119 Commits
Author SHA1 Message Date
mesh-admin 26a58c7a79 Merge pull request 'Ask a seat once more when the controller refused as handing over (hq issue 289)' (#20) from fix/a-verb-survives-the-handover into main 2026-10-07 00:40:21 +00:00
mesh-admin 7e83940fcc Merge pull request 'Console: a seat and a module of one name are each reached (hq issue 287)' (#19) from fix/a-seat-and-a-module-of-one-name into main 2026-10-07 00:27:49 +00:00
jochen 2f045e042f Ask a seat once more when the controller refused as handing over (hq issue 289)
mesh/merge-gate pass: builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/a-verb-survives-the-handover delivered: every member is delivered
A controller being replaced refuses a call it cannot serve, marked retry:
handing-over; the one after it answers the same call.
2026-10-07 02:13:17 +02:00
jochen 41911f7f15 Check again, judged by the controller with the gate's fix (novox/hq issue 285)
mesh/merge-gate pass: builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-07 02:06:52 +02:00
jochen 0a5dbc7e9a Cite the hq issues by the numbers they were given: 285, 286, 287
mesh/merge-gate pass: builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (0 of …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-07 02:00:11 +02:00
jochen dac4d423a8 Reach a seat and a module of one name each: list the seat's verbs, and route a machine's address to the module
mesh/merge-gate pass: builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (0 of …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
mesh-delivery is the delivery's seat and the module holding it, which answers the seat's verbs with
tools of the same names. Discovery keyed the seat's verbs and the module's tools in one namespace, so
the module's tool took the key and the seat was listed with no verb; and every address with the name
resolved to the seat, so neither the seat's verbs nor the module's tools could be called. The keys are
apart, and `<node>/<module>.<tool>` reaches the module when the module serves it (novox/hq issue 284).
2026-10-07 01:37:46 +02:00
mesh-admin 01c98db0a4 Merge pull request 'A merge check of its own; every test on a bus of its own; git in the TypeScript toolchain (hq ADR 0238)' (#18) from feat/a-merge-check-of-its-own into main
mesh/delivery delivered
2026-10-06 20:55:58 +00:00
jochen b86a6ca228 Give the TypeScript toolchain git, for the merge checks that run in it (hq ADR 0237)
A repository's own merge-check.sh may declare this toolchain, and a suite that reads a
repository's history failed with spawnSync git ENOENT. Nothing compiled here calls it.
2026-10-06 22:06:16 +02:00
jochen 14bff5dfd7 Check the tool runtime's own code before it merges, every test on a bus of its own (hq ADR 0237)
A merge-check.sh, the repository's layer of the mesh's merge check (mesh/repo-check):
format, vet, and node-tools' suite under the race detector. The tests shared one bus and
had to run one package at a time; like the controller's (#97), each now starts a server
of its own at the release go.mod pins, held to the catalogue's bus image by a test. What
cannot run in the check — bundles that need @novox/mesh-sdk — is said as not tested.
2026-10-06 22:04:46 +02:00
mesh-admin c2a0683115 Merge pull request 'node-tools: say whose words a refused event is (hq issue 276)' (#17) from fix/an-event-handler-answers-what-it-did into main
mesh/delivery delivered
2026-10-06 16:17:05 +00:00
jochen 1792b0d9ba node-tools: say whose words a refused event is (hq issue 276)
"plex did not take radarr.download.completed: Unexpected end of JSON input"
read as the runtime failing to parse the bundle's answer. It was plex's own
handler error, relayed. A bundle's error answer is now a launch.Refused, and
the line says the handler answered an error, quotes it, names the event id
and the delay before the next offer; the runtime's own failures (no answer
in time, bundle exited) are said as before.

The rule, unchanged and now tested: any mesh/event answer that is not an
error takes the event, whatever its result says, including none.
2026-10-06 18:14:50 +02:00
mesh-admin f75780f1d0 Merge pull request 'Say the node tools are there, every minute (hq to-be 45 Phase 1, S11)' (#16) from feat/a-core-that-cannot-fail-silently-phase-1 into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-06 08:34:15 +00:00
jochen 0307df0850 Say the node tools are there, every minute (hq to-be 45 Phase 1, S11)
A machine whose node-engine is heard and whose runtime is gone is a
machine nobody can ask anything, and nothing said so. The runtime now
says on mesh.control.<node>.tools-alive, every minute, that it is there,
with its interval and build; the controller raises tools-silent after
three missed. Core NATS, like the host's heartbeat: a lost one is the
next one. A heartbeat the bus refuses is logged when that starts and
when it stops.
2026-10-06 10:06:29 +02:00
mesh-admin 730b4047f0 Merge pull request 'Say a timeout is not a failure, and where the controller keeps the answer (hq issue 265)' (#15) from fix/a-timeout-is-not-a-failure into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-05 23:33:12 +00:00
jochen fce3dcb98d Say a timeout is not a failure, and where the controller keeps the answer (hq issue 265) 2026-10-06 01:14:59 +02:00
mesh-admin 9730bd89c3 Merge pull request 'Console: keep a mesh seat's node argument, and refuse what a call would drop (hq issue 244)' (#14) from fix/console-keeps-a-mesh-seats-node into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-05 22:46:47 +00:00
jochen 3a7c9645e0 Keep a mesh seat's node argument, and refuse what a call would drop
The console took node out of every schema and every call, so the
controller's push, plan, assign, pin and settings were described without
the machine and called without it: a push naming one machine ran as a
push of every machine behind (hq issue 244). node is now taken out only
where the address names the machine, a different machine there is
refused, and an argument a seat's verb does not declare is refused.
2026-10-06 00:37:15 +02:00
mesh-admin 93c1ad8c4a Merge pull request 'Console: wait for every runtime that answered, name the ones it missed, add mesh_runtimes' (#13) from fix/console-sees-every-runtime into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-05 13:38:48 +00:00
jochen 52a0cc4fcb Forget a runtime's old instance once it answers under a new one, so a restart is not reported as a missed answer 2026-10-05 15:38:30 +02:00
jochen 379d19d907 Wait for every runtime discovery hears from, and name the ones it misses
The console gathered $SRV.INFO answers for a fixed 750 ms. The laptop's
runtime (48 modules, 341 endpoints, 164 kB) answers last every time: its
answer crosses to the broker on another machine and back, a median of
365 ms on a quiet link and 813 ms in one of 25 rounds measured. When it
missed the window the console said its modules ran nowhere ("nothing in
the mesh is called slack") or only on another machine.

Discovery now asks $SRV.PING alongside $SRV.INFO and waits, past the
window and up to 5 s, for every instance that answered PING. A runtime
that never sends what it serves, or answered before and not now, is
named in every answer that might concern it instead of the module being
called missing. An answer still too large after first-line descriptions
drops them, and says so in its metadata.

mesh_runtimes reports, per runtime, its machine, how long its answer
took, its size, its modules and tools, whether it was shortened and when
it was last heard, and the runtimes and machines not heard.
2026-10-05 15:36:10 +02:00
mesh-admin 8b789578c1 Merge pull request 'Keep a machine's discovery answer under the bus's message limit' (#52) from fix/discovery-fits-in-a-message into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-04 15:22:56 +00:00
jochen b3ebdd5edd Keep a machine's discovery answer under the bus's message limit
Every tool's schema in one $SRV.INFO reply outgrew max_payload on machines
serving 137-206 tools, and the refused reply was dropped silently, so search
and a machine's view went empty. Module tools now announce without schema;
describe and tools/list ask the module for it. An answer still too large has
its descriptions cut to a line, and a failed reply is logged.
2026-10-04 17:06:20 +02:00
mesh-admin 51c79d8461 Merge pull request 'The console says an account was refused only when the bus refused it' (#51) from fix/console-says-a-refusal-only-for-a-refusal into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-04 10:01:18 +00:00
jochen 8405e32efc The console says an account was refused only when the bus refused it
"authorization" alone matched a tool's own answer mentioning the word — the
runtime refusing a state value with an Authorization header read as
"this account may not call".
2026-10-04 11:31:42 +02:00
mesh-admin 670a7884ff Merge pull request 'Module state is hq ADR 0201 after all' (#50) from fix/module-state-is-0201 into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-04 09:03:18 +00:00
jochen 779ea67ea6 Module state is hq ADR 0201 after all: the derived-value record moved to 0202 on hq main 2026-10-04 11:02:43 +02:00
mesh-admin b149e9fcd6 Merge pull request 'node-tools provides its endpoint at node scope (hq to-be 40 WP1)' (#34) from feat/claude-code-agent into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-04 09:01:26 +00:00
mesh-admin 168cf02829 Merge pull request 'Module state is hq ADR 0202 (0201 landed first for a provider's derivations)' (#49) from fix/adr-0202-module-state into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-04 09:01:23 +00:00
jochen 70ff0f84af Module state is hq ADR 0202: 0201 landed first for a provider's derivations 2026-10-04 03:44:51 +02:00
mesh-admin 328550f920 Merge pull request 'The runtime serves a bundle its module's state (hq ADR 0201)' (#48) from feat/module-state-on-the-bus into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-04 01:43:37 +00:00
jochen 29c24a2367 The TypeScript SDK's state reaches the runtime, a narrowed watch only its keys (novox/hq ADR 0201) 2026-10-04 02:48:10 +02:00
jochen 1adfcad88e Name the watch a state change is for (novox/hq ADR 0201) 2026-10-04 02:45:31 +02:00
jochen dac8812968 gofmt 2026-10-04 02:45:10 +02:00
jochen 6cba894f01 The runtime serves a bundle its module's state (novox/hq ADR 0201)
mesh/state.get, put, delete, keys and watch on the stdio channel, from the
buckets the membership issues. A watch hands the current values without
deletions, then every change, and is answered once the current values are
delivered. Refused with the reason: state not issued, a reader's write, a
value with a credential-named field — the bus alone would answer a refused
write with a timeout.
2026-10-04 02:45:07 +02:00
mesh-admin 47cdfad754 Merge pull request 'Ask the bus again for a subscription it refused (hq issue 222)' (#47) from fix/issue-222-a-refused-subscription-is-asked-again into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 23:20:49 +00:00
jochen 85810ccc8c Ask the bus again for a subscription it refused (novox/hq issue 222)
A push sends the machine that needs a grant and the bus's machine in the same breath, and the
runtime can subscribe the moment before the bus reloads its user list. Refused once, the
subscription stayed dead until some later membership re-served it, and a newly assigned module ran
unreachable. A refused subject the runtime answers is now asked for again for about five minutes.
2026-10-04 01:20:44 +02:00
mesh-admin 2ba7451229 Merge pull request 'A refused tool subscription is said, never fatal (hq issue 218)' (#46) from fix/a-refused-seat-subscription-is-not-fatal into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 22:13:22 +00:00
jochen 6604d44372 A refused tool subscription is said, never fatal (novox/hq issue 218)
After the controller stopped granting a mesh seat to claimants that do not hold it, an image built
before the runtime followed its membership still subscribed the seat's subject, and the refusal
ended the process: ace's postgres runtime crash-looped. A subject the grants leave out now costs
that subject only, as an announcement's already did (issue 217).
2026-10-04 00:13:15 +02:00
mesh-admin 7b21440962 Merge pull request 'Serve a seat's verbs from the membership once one is issued (hq issue 218)' (#45) from fix/issue-218-the-membership-decides-the-seats into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 21:49:27 +00:00
jochen 0cea8d286e Serve a seat's verbs from the membership once one is issued (novox/hq issue 218)
The runtime added every seat its start-up credential claims even after the mesh issued a
membership without it. A seat held once for the mesh is claimed on every machine running the
module, so ace's postgres announced the store seat the bus then refused it on.
2026-10-03 23:48:15 +02:00
mesh-admin 094a7d0d7c Merge pull request 'The toolchain carries the SDK the mesh last published, and esbuild (hq issue 212, ADR 0193)' (#44) from feat/the-toolchain-follows-the-sdk-and-bundles into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 21:31:59 +00:00
jochen b52669577d The toolchain carries the SDK the mesh last published, and esbuild (hq issue 212, ADR 0193)
mesh-tools stands on mesh-sdk's published package: the build receives its exact version and
installs it after the package.json install, so a release is a new argument and the cached layer
cannot keep an older SDK; the planner orders the toolchain after the SDK, and every bundle after the
toolchain (issue 211). esbuild, a development dependency, is what the builder bundles each
TypeScript entrypoint and launcher into one file with.
2026-10-03 23:26:09 +02:00
mesh-admin 7bd76275f9 Merge pull request 'A refused announcement is said, never fatal (hq issue 217)' (#43) from fix/a-refused-announcement-is-not-fatal into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 21:24:50 +00:00
jochen 6ba0f4dc1e A refused announcement is said, never fatal (hq issue 217)
The raw subscription that answers discovery ran its loop unguarded, so a refusal escaped as an
unhandled rejection and ended the process: every per-module container crash-looped on 2026-10-03
over a subscription that only serves the mesh seeing the runtime. It is now caught, logged, and the
runtime serves on. Tested against a bus whose permissions refuse the subject; fails without it.
2026-10-03 23:24:38 +02:00
mesh-admin bc05658772 Merge pull request 'A mesh seat's holder answers for the machine it runs on (hq ADR 0197)' (#42) from fix/a-mesh-seats-holder-answers-for-its-machine into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 21:23:06 +00:00
jochen 4af59636c3 A mesh seat's holder answers for the machine it runs on (hq ADR 0197)
The controller announces the mesh-controller seat without a machine — the seat is the mesh's — and
the console then reported it as not answering on the machine it is assigned to. An announcement that
names no machine now answers for wherever its module is assigned.
2026-10-03 23:22:56 +02:00
mesh-admin 6e425a000f Merge pull request 'The node's runtime is its modules' bus: it binds each module's consumer and hands its events to the bundle (hq ADR 0198)' (#41) from feat/0198-the-runtime-is-the-bus into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 21:18:33 +00:00
jochen 14b6588839 Merge remote-tracking branch 'origin/main' into feat/0198-the-runtime-is-the-bus 2026-10-03 23:16:43 +02:00
mesh-admin 490aedfd36 Merge pull request 'Announce only on the subjects the grants allow (hq ADR 0197, issue 217)' (#40) from fix/announce-only-what-the-grants-allow into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 21:11:24 +00:00
jochen f11ac6441c Announce only on the subjects the grants allow (hq ADR 0197)
Both runtimes subscribed $SRV.<verb>.> as a wildcard; the grants allow the bare question and the
service's own name and instance. The bus refused the wildcard, and the TypeScript runtime treats a
refused subscription as fatal, so every per-module container crash-looped after the image rolled.
They now subscribe exactly $SRV.<verb>, $SRV.<verb>.<name> and $SRV.<verb>.<name>.<id>.
2026-10-03 22:31:02 +02:00
jochen ffe229308c The node's runtime is its modules' bus: it binds each module's consumer and hands its events to the bundle (hq ADR 0198)
A launched bundle's mesh/subscribe binds the module's own durable consumer — EVENTS, <node>_<module>,
by name as the module's own runtime bound it, so nothing is lost or replayed in the move — and every
event goes to each child of the module that subscribed as mesh/event, acknowledged only when all
answered, negatively acknowledged after a short delay when one failed or died, terminated when it is
not an event. mesh/ask calls a tool as the module. Every launched bundle is started again when it
exits, with backoff, since long-running code waits for no call. Requires SDK 0.1.6.
2026-10-03 22:29:20 +02:00
mesh-admin df4f492a72 Merge pull request 'The mesh's tools are found by address, from what announces itself on the bus (hq ADR 0195, 0197)' (#39) from feat/0197-tools-announce-themselves into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 20:19:53 +00:00
jochen 66e8be0e31 The TypeScript runtime announces what it serves too, in the same services format (hq ADR 0197)
The per-module containers still run this runtime; their tools and the seats they hold (the store's,
the catalogue's) must be found by the console the same way as the node runtime's. It answers
$SRV.PING, $SRV.INFO and $SRV.STATS with one service per process, one endpoint per tool per subject
and per seat verb served, the metadata as the Go runtime writes it.
2026-10-03 22:15:48 +02:00
jochen 7722668220 Every runtime announces what it serves in the NATS services protocol; the console discovers by asking the bus (hq ADR 0197)
The Go runtime answers $SRV.PING, $SRV.INFO and $SRV.STATS (and per name and id) in the
io.nats.micro.v1 format with what it serves at the moment it is asked: one service per runtime
process, since the bus admits one reply per request from each responder, and one endpoint per tool
per subject, its metadata saying module, seat, scope, machine, description, schema and whether the
module is interchangeable. Serving is unchanged.

The console gathers one $SRV.INFO request's answers instead of asking the catalogue's roster and
each module's tools, and reads the controller's records as JSON for what should have answered: an
assignment with tools that did not announce is named, a module without tools never is. The text
parsers of node list and module list are gone. Packages share the test bus: go test -p 1.
2026-10-03 22:14:16 +02:00
jochen e8989f3cf5 Merge remote-tracking branch 'origin/main' into feat/0197-tools-announce-themselves 2026-10-03 22:08:06 +02:00
mesh-admin 915f372a85 Merge pull request 'node-tools in Go: the node's runtime, launch-only, wire-compatible with the TypeScript (hq ADR 0193)' (#38) from feat/0193-node-tools-in-go into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 20:02:15 +00:00
jochen 486dad99a5 Merge remote-tracking branch 'origin/main' into feat/0193-node-tools-in-go 2026-10-03 22:02:03 +02:00
mesh-admin 65f3b68076 Merge pull request 'node-tools launches every bundle it serves; a child's emit is published as its module (hq ADR 0193)' (#36) from feat/0193-the-runtime-launches-every-bundle into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 20:01:58 +00:00
jochen a269a76c87 The console announces five tools and reaches everything by address (hq ADR 0195)
mesh_overview, mesh_machine, mesh_search, mesh_describe and mesh_call walk the mesh's structure;
every tool has one address per layer: <seat>.<verb>, <node>/<seat>.<verb>, <node>/<module>.<tool>,
and <module>.<tool> for a module the mesh issued a plain subject. A stateful module called without
its machine, a node seat without one, a mesh seat with one, or a module on the wrong machine is
refused naming what would work. Answers come from the mesh when asked, kept five seconds, so a tool
that arrives mid-session is found. The flat catalogue stays behind MESH_CONSOLE_FLAT=1 and old
<module>.<tool> names still answer.
2026-10-03 21:54:19 +02:00
jochen e6a33dd969 node-tools is built from Go (hq ADR 0193)
The node's runtime is the Go binary: one static executable the host runs as ./node-tools from its own
bundle. Node.js stays on the machine for the TypeScript bundles the runtime launches. The TypeScript
source stays in the repository: it is the runtime image the per-module containers run until WP4c.
2026-10-03 21:26:06 +02:00
mesh-admin 42e0987c64 Merge pull request 'Require SDK 0.1.5: a launched bundle names its module and emits through the runtime (hq ADR 0193)' (#37) from fix/the-toolchain-carries-sdk-0.1.5 into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 19:23:59 +00:00
jochen ae6bdc9b06 Require SDK 0.1.5: a launched bundle names its module and emits through the runtime (hq ADR 0193)
The toolchain image installs from this package.json with a range; unchanged, Docker reused the
cached install and the image kept SDK 0.1.3 after 0.1.5 was published. Bundles copied that copy, so
a launched bundle registering its seat first served the seat's verbs as its own tools. Requiring
0.1.5 says what the runtime and its bundles need, and invalidates the cached layer.
2026-10-03 21:23:50 +02:00
jochen e45389f75a node-tools in Go: the node's runtime, launch-only, wire-compatible with the TypeScript (hq ADR 0193)
The runtime knows no language, so nothing ties it to Node.js. This ports its serve mode — the
pinned bus connection and patient connect, following memberships, launching every served bundle
over MCP on stdio with its own environment, a child's emit published as its module, each tool,
the tools verb and seat verbs served where the mesh issued them, and the console on loopback —
to one static binary. Same subjects, request and reply bodies, event headers and MCP answers.
The TypeScript stays: it is still the runtime inside the per-module containers until WP4c.
Tests run against a real bus and share the TypeScript fixtures.
2026-10-03 21:21:52 +02:00
jochen b182943c24 node-tools launches every bundle it serves; a child's emit is published as its module (hq ADR 0193)
Every served entrypoint is started as a process speaking MCP over stdio, told its module and node;
one that is not executable is refused by name. The import path, the SDK resolve hook (issue 209)
and the per-registration hand-off go. The one-module form the per-module containers use is still
imported until they move (to-be 38 WP4c). A child's mesh/publish is published as its module and
answered once accepted; a child that dies says why in its own last words. Fixtures are served
through launchers exactly as the builder writes them.
2026-10-03 21:12:15 +02:00
mesh-admin 26e7fcb4cb Merge pull request 'A launched bundle is told the module it serves (hq ADR 0193)' (#35) from feat/0193-a-launched-bundle-is-told-its-module into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 19:08:06 +00:00
jochen 302645dd01 A launched bundle is told the module it serves (hq ADR 0193)
node-tools sets MESH_SERVED_MODULE for each child it launches, so the SDK's stdio loop lists that
module's tools by their names and a seat's verbs as the seat's, whichever was registered first.
2026-10-03 21:07:52 +02:00
jochen de4b63824e node-tools provides its endpoint at node scope, so a consumer on the machine is told the console's port rather than writing it (hq to-be 40 WP1) 2026-10-03 16:02:08 +02:00
mesh-admin 8e30ea9ff5 Merge pull request 'node-tools hands each bundle its own environment (hq ADR 0192)' (#33) from feat/0192-each-bundle-its-own-env into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 13:34:05 +00:00
jochen 193ed0ac63 node-tools hands each bundle its own environment (hq ADR 0192)
The mesh composes every served module's words into MESH_TOOL_ENV; the runtime takes it at start
and removes it from its own environment, then gives each registration's contributor and each
launched child the runtime's words plus its own module's, never another's. Against an SDK
without collectToolsEach it says so and serves with the runtime's words only. The test serves
two imported bundles and one launched, each answering with its own words and none of the others'.
2026-10-03 15:28:49 +02:00
mesh-admin 4537494150 Merge pull request 'One SDK per runtime: a bundle's SDK import resolves to the runtime's copy (hq issue 209)' (#32) from fix/issue-209-one-sdk-per-runtime into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-03 11:07:08 +00:00
jochen 7152148410 One SDK per runtime: a bundle's SDK import resolves to the runtime's copy (hq issue 209)
A bundle carries its dependencies, the SDK among them; imported in-process that copy was a
second SDK with its own tool registry, so a bundle registered its tools into a list the
runtime never read and served nothing, silently. A resolve hook (module.registerHooks, in
thread; module.register is deprecated from Node 26) now sends every import of
@novox/mesh-sdk, from whichever bundle, to the runtime's own copy: one registry, one broker.
The test loads a bundle from a directory holding its own SDK copy and sees its tool served.
2026-10-03 13:00:19 +02:00
mesh-admin 5d488a45f7 Merge pull request 'node-tools is a module beside mesh-tools: the runtime as a bundle, and serve is the console (hq ADR 0175, to-be 38 WP3)' (#31) from feat/wp3-node-tools into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-02 19:57:24 +00:00
jochen c46f9502ee node-tools is a module beside mesh-tools: the runtime as a bundle, and serve is the console (hq ADR 0175, to-be 38 WP3)
One repository, two modules (ADR 0069). `node-tools/` holds the runtime — its code, tests, package
and the manifest of the module the controller composes a process for on every machine it is
assigned to: a bundle of `src/main.js`, the interpreter as a package, a place for the node's
credential, the loopback port the console declared, and leave to call every tool. Nothing about
how it runs: which bundles to load, where the credential is and whose machine it is are the
controller's to compose (WP2). The root module `mesh-tools` keeps the two images TypeScript
bundles are compiled in and a module's own service may run in; it is no longer how tools reach a
node.

As node-tools, `serve` is also the console (ADR 0175 §6): the same process answers MCP on
loopback for whoever is on the machine, through which the tools it serves can be called. A
module's own runtime in a container keeps serving without a listener.

The toolchain image now carries /app/runtime — a package.json saying the compiled files are ES
modules and the production node_modules — for the builder to copy into every TypeScript bundle,
so a bundle unpacked on a machine starts (ADR 0188 §5; the builder's side is the controller's).
Proven here by compiling node-tools with the toolchain's exact flags and starting the result.
The AMQP probe script is gone with the bus it probed.
2026-10-02 21:43:37 +02:00
mesh-admin 2746fd31f0 Merge pull request 'Cite ADR 0188, not 0187: the record was renumbered before it merged' (#30) from fix/cite-adr-0188 into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-02 19:29:30 +00:00
jochen 82d7306ee0 Cite ADR 0188, not 0187: the record was renumbered before it merged (0187 is the dead-tracker record) 2026-10-02 21:29:04 +02:00
mesh-admin 2818f99b17 Merge pull request 'The runtime serves a list of modules on one credential, naming a bundle that fails to load (hq ADR 0175, to-be 38 WP1)' (#29) from feat/the-operators-machine into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-02 19:27:49 +00:00
jochen 1436b02755 A bundle that is not JavaScript is launched and spoken to over MCP on stdio (hq ADR 0187, to-be 38 WP1b)
The runtime imported a bundle into its own process, which only JavaScript can be. Now an entrypoint
that is not a plain JavaScript file — or is one marked executable — is started as a child with the
runtime's environment and asked `tools/list` once and `tools/call` per call; what it lists is
registered exactly as an imported bundle's registrations are, a `<seat>.<verb>` name as the seat's
implementation. So a tools bundle may be in any language, and the mesh's part — the subjects, the
seats, the `tools` answer, a failed bundle named — stays in the runtime and is shared by all of
them. A child that exits mid-call tells the caller so and is started again on its next call.

Proven against a real bus beside the three bundles already there: a Python bundle with no SDK at
all answers its tool and its seat verb; a TypeScript bundle written against the protocol and marked
executable is served through the launcher, shortcut off; a bundle told to exit is relaunched.
2026-10-02 21:24:20 +02:00
mesh-admin 4e0559c31a Merge pull request 'Cite hq ADR 0170, not 0169: the firewall seat's record was renumbered' (#28) from fix/adr-0170-cited into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-02 16:44:07 +00:00
jochen 6390d1d7fb The runtime serves a list of modules on one credential, naming a bundle that fails to load (hq ADR 0175, to-be 38 WP1)
One tool runtime per node, host-side, is what the runtime was written to be; the catalogue built a
container per module around it instead. This lets `serve` take a list — MESH_TOOL_MODULES as
<module>=<entrypoint> entries — and do for every assigned module what it did for one: read that
module's membership and follow it, serve its tools where the membership says, serve each held
seat's verbs on the seat's subjects. The seats come from the memberships now, so the node's
credential carries no claims; a module's own runtime still reads its credential's, so nothing
built today changes behaviour. A bare path in MESH_TOOL_MODULES stays the one-module form.

A bundle that throws on import is said in the log and in what `tools` answers for its module
(`failed`), which discovery lists with the reason instead of as "not answering"; the other bundles
serve. The filter that dropped every registration under a name but the one module goes; what
stays is that a registration under a seat's name is served only where some served module claims
the seat. A tool runs attributed to its module, so an event it emits lands on the module's
subject and not the runtime's. MESH_OPERATOR_ACCOUNT and MESH_OPERATOR_HOME are read and said;
tools take them from their environment.

Proven against a real bus: three bundles, one broken; five tools and two seat verbs answer on
their subjects; `tools` names the failed bundle; a membership re-issued mid-run re-serves.
2026-10-02 18:22:30 +02:00
jschoubben 020003ea3f Cite hq ADR 0170, not 0169: the firewall seat's record was renumbered after a collision on hq main 2026-10-02 14:52:28 +02:00
mesh-admin 809f07d085 Merge pull request 'A node-scoped seat's verb is callable through the console, naming its machine (hq ADR 0169)' (#27) from fix/a-node-scoped-seats-verb-is-callable-through-the-console into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-02 12:24:02 +00:00
jschoubben fb0dcd0cc1 A node-scoped seat's verb is callable through the console, naming its machine (hq ADR 0169)
The listing left node-scoped seats out, so <seat>.<verb> never resolved as a
seat's and the call went to a module subject nothing served. Listed now with
their scope: the schema requires the machine, the call carries it in the
subject, and a call without one is refused in words.
2026-10-02 14:23:26 +02:00
mesh-admin a3ace58362 Merge pull request 'A registration under a stranger's name is said and skipped, not fatal' (#26) from fix/a-registration-under-a-strangers-name-is-refused-not-fatal into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-01 14:47:31 +00:00
jschoubben feb13c1de6 A registration under a stranger's name is said and skipped, not fatal
A module implementing a seat its credential does not (yet) claim registered tools under the seat's
name; the runtime treated them as the module's own, refused to serve another module's key, and the
whole runtime restarted in a loop — postgres on the control node, 2026-10-01, whose credential predated
the claims. Such a registration is now named in the log and left out; the module's own tools serve.
2026-10-01 16:47:03 +02:00
mesh-admin e6676068c7 Merge pull request 'The membership is read by its subject' (#25) from fix/the-membership-is-read-by-its-subject into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-01 14:22:26 +00:00
jschoubben a01b5f5b78 The membership is read by its subject
The runtime asked the assignments stream's root for the last message by subject in the body, and the
mesh grants a module's account only the subject-addressed form of the direct get — the server refused
every read (Publish Violation on $JS.API.DIRECT.GET.ASSIGNMENTS for every module on the new runtime),
so every runtime kept the derived shape. The address is now the stream then the subject, nothing in the
body, which is the one address the account has.
2026-10-01 16:16:57 +02:00
mesh-admin ab8b13f512 Merge pull request 'A seat's verb keeps a node of its own; only a module's tool gives it to the subject' (#24) from fix/a-seat-verbs-node-is-its-own into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-01 13:44:22 +00:00
jschoubben ce8c37a7fc The listing test tells the seat's two verbs apart: one keeps its own node, the other takes none 2026-10-01 15:43:36 +02:00
jschoubben 809e2b11ec The listing test indexes the module's tool after the seat's two verbs 2026-10-01 15:42:58 +02:00
jschoubben 670b486ac0 A seat's verb keeps a node of its own; only a module's tool gives it to the subject
The console moved every call's node into the subject (ADR 0159), so mesh-controller.push {node: x}
became a call to the seat's verb on machine x, which nothing serves — the mesh's own verbs could not
be given a machine from the console at all. A role's verb takes no machine from the console; its
arguments are its own.
2026-10-01 15:42:35 +02:00
mesh-admin 56f82588af Merge pull request 'A runtime serves what the mesh issued it, and a seat's verbs are implemented under the seat's name (hq ADR 0160)' (#23) from feat/a-runtime-serves-what-it-is-issued into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-01 13:23:55 +00:00
jschoubben e7b98f1fbc A runtime serves what the mesh issued it, and a seat's verbs are implemented under the seat's name (hq ADR 0160)
The one address a runtime derives for itself is mesh.assignment.<node>.<module>. It reads the
membership there with a direct get on the ASSIGNMENTS stream, serves each tool exactly where the
membership says — the plain subject in the module's queue when the mesh issued one, this machine's
beside it — and follows the subject, re-serving when a new membership arrives. A mesh that has issued
nothing yet gets the shape it always derived, and the log says so.

A seat's verbs are the role's, not the software's (ADR 0159): a module implements them with
registerModuleTools("<seat>", …), the runtime serves that on the seat's subjects when the credential
claims the seat, and never lists it among the module's own tools. A module named like its seat
registers once and is both.

The tools answer carries each tool's subjects, and the console and CLI call the subject the listing
gave them instead of composing one.
2026-10-01 15:17:14 +02:00
mesh-admin 278a25b3e5 Merge pull request 'A tool call names the machine it is for, every answer says which machine answered, and a holder's runtime serves its seat's verbs (ADR 0159)' (#22) from feat/a-tool-call-names-the-machine into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
2026-10-01 12:01:40 +00:00
jschoubben c65f1993ba A tool call names the machine it is for, every answer says which machine answered, and a holder's
runtime serves its seat's verbs (novox/hq ADR 0159)

A module on several machines served one subject in one queue group, so a call reached whichever
instance answered first and nobody could ask one machine's instance. Now each instance also serves
its subject with its machine as the last token, `<module>.<tool>@<node>` addresses it, and every
answer carries the machine that gave it: the console lists `node` on every module tool, strips it
into the subject, and appends "answered by <node>" to the answer; `mesh call` prints it.

And a seat's verbs are served by whoever claims the seat: the credential names the claimed seats
and the verbs each promises, the runtime serves each verb with the module's tool of the same name
on the seat's own subject, and the bus — which admits only the holder's subscription — decides where
that serving is real. Design 33 §3 and §4, built.
2026-10-01 13:58:22 +02:00
jschoubben 6a91d144b3 Merge pull request 'The console lists and calls a role's tools' (#21) from feat/the-mesh-answers-for-itself into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
Reviewed-on: http://git.novox.be/novox/mesh-tools/pulls/21
2026-09-30 15:54:31 +00:00
jschoubben 71965ef958 The console lists and calls a role's tools
seat:<seat>.<verb> addresses a role's tool (with @<node> for a node-scoped seat); the listing asks the
mesh-controller seat's tools verb beside the modules and marks a role's tools; <seat>.<verb> resolves
to the seat when the seat declares that verb, a module's own name otherwise (novox/hq ADR 0154).
2026-09-30 17:41:56 +02:00
jschoubben dea98e509a Merge pull request 'The console: mesh serve on loopback, and every runtime answers tools' (#20) from feat/the-console into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
Reviewed-on: http://git.novox.be/novox/mesh-tools/pulls/20
2026-09-30 14:46:54 +00:00
jschoubben 80b02740ab The console: mesh serve on loopback, and every runtime answers tools
The runtime serves a tools verb per module with names, descriptions and schemas (design 34 §3), and
refuses a module naming its own tool tools. Discovery asks catalog_modules then each module, naming
what did not answer. One MCP handler over two transports: stdio (mesh mcp) and loopback HTTP (mesh
serve, the mesh-console module, novox/hq ADR 0152); serve refuses any bind but loopback. tools/call
may go through a running console with --console and no credential.
2026-09-30 16:19:22 +02:00
mesh-admin 621d033d53 Merge pull request 'One consumer, one reader, however many patterns a module registers' (#19) from fix/one-consumer-one-loop into main 2026-09-28 14:25:47 +00:00
jschoubben 38831c5c56 One consumer, one reader, however many patterns a module registers
A module has exactly one durable consumer, and each subscribe() started its own reader of it. Two
readers split the stream between them, and a reader that receives a message its own pattern does not
match acknowledges it — which is the right answer for a filter wider than anything registered, and
silent loss when the message was another handler's. The first module to subscribe twice would have
dropped roughly half of each kind of event with nothing reporting it.

Every registration is now dispatched from one reader, and a message is acknowledged once every handler
it is for has taken it.
2026-09-28 16:25:42 +02:00
mesh-admin fdad2f3268 Merge pull request 'A module answers the word the mesh asks: prepare' (#18) from feat/a-module-answers-prepare into main 2026-09-28 10:45:02 +00:00
jschoubben 81972a4995 A module answers the word the mesh asks: prepare
The runtime gains `prepare`, which brings this module's state to the shape this version needs and
exits (novox/hq ADR 0135). The entrypoints come from MESH_PREPARE, which a module's own image names
beside the entrypoints it already lists there — the module knows which of its files prepares its
state and nothing else could. No broker is connected: preparation runs before the version that would
use it. An empty list fails rather than passing quietly, because the mesh asks this only of a module
whose manifest says it prepares something, and exiting 0 would let that version serve against a
state nobody shaped.
2026-09-28 12:45:00 +02:00
mesh-admin 10e8191717 Merge pull request 'The runtime hears on its own inbox' (#17) from fix/the-runtime-hears-on-its-own-inbox into main 2026-09-28 02:30:24 +00:00
jschoubben d703cebff4 The runtime hears on its own inbox
Every user's inbox is private to it and the grant names it; a reply space the client invented was
refused, and with it every pull for the next message and every answer to a tool call.
2026-09-28 04:30:22 +02:00
mesh-admin 46b56d53a6 Merge pull request 'The pin is the only check: the runtime stops verifying the bus's name' (#16) from fix/the-pin-is-the-only-check into main 2026-09-28 02:03:46 +00:00
jschoubben d4a2802342 The pin is the only check: the runtime stops verifying the bus's name
Every module on the new runtime reached the handshake and failed on 'does not match
certificate's altnames': the bus's certificate names the seat, not the address a machine dials
it by, and pinning the exact certificate already decides everything a name check could. The
client's transport spreads the TLS options into Node's tls.connect, so the hostname check is
replaced with one that passes and the pinned certificate is the one authority accepted.
2026-09-28 04:03:43 +02:00
mesh-admin 8acfa7a07d Merge pull request 'One bus: the runtime pins the certificate after the server speaks, and the old transport goes' (#15) from feat/one-bus into main 2026-09-28 01:09:01 +00:00
jschoubben f0104b7846 One bus: the runtime pins the certificate after the server speaks, and the old transport goes
Every module that dialled the new bus failed its handshake with "wrong version
number": the runtime pinned the server's certificate by a raw TLS connection to a
port on which the server speaks first, in the clear. The pin is taken after the
INFO line now, on the same socket, and then the real connection verifies against
exactly that certificate.

And the old transport is deleted — its client, its tests, its dependency — with
the wire-compatibility pins that only existed for the move (novox/hq ADR 0131,
design 28 task 5.5). A credential names the bus, and there is one.
2026-09-28 03:08:59 +02:00
mesh-admin cd26131c61 Merge pull request 'The runtime speaks the bus its credential names' (#14) from feat/the-runtime-speaks-the-bus-its-credential-names into main 2026-09-28 00:42:29 +00:00
jschoubben 9e3ff6fa45 The runtime speaks the bus its credential names
A module moved to the bus being built was handed a credential for it — nats://
with user, password and fingerprint beside the address — and nothing else in its
environment changed. The runtime always dialled the old bus, so every moved module
kept serving and answered nobody. The scheme in the credential is enough to know
which bus to speak; the nats broker was already written and never chosen.
2026-09-28 02:42:26 +02:00
jschoubben a4447f1251 Merge pull request 'A person's own client, and pins that the wire did not change' (#13) from feat/nats-genesis into main 2026-09-27 17:20:07 +00:00
jschoubben 3d55aeb1d8 The old bus's wire is pinned unchanged, because this has to merge to a running mesh
Every module's event names were converted from the old bus's routing keys to local
names, and this client maps them back. If that mapping is wrong anywhere a live mesh's
events stop being delivered — silently, because a binding that matches nothing is not an
error.

So the mapping is pinned against the literal routing keys the mesh published before,
taken from the manifests as they were: what each module now emits, what each now binds,
and that a handler still matches what the bus delivers. Including the audit logger's
"everything", which must stay `#` on this bus.

And a key already in the old form is left alone, so a module built from an older manifest
keeps working beside one built from a current manifest — which is the state the mesh will
actually be in between deployments.
2026-09-27 17:37:19 +02:00
jschoubben 9acc40145a A person's client: the mesh's tools from a workstation
Design 25 §7's second item. Two surfaces over one thing — a command line for somebody
at a terminal, an MCP server for an agent — and both are adapters over the same three
calls: what tools are there, what does this one take, call it. A second way of reaching
a tool would be a second thing to keep correct.

It uses the client a module's runtime uses. Not a bridge and not a second protocol: a
person connects as their own bus user and publishes on the tool subjects their account
permits, so "what may this person do" is answered by the same permission list that
answers it for a module, and an audit has nothing separate to read.

`mesh tools` lists what the *catalogue* has, not what this credential may call. The two
differ and the difference is the point: somebody seeing only their own tools cannot tell
"not installed" from "not yours", and those need different people to fix them.

A failed call says which of three things happened, because the remedies are in three
different places: nobody serves that tool, this credential may not call it, or the tool
itself was slow. Without that they are one timeout and a stack trace.

The MCP surface decides nothing. The tool names are the ones a person types, the schemas
are the modules' own, and an answer is passed through unshaped — an adapter that
summarised somebody else's answer would be deciding what matters in it. A tool that fails
comes back as a tool error rather than a protocol error, because the request was
well-formed and the mesh answered it.

Written against the protocol directly: it is three methods and one framing, and a
dependency here would be a dependency on every workstation.

Tests drive both surfaces against a real bus, including that a host's notification is
answered with nothing and an unknown method is refused. They run one file at a time,
because each stands up a module serving the same tool subjects and run together their
requests get split between them — which showed up as one test reading another's answer.
2026-09-27 17:03:13 +02:00
jschoubben fbeb373d1a Both clients map local event names to their own wire
A module names its events locally and each transport works out where they land.
That is what design 29 says and what neither client did: both passed the name
straight through, which happened to be right on the old bus because modules were
writing routing keys, and wrong on the new one (novox/hq 04-ISSUES/127).

The old bus's client now turns a local name into `module.<emitter>.<event>` on the
way out and back on the way in. Without that, converting the modules to local
names would have broken the mesh that is actually running.

**A handler and a manifest now say the same thing.** The key a module sees was the
event name alone, so a manifest declaring `consumes: builder.built` produced a
pattern that could never match what it was compared against — and a module
consuming one event from two emitters could only tell them apart by reading a
header. The subject already carries the emitter, so naming it in the key makes a
mismatch between manifest and code a typo instead of a category error.

Both matchers accept `**` for the rest of a name, which is how a manifest spells
it; the old bus's `#` still works, because both buses ship until the rollout.
2026-09-27 14:42:40 +02:00
jschoubben 2197c36fef Describe the client on its own terms
Same cleanup: the comments explained each decision by contrast with what
came before instead of stating it. The certificate constraint stays — it is
a fact about the mesh's certificates, not a comparison.
2026-09-26 23:51:00 +02:00
jschoubben 19560ca6a7 Hold the runtime's NATS client to the shared fixtures
Read back from the stream rather than from the client that wrote it, so the
check is what reached the wire. The runner lives with the implementation;
the fixture stays in one place.
2026-09-26 23:40:59 +02:00
jschoubben c1517c39a0 The tool runtime's client on NATS, behind the unchanged contract
Task 3.6 of novox/hq ADR 0116. A module is still written against request,
handle, publish, subscribe, close; only what is underneath changes. main.ts
still selects the AMQP client — steps 1 to 4 leave every node on AMQP, so
this ships beside it and is selected at the rollout.

Round-tripped against a real server (test/roundtrip.mjs): a tool answered
across two connections, a throwing handler reaching the caller as an error
rather than a timeout, an event delivered once with its key, body, node and
event id intact, and an event landing under its emitter's own namespace.

Three things the compiler and the server corrected:

- the envelope's field is `key`, not `type`, and the payload is `env.body`
  with metadata in headers — not the whole envelope re-encoded. An
  implementation that nested the envelope would pass all its own tests and
  agree with nobody, which is what the conformance suite exists to stop.
- the NATS client's TLS options are PEM strings with no verify hook, so the
  AMQP client's `checkServerIdentity: () => undefined` has no equivalent.
  The fingerprint check still happens and is still the guarantee, but the
  bus's certificate must now carry a SAN matching the address nodes dial.
  That is a constraint on the mesh's certificates, recorded where it bites.
- a durable consumer is bound, never created: a module's account cannot
  reach the JetStream API, and a runtime creating its own would be a module
  choosing its own delivery semantics.
2026-09-26 23:29:17 +02:00
jschoubben 6b380b67e0 Merge pull request 'The tool runtime's recipe starts FROM the base its manifest declares (ADR 0097)' (#12) from multiple-fixes into main 2026-09-21 22:58:23 +02:00
jschoubben 2990b2d5a4 The tool runtime's recipe starts FROM the base its manifest declares (novox/hq ADR 0097) 2026-09-21 22:16:10 +02:00
105 changed files with 11997 additions and 758 deletions
+1
View File
@@ -1,2 +1,3 @@
node_modules/
dist/
.mesh-build/
+45 -18
View File
@@ -1,4 +1,8 @@
# Three stages, two published images: the one modules are COMPILED in, and the one they RUN in.
ARG NODE_BASE=node:22-bookworm-slim
# The mesh-tools module: the two images every TypeScript module is COMPILED in and may RUN in. The
# runtime itself ships as the node-tools module's bundle (node-tools/, novox/hq ADR 0175, to-be 38
# WP3); these images are the toolchain for TypeScript bundles and the base a module's own service
# may still be built on. They are no longer how tools reach a node.
#
# **They were the same image, and that was a mistake.** A module's recipe starts from this and
# invokes the compiler out of it, so the compiler had to be here — and because the same image was
@@ -13,41 +17,64 @@
# docker may carry no buildx.
# ---- deps: node_modules resolved from the mesh's registry, credential and all ----------------
FROM node:22-bookworm-slim AS deps
FROM ${NODE_BASE} AS deps
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY package.json ./
COPY node-tools/package.json ./
# The builder writes .npmrc into the build context; it authenticates to the mesh's package registry
# for the @novox scope, which is where @novox/mesh-sdk resolves. This stage is not published, so the
# credential travels no further than here. Development dependencies included: the compiler is one.
COPY .npmrc ./.npmrc
RUN npm install --no-audit --no-fund
# **The SDK this image carries is the one the mesh last published** (novox/hq issue 212). The range in
# package.json is resolved once and the layer above is cached, so a release reached no toolchain until
# that file changed. The exact version arrives as a build argument from the SDK module's published
# package (module.json `build.on`), so a new release is a new argument, this layer runs again — and
# the planner orders this module after the SDK, so a release rebuilds the toolchain and, after it,
# every bundle compiled in it (issue 211).
ARG MESH_SDK=@novox/mesh-sdk@latest
RUN npm install --no-audit --no-fund "${MESH_SDK}"
# ---- toolchain: what a module is compiled in, WITHOUT the credential --------------------------
FROM node:22-bookworm-slim AS toolchain
# ---- compiling: the runtime's own code built, WITHOUT the credential -------------------------
FROM ${NODE_BASE} AS compiling
WORKDIR /app
COPY package.json ./
COPY node-tools/package.json ./
# The resolved libraries, but not the .npmrc that resolved them.
COPY --from=deps /app/node_modules ./node_modules
# The toolkit arrives compiled. It used to arrive as sources, and this compiled it by hand — the
# hook that builds it on install was running all along, and the result was then packed out of the
# package, because with no explicit file list npm falls back to .gitignore and that ignores the
# build output. Fixed where it belonged, in the toolkit.
COPY tsconfig.json ./
COPY src ./src
COPY node-tools/tsconfig.json ./
COPY node-tools/src ./src
RUN npm run build
# ---- what the running image needs, and nothing else -------------------------------------------
# ---- what a running bundle needs, and nothing else -------------------------------------------
# Its own stage so the toolchain image keeps its build tools while the runtime image does not.
FROM toolchain AS lean
FROM compiling AS lean
RUN npm prune --omit=dev
# ---- runtime: what a module runs in -----------------------------------------------------------
FROM node:22-bookworm-slim AS runtime
# ---- toolchain: what a TypeScript bundle is compiled in ---------------------------------------
# The compiler, and esbuild (a development dependency) at /app/node_modules/esbuild: the builder
# bundles every entrypoint and launcher into one file with it (novox/hq ADR 0193), so a bundle
# carries what it imports and not this image's node_modules.
# Beside the compiler, at /app/runtime, what every TypeScript bundle runs with: the production
# dependencies the SDK and the runtime need, and a package.json saying the compiled files are ES
# modules. The builder copies this directory whole into a compiled bundle (novox/hq ADR 0188 §5),
# so a bundle unpacked on a machine starts — a `.js` without that package.json is read as
# CommonJS, and an import of `nats` without node_modules beside it resolves to nothing.
FROM compiling AS toolchain
COPY --from=lean /app/node_modules /app/runtime/node_modules
RUN printf '{"type":"module","private":true}\n' > /app/runtime/package.json
# **git, because a pull request's merge check runs in this image too** (novox/hq ADR 0237): a repository's
# own merge-check.sh that declares this toolchain runs here, and a suite that reads a repository's history
# (the lab's last-run record) ran into "spawnSync git ENOENT". Nothing compiled here calls it.
RUN apt-get update \
&& apt-get install -y --no-install-recommends git \
&& rm -rf /var/lib/apt/lists/*
# ---- runtime: what a module's own service may run in ------------------------------------------
FROM ${NODE_BASE} AS runtime
WORKDIR /app
COPY package.json ./
COPY node-tools/package.json ./
COPY --from=lean /app/node_modules ./node_modules
COPY --from=toolchain /app/dist ./dist
COPY --from=compiling /app/dist ./dist
ENTRYPOINT ["node", "dist/main.js"]
+73 -17
View File
@@ -1,32 +1,88 @@
# mesh-tools
The Novox Mesh **tool runtime** — the per-node process that makes a module's tools actually serve.
Two modules in one repository (novox/hq ADR 0069), one piece of software:
A module ships its tools (built on [`@novox/mesh-sdk`](https://git.novox.be/novox/mesh-sdk)); this
runtime is what loads them and puts them on the mesh. It:
- **`node-tools`** (`node-tools/`) — the node's **tool runtime** as a module (ADR 0175, to-be 38
WP3): one process per machine the host runs from this bundle, serving every assigned module's tools
and every held seat's verbs on the bus, and answering MCP on the machine's loopback — the console
(design 34). The code, its tests and the `mesh` client all live there.
- **`mesh-tools`** (this directory) — the two images TypeScript bundles are compiled in and a module's
own *service* may still run in. Built from the same code; no longer how tools reach a node.
1. connects the mesh broker (novox/hq ADR 0001) — a concrete AMQP implementation of the sdk's
`Broker` contract;
2. imports the assigned modules' compiled tool entrypoints, each of which registers its tools as it
loads;
3. serves them through the sdk's `serveTools` harness, answering `tools.invoke` over the broker.
The runtime:
1. connects the mesh bus on the node's credential — a concrete implementation of the sdk's `Broker`
contract;
2. reads one membership per module it serves — what the mesh issued that module on this machine
(ADR 0160): where its tools are answered, which seats it holds — and follows each live;
3. imports each module's compiled tool entrypoints, each of which registers its tools as it loads,
guarded: a bundle that throws is named, in the log and in what `tools` answers for its module,
and the others serve;
4. serves every module's tools on that module's subjects and every held seat's verbs on the seat's.
A bundle that is not plain JavaScript — a Go or Rust binary, a Python script, or a JavaScript file
marked executable — is **launched** rather than imported (novox/hq ADR 0188): the runtime starts it
as a child with its own environment and speaks MCP over stdio to it, `tools/list` once and
`tools/call` per call. A tool it lists as `<seat>.<verb>` is the seat's implementation. A child that
exits is named in the log and started again on its next call. So a tools bundle may be written in
any language; the mesh's SDK for each is the stdio loop and nothing more (`node-tools/src/launch.ts`
is the runtime's side of it).
Everything hard — dispatch, collection, duplicate-name safety — is the sdk's. This is the thin
wrapper that binds the broker and loads the modules. Keeping the AMQP client here, out of the sdk,
is deliberate: a broker-client change never rebuilds a module (ADR 0039).
wrapper that binds the bus and loads the modules. Keeping the bus client here, out of the sdk, is
deliberate: a bus-client change never rebuilds a module (ADR 0039). The runtime is module-agnostic:
it knows bundles and subjects, nothing of what any module does. A tool that needs root escalates
itself — root is the module's concern, not the runtime's.
## Running it
```
MESH_BROKER_URL amqp://… the mesh broker
MESH_TOOL_MODULES /a/tools/index.js,… the assigned modules' compiled tool entrypoints
MESH_BROKER_FILE the node's sealed credential, as the mesh delivered it
MESH_TOOL_MODULES alpha=/…/alpha/tools/index.js,beta=/…/beta/dist/index.js,…
the modules to serve and their compiled entrypoints; several entries may
name one module. A bare path is an entrypoint of the credential's own module
— the one-module form a per-module container still sets.
MESH_OPERATOR_ACCOUNT whose machine this is, and MESH_OPERATOR_HOME where their home is; set by
the mesh when the node has an account, read by tools from their environment
MESH_BROKER_URL a plain URL instead of the credential, for the bootstrap case
```
`node dist/main.js`, or the container (`Dockerfile`). On a node the host resolves both variables
and starts it like any other supervised workload.
`node dist/main.js`. On a node the controller composes the variables and the host supervises the
process like any other host-side workload (novox/hq to-be 38). As `node-tools` the same process is
the console: MCP on `127.0.0.1:4270` (or `MESH_CONSOLE_LISTEN`). The container (`Dockerfile`) is how
a module's own *service* may still be built; it is no longer how tools reach a node.
## `mesh` — the tools for whoever is on a machine
The same package carries the client (novox/hq design 25 §7, design 34): `mesh tools`, `mesh call
<module>.<tool> [json]`, `mesh mcp` (an MCP server over stdio for a program a person starts) and
`mesh serve` (the **console**: MCP over HTTP on a machine's loopback, started by the mesh as the
`mesh-console` module on the credential in `MESH_BROKER_FILE` — novox/hq ADR 0152). `mesh serve`
refuses to bind anything but loopback. With `--console <url>`, `tools` and `call` go through a console
already on the machine and need no credential.
The console announces six tools and reaches everything else by address (novox/hq ADR 0195):
`mesh_overview` (the mesh's seats and machines), `mesh_machine` (one machine's seats and modules),
`mesh_search` (a tool by words), `mesh_describe` (one tool's arguments), `mesh_call` (call one by
address) and `mesh_runtimes` (which runtimes answered discovery: per runtime its machine, how long its
answer took, its size in bytes, how many modules and tools it announced, whether it was shortened to
fit the bus, when it was last heard — and who was expected and not heard).
Discovery asks the bus (ADR 0197): every runtime answers the NATS services protocol's `$SRV.PING` and
`$SRV.INFO` with what it serves at that moment, and the console reads where the controller's records
place each module. The console waits at least 750 ms, and up to 5 s for every runtime that answered
PING to send what it serves — a large runtime's answer crossing the bus to a broker on another machine
can arrive well after the rest. A runtime that said it is there and did not say what it serves, or
that answered earlier and not now, is named: the console never calls a module missing, or on another
machine, while a runtime that might serve it was not heard. A runtime whose answer outgrows the bus
announces first-line descriptions, then none, and says so. A module may not name a tool of its own
`tools`; the runtime refuses it at load.
## Verified
`npm test` stands up LavinMQ (the mesh's broker) and proves the whole path over real AMQP: the
runtime serves a registered tool, a separate connection invokes it by name and gets the result, and
an unknown tool is refused over the wire.
`npm test` runs against a real NATS server with JetStream (`MESH_TEST_NATS`, see any test's header
for the one-line `docker run`) and proves the whole path over the wire: the runtime serves a
registered tool, a separate connection invokes it by name and gets the result, an unknown tool is
refused, a runtime serves exactly the subjects it is issued and re-serves on a new membership, and
the node's runtime serves three modules' bundles on one credential — one of them broken, named and
not fatal — with every seat verb answering where the membership put it.
-48
View File
@@ -1,48 +0,0 @@
import amqp from "amqplib";
const PORT = process.argv[2];
const MPORT = process.argv[3];
const B = `http://127.0.0.1:${MPORT}`;
const AUTH = "Basic " + Buffer.from("guest:guest").toString("base64");
async function api(method, path, body) {
const r = await fetch(B + path, {
method,
headers: { "content-type": "application/json", authorization: AUTH },
body: body ? JSON.stringify(body) : undefined,
});
if (r.status >= 300 && r.status !== 404) throw new Error(`${method} ${path} -> ${r.status}`);
}
await api("PUT", "/api/exchanges/%2f/mesh.events.dead", { type: "topic", durable: true });
await api("PUT", "/api/users/al", { password: "s", tags: "" });
const Q = "anchor.al.events";
const D = "mesh.events.dead";
const q = Q.replace(/\./g, "\\.");
const d = D.replace(/\./g, "\\.");
// configure, write, read patterns per grant on the dead exchange
const combos = {
"none": { configure: `^${q}$`, write: `^${q}$`, read: `^${q}$` },
"read-dead": { configure: `^${q}$`, write: `^${q}$`, read: `^(${q}|${d})$` },
"write-dead": { configure: `^${q}$`, write: `^(${q}|${d})$`, read: `^${q}$` },
"configure-dead": { configure: `^(${q}|${d})$`, write: `^${q}$`, read: `^${q}$` },
"read+write-dead": { configure: `^${q}$`, write: `^(${q}|${d})$`, read: `^(${q}|${d})$` },
};
let i = 0;
for (const [label, perms] of Object.entries(combos)) {
await api("PUT", "/api/permissions/%2f/al", perms);
const queue = `${Q}.${i++}`; // fresh each time
try {
const c = await amqp.connect(`amqp://al:s@127.0.0.1:${PORT}/`);
const ch = await c.createChannel();
ch.on("error", () => {});
await ch.assertQueue(queue, { durable: true, deadLetterExchange: D });
console.log(`${label}: declare-with-DLX OK`);
await c.close();
} catch (e) {
console.log(`${label}: FAIL - ${String(e.message).slice(0, 70)}`);
}
}
+45
View File
@@ -0,0 +1,45 @@
#!/bin/sh
# mesh-check-toolchain: go
#
# The tool runtime's own check (novox/hq ADR 0237 as amended): the second layer of a pull request's merge
# check, `mesh/repo-check`, run by the build seat in the mesh's Go toolchain — and by hand.
#
# The gate — every machine of the facts snapshot composed with this change — is the build seat's first
# layer (`mesh/merge-gate`), run because the mesh's module graph builds node-tools and mesh-tools from
# here. This is the code's own: node-tools (Go) formatted, vetted and its suite, every test on a bus of its
# own at the release the mesh runs (internal/meshtest), packages in parallel, under the race detector when
# the toolchain has a C compiler.
#
# **Said, never passed silently**: the runtime's and the console's tests launch TypeScript and Python
# bundles that import the mesh's own package (@novox/mesh-sdk), which comes from the package registry — and
# a check of code nobody has approved is given no credential for it. Where node and that package are not
# present those two packages are not tested here, and the TypeScript in node-tools/src neither.
set -eu
cd node-tools
unformatted=$(gofmt -l cmd internal)
if [ -n "$unformatted" ]; then
echo "not gofmt'd:"
echo "$unformatted"
exit 1
fi
CGO_ENABLED=0 go vet ./...
packages=$(go list ./...)
bundleless=""
if ! command -v node >/dev/null 2>&1 || [ ! -d node_modules/@novox/mesh-sdk ]; then
echo "NOT TESTED HERE: internal/runtime and internal/console launch bundles that need node and @novox/mesh-sdk"
packages=$(printf '%s\n' "$packages" | grep -v -e '/internal/runtime$' -e '/internal/console$')
# What of the console launches no bundle runs anyway: how an address resolves (novox/hq issue 287).
bundleless='^TestASeatAndAModuleOfOneNameAreEachReached$'
fi
if command -v gcc >/dev/null 2>&1; then
CGO_ENABLED=1 go test -race -count=1 $packages
else
echo "NOT RACE-CHECKED: the toolchain holds no C compiler; the suite runs without the race detector"
CGO_ENABLED=0 go test -count=1 $packages
fi
if [ -n "$bundleless" ]; then
CGO_ENABLED=0 go test -count=1 -run "$bundleless" ./internal/console
fi
echo "NOT TESTED HERE: node-tools/src (TypeScript) needs @novox/mesh-sdk from the package registry"
+11
View File
@@ -16,6 +16,17 @@
"from": "Dockerfile",
"target": "runtime"
}
],
"on": [
{
"arg": "NODE_BASE",
"image": "node@sha256:48e4b67d85f87bd551df43704e24d252f56cc5f8e9718841aace50f19948f0f9"
},
{
"arg": "MESH_SDK",
"module": "mesh-sdk",
"artifact": "lib"
}
]
},
"resources": []
+11
View File
@@ -0,0 +1,11 @@
# node-tools
The node's tool runtime as a module (novox/hq ADR 0175, to-be 38 WP3). Assigned to a machine, it is
one process the host runs from this bundle, as the operator's account: it serves every assigned
module's tools and every held seat's verbs on the bus, and answers MCP on the machine's loopback —
the console (design 34). The controller composes the process (which bundles to load, where the
credential is, whose machine it is); this manifest says only what the machine must have for it: the
interpreter, a place for the credential, the loopback port, and leave to call every tool.
The code is the `mesh-tools` package in this directory; the module at the repository root,
`mesh-tools`, builds the images TypeScript bundles are compiled in. See the repository README.
+138
View File
@@ -0,0 +1,138 @@
// node-tools — the node's tool runtime, in Go (novox/hq ADR 0175, ADR 0193; to-be 38 WP4d).
//
// One process per machine: it connects to the bus on the node's credential, launches every assigned
// module's tools bundle and serves its tools and its seats' verbs; as the node-tools module — or
// wherever MESH_CONSOLE_LISTEN says — it is also the console, MCP over HTTP on loopback.
//
// MESH_BROKER_FILE the credential the mesh sealed to this machine for the runtime
// MESH_TOOL_MODULES <module>=<entrypoint>,… the bundles to launch
// MESH_TOOL_ENV {"<module>": {"<word>": "<value>"}} what each is given (ADR 0192)
// MESH_OPERATOR_ACCOUNT whose machine this is, and MESH_OPERATOR_HOME their home
// MESH_CONSOLE_LISTEN where the console listens, overriding 127.0.0.1:4270
package main
import (
"encoding/json"
"fmt"
"log"
"math/rand/v2"
"os"
"os/signal"
"syscall"
"time"
"github.com/novox/mesh-tools/node-tools/internal/alive"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/console"
"github.com/novox/mesh-tools/node-tools/internal/runtime"
)
// runtimeModule is the module that is the node's tool runtime; on its credential, serving is also
// the console.
const runtimeModule = "node-tools"
// consoleListen is where the console listens when nothing says otherwise.
const consoleListen = "127.0.0.1:4270"
func main() {
log.SetFlags(0)
if len(os.Args) > 1 && os.Args[1] != "serve" {
fmt.Fprintf(os.Stderr, "node-tools: %q is not a mode; this runtime serves (novox/hq ADR 0193)\n", os.Args[1])
os.Exit(2)
}
cred := credential()
conn := connectPatiently(cred)
served, err := runtime.ServedModulesFrom(os.Getenv(runtime.ToolModules), cred.Module)
if err != nil {
log.Fatalf("node-tools: %v", err)
}
envs, err := runtime.TakeToolEnvs()
if err != nil {
log.Fatalf("node-tools: %v", err)
}
stop, err := runtime.Run(conn, served, envs, log.Printf)
if err != nil {
log.Fatalf("node-tools: %v", err)
}
// And says it is there (novox/hq to-be 45 S11): the machine's runtime, on its machine's name. A
// runtime on another credential — a module's own, in a test — is not the machine's.
stopSaying := func() {}
if cred.Module == runtimeModule && cred.Node != "" {
stopSaying = alive.Keep(conn, cred.Node, alive.Every, log.Printf)
}
listen := os.Getenv("MESH_CONSOLE_LISTEN")
if listen == "" && cred.Module == runtimeModule {
listen = consoleListen
}
var up *console.Listening
if listen != "" {
node := cred.Node
if node == "" {
node = "?"
}
who := node + "." + cred.Module
up, err = console.Serve(console.NewSurface(conn, who), listen)
if err != nil {
log.Fatalf("node-tools: %v", err)
}
log.Printf("mesh console listening on http://%s/mcp as %s", up.Address, who)
}
signals := make(chan os.Signal, 1)
signal.Notify(signals, syscall.SIGTERM, syscall.SIGINT)
<-signals
stopSaying()
stop()
if up != nil {
_ = up.Close()
}
conn.Close()
}
// credential reads the credential the mesh delivered; a missing or unreadable one is a fault of
// configuration, said and final.
func credential() bus.Credential {
file := os.Getenv("MESH_BROKER_FILE")
if file == "" {
if url := os.Getenv("MESH_BROKER_URL"); url != "" {
return bus.Credential{URL: url, Module: runtimeModule}
}
fmt.Fprintln(os.Stderr, "node-tools: set MESH_BROKER_FILE (a sealed credential) or MESH_BROKER_URL — there is no broker to reach")
os.Exit(1)
}
raw, err := os.ReadFile(file)
if err != nil {
fmt.Fprintf(os.Stderr, "node-tools: cannot read the broker credential at %s: %v\n", file, err)
os.Exit(1)
}
var cred bus.Credential
if err := json.Unmarshal(raw, &cred); err != nil {
fmt.Fprintf(os.Stderr, "node-tools: cannot read the broker credential at %s: %v\n", file, err)
os.Exit(1)
}
if cred.URL == "" {
fmt.Fprintf(os.Stderr, "node-tools: %s carries no url — it is not a broker credential\n", file)
os.Exit(1)
}
return cred
}
// connectPatiently retries while the bus is merely not reachable yet — the normal case at startup —
// and gives up at once on what waiting cannot fix (issue 058).
func connectPatiently(cred bus.Credential) *bus.Conn {
for delay := 2 * time.Second; ; delay = min(delay*2, 30*time.Second) {
conn, err := bus.Connect(cred)
if err == nil {
return conn
}
if fatal := bus.Fatal(err); fatal != "" {
fmt.Fprintf(os.Stderr, "node-tools: %s — waiting will not fix this; giving up\n", fatal)
os.Exit(1)
}
wait := delay + time.Duration(rand.IntN(1000))*time.Millisecond
fmt.Fprintf(os.Stderr, "node-tools: the broker is not reachable yet (%v); retrying in %ds\n", err, int(wait.Round(time.Second)/time.Second))
time.Sleep(wait)
}
}
+22
View File
@@ -0,0 +1,22 @@
module github.com/novox/mesh-tools/node-tools
go 1.26.0
require (
github.com/nats-io/nats.go v1.54.0
golang.org/x/sys v0.48.0
)
require (
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/nats-server/v2 v2.11.17 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
go.uber.org/automaxprocs v1.6.0 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/time v0.15.0 // indirect
)
+27
View File
@@ -0,0 +1,27 @@
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
github.com/nats-io/nkeys v0.4.16/go.mod h1:llLgWoI0o4z/Q57q2R1kHfmocyhGV6VG/U18Glg1Afs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=
go.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
+80
View File
@@ -0,0 +1,80 @@
// Package alive is the node tools saying they are there (novox/hq to-be 45 §3, S11).
//
// **A machine heard and its runtime gone is a machine nobody can ask anything.** The node-engine says
// every minute that the machine is there; nothing said whether the runtime serving every module's
// tools and every held seat's verbs was. Now it says so too, on its own subject, every interval, with
// the interval — the controller's watchdog raises `tools-silent` after three missed — and its build.
// Core NATS, kept by nobody, as the host's heartbeat: a lost one is the next one.
package alive
import (
"encoding/json"
"runtime/debug"
"sync"
"time"
)
// Every is how often the runtime says it is there.
const Every = 60 * time.Second
// Subject is where one machine's node tools say it.
func Subject(node string) string { return "mesh.control." + node + ".tools-alive" }
// Beat is what is said: the machine, the interval, and the build.
type Beat struct {
Node string `json:"node"`
IntervalSeconds int `json:"interval_seconds"`
Version string `json:"version,omitempty"`
}
// Sayer publishes one message, kept by nobody.
type Sayer interface {
Say(subject string, body []byte) error
}
// Version is this build's commit, as the toolchain stamped it; empty when it did not.
func Version() string {
info, ok := debug.ReadBuildInfo()
if !ok {
return ""
}
for _, s := range info.Settings {
if s.Key == "vcs.revision" {
return s.Value
}
}
return ""
}
// Keep says the runtime is there now and every interval until the returned function is called. A
// heartbeat that cannot be said is logged when that starts and when it stops — not every minute.
func Keep(bus Sayer, node string, every time.Duration, logf func(string, ...any)) func() {
body, _ := json.Marshal(Beat{Node: node, IntervalSeconds: int(every / time.Second), Version: Version()})
done := make(chan struct{})
var once sync.Once
go func() {
failing := ""
tick := time.NewTicker(every)
defer tick.Stop()
for {
why := ""
if err := bus.Say(Subject(node), body); err != nil {
why = err.Error()
}
if why != failing {
if why != "" {
logf("node-tools: could not say this runtime is there — the mesh will call it silent: %s", why)
} else {
logf("node-tools: says again that this runtime is there")
}
failing = why
}
select {
case <-done:
return
case <-tick.C:
}
}
}()
return func() { once.Do(func() { close(done) }) }
}
+74
View File
@@ -0,0 +1,74 @@
package alive
import (
"encoding/json"
"errors"
"sync"
"testing"
"time"
)
type said struct {
mu sync.Mutex
subs []string
body [][]byte
fail error
}
func (s *said) Say(subject string, body []byte) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.fail != nil {
return s.fail
}
s.subs, s.body = append(s.subs, subject), append(s.body, body)
return nil
}
func (s *said) count() int {
s.mu.Lock()
defer s.mu.Unlock()
return len(s.subs)
}
// **The runtime says it is there at once and every interval, as its machine, with the interval**: the
// controller's bound is three of them (to-be 45 S11).
func TestTheRuntimeSaysItIsThereEveryInterval(t *testing.T) {
s := &said{}
stop := Keep(s, "anchor", 20*time.Millisecond, t.Logf)
time.Sleep(70 * time.Millisecond)
stop()
stop()
if n := s.count(); n < 3 {
t.Fatalf("said %d times in three intervals", n)
}
var beat Beat
if err := json.Unmarshal(s.body[0], &beat); err != nil || beat.Node != "anchor" || beat.IntervalSeconds != 0 ||
s.subs[0] != "mesh.control.anchor.tools-alive" {
t.Fatalf("%s on %s: %v", s.body[0], s.subs[0], err)
}
n := s.count()
time.Sleep(50 * time.Millisecond)
if s.count() != n {
t.Fatal("said after it was stopped")
}
}
// **A heartbeat the bus will not take is said once, and again when it is taken**: not a line a minute.
func TestAHeartbeatThatCannotBeSaidIsLoggedOnce(t *testing.T) {
s := &said{fail: errors.New("permissions violation")}
var mu sync.Mutex
var lines []string
stop := Keep(s, "anchor", 10*time.Millisecond, func(f string, a ...any) {
mu.Lock()
lines = append(lines, f)
mu.Unlock()
})
time.Sleep(60 * time.Millisecond)
stop()
mu.Lock()
defer mu.Unlock()
if len(lines) != 1 {
t.Fatalf("logged %d lines for one failure", len(lines))
}
}
+379
View File
@@ -0,0 +1,379 @@
// Package announce answers the NATS services protocol's discovery for what a runtime serves, and
// gathers the answers (novox/hq ADR 0197).
//
// A runtime does not re-serve its tools through a services library: serving is unchanged. It answers
// `$SRV.PING`, `$SRV.INFO` and `$SRV.STATS` — and the same followed by its service name, and by its
// name and id — in the format NATS's own tools read, with what it is serving at the moment it is
// asked. One service per runtime process: the bus admits one reply per request from each responder,
// so a runtime serving many modules and seats answers once, one endpoint per tool per subject, and
// says in each endpoint's metadata which module, seat, scope and machine it is.
package announce
import (
"encoding/json"
"strings"
"time"
"github.com/nats-io/nats.go/micro"
"github.com/novox/mesh-tools/node-tools/internal/bus"
)
// Version is the announced service version (semver, as the protocol requires).
const Version = "0.1.0"
// Window is how long the console gathers discovery answers at the least: every instance answers one
// request, and how many will is what is being found out.
var Window = 750 * time.Millisecond
// Patience is how long the console waits, at the most, for the full answer of a service that has
// said it is there. **A fixed window lost the largest runtime** (2026-10-05): the laptop's answer —
// 48 modules, 341 endpoints, 164 kB — went up to the broker on another machine and back, arrived
// last every time (a median of 365 ms, one in 25 after 813 ms on a quiet link, later still while the
// runtime re-served), and when it missed the 750 ms window the console said its modules ran nowhere.
// A PING answer is a hundred bytes and arrives at once, so who is there is known early; what each
// serves is waited for until it arrives or this passes, and a service that never sends it is named.
var Patience = 5 * time.Second
// Shortened is the service metadata key a runtime sets when its announcement was cut to fit the bus.
const Shortened = "shortened"
// Kinds of endpoint.
const (
KindTool = "tool" // a module's own tool
KindSeat = "seat" // a seat's verb, served by the module holding it
)
// Endpoint is one tool served on one subject, as it is announced.
type Endpoint struct {
Kind string
Module string // the module whose code answers
Tool string // the tool's or the verb's name
Seat string // for a seat's verb
Scope string // "mesh" or "node", for a seat's verb
Node string
Description string
Schema json.RawMessage
Interchangeable bool
Subject string
Queue string
}
// Name is the endpoint's name as the protocol allows it — letters, digits, `-` and `_` — the
// prefix and the tool joined by `__`; the metadata, not the name, is what identifies it.
func (e Endpoint) Name() string {
prefix := e.Module
if e.Kind == KindSeat {
prefix = e.Seat
}
return clean(prefix) + "__" + clean(e.Tool)
}
func clean(s string) string {
var b strings.Builder
for _, r := range s {
if r == '-' || r == '_' || (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') {
b.WriteRune(r)
} else {
b.WriteRune('_')
}
}
return b.String()
}
func (e Endpoint) info() micro.EndpointInfo {
md := map[string]string{
"kind": e.Kind, "module": e.Module, "tool": e.Tool, "node": e.Node,
"description": e.Description, "interchangeable": boolWord(e.Interchangeable),
}
// **A seat's verb carries its schema; a module's tool does not.** Every tool's schema, once per
// subject it answers on, made a runtime's one answer outgrow the bus's largest message (1 MiB) once
// machines served a couple of hundred tools — and the answer that cannot be sent is silence: the
// machine vanished from discovery (2026-10-04). A module's tool's schema is asked of the runtime that
// serves it when a person describes it (its `tools` verb); a seat's verbs are few, and a seat has no
// `tools` verb of its own to ask.
if e.Kind == KindSeat {
schema := strings.TrimSpace(string(e.Schema))
if schema == "" || schema == "null" {
schema = "{}"
}
md["schema"] = schema
}
if e.Kind == KindSeat {
md["seat"] = e.Seat
md["scope"] = e.Scope
}
return micro.EndpointInfo{Name: e.Name(), Subject: e.Subject, QueueGroup: e.Queue, Metadata: md}
}
func boolWord(b bool) string {
if b {
return "true"
}
return "false"
}
// Service is who answers: the runtime's name, its instance, and a word about it.
type Service struct {
Name string
ID string
Description string
Metadata map[string]string
}
func (s Service) identity() micro.ServiceIdentity {
md := s.Metadata
if md == nil {
md = map[string]string{}
}
return micro.ServiceIdentity{Name: s.Name, ID: s.ID, Version: Version, Metadata: md}
}
// Info is the info_response for these endpoints.
func Info(s Service, endpoints []Endpoint) micro.Info {
out := micro.Info{ServiceIdentity: s.identity(), Type: micro.InfoResponseType,
Description: s.Description, Endpoints: []micro.EndpointInfo{}}
for _, e := range endpoints {
out.Endpoints = append(out.Endpoints, e.info())
}
return out
}
// Serve answers discovery for one service until stopped, asking `current` for its endpoints each
// time — so what is announced is what is served now, re-served memberships included.
func Serve(conn *bus.Conn, s Service, current func() []Endpoint) (func(), error) {
started := time.Now().UTC()
answer := func(subject string, _ []byte) []byte {
parts := strings.Split(subject, ".")
if len(parts) < 2 || parts[0] != "$SRV" {
return nil
}
if len(parts) >= 3 && parts[2] != s.Name {
return nil // another service's
}
if len(parts) >= 4 && parts[3] != s.ID {
return nil // another instance's
}
var v any
switch parts[1] {
case "PING":
v = micro.Ping{ServiceIdentity: s.identity(), Type: micro.PingResponseType}
case "INFO":
v = Info(s, current())
case "STATS":
st := micro.Stats{ServiceIdentity: s.identity(), Type: micro.StatsResponseType, Started: started,
Endpoints: []*micro.EndpointStats{}}
for _, e := range current() {
st.Endpoints = append(st.Endpoints, &micro.EndpointStats{Name: e.Name(), Subject: e.Subject, QueueGroup: e.Queue})
}
v = st
default:
return nil
}
body, err := json.Marshal(v)
if err != nil {
return nil
}
// Too large for the bus is no answer at all, and was silent: shorten every description to its
// first line and say so, rather than vanish; still too large, leave descriptions out.
if limit := conn.MaxPayload(); limit > 0 && int64(len(body)) > limit {
if info, ok := v.(micro.Info); ok {
body = shorten(conn.Logf, info, body, limit)
}
}
return body
}
var stops []func()
for _, verb := range []string{"PING", "INFO", "STATS"} {
// Exactly the questions asked of every service and of this one by name and instance — what the
// grants allow (novox/hq ADR 0197). A wildcard is refused by the bus.
for _, subject := range []string{"$SRV." + verb, "$SRV." + verb + "." + s.Name, "$SRV." + verb + "." + s.Name + "." + s.ID} {
stop, err := conn.Raw(subject, answer)
if err != nil {
for _, st := range stops {
st()
}
return func() {}, err
}
stops = append(stops, stop)
}
}
return func() {
for _, st := range stops {
st()
}
}, nil
}
// shorten is an info_response cut to fit the bus: descriptions to their first line, then none, the
// service's metadata saying which (so discovery can tell a person what it was given).
func shorten(logf func(string, ...any), info micro.Info, body []byte, limit int64) []byte {
md := map[string]string{}
for k, v := range info.Metadata {
md[k] = v
}
info.Metadata = md
for _, step := range []string{"descriptions cut to their first line", "descriptions left out"} {
for i := range info.Endpoints {
e := info.Endpoints[i].Metadata
if step == "descriptions left out" {
delete(e, "description")
} else {
e["description"] = firstLine(e["description"])
}
}
md[Shortened] = step
shorter, err := json.Marshal(info)
if err != nil {
return body
}
logf("[mesh-tools] what this runtime serves is %d bytes, beyond the bus's %d; announced with %s (%d bytes)",
len(body), limit, step, len(shorter))
if int64(len(shorter)) <= limit {
return shorter
}
body = shorter
}
return body
}
// Heard is one service instance's answer to discovery: what it serves, how long it took to arrive,
// and how large it was.
type Heard struct {
Info micro.Info
Took time.Duration
Bytes int
Shortened string // why its descriptions are not whole, or ""
}
// Machine is the machine an instance runs on: its metadata's, or the one its endpoints name.
func (h Heard) Machine() string {
if n := h.Info.Metadata["node"]; n != "" {
return n
}
for _, e := range h.Info.Endpoints {
if n := e.Metadata["node"]; n != "" {
return n
}
}
return ""
}
// Instance names one service instance.
type Instance struct {
Name string
ID string
Machine string
}
// Discovery is what one round of discovery heard, and who said it was there but did not say what it
// serves in time.
type Discovery struct {
At time.Time
Waited time.Duration
Heard []Heard
Silent []Instance
}
// Gather asks every service on the bus who it is and what it serves, and waits at least the window
// and at most the patience: until every instance that answered PING has answered INFO. An answer
// that is not the protocol's is skipped.
func Gather(conn *bus.Conn) (Discovery, error) {
start := time.Now()
d := Discovery{At: start.UTC()}
pings, stopPings, err := conn.Collect("$SRV.PING", nil)
if err != nil {
return d, err
}
defer stopPings()
infos, stopInfos, err := conn.Collect("$SRV.INFO", nil)
if err != nil {
return d, err
}
defer stopInfos()
type key struct{ name, id string }
pinged := map[key]int{}
answered := map[key]int{}
where := map[key]string{}
var order []key
complete := func() bool {
for k, n := range pinged {
if answered[k] < n {
return false
}
}
return true
}
least := time.NewTimer(Window)
defer least.Stop()
most := time.NewTimer(Patience)
defer most.Stop()
windowOver := false
for {
select {
case a := <-pings:
var p micro.Ping
if json.Unmarshal(a.Data, &p) != nil || p.Type != micro.PingResponseType {
continue
}
k := key{p.Name, p.ID}
if _, seen := pinged[k]; !seen {
order = append(order, k)
}
pinged[k]++
if where[k] == "" {
where[k] = p.Metadata["node"]
}
case a := <-infos:
var i micro.Info
if json.Unmarshal(a.Data, &i) != nil || i.Type != micro.InfoResponseType {
continue
}
answered[key{i.Name, i.ID}]++
d.Heard = append(d.Heard, Heard{Info: i, Took: a.At.Sub(start), Bytes: len(a.Data), Shortened: i.Metadata[Shortened]})
case <-least.C:
windowOver = true
case <-most.C:
d.Waited = time.Since(start)
for _, k := range order {
if answered[k] < pinged[k] {
d.Silent = append(d.Silent, Instance{Name: k.name, ID: k.id, Machine: where[k]})
}
}
return d, nil
}
if windowOver && complete() {
d.Waited = time.Since(start)
return d, nil
}
}
}
// Endpoints reads an info_response's endpoints back into what they announce.
func Endpoints(i micro.Info) []Endpoint {
var out []Endpoint
for _, e := range i.Endpoints {
md := e.Metadata
out = append(out, Endpoint{
Kind: md["kind"], Module: md["module"], Tool: md["tool"], Seat: md["seat"], Scope: md["scope"],
Node: md["node"], Description: md["description"], Schema: json.RawMessage(md["schema"]),
Interchangeable: md["interchangeable"] == "true", Subject: e.Subject, Queue: e.QueueGroup,
})
}
return out
}
// firstLine is a description's first sentence or line, at most 160 characters.
func firstLine(s string) string {
if i := strings.IndexAny(s, "\n"); i >= 0 {
s = s[:i]
}
if i := strings.Index(s, ". "); i >= 0 {
s = s[:i+1]
}
if len(s) > 160 {
s = s[:157] + "..."
}
return s
}
@@ -0,0 +1,54 @@
package announce
import (
"encoding/json"
"fmt"
"strings"
"testing"
)
// What a machine serves fits the bus's largest message (1 MiB) with room to spare at several hundred
// tools: a module's tool is announced without its schema, which made a machine with ~200 tools outgrow it
// and vanish from discovery (2026-10-04). A seat's verb keeps its schema.
func TestAMachinesAnnouncementFitsTheBusAtSeveralHundredTools(t *testing.T) {
schema := json.RawMessage(`{"type":"object","properties":{` + strings.Repeat(`"field_with_a_long_name":{"type":"string","description":"a description of what this argument means, long enough to matter"},`, 12) + `"last":{"type":"string"}}}`)
description := strings.Repeat("A tool that does something useful, described at the length the catalogue's tools are. ", 3)
var endpoints []Endpoint
for i := 0; i < 400; i++ {
for _, subject := range []string{fmt.Sprintf("mesh.mod.m%d.tool.t", i), fmt.Sprintf("mesh.mod.m%d.tool.t.laptop", i)} {
endpoints = append(endpoints, Endpoint{Kind: KindTool, Module: fmt.Sprintf("m%d", i), Tool: "t", Node: "laptop",
Description: description, Schema: schema, Subject: subject})
}
}
endpoints = append(endpoints, Endpoint{Kind: KindSeat, Module: "m0", Tool: "verb", Seat: "a-seat", Scope: "mesh",
Description: "a verb", Schema: schema, Subject: "mesh.seat.a-seat.tool.verb"})
body, err := json.Marshal(Info(Service{Name: "node-tools", ID: "laptop"}, endpoints))
if err != nil {
t.Fatal(err)
}
if len(body) > 1<<20 {
t.Fatalf("400 tools announce %d bytes, beyond the bus's 1 MiB", len(body))
}
t.Logf("400 tools on two subjects each announce %d bytes", len(body))
back := Endpoints(Info(Service{Name: "node-tools", ID: "laptop"}, endpoints))
if len(back[0].Schema) != 0 {
t.Fatalf("a module's tool still carries its schema: %s", back[0].Schema)
}
if seat := back[len(back)-1]; !strings.Contains(string(seat.Schema), "field_with_a_long_name") {
t.Fatalf("a seat's verb lost its schema: %s", seat.Schema)
}
withSchemas := 0
for range endpoints {
withSchemas += len(schema) * 2 // escaped inside a string
}
t.Logf("with every schema it would have been over %d bytes", len(body)+withSchemas)
}
func TestFirstLineShortensADescription(t *testing.T) {
if got := firstLine("One thing. And more after it."); got != "One thing." {
t.Fatalf("%q", got)
}
if got := firstLine(strings.Repeat("x", 300)); len(got) != 160 {
t.Fatalf("%d", len(got))
}
}
+156
View File
@@ -0,0 +1,156 @@
package announce
import (
"encoding/json"
"strings"
"testing"
"time"
"github.com/nats-io/nats.go/micro"
"github.com/novox/mesh-tools/node-tools/internal/bus"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
func connect(t *testing.T, module, node string) *bus.Conn {
t.Helper()
c, err := bus.Connect(bus.Credential{URL: mt.URL(t), Module: module, Node: node})
if err != nil {
t.Fatal(err)
}
c.Logf = func(string, ...any) {}
t.Cleanup(c.Close)
return c
}
// lagging answers PING at once and INFO after `lag` — or never, with a negative lag: a runtime whose
// one large answer is slow to cross the bus, as the laptop's was (2026-10-05).
func lagging(t *testing.T, conn *bus.Conn, s Service, lag time.Duration, endpoints []Endpoint) {
t.Helper()
ping, _ := json.Marshal(micro.Ping{ServiceIdentity: s.identity(), Type: micro.PingResponseType})
stop, err := conn.Raw("$SRV.PING", func(string, []byte) []byte { return ping })
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
stop, err = conn.Raw("$SRV.INFO", func(string, []byte) []byte {
if lag < 0 {
return nil
}
time.Sleep(lag)
body, _ := json.Marshal(Info(s, endpoints))
return body
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
conn.Flush()
}
// A runtime whose answer arrives after the window is waited for, because it said it was there; one
// that never says what it serves is named, not dropped. With the fixed 750 ms window the laptop's
// modules were said to run nowhere whenever its answer was late (2026-10-05).
func TestDiscoveryWaitsForEveryRuntimeThatSaidItIsThere(t *testing.T) {
was := Patience
Patience = 2500 * time.Millisecond
t.Cleanup(func() { Patience = was })
fast := connect(t, "node-tools", "desk")
stopFast, err := Serve(fast, Service{Name: "node-tools", ID: "desk", Metadata: map[string]string{"node": "desk"}}, func() []Endpoint {
return []Endpoint{{Kind: KindTool, Module: "alpha", Tool: "one", Node: "desk", Subject: "mesh.mod.alpha.tool.one.desk"}}
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(stopFast)
fast.Flush()
slow := connect(t, "node-tools", "laptop")
lagging(t, slow, Service{Name: "node-tools", ID: "laptop", Metadata: map[string]string{"node": "laptop"}}, Window+400*time.Millisecond,
[]Endpoint{{Kind: KindTool, Module: "slack", Tool: "slack_check", Node: "laptop", Subject: "mesh.mod.slack.tool.slack_check.laptop"}})
mute := connect(t, "node-tools", "sleeper")
lagging(t, mute, Service{Name: "node-tools", ID: "sleeper", Metadata: map[string]string{"node": "sleeper"}}, -1, nil)
asker := connect(t, "console", "desk")
start := time.Now()
d, err := Gather(asker)
if err != nil {
t.Fatal(err)
}
took := time.Since(start)
heard := map[string]Heard{}
for _, h := range d.Heard {
heard[h.Info.ID] = h
}
if h, ok := heard["laptop"]; !ok {
t.Fatalf("the slow runtime was not heard: %+v", d.Heard)
} else if h.Took <= Window || h.Machine() != "laptop" || h.Bytes == 0 {
t.Errorf("the slow runtime's answer: took %s (window %s), machine %q, %d bytes", h.Took, Window, h.Machine(), h.Bytes)
}
if _, ok := heard["desk"]; !ok {
t.Errorf("the fast runtime was not heard: %+v", d.Heard)
}
if len(d.Silent) != 1 || d.Silent[0].ID != "sleeper" || d.Silent[0].Machine != "sleeper" {
t.Errorf("the runtime that never said what it serves is not named: %+v", d.Silent)
}
if took < Patience || took > Patience+time.Second {
t.Errorf("waited %s with a silent runtime; patience is %s", took, Patience)
}
}
// With nobody slow, discovery takes the window and no longer.
func TestDiscoveryDoesNotWaitWhenEveryoneAnswered(t *testing.T) {
fast := connect(t, "node-tools", "desk")
stop, err := Serve(fast, Service{Name: "node-tools", ID: "desk"}, func() []Endpoint { return nil })
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
fast.Flush()
start := time.Now()
d, err := Gather(connect(t, "console", "desk"))
if err != nil {
t.Fatal(err)
}
if took := time.Since(start); took > Window+500*time.Millisecond || len(d.Silent) != 0 || len(d.Heard) == 0 {
t.Errorf("took %s, heard %d, silent %+v", took, len(d.Heard), d.Silent)
}
}
// Too large even with first lines, descriptions are left out — and the answer says which, without
// touching the service's own metadata.
func TestAnAnswerTooLargeIsShortenedUntilItFits(t *testing.T) {
long := strings.Repeat("A sentence that is long. ", 20)
var endpoints []Endpoint
for i := 0; i < 50; i++ {
endpoints = append(endpoints, Endpoint{Kind: KindTool, Module: "m", Tool: "t" + strings.Repeat("x", i), Node: "laptop",
Description: long, Subject: "mesh.mod.m.tool.t"})
}
own := map[string]string{"node": "laptop"}
info := Info(Service{Name: "node-tools", ID: "laptop", Metadata: own}, endpoints)
full, _ := json.Marshal(info)
firstLines := 0
for range endpoints {
firstLines += len(firstLine(long))
}
limit := int64(len(full) - len(long)*len(endpoints) + firstLines/2) // first lines alone do not fit
var said []string
got := shorten(func(f string, a ...any) { said = append(said, f) }, info, full, limit)
if int64(len(got)) > limit {
t.Fatalf("still %d bytes, beyond %d", len(got), limit)
}
var back micro.Info
if err := json.Unmarshal(got, &back); err != nil {
t.Fatal(err)
}
if back.Metadata[Shortened] != "descriptions left out" || back.Metadata["node"] != "laptop" {
t.Errorf("metadata: %v", back.Metadata)
}
if _, has := own[Shortened]; has {
t.Error("the service's own metadata was changed")
}
if len(said) != 2 {
t.Errorf("each step is said: %v", said)
}
}
+899
View File
@@ -0,0 +1,899 @@
// Package bus is the node runtime's connection to the mesh bus, on NATS (novox/hq design 25, design
// 29, ADR 0160, ADR 0175). It is the Go port of node-tools' broker-nats.ts, and speaks the same
// wire: the same subjects, the same JSON request and reply bodies, the same event headers.
//
// mesh.mod.<module>.event.<type> an event a module emits
// mesh.mod.<module>.tool.<tool> a tool a module serves
// mesh.seat.<seat>.tool.<verb> a role's verb, answered by whoever holds the seat
package bus
import (
"context"
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"log"
"regexp"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-tools/node-tools/internal/wire"
)
// RequestTimeout is how long a call waits for its answer — what modules already expect. The
// controller answers its verbs well inside it (its AnswerWithin, ten seconds), with a call id while a
// verb still runs, so a caller is never left guessing past this (novox/hq issue 265).
const RequestTimeout = 30 * time.Second
const assignmentsStream = "ASSIGNMENTS"
// Credential is a broker credential as the mesh delivers it (novox/hq ADR 0120).
type Credential struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node,omitempty"`
Module string `json:"module,omitempty"`
User string `json:"user,omitempty"`
Password string `json:"password,omitempty"`
Claims []Claim `json:"claims,omitempty"`
}
// Claim is a seat a module claims, with the verbs it promises (novox/hq ADR 0159).
type Claim struct {
Seat string `json:"seat"`
Scope string `json:"scope,omitempty"`
Serves []string `json:"serves,omitempty"`
}
// Membership is what the mesh issued one assignment (novox/hq ADR 0160).
type Membership struct {
Node string `json:"node"`
Module string `json:"module"`
Serves []Served `json:"serves"`
Seats []SeatVerb `json:"seats,omitempty"`
Emits string `json:"emits"`
Reaches map[string][]string `json:"reaches,omitempty"`
Tools string `json:"tools"`
// State is every bucket the module's code may reach, by the name it uses (novox/hq ADR 0201).
State []StateIssued `json:"state,omitempty"`
}
// Served is an address a tool is answered on; `{tool}` stands for the tool's name.
type Served struct {
Subject string `json:"subject"`
Queue string `json:"queue,omitempty"`
}
// SeatVerb is one verb of a seat a module holds, where it is answered.
type SeatVerb struct {
Seat string `json:"seat"`
Verb string `json:"verb"`
Subject string `json:"subject"`
}
// Envelope is an event as a module emits it: the body is the payload, the metadata rides as headers
// (novox/hq ADR 0042).
type Envelope struct {
Key string `json:"key"`
Node string `json:"node,omitempty"`
Body json.RawMessage `json:"body"`
Headers map[string]string `json:"headers,omitempty"`
}
// Answered is a call's result and the machine that gave it (novox/hq ADR 0159).
type Answered struct {
Result json.RawMessage
Node string
}
// Handler answers one request body.
type Handler func(body json.RawMessage) (any, error)
// ErrPin is a bus whose certificate is not the one the mesh pinned: final, never retried.
var ErrPin = errors.New("the bus's certificate does not match the pin")
// Fatal is why a connection failure is final rather than "not yet", or "" when waiting may fix it —
// the same classification the TypeScript runtime makes.
func Fatal(err error) string {
if err == nil {
return ""
}
msg := err.Error()
if errors.Is(err, ErrPin) || strings.Contains(msg, ErrPin.Error()) {
return "the bus's certificate does not match the pin"
}
if regexp.MustCompile(`(?i)invalid url|no servers available for connection: .*url`).MatchString(msg) ||
strings.Contains(msg, "nats: invalid url") {
return "the bus address is not a usable URL"
}
if regexp.MustCompile(`(?i)authorization violation|user authentication expired|permissions violation`).MatchString(msg) {
return "the bus refused this account"
}
return ""
}
func normalizeFingerprint(f string) string {
f = strings.TrimSpace(f)
if len(f) > 7 && strings.EqualFold(f[:7], "sha256:") {
f = f[7:]
}
return strings.ToLower(strings.ReplaceAll(f, ":", ""))
}
// pinned accepts exactly the certificate with this SHA-256 and no other. The pin is the only check:
// the bus's certificate names the seat, not the address a machine dials it by.
func pinned(want string) *tls.Config {
want = normalizeFingerprint(want)
return &tls.Config{
InsecureSkipVerify: true, //nolint:gosec // replaced by the pin, which is stricter
MinVersion: tls.VersionTLS12,
VerifyPeerCertificate: func(raw [][]byte, _ [][]*x509.Certificate) error {
if len(raw) == 0 {
return fmt.Errorf("%w: it presented none", ErrPin)
}
sum := sha256.Sum256(raw[0])
if got := hex.EncodeToString(sum[:]); got != want {
return fmt.Errorf("%w: it presented %s, not the pinned %s", ErrPin, got, want)
}
return nil
},
}
}
// Conn is the runtime's connection: the module it is, the memberships it follows, the subjects it
// answers.
type Conn struct {
nc *nats.Conn
js nats.JetStreamContext
self string
node string
cred Credential
mu sync.Mutex
issued map[string]*Membership // module → membership; present with nil = followed, none issued
onNew []func(Membership)
subs []*nats.Subscription
Logf func(format string, args ...any)
// answering is every subject this runtime answers on, so a subscription the bus refused can be
// asked for again (novox/hq issue 222).
answering map[string]*answered
// RetryAfter is how long after a refusal a subscription is asked for again, by attempt; past the
// last, it is given up and said. A variable so a test need not wait minutes.
RetryAfter []time.Duration
}
// answered is one subject the runtime answers on: what to subscribe again with, and how often it was
// refused.
type answered struct {
subject, queue string
cb nats.MsgHandler
sub *nats.Subscription
refusals int
stopped bool
}
// retryAfter is the default wait before each new attempt: a push sends the bus's user list in the
// same breath as the machine that needs it, and the bus reloads it a moment later — five minutes
// covers a slow push, and a grant that never comes is said and left.
var retryAfter = []time.Duration{2 * time.Second, 5 * time.Second, 10 * time.Second, 20 * time.Second,
30 * time.Second, 60 * time.Second, 60 * time.Second, 120 * time.Second}
var (
refusedSubject = regexp.MustCompile(`Subscription to "(\S+)"`)
refusedQueue = regexp.MustCompile(`using queue "(\S+)"`)
)
func answeringKey(subject, queue string) string { return subject + "\x00" + queue }
// refused is the bus saying no to a subscription. **Asked again, not given up** (novox/hq issue 222):
// what an account may answer is the bus's user list, written on the bus's machine, and a push that
// assigns a module somewhere sends that machine and the bus's in the same breath — the runtime can
// subscribe in the moment before the bus has reloaded. Refused once, the subscription stayed dead
// until some later membership happened to re-serve it, and the module ran unreachable meanwhile.
func (c *Conn) refused(err error) {
if !errors.Is(err, nats.ErrPermissionViolation) {
return
}
m := refusedSubject.FindStringSubmatch(err.Error())
if len(m) < 2 {
return
}
queue := ""
if q := refusedQueue.FindStringSubmatch(err.Error()); len(q) >= 2 {
queue = q[1]
}
c.mu.Lock()
a := c.answering[answeringKey(m[1], queue)]
if a == nil || a.stopped {
c.mu.Unlock()
return
}
waits := c.RetryAfter
if waits == nil {
waits = retryAfter
}
if a.refusals >= len(waits) {
c.mu.Unlock()
c.Logf("[mesh-tools] the bus still refuses %s after %d attempts; not served here until the mesh issues it again", a.subject, a.refusals)
return
}
wait := waits[a.refusals]
a.refusals++
attempt := a.refusals
c.mu.Unlock()
c.Logf("[mesh-tools] the bus refused %s; asking again in %s (attempt %d)", a.subject, wait, attempt)
time.AfterFunc(wait, func() {
c.mu.Lock()
defer c.mu.Unlock()
if a.stopped {
return
}
if a.sub != nil {
_ = a.sub.Unsubscribe()
}
var sub *nats.Subscription
var err error
if a.queue != "" {
sub, err = c.nc.QueueSubscribe(a.subject, a.queue, a.cb)
} else {
sub, err = c.nc.Subscribe(a.subject, a.cb)
}
if err != nil {
c.Logf("[mesh-tools] asking again for %s failed: %v", a.subject, err)
return
}
a.sub = sub
c.subs = append(c.subs, sub)
})
}
// Connect dials the bus as the credential's module. A module's subjects come from its credential,
// never from its calls (ADR 0074).
func Connect(cred Credential) (*Conn, error) {
if cred.Module == "" {
return nil, errors.New("a broker credential with no module: the runtime derives its subjects " +
"from the account the mesh issued, and cannot guess which module it is")
}
node := cred.Node
if node == "" {
node = "?"
}
var c *Conn
opts := []nats.Option{
nats.Name(node + "." + cred.Module),
nats.ErrorHandler(func(_ *nats.Conn, _ *nats.Subscription, err error) {
if c != nil {
c.refused(err)
}
}),
// Reconnect forever: the bus restarting is an upgrade, not a reason to exit.
nats.MaxReconnects(-1),
}
if cred.User != "" {
opts = append(opts, nats.UserInfo(cred.User, cred.Password),
// Its own inbox: every user's inbox is private to it (design 25 §4).
nats.CustomInboxPrefix("_INBOX."+cred.User))
}
if strings.TrimSpace(cred.Fingerprint) != "" {
opts = append(opts, nats.Secure(pinned(cred.Fingerprint)))
}
nc, err := nats.Connect(cred.URL, opts...)
if err != nil {
return nil, err
}
js, err := nc.JetStream()
if err != nil {
nc.Close()
return nil, err
}
c = &Conn{nc: nc, js: js, self: cred.Module, node: cred.Node, cred: cred,
issued: map[string]*Membership{}, Logf: log.Printf, answering: map[string]*answered{}}
c.Follow(cred.Module)
return c, nil
}
// Module is what this connection is.
func (c *Conn) Module() string { return c.self }
// Node is the machine this connection's account is scoped to.
func (c *Conn) Node() string { return c.node }
// Credential is what this connection was opened with.
func (c *Conn) Credential() Credential { return c.cred }
// MembershipSubject is the one address a runtime derives for an assignment (ADR 0160).
func MembershipSubject(node, module string) string {
return "mesh.assignment." + node + "." + module
}
// Follow reads a module's membership on this machine once and follows it live, so its tools are
// served where the mesh issued them (ADR 0175).
func (c *Conn) Follow(module string) {
c.mu.Lock()
if c.node == "" {
c.mu.Unlock()
return
}
if _, has := c.issued[module]; has {
c.mu.Unlock()
return
}
c.issued[module] = nil
c.mu.Unlock()
subject := MembershipSubject(c.node, module)
// The subject-addressed direct get: the one address the mesh grants this account on the
// stream's API.
if got, err := c.nc.Request("$JS.API.DIRECT.GET."+assignmentsStream+"."+subject, nil, 5*time.Second); err == nil {
if got.Header.Get("Status") == "" && len(got.Data) > 0 {
var m Membership
if json.Unmarshal(got.Data, &m) == nil {
c.mu.Lock()
c.issued[module] = &m
c.mu.Unlock()
}
}
}
if c.Membership(module) == nil {
c.Logf("[mesh-tools] no membership issued for %s on %s yet; serving the derived shape until one arrives", module, c.node)
}
sub, err := c.nc.Subscribe(subject, func(msg *nats.Msg) {
var m Membership
if err := json.Unmarshal(msg.Data, &m); err != nil {
c.Logf("[mesh-tools] a membership arrived that is not one: %v", err)
return
}
c.mu.Lock()
c.issued[module] = &m
handlers := append([]func(Membership){}, c.onNew...)
c.mu.Unlock()
c.Logf("[mesh-tools] %s on %s was issued a new membership; re-serving on it", module, c.node)
for _, h := range handlers {
h(m)
}
_ = c.nc.Flush()
})
if err == nil {
c.track(sub)
}
}
// Membership is what the mesh issued a module here, or nil when nothing has been issued.
func (c *Conn) Membership(module string) *Membership {
c.mu.Lock()
defer c.mu.Unlock()
return c.issued[module]
}
// Following says whether this connection follows a module's membership.
func (c *Conn) Following(module string) bool {
c.mu.Lock()
defer c.mu.Unlock()
_, has := c.issued[module]
return has
}
// Serving is every module whose membership this connection follows.
func (c *Conn) Serving() []string {
c.mu.Lock()
defer c.mu.Unlock()
out := make([]string, 0, len(c.issued))
for m := range c.issued {
out = append(out, m)
}
return out
}
// OnMembership is called with every new membership any followed module is issued.
func (c *Conn) OnMembership(h func(Membership)) {
c.mu.Lock()
c.onNew = append(c.onNew, h)
c.mu.Unlock()
}
func (c *Conn) track(s *nats.Subscription) {
c.mu.Lock()
c.subs = append(c.subs, s)
c.mu.Unlock()
}
// servedOn is where a served module's tool is answered: its membership's subjects when issued, the
// derived shape otherwise (the shape the mesh issues on day one).
func (c *Conn) servedOn(module, tool string) []Served {
if m := c.Membership(module); m != nil {
out := make([]Served, 0, len(m.Serves)+1)
for _, s := range m.Serves {
out = append(out, Served{Subject: strings.ReplaceAll(s.Subject, "{tool}", tool), Queue: s.Queue})
}
if tool == "tools" && m.Tools != "" {
found := false
for _, s := range out {
found = found || s.Subject == m.Tools
}
if !found {
out = append([]Served{{Subject: m.Tools, Queue: "serve." + module}}, out...)
}
}
return out
}
base := "mesh.mod." + module + ".tool." + tool
out := []Served{{Subject: base, Queue: "serve." + module}}
if c.node != "" {
out = append(out, Served{Subject: base + "." + c.node})
}
return out
}
type reply struct {
Result any `json:"result,omitempty"`
Error string `json:"error,omitempty"`
Node string `json:"node,omitempty"`
}
// answerOn answers one subject with one handler, and says which machine answered (ADR 0159).
func (c *Conn) answerOn(subject, queue string, h Handler) (func(), error) {
cb := func(msg *nats.Msg) {
go func() {
var r reply
result, err := h(json.RawMessage(msg.Data))
if err != nil {
r.Error = err.Error()
} else {
r.Result = nullable(result)
}
r.Node = c.node
body, _ := wire.Marshal(r)
_ = msg.Respond(body)
}()
}
var sub *nats.Subscription
var err error
if queue != "" {
sub, err = c.nc.QueueSubscribe(subject, queue, cb)
} else {
sub, err = c.nc.Subscribe(subject, cb)
}
if err != nil {
return func() {}, err
}
c.track(sub)
a := &answered{subject: subject, queue: queue, cb: cb, sub: sub}
c.mu.Lock()
c.answering[answeringKey(subject, queue)] = a
c.mu.Unlock()
return func() {
c.mu.Lock()
a.stopped = true
if c.answering[answeringKey(subject, queue)] == a {
delete(c.answering, answeringKey(subject, queue))
}
current := a.sub
c.mu.Unlock()
_ = current.Unsubscribe()
}, nil
}
// nullable keeps a nil result as JSON null rather than dropping the key: the TypeScript reply always
// carries `result` when the handler did not throw.
func nullable(v any) any {
if v == nil {
return json.RawMessage("null")
}
return v
}
// HandleSubject answers a subject outright: a seat's verb where the mesh issued it.
func (c *Conn) HandleSubject(subject string, h Handler) (func(), error) {
return c.answerOn(subject, "", h)
}
// Handle serves `<module>.<tool>` where the mesh issued that module, and follows its membership:
// when a new one arrives, it serves where it now says and stops where it no longer does.
func (c *Conn) Handle(key string, h Handler) (func(), error) {
if strings.HasPrefix(key, "seat:") {
subject, err := ToolSubject(key, c.self)
if err != nil {
return func() {}, err
}
return c.answerOn(subject, "", h)
}
module, tool := c.self, key
if dot := strings.Index(key, "."); dot >= 0 {
module, tool = key[:dot], key[dot+1:]
}
if module != c.self && !c.Following(module) {
return func() {}, fmt.Errorf("%s cannot serve %s: a module serves its own tools, and a runtime "+
"those of the modules it follows", c.self, key)
}
var mu sync.Mutex
var stops []func()
serve := func() {
mu.Lock()
defer mu.Unlock()
for _, s := range stops {
s()
}
stops = nil
for _, s := range c.servedOn(module, tool) {
if stop, err := c.answerOn(s.Subject, s.Queue, h); err == nil {
stops = append(stops, stop)
} else {
c.Logf("[mesh-tools] cannot serve %s on %s: %v", key, s.Subject, err)
}
}
}
serve()
c.OnMembership(func(m Membership) {
if m.Module == module {
serve()
}
})
return func() {
mu.Lock()
defer mu.Unlock()
for _, s := range stops {
s()
}
stops = nil
}, nil
}
// reachedAt is where a call by key goes: a subject this connection's own membership says it
// reaches — the machine's when named — else the derived shape.
func (c *Conn) reachedAt(key string) (string, error) {
name, wanted, _ := strings.Cut(key, "@")
if m := c.Membership(c.self); m != nil {
if reach := m.Reaches[name]; len(reach) > 0 {
if wanted == "" {
return reach[0], nil
}
for _, s := range reach {
if strings.HasSuffix(s, "."+wanted) {
return s, nil
}
}
}
}
return ToolSubject(key, c.self)
}
// Ask calls a tool by key — or on a subject the mesh listed for it — and learns which machine
// answered. Core request/reply: a tool call is never persisted (design 25 §3).
func (c *Conn) Ask(key string, body any, on string) (Answered, error) {
subject := on
if subject == "" {
s, err := c.reachedAt(key)
if err != nil {
return Answered{}, err
}
subject = s
}
data, err := wire.Marshal(body)
if err != nil {
return Answered{}, err
}
answered, handingOver, err := c.askOnce(subject, data)
if !handingOver {
return answered, err
}
// **Refused because the controller is handing over: asked once more** (novox/hq issue 289). The
// controller being replaced did nothing and said so; the one after it answers the same call. Once:
// a second refusal is a controller that is not coming back, and the caller is told.
time.Sleep(HandoverPause)
answered, again, err := c.askOnce(subject, data)
if err != nil && again {
return Answered{}, fmt.Errorf("asked twice, %s apart, and both times refused as a handover: %w", HandoverPause, err)
}
return answered, err
}
// HandoverPause is how long a call refused as a handover waits before it is asked once more: the
// supervisor's restart of the controller, which stops the one being replaced and starts the next. A
// variable so a test need not wait.
var HandoverPause = 3 * time.Second
// RetryHandingOver is the mark the controller puts on a call it refused because it is being replaced
// (mesh-controller internal/link, `"retry": "handing-over"`).
const RetryHandingOver = "handing-over"
// askOnce asks once, and says whether the answer was a refusal marked to be asked again.
func (c *Conn) askOnce(subject string, data []byte) (Answered, bool, error) {
msg, err := c.nc.Request(subject, data, RequestTimeout)
if err != nil {
if errors.Is(err, nats.ErrNoResponders) {
return Answered{}, false, errors.New("503 no responders")
}
if errors.Is(err, nats.ErrTimeout) {
return Answered{}, false, errors.New("timeout")
}
return Answered{}, false, err
}
var r struct {
Result json.RawMessage `json:"result"`
Error string `json:"error"`
Node string `json:"node"`
Retry string `json:"retry"`
}
if err := json.Unmarshal(msg.Data, &r); err != nil {
return Answered{}, false, err
}
if r.Error != "" {
return Answered{}, r.Retry == RetryHandingOver, errors.New(r.Error)
}
return Answered{Result: r.Result, Node: r.Node}, false, nil
}
// PublishAs emits an event as a module: published into JetStream and awaited, de-duplicated by its
// own id (ADR 0042). The body is the payload; the metadata rides as headers.
func (c *Conn) PublishAs(module string, env Envelope) error {
msg := nats.NewMsg("mesh.mod." + module + ".event." + env.Key)
for k, v := range env.Headers {
msg.Header.Set(k, v)
}
if env.Headers["content-type"] == "" {
msg.Header.Set("content-type", "application/json")
}
if env.Node != "" {
msg.Header.Set("x-node", env.Node)
}
body := env.Body
if len(body) == 0 {
body = json.RawMessage("null")
}
msg.Data = body
var opts []nats.PubOpt
if id := env.Headers["x-event-id"]; id != "" {
opts = append(opts, nats.MsgId(id))
}
_, err := c.js.PublishMsg(msg, opts...)
return err
}
// Say publishes one message on core NATS, kept by nobody: a heartbeat, whose loss is the next one.
func (c *Conn) Say(subject string, body []byte) error { return c.nc.Publish(subject, body) }
// ServedOn is where a served module's tool is answered right now: the membership's subjects when
// issued, the derived shape otherwise — what Handle subscribes, for what announces it (ADR 0197).
func (c *Conn) ServedOn(module, tool string) []Served { return c.servedOn(module, tool) }
// Raw answers one subject with a function of the request, not a tool's reply envelope: the NATS
// services protocol's discovery subjects answer in their own format (novox/hq ADR 0197). A nil
// answer is no reply — the request was for another service.
func (c *Conn) Raw(subject string, answer func(subject string, data []byte) []byte) (func(), error) {
sub, err := c.nc.Subscribe(subject, func(msg *nats.Msg) {
if body := answer(msg.Subject, msg.Data); body != nil {
// Said, never dropped: an answer the bus will not carry was silence before (2026-10-04).
if err := msg.Respond(body); err != nil {
c.Logf("[mesh-tools] could not answer %s (%d bytes): %v", msg.Subject, len(body), err)
}
}
})
if err != nil {
return func() {}, err
}
c.track(sub)
return func() { _ = sub.Unsubscribe() }, nil
}
// Arrival is one answer to a request, and when it arrived.
type Arrival struct {
Data []byte
At time.Time
}
// Collect publishes one request and hands every answer to the channel as it arrives, until stopped:
// for a caller that decides for itself when it has heard enough — discovery waits for every service
// that said it is there, not for a fixed moment (2026-10-05). Answers beyond what the caller has
// taken are kept up to the channel's room; an empty answer is no answer.
func (c *Conn) Collect(subject string, body []byte) (<-chan Arrival, func(), error) {
inbox := c.nc.NewRespInbox()
out := make(chan Arrival, 1024)
var mu sync.Mutex
stopped := false
sub, err := c.nc.Subscribe(inbox, func(msg *nats.Msg) {
if len(msg.Data) == 0 {
return
}
a := Arrival{Data: msg.Data, At: time.Now()}
mu.Lock()
defer mu.Unlock()
if stopped {
return
}
select {
case out <- a:
default:
c.Logf("[mesh-tools] more answers to %s than are kept; one (%d bytes) dropped", subject, len(msg.Data))
}
})
if err != nil {
return nil, func() {}, err
}
stop := func() {
_ = sub.Unsubscribe()
mu.Lock()
stopped = true
mu.Unlock()
}
if err := c.nc.PublishRequest(subject, inbox, body); err != nil {
stop()
return nil, func() {}, err
}
return out, stop, nil
}
// MaxPayload is the largest message the bus carries, as the server told this connection.
func (c *Conn) MaxPayload() int64 { return c.nc.MaxPayload() }
// Flush waits until the bus has every subscription made so far, so what is served is answerable
// when this returns.
func (c *Conn) Flush() { _ = c.nc.Flush() }
// Close unsubscribes everything and drains, so an in-flight reply is finished rather than dropped.
func (c *Conn) Close() {
c.mu.Lock()
subs := c.subs
c.subs = nil
c.mu.Unlock()
for _, s := range subs {
_ = s.Unsubscribe()
}
_ = c.nc.Drain()
}
// ToolSubject is a tool's subject. A bare name is this module's own; `<module>.<tool>` another's;
// `seat:<seat>.<verb>` a role's, with `@<node>` for a node-scoped seat (design 33 §4).
func ToolSubject(key, self string) (string, error) {
if strings.HasPrefix(key, "seat:") {
rest := strings.TrimPrefix(key, "seat:")
dot := strings.Index(rest, ".")
if dot < 0 {
return "", fmt.Errorf("%q names a seat and no verb: seat:<seat>.<verb>", key)
}
seat := rest[:dot]
verb, node, _ := strings.Cut(rest[dot+1:], "@")
if node != "" {
return "mesh.seat." + seat + ".tool." + verb + "." + node, nil
}
return "mesh.seat." + seat + ".tool." + verb, nil
}
name, node, _ := strings.Cut(key, "@")
var base string
if dot := strings.Index(name, "."); dot < 0 {
base = "mesh.mod." + self + ".tool." + name
} else {
base = "mesh.mod." + name[:dot] + ".tool." + name[dot+1:]
}
if node != "" {
return base + "." + node, nil
}
return base, nil
}
// SeatToolSubject is a seat's verb as its holder serves it: flat for a mesh seat, carrying the
// machine for a node-scoped one.
func SeatToolSubject(seat, verb, scope, node string) string {
base := "mesh.seat." + seat + ".tool." + verb
if scope == "node" && node != "" {
return base + "." + node
}
return base
}
// AskAs calls a tool on a module's behalf (novox/hq ADR 0198): a bare key is that module's own tool,
// `<module>.<tool>` another's, `seat:<seat>.<verb>[@<node>]` a role's — resolved through what the
// mesh issued that module to reach, as its own runtime resolved it, else the derived shape.
func (c *Conn) AskAs(module, key string, body any) (Answered, error) {
name, wanted, _ := strings.Cut(key, "@")
subject := ""
if m := c.Membership(module); m != nil {
if reach := m.Reaches[name]; len(reach) > 0 {
subject = reach[0]
if wanted != "" {
subject = ""
for _, s := range reach {
if strings.HasSuffix(s, "."+wanted) {
subject = s
}
}
}
}
}
if subject == "" {
s, err := ToolSubject(key, module)
if err != nil {
return Answered{}, err
}
subject = s
}
return c.Ask(key, body, subject)
}
// EventsStream is where every module's events land, and ConsumerOf the durable consumer the
// controller makes for a module on a machine: the same names the module's own runtime bound
// (node-tools/src/broker-nats.ts), so moving the module into the node's runtime neither loses an
// event nor sees one twice.
const EventsStream = "EVENTS"
// ConsumerOf is a module's durable consumer on a machine: `<node>_<module>`.
func ConsumerOf(node, module string) string {
if node == "" {
node = "?"
}
return node + "_" + module
}
// NakDelay is how long an event a handler failed waits before it is offered again: a transient cause
// gets another attempt, a permanent one exhausts the consumer's max-deliver rather than spinning.
var NakDelay = 5 * time.Second
// ConsumeAs reads a module's durable consumer and hands each event to deliver (novox/hq ADR 0198):
// acknowledged when deliver returns nil, negatively acknowledged after NakDelay when it returns an
// error, terminated when it is not an event at all. The consumer is the controller's to create; this
// binds to it and never makes one. Runs until stopped.
func (c *Conn) ConsumeAs(module string, deliver func(Envelope) error) (func(), error) {
durable := ConsumerOf(c.node, module)
js, err := jetstream.New(c.nc)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
// Bound by name, never created and never matched against a subject: the consumer and its
// filters are the controller's (design 29 §3), exactly as the module's own runtime bound it.
consumer, err := js.Consumer(ctx, EventsStream, durable)
if err != nil {
return nil, fmt.Errorf("binding %s's consumer %s: %w", module, durable, err)
}
reading, err := consumer.Consume(func(msg jetstream.Msg) {
env, ok := envelopeOf(msg.Subject(), msg.Headers(), msg.Data())
if !ok {
// Unparseable: redelivering bytes no version of a handler can read is a loop.
_ = msg.Term()
return
}
if err := deliver(env); err != nil {
_ = msg.NakWithDelay(NakDelay)
return
}
_ = msg.Ack()
})
if err != nil {
return nil, fmt.Errorf("reading %s's consumer %s: %w", module, durable, err)
}
var once sync.Once
return func() { once.Do(reading.Stop) }, nil
}
// envelopeOf rebuilds the envelope a module sees from a delivered event: its key is the emitter and
// the event, recovered from the subject; its metadata rides as headers (novox/hq ADR 0042).
func envelopeOf(subject string, header nats.Header, data []byte) (Envelope, bool) {
if !json.Valid(data) {
return Envelope{}, false
}
headers := map[string]string{}
for k := range header {
headers[k] = header.Get(k)
}
return Envelope{Key: keyOf(subject), Node: headers["x-node"], Body: json.RawMessage(data), Headers: headers}, true
}
// keyOf is the key a module sees for an event subject: `mesh.mod.<emitter>.event.<event>` is
// `<emitter>.<event>` — the vocabulary its manifest names what it consumes in.
func keyOf(subject string) string {
before, event, found := strings.Cut(subject, ".event.")
if !found {
return subject
}
parts := strings.Split(before, ".")
if emitter := parts[len(parts)-1]; emitter != "" {
return emitter + "." + event
}
return event
}
+48
View File
@@ -0,0 +1,48 @@
package bus
import (
"errors"
"fmt"
"testing"
)
func TestSubjectsAreTheOnesTheTypeScriptRuntimeUses(t *testing.T) {
for key, want := range map[string]string{
"status": "mesh.mod.self.tool.status",
"alpha.one": "mesh.mod.alpha.tool.one",
"alpha.one@anchor": "mesh.mod.alpha.tool.one.anchor",
"seat:node-shelf.list": "mesh.seat.node-shelf.tool.list",
"seat:node-shelf.list@anchor": "mesh.seat.node-shelf.tool.list.anchor",
} {
if got, err := ToolSubject(key, "self"); err != nil || got != want {
t.Errorf("%s: %s %v, want %s", key, got, err, want)
}
}
if _, err := ToolSubject("seat:nothing", "self"); err == nil {
t.Error("a seat with no verb was accepted")
}
if SeatToolSubject("s", "v", "node", "n") != "mesh.seat.s.tool.v.n" || SeatToolSubject("s", "v", "mesh", "n") != "mesh.seat.s.tool.v" {
t.Error("seat subjects")
}
}
func TestAFingerprintIsReadHoweverItIsWritten(t *testing.T) {
for _, f := range []string{"sha256:AB:CD:ef", "abcdef", "ABCDEF", "SHA256:abcdef"} {
if got := normalizeFingerprint(f); got != "abcdef" {
t.Errorf("%s → %s", f, got)
}
}
}
func TestWhatWaitingCannotFixIsFinal(t *testing.T) {
for err, want := range map[error]string{
fmt.Errorf("x509: %w: it presented aa", ErrPin): "the bus's certificate does not match the pin",
errors.New("nats: Authorization Violation"): "the bus refused this account",
errors.New("nats: invalid url"): "the bus address is not a usable URL",
errors.New("dial tcp 10.0.0.1:4222: connect: connection refused"): "",
} {
if got := Fatal(err); got != want {
t.Errorf("%v → %q, want %q", err, got, want)
}
}
}
+73
View File
@@ -0,0 +1,73 @@
package bus_test
import (
"encoding/json"
"strings"
"sync/atomic"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
// **A call the controller refused because it is handing over is asked once more** (novox/hq issue 289):
// the controller being replaced did nothing and marked its refusal, and the one after it answers. A
// refusal not so marked is final, and a second handover refusal is said, not asked a third time.
func TestAHandoverRefusalIsAskedOnceMore(t *testing.T) {
was := bus.HandoverPause
bus.HandoverPause = 10 * time.Millisecond
t.Cleanup(func() { bus.HandoverPause = was })
url := meshtest.URL(t)
seat, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer seat.Close()
c, err := bus.Connect(bus.Credential{URL: url, Module: "console", Node: "anchor"})
if err != nil {
t.Fatal(err)
}
defer c.Close()
serve := func(t *testing.T, verb string, answers ...map[string]any) *atomic.Int32 {
t.Helper()
var asked atomic.Int32
sub, err := seat.Subscribe(bus.SeatToolSubject("mesh-controller", verb, "", ""), func(m *nats.Msg) {
n := int(asked.Add(1)) - 1
if n >= len(answers) {
n = len(answers) - 1
}
body, _ := json.Marshal(answers[n])
_ = m.Respond(body)
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = sub.Unsubscribe() })
_ = seat.Flush()
return &asked
}
handingOver := map[string]any{"error": "the controller is handing over to the next one; nothing was done, ask again",
"retry": bus.RetryHandingOver}
asked := serve(t, "rotate", handingOver, map[string]any{"result": "rotated"})
got, err := c.Ask("seat:mesh-controller.rotate", map[string]any{}, "")
if err != nil || string(got.Result) != `"rotated"` || asked.Load() != 2 {
t.Fatalf("after a handover refusal: %s %v, asked %d times", got.Result, err, asked.Load())
}
asked = serve(t, "push", map[string]any{"error": "refused for its own reason"})
if _, err := c.Ask("seat:mesh-controller.push", map[string]any{}, ""); err == nil || asked.Load() != 1 {
t.Fatalf("an unmarked refusal was asked again: %v, asked %d times", err, asked.Load())
}
asked = serve(t, "status", handingOver)
_, err = c.Ask("seat:mesh-controller.status", map[string]any{}, "")
if err == nil || asked.Load() != 2 || !strings.Contains(err.Error(), "both times refused as a handover") {
t.Fatalf("two handover refusals: %v, asked %d times", err, asked.Load())
}
}
+86
View File
@@ -0,0 +1,86 @@
package bus
import (
"encoding/json"
"fmt"
"os"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
)
// novox/hq issue 222: a subscription the bus refused is asked for again. A push sends the machine
// that needs a grant and the bus's machine together; the runtime may subscribe the moment before the
// bus reloads, and a refusal must not leave the module unreachable until some later membership.
func TestARefusedSubscriptionIsAskedForAgain(t *testing.T) {
url := os.Getenv("MESH_TEST_NATS")
if url == "" {
t.Skip("MESH_TEST_NATS unset")
}
c, err := Connect(Credential{URL: url, Module: "alpha", Node: "anchor"})
if err != nil {
t.Fatal(err)
}
defer c.Close()
var mu sync.Mutex
var said []string
c.Logf = func(f string, a ...any) { mu.Lock(); said = append(said, fmt.Sprintf(f, a...)); mu.Unlock() }
c.RetryAfter = []time.Duration{50 * time.Millisecond, 50 * time.Millisecond}
subject := "mesh.mod.alpha.tool.ping.anchor"
stop, err := c.answerOn(subject, "", func(json.RawMessage) (any, error) { return "pong", nil })
if err != nil {
t.Fatal(err)
}
first := c.answering[answeringKey(subject, "")].sub
// What the bus says when the grant is not there yet.
c.refused(fmt.Errorf("%w: Permissions Violation for Subscription to %q", nats.ErrPermissionViolation, subject))
deadline := time.Now().Add(2 * time.Second)
for {
c.mu.Lock()
again := c.answering[answeringKey(subject, "")].sub
c.mu.Unlock()
if again != first {
break
}
if time.Now().After(deadline) {
t.Fatal("the refused subscription was not asked for again")
}
time.Sleep(10 * time.Millisecond)
}
asker, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer asker.Close()
reply, err := asker.Request(subject, []byte("{}"), 2*time.Second)
if err != nil {
t.Fatalf("the subject asked for again does not answer: %v", err)
}
if !strings.Contains(string(reply.Data), "pong") {
t.Fatalf("the subject asked for again answered %s", reply.Data)
}
// Past its attempts it is given up, and said.
for i := 0; i < 3; i++ {
c.refused(fmt.Errorf("%w: Permissions Violation for Subscription to %q", nats.ErrPermissionViolation, subject))
time.Sleep(80 * time.Millisecond)
}
mu.Lock()
gaveUp := strings.Contains(strings.Join(said, "\n"), "still refuses "+subject)
mu.Unlock()
if !gaveUp {
t.Errorf("never gave up on a subject that stays refused:\n%s", strings.Join(said, "\n"))
}
// A subject the runtime stopped answering is not asked for again.
stop()
c.refused(fmt.Errorf("%w: Permissions Violation for Subscription to %q", nats.ErrPermissionViolation, subject))
if _, still := c.answering[answeringKey(subject, "")]; still {
t.Error("a stopped subject is still tracked")
}
}
+357
View File
@@ -0,0 +1,357 @@
package bus
import (
"context"
"encoding/json"
"errors"
"fmt"
"regexp"
"sort"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go/jetstream"
)
// A module's state on the bus (novox/hq ADR 0201): key-value buckets the controller creates from what
// the module declared, and issues to each assignment in its membership by the name the module uses —
// its own state by the local name, another module's as `<module>.<name>`.
//
// **The runtime keeps each module to its own buckets.** One account per machine carries every module
// on it, so the bus enforces only the union; and a write the bus refuses reaches the writer as a
// timeout, not a refusal (measured, novox/hq research 024). So what a module may reach is decided here,
// from its membership, and refused with the reason before anything is sent.
// StateIssued is one bucket an assignment may reach (ADR 0201).
type StateIssued struct {
Name string `json:"name"`
Bucket string `json:"bucket"`
Writes bool `json:"writes,omitempty"`
}
// StateEntry is one key's current value.
type StateEntry struct {
Key string `json:"key"`
Value json.RawMessage `json:"value"`
Revision uint64 `json:"revision"`
}
// StateChange is one change a watch delivers: a key put or deleted.
type StateChange struct {
State string `json:"state"`
Key string `json:"key"`
Op string `json:"op"`
Value json.RawMessage `json:"value,omitempty"`
Revision uint64 `json:"revision"`
// Current is true for a value that was there when the watch began, false for a change since.
Current bool `json:"current"`
}
// StateTimeout bounds one state operation on the bus.
var StateTimeout = 10 * time.Second
// stateKey is a key the bus can hold: letters, digits and `-/_=.`, no leading or trailing dot. A
// module's convention for naming a machine in a key (`all.<server>`, `<machine>.<server>`) fits it.
var stateKey = regexp.MustCompile(`^[-/_=a-zA-Z0-9]+(\.[-/_=a-zA-Z0-9]+)*$`)
// issuedState is the bucket a module may reach by a name, and whether it may write it — or the reason
// it may not reach it at all.
func (c *Conn) issuedState(module, name string) (StateIssued, error) {
m := c.Membership(module)
if m == nil {
return StateIssued{}, fmt.Errorf("%s has no membership issued on %s yet, so no state of it is reachable "+
"until the mesh issues one (novox/hq ADR 0201)", module, c.node)
}
var names []string
for _, s := range m.State {
if s.Name == name {
return s, nil
}
names = append(names, s.Name)
}
sort.Strings(names)
issued := "none"
if len(names) > 0 {
issued = strings.Join(names, ", ")
}
return StateIssued{}, fmt.Errorf("%s keeps and reads no state called %q: it declares what it keeps under "+
"`state` and what it reads under `reads` as <module>.<name>, and was issued: %s (novox/hq ADR 0201)",
module, name, issued)
}
func (c *Conn) bucket(ctx context.Context, s StateIssued) (jetstream.KeyValue, error) {
js, err := jetstream.New(c.nc)
if err != nil {
return nil, err
}
kv, err := js.KeyValue(ctx, s.Bucket)
if err != nil {
if errors.Is(err, jetstream.ErrBucketNotFound) {
return nil, fmt.Errorf("the state %q is issued and its bucket is not on the bus yet: the controller "+
"creates it from the catalogue on its next raise", s.Name)
}
return nil, err
}
return kv, nil
}
func checkKey(key string) error {
if !stateKey.MatchString(key) {
return fmt.Errorf("%q is not a key the bus can hold: letters, digits and -/_=, in dot-separated "+
"names", key)
}
return nil
}
// StateGet is one key's current value, or nil when it has none.
func (c *Conn) StateGet(module, name, key string) (*StateEntry, error) {
s, err := c.issuedState(module, name)
if err != nil {
return nil, err
}
if err := checkKey(key); err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
kv, err := c.bucket(ctx, s)
if err != nil {
return nil, err
}
e, err := kv.Get(ctx, key)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, nil
}
if err != nil {
return nil, err
}
return &StateEntry{Key: e.Key(), Value: valueOf(e.Value()), Revision: e.Revision()}, nil
}
// StatePut writes one key, as the module, where the module keeps the state. It answers the revision.
func (c *Conn) StatePut(module, name, key string, value json.RawMessage) (uint64, error) {
s, err := c.writable(module, name)
if err != nil {
return 0, err
}
if err := checkKey(key); err != nil {
return 0, err
}
if len(value) == 0 || !json.Valid(value) {
return 0, fmt.Errorf("a state value is JSON")
}
if field := credentialField(value); field != "" {
return 0, fmt.Errorf("%s's %s.%s carries a field %q, which names a credential: no secret is kept in state, "+
"sealed or not — a bucket is a stream, and a machine joining a year later reads it whole. Name the "+
"secret and fetch it on request/reply (novox/hq ADR 0201, design 32 §10)", module, name, key, field)
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
kv, err := c.bucket(ctx, s)
if err != nil {
return 0, err
}
return kv.Put(ctx, key, value)
}
// StateDelete removes one key, as the module, where the module keeps the state. A key that was not
// there is not an error: what is asked for is that it is gone.
func (c *Conn) StateDelete(module, name, key string) error {
s, err := c.writable(module, name)
if err != nil {
return err
}
if err := checkKey(key); err != nil {
return err
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
kv, err := c.bucket(ctx, s)
if err != nil {
return err
}
return kv.Delete(ctx, key)
}
// StateKeys is every key with a current value, sorted.
func (c *Conn) StateKeys(module, name string) ([]string, error) {
s, err := c.issuedState(module, name)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
kv, err := c.bucket(ctx, s)
if err != nil {
return nil, err
}
lister, err := kv.ListKeys(ctx)
if err != nil {
return nil, err
}
defer func() { _ = lister.Stop() }()
keys := []string{}
for k := range lister.Keys() {
keys = append(keys, k)
}
sort.Strings(keys)
return keys, nil
}
func (c *Conn) writable(module, name string) (StateIssued, error) {
s, err := c.issuedState(module, name)
if err != nil {
return s, err
}
if !s.Writes {
owner := name
if dot := strings.LastIndex(name, "."); dot > 0 {
owner = name[:dot]
}
return s, fmt.Errorf("%s reads %s and does not keep it: only %s's own instances write it (novox/hq ADR 0201)",
module, name, owner)
}
return s, nil
}
// StateWatch hands deliver the current value of every key matching the pattern — none that is
// deleted — and then every change, in order (ADR 0201). It returns once the current values are
// delivered; deliver is called from one goroutine, one change at a time, and an error from it is said
// and the watch goes on: state is not a queue, and the next change, or the next start, reads it again.
// The pattern is a key, with `*` for one name and `**` for the rest; empty is every key.
func (c *Conn) StateWatch(module, name, pattern string, deliver func(StateChange) error) (stop func(), err error) {
s, err := c.issuedState(module, name)
if err != nil {
return nil, err
}
filter := ">"
if pattern != "" {
parts := strings.Split(pattern, ".")
for i, p := range parts {
if p == "**" {
if i != len(parts)-1 {
return nil, fmt.Errorf("%q: `**` stands for the rest of a key, so it comes last", pattern)
}
parts[i] = ">"
continue
}
if p != "*" && !stateKey.MatchString(p) {
return nil, fmt.Errorf("%q is not a key pattern: names, `*` for one and `**` for the rest", pattern)
}
}
filter = strings.Join(parts, ".")
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
kv, err := c.bucket(ctx, s)
cancel()
if err != nil {
return nil, err
}
watching, stopWatching := context.WithCancel(context.Background())
w, err := kv.Watch(watching, filter)
if err != nil {
stopWatching()
return nil, err
}
var once sync.Once
stop = func() {
once.Do(func() {
_ = w.Stop()
stopWatching()
})
}
current := make(chan struct{})
go func() {
initial := true
for e := range w.Updates() {
if e == nil {
// The end of what was there when the watch began.
if initial {
initial = false
close(current)
}
continue
}
change := StateChange{State: name, Key: e.Key(), Revision: e.Revision(), Current: initial}
switch e.Operation() {
case jetstream.KeyValuePut:
change.Op = "put"
change.Value = valueOf(e.Value())
default:
// A deletion among the current values is a key that is not there: not handed over
// (measured, research 024 — the server sends its marker among the initial values).
if initial {
continue
}
change.Op = "delete"
}
if err := deliver(change); err != nil {
c.Logf("[mesh-tools] %s did not take %s %s of %s: %v; its next change, or its next start, reads it again",
module, change.Op, change.Key, name, err)
}
}
if initial {
close(current)
}
}()
select {
case <-current:
return stop, nil
case <-time.After(StateTimeout):
stop()
return nil, fmt.Errorf("the current values of %s did not arrive in %s", name, StateTimeout)
}
}
func valueOf(raw []byte) json.RawMessage {
if len(raw) == 0 || !json.Valid(raw) {
b, _ := json.Marshal(string(raw))
return b
}
return json.RawMessage(raw)
}
// credentialEndings are the ends of a field name that say its value is a credential. A guard against
// the ordinary mistake, not a determined one: a sealed value is plain text to anything inspecting it,
// so the rule is checked where it can be and said to be partial (ADR 0201).
var credentialEndings = []string{"password", "passwd", "secret", "token", "credential", "credentials",
"authorization", "apikey", "privatekey", "accesskey", "cookie"}
// credentialField is the first field anywhere in a JSON value whose name says it is a credential, or "".
func credentialField(value json.RawMessage) string {
var v any
if json.Unmarshal(value, &v) != nil {
return ""
}
var walk func(any) string
walk = func(v any) string {
switch t := v.(type) {
case map[string]any:
keys := make([]string, 0, len(t))
for k := range t {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
norm := strings.NewReplacer("-", "", "_", "", " ", "").Replace(strings.ToLower(k))
for _, end := range credentialEndings {
if strings.HasSuffix(norm, end) {
return k
}
}
if found := walk(t[k]); found != "" {
return found
}
}
case []any:
for _, x := range t {
if found := walk(x); found != "" {
return found
}
}
}
return ""
}
return walk(v)
}
+23
View File
@@ -0,0 +1,23 @@
package bus
import (
"encoding/json"
"testing"
)
// A value naming a credential anywhere in it is found (novox/hq ADR 0201) — the guard against the
// ordinary mistake — and a value that only mentions tokens as a count is not.
func TestACredentialNamedFieldIsFoundAnywhereInAValue(t *testing.T) {
for value, want := range map[string]string{
`{"url":"http://x","headers":{"Authorization":"Bearer s"}}`: "Authorization",
`[{"name":"a","env":{"API_KEY":"s"}}]`: "API_KEY",
`{"refresh_token":"s"}`: "refresh_token",
`{"clientSecret":"s"}`: "clientSecret",
`{"maxTokens":4096,"name":"a","generation":3}`: "",
`"just a string"`: "",
} {
if got := credentialField(json.RawMessage(value)); got != want {
t.Errorf("%s: found %q, want %q", value, got, want)
}
}
}
+953
View File
@@ -0,0 +1,953 @@
package console
// The mesh's tools found by address, not announced whole (novox/hq ADR 0195, to-be 34 §3a).
//
// The console announces six tools — five to find and call, and mesh_runtimes to say which runtimes
// discovery heard. Everything the mesh answers is reached through them by one address per layer:
//
// <seat>.<verb> a seat held once for the mesh — its holder answers
// <node>/<seat>.<verb> a seat held once per machine — that machine's holder answers
// <node>/<module>.<tool> a module assigned to a machine — that assignment answers
// <module>.<tool> also, for a module whose instances are interchangeable (ADR 0160)
//
// A module that is not interchangeable is called with its machine or refused, naming the machines
// it runs on: "whichever answers" is no answer for state a machine holds.
//
// Every discovery verb asks the mesh when it is called — kept a few seconds at most, never for a
// session — so a tool that arrived a minute ago is found without the client reconnecting.
import (
"encoding/json"
"fmt"
"sort"
"strings"
"sync"
"time"
"github.com/novox/mesh-tools/node-tools/internal/announce"
"github.com/novox/mesh-tools/node-tools/internal/bus"
)
// IndexKept is how long what the mesh answered is kept before it is asked again: long enough that
// one agent turn's search, describe and call ask once, short enough that nothing goes stale.
var IndexKept = 5 * time.Second
// The six tools the console announces. Names of the API's kind — letters, digits, `_`, `-`.
const (
verbOverview = "mesh_overview"
verbMachine = "mesh_machine"
verbSearch = "mesh_search"
verbDescribe = "mesh_describe"
verbCall = "mesh_call"
verbRuntimes = "mesh_runtimes"
)
// searchCap is how many matches a search answers before it says how many more there were.
const searchCap = 25
const grammar = "Addresses: `<seat>.<verb>` for a seat held once for the mesh (e.g. `mesh-controller.nodes`); " +
"`<node>/<seat>.<verb>` for a seat every machine holds (e.g. `ace/node-packet-filter.rules`); " +
"`<node>/<module>.<tool>` for a module on one machine (e.g. `novox/postgres.postgres_list_databases`); " +
"and `<module>.<tool>` also for a module whose instances are interchangeable."
// discovery is the six tools as tools/list announces them.
func discovery() []map[string]any {
str := func(desc string) map[string]any { return map[string]any{"type": "string", "description": desc} }
obj := func(props map[string]any, required ...string) map[string]any {
s := map[string]any{"type": "object", "properties": props}
if len(required) > 0 {
s["required"] = required
}
return s
}
return []map[string]any{
{"name": verbOverview, "inputSchema": obj(map[string]any{}),
"description": "The mesh at a glance: the seats it holds once for the whole mesh with their verbs, the seats " +
"every machine holds, and its machines. Start here, then `mesh_machine` for one machine. " + grammar},
{"name": verbMachine, "inputSchema": obj(map[string]any{"node": str("the machine, as mesh_overview names it")}, "node"),
"description": "One machine: the seats it holds with their verbs, and the modules assigned to it with their tools — " +
"each with the address to describe or call it by. " + grammar},
{"name": verbSearch, "inputSchema": obj(map[string]any{"query": str("words to find in tool names and descriptions, e.g. `postgres databases`")}, "query"),
"description": "Find tools anywhere in the mesh by words: every match's address and a line of what it does, across the " +
"mesh's seats, the machines' seats and every module on every machine. " + grammar},
{"name": verbDescribe, "inputSchema": obj(map[string]any{"address": str("the tool's address")}, "address"),
"description": "What one tool does and the arguments it takes, as a JSON schema. The machine is in the address, " +
"never an argument. " + grammar},
{"name": verbCall, "inputSchema": obj(map[string]any{
"address": str("the tool's address"),
"arguments": map[string]any{"type": "object", "description": "the tool's arguments, as mesh_describe gives its schema"},
}, "address"),
"description": "Call one tool by its address with its arguments; the answer says which machine gave it. " + grammar},
{"name": verbRuntimes, "inputSchema": obj(map[string]any{}),
"description": "Which runtimes answered discovery, asked now: for each, its machine, how long its answer took to " +
"arrive, its size in bytes, how many modules and tools it announced, whether it was shortened to fit the " +
"bus, and when it was last heard — and every runtime or machine expected that was not heard. Read-only; " +
"for when an address is said to be missing or on another machine."},
}
}
func isDiscovery(name string) bool {
switch name {
case verbOverview, verbMachine, verbSearch, verbDescribe, verbCall, verbRuntimes:
return true
}
return false
}
// seatInfo is a seat as the mesh's records define it, and who holds it where.
type seatInfo struct {
Seat string
Scope string // "mesh" or "node"
Verbs []Tool
Holders []holder
}
type holder struct {
Module string `json:"module"`
Node string `json:"node"`
}
// moduleInfo is a module that answers tools: where it runs, whether any instance will do, and its
// tools as one of its instances described them.
type moduleInfo struct {
Module string
On []string
Interchangeable bool
Tools []Tool
}
// index is what the mesh answered about itself, at one moment.
type index struct {
Seats []seatInfo
Machines []string
Modules map[string]*moduleInfo
NotAnswering []string
Listing *Listing // the flat catalogue the call path resolves subjects with
Discovery announce.Discovery
// Recorded is where the controller's records place each module that declares tools.
Recorded map[string][]string
// Unheard is every runtime whose answer this index lacks: it said it was there and did not say
// what it serves in time, or it answered before and not now. An address it might answer is never
// called missing while it is here (2026-10-05).
Unheard []unheard
}
// unheard is one runtime discovery did not hear in full, and why.
type unheard struct {
Runtime string `json:"runtime"`
Machine string `json:"machine,omitempty"`
Why string `json:"why"`
}
// unheardOn is what is unheard on one machine; with no machine, everything unheard.
func (x *index) unheardOn(node string) []unheard {
var out []unheard
for _, u := range x.Unheard {
if node == "" || u.Machine == node || u.Machine == "" {
out = append(out, u)
}
}
return out
}
// sayUnheard is the unheard, as a sentence's end.
func sayUnheard(us []unheard) string {
var parts []string
for _, u := range us {
who := u.Runtime
if u.Machine != "" {
who += " on " + u.Machine
}
parts = append(parts, who+" ("+u.Why+")")
}
return strings.Join(parts, "; ")
}
func (x *index) seat(name string) *seatInfo {
for i := range x.Seats {
if x.Seats[i].Seat == name {
return &x.Seats[i]
}
}
return nil
}
func findTool(tools []Tool, name string) *Tool {
for i := range tools {
if tools[i].Name == name {
return &tools[i]
}
}
return nil
}
// controllerOutput is a controller seat verb's answer: the command's printed output.
func controllerOutput(conn *bus.Conn, verb string) (string, error) {
got, err := conn.Ask("seat:mesh-controller."+verb, map[string]any{}, "")
if err != nil {
return "", err
}
var r struct {
Output string `json:"output"`
OK *bool `json:"ok"`
}
if json.Unmarshal(got.Result, &r) != nil {
return "", fmt.Errorf("mesh-controller.%s answered something that is not its output", verb)
}
if r.OK != nil && !*r.OK {
return "", fmt.Errorf("mesh-controller.%s: %s", verb, strings.TrimSpace(r.Output))
}
return r.Output, nil
}
// jsonIn is the JSON document a command printed, after any lines it said first: a seat verb runs
// the controller's command, and a command may warn before it answers.
func jsonIn(output string) string {
t := strings.TrimSpace(output)
if strings.HasPrefix(t, "{") || strings.HasPrefix(t, "[") {
return t
}
for _, open := range []string{"\n{", "\n["} {
if i := strings.Index(output, open); i >= 0 {
return strings.TrimSpace(output[i+1:])
}
}
return ""
}
// recordedModule is a module as the controller's records hold it (`module list --json`): where the
// mesh assigned it, and whether it declares tools — what should announce itself, and where.
type recordedModule struct {
Module string `json:"module"`
On []string `json:"on"`
Tools bool `json:"tools"`
}
// recordedMachine is a machine as the controller's records hold it (`node list --json`).
type recordedMachine struct {
Name string `json:"name"`
}
// indexOn asks the mesh what it holds (novox/hq ADR 0197): what answers, from every runtime's own
// announcement on the bus — one `$SRV.INFO` request — and what should, from the controller's records
// read as JSON. Nothing is inferred from a roster and nothing is parsed from print.
func indexOn(conn *bus.Conn) (*index, error) {
var wg sync.WaitGroup
var nodesOut, modulesOut string
wg.Add(2)
go func() { defer wg.Done(); nodesOut, _ = controllerOutput(conn, "nodes") }()
go func() { defer wg.Done(); modulesOut, _ = controllerOutput(conn, "modules") }()
d, err := announce.Gather(conn)
wg.Wait()
if err != nil {
return nil, err
}
var endpoints []announce.Endpoint
for _, h := range d.Heard {
endpoints = append(endpoints, announce.Endpoints(h.Info)...)
}
x, announced, machines := indexOfEndpoints(endpoints)
l := x.Listing
x.Discovery = d
for _, s := range d.Silent {
x.Unheard = append(x.Unheard, unheard{Runtime: s.Name, Machine: s.Machine,
Why: fmt.Sprintf("it answered PING; what it serves did not arrive within %s", announce.Patience)})
}
// What should have answered: every assignment of a module that declares tools. Silence is named;
// a module with no tools is never a name here.
var recorded []recordedModule
if json.Unmarshal([]byte(jsonIn(modulesOut)), &recorded) == nil {
for _, m := range recorded {
if !m.Tools {
continue
}
x.Recorded[m.Module] = append([]string{}, m.On...)
for _, n := range m.On {
// A holder of a seat held once for the mesh announces no machine — the seat is the mesh's,
// not a machine's — so what it announced without one answers for wherever it is assigned.
if !announced[m.Module][n] && !announced[m.Module][""] {
l.NotAnswering = append(l.NotAnswering, m.Module+" on "+n)
}
}
}
} else {
l.NotAnswering = append(l.NotAnswering, "mesh-controller (its records of the modules did not answer, so what is missing cannot be said)")
}
for _, u := range x.Unheard {
l.NotAnswering = append(l.NotAnswering, "the runtime "+sayUnheard([]unheard{u}))
}
sort.Strings(l.NotAnswering)
x.NotAnswering = l.NotAnswering
var known []recordedMachine
if json.Unmarshal([]byte(jsonIn(nodesOut)), &known) == nil {
for _, n := range known {
if n.Name != "" {
machines[n.Name] = true
}
}
}
for n := range machines {
x.Machines = append(x.Machines, n)
}
sort.Strings(x.Machines)
return x, nil
}
func containsHolder(hs []holder, h holder) bool {
for _, x := range hs {
if x == h {
return true
}
}
return false
}
func (s *Surface) index() (*index, error) { return s.indexAsked(false) }
// indexAsked is what the mesh answered, asked again when `fresh` or when the kept answer is old.
func (s *Surface) indexAsked(fresh bool) (*index, error) {
s.mu.Lock()
if !fresh && s.idx != nil && time.Since(s.idxAt) <= IndexKept {
x := s.idx
s.mu.Unlock()
return x, nil
}
s.mu.Unlock()
x, err := indexOn(s.conn)
if err != nil {
return nil, err
}
s.mu.Lock()
s.remember(x)
s.idx, s.idxAt = x, time.Now()
s.mu.Unlock()
return x, nil
}
// indexOfEndpoints is what the announced endpoints say: every seat with its verbs and holders, every
// module with its tools and machines, the flat listing, and which module announced on which machine.
func indexOfEndpoints(endpoints []announce.Endpoint) (*index, map[string]map[string]bool, map[string]bool) {
l := &Listing{Tools: []Tool{}, NotAnswering: []string{}}
x := &index{Modules: map[string]*moduleInfo{}, Listing: l, Recorded: map[string][]string{}}
announced := map[string]map[string]bool{} // module → node → announced something
seats := map[string]*seatInfo{}
// <module>.<tool>, or seat:<seat>.<verb>, → index in l.Tools. **Apart**, because a seat and a module may
// share a name — mesh-delivery is the delivery's seat and the module holding it (novox/hq ADR 0239), and
// the module answers the seat's verbs with tools of the same names: in one namespace the module's tool
// took the key first, and the seat was listed with no verb at all (novox/hq issue 287).
toolAt := map[string]int{}
machines := map[string]bool{}
for _, e := range endpoints {
if e.Node != "" {
machines[e.Node] = true
}
if announced[e.Module] == nil {
announced[e.Module] = map[string]bool{}
}
announced[e.Module][e.Node] = true
switch e.Kind {
case announce.KindSeat:
st := seats[e.Seat]
if st == nil {
st = &seatInfo{Seat: e.Seat, Scope: e.Scope}
seats[e.Seat] = st
}
if st.Scope != "node" && e.Scope == "node" {
st.Scope = "node"
}
h := holder{Module: e.Module, Node: e.Node}
if !containsHolder(st.Holders, h) {
st.Holders = append(st.Holders, h)
}
key := "seat:" + e.Seat + "." + e.Tool
if _, have := toolAt[key]; !have {
toolAt[key] = len(l.Tools)
t := Tool{Module: e.Seat, Name: e.Tool, Description: e.Description, Input: e.Schema, Seat: true, Scope: st.Scope}
l.Tools = append(l.Tools, t)
st.Verbs = append(st.Verbs, t)
}
case announce.KindTool:
m := x.Modules[e.Module]
if m == nil {
m = &moduleInfo{Module: e.Module}
x.Modules[e.Module] = m
}
if e.Node != "" && !contains(m.On, e.Node) {
m.On = append(m.On, e.Node)
}
m.Interchangeable = m.Interchangeable || e.Interchangeable
key := e.Module + "." + e.Tool
i, have := toolAt[key]
if !have {
i = len(l.Tools)
toolAt[key] = i
l.Tools = append(l.Tools, Tool{Module: e.Module, Name: e.Tool, Description: e.Description, Input: e.Schema})
}
if !contains(l.Tools[i].Subjects, e.Subject) {
l.Tools[i].Subjects = append(l.Tools[i].Subjects, e.Subject)
}
}
}
// A tool's subjects as a call looks them up: the plain one any instance answers first, then each
// machine's.
for i := range l.Tools {
t := &l.Tools[i]
if t.Seat {
continue
}
plain := "mesh.mod." + t.Module + ".tool." + t.Name
sort.SliceStable(t.Subjects, func(a, b int) bool {
if (t.Subjects[a] == plain) != (t.Subjects[b] == plain) {
return t.Subjects[a] == plain
}
return t.Subjects[a] < t.Subjects[b]
})
}
for _, t := range l.Tools {
if !t.Seat {
x.Modules[t.Module].Tools = append(x.Modules[t.Module].Tools, t)
}
}
for _, m := range x.Modules {
sort.Strings(m.On)
}
for _, st := range seats {
sort.Slice(st.Holders, func(a, b int) bool { return st.Holders[a].Node < st.Holders[b].Node })
x.Seats = append(x.Seats, *st)
}
sort.Slice(x.Seats, func(i, j int) bool { return x.Seats[i].Seat < x.Seats[j].Seat })
sort.SliceStable(l.Tools, func(i, j int) bool {
return l.Tools[i].Module+"."+l.Tools[i].Name < l.Tools[j].Module+"."+l.Tools[j].Name
})
return x, announced, machines
}
// target is what an address resolves to.
type target struct {
Address string
Key string // the call key the existing path takes: seat:<s>.<v>[@node] or <m>.<t>[@node]
Name string // <seat>.<verb> or <module>.<tool>, for the listing's subject lookup
Node string
Tool Tool
Seat bool
}
// resolve turns an address into exactly one target, or says why it cannot.
func resolve(x *index, address string) (target, error) {
address = strings.TrimSpace(address)
node, rest, hasNode := strings.Cut(address, "/")
if !hasNode {
rest, node = address, ""
}
dot := strings.Index(rest, ".")
if dot <= 0 || dot == len(rest)-1 || strings.Contains(rest, "/") {
return target{}, fmt.Errorf("%q is not an address. %s", address, grammar)
}
prefix, name := rest[:dot], rest[dot+1:]
s := x.seat(prefix)
if s != nil && x.Modules[prefix] != nil && findTool(x.Modules[prefix].Tools, name) != nil {
// **A seat and a module of one name** (novox/hq issue 287): mesh-delivery is the delivery's seat and
// the module holding it. The module's tool is meant when the seat has no such verb, or when a machine
// is named for a seat held once for the mesh — `<node>/<module>.<tool>` is a module on one machine.
if findTool(s.Verbs, name) == nil || (node != "" && s.Scope != "node") {
s = nil
}
}
if s != nil {
verb := findTool(s.Verbs, name)
if verb == nil {
if len(s.Verbs) == 0 {
return target{}, fmt.Errorf("the seat %s has no verb %s: its holder announced none", prefix, name)
}
return target{}, fmt.Errorf("the seat %s has no verb %s; it has %s", prefix, name, toolNames(s.Verbs))
}
if s.Scope == "node" {
if node == "" {
return target{}, fmt.Errorf("%s is held once per machine: write <node>/%s — it is held on %s",
prefix, rest, orNobody(nodesOf(s.Holders)))
}
return target{Address: node + "/" + rest, Key: "seat:" + rest + "@" + node, Name: rest, Node: node, Tool: *verb, Seat: true}, nil
}
if node != "" {
return target{}, fmt.Errorf("%s is held once for the whole mesh: write %s, without a machine", prefix, rest)
}
return target{Address: rest, Key: "seat:" + rest, Name: rest, Tool: *verb, Seat: true}, nil
}
m := x.Modules[prefix]
if m == nil {
if on := x.Recorded[prefix]; len(on) > 0 {
why := "it did not announce itself on the bus"
if us := x.unheardOnAny(on); len(us) > 0 {
why = "discovery did not hear in full from " + sayUnheard(us)
}
return target{}, fmt.Errorf("%s is assigned to %s in the mesh's records, but nothing that answered discovery serves it: %s. "+
"Ask again in a moment; mesh_runtimes shows which runtimes answered", prefix, strings.Join(on, ", "), why)
}
if len(x.Unheard) > 0 {
return target{}, fmt.Errorf("nothing that answered discovery is called %s, but not every runtime answered: %s — "+
"%s may be theirs. Ask again in a moment; mesh_runtimes shows which runtimes answered", prefix, sayUnheard(x.Unheard), prefix)
}
return target{}, fmt.Errorf("nothing in the mesh is called %s: no seat, and no module that answers tools. "+
"mesh_search finds a tool by words", prefix)
}
tool := findTool(m.Tools, name)
if tool == nil {
return target{}, fmt.Errorf("%s has no tool %s; it has %s", prefix, name, toolNames(m.Tools))
}
if node == "" {
if !m.Interchangeable {
return target{}, fmt.Errorf("%s keeps state on each machine it runs on, so a call names the machine: "+
"write <node>/%s — it runs on %s", prefix, rest, orNobody(m.On))
}
return target{Address: rest, Key: rest, Name: rest, Tool: *tool}, nil
}
if len(m.On) > 0 && !contains(m.On, node) {
if us := x.unheardOn(node); len(us) > 0 || contains(x.Recorded[prefix], node) {
why := "it did not announce itself there"
if len(us) > 0 {
why = "discovery did not hear in full from " + sayUnheard(us)
}
return target{}, fmt.Errorf("%s on %s did not answer discovery: %s. It was heard on %s. "+
"Ask again in a moment; mesh_runtimes shows which runtimes answered", prefix, node, why, orNobody(m.On))
}
return target{}, fmt.Errorf("%s does not run on %s; it runs on %s", prefix, node, orNobody(m.On))
}
return target{Address: node + "/" + rest, Key: rest + "@" + node, Name: rest, Node: node, Tool: *tool}, nil
}
// unheardOnAny is what is unheard on any of these machines.
func (x *index) unheardOnAny(nodes []string) []unheard {
var out []unheard
for _, u := range x.Unheard {
if u.Machine == "" || contains(nodes, u.Machine) {
out = append(out, u)
}
}
return out
}
func contains(xs []string, s string) bool {
for _, x := range xs {
if x == s {
return true
}
}
return false
}
func toolNames(ts []Tool) string {
names := make([]string, 0, len(ts))
for _, t := range ts {
names = append(names, t.Name)
}
sort.Strings(names)
return strings.Join(names, ", ")
}
func nodesOf(hs []holder) []string {
var out []string
for _, h := range hs {
if h.Node != "" && !contains(out, h.Node) {
out = append(out, h.Node)
}
}
sort.Strings(out)
return out
}
func orNobody(nodes []string) string {
if len(nodes) == 0 {
return "no machine the mesh knows of"
}
return strings.Join(nodes, ", ")
}
// firstLine is a description's first line, for a list.
func firstLine(s string) string {
s = strings.TrimSpace(s)
if i := strings.IndexAny(s, "\n"); i >= 0 {
s = s[:i]
}
if len(s) > 160 {
s = s[:157] + "…"
}
return s
}
// schemaAsPassed is a tool's schema as an agent passes it at this address. Where the address names
// the machine, `node` is taken out: the machine is in the address. Where it does not — a seat held
// once for the mesh, or an interchangeable module — `node` is the tool's own argument, and is kept.
//
// **Taking it out everywhere made the mesh's own verbs lie** (novox/hq issue 244): the controller's
// `push`, `plan`, `assign`, `pin` and `settings` take the machine they act on as `node`, and were
// described without it — `push` as taking nothing at all, so a push naming one machine arrived as a
// push of every machine behind.
func schemaAsPassed(t target) map[string]any {
if t.Node == "" {
return asSchema(t.Tool.Input)
}
return schemaWithoutNode(t.Tool.Input)
}
// argumentsAsSent are a call's arguments as the tool at this address receives them, or why the call
// is refused. **Nothing given is dropped without a word** (novox/hq issue 244):
// - where the address names the machine, a `node` naming the same machine is redundant and taken
// out; one naming another machine is refused — which of the two was meant is not guessed;
// - where it does not, `node` is the tool's own argument and goes to it like any other;
// - a seat's schema is the mesh's record of the verb, so an argument a seat's verb does not
// declare is refused here, naming it, rather than sent to be passed over.
func argumentsAsSent(t target, given map[string]any) (map[string]any, error) {
args := map[string]any{}
for k, v := range given {
args[k] = v
}
if n, has := args["node"]; has && t.Node != "" {
if named, _ := n.(string); strings.TrimSpace(named) != t.Node {
return nil, fmt.Errorf("%s names the machine %s in its address and %v in its arguments: say it once, in the address",
t.Address, t.Node, n)
}
delete(args, "node")
}
if !t.Seat {
return args, nil
}
declared := map[string]bool{}
var names []string
if p, ok := schemaAsPassed(t)["properties"].(map[string]any); ok {
for k := range p {
declared[k] = true
names = append(names, k)
}
}
sort.Strings(names)
var strangers []string
for k := range args {
if !declared[k] {
strangers = append(strangers, k)
}
}
if len(strangers) > 0 {
sort.Strings(strangers)
takes := "nothing"
if len(names) > 0 {
takes = strings.Join(names, ", ")
}
return nil, fmt.Errorf("%s takes no argument %s — it takes %s (mesh_describe %s); nothing was sent",
t.Address, strings.Join(strangers, ", "), takes, t.Address)
}
return args, nil
}
// schemaWithoutNode is a tool's schema with `node` taken out, for an address that names the machine.
func schemaWithoutNode(raw json.RawMessage) map[string]any {
schema := asSchema(raw)
out := map[string]any{}
for k, v := range schema {
out[k] = v
}
if p, ok := schema["properties"].(map[string]any); ok {
props := map[string]any{}
for k, v := range p {
if k != "node" {
props[k] = v
}
}
out["properties"] = props
}
if r, ok := schema["required"].([]any); ok {
var keep []any
for _, k := range r {
if k != "node" {
keep = append(keep, k)
}
}
if len(keep) == 0 {
delete(out, "required")
} else {
out["required"] = keep
}
}
return out
}
// answerText is a discovery verb's answer as MCP content: JSON, indented.
func answerText(v any) map[string]any {
var b strings.Builder
enc := json.NewEncoder(&b)
enc.SetEscapeHTML(false) // `<node>/…` is read by an agent, not embedded in a page
enc.SetIndent("", " ")
_ = enc.Encode(v)
return map[string]any{"content": []map[string]any{{"type": "text", "text": strings.TrimRight(b.String(), "\n")}}}
}
func failure(text string) map[string]any {
return map[string]any{"content": []map[string]any{{"type": "text", "text": text}}, "isError": true}
}
// discover answers one of the discovery verbs.
func (s *Surface) discover(name string, args map[string]any) map[string]any {
if name == verbRuntimes {
return s.runtimesAnswer()
}
x, err := s.index()
if err != nil {
return failure("the mesh's discovery failed: " + err.Error())
}
str := func(k string) string { v, _ := args[k].(string); return strings.TrimSpace(v) }
switch name {
case verbOverview:
type verbLine struct {
Address string `json:"address"`
Description string `json:"description"`
}
var mesh, node []map[string]any
for _, st := range x.Seats {
var verbs []verbLine
for _, v := range st.Verbs {
addr := st.Seat + "." + v.Name
if st.Scope == "node" {
addr = "<node>/" + addr
}
verbs = append(verbs, verbLine{addr, firstLine(v.Description)})
}
entry := map[string]any{"seat": st.Seat, "verbs": verbs}
if st.Scope == "node" {
entry["held on"] = nodesOf(st.Holders)
node = append(node, entry)
} else {
if h := nodesOf(st.Holders); len(h) > 0 {
entry["held on"] = h
}
mesh = append(mesh, entry)
}
}
modules := 0
tools := 0
for _, m := range x.Modules {
modules++
tools += len(m.Tools)
}
return answerText(map[string]any{
"seats of the mesh": mesh,
"seats every machine": node,
"machines": x.Machines,
"modules with tools": fmt.Sprintf("%d modules, %d tools — mesh_machine lists a machine's, mesh_search finds one", modules, tools),
"not answering": x.NotAnswering,
})
case verbMachine:
node := str("node")
if node == "" {
return failure("mesh_machine needs `node`: one of " + orNobody(x.Machines))
}
if !contains(x.Machines, node) {
return failure(fmt.Sprintf("the mesh knows no machine %q; it has %s", node, orNobody(x.Machines)))
}
var seats []map[string]any
for _, st := range x.Seats {
if st.Scope != "node" || !contains(nodesOf(st.Holders), node) {
continue
}
var verbs []string
for _, v := range st.Verbs {
verbs = append(verbs, node+"/"+st.Seat+"."+v.Name)
}
var by string
for _, h := range st.Holders {
if h.Node == node {
by = h.Module
}
}
seats = append(seats, map[string]any{"seat": st.Seat, "held by": by, "verbs": verbs})
}
var modules []map[string]any
names := make([]string, 0, len(x.Modules))
for n := range x.Modules {
names = append(names, n)
}
sort.Strings(names)
for _, n := range names {
m := x.Modules[n]
if !contains(m.On, node) {
continue
}
var tools []map[string]string
for _, t := range m.Tools {
tools = append(tools, map[string]string{"address": node + "/" + m.Module + "." + t.Name, "does": firstLine(t.Description)})
}
entry := map[string]any{"module": m.Module, "tools": tools}
if m.Interchangeable {
entry["interchangeable"] = "any instance answers <module>.<tool> as well"
}
modules = append(modules, entry)
}
out := map[string]any{"machine": node, "seats": seats, "modules": modules}
if seats == nil {
out["seats"] = []map[string]any{}
}
if modules == nil {
out["modules"] = []map[string]any{}
}
if us := x.unheardOn(node); len(us) > 0 {
out["incomplete"] = "discovery did not hear in full from " + sayUnheard(us) +
"; what is listed may be missing what it serves. Ask again in a moment; mesh_runtimes shows which runtimes answered"
}
return answerText(out)
case verbSearch:
query := strings.ToLower(str("query"))
if query == "" {
return failure("mesh_search needs `query`: words to find, e.g. `postgres databases`")
}
words := strings.Fields(query)
type hit struct {
Address string `json:"address"`
Does string `json:"does"`
Also string `json:"also,omitempty"`
}
var hits []hit
matches := func(parts ...string) bool {
hay := strings.ToLower(strings.Join(parts, " "))
for _, w := range words {
if !strings.Contains(hay, w) {
return false
}
}
return true
}
for _, st := range x.Seats {
for _, v := range st.Verbs {
if !matches(st.Seat, v.Name, v.Description) {
continue
}
if st.Scope == "node" {
on := nodesOf(st.Holders)
first := "<node>"
also := ""
if len(on) > 0 {
first = on[0]
if len(on) > 1 {
also = "also on " + strings.Join(on[1:], ", ")
}
}
hits = append(hits, hit{first + "/" + st.Seat + "." + v.Name, firstLine(v.Description), also})
} else {
hits = append(hits, hit{st.Seat + "." + v.Name, firstLine(v.Description), ""})
}
}
}
names := make([]string, 0, len(x.Modules))
for n := range x.Modules {
names = append(names, n)
}
sort.Strings(names)
for _, n := range names {
m := x.Modules[n]
for _, t := range m.Tools {
if !matches(m.Module, t.Name, t.Description) {
continue
}
switch {
case m.Interchangeable:
hits = append(hits, hit{m.Module + "." + t.Name, firstLine(t.Description), "any instance; on " + orNobody(m.On)})
case len(m.On) == 0:
hits = append(hits, hit{"<node>/" + m.Module + "." + t.Name, firstLine(t.Description), "the mesh places it on no machine"})
default:
also := ""
if len(m.On) > 1 {
also = "also on " + strings.Join(m.On[1:], ", ")
}
hits = append(hits, hit{m.On[0] + "/" + m.Module + "." + t.Name, firstLine(t.Description), also})
}
}
}
out := map[string]any{"matches": hits}
if len(hits) > searchCap {
out["matches"] = hits[:searchCap]
out["more"] = fmt.Sprintf("%d more; narrow the words", len(hits)-searchCap)
}
if len(hits) == 0 {
out["matches"] = []hit{}
out["hint"] = "nothing matched every word; try fewer words, or mesh_overview and mesh_machine to browse"
}
if len(x.Unheard) > 0 {
out["incomplete"] = "discovery did not hear in full from " + sayUnheard(x.Unheard) +
"; what they serve is not searched. mesh_runtimes shows which runtimes answered"
}
return answerText(out)
case verbDescribe:
t, err := resolve(x, str("address"))
if err != nil {
return failure(err.Error())
}
if !t.Seat && len(t.Tool.Input) == 0 {
// Discovery carries a seat's schema, not a module tool's: asked of the runtime serving it.
t.Tool.Input = inputOf(s.conn, t.Tool.Module, t.Tool.Name)
}
description := t.Tool.Description
if description == "" {
description = t.Name
}
return answerText(map[string]any{"address": t.Address, "description": description,
"arguments": schemaAsPassed(t)})
case verbCall:
t, err := resolve(x, str("address"))
if err != nil {
return failure(err.Error())
}
given, _ := args["arguments"].(map[string]any)
callArgs, err := argumentsAsSent(t, given)
if err != nil {
return failure(err.Error())
}
var got bus.Answered
if t.Seat {
got, err = s.conn.Ask(t.Key, callArgs, "")
} else {
got, err = callTool(s.conn, t.Key, callArgs, nil, x.Listing)
}
if err != nil {
return failure(whyItFailed(t.Address, err))
}
content := []map[string]any{{"type": "text", "text": pretty(got.Result)}}
if got.Node != "" {
content = append(content, map[string]any{"type": "text", "text": "answered by " + got.Node})
}
return map[string]any{"content": content}
}
return failure("no discovery verb " + name)
}
// inputOf is a module tool's input schema, asked of a runtime that serves the module — its `tools` verb,
// answered on the module's plain subject by any instance — because discovery no longer carries it: every
// tool's schema made a machine's one discovery answer outgrow the bus (2026-10-04). Nil when nothing
// answers, which describes the tool as taking nothing rather than failing the description.
func inputOf(conn *bus.Conn, module, tool string) json.RawMessage {
got, err := conn.Ask(module+".tools", map[string]any{}, "")
if err != nil {
return nil
}
var answer struct {
Tools []struct {
Name string `json:"name"`
Input json.RawMessage `json:"input"`
} `json:"tools"`
}
if json.Unmarshal(got.Result, &answer) != nil {
return nil
}
for _, t := range answer.Tools {
if t.Name == tool {
return t.Input
}
}
return nil
}
+243
View File
@@ -0,0 +1,243 @@
package console
import (
"encoding/json"
"fmt"
"strings"
"sync/atomic"
"testing"
"time"
"github.com/novox/mesh-tools/node-tools/internal/announce"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
"github.com/novox/mesh-tools/node-tools/internal/runtime"
)
// text is a tool result's first text, and whether it was an error.
func text(t *testing.T, reply map[string]any) (string, bool) {
t.Helper()
result, ok := reply["result"].(map[string]any)
if !ok {
t.Fatalf("no result: %v", reply)
}
content := result["content"].([]any)
isErr, _ := result["isError"].(bool)
var parts []string
for _, c := range content {
parts = append(parts, c.(map[string]any)["text"].(string))
}
return strings.Join(parts, "\n"), isErr
}
func call(t *testing.T, endpoint, tool string, args map[string]any) (string, bool) {
t.Helper()
body, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": 9, "method": "tools/call",
"params": map[string]any{"name": tool, "arguments": args}})
return text(t, post(t, endpoint, string(body)))
}
// novox/hq ADR 0195: the console announces six tools, and everything the mesh answers is reached
// through them by one address per layer.
func TestTheMeshsToolsAreFoundByAddress(t *testing.T) {
was := IndexKept
IndexKept = 0 // every discovery asks the mesh, so a module arriving mid-test is found
t.Cleanup(func() { IndexKept = was })
mesh := mt.New(t)
// alpha: interchangeable (the mesh issued it a plain subject); beta: state on its machine, holds
// the node-shelf seat there.
mesh.Issue(t, mt.MembershipOf("alpha", "desk", true, nil))
mesh.Issue(t, mt.MembershipOf("beta", "desk", false, map[string][]string{"node-shelf": {"list", "clear"}}))
nodeTools := connect(t, "node-tools", "desk")
stop, err := runtime.Run(nodeTools, []runtime.Served{
{Module: "alpha", Entrypoints: []string{mt.Fixture("many-alpha.serve.mjs")}},
{Module: "beta", Entrypoints: []string{mt.Fixture("many-beta.serve.mjs")}},
}, nil, (&mt.Logs{}).Logf)
if err != nil {
t.Fatal(err)
}
defer stop()
// Nothing may be asked for a roster or a module's `tools` any more (ADR 0197): counted.
var asked atomic.Int32
watcher := connect(t, "watcher", "")
for _, subject := range []string{"mesh.mod.*.tool.tools", "mesh.mod.*.tool.tools.*", "mesh.mod.mesh-catalog.>"} {
stop, err := watcher.Raw(subject, func(string, []byte) []byte { asked.Add(1); return nil })
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
}
// The controller: its records as JSON, as its seat verbs answer them, and its own seat's verbs
// announced on the bus like every runtime's.
controller := connect(t, "mesh-controller", "bench")
out := func(s string) map[string]any { return map[string]any{"output": s, "ok": true} }
serve := func(verb string, answer func() any) {
stop, err := controller.HandleSubject("mesh.seat.mesh-controller.tool."+verb, func(json.RawMessage) (any, error) {
return answer(), nil
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
}
serve("nodes", func() any {
return out("the bus's user list leaves out 2 user(s)\n" +
`[{"name":"bench","heard":"3m ago","mode":"converged","id":"1f2e"},{"name":"desk","heard":"here","mode":"converged","id":"9a8b"}]`)
})
var gammaOn atomic.Value
gammaOn.Store("[]")
serve("modules", func() any {
return out(fmt.Sprintf(`[{"module":"alpha","on":["desk"],"tools":true},{"module":"beta","on":["desk"],"tools":true},`+
`{"module":"gamma","on":%s,"tools":true},{"module":"delta","on":["desk"],"tools":false},`+
`{"module":"epsilon","on":["bench"],"tools":true},{"module":"mesh-controller","on":["bench"],"tools":true}]`, gammaOn.Load().(string)))
})
stopAnn, err := announce.Serve(controller, announce.Service{Name: "mesh-controller", ID: "bench"}, func() []announce.Endpoint {
return []announce.Endpoint{{Kind: announce.KindSeat, Module: "mesh-controller", Seat: "mesh-controller", Scope: "mesh",
// As the live controller announces: a mesh seat's holder names no machine.
Tool: "nodes", Node: "", Description: "Every machine the mesh knows.", Schema: json.RawMessage(`{}`),
Subject: "mesh.seat.mesh-controller.tool.nodes"}}
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(stopAnn)
controller.Flush()
watcher.Flush()
up, err := Serve(NewSurface(nodeTools, "desk.node-tools"), "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
defer up.Close()
endpoint := "http://" + up.Address + "/mcp"
// Six tools, nothing else.
listed := post(t, endpoint, `{"jsonrpc":"2.0","id":2,"method":"tools/list"}`)["result"].(map[string]any)
var names []string
for _, x := range listed["tools"].([]any) {
name := x.(map[string]any)["name"].(string)
names = append(names, name)
for _, r := range name {
if !(r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9' || r == '_' || r == '-') || len(name) > 64 {
t.Errorf("%q is not a name the API takes", name)
}
}
}
if got := strings.Join(names, ","); got != "mesh_overview,mesh_machine,mesh_search,mesh_describe,mesh_call,mesh_runtimes" {
t.Errorf("announced %s", got)
}
// The overview names the mesh's seats, the machines' seats and the machines.
overview, isErr := call(t, endpoint, "mesh_overview", nil)
if isErr || !strings.Contains(overview, "mesh-controller.nodes") || !strings.Contains(overview, "<node>/node-shelf.list") ||
!strings.Contains(overview, `"bench"`) || !strings.Contains(overview, `"desk"`) {
t.Errorf("overview: %s", overview)
}
// Silence is named only where tools should have answered: epsilon declares tools on bench and
// nothing there announced it; delta declares none and is never a name.
if strings.Contains(overview, "mesh-controller on bench") {
t.Errorf("a mesh seat's holder that announced no machine is called silent on its machine:\n%s", overview)
}
if !strings.Contains(overview, "epsilon on bench") || strings.Contains(overview, "delta") {
t.Errorf("not answering: %s", overview)
}
machine, isErr := call(t, endpoint, "mesh_machine", map[string]any{"node": "desk"})
if isErr || !strings.Contains(machine, "desk/node-shelf.list") || !strings.Contains(machine, "desk/beta.three") ||
!strings.Contains(machine, "desk/alpha.one") {
t.Errorf("machine: %s", machine)
}
// A mesh seat, a node seat, an assignment and an interchangeable module, each by address.
if got, isErr := call(t, endpoint, "mesh_call", map[string]any{"address": "mesh-controller.nodes"}); isErr || !strings.Contains(got, "converged") {
t.Errorf("mesh seat: %s", got)
}
if got, isErr := call(t, endpoint, "mesh_call", map[string]any{"address": "desk/node-shelf.list"}); isErr || !strings.Contains(got, `"a"`) {
t.Errorf("node seat: %s", got)
}
if got, isErr := call(t, endpoint, "mesh_call", map[string]any{"address": "desk/beta.three"}); isErr ||
!strings.Contains(got, `"beta": 3`) || !strings.Contains(got, "answered by desk") {
t.Errorf("assignment: %s", got)
}
if got, isErr := call(t, endpoint, "mesh_call", map[string]any{"address": "alpha.one"}); isErr || !strings.Contains(got, `"alpha": 1`) {
t.Errorf("interchangeable module: %s", got)
}
// Refused, by name, where the address does not say enough or says the wrong thing.
for address, want := range map[string]string{
"beta.three": "keeps state on each machine it runs on, so a call names the machine: write <node>/beta.three — it runs on desk",
"node-shelf.list": "held once per machine: write <node>/node-shelf.list — it is held on desk",
"desk/mesh-controller.nodes": "held once for the whole mesh",
"bench/beta.three": "beta does not run on bench; it runs on desk",
"nonsense": "is not an address",
} {
got, isErr := call(t, endpoint, "mesh_call", map[string]any{"address": address})
if !isErr || !strings.Contains(got, want) {
t.Errorf("%s: %s (want %q)", address, got, want)
}
}
// Discovery itself asked nothing but the bus.
if n := asked.Load(); n != 0 {
t.Errorf("discovery asked a roster or a module's tools %d time(s); it asks the bus", n)
}
// Described without `node`: the address carries the machine.
described, isErr := call(t, endpoint, "mesh_describe", map[string]any{"address": "desk/beta.three"})
if isErr || strings.Contains(described, `"node"`) || !strings.Contains(described, `"address": "desk/beta.three"`) {
t.Errorf("describe: %s", described)
}
// Its schema, which discovery no longer carries for a module's tool, asked of the runtime serving it.
if !strings.Contains(described, `"verbose"`) {
t.Errorf("describe lost the tool's arguments: %s", described)
}
// Search finds across the layers; a module that starts serving after the first answer is found.
if got, _ := call(t, endpoint, "mesh_search", map[string]any{"query": "shelf"}); !strings.Contains(got, "desk/node-shelf.list") {
t.Errorf("search a node seat: %s", got)
}
if got, _ := call(t, endpoint, "mesh_search", map[string]any{"query": "gamma"}); strings.Contains(got, "gamma.given") {
t.Fatalf("gamma was found before it served: %s", got)
}
mesh.Issue(t, mt.MembershipOf("gamma", "desk", false, nil))
gammaOn.Store(`["desk"]`)
late := connect(t, "node-tools", "desk")
stopLate, err := runtime.Run(late, []runtime.Served{{Module: "gamma", Entrypoints: []string{mt.Fixture("env-gamma.serve.mjs")}}},
nil, (&mt.Logs{}).Logf)
if err != nil {
t.Fatal(err)
}
defer stopLate()
var found string
for i := 0; i < 30; i++ {
found, _ = call(t, endpoint, "mesh_search", map[string]any{"query": "gamma given"})
if strings.Contains(found, "desk/gamma.given") {
break
}
time.Sleep(100 * time.Millisecond)
}
if !strings.Contains(found, "desk/gamma.given") {
t.Errorf("a module that arrived later was not found: %s", found)
}
// Only describing a module's tool asks its runtime — once, for the schema discovery no longer carries
// (2026-10-04: every tool's schema made a machine's discovery answer outgrow the bus).
if n := asked.Load(); n != 1 {
t.Errorf("asked a module's tools %d time(s); describing one tool asks once, and discovery never", n)
}
// The old names still answer, unannounced.
if got, isErr := call(t, endpoint, "alpha.one", nil); isErr || !strings.Contains(got, `"alpha": 1`) {
t.Errorf("an old name: %s", got)
}
}
func TestTheControllersRecordsAreReadAsJSON(t *testing.T) {
if got := jsonIn("a warning printed first\n[{\"name\":\"ace\"}]"); got != `[{"name":"ace"}]` {
t.Errorf("an array after a warning: %q", got)
}
if got := jsonIn(`{"seats":[]}`); got != `{"seats":[]}` {
t.Errorf("an object: %q", got)
}
}
+155
View File
@@ -0,0 +1,155 @@
package console
import (
"encoding/json"
"errors"
"fmt"
"regexp"
"strings"
"github.com/novox/mesh-tools/node-tools/internal/bus"
)
// Tool is a tool as its module — or, for a role's tool, the mesh's records — describes it.
type Tool struct {
Module string
Name string
Description string
Input json.RawMessage
Seat bool
Scope string
Subjects []string
}
// Listing is what the mesh could say about its tools; silence is named, never dropped (design 34 §3).
type Listing struct {
Tools []Tool
NotAnswering []string
}
// Seats is the seats and their verbs, so `<seat>.<verb>` resolves to the role.
type Seats map[string]map[string]bool
func seatsIn(l *Listing) Seats {
out := Seats{}
for _, t := range l.Tools {
if !t.Seat {
continue
}
if out[t.Module] == nil {
out[t.Module] = map[string]bool{}
}
out[t.Module][t.Name] = true
}
return out
}
// toolKey is the key a call uses: a role's when the prefix is a seat declaring that verb.
func toolKey(name string, seats Seats) string {
if strings.HasPrefix(name, "seat:") {
return name
}
dot := strings.Index(name, ".")
if dot < 0 {
return name
}
if seats[name[:dot]][name[dot+1:]] {
return "seat:" + name
}
return name
}
// toolsOn is the flat catalogue (MESH_CONSOLE_FLAT=1): what announced itself on the bus, every
// tool and seat verb, and the assignments with tools that did not (novox/hq ADR 0197).
func toolsOn(conn *bus.Conn) (*Listing, error) {
x, err := indexOn(conn)
if err != nil {
return nil, err
}
return x.Listing, nil
}
// callTool calls `<module>.<tool>[@<node>]`, on the subject the listing names for it when it names one.
func callTool(conn *bus.Conn, key string, args any, seats Seats, l *Listing) (bus.Answered, error) {
name, node, _ := strings.Cut(key, "@")
if !strings.Contains(name, ".") {
return bus.Answered{}, fmt.Errorf("%q does not name a tool: write <module>.<tool>, as `mesh tools` lists "+
"them, or <module>.<tool>@<node> for the instance on one machine", key)
}
resolved := toolKey(name, seats)
if node != "" {
resolved += "@" + node
}
return conn.Ask(resolved, args, subjectListed(name, node, l))
}
func subjectListed(name, node string, l *Listing) string {
if l == nil {
return ""
}
dot := strings.Index(name, ".")
module, tool := name[:dot], name[dot+1:]
for _, t := range l.Tools {
if t.Module == module && t.Name == tool && !t.Seat {
if len(t.Subjects) == 0 {
return ""
}
if node == "" {
return t.Subjects[0]
}
for _, s := range t.Subjects {
if strings.HasSuffix(s, "."+node) {
return s
}
}
return ""
}
}
return ""
}
var (
noResponders = regexp.MustCompile(`(?i)no responders|503`)
// The bus's own two refusals, by their whole phrase. "authorization" alone also matched a tool's own
// answer that merely mentions the word — the runtime refusing a state value with an Authorization
// header (novox/hq ADR 0201) read as "this account may not call", which sent the reader the wrong way.
refused = regexp.MustCompile(`(?i)permissions violation|authorization violation`)
timedOut = regexp.MustCompile(`(?i)timeout`)
)
// controllerSeat is the seat whose holder keeps what came of every call (novox/hq issue 265).
const controllerSeat = "mesh-controller"
// whyItFailed says why a call failed, so the remedy is in the words.
func whyItFailed(key string, err error) string {
if err == nil {
err = errors.New("failed")
}
msg := err.Error()
switch {
case noResponders.MatchString(msg):
extra := ""
if strings.HasPrefix(key, "seat:") {
extra = ", or nothing holds that seat"
}
return "nothing serves " + key + ". The module may not be assigned to any machine, or it is down" +
extra + " — `mesh tools` lists what answered."
case refused.MatchString(msg):
return "this account may not call " + key + ". What it may call was fixed when it was issued — a " +
"person's by `operator issue`, the console's by its manifest."
case timedOut.MatchString(msg):
// **A timeout is not a failure** (novox/hq issue 265). Something took the call, so it may
// still be running and may already have done what was asked: a push said this while it pushed.
said := key + " did not answer within " + bus.RequestTimeout.String() + ". That is not a failure: " +
"something is serving it, so it may still be running and may already have done what was " +
"asked — check before repeating a call that changes the mesh."
if strings.HasPrefix(strings.TrimPrefix(key, "seat:"), controllerSeat+".") {
// The controller answers every call within seconds, with an id while it runs, so silence
// from it is an answer lost on the way — which it keeps.
said += " The controller answers every call within seconds, so its answer was lost on the " +
"way; " + controllerSeat + ".calls lists its recent calls and what came of each."
}
return said
}
return key + " failed: " + msg
}
+131
View File
@@ -0,0 +1,131 @@
package console
import (
"bytes"
"encoding/json"
"io"
"net/http"
"strings"
"testing"
"github.com/novox/mesh-tools/node-tools/internal/bus"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
"github.com/novox/mesh-tools/node-tools/internal/runtime"
)
func connect(t *testing.T, module, node string) *bus.Conn {
t.Helper()
c, err := bus.Connect(bus.Credential{URL: mt.URL(t), Module: module, Node: node})
if err != nil {
t.Fatal(err)
}
c.Logf = func(string, ...any) {}
t.Cleanup(c.Close)
return c
}
func post(t *testing.T, endpoint string, body string) map[string]any {
t.Helper()
res, err := http.Post(endpoint, "application/json", bytes.NewBufferString(body))
if err != nil {
t.Fatal(err)
}
defer res.Body.Close()
raw, _ := io.ReadAll(res.Body)
var out map[string]any
if err := json.Unmarshal(raw, &out); err != nil {
t.Fatalf("%d %s", res.StatusCode, raw)
}
return out
}
// As node-tools, the runtime serves the bundles and is the console on loopback: the listing is what
// the modules and the mesh's records answered, and a call reaches the module on the machine named.
func TestTheConsoleListsAndCallsOverHTTP(t *testing.T) {
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("alpha", "desk", true, nil))
mesh.Issue(t, mt.MembershipOf("beta", "desk", false, map[string][]string{"node-shelf": {"list", "clear"}}))
nodeTools := connect(t, "node-tools", "desk")
stop, err := runtime.Run(nodeTools, []runtime.Served{
{Module: "alpha", Entrypoints: []string{mt.Fixture("many-alpha.serve.mjs")}},
{Module: "beta", Entrypoints: []string{mt.Fixture("many-beta.serve.mjs")}},
}, nil, (&mt.Logs{}).Logf)
if err != nil {
t.Fatal(err)
}
defer stop()
// The controller's records (ADR 0197): ghost declares tools on desk and nothing announces it.
controller := connect(t, "mesh-controller", "")
stopSeat, _ := controller.HandleSubject("mesh.seat.mesh-controller.tool.modules", func(json.RawMessage) (any, error) {
return map[string]any{"ok": true, "output": `[{"module":"alpha","on":["desk"],"tools":true},` +
`{"module":"beta","on":["desk"],"tools":true},{"module":"ghost","on":["desk"],"tools":true}]`}, nil
})
defer stopSeat()
controller.Flush()
flat := NewSurface(nodeTools, "desk.node-tools")
flat.Flat = true // the whole catalogue, as before ADR 0195: still reachable, no longer announced
up, err := Serve(flat, "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
defer up.Close()
endpoint := "http://" + up.Address + "/mcp"
init := post(t, endpoint, `{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{}}}`)
if !strings.Contains(init["result"].(map[string]any)["instructions"].(string), "reached as desk.node-tools") {
t.Errorf("initialize: %v", init)
}
listed := post(t, endpoint, `{"jsonrpc":"2.0","id":2,"method":"tools/list"}`)["result"].(map[string]any)
var names []string
for _, x := range listed["tools"].([]any) {
names = append(names, x.(map[string]any)["name"].(string))
}
if got := strings.Join(names, ","); got != "alpha.one,alpha.two,beta.five,beta.four,beta.three,node-shelf.clear,node-shelf.list" {
t.Errorf("listed %s", got)
}
if got := listed["_meta"].(map[string]any)["notAnswering"]; len(got.([]any)) != 1 || got.([]any)[0] != "ghost on desk" {
t.Errorf("not answering: %v", got)
}
for _, x := range listed["tools"].([]any) {
tool := x.(map[string]any)
if tool["name"] == "node-shelf.list" {
schema := tool["inputSchema"].(map[string]any)
if req, _ := schema["required"].([]any); len(req) != 1 || req[0] != "node" {
t.Errorf("a node seat's verb does not require node: %v", schema)
}
}
}
called := post(t, endpoint, `{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"alpha.one","arguments":{"node":"desk"}}}`)
content := called["result"].(map[string]any)["content"].([]any)
var got map[string]any
_ = json.Unmarshal([]byte(content[0].(map[string]any)["text"].(string)), &got)
if got["alpha"] != float64(1) || content[1].(map[string]any)["text"] != "answered by desk" {
t.Errorf("called: %v", called)
}
seat := post(t, endpoint, `{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"node-shelf.list","arguments":{"node":"desk"}}}`)
if text := seat["result"].(map[string]any)["content"].([]any)[0].(map[string]any)["text"].(string); !strings.Contains(text, `"a"`) {
t.Errorf("seat verb: %v", seat)
}
refused := post(t, endpoint, `{"jsonrpc":"2.0","id":5,"method":"tools/call","params":{"name":"node-shelf.list","arguments":{}}}`)
if refused["error"] == nil {
t.Errorf("a node seat's verb was called without its machine: %v", refused)
}
absent := post(t, endpoint, `{"jsonrpc":"2.0","id":6,"method":"tools/call","params":{"name":"ghost.boo","arguments":{}}}`)
result := absent["result"].(map[string]any)
if result["isError"] != true || !strings.Contains(result["content"].([]any)[0].(map[string]any)["text"].(string), "nothing serves ghost.boo") {
t.Errorf("an absent tool: %v", absent)
}
res, err := http.Post(endpoint, "application/json", strings.NewReader(`{"jsonrpc":"2.0","method":"notifications/initialized"}`))
if err != nil || res.StatusCode != 202 {
t.Errorf("a notification: %v %v", res, err)
}
}
func TestTheConsoleListensOnLoopbackAndNowhereElse(t *testing.T) {
if _, err := Serve(NewSurface(nil, "x"), "0.0.0.0:0"); err == nil || !strings.Contains(err.Error(), "loopback and nowhere else") {
t.Errorf("a non-loopback console was not refused: %v", err)
}
}
+125
View File
@@ -0,0 +1,125 @@
package console
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/http"
"strconv"
"strings"
"github.com/novox/mesh-tools/node-tools/internal/wire"
)
// bodyLimit is the most a request body may be.
const bodyLimit = 1 << 20
var loopback = map[string]bool{"127.0.0.1": true, "::1": true, "localhost": true, "[::1]": true}
// Listening is a console that listens: where, with the port the machine gave, and how to stop it.
type Listening struct {
Address string
Close func() error
}
// Serve listens on host:port, refused unless the host is loopback — said before binding, so a
// console that would open to a network is a startup failure (ADR 0152).
func Serve(s *Surface, listen string) (*Listening, error) {
at := strings.LastIndex(listen, ":")
if at < 0 {
return nil, fmt.Errorf("%q is not host:port", listen)
}
host, portText := listen[:at], listen[at+1:]
if !loopback[host] {
return nil, fmt.Errorf(`the console listens on loopback and nowhere else (novox/hq ADR 0152): %q is not this `+
"machine's own address — whoever is on the machine owns the mesh there, and nobody else may reach this", host)
}
port, err := strconv.Atoi(portText)
if err != nil || port < 0 || port > 65535 {
return nil, fmt.Errorf("%q is not a port", portText)
}
ln, err := net.Listen("tcp", net.JoinHostPort(strings.Trim(host, "[]"), portText))
if err != nil {
return nil, err
}
server := &http.Server{Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { route(w, r, s) })}
go func() { _ = server.Serve(ln) }()
bound := ln.Addr().(*net.TCPAddr).Port
return &Listening{Address: host + ":" + strconv.Itoa(bound), Close: func() error { return server.Shutdown(context.Background()) }}, nil
}
func route(w http.ResponseWriter, r *http.Request, s *Surface) {
switch r.URL.Path {
case "/":
w.Header().Set("content-type", "text/plain; charset=utf-8")
_, _ = io.WriteString(w, "the mesh's console: MCP over HTTP at POST /mcp (novox/hq design 34)\n")
return
case "/mcp":
default:
writeJSON(w, 404, map[string]any{"error": "the console serves /mcp and nothing else"})
return
}
switch r.Method {
case http.MethodPost:
case http.MethodDelete:
w.WriteHeader(204) // no session to end
return
default:
w.Header().Set("allow", "POST, DELETE")
w.WriteHeader(405)
return
}
body, err := io.ReadAll(io.LimitReader(r.Body, bodyLimit+1))
if err != nil || len(body) > bodyLimit {
if err == nil {
err = fmt.Errorf("the request is larger than %d bytes", bodyLimit)
}
writeJSON(w, 413, map[string]any{"jsonrpc": "2.0", "id": nil, "error": map[string]any{"code": -32600, "message": err.Error()}})
return
}
trimmed := strings.TrimSpace(string(body))
if strings.HasPrefix(trimmed, "[") {
var batch []Request
if err := json.Unmarshal(body, &batch); err != nil {
writeJSON(w, 400, map[string]any{"jsonrpc": "2.0", "id": nil, "error": map[string]any{"code": -32700, "message": "the body is not JSON"}})
return
}
replies := []*Reply{}
for _, req := range batch {
if reply := s.Handle(req); reply != nil {
replies = append(replies, reply)
}
}
if len(replies) == 0 {
w.WriteHeader(202)
return
}
writeJSON(w, 200, replies)
return
}
var req Request
if err := json.Unmarshal(body, &req); err != nil {
writeJSON(w, 400, map[string]any{"jsonrpc": "2.0", "id": nil, "error": map[string]any{"code": -32700, "message": "the body is not JSON"}})
return
}
reply := s.Handle(req)
if reply == nil {
w.WriteHeader(202)
return
}
writeJSON(w, 200, reply)
}
func writeJSON(w http.ResponseWriter, status int, body any) {
text, err := wire.Marshal(body)
if err != nil {
text, _ = json.Marshal(map[string]any{"error": errors.New("unencodable answer").Error()})
}
w.Header().Set("content-type", "application/json; charset=utf-8")
w.Header().Set("content-length", strconv.Itoa(len(text)))
w.WriteHeader(status)
_, _ = w.Write(text)
}
+329
View File
@@ -0,0 +1,329 @@
// Package console is the mesh's tools as an MCP server on a machine's loopback (novox/hq design 34,
// ADR 0152, ADR 0175 §6): the Go port of node-tools' http.ts, mcp.ts and client.ts. A thin adapter:
// every tool listed is one a module answered for, the schema is the module's, the answer the module's.
package console
import (
"encoding/json"
"os"
"strings"
"sync"
"time"
"github.com/novox/mesh-tools/node-tools/internal/bus"
)
// Protocol is the MCP version spoken to an agent host.
const Protocol = "2025-03-26"
// ListingKept is how long a fetched tool list is kept before the modules are asked again.
var ListingKept = 30 * time.Second
// Request is one JSON-RPC message from a host.
type Request struct {
JSONRPC string `json:"jsonrpc"`
ID json.RawMessage `json:"id,omitempty"`
Method string `json:"method"`
Params json.RawMessage `json:"params,omitempty"`
}
// Reply is one JSON-RPC answer.
type Reply struct {
JSONRPC string `json:"jsonrpc"`
ID json.RawMessage `json:"id"`
Result any `json:"result,omitempty"`
Error *RPCError `json:"error,omitempty"`
}
// RPCError is a protocol-level refusal.
type RPCError struct {
Code int `json:"code"`
Message string `json:"message"`
}
// Surface answers MCP requests over one bus connection, as one account.
type Surface struct {
conn *bus.Conn
who string
mu sync.Mutex
known *Listing
at time.Time
idx *index
idxAt time.Time
// runtimes is every runtime discovery has heard, as it last heard it (mesh_runtimes).
runtimes map[string]*runtimeSeen
// Flat announces the whole catalogue, as the console did before ADR 0195: for a person reading
// it or a client that wants it. Off by default; MESH_CONSOLE_FLAT=1 turns it on.
Flat bool
}
// NewSurface is the surface over a connection, as `who`.
func NewSurface(conn *bus.Conn, who string) *Surface {
return &Surface{conn: conn, who: who, Flat: os.Getenv("MESH_CONSOLE_FLAT") == "1"}
}
func (s *Surface) listing() (*Listing, error) {
s.mu.Lock()
if s.known != nil && time.Since(s.at) <= ListingKept {
l := s.known
s.mu.Unlock()
return l, nil
}
s.mu.Unlock()
l, err := toolsOn(s.conn)
if err != nil {
return nil, err
}
s.mu.Lock()
s.known, s.at = l, time.Now()
s.mu.Unlock()
return l, nil
}
func isNotification(id json.RawMessage) bool {
t := strings.TrimSpace(string(id))
return t == "" || t == "null"
}
func answer(id json.RawMessage, result any) *Reply {
return &Reply{JSONRPC: "2.0", ID: idOrNull(id), Result: result}
}
func refuse(id json.RawMessage, code int, message string) *Reply {
return &Reply{JSONRPC: "2.0", ID: idOrNull(id), Error: &RPCError{Code: code, Message: message}}
}
func idOrNull(id json.RawMessage) json.RawMessage {
if isNotification(id) {
return json.RawMessage("null")
}
return id
}
// Handle answers one request; nil for a notification, which expects none.
func (s *Surface) Handle(r Request) *Reply {
notification := isNotification(r.ID)
switch r.Method {
case "initialize":
return answer(r.ID, map[string]any{
"protocolVersion": Protocol,
"capabilities": map[string]any{"tools": map[string]any{}},
"serverInfo": map[string]any{"name": "mesh", "version": "1"},
"instructions": s.instructions(),
})
case "notifications/initialized":
return nil
case "ping":
if notification {
return nil
}
return answer(r.ID, map[string]any{})
case "tools/list":
if !s.Flat {
return answer(r.ID, map[string]any{"tools": discovery()})
}
l, err := s.listing()
if err != nil {
return refuse(r.ID, -32603, "the mesh's discovery failed: "+err.Error())
}
tools := make([]map[string]any, 0, len(l.Tools))
inputs := map[string]map[string]json.RawMessage{} // module → tool → input, asked once per module
for i, t := range l.Tools {
if !t.Seat && len(t.Input) == 0 {
if inputs[t.Module] == nil {
inputs[t.Module] = toolInputs(s.conn, t.Module)
}
l.Tools[i].Input = inputs[t.Module][t.Name]
t = l.Tools[i]
}
var schema map[string]any
switch {
case t.Seat && t.Scope != "node":
schema = asSchema(t.Input)
case t.Seat:
schema = withNode(asSchema(t.Input), "the machine whose seat answers; required, the seat is held once per machine", true)
default:
schema = withNode(asSchema(t.Input), "", false)
}
description := t.Description
if description == "" {
description = t.Name + ", served by " + t.Module
}
tools = append(tools, map[string]any{"name": t.Module + "." + t.Name, "description": description, "inputSchema": schema})
}
return answer(r.ID, map[string]any{"tools": tools, "_meta": map[string]any{"notAnswering": l.NotAnswering}})
case "tools/call":
var p struct {
Name string `json:"name"`
Arguments map[string]any `json:"arguments"`
}
_ = json.Unmarshal(r.Params, &p)
if isDiscovery(p.Name) {
if p.Arguments == nil {
p.Arguments = map[string]any{}
}
return answer(r.ID, s.discover(p.Name, p.Arguments))
}
args := map[string]any{}
for k, v := range p.Arguments {
args[k] = v
}
l, _ := s.listing()
var roles Seats
if l != nil {
roles = seatsIn(l)
}
bare, _, _ := strings.Cut(p.Name, "@")
isSeatVerb := roles != nil && strings.HasPrefix(toolKey(bare, roles), "seat:")
nodeScoped := false
if isSeatVerb && l != nil {
for _, t := range l.Tools {
if t.Seat && t.Scope == "node" && t.Module+"."+t.Name == bare {
nodeScoped = true
}
}
}
takesNode := !isSeatVerb || nodeScoped
node := ""
if takesNode {
if n, ok := args["node"].(string); ok {
node = n
}
delete(args, "node")
}
if nodeScoped && node == "" && !strings.Contains(p.Name, "@") {
return refuse(r.ID, -32602, p.Name+" is a machine's seat's verb: name the machine with `node`")
}
name := p.Name
if node != "" && !strings.Contains(p.Name, "@") {
name = p.Name + "@" + node
}
got, err := callTool(s.conn, name, args, roles, l)
if err != nil {
return answer(r.ID, map[string]any{
"content": []map[string]any{{"type": "text", "text": whyItFailed(name, err)}},
"isError": true,
})
}
content := []map[string]any{{"type": "text", "text": pretty(got.Result)}}
if got.Node != "" {
content = append(content, map[string]any{"type": "text", "text": "answered by " + got.Node})
}
return answer(r.ID, map[string]any{"content": content})
}
if notification {
return nil
}
return refuse(r.ID, -32601, "mesh's MCP surface has no "+r.Method)
}
// pretty is a module's answer as JSON text, indented as JSON.stringify(result, null, 2) writes it.
func pretty(raw json.RawMessage) string {
if len(raw) == 0 {
return "null"
}
var v any
if json.Unmarshal(raw, &v) != nil {
return string(raw)
}
b, err := json.MarshalIndent(v, "", " ")
if err != nil {
return string(raw)
}
return strings.NewReplacer(`<`, "<", `>`, ">", `&`, "&").Replace(string(b))
}
// asSchema is a module's declared input as a JSON schema: wrapped when it is a bare map of
// properties, passed through when it is a schema, empty when nothing was declared.
func asSchema(raw json.RawMessage) map[string]any {
var given map[string]any
if json.Unmarshal(raw, &given) != nil || given == nil {
return map[string]any{"type": "object", "properties": map[string]any{}}
}
if given["type"] == "object" {
return given
}
if _, has := given["properties"]; has {
return given
}
if len(given) == 0 {
return map[string]any{"type": "object", "properties": map[string]any{}}
}
return map[string]any{"type": "object", "properties": given}
}
// withNode adds the optional — or, for a node seat, required — `node` argument (ADR 0159).
func withNode(schema map[string]any, description string, required bool) map[string]any {
properties := map[string]any{}
if p, ok := schema["properties"].(map[string]any); ok {
for k, v := range p {
properties[k] = v
}
}
if _, has := properties["node"]; !has {
if description == "" {
description = "the machine to ask, when this module runs on several; else whichever answers, and the answer says which"
}
properties["node"] = map[string]any{"type": "string", "description": description}
}
out := map[string]any{}
for k, v := range schema {
out[k] = v
}
out["type"] = "object"
out["properties"] = properties
if required {
have := []any{}
if r, ok := schema["required"].([]any); ok {
have = r
}
hasNode := false
for _, x := range have {
hasNode = hasNode || x == "node"
}
if !hasNode {
have = append(have, "node")
}
out["required"] = have
}
return out
}
// instructions is what an agent host is told about this surface when it connects.
func (s *Surface) instructions() string {
if s.Flat {
return "These are the tools of a Novox mesh, reached as " + s.who + ". Every call goes to the module " +
"that serves it; what may be called was fixed when this account was issued, so a " +
"refusal means the account, not the tool. The list is what the running modules " +
"answered, plus every role's tools from the mesh's records — the mesh's own verbs " +
"(mesh-controller.status, .push, .assign …) among them; a module that did not answer " +
"is named in the list's _meta and can still be called by <module>.<tool>."
}
return "The tools of a Novox mesh, reached as " + s.who + ", found by address rather than listed " +
"whole (novox/hq ADR 0195). mesh_overview shows the mesh's seats and machines; mesh_machine one " +
"machine's seats and modules; mesh_search finds a tool by words; mesh_describe gives one tool's " +
"arguments; mesh_call calls it; mesh_runtimes says which machines' runtimes answered discovery, how fast " +
"and how much — ask it when an address is said to be missing or on another machine. " + grammar + " What may be called was fixed when this account " +
"was issued, so a refusal means the account, not the tool."
}
// toolInputs is every tool's input schema of one module, asked of a runtime that serves it.
func toolInputs(conn *bus.Conn, module string) map[string]json.RawMessage {
out := map[string]json.RawMessage{}
got, err := conn.Ask(module+".tools", map[string]any{}, "")
if err != nil {
return out
}
var answer struct {
Tools []struct {
Name string `json:"name"`
Input json.RawMessage `json:"input"`
} `json:"tools"`
}
if json.Unmarshal(got.Result, &answer) == nil {
for _, t := range answer.Tools {
out[t.Name] = t.Input
}
}
return out
}
@@ -0,0 +1,52 @@
package console
import (
"encoding/json"
"strings"
"testing"
)
// novox/hq issue 244: `node` is taken out only where the address names the machine. A seat held once
// for the mesh takes the machine it acts on as `node`, and is described and called with it — on
// 2026-10-05 the controller's push was described as taking nothing, and a push naming one machine
// ran as a push of every machine behind.
func TestAMeshSeatsNodeIsItsOwnArgument(t *testing.T) {
push := Tool{Module: "mesh-controller", Name: "push", Seat: true, Scope: "mesh",
Input: json.RawMessage(`{"type":"object","properties":{"node":{"type":"string"},"behind":{"type":"string"}}}`)}
mesh := target{Address: "mesh-controller.push", Tool: push, Seat: true}
props := schemaAsPassed(mesh)["properties"].(map[string]any)
if _, has := props["node"]; !has {
t.Fatalf("a mesh seat's verb was described without its node: %v", props)
}
sent, err := argumentsAsSent(mesh, map[string]any{"node": "g1"})
if err != nil || sent["node"] != "g1" {
t.Fatalf("a mesh seat's verb was called without the machine it was given: %v %v", sent, err)
}
// An argument the seat's verb does not declare is refused, naming it, never sent to be dropped.
if _, err := argumentsAsSent(mesh, map[string]any{"machine": "g1"}); err == nil || !strings.Contains(err.Error(), "machine") {
t.Fatalf("an undeclared argument was sent: %v", err)
}
// A machine's seat: the address names the machine, so `node` is out of the schema; given the same
// machine again it is taken out, given another it is refused.
shelf := Tool{Module: "node-shelf", Name: "list", Seat: true, Scope: "node",
Input: json.RawMessage(`{"type":"object","properties":{"limit":{"type":"string"}}}`)}
onDesk := target{Address: "desk/node-shelf.list", Node: "desk", Tool: shelf, Seat: true}
if _, has := schemaAsPassed(onDesk)["properties"].(map[string]any)["node"]; has {
t.Fatal("a machine's seat was described with a node the address already names")
}
if sent, err := argumentsAsSent(onDesk, map[string]any{"node": "desk", "limit": "3"}); err != nil || sent["node"] != nil || sent["limit"] != "3" {
t.Fatalf("the same machine named twice: %v %v", sent, err)
}
if _, err := argumentsAsSent(onDesk, map[string]any{"node": "bench"}); err == nil || !strings.Contains(err.Error(), "bench") {
t.Fatalf("another machine in the arguments than in the address was dropped: %v", err)
}
// A module's tool is its own: what it takes beyond node is the module's to judge.
beta := target{Address: "desk/beta.three", Node: "desk", Tool: Tool{Module: "beta", Name: "three"}}
if sent, err := argumentsAsSent(beta, map[string]any{"node": "desk", "verbose": true}); err != nil || sent["verbose"] != true || sent["node"] != nil {
t.Fatalf("a module's tool: %v %v", sent, err)
}
}
@@ -0,0 +1,57 @@
package console
import (
"strings"
"testing"
"github.com/novox/mesh-tools/node-tools/internal/announce"
)
// **Issue 287**: mesh-delivery is the delivery's seat, held once for the mesh, and the module holding it,
// which answers the seat's verbs with tools of the same names and has tools of its own. Discovery put the
// module's tool and the seat's verb under one key, so the seat was listed with no verb; and every address
// with the name resolved to the seat — `mesh-delivery.deliveries` and `novox/mesh-delivery.delivery_status`
// alike were refused, the seat's verbs and the module's tools both out of reach.
func TestASeatAndAModuleOfOneNameAreEachReached(t *testing.T) {
var endpoints []announce.Endpoint
for _, tool := range []string{"deliveries", "delivery_status"} {
endpoints = append(endpoints, announce.Endpoint{Kind: announce.KindTool, Module: "mesh-delivery", Tool: tool,
Node: "anchor", Subject: "mesh.mod.mesh-delivery.tool." + tool + ".anchor"})
}
for _, verb := range []string{"deliveries", "stalled"} {
endpoints = append(endpoints, announce.Endpoint{Kind: announce.KindSeat, Module: "mesh-delivery", Seat: "mesh-delivery",
Tool: verb, Scope: "mesh", Subject: "mesh.seat.mesh-delivery.tool." + verb})
}
x, _, _ := indexOfEndpoints(endpoints)
s := x.seat("mesh-delivery")
if s == nil || len(s.Verbs) != 2 {
t.Fatalf("the seat is listed with verbs %v, not its two", s)
}
if m := x.Modules["mesh-delivery"]; m == nil || len(m.Tools) != 2 {
t.Fatalf("the module is listed with tools %v, not its two", m)
}
for address, want := range map[string]struct {
seat bool
key string
}{
"mesh-delivery.deliveries": {true, "seat:mesh-delivery.deliveries"},
"mesh-delivery.stalled": {true, "seat:mesh-delivery.stalled"},
"anchor/mesh-delivery.delivery_status": {false, "mesh-delivery.delivery_status@anchor"},
"anchor/mesh-delivery.deliveries": {false, "mesh-delivery.deliveries@anchor"},
} {
got, err := resolve(x, address)
if err != nil {
t.Errorf("%s: %v", address, err)
continue
}
if got.Seat != want.seat || got.Key != want.key {
t.Errorf("%s resolves to %+v, not the %s", address, got, want.key)
}
}
// A module tool of the name without a machine: the module keeps state on its machine, so it says so.
if _, err := resolve(x, "mesh-delivery.delivery_status"); err == nil || !strings.Contains(err.Error(), "<node>/") {
t.Errorf("the module's tool without a machine: %v", err)
}
}
@@ -0,0 +1,37 @@
package console
import (
"errors"
"strings"
"testing"
)
// A refusal is said only for the bus's own refusals; a tool's answer that mentions authorization is the
// tool's answer, not the account's.
func TestARefusalIsSaidOnlyForTheBussOwn(t *testing.T) {
for msg, refusal := range map[string]bool{
`nats: Permissions Violation for Publish to "mesh.mod.x.tool.y"`: true,
"nats: Authorization Violation": true,
`claude-code's servers.all.x carries a field "Authorization", which names a credential`: false,
} {
got := strings.HasPrefix(whyItFailed("x.y", errors.New(msg)), "this account may not call")
if got != refusal {
t.Errorf("%q read as a refusal: %v, want %v", msg, got, refusal)
}
}
}
// **A timeout says it is not a failure** (novox/hq issue 265): a push that outlasted the wait had
// pushed, and the caller read "did not answer in time". And from the controller, where its answer is.
func TestATimeoutIsNotAFailure(t *testing.T) {
got := whyItFailed("seat:mesh-controller.push", errors.New("timeout"))
for _, want := range []string{"not a failure", "may already have done", "mesh-controller.calls"} {
if !strings.Contains(got, want) {
t.Errorf("a controller timeout does not say %q: %s", want, got)
}
}
if other := whyItFailed("novox/postgres.postgres_list_databases", errors.New("timeout")); !strings.Contains(other, "not a failure") ||
strings.Contains(other, "calls") {
t.Errorf("a module's timeout: %s", other)
}
}
+200
View File
@@ -0,0 +1,200 @@
package console
// What discovery heard from each runtime (2026-10-05): the console said a module ran nowhere when the
// largest runtime's answer arrived after the window closed, and nothing showed that it had. Every
// round of discovery is remembered per runtime, so a runtime heard before and not now is named, and
// `mesh_runtimes` says who answered, how fast and how much.
import (
"fmt"
"sort"
"time"
"github.com/novox/mesh-tools/node-tools/internal/announce"
)
// RuntimeForgotten is how long a runtime heard once is expected again: within it, its silence is said
// wherever an address it might answer is asked for; after it, it is listed only by mesh_runtimes.
var RuntimeForgotten = 15 * time.Minute
// runtimeSeen is one runtime instance as discovery last heard it.
type runtimeSeen struct {
Name string
ID string
Machine string
LastHeard time.Time
Took time.Duration
Bytes int
Modules int
Tools int
SeatVerbs int
Shortened string
Answered bool // in the latest round
Why string // why not, when it did not
}
func runtimeKey(name, id string) string { return name + "/" + id }
// seenOf is what one answer announced, counted.
func seenOf(h announce.Heard, at time.Time) *runtimeSeen {
modules := map[string]bool{}
tools := map[string]bool{}
verbs := map[string]bool{}
for _, e := range announce.Endpoints(h.Info) {
switch e.Kind {
case announce.KindTool:
modules[e.Module] = true
tools[e.Module+"."+e.Tool] = true
case announce.KindSeat:
verbs[e.Seat+"."+e.Tool] = true
}
}
return &runtimeSeen{Name: h.Info.Name, ID: h.Info.ID, Machine: h.Machine(), LastHeard: at, Took: h.Took,
Bytes: h.Bytes, Modules: len(modules), Tools: len(tools), SeatVerbs: len(verbs), Shortened: h.Shortened, Answered: true}
}
// remember records a round of discovery and adds to the index every runtime heard lately that did not
// answer this time. Called with s.mu held.
func (s *Surface) remember(x *index) {
if s.runtimes == nil {
s.runtimes = map[string]*runtimeSeen{}
}
at := x.Discovery.At
if at.IsZero() {
at = time.Now().UTC()
}
heard := map[string]bool{}
for _, h := range x.Discovery.Heard {
k := runtimeKey(h.Info.Name, h.Info.ID)
seen := seenOf(h, at)
if heard[k] {
// Two instances under one name and id: both counted, the slower answer's time kept.
prev := s.runtimes[k]
seen.Modules += prev.Modules
seen.Tools += prev.Tools
seen.SeatVerbs += prev.SeatVerbs
seen.Bytes += prev.Bytes
if prev.Took > seen.Took {
seen.Took = prev.Took
}
}
heard[k] = true
s.runtimes[k] = seen
}
// A runtime restarted answers under a new instance id: the old one is gone, not silent, and saying
// it was missed would hide nothing but cry wolf after every restart.
now := map[string]bool{}
for k := range heard {
now[s.runtimes[k].Name+"@"+s.runtimes[k].Machine] = true
}
for k, r := range s.runtimes {
if !heard[k] && r.Machine != "" && now[r.Name+"@"+r.Machine] {
delete(s.runtimes, k)
}
}
silent := map[string]string{}
for _, u := range x.Discovery.Silent {
silent[runtimeKey(u.Name, u.ID)] = fmt.Sprintf("it answered PING; what it serves did not arrive within %s", announce.Patience)
}
for k, r := range s.runtimes {
if heard[k] {
continue
}
r.Answered = false
if why, ok := silent[k]; ok {
r.Why = why
continue // already in the index's unheard
}
r.Why = "it did not answer this round of discovery"
if time.Since(r.LastHeard) <= RuntimeForgotten {
x.Unheard = append(x.Unheard, unheard{Runtime: r.Name, Machine: r.Machine,
Why: fmt.Sprintf("it answered discovery %s ago, not now", time.Since(r.LastHeard).Round(time.Second))})
}
}
for _, u := range x.Discovery.Silent {
k := runtimeKey(u.Name, u.ID)
if _, known := s.runtimes[k]; !known {
s.runtimes[k] = &runtimeSeen{Name: u.Name, ID: u.ID, Machine: u.Machine, Why: silent[k]}
}
}
}
// runtimesAnswer is mesh_runtimes: discovery asked now, and every runtime as it was heard.
func (s *Surface) runtimesAnswer() map[string]any {
x, err := s.indexAsked(true)
if err != nil {
return failure("the mesh's discovery failed: " + err.Error())
}
s.mu.Lock()
var all []runtimeSeen
for _, r := range s.runtimes {
all = append(all, *r)
}
s.mu.Unlock()
sort.Slice(all, func(i, j int) bool {
if all[i].Machine != all[j].Machine {
return all[i].Machine < all[j].Machine
}
return runtimeKey(all[i].Name, all[i].ID) < runtimeKey(all[j].Name, all[j].ID)
})
now := time.Now()
var answered, notHeard []map[string]any
onMachine := map[string]bool{}
for _, r := range all {
entry := map[string]any{"runtime": r.Name, "instance": r.ID}
if r.Machine != "" {
entry["machine"] = r.Machine
}
if !r.LastHeard.IsZero() {
entry["last heard"] = r.LastHeard.Format(time.RFC3339)
entry["last heard, ago"] = now.Sub(r.LastHeard).Round(time.Second).String()
entry["took"] = r.Took.Round(time.Millisecond).String()
entry["bytes"] = r.Bytes
entry["modules"] = r.Modules
entry["tools"] = r.Tools
entry["seat verbs"] = r.SeatVerbs
entry["shortened"] = r.Shortened != ""
if r.Shortened != "" {
entry["shortened, how"] = r.Shortened
}
}
if r.Answered {
onMachine[r.Machine] = true
answered = append(answered, entry)
} else {
entry["why"] = r.Why
if r.LastHeard.IsZero() {
entry["last heard"] = "never in full, since this console started"
}
notHeard = append(notHeard, entry)
}
}
for _, m := range x.Machines {
if !onMachine[m] && !machineListed(notHeard, m) {
notHeard = append(notHeard, map[string]any{"machine": m,
"why": "the mesh knows this machine and no runtime on it has answered discovery since this console started"})
}
}
if answered == nil {
answered = []map[string]any{}
}
if notHeard == nil {
notHeard = []map[string]any{}
}
return answerText(map[string]any{
"asked": x.Discovery.At.Format(time.RFC3339),
"waited": x.Discovery.Waited.Round(time.Millisecond).String(),
"window": fmt.Sprintf("at least %s; up to %s for a runtime that answered PING", announce.Window, announce.Patience),
"answered": answered,
"not heard": notHeard,
})
}
func machineListed(entries []map[string]any, machine string) bool {
for _, e := range entries {
if e["machine"] == machine {
return true
}
}
return false
}
@@ -0,0 +1,219 @@
package console
import (
"encoding/json"
"strings"
"testing"
"time"
"github.com/nats-io/nats.go/micro"
"github.com/novox/mesh-tools/node-tools/internal/announce"
"github.com/novox/mesh-tools/node-tools/internal/bus"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
// slowRuntime answers PING at once and INFO after `lag`, or never with a negative lag.
func slowRuntime(t *testing.T, conn *bus.Conn, node string, lag time.Duration, endpoints []announce.Endpoint) {
t.Helper()
id := micro.ServiceIdentity{Name: "node-tools", ID: node, Version: announce.Version, Metadata: map[string]string{"node": node}}
ping, _ := json.Marshal(micro.Ping{ServiceIdentity: id, Type: micro.PingResponseType})
stop, err := conn.Raw("$SRV.PING", func(string, []byte) []byte { return ping })
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
stop, err = conn.Raw("$SRV.INFO", func(string, []byte) []byte {
if lag < 0 {
return nil
}
time.Sleep(lag)
body, _ := json.Marshal(announce.Info(announce.Service{Name: id.Name, ID: id.ID, Metadata: id.Metadata}, endpoints))
return body
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
conn.Flush()
}
// The laptop's symptom (2026-10-05), reproduced: its runtime's one large answer reached the console
// after the 750 ms window, and the console said its modules ran nowhere — "nothing in the mesh is
// called slack" — or ran only on another machine. Now a runtime that said it is there is waited for,
// one that never says what it serves is named wherever an address it might answer is asked for, and
// mesh_runtimes shows who answered, how fast and how much.
func TestARuntimeThatAnswersLateIsFoundAndOneThatNeverDoesIsNamed(t *testing.T) {
keptWas, patienceWas := IndexKept, announce.Patience
// Kept for the test: one round of discovery answers every question but mesh_runtimes, which asks anew.
IndexKept, announce.Patience = time.Minute, 2500*time.Millisecond
t.Cleanup(func() { IndexKept, announce.Patience = keptWas, patienceWas })
mt.New(t)
laptop := connect(t, "node-tools", "laptop")
stopTool, err := laptop.HandleSubject("mesh.mod.slack.tool.slack_check.laptop", func(json.RawMessage) (any, error) {
return map[string]any{"ok": true}, nil
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(stopTool)
slowRuntime(t, laptop, "laptop", announce.Window+400*time.Millisecond, []announce.Endpoint{
{Kind: announce.KindTool, Module: "slack", Tool: "slack_check", Node: "laptop", Description: "Is Slack started once?",
Subject: "mesh.mod.slack.tool.slack_check.laptop"},
{Kind: announce.KindTool, Module: "nextcloud-client", Tool: "nextcloud_check", Node: "laptop",
Subject: "mesh.mod.nextcloud-client.tool.nextcloud_check.laptop"},
})
sleeper := connect(t, "node-tools", "sleeper")
slowRuntime(t, sleeper, "sleeper", -1, nil)
desktop := connect(t, "node-tools", "desktop")
slowRuntime(t, desktop, "desktop", 0, []announce.Endpoint{
{Kind: announce.KindTool, Module: "nextcloud-client", Tool: "nextcloud_check", Node: "desktop",
Subject: "mesh.mod.nextcloud-client.tool.nextcloud_check.desktop"},
})
controller := connect(t, "mesh-controller", "bench")
for verb, output := range map[string]string{
"nodes": `[{"name":"laptop"},{"name":"sleeper"},{"name":"desktop"},{"name":"bench"}]`,
"modules": `[{"module":"slack","on":["laptop"],"tools":true},{"module":"nap","on":["sleeper"],"tools":true},` +
`{"module":"nextcloud-client","on":["laptop","desktop","sleeper"],"tools":true}]`,
} {
output := output
stop, err := controller.HandleSubject("mesh.seat.mesh-controller.tool."+verb, func(json.RawMessage) (any, error) {
return map[string]any{"output": output, "ok": true}, nil
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
}
controller.Flush()
console := connect(t, "node-tools", "laptop-console")
up, err := Serve(NewSurface(console, "laptop.node-tools"), "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
defer up.Close()
endpoint := "http://" + up.Address + "/mcp"
// The late answer is waited for: the laptop's module is found and called.
if got, isErr := call(t, endpoint, "mesh_call", map[string]any{"address": "laptop/slack.slack_check"}); isErr || !strings.Contains(got, `"ok": true`) {
t.Errorf("a late runtime's module: %s", got)
}
if got, isErr := call(t, endpoint, "mesh_call", map[string]any{"address": "laptop/nextcloud-client.nextcloud_check"}); isErr && strings.Contains(got, "does not run on laptop") {
t.Errorf("a late runtime's module was said to run elsewhere: %s", got)
}
machine, _ := call(t, endpoint, "mesh_machine", map[string]any{"node": "laptop"})
if !strings.Contains(machine, "laptop/slack.slack_check") {
t.Errorf("the late runtime's machine lists nothing: %s", machine)
}
// The silent one is named, never called missing.
for address, want := range map[string]string{
"sleeper/nap.doze": "nap is assigned to sleeper in the mesh's records, but nothing that answered discovery serves it: discovery did not hear in full from node-tools on sleeper (it answered PING",
"sleeper/nextcloud-client.nextcloud_check": "nextcloud-client on sleeper did not answer discovery: discovery did not hear in full from node-tools on sleeper",
} {
got, isErr := call(t, endpoint, "mesh_call", map[string]any{"address": address})
if !isErr || !strings.Contains(got, want) || strings.Contains(got, "nothing in the mesh is called") || strings.Contains(got, "does not run on") {
t.Errorf("%s: %s\nwant %q", address, got, want)
}
}
machine, _ = call(t, endpoint, "mesh_machine", map[string]any{"node": "sleeper"})
if !strings.Contains(machine, `"incomplete"`) || !strings.Contains(machine, `"modules": []`) {
t.Errorf("a machine whose runtime did not answer: %s", machine)
}
overview, _ := call(t, endpoint, "mesh_overview", nil)
if !strings.Contains(overview, "the runtime node-tools on sleeper") {
t.Errorf("overview does not name the silent runtime: %s", overview)
}
// mesh_runtimes: who answered, how fast, how much; who did not.
got, isErr := call(t, endpoint, "mesh_runtimes", nil)
if isErr {
t.Fatal(got)
}
var report struct {
Answered []map[string]any `json:"answered"`
NotHeard []map[string]any `json:"not heard"`
}
if err := json.Unmarshal([]byte(got), &report); err != nil {
t.Fatalf("%v: %s", err, got)
}
var lap map[string]any
for _, r := range report.Answered {
if r["machine"] == "laptop" {
lap = r
}
}
if lap == nil {
t.Fatalf("the laptop's runtime is not listed as answering: %s", got)
}
took, _ := time.ParseDuration(lap["took"].(string))
if took < announce.Window || lap["bytes"].(float64) <= 0 || lap["modules"].(float64) != 2 || lap["tools"].(float64) != 2 ||
lap["shortened"] != false || lap["last heard"] == nil {
t.Errorf("the laptop's runtime: %v", lap)
}
if !strings.Contains(got, `"machine": "sleeper"`) || !strings.Contains(got, "answered PING") {
t.Errorf("the silent runtime is not listed: %s", got)
}
silent := false
for _, r := range report.NotHeard {
silent = silent || r["machine"] == "sleeper"
}
if !silent {
t.Errorf("sleeper is not among the not heard: %s", got)
}
}
// A runtime heard once and silent now is named in the answers about its machine.
func TestARuntimeHeardBeforeAndNotNowIsNamed(t *testing.T) {
s := &Surface{}
before := &index{Modules: map[string]*moduleInfo{}, Recorded: map[string][]string{}, Discovery: announce.Discovery{At: time.Now().UTC(),
Heard: []announce.Heard{{Info: micro.Info{ServiceIdentity: micro.ServiceIdentity{Name: "node-tools", ID: "laptop",
Metadata: map[string]string{"node": "laptop"}}}, Took: 300 * time.Millisecond, Bytes: 1000}}}}
s.remember(before)
now := &index{Modules: map[string]*moduleInfo{"nextcloud-client": {Module: "nextcloud-client", On: []string{"desktop"},
Tools: []Tool{{Module: "nextcloud-client", Name: "nextcloud_check"}}}}, Recorded: map[string][]string{},
Discovery: announce.Discovery{At: time.Now().UTC()}}
s.remember(now)
if len(now.Unheard) != 1 || now.Unheard[0].Machine != "laptop" {
t.Fatalf("unheard: %+v", now.Unheard)
}
_, err := resolve(now, "laptop/nextcloud-client.nextcloud_check")
if err == nil || !strings.Contains(err.Error(), "nextcloud-client on laptop did not answer discovery") ||
!strings.Contains(err.Error(), "answered discovery") {
t.Errorf("%v", err)
}
_, err = resolve(now, "laptop/slack.slack_check")
if err == nil || strings.Contains(err.Error(), "nothing in the mesh is called") || !strings.Contains(err.Error(), "slack may be theirs") {
t.Errorf("%v", err)
}
// Nobody unheard: the plain answers stand.
quiet := &index{Modules: now.Modules, Recorded: map[string][]string{}}
if _, err := resolve(quiet, "laptop/nextcloud-client.nextcloud_check"); err == nil || !strings.Contains(err.Error(), "does not run on laptop; it runs on desktop") {
t.Errorf("%v", err)
}
if _, err := resolve(quiet, "laptop/slack.slack_check"); err == nil || !strings.Contains(err.Error(), "nothing in the mesh is called slack") {
t.Errorf("%v", err)
}
}
// A runtime restarted answers under a new instance id; its old instance is gone, not unheard.
func TestARestartedRuntimeIsNotNamedUnheard(t *testing.T) {
heard := func(id string) *index {
return &index{Modules: map[string]*moduleInfo{}, Recorded: map[string][]string{}, Discovery: announce.Discovery{At: time.Now().UTC(),
Heard: []announce.Heard{{Info: micro.Info{ServiceIdentity: micro.ServiceIdentity{Name: "node-tools", ID: id,
Metadata: map[string]string{"node": "laptop"}}}}}}}
}
s := &Surface{}
s.remember(heard("before-restart"))
after := heard("after-restart")
s.remember(after)
if len(after.Unheard) != 0 {
t.Errorf("a restarted runtime was named unheard: %+v", after.Unheard)
}
if len(s.runtimes) != 1 {
t.Errorf("the old instance is still kept: %d runtimes", len(s.runtimes))
}
}
@@ -0,0 +1,89 @@
package launch
import (
"encoding/json"
"errors"
"os"
"path/filepath"
"sync"
"testing"
"time"
)
// A bundle that answers `mesh/event` as $ANSWER says: an error, a result, or an answer with neither.
const answeringBundle = `#!/bin/sh
while IFS= read -r line; do
id=$(printf '%s' "$line" | sed -n 's/.*"id":\([0-9]*\)[,}].*/\1/p')
case "$line" in
*'"method":"initialize"'*) printf '{"jsonrpc":"2.0","id":%s,"result":{}}\n' "$id" ;;
*'"method":"tools/list"'*)
printf '{"jsonrpc":"2.0","id":%s,"result":{"tools":[]}}\n' "$id"
printf '{"jsonrpc":"2.0","id":"s1","method":"mesh/subscribe","params":{"pattern":"#"}}\n' ;;
*'"method":"mesh/event"'*)
case "$ANSWER" in
error) printf '{"jsonrpc":"2.0","id":%s,"error":{"code":-32000,"message":"Unexpected end of JSON input"}}\n' "$id" ;;
result) printf '{"jsonrpc":"2.0","id":%s,"result":{}}\n' "$id" ;;
bare) printf '{"jsonrpc":"2.0","id":%s}\n' "$id" ;;
esac ;;
esac
done
`
type subscribing struct {
mu sync.Mutex
deliver func(json.RawMessage) error
}
func (b *subscribing) Publish(json.RawMessage) error { return nil }
func (b *subscribing) Ask(json.RawMessage) (json.RawMessage, error) { return nil, nil }
func (b *subscribing) State(string, json.RawMessage) (json.RawMessage, error) { return nil, nil }
func (b *subscribing) Watch(json.RawMessage, func(json.RawMessage) error) (func(), error) {
return func() {}, nil
}
func (b *subscribing) Subscribe(d func(json.RawMessage) error) error {
b.mu.Lock()
defer b.mu.Unlock()
b.deliver = d
return nil
}
func deliverTo(t *testing.T, answer string) error {
t.Helper()
entry := filepath.Join(t.TempDir(), "bundle")
if err := os.WriteFile(entry, []byte(answeringBundle), 0o755); err != nil {
t.Fatal(err)
}
b := &subscribing{}
l, err := Start("plex", entry, append(os.Environ(), "ANSWER="+answer), b, t.Logf)
if err != nil {
t.Fatal(err)
}
t.Cleanup(l.Stop)
for i := 0; ; i++ {
b.mu.Lock()
d := b.deliver
b.mu.Unlock()
if d != nil {
return d(json.RawMessage(`{"key":"radarr.download.completed","body":{"title":"x"}}`))
}
if i > 100 {
t.Fatal("the bundle never subscribed")
}
time.Sleep(20 * time.Millisecond)
}
}
// An event is taken by any answer that is not an error, and not taken by an error, which reaches the
// runtime as the bundle's own words — not as something the runtime failed to read (hq issue 276).
func TestAnEventIsTakenByAnyAnswerThatIsNotAnError(t *testing.T) {
for _, answer := range []string{"result", "bare"} {
if err := deliverTo(t, answer); err != nil {
t.Errorf("an answer %q did not take the event: %v", answer, err)
}
}
err := deliverTo(t, "error")
var refused *Refused
if !errors.As(err, &refused) || refused.Method != "mesh/event" || refused.Message != "Unexpected end of JSON input" {
t.Fatalf("an error answer was not the bundle's refusal of mesh/event: %#v", err)
}
}
+563
View File
@@ -0,0 +1,563 @@
// Package launch starts a bundle the runtime serves and speaks MCP over stdio to it (novox/hq ADR
// 0188, ADR 0193): `initialize`, `tools/list` once, `tools/call` per call. A Go binary, a Python
// script and a Node launcher are the same thing here: an executable that answers those. The runtime
// knows no language; it starts the path it is given.
package launch
import (
"bufio"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"os/exec"
"regexp"
"strings"
"sync"
"syscall"
"time"
"golang.org/x/sys/unix"
"github.com/novox/mesh-tools/node-tools/internal/wire"
)
// Protocol is the MCP version spoken to a bundle.
const Protocol = "2025-03-26"
// How long a child has for its handshake, and a call before the caller is told it is slow.
var (
HandshakeTimeout = 10 * time.Second
CallTimeout = 30 * time.Second
)
// Tool is one tool a launched bundle listed, and how to call it.
type Tool struct {
Name string
Description string
Input json.RawMessage
Run func(args json.RawMessage) (json.RawMessage, error)
}
// Registration is the tools a bundle listed under one name: its module's, or a seat's.
type Registration struct {
Module string
Tools []Tool
}
// Bus is what a launched bundle reaches the mesh through (novox/hq ADR 0193, ADR 0198): the runtime
// publishes, asks and subscribes on the module's behalf. Subscribe is asked each time the module's
// code subscribes; the runtime binds the module's consumer once and calls deliver for every event,
// acknowledging it on the bus when deliver returns nil.
//
// State and Watch reach the module's state (novox/hq ADR 0201): State answers `get`, `put`, `delete`
// and `keys`; Watch hands deliver the current values and then every change, and returns once the
// current values are delivered — the watch is the child's, and stops when the child does.
type Bus interface {
Publish(params json.RawMessage) error
Ask(params json.RawMessage) (json.RawMessage, error)
Subscribe(deliver func(envelope json.RawMessage) error) error
State(verb string, params json.RawMessage) (json.RawMessage, error)
Watch(params json.RawMessage, deliver func(change json.RawMessage) error) (stop func(), err error)
}
// EventTimeout bounds how long a bundle has to handle one event before it is offered again.
var EventTimeout = 2 * time.Minute
// Refused is a bundle's error answer to what the runtime asked it: the bundle's own words, as opposed
// to the runtime not reaching it (not running, exited, no answer in time). For `mesh/event` it is the
// one way a bundle says it did not take an event: any answer that is not an error — `{}`, `null`, no
// result at all — takes it (mesh-sdk, "Taking an event"; novox/hq issue 276).
type Refused struct {
Method string
Message string
}
func (r *Refused) Error() string { return r.Message }
// Restart backoff: a bundle that exits is started again at once, then after growing pauses while it
// keeps exiting, back to at once once it has run a while.
var (
RestartFirst = 500 * time.Millisecond
RestartMost = 30 * time.Second
RestartSettle = time.Minute
)
// Launched is a running bundle: what it registered, and how to stop it.
type Launched struct {
Registrations []Registration
Stop func()
}
// Executable says whether an entrypoint can be started: a bundle the runtime serves is executable,
// and one that is not was not built to be served (ADR 0193).
func Executable(path string) bool {
info, err := os.Stat(path)
if err != nil || info.IsDir() {
return false
}
return info.Mode().Perm()&0o111 != 0
}
type message struct {
JSONRPC string `json:"jsonrpc,omitempty"`
ID json.RawMessage `json:"id,omitempty"`
Method string `json:"method,omitempty"`
Params json.RawMessage `json:"params,omitempty"`
Result json.RawMessage `json:"result,omitempty"`
Error *struct {
Code int `json:"code"`
Message string `json:"message"`
} `json:"error,omitempty"`
}
type child struct {
cmd *exec.Cmd
stdin io.WriteCloser
writeMu sync.Mutex
mu sync.Mutex
pending map[int64]chan message
next int64
dead chan struct{}
why error
// watches are the state watches this child asked for, stopped when it exits (ADR 0201): a child
// that starts again watches again, as its code runs again.
watches []func()
}
func (c *child) write(m any) error {
b, err := wire.Marshal(m)
if err != nil {
return err
}
c.writeMu.Lock()
defer c.writeMu.Unlock()
_, err = c.stdin.Write(append(b, '\n'))
return err
}
var stackLine = regexp.MustCompile(`^\s+at\s`)
// Start launches one bundle and learns its tools. It fails when the child cannot be started or does
// not complete the handshake. A child that exits later is started again on its next call.
func Start(module, entry string, env []string, mesh Bus, logf func(string, ...any)) (*Launched, error) {
var mu sync.Mutex
var current *child
// The child that last subscribed is the one events are handed to: a restarted child subscribes
// again as its code is imported, and from then on the module's events are its.
var subscriber *child
stopped := false
deliver := func(envelope json.RawMessage) error {
mu.Lock()
c := subscriber
mu.Unlock()
if c == nil {
return errors.New(module + "'s bundle is not running to take its events")
}
// Taken unless the answer is an error; what a result says is not read (see Refused).
_, err := c.ask(module, "mesh/event", map[string]any{"envelope": envelope}, EventTimeout)
return err
}
var restart func(after time.Duration)
var startedAt time.Time
pause := RestartFirst
start := func() (*child, error) {
cmd := exec.Command(entry)
cmd.Env = env
stdin, err := cmd.StdinPipe()
if err != nil {
return nil, err
}
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, err
}
stderr, err := cmd.StderrPipe()
if err != nil {
return nil, err
}
if err := cmd.Start(); err != nil {
return nil, err
}
c := &child{cmd: cmd, stdin: stdin, pending: map[int64]chan message{}, next: 1, dead: make(chan struct{})}
var lastSaid string
var saidMu sync.Mutex
stderrDone := make(chan struct{})
go func() {
defer close(stderrDone)
scan := bufio.NewScanner(stderr)
scan.Buffer(make([]byte, 64*1024), 1<<20)
for scan.Scan() {
line := scan.Text()
if strings.TrimSpace(line) == "" {
continue
}
logf("[%s] %s", module, line)
if !stackLine.MatchString(line) && !strings.HasPrefix(line, "Node.js v") {
saidMu.Lock()
lastSaid = strings.TrimSpace(line)
saidMu.Unlock()
}
}
}()
go func() {
scan := bufio.NewScanner(stdout)
scan.Buffer(make([]byte, 64*1024), 16<<20)
for scan.Scan() {
line := strings.TrimSpace(scan.Text())
if line == "" {
continue
}
var m message
if err := json.Unmarshal([]byte(line), &m); err != nil {
if len(line) > 120 {
line = line[:120]
}
logf("[mesh-tools] %s's bundle said something that is not a reply: %s", module, line)
continue
}
// The bundle asks the runtime (ADR 0193, ADR 0198): to emit, to call a tool, to hand it the
// module's events. Each on the module's behalf, answered when done; nothing else.
if m.Method != "" {
go func(m message) {
var result any = map[string]any{}
var err error
switch m.Method {
case "mesh/publish":
err = mesh.Publish(m.Params)
case "mesh/ask":
var asked json.RawMessage
if asked, err = mesh.Ask(m.Params); err == nil {
result = asked
}
case "mesh/subscribe":
mu.Lock()
subscriber = c
mu.Unlock()
err = mesh.Subscribe(deliver)
case "mesh/state.get", "mesh/state.put", "mesh/state.delete", "mesh/state.keys":
var answered json.RawMessage
if answered, err = mesh.State(strings.TrimPrefix(m.Method, "mesh/state."), m.Params); err == nil {
result = answered
}
case "mesh/state.watch":
// Each change is asked of this child, in order, naming the watch it is for by
// the id the child asked it with — two watches of one state are two handlers;
// the watch is answered once the current values have been handed over, so a
// bundle that awaits it has the whole of the state before it goes on (ADR 0201).
var stop func()
watch := m.ID
stop, err = mesh.Watch(m.Params, func(change json.RawMessage) error {
_, err := c.ask(module, "mesh/state", map[string]any{"watch": watch, "change": change}, EventTimeout)
return err
})
if err == nil {
c.mu.Lock()
if c.why != nil {
c.mu.Unlock()
stop()
} else {
c.watches = append(c.watches, stop)
c.mu.Unlock()
}
}
default:
if len(m.ID) > 0 {
_ = c.write(map[string]any{"jsonrpc": "2.0", "id": m.ID,
"error": map[string]any{"code": -32601, "message": "the runtime answers no " + m.Method + " from a bundle"}})
}
return
}
if len(m.ID) == 0 {
return
}
if err != nil {
_ = c.write(map[string]any{"jsonrpc": "2.0", "id": m.ID,
"error": map[string]any{"code": -32000, "message": err.Error()}})
return
}
_ = c.write(map[string]any{"jsonrpc": "2.0", "id": m.ID, "result": result})
}(m)
continue
}
var id int64
if json.Unmarshal(m.ID, &id) != nil {
continue
}
c.mu.Lock()
ch := c.pending[id]
delete(c.pending, id)
c.mu.Unlock()
if ch != nil {
ch <- m
}
}
<-stderrDone
err := cmd.Wait()
code := "0"
if exit := (*exec.ExitError)(nil); errors.As(err, &exit) {
if status, ok := exit.Sys().(syscall.WaitStatus); ok && status.Signaled() {
code = unix.SignalName(status.Signal())
} else {
code = fmt.Sprint(exit.ExitCode())
}
}
saidMu.Lock()
why := fmt.Sprintf("%s's bundle exited (%s)", module, code)
if lastSaid != "" {
why += ": " + lastSaid
}
saidMu.Unlock()
c.mu.Lock()
c.why = errors.New(why)
watches := c.watches
c.watches = nil
c.mu.Unlock()
for _, stop := range watches {
stop()
}
close(c.dead)
mu.Lock()
// Only a child that was serving is brought back; one that died in its own handshake was
// never accepted, and whoever started it was told why.
wasServing := current == c
if current == c {
current = nil
}
if subscriber == c {
subscriber = nil
}
wasStopped := stopped
// Started again at once if it ran a while; after a growing pause while it keeps exiting.
wait := RestartFirst
if time.Since(startedAt) < RestartSettle {
wait = pause
if pause *= 2; pause > RestartMost {
pause = RestartMost
}
} else {
pause = RestartFirst
}
mu.Unlock()
if !wasStopped && wasServing {
// Every bundle stays up (ADR 0198): code that runs long — a handler, a provisioner —
// is not waiting for a call to bring it back, and a tool bundle back early costs nothing.
logf("[mesh-tools] %s; started again in %s", why, wait.Round(100*time.Millisecond))
restart(wait)
}
}()
if _, err := c.ask(module, "initialize", map[string]any{"protocolVersion": Protocol, "capabilities": map[string]any{},
"clientInfo": map[string]any{"name": "node-tools", "version": "1"}}, HandshakeTimeout); err != nil {
_ = cmd.Process.Kill()
return nil, err
}
_ = c.write(map[string]any{"jsonrpc": "2.0", "method": "notifications/initialized"})
return c, nil
}
asking := func(method string, params any, timeout time.Duration) (json.RawMessage, error) {
mu.Lock()
c := current
mu.Unlock()
if c == nil {
fresh, err := start()
if err != nil {
return nil, err
}
mu.Lock()
if current == nil {
current = fresh
startedAt = time.Now()
} else {
_ = fresh.cmd.Process.Signal(syscall.SIGTERM)
fresh = current
}
c = fresh
mu.Unlock()
}
return c.ask(module, method, params, timeout)
}
restart = func(after time.Duration) {
go func() {
time.Sleep(after)
mu.Lock()
if stopped || current != nil {
mu.Unlock()
return
}
mu.Unlock()
fresh, err := start()
mu.Lock()
defer mu.Unlock()
if err != nil {
if !stopped {
wait := pause
if pause *= 2; pause > RestartMost {
pause = RestartMost
}
logf("[mesh-tools] %s's bundle did not start again: %v; trying in %s", module, err, wait)
go restart(wait)
}
return
}
if stopped {
_ = fresh.cmd.Process.Signal(syscall.SIGTERM)
return
}
if current == nil {
current = fresh
startedAt = time.Now()
} else {
_ = fresh.cmd.Process.Signal(syscall.SIGTERM)
}
}()
}
first, err := start()
if err != nil {
return nil, err
}
mu.Lock()
current = first
startedAt = time.Now()
mu.Unlock()
raw, err := asking("tools/list", map[string]any{}, HandshakeTimeout)
if err != nil {
return nil, err
}
var listed struct {
Tools []struct {
Name string `json:"name"`
Description string `json:"description"`
InputSchema json.RawMessage `json:"inputSchema"`
} `json:"tools"`
}
if err := json.Unmarshal(raw, &listed); err != nil {
return nil, fmt.Errorf("%s's bundle listed its tools in a shape that is not MCP's: %w", module, err)
}
order := []string{}
groups := map[string][]Tool{}
for _, t := range listed.Tools {
under, name := module, t.Name
if dot := strings.Index(t.Name, "."); dot >= 0 {
under, name = t.Name[:dot], t.Name[dot+1:]
}
full := t.Name
input := t.InputSchema
if len(input) == 0 || string(input) == "null" {
input = json.RawMessage("{}")
}
tool := Tool{Name: name, Description: t.Description, Input: input,
Run: func(args json.RawMessage) (json.RawMessage, error) {
if len(args) == 0 || string(args) == "null" {
args = json.RawMessage("{}")
}
res, err := asking("tools/call", map[string]any{"name": full, "arguments": args}, CallTimeout)
if err != nil {
return nil, err
}
var called struct {
Content []struct {
Type string `json:"type"`
Text string `json:"text"`
} `json:"content"`
IsError bool `json:"isError"`
}
_ = json.Unmarshal(res, &called)
text := ""
for _, c := range called.Content {
if c.Type == "text" {
text = c.Text
break
}
}
if called.IsError {
if text == "" {
text = module + "." + full + " failed"
}
return nil, errors.New(text)
}
// The bundle's answer is JSON as text; handed back as the value it encodes.
if json.Valid([]byte(text)) && text != "" {
return json.RawMessage(text), nil
}
b, _ := json.Marshal(text)
return b, nil
}}
if _, seen := groups[under]; !seen {
order = append(order, under)
}
groups[under] = append(groups[under], tool)
}
out := &Launched{Stop: func() {
mu.Lock()
stopped = true
c := current
current = nil
mu.Unlock()
if c != nil && c.cmd.Process != nil {
_ = c.cmd.Process.Signal(syscall.SIGTERM)
}
}}
for _, under := range order {
out.Registrations = append(out.Registrations, Registration{Module: under, Tools: groups[under]})
}
return out, nil
}
func (c *child) ask(module, method string, params any, timeout time.Duration) (json.RawMessage, error) {
c.mu.Lock()
if c.why != nil {
c.mu.Unlock()
return nil, c.why
}
id := c.next
c.next++
ch := make(chan message, 1)
c.pending[id] = ch
c.mu.Unlock()
if err := c.write(map[string]any{"jsonrpc": "2.0", "id": id, "method": method, "params": params}); err != nil {
c.mu.Lock()
delete(c.pending, id)
c.mu.Unlock()
select {
case <-c.dead:
return nil, c.deathReason()
case <-time.After(100 * time.Millisecond):
return nil, err
}
}
timer := time.NewTimer(timeout)
defer timer.Stop()
select {
case m := <-ch:
if m.Error != nil {
msg := m.Error.Message
if msg == "" {
msg = "the bundle refused the request"
}
return nil, &Refused{Method: method, Message: msg}
}
return m.Result, nil
case <-c.dead:
return nil, c.deathReason()
case <-timer.C:
c.mu.Lock()
delete(c.pending, id)
c.mu.Unlock()
return nil, fmt.Errorf("%s's bundle did not answer %s in %ds", module, method, int(timeout/time.Second))
}
}
func (c *child) deathReason() error {
c.mu.Lock()
defer c.mu.Unlock()
if c.why != nil {
return c.why
}
return errors.New("the bundle exited")
}
+233
View File
@@ -0,0 +1,233 @@
// Package meshtest raises what the controller would, for tests against a real bus: the ASSIGNMENTS
// and EVENTS streams, memberships issued by hand, and a fixture's path.
//
// **Every test has a bus of its own** (novox/hq ADR 0237, the controller's internal/testbus): a server
// linked in at the release go.mod pins — the release the mesh runs, held so by a test beside this one —
// started for the one test and gone after it. The packages shared one bus and had to run one at a time:
// each test raised the streams afresh, and the console discovers every runtime that announces itself on
// the bus (ADR 0197), so a runtime from another package's test was, correctly, found. Now the suite runs
// as Go runs it, in parallel and under the race detector. A person may still point a run at a bus of
// their own with MESH_TEST_NATS_EXTERNAL=1 and MESH_TEST_NATS; then the run is theirs to serialise.
package meshtest
import (
"encoding/json"
"os"
"path/filepath"
"runtime"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats-server/v2/server"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-tools/node-tools/internal/bus"
)
// Version is the release of the server the tests run.
const Version = server.VERSION
// URL is the bus of this test alone: started at its first ask, the same one at every ask after — a test
// that dials twice, or hands the address to the code it tests, reaches one bus — and shut down when the
// test ends.
func URL(t *testing.T) string {
t.Helper()
if os.Getenv("MESH_TEST_NATS_EXTERNAL") == "1" {
if url := os.Getenv("MESH_TEST_NATS"); url != "" {
return url
}
t.Fatal("MESH_TEST_NATS_EXTERNAL=1 and MESH_TEST_NATS names no bus")
}
if url, ok := buses.Load(t); ok {
return url.(string)
}
opts := &server.Options{Host: "127.0.0.1", Port: server.RANDOM_PORT, JetStream: true, StoreDir: t.TempDir(),
NoLog: true, NoSigs: true}
s, err := server.NewServer(opts)
if err != nil {
t.Fatalf("a bus for this test could not be made: %v", err)
}
go s.Start()
if !s.ReadyForConnections(30 * time.Second) {
s.Shutdown()
t.Fatal("a bus for this test did not come up within 30s")
}
url := s.ClientURL()
buses.Store(t, url)
t.Cleanup(func() {
buses.Delete(t)
s.Shutdown()
s.WaitForShutdown()
})
return url
}
// buses are the running tests' buses, by test.
var buses sync.Map
// Mesh is the controller's job, done by hand.
type Mesh struct {
nc *nats.Conn
js nats.JetStreamContext
}
// New raises the streams afresh.
func New(t *testing.T) *Mesh {
t.Helper()
nc, err := nats.Connect(URL(t))
if err != nil {
t.Fatal(err)
}
js, _ := nc.JetStream()
for _, s := range []string{"ASSIGNMENTS", "EVENTS"} {
_ = js.DeleteStream(s)
}
if _, err := js.AddStream(&nats.StreamConfig{Name: "ASSIGNMENTS", Subjects: []string{"mesh.assignment.>"},
MaxMsgsPerSubject: 1, AllowDirect: true}); err != nil {
t.Fatal(err)
}
if _, err := js.AddStream(&nats.StreamConfig{Name: "EVENTS", Subjects: []string{"mesh.mod.*.event.>"}}); err != nil {
t.Fatal(err)
}
t.Cleanup(nc.Close)
return &Mesh{nc: nc, js: js}
}
// Issue publishes a membership.
func (m *Mesh) Issue(t *testing.T, mem bus.Membership) {
t.Helper()
body, _ := json.Marshal(mem)
if _, err := m.js.Publish(bus.MembershipSubject(mem.Node, mem.Module), body); err != nil {
t.Fatal(err)
}
}
// NextEvent is the subject the next event under a pattern lands on.
func (m *Mesh) NextEvent(t *testing.T, pattern string) <-chan string {
t.Helper()
ch := make(chan string, 1)
sub, err := m.nc.Subscribe(pattern, func(msg *nats.Msg) {
select {
case ch <- msg.Subject:
default:
}
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = sub.Unsubscribe() })
_ = m.nc.Flush()
return ch
}
// MembershipOf is a membership as the controller issues one on a machine.
func MembershipOf(module, node string, plain bool, seats map[string][]string) bus.Membership {
own := "mesh.mod." + module
serves := []bus.Served{{Subject: own + ".tool.{tool}." + node}}
if plain {
serves = append(serves, bus.Served{Subject: own + ".tool.{tool}", Queue: "serve." + module})
}
var verbs []bus.SeatVerb
for seat, vs := range seats {
for _, v := range vs {
verbs = append(verbs, bus.SeatVerb{Seat: seat, Verb: v, Subject: "mesh.seat." + seat + ".tool." + v + "." + node})
}
}
return bus.Membership{Node: node, Module: module, Serves: serves, Seats: verbs,
Emits: own + ".event.{event}", Tools: own + ".tool.tools"}
}
// Fixture is a path in node-tools/test/fixtures, which these tests share with the TypeScript ones.
func Fixture(name string) string {
_, here, _, _ := runtime.Caller(0)
return filepath.Join(filepath.Dir(here), "..", "..", "test", "fixtures", name)
}
// Until retries while the bus answers "no responders" — something not yet served.
func Until(t *testing.T, try func() error) {
t.Helper()
var err error
for i := 0; i < 50; i++ {
if err = try(); err == nil {
return
}
time.Sleep(100 * time.Millisecond)
}
t.Fatal(err)
}
// Logs collects what the runtime says.
type Logs struct{ lines []string }
// Logf is a logger that keeps the lines.
func (l *Logs) Logf(format string, args ...any) {
l.lines = append(l.lines, sprintf(format, args...))
}
// Has says whether a line contains every fragment.
func (l *Logs) Has(fragments ...string) bool {
for _, line := range l.lines {
all := true
for _, f := range fragments {
all = all && strings.Contains(line, f)
}
if all {
return true
}
}
return false
}
// All is every line.
func (l *Logs) All() string { return strings.Join(l.lines, "\n") }
// Consumer makes a module's durable consumer on a machine as the controller does: pull, on the
// EVENTS stream, filtered to what the module consumes, with a short ack wait so a test sees a
// redelivery in seconds rather than the mesh's minutes.
func (m *Mesh) Consumer(t *testing.T, node, module string, filters []string, ackWait time.Duration) {
t.Helper()
cfg := &nats.ConsumerConfig{Durable: node + "_" + module, AckPolicy: nats.AckExplicitPolicy,
AckWait: ackWait, MaxDeliver: 10, DeliverPolicy: nats.DeliverNewPolicy}
if len(filters) == 1 {
cfg.FilterSubject = filters[0]
} else {
cfg.FilterSubjects = filters
}
if _, err := m.js.AddConsumer("EVENTS", cfg); err != nil {
t.Fatal(err)
}
}
// Emit publishes an event as a module would, into the EVENTS stream.
func (m *Mesh) Emit(t *testing.T, subject string, body any) {
t.Helper()
data, _ := json.Marshal(body)
if _, err := m.js.Publish(subject, data); err != nil {
t.Fatal(err)
}
}
// Pending is what a module's consumer still holds: delivered and not acknowledged, and not yet
// delivered.
func (m *Mesh) Pending(t *testing.T, node, module string) (ackPending, notDelivered uint64) {
t.Helper()
info, err := m.js.ConsumerInfo("EVENTS", node+"_"+module)
if err != nil {
t.Fatal(err)
}
return uint64(info.NumAckPending), info.NumPending
}
// Bucket makes a module's state afresh as the controller does from the catalogue (novox/hq ADR 0201),
// and answers it for writing what is there before a bundle starts.
func (m *Mesh) Bucket(t *testing.T, name string) nats.KeyValue {
t.Helper()
_ = m.js.DeleteKeyValue(name)
kv, err := m.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: name, History: 1, MaxValueSize: 256 * 1024})
if err != nil {
t.Fatal(err)
}
return kv
}
@@ -0,0 +1,75 @@
package meshtest
import (
"encoding/json"
"os"
"path/filepath"
"regexp"
"strings"
"testing"
)
// **The bus the tests run is the bus the mesh runs** (novox/hq ADR 0227 rule 9, ADR 0237): the server linked
// into the tests is held to the release the catalogue's bus image is — read from beside this checkout, as
// the build seat clones it for a merge check — and, given the facts snapshot, to the release the mesh's bus
// server says it runs. A bus upgrade moves the catalogue's pin; this then fails until go.mod's moves with it.
func TestTheBusTheTestsRunIsTheBusTheMeshRuns(t *testing.T) {
beside := os.Getenv("MESH_CHECK_BESIDE")
if beside == "" {
beside = filepath.Join("..", "..", "..", "..")
}
dockerfile := filepath.Join(beside, "mesh-catalog", "modules", "nats", "Dockerfile")
raw, err := os.ReadFile(dockerfile)
switch {
case err == nil:
pinned := regexp.MustCompile(`upstream: nats ([0-9.]+)-alpine`).FindSubmatch(raw)
if pinned == nil {
t.Fatalf("%s says no release its digest is", dockerfile)
}
if string(pinned[1]) != Version {
t.Errorf("the tests run bus %s and the catalogue's bus image is %s: move go.mod's nats-server pin with the image",
Version, pinned[1])
}
case os.IsNotExist(err):
t.Logf("the catalogue is not beside this checkout, so its bus image is not compared")
default:
t.Fatal(err)
}
path := os.Getenv("MESH_FACTS")
if path == "" {
t.Logf("no MESH_FACTS: the bus the mesh runs is compared in a merge check, which has it")
return
}
body, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
var f struct {
Taken string `json:"taken"`
Versions struct {
Bus string `json:"bus"`
} `json:"versions"`
}
if err := json.Unmarshal(body, &f); err != nil {
t.Fatal(err)
}
if f.Versions.Bus != "" && strings.TrimPrefix(f.Versions.Bus, "v") != Version {
t.Errorf("the tests run bus %s and the mesh's bus runs %s (facts of %s)", Version, f.Versions.Bus, f.Taken)
}
}
// Each test is given a bus of its own, and the same one at every ask.
func TestEachTestHasABusOfItsOwn(t *testing.T) {
var first string
t.Run("one", func(t *testing.T) {
first = URL(t)
if URL(t) != first {
t.Fatal("two buses for one test")
}
})
t.Run("two", func(t *testing.T) {
if URL(t) == first {
t.Fatal("two tests were given one bus")
}
})
}
+5
View File
@@ -0,0 +1,5 @@
package meshtest
import "fmt"
func sprintf(format string, args ...any) string { return fmt.Sprintf(format, args...) }
@@ -0,0 +1,121 @@
package runtime
import (
"encoding/json"
"sort"
"strings"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/micro"
"github.com/novox/mesh-tools/node-tools/internal/bus"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
// novox/hq ADR 0197: what a runtime serves, it announces — the NATS services protocol's discovery,
// read here with NATS's own types, each endpoint a subject actually served — and the announcement
// follows a re-issued membership.
func TestTheRuntimeAnnouncesWhatItServesInTheServicesProtocol(t *testing.T) {
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("alpha", "anchor", false, nil))
mesh.Issue(t, mt.MembershipOf("beta", "anchor", false, map[string][]string{"node-shelf": {"list", "clear"}}))
nodeTools := connect(t, "node-tools", "anchor")
stop, err := Run(nodeTools, []Served{
{"alpha", []string{mt.Fixture("many-alpha.serve.mjs")}},
{"beta", []string{mt.Fixture("many-beta.serve.mjs")}},
}, nil, (&mt.Logs{}).Logf)
if err != nil {
t.Fatal(err)
}
defer stop()
nc, err := nats.Connect(mt.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
info := func() micro.Info {
t.Helper()
msg, err := nc.Request("$SRV.INFO.node-tools.anchor", nil, 2*time.Second)
if err != nil {
t.Fatal(err)
}
var i micro.Info
if err := json.Unmarshal(msg.Data, &i); err != nil {
t.Fatal(err)
}
return i
}
subjects := func(i micro.Info) string {
var out []string
for _, e := range i.Endpoints {
out = append(out, e.Subject+"|"+e.QueueGroup+"|"+e.Metadata["kind"]+"|"+e.Metadata["module"]+"|"+e.Metadata["seat"]+"|"+e.Metadata["scope"]+"|"+e.Metadata["interchangeable"])
}
sort.Strings(out)
return strings.Join(out, "\n")
}
got := info()
if got.Type != micro.InfoResponseType || got.Name != "node-tools" || got.ID != "anchor" || got.Version == "" {
t.Errorf("identity: %+v", got.ServiceIdentity)
}
want := strings.Join([]string{
"mesh.mod.alpha.tool.one.anchor||tool|alpha|||false",
"mesh.mod.alpha.tool.two.anchor||tool|alpha|||false",
"mesh.mod.beta.tool.five.anchor||tool|beta|||false",
"mesh.mod.beta.tool.four.anchor||tool|beta|||false",
"mesh.mod.beta.tool.three.anchor||tool|beta|||false",
"mesh.seat.node-shelf.tool.clear.anchor||seat|beta|node-shelf|node|false",
"mesh.seat.node-shelf.tool.list.anchor||seat|beta|node-shelf|node|false",
}, "\n")
if s := subjects(got); s != want {
t.Errorf("announced:\n%s\nwant:\n%s", s, want)
}
for _, e := range got.Endpoints {
if e.Metadata["node"] != "anchor" || e.Metadata["description"] == "" {
t.Errorf("endpoint metadata: %+v", e)
}
// A seat's verb carries its schema; a module's tool does not — it outgrew the bus (2026-10-04).
_, hasSchema := e.Metadata["schema"]
if e.Metadata["kind"] == "seat" && !json.Valid([]byte(e.Metadata["schema"])) {
t.Errorf("a seat's verb without a schema: %+v", e)
}
if e.Metadata["kind"] == "tool" && hasSchema {
t.Errorf("a module's tool still announces its schema: %+v", e)
}
}
// Every subject announced is answered.
asker := connect(t, "console", "workstation")
for _, e := range got.Endpoints {
if _, err := asker.Ask("", map[string]any{}, e.Subject); err != nil {
t.Errorf("announced %s and does not answer it: %v", e.Subject, err)
}
}
// PING answers with the same identity, and a request for another service is not answered.
if msg, err := nc.Request("$SRV.PING", nil, 2*time.Second); err != nil || !strings.Contains(string(msg.Data), micro.PingResponseType) {
t.Errorf("ping: %v %v", msg, err)
}
if _, err := nc.Request("$SRV.INFO.somebody-else", nil, 300*time.Millisecond); err == nil {
t.Error("answered a request for another service")
}
// alpha is re-issued a plain subject: the announcement says so, and says it is interchangeable.
mesh.Issue(t, mt.MembershipOf("alpha", "anchor", true, nil))
var after string
for i := 0; i < 40; i++ {
after = subjects(info())
if strings.Contains(after, "mesh.mod.alpha.tool.one|serve.alpha|tool|alpha|||true") {
break
}
time.Sleep(100 * time.Millisecond)
}
if !strings.Contains(after, "mesh.mod.alpha.tool.one|serve.alpha|tool|alpha|||true") ||
!strings.Contains(after, "mesh.mod.alpha.tool.one.anchor||tool|alpha|||true") {
t.Errorf("after a re-issued membership:\n%s", after)
}
_ = bus.Served{}
}
+157
View File
@@ -0,0 +1,157 @@
package runtime
import (
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/launch"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
// A logger safe to write from the runtime's goroutines.
type lines struct {
mu sync.Mutex
l []string
}
func (s *lines) logf(format string, args ...any) {
s.mu.Lock()
defer s.mu.Unlock()
s.l = append(s.l, strings.TrimSpace(sprintf(format, args...)))
}
func (s *lines) all() string {
s.mu.Lock()
defer s.mu.Unlock()
return strings.Join(s.l, "\n")
}
func sprintf(format string, args ...any) string { return fmtSprintf(format, args...) }
func countLines(t *testing.T, path, want string) int {
t.Helper()
b, _ := os.ReadFile(path)
n := 0
for _, l := range strings.Split(string(b), "\n") {
if l == want {
n++
}
}
return n
}
// novox/hq ADR 0198: a module's long-running code is launched by the node's runtime, and the runtime
// is its bus — it binds the module's own consumer, hands each event to the bundle, and acknowledges it
// only when the bundle has taken it; one the bundle failed or died on is offered again.
func TestTheRuntimeHandsAModulesEventsToItsBundleAndAcknowledgesThemOnlyWhenTaken(t *testing.T) {
mesh := mt.New(t)
restore := bus.NakDelay
bus.NakDelay = 300 * time.Millisecond
t.Cleanup(func() { bus.NakDelay = restore })
firstPause := launch.RestartFirst
launch.RestartFirst = 100 * time.Millisecond
t.Cleanup(func() { launch.RestartFirst = firstPause })
mesh.Issue(t, mt.MembershipOf("watcher", "anchor", false, nil))
mesh.Issue(t, mt.MembershipOf("beta", "anchor", true, nil))
// The controller's consumer for the module, as its own runtime bound it: anchor_watcher on EVENTS.
mesh.Consumer(t, "anchor", "watcher", []string{"mesh.mod.alpha.event.>"}, 2*time.Second)
nodeTools := connect(t, "node-tools", "anchor")
asker := connect(t, "console", "workstation")
dir := t.TempDir()
log := filepath.Join(dir, "watch.log")
said := &lines{}
stop, err := Run(nodeTools, []Served{
{Module: "watcher", Entrypoints: []string{mt.Fixture("watcher.serve.mjs")}},
{Module: "beta", Entrypoints: []string{mt.Fixture("many-beta.serve.mjs")}},
}, map[string]map[string]string{"watcher": {"WATCH_LOG": log}}, said.logf)
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
if !strings.Contains(said.all(), "watcher's events arrive on its consumer anchor_watcher") {
t.Fatalf("the module's consumer was not bound:\n%s", said.all())
}
// Handled once, acknowledged once.
mesh.Emit(t, "mesh.mod.alpha.event.happened", map[string]any{"n": 1})
mt.Until(t, func() error {
if countLines(t, log, "handled 1") != 1 {
return errorf("event 1 not handled yet")
}
return nil
})
// The handler fails the first time: not acknowledged, offered again, then handled.
mesh.Emit(t, "mesh.mod.alpha.event.happened", map[string]any{"n": 2, "fail": true})
// The bundle dies on the first offer: not acknowledged, the bundle is started again and the
// event is offered again once its ack wait has passed.
mesh.Emit(t, "mesh.mod.alpha.event.happened", map[string]any{"n": 3, "die": true})
deadline := time.Now().Add(20 * time.Second)
for time.Now().Before(deadline) {
if countLines(t, log, "handled 2") == 1 && countLines(t, log, "handled 3") == 1 {
break
}
time.Sleep(200 * time.Millisecond)
}
if countLines(t, log, "handled 2") != 1 || countLines(t, log, "handled 3") != 1 {
b, _ := os.ReadFile(log)
t.Fatalf("a failed or interrupted event was not offered again and handled once:\n%s\nruntime:\n%s", b, said.all())
}
if countLines(t, log, "started") < 2 {
t.Errorf("the bundle that died was not started again: %s", said.all())
}
mt.Until(t, func() error {
ack, notYet := mesh.Pending(t, "anchor", "watcher")
if ack != 0 || notYet != 0 {
return errorf("still pending: %d unacknowledged, %d undelivered", ack, notYet)
}
return nil
})
if countLines(t, log, "handled 1") != 1 {
t.Errorf("event 1 was handled more than once")
}
// A tool asks another module's tool through the runtime, as the module.
got, err := call(t, asker, "watcher.relay@anchor", map[string]any{})
if err != nil {
t.Fatal(err)
}
same(t, got, `{"beta":3}`)
}
// A long-running bundle that exits is started again, without waiting for a call (ADR 0198).
func TestALongRunningBundleThatExitsIsStartedAgain(t *testing.T) {
mesh := mt.New(t)
firstPause := launch.RestartFirst
launch.RestartFirst = 100 * time.Millisecond
t.Cleanup(func() { launch.RestartFirst = firstPause })
mesh.Issue(t, mt.MembershipOf("flaky", "anchor", false, nil))
nodeTools := connect(t, "node-tools", "anchor")
log := filepath.Join(t.TempDir(), "flaky.log")
said := &lines{}
stop, err := Run(nodeTools, []Served{{Module: "flaky", Entrypoints: []string{mt.Fixture("flaky.serve.mjs")}}},
map[string]map[string]string{"flaky": {"FLAKY_LOG": log}}, said.logf)
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
mt.Until(t, func() error {
if countLines(t, log, "started") < 3 {
return errorf("started %d time(s)", countLines(t, log, "started"))
}
return nil
})
if !strings.Contains(said.all(), "flaky's bundle exited (3)") || !strings.Contains(said.all(), "started again in") {
t.Errorf("the restart was not said:\n%s", said.all())
}
}
func fmtSprintf(format string, args ...any) string { return fmt.Sprintf(format, args...) }
func errorf(format string, args ...any) error { return fmt.Errorf(format, args...) }
+668
View File
@@ -0,0 +1,668 @@
// Package runtime is the node's tool runtime (novox/hq ADR 0175, ADR 0193), the Go port of
// node-tools' runtime.ts in its launch-only form: it launches every assigned module's bundle,
// serves each module's tools on that module's subjects and each held seat's verbs on the seat's,
// and answers for every module the verb that says what it serves. It imports nothing and knows no
// language.
package runtime
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"sync"
"github.com/novox/mesh-tools/node-tools/internal/announce"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/launch"
)
// ToolsVerb is the verb every module's runtime answers for it (ADR 0152): its tools, from the code
// that answers them.
const ToolsVerb = "tools"
// Words the mesh sets for the runtime (ADR 0175, ADR 0192).
const (
ToolModules = "MESH_TOOL_MODULES"
ToolEnv = "MESH_TOOL_ENV"
OperatorAccount = "MESH_OPERATOR_ACCOUNT"
OperatorHome = "MESH_OPERATOR_HOME"
)
// Served is one module this runtime serves and its entrypoints.
type Served struct {
Module string
Entrypoints []string
}
// ToolsAnswer is what `tools` answers for one module.
type ToolsAnswer struct {
Module string `json:"module"`
Tools []ListedTool `json:"tools"`
Failed string `json:"failed,omitempty"`
}
// ListedTool is one tool as a module's `tools` answer lists it.
type ListedTool struct {
Name string `json:"name"`
Description string `json:"description"`
Input json.RawMessage `json:"input"`
Subjects []string `json:"subjects,omitempty"`
}
// ServedModulesFrom reads MESH_TOOL_MODULES: `<module>=<entrypoint>` entries, comma-separated, several
// per module. The one-module form — a bare path, or the runtime's own module — is the per-module
// containers' (to-be 38 WP4c) and refused here: the node's runtime imports nothing.
func ServedModulesFrom(spec, own string) ([]Served, error) {
order := []string{}
by := map[string][]string{}
for _, raw := range strings.Split(spec, ",") {
entry := strings.TrimSpace(raw)
if entry == "" {
continue
}
module, path, ok := strings.Cut(entry, "=")
module, path = strings.TrimSpace(module), strings.TrimSpace(path)
if !ok || module == "" || path == "" || module == own {
return nil, fmt.Errorf("%s: %q is not <module>=<entrypoint> of another module; the node's "+
"runtime launches the bundles it is given and imports nothing (novox/hq ADR 0193)", ToolModules, entry)
}
if _, seen := by[module]; !seen {
order = append(order, module)
}
by[module] = append(by[module], path)
}
out := make([]Served, 0, len(order))
for _, m := range order {
out = append(out, Served{Module: m, Entrypoints: by[m]})
}
return out, nil
}
// TakeToolEnvs reads the composed environments (ADR 0192) and removes them from the process's, so no
// bundle finds another's there.
func TakeToolEnvs() (map[string]map[string]string, error) {
raw := os.Getenv(ToolEnv)
os.Unsetenv(ToolEnv)
out := map[string]map[string]string{}
if raw == "" {
return out, nil
}
var parsed map[string]map[string]any
if err := json.Unmarshal([]byte(raw), &parsed); err != nil {
return nil, fmt.Errorf(`%s is not JSON of the shape {"<module>": {"<word>": "<value>"}}: %w`, ToolEnv, err)
}
for module, words := range parsed {
own := map[string]string{}
for k, v := range words {
if s, ok := v.(string); ok {
own[k] = s
} else {
b, _ := json.Marshal(v)
own[k] = string(b)
}
}
out[module] = own
}
return out, nil
}
type registration struct {
module string // the module's own name, or a seat's
owner string // the module whose bundle made it
tools []launch.Tool
}
// Run launches, binds and serves. It answers a stop function.
func Run(conn *bus.Conn, served []Served, envs map[string]map[string]string, logf func(string, ...any)) (func(), error) {
if account := os.Getenv(OperatorAccount); account != "" {
home := ""
if h := os.Getenv(OperatorHome); h != "" {
home = " (home " + h + ")"
}
logf("[mesh-tools] the operator's account here is %s%s", account, home)
}
modules := make([]string, 0, len(served))
isServed := map[string]bool{}
for _, s := range served {
modules = append(modules, s.Module)
isServed[s.Module] = true
conn.Follow(s.Module)
}
node := conn.Node()
base := os.Environ()
envFor := func(module string) []string {
words := map[string]string{}
for _, kv := range base {
if k, v, ok := strings.Cut(kv, "="); ok && k != ToolEnv {
words[k] = v
}
}
for k, v := range envs[module] {
words[k] = v
}
words["MESH_SERVED_MODULE"] = module
words["MESH_MODULE"] = module
if node != "" {
words["MESH_NODE"] = node
}
out := make([]string, 0, len(words))
for k, v := range words {
out = append(out, k+"="+v)
}
sort.Strings(out)
return out
}
// The module's events, for every child of it that subscribes (ADR 0198): one consumer per module,
// bound the first time any of its children subscribes, each event handed to every child that did.
events := &consumers{conn: conn, logf: logf, of: map[string]*moduleEvents{}}
failed := map[string]string{}
var registrations []registration
var stops []func()
for _, s := range served {
for _, entry := range s.Entrypoints {
path, _ := filepath.Abs(entry)
module := s.Module
fail := func(why string) {
failed[module] = why
logf("[mesh-tools] %s's bundle %s failed to load: %s; its tools are not served here", module, entry, why)
}
if !launch.Executable(path) {
fail(path + " is not executable; a bundle the runtime serves is started, never imported, and its build makes it executable (novox/hq ADR 0193)")
continue
}
child, err := launch.Start(module, path, envFor(module), events.forModule(module), logf)
if err != nil {
fail(err.Error())
continue
}
stops = append(stops, child.Stop)
for _, r := range child.Registrations {
registrations = append(registrations, registration{module: r.Module, owner: module, tools: r.Tools})
}
}
}
claimed := map[string]bool{}
for _, m := range modules {
if mem := conn.Membership(m); mem != nil {
for _, s := range mem.Seats {
claimed[s.Seat] = true
}
}
}
var own []registration
for _, r := range registrations {
switch {
case isServed[r.module]:
own = append(own, r)
case claimed[r.module]:
default:
logf(`[mesh-tools] %s registers tools under "%s", which is neither a module served here nor a seat one of them claims; not served until the mesh issues the claim`, r.owner, r.module)
}
}
stopAll := func() {
events.stopAll()
for i := len(stops) - 1; i >= 0; i-- {
stops[i]()
}
}
for _, r := range own {
seen := map[string]bool{}
for _, t := range r.tools {
if t.Name == ToolsVerb {
stopAll()
return nil, fmt.Errorf(`%s names a tool "%s", which is the verb the runtime answers for every module with what it serves (novox/hq ADR 0152) — refused, rename it`, r.module, ToolsVerb)
}
if seen[t.Name] {
stopAll()
return nil, fmt.Errorf("%s exposes two tools named %s — refused", r.module, t.Name)
}
seen[t.Name] = true
}
}
var names []string
byModule := map[string][]launch.Tool{}
for _, r := range own {
for _, t := range r.tools {
t := t
stop, err := conn.Handle(r.module+"."+t.Name, func(body json.RawMessage) (any, error) {
return t.Run(argsOf(body))
})
if err != nil {
logf("[mesh-tools] cannot serve %s.%s: %v", r.module, t.Name, err)
continue
}
names = append(names, r.module+"."+t.Name)
stops = append(stops, stop)
}
byModule[r.module] = append(byModule[r.module], r.tools...)
}
for _, module := range modules {
module := module
tools := byModule[module]
why := failed[module]
if len(tools) == 0 && why == "" {
continue // a pure-events module: silent, as it always was
}
stop, err := conn.Handle(module+"."+ToolsVerb, func(json.RawMessage) (any, error) {
answer := ToolsAnswer{Module: module, Tools: []ListedTool{}, Failed: why}
for _, t := range tools {
answer.Tools = append(answer.Tools, ListedTool{Name: t.Name, Description: t.Description,
Input: t.Input, Subjects: subjectsOf(conn, module, t.Name)})
}
return answer, nil
})
if err == nil {
stops = append(stops, stop)
}
}
failedNames := make([]string, 0, len(failed))
for _, m := range modules {
if _, f := failed[m]; f {
failedNames = append(failedNames, m)
}
}
line := fmt.Sprintf("[mesh-tools] serving %d tool(s) for %d module(s): %s", len(names), len(modules), orNone(names))
if len(failedNames) > 0 {
line += fmt.Sprintf("; not serving %s, whose bundle(s) failed to load", strings.Join(failedNames, ", "))
}
logf("%s", line)
stops = append(stops, serveSeats(conn, modules, registrations, logf))
// **What it serves, it announces** (novox/hq ADR 0197): the NATS services protocol's discovery,
// answered with what is served at the moment it is asked — re-served memberships included.
announced, err := announce.Serve(conn, announce.Service{
Name: conn.Module(), ID: instanceOf(conn),
Description: "the mesh's tool runtime on " + node + ": every assigned module's tools and the seats they hold",
Metadata: map[string]string{"node": node},
}, func() []announce.Endpoint { return endpointsOf(conn, own, registrations, modules) })
if err != nil {
logf("[mesh-tools] cannot announce what it serves: %v", err)
} else {
stops = append(stops, announced)
}
conn.Flush()
return stopAll, nil
}
// instanceOf is this runtime's instance on the bus: its machine, which is what tells two instances of
// one service apart; the connection's module where it has no machine.
func instanceOf(conn *bus.Conn) string {
if n := conn.Node(); n != "" {
return n
}
return conn.Module()
}
// endpointsOf is everything this runtime serves now: each served module's tools on every subject the
// mesh issued for them, and each held seat's verbs on the seat's subject (ADR 0197).
func endpointsOf(conn *bus.Conn, own []registration, registrations []registration, modules []string) []announce.Endpoint {
node := conn.Node()
var out []announce.Endpoint
for _, r := range own {
for _, t := range r.tools {
served := conn.ServedOn(r.module, t.Name)
interchangeable := false
for _, s := range served {
interchangeable = interchangeable || s.Subject == "mesh.mod."+r.module+".tool."+t.Name
}
for _, s := range served {
out = append(out, announce.Endpoint{Kind: announce.KindTool, Module: r.module, Tool: t.Name,
Node: node, Description: t.Description, Schema: t.Input, Interchangeable: interchangeable,
Subject: s.Subject, Queue: s.Queue})
}
}
}
impl := map[string]map[string]launch.Tool{}
for _, r := range registrations {
if impl[r.module] == nil {
impl[r.module] = map[string]launch.Tool{}
}
for _, t := range r.tools {
impl[r.module][t.Name] = t
}
}
have := map[string]bool{}
for _, module := range modules {
m := conn.Membership(module)
if m == nil {
continue
}
for _, v := range m.Seats {
t, ok := impl[v.Seat][v.Verb]
if !ok || have[v.Subject] {
continue
}
have[v.Subject] = true
scope := "mesh"
if node != "" && strings.HasSuffix(v.Subject, "."+node) {
scope = "node"
}
out = append(out, announce.Endpoint{Kind: announce.KindSeat, Module: module, Tool: v.Verb, Seat: v.Seat,
Scope: scope, Node: node, Description: t.Description, Schema: t.Input, Subject: v.Subject})
}
}
return out
}
func orNone(names []string) string {
if len(names) == 0 {
return "(none)"
}
return strings.Join(names, ", ")
}
// argsOf is a call's arguments as the tool receives them: an object, `{}` for none.
func argsOf(body json.RawMessage) json.RawMessage {
trimmed := strings.TrimSpace(string(body))
if trimmed == "" || trimmed == "null" {
return json.RawMessage("{}")
}
return body
}
// subjectsOf is where a tool is answered as the mesh issued it: the plain subject first, then this
// machine's; nothing before a membership is issued.
func subjectsOf(conn *bus.Conn, module, tool string) []string {
m := conn.Membership(module)
if m == nil {
return nil
}
var plain, mine []string
for _, s := range m.Serves {
subject := strings.ReplaceAll(s.Subject, "{tool}", tool)
if s.Queue != "" {
plain = append(plain, subject)
} else {
mine = append(mine, subject)
}
}
return append(plain, mine...)
}
// serveSeats serves every verb of every seat a served module holds, where the mesh issued it, by
// the tool of the same name registered under the seat's name (ADR 0159, 0160) — and serves again
// whenever a membership changes. Whether this machine holds the seat is the bus's to decide.
func serveSeats(conn *bus.Conn, modules []string, registrations []registration, logf func(string, ...any)) func() {
impl := map[string]map[string]launch.Tool{}
for _, r := range registrations {
if impl[r.module] == nil {
impl[r.module] = map[string]launch.Tool{}
}
for _, t := range r.tools {
impl[r.module][t.Name] = t
}
}
var mu sync.Mutex
var stops []func()
serve := func() {
mu.Lock()
defer mu.Unlock()
for _, s := range stops {
s()
}
stops = nil
have := map[string]bool{}
for _, module := range modules {
m := conn.Membership(module)
if m == nil {
continue
}
for _, v := range m.Seats {
if have[v.Subject] {
continue
}
have[v.Subject] = true
t, ok := impl[v.Seat][v.Verb]
if !ok {
logf("[mesh-tools] %s claims %s and implements no %s, which that seat promises; not served", module, v.Seat, v.Verb)
continue
}
stop, err := conn.HandleSubject(v.Subject, func(body json.RawMessage) (any, error) {
return t.Run(argsOf(body))
})
if err != nil {
logf("[mesh-tools] cannot serve %s's %s on %s: %v", v.Seat, v.Verb, v.Subject, err)
continue
}
stops = append(stops, stop)
logf("[mesh-tools] serving %s's %s on %s, admitted where %s holds the seat", v.Seat, v.Verb, v.Subject, module)
}
}
}
serve()
conn.OnMembership(func(bus.Membership) { go func() { serve(); conn.Flush() }() })
return func() {
mu.Lock()
defer mu.Unlock()
for _, s := range stops {
s()
}
stops = nil
}
}
// consumers holds, per module the runtime serves, the one durable consumer its events arrive on and
// the children its events are handed to (novox/hq ADR 0198).
type consumers struct {
conn *bus.Conn
logf func(string, ...any)
mu sync.Mutex
of map[string]*moduleEvents
}
type moduleEvents struct {
stop func()
delivers []*func(json.RawMessage) error
}
// stopAll unbinds every module's consumer.
func (c *consumers) stopAll() {
c.mu.Lock()
defer c.mu.Unlock()
for _, m := range c.of {
if m.stop != nil {
m.stop()
}
}
c.of = map[string]*moduleEvents{}
}
// forModule is the bus one launched bundle of a module reaches the mesh through.
func (c *consumers) forModule(module string) launch.Bus {
return &moduleBus{all: c, module: module}
}
type moduleBus struct {
all *consumers
module string
mu sync.Mutex
deliver *func(json.RawMessage) error
}
// Publish emits an event as the module (ADR 0193).
func (b *moduleBus) Publish(params json.RawMessage) error {
var env bus.Envelope
if err := json.Unmarshal(params, &env); err != nil {
return fmt.Errorf("not an event envelope: %w", err)
}
return b.all.conn.PublishAs(b.module, env)
}
// Ask calls a tool as the module: `{key, body}`, answered with the tool's result (ADR 0198).
func (b *moduleBus) Ask(params json.RawMessage) (json.RawMessage, error) {
var asked struct {
Key string `json:"key"`
Body json.RawMessage `json:"body"`
}
if err := json.Unmarshal(params, &asked); err != nil || asked.Key == "" {
return nil, fmt.Errorf("mesh/ask names no tool: {key, body}")
}
body := any(asked.Body)
if len(asked.Body) == 0 {
body = map[string]any{}
}
answered, err := b.all.conn.AskAs(b.module, asked.Key, body)
if err != nil {
return nil, err
}
if len(answered.Result) == 0 {
return json.RawMessage("null"), nil
}
return answered.Result, nil
}
// Subscribe hands this bundle the module's events. The consumer is bound once per module; each of
// the module's children that subscribed is handed every event, and the event is acknowledged only
// when all of them took it — one consumer split between two readers would give each half.
func (b *moduleBus) Subscribe(deliver func(json.RawMessage) error) error {
b.mu.Lock()
if b.deliver == nil {
d := deliver
b.deliver = &d
} else {
*b.deliver = deliver
}
mine := b.deliver
b.mu.Unlock()
c := b.all
c.mu.Lock()
defer c.mu.Unlock()
m := c.of[b.module]
if m == nil {
m = &moduleEvents{}
c.of[b.module] = m
}
listed := false
for _, d := range m.delivers {
if d == mine {
listed = true
}
}
if !listed {
m.delivers = append(m.delivers, mine)
}
if m.stop != nil {
return nil
}
module := b.module
stop, err := c.conn.ConsumeAs(module, func(env bus.Envelope) error {
raw, err := json.Marshal(env)
if err != nil {
return err
}
c.mu.Lock()
targets := append([]*func(json.RawMessage) error(nil), c.of[module].delivers...)
c.mu.Unlock()
for _, d := range targets {
if err := (*d)(raw); err != nil {
c.logf("[mesh-tools] %s did not take %s%s: %s; offered again in %s", module, env.Key,
eventRef(env), whyNotTaken(err), bus.NakDelay)
return err
}
}
return nil
})
if err != nil {
return err
}
m.stop = stop
c.logf("[mesh-tools] %s's events arrive on its consumer %s", module, bus.ConsumerOf(c.conn.Node(), module))
return nil
}
// whyNotTaken says why a bundle did not take an event, so the line names who failed: the module's
// handler, in its own words — an error it answered — or the runtime not reaching it. A bundle's
// handler error read bare ("Unexpected end of JSON input") looked like the runtime's (issue 276).
func whyNotTaken(err error) string {
var refused *launch.Refused
if errors.As(err, &refused) {
return fmt.Sprintf("its handler answered an error, in its own words: %q (an event is taken by any answer that is not an error)", refused.Message)
}
return err.Error()
}
// eventRef names the event a line is about by its id, when it has one, so its offers can be told
// apart from another event's.
func eventRef(env bus.Envelope) string {
if id := env.Headers["x-event-id"]; id != "" {
return " (event " + id + ")"
}
return ""
}
// stateAsked is what a bundle names when it reaches its state (ADR 0201): the state by the name its
// module uses, a key, and for a put the value.
type stateAsked struct {
State string `json:"state"`
Key string `json:"key"`
Value json.RawMessage `json:"value"`
}
// State answers a bundle's `get`, `put`, `delete` and `keys` on its module's state (ADR 0201). The
// runtime refuses, with the reason, a state the module was not issued and a write to one it only reads.
func (b *moduleBus) State(verb string, params json.RawMessage) (json.RawMessage, error) {
var asked stateAsked
if err := json.Unmarshal(params, &asked); err != nil || asked.State == "" {
return nil, fmt.Errorf("mesh/state.%s names no state: {state, key, value}", verb)
}
conn := b.all.conn
var answer any
switch verb {
case "get":
entry, err := conn.StateGet(b.module, asked.State, asked.Key)
if err != nil {
return nil, err
}
if entry == nil {
return json.RawMessage("null"), nil
}
answer = entry
case "put":
revision, err := conn.StatePut(b.module, asked.State, asked.Key, asked.Value)
if err != nil {
return nil, err
}
answer = map[string]any{"revision": revision}
case "delete":
if err := conn.StateDelete(b.module, asked.State, asked.Key); err != nil {
return nil, err
}
answer = map[string]any{}
case "keys":
keys, err := conn.StateKeys(b.module, asked.State)
if err != nil {
return nil, err
}
answer = keys
default:
return nil, fmt.Errorf("the runtime answers no mesh/state.%s", verb)
}
return json.Marshal(answer)
}
// Watch hands a bundle its module's state as it is and as it changes (ADR 0201): `{state, key}`, the
// key a pattern with `*` and `**`, empty for every key.
func (b *moduleBus) Watch(params json.RawMessage, deliver func(json.RawMessage) error) (func(), error) {
var asked stateAsked
if err := json.Unmarshal(params, &asked); err != nil || asked.State == "" {
return nil, fmt.Errorf("mesh/state.watch names no state: {state, key}")
}
return b.all.conn.StateWatch(b.module, asked.State, asked.Key, func(change bus.StateChange) error {
raw, err := json.Marshal(change)
if err != nil {
return err
}
return deliver(raw)
})
}
+268
View File
@@ -0,0 +1,268 @@
package runtime
import (
"encoding/json"
"errors"
"os"
"strings"
"testing"
"time"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/launch"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
func connect(t *testing.T, module, node string) *bus.Conn {
t.Helper()
c, err := bus.Connect(bus.Credential{URL: mt.URL(t), Module: module, Node: node})
if err != nil {
t.Fatal(err)
}
c.Logf = func(string, ...any) {}
t.Cleanup(c.Close)
return c
}
func call(t *testing.T, asker *bus.Conn, key string, args any) (string, error) {
t.Helper()
got, err := asker.Ask(key, args, "")
return string(got.Result), err
}
func same(t *testing.T, got, want string) {
t.Helper()
var a, b any
if json.Unmarshal([]byte(got), &a) != nil || json.Unmarshal([]byte(want), &b) != nil {
t.Fatalf("not JSON: got %s want %s", got, want)
}
ga, _ := json.Marshal(a)
gb, _ := json.Marshal(b)
if string(ga) != string(gb) {
t.Errorf("got %s, want %s", got, want)
}
}
// The node's runtime serves five modules' bundles on one credential — two TypeScript, one broken,
// one Python, one written against the protocol — and follows a re-issued membership (ADR 0175, 0193).
func TestTheNodesRuntimeServesFiveModulesAndFollowsAReissuedMembership(t *testing.T) {
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("alpha", "anchor", true, nil))
mesh.Issue(t, mt.MembershipOf("beta", "anchor", false, map[string][]string{"node-shelf": {"list", "clear"}}))
mesh.Issue(t, mt.MembershipOf("gamma", "anchor", false, nil))
mesh.Issue(t, mt.MembershipOf("delta", "anchor", false, map[string][]string{"node-lamp": {"on"}}))
mesh.Issue(t, mt.MembershipOf("epsilon", "anchor", false, nil))
nodeTools := connect(t, "node-tools", "anchor")
asker := connect(t, "console", "workstation")
logs := &mt.Logs{}
t.Setenv("MESH_OPERATOR_ACCOUNT", "somebody")
t.Setenv("MESH_OPERATOR_HOME", "/home/somebody")
stop, err := Run(nodeTools, []Served{
{"alpha", []string{mt.Fixture("many-alpha.serve.mjs")}},
{"beta", []string{mt.Fixture("many-beta.serve.mjs")}},
{"gamma", []string{mt.Fixture("many-broken.serve.mjs")}},
{"delta", []string{mt.Fixture("many-delta.py")}},
{"epsilon", []string{mt.Fixture("many-epsilon.mjs")}},
}, nil, logs.Logf)
if err != nil {
t.Fatal(err)
}
defer stop()
if !logs.Has("the operator's account here is somebody (home /home/somebody)") {
t.Errorf("the operator was not said:\n%s", logs.All())
}
if !logs.Has("gamma's bundle", "many-broken.serve.mjs failed to load: gamma's bundle exited (1): Error: gamma's bundle cannot find its client; its tools are not served here") {
t.Errorf("the broken bundle was not named with its own words:\n%s", logs.All())
}
if !logs.Has("serving 8 tool(s) for 5 module(s): alpha.one, alpha.two, beta.three, beta.four, beta.five, delta.greet, delta.die, epsilon.seven; not serving gamma") {
t.Errorf("not serving what it should:\n%s", logs.All())
}
for key, want := range map[string]string{
"alpha.one": `{"alpha":1}`, "alpha.one@anchor": `{"alpha":1}`, "beta.three@anchor": `{"beta":3}`,
"beta.four@anchor": `{"beta":4}`, "beta.five@anchor": `{"beta":5}`,
"seat:node-shelf.list@anchor": `{"shelf":["a","b"]}`, "seat:node-shelf.clear@anchor": `{"cleared":true}`,
"seat:node-lamp.on@anchor": `{"on":true,"language":"python"}`, "epsilon.seven@anchor": `{"epsilon":7,"via":"stdio"}`,
} {
got, err := call(t, asker, key, map[string]any{})
if err != nil {
t.Errorf("%s: %v", key, err)
continue
}
same(t, got, want)
}
if _, err := call(t, asker, "beta.three", map[string]any{}); err == nil || !strings.Contains(err.Error(), "no responders") {
t.Errorf("beta was not issued the plain subject, and answered on it: %v", err)
}
got, err := call(t, asker, "delta.greet@anchor", map[string]any{"who": "mesh"})
if err != nil {
t.Fatal(err)
}
same(t, got, `{"greeting":"hello mesh","language":"python"}`)
// A launched tool that emits does so as its module, through the runtime.
landed := mesh.NextEvent(t, "mesh.mod.*.event.>")
got, err = call(t, asker, "alpha.two", map[string]any{})
if err != nil {
t.Fatal(err)
}
same(t, got, `{"alpha":2}`)
select {
case subject := <-landed:
if subject != "mesh.mod.alpha.event.happened" {
t.Errorf("the event landed on %s", subject)
}
case <-time.After(5 * time.Second):
t.Error("the event never landed")
}
// A bundle that dies mid-call is said, and started again on its next call.
if _, err := call(t, asker, "delta.die@anchor", map[string]any{}); err == nil {
t.Error("a bundle that died answered")
}
got, err = call(t, asker, "delta.greet@anchor", map[string]any{"who": "again"})
if err != nil {
t.Fatalf("not started again: %v", err)
}
same(t, got, `{"greeting":"hello again","language":"python"}`)
// `tools` answers for each, and why gamma serves nothing.
gamma, err := call(t, asker, "gamma.tools@anchor", map[string]any{})
if err != nil {
t.Fatal(err)
}
same(t, gamma, `{"module":"gamma","tools":[],"failed":"gamma's bundle exited (1): Error: gamma's bundle cannot find its client"}`)
beta, _ := call(t, asker, "beta.tools@anchor", map[string]any{})
var answer ToolsAnswer
_ = json.Unmarshal([]byte(beta), &answer)
if len(answer.Tools) != 3 || answer.Tools[0].Name != "three" || strings.Join(answer.Tools[0].Subjects, ",") != "mesh.mod.beta.tool.three.anchor" {
t.Errorf("beta's tools answer: %s", beta)
}
// Re-issued mid-run, now answering for the module anywhere: served without a restart.
mesh.Issue(t, mt.MembershipOf("beta", "anchor", true, map[string][]string{"node-shelf": {"list", "clear"}}))
mt.Until(t, func() error { _, err := call(t, asker, "beta.three", map[string]any{}); return err })
got, err = call(t, asker, "seat:node-shelf.list@anchor", map[string]any{})
if err != nil {
t.Fatal(err)
}
same(t, got, `{"shelf":["a","b"]}`)
}
// Each bundle is given its own environment and none of another's (ADR 0192), and the composed
// environments are not left in the runtime's.
func TestEachBundleIsGivenItsOwnEnvironment(t *testing.T) {
mesh := mt.New(t)
for _, m := range []string{"gamma", "delta", "zeta"} {
mesh.Issue(t, mt.MembershipOf(m, "anchor", false, nil))
}
nodeTools := connect(t, "node-tools", "anchor")
asker := connect(t, "console", "workstation")
t.Setenv("MESH_OPERATOR_ACCOUNT", "somebody")
t.Setenv(ToolEnv, `{"gamma":{"GAMMA_CONFIG_FILE":"/var/lib/mesh/gamma/config.json"},"delta":{"DELTA_TOKEN_FILE":"/var/lib/mesh/delta/token"},"zeta":{"ZETA_URL":"http://127.0.0.1:3000"}}`)
envs, err := TakeToolEnvs()
if err != nil {
t.Fatal(err)
}
if _, left := os.LookupEnv(ToolEnv); left {
t.Error("the composed environments were left in the runtime's")
}
logs := &mt.Logs{}
stop, err := Run(nodeTools, []Served{
{"gamma", []string{mt.Fixture("env-gamma.serve.mjs")}},
{"delta", []string{mt.Fixture("env-delta.serve.mjs")}},
{"zeta", []string{mt.Fixture("env-zeta.mjs")}},
}, envs, logs.Logf)
if err != nil {
t.Fatal(err)
}
defer stop()
for key, want := range map[string]string{
"gamma.given@anchor": `{"mine":"/var/lib/mesh/gamma/config.json","theirs":null,"runtime":"somebody","composed":null}`,
"delta.given@anchor": `{"mine":"/var/lib/mesh/delta/token","theirs":null}`,
"zeta.given@anchor": `{"mine":"http://127.0.0.1:3000","theirs":null,"composed":null}`,
} {
got, err := call(t, asker, key, map[string]any{})
if err != nil {
t.Fatalf("%s: %v\n%s", key, err, logs.All())
}
same(t, got, want)
}
}
// An entrypoint that is not executable is refused by name, and the others serve (ADR 0193).
func TestAnEntrypointThatIsNotExecutableIsRefused(t *testing.T) {
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("alpha", "anchor", false, nil))
mesh.Issue(t, mt.MembershipOf("plain", "anchor", false, nil))
nodeTools := connect(t, "node-tools", "anchor")
asker := connect(t, "console", "workstation")
logs := &mt.Logs{}
stop, err := Run(nodeTools, []Served{
{"alpha", []string{mt.Fixture("many-alpha.serve.mjs")}},
{"plain", []string{mt.Fixture("many-alpha.mjs")}},
}, nil, logs.Logf)
if err != nil {
t.Fatal(err)
}
defer stop()
if !logs.Has("plain's bundle", "many-alpha.mjs failed to load:", "is not executable; a bundle the runtime serves is started, never imported") {
t.Errorf("the non-executable entrypoint was not refused by name:\n%s", logs.All())
}
got, err := call(t, asker, "alpha.one@anchor", map[string]any{})
if err != nil {
t.Fatal(err)
}
same(t, got, `{"alpha":1}`)
}
// A launched bundle is told the module it serves, so its seat's verbs stay the seat's (ADR 0193).
func TestALaunchedBundleRegisteringItsSeatFirstServesTheSeat(t *testing.T) {
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("theta", "anchor", false, map[string][]string{"node-shelf": {"list"}}))
nodeTools := connect(t, "node-tools", "anchor")
asker := connect(t, "console", "workstation")
stop, err := Run(nodeTools, []Served{{"theta", []string{mt.Fixture("served-seat-first.mjs")}}},
map[string]map[string]string{"theta": {"THETA_WORD": "given"}}, (&mt.Logs{}).Logf)
if err != nil {
t.Fatal(err)
}
defer stop()
got, err := call(t, asker, "theta.own@anchor", map[string]any{})
if err != nil {
t.Fatal(err)
}
same(t, got, `{"theta":"given"}`)
got, err = call(t, asker, "seat:node-shelf.list@anchor", map[string]any{})
if err != nil {
t.Fatal(err)
}
same(t, got, `{"shelf":["x"]}`)
}
func TestToolModulesNamesOtherModulesOnly(t *testing.T) {
got, err := ServedModulesFrom(" alpha=/a/tools/index.serve.mjs, beta=/b/one, beta=/b/two ", "node-tools")
if err != nil || len(got) != 2 || got[1].Module != "beta" || len(got[1].Entrypoints) != 2 {
t.Fatalf("%v %v", got, err)
}
for _, bad := range []string{"/mine/index.js", "node-tools=/own.js", "=/x"} {
if _, err := ServedModulesFrom(bad, "node-tools"); err == nil {
t.Errorf("%q was accepted", bad)
}
}
}
// A line about an event a bundle did not take says who failed: the handler in its own words, or the
// runtime not reaching it (hq issue 276).
func TestAnUntakenEventSaysWhoFailed(t *testing.T) {
handler := whyNotTaken(&launch.Refused{Method: "mesh/event", Message: "Unexpected end of JSON input"})
if !strings.Contains(handler, `its handler answered an error, in its own words: "Unexpected end of JSON input"`) {
t.Errorf("a handler's error is not said as the handler's: %s", handler)
}
if got := whyNotTaken(errors.New("plex's bundle did not answer mesh/event in 120s")); got != "plex's bundle did not answer mesh/event in 120s" {
t.Errorf("the runtime's own failure is not said as it is: %s", got)
}
if got := eventRef(bus.Envelope{Headers: map[string]string{"x-event-id": "e1"}}); got != " (event e1)" {
t.Errorf("the event is not named by its id: %q", got)
}
}
+178
View File
@@ -0,0 +1,178 @@
package runtime
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-tools/node-tools/internal/bus"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
// novox/hq ADR 0201: a module keeps its current state in buckets it declares, and its code reaches
// them through the runtime. The owner's instances write and read; a reader only reads; a watch hands
// the current values — none that is deleted — and then every change; the runtime refuses, with the
// reason, what the module was not issued, a write to state it only reads, and a value naming a
// credential.
func TestABundleKeepsAndWatchesItsStateThroughTheRuntime(t *testing.T) {
mesh := mt.New(t)
kv := mesh.Bucket(t, "keeper_servers")
// What was there before anything started: one value, and one key since deleted.
if _, err := kv.Put("all.one", []byte(`{"url":"http://one"}`)); err != nil {
t.Fatal(err)
}
if _, err := kv.Put("all.gone", []byte(`{}`)); err != nil {
t.Fatal(err)
}
if err := kv.Delete("all.gone"); err != nil {
t.Fatal(err)
}
keeper := mt.MembershipOf("keeper", "anchor", false, nil)
keeper.State = []bus.StateIssued{{Name: "servers", Bucket: "keeper_servers", Writes: true}}
peeker := mt.MembershipOf("peeker", "anchor", false, nil)
peeker.State = []bus.StateIssued{{Name: "keeper.servers", Bucket: "keeper_servers"}}
mesh.Issue(t, keeper)
mesh.Issue(t, peeker)
nodeTools := connect(t, "node-tools", "anchor")
asker := connect(t, "console", "workstation")
dir := t.TempDir()
keeperLog, peekerLog := filepath.Join(dir, "keeper.log"), filepath.Join(dir, "peeker.log")
said := &lines{}
stop, err := Run(nodeTools, []Served{
{Module: "keeper", Entrypoints: []string{mt.Fixture("state-keeper.mjs")}},
{Module: "peeker", Entrypoints: []string{mt.Fixture("state-keeper.mjs")}},
}, map[string]map[string]string{
"keeper": {"STATE_NAME": "servers", "STATE_LOG": keeperLog},
"peeker": {"STATE_NAME": "keeper.servers", "STATE_LOG": peekerLog},
}, said.logf)
if err != nil {
t.Fatal(err)
}
t.Cleanup(stop)
// Both read the whole current state at start: the value, not the deleted key, then the end of it.
for _, log := range []string{keeperLog, peekerLog} {
waitFor(t, log, "current")
lines := read(t, log)
if lines[0] != `was put all.one {"url":"http://one"}` || lines[1] != "current" || strings.Contains(strings.Join(lines, "\n"), "all.gone") {
t.Fatalf("the current state was not handed over as it is:\n%s\nruntime:\n%s", strings.Join(lines, "\n"), said.all())
}
}
// The owner writes; every watch sees the change.
got, err := call(t, asker, "keeper.put@anchor", map[string]any{"state": "servers", "key": "anchor.two", "value": map[string]any{"url": "http://two"}})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(got, `"revision"`) {
t.Fatalf("a put answered %s", got)
}
waitFor(t, keeperLog, `now put anchor.two {"url":"http://two"}`)
waitFor(t, peekerLog, `now put anchor.two {"url":"http://two"}`)
// Get and keys, by the owner and by the reader.
got, err = call(t, asker, "peeker.get@anchor", map[string]any{"state": "keeper.servers", "key": "anchor.two"})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(got, `"value":{"url":"http://two"}`) {
t.Fatalf("the reader's get answered %s", got)
}
got, err = call(t, asker, "peeker.get@anchor", map[string]any{"state": "keeper.servers", "key": "nothing.here"})
if err != nil || got != "null" {
t.Fatalf("a key with no value answered %s, %v", got, err)
}
got, err = call(t, asker, "keeper.keys@anchor", map[string]any{"state": "servers"})
if err != nil {
t.Fatal(err)
}
same(t, got, `["all.one","anchor.two"]`)
// Refused, with the reason, before anything is sent.
for _, c := range []struct {
key, why string
args map[string]any
}{
{"peeker.put@anchor", "peeker reads keeper.servers and does not keep it",
map[string]any{"state": "keeper.servers", "key": "x", "value": 1}},
{"keeper.get@anchor", `keeper keeps and reads no state called "other"`,
map[string]any{"state": "other", "key": "x"}},
{"keeper.put@anchor", `carries a field "accessToken", which names a credential`,
map[string]any{"state": "servers", "key": "x", "value": map[string]any{"headers": map[string]any{"accessToken": "s"}}}},
{"keeper.put@anchor", "is not a key the bus can hold",
map[string]any{"state": "servers", "key": "a..b", "value": 1}},
} {
if _, err := call(t, asker, c.key, c.args); err == nil || !strings.Contains(err.Error(), c.why) {
t.Errorf("%s %v: want refused for %q, got %v", c.key, c.args, c.why, err)
}
}
// A delete reaches every watch.
if _, err := call(t, asker, "keeper.del@anchor", map[string]any{"state": "servers", "key": "all.one"}); err != nil {
t.Fatal(err)
}
waitFor(t, peekerLog, "now delete all.one")
waitFor(t, keeperLog, "now delete all.one")
}
func read(t *testing.T, path string) []string {
t.Helper()
b, _ := os.ReadFile(path)
return strings.Split(strings.TrimSpace(string(b)), "\n")
}
func waitFor(t *testing.T, path, line string) {
t.Helper()
mt.Until(t, func() error {
for _, l := range read(t, path) {
if l == line {
return nil
}
}
return errorf("%s has no %q yet: %s", filepath.Base(path), line, strings.Join(read(t, path), " | "))
})
}
// The TypeScript SDK against the runtime (ADR 0201): two watches of one state, each handed only its
// own changes, the current values handled before the module's import goes on.
func TestTheSDKsStateReachesTheRuntime(t *testing.T) {
mesh := mt.New(t)
kv := mesh.Bucket(t, "sdkkeeper_servers")
if _, err := kv.Put("all.one", []byte(`{"url":"http://one"}`)); err != nil {
t.Fatal(err)
}
m := mt.MembershipOf("sdkkeeper", "anchor", false, nil)
m.State = []bus.StateIssued{{Name: "servers", Bucket: "sdkkeeper_servers", Writes: true}}
mesh.Issue(t, m)
nodeTools := connect(t, "node-tools", "anchor")
asker := connect(t, "console", "workstation")
log := filepath.Join(t.TempDir(), "sdk.log")
said := &lines{}
stop, err := Run(nodeTools, []Served{{Module: "sdkkeeper", Entrypoints: []string{mt.Fixture("state-sdk.serve.mjs")}}},
map[string]map[string]string{"sdkkeeper": {"STATE_LOG": log}}, said.logf)
if err != nil {
t.Fatalf("%v\n%s", err, said.all())
}
t.Cleanup(stop)
waitFor(t, log, "current")
if got := strings.Join(read(t, log), " | "); got != "was put all.one | current" {
t.Fatalf("the import went on with %q", got)
}
if _, err := call(t, asker, "sdkkeeper.register@anchor", map[string]any{"key": "anchor.two", "url": "http://two"}); err != nil {
t.Fatal(err)
}
waitFor(t, log, "now put anchor.two")
waitFor(t, log, "narrow put anchor.two")
if _, err := call(t, asker, "sdkkeeper.register@anchor", map[string]any{"key": "all.three", "url": "http://three"}); err != nil {
t.Fatal(err)
}
waitFor(t, log, "now put all.three")
for _, l := range read(t, log) {
if l == "narrow put all.three" {
t.Fatalf("a narrowed watch was handed a key outside it: %s", strings.Join(read(t, log), " | "))
}
}
}
+18
View File
@@ -0,0 +1,18 @@
// Package wire is JSON as the TypeScript runtime writes it: no HTML escaping of <, > and &.
package wire
import (
"bytes"
"encoding/json"
)
// Marshal encodes v the way JSON.stringify does, without a trailing newline.
func Marshal(v any) ([]byte, error) {
var b bytes.Buffer
enc := json.NewEncoder(&b)
enc.SetEscapeHTML(false)
if err := enc.Encode(v); err != nil {
return nil, err
}
return bytes.TrimRight(b.Bytes(), "\n"), nil
}
+56
View File
@@ -0,0 +1,56 @@
{
"module": "node-tools",
"version": "1",
"slug": "node-tools",
"invokes": [
"*"
],
"provides": [
{
"name": "mcp-endpoint",
"scope": "node"
}
],
"serves": {
"mcp-endpoint": {
"port": 4270
}
},
"own-secrets": {
"broker": "${dir:mesh-state}/broker"
},
"listens": [
{
"name": "mcp",
"port": 4270,
"protocol": "tcp",
"from": "machine",
"why": "the mesh's tools for whoever is on this machine, over MCP on loopback; the machine's login is the authority (novox/hq ADR 0152, 0175)"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0755",
"place": "mesh"
},
{
"id": "interpreter",
"type": "package",
"package": "nodejs"
}
],
"build": {
"artifacts": [
{
"name": "runtime",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/node-tools",
"binary": "node-tools"
}
]
}
}
+76
View File
@@ -0,0 +1,76 @@
{
"name": "@novox/mesh-tools",
"version": "0.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@novox/mesh-tools",
"version": "0.1.0",
"dependencies": {
"@novox/mesh-sdk": "^0.1.0",
"nats": "^2.29.0"
},
"bin": {
"mesh": "dist/mesh.js",
"mesh-tools": "dist/main.js"
},
"devDependencies": {
"@types/node": "^22.20.1",
"typescript": "^5.9.3"
}
},
"node_modules/@novox/mesh-sdk": {
"version": "0.1.1"
},
"node_modules/@types/node": {
"version": "22.20.1",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~6.21.0"
}
},
"node_modules/nats": {
"version": "2.29.3",
"license": "Apache-2.0",
"dependencies": {
"nkeys.js": "1.1.0"
},
"engines": {
"node": ">= 14.0.0"
}
},
"node_modules/nkeys.js": {
"version": "1.1.0",
"license": "Apache-2.0",
"dependencies": {
"tweetnacl": "1.0.3"
},
"engines": {
"node": ">=10.0.0"
}
},
"node_modules/tweetnacl": {
"version": "1.0.3",
"license": "Unlicense"
},
"node_modules/typescript": {
"version": "5.9.3",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
},
"node_modules/undici-types": {
"version": "6.21.0",
"dev": true,
"license": "MIT"
}
}
}
+8 -6
View File
@@ -4,19 +4,21 @@
"description": "The Novox Mesh tool runtime \u2014 binds the mesh broker and serves the assigned modules' tools.",
"type": "module",
"bin": {
"mesh-tools": "./dist/main.js"
"mesh-tools": "./dist/main.js",
"mesh": "./dist/mesh.js"
},
"scripts": {
"build": "tsc",
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
"pretest": "tsc",
"test": "node --test --test-concurrency=1 --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.0",
"amqplib": "^0.10.9"
"@novox/mesh-sdk": "^0.1.6",
"nats": "^2.29.0"
},
"devDependencies": {
"@types/amqplib": "^0.10.8",
"@types/node": "^22.20.1",
"typescript": "^5.9.3"
"typescript": "^5.9.3",
"esbuild": "^0.25.0"
}
}
+139
View File
@@ -0,0 +1,139 @@
// What a runtime serves, it announces (novox/hq ADR 0197): the NATS services protocol's discovery —
// `$SRV.PING`, `$SRV.INFO`, `$SRV.STATS`, and the same followed by the service's name and its id —
// answered in the io.nats.micro.v1 format with what is served at the moment of the request. Serving is
// unchanged; this only says what is served. One service per runtime process, because the bus admits
// one reply per request from each responder: one endpoint per tool per subject, its metadata saying
// which module, seat, scope and machine it is. The same shape the Go runtime answers.
import { StringCodec } from "nats";
import { asSchema } from "@novox/mesh-sdk/stdio";
import type { ToolDefinition } from "@novox/mesh-sdk/tools";
import type { RuntimeBroker } from "./broker-nats.js";
const sc = StringCodec();
export const VERSION = "0.1.0";
export const INFO_RESPONSE = "io.nats.micro.v1.info_response";
export const PING_RESPONSE = "io.nats.micro.v1.ping_response";
export const STATS_RESPONSE = "io.nats.micro.v1.stats_response";
/** One tool served on one subject, as announced. */
export interface Endpoint {
kind: "tool" | "seat";
module: string;
tool: string;
seat?: string;
scope?: "mesh" | "node";
node: string;
description: string;
schema: unknown;
interchangeable: boolean;
subject: string;
queue?: string;
}
/** A seat's verb as the runtime serves it, with the definition that answers it. */
export interface ServedSeatVerb {
seat: string;
verb: string;
subject: string;
holder: string;
tool: ToolDefinition;
}
export interface Service {
name: string;
id: string;
description: string;
metadata: Record<string, string>;
}
/** The endpoint's name as the protocol allows it; the metadata, not the name, identifies it. */
function nameOf(e: Endpoint): string {
const clean = (s: string) => s.replace(/[^A-Za-z0-9_-]/g, "_");
return `${clean(e.kind === "seat" ? e.seat ?? "" : e.module)}__${clean(e.tool)}`;
}
/** The info_response for these endpoints. */
export function info(s: Service, endpoints: Endpoint[]): Record<string, unknown> {
return {
name: s.name, id: s.id, version: VERSION, metadata: s.metadata, type: INFO_RESPONSE, description: s.description,
endpoints: endpoints.map((e) => {
const metadata: Record<string, string> = {
kind: e.kind, module: e.module, tool: e.tool, node: e.node, description: e.description,
schema: JSON.stringify(e.schema ?? {}), interchangeable: e.interchangeable ? "true" : "false",
};
if (e.kind === "seat") {
metadata.seat = e.seat ?? "";
metadata.scope = e.scope ?? "mesh";
}
return { name: nameOf(e), subject: e.subject, queue_group: e.queue ?? "", metadata };
}),
};
}
/** Everything served now: each module's tools on every subject issued for them, and each held
* seat's verbs on the seat's subject. */
export function endpointsOf(
broker: RuntimeBroker,
own: { module: string; tools: ToolDefinition[] }[],
seats: ServedSeatVerb[],
): Endpoint[] {
const node = broker.node ?? "";
const out: Endpoint[] = [];
for (const { module, tools } of own) {
for (const t of tools) {
const served = broker.servedOn ? broker.servedOn(module, t.name) : [];
const interchangeable = served.some((s) => s.subject === `mesh.mod.${module}.tool.${t.name}`);
for (const s of served) {
out.push({ kind: "tool", module, tool: t.name, node, description: t.description, schema: asSchema(t.input),
interchangeable, subject: s.subject, queue: s.queue });
}
}
}
for (const v of seats) {
out.push({ kind: "seat", module: v.holder, tool: v.verb, seat: v.seat,
scope: node && v.subject.endsWith(`.${node}`) ? "node" : "mesh", node, description: v.tool.description,
schema: asSchema(v.tool.input), interchangeable: false, subject: v.subject });
}
return out;
}
/** Answer discovery for one service until stopped. */
export function announce(broker: RuntimeBroker, s: Service, current: () => Endpoint[]): () => void {
const started = new Date().toISOString();
const identity = { name: s.name, id: s.id, version: VERSION, metadata: s.metadata };
const answer = (subject: string): Uint8Array | undefined => {
const parts = subject.split(".");
if (parts[0] !== "$SRV" || parts.length < 2) return undefined;
if (parts.length >= 3 && parts[2] !== s.name) return undefined; // another service's
if (parts.length >= 4 && parts[3] !== s.id) return undefined; // another instance's
let v: unknown;
switch (parts[1]) {
case "PING":
v = { ...identity, type: PING_RESPONSE };
break;
case "INFO":
v = info(s, current());
break;
case "STATS":
v = { ...identity, type: STATS_RESPONSE, started, endpoints: current().map((e) => ({
name: nameOf(e), subject: e.subject, queue_group: e.queue ?? "", num_requests: 0, num_errors: 0,
last_error: "", processing_time: 0, average_processing_time: 0 })) };
break;
default:
return undefined;
}
return sc.encode(JSON.stringify(v));
};
const stops: (() => void)[] = [];
// Exactly the questions asked of every service and of this one by name and instance — what the
// grants allow (novox/hq ADR 0197). A wildcard is refused by the bus, and a refused subscription
// ends a runtime: on 2026-10-03 it crash-looped every container that announced itself.
for (const verb of ["PING", "INFO", "STATS"]) {
for (const subject of [`$SRV.${verb}`, `$SRV.${verb}.${s.name}`, `$SRV.${verb}.${s.name}.${s.id}`]) {
stops.push(broker.raw!(subject, (subj) => answer(subj)));
}
}
return () => stops.forEach((stop) => stop());
}
+677
View File
@@ -0,0 +1,677 @@
// The tool runtime's broker client, on NATS.
//
// **The sdk's contract does not change** (novox/hq ADR 0106, ADR 0039): a module is written
// against `request`, `handle`, `publish`, `subscribe`, `close`, and the runtime implements them.
// That is why a module built before any of this runs on the new runtime without a rebuild, and
// why the sdk's own diff for the whole bus change is three comments.
//
// Underneath, everything is a subject and durability is JetStream (novox/hq design 25,
// design 29).
//
// mesh.mod.<module>.event.<type> an event this module emits
// mesh.mod.<module>.tool.<tool> a tool this module serves
// mesh.seat.<seat>.accept.<verb> work submitted to a role
//
// The module never writes one of those: it names its events and tools locally and the mesh
// derives the subject (design 29 §1), so reorganising the subject space leaves every module
// correct.
import { AsyncLocalStorage } from "node:async_hooks";
import { createHash } from "node:crypto";
import net from "node:net";
import tls from "node:tls";
import { connect as natsConnect, headers as natsHeaders, StringCodec, type JsMsg, type Subscription, type TlsOptions } from "nats";
import type { Broker, Envelope, EventHeaders } from "@novox/mesh-sdk/messaging";
const sc = StringCodec();
/** Requests wait this long for an answer before failing. Unchanged from what modules already
* expect, so a module's timeout handling is not something the bus quietly redefines. */
const REQUEST_TIMEOUT_MS = 30_000;
export class PinMismatchError extends Error {}
/** A broker credential as the mesh delivers it (novox/hq ADR 0120): the bus's address, the
* fingerprint of the certificate it must present, and the node and module the account is scoped
* to — the runtime derives its subjects from those rather than being told them. */
export interface Credential {
url: string;
fingerprint?: string;
node?: string;
module?: string;
user?: string;
password?: string;
/** The seats this module claims, with the verbs each promises (novox/hq ADR 0159). The runtime
* serves each claimed seat's verbs with its tools of the same name; the bus admits only the
* holder's subscription, so claiming and not holding costs a refused subscription and nothing else. */
claims?: { seat: string; scope?: string; serves?: string[] }[];
}
/**
* What the mesh issued this assignment (novox/hq ADR 0160): where its tools are served, in which
* queue, the verbs of the seats it holds, where its events land, what it may reach. Read from the
* ASSIGNMENTS stream at `mesh.assignment.<node>.<module>` — the one subject a runtime derives for
* itself — and followed live. Absent for a mesh older than the membership, and then the runtime
* serves the shape it always derived, and says so.
*/
export interface Membership {
node: string;
module: string;
/** Addresses a tool is answered on; `{tool}` stands for the tool's name. */
serves: { subject: string; queue?: string }[];
seats?: { seat: string; verb: string; subject: string }[];
emits: string;
reaches?: Record<string, string[]>;
tools: string;
}
/** What a tool call answers: the module's own result, and which machine answered it
* (novox/hq ADR 0159) — a module on several machines is otherwise an answer from nowhere. */
export interface Answered<Res> {
result: Res;
node?: string;
}
/** The bus as the runtime sees it: the sdk's contract, and the few things only the runtime needs —
* an answer that says which machine gave it, serving a subject that is not a module's own tool
* (a seat's verb), and following the memberships of the modules it serves beside its own. */
export interface RuntimeBroker extends Broker {
/** Call a tool by key, or — when `on` names a subject the mesh listed for it (ADR 0160) — there. */
ask<Req, Res>(key: string, body: Req, on?: string): Promise<Answered<Res>>;
handleSubject<Req, Res>(subject: string, handler: (body: Req) => Promise<Res>): Promise<() => void>;
/**
* Serve another module's tools from this connection: read its membership on this machine and
* follow it, so `handle("<module>.<tool>")` is served where the mesh issued that module (novox/hq
* ADR 0175: one runtime per node, every assigned module's tools). The account must be allowed
* to read that membership and to subscribe its subjects — the node's is; a module's own is not,
* and is refused by the bus, not here.
*/
follow(module: string): Promise<void>;
/** What the mesh issued this assignment — this module's when none is named — or undefined when
* nothing has been issued yet. */
membership(module?: string): Membership | undefined;
/** Called when the mesh issues a new membership to any module this connection follows; the
* runtime re-serves on it. The membership says which module it is for. */
onMembership(handler: (m: Membership) => void): void;
/** The modules whose memberships this connection follows: its own and every one `follow` added. */
serving(): string[];
/** The module this connection is: what its credential named, and what a bare key serves as. */
readonly module: string;
/** Where a served module's tool is answered right now (ADR 0197: what it serves, it announces). */
servedOn?(module: string, tool: string): { subject: string; queue?: string }[];
/** Answer a subject in a format of its own, not a tool's reply envelope — the NATS services
* protocol's discovery (novox/hq ADR 0197). An undefined answer is no reply. */
raw?(subject: string, answer: (subject: string, data: Uint8Array) => Uint8Array | undefined): () => void;
/** The machine this connection serves on, when its credential names one. */
readonly node?: string;
}
/**
* Which module's tool is at work on this connection, when one is (novox/hq ADR 0175). One runtime
* carries many modules' tools, and an event a tool emits must land on the emitting *module's*
* subject, not the runtime's — so the runtime runs each tool inside this store, and `publish`
* reads the module from it. Outside a tool the connection's own module stands.
*/
export const atWork = new AsyncLocalStorage<{ module: string }>();
const ASSIGNMENTS_STREAM = "ASSIGNMENTS";
/** The one address a runtime derives for itself (ADR 0160). */
export function membershipSubject(node: string, module: string): string {
return `mesh.assignment.${node}.${module}`;
}
/** Whether a connection failure is worth retrying, or is a fact about this configuration that
* retrying cannot change. The runtime's supervisor asks this and does not need to know what it
* is connected to. */
export function fatalBrokerReason(err: unknown): string | null {
if (err instanceof PinMismatchError) return "the bus's certificate does not match the pin";
const e = err as { code?: string; message?: string };
const message = typeof e?.message === "string" ? e.message : String(err);
if (e?.code === "ERR_INVALID_URL" || /invalid url/i.test(message)) {
return "the bus address is not a usable URL";
}
if (/authorization violation|user authentication expired|permissions violation/i.test(message)) {
return "the bus refused this account";
}
return null;
}
/**
* Connect to the mesh bus and return a Broker.
*
* **A module's subjects come from its credential, not from its calls.** `node` and `module` name
* the account the mesh issued, and every subject this client publishes or subscribes is derived
* from them — so a module cannot name another's namespace even by mistake, and what it emits
* matches what the mesh authorised (ADR 0074's identity rule).
*/
export async function connectNats(
target: string | Credential,
opts: { module?: string } = {},
): Promise<RuntimeBroker> {
const cred: Credential = typeof target === "string" ? { url: target } : target;
const self = cred.module ?? opts.module;
if (!self) {
throw new Error(
"a broker credential with no module: the runtime derives its subjects from the account " +
"the mesh issued, and cannot guess which module it is",
);
}
const conn = await natsConnect({
servers: cred.url,
user: cred.user,
pass: cred.password,
name: `${cred.node ?? "?"}.${self}`,
tls: cred.fingerprint ? await pinnedTls(cred.url, cred.fingerprint) : undefined,
// Its own inbox, not a random one: every user's inbox is private to it (design 25 §4), and the
// grant names `_INBOX.<user>.>` — a reply space the client invented would be refused, and with
// it every pull for the next message and every answer to a tool call.
inboxPrefix: cred.user ? `_INBOX.${cred.user}` : undefined,
// Reconnect forever: the bus being restarted is an upgrade, not a reason for every module on
// the mesh to exit. `close()` stays the only thing that ends the connection.
maxReconnectAttempts: -1,
});
const js = conn.jetstream();
const subs: Subscription[] = [];
// Every registration, and the one reader that dispatches to them. A module has one durable
// consumer; the loop belongs to the connection rather than to a subscription.
const listeners: { pattern: string; handler: (env: Envelope<unknown>) => Promise<void> }[] = [];
let reading: Awaited<ReturnType<Awaited<ReturnType<typeof js.consumers.get>>["consume"]>> | undefined;
let closed = false;
const node = cred.node;
// The memberships, one per module this connection serves, each read once and followed. A
// module's own runtime follows one, its own; the node's runtime follows one per assigned module
// (novox/hq ADR 0175) — the same subject shape, the same stream, read as many times as there are
// modules, and nothing on the bus learns a new shape for it. A direct get is one request on the
// stream's API, which is the whole of what an account may ask JetStream for a subject it is
// granted; a 404 is a mesh that has not issued one, which is a fact to say and not an error to
// retry.
const issued = new Map<string, Membership | undefined>();
const issuedHandlers: ((m: Membership) => void)[] = [];
const follow = async (module: string): Promise<void> => {
if (!node || issued.has(module)) return;
issued.set(module, undefined);
const subjectOfTheirs = membershipSubject(node, module);
try {
// The subject-addressed form of a direct get — the stream, then the subject, nothing in
// the body — because that is the one address the mesh grants this account on the
// stream's API; the body form asks the stream's root, which it may not.
const got = await conn.request(`$JS.API.DIRECT.GET.${ASSIGNMENTS_STREAM}.${subjectOfTheirs}`,
new Uint8Array(0), { timeout: 5_000 });
const status = got.headers?.code ?? 0;
if (status === 0 && got.data.length > 0) {
issued.set(module, JSON.parse(sc.decode(got.data)) as Membership);
}
} catch {
// Not readable here: an older mesh, a stream not yet asserted, or no grant. Said below.
}
if (!issued.get(module)) {
console.log(`[mesh-tools] no membership issued for ${module} on ${node} yet; serving the derived shape until one arrives`);
}
try {
const live = conn.subscribe(subjectOfTheirs);
subs.push(live);
void (async () => {
for await (const msg of live) {
try {
const m = JSON.parse(sc.decode(msg.data)) as Membership;
issued.set(module, m);
console.log(`[mesh-tools] ${module} on ${node} was issued a new membership; re-serving on it`);
for (const h of issuedHandlers) h(m);
} catch (err) {
console.log(`[mesh-tools] a membership arrived that is not one: ${err}`);
}
}
})();
} catch {
// A subscription this account may not make is a mesh older than the membership.
}
};
await follow(self);
/** The subjects a tool of a served module is served on: from that module's membership when
* issued, derived otherwise (the shape the mesh issues on day one, so the two agree). */
const servedOn = (module: string, tool: string): { subject: string; queue?: string }[] => {
const m = issued.get(module);
if (m) {
const out = m.serves.map((s) => ({ subject: s.subject.replace("{tool}", tool), queue: s.queue }));
// The verb that lists what this module serves is answered on the mesh's plain address for it
// whatever the placement — one answer suffices, so a queue — and on this machine's beside it.
if (tool === "tools" && m.tools && !out.some((s) => s.subject === m.tools)) {
out.unshift({ subject: m.tools, queue: `serve.${module}` });
}
return out;
}
const base = `mesh.mod.${module}.tool.${tool}`;
const out: { subject: string; queue?: string }[] = [{ subject: base, queue: `serve.${module}` }];
if (node) out.push({ subject: `${base}.${node}` });
return out;
};
/** Where a call by key goes: a subject this connection's own membership says it reaches, when it
* says one — the machine's when named — else the derived shape. */
const reachedAt = (key: string): string => {
const [name, wanted] = key.split("@", 2);
const reach = issued.get(self)?.reaches?.[name];
if (reach && reach.length > 0) {
if (wanted) {
const at = reach.find((s) => s.endsWith(`.${wanted}`));
if (at) return at;
} else {
return reach[0];
}
}
return toolSubject(key, self);
};
/** Answer one subject with one handler, and say which machine answered (novox/hq ADR 0159). */
const answerOn = <Req, Res>(
subject: string,
queue: string | undefined,
handler: (body: Req) => Promise<Res>,
): (() => void) => {
const sub = conn.subscribe(subject, queue ? { queue } : {});
subs.push(sub);
void (async () => {
// **A refused subscription is said, never fatal** (novox/hq 04-ISSUES/218, as 217 for the
// announcements). The grants are the mesh's word on what this account may answer; a subject
// they leave out — a seat claimed here and held elsewhere — costs that subject, never the
// module's other tools, its handlers and its provisioning. Unhandled, the refusal ended the
// process and a module's runtime crash-looped on 2026-10-04.
try {
for await (const msg of sub) {
let reply: { result?: Res; error?: string; node?: string };
try {
reply = { result: await handler(JSON.parse(sc.decode(msg.data)) as Req) };
} catch (err) {
// The caller is told, rather than left to time out: a handler that threw is a
// different failure from a tool nobody serves, and only one of them is worth retrying.
reply = { error: err instanceof Error ? err.message : String(err) };
}
if (node) reply.node = node;
msg.respond(sc.encode(JSON.stringify(reply)));
}
} catch (err) {
console.log(`[mesh-tools] the bus refused ${subject}: ${err instanceof Error ? err.message : String(err)}; ` +
"not served here, and the rest serves on");
}
})();
return () => sub.unsubscribe();
};
/**
* Ask one question and await one answer, with the machine that gave it.
*
* Core NATS request/reply, not JetStream: a tool call must never be persisted (design 25 §3),
* and a lost one is a timeout the caller already handles. The reply travels on the inbox the
* request carries, which the responder may answer because its account has `allow_responses`
* — one reply to a message it actually received, and nothing wider.
*/
const ask = async <Req, Res>(key: string, body: Req, on?: string): Promise<Answered<Res>> => {
const msg = await conn.request(on ?? reachedAt(key), sc.encode(JSON.stringify(body)), {
timeout: REQUEST_TIMEOUT_MS,
});
const reply = JSON.parse(sc.decode(msg.data)) as { result?: Res; error?: string; node?: string };
if (reply.error) throw new Error(reply.error);
return { result: reply.result as Res, node: reply.node };
};
return {
async request<Req, Res>(key: string, body: Req): Promise<Res> {
return (await ask<Req, Res>(key, body)).result;
},
ask,
/**
* Answer a question, two ways (novox/hq ADR 0159): on the module's subject in a queue group,
* so several machines may serve one tool and exactly one of them answers each call; and on the
* same subject with this machine as its last token, so a caller that names the machine reaches
* this instance and no other. A runtime that does not know its machine serves only the first,
* which is how it always behaved.
*/
async handle<Req, Res>(key: string, handler: (body: Req) => Promise<Res>): Promise<() => void> {
// A seat's verb named outright is served on the seat's subject as given, for a holder that
// knows its role without a membership; everything else is a served module's own tool, served
// where the mesh issued that module (ADR 0160). A key naming a module this connection does
// not follow is not served here at all: a module serves its own tools, and the node's
// runtime those of the modules it was given (ADR 0175) — never a stranger's.
if (key.startsWith("seat:")) {
const stop = answerOn(toolSubject(key, self), undefined, handler);
return () => stop();
}
const dot = key.indexOf(".");
const module = dot < 0 ? self : key.slice(0, dot);
if (!issued.has(module) && module !== self) {
throw new Error(
`${self} cannot serve ${key}: a module serves its own tools, and a runtime those of the ` +
"modules it follows",
);
}
const tool = dot < 0 ? key : key.slice(dot + 1);
let stops = servedOn(module, tool).map((s) => answerOn(s.subject, s.queue, handler));
// When that module's new membership arrives, serve where it now says and stop serving where
// it no longer does.
issuedHandlers.push((m) => {
if (m.module !== module) return;
stops.forEach((stop) => stop());
stops = servedOn(module, tool).map((s) => answerOn(s.subject, s.queue, handler));
});
return () => stops.forEach((stop) => stop());
},
module: self,
node,
servedOn,
raw(subject: string, answer: (subject: string, data: Uint8Array) => Uint8Array | undefined): () => void {
const sub = conn.subscribe(subject);
subs.push(sub);
void (async () => {
// **A refusal here is said, never fatal** (novox/hq 04-ISSUES/217). This serves discovery —
// what the runtime says about itself — not the work; a bus that refuses it costs the mesh
// seeing this runtime, not the runtime's tools and handlers. Unhandled, the refusal ended the
// process and every per-module container crash-looped on 2026-10-03.
try {
for await (const msg of sub) {
const body = answer(msg.subject, msg.data);
if (body) msg.respond(body);
}
} catch (err) {
console.log(`[mesh-tools] the bus refused ${subject}: ${err instanceof Error ? err.message : String(err)}; ` +
"discovery will not see this runtime there, and it serves on");
}
})();
return () => sub.unsubscribe();
},
follow,
serving: () => [...issued.keys()],
membership: (module?: string) => issued.get(module ?? self),
onMembership: (handler: (m: Membership) => void) => {
issuedHandlers.push(handler);
},
async handleSubject<Req, Res>(subject: string, handler: (body: Req) => Promise<Res>): Promise<() => void> {
return answerOn(subject, undefined, handler);
},
/**
* Emit an event.
*
* Published into JetStream and awaited, so a publish the bus never accepted fails the emit
* rather than vanishing — at-least-once starts at the emitter, not only the consumer
* (ADR 0042).
*
* `msgID` is the event's own id, so a redelivery after a crash between publishing and
* acknowledging is de-duplicated by the server inside its window rather than seen twice.
*/
async publish<T>(env: Envelope<T>): Promise<void> {
// **The body is the payload and the metadata rides as headers** (ADR 0042). That shape
// is what the conformance suite pins: an implementation that nested the whole envelope in
// the body would pass every one of its own tests and agree with nobody.
const meta = (env.headers ?? {}) as Record<string, string>;
const h = natsHeaders();
for (const [k, v] of Object.entries(meta)) {
if (v != null) h.set(k, String(v));
}
if (!meta["content-type"]) h.set("content-type", "application/json");
if (env.node) h.set("x-node", env.node);
// The emitting module's subject: the tool at work's when a served module's tool emits from
// the node's runtime (ADR 0175), this connection's own otherwise.
await js.publish(eventSubject(env.key, atWork.getStore()?.module ?? self), sc.encode(JSON.stringify(env.body)), {
headers: h,
// De-duplicated by the server inside its window, so a redelivery after a crash between
// publishing and acknowledging is not seen twice. Only the emitter can make this id.
msgID: meta["x-event-id"],
});
},
/**
* React to events.
*
* The durable consumer is the **controller's** to create, from what this module declared it
* consumes (design 29 §3) — this binds to it and never creates one. A runtime that created
* its own would be a module deciding its own delivery semantics, and its account cannot
* reach the JetStream API to do it anyway.
*
* **One consumer, one loop, however many patterns a module registers.** A module has exactly one
* durable consumer, so two loops reading it would each take half the messages — and a loop that
* received one its own pattern does not match acknowledges it, which is the right answer for a
* filter wider than anything registered and silent loss when it is another handler's. Every
* registration is therefore dispatched from one reader, and a message is acknowledged once every
* handler it is for has taken it.
*/
async subscribe<T>(
pattern: string,
handler: (env: Envelope<T>) => Promise<void>,
): Promise<() => void> {
const listener = { pattern, handler: handler as (env: Envelope<unknown>) => Promise<void> };
listeners.push(listener);
if (!reading) {
const durable = `${cred.node ?? "?"}_${self}`;
const consumer = await js.consumers.get("EVENTS", durable);
const messages = await consumer.consume();
reading = messages;
void (async () => {
for await (const msg of messages) {
await deliver(msg, listeners);
}
})();
}
return () => {
const at = listeners.indexOf(listener);
if (at >= 0) listeners.splice(at, 1);
if (listeners.length === 0 && reading) {
void reading.close();
reading = undefined;
}
};
},
async close(): Promise<void> {
if (closed) return;
closed = true;
for (const sub of subs) sub.unsubscribe();
// Drain rather than close: an in-flight reply is finished instead of dropped, which for a
// tool call is the difference between an answer and an unexplained timeout at the caller.
await conn.drain();
},
};
}
/**
* Deliver one event to every handler it is for, acknowledging only once each has taken it.
*
* Several registrations share one durable consumer, so matching happens here rather than by having
* each registration read the stream: two readers of one consumer would split it between them, and a
* message that reached the wrong one would be acknowledged as not-for-me and lost.
*/
async function deliver(
msg: JsMsg,
listeners: { pattern: string; handler: (env: Envelope<unknown>) => Promise<void> }[],
): Promise<void> {
let env: Envelope<unknown>;
try {
env = toEnvelope<unknown>(msg);
} catch {
// Unparseable: acknowledge it. Redelivering a message no version of this code can read is
// an infinite loop, and the stream's dead-letter is for handlers that fail, not for bytes
// that were never an envelope.
msg.term();
return;
}
const forThis = listeners.filter((l) => topicMatches(l.pattern, env.key));
if (forThis.length === 0) {
// The consumer's filters are the controller's, derived from what the module declared it
// consumes, and may be wider than anything it registered a handler for. Acknowledge it, or it
// would be redelivered until it expired.
msg.ack();
return;
}
try {
for (const l of forThis) await l.handler(env);
msg.ack();
} catch {
// Negative-acknowledge with a delay, so a handler failing on a transient cause gets another
// attempt, and one failing permanently exhausts max-deliver and dead-letters rather than
// spinning. The consumer's limits are the controller's; this only says "not done".
msg.nak(5_000);
}
}
/** Rebuild the envelope a module sees, from the subject, the headers and the payload — the
* mirror of publish, and the reason both live beside each other. */
function toEnvelope<T>(msg: JsMsg): Envelope<T> {
const headers: Record<string, string> = {};
if (msg.headers) {
for (const k of msg.headers.keys()) headers[k] = msg.headers.get(k);
}
return {
// The event's own key, recovered from the subject: `mesh.mod.<module>.event.<key>`. The
// module never sees the subject, only the key it declared.
key: keyFromSubject(msg.subject),
node: headers["x-node"] ?? "",
body: JSON.parse(sc.decode(msg.data)) as T,
headers: headers as EventHeaders,
};
}
/** The event key a module sees: the emitter and the event, which is exactly how its manifest names
* what it consumes (novox/hq design 29 §1, 04-ISSUES/127).
*
* **One vocabulary for the declaration and the handler.** This returned the event name alone, so a
* manifest declaring `consumes: builder.built` produced a handler pattern that could never match
* the key it was compared against — and a module consuming the same event from two emitters could
* not tell them apart except by reading a header. The subject already carries the emitter; naming it
* here makes a mismatch between manifest and code a typo rather than a category error. */
function keyFromSubject(subject: string): string {
const marker = ".event.";
const at = subject.indexOf(marker);
if (at < 0) return subject;
const event = subject.slice(at + marker.length);
// `mesh.mod.<emitter>.event.…` — the emitter is the token before the marker.
const before = subject.slice(0, at).split(".");
const emitter = before[before.length - 1];
return emitter ? `${emitter}.${event}` : event;
}
/** A module's own event subject. Derived, never taken from the caller: the module names its
* event and the mesh decides where it lands (design 29 §1). */
function eventSubject(type: string, self: string): string {
return `mesh.mod.${self}.event.${type}`;
}
/** A tool's subject. A bare name is this module's own tool; `<module>.<tool>` addresses
* another's, which is how a request reaches a module that is not this one; `seat:<seat>.<verb>`
* addresses a role's tool, answered by whoever holds the seat (novox/hq ADR 0132) — with
* `seat:<seat>.<verb>@<node>` for a node-scoped seat, whose tool carries the machine (design 33 §4). */
function toolSubject(key: string, self: string): string {
if (key.startsWith("seat:")) {
const rest = key.slice("seat:".length);
const dot = rest.indexOf(".");
if (dot < 0) throw new Error(`"${key}" names a seat and no verb: seat:<seat>.<verb>`);
const seat = rest.slice(0, dot);
const [verb, node] = rest.slice(dot + 1).split("@", 2);
return node ? `mesh.seat.${seat}.tool.${verb}.${node}` : `mesh.seat.${seat}.tool.${verb}`;
}
// `<module>.<tool>@<node>` names the machine (novox/hq ADR 0159): the same subject with the
// machine as its last token, which is what that instance serves beside the queue.
const [name, node] = key.split("@", 2);
const dot = name.indexOf(".");
const base = dot < 0
? `mesh.mod.${self}.tool.${name}`
: `mesh.mod.${name.slice(0, dot)}.tool.${name.slice(dot + 1)}`;
return node ? `${base}.${node}` : base;
}
/** A seat's verb, as its holder serves it: flat for a mesh seat, carrying the machine for a
* node-scoped one (design 33 §4) — the same shape the controller grants. */
export function seatToolSubject(seat: string, verb: string, scope: string | undefined, node: string | undefined): string {
const base = `mesh.seat.${seat}.tool.${verb}`;
return scope === "node" && node ? `${base}.${node}` : base;
}
function normalizeFingerprint(fingerprint: string): string {
return fingerprint.replace(/^sha256:/i, "").replace(/:/g, "").toLowerCase();
}
/**
* Dial once to see the certificate, and refuse unless it is exactly the one the mesh pinned.
* A certificate authority is not consulted: the mesh issued this and knows its fingerprint,
* which is stronger than trusting whoever a machine's trust store happens to contain.
*
* **The pin is the only check.** What comes back is handed to the client as its TLS options, and
* the client's transport spreads them into Node's own `tls.connect` — so the pinned certificate
* is the one authority the handshake accepts, and the hostname check beside it is replaced with
* one that always passes. Pinning the exact certificate makes verifying its name redundant, and
* the bus's certificate names the seat (`mesh-broker`), not the address a machine happens to
* dial it by: every module on the mesh met "does not match certificate's altnames" the first time
* it reached the handshake (2026-09-28).
*/
async function pinnedTls(rawUrl: string, fingerprint: string): Promise<TlsOptions> {
const url = new URL(rawUrl.includes("://") ? rawUrl : `nats://${rawUrl}`);
const port = url.port ? Number(url.port) : 4222;
// **The bus speaks first, in the clear.** A NATS server sends its INFO line before TLS begins,
// and only then expects the client to start the handshake; a raw TLS connect to that port reads
// the INFO line as a TLS record and fails with "wrong version number" — which is what every
// module met the first time it dialled the bus being built (2026-09-28). So: connect, wait for
// INFO, then start TLS on the same socket, and read the certificate the server presents.
const certificate = await new Promise<tls.DetailedPeerCertificate>((resolve, reject) => {
const plain = net.connect({ host: url.hostname, port }, () => {});
let seenInfo = false;
let buffered = "";
plain.on("error", reject);
plain.on("data", (chunk: Buffer) => {
if (seenInfo) return;
buffered += chunk.toString("utf8");
if (!buffered.includes("\r\n")) return;
seenInfo = true;
plain.removeAllListeners("data");
const secure = tls.connect(
{ socket: plain, rejectUnauthorized: false, servername: url.hostname },
() => {
const peer = secure.getPeerCertificate(true);
secure.end();
resolve(peer);
},
);
secure.on("error", reject);
});
});
const seen = createHash("sha256").update(certificate.raw).digest("hex");
if (seen !== normalizeFingerprint(fingerprint)) {
throw new PinMismatchError(
`the bus at ${url.hostname}:${port} presented ${seen}, not the pinned ${normalizeFingerprint(fingerprint)}`,
);
}
const pem = `-----BEGIN CERTIFICATE-----\n${certificate.raw.toString("base64").replace(/(.{64})/g, "$1\n")}\n-----END CERTIFICATE-----\n`;
// Node's option, not the client's: the transport passes the whole object on. `undefined` from
// checkServerIdentity is "the name is fine"; the pin above already decided the rest.
return { ca: pem, checkServerIdentity: () => undefined } as TlsOptions;
}
/** The mesh's topic matching: `*` is one token, `#` the rest. This is the module's vocabulary —
* a module's `consumes` pattern is matched here, and the subject it becomes is the mesh's
* business, not the module's. */
export function topicMatches(pattern: string, key: string): boolean {
return matchFrom(pattern.split("."), 0, key.split("."), 0);
}
function matchFrom(p: string[], pi: number, k: string[], ki: number): boolean {
if (pi === p.length) return ki === k.length;
// `**` is the mesh's wildcard for the rest of a name; `#` is the old bus's, accepted so a pattern
// written either way behaves the same while both buses ship (novox/hq design 29 §1).
if (p[pi] === "#" || p[pi] === "**") {
for (let skip = ki; skip <= k.length; skip++) {
if (matchFrom(p, pi + 1, k, skip)) return true;
}
return false;
}
if (ki === k.length) return false;
if (p[pi] !== "*" && p[pi] !== k[ki]) return false;
return matchFrom(p, pi + 1, k, ki + 1);
}
+280
View File
@@ -0,0 +1,280 @@
/**
* The mesh's tools, for whoever is on a machine (novox/hq design 25 §7, design 34).
*
* Two surfaces over one thing. A command line, for somebody at a terminal; an MCP server, for an
* agent. Both are adapters over the same three calls — what tools are there, what does this one take,
* call it — because a second way of reaching a tool is a second thing to keep correct.
*
* **It uses the same client a module's runtime uses.** Not a second protocol and not a bridge: the
* caller connects as its own bus user — a person's, or the console's — publishes on the tool subjects
* that account permits, and the server refuses anything else. So "what may this ask" is answered by the
* same permission list that answers it for a module, and there is nothing here for an audit to read
* separately.
*
* What the caller may NOT do is the more interesting half, and none of it is enforced here — it is the
* account (design 25 §4): it cannot publish an event, so it cannot claim a module said something; it
* has no consumer, so there is no delivery to acknowledge; and it cannot answer a request, so it cannot
* impersonate a module on a bus where anyone may serve a tool.
*/
import { readFile } from "node:fs/promises";
import type { Broker } from "@novox/mesh-sdk/messaging";
import { connectNats, type Answered, type Credential } from "./broker-nats.js";
import { TOOLS_VERB, type ToolsAnswer } from "./runtime.js";
/** Where the catalogue answers which modules the mesh holds. */
const CATALOGUE_MODULES = "mesh-catalog.catalog_modules";
/** Where the mesh answers every role's tools, from its records: the mesh-controller seat's own
* `tools` verb (novox/hq ADR 0154, design 33 §5). */
const SEAT_TOOLS = "seat:mesh-controller.tools";
/** A tool as its module describes it. */
export interface Tool {
/** The module that serves it — or, for a role's tool, the seat. */
module: string;
name: string;
description?: string;
/** The JSON schema of what it takes, as the module declared it. */
input?: unknown;
/** True for a role's tool: addressed to the seat, answered by whoever holds it (ADR 0132). */
seat?: boolean;
/** A role's scope: `node` for a seat held once per machine, whose verb is asked of one machine
* (design 33 §4) and takes `node` for it; `mesh` or absent otherwise. */
scope?: string;
/** Where the tool is answered, as the mesh issued it (ADR 0160): the plain subject first when the
* module answers for itself anywhere, then one per machine. Absent for a runtime older than this. */
subjects?: string[];
}
/**
* What the mesh could say about its tools when asked (design 34 §3).
*
* **Silence is named, never dropped.** A module the catalogue holds and nothing answered for is in
* `notAnswering`, because a tool that is not offered looks exactly like a tool that does not exist,
* and those need different people to fix them.
*/
export interface Listing {
tools: Tool[];
/** Modules the catalogue holds whose runtime did not answer `tools`: not assigned, not up, or built
* before the runtime answered it. Each may still be called by name. The mesh's own records are
* listed here as `mesh-controller (seat)` when the control plane did not answer. */
notAnswering: string[];
}
/** The seats and the verbs each declares, from the last listing, so a call can tell a role's tool
* from a module's when the two share a prefix (a module and a seat may share a name). */
export type Seats = Map<string, Set<string>>;
/** The roles' tools, keyed the way `toolKey` names them. */
export function seatsIn(have: Listing): Seats {
const seats: Seats = new Map();
for (const t of have.tools) {
if (!t.seat) continue;
if (!seats.has(t.module)) seats.set(t.module, new Set());
seats.get(t.module)!.add(t.name);
}
return seats;
}
/** The key a call uses for `<prefix>.<name>`: a role's when the prefix is a seat declaring that
* verb, a module's otherwise. Both names for one capability are deliberate and bounded (ADR 0132);
* the seat wins only for a verb it actually declares, so a module's own tool is never shadowed. */
export function toolKey(name: string, seats?: Seats): string {
if (name.startsWith("seat:")) return name;
const dot = name.indexOf(".");
if (dot < 0) return name;
const prefix = name.slice(0, dot);
const verb = name.slice(dot + 1);
if (seats?.get(prefix)?.has(verb)) return `seat:${prefix}.${verb}`;
return name;
}
/**
* A person's credential, as `operator issue` prints it.
*
* The same shape a module is handed, minus the parts a module needs and a person does not: no node,
* because a person is not on a machine, and no module, because they are not one.
*/
export interface PersonCredential extends Credential {
person?: string;
invokes?: string[];
}
/** Read the credential from the file `operator issue` produced. */
export async function credentialFrom(path: string): Promise<PersonCredential> {
const raw = await readFile(path, "utf8");
let held: PersonCredential;
try {
held = JSON.parse(raw) as PersonCredential;
} catch (e) {
throw new Error(
`${path} is not a credential this mesh issued: ${(e as Error).message}. ` +
"It is the JSON `operator issue` printed, saved verbatim.",
);
}
if (!held.url || !held.user || !held.password) {
throw new Error(
`${path} names no bus, user or password. It is the JSON \`operator issue\` printed, saved ` +
"verbatim — not an edited copy of it.",
);
}
return held;
}
/** Connect as this person. The module name the runtime wants is their own user, because every subject
* it derives is for a tool somebody else serves. */
export async function connectAs(held: PersonCredential): Promise<Broker> {
return connectNats({ ...held, module: held.user });
}
/**
* Connect as the console: the module credential the mesh delivered (novox/hq ADR 0152), read from
* the same variable every runtime reads. It names the node and the module, so the account's inbox
* and subjects derive from what the mesh authorised and from nothing in this process's environment.
*/
export async function connectAsTheConsole(path: string): Promise<{ bus: Broker; who: string }> {
const raw = await readFile(path, "utf8");
let held: Credential;
try {
held = JSON.parse(raw) as Credential;
} catch (e) {
throw new Error(`${path} is not a broker credential: ${(e as Error).message}`);
}
if (!held.url || !held.module || !held.user) {
throw new Error(
`${path} names no bus, module or user: the console runs on the credential the mesh sealed to ` +
"this machine for it, and nothing else",
);
}
return { bus: await connectNats(held), who: `${held.node ?? "?"}.${held.module}` };
}
/**
* What tools the mesh has, asked of the modules (design 34 §3).
*
* The catalogue says which modules the mesh holds; each module says what it serves, through the one
* verb its runtime answers for it. **Asked, not configured**: a client carrying its own list would be a
* list that goes stale the first time a module is assigned. Every module is asked at once, and the bus
* refuses at once a request nothing serves, so the cost is bounded by the modules that are up.
*/
export async function toolsOn(bus: Broker): Promise<Listing> {
const [answered, roles] = await Promise.all([
bus.request<Record<string, never>, { modules?: { module: string }[] }>(CATALOGUE_MODULES, {}),
// The roles' tools, from the mesh's records (design 33 §5). Asked beside the modules rather
// than first: a control plane that is restarting must not hide every module's tools with it.
bus
.request<Record<string, never>, { seats?: { seat: string; scope?: string; tools?: ToolsAnswer["tools"] }[] }>(
SEAT_TOOLS,
{},
)
.catch(() => undefined),
]);
const names = (answered.modules ?? []).map((m) => m.module).filter((m) => typeof m === "string");
const asked = await Promise.allSettled(
names.map((module) => bus.request<Record<string, never>, ToolsAnswer>(`${module}.${TOOLS_VERB}`, {})),
);
const tools: Tool[] = [];
const notAnswering: string[] = [];
if (roles) {
for (const s of roles.seats ?? []) {
// A node-scoped seat's tool is asked of one machine (design 33 §4): listed with its scope, so
// a caller names the machine and the call carries it — `seat:<seat>.<verb>@<node>`. Left out
// of the listing, the verb never resolved as a seat's and nothing served it (ADR 0170).
for (const t of s.tools ?? []) {
tools.push({ module: s.seat, name: t.name, description: t.description, input: t.input, seat: true, scope: s.scope });
}
}
} else {
notAnswering.push("mesh-controller (seat)");
}
asked.forEach((outcome, i) => {
const module = names[i]!;
if (outcome.status === "fulfilled" && typeof outcome.value?.failed === "string") {
// The runtime answered for it and serves nothing: the bundle failed to load (ADR 0175). Said
// with the reason, because "not answering" would send somebody to check an assignment that
// is fine.
notAnswering.push(`${module} (its tools bundle failed to load: ${outcome.value.failed})`);
} else if (outcome.status === "fulfilled" && Array.isArray(outcome.value?.tools)) {
for (const t of outcome.value.tools) {
tools.push({ module, name: t.name, description: t.description, input: t.input, subjects: t.subjects });
}
} else {
notAnswering.push(module);
}
});
tools.sort((a, b) => `${a.module}.${a.name}`.localeCompare(`${b.module}.${b.name}`));
notAnswering.sort();
return { tools, notAnswering };
}
/** Call one tool. The key is `<module>.<tool>`, which is what a person types and what the account
* permits — one vocabulary, so a refusal names the thing they asked for. */
/** Call a tool and learn which machine answered (novox/hq ADR 0159). `<module>.<tool>@<node>` asks
* the instance on one machine; without it, whichever instance answers first does, and the answer
* says which. */
export async function callTool(
bus: Broker,
key: string,
args: unknown,
seats?: Seats,
listing?: Listing,
): Promise<Answered<unknown>> {
const at = key.indexOf("@");
const name = at < 0 ? key : key.slice(0, at);
const node = at < 0 ? "" : key.slice(at + 1);
if (!name.includes(".")) {
throw new Error(
`"${key}" does not name a tool: write <module>.<tool>, as \`mesh tools\` lists them, ` +
"or <module>.<tool>@<node> for the instance on one machine",
);
}
const resolved = toolKey(name, seats) + (node ? `@${node}` : "");
// Where the tool is answered is the module's to say and the mesh's to issue (ADR 0160): when the
// listing carried subjects for it, the call goes to one of those and composes nothing.
const on = subjectListed(name, node, listing);
const asking = bus as Broker & { ask?: <Req, Res>(k: string, b: Req, on?: string) => Promise<Answered<Res>> };
if (typeof asking.ask === "function") return asking.ask<unknown, unknown>(resolved, args ?? {}, on);
return { result: await bus.request<unknown, unknown>(resolved, args ?? {}) };
}
/** The subject the listing says answers `<module>.<tool>` — the machine's when one is named, else
* the plain one — or undefined when the listing said none, and the key is composed as before. */
export function subjectListed(name: string, node: string, listing?: Listing): string | undefined {
if (!listing) return undefined;
const dot = name.indexOf(".");
const module = name.slice(0, dot);
const tool = name.slice(dot + 1);
const found = listing.tools.find((t) => t.module === module && t.name === tool && !t.seat);
const subjects = found?.subjects ?? [];
if (subjects.length === 0) return undefined;
if (node) return subjects.find((s) => s.endsWith(`.${node}`));
return subjects[0];
}
/**
* Why a call failed, said so that the remedy is in the words.
*
* Three answers a person actually gets, and they need different things done: nobody serves that tool,
* the mesh refused this account, or the tool itself failed. Without this they are one timeout and a
* stack trace.
*/
export function whyItFailed(key: string, err: unknown): string {
const message = err instanceof Error ? err.message : String(err);
if (/no responders|503/i.test(message)) {
return `nothing serves ${key}. The module may not be assigned to any machine, or it is down` +
(key.startsWith("seat:") ? ", or nothing holds that seat" : "") +
" — `mesh tools` lists what answered.";
}
if (/permissions violation|authorization/i.test(message)) {
return `this account may not call ${key}. What it may call was fixed when it was issued — a ` +
"person's by `operator issue`, the console's by its manifest.";
}
if (/timeout/i.test(message)) {
return `${key} did not answer in time. Something is serving it, so this is the tool being slow ` +
"rather than absent.";
}
return `${key} failed: ${message}`;
}
+165
View File
@@ -0,0 +1,165 @@
/**
* The console's endpoint: MCP over HTTP, on a machine's loopback (novox/hq ADR 0152, design 34 §2).
*
* **Loopback is the authority boundary.** Whoever can connect is on the machine, and whoever is on the
* machine is the account that owns the mesh there (ADR 0034, ADR 0144). So there is no token and no
* login here, and the one thing this file enforces is that it binds nothing else: a console reachable
* from another machine would be authority over the mesh handed to whoever finds the port.
*
* The transport is the streamable-HTTP shape an agent host speaks: `POST /mcp` with one JSON-RPC
* message, answered with one JSON body. No session, because the surface holds nothing per caller; no
* event stream, because nothing here has anything to say unasked.
*/
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
import type { Broker } from "@novox/mesh-sdk/messaging";
import { mcpSurface, type Reply, type Request } from "./mcp.js";
/** The most a request body may be. A tool's arguments are small; a megabyte is somebody else's file. */
const BODY_LIMIT = 1 << 20;
export interface Listening {
/** Where it listens, as `host:port`, with the port the machine actually gave. */
address: string;
close(): Promise<void>;
}
/** Hosts that are this machine and no other. */
const loopback = new Set(["127.0.0.1", "::1", "localhost", "[::1]"]);
/**
* Listen on `host:port`. Refused unless the host is loopback — said before binding, so a manifest or
* a flag that would open the console to a network is a startup failure rather than something
* discovered by whoever finds it.
*/
export async function serveMcpHttp(bus: Broker, who: string, listen: string): Promise<Listening> {
const at = listen.lastIndexOf(":");
if (at < 0) {
throw new Error(`"${listen}" is not host:port`);
}
const host = listen.slice(0, at);
const port = Number(listen.slice(at + 1));
if (!loopback.has(host)) {
throw new Error(
`the console listens on loopback and nowhere else (novox/hq ADR 0152): "${host}" is not this ` +
"machine's own address — whoever is on the machine owns the mesh there, and nobody else may reach this",
);
}
if (!Number.isInteger(port) || port < 0 || port > 65535) {
throw new Error(`"${listen.slice(at + 1)}" is not a port`);
}
const surface = mcpSurface(bus, who);
const server = createServer((req, res) => {
void route(req, res, surface.handle).catch((e) => {
json(res, 500, { jsonrpc: "2.0", id: null, error: { code: -32603, message: String(e) } });
});
});
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(port, host.replace(/^\[|\]$/g, ""), () => resolve());
});
const bound = server.address();
const address = typeof bound === "object" && bound ? `${host}:${bound.port}` : listen;
return {
address,
close: () =>
new Promise<void>((resolve) => {
server.close(() => resolve());
}),
};
}
async function route(
req: IncomingMessage,
res: ServerResponse,
handle: (r: Request) => Promise<Reply | undefined>,
): Promise<void> {
const path = (req.url ?? "/").split("?")[0];
if (path === "/") {
res.writeHead(200, { "content-type": "text/plain; charset=utf-8" });
res.end("the mesh's console: MCP over HTTP at POST /mcp (novox/hq design 34)\n");
return;
}
if (path !== "/mcp") {
json(res, 404, { error: "the console serves /mcp and nothing else" });
return;
}
switch (req.method) {
case "POST":
break;
case "DELETE":
// A host ending a session. There is no session to end; saying so is the truthful answer.
res.writeHead(204).end();
return;
case "GET":
// A host opening an event stream. The console has nothing to say unasked.
res.writeHead(405, { allow: "POST, DELETE" }).end();
return;
default:
res.writeHead(405, { allow: "POST, DELETE" }).end();
return;
}
let body: string;
try {
body = await read(req);
} catch (e) {
json(res, 413, { jsonrpc: "2.0", id: null, error: { code: -32600, message: String(e) } });
return;
}
let parsed: unknown;
try {
parsed = JSON.parse(body);
} catch {
json(res, 400, { jsonrpc: "2.0", id: null, error: { code: -32700, message: "the body is not JSON" } });
return;
}
// One message, or a batch of them; a batch is answered as a batch. A notification gets no reply
// and, alone, no body: 202 is how the transport says "heard".
if (Array.isArray(parsed)) {
const replies = (await Promise.all(parsed.map((r) => handle(r as Request)))).filter(Boolean);
if (replies.length === 0) {
res.writeHead(202).end();
} else {
json(res, 200, replies);
}
return;
}
const reply = await handle(parsed as Request);
if (!reply) {
res.writeHead(202).end();
return;
}
json(res, 200, reply);
}
function read(req: IncomingMessage): Promise<string> {
return new Promise((resolve, reject) => {
let size = 0;
const chunks: Buffer[] = [];
req.on("data", (chunk: Buffer) => {
size += chunk.length;
if (size > BODY_LIMIT) {
reject(new Error(`the request is larger than ${BODY_LIMIT} bytes`));
req.destroy();
return;
}
chunks.push(chunk);
});
req.on("end", () => resolve(Buffer.concat(chunks).toString("utf8")));
req.on("error", reject);
});
}
function json(res: ServerResponse, status: number, body: unknown): void {
const text = JSON.stringify(body);
res.writeHead(status, {
"content-type": "application/json; charset=utf-8",
"content-length": Buffer.byteLength(text),
});
res.end(text);
}
+192
View File
@@ -0,0 +1,192 @@
// A tools bundle as a process the runtime launches (novox/hq ADR 0188).
//
// The runtime does not run a tool's code itself when the bundle is not JavaScript: it starts the
// bundle's executable as a child with the runtime's environment and speaks MCP over stdio to it —
// `initialize`, `tools/list` once, `tools/call` per call. A Rust binary, a Go binary, a Python
// script and a Node script are the same thing from here: a process that answers those. Everything
// the mesh adds — the subjects from the membership, the held seats, the `tools` answer, a bundle
// that failed named and the others serving — is the runtime's, outside this file.
//
// A tool the child lists as `<seat>.<verb>` is the module's implementation of that seat's verb;
// any other name is the module's own tool. The same rule the in-process registration follows.
import { spawn, type ChildProcess } from "node:child_process";
import { accessSync, constants } from "node:fs";
import type { ToolDefinition } from "@novox/mesh-sdk/tools";
import { broker, type Envelope } from "@novox/mesh-sdk/messaging";
import { atWork } from "./broker-nats.js";
/** The protocol version this speaks; a bundle says the same. */
export const PROTOCOL = "2025-03-26";
/** How long a child has to answer `initialize` and `tools/list` before it is a failed bundle, and
* how long a call may take before the caller is told the tool is slow rather than absent. */
const HANDSHAKE_MS = 10_000;
const CALL_MS = 30_000;
/** Whether an entrypoint is launched as a process rather than imported: anything that is not a
* plain JavaScript file, and a JavaScript file marked executable — a bundle written against the
* protocol in TypeScript, served the same way as any other language. */
export function launches(entry: string): boolean {
const javascript = /\.(m|c)?js$/.test(entry);
let executable = false;
try {
accessSync(entry, constants.X_OK);
executable = true;
} catch {
// not executable, or not there — importing will say which
}
return !javascript || executable;
}
/** What a launched bundle registers: the groups the in-process path would have, by name. */
export interface Launched {
registrations: { module: string; tools: ToolDefinition[] }[];
stop(): void;
}
interface Pending {
resolve(v: any): void;
reject(e: Error): void;
timer: NodeJS.Timeout;
}
/**
* Launch a bundle and learn its tools. Rejects when the child cannot be started or does not complete
* the handshake, which the runtime records as the bundle having failed. A child that exits later is
* started again on the next call, once; a call in flight when it died is told so.
*/
export async function launch(module: string, entry: string, env: NodeJS.ProcessEnv = process.env): Promise<Launched> {
let child: ChildProcess | undefined;
let nextId = 1;
const pending = new Map<number, Pending>();
let stopped = false;
const start = async (): Promise<void> => {
const proc = spawn(entry, [], { stdio: ["pipe", "pipe", "pipe"], env });
child = proc;
let buffered = "";
proc.stdout!.on("data", (chunk: Buffer) => {
buffered += chunk.toString("utf8");
let at: number;
while ((at = buffered.indexOf("\n")) >= 0) {
const line = buffered.slice(0, at).trim();
buffered = buffered.slice(at + 1);
if (!line) continue;
let reply: { id?: number | string; method?: string; params?: unknown; result?: unknown; error?: { message?: string } };
try {
reply = JSON.parse(line);
} catch {
console.log(`[mesh-tools] ${module}'s bundle said something that is not a reply: ${line.slice(0, 120)}`);
continue;
}
// **The bundle asks the runtime to emit** (novox/hq ADR 0193): published on the bus as this
// module, and answered once the bus has accepted it, so the tool's emit means what it means
// in-process. Nothing else a bundle may ask.
if (typeof reply.method === "string") {
const id = reply.id;
const answer = (m: Record<string, unknown>) => proc.stdin!.write(JSON.stringify({ jsonrpc: "2.0", id, ...m }) + "\n");
if (reply.method !== "mesh/publish") {
if (id !== undefined) answer({ error: { code: -32601, message: `the runtime answers no ${reply.method} from a bundle` } });
continue;
}
atWork.run({ module }, () => broker().publish(reply.params as Envelope<unknown>))
.then(() => { if (id !== undefined) answer({ result: {} }); })
.catch((err: unknown) => { if (id !== undefined) answer({ error: { code: -32000, message: err instanceof Error ? err.message : String(err) } }); });
continue;
}
const waiting = typeof reply.id === "number" ? pending.get(reply.id) : undefined;
if (!waiting) continue;
pending.delete(reply.id as number);
clearTimeout(waiting.timer);
if (reply.error) waiting.reject(new Error(reply.error.message ?? "the bundle refused the request"));
else waiting.resolve(reply.result);
}
});
// stderr is the bundle's log; kept under the module's name so a fault reads where it belongs.
// The last thing it said is kept, so a bundle that dies says why in its own words, not by code.
let lastSaid = "";
proc.stderr!.on("data", (chunk: Buffer) => {
for (const line of chunk.toString("utf8").split("\n")) {
if (!line.trim()) continue;
console.log(`[${module}] ${line}`);
if (/\S/.test(line) && !/^\s+at\s/.test(line) && !/^Node\.js v/.test(line)) lastSaid = line.trim();
}
});
const exited = new Promise<never>((_, reject) => {
proc.once("error", (err) => reject(err));
proc.once("exit", (code, signal) => {
const why = `${module}'s bundle exited (${signal ?? code})` + (lastSaid ? `: ${lastSaid}` : "");
for (const [id, p] of pending) {
pending.delete(id);
clearTimeout(p.timer);
p.reject(new Error(why));
}
if (child === proc) child = undefined;
if (!stopped) console.log(`[mesh-tools] ${why}; started again on its next call`);
reject(new Error(why));
});
});
const ask = (method: string, params: unknown, ms: number): Promise<any> =>
Promise.race([
new Promise<any>((resolve, reject) => {
const id = nextId++;
const timer = setTimeout(() => {
pending.delete(id);
reject(new Error(`${module}'s bundle did not answer ${method} in ${ms / 1000}s`));
}, ms);
pending.set(id, { resolve, reject, timer });
proc.stdin!.write(JSON.stringify({ jsonrpc: "2.0", id, method, params }) + "\n");
}),
exited,
]);
exited.catch(() => {}); // observed through the race; never unhandled
(proc as ChildProcess & { ask?: typeof ask }).ask = ask;
await ask("initialize", { protocolVersion: PROTOCOL, capabilities: {}, clientInfo: { name: "node-tools", version: "1" } }, HANDSHAKE_MS);
proc.stdin!.write(JSON.stringify({ jsonrpc: "2.0", method: "notifications/initialized" }) + "\n");
};
const asking = async (method: string, params: unknown, ms: number): Promise<any> => {
if (!child) await start();
return (child as ChildProcess & { ask: (m: string, p: unknown, ms: number) => Promise<any> }).ask(method, params, ms);
};
await start();
const listed = (await asking("tools/list", {}, HANDSHAKE_MS)) as { tools?: { name: string; description?: string; inputSchema?: unknown }[] };
const groups = new Map<string, ToolDefinition[]>();
for (const t of listed.tools ?? []) {
const dot = t.name.indexOf(".");
const under = dot < 0 ? module : t.name.slice(0, dot);
const name = dot < 0 ? t.name : t.name.slice(dot + 1);
const tools = groups.get(under) ?? [];
tools.push({
name,
description: t.description ?? "",
input: (t.inputSchema as Record<string, unknown> | undefined) ?? {},
run: async (args) => {
const result = (await asking("tools/call", { name: t.name, arguments: args ?? {} }, CALL_MS)) as {
content?: { type: string; text?: string }[];
isError?: boolean;
};
const text = result?.content?.find((c) => c.type === "text")?.text ?? "";
if (result?.isError) throw new Error(text || `${module}.${t.name} failed`);
// The bundle's answer is JSON as text (that is what every MCP host renders); handed back as
// the value it encodes so a caller on the bus sees what an in-process tool would return.
try {
return JSON.parse(text);
} catch {
return text;
}
},
});
groups.set(under, tools);
}
return {
registrations: [...groups].map(([under, tools]) => ({ module: under, tools })),
stop: () => {
stopped = true;
child?.kill("SIGTERM");
child = undefined;
},
};
}
+128 -15
View File
@@ -1,10 +1,14 @@
// The runnable entrypoint. Three modes:
//
// mesh-tools serve — bind the broker and serve the assigned modules until
// stopped. A module entrypoint that subscribes to events (on("#"))
// stopped; as the node-tools module, also the console on loopback. A module entrypoint that subscribes to events (on("#"))
// starts consuming as it is imported, so this also runs consumers.
// mesh-tools emit TYPE [JSON] emit one event onto the mesh and exit — an operable primitive,
// and what an events test uses to put a message on the wire.
// mesh-tools prepare bring this module's state to the shape this version needs and exit
// — the runtime's answer to the word the mesh asks every module
// (novox/hq ADR 0135). The entrypoints come from MESH_PREPARE, which
// the module's own image names beside MESH_TOOL_MODULES.
// mesh-tools run ENTRYPOINT run one compiled module entrypoint to completion and exit — the
// runtime side of a run-once step (novox/hq ADR 0052). It imports
// the given entrypoint, whose top-level code does its work — seed a
@@ -17,14 +21,19 @@
// MESH_BROKER_FILE a sealed {url, fingerprint} the mesh delivered (novox/hq ADR 0043) — an
// amqps account scoped to this module. Preferred: a module holds its own.
// MESH_BROKER_URL a plain URL, for the bootstrap/admin case before a module has an account.
// MESH_TOOL_MODULES /path/a,/path/b,… compiled module entrypoints (serve mode)
// MESH_TOOL_MODULES <module>=/path/a,… the modules to serve and their compiled entrypoints (serve
// mode; ADR 0175); a bare path is an entrypoint of the credential's own module
// MESH_PREPARE /path/a,/path/b,… compiled entrypoints that prepare this module's state
// MESH_MODULE / MESH_NODE the identity stamped onto emitted events (ADR 0042)
import { readFileSync } from "node:fs";
import { pathToFileURL } from "node:url";
import { connectAmqp, fatalBrokerReason } from "./broker-amqp.js";
import type { Credential } from "./broker-amqp.js";
import { runTools } from "./runtime.js";
import { connectNats, fatalBrokerReason as fatalNatsReason, type Credential } from "./broker-nats.js";
import { takeToolEnvs, runTools, type ServedModule } from "./runtime.js";
import { serveMcpHttp, type Listening } from "./http.js";
/** The credential this process connected with, for what it says beyond the connection (ADR 0159). */
let lastCredential: Credential | undefined;
import { invokeTool } from "@novox/mesh-sdk/tools";
import { useBroker } from "@novox/mesh-sdk/messaging";
import type { Broker } from "@novox/mesh-sdk/messaging";
@@ -35,12 +44,19 @@ import { emit } from "@novox/mesh-sdk/events";
* over the plain bootstrap URL otherwise. A scoped module assumes the foundation's exchanges exist —
* its account may not declare them (ADR 0043).
*/
// fatalBrokerReasonFor is the reason a connection failure is final rather than "not yet", for
// whichever bus this runtime is on — each transport knows its own refusals.
function fatalBrokerReasonFor(err: unknown): string | null {
return fatalNatsReason(err);
}
async function connectBroker(): Promise<Broker> {
const file = process.env.MESH_BROKER_FILE;
if (file) {
let credential: Credential;
try {
credential = JSON.parse(readFileSync(file, "utf8")) as Credential;
lastCredential = credential;
} catch (err) {
console.error(`mesh-tools: cannot read the broker credential at ${file}: ${err}`);
process.exit(1);
@@ -54,7 +70,13 @@ async function connectBroker(): Promise<Broker> {
// what the environment says.
if (credential.node) process.env.MESH_NODE = credential.node;
if (credential.module) process.env.MESH_MODULE = credential.module;
return connectAmqp(credential, { assumeExchanges: true });
// **The credential names the bus.** A module moved to the bus being built was handed a
// credential for it — `nats://…` with user, password and fingerprint beside the address — and
// nothing else in its environment changed (design 25; novox/hq design 28 task 5.2). The scheme
// is enough to know which bus to speak; a runtime that always dialled the old one would keep
// serving and answer nobody.
// One bus (novox/hq ADR 0131, design 28 task 5.5): the credential names it, and it is this.
return connectNats(credential);
}
const url = process.env.MESH_BROKER_URL;
if (!url) {
@@ -63,7 +85,7 @@ async function connectBroker(): Promise<Broker> {
);
process.exit(1);
}
return connectAmqp(url);
return connectNats({ url });
}
/**
@@ -83,7 +105,7 @@ async function connectBrokerPatiently(): Promise<Broker> {
try {
return await connectBroker();
} catch (err) {
const fatal = fatalBrokerReason(err);
const fatal = fatalBrokerReasonFor(err);
if (fatal !== null) {
console.error(`mesh-tools: ${fatal} — waiting will not fix this; giving up`);
throw err;
@@ -98,17 +120,68 @@ async function connectBrokerPatiently(): Promise<Broker> {
}
}
async function serve(): Promise<void> {
const moduleEntrypoints = (process.env.MESH_TOOL_MODULES ?? "")
.split(",")
.map((s) => s.trim())
.filter(Boolean);
/**
* What MESH_TOOL_MODULES names (novox/hq ADR 0175, to-be 38 WP1): `<module>=<entrypoint>` entries,
* comma-separated, several per module allowed — the node's runtime serving every assigned module's
* bundle. A bare path is the one-module form the per-module containers still set: an entrypoint of
* the credential's own module. Both may appear; the result is one list of modules.
*/
export function servedModulesFrom(spec: string, own: string | undefined): { serves: ServedModule[]; moduleEntrypoints: string[] } {
const serves = new Map<string, string[]>();
const moduleEntrypoints: string[] = [];
for (const raw of spec.split(",")) {
const entry = raw.trim();
if (!entry) continue;
const eq = entry.indexOf("=");
if (eq < 0) {
moduleEntrypoints.push(entry);
continue;
}
const module = entry.slice(0, eq).trim();
const path = entry.slice(eq + 1).trim();
if (!module || !path) {
throw new Error(`MESH_TOOL_MODULES: "${entry}" is neither <module>=<entrypoint> nor an entrypoint of this module`);
}
if (module === own) {
moduleEntrypoints.push(path);
continue;
}
serves.set(module, [...(serves.get(module) ?? []), path]);
}
return { serves: [...serves].map(([module, entrypoints]) => ({ module, entrypoints })), moduleEntrypoints };
}
/** The module that is the node's tool runtime (novox/hq ADR 0175, to-be 38 WP3): on its credential,
* `serve` is also the console — MCP on the machine's loopback (design 34). */
export const RUNTIME_MODULE = "node-tools";
/** Where the console listens when the runtime is node-tools and nothing says otherwise: the port
* the module's manifest declares `from: machine`. MESH_CONSOLE_LISTEN overrides it either way. */
const CONSOLE_LISTEN = "127.0.0.1:4270";
async function serve(): Promise<void> {
const broker = await connectBrokerPatiently();
const stop = await runTools({ broker, moduleEntrypoints });
// Parsed after connecting: a bare entrypoint belongs to the module the credential names.
const { serves, moduleEntrypoints } = servedModulesFrom(process.env.MESH_TOOL_MODULES ?? "", lastCredential?.module);
// Each module's environment, composed by the mesh (ADR 0192): taken before any bundle is imported.
const envs = takeToolEnvs();
const stop = await runTools({ broker, serves, moduleEntrypoints, credential: lastCredential, envs });
// The console is this runtime's serving mode (ADR 0175 §6): as node-tools, or wherever the
// listen address is given, the same process answers MCP on loopback for whoever is on the
// machine. A module's own runtime in a container on the machine's network does not — two of
// them on one port would be the fault, and the console is one per machine.
const listen = process.env.MESH_CONSOLE_LISTEN ?? (lastCredential?.module === RUNTIME_MODULE ? CONSOLE_LISTEN : "");
let consoleUp: Listening | undefined;
if (listen) {
const who = `${lastCredential?.node ?? "?"}.${lastCredential?.module ?? RUNTIME_MODULE}`;
consoleUp = await serveMcpHttp(broker, who, listen);
console.log(`mesh console listening on http://${consoleUp.address}/mcp as ${who}`);
}
const shutdown = async (): Promise<void> => {
stop();
await consoleUp?.close();
await broker.close();
process.exit(0);
};
@@ -166,12 +239,49 @@ async function runEntry(entrypoint: string): Promise<void> {
await import(pathToFileURL(entrypoint).href);
}
/**
* Bring this module's state to the shape this version needs, and exit — the runtime's answer to the
* one word the mesh asks every module (novox/hq ADR 0135).
*
* The entrypoints come from `MESH_PREPARE`, which a module's own image sets beside the entrypoints it
* already lists there: the module knows which of its files prepares its state, and nothing else could.
* Each is imported in the order given, to completion, with no broker — preparation runs before the
* version that would use it, so there is nothing yet to talk to.
*
* **An empty list is a failure, not a no-op.** The mesh only asks this of a module whose manifest says
* it prepares something; a module that says so and names nothing has been built wrong, and exiting 0
* would let that version serve against a state nobody shaped.
*/
async function prepareState(): Promise<void> {
const named = (process.env.MESH_PREPARE ?? "")
.split(",")
.map((entry) => entry.trim())
.filter((entry) => entry !== "");
if (named.length === 0) {
console.error(
"mesh-tools prepare: this module was asked to prepare its state and its image names nothing " +
"to do it with — set MESH_PREPARE to the compiled entrypoint(s) that prepare it, the way " +
"MESH_TOOL_MODULES names the ones it serves",
);
process.exit(1);
}
for (const entrypoint of named) {
console.log(`[mesh-tools] preparing with ${entrypoint}`);
await runEntry(entrypoint);
}
console.log(`[mesh-tools] prepared: ${named.length} entrypoint(s) ran to completion`);
}
async function main(): Promise<void> {
const [command, ...rest] = process.argv.slice(2);
if (command === "run") {
await runEntry(rest[0] ?? "");
return;
}
if (command === "prepare") {
await prepareState();
return;
}
if (command === "invoke") {
const [module, tool] = rest;
if (!module || !tool) {
@@ -193,4 +303,7 @@ async function main(): Promise<void> {
await serve();
}
void main();
// Only when run, so a test can import the pieces.
if (process.argv[1] && import.meta.url === new URL(`file://${process.argv[1]}`).href) {
void main();
}
+258
View File
@@ -0,0 +1,258 @@
/**
* The mesh's tools as an MCP server (novox/hq design 25 §7, design 34).
*
* **A thin adapter and nothing more.** Every tool an agent sees is one a module answered for and one
* this account may call; the schema is the module's own; the answer is the module's own. Nothing here
* decides anything, which is why it is short — an MCP surface that reshaped arguments or summarised
* answers would be a second definition of what a tool is, and the module's code is the first.
*
* Implemented against the protocol directly rather than through a library: the surface is three
* methods and one framing, and a dependency here would be a dependency on every machine.
*
* One handler, two transports. Over stdio for a program a person starts (`mesh mcp`), over HTTP on a
* machine's loopback for the console the mesh assigns there (`mesh serve`, http.ts). The handler does
* not know which asked.
*/
import type { Broker } from "@novox/mesh-sdk/messaging";
import { callTool, seatsIn, toolKey, toolsOn, whyItFailed, type Listing, type Seats } from "./client.js";
/** The protocol version this speaks. Stated, because a host that wants another should be told so
* rather than discovering it through a shape it did not expect. */
export const PROTOCOL = "2025-03-26";
export interface Request {
jsonrpc: string;
id?: number | string | null;
method: string;
params?: Record<string, unknown>;
}
export interface Reply {
jsonrpc: "2.0";
id: Request["id"];
result?: unknown;
error?: { code: number; message: string };
}
/** How long a fetched tool list is kept before the modules are asked again. An agent asks on every
* turn; the mesh changes on the order of minutes. */
export const LISTING_KEPT_MS = 30_000;
export interface Surface {
/** Answer one request; undefined for a notification, which expects none. */
handle(request: Request): Promise<Reply | undefined>;
}
/**
* The surface over one bus connection, as one account.
*
* The tool list is fetched when first asked and kept for a short while (design 34 §3): asking every
* module on every `tools/list` would fan out on every agent turn for something nobody changed, and
* never refreshing would hide a module assigned a moment ago.
*/
export function mcpSurface(bus: Broker, who: string): Surface {
let known: { listing: Listing; at: number } | undefined;
const answer = (id: Request["id"], result: unknown): Reply => ({ jsonrpc: "2.0", id, result });
const refuse = (id: Request["id"], code: number, message: string): Reply => ({
jsonrpc: "2.0",
id,
error: { code, message },
});
const listing = async (): Promise<Listing> => {
if (!known || Date.now() - known.at > LISTING_KEPT_MS) {
known = { listing: await toolsOn(bus), at: Date.now() };
}
return known.listing;
};
// The roles the last listing knew, so `<seat>.<verb>` resolves to the seat. Fetched once if a
// call arrives before any list did; a listing that failed leaves no roles, and the name is then
// a module's, which is the right fallback for a mesh whose control plane is away.
const roles = async (): Promise<Seats | undefined> => {
try {
return seatsIn(await listing());
} catch {
return undefined;
}
};
return {
async handle(request) {
// A notification has no id and expects no answer; `initialized` is the one every host sends.
const notification = request.id === undefined || request.id === null;
switch (request.method) {
case "initialize":
return answer(request.id, {
protocolVersion: PROTOCOL,
capabilities: { tools: {} },
serverInfo: { name: "mesh", version: "1" },
// Said in the handshake, because an agent that knows whose authority it is acting under
// can say so when a call is refused — and a refusal is the one thing here that is not
// the mesh's fault or the tool's.
instructions:
`These are the tools of a Novox mesh, reached as ${who}. Every call goes to the module ` +
`that serves it; what may be called was fixed when this account was issued, so a ` +
`refusal means the account, not the tool. The list is what the running modules ` +
`answered, plus every role's tools from the mesh's records — the mesh's own verbs ` +
`(mesh-controller.status, .push, .assign …) among them; a module that did not answer ` +
`is named in the list's _meta and can still be called by <module>.<tool>.`,
});
case "notifications/initialized":
return undefined;
case "ping":
return notification ? undefined : answer(request.id, {});
case "tools/list": {
let have: Listing;
try {
have = await listing();
} catch (e) {
return refuse(request.id, -32603, whyItFailed("mesh-catalog.catalog_modules", e));
}
return answer(request.id, {
tools: have.tools.map((t) => ({
name: `${t.module}.${t.name}`,
description: t.description ?? `${t.name}, served by ${t.module}`,
// The module's own schema, passed through — with `node`, the machine to ask when
// the module runs on several (novox/hq ADR 0159); a seat's verb takes none, the
// seat's scope decides. An empty object is a tool that takes nothing, which is a
// real answer and not a missing one.
inputSchema: t.seat && t.scope !== "node" ? asSchema(t.input)
: t.seat ? withNode(asSchema(t.input), "the machine whose seat answers; required, the seat is held once per machine", true)
: withNode(asSchema(t.input)),
})),
// Silence, named (design 34 §3): the modules the catalogue holds and nothing answered
// for. Not a tool, so not in `tools`; not dropped either.
_meta: { notAnswering: have.notAnswering },
});
}
case "tools/call": {
const given = String(request.params?.name ?? "");
const args = { ...((request.params?.arguments as Record<string, unknown> | undefined) ?? {}) };
// The machine, when the caller names one, travels in the subject and never reaches the
// module's arguments (novox/hq ADR 0159) — for a module's tool. A seat's verb takes no
// machine from the console (the seat's scope decides), so a `node` among its arguments
// is the verb's own, as `push` and `assign` take one, and is handed through untouched.
const have = await listing().catch(() => undefined);
const roles = have && seatsIn(have);
const bare = given.split("@", 1)[0];
const isSeatVerb = roles ? toolKey(bare, roles).startsWith("seat:") : false;
// A node-scoped seat's verb is asked of one machine (design 33 §4, ADR 0170): `node`
// names it and travels in the subject, as for a module's tool.
const nodeScoped = isSeatVerb && (have?.tools.some((t) => t.seat && t.scope === "node" &&
`${t.module}.${t.name}` === bare) ?? false);
const takesNode = !isSeatVerb || nodeScoped;
const node = takesNode && typeof args.node === "string" && args.node !== "" ? args.node : "";
if (takesNode) delete args.node;
if (nodeScoped && !node && !given.includes("@")) {
return refuse(request.id, -32602, `${given} is a machine's seat's verb: name the machine with \`node\``);
}
const name = node && !given.includes("@") ? `${given}@${node}` : given;
try {
const { result, node: answeredBy } = await callTool(bus, name, args, roles, have);
// Text, because that is what every host renders. The content is the module's answer
// as JSON, unshaped: an adapter that flattened it would be deciding what matters in
// somebody else's answer. Which machine answered follows it as its own line.
const content: { type: string; text: string }[] = [
{ type: "text", text: JSON.stringify(result, null, 2) },
];
if (answeredBy) content.push({ type: "text", text: `answered by ${answeredBy}` });
return answer(request.id, { content });
} catch (e) {
// **An error the agent can act on, not a stack.** isError rather than a protocol
// failure, because the call was well-formed and the mesh answered it — with a refusal,
// an absence or a fault, and the words say which.
return answer(request.id, {
content: [{ type: "text", text: whyItFailed(name, e) }],
isError: true,
});
}
}
default:
return notification ? undefined : refuse(request.id, -32601, `mesh's MCP surface has no ${request.method}`);
}
},
};
}
/**
* A module's declared input as a JSON schema an agent can read.
*
* The sdk keeps a tool's input opaque, and the catalogue's modules write it as a bare map of
* property to description — `{ module: { type, description } }` — which is the `properties` of a
* schema rather than a schema. Wrapped here when that is what arrived; passed through when a module
* already wrote a schema; an empty object when it declared nothing. The module's words are kept
* either way.
*/
export function asSchema(input: unknown): Record<string, unknown> {
if (!input || typeof input !== "object" || Array.isArray(input)) {
return { type: "object", properties: {} };
}
const given = input as Record<string, unknown>;
if (given.type === "object" || "properties" in given) return given;
if (Object.keys(given).length === 0) return { type: "object", properties: {} };
return { type: "object", properties: given };
}
/**
* Serve over stdio until stdin closes, which is how a host ends a session.
*/
export async function serveMcp(bus: Broker, who: string): Promise<void> {
const surface = mcpSurface(bus, who);
const say = (message: unknown) => {
process.stdout.write(`${JSON.stringify(message)}\n`);
};
for await (const line of lines()) {
let request: Request;
try {
request = JSON.parse(line) as Request;
} catch {
// Unparseable, and with no id there is nobody to tell. Skipped rather than answered, because a
// reply to a request that was never framed is noise on the same channel.
continue;
}
const reply = await surface.handle(request);
if (reply) say(reply);
}
}
/** stdin as newline-framed messages, which is what MCP over stdio is. */
async function* lines(): AsyncGenerator<string> {
let buffered = "";
for await (const chunk of process.stdin) {
buffered += (chunk as Buffer).toString("utf8");
let at: number;
while ((at = buffered.indexOf("\n")) >= 0) {
const line = buffered.slice(0, at).trim();
buffered = buffered.slice(at + 1);
if (line !== "") yield line;
}
}
if (buffered.trim() !== "") yield buffered.trim();
}
/** Every module tool takes an optional `node`: the machine to ask when the module runs on several
* (novox/hq ADR 0159). Added to the listing, stripped before the call, never seen by the module. */
function withNode(schema: Record<string, unknown>, description?: string, required = false): Record<string, unknown> {
const properties = { ...((schema.properties as Record<string, unknown> | undefined) ?? {}) };
if (!("node" in properties)) {
properties.node = {
type: "string",
description: description ??
"the machine to ask, when this module runs on several; else whichever answers, and the answer says which",
};
}
const out: Record<string, unknown> = { ...schema, type: "object", properties };
if (required) {
const have = Array.isArray(schema.required) ? (schema.required as string[]) : [];
out.required = have.includes("node") ? have : [...have, "node"];
}
return out;
}
+278
View File
@@ -0,0 +1,278 @@
#!/usr/bin/env node
/**
* `mesh` — the mesh's tools, for whoever is on a machine (novox/hq design 25 §7, design 34).
*
* Four verbs and nothing else. What tools are there, call one, serve the same two to an agent over
* stdio, and serve them on a machine's loopback as the console the mesh assigns. Deliberately thin:
* everything that could be a decision is one the mesh already made, and a client that grew opinions
* would be a second place the mesh's behaviour is defined.
*
* mesh tools what the running modules answer
* mesh call <module>.<tool> [json] call one, arguments as JSON on the command line or on stdin
* mesh mcp the same two, as an MCP server over stdio
* mesh serve [--listen host:port] the console: MCP over HTTP on this machine's loopback
*
* Who it speaks as, in order of preference:
* MESH_BROKER_FILE the module credential the mesh delivered — the console's (ADR 0152)
* MESH_CREDENTIAL / --credential <file> a person's, as `operator issue` printed it (design 25 §7)
* MESH_CONSOLE / --console <url> no credential: `tools` and `call` go through a console
* already running on this machine, over loopback HTTP
*/
import { readFile } from "node:fs/promises";
import type { Broker } from "@novox/mesh-sdk/messaging";
import {
callTool,
connectAs,
connectAsTheConsole,
credentialFrom,
seatsIn,
toolsOn,
whyItFailed,
type Listing,
} from "./client.js";
import { serveMcpHttp } from "./http.js";
import { serveMcp } from "./mcp.js";
const usage = `mesh tools
mesh call <module>.<tool> [json]
mesh mcp
mesh serve [--listen host:port]
--credential <file> a person's credential, the JSON \`operator issue\` printed; default $MESH_CREDENTIAL
--console <url> a console on this machine to ask through instead; default $MESH_CONSOLE
--listen <host:port> where \`serve\` listens; loopback only; default $MESH_CONSOLE_LISTEN or 127.0.0.1:4270
MESH_BROKER_FILE the module credential the mesh delivered, which \`serve\` runs on`;
/** What the console listens on when nothing says otherwise. */
const DEFAULT_LISTEN = "127.0.0.1:4270";
async function main(argv: string[]): Promise<number> {
const args = [...argv];
let credentialPath = process.env.MESH_CREDENTIAL ?? "";
let consoleUrl = process.env.MESH_CONSOLE ?? "";
let listen = process.env.MESH_CONSOLE_LISTEN ?? DEFAULT_LISTEN;
for (let i = 0; i < args.length; i++) {
const take = () => {
const v = args[i + 1] ?? "";
args.splice(i, 2);
i--;
return v;
};
if (args[i] === "--credential") credentialPath = take();
else if (args[i] === "--console") consoleUrl = take();
else if (args[i] === "--listen") listen = take();
}
const verb = args.shift();
if (!verb || verb === "help" || verb === "--help") {
console.log(usage);
return verb ? 0 : 1;
}
// Through a console already on this machine: no credential to hold, which is the point of one.
if (consoleUrl && (verb === "tools" || verb === "call")) {
return verb === "tools" ? listingVia(consoleUrl) : callingVia(consoleUrl, args);
}
const { bus, who } = await connecting(credentialPath, verb);
try {
switch (verb) {
case "tools":
return await listing(bus, who);
case "call":
return await calling(bus, args);
case "mcp":
// Serves until stdin closes, which is how an MCP host ends a session.
await serveMcp(bus, who);
return 0;
case "serve": {
const up = await serveMcpHttp(bus, who, listen);
console.log(`mesh console listening on http://${up.address}/mcp as ${who}`);
await new Promise<void>((resolve) => {
process.once("SIGTERM", () => resolve());
process.once("SIGINT", () => resolve());
});
await up.close();
return 0;
}
default:
console.error(`mesh has no "${verb}".\n\n${usage}`);
return 1;
}
} finally {
await bus.close();
}
}
/**
* Who this process is on the bus. The module credential first: a console is started by the mesh with
* MESH_BROKER_FILE and nothing else, and must not fall back to a person's file lying around.
*/
async function connecting(credentialPath: string, verb: string): Promise<{ bus: Broker; who: string }> {
const delivered = process.env.MESH_BROKER_FILE;
if (delivered) {
return connectAsTheConsole(delivered);
}
if (!credentialPath) {
throw new Error(
verb === "serve"
? "no credential: the console runs on MESH_BROKER_FILE, the module credential the mesh " +
"delivered; to run it by hand, pass --credential <file> with a person's credential"
: "no credential: set MESH_CREDENTIAL or pass --credential <file> (the JSON `operator issue` " +
"printed, saved verbatim), or --console <url> to ask through a console on this machine",
);
}
const held = await credentialFrom(credentialPath);
return { bus: await connectAs(held), who: held.person ?? held.user ?? "somebody" };
}
function printListing(have: Listing, who?: string): void {
if (have.tools.length === 0) {
console.log("no running module answered with any tool");
}
// **What the modules answered, not what this account may call.** The two differ and the
// difference is the point: an account seeing only its own tools cannot tell "not installed" from
// "not yours", and those need different people to fix them.
for (const t of have.tools) {
const name = `${t.module}.${t.name}`;
const line = t.description ? `${name.padEnd(36)} ${t.description}` : name;
console.log(t.seat ? `${line} (a role's tool: answered by whoever holds the ${t.module} seat)` : line);
}
if (have.notAnswering.length > 0) {
console.log(
`\nheld by the mesh and not answering: ${have.notAnswering.join(", ")} — not assigned, not up, ` +
"or built before the runtime answered `tools`; each can still be called by name",
);
}
if (who) {
console.log(`\nasked as ${who}; what ${who} may call was fixed when the account was issued.`);
}
}
async function listing(bus: Broker, who?: string): Promise<number> {
let have: Listing;
try {
have = await toolsOn(bus);
} catch (e) {
console.error(whyItFailed("mesh-catalog.catalog_modules", e));
return 1;
}
printListing(have, who);
return 0;
}
async function calling(bus: Broker, args: string[]): Promise<number> {
const key = args.shift();
if (!key) {
console.error("mesh call <module>.<tool> [json]");
return 1;
}
const parsed = await argumentsFrom(args);
if (parsed === undefined) return 1;
try {
// `<seat>.<verb>` reaches the role when the mesh lists that verb for the seat; `seat:` says so
// outright and asks nothing first.
const have = key.startsWith("seat:") ? undefined : await toolsOn(bus).catch(() => undefined);
const { result, node } = await callTool(bus, key, parsed, have && seatsIn(have), have);
console.log(JSON.stringify(result, null, 2));
if (node) console.error(`answered by ${node}`);
return 0;
} catch (e) {
console.error(whyItFailed(key, e));
return 1;
}
}
/** The console's answer to one MCP request, over loopback HTTP. */
async function viaConsole(consoleUrl: string, method: string, params?: unknown): Promise<any> {
const endpoint = consoleUrl.endsWith("/mcp") ? consoleUrl : `${consoleUrl.replace(/\/$/, "")}/mcp`;
const res = await fetch(endpoint, {
method: "POST",
headers: { "content-type": "application/json", accept: "application/json" },
body: JSON.stringify({ jsonrpc: "2.0", id: 1, method, params }),
});
if (!res.ok) {
throw new Error(`the console at ${endpoint} answered ${res.status}`);
}
const reply = (await res.json()) as { result?: any; error?: { message: string } };
if (reply.error) throw new Error(reply.error.message);
return reply.result;
}
async function listingVia(consoleUrl: string): Promise<number> {
try {
const result = await viaConsole(consoleUrl, "tools/list");
const have: Listing = {
tools: (result.tools ?? []).map((t: { name: string; description?: string; inputSchema?: unknown }) => {
const at = t.name.indexOf(".");
return { module: t.name.slice(0, at), name: t.name.slice(at + 1), description: t.description, input: t.inputSchema };
}),
notAnswering: result._meta?.notAnswering ?? [],
};
printListing(have);
return 0;
} catch (e) {
console.error(e instanceof Error ? e.message : String(e));
return 1;
}
}
async function callingVia(consoleUrl: string, args: string[]): Promise<number> {
const key = args.shift();
if (!key) {
console.error("mesh call <module>.<tool> [json]");
return 1;
}
const parsed = await argumentsFrom(args);
if (parsed === undefined) return 1;
try {
const result = await viaConsole(consoleUrl, "tools/call", { name: key, arguments: parsed });
const text = result?.content?.[0]?.text ?? JSON.stringify(result);
if (result?.isError) {
console.error(text);
return 1;
}
console.log(text);
return 0;
} catch (e) {
console.error(e instanceof Error ? e.message : String(e));
return 1;
}
}
/** A call's arguments: JSON on the command line, else on stdin, else nothing. Undefined when what
* was given is not JSON, after saying so. */
async function argumentsFrom(args: string[]): Promise<unknown> {
const raw = args.length > 0 ? args.join(" ") : await maybeStdin();
if (raw.trim() === "") return {};
try {
return JSON.parse(raw);
} catch (e) {
console.error(`the arguments are not JSON: ${(e as Error).message}`);
return undefined;
}
}
/** Arguments on stdin, for a call whose JSON is too long or too quoted to type. Empty when stdin is a
* terminal, so `mesh call x.y` with no arguments does not hang waiting for something nobody is
* typing. */
async function maybeStdin(): Promise<string> {
if (process.stdin.isTTY) return "";
const chunks: Buffer[] = [];
for await (const chunk of process.stdin) chunks.push(chunk as Buffer);
return Buffer.concat(chunks).toString("utf8");
}
// Only when run, so a test can import the pieces.
if (process.argv[1] && import.meta.url === new URL(`file://${process.argv[1]}`).href) {
main(process.argv.slice(2))
.then((code) => process.exit(code))
.catch((e) => {
console.error(e instanceof Error ? e.message : String(e));
process.exit(1);
});
}
export { main, usage };
export const _readFile = readFile;
+384
View File
@@ -0,0 +1,384 @@
// The tool runtime — the per-node process that makes the mesh's tools actually serve (novox/hq
// ADR 0175). It binds the mesh broker, loads the served modules' tool bundles (each of which calls
// registerModuleTools as it imports), and serves every module's tools on that module's subjects and
// every held seat's verbs on the seat's. Everything hard — dispatch, collection, duplicate-name
// safety — is the sdk's; this is the wrapper.
//
// One runtime, many modules. It was written for one module per process and ran that way in a
// container per module; it now serves a list, as the one process per node the host supervises,
// and the per-module shape is the list with one entry. A bundle that fails to import is named —
// in the log and in what `tools` answers for it — and the others serve.
import { pathToFileURL } from "node:url";
import { resolve } from "node:path";
import { useBroker } from "@novox/mesh-sdk/messaging";
import { collectTools, toolKey, type ToolDefinition } from "@novox/mesh-sdk/tools";
import type { Broker } from "@novox/mesh-sdk/messaging";
import { atWork, seatToolSubject, type Credential, type RuntimeBroker } from "./broker-nats.js";
import { launch, launches } from "./launch.js";
import { announce, endpointsOf, type ServedSeatVerb } from "./announce.js";
/**
* The one verb every module's runtime answers for it (novox/hq ADR 0152, design 34 §3): the
* module's tool names, descriptions and argument schemas, from the code that answers them and
* from nowhere else. Discovery asks the module, because a copy kept anywhere else drifts.
*/
export const TOOLS_VERB = "tools";
/** What `tools` answers for one module. */
export interface ToolsAnswer {
module: string;
tools: {
name: string;
description: string;
input: Readonly<Record<string, unknown>>;
/** Where this tool is answered, as the mesh issued it (ADR 0160): the module's plain subject
* first when there is one, then this machine's. A caller composes nothing. */
subjects?: string[];
}[];
/** Why this module serves nothing here, when its bundle failed to load (ADR 0175): said where
* discovery looks, so a module that is silent and one that is broken are told apart. */
failed?: string;
}
/** One module this runtime serves: its name and its compiled tool entrypoints. */
export interface ServedModule {
module: string;
/** Absolute paths to the module's compiled tool entrypoints (e.g. .../umami/tools/index.js). */
entrypoints: string[];
}
export interface RuntimeOptions {
/** The mesh broker to serve over. */
broker: Broker;
/** The modules to serve, each with its entrypoints. */
serves?: ServedModule[];
/** The credential's own module's entrypoints — the one-module form, which the per-module
* containers still use; the same as naming the credential's module in `serves`. */
moduleEntrypoints?: string[];
/** The credential the mesh delivered, for what it says about the seats this module claims
* (novox/hq ADR 0159). Absent for a runtime started by hand, which then serves no seat its
* memberships do not name. */
credential?: Credential;
/** What each served module's bundles are given (novox/hq ADR 0192): module → words, composed by
* the mesh per machine. A module absent here is given the runtime's own words and nothing more. */
envs?: ReadonlyMap<string, Readonly<Record<string, string>>>;
}
/** The variable the mesh composes every served module's environment into, as JSON (ADR 0192). Read
* once at start and removed from the process's environment, so no bundle finds another's there. */
export const TOOL_ENV = "MESH_TOOL_ENV";
/** Read and remove the composed environments from an environment (the process's, by default). */
export function takeToolEnvs(env: NodeJS.ProcessEnv = process.env): Map<string, Record<string, string>> {
const raw = env[TOOL_ENV];
delete env[TOOL_ENV];
const out = new Map<string, Record<string, string>>();
if (!raw) return out;
let parsed: unknown;
try {
parsed = JSON.parse(raw);
} catch {
throw new Error(`${TOOL_ENV} is not JSON; the mesh composes it as {"<module>": {"<word>": "<value>"}}`);
}
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
throw new Error(`${TOOL_ENV} is not an object of modules`);
}
for (const [module, words] of Object.entries(parsed as Record<string, unknown>)) {
if (!words || typeof words !== "object" || Array.isArray(words)) {
throw new Error(`${TOOL_ENV}: ${module}'s environment is not an object of words`);
}
const own: Record<string, string> = {};
for (const [k, v] of Object.entries(words as Record<string, unknown>)) own[k] = String(v);
out.set(module, own);
}
return out;
}
/** Two environment words the mesh sets for the node's runtime and every tool reads from its
* environment: whose machine this is (novox/hq to-be 37 §3, ADR 0175). */
export const OPERATOR_ACCOUNT = "MESH_OPERATOR_ACCOUNT";
export const OPERATOR_HOME = "MESH_OPERATOR_HOME";
/** Load the modules, bind the broker, and serve. Returns a stop function that unhooks serving. */
export async function runTools(opts: RuntimeOptions): Promise<() => void> {
useBroker(() => opts.broker);
const runtime = opts.broker as RuntimeBroker;
// Whose runtime this is: the credential's module, or the connection's own when a runtime is
// started by hand without one — the broker was told its module when it connected.
const self = opts.credential?.module ?? (typeof runtime.module === "string" ? runtime.module : undefined);
// What to serve: the list, with the one-module form folded in as the credential's own entry.
const served = new Map<string, string[]>();
for (const s of opts.serves ?? []) {
served.set(s.module, [...(served.get(s.module) ?? []), ...s.entrypoints]);
}
const ownEntrypoints = opts.moduleEntrypoints ?? [];
if (opts.moduleEntrypoints?.length) {
if (!self) {
throw new Error(
"entrypoints were given with no module to serve them as: name the module (MESH_TOOL_MODULES " +
"as <module>=<entrypoint>) or connect on a credential that names one",
);
}
served.set(self, [...(served.get(self) ?? []), ...opts.moduleEntrypoints]);
}
// The operator's machine, said once so a tool's behaviour under it can be read back from the
// log. Tools read the two words from their own environment, which is this process's.
const account = process.env[OPERATOR_ACCOUNT];
if (account) {
console.log(`[mesh-tools] the operator's account here is ${account}` +
(process.env[OPERATOR_HOME] ? ` (home ${process.env[OPERATOR_HOME]})` : ""));
}
// Follow every served module's membership before loading anything, so what each is issued is
// known when its tools are bound. A module's own runtime already follows its own.
if (typeof runtime.follow === "function") {
for (const module of served.keys()) await runtime.follow(module);
}
// What each module's bundles are given: the runtime's own words, and over them the module's own.
const envs = opts.envs ?? new Map<string, Record<string, string>>();
const envFor = (module: string): NodeJS.ProcessEnv => ({ ...process.env, ...(envs.get(module) ?? {}) });
// Every bundle this runtime serves is launched as a process and spoken to over MCP on stdio (ADR
// 0188, ADR 0193): given the runtime's words and its module's own, and told the module it serves
// it as. The runtime knows no language; an entrypoint that is not executable was not built to be
// served, and is refused by name. A bundle that fails to start is named, and the others serve
// (ADR 0175: one faulty bundle must not take the node's tools down).
//
// The one-module form — the credential's own module's entrypoints, which the per-module containers
// still use for their event handlers and provisioners until they move (to-be 38 WP4c) — is imported
// into this process as before: one module, one SDK, its container's own environment.
// The machine this runtime serves, which an event a launched tool emits is stamped with.
const node = opts.credential?.node ?? (typeof (runtime as unknown as { node?: unknown }).node === "string" ? (runtime as unknown as { node: string }).node : undefined);
const failed = new Map<string, string>();
const launched: { module: string; owner: string; tools: ToolDefinition[] }[] = [];
const children: Array<() => void> = [];
for (const [module, entrypoints] of served) {
const imported = module === self && ownEntrypoints.length > 0;
for (const entry of entrypoints) {
const path = resolve(entry);
try {
if (imported) {
await import(pathToFileURL(path).href);
continue;
}
if (!launches(path)) {
throw new Error(`${path} is not executable; a bundle the runtime serves is started, never imported, and its build makes it executable (novox/hq ADR 0193)`);
}
const child = await launch(module, path, {
...envFor(module), MESH_SERVED_MODULE: module, MESH_MODULE: module,
...(node ? { MESH_NODE: node } : {}),
});
children.push(child.stop);
for (const r of child.registrations) launched.push({ ...r, owner: module });
} catch (err) {
const why = err instanceof Error ? err.message : String(err);
failed.set(module, why);
console.log(`[mesh-tools] ${module}'s bundle ${entry} failed to load: ${why}; its tools are not served here`);
}
}
}
// A registration under a served module's name is that module's tools, served on its subjects.
// One under a seat's name is the module's implementation of that seat's verbs (ADR 0159, 0160):
// served on the seat's subjects by serveClaimedSeats where some served module claims the seat,
// never as a module's tools and never listed among them. A module named like its seat (the
// catalogue is the mesh-catalog seat) registers once and is both. Anything else is said and left
// out rather than fatal — on 2026-10-01 the credential of a module that had just learned to
// implement a seat did not yet name the claim, and the whole runtime restarted for it.
const claimed = seatsClaimed(served.keys(), self, opts.credential, runtime);
const registrations = [
...collectTools().map((r) => ({ ...r, owner: self ?? r.module })),
...launched,
];
const ownRegistrations = registrations.filter(({ module, owner: by }) => {
if (served.has(module)) return true;
if (claimed.has(module)) return false;
console.log(`[mesh-tools] ${by} registers tools under "${module}", which is neither a module served here nor a seat one of them claims; not served until the mesh issues the claim`);
return false;
});
const stops: Array<() => void> = [...children];
const stop = (): void => stops.splice(0).forEach((s) => s());
// Refused before anything is bound if a module named a tool of its own `tools`: one name
// answering two things is the fault nobody can diagnose afterwards, and the runtime is the only
// place that sees both. Likewise two tools of one module under one name.
for (const { module, tools: own } of ownRegistrations) {
if (own.some((t) => t.name === TOOLS_VERB)) {
throw new Error(
`${module} names a tool "${TOOLS_VERB}", which is the verb the runtime answers for every ` +
"module with what it serves (novox/hq ADR 0152) — refused, rename it",
);
}
const seen = new Set<string>();
for (const t of own) {
if (seen.has(t.name)) throw new Error(`${module} exposes two tools named ${t.name} — refused`);
seen.add(t.name);
}
}
// Each tool on its own key, namespaced by its module (ADR 0047); where that key is answered is
// the broker's to know from the module's membership (ADR 0160). A tool runs attributed to its
// module, so what it emits lands on the module's subject and not the runtime's.
const names: string[] = [];
for (const { module, tools: own } of ownRegistrations) {
for (const t of own) {
names.push(toolKey(module, t.name));
stops.push(await opts.broker.handle(toolKey(module, t.name), (args: Record<string, unknown> | undefined) =>
atWork.run({ module }, () => t.run(args ?? {}))));
}
}
// And, for every served module, the verb that says what it serves — nothing, and why, for a
// module whose bundle failed. A module that registered nothing and did not fail is a pure-events
// module (the audit logger), whose scoped account may not declare the serve queue; it is left
// silent as it always was.
const byModule = new Map<string, ToolDefinition[]>();
for (const { module, tools: own } of ownRegistrations) {
byModule.set(module, [...(byModule.get(module) ?? []), ...own]);
}
for (const module of served.keys()) {
const own = byModule.get(module) ?? [];
const why = failed.get(module);
if (own.length === 0 && !why) continue;
const subjectsOf = (tool: string): string[] | undefined => {
const m = typeof runtime.membership === "function" ? runtime.membership(module) : undefined;
if (!m) return undefined;
const plain = m.serves.filter((s) => s.queue).map((s) => s.subject.replace("{tool}", tool));
const mine = m.serves.filter((s) => !s.queue).map((s) => s.subject.replace("{tool}", tool));
return [...plain, ...mine];
};
stops.push(await opts.broker.handle(toolKey(module, TOOLS_VERB), async (): Promise<ToolsAnswer> => ({
module,
tools: own.map((t) => ({ name: t.name, description: t.description, input: t.input, subjects: subjectsOf(t.name) })),
...(why ? { failed: why } : {}),
})));
}
console.log(`[mesh-tools] serving ${names.length} tool(s) for ${served.size} module(s): ${names.join(", ") || "(none)"}` +
(failed.size ? `; not serving ${[...failed.keys()].join(", ")}, whose bundle(s) failed to load` : ""));
const seats = await serveClaimedSeats(runtime, [...served.keys()], self, opts.credential, registrations);
stops.push(seats.stop);
// **What it serves, it announces** (novox/hq ADR 0197), asked at the moment of the request.
if (typeof runtime.raw === "function") {
stops.push(announce(runtime, {
name: self ?? "runtime", id: runtime.node ?? self ?? "runtime",
description: `the tool runtime of ${self ?? "a module"}${runtime.node ? ` on ${runtime.node}` : ""}`,
metadata: runtime.node ? { node: runtime.node } : {},
}, () => endpointsOf(runtime, ownRegistrations, seats.serving())));
}
return () => stop();
}
/** The seats some served module claims: from the credential for its own module, and from every
* served module's membership (ADR 0160) — the node's runtime holds no claims of its own. */
function seatsClaimed(
modules: Iterable<string>,
self: string | undefined,
credential: Credential | undefined,
runtime: RuntimeBroker,
): Set<string> {
const out = new Set<string>();
for (const c of credential?.claims ?? []) if (credential?.module === self) out.add(c.seat);
for (const module of modules) {
const m = typeof runtime.membership === "function" ? runtime.membership(module) : undefined;
for (const s of m?.seats ?? []) out.add(s.seat);
}
return out;
}
/** One seat's verb, where its callers ask, and which served module holds the seat. */
interface SeatVerb {
seat: string;
verb: string;
subject: string;
holder: string;
}
/**
* Holding a seat means serving its tools (design 33 §3, novox/hq ADR 0159). What a served module
* claims and promises comes from its membership (ADR 0160) — and, for a module's own runtime, from
* its credential, which named the claims before memberships did. Each verb is served on the seat's
* own subject by the tool of the same name registered under the seat's name. Whether this instance
* *holds* the seat is the bus's to decide: only the holder's account may subscribe the seat's
* subjects, so a claimant that does not hold it here is refused the subscription and serves nothing
* — never a failure of its own tools.
*/
async function serveClaimedSeats(
broker: RuntimeBroker,
served: string[],
self: string | undefined,
credential: Credential | undefined,
registrations: { module: string; owner: string; tools: ToolDefinition[] }[],
): Promise<{ stop: () => void; serving: () => ServedSeatVerb[] }> {
if (typeof broker.handleSubject !== "function") return { stop: () => {}, serving: () => [] };
// A seat's verbs are the role's, not the software's (ADR 0159): implemented under the seat's
// name — `registerModuleTools("mesh-store", …)` — and never confused with the module's own tools.
const implementations = new Map<string, Map<string, (args: Record<string, unknown>) => Promise<unknown>>>();
const definitions = new Map<string, Map<string, ToolDefinition>>();
for (const { module, tools } of registrations) {
const defs = definitions.get(module) ?? new Map<string, ToolDefinition>();
for (const t of tools) defs.set(t.name, t);
definitions.set(module, defs);
}
for (const { module, owner, tools } of registrations) {
const verbs = implementations.get(module) ?? new Map<string, (args: Record<string, unknown>) => Promise<unknown>>();
for (const t of tools) verbs.set(t.name, (args) => atWork.run({ module: owner }, () => t.run(args)));
implementations.set(module, verbs);
}
/** Every verb of every seat a served module claims, where the mesh issued it. */
const wanted = (): SeatVerb[] => {
const out: SeatVerb[] = [];
const have = new Set<string>();
const add = (v: SeatVerb): void => {
if (have.has(v.subject)) return;
have.add(v.subject);
out.push(v);
};
for (const module of served) {
const m = typeof broker.membership === "function" ? broker.membership(module) : undefined;
for (const s of m?.seats ?? []) add({ seat: s.seat, verb: s.verb, subject: s.subject, holder: module });
// The credential's claims, for the module's own runtime, ONLY until the mesh issues a
// membership (novox/hq issue 218). The credential names what the module claims; the membership
// names what it holds on this machine. A seat held once for the mesh is claimed by every
// machine running the module and held by one, so once a membership exists it decides: a claim
// it leaves out is not held here, and serving it anyway announced the seat from a machine the
// bus then refused it on.
if (module !== self || m) continue;
for (const claim of credential?.claims ?? []) {
for (const verb of claim.serves ?? []) {
const subject = seatToolSubject(claim.seat, verb, claim.scope, credential?.node);
add({ seat: claim.seat, verb, subject, holder: module });
}
}
}
return out;
};
let stops: (() => void)[] = [];
let servingNow: ServedSeatVerb[] = [];
const serve = async (): Promise<void> => {
stops.forEach((s) => s());
stops = [];
servingNow = [];
for (const v of wanted()) {
const run = implementations.get(v.seat)?.get(v.verb);
const tool = definitions.get(v.seat)?.get(v.verb);
if (!run) {
console.log(`[mesh-tools] ${v.holder} claims ${v.seat} and implements no ${v.verb}, which that seat promises; not served`);
continue;
}
stops.push(await broker.handleSubject(v.subject, run));
if (tool) servingNow.push({ ...v, tool });
console.log(`[mesh-tools] serving ${v.seat}'s ${v.verb} on ${v.subject}, admitted where ${v.holder} holds the seat`);
}
};
await serve();
// A membership issued to any served module may add, move or withdraw a seat's verbs.
if (typeof broker.onMembership === "function") broker.onMembership(() => void serve());
return { stop: () => stops.forEach((s) => s()), serving: () => servingNow };
}
+69
View File
@@ -0,0 +1,69 @@
/**
* What a runtime serves, it announces (novox/hq ADR 0197): the TypeScript runtime the per-module
* containers still run answers the NATS services protocol's discovery in the same shape as the Go
* tool runtime — its module's tools on every subject issued, and the seat verbs it serves.
*
* MESH_TEST_NATS=nats://127.0.0.1:14232 node --test --experimental-strip-types test/announce.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
import { fileURLToPath } from "node:url";
import { connect, StringCodec } from "nats";
import { resetTools } from "@novox/mesh-sdk/tools";
import { connectNats, membershipSubject } from "../dist/broker-nats.js";
import { runTools } from "../dist/runtime.js";
const url = process.env.MESH_TEST_NATS;
const fixture = (name: string) => fileURLToPath(new URL(`./fixtures/${name}`, import.meta.url));
const sc = StringCodec();
test("the runtime answers $SRV.INFO with what it serves, in the services protocol's format", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const nc = await connect({ servers: url });
const jsm = await nc.jetstreamManager();
try {
await jsm.streams.delete("ASSIGNMENTS");
} catch {
// none yet
}
await jsm.streams.add({ name: "ASSIGNMENTS", subjects: ["mesh.assignment.>"], max_msgs_per_subject: 1, allow_direct: true } as never);
await nc.jetstream().publish(membershipSubject("anchor", "shop"), sc.encode(JSON.stringify({
node: "anchor", module: "shop",
serves: [{ subject: "mesh.mod.shop.tool.{tool}.anchor" }, { subject: "mesh.mod.shop.tool.{tool}", queue: "serve.shop" }],
emits: "mesh.mod.shop.event.{event}", tools: "mesh.mod.shop.tool.tools",
})));
const shop = await connectNats({ url, node: "anchor", module: "shop" });
let stop = () => {};
try {
stop = await runTools({ broker: shop, moduleEntrypoints: [fixture("shop-tools.mjs")] });
const msg = await nc.request("$SRV.INFO.shop.anchor", sc.encode(""), { timeout: 2000 });
const info = JSON.parse(sc.decode(msg.data)) as {
type: string; name: string; id: string; version: string;
endpoints: { name: string; subject: string; queue_group: string; metadata: Record<string, string> }[];
};
assert.equal(info.type, "io.nats.micro.v1.info_response");
assert.equal(info.name, "shop");
assert.equal(info.id, "anchor");
assert.ok(info.version);
const price = info.endpoints.filter((e) => e.metadata.tool === "price").map((e) => `${e.subject}|${e.queue_group}`).sort();
assert.deepEqual(price, ["mesh.mod.shop.tool.price.anchor|", "mesh.mod.shop.tool.price|serve.shop"]);
const one = info.endpoints.find((e) => e.metadata.tool === "price")!;
assert.equal(one.metadata.kind, "tool");
assert.equal(one.metadata.module, "shop");
assert.equal(one.metadata.node, "anchor");
assert.equal(one.metadata.interchangeable, "true");
assert.ok(JSON.parse(one.metadata.schema).type === "object");
// Ping answers with the same identity; another service's request is not answered.
const ping = JSON.parse(sc.decode((await nc.request("$SRV.PING", sc.encode(""), { timeout: 2000 })).data));
assert.equal(ping.type, "io.nats.micro.v1.ping_response");
await assert.rejects(nc.request("$SRV.INFO.somebody-else", sc.encode(""), { timeout: 300 }));
} finally {
stop();
await shop.close();
await nc.close();
resetTools();
}
});
+188
View File
@@ -0,0 +1,188 @@
/**
* A person's client, against a real bus.
*
* What is worth checking is not that a request/reply works — the runtime's own tests cover that — but
* that the two surfaces are the same thing. An agent and a person must see the same tools and get the
* same answers, or the MCP surface becomes a second definition of what a tool is.
*
* docker run -d --rm --name t -p 14232:4222 nats:2.10-alpine -js
* MESH_TEST_NATS=nats://127.0.0.1:14232 node --test --experimental-strip-types test/client.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
// The built output, not the source: the client imports its siblings as `.js`, which is what ships and
// what every other file here does, and cannot be loaded as TypeScript directly. `pretest` builds.
import { connectNats } from "../dist/broker-nats.js";
import { callTool, seatsIn, toolKey, toolsOn, whyItFailed } from "../dist/client.js";
const url = process.env.MESH_TEST_NATS;
/** A mesh as discovery sees it (design 34 §3): the catalogue holding three modules, two of them up
* and answering `tools`, one held and not running. Two connections, because a person and a module
* are different users even in a test. */
async function aMeshWithTools() {
const catalogue = await connectNats({ url: url!, module: "mesh-catalog" });
const shop = await connectNats({ url: url!, module: "shop" });
await catalogue.handle("catalog_modules", async () => ({
modules: [{ module: "shop" }, { module: "mesh-catalog" }, { module: "ghost" }],
}));
await catalogue.handle("tools", async () => ({
module: "mesh-catalog",
tools: [{ name: "catalog_modules", description: "what modules the mesh has", input: {} }],
}));
await shop.handle("tools", async () => ({
module: "shop",
tools: [{ name: "price", description: "what something costs", input: { type: "object" } }],
}));
await shop.handle("price", async (body: { of?: string }) => ({ of: body.of ?? "nothing", cost: 12 }));
// And the mesh's own records, served by the holder of the mesh-controller seat (ADR 0154): one
// role's tool, and the verb that lists every role's.
const controller = await connectNats({ url: url!, module: "mesh-controller" });
await controller.handle("seat:mesh-controller.tools", async () => ({
seats: [
{ seat: "mesh-controller", scope: "mesh", tools: [{ name: "status", description: "what is wrong", input: {} }] },
{ seat: "node-dns-resolver", scope: "node", tools: [{ name: "lookup", description: "one machine's", input: {} }] },
],
}));
await controller.handle("seat:mesh-controller.status", async () => ({ output: "all quiet", ok: true }));
return {
async close() {
await catalogue.close();
await shop.close();
await controller.close();
},
};
}
test("a person sees what the running modules answer, sorted, and who did not answer", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const mesh = await aMeshWithTools();
const person = await connectNats({ url, module: "person.ada" });
try {
const began = Date.now();
const have = await toolsOn(person);
assert.deepEqual(
have.tools.map((x) => `${x.module}.${x.name}`),
["mesh-catalog.catalog_modules", "mesh-controller.status", "node-dns-resolver.lookup", "shop.price"],
"the list is what the modules answered plus every role's tools, in a stable order",
);
// A role's tool is marked as one; a node-scoped seat's carries its scope, so a caller names
// the machine and the verb resolves as the seat's (design 33 §4, ADR 0170).
assert.ok(have.tools.find((x) => x.module === "mesh-controller")!.seat);
const lookup = have.tools.find((x) => x.module === "node-dns-resolver")!;
assert.ok(lookup.seat && lookup.scope === "node");
assert.equal(toolKey("node-dns-resolver.lookup", seatsIn(have)), "seat:node-dns-resolver.lookup");
// Silence is named, never dropped: a module the catalogue holds and nothing answered for.
assert.deepEqual(have.notAnswering, ["ghost"]);
// And at once: a module that is not running costs nothing, or the list is unusable.
assert.ok(Date.now() - began < 5_000, "an absent module waited out the timeout");
} finally {
await person.close();
await mesh.close();
}
});
test("a person calls a tool and gets the module's own answer, unshaped", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const mesh = await aMeshWithTools();
const person = await connectNats({ url, module: "person.ada" });
try {
const answer = await callTool(person, "shop.price", { of: "a hat" });
assert.deepEqual(answer.result, { of: "a hat", cost: 12 });
} finally {
await person.close();
await mesh.close();
}
});
test("a tool nobody serves says so at once, and says what to do about it", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const person = await connectNats({ url: url!, module: "person.ada" });
try {
const began = Date.now();
await assert.rejects(() => callTool(person, "ghost.missing", {}));
// At once, not after the whole wait: "that module is down" and "that tool is slow" need
// different things done, and a timeout cannot tell them apart.
assert.ok(Date.now() - began < 5_000, "a tool nobody serves waited out the timeout");
} finally {
await person.close();
}
});
test("a name that is not <module>.<tool> is refused before anything is sent", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const person = await connectNats({ url: url!, module: "person.ada" });
try {
await assert.rejects(() => callTool(person, "price", {}), /does not name a tool/);
} finally {
await person.close();
}
});
test("each way a call fails says what to do about it", () => {
// The three answers a person actually gets. Without this they are one timeout and a stack trace,
// and the remedies are in three different places.
assert.match(whyItFailed("shop.price", new Error("no responders")), /nothing serves shop\.price/);
assert.match(
whyItFailed("shop.price", new Error("Permissions Violation for Publish")),
/may not call shop\.price/,
);
assert.match(whyItFailed("shop.price", new Error("timeout")), /did not answer in time/);
assert.match(whyItFailed("shop.price", new Error("something else")), /something else/);
});
test("a role's tool is reached through the seat, and a module's own name is never shadowed", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const { seatsIn, toolKey } = await import("../dist/client.js");
const mesh = await aMeshWithTools();
const person = await connectNats({ url, module: "person.ada" });
try {
const roles = seatsIn(await toolsOn(person));
assert.equal(toolKey("mesh-controller.status", roles), "seat:mesh-controller.status");
assert.equal(toolKey("mesh-controller.other", roles), "mesh-controller.other", "a verb the seat does not declare is a module's");
assert.equal(toolKey("shop.price", roles), "shop.price");
const answer = await callTool(person, "mesh-controller.status", {}, roles);
assert.deepEqual(answer.result, { output: "all quiet", ok: true });
const direct = await callTool(person, "seat:mesh-controller.status", {});
assert.deepEqual(direct.result, { output: "all quiet", ok: true });
} finally {
await person.close();
await mesh.close();
}
});
// A module on two machines (novox/hq ADR 0159): a call names the machine and reaches that instance
// and no other; a call that names none reaches one of them and says which; a seat's verb is served
// by the claimant's tool of the same name on the seat's own subject.
test("a tool call names the machine it is for, and every answer says which machine answered", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const onAnchor = await connectNats({ url: url!, module: "store", node: "anchor" });
const onHome = await connectNats({ url: url!, module: "store", node: "home-server" });
const asker = await connectNats({ url: url!, module: "console", node: "workstation" });
try {
await onAnchor.handle("databases", async () => ({ at: "anchor" }));
await onHome.handle("databases", async () => ({ at: "home-server" }));
const home = await callTool(asker, "store.databases@home-server", {});
assert.deepEqual(home.result, { at: "home-server" });
assert.equal(home.node, "home-server");
const anchor = await callTool(asker, "store.databases@anchor", {});
assert.deepEqual(anchor.result, { at: "anchor" });
assert.equal(anchor.node, "anchor");
const whichever = await callTool(asker, "store.databases", {});
assert.ok(["anchor", "home-server"].includes(whichever.node ?? ""), `an unnamed call still says who answered: ${whichever.node}`);
assert.deepEqual(whichever.result, { at: whichever.node });
// A seat's verb, served by the claimant on the seat's subject; a mesh seat's is flat.
await onAnchor.handleSubject("mesh.seat.mesh-store.tool.databases", async () => ({ seat: "mesh-store", at: "anchor" }));
const viaSeat = await callTool(asker, "seat:mesh-store.databases", {});
assert.deepEqual(viaSeat.result, { seat: "mesh-store", at: "anchor" });
assert.equal(viaSeat.node, "anchor");
} finally {
await onAnchor.close();
await onHome.close();
await asker.close();
}
});
+54
View File
@@ -0,0 +1,54 @@
// The TypeScript implementation, held to the shared fixtures (novox/hq ADR 0074, design 19).
//
// Run against a NATS server, because the question is what actually reaches the wire:
//
// docker run -d --rm --name c -p 14222:4222 nats:2.10-alpine -js
// node test/conformance.mjs
//
// **The runner lives with the implementation it exercises; the fixture does not.** It is read
// from the sdk's conformance directory by sibling path — the same file the Go suite reads. A
// fixture copied into each implementation is two fixtures, and two fixtures drift, which is the
// failure the suite exists to prevent.
import { readFileSync } from "node:fs";
import { connect } from "nats";
const clientPath = process.argv[2] ?? "../dist/broker-nats.js";
const { connectNats } = await import(clientPath);
const f = JSON.parse(readFileSync(new URL("../../mesh-sdk/conformance/events/module-event.json", import.meta.url)));
const URL_ = process.env.MESH_TEST_NATS ?? "nats://127.0.0.1:14222";
let failed = 0;
const check = (ok, what) => { console.log(` ${ok ? "ok " : "FAIL"} ${what}`); if (!ok) failed++; };
const admin = await connect({ servers: URL_ });
const jsm = await admin.jetstreamManager();
await jsm.streams.add({ name: "EVENTS", subjects: ["mesh.mod.*.event.>", "mesh.seat.*.event.>"] });
const shop = await connectNats({ url: URL_, node: f.given.node, module: f.given.module });
await shop.publish({ key: f.given.key, node: f.given.node, body: f.given.body, headers: f.given.headers });
// What actually landed, read back from the stream rather than from the client that wrote it.
const msg = await jsm.streams.getMessage("EVENTS", { last_by_subj: f.wire.subject });
check(!!msg, `it lands on ${f.wire.subject}`);
if (msg) {
const got = {};
if (msg.header) for (const k of msg.header.keys()) got[k] = msg.header.get(k);
for (const h of f.wire.requiredHeaders) {
check(got[h] !== undefined && got[h] !== "", `${h} is set`);
}
check(got["content-type"] === f.wire.headerFormats["content-type"], "content-type is as pinned");
check(new RegExp(f.wire.headerFormats["x-event-id"]).test(got["x-event-id"]), "x-event-id is as pinned");
check(!Number.isNaN(Date.parse(got["x-time"])), "x-time parses as a date");
check(got["x-source"] === f.given.module, "x-source agrees with the subject's module");
const payload = JSON.parse(new TextDecoder().decode(msg.data));
check(payload.envelope === undefined && payload.key === undefined,
"the payload is the body alone, not the envelope (the fixture refuses nesting)");
check(JSON.stringify(payload) === JSON.stringify(f.given.body),
"the body round-trips — semantic, not byte-exact, per the README");
}
await shop.close(); await admin.close();
console.log(failed ? `\n${failed} failed` : "\nall passed");
process.exit(failed ? 1 : 0);
+6
View File
@@ -0,0 +1,6 @@
// A module naming a tool after the verb the runtime answers for every module — refused at load.
import { registerModuleTools } from "@novox/mesh-sdk/tools";
registerModuleTools("clash", () => [
{ name: "tools", description: "mine, not the runtime's", input: {}, run: async () => ({}) },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for clash-tools.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./clash-tools.mjs");
await serveRegisteredOverStdio();
+4
View File
@@ -0,0 +1,4 @@
import { registerModuleTools } from "@novox/mesh-sdk/tools";
registerModuleTools("delta", (env) => [
{ name: "given", description: "what delta was given", input: {}, run: async () => ({ mine: env.DELTA_TOKEN_FILE ?? null, theirs: env.GAMMA_CONFIG_FILE ?? null }) },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for env-delta.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./env-delta.mjs");
await serveRegisteredOverStdio();
+5
View File
@@ -0,0 +1,5 @@
// A bundle that reads what it was given (novox/hq ADR 0192): its contributor's environment.
import { registerModuleTools } from "@novox/mesh-sdk/tools";
registerModuleTools("gamma", (env) => [
{ name: "given", description: "what gamma was given", input: {}, run: async () => ({ mine: env.GAMMA_CONFIG_FILE ?? null, theirs: env.DELTA_TOKEN_FILE ?? null, runtime: env.MESH_OPERATOR_ACCOUNT ?? null, composed: env.MESH_TOOL_ENV ?? null }) },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for env-gamma.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./env-gamma.mjs");
await serveRegisteredOverStdio();
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env node
// Launched (ADR 0188): its environment is the child's own.
import { serveStdio } from "@novox/mesh-sdk/stdio";
await serveStdio("zeta", [
{ name: "given", description: "what zeta was given", input: {}, run: async () => ({ mine: process.env.ZETA_URL ?? null, theirs: process.env.GAMMA_CONFIG_FILE ?? null, composed: process.env.MESH_TOOL_ENV ?? null }) },
]);
+7
View File
@@ -0,0 +1,7 @@
#!/usr/bin/env node
// A long-running bundle that keeps exiting (novox/hq ADR 0198): the runtime starts it again each time.
import { appendFileSync } from "node:fs";
import { serveStdio } from "@novox/mesh-sdk/stdio";
appendFileSync(process.env.FLAKY_LOG, "started\n");
setTimeout(() => process.exit(3), 300);
await serveStdio("flaky", []);
+9
View File
@@ -0,0 +1,9 @@
// One of several bundles the node's runtime loads (novox/hq ADR 0175): a module with two tools.
import { registerModuleTools } from "@novox/mesh-sdk/tools";
import { emit } from "@novox/mesh-sdk/events";
registerModuleTools("alpha", () => [
{ name: "one", description: "alpha's first", input: {}, run: async () => ({ alpha: 1 }) },
// A tool that emits: the event must land on alpha's subject, not the runtime's.
{ name: "two", description: "alpha's second, which emits", input: {}, run: async () => { await emit("happened", { by: "alpha" }); return { alpha: 2 }; } },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for many-alpha.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./many-alpha.mjs");
await serveRegisteredOverStdio();
+14
View File
@@ -0,0 +1,14 @@
// One of several bundles the node's runtime loads (novox/hq ADR 0175): a module with three tools of
// its own and the implementation of a node seat's two verbs under the seat's name (ADR 0159).
import { registerModuleTools } from "@novox/mesh-sdk/tools";
registerModuleTools("beta", () => [
{ name: "three", description: "beta's", input: { verbose: { type: "boolean", description: "say more" } }, run: async () => ({ beta: 3 }) },
{ name: "four", description: "beta's", input: {}, run: async () => ({ beta: 4 }) },
{ name: "five", description: "beta's", input: {}, run: async () => ({ beta: 5 }) },
]);
registerModuleTools("node-shelf", () => [
{ name: "list", description: "what is on the shelf", input: {}, run: async () => ({ shelf: ["a", "b"] }) },
{ name: "clear", description: "take it all off", input: {}, run: async () => ({ cleared: true }) },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for many-beta.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./many-beta.mjs");
await serveRegisteredOverStdio();
+3
View File
@@ -0,0 +1,3 @@
// A bundle that throws on import — the fault ADR 0175 names as what got harder: one module's bundle
// must not take the node's other tools down.
throw new Error("gamma's bundle cannot find its client");
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for many-broken.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./many-broken.mjs");
await serveRegisteredOverStdio();
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env python3
# A tools bundle in a second language (novox/hq ADR 0188): MCP over stdio, no SDK, no dependencies.
# One tool of its own, one seat verb, and one that exits the process mid-call.
import json, sys
def say(m):
m["jsonrpc"] = "2.0"; sys.stdout.write(json.dumps(m) + "\n"); sys.stdout.flush()
TOOLS = [
{"name": "greet", "description": "say hello", "inputSchema": {"type": "object", "properties": {"who": {"type": "string"}}}},
{"name": "node-lamp.on", "description": "the seat's verb", "inputSchema": {"type": "object", "properties": {}}},
{"name": "die", "description": "exit without answering", "inputSchema": {"type": "object", "properties": {}}},
]
for line in sys.stdin:
req = json.loads(line); rid = req.get("id"); m = req.get("method"); p = req.get("params") or {}
if m == "initialize":
say({"id": rid, "result": {"protocolVersion": "2025-03-26", "capabilities": {"tools": {}}, "serverInfo": {"name": "delta", "version": "1"}}})
elif m == "tools/list":
say({"id": rid, "result": {"tools": TOOLS}})
elif m == "tools/call":
name = p.get("name"); args = p.get("arguments") or {}
if name == "greet":
say({"id": rid, "result": {"content": [{"type": "text", "text": json.dumps({"greeting": "hello " + args.get("who", "world"), "language": "python"})}]}})
elif name == "node-lamp.on":
say({"id": rid, "result": {"content": [{"type": "text", "text": json.dumps({"on": True, "language": "python"})}]}})
elif name == "die":
print("delta: told to die", file=sys.stderr); sys.exit(3)
else:
say({"id": rid, "error": {"code": -32602, "message": "no such tool"}})
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env node
// A TypeScript bundle written against the protocol and marked executable: served through the
// launcher like any other language, with the in-process shortcut off (novox/hq ADR 0188).
import { serveStdio } from "@novox/mesh-sdk/stdio";
await serveStdio("epsilon", [
{ name: "seven", description: "epsilon's", input: {}, run: async () => ({ epsilon: 7, via: "stdio" }) },
]);
+10
View File
@@ -0,0 +1,10 @@
# A bus that refuses one subject: what the mesh's grants do to a subject they do not name (issue 217).
authorization {
users = [
{ user: "runtime", password: "runtime", permissions: {
publish: { allow: [">"] }
subscribe: { allow: [">"], deny: ["$SRV.PING.>", "mesh.seat.held-elsewhere.>"] }
} }
]
}
jetstream: enabled
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env node
// A launched bundle registering its seat before its own tools, served through the SDK's loop
// (novox/hq ADR 0193): which is the seat's and which the module's comes from the name it is given.
import { registerModuleTools } from "@novox/mesh-sdk/tools";
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
registerModuleTools("node-shelf", () => [{ name: "list", description: "the shelf", input: {}, run: async () => ({ shelf: ["x"] }) }]);
registerModuleTools("theta", (env) => [{ name: "own", description: "theta's", input: {}, run: async () => ({ theta: env.THETA_WORD ?? null }) }]);
await serveRegisteredOverStdio();
+12
View File
@@ -0,0 +1,12 @@
// The shop again, in a file of its own: a module imported once stays imported, so a second test needs a second entrypoint.
import { registerModuleTools } from "@novox/mesh-sdk/tools";
registerModuleTools("shop", () => [
{
name: "price",
description: "what something costs",
input: { of: { type: "string", description: "the thing" } },
run: async (args) => ({ of: args.of ?? "nothing", cost: 12 }),
},
{ name: "refund", description: "give it back", input: {}, run: async () => ({ done: true }) },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for shop-seat.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./shop-seat.mjs");
await serveRegisteredOverStdio();
+12
View File
@@ -0,0 +1,12 @@
// A module's tool entrypoint, as the runtime imports one: registers and returns.
import { registerModuleTools } from "@novox/mesh-sdk/tools";
registerModuleTools("shop", () => [
{
name: "price",
description: "what something costs",
input: { of: { type: "string", description: "the thing" } },
run: async (args) => ({ of: args.of ?? "nothing", cost: 12 }),
},
{ name: "refund", description: "give it back", input: {}, run: async () => ({ done: true }) },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for shop-tools.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./shop-tools.mjs");
await serveRegisteredOverStdio();
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env node
// A bundle that keeps and reads state through the runtime (novox/hq ADR 0201), written against the
// protocol with no SDK: on start it watches STATE_NAME and writes every change it is handed to
// STATE_LOG; its tools put, get, delete and list keys of whichever state they name.
import { appendFileSync } from "node:fs";
import { createInterface } from "node:readline";
const say = (m) => process.stdout.write(JSON.stringify({ jsonrpc: "2.0", ...m }) + "\n");
let next = 1;
const waiting = new Map();
const ask = (method, params) => new Promise((resolve, reject) => {
const id = `keeper-${next++}`;
waiting.set(id, { resolve, reject });
say({ id, method, params });
});
const log = (line) => appendFileSync(process.env.STATE_LOG, line + "\n");
const text = (v) => ({ content: [{ type: "text", text: JSON.stringify(v ?? null) }] });
const tool = (name) => ({ name, description: name, inputSchema: { type: "object", properties: {} } });
const TOOLS = ["put", "get", "del", "keys"].map(tool);
createInterface({ input: process.stdin }).on("line", async (line) => {
const m = JSON.parse(line);
if (m.method === undefined && waiting.has(m.id)) {
const w = waiting.get(m.id); waiting.delete(m.id);
m.error ? w.reject(new Error(m.error.message)) : w.resolve(m.result);
return;
}
const p = m.params ?? {};
switch (m.method) {
case "initialize":
say({ id: m.id, result: { protocolVersion: "2025-03-26", capabilities: { tools: {} }, serverInfo: { name: "keeper", version: "1" } } });
// Watched once the runtime has the handshake: the current values, then every change.
ask("mesh/state.watch", { state: process.env.STATE_NAME })
.then(() => log("current"), (e) => log("watch refused: " + e.message));
return;
case "tools/list":
say({ id: m.id, result: { tools: TOOLS } });
return;
case "mesh/state": {
const c = p.change;
log(`${c.current ? "was" : "now"} ${c.op} ${c.key}${c.value !== undefined ? " " + JSON.stringify(c.value) : ""}`);
say({ id: m.id, result: {} });
return;
}
case "tools/call": {
const a = p.arguments ?? {};
const verb = { put: "put", get: "get", del: "delete", keys: "keys" }[p.name];
try {
say({ id: m.id, result: text(await ask(`mesh/state.${verb}`, { state: a.state, key: a.key, value: a.value })) });
} catch (e) {
say({ id: m.id, result: { content: [{ type: "text", text: e.message }], isError: true } });
}
return;
}
default:
if (m.id !== undefined) say({ id: m.id, error: { code: -32601, message: "no " + m.method } });
}
});
+13
View File
@@ -0,0 +1,13 @@
// A module that keeps state through the SDK (novox/hq ADR 0201): it watches its servers as it is
// imported, and writes each change it is handed to STATE_LOG; one tool registers a server.
import { appendFileSync } from "node:fs";
import { state } from "@novox/mesh-sdk/state";
import { registerModuleTools } from "@novox/mesh-sdk/tools";
const servers = state("servers");
await servers.watch((c) => appendFileSync(process.env.STATE_LOG, `${c.current ? "was" : "now"} ${c.op} ${c.key}\n`));
await servers.watch((c) => appendFileSync(process.env.STATE_LOG, `narrow ${c.op} ${c.key}\n`), { key: "anchor.*" });
appendFileSync(process.env.STATE_LOG, "current\n");
registerModuleTools("sdkkeeper", () => [
{ name: "register", description: "put a server", input: {}, run: async (a) => servers.put(a.key, { url: a.url }) },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for state-sdk.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./state-sdk.mjs");
await serveRegisteredOverStdio();
+13
View File
@@ -0,0 +1,13 @@
// A copy for the issue 218 test: a fixture registers once per process, at import.
// A module that is both software and a role: postgres's own tools under its name, and its
// implementation of the mesh-store seat's verbs under the seat's (novox/hq ADR 0159, 0160).
import { registerModuleTools } from "@novox/mesh-sdk/tools";
registerModuleTools("postgres", () => [
{ name: "postgres_create_database", description: "make one", input: {}, run: async () => ({ made: true }) },
{ name: "databases", description: "postgres's own listing", input: {}, run: async () => ({ software: "postgres" }) },
]);
registerModuleTools("mesh-store", () => [
{ name: "databases", description: "what the store holds", input: {}, run: async () => ({ seat: "mesh-store" }) },
]);
+12
View File
@@ -0,0 +1,12 @@
// A module that is both software and a role: postgres's own tools under its name, and its
// implementation of the mesh-store seat's verbs under the seat's (novox/hq ADR 0159, 0160).
import { registerModuleTools } from "@novox/mesh-sdk/tools";
registerModuleTools("postgres", () => [
{ name: "postgres_create_database", description: "make one", input: {}, run: async () => ({ made: true }) },
{ name: "databases", description: "postgres's own listing", input: {}, run: async () => ({ software: "postgres" }) },
]);
registerModuleTools("mesh-store", () => [
{ name: "databases", description: "what the store holds", input: {}, run: async () => ({ seat: "mesh-store" }) },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for store-seat-unclaimed.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./store-seat-unclaimed.mjs");
await serveRegisteredOverStdio();
+12
View File
@@ -0,0 +1,12 @@
// A module that is both software and a role: postgres's own tools under its name, and its
// implementation of the mesh-store seat's verbs under the seat's (novox/hq ADR 0159, 0160).
import { registerModuleTools } from "@novox/mesh-sdk/tools";
registerModuleTools("postgres", () => [
{ name: "postgres_create_database", description: "make one", input: {}, run: async () => ({ made: true }) },
{ name: "databases", description: "postgres's own listing", input: {}, run: async () => ({ software: "postgres" }) },
]);
registerModuleTools("mesh-store", () => [
{ name: "databases", description: "what the store holds", input: {}, run: async () => ({ seat: "mesh-store" }) },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for store-seat.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./store-seat.mjs");
await serveRegisteredOverStdio();
+24
View File
@@ -0,0 +1,24 @@
// A module whose code runs long (novox/hq ADR 0198): it subscribes as it is imported, handles each
// event once it can, fails the first time it is asked to, dies the first time it is told to, and one
// tool asks another module's tool through the runtime. What it did is written to WATCH_LOG.
import { appendFileSync, existsSync, writeFileSync } from "node:fs";
import { on } from "@novox/mesh-sdk/events";
import { broker } from "@novox/mesh-sdk/messaging";
import { registerModuleTools } from "@novox/mesh-sdk/tools";
const log = process.env.WATCH_LOG;
const once = (mark) => {
const file = `${log}.${mark}`;
if (existsSync(file)) return false;
writeFileSync(file, "1");
return true;
};
appendFileSync(log, "started\n");
await on("alpha.happened", async (e) => {
if (e.body.fail && once(`fail-${e.body.n}`)) throw new Error(`not yet ${e.body.n}`);
if (e.body.die && once(`die-${e.body.n}`)) process.exit(7);
appendFileSync(log, `handled ${e.body.n}\n`);
});
registerModuleTools("watcher", () => [
{ name: "relay", description: "asks beta", input: {}, run: async () => broker().request("beta.three", {}) },
]);
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env node
// The launcher the builder writes beside an entrypoint (novox/hq ADR 0193), for watcher.mjs.
import { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";
await import("./watcher.mjs");
await serveRegisteredOverStdio();
+113
View File
@@ -0,0 +1,113 @@
/**
* The console: the same surface over HTTP on loopback, started the way the mesh starts it — on the
* module credential in MESH_BROKER_FILE (novox/hq ADR 0152, design 34 §2).
*
* docker run -d --rm --name t -p 14232:4222 nats:2.10-alpine -js
* MESH_TEST_NATS=nats://127.0.0.1:14232 node --test --experimental-strip-types test/http.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
import { spawn, type ChildProcess } from "node:child_process";
import { mkdtemp, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { connectNats } from "../dist/broker-nats.js";
import { serveMcpHttp } from "../dist/http.js";
const url = process.env.MESH_TEST_NATS;
async function aMesh(t: { after: (fn: () => Promise<void> | void) => void }) {
const catalogue = await connectNats({ url: url!, module: "mesh-catalog" });
const shop = await connectNats({ url: url!, module: "shop" });
await catalogue.handle("catalog_modules", async () => ({ modules: [{ module: "shop" }] }));
await shop.handle("tools", async () => ({
module: "shop",
tools: [{ name: "price", description: "what something costs", input: {} }],
}));
await shop.handle("price", async (body: { of?: string }) => ({ of: body.of ?? "nothing", cost: 12 }));
t.after(async () => {
await catalogue.close();
await shop.close();
});
}
/** `mesh serve` as the mesh runs it: MESH_BROKER_FILE, a listen address, nothing else. */
async function aConsole(t: { after: (fn: () => Promise<void> | void) => void }): Promise<string> {
const dir = await mkdtemp("/tmp/mesh-console-");
const credential = join(dir, "broker");
await writeFile(credential, JSON.stringify({ url, node: "desk", module: "mesh-console", user: "desk.mesh-console", password: "x" }));
const child: ChildProcess = spawn(process.execPath, ["dist/mesh.js", "serve", "--listen", "127.0.0.1:0"], {
env: { ...process.env, MESH_BROKER_FILE: credential, MESH_CREDENTIAL: "" },
stdio: ["ignore", "pipe", "pipe"],
});
t.after(() => {
child.kill("SIGTERM");
});
return new Promise((resolve, reject) => {
let out = "";
let err = "";
child.stdout!.on("data", (d) => {
out += d.toString();
const m = /listening on (http:\/\/[^/]+\/mcp)/.exec(out);
if (m) resolve(m[1]!);
});
child.stderr!.on("data", (d) => (err += d.toString()));
child.on("exit", (code) => reject(new Error(`serve exited ${code}: ${err}`)));
});
}
async function post(endpoint: string, body: unknown): Promise<{ status: number; json?: any }> {
const res = await fetch(endpoint, {
method: "POST",
headers: { "content-type": "application/json", accept: "application/json" },
body: JSON.stringify(body),
});
const text = await res.text();
return { status: res.status, json: text ? JSON.parse(text) : undefined };
}
test("the console answers a host on loopback, as the account the mesh gave it", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
await aMesh(t);
const endpoint = await aConsole(t);
const hello = await post(endpoint, { jsonrpc: "2.0", id: 1, method: "initialize", params: {} });
assert.equal(hello.status, 200);
assert.match(hello.json.result.instructions, /desk\.mesh-console/, "the handshake names the console's account");
const heard = await post(endpoint, { jsonrpc: "2.0", method: "notifications/initialized" });
assert.equal(heard.status, 202, "a notification is heard and not answered");
const listed = await post(endpoint, { jsonrpc: "2.0", id: 2, method: "tools/list" });
assert.deepEqual(listed.json.result.tools.map((x: { name: string }) => x.name), ["shop.price"]);
const called = await post(endpoint, {
jsonrpc: "2.0", id: 3, method: "tools/call", params: { name: "shop.price", arguments: { of: "a hat" } },
});
assert.deepEqual(JSON.parse(called.json.result.content[0].text), { of: "a hat", cost: 12 });
// A person's client through the same endpoint, with no credential of its own.
const { main } = await import("../dist/mesh.js");
const logged: string[] = [];
const was = console.log;
console.log = (line: string) => logged.push(String(line));
try {
assert.equal(await main(["tools", "--console", endpoint]), 0);
} finally {
console.log = was;
}
assert.ok(logged.some((l) => l.startsWith("shop.price")), `the client did not list through the console: ${logged}`);
});
test("the console binds loopback and nowhere else", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const bus = await connectNats({ url, module: "mesh-console", node: "desk" });
try {
await assert.rejects(() => serveMcpHttp(bus, "desk.mesh-console", "0.0.0.0:0"), /loopback and nowhere else/);
const up = await serveMcpHttp(bus, "desk.mesh-console", "127.0.0.1:0");
assert.match(up.address, /^127\.0\.0\.1:\d+$/);
await up.close();
} finally {
await bus.close();
}
});
+196
View File
@@ -0,0 +1,196 @@
/**
* The MCP surface, driven the way a host drives it.
*
* **The claim worth checking is that it is the same thing the command line is.** An agent and a
* person must see the same tools and get the same answers, or this becomes a second definition of what
* a tool is — which is exactly what a thin adapter is supposed to avoid.
*
* docker run -d --rm --name t -p 14232:4222 nats:2.10-alpine -js
* MESH_TEST_NATS=nats://127.0.0.1:14232 node --test --experimental-strip-types test/mcp.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
import { spawn } from "node:child_process";
import { connectNats } from "../dist/broker-nats.js";
const url = process.env.MESH_TEST_NATS;
/** A module answering the catalogue's list and one tool, plus a credential file the client reads. */
async function aMeshAndACredential(t: { after: (fn: () => Promise<void> | void) => void }) {
const catalogue = await connectNats({ url: url!, module: "mesh-catalog" });
const shop = await connectNats({ url: url!, module: "shop" });
await catalogue.handle("catalog_modules", async () => ({
modules: [{ module: "shop" }, { module: "ghost" }],
}));
await shop.handle("tools", async () => ({
module: "shop",
tools: [{ name: "price", description: "what something costs", input: { of: { type: "string" } } }],
}));
await shop.handle("price", async (body: { of?: string }) => ({ of: body.of ?? "nothing", cost: 12 }));
const controller = await connectNats({ url: url!, module: "mesh-controller" });
await controller.handle("seat:mesh-controller.tools", async () => ({
seats: [
{ seat: "mesh-controller", scope: "mesh", tools: [
{ name: "status", description: "what is wrong", input: {} },
{ name: "push", description: "tell a machine", input: { node: { type: "string" } } },
] },
// A seat held once per machine (design 33 §4, ADR 0170): its verb is asked of one.
{ seat: "node-dns-resolver", scope: "node", tools: [{ name: "lookup", description: "one machine's", input: {} }] },
],
}));
await controller.handle("seat:node-dns-resolver.lookup@anchor", async () => ({ machine: "anchor", answered: true }));
await controller.handle("seat:mesh-controller.status", async () => ({ output: "all quiet", ok: true }));
await controller.handle("seat:mesh-controller.push", async (body: { node?: string }) => ({ told: body.node ?? "nobody" }));
t.after(async () => {
await catalogue.close();
await shop.close();
await controller.close();
});
const { mkdtemp, writeFile } = await import("node:fs/promises");
const { join } = await import("node:path");
const dir = await mkdtemp("/tmp/mesh-client-");
const path = join(dir, "credential.json");
await writeFile(
path,
JSON.stringify({ url, user: "person.ada", password: "x", person: "ada", invokes: ["shop.price"] }),
);
return path;
}
/** Drive `mesh mcp` over stdio and collect the replies, as a host would. */
function driving(credential: string, requests: unknown[]): Promise<Record<string, any>[]> {
return new Promise((resolve, reject) => {
const child = spawn(process.execPath, ["dist/mesh.js", "mcp", "--credential", credential], {
stdio: ["pipe", "pipe", "pipe"],
});
let out = "";
let err = "";
child.stdout.on("data", (d) => (out += d.toString()));
child.stderr.on("data", (d) => (err += d.toString()));
child.on("error", reject);
child.on("close", () => {
const replies = out
.split("\n")
.filter((l) => l.trim() !== "")
.map((l) => JSON.parse(l) as Record<string, any>);
if (replies.length === 0 && err !== "") reject(new Error(err));
else resolve(replies);
});
for (const r of requests) child.stdin.write(`${JSON.stringify(r)}\n`);
child.stdin.end();
});
}
test("a host initialises, lists the mesh's tools and calls one", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const credential = await aMeshAndACredential(t);
const replies = await driving(credential, [
{ jsonrpc: "2.0", id: 1, method: "initialize", params: {} },
{ jsonrpc: "2.0", method: "notifications/initialized" },
{ jsonrpc: "2.0", id: 2, method: "tools/list" },
{ jsonrpc: "2.0", id: 3, method: "tools/call", params: { name: "shop.price", arguments: { of: "a hat" } } },
]);
const byId = new Map(replies.map((r) => [r.id, r]));
// A notification is answered with nothing, or a host waiting on ids sees a reply it cannot match.
assert.equal(replies.length, 3, `expected three replies, got ${JSON.stringify(replies)}`);
const hello = byId.get(1)!.result;
assert.equal(hello.protocolVersion, "2025-03-26");
assert.ok(hello.capabilities.tools, "a server offering no tools is not this one");
assert.match(hello.instructions, /ada/, "the handshake says whose authority a call is made under");
const listed = byId.get(2)!.result.tools;
assert.deepEqual(listed.map((x: { name: string }) => x.name),
["mesh-controller.push", "mesh-controller.status", "node-dns-resolver.lookup", "shop.price"],
"the modules' tools and the roles', named the way a person names them");
// A node-scoped seat's verb takes the machine, and requires it (ADR 0170).
const lookup = listed[2];
assert.equal(lookup.inputSchema.properties.node.type, "string");
assert.deepEqual(lookup.inputSchema.required, ["node"]);
const price = listed[3];
assert.ok(price.inputSchema, "a tool with no schema is one an agent cannot call");
// A module's bare property map arrives as a schema an agent can read, its words kept — and
// `node`, the machine to ask when the module runs on several (novox/hq ADR 0159), beside them.
assert.deepEqual(price.inputSchema.properties.of, { type: "string" });
assert.equal(price.inputSchema.properties.node.type, "string", "a module's tool takes the machine to ask");
assert.ok(!listed[1].inputSchema.properties?.node, "a seat's verb takes no machine; the seat's scope decides");
assert.equal(listed[0].inputSchema.properties?.node?.type, "string", "a seat's verb that takes a node of its own keeps it");
// Silence is named: the module the catalogue holds and nothing answered for.
assert.deepEqual(byId.get(2)!.result._meta.notAnswering, ["ghost"]);
const called = byId.get(3)!.result;
assert.ok(!called.isError, `the call failed: ${JSON.stringify(called)}`);
// The module's own answer, unshaped. An adapter that summarised it would be deciding what matters
// in somebody else's answer.
assert.deepEqual(JSON.parse(called.content[0].text), { of: "a hat", cost: 12 });
});
test("a tool nobody serves comes back as an error the agent can act on", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const credential = await aMeshAndACredential(t);
const replies = await driving(credential, [
{ jsonrpc: "2.0", id: 1, method: "tools/call", params: { name: "ghost.missing", arguments: {} } },
]);
const result = replies[0].result;
// isError, not a protocol failure: the call was well-formed and the mesh answered it — with an
// absence. A JSON-RPC error would tell the agent its request was malformed, which it was not.
assert.ok(result?.isError, `expected a tool error, got ${JSON.stringify(replies[0])}`);
assert.match(result.content[0].text, /nothing serves ghost\.missing/);
});
test("a method this surface does not have is refused, and a notification is not", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const credential = await aMeshAndACredential(t);
const replies = await driving(credential, [
{ jsonrpc: "2.0", id: 1, method: "resources/list" },
{ jsonrpc: "2.0", method: "notifications/cancelled" },
]);
assert.equal(replies.length, 1, "a notification was answered");
assert.equal(replies[0].error.code, -32601);
assert.match(replies[0].error.message, /resources\/list/);
});
// A seat's verb that takes a machine as its own argument — `push <node>` — keeps it: the console
// moves `node` into the subject for a module's tool only (ADR 0159), never for a role's verb.
test("a seat's verb keeps a node of its own; only a module's tool gives it to the subject", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const credential = await aMeshAndACredential(t);
const replies = await driving(credential, [
{ jsonrpc: "2.0", id: 1, method: "tools/call", params: { name: "mesh-controller.push", arguments: { node: "anchor" } } },
]);
const result = replies[0].result;
assert.ok(!result.isError, JSON.stringify(replies[0]));
assert.deepEqual(JSON.parse(result.content[0].text), { told: "anchor" });
});
test("a host calls the mesh's own verb through the seat", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const credential = await aMeshAndACredential(t);
const replies = await driving(credential, [
{ jsonrpc: "2.0", id: 1, method: "tools/call", params: { name: "mesh-controller.status", arguments: {} } },
]);
const result = replies[0].result;
assert.ok(!result.isError, JSON.stringify(replies[0]));
assert.deepEqual(JSON.parse(result.content[0].text), { output: "all quiet", ok: true });
});
test("a node-scoped seat's verb is asked of the machine named, and refused without one", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const credential = await aMeshAndACredential(t);
const replies = await driving(credential, [
{ jsonrpc: "2.0", id: 1, method: "initialize", params: {} },
{ jsonrpc: "2.0", id: 2, method: "tools/call", params: { name: "node-dns-resolver.lookup", arguments: { node: "anchor" } } },
{ jsonrpc: "2.0", id: 3, method: "tools/call", params: { name: "node-dns-resolver.lookup", arguments: {} } },
]);
const byId = new Map(replies.map((r) => [r.id, r]));
const answered = byId.get(2)!.result;
assert.ok(!answered.isError, JSON.stringify(answered));
assert.match(answered.content[0].text, /"machine": "anchor"/, "the machine's holder answered");
assert.match(byId.get(3)!.error?.message ?? JSON.stringify(byId.get(3)), /name the machine/);
});
+255
View File
@@ -0,0 +1,255 @@
/**
* The mesh issues an assignment's subjects, and a runtime serves what it is issued (novox/hq ADR
* 0160). A runtime derives one address for itself — `mesh.assignment.<node>.<module>` — reads the
* membership there, serves exactly what it says, and re-serves when a new one arrives. Against a
* real bus with JetStream, because the membership is a direct get on a stream.
*
* docker run -d --rm --name t -p 14232:4222 nats:2.10-alpine -js
* MESH_TEST_NATS=nats://127.0.0.1:14232 node --test --experimental-strip-types test/membership.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
import { fileURLToPath } from "node:url";
import { connect, StringCodec } from "nats";
import { resetTools } from "@novox/mesh-sdk/tools";
import { connectNats, membershipSubject } from "../dist/broker-nats.js";
import { callTool, subjectListed } from "../dist/client.js";
import { runTools } from "../dist/runtime.js";
const url = process.env.MESH_TEST_NATS;
const fixture = (name: string) => fileURLToPath(new URL(`./fixtures/${name}`, import.meta.url));
const sc = StringCodec();
/** The ASSIGNMENTS stream as the controller asserts it: last-per-subject, readable by direct get. */
async function anAssignmentsStream() {
const nc = await connect({ servers: url! });
const jsm = await nc.jetstreamManager();
try {
await jsm.streams.delete("ASSIGNMENTS");
} catch {
// none yet
}
await jsm.streams.add({
name: "ASSIGNMENTS",
subjects: ["mesh.assignment.>"],
max_msgs_per_subject: 1,
allow_direct: true,
} as never);
return {
async issue(m: object & { node: string; module: string }) {
await nc.jetstream().publish(membershipSubject(m.node, m.module), sc.encode(JSON.stringify(m)));
},
async close() {
await nc.close();
},
};
}
test("a runtime serves exactly the subjects it is issued, and the listing carries them", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const stream = await anAssignmentsStream();
// The mesh placed shop on two machines that are not interchangeable: each is served by name only.
await stream.issue({
node: "anchor",
module: "shop",
serves: [{ subject: "mesh.mod.shop.tool.{tool}.anchor" }],
emits: "mesh.mod.shop.event.{event}",
tools: "mesh.mod.shop.tool.tools",
});
const shop = await connectNats({ url, node: "anchor", module: "shop" });
const asker = await connectNats({ url, module: "console", node: "workstation" });
let stop = () => {};
try {
stop = await runTools({ broker: shop, moduleEntrypoints: [fixture("shop-tools.mjs")] });
assert.equal(shop.membership()?.module, "shop", "the runtime read what the mesh issued");
// By name it answers; the plain subject was not issued, so nothing serves it.
const named = await callTool(asker, "shop.price@anchor", { of: "a hat" });
assert.deepEqual(named.result, { of: "a hat", cost: 12 });
assert.equal(named.node, "anchor");
await assert.rejects(callTool(asker, "shop.price", {}), /no responders|503/i, "a subject not issued is not served");
// The tools answer says where each is served, and the client composes nothing.
const tools = await asker.request<Record<string, never>, { tools: { name: string; subjects?: string[] }[] }>(
"shop.tools",
{},
);
assert.deepEqual(tools.tools.find((x) => x.name === "price")?.subjects, ["mesh.mod.shop.tool.price.anchor"]);
const listing = { tools: [{ module: "shop", name: "price", subjects: ["mesh.mod.shop.tool.price.anchor"] }], notAnswering: [] };
assert.equal(subjectListed("shop.price", "anchor", listing as never), "mesh.mod.shop.tool.price.anchor");
assert.equal(subjectListed("shop.price", "elsewhere", listing as never), undefined);
// The mesh re-issues the membership with the plain subject too (the modules became
// interchangeable); the runtime re-serves on it without a restart.
await stream.issue({
node: "anchor",
module: "shop",
serves: [{ subject: "mesh.mod.shop.tool.{tool}", queue: "serve.shop" }, { subject: "mesh.mod.shop.tool.{tool}.anchor" }],
emits: "mesh.mod.shop.event.{event}",
tools: "mesh.mod.shop.tool.tools",
});
await new Promise((r) => setTimeout(r, 300));
const plain = await callTool(asker, "shop.price", { of: "a coat" });
assert.deepEqual(plain.result, { of: "a coat", cost: 12 });
assert.equal(plain.node, "anchor");
} finally {
stop();
await asker.close();
await shop.close();
await stream.close();
}
});
test("a seat's verbs are implemented under the seat's name, served where issued, never listed as the module's", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const stream = await anAssignmentsStream();
await stream.issue({
node: "anchor",
module: "postgres",
serves: [{ subject: "mesh.mod.postgres.tool.{tool}", queue: "serve.postgres" }, { subject: "mesh.mod.postgres.tool.{tool}.anchor" }],
seats: [{ seat: "mesh-store", verb: "databases", subject: "mesh.seat.mesh-store.tool.databases" }],
emits: "mesh.mod.postgres.event.{event}",
tools: "mesh.mod.postgres.tool.tools",
});
const credential = { url, node: "anchor", module: "postgres", claims: [{ seat: "mesh-store", scope: "mesh", serves: ["databases", "query"] }] };
const pg = await connectNats(credential);
const asker = await connectNats({ url, module: "console", node: "workstation" });
let stop = () => {};
try {
stop = await runTools({ broker: pg, credential, moduleEntrypoints: [fixture("store-seat.mjs")] });
// The module's own `databases` and the seat's are two tools: postgres's on its subject, the
// store's on the seat's, each answering as itself.
const own = await callTool(asker, "postgres.databases", {});
assert.deepEqual(own.result, { software: "postgres" });
const seat = await callTool(asker, "seat:mesh-store.databases", {});
assert.deepEqual(seat.result, { seat: "mesh-store" });
assert.equal(seat.node, "anchor");
// What the seat does not promise — creating a database — is postgres's tool and not the store's.
await assert.rejects(callTool(asker, "seat:mesh-store.postgres_create_database", {}), /no responders|503/i);
// And the module's `tools` lists only postgres's own, never the seat's implementation.
const tools = await asker.request<Record<string, never>, { module: string; tools: { name: string }[] }>("postgres.tools", {});
assert.deepEqual(tools.tools.map((x) => x.name).sort(), ["databases", "postgres_create_database"]);
await assert.rejects(asker.request("mesh-store.tools", {}), /no responders|503/i, "a seat is not a module with a tools verb");
} finally {
stop();
await asker.close();
await pg.close();
await stream.close();
}
});
// novox/hq issue 218: the store seat is claimed by every machine running postgres and held by one.
// Where the mesh issued a membership without the seat, the claim in the credential serves nothing:
// the module's own tools answer, the seat's verbs do not, and the runtime does not announce them.
test("a claimant the membership does not make the holder serves none of the seat's verbs", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const stream = await anAssignmentsStream();
await stream.issue({
node: "elsewhere",
module: "postgres",
serves: [{ subject: "mesh.mod.postgres.tool.{tool}.elsewhere" }],
emits: "mesh.mod.postgres.event.{event}",
tools: "mesh.mod.postgres.tool.tools",
});
const credential = { url, node: "elsewhere", module: "postgres", claims: [{ seat: "mesh-store", scope: "mesh", serves: ["databases", "query"] }] };
const pg = await connectNats(credential);
const asker = await connectNats({ url, module: "console", node: "workstation" });
let stop = () => {};
try {
stop = await runTools({ broker: pg, credential, moduleEntrypoints: [fixture("store-claimant.mjs")] });
assert.deepEqual((await callTool(asker, "postgres.databases@elsewhere", {})).result, { software: "postgres" });
await assert.rejects(callTool(asker, "seat:mesh-store.databases", {}), /no responders|503/i, "the store is not held here");
} finally {
stop();
await asker.close();
await pg.close();
await stream.close();
}
});
test("a module named like its seat registers once, and answers as the module and as the seat", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const stream = await anAssignmentsStream();
await stream.issue({
node: "anchor",
module: "shop",
serves: [{ subject: "mesh.mod.shop.tool.{tool}", queue: "serve.shop" }, { subject: "mesh.mod.shop.tool.{tool}.anchor" }],
seats: [{ seat: "shop", verb: "price", subject: "mesh.seat.shop.tool.price" }],
emits: "mesh.mod.shop.event.{event}",
tools: "mesh.mod.shop.tool.tools",
});
const credential = { url, node: "anchor", module: "shop", claims: [{ seat: "shop", scope: "mesh", serves: ["price"] }] };
const shop = await connectNats(credential);
const asker = await connectNats({ url, module: "console", node: "workstation" });
let stop = () => {};
try {
stop = await runTools({ broker: shop, credential, moduleEntrypoints: [fixture("shop-seat.mjs")] });
assert.deepEqual((await callTool(asker, "shop.price", { of: "a hat" })).result, { of: "a hat", cost: 12 });
assert.deepEqual((await callTool(asker, "seat:shop.price", { of: "a hat" })).result, { of: "a hat", cost: 12 });
const tools = await asker.request<Record<string, never>, { tools: { name: string }[] }>("shop.tools", {});
assert.deepEqual(tools.tools.map((x) => x.name), ["price", "refund"]);
} finally {
stop();
await asker.close();
await shop.close();
await stream.close();
}
});
test("a mesh that has issued nothing yet gets the derived shape, and says so", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
const stream = await anAssignmentsStream();
const said: string[] = [];
const log = console.log;
console.log = (...a: unknown[]) => said.push(a.join(" "));
let lone;
try {
lone = await connectNats({ url, node: "home-server", module: "lone" });
} finally {
console.log = log;
}
const asker = await connectNats({ url, module: "console", node: "workstation" });
try {
assert.equal(lone.membership(), undefined);
assert.ok(said.some((s) => /no membership issued for lone on home-server/.test(s)), said.join("\n"));
await lone.handle("ping", async () => ({ pong: true }));
assert.deepEqual((await callTool(asker, "lone.ping", {})).result, { pong: true });
assert.deepEqual((await callTool(asker, "lone.ping@home-server", {})).result, { pong: true });
} finally {
await asker.close();
await lone.close();
await stream.close();
}
});
// A module that implements a seat its credential does not (yet) claim is not served for it and does
// not fall over either: the runtime says so and serves the module's own tools.
test("a registration under a seat the credential does not claim is said and skipped, not fatal", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const stream = await anAssignmentsStream();
const credential = { url, node: "anchor", module: "postgres" };
const pg = await connectNats(credential);
const asker = await connectNats({ url, module: "console", node: "workstation" });
const said: string[] = [];
const log = console.log;
console.log = (...a: unknown[]) => said.push(a.join(" "));
let stop = () => {};
try {
stop = await runTools({ broker: pg, credential, moduleEntrypoints: [fixture("store-seat-unclaimed.mjs")] });
console.log = log;
assert.ok(said.some((s) => /registers tools under "mesh-store".*not served/.test(s)), said.join("\n"));
assert.deepEqual((await callTool(asker, "postgres.databases", {})).result, { software: "postgres" });
await assert.rejects(callTool(asker, "seat:mesh-store.databases", {}), /no responders|503/i);
} finally {
console.log = log;
stop();
await asker.close();
await pg.close();
await stream.close();
}
});
+350
View File
@@ -0,0 +1,350 @@
/**
* One runtime per node serves every assigned module's tools (novox/hq ADR 0175, to-be 38 WP1). The
* node's runtime is handed a list of modules and their bundles on the node's credential; it reads
* one membership per module, serves each module's tools on that module's subjects and each held
* seat's verbs on the seat's, names a bundle that fails to load without dropping the others, and
* re-serves a module whose membership is re-issued mid-run. Against a real bus with JetStream.
*
* docker run -d --rm --name t -p 14232:4222 nats:2.10-alpine -js
* MESH_TEST_NATS=nats://127.0.0.1:14232 node --test --experimental-strip-types test/node-runtime.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
import { fileURLToPath } from "node:url";
import { cpSync, mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { connect, StringCodec } from "nats";
import { resetTools } from "@novox/mesh-sdk/tools";
import { connectNats, membershipSubject } from "../dist/broker-nats.js";
import { callTool, toolsOn } from "../dist/client.js";
import { servedModulesFrom } from "../dist/main.js";
import { runTools, takeToolEnvs } from "../dist/runtime.js";
const url = process.env.MESH_TEST_NATS;
const fixture = (name: string) => fileURLToPath(new URL(`./fixtures/${name}`, import.meta.url));
const sc = StringCodec();
/** The controller's job, done by hand: the ASSIGNMENTS stream (last-per-subject, direct get) and an
* EVENTS stream for what a tool emits. */
async function aMesh() {
const nc = await connect({ servers: url! });
const jsm = await nc.jetstreamManager();
for (const name of ["ASSIGNMENTS", "EVENTS"]) {
try {
await jsm.streams.delete(name);
} catch {
// none yet
}
}
await jsm.streams.add({ name: "ASSIGNMENTS", subjects: ["mesh.assignment.>"], max_msgs_per_subject: 1, allow_direct: true } as never);
await jsm.streams.add({ name: "EVENTS", subjects: ["mesh.mod.*.event.>"] });
return {
async issue(m: object & { node: string; module: string }) {
await nc.jetstream().publish(membershipSubject(m.node, m.module), sc.encode(JSON.stringify(m)));
},
/** The next subject an event lands on, under a pattern. */
nextEvent(pattern: string): Promise<string> {
const sub = nc.subscribe(pattern, { max: 1 });
return (async () => {
for await (const m of sub) return m.subject;
throw new Error("no event");
})();
},
async close() {
await nc.close();
},
};
}
/** A membership as the controller issues one on a machine, with the module's own subject when it
* answers for the module anywhere, and the verbs of the node seats it holds. */
function membershipOf(module: string, node: string, opts: { plain?: boolean; seats?: Record<string, string[]> } = {}) {
const own = `mesh.mod.${module}`;
const serves: { subject: string; queue?: string }[] = [{ subject: `${own}.tool.{tool}.${node}` }];
if (opts.plain) serves.push({ subject: `${own}.tool.{tool}`, queue: `serve.${module}` });
const seats = Object.entries(opts.seats ?? {}).flatMap(([seat, verbs]) =>
verbs.map((verb) => ({ seat, verb, subject: `mesh.seat.${seat}.tool.${verb}.${node}` })));
return { node, module, serves, seats, emits: `${own}.event.{event}`, tools: `${own}.tool.tools` };
}
/** Wait for something to be served: the bus answers "no responders" at once until it is. */
async function until<T>(attempt: () => Promise<T>, tries = 50): Promise<T> {
for (let i = 0; ; i++) {
try {
return await attempt();
} catch (e) {
if (i >= tries) throw e;
await new Promise((r) => setTimeout(r, 100));
}
}
}
test("MESH_TOOL_MODULES names modules and their entrypoints; a bare path is the credential's own module's", () => {
const have = servedModulesFrom(" alpha=/a/tools/index.js, beta=/b/one.js ,beta=/b/two.js, /mine/index.js ,node-tools=/own/x.js", "node-tools");
assert.deepEqual(have.serves, [
{ module: "alpha", entrypoints: ["/a/tools/index.js"] },
{ module: "beta", entrypoints: ["/b/one.js", "/b/two.js"] },
]);
// The credential's own module, named or bare, is the one-module form either way.
assert.deepEqual(have.moduleEntrypoints, ["/mine/index.js", "/own/x.js"]);
assert.throws(() => servedModulesFrom("=/nothing.js", undefined), /neither <module>=<entrypoint>/);
assert.deepEqual(servedModulesFrom("", "x"), { serves: [], moduleEntrypoints: [] });
});
test("the node's runtime serves five modules' bundles on one credential — two of them launched, one broken — and follows a re-issued membership", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const mesh = await aMesh();
// Three modules assigned to the machine: alpha answers for itself anywhere, beta only here and
// holds the node-shelf seat, gamma's bundle is broken.
await mesh.issue(membershipOf("alpha", "anchor", { plain: true }));
await mesh.issue(membershipOf("beta", "anchor", { seats: { "node-shelf": ["list", "clear"] } }));
await mesh.issue(membershipOf("gamma", "anchor"));
// Two more, launched rather than loaded (ADR 0188): delta is Python and holds the node-lamp seat;
// epsilon is TypeScript written against the protocol and marked executable.
await mesh.issue(membershipOf("delta", "anchor", { seats: { "node-lamp": ["on"] } }));
await mesh.issue(membershipOf("epsilon", "anchor"));
// The node's credential: the runtime module's name, no claims (seats come from the memberships).
const credential = { url, node: "anchor", module: "node-tools" };
const nodeTools = await connectNats(credential);
const asker = await connectNats({ url, module: "console", node: "workstation" });
const said: string[] = [];
const log = console.log;
console.log = (...a: unknown[]) => said.push(a.join(" "));
let stop = () => {};
try {
process.env.MESH_OPERATOR_ACCOUNT = "somebody";
process.env.MESH_OPERATOR_HOME = "/home/somebody";
stop = await runTools({
broker: nodeTools,
credential,
serves: [
{ module: "alpha", entrypoints: [fixture("many-alpha.serve.mjs")] },
{ module: "beta", entrypoints: [fixture("many-beta.serve.mjs")] },
{ module: "gamma", entrypoints: [fixture("many-broken.serve.mjs")] },
{ module: "delta", entrypoints: [fixture("many-delta.py")] },
{ module: "epsilon", entrypoints: [fixture("many-epsilon.mjs")] },
],
});
console.log = log;
assert.deepEqual(nodeTools.serving().sort(), ["alpha", "beta", "delta", "epsilon", "gamma", "node-tools"]);
assert.ok(said.some((s) => /the operator's account here is somebody \(home \/home\/somebody\)/.test(s)), said.join("\n"));
assert.ok(said.some((s) => /gamma's bundle .*many-broken\.serve\.mjs failed to load: gamma's bundle exited \(1\): Error: gamma's bundle cannot find its client; its tools are not served here/.test(s)), said.join("\n"));
assert.ok(said.some((s) => /serving 8 tool\(s\) for 5 module\(s\): alpha\.one, alpha\.two, beta\.three, beta\.four, beta\.five, delta\.greet, delta\.die, epsilon\.seven; not serving gamma/.test(s)), said.join("\n"));
// Five tools answer, each where its module's membership says: alpha anywhere and here, beta here only.
assert.deepEqual((await callTool(asker, "alpha.one", {})).result, { alpha: 1 });
assert.deepEqual((await callTool(asker, "alpha.one@anchor", {})).result, { alpha: 1 });
assert.deepEqual((await callTool(asker, "beta.three@anchor", {})).result, { beta: 3 });
assert.deepEqual((await callTool(asker, "beta.four@anchor", {})).result, { beta: 4 });
assert.deepEqual((await callTool(asker, "beta.five@anchor", {})).result, { beta: 5 });
await assert.rejects(callTool(asker, "beta.three", {}), /no responders|503/i, "beta was not issued the module's plain subject");
// Two seat verbs answer on the seat's subjects, held by beta.
assert.deepEqual((await callTool(asker, "seat:node-shelf.list@anchor", {})).result, { shelf: ["a", "b"] });
assert.deepEqual((await callTool(asker, "seat:node-shelf.clear@anchor", {})).result, { cleared: true });
// A bundle in another language answers the same way, its seat verb among them; so does a
// TypeScript bundle served through the protocol rather than imported.
assert.deepEqual((await callTool(asker, "delta.greet@anchor", { who: "mesh" })).result, { greeting: "hello mesh", language: "python" });
assert.deepEqual((await callTool(asker, "seat:node-lamp.on@anchor", {})).result, { on: true, language: "python" });
assert.deepEqual((await callTool(asker, "epsilon.seven@anchor", {})).result, { epsilon: 7, via: "stdio" });
// A child that exits mid-call tells the caller so and is started again on the next call.
await assert.rejects(callTool(asker, "delta.die@anchor", {}), /delta's bundle exited \(3\)/);
assert.deepEqual((await callTool(asker, "delta.greet@anchor", {})).result, { greeting: "hello world", language: "python" });
// A tool that emits does so as its module, not as the runtime.
const landed = mesh.nextEvent("mesh.mod.*.event.>");
assert.deepEqual((await callTool(asker, "alpha.two", {})).result, { alpha: 2 });
assert.equal(await landed, "mesh.mod.alpha.event.happened");
// `tools` answers for each: what alpha and beta serve, and why gamma serves nothing.
const gamma = await asker.request<Record<string, never>, { module: string; tools: unknown[]; failed?: string }>("gamma.tools@anchor", {});
assert.deepEqual(gamma, { module: "gamma", tools: [], failed: "gamma's bundle exited (1): Error: gamma's bundle cannot find its client" });
const beta = await asker.request<Record<string, never>, { tools: { name: string; subjects?: string[] }[] }>("beta.tools@anchor", {});
assert.deepEqual(beta.tools.map((x) => x.name), ["three", "four", "five"]);
assert.deepEqual(beta.tools[0]!.subjects, ["mesh.mod.beta.tool.three.anchor"]);
// And discovery says so, with the reason, beside the modules that answered.
const catalogue = await connectNats({ url, module: "mesh-catalog" });
await catalogue.handle("catalog_modules", async () => ({ modules: [{ module: "alpha" }, { module: "beta" }, { module: "gamma" }, { module: "delta" }, { module: "epsilon" }] }));
try {
const have = await toolsOn(asker);
assert.deepEqual(have.tools.map((x) => `${x.module}.${x.name}`), ["alpha.one", "alpha.two", "beta.five", "beta.four", "beta.three", "delta.die", "delta.greet", "epsilon.seven"]);
assert.deepEqual(have.notAnswering, ["gamma (its tools bundle failed to load: gamma's bundle exited (1): Error: gamma's bundle cannot find its client)", "mesh-controller (seat)"]);
} finally {
await catalogue.close();
}
// The mesh re-issues beta's membership mid-run — now answering for the module anywhere — and
// the runtime serves the new subject without a restart.
await mesh.issue(membershipOf("beta", "anchor", { plain: true, seats: { "node-shelf": ["list", "clear"] } }));
assert.deepEqual((await until(() => callTool(asker, "beta.three", {}))).result, { beta: 3 });
assert.deepEqual((await callTool(asker, "seat:node-shelf.list@anchor", {})).result, { shelf: ["a", "b"] });
} finally {
console.log = log;
delete process.env.MESH_OPERATOR_ACCOUNT;
delete process.env.MESH_OPERATOR_HOME;
stop();
await asker.close();
await nodeTools.close();
await mesh.close();
resetTools();
}
});
test("a bundle carrying its own copy of the SDK registers into the runtime's registry, and its tools are served (issue 209)", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
let dir = "";
let stop = () => {};
const closing: Array<() => Promise<void>> = [];
const said: string[] = [];
const log = console.log;
try {
// A bundle as the toolchain packs one: its compiled entrypoint, a package.json saying ES modules,
// and its dependencies copied in — the SDK among them, a second copy beside the runtime's own,
// and a dependency of the bundle's own that the runtime does not carry.
dir = mkdtempSync(join(tmpdir(), "mesh-bundle-"));
const sdk = realpathSync(fileURLToPath(new URL("../node_modules/@novox/mesh-sdk/", import.meta.url)));
cpSync(sdk, join(dir, "node_modules", "@novox", "mesh-sdk"), { recursive: true });
mkdirSync(join(dir, "node_modules", "zeta-flavour"), { recursive: true });
writeFileSync(join(dir, "node_modules", "zeta-flavour", "package.json"), '{"name":"zeta-flavour","type":"module","main":"index.js"}\n');
writeFileSync(join(dir, "node_modules", "zeta-flavour", "index.js"), 'export const flavour = "the bundle\'s own";\n');
writeFileSync(join(dir, "package.json"), '{"type":"module","private":true}\n');
writeFileSync(join(dir, "index.serve.mjs"),
'#!/usr/bin/env node\nimport { serveRegisteredOverStdio } from "@novox/mesh-sdk/stdio";\nawait import("./index.js");\nawait serveRegisteredOverStdio();\n', { mode: 0o755 });
writeFileSync(join(dir, "index.js"),
'import { registerModuleTools } from "@novox/mesh-sdk/tools";\n' +
'import { flavour } from "zeta-flavour";\n' +
'registerModuleTools("zeta", () => [{ name: "probe", description: "answers", input: {}, run: async () => ({ zeta: true, flavour }) }]);\n');
const mesh = await aMesh();
closing.push(() => mesh.close());
await mesh.issue(membershipOf("zeta", "anchor"));
const credential = { url, node: "anchor", module: "node-tools" };
const nodeTools = await connectNats(credential);
closing.push(() => nodeTools.close());
const asker = await connectNats({ url, module: "console", node: "workstation" });
closing.push(() => asker.close());
console.log = (...a: unknown[]) => said.push(a.join(" "));
stop = await runTools({ broker: nodeTools, credential, serves: [{ module: "zeta", entrypoints: [join(dir, "index.serve.mjs")] }] });
console.log = log;
assert.ok(said.some((s) => /serving 1 tool\(s\) for 1 module\(s\): zeta\.probe/.test(s)), said.join("\n"));
// The SDK is the runtime's (the registration arrived); the bundle's other dependency is its own.
assert.deepEqual((await callTool(asker, "zeta.probe@anchor", {})).result, { zeta: true, flavour: "the bundle's own" });
} finally {
console.log = log;
stop();
for (const close of closing.reverse()) await close();
if (dir) rmSync(dir, { recursive: true, force: true });
resetTools();
}
});
test("each bundle is given its own environment and none of another's, imported or launched (ADR 0192)", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const mesh = await aMesh();
for (const m of ["gamma", "delta", "zeta"]) await mesh.issue(membershipOf(m, "anchor"));
const credential = { url, node: "anchor", module: "node-tools" };
const nodeTools = await connectNats(credential);
const asker = await connectNats({ url, module: "console", node: "workstation" });
const log = console.log;
let stop = () => {};
const before = process.env.MESH_TOOL_ENV;
try {
process.env.MESH_OPERATOR_ACCOUNT = "somebody";
process.env.MESH_TOOL_ENV = JSON.stringify({
gamma: { GAMMA_CONFIG_FILE: "/var/lib/mesh/gamma/config.json" },
delta: { DELTA_TOKEN_FILE: "/var/lib/mesh/delta/token" },
zeta: { ZETA_URL: "http://127.0.0.1:3000" },
});
const envs = takeToolEnvs();
assert.equal(process.env.MESH_TOOL_ENV, undefined, "the composed environments were left in the process's");
console.log = () => {};
stop = await runTools({
broker: nodeTools, credential, envs,
serves: [
{ module: "gamma", entrypoints: [fixture("env-gamma.serve.mjs")] },
{ module: "delta", entrypoints: [fixture("env-delta.serve.mjs")] },
{ module: "zeta", entrypoints: [fixture("env-zeta.mjs")] },
],
});
console.log = log;
assert.deepEqual((await callTool(asker, "gamma.given@anchor", {})).result,
{ mine: "/var/lib/mesh/gamma/config.json", theirs: null, runtime: "somebody", composed: null });
assert.deepEqual((await callTool(asker, "delta.given@anchor", {})).result,
{ mine: "/var/lib/mesh/delta/token", theirs: null });
assert.deepEqual((await callTool(asker, "zeta.given@anchor", {})).result,
{ mine: "http://127.0.0.1:3000", theirs: null, composed: null });
} finally {
console.log = log;
if (before === undefined) delete process.env.MESH_TOOL_ENV; else process.env.MESH_TOOL_ENV = before;
delete process.env.MESH_OPERATOR_ACCOUNT;
stop();
await asker.close();
await nodeTools.close();
await mesh.close();
resetTools();
}
});
test("a launched bundle is told the module it serves, so its seat's verbs stay the seat's (ADR 0193)", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const mesh = await aMesh();
await mesh.issue(membershipOf("theta", "anchor", { seats: { "node-shelf": ["list"] } }));
const credential = { url, node: "anchor", module: "node-tools" };
const nodeTools = await connectNats(credential);
const asker = await connectNats({ url, module: "console", node: "workstation" });
const log = console.log;
let stop = () => {};
try {
console.log = () => {};
stop = await runTools({ broker: nodeTools, credential, envs: new Map([["theta", { THETA_WORD: "given" }]]),
serves: [{ module: "theta", entrypoints: [fixture("served-seat-first.mjs")] }] });
console.log = log;
assert.deepEqual((await callTool(asker, "theta.own@anchor", {})).result, { theta: "given" });
assert.deepEqual((await callTool(asker, "seat:node-shelf.list@anchor", {})).result, { shelf: ["x"] });
} finally {
console.log = log;
stop();
await asker.close();
await nodeTools.close();
await mesh.close();
resetTools();
}
});
test("an entrypoint that is not executable is refused by name, and the others serve (ADR 0193)", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const mesh = await aMesh();
for (const m of ["alpha", "plain"]) await mesh.issue(membershipOf(m, "anchor"));
const credential = { url, node: "anchor", module: "node-tools" };
const nodeTools = await connectNats(credential);
const asker = await connectNats({ url, module: "console", node: "workstation" });
const said: string[] = [];
const log = console.log;
let stop = () => {};
try {
console.log = (...a: unknown[]) => said.push(a.join(" "));
stop = await runTools({ broker: nodeTools, credential, serves: [
{ module: "alpha", entrypoints: [fixture("many-alpha.serve.mjs")] },
{ module: "plain", entrypoints: [fixture("many-alpha.mjs")] },
] });
console.log = log;
assert.ok(said.some((s) => /plain's bundle .*many-alpha\.mjs failed to load: .* is not executable; a bundle the runtime serves is started, never imported/.test(s)), said.join("\n"));
assert.deepEqual((await callTool(asker, "alpha.one@anchor", {})).result, { alpha: 1 });
} finally {
console.log = log;
stop();
await asker.close();
await nodeTools.close();
await mesh.close();
resetTools();
}
});
+63
View File
@@ -0,0 +1,63 @@
/**
* The runtime as the node-tools module (novox/hq ADR 0175 §6, to-be 38 WP3): started the way the
* host starts it — `main.js` with the node's credential and MESH_TOOL_MODULES — it serves the bundles
* AND answers MCP on loopback as the console, through which a tool it serves can be called.
*
* docker run -d --rm --name t -p 14232:4222 nats:2.10-alpine -js
* MESH_TEST_NATS=nats://127.0.0.1:14232 node --test --experimental-strip-types test/node-tools-serve.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
import { spawn, type ChildProcess } from "node:child_process";
import { mkdtemp, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { connectNats } from "../dist/broker-nats.js";
const url = process.env.MESH_TEST_NATS;
const fixture = (name: string) => fileURLToPath(new URL(`./fixtures/${name}`, import.meta.url));
async function post(endpoint: string, body: unknown): Promise<any> {
const res = await fetch(endpoint, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
return res.json();
}
test("as node-tools, serve loads the bundles and is the console on loopback", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
// The catalogue, for discovery; the node credential names the runtime module and no claims.
const catalogue = await connectNats({ url, module: "mesh-catalog" });
await catalogue.handle("catalog_modules", async () => ({ modules: [{ module: "alpha" }] }));
t.after(() => catalogue.close());
const dir = await mkdtemp("/tmp/node-tools-");
const credential = join(dir, "broker");
await writeFile(credential, JSON.stringify({ url, node: "desk", module: "node-tools", user: "desk.node-tools", password: "x" }));
const child: ChildProcess = spawn(process.execPath, ["dist/main.js"], {
env: {
...process.env,
MESH_BROKER_FILE: credential,
MESH_TOOL_MODULES: `alpha=${fixture("many-alpha.serve.mjs")}`,
MESH_CONSOLE_LISTEN: "127.0.0.1:0",
},
stdio: ["ignore", "pipe", "pipe"],
});
t.after(() => {
child.kill("SIGTERM");
});
const endpoint = await new Promise<string>((resolve, reject) => {
let out = "";
let err = "";
child.stdout!.on("data", (d) => {
out += d.toString();
const m = /listening on (http:\/\/[^/]+\/mcp) as desk\.node-tools/.exec(out);
if (m) resolve(m[1]!);
});
child.stderr!.on("data", (d) => (err += d.toString()));
child.on("exit", (code) => reject(new Error(`serve exited ${code}: ${err}`)));
});
const listed = await post(endpoint, { jsonrpc: "2.0", id: 1, method: "tools/list" });
assert.deepEqual(listed.result.tools.map((x: any) => x.name).filter((n: string) => n.startsWith("alpha.")), ["alpha.one", "alpha.two"]);
// A tool the same process serves on the bus, called through the console it also is.
const called = await post(endpoint, { jsonrpc: "2.0", id: 2, method: "tools/call", params: { name: "alpha.one", arguments: { node: "desk" } } });
assert.deepEqual(JSON.parse(called.result.content[0].text), { alpha: 1 });
});
+80
View File
@@ -0,0 +1,80 @@
import { spawn } from "node:child_process";
import { test } from "node:test";
import assert from "node:assert/strict";
import { fatalBrokerReason, PinMismatchError, topicMatches } from "../src/broker-nats.ts";
// novox/hq issue 058 (and its review): serve mode retries a broker that is not up yet, but must
// give up at once on a failure waiting cannot fix — otherwise a permanent fault loops for ever
// disguised as "not reachable". This is the classifier that draws the line; the bed cannot test it
// (it starts the consumer only after the broker is up), so it is proven here.
test("a broker that is not up yet is retryable, not fatal", () => {
for (const err of [
Object.assign(new Error("connect ECONNREFUSED 10.42.0.1:5671"), { code: "ECONNREFUSED" }),
Object.assign(new Error("connect ETIMEDOUT"), { code: "ETIMEDOUT" }),
Object.assign(new Error("getaddrinfo EAI_AGAIN anchor.internal"), { code: "EAI_AGAIN" }),
new Error("timed out fetching the broker's certificate"),
]) {
assert.equal(fatalBrokerReason(err), null, `should retry: ${(err as Error).message}`);
}
});
test("a certificate that does not match the pin is fatal, by type not by message", () => {
// Typed, so rewording the message cannot turn an impostor back into an infinite retry.
assert.notEqual(fatalBrokerReason(new PinMismatchError("anything at all")), null);
// A plain Error with pin-ish words is NOT treated as the pin case — only the type is.
assert.equal(fatalBrokerReason(new Error("the pinned value was fine")), null);
});
test("a malformed broker URL is fatal — it never parses on the next try", () => {
assert.notEqual(fatalBrokerReason(Object.assign(new Error("Invalid URL"), { code: "ERR_INVALID_URL" })), null);
assert.notEqual(fatalBrokerReason(new Error("Invalid URL: not-a-url")), null);
});
test("a refused login is fatal — a wrong or revoked credential, not an absent broker", () => {
// The bus refuses a login in its own words; each is final, because the next try says the same.
for (const msg of ["Authorization Violation", "nats: user authentication expired", "Permissions Violation for Subscription to \"x\""]) {
assert.notEqual(fatalBrokerReason(new Error(msg)), null, `should be fatal: ${msg}`);
}
});
test("a non-Error value does not crash the classifier", () => {
assert.equal(fatalBrokerReason("just a string"), null);
assert.equal(fatalBrokerReason(undefined), null);
});
// **A module asked to prepare its state and naming nothing is a failure, not a no-op** (novox/hq
// ADR 0135). The mesh asks this only of a module whose manifest says it prepares something, so an
// image that names nothing was built wrong, and exiting 0 would let that version serve against a
// state nobody shaped.
test("preparing with nothing named fails rather than passing quietly", async () => {
const runtime = new URL("../dist/main.js", import.meta.url).pathname;
const ran = await new Promise<{ code: number | null; said: string }>((resolve) => {
const child = spawn(process.execPath, [runtime, "prepare"], {
env: { ...process.env, MESH_PREPARE: "" },
});
let said = "";
child.stderr.on("data", (chunk) => (said += String(chunk)));
child.on("close", (code) => resolve({ code, said }));
});
assert.notEqual(ran.code, 0, "a module that prepares nothing exited 0, so its version would serve");
assert.match(ran.said, /MESH_PREPARE/);
});
// **One durable consumer feeds one reader, however many patterns a module registers.**
//
// A module has exactly one consumer, so two readers of it would each take half the messages — and a
// reader that received one its own pattern does not match acknowledges it, which is right for a
// filter wider than anything registered and silent loss when it is another handler's. The matching is
// therefore pure and tested as such: what a message is for is decided by the patterns registered, not
// by which reader happened to fetch it.
test("a message is for every pattern that matches it, and nothing else", () => {
const registered = ["mesh-build-machine.built", "mesh-controller.built-before"];
const matched = (key: string) => registered.filter((p) => topicMatches(p, key));
assert.deepEqual(matched("mesh-build-machine.built"), ["mesh-build-machine.built"]);
assert.deepEqual(matched("mesh-controller.built-before"), ["mesh-controller.built-before"]);
// Nothing registered for it: the consumer's filter is the controller's and may be wider.
assert.deepEqual(matched("mesh-catalog.upgraded"), []);
// And a handler that asked for everything gets both, which is what the audit logger does.
assert.deepEqual(["#"].filter((p) => topicMatches(p, "mesh-controller.built-before")), ["#"]);
});
+68
View File
@@ -0,0 +1,68 @@
/**
* A refused announcement is said and never fatal (novox/hq 04-ISSUES/217): against a bus whose
* permissions refuse one discovery subject, the runtime's raw subscription is refused, logged, and
* the process keeps serving — its tools still answer.
*
* docker run -d --rm --name t -p 14233:4222 -v $PWD/test/fixtures/refusing-nats.conf:/c.conf nats:2.10-alpine -c /c.conf
* MESH_TEST_REFUSING_NATS=nats://127.0.0.1:14233 node --test --experimental-strip-types test/refused.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
import { connectNats } from "../dist/broker-nats.js";
import { callTool } from "../dist/client.js";
const url = process.env.MESH_TEST_REFUSING_NATS;
test("a refused discovery subscription is logged and the runtime serves on", async (t) => {
if (!url) return t.skip("MESH_TEST_REFUSING_NATS unset");
const bus = await connectNats({ url, user: "runtime", password: "runtime", module: "alpha", node: "anchor" });
const said: string[] = [];
const log = console.log;
console.log = (...a: unknown[]) => said.push(a.join(" "));
const crashed: unknown[] = [];
const onRejection = (e: unknown) => crashed.push(e);
process.on("unhandledRejection", onRejection);
try {
(bus as unknown as { raw: (s: string, f: () => Uint8Array | undefined) => () => void }).raw("$SRV.PING.>", () => undefined);
const stop = await bus.handle("alpha.ping", async () => ({ pong: true }));
for (let i = 0; i < 50 && !said.some((s) => s.includes("the bus refused $SRV.PING.>")); i++) await new Promise((r) => setTimeout(r, 50));
console.log = log;
assert.ok(said.some((s) => /the bus refused \$SRV\.PING\.>.*serves on/.test(s)), said.join("\n"));
assert.equal(crashed.length, 0, `the refusal escaped: ${String(crashed[0])}`);
stop();
} finally {
console.log = log;
process.off("unhandledRejection", onRejection);
await bus.close();
}
});
// novox/hq issue 218: a tool subject the grants leave out — a seat claimed here and held elsewhere —
// is refused, said, and the module's other tools still answer.
test("a refused tool subscription is logged and the module's other tools answer", async (t) => {
if (!url) return t.skip("MESH_TEST_REFUSING_NATS unset");
const bus = await connectNats({ url, user: "runtime", password: "runtime", module: "alpha", node: "anchor" });
const asker = await connectNats({ url, user: "runtime", password: "runtime", module: "console", node: "workstation" });
const said: string[] = [];
const log = console.log;
console.log = (...a: unknown[]) => said.push(a.join(" "));
const crashed: unknown[] = [];
const onRejection = (e: unknown) => crashed.push(e);
process.on("unhandledRejection", onRejection);
try {
const refused = await bus.handleSubject!("mesh.seat.held-elsewhere.tool.databases", async () => ({ seat: true }));
const stop = await bus.handle("alpha.ping", async () => ({ pong: true }));
for (let i = 0; i < 50 && !said.some((s) => s.includes("the bus refused mesh.seat.held-elsewhere")); i++) await new Promise((r) => setTimeout(r, 50));
console.log = log;
assert.ok(said.some((s) => /the bus refused mesh\.seat\.held-elsewhere\.tool\.databases.*serves on/.test(s)), said.join("\n"));
assert.equal(crashed.length, 0, `the refusal escaped: ${String(crashed[0])}`);
assert.deepEqual((await callTool(asker, "alpha.ping@anchor", {})).result, { pong: true });
stop();
refused();
} finally {
console.log = log;
process.off("unhandledRejection", onRejection);
await asker.close();
await bus.close();
}
});
+57
View File
@@ -0,0 +1,57 @@
// Round-trip the runtime's NATS client against a real server: a tool call answered, and an
// event emitted and received with its envelope intact.
import { connect } from "nats";
import { connectNats } from "../dist/broker-nats.js";
const URL = "nats://127.0.0.1:14222";
// The controller's job, done by hand here: the stream and the module's durable consumer.
const admin = await connect({ servers: URL });
const jsm = await admin.jetstreamManager();
await jsm.streams.add({ name: "EVENTS", subjects: ["mesh.mod.*.event.>", "mesh.seat.*.event.>"] });
await jsm.consumers.add("EVENTS", {
durable_name: "one_audit", ack_policy: "explicit",
filter_subjects: ["mesh.mod.shop.event.order.placed"],
});
const shop = await connectNats({ url: URL, node: "one", module: "shop" });
const audit = await connectNats({ url: URL, node: "one", module: "audit" });
let failures = 0;
const check = (ok, what) => { console.log(` ${ok ? "ok " : "FAIL"} ${what}`); if (!ok) failures++; };
// A tool, served and called.
await shop.handle("price", async (body) => ({ total: body.qty * 3 }));
const answer = await audit.request("shop.price", { qty: 4 });
check(answer.total === 12, "a tool call is answered across two connections");
// A handler that throws reaches the caller as an error, not a timeout.
await shop.handle("boom", async () => { throw new Error("no"); });
let threw = null;
try { await audit.request("shop.boom", {}); } catch (e) { threw = e.message; }
check(threw === "no", "a handler that throws answers the caller instead of timing out");
// An event, emitted and received with its envelope intact.
const seen = [];
await audit.subscribe("order.placed", async (env) => { seen.push(env); });
await shop.publish({
key: "order.placed", node: "one", body: { id: "a1" },
headers: { "x-event-id": "e1", "x-node": "one", "content-type": "application/json" },
});
await new Promise((r) => setTimeout(r, 800));
check(seen.length === 1, `exactly one delivery (saw ${seen.length})`);
if (seen[0]) {
check(seen[0].key === "order.placed", "the key survives the subject round trip");
check(seen[0].body?.id === "a1", "the body is the payload, not the whole envelope");
check(seen[0].node === "one", "the node comes back from the headers");
check(seen[0].headers?.["x-event-id"] === "e1", "the event id survives as a header");
}
// A module cannot reach into another's namespace by naming its own event oddly.
await shop.publish({ key: "other", node: "one", body: {}, headers: { "x-event-id": "e2" } });
const msg = await jsm.streams.getMessage("EVENTS", { last_by_subj: "mesh.mod.shop.event.other" });
check(!!msg, "an event lands under the emitting module's own namespace");
await shop.close(); await audit.close(); await admin.close();
console.log(failures ? `\n${failures} failed` : "\nall passed");
process.exit(failures ? 1 : 0);
+60
View File
@@ -0,0 +1,60 @@
/**
* Every module's runtime answers `tools` for it (novox/hq ADR 0152, design 34 §3): the names,
* descriptions and schemas from the code that answers them. Against a real bus, because the claim is
* what a second connection gets back.
*
* docker run -d --rm --name t -p 14232:4222 nats:2.10-alpine -js
* MESH_TEST_NATS=nats://127.0.0.1:14232 node --test --experimental-strip-types test/runtime-tools.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
import { fileURLToPath } from "node:url";
import { resetTools } from "@novox/mesh-sdk/tools";
import { connectNats } from "../dist/broker-nats.js";
import { runTools } from "../dist/runtime.js";
const url = process.env.MESH_TEST_NATS;
const fixture = (name: string) => fileURLToPath(new URL(`./fixtures/${name}`, import.meta.url));
test("a module registering two tools answers three names, the third being what it serves", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const shop = await connectNats({ url, node: "one", module: "shop" });
const asker = await connectNats({ url, module: "person.ada" });
const stop = await runTools({ broker: shop, moduleEntrypoints: [fixture("shop-tools.mjs")] });
try {
const answer = await asker.request<Record<string, never>, { module: string; tools: { name: string; input: unknown }[] }>(
"shop.tools",
{},
);
assert.equal(answer.module, "shop");
assert.deepEqual(answer.tools.map((x) => x.name), ["price", "refund"]);
// The schema travels with the name: a name alone is not callable by something that has never
// seen the mesh before.
assert.deepEqual(answer.tools[0].input, { of: { type: "string", description: "the thing" } });
// And the tools themselves still answer beside it.
const priced = await asker.request<{ of: string }, { cost: number }>("shop.price", { of: "a hat" });
assert.equal(priced.cost, 12);
} finally {
stop();
await asker.close();
await shop.close();
}
});
test("a module naming a tool of its own `tools` is refused at load", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
resetTools();
const clash = await connectNats({ url, node: "one", module: "clash" });
try {
await assert.rejects(
() => runTools({ broker: clash, moduleEntrypoints: [fixture("clash-tools.mjs")] }),
/names a tool "tools"/,
);
} finally {
resetTools();
await clash.close();
}
});

Some files were not shown because too many files have changed in this diff Show More