Commit Graph
38 Commits
Author SHA1 Message Date
jschoubben fbeb373d1a Both clients map local event names to their own wire
A module names its events locally and each transport works out where they land.
That is what design 29 says and what neither client did: both passed the name
straight through, which happened to be right on the old bus because modules were
writing routing keys, and wrong on the new one (novox/hq 04-ISSUES/127).

The old bus's client now turns a local name into `module.<emitter>.<event>` on the
way out and back on the way in. Without that, converting the modules to local
names would have broken the mesh that is actually running.

**A handler and a manifest now say the same thing.** The key a module sees was the
event name alone, so a manifest declaring `consumes: builder.built` produced a
pattern that could never match what it was compared against — and a module
consuming one event from two emitters could only tell them apart by reading a
header. The subject already carries the emitter, so naming it in the key makes a
mismatch between manifest and code a typo instead of a category error.

Both matchers accept `**` for the rest of a name, which is how a manifest spells
it; the old bus's `#` still works, because both buses ship until the rollout.
2026-09-27 14:42:40 +02:00
jschoubben 2197c36fef Describe the client on its own terms
Same cleanup: the comments explained each decision by contrast with what
came before instead of stating it. The certificate constraint stays — it is
a fact about the mesh's certificates, not a comparison.
2026-09-26 23:51:00 +02:00
jschoubben 19560ca6a7 Hold the runtime's NATS client to the shared fixtures
Read back from the stream rather than from the client that wrote it, so the
check is what reached the wire. The runner lives with the implementation;
the fixture stays in one place.
2026-09-26 23:40:59 +02:00
jschoubben c1517c39a0 The tool runtime's client on NATS, behind the unchanged contract
Task 3.6 of novox/hq ADR 0116. A module is still written against request,
handle, publish, subscribe, close; only what is underneath changes. main.ts
still selects the AMQP client — steps 1 to 4 leave every node on AMQP, so
this ships beside it and is selected at the rollout.

Round-tripped against a real server (test/roundtrip.mjs): a tool answered
across two connections, a throwing handler reaching the caller as an error
rather than a timeout, an event delivered once with its key, body, node and
event id intact, and an event landing under its emitter's own namespace.

Three things the compiler and the server corrected:

- the envelope's field is `key`, not `type`, and the payload is `env.body`
  with metadata in headers — not the whole envelope re-encoded. An
  implementation that nested the envelope would pass all its own tests and
  agree with nobody, which is what the conformance suite exists to stop.
- the NATS client's TLS options are PEM strings with no verify hook, so the
  AMQP client's `checkServerIdentity: () => undefined` has no equivalent.
  The fingerprint check still happens and is still the guarantee, but the
  bus's certificate must now carry a SAN matching the address nodes dial.
  That is a constraint on the mesh's certificates, recorded where it bites.
- a durable consumer is bound, never created: a module's account cannot
  reach the JetStream API, and a runtime creating its own would be a module
  choosing its own delivery semantics.
2026-09-26 23:29:17 +02:00
jschoubben 6b380b67e0 Merge pull request 'The tool runtime's recipe starts FROM the base its manifest declares (ADR 0097)' (#12) from multiple-fixes into main 2026-09-21 22:58:23 +02:00
jschoubben 2990b2d5a4 The tool runtime's recipe starts FROM the base its manifest declares (novox/hq ADR 0097) 2026-09-21 22:16:10 +02:00
jschoubben bca504b521 Merge pull request 'The patient reconnect gives up on permanent failures (058 review)' (#11) from fix/patient-connect-fatal-set into main 2026-09-20 13:41:57 +02:00
jschoubben 5b111da4a9 The patient reconnect gives up on permanent failures (issue 058 review)
The retry loop treated everything but a cert-pin mismatch as transient,
so a refused login (revoked/mis-sealed credential) or a malformed broker
URL retried for ever logging 'not reachable yet' — the silent
non-progress the fix set out to remove, and a contradiction of its own
docstring. fatalBrokerReason now classifies those three as fatal and
everything else (connection refused, timeout, DNS) as retryable, with a
unit test covering the split — the honest proof the bed cannot give,
since it only ever starts the consumer after the broker is up.

The pin case is now a typed PinMismatchError caught by instanceof, not a
prose substring a reword could silently downgrade to an infinite retry
against an impostor. Added a little jitter so modules do not stampede a
recovering broker in lockstep.
2026-09-20 13:30:52 +02:00
jschoubben 0ea3db3b20 Merge pull request 'Serve mode waits for its broker instead of crash-looping (issue 058)' (#10) from fix/the-runtime-waits-for-its-broker into main 2026-09-20 12:53:07 +02:00
jschoubben 1a55e8f205 Serve mode waits for its broker instead of crash-looping (issue 058)
At startup 'the broker is not reachable yet' is the normal case — a
container comes up in seconds, the overlay tunnel a moment later.
Exiting delegated the retry to the container runtime, which read as a
crash-loop to every restart-counting health check and every person
watching docker ps. Serve mode now retries with capped backoff, aloud,
indefinitely; a pinned-certificate mismatch still refuses at once, and
one-shot commands (emit, invoke, run) still fail fast.
2026-09-18 02:06:53 +02:00
jschoubben a99b0d3029 Merge pull request 'Resolve the SDK by version; mesh-controller/foundation rename' (#9) from feat/a-bed-that-hands-over-nothing into main 2026-09-16 23:25:45 +02:00
jschoubben 813f2e0db3 Rename mesh-control -> mesh-controller, substrate -> foundation
One name per thing, per the HQ glossary: the module/container/image/binary/repo
becomes mesh-controller, the seat the-controller, and the store+broker pair the
foundation (embedded base bundles, default template and example lock renamed with
their go:embed directives). No behaviour change — a pure vocabulary rename.

Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
2026-09-16 18:40:40 +02:00
jschoubben b00468d4c0 Resolve the SDK in a throwaway deps stage, not with a buildkit secret
A machine's docker may carry no buildx, so --mount=type=secret cannot be relied
on. Instead a deps stage copies in the builder-written .npmrc, resolves
node_modules from the mesh's registry, and the toolchain stage copies those
node_modules out without the credential — so it is in no published layer.

Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
2026-09-16 11:48:16 +02:00
jschoubben b618057fb1 Resolve the SDK by version from the registry, not from a git URL
package.json names @novox/mesh-sdk by version and the install is a
buildkit-secret-mounted resolve from the mesh's package registry, closing the
git-URL half of issue 053. The lock is regenerated against the registry (a
follow-up switches install to ci with a committed lock).

Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
2026-09-16 10:27:26 +02:00
jschoubben 6b6ec68a43 Merge pull request 'Stop compiling the toolkit by hand; it arrives compiled now' (#8) from feat/no-workaround-for-a-package-that-works into main 2026-09-14 11:09:06 +02:00
jschoubben 42b2f9cca0 Stop compiling the toolkit by hand; it arrives compiled now 2026-09-14 11:05:57 +02:00
jschoubben b6fa6575c1 Merge pull request 'Build and run are two images, from one recipe' (#7) from feat/build-and-run-are-two-images into main 2026-09-14 02:02:35 +02:00
jschoubben 62e2aa7708 Build and run are two images, from one recipe
A module's recipe starts from this image and invokes the compiler out of it, so
the compiler had to be here. The same image was also what every module ran in,
so every running container on every machine carried a compiler it would never
invoke: 23 of the 28 MB of libraries. An earlier attempt to prune the build
tools produced a smaller image that nothing could be built on, and the comment
defending their return made a workaround look like a decision.

One recipe, because the two must agree about the operating system, the language
version and the library, and two files drift.
2026-09-14 01:57:05 +02:00
jschoubben 0383c1987a Merge pull request 'runtime: a run subcommand to run a module entrypoint to completion (ADR 0052)' (#6) from feat/run-once-entry into main 2026-09-13 11:16:12 +02:00
jschoubben 77493642d8 Remove what was only there to move a commit
Two lines added to prove that changing this image makes everything standing on
it go stale. The proof worked and the lines were never meant to stay: nothing
reads MESH_RUNTIME.
2026-09-13 11:15:24 +02:00
jschoubben 8ad37c33cd Change the base for real, so the image moves with the commit 2026-09-13 02:50:15 +02:00
jschoubben 8318c78125 Move the base, to see whether the mesh notices what it reaches 2026-09-13 02:47:14 +02:00
jschoubben 90a15cde20 The runtime keeps the compiler, because it is also the build environment
Every module's recipe starts from this image and invokes the compiler out of
it. Pruning build dependencies made a smaller image that nothing could be
built on.
2026-09-13 02:45:16 +02:00
jschoubben 78257cf151 Compile the toolkit after installing it, because npm does not
It declares the hook npm is supposed to run after a git install, and this npm
does not run it — so the package arrives as sources with every entry point
pointing at a compiled directory that is not there.
2026-09-13 02:44:12 +02:00
jschoubben a5d65ada53 The build stage can fetch a git dependency, and only it can 2026-09-13 02:39:54 +02:00
jschoubben a174dfd404 The runtime every module stands on is built from its own repository
It copied in a compiled directory that is not in source control and resolved
the toolkit to a sibling checkout, so only a workstation with two repositories
side by side could produce it — and its fingerprint was then typed into every
module by hand. Nothing could rebuild it, so nothing could check it, and the
rule that catches a base moving had no version on the far end of its edge.

The recipe now also says what the image in service actually is. It claimed
Alpine and has been serving Debian for as long as nobody could rebuild it.
2026-09-13 02:39:11 +02:00
jschoubben 9af4d3fae9 Merge pull request 'mesh-tools run: a one-shot entrypoint invocation (ADR 0052 runtime side)' (#5) from feat/run-once-entry into main 2026-09-06 00:48:07 +02:00
jschoubben 991fb6faec runtime: a 'run' subcommand to run a module entrypoint to completion
The run-once step of ADR 0052 reuses a module's runtime image but must run
its bootstrap/seed entrypoint offline and exit, not the broker-bound serve
loop. Add 'mesh-tools run <entrypoint>': import the compiled entrypoint,
await its top-level work, exit — no broker, so a first-boot seed runs before
the module has anything to talk to. Exit code is the step's, which is how the
host gates the container that depends on it.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-06 00:17:13 +02:00
jschoubben 056733b519 Merge pull request 'Resync hq ADR references (0044-0054 -> 0039-0049)' (#4) from feat/adr-ref-resync into main 2026-09-05 12:45:51 +02:00
jschoubben ef540042bd Resync hq ADR references 0044-0054 -> 0039-0049 after the hq record reconciliation 2026-09-05 12:45:25 +02:00
jschoubben b936812f21 Merge pull request 'runtime: serve each tool over mesh.rpc, and an invoke subcommand (ADR 0052)' (#3) from events/tool-serving into main 2026-09-05 03:02:46 +02:00
jschoubben 4558248f16 runtime: RPC replies ride mesh.rpc; an invoke subcommand (ADR 0052)
Replies go through the RPC exchange keyed by the caller's reply-queue name, not
the default exchange — so a serving module's scoped account answers with write
on mesh.rpc alone, never the default exchange (which would let it publish into
any queue). 'mesh-tools invoke <module> <tool> [args]' is the caller's side, the
sibling of emit. Verified against a real broker: a scoped account serves its
tool and is refused another module's serve queue.
2026-09-04 21:56:04 +02:00
jschoubben bf5339cec2 runtime: take node and module identity from the sealed credential
The mesh scoped the account to a node and module; the credential now carries
both, so the runtime names its queue and stamps its events as the mesh
authorised without a manifest interpolating a node the vocabulary has no token
for. Verified: with only MESH_BROKER_FILE, the audit logger consumed as
anchor/audit-logger.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-04 01:53:23 +02:00
jschoubben 04a689e008 runtime: connect with a sealed credential, scoped, over pinned amqps (ADR 0048)
A module reads its broker credential from MESH_BROKER_FILE — the sealed
{url,fingerprint} the mesh delivered — and connects over amqps pinned to
exactly that certificate. The pin is two-phase (fetch cert, verify, then
trust only it), because Node's checkServerIdentity does not run under
rejectUnauthorized:false, so a naive connect-then-check would already have
sent the password to whoever answered.

A scoped module (assumeExchanges) never declares the exchanges (its account
may not) nor its own queue with a dead-letter (the broker refuses that to a
non-administrator) — the mesh pre-declared the queue, so it passively checks
it, binds and consumes. The RPC reply queue is lazy, and a module that
registered no tools serves none: a pure-events consumer touches only what its
account allows.

Verified end-to-end against a real broker as the scoped account: the audit
logger consumes # and records events, over an account that is not the
broker's own.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-04 01:51:17 +02:00
jschoubben 99ce1e1252 runtime: an emit primitive, so an events test can put a message on the wire
'mesh-tools emit <type> [json]' connects, emits one ADR 0047 event (awaiting
the publish confirm), and exits. The serve path already runs a module's
on('#') subscription as an import side effect, so the runtime hosts both an
emitter and the audit-logger consumer.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-04 00:56:37 +02:00
jschoubben f85d1dc303 broker: the ADR 0047 wire shape, and the event failure paths
The AMQP adapter now honours the full contract: events published
persistent with metadata in headers; a durable per-consumer queue
(<node>.<module>.events) with prefetch and a dead-letter exchange
(mesh.events.dead); manual ack for at-least-once.

Failure paths, not just the happy one:
- a confirm channel, so a publish the broker never accepted fails the
  emit rather than vanishing — at-least-once starts at the emitter;
- a handler that keeps failing is requeued once, then dead-lettered
  (poison set aside, never looping);
- an undecodable body is dead-lettered at once — it never decodes on
  redelivery, and must not wedge the queue.

Binding-conformance tests against a disposable broker (a stand-in for the
mesh-hosted broker, ADR 0001): headers on the wire with a pure body, the
redelivery-limit dead-letter, and the poison-body dead-letter.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-04 00:26:09 +02:00
jschoubben 201a413f69 broker: split events (mesh.events) from tool RPC (mesh.rpc)
Per ADR 0047: two topic exchanges kept apart, so a '#' subscription on
mesh.events is a clean audit of module/mesh/node events without the
request/reply tool traffic. Header-in-AMQP, persistent messages, per-
consumer durable dead-lettered queues and prefetch are the remaining
alignment to 0047.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-03 23:50:56 +02:00
jschoubben 9eddcdb0a0 mesh-tools — the tool runtime and AMQP broker binding
The per-node process that makes a module's tools serve on the mesh:
- broker-amqp.ts: a concrete AMQP implementation of the sdk's Broker
  contract (request/reply over a reply queue + correlation id, publish/
  subscribe over a topic exchange). Kept here, not in the sdk, so a
  broker-client change never rebuilds a module (ADR 0044).
- runtime.ts: bind the broker, import the assigned modules' tool
  entrypoints (each registers as it loads), serveTools. The thin wrapper.
- main.ts: the entrypoint, configured by MESH_BROKER_URL + MESH_TOOL_MODULES.
- Dockerfile: the container a node runs it as.

Verified over a REAL broker: the test spins LavinMQ (the mesh's broker),
the runtime serves a registered tool, a separate connection invokes it by
name over AMQP and gets the result, and an unknown tool is refused over
the wire. So 'modules can serve' is now running-on-the-mesh, not just
proven in a mock.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-03 23:21:01 +02:00