jschoubben
814c9e563f
A module does declare requirements the provisioner fulfils — but the broker it gets that way is a private vhost, the analog of a database, not the mesh's bus. Two modules of the new mesh depend on it, so the compatibility broker was never single-purpose and its retirement would have stranded them. NATS is the heart: one bus, a module's messaging is subjects on it scoped by what it declares, and no module is handed a server of its own. The seat delivers nothing; the interface retires with the broker. Also closes the EVENTS question — one stream, on the bootstrap argument, not preference. Designs 25 and 28 go in-progress: step 1 is starting. The insight check caught a false positive on its own first real use — its bold-run pattern crossed newlines and joined an unrelated `**` to the marker. Constrained to one line, still catching all four bad shapes.
5.4 KiB
topic, status, date, deciders, reconstructed, extends
| topic | status | date | deciders | reconstructed | extends |
|---|---|---|---|---|---|
| the mesh | accepted | 2026-09-23 | jochen | false | 02-DECISIONS/0002-nodes-communicate-over-a-broker.md |
106. The bus is NATS
Context
The mesh's bus is an AMQP broker. Research 014 measured what that costs and what it would take to change: AMQP is spoken in three places of the mesh's own code — the controller's link, the host's link, the tool runtime's client — and in none of the sdk or the modules, because ADR 0039 kept the client out of the sdk for exactly this. The predecessor's world is AMQP and is being retired module by module; it cannot move and does not need to.
The operator's direction: NATS, native — the mesh built on it, not bridged to it — and the tools a person reaches from a workstation designed on it rather than built quickly on what exists.
Considered Options
- Stay on AMQP. Rejected by the operator.
- NATS behind a bridge, the mesh unchanged. Rejected: two buses, every guarantee crossing a seam, and the reason for changing — one native, simple, subject-addressed bus with request/reply and accounts built in — lost at the seam.
- NATS native, built in the lab in parallel with the migration, cut over in one rehearsed rollout after the migration's core is done. Adopted.
Decision
The mesh's bus is NATS. Every link the mesh has — control, node queues, builds, enrolment,
reports, events, tool invocation — is carried on NATS subjects; durability, catch-up and
hold-unacknowledged-and-retry (ADR 0083) are
JetStream's; a module's account is a NATS account with per-subject permissions derived from its
emits and consumes (ADR 0043),
written as configuration the host declares and the server reloads — never through a management
API. The broker module changes; the seat mesh-broker does not
(ADR 0079).
The sdk's broker contract does not change. Modules are written against request, handle,
publish, subscribe; the runtime implements them on NATS; no module speaks a protocol.
The AMQP broker the mesh adopted becomes the predecessor's compatibility broker, kept as a module with one purpose — the predecessor's clients — and retired with the last of them.
It is built beside the migration and cut over after its core. The bus is not changed under a half-migrated node. A lab bed proves the whole path — enrolment, the store window, upgrades, tool invocation — on NATS before any node's bus moves, and the move is one rollout: controller, every host, every runtime together.
What a person reaches the mesh's tools with is designed on NATS, as part of the same design: a person's account, scoped like a module's, and a client that speaks the bus directly.
Consequences
- The architecture — subjects, streams, accounts, the enrolment handshake, the person's client — is written as a to-be design before code, and reviewed.
- Eight records are touched: 0002 and 0033 (the substrate names a broker — now NATS), 0041 and
0042 (events keep their shape; the exchange becomes a subject prefix), 0043 (accounts as
configuration), 0078 (the broker module is
nats), 0083 (JetStream carries the guarantee), 0039 (unchanged, and the reason this is possible). - The lab beds that prove the bus are re-run on NATS; none is skipped.
- Until the cutover, nothing changes on any node.
How it is checked
A lab bed raises a mesh on NATS from genesis and proves: a node enrols over TLS with a claimed
token; a push composes, is held while the store restarts and applies after; an upgrade rolls out;
a module's tools are invoked from another node and from a person's client; an event dead-letters
after its deliveries are exhausted; a module's account cannot publish outside its emits nor
subscribe outside its consumes. Then the cutover bed: a mesh on AMQP with the predecessor's
compatibility broker beside it moves its bus in one rollout with every node reporting afterwards.
Progressive insight
Progressive insight — 2026-09-26. The compatibility broker was not single-purpose when this was written. This record says the adopted AMQP broker is "kept as a module with one purpose — the predecessor's clients". Two modules of the new mesh also depended on it, through a
requires: ["amqp"]grant its provisioner answered with a private vhost —amqp-pingandamqp-email-forwarder. On the retirement condition below, both would have been left requiring something no provider answers. ADR 0117 resolves it by moving them onto the bus and retiring the interface, which makes this record's sentence true rather than merely intended. The decision — the bus is NATS, the AMQP broker becomes the predecessor's compatibility broker and retires with the last of them — is unchanged.