jschoubben
85f972749a
0117 went a step further than it had grounds for. It was right that the bus is the only bus, and wrong that the amqp interface must therefore retire — because it conflated two reasons to want a broker. Using one to reach another module is a second bus and stays refused. Needing an AMQP broker as a backing service, the way something needs a database, is ordinary, and forbidding it would make the mesh unable to run normal software while calling that architecture. So the broker becomes a plain provider module: no seat, not foundation, never raised at genesis, no retirement condition. lavinmq now claims nothing and provides amqp; nats claims mesh-broker and provides nothing. The rule that survives is about direction, not software: inter-module communication goes over the bus. A module may hold a broker for itself; it may not use one as a channel to another module. That is a review judgement where 0117 could have used a parser, which is the honest cost. 0106's progressive insight was itself wrong and is corrected by a second one there — nothing moves off the old broker, so its "one purpose" sentence does not become true, it is just not what that server is. The insight check needed two fixes it found itself: a date may carry trailing words, and a bold run with a link is discussing an insight rather than marking one. All four bad shapes still fire.
108 lines
6.3 KiB
Markdown
108 lines
6.3 KiB
Markdown
---
|
|
topic: the mesh
|
|
status: accepted
|
|
date: 2026-09-23
|
|
deciders: jochen
|
|
reconstructed: false
|
|
extends: 02-DECISIONS/0002-nodes-communicate-over-a-broker.md
|
|
---
|
|
|
|
# 106. The bus is NATS
|
|
|
|
## Context
|
|
|
|
The mesh's bus is an AMQP broker. [Research 014](../01-RESEARCH/014-the-bus-on-nats/00-overview.md)
|
|
measured what that costs and what it would take to change: AMQP is spoken in three places of the
|
|
mesh's own code — the controller's link, the host's link, the tool runtime's client — and in none of
|
|
the sdk or the modules, because [ADR 0039](0039-what-the-sdk-holds-and-refuses.md) kept the client
|
|
out of the sdk for exactly this. The predecessor's world is AMQP and is being retired module by
|
|
module; it cannot move and does not need to.
|
|
|
|
The operator's direction: NATS, native — the mesh built on it, not bridged to it — and the tools a
|
|
person reaches from a workstation designed on it rather than built quickly on what exists.
|
|
|
|
## Considered Options
|
|
|
|
1. **Stay on AMQP.** Rejected by the operator.
|
|
2. **NATS behind a bridge**, the mesh unchanged. Rejected: two buses, every guarantee crossing a
|
|
seam, and the reason for changing — one native, simple, subject-addressed bus with request/reply
|
|
and accounts built in — lost at the seam.
|
|
3. **NATS native, built in the lab in parallel with the migration, cut over in one rehearsed
|
|
rollout after the migration's core is done.** Adopted.
|
|
|
|
## Decision
|
|
|
|
**The mesh's bus is NATS.** Every link the mesh has — control, node queues, builds, enrolment,
|
|
reports, events, tool invocation — is carried on NATS subjects; durability, catch-up and
|
|
hold-unacknowledged-and-retry ([ADR 0083](0083-one-push-leaves-the-mesh-consistent.md)) are
|
|
JetStream's; a module's account is a NATS account with per-subject permissions derived from its
|
|
`emits` and `consumes` ([ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md)),
|
|
written as configuration the host declares and the server reloads — never through a management
|
|
API. The broker module changes; the seat `mesh-broker` does not
|
|
([ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md)).
|
|
|
|
**The sdk's broker contract does not change.** Modules are written against `request`, `handle`,
|
|
`publish`, `subscribe`; the runtime implements them on NATS; no module speaks a protocol.
|
|
|
|
**The AMQP broker the mesh adopted becomes the predecessor's compatibility broker**, kept as a
|
|
module with one purpose — the predecessor's clients — and retired with the last of them.
|
|
|
|
**It is built beside the migration and cut over after its core.** The bus is not changed under a
|
|
half-migrated node. A lab bed proves the whole path — enrolment, the store window, upgrades, tool
|
|
invocation — on NATS before any node's bus moves, and the move is one rollout: controller, every
|
|
host, every runtime together.
|
|
|
|
**What a person reaches the mesh's tools with is designed on NATS**, as part of the same design: a
|
|
person's account, scoped like a module's, and a client that speaks the bus directly.
|
|
|
|
## Consequences
|
|
|
|
- The architecture — subjects, streams, accounts, the enrolment handshake, the person's client — is
|
|
written as a to-be design before code, and reviewed.
|
|
- Eight records are touched: 0002 and 0033 (the substrate names a broker — now NATS), 0041 and
|
|
0042 (events keep their shape; the exchange becomes a subject prefix), 0043 (accounts as
|
|
configuration), 0078 (the broker module is `nats`), 0083 (JetStream carries the guarantee), 0039
|
|
(unchanged, and the reason this is possible).
|
|
- The lab beds that prove the bus are re-run on NATS; none is skipped.
|
|
- Until the cutover, nothing changes on any node.
|
|
|
|
## How it is checked
|
|
|
|
A lab bed raises a mesh on NATS from genesis and proves: a node enrols over TLS with a claimed
|
|
token; a push composes, is held while the store restarts and applies after; an upgrade rolls out;
|
|
a module's tools are invoked from another node and from a person's client; an event dead-letters
|
|
after its deliveries are exhausted; a module's account cannot publish outside its `emits` nor
|
|
subscribe outside its `consumes`. Then the cutover bed: a mesh on AMQP with the predecessor's
|
|
compatibility broker beside it moves its bus in one rollout with every node reporting afterwards.
|
|
|
|
## Progressive insight
|
|
|
|
> **Progressive insight — 2026-09-26.** *The compatibility broker was not single-purpose when this
|
|
> was written.* This record says the adopted AMQP broker is "kept as a module with one purpose —
|
|
> the predecessor's clients". Two modules of the new mesh also depended on it, through a `requires:
|
|
> ["amqp"]` grant its provisioner answered with a private vhost — `amqp-ping` and
|
|
> `amqp-email-forwarder`. On the retirement condition below, both would have been left requiring
|
|
> something no provider answers.
|
|
> [ADR 0117](0117-the-bus-is-the-only-broker.md) resolves it by moving them onto the bus and
|
|
> retiring the interface, which makes this record's sentence true rather than merely intended. The
|
|
> decision — the bus is NATS, the AMQP broker becomes the predecessor's compatibility broker and
|
|
> retires with the last of them — is unchanged.
|
|
|
|
> **Progressive insight — 2026-09-26, correcting the one above.** *The broker is not a
|
|
> compatibility module at all, and the sentence does not become true.* The insight above said
|
|
> [ADR 0117](0117-the-bus-is-the-only-broker.md) would make "one purpose — the predecessor's
|
|
> clients" true by moving the mesh's own modules off it.
|
|
> [ADR 0119](0119-amqp-is-a-provision-not-the-bus.md) supersedes that: nothing moves off, because
|
|
> a module may legitimately need an AMQP broker as a backing service the way it needs a database.
|
|
> The broker becomes **an ordinary provider module** — no seat, not foundation, not raised at
|
|
> genesis, and with no retirement condition, because the day its last client disappears is not a
|
|
> day anything is waiting for. What this record decided — the mesh's bus is NATS — is untouched
|
|
> by both; what was wrong was the sentence describing what happens to the old server, twice.
|
|
|
|
## References
|
|
|
|
- [research 014](../01-RESEARCH/014-the-bus-on-nats/00-overview.md)
|
|
- [ADR 0002](0002-nodes-communicate-over-a-broker.md), [0033](0033-the-substrate-is-a-store-and-a-broker.md),
|
|
[0039](0039-what-the-sdk-holds-and-refuses.md), [0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md),
|
|
[0078](0078-the-store-and-broker-are-modules.md), [0083](0083-one-push-leaves-the-mesh-consistent.md)
|