novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

The bus in five steps: a decomposition, a breakdown, and progressive insight #134

Merged
jschoubben merged 6 commits from feat/the-bus-in-five-steps into main 2026-09-26 17:05:44 +00:00
Conversation 1 Commits 6 Files Changed 9
+294 -32
4 changed files with 294 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit fe0c1e9da2 - Show all commits
02-DECISIONS/0115-the-bus-is-built-in-five-steps.md
+142
View File
@@ -0,0 +1,142 @@
---
topic: the mesh
status: accepted
date: 2026-09-26
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0106-the-bus-is-nats.md
---
# 115. The bus is built in five steps, and the protocol moves with it
## Context
[ADR 0106](0106-the-bus-is-nats.md) decided the bus is NATS and described the change as one thing:
built beside the migration, cut over in one rollout after its core. The architecture was written as
[design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) and revised once after review. What neither
says is how the work is divided, and three gaps follow from that.
**The whole of the build is one point.** Design 25 §9 numbers four items. The first — "the `nats`
module, the controller's and host's link on NATS, the runtime's client — built and proven in the
lab" — is every line of code the change requires; the other three are the rollout. A step of that
size ends at nothing provable until it ends at everything, which is the failure
[design 22](../03-DESIGN/01-to-be/22-the-work-ahead.md) opens by naming: *a phase that ends at a
claim is a phase that went missing without anything complaining.*
**There is no adoption path.** Design 25 §5 says the broker "is raised at genesis like the store,
adopted as a module in the same phase" — which describes a mesh being raised from nothing. The mesh
this is for is already running, and a running node gets a foundation module by adoption in place
([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)), not by genesis. §9 goes
straight from "built beside" to "one rollout" and never crosses that gap.
**The wire changes and the protocol specification does not know it.** Design 25 §8 says a module
sees nothing new. That is true of the SDK's contract — `request`, `handle`, `publish`, `subscribe`
— and false of the wire underneath it.
[ADR 0074](0074-the-wire-is-specified-not-the-types.md) settled that what an SDK implements is a
*specified wire*, checked by fixtures that must match byte for byte, precisely because two
implementations that disagree about an envelope do not fail to compile. That specification is
[design 19](../03-DESIGN/01-to-be/19-the-module-protocol.md), and it is written entirely in AMQP:
exchanges, a durable per-consumer queue named `<node>.<module>.events`, a shared `serve.<key>`
queue — sixteen occurrences of an AMQP term across the document. Design 25 does not cite design 19
anywhere, and design 19 does not cite ADR 0106. So the record that says *what two implementations
may not disagree about* still describes the bus being replaced.
## Considered Options
1. **Keep ADR 0106's shape — build it all, cut over once.** Rejected: not for its rollout, which
is right, but because it leaves the build a single step of unknown length with no intermediate
anyone can run. The three gaps above were found by dividing it; they were invisible while it
was one item.
2. **Cut over incrementally by traffic kind** — events to NATS first, then tools, then control,
the mesh on two buses meanwhile. Rejected: ADR 0106 already rejected two buses, and this is
that with extra steps. The store-window guarantee
([ADR 0083](0083-one-push-leaves-the-mesh-consistent.md)) is exactly the one that cannot cross
a seam, and control is exactly the traffic that carries it.
3. **Five steps, each ending at something provable, the cutover still one rollout.** The build is
divided; the bus still moves once. Adopted.
## Decision
**The bus is built in five steps. Each ends at something a lab bed proves, and no step's proof
waits for the one after it. The cutover remains a single rollout** — dividing the build does not
divide the bus.
**Step 1 — genesis raises the broker.** The `nats` module and a mesh raised on it from nothing.
This is built and proven even though the mesh it is for will never travel this path, because
genesis is where the foundation is *defined*: the only place the mesh comes from nothing, and the
definition every other path is measured against. A genesis path that exists only on paper is one
nobody discovers is wrong until there is a second mesh.
**Step 2 — adoption puts the broker in the seat.** A mesh already running receives the broker by
adoption in place, and the seat it claims is **`mesh-broker`** — unchanged.
[ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md) named the foundation seats
after the server's *role* rather than the product for exactly this case, and ADR 0106 restated it:
the broker module changes, the seat does not. A seat named after the product would have to be
renamed by every change the seat exists to survive.
**Step 3 — the protocol gets a NATS binding.** ADR 0074 stands unamended: the mesh defines a module
protocol, an SDK is an implementation of it in one language and nothing more, the protocol is split
per capability, and conformance is executable fixtures per capability rather than prose. What
changes is what the specification specifies. Design 19's wire section is rewritten from exchanges
and queues to subjects and streams; the fixtures are recaptured on NATS; every SDK claims the
capabilities it passes, and a language may arrive with connection and events alone.
**The SDK gains no conveniences in the process.** [ADR 0039](0039-what-the-sdk-holds-and-refuses.md)
refuses frequent-and-cascading code in the shared library, and ADR 0074 restates it — an SDK is
"not a convenience layer, not a place for helpers to accumulate." A new transport is the moment
that pressure is highest and the reason to hold hardest: the predecessor's shared library is the
cautionary tale ADR 0039 opens with, and it did not become that in one decision. Code shared among
a module's own features stays in that module.
**Step 4 — the core speaks NATS.** The controller's link, the host's link and the tool runtime's
client, and with them the flows that are today carried by something other than the bus: a build
source's change reaching the builder, an installation, a module's own reports. Each is a
conversion with a named before and after, not a rewrite. Observation — heartbeats, conditions,
key-value state — is [research 017](../01-RESEARCH/017-a-mesh-that-heals-itself/00-overview.md)'s
and stays there; that effort already reserves it for after the move, and this step does not
pre-empt its design.
**Step 5 — deployment.** The rollout ADR 0106 decided, unchanged: the controller, every host and
every runtime move together, each node confirmed to have heard before AMQP stops. What this step
adds is that steps 1 to 4 *ship ahead of it* without moving any node's bus — the module exists in
the catalogue, the protocol is specified, the code is written and beds pass, and the running mesh
is still on AMQP throughout. The bus moves on one day, at the end, once.
## Consequences
- **Design 25 §9 is replaced by these five steps** and §10's beds are attributed to the step each
proves, so no proof waits for the last step.
- **Design 19 is stale in its wire section from today** and says so in its own frontmatter and
opening until step 3 rewrites it. A specification that describes the bus being replaced is worse
than an absent one, because it reads as current.
- **`nats-broker` is not a seat.** The module is `nats`; the seat is `mesh-broker`.
- **Steps 1 to 4 leave every node on AMQP.** A step can be abandoned, or reordered after step 2,
without a rollback — the cost of being wrong is bounded until step 5.
- **What got harder:** five steps mean five proofs rather than one, and step 3 rewrites a design
other designs cite, so their references are checked when it lands. Dividing the work does not
reduce it.
## How it is checked
- **Per step, a bed, and the bed named in design 25 §10 against the step it belongs to.** Step 1:
a mesh raised on NATS from genesis. Step 2: the broker adopted into a mesh already running, and
a second holder of `mesh-broker` refused at resolution. Step 3: the conformance suite passing per
capability, on NATS, for every SDK that claims it — and a module built before the binding serving
its tools unchanged. Step 4: each converted flow proved against the behaviour it replaced. Step 5:
the cutover bed, then the rollout with every node reporting.
- **Step 3 is not done when the code runs.** It is done when the fixtures match byte for byte
across implementations, which is ADR 0074's own test and the only one that catches two SDKs
quietly ignoring each other.
- **A step that cannot name what its bed proves is not a step**, and is divided further before it
is started.
## References
- [ADR 0106](0106-the-bus-is-nats.md) — the decision this divides.
- [ADR 0074](0074-the-wire-is-specified-not-the-types.md) — the protocol and its conformance suite;
step 3 is its NATS binding, not a replacement.
- [ADR 0039](0039-what-the-sdk-holds-and-refuses.md) — why step 3 adds no helpers.
- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md) — why the seat is
`mesh-broker`.
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) — the adoption step 2 uses.
- [design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md), [design 19](../03-DESIGN/01-to-be/19-the-module-protocol.md) — the two documents this changes.
02-DECISIONS/README.md
+1
View File
@@ -95,6 +95,7 @@ python3 00-META/checks/index.py fail if stale
- **0104** — [A provision may be answered by an adapter to the predecessor](0104-a-provision-may-be-answered-by-an-adapter-to-the-predecessor.md)
- **0105** — [The mesh adopts the predecessor's tunnel in place](0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md)
- **0106** — [The bus is NATS](0106-the-bus-is-nats.md)
- **0115** — [The bus is built in five steps, and the protocol moves with it](0115-the-bus-is-built-in-five-steps.md)
### Its tiers, from the bottom up
03-DESIGN/01-to-be/19-the-module-protocol.md
+24 -1
View File
@@ -5,9 +5,11 @@ code:
- mesh-sdk src
- mesh-tools src/broker-amqp.ts
- mesh-controller internal/link
updated: 2026-09-21
updated: 2026-09-26
decisions:
- 02-DECISIONS/0095-the-control-plane-is-the-way-to-ask-a-module.md
- 02-DECISIONS/0106-the-bus-is-nats.md
- 02-DECISIONS/0115-the-bus-is-built-in-five-steps.md
- 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
- 02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md
@@ -23,6 +25,27 @@ language and nothing more ([ADR 0074](../../02-DECISIONS/0074-the-wire-is-specif
This is a specification, so it says what is required rather than how anything is arranged. Where it
describes current behaviour that is *not yet* specified-and-conformed, it says so.
> **The wire below is the bus being replaced.** *2026-09-26.*
> [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md) moved the mesh's bus to NATS. Everything
> in this document that names an exchange, a queue or a routing key — the event exchanges, the
> durable `<node>.<module>.events` queue, the shared `serve.<key>` queue — describes the transport
> being retired, and the conformance fixtures were captured against it.
>
> What does **not** change is this document's model, which is the part ADR 0074 decided: a floor
> plus independent capabilities, an SDK that implements what it claims and is legitimate when it
> claims less, identity taken from the sealed credential rather than the environment, at-least-once
> with dedup on `x-event-id`, and conformance as executable fixtures per capability rather than
> prose. The envelope keeps its shape ([ADR 0042](../../02-DECISIONS/0042-the-shape-of-an-event-on-the-wire.md));
> it becomes the message body.
>
> Rewriting the wire sections onto the subjects and streams of
> [design 25](25-the-bus-on-nats.md) §2–§3, and recapturing the fixtures there, is **step 3 of
> [ADR 0115](../../02-DECISIONS/0115-the-bus-is-built-in-five-steps.md)**. Until that lands, read
> the sections below for what two implementations may not disagree *about*, and design 25 for what
> they will disagree about it *on*. A specification that silently described a retired transport
> would be worse than an absent one, because it reads as current — hence this note rather than a
> quiet edit.
## The shape of it
A **floor** every implementation needs, and three **capabilities** that are independent of each
03-DESIGN/01-to-be/25-the-bus-on-nats.md
+127 -31
View File
@@ -6,9 +6,14 @@ code:
- mesh-host internal/link (to be replaced)
- mesh-tools src/broker-amqp.ts (to be replaced)
- mesh-catalog modules/nats (to be written)
updated: 2026-09-24
- mesh-sdk src (the protocol's NATS binding, step 3)
updated: 2026-09-26
decisions:
- 02-DECISIONS/0106-the-bus-is-nats.md
- 02-DECISIONS/0115-the-bus-is-built-in-five-steps.md
- 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
- 02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
- 02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md
- 02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
@@ -221,39 +226,94 @@ bridged. It is three things:
Nothing is built of this before §10's bed passes; the MCP surface is a thin adapter over (2).
## 8. What a module sees
## 8. What a module sees, and what the wire does
Nothing new. `publish` on an envelope becomes a publish on `mesh.events.<module>.<event>`;
`subscribe` with a pattern becomes a durable JetStream consumer on the matching subject filter;
`request`/`handle` become a NATS request and a queue-group subscription on
`mesh.tools.<module>.<tool>`. The envelope's shape ([ADR 0042](../../02-DECISIONS/0042-the-shape-of-an-event-on-the-wire.md))
is unchanged; it is the message body. A module built today runs on the new runtime without a
rebuild — that is the test of ADR 0039, and it is in §10.
**The contract a module is written against does not change.** `publish` on an envelope becomes a
publish on `mesh.events.<module>.<event>`; `subscribe` with a pattern becomes a durable JetStream
consumer on the matching subject filter; `request`/`handle` become a NATS request and a queue-group
subscription on `mesh.tools.<module>.<tool>`. The envelope's shape
([ADR 0042](../../02-DECISIONS/0042-the-shape-of-an-event-on-the-wire.md)) is unchanged; it is the
message body. A module built today runs on the new runtime without a rebuild — that is the test of
ADR 0039, and it is in §10.
## 9. Moving from the bus the mesh has
**The wire underneath it changes completely, and that is a specification, not an implementation
detail.** Revision, second review ([ADR 0115](../../02-DECISIONS/0115-the-bus-is-built-in-five-steps.md)):
an earlier draft of this section said "nothing new" and stopped there, which read as though the
change were contained inside the runtime. It is not.
[ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) settled that an SDK is an
implementation of a *specified wire*, checked by fixtures that must match byte for byte — because
two implementations that disagree about an envelope do not fail to compile, they ignore each other
while both keep running. That specification is
[design 19](19-the-module-protocol.md), and it is written in exchanges, a durable per-consumer queue
named `<node>.<module>.events`, and a shared `serve.<key>` queue. Every one of those is gone here.
Per ADR 0106: built beside, cut over once, after the core.
So design 19 is rewritten from exchanges and queues to the subjects and streams of §2 and §3, its
fixtures are recaptured on NATS, and each SDK re-claims the capabilities it passes. That is step 3
of §9, and until it lands design 19 says in its own opening that its wire section describes the bus
being replaced. What is *not* rewritten: ADR 0074's model — protocol split per capability, an SDK
that implements the floor and events alone is legitimate, conformance executable per capability —
and ADR 0039's refusal. A new transport is when the pressure to grow the shared library is highest;
nothing is added to it here.
1. The `nats` module, the controller's and host's link on NATS, the runtime's client — built and
proven in the lab (§10) while the migration continues on AMQP. Modules converted meanwhile
target the sdk contract and are untouched by this.
2. The cutover is one rollout, previewed: the controller assigns `nats` to the hub (raised beside
the AMQP broker on its own ports), composes every node's and module's account into it, then
rolls out the controller, every host and every runtime built for NATS. Each node's host connects
to the new bus as it comes up and reports; the controller confirms every node heard before it
stops listening on AMQP. The predecessor's clients never notice: their broker is the
compatibility module and stays.
3. The AMQP-side mesh accounts are removed from the compatibility broker; it keeps only the
predecessor's users. The bus's port settings follow ADR 0100 like any port.
4. The compatibility broker retires when its retirement condition holds.
## 9. Moving from the bus the mesh has: five steps
What is not done: no dual-bus period for the mesh's own traffic, no bridge, no module rebuilt.
Per ADR 0106 the bus moves once. Per
[ADR 0115](../../02-DECISIONS/0115-the-bus-is-built-in-five-steps.md) the *build* is five steps,
each ending at something §10 proves, so that no part of this waits on the whole of it. Dividing the
build does not divide the bus: steps 1 to 4 leave every node on AMQP, and step 5 is still one
rollout.
**Step 1 — genesis raises the broker.** The `nats` module of §5, and a mesh raised on it from
nothing. This is built and proven although the mesh it is for will never travel this path: genesis
is where the foundation is *defined*, the one place the mesh comes from nothing, and the definition
every other path is measured against. A genesis path that exists only on paper is one nobody finds
wrong until there is a second mesh. *Ends at: the genesis bed.*
**Step 2 — adoption puts the broker in its seat.** A mesh already running does not get a foundation
module by being raised again; it adopts one in place
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)). The
server is raised beside the AMQP broker on its own ports, carrying no mesh traffic yet, and the
`nats` module is adopted onto it. The seat it claims is **`mesh-broker`**, unchanged — the
foundation seats are named after the server's role rather than the product
([ADR 0079](../../02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md)) for
exactly this case, and a seat named after the product would need renaming by every change the seat
exists to survive. *Ends at: the adoption bed.*
**Step 3 — the protocol gets its NATS binding.** §8's other half: design 19 rewritten from
exchanges and queues to subjects and streams, its conformance fixtures recaptured on NATS, each SDK
re-claiming the capabilities it passes. ADR 0074's model is unamended and ADR 0039's refusal holds —
no helper layer arrives with the new transport. A language may still arrive in pieces: connection
and events first, tools and provisioning when something needs them. *Ends at: the conformance suite,
per capability, per implementation.*
**Step 4 — the core speaks NATS.** The controller's link, the host's link, the tool runtime's
client — and with them the flows carried today by something other than the bus: a build source's
change reaching the builder, an installation, a module's own reports. Each is a conversion with a
named before and after. Observation — heartbeats, conditions, key-value state — belongs to
[research 017](../../01-RESEARCH/017-a-mesh-that-heals-itself/00-overview.md), which already
reserves it for after the move; this step does not pre-empt its design. Modules converted meanwhile
target the sdk contract and are untouched by any of it. *Ends at: each converted flow proved against
the behaviour it replaced.*
**Step 5 — the rollout.** Unchanged from ADR 0106 and previewed: the controller composes every
node's and module's account into the server standing since step 2, then rolls out the controller,
every host and every runtime built for NATS. Each node's host connects to the new bus as it comes up
and reports; the controller confirms every node heard before it stops listening on AMQP. The
predecessor's clients never notice — their broker is the compatibility module and stays. Then the
AMQP-side mesh accounts are removed from it, leaving only the predecessor's users, and it retires
when its retirement condition holds. The bus's port settings follow ADR 0100 like any port.
*Ends at: the cutover bed, then the rollout itself.*
What is not done, at any step: no dual-bus period for the mesh's own traffic, no bridge, no module
rebuilt.
## 10. How it is checked
Two lab beds, both required green before any node's bus moves.
**A bed per step, and each is green before the step after it starts** — the division in §9 is only
real if the proofs divide with it. A step that cannot name what its bed proves is not a step, and is
divided further before it is started.
**The bus bed** — a mesh raised on NATS from genesis:
**Step 1 — the genesis bed**, a mesh raised on NATS from nothing:
- a node enrols over TLS with a claimed token, and the enrolment user cannot read a declaration;
- a push composes; the store is stopped; the push is held (nak with delay), the store returns, the
push applies, nothing was lost or duplicated;
@@ -273,12 +333,40 @@ Two lab beds, both required green before any node's bus moves.
field a consumer's ack has already claimed;
- the `nats` container is not recreated when only its composed configuration file changes, and
a change to that file is live (a new user can connect, a revoked one cannot) within one
watcher-poll interval, without a restart;
- a module built before this design serves its tools unchanged on the new runtime.
watcher-poll interval, without a restart.
**The cutover bed** — a mesh on AMQP with a predecessor stand-in on the compatibility broker moves
its bus in one rollout; every node reports on NATS afterwards; the stand-in's client on AMQP is
still connected throughout.
**Step 2 — the adoption bed**, a mesh already running that has never had this server:
- the server is raised beside the AMQP broker on its own ports and the `nats` module is adopted
onto it in place, holding the data and the configuration it was raised with;
- the seat it claims is `mesh-broker`, and a second assignment of it anywhere in the mesh is
refused at resolution — *one per mesh*, as ADR 0079 requires;
- every node stays on AMQP throughout and nothing routes to the adopted server. This is the
check that makes steps 3 and 4 safe to run against a live mesh: adoption that quietly carried
traffic would be step 5 arriving early and unrehearsed.
**Step 3 — conformance, per capability, per implementation:**
- the fixtures of design 19, recaptured on NATS, produced and consumed byte for byte by every SDK
that claims the capability — the test ADR 0074 set, and the only one that catches two
implementations quietly ignoring each other;
- an SDK implementing connection and events alone passes those two and claims nothing more,
rather than failing as a whole;
- a module built before this design serves its tools unchanged on the new runtime — the test of
ADR 0039;
- the shared library gained nothing but the binding: its surface is the protocol and the
primitives, and a helper that arrived with the transport is a review failure, not a detail.
**Step 4 — each converted flow against the behaviour it replaced:**
- a build source's change reaches the builder over the bus, and the build that follows is the one
the change asked for;
- an installation completes over the bus, with the same outcome the path it replaces produced;
- a module's reports arrive, and a node that was unreachable catches up rather than losing them;
- nothing of research 017's observation work is present — no heartbeat semantics, no condition
store — because that design is not written yet and a flow built ahead of it would have to be
rebuilt.
**Step 5 — the cutover bed:** a mesh on AMQP with a predecessor stand-in on the compatibility
broker moves its bus in one rollout; every node reports on NATS afterwards; the stand-in's client
on AMQP is still connected throughout.
Unit tests hold the controller to composing accounts from `emits`/`consumes` and nothing else, to
creating the four streams and asserting them idempotently, and to spending a token exactly once;
@@ -287,7 +375,15 @@ to mapping the sdk contract onto subjects exactly as §8 says.
## 11. Open, for the review
**Closed by this revision** (first review, recorded in `MIGRATION-LOG.md`, 2026-09-24): the
**Closed by the second revision** (2026-09-26, [ADR 0115](../../02-DECISIONS/0115-the-bus-is-built-in-five-steps.md)):
the build was one undivided item (§9 is now five steps, each with its own bed in §10); there was no
adoption path for a mesh already running (§9 step 2); and §8 said a module sees "nothing new"
without distinguishing the sdk's contract, which does not change, from the specified wire, which
changes entirely and is design 19's (§8, §9 step 3). Two smaller corrections: the seat is
`mesh-broker` and not the product's name, and the shared library gains no conveniences with the new
transport.
**Closed by the first revision** (recorded in `MIGRATION-LOG.md`, 2026-09-24): the
`reload-on`/container mismatch (§5), the eaten reply subject on a CONTROL-stream message (§2, §6),
the missing ack permission (§4), and the un-scoped reply inbox under one account (§4). Each is
named where it was wrong, not silently fixed, so a reader comparing against the first version can