PR #133 landed a different 0115 while this branch was open. The bus record is now 0116, with every citation in designs 19, 25, 28 and the index following it. Note: cycle.py and records.py both fail on main as merged, on that record — nothing cites it, and it rests on 0112, which is still proposed. Both pre-date this branch and are left for their own change.
257 lines
16 KiB
Markdown
257 lines
16 KiB
Markdown
---
|
|
layer: to-be
|
|
status: proposed
|
|
code: []
|
|
updated: 2026-09-26
|
|
decisions:
|
|
- 02-DECISIONS/0116-the-bus-is-built-in-five-steps.md
|
|
- 02-DECISIONS/0106-the-bus-is-nats.md
|
|
- 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
|
|
- 02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md
|
|
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
|
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
|
|
---
|
|
|
|
# 28. Building the bus
|
|
|
|
**The work of [ADR 0116](../../02-DECISIONS/0116-the-bus-is-built-in-five-steps.md)'s five steps,
|
|
in the order its dependencies allow, with what each ends at.**
|
|
[Design 25](25-the-bus-on-nats.md) is the architecture and stays the authority on *what is built*;
|
|
this document holds only the order, the sizes and the proofs, and it is wrong the moment it
|
|
disagrees with design 25 rather than the other way round.
|
|
|
|
Each step ends at something runnable. A step that cannot name what its bed proves is not a step,
|
|
and is divided further before it is started.
|
|
|
|
## How this is built, and when it is run
|
|
|
|
**Written as code with unit tests, committed per change, and taken to the lab once the pieces that
|
|
would change the outcome are in place.** The mistake this avoids is the one
|
|
[design 22](22-the-work-ahead.md) records: running a long bed against a mesh mid-transformation and
|
|
debugging paths the next step deletes. Where a fault can be reasoned out of the code path, it is —
|
|
reading, not running.
|
|
|
|
So the beds below are acceptance tests at the end of assembled work, not the tool for finding each
|
|
bug, and a step's bed is run when that step is finished rather than while it is being written.
|
|
|
|
## What the work is, measured
|
|
|
|
Counted 2026-09-26, non-test source only. The point of counting is that none of this is unknown
|
|
territory: every piece has a shape already standing beside it.
|
|
|
|
| Piece | Today | Size | Becomes |
|
|
|---|---|---|---|
|
|
| the controller's link | Go, one package | ~1 800 lines | the same package on NATS |
|
|
| the host's link | Go, one package, mirroring the contracts rather than importing them | ~1 000 lines | the same, on NATS |
|
|
| the tool runtime's client | TypeScript, one file | ~390 lines | the same, on NATS |
|
|
| the sdk's messaging surface | TypeScript: messaging, events, tools, contracts, primitives | ~360 lines across five | **unchanged**, see below |
|
|
| the broker module | the adopted AMQP broker: client, tools, provisioner, bootstrap, image, manifest | ~340 lines of module code | the `nats` module, same shape |
|
|
| the beds | 39 lab scenarios, including a broker bed, an adoption bed, a genesis bed and a store-window bed | — | four analogues and one new |
|
|
|
|
**Two measurements are worth stating on their own, because they change what the steps are.**
|
|
|
|
**The sdk speaks no AMQP, and never did.** The word appears in its source three times, in three
|
|
comments; its messaging module says in as many words that it "carries the contract, not a specific
|
|
AMQP client build." [ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md) put the
|
|
client in the runtime, and the payoff is collected here: **no module is rebuilt for this change, and
|
|
the sdk's own diff is three comments.** That is the whole reason a bus can be replaced under a live
|
|
mesh at all.
|
|
|
|
**The wire therefore has three implementations, not two, and no suite pins any of them.**
|
|
[ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) spoke of "the existing
|
|
two implementations" — Go and TypeScript. Measured, the Go side is *two separate packages* that
|
|
mirror rather than share (the host imports nothing, by
|
|
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md)), so the count is the controller's link, the
|
|
host's link, and the runtime's client. And a search for conformance fixtures finds none anywhere in
|
|
the four repositories: design 22's Phase 1.2 — the suite — has not been built.
|
|
|
|
> **This corrected the record.** ADR 0116 said step 3's fixtures were *recaptured* on NATS. There
|
|
> is nothing to recapture, so step 3 **builds** the suite, and its first job is to pin the wire the
|
|
> mesh has before changing it — a suite written only against the new bus certifies whatever the new
|
|
> bus happens to do. A fact went stale while the decision stood, which is a **progressive insight**
|
|
> ([`02-DECISIONS/README.md`](../../02-DECISIONS/README.md)): it is marked and dated in ADR 0116
|
|
> itself rather than left to be discovered here.
|
|
|
|
## The order the work actually allows
|
|
|
|
**The five steps are chunks of capability; the build order is not simply 1 to 5, and pretending
|
|
otherwise would put two beds where they cannot run.** Three edges decide it:
|
|
|
|
- **A specification precedes the implementations it governs.** ADR 0074's whole argument is that
|
|
agreement is specified and checked, not hoped for. So the wire's NATS binding is written *before*
|
|
the three implementations are, even though it is step 3 — and its conformance half can only
|
|
*finish* once two implementations exist to disagree.
|
|
- **A mesh cannot be raised on a bus nothing speaks.** A bed that raises a mesh on NATS from
|
|
genesis — enrolling a node, holding a push while the store restarts, rolling out an upgrade —
|
|
needs the controller and the host to speak NATS already. That is the implementations, and they
|
|
arrive with step 3.
|
|
- **Adoption needs the module and nothing else.** Step 2 puts a correctly configured server into a
|
|
running mesh that continues to ignore it, which depends on no link at all.
|
|
|
|
So step 1's bed proves *the server, from genesis, configured* — not a mesh living on it. The full
|
|
genesis bed is step 4's, where it can first run.
|
|
|
|
> **This corrected the record too.** ADR 0116 first attributed "a mesh raised on NATS from genesis"
|
|
> to step 1. That bed cannot run until the links exist, and a step whose proof cannot run is the
|
|
> exact failure the record was written to prevent — so step 1 now ends at the server standing,
|
|
> correctly configured and carrying nothing, and the full bed is named under step 4. The five
|
|
> steps, their names, their order and the single rollout are unchanged; only where two beds run has
|
|
> moved. Marked and dated in ADR 0116 as a progressive insight, with what the record said before.
|
|
|
|
```
|
|
step 1 module, genesis places it ──┐
|
|
step 2 adoption into a running mesh ──┤ neither needs a link
|
|
│
|
|
step 3 the wire specified ──► three implementations ──► the suite
|
|
│
|
|
step 4 the flows, and the full genesis bed
|
|
│
|
|
step 5 the rollout
|
|
```
|
|
|
|
## Step 1 — the module, and genesis raises it
|
|
|
|
**Why here.** Everything else needs a server to talk to, and genesis is where the foundation is
|
|
defined. The mesh this is for will never travel this path — it is already running, and takes step 2
|
|
— but genesis is the definition every other path is measured against, and one that exists only on
|
|
paper is wrong until there is a second mesh to find out.
|
|
|
|
- [ ] 1.1 the `nats` module: manifest, image, one container, its client, TLS and monitoring ports,
|
|
JetStream on a named volume — the shape of design 25 §5, and the same shape the broker module
|
|
beside it already has
|
|
- [ ] 1.2 the composed configuration as a **directory** resource, and the entrypoint that watches
|
|
the one file and signals the server itself — design 25 §5's correction, kept inside the module
|
|
because a container has no reload and a recreate would drop every connection the mesh has
|
|
- [ ] 1.3 the controller composes that file: accounts, permissions, TLS, JetStream — permissions
|
|
derived from `emits` and `consumes` and nothing else, plus each user's own ack subject and its
|
|
own inbox prefix (design 25 §4)
|
|
- [ ] 1.4 the four streams, created at genesis and asserted idempotently on start, by the controller
|
|
as their only writer
|
|
- [ ] 1.5 genesis raises it as foundation, claiming the seat **`mesh-broker`** — the seat is the
|
|
server's role, not the product
|
|
- [ ] 1.6 the genesis-broker bed
|
|
|
|
**Done when.** A mesh raised from nothing has the server standing with the streams asserted and
|
|
every account and permission composed from the manifests; a user cannot publish outside its
|
|
`emits`, subscribe outside its `consumes`, ack another user's delivery or subscribe another's inbox
|
|
prefix; the monitoring port is refused from anything but the private network; a change to the
|
|
composed file is live within one watcher interval without a restart, and the container is not
|
|
recreated by it.
|
|
|
|
**The permission checks belong here rather than later** because the server enforces them itself — a
|
|
plain client proves them, no link required — and they are the whole of what ADR 0043 asks for. No
|
|
mesh traffic is on the bus yet; that is step 4's bed, not this one's.
|
|
|
|
## Step 2 — adoption puts it in the seat
|
|
|
|
**Why here.** It needs only step 1's module, it is the path the mesh that exists will actually take,
|
|
and it is what makes steps 3 and 4 safe to develop against a live mesh. A running mesh does not get
|
|
a foundation module by being raised again; it adopts one in place
|
|
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)).
|
|
|
|
- [ ] 2.1 the server raised beside the existing broker on its own ports, carrying nothing
|
|
- [ ] 2.2 the `nats` module adopted onto it in place, holding the data and configuration it was
|
|
raised with
|
|
- [ ] 2.3 the seat claim, and the resolver's refusal of a second holder mesh-wide
|
|
- [ ] 2.4 the adoption bed
|
|
|
|
**Done when.** A mesh already running has the server adopted, holding `mesh-broker`; a second
|
|
assignment anywhere is refused at resolution — *one per mesh*; and every node is still on the old
|
|
bus with nothing routed to the new one. **That last check is the point of the step**: adoption that
|
|
quietly carried traffic would be step 5 arriving early and unrehearsed.
|
|
|
|
## Step 3 — the protocol on NATS
|
|
|
|
**Why here.** The implementations cannot be written against an unwritten wire, and this is the step
|
|
that decides what "agreeing" means for everything after it. It is the largest step and the one that
|
|
pays for itself furthest away.
|
|
|
|
- [ ] 3.1 **the suite first, on the bus the mesh has** — fixtures for the envelope and its required
|
|
headers, the contributions file, a served tool call and a grant, capturing what the three
|
|
implementations do *today*. Written first because a suite born on the new bus certifies
|
|
whatever the new bus happens to do
|
|
- [ ] 3.2 design 19 rewritten from exchanges, queues and routing keys to the subjects and streams of
|
|
design 25 §2–§3, per capability, with ADR 0074's model untouched: floor plus capabilities, an
|
|
implementation legitimate when it claims less, identity from the sealed credential, dedup on
|
|
`x-event-id`
|
|
- [ ] 3.3 the fixtures restated on NATS, and the capability each implementation claims
|
|
- [ ] 3.4 the controller's link on NATS
|
|
- [ ] 3.5 the host's link on NATS — mirroring, still importing nothing
|
|
- [ ] 3.6 the tool runtime's client on NATS, behind the unchanged sdk contract
|
|
- [ ] 3.7 the sdk's three stale comments, and nothing else in it
|
|
|
|
**Done when.** The fixtures are produced and consumed byte for byte by every implementation that
|
|
claims the capability, and a module built before any of this serves its tools unchanged on the new
|
|
runtime. **The step is not done when the code runs** — two implementations that disagree about an
|
|
envelope do not fail to compile, they ignore each other while both keep running, which is the
|
|
failure ADR 0074 exists to catch.
|
|
|
|
**And the shared library gained nothing but the binding.** A new transport is when the pressure to
|
|
add conveniences is highest, and ADR 0039's rule does not bend for it: a helper that arrives with
|
|
the bus is a review failure, not a detail. Code shared among a module's own features stays in that
|
|
module.
|
|
|
|
## Step 4 — the core speaks it
|
|
|
|
**Why here.** The links exist from step 3, so the flows that are not on the bus at all can move onto
|
|
it, and the beds that need a mesh living on NATS can finally run.
|
|
|
|
- [ ] 4.1 **the full genesis bed** — a mesh raised on NATS from nothing and living on it: a node
|
|
enrols over TLS with a claimed token and the enrolment user cannot read a declaration; a push
|
|
is held while the store restarts and applies after, nothing lost or duplicated; a node that
|
|
was away gets exactly the newest declaration and refuses a replayed older one by sequence; an
|
|
upgrade rolls out to two nodes; an event dead-letters after `max-deliver`; and an enrolment
|
|
held by a `nak`-with-delay cycle still reaches the enrolling node, proving the reply travels
|
|
in the payload and not the transport field the consumer's ack has claimed. The server-enforced
|
|
permissions were proved at step 1 and are not re-proved here
|
|
- [ ] 4.2 a build source's change reaches the builder over the bus, and the build that follows is
|
|
the one the change asked for
|
|
- [ ] 4.3 an installation completes over the bus, with the same outcome as the path it replaces
|
|
- [ ] 4.4 a person's client: the account, the client that speaks the bus, and the tool surface over
|
|
it (design 25 §7) — a module's tool invoked from another node and from a person, refused from
|
|
an account that may not
|
|
- [ ] 4.5 reports and catch-up: a node that was unreachable catches up rather than losing them
|
|
|
|
**Done when.** Each converted flow is proved against the behaviour it replaced, and the full genesis
|
|
bed is green. **Observation is not in this step** — heartbeats, conditions and key-value state are
|
|
[research 017](../../01-RESEARCH/017-a-mesh-that-heals-itself/00-overview.md)'s, that effort already
|
|
reserves them for after the move, and a flow built ahead of its design would be rebuilt.
|
|
|
|
## Step 5 — the rollout
|
|
|
|
**Why here.** It is the only step that moves a node's bus, and it moves every node's at once.
|
|
|
|
- [ ] 5.1 the cutover bed: a mesh on AMQP with a predecessor stand-in on the compatibility broker
|
|
moves its bus in one rollout, every node reporting on NATS afterwards, the stand-in's own
|
|
client still connected throughout
|
|
- [ ] 5.2 the rollout: accounts composed, then the controller, every host and every runtime
|
|
together; every node confirmed heard before AMQP stops
|
|
- [ ] 5.3 the mesh's accounts removed from the compatibility broker, leaving the predecessor's users
|
|
- [ ] 5.4 the compatibility broker retires when its condition holds — no client connected for the
|
|
period the operator sets
|
|
|
|
**Done when.** Every node reports on NATS and the predecessor's clients never noticed.
|
|
|
|
## The through-line
|
|
|
|
The order is dependency, not preference. **Steps 1 to 4 leave every node on AMQP**, so the cost of
|
|
being wrong is bounded until the last step: a step may be abandoned, or reordered after step 2,
|
|
without a rollback. The server stands before anything speaks to it; the wire is specified before it
|
|
is implemented three times; the flows move once there is something to move them onto; and the bus
|
|
itself moves once, at the end, on one day.
|
|
|
|
## What is deliberately not here
|
|
|
|
- **Observation** — research 017's, after the move, by its own design.
|
|
- **Leaf nodes** — design 25 §11 keeps this out of scope and says so; a leaf per machine is a later
|
|
question, noted so it is not forgotten.
|
|
- **The predecessor's world.** It is AMQP, it cannot move, and it does not need to: its broker is
|
|
the compatibility module until its last client is gone.
|
|
|
|
## How this list is kept true
|
|
|
|
A task is ticked when its change is committed, not when it is written. A step is done when its bed
|
|
is green, not when its tasks are ticked. If a step's tasks are all ticked and its bed has not run,
|
|
the step is **in progress** and this document says so — that gap is the thing the whole shape is
|
|
built to make visible.
|