diff --git a/03-DESIGN/01-to-be/25-the-bus-on-nats.md b/03-DESIGN/01-to-be/25-the-bus-on-nats.md index beaa57a..0b0fc45 100644 --- a/03-DESIGN/01-to-be/25-the-bus-on-nats.md +++ b/03-DESIGN/01-to-be/25-the-bus-on-nats.md @@ -307,6 +307,9 @@ when its retirement condition holds. The bus's port settings follow ADR 0100 lik What is not done, at any step: no dual-bus period for the mesh's own traffic, no bridge, no module rebuilt. +**The work of these five, broken down and measured, is [design 28](28-building-the-bus.md)** — +including the two places their dependencies put a bed later than the step that names it. + ## 10. How it is checked **A bed per step, and each is green before the step after it starts** — the division in §9 is only diff --git a/03-DESIGN/01-to-be/28-building-the-bus.md b/03-DESIGN/01-to-be/28-building-the-bus.md new file mode 100644 index 0000000..ca5f952 --- /dev/null +++ b/03-DESIGN/01-to-be/28-building-the-bus.md @@ -0,0 +1,251 @@ +--- +layer: to-be +status: proposed +code: [] +updated: 2026-09-26 +decisions: + - 02-DECISIONS/0115-the-bus-is-built-in-five-steps.md + - 02-DECISIONS/0106-the-bus-is-nats.md + - 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md + - 02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md + - 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md + - 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md +--- + +# 28. Building the bus + +**The work of [ADR 0115](../../02-DECISIONS/0115-the-bus-is-built-in-five-steps.md)'s five steps, +in the order its dependencies allow, with what each ends at.** +[Design 25](25-the-bus-on-nats.md) is the architecture and stays the authority on *what is built*; +this document holds only the order, the sizes and the proofs, and it is wrong the moment it +disagrees with design 25 rather than the other way round. + +Each step ends at something runnable. A step that cannot name what its bed proves is not a step, +and is divided further before it is started. + +## How this is built, and when it is run + +**Written as code with unit tests, committed per change, and taken to the lab once the pieces that +would change the outcome are in place.** The mistake this avoids is the one +[design 22](22-the-work-ahead.md) records: running a long bed against a mesh mid-transformation and +debugging paths the next step deletes. Where a fault can be reasoned out of the code path, it is — +reading, not running. + +So the beds below are acceptance tests at the end of assembled work, not the tool for finding each +bug, and a step's bed is run when that step is finished rather than while it is being written. + +## What the work is, measured + +Counted 2026-09-26, non-test source only. The point of counting is that none of this is unknown +territory: every piece has a shape already standing beside it. + +| Piece | Today | Size | Becomes | +|---|---|---|---| +| the controller's link | Go, one package | ~1 800 lines | the same package on NATS | +| the host's link | Go, one package, mirroring the contracts rather than importing them | ~1 000 lines | the same, on NATS | +| the tool runtime's client | TypeScript, one file | ~390 lines | the same, on NATS | +| the sdk's messaging surface | TypeScript: messaging, events, tools, contracts, primitives | ~360 lines across five | **unchanged**, see below | +| the broker module | the adopted AMQP broker: client, tools, provisioner, bootstrap, image, manifest | ~340 lines of module code | the `nats` module, same shape | +| the beds | 39 lab scenarios, including a broker bed, an adoption bed, a genesis bed and a store-window bed | — | four analogues and one new | + +**Two measurements are worth stating on their own, because they change what the steps are.** + +**The sdk speaks no AMQP, and never did.** The word appears in its source three times, in three +comments; its messaging module says in as many words that it "carries the contract, not a specific +AMQP client build." [ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md) put the +client in the runtime, and the payoff is collected here: **no module is rebuilt for this change, and +the sdk's own diff is three comments.** That is the whole reason a bus can be replaced under a live +mesh at all. + +**The wire therefore has three implementations, not two, and no suite pins any of them.** +[ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) spoke of "the existing +two implementations" — Go and TypeScript. Measured, the Go side is *two separate packages* that +mirror rather than share (the host imports nothing, by +[ADR 0005](../../02-DECISIONS/0005-the-node-host.md)), so the count is the controller's link, the +host's link, and the runtime's client. And a search for conformance fixtures finds none anywhere in +the four repositories: design 22's Phase 1.2 — the suite — has not been built. + +> **A correction to ADR 0115, found in the measuring.** That record says step 3's fixtures are +> *recaptured* on NATS. There is nothing to recapture: the suite does not exist. Step 3 **builds** +> it, and its first job is to pin the wire the mesh has before changing it, because a suite written +> only against the new bus certifies whatever the new bus happens to do. The record's decision is +> unaffected; the word was wrong, and is corrected here rather than edited there +> ([`02-DECISIONS/README.md`](../../02-DECISIONS/README.md): a record's meaning is never edited). + +## The order the work actually allows + +**The five steps are chunks of capability; the build order is not simply 1 to 5, and pretending +otherwise would put two beds where they cannot run.** Three edges decide it: + +- **A specification precedes the implementations it governs.** ADR 0074's whole argument is that + agreement is specified and checked, not hoped for. So the wire's NATS binding is written *before* + the three implementations are, even though it is step 3 — and its conformance half can only + *finish* once two implementations exist to disagree. +- **A mesh cannot be raised on a bus nothing speaks.** A bed that raises a mesh on NATS from + genesis — enrolling a node, holding a push while the store restarts, rolling out an upgrade — + needs the controller and the host to speak NATS already. That is the implementations, and they + arrive with step 3. +- **Adoption needs the module and nothing else.** Step 2 puts a correctly configured server into a + running mesh that continues to ignore it, which depends on no link at all. + +So step 1's bed proves *the server, from genesis, configured* — not a mesh living on it. The full +genesis bed is step 4's, where it can first run. + +> **This differs from ADR 0115's check list, deliberately.** That record attributes "a mesh raised +> on NATS from genesis" to step 1. Measuring the dependency showed that bed cannot run until the +> links exist, and a step whose proof cannot run is the exact failure the record was written to +> prevent. The five steps, their names and the single rollout are unchanged; only where two beds +> run has moved. If that reads as a change of meaning rather than a correction of fact, the fix is +> a superseding record, not an edit. + +``` +step 1 module, genesis places it ──┐ +step 2 adoption into a running mesh ──┤ neither needs a link + │ +step 3 the wire specified ──► three implementations ──► the suite + │ +step 4 the flows, and the full genesis bed + │ +step 5 the rollout +``` + +## Step 1 — the module, and genesis raises it + +**Why here.** Everything else needs a server to talk to, and genesis is where the foundation is +defined. The mesh this is for will never travel this path — it is already running, and takes step 2 +— but genesis is the definition every other path is measured against, and one that exists only on +paper is wrong until there is a second mesh to find out. + +- [ ] 1.1 the `nats` module: manifest, image, one container, its client, TLS and monitoring ports, + JetStream on a named volume — the shape of design 25 §5, and the same shape the broker module + beside it already has +- [ ] 1.2 the composed configuration as a **directory** resource, and the entrypoint that watches + the one file and signals the server itself — design 25 §5's correction, kept inside the module + because a container has no reload and a recreate would drop every connection the mesh has +- [ ] 1.3 the controller composes that file: accounts, permissions, TLS, JetStream — permissions + derived from `emits` and `consumes` and nothing else, plus each user's own ack subject and its + own inbox prefix (design 25 §4) +- [ ] 1.4 the four streams, created at genesis and asserted idempotently on start, by the controller + as their only writer +- [ ] 1.5 genesis raises it as foundation, claiming the seat **`mesh-broker`** — the seat is the + server's role, not the product +- [ ] 1.6 the genesis-broker bed + +**Done when.** A mesh raised from nothing has the server standing with the streams asserted and +every account and permission composed from the manifests; the monitoring port is refused from +anything but the private network; a change to the composed file is live within one watcher interval +without a restart, and the container is not recreated by it. No mesh traffic is on it yet — that is +step 4's bed, not this one's. + +## Step 2 — adoption puts it in the seat + +**Why here.** It needs only step 1's module, it is the path the mesh that exists will actually take, +and it is what makes steps 3 and 4 safe to develop against a live mesh. A running mesh does not get +a foundation module by being raised again; it adopts one in place +([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)). + +- [ ] 2.1 the server raised beside the existing broker on its own ports, carrying nothing +- [ ] 2.2 the `nats` module adopted onto it in place, holding the data and configuration it was + raised with +- [ ] 2.3 the seat claim, and the resolver's refusal of a second holder mesh-wide +- [ ] 2.4 the adoption bed + +**Done when.** A mesh already running has the server adopted, holding `mesh-broker`; a second +assignment anywhere is refused at resolution — *one per mesh*; and every node is still on the old +bus with nothing routed to the new one. **That last check is the point of the step**: adoption that +quietly carried traffic would be step 5 arriving early and unrehearsed. + +## Step 3 — the protocol on NATS + +**Why here.** The implementations cannot be written against an unwritten wire, and this is the step +that decides what "agreeing" means for everything after it. It is the largest step and the one that +pays for itself furthest away. + +- [ ] 3.1 **the suite first, on the bus the mesh has** — fixtures for the envelope and its required + headers, the contributions file, a served tool call and a grant, capturing what the three + implementations do *today*. Written first because a suite born on the new bus certifies + whatever the new bus happens to do +- [ ] 3.2 design 19 rewritten from exchanges, queues and routing keys to the subjects and streams of + design 25 §2–§3, per capability, with ADR 0074's model untouched: floor plus capabilities, an + implementation legitimate when it claims less, identity from the sealed credential, dedup on + `x-event-id` +- [ ] 3.3 the fixtures restated on NATS, and the capability each implementation claims +- [ ] 3.4 the controller's link on NATS +- [ ] 3.5 the host's link on NATS — mirroring, still importing nothing +- [ ] 3.6 the tool runtime's client on NATS, behind the unchanged sdk contract +- [ ] 3.7 the sdk's three stale comments, and nothing else in it + +**Done when.** The fixtures are produced and consumed byte for byte by every implementation that +claims the capability, and a module built before any of this serves its tools unchanged on the new +runtime. **The step is not done when the code runs** — two implementations that disagree about an +envelope do not fail to compile, they ignore each other while both keep running, which is the +failure ADR 0074 exists to catch. + +**And the shared library gained nothing but the binding.** A new transport is when the pressure to +add conveniences is highest, and ADR 0039's rule does not bend for it: a helper that arrives with +the bus is a review failure, not a detail. Code shared among a module's own features stays in that +module. + +## Step 4 — the core speaks it + +**Why here.** The links exist from step 3, so the flows that are not on the bus at all can move onto +it, and the beds that need a mesh living on NATS can finally run. + +- [ ] 4.1 **the full genesis bed** — a mesh raised on NATS from nothing: a node enrols over TLS with + a claimed token and the enrolment user cannot read a declaration; a push is held while the + store restarts and applies after, nothing lost or duplicated; a node that was away gets + exactly the newest declaration and refuses a replayed older one by sequence; an upgrade rolls + out to two nodes; an event dead-letters after `max-deliver`; a module cannot publish outside + its `emits`, ack another module's delivery, or subscribe another's inbox prefix; and an + enrolment held by a `nak`-with-delay cycle still reaches the enrolling node, proving the reply + travels in the payload and not the transport field the consumer's ack has claimed +- [ ] 4.2 a build source's change reaches the builder over the bus, and the build that follows is + the one the change asked for +- [ ] 4.3 an installation completes over the bus, with the same outcome as the path it replaces +- [ ] 4.4 a person's client: the account, the client that speaks the bus, and the tool surface over + it (design 25 §7) — a module's tool invoked from another node and from a person, refused from + an account that may not +- [ ] 4.5 reports and catch-up: a node that was unreachable catches up rather than losing them + +**Done when.** Each converted flow is proved against the behaviour it replaced, and the full genesis +bed is green. **Observation is not in this step** — heartbeats, conditions and key-value state are +[research 017](../../01-RESEARCH/017-a-mesh-that-heals-itself/00-overview.md)'s, that effort already +reserves them for after the move, and a flow built ahead of its design would be rebuilt. + +## Step 5 — the rollout + +**Why here.** It is the only step that moves a node's bus, and it moves every node's at once. + +- [ ] 5.1 the cutover bed: a mesh on AMQP with a predecessor stand-in on the compatibility broker + moves its bus in one rollout, every node reporting on NATS afterwards, the stand-in's own + client still connected throughout +- [ ] 5.2 the rollout: accounts composed, then the controller, every host and every runtime + together; every node confirmed heard before AMQP stops +- [ ] 5.3 the mesh's accounts removed from the compatibility broker, leaving the predecessor's users +- [ ] 5.4 the compatibility broker retires when its condition holds — no client connected for the + period the operator sets + +**Done when.** Every node reports on NATS and the predecessor's clients never noticed. + +## The through-line + +The order is dependency, not preference. **Steps 1 to 4 leave every node on AMQP**, so the cost of +being wrong is bounded until the last step: a step may be abandoned, or reordered after step 2, +without a rollback. The server stands before anything speaks to it; the wire is specified before it +is implemented three times; the flows move once there is something to move them onto; and the bus +itself moves once, at the end, on one day. + +## What is deliberately not here + +- **Observation** — research 017's, after the move, by its own design. +- **Leaf nodes** — design 25 §11 keeps this out of scope and says so; a leaf per machine is a later + question, noted so it is not forgotten. +- **The predecessor's world.** It is AMQP, it cannot move, and it does not need to: its broker is + the compatibility module until its last client is gone. + +## How this list is kept true + +A task is ticked when its change is committed, not when it is written. A step is done when its bed +is green, not when its tasks are ticked. If a step's tasks are all ticked and its bed has not run, +the step is **in progress** and this document says so — that gap is the thing the whole shape is +built to make visible. diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index 3c8f3d3..7e08068 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -34,8 +34,10 @@ document is written and this one's status becomes `implemented`. | [`22-the-work-ahead.md`](22-the-work-ahead.md) | Everything decided and not yet built, in dependency order, each phase ending at a run | [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md), [ADR 0075](../../02-DECISIONS/0075-two-stores-and-which-provides-what.md), [ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md) | | [`23-choosing-a-provider.md`](23-choosing-a-provider.md) | Which of several providers of a kind serves a consumer, and when a module carries its own instead | [ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md), [ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md) | | [`24-the-secrets-vault.md`](24-the-secrets-vault.md) | The module that owns a secret — a `secret` provision, and the boundary of what it owns | [ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md), [ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md), [ADR 0048](../../02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md) | +| [`25-the-bus-on-nats.md`](25-the-bus-on-nats.md) | **Proposed.** The architecture of the mesh's bus on NATS: what rides which subject under which guarantee and whose account, how a node joins, how a person reaches a tool, and how the mesh moves from the bus it has | [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md), [ADR 0115](../../02-DECISIONS/0115-the-bus-is-built-in-five-steps.md), [ADR 0043](../../02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md), [ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md) | | [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) | | [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) | +| [`28-building-the-bus.md`](28-building-the-bus.md) | **Proposed.** The five steps of the bus work in the order their dependencies allow, each ending at a bed — with the surface measured, so no step's size is a guess | [ADR 0115](../../02-DECISIONS/0115-the-bus-is-built-in-five-steps.md), [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md), [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) | ## Not yet written