diff --git a/03-DESIGN/01-to-be/28-building-the-bus.md b/03-DESIGN/01-to-be/28-building-the-bus.md index 51363f4..7c16fea 100644 --- a/03-DESIGN/01-to-be/28-building-the-bus.md +++ b/03-DESIGN/01-to-be/28-building-the-bus.md @@ -5,7 +5,7 @@ code: - mesh-catalog modules/nats - mesh-controller internal/catalogue - mesh-lab scenarios -updated: 2026-09-26 +updated: 2026-09-27 decisions: - 02-DECISIONS/0116-the-bus-is-built-in-five-steps.md - 02-DECISIONS/0106-the-bus-is-nats.md @@ -242,36 +242,48 @@ pays for itself furthest away. `x-event-id`. Claims checked against a running server are marked *verified* in the text, so a reader can tell what was measured from what was reasoned. One limitation lifts with the transport: a module may now call another's tool, which issue 049 recorded it could not. -- [~] 3.4 the controller's link on NATS — **the seam exists and the outbound half is through - it.** `Bus` is stated in the mesh's words (publish an event, declare to a node) rather than - a transport's, with an AMQP and a NATS implementation, both shipping: steps 1 to 4 leave - every node on AMQP, and both shipping is what lets one conformance fixture hold them to the - same envelope. The NATS one is checked against a real server, reading back from the stream - rather than from the code that wrote it. +- [x] 3.4 the controller's link on NATS — **both halves are through the seam, and the store + window is the server's.** `Bus` states the outbound in the mesh's words (publish an event, + declare to a node) and `Control` states the inbound (took it, dropped it, held it for the + store); each has an AMQP and a NATS implementation, and both ship, because steps 1 to 4 + leave every node on AMQP and both shipping is what holds them to one envelope. - The seam turned out to be eight call sites — the same smallness that said this bus could be - replaced at all. + The outbound seam turned out to be eight call sites; the inbound was the larger half, and + the reason: every handler took the transport's own delivery type, so the loop could not move + without moving enrolment, reports, builds, upgrades and catch-up with it in one breath. - **The consume side's hard part is decided and tested**: the store window is a `nak` with a - delay rather than a delivery held in memory, which also means a controller restarting - mid-window loses nothing. Moving the holding into the server costs one thing — an older - report is redelivered after a newer was applied — and a report already carries the digest - of the declaration it is about, so supersession becomes a check rather than something the - controller remembers. Pure and tested without a bus, a store or a clock. + **The window (ADR 0083) is now what decides, once, for both.** On the bus the mesh has, + holding a message means an unacknowledged delivery kept in the controller, bounded by the + prefetch and lost if it stops. On the bus being built it is a `nak` with a delay: the + message stays the server's and the controller keeps only the moment it first could not take + it, so one that restarts mid-window has nothing to lose. Seven claims about that were asked + of a running server rather than reasoned — a report heard and gone from the work queue, one + held through a store outage and recorded when it returned, one let go once the bound passed, + a superseded one settled without being acted on, a heartbeat heard and nothing persisted, + both followed events acknowledged on a stream the controller had no ack subject for, and the + enrolment answer arriving at the address the request carried in its payload. - **Asking a tool is through the seam and loses two problems**: there is no reply queue to - declare and no correlation to check, because each account has one inbox prefix and an - answer cannot reach the wrong asker — which settles a cost the build code records having - paid, where every asker saw every result. And a tool nobody serves says so at once instead - of after the whole wait, which is the difference between "that module is down" and "that - tool is slow". + **Three things the wiring forced into the open.** - **A build is a different shape, not the same one.** It takes minutes, so it is work - submitted to a queue with the outcome returning to a reply subject the request carries — - the pattern design 25 §2 already sets for anything crossing a stream. It touches the - builder as well, so it travels with that conversion in step 4. + *Supersession is asked before the store, not after.* A report about a declaration the mesh + has moved past would otherwise wait out a restarting store to be written, and then overwrite + what the node is doing now. - Still outstanding: wiring the window decision into the loop, enrolment, and serving. + *Half of a report is not about a declaration, and that half is never stale.* What the machine + **is** — the tunnel it took over, the ports its own bundle holds, what an adopted node found, + a node moving its overlay key — reaches the mesh on a report and nowhere else. A rekey set + aside as stale is a node whose overlay key never moves, and no retry is coming, because the + node said it once. So staleness is asked only of a report that is purely an apply's account. + + *The controller could not have consumed a module event at all.* Its account granted no event + subject to subscribe and no ack subject on the events stream, so every announcement would + have been redelivered for ever, refused by the permission list it already had. Both are now + granted, each subject named rather than by pattern — a controller subscribing every event in + the mesh is a permission list that has stopped saying what it is for. Its consumers are + **named beside the mesh's own streams rather than derived**, because the controller files no + manifest and authority cannot come from a declaration that does not exist. + + Still outstanding: a build's own shape, which travels with the builder in step 4. - [~] 3.5 the host's link on NATS — **the outbound half is through a seam**, mirroring the controller's and still importing nothing of the mesh's own (ADR 0005): the host's own interface over its own libraries, agreeing with the controller only because a fixture holds @@ -349,6 +361,17 @@ module. **Why here.** The links exist from step 3, so the flows that are not on the bus at all can move onto it, and the beds that need a mesh living on NATS can finally run. +> **A blocker surfaced here that is not this step's to fix.** Every event name in the catalogue is +> still written the way a routing key on the bus the mesh has is written, so the derivation design 29 +> §1 specifies turns a consumer's declaration into a subject **no emitter publishes** — thirty-seven +> manifests, and one that cannot be composed at all. Nothing fails on the bus the mesh runs on +> today, where a routing key is matched literally; it fails on the first mesh raised on the new bus +> and not before, which is why wiring the controller's own subscription is what found it. Opened as +> [issue 127](../../04-ISSUES/127-a-module-event-derives-a-subject-nothing-publishes/00-report.md). +> It holds 4.2, 4.3 and the catch-up half of 4.5; the node-facing flows — enrolment, reports, +> heartbeats, a build's outcome — are unaffected, because those subjects are the mesh's own and +> derive from nothing a module declares. + - [ ] 4.1 **the full genesis bed** — a mesh raised on NATS from nothing and living on it: a node enrols over TLS with a claimed token and the enrolment user cannot read a declaration; a push is held while the store restarts and applies after, nothing lost or duplicated; a node that @@ -358,8 +381,10 @@ it, and the beds that need a mesh living on NATS can finally run. in the payload and not the transport field the consumer's ack has claimed. The server-enforced permissions were proved at step 1 and are not re-proved here - [ ] 4.2 a build source's change reaches the builder over the bus, and the build that follows is - the one the change asked for -- [ ] 4.3 an installation completes over the bus, with the same outcome as the path it replaces + the one the change asked for — **blocked by + [issue 127](../../04-ISSUES/127-a-module-event-derives-a-subject-nothing-publishes/00-report.md)** +- [ ] 4.3 an installation completes over the bus, with the same outcome as the path it replaces — + **blocked by the same** - [~] 4.4 a person's client — **the account is done**: a person is not a module and holds no seat, so their authority is a list of tools (or `*` for an administrator) and nothing else. Held to four properties, each a way of being wrong that would not announce itself: nothing @@ -370,7 +395,13 @@ it, and the beds that need a mesh living on NATS can finally run. Still to build: the client program itself — the command line and the MCP surface over it. It needs nothing from the consume side, so it is not blocked by step 3. -- [ ] 4.5 reports and catch-up: a node that was unreachable catches up rather than losing them +- [~] 4.5 reports and catch-up: a node that was unreachable catches up rather than losing them — + **the reports half is in and proved against a server** (3.4): held through the store's + absence by the server rather than by the controller, superseded ones settled by the digest + they carry. The catch-up half is where issue 127 bites hardest: the controller replays a + build announcement under its **own** name rather than the builder's, so a catalogue + filtering the builder's subject hears nothing. Whether the controller may sign an event as + another module is a design question, not a wiring one, and it is open in that issue. **Done when.** Each converted flow is proved against the behaviour it replaced, and the full genesis bed is green. **Observation is not in this step** — heartbeats, conditions and key-value state are diff --git a/04-ISSUES/127-a-module-event-derives-a-subject-nothing-publishes/00-report.md b/04-ISSUES/127-a-module-event-derives-a-subject-nothing-publishes/00-report.md new file mode 100644 index 0000000..09b6733 --- /dev/null +++ b/04-ISSUES/127-a-module-event-derives-a-subject-nothing-publishes/00-report.md @@ -0,0 +1,93 @@ +--- +status: open +opened: 2026-09-27 +located-in: [] +fixed-by: +amended-design: +--- + +# 127 — A module's event derives a subject nothing publishes + +## What was observed + +[Design 29](../../03-DESIGN/01-to-be/29-what-a-module-declares.md) §1 says a module names an event +locally and the mesh derives the subject: `emits: order.placed` becomes +`mesh.mod..event.order.placed`, and a consumer declaring `consumes: shop.order.placed` +subscribes the emitter's own subject. That derivation is built and tested. + +**Every event name in the catalogue is still written the way a routing key on the bus the mesh has +is written** — `module..` — and the derivation reads it as `.`. Asked +of the composer directly, with the module names and declarations the catalogue holds today: + +| declared | derived | +|---|---| +| `builder` emits `module.builder.built` | publish `mesh.mod.builder.event.module.builder.built` | +| the catalogue consumes `module.builder.built` | subscribe `mesh.mod.module.event.builder.built` | +| a media module emits `module..download.completed` | publish `mesh.mod..event.module..download.completed` | +| a player consumes `module.*.download.completed` | subscribe `mesh.mod.module.event.*.download.completed` | + +The consumer's subject names a module called `module`. **No cross-module subscription in the +catalogue matches what any emitter publishes.** Thirty-seven manifests declare events; every one of +their consume declarations derives this way. + +Two further consequences of the same cause, found in the same check: + +- One module declares `consumes: "#"` — the wildcard of the bus the mesh has, which is not a + subject at all. The composer **refuses it outright**, so that module's account cannot be composed + and the module cannot be assigned. +- One module emits under a name that is not its own — it declares `module..image.pushed` + while being a differently named module — which the derivation puts inside *its* namespace. Whether + that is legitimate is a design question: design 29 §2 makes an event's source a fact the server + enforces, and this is a module claiming another's name in its own event. + +None of it fails on the bus the mesh runs on today, where a routing key is matched literally and +nothing derives anything. It fails only once the subject is derived — which is to say it fails on +the first mesh raised on the new bus, and not before. + +Evidence: run against the controller's own `PermissionsFor` on the current feature branch, with the +declarations read from the catalogue's manifests. Found while wiring the controller's consume side +(design 28 step 3.4), when the controller's own subscription had to be written and the subject it +would have to name turned out not to be the one design 29 specifies. + +## Why it matters beyond this instance + +**This is the failure [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) +exists to catch, arriving by a route the conformance suite does not cover.** Two implementations +that disagree about an envelope do not fail to compile — they ignore each other while both keep +running. Here it is not two implementations disagreeing but a *declaration* and a *derivation* +disagreeing, and the symptom is identical: every service starts, every log is quiet, and nothing +reacts to anything. + +The fixtures cannot catch it. They pin one emitter's envelope against one subject, and both halves +of that pair are correct. What is wrong is only visible when an emitter's derived subject is set +beside a consumer's derived subject — a check nothing performs, because until the subject was +derived there was nothing to compare. + +It also means the rule 3.8 established is weaker than it reads. That task asserted **no manifest +contains a subject**, which holds: a manifest contains a local name. What nothing asserts is that a +local name derives to a subject some emitter actually publishes, and the rule as stated is satisfied +by thirty-seven manifests whose names derive to nothing. + +And it blocks work already scheduled. Design 28's step 4.2 (a build source's change reaching the +builder over the bus) and 4.3 (an installation completing over the bus) are both event flows through +exactly these pairs, and the catch-up flow the controller answers is a third — the controller +currently replays a build announcement under its *own* name rather than the builder's, which a +consumer filtering the builder's subject will not hear either. + +## Open questions + +- Is a local name converted per manifest (`emits: built`), or does the derivation keep accepting the + old form and strip a redundant prefix? The first is thirty-seven manifests and a rule that can be + checked; the second is a rule that cannot, because `module.foo.bar` is also a legitimate three-part + local name. +- What checks the pair? An emitter's derived subject against every consumer's derived subject is a + whole-catalogue check, not a per-manifest one — and a module lives in its own repository and may + be registered long after the catalogue was checked. +- What are `#` and `*` in a consumed name? The bus the mesh has and the bus being built spell + wildcards differently, and a `consumes` pattern is the one place a module writes one. +- May a module emit an event named after another module, and if not, what does the module that does + it today declare instead? +- Who replays? A catch-up answer published by the controller under a builder's subject is the + controller signing an event as another module, which is the thing the derived namespace prevents. + If it must not, then a replay is a different message from an announcement, and the consumer needs + to be told so.