Issue 010 fixed: origins keep the bundle and the mesh apart

The store records where each resource came from and each origin removes only
its own. Verified on the scenario that caused it -- eleven resources raised,
enrolled, sent the same two-resource declaration, and the store, broker and
control plane were all still running. A later declaration dropping a resource
still removed it, so removal by omission survived the fix.

Two more faults found while fixing it, both the same shape. A report published
to a routing key nobody bound vanishes: the broker accepts it, finds no queue,
drops it, and tells the publisher nothing -- so nodes announced what they had
applied into a void. And publishReport was discarding its error, so a node that
could not tell the mesh looked exactly like one that had.

Reports are mandatory now, so an unroutable one comes back and is said out
loud, and the binding covers every key a node may publish.
This commit is contained in:
2026-08-29 16:43:28 +02:00
parent 594ea10b07
commit 6bcf0e4f9f
@@ -1,8 +1,9 @@
---
status: open
status: fixed
opened: 2026-08-29
located-in: [mesh-host, mesh-control]
fixed-by:
- "mesh-host: the store records where each resource came from — carried or declared — and each origin removes only its own. Verified in the lab on the exact scenario that caused this: the substrate survived, and a later declaration still removed what it had itself declared."
amended-design:
---
@@ -55,15 +56,28 @@ So the first node is left in a state no other node is in, and the ordinary path
- **Special-casing the first node.** [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)
is explicit that its specialness lasts two commands, and this would extend it for ever.
## The shape of an answer, not yet chosen
## The fix
The store needs to record **where a resource came from** — carried, or declared — and reconcile
each against its own source. A declaration would then remove only what the mesh previously
declared, and the bundle would remain the bundle's business until the mesh is told about it.
**The store records where each resource came from** — `carried` or `declared` — and each origin
removes only its own. A declaration removes what the mesh previously declared and never what the
bundle raised; reconciling the bundle removes what the bundle previously raised and never what the
mesh assigned.
That leaves a real question behind it: **what happens when the mesh eventually does declare the
substrate**, which it must, or the substrate can never be upgraded. Two sources claiming the same
container is the ambiguity this issue is made of, moved rather than removed.
State written before the field existed reads as `carried`, because everything a host had applied
at that point came from its bundle — there was no other way to tell it anything. Guessing the
other way would have the first upgrade remove the substrate, which is this fault arriving through
the change that fixes it.
**Verified on the scenario that caused it.** A first node raised eleven resources, enrolled, and
was sent the same two-resource declaration. Both applied; the store, the broker and the control
plane were still running afterwards. A second declaration dropping one resource removed that
resource and nothing else, so removal by omission still works — which is the property that had to
survive the fix.
**What remains open** is what happens when the mesh eventually declares the substrate, which it
must, or the substrate can never be upgraded. Two sources claiming one container is the ambiguity
this issue is made of, narrowed rather than removed: it can no longer happen by accident, and
nothing yet says what it means when it happens on purpose.
## How it was found
@@ -73,3 +87,18 @@ this fell out of it.
The declaration was two lines and destroyed a working mesh in under a second, which is worth
holding on to: this is not an edge case reached by trying, it is the first thing that happens.
## Two more faults found while fixing it
Both of the same shape, and worth recording because the shape is the point.
**A report published to an unbound routing key vanishes.** The control plane bound `enrol` and not
`report`, so nodes announced what they had applied into a void — the broker accepted each message,
found no queue for it, and dropped it. The publisher was told nothing. Reports are now published
`mandatory`, so anything unroutable comes back and is said out loud, and the binding covers every
key a node may publish.
**A publish failure was being swallowed.** `publishReport` discarded its error, so a node that
could not tell the mesh what it had done looked identical to one that had. That is the fault this
repository keeps cataloguing, written by hand into the newest code in it.