Issue 010: the first declaration destroys the substrate
Found in the lab, doing the ordinary thing: raise a first node, enrol it, send it a declaration. Both declared resources applied correctly and every container on the machine was removed -- the store, the broker, and the control plane that had sent the message. The link died mid-sentence because the broker carrying it had just been torn down by what it carried. Nothing is behaving incorrectly. Apply removes what the store holds and the declaration does not name, which is what reconciliation means. The fault is that the carried bundle and mesh declarations share one store, so the host cannot tell what this machine raised for itself before there was a mesh from what the mesh told it to have. It is invisible until those two meet, which happens exactly once per mesh: on the first node, after enrolment, the moment the control plane first speaks. The report says what is not the answer, including the tempting one -- having the control plane send the substrate back. It cannot: it was never told what the bundle contained, and the bundle exists precisely because there was no control plane to ask.
This commit is contained in:
@@ -0,0 +1,75 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-08-29
|
||||
located-in: [mesh-host, mesh-control]
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 010 — The first declaration a node receives destroys the substrate it raised
|
||||
|
||||
## Symptom
|
||||
|
||||
A first node was raised from its carried bundle: container runtime, store, two context databases,
|
||||
their schemas, broker, control plane. Eleven resources, all running. It then enrolled against the
|
||||
control plane on its own machine, held its link open, and was sent a declaration naming two
|
||||
resources — a directory and a file.
|
||||
|
||||
Both were applied correctly. And **every container on the machine was removed**: the store, the
|
||||
broker, and the control plane that had sent the declaration. The link died mid-sentence with
|
||||
`the link closed: Exception (501) Reason: "EOF"`, because the broker carrying it had just been
|
||||
torn down by the message it carried.
|
||||
|
||||
Afterwards `mesh-host owned` listed two resources. The mesh had deleted itself.
|
||||
|
||||
## What is actually wrong
|
||||
|
||||
Nothing in the code is behaving incorrectly. `apply` removes what the store holds and the incoming
|
||||
declaration does not name, which is what reconciliation means — the declaration is the desired
|
||||
state, not a patch, and anything else would make it impossible to remove a resource by omission.
|
||||
|
||||
**The fault is that the carried bundle and mesh declarations share one store.** The host cannot
|
||||
tell "this machine raised this for itself before there was a mesh" from "the mesh told this
|
||||
machine to have this", so the second overwrites the first completely.
|
||||
|
||||
That is invisible until the two meet, which happens exactly once per mesh: on the first node,
|
||||
after enrolment, at the moment the control plane first speaks.
|
||||
|
||||
## Why it matters more than a footgun
|
||||
|
||||
**The first node is the only node where the substrate is not the mesh's doing.** Every other node
|
||||
receives everything it runs from the control plane, so a complete declaration is complete by
|
||||
construction. The first node raised its own substrate from a file it carried, and the control
|
||||
plane has never been told about it — so the control plane cannot include it in a declaration even
|
||||
if it wanted to.
|
||||
|
||||
So the first node is left in a state no other node is in, and the ordinary path destroys it.
|
||||
|
||||
## What is not the answer
|
||||
|
||||
- **Making the control plane send the substrate back.** It does not know what the bundle contained
|
||||
and should not: the bundle exists precisely because there was no control plane yet.
|
||||
- **Making apply stop removing orphans.** Removal by omission is how a declaration says *stop
|
||||
running this*, and losing it costs the property that a node converges on what it was told rather
|
||||
than accumulating.
|
||||
- **Special-casing the first node.** [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)
|
||||
is explicit that its specialness lasts two commands, and this would extend it for ever.
|
||||
|
||||
## The shape of an answer, not yet chosen
|
||||
|
||||
The store needs to record **where a resource came from** — carried, or declared — and reconcile
|
||||
each against its own source. A declaration would then remove only what the mesh previously
|
||||
declared, and the bundle would remain the bundle's business until the mesh is told about it.
|
||||
|
||||
That leaves a real question behind it: **what happens when the mesh eventually does declare the
|
||||
substrate**, which it must, or the substrate can never be upgraded. Two sources claiming the same
|
||||
container is the ambiguity this issue is made of, moved rather than removed.
|
||||
|
||||
## How it was found
|
||||
|
||||
In the lab, on a sealed machine, by doing the ordinary thing: raise a first node, enrol it, and
|
||||
tell it something. It was not a test of this — it was the first end-to-end run of the link, and
|
||||
this fell out of it.
|
||||
|
||||
The declaration was two lines and destroyed a working mesh in under a second, which is worth
|
||||
holding on to: this is not an edge case reached by trying, it is the first thing that happens.
|
||||
Reference in New Issue
Block a user