Issue 010: the first declaration destroys the substrate

Found in the lab, doing the ordinary thing: raise a first node, enrol it, send
it a declaration. Both declared resources applied correctly and every container
on the machine was removed -- the store, the broker, and the control plane that
had sent the message. The link died mid-sentence because the broker carrying it
had just been torn down by what it carried.

Nothing is behaving incorrectly. Apply removes what the store holds and the
declaration does not name, which is what reconciliation means. The fault is
that the carried bundle and mesh declarations share one store, so the host
cannot tell what this machine raised for itself before there was a mesh from
what the mesh told it to have.

It is invisible until those two meet, which happens exactly once per mesh: on
the first node, after enrolment, the moment the control plane first speaks.

The report says what is not the answer, including the tempting one -- having
the control plane send the substrate back. It cannot: it was never told what
the bundle contained, and the bundle exists precisely because there was no
control plane to ask.
This commit is contained in:
2026-08-29 16:23:01 +02:00
parent 02afb7516b
commit 594ea10b07
@@ -0,0 +1,75 @@
---
status: open
opened: 2026-08-29
located-in: [mesh-host, mesh-control]
fixed-by:
amended-design:
---
# 010 — The first declaration a node receives destroys the substrate it raised
## Symptom
A first node was raised from its carried bundle: container runtime, store, two context databases,
their schemas, broker, control plane. Eleven resources, all running. It then enrolled against the
control plane on its own machine, held its link open, and was sent a declaration naming two
resources — a directory and a file.
Both were applied correctly. And **every container on the machine was removed**: the store, the
broker, and the control plane that had sent the declaration. The link died mid-sentence with
`the link closed: Exception (501) Reason: "EOF"`, because the broker carrying it had just been
torn down by the message it carried.
Afterwards `mesh-host owned` listed two resources. The mesh had deleted itself.
## What is actually wrong
Nothing in the code is behaving incorrectly. `apply` removes what the store holds and the incoming
declaration does not name, which is what reconciliation means — the declaration is the desired
state, not a patch, and anything else would make it impossible to remove a resource by omission.
**The fault is that the carried bundle and mesh declarations share one store.** The host cannot
tell "this machine raised this for itself before there was a mesh" from "the mesh told this
machine to have this", so the second overwrites the first completely.
That is invisible until the two meet, which happens exactly once per mesh: on the first node,
after enrolment, at the moment the control plane first speaks.
## Why it matters more than a footgun
**The first node is the only node where the substrate is not the mesh's doing.** Every other node
receives everything it runs from the control plane, so a complete declaration is complete by
construction. The first node raised its own substrate from a file it carried, and the control
plane has never been told about it — so the control plane cannot include it in a declaration even
if it wanted to.
So the first node is left in a state no other node is in, and the ordinary path destroys it.
## What is not the answer
- **Making the control plane send the substrate back.** It does not know what the bundle contained
and should not: the bundle exists precisely because there was no control plane yet.
- **Making apply stop removing orphans.** Removal by omission is how a declaration says *stop
running this*, and losing it costs the property that a node converges on what it was told rather
than accumulating.
- **Special-casing the first node.** [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)
is explicit that its specialness lasts two commands, and this would extend it for ever.
## The shape of an answer, not yet chosen
The store needs to record **where a resource came from** — carried, or declared — and reconcile
each against its own source. A declaration would then remove only what the mesh previously
declared, and the bundle would remain the bundle's business until the mesh is told about it.
That leaves a real question behind it: **what happens when the mesh eventually does declare the
substrate**, which it must, or the substrate can never be upgraded. Two sources claiming the same
container is the ambiguity this issue is made of, moved rather than removed.
## How it was found
In the lab, on a sealed machine, by doing the ordinary thing: raise a first node, enrol it, and
tell it something. It was not a test of this — it was the first end-to-end run of the link, and
this fell out of it.
The declaration was two lines and destroyed a working mesh in under a second, which is worth
holding on to: this is not an edge case reached by trying, it is the first thing that happens.