Approve 0057-0059, with four corrections from review

Not approved as drafted -- four things came out of checking them against each
other, and one was a bug that would have broken every upgrade.

The bug: 0059 specified Restart=on-failure while 0057 has the host restart onto
a new binary by exiting CLEANLY. on-failure does not restart a process that
exited zero, so every upgraded node would have been left stopped, having
successfully upgraded. Found by reading the two records against each other
rather than by either alone. Now Restart=always in all three places that
mention it.

The host cannot run in a container, and the reason is decisive rather than
stylistic: step 0 of the substrate bootstrap installs the container runtime, so
a host inside a container would need the thing it exists to install. It would
also break 0041 -- copy it onto a machine and run it stops being true when the
machine must already have a runtime. Everything above tier 0 is a container;
the host is not. That split is the tier boundary, not an inconsistency.

systemd is named rather than abstracted. An init is not a dependency in 0041's
sense: 0041 is about what must be installed before the host works, and an init
is not installed, it is what the machine already is. The unit file is the only
systemd-specific artefact and it belongs to the package, so a machine with a
different supervisor ships a different package.

The mesh is a watchdog, and my first draft was half an answer. Recovery must be
local -- nothing dials a node, and a host that cannot start cannot report. But
detection is the mesh's, and a local supervisor structurally cannot do it: it
sees one process failing and cannot tell a broken machine from a broken
release. Only something watching every node can, and that distinction decides
whether the response is "fix this machine" or "stop shipping this version". So
a host rollout is staged -- a few nodes, wait for heartbeats, continue or stop
on silence. Local rollback still needed, because the canary nodes break and
because a node offline during the rollout gets the declaration later with no
batch around it.

The first declaration is the overlay and nothing else. Forced, because a node's
address and peers are assigned rather than chosen. But also the way back in: a
node reachable over the overlay can be fixed by hand if a later declaration
breaks it, and a large first declaration risks a node that is broken and
unreachable at once.

Also stated plainly, because it reads as a contradiction: nodes reach each
other over the overlay and every node consumes from the broker; what 0039
forbids is an inbound CONTROL surface, not reachability.

And in 06: no node holds a credential to any control-plane store, for reads or
writes. Four ADRs already say this separately and none of them said it in one
place. Nodes state over the broker; the owning context writes. With a note that
most high-frequency writes are observability's, not the registry's -- routing
logs into the registry would be the shared-schema mistake arriving through a
door marked performance.
This commit is contained in:
2026-08-27 22:12:12 +02:00
parent 605c9fd441
commit 19997d56c3
6 changed files with 218 additions and 22 deletions
@@ -10,6 +10,9 @@ decisions:
- 02-DECISIONS/0039-the-link-is-the-security-boundary.md
- 02-DECISIONS/0043-a-declaration-is-an-ordered-list-of-owned-resources.md
- 02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md
- 02-DECISIONS/0057-the-host-is-a-root-service-installed-as-a-package.md
- 02-DECISIONS/0058-delivery-ends-in-a-declaration.md
- 02-DECISIONS/0059-a-host-that-cannot-start-rolls-itself-back.md
---
# The node lifecycle
@@ -97,6 +100,48 @@ derived centrally and pushed down
([ADR 0038](../../02-DECISIONS/0038-a-node-joins-by-linking-first.md),
[`08-connectivity.md`](08-connectivity.md)).
### The first declaration is the overlay, and nothing else
**The mesh makes a node reachable before it makes it useful.** Step 5 is not one declaration
carrying everything the node will ever run. It is two, in order:
```
first the overlay — this node's address, its keys, its peers, its names
then everything else — packages, containers, services, files
```
Three reasons, and the third is the one that matters when something goes wrong:
- **It is forced.** A node cannot join the overlay before contacting the mesh, because its
address and peer set are *assigned* — it generates a keypair, publishes the public half, and
receives the rest ([`08-connectivity.md`](08-connectivity.md)). So the overlay is the first
thing the mesh can give it, and it should be.
- **It is what [ADR 0038](../../02-DECISIONS/0038-a-node-joins-by-linking-first.md) already
says:** *a joining node does the minimum to be reachable, and nothing else.*
- **It is the way back in.** Once the overlay is up, the node is reachable over it — by SSH, by
anything. If a later declaration breaks the machine, there is a route to it that does not
depend on the mesh's control path working. **Sending a large first declaration risks a node
that is broken and unreachable at the same time**, and those two failures are much worse
together than separately.
### Reachable is not the same as having a control surface
Worth stating plainly, because the two rules read as a contradiction and are not.
| | |
|---|---|
| **every node reaches every other node** | over the overlay — SSH, services, ordinary traffic. This is the point of having one |
| **every node consumes from the broker** | its own queue, over its own outbound connection ([ADR 0001](../../02-DECISIONS/0001-nodes-communicate-over-a-broker.md)) |
| **nothing dials a node to control it** | the host has no inbound control surface ([ADR 0039](../../02-DECISIONS/0039-the-link-is-the-security-boundary.md)) |
**[ADR 0039](../../02-DECISIONS/0039-the-link-is-the-security-boundary.md) is about the control
channel, not about network reachability.** What it forbids is a listening thing that accepts
instructions and changes the machine. A node being reachable on the overlay — the whole purpose
of the overlay — is untouched by it, and so is a person opening a shell on it.
The distinction is *who can tell this machine what to be*: only the control plane, only over the
link the node opened, only in declarations of known shape.
---
## hosted → enrolled: the first node