Order the records the way the system is learned
Jochen asked whether the order made sense. It did not -- it followed when things happened to be decided, which after consolidation is fictional anyway since record 5 alone folds decisions taken across a week. Concretely wrong before: the domain statement sat at 8, after five engineering rules; the constitution was scattered across 5, 12 and 17; the tiers landed at 15, 16, 21 and 22 with process records in between. Now it walks: what the mesh is (1-3), its tiers from the bottom up (4-8), what runs on them and how it gets there (9-10), how it is built (11-16), how it is checked (17-18), how we work (19-23). Two things made this safe rather than free. It is a permutation, not a compaction, so the renames go through temporary names -- otherwise two files want one slot and one is lost. And the reference rewrite is a single simultaneous pass, because almost every number moved into a slot another number was vacating; replacing one at a time would have cascaded and pointed things at the wrong record while still resolving. Verified: 284 [ADR NNNN](path) links across the repository, all with matching text and target. The ordering principle is now stated in 19 rather than left implicit -- the repository already said "the numbering is the flow" about its folders, and there was no reason for the records to be the exception.
This commit is contained in:
@@ -4,17 +4,17 @@ status: designed
|
||||
code: []
|
||||
updated: 2026-08-27
|
||||
decisions:
|
||||
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
|
||||
- 02-DECISIONS/0016-the-node-host.md
|
||||
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
|
||||
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
|
||||
- 02-DECISIONS/0016-the-node-host.md
|
||||
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
|
||||
- 02-DECISIONS/0016-the-node-host.md
|
||||
- 02-DECISIONS/0023-delivery.md
|
||||
- 02-DECISIONS/0016-the-node-host.md
|
||||
- 02-DECISIONS/0016-the-node-host.md
|
||||
- 02-DECISIONS/0016-the-node-host.md
|
||||
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
---
|
||||
|
||||
# The node lifecycle
|
||||
@@ -42,11 +42,11 @@ questions that were not being asked live.
|
||||
|
||||
**Only `enrolled` and `disconnected` are nodes**, and they are the same node in two situations
|
||||
rather than two kinds of thing
|
||||
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)). **`hosted` is not a
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). **`hosted` is not a
|
||||
node** — it is a machine with a program on it that has not been told which mesh it belongs to.
|
||||
|
||||
There is no state for *the first node*. That is the point of
|
||||
[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md): the first node walks the
|
||||
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md): the first node walks the
|
||||
same path, in an unusual order.
|
||||
|
||||
---
|
||||
@@ -54,7 +54,7 @@ same path, in an unusual order.
|
||||
## unmanaged → hosted: installing
|
||||
|
||||
In the machine's own idiom, because the package manager and the init file are the system's
|
||||
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)):
|
||||
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)):
|
||||
|
||||
```
|
||||
# Alpine — the intended first node
|
||||
@@ -67,7 +67,7 @@ systemctl enable --now nox-mesh-host
|
||||
```
|
||||
|
||||
Two lines each, and the init file behind them is four
|
||||
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)) — it says
|
||||
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)) — it says
|
||||
*run the launcher at boot* and nothing else, so a third system is transcription rather than a
|
||||
port.
|
||||
|
||||
@@ -103,7 +103,7 @@ WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
**Two lines of policy, and that is deliberate**
|
||||
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)). The init is
|
||||
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). The init is
|
||||
asked to *start this at boot* and *start it again if it exits*, and nothing else. Both are
|
||||
expressible in OpenRC, runit, s6 and an Android `init.rc`, so porting this file is transcription
|
||||
rather than design.
|
||||
@@ -135,7 +135,7 @@ nox-mesh-host enrol --token <one-time token>
|
||||
```
|
||||
|
||||
The token carries three things and is carried by a person
|
||||
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)): the broker's
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)): the broker's
|
||||
address, the fingerprint to expect, and the right to join once.
|
||||
|
||||
What happens, in order:
|
||||
@@ -153,7 +153,7 @@ a container runtime, an architecture. The profile is not a diagnostic; it is the
|
||||
|
||||
**The node computes nothing about the mesh.** It needs one peer to reach; the whole overlay is
|
||||
derived centrally and pushed down
|
||||
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md),
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md),
|
||||
[`08-connectivity.md`](08-connectivity.md)).
|
||||
|
||||
### The first declaration is the overlay, and nothing else
|
||||
@@ -172,7 +172,7 @@ Three reasons, and the third is the one that matters when something goes wrong:
|
||||
address and peer set are *assigned* — it generates a keypair, publishes the public half, and
|
||||
receives the rest ([`08-connectivity.md`](08-connectivity.md)). So the overlay is the first
|
||||
thing the mesh can give it, and it should be.
|
||||
- **It is what [ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) already
|
||||
- **It is what [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) already
|
||||
says:** *a joining node does the minimum to be reachable, and nothing else.*
|
||||
- **It is the way back in.** Once the overlay is up, the node is reachable over it — by SSH, by
|
||||
anything. If a later declaration breaks the machine, there is a route to it that does not
|
||||
@@ -187,10 +187,10 @@ Worth stating plainly, because the two rules read as a contradiction and are not
|
||||
| | |
|
||||
|---|---|
|
||||
| **every node reaches every other node** | over the overlay — SSH, services, ordinary traffic. This is the point of having one |
|
||||
| **every node consumes from the broker** | its own queue, over its own outbound connection ([ADR 0001](../../02-DECISIONS/0001-nodes-communicate-over-a-broker.md)) |
|
||||
| **nothing dials a node to control it** | the host has no inbound control surface ([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)) |
|
||||
| **every node consumes from the broker** | its own queue, over its own outbound connection ([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)) |
|
||||
| **nothing dials a node to control it** | the host has no inbound control surface ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) |
|
||||
|
||||
**[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) is about the control
|
||||
**[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) is about the control
|
||||
channel, not about network reachability.** What it forbids is a listening thing that accepts
|
||||
instructions and changes the machine. A node being reachable on the overlay — the whole purpose
|
||||
of the overlay — is untouched by it, and so is a person opening a shell on it.
|
||||
@@ -232,7 +232,7 @@ used months later on node two.
|
||||
## Two kinds of host
|
||||
|
||||
Everything above assumes a machine with an init that runs the host at boot. Not every machine
|
||||
has one ([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)).
|
||||
has one ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)).
|
||||
|
||||
| | **resident** | **episodic** |
|
||||
|---|---|---|
|
||||
@@ -245,7 +245,7 @@ has one ([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)).
|
||||
| can be the first node | yes | **no** |
|
||||
|
||||
**An episodic host being killed is disconnection, not failure.** That is
|
||||
[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) doing the work it was written
|
||||
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) doing the work it was written
|
||||
for: reachability is state, not class. Everything the design already does for a laptop that
|
||||
closes — an authoritative local store, reconcile on start, *last heard from* reported without an
|
||||
alarm — is what an episodic host needs, at a shorter period.
|
||||
@@ -258,7 +258,7 @@ empty placeholder waiting to be filled in.
|
||||
**Two things this changes for anything reading the mesh.** *Last heard from* is a much weaker
|
||||
signal on an episodic host — a healthy phone looks like a dead server — so a reader has to know
|
||||
which kind it is looking at. And a declaration may take a long time to land, which makes
|
||||
[ADR 0023](../../02-DECISIONS/0023-delivery.md)'s separation of
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md)'s separation of
|
||||
*outstanding* from *failed* load-bearing rather than tidy.
|
||||
|
||||
**Still open:** how an episodic host is started in practice — an APK with a foreground service,
|
||||
@@ -272,7 +272,7 @@ Adoption is not a state. It is what the **first apply** does when it is told to
|
||||
machine already has ([research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md)).
|
||||
|
||||
A candidate machine is not empty. It has a package manager, probably a container runtime,
|
||||
configuration somebody chose. [ADR 0016](../../02-DECISIONS/0016-the-node-host.md)
|
||||
configuration somebody chose. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
|
||||
says the host never touches what it did not create — adoption is the deliberate act of taking
|
||||
ownership of exactly that, so it is a companion to that rule rather than an exception:
|
||||
|
||||
@@ -298,8 +298,8 @@ outcome **derived** from the worst line rather than stated alongside it.
|
||||
|
||||
**Changes are pushed, not polled.** A declaration arrives as a message on the link and the host
|
||||
applies it then. The link is already open and outbound
|
||||
([ADR 0001](../../02-DECISIONS/0001-nodes-communicate-over-a-broker.md),
|
||||
[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)) — asking it
|
||||
([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md),
|
||||
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) — asking it
|
||||
repeatedly whether anything has changed would be slower to land *and* constant traffic to learn
|
||||
nothing.
|
||||
|
||||
@@ -326,11 +326,11 @@ So the two periodic things do different jobs and should not be conflated:
|
||||
without a heartbeat that is indistinguishable from a node that stopped. With one, *last heard
|
||||
from* is a fact beside every node — which is what
|
||||
[how long disconnected](#how-long-disconnected-and-who-is-told) reports and what
|
||||
[ADR 0016](../../02-DECISIONS/0016-the-node-host.md) exists
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) exists
|
||||
because a stuck node cannot send.
|
||||
|
||||
**Rebooting mid-apply is safe by construction.** The store records each resource *after* it
|
||||
worked ([ADR 0014](../../02-DECISIONS/0014-a-picture-is-read-from-what-runs.md)), so a host that
|
||||
worked ([ADR 0018](../../02-DECISIONS/0018-a-picture-is-read-from-what-runs.md)), so a host that
|
||||
dies half way through comes back, finds the completed ones already matching, and applies the
|
||||
rest. The rule that exists to stop the host lying about what it did also makes it crash-safe.
|
||||
|
||||
@@ -357,7 +357,7 @@ runtime because a declaration changed would stop every container on the node.
|
||||
## enrolled ⇄ disconnected
|
||||
|
||||
Not a failure. Not degraded. A situation
|
||||
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)).
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
|
||||
|
||||
A disconnected node **keeps reconciling against its own store**, so it goes on holding its
|
||||
machine in the last state it was told to hold. A laptop shut for a week comes back and
|
||||
@@ -365,7 +365,7 @@ reconciles; it does not come back and ask what it is.
|
||||
|
||||
What it cannot do: receive new declarations, be granted anything new, or have its certificates
|
||||
renewed — which is the clock on the whole arrangement
|
||||
([ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)).
|
||||
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)).
|
||||
|
||||
**How long it has been disconnected is a fact the mesh must hold**, and nothing holds it today.
|
||||
Without it, a node running last month's assignments looks exactly like one that is current.
|
||||
@@ -384,7 +384,7 @@ nox-mesh-host profile # what can this machine actually do?
|
||||
|
||||
`apply FILE` accepts actions, because someone who can write that file and run this binary as
|
||||
root can already do anything it can. The bound in
|
||||
[ADR 0016](../../02-DECISIONS/0016-the-node-host.md) is on what a
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) is on what a
|
||||
**remote** party may push, not on what a person at the machine may do.
|
||||
|
||||
This replaces the three hand-run scripts that exist today — first node, joining, rescue — with
|
||||
@@ -401,14 +401,14 @@ what it owns by the table above, reports, and drops its identity. The machine ke
|
||||
installed and is back to `hosted`. Nothing is left behind that anybody has to remember.
|
||||
|
||||
**The node is gone.** Stolen, dead, or simply unreachable. The mesh cannot tell it anything, and
|
||||
by [ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) it will go on reconciling
|
||||
by [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) it will go on reconciling
|
||||
its last declaration **forever**.
|
||||
|
||||
That is the honest consequence of making disconnection ordinary, and the answer is not to make
|
||||
the host expire. It is that **the node holds nothing that outlives revocation**: its identity is
|
||||
its own, and every grant it holds is a per-node credential at the provider
|
||||
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md),
|
||||
[ADR 0020](../../02-DECISIONS/0020-a-context-owns-its-store.md)). Revoking is done at the
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md),
|
||||
[ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md)). Revoking is done at the
|
||||
database, the broker, the object store — not on the machine.
|
||||
|
||||
So a lost node keeps *running* and stops being able to *reach* anything. That is the best
|
||||
@@ -438,7 +438,7 @@ remains locally authoritative for *operating*; the copy exists only for this.
|
||||
## Upgrading the host
|
||||
|
||||
The host is delivered like anything else
|
||||
([ADR 0023](../../02-DECISIONS/0023-delivery.md)), and this is worth
|
||||
([ADR 0010](../../02-DECISIONS/0010-delivery.md)), and this is worth
|
||||
walking through because tier 0 looks like it should be special and is not.
|
||||
|
||||
```
|
||||
@@ -471,7 +471,7 @@ the test of whether this is really uniform.
|
||||
3 it finishes the apply and reports never mid-way
|
||||
4 it exits 0 having finished, not having been stopped
|
||||
5 the launcher starts it again on the new binary — it supervises the host
|
||||
rather than exec'ing it (ADR 0016), so this
|
||||
rather than exec'ing it (ADR 0005), so this
|
||||
needs nothing from the init
|
||||
6 the new host reconciles on start trigger 1, confirming the machine still matches
|
||||
```
|
||||
@@ -489,7 +489,7 @@ own apply completes. A node must therefore report the version it is **running**,
|
||||
installed — otherwise the mesh believes an upgrade landed at step 1.
|
||||
|
||||
**A version that crashes on start rolls itself back**
|
||||
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)).
|
||||
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)).
|
||||
|
||||
What the init starts is not the host but a **launcher**, and the launcher is where the policy
|
||||
lives:
|
||||
@@ -551,7 +551,7 @@ credentials still valid — the case
|
||||
**The host reports what it owns, and the mesh keeps the last report.**
|
||||
|
||||
The store stays locally authoritative — a node operates from its own copy and needs nothing to
|
||||
do so ([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)). What changes is that
|
||||
do so ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). What changes is that
|
||||
the mesh holds a **copy for recovery**, refreshed on every apply report.
|
||||
|
||||
So a node that loses its state file re-enrols, receives both the declaration *and* the record of
|
||||
@@ -616,12 +616,12 @@ keeps cataloguing.
|
||||
### Where the enrolment token comes from
|
||||
|
||||
**`mesh-control token issue` prints it once**, to the person running it. Single-use, and it
|
||||
expires whether used or not ([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)).
|
||||
expires whether used or not ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
|
||||
|
||||
It is carried by hand — read off a screen, pasted into a terminal. That is the design rather than
|
||||
a gap in it: its authenticity comes from the channel it travelled, which is what
|
||||
lets a node verify a mesh it has never spoken to
|
||||
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)). A token emailed,
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). A token emailed,
|
||||
committed, or dropped in shared storage has lost the only property that makes it worth carrying.
|
||||
|
||||
**On the first node it comes from the control plane that was raised two commands ago**, which is
|
||||
@@ -632,10 +632,10 @@ the same command against a mesh that is one machine old.
|
||||
## Still open
|
||||
|
||||
- ~~**Automatic rollback of a bad host version.**~~ **Resolved** by
|
||||
[ADR 0016](../../02-DECISIONS/0016-the-node-host.md): a launcher
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md): a launcher
|
||||
counts failed starts and rolls back — shipped by the package, not the host binary, because a
|
||||
binary that will not start cannot recover itself. It rolls back once; a second failure means
|
||||
the machine is the problem, not the binary.
|
||||
- **How a previous declaration is retained and chosen**, which is what rollback of anything else
|
||||
would use ([ADR 0023](../../02-DECISIONS/0023-delivery.md)).
|
||||
would use ([ADR 0010](../../02-DECISIONS/0010-delivery.md)).
|
||||
- **A node returning after months** applies a very large jump in one go. Correct, and untested.
|
||||
|
||||
Reference in New Issue
Block a user