|
|
|
@@ -0,0 +1,310 @@
|
|
|
|
|
---
|
|
|
|
|
layer: to-be
|
|
|
|
|
status: designed
|
|
|
|
|
code: []
|
|
|
|
|
updated: 2026-08-27
|
|
|
|
|
decisions:
|
|
|
|
|
- 02-DECISIONS/0036-a-node-is-a-managed-machine.md
|
|
|
|
|
- 02-DECISIONS/0037-the-host-applies-it-does-not-decide.md
|
|
|
|
|
- 02-DECISIONS/0038-a-node-joins-by-linking-first.md
|
|
|
|
|
- 02-DECISIONS/0039-the-link-is-the-security-boundary.md
|
|
|
|
|
- 02-DECISIONS/0043-a-declaration-is-an-ordered-list-of-owned-resources.md
|
|
|
|
|
- 02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
# The node lifecycle
|
|
|
|
|
|
|
|
|
|
How a Linux machine becomes a node, stays one, and stops being one.
|
|
|
|
|
|
|
|
|
|
[`05-the-node-host.md`](05-the-node-host.md) describes the host as a component. This describes
|
|
|
|
|
it as something that runs for years on a machine somebody else also uses — which is where the
|
|
|
|
|
questions that were not being asked live.
|
|
|
|
|
|
|
|
|
|
## The states
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
unmanaged ──install──► hosted ──enrol──► enrolled ⇄ disconnected
|
|
|
|
|
▲ │
|
|
|
|
|
└─────release──────┘
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
| State | Has | Can |
|
|
|
|
|
|---|---|---|
|
|
|
|
|
| **unmanaged** | nothing of ours | — it is a Linux machine |
|
|
|
|
|
| **hosted** | the host, no identity | apply a local file, apply its bundle |
|
|
|
|
|
| **enrolled** | identity, link, store | everything; this is *a node* |
|
|
|
|
|
| **disconnected** | identity, store, no link | hold its machine in the last state it was told |
|
|
|
|
|
|
|
|
|
|
**Only `enrolled` and `disconnected` are nodes**, and they are the same node in two situations
|
|
|
|
|
rather than two kinds of thing
|
|
|
|
|
([ADR 0036](../../02-DECISIONS/0036-a-node-is-a-managed-machine.md)). **`hosted` is not a
|
|
|
|
|
node** — it is a machine with a program on it that has not been told which mesh it belongs to.
|
|
|
|
|
|
|
|
|
|
There is no state for *the first node*. That is the point of
|
|
|
|
|
[ADR 0038](../../02-DECISIONS/0038-a-node-joins-by-linking-first.md): the first node walks the
|
|
|
|
|
same path, in an unusual order.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## unmanaged → hosted: installing
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
pacman -S nox-mesh-host
|
|
|
|
|
systemctl enable --now nox-mesh-host
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Or, where there is no repository to install from:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
curl -fsSL https://<release>/mesh-host-<version>-x86_64.tar.gz | tar -xz -C /usr/local/bin
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
**The tarball must never acquire a dependency**, because the mesh's own package repository is
|
|
|
|
|
hosted on the mesh. Any route that needs the mesh in order to install the thing that joins the
|
|
|
|
|
mesh is a circle — unusable on a first node, and unusable by whoever is repairing a mesh that is
|
|
|
|
|
down, which is exactly when it is wanted.
|
|
|
|
|
|
|
|
|
|
At this point the host is running and **doing nothing**. It has no identity, so there is nobody
|
|
|
|
|
to link to and nothing to apply. It answers `profile`, `inventory` and `version`, and waits.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## hosted → enrolled: the ordinary case
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
nox-mesh-host enrol --token <one-time token>
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The token carries three things and is carried by a person
|
|
|
|
|
([ADR 0051](../../02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md)): the broker's
|
|
|
|
|
address, the fingerprint to expect, and the right to join once.
|
|
|
|
|
|
|
|
|
|
What happens, in order:
|
|
|
|
|
|
|
|
|
|
1. the host dials the broker at the address in the token, **over the underlay**;
|
|
|
|
|
2. it checks the broker's certificate against the pinned fingerprint — *before* sending anything;
|
|
|
|
|
3. it presents the one-time secret and receives its **own durable identity**;
|
|
|
|
|
4. it reports its `profile` and `inventory` upward;
|
|
|
|
|
5. the control plane decides what this machine should be, and sends a declaration;
|
|
|
|
|
6. the host applies it, reads back, and reports.
|
|
|
|
|
|
|
|
|
|
**Step 4 is the one that is easy to miss and is what makes step 5 possible.** The control plane
|
|
|
|
|
cannot decide what a machine should run without knowing what it *can* run — a graphical session,
|
|
|
|
|
a container runtime, an architecture. The profile is not a diagnostic; it is the input.
|
|
|
|
|
|
|
|
|
|
**The node computes nothing about the mesh.** It needs one peer to reach; the whole overlay is
|
|
|
|
|
derived centrally and pushed down
|
|
|
|
|
([ADR 0038](../../02-DECISIONS/0038-a-node-joins-by-linking-first.md),
|
|
|
|
|
[`08-connectivity.md`](08-connectivity.md)).
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## hosted → enrolled: the first node
|
|
|
|
|
|
|
|
|
|
The same path, with the mesh built in the middle of it.
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
# 1 — raise the substrate and the control plane from the carried bundle
|
|
|
|
|
nox-mesh-host reconcile
|
|
|
|
|
|
|
|
|
|
# 2 — the control plane now exists, and issues the first token
|
|
|
|
|
mesh-control token issue
|
|
|
|
|
|
|
|
|
|
# 3 — the machine joins the mesh it just raised
|
|
|
|
|
nox-mesh-host enrol --token <token>
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Step 1 is the bootstrap from [`07-the-substrate.md`](07-the-substrate.md): Docker, then
|
|
|
|
|
PostgreSQL, then the database, then the schema, then the control plane. It needs no identity
|
|
|
|
|
because nothing is being asked of anyone — the host is applying a declaration it already
|
|
|
|
|
carries, to the machine it is already on.
|
|
|
|
|
|
|
|
|
|
**After step 3 the first node is not special in any way**, which is the property `adopt.sh` and
|
|
|
|
|
the bootstrap script never had. Its specialness lasted two commands.
|
|
|
|
|
|
|
|
|
|
**And enrolment is exercised on node one.** The path every other node depends on is walked
|
|
|
|
|
immediately, against a control plane on the same machine, rather than being written and first
|
|
|
|
|
used months later on node two.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## Adoption: what happens to what is already there
|
|
|
|
|
|
|
|
|
|
Adoption is not a state. It is what the **first apply** does when it is told to own something a
|
|
|
|
|
machine already has ([research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md)).
|
|
|
|
|
|
|
|
|
|
A candidate machine is not empty. It has a package manager, probably a container runtime,
|
|
|
|
|
configuration somebody chose. [ADR 0043](../../02-DECISIONS/0043-a-declaration-is-an-ordered-list-of-owned-resources.md)
|
|
|
|
|
says the host never touches what it did not create — adoption is the deliberate act of taking
|
|
|
|
|
ownership of exactly that, so it is a companion to that rule rather than an exception:
|
|
|
|
|
|
|
|
|
|
> *never, unless adoption made it the host's* — with adoption **explicit, recorded, and visible
|
|
|
|
|
> in what the host says it owns.**
|
|
|
|
|
|
|
|
|
|
Three rules, all earned:
|
|
|
|
|
|
|
|
|
|
**The original is kept before anything is written.** A one-way door on a working machine is not
|
|
|
|
|
an installation. This is a *never* rule, and it earns that from the worst loss in this record —
|
|
|
|
|
a tool acting on a path it did not own.
|
|
|
|
|
|
|
|
|
|
**On conflict, the machine's configuration wins.** Adoption always completes; the conflict is
|
|
|
|
|
flagged and reconciled afterwards. A machine in use keeps working exactly as it did.
|
|
|
|
|
|
|
|
|
|
**Adoption produces a briefing**, not just a result: what it found, what it took over, and what
|
|
|
|
|
it could not resolve — with each line marked `ok`, `kept`, `unknown` or `failed`, and the overall
|
|
|
|
|
outcome **derived** from the worst line rather than stated alongside it.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## enrolled: what running actually looks like
|
|
|
|
|
|
|
|
|
|
Four reconcile triggers
|
|
|
|
|
([ADR 0057](../../02-DECISIONS/0057-the-host-is-a-root-service-installed-as-a-package.md)):
|
|
|
|
|
|
|
|
|
|
| | |
|
|
|
|
|
|---|---|
|
|
|
|
|
| **start** | the machine may have changed while nothing was running |
|
|
|
|
|
| **a declaration arrives** | the ordinary path |
|
|
|
|
|
| **every ten minutes** | drift — something other than the host changed the machine |
|
|
|
|
|
| **reconnect** | declarations may have been missed |
|
|
|
|
|
|
|
|
|
|
**Rebooting mid-apply is safe, and it is safe by construction.** The store records each resource
|
|
|
|
|
*after* it worked ([ADR 0035](../../02-DECISIONS/0035-a-picture-is-read-from-what-runs.md)), so
|
|
|
|
|
a host that dies half way through comes back, finds the completed ones already matching, and
|
|
|
|
|
applies the rest. The rule that exists to stop the host lying about what it did also makes it
|
|
|
|
|
crash-safe.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## Updating what the node holds
|
|
|
|
|
|
|
|
|
|
An ordinary declaration. Someone assigns a module; the control plane recomputes what that node
|
|
|
|
|
should be and sends it; the host applies the difference and removes what is no longer declared.
|
|
|
|
|
|
|
|
|
|
**Removal is not symmetric, and the asymmetry is the design:**
|
|
|
|
|
|
|
|
|
|
| | on being undeclared |
|
|
|
|
|
|---|---|
|
|
|
|
|
| file, directory | **removed** |
|
|
|
|
|
| container | **removed** — the host created it |
|
|
|
|
|
| service | **stopped**; the unit file is not the host's to delete |
|
|
|
|
|
| package | **left installed** — *forgotten*, not removed |
|
|
|
|
|
| action | **forgotten** — it left nothing the host owns |
|
|
|
|
|
|
|
|
|
|
The host removes what it *made* and leaves what it merely *configured*. Uninstalling a container
|
|
|
|
|
runtime because a declaration changed would stop every container on the node.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## Updating the host itself
|
|
|
|
|
|
|
|
|
|
A declaration too — `package: nox-mesh-host`, at a version. The package manager writes the new
|
|
|
|
|
binary; the running process is undisturbed, because Unix keeps the running executable's inode.
|
|
|
|
|
|
|
|
|
|
**Then the host exits, and the supervisor restarts it on the new binary.** After the apply
|
|
|
|
|
completes, never during it; only when the executable actually changed; exit zero, so a restart
|
|
|
|
|
is what happens next rather than a failure a supervisor backs off from.
|
|
|
|
|
|
|
|
|
|
**The host never asks the service manager to restart it.** That is the host killing itself
|
|
|
|
|
part-way through an apply. It stops by finishing.
|
|
|
|
|
|
|
|
|
|
**A version-skewed fleet is normal**, because each node restarts when its own apply finishes.
|
|
|
|
|
What a node reports must be the **running** version, not the installed one, or the mesh will
|
|
|
|
|
believe an upgrade landed before it took effect.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## enrolled ⇄ disconnected
|
|
|
|
|
|
|
|
|
|
Not a failure. Not degraded. A situation
|
|
|
|
|
([ADR 0036](../../02-DECISIONS/0036-a-node-is-a-managed-machine.md)).
|
|
|
|
|
|
|
|
|
|
A disconnected node **keeps reconciling against its own store**, so it goes on holding its
|
|
|
|
|
machine in the last state it was told to hold. A laptop shut for a week comes back and
|
|
|
|
|
reconciles; it does not come back and ask what it is.
|
|
|
|
|
|
|
|
|
|
What it cannot do: receive new declarations, be granted anything new, or have its certificates
|
|
|
|
|
renewed — which is the clock on the whole arrangement
|
|
|
|
|
([ADR 0053](../../02-DECISIONS/0053-one-control-plane-and-no-failover.md)).
|
|
|
|
|
|
|
|
|
|
**How long it has been disconnected is a fact the mesh must hold**, and nothing holds it today.
|
|
|
|
|
Without it, a node running last month's assignments looks exactly like one that is current.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## Rescue
|
|
|
|
|
|
|
|
|
|
The host is still a command-line tool, and that is what rescue is:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
nox-mesh-host owned # what do you think you own?
|
|
|
|
|
nox-mesh-host apply repair.json # apply something by hand, locally
|
|
|
|
|
nox-mesh-host profile # what can this machine actually do?
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
`apply FILE` accepts actions, because someone who can write that file and run this binary as
|
|
|
|
|
root can already do anything it can. The bound in
|
|
|
|
|
[ADR 0047](../../02-DECISIONS/0047-the-bundle-may-carry-actions-the-link-may-not.md) is on what a
|
|
|
|
|
**remote** party may push, not on what a person at the machine may do.
|
|
|
|
|
|
|
|
|
|
This replaces the three hand-run scripts that exist today — first node, joining, rescue — with
|
|
|
|
|
one binary that has always been the same binary.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## enrolled → hosted: retiring a node
|
|
|
|
|
|
|
|
|
|
Two cases, and they are genuinely different.
|
|
|
|
|
|
|
|
|
|
**Graceful.** The control plane sends a final declaration that names nothing. The host removes
|
|
|
|
|
what it owns by the table above, reports, and drops its identity. The machine keeps the host
|
|
|
|
|
installed and is back to `hosted`. Nothing is left behind that anybody has to remember.
|
|
|
|
|
|
|
|
|
|
**The node is gone.** Stolen, dead, or simply unreachable. The mesh cannot tell it anything, and
|
|
|
|
|
by [ADR 0036](../../02-DECISIONS/0036-a-node-is-a-managed-machine.md) it will go on reconciling
|
|
|
|
|
its last declaration **forever**.
|
|
|
|
|
|
|
|
|
|
That is the honest consequence of making disconnection ordinary, and the answer is not to make
|
|
|
|
|
the host expire. It is that **the node holds nothing that outlives revocation**: its identity is
|
|
|
|
|
its own, and every grant it holds is a per-node credential at the provider
|
|
|
|
|
([ADR 0039](../../02-DECISIONS/0039-the-link-is-the-security-boundary.md),
|
|
|
|
|
[ADR 0045](../../02-DECISIONS/0045-a-context-owns-its-store.md)). Revoking is done at the
|
|
|
|
|
database, the broker, the object store — not on the machine.
|
|
|
|
|
|
|
|
|
|
So a lost node keeps *running* and stops being able to *reach* anything. That is the best
|
|
|
|
|
available outcome and it is worth stating plainly rather than implying the mesh can reach out and
|
|
|
|
|
switch a machine off, which it cannot and should not be able to.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## Losing the store
|
|
|
|
|
|
|
|
|
|
Worth its own section because the failure is quiet.
|
|
|
|
|
|
|
|
|
|
If `/var/lib/mesh-host/state.json` is lost — a reinstall, a replaced disk — the host loses
|
|
|
|
|
**its record of what it owns**, not its ability to work. It re-enrols, receives the declaration
|
|
|
|
|
again, and re-applies it.
|
|
|
|
|
|
|
|
|
|
**What does not come back is removal.** Resources it applied under an older declaration and no
|
|
|
|
|
longer holds a record of become unowned: the host will not touch them, because it never touches
|
|
|
|
|
what it did not create. They sit there, unmanaged, indefinitely.
|
|
|
|
|
|
|
|
|
|
The store is therefore the one piece of node state that matters, and *how it is protected* is
|
|
|
|
|
not designed.
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## Open
|
|
|
|
|
|
|
|
|
|
- **Re-enrolling as the same node.** A machine that lost its identity gets a new token — but
|
|
|
|
|
whether the mesh treats it as the same node or a new one is an operator's decision today, and
|
|
|
|
|
nothing supports either.
|
|
|
|
|
- **Protecting the store.** Above. Its loss is silent and permanent.
|
|
|
|
|
- **How long disconnected, and who is told.** The fact is not held anywhere.
|
|
|
|
|
- **Whether a `failed` adoption line still lets adoption complete.** Reopened by
|
|
|
|
|
[research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) and not decided:
|
|
|
|
|
*flags inform* was decided about conflicts, and a failure is different in kind.
|
|
|
|
|
- **What a briefing looks like.** It is the first thing a session on a new node sees, which
|
|
|
|
|
makes it an interface rather than a log.
|
|
|
|
|
- **Where the enrolment token comes from, operationally.** A person carries it. Nothing says how
|
|
|
|
|
it is generated, shown, or transported, and it is now the only secret in adoption.
|