Designed with no reference to what came before, which was asked for. The system this replaces has features — several deployable units inside one module — and they are deliberately absent. That closes something ADR 0001 has been carrying as an open prerequisite. It lists "named features with per-node opt-in" as required, or "every independently deployable unit becomes a module again and the count returns". The premise was right and the remedy already exists in another form: several modules, assignment per node, and a module with requirements and no files of its own. `networking` is exactly that. The count does not return because what made it return — a module is expensive, so put several things in one — is gone. A module here is a manifest and usually nothing else. The manifest in a repository names artifacts; the manifest the mesh holds names digests. Two documents, because a digest is not knowable until something is built and a repository carrying one is wrong the moment anybody edits anything. The builder runs on a node. Building needs a container runtime and a working tree, and what the control plane may send a machine is bounded by the declaration language. A control plane holding a container socket would be the one component that can do anything anywhere. And the host's vocabulary grew from six shapes to eight — user and archive — with the reasoning for each and for the refusals that came with them. The count is asserted by a test precisely because every addition widens what a compromised control plane can express.
285 lines
14 KiB
Markdown
285 lines
14 KiB
Markdown
---
|
|
layer: to-be
|
|
status: in-progress
|
|
code: [mesh-host]
|
|
updated: 2026-08-30
|
|
decisions:
|
|
- 02-DECISIONS/0019-how-this-repository-works.md
|
|
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
|
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
|
- 02-DECISIONS/0010-delivery.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
---
|
|
|
|
# The node host
|
|
|
|
Tier 0. The one thing ever installed by hand, and the only thing that changes a machine.
|
|
|
|
## What it is
|
|
|
|
A **statically linked binary that requires nothing to be present** — copy it onto a machine and
|
|
run it, and that is the whole installation
|
|
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). Written in Go, because the
|
|
job is system-level and because the host shares no code with any other tier.
|
|
|
|
A single binary with one job: **apply declared state on this machine**
|
|
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). Overlay
|
|
membership, packet filtering, packages, services, containers and filesystems are not six
|
|
concerns it carries; they are six instances of the one.
|
|
|
|
It replaces three things that exist today
|
|
([`00-as-is/05`](../00-as-is/05-runtime-and-installation.md)): the three hand-run bootstrap
|
|
scripts — first node, joining, rescue — and the synchronisers that rewrite managed files
|
|
([`00-as-is/06`](../00-as-is/06-configuration-and-secrets.md)).
|
|
|
|
**What it is not:** it does not decide anything that needs another node, it never queries the
|
|
mesh database, and it has no listening surface.
|
|
|
|
## The parts
|
|
|
|
| Part | Owns |
|
|
|---|---|
|
|
| `apply` | reconciling declared state on this machine |
|
|
| `store` | local state, authoritative while disconnected |
|
|
| `link` | the single outbound connection to the control plane |
|
|
| `profile` | what this machine can be asked to do |
|
|
| `inventory` | what this machine is and has |
|
|
| `substrate.lock` | the pinned tier-1 descriptor, appliable with no mesh present |
|
|
|
|
### apply
|
|
|
|
Takes a **declaration** and makes the machine match it. Idempotent: applying the same
|
|
declaration twice changes nothing the second time, and applying it to a drifted machine
|
|
returns it.
|
|
|
|
Three properties, each following a recorded decision:
|
|
|
|
**A failed step fails the apply.** Not "logs and continues"
|
|
([ADR 0010](../../02-DECISIONS/0010-delivery.md)). A partial apply that
|
|
reports success is the mesh's most expensive shape.
|
|
|
|
**Each applier reads back.** Setting a value is not evidence the value took. The firewall is
|
|
asked whether the rule loaded; conntrack is asked what timeout it holds. This is `how-we-build`
|
|
§5 as a component requirement rather than a review habit.
|
|
|
|
**What was applied is recorded after it works, never before**
|
|
([ADR 0018](../../02-DECISIONS/0018-a-picture-is-read-from-what-runs.md)). A failed apply leaves
|
|
the machine in whatever state it reached, and nothing must claim otherwise.
|
|
|
|
### store
|
|
|
|
Local, and **authoritative while disconnected**. Not a cache of the control plane — the record
|
|
of what this node has applied and what it currently holds.
|
|
|
|
This is structural rather than convenient: if disconnection is an ordinary situation rather
|
|
than an exception ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)), the
|
|
store is what makes it ordinary. A laptop shut for a week comes back and reconciles; it does
|
|
not come back and ask what it is.
|
|
|
|
### link
|
|
|
|
The node's one connection to the control plane, and its security boundary
|
|
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
|
|
|
|
It is the broker connection that already exists
|
|
([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)) — outbound,
|
|
node-initiated, per-node addressed — carrying **per-node identity instead of a shared
|
|
credential**. The node owns no password. It owns an identity, and that identity is what it
|
|
presents.
|
|
|
|
What arrives is bounded by form: **declarations of known shape, never a command to run.**
|
|
|
|
**It must fail legibly.** A node that cannot link says which side refused and on what grounds.
|
|
A boundary that refuses without saying why is worse than the password it replaced.
|
|
|
|
### profile
|
|
|
|
What this machine can be asked to do — a graphical session, a container runtime, an
|
|
architecture, a network position.
|
|
|
|
**Detected, never assumed.** A package being installed does not mean a capability is present
|
|
([04-ISSUES/007](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md)): a
|
|
capability is real when it is present, running and working, and the difference is the whole
|
|
point of detecting it.
|
|
|
|
The profile is what makes [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)
|
|
work: a node is a node, and what varies between them is here rather than in the definition.
|
|
|
|
### inventory
|
|
|
|
What this machine *is* — its identity, what it holds, what it has applied. Reported upward over
|
|
the link; never asked downward.
|
|
|
|
## What it is not
|
|
|
|
- It does not decide anything that needs another node.
|
|
- It never queries the mesh database.
|
|
- It has no listening surface.
|
|
- **It does not manage its own unit.** It manages `service` resources and its own unit is one —
|
|
the temptation is obvious and it ends with a host stopping itself half way through an apply,
|
|
leaving a machine with nothing running to fix it. The installation owns the host; the host owns
|
|
everything else.
|
|
|
|
**How it is installed, enrolled, run, upgraded and retired is
|
|
[`09-the-node-lifecycle.md`](09-the-node-lifecycle.md)**, in full and in one place. This document
|
|
is the component; that one is what happens to it.
|
|
|
|
## Where a declaration comes from
|
|
|
|
One behaviour, two sources
|
|
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)):
|
|
|
|
| Situation | Source |
|
|
|---|---|
|
|
| no mesh reachable | `substrate.lock` — the pinned bundle the host carries |
|
|
| mesh reachable | the control plane, over the link |
|
|
|
|
**The first node is not a different kind of node.** It is a node whose mesh is not up yet. It
|
|
applies the bundle it carries, the control plane comes up on top of it, and from that moment it
|
|
takes declarations like every other node. Its specialness is temporary and self-erasing.
|
|
|
|
**A joining node does the minimum to be reachable and nothing else** — an identity, an address,
|
|
one peer — and then stops deciding. It does not compute the overlay; it needs one peer to reach
|
|
the mesh, and the full peer set arrives derived.
|
|
|
|
## What a declaration is
|
|
|
|
Settled by [ADR 0005](../../02-DECISIONS/0005-the-node-host.md).
|
|
|
|
**JSON**, because the host has no dependencies to spend and the standard library carries no
|
|
YAML. **An ordered list of typed resources**, each with a stable identity — the order is stated
|
|
rather than derived, because deriving it would be the host deciding the thing most likely to
|
|
differ from what the control plane intended.
|
|
|
|
**Unknown is refused, never skipped.** An unknown version, type or field refuses the whole
|
|
declaration. A host that skipped what it did not understand would apply most of it and report
|
|
success.
|
|
|
|
**Complete for what the host owns, and only that.** It removes what it previously applied and
|
|
is no longer declared — a fact it holds, from the store, rather than an inference — and never
|
|
removes anything it did not create.
|
|
|
|
**Addressed.** A host with an identity refuses a declaration addressed elsewhere; a host
|
|
without one, applying the bundle it carries, has nothing to check against.
|
|
|
|
## Build order
|
|
|
|
Staged so each stage is verifiable in the lab before the next exists.
|
|
|
|
**1 — profile and inventory.** The host runs on a machine, detects what it can do, and reports
|
|
what it is. No control plane, no declarations, no network. Verifiable immediately: the lab's
|
|
`place:` gains its first implementation, and a raised scenario finally contains something.
|
|
|
|
**2 — apply, from the bundle.** The host applies `substrate.lock` with no mesh present. This is
|
|
the first node's path, and it is the claim the skeleton's Move 1 rests on and has never proved:
|
|
that one host can raise the substrate alone.
|
|
|
|
Raising the substrate needs six shapes in the host's vocabulary, and **all six are built**:
|
|
|
|
| | | |
|
|
|---|---|---|
|
|
| `directory`, `file` | **built** | no machine dependency at all |
|
|
| `service` | **built** | running/stopped **and** enabled/disabled at boot — a unit started but not enabled stops being true at the next reboot |
|
|
| `package` | **built** | present, never upgraded, and **never uninstalled** — the host cannot know what else needs it, so dropping one is *forgotten*, not *removed* |
|
|
| `container` | **built** | pinned by digest ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)); identified by a label carrying a digest of the declaration that made it, because a runtime normalises what it is given and that is indistinguishable from drift |
|
|
| `action` | **built** | bundle-only ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)); verify is mandatory and is the idempotency check as well as the read-back |
|
|
|
|
**The parser enforces the boundary rather than the caller remembering it.** `Parse` refuses an
|
|
action and is what the link uses; `ParseTrusted` permits one and is what the bundle uses. The
|
|
safe path is the default and the permissive one has to be named.
|
|
|
|
**The lab still cannot exercise the last three**, and that is now the only thing in the way: a
|
|
scenario is a closed address space, so nothing can be fetched there, and its machines carry no
|
|
container runtime. All three were instead verified against a real machine — a container created,
|
|
labelled, replaced when its declaration changed, exec'd into and removed; an action that exits
|
|
zero and satisfies nothing failing the apply. That is lab-installation work
|
|
([`04-lab-installation.md`](04-lab-installation.md)) rather than a constraint on the design, but
|
|
until it is done the substrate bootstrap has no end-to-end test.
|
|
|
|
**3 — link and store.** The node connects, receives declarations, and holds what it applied.
|
|
|
|
**4 — enrolment.** The one genuinely new mechanism in
|
|
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md); everything else there
|
|
is configuration of what already runs.
|
|
|
|
Stage 1 is deliberately the smallest useful thing. The lab currently raises **empty machines**
|
|
([`00-as-is/11`](../00-as-is/11-the-lab.md)) because `place:` has nothing to place; stage 1 ends
|
|
that, and every later stage is tested by a lab that already works.
|
|
|
|
## How it is verified
|
|
|
|
**The lab is the harness.** A scenario places a host on a machine and asserts what it did —
|
|
against a real hypervisor, with the boundary never mocked
|
|
([ADR 0017](../../02-DECISIONS/0017-a-test-defends-a-decision.md)).
|
|
|
|
Each decision above owes a test:
|
|
|
|
| Decision | What asserts it |
|
|
|---|---|
|
|
| 0037 — the host never queries the mesh database | no database client in the dependency tree; a dependency-direction lint failing on an upward import |
|
|
| 0038 — one behaviour, two sources | the same code path raises a first node and joins a second |
|
|
| 0039 — a node holds no shared credential | a raised node's store contains no credential to any service |
|
|
| 0036 — disconnection is a situation | a node cut off and returned reconciles without being re-adopted |
|
|
| 0008 — a failed step fails the apply | an apply with a failing step reports failure |
|
|
|
|
## Open
|
|
|
|
- **Whether one host can raise the substrate alone.** Move 1 assumes it. Stage 2 tests it, and
|
|
if it is false the tier boundary moves.
|
|
- **What the host carries versus what it finds.** It manages `wg`, `nft`, `pacman`, `docker`;
|
|
it does not contain them, and how it obtains one it lacks is undecided —
|
|
[04-ISSUES/007](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md).
|
|
- **Six vocabularies.** Zero dependencies, but the host must still know what a peer, a rule, a
|
|
package, a unit, a container and a dataset *are*. Nothing has measured that surface, and it is
|
|
the residue of the question [`host-size.md`](../../01-RESEARCH/006-mesh-from-scratch/host-size.md)
|
|
answered.
|
|
- **Rescue.** [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) suggests it is
|
|
a node whose local state is discarded so the mesh re-derives it, and does not decide it.
|
|
- **What may expire.** An identity needing refresh to stay valid would make a laptop fail for
|
|
being a laptop ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
|
|
|
|
## What was added to the vocabulary, and why each cost was worth paying
|
|
|
|
*Written 2026-08-30. Every addition widens what a compromised control plane can express, so the
|
|
count is asserted by a test and a change to it is a decision rather than a convenience.*
|
|
|
|
Six shapes raised the substrate. Two more exist because most of what a person installs is not a
|
|
service:
|
|
|
|
| | why |
|
|
|---|---|
|
|
| **user** | a shell, a terminal, a chat client, a desktop are a package plus configuration **in somebody's home**. A mesh with no user owns `/etc` and nothing anybody looks at |
|
|
| **archive** | a theme is hundreds of files. Inlining them makes every declaration enormous and rewrites all of them when one changes |
|
|
|
|
And `file` gained two fields: `bytes`, because a wallpaper is not a string, and `owner`, because a
|
|
file in a home belongs to somebody.
|
|
|
|
**`user` also makes a login shell declared state.** `chsh` is a command, the link may not carry
|
|
one, and a shell that could only be set by hand is a shell the mesh cannot manage — which is most
|
|
of the reason to manage a machine.
|
|
|
|
### The refusals that came with them
|
|
|
|
- **A file says what is in it exactly once.** `content`, `bytes` and `sealed` are exclusive, so
|
|
*what is in this file* is answerable by looking rather than by knowing which field wins.
|
|
- **Groups are added, never pruned.** The tool that sets them replaces the set unless told
|
|
otherwise, which would silently remove every group that makes a login able to use the machine.
|
|
A machine's own groups are not the mesh's to know about.
|
|
- **An archive is pinned by digest, checked before a single file is written.** This is the one
|
|
place the host reaches out on its own — everywhere else it holds one outbound connection and
|
|
fetches nothing — so the only thing making those bytes safe to unpack is that they hash to what
|
|
was declared.
|
|
- **An entry naming a path outside the archive is refused, not sanitised.** Rewriting it to land
|
|
inside would put a file somewhere nobody asked for and report success. The first implementation
|
|
quietly relocated it, and a test caught that.
|
|
- **Symlinks and device nodes are refused rather than skipped**, or an archive needing one arrives
|
|
silently incomplete.
|
|
|
|
**A partial host does archives and refuses users**: an archive needs a filesystem and a way to
|
|
fetch; a user needs a user database the host is allowed to write. |