409 lines
23 KiB
Markdown
409 lines
23 KiB
Markdown
---
|
|
layer: to-be
|
|
status: in-progress
|
|
code: [mesh-host]
|
|
updated: 2026-09-22
|
|
decisions:
|
|
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
|
- 02-DECISIONS/0019-how-this-repository-works.md
|
|
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
|
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
|
- 02-DECISIONS/0010-delivery.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
- 02-DECISIONS/0005-the-node-host.md
|
|
---
|
|
|
|
# The node host
|
|
|
|
Tier 0. The one thing ever installed by hand, and the only thing that changes a machine.
|
|
|
|
## What it is
|
|
|
|
A **statically linked binary that requires nothing to be present** — copy it onto a machine and
|
|
run it, and that is the whole installation
|
|
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). Written in Go, because the
|
|
job is system-level and because the host shares no code with any other tier.
|
|
|
|
A single binary with one job: **apply declared state on this machine**
|
|
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). Overlay
|
|
membership, packet filtering, packages, services, containers and filesystems are not six
|
|
concerns it carries; they are six instances of the one.
|
|
|
|
It replaces three things that exist today
|
|
([`00-as-is/05`](../00-as-is/05-runtime-and-installation.md)): the three hand-run bootstrap
|
|
scripts — first node, joining, rescue — and the synchronisers that rewrite managed files
|
|
([`00-as-is/06`](../00-as-is/06-configuration-and-secrets.md)).
|
|
|
|
**What it is not:** it does not decide anything that needs another node, it never queries the
|
|
mesh database, and it has no listening surface.
|
|
|
|
## The parts
|
|
|
|
| Part | Owns |
|
|
|---|---|
|
|
| `apply` | reconciling declared state on this machine |
|
|
| `store` | local state, authoritative while disconnected |
|
|
| `link` | the single outbound connection to the controller |
|
|
| `profile` | what this machine can be asked to do |
|
|
| `inventory` | what this machine is and has |
|
|
| `foundation.lock` | the pinned tier-1 descriptor, appliable with no mesh present |
|
|
|
|
### apply
|
|
|
|
Takes a **declaration** and makes the machine match it. Idempotent: applying the same
|
|
declaration twice changes nothing the second time, and applying it to a drifted machine
|
|
returns it.
|
|
|
|
Three properties, each following a recorded decision:
|
|
|
|
**A failed step fails the apply.** Not "logs and continues"
|
|
([ADR 0010](../../02-DECISIONS/0010-delivery.md)). A partial apply that
|
|
reports success is the mesh's most expensive shape.
|
|
|
|
**Each applier reads back.** Setting a value is not evidence the value took. The firewall is
|
|
asked whether the rule loaded; conntrack is asked what timeout it holds. This is `how-we-build`
|
|
§5 as a component requirement rather than a review habit.
|
|
|
|
**What was applied is recorded after it works, never before**
|
|
([ADR 0018](../../02-DECISIONS/0018-a-picture-is-read-from-what-runs.md)). A failed apply leaves
|
|
the machine in whatever state it reached, and nothing must claim otherwise.
|
|
|
|
### store
|
|
|
|
Local, and **authoritative while disconnected**. Not a cache of the controller — the record
|
|
of what this node has applied and what it currently holds.
|
|
|
|
This is structural rather than convenient: if disconnection is an ordinary situation rather
|
|
than an exception ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)), the
|
|
store is what makes it ordinary. A laptop shut for a week comes back and reconciles; it does
|
|
not come back and ask what it is.
|
|
|
|
### link
|
|
|
|
The node's one connection to the controller, and its security boundary
|
|
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
|
|
|
|
It is the broker connection that already exists
|
|
([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)) — outbound,
|
|
node-initiated, per-node addressed — carrying **per-node identity instead of a shared
|
|
credential**. The node owns no password. It owns an identity, and that identity is what it
|
|
presents.
|
|
|
|
What arrives is bounded by form: **declarations of known shape, never a command to run.**
|
|
|
|
**It must fail legibly.** A node that cannot link says which side refused and on what grounds.
|
|
A boundary that refuses without saying why is worse than the password it replaced.
|
|
|
|
### profile
|
|
|
|
What this machine can be asked to do — a graphical session, a container runtime, an
|
|
architecture, a network position.
|
|
|
|
**Detected, never assumed.** A package being installed does not mean a capability is present
|
|
([04-ISSUES/007](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md)): a
|
|
capability is real when it is present, running and working, and the difference is the whole
|
|
point of detecting it.
|
|
|
|
The profile is what makes [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)
|
|
work: a node is a node, and what varies between them is here rather than in the definition.
|
|
|
|
### inventory
|
|
|
|
What this machine *is* — its identity, what it holds, what it has applied. Reported upward over
|
|
the link; never asked downward.
|
|
|
|
## What it is not
|
|
|
|
- It does not decide anything that needs another node.
|
|
- It never queries the mesh database.
|
|
- It has no listening surface.
|
|
- **It does not manage its own unit.** It manages `service` resources and its own unit is one —
|
|
the temptation is obvious and it ends with a host stopping itself half way through an apply,
|
|
leaving a machine with nothing running to fix it. The installation owns the host; the host owns
|
|
everything else.
|
|
|
|
**How it is installed, enrolled, run, upgraded and retired is
|
|
[`09-the-node-lifecycle.md`](09-the-node-lifecycle.md)**, in full and in one place. This document
|
|
is the component; that one is what happens to it.
|
|
|
|
## What it finds, on an adopted node
|
|
|
|
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* A declaration says whether the node is adopted, and which of its
|
|
modules have been **taken**. *Found* is a file at a declared path, or a container at a declared
|
|
name, that the host's store has no record of writing. On an adopted node the host keeps what it
|
|
found for any module not yet taken: it records a found file's original content before anything
|
|
else, and it reports the file or container as held — a report that says what it holds, so an
|
|
adopted node never reads as converged. Once the module is taken, its resources converge like any
|
|
other. What is held is never removed, even when its module is unassigned, and a held file or
|
|
container that changes while held is reported as changed by something else, not reverted or
|
|
restarted. The host also
|
|
converges a new resource, the **opening** — a port made reachable through the firewall it found
|
|
([08-connectivity](08-connectivity.md)) — and reports which firewall it found. This is the
|
|
companion the host's ownership rule needed: *never touch what you did not create, unless adoption
|
|
made it yours — and while the node is adopted, not until its module is taken.* *How it is
|
|
checked:* unit tests hold the host to keeping a found file and container, converging them once
|
|
taken, never removing a held file and reporting one that changed; the adoption bed asserts a found
|
|
file byte for byte unchanged until its module is taken.
|
|
|
|
## Where a declaration comes from
|
|
|
|
One behaviour, two sources
|
|
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)):
|
|
|
|
| Situation | Source |
|
|
|---|---|
|
|
| no mesh reachable | `foundation.lock` — the pinned bundle the host carries |
|
|
| mesh reachable | the controller, over the link |
|
|
|
|
**The first node is not a different kind of node.** It is a node whose mesh is not up yet. It
|
|
applies the bundle it carries, the controller comes up on top of it, and from that moment it
|
|
takes declarations like every other node. Its specialness is temporary and self-erasing.
|
|
|
|
**A joining node does the minimum to be reachable and nothing else** — an identity, an address,
|
|
one peer — and then stops deciding. It does not compute the overlay; it needs one peer to reach
|
|
the mesh, and the full peer set arrives derived.
|
|
|
|
## What a declaration is
|
|
|
|
Settled by [ADR 0005](../../02-DECISIONS/0005-the-node-host.md).
|
|
|
|
**JSON**, because the host has no dependencies to spend and the standard library carries no
|
|
YAML. **An ordered list of typed resources**, each with a stable identity — the order is stated
|
|
rather than derived, because deriving it would be the host deciding the thing most likely to
|
|
differ from what the controller intended.
|
|
|
|
**Unknown is refused, never skipped.** An unknown version, type or field refuses the whole
|
|
declaration. A host that skipped what it did not understand would apply most of it and report
|
|
success.
|
|
|
|
**Complete for what the host owns, and only that.** It removes what it previously applied and
|
|
is no longer declared — a fact it holds, from the store, rather than an inference — and never
|
|
removes anything it did not create.
|
|
|
|
**Addressed.** A host with an identity refuses a declaration addressed elsewhere; a host
|
|
without one, applying the bundle it carries, has nothing to check against.
|
|
|
|
## Build order
|
|
|
|
Staged so each stage is verifiable in the lab before the next exists.
|
|
|
|
**1 — profile and inventory.** The host runs on a machine, detects what it can do, and reports
|
|
what it is. No controller, no declarations, no network. Verifiable immediately: the lab's
|
|
`place:` gains its first implementation, and a raised scenario finally contains something.
|
|
|
|
**2 — apply, from the bundle.** The host applies `foundation.lock` with no mesh present. This is
|
|
the first node's path, and it is the claim the skeleton's Move 1 rests on and has never proved:
|
|
that one host can raise the foundation alone.
|
|
|
|
Raising the foundation uses **four** shapes — `package`, `container`, `service`, `action` —
|
|
counted from the bundle that exists rather than reasoned about. `file` and `directory` are listed
|
|
below because they are the cheapest to be sure of and a foundation that needed them would find them
|
|
ready; the current bundle simply does not. **All of them are built:**
|
|
|
|
| | | |
|
|
|---|---|---|
|
|
| `directory`, `file` | **built** | no machine dependency at all |
|
|
| `service` | **built** | running/stopped **and** enabled/disabled at boot — a unit started but not enabled stops being true at the next reboot |
|
|
| `package` | **built** | present, never upgraded, and **never uninstalled** — the host cannot know what else needs it, so dropping one is *forgotten*, not *removed* |
|
|
| `container` | **built** | pinned by digest ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)); identified by a label carrying a digest of the declaration that made it, because a runtime normalises what it is given and that is indistinguishable from drift |
|
|
| `action` | **built** | bundle-only ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)); verify is mandatory and is the idempotency check as well as the read-back |
|
|
|
|
One more shape is decided and not yet built: **`opening`**, on adopted nodes only
|
|
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)) — a port
|
|
made reachable, from where, on the incoming or forwarded path, through the firewall found on the
|
|
machine, marked as the mesh's and re-checked on every reconcile.
|
|
|
|
**A service says what it must reflect, and that is declared state rather than a command.**
|
|
`restart-on` names files whose change means the unit must be restarted — because a running service
|
|
does not re-read its configuration, and replacing a file, finding the service already running and
|
|
doing nothing leaves a machine behaving the way it did before while every check passes. A *command*
|
|
to restart would be an action, and the link may not carry one, so this is the shape that rule
|
|
leaves rather than a way around it.
|
|
|
|
**It may name a file another module put there**, written `<module>.<id>`. The case that needed it:
|
|
a resolver restarting when the mesh rewrites the names, which are computed by the mesh and belong
|
|
to its module rather than to the daemon's. Without it the daemon serves the names it started with
|
|
for ever — every machine that joined afterwards unreachable by name, and every check passing. An
|
|
unqualified name still means *my own*, so the common case reads as it always did.
|
|
|
|
**An action's verify is the definition of what the action is for**, and the action's own idea of
|
|
being finished must be the same one. *Written 2026-08-31, after this went wrong.* If an action
|
|
waits on one test and its verify reads back another, the two can disagree — and then the action
|
|
succeeds into a state its own verify rejects. The host says so accurately and uselessly: *the
|
|
action ran without error and its own verify still fails.* It is intermittent, it reads as a slow
|
|
machine, and the remedy people reach for is a longer timeout, which cannot help.
|
|
[04-ISSUES/017](../../04-ISSUES/017-an-action-succeeded-into-a-state-its-verify-rejects/00-report.md)
|
|
is that, in the one action the whole bootstrap depends on.
|
|
|
|
**The parser enforces the boundary rather than the caller remembering it.** `Parse` refuses an
|
|
action and is what the link uses; `ParseTrusted` permits one and is what the bundle uses. The
|
|
safe path is the default and the permissive one has to be named.
|
|
|
|
**The lab still cannot exercise the last three**, and that is now the only thing in the way: a
|
|
scenario is a closed address space, so nothing can be fetched there, and its machines carry no
|
|
container runtime. All three were instead verified against a real machine — a container created,
|
|
labelled, replaced when its declaration changed, exec'd into and removed; an action that exits
|
|
zero and satisfies nothing failing the apply. That is lab-installation work
|
|
([`04-lab-installation.md`](04-lab-installation.md)) rather than a constraint on the design, but
|
|
until it is done the foundation bootstrap has no end-to-end test.
|
|
|
|
**3 — link and store.** The node connects, receives declarations, and holds what it applied.
|
|
|
|
**4 — enrolment.** The one genuinely new mechanism in
|
|
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md); everything else there
|
|
is configuration of what already runs.
|
|
|
|
Stage 1 is deliberately the smallest useful thing. The lab currently raises **empty machines**
|
|
([`00-as-is/11`](../00-as-is/11-the-lab.md)) because `place:` has nothing to place; stage 1 ends
|
|
that, and every later stage is tested by a lab that already works.
|
|
|
|
## How it is verified
|
|
|
|
**The lab is the harness.** A scenario places a host on a machine and asserts what it did —
|
|
against a real hypervisor, with the boundary never mocked
|
|
([ADR 0017](../../02-DECISIONS/0017-a-test-defends-a-decision.md)).
|
|
|
|
Each decision above owes a test:
|
|
|
|
| Decision | What asserts it |
|
|
|---|---|
|
|
| 0037 — the host never queries the mesh database | no database client in the dependency tree; a dependency-direction lint failing on an upward import |
|
|
| 0038 — one behaviour, two sources | the same code path raises a first node and joins a second |
|
|
| 0039 — a node holds no shared credential | a raised node's store contains no credential to any service |
|
|
| 0036 — disconnection is a situation | a node cut off and returned reconciles without being re-adopted |
|
|
| 0008 — a failed step fails the apply | an apply with a failing step reports failure |
|
|
|
|
## What a machine says about itself, and what the mesh keeps of it
|
|
|
|
*2026-08-31, from finding that half of it was being discarded.*
|
|
|
|
**A capability is detected and never assumed** ([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)),
|
|
so the only account of what a machine can do is the one the machine gave. That account has two
|
|
halves and the mesh was keeping one:
|
|
|
|
| | |
|
|
|---|---|
|
|
| **the yes or no** | gates an assignment — *this machine has no seat, and nothing can be installed to fix that* |
|
|
| **the detail** | carries a value — `seat: card1-DP-1`, an architecture, an amount of memory |
|
|
|
|
They are **one fact read two ways**: *can this run here* and *what should it be configured as*. A
|
|
module that must not be assigned without an OLED panel and one that dims itself differently on one
|
|
are reading the same line. Keeping only the first read makes the second unanswerable, and there is
|
|
nowhere else to get it — the detector is the only thing that looked.
|
|
|
|
**The reason an absent capability is absent goes the same way, and it is the half a person needs
|
|
most.** *This machine has no container runtime* is the answer; *docker is not installed* is why.
|
|
The first is the mesh's to say and the second is only the machine's.
|
|
|
|
### The eight, and what each one is evidence of
|
|
|
|
*Written 2026-08-31 from `internal/profile/detectors.go`, because the set was implemented and
|
|
enumerated in no document. A vocabulary a module writes against, that exists only in code, is one
|
|
nobody can write against without reading the code.*
|
|
|
|
| capability | what a detection proves |
|
|
|---|---|
|
|
| `container-runtime` | a runtime is **running**, not installed |
|
|
| `package-manager` | the machine's own package manager works |
|
|
| `service-manager` | an init that can be asked for state — including *degraded*, which reports on stdout and exits non-zero |
|
|
| `firewall` | a filter this host can write rules into |
|
|
| `overlay` | the private network can be joined |
|
|
| `graphical-session` | a display server **is running** — state |
|
|
| `seat` | hardware where one **could** run — and assignment needs this one, not the row above |
|
|
| `privileged` | the host can change the machine |
|
|
|
|
**`seat` and `graphical-session` are the pair worth reading twice**, because collapsing them is
|
|
the obvious economy and it is wrong in both directions: a machine with a seat and no session can
|
|
be given a display server, and a machine with a session running is not thereby able to host a
|
|
second one.
|
|
|
|
**A detection runs something that only succeeds if the thing is *functioning*, never `--version`.**
|
|
A version string proves a binary is on disk, which
|
|
[`04-ISSUES/007`](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md)
|
|
records as false in the way that matters: the package was installed and the daemon was not
|
|
running.
|
|
|
|
**Never reported and reported nothing stay different.** One machine has not run the host yet; the
|
|
other ran it and can do nothing. Both refuse everything that requires a capability, and the
|
|
remedies are not remotely alike.
|
|
|
|
*Checked by recording a profile with a present capability carrying a value and an absent one
|
|
carrying its reason, and requiring both to survive — and by requiring a machine that never
|
|
reported to be distinguishable from one that reported an empty list.*
|
|
|
|
## Open
|
|
|
|
- **Whether one host can raise the foundation alone.** Move 1 assumes it. Stage 2 tests it, and
|
|
if it is false the tier boundary moves.
|
|
- **What the host carries versus what it finds.** It manages `wg`, `nft`, `pacman`, `docker`;
|
|
it does not contain them, and how it obtains one it lacks is undecided —
|
|
[04-ISSUES/007](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md).
|
|
- **Six vocabularies.** Zero dependencies, but the host must still know what a peer, a rule, a
|
|
package, a unit, a container and a dataset *are*. Nothing has measured that surface, and it is
|
|
the residue of the question [`host-size.md`](../../01-RESEARCH/006-mesh-from-scratch/host-size.md)
|
|
answered.
|
|
- **Rescue.** [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) suggests it is
|
|
a node whose local state is discarded so the mesh re-derives it, and does not decide it.
|
|
- **What may expire.** An identity needing refresh to stay valid would make a laptop fail for
|
|
being a laptop ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
|
|
|
|
## What was added to the vocabulary, and why each cost was worth paying
|
|
|
|
*Written 2026-08-30. Every addition widens what a compromised controller can express, so the
|
|
count is asserted by a test and a change to it is a decision rather than a convenience.*
|
|
|
|
Four shapes raise the foundation. Five more exist because most of what a person installs is not a
|
|
service:
|
|
|
|
| | why |
|
|
|---|---|
|
|
| **file**, **directory** | the foundation needs neither, and almost everything else does |
|
|
| **user** | a shell, a terminal, a chat client, a desktop are a package plus configuration **in somebody's home**. A mesh with no user owns `/etc` and nothing anybody looks at |
|
|
| **archive** | a theme is hundreds of files. Inlining them makes every declaration enormous and rewrites all of them when one changes |
|
|
| **network** | a module of several containers has to let them reach each other by name, and doing it with an action would create something nothing could ever remove ([ADR 0029](../../02-DECISIONS/0029-a-network-is-a-shape-because-an-action-cannot-be-undone.md)) |
|
|
|
|
And `file` gained two fields: `bytes`, because a wallpaper is not a string, and `owner`, because a
|
|
file in a home belongs to somebody.
|
|
|
|
**`user` also makes a login shell declared state.** `chsh` is a command, the link may not carry
|
|
one, and a shell that could only be set by hand is a shell the mesh cannot manage — which is most
|
|
of the reason to manage a machine.
|
|
|
|
### The refusals that came with them
|
|
|
|
- **A file says what is in it exactly once.** `content`, `bytes` and `sealed` are exclusive, so
|
|
*what is in this file* is answerable by looking rather than by knowing which field wins.
|
|
- **Groups are added, never pruned.** The tool that sets them replaces the set unless told
|
|
otherwise, which would silently remove every group that makes a login able to use the machine.
|
|
A machine's own groups are not the mesh's to know about.
|
|
- **An archive is pinned by digest, checked before a single file is written.** This is the one
|
|
place the host reaches out on its own — everywhere else it holds one outbound connection and
|
|
fetches nothing — so the only thing making those bytes safe to unpack is that they hash to what
|
|
was declared.
|
|
- **An entry naming a path outside the archive is refused, not sanitised.** Rewriting it to land
|
|
inside would put a file somewhere nobody asked for and report success. The first implementation
|
|
quietly relocated it, and a test caught that.
|
|
- **Symlinks and device nodes are refused rather than skipped**, or an archive needing one arrives
|
|
silently incomplete.
|
|
|
|
**A partial host does archives and refuses users**: an archive needs a filesystem and a way to
|
|
fetch; a user needs a user database the host is allowed to write.
|
|
|
|
## A machine becomes the last thing it was told
|
|
|
|
Every declaration is complete, so applying an old one is never wrong, only wasted — and under a
|
|
flurry of pushes a machine spent minutes becoming things the mesh had moved past
|
|
([issue 031](../../04-ISSUES/031-a-machine-becomes-each-thing-it-was-told-in-turn/00-report.md)).
|
|
So the host looks at what is already waiting before it applies anything: it holds a small window
|
|
of unacknowledged declarations, applies the newest, and sets the rest aside — each **reported as
|
|
superseded**, naming the one applied instead, because silence would read as a machine that
|
|
ignored an instruction and "applied" would be a lie. Applying stays one at a time; only seeing
|
|
is not. **Checked** by the link's unit tests on the drain, and by the genesis bed's settle wait,
|
|
which counts on a node catching up to the newest declaration rather than the oldest.
|
|
|