Review of the fix for hq issue 104 found three faults in it. A file applied
on an enrolled node — the mesh's own last declaration included — is applied
as the bundle is, so its resources are recorded as the machine's own and
what the mesh declared reads as undeclared: the plan removed the foundation.
`apply FILE` is for a machine the mesh has not spoken to, and is now refused
saying so whenever declared.json exists. The plan looked at what is held
before what the declaration says is taken, so the one cutover ADR 0100 says
must be previewed read as a hold; it now decides in holdOnAdopted's order,
models a step run inside a held container, and a test holds the plan's
sequence to the apply's outcomes. Genesis wrote the mode on every run, so a
re-run after `converge` left the state saying adopted while the kept,
signed declaration said converged, and the reconcile loop refused every five
minutes with no delivery coming to end it: genesis now writes the mode only
when none is recorded, and where the state and the verified kept declaration
disagree, the kept declaration wins and the repair is said.
Also: a file lock beside the state, taken by the link service, the host's
own commands and the installer alike, so a `reconcile` run by hand no
longer races the loop's save — chosen over refusing while a named service is
active, which would miss a `mesh-host run` started by hand; `--json
--dry-run` emits {plan} like an apply emits {plan, report}; the README's
duplicate flag line; and the bundle refusal is about the digest, not a claim
the carried bytes can never match what genesis applied.
233 lines
12 KiB
Markdown
233 lines
12 KiB
Markdown
# mesh-host
|
|
|
|
Tier 0 of the Novox Mesh. The one thing ever installed by hand, and the only thing that changes
|
|
a machine.
|
|
|
|
```
|
|
scp mesh-host root@machine:/usr/local/bin/
|
|
mesh-host profile
|
|
```
|
|
|
|
That is the whole installation. One statically linked binary, nothing else present, no runtime
|
|
to install first ([`novox/hq` ADR 0005](https://git.novox.be/novox/hq)).
|
|
|
|
## What it is for
|
|
|
|
**Apply declared state on this machine.** Overlay membership, packet filtering, packages,
|
|
services, containers and filesystems are not six concerns it carries; they are six instances of
|
|
the one.
|
|
|
|
**It does not decide.** Anything needing knowledge of another node is the control plane's, and
|
|
the host never queries the mesh database. It receives declarations and applies them.
|
|
|
|
## Joining a mesh
|
|
|
|
```
|
|
mesh-host enrol --token <token> --name <what this machine is called>
|
|
```
|
|
|
|
**The node generates its own identity** — an Ed25519 keypair whose private half never leaves the
|
|
machine. The mesh records the public half. Nothing is issued to this node; it arrives holding its
|
|
identity, and what it receives is being known.
|
|
|
|
**The broker's certificate is checked before this machine sends anything.** The token pins a
|
|
fingerprint; the connection is refused if what answers presents anything else. That refusal has
|
|
its own error and says plainly that retrying will not help, because it does not mean the network
|
|
is down — it means the mesh was substituted, and since this host applies whatever the link
|
|
delivers, that would be the whole machine.
|
|
|
|
There is no certificate authority involved and no hostname check. At bootstrap the broker is
|
|
self-signed and reached at an address rather than a name, so there is nothing to trace and nothing
|
|
to match. One exact certificate, or nothing, which is stricter than either.
|
|
|
|
**An already-enrolled machine refuses to enrol again.** The mesh believes its first identity, so
|
|
replacing it is deliberate: remove the identity file first.
|
|
|
|
**What is not built is the link itself.** Enrolment verifies the broker and generates the identity,
|
|
and then stops, having saved nothing — so it can be run again unchanged.
|
|
|
|
## What exists today
|
|
|
|
**Stages 1 and 2.** It reports what a machine is, and it applies a declaration to one. It
|
|
connects to nothing and listens on nothing — what it applies comes from a file.
|
|
|
|
```
|
|
mesh-host profile what this machine can be asked to do
|
|
mesh-host inventory what this machine is, and what it holds
|
|
mesh-host apply FILE make this machine match a declaration from a file
|
|
mesh-host reconcile make this machine match what the mesh last told it — or, before
|
|
any mesh has, the bundle this host carries
|
|
mesh-host bundle show what this host carries
|
|
mesh-host owned what this host has applied and still owns
|
|
--json machine-readable
|
|
--state where this node keeps what it knows
|
|
--dry-run say what applying would change, and change nothing
|
|
```
|
|
|
|
```
|
|
$ mesh-host profile
|
|
linux/amd64
|
|
|
|
yes container-runtime 29.7.2
|
|
no firewall nft exited 1: Operation not permitted (you must be root)
|
|
yes graphical-session x11: :1
|
|
yes overlay wg0
|
|
yes package-manager pacman 7.1.0
|
|
no privileged effective uid 1000, not 0
|
|
yes service-manager degraded
|
|
|
|
cannot be asked to: [firewall privileged]
|
|
```
|
|
|
|
## Applying
|
|
|
|
A declaration is JSON, versioned, and an **ordered list** of resources — the order is stated
|
|
rather than derived, because deriving it would be the host deciding
|
|
([`novox/hq` ADR 0005](https://git.novox.be/novox/hq)). The vocabulary is `directory`, `file`
|
|
and `service`, and **anything outside it refuses the whole declaration**: a host that skipped
|
|
what it did not understand would apply most of a declaration and report success.
|
|
|
|
```json
|
|
{"declaration":1,"resources":[
|
|
{"id":"mesh-etc","type":"directory","path":"/etc/mesh","mode":"0755"},
|
|
{"id":"node-conf","type":"file","path":"/etc/mesh/node.conf","content":"role = anchor\n","mode":"0640"},
|
|
{"id":"journal","type":"service","unit":"systemd-journald.service","state":"running"}
|
|
]}
|
|
```
|
|
|
|
**It converges rather than executes.** Applying twice changes nothing the second time; applying
|
|
to a drifted machine returns it. A mode is *maintained*, not merely set — a permission applied
|
|
at creation is not a permission held.
|
|
|
|
**It owns a footprint, and only that.** What it applied and is no longer declared is removed;
|
|
what it did not create is never touched. It knows which is which because it recorded what it
|
|
did, after each thing worked.
|
|
|
|
**A failed step fails the apply.** No step runs after a failure, and the error carries what had
|
|
already been done — the machine is in whatever state that left it, and pretending otherwise is
|
|
the fault this exists to prevent.
|
|
|
|
## The bundle a host carries
|
|
|
|
A host built for a machine carries its declaration **inside the binary**:
|
|
|
|
```
|
|
make host BUNDLE=path/to/foundation.lock
|
|
```
|
|
|
|
`mesh-host reconcile` then applies it, on a machine the mesh has told nothing yet. That is the
|
|
first node's path — no mesh present, nothing fetched, nothing else copied onto the machine. Once
|
|
the mesh has spoken, `reconcile` holds the machine to what it last said and never to the bundle,
|
|
which genesis consumed; a bundle or a file is refused when it says the other mode than the node
|
|
is in; and `apply FILE` is refused altogether once the mesh has spoken — a file is applied as the
|
|
bundle is, its resources recorded as the machine's own, so on an enrolled node it would plan to
|
|
remove the foundation. `apply FILE` is for a machine the mesh has not spoken to. (hq's to-be
|
|
node lifecycle describes `apply repair.json` as a rescue on an enrolled node; that line is being
|
|
amended in hq, and no rescue path exists here yet.) Both commands say what they would change
|
|
before changing anything, and `--dry-run` is that alone. `copy it and run it` stops being true the moment
|
|
a second file has to arrive with it, which is why the bundle is embedded rather than beside it.
|
|
|
|
**A default build carries nothing and refuses to reconcile**, saying so. A host that applied
|
|
nothing and reported success would look exactly like one that raised a first node, and the
|
|
difference would surface later as a mesh that never came up with nothing to point at.
|
|
|
|
Stages 3 and 4 — the link, and enrolment — are designed and not built.
|
|
|
|
## What stage 2 does not yet prove
|
|
|
|
The design defines stage 2 as *the host applies `foundation.lock` with no mesh present*, and
|
|
calls out the claim underneath it: **that one host can raise the foundation alone**.
|
|
|
|
The mechanism is proved — a sealed machine, one binary, and it configures itself from what it
|
|
carries. The claim is not. The foundation is four container services, and:
|
|
|
|
- the vocabulary has no container type, because a container needs an image and where images
|
|
come from is open ([`novox/hq` research 012](https://git.novox.be/novox/hq));
|
|
- what belongs in a foundation is not known — the closure for a one-node mesh is what
|
|
research 011 and 012 exist to answer;
|
|
- and the machine used to test this has no container runtime, because a sealed network cannot
|
|
install one.
|
|
|
|
So `foundation.lock` here is a real bundle with a placeholder's content. Saying that plainly
|
|
beats shipping a host that claims a foundation it has never raised.
|
|
|
|
## A capability is detected, never assumed
|
|
|
|
The reason this is the first thing built rather than a detail of it.
|
|
|
|
**An installed package is not a capability.** A container client on disk with its daemon down
|
|
looks exactly like a working runtime, and a node assigned work on that basis fails at the
|
|
moment the work arrives. So every detector runs something that only succeeds if the thing is
|
|
**functioning** — the daemon is asked for its version, the package database is queried, the
|
|
firewall is asked to list a ruleset, which needs the privilege as well as the tool.
|
|
|
|
**Every verdict says how it knows.** A capability reported absent with no reason is a fault
|
|
nobody can act on. The reason is what a person reads when a node will not take work they
|
|
expected it to take.
|
|
|
|
**A unit that does not exist is not a unit that is stopped.** `systemctl is-active` says
|
|
`inactive` for both, so declaring a unit stopped reported success for a unit the host cannot
|
|
manage at all. `LoadState` separates them. Found by applying inside a raised machine, not by
|
|
reasoning — and its sibling: removing an orphaned service whose unit has since been uninstalled
|
|
used to fail the whole apply, which left a node able to apply *nothing*, ever.
|
|
|
|
**Exit codes are not the whole answer.** Found by running against a real machine rather than by
|
|
reasoning: `systemctl is-system-running` exits non-zero for every state except `running` —
|
|
including `degraded`, which means some units failed and the init is emphatically there. Reading
|
|
the exit code reported no service manager on a machine whose init it was. That is the same
|
|
fault in the mirror — installed-but-broken reported present, working-but-imperfect reported
|
|
absent — and both place work wrongly.
|
|
|
|
## Building
|
|
|
|
```
|
|
go test ./... structure and logic, and the same checks against this machine
|
|
CGO_ENABLED=0 go build -ldflags="-s -w" -o mesh-host ./cmd/mesh-host
|
|
```
|
|
|
|
Roughly 3 MB, static, no dynamic dependencies. Cross-compiles with `GOOS`/`GOARCH`; a host is
|
|
built once per architecture and copied, never built on the machine it runs on.
|
|
|
|
**Mocking the boundary is forbidden** ([`novox/hq` ADR 0017](https://git.novox.be/novox/hq)).
|
|
Every detector is exercised against a fake runner for its logic *and* against this machine for
|
|
its behaviour. The tests do not assert which capabilities a machine has — that varies, and is
|
|
the point of detecting — they assert that detection tells the truth about whatever is there.
|
|
|
|
## Where the reasoning lives
|
|
|
|
Design and decisions are in [`novox/hq`](https://git.novox.be/novox/hq), not here. This
|
|
repository carries implementation and does not carry decisions.
|
|
|
|
- `03-DESIGN/01-to-be/05-the-node-host.md` — what this is and the order it is built in
|
|
- `02-DECISIONS/0037-the-host-applies-it-does-not-decide.md` — the one concern
|
|
- `02-DECISIONS/0038-a-node-joins-by-linking-first.md` — one behaviour, two sources
|
|
- `02-DECISIONS/0039-the-link-is-the-security-boundary.md` — a node owns no password
|
|
- `02-DECISIONS/0041-the-host-depends-on-nothing.md` — why this is a static binary, and Go
|
|
- `04-ISSUES/007-an-installed-package-is-not-a-capability` — why detection works this way
|
|
|
|
## Checks that cross into the control plane's repository
|
|
|
|
Two things are agreed between this repository and `novox/mesh-controller`, and each is a separate
|
|
struct on each side. A field renamed on one of them fails **silently** — the crossing succeeds and
|
|
something is simply absent — so both are checked by handing one side's real output to the other's
|
|
real parser. Neither runs by default; each skips with a reason, because a repository that fails
|
|
without its neighbour checked out is a repository nobody can build.
|
|
|
|
**What the mesh sends, read by this host:**
|
|
|
|
```
|
|
mesh-controller: ./build/mesh-controller plan <node> --json > /tmp/d.json
|
|
mesh-host: MESH_EMITTED=/tmp/d.json go test ./internal/declaration/ -v
|
|
```
|
|
|
|
**What this node says when it joins, read by the mesh:**
|
|
|
|
```
|
|
mesh-host: MESH_ENROL_OUT=/tmp/enrol.json go test ./internal/link/
|
|
mesh-controller: MESH_ENROL=/tmp/enrol.json make check
|
|
```
|
|
|
|
The second writes the private half of the sealing key beside the request, so the mesh's suite can
|
|
prove that what it sealed is openable rather than merely present. A key that is correctly named
|
|
and simply *wrong* passes every check that only looks at the message.
|