jschoubben f4143806c2 Build the rollback mechanism, and test it
ADR 0059's recovery path: the pieces that run when the host will not start.

internal/upgrade -- two facts, neither of them the host judging its health.
Whether the executable this process started from has been replaced on disk, and
which version last completed a reconcile.

The first design was wrong and the tests caught it, not review. It asked
/proc/self/exe whether it was marked deleted. That is Linux procfs behaviour
rather than a fact about files, and it catches only unlink -- a binary swapped
by rename onto the same path reads as untouched, which is exactly what a
package manager does. Now the identity is captured at start and compared later:
no procfs, and neither case missed.

known-good is one bare line. The reader is a shell script on a machine where
the host is failing to start, so it must not need a parser to be present and
working. Written only after a clean apply, which is the whole claim -- not
health, because a disconnected node is ordinary and a failing resource is the
machine's problem rather than the binary's.

packaging/ -- the unit, the rollback unit, and the rollback script. The script
shares no code with the host and calls none of it: a binary that cannot start
cannot be its own recovery. POSIX sh, nothing that has to be installed. The
unit carries Restart=always with a comment saying why on-failure would break
every upgrade.

Both are tested and both sets of tests were confirmed to bite. Injecting five
faults broke exactly the intended tests -- except one, and chasing why it did
not found a placebo assertion I had written: `check "exits zero" ... "0" "0"`
compares a literal to itself and can never fail. Replaced with the real exit
code, after which the injection bites.

Also caught: an injection that produced a build failure rather than a test
failure, which my grep read as "no failure". Re-run so it compiled, and the
test did bite.

The script test runs in `make check`, so it is a gate rather than something
that was run once.

Verified against the real binary: known-good is written beside the store after
a clean apply and is NOT written after a failed one.
2026-08-27 22:24:31 +02:00

mesh-host

Tier 0 of the Novox Mesh. The one thing ever installed by hand, and the only thing that changes a machine.

scp mesh-host root@machine:/usr/local/bin/
mesh-host profile

That is the whole installation. One statically linked binary, nothing else present, no runtime to install first (novox/hq ADR 0041).

What it is for

Apply declared state on this machine. Overlay membership, packet filtering, packages, services, containers and filesystems are not six concerns it carries; they are six instances of the one.

It does not decide. Anything needing knowledge of another node is the control plane's, and the host never queries the mesh database. It receives declarations and applies them.

What exists today

Stages 1 and 2. It reports what a machine is, and it applies a declaration to one. It connects to nothing and listens on nothing — what it applies comes from a file.

mesh-host profile        what this machine can be asked to do
mesh-host inventory      what this machine is, and what it holds
mesh-host apply FILE     make this machine match a declaration from a file
mesh-host reconcile      make this machine match the declaration this host carries
mesh-host bundle         show what this host carries
mesh-host owned          what this host has applied and still owns
  --json                 machine-readable
  --state                where this node keeps what it knows
  --dry-run              read and check the declaration, change nothing
$ mesh-host profile
linux/amd64

  yes  container-runtime  29.7.2
  no   firewall           nft exited 1: Operation not permitted (you must be root)
  yes  graphical-session  x11: :1
  yes  overlay            wg0
  yes  package-manager    pacman 7.1.0
  no   privileged         effective uid 1000, not 0
  yes  service-manager    degraded

cannot be asked to: [firewall privileged]

Applying

A declaration is JSON, versioned, and an ordered list of resources — the order is stated rather than derived, because deriving it would be the host deciding (novox/hq ADR 0043). The vocabulary is directory, file and service, and anything outside it refuses the whole declaration: a host that skipped what it did not understand would apply most of a declaration and report success.

{"declaration":1,"resources":[
  {"id":"mesh-etc","type":"directory","path":"/etc/mesh","mode":"0755"},
  {"id":"node-conf","type":"file","path":"/etc/mesh/node.conf","content":"role = anchor\n","mode":"0640"},
  {"id":"journal","type":"service","unit":"systemd-journald.service","state":"running"}
]}

It converges rather than executes. Applying twice changes nothing the second time; applying to a drifted machine returns it. A mode is maintained, not merely set — a permission applied at creation is not a permission held.

It owns a footprint, and only that. What it applied and is no longer declared is removed; what it did not create is never touched. It knows which is which because it recorded what it did, after each thing worked.

A failed step fails the apply. No step runs after a failure, and the error carries what had already been done — the machine is in whatever state that left it, and pretending otherwise is the fault this exists to prevent.

The bundle a host carries

A host built for a machine carries its declaration inside the binary:

make host BUNDLE=path/to/substrate.lock

mesh-host reconcile then applies it. That is the first node's path — no mesh present, nothing fetched, nothing else copied onto the machine. copy it and run it stops being true the moment a second file has to arrive with it, which is why the bundle is embedded rather than beside it.

A default build carries nothing and refuses to reconcile, saying so. A host that applied nothing and reported success would look exactly like one that raised a first node, and the difference would surface later as a mesh that never came up with nothing to point at.

Stages 3 and 4 — the link, and enrolment — are designed and not built.

What stage 2 does not yet prove

The design defines stage 2 as the host applies substrate.lock with no mesh present, and calls out the claim underneath it: that one host can raise the substrate alone.

The mechanism is proved — a sealed machine, one binary, and it configures itself from what it carries. The claim is not. The substrate is four container services, and:

  • the vocabulary has no container type, because a container needs an image and where images come from is open (novox/hq research 012);
  • what belongs in a substrate is not known — the closure for a one-node mesh is what research 011 and 012 exist to answer;
  • and the machine used to test this has no container runtime, because a sealed network cannot install one.

So substrate.lock here is a real bundle with a placeholder's content. Saying that plainly beats shipping a host that claims a substrate it has never raised.

A capability is detected, never assumed

The reason this is the first thing built rather than a detail of it.

An installed package is not a capability. A container client on disk with its daemon down looks exactly like a working runtime, and a node assigned work on that basis fails at the moment the work arrives. So every detector runs something that only succeeds if the thing is functioning — the daemon is asked for its version, the package database is queried, the firewall is asked to list a ruleset, which needs the privilege as well as the tool.

Every verdict says how it knows. A capability reported absent with no reason is a fault nobody can act on. The reason is what a person reads when a node will not take work they expected it to take.

A unit that does not exist is not a unit that is stopped. systemctl is-active says inactive for both, so declaring a unit stopped reported success for a unit the host cannot manage at all. LoadState separates them. Found by applying inside a raised machine, not by reasoning — and its sibling: removing an orphaned service whose unit has since been uninstalled used to fail the whole apply, which left a node able to apply nothing, ever.

Exit codes are not the whole answer. Found by running against a real machine rather than by reasoning: systemctl is-system-running exits non-zero for every state except running — including degraded, which means some units failed and the init is emphatically there. Reading the exit code reported no service manager on a machine whose init it was. That is the same fault in the mirror — installed-but-broken reported present, working-but-imperfect reported absent — and both place work wrongly.

Building

go test ./...                       structure and logic, and the same checks against this machine
CGO_ENABLED=0 go build -ldflags="-s -w" -o mesh-host ./cmd/mesh-host

Roughly 3 MB, static, no dynamic dependencies. Cross-compiles with GOOS/GOARCH; a host is built once per architecture and copied, never built on the machine it runs on.

Mocking the boundary is forbidden (novox/hq ADR 0034). Every detector is exercised against a fake runner for its logic and against this machine for its behaviour. The tests do not assert which capabilities a machine has — that varies, and is the point of detecting — they assert that detection tells the truth about whatever is there.

Where the reasoning lives

Design and decisions are in novox/hq, not here. This repository carries implementation and does not carry decisions.

  • 03-DESIGN/01-to-be/05-the-node-host.md — what this is and the order it is built in
  • 02-DECISIONS/0037-the-host-applies-it-does-not-decide.md — the one concern
  • 02-DECISIONS/0038-a-node-joins-by-linking-first.md — one behaviour, two sources
  • 02-DECISIONS/0039-the-link-is-the-security-boundary.md — a node owns no password
  • 02-DECISIONS/0041-the-host-depends-on-nothing.md — why this is a static binary, and Go
  • 04-ISSUES/007-an-installed-package-is-not-a-capability — why detection works this way
S
Description
Novox Mesh — tier 0. One statically linked binary that requires nothing present. Applies declared state on a machine; decides nothing.
Readme
3.3 MiB
Languages
Go 98.2%
Shell 1.5%
Makefile 0.3%