hq/02-DECISIONS/0141-the-host-delivers-its-own-successor.md

topic, status, date, deciders, reconstructed, extends

topic	status	date	deciders	reconstructed	extends
what runs on it	accepted	2026-09-29	jochen	false	02-DECISIONS/0005-the-node-host.md

141. The host delivers its own successor, and versions live side by side

Context

Issue 142. A merge builds every changed module and the control plane — which is itself a module — and the result reaches the machines running it with nobody asking. The host is the exception: it is not a build target, no declaration delivers it, and every machine in this mesh runs a byte-identical binary that somebody built on a workstation and copied out.

The half that recovers from a bad host exists. internal/upgrade can tell that the executable this process started from was replaced on disk, and it records which version last completed a reconcile. The launcher counts consecutive failed starts, calls a rollback at the limit, and treats a clean exit as the host standing aside so that the next loop runs whatever is on disk now. That supervision is complete and correct.

Two things make it dead code:

• Replaced() is called by nothing but its own tests. Nothing tells the running host that a successor is waiting.
• The rollback resolves a version through the machine's package manager — pacman -U from the package cache. No machine here has the host installed as a package, so the recovery cannot run on any of them; and being written in one package manager's terms, it cannot run on two of the three operating systems the host is built for — ADR 0005 builds one binary per operating system, pinned at link time.

The record already points at the answer. What is kept is a version, not a path. Keeping a version is only useful to something that can choose between versions present on the machine, which is what the package manager was being asked to do. The versions can simply be on disk.

Considered Options

1. Deliver the host as a package, as the rollback assumes. Rejected: it needs a package built and a repository trusted per operating system, three of each, and the existing package resource asserts presence and deliberately never a version — "version is the package manager's business and the mesh does not hold a second opinion about it" — so it cannot ask for a particular host anyway. Heaviest of the three and the only one that is different on every machine.
2. Write the new binary over the running one. Rejected on a fact: a running executable cannot be truncated, and archive opens what it unpacks with O_TRUNC. It could be made to write and rename, which is better hygiene and worth doing for its own sake, but it buys nothing here that option 3 does not, and it leaves rollback with nowhere to go back to.
3. Versions side by side; the newest retires the old. Adopted.

Decision

A host version is delivered as an archive into a directory named for it, and never over a running one. The declaration names it like any other archive — fetched by digest, the digest checked before anything is unpacked. Nothing new travels, no new resource kind, and no change to how archives are applied, because the path being written is not the path being executed.

The launcher starts the most recently delivered version. That is what "the newest" means: the version whose directory arrived last. It reads no pointer and follows no link — the mesh creates no links (ADR 0012) — and the version is in the path, so nothing has to be told what is running.

The running host stands aside for a successor, and only between reconciles. Finding a newer version delivered, it finishes the reconcile it is in and exits cleanly. The launcher already reads a clean exit as exactly this and starts what is on disk now. A host that stood aside mid-apply is the half-configured machine this project exists to prevent, so the check happens at the boundary and nowhere else.

A version that completes a reconcile records itself, and retires what came before it. The known-good record is written as it is today. Then versions older than the one before the running one are removed: the running version and its predecessor are kept, which is exactly what a rollback needs, and nothing else accumulates.

Rollback starts the previous version instead of reinstalling a package. At the failure limit the launcher pins the known-good version and starts that, once. The second failure is still a different diagnosis — the previously working version does not run either, so it is the machine and not the binary — and the halt is unchanged. No package manager, no package cache, and the same script on every operating system.

A machine says which host version it is running, on the report it already sends, beside the other facts it states about itself. Without it nothing can say a machine is behind, so "every machine current with its source" cannot include the host.

Progressive insight — 2026-09-29, the same day

The delivery is not "nothing new", and this record said it was. The decision above stands and is built: versions side by side, the newest runs, the running host stands aside between reconciles, a completed reconcile retires what is older than the predecessor, rollback picks a directory. What was wrong was a claim about how a version reaches a machine. The paragraph on delivery said the declaration "names it like any other archive… nothing new travels, no new resource kind"; the second half is true and the first is not, because two things the delivery needs do not exist:

• Nothing can compile it. A bundle artifact is compiled by a closed list of toolchains — typescript and python — whose own comment says adding a language is a decision, because a language used by modules needs an SDK carrying the broker client, the event envelope and tool serving. The host uses none of that: it is what applies modules, not one of them. So the obligation that list warns about attaches to a module written in a language, not to the language being buildable, and the control plane — also written in Go — is built as an image from a Dockerfile rather than through a toolchain at all.
• A version cannot reach the path. An archive resource names a fixed path in the manifest, and nothing interpolates the built version into it, so nothing can ask for …/versions/<version>/.

Neither changes what was decided, which options were weighed, or any consequence: the shape is unaffected and the host half is merged and tested. What it changes is the cost, which this record understated as none. The remaining work is a way to build the host and a way to name a version in a path, and until both exist nothing delivers a version and every machine takes the fallback — which is what every machine does today.

Consequences

• The host becomes a build target and a module — a module whose resource is the next host, applied by the host that is running. The bootstrap is not circular because the two are different versions in different directories.
• Rollback becomes usable on every machine, having been usable on none. It also stops being written in one operating system's terms.
• One copy by hand remains, once. The first host that understands versioned directories cannot be fetched by a host that does not. That copy is the last, and it is the honest cost of the change rather than a step in the design.
• Two versions occupy disk instead of one. About nine megabytes. The predecessor is the price of a rollback that does not depend on a cache somebody else may clean.
• What got harder: a host must now be able to find its own successor and to judge when it is safe to stand aside. Both are between reconciles, which is the only moment the host is not mid-change.
• A machine that is never told a newer version keeps running what it has, indefinitely and visibly, because its report says which version that is.

How it is checked

• A delivered version is run, and the old one is not. A bed delivers a second version to a machine running the first; the host exits between reconciles, the launcher starts the new one, and the machine reports the new version. This fails against the previous behaviour, where nothing notices a delivered version at all.
• It stands aside between reconciles and never inside one. Asserted by delivering a version while an apply is in flight: the apply completes, and the exit follows it.
• A version that will not start is rolled back to its predecessor, once, and the second failure halts with the machine named rather than the binary — asserted with no package manager involved.
• A completed reconcile retires what is older than the predecessor, and never the predecessor itself, because that is what a rollback needs. Asserted on the directory afterwards.
• The report names the running version, asserted end to end rather than on the function that reads it, since the point is that the control plane can tell a machine is behind.
• The launcher picks the newest delivered version with no pointer file and no link, asserted by delivering two and checking which runs.

References

• ADR 0005 — the host, and what its supervision is for
• ADR 0010 — a declaration is owned resources; this adds no kind to it
• ADR 0012 — why the version is in the path
• ADR 0005, it is built per operating system — why a rollback written in one package manager's terms was wrong for two of three
• issue 142 — the measurement