novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Per-OS hosts, and an init asked for only start and restart

Browse Source 
0060 -- the host is built per operating system. systemd and pacman are the Arch
host's implementation, not abstractions the mesh has to grow. They are not
independent choices: a machine has pacman because it is Arch, and the package
manager, service manager and packaging format arrive together as one decision
somebody made at install time.

Rejected abstracting them, and the reason is correctness rather than effort.
The service applier reads LoadState to tell "not installed" apart from
"stopped", which is what stops it reporting absence as success. An interface
spanning systemd and OpenRC degrades to what both express, and the lowest
common denominator is exactly where that fault lives.

Almost all of it is shared -- the vocabulary, store, apply loop, read-back
discipline, refusal model, bundle and link are portable. Two appliers differ.
And delivery was already per-OS, since a .pkg.tar.zst is an Arch artifact, so
this is the seam that already existed.

Android is the interesting case rather than Debian: no service manager, no
package installation, usually no root. Such a host implements file, directory
and action and refuses the rest -- the same refusal a host already gives an
unknown type, with a different reason. Those three are the portable floor.

The container runtime is deliberately left open: it is not an OS split, since
Arch runs docker or podman.

0061 -- the init is asked for start-at-boot and restart-on-exit, and nothing
else. Both are expressible in OpenRC, runit, s6 and an Android init.rc.
Counting failed starts and rolling back moves into a launcher, because that is
the one piece which must work when the host does not, and a script with a
counter can be tested where OnFailure= can only be hoped for. Supersedes 0059,
keeping its reasoning in full.

The checker found all six places citing 0059 and refused the commit until they
named the replacement.
This commit is contained in:
jschoubben
2026-08-27 23:46:11 +02:00
parent dcc4b8339c
commit e1ad39b500
5 changed files with 267 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
02-DECISIONS/0059-a-host-that-cannot-start-rolls-itself-back.md
+2 -1
View File
@@ -1,5 +1,6 @@
---
status: accepted
status: superseded
superseded-by: 02-DECISIONS/0061-the-host-asks-an-init-for-start-and-restart.md
date: 2026-08-27
deciders: jochen
reconstructed: false
02-DECISIONS/0060-the-host-is-built-per-operating-system.md
+112
View File
@@ -0,0 +1,112 @@
---
status: accepted
date: 2026-08-28
deciders: jochen
reconstructed: false
extends: 0057-the-host-is-a-root-service-installed-as-a-package.md
---
# 60. The host is built per operating system
## Context
Three of the host's six shapes need something from the machine: `service` needs a service
manager, `package` needs a package manager, `container` needs a container runtime. The other
three — `file`, `directory`, `action` — need only a filesystem and the ability to run something.
The host names those capabilities generically and implements them specifically. The detector
reports `container-runtime`, `package-manager`, `service-manager`; the appliers call `docker`,
`pacman` and `systemctl`. **So the design says capability and the code says Arch**, and nothing
records which of those is the intent.
The question that surfaced it: what happens on a machine that has podman, or one that does not
run systemd? And underneath it, a live contradiction —
[research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) decides that on
conflict during adoption *the machine's configuration is kept*, so a machine with podman keeps
podman, and then the container applier calls `docker` and fails.
## Considered options
1. **Abstract each capability behind an interface.** One host, adapters per service manager and
package manager. Rejected, and the reason is correctness rather than effort: the service
applier reads `LoadState` to tell *not installed* apart from *stopped*, which is what stops it
reporting absence as success. An interface spanning systemd and OpenRC degrades to what both
can express, and **the lowest common denominator is exactly where that fault lives**.
2. **Support one operating system and say so.** Honest, and it makes every other machine
permanently out of scope rather than not-yet.
3. **A host per operating system.** Chosen.
## Decision
**The host is built for an operating system family, and `systemd` and `pacman` are the Arch
host's implementation rather than abstractions the mesh has to grow.**
```
mesh-host-arch-x86_64 pacman · systemctl · docker
mesh-host-debian-x86_64 apt · systemctl · docker (when there is a machine)
mesh-host-alpine-x86_64 apk · rc-service · podman (when there is a machine)
```
**These are not independent choices and treating them as such was the error.** A machine has
pacman *because* it is Arch. The package manager, the service manager and the packaging format
arrive together, as one decision somebody made when they installed the operating system.
### Almost all of it is shared
Not a rewrite per operating system. The declaration vocabulary, the store, the apply loop, the
read-back discipline, the refusal model, the bundle and the link are all portable. **What differs
is two appliers**, and the rest is compiled around them.
### The control plane names the package, because the host does not decide
A container runtime is `docker` on Arch and `docker.io` on Debian. Mapping *this node needs a
container runtime* to a package name is **deciding**, which
[ADR 0037](0037-the-host-applies-it-does-not-decide.md) puts outside the host.
It needs no new mechanism: the profile already reports what the machine is, so the declaration a
node receives is already tailored to that node. The host receives a package name and installs it.
### A host that cannot implement a shape refuses it
The interesting case is not Debian, it is **Android** — no service manager it will lend us, no
package installation, usually no root. Such a host implements `file`, `directory` and `action`,
and nothing else.
That needs no new mechanism either. A host already refuses a type it does not know; *this host
does not implement `package`* is the same refusal with a different reason, and the profile
reports which shapes it implements so the control plane never sends one it cannot do.
**`file`, `directory` and `action` are the portable floor.** They work anywhere there is a
filesystem and a way to run something, which makes a partial host a real thing rather than a
broken one.
## Consequences
- **Each implementation stays as sharp as its operating system allows.** The `LoadState`
distinction survives because the Arch host knows it is systemd. Nothing is degraded to fit an
interface spanning systems we do not run.
- **Delivery already worked this way**, which is the strongest sign this is the right seam: the
host ships as a package from the mesh's own repository
([ADR 0058](0058-delivery-ends-in-a-declaration.md)), and a `.pkg.tar.zst` is an Arch artifact.
A per-OS binary is consistent with what was already decided rather than an addition to it.
- **The container runtime is left unresolved on purpose**, because it is not an OS split: Arch
runs docker or podman. Per-OS hosts answer the service and package managers and leave the
runtime a genuine choice *within* a host. That is a separate decision.
- **A second operating system is now additive rather than a redesign** — write two appliers, ship
a package. And it will be designed against a real machine rather than a guess, which is the
point of not building the abstraction now.
- **The profile has to report the operating system**, and today it reports capabilities without
saying which system they belong to. Small, and needed before the control plane can tailor a
package name.
- **Nothing states the machine requirements yet.** A machine missing a capability fails at apply
time rather than being refused up front, even though the host already detects it. That is a
gap this record makes visible and does not close.
## References
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why the host is handed a package name.
- [ADR 0041](0041-the-host-depends-on-nothing.md) — one static binary, now per system as well as
per architecture.
- [ADR 0058](0058-delivery-ends-in-a-declaration.md) — delivery, which was already per-OS.
- [Research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) — adoption keeping
the machine's configuration, which hardcoding a runtime contradicts.
02-DECISIONS/0061-the-host-asks-an-init-for-start-and-restart.md
+109
View File
@@ -0,0 +1,109 @@
---
status: accepted
date: 2026-08-28
deciders: jochen
reconstructed: false
supersedes: 0059-a-host-that-cannot-start-rolls-itself-back.md
extends: 0060-the-host-is-built-per-operating-system.md
---
# 61. The host asks an init for start and restart, and nothing else
## Context
[ADR 0059](0059-a-host-that-cannot-start-rolls-itself-back.md) built the node's recovery out of
systemd's own features: `StartLimitBurst` to decide a binary is broken, `OnFailure` to run a
rollback unit. It works, and it makes recovery — the thing that matters most when a node is
stuck — the most systemd-specific part of the whole host.
[ADR 0060](0060-the-host-is-built-per-operating-system.md) makes the host per operating system,
which raises the obvious question: how much of an init does the host actually need?
Counted honestly, three things, and only one of them is special:
| | any init? |
|---|---|
| start at boot | **yes** |
| restart it when it exits | **yes** |
| give up after N failures and run something else | **no** — that is systemd's `StartLimitBurst` and `OnFailure` |
So the recovery mechanism is the only reason the host needs *this* init rather than *an* init.
And it is the part that must work on a machine where the host does not, which makes "it is
expressed in unit-file syntax" a poor place for it: unit syntax is not something we can test, and
the one time it runs is the one time nobody can afford it to be wrong.
**The substrate does not need systemd either**, which is what makes this worth doing rather than
merely tidy. The bootstrap is package → container → action → container, and every mesh workload
is a container the runtime restarts. Nothing in it declares a `service`.
## Decision
**An init is asked for two things: start this at boot, and start it again if it exits.** Both are
expressible in systemd, OpenRC, runit, s6 and an Android `init.rc`.
**Everything else moves into a launcher**, which is what the init actually starts:
```
init ──► nox-mesh-host-launch ──► nox-mesh-host
│
├─ halted? say so and stop. a person has to look
├─ count this start attempt
├─ too many, and not yet rolled back? roll back, then start
├─ too many, and already rolled back? halt — the machine is the problem
└─ otherwise start the host
host, on a completed reconcile ──► clears the counter, records known-good
```
**This keeps everything ADR 0059 decided and changes only where it lives.** Two watchdogs still,
and neither substitutes for the other: the mesh stages a host rollout and stops when nodes go
quiet; the node recovers itself. Recovery is still local, because nothing dials a node and a host
that cannot start cannot report. It rolls back once, because a second failure of a
previously-working binary is a different diagnosis. The rollback still shares no code with the
host, because a binary that will not start cannot be its own recovery.
### What is gained by moving it
- **It becomes testable.** A shell script with a counter can be run against a stub package
manager and asserted, which is how the rollback script is already tested. `OnFailure=` can be
read and hoped for.
- **The host becomes runnable under any init**, which is what makes an Alpine or Android host
possible later rather than blocked on porting the recovery.
- **The give-up policy stops being configuration and becomes code we own.** Three attempts is a
decision, and it should live where decisions are read and tested.
### What it costs
- **One more process in the chain**, and it runs before the host on every start.
- **The counter is the whole mechanism, and it is the part to get right.** Never cleared, and
the node rolls back on a healthy boot; cleared too eagerly, and it never rolls back at all.
It is cleared by the host on a **completed reconcile** — the same event that records
known-good, for the same reason.
- **The launcher is a thing that can itself be broken**, and nothing recovers it. That is one
turtle down from where we were, not zero: the alternative was unit syntax, which also cannot
recover itself and additionally cannot be tested.
## Consequences
- **`Restart=always` stays load-bearing and stays subtle.** The host restarts onto a new binary
by exiting cleanly ([ADR 0057](0057-the-host-is-a-root-service-installed-as-a-package.md)), so
whatever supervises must restart on a zero exit. This caught out an earlier draft of 0059,
which specified `on-failure` and would have left every upgraded node stopped.
- **The unit file becomes trivial**, which is the point: start, restart, a state directory.
Nothing in it encodes policy, so porting it is transcription rather than design.
- **A halted node is silent**, unchanged from 0059 and still the last gap. What notices is the
mesh seeing a node it has not heard from.
- **The rollback script grows into a launcher** rather than being replaced. Its tested behaviour —
roll back once, refuse to guess with no known-good, fail loudly when the package is not
cached — carries over and is where the new counter logic joins it.
- **`service` survives as a shape**, and this record does not remove it. Almost nothing in the
design declares one, but adoption takes over machines already in use whose units somebody
chose, and saying the mesh may never manage those is a larger decision than this one.
## References
- [ADR 0059](0059-a-host-that-cannot-start-rolls-itself-back.md) — superseded; its reasoning
about two watchdogs, rolling back once, and recovery being local is kept in full.
- [ADR 0060](0060-the-host-is-built-per-operating-system.md) — why this question was asked.
- [ADR 0057](0057-the-host-is-a-root-service-installed-as-a-package.md) — restart-by-exiting,
which constrains what the init must do.