hq/02-DECISIONS/0061-the-host-asks-an-init-for-start-and-restart.md

status, date, deciders, reconstructed, supersedes, extends

status	date	deciders	reconstructed	supersedes	extends
accepted	2026-08-28	jochen	false	0059-a-host-that-cannot-start-rolls-itself-back.md	0060-the-host-is-built-per-operating-system.md

61. The host asks an init for start and restart, and nothing else

Context

ADR 0059 built the node's recovery out of systemd's own features: StartLimitBurst to decide a binary is broken, OnFailure to run a rollback unit. It works, and it makes recovery — the thing that matters most when a node is stuck — the most systemd-specific part of the whole host.

ADR 0060 makes the host per operating system, which raises the obvious question: how much of an init does the host actually need?

Counted honestly, three things, and only one of them is special:

	any init?
start at boot	yes
restart it when it exits	yes
give up after N failures and run something else	no — that is systemd's StartLimitBurst and OnFailure

So the recovery mechanism is the only reason the host needs this init rather than an init. And it is the part that must work on a machine where the host does not, which makes "it is expressed in unit-file syntax" a poor place for it: unit syntax is not something we can test, and the one time it runs is the one time nobody can afford it to be wrong.

The substrate does not need systemd either, which is what makes this worth doing rather than merely tidy. The bootstrap is package → container → action → container, and every mesh workload is a container the runtime restarts. Nothing in it declares a service.

Decision

An init is asked for two things: start this at boot, and start it again if it exits. Both are expressible in systemd, OpenRC, runit, s6 and an Android init.rc.

Everything else moves into a launcher, which is what the init actually starts:

init ──► nox-mesh-host-launch ──► nox-mesh-host
              │
              ├─ halted?         say so and stop. a person has to look
              ├─ count this start attempt
              ├─ too many, and not yet rolled back?   roll back, then start
              ├─ too many, and already rolled back?   halt — the machine is the problem
              └─ otherwise                            start the host

host, on a completed reconcile ──► clears the counter, records known-good

This keeps everything ADR 0059 decided and changes only where it lives. Two watchdogs still, and neither substitutes for the other: the mesh stages a host rollout and stops when nodes go quiet; the node recovers itself. Recovery is still local, because nothing dials a node and a host that cannot start cannot report. It rolls back once, because a second failure of a previously-working binary is a different diagnosis. The rollback still shares no code with the host, because a binary that will not start cannot be its own recovery.

What is gained by moving it

• It becomes testable. A shell script with a counter can be run against a stub package manager and asserted, which is how the rollback script is already tested. OnFailure= can be read and hoped for.
• The host becomes runnable under any init, which is what makes an Alpine or Android host possible later rather than blocked on porting the recovery.
• The give-up policy stops being configuration and becomes code we own. Three attempts is a decision, and it should live where decisions are read and tested.

What it costs

• One more process in the chain, and it runs before the host on every start.
• The counter is the whole mechanism, and it is the part to get right. Never cleared, and the node rolls back on a healthy boot; cleared too eagerly, and it never rolls back at all. It is cleared by the host on a completed reconcile — the same event that records known-good, for the same reason.
• The launcher is a thing that can itself be broken, and nothing recovers it. That is one turtle down from where we were, not zero: the alternative was unit syntax, which also cannot recover itself and additionally cannot be tested.

Consequences

• Restart=always stays load-bearing and stays subtle. The host restarts onto a new binary by exiting cleanly (ADR 0057), so whatever supervises must restart on a zero exit. This caught out an earlier draft of 0059, which specified on-failure and would have left every upgraded node stopped.
• The unit file becomes trivial, which is the point: start, restart, a state directory. Nothing in it encodes policy, so porting it is transcription rather than design.
• A halted node is silent, unchanged from 0059 and still the last gap. What notices is the mesh seeing a node it has not heard from.
• The rollback script grows into a launcher rather than being replaced. Its tested behaviour — roll back once, refuse to guess with no known-good, fail loudly when the package is not cached — carries over and is where the new counter logic joins it.
• service survives as a shape, and this record does not remove it. Almost nothing in the design declares one, but adoption takes over machines already in use whose units somebody chose, and saying the mesh may never manage those is a larger decision than this one.

References

• ADR 0059 — superseded; its reasoning about two watchdogs, rolling back once, and recovery being local is kept in full.
• ADR 0060 — why this question was asked.
• ADR 0057 — restart-by-exiting, which constrains what the init must do.