Issue 194: the host's own former archive stops every machine applying anything

Rule 5 of ADR 0163 (a former target is removed) met issue 162 (an archive has no removal) in the host's own archive, the first time a host carrying former targets replaced itself; every machine applied nothing from then on.
2026-10-02 02:42:19 +02:00
13 changed files with 71 additions and 129 deletions
@@ -131,32 +131,6 @@ the plan says it too.
 | Genesis raises the forge as its module declares it | a genesis test that raises, assigns, and finds the module holding rather than raising a second |
 | Live | the next cutover on an adopted machine: `take` shows the comparison, refuses the downgrade if there is one, and the service keeps its configuration and its secret |
 ## Built, 2026-10-02
 > **Progressive insight — 2026-10-02.** The decision stands; these are the facts of its building.
 Built across mesh-host 63 and 64 and mesh-controller 201, 202 and the pull request that followed
 them. Rule 1: `take` previews every held thing's comparison and ends with a digest; `take --yes
 <digest>` acts on exactly that preview, and a changed preview or an account older than the flip
 allows is refused, as the flip's are. A published port's reach is said as the machine reported it,
 behind the found firewall whose rules are not read. Rule 2: an older image, a differing file and a
 minted, unaccepted secret for found data refuse, overridden by `--downgrade`, `--replace <path>` and
 `--mint <name>`; the secrets a module holds on a machine are read with where each came from. Rule 3:
 `secret accept --provider` reaches a required secret. Rule 4: the per-machine setting is `networks`,
 a container id to the found networks it keeps; judged for an adopted machine only, joined by the host
 after the container runs, part of the container's spec, named in the preview. Rule 5: the host's
 facts, former targets and strays. Rule 6: one judgement, run where a setting is stored and where a
 machine is composed; a module whose stored setting its definition can no longer compose is left out
 of the declaration, the envelope says so, the host keeps that module's things, and `plan` and `push`
 say it by name. A key that reaches nothing is refused where stored and said by `plan`, and never
 costs a module. Rule 7: genesis raises the forge under the module's container name, with its image
 digest and its data directory; the network is the one difference left, said by the take, because the
 bootstrap forge reaches the store on the machine's loopback.
 **Not yet proven live.** Every machine of this mesh is converged, so the table's last row — a take
 on an adopted machine — waits for the next adoption. What is live is what the rows above it check.
 Issues 086, 098, 099, 100 and 101 stay located until that row is read.
 ## References
 - [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md), [ADR 0103](0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md), [ADR 0104](0104-a-provision-may-be-answered-by-an-adapter-to-the-predecessor.md), [ADR 0162](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)
@@ -2,7 +2,7 @@
 layer: to-be
 status: in-progress
 code: [mesh-host]
-updated: 2026-10-02
+updated: 2026-10-01
 decisions:
  - 02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md
  - 02-DECISIONS/0141-the-host-delivers-its-own-successor.md
@@ -163,19 +163,6 @@ a resource's former targets, removes a container or file it wrote under a name t
 longer names, never removes what was found, and reports what runs on the machine that it neither
 wrote nor holds. *How it is checked:* ADR 0163's table.
 **What the host joins, keeps and raises for a take** — revision, 2026-10-02
 ([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 4, 6 and 7). A
 container may name networks it also joins once it runs — the found network a per-machine setting keeps
 for a taken container while a neighbour still resolves it there; joined after the run, part of the
 container's spec, refused when it cannot be joined. A declaration may name the modules the mesh left
 out of it because a stored setting cannot compose with the module's definition: the host keeps what it
 wrote and holds for a left-out module and says so, where absence used to read as removal. And genesis
 raises the bootstrap forge under the forge module's container name, with the module's image digest and
 its data directory, so the module holds it by the found rule; the network is the one difference a take
 has left to say. *How it is checked:* a host test joins a kept network and refuses one it cannot; a
 host test keeps a left-out module's record and hold and removes an absent module's; a bootstrap test
 holds the installer's constants to the module's manifest where the catalogue is checked out beside it.
 **Found reaches every kind that can touch what the machine has**
 ([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)). For a module not yet taken, a directory present with no record
 keeps its mode and owner, a unit present with no record keeps its state and boot setting, a
@@ -8,7 +8,7 @@ code:
  - mesh-host packaging/nox-mesh-host-network.sh
  - mesh-controller internal/token
  - mesh-controller internal/inventory/nodes.go
-updated: 2026-10-02
+updated: 2026-10-01
 decisions:
  - 02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md
  - 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
@@ -323,21 +323,6 @@ network are said. `take --yes <digest>` cuts over what was previewed, as the fli
 container may keep a found network by a per-machine setting while its neighbours are not yet taken.
 *How it is checked:* ADR 0163's table.
 **A setting is judged where it is stored, and the take's words** — revision, 2026-10-02
 ([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1, 2, 4 and 6).
 The preview ends with a digest of what it said; `take --yes <digest>` acts on that preview and nothing
 else, and a preview that has changed since, or an account of the machine older than the flip allows, is
 refused as the flip's is. A module the machine holds nothing for has nothing to compare, and `--yes`
 suffices. The overrides are `--downgrade`, `--replace <path>` and `--mint <name>`; the per-machine
 setting that keeps a found network is `networks`, a container id to the networks it keeps, accepted
 for an adopted machine only. Storing a setting composes it against the module's current definition and
 refuses, naming node, module, layer and key, what cannot compose or reaches nothing. A definition that
 later moves under a stored setting costs that module its place in the machine's declaration, said by
 name in `plan`, `push` and the declaration itself, and the machine is told everything else; a stray
 setting no longer refuses the machine where it is read. *How it is checked:* controller tests over the
 one judgement — refused where stored, a module left out where composed, the envelope naming it — and
 over a take's digest, staleness and secrets.
 A candidate machine is not empty. It has a package manager, probably a container runtime,
 configuration somebody chose. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
 says the host never touches what it did not create — adoption is the deliberate act of taking
@@ -40,9 +40,3 @@ it changes before it changes it, and for taking a module this one does not.
 [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the preview names a narrowing. Building follows,
 host first, then the controller's `take`.
 ## Built, 2026-10-02
 mesh-controller 201 and the pull request after it: the preview names it, and `take --yes <digest>`
 acts on the preview that was read. Stays located until a take is read on an adopted machine — every
 machine of this mesh is converged today, so the record's live row has not been run.
@@ -53,13 +53,3 @@ network, or it is not a takeover.
 [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 7: genesis raises as the module declares. Building follows,
 host first, then the controller's `take`.
 ## Built in part, 2026-10-02
 mesh-host 64: genesis raises the forge under the module's container name (`gitea`), pinned to the
 module's image digest, with the module's data directory mounted at `/data` — so the module finds it,
 holds it, and a take compares equal images and the same data. A test holds the installer's constants
 to the module's manifest where the catalogue is checked out beside it. The network is the difference
 left: the bootstrap forge runs on the machine's network to reach the store on its loopback, the module
 runs bridged and publishes its ports, and the take says so. Closing waits for group 9's genesis test —
 a mesh raised, the module assigned, and the module found holding rather than raising a second forge.
@@ -1,8 +1,8 @@
 ---
-status: resolved
+status: located
 opened: 2026-09-23
 located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
-fixed-by: mesh-controller (the pull request after 201: JudgeSettings, LeftOut), mesh-host 64 (left_out kept)
+fixed-by:
 amended-design:
 ---
@@ -63,12 +63,3 @@ knowing the code.
 [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 6: judged where stored; an impossible statement costs a module. Building follows,
 host first, then the controller's `take`.
 ## Resolved, 2026-10-02
 One judgement, in the catalogue, run where a setting is stored and where a machine is composed. Stored,
 a setting that cannot compose with the module's current definition is refused naming the node, the
 module, the layer and the key; a key that reaches nothing is refused there too. Composed, a definition
 that moved under a stored setting leaves that module out of the machine's declaration — the envelope
 names it, the host keeps what it holds and wrote for it, `plan` and `push` say it — and the machine is
 told everything else. A stray setting no longer refuses the whole machine where it is read.
@@ -1,8 +1,8 @@
 ---
-status: resolved
+status: located
 opened: 2026-09-23
 located-in: [mesh-controller cmd/mesh-controller/adoption.go (take previews nothing), mesh-host internal/apply (the comparison and the record)]
-fixed-by: mesh-host 63 (former targets removed, strays reported), mesh-controller 201/202 (strays shown)
+fixed-by:
 amended-design:
 ---
@@ -80,11 +80,3 @@ found, and so would be kept for ever on purpose.
 [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 5: former targets are removed and strays reported. Building follows,
 host first, then the controller's `take`.
 ## Resolved, 2026-10-02
 mesh-host 63: the host's record keeps a resource's former targets, removes a container or file it
 wrote under a name the declaration no longer names, never what was found, and reports strays — what
 runs on the machine that the mesh neither wrote nor holds. mesh-controller 201 and 202 show strays
 on `node show` for an adopted and a converged machine alike; the live mesh reported four on the
 control node the evening it rolled.
@@ -68,9 +68,3 @@ written.
 [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the difference is shown and a differing file refuses. Building follows,
 host first, then the controller's `take`.
 ## Built, 2026-10-02
 mesh-host 63 reports the difference between the kept original and the declared content; mesh-controller
 201 shows it in the preview and refuses a differing file unless `--replace <path>` names it, or the
 module declares the file partially. Stays located until a take is read on an adopted machine.
@@ -65,9 +65,3 @@ expected rate.
 [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 2: the images are compared by age and a downgrade refuses. Building follows,
 host first, then the controller's `take`.
 ## Built, 2026-10-02
 mesh-host 63 reports the found image and both images' creation dates; mesh-controller 201 says
 DOWNGRADE and refuses unless `--downgrade` is said. Stays located until a take is read on an adopted
 machine.
@@ -70,11 +70,3 @@ the module can only be installed fresh.
 [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 2 and 3: a minted secret for found data refuses; secret accept reaches required secrets. Building follows,
 host first, then the controller's `take`.
 ## Built, 2026-10-02
 `secret accept <node> <module> <name> --provider <node>` reaches a required secret (mesh-controller 201).
 The pull request after it reads every secret a module holds on a machine with its origin, and a take
 of a module whose data was found refuses a minted, unaccepted one — naming the accept that carries
 the existing value in, or `--mint <name>` to let the service take the new one. Stays located until
 a take is read on an adopted machine.
@@ -66,10 +66,3 @@ exercise.
 [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 1 and 4: the neighbours are named; a found network may be kept by a setting. Building follows,
 host first, then the controller's `take`.
 ## Built, 2026-10-02
 The preview names every neighbour on a found network (mesh-controller 201). The pull request after it
 adds the per-machine setting `networks` — a container id to the found networks it keeps — judged for an
 adopted machine only, and mesh-host 64 has the taken container join each once it runs. Stays located
 until a take is read on an adopted machine.
@@ -1,9 +1,7 @@
 ---
-status: resolved
+status: located
 opened: 2026-09-26
 located-in: [mesh-host internal/apply]
 fixed-by: mesh-host 63 (every written field compared), mesh-controller 201 (build says the policy)
 amended-design:
 ---
 # 126 — a volume path is not in the spec comparison, and a roll-out raced a data move
@@ -52,9 +50,3 @@ the install-page junk was discarded twice.
 [ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rules 5 and 7: every field compared; build says the policy. Building follows,
 host first, then the controller's `take`.
 ## Resolved, 2026-10-02
 mesh-host 63: every field the host writes is compared before a container is called current, volumes
 and paths included. mesh-controller 201: `build` and the take-in say when a module's policy rolls a
 result out at once; under ADR 0162 the plan says it too.
@@ -0,0 +1,64 @@
 ---
 status: located
 opened: 2026-10-02
 located-in: [mesh-host internal/apply (removeOrphan: a former target of a kind with no removal was fatal), mesh-host internal/store (Record keeps a former target for every kind, the host's own archive included)]
 fixed-by:
 amended-design:
 ---
 # 194 — The host's own former archive stops every machine applying anything
 ## What was observed
 2026-10-02, 00:34Z, on all four machines of this mesh, the first time a host carrying former
 targets ([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 5,
 built in mesh-host 63) replaced itself with a newer host (mesh-host 64).
 The host delivers its own successor as an archive whose target is a versioned directory
 ([ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md)): every new version
 is the same resource with a new target. Since mesh-host 63 the record keeps a resource's former
 target so the next apply removes what the host wrote under it
 ([issue 097](../097-a-resource-that-changes-target-leaves-the-old-one-behind/00-report.md)). So
 the new host's first apply found the previous version's directory as a former target of its own
 archive, and asked the removal for an archive — which does not exist
 ([issue 162](../162-an-archive-cannot-be-undeclared/00-report.md)):
 ```
 applying "mesh-host.next@former:/usr/lib/nox-mesh-host/versions/3c906749ad27": no way to remove a "archive"
 0 resource(s) were applied and remain
 ```
 Orphans are removed before any resource is applied on a converged machine, so the refusal ended
 every apply at its first step. Every machine reported `failed`, applied nothing, and would have
 gone on doing so: a host fix is itself an archive the same apply would have to write, and the apply
 never reached it. The machines kept running what they had; nothing new from the mesh could land.
 ## Why it matters beyond this instance
 Two rules that are each right met in the one resource the host cannot afford to stop on. Rule 5
 says a former target is removed and said; issue 162 says an archive has no removal, deliberately,
 so an unassignment nothing can undo is never reported as done. Neither rule was wrong; their
 meeting was never tested, because the bed that would have found it is a host replacing itself
 under the new rule, and the first such replacement was the live one. The fix is narrow: a former
 target of a kind the host cannot remove is left in place, said, and forgotten — never fatal,
 because nobody dropped it. An archive the declaration dropped still refuses, as 162 has it.
 ## What it took to recover
 The broken host cannot apply its own fix: the fix is delivered as an archive, and the apply fails
 before writing anything. On each machine the host's record (`/var/lib/mesh-host/state.json`) had to
 lose the one `@former:` entry by hand, once, so that the next push could write the fixed archive and
 stand aside for it. A manual edit of the host's record is otherwise never done; it is written here
 because the alternative was four machines that could apply nothing.
 ## Open questions
 - Should `Record` keep a former target for a kind the host cannot remove at all? The trace is
  useful; the removal it implies is not. Keeping it and letting the apply forget it is what the fix
  does; not recording it would be quieter.
 - Should the host's own versions directory be cleaned by the launcher rather than by the apply —
  the one archive whose former targets are genuinely removable, by the thing that knows which one
  runs?
 - Is there a bed that replaces a host under the current rules before the live mesh does
  (the proof row of [ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md) was
  a single crossover, before former targets existed)?