--- status: resolved opened: 2026-10-02 located-in: [mesh-host internal/apply (removeOrphan: a former target of a kind with no removal was fatal), mesh-host internal/store (Record keeps a former target for every kind, the host's own archive included)] fixed-by: mesh-host 65 — a former target of a kind the host cannot remove is left in place, said and forgotten; a dropped archive still refuses amended-design: --- # 194 — The host's own former archive stops every machine applying anything ## What was observed 2026-10-02, 00:34Z, on all four machines of this mesh, the first time a host carrying former targets ([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 5, built in mesh-host 63) replaced itself with a newer host (mesh-host 64). The host delivers its own successor as an archive whose target is a versioned directory ([ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md)): every new version is the same resource with a new target. Since mesh-host 63 the record keeps a resource's former target so the next apply removes what the host wrote under it ([issue 097](../097-a-resource-that-changes-target-leaves-the-old-one-behind/00-report.md)). So the new host's first apply found the previous version's directory as a former target of its own archive, and asked the removal for an archive — which does not exist ([issue 162](../162-an-archive-cannot-be-undeclared/00-report.md)): ``` applying "mesh-host.next@former:/usr/lib/nox-mesh-host/versions/3c906749ad27": no way to remove a "archive" 0 resource(s) were applied and remain ``` Orphans are removed before any resource is applied on a converged machine, so the refusal ended every apply at its first step. Every machine reported `failed`, applied nothing, and would have gone on doing so: a host fix is itself an archive the same apply would have to write, and the apply never reached it. The machines kept running what they had; nothing new from the mesh could land. ## Why it matters beyond this instance Two rules that are each right met in the one resource the host cannot afford to stop on. Rule 5 says a former target is removed and said; issue 162 says an archive has no removal, deliberately, so an unassignment nothing can undo is never reported as done. Neither rule was wrong; their meeting was never tested, because the bed that would have found it is a host replacing itself under the new rule, and the first such replacement was the live one. The fix is narrow: a former target of a kind the host cannot remove is left in place, said, and forgotten — never fatal, because nobody dropped it. An archive the declaration dropped still refuses, as 162 has it. ## What it took to recover The broken host cannot apply its own fix: the fix is delivered as an archive, and the apply fails before writing anything. On each machine the host's record (`/var/lib/mesh-host/state.json`) had to lose the one `@former:` entry by hand, once, so that the next push could write the fixed archive and stand aside for it. A manual edit of the host's record is otherwise never done; it is written here because the alternative was four machines that could apply nothing. ## Open questions - Should `Record` keep a former target for a kind the host cannot remove at all? The trace is useful; the removal it implies is not. Keeping it and letting the apply forget it is what the fix does; not recording it would be quieter. - Should the host's own versions directory be cleaned by the launcher rather than by the apply — the one archive whose former targets are genuinely removable, by the thing that knows which one runs? - Is there a bed that replaces a host under the current rules before the live mesh does (the proof row of [ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md) was a single crossover, before former targets existed)? ## Resolved, 2026-10-02 mesh-host 65, merged 07:35Z. Recovered as the record above says: the operator dropped the one `@former:` entry from each machine's host record and pushed; the fixed host then ran on all four and its first apply said `forgotten mesh-host.next@former:… a former target left in place` and applied the rest. The open questions stand as questions for the host's own versions, not as faults.