72 lines
4.2 KiB
Markdown
72 lines
4.2 KiB
Markdown
---
|
|
status: resolved
|
|
opened: 2026-10-02
|
|
located-in: [mesh-host internal/apply (removeOrphan: a former target of a kind with no removal was fatal), mesh-host internal/store (Record keeps a former target for every kind, the host's own archive included)]
|
|
fixed-by: mesh-host 65 — a former target of a kind the host cannot remove is left in place, said and forgotten; a dropped archive still refuses
|
|
amended-design:
|
|
---
|
|
|
|
# 194 — The host's own former archive stops every machine applying anything
|
|
|
|
## What was observed
|
|
|
|
2026-10-02, 00:34Z, on all four machines of this mesh, the first time a host carrying former
|
|
targets ([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md), rule 5,
|
|
built in mesh-host 63) replaced itself with a newer host (mesh-host 64).
|
|
|
|
The host delivers its own successor as an archive whose target is a versioned directory
|
|
([ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md)): every new version
|
|
is the same resource with a new target. Since mesh-host 63 the record keeps a resource's former
|
|
target so the next apply removes what the host wrote under it
|
|
([issue 097](../097-a-resource-that-changes-target-leaves-the-old-one-behind/00-report.md)). So
|
|
the new host's first apply found the previous version's directory as a former target of its own
|
|
archive, and asked the removal for an archive — which does not exist
|
|
([issue 162](../162-an-archive-cannot-be-undeclared/00-report.md)):
|
|
|
|
```
|
|
applying "mesh-host.next@former:/usr/lib/nox-mesh-host/versions/3c906749ad27": no way to remove a "archive"
|
|
0 resource(s) were applied and remain
|
|
```
|
|
|
|
Orphans are removed before any resource is applied on a converged machine, so the refusal ended
|
|
every apply at its first step. Every machine reported `failed`, applied nothing, and would have
|
|
gone on doing so: a host fix is itself an archive the same apply would have to write, and the apply
|
|
never reached it. The machines kept running what they had; nothing new from the mesh could land.
|
|
|
|
## Why it matters beyond this instance
|
|
|
|
Two rules that are each right met in the one resource the host cannot afford to stop on. Rule 5
|
|
says a former target is removed and said; issue 162 says an archive has no removal, deliberately,
|
|
so an unassignment nothing can undo is never reported as done. Neither rule was wrong; their
|
|
meeting was never tested, because the bed that would have found it is a host replacing itself
|
|
under the new rule, and the first such replacement was the live one. The fix is narrow: a former
|
|
target of a kind the host cannot remove is left in place, said, and forgotten — never fatal,
|
|
because nobody dropped it. An archive the declaration dropped still refuses, as 162 has it.
|
|
|
|
## What it took to recover
|
|
|
|
The broken host cannot apply its own fix: the fix is delivered as an archive, and the apply fails
|
|
before writing anything. On each machine the host's record (`/var/lib/mesh-host/state.json`) had to
|
|
lose the one `@former:` entry by hand, once, so that the next push could write the fixed archive and
|
|
stand aside for it. A manual edit of the host's record is otherwise never done; it is written here
|
|
because the alternative was four machines that could apply nothing.
|
|
|
|
## Open questions
|
|
|
|
- Should `Record` keep a former target for a kind the host cannot remove at all? The trace is
|
|
useful; the removal it implies is not. Keeping it and letting the apply forget it is what the fix
|
|
does; not recording it would be quieter.
|
|
- Should the host's own versions directory be cleaned by the launcher rather than by the apply —
|
|
the one archive whose former targets are genuinely removable, by the thing that knows which one
|
|
runs?
|
|
- Is there a bed that replaces a host under the current rules before the live mesh does
|
|
(the proof row of [ADR 0141](../../02-DECISIONS/0141-the-host-delivers-its-own-successor.md) was
|
|
a single crossover, before former targets existed)?
|
|
|
|
## Resolved, 2026-10-02
|
|
|
|
mesh-host 65, merged 07:35Z. Recovered as the record above says: the operator dropped the one
|
|
`@former:` entry from each machine's host record and pushed; the fixed host then ran on all four and
|
|
its first apply said `forgotten mesh-host.next@former:… a former target left in place` and applied the
|
|
rest. The open questions stand as questions for the host's own versions, not as faults.
|