A former target of a kind the host cannot remove is left in place and said, never fatal (hq issue 194) #65

2026-10-02T00:44:22Z

mesh-admin commented

2026-10-02 00:44:22 +00:00

Live fault, every machine. After host 64 rolled (00:34Z), each machine's first apply under the new host failed at its first step:

applying "mesh-host.next@former:/usr/lib/nox-mesh-host/versions/3c906749ad27": no way to remove a "archive"
0 resource(s) were applied and remain

ADR 0163 rule 5 (host 63: a former target is removed on the next apply) met issue 162 (an archive has no removal) in the host's own archive, whose target is a new directory every version. Orphans are removed before any resource on a converged machine, so nothing from the mesh can land anywhere until the record changes — the fix included, since it is an archive too. hq issue 194 (PR 274) has the account.

The fix. remove returns a sentinel for a kind it has no removal for; removeOrphan treats that as fatal only for an orphan the declaration dropped (162 unchanged) and, for a former target, forgets the record and says forgotten … a former target left in place. Test: a declaration with the host's archive current and its former on record applies whole; a dropped archive still refuses. Full suite green, 20 packages.

Recovery, by hand, once per machine (the broken host cannot apply this): merge and let it build; on each machine drop the one @former: entry from /var/lib/mesh-host/state.json; push <node>. The apply writes this archive, stands aside, and the fixed host forgets the former it finds. Then status is clean and mesh-controller 206 can follow.

**Live fault, every machine.** After host 64 rolled (00:34Z), each machine's first apply under the new host failed at its first step: ``` applying "mesh-host.next@former:/usr/lib/nox-mesh-host/versions/3c906749ad27": no way to remove a "archive" 0 resource(s) were applied and remain ``` ADR 0163 rule 5 (host 63: a former target is removed on the next apply) met issue 162 (an archive has no removal) in the host's own archive, whose target is a new directory every version. Orphans are removed before any resource on a converged machine, so nothing from the mesh can land anywhere until the record changes — the fix included, since it is an archive too. hq issue 194 (PR 274) has the account. **The fix.** `remove` returns a sentinel for a kind it has no removal for; `removeOrphan` treats that as fatal only for an orphan the declaration dropped (162 unchanged) and, for a former target, forgets the record and says `forgotten … a former target left in place`. Test: a declaration with the host's archive current and its former on record applies whole; a dropped archive still refuses. Full suite green, 20 packages. **Recovery, by hand, once per machine** (the broken host cannot apply this): merge and let it build; on each machine drop the one `@former:` entry from `/var/lib/mesh-host/state.json`; `push <node>`. The apply writes this archive, stands aside, and the fixed host forgets the former it finds. Then `status` is clean and mesh-controller 206 can follow.

mesh-admin added 1 commit 2026-10-02 00:44:22 +00:00

A former target of a kind the host cannot remove is left in place and said, never fatal (hq issue 194) c47aa5d9b3

The host delivers its own successor as an archive whose target is a new
directory each version, and since mesh-host 63 the record keeps a resource's
former target for the next apply to remove. An archive has no removal (issue
162), so the first host that replaced itself under that rule refused its own
former version at the first step of every apply, and all four machines applied
nothing from then on. A former target nobody dropped is forgotten and said;
an archive the declaration dropped still refuses.

mesh-admin merged commit e12ca3f4dd into main

2026-10-02 07:36:57 +00:00

mesh-admin deleted branch fix/a-former-target-without-a-removal-is-left-and-said

2026-10-02 07:36:57 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-host#65