Merge pull request 'Issues 233, 234: a stale declaration removed four modules, and the host could not recover' (#358) from issues/233-234-a-stale-declaration-and-a-host-that-cannot-recover into main

2026-10-04 13:09:47 +00:00
parent 438162b5a5 9cc00d57ea
commit 2f41b4124d
2 changed files with 127 additions and 0 deletions
@@ -0,0 +1,60 @@
+---
+status: open
+opened: 2026-10-04
+located-in: []
+fixed-by:
+amended-design:
+---
+
+# 233 — A host without its package manager's configuration refuses the declaration that would restore it
+
+## What was observed
+
+2026-10-04, on a workstation. The package manager's module writes that tool's main configuration
+file whole, keeping the file it found. A declaration that no longer named the module
+([issue 234](../234-a-later-declaration-left-out-four-assigned-modules-and-a-machine-applied-it/00-report.md))
+reached a host older than the fix that gives a written-over file back its original. The host
+removed the file, and the kept original stayed where the host keeps such files.
+
+From the next declaration on, the host refused every declaration whole:
+
+```
+refused a declaration: this is the arch host and pacman does not answer here. Either this machine
+is not Arch, or its package database is broken: pacman exited 1: error: config file … could not be
+read: No such file or directory
+```
+
+The machine stayed in that state for an hour and a half. It retried every five minutes and was
+refused every time. Two declarations it refused would have repaired it:
+
+- the current one, which names the package manager's module again and so writes the file;
+- the one carrying the newer host, which gives a kept original back.
+
+Neither could be applied. A person restored the kept original by hand, and the next push applied.
+
+`status` showed the machine as `refused` with the error above. That is correct, but nothing said
+that the mesh's own tools could no longer reach it.
+
+## Why it matters beyond this instance
+
+The host checks that the package manager answers before it applies anything. That check is right for
+a machine that is not what the mesh thinks it is. Here the check depends on a file that the mesh's own
+modules own and can remove. Once that file is gone:
+
+- every module's change waits behind it, including the host's own upgrade;
+- the only way back is a person on the machine;
+- so a fault that one declaration caused, and a later declaration would fix, cannot be undone
+  through the mesh.
+
+The same shape holds for anything the host probes before applying. If what it probes is one of
+the mesh's own resources, one bad declaration can wedge the machine.
+
+## Open questions
+
+1. Should a probe that fails refuse the declaration whole? Or should it fail only the resources that
+   need the tool, and apply the rest (files, units, the host's own upgrade)? The rest may include
+   the very resource that restores the tool.
+2. Should the host refuse to remove a file a seat's holder needs to answer, or warn before it does?
+3. Is a machine that refuses every declaration for longer than one apply a fault the mesh raises by
+   itself ([issue 187](../187-the-mesh-tells-nobody-when-it-stops-working/00-report.md))? Today it
+   appears only to someone who asks for `status`.
@@ -0,0 +1,67 @@
+---
+status: open
+opened: 2026-10-04
+located-in: []
+fixed-by:
+amended-design:
+---
+
+# 234 — A later declaration left out four assigned modules, and a machine applied it
+
+## What was observed
+
+2026-10-04, on a workstation. Four modules (the package manager's, sudo, localization and the
+container runtime's) had been assigned and pushed, and the host was applying them in one long apply,
+about eleven minutes on a loaded machine. Seven more declarations arrived while it worked. When the
+apply ended, the host logged six times `set aside a declaration: a newer one arrived with it` and
+applied the one it kept.
+
+The host chooses by the sequence a declaration carries
+([issue 107](../107-a-declaration-carries-no-order/00-report.md)), so the one it kept was the latest
+the controller had sent. That declaration did not name the four modules. Over the next eight minutes
+the host undeclared all of them:
+
+```
+removed sudo.operator (…)
+removed pacman.config (…)
+forgotten pacman.package (pacman)
+removed localization.locale (…)
+removed docker.prune-service (…)
+```
+
+Throughout, the controller's assignments named the four modules on that machine, and they still do:
+`plan` for the machine lists them. Nobody had unassigned them.
+
+What happened around it:
+
+- A catalogue asked the controller to catch up twice in the same minutes, and every builder rebuilt
+  the same catalogue commit.
+- The controller daemon restarted three minutes after the stale declaration was applied.
+- Another session was working on the mesh and pushing at the same time.
+
+The controller logs no send. Which process sent the stale declaration, and from what view, cannot be
+read back from anything the mesh keeps.
+
+## Why it matters beyond this instance
+
+A declaration is the mesh's word on what a machine should be, and the host applies it in full,
+including removing what it does not name. A stale one is not harmless: here it removed four modules,
+and on a host older than the fix for written-over files it deleted four system files outright
+([issue 233](../233-a-host-without-its-package-managers-configuration-refuses-the-declaration-that-would-restore-it/00-report.md)).
+
+[Issue 204](../204-a-controller-handover-re-sent-every-node-a-stale-declaration/00-report.md) was this
+family once before and is marked resolved. Ordering by sequence (issue 107) protects against an old
+declaration arriving late. It does not protect against a declaration that is new in sequence but
+composed from an old view.
+
+## Open questions
+
+1. Where can a declaration be composed with fewer modules than the assignments hold? Candidates:
+   - a send from a process with an out-of-date view;
+   - composition while a module's build is being replaced;
+   - a plan sending what it composed when it was made.
+2. Should every send be recorded with its sequence and its sender, so `status` can show what a machine
+   was last told and by whom?
+3. Should a declaration that removes something carry a check the host can verify? For example, the
+   assignment generation it was composed from, so a host refuses one older than the generation it has
+   already applied.