Merge pull request 'Issues 257, 261: a reconcile applied an older declaration over a newer one' (#109) from issues/257-261-a-reconcile-applied-an-older-declaration into main

2026-10-05 19:44:14 +00:00
parent e6fbb86373 f391c5c36c
commit 4fac135a46
3 changed files with 72 additions and 3 deletions
@@ -1,8 +1,8 @@
 ---
-status: diagnosing
+status: located
 opened: 2026-10-05
-located-in: []
-fixed-by:
+located-in: [mesh-host]
+fixed-by: novox/mesh-host#24 (89a7796)
 amended-design:
 ---

@@ -29,3 +29,32 @@ sequence numbers for two sends at 20:22:57.
 a declaration the controller cannot place says nothing about the build. The plan should say "it
 applied something other than what was recorded as sent" instead of "waiting", so a reader acts in
 minutes rather than at the bound.
+
+## 2026-10-05, later: found
+
+**The lead above was wrong.** It was not two sends. The machine's own five-minute reconcile did it.
+
+The host re-applies what it was last told every five minutes. That reconcile read the kept
+declaration, then waited for any apply in progress. A declaration the link is applying is kept only
+once its apply ends. On the anchor:
+
+- the reconcile was due at 20:22:58;
+- the controller's send arrived at 20:22:57;
+- the reconcile read the declaration kept before that send, waited, and applied it after the link's
+  apply;
+- its report named that older declaration.
+
+That explains every fact above:
+
+- the report's digest matched nothing recorded as sent;
+- the other machines, not due, matched;
+- the next push matched.
+
+**Seen a second time, with a visible effect** (issue 261): a module assigned on the laptop was
+applied and then given back 29 seconds later by the reconcile due during that apply.
+
+**Fixed** in mesh-host: the reconcile reads the kept declaration once it holds the apply lock. A test
+reproduces the interleaving: it fails with the old order and passes with the fix.
+
+The plan's "waiting" wording, raised above as a second fault, is unchanged. With the cause gone, a
+report naming an unrecorded declaration should no longer occur.
@@ -0,0 +1,40 @@
+---
+status: located
+opened: 2026-10-05
+located-in: [mesh-host]
+fixed-by: novox/mesh-host#24 (89a7796)
+amended-design:
+---
+
+# 261. A module was applied and given back half a minute later
+
+## Symptom
+
+The `hosts` module was assigned to the laptop and pushed. The laptop's host applied it: the module's
+block was written at the start of `/etc/hosts`, its tools were installed, and the host reported the
+declaration it had been sent. Twenty-nine seconds later the same host gave the block and the tools
+back. Nothing on the controller had sent a declaration without the module. Five minutes later the
+module was back.
+
+The host's journal on the laptop, in order:
+
+- `updated hosts.own (/etc/hosts): the mesh's region added at the start`
+- `applied 342 resource(s)`
+- `restored hosts.own (/etc/hosts)`
+- `removed hosts.bundle-tools`
+
+## Cause
+
+The host's five-minute reconcile was due during that apply. It read the declaration kept before the
+push, waited for the push's apply to finish, and then applied the older declaration over it. That
+older declaration did not have the module, so the module was given back. The next reconcile read the
+newer declaration, which had been kept by then, and applied the module again.
+
+This is the same fault as [issue 257](../257-a-plan-waited-on-a-declaration-its-first-machine-never-reported/00-report.md),
+where it showed only as a report naming a declaration nobody had recorded sending.
+
+## Fix
+
+The reconcile reads the kept declaration once it holds the apply lock, so it always applies the
+latest declaration the mesh sent. This is checked by mesh-host's test
+`TestAReconcileAppliesWhatWasKeptWhenItsTurnComes`, which fails with the old order.