A version that fails on one machine while another runs it healthy is that machine's failure, and goes no further meanwhile (issue 1496, ADR 0557 §1-§3) #81

Merged
mesh-tools merged 5 commits from fix/1496-start-health into main 2026-10-26 21:37:59 +00:00
5 Commits
Author SHA1 Message Date
jschoubben f007ca4227 A node that runs a version alone-failed elsewhere still gets its want at it (ADR 0557 §2, issue 1496)
mesh/delivery merged: source.main moved to novox/mesh-delivery@bd107d20154c
mesh/delivery-group group fix/1496-start-health merged: every member is merged
mesh/check checking f007ca42279a on 4b36cc29a6ae | novox/mesh-delivery#81 in group fix/1496-start-health
Review of 50c4674: the bar on new wants naming F while a node fails it alone
also caught a node that runs F with no live want, the issue-1945 recovery path:
it went on neither list and its node-engine gave the module up. The bar now
holds only a node that neither wants nor runs F.
2026-10-26 22:34:25 +01:00
jschoubben 50c4674dfa While a node fails a version alone, no further node gets a want for it (ADR 0557 §2, issue 1496)
ADR 0557 version 9: a failure that is one node's holds nothing and sets nothing
back, but the version reaches no node that does not want it yet until it is
over. A node whose want does not name F gets no new want naming it (rule 2, in
words naming the failing nodes); wants already written stay.
2026-10-26 22:30:00 +01:00
jschoubben 92aedb9365 The first pass after a start clears failed-alone for every module and known node (ADR 0557, issue 1496)
Review round 2 of 16e4e61: gating the first-pass clear on a failed have left a
condition an earlier process raised standing for good once its node recovered
while no assigner watched. The first pass now clears it for every module and
every node the presence bucket knows, a quiet clear at the router for one that
is not open.
2026-10-26 22:21:00 +01:00
jschoubben 16e4e61a0f Only an online healthy witness, and only a first failure, keep a version from being held; failed-alone clears when a node leaves (ADR 0557, issue 1496)
Review of 2ee7589: a node that sleeps after running F healthy kept its stale
have as a witness, so F stayed unheld while it failed on the online nodes; and a
failed-alone condition was never cleared once its node left the module.

A witness must now be online by the watchman's presence. F failing on a second
node is no longer one node's failure: the hold and the set-back stand (ADR 0557
version 6, §2). A remembered failed-alone key whose node is no longer one of the
module's nodes is cleared once; the first pass after a start clears it for every
node the presence bucket knows, for a module with a failed have.
2026-10-26 22:16:38 +01:00
jschoubben 2ee7589474 A version healthy on another node is not held for one node's failed health check (ADR 0557, issue 1496)
On ace, 2026-10-26 21:19, a newly assigned mesh-build-node failed its first
health check at 8a98386f (later 31be2f24) while shanks, g14 and novox had run it
healthy for an hour. Being first by name, ace was the canary, and the assigner
held the version and moved the three healthy nodes back.

A failed health check of the newest fingerprint now holds it only when no other
node's have names it healthy (ADR 0557 §2 keeps ADR 0429 §6's hold and set-back
for that case). Otherwise the failure is that node's (ADR 0557 §1): nothing is
held or set back, the node keeps its want, and the assigner raises the node's
condition once (failed-alone.<node>.<module>), naming the node, the module, the
fingerprint and the have's words, and clears it once when it ends; the first
pass after a start clears what a previous process may have raised.
2026-10-26 22:06:32 +01:00