Review round 1 of issue 1945's fix found two ways the guard could let a loss
through or hide one from the operator.
A publish deny was never compared: grants.User carried no deny, so a widened
allow that brings a new deny (a lost publish right) passed as an added right.
The user now carries its deny; a deny the shadow adds is a loss, and while
nats_users answers no deny for the loaded list, a user whose grants differ
and whose shadow grant has a deny refuses, named as a deny the loaded list
cannot be compared on. None of today's 41 differing users carries a deny
(only the four node-tools users do, and they are the same in both lists).
The ask said "takes none away" whenever rights were added, whatever else
differed. Its words now name the added rights only when nothing else refuses
(Refusals: unread list, Removing, a missing hash), and otherwise say what the
yes would be refused for, naming the users; Decide uses the same Refusals.
issue 1945
The operator's yes to take-over was refused twice (2026-10-26 13:48 and
2026-10-27 21:19 UTC) because 41 users differ, every difference a right the
grants part's list adds. The controller's list is not changed (ADR 0547 §1),
so ADR 0547 §8's equality could never hold again. ADR 0562 changes it:
take-over is allowed when no user is added or dropped, no loaded user loses a
publish, subscribe or responses right and every hash is held; the ask names
the added rights by user and kind, and a yes after they changed is refused.
Tests: rows from the captured live comparison of 21:19 (testdata, no
hashes) in the compare table, and rows in the take-over table for a right
removed, added rights changed since the ask, and only added rights recorded.
issue 1945
A node composing a consumer reads the served keys of the trunk's manifest of a
provider elsewhere, which may serve a setting the build it runs does not.
mesh-grants declares each module's served settings (catalogue.ServedSettings), so
the user list grants every other node's agent those keys of mesh-config_resolved
alone: shanks then settles what novox's mailu serves. The assigner's check of
every node reads no build, archive or served value: it composes as before.
mesh/check being carried to ec0b7e2dcc06 on da514816204b | from 24cb379434f5
A dropped verified-sender comes from a root-free word that failed for one
pass, not from a release, and losing it cut the operator's approvals on
2026-10-27. Holdings.CapabilitiesLost holds such a membership with its own
reason while the holder is assigned and its manifest still asks for it; a
dropped tool or field and a moved seat stay published.
Holding the user list on any loss would freeze it after take-over at the
first release that drops a tool or moves a seat (ADR 0547 §7), with nothing
to accept a deliberate loss; memberships had the same gap. The lead's decision
in review round 1: one pure guard, Holdings.LastHolders, holds the list and
each membership only when a seat held in force would have no holder in what
is composed now while an assigned module still claims it, the 09:21 kind.
A dropped tool, verb or field, or a seat handed to another node, is written.
The user list of 2026-10-27 09:21 granted no holder for nine seats: the seeded
placements hold only the seats the controller had on record, and the grants part
read those holds as every holder (records.go heldWherePlaced). A placement's
holds are now the holders on record, as catalogue.World's Holdings are: a mesh
seat no placement holds is held by derivation, wherever its claimant is placed.
The membership loss guard (Losses) did not cover the user list: after take-over
the list that lost those seats went out. UserLosses applies the same guard per
user against the loaded list, and holdsBack holds a list that would lose a grant,
or whose loaded grants cannot be read, raising the held warning.
Quick fix chosen over moving the on-record rule into mesh-sdk: mesh-grants is the
only part reading placement holds itself; the others pass them to the catalogue.
mesh/check being carried to bb63d83f849e on c38e81c7f5e9 | from 2915d74822cd
After the placement seed's revert the rules that hold a want (a hold, rules 3 and 5) kept g14's and shanks's Self wants: the held branch copied them whole, and the rule branch wrote a want naming no composition, which EncodeWant refuses. Module.Self, set once from the placement, is now the one source, and every want the rules write or keep is derived from it: an unplaced node's kept want names the controller's composition of the same fingerprint, asked as any record move; with none, it stays and rule 7 says why.
mesh/check being carried to efadc691e7ea on 475bb4bb0b8f | from 4af36c69a761
view_reach asked for mesh-delivery-view.reach, a name the mesh never issued, so it failed on the machines;
the same line was not gofmt'd, which failed the try of main. Quick fix: the name. Thorough for this module: the
manifest test now refuses any stdio.State literal in cmd/ that is neither an own state nor listed in reads.
mesh/check checking f007ca42279a on 4b36cc29a6ae | novox/mesh-delivery#81 in group fix/1496-start-health
Review of 50c4674: the bar on new wants naming F while a node fails it alone
also caught a node that runs F with no live want, the issue-1945 recovery path:
it went on neither list and its node-engine gave the module up. The bar now
holds only a node that neither wants nor runs F.
ADR 0557 version 9: a failure that is one node's holds nothing and sets nothing
back, but the version reaches no node that does not want it yet until it is
over. A node whose want does not name F gets no new want naming it (rule 2, in
words naming the failing nodes); wants already written stay.
Review round 2 of 16e4e61: gating the first-pass clear on a failed have left a
condition an earlier process raised standing for good once its node recovered
while no assigner watched. The first pass now clears it for every module and
every node the presence bucket knows, a quiet clear at the router for one that
is not open.
Review of 2ee7589: a node that sleeps after running F healthy kept its stale
have as a witness, so F stayed unheld while it failed on the online nodes; and a
failed-alone condition was never cleared once its node left the module.
A witness must now be online by the watchman's presence. F failing on a second
node is no longer one node's failure: the hold and the set-back stand (ADR 0557
version 6, §2). A remembered failed-alone key whose node is no longer one of the
module's nodes is cleared once; the first pass after a start clears it for every
node the presence bucket knows, for a module with a failed have.
On ace, 2026-10-26 21:19, a newly assigned mesh-build-node failed its first
health check at 8a98386f (later 31be2f24) while shanks, g14 and novox had run it
healthy for an hour. Being first by name, ace was the canary, and the assigner
held the version and moved the three healthy nodes back.
A failed health check of the newest fingerprint now holds it only when no other
node's have names it healthy (ADR 0557 §2 keeps ADR 0429 §6's hold and set-back
for that case). Otherwise the failure is that node's (ADR 0557 §1): nothing is
held or set back, the node keeps its want, and the assigner raises the node's
condition once (failed-alone.<node>.<module>), naming the node, the module, the
fingerprint and the have's words, and clears it once when it ends; the first
pass after a start clears what a previous process may have raised.
jschoubben
mesh-tools