mesh-catalog #562's step passed on shanks at 10:52:16Z and its end was heard at 10:52:17Z (its own watch was deaf;
HeardEnds read it at the nudge of that same step end). The checker was judging it (kept verdicts, composition) when
checker_status, reading only the steps state, named it lost: no step runs and the newest one ended. checker_ask_again,
called on that at 10:52:27Z, ended the judging (context canceled) and the check ran whole again, passing at 11:00:16Z.
No step end was lost; the read tools could not tell judging from lost.
The running check now writes, in its state asks (already its own, by verdict key), the step whose end it heard and
judges, and since when; its next ask or its end clears it. checker_status names such a candidate under judging, not
lost, and checker_ask_again refuses it. A judging older than the step bound counts as lost (the checker that heard it
stopped). State on the bus, written on the step-end event; no timer.
Quick fix: a grace time after a step's end in lostWhy. Thorough fix (chosen): the checker says on the bus what it
judges, so the read side never guesses; a grace would misname any judging longer than it (a composition wait).
Test: a row in TestACheckWhoseAskJudgedNothingIsAskedAgainOrRead holds the composition and reads status and
ask-again while the step's end is judged; without the fix it names the check lost and the ask again fails it.
Recovery of photos and link2pay on novox, 2026-10-29: their composition and grant were lost and no tool wrote a done key again.
Quick fix: rewrite the done key unchanged as a new revision. Thorough fix: ADR 0547, the node agent composes its own and the
assigner stops waiting on controller compositions (design 47).
mesh/check pass d54eab4052d1 on c1b4533201c8 | unit 3318 run, 1951 kept, 38 skipped | integration 3994 run, 2125 kept, 38 skipped | over bound: unit, …
On novox on 2026-10-29 the assigner wrote no want for de-spiegel and link2pay (rule 5: no provider of route runs
healthy) nor photos (rule 5: no provider of s3-bucket), and none where no node of a module was online (rule 3). A
module with no want is on no list and nothing applies it; a site runs while the proxy in front of it is down.
- Quick fix: skip rules 3 and 5 for the three modules' first wants. Leaves rule 5 holding every move and the
canary holding new assignments.
- Thorough fix, chosen: rule 5 holds no want; it is said per node (Outcome.Unreached, journaled once per words by the
pass). The canary (rule 3) no longer holds the first want of a node that does not run the module. A node that runs
it keeps the canary order: at an older composed and granted fingerprint of the new form its first want stays at what
it runs; at an old-form build it waits for the canary as before (issue 1945); at an older fingerprint with no
composition or grant it gets F. This departs from ADR 0429 §6, which makes every provider healthy a rule of every want.
Tests: rows changed in the existing rule tests, and rows for today's modules (route-proxy down: de-spiegel and
link2pay wanted; minio not healthy: photos moves; novox offline: a first want written, a move kept back).
mesh/check failed: ace: the check of novox/mesh-host#168 failed: the pin of git.novox.be/novox/mesh-sdk/go at commit d4c61ee8ffd0 is on no pull reques…
mesh/check failed: shanks: novox/mesh-delivery#105 does not merge into the trunk: files in conflict: modules/mesh-delivery-assigner/go.mod, modules/me…
mesh/delivery-group group feat/774-vault-mints-l2 checking: the checker's verdict is out of date: novox/mesh-sdk@8eafc0c5ce3c was judged on trunk 53f26493697e,…
Why: after the lease write the assignments are frozen, so the part's union kept a module the catalogue takes off a node
with its account, grant, membership and user-list entry. The lease is read as the assigner reads it
(buscompose.LeaseHandedAway); before it, or with no placement on the bus, the union stands. The read stays declared until
the first change after the lease write, named in modules/assignments-readers with the assigner; a TestGrants row refuses
any other module.json that reads mesh-controller.assignments, as mesh-catalog's checks/placement does for the catalogue.
Why: the controller's grant writer skipped link2pay, snake, f1-circuits and txt-game in silence (it read the fingerprint by
their unchanged commit, a build of the old form) and the take-in said 'its grant written'. The part already reads the
fingerprint from the indexer's value, so those four are granted at their src2 done key's fingerprint; a row proves it.
What a pass leaves without a grant (no fingerprint of the new form with the indexer's none2, a refused manifest, no
hashed account) is now said rather than passed over. Quick: log it. Thorough (chosen): the pass's Missing, shown by the
grants tool, and a warning condition raised and cleared on change, as consumer-differs is.
Why: from the lease write the grants part's state grants is the source and the controller's bucket otherwise; the assigner follows the lease key (a pass only when the source changes), and neither bucket stops its start. Needs the group's mesh-sdk pin (GrantSource, LeaseHandedAway, the assigner a named reader of the lease).
Why: the assigner switches its grants source when the controller's lease is handed away; until then the written keys are compared with the controller's, so the switch rests on a measured equality. Needs the group's mesh-sdk pin (PartGrantKey) before it builds.
mesh/check pass 2d8ad2e66a8f on 38f89c6976c9 | carried from 1f6f4eda3208 | unit 470 run, 4760 kept, 13 skipped | integration 503 run, 4844 kept, 22 sk…
Once the gate's one-hour ask bound goes (mesh-catalog fix/1733-no-hour), the
gate asks the operator's Approve only on a verdict, so every path that left
a candidate without one now writes one, from an event and with no timer:
- a group with a member whose repository's trunk head is not known (a pull
request the checker cannot claim) is not judged in part: its verdict fails
naming that repository, and every status of the group says so (a pass of
the others would have let it merge with that member never checked);
- the verdict of heads that cannot be judged replaces whatever the key held,
such as a running verdict left by a check that can no longer run;
- a verdict key deleted is judged again on its delete (the checker watches its
verdicts), and a check running when its key is deleted writes it anew.
Tests: three rows at the end of the no-trunk test; counts unchanged.
mesh/check checking e7888024a826 on 769815ac0fea | novox/mesh-delivery#103 in group fix/lost-check
Re-review of fix/lost-check: writing an empty list at the end kept one key
for every verdict key ever checked. Tests: every row of the lost-ask table
proves the key gone at the end; one row more (deaf after its build module
replaced it, asked again by hand); counts unchanged.
Since 9db9499 the checker asks no check while no node record is on the bus; the bed wrote none, so eight integration tests failed on main (PR #101's check). The fake build module now records the names an ask carries, and the first test asserts n1 is named.
Review of fix/lost-check: an ask no node took yet has no step key, so
asking again then would run the check twice (issue 1791's kind). The
checker keeps the asks each running check put in a new state asks, and the
tool refuses unless every one of them has a step key. Asking again under the
same ask id is not done: the stream drops a second put of an id within its
de-duplication window and keeps both past it, so the waiting copy is never
replaced.
Tests: the hand row of the lost-ask table refuses while its ask waits;
counts unchanged.
The checker's HeardEnds moved to mesh-sdk buildrun (fix/lost-check,
6aeb401): both modules pin it, and the builder, which waits on the same
Asker, now reads an ask's step at each step end of its build module too.
checker_ask_again and checker_status's lost stay in the checker.
Tests: one row in the builder's ask table (its watch deaf, read at a
nudge, done with no new ask); counts unchanged.
mesh-catalog #553's verdict stayed running after its step mvsgthyjed5786
passed on shanks at 19:09:31Z: the watch of that one step key, opened at
18:46:05Z, heard neither its start nor its end and never closed, so the
wait had no bound running and nothing would ever judge it.
- HeardEnds stands between the asker and each ask's watch: at every step
end of the same build module (one watch per steps state), at a
reconnect, the asks waited on read their keys now and are judged; a
watch that closes is opened again, at most 3 times.
- A step the build module's next process ended (RestartReason, a delivery
replacing it) counts as a lost ask: asked again under the claim, at most
LostAskTries, then failed as not heard, judged again when the nodes change.
- checker_ask_again {group, why} ends a running verdict as asked again by
hand; the checker hears its verdicts, ends the wait and claims it anew.
Refused while a step of its pull requests runs, or for a verdict that
is not running.
- checker_status names such checks under lost, and each step's what.
Tests: one table replaces the given-up test (rows for given up, replaced,
deaf and nudged, deaf and asked by hand); counts unchanged (147 unit, 10
integration in the checker).
jschoubben
mesh-tools