S1 is urgent after half an hour for a control node (the node running mesh-controller). S9 also hears the bus module's
slow consumers and closed connections, each closure counted once. S21 is one finding per heard machine from its failed
haves and its provisioner clients' reports (PROV_APPLIED). DEAD_LETTERS is counted again at a letter's age and every
five minutes while letters are held, on the watchman's own pulse. needs-operator waits for L6: its inputs are not on the
bus. Pins mesh-sdk b6ef01d.
The signals rows S1, S8, S9, S10, S11, S21 and the haves' unhealthy and needs-operator are pulses whose expiry is a finding;
stuck keys and presence are its own findings; it keeps the dead letters from the lease write on its own consumer, lists and
shows them, and refuses deliver and drop until the hand-act log's home (issue 1953). Tools read every state it keeps and
start resync. Pins mesh-sdk 2a04bf0.
Recovery of photos and link2pay on novox, 2026-10-29: their composition and grant were lost and no tool wrote a done key again.
Quick fix: rewrite the done key unchanged as a new revision. Thorough fix: ADR 0547, the node agent composes its own and the
assigner stops waiting on controller compositions (design 47).
mesh/check pass d54eab4052d1 on c1b4533201c8 | unit 3318 run, 1951 kept, 38 skipped | integration 3994 run, 2125 kept, 38 skipped | over bound: unit, …
On novox on 2026-10-29 the assigner wrote no want for de-spiegel and link2pay (rule 5: no provider of route runs
healthy) nor photos (rule 5: no provider of s3-bucket), and none where no node of a module was online (rule 3). A
module with no want is on no list and nothing applies it; a site runs while the proxy in front of it is down.
- Quick fix: skip rules 3 and 5 for the three modules' first wants. Leaves rule 5 holding every move and the
canary holding new assignments.
- Thorough fix, chosen: rule 5 holds no want; it is said per node (Outcome.Unreached, journaled once per words by the
pass). The canary (rule 3) no longer holds the first want of a node that does not run the module. A node that runs
it keeps the canary order: at an older composed and granted fingerprint of the new form its first want stays at what
it runs; at an old-form build it waits for the canary as before (issue 1945); at an older fingerprint with no
composition or grant it gets F. This departs from ADR 0429 §6, which makes every provider healthy a rule of every want.
Tests: rows changed in the existing rule tests, and rows for today's modules (route-proxy down: de-spiegel and
link2pay wanted; minio not healthy: photos moves; novox offline: a first want written, a move kept back).
mesh/check failed: ace: the check of novox/mesh-host#168 failed: the pin of git.novox.be/novox/mesh-sdk/go at commit d4c61ee8ffd0 is on no pull reques…
mesh/check failed: shanks: novox/mesh-delivery#105 does not merge into the trunk: files in conflict: modules/mesh-delivery-assigner/go.mod, modules/me…
mesh/delivery-group group feat/774-vault-mints-l2 checking: the checker's verdict is out of date: novox/mesh-sdk@8eafc0c5ce3c was judged on trunk 53f26493697e,…
Why: after the lease write the assignments are frozen, so the part's union kept a module the catalogue takes off a node
with its account, grant, membership and user-list entry. The lease is read as the assigner reads it
(buscompose.LeaseHandedAway); before it, or with no placement on the bus, the union stands. The read stays declared until
the first change after the lease write, named in modules/assignments-readers with the assigner; a TestGrants row refuses
any other module.json that reads mesh-controller.assignments, as mesh-catalog's checks/placement does for the catalogue.
Why: the controller's grant writer skipped link2pay, snake, f1-circuits and txt-game in silence (it read the fingerprint by
their unchanged commit, a build of the old form) and the take-in said 'its grant written'. The part already reads the
fingerprint from the indexer's value, so those four are granted at their src2 done key's fingerprint; a row proves it.
What a pass leaves without a grant (no fingerprint of the new form with the indexer's none2, a refused manifest, no
hashed account) is now said rather than passed over. Quick: log it. Thorough (chosen): the pass's Missing, shown by the
grants tool, and a warning condition raised and cleared on change, as consumer-differs is.
Why: from the lease write the grants part's state grants is the source and the controller's bucket otherwise; the assigner follows the lease key (a pass only when the source changes), and neither bucket stops its start. Needs the group's mesh-sdk pin (GrantSource, LeaseHandedAway, the assigner a named reader of the lease).
Why: the assigner switches its grants source when the controller's lease is handed away; until then the written keys are compared with the controller's, so the switch rests on a measured equality. Needs the group's mesh-sdk pin (PartGrantKey) before it builds.
mesh/check pass 2d8ad2e66a8f on 38f89c6976c9 | carried from 1f6f4eda3208 | unit 470 run, 4760 kept, 13 skipped | integration 503 run, 4844 kept, 22 sk…
Once the gate's one-hour ask bound goes (mesh-catalog fix/1733-no-hour), the
gate asks the operator's Approve only on a verdict, so every path that left
a candidate without one now writes one, from an event and with no timer:
- a group with a member whose repository's trunk head is not known (a pull
request the checker cannot claim) is not judged in part: its verdict fails
naming that repository, and every status of the group says so (a pass of
the others would have let it merge with that member never checked);
- the verdict of heads that cannot be judged replaces whatever the key held,
such as a running verdict left by a check that can no longer run;
- a verdict key deleted is judged again on its delete (the checker watches its
verdicts), and a check running when its key is deleted writes it anew.
Tests: three rows at the end of the no-trunk test; counts unchanged.
mesh/check checking e7888024a826 on 769815ac0fea | novox/mesh-delivery#103 in group fix/lost-check
Re-review of fix/lost-check: writing an empty list at the end kept one key
for every verdict key ever checked. Tests: every row of the lost-ask table
proves the key gone at the end; one row more (deaf after its build module
replaced it, asked again by hand); counts unchanged.
Since 9db9499 the checker asks no check while no node record is on the bus; the bed wrote none, so eight integration tests failed on main (PR #101's check). The fake build module now records the names an ask carries, and the first test asserts n1 is named.
Review of fix/lost-check: an ask no node took yet has no step key, so
asking again then would run the check twice (issue 1791's kind). The
checker keeps the asks each running check put in a new state asks, and the
tool refuses unless every one of them has a step key. Asking again under the
same ask id is not done: the stream drops a second put of an id within its
de-duplication window and keeps both past it, so the waiting copy is never
replaced.
Tests: the hand row of the lost-ask table refuses while its ask waits;
counts unchanged.
The checker's HeardEnds moved to mesh-sdk buildrun (fix/lost-check,
6aeb401): both modules pin it, and the builder, which waits on the same
Asker, now reads an ask's step at each step end of its build module too.
checker_ask_again and checker_status's lost stay in the checker.
Tests: one row in the builder's ask table (its watch deaf, read at a
nudge, done with no new ask); counts unchanged.
mesh-catalog #553's verdict stayed running after its step mvsgthyjed5786
passed on shanks at 19:09:31Z: the watch of that one step key, opened at
18:46:05Z, heard neither its start nor its end and never closed, so the
wait had no bound running and nothing would ever judge it.
- HeardEnds stands between the asker and each ask's watch: at every step
end of the same build module (one watch per steps state), at a
reconnect, the asks waited on read their keys now and are judged; a
watch that closes is opened again, at most 3 times.
- A step the build module's next process ended (RestartReason, a delivery
replacing it) counts as a lost ask: asked again under the claim, at most
LostAskTries, then failed as not heard, judged again when the nodes change.
- checker_ask_again {group, why} ends a running verdict as asked again by
hand; the checker hears its verdicts, ends the wait and claims it anew.
Refused while a step of its pull requests runs, or for a verdict that
is not running.
- checker_status names such checks under lost, and each step's what.
Tests: one table replaces the given-up test (rows for given up, replaced,
deaf and nudged, deaf and asked by hand); counts unchanged (147 unit, 10
integration in the checker).
mesh/check failed: integration: meshbuildgo/cmd/mesh-build-go.TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus: the compose ask mvsg85b34d25b4 never ende…
Deletes the placement and candidate writes, the reread tool and its forge reads, the two buckets and the placement-changed event.
D1a and D1b are one commit: the index package, the named reads, the tools and the manifest do not compile apart.
mesh/check being carried to 726bbf1f1ffa on 2ac146078f47 | from e753009fbc71
The view remembered a reach value, a reached head and a motion event as written or said before the bus took them; a write refused at its start (no responders) was never made again while the derivation stayed the same, and mesh-delivery.show said mesh-delivery-view not running on shanks while node_haves said it running.
Quick fix: forget only the refused reach write. Thorough fix, chosen: every write and event the view makes after its lock (reach, reached head, delivered revision, motion) answers its error, and the model forgets what was refused, so the next change it takes in makes it again; one place, the writer of the state both mesh-delivery.show and knowledge-base.progress read.
Asking whether the merge contains the held commit refused forward merges
forever on a diverged or rewritten main or a squash-merged registered head,
failed every later merge when the forge no longer had the held commit, and
cost one forge call per held commit. The question is now the reverse: a merge
is kept back only when it is in the taken merge and differs from it. A merged
message asks it once, before it writes anything, and a late message writes no
module, deletion, placement or taken key. A take-in asks it once for its
module. A 404 is not known to be older and is written, and said; any other
failure to say leaves the message for redelivery.
The indexer wrote mesh-catalog's merge 9cdac09a back over the later 69056948,
rolling kernel and mesh-build-image back on the nodes: a merged message's
commit was written whatever the held one was. Before writing or deleting a
module's value, the indexer now asks the forge whether the merge contains the
held commit, keeps the held value when it does not and says so, and leaves the
message for redelivery when the forge cannot say. Each applied merged message
is said in one line with the modules written.
No tool showed what the indexer heard of FORGE nor a modules key's history
with times, so the cause could not be proven: the indexer now records each
FORGE message it handles (sequence, deliveries, times, commit, outcome) in its
state heard, read by indexer_heard, and indexer_module_history reads
modules.<module>'s kept values with their times.
Quick fix (Src order in writeOverlay) rejected: Src is 0 once the source moved
on, as here. Thorough fix chosen: the order is the forge's, on every path that
writes a module (merged message, take-in).
jschoubben