mesh/check pass 2d8ad2e66a8f on 38f89c6976c9 | carried from 1f6f4eda3208 | unit 470 run, 4760 kept, 13 skipped | integration 503 run, 4844 kept, 22 sk…
Once the gate's one-hour ask bound goes (mesh-catalog fix/1733-no-hour), the
gate asks the operator's Approve only on a verdict, so every path that left
a candidate without one now writes one, from an event and with no timer:
- a group with a member whose repository's trunk head is not known (a pull
request the checker cannot claim) is not judged in part: its verdict fails
naming that repository, and every status of the group says so (a pass of
the others would have let it merge with that member never checked);
- the verdict of heads that cannot be judged replaces whatever the key held,
such as a running verdict left by a check that can no longer run;
- a verdict key deleted is judged again on its delete (the checker watches its
verdicts), and a check running when its key is deleted writes it anew.
Tests: three rows at the end of the no-trunk test; counts unchanged.
Review of fix/lost-check: an ask no node took yet has no step key, so
asking again then would run the check twice (issue 1791's kind). The
checker keeps the asks each running check put in a new state asks, and the
tool refuses unless every one of them has a step key. Asking again under the
same ask id is not done: the stream drops a second put of an id within its
de-duplication window and keeps both past it, so the waiting copy is never
replaced.
Tests: the hand row of the lost-ask table refuses while its ask waits;
counts unchanged.
mesh-catalog #553's verdict stayed running after its step mvsgthyjed5786
passed on shanks at 19:09:31Z: the watch of that one step key, opened at
18:46:05Z, heard neither its start nor its end and never closed, so the
wait had no bound running and nothing would ever judge it.
- HeardEnds stands between the asker and each ask's watch: at every step
end of the same build module (one watch per steps state), at a
reconnect, the asks waited on read their keys now and are judged; a
watch that closes is opened again, at most 3 times.
- A step the build module's next process ended (RestartReason, a delivery
replacing it) counts as a lost ask: asked again under the claim, at most
LostAskTries, then failed as not heard, judged again when the nodes change.
- checker_ask_again {group, why} ends a running verdict as asked again by
hand; the checker hears its verdicts, ends the wait and claims it anew.
Refused while a step of its pull requests runs, or for a verdict that
is not running.
- checker_status names such checks under lost, and each step's what.
Tests: one table replaces the given-up test (rows for given up, replaced,
deaf and nudged, deaf and asked by hand); counts unchanged (147 unit, 10
integration in the checker).
A repository never merged through the gate has no entry in source.main, so its pull requests were dropped without a log
line, a verdict or a status, and checker_status counted them as new. Now the verdict says it cannot be judged and why,
the forge status is error with those words, checker_status names it under cannot_be_judged, the log says it once per
head, and the heads are judged whole once the trunk head is known. Quick fix: this. Thorough fix (the gate seeding
source.main from the pull message's base head) is still to do: it needs mesh-sdk, the forge bridge and the gate.
mesh-grants moves to the same mesh-sdk (3203358c), so its shadow user list grants the checker and the builder the
DEAD_LETTERS reads the controller's list will, and the take-over is not blocked by a difference. The checker and the
builder hear DEAD_LETTERS through buildrun.DeadLetterHearing: in the background, again at every reconnect and every ask
waited for while deaf, and keep whether they hear it in their new state hearing, which checker_status and builder_stuck
say as dead_letters. checker_try says that a try ended by the not-taken bound leaves its ask in the stream.
Not done as asked: a module health tool. The catalogue refuses a tool check on a module with no runtime of its own and no
other check beside it (ADR 0227 rule 8, ADR 0366 §1), so the deaf state is said in the status tools and the log.
builder_stuck, checker_kept, checker_verdicts, checker_status, checker_previews, checker_pull and checker_try listed a
bucket of thousands of keys and read each value (builder_stuck also its history) one round trip at a time before
filtering, so they timed out at 30 s. Now keys are narrowed by what the key carries first, history is read only for rows
kept, values are read sixteen at a time, at most 1000 a call with the answer saying it was cut, steps newest first, a new
try reads only the newest 64 tries, and each tool logs the values read and the time taken. Same kind as the view's fix
(PR 52); the helper is copied, not put in mesh-sdk, as a few lines do not earn a group delivery.
mesh/check checking 4ffe5f4d5243 on 85aca46348a0 | novox/mesh-delivery#25 in group feat/fastcheck
A new pull request waited 25 to 45 minutes behind every open pull request's re-judge after a
merge. The semaphore becomes a ranked admission (ADR 0536 §4); checker_status names the
waiting checks by kind.
mesh/check pass e76202876032 on b714eab995a5 | unit 422 run, 0 kept | integration 467 run, 0 kept | over bound: integration (slowest integration mesh-…
The checker's own places are released when a step ends or its wait passes the step bound; checker_status read every
'running' key in the build modules' steps buckets as a step in flight, so keys left by stopped, restarted or unassigned
build modules looked like full nodes. It now lists them under left_running with why (past the bound, module not on the
node, module not running), and leaves hq-only groups out of the unjudged candidates.
The gate decides on mesh/check alone and matches an Approve's members by the heads hash over lowercase repositories
and heads. A success judged on an old trunk stood on the forge while the next check waited for a place or a delivery,
and a candidate whose repository the forge named in capitals carried a hash the gate never finds. A check that ends
after its trunk moved now owes pending, not its verdict.
jschoubben