Tests no longer race the wall clock; the checker invokes compose-candidate again #143

2026-11-02T22:09:05Z

mesh-tools commented

2026-11-02 22:09:05 +00:00

Issue 1126 (the controller's removal), two follow-ups of release sdk1, one check:

Every mesh-delivery test that raced the wall clock is made robust, after #142's check failed at 22:35 on two buildrun tests ("did not end within 180ms / 60ms") on a loaded shanks: the second failure tonight of that kind (the first: mesh-sdk's retry tests, fixed in f02f142). Quick fix (the two tests) against thorough fix (every timing-bound test of the repository): thorough chosen. buildrun's asker tests use the steps' own times (bound 1 h, nothing sleeps); the builder's decline bound reads Builder.Now (nil in production: the wall clock as before); the checker logs a merge-turn change, which its integration test waits on instead of sleeping; the teller's test bus gains its own ends bucket (its suite 34 s -> 12.6 s) and restores its package-level now; poll deadlines 10 s unit / 20 s integration. Proven in a half-core cgroup beside a heavy suite: buildrun failed 5 of 20 runs before, 30 of 30 passed after. No test added.
mesh-delivery-checker invokes seat:node-engine.compose-candidate again (reverts #142): the new indexer runs on novox and all four node-engines serve the verb, so a check composes on the machines instead of passing with a warning. The reviewer ran the running indexer's whole-shelf judgement over 157 manifests with this head: none refused, so tonight's loop cannot recur.

Reviewed-by: review-1126@shanks at 34a607a1f5

https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL

Issue 1126 (the controller's removal), two follow-ups of release sdk1, one check: 1. Every mesh-delivery test that raced the wall clock is made robust, after #142's check failed at 22:35 on two buildrun tests ("did not end within 180ms / 60ms") on a loaded shanks: the second failure tonight of that kind (the first: mesh-sdk's retry tests, fixed in f02f142). Quick fix (the two tests) against thorough fix (every timing-bound test of the repository): thorough chosen. buildrun's asker tests use the steps' own times (bound 1 h, nothing sleeps); the builder's decline bound reads Builder.Now (nil in production: the wall clock as before); the checker logs a merge-turn change, which its integration test waits on instead of sleeping; the teller's test bus gains its own ends bucket (its suite 34 s -> 12.6 s) and restores its package-level now; poll deadlines 10 s unit / 20 s integration. Proven in a half-core cgroup beside a heavy suite: buildrun failed 5 of 20 runs before, 30 of 30 passed after. No test added. 2. mesh-delivery-checker invokes seat:node-engine.compose-candidate again (reverts #142): the new indexer runs on novox and all four node-engines serve the verb, so a check composes on the machines instead of passing with a warning. The reviewer ran the running indexer's whole-shelf judgement over 157 manifests with this head: none refused, so tonight's loop cannot recur. Reviewed-by: review-1126@shanks at 34a607a1f5cf44888f588d0dfca423b1b69b5e33 https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL

mesh-tools added 2 commits 2026-11-02 22:09:05 +00:00

Tests wait on events or the clock they are given, never on milliseconds a loaded machine outlasts f73b0bf6f6

The check of 2026-11-02 22:35 on a loaded shanks failed two buildrun tests on wall-clock bounds of 60 ms, the
second failure of this kind tonight (the first: mesh-sdk's busretry tests, f02f142).

Quick fix: give the two buildrun asker tests bounds of seconds.
Thorough fix (chosen, because the kind recurs across every module of the repository): every test that raced a
wall clock now waits on an event, steps a clock it is given, or has a bound only a failing test reaches.
- buildrun asker tests: the steps' own times stand for the clock (bound an hour, starts and phases written in the
past), no goroutine paces phases with sleeps.
- builder: the decline bound reads Builder.Now; the decline test steps it; the stream-wait test waits for five
check reads instead of 100 ms.
- checker: the lost-ask try bounds an untaken ask at 200 ms only where that is what ends it; the program says when a
merge turn changes, and the integration test waits on that line instead of 300 ms.
- teller: the bed makes the teller's own ends bucket (a 10 s stall at start the tests raced by 0.5 s), waits for its
warrant subscription instead of 500 ms pauses, and the package-level now is restored with t.Cleanup.
- watchman: the recount, run in its own goroutine, is waited for instead of 10 ms.
- polls of 2-5 s become 10 s (unit) and 20 s (integration).

Under a throttled cgroup (half a core shared with two busy loops) beside the node-engine's suite, the buildrun
tests failed 5 of 20 runs before and passed 30 of 30 after; no test added.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL

Give the checker back its compose-candidate invoke, now that the new indexer runs

mesh/delivery merged: source.main moved to novox/mesh-delivery@a62276bcafad

Details

mesh/check being carried to 34a607a1f5cf on a62276bcafad | from 15d28ddf5bd3

34a607a1f5

The invoke was left out (mesh-delivery #142) only while the indexer built before the verb ran; the new indexer and the
node-engine's compose-candidate run on every machine since 22:57, so a check composes on the machines again instead of
passing with a warning (ADR 0593 §7, §9).

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL

mesh-tools commented

2026-11-02 22:09:10 +00:00

Delivery novox/mesh-delivery@34a607a1f5cf — merged since 2026-11-02T22:17:57Z

Source — source.main moved to novox/mesh-delivery@a62276bcaf. Whether it reached the machines is mesh-delivery-view's, derived from source.main, the wants and the haves.

Transitions

2026-11-02 22:09 (new) → proposed (announced): novox/mesh-delivery#143's head announced
2026-11-02 22:09 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false)
2026-11-02 22:17 ready → merged (merged): on the trunk its modules follow: merged there

The commit's note under refs/notes/mesh-plan keeps every transition of this change up to its merge: git log --notes=mesh-plan.

**Delivery** `novox/mesh-delivery@34a607a1f5cf` — **merged** since 2026-11-02T22:17:57Z **Source** — source.main moved to novox/mesh-delivery@a62276bcafad. Whether it reached the machines is `mesh-delivery-view`'s, derived from `source.main`, the wants and the haves. **Transitions** - 2026-11-02 22:09 (new) → proposed (announced): novox/mesh-delivery#143's head announced - 2026-11-02 22:09 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false) - 2026-11-02 22:17 ready → merged (merged): on the trunk its modules follow: merged there The commit's note under `refs/notes/mesh-plan` keeps every transition of this change up to its merge: `git log --notes=mesh-plan`.

mesh-tools merged commit a62276bcaf into main

2026-11-02 22:17:54 +00:00

mesh-tools deleted branch fix/timing-tests

2026-11-02 22:17:54 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-delivery#143