Phase 5 (3/4): every test on a bus of its own, at the release the mesh runs (hq ADR 0237) #97

Merged
mesh-admin merged 1 commits from feat/a-suite-that-cannot-flake into main 2026-10-06 19:19:42 +00:00
Contributor

hq ADR 0237 decision 7 (versions tested as run; a suite a gate can trust).

  • internal/testbus: a nats-server per test (linked in, v2.11.17 = the catalogue's bus image), memoised per test, shut down after. Every live test that read MESH_TEST_NATS now gets its own bus; MESH_TEST_NATS_EXTERNAL=1 points a run at a bus of a person's choosing.
  • TestTheBusTheTestsRunIsTheBusTheMeshRuns: holds the pin to the catalogue's nats Dockerfile release and, with MESH_FACTS, to the release the mesh runs. TestTheStoreTheTestsRunIsTheStoreTheMeshRuns: the store's major vs the facts.
  • TestAWaiterPinsNoConnectionAndGivesUp read a timing (most connections held at one look) and failed on a loaded machine; it now reads the state it means (fewest held across the wait).
  • make check: packages in parallel, -race, -timeout 15m.
  • go.mod + vendor: github.com/nats-io/nats-server/v2 v2.11.17 (and its deps).

On the reported flakes: on main, TestNatsTwoMachinesShareTheWorkAndNeitherIsHandedMoreThanItCanTake did not hang under -race against a fresh bus, alone or with its package, nor did the whole suite in parallel against a fresh bus; the named failures (...PausedAsItsPullWasAnswered..., ...StoreCannotTake..., TheBusIsWhatTheControllerDefines, the waiter) come from packages sharing one bus's fixed-name objects and from the waiter's timing read. With this branch: the whole suite run 3x in parallel under -race, zero failures (siblings at main).

Independent of #95/#96 (no shared files but go.mod/vendor, which those do not touch). After it merges, #96's merge-check.sh can drop -p 1.

hq ADR 0237 decision 7 (versions tested as run; a suite a gate can trust). - `internal/testbus`: a nats-server per test (linked in, v2.11.17 = the catalogue's bus image), memoised per test, shut down after. Every live test that read `MESH_TEST_NATS` now gets its own bus; `MESH_TEST_NATS_EXTERNAL=1` points a run at a bus of a person's choosing. - `TestTheBusTheTestsRunIsTheBusTheMeshRuns`: holds the pin to the catalogue's nats Dockerfile release and, with `MESH_FACTS`, to the release the mesh runs. `TestTheStoreTheTestsRunIsTheStoreTheMeshRuns`: the store's major vs the facts. - `TestAWaiterPinsNoConnectionAndGivesUp` read a timing (most connections held at one look) and failed on a loaded machine; it now reads the state it means (fewest held across the wait). - `make check`: packages in parallel, `-race`, `-timeout 15m`. - go.mod + vendor: github.com/nats-io/nats-server/v2 v2.11.17 (and its deps). On the reported flakes: on main, `TestNatsTwoMachinesShareTheWorkAndNeitherIsHandedMoreThanItCanTake` did not hang under -race against a fresh bus, alone or with its package, nor did the whole suite in parallel against a fresh bus; the named failures (`...PausedAsItsPullWasAnswered...`, `...StoreCannotTake...`, `TheBusIsWhatTheControllerDefines`, the waiter) come from packages sharing one bus's fixed-name objects and from the waiter's timing read. With this branch: the whole suite run 3x in parallel under -race, zero failures (siblings at main). Independent of #95/#96 (no shared files but go.mod/vendor, which those do not touch). After it merges, #96's merge-check.sh can drop `-p 1`.
mesh-admin added 1 commit 2026-10-06 19:18:02 +00:00
The live tests reached one shared bus and assert, read and remove the mesh's own objects by
their fixed names, so packages run in parallel deleted what each other read and the suite
passed only one package at a time; a red suite read as noise. internal/testbus starts a server
per test, linked in at the nats-server release go.mod pins, and a test holds that pin to the
catalogue's bus image and to the facts snapshot's bus when there is one, so the tests never run
a bus the mesh does not. The waiter test read a timing (the most connections held at one look)
and now reads the state it means (the fewest held across the wait). make check runs the packages
in parallel under the race detector, with a timeout.
mesh-admin merged commit 1c26235bc1 into main 2026-10-06 19:19:42 +00:00
mesh-admin deleted branch feat/a-suite-that-cannot-flake 2026-10-06 19:19:42 +00:00
Author
Contributor

Delivery novox/mesh-controller@1c26235bc18a — delivered since 2026-10-06T23:05:28Z

Machines

  • build-agent on ace: passed — healthy 3 times over 2m34s
  • build-agent on the rest running it: sent
  • mesh-controller on novox: passed — healthy 3 times over 4m1s
  • mesh-controller on the rest running it: sent
  • route-proxy on ace: passed — healthy 3 times over 2m32s
  • route-proxy on the rest running it: sent

Transitions

  • 2026-10-06 23:05 (new) → proposed (announced): novox/mesh-controller#97 merged; its head was not heard by this owner, so it is made from the forge's word
  • 2026-10-06 23:05 proposed → held (merged-unchecked): merged without a passing check — a verdict known with it is taken first: only a person decides that it goes on
  • 2026-10-06 23:05 held → delivering (go): its walk started on a word that was not this owner's: the controller's own path, which waits for nobody, or a person's plans go
  • 2026-10-06 23:05 delivering → delivered (done): its walk is done: every machine of its deploy plan passed, or was left as its policy says

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

<!-- mesh-delivery:view --> **Delivery** `novox/mesh-controller@1c26235bc18a` — **delivered** since 2026-10-06T23:05:28Z **Machines** - build-agent on ace: passed — healthy 3 times over 2m34s - build-agent on the rest running it: sent - mesh-controller on novox: passed — healthy 3 times over 4m1s - mesh-controller on the rest running it: sent - route-proxy on ace: passed — healthy 3 times over 2m32s - route-proxy on the rest running it: sent **Transitions** - 2026-10-06 23:05 (new) → proposed (announced): novox/mesh-controller#97 merged; its head was not heard by this owner, so it is made from the forge's word - 2026-10-06 23:05 proposed → held (merged-unchecked): merged without a passing check — a verdict known with it is taken first: only a person decides that it goes on - 2026-10-06 23:05 held → delivering (go): its walk started on a word that was not this owner's: the controller's own path, which waits for nobody, or a person's `plans go` - 2026-10-06 23:05 delivering → delivered (done): its walk is done: every machine of its deploy plan passed, or was left as its policy says The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
This repo is archived. You cannot comment on pull requests.
No Reviewers
1 Participants
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: novox/mesh-controller#97