Prove a write by compare-and-set loses no update under contention, on a real bus (hq research 042) #25

Merged
mesh-admin merged 2 commits from feat/a-counter-by-compare-and-set into main 2026-10-10 12:37:43 +00:00
Contributor

Part of the issue tracker decision, hq research 042 (01-RESEARCH/042-the-mesh-keeps-its-own-issues); its ADR has no number yet. Companion: novox/mesh-sdk#16 (State(...).Increment).

Finding: the runtime already carries the expected revision to the bus: mesh/state.create is KeyValue.Create, mesh/state.update is KeyValue.Update(key, value, revision) with revision 0 refused, and a lost write (ErrKeyExists / wrong last sequence) is bus.ErrStateChanged answered to the bundle with code -32010 (#23, hq ADR 0259). Old SDKs only send get/put/delete/keys/watch, which are unchanged. No runtime code changes here: tests only.

Tests

  • internal/runtime TestTwoWritersIncrementingOneCounterOnTheBusLoseNoUpdate: on an embedded nats-server with JetStream (internal/meshtest, a bus per test, so it never skips), two writers x 1000 increments through the module's bus as a bundle asks (mesh/state.get then create/update at the revision read, retry when the code is -32010, bounded) end at 2000, every number once, rising per writer. A local run logged 1646 lost writes retried, so the contention is real.
  • TestAStaleWriteIsRefusedAndWritesNothing: update at revision 0 refused (not as a lost write) and writes nothing; second create, stale revision and future revision each refused with -32010 and ErrStateChanged, value unchanged; a deleted key can be created again.
  • internal/launch TestALostWriteIsAnsweredWithItsOwnCodeAndNothingElseIs: the code reaches the bundle wrapped or not; a plain refusal is -32000.
  • merge-check.sh: both runtime tests added to the list that runs where no bundle can be launched, so mesh/repo-check runs them.

Mutations, each killed: Update as Put, Create as Put, conflict not mapped, wrong-sequence not recognised, conflict answered as success, zero-revision guard removed, launch dropping the code, the bus error carrying the wrong code, the runtime dropping the asked revision.

sh merge-check.sh passes locally (race detector on).

Part of the issue tracker decision, hq research 042 (`01-RESEARCH/042-the-mesh-keeps-its-own-issues`); its ADR has no number yet. Companion: novox/mesh-sdk#16 (`State(...).Increment`). **Finding:** the runtime already carries the expected revision to the bus: `mesh/state.create` is `KeyValue.Create`, `mesh/state.update` is `KeyValue.Update(key, value, revision)` with revision 0 refused, and a lost write (`ErrKeyExists` / wrong last sequence) is `bus.ErrStateChanged` answered to the bundle with code -32010 (#23, hq ADR 0259). Old SDKs only send get/put/delete/keys/watch, which are unchanged. No runtime code changes here: tests only. **Tests** - `internal/runtime` `TestTwoWritersIncrementingOneCounterOnTheBusLoseNoUpdate`: on an embedded nats-server with JetStream (internal/meshtest, a bus per test, so it never skips), two writers x 1000 increments through the module's bus as a bundle asks (`mesh/state.get` then `create`/`update` at the revision read, retry when the code is -32010, bounded) end at 2000, every number once, rising per writer. A local run logged 1646 lost writes retried, so the contention is real. - `TestAStaleWriteIsRefusedAndWritesNothing`: update at revision 0 refused (not as a lost write) and writes nothing; second create, stale revision and future revision each refused with -32010 and `ErrStateChanged`, value unchanged; a deleted key can be created again. - `internal/launch` `TestALostWriteIsAnsweredWithItsOwnCodeAndNothingElseIs`: the code reaches the bundle wrapped or not; a plain refusal is -32000. - `merge-check.sh`: both runtime tests added to the list that runs where no bundle can be launched, so `mesh/repo-check` runs them. **Mutations, each killed:** Update as Put, Create as Put, conflict not mapped, wrong-sequence not recognised, conflict answered as success, zero-revision guard removed, launch dropping the code, the bus error carrying the wrong code, the runtime dropping the asked revision. `sh merge-check.sh` passes locally (race detector on).
mesh-admin added 1 commit 2026-10-10 10:31:29 +00:00
Prove a write by compare-and-set loses no update under contention, on a real bus (hq research 042)
mesh/delivery-group group feat/a-counter-by-compare-and-set checking: 0 of 2 member(s) ready
mesh/merge-gate pass: builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of …
mesh/repo-check pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; it…
mesh/delivery superseded: a newer head of the same pull request
1eaa1abbe9
The issues module will hand out numbers through mesh/state.create and update. Two writers incrementing one
counter a thousand times each through the module's bus now end at two thousand with every number handed out
once; a stale, zero or future revision is refused with the lost-write code and writes nothing; a deleted key
can be created again; and the lost-write code reaches a bundle wrapped or not, and no other refusal carries it.
Both runtime tests run in the check's bundleless list, so mesh/repo-check runs them.

Part of the issue tracker decision, hq research 042.
Author
Contributor

Delivery novox/mesh-tools@d0128adf5bd0 — delivered since 2026-10-10T12:43:12Z

Delivery plan — builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step

  • tier 0: mesh-tools, node-tools
  • ace: receives node-tools
  • g14: receives node-tools
  • novox: receives node-tools
  • shanks: receives node-tools
  • node-tools provides mcp-endpoint: its consumers are sent again after it

Group feat/a-counter-by-compare-and-set, in order: novox/mesh-sdk@9c548753b8 → novox/mesh-tools@d0128adf5b

  • composed together: pass — every machine composes with the change as it did without (4 of 4 compose)

Machines

  • mesh-tools on every machine running it: sent
  • node-tools on ace: passed — healthy 3 times over 2m29s
  • node-tools on the rest running it: sent

Transitions

  • 2026-10-10 10:33 (new) → proposed (announced): novox/mesh-tools#25's head announced
  • 2026-10-10 10:36 proposed → checked (checked): the verdict names this commit
  • 2026-10-10 10:36 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
  • 2026-10-10 12:39 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
  • 2026-10-10 12:39 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
  • 2026-10-10 12:43 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

<!-- mesh-delivery:view --> **Delivery** `novox/mesh-tools@d0128adf5bd0` — **delivered** since 2026-10-10T12:43:12Z **Delivery plan** — builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step - tier 0: mesh-tools, node-tools - ace: receives node-tools - g14: receives node-tools - novox: receives node-tools - shanks: receives node-tools - node-tools provides mcp-endpoint: its consumers are sent again after it **Group** `feat/a-counter-by-compare-and-set`, in order: novox/mesh-sdk@9c548753b800 → novox/mesh-tools@d0128adf5bd0 - composed together: pass — every machine composes with the change as it did without (4 of 4 compose) **Machines** - mesh-tools on every machine running it: sent - node-tools on ace: passed — healthy 3 times over 2m29s - node-tools on the rest running it: sent **Transitions** - 2026-10-10 10:33 (new) → proposed (announced): novox/mesh-tools#25's head announced - 2026-10-10 10:36 proposed → checked (checked): the verdict names this commit - 2026-10-10 10:36 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-10 12:39 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-10 12:39 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit - 2026-10-10 12:43 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
jschoubben added 1 commit 2026-10-10 10:33:45 +00:00
Fail the contention test when the two writers never collided: it would prove nothing (the review of 2026-10-10)
mesh/merge-gate pass: builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of …
mesh/repo-check pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; it…
mesh/delivery delivered
mesh/delivery-group group feat/a-counter-by-compare-and-set merged: every member is merged
d0128adf5b
Author
Contributor

Merge check at 1eaa1abb

  • mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-tools, node-tools)
  • mesh/repo-check: PASS — THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; its merge-check.sh passed

Delivery plan — builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step

  • tier 0: mesh-tools, node-tools
  • ace: receives node-tools
  • g14: receives node-tools
  • novox: receives node-tools
  • shanks: receives node-tools
  • node-tools provides mcp-endpoint: its consumers are sent again after it

Run by the build seat on ace as build-1791628315184484881 (builds --log build-1791628315184484881).

--- the gate: mesh-tools, node-tools
mesh-tools: ok
node-tools: ok, invokes every tool
long-running resources without health: 0
files asking for a setting without saying whether it is trusted: 0
2 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 0 module(s) changed against 4 machine(s), as the snapshot of 2026-10-10T10:27:18Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-10T10:27:18Z

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 2 module(s) in 1 tier(s): mesh-tools, node-tools
--- THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges
--- its merge-check.sh (go toolchain)
NOT TESTED HERE: internal/runtime and internal/console launch bundles that need node and @novox/mesh-sdk
?   	github.com/novox/mesh-tools/node-tools/cmd/node-tools	[no test files]
ok  	github.com/novox/mesh-tools/node-tools/internal/alive	1.191s
ok  	github.com/novox/mesh-tools/node-tools/internal/announce	4.357s
ok  	github.com/novox/mesh-tools/node-tools/internal/bus	7.654s
ok  	github.com/novox/mesh-tools/node-tools/internal/launch	1.088s
ok  	github.com/novox/mesh-tools/node-tools/internal/meshtest	1.125s
?   	github.com/novox/mesh-tools/node-tools/internal/wire	[no test files]
ok  	github.com/novox/mesh-tools/node-tools/internal/console	0.005s
ok  	github.com/novox/mesh-tools/node-tools/internal/runtime	0.473s
NOT TESTED HERE: node-tools/src (TypeScript) needs @novox/mesh-sdk from the package registry
**Merge check** at `1eaa1abb` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-tools, node-tools) - `mesh/repo-check`: **PASS** — THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; its merge-check.sh passed **Delivery plan** — builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step - tier 0: mesh-tools, node-tools - ace: receives node-tools - g14: receives node-tools - novox: receives node-tools - shanks: receives node-tools - node-tools provides mcp-endpoint: its consumers are sent again after it Run by the build seat on ace as `build-1791628315184484881` (`builds --log build-1791628315184484881`). ``` --- the gate: mesh-tools, node-tools mesh-tools: ok node-tools: ok, invokes every tool long-running resources without health: 0 files asking for a setting without saying whether it is trusted: 0 2 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 0 module(s) changed against 4 machine(s), as the snapshot of 2026-10-10T10:27:18Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-10T10:27:18Z machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 2 module(s) in 1 tier(s): mesh-tools, node-tools --- THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges --- its merge-check.sh (go toolchain) NOT TESTED HERE: internal/runtime and internal/console launch bundles that need node and @novox/mesh-sdk ? github.com/novox/mesh-tools/node-tools/cmd/node-tools [no test files] ok github.com/novox/mesh-tools/node-tools/internal/alive 1.191s ok github.com/novox/mesh-tools/node-tools/internal/announce 4.357s ok github.com/novox/mesh-tools/node-tools/internal/bus 7.654s ok github.com/novox/mesh-tools/node-tools/internal/launch 1.088s ok github.com/novox/mesh-tools/node-tools/internal/meshtest 1.125s ? github.com/novox/mesh-tools/node-tools/internal/wire [no test files] ok github.com/novox/mesh-tools/node-tools/internal/console 0.005s ok github.com/novox/mesh-tools/node-tools/internal/runtime 0.473s NOT TESTED HERE: node-tools/src (TypeScript) needs @novox/mesh-sdk from the package registry ```
Author
Contributor

Merge check at d0128adf

  • mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-tools, node-tools)
  • mesh/repo-check: PASS — THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; its merge-check.sh passed

Delivery plan — builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step

  • tier 0: mesh-tools, node-tools
  • ace: receives node-tools
  • g14: receives node-tools
  • novox: receives node-tools
  • shanks: receives node-tools
  • node-tools provides mcp-endpoint: its consumers are sent again after it

Run by the build seat on g14 as build-1791628437599630289 (builds --log build-1791628437599630289).

--- the gate: mesh-tools, node-tools
mesh-tools: ok
node-tools: ok, invokes every tool
long-running resources without health: 0
files asking for a setting without saying whether it is trusted: 0
2 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 0 module(s) changed against 4 machine(s), as the snapshot of 2026-10-10T10:27:18Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-10T10:27:18Z

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 2 module(s) in 1 tier(s): mesh-tools, node-tools
--- THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges
--- its merge-check.sh (go toolchain)
NOT TESTED HERE: internal/runtime and internal/console launch bundles that need node and @novox/mesh-sdk
?   	github.com/novox/mesh-tools/node-tools/cmd/node-tools	[no test files]
ok  	github.com/novox/mesh-tools/node-tools/internal/alive	1.191s
ok  	github.com/novox/mesh-tools/node-tools/internal/announce	4.358s
ok  	github.com/novox/mesh-tools/node-tools/internal/bus	7.774s
ok  	github.com/novox/mesh-tools/node-tools/internal/launch	0.069s
ok  	github.com/novox/mesh-tools/node-tools/internal/meshtest	1.114s
?   	github.com/novox/mesh-tools/node-tools/internal/wire	[no test files]
ok  	github.com/novox/mesh-tools/node-tools/internal/console	0.004s
ok  	github.com/novox/mesh-tools/node-tools/internal/runtime	0.476s
NOT TESTED HERE: node-tools/src (TypeScript) needs @novox/mesh-sdk from the package registry
**Merge check** at `d0128adf` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-tools, node-tools) - `mesh/repo-check`: **PASS** — THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; its merge-check.sh passed **Delivery plan** — builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step - tier 0: mesh-tools, node-tools - ace: receives node-tools - g14: receives node-tools - novox: receives node-tools - shanks: receives node-tools - node-tools provides mcp-endpoint: its consumers are sent again after it Run by the build seat on g14 as `build-1791628437599630289` (`builds --log build-1791628437599630289`). ``` --- the gate: mesh-tools, node-tools mesh-tools: ok node-tools: ok, invokes every tool long-running resources without health: 0 files asking for a setting without saying whether it is trusted: 0 2 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 0 module(s) changed against 4 machine(s), as the snapshot of 2026-10-10T10:27:18Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-10T10:27:18Z machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 2 module(s) in 1 tier(s): mesh-tools, node-tools --- THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges --- its merge-check.sh (go toolchain) NOT TESTED HERE: internal/runtime and internal/console launch bundles that need node and @novox/mesh-sdk ? github.com/novox/mesh-tools/node-tools/cmd/node-tools [no test files] ok github.com/novox/mesh-tools/node-tools/internal/alive 1.191s ok github.com/novox/mesh-tools/node-tools/internal/announce 4.358s ok github.com/novox/mesh-tools/node-tools/internal/bus 7.774s ok github.com/novox/mesh-tools/node-tools/internal/launch 0.069s ok github.com/novox/mesh-tools/node-tools/internal/meshtest 1.114s ? github.com/novox/mesh-tools/node-tools/internal/wire [no test files] ok github.com/novox/mesh-tools/node-tools/internal/console 0.004s ok github.com/novox/mesh-tools/node-tools/internal/runtime 0.476s NOT TESTED HERE: node-tools/src (TypeScript) needs @novox/mesh-sdk from the package registry ```
mesh-admin merged commit c761c22c2a into main 2026-10-10 12:37:43 +00:00
mesh-admin deleted branch feat/a-counter-by-compare-and-set 2026-10-10 12:37:43 +00:00
Sign in to join this conversation.
No Reviewers
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: novox/mesh-tools#25