An act's answer is said from one resolution after its write (hq issue 237) #253

Merged
mesh-admin merged 1 commits from fix/237-one-resolution-for-assign into main 2026-10-12 13:31:03 +00:00
Contributor

What happened (issue 237, 2026-10-04). A workstation already ran a module depending on node-hotkeys; assigning the holder answered "asus-zephyrus-g14 now has node-hotkeys held" and then " is left out of the rest of the mesh: it does not resolve … assign one that holds it: triggerhappy". plan right after resolved.

Diagnosis (the issue's question 1). The tail did not come from a cached view or the catalogue. It came from a read made before the write: assignWith calls recordDerivedHolders (issue 170) before recording the assignment. That calls theRestOfTheMesh(""), which resolves every node with the old assignments and prints left out … to standard error. The controller seat's verb answers with stdout followed by stderr (answerOf), so the stale line came last.

What changes (the issue's question 2, decided 2026-10-11).

  • theRestOfTheMesh says a left-out machine through sayLeftOut(ctx, …). It still goes to stderr by default, so every other reader keeps issue 188's line. An act sets leftOutSaidBy(ctx, io.Discard).
  • assign takes one planFor of the node after the write (resolvedAfter). The node's "now has … held" and "but … depends on" lines come from that resolution's Unheld and modules (the closure, as plan counts them), through unheldChangeTo. Whether the node resolves comes from the same resolution. Before, the after-side was judged over before+added names.
  • unassign uses the same resolution. It now also says, once and inside its answer, when its node does not resolve after the act. Before, that was said only on stderr, once per other machine.
  • Every other machine is still said by blockedElsewhere, which is read after the write.
  • settings composes no resolution in its answer, so there was nothing to change there.

Limit I chose. When the node does not resolve, there is no closure to read. The unmet dependencies are then judged over the assignments read after the write. That set can only find more unmet seats than the closure would, so a "now has … held" line can never contradict the refusal.

Replay. TestReplay237 (replay237_test.go) uses only what main had (register, assignCommand, planFor). It replays the incident with a container runtime: a node runs web (a container) from before the rule, and docker is assigned. On main it fails with the incident's exact text ("laptop is left out of the rest of the mesh … assign one that holds it: docker"). With the fix it passes. It still needs registering in mesh-lab replays/register.go.

Tests.

  • one_resolution_test.go:
    • an unassign that leaves its node unresolved says so exactly once in its answer, and carries no left out line (fails on main: said 0 times, plus the stderr line);
    • an unassign that leaves the node resolved says nothing of it;
    • an assign that leaves the node unresolved returns exactly planFor's refusal and no "now has" line;
    • sayLeftOut still reaches stderr without an act's writer.
  • The existing unheldChange/seat-dependency tests pass unchanged.
  • sh merge-check.sh passes locally against a throwaway PostgreSQL (gofmt, vet, go test -race, store tests).

Reviewers' checklist.

  • Every line of assign/unassign's answer about the node comes from the resolution after the write.
  • Other callers of theRestOfTheMesh (status, holdings, delivery, source, seats, build source) still print the left-out line.
  • The replay fails on main and passes here.

Core: the controller. One controller rollout at a time; merge only while mesh-controller.plans says done. It does not touch #188's or #251's files.

Fixes issue 237.

**What happened (issue 237, 2026-10-04).** A workstation already ran a module depending on `node-hotkeys`; assigning the holder answered "asus-zephyrus-g14 now has node-hotkeys held" and then "<node> is left out of the rest of the mesh: it does not resolve … assign one that holds it: triggerhappy". `plan` right after resolved. **Diagnosis (the issue's question 1).** The tail did not come from a cached view or the catalogue. It came from a read made **before the write**: `assignWith` calls `recordDerivedHolders` (issue 170) before recording the assignment. That calls `theRestOfTheMesh("")`, which resolves every node with the old assignments and prints `left out …` to **standard error**. The controller seat's verb answers with stdout followed by stderr (`answerOf`), so the stale line came last. **What changes (the issue's question 2, decided 2026-10-11).** - `theRestOfTheMesh` says a left-out machine through `sayLeftOut(ctx, …)`. It still goes to stderr by default, so every other reader keeps issue 188's line. An act sets `leftOutSaidBy(ctx, io.Discard)`. - `assign` takes **one** `planFor` of the node after the write (`resolvedAfter`). The node's "now has … held" and "but … depends on" lines come from that resolution's `Unheld` and modules (the closure, as `plan` counts them), through `unheldChangeTo`. Whether the node resolves comes from the same resolution. Before, the after-side was judged over `before+added` names. - `unassign` uses the same resolution. It now also says, once and inside its answer, when its node does not resolve after the act. Before, that was said only on stderr, once per other machine. - Every other machine is still said by `blockedElsewhere`, which is read after the write. - `settings` composes no resolution in its answer, so there was nothing to change there. **Limit I chose.** When the node does not resolve, there is no closure to read. The unmet dependencies are then judged over the assignments read after the write. That set can only find more unmet seats than the closure would, so a "now has … held" line can never contradict the refusal. **Replay.** `TestReplay237` (`replay237_test.go`) uses only what main had (register, `assignCommand`, `planFor`). It replays the incident with a container runtime: a node runs `web` (a container) from before the rule, and `docker` is assigned. On main it fails with the incident's exact text ("laptop is left out of the rest of the mesh … assign one that holds it: docker"). With the fix it passes. It still needs registering in mesh-lab `replays/register.go`. **Tests.** - `one_resolution_test.go`: - an unassign that leaves its node unresolved says so exactly once in its answer, and carries no `left out` line (fails on main: said 0 times, plus the stderr line); - an unassign that leaves the node resolved says nothing of it; - an assign that leaves the node unresolved returns exactly `planFor`'s refusal and no "now has" line; - `sayLeftOut` still reaches stderr without an act's writer. - The existing `unheldChange`/seat-dependency tests pass unchanged. - `sh merge-check.sh` passes locally against a throwaway PostgreSQL (gofmt, vet, `go test -race`, store tests). **Reviewers' checklist.** - [ ] Every line of `assign`/`unassign`'s answer about the node comes from the resolution after the write. - [ ] Other callers of `theRestOfTheMesh` (status, holdings, delivery, source, seats, build source) still print the left-out line. - [ ] The replay fails on main and passes here. **Core**: the controller. One controller rollout at a time; merge only while `mesh-controller.plans` says done. It does not touch #188's or #251's files. Fixes issue 237.
Author
Contributor

Delivery novox/mesh-controller@3be6e86f77c0 — delivered since 2026-10-12T13:36:46Z

Delivery plan — builds mesh-controller, route-proxy → ace, novox; no bus step

  • tier 0: mesh-controller, route-proxy
  • ace: receives route-proxy
  • novox: receives mesh-controller, route-proxy
  • route-proxy provides route: its consumers are sent again after it
  • route-proxy keeps data (ADR 0233): what it holds is backed up before it moves
  • grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before route-proxy is judged on its first machine

Machines

  • mesh-controller on novox: passed — healthy 3 times over 4m7s
  • mesh-controller on the rest running it: sent

Transitions

  • 2026-10-12 11:57 (new) → proposed (announced): novox/mesh-controller#253's head announced
  • 2026-10-12 12:07 proposed → checked (checked): the verdict names this commit
  • 2026-10-12 12:07 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
  • 2026-10-12 13:31 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
  • 2026-10-12 13:31 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
  • 2026-10-12 13:36 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

<!-- mesh-delivery:view --> **Delivery** `novox/mesh-controller@3be6e86f77c0` — **delivered** since 2026-10-12T13:36:46Z **Delivery plan** — builds mesh-controller, route-proxy → ace, novox; no bus step - tier 0: mesh-controller, route-proxy - ace: receives route-proxy - novox: receives mesh-controller, route-proxy - route-proxy provides route: its consumers are sent again after it - route-proxy keeps data (ADR 0233): what it holds is backed up before it moves - grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before route-proxy is judged on its first machine **Machines** - mesh-controller on novox: passed — healthy 3 times over 4m7s - mesh-controller on the rest running it: sent **Transitions** - 2026-10-12 11:57 (new) → proposed (announced): novox/mesh-controller#253's head announced - 2026-10-12 12:07 proposed → checked (checked): the verdict names this commit - 2026-10-12 12:07 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-12 13:31 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-12 13:31 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit - 2026-10-12 13:36 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
Author
Contributor

Merge check at 86044520

  • mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-controller)
  • mesh/repo-check: FAIL — its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.71s)

Delivery plan — builds mesh-controller → novox; no bus step

  • tier 0: mesh-controller
  • novox: receives mesh-controller
  • read by no module's build: cmd/mesh-controller/one_resolution_test.go, cmd/mesh-controller/replay237_test.go

Run by the build seat on ace as build-1791804712270050743 (builds --log build-1791804712270050743).

--- the gate: mesh-controller
mesh-controller: ok, 57 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-12T11:32:09Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-12T11:32:09Z

modules the change moves:
  - mesh-controller (changed)

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 1 module(s) in 1 tier(s): mesh-controller

notes:
  - cmd/mesh-controller/one_resolution_test.go and cmd/mesh-controller/replay237_test.go read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
ok  	github.com/novox/mesh-controller/cmd/mesh-builder	1.045s
ok  	github.com/novox/mesh-controller/cmd/mesh-controller	250.748s
ok  	github.com/novox/mesh-controller/examples/objectstore-provisioner	1.020s
ok  	github.com/novox/mesh-controller/examples/postgres-provisioner	1.030s
ok  	github.com/novox/mesh-controller/examples/redis-provisioner	1.019s
ok  	github.com/novox/mesh-controller/examples/route-proxy	1.227s
ok  	github.com/novox/mesh-controller/internal/artifacts	1.069s
?   	github.com/novox/mesh-controller/internal/beside	[no test files]
--- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.71s)
    genesis_template_test.go:29: judged against /var/lib/build-agent/workspace/check/mesh-host as cloned beside this check
    genesis_template_test.go:48: the installer's user list lets the controller publish [mesh.seat.mesh-delivery.tool.groups], which it does not derive
FAIL
FAIL	github.com/novox/mesh-controller/internal/broker	3.375s
ok  	github.com/novox/mesh-controller/internal/builder	16.987s
ok  	github.com/novox/mesh-controller/internal/catalogue	8.701s
ok  	github.com/novox/mesh-controller/internal/conditions	2.909s
ok  	github.com/novox/mesh-controller/internal/envfile	1.088s
ok  	github.com/novox/mesh-controller/internal/facts	1.074s
ok  	github.com/novox/mesh-controller/internal/identity	5.162s
ok  	github.com/novox/mesh-controller/internal/inventory	95.713s
ok  	github.com/novox/mesh-controller/internal/lease	17.728s
ok  	github.com/novox/mesh-controller/internal/licences	4.672s
ok  	github.com/novox/mesh-controller/internal/licences/adapters	1.070s
ok  	github.com/novox/mesh-controller/internal/link	60.298s
ok  	github.com/novox/mesh-controller/internal/lint	2.559s
ok  	github.com/novox/mesh-controller/internal/outward	1.125s
ok  	github.com/novox/mesh-controller/internal/overlay	1.094s
ok  	github.com/novox/mesh-controller/internal/secrets	1.543s
ok  	github.com/novox/mesh-controller/internal/store	3.756s
ok  	github.com/novox/mesh-controller/internal/testbus	1.301s
ok  	github.com/novox/mesh-controller/internal/token	1.023s
FAIL
--- what failed, picked from the whole of its output (the whole is in the build's log)
--- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.71s)
    genesis_template_test.go:29: judged against /var/lib/build-agent/workspace/check/mesh-host as cloned beside this check
    genesis_template_test.go:48: the installer's user list lets the controller publish [mesh.seat.mesh-delivery.tool.groups], which it does not derive
FAIL	github.com/novox/mesh-controller/internal/broker	3.375s
**Merge check** at `86044520` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-controller) - `mesh/repo-check`: **FAIL** — its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.71s) **Delivery plan** — builds mesh-controller → novox; no bus step - tier 0: mesh-controller - novox: receives mesh-controller - read by no module's build: cmd/mesh-controller/one_resolution_test.go, cmd/mesh-controller/replay237_test.go Run by the build seat on ace as `build-1791804712270050743` (`builds --log build-1791804712270050743`). ``` --- the gate: mesh-controller mesh-controller: ok, 57 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8) long-running resources without health: 1 files asking for a setting without saying whether it is trusted: 1 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-12T11:32:09Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-12T11:32:09Z modules the change moves: - mesh-controller (changed) machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 1 module(s) in 1 tier(s): mesh-controller notes: - cmd/mesh-controller/one_resolution_test.go and cmd/mesh-controller/replay237_test.go read by no module's build, and rebuild nothing (issue 280) --- its merge-check.sh (go toolchain) ok github.com/novox/mesh-controller/cmd/mesh-builder 1.045s ok github.com/novox/mesh-controller/cmd/mesh-controller 250.748s ok github.com/novox/mesh-controller/examples/objectstore-provisioner 1.020s ok github.com/novox/mesh-controller/examples/postgres-provisioner 1.030s ok github.com/novox/mesh-controller/examples/redis-provisioner 1.019s ok github.com/novox/mesh-controller/examples/route-proxy 1.227s ok github.com/novox/mesh-controller/internal/artifacts 1.069s ? github.com/novox/mesh-controller/internal/beside [no test files] --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.71s) genesis_template_test.go:29: judged against /var/lib/build-agent/workspace/check/mesh-host as cloned beside this check genesis_template_test.go:48: the installer's user list lets the controller publish [mesh.seat.mesh-delivery.tool.groups], which it does not derive FAIL FAIL github.com/novox/mesh-controller/internal/broker 3.375s ok github.com/novox/mesh-controller/internal/builder 16.987s ok github.com/novox/mesh-controller/internal/catalogue 8.701s ok github.com/novox/mesh-controller/internal/conditions 2.909s ok github.com/novox/mesh-controller/internal/envfile 1.088s ok github.com/novox/mesh-controller/internal/facts 1.074s ok github.com/novox/mesh-controller/internal/identity 5.162s ok github.com/novox/mesh-controller/internal/inventory 95.713s ok github.com/novox/mesh-controller/internal/lease 17.728s ok github.com/novox/mesh-controller/internal/licences 4.672s ok github.com/novox/mesh-controller/internal/licences/adapters 1.070s ok github.com/novox/mesh-controller/internal/link 60.298s ok github.com/novox/mesh-controller/internal/lint 2.559s ok github.com/novox/mesh-controller/internal/outward 1.125s ok github.com/novox/mesh-controller/internal/overlay 1.094s ok github.com/novox/mesh-controller/internal/secrets 1.543s ok github.com/novox/mesh-controller/internal/store 3.756s ok github.com/novox/mesh-controller/internal/testbus 1.301s ok github.com/novox/mesh-controller/internal/token 1.023s FAIL --- what failed, picked from the whole of its output (the whole is in the build's log) --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.71s) genesis_template_test.go:29: judged against /var/lib/build-agent/workspace/check/mesh-host as cloned beside this check genesis_template_test.go:48: the installer's user list lets the controller publish [mesh.seat.mesh-delivery.tool.groups], which it does not derive FAIL github.com/novox/mesh-controller/internal/broker 3.375s ```
Author
Contributor

Verdict: approve at 8604452

Independent review from a fresh clone (not the forge checkout), with all four checks the reviewers' checklist and this review's brief asked for run first-hand rather than taken on the PR's word.

Scope. git diff $(git merge-base main 8604452) 8604452 touches only acts.go, plan.go, and the two new test files — matches the PR diff. A raw git diff main 8604452 shows more because main has since absorbed #188 and #252, neither of which this branch touches.

The self-contradiction is fixed, proven both ways.

  • Laid replay237_test.go alone over current main (no fix) and ran it against a throwaway Postgres: it fails with the incident's own text — "laptop is left out ... does not resolve ... assign one that holds it: docker" beside "now has ... held" — reproducing issue 237 exactly.
  • On 8604452, TestReplay237 and all four one_resolution_test.go cases pass.

Read the fix, not just the description.

  • leftOutSaidBy(ctx, io.Discard) is set once at the top of assignWith/unassign and reaches recordDerivedHolders's pre-write theRestOfTheMesh call (the stale read issue 237 diagnosed) and the post-write resolvedAfter/planFor call. blockedElsewhere never goes through sayLeftOut — it calls planFor per other node and reads the returned error directly — so nothing is lost there, only a redundant stderr echo for the acting node's own resolution is silenced.
  • Every other caller of theRestOfTheMesh (checks.go, delivery.go, holdings.go, build_source.go, source.go ×2, seats.go) runs with a fresh ctx, so sayLeftOut's os.Stderr default still fires — issue 188's line is intact everywhere else. settingsCommand calls neither planFor nor theRestOfTheMesh, confirming there was nothing to change there.
  • The limit chosen holds up structurally: in resolvedAfter's error branch, unheldOver is judged over the directly-assigned names, a subset of the real closure, so it can only ever find a seat unmet when the closure would agree or do worse — never the reverse. And unheldChangeTo builds the "now has … held" lines and the "but … depends on" lines from disjoint key sets, so the same seat can never appear in both; the exact shape of the original bug is now structurally impossible, not just avoided by the replay's one scenario.

The red repo-check is confirmed pre-existing and unrelated, not taken on the builder's word. I ran TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose myself with MESH_CHECK_BESIDE pointed at a fresh clone of mesh-host main, on both 8604452 and on mesh-controller main (4ab441c): identical failure both times ("lets the controller publish [mesh.seat.mesh-delivery.tool.groups], which it does not derive"). mesh-host main's installer template already grants that permission (#85, carried by #250); mesh-controller main hasn't merged #250 yet to derive it. Nothing here touches that path. mesh/merge-gate already reads PASS (4 of 4 compose).

Full local verification.

  • gofmt -l cmd internal examples: clean.
  • CGO_ENABLED=0 go vet ./...: clean.
  • CGO_ENABLED=1 go test -race -count=1 -timeout 30m ./... against a throwaway PostgreSQL 17 (make postgres, port 55532): every package ok, including internal/broker (which passes here because without MESH_CHECK_BESIDE it judges against the captured testdata/beside snapshot, not a live mesh-host clone — consistent with the finding above, not a contradiction of it).

One nit, not a blocker. TestReplay237 and the four one_resolution_test.go cases all go through aMesh(t), which raises a real Postgres via inventory.ForTest — confirmed, since TestReplay237 explicitly SKIPs with "no MESH_TEST_POSTGRES; run make check to raise one" when no store is given. None of the five were added to testdata/store-tests, unlike this repository's own convention (e.g. #249) for guarding against a silent skip-as-pass in a misconfigured check. Worth adding in a follow-up; the repo-check's own log shows them genuinely running (not skipped) in this run, so it does not block.

Reviewers' checklist: every line of assign/unassign's answer about the node comes from the resolution after the write — yes. Other callers of theRestOfTheMesh still print the left-out line — yes. The replay fails on main and passes here — yes, reproduced directly.

Verdict: approve at 8604452 Independent review from a fresh clone (not the forge checkout), with all four checks the reviewers' checklist and this review's brief asked for run first-hand rather than taken on the PR's word. **Scope.** `git diff $(git merge-base main 8604452) 8604452` touches only acts.go, plan.go, and the two new test files — matches the PR diff. A raw `git diff main 8604452` shows more because main has since absorbed #188 and #252, neither of which this branch touches. **The self-contradiction is fixed, proven both ways.** - Laid `replay237_test.go` alone over current main (no fix) and ran it against a throwaway Postgres: it fails with the incident's own text — "laptop is left out ... does not resolve ... assign one that holds it: docker" beside "now has ... held" — reproducing issue 237 exactly. - On 8604452, `TestReplay237` and all four `one_resolution_test.go` cases pass. **Read the fix, not just the description.** - `leftOutSaidBy(ctx, io.Discard)` is set once at the top of `assignWith`/`unassign` and reaches `recordDerivedHolders`'s pre-write `theRestOfTheMesh` call (the stale read issue 237 diagnosed) and the post-write `resolvedAfter`/`planFor` call. `blockedElsewhere` never goes through `sayLeftOut` — it calls `planFor` per other node and reads the returned error directly — so nothing is lost there, only a redundant stderr echo for the acting node's own resolution is silenced. - Every other caller of `theRestOfTheMesh` (checks.go, delivery.go, holdings.go, build_source.go, source.go ×2, seats.go) runs with a fresh ctx, so `sayLeftOut`'s `os.Stderr` default still fires — issue 188's line is intact everywhere else. `settingsCommand` calls neither `planFor` nor `theRestOfTheMesh`, confirming there was nothing to change there. - The limit chosen holds up structurally: in `resolvedAfter`'s error branch, `unheldOver` is judged over the directly-assigned names, a subset of the real closure, so it can only ever find a seat unmet when the closure would agree or do worse — never the reverse. And `unheldChangeTo` builds the "now has … held" lines and the "but … depends on" lines from disjoint key sets, so the same seat can never appear in both; the exact shape of the original bug is now structurally impossible, not just avoided by the replay's one scenario. **The red repo-check is confirmed pre-existing and unrelated, not taken on the builder's word.** I ran `TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose` myself with `MESH_CHECK_BESIDE` pointed at a fresh clone of mesh-host main, on both 8604452 and on mesh-controller main (4ab441c): identical failure both times ("lets the controller publish [mesh.seat.mesh-delivery.tool.groups], which it does not derive"). mesh-host main's installer template already grants that permission (#85, carried by #250); mesh-controller main hasn't merged #250 yet to derive it. Nothing here touches that path. `mesh/merge-gate` already reads PASS (4 of 4 compose). **Full local verification.** - `gofmt -l cmd internal examples`: clean. - `CGO_ENABLED=0 go vet ./...`: clean. - `CGO_ENABLED=1 go test -race -count=1 -timeout 30m ./...` against a throwaway PostgreSQL 17 (`make postgres`, port 55532): every package ok, including `internal/broker` (which passes here because without `MESH_CHECK_BESIDE` it judges against the captured `testdata/beside` snapshot, not a live mesh-host clone — consistent with the finding above, not a contradiction of it). **One nit, not a blocker.** `TestReplay237` and the four `one_resolution_test.go` cases all go through `aMesh(t)`, which raises a real Postgres via `inventory.ForTest` — confirmed, since `TestReplay237` explicitly SKIPs with "no MESH_TEST_POSTGRES; run `make check` to raise one" when no store is given. None of the five were added to `testdata/store-tests`, unlike this repository's own convention (e.g. #249) for guarding against a silent skip-as-pass in a misconfigured check. Worth adding in a follow-up; the repo-check's own log shows them genuinely running (not skipped) in this run, so it does not block. Reviewers' checklist: every line of `assign`/`unassign`'s answer about the node comes from the resolution after the write — yes. Other callers of `theRestOfTheMesh` still print the left-out line — yes. The replay fails on main and passes here — yes, reproduced directly.
jschoubben added 1 commit 2026-10-12 11:56:41 +00:00
Say an act's answer from one resolution after its write (issue 237)
mesh/merge-gate pass: builds mesh-controller, route-proxy → ace, novox; no bus step; every machine composes with the change as it did without (4 of 4 compo…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
3be6e86f77
assign read the whole mesh before its write, to record the seats held by
derivation, and that read printed on standard error that the node did not
resolve for lack of the seat the act was about to give it. The seat verb
answers with both streams, so the stale line came last and named the
assignment just made as its remedy.

The reads inside an act no longer print 'left out' lines; the node's
unmet seat dependencies and whether it resolves come from one planFor
taken after the write, and every other machine from blockedElsewhere.
unassign now says when its node does not resolve, from the same
resolution.
jschoubben force-pushed fix/237-one-resolution-for-assign from 8604452015 to 3be6e86f77 2026-10-12 11:56:41 +00:00 Compare
Author
Contributor

Merge check at 3be6e86f

  • mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-controller, route-proxy)
  • mesh/repo-check: PASS — its merge-check.sh passed

Delivery plan — builds mesh-controller, route-proxy → ace, novox; no bus step

  • tier 0: mesh-controller, route-proxy
  • ace: receives route-proxy
  • novox: receives mesh-controller, route-proxy
  • route-proxy provides route: its consumers are sent again after it
  • route-proxy keeps data (ADR 0233): what it holds is backed up before it moves
  • grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before route-proxy is judged on its first machine

Run by the build seat on g14 as build-1791806223919801910 (builds --log build-1791806223919801910).

--- the gate: mesh-controller, route-proxy
mesh-controller: ok, 57 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-12T11:52:38Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-12T11:52:38Z

modules the change moves:
  - mesh-controller (changed)

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 3 module(s) in 3 tier(s): build-agent, mesh-controller, route-proxy
--- its merge-check.sh (go toolchain)
ok  	github.com/novox/mesh-controller/cmd/mesh-builder	1.039s
ok  	github.com/novox/mesh-controller/cmd/mesh-controller	252.352s
ok  	github.com/novox/mesh-controller/examples/objectstore-provisioner	1.016s
ok  	github.com/novox/mesh-controller/examples/postgres-provisioner	1.024s
ok  	github.com/novox/mesh-controller/examples/redis-provisioner	1.021s
ok  	github.com/novox/mesh-controller/examples/route-proxy	1.152s
ok  	github.com/novox/mesh-controller/internal/artifacts	1.051s
?   	github.com/novox/mesh-controller/internal/beside	[no test files]
ok  	github.com/novox/mesh-controller/internal/broker	4.070s
ok  	github.com/novox/mesh-controller/internal/builder	14.176s
ok  	github.com/novox/mesh-controller/internal/catalogue	6.948s
ok  	github.com/novox/mesh-controller/internal/conditions	2.470s
ok  	github.com/novox/mesh-controller/internal/envfile	1.032s
ok  	github.com/novox/mesh-controller/internal/facts	1.032s
ok  	github.com/novox/mesh-controller/internal/identity	4.334s
ok  	github.com/novox/mesh-controller/internal/inventory	86.697s
ok  	github.com/novox/mesh-controller/internal/lease	17.699s
ok  	github.com/novox/mesh-controller/internal/licences	3.920s
ok  	github.com/novox/mesh-controller/internal/licences/adapters	1.007s
ok  	github.com/novox/mesh-controller/internal/link	59.952s
ok  	github.com/novox/mesh-controller/internal/lint	2.231s
ok  	github.com/novox/mesh-controller/internal/outward	1.037s
ok  	github.com/novox/mesh-controller/internal/overlay	1.028s
ok  	github.com/novox/mesh-controller/internal/secrets	1.465s
ok  	github.com/novox/mesh-controller/internal/store	3.313s
ok  	github.com/novox/mesh-controller/internal/testbus	1.181s
ok  	github.com/novox/mesh-controller/internal/token	1.021s
store tests, each run against the store:
  ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeWalksTheOthersAgain (0.34s)
  ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeEndsTheMergesThatContainIt (0.40s)
  ./cmd/mesh-controller --- PASS: TestAStopWalksTheOthersAgainInTheOpenBatch (0.37s)
  ./cmd/mesh-controller --- PASS: TestAStopNamesWhatStaysOnTheMachinesAndWhatStillCarriesIt (0.36s)
  ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeLeavesItOutOfABatch (0.34s)
  ./cmd/mesh-controller --- PASS: TestSavingPlansAndMovingMergesIsOneAct (0.33s)
  ./cmd/mesh-controller --- PASS: TestNodeShowAndStatusReadTheLastSendFromTheStore (0.39s)
  ./cmd/mesh-controller --- PASS: TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds (0.26s)
  ./cmd/mesh-controller --- PASS: TestAWalkLetGoAndStoppedByVerbsIsSaidStartedAndEnded (0.40s)
  ./cmd/mesh-controller --- PASS: TestAWalkStoppedOrClosedByAPersonIsSaidEndedStopped (0.39s)
  ./cmd/mesh-controller --- PASS: TestAWalkRetriedByAPersonIsSaidStartedAgain (0.36s)
  ./cmd/mesh-controller --- PASS: TestABacklogReleasedByAPersonIsSaidStarted (0.35s)
  ./cmd/mesh-controller --- PASS: TestAWalkEndedByCancelOrClearIsSaidEnded (0.39s)
  ./cmd/mesh-controller --- PASS: TestAWalkRestartedByARebuildIsSaidStarted (0.36s)
  ./cmd/mesh-controller --- PASS: TestAWalkStoppedByHandAndRetriedEndsFailedWhenItFailsAgain (0.39s)
  ./cmd/mesh-controller --- PASS: TestReplay237 (0.36s)
  ./cmd/mesh-controller --- PASS: TestAnUnassignThatLeavesItsNodeUnresolvedSaysSoOnceInItsAnswer (0.36s)
  ./cmd/mesh-controller --- PASS: TestAnUnassignThatLeavesItsNodeResolvedSaysNothingOfIt (0.35s)
  ./cmd/mesh-controller --- PASS: TestAnAssignThatLeavesItsNodeUnresolvedSaysOnlyThePlansRefusal (0.37s)
  ./internal/inventory --- PASS: TestTheLastSendSaysWhatTheMachineAnswered (0.20s)
  ./internal/inventory --- PASS: TestAReportIsTheLastSendsAnswerOnlyWhenItIsAboutThatSend (0.22s)
  ./internal/inventory --- PASS: TestAnIssuedValueIsSealedToEveryHolderOnceAndNotAgain (0.23s)
  ./internal/inventory --- PASS: TestAnIssuedValueIsSealedAgainAfterAKeyChangeOrAnAcceptance (0.21s)
  ./internal/inventory --- PASS: TestAConditionsLooksAreKeptBesideTheBucket (0.19s)
  ./internal/inventory --- PASS: TestAWalksStartIsKeptAndARefusedSaveLeavesItUnsaid (0.19s)
**Merge check** at `3be6e86f` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-controller, route-proxy) - `mesh/repo-check`: **PASS** — its merge-check.sh passed **Delivery plan** — builds mesh-controller, route-proxy → ace, novox; no bus step - tier 0: mesh-controller, route-proxy - ace: receives route-proxy - novox: receives mesh-controller, route-proxy - route-proxy provides route: its consumers are sent again after it - route-proxy keeps data (ADR 0233): what it holds is backed up before it moves - grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before route-proxy is judged on its first machine Run by the build seat on g14 as `build-1791806223919801910` (`builds --log build-1791806223919801910`). ``` --- the gate: mesh-controller, route-proxy mesh-controller: ok, 57 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8) long-running resources without health: 1 files asking for a setting without saying whether it is trusted: 1 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-12T11:52:38Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-12T11:52:38Z modules the change moves: - mesh-controller (changed) machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 3 module(s) in 3 tier(s): build-agent, mesh-controller, route-proxy --- its merge-check.sh (go toolchain) ok github.com/novox/mesh-controller/cmd/mesh-builder 1.039s ok github.com/novox/mesh-controller/cmd/mesh-controller 252.352s ok github.com/novox/mesh-controller/examples/objectstore-provisioner 1.016s ok github.com/novox/mesh-controller/examples/postgres-provisioner 1.024s ok github.com/novox/mesh-controller/examples/redis-provisioner 1.021s ok github.com/novox/mesh-controller/examples/route-proxy 1.152s ok github.com/novox/mesh-controller/internal/artifacts 1.051s ? github.com/novox/mesh-controller/internal/beside [no test files] ok github.com/novox/mesh-controller/internal/broker 4.070s ok github.com/novox/mesh-controller/internal/builder 14.176s ok github.com/novox/mesh-controller/internal/catalogue 6.948s ok github.com/novox/mesh-controller/internal/conditions 2.470s ok github.com/novox/mesh-controller/internal/envfile 1.032s ok github.com/novox/mesh-controller/internal/facts 1.032s ok github.com/novox/mesh-controller/internal/identity 4.334s ok github.com/novox/mesh-controller/internal/inventory 86.697s ok github.com/novox/mesh-controller/internal/lease 17.699s ok github.com/novox/mesh-controller/internal/licences 3.920s ok github.com/novox/mesh-controller/internal/licences/adapters 1.007s ok github.com/novox/mesh-controller/internal/link 59.952s ok github.com/novox/mesh-controller/internal/lint 2.231s ok github.com/novox/mesh-controller/internal/outward 1.037s ok github.com/novox/mesh-controller/internal/overlay 1.028s ok github.com/novox/mesh-controller/internal/secrets 1.465s ok github.com/novox/mesh-controller/internal/store 3.313s ok github.com/novox/mesh-controller/internal/testbus 1.181s ok github.com/novox/mesh-controller/internal/token 1.021s store tests, each run against the store: ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeWalksTheOthersAgain (0.34s) ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeEndsTheMergesThatContainIt (0.40s) ./cmd/mesh-controller --- PASS: TestAStopWalksTheOthersAgainInTheOpenBatch (0.37s) ./cmd/mesh-controller --- PASS: TestAStopNamesWhatStaysOnTheMachinesAndWhatStillCarriesIt (0.36s) ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeLeavesItOutOfABatch (0.34s) ./cmd/mesh-controller --- PASS: TestSavingPlansAndMovingMergesIsOneAct (0.33s) ./cmd/mesh-controller --- PASS: TestNodeShowAndStatusReadTheLastSendFromTheStore (0.39s) ./cmd/mesh-controller --- PASS: TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds (0.26s) ./cmd/mesh-controller --- PASS: TestAWalkLetGoAndStoppedByVerbsIsSaidStartedAndEnded (0.40s) ./cmd/mesh-controller --- PASS: TestAWalkStoppedOrClosedByAPersonIsSaidEndedStopped (0.39s) ./cmd/mesh-controller --- PASS: TestAWalkRetriedByAPersonIsSaidStartedAgain (0.36s) ./cmd/mesh-controller --- PASS: TestABacklogReleasedByAPersonIsSaidStarted (0.35s) ./cmd/mesh-controller --- PASS: TestAWalkEndedByCancelOrClearIsSaidEnded (0.39s) ./cmd/mesh-controller --- PASS: TestAWalkRestartedByARebuildIsSaidStarted (0.36s) ./cmd/mesh-controller --- PASS: TestAWalkStoppedByHandAndRetriedEndsFailedWhenItFailsAgain (0.39s) ./cmd/mesh-controller --- PASS: TestReplay237 (0.36s) ./cmd/mesh-controller --- PASS: TestAnUnassignThatLeavesItsNodeUnresolvedSaysSoOnceInItsAnswer (0.36s) ./cmd/mesh-controller --- PASS: TestAnUnassignThatLeavesItsNodeResolvedSaysNothingOfIt (0.35s) ./cmd/mesh-controller --- PASS: TestAnAssignThatLeavesItsNodeUnresolvedSaysOnlyThePlansRefusal (0.37s) ./internal/inventory --- PASS: TestTheLastSendSaysWhatTheMachineAnswered (0.20s) ./internal/inventory --- PASS: TestAReportIsTheLastSendsAnswerOnlyWhenItIsAboutThatSend (0.22s) ./internal/inventory --- PASS: TestAnIssuedValueIsSealedToEveryHolderOnceAndNotAgain (0.23s) ./internal/inventory --- PASS: TestAnIssuedValueIsSealedAgainAfterAKeyChangeOrAnAcceptance (0.21s) ./internal/inventory --- PASS: TestAConditionsLooksAreKeptBesideTheBucket (0.19s) ./internal/inventory --- PASS: TestAWalksStartIsKeptAndARefusedSaveLeavesItUnsaid (0.19s) ```
mesh-admin merged commit 5facd8b08e into main 2026-10-12 13:31:03 +00:00
mesh-admin deleted branch fix/237-one-resolution-for-assign 2026-10-12 13:31:04 +00:00
This repo is archived. You cannot comment on pull requests.
No Reviewers
2 Participants
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: novox/mesh-controller#253