Wait until the node applied the genesis document before the sdk step asks for a build (issue 615) #105

2026-10-13T09:00:19Z

mesh-admin commented

2026-10-13 09:00:19 +00:00

Issue 615 (genesis's sdk step can ask for its build before the build agent has the genesis document). Related: hq ADR 0336 §7, ADR 0010 (push <node> --wait).

Cause: the bed runs at 07:05 (mesh-host 7d1d6bf) and at 10:49 (4dde21f) print the same lines from step 13 through step 14's "pushed anchor / bus users placed". #104 changed neither step 14 nor the push. The packages step's push returns as soon as the declaration is sent. The sdk step asks for the build at once, and the build agent reads the document from the file the node writes when it applies that push. So it was a race: 07:05 won it; 10:49 lost it with "neither its binding nor the genesis document".

Fix: deliverGenesisRegistry pushes with push anchor --wait <--wait> (new pushNodeApplied). The controller's own wait ends when the node reports it applied exactly that declaration, and fails loudly on a refusal or failed resources. The controller is given twice --wait of patience. Every other push still returns on sending.

Tests: TestThePackagesStepWaitsUntilTheNodeAppliedTheGenesisDocument fails on main. TestAnOrdinaryPushDoesNotWait holds the other pushes as they are. The 608 tests now find the push by its command, not by its suffix. merge-check.sh passes. There is no nats-server here, so the bus tests are skipped, and the script says so.

Fixes issue 615 (lab proof: the genesis-keeps-its-bus bed).

Issue 615 (genesis's sdk step can ask for its build before the build agent has the genesis document). Related: hq ADR 0336 §7, ADR 0010 (`push <node> --wait`). **Cause**: the bed runs at 07:05 (mesh-host 7d1d6bf) and at 10:49 (4dde21f) print the same lines from step 13 through step 14's "pushed anchor / bus users placed". #104 changed neither step 14 nor the push. The packages step's push returns as soon as the declaration is sent. The sdk step asks for the build at once, and the build agent reads the document from the file the node writes when it applies that push. So it was a race: 07:05 won it; 10:49 lost it with "neither its binding nor the genesis document". **Fix**: `deliverGenesisRegistry` pushes with `push anchor --wait <--wait>` (new `pushNodeApplied`). The controller's own wait ends when the node reports it applied exactly that declaration, and fails loudly on a refusal or failed resources. The controller is given twice `--wait` of patience. Every other push still returns on sending. **Tests**: `TestThePackagesStepWaitsUntilTheNodeAppliedTheGenesisDocument` fails on main. `TestAnOrdinaryPushDoesNotWait` holds the other pushes as they are. The 608 tests now find the push by its command, not by its suffix. merge-check.sh passes. There is no nats-server here, so the bus tests are skipped, and the script says so. Fixes issue 615 (lab proof: the genesis-keeps-its-bus bed).

mesh-admin added 1 commit 2026-10-13 09:00:19 +00:00

Wait until the node applied the genesis document before the sdk step asks for a build (hq issue 615)

mesh/merge-gate head 89fd0c06e102 merged into trunk 17038b2a074b — pass: the change touches no module of the mesh's graph

Details

mesh/repo-check head 89fd0c06e102 merged into trunk 17038b2a074b — pass: its merge-check.sh passed

Details

mesh/delivery delivered

Details

89fd0c06e1

The packages step sealed the build agent its genesis document and pushed, and the push returned as soon
as it was sent. The sdk step then asked for the SDK's build at once, and the agent reads the document
from the file the node writes when it applies that push: one genesis found it in time, the next was
refused for having neither a binding nor the document. That one push now waits until the node reports
it applied it (push --wait); every other push still returns on sending.

mesh-admin commented

2026-10-13 09:00:36 +00:00

Delivery novox/mesh-host@89fd0c06e102 — delivered since 2026-10-13T09:09:28Z

Delivery plan — builds nothing: the change touches no module of the mesh's graph

Transitions

2026-10-13 09:00 (new) → proposed (announced): novox/mesh-host#105's head announced
2026-10-13 09:01 proposed → checked (checked): a verdict names this commit and the trunk's head as its trunk commit
2026-10-13 09:01 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
2026-10-13 09:09 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
2026-10-13 09:09 published → delivered (done): nothing for a walk to move

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

**Delivery** `novox/mesh-host@89fd0c06e102` — **delivered** since 2026-10-13T09:09:28Z **Delivery plan** — builds nothing: the change touches no module of the mesh's graph **Transitions** - 2026-10-13 09:00 (new) → proposed (announced): novox/mesh-host#105's head announced - 2026-10-13 09:01 proposed → checked (checked): a verdict names this commit and the trunk's head as its trunk commit - 2026-10-13 09:01 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-13 09:09 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-13 09:09 published → delivered (done): nothing for a walk to move The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.

mesh-admin reviewed

2026-10-13 09:08:36 +00:00

mesh-admin left a comment

Verdict: approve at 89fd0c06e1

Issue 615's report and the PR body agree on the cause: the packages step's push of the genesis document returned once mesh-controller sent the declaration, and the sdk step's build ask raced the node's apply of the file the build agent reads it from. The fix changes exactly that one push.

The wait is bounded and the failure is loud. In internal/bootstrap/module.go, pushNodeApplied calls pushing(..., applied=true), which appends "--wait", o.Wait.String() to the controller command and gives the controller's own tell() a context timeout of 2*o.Wait ("the controller's own wait, and time to compose and send before it starts"). In mesh-controller's waitForApplied (cmd/mesh-controller/push.go), the wait is a bounded loop: deadline := time.Now().Add(wait), polling the node's reported digest every 500ms, and it fails loudly by name on every bad outcome — "%s did not report applying what it was sent within %s (it may still be converging; check status)" on a timeout, "%s applied what it was sent but %d resource(s) failed" on a failed resource, and "%s refused what it was sent: %s" on a refusal, each naming the node. mesh-host's own tell() wraps any such failure as "<container> push anchor --wait <duration> was refused: %w" plus the controller's own output, and the installer's say(" sealing the build agent the genesis document for the forge it raised") names the step immediately before the call, so a failure here is clearly attributable to this step in the log even though the installer does not number steps literally ("14", "15") in its own output. One nitpick, not blocking: pushing()'s generic wrap ("What was registered and assigned is registered and assigned, and this node has not been sent it... a push the mesh refused sent nothing at all") is reused unchanged for the --wait path, and is not quite accurate there — a --wait timeout or a reported failure/refusal means the declaration WAS sent and the node did or did not apply it, not that "nothing was sent at all"; the real reason is still visible in the indented output beneath it, so this is cosmetic.
push <node> --wait exists, unchanged, in both commits named. git show cd6757f:cmd/mesh-controller/push.go has wait := set.Duration("wait", 0, "for a named node, how long to wait for it to report applying what it was sent (0: do not wait)"), used only for a single named node via waitForApplied(ctx, inv, args[0], sentDigest[args[0]], *wait). git diff cd6757f main -- cmd/mesh-controller/push.go produced no output: push.go is byte-identical between cd6757f and main (1826b68). It waits for the node's own inventory report (inv.DoingOf) to show Declared equal to the digest just sent, with outcome Applied, Failed, or Refused — i.e. for the node to report having applied exactly that declaration, as ADR 0010 and the PR describe.
I audited every pushNode/pushNodeApplied call site in internal/bootstrap (module.go's generic installModule, control.go's InstallControlPlane, builder.go's InstallBuilder, phase2.go's installModule-per-module and PlaceOnTheNetwork, phase3.go's installProvider shared by InstallStore/InstallBroker/InstallVault, and phase_packages.go's deliverGenesisRegistry). Only InstallControlPlane's push (control.go) has a comparable shape — its very next lines, waitForContainer and waitForTheControlPlane, depend on the node having applied the push — but it already gets that guarantee a different way, by polling the container directly, so it does not also need --wait. A repo-wide grep for the pattern this fix defends against ("the node writes", "reads the document") turns up exactly one place outside this diff's own comments: phase_packages.go. No other genesis push has a following step that reads, straight off the node, a file that only that push's own apply writes, so no other push needs this fix.
go vet ./internal/bootstrap/... is clean at the head. go test -race ./internal/bootstrap/... passes at the head (ok github.com/novox/mesh-host/internal/bootstrap 3.935s). Checked out main (17038b2a07) with only the PR's new test file overlaid: TestThePackagesStepWaitsUntilTheNodeAppliedTheGenesisDocument fails there with "the genesis document's push does not wait for the node to apply it: docker exec mesh-controller /mesh-controller push anchor", and TestAnOrdinaryPushDoesNotWait passes on main too, as expected. At the head, ./merge-check.sh exits 0 with every package reporting ok and prints "NOT RUN AGAINST A BUS: the toolchain holds no nats-server; the tests that need one are skipped", matching what the PR body claims.

Gitea's own required checks on this head are both green: mesh/merge-gate is "success" — "head 89fd0c06e1 merged into trunk 17038b2a07 — pass: the change touches no module of the mesh's graph" — and mesh/repo-check is "success" — "head 89fd0c06e1 merged into trunk 17038b2a07 — pass: its merge-check.sh passed". mesh/delivery is also "success": "ready: it delivers once merged". No review was posted on this pull request before this one.

Verdict: approve at 89fd0c06e10221b2b044ea7c01b17ca6da80b4c7 Issue 615's report and the PR body agree on the cause: the packages step's push of the genesis document returned once mesh-controller sent the declaration, and the sdk step's build ask raced the node's apply of the file the build agent reads it from. The fix changes exactly that one push. 1. The wait is bounded and the failure is loud. In internal/bootstrap/module.go, pushNodeApplied calls pushing(..., applied=true), which appends "--wait", o.Wait.String() to the controller command and gives the controller's own `tell()` a context timeout of 2*o.Wait ("the controller's own wait, and time to compose and send before it starts"). In mesh-controller's waitForApplied (cmd/mesh-controller/push.go), the wait is a bounded loop: `deadline := time.Now().Add(wait)`, polling the node's reported digest every 500ms, and it fails loudly by name on every bad outcome — "%s did not report applying what it was sent within %s (it may still be converging; check `status`)" on a timeout, "%s applied what it was sent but %d resource(s) failed" on a failed resource, and "%s refused what it was sent: %s" on a refusal, each naming the node. mesh-host's own tell() wraps any such failure as "`<container> push anchor --wait <duration>` was refused: %w" plus the controller's own output, and the installer's `say(" sealing the build agent the genesis document for the forge it raised")` names the step immediately before the call, so a failure here is clearly attributable to this step in the log even though the installer does not number steps literally ("14", "15") in its own output. One nitpick, not blocking: pushing()'s generic wrap ("What was registered and assigned is registered and assigned, and this node has not been sent it... a push the mesh refused sent nothing at all") is reused unchanged for the --wait path, and is not quite accurate there — a --wait timeout or a reported failure/refusal means the declaration WAS sent and the node did or did not apply it, not that "nothing was sent at all"; the real reason is still visible in the indented output beneath it, so this is cosmetic. 2. `push <node> --wait` exists, unchanged, in both commits named. `git show cd6757f:cmd/mesh-controller/push.go` has `wait := set.Duration("wait", 0, "for a named node, how long to wait for it to report applying what it was sent (0: do not wait)")`, used only for a single named node via `waitForApplied(ctx, inv, args[0], sentDigest[args[0]], *wait)`. `git diff cd6757f main -- cmd/mesh-controller/push.go` produced no output: push.go is byte-identical between cd6757f and main (1826b68). It waits for the node's own inventory report (`inv.DoingOf`) to show `Declared` equal to the digest just sent, with outcome Applied, Failed, or Refused — i.e. for the node to report having applied exactly that declaration, as ADR 0010 and the PR describe. 3. I audited every pushNode/pushNodeApplied call site in internal/bootstrap (module.go's generic installModule, control.go's InstallControlPlane, builder.go's InstallBuilder, phase2.go's installModule-per-module and PlaceOnTheNetwork, phase3.go's installProvider shared by InstallStore/InstallBroker/InstallVault, and phase_packages.go's deliverGenesisRegistry). Only InstallControlPlane's push (control.go) has a comparable shape — its very next lines, waitForContainer and waitForTheControlPlane, depend on the node having applied the push — but it already gets that guarantee a different way, by polling the container directly, so it does not also need `--wait`. A repo-wide grep for the pattern this fix defends against ("the node writes", "reads the document") turns up exactly one place outside this diff's own comments: phase_packages.go. No other genesis push has a following step that reads, straight off the node, a file that only that push's own apply writes, so no other push needs this fix. 4. go vet ./internal/bootstrap/... is clean at the head. go test -race ./internal/bootstrap/... passes at the head (`ok github.com/novox/mesh-host/internal/bootstrap 3.935s`). Checked out main (17038b2a074b) with only the PR's new test file overlaid: `TestThePackagesStepWaitsUntilTheNodeAppliedTheGenesisDocument` fails there with "the genesis document's push does not wait for the node to apply it: docker exec mesh-controller /mesh-controller push anchor", and `TestAnOrdinaryPushDoesNotWait` passes on main too, as expected. At the head, ./merge-check.sh exits 0 with every package reporting `ok` and prints "NOT RUN AGAINST A BUS: the toolchain holds no nats-server; the tests that need one are skipped", matching what the PR body claims. Gitea's own required checks on this head are both green: `mesh/merge-gate` is "success" — "head 89fd0c06e102 merged into trunk 17038b2a074b — pass: the change touches no module of the mesh's graph" — and `mesh/repo-check` is "success" — "head 89fd0c06e102 merged into trunk 17038b2a074b — pass: its merge-check.sh passed". `mesh/delivery` is also "success": "ready: it delivers once merged". No review was posted on this pull request before this one.

mesh-admin merged commit f0d39b4a4a into main

2026-10-13 09:08:59 +00:00

mesh-admin deleted branch fix/615-wait-for-the-genesis-document

2026-10-13 09:09:00 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-host#105