Ask the plans again when the moved path starts before its runtime answers (issue 943) #336

2026-10-16T16:53:04Z

mesh-tools commented

2026-10-16 16:53:04 +00:00

Issue 943 (critical). postgres's moved path sometimes starts before its runtime connects it to the bus; its first watch is refused ('before Serve: a bundle reaches the bus through the runtime that launched it') and Run returned, so the provider answered nobody and retired nothing until its next restart: novox at 16:52, 18:29 and 18:48, ace at 16:22 (tool runner journals). It keeps issue 937's retirement condition on novox from clearing.

Fix. movedRun.Run asks the watch again, the wait doubling from FirstRetry (a second) to a minute, until answered or ctx ends; said at try 4 and every 20th. Identical in keycloak, which has the same race.

Tests. early_test.go per module: a bus refusing the first three watches; fails with main's Run, passes now. postgres 0.13 s, keycloak 0.20 s; -race passes.

Review. Independent review, round 1 of at most 3: approved at tree f99843a9 = this head's tree. No issues opened.

Issue 943 (critical). postgres's moved path sometimes starts before its runtime connects it to the bus; its first watch is refused ('before Serve: a bundle reaches the bus through the runtime that launched it') and Run returned, so the provider answered nobody and retired nothing until its next restart: novox at 16:52, 18:29 and 18:48, ace at 16:22 (tool runner journals). It keeps issue 937's retirement condition on novox from clearing. **Fix.** movedRun.Run asks the watch again, the wait doubling from FirstRetry (a second) to a minute, until answered or ctx ends; said at try 4 and every 20th. Identical in keycloak, which has the same race. **Tests.** early_test.go per module: a bus refusing the first three watches; fails with main's Run, passes now. postgres 0.13 s, keycloak 0.20 s; -race passes. **Review.** Independent review, round 1 of at most 3: approved at tree f99843a9 = this head's tree. No issues opened.

mesh-tools added 1 commit 2026-10-16 16:53:04 +00:00

Ask the plans again when the moved path starts before its runtime answers (issue 943)

mesh/merge-gate head 6005c6001e2f merged into trunk 832a0afd1814 — pass: builds keycloak, postgres, sent nowhere; no bus step; 3 wait(s) for a person; ever…

Details

mesh/repo-check head 6005c6001e2f merged into trunk 832a0afd1814 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…

Details

mesh/delivery delivered

Details

6005c6001e

postgres and keycloak start their moved path beside Serve, and a watch
asked before Serve runs is refused by the runtime. Run returned on that
refusal and main only logged it, so a start that lost the race left the
provider answering nobody and retiring nothing until its next restart:
postgres on novox at 16:52, 18:29 and 18:48, on ace at 16:22, which kept
issue 937's retirement condition on novox from clearing. Run now asks
again, the wait doubling from a second to a minute, until the runtime
answers or its context ends.

mesh-tools added the mesh/ready label 2026-10-16 16:53:09 +00:00

mesh-tools commented

2026-10-16 16:53:09 +00:00

ready at 6005c6001e

Reviewed by review-937@novox: approved round 1 of at most 3 at tree f99843a9, which this head carries: the moved path retries its first watch until the runtime answers; no change asked. Marked through mesh-delivery.mark-ready (novox/hq issue 913).

ready at 6005c6001e2f38cca9dad3e6b86fd09c5d3f2a9a Reviewed by review-937@novox: approved round 1 of at most 3 at tree f99843a9, which this head carries: the moved path retries its first watch until the runtime answers; no change asked. Marked through mesh-delivery.mark-ready (novox/hq issue 913).

mesh-tools commented

2026-10-16 16:53:11 +00:00

Delivery novox/mesh-catalog@6005c6001e2f — delivered since 2026-10-16T16:58:56Z

Delivery plan — builds keycloak, postgres, sent nowhere; no bus step; 3 wait(s) for a person

tier 0: keycloak, postgres
ace: waits for a person: postgres
novox: waits for a person: keycloak, postgres
keycloak waits for a person: its policy records (module) — unless on a machine its move recreates none of its containers and moves none of its data, which the walk judges and then sends
keycloak provides oidc-client: its consumers are sent again after it
keycloak keeps data (ADR 0233): what it holds is backed up before it moves
postgres waits for a person: its policy records (module) — unless on a machine its move recreates none of its containers and moves none of its data, which the walk judges and then sends
postgres provides postgres-database: its consumers are sent again after it
postgres keeps data (ADR 0233): what it holds is backed up before it moves

Machines

keycloak on every machine running it: sent
postgres on every machine running it: sent

Transitions

2026-10-16 16:53 (new) → proposed (announced): novox/mesh-catalog#336's head announced
2026-10-16 16:55 proposed → checked (checked): a verdict names this commit and the trunk's head as its trunk commit
2026-10-16 16:55 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
2026-10-16 16:55 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
2026-10-16 16:55 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
2026-10-16 16:58 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

**Delivery** `novox/mesh-catalog@6005c6001e2f` — **delivered** since 2026-10-16T16:58:56Z **Delivery plan** — builds keycloak, postgres, sent nowhere; no bus step; 3 wait(s) for a person - tier 0: keycloak, postgres - ace: waits for a person: postgres - novox: waits for a person: keycloak, postgres - keycloak waits for a person: its policy records (module) — unless on a machine its move recreates none of its containers and moves none of its data, which the walk judges and then sends - keycloak provides oidc-client: its consumers are sent again after it - keycloak keeps data (ADR 0233): what it holds is backed up before it moves - postgres waits for a person: its policy records (module) — unless on a machine its move recreates none of its containers and moves none of its data, which the walk judges and then sends - postgres provides postgres-database: its consumers are sent again after it - postgres keeps data (ADR 0233): what it holds is backed up before it moves **Machines** - keycloak on every machine running it: sent - postgres on every machine running it: sent **Transitions** - 2026-10-16 16:53 (new) → proposed (announced): novox/mesh-catalog#336's head announced - 2026-10-16 16:55 proposed → checked (checked): a verdict names this commit and the trunk's head as its trunk commit - 2026-10-16 16:55 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-16 16:55 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-16 16:55 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit - 2026-10-16 16:58 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.

mesh-tools commented

2026-10-16 16:55:40 +00:00

Merge check at 6005c600
Judged: head 6005c6001e2f merged into trunk 832a0afd1814; read beside it: mesh-controller c7cb9f0a2cd3, mesh-controller-main c7cb9f0a2cd3, mesh-host cd13f7726ef0, mesh-tools 7412c5b6c9c5

mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: keycloak, postgres)
mesh/repo-check: PASS — its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-check-typescript.sh (typescript)

Delivery plan — builds keycloak, postgres, sent nowhere; no bus step; 3 wait(s) for a person

tier 0: keycloak, postgres
ace: waits for a person: postgres
novox: waits for a person: keycloak, postgres
keycloak waits for a person: its policy records (module) — unless on a machine its move recreates none of its containers and moves none of its data, which the walk judges and then sends
keycloak provides oidc-client: its consumers are sent again after it
keycloak keeps data (ADR 0233): what it holds is backed up before it moves
postgres waits for a person: its policy records (module) — unless on a machine its move recreates none of its containers and moves none of its data, which the walk judges and then sends
postgres provides postgres-database: its consumers are sent again after it
postgres keeps data (ADR 0233): what it holds is backed up before it moves
read by no module's build: modules/keycloak/cmd/keycloak-provider/early_test.go, modules/postgres/cmd/postgres-provider/early_test.go

Run by the build seat on ace as build-1792169586771094221 (builds --log build-1792169586771094221).

--- the gate: keycloak, postgres
keycloak: ok, keeps state bound, reads mesh-controller.assignments; WARNING: runtime-config ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: server stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
postgres: ok, keeps state bound, reads mesh-controller.assignments, keeps store (valuable), dumps (rebuildable), ready: server by tcp on database every 30s
long-running resources without health: 1
files asking for a setting without saying whether it is trusted: 1
2 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 0 module(s) changed against 4 machine(s), as the snapshot of 2026-10-16T16:34:22Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-16T16:34:22Z

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-config, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-provisioner, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 2 module(s) in 1 tier(s): keycloak, postgres

notes:
  - modules/keycloak/cmd/keycloak-provider/early_test.go and modules/postgres/cmd/postgres-provider/early_test.go read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
ok  	git.novox.be/novox/mesh-catalog/checks/words	0.002s
words: 12 retired words and 70 daemons' lines, none in the 53630 strings of 801 files
ok  	git.novox.be/novox/mesh-catalog/checks/agenthome	0.010s
agent-home: no archive and no file written into under the agent's home
ok  	git.novox.be/novox/mesh-catalog/checks/mailstore	0.008s
mail-store: claimed only by mesh-mail
ok  	git.novox.be/novox/mesh-catalog/checks/secrets	0.012s
secrets: no setting can replace a vault secret in a mergeable file (1 known, nested)
ok  	git.novox.be/novox/mesh-catalog/checks/predefined	0.218s
predefined.json: 2 key(s), none removed that a module names
ok  	git.novox.be/novox/mesh-catalog/checks/config	0.014s
config: no moved module says settings, takes-settings or ${setting:<key>}, or takes a whole layer (0 not yet moved)
ok  	git.novox.be/novox/mesh-catalog/checks/templates	0.010s
templates: no module keeps a file's text inline but the 0 listed
ok  	git.novox.be/novox/mesh-catalog/checks/paths	0.005s
paths: no module names another's state on its machine (3 known, listed to go)
long-running resources without health: 61, as kept
ok  	git.novox.be/novox/mesh-catalog/checks/deps	0.197s
testing modules/keycloak
ok  	keycloak/cmd/keycloak-provider	1.382s
testing modules/postgres
ok  	postgres/cmd/postgres-provider	1.356s
--- its merge-check-typescript.sh (typescript toolchain)

**Merge check** at `6005c600` Judged: head `6005c6001e2f` merged into trunk `832a0afd1814`; read beside it: mesh-controller `c7cb9f0a2cd3`, mesh-controller-main `c7cb9f0a2cd3`, mesh-host `cd13f7726ef0`, mesh-tools `7412c5b6c9c5` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: keycloak, postgres) - `mesh/repo-check`: **PASS** — its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-check-typescript.sh (typescript) **Delivery plan** — builds keycloak, postgres, sent nowhere; no bus step; 3 wait(s) for a person - tier 0: keycloak, postgres - ace: waits for a person: postgres - novox: waits for a person: keycloak, postgres - keycloak waits for a person: its policy records (module) — unless on a machine its move recreates none of its containers and moves none of its data, which the walk judges and then sends - keycloak provides oidc-client: its consumers are sent again after it - keycloak keeps data (ADR 0233): what it holds is backed up before it moves - postgres waits for a person: its policy records (module) — unless on a machine its move recreates none of its containers and moves none of its data, which the walk judges and then sends - postgres provides postgres-database: its consumers are sent again after it - postgres keeps data (ADR 0233): what it holds is backed up before it moves - read by no module's build: modules/keycloak/cmd/keycloak-provider/early_test.go, modules/postgres/cmd/postgres-provider/early_test.go Run by the build seat on ace as `build-1792169586771094221` (`builds --log build-1792169586771094221`). ``` --- the gate: keycloak, postgres keycloak: ok, keeps state bound, reads mesh-controller.assignments; WARNING: runtime-config ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: server stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8) postgres: ok, keeps state bound, reads mesh-controller.assignments, keeps store (valuable), dumps (rebuildable), ready: server by tcp on database every 30s long-running resources without health: 1 files asking for a setting without saying whether it is trusted: 1 2 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 0 module(s) changed against 4 machine(s), as the snapshot of 2026-10-16T16:34:22Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-16T16:34:22Z machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-config, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-provisioner, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 2 module(s) in 1 tier(s): keycloak, postgres notes: - modules/keycloak/cmd/keycloak-provider/early_test.go and modules/postgres/cmd/postgres-provider/early_test.go read by no module's build, and rebuild nothing (issue 280) --- its merge-check.sh (go toolchain) ok git.novox.be/novox/mesh-catalog/checks/words 0.002s words: 12 retired words and 70 daemons' lines, none in the 53630 strings of 801 files ok git.novox.be/novox/mesh-catalog/checks/agenthome 0.010s agent-home: no archive and no file written into under the agent's home ok git.novox.be/novox/mesh-catalog/checks/mailstore 0.008s mail-store: claimed only by mesh-mail ok git.novox.be/novox/mesh-catalog/checks/secrets 0.012s secrets: no setting can replace a vault secret in a mergeable file (1 known, nested) ok git.novox.be/novox/mesh-catalog/checks/predefined 0.218s predefined.json: 2 key(s), none removed that a module names ok git.novox.be/novox/mesh-catalog/checks/config 0.014s config: no moved module says settings, takes-settings or ${setting:<key>}, or takes a whole layer (0 not yet moved) ok git.novox.be/novox/mesh-catalog/checks/templates 0.010s templates: no module keeps a file's text inline but the 0 listed ok git.novox.be/novox/mesh-catalog/checks/paths 0.005s paths: no module names another's state on its machine (3 known, listed to go) long-running resources without health: 61, as kept ok git.novox.be/novox/mesh-catalog/checks/deps 0.197s testing modules/keycloak ok keycloak/cmd/keycloak-provider 1.382s testing modules/postgres ok postgres/cmd/postgres-provider 1.356s --- its merge-check-typescript.sh (typescript toolchain) ```

mesh-tools merged commit 3817099b2c into main

2026-10-16 16:55:41 +00:00

mesh-tools deleted branch fix/943-moved-path-waits-for-runtime

2026-10-16 16:55:41 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-catalog#336