Issue 386: a module whose only fault was a secret the operator had not given yet was stated unhealthy, failed its first-node gate and was put back.
internal/link: ReadToolAnswer and AskTool read the tool's answer through mesh-sdk go/health (healthy, unhealthy, waits); StateWaiting = "waiting"; ResourceHealth gains waits (part, secret or setting, what).
internal/liveness: a tool look that says waits states the resource waiting with its waits and the answer's why, on one look, inside the grace too, and counts as no failing look; the next healthy or unhealthy answer replaces it (unhealthy still after the declared failing looks). An answer without state is judged as before; a malformed wait is unhealthy (the SDK's Read).
The report and the change event carry the waits.
Tests: internal/liveness/waiting_test.go (waiting in grace with its waits; stays waiting after grace and is no failing look; healthy after a wait; unhealthy after a wait follows the failing looks; no-state answers as before; a malformed wait is unhealthy), internal/link TestAToolsWaitIsReadWithWhatItWaitsFor. merge-check.sh passes locally; the tests that need a real bus were skipped (no nats-server here).
Depends on novox/mesh-sdk#18 (pinned to its branch commit 33917f91 as a pseudo-version; bump to the merge commit once it merges). Decision: ADR 0283, novox/hq#294.
Rollout order: after the controller. An older controller reads waiting as not healthy and raises nothing, so its gate fails at the bound as today.
Issue 386: a module whose only fault was a secret the operator had not given yet was stated unhealthy, failed its first-node gate and was put back.
- internal/link: ReadToolAnswer and AskTool read the tool's answer through mesh-sdk go/health (healthy, unhealthy, waits); StateWaiting = "waiting"; ResourceHealth gains `waits` (part, secret or setting, what).
- internal/liveness: a tool look that says waits states the resource `waiting` with its waits and the answer's why, on one look, inside the grace too, and counts as no failing look; the next healthy or unhealthy answer replaces it (unhealthy still after the declared failing looks). An answer without state is judged as before; a malformed wait is unhealthy (the SDK's Read).
- The report and the change event carry the waits.
Tests: internal/liveness/waiting_test.go (waiting in grace with its waits; stays waiting after grace and is no failing look; healthy after a wait; unhealthy after a wait follows the failing looks; no-state answers as before; a malformed wait is unhealthy), internal/link TestAToolsWaitIsReadWithWhatItWaitsFor. merge-check.sh passes locally; the tests that need a real bus were skipped (no nats-server here).
Depends on novox/mesh-sdk#18 (pinned to its branch commit 33917f91 as a pseudo-version; bump to the merge commit once it merges). Decision: ADR 0283, novox/hq#294.
Rollout order: after the controller. An older controller reads `waiting` as not healthy and raises nothing, so its gate fails at the bound as today.
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
A module whose only fault was a secret the operator had not given yet was
stated unhealthy, failed its first-node gate and was put back. The engine
now reads the tool's answer through mesh-sdk go/health and states such a
resource waiting, with what it waits for, on one look; a malformed wait
stays unhealthy.
Deliverynovox/mesh-host@9545121ec93b — delivered since 2026-10-10T20:08:35Z
Delivery plan — builds mesh-host → ace, g14, novox, shanks; no bus step
tier 0: mesh-host
ace: receives mesh-host
g14: receives mesh-host
novox: receives mesh-host
shanks: receives mesh-host
Machines
mesh-host on ace: passed — healthy 3 times over 2m31s
mesh-host on the rest running it: sent
Transitions
2026-10-10 19:14 (new) → proposed (announced): novox/mesh-host#78's head announced
2026-10-10 19:17 proposed → checked (checked): the verdict names this commit
2026-10-10 19:17 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
2026-10-10 20:05 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
2026-10-10 20:05 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
2026-10-10 20:08 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says
The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.
<!-- mesh-delivery:view -->
**Delivery** `novox/mesh-host@9545121ec93b` — **delivered** since 2026-10-10T20:08:35Z
**Delivery plan** — builds mesh-host → ace, g14, novox, shanks; no bus step
- tier 0: mesh-host
- ace: receives mesh-host
- g14: receives mesh-host
- novox: receives mesh-host
- shanks: receives mesh-host
**Machines**
- mesh-host on ace: passed — healthy 3 times over 2m31s
- mesh-host on the rest running it: sent
**Transitions**
- 2026-10-10 19:14 (new) → proposed (announced): novox/mesh-host#78's head announced
- 2026-10-10 19:17 proposed → checked (checked): the verdict names this commit
- 2026-10-10 19:17 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
- 2026-10-10 20:05 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
- 2026-10-10 20:05 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
- 2026-10-10 20:08 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says
The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
Independent review against ADR 0283 decision 2 and its 'How it is checked' row 2.
What I checked:
Decision 2 is implemented exactly. A tool answer of waits with readable waits is kept as a wait in Record. That counts as no failing look, applies inside the grace too, and is stated waiting with its waits by ready(). The next answer clears k.Waits first, so it replaces the wait. An unhealthy answer after a wait still needs the declared failing looks after the grace. An answer without state goes through health.Read, which reads healthy as before. A malformed wait is read as unhealthy and excuses nothing.
Nothing else in health judging changes. Held, blind/unknown, restarting and down are all decided before ready() in judge(), so a wait that turns into a crash or a restart loop is still said unhealthy. Only tool checks return waits; http, tcp, runtime, exec and unit checks return nil waits. A change of check declaration and a fresh start both clear the waits. Waits is persisted in kept, so the waiting state survives an engine restart. The change event fires on waiting<->healthy, because the State/Reason word changes. Statement.Healthy() reads waiting as not healthy, so the health statement is said again on the unhealthy cadence.
Tests: sh merge-check.sh passes; every package is ok (the NATS-bus tests are skipped because the toolchain has no nats-server). The new tests cover row 2: waiting with its waits inside the grace, waiting after the grace with no failing streak, the next healthy answer is healthy, an unhealthy answer after a wait follows the failing looks, an answer without state is judged as before, and a malformed wait is unhealthy. There is also a link-level test of the reader and of the wire JSON.
Secrets: waits carry part, secret or setting name, and what in words. The SDK's Wait.Check rejects a path or an address. Nothing new reads a file.
Backward compatibility: the report and the event only gain waits,omitempty. In mesh-controller main, module_health.go raises a condition only on StateUnhealthy, and its gate reads any other state as not yet ('is waiting'), so an older controller raises nothing, as the ADR says.
The code is Go only. The added lines contain no retired word.
Findings (none blocking):
internal/liveness/readiness.go:216: a wait does not reset k.Passed. A resource that passed, then waited, then answers unhealthy once after the grace is stated healthy for that one look (Failing 1 < Looks 2, Passed true), not starting. A gate or condition reading that one statement sees a stale pass. This is the same tolerance as healthy->one failing look before this PR, but the wait has 'replaced' the earlier pass. Suggestion: clear Passed when a wait is recorded (k.Passed, k.Failing, ... = false, 0, ...), and assert != Healthy in TestAnUnhealthyAnswerAfterAWaitFollowsTheFailingLooks.
internal/liveness/liveness.go:371: the change word is State/Reason, and Reason holds the module's why but not its waits. If the waits change (for example, a second source starts waiting) while why stays the same, no change is flagged. The periodic re-say still carries the new waits. Consider adding the waits to the word.
Some edge cases have no test of their own, though the code handles them: a crash or restart loop beating a wait, a check-declaration change clearing waits, waits kept across an engine restart (the judge reloaded from its file), and changed set on waiting<->healthy. Worth adding with finding 1's fix or in a follow-up.
APPROVE at 3abb07f1e25b8da73675b341aa2e57986ce445dd
Independent review against ADR 0283 decision 2 and its 'How it is checked' row 2.
What I checked:
- Decision 2 is implemented exactly. A tool answer of `waits` with readable waits is kept as a wait in `Record`. That counts as no failing look, applies inside the grace too, and is stated `waiting` with its waits by `ready()`. The next answer clears `k.Waits` first, so it replaces the wait. An unhealthy answer after a wait still needs the declared failing looks after the grace. An answer without `state` goes through `health.Read`, which reads `healthy` as before. A malformed wait is read as unhealthy and excuses nothing.
- Nothing else in health judging changes. Held, blind/unknown, restarting and down are all decided before `ready()` in `judge()`, so a wait that turns into a crash or a restart loop is still said unhealthy. Only tool checks return waits; http, tcp, runtime, exec and unit checks return nil waits. A change of check declaration and a fresh start both clear the waits. `Waits` is persisted in `kept`, so the waiting state survives an engine restart. The change event fires on waiting<->healthy, because the State/Reason word changes. `Statement.Healthy()` reads waiting as not healthy, so the health statement is said again on the unhealthy cadence.
- Tests: `sh merge-check.sh` passes; every package is ok (the NATS-bus tests are skipped because the toolchain has no nats-server). The new tests cover row 2: waiting with its waits inside the grace, waiting after the grace with no failing streak, the next healthy answer is healthy, an unhealthy answer after a wait follows the failing looks, an answer without state is judged as before, and a malformed wait is unhealthy. There is also a link-level test of the reader and of the wire JSON.
- Secrets: waits carry part, secret or setting name, and `what` in words. The SDK's `Wait.Check` rejects a path or an address. Nothing new reads a file.
- Backward compatibility: the report and the event only gain `waits,omitempty`. In mesh-controller main, `module_health.go` raises a condition only on `StateUnhealthy`, and its gate reads any other state as not yet ('is waiting'), so an older controller raises nothing, as the ADR says.
- The code is Go only. The added lines contain no retired word.
Findings (none blocking):
1. internal/liveness/readiness.go:216: a wait does not reset `k.Passed`. A resource that passed, then waited, then answers unhealthy once after the grace is stated `healthy` for that one look (Failing 1 < Looks 2, Passed true), not `starting`. A gate or condition reading that one statement sees a stale pass. This is the same tolerance as healthy->one failing look before this PR, but the wait has 'replaced' the earlier pass. Suggestion: clear `Passed` when a wait is recorded (`k.Passed, k.Failing, ... = false, 0, ...`), and assert `!= Healthy` in TestAnUnhealthyAnswerAfterAWaitFollowsTheFailingLooks.
2. internal/liveness/liveness.go:371: the change word is State/Reason, and Reason holds the module's `why` but not its waits. If the waits change (for example, a second source starts waiting) while `why` stays the same, no change is flagged. The periodic re-say still carries the new waits. Consider adding the waits to the word.
3. Some edge cases have no test of their own, though the code handles them: a crash or restart loop beating a wait, a check-declaration change clearing waits, waits kept across an engine restart (the judge reloaded from its file), and `changed` set on waiting<->healthy. Worth adding with finding 1's fix or in a follow-up.
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
A resource that passed, then waited, then failed once after its grace was
still stated healthy from the old pass; and a change of what it waits for
alone raised no change event, so the controller could miss it.
This is a re-review of the changes since 3abb07f1. They address all three findings from review 5.
Fixed. Record now clears Passed when it records a wait (internal/liveness/readiness.go:217). A pass from before a wait no longer carries through it, so one failing look after the grace is said starting, not healthy. TestAnUnhealthyAnswerAfterAWaitFollowsTheFailingLooks now asserts that the state is not healthy, and TestAPassBeforeAWaitIsNotKeptThroughIt covers the order pass, then wait, then fail.
Fixed. The change word now includes the resource's waits (waitsWord, internal/liveness/liveness.go:372 and :584). TestAChangeOfWaitingOrOfItsWaitsIsAChange checks four cases: healthy to waiting is a change, the same wait again is not, an added wait is a change, and waiting to healthy is a change.
Fixed. Three new tests cover the remaining edge cases. TestDownBeatsAWait: a stopped process is said unhealthy/down with no waits. TestAChangedCheckClearsTheWaits: a changed check declaration clears the waits. TestWaitsAreKeptAcrossTheEnginesRestart: a judge reloaded from its file is still waiting with the same wait.
sh merge-check.sh passes in every package; the NATS-bus tests are skipped because the toolchain has no nats-server. Nothing else changed since the last review, and none of the earlier checks changes its outcome. I have no new findings.
APPROVE at 9545121ec93b57774eeeb8f4475a6335b16f434d
This is a re-review of the changes since 3abb07f1. They address all three findings from review 5.
1. Fixed. `Record` now clears `Passed` when it records a wait (internal/liveness/readiness.go:217). A pass from before a wait no longer carries through it, so one failing look after the grace is said `starting`, not `healthy`. TestAnUnhealthyAnswerAfterAWaitFollowsTheFailingLooks now asserts that the state is not healthy, and TestAPassBeforeAWaitIsNotKeptThroughIt covers the order pass, then wait, then fail.
2. Fixed. The change word now includes the resource's waits (`waitsWord`, internal/liveness/liveness.go:372 and :584). TestAChangeOfWaitingOrOfItsWaitsIsAChange checks four cases: healthy to waiting is a change, the same wait again is not, an added wait is a change, and waiting to healthy is a change.
3. Fixed. Three new tests cover the remaining edge cases. TestDownBeatsAWait: a stopped process is said unhealthy/down with no waits. TestAChangedCheckClearsTheWaits: a changed check declaration clears the waits. TestWaitsAreKeptAcrossTheEnginesRestart: a judge reloaded from its file is still waiting with the same wait.
`sh merge-check.sh` passes in every package; the NATS-bus tests are skipped because the toolchain has no nats-server. Nothing else changed since the last review, and none of the earlier checks changes its outcome. I have no new findings.
mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-host)
mesh/repo-check: PASS — its merge-check.sh passed
Delivery plan — builds mesh-host → ace, g14, novox, shanks; no bus step
tier 0: mesh-host
ace: receives mesh-host
g14: receives mesh-host
novox: receives mesh-host
shanks: receives mesh-host
read by no module's build: internal/link/asktool_nats_test.go, internal/link/health_test.go, internal/link/replay331_test.go, internal/liveness/readiness_test.go, internal/liveness/replay_test.go, internal/liveness/waiting_test.go
Run by the build seat on novox as build-1791659490375954891 (builds --log build-1791659490375954891).
--- the gate: mesh-host
mesh-host: ok
long-running resources without health: 0
files asking for a setting without saying whether it is trusted: 0
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 0 module(s) changed against 4 machine(s), as the snapshot of 2026-10-10T19:11:59Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-10T19:11:59Z
machines:
- a machine (f35): composes
- holds mesh-dns-resolver (idt): composes
- holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
- a machine (yvbpsm): composes
a merge would rebuild 1 module(s) in 1 tier(s): mesh-host
notes:
- internal/link/asktool_nats_test.go, internal/link/health_test.go, internal/link/replay331_test.go, internal/liveness/readiness_test.go, internal/liveness/replay_test.go and internal/liveness/waiting_test.go read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
go: downloading git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c
ok github.com/novox/mesh-host/cmd/mesh-bootstrap 1.018s
ok github.com/novox/mesh-host/cmd/mesh-host 12.873s
ok github.com/novox/mesh-host/examples 1.020s
ok github.com/novox/mesh-host/internal/accounts 11.854s
ok github.com/novox/mesh-host/internal/apply 33.470s
ok github.com/novox/mesh-host/internal/bootstrap 8.776s
ok github.com/novox/mesh-host/internal/bundle 1.017s
ok github.com/novox/mesh-host/internal/declaration 1.062s
ok github.com/novox/mesh-host/internal/firewall 1.856s
ok github.com/novox/mesh-host/internal/identity 3.592s
ok github.com/novox/mesh-host/internal/image 1.025s
ok github.com/novox/mesh-host/internal/inventory 1.022s
ok github.com/novox/mesh-host/internal/link 11.053s
ok github.com/novox/mesh-host/internal/liveness 6.098s
ok github.com/novox/mesh-host/internal/meshcli 1.330s
ok github.com/novox/mesh-host/internal/network 1.031s
ok github.com/novox/mesh-host/internal/outward 1.018s
ok github.com/novox/mesh-host/internal/profile 1.497s
ok github.com/novox/mesh-host/internal/reachable 1.223s
ok github.com/novox/mesh-host/internal/store 5.920s
ok github.com/novox/mesh-host/internal/system 1.017s
ok github.com/novox/mesh-host/internal/tunnel 1.019s
ok github.com/novox/mesh-host/internal/units 1.014s
ok github.com/novox/mesh-host/internal/upgrade 4.243s
ok github.com/novox/mesh-host/internal/witness 40.620s
ok github.com/novox/mesh-host/packaging 1.010s
? github.com/novox/mesh-host/rootsearch [no test files]
ok github.com/novox/mesh-host/validate 1.010s
**Merge check** at `3abb07f1`
- `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-host)
- `mesh/repo-check`: **PASS** — its merge-check.sh passed
**Delivery plan** — builds mesh-host → ace, g14, novox, shanks; no bus step
- tier 0: mesh-host
- ace: receives mesh-host
- g14: receives mesh-host
- novox: receives mesh-host
- shanks: receives mesh-host
- read by no module's build: internal/link/asktool_nats_test.go, internal/link/health_test.go, internal/link/replay331_test.go, internal/liveness/readiness_test.go, internal/liveness/replay_test.go, internal/liveness/waiting_test.go
Run by the build seat on novox as `build-1791659490375954891` (`builds --log build-1791659490375954891`).
```
--- the gate: mesh-host
mesh-host: ok
long-running resources without health: 0
files asking for a setting without saying whether it is trusted: 0
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 0 module(s) changed against 4 machine(s), as the snapshot of 2026-10-10T19:11:59Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-10T19:11:59Z
machines:
- a machine (f35): composes
- holds mesh-dns-resolver (idt): composes
- holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
- a machine (yvbpsm): composes
a merge would rebuild 1 module(s) in 1 tier(s): mesh-host
notes:
- internal/link/asktool_nats_test.go, internal/link/health_test.go, internal/link/replay331_test.go, internal/liveness/readiness_test.go, internal/liveness/replay_test.go and internal/liveness/waiting_test.go read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
go: downloading git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c
ok github.com/novox/mesh-host/cmd/mesh-bootstrap 1.018s
ok github.com/novox/mesh-host/cmd/mesh-host 12.873s
ok github.com/novox/mesh-host/examples 1.020s
ok github.com/novox/mesh-host/internal/accounts 11.854s
ok github.com/novox/mesh-host/internal/apply 33.470s
ok github.com/novox/mesh-host/internal/bootstrap 8.776s
ok github.com/novox/mesh-host/internal/bundle 1.017s
ok github.com/novox/mesh-host/internal/declaration 1.062s
ok github.com/novox/mesh-host/internal/firewall 1.856s
ok github.com/novox/mesh-host/internal/identity 3.592s
ok github.com/novox/mesh-host/internal/image 1.025s
ok github.com/novox/mesh-host/internal/inventory 1.022s
ok github.com/novox/mesh-host/internal/link 11.053s
ok github.com/novox/mesh-host/internal/liveness 6.098s
ok github.com/novox/mesh-host/internal/meshcli 1.330s
ok github.com/novox/mesh-host/internal/network 1.031s
ok github.com/novox/mesh-host/internal/outward 1.018s
ok github.com/novox/mesh-host/internal/profile 1.497s
ok github.com/novox/mesh-host/internal/reachable 1.223s
ok github.com/novox/mesh-host/internal/store 5.920s
ok github.com/novox/mesh-host/internal/system 1.017s
ok github.com/novox/mesh-host/internal/tunnel 1.019s
ok github.com/novox/mesh-host/internal/units 1.014s
ok github.com/novox/mesh-host/internal/upgrade 4.243s
ok github.com/novox/mesh-host/internal/witness 40.620s
ok github.com/novox/mesh-host/packaging 1.010s
? github.com/novox/mesh-host/rootsearch [no test files]
ok github.com/novox/mesh-host/validate 1.010s
```
mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-host)
mesh/repo-check: PASS — its merge-check.sh passed
Delivery plan — builds mesh-host → ace, g14, novox, shanks; no bus step
tier 0: mesh-host
ace: receives mesh-host
g14: receives mesh-host
novox: receives mesh-host
shanks: receives mesh-host
read by no module's build: internal/link/asktool_nats_test.go, internal/link/health_test.go, internal/link/replay331_test.go, internal/liveness/readiness_test.go, internal/liveness/replay_test.go, internal/liveness/waiting_test.go
Run by the build seat on shanks as build-1791659675974941673 (builds --log build-1791659675974941673).
--- the gate: mesh-host
mesh-host: ok
long-running resources without health: 0
files asking for a setting without saying whether it is trusted: 0
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 0 module(s) changed against 4 machine(s), as the snapshot of 2026-10-10T19:11:59Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-10T19:11:59Z
machines:
- a machine (f35): composes
- holds mesh-dns-resolver (idt): composes
- holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
- a machine (yvbpsm): composes
a merge would rebuild 1 module(s) in 1 tier(s): mesh-host
notes:
- internal/link/asktool_nats_test.go, internal/link/health_test.go, internal/link/replay331_test.go, internal/liveness/readiness_test.go, internal/liveness/replay_test.go and internal/liveness/waiting_test.go read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
go: downloading git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c
ok github.com/novox/mesh-host/cmd/mesh-bootstrap 1.015s
ok github.com/novox/mesh-host/cmd/mesh-host 5.508s
ok github.com/novox/mesh-host/examples 1.018s
ok github.com/novox/mesh-host/internal/accounts 1.462s
ok github.com/novox/mesh-host/internal/apply 4.722s
ok github.com/novox/mesh-host/internal/bootstrap 1.628s
ok github.com/novox/mesh-host/internal/bundle 1.057s
ok github.com/novox/mesh-host/internal/declaration 1.098s
ok github.com/novox/mesh-host/internal/firewall 1.854s
ok github.com/novox/mesh-host/internal/identity 1.144s
ok github.com/novox/mesh-host/internal/image 1.053s
ok github.com/novox/mesh-host/internal/inventory 1.065s
ok github.com/novox/mesh-host/internal/link 10.501s
ok github.com/novox/mesh-host/internal/liveness 2.141s
ok github.com/novox/mesh-host/internal/meshcli 1.376s
ok github.com/novox/mesh-host/internal/network 1.076s
ok github.com/novox/mesh-host/internal/outward 1.067s
ok github.com/novox/mesh-host/internal/profile 1.551s
ok github.com/novox/mesh-host/internal/reachable 1.055s
ok github.com/novox/mesh-host/internal/store 1.239s
ok github.com/novox/mesh-host/internal/system 1.053s
ok github.com/novox/mesh-host/internal/tunnel 1.058s
ok github.com/novox/mesh-host/internal/units 1.040s
ok github.com/novox/mesh-host/internal/upgrade 1.173s
ok github.com/novox/mesh-host/internal/witness 4.796s
ok github.com/novox/mesh-host/packaging 1.045s
? github.com/novox/mesh-host/rootsearch [no test files]
ok github.com/novox/mesh-host/validate 1.038s
**Merge check** at `9545121e`
- `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-host)
- `mesh/repo-check`: **PASS** — its merge-check.sh passed
**Delivery plan** — builds mesh-host → ace, g14, novox, shanks; no bus step
- tier 0: mesh-host
- ace: receives mesh-host
- g14: receives mesh-host
- novox: receives mesh-host
- shanks: receives mesh-host
- read by no module's build: internal/link/asktool_nats_test.go, internal/link/health_test.go, internal/link/replay331_test.go, internal/liveness/readiness_test.go, internal/liveness/replay_test.go, internal/liveness/waiting_test.go
Run by the build seat on shanks as `build-1791659675974941673` (`builds --log build-1791659675974941673`).
```
--- the gate: mesh-host
mesh-host: ok
long-running resources without health: 0
files asking for a setting without saying whether it is trusted: 0
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 0 module(s) changed against 4 machine(s), as the snapshot of 2026-10-10T19:11:59Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-10T19:11:59Z
machines:
- a machine (f35): composes
- holds mesh-dns-resolver (idt): composes
- holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
- a machine (yvbpsm): composes
a merge would rebuild 1 module(s) in 1 tier(s): mesh-host
notes:
- internal/link/asktool_nats_test.go, internal/link/health_test.go, internal/link/replay331_test.go, internal/liveness/readiness_test.go, internal/liveness/replay_test.go and internal/liveness/waiting_test.go read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
go: downloading git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c
ok github.com/novox/mesh-host/cmd/mesh-bootstrap 1.015s
ok github.com/novox/mesh-host/cmd/mesh-host 5.508s
ok github.com/novox/mesh-host/examples 1.018s
ok github.com/novox/mesh-host/internal/accounts 1.462s
ok github.com/novox/mesh-host/internal/apply 4.722s
ok github.com/novox/mesh-host/internal/bootstrap 1.628s
ok github.com/novox/mesh-host/internal/bundle 1.057s
ok github.com/novox/mesh-host/internal/declaration 1.098s
ok github.com/novox/mesh-host/internal/firewall 1.854s
ok github.com/novox/mesh-host/internal/identity 1.144s
ok github.com/novox/mesh-host/internal/image 1.053s
ok github.com/novox/mesh-host/internal/inventory 1.065s
ok github.com/novox/mesh-host/internal/link 10.501s
ok github.com/novox/mesh-host/internal/liveness 2.141s
ok github.com/novox/mesh-host/internal/meshcli 1.376s
ok github.com/novox/mesh-host/internal/network 1.076s
ok github.com/novox/mesh-host/internal/outward 1.067s
ok github.com/novox/mesh-host/internal/profile 1.551s
ok github.com/novox/mesh-host/internal/reachable 1.055s
ok github.com/novox/mesh-host/internal/store 1.239s
ok github.com/novox/mesh-host/internal/system 1.053s
ok github.com/novox/mesh-host/internal/tunnel 1.058s
ok github.com/novox/mesh-host/internal/units 1.040s
ok github.com/novox/mesh-host/internal/upgrade 1.173s
ok github.com/novox/mesh-host/internal/witness 4.796s
ok github.com/novox/mesh-host/packaging 1.045s
? github.com/novox/mesh-host/rootsearch [no test files]
ok github.com/novox/mesh-host/validate 1.038s
```
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Issue 386: a module whose only fault was a secret the operator had not given yet was stated unhealthy, failed its first-node gate and was put back.
waits(part, secret or setting, what).waitingwith its waits and the answer's why, on one look, inside the grace too, and counts as no failing look; the next healthy or unhealthy answer replaces it (unhealthy still after the declared failing looks). An answer without state is judged as before; a malformed wait is unhealthy (the SDK's Read).Tests: internal/liveness/waiting_test.go (waiting in grace with its waits; stays waiting after grace and is no failing look; healthy after a wait; unhealthy after a wait follows the failing looks; no-state answers as before; a malformed wait is unhealthy), internal/link TestAToolsWaitIsReadWithWhatItWaitsFor. merge-check.sh passes locally; the tests that need a real bus were skipped (no nats-server here).
Depends on novox/mesh-sdk#18 (pinned to its branch commit 33917f91 as a pseudo-version; bump to the merge commit once it merges). Decision: ADR 0283, novox/hq#294.
Rollout order: after the controller. An older controller reads
waitingas not healthy and raises nothing, so its gate fails at the bound as today.Delivery
novox/mesh-host@9545121ec93b— delivered since 2026-10-10T20:08:35ZDelivery plan — builds mesh-host → ace, g14, novox, shanks; no bus step
Machines
Transitions
The commit's note under
refs/notes/mesh-plankeeps every transition:git log --notes=mesh-plan.APPROVE at
3abb07f1e2Independent review against ADR 0283 decision 2 and its 'How it is checked' row 2.
What I checked:
waitswith readable waits is kept as a wait inRecord. That counts as no failing look, applies inside the grace too, and is statedwaitingwith its waits byready(). The next answer clearsk.Waitsfirst, so it replaces the wait. An unhealthy answer after a wait still needs the declared failing looks after the grace. An answer withoutstategoes throughhealth.Read, which readshealthyas before. A malformed wait is read as unhealthy and excuses nothing.ready()injudge(), so a wait that turns into a crash or a restart loop is still said unhealthy. Only tool checks return waits; http, tcp, runtime, exec and unit checks return nil waits. A change of check declaration and a fresh start both clear the waits.Waitsis persisted inkept, so the waiting state survives an engine restart. The change event fires on waiting<->healthy, because the State/Reason word changes.Statement.Healthy()reads waiting as not healthy, so the health statement is said again on the unhealthy cadence.sh merge-check.shpasses; every package is ok (the NATS-bus tests are skipped because the toolchain has no nats-server). The new tests cover row 2: waiting with its waits inside the grace, waiting after the grace with no failing streak, the next healthy answer is healthy, an unhealthy answer after a wait follows the failing looks, an answer without state is judged as before, and a malformed wait is unhealthy. There is also a link-level test of the reader and of the wire JSON.whatin words. The SDK'sWait.Checkrejects a path or an address. Nothing new reads a file.waits,omitempty. In mesh-controller main,module_health.goraises a condition only onStateUnhealthy, and its gate reads any other state as not yet ('is waiting'), so an older controller raises nothing, as the ADR says.Findings (none blocking):
k.Passed. A resource that passed, then waited, then answers unhealthy once after the grace is statedhealthyfor that one look (Failing 1 < Looks 2, Passed true), notstarting. A gate or condition reading that one statement sees a stale pass. This is the same tolerance as healthy->one failing look before this PR, but the wait has 'replaced' the earlier pass. Suggestion: clearPassedwhen a wait is recorded (k.Passed, k.Failing, ... = false, 0, ...), and assert!= Healthyin TestAnUnhealthyAnswerAfterAWaitFollowsTheFailingLooks.whybut not its waits. If the waits change (for example, a second source starts waiting) whilewhystays the same, no change is flagged. The periodic re-say still carries the new waits. Consider adding the waits to the word.changedset on waiting<->healthy. Worth adding with finding 1's fix or in a follow-up.APPROVE at
9545121ec9This is a re-review of the changes since
3abb07f1. They address all three findings from review 5.Recordnow clearsPassedwhen it records a wait (internal/liveness/readiness.go:217). A pass from before a wait no longer carries through it, so one failing look after the grace is saidstarting, nothealthy. TestAnUnhealthyAnswerAfterAWaitFollowsTheFailingLooks now asserts that the state is not healthy, and TestAPassBeforeAWaitIsNotKeptThroughIt covers the order pass, then wait, then fail.waitsWord, internal/liveness/liveness.go:372 and :584). TestAChangeOfWaitingOrOfItsWaitsIsAChange checks four cases: healthy to waiting is a change, the same wait again is not, an added wait is a change, and waiting to healthy is a change.sh merge-check.shpasses in every package; the NATS-bus tests are skipped because the toolchain has no nats-server. Nothing else changed since the last review, and none of the earlier checks changes its outcome. I have no new findings.Merge check at
3abb07f1mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-host)mesh/repo-check: PASS — its merge-check.sh passedDelivery plan — builds mesh-host → ace, g14, novox, shanks; no bus step
Run by the build seat on novox as
build-1791659490375954891(builds --log build-1791659490375954891).Merge check at
9545121emesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-host)mesh/repo-check: PASS — its merge-check.sh passedDelivery plan — builds mesh-host → ace, g14, novox, shanks; no bus step
Run by the build seat on shanks as
build-1791659675974941673(builds --log build-1791659675974941673).