Raise a machine that refuses or only partly applies its declarations past one apply interval (issue 233) #261

Closed
mesh-admin wants to merge 2 commits from fix/233-a-failed-probe-fails-only-what-needs-the-tool into main
Contributor

What

A new row S21 in the signals table adds the condition apply-failing on the machine. It is raised once a machine's reports have said the same failure (refused whole, or applied only in part) for longer than one apply interval (5 minutes, the node-engine's ReconcileEvery), over at least two reports. It is not raised while the machine is silent or asleep, because S1 covers that, and a clean apply clears it. It reads the failing-since and count the store already keeps (issue 065, NotDoingWhatTheyWereTold), so there is no schema change.

When the node-engine names a package manager that does not answer (key the package manager (pacman), mesh-host PR on the same branch name), the condition names the tool in the summary and the headline (" cannot install packages"). The probe's words go in the evidence (Said), so the configuration file's path stays out of the summary. Otherwise it gives the refusal or the failed resource ids, with the machine's words in the evidence. Plain words are registered as a fallback.

Why

Issue 233, question 3, decided 2026-10-11: a machine whose declarations are refused, or applied only in part, for longer than one apply interval raises a condition by itself (issue 187's family). On 2026-10-04 a workstation refused every declaration for 90 minutes, and the only sign was a line in status.

How it was tested

  • apply_failing_test.go: a machine whose package manager has not answered for 90 minutes is raised. The test checks the key, that the summary names pacman and the count and carries no path, that the evidence carries the probe's words, and the plain words. A whole refusal is also raised. A single report (a laptop back after a week) and a silent machine are not raised.
  • The generated signals-table test walks S21: inside the bound (5 minutes, 2 reports) nothing is raised; past it (5 minutes and 1 second) the condition is raised and then cleared through the keeper. The plain-words test holds its wording.
  • sh merge-check.sh: green (the store tests were skipped locally because no MESH_TEST_POSTGRES is set; the build seat runs them).

The first commit carried a controller binary by mistake. The second commit removes it, so the PR's diff is clean. Squash-merging leaves it out of history.

Fixes issue 233

## What A new row S21 in the signals table adds the condition `apply-failing` on the machine. It is raised once a machine's reports have said the same failure (refused whole, or applied only in part) for longer than one apply interval (5 minutes, the node-engine's ReconcileEvery), over at least two reports. It is not raised while the machine is silent or asleep, because S1 covers that, and a clean apply clears it. It reads the failing-since and count the store already keeps (issue 065, `NotDoingWhatTheyWereTold`), so there is no schema change. When the node-engine names a package manager that does not answer (key `the package manager (pacman)`, mesh-host PR on the same branch name), the condition names the tool in the summary and the headline ("<machine> cannot install packages"). The probe's words go in the evidence (`Said`), so the configuration file's path stays out of the summary. Otherwise it gives the refusal or the failed resource ids, with the machine's words in the evidence. Plain words are registered as a fallback. ## Why Issue 233, question 3, decided 2026-10-11: a machine whose declarations are refused, or applied only in part, for longer than one apply interval raises a condition by itself (issue 187's family). On 2026-10-04 a workstation refused every declaration for 90 minutes, and the only sign was a line in `status`. ## How it was tested - `apply_failing_test.go`: a machine whose package manager has not answered for 90 minutes is raised. The test checks the key, that the summary names pacman and the count and carries no path, that the evidence carries the probe's words, and the plain words. A whole refusal is also raised. A single report (a laptop back after a week) and a silent machine are not raised. - The generated signals-table test walks S21: inside the bound (5 minutes, 2 reports) nothing is raised; past it (5 minutes and 1 second) the condition is raised and then cleared through the keeper. The plain-words test holds its wording. - `sh merge-check.sh`: green (the store tests were skipped locally because no MESH_TEST_POSTGRES is set; the build seat runs them). The first commit carried a controller binary by mistake. The second commit removes it, so the PR's diff is clean. Squash-merging leaves it out of history. Fixes issue 233
mesh-admin added 2 commits 2026-10-12 12:33:17 +00:00
Take out the controller binary a local build left in the change (issue 233)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery-group group fix/233-a-failed-probe-fails-only-what-needs-the-tool ready: every member ready, and composed together they pass
mesh/delivery stopped: the pull request closed unmerged
2043b0273e
Author
Contributor

Delivery novox/mesh-controller@2043b0273edf — stopped since 2026-10-12T12:50:43Z

Delivery plan — builds mesh-controller → novox; no bus step

  • tier 0: mesh-controller
  • novox: receives mesh-controller

Group fix/233-a-failed-probe-fails-only-what-needs-the-tool, in order: novox/mesh-host@cb0a07ba7a → novox/mesh-controller@2043b0273e

Transitions

  • 2026-10-12 12:33 (new) → proposed (announced): novox/mesh-controller#261's head announced
  • 2026-10-12 12:42 proposed → checked (checked): the verdict names this commit
  • 2026-10-12 12:42 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
  • 2026-10-12 12:50 ready → stopped (closed): the pull request closed unmerged

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

<!-- mesh-delivery:view --> **Delivery** `novox/mesh-controller@2043b0273edf` — **stopped** since 2026-10-12T12:50:43Z **Delivery plan** — builds mesh-controller → novox; no bus step - tier 0: mesh-controller - novox: receives mesh-controller **Group** `fix/233-a-failed-probe-fails-only-what-needs-the-tool`, in order: novox/mesh-host@cb0a07ba7ac9 → novox/mesh-controller@2043b0273edf - novox/mesh-host@cb0a07ba7ac9 before novox/mesh-controller@2043b0273edf: engine before controller - composed together: pass — every machine composes with the change as it did without (4 of 4 compose) **Transitions** - 2026-10-12 12:33 (new) → proposed (announced): novox/mesh-controller#261's head announced - 2026-10-12 12:42 proposed → checked (checked): the verdict names this commit - 2026-10-12 12:42 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-12 12:50 ready → stopped (closed): the pull request closed unmerged The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
Author
Contributor

Merge check at 2043b027

  • mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-controller)
  • mesh/repo-check: PASS — its merge-check.sh passed

Delivery plan — builds mesh-controller → novox; no bus step

  • tier 0: mesh-controller
  • novox: receives mesh-controller
  • read by no module's build: cmd/mesh-controller/apply_failing_test.go, cmd/mesh-controller/signals_test.go

Run by the build seat on g14 as build-1791808414964274385 (builds --log build-1791808414964274385).

--- the gate: mesh-controller
mesh-controller: ok, 57 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-12T12:29:08Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-12T12:29:08Z

modules the change moves:
  - mesh-controller (changed)

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 1 module(s) in 1 tier(s): mesh-controller

notes:
  - cmd/mesh-controller/apply_failing_test.go and cmd/mesh-controller/signals_test.go read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
ok  	github.com/novox/mesh-controller/cmd/mesh-builder	1.051s
ok  	github.com/novox/mesh-controller/cmd/mesh-controller	260.933s
ok  	github.com/novox/mesh-controller/examples/objectstore-provisioner	1.026s
ok  	github.com/novox/mesh-controller/examples/postgres-provisioner	1.036s
ok  	github.com/novox/mesh-controller/examples/redis-provisioner	1.021s
ok  	github.com/novox/mesh-controller/examples/route-proxy	1.163s
ok  	github.com/novox/mesh-controller/internal/artifacts	1.056s
?   	github.com/novox/mesh-controller/internal/beside	[no test files]
ok  	github.com/novox/mesh-controller/internal/broker	5.977s
ok  	github.com/novox/mesh-controller/internal/builder	16.113s
ok  	github.com/novox/mesh-controller/internal/catalogue	7.536s
ok  	github.com/novox/mesh-controller/internal/conditions	3.057s
ok  	github.com/novox/mesh-controller/internal/envfile	1.035s
ok  	github.com/novox/mesh-controller/internal/facts	1.036s
ok  	github.com/novox/mesh-controller/internal/identity	12.793s
ok  	github.com/novox/mesh-controller/internal/inventory	99.702s
ok  	github.com/novox/mesh-controller/internal/lease	17.937s
ok  	github.com/novox/mesh-controller/internal/licences	12.325s
ok  	github.com/novox/mesh-controller/internal/licences/adapters	1.014s
ok  	github.com/novox/mesh-controller/internal/link	66.150s
ok  	github.com/novox/mesh-controller/internal/lint	2.110s
ok  	github.com/novox/mesh-controller/internal/outward	1.033s
ok  	github.com/novox/mesh-controller/internal/overlay	1.023s
ok  	github.com/novox/mesh-controller/internal/secrets	1.403s
ok  	github.com/novox/mesh-controller/internal/store	3.544s
ok  	github.com/novox/mesh-controller/internal/testbus	1.208s
ok  	github.com/novox/mesh-controller/internal/token	1.017s
store tests, each run against the store:
  ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeWalksTheOthersAgain (0.40s)
  ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeEndsTheMergesThatContainIt (0.40s)
  ./cmd/mesh-controller --- PASS: TestAStopWalksTheOthersAgainInTheOpenBatch (0.39s)
  ./cmd/mesh-controller --- PASS: TestAStopNamesWhatStaysOnTheMachinesAndWhatStillCarriesIt (0.40s)
  ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeLeavesItOutOfABatch (0.36s)
  ./cmd/mesh-controller --- PASS: TestSavingPlansAndMovingMergesIsOneAct (0.36s)
  ./cmd/mesh-controller --- PASS: TestNodeShowAndStatusReadTheLastSendFromTheStore (0.41s)
  ./cmd/mesh-controller --- PASS: TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds (0.27s)
  ./cmd/mesh-controller --- PASS: TestAWalkLetGoAndStoppedByVerbsIsSaidStartedAndEnded (0.41s)
  ./cmd/mesh-controller --- PASS: TestAWalkStoppedOrClosedByAPersonIsSaidEndedStopped (0.52s)
  ./cmd/mesh-controller --- PASS: TestAWalkRetriedByAPersonIsSaidStartedAgain (0.40s)
  ./cmd/mesh-controller --- PASS: TestABacklogReleasedByAPersonIsSaidStarted (0.52s)
  ./cmd/mesh-controller --- PASS: TestAWalkEndedByCancelOrClearIsSaidEnded (0.43s)
  ./cmd/mesh-controller --- PASS: TestAWalkRestartedByARebuildIsSaidStarted (0.37s)
  ./cmd/mesh-controller --- PASS: TestAWalkStoppedByHandAndRetriedEndsFailedWhenItFailsAgain (0.43s)
  ./internal/inventory --- PASS: TestTheLastSendSaysWhatTheMachineAnswered (0.52s)
  ./internal/inventory --- PASS: TestAReportIsTheLastSendsAnswerOnlyWhenItIsAboutThatSend (0.26s)
  ./internal/inventory --- PASS: TestAnIssuedValueIsSealedToEveryHolderOnceAndNotAgain (0.23s)
  ./internal/inventory --- PASS: TestAnIssuedValueIsSealedAgainAfterAKeyChangeOrAnAcceptance (0.23s)
  ./internal/inventory --- PASS: TestAConditionsLooksAreKeptBesideTheBucket (0.22s)
  ./internal/inventory --- PASS: TestAWalksStartIsKeptAndARefusedSaveLeavesItUnsaid (0.21s)
**Merge check** at `2043b027` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-controller) - `mesh/repo-check`: **PASS** — its merge-check.sh passed **Delivery plan** — builds mesh-controller → novox; no bus step - tier 0: mesh-controller - novox: receives mesh-controller - read by no module's build: cmd/mesh-controller/apply_failing_test.go, cmd/mesh-controller/signals_test.go Run by the build seat on g14 as `build-1791808414964274385` (`builds --log build-1791808414964274385`). ``` --- the gate: mesh-controller mesh-controller: ok, 57 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8) long-running resources without health: 1 files asking for a setting without saying whether it is trusted: 1 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-12T12:29:08Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-12T12:29:08Z modules the change moves: - mesh-controller (changed) machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 1 module(s) in 1 tier(s): mesh-controller notes: - cmd/mesh-controller/apply_failing_test.go and cmd/mesh-controller/signals_test.go read by no module's build, and rebuild nothing (issue 280) --- its merge-check.sh (go toolchain) ok github.com/novox/mesh-controller/cmd/mesh-builder 1.051s ok github.com/novox/mesh-controller/cmd/mesh-controller 260.933s ok github.com/novox/mesh-controller/examples/objectstore-provisioner 1.026s ok github.com/novox/mesh-controller/examples/postgres-provisioner 1.036s ok github.com/novox/mesh-controller/examples/redis-provisioner 1.021s ok github.com/novox/mesh-controller/examples/route-proxy 1.163s ok github.com/novox/mesh-controller/internal/artifacts 1.056s ? github.com/novox/mesh-controller/internal/beside [no test files] ok github.com/novox/mesh-controller/internal/broker 5.977s ok github.com/novox/mesh-controller/internal/builder 16.113s ok github.com/novox/mesh-controller/internal/catalogue 7.536s ok github.com/novox/mesh-controller/internal/conditions 3.057s ok github.com/novox/mesh-controller/internal/envfile 1.035s ok github.com/novox/mesh-controller/internal/facts 1.036s ok github.com/novox/mesh-controller/internal/identity 12.793s ok github.com/novox/mesh-controller/internal/inventory 99.702s ok github.com/novox/mesh-controller/internal/lease 17.937s ok github.com/novox/mesh-controller/internal/licences 12.325s ok github.com/novox/mesh-controller/internal/licences/adapters 1.014s ok github.com/novox/mesh-controller/internal/link 66.150s ok github.com/novox/mesh-controller/internal/lint 2.110s ok github.com/novox/mesh-controller/internal/outward 1.033s ok github.com/novox/mesh-controller/internal/overlay 1.023s ok github.com/novox/mesh-controller/internal/secrets 1.403s ok github.com/novox/mesh-controller/internal/store 3.544s ok github.com/novox/mesh-controller/internal/testbus 1.208s ok github.com/novox/mesh-controller/internal/token 1.017s store tests, each run against the store: ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeWalksTheOthersAgain (0.40s) ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeEndsTheMergesThatContainIt (0.40s) ./cmd/mesh-controller --- PASS: TestAStopWalksTheOthersAgainInTheOpenBatch (0.39s) ./cmd/mesh-controller --- PASS: TestAStopNamesWhatStaysOnTheMachinesAndWhatStillCarriesIt (0.40s) ./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeLeavesItOutOfABatch (0.36s) ./cmd/mesh-controller --- PASS: TestSavingPlansAndMovingMergesIsOneAct (0.36s) ./cmd/mesh-controller --- PASS: TestNodeShowAndStatusReadTheLastSendFromTheStore (0.41s) ./cmd/mesh-controller --- PASS: TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds (0.27s) ./cmd/mesh-controller --- PASS: TestAWalkLetGoAndStoppedByVerbsIsSaidStartedAndEnded (0.41s) ./cmd/mesh-controller --- PASS: TestAWalkStoppedOrClosedByAPersonIsSaidEndedStopped (0.52s) ./cmd/mesh-controller --- PASS: TestAWalkRetriedByAPersonIsSaidStartedAgain (0.40s) ./cmd/mesh-controller --- PASS: TestABacklogReleasedByAPersonIsSaidStarted (0.52s) ./cmd/mesh-controller --- PASS: TestAWalkEndedByCancelOrClearIsSaidEnded (0.43s) ./cmd/mesh-controller --- PASS: TestAWalkRestartedByARebuildIsSaidStarted (0.37s) ./cmd/mesh-controller --- PASS: TestAWalkStoppedByHandAndRetriedEndsFailedWhenItFailsAgain (0.43s) ./internal/inventory --- PASS: TestTheLastSendSaysWhatTheMachineAnswered (0.52s) ./internal/inventory --- PASS: TestAReportIsTheLastSendsAnswerOnlyWhenItIsAboutThatSend (0.26s) ./internal/inventory --- PASS: TestAnIssuedValueIsSealedToEveryHolderOnceAndNotAgain (0.23s) ./internal/inventory --- PASS: TestAnIssuedValueIsSealedAgainAfterAKeyChangeOrAnAcceptance (0.23s) ./internal/inventory --- PASS: TestAConditionsLooksAreKeptBesideTheBucket (0.22s) ./internal/inventory --- PASS: TestAWalksStartIsKeptAndARefusedSaveLeavesItUnsaid (0.21s) ```
mesh-admin closed this pull request 2026-10-12 12:50:24 +00:00
This repo is archived. You cannot comment on pull requests.
No Reviewers
1 Participants
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: novox/mesh-controller#261