Compare commits

..
Author SHA1 Message Date
jschoubben 91a7c1eaff Keep a plan's backups in the store: a migration and the plan's read and write (issues 1065, 1066)
mesh/repo-check waiting in the build queue since 00:05: asked by the merge tool against trunk 7c9c187ca6e2
mesh/delivery stopped: Operator's word (2026-10-18 00:09): split the group. After mesh-catalog #353 merged on the group's Approve (23:48:51), #375's merg…
mesh/delivery-group group feat/backup-before-send stopped: a member was stopped
mesh/merge-gate building on shanks since 00:10
2026-10-17 23:25:57 +02:00
jschoubben 08e000bb56 Read the backup holders of every machine, and the notes fixture's data at a resource (issues 1065, 1066)
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 08e000bb56aa merged into trunk 7c9c187ca6e2 beside mesh-catalog#353@c898a937 — fail: its merge-check.sh failed: --- FAIL: TestADeliver…
mesh/delivery-group group feat/backup-before-send rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-17 23:04:52 +02:00
jschoubben a39d6ce78d Fail a backup whose holder cannot be asked at the bound, not retry forever (issue 1066)
mesh/merge-gate head a39d6ce78d19 merged into trunk 7c9c187ca6e2 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head a39d6ce78d19 merged into trunk 7c9c187ca6e2 — fail: its merge-check.sh failed: --- FAIL: TestADeliveryThatMovesABackedUpModuleWaitsFor…
mesh/delivery-group group feat/backup-before-send rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-17 22:53:59 +02:00
jschoubben c531c78eb5 Hold a plan whose backup failed or stayed silent, say it, and ask again by a verb (issue 1066) 2026-10-17 22:48:28 +02:00
jschoubben 9a20547dbd Ask the backup seats before the first tier (issue 1065) 2026-10-17 22:45:09 +02:00
mesh-tools 7c9c187ca6 Merge pull request 'A group is one batch and one delivery plan, ordered by the module graph and gated by its composed check; unit and integration tests split (issues 977, 895)' (#374) from feat/one-group-one-plan into main 2026-10-17 20:11:00 +00:00
jschoubben 2d24fbb716 Drop the two deleted release-branch tests from the store tests list (issue 977)
mesh/merge-gate head 2d24fbb71611 merged into trunk b77a298a7eee beside mesh-catalog#352@07717974 — pass: builds build-agent, mesh-controller → ace, g14, n…
mesh/repo-check head 2d24fbb71611 merged into trunk b77a298a7eee beside mesh-catalog#352@07717974 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh):…
mesh/delivery delivered
mesh/delivery-group group feat/one-group-one-plan delivered: every member is delivered
2026-10-17 21:49:58 +02:00
jschoubben 9cdc7d6276 Bound the unit level at 5 s, twice the measure, while issue 1055 brings it to 1 s (issue 895)
mesh/merge-gate head 9cdc7d6276d2 merged into trunk b77a298a7eee beside mesh-catalog#352@07717974 — pass: builds build-agent, mesh-controller → ace, g14, n…
mesh/repo-check head 9cdc7d6276d2 merged into trunk b77a298a7eee beside mesh-catalog#352@07717974 — fail: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh):…
mesh/delivery-group group feat/one-group-one-plan rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-17 21:40:42 +02:00
jschoubben 1a309a3783 Merge remote-tracking branch 'origin/main' into feat/one-group-one-plan
# Conflicts:
#	merge-check.sh
2026-10-17 21:39:24 +02:00
mesh-tools b77a298a7e Merge pull request 'merge-check.sh runs the integration level with the integration tag and times each level, before any test carries the tag (issue 895)' (#373) from fix/895-script-first into main 2026-10-17 19:39:02 +00:00
jschoubben 64415f8984 Merge branch 'feat/one-group-one-plan' into g/977 2026-10-17 21:37:37 +02:00
jschoubben 07b050790e Review of issue 977: keep the plan's own order for its gates, hold a group only for a composed mesh-delivery, keep mark-ready one release, compare with the batch's start, fail a lone cycle (issue 977) 2026-10-17 21:37:37 +02:00
jschoubben 8db72edd3c merge-check.sh runs and vets the integration level with the integration tag and times each level; the unit bound is not enforced yet (issue 895)
mesh/merge-gate head 8db72edd3c69 merged into trunk d5d54498358e — pass: the change touches no module of the mesh's graph
mesh/repo-check head 8db72edd3c69 merged into trunk d5d54498358e — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery delivered
2026-10-17 21:32:53 +02:00
jschoubben 66dc501fc4 Split the unit level from the integration level in cmd, builder, link, catalogue, inventory; cheap fixtures for the archive-entry bound, the registry waits and the catalogue copy (issue 895) 2026-10-17 21:32:23 +02:00
jschoubben aac4bae5a4 A group member's own check runs its repository's check alone, a cycle fails the composed check, and mark-ready leaves the delivery seat (issue 977, part D, ADR 0403) 2026-10-17 21:22:55 +02:00
jschoubben 8b3a9163d0 Hold a group's batch open for its last member and keep both kinds of member in one batch and one plan (issue 977, part C, ADR 0418) 2026-10-17 21:06:33 +02:00
jschoubben e43d52c1ea Draw a plan's order per module from the graph and the manifests' change, and read no after: line (issue 977, part B, ADR 0419) 2026-10-17 20:57:30 +02:00
jschoubben 0d83c6ffc9 Merge branch 'feat/one-group-one-plan' into g/977 2026-10-17 20:43:10 +02:00
jschoubben b752b1f1df Remove the release-branch judge from the controller, the wire and the build seat (issue 977, part A, ADR 0401) 2026-10-17 20:43:06 +02:00
jschoubben d209de58b7 Merge branch 'feat/one-group-one-plan' into t/895 2026-10-17 20:41:57 +02:00
jschoubben 14fe2ad736 Split the unit level from the integration level outside cmd, builder, link, catalogue, inventory; merge-check.sh times and bounds the unit level (issue 895) 2026-10-17 20:41:57 +02:00
jschoubben d977f64ec9 Read an empty source ref as the trunk and say a merge into another branch starts no delivery (issue 977, part A, issue 889) 2026-10-17 20:37:58 +02:00
mesh-tools d5d5449835 Merge pull request 'build-agent and route-proxy live whole in mesh-controller (issue 1019, ADR 0420)' (#370) from feat/whole-modules into main 2026-10-17 17:31:58 +00:00
jschoubben 874a91a3c6 Ask the checks again on today's main (issue 1019)
mesh/merge-gate head 874a91a3c64f merged into trunk 4669a8d1abe9 beside mesh-catalog#350@795806ef — pass: builds mesh-controller, route-proxy, new: modules…
mesh/repo-check head 874a91a3c64f merged into trunk 4669a8d1abe9 beside mesh-catalog#350@795806ef — pass: its merge-check.sh passed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792258323655869835 (novox/mesh-controller at d5d54498 into ma…
mesh/delivery-group group feat/whole-modules delivering: 1 of 2 delivered; novox/mesh-catalog@795806efa7ad, stopped for a member delivered since, waits for a l…
mesh-controller #372 merged 3 s before the operator's Approve; mesh-delivery put the group back
to proposed and asks no check again (issue opened 19:2x), so a new head asks it.
2026-10-17 19:21:21 +02:00
mesh-tools 4669a8d1ab Merge pull request 'Judge machines with the node-engine that knows unknown and knows (issue 1024)' (#372) from feat/owned-paths-controller into main 2026-10-17 17:10:03 +00:00
mesh-tools 34197afa01 Judge machines with the node-engine that knows unknown and knows (issue 1024, ADR 0422)
mesh/merge-gate head 34197afa0181 merged into trunk c43772fd94eb — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 34197afa0181 merged into trunk c43772fd94eb — pass: its merge-check.sh passed
mesh/delivery delivered
The gate composes every machine with the node-engine's own validator, vendored at mesh-host of 2026-10-15, which refuses the `unknown` and `knows` keys mesh-host #117 added; every catalogue manifest using them was refused on every machine (mesh-catalog #349). The pin moves to #117's merge, 6469e8238443, delivered; only the declaration validator changes in vendor/. A test holds the gate to taking such a directory.
2026-10-17 16:43:49 +00:00
jschoubben 0125ccd7d2 A member's own merge gate sees a move through the core clones beside it (issue 1043)
mesh/merge-gate out of date: judged beside mesh-catalog#350 at c47d94925811, it is now at 795806efa7ad; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged beside mesh-catalog#350 at c47d94925811, it is now at 795806efa7ad; judged again on a recheck or when it is merged
mesh/delivery-group group feat/whole-modules checking: composed check out of date: novox/mesh-controller@0125ccd7d200 was judged against trunk c43772fd94eb, th…
mesh/delivery superseded: a newer head of the same pull request
2026-10-17 18:40:34 +02:00
jschoubben fe5472bab4 Tests read build-agent and route-proxy from this repository when they live here (issue 1019) 2026-10-17 18:37:33 +02:00
jschoubben 48f5a7d3a9 module check refuses an artifact context naming another repository (issue 1019)
mesh/delivery-group group feat/whole-modules rejected: its order contradicts itself: novox/mesh-catalog@c47d94925811 and novox/mesh-controller@48f5a7d3a99a: bu…
mesh/merge-gate head 48f5a7d3a99a merged into trunk c43772fd94eb beside mesh-catalog#350@c47d9492 — fail: builds mesh-controller, route-proxy, new: modules…
mesh/repo-check head 48f5a7d3a99a merged into trunk c43772fd94eb beside mesh-catalog#350@c47d9492 — fail: its merge-check.sh failed: --- FAIL: TestTheBuild…
mesh/delivery superseded: a newer head of the same pull request
2026-10-17 18:18:22 +02:00
jschoubben 4864a43413 The merge gate refuses an artifact context naming another repository (issue 1019) 2026-10-17 18:17:10 +02:00
jschoubben cd88dbd336 One route-proxy Dockerfile: the module's, which the Makefile builds (issue 1019) 2026-10-17 18:15:52 +02:00
jschoubben 65d570f114 Vendor route-proxy's SDK so its provider builds offline (issue 1019) 2026-10-17 18:15:43 +02:00
jschoubben 1586bd1134 Bring build-agent and route-proxy in whole (issue 1019) 2026-10-17 18:15:33 +02:00
mesh-tools c43772fd94 Merge pull request 'The controller accepts a reviewed move of a module between repositories on the operator's Approve (issue 1025, ADR 0420)' (#369) from feat/move-a-module into main 2026-10-17 15:39:57 +00:00
jschoubben 7ee2b42230 issue 1025: the nested-module check is a file of its own, and the gate test keeps the app module in its repository
mesh/merge-gate head 7ee2b42230dc merged into trunk b8c810a3499c beside mesh-catalog#348@40eed3a0 — pass: builds build-agent, mesh-controller, route-proxy …
mesh/repo-check head 7ee2b42230dc merged into trunk b8c810a3499c beside mesh-catalog#348@40eed3a0 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh):…
mesh/delivery delivered
mesh/delivery-group group feat/move-a-module delivered: every member is delivered
2026-10-17 17:27:46 +02:00
jschoubben e42becb479 issue 1025: the bus test makes the controller's consumers before it publishes mesh-delivery's word
mesh/merge-gate head e42becb479a4 merged into trunk b8c810a3499c beside mesh-catalog#348@40eed3a0 — pass: builds build-agent, mesh-controller, route-proxy …
mesh/repo-check head e42becb479a4 merged into trunk b8c810a3499c beside mesh-catalog#348@40eed3a0 — fail: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh):…
mesh/delivery-group group feat/move-a-module rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-17 17:18:56 +02:00
jschoubben 43b59c5dff issue 1025: the merge check vets and tests every nested Go module (ADR 0420 §5) 2026-10-17 15:58:46 +02:00
jschoubben 81715273d3 issue 1025: the operator's Approve of a reviewed group registers a moved module from its new repository (ADR 0420 §7) 2026-10-17 15:56:49 +02:00
jschoubben 085b661936 issue 1025: the merge gate judges a group's move of a module as a move (ADR 0420 §6) 2026-10-17 15:43:28 +02:00
jschoubben 21414307c8 issue 1025: a context naming the module's own repository is its own build source (ADR 0420 §4) 2026-10-17 15:41:35 +02:00
mesh-tools b8c810a349 Merge pull request 'Rollouts advance on events, and builds and checks wait on three work queues (issue 974, design 47 milestone 2)' (#368) from feat/events-and-queues into main 2026-10-17 12:52:34 +00:00
jschoubben 0f21fe7888 issue 974: review 2: a check stopped by its key says no verdict, an error judges nothing, and an unacknowledged check goes to a second builder
mesh/merge-gate head 0f21fe7888be merged into trunk 7d80155d1636 beside mesh-catalog#345@94f65ccc — pass: builds build-agent, mesh-controller, route-proxy …
mesh/repo-check head 0f21fe7888be merged into trunk 7d80155d1636 beside mesh-catalog#345@94f65ccc — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/events-and-queues delivered: every member is delivered
2026-10-17 14:12:50 +02:00
jschoubben bd257afbd5 issue 974: the builders say the checks they run in a bucket that ages, and a builder drops a waiting request for a head and trunk commit another is running 2026-10-17 14:10:26 +02:00
jschoubben 357e08a4f6 issue 974: a broker test shows the build agent's reads reach its module principal as a read and watch of the pull requests' keys 2026-10-17 14:04:27 +02:00
jschoubben a83b921af0 issue 974: acceptance 5: two merges within a minute leave each open pull request one queued check 2026-10-17 14:03:26 +02:00
jschoubben 3d09d5235f issue 974: the checks are queued by pull request on two work-queue streams, taken after a delivery build, dropped when judged, stopped when their key moves; the reordering and the forge-asking end are gone 2026-10-17 14:01:33 +02:00
jschoubben f883830975 issue 974: two work-queue streams for the checks of pull requests (CHECKS_READY, CHECKS_OTHER), their workers, grants and writer 2026-10-17 13:53:22 +02:00
jschoubben 96325c295c issue 974: the serving controller follows its own plan-moved, so a plan moved from the terminal is looked at 2026-10-17 13:47:47 +02:00
jschoubben 9ca2c2bfb3 issue 974: review 1: a verb's end, a provisioner client's word and a failed key read each ask or retry a look; the silent bucket ages; the acceptance test starts the rollout through the serving wiring 2026-10-17 13:45:41 +02:00
jschoubben 20c8d9454a issue 974: the step's acceptance test: a rollout advances on events with no ticker, a silent machine is named when its key expires, a replaced controller resumes 2026-10-17 13:33:49 +02:00
jschoubben 3d192a4857 issue 974: every plan is written through the one hook that says plan-moved, proven by reading the sources and by the moves a merge and a cut make 2026-10-17 13:30:27 +02:00
jschoubben 8af77c2c75 issue 974: a rollout advances on events (a look asked by each plan saved, report and expired silent key), the 30 s ticker goes, and each tier send is said as sent 2026-10-17 13:28:34 +02:00
jschoubben fd522f4322 issue 974: a rollout's wait for a machine is a key on the bus that its report deletes and its expiry names silent 2026-10-17 13:18:27 +02:00
mesh-tools 7d80155d16 Merge pull request 'No merge is dropped, and the merge window lives on the bus (issues 973 step 2, 921)' (#367) from feat/merge-window-on-the-bus into main 2026-10-17 10:54:04 +00:00
jschoubben bafc31bf85 The ssh-client test offers every key ssh-client's files ask of forge-account, read from them (issue 985)
mesh/merge-gate head bafc31bf8568 merged into trunk b0d489a86c23 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head bafc31bf8568 merged into trunk b0d489a86c23 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-17 12:43:21 +02:00
jschoubben fbf5a3f909 The held-merges integration test waits for the batch's window to close, as a joined batch's window opens again (issue 921) 2026-10-17 12:42:30 +02:00
jschoubben 20e14872ef Make the merges consumer before the events consumer narrows, from the old one's ack floor, so no merge is lost to the change (issue 973)
mesh/merge-gate head 20e14872ef1e merged into trunk b0d489a86c23 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 20e14872ef1e merged into trunk b0d489a86c23 — fail: its merge-check.sh failed: --- FAIL: TestSSHClientOwnsTheOperatorsSSHConfig (0.07s…
mesh/delivery superseded: a newer head of the same pull request
2026-10-17 12:33:18 +02:00
jschoubben 5702187695 Leave a merge unsettled whenever the controller's context ended, whatever the error (issue 921) 2026-10-17 12:30:49 +02:00
jschoubben a79182e53b Give the forge's merges a durable consumer of their own: no max-deliver, no reset, a growing delay while held (issue 973) 2026-10-17 12:28:27 +02:00
jschoubben ff16938096 Leave a held merge unsettled when the controller ends or the hold fails, never take it (issue 921) 2026-10-17 12:20:40 +02:00
jschoubben e3f22c737f Round a merge window key's time to live up to a whole second, as the server rounds down (issue 973) 2026-10-17 12:20:24 +02:00
jschoubben 2b99d455fc Integration test of the merge path on a test bus: a held merge is recorded and carried, a window closes by its key's expiry (issues 921, 973) 2026-10-17 12:14:43 +02:00
jschoubben 9e99f31c23 Remove the five-minute merge catch-up, its signal S5 and its watchdog: the event and its durable consumer keep every merge (issue 973) 2026-10-17 12:12:19 +02:00
jschoubben df39a499ef Close a batch's merge window by the expiry of a key on the bus, heard by a watcher, and drop the five-second cutter (issue 973) 2026-10-17 12:10:30 +02:00
jschoubben 11725908ec The merge window's bucket: keys that expire, written by the controller (issue 973) 2026-10-17 12:07:57 +02:00
jschoubben 379a52c2e8 Hold a merge the controller cannot record now, and hand it over again, instead of taking it (issue 921) 2026-10-17 12:03:00 +02:00
mesh-tools b0d489a86c Merge pull request 'A batch that shares nothing with a running rollout starts beside it (issue 973, step 1)' (#366) from feat/merge-window-on-the-bus into main 2026-10-16 23:11:17 +00:00
jschoubben 6355176fda The ssh-client test offers the forge from another node, as the catalogue's ssh-client now requires forge-account (issue 985)
mesh/merge-gate head 6355176fdab0 merged into trunk 6a62a7e552ea — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 6355176fdab0 merged into trunk 6a62a7e552ea — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-17 01:00:40 +02:00
jschoubben 300d3c1bf4 Cut a closed batch beside the open walks it shares nothing with, queue a related one with the reason, fold only related waiting walks (issue 973)
mesh/merge-gate head 300d3c1bf466 merged into trunk 6a62a7e552ea — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 300d3c1bf466 merged into trunk 6a62a7e552ea — fail: its merge-check.sh failed: --- FAIL: TestSSHClientOwnsTheOperatorsSSHConfig (0.01s)
mesh/delivery superseded: a newer head of the same pull request
2026-10-17 00:49:53 +02:00
jschoubben baf1501d81 A walk, a retry and a let-go wait only for a started walk they are related to (issue 973) 2026-10-17 00:46:27 +02:00
jschoubben a5ba47e23c Name the open walk a change is related to, own path related to all (issue 973) 2026-10-17 00:45:00 +02:00
mesh-tools 6a62a7e552 Merge pull request 'The merge gate composes each machine with its engine's contract, so moved modules are handed there as on the mesh (issue 969)' (#365) from fix/969-gate-composes-handing into main 2026-10-16 21:53:07 +00:00
fix-969 5890eb104d List the gate's store test so it cannot skip unseen (issue 969)
mesh/merge-gate head 5890eb104da0 merged into trunk 1cdf12df00b5 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 5890eb104da0 merged into trunk 1cdf12df00b5 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-16 23:18:14 +02:00
fix-969 262b39ae09 Test the gate's handing end to end: snapshot, store, and the composition either side (issue 969) 2026-10-16 23:16:38 +02:00
fix-969 2f2a8bb57e Compose each machine in the merge gate with its engine's contract, so moved modules are handed there as on the mesh (issue 969) 2026-10-16 23:12:16 +02:00
mesh-tools 1cdf12df00 Merge pull request 'Test the refusal of a jail whose pattern names the host twice' (#364) from fix/959-jail-pattern-test into main 2026-10-16 20:19:49 +00:00
jschoubben f614eeb445 Test the refusal of a jail whose pattern names the host twice
mesh/merge-gate head f614eeb445a1 merged into trunk 53a3c91f53ce — pass: the change touches no module of the mesh's graph
mesh/repo-check head f614eeb445a1 merged into trunk 53a3c91f53ce — pass: its merge-check.sh passed
mesh/delivery delivered
Main refuses such a jail (fail2ban would refuse its whole configuration and
keep no bans), but no test held it: the only one sat untracked in an old
worktree. Issue 959.
2026-10-16 22:04:36 +02:00
mesh-tools 53a3c91f53 Merge pull request 'Feature providers-after-switch-on, step 1: plans keep their route names and name the credential tool (issues 923, 928)' (#363) from feat/providers-after-switch-on into main 2026-10-16 19:12:52 +00:00
fix-ctl-pas df5f01bfa1 Name the credential tool of a software-issued offer in the plan (issue 928)
mesh/merge-gate head df5f01bfa1ee merged into trunk c7cb9f0a2cd3 beside mesh-catalog#337@d2992f81 — pass: builds build-agent, mesh-controller → ace, g14, n…
mesh/repo-check head df5f01bfa1ee merged into trunk c7cb9f0a2cd3 beside mesh-catalog#337@d2992f81 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/providers-after-switch-on delivered: every member is delivered
2026-10-16 19:35:56 +02:00
fix-ctl-pas 32014ac6de Keep a provision's contributes, at, from, as and names in the sent plans (issue 923) 2026-10-16 19:31:58 +02:00
mesh-tools c7cb9f0a2c Merge pull request 'Give a channel's verified sender back when the root-free judgement passes again (issue 930)' (#362) from fix/930-root-free-recomposes-verified-sender into main 2026-10-16 15:53:20 +00:00
jschoubben 87410d79ed Issue a verified sender again only over the membership that was read, so a newer push is not overwritten (issue 930)
mesh/merge-gate head 87410d79edd6 merged into trunk e231185572ba — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 87410d79edd6 merged into trunk e231185572ba — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-16 17:46:18 +02:00
jschoubben 1c3c3edc7d Prove on a real bus a lost verified sender is issued again once the judgement passes (issue 930) 2026-10-16 17:43:25 +02:00
jschoubben b2e2bf0638 Give a channel's verified sender back when the root-free judgement passes again, and say when it is dropped (issue 930)
A push composed during a failed judgement issued the router a membership without the verified sender and nothing composed it again, so every approval was refused. The self-check now issues it again within one run; a composition says which kind lost it and why.
2026-10-16 17:42:23 +02:00
mesh-tools e231185572 Merge pull request 'Grant the controller mesh-config's config verb in its own name (issue 925)' (#361) from fix/925-controller-asks-mesh-config into main 2026-10-16 14:46:14 +00:00
mesh-tools b12755c163 Prove on a real bus the controller asks mesh-config in its own name (issue 925)
mesh/merge-gate head b12755c16313 merged into trunk ead062569ed3 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head b12755c16313 merged into trunk ead062569ed3 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-16 14:39:53 +00:00
mesh-tools abf37c7212 Grant the controller mesh-config's config verb in its own name (issue 925)
mesh/merge-gate error: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; the check could not run: killed by hand
mesh/repo-check error: the check could not run: killed by hand
mesh/delivery superseded: a newer head of the same pull request
move-config asks mesh-config.config on the subject that names the
controller as caller; no grant named it, so the bus refused the move.
2026-10-16 14:36:19 +00:00
mesh-tools ead062569e Merge pull request 'The seat node-files: write, copy, move and delete a file on a node through the mesh, with trash and owned paths (issue 900, ADR 0394)' (#360) from feat/900-node-files into main 2026-10-16 14:25:00 +00:00
jschoubben f4ddddd59d Merge remote-tracking branch 'origin/main' into feat/900-node-files
mesh/merge-gate head f4ddddd59d9a merged into trunk f45b4cb21301 beside mesh-catalog#332@d4a92608 — pass: builds build-agent, mesh-controller → ace, g14, n…
mesh/repo-check head f4ddddd59d9a merged into trunk f45b4cb21301 beside mesh-catalog#332@d4a92608 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/900-node-files delivered: every member is delivered
2026-10-16 16:15:44 +02:00
mesh-tools f45b4cb213 Merge pull request 'Issue the provisioner client its own account when its module declares no broker (issue 917)' (#359) from fix/917-provisioner-account into main 2026-10-16 14:11:26 +00:00
jschoubben 407dc86a74 Merge pull request 'The delivery seat promises mark-ready: a reviewer marks a head ready through one audited verb (issue 913)' (#357) from feat/913-mark-ready into main
Reviewed-on: #357
2026-10-16 13:47:28 +00:00
jschoubben 210f7cf5e6 Merge remote-tracking branch 'origin/main' into feat/900-node-files
mesh/delivery-group group feat/900-node-files checking: composed check out of date: novox/mesh-controller@210f7cf5e64b was judged against trunk 611e2275cf7c, t…
mesh/merge-gate out of date: judged beside mesh-catalog#332 merged onto 3db85a0dde83, its main is now 8b53a3663244; judged again on a recheck or when it is…
mesh/repo-check out of date: judged beside mesh-catalog#332 merged onto 3db85a0dde83, its main is now 8b53a3663244; judged again on a recheck or when it is…
mesh/delivery superseded: a newer head of the same pull request
2026-10-16 15:10:03 +02:00
mesh-tools 611e2275cf Merge pull request 'The controller's catalogue tests judge the catalogue as the merge check does, so main passes again (issue 914)' (#358) from fix/914-catalogue-tests-judge-like-the-check into main 2026-10-16 13:09:43 +00:00
mesh-tools eea5866216 List the client's assign test among the store tests
mesh/merge-gate head eea58662163e merged into trunk 407dc86a74df — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head eea58662163e merged into trunk 407dc86a74df — pass: its merge-check.sh passed
mesh/delivery delivered
The merge check proves only listed store tests ran; unlisted, the one
test that shows assign issuing the client's account could skip unseen
(review of issue 917).
2026-10-16 13:03:06 +00:00
mesh-tools b5da0b2161 Issue the provisioner client its own account when its module declares no broker
mesh/merge-gate error: builds mesh-controller → novox; no bus step; the check could not run: killed by hand
mesh/repo-check error: the check could not run: killed by hand
mesh/delivery superseded: a newer head of the same pull request
The catalogue's mesh-provisioner-client declares only provisioner-broker:
the client speaks on the bus as itself. module issue refused it for
lacking a broker, assign minted nothing, and push refused it for lacking
an account, so no machine could run the client (issue 917).
2026-10-16 13:01:03 +00:00
jschoubben c860426c55 Narrow the bar test to what the controller owns for a moved bar; rendering is issue 915 (issue 914)
mesh/merge-gate head c860426c555a merged into trunk a20680dfe769 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head c860426c555a merged into trunk a20680dfe769 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-16 14:52:13 +02:00
jschoubben 65f24cd45d Judge catalogue tests as the check does: the predefined list and templates read from the tree (issue 914) 2026-10-16 14:50:40 +02:00
jschoubben 9cb08299b3 Merge remote-tracking branch 'origin/main' into feat/900-node-files
# Conflicts:
#	internal/catalogue/seats_test.go
2026-10-16 14:49:29 +02:00
jschoubben 114c99e02a The delivery seat promises mark-ready: a reviewer marks a head ready through one audited verb (issue 913)
mesh/merge-gate head 114c99e02a52 merged into trunk 611e2275cf7c beside mesh-catalog#331@2d1b2036 — pass: builds build-agent, mesh-controller → ace, g14, n…
mesh/repo-check head 114c99e02a52 merged into trunk 611e2275cf7c beside mesh-catalog#331@2d1b2036 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/913-mark-ready delivered: every member is delivered
2026-10-16 14:31:04 +02:00
mesh-tools a20680dfe7 Merge pull request 'Seal core credentials by the controller while the vault does not hold them (issue 908, ADR 0396)' (#356) from fix/908-seal-core-until-vault-holds into main 2026-10-16 10:38:54 +00:00
mesh-tools ebdbba213e List the renamed store test of issue 908 in testdata/store-tests
mesh/merge-gate head ebdbba213e73 merged into trunk 18c58b4903ad — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head ebdbba213e73 merged into trunk 18c58b4903ad — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-16 10:33:45 +00:00
mesh-tools 5eb5ea1774 Seal core credentials by the controller while the vault does not hold them, so the bus is never left out of a declaration (issue 908, ADR 0396)
mesh/merge-gate head 5eb5ea177482 merged into trunk 18c58b4903ad — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 5eb5ea177482 merged into trunk 18c58b4903ad — fail: its merge-check.sh failed: 1 store test(s) listed in testdata/store-tests did not …
mesh/delivery superseded: a newer head of the same pull request
2026-10-16 10:27:53 +00:00
jschoubben 5fa523f608 issue 900: restore takes a reason and an override; owned paths cite ADR 0394 section 5 2026-10-16 11:30:59 +02:00
jschoubben 9a5c02bd60 issue 900: gofmt files.go 2026-10-16 11:25:40 +02:00
jschoubben 9ce77e4e7b issue 900: node-files descriptions follow ADR 0394 (operator account, writes do not refuse secret places, overwrite keeps trash) 2026-10-16 11:25:34 +02:00
jschoubben 470c97654b issue 900: add the seat node-files beside node-reads, with its verbs, holder rule and owned-paths word 2026-10-16 11:16:44 +02:00
mesh-tools 18c58b4903 Merge pull request 'Feature 57 (config and provisioning): release mesh-controller' (#355) from release/57-config-and-provisioning into main 2026-10-16 08:42:05 +00:00
mesh-tools 8c381172e7 Restore the blank last line the patch route dropped from seven golden files, so the branch holds the reviewed tree (issue 875)
mesh/merge-gate head 8c381172e7ea merged into trunk d70bb1f38867 beside mesh-host#116@bff1ac14 — pass: builds build-agent, mesh-controller, route-proxy → a…
mesh/repo-check head 8c381172e7ea merged into trunk d70bb1f38867 beside mesh-host#116@bff1ac14 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group release/57-config-and-provisioning delivered: every member is delivered
2026-10-16 08:29:21 +00:00
mesh-tools d984816844 Feature 57 (config and provisioning): release mesh-controller, the reviewed release branch merged with main, on top of main 2026-10-16 08:28:38 +00:00
mesh-tools d70bb1f388 Merge pull request 'Judge a delivery group's composed check by the group's own controller, release branch included (issue 867, ADR 0239 §14)' (#352) from fix/867-group-check-judged-by-its-controller into main 2026-10-16 01:04:00 +00:00
mesh-toolsandClaude 151faa10a1 Judge a delivery group's composed check by the group's own controller, release branch included (issue 867, ADR 0239 §14)
mesh/merge-gate head 151faa10a13b merged into trunk a07dd78c3e90 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 151faa10a13b merged into trunk a07dd78c3e90 — pass: its merge-check.sh passed
mesh/delivery delivered
The composed check of mesh-controller#346 and mesh-host#114 into feat/57-config-and-provisioning was judged by the release branch's controller, which cannot build the node-engine member's validator, so the group could never pass. ADR 0239 §14 has the group's own controller judge when a member moves it; ADR 0358 §2 changes the judge only for a pull request into a release branch. A group member in mesh-controller now judges, with the mesh-host member's validator, built in the isolated judge environment and kept by content; the statuses say the group's controller judged.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-16 00:59:27 +00:00
mesh-tools a07dd78c3e Merge pull request 'Ask mesh-delivery for a recheck's group on a bus connection of its own (issue 867, ADR 0382 §1)' (#351) from fix/867-recheck-asks-on-the-bus into main 2026-10-16 00:33:18 +00:00
mesh-toolsandClaude 3fdbea4204 Ask mesh-delivery for a recheck's group on a bus connection of its own (issue 867, ADR 0382 §1)
mesh/merge-gate head 3fdbea420440 merged into trunk af60d2719c0c — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 3fdbea420440 merged into trunk af60d2719c0c — pass: its merge-check.sh passed
mesh/delivery delivered
`delivery check` runs as a command of its own, where no serving connection exists, so a recheck said its siblings could not be learned ("this controller is not on the bus") and mesh-host#114 was judged without its sibling. groupsAsked now asks through onTheBus, which lends the serving connection or dials one, as the hand-act log does. TestTheRecheckVerbAsksMeshDeliveryOnABusOfItsOwn drives the verb against a real responder.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-16 00:27:57 +00:00
mesh-tools af60d2719c Merge pull request 'Find a rechecked pull request's group by its repository and number (issue 867, ADR 0382 §1)' (#350) from fix/867-recheck-finds-its-group into main 2026-10-16 00:17:06 +00:00
mesh-toolsandClaude de5292323f Find a rechecked pull request's group by its repository and number (issue 867, ADR 0382 §1)
mesh/merge-gate head de5292323f33 merged into trunk a47976be4e69 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head de5292323f33 merged into trunk a47976be4e69 — pass: its merge-check.sh passed
mesh/delivery delivered
A recheck rebuilds the pull request without its head branch name, so the sibling lookup found no group and said nothing: mesh-host#114's recheck was judged without its sibling. With no branch name, the group is the one whose member_details hold the pull request; an older mesh-delivery says the siblings could not be learned. TestReplay867Recheck replays it.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-16 00:13:06 +00:00
mesh-tools a47976be4e Merge pull request 'Check a delivery group's member beside its siblings' merges (issue 867, ADR 0382)' (#348) from fix/867-group-siblings into main 2026-10-15 23:53:01 +00:00
mesh-tools 4446027108 Merge pull request 'Compose the forge in its manifest tests with the operator's account (issue 869)' (#349) from fix/869-forge-tests-name-an-account into main 2026-10-15 23:46:21 +00:00
mesh-toolsandClaude e42d72a80a Compose the forge in its manifest tests with the operator's account, as the mesh does (issue 869)
mesh/merge-gate head e42d72a80a62 merged into trunk 7543f64911c5 — pass: the change touches no module of the mesh's graph
mesh/repo-check head e42d72a80a62 merged into trunk 7543f64911c5 — pass: its merge-check.sh passed
mesh/delivery delivered
The gitea manifest names its patch spool's owner ${machine:account} since mesh-catalog d4b8b247 (issue 852); these tests gave the machine no account, so main failed its own merge check.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-15 23:36:30 +00:00
mesh-toolsandClaude 37fd73f378 Check a delivery group's member beside its siblings' merges, judged by the group's controller (issue 867, ADR 0382)
mesh/merge-gate head 37fd73f3781e merged into trunk 4446027108fd — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 37fd73f3781e merged into trunk 4446027108fd — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/867-group-siblings delivered: every member is delivered
A mesh-host change and a mesh-controller change that need each other could not both pass their own checks: each read the other at its base. The check now asks mesh-delivery for the group's open siblings into a base of the same name, reads each at its head merged onto its base, builds the judge from a controller sibling's merge in the isolated judge environment keyed on its inputs, and names the siblings in the verdict. With no answer from mesh-delivery the verdict says it could not learn them; a pull request with no sibling is checked as before. TestReplay867 replays the failure.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-15 23:27:12 +00:00
mesh-tools 7543f64911 Merge pull request 'Mail seats: a declared seat carries needs; a secret given is told once (issues 802, 835)' (#345) from feat/mail-seats into main 2026-10-15 17:44:20 +00:00
jochen 5cf8938f55 Let the genesis test accept a degradable grant missing from the installer's list (issue 835)
mesh/merge-gate head 5cf8938f55b3 merged into trunk 0ab61d1be8c5 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 5cf8938f55b3 merged into trunk 0ab61d1be8c5 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/mail-seats delivered: every member is delivered
Forge check of mesh-controller#345: the merge check clones mesh-host at the commit the
machines run, not its main, so a change that only edits mesh-host's examples (the
operator-channel.tell grant) can never be seen there until mesh-host's own module next
builds. operator-channel.tell is the one grant with a fallback for a refusal (the
secret-given condition, ADR 0380 §9), so mark it degradable and let
TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose pass without it while every
other grant stays required.
2026-10-15 19:29:13 +02:00
jochen a18c17d3cc Name the machine in the secret-given message (issue 835, review round 2)
mesh/delivery-group group feat/mail-seats delivering: 1 of 3 delivered
mesh/merge-gate head a18c17d3cc5f merged into trunk 0ab61d1be8c5 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head a18c17d3cc5f merged into trunk 0ab61d1be8c5 — fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheContro…
mesh/delivery superseded: a newer head of the same pull request
ADR 0380 §1 says the message names the module and the machine it is
for; the words named no machine at the terminal and only the desk's
machine at the desk. The headline now ends "on <node>" and the tell
carries "subject": <node> in both cases.

Also covers two gaps review round 2 flagged: a tell failure other
than ErrNothingServes/ErrGrantRefused surfaces and raises no
condition, and the terminal's hand act now names who asked (not just
where), extracted into terminalGiveHandAct so it is testable on its
own.
2026-10-15 18:45:03 +02:00
jochen 5f24a47df6 Refresh captured mesh-host at c3b2fce, carrying the operator-channel.tell grant (issue 835)
mesh-host joined feat/mail-seats to add the controller's new
operator-channel.tell publish subject to its genesis user list. Moved
per testdata/beside/CAPTURED's own recipe (git archive, never by hand);
the refreshed snapshot also carries main's drift since the last capture
(ADR 0338's store placement). TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose
now passes; merge-check.sh green.
2026-10-15 17:55:30 +02:00
jochen 435468c151 A secret given tells the operator once, opens no condition (issue 835)
ADR 0380: giving a module's own secret now asks the operator channel's
tell over the bus, naming the secret, where, when and who asked, and
raises no secret-given condition. Until nothing serves tell yet, or the
controller's grant is refused (link.ErrNothingServes / the new
link.ErrGrantRefused), the condition is raised exactly as before, so the
change composes whichever side of issue 797 merges first. A successful
tell also retires every open secret-given condition. The terminal's own
secret accept now records a hand act too, as the desk path already did.

The controller's bus grant for operator-channel.tell is added to the
composed config (golden fixture updated).
2026-10-15 17:45:45 +02:00
jochen b5a441a978 Let a module-declared seat carry needs, so mail-notifier can need node-notifier (issue 802)
ADR 0220's dependency mechanism served compiled seats only and retired its one user with
node-resolver-config (commit f68521da). ADR 0378 S8 revives it for module-declared seats:
SeatDeclaration gains Needs, judged by the catalogue check (node scope only, a name that
resolves to a node seat somewhere in the catalogue, never itself), and DependsOn now takes
the catalogue so a claimed seat's needs are found wherever they are declared.
2026-10-15 17:18:23 +02:00
mesh-tools 0ab61d1be8 Merge pull request 'Mail and bus hardening: restricted streams, recorded moves that restart nothing, desk prompts, abandoned calls (issues 727, 728, 785, 789, 711)' (#344) from feat/mail-hardening into main 2026-10-15 14:34:24 +00:00
jochen 2347a4c73b Merge branch 'feat/mail-hardening' into mh-711-728-r1
mesh/merge-gate head 2347a4c73bd0 merged into trunk 20e58907b0f8 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 2347a4c73bd0 merged into trunk 20e58907b0f8 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/mail-hardening delivered: every member is delivered
2026-10-15 15:43:34 +02:00
jochen ceffbf500a Merge branch 'feat/mail-hardening' into mh-711-728-r1 2026-10-15 15:29:15 +02:00
jochen d55e148a1d Never wait NextMsg's bound for a dead letter held already overcounts (issue 728) 2026-10-15 15:28:56 +02:00
jochen ab74bbb56b Merge branch 'feat/mail-hardening' into mh-789-r1 2026-10-15 15:27:13 +02:00
jochen 320dde41d9 Count an own secret's prompt by character and refuse {member} on a single secret (issue 789) 2026-10-15 15:27:06 +02:00
jochen e603b2cae1 Bound MarkAbandoned's compare-and-set retries, giving up after 10 tries (issue 711) 2026-10-15 15:20:43 +02:00
jochen 87780f482b Merge branch 'feat/mail-hardening' into mh-785-r1 2026-10-15 15:15:55 +02:00
jochen 867a3f81a9 Hold a record-policy move that changes a non-file/directory resource (issue 785)
ADR 0377 §3's allowlist names only tools bundles, files no container mounts
and holds no data, and manifest fields that recreate nothing. Holds only
checked containers, files, directories and processes/services; any other
resource type (archive, package, user, network, ...) fell through unheld.
Add a default case that holds and names the resource type and id.
2026-10-15 14:59:06 +02:00
jochen 9aeb5f4493 Keep a restricted message past its stream's age as a record without its body, as ADR 0376 §3 says (issue 728)
Reverts 45b6c347, which kept nothing of such a message and raised an aged-out
condition: that departed from an accepted decision on a lead's word alone. The
concern behind it (the record keeps the subject, with no age) is its own issue.
2026-10-15 14:56:24 +02:00
jochen 15a374659e Merge branch 'feat/mail-hardening' into mh-711 2026-10-15 14:47:09 +02:00
jochen 868e3ae377 Never mark a call abandoned over its recorded finish when a controller starts (issue 711)
A starting controller's sweep read each call it saw running and wrote it back
abandoned unconditionally, so a finish written in between, with its kept answer,
was lost. The mark is now a compare-and-set on the call's record: a finish already
recorded, or racing the mark, stands.
2026-10-15 14:47:09 +02:00
jochen ce77a2caaa Merge branch 'feat/mail-hardening' into mh-785 2026-10-15 14:27:37 +02:00
jochen 0d1bb035cb Let a recorded module's move that restarts nothing roll out with its walk (issue 785)
Under a module's own record (its manifest's, or its irreplaceable data's), every
move waited for a person's push, even one that only changed a tools bundle: the
database server stayed behind on ace on 2026-10-15 although its walk said it
recreated none of its containers. ADR 0377 lets such a move roll out.

A move is judged per machine against the build that machine was last sent: it
goes when no container, mounted file, data path, data declaration, provider or
(for a module with data) long-running process changes. A person's record, the
bus and any move that cannot be judged still wait. The walk keeps per machine
which way each recorded module went and why, says it in its note and log, and
status names a machine behind with `push <node>` only for a move that waits.
2026-10-15 14:27:37 +02:00
jochen 45b6c347b2 Keep nothing of a restricted message past its stream's age, only count it (issue 728)
A record kept without its body still carried its subject, which names whose mail it was, with no age. The lead decided such a message is not kept at all: the notice is taken, the message counted, and the log and an aged-out condition say how many messages of the stream aged out undelivered. Unrestricted streams are unchanged.
2026-10-15 14:22:07 +02:00
jochen d03e24f423 Age a restricted stream's dead letters out with it and never show their body (issue 728)
DEAD_LETTERS kept a message given up on MAIL forever and showed it whole to any caller of dead-letters, outside the grant and past MAIL's week. Per ADR 0376 §3-§6: the kept copy carries a per-message time to live of what is left of its stream's age (a copy already past it is kept without its body), DEAD_LETTERS and EVENTS allow per-message TTL, the list and show give no body or own headers for it, delivering it again needs a minute left and the copy keeps the time left, and its max-deliveries condition clears saying how many aged out undelivered and from which stream.
2026-10-15 14:11:37 +02:00
jochen f75cd0239d Hear a restricted stream only from a module's own account, never the tool runner's (issue 727)
The tool runner's bus account carried the heard grants, consumer and bind word of every module it runs, and every agent session on that machine reaches the bus as that account, so any session could pull a granted module's consumer on MAIL. Per ADR 0376 §1 the controller now composes a restricted stream's consumer, grants and bind word only for a module that declares runs-as; registration refuses a consumes of a restricted stream without runs-as (§2), and a grant that composes nothing is said in doctor (D-heard) and in the controller's log on every raise and send.
2026-10-15 14:11:37 +02:00
jochen f7928a2bf7 Let an own secret's manifest declare a desk prompt (issue 789)
A family like mesh-mail's mail-credential-* names only the module and the secret's key at the desk, so the operator cannot tell which account is asked. An own secret (or a family) may now declare a one-line prompt in its manifest, with {member} filled by a family member's suffix; the module check refuses one that is not one line, over 120 characters, or holds another placeholder. desk_secret.go reads it from the controller's own catalogue and sends it to the desk as what, never from the call or the module's runtime; omitted when none is declared, so an old desk or module behaves as today.
2026-10-15 13:47:20 +02:00
mesh-tools 20e58907b0 Merge pull request 'Read the borrowed-words log without the logger's clock (issue 752)' (#343) from 752-log-without-clock into main 2026-10-15 00:43:53 +00:00
mesh-toolsandClaude Opus 5.5 f877c33c7c Read the borrowed-words log without the logger's clock, so the test passes at any time of day (issue 752)
mesh/merge-gate head f877c33c7c18 merged into trunk 0b7e740b299c — pass: builds build-agent → ace, g14, novox, shanks; no bus step; every machine composes …
mesh/repo-check head f877c33c7c18 merged into trunk 0b7e740b299c — pass: its merge-check.sh passed
mesh/delivery delivered
The test forbids "23:3" in the log line, and the logger's own timestamp held it from 23:30
to 23:39 every night, so the check went red for no reason in that window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:51:05 +00:00
mesh-tools 0b7e740b29 Merge pull request 'Let the tool runner's account hear the event families its manifest names (issue 745, ADR 0367)' (#342) from 745-mesh-wait into main 2026-10-14 23:14:41 +00:00
mesh-toolsandClaude Opus 5.5 755457cc37 Let the tool runner's account hear the event families its manifest names (issue 745, ADR 0367)
mesh/merge-gate head 755457cc379e merged into trunk 4cc6b6c566a5 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 755457cc379e merged into trunk 4cc6b6c566a5 — pass: its merge-check.sh passed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792019685173332639 (novox/mesh-controller at 0b7e740b into ma…
A session on the mesh MCP server polls for every verdict, answer and delivery because the tool
runner's account hears no event. ADR 0367 lets the tool runner's module name, one by one, the
families a session may wait on: the manifest's `waits`, refused for any other module and for a
wildcard. The controller grants the runtime's account a plain subscription on those subjects
alone, makes no durable consumer for them, and issues the same list in the tool runner's
membership, so the grant and what `mesh_wait` offers come from one composition and no call or
setting widens it. A family on a seat whose events end in a kind or an asker must name it, and is
refused at composition without it, so it is never granted a subject nothing publishes on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:05:14 +00:00
mesh-admin 4cc6b6c566 Merge pull request 'Say a read-only mesh-cli call in no line of its own, and count them hourly (hq issue 762)' (#341) from fix/762-a-read-call-writes-no-journal-line into main 2026-10-14 22:33:08 +00:00
jschoubben 9e11790bd7 Say a read-only mesh-cli call in no line of its own, and count them hourly (hq issue 762)
mesh/merge-gate head 9e11790bd74a merged into trunk 3ed08c3464e3 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 9e11790bd74a merged into trunk 3ed08c3464e3 — pass: its merge-check.sh passed
mesh/delivery delivered
A page polling plans every few seconds wrote about 14,000 journal lines a day. A line is judged a
read by the generic command verb's reading forms (ADR 0266), on its every word, so plans go,
conditions silence and settings set are still said; a refused or failed read is said too.
2026-10-15 00:28:17 +02:00
mesh-admin 3ed08c3464 Merge pull request 'Take no rebuild into a waiting rollout, and ask what a tier left unasked (hq issue 754)' (#340) from fix/754-a-rebuild-in-the-wait-is-not-the-walks-build into main 2026-10-14 22:06:58 +00:00
jschoubben 9f072c110c Take no rebuild into a waiting walk, and ask what a tier left unasked (hq issue 754)
mesh/merge-gate head 9f072c110c14 merged into trunk 60cc9ec363df — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 9f072c110c14 merged into trunk 60cc9ec363df — pass: its merge-check.sh passed
mesh/delivery delivered
A rebuild that landed while a walk waited for mesh-delivery's word was taken as
the walk's build of that module; once let go, the walk asked a tier only when
every module of it was unasked, so it never asked the rest and stood still.

A walk now takes only the outcomes of builds it asked, and none while it waits;
a rebuild does not join a waiting walk and says it runs beside it; and a tier
asked in part is asked the rest, including a module held built that the walk
never asked for, which is somebody else's build and not sent as the walk's.
2026-10-15 00:02:28 +02:00
mesh-admin 60cc9ec363 Merge pull request 'End the check of a pull request's old head when a new head is checked (hq issue 726)' (#339) from fix/726-a-new-push-ends-the-old-check into main 2026-10-14 20:28:16 +00:00
mesh-admin 5d22d7df86 Merge pull request 'A module moves only when its program changes (hq issue 721)' (#336) from fix/a-module-moves-only-when-its-program-changes into main 2026-10-14 20:21:05 +00:00
mesh-admin 86057f4551 Merge pull request 'A module may declare a stream of its own, heard only by the consumers its trusted config value grants (hq issue 718, ADR 0363)' (#338) from fix-718-module-streams-and-grants into main 2026-10-14 20:13:33 +00:00
jschoubben e4acab32d6 End the check of a pull request's old head when a new head is checked
mesh/merge-gate head e4acab32d6a2 merged into trunk 60cc9ec363df — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head e4acab32d6a2 merged into trunk 60cc9ec363df — pass: its merge-check.sh passed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792009731456564734 (novox/mesh-controller at 60cc9ec3 into ma…
A check judges one head; once the pull request has another, its verdict
answers nobody and its run holds a build machine the new head waits
behind. Asking a check of R#N at H now drops every waiting check of R#N
at another head and kills every one in flight, saying each in the log.
Announcements are unordered and a recheck asks a recorded head, so this
happens only when the forge confirms H is the pull request's current
head; otherwise nothing ends and that is said. A one-shot recheck ends
them within the call. Tries, walk builds, group checks, other pull
requests' checks and rechecks of the same head are left alone.

novox/hq issue 726
2026-10-14 22:10:15 +02:00
jochen 743e6478a7 Remove a restricted stream's consumer when the bus refuses to narrow it, so its wider filters deliver nothing more (ADR 0363 §6, review of PR 338)
mesh/merge-gate head 743e6478a76c merged into trunk c2ec0a601acb — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 743e6478a76c merged into trunk c2ec0a601acb — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-14 22:05:47 +02:00
jochen ede2772b3c Deliver a module stream's given-up message again only while its consumer is still granted it (ADR 0363 §7)
mesh/merge-gate head ede2772b3c90 merged into trunk c2ec0a601acb — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head ede2772b3c90 merged into trunk c2ec0a601acb — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-14 21:55:55 +02:00
jschoubben b34055f580 Register the compared module after the fixture's own registration (review of #336)
mesh/merge-gate head b34055f580d4 merged into trunk 86057f455194 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head b34055f580d4 merged into trunk 86057f455194 — pass: its merge-check.sh passed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792009731456564734 (novox/mesh-controller at 60cc9ec3 into ma…
mesh/delivery-group group fix/a-module-moves-only-when-its-program-changes delivered: every member is delivered
2026-10-14 21:53:50 +02:00
jochen ee13e75bf3 Let a module declare a stream of its own and restrict who hears it by a trusted config value (hq issue 718, ADR 0363)
mesh-mail's events must live on a stream of their own and reach only the modules the operator granted
an account (ADR 0359 §2, §4). The controller now creates a module's declared stream, publishes its events
on mesh.stream.<module>.event.<event>, and composes a consumer's durable consumer, filters and bus grants
on a restricted stream only where its consumes asks and the provider's trusted value names it; a consumer
that loses its grant has its consumer removed on the next assertion. Nothing changes while no manifest
declares a stream.
2026-10-14 21:52:45 +02:00
jschoubben 480657d95b Register the module a walk's store test compares with, as the mesh runs it (review of #336)
mesh/merge-gate head 480657d95b9b merged into trunk c2ec0a601acb — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 480657d95b9b merged into trunk c2ec0a601acb — fail: its merge-check.sh failed: --- FAIL: TestAWalkWhoseEveryBuildCameOutTheSameIsDone …
mesh/delivery-group group fix/a-module-moves-only-when-its-program-changes rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-14 21:43:20 +02:00
jschoubben 38c87a7e47 Compare a build with the one the mesh registered, not the newest, which may be one put back (review of #336, hq issue 721)
mesh/merge-gate head 38c87a7e4714 merged into trunk c2ec0a601acb — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 38c87a7e4714 merged into trunk c2ec0a601acb — fail: its merge-check.sh failed: --- FAIL: TestAWalkWhoseEveryBuildCameOutTheSameIsDone …
mesh/delivery-group group fix/a-module-moves-only-when-its-program-changes rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-14 21:30:09 +02:00
jschoubben 6acce60f8b List the store test of a walk whose every build came out the same (hq issues 611, 721)
mesh/merge-gate head 6acce60f8bd3 merged into trunk c2ec0a601acb — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 6acce60f8bd3 merged into trunk c2ec0a601acb — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-14 21:23:46 +02:00
jschoubben 4390589c61 Leave out a dependent of a base that came out the same, and end a walk with every build the same as done (ADR 0267 rule 3; hq issues 607, 611, 721)
mesh/merge-gate head 4390589c61a4 merged into trunk c2ec0a601acb — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 4390589c61a4 merged into trunk c2ec0a601acb — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-14 21:19:14 +02:00
jschoubben e6a28a2d28 Say why the walk moves every module when the checks' builds cannot be read (lint, hq issue 721)
mesh/delivery-group group fix/a-module-moves-only-when-its-program-changes checking: 1 of 2 member(s) ready
mesh/merge-gate head e6a28a2d2814 merged into trunk c2ec0a601acb — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head e6a28a2d2814 merged into trunk c2ec0a601acb — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-14 21:15:24 +02:00
jschoubben 32bd8b5d5f Leave a module out of a walk when its check built it the same as what runs (hq issue 721)
mesh/merge-gate head 32bd8b5d5fb2 merged into trunk c2ec0a601acb — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 32bd8b5d5fb2 merged into trunk c2ec0a601acb — fail: its merge-check.sh failed: --- FAIL: TestNoReaderAnswersEmptyForAnError (2.20s)
mesh/delivery-group group fix/a-module-moves-only-when-its-program-changes rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
A one-sentence change in a package mesh-builder imports but never reaches moved
build-agent. A merge check now compares each module it builds with the build the
mesh holds, the whole image without file times, labels or creation time, and a
walk is cut without a module every check of its pull requests found the same,
while the mesh still holds that build. A comparison that fails moves the module
as before.
2026-10-14 21:07:25 +02:00
mesh-admin c2ec0a601a Merge pull request 'A one-machine walk says its report was its run (hq issue 720)' (#335) from fix/720-a-one-machine-walk-says-its-report into main 2026-10-14 18:59:22 +00:00
jschoubben 34050dc6a3 Say a one-machine walk's report was its run: no gate reads it since ADR 0360 (hq issue 720)
mesh/merge-gate head 34050dc6a3c5 merged into trunk a5500f732f77 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 34050dc6a3c5 merged into trunk a5500f732f77 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-14 20:55:04 +02:00
mesh-admin a5500f732f Merge pull request 'Run the slow test packages in shards and build judges from a warm Go cache (hq issue 717)' (#334) from fix/717-the-suite-runs-in-under-two-minutes into main 2026-10-14 18:46:47 +00:00
jschoubben 3ec4473b29 Run the slow test packages in shards and stop compiling the standard library per judge test (hq issue 717)
mesh/merge-gate head 3ec4473b29b2 merged into trunk df8c1ccf1fff — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 3ec4473b29b2 merged into trunk df8c1ccf1fff — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery delivered
The controller's check spent 3-5 minutes in cmd/mesh-controller, whose few hundred store tests run one
after another, while most packages end in seconds. merge-check.sh now compiles each slow package once and
deals its tests round-robin into processes that run beside each other and beside the rest of the suite;
every test still runs once, under the race detector, through its package's TestMain. The listed store
tests run beside the suite instead of after it.

internal/builder's judge tests ran a real go build with a fresh Go cache each, compiling the standard
library every time: half the suite's CPU. The fake runtime fills an empty cache from one warmed per
process. The lease tests run in parallel. The link test of slow work now sets its one-second window after
the serving loop asserted the controller's thirty-second one, so it proves the heartbeat (it passed
without it) in half the time.
2026-10-14 20:38:35 +02:00
mesh-admin df8c1ccf1f Merge pull request 'Build a check's builds where the host's engine sees them, a tier at once and beside the tests (hq issues 713, 716)' (#333) from fix/713-the-walk-reuses-its-checks-builds into main 2026-10-14 18:36:18 +00:00
jschoubben bbee20adf9 Build a merge's modules of a tier at once, and beside the check's tests where the judge cache allows (hq issue 716)
mesh/merge-gate head bbee20adf9b9 merged into trunk 622fb1e9cbac — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head bbee20adf9b9 merged into trunk 622fb1e9cbac — pass: its merge-check.sh passed
mesh/delivery delivered
A check built its images one after another, after merge-check.sh. With a judge cache the
builds run during the tests, out of their reach; a failed check lets go of them.
2026-10-14 20:20:49 +02:00
jschoubben afed887217 Build a check's builds for its merge where the host's container engine sees them (hq issue 713)
The check built in a temporary directory of the build holder's container, which the
host's engine mounted empty: every check build failed and every walk built again.
2026-10-14 20:14:20 +02:00
mesh-admin 622fb1e9cb Merge pull request 'Judge a pull request into a release branch by that branch's controller and repositories (issue 704, ADR 0358)' (#329) from fix/704-release-branch-judge into main 2026-10-14 18:09:03 +00:00
mesh-adminandClaude Opus 5.5 5d5bdf9e32 Refuse the replays without a judge cache rather than run them from a volume
mesh/merge-gate head 5d5bdf9e32d1 merged into trunk 4af9f1147de6 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 5d5bdf9e32d1 merged into trunk 4af9f1147de6 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/704-release-branch-judge delivered: every member is delivered
A check's own volume is no place for the replays: the store replay binds a
directory it makes there into other containers by path, which the container
runtime would create on the build node's own disk (review 12308). Without the
build agent's judge cache the replays are refused and the gate says so as an
error; replays the runtime cannot start are the check's error, not the
change's; and an id that cannot name a place in the judge cache is refused
(hq issue 709, issue 704).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:58:07 +00:00
mesh-admin 4af9f1147d Merge pull request 'Send a merge's builds to every machine at once, with no first-machine gate (hq ADR 0360)' (#332) from feat/lean-rollout into main 2026-10-14 17:53:34 +00:00
jschoubben 704b21b3c5 Carry the new controller's user list to the bus when its walk ends, as the gate's pass did (hq ADR 0236 §34, ADR 0360)
mesh/merge-gate head 704b21b3c547 merged into trunk 76304d9db817 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 704b21b3c547 merged into trunk 76304d9db817 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-14 19:36:28 +02:00
jschoubben 5b455b8a24 Keep the first-machine gate behind a switch for walks begun before hq ADR 0360, and test both paths
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head 5b455b8a2438 merged into trunk 76304d9db817 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 5b455b8a2438 merged into trunk 76304d9db817 — fail: its merge-check.sh failed: --- FAIL: TestARenewalWrittenThoughItsAnswerWasLostIsSt…
2026-10-14 19:35:07 +02:00
jschoubben 0e97413875 Assert the backlog walk sends every machine at once and passes on the report (hq ADR 0360)
mesh/merge-gate head 0e974138751e merged into trunk 76304d9db817 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 0e974138751e merged into trunk 76304d9db817 — fail: its merge-check.sh failed: --- FAIL: TestAReusedCheckBuildIsRegisteredByItsDigestA…
mesh/delivery superseded: a newer head of the same pull request
2026-10-14 19:21:42 +02:00
jschoubben da8a5e9bc0 Send a merge's builds to every machine at once, with no first-machine gate (hq ADR 0360)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head da8a5e9bc0f7 merged into trunk 76304d9db817 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head da8a5e9bc0f7 merged into trunk 76304d9db817 — fail: its merge-check.sh failed: --- FAIL: TestAReusedCheckBuildIsRegisteredByItsDigestA…
The operator chose the leanest rollout: no gate, no one machine first, failures loud and fixed forward.
A walk sends every rolled-out module to every machine running it in one step, built-by orders no tier,
and a walk ends on its machines' reports. The backlog walk sends every machine at once and passes each
on its applied report.
2026-10-14 19:19:49 +02:00
mesh-adminandClaude Opus 5.5 847ffd6d12 Never run the replays from the workspace, with or without a judge cache
mesh/merge-gate out of date: judged against trunk 76304d9db817, the trunk is now 4af9f1147de6; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 76304d9db817, the trunk is now 4af9f1147de6; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
Without a judge cache the replays still ran mesh-lab's checkout in the workspace
with the container runtime's socket, so a self-judging pull request could write
a test into them (hq issue 709, review 12303). They now run from a copy in a
container volume of the check's own, made before anything of the change runs,
with a fresh home and throwaway caches, removed after the check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:34:00 +00:00
mesh-adminandClaude Opus 5.5 0214b45469 Run mesh-lab's replays where no pull request reaches
mesh/merge-gate head 0214b45469ec merged into trunk 76304d9db817 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 0214b45469ec merged into trunk 76304d9db817 — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The replays hold the container runtime's socket, yet took their HOME, Go
environment, build cache and module cache from the workspace a pull request's
own script writes, so a planted -toolexec hook ran as root on the build node
(hq issue 709). They now run from a copy of mesh-lab made before anything of
the change runs, with a fresh home and the judge cache's caches, as a judge is
built (ADR 0358 §6).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:15:19 +00:00
mesh-adminandClaude Opus 5.5 2bee290eea Keep judge builds warm in the build agent's judge cache, and leave nothing on the host
mesh/delivery-group group fix/704-release-branch-judge rejected: its order contradicts itself: novox/mesh-catalog@47ef2644aca1 and novox/mesh-controller@2bee29…
mesh/merge-gate head 2bee290eea56 merged into trunk 76304d9db817 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 2bee290eea56 merged into trunk 76304d9db817 — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
Review of #329: a judge's home made with a temporary directory inside the build
agent is resolved on the host by the container runtime, leaving a cold cache
behind on every check, and a kept judge was lost on every redeploy. Judge builds
now use the judge cache the build agent declares (MESH_JUDGE_CACHE), mounted into
judge builds only: one warm Go cache, the kept judges, and a fresh home per check
removed after it (hq ADR 0358 §6, issue 704, issue 482).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:53:05 +00:00
mesh-admin 76304d9db8 Merge pull request 'Keep the controller lease through a stalled renewal until its TTL passes (hq issue 477)' (#331) from fix-477-lease-survives-stall into main 2026-10-14 13:35:23 +00:00
mesh-adminandClaude Opus 5.5 b7084a19e9 Build every judge where no pull request can reach, and never read mesh-lab at a release
mesh/merge-gate out of date: judged against trunk c975dbf7b053, the trunk is now 76304d9db817; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk c975dbf7b053, the trunk is now 76304d9db817; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
Review of #329: a judge built with HOME in the workspace runs git under a
configuration a pull request's script can plant, so a hook could rewrite the
judge's sources before they compile and the result be kept under an honest key
(issue 482, issue 704). Every judge now builds with its own HOME, configuration
and Go cache outside the workspace, no git configuration and no VCS stamping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:30:46 +00:00
mesh-admin c975dbf7b0 Merge pull request 'A module may declare its unhealthy urgent, with why (hq ADR 0355, issue 691)' (#330) from adr-355-urgent-unhealthy into main 2026-10-14 13:21:56 +00:00
jochen 98e8568749 Keep the controller lease through a renewal the bus did not answer, until its TTL passes
mesh/merge-gate head 98e856874913 merged into trunk c975dbf7b053 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 98e856874913 merged into trunk c975dbf7b053 — pass: its merge-check.sh passed
mesh/delivery delivered
A bus stall of a few seconds made a renewal time out, and the controller
called that a lost lease, exited and raised the urgent lease-lost alert,
though no other controller could take a key that still lived. Now only
another holder seen, or the key's TTL passed unrenewed, loses the lease;
new acts stop at half the TTL unrenewed and resume on the next renewal
(novox/hq issue 477).
2026-10-14 15:20:19 +02:00
jochen 58889235d1 Raise a module's unhealthy urgent when its manifest says why (hq ADR 0355, issue 691)
mesh/merge-gate head 58889235d163 merged into trunk 535c7641ac41 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 58889235d163 merged into trunk 535c7641ac41 — pass: its merge-check.sh passed
mesh/delivery delivered
A broken service people rely on, such as mail, was said only as a warning on
the desktop. A module may now declare unhealthy urgent with what people lose;
the module check refuses it without why or with another severity.
2026-10-14 15:13:57 +02:00
mesh-adminandClaude Opus 5.5 48b2906b5c Keep mesh-lab at its trunk and build the release judge with a cache of its own
mesh/merge-gate out of date: judged against trunk 535c7641ac41, the trunk is now c975dbf7b053; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 535c7641ac41, the trunk is now c975dbf7b053; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
Review of #329: mesh-lab's replays run with the container runtime's socket, so
a same-named branch must never be what runs there; the shared Go cache and HOME
are writable by a pull request's own script, so the judge kept for a head must
not be built from them (issue 704, issue 482). A forge answer naming no branch
is an error, and a trunk is the branch a repository follows, not a name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:13:30 +00:00
mesh-adminandClaude Opus 5.5 ea7d91bff1 Say which validator judged a release-branch check, and prove clause 5
mesh/merge-gate head ea7d91bff190 merged into trunk 535c7641ac41 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head ea7d91bff190 merged into trunk 535c7641ac41 — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
ADR 0358 version 4 asks the verdict to say whether mesh-host's release-branch
validator or the vendored one judged, and checks that a release branch's own
pull request into its trunk is judged as any trunk pull request (issue 704).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 12:56:55 +00:00
mesh-adminandClaude Opus 5.5 4a75cc51dd Judge a pull request into a release branch by that branch's controller and repositories
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head 4a75cc51dda6 merged into trunk 535c7641ac41 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 4a75cc51dda6 merged into trunk 535c7641ac41 — pass: its merge-check.sh passed
A release branch could never carry a change that needs another repository's
release-branch change: its pull requests were judged by the running controller
and against the other repositories' main (hq issue 704, ADR 0358).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 12:48:44 +00:00
mesh-admin 535c7641ac Merge pull request 'A fresh check keeps a build whose key is unchanged instead of building it again (hq issue 703)' (#328) from fix/703-a-fresh-check-keeps-an-unchanged-build into main 2026-10-14 08:15:56 +00:00
jschoubben c707bf18d8 Say which check made a build the walk does not take, and test the merged heads
mesh/merge-gate head c707bf18d8bf merged into trunk 04730b54a8f2 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head c707bf18d8bf merged into trunk 04730b54a8f2 — pass: its merge-check.sh passed
mesh/delivery delivered
A build of another pull request or of a head the forge did not merge is
in no verdict the walk is handed; the walk now names the check that made
it, so its "built:" line says which head it was (novox/hq issue 703).
2026-10-14 10:05:31 +02:00
jschoubben 060f2a9ca5 Build for the merge from a copy its tests never touch, and keep or take only the same head's build
mesh/merge-gate head 060f2a9ca5fd merged into trunk 04730b54a8f2 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 060f2a9ca5fd merged into trunk 04730b54a8f2 — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The gate and the repository's check mount the workspace read-write, so a
build from that tree could hold what the tests wrote under a key that
describes the merge. A check now clones the merge outside the workspace
before anything of the change runs, and builds from that. A check keeps,
and a walk takes, only a build an earlier check of the same pull request
at the same head made: another pull request's build, or an earlier
head's, is never registered. The walk reads the verdicts once
(novox/hq issue 703, review of #328).
2026-10-14 09:56:23 +02:00
jschoubben 1666747a22 Keep a check build whose key is unchanged instead of building it again
mesh/merge-gate head 1666747a2246 merged into trunk 04730b54a8f2 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 1666747a2246 merged into trunk 04730b54a8f2 — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A fresh check asked when only the trunk moved rebuilt every image, though
the reuse key said nothing it is built from changed: a controller check
took about 12 minutes instead of 5.5. The check now keeps a build kept
under its key when a verdict of the build seat names that key and every
digest, names it as the build's own check so a walk takes it, and the
cleanup keeps it while any check names it (novox/hq issue 703).
2026-10-14 09:41:39 +02:00
mesh-admin 04730b54a8 Merge pull request 'S9 raises every bus user's slow consumer and closed connection, from the bus module's events (hq issue 270, ADR 0357)' (#326) from fix/270-the-bus-module-says-every-users-slow-consumer into main 2026-10-14 07:14:33 +00:00
jochen 45870ab31c Bound the read of bus-module events, keep S9 seeing when it fails, count the controller's own slow consumer once (hq issue 270, review of #326)
mesh/merge-gate head 45870ab31cc5 merged into trunk 3479a35462b2 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 45870ab31cc5 merged into trunk 3479a35462b2 — pass: its merge-check.sh passed
mesh/delivery delivered
A storm of closures must not spend a watchdog tick, so the read stops at 2000
events and says the count is a floor. A failed read is its own condition
rather than blinding S9, so the controller's own signals are still said. The
controller's fall behind, told to its client and closed by the server, is one
fault, counted once.
2026-10-14 09:02:47 +02:00
jochen fd9308d328 Raise every bus user's slow consumer and closed connection as S9, from the bus module's events (hq issue 270, ADR 0357)
mesh/delivery-group group fix/270-the-bus-module-says-every-users-slow-consumer rejected: a member's own check failed
mesh/merge-gate head fd9308d3287e merged into trunk 3479a35462b2 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head fd9308d3287e merged into trunk 3479a35462b2 — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The controller heard a slow consumer only on its own connection: the server
says another user's only in its record of closed connections. The bus module
now says that record as events; the watchdogs read the last hour back on every
tick and raise one condition per user and kind, naming the module and machine.
The module check lets the bus module emit, since its tool runner publishes.
2026-10-14 08:41:47 +02:00
mesh-admin 3479a35462 Merge pull request 'List the provider-hold tests among the store tests (hq issue 145, to-be 48 Phase C)' (#319) from fix/145-a-consumer-is-held-under-its-provider into main
mesh/merge-gate head 3479a35462b2 merged into trunk 221fe5a60d1f — fail: head 3479a35462b2 does not merge cleanly into trunk 221fe5a60d1f: in conflict test…
mesh/repo-check head 3479a35462b2 merged into trunk 221fe5a60d1f — fail: head 3479a35462b2 does not merge cleanly into trunk 221fe5a60d1f: in conflict test…
mesh/delivery stopped: the pull request closed unmerged
2026-10-14 05:18:00 +00:00
jochen cebc593627 List the provider-hold tests among the store tests, so a skip cannot pass unseen (hq issue 145)
mesh/merge-gate head cebc593627d3 merged into trunk 2959c8143536 — pass: builds build-agent → ace, g14, novox, shanks; no bus step; every machine composes …
mesh/repo-check head cebc593627d3 merged into trunk 2959c8143536 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/145-a-consumer-is-held-under-its-provider delivered: every member is delivered
The rule 5 test of ADR 0240 (one provider down, three consumers, one
condition) and its healthy-provider twin need the store and skip without
one; unlisted, the merge check passed them without running them.
2026-10-14 03:21:50 +02:00
mesh-admin 2959c81435 Merge pull request 'Link a build's bases into its repository before pushing it (issue 694)' (#318) from fix/694-a-push-holds-its-bases into main 2026-10-14 01:19:24 +00:00
jochen 8933864a49 Link a build's bases into its repository before pushing it (issue 694)
mesh/merge-gate head 8933864a49d0 merged into trunk df9e5728a925 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 8933864a49d0 merged into trunk df9e5728a925 — pass: its merge-check.sh passed
mesh/delivery delivered
A runtime on containerd's image store, the default on a fresh docker 29,
need not hold the compressed layers of a base it built FROM: a layer it
had unpacked already is never fetched. Its push then fails on that layer
although the registry holds it under the base's mirror. A push reads
locally only what the target repository lacks, so mount the base's blobs
there first.
2026-10-14 02:59:53 +02:00
mesh-admin df9e5728a9 Merge pull request 'Try a pushed branch on the build seat before its pull request (hq issue 686)' (#314) from fix/686-a-branch-is-tested-before-its-pull-request into main 2026-10-14 00:57:46 +00:00
mesh-admin 2d18f438c5 Merge pull request 'A no-move tier waits for no new report, so its walk asks the next tier (issue 648)' (#309) from fix/648-a-no-move-tier-waits-for-nothing into main 2026-10-14 00:32:47 +00:00
jschoubben 3ce5a0239b Keep a try off the forge and bound how many run (hq issue 686)
mesh/merge-gate head 3ce5a0239bb1 merged into trunk 2d18f438c516 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 3ce5a0239bb1 merged into trunk 2d18f438c516 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery delivered
Review of #314: a try's start named its commit, which the forge's holder
matches to an open pull request's head and marks pending. Its start now
names no repository or commit. One try per commit and four in all, a try
never ages ahead of deliveries or pull request checks, and its id is
checked before it is a bus subject.
2026-10-14 02:29:28 +02:00
jochen 70885ab3c4 Share one prefix between the no-move Why's writer and its reader (review of #309, issue 648)
mesh/delivery delivered
mesh/merge-gate head 70885ab3c4a3 merged into trunk ca3f43c400ba — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 70885ab3c4a3 merged into trunk ca3f43c400ba — pass: its merge-check.sh passed
2026-10-14 02:22:25 +02:00
jschoubben 65508d9df9 Try a pushed branch on the build seat before its pull request (hq issue 686)
mesh/merge-gate head 65508d9df9f4 merged into trunk ca3f43c400ba — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 65508d9df9f4 merged into trunk ca3f43c400ba — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery superseded: a newer head of the same pull request
Store tests skip without a store and no agent may start one, so each failure
cost a 15-30 minute review round. The try verb asks the build seat for the
repository's merge-check.sh on the ref merged into the trunk, as a check in
the build queue, and reads its outcome from the build's log: no status on
the forge, nothing registered, nothing built for a merge.
2026-10-14 02:20:37 +02:00
jschoubben 8123ca5654 Test that a branch is tried before its pull request (hq issue 686)
A builder's store tests first ran in the pull request's merge check, a review
round late. These fail on main: a try asks the repository's own check alone,
its outcome sets no status and registers nothing, an unknown ref is refused
before anything is asked, and the answer carries the failing lines.
2026-10-14 02:20:37 +02:00
jochen d9b86376d3 A no-move tier waits for no new report, so its walk asks the next tier (issue 648)
A build made from the source every machine running it runs is marked sent
and sent nothing, yet the tier's wait asked each machine for a report after
that marking. None follows, and the walk waited for ever. Each machine is
now judged by whether it answered its last send, the one whose builds made
the build no move.
2026-10-14 02:18:53 +02:00
mesh-admin ca3f43c400 Merge pull request 'The rest go at once after the first passes, and a gate survives a new controller (hq ADR 0353)' (#312) from fix/the-rest-in-parallel-and-gates-survive-a-new-controller into main 2026-10-13 23:43:02 +00:00
jschoubben b482122d66 Say a read error of a silent rest, and keep the node-engine one at a time in a kept step (hq ADR 0353)
mesh/delivery delivered
mesh/merge-gate head b482122d6646 merged into trunk 1601cb17432e — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head b482122d6646 merged into trunk 1601cb17432e — pass: its merge-check.sh passed
An empty answer on a read error would clear release-silent unseen.
2026-10-14 01:26:00 +02:00
jschoubben 2f5cc146fe Keep a walk's sends to what was kept, and say a silent rest (hq ADR 0353 review)
mesh/merge-gate head 2f5cc146fe46 merged into trunk 1601cb17432e — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 2f5cc146fe46 merged into trunk 1601cb17432e — fail: its merge-check.sh failed: --- FAIL: TestAFirstMachineThatFailsStillStopsTheWalk (…
mesh/delivery superseded: a newer head of the same pull request
A release's own gates held its other sends back for ever; an adopted send
lost what the machine ran before; a kept send could grow a build no gate
passed; a take-over matched by commit alone; a rest that went silent ended
done. The node-engine and the resolver still go one machine at a time, since
a fault that cuts a machine off shows only as silence.
2026-10-14 01:18:51 +02:00
jschoubben c2ea9b4c51 Send the rest at once after the first passes, and resume a stored gate (hq ADR 0353)
mesh/repo-check head c2ea9b4c5187 merged into trunk 1601cb17432e — fail: its merge-check.sh failed: --- FAIL: TestAReleaseSendsTheRestAtOnceAfterTheFirstPa…
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head c2ea9b4c5187 merged into trunk 1601cb17432e — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
A release now sends every other machine in one step once its first machine
passed, each judged by its own gate; a failing one is put back without
stopping the others, a silent one is left. A send is kept before it is made,
and a machine already sent what a gate is for is not sent again: the gate
counts from that send and takes over another walk's readings of the same
build there.
2026-10-14 01:06:49 +02:00
jschoubben b03d257f81 Test that the rest go at once after the first passes and a gate survives a new controller (hq ADR 0353)
The release walked four machines one after another in eleven minutes, and a
new controller re-sent and re-judged a build its machine already ran.
2026-10-14 01:06:49 +02:00
mesh-admin 1601cb1743 Merge pull request 'A walk's first-machine gate waits for a busy container, and mesh-builder busy drains the agent (issue 594, hq ADR 0335)' (#283) from feat/335-a-busy-container-is-replaced-when-idle into main 2026-10-13 22:20:55 +00:00
jschoubben 873b19566a Read the delivery group's order again, now the catalogue and hq left it
mesh/delivery delivered
mesh/merge-gate head 873b19566a82 merged into trunk 72214ae989b3 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 873b19566a82 merged into trunk 72214ae989b3 — pass: its merge-check.sh passed
mesh/delivery-group group feat/335-a-busy-container-is-replaced-when-idle delivered: every member is delivered
2026-10-14 00:08:44 +02:00
jschoubben b0753e2aac Let a drain lapse, bound the gate's wait on the controller's clock, keep a busy wait out of apply time
mesh/delivery superseded: a newer head of the same pull request
mesh/delivery-group group feat/335-a-busy-container-is-replaced-when-idle checking: 1 of 2 member(s) ready
mesh/merge-gate head b0753e2aac95 merged into trunk 72214ae989b3 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head b0753e2aac95 merged into trunk 72214ae989b3 — pass: its merge-check.sh passed
Review of #283: a drain whose replacement was withdrawn held the agent for
ever with its health green; the gate trusted the engine's until; a machine
of the rest waiting at the bound read as silent and its apply time held the
wait; an unknown mesh-builder word exited 0; plan said nothing of a busy an
older engine is not sent; and a node with a report and no send broke the
reading of every last report (the store test's NULL).
2026-10-14 00:07:31 +02:00
jschoubben a69c0e50dc Merge origin/main into feat/335, and pin mesh-host at #98's new head 2026-10-14 00:01:21 +02:00
mesh-admin 72214ae989 Merge pull request 'One build per change: the check builds what its merge moves, the walk takes it by key (ADR 0350, issue 649)' (#311) from feat/350-one-build-per-change into main 2026-10-13 21:30:25 +00:00
jschoubben 5c606fd1d4 Take only a check build its verdict names, and keep a build another check still names
mesh/merge-gate head 5c606fd1d48b merged into trunk 372faebf2824 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 5c606fd1d48b merged into trunk 372faebf2824 — pass: its merge-check.sh passed (run 2)
mesh/delivery delivered
mesh/delivery-group group feat/350-one-build-per-change delivered: every member is delivered
The artifact store takes writes from anything on its network, so a record
planted under another merge's key could have been registered: the walk now
takes a scratch build only when the build seat's own verdict for one of its
pull requests names its key and digests. Deleting by digest also took builds
another check shared (review of #311; novox/hq ADR 0350).
2026-10-13 23:16:55 +02:00
mesh-admin 372faebf28 Merge pull request 'The controller passes its first-node gate on one reading (ADR 0351)' (#310) from feat/351-the-controller-passes-on-one-reading into main 2026-10-13 21:14:27 +00:00
jschoubben 675466a8c3 Hold the store tests' modules as the settle time did, and fault a dip after any healthy reading
mesh/delivery delivered
mesh/merge-gate head 675466a8c35f merged into trunk bff5b7f0e4f8 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 675466a8c35f merged into trunk bff5b7f0e4f8 — pass: its merge-check.sh passed
Two tier tests relied on the two-minute settle to keep a module unjudged; the
witness window and the gate's spacing now hold it. A healthy reading taken
before the controller's floor counted no pass, so a dip after it did not take
the two-minute pass away (novox/hq ADR 0352 decision 3).
2026-10-13 23:07:15 +02:00
jschoubben cd49b50c40 Build a change once: the check builds what its merge moves, the walk takes it by key
mesh/merge-gate out of date: judged against trunk bff5b7f0e4f8, the trunk is now 372faebf2824; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk bff5b7f0e4f8, the trunk is now 372faebf2824; judged again on a recheck or when it is merged
mesh/delivery-group group feat/350-one-build-per-change checking: composed check out of date: novox/mesh-controller@cd49b50c4075 was judged against trunk bff5b…
mesh/delivery superseded: a newer head of the same pull request
A merge built every moved module a second time although its check had judged
the same tree (issue 649). The check now builds into the scratch namespace
under a reuse key (source fingerprint plus the builder's own digest), and the
walk of the trunk commit copies a build of the same key by digest instead of
building, saying per module why when it cannot (novox/hq ADR 0350).
2026-10-13 23:04:40 +02:00
jschoubben 41eefc2f8c Test that a healthy node-engine passes at two minutes on one reading
mesh/merge-gate head 41eefc2f8c3e merged into trunk bff5b7f0e4f8 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 41eefc2f8c3e merged into trunk bff5b7f0e4f8 — fail: its merge-check.sh failed: --- FAIL: TestACoreBehindWhileTheNodeToolsSettleFailsNo…
mesh/delivery superseded: a newer head of the same pull request
2026-10-13 22:58:37 +02:00
jschoubben c3946e5d72 Pass the node-engine and resolver on their third reading, at most two minutes
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head c3946e5d72c2 merged into trunk bff5b7f0e4f8 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head c3946e5d72c2 merged into trunk bff5b7f0e4f8 — fail: its merge-check.sh failed: --- FAIL: TestACoreBehindWhileTheNodeToolsSettleFailsNo…
The operator wants these gates as fast as they can be: counting readings only after
two minutes added about 80 s to every node-engine rollout for no safety. Count
from the send, pass on the third healthy reading, and pass a module healthy at
every reading at two minutes on what it has (novox/hq ADR 0352).
2026-10-13 22:55:46 +02:00
jschoubben 524295926f Pass the controller's first-node gate on one reading after the witness window
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head 524295926f76 merged into trunk bff5b7f0e4f8 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 524295926f76 merged into trunk bff5b7f0e4f8 — fail: its merge-check.sh failed: --- FAIL: TestACoreBehindWhileTheNodeToolsSettleFailsNo…
The controller runs on one machine, so three readings over two minutes guarded no
other machine and held every walk behind it. Judge each module by its own rule:
one reading for the controller and the tool runner once the node-engine's witness
window has ended, one for mesh-delivery, three for the node-engine and the
resolver with the first two minutes after the send (novox/hq ADR 0351). The
gate's verdict names the rule it applied.
2026-10-13 22:39:32 +02:00
mesh-admin bff5b7f0e4 Merge pull request 'A settings proposal reads at a glance, its values whole where the sender is proven (issues 383 and 639)' (#304) from fix/383-an-ask-reads-at-a-glance into main 2026-10-13 19:22:06 +00:00
mesh-admin cef22a2526 Merge pull request 'A provider serves its consumers its own composed names (ADR 0346, issue 624)' (#306) from feat/346-a-provider-serves-its-own-names into main 2026-10-13 19:11:16 +00:00
jochen deef9421a2 A change's line the router would refuse is named with its value withheld, or refused at propose (the review of #304)
mesh/merge-gate head deef9421a276 merged into trunk cef22a25264b — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head deef9421a276 merged into trunk cef22a25264b — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/383-an-ask-reads-at-a-glance delivered: every member is delivered
{a:1, b:2, c:3, password:hunter2} was proposed with "+ password: hunter2" in its Details: the router
withheld the line unnamed and Approve set a value the operator never saw (ADR 0347 §1, §3, §4). Each line is
now judged as the router judges it, masked under the content rule and whole under the secret rule; a line
refused is said "+ password (value withheld)", and any line the router would still refuse is refused at
propose in the controller's words, never echoing a secret-shaped key. SDK at mesh-sdk 5933eb7 (no two asks
digest the same).
2026-10-13 20:45:25 +02:00
jochen 3c19e9ee93 A settings proposal's summary says the layer is sent at once by a walk (ADR 0344 with ADR 0345)
Rebased on #305: Approve now sends the layer by a walk, so the summary's "takes it at its next push" was
false. The summary says it is set and sent at once by a walk to where it reaches, one machine at a time,
each judged; the rest of what Approve does (onApprove) leads the Details, beside every key whole.
2026-10-13 20:45:25 +02:00
jochen 3d94f87bb0 The glance's cut says only what the note holds there: whole beside the whole details, else in the note (ADR 0347) 2026-10-13 20:45:25 +02:00
jochen 4644f25edd A settings proposal carries every key with its whole value in its Details (issue 383)
The review of #304: with more than three keys the ask showed only the names of the rest, a long value was
cut without a word, and the Details held no key, so Approve bound values the operator never saw. The
Details now list every key uncut — masked, and whole in WholeDetails for a channel that proves who
answers — before who proposed it and the fingerprint; the glance says what it cut. Vendors mesh-sdk at
f474544 and captures its conformance events there.
2026-10-13 20:45:25 +02:00
jochen c58e0a571e A settings proposal reads at a glance, its values whole where the sender is proven (issues 383 and 639)
The operator could not read the settings proposals on the phone: the change, the fingerprint and a how-to
paragraph in one wall of plain text, and the values masked as ‹path› and ‹address›, so the approval was
blind.

The ask is now a summary of what changes where ("Changes 3 setting(s) of mounts on shanks; approved, the
layer is set at once and shanks takes it at its next push."), then at most three points, one key each (more
keys: the first two and the names of the rest). Who proposed it, the fingerprint and how to read it whole
are its Details, which the router notes after the answers. The change is carried twice: under the content
rule, and whole (asks.Ask.Whole), shown only on a channel that proves who answers, a value shaped like a
secret withheld in both (from the earlier work on issue 383, #202).

Vendors mesh-sdk at 9588871 (fix/639-an-ask-reads-at-a-glance: Message.Note, asks.Glance), and captures its
conformance events at that commit.
2026-10-13 20:45:25 +02:00
mesh-admin 332fbaa9b1 Merge pull request 'settings: no layer replaces a value the vault fills, at any depth (issue 610)' (#308) from fix/610-no-layer-replaces-a-vault-secret into main 2026-10-13 18:33:41 +00:00
jochen 7a0c219f96 List the provider-elsewhere store test, read a machine's domain as its profile, reuse the settings read once, and refuse a nested own name (hq ADR 0346, review 228)
mesh/merge-gate head 7a0c219f961e merged into trunk 332fbaa9b175 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 7a0c219f961e merged into trunk 332fbaa9b175 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-13 20:32:51 +02:00
jochen 985423ae61 Say that a provider's own name is not on its consumers' entries (hq ADR 0346) 2026-10-13 20:29:17 +02:00
jochen cfacda2625 Let a provider serve its consumers its own composed names (hq ADR 0346, issue 624)
The forge writes every package tarball at its public name, and its
consumers had no source for that name but a second copy kept by hand,
though the mesh composes it for the forge's route. A served value may now
name the provider's own names; a name its machine does not compose leaves
the key out, and the plan says so.
2026-10-13 20:29:17 +02:00
mesh-admin bf6beec465 settings: no layer replaces a value the vault fills, at any depth (issue 610)
mesh/merge-gate head bf6beec465c2 merged into trunk 9f3aa91e812b — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head bf6beec465c2 merged into trunk 9f3aa91e812b — pass: its merge-check.sh passed
mesh/delivery delivered
A mergeable file's protected keys were checked at the top level only, so a layer could still set searxng's server.secret_key under the server key its file takes. The merge now refuses any layer value that would replace a ${secret:} placeholder in the file's own content, at any depth, whatever the module's protected list says.
2026-10-13 17:43:24 +00:00
mesh-admin 9f3aa91e81 Merge pull request 'List the settings walk's store tests, and drop a refusal check that never matched (issue 641 review)' (#307) from fix/641-list-the-settings-store-tests into main 2026-10-13 17:35:00 +00:00
jochen 4bdcef3f73 List the settings walk's store tests, and drop a refusal check that never matched (issue 641 review)
mesh/merge-gate head 4bdcef3f73b5 merged into trunk 8ddf8b700558 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 4bdcef3f73b5 merged into trunk 8ddf8b700558 — pass: its merge-check.sh passed
mesh/delivery delivered
A store test not named in testdata/store-tests skips unseen without a store
(issue 459). The check after the send could not match: the send turns its
refusals into words; the composition before it already fails the walk.
2026-10-13 19:24:02 +02:00
mesh-admin 8ddf8b7005 Merge pull request 'Send an approved settings layer by a walk at once, so the operator's Approve takes effect (issue 641, ADR 0344)' (#305) from fix/641-an-approved-setting-is-sent into main 2026-10-13 17:22:20 +00:00
jochen 8bbb1b19ae A settings gate keeps the module's build, and a walk that cannot keep a build fails (issue 641 review)
mesh/merge-gate head 8bbb1b19ae63 merged into trunk 6f1a05366c1d — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 8bbb1b19ae63 merged into trunk 6f1a05366c1d — pass: its merge-check.sh passed
mesh/delivery delivered
Without a build on the gate, a merge walk sending the machine a new build of
the module was read as the settings send's own report. A refusal for a last
send not known, or a kept build gone, never clears, so retrying it every
tick held the walk of waiting builds; it now fails the walk with why. Every
failure names the machines passed over as away (ADR 0344 §6).
2026-10-13 17:33:07 +02:00
jochen e32ca00aed Test that a settings send moves no ungated build waiting on its machine (issue 641, ADR 0344 §3)
mesh/merge-gate head e32ca00aed9d merged into trunk 6f1a05366c1d — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head e32ca00aed9d merged into trunk 6f1a05366c1d — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-13 16:55:49 +02:00
jochen 1219ec8229 A settings walk sends the layer alone, every build kept, and stops before a data move (issue 641, ADR 0344)
mesh/merge-gate head 1219ec822943 merged into trunk 6f1a05366c1d — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 1219ec822943 merged into trunk 6f1a05366c1d — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A judged send carries what waits on its machine, which would have released
builds held after a failed walk without a person's word. The settings walk
now keeps every build of its machine, refuses a send that would move a
running module's data, and never says delivered when it reached no machine.
2026-10-13 16:50:10 +02:00
jochen e324f5d821 Send an approved settings layer by a walk at once, so the operator's Approve is the act that takes effect (issue 641, ADR 0344)
mesh/merge-gate head e324f5d82162 merged into trunk 6f1a05366c1d — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head e324f5d82162 merged into trunk 6f1a05366c1d — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The warrant set the layer and left the push as a second act nobody took: the
operator approved a betterbird change on Telegram and nothing reached g14.
Now the warrant opens a settings walk: the machines the layer reaches, one at
a time, the module judged on each at its gate, its end told once through
plan-moved. The ask and the proposer's answer say so instead of naming push.
2026-10-13 16:39:47 +02:00
mesh-admin 6f1a05366c Merge pull request 'Admit a port from the machine's own local network, and never a public subnet (issue 154, ADR 0340)' (#299) from fix/154-local-network into main 2026-10-13 13:07:11 +00:00
mesh-admin 0a1dc5f93f Merge pull request 'seat: a push to a named machine waits for the machine's answer, so its call says how it went (issue 629)' (#301) from fix/629-a-push-says-how-it-went into main 2026-10-13 12:44:51 +00:00
mesh-admin ae0b384497 push-wait is untrusted, so it is set in one call like any preference (issue 629)
mesh/merge-gate head ae0b384497c2 merged into trunk f3875a6b6019 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head ae0b384497c2 merged into trunk f3875a6b6019 — pass: its merge-check.sh passed
mesh/delivery delivered
The gate warned that the push file asks for a setting without saying whether it is trusted, which makes push-wait terminal-only.
2026-10-13 12:38:31 +00:00
mesh-admin f3875a6b60 Merge pull request 'catalogue: say in the comments that a package may be declared by several modules (issue 628)' (#300) from fix/628-packages-are-shared into main 2026-10-13 12:38:30 +00:00
mesh-admin f09c5ebdec push: the wait is the call's or the controller's setting push-wait, never a length in the code (issue 629)
mesh/merge-gate head f09c5ebdec6c merged into trunk 584aa2d7d7a5 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head f09c5ebdec6c merged into trunk 584aa2d7d7a5 — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The operator's rule: a wait is never hard-coded, at most a default is configured. The push verb takes `wait`; without it the controller's setting `push-wait` (module default 2m) applies; a setting that cannot be read waits for nothing, as a push did before, and says so.
2026-10-13 12:09:43 +00:00
mesh-admin 6d2e4c299a seat: a push to a named machine waits for the machine's answer, so its call says how it went (issue 629)
mesh/merge-gate head 6d2e4c299a34 merged into trunk 584aa2d7d7a5 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 6d2e4c299a34 merged into trunk 584aa2d7d7a5 — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The seat ran `push <node> --wait 0`, so a push's call ended at "sent" and nobody learned whether the machine applied it. The caller is answered before a push starts (answersFirst, issue 265), so waiting holds nobody: the call now waits up to two minutes for the machine's own report and keeps it, applied, refused with the machine's reason, or each failed resource with its error.
2026-10-13 12:03:53 +00:00
mesh-admin b654b320a1 catalogue: say in the comments that a package may be declared by several modules (issue 628)
mesh/merge-gate head b654b320a18a merged into trunk 584aa2d7d7a5 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head b654b320a18a merged into trunk 584aa2d7d7a5 — pass: its merge-check.sh passed
mesh/delivery delivered
The resolver follows ADR 0303, but four comments still stated ADR 0210's retired rule that two modules declaring one package collide, and an agent took them as the rule. What stays exclusive is what a module owns: a path, a unit, a name, and a package declared present by one module and absent by another.
2026-10-13 11:52:17 +00:00
jochen 4cc8588afb Test that a global IPv6 subnet is public and never counts as the local network (issue 154, ADR 0340)
mesh/merge-gate head 4cc8588afb87 merged into trunk 0a1dc5f93fcb — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 4cc8588afb87 merged into trunk 0a1dc5f93fcb — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/154-local-network delivered: every member is delivered
The review asked for the IPv6 side of the rule that local-network cannot open a port to the internet.
2026-10-13 13:20:50 +02:00
mesh-admin 584aa2d7d7 Merge pull request 'Never make a credential only a person can give, and give it once per provider (hq issue 164, ADR 0337)' (#298) from fix/164-an-accepted-credential-is-never-minted into main 2026-10-13 09:52:07 +00:00
jochen c9dd07f919 Never make a credential only a person can give, and give it once per provider (hq issue 164, ADR 0337)
mesh/merge-gate head c9dd07f919a2 merged into trunk 404b4409e70e — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head c9dd07f919a2 merged into trunk 404b4409e70e — pass: its merge-check.sh passed (run 2)
mesh/delivery delivered
A Servarr app's API key, an indexer's key and a downloader's password are values the provider's
software holds; the pair path made a random value for every consumer of them, sealed it to both ends
and reported success, and the provider refused it. An offer may now say `"credential": {"issued-by":
"outside"}`: SecretFor never makes a value for it, the plan reads such a pair as given or not given
and sends the consumer no credential file until a person gives it, without refusing its machine;
remakeShared and SharedSecretFor never make a provider's own secret issued outside either.
`secret accept <provider's node> <provider module> <provision>` seals one value to every consumer that
binds the provision from that provider, the provider and the operator, under one generation; the
per-pair form is refused for such a provision, and rotate names the accept. A consumer may wait for a
provision's credential: the controller excuses the wait only when the store bears it out, and its
needs-operator condition names the one accept line.

Adds the replay of issue 164 (TestReplay164), written with only what the controller had before.
2026-10-13 11:37:31 +02:00
mesh-admin 404b4409e7 Merge pull request 'A built module keeps its build edges: a new SDK plans the toolchain and its TypeScript bundles (hq issue 614)' (#296) from fix/614-a-built-module-keeps-its-build-edges into main 2026-10-13 09:35:26 +00:00
jochen 6e6cd5e26d Admit a port from the machine's own local network, and never a public subnet (issue 154, ADR 0340)
mesh/delivery-group group fix/154-local-network checking: 0 of 2 member(s) ready
mesh/merge-gate out of date: judged against trunk 1826b68a22b5, the trunk is now 404b4409e70e; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 1826b68a22b5, the trunk is now 404b4409e70e; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
A local service could only be declared public, which says the internet. local-network is a source and a reach: the private network and the private networks the machine reported on a link facing outside, judged here, shown by plan and node, and said in the rule set when there are none.
2026-10-13 11:20:22 +02:00
jochen e658a14296 Read a built module's build edges from what is kept of it, so a new SDK rebuilds its toolchain and bundles (hq issue 614)
mesh/merge-gate head e658a14296f9 merged into trunk 1826b68a22b5 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head e658a14296f9 merged into trunk 1826b68a22b5 — pass: its merge-check.sh passed
mesh/delivery delivered
A registered manifest carries no build, so mesh-tools' build.on naming the
SDK's package and every bundle's toolchain were edges only before a module was
first built: mesh-sdk 0.1.14 planned "sent nowhere". The relation now reads a
package base from the recorded name@version against the package the mesh holds,
and the toolchain from the bundles the resolved manifest keeps. Only records the
mesh already keeps are read, so it holds for every module built before it and a
rollback reads the same records. Replay R614.
2026-10-13 11:08:17 +02:00
mesh-admin 1826b68a22 Merge pull request 'Say which modules a walk left unchanged, so a merge that moved nothing is not read as stranded (hq issue 611)' (#295) from fix/611-an-unchanged-build-is-delivered into main 2026-10-13 08:25:45 +00:00
jochen 05576db018 Say which modules a walk left unchanged, so a merge that moved nothing is not read as stranded (hq issue 611)
mesh/merge-gate head 05576db018a4 merged into trunk a688dcff394b — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 05576db018a4 merged into trunk a688dcff394b — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/611-an-unchanged-build-is-delivered delivered: every member is delivered
A delivery plan made while another walk built a module reads it whole and names it; the walk, cut later,
finds none of the changed files in the build source that build recorded and moves nothing. Its record named
no module, so mesh-delivery read 'matched what runs' as 'never built', stopped the delivery and asked the
operator about a change already on every machine. The walk now says each such module in delivery.unchanged,
with why; its modules keep meaning what it builds.
2026-10-13 08:18:19 +02:00
mesh-admin a688dcff39 Merge pull request 'Add the replay of issue 239: a module's name is not taken over by another repository (hq issue 239)' (#294) from fix/239-replay into main 2026-10-13 05:37:08 +00:00
mesh-admin 8f1aec0832 Merge pull request 'Add the replay of issue 363: a README-only merge made a new digest of an unchanged Go program (hq issue 363)' (#293) from fix/363-replay into main 2026-10-13 05:08:13 +00:00
jochen 95556b8def Add the replay of issue 239: a module's name is not taken over by a build from another repository
mesh/merge-gate head 95556b8def3f merged into trunk 8f1aec0832b1 — pass: the change touches no module of the mesh's graph
mesh/repo-check head 95556b8def3f merged into trunk 8f1aec0832b1 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/239-replay delivered: every member is delivered
The fix (#174, hq ADR 0266 §7) refuses an outcome built from a repository other than the module's
registered one; nothing replayed the photo app's outage that showed why. The replay uses only what
the controller had before the fix, so the prover can show it fails there and passes on the fix.
2026-10-13 07:02:05 +02:00
jochen 94c928b1f5 Add the replay of issue 363: a README-only merge made a new digest of an unchanged Go program (hq issue 363)
mesh/merge-gate head 94c928b1f5af merged into trunk cd6757f1e3c8 — pass: builds build-agent, route-proxy → ace, g14, novox, shanks; no bus step; every mach…
mesh/repo-check head 94c928b1f5af merged into trunk cd6757f1e3c8 — pass: its merge-check.sh passed
mesh/delivery covered after its stop: a later merge that contains it was delivered: novox/mesh-controller@e658a14296f9 (merged as 404b4409 into main, wal…
mesh/delivery-group group fix/363-replay delivered: every member is delivered
2026-10-13 07:00:44 +02:00
mesh-admin cd6757f1e3 Merge pull request 'Stamp no commit into a Go bundle, so one build source is one digest (hq issue 363, ADR 0267 rule 6)' (#292) from fix/363-a-go-bundle-does-not-vary-with-its-commit into main 2026-10-13 04:59:09 +00:00
jochen ef3e5ae9b0 Stamp no commit in this repository's own image recipes either (hq issue 363, review of #292)
mesh/merge-gate head ef3e5ae9b0dc merged into trunk 1d244d5aa24e — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head ef3e5ae9b0dc merged into trunk 1d244d5aa24e — pass: its merge-check.sh passed
mesh/delivery delivered
The genesis image copies the whole clone with .git, so its controller
carried the commit it was built at, as the bundles did.
2026-10-13 06:47:18 +02:00
mesh-admin 1d244d5aa2 Merge pull request 'Add the replay of issue 348, and refuse bind-dynamic now the catalogue binds once (hq issue 348)' (#291) from fix/348-replay into main 2026-10-13 04:39:40 +00:00
jochen bcdb1ad4f8 Stamp no commit into a Go bundle, so one build source is one digest (hq issue 363, ADR 0267 rule 6)
mesh/merge-gate out of date: judged against trunk e0d69ab7a05b, the trunk is now 1d244d5aa24e; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk e0d69ab7a05b, the trunk is now 1d244d5aa24e; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
A module built from its repository's root is compiled with the clone's .git
in view, and Go stamped the commit into the binary: a merge that changed only
a README made a new digest of an unchanged program.
2026-10-13 06:29:44 +02:00
mesh-admin e0d69ab7a0 Merge pull request 'mesh-builder: read the genesis document until the registry binds, and refuse a registry build with neither (hq issue 121, ADR 0336)' (#289) from fix/121-genesis-package-registry into main 2026-10-13 04:22:05 +00:00
jochen 6eea92c323 Add the replay of issue 348, and refuse bind-dynamic now the catalogue binds once (hq issue 348)
mesh/merge-gate head 6eea92c323f4 merged into trunk e0d69ab7a05b — pass: builds build-agent → ace, g14, novox, shanks; no bus step; every machine composes …
mesh/repo-check head 6eea92c323f4 merged into trunk e0d69ab7a05b — pass: its merge-check.sh passed (run 2)
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-controller@ef3e5ae9b0dc (merged as cd6757f1 into main, walk plan-17918675…
mesh/delivery-group group fix/348-replay delivered: every member is delivered
The replay holds the operator's acceptance: a build sent after the resolver
stopped answering is not put back for that fault, and a restarted engine's
undecided statement does not forget it. Written only with what the controller
had before mesh-controller #179, so mesh-lab's prover lays it over the commit
before: it fails there and passes on the fix and on main.

The resolver's manifest test took either binding until the catalogue's main
bound once; mesh-catalog #161 (105cc7c0) did, so bind-dynamic is refused.
2026-10-13 06:09:09 +02:00
mesh-admin b12c4b82e8 Merge pull request 'Add the replay of issue 349: a merge acted on late superseded the merge after it (hq issue 349)' (#290) from fix/349-replay into main 2026-10-13 03:54:10 +00:00
jochen 5740874ce3 Drive the replay of issue 349 to where main decides (review of #290)
mesh/merge-gate head 5740874ce3b9 merged into trunk 1c9bc9cf0041 — pass: the change touches no module of the mesh's graph
mesh/repo-check head 5740874ce3b9 merged into trunk 1c9bc9cf0041 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/349-replay delivered: every member is delivered
Since ADR 0276 the earlier merge waits in a queued batch with no modules, and the
replay skipped it, so its pass on main proved nothing. The replay now ends the
later merge's walk and has another repository's merge heard, which cuts the batch,
and fails on any plan of the two merges left standing with nothing planned.
2026-10-13 05:42:52 +02:00
jochen bbf70fbda4 Add the replay of issue 349: a merge acted on late superseded the merge after it (hq issue 349)
Two catalogue merges acted on in reverse order left the fixed module planned at
the commit without the fix. The replay uses only what the controller had before
Listed among the store tests, so a check without a database cannot pass its skip.
2026-10-13 05:42:52 +02:00
mesh-admin 1c9bc9cf00 Merge pull request 'Let a trusted file that names its keys hold only those keys at the terminal (hq issue 586, ADR 0329 §5)' (#286) from fix/586-a-named-key-is-trusted-alone into main 2026-10-13 03:38:13 +00:00
jochen 3275a16beb Let a trusted file that names its keys hold only those keys at the terminal (hq issue 586, ADR 0329 §5)
mesh/merge-gate head 3275a16bebbb merged into trunk 584ee991961c — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 3275a16bebbb merged into trunk 584ee991961c — pass: its merge-check.sh passed
mesh/delivery delivered
A module with a trusted mergeable file had its whole layer refused through the
settings verb, because such a file took any key. Once a module's files name the
keys they take (ADR 0326), each file takes only those, so only a trusted file's
named keys are the terminal's: betterbird's tags, in a file nothing trusts, is set
through the verb, while its accounts and identities stay the terminal's or the
operator's warrant. A module whose files name no keys keeps the whole-layer rule.
2026-10-13 05:22:35 +02:00
mesh-admin 584ee99196 Merge pull request 'Add the replay of issue 340: a verb set what every agent session obeys (hq issue 340)' (#288) from fix/340-replay into main 2026-10-13 01:04:46 +00:00
jochen e44c726a38 List TestReplay340 among the store tests, so a check without a database cannot count its skip as a pass (hq issue 340, review 144)
mesh/merge-gate head e44c726a3812 merged into trunk 49102e1645c9 — pass: the change touches no module of the mesh's graph
mesh/repo-check head e44c726a3812 merged into trunk 49102e1645c9 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/340-replay delivered: every member is delivered
2026-10-13 02:57:35 +02:00
jochen a641ed7d15 Let the build agent run on the genesis document until its registry binds (hq issue 121, ADR 0336)
mesh/merge-gate head a641ed7d15c7 merged into trunk b12c4b82e820 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head a641ed7d15c7 merged into trunk b12c4b82e820 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/121-genesis-package-registry delivered: every member is delivered
With npm-package-registry optional, the binding file the agent's environment
names may be absent, and mesh-builder failed every build on reading it. It now
reads the genesis document the installer seals at genesis's packages step when
no binding is present, prefers the binding when both are, and each build's log
says which it resolved from (§4). A build that needs the registry - a package,
an image naming .npmrc, TypeScript with an @novox/ dependency - on an agent with
neither is refused before anything is built or published, naming the machine,
the unbound requirement and gitea (§6); it never falls back to the public
registry. The captured catalogue moves to the mesh-catalog change it is judged
with.
2026-10-13 02:53:41 +02:00
jochen 2a13917d6e Add the replay of issue 340: a verb set what every agent session obeys (hq issue 340)
The settings verb took the agent module's managed settings and an empty
runtime configuration's url from any caller. The replay uses only what the
controller had before #172, so mesh-lab's prover can show it fails there.
2026-10-13 02:52:28 +02:00
mesh-admin 49102e1645 Merge pull request 'Add the replays of issues 279 and 412 (hq issues 279, 412)' (#285) from fix/279-412-replays into main 2026-10-13 00:45:26 +00:00
mesh-admin eb7ee921b5 Merge pull request 'Replay issue 166 with only what the catalogue had before its fix (for mesh-lab R166)' (#287) from test/166-replay into main 2026-10-13 00:32:48 +00:00
jochen b1baa8d8ba Replay issue 166 with only what the catalogue had before its fix (for mesh-lab R166)
mesh/merge-gate head b1baa8d8baf0 merged into trunk 4473bc7d7315 — pass: the change touches no module of the mesh's graph
mesh/repo-check head b1baa8d8baf0 merged into trunk 4473bc7d7315 — pass: its merge-check.sh passed
mesh/delivery delivered
A replay must fail on the commit before its fix and pass on it; the fix's own tests read what the fix
added, so this one uses only ParseManifest, Resolve and the resolve tests' helpers, and can be laid
over the commit before #281.
2026-10-13 02:27:07 +02:00
mesh-admin 4473bc7d73 Merge pull request 'Let a requirement be optional, binding only a provider that already runs (hq issue 166, ADR 0330)' (#281) from fix/166-optional-requirement into main 2026-10-13 00:05:33 +00:00
jochen 3bdab29a5a Add the replays of issues 279 and 412, written with only what the catalogue had before each fix
mesh/merge-gate head 3bdab29a5a0e merged into trunk eb7ee921b533 — pass: the change touches no module of the mesh's graph
mesh/repo-check head 3bdab29a5a0e merged into trunk eb7ee921b533 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/279-412-replays delivered: every member is delivered
So the lab's prover can lay them over the commit before mesh-controller #260 and #267 and see them fail
there and pass on the fix (hq issues 279, 412).
2026-10-13 02:02:20 +02:00
jochen 79a3396340 Store optional requirements beside a plain requires list (hq issue 166, review of #281)
mesh/merge-gate head 79a3396340a7 merged into trunk 454a3db69a1e — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 79a3396340a7 merged into trunk 454a3db69a1e — pass: its merge-check.sh passed
mesh/delivery delivered
A controller older than optional requirements reads `requires` as names, and Inventory.Catalogue
stops at the first manifest it cannot read: the long form stored inside `requires` would have left
a rollback below this controller composing no machine. Stored, `requires` stays plain and the
optional names sit in `optional-requires`, a key an older controller sets aside, leaving out only
the module carrying it. A refusal of a hard requirement now names a module that wants it hard.
2026-10-13 01:53:18 +02:00
jochen 6ec4139871 Let a requirement be optional, binding only a provider that already runs (hq issue 166, ADR 0330)
Making every soft dependency a provision made it hard: a mesh without influxdb could not assign
grafana, and one with influxdb in its catalogue would install it for grafana. A requirement may now
be written {"name": ..., "optional": true}. It resolves as a hard one when a provider is assigned or
reachable, and is otherwise absent: never refused, nothing installed for it. Its binding, credential
and only-when-bound files are left out, its ${bound:...|default:...} placeholders take their
defaults, the module check refuses one that could render empty, and plan and node list it unbound.
2026-10-13 01:53:18 +02:00
mesh-admin 454a3db69a Merge pull request 'Refuse a declaring module's mergeable file that names no keys (hq issue 574, ADR 0326 §3)' (#282) from fix/574-a-declaring-module-names-its-files-keys into main 2026-10-12 23:28:21 +00:00
mesh-admin 7b2a9eae80 Merge pull request 'Read the one genesis bundle mesh-host carries now (issue 146)' (#284) from fix/146-controller-test-one-bundle into main 2026-10-12 23:21:14 +00:00
jochen 5795e03fad Read the one genesis bundle mesh-host carries now (hq issue 146, ADR 0331)
mesh/merge-gate head 5795e03fad7b merged into trunk cbac703e817d — pass: the change touches no module of the mesh's graph
mesh/repo-check head 5795e03fad7b merged into trunk cbac703e817d — pass: its merge-check.sh passed
mesh/delivery delivered
mesh-host #96 replaced foundation-first-node-nats.lock with foundation-first-node.lock,
and the test that holds the installer's first user list to the controller's composition
still read the removed file, so repo-check failed on main and every pull request. The
captured copy moves to mesh-host 7a41e12, with the placement guard at the same commit.
2026-10-13 01:16:10 +02:00
mesh-admin cbac703e81 Merge pull request 'Say a build handed out again restarted: its log, the queue and the check's verdict (issue 594, part)' (#280) from fix/594-a-replaced-agent-finishes-its-build into main 2026-10-12 22:55:59 +00:00
jochen 18221c4079 Judge ADR 0326 §3 only where a definition is written (hq issue 574)
mesh/merge-gate head 18221c4079d1 merged into trunk 7b2a9eae8020 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 18221c4079d1 merged into trunk 7b2a9eae8020 — pass: its merge-check.sh passed
mesh/delivery delivered
ParseManifest also reads stored builds again: a build's manifest at a
commit, the gate, a release, a module put back. betterbird's builds from
before its accounts file named its keys would have become uncomposable
there, so a send to g14 or shanks could fail and a rollback be refused.
The refusal now runs in module check, module add, a build taken in, the
builder and the merge gate's registration, and a file is no longer
reported by both §2 and §3.
2026-10-13 00:54:26 +02:00
jschoubben b1546ed28a Hold a walk's first-machine gate while its busy container finishes its build
mesh/delivery superseded: a newer head of the same pull request
mesh/delivery-group group feat/335-a-busy-container-is-replaced-when-idle rejected: its order contradicts itself: novox/mesh-catalog@190608193764, novox/mesh-c…
mesh/merge-gate out of date: judged against trunk 56d6313262e8, the trunk is now cbac703e817d; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 56d6313262e8, the trunk is now cbac703e817d; judged again on a recheck or when it is merged
A merge into mesh-controller replaced every build agent and threw away the
checks in flight (issue 594). The manifest's container busy is checked and
sent only to an engine that reads it; a report's busy waits are kept, the
gate starts at the report that says the container replaced, and the wait is
a phase of the delivery time. mesh-builder busy drains the agent and answers
while a build is in hand (novox/hq ADR 0335).
2026-10-13 00:53:10 +02:00
jochen db0cb1e15e Refuse a declaring module's mergeable file that names no keys (hq issue 574, ADR 0326 §3)
mesh/merge-gate head db0cb1e15e39 merged into trunk 56d6313262e8 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head db0cb1e15e39 merged into trunk 56d6313262e8 — fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheContro…
mesh/delivery superseded: a newer head of the same pull request
A module that declares a setting and owns a mergeable file naming no keys
still let every key reach that file. The refusal was held back in #272
because betterbird was one; mesh-catalog #252 named its keys, and every
module in mesh-catalog and mesh-media-catalog main now passes, so the
module check and registration refuse it again. A stored definition is
still composed as it is, never left out of a machine.
2026-10-13 00:40:24 +02:00
jschoubben ee1d7e5cae Mark a rerun check with a short (run N) after its verdict, and word a re-hand-out neutrally
mesh/merge-gate head ee1d7e5cae12 merged into trunk 56d6313262e8 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head ee1d7e5cae12 merged into trunk 56d6313262e8 — pass: its merge-check.sh passed
mesh/delivery delivered
The forge clips a status to 140 characters, so the long sentence pushed the
verdict's own words out; and a paused holder's hand-back or a bus delay also
count as a delivery, so the cause is not claimed (novox/hq issue 594).
2026-10-13 00:31:57 +02:00
jschoubben 43cab56452 Say a build handed out again restarted, in its log, the queue and the check's verdict
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head 43cab564523d merged into trunk 56d6313262e8 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 43cab564523d merged into trunk 56d6313262e8 — pass: its merge-check.sh passed
A build agent replaced mid-build leaves its ask to be handed out again, and
the second run looked like a first: the queue and the pull request showed
nothing of the 5-8 minutes lost (novox/hq issue 594).
2026-10-13 00:26:08 +02:00
mesh-admin 56d6313262 Merge pull request 'The controller's check stops waiting on fixtures and fixed intervals (issue 591)' (#279) from fix/591-the-controller-check-runs-in-minutes into main 2026-10-12 22:22:24 +00:00
jschoubben 7f3ff32361 Give a fast-renewing test lease a serving controller's time to renew, and prove the minted hash's cost
mesh/merge-gate head 7f3ff3236149 merged into trunk 21ac8fd8586d — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 7f3ff3236149 merged into trunk 21ac8fd8586d — pass: its merge-check.sh passed
mesh/delivery delivered
A renewal timed out at the renew interval, so the lease test's 1 s renewals
could lose the lease on a slow build machine (novox/hq issue 591, review of #279).
2026-10-13 00:08:48 +02:00
mesh-admin 21ac8fd858 Merge pull request 'Let a mergeable file name the setting keys it takes (hq issue 168, ADR 0326)' (#272) from fix/168-a-file-names-the-settings-it-takes into main 2026-10-12 22:07:08 +00:00
jschoubben 2819915e8e Stop the controller's tests waiting out fixed intervals
mesh/merge-gate out of date: judged against trunk e63af0df17f5, the trunk is now 21ac8fd8586d; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk e63af0df17f5, the trunk is now 21ac8fd8586d; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
Reading the build events waited two seconds on every queue read, the race
detector's exit sleep cost a second per command line a test runs, and the
lease and plan-wait tests waited out production intervals (novox/hq issue 591).
2026-10-12 23:53:06 +02:00
jschoubben 7892019d9e Copy each store test's database from a template migrated once, and hash test bus passwords at bcrypt's least cost
Running ~90 migrations into a fresh database per store test, and bcrypt at
the default cost under the race detector, were most of the controller's
check (novox/hq issue 591).
2026-10-12 23:53:06 +02:00
mesh-admin e63af0df17 Merge pull request 'End a walk's apply phase at each machine's own report of its send (hq issue 582)' (#278) from fix/582-apply-ends-at-the-machines-own-report into main 2026-10-12 21:36:33 +00:00
mesh-admin 961f2e82ea Merge pull request 'Hand a delivery build out before pull-request checks (issue 558)' (#276) from fix/558-a-delivery-build-goes-first into main 2026-10-12 21:20:19 +00:00
jschoubben e727589e48 End a walk's apply phase at each machine's own report of its send (hq issue 582)
mesh/merge-gate head e727589e483b merged into trunk 961f2e82ea1c — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head e727589e483b merged into trunk 961f2e82ea1c — pass: its merge-check.sh passed
mesh/delivery delivered
The walk records its send to the rest only after every machine was sent,
which takes 20-28 s, so the machine sent first fell outside the 2 s slack,
its report was missed and the phase ended at the next walk's send to it.
Keep each machine's own send time with what it was sent, match its apply
duration by that send and digest, and show each machine's report in the
walk's phases (ADR 0282 decision 6).
2026-10-12 23:18:28 +02:00
mesh-admin 2f157148c6 Merge pull request 'node shows what a machine said it is made of, with its age (issue 160)' (#269) from fix/160-node-shows-what-a-machine-is into main 2026-10-12 20:37:06 +00:00
mesh-admin 13be75eb65 Merge pull request 'Replay issue 201 with only what the controller had before its fix (for mesh-lab R201)' (#275) from test/201-replay into main 2026-10-12 20:22:37 +00:00
mesh-admin 13711fb5b9 Merge pull request 'Mark that the build agent takes work, so its health check can see it ready (hq issue 544)' (#277) from fix/544-the-build-agent-says-it-is-ready into main 2026-10-12 19:58:09 +00:00
jschoubben fc573f951a Remove only a move's proven second copy, and never delete a handed-out original (hq issue 558)
mesh/merge-gate head fc573f951a47 merged into trunk 2f157148c6c8 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head fc573f951a47 merged into trunk 2f157148c6c8 — pass: its merge-check.sh passed
mesh/delivery delivered
Second review of #276: a dead letter delivered again beside its original
shares its request id and was removed as a copy, losing the ask; and a
publish that outlived the pause could delete an original a holder had
taken. A copy now names the message it copies, removals are said, the
publish waits inside the pause's margin, and an original handed out
meanwhile keeps its place while the copy goes.
2026-10-12 21:46:49 +02:00
mesh-admin c0c379af2b Merge pull request 'Say the class of what a change plan moves, for a fix's budget (issue 403, hq ADR 0313)' (#265) from fix/403-fix-time into main 2026-10-12 19:41:26 +00:00
jschoubben 1ded7dff34 Make a queue move lossless and run its passes on the acting controller alone (hq issue 558)
mesh/merge-gate out of date: judged against trunk 4dbd4066ca54, the trunk is now c0c379af2b3c; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 4dbd4066ca54, the trunk is now c0c379af2b3c; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
Review of #276: a move deleted before it published, so a failed publish
lost an ask; a verb's own process ran a pass beside the serving one, so
one could resume the queue while the other moved asks and a build ran
twice. The copy is now published first and second copies removed by id,
asks only signal the acting controller's loop, moves stop with 5 s of
the pause left, a cancel finds a moved ask by id, and an unreadable ask
keeps its age.
2026-10-12 21:38:11 +02:00
jschoubben 50c55a773e Mark that the build agent takes work, so its health check can see it ready (novox/hq issue 544)
mesh/merge-gate head 50c55a773eb8 merged into trunk c0c379af2b3c — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 50c55a773eb8 merged into trunk c0c379af2b3c — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/544-the-build-agent-says-it-is-ready delivered: every member is delivered
The agent listens on nothing, so the node-engine could judge it only alive and
held it starting for the whole grace on every machine a walk moved it to. It
now writes a mark once it is bound to its worker, after its seat verbs are
served, and 'mesh-builder ready' passes only while the mark is there.
2026-10-12 21:33:12 +02:00
jochen daf52b3aeb Merge remote-tracking branch 'origin/main' into fix/403-fix-time
mesh/merge-gate head daf52b3aeb79 merged into trunk 4dbd4066ca54 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head daf52b3aeb79 merged into trunk 4dbd4066ca54 — pass: its merge-check.sh passed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1791834103351744736 (novox/mesh-controller at c0c379af into ma…
mesh/delivery-group group fix/403-fix-time delivered: every member is delivered
# Conflicts:
#	internal/link/events.go
2026-10-12 21:28:52 +02:00
jschoubben c6e234455f Hand a delivery build out before pull-request checks (hq issue 558)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate out of date: judged against trunk 4dbd4066ca54, the trunk is now c0c379af2b3c; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 4dbd4066ca54, the trunk is now c0c379af2b3c; judged again on a recheck or when it is merged
A merged change's walk waited minutes on its build behind checks in a
first-come queue. The queue now hands deliveries out first, a check that
has waited ten minutes before deliveries asked after it, and says each
ask's kind; the order is kept in the work queue itself.
2026-10-12 21:25:46 +02:00
mesh-admin 4dbd4066ca Merge pull request 'A seat names the paths its holder owns, and a claim or a declaration without the seat is refused (issue 412, hq ADR 0287)' (#267) from fix/412-a-seat-names-its-paths into main 2026-10-12 19:22:11 +00:00
jochen 45a1783ba9 Replay issue 201 with only what the controller had before its fix
mesh/merge-gate head 45a1783ba9a4 merged into trunk 13711fb5b91b — pass: the change touches no module of the mesh's graph
mesh/repo-check head 45a1783ba9a4 merged into trunk 13711fb5b91b — pass: its merge-check.sh passed
mesh/delivery delivered
The fix's own TestReplay201 reads what #273 added (sendScope.puttingBack,
recordedKept's core keep, the condition), so it cannot be laid over the
commit before. TestReplay201ThePlansSend composes a plan's send to the
control node through keepingRecorded, planFor and declarationFor and holds
that it carries the controller the machine was last sent, not the older
record; it fails on 84deb53^1 and passes on 84deb53. For mesh-lab's R201
(hq issue 201).
2026-10-12 21:17:31 +02:00
mesh-admin 84deb530ee Merge pull request 'Keep the controller and node-engine at the build a machine was last sent over an older record (hq issue 201)' (#273) from fix/201-a-push-never-downgrades-the-core into main 2026-10-12 18:52:04 +00:00
jochen b3787e4d8e Merge origin/main into fix/201-a-push-never-downgrades-the-core
mesh/merge-gate head b3787e4d8eb2 merged into trunk a681227af373 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head b3787e4d8eb2 merged into trunk a681227af373 — pass: its merge-check.sh passed
mesh/delivery delivered
# Conflicts:
#	testdata/store-tests
2026-10-12 20:31:19 +02:00
mesh-admin a681227af3 Merge pull request 'Add issue 401's replay, written with what the controller had before its fix (hq issue 401)' (#274) from fix/401-replay into main 2026-10-12 18:22:09 +00:00
mesh-admin d8f36a8fb7 Merge pull request 'Read an xz release in Go, so the build seat needs no xz tool (hq issue 345)' (#264) from fix/345-xz-in-go into main 2026-10-12 17:35:56 +00:00
jochen f0fd7fe886 Leave a kept core move out of what a gate judges (issue 201, review 91)
mesh/merge-gate head f0fd7fe8868e merged into trunk a681227af373 — fail: builds mesh-controller → novox; no bus step; head f0fd7fe8868e does not merge clea…
mesh/repo-check head f0fd7fe8868e merged into trunk a681227af373 — fail: head f0fd7fe8868e does not merge cleanly into trunk a681227af373: in conflict test…
mesh/delivery superseded: a newer head of the same pull request
gatedSend took its judged moves from the record, so on a machine kept at
a newer controller the gate judged the controller moving back to the
record, read its own send's kept controller as a later build there, and
was superseded with nothing judged. The kept moves are now left out
there as in ungatedIn (withoutKept). The doc says the bus step keeps
too, and the three store tests are listed.
2026-10-12 19:16:26 +02:00
jochen 41a9ff15dd Merge origin/main into fix/201-a-push-never-downgrades-the-core 2026-10-12 19:07:20 +02:00
mesh-admin ba685b17f5 Merge pull request 'Let plans wait for a plan to move and answer it as data (issue 229)' (#266) from fix/229-plans-wait into main 2026-10-12 17:06:50 +00:00
jochen d170491256 Keep the core over an older record instead of refusing the send (issue 201, review 90)
mesh/merge-gate out of date: judged against trunk f8daec01d3f0, the trunk is now ba685b17f5e1; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk f8daec01d3f0, the trunk is now ba685b17f5e1; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
Refusing the whole send wedged a machine whose gate put back the
controller and whose put-back send was refused: every later automatic
send there was refused, retried each tick, and held up every other walk.

A send now composes the controller and the node-engine at the build the
machine was last sent when the mesh's record is older, and carries the
rest; nothing is refused or retried, and a plan whose own build is the
older one ends with its gate superseded. A sent build that failed its
gate is never kept, so the next send completes a put-back. The condition
names the two ways out and clears on any send that carries the record,
a person's push included.
2026-10-12 18:51:17 +02:00
jochen 6bb1c9b768 Merge remote-tracking branch 'origin/main' into fix/201-a-push-never-downgrades-the-core 2026-10-12 18:40:12 +02:00
mesh-admin f8daec01d3 Merge pull request 'Read a plan by its id as a parameter, never as SQL (issue 561)' (#268) from fix/561-plan-id-parameter into main 2026-10-12 16:37:07 +00:00
jochen f3c15e00af Add issue 401's replay, written with what the controller had before its fix, so the lab can prove it
mesh/merge-gate head f3c15e00af50 merged into trunk d8f36a8fb706 — pass: the change touches no module of the mesh's graph
mesh/repo-check head f3c15e00af50 merged into trunk d8f36a8fb706 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/401-replay delivered: every member is delivered
The fix's own test reads GivenOwnSecret, which the commit before does not have, so it cannot be laid over it. TestReplay401 asserts the same outcome with register, assign, planFor, declarationFor and the inventory's secret reads; it fails on 1e28342^1 and passes on main. Named in testdata/store-tests so a skip without the store fails the check. Replay for hq issue 401.
2026-10-12 18:36:22 +02:00
jochen 9038d4b0d3 Refuse a send that puts the controller or node-engine behind its machine's build (issue 201)
mesh/merge-gate out of date: judged against trunk f10aedac5880, the trunk is now f8daec01d3f0; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk f10aedac5880, the trunk is now f8daec01d3f0; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
On 2026-10-02 a plan sent the control node a controller older than the one
running there, whose seat row the newer build had widened; the older binary
refused the row and the mesh went quiet for ten minutes. ADR 0185 took the
outage out of it, and the race stayed open.

Every send but a person's push now refuses a move of mesh-controller or
mesh-host to a build asked before the one its machine was last sent (the
builds' registration order, issue 219), whatever composed it — gated or not.
A gate's own put-back is the one exception, for the modules it puts back,
and says so in its send's scope. The refusal is the send's error, which the
plan keeps as its note, and a warning condition on the machine, cleared by
the next send there that carries no such move.
2026-10-12 18:21:17 +02:00
mesh-admin f10aedac58 Merge pull request 'A merge check judges the head merged into its trunk, and names the trunk commit (hq ADR 0323, issue 341)' (#271) from feat/341-a-merge-check-judges-the-merge into main 2026-10-12 16:11:27 +00:00
jochen b538397055 Name the issue that holds the refusal back (hq issue 574)
mesh/merge-gate head b5383970550b merged into trunk e63af0df17f5 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head b5383970550b merged into trunk e63af0df17f5 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-12 18:00:49 +02:00
jochen dd10889209 Hold back refusing a declaring module's silent mergeable file (hq issue 168)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
betterbird declares a preference and owns a mergeable file that names no
keys, and the catalogue can only name them once this controller runs: the
refusal made the catalogue check fail against catalogue main. It follows
once mesh-catalog names the keys.
2026-10-12 18:00:33 +02:00
jschoubben 875ebc093e Format PullUpdated, and keep the announced base head on a rechecked member
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/341-a-merge-check-judges-the-merge delivered: every member is delivered
A head checked again through mesh-delivery reached the direct verdict with no
base head, so a head no build judges was answered with no trunk commit and the
merge tool would refuse it (novox/hq ADR 0323).
2026-10-12 17:52:12 +02:00
mesh-admin 1e28342ede Merge pull request 'Make no value for an own secret issued outside the mesh that nobody gave (issue 401)' (#262) from fix/401-an-outside-secret-not-given-is-not-made into main 2026-10-12 15:48:12 +00:00
jochen 286257d802 Let a mergeable file name the setting keys it takes (hq issue 168, ADR 0326)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestModuleCheckPassesTheCatalogue (0.89s); and TestEveryCatalogueManifestParses, TestNoCatalogue…
mesh/delivery superseded: a newer head of the same pull request
Every key of a module's layers merged into every mergeable file it owns:
the mesh's own words, a key meant for a ${setting:} in another file, a
key for a second configuration file. A module could not own two
configurable files. A file now names its keys in takes-settings and
takes only those; a key no file names is stray where it is set; a
module that declares a setting names the keys of every mergeable file,
and one that declares none keeps today's form (ADR 0164 §15).
2026-10-12 17:47:05 +02:00
jschoubben bc623bc1f9 Name the announced trunk commit on a verdict no build judges
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: not gofmt'd by the toolchain's gofmt: internal/link/events.go
mesh/delivery-group group feat/341-a-merge-check-judges-the-merge rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
A change touching no module, in a repository with no merge-check.sh, was
answered by the controller with no trunk commit, so the merge tool refused it,
asked again and refused it for ever (novox/mesh-tools-go; novox/hq ADR 0323).
2026-10-12 17:44:18 +02:00
jochen 67d54d9d09 Say an account without its time as age unknown, not as two thousand years (issue 160)
mesh/merge-gate head 67d54d9d09b2 merged into trunk 13be75eb65c0 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 67d54d9d09b2 merged into trunk 13be75eb65c0 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-12 17:43:38 +02:00
jschoubben ce9dc8e157 Run no hook and no signature setting when a check makes its merge
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
A build machine's or a clone's git configuration must not run code or change
the merge a check judges (novox/hq ADR 0323).
2026-10-12 17:39:59 +02:00
mesh-admin c672b3df6d Merge pull request 'Stop composing ADR 0312's group grant, so deliveries to the media machine pass again (hq issue 567)' (#270) from fix/567-revert-the-group-grant into main 2026-10-12 15:36:12 +00:00
jschoubben 25deb499bf Say why a merge's changed files read as none on an error
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-12 17:31:36 +02:00
jschoubben 83a9cf6e3a Judge a pull request's head merged into its trunk, and name the trunk commit
A check judged the head alone, so two pull requests that each passed could
break the trunk together (node-engine #66 on 2026-10-09), and a recheck after
the trunk moved judged the same head again. The build seat now merges the head
into the trunk's head in its own clone, never pushed, fails a head that does not
merge cleanly naming the files, and the verdict names the head, the trunk
commit and the commits read beside it, for the forge and mesh-delivery to weigh
against the trunk's next move (novox/hq ADR 0323, issue 341).
2026-10-12 17:27:01 +02:00
jschoubben 7b050abdcb Stop composing ADR 0312's group grant, so deliveries to the media machine pass again
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
Since #256 delivered, the node-engine judges every read-write access by the path's group alone and
states six media modules unhealthy on ace, though they write their folders as the owner; every
delivery there fails its gate (hq issue 567). ADR 0325 replaces ADR 0312; until it is built, the
controller sends no grant, as before #256.

This reverts commit 3cfc121670.
2026-10-12 17:25:46 +02:00
jochen 7953d2889d Show what a machine said it is made of in node, with its age (issue 160)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The architecture and kernel a machine reported were kept and read by nothing, and the facts the
node-engine now says (memory, filesystems, processor, distribution, virtual or physical, uptime,
timezone) would have joined them. node shows every one in one block with the age of the account,
and a fact the machine could not determine as unknown with its why, never as a zero.
2026-10-12 17:23:07 +02:00
jochen c3b4f187e1 Name issue 401's two store tests, so a skip without the store fails the check (issue 459's rule)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1791826579427234545 (novox/mesh-controller at d8f36a8f into ma…
2026-10-12 17:15:18 +02:00
jochen edd33f21d2 Make no value for an own secret issued outside the mesh that nobody gave (issue 401)
The send minted a random value for a plain own secret marked issued-by outside and placed it, so a
module could not tell "not given" from "given and wrong" (ADR 0228). It now places only a value a
person gave: not given is no file, an empty value where a module's file names it, and nothing to
restart on, as ADR 0283 decided for a family's member. A value the mesh made for one earlier is not
read as given.
2026-10-12 17:15:12 +02:00
jochen 24347ea5a0 Step back from a rendered resolver file only, and compare a declared path cleaned (issue 412, review 80)
mesh/merge-gate head 24347ea5a02b merged into trunk 84deb530ee5a — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 24347ea5a02b merged into trunk 84deb530ee5a — pass: its merge-check.sh passed
mesh/delivery delivered
A resource on the resolver file was left out by the check and still composed by the declaration, so an
uplink declaring it as a resource beside the resolver passed with two writers; it is refused again, as
before. And /etc//hosts or /etc/../etc/hosts is /etc/hosts to the machine, so it is compared cleaned.
A seat names no root or top-level directory.
2026-10-12 17:08:55 +02:00
jochen 5ad36127ee Hold the plans schema's longest bound to the value, and say what a slow store gives (review 75)
mesh/merge-gate head 5ad36127ee8b merged into trunk f8daec01d3f0 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 5ad36127ee8b merged into trunk f8daec01d3f0 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-12 16:50:59 +02:00
jochen a1a6f6493d Refuse a plans argument that is not a plan's id (issue 561)
mesh/merge-gate head a1a6f6493d50 merged into trunk f10aedac5880 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head a1a6f6493d50 merged into trunk f10aedac5880 — pass: its merge-check.sh passed
mesh/delivery delivered
Every argument of plans that names a plan (id, wait, stop, close, retry,
go) is checked once where the call becomes a command line, against the
ids the mesh gives plans (plan-<digits>, release-<digits>), so a quote
from a caller never reaches a command or the store (review 75 on #266).
2026-10-12 16:46:29 +02:00
jochen e9f060e0be Read a plan by its id as a parameter, never as SQL (issue 561)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
PlanByID pasted its id into the query, and the id comes from a caller of
the plans verb: an id holding a quote changed the query (x' or '1'='1
answered another plan). The id is now $1, and RecentPlans's limit too, so
no plan query is built from a value.
2026-10-12 16:33:23 +02:00
mesh-admin 3cfc121670 Merge pull request 'Compose each read-write access's grant for the node-engine (hq ADR 0312, issue 537)' (#256) from feat/537-a-read-write-access-grants-the-paths-group into main 2026-10-12 14:20:27 +00:00
jochen a3bf5da9b1 Name the paths a seat's holder owns, and refuse a claim or a declaration without the seat (issue 412)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
Which module owns /etc/hosts or /etc/resolv.conf on a machine was answered only by reading every
manifest, and a module not holding node-hostname could declare /etc/hosts where nobody held it.
hq ADR 0287 makes the paths part of the seat's definition: node-hostname names /etc/hosts and
/etc/hostname, node-uplink names /etc/resolv.conf stepping back for node-resolver, node-resolver
names /etc/resolv.conf.

CanHold refuses a holder that does not declare every path (registration and handover), registration
refuses a module declaring a seat's path while claiming none of the seats naming it, a module's own
seat names paths under the same rules (node only, literal, not named by another seat), composition
refuses a generated resource on a seat's path, the uplink's step-back is read from the seat instead
of a special case, and `seats` shows each seat's paths and who owns each on each machine.
2026-10-12 16:19:49 +02:00
jochen d20b52cb19 Let plans wait for a plan to move and answer it as data (issue 229)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
An agent following a rollout had plans answer once, in prose, and polled the
console with a curl loop of its own to be told a plan moved. plans now takes
wait (a plan's id), until (moved, tier or done) and bound, and answers when the
plan moves or the bound passes, saying which, with the plan as data and its
line as the output.

A call's one answer has to reach the console inside its thirty seconds, so
link.AnswersWithin lets a handler that waits on purpose hold its answer up to
LongestAnswer (25s) instead of AnswerWithin; the bound is at most 20s, and a
caller following a plan further asks again.
2026-10-12 16:18:54 +02:00
jochen 7ca5a45ecc Class a new module by its own name, so adding a core module is not called leaf (issue 403, review 69)
mesh/delivery-group group fix/403-fix-time checking: composed check out of date: novox/mesh-controller@7ca5a45ecc5a was judged against trunk ?, the trunk is no…
mesh/merge-gate out of date: judged against trunk 84deb530ee5a, the trunk is now 4dbd4066ca54; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 84deb530ee5a, the trunk is now 4dbd4066ca54; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
2026-10-12 16:16:02 +02:00
mesh-admin db7e4869ae Merge pull request 'Raise a machine that refuses or only partly applies its declarations past one apply interval (issue 233)' (#263) from fix/233-a-failed-probe-fails-only-what-needs-the-tool-2 into main 2026-10-12 14:11:10 +00:00
jochen 539a1c16cd Refuse an xz file of more than 1,024 blocks before decoding it (review 68, hq issue 345)
mesh/merge-gate head 539a1c16cd4e merged into trunk ba685b17f5e1 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 539a1c16cd4e merged into trunk ba685b17f5e1 — pass: its merge-check.sh passed
mesh/delivery delivered
The xz reader keeps one delta or BCJ decoder for every block of a stream and frees them only when the
stream ends, so a file of many tiny blocks grows the heap with its length: about 67 MiB live for a 6 MB
`xz --block-size=1 --delta` file, about 5.6 GiB at the 512 MiB download cap.

The file's framing is now walked first, without decompressing: stream header, each block header, the
LZMA2 chunk headers (each carries its compressed size, so a chunk is skipped unread), padding, check,
index and footer. A file of more than 1,024 blocks or 16 streams is refused, as is an index whose block
count is not the count walked. The index alone is not trusted because the reader compares it with the
blocks only after decoding them all, so a lying index would still be decoded. Refusing delta and BCJ
outright was not chosen: it needs the same walk to see each block header, and would refuse sound
releases filtered with BCJ.

1,024 blocks keep at most about 0.35 MiB of decoders. Betterbird's archive is 1 stream with 1 block;
multi-threaded xz cuts blocks of 3 times the dictionary, so 512 MiB in blocks of 512 KiB still fits.
The library stays unforked.
2026-10-12 16:08:33 +02:00
mesh-admin 9eddf36503 Merge pull request 'A directory's owner may follow the account its module runs as (issue 279)' (#260) from fix/279-owner-follows-a-setting into main 2026-10-12 14:00:23 +00:00
jochen 05eccbf22b Say the class of what a change plan moves, so a fix's budget follows its modules (issue 403)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery-group group fix/403-fix-time ready: every member ready, and composed together they pass
mesh/delivery superseded: a newer head of the same pull request
The issue tracker holds a fix to the budget of the slowest-budgeted module it
moves, and gives none to a fix that moves no module (hq ADR 0313 §2). The
walk's class covers every merge a walk carries; the plan's class is the
change's own: core, leaf, or none.
2026-10-12 15:50:11 +02:00
jochen 01fc9e1d73 Read xz with a real dictionary maximum (review 65, hq issue 345)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
ulikunitz/xz's ReaderConfig.DictCap is a minimum: a stream asking for a
1 GiB or 4 GiB dictionary was given it. github.com/therootcompany/xz, a port
of xz-embedded, takes a maximum and refuses a block whose LZMA2 dictionary is
larger before allocating it. 64 MiB, xz's largest preset.
2026-10-12 15:39:34 +02:00
jochen d4de4543cc Judge S21 by time since the first failing report while the machine is heard, not by how many reports said it (issue 233 review)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/233-a-failed-probe-fails-only-what-needs-the-tool-2 delivered: every member is delivered
2026-10-12 15:39:08 +02:00
jochen a371de37fe Keep a directory owner's settings at the controller's terminal (issue 279)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
Root chowns a directory to the number its owner's setting names, so a key
an owner asks for must not change through the settings verb: any caller
could have set puid to 0 or to its own uid. TerminalKeys now takes every
${setting:…} in a directory's owner, whatever else asks for it, and the
owner is written as the parsed number so "0001" never reaches the
node-engine as text (review 60).
2026-10-12 15:37:29 +02:00
jochen 77acb40092 Let a directory's owner follow the account its module runs as (issue 279)
A media manager runs as the puid/pgid its assignment sets per node, but a
declared directory's owner was literal, so a folder the app must write was
refused on a machine whose identity was not the image's default. A
directory's owner now takes ${setting:<key>} (user, or user:group), filled
at composition and refused unless the value is a whole number, and the
module check refuses any other shape or a key the module asks for nowhere
else.
2026-10-12 15:37:23 +02:00
mesh-admin 5facd8b08e Merge pull request 'An act's answer is said from one resolution after its write (hq issue 237)' (#253) from fix/237-one-resolution-for-assign into main 2026-10-12 13:31:03 +00:00
mesh-admin 72edb458cc Merge pull request 'The node-reads seat: its verbs, its holder's own account, and the node's secret places (hq ADR 0310, issue 418)' (#257) from fix/418-node-reads into main 2026-10-12 13:20:23 +00:00
jochen 10f76cafc4 Read an xz release in Go, so the build seat needs no xz tool (hq issue 345)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
Betterbird's pinned .tar.xz failed on the build seat with "this build seat has
no xz to read it": the seat's image is the catalogue's build-agent recipe,
which installs git and docker-cli alone, and xz had been added only to
cmd/mesh-builder/Dockerfile, which the seat does not build from. The stream
is now read with github.com/ulikunitz/xz (vendored), its dictionary capped
at xz's own largest preset, and what it unpacks to bounded as before; the
xz line in the unused Dockerfile is taken back out.
2026-10-12 15:10:52 +02:00
mesh-admin 2843834813 Merge pull request 'The build queue answers as data too (issue 455)' (#259) from fix/455-the-queue-answers-as-data into main 2026-10-12 13:04:26 +00:00
jochen 3e10ad1535 Raise a machine that refuses or only partly applies its declarations past one apply interval, naming the package manager and its words, so the mesh says it rather than only status (issue 233)
mesh/delivery-group group fix/233-a-failed-probe-fails-only-what-needs-the-tool-2 ready: every member ready, and composed together they pass
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-12 14:49:33 +02:00
mesh-admin 27b13e4586 Merge pull request 'Ask a node's engine which of its directories the walk would refuse, read-only (issue 358)' (#258) from fix/358-judge-directories-read-only into main 2026-10-12 12:48:56 +00:00
mesh-admin 142a5ad71d Merge pull request 'An archive may be an upstream release pinned by URL and sha256, mirrored by the build seat (hq issue 345, ADR 0316)' (#255) from fix/345-builder-archive-pins into main 2026-10-12 12:33:58 +00:00
jochen 63a99d4323 Name the vault's placed directories among a node's secret places, fail loudly on one that cannot be resolved, and keep the reads account out of every group (review 54, issue 418)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/418-node-reads delivered: every member is delivered
The real mesh-vault manifest declares its directories without a path, so the
list left its state and ledger out, and a place still holding a placeholder
vanished from it without a word. The provider's directories are now resolved
where the node places them, an unresolved place fails the composition naming
the module, and a node-reads holder whose account is put in a group is refused.
2026-10-12 14:30:00 +02:00
jschoubben 0cd8bc0bfb Answer the build queue as data too, so a check can say how far back it waits
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The queue verb answered only a table, so nothing outside the controller
could say an ask's place in the queue. It now asks its command for JSON:
each ask's state, kind, repository, commit, machine, since and position
among the waiting, with the table carried beside it and answered as the
verb's output, both written from one read. A repository's password, if
one were ever written into it, is left out of what is shown.

novox/hq issue 455
2026-10-12 14:22:22 +02:00
jschoubben 7385d9c092 Ask a node's engine which of its directories the walk would refuse, read-only (issue 358)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/358-judge-directories-read-only delivered: every member is delivered
Before the node-engine gives a mesh directory its mode and owner only through
a walk that follows no link another account could make, the operator is to
see which directories that walk would refuse on every node, the control-node
included, where no tool reads a path. `node judge-directories <node>` signs
the ask as a hand-over is, under its own context, and prints the engine's
one line per directory; it changes nothing, so it is a seat verb an agent may
call. The node is granted the subject, the controller alone its publish.
2026-10-12 14:17:01 +02:00
mesh-admin db9e9c5b87 Merge pull request 'Advise a recheck for an unanswered check on a ready or rejected delivery (issue 451)' (#254) from fix/451-a-recheck-is-the-step into main 2026-10-12 12:14:43 +00:00
jochen 4ead5a3df7 Define the node-reads seat, refuse a holder without an account of its own, and tell it where its node keeps a secret (hq ADR 0310, issue 418)
mesh/delivery-group group fix/418-node-reads rejected: a member's own check failed
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
Reading a file, searching one, listing a directory, the owner and mode of each
component of a path and looking up an account had no mesh tool, so they went
through the login shell's execute as the operator's account. node-reads is a
node seat with six verbs, each saying the commands it replaces. Its holder must
run as an account of its own (ADR 0310 §3), and its own runtime is given
MESH_SECRET_PLACES: the directory of every module's own secret, every credential
file and grant directory, the vault's directories and the sealed-secret keep,
so it refuses them by name before any read (§5).
2026-10-12 13:57:24 +02:00
jschoubben a23b190747 Merge remote-tracking branch 'origin/main' into fix/451-a-recheck-is-the-step
mesh/merge-gate pass: builds mesh-controller, route-proxy → ace, novox; no bus step; every machine composes with the change as it did without (4 of 4 compo…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/451-a-recheck-is-the-step delivered: every member is delivered
2026-10-12 13:56:52 +02:00
jochen 3be6e86f77 Say an act's answer from one resolution after its write (issue 237)
mesh/merge-gate pass: builds mesh-controller, route-proxy → ace, novox; no bus step; every machine composes with the change as it did without (4 of 4 compo…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
assign read the whole mesh before its write, to record the seats held by
derivation, and that read printed on standard error that the node did not
resolve for lack of the seat the act was about to give it. The seat verb
answers with both streams, so the stale line came last and named the
assignment just made as its remedy.

The reads inside an act no longer print 'left out' lines; the node's
unmet seat dependencies and whether it resolves come from one planFor
taken after the write, and every other machine from blockedElsewhere.
unassign now says when its node does not resolve, from the same
resolution.
2026-10-12 13:53:19 +02:00
jochen b0c4b849a7 Judge an upstream pin's host like any name, and harden the upstream unpack (review 35, hq issue 345)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The url was skipped by the installation check, so an installation's host or
an address passed; it is judged whole now, the vendor's host declared in the
artifact's names-on-purpose, and an address or a reserved name as the host
is refused. Only the version's dotted-quad reading is skipped. The unpack
caps its entries, drains xz before waiting for it, reads a pax global
header and a nested within's parents, and a redirect off https is not
followed.
2026-10-12 13:51:35 +02:00
jochen b849af53dd Let an archive be an upstream release pinned by URL and sha256, mirrored by the build seat (hq issue 345, ADR 0316)
A mail client's release is about 80 MB per version, too large to keep in a
repository at every upgrade, and an add-on whose licence forbids copying it
to third parties may not be in a repository at all. An archive artifact may
now name url, sha256 and version (and file or within) instead of from: the
build seat fetches it once, refuses it when its digest is not the pin's and
names both, keeps it in its workspace under the digest (a vendor that serves
only its newest version cannot break a later build of the same pin), and
packs it like any archive, so a node gets the mesh's own archive by digest
and never asks the vendor. The build seat's image carries xz, which Go's
standard library does not read. The installation check takes the vendor's
address and version of such a pin for the world's, as a registry is an
image's.
2026-10-12 13:51:35 +02:00
jschoubben 74bd90df7e Pin the node-engine at its merge of the access grant, and capture it beside (hq ADR 0312, issue 537)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1791819383983869952 (novox/mesh-controller at c672b3df into ma…
2026-10-12 13:47:28 +02:00
jschoubben 30599c1858 Pin the node-engine at its reviewed grant, and hold the mount match to the path itself (hq ADR 0312, issue 537 review)
A path beginning with the same letters is another path, and a read-only mount
of a read-write access is still granted: the grant follows what mounts it.
2026-10-12 13:47:17 +02:00
jschoubben 10d4712f82 Name who a read-write access grants its path's group to, so modules write shared data through the group the operator gave it (hq ADR 0312, issue 537)
Each read-write access is composed with a grant: the module's containers that
mount its path and the account its tools run as (the operator's, or the module's
own under ADR 0259 §8); that runtime restarts on the access, so it holds a group
the engine put its account in. A read access grants nothing. Sent only to an
engine whose statement says GrantContract (4), since an older one refuses the
field. Pins the node-engine at the branch that reads it.
2026-10-12 13:47:17 +02:00
jschoubben e737e13bbb Advise a recheck for an unanswered check on a ready or rejected delivery (issue 451)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.66s)
mesh/delivery-group group fix/451-a-recheck-is-the-step rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
That check answered once, so the build queue is the wrong place to look.
mesh-delivery names the delivery's state as data (held); the controller
reads it and never the words. A line without it keeps the advice of 438.
2026-10-12 13:47:11 +02:00
mesh-admin 4b1dee6b79 Merge pull request 'A merge that shares nothing with the assembling batch, and whose group waits for no one, is cut at once (hq issue 525, ADR 0304 §1, ADR 0306 §2-3)' (#250) from feat/525-a-lone-merge-skips-the-window into main 2026-10-12 11:46:59 +00:00
mesh-admin 4ab441ce16 Merge pull request 'Ask only once the bus enforces the controller's own grant, judged for real and failing closed (review of hq issue 353)' (#188) from fix/353-the-asker-judges-the-controllers-own-grant into main 2026-10-12 11:34:14 +00:00
jochen f996cf7b6b Take an older bus module's allowed as held, and name only remedies that work (review 31 of PR 188, hq issue 353)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
After mesh-catalog #166 merges and before the bus upgrade, the bus's
machine cannot be pushed, so any change to the composed list made the
list check read behind for good: every ask was withheld and the condition
named a push that is refused. An older module's allowed now counts as
held, the unconfirmed reload said once; the worst case is one ask refused
in the seconds before a reload, which is asked again (issue 369).

The list check stays for a module that says not allowed or cannot answer,
and its remedy is the plan's own reason when the list cannot be composed,
the bus upgrade while sends there wait on it, and a push otherwise. An
in_force false is a wait only when the module gives both times; otherwise
what is not known is named.
2026-10-12 13:21:32 +02:00
jochen bcfcfaaaf3 Keep asking on an older bus module and name the real remedy for each grant not held (review of hq issue 353)
A bus module older than its in_force answer read as not held, so a merge
before mesh-catalog #166 is live would silence every ask until a person's
bus upgrade. Such an answer now falls back to the check main makes (is the
list sent to the bus's machine the one composed now), says the reload is
not confirmed, and each case names its own remedy: push <node> for a list
behind or not allowing it, a ~5 s wait for a reload, the bus upgrade when
an older module's list cannot be judged.
2026-10-12 13:21:32 +02:00
jschoubben a3181e356b Ask only once the bus enforces the controller's own grant, judged for real and failing closed (review of hq issue 353)
change held asking back, a list sent was taken for a list the bus enforces, a plan that could not be
worked out read as not behind, and no test held the real judgement. Now the asker asks only when the
composition grants the controller itself to publish its ask, and the bus's own module says that grant
is in force on the running server (nats_user_can's in_force, mesh-catalog); each fact not known is not
held, said with why. brokerBehind takes a list nobody could compose as behind.
2026-10-12 13:21:32 +02:00
mesh-admin dd2fd28c92 Merge pull request 'closedNames waits for the verb's own connection by name, not the first closed one (issue 543)' (#252) from fix/543-wait-for-the-verbs-own-connection-by-name into main 2026-10-12 11:21:18 +00:00
mesh-admin 9d165c7500 Merge pull request 'plan-moved says what a walk touches, its bound and its outcome when it starts and ends (ADR 0308, issue 532)' (#247) from adr-0308-walk-touches into main 2026-10-12 11:08:53 +00:00
jochen 7630072e63 Wait for the verb's own connection by name, not the first closed one (issue 543)
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
closedNames returned as soon as the bus's closed-connections list held
anything, which could be the test's own setup connection closing before
the verb subprocess's connection did. That raced TestAVerbsOwnProcessNamesItsConnection
red about once in 80 runs. It now waits until the connection it is named
for appears, or the deadline passes.
2026-10-12 13:05:15 +02:00
jochen 1e9dcf0667 Say every walk a verb keeps, and clear a stop from a walk that walks again
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
cancel, clear and rebuild kept a walk's end or restart without saying it, as plans stop, retry and
release-backlog did before them. The controller's run now says every save a verb makes on the bus, so the
next verb cannot lose a mark the same way. A stop is cleared when a walk is resumed, so one that fails again
ends failed and the search for its merge runs (review 21, ADR 0308, issue 532).
2026-10-12 12:54:20 +02:00
jochen cd6610cdf7 Say a walk's start or end when a person stops, closes, retries or releases it, and a hand stop as stopped
plans stop|close, plans retry and upgrade release-backlog kept a walk's start or end mark without saying it,
so the router would hold what the walk touched until its bound; each now runs on the bus as plans go does.
A hand stop keeps who stopped it, so its outcome is stopped, not failed (review 15, ADR 0308, issue 532).
2026-10-12 12:54:20 +02:00
jochen facc551ff2 Say what a walk touches, its bound and its outcome when it starts and ends (ADR 0308, issue 532)
The router holds what an open walk touches and tells its result once; it can do so
only if plan-moved names the walk's start and end, the modules and machines it moves,
what a moved tool runner or node-engine carries there, and how long to hold.
2026-10-12 12:54:20 +02:00
jschoubben 71b3aeaf2c Keep a group member of the other kind's batch from being overtaken (hq issue 525)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A merge's group member counted as merged when its merge waited in any batch,
but kinds never share a batch: the later member got a batch of its own, cut
at once, and walked before the earlier member's batch was cut, breaking the
group's order (ADR 0276 §5). Only a member kept in a batch of the merge's own
kind counts as merged now; one kept in the other kind's keeps the merge
waiting, and its batch names that member, so the two are cut in the order
their windows close (review of #250).
2026-10-12 12:54:11 +02:00
mesh-admin fafa3fd9ef Merge pull request 'Require the service manager's reset-failed, wanted-by and unlink-dangling verbs (issue 332)' (#248) from fix/332-require-service-manager-verbs into main 2026-10-12 10:53:20 +00:00
jschoubben 6e5a1df575 Judge the installer's user list against mesh-host with the groups grant (hq issue 525)
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.78s)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
The controller now asks mesh-delivery.groups, and mesh-host a76baa4b carries
that grant in the installer's first user list (mesh-host #85). The captured
copy is moved to that commit so the genesis test judges the list that ships.
2026-10-12 12:31:50 +02:00
mesh-admin d881f4764d Merge pull request 'Keep a condition's looks in the store, so a look that changes nothing writes no bucket (issue 420)' (#249) from fix/420-conditions-bucket-on-transition into main 2026-10-12 10:27:26 +00:00
jochen 61d00a234a Give the store tests' service manager holders the three verbs it now must serve (issue 332)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
They named only the eight verbs from before issue 332 and were refused once the seat required
reset-failed, wanted-by and unlink-dangling.
2026-10-12 12:25:05 +02:00
jschoubben 4d203b5a5d Ask mesh-delivery whether a merge's group still assembles (hq issue 525)
A merge from any head waited for the window because the controller could not
see a group's members, so the window skip changed nothing in production and
unrelated merges walked one after another. The controller now asks
mesh-delivery's groups, within 3 s and before the plans are held: a merge
waits only when its head's group has another member whose pull request is
open and whose merge it does not keep yet (ADR 0306 §2). No answer means it
waits, and its batch says why. A headed merge in no waiting group and related
to no batch is cut at once, as one from no head is (ADR 0304 §1, ADR 0306 §3).

The controller's grant names mesh-delivery.groups. The installer's first user
list in mesh-host must carry it too before the genesis test can pass.
2026-10-12 12:14:46 +02:00
jschoubben 6196a4815e Let a merge that shares nothing skip the merge window (hq issue 525)
A merge waited the whole window even when nothing assembling shared anything
with it. It now joins only a batch it is related to (ADR 0304 §1, read by ADR
0306 §3: repository, a module the cut would move, a graph edge, a group head),
folding the newer into the oldest when it relates to two. A merge related to
none starts a batch whose window is closed at once (PlanBatch.CutAtOnce),
unless its delivery group may still have a member to merge (ADR 0306 §2).

The controller cannot see a group's unmerged members (that is mesh-delivery's
record), so a merge from any head still waits, in a batch of its own: the safe
error is to wait. Walks stay one at a time; a batch cut at once queues behind
an open walk. A stop's merges walked again join a related batch, or their own.
2026-10-12 12:14:46 +02:00
jochen dcdafd37f0 Keep a condition's looks in the store, so a look that changes nothing writes no bucket (issue 420)
mesh/delivery delivered
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
Every look rewrote the condition's value in the conditions bucket, because the
value carried the count of looks, the last look and the newest evidence: 131 647
writes in four days for twelve open conditions, against ADR 0289's rule that a
bucket is written only when what it describes changed. The looks now live in the
controller's store (migration 0095, condition_looks), the bucket holds what a
condition says and is written only when that changes, and the conditions verb and
the condition events put the looks back.
2026-10-12 12:11:13 +02:00
mesh-admin f549647d18 Merge pull request 'Give the unheld log test its own writer and memory, so it passes on a repeat (issue 486)' (#246) from fix/486-the-unheld-log-test-stands-alone into main 2026-10-12 10:10:39 +00:00
jochen 0a0f5b4a5a Require the service manager's reset-failed, wanted-by and unlink-dangling verbs now every holder serves them (issue 332)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestAVerbsOwnProcessNamesItsConnection (4.62s); and TestASeatsVerbGainsTheArgumentsTheBinaryNames
mesh/delivery superseded: a newer head of the same pull request
ADR 0246 step 3: the systemd module, the seat's only holder, serves all three on every node,
so a holder that stops serving one is refused rather than leaving a node without them.
2026-10-12 12:03:37 +02:00
mesh-admin 22a2f73f7b Merge pull request 'plans names the hold a waiting walk waits on (issue 529)' (#245) from fix/529-a-waiting-walk-is-answered-or-named into main 2026-10-12 09:55:17 +00:00
mesh-admin d854598d8c Merge pull request 'Wait out a forge not served for a moment when registering a build (issue 486)' (#244) from fix/486-the-forge-is-waited-out into main 2026-10-12 09:43:50 +00:00
jschoubben ac5d34f613 Give the unheld log test its own writer and memory, so it passes on a repeat (novox/hq issue 486)
mesh/delivery delivered
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
logUnheld kept what it had logged in package state and the test swapped os.Stderr, so a second run in one process logged nothing and failed.
2026-10-12 11:41:41 +02:00
jschoubben b03ec3b64e Offer no hand start for a walk that is only waiting its turn (issue 529)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/529-a-waiting-walk-is-answered-or-named delivered: every member is delivered
A hand start there would skip its group's order; point at it only for a
walk nothing holds, and say why nothing holds it.
2026-10-12 11:34:54 +02:00
jschoubben 4aacd98f8d Cap the forge wait as a whole, and wait out only a forge not served (novox/hq issue 486)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The registration holds the controller's one receive loop. Checking the bound
only between asks let an ask in flight hold it near two minutes, the
deaf-controller limit. The whole wait now runs under a 60 s deadline, still
says it waited when that deadline ends it, and a tool that did not answer in
time is not asked again: it has already held the loop its whole wait.
2026-10-12 11:32:14 +02:00
jschoubben ed037784e2 Name the hold a waiting walk waits on in plans (issue 529)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
mesh/delivery-group group fix/529-a-waiting-walk-is-answered-or-named checking: 0 of 2 member(s) ready
A walk whose delivery was held for a person said only that it waits for
mesh-delivery's word and that plans go starts it by hand, pointing the
reader at a hand start instead of the person's decision. Ask mesh-delivery
what holds each waiting walk and say the held delivery, its reason and
mesh-delivery.release, or the delivery not at its turn.
2026-10-12 11:27:05 +02:00
jschoubben 47205403d3 Wait out a forge not served for a moment when registering a build (novox/hq issue 486)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A registration asked the forge while node-tools re-served after a tool-runner
rollout, heard "nothing serves gitea.gitea_api", and failed a good build's
plan. The bus's not-served answer is now link.ErrNothingServes (and a timeout
link.ErrNoAnswer), and the registration asks again for up to 90 s on those
alone, never on a refusal; past the bound it fails saying how long it waited.
2026-10-12 11:23:03 +02:00
mesh-admin da1473e9db Merge pull request 'Say a unit's restart count as what it counts, not since its state began (issue 458)' (#243) from fix/458-restarts-say-what-they-count into main 2026-10-12 08:56:34 +00:00
jschoubben cde0dc287b Say a restart count runs from when the node-engine began watching it
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/458-restarts-say-what-they-count delivered: every member is delivered
The engine reports its own count, kept across recreates and starts by hand, not
systemd's NRestarts or the runtime's RestartCount (novox/hq issue 458).
2026-10-12 10:46:38 +02:00
jschoubben 63079e2c67 Say a restart count is automatic restarts since the last start or creation
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
NRestarts and RestartCount reset on a start by hand or a recreation, so
"whole life" was as wrong as "since" (novox/hq issue 458).
2026-10-12 10:45:40 +02:00
jschoubben 0d08d5c730 Say a unit's restart count is its whole life, not since its state began
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The count comes from systemd's NRestarts or the runtime's restart count; beside
"since" it read as thousands of restarts tonight (novox/hq issue 458).
2026-10-12 10:42:19 +02:00
mesh-admin 3bd984c41e Merge pull request 'Let a module keep a log the controller creates and never removes, and warn before a bucket or log is full (ADR 0297)' (#231) from adr-0297-module-logs into main 2026-10-12 00:42:55 +00:00
mesh-admin ed1b106c19 Merge pull request 'Say what relates two changes, before anything acts on it (hq issue 525)' (#242) from feat/525-what-relates-two-changes into main 2026-10-12 00:35:18 +00:00
jochen 6b77dd96f4 Merge remote-tracking branch 'origin/main' into adr-0297-module-logs
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-12 02:29:00 +02:00
jschoubben 9460689fff Relate a batch whose moves are not known to every change
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The review found that a merge kept without its moves, judged again at
the cut, gave its batch no modules, so it could read as unrelated to a
walk it will build beside. An unknown change is never unrelated.
2026-10-12 02:06:54 +02:00
jschoubben 96005a8bfd Say what relates two changes, so unrelated ones can walk beside each other
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
The helper decides, by repository, module (widened to dependents), delivery
group head or module graph edge, whether two changes are related, as novox/hq
ADR 0304 decision 1 reads in ADR 0306 decision 3. Nothing calls it yet; it is
the first step of issue 525's parallel walks.
2026-10-12 02:04:38 +02:00
mesh-admin b12da96906 Merge pull request 'Hold a leaf delivery to three minutes (hq ADR 0304 decision 4, issue 525)' (#241) from feat/525-leaf-budget-three-minutes into main 2026-10-12 00:02:24 +00:00
mesh-admin eba0a301cc Merge pull request 'Judge a leaf module on one healthy reading, a core module on three (hq issue 524)' (#240) from fix/524-judgement-by-class into main 2026-10-11 23:52:00 +00:00
jschoubben e6bd8f1dee Say a leaf delivery's budget as three minutes in the times verb's description
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/525-leaf-budget-three-minutes delivered: every member is delivered
2026-10-12 01:46:26 +02:00
jschoubben 5b9c81bf26 Hold a leaf delivery to three minutes (hq ADR 0304 decision 4, issue 525)
mesh/delivery-group group feat/525-leaf-budget-three-minutes checking: 0 of 2 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
ADR 0304 lowered a leaf module's delivery budget from five minutes to
three, the time its measured phases support once a leaf is judged on
one reading and waits only for what it shares.
2026-10-12 01:44:18 +02:00
jschoubben d8ec8baa22 Judge a send on no machine as the core, and test a send carrying the core with the real classes
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; it…
mesh/delivery delivered
The review found that a send whose first machine runs none of its
modules would pass at once with no reading, and that every mixed send
in the tests judged its leaf as the core.
2026-10-12 01:38:09 +02:00
mesh-admin 9c750a80de Merge pull request 'A provider's software-issued credential is taken from the provider, sealed (issue 526, ADR 0305)' (#239) from feat/526-a-provider-delivers-what-its-software-issued into main 2026-10-11 23:29:10 +00:00
jschoubben 4ee131e990 Let the controller ask an issued credential's tool in its own name, and seal it again over a remade pair (hq issue 526, ADR 0305)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/526-a-provider-delivers-what-its-software-issued delivered: every member is delivered
The bus left the controller out of the caller-named grants, so issued could never reach plex's tool; the
controller is now granted exactly the tools of offers whose software issues the credential, composed from
the records, so the installer's first user list is unchanged. A pair remade on a key change or accepted by
hand kept the issued stamp and was never sealed again; those writes now clear it.
2026-10-12 01:21:20 +02:00
jschoubben faf8b76b84 Cite ADR 0305's clauses by their accepted-for-review numbers (issue 526)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-12 01:09:26 +02:00
jschoubben ad3b4f185b Judge a leaf module on one healthy reading, a core module on three (hq issue 524)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; it…
mesh/delivery superseded: a newer head of the same pull request
ADR 0282 decided readings by class and ADR 0304 took a leaf module to
one, but the gate never learned it: every build, leaf or core, needed
three readings over at least two minutes, so a container module that
was healthy at once still waited about two and a half minutes on its
first machine. The number of readings and the settle time now follow
the module's class: three after two minutes for the controller's own
path and the resolver, one for every other module. Tests that judge a
gate's other rules keep leaf modules on three readings.
2026-10-12 01:08:14 +02:00
jschoubben db86dd52c8 Take a provider's software-issued credential from the provider, sealed (hq issue 526, ADR 0305)
mesh/delivery-group group feat/526-a-provider-delivers-what-its-software-issued rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
A token a media server got from its vendor reached its consumers only as a value the mesh minted, which
the server refused, or as a copy typed by hand. An offer may now say its software issues the credential;
the new issued verb carries no value and asks the provider's tool, in the controller's own name, for the
value sealed to one call, then seals it to every pair that does not hold it yet.
2026-10-12 01:07:16 +02:00
mesh-admin b26499bb9f Merge pull request 'The store-tests step fails on a malformed line, a failed go test and a shortened list (issue 515)' (#238) from fix/515-store-tests-step into main 2026-10-11 22:35:40 +00:00
jschoubben e25fdf776f Make the store-tests step fail on a malformed line, a failed go test and a shortened list
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; it…
mesh/delivery delivered
The step let tests escape it: a one-field line ran nothing, go test's exit
status was ignored, a pull request could drop entries main lists, and a
result written mid-line by go test -v read as NOT RUN, hiding a FAIL
(novox/hq issue 515).
2026-10-12 00:28:32 +02:00
mesh-admin e1953ad3f5 Merge pull request 'Let several modules declare one package, and refuse only present beside absent (hq ADR 0303)' (#237) from feat/303-shared-packages into main 2026-10-11 20:39:23 +00:00
jschoubben 6d2138217f Let several modules declare one package, and refuse only present beside absent (hq ADR 0303)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
Assigning the media-importer beside youtube was refused because both
declare ffmpeg. The node-engine installs a missing package and never
removes one a declaration stopped naming, so a second declaration
changes nothing on the machine; what two modules can contradict is one
declaring a package present and another absent, which stays refused.
2026-10-11 21:55:20 +02:00
mesh-admin 0e86664191 Merge pull request 'delivery-stop leaves merges out and walks the others again (hq issue 459, ADR 0299)' (#232) from feat/459-a-stop-re-walks-the-others into main 2026-10-11 19:31:57 +00:00
mesh-admin 92aa3fb179 Merge pull request 'A build-log bus test waits for the stream to store both lines (hq issue 507)' (#236) from fix/507-a-build-log-test-waits into main 2026-10-11 19:12:47 +00:00
mesh-admin c2abb3759a Merge pull request 'Stop tracking built binaries at the repository root (issue 470)' (#235) from fix/470-no-built-binary-is-tracked into main 2026-10-11 19:07:25 +00:00
jschoubben 0fb7de7ef8 Merge remote-tracking branch 'origin/main' into feat/459-a-stop-re-walks-the-others
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/459-a-stop-re-walks-the-others delivered: every member is delivered
# Conflicts:
#	testdata/store-tests
2026-10-11 21:06:47 +02:00
jschoubben bc4662d656 End the line a walk's ask leaves open as each ADR 0299 store test ends, so its pass is read (issue 459)
A walk's ask prints "  tier N: " and the build ask the rest; the tests' fake
ask prints nothing, so go test -v put three tests' --- PASS after it and the
store-tests step read them as not run.
2026-10-11 21:06:44 +02:00
mesh-admin ece3759647 Merge pull request 'Say each machine's last send in status and node show (hq issue 485)' (#234) from fix/485-the-last-send-is-said into main 2026-10-11 19:06:00 +00:00
jschoubben 8d73f705af Wait for the build's log lines to be stored before counting them (hq issue 507)
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A line is said with a plain publish, so a live subscriber can hold both
lines while the stream has stored one or none yet; the test read the
stream once and failed under load. It now waits up to 5 s for both.
2026-10-11 21:05:04 +02:00
jschoubben 0d4d94ff24 Stop tracking built binaries at the repository root, and ignore them (hq issue 470)
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
Built artifacts belong in build/; three binaries (51 MB) were committed by a local build and rode in every clone and build context.
2026-10-11 20:52:45 +02:00
jschoubben e2801773c3 Take an ordered engine's report as a send's answer only by its epoch and sequence (novox/hq issue 485)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A periodic report about an identical earlier send names the same digest and
can arrive after a later send the machine never received, so the digest and
the time read it as applied. The digest-and-time rule stays for engines that
report no order. Tests now hold the digest condition and the sequence one.
2026-10-11 20:44:54 +02:00
jschoubben fae6bd0a87 Say each machine's last send in status and node show, and as fields in status --json (novox/hq issue 485)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
The send record of issue 234 is now readable only through the controller's
verbs (issue 506), and only plan said it. status says per machine when it was
last sent a declaration, by whom, from which generation and what it answered;
node show says it in full with the refusal's words. A machine's answer is read
from its report only when that report is about the send and came after it.
2026-10-11 20:40:48 +02:00
mesh-admin afc7dd8f68 Merge pull request 'Fail the repo check when a listed store test did not run against the store (hq issue 459)' (#233) from feat/459-store-tests-are-proven into main 2026-10-11 18:31:45 +00:00
jschoubben 201bd28278 Keep a walk's withheld modules on its record, name every machine a stopped send reached, and list the stop's store tests (review of #232)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check fail: its merge-check.sh failed: 3 store test(s) listed in testdata/store-tests did not run and pass against the store
mesh/delivery-group group feat/459-a-stop-re-walks-the-others rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-11 20:25:20 +02:00
jschoubben 5446be1abb Merge remote-tracking branch 'origin/feat/459-store-tests-are-proven' into feat/459-a-stop-re-walks-the-others 2026-10-11 20:23:14 +02:00
jschoubben 932203df20 Leave the store tests' proof to the check of its own (#233) 2026-10-11 20:23:14 +02:00
jschoubben 949990f5a8 Merge remote-tracking branch 'origin/main' into feat/459-a-stop-re-walks-the-others 2026-10-11 20:23:12 +02:00
jschoubben 731143f5b9 Fail the repo check when a listed store test did not run against the store (hq issue 459)
mesh/merge-gate pass: builds build-agent → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; it…
mesh/delivery covered after its stop: a later merge that contains it was delivered: novox/mesh-controller@e2801773c300 (merged as ece37596 into main, wal…
A test that needs the store skips without one, and a skip passes unseen, so a
change's store tests could pass the check without ever running. Tests named in
testdata/store-tests must each run and pass, said by name.
2026-10-11 20:22:56 +02:00
mesh-admin b555e995b0 Merge pull request 'A walk sends the bus's user list alone before its gate, so a new grant is on the bus when the first machine is judged (issue 490)' (#230) from fix/490-grants-reach-the-bus-before-the-gate into main 2026-10-11 18:19:33 +00:00
jschoubben 309475bf6f Walk a stopped walk's merges again in the open batch, in one act, and say what stays on the machines (review of #232, hq issue 459)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the change's judges the pull requests after it merges; it…
mesh/delivery-group group feat/459-a-stop-re-walks-the-others ready: every member ready, and composed together they pass
mesh/delivery superseded: a newer head of the same pull request
A new batch beside the open one walked an older commit after a newer one, and a
stop that failed partway could leave a merge on two plans. A module shared with
the stopped merge, built at its branch, would deliver it; it is left out or
withheld. A batch not yet cut can drop a merge, so mesh-delivery can stop a
merge walked again. The stopped walk names what it left on which machines.
2026-10-11 20:12:27 +02:00
jschoubben fbfdded74b Let delivery-stop leave merges out and walk the others again, so a group stop ends only its own change (hq issue 459, ADR 0299)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery-group group feat/459-a-stop-re-walks-the-others ready: every member ready, and composed together they pass
mesh/delivery superseded: a newer head of the same pull request
One walk carries every merge of its batch, so ending it for one group member
ended every other delivery on it. delivery-stop now takes without: the walk
ends, every other merge it answered is batched again, and each later merge of
a left-out merge's repository is left out too, since it contains it.
2026-10-11 19:54:12 +02:00
mesh-admin 315cbd1f54 Merge pull request 'Issue 496: a definition's resolved directory and file paths are judged by the node-engine's rules where the definition is judged' (#228) from fix/496-a-resolved-path-is-judged into main 2026-10-11 17:49:40 +00:00
jschoubben eb8233ac7c Give the grants step's restic fixture a check beside its tool, as the module rules require (repo-check of #230)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A tool check alone is refused, so the store test's c2 never registered;
the fixtures are now parsed by a test that needs no store.
2026-10-11 19:43:50 +02:00
mesh-admin 3b017b228c Merge pull request 'Say a module assigned and left out of its machine's declaration as a condition (issue 380)' (#223) from fix/380-a-left-out-module-is-said into main 2026-10-11 17:36:58 +00:00
jochen bf82d30865 Keep a fill warning for an unreadable bucket or log only when its condition was read open
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
Second review of #231: when neither the stream nor its condition can be
read, nothing is known of its fill, so none is said; the unread warning
covers it. Only a condition read back as open is kept, with its own words.
2026-10-11 19:34:42 +02:00
jochen c432b9b5f6 Name the fill condition for buckets and logs, and judge each one on its own
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
Review of #231: "store" is the glossary's word for the postgres server, so
the condition is bucket-or-log-filling. And one stream that cannot be read
no longer fails the whole check: it is said as unread with its reason, an
open fill condition for it is kept, and every other bucket and log is
still judged (ADR 0297 §6).
2026-10-11 19:30:59 +02:00
jschoubben 325575b292 Keep the builds of the grants step's machine alone, not of every machine its composition resolves (repo-check of #230)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheGrantsStepSendsTheUserListAndNothingElse (2.14s)
mesh/delivery superseded: a newer head of the same pull request
Composing the bus's machine resolves the others on the same context, and
a machine never recorded as sent refused the whole composition.
2026-10-11 19:30:10 +02:00
jschoubben c385ed2a17 Merge remote-tracking branch 'origin/main' into fix/380-a-left-out-module-is-said
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-11 19:28:39 +02:00
jochen 7bd61332cb Let a module declare a log the controller creates and never removes (ADR 0297)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The issue tracker's history lasts seven days on the events stream and its
bucket fills toward a cap with no warning (issues 501, 502). A module may
now declare `logs`; the controller creates each as a stream on the bus
(create-or-update, never deleted or recreated, reported when undeclared),
issues it to the owner's assignments in their membership, grants the
runtime exactly the append and direct-read subjects, and raises a
condition when any module's bucket or log reaches 75% of its cap, cleared
below 70%.
2026-10-11 19:22:43 +02:00
jschoubben 0f853e93fa Exempt only the user list's content from the grants step's guard, and test the guard in the send itself (review of #230)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheGrantsStepSendsTheUserListAndNothingElse (1.69s)
mesh/delivery superseded: a newer head of the same pull request
The guard let any field of the list's resource move; its path or mode
changing is a change the step must not carry. The store test now calls
sendToEach, so removing the guard there fails it.
2026-10-11 19:17:28 +02:00
jschoubben 78205d7405 Judge the grants step on the declaration it sends, so nothing composed between the check and the send slips through (review of #230)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheGrantsStepSendsTheUserListAndNothingElse (2.55s)
The guard composed its own declaration ahead of the send; now sendToEach
judges the one it composed and stamped, and refuses it unsent.
2026-10-11 19:12:33 +02:00
jschoubben 538c4d5115 Merge main (#227) into the grants step's branch 2026-10-11 19:10:09 +02:00
jschoubben fbeffb608d Refuse the grants step when its send would change more than the bus's user list (review of #230)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheGrantsStepSendsTheUserListAndNothingElse (1.68s)
Kept builds are not the whole declaration: a new assignment, settings or
a provision's logins went out as the mesh holds them now. The step now
compares what it would send with the last send's summary and fails, said,
when anything but the user list differs; ungatedIn reads the kept builds
instead of passing the step unread.
2026-10-11 19:09:23 +02:00
mesh-admin fad655532d Merge pull request 'A declaration says the assignment generation it came from, and every send is recorded (issue 234)' (#227) from fix/234-a-declaration-says-the-generation-it-came-from into main 2026-10-11 17:08:47 +00:00
jschoubben e1367d185a Send the bus's user list alone before a walk's gate, so a new grant is on the bus when the first machine is judged (issue 490)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A walk left the machine holding the bus out of a gated send whenever a
build waited there for that same gate, so a module's new health tool was
refused by the bus on its first machine and the gate could not pass. The
grants step sends that machine its declaration with every build kept,
before the first machine's send; plans and the change plan say it.
2026-10-11 19:01:41 +02:00
jschoubben 0c675bcdb5 Merge remote-tracking branch 'origin/main' into fix/234-a-declaration-says-the-generation-it-came-from
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-11 19:00:24 +02:00
jschoubben 8115f1ac42 Judge a definition's resolved paths where the definition is judged, by the node-engine's own rules, so a refusal never freezes a machine (review of #228, issue 496)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-11 18:59:07 +02:00
jschoubben 93289dc88b Merge remote-tracking branch 'origin/main' into fix/496-a-resolved-path-is-judged 2026-10-11 18:56:02 +02:00
mesh-admin 9edc5c758d Merge pull request 'A controller test judges the runtime's daemon.json by the docker module's own keys, not live-restore alone (issue 498)' (#229) from fix/498-docker-gives-log-rotation-too into main 2026-10-11 16:52:51 +00:00
jschoubben 483c387b72 Judge the runtime's daemon.json against the docker module's own keys, not a frozen copy
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The test pinned daemon.json to live-restore alone, so every controller check failed once the
docker module gained log rotation as a preference (mesh-catalog #205, #217). It still refuses
dns (ADR 0196) and any key the module does not declare, and checks the log defaults where the
module declares them (novox/hq issue 498).
2026-10-11 18:46:31 +02:00
jschoubben 174c8b5d53 Judge every path the controller resolves for a directory or file where the plan is composed, so the merge gate refuses a directory in Docker's data (issue 496)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheResolverAndWhatAsksItComposeOnOneMachine (0.02s)
mesh/delivery superseded: a newer head of the same pull request
2026-10-11 18:19:22 +02:00
mesh-admin fe2e5e91b5 Merge pull request 'A check never sees the forge credential, and what it publishes is redacted (issue 462)' (#226) from fix/462-a-check-never-sees-the-forge-credential into main 2026-10-11 10:19:01 +00:00
jschoubben 5dd0e3c056 Raise S20 for every generation refusal, ask for a push when the counter was behind, and write a send's generation with its digest (hq issue 234 review)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheResolverAndWhatAsksItComposeOnOneMachine (0.03s)
mesh/delivery superseded: a newer head of the same pull request
A refusal of a send the mesh never recorded went only to stderr; it is now kept
with its sender said to be unknown, and raised. A counter behind the machine (a
store put back) is raised and the condition names push, since the receive loop
must not push. The node's digest and generation are one transaction, and a send
record that fails is said loudly without failing the send. The trigger ignores
an update that changes nothing, and the put-back mark is dropped: a put-back is
composed afresh with the current generation.
2026-10-11 11:55:27 +02:00
mesh-admin 51db0b5273 Merge pull request 'broker: each credential may call tools only in its own name on the caller-named subjects (hq issue 365)' (#224) from fix/365-a-tool-call-names-its-caller into main 2026-10-11 09:31:36 +00:00
jschoubben 313efa826c Say in every declaration the assignment generation it came from, and record every send (hq issue 234)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A declaration newer in sequence than every other named four fewer modules
than the assignments held, a machine applied it, and nothing the mesh kept
said who sent it or from what view. The store now raises an assignment
generation in the same transaction as every assignment change (a trigger,
so a cascade counts too); a send reads it before composing, carries it to
engines that said they read it, marks a gate's put-back, and is recorded
with its sequence, sender, generation and modules. The would-send is
stamped with what was last sent, so nothing reads behind for it; a refusal
is kept on the send it refused and raised as S20 naming the sender; plan
says what the machine was last told.
2026-10-11 11:30:16 +02:00
jschoubben 72fde0ee1a Capture the SDK's conformance fixtures at the commit the node-engine's pin moved it to (hq issue 449's rule) 2026-10-11 11:30:16 +02:00
jschoubben 5c4fa43f8b Leave no package-registry credential or clone userinfo in the workspace a check mounts (issue 462)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A build wrote .npmrc into its source tree and never removed it, and its clone
recorded the URL's userinfo; a later check's container mounts the workspace as
HOME. The .npmrc and the tree now go when the build ends, clones record their
URL without userinfo, and a check first removes any .npmrc an older builder left.
2026-10-11 11:24:58 +02:00
jschoubben 7bd04342b6 Pin the node-engine at its pull request's generation refusal, so the validator reads a declaration's generation (hq issue 234) 2026-10-11 11:23:12 +02:00
jschoubben c494ed03a7 Keep the forge credential out of what a check's container sees, and redact what a check publishes (issue 462)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
The toolchain container mounts the workspace as HOME, so the credential kept
there, and a clone's recorded userinfo, were readable by any pull request; its
output is kept on the bus for days. The credential now lives in a private
directory outside the workspace only while cloning, clones record their URL
without userinfo, and every line said to the build's log and the verdict
passes a redactor copied from the journal tool (sharing it: issue 471).
2026-10-11 11:20:10 +02:00
mesh-admin 02c61b983c Merge pull request 'A build waits out a registry held still, and a retry asks every failed build of the tier (issue 457)' (#225) from fix/457-a-build-waits-out-a-registry-pause into main 2026-10-11 09:17:09 +00:00
jschoubben c14fe2776c Restore the tracked controller binary a local build overwrote, and hold that D1 never clears a wait (review of #223)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The binary in 419d662 came from a build into the worktree, not from the
change. node show says send, as the glossary does. A test now reconciles D1
beside a wait's needs-operator, which a wait raised under D1's source would fail.
2026-10-11 11:14:03 +02:00
jschoubben 419d662ad8 Use the glossary's words, say which modules raise a condition, and judge left-out modules in D1 (review of #223)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
The operator reads manifest, declaration and send, not definition,
composition and update. D1 already resolves every machine, so the
left-out judgement rides on it instead of resolving each machine again.
A test holds that a wait's own needs-operator still clears beside it.
2026-10-11 11:10:22 +02:00
mesh-admin 3b6b54a501 Merge pull request 'The SDK conformance test reads the SDK the controller pins, never a desktop's checkout (issue 449)' (#220) from fix/449-the-conformance-test-reads-what-it-carries into main 2026-10-11 09:08:57 +00:00
jschoubben 83ce19b9c0 Say a registry came back only when the call worked, and retry from toRetry's set (issue 457 review)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The streaming blob PUT is left unwaited, with why, since its body cannot be
read twice and the POST before it already waited.
2026-10-11 11:05:30 +02:00
jschoubben 7758301444 Ask every failed build of the tier on a retry, not only the first (issue 457)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
An outcome arriving after its plan failed is kept only in the build records,
so a retry read from the plan alone asked one failed build and the records
then failed the plan again on the next. Settle the tier from the records first.
2026-10-11 10:51:41 +02:00
jschoubben 094d3d5bc6 Wait out a registry held still, for up to five minutes, instead of failing the build (issue 457)
The store's nightly collection stops the registry for about a minute and a
half; builds in that window failed on connection refused and failed their
whole plans. A refusal is now waited for with a growing pause, said in the
build's log, and still fails the build past the bound.
2026-10-11 10:51:41 +02:00
mesh-admin e7bcc107c8 Merge pull request 'A failed repository check names what failed, from its whole output (issue 460)' (#222) from fix/460-a-failed-check-names-what-failed into main 2026-10-11 08:47:56 +00:00
jschoubben ebca7816bf inventory: a person's one tool is granted naming that person as the caller too (hq issue 365)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/365-a-tool-call-names-its-caller delivered: every member is delivered
2026-10-11 05:47:40 +02:00
jschoubben 9bed7d6398 broker: grant each credential the tool calls that name it as the caller, and no other (hq issue 365)
Every grant to call a tool is granted again under the call kind, with the
credential's own bus user as the last token, and every grant to answer one is
granted for calls naming any caller: the caller of a tool call becomes a fact
the bus enforces, as ADR 0259 section 3 made the asker of an ask. A kind of its
own because every existing tool grant is a wildcard that would match any caller
appended. The old tool grants stay for one release so nothing loses its way to a
tool (hq issue 464); the controller's own grants move with that issue, since
they are also the installer's first user list.
2026-10-11 05:25:29 +02:00
jschoubben a330c564ba Say a module assigned and left out of its machine's composition as a condition (issue 380)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A push leaves out a module whose settings do not compose and sends the rest,
which is right, but only plan said so: nfs-server ran nowhere for days while
the operator believed it ran. The self-check now raises needs-operator for a
setting nobody gave, naming the setting and the command, and left-out for any
other cause; both warnings, cleared once the module composes or is unassigned.
status and node show list the same modules as assigned, not applied.
2026-10-11 04:43:58 +02:00
mesh-admin 0a2e58c070 Merge pull request 'Deliver again an ask the controller made, removing the original from its queue (issue 334, part)' (#219) from fix/334-a-given-up-ask-can-be-delivered-again into main 2026-10-11 02:36:26 +00:00
jschoubben 1747e33923 Name what failed in a repository check from its whole output, not its tail (issue 460)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A long run's logs pushed the --- FAIL lines out of the last 200 lines the
summary was named from, so a failed check could not say which test failed.
Pick the lines that name a failure as the output streams, keep them whole at
the end of the verdict's report, and say the whole output in the build's log.
2026-10-11 04:24:34 +02:00
jschoubben 9b6acf0ef5 Read the captured SDK, saying so, when the seat placed no clone beside the check (issue 449)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The seat runs the running controller's besideRefs, which clones no mesh-sdk until this
change has rolled out, so a missing clone failing the test kept this change from ever
passing its own check. A follow-up makes it a failure again after the rollout.
2026-10-11 04:20:54 +02:00
mesh-admin 45b4ae92bc Merge pull request 'A provider whose wait fails its check lists who waits on it (issue 450)' (#221) from fix/450-a-provider-whose-wait-fails-lists-its-waiters into main 2026-10-11 02:19:02 +00:00
jschoubben 4f5fab81fe Read the SDK fixtures at the pin of the tree under check, not the running controller's (issue 449 review)
A pull request moving the SDK passed its check against the old SDK and then failed
everywhere once it rolled out. In a merge check the test now reads the clone's history
at the tree's own go.mod pin, and holds the captured copy to the clone byte for byte.
2026-10-11 04:19:01 +02:00
jschoubben 4632b6a508 Judge the SDK conformance fixtures against the SDK the controller pins, never a desktop's checkout (issue 449)
A check clones mesh-sdk beside the controller at the commit go.mod pins; elsewhere
the test reads a copy captured at that commit, held to go.mod. A missing clone fails
instead of skipping.
2026-10-11 04:19:01 +02:00
mesh-admin 9d6a12eb53 Merge pull request 'Say an unanswered merge check's time in the operator's zone (issue 443)' (#218) from fix/443-a-stalled-lines-times-are-local into main 2026-10-11 01:51:53 +00:00
jschoubben 984d85865d Give the words' zone through a seam, so no test writes time.Local under the race detector (novox/hq issue 443)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/443-a-stalled-lines-times-are-local delivered: every member is delivered
2026-10-11 03:41:14 +02:00
jschoubben ed45cc6415 Check a provider's waits before saying who waits on it (issue 450)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A provider whose wait fails the check raises unhealthy, but the holding read
its stored statement with the wait unchecked: sayWaiters built the
needs-operator key, found it not open and listed no held consumer. The
holding now reads each statement as judged (ADR 0283 decision 3), in
sayWaiters and in the provider's state its consumers are held by.
2026-10-11 03:29:10 +02:00
jschoubben ef551fdfb6 Remove an ask's original only when its sequence still holds it, and only with a worker (issue 334)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A seat's queue made again numbers from one, so an old dead letter's sequence
can name a live ask; deleting by number alone would drop it silently. An ask
with no worker would wait unseen while counted as delivered.
2026-10-11 03:24:52 +02:00
mesh-admin 7493bd8f18 Merge pull request 'A provider waiting for the operator holds its consumers, and a wait beside another condition is said (issue 405)' (#216) from fix/405-a-waiting-provider-holds-its-consumers into main 2026-10-11 01:23:35 +00:00
mesh-admin 8305e4e3d1 Merge pull request 'A test that reads another repository judges what the check clones, never the desktop's checkout (issue 432)' (#217) from fix/432-a-test-reads-what-it-carries into main 2026-10-11 01:21:14 +00:00
mesh-admin fbc7bc976b Merge pull request 'make check runs merge-check.sh, so it and the gate agree (issue 431)' (#215) from fix/431-make-check-runs-what-the-gate-runs into main 2026-10-11 01:20:19 +00:00
jschoubben f510b46319 Deliver again an ask the controller made, removing the original from its queue (issue 334)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A build ask its seat's worker gave up on could only be dropped, though the
controller already holds the publish on that seat's accepts and the stream
API to remove the original. Asks to other seats stay refused: delivering them
needs a grant ADR 0264 withholds, in the asker's name ADR 0259 protects.
2026-10-11 03:18:34 +02:00
jschoubben 926fde2fda Say an unanswered merge check's time in the operator's zone, from the checks given as data (novox/hq issue 443)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check fail: its merge-check.sh failed: FAIL github.com/novox/mesh-controller/cmd/mesh-controller 268.072s
mesh/delivery-group group fix/443-a-stalled-lines-times-are-local rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-11 03:10:30 +02:00
jschoubben d5cf3b42fe Say a waiting provider in the operator's words, and which state it is when a consumer is held (issue 405 review)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1791681844507629611 (novox/mesh-controller at 7493bd8f into ma…
2026-10-11 03:05:34 +02:00
mesh-admin 47c7877e8d Merge pull request 'A consumer's max-deliveries condition has one watcher and counts what was given up (issue 440)' (#214) from fix/440-one-key-one-watcher into main 2026-10-11 01:04:28 +00:00
jschoubben 68fbd99e89 Say how a check's clones are chosen and what the captured copy carries (issue 432 review)
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-11 03:00:28 +02:00
jschoubben 9a37c143e4 Keep a waiting provider's hold within ADR 0283 (issue 405 review)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
A consumer held under a provider that only waits for the operator now
passes its gate as a wait for a person, carrying the wait, so no walk
waits on the operator's secret. Only consumers of the waiting part are
held; one that cannot be matched is raised on its own and says its
provider waits. needs-operator stays a warning while consumers wait.
2026-10-11 03:00:27 +02:00
jschoubben eb72075104 Judge tests that read another repository against what the check clones, never the desktop's checkout (issue 432)
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
Tests that read ../../../mesh-catalog or ../../../mesh-host gave a verdict
that depended on what sat beside the checkout: a stale or dirty sibling
failed them on a desktop, and a missing one skipped them unseen. They now
read the clone the build seat puts in MESH_CHECK_BESIDE, failing when it is
absent there, and elsewhere a copy captured at a named commit.

The skip had hidden that the builder test read a module retired by ADR 0190.
The systemd reading test no longer counts the machine's own environment.d.
2026-10-11 02:55:04 +02:00
jschoubben d6b867a87a Restart what reads a secret family member when the member is given again (issue 405)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheSecondControllerWaitsAndTakesAHigherEpochOnHandover (4.67s)
mesh/delivery superseded: a newer head of the same pull request
secretsReadBy looked only at secrets declared by name, so a container
mounting a member kept the value it started with.
2026-10-11 02:51:11 +02:00
jschoubben 1dc9961c43 Hold consumers under a provider waiting for the operator, and say a wait beside another condition (issue 405)
A provider whose only part not healthy waits for the operator's secret or
setting let its consumers raise their own unhealthy conditions, and a
wait beside a relogin, a directory used as found or a fault was not said
until the other cleared.
2026-10-11 02:51:11 +02:00
mesh-admin 11ffab887b Merge pull request 'Say an unanswered merge check in its own words, with no action it cannot take (issue 438)' (#213) from fix/438-a-check-that-never-answered-is-said into main 2026-10-11 00:47:10 +00:00
jschoubben cfbbad8209 Count a dead letter by when it was kept, so one kept late still counts (issue 440)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The give-up time is not newer for every letter kept: a notice that could
not be kept is offered again a minute later, so a later give-up can be kept
first and the one after it read as not fresh. The stored time rises with the
stream's sequence. A consumer whose letters vanish between the two reads is
left out of the look instead of counted as a look, and the skip of a
token-less max-deliveries advisory now has a test of its own.
2026-10-11 02:36:58 +02:00
jschoubben d2e9dc6159 Inline the check's teardown so make -n check stays a dry run
mesh/merge-gate pass: builds build-agent → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-controller@ef551fdfb6b2 (merged as 0a2e58c0 into main, walk plan-17916861…
GNU make runs a recipe line holding $(MAKE) even under -n, so a dry run of
check ran the whole suite. novox/hq issue 431.
2026-10-11 02:35:37 +02:00
jschoubben 5557a01f06 Make check run merge-check.sh, so a developer's check and the gate cannot drift
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
make check listed its own steps, and its gofmt walked vendor/, failing on
third-party files no change could fix; the steps after it never ran. It now
raises the throwaway database and runs the script repo-check runs.
novox/hq issue 431.
2026-10-11 02:33:30 +02:00
mesh-admin 5bddb16514 Merge pull request 'A misspelled placeholder is refused, never written out as text (issue 231)' (#211) from fix/231-a-misspelled-placeholder-is-refused into main 2026-10-11 00:31:46 +00:00
jschoubben 671e350f56 Name the build queue and the merge check as the glossary does (issue 438)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/438-a-check-that-never-answered-is-said delivered: every member is delivered
Review of #213: the queue is the controller's build queue, not the build seat's,
and the merge check is the pair, so the headline says the pull request's merge
check has not answered.
2026-10-11 02:30:19 +02:00
jschoubben f83fcdc15f Give a consumer's max-deliveries key one watcher, counting what was given up (issue 440)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The dead-letter row and a max-deliveries advisory without a token both said
bus.<stream>.<consumer>.max-deliveries: each look added an observation and a
line of evidence through the row, and the two overwrote each other's words.
The row owns the key since issue 330, so the advisory watcher leaves it, and
the row now says when the newest held message was given up, so a letter held
for a day no longer reads as observed every 30 seconds. Times without Happened
is refused, since the raise ignored it and an update counted it.
2026-10-11 02:30:09 +02:00
jschoubben 585caa4371 Say an unanswered merge check in its own words, with no action it cannot take (issue 438)
mesh/delivery-group group fix/438-a-check-that-never-answered-is-said checking: 0 of 2 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh-delivery now says a head whose merge check started and never answered as a
stalled line in state unanswered. Through the generic delivery words it would read
as a delivery and offer a Stop that mesh-delivery refuses, since no delivery of
the head exists; it now names the checks waited on and what the operator can do.
2026-10-11 02:25:11 +02:00
mesh-admin f8d08b0637 Merge pull request 'A bus refusal is counted by what the bus said, not by the controller's looks (issue 402)' (#212) from fix/402-a-refusal-is-counted-once into main 2026-10-11 00:21:00 +00:00
jschoubben dc62fd0075 Let a shell parameter operator after a known word pass, so ${PORT:-8080} is not refused (issue 231)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The case-insensitive rule for the mesh's namespace words took ${PORT:-8080},
${SHELL:-/bin/sh}, ${SECRET:?unset} and ${dir:-/tmp} for misspellings, though they
are among the commonest lines of a script or env file. A :-, :=, :+ or :? after the
colon is the shell's, whatever the word's case.
2026-10-11 02:20:43 +02:00
jschoubben 525b2c1a11 Sweep the definition, not the filled values, and judge each field alike at check and composition (issue 231)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
The first sweep ran at composition over the resource after settings, bindings and
machine facts were filled, so an operator's value such as ${labels:instance} was
refused as a misspelling its author never wrote, and the whole machine got nothing.
Both points now sweep the resource as the manifest wrote it, against one table of
which fields each pass fills, so the check and composition refuse the same things.
Any ${<known namespace>: its pattern does not take is refused whatever its case or key.

Issue 231's undeclared-setting half is not met here: refusing a key the module does
not declare (ADR 0164) is not built and is handed to issue 398.
2026-10-11 02:15:37 +02:00
jschoubben c11222026a Count a bus refusal by what the bus said, not by how often the controller looked (issue 402)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
S9 reads the one remembered advisory again on every look for an hour, and
each look was kept as an observation and a line of evidence, so a single
refusal read as one repeated every 30 seconds. An observation may now say
when what it reads happened and how often; the keeper counts that, and a
look with nothing newer adds nothing. Sources that look at the present
keep counting their looks.
2026-10-11 02:10:48 +02:00
jschoubben 360c318e85 Refuse a placeholder no pass consumes, so a misspelling never reaches a machine as text (issue 231)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-11 02:06:14 +02:00
mesh-admin f008fab9d8 Merge pull request 'A kept call holds a command's first word, never the line (issue 397)' (#210) from fix/397-a-kept-call-holds-no-command-line into main 2026-10-10 23:23:49 +00:00
jschoubben 15a9919df5 A kept command's first word is parted by any whitespace, and capped (review of issue 397)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The review of #210 found that the fix left the hole open and widened
another.

A command's words were cut on a space alone, while the verb's own
splitter parts them on a space, a tab or a newline. The same line
written with tabs found no space, so all of it was kept — the very hole
this closes. Its words are now parted as the splitter parts them.

And because the command arm sits above the arm that caps a string at
120 bytes, a command kept whole was no longer capped: for a 300-byte
single token the change kept more than the code it replaced. The first
word now carries the same cap.

A one-word command is still kept whole, but the comment no longer
claims it carries nothing to withhold: the record is written before the
verb judges the line, so one word may be a token or compact JSON. The
cap is what bounds that.

The test now says the whole of what is kept for each line, which is
also what proves the rest is gone, and looks for forbidden words as
whole words rather than as substrings — so "set" is back in the list,
and "show" cannot hide in a word such as "shown". All eight cases fail
against the unfixed code.
2026-10-11 01:04:40 +02:00
jschoubben 1390836b73 A kept call holds a command's first word, never the line (issue 397)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
kept() withheld the arguments line, values, secret and stdin, and kept
the command argument of the generic command verb verbatim. A line such
as `settings set <module> '<value>' --node <node>` therefore put the
value into the calls record, which answers anyone who may call the seat.

It is now kept as a mesh-cli line is: its first word, with the rest said
to have been given. A command of one word is kept whole, since there is
nothing after it to withhold, and one that is not a string is kept as
"(given, not kept)".

The record as it stands holds no such line: its whole answer, read at
2026-10-10 22:52 UTC, carries no call of the command verb. That says
nothing of calls older than its window.
2026-10-11 00:53:37 +02:00
mesh-admin dac7e5f7f6 Merge pull request 'Say a part waiting for the operator as needs-operator and pass its gate; add secret families (issue 386, ADR 0283)' (#209) from feat/386-a-wait-for-the-operator into main 2026-10-10 19:49:29 +00:00
mesh-admin c06a2cd972 Merge pull request 'Say a walk's phases out of order unknown, never a negative time (hq issue 382)' (#208) from fix/382-phases-out-of-order into main 2026-10-10 19:36:52 +00:00
jochen d5f6b67b94 Say a part waiting for the operator as needs-operator and pass its gate; add secret families (issue 386, hq ADR 0283)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A module waiting for the operator's secret failed its first-node gate, held
every later walk and was said as 'nothing for you to do'. The node-engine's
new waiting state is checked against the manifest and the secrets given,
read by the gate as a wait for a person, and raised as needs-operator naming
the act. A secret family gives each part its own one-line secret, which the
mesh never makes, so the desk prompt can take each password.
2026-10-10 21:19:58 +02:00
jochen 12d4025d30 Say a walk's phases out of order unknown, never a negative time (hq issue 382)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
#206's own walk kept its first send 783 ms before its build, and delivery
walks said send-first measured at -783 ms. A moment recorded before the
one before it now makes both phases unknown: the earlier one's time joins
the unknown time, the later one has none, and the walk goes on from the
later moment. Measured phases and unknown time still add up to the total.
2026-10-10 21:15:48 +02:00
mesh-admin 1ca4d8ce60 Merge pull request 'Test that times is optional on the delivery seat (hq issue 382)' (#207) from test/382-times-optional into main 2026-10-10 19:13:56 +00:00
jochen a4f93b52a8 Test that a delivery seat holder without times still holds the seat, and one serving it is not refused (hq issue 382, review of #206)
mesh/merge-gate pass: builds build-agent, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 o…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-10 20:55:43 +02:00
mesh-admin a2a671adb7 Merge pull request 'Keep each walk's phases and say a delivery over its budget (hq ADR 0282 slice 1, issue 382)' (#206) from feat/382-walk-phases into main 2026-10-10 18:49:18 +00:00
jochen 412f11b079 Mark times optional on the delivery seat until mesh-delivery serves it (hq ADR 0282, design 33 §7)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-10 20:35:24 +02:00
jochen 5d4eb974ab Promise times among the mesh-delivery seat's verbs, so its holder may serve it (hq ADR 0282)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.75s)
mesh/delivery superseded: a newer head of the same pull request
2026-10-10 20:22:34 +02:00
jochen 6805e2bcb3 Walk phases: a report past the silent bound never moves a walk's end; keep phases outside the hold on the plans; first-node gate in words (review of #206)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.71s)
2026-10-10 20:20:07 +02:00
jochen c640341c50 Keep each walk's phases and say a delivery over its budget (hq ADR 0282 slice 1, issue 382)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.62s)
The operator's budget (a leaf module running everywhere within five minutes
of its merge, a core module within ten) cannot be held to without knowing
where a walk's time goes. A walk now keeps when its batch's window closed,
when it was cut and its class (migration 0093), each gate reading, and what
each machine of the rest was sent; 'delivery walks' and plan-moved say its
phases from the merge to every machine of the rest reporting the build
applied, and ended walks keep them as walk-phase durations per class. Probe
D16 reads mesh-delivery's 'times' and raises delivery.<class>.over-budget.
A phase that cannot be measured is said unknown, never zero. Measurement
only: no walk is held, sent or judged differently.
2026-10-10 20:15:01 +02:00
mesh-admin 690b75f659 Merge pull request 'Say where a command line's quote is left open and how a quote is written (hq issue 294)' (#205) from fix/294-quote-in-a-quoted-value into main 2026-10-10 17:21:57 +00:00
jochen d52de218c8 Quote none of the line in the unclosed-quote refusal: a refusal is kept on the bus as the call's answer (hq issue 294)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-10 18:39:20 +02:00
jochen 6188e6b1da Say where a command line's quote is left open and how a quote is written (hq issue 294)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
An apostrophe inside a single-quoted value ends that quote, and the line then failed as a bare
'unclosed quote' with no position and no way out named. The splitter's rules stay the shell's; the
refusal now names the character, shows the line from there and gives the two ways to write a quote.
2026-10-10 18:36:47 +02:00
mesh-admin 798738cc12 Merge pull request 'A shrink is read against the item's own path: a moved directory starts its size history again (hq issue 368)' (#204) from fix/368-data-shrank-path-change into main 2026-10-10 15:56:12 +00:00
jochen babfef4f3a Keep one reading when two paths are measured at one moment, rather than failing the machine's record (hq issue 368 review)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-10 17:48:49 +02:00
jochen 96b0966ad8 Read a shrink against the item's own path, so a moved directory starts its size history again (hq issue 368)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A reading now keeps the path it was measured at, and the peak is read only from
readings at the item's path now. Before, an agent's home that moved to its own
account was compared with the operator's home it left, and data-shrank was
raised for data that was never lost. Existing readings take their item's path
now, so a shrink that is real stays raised.
2026-10-10 17:42:46 +02:00
mesh-admin 4d30b5de13 Merge pull request 'A module's own secret is asked for by a verb and typed at the desk, bounded, the prompt naming who asked (hq ADR 0277)' (#201) from feat/0277-secret-ask into main 2026-10-10 13:34:20 +00:00
jochen c97942d591 A module's own secret is asked for by a verb and typed at the desk, bounded, the prompt naming who asked (hq ADR 0277)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The give verb opened the desk's hidden prompt for a module's own secret, with the desk named. Now the
secret-ask verb (secret ask <node> <module> <name> [--at <desk>]) opens the same prompt from anywhere on
the mesh, with the desk the module's machine unless named, and give composes the same line. Every ask
is recorded in the store before the prompt opens (migration 0091): one open ask per secret, three an
hour, so an agent cannot keep a prompt in front of the operator. The prompt names who asked, from the
bus's word on the caller cut to a name's characters, never an argument of the call. The value stays
typed at the desk, sealed to the one call and then to the module's machine, never in an argument, a
log or an event; a secret the mesh makes itself and a trusted party's secret are refused as before.
2026-10-10 15:24:08 +02:00
mesh-admin 9c69b9da17 Merge pull request 'An own-path merge never shares a batch with a catalogue merge, and plans names a walk by what it moves (tracker issues 377, 378)' (#200) from fix/377-378-own-path-batches-and-named-plan-lines into main 2026-10-10 13:14:37 +00:00
mesh-admin 00065dbdaf Merge pull request 'A trusted setting is proposed through the settings verb and set only on the operator's warrant (hq ADR 0277)' (#199) from feat/0277-settings-propose into main 2026-10-10 12:59:13 +00:00
jochen 16362e1bbd A deferred walk is one whose note says so: a failed first ask is watched as before (review of #200)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-10 14:51:27 +02:00
jochen 3496b58f66 A walk let go behind another reads as a wait: its note on the line, never late, and no tier of it watched (hq ADR 0276 review of #200)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
2026-10-10 14:47:11 +02:00
mesh-admin d1cc9b4e20 Merge pull request 'Ask again after a refusal, with a growing wait (issues 369 and 373)' (#198) from fix/373-one-message-per-condition into main 2026-10-10 12:45:07 +00:00
jochen cc11de2513 A trusted setting is proposed through the settings verb and set only on the operator's warrant (hq ADR 0277)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
Issue 339 made every trusted setting the controller's terminal's alone, so an agent could only hand the
operator a line to type at the control node. Now anyone the bus admits may PROPOSE a layer: settings
propose keeps the proposal in the controller's own asks (the asked bucket, which the controller alone
writes), judged as settings set judges, and asks the operator on the operator channel at the level
approve with every key, its exact new value (in its shape where a path or an address may not leave the
mesh), the was of a changed key, the removed keys and the layer's fingerprint. The serving controller
sets the layer on the warrant alone: once, for the ask it holds, only when the record's values still
digest to what the option bound and the layer is still the one shown, with the terminal's judgement and
history, and keeps who approved it beside the layer, which settings says back (migration 0090). Decline,
expiry, a cancel, a replacement or the router's refusal discard it; nothing is asked when no router, no
grant or no channel can carry it. settings proposals lists them. The push afterwards is a separate act.
2026-10-10 14:42:40 +02:00
jochen 04fcce2fcc An own-path batch is cut first and folds no waiting walk; a walk let go beside a started one starts once it ended; a late catalogue merge is not answered by a walk that waited for nobody (hq ADR 0276 review, tracker issue 377)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-10 14:33:10 +02:00
mesh-admin b9ceeace41 Merge pull request 'Merges are assembled in a rolling window, and each batch is walked once (hq ADR 0276, issue 362)' (#197) from fix/362-a-walk-answers-every-merge-it-contains into main 2026-10-10 12:26:20 +00:00
jochen 2e0a7d1cbd Review: an answered retry never brings its old refusal back (issue 369)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/373-one-message-per-condition delivered: every member is delivered
A refusal stood for its part whenever no ask was open, so after the retry
was taken and answered, "Questions for you not delivered" came back with
the old words for an hour, then again after each answer. A refusal is now
current only while no ask was made after it that the router did not refuse;
the verdict wait applies only to that newer ask. Tests reconcile inside the
wait and after an answer.
2026-10-10 14:14:07 +02:00
jochen 3ced96fc6f A merge on the controller's own path never shares a batch with one that waits for the delivery's word (hq ADR 0276, decided during the build)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-10 14:12:18 +02:00
jochen d3b4549611 Ask again after a refusal, with a growing wait (issues 369 and 373)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
mesh/delivery-group group fix/373-one-message-per-condition ready: every member ready, and composed together they pass
The router refused an ask while its channels had not yet said they could
send; both could twenty minutes later, and the controller repeated that
refusal every minute for eleven hours, escalating "Questions for you not
delivered" on it, because a refused ask was asked again only when the
channels' claims changed.

A refused ask is now asked again after 1 min, doubling with each refusal
in a row, at most 30 min, and at once when the channels change. While the
router's word on an ask made again is awaited (2 min), the condition
stands as it was, so it neither flaps nor clears early; once the ask is
taken it clears; a new refusal is said in its own words. Its explanation
no longer says its verdict twice.
2026-10-10 14:02:59 +02:00
jochen 065cfa0d1d Owe a merge to the record from its branch's first kept merge (hq ADR 0276 review)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
2026-10-10 13:54:40 +02:00
jochen 0e0e143055 Keep a merge a cut made history, and build a batch's walk at the branch (hq ADR 0276 review)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
2026-10-10 13:48:31 +02:00
jochen 2887691414 Walk the earlier merges of a failed walk as news (hq ADR 0276 review)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check fail: its merge-check.sh failed: FAIL github.com/novox/mesh-controller/cmd/mesh-controller 256.824s
mesh/delivery superseded: a newer head of the same pull request
2026-10-10 13:41:42 +02:00
jochen e1499d4196 A late merge is named however its modules read since the cut; a walk builds the commits it carries (hq ADR 0276 review) 2026-10-10 13:39:21 +02:00
jochen 96fc4209d3 Assemble merges in a rolling window and walk each batch once (hq ADR 0276, issue 362)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
Every merge opened a walk and the next merge of the branch superseded it: two
catalogue merges 18 s apart left a walk no delivery held, and the operator
started it by hand 58 minutes later. A merge now joins the open batch, kept in
the store (migration 0089), which is cut into one walk when no merge came for
merge-window (90 s) or at merge-window-at-most (10 min): one commit per
repository, the latest of its branch, with every file the batch's merges
changed. One walk at a time; a started walk is never superseded, a waiting one
is folded into the next. The walk names every merge it answers on the wire
(delivery.merges, taken_over_by, batch). A failed walk walks its earlier merges
alone, newest first, until one is delivered. A delivery group's order becomes
tier edges inside the walk. plans shows the batch assembling; S18 and S19
bound its waits; S16 names the merges a waiting walk answers.
2026-10-10 13:20:04 +02:00
mesh-admin 8a53532d89 Merge pull request 'A plan says why each module is in it (hq ADR 0267, issue 363)' (#196) from fix/0267-a-plan-says-why-each-module-is-in-it into main 2026-10-10 11:04:07 +00:00
jochen d0161fac52 Say a deleted module's reason, read a context's reason at the merged branch, and keep the snapshot's bytes (hq ADR 0267, review)
mesh/delivery delivered
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
2026-10-10 12:56:22 +02:00
jochen 887de9b5f2 Say why each module is in a plan: its build source's changed files, or why it is read whole (hq ADR 0267, issue 363)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
A what-if named the build seat's holder as packaging the controller's
source, "rebuilt without their own source moving", while it was in the
plan because an open plan had not built it yet. The what-if and the merge
log now say, per module, which changed files of its build source moved it,
or that it is read whole and why: no build source recorded, a newer build
failed, or a plan has not built it yet.
2026-10-10 12:49:44 +02:00
jochen 1560c498b6 Ask the rollback test's failed build after the registered one, whenever it runs
Its id named 2026-10-10 02:40 UTC; once the clock passed that, the
registered build read as newer and the test failed on main.
2026-10-10 12:49:44 +02:00
mesh-admin df6d72aec2 Merge pull request 'Record a module's act on the operator's warrant from the router's own record (hq ADR 0274)' (#194) from feat/warranted-hand-act into main 2026-10-10 10:39:34 +00:00
mesh-admin a6bc0936e1 Merge pull request 'The gate's module check notes a seat another module declares, which it was not given (hq issue 364)' (#195) from fix/364-a-touched-manifest-uses-a-seat-another-module-declares into main 2026-10-10 10:06:27 +00:00
mesh-admin 513ebd0577 Merge pull request 'A merge moves a module only when its build source holds a changed file (hq ADR 0267, issue 363)' (#193) from fix/0267-a-merge-moves-what-its-build-source-holds into main 2026-10-10 02:00:18 +00:00
jochen 88e84a4dd7 warranted says what the operator chose, never that the module acted (hq ADR 0274 review)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
2026-10-10 03:55:44 +02:00
jochen 7b5c063cd3 The gate's module check notes a seat another module declares, which it was not given (hq issue 364)
mesh/delivery delivered
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
The gate passes only the manifests a change touches, so a module that starts using the operator channel
was refused for a declaration it could not see. Over every manifest it stays a refusal.
2026-10-10 03:53:07 +02:00
jochen abd3078491 warranted takes no word of the caller's: the record is the router's alone (hq ADR 0274 review)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
A caller's own line, recorded first under the one id the ask decides, would stand for every node's.
2026-10-10 03:52:04 +02:00
jochen 0e5aed1253 Record a module's act on the operator's warrant from the router's own record (hq ADR 0274)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A module that asks the operator acts with its own grants, and the hand-act log is where a person's
decisions are read back. The new verb warranted records who chose, how and with which proofs from the
router's record, never the caller's word, once per ask however many instances ask.
2026-10-10 03:40:49 +02:00
mesh-admin 6d3523ff10 Merge pull request 'A build says its build source; an image compiling Go is handed only that (hq ADR 0267, issue 363)' (#192) from fix/0267-a-build-says-its-build-source into main 2026-10-10 01:34:45 +00:00
jochen bd35b1c06c Judge a packaging module's news by plans that built it, over every plan since its build (hq ADR 0267, review)
mesh/delivery delivered
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
A plan that closed without building a module hid a missed merge from it;
a window of recent plans let an old closure back in once the plan that
overtook it slid out; a merge the forge gave no time was acted on again
every pass. A plan answering the merge's own commit is now a look at it,
and clocks a little apart do not make a merge history.
2026-10-10 03:23:50 +02:00
jochen 150f038ff8 Read a module whole while a plan has overtaken its build source, and plan every view alike (hq ADR 0267, review)
A failed build, or a plan closed before reaching a module, left the
closure its last good build said, and a fix-forward to a newly imported
package would have moved nothing. A missed merge moving only a module
that packages the repository was never caught up, and an older merge
read as history for it through a look that was not its own. The gate,
a pull request's check, the what-if and a delivery's order now read the
same view the merge handler does.
2026-10-10 03:20:36 +02:00
jochen 6c616838a5 Move a module on a merge only when its build source holds a changed file (hq ADR 0267, issue 363)
Every merge to the controller's repository planned the controller, the
build seat's holder and the route proxy in three gated tiers, whatever it
changed (issue 338). The planner now maps a merge's files onto the build
source each module's newest trunk build said: a README moves nothing, the
controller's command the controller alone, the proxy's program the proxy
alone. A module with none said, or one an open plan has yet to build, is
read whole as before. Sharing a repository draws no packages edge any more,
and one recorded before neither widens nor orders a plan.
2026-10-10 03:20:36 +02:00
jochen de55c63e12 Hold a cgo file's directory whole: its preamble may include from below it (hq ADR 0267, review)
mesh/delivery delivered
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
2026-10-10 03:20:35 +02:00
jochen abe5f7dc17 Say a build source only for the trunk's head, and hold what C, assembly and a new go.mod reach (hq ADR 0267, review)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A hand build of an older trunk commit said a closure lacking what was
imported since, and the planner would have mapped the next merge onto it.
C and assembly beside Go may include files below their directory, and a
go.mod made above a package moves it out of its module: each is now held.
2026-10-10 03:11:06 +02:00
jochen 4d1b81b6cb Say what a build was made from, and hand an image compiling Go only that (hq ADR 0267, issue 363)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A merge to the controller's repository moved the route proxy and the build
seat's holder whatever it changed, because nothing said which files their
builds read. A build of a trunk commit now says its build source per
repository: a Go program's import closure, an archive's directory, an
image's recipe and the package it names in the new 'compiles' field. That
image is built from its build source alone, so a recipe reading past it
fails by name, and its fingerprint is over what it was handed.
2026-10-10 02:47:35 +02:00
mesh-admin 9517f590ac Merge pull request 'Ask a node's engine for a fresh setuid search at the terminal (hq issue 361)' (#191) from fix/361-node-setuid-search-at-the-terminal into main 2026-10-10 00:06:33 +00:00
jochen 9cef820117 Ask a node's engine for a fresh setuid search at the terminal (hq issue 361)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
After the operator removes by hand what the last search found, no apply says so and nothing searched again
until the next day. node setuid-search <node> signs the ask as a hand-over is, under its own context, on a
subject only the node's engine hears and only the controller may publish.
2026-10-10 01:56:56 +02:00
mesh-admin 272ca2a578 Merge pull request 'Keep quiet for the setuid search the engine now runs to completion (hq issue 361)' (#190) from fix/361-the-setuid-search-runs-to-completion into main 2026-10-09 23:28:36 +00:00
jochen 7160d95323 Pin the node-engine at its merge of the resumable setuid walk (hq issue 361)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-10 01:19:17 +02:00
jochen 9551bc2380 Say the setuid search's quiet in the tests' words, and pin the node-engine at its reviewed head (hq issue 361)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery-group group fix/361-the-setuid-search-runs-to-completion delivering: 0 of 2 delivered
mesh/delivery superseded: a newer head of the same pull request
2026-10-10 01:11:24 +02:00
jochen cdaba36eca Pin the node-engine at its pull request's resumable walk (hq issue 361)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery-group group fix/361-the-setuid-search-runs-to-completion checking: 1 of 2 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
2026-10-10 01:01:18 +02:00
jochen eaf5195998 Keep quiet for the setuid search the engine now runs to completion (hq issue 361)
The node-engine's search has no bound any more: it runs at idle priority
and judges from its last complete, fresh result. The controller's quiet
while an agent account waits is the engine's rootsearch.Quiet, not the
old fifteen-minute bound, and the node-engine is pinned at its pull
request.
2026-10-10 00:54:52 +02:00
mesh-admin a6f831a633 Merge pull request 'Say a secret given in plain words, and log words the keeper refuses (hq issue 359)' (#189) from fix/the-secret-given-words-pass-plain into main 2026-10-09 22:00:15 +00:00
jochen a138c045bb Log a refused wording without what it quotes, and keep the secret-given words within bounds
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The reviewer found that the logged reason quoted the refused fragment: a
hash-shaped secret would reach the journal, and a changing clock time
defeated the once-per-kind dedupe. The reason is now logged without its
quoted fragment, the test resets the dedupe so it repeats, and a module
name too long for the headline falls back to the machine.
2026-10-09 23:44:19 +02:00
jochen 988e501ccc Say a secret given in plain words, and log words the keeper refuses
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
The secret-given condition's explanation carried a clock time, which the
plain rule refuses, so the operator read the scope's fallback ("needs a
look") instead of what changed. Its words now carry no time and say the
secret's name as words; a test holds them to the rule through a keeper.
With no test hook set, the keeper logs a refused or missing wording once
per kind and reason, so a fallback is never silent again (hq issue 359).
2026-10-09 23:41:22 +02:00
mesh-admin 19eefb66c6 Merge pull request 'node hand-over: the terminal hands a directory used as found to the mesh, asked of the node's engine (hq issue 356)' (#187) from fix/356-node-hand-over-at-the-terminal into main 2026-10-09 17:57:17 +00:00
jochen 081d9244d6 Merge remote-tracking branch 'origin/main' into fix/356-node-hand-over-at-the-terminal
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-09 19:44:55 +02:00
jochen b55a38ca9f Sign the hand-over ask, and fail the line on the engine's refusal (hq issue 356, review)
The subject proved nothing: the bus lets any principal allowed to answer reply to a message it received on the reply subject that message named, so a tool server — the operator's account, every agent — could deliver a hand-over to an engine. The controller now signs the ask with the mesh's key over a fixed context (node, path, who asked, a minute's expiry, a fresh nonce), as declarations are signed, and the engine verifies it. The writers table gains the row for mesh.node.*.ask.hand-over; the subject's comment no longer claims who the engine hears. The line's known-node check and the refusal branch are tested; every check was removed in turn and a test failed.
2026-10-09 19:44:55 +02:00
mesh-admin 747734687e Merge pull request 'Ask the operator only once the bus holds the controller's grant to ask (hq issue 353)' (#186) from fix/353-the-controller-asks-only-once-the-bus-holds-its-grant into main 2026-10-09 17:34:08 +00:00
jochen 9006c82393 node hand-over: the terminal hands a directory used as found to the mesh, asked of the node's engine (hq issue 356)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A module's condition told the operator to run the node-engine's hand-over at the machine, as root (issue 339), and the mesh had no channel for it. Now node hand-over <node> <path> is the controller's terminal's — a node subcommand that is not a read, so every verb and mesh-cli outside the terminal refuse it — and asks that node's engine on mesh.node.<node>.ask.hand-over, a request only the controller may publish and only that node's engine may hear and answer (its grant gains the subject and the right to answer what it was asked). The line's node and path are judged before anything is asked; the engine's answer is printed, a refusal as a refusal. Every text addressed to the operator names the nox line (ADR 0272); the condition's words stay plain.
2026-10-09 18:38:09 +02:00
jschoubben 0c8c9ffae9 Ask the operator only once the bus holds the controller's grant to ask (hq issue 353)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The grant is composed from the router's assignment and reaches the bus when its machine is next
pushed. Between assign and push the record said a router was here and the bus refused every ask
(seven refusals on 2026-10-09, 17:54 to 17:56). The asker now judges, as a push does, whether the
bus's machine was last sent the user list composed now; while it was not, nothing is published, it
is said once, and the conditions that need the operator are raised as undelivered, naming the push
that carries the grant.
2026-10-09 18:13:58 +02:00
mesh-admin 1c7c385839 Merge pull request 'A gate judges its own send and the build it sent, and never puts the controller back behind its store (hq issue 352)' (#185) from fix/a-gate-outlives-the-controller-and-judges-the-build-it-sent into main 2026-10-09 16:10:13 +00:00
mesh-admin 65ff6159ad Merge pull request 'mesh-cli: give a terminal line the standard input mesh-cli carried, refuse it to any other, keep it nowhere (hq ADR 0259 §10, ADR 0272)' (#184) from feat/a-terminal-line-takes-standard-input into main 2026-10-09 16:10:11 +00:00
jschoubben e6e1e3bc89 A gate judges its own send and the build it sent, and never puts the controller back behind its store (hq issue 352)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
On 2026-10-09 a release's gate on the control node read the machine's
report against a newer send another plan had just made there, failed
three builds the machine had reported healthy, and put them back on
every machine to a controller older than the store's schema; that
controller then passed the newer plan's gate from its own health.

- A gate keeps what its send carried (digest, sequence) and reads the
  report against it; a report on the last send is on it too.
- A gate judges only the build the machine was last sent: another build
  there supersedes the judging — no verdict, nothing put back.
- A controller is told its build (MESH_CONTROLLER_VERSION, ${version}
  in a process's env) and records how far it reads the store's schema;
  a put-back to a build that reaches less, or never said, is refused
  and the current build kept, said as urgent.
- A release's open gate holds other sends of its modules there, and a
  plan's own first send waits on it.
2026-10-09 17:15:40 +02:00
jschoubben 07e59c535e Pin mesh-host at its main (d8ff154), where the installer's first user list carries the controller's ask grants
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer delivery to the same trunk took over its walk
mesh/delivery-group group feat/a-terminal-line-takes-standard-input delivered: every member is delivered
The repo-check reads the installer's user list at the pinned commit, and the old pin predated mesh-host
#59 and #68: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose failed on main's own grants.
2026-10-09 17:10:16 +02:00
jschoubben ba97297f66 mesh-cli: give a terminal line the standard input mesh-cli carried, refuse it to any other, keep it nowhere
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.62s)
mesh/delivery-group group feat/a-terminal-line-takes-standard-input rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
A secret given at the controller's terminal through mesh-cli (secret accept … --from -) never reached the
line: every line ran with no standard input. A line that runs as the terminal now reads what mesh-cli
carried (at most 64 KiB); an ordinary call carrying any is refused and nothing runs; the calls record keeps
only that some was given, the journal and the answer nothing of it.
2026-10-09 17:01:36 +02:00
mesh-admin e6b00e2e51 Merge pull request 'give: take a module's own secret through a hidden prompt at the operator's desk (hq ADR 0259 §10)' (#156) from feat/a-secret-given-at-the-desk into main 2026-10-09 14:30:47 +00:00
jschoubben fa19a2d718 give: test that a trusted party's secret given at the terminal is announced before it is kept, and refuse an unknown machine before anybody types
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The terminal path's order is one function, keepGiven, so the test fails when the announcement before a
trusted party's secret is removed, and when a failed announcement still keeps it. give also refuses a
machine the mesh does not know, as the secret's or as the desk, before the prompt (the final review).
2026-10-09 16:22:47 +02:00
jschoubben 3e5086a2f6 give: never take a trusted party's secret at a desk, and announce it before it is kept (hq ADR 0259 §10, the confirmation review's N1-give)
A desk's prompt is answered over the desk machine's bus, on a subject that machine's runtime answers;
on a desk machine agents run as the operator, who holds that credential, so an agent could answer first
with a bot token of its own sealed to the call's key. The secret of a module running as an account of
its own (the Telegram bot's token) is now refused at the desk, before anybody is asked, naming the line
to type at the controller's terminal; there it is announced on every channel, the old one among them,
before it is kept, and not kept when that announcement fails. What is typed at the terminal is not
echoed. Tests: the refusal with nobody asked, and only the desk machine's launcher may answer its
prompt (MaySubscribe).
2026-10-09 16:22:47 +02:00
jschoubben ed331eb972 Let the give verb's exact line past the terminal rule for secrets, and nothing else (hq ADR 0259 §10, ADR 0266)
Restacked on #157, which carries #164's rule that no verb runs secret accept. give's line carries no value:
the operator types it into the desk's hidden prompt, sealed to the call and then to the module's machine.
Only that exact line passes: a value, a file, a provider or any extra word stays the terminal's.
2026-10-09 16:22:47 +02:00
jschoubben be59f29f46 give: take only a value a person holds, ask the desk by name, let the controller alone ask it, and announce every value given
The review of 2026-10-09 (M4):
- give refuses broker (the bus account issue mints) and any own secret the mesh may make itself;
- the desk's prompt is asked by module, secret and machine, never with words of the caller's, and the
  bus denies the prompt's subjects to every principal but the controller, however wide its grant, so the
  prompt's 'the controller asks' is the bus's word (Permissions.PublishDeny, broker.ControllerOnly);
- secret accept with a value is refused through a verb: a value comes from the terminal or the desk;
- every value given for an own secret, at the terminal or the desk, raises the urgent condition
  secret-given on every channel, until the operator silences it.
2026-10-09 16:22:47 +02:00
jochen 5689553406 Take a module's own secret through a hidden prompt on the operator's desk, so a bot token never passes through an agent's session (hq ADR 0259 §10) 2026-10-09 16:22:47 +02:00
jschoubben 0559b80887 Move to mesh-sdk 16984aa, rebased on its main, which refuses a warrant with no time or for an ask with no expiry
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.56s)
mesh/delivery stopped: the pull request closed unmerged
2026-10-09 16:22:34 +02:00
jschoubben 74d35600a6 Keep an approval asked through a silence of its condition (hq ADR 0259, the confirmation review's M1)
Choosing Silence silenced the condition, the condition was no longer wanted, and the next reconcile
cancelled the Restart or Release ask beside it: an acknowledgement, which any desk click may give,
took an approval back. An open approval ask now stays until it is answered or expires while its
condition is open and silenced with the same answers. The test silences as the controller does; it
failed before (0 open) and passes, and the kept Restart is performed on its warrant.
2026-10-09 16:22:34 +02:00
jschoubben d19c9ed5b7 Start a rehearsal only at the controller's terminal as main now judges it (hq ADR 0259, ADR 0272)
rehearse refused only a verb's process. Since mesh-cli (ADR 0272 §4) the serving controller runs an
ordinary mesh-cli line without a verb, naming its caller: such a line, from an agent's account, read as
the terminal and could start a question the operator did not ask. rehearse now asks
startedAtTheTerminal. And main's mesh-cli test helper asked is cliAsked, beside the asker's asked.
2026-10-09 16:22:34 +02:00
jschoubben 799eec0a5a Ask an acknowledgement apart from an approval, change every kept ask by compare-and-set, and rehearse rather than drill (hq ADR 0259, review M1/L2/L3/L7)
- M1: a condition offering both kinds of answer is asked twice: its authorising answers about the
  condition, its acknowledging ones (Silence) apart, so an answer from a channel that only acknowledges
  never ends an approval.
- L2: the asked store creates once and changes only over the revision it read, deciding again on what it
  reads; a stale cancel no longer writes over an act.
- L3: every ask is kept before it is published, one whose publishing failed is marked unsent and asked
  again, and a cancel is kept before it is said. The terminal's test question is now `rehearse`, so it is
  not called what the glossary calls a drill; its two answers are both approve-level.
- L7: two deliveries of one warrant to two controllers at once act exactly once, on a real bus.
- Re-vendored onto mesh-sdk 76902998 (canonical digests): an option binds an asks.Act with each argument
  as arg.<name>.
- The lab's bus fixture composes verified-sender only where the lab says its machine is root-free
  (MESH_LAB_ASKS_ROOT_FREE=true).
2026-10-09 16:22:34 +02:00
jschoubben ad406e81b8 Say loudly when a condition that needs the operator could not be asked on any channel (hq ADR 0259)
With no router, or an ask the router refused and nothing changed since, the controller asked nothing
and said it only in its own log. It now keeps a condition of its own, asks-undelivered, naming the
conditions not asked and why, cleared once each can be asked again.
2026-10-09 16:22:34 +02:00
jschoubben 646c5e53db Add drill: an ask the operator starts at the controller's terminal, whose approval performs nothing and is recorded (hq ADR 0259)
The live acceptance needs an approval the operator can ask for at will and that changes nothing. A
drill is asked like any condition's ask, bound to its own act, claimed once on its warrant and recorded
as a warrant hand-act with who answered, through which channel and the proofs. A verb's process may not
start one, so no agent asks the operator a question they did not start.
2026-10-09 16:22:34 +02:00
jschoubben 2190e2c664 Compose and raise the bus of the lab's proof of the operator's answers, as this controller would (hq ADR 0259)
mesh-lab's asks proof runs the router, the Telegram channel and an asker on a real bus. Its accounts,
streams, workers, buckets and memberships come from this test at the controller's commit, so the lab
proves the composition and not a copy of it. Skipped unless the lab asks.
2026-10-09 16:22:34 +02:00
jschoubben 0909d7b125 Bind each asked option to the exact act, and perform only that act on its warrant (hq ADR 0259 §6)
Every option the controller asks with carries the digest of its verb, machine, arguments and level
(the SDK's Option.Binds). A warrant must name the digest of the ask the controller keeps, and before
acting the controller checks that the act it is about to perform is the one the option bound: a
record changed after the ask is refused, never performed. mesh-sdk moves to d4077b4.
2026-10-09 16:22:34 +02:00
jochen 744b0b9162 Ask at most three at a time, wait out a refusal, need a router, and act only on a claimed open ask, as the review asked (hq ADR 0259) 2026-10-09 16:22:34 +02:00
jochen 8de4dc7951 Ask the operator for a condition's answers and act on the warrant, so release, stop, start and restart can be answered from any channel that proves who answered (hq ADR 0259) 2026-10-09 16:22:34 +02:00
mesh-admin 2913c54c29 Merge pull request 'Kinded benches, verbs named by their caller, proofs and records (hq ADR 0259 §3)' (#154) from feat/asks-answered-on-any-channel into main 2026-10-09 14:17:43 +00:00
mesh-admin ef26d4cb0f Merge pull request 'Test the newest merge's order, a second reopening's gaps and sub-second merge times (hq issues 348, 349)' (#183) from fix/348-349-test-gaps into main 2026-10-09 13:55:14 +00:00
jschoubben d9a730307c Test the newest merge's order, a second reopening's gaps and sub-second merge times (hq issues 348, 349)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The review of PR 179 found four paths no test held: which of two earlier
plans NewestMergeOf takes, a tie between them, gaps kept across a second
reopening in one keeper, and a merge time's fraction of a second.
2026-10-09 15:29:31 +02:00
mesh-admin 9ca7952d5e Merge pull request 'Judge a send by when a fault began, not when it was last raised (hq issue 348)' (#179) from fix/a-fault-from-before-a-send-fails-no-gate into main 2026-10-09 13:25:21 +00:00
jschoubben 58cb586c37 Answer the review of hq issues 348 and 349: gaps, parts, the newest merge in any state
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
- A reopened fault keeps its gaps: it was there at a send unless the send
  fell in one, so a send that breaks a machine recovered before it still
  fails its gate (A2).
- An undecided part holds only the conditions that name it (A4).
- D2 holds a silent resolver for the next run again, refused or not: a
  burst of refusals is also a restart (A3).
- A late merge is planned at the newest planned merge of its branch in any
  state, not only an open one (A1); merge times to the nanosecond (A5).
2026-10-09 15:00:56 +02:00
jschoubben c3c56a69e2 Take either binding until the catalogue's main binds once (hq issue 348)
The test reads the catalogue beside it, which the build seat checks out at
main; mesh-catalog PR 161 changes the binding, so the two land in either
order.
2026-10-09 15:00:56 +02:00
jschoubben 63b87b6f7b Hold the resolver to bind-interfaces, as mesh-catalog PR 161 makes it (hq issue 348) 2026-10-09 15:00:56 +02:00
jschoubben 997a4925b0 Order a branch's plans by its merges, and build the newest commit (hq issue 349)
A merge acted on late by the catch-up made its plan after the plan of the
merge that followed it, superseded it by creation time, and folded its
unbuilt modules into a plan at the older commit: on 2026-10-09 the
forge's security fix (a082615b) was superseded by 8ff8197a. A plan now
keeps its merge time (migration 0086), supersession follows it, and a
merge older than an open plan of its branch is planned at that plan's
commit, which contains it.
2026-10-09 15:00:56 +02:00
jschoubben 077ddf0eb8 Judge a send by when a fault began, not when it was last raised (hq issue 348)
On 2026-10-09 the control node's resolver refused from 10:57:57 UTC. A
node-engine restarted by the 10:59:34 send said its names undecided, that
statement cleared the network condition, the next look raised it again
after the send, and the gate put back two builds for a fault older than
them.

- A condition keeps First across a reopening; the gate reads Began.
- An undecided network statement (unknown, starting) clears nothing.
- D10 counts what a release's tier names as rolling, so a walked
  node-engine is not core-behind on its own first machine.
- D2 raises a resolver that refuses every try at once: a refusal is an
  answer, not a loaded resolver (issue 277).
2026-10-09 15:00:56 +02:00
jschoubben c544c2a17b Key root-not-free apart from DA, keep ADR 0266's quiet window there, and judge at one clock (hq ADR 0259 §8)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer delivery to the same trunk took over its walk
mesh/delivery-group group feat/asks-answered-on-any-channel delivered: every member is delivered
The confirmation review of 2026-10-09 found D-root and DA writing one key, machine.<m>.agent-root, from
two probes with different words, so it flapped every run; D-root is now root-not-free. D-root raised the
urgent condition after every node-engine restart while the first setuid search ran; it now keeps the
same quiet window as DA, and the root-free verb still answers that machine not free. agentConfined
takes the judging clock.
2026-10-09 13:55:06 +02:00
jschoubben 5866b2db94 Hold a private kind's holder to an account of its own, as a verified one is (hq ADR 0259 §7, §8)
A private kind is where the router shows a link's code, and the code makes an account the operator's.
Registration refused a verified-sender holder on the machine's runtime and let a private one stand;
it now refuses both (the confirmation review of 2026-10-09, low).
2026-10-09 13:51:54 +02:00
jschoubben 983bf65141 Answer the controller's own verbs, root-free among them, from the serving controller alone (hq ADR 0259 §8)
The confirmation review asked who may answer root-free on the bus. Composed from the controller's own
manifest, the module principal of the machine running the controller and that machine's runtime were
granted the controller seat's tool subjects too: either could answer root-free, and the runtime's
credential is one an agent on that machine may hold. The controller's seat is now served by the
controller principal alone, in grants and memberships; TestOnlyTheServingControllerMayAnswerRootFree
failed before (3 answerers) and passes. And a machine waiting for its first setuid search is not
root-free, whatever ADR 0266's quiet window does to the self-check.
2026-10-09 13:51:18 +02:00
jschoubben 0e46b8302d Let root-free take its machines as a list, as the router names them
The router's contract names the machines as a JSON array. A verb's argument declared a list now takes
an array of names (or one text separated by commas), and refuses anything else in it.
2026-10-09 13:48:56 +02:00
jschoubben 4c375dafed Judge a machine root-free only on a positive, fresh measure, and believe a verified sender only there (hq ADR 0259 §8, review H2/H3)
The agent-root probe read the sudo module's answer, given in the machine's runtime as the very account
an agent could become, and took a missing account, a missing answer or no accounts as a pass. One
judgement now decides: the machine names an agent account its node-engine judged unable to become
root within 15 minutes (agentConfined, mesh-controller #164), and the login shell's execute is not
served there; anything not read is not free. The probe raises agent-root on it, the new root-free
verb answers it live for the router, and a push composes verified-sender for a kind only while its
machine and the router's pass it.
2026-10-09 13:48:56 +02:00
jschoubben e2a45b18ba Refuse a module of its own account running as the node's operator or agent account (hq ADR 0259 §8, review L4) 2026-10-09 13:48:56 +02:00
jschoubben 5fa8e40667 Raise agent-root where who can become root is not measured, so the router never reads a missing measure as a no (hq ADR 0259 §8)
A machine where the router or a verified channel runs and the sudo module is absent or does not answer
made the probe fail to run, which raises nothing the router reads, so it went on approving there. Each
such machine now raises the same urgent condition, saying it was not measured.
2026-10-09 13:48:56 +02:00
jschoubben b3fd360ddb Count the login shell where its execute is served, not where its seat is held (hq ADR 0268)
The control-node withholds execute through its holder's setting since ADR 0268, so probe D-root read a
closed path as open. It now counts the verb as served while the holder's setting for that machine is
serve, or the bus hears execute answered there, or the bus could not be asked: a withheld value not yet
pushed, or a holder answering against its setting, is never taken for closed.
2026-10-09 13:48:56 +02:00
jochen 9372e80cec Serve a trusted holder from a runtime of its own account, refuse it in the machine's runtime, and say while an agent can become root where it runs (hq ADR 0259 §8) 2026-10-09 13:48:56 +02:00
jochen c9be75b13e Carry a channel's capabilities on its claim and tell the router every kind, so an answer is judged by the controller's record and not the channel's word 2026-10-09 13:48:56 +02:00
jochen f5f315cc95 Grant a seat's traffic by caller and by kind, so an ask's asker and a channel's kind are facts the bus enforces (hq ADR 0259) 2026-10-09 13:48:56 +02:00
1204 changed files with 152425 additions and 14636 deletions
+5
View File
@@ -3,3 +3,8 @@
# A built binary. The lab writes one here when pointed at the repository root by mistake;
# built artifacts belong in build/, which is already ignored.
/mesh-builder
/mesh-controller
/postgres-provisioner
/redis-provisioner
# And where `go build` in cmd/mesh-controller leaves it (novox/hq issue 233).
/cmd/mesh-controller/mesh-controller
+3 -1
View File
@@ -27,9 +27,11 @@ WORKDIR /src
COPY go.mod go.sum ./
COPY vendor/ vendor/
# `COPY . .` brings `.git` along, so Go is told not to stamp the commit: the version is said by -X, and a
# rebuild of an unchanged program is the same bytes (novox/hq ADR 0267 rule 6, issue 363).
COPY . .
ARG VERSION=development
RUN CGO_ENABLED=0 GOFLAGS=-mod=vendor GOPROXY=off go build -trimpath \
RUN CGO_ENABLED=0 GOFLAGS=-mod=vendor GOPROXY=off go build -trimpath -buildvcs=false \
-ldflags "-s -w -X main.version=${VERSION}" \
-o /mesh-controller ./cmd/mesh-controller
+25 -13
View File
@@ -95,31 +95,43 @@ PROXY_IMAGE ?= mesh-route-proxy:$(VERSION)
PROXY_DEV_TAG ?= mesh-route-proxy:development
proxy-image:
docker build --build-arg GO_BASE=$(GO_BASE) -f examples/route-proxy/Dockerfile -t $(PROXY_IMAGE) -t $(PROXY_DEV_TAG) .
docker build --build-arg GO_BASE=$(GO_BASE) -f modules/route-proxy/Dockerfile -t $(PROXY_IMAGE) -t $(PROXY_DEV_TAG) .
@echo
@docker image inspect $(PROXY_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The whole gate. Raises a database, runs everything against it, and takes it down again --
# including when the tests fail, which is why the teardown is not conditional.
# The whole check. Raises a database, runs the repository's own check against it -- merge-check.sh,
# the very script `mesh/repo-check` runs -- and takes the database down again, including when the
# check fails, which is why the teardown is not conditional and the script's exit status is the
# target's.
#
# **Packages in parallel, under the race detector, each test on a bus of its own** (internal/testbus).
# It was one package at a time against one shared bus, because the live tests assert, read and remove
# the mesh's own objects by their fixed names, and two packages at once deleted what the other read; the
# suite was red run as Go runs it and read as noise. A bus per test, of the release the mesh runs, made
# it the same in any order. The timeout bounds a hang to a failure with a stack, never a stalled gate.
check: fmt vet postgres
@go test -race -timeout 15m ./... ; status=$$? ; $(MAKE) postgres-stop ; exit $$status
# **It calls the script rather than restating it** (novox/hq issue 431): this target used to list its
# own steps, its formatting step walked vendor/ where the script's does not, and it failed on
# third-party code no change could fix -- so a developer's check and the gate disagreed, and the steps
# after formatting never ran through it. The script is the one list of steps; this target only gives it
# a database (MESH_TEST_POSTGRES, exported above).
#
# **Two levels** (issue 895): the unit level, `go test ./...` untagged, timed and held to a bound; and the integration
# level, `-tags integration`, the tests that start a bus, a store or another process. Each is said with its time.
#
# **Packages in parallel, under the race detector, each test on a bus of its own** (internal/testbus),
# as the script runs them: a bus per test, of the release the mesh runs, makes the suite the same in any
# order, and the timeout bounds a hang to a failure with a stack, never a stalled gate.
check: postgres
@sh merge-check.sh ; status=$$? ; docker rm -f $(PG_CONTAINER) >/dev/null 2>&1 || true ; exit $$status
# Without a database the store's tests skip rather than fail, so this is the honest subset and not
# the gate.
# The unit level alone (untagged: no bus, no store, no other process), which a developer runs continuously. The tests
# that need a bus or the store carry `//go:build integration` and run in `check`, not here; without a database the
# store's tests would skip rather than fail anyway.
test:
go test -timeout 15m ./...
vet:
go vet ./...
# Quick steps for a developer, not part of `check`. The directories gofmt reads must be the ones
# merge-check.sh lists, never `.`, which walks vendor/ (novox/hq issue 431).
fmt:
@unformatted=$$(gofmt -l . 2>/dev/null) ; \
@unformatted=$$(gofmt -l cmd internal examples) ; \
if [ -n "$$unformatted" ] ; then echo "not gofmt'd:" ; echo "$$unformatted" ; exit 1 ; fi
postgres:
+6
View File
@@ -0,0 +1,6 @@
# Release 57: config and provisioning
This branch carries the work of hq design 57 (ADR 0348, ADR 0349) in this repository. Each package
merges here through its own reviewed pull request; the branch reaches main as one pull request, in one
delivery group with the branch of the same name in mesh-controller, mesh-host, mesh-sdk, mesh-catalog
and hq, after a rehearsal in the lab. This file is deleted by the last commit before that pull request.
+29
View File
@@ -0,0 +1,29 @@
#!/bin/sh
# The unit tests of every Go module of its own below this repository's root, run from the root, for merge-check.sh
# and for the test that tries it on a tree (merge_check_nested_test.go). Its own file so that the test needs no
# particular merge-check.sh: the check that judges a change to merge-check.sh is main's.
set -eu
cgo="${cgo:-0}"
race="${race:-}"
# nested_modules vets and tests every Go module of its own below the root, `modules/<name>/go.mod` (novox/hq ADR 0420
# §5, issue 1025): `./...` stops at a nested go.mod, so a module that lives whole in this repository would have its
# unit tests run by nobody. None is a pass. Each runs in its own directory, outside the vendor directory this
# repository's own packages are built from; a nested module vendors what the forge holds (its own `vendor/`, so the
# check runs offline), as this repository's root module does (issue 1019). A failing module fails the check; the others are still run.
nested_modules() {
nested=0
for gomod in modules/*/go.mod; do
[ -e "$gomod" ] || continue
dir=$(dirname "$gomod")
echo "nested module $dir:"
if ! (cd "$dir" && GOFLAGS= CGO_ENABLED=0 go vet ./... &&
GOFLAGS= CGO_ENABLED="${cgo:-0}" go test ${race:-} -count=1 -timeout 30m ./...); then
echo "nested module $dir FAILED"
nested=1
fi
done
return $nested
}
nested_modules
+1 -1
View File
@@ -11,7 +11,7 @@ WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -ldflags '-s -w' -o /mesh-builder ./cmd/mesh-builder
RUN CGO_ENABLED=0 go build -trimpath -buildvcs=false -ldflags '-s -w' -o /mesh-builder ./cmd/mesh-builder
FROM ${ALPINE_BASE}
# git to clone what it is asked to build, and the docker client to build and push it. The daemon
+146
View File
@@ -0,0 +1,146 @@
package main
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"time"
)
// **The build agent says whether it is busy, and being asked drains it** (novox/hq ADR 0335 §1, §5). Its
// container declares `busy` as `mesh-builder busy`: the node-engine runs it inside the container before it
// replaces it. It exits 0 while a build is in hand and 1 when none is; and from the first ask the agent takes
// no new build request, so its answer ends once the build in hand ends — a replaced builder finishes the check
// it runs, instead of the build queue handing it out again from the start (issue 594).
//
// Two marks in the container's own temporary directory, as the ready mark (ready.go): a new container starts
// without them, and a restarted one removes them before it takes work again.
//
// - in hand: written by the agent when it claims a request, before it reads whether it is drained; removed
// when the build ends.
// - drained: written by the ask, before it reads whether a build is in hand.
//
// Each side writes its own mark before it reads the other's, so an ask racing a fetch sees the build in hand,
// or the agent sees the drain and hands the request back; never neither.
func inHandMark() string { return filepath.Join(os.TempDir(), "mesh-builder.in-hand") }
func drainMark() string { return filepath.Join(os.TempDir(), "mesh-builder.drained") }
// The drain lapses when no ask has come for drainLapse: the node-engine asks every 20 seconds for as long as it
// really waits, so an agent whose replacement was withdrawn — the declaration put back, the walk stopped —
// takes work again rather than staying drained for ever (review of ADR 0335). And an agent drained for longer
// than the node-engine's bound and the slack says it is not ready: the replacement it was drained for never
// came, which is a fault to see.
const (
drainLapse = 2 * time.Minute
drainTooLong = 15*time.Minute + 5*time.Minute
)
// drainedAt is when the drain began and when it was last asked, from its mark; false when there is no mark.
func drainedAt() (began, asked time.Time, ok bool) {
info, err := os.Stat(drainMark())
if err != nil {
return time.Time{}, time.Time{}, false
}
asked = info.ModTime()
began = asked
if raw, err := os.ReadFile(drainMark()); err == nil {
if t, err := time.Parse(time.RFC3339, strings.TrimSpace(string(raw))); err == nil && !t.After(asked) {
began = t
}
}
return began, asked, true
}
// drained says the agent was asked whether it is busy within drainLapse: it takes no new build request.
func drained() bool { return drainedSince(time.Now()) }
func drainedSince(now time.Time) bool {
_, asked, ok := drainedAt()
return ok && now.Sub(asked) < drainLapse
}
// drainedTooLong is the health check's second half: nil unless the agent has been drained, asked all along, for
// longer than its node-engine's bound and the slack.
func drainedTooLong(now time.Time) error {
began, asked, ok := drainedAt()
if !ok || now.Sub(asked) >= drainLapse || now.Sub(began) <= drainTooLong {
return nil
}
return fmt.Errorf("drained since %s and still asked: the replacement its node-engine waits for has not come "+
"within %s (novox/hq ADR 0335)", began.UTC().Format(time.RFC3339), drainTooLong)
}
// claim marks a request in hand, unless the agent is drained: then nothing is in hand, and the request goes
// back to the queue.
func claim() bool {
// Made, never written over: one build at a time, and a mark already there is a build in hand.
f, err := os.OpenFile(inHandMark(), os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644)
if err != nil {
// Unmarked, an ask would say idle while this build runs: the build is not taken.
fmt.Fprintf(os.Stderr, "cannot mark a build in hand (%s), so it is handed back: %v\n", inHandMark(), err)
return false
}
_, err = f.WriteString(time.Now().UTC().Format(time.RFC3339) + "\n")
if closeErr := f.Close(); err == nil {
err = closeErr
}
if err != nil {
_ = os.Remove(inHandMark())
fmt.Fprintf(os.Stderr, "cannot mark a build in hand (%s), so it is handed back: %v\n", inHandMark(), err)
return false
}
if drained() {
_ = os.Remove(inHandMark())
fmt.Fprintln(os.Stderr, "asked whether busy, so this agent takes no new build: handed back")
return false
}
return true
}
// holding says what is in hand, for the ask to print: written once the request is read.
func holding(what string) {
_ = os.WriteFile(inHandMark(), []byte(time.Now().UTC().Format(time.RFC3339)+" "+what+"\n"), 0o644)
}
// released ends the build in hand.
func released() {
if err := os.Remove(inHandMark()); err != nil && !errors.Is(err, os.ErrNotExist) {
fmt.Fprintf(os.Stderr, "cannot remove the mark of the build in hand (%s): %v\n", inHandMark(), err)
}
}
// undrained clears both marks at the start of a holder: a container started again is asked afresh.
func undrained() error {
for _, mark := range []string{drainMark(), inHandMark()} {
if err := os.Remove(mark); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove %s: %w", mark, err)
}
}
return nil
}
// busy is the ask: it drains the agent, then says whether a build is in hand. Nil — exit 0 — while one is.
func busy() (string, error) {
// The mark keeps when the drain began while it is asked within drainLapse, and is written again on every
// ask: its time is the last ask.
began := time.Now().UTC()
if was, _, ok := drainedAt(); ok && drained() {
began = was.UTC()
}
if err := os.WriteFile(drainMark(), []byte(began.Format(time.RFC3339)+"\n"), 0o644); err != nil {
// Not drained, it could take the next build: said, and answered busy only for what is in hand now.
fmt.Fprintf(os.Stderr, "cannot mark this agent drained (%s): %v\n", drainMark(), err)
}
held, err := os.ReadFile(inHandMark())
if err != nil {
return "", errors.New("idle: no build in hand, and this agent takes no new one")
}
since, what, _ := strings.Cut(strings.TrimSpace(string(held)), " ")
if what == "" {
what = "a build"
}
return fmt.Sprintf("busy: %s, in hand since %s; taking no new build", what, since), nil
}
+162
View File
@@ -0,0 +1,162 @@
package main
import (
"os"
"strings"
"sync"
"testing"
"time"
)
// `mesh-builder busy` answers 0 while a build is in hand and non-zero when idle, and after its first ask the
// agent takes no new build request (novox/hq ADR 0335 §5).
func TestBusyAnswersWhileABuildIsInHandAndDrainsTheAgent(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
if drained() {
t.Fatal("an agent never asked is drained")
}
// A build is claimed and runs.
if !claim() {
t.Fatal("an agent never asked refused a build")
}
holding("novox/mesh-controller (check-280)")
said, err := busy()
if err != nil || !strings.Contains(said, "novox/mesh-controller (check-280)") {
t.Fatalf("an agent with a build in hand did not answer busy with what it holds: %q %v", said, err)
}
// From that ask on it takes nothing new: the fetch is not made, and one fetched is handed back.
if !drained() {
t.Fatal("the ask did not drain the agent")
}
if claim() {
t.Fatal("a drained agent took a new build")
}
// The build in hand is still in hand after the refused claim, and busy goes on saying so.
if _, err := busy(); err != nil {
t.Fatalf("a refused claim ended the build in hand: %v", err)
}
// It ends: idle, and still drained.
released()
if _, err := busy(); err == nil {
t.Fatal("an agent with nothing in hand answered busy")
}
if claim() {
t.Fatal("a drained agent took a build after its last one ended")
}
// Started again, it is asked afresh.
if err := undrained(); err != nil {
t.Fatal(err)
}
if drained() || !claim() {
t.Fatal("a holder started again is still drained")
}
released()
}
// An ask racing a fetch sees the build in hand, or the agent sees the drain and hands the request back: never
// neither, which would replace the container under a build just taken.
func TestAnAskRacingAFetchNeverMissesTheBuild(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
for i := 0; i < 200; i++ {
if err := undrained(); err != nil {
t.Fatal(err)
}
var claimed bool
var askErr error
var wg sync.WaitGroup
wg.Add(2)
go func() { defer wg.Done(); claimed = claim() }()
go func() { defer wg.Done(); _, askErr = busy() }()
wg.Wait()
if claimed && askErr != nil {
t.Fatalf("round %d: the agent took a build and the ask said idle", i)
}
released()
}
}
// A drain no longer asked lapses: an agent whose replacement was withdrawn after the first ask (a put-back, a
// revert, a stopped walk) takes work again after drainLapse, rather than staying drained with its health green.
func TestADrainNoLongerAskedLapses(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
if !drained() || claim() {
t.Fatal("an agent just asked takes work")
}
// The last ask was three minutes ago.
old := time.Now().Add(-3 * time.Minute)
if err := os.Chtimes(drainMark(), old, old); err != nil {
t.Fatal(err)
}
if drained() {
t.Fatal("a drain no longer asked for three minutes still holds")
}
if !claim() {
t.Fatal("an agent whose drain lapsed takes no work")
}
released()
}
// An agent drained, and asked all along, for longer than its node-engine's bound and the slack is not ready: the
// replacement it was drained for never came. One asked again after a lapse starts a new drain.
func TestAnAgentDrainedTooLongIsNotReady(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
taking("mesh-build-machine")
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
now := time.Now()
if err := drainedTooLong(now); err != nil {
t.Fatalf("an agent drained a moment ago is not ready: %v", err)
}
// The drain began 25 minutes ago, and the node-engine asked a moment ago.
began := now.Add(-25 * time.Minute).UTC().Format(time.RFC3339)
if err := os.WriteFile(drainMark(), []byte(began+"\n"), 0o644); err != nil {
t.Fatal(err)
}
err := drainedTooLong(time.Now())
if err == nil || !strings.Contains(err.Error(), "has not come") {
t.Fatalf("an agent drained for 25 minutes reads ready: %v", err)
}
// Each ask keeps when the drain began.
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
if err := drainedTooLong(time.Now()); err == nil {
t.Fatal("an ask started the drain again")
}
// The asks stopped: the drain lapsed, and so did the fault.
old := time.Now().Add(-3 * time.Minute)
if err := os.Chtimes(drainMark(), old, old); err != nil {
t.Fatal(err)
}
if err := drainedTooLong(time.Now()); err != nil {
t.Fatalf("a lapsed drain still reads not ready: %v", err)
}
}
// An ask this build does not know exits non-zero: an older image asked `busy` reads idle, not busy for the
// node-engine's whole bound.
func TestAnUnknownCommandFails(t *testing.T) {
was := os.Args
t.Cleanup(func() { os.Args = was })
os.Args = []string{"mesh-builder", "some-later-word"}
if err := run(); err == nil {
t.Fatal("an unknown command exited 0")
}
os.Args = []string{"mesh-builder", "help"}
if err := run(); err != nil {
t.Fatalf("help failed: %v", err)
}
}
+38
View File
@@ -0,0 +1,38 @@
package main
import (
"testing"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// The build seat hands the check the judge cache the build agent declares (MESH_JUDGE_CACHE), never a path of its own.
func TestTheBuildSeatHandsTheCheckTheJudgeCacheItDeclares(t *testing.T) {
t.Setenv("MESH_JUDGE_CACHE", "/var/lib/mesh/build-agent/judge-cache")
if got := judgeCache(); got != "/var/lib/mesh/build-agent/judge-cache" {
t.Fatalf("the judge cache the manifest declares is read as %q", got)
}
t.Setenv("MESH_JUDGE_CACHE", "")
if got := judgeCache(); got != "" {
t.Fatalf("with no judge cache declared, %q is used", got)
}
}
// The siblings a check is asked to read reach the build seat's spec and, as read, the outcome (novox/hq ADR 0382).
func TestTheSiblingsAnAskNamesReachTheSpecAndTheOutcome(t *testing.T) {
spec := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main",
Siblings: []link.CheckSibling{{Repository: "novox/mesh-controller", Number: 346, Head: "a", Base: "main", Dir: "mesh-controller"}},
SiblingsUnknown: "why"}})
if len(spec.Siblings) != 1 || spec.Siblings[0] != (builder.Sibling{Repository: "novox/mesh-controller", Number: 346,
Head: "a", Base: "main", Dir: "mesh-controller"}) || spec.SiblingsUnknown != "why" {
t.Fatalf("the spec reads %+v", spec)
}
out := siblingsOut([]builder.JudgedSibling{{Repository: "novox/mesh-controller", Number: 346, Head: "a", Base: "main", BaseCommit: "b"}})
if len(out) != 1 || out[0].BaseCommit != "b" || out[0].Number != 346 {
t.Fatalf("the outcome reads %+v", out)
}
if plain := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "o", Repo: "r"}}); plain.Siblings != nil {
t.Fatalf("a request that names none has siblings %+v", plain.Siblings)
}
}
+31
View File
@@ -41,6 +41,9 @@ type holder struct {
say func(link.HolderState) error
// remove runs what a kill needs outside the build: the containers left behind.
remove builder.Runner
// scratch is where check builds are kept (novox/hq ADR 0350 §5): what the week's sweep lets go of. Nil where
// this machine publishes to no artifact store.
scratch builder.Scratch
mu sync.Mutex
paused bool
@@ -131,6 +134,34 @@ func (h *holder) stateWhilePaused(ctx context.Context, every time.Duration) {
}
}
// expireChecks lets go of every check build older than a week (novox/hq ADR 0350 §5), at start and every while
// after. Every holder sweeps the one store; what one let go of, another finds gone.
func (h *holder) expireChecks(ctx context.Context, every time.Duration, now func() time.Time) {
sweep := func() {
if h.scratch == nil {
return
}
n, err := h.scratch.Expire(ctx, builder.CheckBuildsKept, now())
switch {
case err != nil:
fmt.Fprintf(os.Stderr, "check builds older than %s: %d let go of, and not all: %v\n", builder.CheckBuildsKept, n, err)
case n > 0:
fmt.Fprintf(os.Stderr, "check builds older than %s: %d check(s)' builds let go of\n", builder.CheckBuildsKept, n)
}
}
sweep()
tick := time.NewTicker(every)
defer tick.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
sweep()
}
}
}
// begin records the build this machine took, with the cancel that ends it.
func (h *holder) begin(request link.BuildRequest, cancel context.CancelFunc) *running {
r := &running{request: request, started: time.Now(), cancel: cancel, done: make(chan struct{})}
+377 -63
View File
@@ -18,12 +18,16 @@ package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"net/url"
"os"
"os/signal"
"sort"
"strings"
"sync"
"sync/atomic"
"syscall"
"time"
@@ -54,6 +58,8 @@ is dialled except the broker.
MESH_BINDING a file the mesh wrote saying where the artifact store is
MESH_PACKAGE_BINDING a file the mesh wrote saying where the package registry is
MESH_NPM_TOKEN_FILE a file the mesh sealed holding the token for it
MESH_GENESIS_PACKAGE_REGISTRY the document the installer sealed at genesis, read while no
binding is present (novox/hq ADR 0336)
MESH_NPM_REGISTRY a package registry URL, when the mesh has not said (a person, the bootstrap)
MESH_NPM_TOKEN the token for it, likewise
MESH_NPM_SCOPE the scope it answers for (default: @novox)
@@ -66,6 +72,17 @@ broker to take work from or a registry to publish into:
Without --registry the artifacts stay in this machine's container runtime, named by the
digest of their own configuration. The result is printed as JSON.
mesh-builder ready
exits 0 once the agent running in this container is bound to its build work and pulls it, and
1 before: its container's health check.
mesh-builder busy
exits 0 while the agent running in this container has a build in hand, and 1 when it has none;
from the first ask the agent takes no new build: its container's busy command, asked before the
container is replaced (novox/hq ADR 0335).
`
func run() error {
@@ -76,12 +93,42 @@ func run() error {
return nil
case "build":
return buildOnce(context.Background(), os.Args[2:])
default:
case "ready":
// The container's health check (novox/hq issue 544): exits non-zero until the agent takes work, and
// once it has been drained longer than its node-engine would wait to replace it (ADR 0335).
if err := ready(); err != nil {
return err
}
return drainedTooLong(time.Now())
case "busy":
// The container's busy command (novox/hq ADR 0335): drains the agent, and exits non-zero once
// nothing is in hand.
said, err := busy()
if err != nil {
return err
}
fmt.Println(said)
return nil
case "help", "-h", "--help":
fmt.Print(usage)
return nil
default:
// **An ask this build does not know fails** (novox/hq ADR 0335): an image older than `busy`, asked
// it, must read as idle — exiting 0 would hold its container for the node-engine's whole bound.
fmt.Fprint(os.Stderr, usage)
return fmt.Errorf("%q is not a command this mesh-builder knows", os.Args[1])
}
}
if err := notReady(); err != nil {
return err
}
// **The builder's own version is part of every reuse key** (novox/hq ADR 0350 §1): a builder that builds
// differently never takes a build another one made.
builder.Version = builder.SelfVersion(version)
if err := undrained(); err != nil {
return err
}
credential, err := brokerFrom()
if err != nil {
return err
@@ -134,7 +181,13 @@ func run() error {
// **What an earlier holder here left goes before anything is taken** (novox/hq issue 306): a holder
// recreated mid-check left the check's store and bus running, and the ask went to another machine.
h.removeLeftBehind()
machine := link.MachineOverNATSWith(js, on, seat, link.MachineOptions{Paused: h.Paused})
// Paused or drained, it takes nothing new (novox/hq ADR 0219, ADR 0335): a drained agent finishes the build
// in hand and then says idle, and its container is replaced between builds.
machine := link.MachineOverNATSWith(js, on, seat, link.MachineOptions{
Paused: func() bool { return h.Paused() || drained() },
Taking: func() { taking(seat) },
Claim: claim,
})
defer machine.Close()
// The seat's verbs, on this machine's subjects. Only the seat that declares them: the retired
@@ -158,8 +211,13 @@ func run() error {
fmt.Fprintf(os.Stderr, "building for the mesh, publishing to %s\n", registry)
publisher := builder.Registry{Address: registry, Run: builder.Command}
// Check builds are let go of after a week (novox/hq ADR 0350 §5).
h.scratch = publisher
go h.expireChecks(ctx, time.Hour, time.Now)
return machine.Take(ctx, func(ctx context.Context, work link.Build) {
defer released()
holding(work.Request().Repository + " (" + work.Request().ID + ")")
answer(ctx, publisher, on, workspace, work, h)
})
}
@@ -216,7 +274,12 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
// role's own events under the build's id (novox/hq ADR 0157) — so whoever asked, and anybody
// watching, reads the same lines this container's log holds, live, and after the fact from the
// stream. Said first, before anything runs, so a build that hangs is one that visibly started.
// One line at a time: a check's builds for its merge say their lines while its tests say theirs (novox/hq
// issue 716).
var saying sync.Mutex
say := func(step, message string) {
saying.Lock()
defer saying.Unlock()
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
work.Say(step, message)
if mine != nil && step != "run" && step != "output" {
@@ -228,6 +291,26 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
if err := work.Began(ctx); err != nil {
fmt.Fprintf(os.Stderr, "cannot say a build started: %v\n", err)
}
// **A running check watches its pull request's key** (novox/hq ADR 0406 §4, issue 974): when the head or the trunk
// commit in it moves, the check is for a state nobody wants any more, and ends itself — nothing is asked of the forge
// (issue 726). A group's check, a try and a build watch nothing.
var superseded atomic.Bool
if w, ok := work.(interface {
WatchPull(context.Context, func(link.PullKey)) (func(), error)
}); ok && request.Check != nil {
stopWatching, err := w.WatchPull(building, func(k link.PullKey) {
superseded.Store(true)
say("superseded", fmt.Sprintf("the pull request's key moved to %.12s on %.12s while this checked %.12s on %.12s: stopped",
k.Head, k.Trunk, request.Ref, request.Check.Trunk))
cancel()
})
if err != nil {
fmt.Fprintf(os.Stderr, "the key of %s/%s#%d could not be watched, so its check runs on if the key moves: %v\n",
request.Check.Owner, request.Check.Repo, request.Check.Number, err)
} else {
defer stopWatching()
}
}
result := link.BuildResult{
ID: request.ID, Repository: request.Repository, Path: request.Path,
@@ -242,7 +325,11 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
}
say("build", what)
npmrc, err := packagesFrom()
npmrc, err := packagesFrom(on)
if err == nil {
say("packages", saysRegistry(npmrc))
}
registryHere, isRegistry := publisher.(builder.Registry)
var built builder.Result
if request.Check != nil {
// **A pull request's merge check, not a build** (novox/hq to-be 45 §9): nothing is built,
@@ -252,11 +339,37 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
if r, ok := publisher.(builder.Registry); ok {
registry = r.Address
}
spec := checkSpecOf(request)
// The judge cache the build agent declares as its own data, never in the workspace: only judge builds and
// mesh-lab's replays mount it, never a check's containers, so no pull request's script can write what a judge
// is built with or kept as, or what the replays run (novox/hq ADR 0358 §6, issue 709).
spec.JudgeCache = judgeCache()
// What the merge would move, built by the check for the walk to take (novox/hq ADR 0350 §1): into the
// scratch namespace. **Never with the package registry's credential**: this is code nobody has approved
// yet, and a recipe could read the credential it is handed (issue 462). A module that resolves packages
// at build time has no reuse key anyway, so nothing is lost by refusing it here.
if isRegistry && len(spec.Builds) > 0 {
spec.BuildWith = &builder.CheckBuilding{Publish: publisher, Scratch: registryHere, Held: request.Held,
Running: request.Check.Running, Seats: request.Seats, Named: namedOf(request.Check.Named)}
}
var v builder.CheckVerdict
v, err = builder.Check(building, builder.Command, checkSpecOf(request), workspace, registry, forgeFrom(), say)
v, err = builder.Check(building, builder.Command, spec, workspace, registry, forgeFrom(), say)
if err == nil {
result.Check = &link.CheckOutcome{Verdict: v.Verdict, Summary: v.Summary, Report: v.Report,
Took: v.Took.Round(time.Second).String(), Gate: layerOf(v.Gate), RepoCheck: layerOf(v.Repo)}
Took: v.Took.Round(time.Second).String(), Gate: layerOf(v.Gate), RepoCheck: layerOf(v.Repo),
Head: v.Head, Trunk: v.Trunk, Beside: v.Beside, Built: checkBuiltOf(v.Built),
Validator: v.Validator, Siblings: siblingsOut(v.Siblings), SiblingsUnknown: v.SiblingsUnknown, JudgeFrom: v.JudgeFrom}
// What each of a group's heads was judged as (novox/hq ADR 0323 §6).
for _, m := range v.Members {
result.Check.Members = append(result.Check.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo,
Commit: m.Head, Trunk: m.Trunk})
}
// A branch tried before its pull request reads its outcome from this log, whole (novox/hq issue 686).
if request.Check.Try {
if line, err := json.Marshal(triedOf(v)); err == nil {
say(link.StepTried, string(line))
}
}
}
} else if err == nil {
// The package-registry credential is a build input, so it is resolved before the clone: a
@@ -264,9 +377,22 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
// after a clone that then fails at npm ci.
// Every container it starts is labelled with its id, so a kill finds what outlived the
// docker client (ADR 0219).
built, err = builder.Build(building, builder.Labelled(builder.Command, request.ID), publisher,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats)
if request.Reuse != nil && isRegistry && !request.DryRun {
// **A walk's build takes the check's build of the same key** (novox/hq ADR 0350 §2).
reuse := builder.Reuse{Scratch: registryHere}
for _, p := range request.Reuse.Pulls {
reuse.Pulls = append(reuse.Pulls, builder.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number})
}
// Only what a verdict names is taken (ADR 0350 §2).
reuse.Named = namedOf(request.Reuse.Builds)
built, err = builder.BuildReusing(building, builder.Labelled(builder.Command, request.ID), publisher, reuse,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats)
} else {
built, err = builder.Build(building, builder.Labelled(builder.Command, request.ID), publisher,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats)
}
}
// What it copied into the store, whatever became of the build (novox/hq ADR 0257).
result.Mirrored = built.Mirrored
@@ -276,7 +402,11 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
if mine != nil {
killed = h.returned(mine) && err != nil
}
if killed {
if superseded.Load() && err != nil {
// Ended by its own key moving on (ADR 0406 §4): said as the check's failure, as a kill is.
result.Failed = link.SupersededByKey
say("failed", link.SupersededByKey)
} else if killed {
// **Killed by hand is the outcome, whatever the build was doing** (novox/hq ADR 0219): the
// error it ended with is the kill's consequence, not a fault of the source.
result.Failed = link.KilledByHand
@@ -302,10 +432,31 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
}
result.Against = built.Against
result.SourceFingerprint = built.Source
// What the walk did instead of building, or why it built (novox/hq ADR 0350 §3, §4).
result.ReuseKey = built.ReuseKey
if built.Reused != nil {
f := built.Reused.From
result.Reused = built.Reused.Words()
result.ReusedFrom = &link.ReusedFrom{Pull: link.PullRef{Owner: f.Pull.Owner, Repo: f.Pull.Repo,
Number: f.Pull.Number}, Head: f.Head, Trunk: f.Trunk, Check: f.Check, Key: f.Key}
} else if request.Reuse != nil {
result.BuiltWhy = built.Why
if result.BuiltWhy == "" {
result.BuiltWhy = "it has no artifact to build"
}
}
result.Trunk, result.OnTrunk, result.Branches = built.Trunk, built.OnTrunk, built.Branches
for _, r := range built.Read {
result.Read = append(result.Read, link.ReadRepository{Repository: r.Repository, Ref: r.Ref})
}
for _, s := range built.Sources {
result.Sources = append(result.Sources, link.BuildSource{Repository: s.Repository, Ref: s.Ref, Paths: s.Paths})
}
// The list of predefined keys the manifest was judged against, for the controller to publish
// (novox/hq ADR 0348 §7).
if p := built.Predefined; p != nil {
result.Predefined = &link.PredefinedList{Keys: p.Keys, Generation: p.Generation}
}
say("built", built.Manifest.Module+" from "+short(built.Commit))
}
}
@@ -345,17 +496,88 @@ func checkSpecOf(request link.BuildRequest) builder.CheckSpec {
spec := builder.CheckSpec{ID: request.ID, Repository: request.Repository, Ref: request.Ref,
Owner: c.Owner, Repo: c.Repo, Number: c.Number, Paths: c.Paths, Beside: map[string]builder.Beside{},
Modules: c.Modules, New: c.New, Manifests: c.Manifests, Judge: c.Judge, Base: c.Base,
Toolchain: builder.ToolchainOf(request.Held), Toolchains: builder.ToolchainsOf(request.Held)}
Toolchain: builder.ToolchainOf(request.Held), Toolchains: builder.ToolchainsOf(request.Held),
Tests: c.Tests, StoreOnly: c.StoreOnly, SiblingsUnknown: c.SiblingsUnknown}
for _, s := range c.Siblings {
spec.Siblings = append(spec.Siblings, builder.Sibling{Repository: s.Repository, Number: s.Number, Head: s.Head,
Base: s.Base, Dir: s.Dir})
}
for dir, b := range c.Beside {
spec.Beside[dir] = builder.Beside{Repository: b.Repository, Ref: b.Ref}
}
for _, b := range c.Builds {
spec.Builds = append(spec.Builds, builder.CheckBuildSpec{Module: b.Module, Repository: b.Repository, Path: b.Path,
Ref: b.Ref, Merged: b.Merged, Tier: b.Tier, Skip: b.Skip})
}
for _, m := range c.Members {
spec.Group = append(spec.Group, builder.GroupHead{Owner: m.Owner, Repo: m.Repo, Repository: m.Repository,
Ref: m.Ref, Paths: m.Paths})
Ref: m.Ref, Paths: m.Paths, Base: m.Base})
}
return spec
}
// siblingsOut is the siblings a check read as the outcome carries them (novox/hq ADR 0382 §4).
func siblingsOut(read []builder.JudgedSibling) []link.CheckSibling {
var out []link.CheckSibling
for _, s := range read {
out = append(out, link.CheckSibling{Repository: s.Repository, Number: s.Number, Head: s.Head, Base: s.Base,
BaseCommit: s.BaseCommit})
}
return out
}
// judgeCache is the judge cache the build agent's manifest declares (MESH_JUDGE_CACHE, novox/hq ADR 0358 §6): a
// directory mounted at the same path on the host, so a judge build's mounts name what the agent sees. Empty when the
// agent declares none, and every judge is then built cold under its check's root.
func judgeCache() string { return os.Getenv("MESH_JUDGE_CACHE") }
// triedOf is a try's outcome as its log line says it (novox/hq issue 686): the repository's own check's verdict,
// or the check's where that did not run — a head that does not merge — and the lines naming what failed.
func triedOf(v builder.CheckVerdict) link.Tried {
t := link.Tried{Verdict: v.Verdict, Summary: v.Summary, Took: v.Took.Round(time.Second).String(),
Head: v.Head, Trunk: v.Trunk}
if v.Repo != nil && (v.Gate == nil || v.Gate.Verdict == "pass") {
t.Verdict, t.Summary = v.Repo.Verdict, v.Repo.Summary
}
if v.Repo != nil && v.Repo.Failed != "" {
t.Failed = strings.Split(v.Repo.Failed, "\n")
}
if t.Verdict == "" {
t.Verdict = "error"
}
return t
}
// namedOf are the builds the seat's verdicts name, as the builder believes a scratch record by them (novox/hq ADR
// 0350 §2, issue 703): each module's key with every artifact's digest.
func namedOf(builds []link.CheckedBuild) []builder.NamedBuild {
var out []builder.NamedBuild
for _, b := range builds {
named := builder.NamedBuild{Pull: builder.PullRef{Owner: b.Pull.Owner, Repo: b.Pull.Repo,
Number: b.Pull.Number}, Check: b.Check, Module: b.Built.Module, Key: b.Built.Key,
Digests: map[string]string{}}
for _, m := range b.Built.Made {
named.Digests[m.Name] = builder.DigestOf(m.Reference)
}
out = append(out, named)
}
return out
}
// checkBuiltOf is a check's builds for its merge as the outcome carries them (novox/hq ADR 0350 §1).
func checkBuiltOf(built []builder.CheckBuilt) []link.CheckBuilt {
var out []link.CheckBuilt
for _, b := range built {
c := link.CheckBuilt{Module: b.Module, Key: b.Key, NotBuilt: b.NotBuilt, Took: b.Took.Round(time.Second).String(),
Kept: b.Kept, Same: b.Same, SameAs: b.SameAs}
for _, m := range b.Made {
c.Made = append(c.Made, link.MadeArtifact{Name: m.Name, Kind: m.Kind, Reference: m.Reference})
}
out = append(out, c)
}
return out
}
// layerOf is one layer of a check as the outcome carries it.
func layerOf(l *builder.Layer) *link.CheckLayer {
if l == nil {
@@ -365,79 +587,171 @@ func layerOf(l *builder.Layer) *link.CheckLayer {
}
// packagesFrom is where a build resolves the mesh's own published packages — the SDK above all
// (novox/hq ADR 0076, issue 053).
// (novox/hq ADR 0076, issue 053) — on this machine.
//
// Preferably from the mesh: a package-registry binding names the endpoint the way the artifact
// store's binding does, and a sealed token file the credential the way the broker's does. The
// environment variables remain for a builder run by a person, and for the bootstrap, where there is
// no registry yet — there the result is disabled and a build that needs no mesh-published dependency
// builds anyway.
func packagesFrom() (builder.Npmrc, error) {
// **The binding wins, then the genesis document** (novox/hq ADR 0336 §3, §4). The build agent's
// npm-package-registry requirement is optional: on genesis's node, before the gitea module is assigned,
// nothing provides it, and the installer sealed the forge it raised to this agent as one document
// instead. Once the requirement binds, the binding names the endpoint the way the artifact store's
// does and a sealed token file the credential, and the genesis document is left unread. A binding
// file named and absent is a requirement not bound here, never a fault: the environment names the
// path whether or not the mesh wrote it.
//
// The environment variables remain for a builder run by a person. With none of the three the result
// is disabled and says this machine, so a build that needs the registry is refused naming it (§6),
// and a build that needs none builds anyway.
func packagesFrom(machine string) (builder.Npmrc, error) {
scope := strings.TrimSpace(os.Getenv("MESH_NPM_SCOPE"))
if scope == "" {
scope = "@novox"
}
none := builder.Npmrc{Machine: machine}
registry := strings.TrimSpace(os.Getenv("MESH_NPM_REGISTRY"))
var username string
if path := strings.TrimSpace(os.Getenv("MESH_PACKAGE_BINDING")); path != "" {
raw, err := os.ReadFile(path)
if err != nil {
return builder.Npmrc{}, fmt.Errorf("cannot read what the mesh said about the package registry: %w", err)
switch {
case errors.Is(err, os.ErrNotExist):
// Not bound on this machine: the genesis document, or nothing.
case err != nil:
return none, fmt.Errorf("cannot read what the mesh said about the package registry: %w", err)
default:
return fromBinding(path, raw, scope, machine)
}
var told struct {
From string `json:"from"`
At string `json:"at"`
As string `json:"as"`
Serves map[string]any `json:"serves"`
}
if path := strings.TrimSpace(os.Getenv("MESH_GENESIS_PACKAGE_REGISTRY")); path != "" {
raw, err := os.ReadFile(path)
switch {
case errors.Is(err, os.ErrNotExist):
// Not this mesh's genesis node, or the installer has not sealed it yet.
case err != nil:
return none, fmt.Errorf("cannot read the genesis document for the package registry: %w", err)
default:
return fromGenesis(path, raw, scope, machine)
}
if err := json.Unmarshal(raw, &told); err != nil {
return builder.Npmrc{}, fmt.Errorf("%s is not a binding: %w", path, err)
}
if told.At == "" {
return builder.Npmrc{}, fmt.Errorf(
"%s says the package registry is on %q and gives no address for it", path, told.From)
}
// Composed from what the provider serves, so nothing here knows gitea's URL shape from
// another registry's: it states its port, the path its registry answers on, and the scheme.
scheme := "https"
if s, ok := told.Serves["scheme"]; ok {
scheme = fmt.Sprintf("%v", s)
}
port, ok := told.Serves["port"]
if !ok {
return builder.Npmrc{}, fmt.Errorf("%s says nothing about which port the package registry answers on", path)
}
npmPath, ok := told.Serves["npm-path"]
if !ok {
return builder.Npmrc{}, fmt.Errorf("%s says nothing about the path the package registry answers on", path)
}
registry = fmt.Sprintf("%s://%s:%v%v", scheme, told.At, port, npmPath)
username = told.As
}
// The credential the mesh sealed to this machine. The mesh authenticates the ordinary way — a
// generated password the provider only applies (novox/hq ADR 0048) — so with a username this is
// a password (basic auth); without one it is a bearer token a provider minted.
// A builder run by a person.
registry := strings.TrimSpace(os.Getenv("MESH_NPM_REGISTRY"))
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
if path := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN_FILE")); path != "" {
raw, err := os.ReadFile(path)
username := strings.TrimSpace(os.Getenv("MESH_NPM_USER"))
if registry == "" && secret == "" {
return none, nil
}
n := builder.Npmrc{Scope: scope, Registry: registry, From: builder.FromEnvironment, Machine: machine}
if username != "" {
n.Username, n.Password = username, secret
} else {
n.Token = secret
}
return n, nil
}
// fromBinding composes the registry from the npm-package-registry binding the mesh wrote and the
// credential it sealed beside it. The mesh authenticates the ordinary way — a generated password the
// provider only applies (novox/hq ADR 0048) — so with a username the secret is a password (basic
// auth); without one it is a bearer token a provider minted.
func fromBinding(path string, raw []byte, scope, machine string) (builder.Npmrc, error) {
none := builder.Npmrc{Machine: machine}
var told struct {
From string `json:"from"`
At string `json:"at"`
As string `json:"as"`
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal(raw, &told); err != nil {
return none, fmt.Errorf("%s is not a binding: %w", path, err)
}
if told.At == "" {
return none, fmt.Errorf(
"%s says the package registry is on %q and gives no address for it", path, told.From)
}
// Composed from what the provider serves, so nothing here knows gitea's URL shape from
// another registry's: it states its port, the path its registry answers on, and the scheme.
scheme := "https"
if s, ok := told.Serves["scheme"]; ok {
scheme = fmt.Sprintf("%v", s)
}
port, ok := told.Serves["port"]
if !ok {
return none, fmt.Errorf("%s says nothing about which port the package registry answers on", path)
}
npmPath, ok := told.Serves["npm-path"]
if !ok {
return none, fmt.Errorf("%s says nothing about the path the package registry answers on", path)
}
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
if file := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN_FILE")); file != "" {
raw, err := os.ReadFile(file)
if err != nil {
return builder.Npmrc{}, fmt.Errorf("cannot read this builder's package-registry credential: %w", err)
return none, fmt.Errorf("cannot read this builder's package-registry credential: %w", err)
}
secret = strings.TrimSpace(string(raw))
}
if u := strings.TrimSpace(os.Getenv("MESH_NPM_USER")); u != "" {
username = u
n := builder.Npmrc{Scope: scope, Registry: fmt.Sprintf("%s://%s:%v%v", scheme, told.At, port, npmPath),
From: builder.FromBinding, Machine: machine}
if told.As != "" {
n.Username, n.Password = told.As, secret
} else {
n.Token = secret
}
return n, nil
}
if registry == "" && secret == "" {
return builder.Npmrc{}, nil
// GenesisRegistry is the genesis document (novox/hq ADR 0336 §3): what the installer sealed to the
// build agent on its node at genesis's packages step, as the `genesis-package-registry` own-secret —
// the forge it raised, the port this node gave the package registry, its npm path, and the account it
// made with that account's password. mesh-host writes it; the field names are the two repositories'
// agreement.
type GenesisRegistry struct {
Scheme string `json:"scheme"`
At string `json:"at"`
Port int `json:"port"`
NpmPath string `json:"npm-path"`
As string `json:"as"`
Password string `json:"password"`
}
// fromGenesis composes the registry from the genesis document.
func fromGenesis(path string, raw []byte, scope, machine string) (builder.Npmrc, error) {
none := builder.Npmrc{Machine: machine}
var g GenesisRegistry
if err := json.Unmarshal(raw, &g); err != nil {
return none, fmt.Errorf("%s is not the genesis document for the package registry: %w", path, err)
}
if username != "" {
return builder.Npmrc{Scope: scope, Registry: registry, Username: username, Password: secret}, nil
var missing []string
for name, value := range map[string]string{"at": g.At, "npm-path": g.NpmPath, "as": g.As, "password": g.Password} {
if strings.TrimSpace(value) == "" {
missing = append(missing, name)
}
}
return builder.Npmrc{Scope: scope, Registry: registry, Token: secret}, nil
if g.Port <= 0 {
missing = append(missing, "port")
}
if len(missing) > 0 {
sort.Strings(missing)
return none, fmt.Errorf("%s, the genesis document for the package registry, says nothing of %s",
path, strings.Join(missing, ", "))
}
scheme := g.Scheme
if scheme == "" {
scheme = "http"
}
return builder.Npmrc{Scope: scope, Registry: fmt.Sprintf("%s://%s:%d%s", scheme, g.At, g.Port, g.NpmPath),
Username: g.As, Password: g.Password, From: builder.FromGenesis, Machine: machine}, nil
}
// saysRegistry is the line each build's log carries about the package registry (novox/hq ADR 0336
// §4): which of the binding and the genesis document it resolved from, or that it has neither here.
func saysRegistry(n builder.Npmrc) string {
if n.Enabled() {
return "the package registry is resolved from " + n.From
}
machine := n.Machine
if machine == "" {
machine = "this machine"
}
return fmt.Sprintf("no package registry: %s is unbound on %s and no genesis document is here; "+
"a build needing it is refused", builder.RegistryRequirement, machine)
}
// forgeFrom is the git credential this builder may offer a clone, composed from the same binding
+6 -1
View File
@@ -84,10 +84,15 @@ func buildOnce(ctx context.Context, args []string) error {
}
fmt.Fprintln(os.Stderr)
npmrc, err := packagesFrom()
machine := os.Getenv("MESH_NODE")
if machine == "" {
machine, _ = os.Hostname()
}
npmrc, err := packagesFrom(machine)
if err != nil {
return err
}
fmt.Fprintf(os.Stderr, " [packages] %s\n", saysRegistry(npmrc))
built, buildErr := builder.Build(ctx, builder.Command, publisher, repository, *path, *ref, where, bases, npmrc,
forgeFrom(),
func(step, message string) { fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message) })
+102
View File
@@ -0,0 +1,102 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/builder"
)
// The build agent's package registry (novox/hq ADR 0336, issue 121): its npm-package-registry
// requirement is optional, so the binding may be absent, and on genesis's node the installer seals
// the forge it raised as one genesis document instead.
// registryFiles names the three paths the build agent's environment always names, writing only those
// given.
func registryFiles(t *testing.T, binding, token, genesis string) {
t.Helper()
dir := t.TempDir()
for env, content := range map[string]string{
"MESH_PACKAGE_BINDING": binding,
"MESH_NPM_TOKEN_FILE": token,
"MESH_GENESIS_PACKAGE_REGISTRY": genesis,
} {
path := filepath.Join(dir, env)
t.Setenv(env, path)
if content != "" {
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatal(err)
}
}
}
for _, env := range []string{"MESH_NPM_REGISTRY", "MESH_NPM_TOKEN", "MESH_NPM_USER", "MESH_NPM_SCOPE"} {
t.Setenv(env, "")
}
}
const (
aBinding = `{"from":"gitea","at":"anchor.mesh","as":"anchor-agent",
"serves":{"scheme":"http","port":3000,"npm-path":"/api/packages/novox/npm/"}}`
aGenesisDocument = `{"scheme":"http","at":"127.0.0.1","port":3100,"npm-path":"/api/packages/novox/npm/",
"as":"mesh-builder","password":"genesis-secret"}`
)
func TestUntilTheRequirementBindsTheGenesisDocumentIsRead(t *testing.T) {
registryFiles(t, "", "", aGenesisDocument)
n, err := packagesFrom("anchor")
if err != nil {
t.Fatalf("an absent binding beside a genesis document was refused: %v", err)
}
if n.From != builder.FromGenesis || n.Registry != "http://127.0.0.1:3100/api/packages/novox/npm/" ||
n.Username != "mesh-builder" || n.Password != "genesis-secret" || n.Scope != "@novox" {
t.Fatalf("the genesis document was not what the registry came from: %+v", n)
}
file, err := n.File()
if err != nil || !strings.Contains(file, "//127.0.0.1:3100/api/packages/novox/npm/:username=mesh-builder") {
t.Fatalf("the .npmrc is not written from the genesis document: %v\n%s", err, file)
}
if got := saysRegistry(n); !strings.Contains(got, "genesis document") {
t.Errorf("the log does not say the genesis document: %s", got)
}
}
func TestARealBindingWinsOverTheGenesisDocument(t *testing.T) {
registryFiles(t, aBinding, "bound-secret\n", aGenesisDocument)
n, err := packagesFrom("anchor")
if err != nil {
t.Fatal(err)
}
if n.From != builder.FromBinding || n.Registry != "http://anchor.mesh:3000/api/packages/novox/npm/" ||
n.Username != "anchor-agent" || n.Password != "bound-secret" {
t.Fatalf("the binding did not win over the genesis document: %+v", n)
}
if got := saysRegistry(n); !strings.Contains(got, "binding") {
t.Errorf("the log does not say the binding: %s", got)
}
}
func TestWithNeitherTheAgentHasNoRegistryAndSaysWhere(t *testing.T) {
registryFiles(t, "", "", "")
n, err := packagesFrom("workstation")
if err != nil {
t.Fatalf("an unbound optional requirement was a fault: %v", err)
}
if n.Enabled() || n.Machine != "workstation" {
t.Fatalf("a registry came from nowhere: %+v", n)
}
got := saysRegistry(n)
for _, want := range []string{"npm-package-registry", "unbound", "workstation"} {
if !strings.Contains(got, want) {
t.Errorf("the log does not say %q: %s", want, got)
}
}
}
func TestAGenesisDocumentMissingAFieldIsRefusedByName(t *testing.T) {
registryFiles(t, "", "", `{"at":"127.0.0.1","port":3000,"npm-path":"/api/packages/novox/npm/","as":"mesh-builder"}`)
if _, err := packagesFrom("anchor"); err == nil || !strings.Contains(err.Error(), "password") {
t.Fatalf("a genesis document with no password was taken: %v", err)
}
}
+44
View File
@@ -0,0 +1,44 @@
package main
import (
"errors"
"fmt"
"os"
"path/filepath"
)
// **The build agent says it is ready** (novox/hq issue 544). It listens on nothing, so no http or tcp
// check can see it work; it writes a mark once it is bound to its worker and pulls build work, and its
// container's health check runs `mesh-builder ready`, which passes only while the mark is there. Without
// it the node-engine could judge the agent only alive, and every walk that moved it waited out the whole
// grace on each machine.
//
// The mark lives in the container's own temporary directory, never in the workspace the machine keeps:
// a new container starts without it, and a restarted one removes it before it binds again.
func readyMark() string { return filepath.Join(os.TempDir(), "mesh-builder.ready") }
// notReady removes the mark: a holder starting is not taking work until it says so again.
func notReady() error {
if err := os.Remove(readyMark()); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove the mark that this agent takes build work (%s): %w", readyMark(), err)
}
return nil
}
// taking writes the mark. A mark that cannot be written is said, and the agent goes on building: its
// check then says it is not ready, which is the truth the gate should hear.
func taking(seat string) {
if err := os.WriteFile(readyMark(), []byte(seat+"\n"), 0o644); err != nil {
fmt.Fprintf(os.Stderr, "cannot mark that this agent takes build work (%s): %v\n", readyMark(), err)
return
}
fmt.Fprintf(os.Stderr, "taking build work from %s\n", seat)
}
// ready is the health check: nil when this container's agent is bound and pulling build work.
func ready() error {
if _, err := os.Stat(readyMark()); err != nil {
return fmt.Errorf("not taking build work yet: %s is not there", readyMark())
}
return nil
}
+32
View File
@@ -0,0 +1,32 @@
package main
import (
"os"
"testing"
)
// The health check passes only between the agent saying it takes work and its next start (novox/hq
// issue 544).
func TestTheAgentIsReadyOnlyOnceItTakesWork(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := ready(); err == nil {
t.Fatal("an agent that never took work is ready")
}
taking("mesh-build-machine")
if err := ready(); err != nil {
t.Fatalf("an agent taking work is not ready: %v", err)
}
// Started again, it is not ready until it binds again.
if err := notReady(); err != nil {
t.Fatal(err)
}
if err := ready(); err == nil {
t.Fatal("an agent starting again is ready before it takes work")
}
if err := notReady(); err != nil {
t.Fatalf("removing a mark that is not there failed: %v", err)
}
if _, err := os.Stat(readyMark()); err == nil {
t.Fatal("the mark is still there")
}
}
+1
View File
@@ -16,6 +16,7 @@ func TestBuilderDiagnosticsStayOffStdout(t *testing.T) {
"version)": true, // --version
`"stopping")`: true, // the loop.s shutdown line
"usage)": true, // --help text, for a human
"said)": true, // busy: the answer its node-engine reads (novox/hq ADR 0335)
}
for _, file := range []string{"once.go", "main.go"} {
src, err := os.ReadFile(file)
+31
View File
@@ -0,0 +1,31 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/builder"
)
// A try's outcome, as the build seat says it in the build's log (novox/hq issue 686): the repository's own
// check's verdict, how long it took, and each line naming what failed.
func TestATrysOutcomeNamesWhatFailed(t *testing.T) {
tried := triedOf(builder.CheckVerdict{Verdict: "pass", Summary: "the change touches no module", Took: 303 * time.Second,
Head: "c0ffee", Trunk: "ca3f43c", Gate: &builder.Layer{Verdict: "pass"},
Repo: &builder.Layer{Verdict: "fail", Summary: "its merge-check.sh failed: TestX",
Failed: "--- FAIL: TestX (0.01s)\n x_test.go:12: got 1, want 2"}})
if tried.Verdict != "fail" || tried.Took != "5m3s" || len(tried.Failed) != 2 || tried.Trunk != "ca3f43c" ||
tried.Summary != "its merge-check.sh failed: TestX" {
t.Fatalf("a failed try: %+v", tried)
}
// A head that does not merge fails both layers: the check's own word.
conflict := triedOf(builder.CheckVerdict{Verdict: "fail", Summary: "does not merge", Gate: &builder.Layer{Verdict: "fail"},
Repo: &builder.Layer{Verdict: "fail", Summary: "does not merge"}})
if conflict.Verdict != "fail" || conflict.Summary != "does not merge" {
t.Fatalf("a try that does not merge: %+v", conflict)
}
// No verdict is an error, never a pass.
if none := triedOf(builder.CheckVerdict{}); none.Verdict != "error" {
t.Fatalf("a try with no verdict: %+v", none)
}
}
+145
View File
@@ -0,0 +1,145 @@
package main
import (
"context"
"fmt"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// A provision whose credential only a person outside the mesh can give (novox/hq ADR 0337, issue 164, issue 165):
// given once for its provider, at the controller's terminal, and sealed in one act to every consumer that binds the
// provision from that provider now. The value is never read back, printed or kept.
// outsideOffer is the provider module's own name when `secret accept <node> <module> <name>` names a provision that
// module offers with a credential issued outside the mesh, and "" when it names anything else (an own secret, or a
// pair with --provider).
func outsideOffer(ctx context.Context, module, name, provider string) (string, error) {
if provider != "" {
return "", nil
}
open, err := openStores(ctx)
if err != nil {
return "", err
}
defer open.Close()
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return "", err
}
if m, known := shelf[module]; known && m.IssuedOutsideOf(name) {
return module, nil
}
return "", nil
}
// offeredOutsideBy is a module that offers the provision with a credential issued outside the mesh, or "".
func offeredOutsideBy(ctx context.Context, provision string) (string, error) {
open, err := openStores(ctx)
if err != nil {
return "", err
}
defer open.Close()
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return "", err
}
module, _ := catalogue.OffersIssuedOutside(shelf, provision)
return module, nil
}
// consumersFrom is every consumer that binds a provision from one provider now, found by resolving every machine as
// its next send would, and the machines that could not be resolved, with why.
func consumersFrom(ctx context.Context, open *stores, provider, providerModule, provision string) (
[]inventory.PairEnd, []string, error) {
nodes, err := open.inventory.Nodes(ctx)
if err != nil {
return nil, nil, err
}
var out []inventory.PairEnd
var unresolved []string
for _, n := range nodes {
plan, _, err := planFor(ctx, open, n.Name)
if err != nil {
unresolved = append(unresolved, fmt.Sprintf("%s: %s", n.Name, oneLine(err.Error())))
continue
}
out = append(out, bindsFrom(plan, n.Name, provider, providerModule, provision)...)
}
return out, unresolved, nil
}
// bindsFrom is every consumer on one machine whose resolution answers the provision from that provider. Pure.
func bindsFrom(plan catalogue.Resolution, machine, provider, providerModule, provision string) []inventory.PairEnd {
var out []inventory.PairEnd
for _, need := range plan.Needs {
if need.ByRecord || need.Name != provision || need.From != provider {
continue
}
if need.Module != "" && need.Module != providerModule {
continue
}
out = append(out, inventory.PairEnd{Consumer: machine, ConsumerModule: need.For, Local: need.Local})
}
sort.Slice(out, func(i, j int) bool {
return out[i].ConsumerModule+"\x00"+out[i].Local < out[j].ConsumerModule+"\x00"+out[j].Local
})
return out
}
// acceptForProvision gives a provision's credential issued outside the mesh, once for its provider (ADR 0337 §3).
func acceptForProvision(ctx context.Context, provider, providerModule, provision, value string) error {
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
consumers, unresolved, err := consumersFrom(ctx, open, provider, providerModule, provision)
if err != nil {
return err
}
accepted, err := open.inventory.AcceptSecretForProvision(ctx, provision, provider, providerModule, value, consumers)
value = ""
if err != nil {
return err
}
fmt.Print(acceptedSaid(provider, providerModule, provision, accepted, unresolved))
if len(accepted.Sealed) == 0 {
return nil
}
if err := sendTo(ctx, open, accepted.Machines); err != nil {
return fmt.Errorf("%s is sealed and could not be sent: %w; `push %s` sends it", provision, err,
strings.Join(accepted.Machines, "`, `push "))
}
fmt.Printf(" sent %s; each consumer starts again on it\n", strings.Join(accepted.Machines, ", "))
return nil
}
// acceptedSaid is what one accept for a provider did, in words that never carry the value. Pure.
func acceptedSaid(provider, providerModule, provision string, a inventory.ProvisionAccepted, unresolved []string) string {
var b strings.Builder
if len(a.Sealed) == 0 {
fmt.Fprintf(&b, "%s from %s on %s: no consumer binds it now, so nothing was sealed; a consumer that binds "+
"later waits for it, and this line given again seals it to every consumer then\n", provision,
providerModule, provider)
} else {
var names []string
for _, c := range a.Sealed {
names = append(names, fmt.Sprintf("%s on %s%s", c.ConsumerModule, c.Consumer, asLocal(c.Local)))
}
fmt.Fprintf(&b, "%s from %s on %s is given, sealed to %s and to %s\n", provision, providerModule, provider,
strings.Join(names, ", "), provider)
fmt.Fprintf(&b, " the mesh cannot read it back, never makes one of its own for it, and will not rotate it\n")
}
for _, c := range a.Unsealable {
fmt.Fprintf(&b, " not sealed to %s on %s: no sealing key on that machine; give it again once it has one\n",
c.ConsumerModule, c.Consumer)
}
for _, u := range unresolved {
fmt.Fprintf(&b, " not known whether a consumer on %s binds it, which did not resolve; give it again once it does\n", u)
}
return b.String()
}
+27 -4
View File
@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"os"
"strings"
"sync"
"time"
@@ -25,8 +26,11 @@ import (
//
// - **The serving controller** takes it before it does anything else — before it asserts the bus's
// objects, which are the controller's to write — waiting while another holds it, and renews it.
// A renewal refused or failed is the lease lost: the gate closes at once and the process exits, so
// its service manager restarts it as a candidate (serve, in push.go).
// Another holder seen, or the key's age passed with no renewal made, is the lease lost: the gate
// closes at once and the process exits, so its service manager restarts it as a candidate (serve,
// in push.go). A renewal the bus does not answer in time is not a loss (novox/hq issue 477): the
// lease is kept, no new act starts once half the key's age passed unrenewed, and the next renewal
// made opens the gate again.
// - **A command run at a shell** — `push` in the installer, the lab, a person repairing a mesh whose
// controller is down (issue 201) — acts **under the holder's epoch** when a controller holds the
// lease: it is the same mesh's word, composed and sent under the store's hold of each machine like
@@ -63,6 +67,11 @@ type actor struct {
// the highest epoch issued, and what was said of it; zero when it was not (S12).
reset time.Time
resetSaid string
// renewEvery and poll, when set, are how often the serving controller renews the lease it holds and
// looks again at one another holds, in place of the lease's own: a test's, so two controllers hand the
// lease over in a second rather than in a renewal of five (novox/hq issue 591). A renewal is given
// the lease's own RenewEvery to answer however often it is made, so a slow bus does not lose it.
renewEvery, poll time.Duration
}
// theLease is this process's standing to the lease.
@@ -179,7 +188,21 @@ func (a *actor) release() {
// holderOf is this process as the lease's holder.
func holderOf(instance string) lease.Holder {
host, _ := os.Hostname()
return lease.Holder{Instance: instance, Host: host, Build: version}
build := runningBuild()
if build == "" {
build = version
}
return lease.Holder{Instance: instance, Host: host, Build: build}
}
// RunningBuildVar is where the declaration tells this process which build it is (module.json, the
// controller process's env): the version its bundle is delivered as, `${version}` composed by the
// catalogue from the bundle's digest. Empty for a process placed by hand.
const RunningBuildVar = "MESH_CONTROLLER_VERSION"
// runningBuild is the version of the build this process is, or empty when the declaration did not say.
func runningBuild() string {
return strings.TrimSpace(os.Getenv(RunningBuildVar))
}
// serveUnderTheLease takes the lease for the serving controller, waiting while another holds it, and
@@ -208,7 +231,7 @@ func (a *actor) serveUnderTheLease(ctx context.Context, inv *inventory.Inventory
}
say := func(format string, args ...any) { fmt.Printf(format+"\n", args...) }
l, err := lease.Open(ctx, api, broker.LeaseBucket, lease.Options{Holder: holderOf(instance),
Floor: inv.HighestEpoch, Say: say, Health: controllerHealth, Moved: func(was, floor uint64) {
RenewEvery: a.renewEvery, RenewWithin: lease.RenewEvery, Poll: a.poll, Floor: inv.HighestEpoch, Say: say, Health: controllerHealth, Moved: func(was, floor uint64) {
a.mu.Lock()
defer a.mu.Unlock()
a.reset = time.Now()
+78 -21
View File
@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/broker"
"io"
"slices"
"sort"
"strings"
@@ -54,6 +55,9 @@ func assignWith(ctx context.Context, open *stores, node string, opts assignOptio
if err := refusedMovingTheController(modules); err != nil {
return "", err
}
// Every line of the answer comes from the one resolution after the write (novox/hq issue 237):
// what the reads before it would say about a machine left out is not the answer's.
ctx = leftOutSaidBy(ctx, io.Discard)
// Held while it is recorded, so it cannot land between a converge's preview and its flip and
// be taken without ever having been previewed (novox/hq ADR 0100).
ctx, release, err := holdNodes(ctx, open, []string{node})
@@ -125,22 +129,28 @@ func assignWith(ctx context.Context, open *stores, node string, opts assignOptio
for _, line := range settled {
answer += "\n " + line
}
// What this act changed about this node's unmet seat dependencies, and nothing else (novox/hq
// ADR 0207): a dependency of a module just assigned, or one this assignment met. The rest of the
// node's list, and every other node's, is `status`'s.
for _, line := range unheldChange(shelf, node, before, append(append([]string(nil), before...), added...)) {
answer += "\n " + line
}
// Its bus credential, in the same act (novox/hq issue 203): an assignment pushed before its
// credential exists delivers a process that cannot authenticate and crash-loops until somebody
// runs a second verb and a second push. Issued here when the module speaks on the bus and has
// no credential yet; kept when it has one, so re-assigning rotates nothing.
var issued []string
for _, module := range added {
if line := issueOnAssign(ctx, open, node, module); line != "" {
answer += "\n " + line
issued = append(issued, line)
}
}
plan, _, err := planFor(ctx, open, node)
// **The one resolution the rest of the answer is said from** (novox/hq issue 237), taken after
// the write: what this act changed about this node's unmet seat dependencies, and nothing else
// (novox/hq ADR 0207) — a dependency of a module just assigned, or one this assignment met; the
// rest of the node's list, and every other node's, is `status`'s — and whether the node resolves.
after := resolvedAfter(ctx, open, shelf, node)
for _, line := range unheldChangeTo(shelf, node, before, after.modules, after.unheld) {
answer += "\n " + line
}
for _, line := range issued {
answer += "\n " + line
}
plan, err := after.plan, after.err
if err != nil {
// Kept, and still refused. Both halves are the answer, and the rest of the mesh is still
// worth reporting: this machine's refusal is rarely the only consequence.
@@ -193,8 +203,9 @@ func seatDependenciesOnAssign(ctx context.Context, open *stores, node string, mo
if _, err := catalogue.AssignRefusal(shelf, node, assigned, adding); err != nil {
return nil, nil, err
}
// Two modules declaring one package, path or unit is refused before anything is recorded
// (novox/hq ADR 0210, 04-ISSUES/235): kept, the node would not resolve until one came off again.
// Two modules declaring one path or unit, or one package present and absent, is refused before anything
// is recorded (novox/hq ADR 0210, ADR 0303, 04-ISSUES/235): kept, the node would not resolve until one came
// off again. Two modules declaring one package present is not a collision (ADR 0303, issue 628).
if err := catalogue.CollisionRefusal(shelf, node, assigned, adding); err != nil {
return nil, nil, err
}
@@ -218,6 +229,8 @@ func unassign(ctx context.Context, open *stores, node string, modules ...string)
if len(modules) == 0 {
return "", fmt.Errorf("unassign %s names no module", node)
}
// One resolution after the write, as assign's answer (novox/hq issue 237).
ctx = leftOutSaidBy(ctx, io.Discard)
ctx, release, err := holdNodes(ctx, open, []string{node})
if err != nil {
return "", err
@@ -262,15 +275,22 @@ func unassign(ctx context.Context, open *stores, node string, modules ...string)
}
answer := fmt.Sprintf("%s no longer runs %s — run `push %s` to make it so",
node, strings.Join(modules, ", "), node)
var left []string
for _, a := range assigned {
if !slices.Contains(modules, a) {
left = append(left, a)
}
}
for _, line := range unheldChange(shelf, node, assigned, left) {
after := resolvedAfter(ctx, open, shelf, node)
for _, line := range unheldChangeTo(shelf, node, assigned, after.modules, after.unheld) {
answer += "\n " + line
}
// Whether the node itself still resolves, from the same resolution: before, only the rest of the
// mesh's reads said it, on standard error and once per other machine.
if after.err != nil {
if unresolvable(after.err) {
answer += fmt.Sprintf("\n but %s does not resolve now, so nothing will be sent to it:", node)
} else {
answer += fmt.Sprintf("\n but whether %s still resolves could not be read:", node)
}
for _, line := range strings.Split(strings.TrimRight(after.err.Error(), "\n"), "\n") {
answer += "\n " + strings.TrimSpace(line)
}
}
// What it leaves behind that is irreplaceable is kept and retired, never removed (novox/hq ADR 0233).
for _, line := range keptOnUnassign(ctx, open.inventory, node, modules) {
answer += "\n " + line
@@ -278,6 +298,39 @@ func unassign(ctx context.Context, open *stores, node string, modules ...string)
return answer + blockedElsewhere(ctx, open, node), nil
}
// afterAct is the one resolution of a node an act's answer is said from, taken after its write
// (novox/hq issue 237): the plan or why there is none, the modules it has on the node, and its unmet
// seat dependencies.
type afterAct struct {
plan catalogue.Resolution
err error
modules []string
unheld []catalogue.Unheld
}
// resolvedAfter resolves the node as `plan` does, once, after the act's write. When the node resolves,
// its modules and unmet seat dependencies are the resolution's — the closure, so a holder pulled in by
// a requirement counts as `plan` counts it. When it does not, they are judged over the assignments read
// after the write, which can only find more unmet than the closure would: a seat said held there is
// held in the closure too.
func resolvedAfter(ctx context.Context, open *stores, shelf map[string]catalogue.Manifest, node string) afterAct {
plan, _, err := planFor(ctx, open, node)
if err == nil {
out := afterAct{plan: plan, unheld: plan.Unheld}
for _, m := range plan.Modules {
out.modules = append(out.modules, m.Module)
}
return out
}
out := afterAct{err: err}
assigned, readErr := open.inventory.Assigned(ctx, node)
if readErr != nil {
return out
}
out.modules, out.unheld = assigned, unheldOver(shelf, node, assigned)
return out
}
// splitModules is a surface's one `module` field as the modules it names: several, comma-separated,
// are one act (novox/hq ADR 0207), so the holders that depend on each other go on together from the
// command API and the controller seat's verbs as they do from the command line.
@@ -338,10 +391,10 @@ func blockedElsewhere(ctx context.Context, open *stores, except string) string {
}
// issueOnAssign gives a newly assigned module its bus credential, the way `module issue` does, and
// says what it did in one line. Nothing for a module that declares no broker secret; nothing for one
// whose user is already minted (a credential is rotated on purpose, never by re-assigning); and when
// the bus cannot be reached from here, the line names the verb and the push that would refuse the
// module until it is run — never a silent placeholder (novox/hq issue 203).
// says what it did in one line. Nothing for a module that declares no broker secret and runs no
// provisioner client; nothing for one whose user is already minted (a credential is rotated on
// purpose, never by re-assigning); and when the bus cannot be reached from here, the line names the
// verb and the push that would refuse the module until it is run — never a silent placeholder (novox/hq issue 203).
func issueOnAssign(ctx context.Context, open *stores, node, module string) string {
inv := open.inventory
shelf, err := inv.Catalogue(ctx)
@@ -353,6 +406,10 @@ func issueOnAssign(ctx context.Context, open *stores, node, module string) strin
return ""
}
user := broker.Principal{Kind: broker.KindModule, Node: node, Module: module}.Username()
if _, reads := m.OwnSecrets["broker"]; !reads {
// A module running the provisioner client alone is minted the client's account (issue 917).
user = broker.Principal{Kind: broker.KindProvisioner, Node: node, Module: module}.Username()
}
if _, minted, err := inv.BusUserHash(ctx, user); err != nil || minted {
return ""
}
+2
View File
@@ -1,3 +1,5 @@
//go:build integration
package main
import (
@@ -0,0 +1,228 @@
//go:build integration
package main
import (
"encoding/json"
"os"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/licences"
)
func TestTheRegistryTrustAndEveryImageFollowThePortTheNodeGaveTheStore(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aStore())
if _, err := assign(ctx, open, "anchor", "distribution"); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetSettings(ctx, "anchor", "distribution",
map[string]any{catalogue.PortsSetting: map[string]any{"5000": 5101}}); err != nil {
t.Fatal(err)
}
// A module the mesh built, recorded by digest and path, running on the other machine.
if err := open.inventory.RecordBuild(ctx, inventory.Build{
ID: "b1", Repository: "r", Module: "app", Commit: "abc",
Made: []inventory.Artifact{{Name: "server", Kind: "image",
Reference: catalogue.ArtifactStoreScheme + "app/server@" + aDigest}},
}); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "app", Version: "1",
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-app",
"image": catalogue.ArtifactStoreScheme + "app/server@" + aDigest}}})
if _, err := assign(ctx, open, "laptop", "app"); err != nil {
t.Fatal(err)
}
// The runtime's trust is the runtime's module's to write (novox/hq ADR 0222): a stand-in for it
// asks where this machine reaches the store, as the docker module does.
register(t, open, catalogue.Manifest{Module: "runtime", Version: "1",
Resources: []map[string]any{{"id": "daemon", "type": "file", "path": "/etc/docker/daemon.json",
"into": "json", "content": `{"insecure-registries": ["${seat:mesh-artifact-store:reach}"]}` + "\n"}}})
if _, err := assign(ctx, open, "laptop", "runtime"); err != nil {
t.Fatal(err)
}
on := map[string]bool{"anchor": true, "laptop": true}
node, port, found, err := artifactStoreOnNetwork(ctx, open.inventory, on)
if err != nil || !found || node != "anchor" || port != "5101" {
t.Fatalf("the store is found on %q:%q (%v, %v); the node put it on 5101", node, port, found, err)
}
var trust string
for _, r := range composed(t, open, "laptop").Resources {
if r["path"] == "/etc/docker/daemon.json" {
trust, _ = r["content"].(string)
}
if r["id"] == "app.server" && r["image"] != "anchor.internal:5101/app/server@"+aDigest {
t.Errorf("the image the mesh built is fetched as %v", r["image"])
}
}
if !strings.Contains(trust, "anchor.internal:5101") || strings.Contains(trust, ":5000") {
t.Fatalf("the runtime is told to trust %q; the node put the store on 5101", trust)
}
// And a replay to the catalogue says where the store is now.
announced, err := following{open: open}.Announceable(ctx)
if err != nil {
t.Fatal(err)
}
if len(announced) != 1 || announced[0].Made[0].Reference != "anchor.internal:5101/app/server@"+aDigest {
t.Fatalf("the replay announces %+v", announced)
}
}
func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
m, err := catalogue.ParseManifest(raw)
if err != nil {
t.Fatal(err)
}
control, err := withSeatPorts(m).Resolve([]catalogue.Built{{Name: "controller", Kind: catalogue.ArtifactBundle,
Reference: "https://registry.example/mesh-controller/controller.tar.gz", Digest: aDigest}})
if err != nil {
t.Fatal(err)
}
register(t, open, control)
register(t, open, catalogue.Manifest{Module: "postgres", Version: "1",
Claims: []catalogue.Claim{{Name: "mesh-store", Scope: catalogue.ScopeMesh}},
Listens: []catalogue.Listening{{Port: 5432, From: catalogue.FromMesh}},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-store",
"ports": []any{"5432:5432"}, "image": "pg@" + aDigest}}})
register(t, open, catalogue.Manifest{Module: "lavinmq", Version: "1",
Claims: []catalogue.Claim{{Name: "mesh-broker", Scope: catalogue.ScopeMesh}},
Listens: []catalogue.Listening{{Port: 5671, From: catalogue.FromMesh},
{Port: 5672, From: catalogue.FromMesh}},
Guards: []int{15672},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-broker",
"ports": []any{"5671:5671", "5672:5672", "127.0.0.1:15672:15672"}, "image": "mq@" + aDigest}}})
// The control plane's own bus user is the installer's, seeded at genesis before the controller
// runs (SeedBusUser); without it a push now refuses the credential nobody issued (issue 203).
if err := open.inventory.SeedBusUser(ctx, inventory.BusUser{Username: "anchor.mesh-controller",
Kind: inventory.BusController, Node: "anchor", Module: "mesh-controller"}, "bootstrap"); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "mesh-controller"); err != nil {
t.Fatal(err)
}
// The store's module is registered and given its port, and NOT assigned: the state genesis
// leaves a given-port node in before the foundation is adopted as modules (04-ISSUES/085).
if err := open.inventory.SetSettings(ctx, "anchor", "postgres",
map[string]any{catalogue.PortsSetting: map[string]any{"5432": 6852}}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "lavinmq"); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetSettings(ctx, "anchor", "lavinmq",
map[string]any{catalogue.PortsSetting: map[string]any{"5672": 5679}}); err != nil {
t.Fatal(err)
}
var env map[string]any
for _, r := range composed(t, open, "anchor").Resources {
if r["id"] == "mesh-controller.controller" {
env, _ = r["env"].(map[string]any)
}
}
if env == nil {
t.Fatal("the control plane's process is not in its own node's declaration")
}
for key, want := range map[string]string{
"MESH_STORE_INVENTORY_PORT": "6852",
"MESH_STORE_IDENTITY_PORT": "6852",
"MESH_STORE_LICENCES_PORT": "6852",
"MESH_BROKER_AMQP_PORT": "5679",
// Neither given nor assigned by the mesh: the manifest's own number is NOT the answer,
// because the sealed value beside it carries the port genesis wrote (finding F3).
"MESH_BROKER_ADDRESS_PORT": "",
"MESH_BROKER_MANAGEMENT_PORT": "",
} {
if env[key] != want {
t.Errorf("the control plane is told %s=%v; the node says %q", key, env[key], want)
}
}
}
func aLoneNode(t *testing.T) *stores {
t.Helper()
inventory.ForTest(t)
licences.ForTest(t)
open, err := openStores(t.Context())
if err != nil {
t.Fatal(err)
}
t.Cleanup(open.Close)
for _, m := range provided {
if err := open.inventory.Provide(t.Context(), m); err != nil {
t.Fatal(err)
}
}
record, err := open.inventory.AddNode(t.Context(), "anchor")
if err != nil {
t.Fatal(err)
}
reported, _ := json.Marshal(map[string]any{"capabilities": []map[string]any{
{"name": "container-runtime", "present": true}}})
var profile map[string]any
_ = json.Unmarshal(reported, &profile)
if err := open.inventory.RecordProfile(t.Context(), record.ID, profile); err != nil {
t.Fatal(err)
}
if err := open.inventory.RecordSealingKey(t.Context(), record.ID, aPublicKey(t)); err != nil {
t.Fatal(err)
}
return open
}
func TestOnANodeWithNoNetworkTheStoreIsReachedByLoopback(t *testing.T) {
open := aLoneNode(t)
ctx := t.Context()
register(t, open, aStore())
register(t, open, catalogue.Manifest{Module: "builder", Version: "1",
Requires: []string{catalogue.ArtifactStoreProvision},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-builder",
"image": "registry.example/mesh-builder@" + aDigest}}})
if err := open.inventory.RecordBuild(ctx, inventory.Build{
ID: "b1", Repository: "r", Module: "postgres", Commit: "abc",
Made: []inventory.Artifact{{Name: "runtime", Kind: "image",
Reference: catalogue.ArtifactStoreScheme + "postgres/runtime@" + aDigest}},
}); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "postgres", Version: "1",
Resources: []map[string]any{{"id": "runtime", "type": "container", "name": "mesh-postgres",
"image": catalogue.ArtifactStoreScheme + "postgres/runtime@" + aDigest}}})
for _, module := range []string{"distribution", "builder", "postgres"} {
if _, err := assign(ctx, open, "anchor", module); err != nil {
t.Fatal(err)
}
}
if err := open.inventory.SetSettings(ctx, "anchor", "distribution",
map[string]any{catalogue.PortsSetting: map[string]any{"5000": 5100}}); err != nil {
t.Fatal(err)
}
var image any
for _, r := range composed(t, open, "anchor").Resources {
if r["id"] == "postgres.runtime" {
image = r["image"]
}
}
if image != "127.0.0.1:5100/postgres/runtime@"+aDigest {
t.Fatalf("on the store's own node, off any network, the image is fetched as %v", image)
}
held := heldBy(ctx)
if got := held["postgres/runtime"]; got != "127.0.0.1:5100/postgres/runtime@"+aDigest {
t.Fatalf("a builder beside the store is handed the base %q", got)
}
}
-213
View File
@@ -2,13 +2,10 @@ package main
import (
"encoding/json"
"os"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/licences"
"github.com/novox/mesh-controller/internal/link"
)
@@ -86,150 +83,12 @@ func aStore() catalogue.Manifest {
// **The trust a machine writes for the store, and the address every built image is fetched
// through, say the port the node gave the store** — not the catalogue's number.
func TestTheRegistryTrustAndEveryImageFollowThePortTheNodeGaveTheStore(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aStore())
if _, err := assign(ctx, open, "anchor", "distribution"); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetSettings(ctx, "anchor", "distribution",
map[string]any{catalogue.PortsSetting: map[string]any{"5000": 5101}}); err != nil {
t.Fatal(err)
}
// A module the mesh built, recorded by digest and path, running on the other machine.
if err := open.inventory.RecordBuild(ctx, inventory.Build{
ID: "b1", Repository: "r", Module: "app", Commit: "abc",
Made: []inventory.Artifact{{Name: "server", Kind: "image",
Reference: catalogue.ArtifactStoreScheme + "app/server@" + aDigest}},
}); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "app", Version: "1",
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-app",
"image": catalogue.ArtifactStoreScheme + "app/server@" + aDigest}}})
if _, err := assign(ctx, open, "laptop", "app"); err != nil {
t.Fatal(err)
}
// The runtime's trust is the runtime's module's to write (novox/hq ADR 0222): a stand-in for it
// asks where this machine reaches the store, as the docker module does.
register(t, open, catalogue.Manifest{Module: "runtime", Version: "1",
Resources: []map[string]any{{"id": "daemon", "type": "file", "path": "/etc/docker/daemon.json",
"into": "json", "content": `{"insecure-registries": ["${seat:mesh-artifact-store:reach}"]}` + "\n"}}})
if _, err := assign(ctx, open, "laptop", "runtime"); err != nil {
t.Fatal(err)
}
on := map[string]bool{"anchor": true, "laptop": true}
node, port, found, err := artifactStoreOnNetwork(ctx, open.inventory, on)
if err != nil || !found || node != "anchor" || port != "5101" {
t.Fatalf("the store is found on %q:%q (%v, %v); the node put it on 5101", node, port, found, err)
}
var trust string
for _, r := range composed(t, open, "laptop").Resources {
if r["path"] == "/etc/docker/daemon.json" {
trust, _ = r["content"].(string)
}
if r["id"] == "app.server" && r["image"] != "anchor.internal:5101/app/server@"+aDigest {
t.Errorf("the image the mesh built is fetched as %v", r["image"])
}
}
if !strings.Contains(trust, "anchor.internal:5101") || strings.Contains(trust, ":5000") {
t.Fatalf("the runtime is told to trust %q; the node put the store on 5101", trust)
}
// And a replay to the catalogue says where the store is now.
announced, err := following{open}.Announceable(ctx)
if err != nil {
t.Fatal(err)
}
if len(announced) != 1 || announced[0].Made[0].Reference != "anchor.internal:5101/app/server@"+aDigest {
t.Fatalf("the replay announces %+v", announced)
}
}
// **The control plane's own connections say the port the node gave the store and the broker.**
//
// Composed from the control plane's own manifest against a real inventory: the store's module is
// given 6852 on this node the way genesis or an operator gives it, and the control plane's
// process is told so beside the sealed connection genesis wrote.
func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
m, err := catalogue.ParseManifest(raw)
if err != nil {
t.Fatal(err)
}
control, err := withSeatPorts(m).Resolve([]catalogue.Built{{Name: "controller", Kind: catalogue.ArtifactBundle,
Reference: "https://registry.example/mesh-controller/controller.tar.gz", Digest: aDigest}})
if err != nil {
t.Fatal(err)
}
register(t, open, control)
register(t, open, catalogue.Manifest{Module: "postgres", Version: "1",
Claims: []catalogue.Claim{{Name: "mesh-store", Scope: catalogue.ScopeMesh}},
Listens: []catalogue.Listening{{Port: 5432, From: catalogue.FromMesh}},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-store",
"ports": []any{"5432:5432"}, "image": "pg@" + aDigest}}})
register(t, open, catalogue.Manifest{Module: "lavinmq", Version: "1",
Claims: []catalogue.Claim{{Name: "mesh-broker", Scope: catalogue.ScopeMesh}},
Listens: []catalogue.Listening{{Port: 5671, From: catalogue.FromMesh},
{Port: 5672, From: catalogue.FromMesh}},
Guards: []int{15672},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-broker",
"ports": []any{"5671:5671", "5672:5672", "127.0.0.1:15672:15672"}, "image": "mq@" + aDigest}}})
// The control plane's own bus user is the installer's, seeded at genesis before the controller
// runs (SeedBusUser); without it a push now refuses the credential nobody issued (issue 203).
if err := open.inventory.SeedBusUser(ctx, inventory.BusUser{Username: "anchor.mesh-controller",
Kind: inventory.BusController, Node: "anchor", Module: "mesh-controller"}, "bootstrap"); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "mesh-controller"); err != nil {
t.Fatal(err)
}
// The store's module is registered and given its port, and NOT assigned: the state genesis
// leaves a given-port node in before the foundation is adopted as modules (04-ISSUES/085).
if err := open.inventory.SetSettings(ctx, "anchor", "postgres",
map[string]any{catalogue.PortsSetting: map[string]any{"5432": 6852}}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "lavinmq"); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetSettings(ctx, "anchor", "lavinmq",
map[string]any{catalogue.PortsSetting: map[string]any{"5672": 5679}}); err != nil {
t.Fatal(err)
}
var env map[string]any
for _, r := range composed(t, open, "anchor").Resources {
if r["id"] == "mesh-controller.controller" {
env, _ = r["env"].(map[string]any)
}
}
if env == nil {
t.Fatal("the control plane's process is not in its own node's declaration")
}
for key, want := range map[string]string{
"MESH_STORE_INVENTORY_PORT": "6852",
"MESH_STORE_IDENTITY_PORT": "6852",
"MESH_STORE_LICENCES_PORT": "6852",
"MESH_BROKER_AMQP_PORT": "5679",
// Neither given nor assigned by the mesh: the manifest's own number is NOT the answer,
// because the sealed value beside it carries the port genesis wrote (finding F3).
"MESH_BROKER_ADDRESS_PORT": "",
"MESH_BROKER_MANAGEMENT_PORT": "",
} {
if env[key] != want {
t.Errorf("the control plane is told %s=%v; the node says %q", key, env[key], want)
}
}
}
// withSeatPorts is the control plane's manifest with the seat placeholders in its environment —
// added here until module.json carries them (the manifest lands one commit after the code that
@@ -274,79 +133,7 @@ func withSeatPorts(m catalogue.Manifest) catalogue.Manifest {
// aLoneNode is one capable machine with nothing placed on any network — the control-node during
// genesis, before the "network" step, which is after the store, the broker, the vault and the
// catalogue have each been built and pushed (finding F2).
func aLoneNode(t *testing.T) *stores {
t.Helper()
inventory.ForTest(t)
licences.ForTest(t)
open, err := openStores(t.Context())
if err != nil {
t.Fatal(err)
}
t.Cleanup(open.Close)
for _, m := range provided {
if err := open.inventory.Provide(t.Context(), m); err != nil {
t.Fatal(err)
}
}
record, err := open.inventory.AddNode(t.Context(), "anchor")
if err != nil {
t.Fatal(err)
}
reported, _ := json.Marshal(map[string]any{"capabilities": []map[string]any{
{"name": "container-runtime", "present": true}}})
var profile map[string]any
_ = json.Unmarshal(reported, &profile)
if err := open.inventory.RecordProfile(t.Context(), record.ID, profile); err != nil {
t.Fatal(err)
}
if err := open.inventory.RecordSealingKey(t.Context(), record.ID, aPublicKey(t)); err != nil {
t.Fatal(err)
}
return open
}
// **Before the network exists, the store's own node reaches it by loopback** — never refused,
// never handed the scheme: a genesis pushes the store, the broker, the vault and the catalogue to
// a node on no network, and builds the catalogue on a base it must be able to pull.
func TestOnANodeWithNoNetworkTheStoreIsReachedByLoopback(t *testing.T) {
open := aLoneNode(t)
ctx := t.Context()
register(t, open, aStore())
register(t, open, catalogue.Manifest{Module: "builder", Version: "1",
Requires: []string{catalogue.ArtifactStoreProvision},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-builder",
"image": "registry.example/mesh-builder@" + aDigest}}})
if err := open.inventory.RecordBuild(ctx, inventory.Build{
ID: "b1", Repository: "r", Module: "postgres", Commit: "abc",
Made: []inventory.Artifact{{Name: "runtime", Kind: "image",
Reference: catalogue.ArtifactStoreScheme + "postgres/runtime@" + aDigest}},
}); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "postgres", Version: "1",
Resources: []map[string]any{{"id": "runtime", "type": "container", "name": "mesh-postgres",
"image": catalogue.ArtifactStoreScheme + "postgres/runtime@" + aDigest}}})
for _, module := range []string{"distribution", "builder", "postgres"} {
if _, err := assign(ctx, open, "anchor", module); err != nil {
t.Fatal(err)
}
}
if err := open.inventory.SetSettings(ctx, "anchor", "distribution",
map[string]any{catalogue.PortsSetting: map[string]any{"5000": 5100}}); err != nil {
t.Fatal(err)
}
var image any
for _, r := range composed(t, open, "anchor").Resources {
if r["id"] == "postgres.runtime" {
image = r["image"]
}
}
if image != "127.0.0.1:5100/postgres/runtime@"+aDigest {
t.Fatalf("on the store's own node, off any network, the image is fetched as %v", image)
}
held := heldBy(ctx)
if got := held["postgres/runtime"]; got != "127.0.0.1:5100/postgres/runtime@"+aDigest {
t.Fatalf("a builder beside the store is handed the base %q", got)
}
}
+2
View File
@@ -1,3 +1,5 @@
//go:build integration
package main
import (
+21 -3
View File
@@ -408,6 +408,9 @@ func comparisonOf(module string, c comparison, opts takeOptions) (preview string
switch {
case sec.Origin == inventory.OriginAccepted:
fmt.Fprintf(&b, " %s %s: accepted from a person, carried in as it is\n", what, name)
case sec.Origin == inventory.OriginIssued:
// Taken from the provider's software (novox/hq ADR 0305): the value the running service uses.
fmt.Fprintf(&b, " %s %s: issued by the provider's software, carried in as it is\n", what, name)
case opts.Mint[sec.Name]:
fmt.Fprintf(&b, " %s %s: minted by the mesh; the service takes the new value, as --mint said\n", what, name)
case foundData:
@@ -421,8 +424,9 @@ func comparisonOf(module string, c comparison, opts takeOptions) (preview string
}
// And its settings on this machine, composed against its definition (rule 1, rule 6).
for _, layer := range c.layers {
keys := make([]string, 0, len(layer.Values))
for k := range layer.Values {
said := layer.Whole()
keys := make([]string, 0, len(said))
for k := range said {
keys = append(keys, k)
}
sort.Strings(keys)
@@ -719,7 +723,7 @@ func converge(ctx context.Context, open *stores, node string, yes bool, digest s
return "", err
}
derived := derivedFilter{rules: rules, foundation: with.Foundation, mesh: with.Mesh,
outward: plan.PublicDomain != "", outwardLinks: with.OutwardLinks}
outward: plan.PublicDomain != "", outwardLinks: with.OutwardLinks, local: with.LocalNetworks}
filtering, err := inv.FilteringOf(ctx, node)
if err != nil {
return "", err
@@ -927,6 +931,9 @@ type derivedFilter struct {
// The filter constrains what arrives on them; everything arriving elsewhere is this machine's
// own guest and is not filtered.
outwardLinks []string
// local is the machine's local networks, which a port from `local-network` stays open to (novox/hq
// ADR 0340).
local []string
}
// closesOutside is what a narrowing from everywhere to the private network is called: it closes.
@@ -974,6 +981,17 @@ func (d derivedFilter) fate(r inventory.Reach) string {
if !onMesh {
return fmt.Sprintf("%s — declared by %s from the mesh only", closesOutside, by)
}
case catalogue.FromLocalNetwork:
// The private network and this machine's local networks (novox/hq ADR 0340): it closes to
// everything else outside, and says which local networks stay.
if !onMesh {
if len(d.local) == 0 {
return fmt.Sprintf("%s — declared by %s from the local network, and this machine "+
"reported none", closesOutside, by)
}
return fmt.Sprintf("WILL CLOSE to everything outside the private network and %s — declared "+
"by %s from the local network", strings.Join(d.local, ", "), by)
}
}
return fmt.Sprintf("declared by %s (from %s)", by, rule.From)
}
@@ -0,0 +1,180 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// **A refusal the bus said once is counted once** (novox/hq issue 402). The bus refused the controller
// one publish at 18:52:59 UTC on 2026-10-10 and never again, yet S9 looked at the one remembered
// advisory every half minute for the hour it is kept, and the condition said "observed 15 time(s), last
// 13s ago" with a refusal in its evidence every 30 seconds: two of us read it as a refusal going on and
// went looking for a missing grant. The condition counts what the bus said, with when it last said it,
// never how often the controller looked.
func TestARefusalSaidOnceIsCountedOnceHoweverOftenItIsLookedAt(t *testing.T) {
refused := time.Date(2026, 10, 10, 18, 52, 59, 0, time.UTC)
now := refused.Add(10 * time.Second)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
// The words the live condition bus.controller.refused carried on 2026-10-10.
said := "the bus refused the controller: nats: permissions violation: Permissions Violation for " +
`Publish to "mesh.seat.mesh-delivery.tool.times"`
advisory := link.Advisory{Kind: link.AdvisoryRefused, ID: "controller", Said: said,
First: refused, Last: refused, Count: 1}
look := func() conditions.Condition {
t.Helper()
f := &signalFacts{now: now, host: "novox", advisories: []link.Advisory{advisory}}
if err := k.Reconcile(t.Context(), "S9", watchAdvisories(f)); err != nil {
t.Fatal(err)
}
c, found, err := conditions.ReadOne(t.Context(), store, "bus.controller.refused")
if err != nil || !found {
t.Fatalf("bus.controller.refused: found %v, %v", found, err)
}
return c
}
var c conditions.Condition
for range 15 {
c = look()
now = now.Add(30 * time.Second)
}
if c.Observations != 1 || len(c.Evidence) != 1 {
t.Fatalf("one refusal, looked at 15 times, reads as observed %d time(s) with %d evidence lines: %+v",
c.Observations, len(c.Evidence), c.Evidence)
}
if !c.LastObserved.Equal(refused) || !c.Evidence[0].At.Equal(refused) {
t.Fatalf("last observed %s, evidence at %s: the refusal was at %s", c.LastObserved, c.Evidence[0].At, refused)
}
// The bus refuses it three times more between two looks: the condition counts four refusals, says
// the newest, and adds one line of evidence for the look that saw them.
again := now.Add(-5 * time.Second)
advisory.Last, advisory.Count = again, 4
c = look()
if c.Observations != 4 || len(c.Evidence) != 2 || !c.LastObserved.Equal(again) || !c.Evidence[0].At.Equal(again) {
t.Fatalf("four refusals read as observed %d time(s), last %s, evidence %+v", c.Observations, c.LastObserved, c.Evidence)
}
if !strings.Contains(c.Summary, "(4 times since 18:52 UTC)") {
t.Fatalf("summary %q", c.Summary)
}
now = now.Add(30 * time.Second)
if c = look(); c.Observations != 4 || len(c.Evidence) != 2 {
t.Fatalf("a look with nothing new counted: observed %d time(s), evidence %+v", c.Observations, c.Evidence)
}
}
// **A condition that looks at the present still counts its looks** (novox/hq issue 402 changes only
// what reads a record): a machine silent for three looks was observed three times, and says so.
func TestAConditionOfThePresentCountsItsLooks(t *testing.T) {
now := time.Date(2026, 10, 10, 19, 0, 0, 0, time.UTC)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
var c conditions.Condition
for range 3 {
var err error
if c, err = k.Observe(t.Context(), conditions.Observation{Scope: conditions.ScopeMachine, ID: "ace",
Kind: "silent", Machine: "ace", Severity: conditions.Warning, Summary: "ace has not been heard from",
Source: "S1"}); err != nil {
t.Fatal(err)
}
now = now.Add(30 * time.Second)
}
if c.Observations != 3 || len(c.Evidence) != 3 || !c.LastObserved.Equal(now.Add(-30*time.Second)) {
t.Fatalf("observed %d time(s), %d evidence lines, last %s", c.Observations, len(c.Evidence), c.LastObserved)
}
}
// **One key, one watcher** (novox/hq issue 440). A consumer's max-deliveries condition is said from
// DEAD_LETTERS while it holds a message the consumer gave up on (issue 330). A max-deliveries advisory
// without a token names the same key; read beside it, each look added an observation and a line of
// evidence through the dead-letter half, and the two summaries overwrote each other on every look. The
// dead-letter row alone says that key, and counts the messages given up on, never the looks.
func TestAHeldDeadLetterIsCountedByWhatWasGivenUpNotByTheLooks(t *testing.T) {
gaveUp := time.Date(2026, 10, 10, 21, 4, 0, 0, time.UTC)
now := gaveUp.Add(20 * time.Second)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
const key = "bus.EVENTS.media_sonarr.max-deliveries"
held := map[string]int{"EVENTS.media_sonarr": 1}
newest := map[string]time.Time{"EVENTS.media_sonarr": gaveUp}
advisory := link.Advisory{Kind: link.AdvisoryMaxDeliveries, ID: "EVENTS.media_sonarr", Stream: "EVENTS",
Consumer: "media_sonarr", Said: "sonarr on media handed message 7 over 5 times and gave up on it",
First: gaveUp, Last: gaveUp, Count: 1}
look := func() conditions.Condition {
t.Helper()
f := &signalFacts{now: now, host: "novox", advisories: []link.Advisory{advisory},
deadLetters: held, deadLettersNewest: newest}
if err := k.Reconcile(t.Context(), "S9", watchAdvisories(f)); err != nil {
t.Fatal(err)
}
c, found, err := conditions.ReadOne(t.Context(), store, key)
if err != nil || !found {
t.Fatalf("%s: found %v, %v", key, found, err)
}
if !strings.Contains(c.Summary, "dead-letters verb") {
t.Fatalf("the summary is not the dead-letter row's: %q", c.Summary)
}
return c
}
var c conditions.Condition
for range 5 {
c = look()
now = now.Add(30 * time.Second)
}
if c.Observations != 1 || len(c.Evidence) != 1 || !c.LastObserved.Equal(gaveUp) {
t.Fatalf("one message given up on, looked at five times, reads as observed %d time(s), last %s, "+
"with %d evidence lines: %+v", c.Observations, c.LastObserved, len(c.Evidence), c.Evidence)
}
// The consumer gives up on a second message between two looks: two given up on, one more line.
again := now.Add(-10 * time.Second)
held["EVENTS.media_sonarr"], newest["EVENTS.media_sonarr"] = 2, again
c = look()
if c.Observations != 2 || len(c.Evidence) != 2 || !c.LastObserved.Equal(again) {
t.Fatalf("two given up on read as observed %d time(s), last %s, evidence %+v", c.Observations,
c.LastObserved, c.Evidence)
}
now = now.Add(30 * time.Second)
if c = look(); c.Observations != 2 || len(c.Evidence) != 2 {
t.Fatalf("a look with nothing new counted: observed %d time(s), evidence %+v", c.Observations, c.Evidence)
}
}
// The advisory still says the max-deliveries it alone knows: a message given up on that could not be
// kept, under a key of its own (issue 330), which issue 440's change leaves as it was.
func TestAMessageThatCouldNotBeKeptIsStillSaidFromTheAdvisory(t *testing.T) {
at := time.Date(2026, 10, 10, 21, 4, 0, 0, time.UTC)
f := &signalFacts{now: at, host: "novox", advisories: []link.Advisory{{Kind: link.AdvisoryMaxDeliveries,
ID: "EVENTS.media_sonarr", Stream: "EVENTS", Consumer: "media_sonarr", Token: link.AdvisoryNotKept,
Said: "sonarr on media gave up on message 7, and it could not be kept", First: at, Last: at, Count: 1}}}
said := watchAdvisories(f)
if len(said) != 1 || said[0].Key() != "bus.EVENTS.media_sonarr.not-kept" {
t.Fatalf("%+v", said)
}
}
// A max-deliveries advisory without a token names a consumer's dead-letter key, which only DEAD_LETTERS
// says (novox/hq issues 330 and 440): with nothing held, the advisory alone raises nothing.
func TestAMaxDeliveriesAdvisoryAloneRaisesNothing(t *testing.T) {
at := time.Date(2026, 10, 10, 21, 4, 0, 0, time.UTC)
f := &signalFacts{now: at, host: "novox", advisories: []link.Advisory{{Kind: link.AdvisoryMaxDeliveries,
ID: "EVENTS.media_sonarr", Stream: "EVENTS", Consumer: "media_sonarr",
Said: "sonarr on media handed message 7 over 5 times and gave up on it", First: at, Last: at, Count: 1}}}
if said := watchAdvisories(f); len(said) != 0 {
t.Fatalf("said %+v", said)
}
}
+11 -8
View File
@@ -48,7 +48,8 @@ const agentAccountProbe = "DA"
//
// The one judgement: `node show`, the self-check, and ADR 0259's router honouring a verified sender read
// it here.
func agentConfined(ctx context.Context, inv *inventory.Inventory, node string) (named, confined bool, why string, err error) {
// now is the judging clock, threaded so a caller judging several things at one instant judges them all at it.
func agentConfined(ctx context.Context, inv *inventory.Inventory, node string, now time.Time) (named, confined bool, why string, err error) {
n, err := inv.NodeByName(ctx, node)
if err != nil {
return false, false, "", err
@@ -61,7 +62,7 @@ func agentConfined(ctx context.Context, inv *inventory.Inventory, node string) (
if err != nil {
return true, false, "", err
}
confined, why = judgedConfined(n.AgentAccount, h, had, time.Now())
confined, why = judgedConfined(n.AgentAccount, h, had, now)
return true, confined, why, nil
}
@@ -118,9 +119,11 @@ func judgedConfined(agent string, h inventory.NodeHealth, had bool, now time.Tim
}
// searchQuietFor is how long the controller lets an agent account's verdict wait for the node-engine's setuid
// search before that is itself the urgent condition: the engine's bound on one search (link.RootSearchBound, the
// engine's own value), counted from when this controller first saw it waiting, never from the engine's start.
const searchQuietFor = link.RootSearchBound
// search before that is itself the urgent condition: the longest a search is expected to take (link.RootSearchQuiet,
// the engine's own value; novox/hq issue 361 — the search runs to completion, with no bound of its own), counted
// from when this controller first saw it waiting, never from the engine's start. Quiet raises nothing; it never
// makes the agent account confined, which only a healthy verdict from a complete, fresh search does.
const searchQuietFor = link.RootSearchQuiet
// The kinds of an agent account's verdict, for the quiet a search earns.
const (
@@ -206,10 +209,10 @@ func probeAgentAccounts(ctx context.Context, d *doctor) ([]conditions.Observatio
if confined {
continue
}
// Not judged yet only because the first search since the node-engine started is still running: not the
// Not judged yet only because the node-engine's search runs and no complete, fresh one judges: not the
// urgent condition after every restart. The agent is still not confined — ADR 0259's router reads
// agentConfined, not this — and `node show` still says not judged. Loud again once the search fails,
// runs out its bound, or the statement goes stale.
// waits past searchQuietFor, or the statement goes stale.
if quiet {
continue
}
@@ -228,7 +231,7 @@ func agentAccountLines(ctx context.Context, inv *inventory.Inventory, n inventor
return []string{fmt.Sprintf(" agents run as the operator account (%s); no agent account is named",
orNoneKnown(n.Account))}
}
_, confined, why, err := agentConfined(ctx, inv, n.Name)
_, confined, why, err := agentConfined(ctx, inv, n.Name, time.Now())
if err != nil {
return []string{fmt.Sprintf(" agents run as %s (home %s); whether it can become root could NOT be read: %v",
n.AgentAccount, n.AgentHome(), err)}
@@ -0,0 +1,161 @@
//go:build integration
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
snapshot "github.com/novox/mesh-controller/internal/facts"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-host/rootsearch"
)
func TestTheSelfCheckSaysAnAgentAccountThatCanBecomeRoot(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
for _, n := range []string{"anchor", "laptop"} {
if _, err := inv.NodeByName(ctx, n); err != nil {
if _, err := inv.AddNode(ctx, n); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAccount(ctx, n, "ops", ""); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAgentAccount(ctx, "anchor", "agent", ""); err != nil {
t.Fatal(err)
}
d := &doctor{open: open}
found, err := probeAgentAccounts(ctx, d)
if err != nil {
t.Fatal(err)
}
found = onlyMachine(found, "anchor")
if len(found) != 1 || found[0].Machine != "anchor" || found[0].Severity != conditions.Urgent ||
!strings.Contains(found[0].Said, "not judged") {
t.Fatalf("a named agent account with no verdict: %+v", found)
}
w := plainWordings[kindAgentCanBecomeRoot](conditions.Observation{Kind: kindAgentCanBecomeRoot, Machine: "anchor"})
if w.Headline == "" || w.Needs == "" || w.Resolved == "" {
t.Errorf("the condition has no plain words: %+v", w)
}
healthy := inventory.ResourceHealth{Module: "claude-code", Resource: "claude-code.agent-account",
Kind: link.KindAccount, Target: "agent", State: link.StateHealthy, Root: link.RootNever, Account: "agent"}
if _, err := inv.RecordHealth(ctx, inventory.NodeHealth{Node: "anchor", Contract: link.RootContract,
SaidAt: time.Now(), HeardAt: time.Now(), Resources: []inventory.ResourceHealth{healthy}}); err != nil {
t.Fatal(err)
}
if found, err = probeAgentAccounts(ctx, d); err != nil || len(onlyMachine(found, "anchor")) != 0 {
t.Fatalf("a judged agent account still fails: %+v %v", found, err)
}
if named, confined, why, err := agentConfined(ctx, inv, "anchor", time.Now()); err != nil || !named || !confined {
t.Fatalf("agentConfined on anchor: %v %v %q %v", named, confined, why, err)
}
if named, _, why, err := agentConfined(ctx, inv, "laptop", time.Now()); err != nil || named ||
!strings.Contains(why, "operator account") {
t.Fatalf("agentConfined on a machine naming none: %v %q %v", named, why, err)
}
}
func TestTheFactsCarryTheAgentAccountAsAPseudonym(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
if err := open.inventory.SetAccount(ctx, "anchor", "keeper", ""); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetAgentAccount(ctx, "anchor", "warden", ""); err != nil {
t.Fatal(err)
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
body, _ := f.Encode()
if strings.Contains(string(body), "warden") {
t.Error("the agent account's name is in the snapshot")
}
m, ok := f.Machine(snapshot.Pseudonym("machine", "anchor"))
if !ok || m.AgentAccount != snapshot.Pseudonym("account", "warden") || m.Account == m.AgentAccount {
t.Fatalf("the anchor's agent account reads as %q (operator %q)", m.AgentAccount, m.Account)
}
}
func TestASearchStillRunningAfterARestartIsNotUrgent(t *testing.T) {
if searchQuietFor != rootsearch.Quiet {
t.Fatalf("the quiet is %s and the node-engine's %s: they are one value", searchQuietFor, rootsearch.Quiet)
}
// A daily search that finishes within the quiet never leaves the account unjudged between two of them.
if rootsearch.FreshFor < rootsearch.Every+rootsearch.Quiet {
t.Fatalf("a complete search judges for %s, less than a day's search (%s) and the quiet (%s)",
rootsearch.FreshFor, rootsearch.Every, rootsearch.Quiet)
}
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if _, err := inv.NodeByName(ctx, "anchor"); err != nil {
if _, err := inv.AddNode(ctx, "anchor"); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAgentAccount(ctx, "anchor", "agent", ""); err != nil {
t.Fatal(err)
}
d := &doctor{open: open}
say := func(state, reason string) []conditions.Observation {
t.Helper()
// The engine's since is always now: it was just restarted.
v := inventory.ResourceHealth{Module: "claude-code", Resource: "claude-code.agent-account",
Kind: link.KindAccount, Target: "agent", State: state, Reason: reason, Root: link.RootNever,
Account: "agent", Since: time.Now()}
if _, err := inv.RecordHealth(ctx, inventory.NodeHealth{Node: "anchor", Contract: link.RootContract,
SaidAt: time.Now(), HeardAt: time.Now(), Resources: []inventory.ResourceHealth{v}}); err != nil {
t.Fatal(err)
}
found, err := probeAgentAccounts(ctx, d)
if err != nil {
t.Fatal(err)
}
return onlyMachine(found, "anchor")
}
running := link.ReasonRootPending + ": the search for setuid programs, started at 19:21, has not finished yet"
healthy := func() {
t.Helper()
if found := say(link.StateHealthy, ""); len(found) != 0 {
t.Fatalf("a healthy verdict raised: %+v", found)
}
}
if found := say(link.StateUnknown, running); len(found) != 0 {
t.Fatalf("a search first seen now was raised: %+v", found)
}
if _, confined, why, _ := agentConfined(ctx, inv, "anchor", time.Now()); confined || !strings.Contains(why, "not judged") {
t.Fatalf("an account whose search runs was read as confined: %q", why)
}
// The engine restarted again and again, each statement's own since fresh: the controller's clock runs on.
if _, err := inv.RootSearchPending(ctx, "anchor", time.Now().Add(-searchQuietFor-time.Minute)); err != nil {
t.Fatal(err)
}
healthy() // a complete verdict forgets when the waiting began …
if _, err := inv.RootSearchPending(ctx, "anchor", time.Now().Add(-searchQuietFor-time.Minute)); err != nil {
t.Fatal(err) // … and this restart loop began past the bound
}
if found := say(link.StateUnknown, running); len(found) != 1 || found[0].Severity != conditions.Urgent {
t.Fatalf("a search pending past the bound, by the controller's clock, was not urgent: %+v", found)
}
healthy()
incomplete := rootsearch.ReasonIncomplete + ": the search for setuid programs did not finish (last tried at " +
"19:23: timeout); it is tried again later"
if found := say(link.StateUnknown, incomplete); len(found) != 1 || found[0].Severity != conditions.Urgent {
t.Fatalf("a search that did not finish was not urgent: %+v", found)
}
if found := say(link.StateUnhealthy, link.ReasonRoot+": in the group docker; "+running); len(found) != 1 ||
found[0].Severity != conditions.Urgent {
t.Fatalf("a way to root found while the search runs was not urgent: %+v", found)
}
}
+6 -145
View File
@@ -1,14 +1,12 @@
package main
import (
"github.com/novox/mesh-host/rootsearch"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
snapshot "github.com/novox/mesh-controller/internal/facts"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -65,55 +63,6 @@ func TestAnAgentAccountIsConfinedOnlyOnAHealthyVerdictJudgedForRoot(t *testing.T
// DA raises an urgent condition, with plain words, on a machine whose agent account is not judged unable to
// become root; a machine that names none is not its to judge.
func TestTheSelfCheckSaysAnAgentAccountThatCanBecomeRoot(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
for _, n := range []string{"anchor", "laptop"} {
if _, err := inv.NodeByName(ctx, n); err != nil {
if _, err := inv.AddNode(ctx, n); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAccount(ctx, n, "ops", ""); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAgentAccount(ctx, "anchor", "agent", ""); err != nil {
t.Fatal(err)
}
d := &doctor{open: open}
found, err := probeAgentAccounts(ctx, d)
if err != nil {
t.Fatal(err)
}
found = onlyMachine(found, "anchor")
if len(found) != 1 || found[0].Machine != "anchor" || found[0].Severity != conditions.Urgent ||
!strings.Contains(found[0].Said, "not judged") {
t.Fatalf("a named agent account with no verdict: %+v", found)
}
w := plainWordings[kindAgentCanBecomeRoot](conditions.Observation{Kind: kindAgentCanBecomeRoot, Machine: "anchor"})
if w.Headline == "" || w.Needs == "" || w.Resolved == "" {
t.Errorf("the condition has no plain words: %+v", w)
}
healthy := inventory.ResourceHealth{Module: "claude-code", Resource: "claude-code.agent-account",
Kind: link.KindAccount, Target: "agent", State: link.StateHealthy, Root: link.RootNever, Account: "agent"}
if _, err := inv.RecordHealth(ctx, inventory.NodeHealth{Node: "anchor", Contract: link.RootContract,
SaidAt: time.Now(), HeardAt: time.Now(), Resources: []inventory.ResourceHealth{healthy}}); err != nil {
t.Fatal(err)
}
if found, err = probeAgentAccounts(ctx, d); err != nil || len(onlyMachine(found, "anchor")) != 0 {
t.Fatalf("a judged agent account still fails: %+v %v", found, err)
}
if named, confined, why, err := agentConfined(ctx, inv, "anchor"); err != nil || !named || !confined {
t.Fatalf("agentConfined on anchor: %v %v %q %v", named, confined, why, err)
}
if named, _, why, err := agentConfined(ctx, inv, "laptop"); err != nil || named ||
!strings.Contains(why, "operator account") {
t.Fatalf("agentConfined on a machine naming none: %v %q %v", named, why, err)
}
}
func TestTheAgentRootWordsArePlain(t *testing.T) {
w := plainWordings[kindAgentCanBecomeRoot](conditions.Observation{Kind: kindAgentCanBecomeRoot, Machine: "anchor"})
@@ -134,28 +83,6 @@ func onlyMachine(obs []conditions.Observation, machine string) []conditions.Obse
// The snapshot a merge check composes from carries the agent account as a pseudonym (novox/hq ADR 0266),
// so a change is judged against machines that name one, and the name never leaves.
func TestTheFactsCarryTheAgentAccountAsAPseudonym(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
if err := open.inventory.SetAccount(ctx, "anchor", "keeper", ""); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetAgentAccount(ctx, "anchor", "warden", ""); err != nil {
t.Fatal(err)
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
body, _ := f.Encode()
if strings.Contains(string(body), "warden") {
t.Error("the agent account's name is in the snapshot")
}
m, ok := f.Machine(snapshot.Pseudonym("machine", "anchor"))
if !ok || m.AgentAccount != snapshot.Pseudonym("account", "warden") || m.Account == m.AgentAccount {
t.Fatalf("the anchor's agent account reads as %q (operator %q)", m.AgentAccount, m.Account)
}
}
// No verb runs a `node` command that sets something: through the generic `command` verb, `node account`,
// `node agent-account` and every other `node` subcommand but list and show are refused, naming the terminal.
@@ -199,75 +126,9 @@ func TestTheNodeVerbOnlyShows(t *testing.T) {
}
}
// After every node-engine restart its search for setuid programs runs for up to its bound, and the agent account
// is not judged until it ends. DA does not raise that as urgent while the search is within its bound, counted from
// when this controller first saw it waiting — never from the engine's own "since", which a restart resets, so an
// engine restarted in a loop does not keep it quiet. The account is still not confined, and `node show` still
// says not judged; a search that failed, or a way to root found, is urgent at once.
func TestASearchStillRunningAfterARestartIsNotUrgent(t *testing.T) {
if searchQuietFor != rootsearch.Bound {
t.Fatalf("the quiet is %s and the node-engine's bound %s: they are one value", searchQuietFor, rootsearch.Bound)
}
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if _, err := inv.NodeByName(ctx, "anchor"); err != nil {
if _, err := inv.AddNode(ctx, "anchor"); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAgentAccount(ctx, "anchor", "agent", ""); err != nil {
t.Fatal(err)
}
d := &doctor{open: open}
say := func(state, reason string) []conditions.Observation {
t.Helper()
// The engine's since is always now: it was just restarted.
v := inventory.ResourceHealth{Module: "claude-code", Resource: "claude-code.agent-account",
Kind: link.KindAccount, Target: "agent", State: state, Reason: reason, Root: link.RootNever,
Account: "agent", Since: time.Now()}
if _, err := inv.RecordHealth(ctx, inventory.NodeHealth{Node: "anchor", Contract: link.RootContract,
SaidAt: time.Now(), HeardAt: time.Now(), Resources: []inventory.ResourceHealth{v}}); err != nil {
t.Fatal(err)
}
found, err := probeAgentAccounts(ctx, d)
if err != nil {
t.Fatal(err)
}
return onlyMachine(found, "anchor")
}
running := link.ReasonRootPending + ": the search for setuid programs, started at 19:21, has not finished yet"
healthy := func() {
t.Helper()
if found := say(link.StateHealthy, ""); len(found) != 0 {
t.Fatalf("a healthy verdict raised: %+v", found)
}
}
if found := say(link.StateUnknown, running); len(found) != 0 {
t.Fatalf("a search first seen now was raised: %+v", found)
}
if _, confined, why, _ := agentConfined(ctx, inv, "anchor"); confined || !strings.Contains(why, "not judged") {
t.Fatalf("an account whose search runs was read as confined: %q", why)
}
// The engine restarted again and again, each statement's own since fresh: the controller's clock runs on.
if _, err := inv.RootSearchPending(ctx, "anchor", time.Now().Add(-searchQuietFor-time.Minute)); err != nil {
t.Fatal(err)
}
healthy() // a complete verdict forgets when the waiting began …
if _, err := inv.RootSearchPending(ctx, "anchor", time.Now().Add(-searchQuietFor-time.Minute)); err != nil {
t.Fatal(err) // … and this restart loop began past the bound
}
if found := say(link.StateUnknown, running); len(found) != 1 || found[0].Severity != conditions.Urgent {
t.Fatalf("a search pending past the bound, by the controller's clock, was not urgent: %+v", found)
}
healthy()
incomplete := rootsearch.ReasonIncomplete + ": the search for setuid programs did not finish (last tried at " +
"19:23: timeout); it is tried again later"
if found := say(link.StateUnknown, incomplete); len(found) != 1 || found[0].Severity != conditions.Urgent {
t.Fatalf("a search that did not finish was not urgent: %+v", found)
}
if found := say(link.StateUnhealthy, link.ReasonRoot+": in the group docker; "+running); len(found) != 1 ||
found[0].Severity != conditions.Urgent {
t.Fatalf("a way to root found while the search runs was not urgent: %+v", found)
}
}
// Until a complete search for setuid programs judges — none since the node-engine's state was kept, or the last
// older than the engine lets one judge — the agent account is not judged, and the search runs to its end with no
// bound (novox/hq issue 361). DA does not raise that as urgent within searchQuietFor, counted from when this
// controller first saw it waiting — never from the engine's own "since", which a restart resets, so an engine
// restarted in a loop does not keep it quiet. The account is still not confined, and `node show` still says not
// judged; a search that failed, or a way to root found, is urgent at once.
+2
View File
@@ -1,3 +1,5 @@
//go:build integration
package main
import (
+164
View File
@@ -0,0 +1,164 @@
package main
import (
"fmt"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// kindApplyFailing is S21's kind: a machine whose declarations have been refused, or applied only in part,
// for longer than one apply interval (novox/hq issue 233).
const kindApplyFailing = "apply-failing"
// applyEvery is how often a node-engine applies what it was last told when nothing new arrives: its
// ReconcileEvery. A failure still there after one more of them is not one the next apply clears.
const applyEvery = 5 * time.Minute
// packageManagerKey is how a node-engine names a package manager that does not answer among what failed
// (mesh-host's probe.go): "the package manager (pacman)".
const packageManagerKey = "the package manager ("
// watchApplyFailing is S21: **a machine that does not apply what it was sent is said by the mesh, not only
// to someone who asks for `status`** (novox/hq issue 233, issue 187's family). A workstation refused every
// declaration for an hour and a half because its package manager did not answer, and the mesh's only word
// on it was a line in `status`. Raised once the machine's first report of this failure is older than one apply
// interval **while the machine is heard** — judged by time, not by how many reports said it: a node-engine
// says a re-apply only when how it went changed, so the same failure is one report however long it lasts.
// Names the refusal's or the failure's words; cleared by the clean apply the node-engine then says. Not while
// the machine is silent, asleep or not known: S1 says silent, and asleep is not lost.
func watchApplyFailing(f *signalFacts) []conditions.Observation {
heard := map[string]bool{}
for _, m := range f.machines {
if !m.asleep() && !m.lastHeard.IsZero() && f.now.Sub(m.lastHeard) <= heartbeatBound(m.every) {
heard[m.name] = true
}
}
var out []conditions.Observation
said := map[string]bool{}
for _, d := range f.doing {
if !d.Wrong() || d.Since == nil || f.now.Sub(*d.Since) <= applyEvery || !heard[d.Node] {
continue
}
out = append(out, applyFailing(d, f.now))
said[d.Node] = true
}
// **A module's files the provisioner failed to place are a failed apply too** (novox/hq ADR 0362 §3): its client's
// newest word, judged by the same bound — the client says a failure once, and fills again on every change —
// on a machine whose node-engine applied what it was sent.
failed := map[string][]inventory.ProvReport{}
for _, r := range f.provFailed {
if r.Status != inventory.ProvFailed || f.now.Sub(r.At) <= applyEvery || !heard[r.Node] || said[r.Node] {
continue
}
failed[r.Node] = append(failed[r.Node], r)
}
for _, node := range sortedNames(failed) {
out = append(out, provisioningFailing(node, failed[node], f.now))
}
return out
}
// provisioningFailing is one machine's S21 condition for the modules whose files its provisioner client failed to
// place.
func provisioningFailing(m string, failed []inventory.ProvReport, now time.Time) conditions.Observation {
sort.Slice(failed, func(a, b int) bool { return failed[a].Module < failed[b].Module })
var names, words []string
since := failed[0].At
for _, r := range failed {
names = append(names, r.Module)
words = append(words, r.Module+": "+oneLine(orNoReason(r.Reason)))
if r.At.Before(since) {
since = r.At
}
}
o := conditions.Observation{Scope: conditions.ScopeMachine, ID: m, Kind: kindApplyFailing, Machine: m,
Severity: conditions.Warning, Said: strings.Join(words, "; "),
Resolved: m + " applies what it is sent again"}
o.Summary = fmt.Sprintf("%s's provisioner has failed to place the files of %s since %s (bound %s)", m,
strings.Join(names, ", "), since.UTC().Format("2006-01-02 15:04 MST"), ago(applyEvery))
o.Headline = m + " is not applying its update"
o.Explanation = fmt.Sprintf("The provisioner on %s has not been able to write the files of %s for %s. The "+
"details say why.", m, strings.Join(names, ", "), humanDuration(now.Sub(since)))
return o
}
func orNoReason(s string) string {
if s == "" {
return "no reason given"
}
return s
}
// applyFailing is one machine's S21 condition.
func applyFailing(d inventory.Doing, now time.Time) conditions.Observation {
m := d.Node
since := d.Since.UTC().Format("2006-01-02 15:04 MST")
o := conditions.Observation{Scope: conditions.ScopeMachine, ID: m, Kind: kindApplyFailing, Machine: m,
Severity: conditions.Warning, Said: doingWords(d),
Resolved: m + " applies what it is sent again"}
how := "refused its declaration whole"
if d.Outcome != inventory.OutcomeRefused {
how = "applied its declaration only in part"
}
o.Summary = fmt.Sprintf("%s has %s since %s (bound %s): %s", m, how, since, ago(applyEvery), failedNames(d))
o.Headline = m + " is not applying its update"
o.Explanation = fmt.Sprintf("%s has %s for %s, and trying again has not helped. The details say what fails.",
m, plainHow(d), humanDuration(now.Sub(*d.Since)))
if tool := unansweringTool(d); tool != "" {
o.Summary = fmt.Sprintf("%s: %s does not answer, so no package reaches it; everything else it was sent "+
"is applied, since %s (bound %s)", m, tool, since, ago(applyEvery))
o.Headline = fmt.Sprintf("%s cannot install packages", m)
o.Explanation = fmt.Sprintf("The package manager on %s (%s) has not answered for %s, so no package reaches "+
"it. Everything else is still applied, so an update that repairs it can land; if none does, it needs a "+
"look on the machine.", m, tool, humanDuration(now.Sub(*d.Since)))
}
return o
}
// plainHow is how a machine failed, as the operator reads it.
func plainHow(d inventory.Doing) string {
if d.Outcome == inventory.OutcomeRefused {
return "refused every update"
}
return "applied its updates only in part"
}
// unansweringTool is the package manager a machine said does not answer, or "".
func unansweringTool(d inventory.Doing) string {
for _, r := range d.Failed {
if rest, ok := strings.CutPrefix(r.ID, packageManagerKey); ok {
return strings.TrimSuffix(rest, ")")
}
}
return ""
}
// failedNames is what failed, by name: the refusal, or the failed resources' ids.
func failedNames(d inventory.Doing) string {
if d.Outcome == inventory.OutcomeRefused {
return "refused"
}
ids := make([]string, 0, len(d.Failed))
for _, r := range d.Failed {
ids = append(ids, r.ID)
}
sort.Strings(ids)
return "failed " + strings.Join(ids, ", ")
}
// doingWords is the machine's own words on what failed: the evidence, kept inside the mesh.
func doingWords(d inventory.Doing) string {
if d.Outcome == inventory.OutcomeRefused {
return "refused: " + oneLine(d.Refused)
}
var parts []string
for _, r := range d.Failed {
parts = append(parts, r.ID+": "+oneLine(r.Error))
}
sort.Strings(parts)
return strings.Join(parts, "; ")
}
+123
View File
@@ -0,0 +1,123 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// unansweringPackageManager is anchor's last report as the workstation's read on 2026-10-04, once its
// node-engine applies the rest (novox/hq issue 233): the package failed and the package manager did not answer,
// the same failure on times reports since for before now.
func unansweringPackageManager(now time.Time, forHowLong time.Duration, times int) inventory.Doing {
since := now.Add(-forHowLong)
return inventory.Doing{Node: "anchor", Outcome: inventory.OutcomeFailed, At: now.Add(-time.Second),
Since: &since, Times: times, Applied: 41, Failed: []inventory.FailedResource{
{ID: "apply", Error: `applying "pacman.base": nothing can be said about "base": this is the arch host and ` +
"pacman does not answer here. Either this machine is not Arch, or its package database is broken: " +
"pacman exited 1: error: config file /etc/pacman.conf could not be read: No such file or directory"},
{ID: "the package manager (pacman)", Error: "pacman does not answer here, so every package in this " +
"declaration fails and the rest of it is applied (novox/hq issue 233): pacman exited 1: error: config " +
"file /etc/pacman.conf could not be read: No such file or directory"}}}
}
// **A machine whose package manager does not answer is raised by the mesh itself, naming the tool and the
// probe's words** (novox/hq issue 233): the words in the evidence, the path kept out of the summary.
func TestAPackageManagerThatDoesNotAnswerIsRaisedNamingItsWords(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
f := calm(now)
f.doing = []inventory.Doing{unansweringPackageManager(now, 90*time.Minute, 18)}
got := watchApplyFailing(f)
if len(got) != 1 {
t.Fatalf("raised %d conditions, want one: %+v", len(got), got)
}
o := got[0]
if o.Key() != "machine.anchor.apply-failing" || o.Machine != "anchor" {
t.Errorf("the condition is not the machine's: %s", o.Key())
}
if !strings.Contains(o.Summary, "pacman does not answer") || !strings.Contains(o.Summary, "since 2026-10-04 12:30") {
t.Errorf("the summary does not name the tool and since when: %s", o.Summary)
}
if strings.Contains(o.Summary, "/etc/") {
t.Errorf("the summary carries a path, which belongs in the evidence: %s", o.Summary)
}
if !strings.Contains(o.Said, "could not be read") {
t.Errorf("the evidence does not carry the probe's words: %s", o.Said)
}
if !strings.Contains(o.Headline, "cannot install packages") || o.Explanation == "" || o.Resolved == "" {
t.Errorf("the plain words do not say it: %+v", o)
}
}
// **A whole refusal is raised too**, with the node-engine's words: what the workstation's report said before the
// node-engine applied the rest.
func TestARefusalThatGoesOnIsRaised(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
since := now.Add(-time.Hour)
f := calm(now)
f.doing = []inventory.Doing{{Node: "anchor", Outcome: inventory.OutcomeRefused, Since: &since, Times: 12,
Refused: "this is the arch host and pacman does not answer here", At: now.Add(-time.Minute)}}
got := watchApplyFailing(f)
if len(got) != 1 || !strings.Contains(got[0].Summary, "refused its declaration whole") ||
!strings.Contains(got[0].Said, "pacman does not answer") {
t.Fatalf("a refusal that went on for an hour was not raised with its words: %+v", got)
}
}
// **Judged by time while the machine is heard, not by how many reports said it** (review of novox/hq issue 233):
// a node-engine says the same failure once, so one report an apply interval old is a fault that goes on; one
// younger is not yet; and a silent or sleeping machine is S1's.
func TestOneReportOlderThanAnApplyIsRaisedAndAQuietMachineIsNot(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
f := calm(now)
f.doing = []inventory.Doing{unansweringPackageManager(now, 6*time.Minute, 1)}
if got := watchApplyFailing(f); len(got) != 1 {
t.Errorf("one report of a failure six minutes old was not raised: %+v", got)
}
f.doing = []inventory.Doing{unansweringPackageManager(now, 4*time.Minute, 1)}
if got := watchApplyFailing(f); len(got) != 0 {
t.Errorf("a failure four minutes old was raised before one apply could clear it: %+v", got)
}
f.doing = []inventory.Doing{unansweringPackageManager(now, time.Hour, 1)}
f.machines[0].lastHeard = now.Add(-time.Hour)
if got := watchApplyFailing(f); len(got) != 0 {
t.Errorf("a silent machine was raised for its last report: %+v", got)
}
f = calm(now)
f.doing = []inventory.Doing{unansweringPackageManager(now, time.Hour, 1)}
f.doing[0].Node = "a-machine-the-mesh-does-not-know"
if got := watchApplyFailing(f); len(got) != 0 {
t.Errorf("a machine not heard of was raised: %+v", got)
}
}
// **A failed re-apply followed by a clean one clears S21** (review of novox/hq issue 233): the node-engine says
// the clean re-apply, the store keeps it as applied with no failure since, and the condition the failure raised
// is cleared through the keeper on the next tick.
func TestACleanReapplyAfterAFailedOneClearsTheCondition(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
defer k.Close(context.Background())
w := &watchdogs{keeper: k, started: now.Add(-time.Hour)}
failing := calm(now)
failing.doing = []inventory.Doing{unansweringPackageManager(now, 20*time.Minute, 1)}
w.see(t.Context(), failing)
open, err := k.Open(t.Context())
if err != nil || len(open) != 1 || open[0].Kind != kindApplyFailing {
t.Fatalf("the failed re-apply raised %+v (%v), want apply-failing", open, err)
}
clean := calm(now.Add(5 * time.Minute))
clean.doing = nil // the clean re-apply recorded: NotDoingWhatTheyWereTold no longer lists the machine
w.see(t.Context(), clean)
if open, _ := k.Open(t.Context()); len(open) != 0 {
t.Fatalf("the condition stayed open after the clean re-apply: %+v", open)
}
}
+946
View File
@@ -0,0 +1,946 @@
package main
// The controller asks, and acts on the operator's warrant (novox/hq ADR 0259 §6). It holds no channel, no
// identity and no factor: it asks the router like any other module, and performs the answer chosen with its
// own grant.
//
// - **For every open, unsilenced condition that needs the operator and names its answers**, one ask is
// published on the `operator-channel` seat under the controller's own name: the condition's words, its
// actions as options at their levels (Silence acknowledges; Release, Stop, Start and Restart approve),
// answered by the operator, expiring after a day (a week when every option only acknowledges). A
// condition that clears, is silenced, or changes its answers has its ask cancelled; an ask that expired
// unanswered is asked again while the condition lasts; one the router refused is asked again after a wait
// that grows with each refusal in a row, at most half an hour, so a refusal is never believed longer. Each ask is kept in the controller's bucket
// `asked`, so a restart neither asks twice nor forgets.
// - **On a warrant**, heard on the seat's event under the controller's own name (which only the router may
// say), the controller acts once per ask: only for an ask it holds, only for the option it offered at
// that option's level, and only while the condition is still open. It performs the action as itself —
// a silence through its own conditions, any other through the verb the action names — with the warrant's
// words as its why, and records it in the hand-act log as the operator's decision, naming the channel,
// the ask and the proofs. An ask that ended without a choice is recorded and nothing is done.
// - **A warrant it missed** while away is read from the router's record of its asks, under its own name.
import (
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// The asker's name on the seat: the controller's module.
const askerName = broker.ControllerSeat
// How long an ask lasts: a day when an answer approves, a week when every answer only acknowledges.
const (
// askApproveFor is a day less a margin, so an ask is never refused at the router for lasting a day and
// a moment (the SDK's bound is a day).
askApproveFor = 24*time.Hour - 10*time.Minute
askAcknowledgeFor = 7 * 24 * time.Hour
// askEvery is how often what is open is asked about again, beside every change.
askEvery = time.Minute
// askCatchUpAfter is how old an open ask is before the router's record of it is read: a warrant heard
// on the event needs no reading.
askCatchUpAfter = 2 * time.Minute
// askAgainAfterAnswer is how long a condition the operator answered is not asked about again with the
// same answers: what was chosen takes a while to clear it, and asking again at once would ask twice.
askAgainAfterAnswer = time.Hour
// askMostOpen is how many asks the controller holds open at once (the router refuses a fourth): the
// most urgent conditions first, then the oldest.
askMostOpen = asks.MostOpen
// askRefusedRetryMost is the longest a refusal is believed without asking again (novox/hq issue 369): the
// router refused while its channels had not yet said they could send, the channels could send twenty
// minutes later, and the controller repeated that refusal for eleven hours. A refused ask is asked again
// after askEvery, then twice as long after each refusal in a row, never longer than this — and at once when
// the channels change.
askRefusedRetryMost = 30 * time.Minute
// askRefusedCounted is how far back refusals in a row are counted for that wait.
askRefusedCounted = 6 * time.Hour
// askVerdictWait is how long an ask made again after a refusal waits for the router's word before it is
// taken as taken: the router refuses an ask as it reads it, so a refusal comes within seconds. Meanwhile
// the condition saying it was not delivered stands as it was, neither cleared nor raised again.
askVerdictWait = 2 * time.Minute
)
// refusedRetryAfter is how long a part refused n times in a row waits before it is asked again.
func refusedRetryAfter(n int) time.Duration {
wait := askEvery
for i := 1; i < n && wait < askRefusedRetryMost; i++ {
wait *= 2
}
return min(wait, askRefusedRetryMost)
}
// What became of an ask, as the controller keeps it.
const (
askOpen = "open"
askCancelled = "cancelled"
)
// asked is one ask the controller made, as it keeps it.
type asked struct {
ID string `json:"id"`
Condition string `json:"condition"`
// Channels is what the channels were when it was asked (asker.channels): an ask the router refused is asked
// again at once when the condition's answers or the channels change, and otherwise after a wait that grows
// with each refusal in a row (refusedRetryAfter, novox/hq issue 369).
Channels string `json:"channels,omitempty"`
Ask asks.Ask `json:"ask"`
Actions []conditions.Action `json:"actions"`
// Options are the actions by option id.
Options map[string]int `json:"options"`
State string `json:"state"`
Opened time.Time `json:"opened"`
Ended time.Time `json:"ended,omitempty"`
Warrant *asks.Warrant `json:"warrant,omitempty"`
// Acted is what the controller did on the warrant: empty before it did anything, "acting" while it acts,
// then "done", "failed: …" or "nothing: …". Anything but empty is never acted on again.
Acted string `json:"acted,omitempty"`
// Part is which ask of its condition this is (askPart): empty for the one that carries the condition's
// answers, or the authorising ones where it has both; "acknowledge" for its acknowledging answers asked
// apart (the review of 2026-10-09, M1).
Part string `json:"part,omitempty"`
// Rehearsal is an ask started at the controller's terminal (rehearse.go): about no condition, its answers
// perform nothing, and the reconciling of conditions leaves it alone.
Rehearsal bool `json:"rehearsal,omitempty"`
// Proposal is a settings layer proposed through a verb (proposals.go, novox/hq ADR 0277): about no
// condition, set on Approve by the serving controller itself, and left alone by the reconciling of conditions.
Proposal *settingsProposal `json:"proposal,omitempty"`
}
// ofACondition says an ask is one of a condition's: not a rehearsal, not a proposal.
func (r asked) ofACondition() bool { return !r.Rehearsal && r.Proposal == nil }
// partKey is an ask's place among what is asked: its condition and its part.
func partKey(condition, part string) string { return condition + "#" + part }
// partAcknowledge is the part of a condition asked apart for its acknowledging answers.
const partAcknowledge = "acknowledge"
// askPart is one ask a condition is asked with: its part, what it is about, and its answers.
type askPart struct {
name string
about string
actions []conditions.Action
}
// levelOf is an action's level as an option offers it: one that says none is never taken for less than
// approve.
func levelOf(act conditions.Action) asks.Level {
if act.Level == "" {
return asks.Approve
}
return asks.Level(act.Level)
}
// partsOf is the asks a condition is asked with (the review of 2026-10-09, M1): one, when its answers are all
// of one kind; else its authorising answers (Release, Stop, Restart) in one ask, about the condition, and its
// acknowledging ones (Silence) in another. **An acknowledgement never shares an ask with an approval**: a
// channel that only acknowledges would otherwise answer the ask, and end the approval with it.
func partsOf(c conditions.Condition) []askPart {
var ack, auth []conditions.Action
for _, act := range c.Actions {
if levelOf(act) == asks.Acknowledge {
ack = append(ack, act)
} else {
auth = append(auth, act)
}
}
if len(ack) == 0 || len(auth) == 0 {
return []askPart{{about: c.Key, actions: c.Actions}}
}
return []askPart{{about: c.Key, actions: auth},
{name: partAcknowledge, about: c.Key + "." + partAcknowledge, actions: ack}}
}
// askedStore keeps the asks (broker.AskedBucket). **Every write after the first is a compare-and-set** (the
// review of 2026-10-09, L2): an ask is created once, and changed only over the revision it was read at, the
// change decided again on what is read — so two controllers, or two deliveries of one warrant, never write
// over each other, and of two that would act only the one whose write stands does.
type askedStore interface {
Get(ctx context.Context, id string) (*asked, error)
// Create keeps a new ask, and refuses one already kept under its id.
Create(ctx context.Context, a asked) error
// Change applies change to the ask kept under id, by compare-and-set, and says whether its write stood.
// change says whether to write at all; on a write that came between, it is asked again on what is read.
Change(ctx context.Context, id string, change func(*asked) bool) (bool, error)
All(ctx context.Context) ([]asked, error)
}
// askChangeTries is how often a change is read and tried again when another write came between.
const askChangeTries = 5
// asker is the controller asking the operator and acting on the answer.
type asker struct {
open func(ctx context.Context) ([]conditions.Condition, error)
silence func(ctx context.Context, key string, d time.Duration, by, why string) error
store askedStore
// publish puts a message on a subject's stream, de-duplicated by id.
publish func(ctx context.Context, subject string, body []byte, id string) error
// call performs an action's verb with its arguments, as the controller.
call func(ctx context.Context, a conditions.Action, args map[string]string) error
// setLayer sets a proposed settings layer on the operator's warrant (novox/hq ADR 0277); nil cannot.
setLayer setOnWarrant
// record writes the hand-act log.
record func(ctx context.Context, act link.HandAct) error
// routerRecord reads the router's record of an ask for a warrant missed; nil reads nothing.
routerRecord func(ctx context.Context, id string) (*asks.Warrant, error)
// routerHere says whether a router holds the seat and takes asks under the asker's name; nil is yes.
routerHere func(ctx context.Context) (bool, error)
// grantHeld says whether the bus holds the controller's grant to ask: the user list the bus's machine was
// last sent is the one the mesh composes now (novox/hq issue 353). The grant is composed from the router's
// assignment and reaches the bus only when that machine is next pushed, so between `assign` and `push`
// the record says a router is here and the bus refuses every ask. why says what to do; nil is yes.
grantHeld func(ctx context.Context) (held bool, why string, err error)
// channels is what the channels are now, as a fingerprint: who holds which kind, promising what.
channels func(ctx context.Context) string
// raise keeps the asker's own condition (sourceAsker): which conditions needing the operator could not be
// asked, and why. Nil raises nothing (a test that does not look).
raise func(ctx context.Context, obs []conditions.Observation) error
now func() time.Time
logf func(string, ...any)
saidNoRouter bool
saidNoGrant bool
// saidGrantCaveat is whether a grant held with a caveat was said, so it is said once.
saidGrantCaveat bool
mu sync.Mutex
nudged chan struct{}
}
func (a *asker) nudge() {
if a == nil {
return
}
a.mu.Lock()
if a.nudged == nil {
a.nudged = make(chan struct{}, 1)
}
ch := a.nudged
a.mu.Unlock()
select {
case ch <- struct{}{}:
default:
}
}
// keep asks until ctx ends: now, on every change of a condition, and every askEvery.
func (a *asker) keep(ctx context.Context) {
a.nudge()
tick := time.NewTicker(askEvery)
defer tick.Stop()
a.mu.Lock()
nudged := a.nudged
a.mu.Unlock()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
case <-nudged:
}
if err := a.reconcile(ctx); err != nil {
a.logf("what the operator is asked could not be brought up to date: %v", err)
}
}
}
// wants says whether a condition is one to ask about now.
func wants(c conditions.Condition, now time.Time) bool {
return len(c.Actions) > 0 && c.Needs != "" && !c.SilencedAt(now)
}
func sameAsked(a []conditions.Action, b []conditions.Action) bool {
x, _ := json.Marshal(a)
y, _ := json.Marshal(b)
return string(x) == string(y)
}
// reconcile brings what is asked in line with what is open.
func (a *asker) reconcile(ctx context.Context) error {
now := a.now()
if a.routerHere != nil {
here, err := a.routerHere(ctx)
if err != nil {
return err
}
if !here {
if !a.saidNoRouter {
a.logf("no router takes asks under the controller's name (a module declaring %s with its ask "+
"named by its caller, assigned): the operator is asked nothing until one is", broker.AsksSeat)
a.saidNoRouter = true
}
open, err := a.open(ctx)
if err != nil {
return err
}
var unasked []conditions.Condition
for _, c := range open {
if wants(c, now) {
unasked = append(unasked, c)
}
}
return a.sayUnasked(ctx, unasked, "no router takes the controller's asks: no module holding "+
broker.AsksSeat+" that takes an ask under its asker's name is assigned")
}
a.saidNoRouter = false
}
if a.grantHeld != nil {
held, why, err := a.grantHeld(ctx)
if err != nil {
return err
}
if !held {
if !a.saidNoGrant {
a.logf("the bus does not hold the controller's grant to ask yet: %s; the operator is asked nothing until it does", why)
a.saidNoGrant = true
}
open, err := a.open(ctx)
if err != nil {
return err
}
var unasked []conditions.Condition
for _, c := range open {
if wants(c, now) {
unasked = append(unasked, c)
}
}
return a.sayUnasked(ctx, unasked, "the bus does not hold the controller's grant to ask yet: "+why)
}
if why != "" && !a.saidGrantCaveat {
// Held, with a caveat worth one line (an older bus module cannot confirm the reload).
a.logf("the controller asks: %s", why)
a.saidGrantCaveat = true
} else if why == "" {
a.saidGrantCaveat = false
}
a.saidNoGrant = false
}
channels := ""
if a.channels != nil {
channels = a.channels(ctx)
}
open, err := a.open(ctx)
if err != nil {
return err
}
all, err := a.store.All(ctx)
if err != nil {
return err
}
byCondition := map[string]asked{} // by partKey
// What is open and not a condition's — a rehearsal, a proposal — still counts toward what the router holds
// open for the controller (askMostOpen); expired unanswered, it is kept so, as the router says it too.
otherOpen := 0
for _, r := range all {
if r.State == askOpen && !r.ofACondition() && !now.Before(r.Ask.Expires) {
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.State != askOpen || x.Acted != "" {
return false
}
x.State, x.Ended, x.Acted = string(asks.OutcomeExpired), now, "nothing: the ask expired unanswered"
return true
}); err != nil {
return err
}
continue
}
if r.State == askOpen && !r.ofACondition() {
otherOpen++
}
if r.State == askOpen && r.ofACondition() {
k := partKey(r.Condition, r.Part)
if prior, held := byCondition[k]; !held || r.Opened.After(prior.Opened) {
byCondition[k] = r
}
}
}
// A warrant missed while away, read from the router's record: for a condition's ask, and for a proposal's or a
// rehearsal's alike.
if a.routerRecord != nil {
var lookedUp []asked
for _, r := range byCondition {
lookedUp = append(lookedUp, r)
}
for _, r := range all {
if r.State == askOpen && !r.ofACondition() {
lookedUp = append(lookedUp, r)
}
}
for _, r := range lookedUp {
if now.Sub(r.Opened) < askCatchUpAfter {
continue
}
if w, err := a.routerRecord(ctx, r.ID); err == nil && w != nil {
body, _ := json.Marshal(w)
if err := a.Decided(ctx, body); err != nil {
return err
}
}
}
if all, err = a.store.All(ctx); err != nil {
return err
}
byCondition = map[string]asked{}
for _, r := range all {
if r.State == askOpen && r.ofACondition() {
byCondition[partKey(r.Condition, r.Part)] = r
}
}
}
// What the operator answered lately, by condition: not asked again at once; and what the router refused,
// newest first: not asked again until the answers or the channels change.
answered, refused := map[string]asked{}, map[string]asked{}
for _, r := range all {
k := partKey(r.Condition, r.Part)
if r.State == string(asks.OutcomeChosen) && now.Sub(r.Ended) < askAgainAfterAnswer {
answered[k] = r
}
if r.State == string(asks.OutcomeRefused) {
if prior, has := refused[k]; !has || r.Opened.After(prior.Opened) {
refused[k] = r
}
}
}
// Refusals in a row, by part: those since the last ask that was not refused, within askRefusedCounted.
// superseded: a part asked since its last refusal, by an ask the router did not refuse — open, answered,
// expired or cancelled. Its refusal is history then, never said again (the review of PR 198: an answered
// retry brought the refusal back).
inRow, superseded := map[string]int{}, map[string]asked{}
for k, last := range refused {
for _, r := range all {
if partKey(r.Condition, r.Part) == k && r.State != string(asks.OutcomeRefused) && r.State != askUnsent &&
r.Opened.After(last.Opened) && r.Opened.After(superseded[k].Opened) {
superseded[k] = r
}
}
var since time.Time
for _, r := range all {
if partKey(r.Condition, r.Part) == k && r.State != string(asks.OutcomeRefused) && r.State != askUnsent &&
!r.Opened.After(last.Opened) && r.Opened.After(since) {
since = r.Opened
}
}
for _, r := range all {
if partKey(r.Condition, r.Part) == k && r.State == string(asks.OutcomeRefused) && r.Opened.After(since) &&
now.Sub(r.Opened) < askRefusedCounted {
inRow[k]++
}
}
}
wanted := map[string]bool{}
var unasked []conditions.Condition // refused by the router, and nothing it was refused for changed
var refusedWords []string
// The most urgent first, then the oldest: those are asked when no more than askMostOpen may be.
sort.SliceStable(open, func(i, j int) bool {
ui, uj := open[i].Severity == conditions.Urgent, open[j].Severity == conditions.Urgent
if ui != uj {
return ui
}
if !open[i].Raised.Equal(open[j].Raised) {
return open[i].Raised.Before(open[j].Raised)
}
return open[i].Key < open[j].Key
})
openNow := otherOpen
for _, c := range open {
if !wants(c, now) {
continue
}
for _, p := range partsOf(c) {
if r, held := byCondition[partKey(c.Key, p.name)]; held && sameAsked(r.Actions, p.actions) && now.Before(r.Ask.Expires) {
openNow++
}
}
}
for _, c := range open {
if !wants(c, now) {
continue
}
saidUnasked := false
for _, p := range partsOf(c) {
key := partKey(c.Key, p.name)
wanted[key] = true
if r, was := refused[key]; was && sameAsked(r.Actions, p.actions) {
cur, held := byCondition[key]
ended := r.Ended
if ended.IsZero() {
ended = r.Opened
}
later, asked := superseded[key]
waiting := !held && !asked && r.Channels == channels && now.Sub(ended) < refusedRetryAfter(inRow[key])
// Asked again now (the wait over), or lately and the router's word not in yet: said as it was until
// that word, so the condition neither clears nor is raised again at each try.
retrying := !held && !asked && !waiting
verdictDue := held && asked && later.ID == cur.ID && now.Sub(cur.Opened) < askVerdictWait
if waiting || retrying || verdictDue {
if !saidUnasked {
unasked, saidUnasked = append(unasked, c), true
}
if r.Warrant != nil && r.Warrant.Words != "" {
refusedWords = append(refusedWords, r.Warrant.Words)
}
}
if waiting {
continue // refused lately, and nothing it was refused for has changed: asked again after the wait
}
}
if r, done := answered[key]; done && sameAsked(r.Actions, p.actions) {
if _, held := byCondition[key]; !held {
continue
}
}
if r, held := byCondition[key]; held {
switch {
case !sameAsked(r.Actions, p.actions):
if err := a.cancel(ctx, r, "its answers changed"); err != nil {
return err
}
case !now.Before(r.Ask.Expires):
// Expired unanswered: the router says so too; asked again below while it lasts.
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.State != askOpen {
return false
}
x.State, x.Ended = string(asks.OutcomeExpired), now
return true
}); err != nil {
return err
}
openNow--
default:
continue
}
}
if openNow >= askMostOpen {
continue // asked when one of the open ones ends, most urgent first
}
if err := a.ask(ctx, c, p, channels); err != nil {
a.logf("the operator could not be asked about %s: %v", c.Key, err)
continue
}
openNow++
}
}
stillOpen := map[string]conditions.Condition{}
for _, c := range open {
stillOpen[c.Key] = c
}
for key, r := range byCondition {
if wanted[key] {
continue
}
// **A silence never takes an approval back** (the confirmation review of 2026-10-09, M1). Silence is an
// acknowledgement — anyone at the desk may give it — so a condition silenced while its approval is asked
// keeps that ask open, unchanged, until it is answered on a channel that proves who answered, or expires.
// It is not asked again once it ends, while the silence lasts.
if c, open := stillOpen[r.Condition]; open && c.SilencedAt(now) && r.Ask.Highest() != asks.Acknowledge &&
now.Before(r.Ask.Expires) && keepsItsAnswers(c, r) {
continue
}
if err := a.cancel(ctx, r, "the condition ended, was silenced or needs nothing now"); err != nil {
return err
}
}
why := "the router refused the ask"
if len(refusedWords) > 0 {
why += ": " + refusedWords[0]
}
return a.sayUnasked(ctx, unasked, why)
}
// keepsItsAnswers says a condition still offers the answers an ask kept was asked with.
func keepsItsAnswers(c conditions.Condition, r asked) bool {
for _, p := range partsOf(c) {
if partKey(c.Key, p.name) == partKey(r.Condition, r.Part) {
return sameAsked(r.Actions, p.actions)
}
}
return false
}
// sourceAsker raises the asker's own condition.
const sourceAsker = "asker"
// sayUnasked keeps the asker's one condition: while a condition that needs the operator could not be asked
// on any channel, said loudly (failure must be loud), cleared when every one could be.
func (a *asker) sayUnasked(ctx context.Context, unasked []conditions.Condition, why string) error {
if a.raise == nil {
return nil
}
var obs []conditions.Observation
if len(unasked) > 0 {
keys := make([]string, 0, len(unasked))
severity := conditions.Warning
for _, c := range unasked {
keys = append(keys, c.Key)
if c.Severity == conditions.Urgent {
severity = conditions.Urgent
}
}
sort.Strings(keys)
obs = append(obs, conditions.Observation{Scope: conditions.ScopeSeat, ID: broker.AsksSeat, Token: "unasked",
Kind: "asks-undelivered", Severity: severity, Source: sourceAsker,
Summary: fmt.Sprintf("%d condition(s) that need the operator could not be asked on any channel: %s; %s",
len(keys), strings.Join(keys, ", "), why),
Headline: "Questions for you not delivered",
Explanation: "The mesh could not send you its questions on any channel.",
Needs: "answer them from the mesh MCP server, and check why no channel carries them.",
Resolved: "The mesh can ask you again"})
}
if err := a.raise(ctx, obs); err != nil {
a.logf("whether the operator could be asked could not be kept as a condition: %v", err)
}
return nil
}
// optionID is an action's label as an option's id: "Silence for a week" is silence-for-a-week.
func optionID(label string) string {
var b strings.Builder
dash := false
for _, r := range strings.ToLower(label) {
switch {
case r >= 'a' && r <= 'z', r >= '0' && r <= '9':
b.WriteRune(r)
dash = false
case !dash && b.Len() > 0:
b.WriteByte('-')
dash = true
}
}
return strings.TrimSuffix(b.String(), "-")
}
// doesWords is what an action does, in the words an option says it with.
func doesWords(act conditions.Action) string {
switch {
case act.Arguments["silence"] != "":
return "nothing more is said of it for a week"
case act.Verb == "mesh-delivery.release":
return "the delivery goes on"
case act.Verb == "mesh-delivery.stop":
return "the delivery ends"
case act.Verb == broker.ControllerSeat+".plans" && act.Arguments["go"] != "":
return "the delivery starts"
case act.Verb == broker.ControllerSeat+".plans" && act.Arguments["stop"] != "":
return "the delivery is stopped"
case strings.HasSuffix(act.Verb, ".restart"):
return "its service is restarted on " + act.Machine
}
return strings.ToLower(act.Label)
}
// askText is a condition's words as an ask says them: without where an answer is given when no channel can
// give it (FromMeshMCPServer), since the ask is answered on a channel and the router says where else.
func askText(s string) string {
for _, with := range []string{", " + FromMeshMCPServer, " " + FromMeshMCPServer} {
s = strings.ReplaceAll(s, with, ".")
}
return strings.ReplaceAll(s, "..", ".")
}
// askOf is the ask one part of a condition is asked with.
func askOf(id string, c conditions.Condition, p askPart, now time.Time) (asks.Ask, map[string]int) {
q := asks.Ask{ID: id, Headline: c.Headline, Explanation: askText(c.Explanation), Who: asks.Operator,
OnExpiry: "nothing is done, and you are asked again while it lasts", About: p.about,
Urgent: c.Severity == conditions.Urgent}
options := map[string]int{}
approves := false
for i, act := range p.actions {
level := levelOf(act) // an action that says nothing of its level is never taken for less than approve
approves = approves || level != asks.Acknowledge
oid := optionID(act.Label)
options[oid] = i
// Every option binds the exact act it stands for (novox/hq ADR 0259 §6): the verb, the machine and
// every argument. The warrant then authorises that act and no other.
binds, _ := asks.ActDigest(boundAct(act))
q.Options = append(q.Options, asks.Option{ID: oid, Label: act.Label, Does: doesWords(act), Level: level,
Binds: binds})
}
q.Expires = now.Add(askAcknowledgeFor)
if approves {
q.Expires = now.Add(askApproveFor)
}
return q, options
}
// boundAct is what an option's Binds digests: the act exactly as the controller will perform it — its verb,
// machine, level, and each argument as "arg.<name>" — and never its label or words.
func boundAct(act conditions.Action) asks.Act {
out := asks.Act{"verb": act.Verb, "machine": act.Machine, "level": act.Level}
for k, v := range act.Arguments {
out["arg."+k] = v
}
return out
}
func newAskID() string {
var b [8]byte
_, _ = rand.Read(b[:])
return "c" + hex.EncodeToString(b[:])
}
// askUnsent is an ask kept and never published: asked again at the next look.
const askUnsent = "unsent"
// ask publishes one ask about a part of a condition, kept before it is published (the review of 2026-10-09,
// L3): a warrant for it then always finds it, and one whose publishing failed is marked so and asked again.
func (a *asker) ask(ctx context.Context, c conditions.Condition, p askPart, channels string) error {
now := a.now()
id := newAskID()
q, options := askOf(id, c, p, now)
if err := q.Check(now); err != nil {
return err
}
body, err := json.Marshal(q)
if err != nil {
return err
}
if err := a.store.Create(ctx, asked{ID: id, Condition: c.Key, Part: p.name, Ask: q, Actions: p.actions,
Options: options, State: askOpen, Opened: now, Channels: channels}); err != nil {
return fmt.Errorf("the ask could not be kept, so it was not asked: %w", err)
}
if err := a.publish(ctx, asks.AskSubject(askerName), body, "ask."+id); err != nil {
if _, cerr := a.store.Change(ctx, id, func(x *asked) bool {
if x.State != askOpen || x.Acted != "" {
return false
}
x.State, x.Ended, x.Acted = askUnsent, a.now(), "nothing: it could not be published: "+err.Error()
return true
}); cerr != nil {
a.logf("the ask %s could not be published, and could not be marked so: %v", id, cerr)
}
return err
}
a.logf("asked the operator about %s (%s): %d answer(s)", c.Key, id, len(q.Options))
return nil
}
// cancel takes an ask back: kept cancelled first, so a warrant that comes after is refused, then said to the
// router; a cancel the router did not hear leaves the ask to expire there, and nothing is done on it here.
func (a *asker) cancel(ctx context.Context, r asked, why string) error {
stood, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.State != askOpen {
return false
}
x.State, x.Ended = askCancelled, a.now()
return true
})
if err != nil || !stood {
return err
}
body, _ := json.Marshal(map[string]string{"id": r.ID})
if err := a.publish(ctx, asks.CancelSubject(askerName), body, "cancel."+r.ID); err != nil {
a.logf("the ask %s about %s is taken back here, and the router could not be told (%v): it expires there, "+
"and no answer to it is acted on", r.ID, r.Condition, err)
return nil
}
a.logf("took back the ask %s about %s: %s", r.ID, r.Condition, why)
return nil
}
// Decided takes the router's word on one of the controller's asks (link.Decider). An error is returned only
// when what was decided could not be kept, so the word is held and heard again.
func (a *asker) Decided(ctx context.Context, body []byte) error {
var w asks.Warrant
if err := json.Unmarshal(body, &w); err != nil {
a.logf("the router's word on an ask could not be read; ignored: %v", err)
return nil
}
if w.Asker != askerName {
a.logf("REFUSED a warrant for %s's ask %s: the controller acts only on its own", w.Asker, w.Ask)
return nil
}
r, err := a.store.Get(ctx, w.Ask)
if err != nil {
return err
}
if r == nil {
a.logf("REFUSED a warrant for the ask %s, which the controller does not hold", w.Ask)
return nil
}
if r.Acted != "" {
return nil // heard again: acted on once
}
now := a.now()
if w.Outcome != asks.OutcomeChosen {
acted := "nothing: the ask " + string(w.Outcome)
if w.Words != "" {
acted += ": " + w.Words
}
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.Acted != "" {
return false
}
x.State, x.Ended, x.Warrant, x.Acted = string(w.Outcome), now, &w, acted
return true
}); err != nil {
return err
}
a.logf("the ask %s about %s ended %s; nothing is done", r.ID, r.Condition, w.Outcome)
return nil
}
if r.State != askOpen {
// Cancelled, replaced or expired in the controller's own record: no answer to it is acted on.
a.logf("REFUSED a warrant for the ask %s, which is %s in the controller's own record", r.ID, r.State)
return nil
}
option, err := w.For(askerName, r.Ask)
if err != nil {
a.logf("REFUSED a warrant for the ask %s: %v", r.ID, err)
return nil
}
index, offered := r.Options[option.ID]
if !offered || index >= len(r.Actions) {
a.logf("REFUSED a warrant for the ask %s: it chose %s, which no action stands for", r.ID, option.ID)
return nil
}
act := r.Actions[index]
// The act about to be performed is the one the option bound when the controller asked: a record changed
// since is refused, never performed.
if err := option.Performs(boundAct(act)); err != nil {
a.logf("REFUSED a warrant for the ask %s: %v", r.ID, err)
return nil
}
open, err := a.open(ctx)
if err != nil {
return err
}
stillOpen := r.Rehearsal || r.Proposal != nil // a rehearsal and a proposal are about no condition
for _, c := range open {
stillOpen = stillOpen || c.Key == r.Condition
}
if !stillOpen {
// The asker checks the state is still what it asked about before it acts (to-be 46 §10, step 7).
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.State != askOpen || x.Acted != "" {
return false
}
x.State, x.Warrant, x.Ended, x.Acted = string(asks.OutcomeChosen), &w, now,
"nothing: the condition ended before the answer"
return true
}); err != nil {
return err
}
a.logf("%s, for %s, which ended meanwhile: nothing is done", w.Says(), r.Condition)
return nil
}
// Claimed before acting, by compare-and-set: only the delivery whose write stands acts (security review
// of 2026-10-08, finding 9). Not by the warrant's message id, which another publisher could take first:
// the controller's own record decides.
claimed, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.State != askOpen || x.Acted != "" {
return false
}
x.State, x.Warrant, x.Acted = string(asks.OutcomeChosen), &w, "acting"
return true
})
if err != nil {
return err
}
if !claimed {
a.logf("the warrant for the ask %s was already taken by another delivery; nothing more is done", r.ID)
return nil
}
r.Acted = "acting"
why := fmt.Sprintf("%s (ask %s)", w.Says(), r.ID)
args := map[string]string{}
for k, v := range act.Arguments {
args[k] = v
}
if v, takes := args["why"]; takes && v == "" {
args["why"] = why
}
var acted error
outcome := "done"
switch {
case r.Rehearsal && act.Verb == rehearsalVerb:
// A rehearsal's answer performs nothing: it is recorded below as the operator's decision.
case r.Proposal != nil && act.Verb == proposalVerb:
// A proposed settings layer, set by this controller itself on Approve (novox/hq ADR 0277): the act's
// digest of the values is held to the record's own values before anything is set.
outcome, acted = a.decideProposal(ctx, *r, act, w)
if acted == nil && outcome != "" && !strings.HasPrefix(outcome, "nothing") {
outcome = "done: " + outcome
}
case act.Arguments["silence"] != "":
acted = a.silence(ctx, act.Arguments["silence"], conditions.MaxSilence, byWords(w), why)
default:
acted = a.call(ctx, act, args)
}
ended := a.now()
if acted != nil {
outcome = "failed: " + acted.Error()
}
r.Ended, r.Acted = ended, outcome
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.Acted != "acting" {
return false
}
x.Ended, x.Acted = ended, outcome
return true
}); err != nil {
a.logf("%s was acted on (%s), and how it ended could NOT be kept: %v", r.ID, outcome, err)
}
verbArgs := []string{act.Verb}
if act.Machine != "" {
verbArgs = append(verbArgs, "on "+act.Machine)
}
keys := make([]string, 0, len(args))
for k := range args {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
if k != "why" {
verbArgs = append(verbArgs, k+"="+args[k])
}
}
if err := a.record(ctx, link.HandAct{Verb: handActWarrant, Args: verbArgs, Why: why, By: byWords(w),
Cause: conditions.CauseOperatorAnswer, Condition: r.Condition, Via: viaWords(w), Ask: r.ID,
Proofs: w.Proofs, RequestedBy: r.Condition, Outcome: r.Acted}); err != nil {
a.logf("%s was done, and could NOT be recorded in the hand-act log: %v", why, err)
}
a.logf("%s: %s", why, r.Acted)
return nil
}
// handActWarrant is the verb an act the operator chose on a warrant is recorded under: a person's decision,
// never a repair (handActVerbs).
const handActWarrant = "warrant"
// byWords is who chose, as the hand-act log says it: "the operator, as telegram identity 42".
func byWords(w asks.Warrant) string {
if w.By == nil {
return "the operator"
}
return fmt.Sprintf("the %s, as %s identity %s", w.By.Who, w.By.Kind, w.By.Identity)
}
// viaWords is the channel an answer came through: its module and kind, and how the sender was known.
func viaWords(w asks.Warrant) string {
if w.By == nil {
return w.Channel
}
via := w.Channel + " (" + w.By.Kind + ")"
if w.By.Verified != "" {
via += ", " + w.By.Verified
}
return via
}
// errNotGranted is an action whose verb the controller's grant does not name.
var errNotGranted = errors.New("the controller's grant does not name this verb")
+153
View File
@@ -0,0 +1,153 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// busAsker is an asker on a real bus's `asked` bucket, counting what it performs: two of them are two
// controllers sharing one record.
type busAskerRig struct {
mu sync.Mutex
called int
acts int
open []conditions.Condition
sent [][]byte
}
func (rig *busAskerRig) asker(t *testing.T, conn *nats.Conn, now time.Time) *asker {
return &asker{
open: func(context.Context) ([]conditions.Condition, error) {
rig.mu.Lock()
defer rig.mu.Unlock()
return rig.open, nil
},
silence: func(context.Context, string, time.Duration, string, string) error { return nil },
store: busAsked{conn: conn},
publish: func(_ context.Context, subject string, body []byte, _ string) error {
rig.mu.Lock()
defer rig.mu.Unlock()
if subject == asks.AskSubject(askerName) {
rig.sent = append(rig.sent, body)
}
return nil
},
call: func(context.Context, conditions.Action, map[string]string) error {
time.Sleep(20 * time.Millisecond) // long enough for the other delivery to arrive meanwhile
rig.mu.Lock()
defer rig.mu.Unlock()
rig.called++
return nil
},
record: func(context.Context, link.HandAct) error {
rig.mu.Lock()
defer rig.mu.Unlock()
rig.acts++
return nil
},
now: func() time.Time { return now },
logf: t.Logf,
}
}
func askedBus(t *testing.T) *nats.Conn {
t.Helper()
conn, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(conn.Close)
js, err := jetstream.New(conn)
if err != nil {
t.Fatal(err)
}
if _, err := js.CreateKeyValue(context.Background(), jetstream.KeyValueConfig{Bucket: broker.AskedBucket}); err != nil {
t.Fatal(err)
}
return conn
}
// The review of 2026-10-09 (L7): two deliveries of one warrant, to two controllers at once, perform its act
// exactly once and record it once — the record's compare-and-set decides, never the warrant's message id.
func TestTwoAnswersAtOnceActOnce(t *testing.T) {
conn := askedBus(t)
now := time.Date(2026, 10, 9, 14, 0, 0, 0, time.UTC)
rig := &busAskerRig{open: []conditions.Condition{heldCondition()}}
first, second := rig.asker(t, conn, now), rig.asker(t, conn, now)
if err := first.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
if len(rig.sent) != 1 {
t.Fatalf("asked %d times", len(rig.sent))
}
var q asks.Ask
_ = json.Unmarshal(rig.sent[0], &q)
release, _ := q.Option("release")
w := asks.Warrant{Ask: q.ID, Asker: askerName, About: q.About, Outcome: asks.OutcomeChosen, Option: release.ID,
Label: release.Label, Level: release.Level, Channel: "telegram", Proofs: []string{"P1"}, At: now,
AskDigest: q.Digest(), By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"}}
body, _ := json.Marshal(w)
var wg sync.WaitGroup
for _, a := range []*asker{first, second, first, second} {
wg.Add(1)
go func(a *asker) {
defer wg.Done()
if err := a.Decided(context.Background(), body); err != nil {
t.Error(err)
}
}(a)
}
wg.Wait()
if rig.called != 1 || rig.acts != 1 {
t.Fatalf("performed %d time(s), recorded %d time(s)", rig.called, rig.acts)
}
got, err := busAsked{conn: conn}.Get(context.Background(), q.ID)
if err != nil || got == nil || got.Acted != "done" {
t.Fatalf("kept as %+v (%v)", got, err)
}
}
// The review of 2026-10-09 (L2): a write decided on a record read earlier never lands over one made since. A
// cancel read before the answer was acted on leaves the act's record as it is.
func TestAStaleCancelDoesNotWriteOverAnAct(t *testing.T) {
conn := askedBus(t)
now := time.Date(2026, 10, 9, 14, 0, 0, 0, time.UTC)
rig := &busAskerRig{open: []conditions.Condition{heldCondition()}}
a := rig.asker(t, conn, now)
if err := a.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
var q asks.Ask
_ = json.Unmarshal(rig.sent[0], &q)
stale, _ := busAsked{conn: conn}.Get(context.Background(), q.ID)
release, _ := q.Option("release")
w := asks.Warrant{Ask: q.ID, Asker: askerName, About: q.About, Outcome: asks.OutcomeChosen, Option: release.ID,
Label: release.Label, Level: release.Level, Channel: "telegram", At: now, AskDigest: q.Digest(),
By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"}}
body, _ := json.Marshal(w)
if err := a.Decided(context.Background(), body); err != nil {
t.Fatal(err)
}
if err := a.cancel(context.Background(), *stale, "the condition ended"); err != nil {
t.Fatal(err)
}
got, _ := busAsked{conn: conn}.Get(context.Background(), q.ID)
if got.State != string(asks.OutcomeChosen) || got.Acted != "done" {
t.Errorf("a stale cancel wrote over the act: %+v", got)
}
}
File diff suppressed because it is too large Load Diff
+589
View File
@@ -0,0 +1,589 @@
package main
// The asker on the bus: its asks in the controller's bucket `asked`, its asks and cancels published on the
// seat under the controller's name, the verbs a warrant chooses called with the controller's grant, and the
// router's record of its asks read under its name (novox/hq ADR 0259).
import (
"context"
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// askerFrom is the serving controller's asker; nil in any other process.
var askerFrom *asker
// askWithin is how long a verb a warrant chose is given to answer.
const askWithin = time.Minute
type busAsked struct{ conn *nats.Conn }
func (b busAsked) kv(ctx context.Context) (jetstream.KeyValue, error) {
js, err := jetstream.New(b.conn)
if err != nil {
return nil, err
}
return js.KeyValue(ctx, broker.AskedBucket)
}
func (b busAsked) Get(ctx context.Context, id string) (*asked, error) {
kv, err := b.kv(ctx)
if err != nil {
return nil, err
}
e, err := kv.Get(ctx, id)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, nil
}
if err != nil {
return nil, err
}
var r asked
return &r, json.Unmarshal(e.Value(), &r)
}
// Create keeps a new ask under its id, and only where none is kept: never over another.
func (b busAsked) Create(ctx context.Context, r asked) error {
kv, err := b.kv(ctx)
if err != nil {
return err
}
body, err := json.Marshal(r)
if err != nil {
return err
}
_, err = kv.Create(ctx, r.ID, body)
return err
}
// Change applies change to the ask kept under id by compare-and-set on its key's revision (the review of
// 2026-10-09, L2): read, changed, and written only over the revision read; when another write came between,
// read again and asked again, at most askChangeTries times. change says whether to write at all.
func (b busAsked) Change(ctx context.Context, id string, change func(*asked) bool) (bool, error) {
kv, err := b.kv(ctx)
if err != nil {
return false, err
}
for try := 0; try < askChangeTries; try++ {
e, err := kv.Get(ctx, id)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return false, nil
}
if err != nil {
return false, err
}
var r asked
if err := json.Unmarshal(e.Value(), &r); err != nil {
return false, err
}
if !change(&r) {
return false, nil
}
body, err := json.Marshal(r)
if err != nil {
return false, err
}
if _, err := kv.Update(ctx, id, body, e.Revision()); err != nil {
var api *jetstream.APIError
if errors.Is(err, jetstream.ErrKeyExists) || (errors.As(err, &api) && api.ErrorCode == jetstream.JSErrCodeStreamWrongLastSequence) {
continue
}
return false, err
}
return true, nil
}
return false, fmt.Errorf("the ask %s changed under every one of %d tries", id, askChangeTries)
}
func (b busAsked) All(ctx context.Context) ([]asked, error) {
kv, err := b.kv(ctx)
if err != nil {
return nil, err
}
lister, err := kv.ListKeys(ctx)
if err != nil {
return nil, err
}
defer func() { _ = lister.Stop() }()
var out []asked
for k := range lister.Keys() {
e, err := kv.Get(ctx, k)
if err != nil {
continue
}
var r asked
if json.Unmarshal(e.Value(), &r) == nil {
out = append(out, r)
}
}
return out, nil
}
// callAction performs an action's verb as the controller, through the grant that names it.
func callAction(conn *nats.Conn) func(ctx context.Context, a conditions.Action, args map[string]string) error {
return func(ctx context.Context, a conditions.Action, args map[string]string) error {
seat, verb, ok := strings.Cut(a.Verb, ".")
if !ok {
return fmt.Errorf("%q names no seat and verb", a.Verb)
}
body := map[string]any{}
for k, v := range args {
body[k] = v
}
if seat == catalogue.DeliverySeat {
_, err := askDeliveryOwner(ctx, conn, verb, body)
return err
}
granted := false
for _, v := range broker.VerbsTheControllerActsOnAWarrant {
granted = granted || (v.Seat == seat && v.Verb == verb)
}
if !granted {
return fmt.Errorf("%s: %w", a.Verb, errNotGranted)
}
var answer link.Answer
var err error
if a.Machine != "" {
answer, err = link.AskSeatTool(ctx, conn, seat, verb, a.Machine, body, askWithin)
} else {
answer, err = link.AskMeshSeatTool(ctx, conn, seat, verb, body, askWithin)
}
if err != nil {
return err
}
if answer.Error != "" {
return fmt.Errorf("%s refused: %s", a.Verb, answer.Error)
}
return nil
}
}
// routerRecordOf reads the router's record of one of the controller's asks, under its name, and answers
// how it ended when it did: the bucket is the one the asks seat's declarer names as its records.
func routerRecordOf(conn *nats.Conn, inv *inventory.Inventory) func(ctx context.Context, id string) (*asks.Warrant, error) {
return func(ctx context.Context, id string) (*asks.Warrant, error) {
bucket, err := asksRecords(ctx, inv)
if err != nil || bucket == "" {
return nil, err
}
reply, err := conn.RequestWithContext(ctx, "$JS.API.DIRECT.GET.KV_"+bucket+".$KV."+bucket+"."+askerName+"."+id, nil)
if err != nil {
return nil, err
}
if reply.Header.Get("Status") != "" {
return nil, nil // none, or not readable: the event says it
}
var rec struct {
State string `json:"state"`
Warrant *asks.Warrant `json:"warrant"`
}
if json.Unmarshal(reply.Data, &rec) != nil || rec.State == "open" || rec.Warrant == nil {
return nil, nil
}
return rec.Warrant, nil
}
}
// asksRecords is the bucket the asks seat's declarer keeps its record of asks in.
func asksRecords(ctx context.Context, inv *inventory.Inventory) (string, error) {
declared, err := inv.Catalogue(ctx)
if err != nil {
return "", err
}
for _, m := range declared {
for _, s := range m.DefinesSeats {
if s.Name == broker.AsksSeat && len(s.Records) > 0 {
return broker.BucketName(m.Module, s.Records[0]), nil
}
}
}
return "", nil
}
// routerHereIn says whether a module declaring the asks seat, with its ask named by its caller, is assigned:
// without it nothing takes an ask, and asking would only fill a queue nobody reads.
func routerHereIn(inv *inventory.Inventory) func(ctx context.Context) (bool, error) {
return func(ctx context.Context) (bool, error) {
entries, err := inv.Catalogued(ctx)
if err != nil {
return false, err
}
for _, e := range entries {
for _, s := range e.Manifest.DefinesSeats {
if s.Name == broker.AsksSeat && s.NamedByCaller("ask") && len(e.On) > 0 {
return true, nil
}
}
}
return false, nil
}
}
// grantHeldIn says whether the bus holds the controller's grant to ask (novox/hq issue 353), judged on the two
// facts that make it so, and failing closed on each:
//
// 1. **composed**: the user list the mesh composes now grants the controller itself to publish its ask
// (controllerMayAsk) — the controller's own grant, not whether any user of the list changed;
// 2. **in force**: the bus's own module, on the machine holding the bus, says the user list there allows it
// and the server reloaded after that list was written (the nats module's `nats_user_can`, `in_force`) — a
// list sent is not a list the bus enforces until it reloads.
//
// Anything that cannot be worked out is not held, with why, so the conditions that need the operator are said
// as undelivered rather than asked into a refusal.
func grantHeldIn(open *stores, conn *nats.Conn) func(ctx context.Context) (bool, string, error) {
return grantHeldBy(composedGrantIn(open), busGrantIn(open, conn), time.Now)
}
// grantJudge is one of the two facts: whether it holds, and why not.
type grantJudge func(ctx context.Context) (bool, string, error)
// grantHeldBy is the judgement over the two facts, remembered for grantLookEvery: composing the list resolves
// the bus's machine whole, and asking the bus's module is a call.
func grantHeldBy(composed, inForce grantJudge, now func() time.Time) func(ctx context.Context) (bool, string, error) {
var mu sync.Mutex
var at time.Time
var held bool
var why string
return func(ctx context.Context) (bool, string, error) {
mu.Lock()
defer mu.Unlock()
if !at.IsZero() && now().Sub(at) < grantLookEvery {
return held, why, nil
}
held, why = judgeGrant(ctx, composed, inForce)
at = now()
return held, why, nil
}
}
// judgeGrant is held only when both facts hold; an error is a fact not known, never a yes. A fact that holds
// with a caveat (a reload not confirmed) gives that caveat as the why of a yes.
func judgeGrant(ctx context.Context, composed, inForce grantJudge) (bool, string) {
var caveat []string
for _, fact := range []struct {
judge grantJudge
what string
}{{composed, "whether the mesh composes the controller a grant to ask"}, {inForce, "whether the bus enforces it"}} {
if fact.judge == nil {
return false, fact.what + " is not judged here"
}
ok, why, err := fact.judge(ctx)
if err != nil {
return false, fact.what + " could not be worked out: " + err.Error()
}
if !ok {
return false, why
}
if why != "" {
caveat = append(caveat, why)
}
}
return true, strings.Join(caveat, "; ")
}
// controllerMayAsk says whether the controller's own grant, as composed from these records, lets it publish its
// ask (ADR 0259 §3: composed while a router that takes asks under its asker's name is assigned).
func controllerMayAsk(records broker.Records) (bool, error) {
users, err := broker.Users(records)
if err != nil {
return false, err
}
for _, u := range users {
if u.Kind != broker.KindController {
continue
}
perms, err := broker.PermissionsFor(u)
if err != nil {
return false, err
}
return broker.MayPublish(perms, asks.AskSubject(askerName)), nil
}
return false, errors.New("the composition holds no controller")
}
// composedGrantIn is the first fact, from the mesh's records.
func composedGrantIn(open *stores) grantJudge {
return func(ctx context.Context) (bool, string, error) {
if open == nil || open.inventory == nil {
return false, "", errors.New("the mesh's records are not open")
}
records, err := open.inventory.BusRecords(ctx)
if err != nil {
return false, "", err
}
may, err := controllerMayAsk(records)
if err != nil || may {
return may, "", err
}
return false, "the mesh composes the controller no grant to ask: no router that takes asks under its " +
"asker's name is assigned", nil
}
}
// busGrantIn is the second fact, asked of the bus's own module on the machine holding the bus, with the check
// main made before it (novox/hq issue 353, PR 186: is the user list sent to the bus's machine the one composed
// now) kept for the cases the module's word does not settle.
func busGrantIn(open *stores, conn *nats.Conn) grantJudge {
ask := func(ctx context.Context) (string, json.RawMessage, error) {
if open == nil || open.inventory == nil || conn == nil {
return "", nil, errors.New("the mesh's records or the bus are not open")
}
holders, err := seatHolders(ctx, open.inventory)
if err != nil {
return "", nil, err
}
h, held := holders[theBrokerSeat]
if !held || h.Node == "" || h.Module == "" {
return "", nil, nil
}
a, err := link.AskModuleToolOn(ctx, conn, h.Module, busUserCanTool, h.Node, map[string]any{
"user": broker.ControllerName, "action": "publish", "subject": asks.AskSubject(askerName)}, 15*time.Second)
if err != nil {
return h.Node, nil, err
}
if a.Error != "" {
return h.Node, nil, errors.New(a.Error)
}
return h.Node, a.Result, nil
}
list := func(ctx context.Context) (busListState, error) {
if open == nil || open.inventory == nil {
return busListState{}, errors.New("the mesh's records are not open")
}
s, err := brokerListState(ctx, open, nil)
if err != nil || s.Machine == "" || !s.Behind {
return s, err
}
step, err := busHeld(ctx, open.inventory, []string{s.Machine})
if err != nil {
return busListState{}, err
}
s.BusStep = step[s.Machine]
return s, nil
}
return busGrantBy(ask, list)
}
// busListState is the user list on the bus's machine as the controller's record knows it: whether the list
// sent there is behind the one composed now, why it could not be composed, and the bus step a send there waits
// on (that machine cannot be pushed until a person's `bus upgrade`).
type busListState struct {
Machine string
Behind bool
Uncomposed error
BusStep string
}
// remedy is the one thing that sends the bus's machine the list composed now: the plan's own reason when it
// cannot be composed, the bus upgrade while sends there wait on it, and otherwise a push of that machine.
func (s busListState) remedy(node string) string {
if s.Machine != "" {
node = s.Machine
}
switch {
case s.Uncomposed != nil:
return fmt.Sprintf("the user list for %s cannot be composed (%v); `plan %s` says what to fix first",
node, s.Uncomposed, node)
case s.BusStep != "":
return fmt.Sprintf("sends to %s wait on the bus's planned step, so a push will not carry it: %s", node, s.BusStep)
}
return fmt.Sprintf("`push %s` carries it", node)
}
// busGrantBy judges the bus's word, and names for each way it is not held the one thing that makes it held:
//
// - **in force**: held;
// - **allowed, from a module older than `in_force`**: held, with the caveat that the reload is not confirmed.
// The file allows the publish, and the bus reloads within about 5 s of a change to it, so the worst case is
// one ask refused in those seconds, which is asked again (issue 369). The list check is not consulted:
// while a bus step is pending the bus's machine cannot be pushed, so any change to the composed list would
// read as behind for good and silence every ask until a person's bus upgrade (the review of PR 188);
// - **allowed, written, not reloaded** (the module gives both times): wait about 5 s;
// - **allowed, in force not known** (the module could not read a time): what is not known, said;
// - **not allowed, or the module cannot answer**: the list check decides — the remedy that sends the list
// when it is behind (busListState.remedy), and, for a module that cannot answer, held while the list sent
// is current, as main judged it.
//
// ask answers the machine holding the bus and its module's answer ("" when nothing holds the bus); list is the
// record's word on the list sent to that machine.
func busGrantBy(ask func(ctx context.Context) (string, json.RawMessage, error),
list func(ctx context.Context) (busListState, error)) grantJudge {
return func(ctx context.Context) (bool, string, error) {
node, raw, askErr := ask(ctx)
if askErr == nil && node == "" {
return false, "no module holds the bus, so nothing says what it enforces", nil
}
var answer busAnswer
if askErr == nil {
if answer, askErr = readBusAnswer(raw); askErr != nil {
askErr = fmt.Errorf("its answer could not be read: %w", askErr)
}
}
if askErr != nil {
if node == "" {
node = "the bus's machine"
}
s, err := list(ctx)
if err != nil {
return false, "", fmt.Errorf("the bus module on %s could not answer (%v), and whether the list sent "+
"there is the one composed now could not be worked out: %w", node, askErr, err)
}
if s.Behind {
return false, fmt.Sprintf("the bus module on %s could not answer (%v), and the user list sent there is "+
"behind what the mesh composes; %s", node, askErr, s.remedy(node)), nil
}
return true, fmt.Sprintf("the bus module on %s could not answer (%v); judged by the user list sent there, "+
"which is the one composed now", node, askErr), nil
}
notAllowed := func(older string) (bool, string, error) {
s, err := list(ctx)
if err != nil {
return false, "", fmt.Errorf("the user list on %s does not allow the controller to ask%s, and whether "+
"the list sent there is the one composed now could not be worked out: %w", node, older, err)
}
if s.Behind {
return false, fmt.Sprintf("the user list on %s does not allow the controller to ask%s: it is behind "+
"what the mesh composes; %s", node, older, s.remedy(node)), nil
}
return false, fmt.Sprintf("the user list on %s does not allow the controller to ask%s, though the list "+
"sent there is the one composed now; %s again", node, older, s.remedy(node)), nil
}
switch {
case answer.InForce == nil && answer.Allowed:
return true, fmt.Sprintf("the user list on %s allows the controller to ask, but its reload is not "+
"confirmed: the bus module there is older than the answer whether a grant is in force, and the bus "+
"upgrade (`bus upgrade`) carries one that confirms it", node), nil
case answer.InForce == nil:
return notAllowed(" (the bus module there is older than the answer whether a grant is in force)")
case !answer.Allowed:
return notAllowed("")
case *answer.InForce:
return true, "", nil
case answer.FileWritten != "" && answer.ServerLoaded != "":
return false, fmt.Sprintf("the bus on %s has the controller's grant to ask written (%s) but has not "+
"reloaded since (last load %s); it reloads in place within about 5 s, so nothing is to be done but wait",
node, answer.FileWritten, answer.ServerLoaded), nil
}
why := answer.Why
if why == "" {
why = "it gives no reason"
}
return false, fmt.Sprintf("the user list on %s allows the controller to ask, but whether the bus enforces it "+
"is not known: %s", node, why), nil
}
}
// busUserCanTool is the bus module's answer to whether a user may do something, and whether it is in force.
const busUserCanTool = "nats_user_can"
// busAnswer is the bus module's answer to busUserCanTool; InForce is nil from a module older than that field,
// and FileWritten and ServerLoaded are given only when it could read both times (mesh-catalog's nats InForce).
type busAnswer struct {
Allowed bool `json:"allowed"`
InForce *bool `json:"in_force"`
Why string `json:"why"`
FileWritten string `json:"file_written"`
ServerLoaded string `json:"server_loaded"`
}
func readBusAnswer(raw json.RawMessage) (busAnswer, error) {
var answer busAnswer
err := json.Unmarshal(raw, &answer)
return answer, err
}
// grantLookEvery is how often the bus's user list is judged against the one its machine was last sent.
const grantLookEvery = 30 * time.Second
// channelsIn is what the channels are now, as a fingerprint: each module claiming a kind of the channel
// bench, where, promising what, and whether of its own account. An ask the router refused is asked again
// once this changes.
func channelsIn(inv *inventory.Inventory) func(ctx context.Context) string {
return func(ctx context.Context) string {
entries, err := inv.Catalogued(ctx)
if err != nil {
return ""
}
var parts []string
for _, e := range entries {
for _, c := range e.Manifest.Claims {
if c.Kind == "" || !catalogue.KindedBenches[c.Name] {
continue
}
on := append([]string(nil), e.On...)
sort.Strings(on)
caps := append([]string(nil), c.Capabilities...)
sort.Strings(caps)
parts = append(parts, fmt.Sprintf("%s/%s=%s@%s[%s]own:%t", c.Name, c.Kind, e.Manifest.Module,
strings.Join(on, ","), strings.Join(caps, ","), e.Manifest.RunsAs != ""))
}
}
sort.Strings(parts)
return strings.Join(parts, ";")
}
}
// startAsking makes the serving controller's asker and hands it the router's words.
func startAsking(ctx context.Context, open *stores, server *link.Server, conn *nats.Conn, keeper *conditions.Keeper) {
js, err := jetstream.New(conn)
if err != nil {
fmt.Printf("the operator cannot be asked: %v\n", err)
return
}
a := servingAsker(open, conn, js, keeper)
if err := server.Decides(a); err != nil {
fmt.Printf("the operator's answers cannot be heard, so nothing is asked: %v\n", err)
return
}
askerFrom = a
go a.keep(ctx)
}
// servingAsker is the serving controller's asker, wired to the mesh: what it reads, publishes, performs and
// records, and the judgements it asks by — the bus's grant to ask among them (novox/hq issue 353).
func servingAsker(open *stores, conn *nats.Conn, js jetstream.JetStream, keeper *conditions.Keeper) *asker {
return &asker{
open: keeper.Open,
silence: func(ctx context.Context, key string, d time.Duration, by, why string) error {
_, err := keeper.Silence(ctx, key, d, by, why)
return err
},
store: busAsked{conn: conn},
publish: func(ctx context.Context, subject string, body []byte, id string) error {
_, err := js.Publish(ctx, subject, body, jetstream.WithMsgID(id))
return err
},
call: callAction(conn),
// A proposed settings layer is set by this controller itself on the warrant (novox/hq ADR 0277).
setLayer: setLayerIn(open),
record: func(ctx context.Context, act link.HandAct) error {
_, err := link.RecordHandAct(ctx, conn, act)
return err
},
routerRecord: routerRecordOf(conn, open.inventory),
routerHere: routerHereIn(open.inventory),
grantHeld: grantHeldIn(open, conn),
channels: channelsIn(open.inventory),
raise: func(ctx context.Context, obs []conditions.Observation) error {
return keeper.Reconcile(ctx, sourceAsker, obs)
},
now: time.Now,
logf: func(format string, args ...any) { fmt.Printf(format+"\n", args...) },
}
}
@@ -0,0 +1,182 @@
//go:build integration
package main
import (
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/overlay"
)
func TestAModuleAssignedAndNotPushedIsAwaitingAPushNotUncomposable(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
pushedBy(t, open, "laptop") // the machine was pushed before; then the module is assigned
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
// The read the rest of the mesh asks still refuses it, with the composer's typed error: nothing
// can be compared about a secret that does not exist (status), and nothing here string-matches.
plan, settings, err := planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
_, err = declarationWith(ctx, open, "laptop", plan, settings, gens, Reading)
var notMade *catalogue.NotMadeError
if !errors.As(err, &notMade) || notMade.Module != "keeper" || notMade.Name != "repository" {
t.Fatalf("a read composition did not say which secret is not made, typed: %v", err)
}
// Foreseen, it composes, names what the push makes, and made nothing.
foreseen, err := declarationWith(ctx, open, "laptop", plan, settings, gens, Foreseeing)
if err != nil || len(foreseen.foreseen) != 1 || foreseen.foreseen[0] != "keeper/repository" {
t.Fatalf("foreseen: %v %v", foreseen.foreseen, err)
}
if _, held, err := open.inventory.ModuleSecretIfIssued(ctx, "laptop", "keeper", "repository"); err != nil || held {
t.Fatalf("asking ahead of the push made the secret (held %v, %v)", held, err)
}
// Within the bound: nothing at all — no urgent, no warning, nobody notified.
if got := d1(t, open); len(got) != 0 {
t.Fatalf("a machine waiting for a push raised %+v", got)
}
// Past the bound: a warning, not urgent, saying what the push will make.
before := awaitingPushBound
awaitingPushBound = -time.Minute
t.Cleanup(func() { awaitingPushBound = before })
got := d1(t, open)
if len(got) != 1 || got[0].Key() != "machine.laptop.awaiting-push" || got[0].Severity != conditions.Warning ||
!strings.Contains(got[0].Summary, "keeper/repository") || !strings.Contains(got[0].Summary, "push laptop") {
t.Fatalf("a machine left un-pushed past the bound: %+v", got)
}
// Pushed: the secret is made and D1 says nothing.
pushedBy(t, open, "laptop")
if got := d1(t, open); len(got) != 0 {
t.Fatalf("a pushed machine still raised %+v", got)
}
}
func TestASecretThePushCannotMakeIsStillUncomposable(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aTalker())
if _, err := assign(ctx, open, "laptop", "talker"); err != nil {
t.Fatal(err)
}
got := d1(t, open)
if len(got) != 1 || got[0].Key() != "machine.laptop.uncomposable" || got[0].Severity != conditions.Urgent ||
!strings.Contains(got[0].Summary, "module issue talker --node laptop") {
t.Fatalf("a push that will be refused was not said urgently: %+v", got)
}
// And a machine whose composition fails for anything else, with a secret waiting beside it, is
// uncomposable for that — the stand-in hides nothing.
register(t, open, aKeeper())
if _, err := assign(ctx, open, "anchor", "keeper"); err != nil {
t.Fatal(err)
}
one, two := rivals()
register(t, open, one)
register(t, open, two)
_, _ = assign(ctx, open, "anchor", "rival-one")
_, _ = assign(ctx, open, "anchor", "rival-two")
keys := map[string]conditions.Severity{}
for _, o := range d1(t, open) {
keys[o.Key()] = o.Severity
}
if keys["machine.anchor.uncomposable"] != conditions.Urgent {
t.Fatalf("a real failure beside a waiting secret: %v", keys)
}
}
func TestAGivenSecretUnderAnOldKeyIsNotForeseen(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
if err := open.inventory.AcceptSecretForModule(ctx, "laptop", "keeper", "repository", "given"); err != nil {
t.Fatal(err)
}
record, err := open.inventory.NodeByName(ctx, "laptop")
if err != nil {
t.Fatal(err)
}
if err := open.inventory.RecordSealingKey(ctx, record.ID, aPublicKey(t)); err != nil {
t.Fatal(err)
}
got := d1(t, open)
if len(got) != 1 || got[0].Key() != "machine.laptop.uncomposable" || !strings.Contains(got[0].Summary, "issue it again") {
t.Fatalf("a given secret under an old key: %+v", got)
}
}
func TestAMachineAwaitsAPushSinceItsOldestAssignmentSinceTheLastSend(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
pushedBy(t, open, "laptop")
sent := time.Now()
since, err := open.inventory.AwaitingSince(ctx, "laptop")
if err != nil || since.After(sent) {
t.Fatalf("nothing assigned since the send: waiting since %v (%v), the send was before %v", since, err, sent)
}
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
since, err = open.inventory.AwaitingSince(ctx, "laptop")
if err != nil || since.Before(sent.Add(-time.Second)) {
t.Fatalf("assigned after the send: waiting since %v (%v), not from the assignment", since, err)
}
}
func TestALastSendIsReadFromTheSendAndTheReport(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
pushedBy(t, open, "laptop")
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
sends, err := readDeliveries(ctx, open.inventory)
if err != nil {
t.Fatal(err)
}
if sends["laptop"].settled("keeper", time.Now()) {
t.Fatal("a holder assigned and not pushed is expected to answer")
}
if !sends["laptop"].settled(overlay.Name, time.Now().Add(time.Hour)) {
t.Fatal("a module the machine was sent long ago is not expected to answer")
}
pushedBy(t, open, "laptop")
sends, err = readDeliveries(ctx, open.inventory)
if err != nil {
t.Fatal(err)
}
if sends["laptop"].settled("keeper", time.Now()) {
t.Fatal("pushed a moment ago and not reported, a holder is already expected")
}
if !sends["laptop"].settled("keeper", time.Now().Add(reportGrace+time.Minute)) {
t.Fatal("pushed past the grace, a holder is not expected")
}
if _, ok := sends["laptop"].carried["keeper"]; !ok {
t.Fatalf("the send's builds do not carry keeper: %v", sends["laptop"].carried)
}
pushedAndApplied(t, open, "laptop")
if sends, err = readDeliveries(ctx, open.inventory); err != nil || !sends["laptop"].settled("keeper", time.Now()) {
t.Fatalf("pushed and reported applied, a holder is not expected to answer (%v)", err)
}
}
-166
View File
@@ -1,15 +1,12 @@
package main
import (
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/overlay"
)
// Between `assign` and `push` a module's own secrets are not made yet: the push makes them. D1 composed
@@ -81,140 +78,14 @@ func d1(t *testing.T, open *stores) []conditions.Observation {
// **THE WINDOW, REPRODUCED**: assigned and not pushed, a module whose own secret the push makes is
// waiting, not uncomposable; past the bound it is a warning naming what the push makes; pushed, nothing.
func TestAModuleAssignedAndNotPushedIsAwaitingAPushNotUncomposable(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
pushedBy(t, open, "laptop") // the machine was pushed before; then the module is assigned
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
// The read the rest of the mesh asks still refuses it, with the composer's typed error: nothing
// can be compared about a secret that does not exist (status), and nothing here string-matches.
plan, settings, err := planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
_, err = declarationWith(ctx, open, "laptop", plan, settings, gens, Reading)
var notMade *catalogue.NotMadeError
if !errors.As(err, &notMade) || notMade.Module != "keeper" || notMade.Name != "repository" {
t.Fatalf("a read composition did not say which secret is not made, typed: %v", err)
}
// Foreseen, it composes, names what the push makes, and made nothing.
foreseen, err := declarationWith(ctx, open, "laptop", plan, settings, gens, Foreseeing)
if err != nil || len(foreseen.foreseen) != 1 || foreseen.foreseen[0] != "keeper/repository" {
t.Fatalf("foreseen: %v %v", foreseen.foreseen, err)
}
if _, held, err := open.inventory.ModuleSecretIfIssued(ctx, "laptop", "keeper", "repository"); err != nil || held {
t.Fatalf("asking ahead of the push made the secret (held %v, %v)", held, err)
}
// Within the bound: nothing at all — no urgent, no warning, nobody notified.
if got := d1(t, open); len(got) != 0 {
t.Fatalf("a machine waiting for a push raised %+v", got)
}
// Past the bound: a warning, not urgent, saying what the push will make.
before := awaitingPushBound
awaitingPushBound = -time.Minute
t.Cleanup(func() { awaitingPushBound = before })
got := d1(t, open)
if len(got) != 1 || got[0].Key() != "machine.laptop.awaiting-push" || got[0].Severity != conditions.Warning ||
!strings.Contains(got[0].Summary, "keeper/repository") || !strings.Contains(got[0].Summary, "push laptop") {
t.Fatalf("a machine left un-pushed past the bound: %+v", got)
}
// Pushed: the secret is made and D1 says nothing.
pushedBy(t, open, "laptop")
if got := d1(t, open); len(got) != 0 {
t.Fatalf("a pushed machine still raised %+v", got)
}
}
// **A REAL FAILURE IS STILL URGENT**: a secret the push would be refused on is not one it will make.
// A bus credential nobody issued (issue 203) fails the push, so D1 says it — urgent, in the push's words.
func TestASecretThePushCannotMakeIsStillUncomposable(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aTalker())
if _, err := assign(ctx, open, "laptop", "talker"); err != nil {
t.Fatal(err)
}
got := d1(t, open)
if len(got) != 1 || got[0].Key() != "machine.laptop.uncomposable" || got[0].Severity != conditions.Urgent ||
!strings.Contains(got[0].Summary, "module issue talker --node laptop") {
t.Fatalf("a push that will be refused was not said urgently: %+v", got)
}
// And a machine whose composition fails for anything else, with a secret waiting beside it, is
// uncomposable for that — the stand-in hides nothing.
register(t, open, aKeeper())
if _, err := assign(ctx, open, "anchor", "keeper"); err != nil {
t.Fatal(err)
}
one, two := rivals()
register(t, open, one)
register(t, open, two)
_, _ = assign(ctx, open, "anchor", "rival-one")
_, _ = assign(ctx, open, "anchor", "rival-two")
keys := map[string]conditions.Severity{}
for _, o := range d1(t, open) {
keys[o.Key()] = o.Severity
}
if keys["machine.anchor.uncomposable"] != conditions.Urgent {
t.Fatalf("a real failure beside a waiting secret: %v", keys)
}
}
// A given secret sealed to a key the machine no longer has is not the mesh's to make again: the push is
// refused on it, so D1 is too.
func TestAGivenSecretUnderAnOldKeyIsNotForeseen(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
if err := open.inventory.AcceptSecretForModule(ctx, "laptop", "keeper", "repository", "given"); err != nil {
t.Fatal(err)
}
record, err := open.inventory.NodeByName(ctx, "laptop")
if err != nil {
t.Fatal(err)
}
if err := open.inventory.RecordSealingKey(ctx, record.ID, aPublicKey(t)); err != nil {
t.Fatal(err)
}
got := d1(t, open)
if len(got) != 1 || got[0].Key() != "machine.laptop.uncomposable" || !strings.Contains(got[0].Summary, "issue it again") {
t.Fatalf("a given secret under an old key: %+v", got)
}
}
// The bound is read from when the machine began waiting: the oldest assignment since its last send.
func TestAMachineAwaitsAPushSinceItsOldestAssignmentSinceTheLastSend(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
pushedBy(t, open, "laptop")
sent := time.Now()
since, err := open.inventory.AwaitingSince(ctx, "laptop")
if err != nil || since.After(sent) {
t.Fatalf("nothing assigned since the send: waiting since %v (%v), the send was before %v", since, err, sent)
}
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
since, err = open.inventory.AwaitingSince(ctx, "laptop")
if err != nil || since.Before(sent.Add(-time.Second)) {
t.Fatalf("assigned after the send: waiting since %v (%v), not from the assignment", since, err)
}
}
// **D3 AND D13 WAIT FOR THE SEND**: a holder is expected to answer on a machine once the machine was sent
// it and had time to report — never between assign and push.
@@ -242,40 +113,3 @@ func TestAHolderIsExpectedOnlyOnceSentAndReported(t *testing.T) {
}
// And through the stores: assigned, not in the last send; pushed and reported, carried and settled.
func TestALastSendIsReadFromTheSendAndTheReport(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
pushedBy(t, open, "laptop")
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
sends, err := readDeliveries(ctx, open.inventory)
if err != nil {
t.Fatal(err)
}
if sends["laptop"].settled("keeper", time.Now()) {
t.Fatal("a holder assigned and not pushed is expected to answer")
}
if !sends["laptop"].settled(overlay.Name, time.Now().Add(time.Hour)) {
t.Fatal("a module the machine was sent long ago is not expected to answer")
}
pushedBy(t, open, "laptop")
sends, err = readDeliveries(ctx, open.inventory)
if err != nil {
t.Fatal(err)
}
if sends["laptop"].settled("keeper", time.Now()) {
t.Fatal("pushed a moment ago and not reported, a holder is already expected")
}
if !sends["laptop"].settled("keeper", time.Now().Add(reportGrace+time.Minute)) {
t.Fatal("pushed past the grace, a holder is not expected")
}
if _, ok := sends["laptop"].carried["keeper"]; !ok {
t.Fatalf("the send's builds do not carry keeper: %v", sends["laptop"].carried)
}
pushedAndApplied(t, open, "laptop")
if sends, err = readDeliveries(ctx, open.inventory); err != nil || !sends["laptop"].settled("keeper", time.Now()) {
t.Fatalf("pushed and reported applied, a holder is not expected to answer (%v)", err)
}
}
@@ -0,0 +1,219 @@
//go:build integration
package main
import (
"encoding/json"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// aBackupSeatOnTheBus is one machine's node-backup holder as a test makes it: it counts the `now` asks it hears and
// answers `backed-up` with the last night it is told (novox/hq ADR 0404).
type aBackupSeatOnTheBus struct {
mu sync.Mutex
nows []string
night map[string]any // module → lastNight
}
func (f *aBackupSeatOnTheBus) serve(t *testing.T, conn *nats.Conn, node string) {
t.Helper()
reply := func(m *nats.Msg, result any) {
body, _ := json.Marshal(map[string]any{"result": result, "error": "", "node": node})
_ = m.Respond(body)
}
now, err := conn.Subscribe(link.NodeSeatToolSubject(catalogue.BackupSeat, "now", node), func(m *nats.Msg) {
var a struct {
Module string `json:"module"`
}
_ = json.Unmarshal(m.Data, &a)
f.mu.Lock()
f.nows = append(f.nows, a.Module)
f.mu.Unlock()
reply(m, map[string]any{"started": a.Module})
})
if err != nil {
t.Fatal(err)
}
read, err := conn.Subscribe(link.NodeSeatToolSubject(catalogue.BackupSeat, "backed-up", node), func(m *nats.Msg) {
f.mu.Lock()
defer f.mu.Unlock()
var out []map[string]any
for module, n := range f.night {
out = append(out, map[string]any{"module": module, "lastNight": n})
}
reply(m, out)
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = now.Unsubscribe(); _ = read.Unsubscribe() })
if err := conn.Flush(); err != nil {
t.Fatal(err)
}
}
func (f *aBackupSeatOnTheBus) answers(module string, ok bool, words string) {
f.mu.Lock()
defer f.mu.Unlock()
f.night = map[string]any{module: map[string]any{"ok": ok, "at": time.Now().UTC().Add(time.Second), "error": words}}
}
func (f *aBackupSeatOnTheBus) asks() int {
f.mu.Lock()
defer f.mu.Unlock()
return len(f.nows)
}
// Acceptance test 3 of design 47, on the controller's side: a delivery plan that moves a module keeping data waits
// for its backups on both machines before it asks a build; a backup that fails holds it, with the machine, the
// module and the backup's words, and a condition says the same; the verb the delivery's release calls asks the
// backups again, and the plan goes on. One bus and one store for the scenarios.
func TestADeliveryThatMovesABackedUpModuleWaitsForItsBackupAndAFailedOneHoldsItAndIsSaid(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
asked := asksRecorded(t)
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
before := handActConn
handActConn = js.Conn()
t.Cleanup(func() { handActConn = before })
if _, err := inv.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "keeper", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.BackupSeat, Scope: catalogue.ScopeNode, Serves: []string{"backed-up", "now", "restore"}}}})
notes, err := catalogue.ParseManifest([]byte(`{"module":"notes","version":"1","data":{"own":[` +
`{"id":"files","path":"${dir:files}","class":"valuable","why":"people's notes"}]},` +
`"resources":[{"id":"files","type":"directory"}]}`))
if err != nil {
t.Fatal(err)
}
if err := inv.RegisterModule(ctx, notes, inventory.Source{Repository: "novox/mesh-catalog", Seat: "git",
Path: "modules/notes", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
for _, node := range []string{"anchor", "laptop"} {
if _, err := assign(ctx, open, node, "keeper"); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, node, "notes"); err != nil {
t.Fatal(err)
}
}
seats := map[string]*aBackupSeatOnTheBus{"anchor": {}, "laptop": {}}
for node, s := range seats {
s.serve(t, js.Conn(), node)
}
// Short waits: the bound and the look are variables, so nothing here waits for the minutes of a real backup.
wasWithin, wasEvery := backupsWithin, backupsPollEvery
backupsWithin, backupsPollEvery = time.Minute, 10*time.Millisecond
t.Cleanup(func() { backupsWithin, backupsPollEvery = wasWithin, wasEvery })
newPlan := func(id string) {
t.Helper()
p := inventory.Plan{ID: id, Repository: "novox/mesh-catalog", Branch: "main", Commit: "c1", Created: time.Now().UTC(),
State: inventory.PlanBuilding, Tiers: [][]string{{"notes"}}, Modules: map[string]*inventory.PlanModule{"notes": {}}}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
}
read := func(id string) inventory.Plan {
t.Helper()
p, err := inv.PlanByID(ctx, id)
if err != nil {
t.Fatal(err)
}
return p
}
// 1. Both machines are asked; while one has not answered, no build is asked.
newPlan("plan-one")
advancePlans(ctx, open)
if seats["anchor"].asks() != 1 || seats["laptop"].asks() != 1 || len(*asked) != 0 {
t.Fatalf("asks anchor=%d laptop=%d builds=%v", seats["anchor"].asks(), seats["laptop"].asks(), *asked)
}
seats["anchor"].answers("notes", true, "")
advancePlans(ctx, open)
if len(*asked) != 0 || read("plan-one").Backups.State != inventory.BackupPending {
t.Fatalf("a tier was asked while a backup was pending: %v %+v", *asked, read("plan-one").Backups)
}
seats["laptop"].answers("notes", true, "")
advancePlans(ctx, open)
if len(*asked) != 1 || read("plan-one").Backups.State != inventory.BackupDone {
t.Fatalf("the tier was not asked once every backup was done: %v %+v", *asked, read("plan-one").Backups)
}
finish := read("plan-one")
finish.State = inventory.PlanDone
if err := inv.SavePlan(ctx, &finish); err != nil {
t.Fatal(err)
}
// 2. A backup that fails holds the plan: not failed, nothing asked, the words kept, a condition raised.
seats["anchor"].night, seats["laptop"].night = nil, nil
newPlan("plan-two")
advancePlans(ctx, open)
seats["anchor"].answers("notes", true, "")
seats["laptop"].answers("notes", false, "no space left on device")
advancePlans(ctx, open)
held := read("plan-two")
if held.State != inventory.PlanBuilding || held.Backups.State != inventory.BackupFailed || len(*asked) != 1 {
t.Fatalf("the plan was not held: %s %+v builds=%v", held.State, held.Backups, *asked)
}
var failed *inventory.PlanBackup
for i := range held.Backups.Asked {
if held.Backups.Asked[i].State == inventory.BackupFailed {
failed = &held.Backups.Asked[i]
}
}
if failed == nil || failed.Machine != "laptop" || failed.Module != "notes" || failed.Words != "no space left on device" ||
strings.Contains(held.Note, "tried again") {
t.Fatalf("the failed backup: %+v note=%q", failed, held.Note)
}
advancePlans(ctx, open) // not retried on the next look
if again := read("plan-two"); again.Backups.State != inventory.BackupFailed || len(*asked) != 1 ||
seats["laptop"].asks() != 2 {
t.Fatalf("a held plan was asked again by itself: %+v builds=%v asks=%d", again.Backups, *asked, seats["laptop"].asks())
}
said, err := probeBackupHeld(ctx, &doctor{open: open})
if err != nil || len(said) != 1 || said[0].Kind != kindBackupHeld || !strings.Contains(said[0].Said, "no space left on device") {
t.Fatalf("the condition: %+v %v", said, err)
}
// 3. The verb the release calls: the failed backup is asked again, and only it, and the plan goes on.
seats["laptop"].answers("notes", false, "no space left on device")
if err := deliveryCommand(ctx, []string{"backups-again", "plan-two", "--why", "a person released it", "--by",
"mesh-delivery for jochen"}); err != nil {
t.Fatal(err)
}
if err := deliveryCommand(ctx, []string{"backups-again", "plan-two", "--why", "again"}); err == nil {
t.Fatal("a plan with no failed backup was asked again")
}
seats["laptop"].night = nil
advancePlans(ctx, open)
if seats["laptop"].asks() != 3 || seats["anchor"].asks() != 2 {
t.Fatalf("asked again: laptop=%d anchor=%d", seats["laptop"].asks(), seats["anchor"].asks())
}
seats["laptop"].answers("notes", true, "")
advancePlans(ctx, open)
if got := read("plan-two"); got.Backups.State != inventory.BackupDone || len(*asked) != 2 {
t.Fatalf("the plan did not go on: %+v builds=%v", got.Backups, *asked)
}
if said, _ := probeBackupHeld(ctx, &doctor{open: open}); len(said) != 0 {
t.Fatalf("the condition stays: %+v", said)
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,698 @@
//go:build integration
package main
import (
"bytes"
"io"
"os"
"slices"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func windowed(t *testing.T) *stores {
t.Helper()
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
settings := t.TempDir() + "/merge-window.json"
if err := os.WriteFile(settings, []byte(`{"merge-window": "90s", "merge-window-at-most": "10m"}`), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv(mergeWindowFileVar, settings)
noGroups(t)
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Seat: "git", Ref: "main", BuiltFrom: "c0",
Head: "c0"}); err != nil {
t.Fatal(err)
}
for _, m := range []struct{ module, repository, path string }{
{"app", "novox/mesh-catalog", "modules/app"},
{"notes", "novox/mesh-catalog", "modules/notes"},
{"one", "novox/one", ""},
{"two", "novox/two", ""},
{"three", "novox/three", ""},
} {
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: m.module, Version: "1"},
inventory.Source{Repository: m.repository, Seat: "git", Path: m.path, Ref: "main", BuiltFrom: "c0",
Head: "c0"}); err != nil {
t.Fatal(err)
}
}
return open
}
func hear(t *testing.T, open *stores, m link.SourceMoved, now time.Time) {
t.Helper()
if err := (following{open: open}).hearMerge(t.Context(), m, now); err != nil {
t.Fatal(err)
}
}
func cutAt(t *testing.T, open *stores, now time.Time) {
t.Helper()
if err := cutBatchesHeld(t.Context(), open, now); err != nil {
t.Fatal(err)
}
}
func TestTwoMergesSecondsApartAreOneWalkAtTheLaterCommit(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksWithPaths(t)
claude := catalogueMerge("553b7191claude", "app", t0)
dunst := catalogueMerge("48bda475dunst", "notes", t0.Add(17*time.Second))
hear(t, open, claude, t0.Add(time.Second))
hear(t, open, dunst, t0.Add(18*time.Second))
if ws, bs := walks(t, open); len(ws) != 0 || len(bs) != 1 {
t.Fatalf("within the window: %d walk(s), %d batch(es)", len(ws), len(bs))
}
cutAt(t, open, t0.Add(18*time.Second+89*time.Second))
if ws, _ := walks(t, open); len(ws) != 0 {
t.Fatalf("cut before the window closed: %+v", ws)
}
cutAt(t, open, t0.Add(18*time.Second+90*time.Second))
ws, bs := walks(t, open)
if len(ws) != 1 || len(bs) != 0 {
t.Fatalf("after the window: %d walk(s), %d batch(es)", len(ws), len(bs))
}
w := ws[0]
if w.Commit != dunst.Commit || len(w.Commits) != 1 || w.Commits[0].Commit != dunst.Commit {
t.Fatalf("the walk is at %s %+v, not the later commit", w.Commit, w.Commits)
}
for _, m := range []string{"app", "notes"} {
if _, in := w.Modules[m]; !in {
t.Fatalf("the walk does not build %s, which one of its merges moved: %v", m, w.Modules)
}
}
want := []inventory.PlanMerge{{Repository: "novox/mesh-catalog", Commit: claude.Commit, Carried: dunst.Commit},
{Repository: "novox/mesh-catalog", Commit: dunst.Commit}}
if w.Delivery == nil || !slices.EqualFunc(w.Delivery.Merges, want, sameMerge) {
t.Fatalf("the walk answers %+v, want %+v", w.Delivery, want)
}
if len(*asked) != 2 || (*asked)[0][2] != "main" || (*asked)[1][2] != "main" {
t.Fatalf("the walk did not ask its modules at the branch, which holds the commit it carries: %v", *asked)
}
// Heard again, as the bus may hand it over twice: never a second merge, nor a second walk.
hear(t, open, claude, t0.Add(5*time.Minute))
if ws, bs := walks(t, open); len(ws) != 1 || len(bs) != 0 {
t.Fatalf("a merge heard twice made %d walk(s) and %d batch(es)", len(ws), len(bs))
}
}
func TestThreeRepositoriesInOneWindowAreOneWalk(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "feat/three")
asksRecorded(t)
for i, r := range []string{"one", "two", "three"} {
hear(t, open, fromHead(repoMerge(r, "c-"+r, t0.Add(time.Duration(i)*20*time.Second)), "feat/three"),
t0.Add(time.Duration(i)*20*time.Second+time.Second))
}
cutAt(t, open, t0.Add(41*time.Second+90*time.Second))
ws, bs := walks(t, open)
if len(ws) != 1 || len(bs) != 0 {
t.Fatalf("%d walk(s), %d batch(es)", len(ws), len(bs))
}
w := ws[0]
if len(w.Commits) != 3 || len(w.Delivery.Merges) != 3 {
t.Fatalf("the walk carries %+v and answers %+v", w.Commits, w.Delivery.Merges)
}
for _, r := range []string{"one", "two", "three"} {
if w.CommitOf("novox/"+r) != "c-"+r {
t.Fatalf("the walk carries %s at %q", r, w.CommitOf("novox/"+r))
}
if _, in := w.Modules[r]; !in {
t.Fatalf("the walk does not build %s: %v", r, w.Modules)
}
}
// Each module's ask is recorded at its own repository's commit.
for _, r := range []string{"one", "two", "three"} {
q, found, err := open.inventory.BuildRequestByID(t.Context(), w.Modules[r].Build)
if err != nil || !found || q.Commit != "c-"+r {
t.Fatalf("%s's build was not recorded at its own commit: %+v %v %v", r, q, found, err)
}
}
}
func TestAMergeRestartsTheWindowAndTheMaximumClosesIt(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "feat/both", "feat/busy")
asksWithPaths(t)
hear(t, open, fromHead(repoMerge("one", "c1", t0), "feat/both"), t0)
hear(t, open, fromHead(repoMerge("two", "c2", t0.Add(80*time.Second)), "feat/both"), t0.Add(80*time.Second))
cutAt(t, open, t0.Add(100*time.Second))
if ws, _ := walks(t, open); len(ws) != 0 {
t.Fatal("the window closed 90 seconds after its first merge, not after its last")
}
cutAt(t, open, t0.Add(170*time.Second))
if ws, _ := walks(t, open); len(ws) != 1 || len(ws[0].Commits) != 2 {
t.Fatalf("the window did not close 90 seconds after its last merge: %+v", ws)
}
busy := windowed(t)
groupsWaiting(t, "feat/both", "feat/busy")
asksWithPaths(t)
var last time.Time
for i := 0; i < 12; i++ {
last = t0.Add(time.Duration(i) * time.Minute)
repo := []string{"one", "two", "three"}[i%3]
hear(t, busy, fromHead(repoMerge(repo, "c"+string(rune('a'+i)), last), "feat/busy"), last)
if ws, _ := walks(t, busy); len(ws) > 0 {
if i != 10 {
t.Fatalf("a busy window closed at merge %d (minute %d), not at its maximum", i, i)
}
break
}
}
ws, _ := walks(t, busy)
if len(ws) != 1 {
t.Fatalf("ten minutes of merges a minute apart were never cut: %d walk(s)", len(ws))
}
}
func TestTheAssemblingBatchIsShown(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/553b7191claude")
asksWithPaths(t)
var said []inventory.Plan
was := inventory.PlanSaved
inventory.PlanSaved = func(_ *inventory.Inventory, p inventory.Plan) { said = append(said, p) }
t.Cleanup(func() { inventory.PlanSaved = was })
now := time.Now().UTC()
claude := catalogueMerge("553b7191claude", "app", now.Add(-20*time.Second))
claude.Number, claude.Title = 174, "claude-code: an agent proposes a section"
dunst := catalogueMerge("48bda475dunst", "notes", now.Add(-2*time.Second))
dunst.Number, dunst.Title = 175, "dunst: the font the operator chose"
hear(t, open, claude, now.Add(-19*time.Second))
hear(t, open, dunst, now.Add(-time.Second))
hear(t, open, fromHead(repoMerge("one", "a6bc0931one", now), dunst.Head), now)
out := captured(t, func() error { return plansCommand(t.Context(), nil) })
lines := strings.Split(out, "\n")
first := lines[0]
for _, want := range []string{"assembling: ", " s left (at the latest ", "grouped: novox/mesh-catalog@48bda475 " +
"(answers 553b7191), novox/one@a6bc0931; plan not yet calculated"} {
if !strings.Contains(first, want) {
t.Fatalf("plans' first line %q does not say %q", first, want)
}
}
// Under it, one line per repository: the pull request, what it answers, what it moves.
if len(lines) < 3 || strings.TrimSpace(lines[1]) != "mesh-catalog #175 dunst: the font the operator chose (answers "+
"#174 claude-code: an agent proposes a section) · app, notes" ||
strings.TrimSpace(lines[2]) != "one a6bc0931 · one" {
t.Fatalf("the grouped list reads %q", lines[1:4])
}
if len(said) == 0 || said[len(said)-1].State != inventory.PlanAssembling || said[len(said)-1].Delivery.Batch == nil ||
len(said[len(said)-1].Delivery.Merges) != 3 {
t.Fatalf("the batch was not said as assembling with its merges: %+v", said)
}
}
func captured(t *testing.T, run func() error) string {
t.Helper()
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
was := os.Stdout
os.Stdout = w
runErr := run()
os.Stdout = was
_ = w.Close()
var b bytes.Buffer
_, _ = io.Copy(&b, r)
if runErr != nil {
t.Fatal(runErr)
}
return b.String()
}
func TestAMergeDuringAWalkJoinsTheNextBatch(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
ws, _ := walks(t, open)
if len(ws) != 1 || !ws[0].Open() {
t.Fatalf("the first batch was not cut: %+v", ws)
}
first := ws[0]
hear(t, open, repoMerge("one", "c2", t0.Add(2*time.Minute)), t0.Add(2*time.Minute))
cutAt(t, open, t0.Add(5*time.Minute))
ws, bs := walks(t, open)
if len(ws) != 1 || len(bs) != 1 || bs[0].State != inventory.PlanQueued || bs[0].Delivery.Batch.Behind != first.ID {
t.Fatalf("the merge during the walk: %d walk(s), batches %+v", len(ws), bs)
}
if got, _ := open.inventory.PlanByID(t.Context(), first.ID); !got.Open() {
t.Fatalf("the started walk was %s by the next merge", got.State)
}
if !strings.HasPrefix(batchWords(bs[0], t0.Add(5*time.Minute)), "queued behind "+first.ID) {
t.Fatalf("a queued batch reads %q", batchWords(bs[0], t0.Add(5*time.Minute)))
}
first.State = inventory.PlanDone
if err := open.inventory.SavePlan(t.Context(), &first); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(6*time.Minute))
ws, bs = walks(t, open)
if len(ws) != 2 || len(bs) != 0 || ws[0].Commit != "c2" {
t.Fatalf("the queued batch was not cut when the walk ended: %+v %+v", ws, bs)
}
}
func TestAWaitingWalkIsFoldedIntoTheNextBatch(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
ctx := t.Context()
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-delivery", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}},
inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/mesh-delivery", Ref: "main",
BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
if _, err := open.inventory.Assign(ctx, "anchor", "mesh-delivery"); err != nil {
t.Fatal(err)
}
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
ws, _ := walks(t, open)
if len(ws) != 1 || !ws[0].Waiting() {
t.Fatalf("the walk does not wait for its word: %+v", ws)
}
waiting := ws[0]
hear(t, open, repoMerge("one", "c2", t0.Add(2*time.Minute)), t0.Add(2*time.Minute))
cutAt(t, open, t0.Add(2*time.Minute+90*time.Second))
ws, bs := walks(t, open)
if len(bs) != 0 || len(ws) != 2 {
t.Fatalf("%d walk(s), %d batch(es)", len(ws), len(bs))
}
folded, err := open.inventory.PlanByID(ctx, waiting.ID)
if err != nil {
t.Fatal(err)
}
newer := ws[0]
if folded.State != inventory.PlanSuperseded || folded.Delivery.TakenOverBy != newer.ID {
t.Fatalf("the waiting walk is %s, taken over by %q", folded.State, folded.Delivery.TakenOverBy)
}
if len(newer.Delivery.Merges) != 2 || newer.CommitOf("novox/one") != "c2" {
t.Fatalf("the newer walk answers %+v", newer.Delivery.Merges)
}
if _, in := newer.Modules["one"]; !in {
t.Fatalf("what the folded walk was to build is not built: %v", newer.Modules)
}
}
func TestALateMergeIsAnsweredByTheWalkOfTheLaterOne(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
later := catalogueMerge("48bda475dunst", "notes", t0.Add(17*time.Second))
hear(t, open, later, t0.Add(18*time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
w := ws[0]
w.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &w); err != nil {
t.Fatal(err)
}
// Moves only notes, which the done walk built from the branch after it: answered there, at once.
hear(t, open, catalogueMerge("553b7191early", "notes", t0), t0.Add(15*time.Minute))
got, _ := open.inventory.PlanByID(ctx, w.ID)
if got.State != inventory.PlanDone || len(got.Delivery.Merges) != 2 ||
!sameMerge(got.Delivery.Merges[0], inventory.PlanMerge{Repository: "novox/mesh-catalog", Commit: "553b7191early",
Carried: later.Commit}) {
t.Fatalf("the late merge is not named by the walk that carried it: %+v", got.Delivery.Merges)
}
// Moves app, which that walk never built: the next batch walks it.
hear(t, open, catalogueMerge("1111aaaaapp", "app", t0.Add(time.Second)), t0.Add(16*time.Minute))
_, bs := walks(t, open)
if len(bs) != 1 || bs[0].Delivery.Merges[0].Commit != "1111aaaaapp" {
t.Fatalf("a late merge moving what the walk never built did not join the next batch: %+v", bs)
}
}
func TestAFailedWalkWalksItsEarlierMergesAlone(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksWithPaths(t)
ctx := t.Context()
// Made before the cut marks their modules seen, as merges are: walked again, they are news all the same.
oldest := catalogueMerge("aaaa0001", "app", t0.Add(-3*time.Minute))
middle := catalogueMerge("bbbb0002", "app", t0.Add(-3*time.Minute+10*time.Second))
newest := catalogueMerge("cccc0003", "notes", t0.Add(-3*time.Minute+20*time.Second))
for i, m := range []link.SourceMoved{oldest, middle, newest} {
hear(t, open, m, t0.Add(time.Duration(i)*10*time.Second+time.Second))
}
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
failed := ws[0]
failed.State, failed.Note = inventory.PlanFailed, "notes failed to build in tier 0"
if err := open.inventory.SavePlan(ctx, &failed); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(3*time.Minute))
got, _ := open.inventory.PlanByID(ctx, failed.ID)
if got.State != inventory.PlanFailed || len(got.Delivery.Merges) != 3 {
t.Fatalf("the failed walk's record changed: %s %+v", got.State, got.Delivery.Merges)
}
ws, _ = walks(t, open)
alone := ws[0]
if alone.ID == failed.ID || alone.Commit != middle.Commit || len(alone.Delivery.Merges) != 1 ||
!sameMerge(alone.Delivery.Merges[0], inventory.PlanMerge{Repository: "novox/mesh-catalog", Commit: middle.Commit}) {
t.Fatalf("the newest earlier merge was not walked alone on its own commit: %+v", alone)
}
if _, in := alone.Modules["app"]; !in || (*asked)[len(*asked)-1] != [3]string{"novox/mesh-catalog", "modules/app",
middle.Commit} {
t.Fatalf("the merge walked alone does not build app at its own commit: %v, asked %v", alone.Modules, *asked)
}
// Retried, the failed walk would name merges the search answers now.
if _, err := retryPlan(ctx, open, failed.ID); err == nil || !strings.Contains(err.Error(), "walked alone") {
t.Fatalf("a failed walk whose merges are searched was retried: %v", err)
}
alone.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &alone); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(4*time.Minute))
ws, bs := walks(t, open)
if ws[0].ID != alone.ID || len(bs) != 0 {
t.Fatalf("the search went on after a merge was delivered: %+v", ws[0])
}
done, _ := open.inventory.PlanByID(ctx, alone.ID)
if len(done.Delivery.Merges) != 2 || !sameMerge(done.Delivery.Merges[0], inventory.PlanMerge{Repository: "novox/mesh-catalog",
Commit: oldest.Commit, Carried: middle.Commit}) {
t.Fatalf("the oldest merge is not answered by the walk that delivered the one after it: %+v", done.Delivery.Merges)
}
// A stopped walk starts no search: a person ended it.
hear(t, open, catalogueMerge("dddd0004", "app", t0.Add(5*time.Minute)), t0.Add(5*time.Minute))
hear(t, open, catalogueMerge("eeee0005", "notes", t0.Add(5*time.Minute+time.Second)), t0.Add(5*time.Minute+time.Second))
cutAt(t, open, t0.Add(8*time.Minute))
ws, _ = walks(t, open)
if _, err := stopWalk(ctx, open.inventory, ws[0].ID, "mesh-delivery for jochen", "not now"); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(9*time.Minute))
if again, _ := walks(t, open); again[0].ID != ws[0].ID {
t.Fatalf("a stopped walk's earlier merge was walked alone: %+v", again[0])
}
}
func TestAGroupsOrderBecomesTiersInsideTheWalk(t *testing.T) {
for _, grouped := range []bool{true, false} {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
for _, m := range []string{"mesh-host"} {
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: m, Version: "1"},
inventory.Source{Repository: "novox/" + m, Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
}
host := repoMerge("mesh-host", "h1", t0)
ctl := repoMerge("mesh-controller", "k1", t0.Add(time.Second))
host.Head, ctl.Head = "feat/together", "feat/together"
if !grouped {
ctl.Head = "feat/alone"
}
hear(t, open, ctl, t0.Add(2*time.Second))
hear(t, open, host, t0.Add(3*time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
if len(ws) != 1 {
t.Fatalf("%d walks", len(ws))
}
tiers := ws[0].Tiers
if grouped {
if len(tiers) != 2 || !slices.Equal(tiers[0], []string{"mesh-host"}) ||
!slices.Equal(tiers[1], []string{"mesh-controller"}) {
t.Fatalf("the group's order is not the walk's tiers: %v", tiers)
}
} else if len(tiers) != 1 {
t.Fatalf("two merges of no group were ordered: %v", tiers)
}
}
}
func TestABatchSurvivesARestart(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "feat/both")
asksWithPaths(t)
hear(t, open, fromHead(repoMerge("one", "c1", t0), "feat/both"), t0)
hear(t, open, fromHead(repoMerge("two", "c2", t0.Add(30*time.Second)), "feat/both"), t0.Add(30*time.Second))
// A new controller: nothing of the first one's but the store.
again, err := openStores(t.Context())
if err != nil {
t.Fatal(err)
}
t.Cleanup(again.Close)
hear(t, again, fromHead(repoMerge("one", "c1", t0), "feat/both"), t0.Add(60*time.Second)) // the bus hands it over again
cutAt(t, again, t0.Add(119*time.Second))
if ws, bs := walks(t, again); len(ws) != 0 || len(bs) != 1 || len(bs[0].Delivery.Merges) != 2 {
t.Fatalf("the restarted controller's batch: %d walk(s), %+v", len(ws), bs)
}
cutAt(t, again, t0.Add(120*time.Second))
ws, bs := walks(t, again)
if len(ws) != 1 || len(bs) != 0 || len(ws[0].Delivery.Merges) != 2 {
t.Fatalf("the restarted controller did not cut the batch at its window: %+v %+v", ws, bs)
}
}
func TestALateMergeOfAOneModuleRepositoryIsNamed(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
c2 := repoMerge("one", "c2", t0.Add(-50*time.Second))
hear(t, open, c2, t0)
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
w := ws[0]
w.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &w); err != nil {
t.Fatal(err)
}
hear(t, open, repoMerge("one", "c1", t0.Add(-60*time.Second)), t0.Add(15*time.Minute))
got, _ := open.inventory.PlanByID(ctx, w.ID)
if len(got.Delivery.Merges) != 2 || !sameMerge(got.Delivery.Merges[0], inventory.PlanMerge{Repository: "novox/one",
Commit: "c1", Carried: "c2"}) {
t.Fatalf("the late merge was not named by the walk that carried it: %+v", got.Delivery.Merges)
}
// A walk of two that built only two: a late merge of one is not its to name.
hear(t, open, repoMerge("two", "d2", t0.Add(16*time.Minute)), t0.Add(16*time.Minute))
cutAt(t, open, t0.Add(18*time.Minute))
ws, _ = walks(t, open)
w = ws[0]
delete(w.Modules, "two")
w.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &w); err != nil {
t.Fatal(err)
}
hear(t, open, repoMerge("two", "d1", t0.Add(15*time.Minute)), t0.Add(20*time.Minute))
if got, _ := open.inventory.PlanByID(ctx, w.ID); len(got.Delivery.Merges) != 1 {
t.Fatalf("a walk that never built what the late merge moves named it: %+v", got.Delivery.Merges)
}
if _, bs := walks(t, open); len(bs) != 1 || bs[0].Delivery.Merges[0].Commit != "d1" {
t.Fatalf("the late merge did not join the next batch: %+v", bs)
}
}
func TestAWalkLetGoBesideAStartedOneStartsOnceItEnded(t *testing.T) {
open := windowed(t)
asked := asksWithPaths(t)
ctx := t.Context()
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(2*time.Minute))
started, _ := walks(t, open)
waiting := inventory.Plan{ID: "plan-waiting", Repository: "novox/one", Branch: "main", Commit: "d1", Created: t0,
State: inventory.PlanBuilding, Tiers: [][]string{{"one"}}, Modules: map[string]*inventory.PlanModule{"one": {}},
Delivery: &inventory.PlanDelivery{Awaits: catalogue.DeliverySeat}}
if err := open.inventory.SavePlan(ctx, &waiting); err != nil {
t.Fatal(err)
}
if _, err := letGo(ctx, open.inventory, waiting.ID, catalogue.DeliverySeat, "its turn"); err != nil {
t.Fatalf("the word was refused beside %s: %v", started[0].ID, err)
}
before := len(*asked)
advanceHeld(ctx, open)
got, _ := open.inventory.PlanByID(ctx, waiting.ID)
if len(*asked) != before || !strings.Contains(got.Note, "starts once "+started[0].ID) {
t.Fatalf("a walk let go beside a started one asked (%d → %d) or does not say it waits: %q", before, len(*asked), got.Note)
}
// Read as a wait, an hour on: its note on the line, never LATE, and no tier of it late for S3.
later := time.Now().Add(time.Hour)
if line := planLine(got, later); !strings.Contains(line, "starts once "+started[0].ID) || strings.Contains(line, "LATE") {
t.Fatalf("a deferred walk reads %q", line)
}
facts, _, err := gatherPlans(ctx, open.inventory, later, nil)
if err != nil {
t.Fatal(err)
}
for _, f := range facts {
if f.id == got.ID {
t.Fatalf("a deferred walk is watched as a tier running late: %+v", f)
}
}
done := started[0]
done.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &done); err != nil {
t.Fatal(err)
}
advanceHeld(ctx, open)
if len(*asked) != before+1 {
t.Fatalf("the walk did not start once the started one ended: asked %v", *asked)
}
}
func TestAMergeOnTheControllersPathNeverSharesABatch(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksWithPaths(t)
ctx := t.Context()
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-delivery", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}},
inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/mesh-delivery", Ref: "main",
BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
if _, err := open.inventory.Assign(ctx, "anchor", "mesh-delivery"); err != nil {
t.Fatal(err)
}
hear(t, open, catalogueMerge("aaaa0001", "app", t0), t0)
hear(t, open, repoMerge("mesh-controller", "k1", t0.Add(10*time.Second)), t0.Add(10*time.Second))
hear(t, open, catalogueMerge("bbbb0002", "notes", t0.Add(20*time.Second)), t0.Add(20*time.Second))
_, bs := walks(t, open)
var ownBatch, catalogueBatch inventory.Plan
for _, b := range bs {
if b.OwnPath() {
ownBatch = b
} else {
catalogueBatch = b
}
}
if len(bs) != 2 || ownBatch.ID == "" || catalogueBatch.ID == "" || len(catalogueBatch.Delivery.Merges) != 2 ||
len(ownBatch.Delivery.Merges) != 1 {
t.Fatalf("a controller merge and two catalogue merges in one window: %+v", bs)
}
if !strings.Contains(batchWords(ownBatch, t0.Add(30*time.Second)), "own path") {
t.Fatalf("the own-path batch does not say so: %q", batchWords(ownBatch, t0.Add(30*time.Second)))
}
// Both windows closed, the controller's batch is cut first and starts; the catalogue batch queues behind it,
// is cut when the controller's walk ended, and waits for its word with both merges.
cutAt(t, open, t0.Add(2*time.Minute))
ws, bs := walks(t, open)
if len(ws) != 1 || ws[0].Waiting() || ws[0].CommitOf("novox/mesh-controller") != "k1" {
t.Fatalf("the controller's batch was not cut first into a started walk: %+v", ws)
}
ownWalk := ws[0]
for _, m := range []string{"app", "notes"} {
if _, in := ownWalk.Modules[m]; in {
t.Fatalf("the controller's walk builds %s, a catalogue module: it skipped its turn", m)
}
}
if len(bs) != 1 || bs[0].State != inventory.PlanQueued || bs[0].Delivery.Batch.Behind != ownWalk.ID || bs[0].OwnPath() {
t.Fatalf("the catalogue batch does not queue behind the controller's walk: %+v", bs)
}
ownWalk.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &ownWalk); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(3*time.Minute))
ws, bs = walks(t, open)
if len(bs) != 0 || ws[0].ID != catalogueBatch.ID || !ws[0].Waiting() || len(ws[0].Delivery.Merges) != 2 {
t.Fatalf("the catalogue batch was not cut into a waiting walk once the controller's ended: %+v %+v", ws, bs)
}
for _, m := range []string{"app", "notes"} {
if _, in := ws[0].Modules[m]; !in {
t.Fatalf("the catalogue walk does not build %s: %v", m, ws[0].Modules)
}
}
for _, a := range *asked {
if a[1] == "modules/app" || a[1] == "modules/notes" {
t.Fatalf("a catalogue module was asked without the word: %v", *asked)
}
}
// A catalogue walk waiting for its word when an own-path batch is cut keeps waiting beside the own-path
// walk, is not folded into it, and its word is taken meanwhile: it starts once the own-path walk ended.
catalogueWalk := ws[0]
hear(t, open, repoMerge("mesh-controller", "k2", t0.Add(4*time.Minute)), t0.Add(4*time.Minute))
cutAt(t, open, t0.Add(6*time.Minute))
ws, _ = walks(t, open)
kept, _ := open.inventory.PlanByID(ctx, catalogueWalk.ID)
if !kept.Waiting() || ws[0].CommitOf("novox/mesh-controller") != "k2" || ws[0].Waiting() {
t.Fatalf("the waiting catalogue walk was not kept waiting beside the controller's walk: %s %+v", kept.State, ws)
}
if _, in := ws[0].Modules["app"]; in {
t.Fatalf("the controller's walk folded the waiting catalogue walk in: %v", ws[0].Modules)
}
if _, err := letGo(ctx, open.inventory, catalogueWalk.ID, catalogue.DeliverySeat, "its turn"); err != nil {
t.Fatal(err)
}
before := len(*asked)
advanceHeld(ctx, open)
if len(*asked) != before {
t.Fatalf("the catalogue walk started beside the controller's: asked %v", (*asked)[before:])
}
own2 := ws[0]
own2.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &own2); err != nil {
t.Fatal(err)
}
advanceHeld(ctx, open)
if len(*asked) < before+1 || (*asked)[before][1] != "modules/app" {
t.Fatalf("the catalogue walk did not start once the controller's ended: %v", (*asked)[before:])
}
}
func TestALateCatalogueMergeIsNotAnsweredByAnOwnPathWalk(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
for _, m := range []struct {
module string
claims []catalogue.Claim
}{
{"nats", nil},
{"mesh-delivery", []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}},
} {
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: m.module, Version: "1", Claims: m.claims},
inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/" + m.module, Ref: "main",
BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
}
if _, err := open.inventory.Assign(ctx, "anchor", "mesh-delivery"); err != nil {
t.Fatal(err)
}
mixed := link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main", Commit: "m1xed", MergedAt: t0.Format(time.RFC3339Nano),
Paths: []string{"modules/nats/module.json", "modules/app/module.json"}, ModuleDirs: []string{"modules/nats", "modules/app"},
ModuleDirsSaid: true}
hear(t, open, mixed, t0.Add(time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
if len(ws) != 1 || ws[0].Waiting() {
t.Fatalf("the mixed merge's walk waited, or was not cut: %+v", ws)
}
done := ws[0]
done.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &done); err != nil {
t.Fatal(err)
}
hear(t, open, catalogueMerge("ear1ier", "app", t0.Add(-30*time.Second)), t0.Add(3*time.Minute))
got, _ := open.inventory.PlanByID(ctx, done.ID)
_, bs := walks(t, open)
if len(got.Delivery.Merges) != 1 || len(bs) != 1 || bs[0].OwnPath() || bs[0].Delivery.Merges[0].Commit != "ear1ier" {
t.Fatalf("the late catalogue merge was answered by the own-path walk (%+v) rather than the next catalogue batch (%+v)",
got.Delivery.Merges, bs)
}
}
+293
View File
@@ -0,0 +1,293 @@
package main
import (
"context"
"encoding/json"
"errors"
"os"
"slices"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The suite's tests of a merge, but for the merge window's own (this file's), plan the merge as it is heard: a
// window of nothing closes at once, so a merge with no walk open is cut into its walk at once, as every merge
// was planned before novox/hq ADR 0276. The window's tests set it through the controller's settings.
func init() {
mergeWindowDefault = 0
// No forge in a test: no manifest is found at any commit, so a merge draws no edge of the manifests' change
// (delivery_edges.go). A test that wants some replaces these (forgeHolding).
manifestAt = func(context.Context, string, string, string, string) (catalogue.Manifest, bool, error) {
return catalogue.Manifest{}, false, nil
}
parentOf = func(context.Context, string, string, string) (string, error) { return "", nil }
}
// windowed is a mesh whose controller's settings give a merge window of 90 seconds and at most 10 minutes, as its
// settings file says them, and
// modules built from the catalogue (app, notes) and from three repositories of their own.
// t0 is the moment a test's clock starts: merges are dated by it, and heard after it.
var t0 = time.Now().UTC().Truncate(time.Second)
// catalogueMerge is a merge of the catalogue changing one module's directory, made at a moment, from a pull
// request's head branch of its own, as the forge announces every merge. Whether its head names a delivery group
// with a member still to merge is mesh-delivery's word, which windowed stubs as no group at all (noGroups); a test
// whose merges wait for the window as a group's says so (groupsWaiting).
func catalogueMerge(commit, module string, made time.Time) link.SourceMoved {
return link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main", Head: "pr/" + commit, Commit: commit,
MergedAt: made.Format(time.RFC3339Nano), Paths: []string{"modules/" + module + "/module.json"},
ModuleDirs: []string{"modules/" + module}, ModuleDirsSaid: true}
}
// repoMerge is a merge of a repository of one module's own, from a head branch of its own (catalogueMerge).
func repoMerge(repo, commit string, made time.Time) link.SourceMoved {
return link.SourceMoved{Owner: "novox", Repo: repo, Base: "main", Head: "pr/" + commit, Commit: commit,
MergedAt: made.Format(time.RFC3339Nano), Paths: []string{"main.go"}}
}
// noGroups stubs mesh-delivery's word on groups for a test: no head names a group.
func noGroups(t *testing.T) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(context.Context, bool, string) ([]groupMember, bool, error) { return nil, false, nil }
t.Cleanup(func() { deliveryGroupOf = was })
}
// groupsWaiting stubs mesh-delivery's word on groups for a test of the merge window's own timing: each head named
// is one whose group mesh-delivery cannot say, so a merge from it waits for the window (ADR 0306 decision 2) and
// nothing holds the batch open; every other head names no group. A group with a member still to merge, which holds
// its batch open however long it takes (ADR 0418 clause 3), is groupStates'.
func groupsWaiting(t *testing.T, heads ...string) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(_ context.Context, _ bool, head string) ([]groupMember, bool, error) {
if slices.Contains(heads, head) {
return nil, false, errors.New("the test's mesh-delivery does not say")
}
return nil, false, nil
}
t.Cleanup(func() { deliveryGroupOf = was })
}
// everyHeadWaits stubs mesh-delivery's word on groups for a test whose merges each wait for the window as they did
// before novox/hq ADR 0304: mesh-delivery cannot say any head's group.
func everyHeadWaits(t *testing.T) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(context.Context, bool, string) ([]groupMember, bool, error) {
return nil, false, errors.New("the test's mesh-delivery does not say")
}
t.Cleanup(func() { deliveryGroupOf = was })
}
// groupStates stubs mesh-delivery's word on one group: its members and their states, which a test changes as the
// members merge.
func groupStates(t *testing.T, head string, states map[string]string) {
t.Helper()
groupStatesFrom(t, head, states, true)
}
// groupStatesFrom is groupStates for a mesh-delivery that does (composed) or does not yet say the group's gate is the
// composed check's: only the first holds the group's batch open for its last member.
func groupStatesFrom(t *testing.T, head string, states map[string]string, composed bool) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(_ context.Context, _ bool, asked string) ([]groupMember, bool, error) {
if asked != head {
return nil, false, nil
}
var members []groupMember
for repository, state := range states {
members = append(members, groupMember{Repository: repository, State: state, Composed: composed})
}
return members, true, nil
}
t.Cleanup(func() { deliveryGroupOf = was })
}
// fromHead is a merge from a head branch a test names: merges from one head are members of one delivery group,
// related by it (novox/hq ADR 0304 decision 1), so they share a batch whatever else they share.
func fromHead(m link.SourceMoved, head string) link.SourceMoved {
m.Head = head
return m
}
// lone is a merge that names no head: known to be no delivery group's, it waits for no window unless it is
// related to a batch assembling (ADR 0304 decision 1).
func lone(m link.SourceMoved) link.SourceMoved {
m.Head = ""
return m
}
// hear is a merge heard at a moment.
// cutAt is the cutter looking at a moment.
// walks is every plan record that is a walk, newest first, and the batches not yet cut.
func walks(t *testing.T, open *stores) (walks, batches []inventory.Plan) {
t.Helper()
recent, err := open.inventory.RecentPlans(t.Context(), 50)
if err != nil {
t.Fatal(err)
}
for _, p := range recent {
if p.Batch() {
batches = append(batches, p)
} else {
walks = append(walks, p)
}
}
return walks, batches
}
// The replay of issue 362: two catalogue merges 18 seconds apart are one batch, cut once into one walk at the
// later commit, which names both merges — the earlier carried by the later.
// Merges of three repositories in one window are one walk, one commit for each.
// Each merge restarts the window: a merge at second 80 keeps the batch open until second 170; the window
// closes at most ten minutes after its first merge however busy.
// `plans` lists the batch being assembled first, in the operator's words: how long is left, when at the latest,
// what is grouped, and that the plan is not yet calculated. It says so on the bus as plan-moved, too.
// captured is what a command printed.
// A merge heard during a walk it is related to (the same repository, issue 973) joins the next batch, which is
// cut when the walk ends; the walk that started is never superseded (ADR 0276 decision 4, ADR 0407 decision 4).
// A walk waiting for its delivery's word is folded into the next batch's walk, which answers its merges; it
// names the walk that took it over.
// A merge heard after a later merge of its branch was walked is answered by that walk when it builds all it
// moves — named at once on a walk done — and joins the next batch when it does not.
// A failed walk marks nothing delivered: its earlier merges are walked alone, on their own commit, the newest
// first; the first delivered answers the older ones, and the search stops.
// A delivery group's order holds inside the walk (ADR 0249 unchanged): merges of two repositories from one head
// branch name, the node-engine's before the controller's, are tiered so; without the group, one tier.
// A restarted controller resumes the window where it stood: the batch, its merges and their times are in the
// store, a merge handed over again is not doubled, and the batch is cut when its window closes.
// The window's settings reach the controller in its settings file, read at every look; one that says no
// duration, or no file, is the default.
func TestTheWindowIsTheControllersSetting(t *testing.T) {
file := t.TempDir() + "/merge-window.json"
t.Setenv(mergeWindowFileVar, file)
for _, c := range []struct {
content string
window, atMost time.Duration
}{
{`{"merge-window": "60s", "merge-window-at-most": "5m"}`, time.Minute, 5 * time.Minute},
{`{"merge-window": 30, "merge-window-at-most": ""}`, 30 * time.Second, mergeWindowAtMostDefault},
{`{"merge-window": "soon"}`, mergeWindowDefault, mergeWindowAtMostDefault},
} {
if err := os.WriteFile(file, []byte(c.content), 0o600); err != nil {
t.Fatal(err)
}
if w, m := mergeWindowOf(); w != c.window || m != c.atMost {
t.Errorf("%s reads as %s and %s, want %s and %s", c.content, w, m, c.window, c.atMost)
}
}
t.Setenv(mergeWindowFileVar, file+".gone")
if w, m := mergeWindowOf(); w != mergeWindowDefault || m != mergeWindowAtMostDefault {
t.Errorf("no file reads as %s and %s", w, m)
}
}
// S16 names every merge a waiting walk answers, never one commit a supersession may have ended (issue 362).
func TestAWaitingWalkNamesTheMergesItAnswers(t *testing.T) {
f := calm(t0)
f.waits = []waitFacts{{id: "plan-2", repository: "novox/mesh-catalog", commit: "48bda475dunst", awaits: "mesh-delivery",
since: t0.Add(-31 * time.Minute), merges: []inventory.PlanMerge{
{Repository: "novox/mesh-catalog", Commit: "553b7191claude", Carried: "48bda475dunst"},
{Repository: "novox/mesh-catalog", Commit: "48bda475dunst"}}}}
got := watchWaits(f)
if len(got) != 1 || !strings.Contains(got[0].Summary, "novox/mesh-catalog@553b7191") ||
!strings.Contains(got[0].Summary, "novox/mesh-catalog@48bda475") {
t.Fatalf("the waiting walk does not name both merges: %+v", got)
}
}
// A merge heard after a later merge of its branch was cut is named by that walk however its modules read since
// the cut, in a repository of one module (review of this change); a walk that never built what it moves names
// nothing, and the merge joins the next batch.
// A related walk waits for the delivery's word too: a waiting walk let go beside a started one it is related to
// (the same repository, issue 973) takes the word and starts once that one ended, asking nothing before.
// A file a merge of the batch removed and a later one brought back is not removed; one removed last is.
func TestARemovedFileIsTheLastMergesWord(t *testing.T) {
ev := func(commit string, paths, removed []string) inventory.BatchedMerge {
m := link.SourceMoved{Owner: "novox", Repo: "one", Base: "main", Commit: commit, Paths: paths, Removed: removed}
b, _ := json.Marshal(m)
return inventory.BatchedMerge{Repository: "novox/one", Branch: "main", Commit: commit, Event: b,
Merged: t0.Add(time.Duration(len(commit)) * time.Second)}
}
got := combinedMerges([]inventory.BatchedMerge{
ev("a", []string{"x/module.json", "y/module.json"}, []string{"x/module.json", "y/module.json"}),
ev("bb", []string{"x/module.json"}, nil),
ev("ccc", []string{"z.go"}, nil)})
if len(got) != 1 || got[0].Commit != "ccc" || !slices.Equal(got[0].Removed, []string{"y/module.json"}) ||
len(got[0].Paths) != 3 {
t.Fatalf("combined as %+v", got)
}
}
// A merge on the controller's own path never shares a batch with one that waits for mesh-delivery's word
// (decided during the build, 2026-10-10): the two are batches of their own, cut one after the other; a walk on
// the own path folds no waiting catalogue walk but batches its merges again behind it; the catalogue's walk
// still waits for the word, so no catalogue delivery skips its turn.
// `plans` names a walk by what it moves (asked by the operator, 2026-10-10): the repository's pull request and
// title, the modules it moves and the machines running them, then the state words; a record naming no pull
// request is named by its commit, and a title is cut at fifty runes.
func TestAPlanLineNamesWhatItMoves(t *testing.T) {
now := time.Date(2026, 10, 10, 12, 0, 0, 0, time.UTC)
p := inventory.Plan{ID: "plan-1", Repository: "novox/mesh-catalog", Branch: "main", Commit: "c1c1c1c1c1",
State: inventory.PlanBuilding, Tiers: [][]string{{"messenger", "telegram"}}, TierEntered: now.Add(-2 * time.Minute),
Modules: map[string]*inventory.PlanModule{"messenger": {State: "asked"}, "telegram": {State: "asked"}},
Commits: []inventory.PlanCommit{{Repository: "novox/mesh-catalog", Branch: "main", Commit: "c1c1c1c1c1"}},
Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{{Repository: "novox/mesh-catalog",
Commit: "c1c1c1c1c1", Number: 175, Title: "a tap shows its outcome", Moves: []string{"telegram", "messenger"}}}}}
running := func(module string) []string {
if module == "messenger" || module == "telegram" {
return []string{"novox"}
}
return nil
}
if got, want := planLineOn(p, now, pauseView{}, tierAtLeast, running),
"mesh-catalog #175 a tap shows its outcome · messenger, telegram → novox · tier 1 of 1, building for 2m0s"; got != want {
t.Fatalf("the line reads %q, want %q", got, want)
}
if got := planLine(p, now); !strings.HasPrefix(got, "mesh-catalog #175 a tap shows its outcome · messenger, telegram · tier") {
t.Fatalf("without the machines the line reads %q", got)
}
old := p
old.Commits, old.Delivery = nil, nil
if got := planLine(old, now); !strings.HasPrefix(got, "mesh-catalog c1c1c1c1 · tier 1 of 1") {
t.Fatalf("a record naming no pull request reads %q", got)
}
long := p
long.Delivery.Merges[0].Title = strings.Repeat("abcdefghij", 6)
if got := planHeadline(long, nil); !strings.Contains(got, "#175 "+strings.Repeat("abcdefghij", 4)+"abcdefghi…") {
t.Fatalf("a long title is not cut at fifty runes: %q", got)
}
}
// sameMerge compares what a walk names of a merge: its repository, commit and the commit carrying it.
func sameMerge(a, b inventory.PlanMerge) bool {
return a.Repository == b.Repository && a.Commit == b.Commit && a.Carried == b.Carried
}
// A catalogue merge heard after a later merge of its branch was walked without the word (a merge that touched
// the bus too, on the controller's own path) is not answered by that walk while the delivery seat has a holder:
// its delivery would skip its turn. It joins the next catalogue batch.
+153
View File
@@ -0,0 +1,153 @@
package main
import (
"context"
"fmt"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// **Every wait of a batch has a bound and a condition** (novox/hq ADR 0276 decision 9): a batch still
// assembling a minute past its maximum while no walk is open is a cut the controller failed to make (S18),
// and a batch waiting behind an open walk longer than that walk's bound waits on a walk gone wrong (S19).
// The kinds S18 and S19 raise.
const (
kindBatchNotCut = "batch-not-cut"
kindBatchBehindWalk = "batch-behind-walk"
// kindGroupNotMerged is raised for a group whose last member has not merged by the merge window's maximum after
// its first (novox/hq ADR 0418 clause 3).
kindGroupNotMerged = "group-not-merged"
)
// batchFacts is one batch not yet cut, as the watchdogs read it.
type batchFacts struct {
id, state, grouped string
// atMost is when its window closes at the latest; closed when it closed.
atMost, closed time.Time
// behind is the open walk it waits behind, and walkBound that walk's bound: a tier's bound for each of its
// tiers.
behind string
walkBound time.Duration
// holds are the groups the batch is held for, by head, with why; created is when the batch was made, the first
// merge of the group heard; atMostWindow is the maximum a window may take (the controller's setting).
holds map[string]string
created time.Time
atMostWindow time.Duration
}
// gatherBatches is every batch not yet cut, with the bound of the walk it waits behind.
func gatherBatches(ctx context.Context, inv *inventory.Inventory, now time.Time) ([]batchFacts, error) {
batches, err := inv.Batches(ctx)
if err != nil || len(batches) == 0 {
return nil, err
}
bounds, err := measuredTierBounds(ctx, inv, now)
if err != nil {
return nil, err
}
var out []batchFacts
for _, b := range batches {
_, atMostWindow := mergeWindowOf()
f := batchFacts{id: b.ID, state: b.State, grouped: groupedWords(b), created: b.Created, atMostWindow: atMostWindow}
if w := b.Delivery; w != nil && w.Batch != nil {
f.holds = w.Batch.Holds
f.atMost, f.closed, f.behind = w.Batch.AtMost, w.Batch.ClosesAt, w.Batch.Behind
if f.atMost.Before(f.closed) {
f.closed = f.atMost
}
if w.Batch.CutAtOnce && !b.Created.IsZero() {
// Its window closed when it was made, which is its latest too (ADR 0304 decision 1).
f.atMost, f.closed = b.Created, b.Created
}
}
if f.behind != "" {
if walk, err := inv.PlanByID(ctx, f.behind); err == nil {
f.walkBound = bounds.of(walk.Repository) * time.Duration(max(1, len(walk.Tiers)))
}
}
out = append(out, f)
}
return out, nil
}
// watchBatchesNotCut is S18: a batch still assembling a minute past its maximum, while no walk is open.
func watchBatchesNotCut(f *signalFacts) []conditions.Observation {
var out []conditions.Observation
for _, b := range f.batches {
if b.state != inventory.PlanAssembling || b.atMost.IsZero() || f.now.Sub(b.atMost) <= batchLateAfter ||
len(b.holds) > 0 { // a batch held for a group waits on a member, which has its own condition
continue
}
late := f.now.Sub(b.atMost)
out = append(out, conditions.Observation{Scope: conditions.ScopePlan, ID: b.id, Kind: kindBatchNotCut,
Severity: conditions.Warning,
Summary: fmt.Sprintf("the batch %s is still assembling %s past its latest close (%s), with no walk open: "+
"the controller did not cut it; grouped: %s", b.id, ago(late), b.atMost.UTC().Format(time.RFC3339), b.grouped),
Said: fmt.Sprintf("assembling since %s past its maximum", ago(late)),
Headline: "Merged changes are not being delivered",
Explanation: fmt.Sprintf("Merges collected for one delivery should have been planned %s ago and were not. "+
"Nothing is lost; the mesh keeps them until it plans them.", humanDuration(late)),
Resolved: "Merged changes are being delivered again"})
}
return out
}
// watchBatchesBehind is S19: a batch waiting behind an open walk longer than that walk's bound, naming it.
func watchBatchesBehind(f *signalFacts) []conditions.Observation {
var out []conditions.Observation
for _, b := range f.batches {
if b.state != inventory.PlanQueued || b.behind == "" || b.walkBound <= 0 || f.now.Sub(b.closed) <= b.walkBound {
continue
}
in := f.now.Sub(b.closed)
out = append(out, conditions.Observation{Scope: conditions.ScopePlan, ID: b.id, Kind: kindBatchBehindWalk,
Severity: conditions.Warning,
Summary: fmt.Sprintf("the batch %s has waited %s behind the walk %s, longer than that walk's bound (%s); "+
"grouped: %s; `plans %s` says where that walk stands", b.id, ago(in), b.behind, ago(b.walkBound),
b.grouped, b.behind),
Said: fmt.Sprintf("queued behind %s for %s", b.behind, ago(in)),
Headline: "Merged changes wait behind a slow delivery",
Explanation: fmt.Sprintf("Merges collected for the next delivery have waited %s for the delivery before "+
"them, which is taking longer than it should. Nothing is lost.", humanDuration(in)),
Resolved: "Merged changes no longer wait behind a slow delivery"})
}
return out
}
// watchGroupsNotMerged is the condition of a batch held for a group whose last member has not merged by the merge
// window's maximum after the first member merged, naming the member and why (novox/hq ADR 0418 clause 3): nothing of
// the group is delivered while it waits.
func watchGroupsNotMerged(f *signalFacts) []conditions.Observation {
var out []conditions.Observation
for _, b := range f.batches {
if len(b.holds) == 0 || b.created.IsZero() || f.now.Sub(b.created) <= b.atMostWindow {
continue
}
heads := make([]string, 0, len(b.holds))
for h := range b.holds {
heads = append(heads, h)
}
sort.Strings(heads)
var why []string
for _, h := range heads {
why = append(why, b.holds[h])
}
waited := f.now.Sub(b.created)
out = append(out, conditions.Observation{Scope: conditions.ScopePlan, ID: b.id,
Kind: kindGroupNotMerged, Severity: conditions.Warning,
Summary: fmt.Sprintf("%s; its first member merged %s ago, past the merge window's maximum (%s), and nothing "+
"of the group is delivered while it waits (batch %s)", strings.Join(why, "; "), ago(waited), ago(b.atMostWindow), b.id),
Said: "a group waits for a member that has not merged",
Headline: "A change waits for the rest of its group to merge",
Explanation: fmt.Sprintf("The changes of a group are delivered together, once the last one is merged. "+
"One has not merged in %s. Nothing is lost; the merged ones are kept and delivered with it.",
humanDuration(waited)),
Resolved: "The group's last change merged"})
}
return out
}
+108
View File
@@ -0,0 +1,108 @@
//go:build integration
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// An unrelated closed batch is cut into a walk beside an open one, and that walk asks its first tier at once
// (issue 973, novox/hq ADR 0407 decision 4): a hotfix is never held by an unrelated delivery.
func TestAnUnrelatedBatchWalksBesideAnOpenWalk(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
ctx := t.Context()
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
if ws, _ := walks(t, open); len(ws) != 1 || !ws[0].Open() {
t.Fatalf("the first walk is not open: %+v", ws)
}
heard := t0.Add(100 * time.Second)
hear(t, open, lone(repoMerge("two", "c2", heard)), heard)
advanceHeld(ctx, open)
ws, bs := walks(t, open)
if len(ws) != 2 || len(bs) != 0 {
t.Fatalf("%d walk(s) and %d batch(es); want the unrelated batch cut beside the open walk", len(ws), len(bs))
}
for _, w := range ws {
if !w.Open() || w.Waiting() || deferred(w) || !askedAny(w) {
t.Errorf("%s (%s) is not walking: open %v, deferred %v, asked %v, note %q", w.ID, w.Named(), w.Open(),
deferred(w), askedAny(w), w.Note)
}
}
}
// A closed batch related to an open walk is queued behind it with the reason, while one that shares nothing is
// cut beside it; the walk that started is never superseded (ADR 0276 decision 4, issue 973).
func TestARelatedBatchWaitsBehindTheOpenWalk(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
ctx := t.Context()
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
first := func() inventory.Plan { ws, _ := walks(t, open); return ws[len(ws)-1] }()
heard := t0.Add(100 * time.Second)
hear(t, open, lone(repoMerge("one", "c2", heard)), heard)
hear(t, open, lone(repoMerge("three", "c3", heard.Add(time.Second))), heard.Add(time.Second))
ws, bs := walks(t, open)
if len(ws) != 2 || len(bs) != 1 {
t.Fatalf("%d walk(s) and %d batch(es); want the unrelated merge walking and the related one queued", len(ws), len(bs))
}
b := bs[0]
words := batchWords(b, heard)
if b.State != inventory.PlanQueued || b.Delivery.Batch.Behind != first.ID || b.CommitOf("novox/one") != "c2" ||
!strings.Contains(words, "queued behind "+first.ID) || !strings.Contains(words, "both change novox/one") {
t.Fatalf("the related batch is %s, behind %q, reading %q", b.State, b.Delivery.Batch.Behind, words)
}
if got, _ := open.inventory.PlanByID(ctx, first.ID); !got.Open() {
t.Fatalf("the started walk was %s by the related merge", got.State)
}
first.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &first); err != nil {
t.Fatal(err)
}
cutAt(t, open, heard.Add(2*time.Second))
if _, bs := walks(t, open); len(bs) != 0 {
t.Fatalf("the queued batch was not cut once the walk it waited for ended: %+v", bs)
}
}
// A batch on the controller's own path never walks beside another walk, and nothing walks beside it, whatever
// the two share (issue 973): it replaces the controller, so it goes alone.
func TestAnOwnPathBatchNeverWalksBeside(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
ctx := t.Context()
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
ws, _ := walks(t, open)
first := ws[0]
heard := t0.Add(100 * time.Second)
hear(t, open, lone(repoMerge("mesh-controller", "k1", heard)), heard)
ws, bs := walks(t, open)
if len(ws) != 1 || len(bs) != 1 || !bs[0].OwnPath() || bs[0].State != inventory.PlanQueued ||
bs[0].Delivery.Batch.Behind != first.ID || !strings.Contains(batchWords(bs[0], heard), "own path") {
t.Fatalf("an own-path batch beside an open walk: %d walk(s), batches %+v", len(ws), bs)
}
first.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &first); err != nil {
t.Fatal(err)
}
cutAt(t, open, heard.Add(time.Second))
ws, _ = walks(t, open)
if len(ws) != 2 || !ws[0].Open() || ws[0].CommitOf("novox/mesh-controller") != "k1" {
t.Fatalf("the own-path batch was not cut once the walk ended: %+v", ws)
}
own := ws[0]
// Unrelated to the own-path walk by every footprint, a merge still waits for it.
later := heard.Add(time.Minute)
hear(t, open, lone(repoMerge("two", "c2", later)), later)
ws, bs = walks(t, open)
if len(ws) != 2 || len(bs) != 1 || bs[0].State != inventory.PlanQueued || bs[0].Delivery.Batch.Behind != own.ID {
t.Fatalf("a merge beside the own-path walk: %d walk(s), batches %+v", len(ws), bs)
}
}
@@ -0,0 +1,102 @@
//go:build integration
package main
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
)
func TestTheIncidentAConsumerStaysBesideItsDataUntilAPersonPinsIt(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if _, err := inv.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
for _, m := range storeManifests() {
register(t, open, m)
}
assignAll := func(pairs ...[2]string) {
for _, a := range pairs {
if _, err := assign(ctx, open, a[0], a[1]); err != nil {
t.Fatalf("assign %s %s: %v", a[0], a[1], err)
}
}
}
// The anchor's store and resolver hold the mesh's seats, on record; the laptop runs its own of each.
assignAll([2]string{"anchor", "store"}, [2]string{"anchor", "resolver"})
for _, seat := range [][2]string{{"mesh-store", "store"}, {"mesh-dns-resolver", "resolver"}} {
if err := inv.HoldSeat(ctx, seat[0], catalogue.ScopeMesh, "anchor", seat[1]); err != nil {
t.Fatal(err)
}
}
assignAll([2]string{"laptop", "store"}, [2]string{"laptop", "resolver"}, [2]string{"laptop", "network"},
[2]string{"laptop", "board"})
plan, _, err := planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
if n := need(t, plan, "board", "postgres-database"); n.From != "laptop" || n.Module != "store" || !n.KeepsData {
t.Fatalf("the consumer was bound to %s/%s (keeps data: %v); its data is beside it", n.From, n.Module, n.KeepsData)
}
if n := need(t, plan, "network", "wildcard-resolution"); n.From != "anchor" || n.KeepsData {
t.Fatalf("the resolver was bound to %s (keeps data: %v); its seat is held on anchor (issue 258)", n.From, n.KeepsData)
}
// Sent, and recorded: only the binding to data.
bindings := boundToData(plan, nil)
if len(bindings) != 1 || bindings[0].Provider != (catalogue.Chosen{Node: "laptop", Module: "store"}) {
t.Fatalf("recorded %+v", bindings)
}
if err := inv.RecordBindings(ctx, "laptop", bindings); err != nil {
t.Fatal(err)
}
if found, err := probeBindings(ctx, &doctor{open: open}); err != nil || len(found) != 0 {
t.Fatalf("a mesh bound where it was sent: %+v, %v", found, err)
}
// The laptop's store taken away: refused, not moved to the anchor's empty one.
if err := inv.Unassign(ctx, "laptop", "store"); err != nil {
t.Fatal(err)
}
if _, _, err := planFor(ctx, open, "laptop"); err == nil || !unresolvable(err) ||
!strings.Contains(err.Error(), "board on laptop is bound to laptop/store") ||
!strings.Contains(err.Error(), "pin laptop postgres-database anchor store") {
t.Fatalf("the consumer's store went and it was answered elsewhere: %v", err)
}
// A person pins the anchor's: it moves, said before it is sent, and the record keeps where it was.
if err := inv.PinProvision(ctx, "laptop", "postgres-database", "anchor", "store"); err != nil {
t.Fatal(err)
}
plan, _, err = planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
if n := need(t, plan, "board", "postgres-database"); n.From != "anchor" {
t.Fatalf("pinned to the anchor and bound to %s", n.From)
}
found, err := probeBindings(ctx, &doctor{open: open})
if err != nil {
t.Fatal(err)
}
if len(found) != 1 || found[0].Kind != kindBindingMoving || found[0].Severity != conditions.Warning ||
!strings.Contains(found[0].Summary, "board's postgres-database moves from laptop/store to anchor/store") {
t.Fatalf("a pinned move is not said before it is sent: %+v", found)
}
if err := inv.RecordBindings(ctx, "laptop", boundToData(plan, nil)); err != nil {
t.Fatal(err)
}
all, err := inv.Bindings(ctx)
if err != nil || len(all) != 1 || all[0].MovedFrom != "laptop/store" {
t.Fatalf("%+v, %v", all, err)
}
if found, err := probeBindings(ctx, &doctor{open: open}); err != nil || len(found) != 0 {
t.Fatalf("a move sent is still said: %+v, %v", found, err)
}
}
-90
View File
@@ -39,96 +39,6 @@ func need(t *testing.T, plan catalogue.Resolution, consumer, provision string) c
// The incident through the stores: the laptop runs its own store and a consumer of it, the anchor's
// store holds the mesh's seat. The consumer stays beside its data, the resolver follows its seat, the
// binding is recorded as sent, and a pin — only a pin — moves it, said before the push that carries it.
func TestTheIncidentAConsumerStaysBesideItsDataUntilAPersonPinsIt(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if _, err := inv.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
for _, m := range storeManifests() {
register(t, open, m)
}
assignAll := func(pairs ...[2]string) {
for _, a := range pairs {
if _, err := assign(ctx, open, a[0], a[1]); err != nil {
t.Fatalf("assign %s %s: %v", a[0], a[1], err)
}
}
}
// The anchor's store and resolver hold the mesh's seats, on record; the laptop runs its own of each.
assignAll([2]string{"anchor", "store"}, [2]string{"anchor", "resolver"})
for _, seat := range [][2]string{{"mesh-store", "store"}, {"mesh-dns-resolver", "resolver"}} {
if err := inv.HoldSeat(ctx, seat[0], catalogue.ScopeMesh, "anchor", seat[1]); err != nil {
t.Fatal(err)
}
}
assignAll([2]string{"laptop", "store"}, [2]string{"laptop", "resolver"}, [2]string{"laptop", "network"},
[2]string{"laptop", "board"})
plan, _, err := planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
if n := need(t, plan, "board", "postgres-database"); n.From != "laptop" || n.Module != "store" || !n.KeepsData {
t.Fatalf("the consumer was bound to %s/%s (keeps data: %v); its data is beside it", n.From, n.Module, n.KeepsData)
}
if n := need(t, plan, "network", "wildcard-resolution"); n.From != "anchor" || n.KeepsData {
t.Fatalf("the resolver was bound to %s (keeps data: %v); its seat is held on anchor (issue 258)", n.From, n.KeepsData)
}
// Sent, and recorded: only the binding to data.
bindings := boundToData(plan, nil)
if len(bindings) != 1 || bindings[0].Provider != (catalogue.Chosen{Node: "laptop", Module: "store"}) {
t.Fatalf("recorded %+v", bindings)
}
if err := inv.RecordBindings(ctx, "laptop", bindings); err != nil {
t.Fatal(err)
}
if found, err := probeBindings(ctx, &doctor{open: open}); err != nil || len(found) != 0 {
t.Fatalf("a mesh bound where it was sent: %+v, %v", found, err)
}
// The laptop's store taken away: refused, not moved to the anchor's empty one.
if err := inv.Unassign(ctx, "laptop", "store"); err != nil {
t.Fatal(err)
}
if _, _, err := planFor(ctx, open, "laptop"); err == nil || !unresolvable(err) ||
!strings.Contains(err.Error(), "board on laptop is bound to laptop/store") ||
!strings.Contains(err.Error(), "pin laptop postgres-database anchor store") {
t.Fatalf("the consumer's store went and it was answered elsewhere: %v", err)
}
// A person pins the anchor's: it moves, said before it is sent, and the record keeps where it was.
if err := inv.PinProvision(ctx, "laptop", "postgres-database", "anchor", "store"); err != nil {
t.Fatal(err)
}
plan, _, err = planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
if n := need(t, plan, "board", "postgres-database"); n.From != "anchor" {
t.Fatalf("pinned to the anchor and bound to %s", n.From)
}
found, err := probeBindings(ctx, &doctor{open: open})
if err != nil {
t.Fatal(err)
}
if len(found) != 1 || found[0].Kind != kindBindingMoving || found[0].Severity != conditions.Warning ||
!strings.Contains(found[0].Summary, "board's postgres-database moves from laptop/store to anchor/store") {
t.Fatalf("a pinned move is not said before it is sent: %+v", found)
}
if err := inv.RecordBindings(ctx, "laptop", boundToData(plan, nil)); err != nil {
t.Fatal(err)
}
all, err := inv.Bindings(ctx)
if err != nil || len(all) != 1 || all[0].MovedFrom != "laptop/store" {
t.Fatalf("%+v, %v", all, err)
}
if found, err := probeBindings(ctx, &doctor{open: open}); err != nil || len(found) != 0 {
t.Fatalf("a move sent is still said: %+v, %v", found, err)
}
}
// What one machine's resolution says against its record.
func TestBindingFindingsSayAKeptMoveAndAMoveNothingAskedFor(t *testing.T) {
+207
View File
@@ -0,0 +1,207 @@
package main
import (
"context"
"errors"
"fmt"
"sort"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/conditions"
)
// **A filling bucket or log is said before it is full** (novox/hq ADR 0297 §6, issue 501).
//
// A module's bucket and its log each have a cap on the bus, and when either is full the bus refuses
// the next write: the module stops doing what it writes for, and nothing said so before. On
// 2026-10-11 the issue tracker's bucket held a sixth of its cap after two days, growing toward a stop
// nobody would have heard coming. So the self-check reads every module's bucket and log, and one at
// fillRaiseAt of its cap or more raises a condition naming the module, the bucket or log, and how full
// it is.
//
// **Cleared by observation below fillClearBelow, not below fillRaiseAt**: a bucket or log hovering at
// its threshold would otherwise be raised and cleared on every run. Between the two, an open condition
// is kept and none is raised.
//
// **One that cannot be read is said, and the rest are still judged.** An unanswered question about one
// stream is no reason to know nothing of the others: it is a finding of its own, naming the bucket or
// log and why, and a fill condition already open for it is kept, because not knowing is not a pass.
// The fill thresholds, in percent of a bucket's or log's cap.
const (
fillRaiseAt = 75
fillClearBelow = 70
)
// The conditions the fill probe raises: one filling, and one that could not be read.
const (
kindBucketOrLogFilling = "bucket-or-log-filling"
kindBucketOrLogUnread = "bucket-or-log-unread"
)
// probeFillID is the probe's id in the registry.
const probeFillID = "D-fill"
// bucketOrLog is one module's bucket or log as the fill probe reads it.
type bucketOrLog struct {
Module string
// Kind is `bucket` or `log`; Name its local name; Stream the stream it is on the bus.
Kind, Name, Stream string
}
// filled is how full one bucket or log is, read from the bus.
type filled struct {
Bytes, Max uint64
}
// percent is how full, in whole percent, rounded down.
func (f filled) percent() uint64 {
if f.Max == 0 {
return 0
}
return f.Bytes * 100 / f.Max
}
// fillKey is where a bucket's or log's fill condition is kept.
func fillKey(s bucketOrLog) string { return conditions.Key(conditions.ScopeBus, s.Stream, "filling") }
// fillObservation is what one bucket's or log's fill says: a finding at fillRaiseAt or above, and,
// while its condition is open, at fillClearBelow or above too; nothing otherwise, which is what clears
// it. The operator's words are the kind's (plain_words.go); the summary and the evidence name the
// module, the bucket or log, and the fill.
func fillObservation(s bucketOrLog, f filled, open bool) (conditions.Observation, bool) {
if f.Max == 0 {
return conditions.Observation{}, false // one with no cap cannot fill
}
// Compared in bytes, not rounded percent: 74.9% is not 75%.
atRaise := f.Bytes*100 >= f.Max*fillRaiseAt
aboveClear := f.Bytes*100 >= f.Max*fillClearBelow
if !atRaise && !(open && aboveClear) {
return conditions.Observation{}, false
}
pct := f.percent()
return conditions.Observation{
Scope: conditions.ScopeBus, ID: s.Stream, Token: "filling", Kind: kindBucketOrLogFilling,
Severity: conditions.Warning,
Summary: fmt.Sprintf("%s's %s %s holds %s of %s, %d%%: at its cap the bus refuses the module's next write",
s.Module, s.Kind, s.Name, mibWords(f.Bytes), mibWords(f.Max), pct),
Said: fmt.Sprintf("module %s, %s %s (stream %s): %d of %d bytes, %d%%", s.Module, s.Kind, s.Name,
s.Stream, f.Bytes, f.Max, pct),
}, true
}
// unreadObservation says one bucket or log could not be read, and why. Asked over the network, so one
// look can be wrong: raised on the second look in a row (confirm.go).
func unreadObservation(s bucketOrLog, why error) conditions.Observation {
return conditions.Observation{
Scope: conditions.ScopeBus, ID: s.Stream, Token: "unread", Kind: kindBucketOrLogUnread,
Severity: conditions.Warning, Confirm: true,
Summary: fmt.Sprintf("%s's %s %s could not be read, so how full it is is not known", s.Module, s.Kind, s.Name),
Said: fmt.Sprintf("module %s, %s %s (stream %s): %v", s.Module, s.Kind, s.Name, s.Stream, why),
}
}
// keptFilling is an open fill condition said again for a bucket or log that could not be read this
// time: not knowing how full it is now is no reason to say it has room. Only ever made from a condition
// read back as open — its own summary and severity, never words made up for it.
func keptFilling(s bucketOrLog, open conditions.Condition, why error) conditions.Observation {
return conditions.Observation{
Scope: conditions.ScopeBus, ID: s.Stream, Token: "filling", Kind: kindBucketOrLogFilling,
Severity: open.Severity, Summary: open.Summary,
Said: fmt.Sprintf("module %s, %s %s (stream %s): not read this time (%v); kept open as it was",
s.Module, s.Kind, s.Name, s.Stream, why),
}
}
// mibWords is a size as a person reads it, in MiB with one decimal.
func mibWords(b uint64) string {
return fmt.Sprintf("%.1f MiB", float64(b)/(1024*1024))
}
// readFill is how full one bucket or log is on the bus; found false when it is not on the bus.
type readFill func(ctx context.Context, s bucketOrLog) (f filled, found bool, err error)
// openFill is the fill condition open under a key, if one is, or why it could not be read.
type openFill func(ctx context.Context, key string) (conditions.Condition, bool, error)
// judgeFills judges every bucket and log on its own: one that cannot be read is said as unread, with
// its fill condition kept only when that condition was read back and is open — when it cannot be read
// either, nothing is said of its fill, which the unread finding already covers. Every other is judged
// by its fill; for one of those, a condition that cannot be read is taken as open, so an unknown never
// clears a fill measured between fillClearBelow and fillRaiseAt.
func judgeFills(ctx context.Context, all []bucketOrLog, read readFill, open openFill) []conditions.Observation {
var out []conditions.Observation
for _, s := range all {
f, found, err := read(ctx, s)
if err != nil {
out = append(out, unreadObservation(s, err))
if c, isOpen, cerr := open(ctx, fillKey(s)); cerr == nil && isOpen {
out = append(out, keptFilling(s, c, err))
}
continue
}
if !found {
continue // not on the bus: created on the controller's next raise, and nothing there can fill
}
_, isOpen, cerr := open(ctx, fillKey(s))
isOpen = isOpen || cerr != nil
if o, said := fillObservation(s, f, isOpen); said {
out = append(out, o)
}
}
sort.SliceStable(out, func(i, j int) bool { return out[i].Key() < out[j].Key() })
return out
}
// declaredBucketsAndLogs is every module's bucket and log the catalogue declares, by its stream.
func declaredBucketsAndLogs(ctx context.Context, d *doctor) ([]bucketOrLog, error) {
buckets, err := d.open.inventory.DeclaredBuckets(ctx)
if err != nil {
return nil, err
}
logs, err := d.open.inventory.DeclaredLogs(ctx)
if err != nil {
return nil, err
}
var out []bucketOrLog
for _, b := range buckets {
out = append(out, bucketOrLog{Module: b.Module, Kind: "bucket", Name: b.Name, Stream: "KV_" + b.Bucket()})
}
for _, l := range logs {
out = append(out, bucketOrLog{Module: l.Module, Kind: "log", Name: l.Name, Stream: l.Stream()})
}
return out, nil
}
// probeFill reads every module's bucket and log on the bus and says each one filling toward its cap,
// and each one that could not be read.
func probeFill(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
all, err := declaredBucketsAndLogs(ctx, d)
if err != nil {
return nil, err
}
js := d.js.Context()
read := func(ctx context.Context, s bucketOrLog) (filled, bool, error) {
if err := ctx.Err(); err != nil {
return filled{}, false, err
}
info, err := js.StreamInfo(s.Stream, nats.Context(ctx))
switch {
case errors.Is(err, nats.ErrStreamNotFound):
return filled{}, false, nil
case err != nil:
return filled{}, false, err
case info.Config.MaxBytes <= 0:
return filled{}, true, nil
}
return filled{Bytes: info.State.Bytes, Max: uint64(info.Config.MaxBytes)}, true, nil
}
open := func(ctx context.Context, key string) (conditions.Condition, bool, error) {
if d.keeper == nil {
return conditions.Condition{}, false, nil
}
return d.keeper.Get(ctx, key)
}
return judgeFills(ctx, all, read, open), nil
}
+165
View File
@@ -0,0 +1,165 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/conditions"
)
// **A filling log or bucket is said at three quarters of its cap and cleared below seven tenths**
// (novox/hq ADR 0297 §6): raised at 75% naming the module, the bucket or log and its fill; not at 74.9%;
// kept between 70% and 75% while it is open, and not raised there when it is not; and cleared — said no
// more — once it reads below 70%, open or not.
func TestAFillingBucketOrLogIsRaisedAtThreeQuartersAndClearedBelowSevenTenths(t *testing.T) {
const mib = 1024 * 1024
log := bucketOrLog{Module: "mesh-issues", Kind: "log", Name: "changes", Stream: "LOG_mesh-issues_changes"}
bucket := bucketOrLog{Module: "mesh-issues", Kind: "bucket", Name: "issues", Stream: "KV_mesh-issues_issues"}
for _, c := range []struct {
name string
of bucketOrLog
bytes uint64
max uint64
open bool
said bool
}{
{"a log at exactly 75%", log, 768 * mib, 1024 * mib, false, true},
{"a bucket at exactly 75%", bucket, 48 * mib, 64 * mib, false, true},
{"a bucket full", bucket, 64 * mib, 64 * mib, false, true},
{"a log just under 75%", log, 768*mib - 1, 1024 * mib, false, false},
{"a bucket at 72%, not open", bucket, 64 * mib * 72 / 100, 64 * mib, false, false},
{"a bucket at 72%, open: kept", bucket, 64 * mib * 72 / 100, 64 * mib, true, true},
{"a log at exactly 70%, open: kept", log, 700 * mib, 1000 * mib, true, true},
{"a log just under 70%, open: cleared", log, 700*mib - 1, 1000 * mib, true, false},
{"a bucket at 10%, open: cleared", bucket, 64 * mib / 10, 64 * mib, true, false},
{"a bucket with no cap", bucket, 100, 0, true, false},
} {
o, said := fillObservation(c.of, filled{Bytes: c.bytes, Max: c.max}, c.open)
if said != c.said {
t.Errorf("%s: said %v, want %v", c.name, said, c.said)
continue
}
if !said {
continue
}
if o.Key() != fillKey(c.of) || o.Kind != kindBucketOrLogFilling || o.Severity != conditions.Warning {
t.Errorf("%s: raised as %s (%s, %s)", c.name, o.Key(), o.Kind, o.Severity)
}
for _, part := range []string{c.of.Module, c.of.Kind + " " + c.of.Name, "%", " of "} {
if !strings.Contains(o.Summary, part) {
t.Errorf("%s: does not name %q: %q", c.name, part, o.Summary)
}
}
if !strings.Contains(o.Said, "bytes") {
t.Errorf("%s: the evidence does not say the fill in bytes: %q", c.name, o.Said)
}
}
o, _ := fillObservation(log, filled{Bytes: 800 * mib, Max: 1024 * mib}, false)
if !strings.Contains(o.Summary, "800.0 MiB of 1024.0 MiB, 78%") {
t.Errorf("the fill is said as %q", o.Summary)
}
}
// **One bucket or log that cannot be read does not stop the others being judged** (review of #231): it is
// said as unread with its reason, a fill condition open for it is kept, and every other bucket and log
// is judged by its own fill.
func TestAnUnreadableBucketOrLogIsSaidAndTheRestAreStillJudged(t *testing.T) {
const mib = 1024 * 1024
broken := bucketOrLog{Module: "a", Kind: "bucket", Name: "broken", Stream: "KV_a_broken"}
brokenOpen := bucketOrLog{Module: "a", Kind: "log", Name: "was-filling", Stream: "LOG_a_was-filling"}
full := bucketOrLog{Module: "b", Kind: "log", Name: "changes", Stream: "LOG_b_changes"}
empty := bucketOrLog{Module: "c", Kind: "bucket", Name: "quiet", Stream: "KV_c_quiet"}
absent := bucketOrLog{Module: "d", Kind: "bucket", Name: "not-yet", Stream: "KV_d_not-yet"}
refusal := errors.New("the bus did not answer")
read := func(_ context.Context, s bucketOrLog) (filled, bool, error) {
switch s {
case broken, brokenOpen:
return filled{}, false, refusal
case full:
return filled{Bytes: 900 * mib, Max: 1000 * mib}, true, nil
case empty:
return filled{Bytes: 1, Max: 64 * mib}, true, nil
}
return filled{}, false, nil
}
open := func(_ context.Context, key string) (conditions.Condition, bool, error) {
if key == fillKey(brokenOpen) {
return conditions.Condition{Key: key, Severity: conditions.Warning,
Summary: "a's log was-filling holds 800.0 MiB of 1024.0 MiB, 78%"}, true, nil
}
return conditions.Condition{}, false, nil
}
got := map[string]conditions.Observation{}
for _, o := range judgeFills(context.Background(), []bucketOrLog{broken, brokenOpen, full, empty, absent}, read, open) {
got[o.Key()] = o
}
for _, s := range []bucketOrLog{broken, brokenOpen} {
o, said := got[conditions.Key(conditions.ScopeBus, s.Stream, "unread")]
if !said || o.Kind != kindBucketOrLogUnread || !o.Confirm || !strings.Contains(o.Said, refusal.Error()) {
t.Errorf("%s could not be read and was said as %+v", s.Stream, o)
}
}
if o, kept := got[fillKey(brokenOpen)]; !kept || !strings.Contains(o.Said, "kept open") {
t.Errorf("an open fill condition of a log not read this time was not kept: %+v", o)
}
if _, said := got[fillKey(broken)]; said {
t.Error("a bucket not read and not filling was said filling")
}
if o, said := got[fillKey(full)]; !said || o.Kind != kindBucketOrLogFilling {
t.Errorf("a log at 90%% beside an unreadable one was not judged: %+v", got)
}
if len(got) != 4 {
t.Errorf("said %d findings, want 4 (two unread, one kept, one filling): %v", len(got), got)
}
}
// Both kinds have plain words of their own, which hold to the plain rule (novox/hq ADR 0253).
func TestAFillingOrUnreadBucketOrLogIsSaidInPlainWords(t *testing.T) {
for _, kind := range []string{kindBucketOrLogFilling, kindBucketOrLogUnread} {
wording, has := plainWordings[kind]
if !has {
t.Errorf("%s has no plain words", kind)
continue
}
if why, ok := conditions.PlainWords(wording(conditions.Observation{Kind: kind})); !ok {
t.Errorf("%s: %s", kind, why)
}
}
}
// **When neither the bucket or log nor its condition can be read, nothing is said of its fill** (second
// review of #231): the unread finding covers it, and no fill is invented for it. For one that is read and
// measured between seven tenths and three quarters, a condition that cannot be read is taken as open, so
// an unknown never clears it.
func TestNothingIsSaidOfAFillWhenNeitherItNorItsConditionCanBeRead(t *testing.T) {
const mib = 1024 * 1024
broken := bucketOrLog{Module: "a", Kind: "log", Name: "changes", Stream: "LOG_a_changes"}
between := bucketOrLog{Module: "b", Kind: "bucket", Name: "issues", Stream: "KV_b_issues"}
read := func(_ context.Context, s bucketOrLog) (filled, bool, error) {
if s == broken {
return filled{}, false, errors.New("the bus did not answer")
}
return filled{Bytes: 72 * mib, Max: 100 * mib}, true, nil
}
open := func(context.Context, string) (conditions.Condition, bool, error) {
return conditions.Condition{}, false, errors.New("the conditions bucket did not answer")
}
got := map[string]conditions.Observation{}
for _, o := range judgeFills(context.Background(), []bucketOrLog{broken, between}, read, open) {
got[o.Key()] = o
}
if o, said := got[fillKey(broken)]; said {
t.Fatalf("a fill was said for a log whose fill and condition could not be read: %+v", o)
}
if _, said := got[conditions.Key(conditions.ScopeBus, broken.Stream, "unread")]; !said {
t.Error("the log that could not be read was not said as unread")
}
if _, kept := got[fillKey(between)]; !kept {
t.Error("a bucket at 72% whose condition could not be read was cleared")
}
if len(got) != 2 {
t.Errorf("said %d findings, want 2: %v", len(got), got)
}
}
+95 -3
View File
@@ -15,6 +15,7 @@ import (
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
@@ -166,6 +167,10 @@ func buildFrom(result link.BuildResult) inventory.Build {
for _, r := range result.Read {
kept.Read = append(kept.Read, inventory.ReadRepository{Repository: r.Repository, Ref: r.Ref})
}
// What it was made from, as files (novox/hq ADR 0267): the planner maps the next merge onto it.
for _, s := range result.Sources {
kept.Sources = append(kept.Sources, inventory.BuildSource{Repository: s.Repository, Ref: s.Ref, Paths: s.Paths})
}
var announced []inventory.Artifact
for _, made := range result.Made {
announced = append(announced, inventory.Artifact{
@@ -178,8 +183,10 @@ func buildFrom(result link.BuildResult) inventory.Build {
kept.Manifest = recordedManifest(result.Manifest, announced)
// The module name comes from the manifest, which only exists when the build got that far.
if len(kept.Manifest) > 0 {
if m, err := catalogue.ParseManifest(kept.Manifest); err == nil {
kept.Module = m.Module
if list, err := judgedWith(result); err == nil {
if m, err := catalogue.ParseManifestWith(kept.Manifest, list); err == nil {
kept.Module = m.Module
}
}
}
return kept
@@ -479,6 +486,7 @@ func buildOneAsked(ctx context.Context, source buildSource, path, ref string, wa
Seats: seatBases(ctx),
DryRun: dryRun,
}
request.Reuse = reuseAsked(ctx, dryRun)
fmt.Printf("asked for %s", source)
if source.Seat != "" {
fmt.Printf(" (%s)", repository)
@@ -597,6 +605,11 @@ func saysWhenThePolicyActs(ctx context.Context, inv *inventory.Inventory, module
// written with the same values.
func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResult) (
catalogue.Manifest, inventory.Build, error) {
// The list of predefined keys as the store keeps it now: another controller process may have kept a newer one.
if err := loadPredefined(ctx, inv); err != nil {
return catalogue.Manifest{}, inventory.Build{}, fmt.Errorf("the list of predefined keys kept cannot be "+
"read: %w", err)
}
kept := buildFrom(result)
if err := inv.RecordBuild(ctx, kept); err != nil {
return catalogue.Manifest{}, kept, err
@@ -607,7 +620,15 @@ func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResu
return catalogue.Manifest{}, kept, fmt.Errorf("%s could not build %s:\n%s",
result.On, result.Repository, result.Failed)
}
manifest, err := catalogue.ParseManifest(kept.Manifest)
// Judged first against the list the build was judged against: the catalogue's at the commit built (novox/hq
// ADR 0348 §7), so a module naming a key its own commit adds is read; and again below against the list the
// mesh publishes, before it is registered.
list, err := judgedWith(result)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: %w", result.On,
result.Repository, short(result.Commit), err)
}
manifest, err := catalogue.ParseManifestWith(kept.Manifest, list)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s and what came back is not a manifest: %w",
result.On, result.Repository, err)
@@ -665,6 +686,12 @@ func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResu
return manifest, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: %w", result.On,
manifest.Module, short(result.Commit), err)
}
// **Never a build in the scratch namespace** (novox/hq ADR 0239 §9, ADR 0350 §2): what a check builds is
// registered only as the walk's copy in the module's own namespace.
if scratch := inScratch(result.Made); scratch != "" {
return manifest, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: %s is in the scratch "+
"namespace, which only a walk's copy leaves (ADR 0350)", result.On, manifest.Module, short(result.Commit), scratch)
}
// **A build that failed its gate is never registered again** (novox/hq ADR 0236): an outcome heard
// twice, or replayed, would otherwise make the build a rollback put back what the module is again,
// and the next push would send it.
@@ -675,6 +702,23 @@ func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResu
"back: it is recorded and not registered again — a newer build is", result.On, manifest.Module,
short(result.Commit))
}
// **The newest trunk build's list of predefined keys is published** (novox/hq ADR 0348 §7): a build of the
// catalogue that may be registered keeps its list when its commit is newer than the one kept. Then the module
// is judged against the list the mesh publishes, as every manifest read again is: one naming a key that list
// does not hold, or of another type, is refused here and never enters the catalogue.
if carried, err := carriedList(result); err != nil {
return manifest, kept, err
} else if carried != nil {
if err := publishPredefined(ctx, inv, result, carried); err != nil {
fmt.Printf("%s: the list of predefined keys it carried is not published: %v\n", result.ID, err)
}
}
if manifest.NamesPredefined() {
if _, err := catalogue.ParseManifest(kept.Manifest); err != nil {
return manifest, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: judged against the "+
"list of predefined keys the mesh publishes, %w", result.On, manifest.Module, short(result.Commit), err)
}
}
// **A build whose source is unchanged is never a move** (novox/hq issue 280): a rebuild made from
// what the build the mesh stands on was made from registers that build's artifacts at the new
// commit, so no machine is sent a new digest for a source nobody changed — an image is not
@@ -710,6 +754,16 @@ func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResu
return manifest, kept, nil
}
// inScratch is the first artifact a build made into the scratch namespace; empty when none is.
func inScratch(made []link.MadeArtifact) string {
for _, m := range made {
if path, kept := catalogue.InArtifactStore(catalogue.Recorded(m.Reference)); kept && strings.HasPrefix(path, builder.ScratchPrefix) {
return m.Reference
}
}
return ""
}
// errOffTheTrunk is a build of a commit off its repository's trunk, which is never published.
var errOffTheTrunk = errors.New("the commit is not on its repository's trunk: a commit off the trunk is checked, " +
"never published (ADR 0238) — merge it, and the merge builds it")
@@ -798,6 +852,9 @@ type answers struct {
sources map[string]inventory.Source
// waiting is every machine not running what the mesh would send it.
waiting []inventory.Machine
// recordedWaits is, per machine, every move of a recorded module that waits there for a person (novox/hq ADR
// 0377 §6): the machine is behind for these, and `push <node>` sends them.
recordedWaits map[string][]string
// reported is every machine's last word beside when it was last sent a declaration — the
// pair that answers "has it caught up", which waiting alone cannot (the sent digest is
// recorded at send, not at apply).
@@ -831,6 +888,9 @@ type answers struct {
// public name on the machine went dark. The holds were correct; they were recorded only in the
// machine's own state file, and the one visible symptom was a count that did not add up.
untaken map[string]map[string]int
// leftOut is, per machine, every module of its set its composition leaves out, and why (novox/hq issue
// 380): assigned and not applied, which every push said only in passing. Not well while there is any.
leftOut map[string][]leftOutModule
// unheld is every module on a machine whose resources are applied through a seat nothing on
// that machine holds (novox/hq ADR 0207), with the modules that could hold it. Reported, not
// refused, until the switch — and while there is any, the mesh is not all well: the order the
@@ -859,6 +919,10 @@ type answers struct {
// applied, is not well: an assignment somebody made is not made yet, or will not be.
pending []inventory.PendingAssignment
pendingUnread string
// lastSent is every machine's last recorded send, by name, with what the machine answered of it (novox/hq
// issue 485); lastSentUnread why it could not be read.
lastSent map[string]inventory.Send
lastSentUnread string
}
// heldBy is every artifact this mesh has built, for a build that may need one as its base.
@@ -870,6 +934,34 @@ type answers struct {
//
// Routed through the artifact store as the network reaches it now (novox/hq 04-ISSUES/102): a
// base is recorded by digest and path, and a build machine needs something it can pull.
// runningBy is, keyed "<module>/<artifact>", each artifact of the build the mesh has registered for its module, as a
// builder reaches the artifact store (novox/hq issue 721): what a check compares its builds with. Empty when it cannot
// be read, said, and then nothing is said unchanged.
func runningBy(ctx context.Context) map[string]string {
open, err := openStores(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read what this mesh runs, so the check compares nothing: %v\n", err)
// empty-on-error: said above; a check given nothing to compare with says no module unchanged, and all move
return nil
}
defer open.Close()
running, err := open.inventory.RegisteredHeld(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read what this mesh runs, so the check compares nothing: %v\n", err)
// empty-on-error: said above; a check given nothing to compare with says no module unchanged, and all move
return nil
}
address, err := whereABuilderReachesTheStore(ctx, open.inventory)
if err != nil || address == "" {
return running
}
routed := make(map[string]string, len(running))
for key, reference := range running {
routed[key] = catalogue.Rerouted(reference, address)
}
return routed
}
func heldBy(ctx context.Context) map[string]string {
open, err := openStores(ctx)
if err != nil {
@@ -0,0 +1,131 @@
//go:build integration
package main
import (
"encoding/json"
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func TestABuildHeardIsRecordedAndRegistered(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
manifest, _ := json.Marshal(map[string]any{"module": "shop", "version": "3"})
m, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory, link.BuildResult{
ID: "b-1", Repository: "http://forge.internal:20000/novox/shop.git", Path: "modules/shop",
Ref: "main", On: "anchor", Commit: "abcdef0123", Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/shop"},
}))
if err != nil {
t.Fatal(err)
}
if m.Module != "shop" {
t.Fatalf("registered %q", m.Module)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if _, held := shelf["shop"]; !held {
t.Fatal("the module a heard build produced is not in the catalogue")
}
src, err := open.inventory.SourceOf(ctx, "shop")
if err != nil || src.Seat != "git" || src.Repository != "novox/shop" || src.BuiltFrom != "abcdef0123" {
t.Fatalf("the source is the seat and the path, never the URL: %+v %v", src, err)
}
builds, err := open.inventory.Builds(ctx, "shop", 5)
if err != nil || len(builds) != 1 || builds[0].ID != "b-1" {
t.Fatalf("the build is not recorded once: %v %v", builds, err)
}
named, _ := json.Marshal(map[string]any{"module": "idp", "version": "1", "resources": []any{
map[string]any{"id": "server", "type": "container", "image": "x@sha256:aa",
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}}}})
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{
ID: "b-2", Repository: "/r", On: "anchor", Commit: "0123456789", Manifest: named})
if err == nil || !strings.Contains(err.Error(), "does not register it") {
t.Fatalf("a definition naming an installation was taken in: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["idp"].Module != "" {
t.Fatal("the refused module was registered anyway")
}
if builds, _ := open.inventory.Builds(ctx, "idp", 5); len(builds) != 1 {
t.Fatalf("the refused build was not recorded: %v", builds)
}
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{ID: "b-3", Repository: "/r", On: "anchor", Failed: "no compiler"})
if err == nil || !strings.Contains(err.Error(), "no compiler") {
t.Fatalf("a failure is said in the builder's words: %v", err)
}
}
func TestABuildAtACommitKeepsTheBranchTheModuleFollows(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
manifest, _ := json.Marshal(map[string]any{"module": "unifi", "version": "1"})
result := func(id, ref, commit string) link.BuildResult {
return link.BuildResult{ID: id, Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/unifi", Ref: ref, On: "anchor", Commit: commit, Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
}
if _, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory, result("b-1", "main", "1111111aaaa"))); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, result("b-2", "9c97a8a", "9c97a8a1d2c3")); err != nil {
t.Fatal(err)
}
src, err := open.inventory.SourceOf(ctx, "unifi")
if err != nil {
t.Fatal(err)
}
if src.Ref != "main" || src.BuiltFrom != "9c97a8a1d2c3" {
t.Errorf("after a build at a commit the module follows %q, built from %q; want main, 9c97a8a1d2c3", src.Ref, src.BuiltFrom)
}
// One new to the catalogue, first built at a commit, follows the default branch.
other, _ := json.Marshal(map[string]any{"module": "letta", "version": "1"})
r := result("b-3", "deadbeef", "deadbeefcafe")
r.Manifest, r.Path = other, "modules/letta"
if _, _, err := takeIn(ctx, open.inventory, r); err != nil {
t.Fatal(err)
}
if src, _ := open.inventory.SourceOf(ctx, "letta"); src.Ref != "" {
t.Errorf("a module first built at a commit follows %q, want the default branch", src.Ref)
}
}
func TestAnOlderBuildHeardLaterDoesNotReplaceTheNewer(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
older := time.Date(2026, 10, 3, 21, 33, 45, 0, time.UTC)
newer := time.Date(2026, 10, 3, 21, 51, 57, 0, time.UTC)
result := func(asked time.Time, image string) link.BuildResult {
manifest, _ := json.Marshal(map[string]any{"module": "postgres", "version": image})
return link.BuildResult{ID: link.NewBuildID(asked), Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/postgres", Ref: "main", On: "anchor", Commit: "efff5415", Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
}
if _, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory, result(newer, "4bcd5f73"))); err != nil {
t.Fatal(err)
}
_, _, err := takeIn(ctx, open.inventory, result(older, "0ab07fa9"))
if !errors.Is(err, inventory.ErrSuperseded) {
t.Fatalf("the older request's outcome was taken in as current: %v", err)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if got := shelf["postgres"].Version; got != "4bcd5f73" {
t.Errorf("postgres is %q; want the newer request's 4bcd5f73", got)
}
if builds, _ := open.inventory.Builds(ctx, "postgres", 5); len(builds) != 2 {
t.Errorf("the late build was not recorded: %v", builds)
}
}
+110 -2
View File
@@ -153,8 +153,97 @@ type forgeFacts struct {
}
// askTheForge asks the forge, through its module's tools on the bus, for a repository's id and its branch's
// protection. A variable so a test needs no forge.
// protection, waiting out a forge not served for a moment (novox/hq issue 486). A variable so a test needs no
// forge.
var askTheForge = func(ctx context.Context, owner, repo, branch string) (forgeFacts, error) {
return forgeWaitedOut(ctx, forgeWait, owner+"/"+repo, func(ctx context.Context) (forgeFacts, error) {
return askTheForgeOnce(ctx, owner, repo, branch)
})
}
// waitOut bounds how long a question is asked again while its answer is "not now": a pause of first, doubling
// to most, for at most bound in all. now and sleep are the clock, so a test waits in no real time.
type waitOut struct {
bound, first, most time.Duration
now func() time.Time
sleep func(context.Context, time.Duration) error
}
// forgeWait is the registration's wait for the forge (novox/hq issue 486). On 2026-10-11 a build's registration
// asked the forge while node-tools on the control-node re-served after a rollout of the tool runner, heard
// "nothing serves gitea.gitea_api", and failed its plan; the forge answered again seconds later. A re-serve takes
// about 10 s, so 60 s waits one out with room to spare. It is a cap on the whole wait, an ask in flight included:
// the registration holds the controller's one receive loop while it waits, and a loop held two minutes reads as
// deaf. The pauses are those the build machine waits the registry with (novox/hq issue 457).
var forgeWait = waitOut{bound: 60 * time.Second, first: time.Second, most: 10 * time.Second,
now: time.Now, sleep: sleepOrDone}
// sleepOrDone pauses for d, or less when ctx ends first.
func sleepOrDone(ctx context.Context, d time.Duration) error {
t := time.NewTimer(d)
defer t.Stop()
select {
case <-ctx.Done():
return ctx.Err()
case <-t.C:
return nil
}
}
// forgeNotNow is an answer that says the forge's tool is not served now — the tool runner re-serving — not that
// the forge refused. Only that is asked again: a refusal, an unreadable answer or a repository with no id is the
// forge's word, and a tool that did not answer in time has already held the receive loop its whole wait.
func forgeNotNow(err error) bool { return errors.Is(err, link.ErrNothingServes) }
// forgeWaitedOut asks the forge about repository, asking again while it answers "not served now", all of it
// within w's bound, an ask in flight included (novox/hq issue 486). Past the bound it fails, saying it waited,
// for how long, and what the forge last said; when the caller's own context ends, it says that instead.
func forgeWaitedOut(ctx context.Context, w waitOut, repository string,
ask func(context.Context) (forgeFacts, error)) (forgeFacts, error) {
capped, cancel := context.WithTimeout(ctx, w.bound)
defer cancel()
began := w.now()
pause := w.first
for asked := 1; ; asked++ {
facts, err := ask(capped)
if err == nil {
if asked > 1 {
fmt.Printf("the forge answered about %s again, after %s (novox/hq issue 486)\n", repository,
w.now().Sub(began).Round(time.Second))
}
return facts, nil
}
if ctx.Err() != nil {
return forgeFacts{}, err // the caller's own context ended: no wait of ours to speak of
}
gaveUp := func() (forgeFacts, error) {
return forgeFacts{}, fmt.Errorf("waited %s for the forge to answer about %s, asking %d times, and it "+
"did not (novox/hq issue 486): %w", w.now().Sub(began).Round(time.Second), repository, asked, err)
}
if capped.Err() != nil {
return gaveUp() // the bound ended an ask in flight
}
if !forgeNotNow(err) {
return forgeFacts{}, err
}
if w.now().Sub(began)+pause > w.bound {
return gaveUp()
}
fmt.Printf("the forge cannot be asked about %s now (%v); asking again in %s, for at most %s in all "+
"(novox/hq issue 486)\n", repository, err, pause, w.bound)
if serr := w.sleep(capped, pause); serr != nil {
if ctx.Err() != nil {
return forgeFacts{}, fmt.Errorf("stopped waiting for the forge to answer about %s: %w", repository,
ctx.Err())
}
return gaveUp()
}
pause = min(2*pause, w.most)
}
}
// askTheForgeOnce asks the forge once, through its module's tools on the bus.
func askTheForgeOnce(ctx context.Context, owner, repo, branch string) (forgeFacts, error) {
js, err := aBus()
if err != nil {
return forgeFacts{}, err
@@ -346,6 +435,20 @@ func mayRegisterFrom(ctx context.Context, inv *inventory.Inventory, module strin
module, sourceWords(built.Repository, built.Seat))
}
}
// **Or the move the operator approved** (novox/hq ADR 0420 §7, issue 1025): a known module built from another
// repository by a merge, whose pull request mesh-delivery merged on the operator's Approve of that head.
// A verb alone, or a merge with no such warrant, is refused as before.
movedWhy := ""
if why != "" && !terminal && !isNew {
accepted, err := approvedMove(ctx, inv, forms, buildID, built)
switch {
case err == nil:
movedWhy, why = accepted, ""
case !errors.Is(err, errNotAMerge):
fmt.Printf("%s: %s is built from another repository, and no approved move covers this build: %v\n",
buildID, module, err)
}
}
if why != "" && !terminal {
return refuse(why)
}
@@ -382,7 +485,8 @@ func mayRegisterFrom(ctx context.Context, inv *inventory.Inventory, module strin
}
// The same repository by the forge's id, not only its name: one deleted and made again is another.
recorded := map[string]int64{}
if !isNew {
// A moved module is registered from the repository the operator approved, not the one it was registered from.
if !isNew && movedWhy == "" {
id, err := inv.SourceIdentity(ctx, module)
if err != nil {
return 0, err
@@ -403,6 +507,10 @@ func mayRegisterFrom(ctx context.Context, inv *inventory.Inventory, module strin
owner, name, m, facts.ID, m, id))
}
}
if movedWhy != "" {
fmt.Printf("%s: %s moved from %s to %s/%s — registered at the operator's Approve: %s\n", buildID, module,
sourceWords(was.Repository, was.Seat), owner, name, movedWhy)
}
return facts.ID, nil
}
@@ -0,0 +1,353 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"os/exec"
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func theCatalogue(t *testing.T) *stores {
t.Helper()
open := aMesh(t)
ctx := t.Context()
for _, b := range []link.BuildResult{
onTrunk("build-sudo", "novox/mesh-catalog", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "1"}),
onTrunk("build-host", "novox/mesh-host", "git", "", map[string]any{"module": "mesh-host", "version": "1"}),
} {
keptAsked(t, open.inventory, b.ID, b.Source.Repository, b.Path, true)
if _, _, err := takeIn(ctx, open.inventory, b); err != nil {
t.Fatal(err)
}
}
return open
}
func TestABuildFromAnAgentsRepositoryIsNotRegisteredAsAModuleTheMeshHolds(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
for _, c := range []struct {
name, repository, path, module string
}{
{"its own repository naming sudo", "agent/sudo", "modules/sudo", "sudo"},
{"a fork of the catalogue", "agent/mesh-catalog", "modules/sudo", "sudo"},
{"a fork of the node-engine", "agent/mesh-host", "", "mesh-host"},
} {
t.Run(c.name, func(t *testing.T) {
id := "build-" + strings.ReplaceAll(c.repository, "/", "-")
keptAsked(t, open.inventory, id, c.repository, c.path, false) // through the build verb
evil := onTrunk(id, c.repository, "git", c.path, map[string]any{"module": c.module, "version": "evil"})
_, _, err := takeIn(ctx, open.inventory, evil)
if !errors.Is(err, errNotItsSource) {
t.Fatalf("a build of %s was taken in as %s: %v", c.repository, c.module, err)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if got := shelf[c.module].Version; got != "1" {
t.Fatalf("%s is now %q, from %s", c.module, got, c.repository)
}
if _, found, _ := open.inventory.BuildByID(ctx, id); !found {
t.Errorf("the refused build %s is not recorded", id)
}
})
}
}
func TestANewModuleFromARepositoryTheCatalogueDoesNotBuildFromIsNotRegistered(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
keptAsked(t, open.inventory, "build-new", "agent/tools", "", false)
_, _, err := takeIn(ctx, open.inventory, onTrunk("build-new", "agent/tools", "git", "",
map[string]any{"module": "agent-tools", "version": "1"}))
if !errors.Is(err, errNotItsSource) {
t.Fatalf("a new module from an agent's repository was taken in: %v", err)
}
// And one asked of nobody here — an outcome on the bus no request was kept for — the same.
_, _, err = takeIn(ctx, open.inventory, onTrunk("build-unasked", "agent/tools", "git", "",
map[string]any{"module": "agent-tools", "version": "1"}))
if !errors.Is(err, errNotItsSource) {
t.Fatalf("an outcome nobody asked for was taken in: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["agent-tools"].Module != "" {
t.Fatal("the refused module is in the catalogue")
}
}
func TestAtTheTerminalAnotherRepositoryIsRegistered(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
keptAsked(t, open.inventory, "build-moved", "novox/sudo", "", true)
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-moved", "novox/sudo", "git", "",
map[string]any{"module": "sudo", "version": "2"})); err != nil {
t.Fatalf("a move the operator asked for at the terminal was refused: %v", err)
}
if src, _ := open.inventory.SourceOf(ctx, "sudo"); src.Repository != "novox/sudo" {
t.Fatalf("sudo is built from %q", src.Repository)
}
if err := open.inventory.RecordBuildRequest(ctx, inventory.BuildRequest{ID: "build-external",
Repository: "http://forge.internal:20000/someone/app.git", For: "build", AtTerminal: true}); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-external", "someone/app", "", "",
map[string]any{"module": "app", "version": "1"})); err != nil {
t.Fatalf("a new module the operator asked for at the terminal was refused: %v", err)
}
}
func TestADeliveryFromTheRegisteredRepositoryIsRegistered(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
rebuilt := onTrunk("build-plan", "novox/mesh-catalog", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "2"})
if err := open.inventory.RecordBuildRequest(ctx, inventory.BuildRequest{ID: rebuilt.ID,
Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/sudo", For: "plan"}); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, rebuilt); err != nil {
t.Fatalf("a plan's build of the module's own repository was refused: %v", err)
}
added := onTrunk("build-merge", "novox/mesh-catalog", "git", "modules/zram", map[string]any{"module": "zram", "version": "1"})
if _, _, err := takeIn(ctx, open.inventory, added); err != nil {
t.Fatalf("a module a merge added to the catalogue repository was refused: %v", err)
}
shelf, _ := open.inventory.Catalogue(ctx)
if shelf["sudo"].Version != "2" || shelf["zram"].Module == "" {
t.Fatalf("not registered: sudo %q, zram %q", shelf["sudo"].Version, shelf["zram"].Module)
}
}
func TestAVerbAsksNoBuildOfARepositoryTheCatalogueDoesNotBuildFrom(t *testing.T) {
theCatalogue(t)
ctx := t.Context()
t.Setenv(verbVar, "build")
t.Setenv(link.CallerVar, "node-tools.anchor, through the mesh-controller seat")
err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, "http://forge.internal:20000/agent/sudo.git")
var policy *heldAtTheTerminal
if !errors.As(err, &policy) {
t.Fatalf("a verb's build of an agent's repository was asked: %v", err)
}
if err := verbMayAsk(ctx, buildSource{Repository: "novox/mesh-catalog", Seat: "git"},
"http://forge.internal:20000/novox/mesh-catalog.git"); err != nil {
t.Fatalf("a verb's build of the catalogue repository was refused: %v", err)
}
t.Setenv(verbVar, "")
t.Setenv(link.CallerVar, "")
if err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, ""); err != nil {
t.Fatalf("the terminal was refused: %v", err)
}
}
func TestARollbackNeverPutsBackABuildFromAnotherRepository(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
inv := open.inventory
fork := onTrunk("build-1791500000000000000", "agent/mesh-catalog", "git", "modules/sudo",
map[string]any{"module": "sudo", "version": "evil"})
fork.Commit = "c0ffee0123456789" // the commit sudo was registered at
keptAsked(t, inv, fork.ID, "agent/mesh-catalog", "modules/sudo", false)
if _, _, err := takeIn(ctx, inv, fork); !errors.Is(err, errNotItsSource) {
t.Fatalf("the fork's build was taken in: %v", err)
}
// Asked after the registered build, whenever the test runs: an id naming a fixed moment read as older
// than the registered build once the clock passed it (2026-10-10 02:40 UTC), and the test failed on main.
failed := onTrunk(fmt.Sprintf("build-%d", time.Now().Add(time.Hour).UnixNano()), "novox/mesh-catalog", "git", "modules/sudo",
map[string]any{"module": "sudo", "version": "2"})
failed.Commit = "badbadbad0123456"
if _, _, err := takeIn(ctx, inv, failed); err != nil {
t.Fatal(err)
}
record, _, err := inv.BuildByID(ctx, failed.ID)
if err != nil {
t.Fatal(err)
}
previous, found, err := inv.PreviousBuild(ctx, "sudo", "c0ffee0123456789", record)
if err != nil {
t.Fatal(err)
}
if found && previous.ID == fork.ID {
t.Fatalf("a rollback would put back the fork's build %s", previous.ID)
}
if !found || previous.ID != "build-sudo" {
t.Fatalf("a rollback puts back %q (found %v), want the registered build-sudo", previous.ID, found)
}
}
func TestATrunkTheForgeDoesNotProtectRegistersNothing(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
theForge.Store("novox/unguarded", forgeFacts{ID: 7, Why: "the rule main lets a person push to main directly"})
t.Cleanup(func() { theForge.Delete("novox/unguarded") })
first := onTrunk("build-unguarded-1", "novox/unguarded", "git", "", map[string]any{"module": "unguarded", "version": "1"})
keptAsked(t, open.inventory, first.ID, "novox/unguarded", "", true)
if _, _, err := takeIn(ctx, open.inventory, first); err != nil {
t.Fatalf("at the terminal: %v", err)
}
again := onTrunk("build-unguarded-2", "novox/unguarded", "git", "", map[string]any{"module": "unguarded", "version": "2"})
if _, _, err := takeIn(ctx, open.inventory, again); !errors.Is(err, errNotItsSource) ||
!strings.Contains(err.Error(), "push to main directly") {
t.Fatalf("a rebuild from an unprotected trunk was taken in: %v", err)
}
theForge.Store("novox/unguarded", errors.New("nothing serves gitea.gitea_api"))
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-unguarded-3", "novox/unguarded", "git", "",
map[string]any{"module": "unguarded", "version": "3"})); !errors.Is(err, errNotItsSource) {
t.Fatalf("a forge that could not say was read as a protected trunk: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["unguarded"].Version != "1" {
t.Fatalf("unguarded is %q", shelf["unguarded"].Version)
}
}
func TestARepositoryMadeAgainUnderItsNameIsNotTheModulesSource(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
theForge.Store("novox/remade", forgeFacts{ID: 100, Guarded: true})
t.Cleanup(func() { theForge.Delete("novox/remade") })
first := onTrunk("build-remade-1", "novox/remade", "git", "", map[string]any{"module": "remade", "version": "1"})
keptAsked(t, open.inventory, first.ID, "novox/remade", "", true)
if _, _, err := takeIn(ctx, open.inventory, first); err != nil {
t.Fatal(err)
}
if id, _ := open.inventory.SourceIdentity(ctx, "remade"); id != 100 {
t.Fatalf("the forge's id was not recorded: %d", id)
}
theForge.Store("novox/remade", forgeFacts{ID: 101, Guarded: true}) // deleted, and made again by an agent
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-remade-2", "novox/remade", "git", "",
map[string]any{"module": "remade", "version": "evil"})); !errors.Is(err, errNotItsSource) ||
!strings.Contains(err.Error(), "made again") {
t.Fatalf("a repository made again under the name was taken in: %v", err)
}
// And a new module from it, beside the one registered from the first, the same.
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-remade-3", "novox/remade", "git", "modules/other",
map[string]any{"module": "other", "version": "1"})); !errors.Is(err, errNotItsSource) {
t.Fatalf("a new module from a repository made again was taken in: %v", err)
}
}
func TestATerminalRequestCoversOnlyWhatItAsked(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
if err := open.inventory.RecordBuildRequest(ctx, inventory.BuildRequest{ID: "build-asked", Repository: "novox/app",
Seat: "git", Path: "modules/app", For: "build", AtTerminal: true}); err != nil {
t.Fatal(err)
}
other := onTrunk("build-asked", "agent/sudo", "git", "modules/app", map[string]any{"module": "sudo", "version": "evil"})
if _, _, err := takeIn(ctx, open.inventory, other); !errors.Is(err, errNotItsSource) {
t.Fatalf("an outcome of another repository under a terminal request's id was taken in: %v", err)
}
elsewhere := onTrunk("build-asked", "novox/app", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "evil"})
if _, _, err := takeIn(ctx, open.inventory, elsewhere); !errors.Is(err, errNotItsSource) {
t.Fatalf("an outcome of another path under a terminal request's id was taken in: %v", err)
}
}
func TestTheServingControllerIsNeverTheTerminal(t *testing.T) {
t.Setenv(verbVar, "")
t.Setenv(link.CallerVar, "")
t.Setenv(servedVar, "")
if !startedAtTheTerminal() {
t.Fatal("a process started by hand is not the terminal")
}
markServed()
if startedAtTheTerminal() {
t.Fatal("the serving controller reads as the terminal")
}
child := exec.Command(os.Args[0], "-test.run=^$")
child.Env = os.Environ()
if !slices.Contains(child.Env, servedVar+"=1") {
t.Fatal("what the serving controller starts does not carry its mark")
}
}
func TestAnUnknownIdentityKeepsTheRecordedOne(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
if err := open.inventory.SetSourceIdentity(ctx, "sudo", 100); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetSourceIdentity(ctx, "sudo", 0); err != nil {
t.Fatal(err)
}
if id, _ := open.inventory.SourceIdentity(ctx, "sudo"); id != 100 {
t.Fatalf("the recorded id became %d", id)
}
}
func TestAModuleMovesOnTheOperatorsApproveOfTheMergeThatMovedIt(t *testing.T) {
const commit = "c0ffee0123456789"
for _, c := range []struct {
what string
change func(*anApprovedMerge)
unkept bool
request string // who asked: "merge", or "build" through a verb
moved bool
}{
{what: "the operator's Approve of the merged head", request: "merge", moved: true},
{what: "no merge mesh-delivery announced", unkept: true, request: "merge"},
{what: "a warrant for other heads", request: "merge", change: func(a *anApprovedMerge) {
a.merge.Heads = map[string]string{"novox/mesh-host@aaaaaaaaaaaa": "e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5"}
}},
{what: "a Decline", request: "merge", change: func(a *anApprovedMerge) {
a.warrant.Option, a.warrant.Label, a.merge.Option = "decline", "Decline", "decline"
}},
{what: "a person who is not the operator", request: "merge", change: func(a *anApprovedMerge) { a.warrant.By.Who = "agent" }},
{what: "a build asked through a verb", request: "build"},
} {
open := theCatalogue(t)
ctx := t.Context()
inv := open.inventory
a := approvedMerge(t)
a.merge.Owner, a.merge.Repo, a.merge.Number, a.merge.HeadSHA = "novox", "mesh-host", 7, "a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1"
if c.change != nil {
c.change(&a)
}
wasKept, wasAsk := keptDeliveryMerge, routerAskOf
keptDeliveryMerge = func(_ context.Context, owner, repo string, number int, head string) (deliveryMerge, bool, error) {
return a.merge, !c.unkept && owner == "novox" && repo == "mesh-host" && number == 7 && head == a.merge.HeadSHA, nil
}
routerAskOf = func(_ context.Context, _ *inventory.Inventory, id string) (*asks.Ask, *asks.Warrant, error) {
if id != a.merge.AskID {
t.Errorf("%s: the router was asked for %q", c.what, id)
}
return &a.ask, &a.warrant, nil
}
t.Cleanup(func() { keptDeliveryMerge, routerAskOf = wasKept, wasAsk })
event, _ := json.Marshal(map[string]any{"owner": "novox", "repo": "mesh-host", "base": "main", "number": 7,
"head_sha": a.merge.HeadSHA, "merge_commit_sha": commit})
if _, err := inv.AddMerge(ctx, inventory.BatchedMerge{Repository: "novox/mesh-host", Branch: "main", Commit: commit,
Merged: time.Now(), Heard: time.Now(), Event: event}); err != nil {
t.Fatal(err)
}
id := fmt.Sprintf("build-%d", time.Now().Add(time.Hour).UnixNano())
if err := inv.RecordBuildRequest(ctx, inventory.BuildRequest{ID: id, Repository: "novox/mesh-host", Seat: "git",
Path: "modules/sudo", Ref: "main", Commit: commit, For: c.request}); err != nil {
t.Fatal(err)
}
moved := onTrunk(id, "novox/mesh-host", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "2"})
moved.Commit = commit
_, _, err := takeIn(ctx, inv, moved)
src, _ := inv.SourceOf(ctx, "sudo")
switch {
case c.moved && err != nil:
t.Errorf("%s: the move was refused: %v", c.what, err)
case c.moved && src.Repository != "novox/mesh-host":
t.Errorf("%s: sudo is built from %q", c.what, src.Repository)
case !c.moved && !errors.Is(err, errNotItsSource):
t.Errorf("%s: the move was not refused as ADR 0266 has it: %v", c.what, err)
case !c.moved && src.Repository != "novox/mesh-catalog":
t.Errorf("%s: a refused move changed sudo's source to %q", c.what, src.Repository)
}
}
}
+6 -258
View File
@@ -3,11 +3,7 @@ package main
import (
"context"
"encoding/json"
"errors"
"hash/fnv"
"os"
"os/exec"
"slices"
"strings"
"sync"
"testing"
@@ -43,141 +39,14 @@ func keptAsked(t *testing.T, inv *inventory.Inventory, id, repository, path stri
}
// theCatalogue is a mesh whose sudo is built from the catalogue repository and whose node-engine from its own.
func theCatalogue(t *testing.T) *stores {
t.Helper()
open := aMesh(t)
ctx := t.Context()
for _, b := range []link.BuildResult{
onTrunk("build-sudo", "novox/mesh-catalog", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "1"}),
onTrunk("build-host", "novox/mesh-host", "git", "", map[string]any{"module": "mesh-host", "version": "1"}),
} {
keptAsked(t, open.inventory, b.ID, b.Source.Repository, b.Path, true)
if _, _, err := takeIn(ctx, open.inventory, b); err != nil {
t.Fatal(err)
}
}
return open
}
func TestABuildFromAnAgentsRepositoryIsNotRegisteredAsAModuleTheMeshHolds(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
for _, c := range []struct {
name, repository, path, module string
}{
{"its own repository naming sudo", "agent/sudo", "modules/sudo", "sudo"},
{"a fork of the catalogue", "agent/mesh-catalog", "modules/sudo", "sudo"},
{"a fork of the node-engine", "agent/mesh-host", "", "mesh-host"},
} {
t.Run(c.name, func(t *testing.T) {
id := "build-" + strings.ReplaceAll(c.repository, "/", "-")
keptAsked(t, open.inventory, id, c.repository, c.path, false) // through the build verb
evil := onTrunk(id, c.repository, "git", c.path, map[string]any{"module": c.module, "version": "evil"})
_, _, err := takeIn(ctx, open.inventory, evil)
if !errors.Is(err, errNotItsSource) {
t.Fatalf("a build of %s was taken in as %s: %v", c.repository, c.module, err)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if got := shelf[c.module].Version; got != "1" {
t.Fatalf("%s is now %q, from %s", c.module, got, c.repository)
}
if _, found, _ := open.inventory.BuildByID(ctx, id); !found {
t.Errorf("the refused build %s is not recorded", id)
}
})
}
}
func TestANewModuleFromARepositoryTheCatalogueDoesNotBuildFromIsNotRegistered(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
keptAsked(t, open.inventory, "build-new", "agent/tools", "", false)
_, _, err := takeIn(ctx, open.inventory, onTrunk("build-new", "agent/tools", "git", "",
map[string]any{"module": "agent-tools", "version": "1"}))
if !errors.Is(err, errNotItsSource) {
t.Fatalf("a new module from an agent's repository was taken in: %v", err)
}
// And one asked of nobody here — an outcome on the bus no request was kept for — the same.
_, _, err = takeIn(ctx, open.inventory, onTrunk("build-unasked", "agent/tools", "git", "",
map[string]any{"module": "agent-tools", "version": "1"}))
if !errors.Is(err, errNotItsSource) {
t.Fatalf("an outcome nobody asked for was taken in: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["agent-tools"].Module != "" {
t.Fatal("the refused module is in the catalogue")
}
}
// The operator at the terminal may still move a module, or add one from a new repository.
func TestAtTheTerminalAnotherRepositoryIsRegistered(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
keptAsked(t, open.inventory, "build-moved", "novox/sudo", "", true)
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-moved", "novox/sudo", "git", "",
map[string]any{"module": "sudo", "version": "2"})); err != nil {
t.Fatalf("a move the operator asked for at the terminal was refused: %v", err)
}
if src, _ := open.inventory.SourceOf(ctx, "sudo"); src.Repository != "novox/sudo" {
t.Fatalf("sudo is built from %q", src.Repository)
}
if err := open.inventory.RecordBuildRequest(ctx, inventory.BuildRequest{ID: "build-external",
Repository: "http://forge.internal:20000/someone/app.git", For: "build", AtTerminal: true}); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-external", "someone/app", "", "",
map[string]any{"module": "app", "version": "1"})); err != nil {
t.Fatalf("a new module the operator asked for at the terminal was refused: %v", err)
}
}
// The delivery's flow is untouched: a merge's rebuild of a module from its own repository, and a merge adding
// a module to a repository the catalogue builds from (novox/hq issue 300), are registered with no terminal.
func TestADeliveryFromTheRegisteredRepositoryIsRegistered(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
rebuilt := onTrunk("build-plan", "novox/mesh-catalog", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "2"})
if err := open.inventory.RecordBuildRequest(ctx, inventory.BuildRequest{ID: rebuilt.ID,
Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/sudo", For: "plan"}); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, rebuilt); err != nil {
t.Fatalf("a plan's build of the module's own repository was refused: %v", err)
}
added := onTrunk("build-merge", "novox/mesh-catalog", "git", "modules/zram", map[string]any{"module": "zram", "version": "1"})
if _, _, err := takeIn(ctx, open.inventory, added); err != nil {
t.Fatalf("a module a merge added to the catalogue repository was refused: %v", err)
}
shelf, _ := open.inventory.Catalogue(ctx)
if shelf["sudo"].Version != "2" || shelf["zram"].Module == "" {
t.Fatalf("not registered: sudo %q, zram %q", shelf["sudo"].Version, shelf["zram"].Module)
}
}
// Through a verb, a build of a repository the catalogue builds nothing from is not even asked: the build node
// would run what the agent wrote.
func TestAVerbAsksNoBuildOfARepositoryTheCatalogueDoesNotBuildFrom(t *testing.T) {
theCatalogue(t)
ctx := t.Context()
t.Setenv(verbVar, "build")
t.Setenv(link.CallerVar, "node-tools.anchor, through the mesh-controller seat")
err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, "http://forge.internal:20000/agent/sudo.git")
var policy *heldAtTheTerminal
if !errors.As(err, &policy) {
t.Fatalf("a verb's build of an agent's repository was asked: %v", err)
}
if err := verbMayAsk(ctx, buildSource{Repository: "novox/mesh-catalog", Seat: "git"},
"http://forge.internal:20000/novox/mesh-catalog.git"); err != nil {
t.Fatalf("a verb's build of the catalogue repository was refused: %v", err)
}
t.Setenv(verbVar, "")
t.Setenv(link.CallerVar, "")
if err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, ""); err != nil {
t.Fatalf("the terminal was refused: %v", err)
}
}
// asTheOperator keeps a build as asked at the controller's terminal, as the operator's first build of a module
// from a repository the catalogue does not yet build from is (novox/hq ADR 0266), and hands it back.
@@ -197,38 +66,6 @@ func asTheOperator(t *testing.T, inv *inventory.Inventory, b link.BuildResult) l
// A rollback puts back only a build of the module's own repository: an agent's build of the module's name,
// recorded and refused, at the very commit the machine ran before (a fork carries it), is never registered by
// the back door of a failed gate.
func TestARollbackNeverPutsBackABuildFromAnotherRepository(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
inv := open.inventory
fork := onTrunk("build-1791500000000000000", "agent/mesh-catalog", "git", "modules/sudo",
map[string]any{"module": "sudo", "version": "evil"})
fork.Commit = "c0ffee0123456789" // the commit sudo was registered at
keptAsked(t, inv, fork.ID, "agent/mesh-catalog", "modules/sudo", false)
if _, _, err := takeIn(ctx, inv, fork); !errors.Is(err, errNotItsSource) {
t.Fatalf("the fork's build was taken in: %v", err)
}
failed := onTrunk("build-1791600000000000000", "novox/mesh-catalog", "git", "modules/sudo",
map[string]any{"module": "sudo", "version": "2"})
failed.Commit = "badbadbad0123456"
if _, _, err := takeIn(ctx, inv, failed); err != nil {
t.Fatal(err)
}
record, _, err := inv.BuildByID(ctx, failed.ID)
if err != nil {
t.Fatal(err)
}
previous, found, err := inv.PreviousBuild(ctx, "sudo", "c0ffee0123456789", record)
if err != nil {
t.Fatal(err)
}
if found && previous.ID == fork.ID {
t.Fatalf("a rollback would put back the fork's build %s", previous.ID)
}
if !found || previous.ID != "build-sudo" {
t.Fatalf("a rollback puts back %q (found %v), want the registered build-sudo", previous.ID, found)
}
}
// theForge is what the forge says in a test, by owner/name: a repository not named here is protected as a
// trunk must be, with an id of its own.
@@ -252,97 +89,15 @@ func init() {
// A module's trunk the forge does not protect — direct pushes, no required status — or a forge that cannot say,
// registers nothing through a verb: the trunk rule means nothing on a branch anyone pushes to.
func TestATrunkTheForgeDoesNotProtectRegistersNothing(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
theForge.Store("novox/unguarded", forgeFacts{ID: 7, Why: "the rule main lets a person push to main directly"})
t.Cleanup(func() { theForge.Delete("novox/unguarded") })
first := onTrunk("build-unguarded-1", "novox/unguarded", "git", "", map[string]any{"module": "unguarded", "version": "1"})
keptAsked(t, open.inventory, first.ID, "novox/unguarded", "", true)
if _, _, err := takeIn(ctx, open.inventory, first); err != nil {
t.Fatalf("at the terminal: %v", err)
}
again := onTrunk("build-unguarded-2", "novox/unguarded", "git", "", map[string]any{"module": "unguarded", "version": "2"})
if _, _, err := takeIn(ctx, open.inventory, again); !errors.Is(err, errNotItsSource) ||
!strings.Contains(err.Error(), "push to main directly") {
t.Fatalf("a rebuild from an unprotected trunk was taken in: %v", err)
}
theForge.Store("novox/unguarded", errors.New("nothing serves gitea.gitea_api"))
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-unguarded-3", "novox/unguarded", "git", "",
map[string]any{"module": "unguarded", "version": "3"})); !errors.Is(err, errNotItsSource) {
t.Fatalf("a forge that could not say was read as a protected trunk: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["unguarded"].Version != "1" {
t.Fatalf("unguarded is %q", shelf["unguarded"].Version)
}
}
// A repository deleted and made again under the module's repository's name is another repository: the forge's
// id, recorded at registration, tells them apart.
func TestARepositoryMadeAgainUnderItsNameIsNotTheModulesSource(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
theForge.Store("novox/remade", forgeFacts{ID: 100, Guarded: true})
t.Cleanup(func() { theForge.Delete("novox/remade") })
first := onTrunk("build-remade-1", "novox/remade", "git", "", map[string]any{"module": "remade", "version": "1"})
keptAsked(t, open.inventory, first.ID, "novox/remade", "", true)
if _, _, err := takeIn(ctx, open.inventory, first); err != nil {
t.Fatal(err)
}
if id, _ := open.inventory.SourceIdentity(ctx, "remade"); id != 100 {
t.Fatalf("the forge's id was not recorded: %d", id)
}
theForge.Store("novox/remade", forgeFacts{ID: 101, Guarded: true}) // deleted, and made again by an agent
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-remade-2", "novox/remade", "git", "",
map[string]any{"module": "remade", "version": "evil"})); !errors.Is(err, errNotItsSource) ||
!strings.Contains(err.Error(), "made again") {
t.Fatalf("a repository made again under the name was taken in: %v", err)
}
// And a new module from it, beside the one registered from the first, the same.
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-remade-3", "novox/remade", "git", "modules/other",
map[string]any{"module": "other", "version": "1"})); !errors.Is(err, errNotItsSource) {
t.Fatalf("a new module from a repository made again was taken in: %v", err)
}
}
// The terminal's mark is the operator's for the repository and path they asked: an outcome of another, under that
// build's id, is not theirs.
func TestATerminalRequestCoversOnlyWhatItAsked(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
if err := open.inventory.RecordBuildRequest(ctx, inventory.BuildRequest{ID: "build-asked", Repository: "novox/app",
Seat: "git", Path: "modules/app", For: "build", AtTerminal: true}); err != nil {
t.Fatal(err)
}
other := onTrunk("build-asked", "agent/sudo", "git", "modules/app", map[string]any{"module": "sudo", "version": "evil"})
if _, _, err := takeIn(ctx, open.inventory, other); !errors.Is(err, errNotItsSource) {
t.Fatalf("an outcome of another repository under a terminal request's id was taken in: %v", err)
}
elsewhere := onTrunk("build-asked", "novox/app", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "evil"})
if _, _, err := takeIn(ctx, open.inventory, elsewhere); !errors.Is(err, errNotItsSource) {
t.Fatalf("an outcome of another path under a terminal request's id was taken in: %v", err)
}
}
// The serving controller, and everything it starts, is never the terminal: a build asked in it reads as asked
// through the mesh even when no verb and no caller is named.
func TestTheServingControllerIsNeverTheTerminal(t *testing.T) {
t.Setenv(verbVar, "")
t.Setenv(link.CallerVar, "")
t.Setenv(servedVar, "")
if !startedAtTheTerminal() {
t.Fatal("a process started by hand is not the terminal")
}
markServed()
if startedAtTheTerminal() {
t.Fatal("the serving controller reads as the terminal")
}
child := exec.Command(os.Args[0], "-test.run=^$")
child.Env = os.Environ()
if !slices.Contains(child.Env, servedVar+"=1") {
t.Fatal("what the serving controller starts does not carry its mark")
}
}
// The forge applies one rule to a branch: the rule named for it, else the first glob that covers it. A stronger
// rule later in the list is not what guards the branch, and is not read as if it were (the confirmation review).
@@ -365,16 +120,9 @@ func TestTheRuleJudgedIsTheOneTheForgeApplies(t *testing.T) {
}
// An id the forge could not give keeps the id already recorded.
func TestAnUnknownIdentityKeepsTheRecordedOne(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
if err := open.inventory.SetSourceIdentity(ctx, "sudo", 100); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetSourceIdentity(ctx, "sudo", 0); err != nil {
t.Fatal(err)
}
if id, _ := open.inventory.SourceIdentity(ctx, "sudo"); id != 100 {
t.Fatalf("the recorded id became %d", id)
}
}
// **The operator's Approve of a reviewed group registers a moved module from its new repository** (novox/hq ADR 0420 §7,
// issue 1025). sudo is built from the catalogue; a merge into the repository of node-engine's module, announced by
// mesh-delivery with the ask it merged on, brings its build from there. Accepted with the operator's Approve of the
// merged head; refused with no kept merge, a warrant for other heads, a Decline, a person who is not the operator, and
// through a verb.
-118
View File
@@ -1,13 +1,9 @@
package main
import (
"encoding/json"
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -15,126 +11,12 @@ import (
// the module registered with its source as the seat and path when the request said so — never the
// URL. A definition naming an installation is recorded and not registered; a failure is recorded
// and said.
func TestABuildHeardIsRecordedAndRegistered(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
manifest, _ := json.Marshal(map[string]any{"module": "shop", "version": "3"})
m, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory, link.BuildResult{
ID: "b-1", Repository: "http://forge.internal:20000/novox/shop.git", Path: "modules/shop",
Ref: "main", On: "anchor", Commit: "abcdef0123", Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/shop"},
}))
if err != nil {
t.Fatal(err)
}
if m.Module != "shop" {
t.Fatalf("registered %q", m.Module)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if _, held := shelf["shop"]; !held {
t.Fatal("the module a heard build produced is not in the catalogue")
}
src, err := open.inventory.SourceOf(ctx, "shop")
if err != nil || src.Seat != "git" || src.Repository != "novox/shop" || src.BuiltFrom != "abcdef0123" {
t.Fatalf("the source is the seat and the path, never the URL: %+v %v", src, err)
}
builds, err := open.inventory.Builds(ctx, "shop", 5)
if err != nil || len(builds) != 1 || builds[0].ID != "b-1" {
t.Fatalf("the build is not recorded once: %v %v", builds, err)
}
named, _ := json.Marshal(map[string]any{"module": "idp", "version": "1", "resources": []any{
map[string]any{"id": "server", "type": "container", "image": "x@sha256:aa",
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}}}})
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{
ID: "b-2", Repository: "/r", On: "anchor", Commit: "0123456789", Manifest: named})
if err == nil || !strings.Contains(err.Error(), "does not register it") {
t.Fatalf("a definition naming an installation was taken in: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["idp"].Module != "" {
t.Fatal("the refused module was registered anyway")
}
if builds, _ := open.inventory.Builds(ctx, "idp", 5); len(builds) != 1 {
t.Fatalf("the refused build was not recorded: %v", builds)
}
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{ID: "b-3", Repository: "/r", On: "anchor", Failed: "no compiler"})
if err == nil || !strings.Contains(err.Error(), "no compiler") {
t.Fatalf("a failure is said in the builder's words: %v", err)
}
}
// novox/hq 04-ISSUES/215: a build asked at a commit is recorded as built from that commit, and the
// module keeps following the branch it followed — a new one, the default branch.
func TestABuildAtACommitKeepsTheBranchTheModuleFollows(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
manifest, _ := json.Marshal(map[string]any{"module": "unifi", "version": "1"})
result := func(id, ref, commit string) link.BuildResult {
return link.BuildResult{ID: id, Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/unifi", Ref: ref, On: "anchor", Commit: commit, Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
}
if _, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory, result("b-1", "main", "1111111aaaa"))); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, result("b-2", "9c97a8a", "9c97a8a1d2c3")); err != nil {
t.Fatal(err)
}
src, err := open.inventory.SourceOf(ctx, "unifi")
if err != nil {
t.Fatal(err)
}
if src.Ref != "main" || src.BuiltFrom != "9c97a8a1d2c3" {
t.Errorf("after a build at a commit the module follows %q, built from %q; want main, 9c97a8a1d2c3", src.Ref, src.BuiltFrom)
}
// One new to the catalogue, first built at a commit, follows the default branch.
other, _ := json.Marshal(map[string]any{"module": "letta", "version": "1"})
r := result("b-3", "deadbeef", "deadbeefcafe")
r.Manifest, r.Path = other, "modules/letta"
if _, _, err := takeIn(ctx, open.inventory, r); err != nil {
t.Fatal(err)
}
if src, _ := open.inventory.SourceOf(ctx, "letta"); src.Ref != "" {
t.Errorf("a module first built at a commit follows %q, want the default branch", src.Ref)
}
}
// novox/hq 04-ISSUES/219: an older request heard after a newer one is recorded and not registered,
// so a push sends what the newer request built.
func TestAnOlderBuildHeardLaterDoesNotReplaceTheNewer(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
older := time.Date(2026, 10, 3, 21, 33, 45, 0, time.UTC)
newer := time.Date(2026, 10, 3, 21, 51, 57, 0, time.UTC)
result := func(asked time.Time, image string) link.BuildResult {
manifest, _ := json.Marshal(map[string]any{"module": "postgres", "version": image})
return link.BuildResult{ID: link.NewBuildID(asked), Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/postgres", Ref: "main", On: "anchor", Commit: "efff5415", Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
}
if _, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory, result(newer, "4bcd5f73"))); err != nil {
t.Fatal(err)
}
_, _, err := takeIn(ctx, open.inventory, result(older, "0ab07fa9"))
if !errors.Is(err, inventory.ErrSuperseded) {
t.Fatalf("the older request's outcome was taken in as current: %v", err)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if got := shelf["postgres"].Version; got != "4bcd5f73" {
t.Errorf("postgres is %q; want the newer request's 4bcd5f73", got)
}
if builds, _ := open.inventory.Builds(ctx, "postgres", 5); len(builds) != 2 {
t.Errorf("the late build was not recorded: %v", builds)
}
}
func TestABuildIDSaysWhenItWasAsked(t *testing.T) {
at := time.Date(2026, 10, 3, 21, 51, 57, 392539762, time.UTC)
@@ -0,0 +1,92 @@
package main
import (
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// **Every user's slow consumer is S9's, not the controller's alone** (novox/hq ADR 0357, issue 270). On
// 2026-10-07 the bus closed two clients on the laptop as slow consumers and nothing raised a condition:
// the controller heard only its own connection. Said by the bus module and read back, a user the bus
// closes is one condition naming the module and its machine, raised at the first look and cleared an hour
// after the last closure.
func TestAUserTheBusClosedAsASlowConsumerIsOneConditionNamingItsModuleAndMachine(t *testing.T) {
closed := time.Date(2026, 10, 14, 12, 0, 0, 0, time.UTC)
now := closed.Add(20 * time.Second)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
advisories := link.BusClosures([]link.BusClosure{
{User: "laptop.claude-code", CID: 10, Reason: "Slow Consumer (Write Deadline)", Opened: closed.Add(-time.Hour),
Closed: closed.Add(-10 * time.Minute)},
{User: "laptop.claude-code", CID: 44, Reason: "Slow Consumer (Write Deadline)", Opened: closed.Add(-time.Minute),
Closed: closed},
{User: "anchor.desk", CID: 7, Reason: "Maximum Message Payload Exceeded", Opened: closed.Add(-time.Hour),
Closed: closed},
}, []string{link.AdvisorySlowConsumer, link.AdvisorySlowConsumer, link.AdvisoryConnectionClosed}, closed.Add(-time.Hour))
look := func() {
t.Helper()
f := &signalFacts{now: now, host: "anchor", advisories: advisories}
if err := k.Reconcile(t.Context(), "S9", watchAdvisories(f)); err != nil {
t.Fatal(err)
}
}
look()
slow, found, err := conditions.ReadOne(t.Context(), store, "bus.laptop.claude-code.slow-consumer")
if err != nil || !found {
t.Fatalf("no slow-consumer condition for the user: %v %v", found, err)
}
if slow.Subject.Machine != "laptop" || slow.Observations != 2 || !slow.LastObserved.Equal(closed) {
t.Fatalf("the slow consumer is said on %q, %d closure(s), last %s: %+v", slow.Subject.Machine, slow.Observations,
slow.LastObserved, slow)
}
if !strings.Contains(slow.Summary, "claude-code on laptop") || strings.Contains(slow.Summary, "laptop.claude-code") {
t.Fatalf("the summary does not name the module in words, or carries the dotted user: %q", slow.Summary)
}
if !strings.Contains(slow.Headline, "Claude-code on laptop") {
t.Fatalf("the headline is %q", slow.Headline)
}
cut, found, err := conditions.ReadOne(t.Context(), store, "bus.anchor.desk.connection-closed")
if err != nil || !found || cut.Subject.Machine != "anchor" || !strings.Contains(cut.Summary, "maximum message payload exceeded") {
t.Fatalf("the user cut off for breaking the bus's rules: %v %v %+v", found, err, cut)
}
// An hour after the newest closure without another, both clear.
now = closed.Add(time.Hour + time.Minute)
look()
for _, key := range []string{"bus.laptop.claude-code.slow-consumer", "bus.anchor.desk.connection-closed"} {
if _, found, _ := k.Get(t.Context(), key); found {
t.Fatalf("%s is still open an hour after the last closure", key)
}
}
}
// **A read of the bus module's events that fails does not blind S9** (review of #326): the controller's own
// signals are still said, and the failed read is a condition of its own.
func TestABusModuleReadThatFailsKeepsTheControllersOwnSignals(t *testing.T) {
at := time.Date(2026, 10, 14, 12, 0, 0, 0, time.UTC)
f := &signalFacts{now: at.Add(time.Minute), host: "anchor",
advisories: []link.Advisory{{Kind: link.AdvisorySlowConsumer, ID: "controller", Said: "the controller fell behind",
First: at, Last: at, Count: 1}},
busFaultsErr: errors.New("nats: timeout")}
keys := map[string]bool{}
for _, o := range watchAdvisories(f) {
keys[o.Key()] = true
}
if !keys["bus.controller.slow-consumer"] || !keys["bus.bus-module.bus-faults-unread"] {
t.Fatalf("observed %v: the controller's own slow consumer and the unread events, both", keys)
}
for _, row := range signalsTable {
if row.Row == "S9" && row.needs(f) != nil {
t.Fatal("the S9 row went blind for a read of the bus module's events")
}
}
}
@@ -0,0 +1,70 @@
//go:build integration
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
func TestASendHeldForTheBusStepIsFoundFromTheWalksItHolds(t *testing.T) {
now := time.Date(2026, 10, 8, 18, 30, 0, 0, time.UTC)
b := aBusOnAnchor()
walk := func(id, repo, note string, updated time.Time) inventory.Plan {
return inventory.Plan{ID: id, Repository: repo, Commit: "c0ffee001122", State: inventory.PlanRolling, Tier: 0,
Tiers: [][]string{{"mesh-host"}}, Modules: map[string]*inventory.PlanModule{"mesh-host": {}}, Note: note,
Updated: updated}
}
for _, c := range []struct {
name string
bus busPending
plans []inventory.Plan
first map[string]time.Time
want []string
since time.Time
}{
{"a walk refused for the bus", b,
[]inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now.Add(-27*time.Minute))}, nil,
[]string{"plan-1"}, now.Add(-27 * time.Minute)},
{"refused earlier than its last save, as this controller saw it", b,
[]inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now.Add(-5*time.Minute))},
map[string]time.Time{"plan-1": now.Add(-28 * time.Minute)}, []string{"plan-1"}, now.Add(-28 * time.Minute)},
{"a walk refused for another reason", b,
[]inventory.Plan{walk("plan-1", "novox/mesh-host", "tier 0: the build seat is paused — tried again", now)}, nil,
nil, time.Time{}},
{"refused for an older bus build than the one held now", func() busPending { o := b; o.to = "99999999cccc"; return o }(),
[]inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now)}, nil, nil, time.Time{}},
{"the bus already runs the build held: its step started", func() busPending {
o := aBusOnAnchor()
o.from = map[string]string{"anchor": o.to}
return o
}(), []inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now)}, nil, nil, time.Time{}},
{"a walk waiting for its delivery's word asks no send", b, func() []inventory.Plan {
p := walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now)
p.Delivery = &inventory.PlanDelivery{Awaits: "mesh-delivery"}
return []inventory.Plan{p}
}(), nil, nil, time.Time{}},
} {
t.Run(c.name, func(t *testing.T) {
got := busWaitsOf(c.plans, c.bus, func(id string) time.Time { return c.first[id] })
var ids []string
for _, w := range got.waits {
ids = append(ids, w.plan)
}
if strings.Join(ids, ",") != strings.Join(c.want, ",") {
t.Fatalf("held for the bus: %v, want %v", ids, c.want)
}
if len(c.want) > 0 {
if !got.waits[0].since.Equal(c.since) {
t.Errorf("waiting since %s, want %s", got.waits[0].since, c.since)
}
if strings.Join(got.machines, ",") != "anchor" || got.to != b.to || got.module != "nats" {
t.Errorf("behind %+v", got)
}
}
})
}
}
-60
View File
@@ -7,7 +7,6 @@ import (
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// novox/hq issue 336: a send held because it would replace the bus outside its planned step waits for a
@@ -27,65 +26,6 @@ func refusedNote(b busPending, tier int) string {
return "tier " + string(rune('0'+tier)) + ": " + errBusWaits.Error() + ": " + held + " — tried again"
}
func TestASendHeldForTheBusStepIsFoundFromTheWalksItHolds(t *testing.T) {
now := time.Date(2026, 10, 8, 18, 30, 0, 0, time.UTC)
b := aBusOnAnchor()
walk := func(id, repo, note string, updated time.Time) inventory.Plan {
return inventory.Plan{ID: id, Repository: repo, Commit: "c0ffee001122", State: inventory.PlanRolling, Tier: 0,
Tiers: [][]string{{"mesh-host"}}, Modules: map[string]*inventory.PlanModule{"mesh-host": {}}, Note: note,
Updated: updated}
}
for _, c := range []struct {
name string
bus busPending
plans []inventory.Plan
first map[string]time.Time
want []string
since time.Time
}{
{"a walk refused for the bus", b,
[]inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now.Add(-27*time.Minute))}, nil,
[]string{"plan-1"}, now.Add(-27 * time.Minute)},
{"refused earlier than its last save, as this controller saw it", b,
[]inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now.Add(-5*time.Minute))},
map[string]time.Time{"plan-1": now.Add(-28 * time.Minute)}, []string{"plan-1"}, now.Add(-28 * time.Minute)},
{"a walk refused for another reason", b,
[]inventory.Plan{walk("plan-1", "novox/mesh-host", "tier 0: the build seat is paused — tried again", now)}, nil,
nil, time.Time{}},
{"refused for an older bus build than the one held now", func() busPending { o := b; o.to = "99999999cccc"; return o }(),
[]inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now)}, nil, nil, time.Time{}},
{"the bus already runs the build held: its step started", func() busPending {
o := aBusOnAnchor()
o.from = map[string]string{"anchor": o.to}
return o
}(), []inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now)}, nil, nil, time.Time{}},
{"a walk waiting for its delivery's word asks no send", b, func() []inventory.Plan {
p := walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now)
p.Delivery = &inventory.PlanDelivery{Awaits: "mesh-delivery"}
return []inventory.Plan{p}
}(), nil, nil, time.Time{}},
} {
t.Run(c.name, func(t *testing.T) {
got := busWaitsOf(c.plans, c.bus, func(id string) time.Time { return c.first[id] })
var ids []string
for _, w := range got.waits {
ids = append(ids, w.plan)
}
if strings.Join(ids, ",") != strings.Join(c.want, ",") {
t.Fatalf("held for the bus: %v, want %v", ids, c.want)
}
if len(c.want) > 0 {
if !got.waits[0].since.Equal(c.since) {
t.Errorf("waiting since %s, want %s", got.waits[0].since, c.since)
}
if strings.Join(got.machines, ",") != "anchor" || got.to != b.to || got.module != "nats" {
t.Errorf("behind %+v", got)
}
}
})
}
}
// **Said at once, not as lateness, and cleared when the step starts**: the walk held only for the bus raises
// the bus's condition on the first tick, before any tier bound, naming what waits, the bus build from and to,
// since when and `bus upgrade`, for the operator; S3 says nothing of it even past its bound; and the
+90 -5
View File
@@ -49,18 +49,66 @@ func assertBusObjects(ctx context.Context, inv *inventory.Inventory, r broker.Ra
// composed from, so a module the mesh grants a consumer's subjects has that consumer waiting.
// Done on every raise, not only when a credential is issued: every module moved onto this bus
// by the rollout was issued on the old one, and came up with nothing to bind to (2026-09-28).
consumers, err := moduleConsumers(ctx, inv)
consumers, records, err := moduleConsumersAndRecords(ctx, inv)
if err != nil {
return nil, err
}
// And a grant of a restricted stream that composes nothing, said on every raise and send (novox/hq ADR 0376 §2).
sayWithheld(records, " ")
// And the work queues of seats that name their caller or their kind, with each holder's worker
// (novox/hq ADR 0259 §3): an ask queues until the router takes it, a channel's work until that kind
// takes it.
trafficStreams, trafficWorkers, err := seatTrafficObjects(ctx, inv)
if err != nil {
return nil, err
}
// And every module's own stream (novox/hq ADR 0363), from the catalogue, before the consumers on it.
modStreams, err := inv.DeclaredStreams(ctx)
if err != nil {
return nil, err
}
// Every one tried, and every failure named: one module's consumer the bus refuses is no reason
// the modules after it in the list hear nothing (novox/hq issue 208, where this runs on each send).
var failed []error
for _, s := range broker.ModuleStreams(modStreams) {
if err := r.EnsureStream(s); err != nil {
failed = append(failed, fmt.Errorf("the module stream %s: %w", s.Name, err))
}
}
for _, s := range trafficStreams {
if err := r.EnsureStream(s); err != nil {
failed = append(failed, fmt.Errorf("the work queue %s: %w", s.Name, err))
}
}
for _, c := range trafficWorkers {
if err := r.EnsureConsumer(c); err != nil {
failed = append(failed, fmt.Errorf("the worker %s on %s: %w", c.Name, c.Stream, err))
}
}
restricted := map[string]bool{}
for _, s := range modStreams {
restricted[s.Name] = s.Restricted
}
for _, c := range consumers {
if err := r.EnsureConsumer(c.Consumer); err != nil {
if err := broker.EnsureHeard(r, c.Consumer, restricted[c.Consumer.Stream]); err != nil {
failed = append(failed, fmt.Errorf("how %s on %s hears what it consumes: %w", c.Module, c.Node, err))
}
}
// **And a consumer no longer granted loses what it reads through** (novox/hq ADR 0363 §6): every consumer
// on a module's stream the composition no longer asks for is removed, on this very send.
if p, prunes := r.(broker.Pruner); prunes && len(modStreams) > 0 {
var wanted []broker.Consumer
for _, c := range consumers {
wanted = append(wanted, c.Consumer)
}
removed, err := broker.PruneHeard(p, modStreams, wanted)
if err != nil {
failed = append(failed, err)
}
for _, gone := range removed {
fmt.Printf(" removed the consumer %s: its module is no longer granted that stream (novox/hq ADR 0363)\n", gone)
}
}
if len(failed) > 0 {
return nil, errors.Join(failed...)
}
@@ -119,15 +167,52 @@ func unassertedObservation(err error) conditions.Observation {
// moduleConsumers is every module's durable consumer, from the records the user list is composed from.
func moduleConsumers(ctx context.Context, inv *inventory.Inventory) ([]broker.ModuleConsumer, error) {
consumers, _, err := moduleConsumersAndRecords(ctx, inv)
return consumers, err
}
// moduleConsumersAndRecords is moduleConsumers with the records they were composed from.
func moduleConsumersAndRecords(ctx context.Context, inv *inventory.Inventory) ([]broker.ModuleConsumer, broker.Records, error) {
records, err := inv.BusRecords(ctx)
if err != nil {
return nil, err
return nil, broker.Records{}, err
}
users, err := broker.Users(records)
if err != nil {
return nil, err
return nil, broker.Records{}, err
}
return broker.ConsumersOf(users), nil
return broker.ConsumersOf(users), records, nil
}
// seatTrafficObjects is the work queues and workers of seats that name their caller or their kind, from
// the records the user list is composed from.
func seatTrafficObjects(ctx context.Context, inv *inventory.Inventory) ([]broker.Stream, []broker.Consumer, error) {
records, err := inv.BusRecords(ctx)
if err != nil {
return nil, nil, err
}
users, err := broker.Users(records)
if err != nil {
return nil, nil, err
}
streams, workers := broker.SeatTrafficObjects(users)
// And the queue of every such seat the catalogue declares, held or not: work queues from registration,
// so what is submitted before a holder is assigned waits for it (the correctness review of 2026-10-08).
declared, err := inv.DeclaredTrafficSeats(ctx)
if err != nil {
return nil, nil, err
}
have := map[string]bool{}
for _, s := range streams {
have[s.Name] = true
}
for _, s := range broker.TrafficQueues(declared) {
if !have[s.Name] {
streams = append(streams, s)
have[s.Name] = true
}
}
return streams, workers, nil
}
// moduleConsumerCount is how many modules hear what they consume, for the raise's one line.
+2
View File
@@ -1,3 +1,5 @@
//go:build integration
package main
import (
+103
View File
@@ -0,0 +1,103 @@
package main
import (
"fmt"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// A first-node gate waits for a busy container (novox/hq ADR 0335 §4).
//
// When a machine reports, on what the gate sent it, that a container of a judged module was left running
// because it said busy, the judging has not started: the build is not yet what runs there. The walk says it
// waits for a busy container on that machine, counts no pass and no bound, and starts the judging — its
// Since — at the report that no longer lists the container waiting. The wait is the node-engine's, bounded
// at 15 minutes; the controller's own bound (busyGateBound past the engine's) is for an engine that stopped
// reporting while it waited, so no walk waits for ever. An engine older than the field never lists one, and
// its gate starts as before.
// busyGateSlack is how long past the engine's bound the controller still waits for the report that says the
// container replaced: a pass of the engine (its asks every 20 seconds) and the report's way here.
var busyGateSlack = 5 * time.Minute
// busyEngineBound is the node-engine's bound on a wait (novox/hq ADR 0335 §3), counted again here on this
// controller's clock from when it first read the wait.
const busyEngineBound = 15 * time.Minute
// busyGateEnds is when the gate stops waiting: the engine's own word on its bound, or this controller's count of
// it from when it first read the wait, whichever is sooner, and the slack — so neither a clock ahead on the
// machine nor a wrong word from it holds a walk longer.
func busyGateEnds(b *inventory.GateBusy) time.Time {
until := b.Seen.Add(busyEngineBound)
if !b.Until.IsZero() && b.Until.Before(until) {
until = b.Until
}
return until.Add(busyGateSlack)
}
// busyWait decides, before a judging, whether the gate waits for a busy container on a machine it judges:
// true while it waits, the gate saying so. False when nothing waits, or no longer: a wait that ended moves the
// gate's Since to its end. Pure but for the gate it is handed.
func busyWait(g *inventory.PlanGate, pairs []judged, f gateFacts, now time.Time) bool {
if g.Busy != nil && g.Busy.Ended != nil {
return false
}
for _, j := range pairs {
b, waits := busyOn(j, f)
if !waits {
continue
}
if g.Busy == nil {
g.Busy = &inventory.GateBusy{Machine: j.node, Module: j.module, Container: b.Container, Since: b.Since,
Seen: now}
}
if g.Busy.Seen.IsZero() {
g.Busy.Seen = now
}
g.Busy.Until, g.Busy.Said = b.Until, b.Said
if now.After(busyGateEnds(g.Busy)) {
// The engine said it would have replaced it by now: the judging starts, and finds the module not
// yet healthy while the machine still lists it waiting (busyStill), until the gate's own bound.
end := now
g.Busy.Ended, g.Busy.PastBound = &end, true
g.Since, g.Passes, g.LastPass, g.Healthy, g.HealthyAt, g.Applied = &end, 0, nil, nil, nil, nil
return false
}
g.Passes, g.LastPass, g.Last, g.Failing = 0, nil, g.Busy.Words(), nil
g.Read(now, false, g.Busy.Words())
return true
}
if g.Busy == nil {
return false
}
// Replaced: the judging starts at the report that said so.
end := now
if r, said := f.reports[g.Busy.Machine]; said && r.At != nil && r.At.After(g.Busy.Since) && r.At.Before(now) {
end = r.At.UTC()
}
g.Busy.Ended = &end
g.Since, g.Passes, g.LastPass, g.Last, g.Healthy, g.HealthyAt, g.Applied = &end, 0, nil, "", nil, nil, nil
return false
}
// busyOn is the container of a judged module its machine last reported, on what the gate sent it, as left
// running because it said busy.
func busyOn(j judged, f gateFacts) (inventory.BusyWait, bool) {
r, said := f.reports[j.node]
if !said || r.At == nil || !f.reportedOn(j.node, r) {
return inventory.BusyWait{}, false
}
return inventory.WaitingFor(r.Busy, j.module)
}
// busyStill is a judged module whose container its machine still lists waiting once the gate stopped waiting
// for it: not yet what was sent, so not healthy, whatever its health says of the container still running.
func busyStill(j judged, f gateFacts) (string, bool) {
b, waits := busyOn(j, f)
if !waits {
return "", false
}
return fmt.Sprintf("%s still runs its busy container %s, past its node-engine's bound of %s",
j.node, b.Container, b.Until.UTC().Format(time.RFC3339)), true
}
+148
View File
@@ -0,0 +1,148 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// A walk whose module's container waits as busy on its first machine says so, and starts that machine's
// first-machine gate only once the machine reports the container replaced (novox/hq ADR 0335 §4).
func TestAGateWaitsForABusyContainerAndStartsWhenItIsReplaced(t *testing.T) {
sent := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
g := &inventory.PlanGate{Machines: []string{"novox"}, Since: &sent,
Sent: map[string]inventory.SentDeclaration{"novox": {Digest: "d1", Sequence: 7}}}
pairs := []judged{{module: "build-agent", node: "novox"}, {module: "mesh-controller", node: "novox"}}
reported := sent.Add(20 * time.Second)
waiting := []inventory.BusyWait{{ID: "build-agent.server", Container: "mesh-build-agent",
State: inventory.BusyWaiting, Since: reported, Until: reported.Add(15 * time.Minute),
Said: "busy: novox/mesh-controller (check of #280), in hand since 00:58:00Z"}}
facts := func(at time.Time, busy []inventory.BusyWait) gateFacts {
// As judgeMoves hands it: the report held against the gate's own send.
return gateFacts{sent: g.Sent, reports: map[string]inventory.Reported{"novox": {Node: "novox",
Outcome: inventory.OutcomeApplied, At: &at, Declared: "d1", ReportedSequence: 7, Busy: busy}}}
}
now := reported.Add(time.Minute)
if !busyWait(g, pairs, facts(reported, waiting), now) {
t.Fatal("a gate whose module's container waits as busy judged it")
}
if !g.Since.Equal(sent) || g.Busy == nil || g.Busy.Machine != "novox" || g.Busy.Ended != nil {
t.Fatalf("the gate does not keep the wait: %+v %+v", g.Since, g.Busy)
}
for _, said := range []string{gateLine(g), g.Last} {
if !strings.Contains(said, "waits for a busy container on novox") {
t.Errorf("the walk does not say it waits for a busy container there: %s", said)
}
}
if !strings.Contains(g.Last, "check of #280") {
t.Errorf("the wait does not say what the container holds: %s", g.Last)
}
// Ten minutes on, still busy, within the engine's bound: still waiting, nothing judged, no bound run.
now = reported.Add(10 * time.Minute)
if !busyWait(g, pairs, facts(reported.Add(9*time.Minute), waiting), now) || g.Passes != 0 {
t.Fatalf("the gate stopped waiting within the bound: %+v", g)
}
// The machine reports it replaced: the judging starts at that report.
replacedAt := reported.Add(12 * time.Minute)
now = replacedAt.Add(5 * time.Second)
replaced := []inventory.BusyWait{{ID: "build-agent.server", Container: "mesh-build-agent",
State: "replaced-when-idle", Since: reported, Until: reported.Add(15 * time.Minute)}}
if busyWait(g, pairs, facts(replacedAt, replaced), now) {
t.Fatal("the gate still waits after the container was replaced")
}
if g.Since == nil || !g.Since.Equal(replacedAt) || g.Busy.Ended == nil || !g.Busy.Ended.Equal(replacedAt) {
t.Fatalf("the gate does not start at the report that said it replaced: since %v, %+v", g.Since, g.Busy)
}
if strings.Contains(gateLine(g), "waits for a busy container") {
t.Errorf("a gate judging still says it waits: %s", gateLine(g))
}
// And a later report listing it waiting again does not hold a gate that started.
if busyWait(g, pairs, facts(now, waiting), now.Add(time.Minute)) {
t.Error("a gate that started waits again")
}
}
// An engine that stops reporting while it waits holds no walk for ever: past the engine's bound and the slack,
// the judging starts, and finds the module not yet healthy while its container is still listed waiting.
func TestAGateWaitsNoLongerThanTheEnginesBound(t *testing.T) {
sent := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
g := &inventory.PlanGate{Machines: []string{"novox"}, Since: &sent}
pairs := []judged{{module: "build-agent", node: "novox"}}
waiting := []inventory.BusyWait{{ID: "build-agent.server", Container: "mesh-build-agent",
State: inventory.BusyWaiting, Since: sent, Until: sent.Add(15 * time.Minute)}}
f := gateFacts{reports: map[string]inventory.Reported{"novox": {Node: "novox", Outcome: inventory.OutcomeApplied,
At: &sent, Current: true, Busy: waiting}}}
now := sent.Add(15*time.Minute + busyGateSlack + time.Second)
if busyWait(g, pairs, f, now) {
t.Fatal("a gate waited past the engine's bound and the slack")
}
if !g.Busy.PastBound || !g.Since.Equal(now) {
t.Fatalf("the gate does not start, said past the bound: %+v %v", g.Busy, g.Since)
}
if why, still := busyStill(pairs[0], f); !still || !strings.Contains(why, "past its node-engine's bound") {
t.Fatalf("a module still running its busy container reads as what was sent: %q", why)
}
}
// A report from an engine older than the field lists nothing: the gate starts at the send, as before; and a
// report not about the gate's send says nothing of it.
func TestAGateOfAnEngineThatNeverWaitsStartsAsBefore(t *testing.T) {
sent := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
g := &inventory.PlanGate{Machines: []string{"ace"}, Since: &sent,
Sent: map[string]inventory.SentDeclaration{"ace": {Digest: "d2", Sequence: 9}}}
pairs := []judged{{module: "build-agent", node: "ace"}}
at := sent.Add(time.Minute)
older := gateFacts{sent: g.Sent, reports: map[string]inventory.Reported{"ace": {Node: "ace", Outcome: inventory.OutcomeApplied,
At: &at, Declared: "d2", ReportedSequence: 9}}}
if busyWait(g, pairs, older, at) || g.Busy != nil || !g.Since.Equal(sent) {
t.Fatalf("a gate on an engine that never waits did not start at the send: %+v %v", g.Busy, g.Since)
}
elsewhere := gateFacts{sent: g.Sent, reports: map[string]inventory.Reported{"ace": {Node: "ace", Outcome: inventory.OutcomeApplied,
At: &at, Declared: "d1", ReportedSequence: 8, Busy: []inventory.BusyWait{{ID: "build-agent.server",
State: inventory.BusyWaiting, Until: at.Add(time.Hour)}}}}}
if busyWait(g, pairs, elsewhere, at) {
t.Fatal("a wait reported about an earlier declaration held the gate")
}
}
// The wait is bounded on this controller's clock: an engine whose clock runs an hour ahead, saying it waits until
// then, holds the gate no longer than 15 minutes and the slack from when the controller first read the wait.
func TestAGateWaitIsBoundedOnTheControllersClock(t *testing.T) {
seen := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
g := &inventory.PlanGate{Machines: []string{"novox"}, Since: &seen}
pairs := []judged{{module: "build-agent", node: "novox"}}
ahead := seen.Add(time.Hour)
f := gateFacts{reports: map[string]inventory.Reported{"novox": {Node: "novox", Outcome: inventory.OutcomeApplied,
At: &seen, Current: true, Busy: []inventory.BusyWait{{ID: "build-agent.server", Container: "mesh-build-agent",
State: inventory.BusyWaiting, Since: ahead, Until: ahead.Add(15 * time.Minute)}}}}}
if !busyWait(g, pairs, f, seen) || !g.Busy.Seen.Equal(seen) {
t.Fatalf("the gate did not start waiting, seen now: %+v", g.Busy)
}
if !busyWait(g, pairs, f, seen.Add(15*time.Minute+busyGateSlack-time.Second)) {
t.Fatal("the gate stopped waiting within the bound on its own clock")
}
if busyWait(g, pairs, f, seen.Add(15*time.Minute+busyGateSlack+time.Second)) || !g.Busy.PastBound {
t.Fatalf("an engine's later word held the gate past the controller's bound: %+v", g.Busy)
}
}
// `plan` says, for a machine whose node-engine is older than the field, each container whose busy is not sent.
func TestAPlanSaysABusyAnOlderEngineIsNotSent(t *testing.T) {
modules := []catalogue.Manifest{{Module: "build-agent", Resources: []map[string]any{
{"id": "server", "type": "container", "busy": map[string]any{"command": "mesh-builder busy"}},
{"id": "agent-env", "type": "file"}}}}
lines := busyNotSent(modules, false)
if len(lines) != 1 || !strings.Contains(lines[0], "build-agent.server declares busy") ||
!strings.Contains(lines[0], "replaced at once") {
t.Fatalf("the plan does not say the busy not sent: %v", lines)
}
if lines := busyNotSent(modules, true); len(lines) != 0 {
t.Fatalf("a busy sent is said as not sent: %v", lines)
}
}
@@ -0,0 +1,36 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"strings"
"testing"
)
func TestCallsIsServedAndSaysWhatItKeeps(t *testing.T) {
handlers, behind, err := seatToolHandlers()
if err != nil || len(behind) != 0 {
t.Fatalf("%v %v", behind, err)
}
calls, ok := handlers["calls"]
if !ok {
t.Fatal("calls is not served")
}
if _, err := calls(context.Background(), json.RawMessage(`{"node":"anchor"}`)); err == nil ||
!strings.Contains(err.Error(), `"node"`) {
t.Errorf("calls took an argument it does not declare: %v", err)
}
if _, err := calls(context.Background(), json.RawMessage(`{"call":"call-0-0"}`)); err == nil ||
!strings.Contains(err.Error(), "not across a restart") {
t.Errorf("an unknown call was not said plainly: %v", err)
}
got, err := calls(context.Background(), json.RawMessage(`{}`))
if err != nil {
t.Fatal(err)
}
if _, listed := got.(map[string]any)["calls"]; !listed {
t.Errorf("calls answered %v", got)
}
}
-28
View File
@@ -1,9 +1,6 @@
package main
import (
"context"
"encoding/json"
"strings"
"testing"
"time"
@@ -52,28 +49,3 @@ func TestAPushAnswersBeforeItSends(t *testing.T) {
}
// `calls` is served, takes a call's id and nothing else, and says plainly when it holds no such call.
func TestCallsIsServedAndSaysWhatItKeeps(t *testing.T) {
handlers, behind, err := seatToolHandlers()
if err != nil || len(behind) != 0 {
t.Fatalf("%v %v", behind, err)
}
calls, ok := handlers["calls"]
if !ok {
t.Fatal("calls is not served")
}
if _, err := calls(context.Background(), json.RawMessage(`{"node":"anchor"}`)); err == nil ||
!strings.Contains(err.Error(), `"node"`) {
t.Errorf("calls took an argument it does not declare: %v", err)
}
if _, err := calls(context.Background(), json.RawMessage(`{"call":"call-0-0"}`)); err == nil ||
!strings.Contains(err.Error(), "not across a restart") {
t.Errorf("an unknown call was not said plainly: %v", err)
}
got, err := calls(context.Background(), json.RawMessage(`{}`))
if err != nil {
t.Fatal(err)
}
if _, listed := got.(map[string]any)["calls"]; !listed {
t.Errorf("calls answered %v", got)
}
}
+53
View File
@@ -2,6 +2,8 @@ package main
import (
"fmt"
"path"
"slices"
"sort"
"strings"
@@ -28,6 +30,14 @@ func changePlanOf(repository, base, head string, r mergeReach, entries []invento
for _, e := range entries {
byName[e.Manifest.Module] = e
}
// The machine holding the bus, whose declaration carries every module's grants (novox/hq issue 490).
holder := ""
for _, e := range entries {
if e.Manifest.BusUsers != "" && e.Manifest.ClaimsSeat(catalogue.BrokerSeat) && len(e.On) > 0 {
holder = e.On[0]
}
}
var granting []string
machines := map[string]*link.MachinePlan{}
machine := func(name string) *link.MachinePlan {
if machines[name] == nil {
@@ -50,6 +60,10 @@ func changePlanOf(repository, base, head string, r mergeReach, entries []invento
machine(on).Receives = append(machine(on).Receives, name)
}
}
if !waits && holder != "" && !(len(e.On) == 1 && e.On[0] == holder) && len(e.On) > 0 &&
!slices.Contains(granting, name) {
granting = append(granting, name)
}
switch {
case (name == "nats" || catalogue.ProvidesBus(e.Manifest)) && len(e.On) > 0:
// Said before the merge (novox/hq issue 336): a new bus build holds every send to the bus's machine
@@ -59,6 +73,11 @@ func changePlanOf(repository, base, head string, r mergeReach, entries []invento
"(issue 280)", busUpgradeNeeded, name, busUpgradeVerb, strings.Join(e.On, ", ")))
case name == "nats" || catalogue.ProvidesBus(e.Manifest):
p.Steps = append(p.Steps, "a planned bus step: the bus is upgraded by `bus upgrade`, never by an ordinary send")
case waits && len(e.On) > 0 && (u.From == catalogue.FromModule || u.From == catalogue.FromData):
// Judged per machine by the walk, against the build each was last sent (novox/hq ADR 0377).
p.Steps = append(p.Steps, fmt.Sprintf("%s waits for a person: its policy records (%s) — unless on a machine "+
"its move recreates none of its containers and moves none of its data, which the walk judges and then sends",
name, orNone(u.From)))
case waits && len(e.On) > 0:
p.Steps = append(p.Steps, fmt.Sprintf("%s waits for a person: its policy records (%s)", name,
orNone(u.From)))
@@ -76,6 +95,13 @@ func changePlanOf(repository, base, head string, r mergeReach, entries []invento
}
}
}
if len(granting) > 0 {
// Said before the merge (novox/hq issue 490): what the walk does when the change alters the bus's
// user list. Whether it does is known only once the builds are registered, so it is said as a rule.
p.Steps = append(p.Steps, fmt.Sprintf("%s: if this changes what the bus's user list says (a new tool, "+
"subject or user), %s is sent that list alone, every build there kept, before %s is judged on its first "+
"machine", grantsStepWord, holder, strings.Join(granting, ", ")))
}
var names []string
for name := range machines {
names = append(names, name)
@@ -85,9 +111,36 @@ func changePlanOf(repository, base, head string, r mergeReach, entries []invento
p.Machines = append(p.Machines, *machines[name])
}
p.Summary = summaryOf(p)
p.Class = changeClassOf(p, entries)
return p
}
// classNone is the class of a change that moves no module: nothing for a walk to move (novox/hq ADR 0313 §2).
const classNone = "none"
// changeClassOf is a change plan's class (novox/hq ADR 0313 §2): that of the slowest-budgeted module it moves,
// itself or as a dependent — core when one is on the controller's own path or holds the mesh's resolver, leaf
// otherwise — and none when it moves nothing. A module it adds a directory for counts by its own class: the
// catalogue names a module's directory by the module, so the directory's last element is its name. The graph holds
// no manifest of it yet, so a new module is core by its name alone (on the controller's own path), never by a
// resolver claim its manifest makes.
func changeClassOf(p link.ChangePlan, entries []inventory.Entry) string {
if len(p.Moved) == 0 && len(p.Dependents) == 0 && len(p.New) == 0 {
return classNone
}
modules := append(append([]string{}, p.Moved...), p.Dependents...)
for _, dir := range p.New {
if name := path.Base(strings.Trim(dir, "/")); name != "." && name != "/" && name != "" {
modules = append(modules, name)
}
}
return classOfModules(modules, entries)
}
// grantsStepWord names the grants step (novox/hq issue 490): the bus's user list sent alone to the machine
// holding the bus ahead of a walk's gate. Not the bus step, which replaces the bus itself.
const grantsStepWord = "grants step"
// busUpgradeNeeded is how a change plan says that merging it holds the bus's machine for a person's step.
const busUpgradeNeeded = "merging this needs a person's bus upgrade"
+85 -2
View File
@@ -11,6 +11,7 @@ import (
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
// moduleCheck judges manifests where they are written, with no mesh (novox/hq ADR 0037, issue 148).
@@ -32,6 +33,9 @@ import (
// provision it wants that a manifest given here offers. An overflow is refused in the pull request
// that introduces it — a new requirement, a lowered bound, a longer slug — instead of on the
// provider's machine when a real machine's name first meets the module's.
// SomeManifestsVar, set by the merge gate, says the manifests given are only some of their repository's.
const SomeManifestsVar = "MESH_MODULE_CHECK_SOME"
func moduleCheck(paths []string, out io.Writer) error {
return moduleCheckFor(paths, catalogue.DefaultLongestMachine, out)
}
@@ -44,6 +48,24 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
shelf := catalogue.Shelf{}
faulted := map[string]bool{}
failed := 0
// The list of predefined keys at the root of the repository above each manifest (novox/hq ADR 0348 §7), read
// once per repository: a manifest naming a key is judged against the list its own tree holds.
lists := map[string]catalogue.PredefinedList{}
listOf := func(path string) (catalogue.PredefinedList, error) {
root := catalogue.RepositoryRoot(filepath.Dir(path))
if root == "" {
return nil, nil
}
if list, read := lists[root]; read {
return list, nil
}
list, err := catalogue.ReadPredefined(root)
if err != nil {
return nil, err
}
lists[root] = list
return list, nil
}
for _, path := range paths {
raw, err := os.ReadFile(path)
if err != nil {
@@ -51,7 +73,18 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
failed++
continue
}
m, err := catalogue.ParseManifest(raw)
list, err := listOf(path)
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
continue
}
m, err := catalogue.ParseManifestWith(raw, list)
if err == nil {
// Each template's text read from the tree beside the manifest, and judged as inline content is
// (novox/hq ADR 0349 §2): a missing template is refused by name.
m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(path))
}
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
@@ -72,6 +105,30 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
failed += len(wrong)
faulted[m.Module] = true
}
// A module that declares a setting names the keys of its mergeable files (novox/hq ADR 0326 §3,
// issue 574): judged where a definition is written, as at registration, never where a stored
// build is read again.
if wrong := catalogue.WrittenSettingsProblems(m); len(wrong) > 0 {
for _, p := range wrong {
fmt.Fprintf(out, "%s: %s\n", path, p)
}
failed += len(wrong)
faulted[m.Module] = true
}
// A module lives whole in one repository (novox/hq ADR 0420 §1, issue 1019): when the check is told the
// repository it runs for (MESH_CHECK_REPOSITORY, set by the build's check), an artifact building from
// another is refused, as the merge gate refuses it. A run by hand names no repository and judges none.
if repo := os.Getenv("MESH_CHECK_REPOSITORY"); repo != "" && m.Build != nil {
for _, a := range m.Build.Artifacts {
if a.Context != nil && a.Context.Repository != "" && !sameRepository(a.Context.Repository,
link.SourceMoved{Owner: ownerOf(repo), Repo: repoOf(repo)}) {
fmt.Fprintf(out, "%s: %s builds artifact %s from %s: a module lives whole in one repository "+
"(novox/hq ADR 0420 §1)\n", path, m.Module, a.Name, a.Context.Repository)
failed++
faulted[m.Module] = true
}
}
}
if named := catalogue.InstallationProblems(m); len(named) > 0 {
for _, p := range named {
fmt.Fprintf(out, "%s: %s\n", path, p)
@@ -85,12 +142,24 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
// Between the manifests: a seat declared twice, a use of a seat nothing declares, a claim on
// a seat that does not exist. Run only over what parsed, because a problem inside one manifest
// has already been said and would be said again here in a worse form.
//
// **Over some of a repository's manifests, a seat none of them declares is a note** (novox/hq issue 364), as
// this command's own word says above: the merge gate passes only the manifests a change touches, and says so
// with SomeManifestsVar, so a module that uses or claims a seat another module declares (the operator
// channel's, a channel bench) was refused there for a manifest it was not given. Given every manifest — the
// catalogue's own check, and registration — it stays a refusal.
some := os.Getenv(SomeManifestsVar) != ""
problems := catalogue.CatalogueProblems(shelf)
sort.Strings(problems)
for _, p := range problems {
if some && catalogue.IsUndeclaredSeat(p) {
fmt.Fprintf(out, "note: %s among the manifests given; registration judges it against the whole catalogue, "+
"and passing the declaring module's manifest too judges it here\n", p)
continue
}
fmt.Fprintln(out, p)
failed++
}
failed += len(problems)
// Between the manifests too: an identity against the bounds of the provisions it wants, which
// only the provider's manifest states.
@@ -164,6 +233,14 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
}
fmt.Fprintf(out, ", keeps state %s", strings.Join(kept, ", "))
}
// And the logs it keeps, with their caps (novox/hq ADR 0297).
if len(m.Logs) > 0 {
kept := make([]string, 0, len(m.Logs))
for _, l := range m.Logs {
kept = append(kept, fmt.Sprintf("%s (%d MiB)", l.Name, l.Cap()))
}
fmt.Fprintf(out, ", keeps log %s", strings.Join(kept, ", "))
}
if len(m.Reads) > 0 {
fmt.Fprintf(out, ", reads %s", strings.Join(m.Reads, ", "))
}
@@ -182,6 +259,12 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
checks = append(checks, fmt.Sprintf("%v by %s", r["id"], catalogue.HealthWords(h)))
}
}
// And its own runtime's, where it runs as its own account (novox/hq ADR 0366).
if own, has := catalogue.OwnRuntimeHealth(m); has {
if h, _, _ := catalogue.ReadHealth(own); h.Kind != "" {
checks = append(checks, fmt.Sprintf("%v by %s", own["id"], catalogue.HealthWords(h)))
}
}
if len(checks) > 0 {
fmt.Fprintf(out, ", ready: %s", strings.Join(checks, "; "))
}
+87
View File
@@ -0,0 +1,87 @@
package main
import (
"context"
"errors"
"fmt"
"os"
"strconv"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/inventory"
)
// A pull request's check builds go when it closes (novox/hq ADR 0350 §5): the build seat keeps them in the artifact
// store's scratch namespace for the walk of its merge to take. **Let go of where the moment is heard**: the
// controller ends the walk answering a merge — after that walk took what it reuses — and the delivery's owner, which
// hears the forge say a pull request closed unmerged, asks the `forget-checks` verb; both let go through the store,
// as the controller's own collection does. The build seat's holders let go of anything a week old on their own,
// so a word lost here leaves a week's builds, never more.
// checkBuildsIn is the scratch namespace as the controller reaches the artifact store.
func checkBuildsIn(ctx context.Context, inv *inventory.Inventory) (builder.Scratch, error) {
shelf, err := inv.Catalogue(ctx)
if err != nil {
return nil, err
}
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return nil, err
}
if address == "" {
return nil, errors.New("the artifact store is not on the mesh's network")
}
return builder.Registry{Address: address}, nil
}
// forgetChecks lets go of a pull request's check builds and answers what it did.
var forgetChecks = func(ctx context.Context, inv *inventory.Inventory, pull builder.PullRef) (string, error) {
scratch, err := checkBuildsIn(ctx, inv)
if err != nil {
return "", err
}
n, err := scratch.Forget(ctx, pull)
if err != nil {
return "", fmt.Errorf("%s: %d check(s)' builds let go of, and not all: %w", pull, n, err)
}
return fmt.Sprintf("%s: %d check(s)' builds let go of", pull, n), nil
}
// forgetChecksOfWalk lets go, on the save that ends a walk, of the check builds of every pull request whose merge
// it answers: the walk took what it reuses. Said, never fatal.
func forgetChecksOfWalk(ctx context.Context, inv *inventory.Inventory, p inventory.Plan) {
if p.WalkSaid != inventory.WalkEnded || p.Delivery == nil {
return
}
for _, pull := range reuseAskOf(&p).Pulls {
said, err := forgetChecks(ctx, inv, builder.PullRef{Owner: pull.Owner, Repo: pull.Repo, Number: pull.Number})
if err != nil {
fmt.Fprintf(os.Stderr, "%s: its check builds stay until the week's sweep: %v\n", p.ID, err)
continue
}
fmt.Printf("%s: %s\n", p.ID, said)
}
}
// forgetChecksCommand is `forget-checks <owner> <repo> <number>`: the verb the delivery's owner asks when the forge
// says a pull request closed unmerged (novox/hq ADR 0350 §5).
func forgetChecksCommand(ctx context.Context, args []string) error {
if len(args) != 3 {
return errors.New("forget-checks <owner> <repo> <number>")
}
number, err := strconv.Atoi(args[2])
if err != nil || number <= 0 {
return fmt.Errorf("%q is not a pull request's number", args[2])
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
said, err := forgetChecks(ctx, open.inventory, builder.PullRef{Owner: args[0], Repo: args[1], Number: number})
if err != nil {
return err
}
fmt.Println(said)
return nil
}
@@ -0,0 +1,102 @@
//go:build integration
package main
import (
"encoding/json"
"reflect"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func TestOnlyAWalksAskTakesAChecksBuildAndADryRunNever(t *testing.T) {
p := &inventory.Plan{Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
{Repository: "novox/shop", Number: 7}, {Repository: "http://forge.invalid/novox/web.git", Number: 3},
{Repository: "novox/shop", Number: 7}, {Repository: "novox/shop"}}}}
asked := reuseAskOf(p)
if want := []link.PullRef{{Owner: "novox", Repo: "shop", Number: 7}, {Owner: "novox", Repo: "web", Number: 3}}; !reflect.DeepEqual(asked.Pulls, want) {
t.Fatalf("the walk names %+v", asked.Pulls)
}
walk := withReuse(t.Context(), asked)
if reuseAsked(walk, false) != asked {
t.Fatal("a walk's ask does not carry its leave")
}
if reuseAsked(walk, true) != nil {
t.Fatal("a dry run carries a leave to take a check's build")
}
if reuseAsked(t.Context(), false) != nil {
t.Fatal("an ask that is not a walk's carries a leave")
}
}
func TestAReusedCheckBuildIsRegisteredByItsDigestAndGatedAsABuiltOne(t *testing.T) {
withTheGate(t) // the first-machine gate's own path (novox/hq ADR 0360 keeps it for earlier walks)
g := aGateMesh(t)
ctx := t.Context()
inv := g.open.inventory
gateEvery = 0
digest := "sha256:" + strings.Repeat("c", 64)
reused := func(id, reference string) link.BuildResult {
manifest := mustJSON(t, catalogue.Manifest{Module: "app", Version: "c3", Resources: []map[string]any{
{"id": "svc", "type": "container", "name": "app", "image": reference}}})
return link.BuildResult{ID: id, Repository: "novox/mesh-catalog", Path: "modules/app", Ref: "main", On: "anchor",
Commit: "c3", Manifest: manifest, Trunk: "main", OnTrunk: true, Branches: []string{"main"},
Made: []link.MadeArtifact{{Name: "server", Kind: "image", Reference: reference}},
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"},
Reused: "reused from the check of novox/mesh-catalog#9 at head abcdefabcdef (merged into trunk 123456123456): server " + digest,
ReusedFrom: &link.ReusedFrom{Pull: link.PullRef{Owner: "novox", Repo: "mesh-catalog", Number: 9},
Head: "abcdefabcdef", Trunk: "123456123456", Key: "r1:" + strings.Repeat("1", 64)}}
}
if _, _, err := takeIn(ctx, inv, reused(link.NewBuildID(time.Now()), "store.internal:5000/scratch/app/server@"+digest)); err == nil ||
!strings.Contains(err.Error(), "scratch") {
t.Fatalf("a build in the scratch namespace was registered: %v", err)
}
id := link.NewBuildID(time.Now().Add(time.Second))
result := reused(id, "store.internal:5000/app/server@"+digest)
if _, _, err := takeIn(ctx, inv, result); err != nil {
t.Fatal(err)
}
current, err := inv.CurrentBuilds(ctx)
if err != nil || current["app"].Commit != "c3" {
t.Fatalf("the reused build is not what app is registered at: %+v %v", current["app"], err)
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(shelf["app"].Resources)
if !strings.Contains(string(raw), "app/server@"+digest) {
t.Fatalf("app is registered as %s, not by the check's digest in its own namespace", raw)
}
// The walk asked it: the walk keeps what its build did, and its gate judges the registered build.
asked := time.Now().UTC()
p := g.plan(t)
p.Commit = "c3"
p.Modules["app"] = &inventory.PlanModule{State: "asked", AskedAt: &asked, Build: id}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
planBuiltSaying(ctx, g.open, "app", "c3", "", asked, id, result.MadeWords())
p = g.plan(t)
if m := p.Modules["app"]; m.State != "built" || !strings.HasPrefix(m.Made, "reused from the check of novox/mesh-catalog#9") {
t.Fatalf("the walk keeps %+v", m)
}
for i := 0; i < 6; i++ {
advancePlans(ctx, g.open)
}
p = g.plan(t)
gate := p.Modules["app"].Gate
if gate == nil || gate.To != "c3" || gate.Verdict != inventory.GatePassed {
t.Fatalf("the gate judged %+v, not the reused build", gate)
}
if sent, _, _ := inv.SentBuilds(ctx, "anchor"); sent["app"] != "c3" {
t.Fatalf("the first machine was sent %v, not the reused build", sent)
}
}
+193
View File
@@ -0,0 +1,193 @@
package main
import (
"context"
"errors"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// One build per change (novox/hq ADR 0350), as the controller asks it and takes it in.
// The check of a change is asked to build what a merge of it would move, in the walk's order: a module of the
// repository checked from the merge, one standing on it from its own repository as the walk builds it, and one
// that reads the repository through a context not at all. A change that moves no module builds nothing.
func TestACheckIsAskedToBuildWhatTheMergeWouldMove(t *testing.T) {
entries := []inventory.Entry{
{Manifest: catalogue.Manifest{Module: "app"}, Source: inventory.Source{Repository: "novox/shop", Seat: "git", Path: "modules/app"}},
{Manifest: catalogue.Manifest{Module: "web"}, Source: inventory.Source{Repository: "novox/web", Seat: "git", Ref: "main"}},
{Manifest: catalogue.Manifest{Module: "docs"}, Source: inventory.Source{Repository: "novox/docs", Seat: "git"}},
}
p := link.PullUpdated{Owner: "novox", Repo: "shop", Number: 7, Base: "main", Commit: "abc"}
scope := checkScope{Reach: mergeReach{Packaging: entries[2:], Plan: inventory.Plan{Tiers: [][]string{{"app", "docs"}, {"web"}}}}}
clone := func(s inventory.Source) (string, error) { return "http://forge.invalid/" + s.Repository + ".git", nil }
got := checkBuildsOf(p, scope, entries, clone)
want := []link.CheckBuild{
{Module: "app", Path: "modules/app", Merged: true, Tier: 0},
{Module: "docs", Tier: 0, Skip: "its build reads novox/shop through a context, which a check cannot hand the merge"},
{Module: "web", Repository: "http://forge.invalid/novox/web.git", Ref: "main", Tier: 1},
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("the check is asked to build\n%+v\nwant\n%+v", got, want)
}
if got := checkBuildsOf(p, checkScope{}, entries, clone); got != nil {
t.Fatalf("a change that moves no module asks for builds: %+v", got)
}
}
// A walk's ask carries its leave to take a check's build, naming the pull requests of the merges it answers; a dry
// run never does, and neither does an ask that is not a walk's.
// A walk ended lets go of its merges' check builds — only once it ended, after it took what it reuses.
func TestAWalkEndedLetsGoOfItsMergesCheckBuilds(t *testing.T) {
var forgot []builder.PullRef
was := forgetChecks
forgetChecks = func(_ context.Context, _ *inventory.Inventory, pull builder.PullRef) (string, error) {
forgot = append(forgot, pull)
return "", nil
}
t.Cleanup(func() { forgetChecks = was })
p := inventory.Plan{ID: "plan-1", Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
{Repository: "novox/shop", Number: 7}}}}
p.WalkSaid = inventory.WalkStarted
forgetChecksOfWalk(t.Context(), nil, p)
p.WalkSaid = ""
forgetChecksOfWalk(t.Context(), nil, p)
if len(forgot) != 0 {
t.Fatalf("a walk not ended let go of %v, which it may still take", forgot)
}
p.WalkSaid = inventory.WalkEnded
forgetChecksOfWalk(t.Context(), nil, p)
if want := []builder.PullRef{{Owner: "novox", Repo: "shop", Number: 7}}; !reflect.DeepEqual(forgot, want) {
t.Fatalf("the walk ended let go of %v", forgot)
}
}
// A build in the scratch namespace is never registered: only the walk's copy in the module's own namespace is.
func TestABuildInTheScratchNamespaceIsNeverRegistered(t *testing.T) {
if got := inScratch([]link.MadeArtifact{{Name: "server", Reference: "store.internal:5000/app/server@sha256:aa"}}); got != "" {
t.Fatalf("a module's own build reads as a scratch build: %s", got)
}
scratch := "store.internal:5000/scratch/app/server@sha256:aa"
if got := inScratch([]link.MadeArtifact{{Name: "server", Reference: scratch}}); got != scratch {
t.Fatalf("a scratch build was not found: %q", got)
}
if got := inScratch([]link.MadeArtifact{{Name: "look", Reference: "http://store.internal:5000/v2/scratch/app/look/blobs/sha256:bb"}}); got == "" {
t.Fatal("a scratch archive was not found")
}
}
// The walk says per module what it did: "reused from …" or "built: <why>" — and a build seat from before says
// neither, which is said.
func TestAWalksBuildSaysWhetherItReusedOrBuilt(t *testing.T) {
if got := (link.BuildResult{Reused: "reused from the check of novox/shop#7"}).MadeWords(); got != "reused from the check of novox/shop#7" {
t.Fatalf("said %q", got)
}
if got := (link.BuildResult{BuiltWhy: "no check built it"}).MadeWords(); got != "built: no check built it" {
t.Fatalf("said %q", got)
}
if got := (link.BuildResult{}).MadeWords(); !strings.Contains(got, "predates ADR 0350") {
t.Fatalf("said %q", got)
}
}
// A walk's module the check's build stood in for is registered by the digest the check built — the copy in the
// module's namespace — and the walk keeps "reused from …" for it; its first-node gate judges that registered
// build as it judges one the walk built. A scratch reference is refused. Needs the store (MESH_TEST_POSTGRES), and
// is named in testdata/store-tests.
// A walk's leave names the builds its own pull requests' verdicts made at the head each merge merged — never an
// earlier head's — read once per walk, and none when they cannot be read: the builder takes nothing a verdict does
// not name (novox/hq ADR 0350 §2, issue 703).
func TestAWalksLeaveNamesTheBuildsItsPullRequestsVerdictsMade(t *testing.T) {
was := checkedBuildsOf
forgetWalkVerdicts()
t.Cleanup(func() { checkedBuildsOf = was; forgetWalkVerdicts() })
var asked [][]link.PullRef
pull := link.PullRef{Owner: "novox", Repo: "shop", Number: 7}
atHead := link.CheckedBuild{Pull: pull, Check: "build-2", Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:2"}}
earlier := link.CheckedBuild{Pull: pull, Check: "build-1", Head: "aaaaaaaaaaaa", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
checkedBuildsOf = func(_ context.Context, pulls []link.PullRef) ([]link.CheckedBuild, error) {
asked = append(asked, pulls)
return []link.CheckedBuild{atHead, earlier}, nil
}
pulls := []link.PullRef{pull}
heads := map[link.PullRef]string{pull: "bbbbbbbbbbbb0123456789"}
for range 3 {
if got := reuseWithVerdicts(t.Context(), "plan-1", &link.ReuseAsk{Pulls: pulls}, heads); !reflect.DeepEqual(got.Builds,
[]link.CheckedBuild{atHead}) {
t.Fatalf("the leave names %+v, not only the merged head's build", got.Builds)
}
}
if !reflect.DeepEqual(asked, [][]link.PullRef{pulls}) {
t.Fatalf("the verdicts were read %d times for one walk: %v", len(asked), asked)
}
// A merge whose head was not said takes nothing.
if got := reuseWithVerdicts(t.Context(), "plan-1", &link.ReuseAsk{Pulls: pulls}, nil); got.Builds != nil {
t.Fatalf("a merge of no said head names %+v", got.Builds)
}
checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) {
return nil, errors.New("no bus")
}
if got := reuseWithVerdicts(t.Context(), "plan-2", &link.ReuseAsk{Pulls: pulls}, heads); got.Builds != nil {
t.Fatalf("verdicts that could not be read named %+v", got.Builds)
}
}
// A check is handed only the builds earlier checks of its own pull request at its own head made: another pull
// request's build, or an earlier head's, is never kept (novox/hq issue 703). None when they cannot be read.
func TestACheckIsHandedOnlyItsOwnHeadsBuilds(t *testing.T) {
was := checkedBuildsOf
t.Cleanup(func() { checkedBuildsOf = was })
pull := link.PullRef{Owner: "novox", Repo: "shop", Number: 7}
mine := link.CheckedBuild{Pull: pull, Check: "build-1", Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
earlier := link.CheckedBuild{Pull: pull, Check: "build-0", Head: "aaaaaaaaaaaa", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
another := link.CheckedBuild{Pull: link.PullRef{Owner: "novox", Repo: "shop", Number: 8}, Check: "build-9",
Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
var asked [][]link.PullRef
checkedBuildsOf = func(_ context.Context, pulls []link.PullRef) ([]link.CheckedBuild, error) {
asked = append(asked, pulls)
return []link.CheckedBuild{mine, earlier, another}, nil
}
p := link.PullUpdated{Owner: "novox", Repo: "shop", Number: 7, Commit: "bbbbbbbbbbbb0123456789"}
builds := []link.CheckBuild{{Module: "app"}}
if got := namedForTheCheck(t.Context(), p, builds); !reflect.DeepEqual(got, []link.CheckedBuild{mine}) ||
!reflect.DeepEqual(asked, [][]link.PullRef{{pull}}) {
t.Fatalf("the check is handed %+v, read for %v", got, asked)
}
checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) {
return nil, errors.New("no bus")
}
if got := namedForTheCheck(t.Context(), p, builds); got != nil {
t.Fatalf("verdicts that could not be read named %+v", got)
}
}
func forgetWalkVerdicts() {
walkVerdicts.Lock()
defer walkVerdicts.Unlock()
walkVerdicts.read = map[string]verdictsRead{}
}
// A walk's merged heads are the forge's word for each merge; a merge announced without its head, or a pull request
// merged at two heads in one walk, has none, and its walk takes no check's build (novox/hq issue 703).
func TestAWalksMergedHeadsAreTheForgesWord(t *testing.T) {
merge := func(number int, head string) inventory.PlanMerge {
return inventory.PlanMerge{Repository: "novox/shop", Number: number, Head: head}
}
p := &inventory.Plan{Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
merge(7, "aaaaaaaaaaaa1"), merge(8, ""), merge(9, "bbbbbbbbbbbb"), merge(9, "cccccccccccc")}}}
heads := mergedHeadsOf(p)
shop := func(n int) link.PullRef { return link.PullRef{Owner: "novox", Repo: "shop", Number: n} }
if heads[shop(7)] != "aaaaaaaaaaaa1" || heads[shop(8)] != "" || heads[shop(9)] != "" {
t.Fatalf("merged heads %v", heads)
}
}
+3 -1
View File
@@ -82,7 +82,9 @@ func checkHereCommand(ctx context.Context, args []string) error {
if _, err := git("fetch", "--quiet", "origin", *base); err != nil {
return fmt.Errorf("cannot fetch %s to say what the change touches: %w", *base, err)
}
changedText, err := git("diff", "--name-only", "origin/"+*base+"...HEAD")
// Without rename detection, so a file moved out of a build source is said under its old name as well:
// its going is a change to the build that held it (novox/hq ADR 0267).
changedText, err := git("diff", "--name-only", "--no-renames", "origin/"+*base+"...HEAD")
if err != nil {
return err
}
+165
View File
@@ -0,0 +1,165 @@
package main
import (
"context"
"fmt"
"regexp"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A delivery group's member is checked beside the other members' heads** (novox/hq ADR 0382, issue 867).
//
// A mesh-host change and a mesh-controller change that need each other could not both pass their own merge checks:
// each was read beside the other's repository at the commit the mesh runs, which holds neither change. So the
// controller asks mesh-delivery's `groups` once for every pull request's check it asks (§1), and names, in the ask,
// the group's other open pull requests into a base of the same name, in a repository the build seat reads beside the
// check — mesh-lab never — each with its whole head. The build seat reads those repositories at the sibling's merge (§2).
// A pull request for which mesh-delivery answered and named no sibling is asked exactly as before (§7).
// wholeCommit is a commit as the forge says a head: forty hexadecimal characters.
var wholeCommit = regexp.MustCompile(`^[0-9a-f]{40}$`)
// besideRepositories are the repositories (lower case, owner/repo) a check reads beside its own, by the directory each
// is found under — the core's, as besideOf clones them, and mesh-sdk beside the controller; never mesh-lab, which is
// always read at its trunk.
func besideRepositories(entries []inventory.Entry) map[string]string {
read := map[string]string{}
for _, e := range entries {
dir, core := coreModules[e.Manifest.Module]
if !core || e.Provided || e.Source.Repository == "" {
continue
}
read[nameOf(e.Source.Repository)] = dir
if dir == "mesh-controller" && e.Source.Seat != "" {
read[nameOf(siblingOf(e.Source.Repository, "mesh-sdk"))] = "mesh-sdk"
}
}
delete(read, "")
return read
}
// nameOf is a recorded repository as owner/repo in lower case, empty when it names no owner.
func nameOf(repository string) string {
owner, repo := sourceOwner(repository), sourceName(repository)
if owner == "" || repo == "" {
return ""
}
return strings.ToLower(owner + "/" + repo)
}
// siblingsFor is the pull request's siblings, as mesh-delivery's groups say them (ADR 0382 §1): the other members of
// the group its head branch names that are open, unmerged and into a base of its own base's name, in a repository the
// check reads beside it (beside: the directories the ask already names). The second answer is why they could not be
// learned, empty when they could: the group's members carry no head or base — a mesh-delivery from before the rule.
func siblingsFor(p link.PullUpdated, entries []inventory.Entry, beside map[string]link.CheckedOut,
lines []groupsLine) ([]link.CheckSibling, string) {
own := strings.ToLower(p.Owner + "/" + p.Repo)
var group *groupsLine
if p.Head == "" {
// A recheck (mesh-delivery's `recheck` through `delivery check`) carries no head branch name: the group is the one
// whose members hold this very pull request, by repository and number.
learnable := false
for i := range lines {
for _, m := range lines[i].Details {
learnable = true
if p.Number != 0 && m.Number == p.Number && strings.ToLower(m.Repository) == own {
group = &lines[i]
}
}
}
if group == nil {
for _, l := range lines {
if len(l.Details) == 0 && len(l.Members) > 0 && !learnable {
return nil, "mesh-delivery named the groups' members without their pull requests (a mesh-delivery from before novox/hq ADR 0382): " +
"the group of a pull request announced without its head branch cannot be found"
}
}
return nil, "" // a lone pull request
}
} else {
for i := range lines {
if lines[i].ID == p.Head {
group = &lines[i]
}
}
if group == nil {
return nil, ""
}
}
if len(group.Details) == 0 && len(group.Members) > 0 {
return nil, "mesh-delivery named the group's members without their heads and bases (a mesh-delivery from before novox/hq ADR 0382)"
}
read := besideRepositories(entries)
var siblings []link.CheckSibling
for _, m := range group.Details {
repository := strings.ToLower(m.Repository)
if repository == own {
continue
}
if !m.Open {
continue
}
if m.Head == "" || m.Base == "" {
return nil, "mesh-delivery named " + m.ID + " without its head or base (a mesh-delivery from before novox/hq ADR 0382)"
}
dir, readBeside := read[repository]
if m.Base != p.Base || !readBeside || dir == "mesh-lab" {
continue
}
if _, there := beside[dir]; !there {
continue
}
head := strings.ToLower(m.Head)
if !wholeCommit.MatchString(head) {
return nil, fmt.Sprintf("mesh-delivery named %s with a head that is not a whole commit (%q)", m.ID, m.Head)
}
siblings = append(siblings, link.CheckSibling{Repository: m.Repository, Number: m.Number, Head: head, Base: m.Base, Dir: dir})
}
sort.Slice(siblings, func(i, j int) bool { return siblings[i].Dir < siblings[j].Dir })
return siblings, ""
}
// withSiblings asks mesh-delivery's `groups` once and says in the request the pull request's siblings, or why they
// could not be learned (ADR 0382 §1): a check never waits on mesh-delivery past groupAskWithin and never fails for it,
// so a fix to mesh-delivery is still checked without it.
func withSiblings(ctx context.Context, request *link.BuildRequest, p link.PullUpdated, entries []inventory.Entry) {
lines, err := groupsAsked(ctx, deliverySeatHeld(entries))
siblingsFrom(request, p, entries, lines, err)
}
// siblingsFrom is withSiblings from mesh-delivery's answer already had.
func siblingsFrom(request *link.BuildRequest, p link.PullUpdated, entries []inventory.Entry, lines []groupsLine, err error) {
if request.Check == nil {
return
}
if err != nil {
request.Check.SiblingsUnknown = fmt.Sprintf("mesh-delivery did not say the group's pull requests (%v): "+
"the check reads no siblings", err)
return
}
request.Check.Siblings, request.Check.SiblingsUnknown = siblingsFor(p, entries, request.Check.Beside, lines)
}
// composedMember says the pull request is a member of a delivery group whose gate is the composed check's (novox/hq
// ADR 0403 clauses 1 and 7): the group has two members or more and mesh-delivery says so on its line. The group is
// found as siblingsFor finds it, by its head branch name, or, for a recheck that carries none, by the pull request's
// repository and number — not by siblings, which name only what the check reads beside it.
func composedMember(p link.PullUpdated, lines []groupsLine) bool {
own := strings.ToLower(p.Owner + "/" + p.Repo)
for _, l := range lines {
found := p.Head != "" && l.ID == p.Head
for _, m := range l.Details {
if p.Head == "" && p.Number != 0 && m.Number == p.Number && strings.ToLower(m.Repository) == own {
found = true
}
}
if found {
return l.ComposedGate && max(len(l.Details), len(l.Members)) >= 2
}
}
return false
}
+248
View File
@@ -0,0 +1,248 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A delivery group's member is checked beside the other members' heads** (novox/hq ADR 0382 §1, §7, issue 867): the
// controller asks mesh-delivery's `groups` once for every check it asks, and names in the ask the group's other open
// pull requests into a base of the same name, in a repository the build seat reads beside it — mesh-lab never.
const (
headHost = "1111111111111111111111111111111111111111"
headController = "2222222222222222222222222222222222222222"
headSDK = "3333333333333333333333333333333333333333"
headLab = "4444444444444444444444444444444444444444"
headHq = "5555555555555555555555555555555555555555"
)
func theCoreOnTheGitSeat() []inventory.Entry {
onSeat := func(module, repository string) inventory.Entry {
e := fromRepo(module, repository, "")
e.Source.Seat = "git"
return e
}
nats := onSeat("nats", "novox/mesh-catalog")
nats.Source.Path = "modules/nats"
held := catalogue.Manifest{Module: "mesh-delivery", Claims: []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}}
return []inventory.Entry{onSeat("mesh-controller", "novox/mesh-controller"), onSeat("mesh-host", "novox/mesh-host"),
onSeat("node-tools", "novox/mesh-tools"), nats, {Manifest: held, On: []string{"anchor"}}}
}
func besideTheCore() map[string]link.CheckedOut {
return map[string]link.CheckedOut{"mesh-controller": {Repository: "c"}, "mesh-host": {Repository: "h"},
"mesh-tools": {Repository: "t"}, "mesh-catalog": {Repository: "k"}, "mesh-sdk": {Repository: "s"}, "mesh-lab": {Repository: "l"}}
}
func member(repository string, number int, head, base string, open bool) memberLine {
return memberLine{ID: repository + "@" + head[:12], Repository: repository, State: "checked", Number: number, Head: head,
Base: base, Open: open}
}
func group(id string, members ...memberLine) groupsLine {
line := groupsLine{ID: id, Details: members}
for _, m := range members {
line.Members = append(line.Members, m.ID+" ("+m.State+")")
}
return line
}
func TestASiblingIsTheGroupsOtherOpenPullRequestIntoABaseOfTheSameName(t *testing.T) {
entries := theCoreOnTheGitSeat()
pull := link.PullUpdated{Owner: "novox", Repo: "mesh-host", Number: 114, Base: "main", Head: "feat/x", Commit: headHost}
for _, c := range []struct {
what string
lines []groupsLine
want []string
unknown string
}{
{"the other member, open, same base", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))},
[]string{"mesh-controller=novox/mesh-controller#346@" + headController + " onto main"}, ""},
{"a member into a base of another name is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "release", true))},
nil, ""},
{"a member not open (merged or closed) is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", false))},
nil, ""},
{"mesh-lab is never one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-lab", 5, headLab, "main", true))},
nil, ""},
{"a repository the check reads nothing of is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/hq", 9, headHq, "main", true))},
nil, ""},
{"mesh-sdk, read beside the controller, is one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-sdk", 7, headSDK, "main", true))},
[]string{"mesh-sdk=novox/mesh-sdk#7@" + headSDK + " onto main"}, ""},
{"another group is none of its business", []groupsLine{group("feat/y",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))},
nil, ""},
{"a group of its own alone has none", []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true))},
nil, ""},
{"an older mesh-delivery names no heads", []groupsLine{{ID: "feat/x",
Members: []string{"novox/mesh-host@111111111111 (checked)", "novox/mesh-controller@222222222222 (checked)"}}},
nil, "without their heads and bases"},
{"a member without its base", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "", true))},
nil, "without its head or base"},
{"a head that is no commit", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController[:12]+"zzzzzzzzzzzzzzzzzzzzzzzzzzzz", "main", true))},
nil, "not a whole commit"},
} {
got, unknown := siblingsFor(pull, entries, besideTheCore(), c.lines)
var said []string
for _, s := range got {
said = append(said, s.Dir+"="+s.Repository+"#"+itoa(s.Number)+"@"+s.Head+" onto "+s.Base)
}
if strings.Join(said, ";") != strings.Join(c.want, ";") {
t.Errorf("%s: siblings %v, wanted %v", c.what, said, c.want)
}
if (c.unknown == "") != (unknown == "") || !strings.Contains(unknown, c.unknown) {
t.Errorf("%s: could not learn %q, wanted %q", c.what, unknown, c.unknown)
}
}
// A pull request into a release branch has siblings into the release branch of the same name (ADR 0358 §1).
release := pull
release.Base = "feat/57"
lines := []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "feat/57", true),
member("novox/mesh-controller", 346, headController, "feat/57", true), member("novox/mesh-tools", 8, headLab, "main", true))}
if got, _ := siblingsFor(release, entries, besideTheCore(), lines); len(got) != 1 || got[0].Dir != "mesh-controller" {
t.Errorf("into a release branch: %+v", got)
}
// A recheck announces the pull request without its head branch: its group is the one holding it (repository, number).
pull.Head = ""
got, unknown := siblingsFor(pull, entries, besideTheCore(), lines)
if len(got) != 1 || got[0].Dir != "mesh-tools" || unknown != "" {
t.Errorf("a recheck of a group member: %+v %q", got, unknown)
}
// A pull request in no group, rechecked, is alone: no sibling, and nothing said.
pull.Number = 999
if got, unknown := siblingsFor(pull, entries, besideTheCore(), lines); got != nil || unknown != "" {
t.Errorf("a lone pull request: %+v %q", got, unknown)
}
if got, unknown := siblingsFor(pull, entries, besideTheCore(), nil); got != nil || unknown != "" {
t.Errorf("no groups at all: %+v %q", got, unknown)
}
// A mesh-delivery that names members without their pull requests cannot find the group of a recheck: it says so.
older := []groupsLine{{ID: "feat/x", Members: []string{"novox/mesh-host@111111111111 (checked)"}}}
pull.Number = 114
if got, unknown := siblingsFor(pull, entries, besideTheCore(), older); got != nil || unknown == "" {
t.Errorf("an older mesh-delivery: %+v %q", got, unknown)
}
}
func itoa(n int) string { b, _ := json.Marshal(n); return string(b) }
// Every check asks mesh-delivery's `groups` once; the ask names the sibling's repository, head and base.
func TestEveryCheckAsksMeshDeliveryOnceAndNamesItsSiblings(t *testing.T) {
entries := theCoreOnTheGitSeat()
asked := 0
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
groupsAsked = func(_ context.Context, held bool) ([]groupsLine, error) {
asked++
if !held {
t.Error("the seat is held, yet it was asked as not held")
}
return []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true),
member("novox/mesh-controller", 346, headController, "main", true))}, nil
}
request := link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main", Beside: besideTheCore()}}
withSiblings(t.Context(), &request, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
if asked != 1 {
t.Fatalf("mesh-delivery was asked %d times for one check", asked)
}
want := link.CheckSibling{Repository: "novox/mesh-controller", Number: 346, Head: headController, Base: "main", Dir: "mesh-controller"}
if got := request.Check.Siblings; len(got) != 1 || got[0] != want || request.Check.SiblingsUnknown != "" {
t.Fatalf("the ask names siblings %+v, unknown %q", got, request.Check.SiblingsUnknown)
}
body, _ := json.Marshal(request.Check)
if !strings.Contains(string(body), `"siblings":[{"repository":"novox/mesh-controller","number":346,"head":"`+headController+
`","base":"main","dir":"mesh-controller"}]`) {
t.Fatalf("on the wire: %s", body)
}
}
// When mesh-delivery does not answer in time (or is not held, or answered without the fields), the check reads no
// sibling and its request says it could not learn them, so a fix to mesh-delivery is still checked without it.
func TestWhenMeshDeliveryDoesNotAnswerTheCheckReadsNoSiblingAndSaysSo(t *testing.T) {
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
for _, c := range []struct {
what string
err error
held bool
}{{"it did not answer", context.DeadlineExceeded, true}, {"nothing serves it", errors.New("nothing serves mesh-delivery.groups"), true},
{"no holder on record", errors.New("no holder of mesh-delivery is on record to say it"), false}} {
entries := theCoreOnTheGitSeat()
if !c.held {
entries = entries[:4]
}
groupsAsked = func(_ context.Context, held bool) ([]groupsLine, error) {
if held != c.held {
t.Errorf("%s: held %v", c.what, held)
}
return nil, c.err
}
request := link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main", Beside: besideTheCore()}}
withSiblings(t.Context(), &request, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
if len(request.Check.Siblings) != 0 || !strings.Contains(request.Check.SiblingsUnknown, c.err.Error()) ||
!strings.Contains(request.Check.SiblingsUnknown, "reads no siblings") {
t.Errorf("%s: siblings %+v, unknown %q", c.what, request.Check.Siblings, request.Check.SiblingsUnknown)
}
}
}
// **A pull request for which mesh-delivery answered and named no sibling is asked exactly as before** (ADR 0382 §7):
// byte for byte the request it was without the rule.
func TestAPullRequestWithNoSiblingIsAskedByteForByteAsBefore(t *testing.T) {
entries := theCoreOnTheGitSeat()
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
for _, answer := range [][]groupsLine{nil, {group("feat/other", member("novox/mesh-controller", 346, headController, "main", true))},
{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true), member("novox/hq", 3, headHq, "main", true))}} {
groupsAsked = func(context.Context, bool) ([]groupsLine, error) { return answer, nil }
build := func() link.BuildRequest {
return link.BuildRequest{ID: "build-1", Repository: "r", Ref: headHost, Check: &link.CheckRequest{Owner: "novox",
Repo: "mesh-host", Number: 114, Base: "main", Beside: besideTheCore(), Modules: []string{"mesh-host"}, Judge: "validator"}}
}
before, _ := json.Marshal(build())
after := build()
withSiblings(t.Context(), &after, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
got, _ := json.Marshal(after)
if string(before) != string(got) {
t.Fatalf("the request changed:\nbefore %s\nafter %s", before, got)
}
}
}
// What the build seat's verdict says of its siblings reaches the forge's holder in the controller's `checked`.
func TestTheVerdictCarriesTheSiblingsAndWhereTheJudgeCameFrom(t *testing.T) {
read := []link.CheckSibling{{Repository: "novox/mesh-controller", Number: 346, Head: headController, Base: "main", BaseCommit: headHost}}
c := checkedOf(link.BuildResult{ID: "b", Ref: headHost, Checked: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114},
Check: &link.CheckOutcome{Verdict: "pass", Summary: "ok", Siblings: read, JudgeFrom: "sibling novox/mesh-controller#346 merge abc",
Validator: "the change's own validator"}})
if len(c.Siblings) != 1 || c.Siblings[0] != read[0] || c.JudgeFrom == "" || c.Validator == "" {
t.Fatalf("the verdict says %+v", c)
}
body, _ := json.Marshal(c)
for _, want := range []string{`"siblings":[{"repository":"novox/mesh-controller","number":346,"head":"` + headController +
`","base":"main","base_commit":"` + headHost + `"}]`, `"judge_from":"sibling novox/mesh-controller#346 merge abc"`} {
if !strings.Contains(string(body), want) {
t.Fatalf("the verdict lacks %s:\n%s", want, body)
}
}
unknown := checkedOf(link.BuildResult{ID: "b", Check: &link.CheckOutcome{Verdict: "pass", SiblingsUnknown: "mesh-delivery did not answer"}})
if body, _ := json.Marshal(unknown); !strings.Contains(string(body), `"siblings_unknown":"mesh-delivery did not answer"`) ||
strings.Contains(string(body), `"siblings":`) {
t.Fatalf("the verdict that could not learn them says %s", body)
}
}
+61 -5
View File
@@ -6,8 +6,10 @@ import (
"path/filepath"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
"testing"
"github.com/novox/mesh-controller/internal/beside"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The check anybody can run is the check registration runs (novox/hq issue 148, ADR 0037): a manifest
@@ -56,10 +58,7 @@ func TestModuleCheckJudgesBetweenTheManifestsGiven(t *testing.T) {
// The real catalogue passes the command, the way it passes the test that used to be the only check.
func TestModuleCheckPassesTheCatalogue(t *testing.T) {
root := filepath.Join("..", "..", "..", "mesh-catalog", "modules")
if _, err := os.Stat(root); err != nil {
t.Skipf("catalogue sibling not present: %v", err)
}
root := beside.Catalogue(t)
paths, err := manifestsUnder(root)
if err != nil || len(paths) == 0 {
t.Fatalf("no manifests under %s: %v", root, err)
@@ -103,3 +102,60 @@ func TestTheControllersManifestServesEveryVerbOfItsSeat(t *testing.T) {
t.Fatalf("the controller's own module.json fails module check: %v\n%s", err, out.String())
}
}
// A module that uses a seat another module declares (novox/hq issue 364): refused over the whole catalogue when the
// declarer is missing, a note when the gate says it gives only the manifests a change touches.
func TestASeatAnotherModuleDeclaresIsANoteOverSomeManifests(t *testing.T) {
dir := t.TempDir()
user := filepath.Join(dir, "user.json")
os.WriteFile(user, []byte(`{"module":"asker","version":"1","uses":["operator-channel"]}`), 0o600)
var out bytes.Buffer
if err := moduleCheck([]string{user}, &out); err == nil {
t.Fatalf("a use of a seat nothing given declares passed the whole-catalogue check:\n%s", out.String())
}
t.Setenv(SomeManifestsVar, "1")
out.Reset()
if err := moduleCheck([]string{user}, &out); err != nil {
t.Fatalf("over some manifests the use was refused:\n%s", out.String())
}
if !strings.Contains(out.String(), "note: asker uses the seat \"operator-channel\"") {
t.Fatalf("the note was not said:\n%s", out.String())
}
}
// **A module lives whole in one repository** (novox/hq ADR 0420 §1, issue 1019): `module check`, told the
// repository it runs for, refuses an artifact whose build context names another; run by hand, it does not know.
func TestModuleCheckRefusesAnArtifactContextNamingAnotherRepository(t *testing.T) {
dir := t.TempDir()
write := func(name, context string) string {
path := filepath.Join(dir, name+".json")
raw := `{"module":"` + name + `","version":"1","build":{"artifacts":[{"name":"server","kind":"image","from":"Dockerfile"` + context + `}]}}`
if err := os.WriteFile(path, []byte(raw), 0o600); err != nil {
t.Fatal(err)
}
return path
}
other := write("other", `,"context":{"seat":"git","repository":"novox/mesh-controller","ref":"main"}`)
own := write("own", `,"context":{"seat":"git","repository":"novox/mesh-catalog","ref":"main"}`)
none := write("none", ``)
t.Setenv("MESH_CHECK_REPOSITORY", "novox/mesh-catalog")
var out bytes.Buffer
if err := moduleCheck([]string{other}, &out); err == nil {
t.Fatalf("a context naming another repository passed:\n%s", out.String())
}
if !strings.Contains(out.String(), "other builds artifact server from novox/mesh-controller: a module lives whole in one repository (novox/hq ADR 0420 §1)") {
t.Errorf("the refusal is not in the words of the gate:\n%s", out.String())
}
for _, path := range []string{own, none} {
out.Reset()
if err := moduleCheck([]string{path}, &out); err != nil {
t.Errorf("%s was refused: %v\n%s", path, err, out.String())
}
}
t.Setenv("MESH_CHECK_REPOSITORY", "")
out.Reset()
if err := moduleCheck([]string{other}, &out); err != nil {
t.Errorf("a local run, with no repository named, refused it: %v\n%s", err, out.String())
}
}
+290 -27
View File
@@ -5,6 +5,8 @@ import (
"encoding/json"
"fmt"
"path"
"regexp"
"runtime/debug"
"slices"
"sort"
"strings"
@@ -143,7 +145,7 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
if err != nil {
return err
}
read, err := inv.ReadRepositories(ctx)
read, err := readForPlanning(ctx, inv)
if err != nil {
return err
}
@@ -152,6 +154,24 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
return err
}
scope := pullScope(p, entries, read, edges)
// **A cycle of edges fails the check, naming every module and edge** (novox/hq ADR 0419 clause 6), a lone pull
// request's as a group's composed check is: no order exists for what it changes.
alone := orderMember{ID: p.Owner + "/" + p.Repo + "@" + twelve(p.Commit), Repository: p.Owner + "/" + p.Repo, Base: p.Base,
Head: p.Commit, Number: p.Number, Paths: p.Paths, PathsTruncated: p.PathsTruncated, Removed: p.Removed, BaseSHA: p.BaseSHA}
if failing, cyclic, err := cycleVerdict(ctx, "", []orderMember{alone},
map[string]orderReach{alone.ID: {Moved: scope.Modules}}, edges, entries); err != nil {
return err
} else if cyclic {
failing.Commit, failing.Trunk = p.Commit, twelve(p.BaseSHA)
fmt.Printf("%s/%s#%d (%.8s): %s\n", p.Owner, p.Repo, p.Number, p.Commit, failing.Summary)
sayChecked(ctx, failing)
return nil
}
// **A member of a group whose gate is the composed check's runs its repository's check alone** (novox/hq ADR 0403
// clause 1): mesh-delivery is asked once, and says so on the group's line. Told nothing, the pull request is
// checked alone, gate included (clause 6).
lines, groupsErr := groupsAsked(ctx, deliverySeatHeld(entries))
composed := groupsErr == nil && composedMember(p, lines)
// The change plan of the commit at hand (ADR 0238): what a merge of it would build and send, posted
// with the verdict whatever the verdict is.
plan := changePlanOf(p.Owner+"/"+p.Repo, p.Base, p.Commit, scope.Reach, entries, func(module string) (inventory.Upgrade, bool) {
@@ -159,9 +179,11 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
return u, err == nil
})
fmt.Print(planText(plan))
direct := link.Checked{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Commit: p.Commit,
ID: link.NewBuildID(time.Now()), Verdict: "pass", Summary: noModuleTouched,
Gate: &link.CheckLayer{Verdict: "pass", Summary: noModuleTouched}, Plan: &plan}
direct := directVerdict(p, plan)
if composed {
scope.Modules, scope.Dependents, scope.New, scope.Manifests = nil, nil, nil, nil
composedGate(&direct)
}
switch {
case !scope.gated() && !scope.Mesh:
fmt.Printf("%s/%s#%d (%.8s): %s, and the repository is not the mesh's: said, nothing run\n",
@@ -179,6 +201,9 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
return err
}
request.Check.Plan = &plan
request.Check.Composed = composed
// **The group's other open pull requests are read beside it** (novox/hq ADR 0382 §1).
siblingsFrom(&request, p, entries, lines, groupsErr)
seat := buildSeatHeld(ctx)
ask, err := askOverOn(seat)
if err != nil {
@@ -188,7 +213,13 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
if err := ask.Ask(ctx, request); err != nil {
return err
}
// **A newer request replaces the one still waiting, and a newer key stops the one running** (novox/hq ADR 0406 §3, §4,
// issue 974): the check queue keeps one request a pull request, and a running check watches its key. Nothing is asked
// of the forge here (issue 726).
what := "its own merge-check.sh alone: " + noModuleTouched
if composed {
what = "its own merge-check.sh alone: the gate is its group's composed check's"
}
if scope.gated() {
what = fmt.Sprintf("the gate over %s (a merge would build %d module(s) in %d tier(s))",
strings.Join(append(append([]string{}, scope.Modules...), prefixedAll("new:", scope.New)...), ", "),
@@ -199,6 +230,38 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
return nil
}
// composedWords is what a group member's own verdict says of its gate layer (novox/hq ADR 0403 clause 1).
const composedWords = "judged by its group's composed check"
// gateComposed is a gate layer's verdict when no gate was run for the member alone: never a pass. Mesh-delivery reads it
// as the composed check's to judge, and the forge's holder sets no mesh/merge-gate from it.
const gateComposed = "composed"
// composedGate says a group member's own verdict has no gate of its own: its gate layer, and the verdict that is
// the gate's, read "composed" in place of a pass.
func composedGate(c *link.Checked) {
c.Verdict, c.Summary = gateComposed, composedWords
c.Gate = &link.CheckLayer{Verdict: gateComposed, Summary: composedWords}
}
// directVerdict is the verdict of a change no build judges — it touches no module, and either the repository is not
// the mesh's or the head holds no merge-check.sh. **It names the trunk commit it holds for** (novox/hq ADR 0323 §3):
// the base branch's head the forge announced with the head, so the merge tool can weigh it against the trunk's head
// and does not refuse it for ever (a verdict naming no trunk is never the trunk's head's). Nothing was built to
// judge: the trunk commit is the one the forge said the change was announced against.
func directVerdict(p link.PullUpdated, plan link.ChangePlan) link.Checked {
trunk := p.BaseSHA
if len(trunk) > 12 {
trunk = trunk[:12]
}
if trunk != "" {
plan.Trunk = trunk
}
return link.Checked{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Commit: p.Commit,
ID: link.NewBuildID(time.Now()), Verdict: "pass", Summary: noModuleTouched,
Gate: &link.CheckLayer{Verdict: "pass", Summary: noModuleTouched}, Plan: &plan, Trunk: trunk}
}
// checkRequestFor is the ask for one pull request's head: the repository as the mesh clones it, the head,
// and what is read beside it.
func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scope checkScope,
@@ -230,7 +293,29 @@ func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scop
if err != nil {
return link.BuildRequest{}, err
}
// Beside it, at what the mesh runs: each core repository by the module the mesh builds from it.
beside, err := besideOf(entries, func(module string) string { return current[module].Commit }, clone)
if err != nil {
return link.BuildRequest{}, err
}
builds := checkBuildsOf(p, scope, entries, clone)
return link.BuildRequest{
ID: link.NewBuildID(time.Now()),
Repository: repository,
Ref: p.Commit,
Held: heldBy(ctx),
Seats: seatBases(ctx),
Source: sourceOnSeat(source),
Check: &link.CheckRequest{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Base: p.Base, Trunk: p.BaseSHA, Ready: p.Ready,
Paths: p.Paths, Beside: beside, Modules: scope.Modules, Dependents: scope.Dependents, New: scope.New,
Manifests: scope.Manifests, Judge: scope.Judge, Builds: builds, Named: namedForTheCheck(ctx, p, builds),
Running: runningBy(ctx)},
}, nil
}
// besideOf is what a check reads beside its own repository, by the directory it is found under: each core repository
// by the module the mesh builds from it, at besideRefs. running is the commit the mesh runs of a module.
func besideOf(entries []inventory.Entry, running func(module string) string,
clone func(inventory.Source) (string, error)) (map[string]link.CheckedOut, error) {
beside := map[string]link.CheckedOut{}
for _, e := range entries {
dir, core := coreModules[e.Manifest.Module]
@@ -239,31 +324,115 @@ func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scop
}
url, err := clone(e.Source)
if err != nil {
return link.BuildRequest{}, err
return nil, err
}
refs := besideRefs(dir, current[e.Manifest.Module].Commit)
refs := besideRefs(dir, running(e.Manifest.Module))
beside[dir] = link.CheckedOut{Repository: url, Ref: refs[dir]}
if dir == "mesh-controller" {
// The controller's main stays its main: it judges only a check the running controller predates.
beside["mesh-controller-main"] = link.CheckedOut{Repository: url, Ref: refs["mesh-controller-main"]}
if e.Source.Seat != "" {
if lab, err := clone(inventory.Source{Seat: e.Source.Seat, Repository: siblingOf(e.Source.Repository,
"mesh-lab")}); err == nil {
beside["mesh-lab"] = link.CheckedOut{Repository: lab, Ref: refs["mesh-lab"]}
for _, sibling := range []string{"mesh-lab", "mesh-sdk"} {
if url, err := clone(inventory.Source{Seat: e.Source.Seat, Repository: siblingOf(e.Source.Repository,
sibling)}); err == nil {
beside[sibling] = link.CheckedOut{Repository: url, Ref: refs[sibling]}
}
}
}
}
}
return link.BuildRequest{
ID: link.NewBuildID(time.Now()),
Repository: repository,
Ref: p.Commit,
Held: heldBy(ctx),
Seats: seatBases(ctx),
Source: sourceOnSeat(source),
Check: &link.CheckRequest{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Base: p.Base,
Paths: p.Paths, Beside: beside, Modules: scope.Modules, Dependents: scope.Dependents, New: scope.New,
Manifests: scope.Manifests, Judge: scope.Judge},
}, nil
return beside, nil
}
// trunkOf is the branch a module's source follows: main when it records none, or records a commit.
func trunkOf(s inventory.Source) string {
if t := strings.TrimSpace(followedBranch(s.Ref)); t != "" {
return t
}
return "main"
}
// sourceName is the name of a recorded repository, a path on the git seat or a URL: novox/mesh-host → mesh-host.
func sourceName(repository string) string {
parts := strings.Split(strings.Trim(strings.TrimSuffix(repository, ".git"), "/"), "/")
if len(parts) < 2 {
return ""
}
return parts[len(parts)-1]
}
// namedForTheCheck are the builds earlier checks of the same pull request at the same head named, read once for the
// check (novox/hq issue 703): asked again because only the trunk moved, the check keeps what they built of a key
// still the same. Another pull request's build, or one of an earlier head, is never kept. None when they cannot be
// read, and the check builds, as before.
func namedForTheCheck(ctx context.Context, p link.PullUpdated, builds []link.CheckBuild) []link.CheckedBuild {
if len(builds) == 0 || p.Commit == "" {
return nil
}
pull := link.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number}
read, err := checkedBuildsOf(ctx, []link.PullRef{pull})
if err != nil {
fmt.Printf(" the earlier checks' builds of %s/%s#%d could not be read, so the check builds: %v\n",
p.Owner, p.Repo, p.Number, err)
// empty-on-error: said above; a check handed no named build builds every module, as before issue 703
return nil
}
var named []link.CheckedBuild
for _, b := range read {
if b.Pull == pull && link.SameHead(b.Head, p.Commit) {
named = append(named, b)
}
}
return named
}
// checkBuildsOf are the modules a merge of the change would build, in the walk's order, as the check builds them
// for the walk to take (novox/hq ADR 0350 §1): a module built from the repository checked, at the branch it merges
// into, from the merge the check makes; any other — a module standing on one — from its own repository at the
// branch it follows, as the walk builds it. A module whose build reads the repository checked through a context,
// or that the merge deletes, cannot be built as the walk would: said, and nothing after it is built. A change that
// moves no module builds nothing.
func checkBuildsOf(p link.PullUpdated, scope checkScope, entries []inventory.Entry,
clone func(inventory.Source) (string, error)) []link.CheckBuild {
if len(scope.Reach.Plan.Tiers) == 0 {
return nil
}
m := link.SourceMoved{Owner: p.Owner, Repo: p.Repo, Base: p.Base, CloneURL: p.CloneURL}
byName := map[string]inventory.Entry{}
for _, e := range entries {
byName[e.Manifest.Module] = e
}
skip := map[string]string{}
for _, e := range scope.Reach.Packaging {
skip[e.Manifest.Module] = "its build reads " + p.Owner + "/" + p.Repo + " through a context, which a check " +
"cannot hand the merge"
}
for _, e := range scope.Reach.Deleted {
skip[e.Manifest.Module] = "the merge deletes it"
}
var out []link.CheckBuild
for tier, names := range scope.Reach.Plan.Tiers {
for _, name := range names {
e, known := byName[name]
b := link.CheckBuild{Module: name, Tier: tier, Skip: skip[name]}
switch {
case !known:
b.Skip = "the catalogue does not hold it"
case b.Skip != "":
case sourceIs(e.Source, m):
b.Merged, b.Path = true, e.Source.Path
default:
url, err := clone(e.Source)
if err != nil {
b.Skip = "its repository cannot be cloned: " + err.Error()
break
}
b.Repository, b.Path, b.Ref = url, e.Source.Path, followedBranch(e.Source.Ref)
}
out = append(out, b)
}
}
return out
}
// besideRefs is the ref each repository is cloned at beside a check, by the directory it is found under:
@@ -271,17 +440,54 @@ func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scop
// catalogue, whose checkout beside is what tests read its files from, at its main, what the next merge
// builds from; and beside the controller its main, for a judge the running controller predates, and the
// lab's main, whose replays every check runs. **One rule, read by the check the controller asks for and by
// the facts snapshot** (Facts.Beside), so a check run by hand clones what the build seat clones.
// the facts snapshot** (Facts.Beside), so a check run by hand clones what the build seat clones. Beside the
// controller also the SDK, checked out at the commit the running controller's go.mod pins (sdkPinned), not
// one a desktop holds (novox/hq issue 449). The checkout only places the clone: the conformance test reads the
// fixtures at the pin of the tree under check, from the clone's history, so a pull request moving the SDK is
// judged against the SDK it moves to (internal/link/conformance_test.go).
func besideRefs(dir, running string) map[string]string {
switch dir {
case "mesh-catalog":
return map[string]string{dir: "main"}
case "mesh-controller":
return map[string]string{dir: running, "mesh-controller-main": "main", "mesh-lab": "main"}
return map[string]string{dir: running, "mesh-controller-main": "main", "mesh-lab": "main",
"mesh-sdk": sdkPinned()}
}
return map[string]string{dir: running}
}
// sdkModule is the Go module of the SDK the controller is built against.
const sdkModule = "git.novox.be/novox/mesh-sdk/go"
// pseudoCommit is the commit a Go pseudo-version names: v0.1.11-0.20261009143344-f047d0a4a970 → f047d0a4a970.
var pseudoCommit = regexp.MustCompile(`-([0-9a-f]{12})$`)
// sdkPinned is the ref of the SDK repository this controller was built from, as its go.mod pins it and its
// build records it: a pseudo-version's commit, or a release's tag (the SDK tags its Go module under go/).
// "main" only for a binary that records no SDK version — a local replace — which a running controller is
// not (novox/hq issue 449).
func sdkPinned() string {
info, ok := debug.ReadBuildInfo()
if !ok {
return "main"
}
for _, dep := range info.Deps {
if dep.Path != sdkModule {
continue
}
if dep.Replace != nil {
dep = dep.Replace
}
if m := pseudoCommit.FindStringSubmatch(dep.Version); m != nil {
return m[1]
}
if strings.HasPrefix(dep.Version, "v") {
return "go/" + dep.Version
}
}
return "main"
}
// siblingOf is another repository of the same owner: novox/mesh-controller → novox/mesh-lab.
func siblingOf(repository, name string) string {
if cut := strings.LastIndex(repository, "/"); cut >= 0 {
@@ -308,6 +514,13 @@ const maxCheckReport = 60 << 10
// checked says a merge check's verdict as the controller's `checked`. Nothing is recorded or
// registered: a check builds nothing (issue 240's rule for a dry run, kept for a check).
func checked(ctx context.Context, result link.BuildResult) {
// **A check stopped by its key says nothing** (novox/hq ADR 0406 §4, issue 974, review 2): it stopped because the head or
// the trunk commit moved on, and the newer request will judge. Said as an error it would name the live head — red
// statuses on the forge, a delivery refused — and written to the key it would stand in the newer request's way.
if result.Failed == link.SupersededByKey {
fmt.Printf("%s: stopped, its pull request's key moved on; the newer check will judge\n", result.ID)
return
}
sayChecked(ctx, checkedOf(result))
}
@@ -322,6 +535,12 @@ func checkedOf(result link.BuildResult) link.Checked {
case result.Check != nil:
c.Verdict, c.Summary, c.Report = result.Check.Verdict, result.Check.Summary, result.Check.Report
c.Gate, c.RepoCheck = result.Check.Gate, result.Check.RepoCheck
// What was judged (novox/hq ADR 0323 §3): the head merged into the trunk commit, and the commits of the
// repositories read beside it.
c.Trunk, c.Beside, c.Took = result.Check.Trunk, result.Check.Beside, result.Check.Took
c.Built = result.Check.Built
c.Validator = result.Check.Validator
c.Siblings, c.SiblingsUnknown, c.JudgeFrom = result.Check.Siblings, result.Check.SiblingsUnknown, result.Check.JudgeFrom
if c.Gate == nil {
// A build seat from before the layers: its verdict is the gate's.
c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
@@ -339,6 +558,12 @@ func checkedOf(result link.BuildResult) link.Checked {
c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
c.RepoCheck = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
}
// **A group member's own check has no gate of its own** (novox/hq ADR 0403 clause 1): what ran was the repository's
// check alone, and its gate layer is "composed", never a pass. A failure or an error stays what it was.
if result.Checked != nil && result.Checked.Composed && result.Check != nil && c.Gate != nil &&
(c.Gate.Verdict == "pass" || c.Gate.Verdict == "warning") {
composedGate(&c)
}
if result.Checked != nil && c.Gate != nil && len(c.Gate.Modules) == 0 {
c.Gate.Modules = append(append([]string{}, result.Checked.Modules...), prefixedAll("new:", result.Checked.New)...)
}
@@ -347,14 +572,20 @@ func checkedOf(result link.BuildResult) link.Checked {
}
if result.Checked != nil {
c.Plan = result.Checked.Plan
if c.Plan != nil && c.Trunk != "" {
// The kept delivery plan records the trunk commit with its other inputs (ADR 0323 §3).
plan := *c.Plan
plan.Trunk = c.Trunk
c.Plan = &plan
}
// A delivery group's composed check (novox/hq ADR 0239): every head it judged, this one first.
if g := result.Checked; g.Group != "" {
c.Group = g.Group
c.Members = append(c.Members, link.CheckedMember{Owner: g.Owner, Repo: g.Repo, Number: g.Number,
Commit: result.Ref})
Commit: result.Ref, Trunk: c.Trunk})
for _, m := range g.Members {
c.Members = append(c.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo, Number: m.Number,
Commit: m.Ref})
Commit: m.Ref, Trunk: memberTrunk(result.Check, m)})
}
}
}
@@ -363,12 +594,40 @@ func checkedOf(result link.BuildResult) link.Checked {
l.Verdict = "error"
}
}
// **A check run again says so** (novox/hq issue 594). The summaries are the lines the pull request
// shows, clipped by the forge to a few words, so they get only a short marker after the verdict's own
// words; the report gets the whole sentence. Layers copied, so the build's own outcome is left as it came.
if mark := link.RunMarker(result.Delivery); mark != "" {
c.Summary += " " + mark
for _, l := range []**link.CheckLayer{&c.Gate, &c.RepoCheck} {
if *l != nil {
layer := **l
layer.Summary += " " + mark
*l = &layer
}
}
c.Report = link.Restarted(result.Delivery) + "\n\n" + c.Report
}
if len(c.Report) > maxCheckReport {
c.Report = "…" + c.Report[len(c.Report)-maxCheckReport:]
}
return c
}
// memberTrunk is the trunk commit the build seat merged one of a group's heads into (novox/hq ADR 0323 §6), empty
// when it said none.
func memberTrunk(outcome *link.CheckOutcome, m link.GroupMember) string {
if outcome == nil {
return ""
}
for _, j := range outcome.Members {
if strings.EqualFold(j.Owner, m.Owner) && strings.EqualFold(j.Repo, m.Repo) && strings.HasPrefix(m.Ref, j.Commit) {
return j.Trunk
}
}
return ""
}
func prefixedAll(prefix string, items []string) []string {
out := make([]string, 0, len(items))
for _, i := range items {
@@ -383,8 +642,12 @@ func sayChecked(ctx context.Context, c link.Checked) {
if c.RepoCheck != nil {
repo = strings.ToUpper(c.RepoCheck.Verdict) + " — " + c.RepoCheck.Summary
}
fmt.Printf("%s: %s/%s#%d at %.8s checked on %s: gate %s — %s; repository %s\n", c.ID, c.Owner, c.Repo, c.Number,
c.Commit, orSomewhere(c.On), strings.ToUpper(c.Verdict), c.Summary, repo)
judged := ""
if c.Trunk != "" {
judged = " merged into trunk " + c.Trunk
}
fmt.Printf("%s: %s/%s#%d at %.8s%s checked on %s: gate %s — %s; repository %s\n", c.ID, c.Owner, c.Repo, c.Number,
c.Commit, judged, orSomewhere(c.On), strings.ToUpper(c.Verdict), c.Summary, repo)
if checkEvents == nil {
return
}
@@ -0,0 +1,56 @@
//go:build integration
package main
import (
"encoding/json"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
func TestACheckStoppedByItsKeyIsAnnouncedAsNoVerdict(t *testing.T) {
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
if err := broker.AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
legacy, err := js.Conn().JetStream()
if err != nil {
t.Fatal(err)
}
was := checkEvents
checkEvents = link.OverNATS{Conn: js.Conn(), JS: legacy}
t.Cleanup(func() { checkEvents = was })
heard := make(chan *nats.Msg, 4)
sub, err := js.Conn().ChanSubscribe(link.SeatEventSubject(link.MeshControllerSeat, link.KeyChecked), heard)
if err != nil {
t.Fatal(err)
}
defer func() { _ = sub.Unsubscribe() }()
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog", Number: 3}
checked(t.Context(), link.BuildResult{ID: "stopped", Ref: "abc", Checked: asked, Failed: link.SupersededByKey})
checked(t.Context(), link.BuildResult{ID: "broken", Ref: "abc", Checked: asked, Failed: "the facts snapshot cannot be read"})
select {
case m := <-heard:
var c link.Checked
_ = json.Unmarshal(m.Data, &c)
if c.ID != "broken" || c.Verdict != "error" {
t.Fatalf("the first verdict said is %+v; want only the broken check's error", c)
}
case <-time.After(5 * time.Second):
t.Fatal("nothing was said")
}
select {
case m := <-heard:
t.Fatalf("a second verdict was said: %s", m.Data)
case <-time.After(300 * time.Millisecond):
}
}
+97
View File
@@ -0,0 +1,97 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"fmt"
"testing"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// Acceptance 5 of milestone 2 (novox/hq ADR 0406 §3, issue 974), live, with the store and a real bus: two merges into one
// repository within a minute, with two open pull requests, leave each pull request with exactly one queued check, by the
// stream's rule. Each merge moves the trunk, and mesh-delivery asks the controller to check both open pull requests again
// against it (`delivery check`, which is the controller's own path for a head announced); nothing moves a request, nothing
// asks the forge, and the queue is read back whole.
func TestTwoMergesWithinAMinuteLeaveEachOpenPullRequestOneQueuedCheck(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
js, _ := aBuildQueue(t)
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
holdTheDeliverySeat(t, open)
was := askDeliveryOwner
t.Cleanup(func() { askDeliveryOwner = was })
askDeliveryOwner = func(_ context.Context, _ *nats.Conn, verb string, _ map[string]any) (json.RawMessage, error) {
if verb != "groups" {
t.Fatalf("asked the owner %s", verb)
}
return json.RawMessage(`[]`), nil
}
// The repository the two pull requests are into, as a module the mesh builds from it.
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
heads := map[int]string{7: "7777777777777777777777777777777777777777", 8: "8888888888888888888888888888888888888888"}
check := func(number int, trunk string) {
t.Helper()
p := orderMember{ID: fmt.Sprintf("novox/mesh-controller@%.12s", heads[number]), Repository: "novox/mesh-controller",
Base: "main", Head: heads[number], Number: number, Paths: []string{"main.go"}, ModuleDirsSaid: true,
BaseSHA: trunk, Ready: number == 8}.pull()
p.MergeCheck, p.MergeCheckSaid = true, true
if err := (following{open: open, oneShot: true}).PullUpdated(ctx, p); err != nil {
t.Fatal(err)
}
}
// Announced against the trunk as it stands, then asked again after each of two merges.
for _, trunk := range []string{"1111111111111111111111111111111111111111", "2222222222222222222222222222222222222222",
"3333333333333333333333333333333333333333"} {
check(7, trunk)
check(8, trunk)
}
q, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil {
t.Fatal(err)
}
seen := map[int]string{}
for _, a := range q.Asks {
c := a.Request.Check
if c == nil {
t.Fatalf("%s is no check", a.ID)
}
if _, twice := seen[c.Number]; twice {
t.Errorf("pull request %d has two queued checks", c.Number)
}
seen[c.Number] = c.Trunk
if a.State != link.AskWaiting {
t.Errorf("%s is %s", a.ID, a.State)
}
if want := "3333333333333333333333333333333333333333"; c.Trunk != want {
t.Errorf("pull request %d is queued against %.8s; want the newest trunk %.8s", c.Number, c.Trunk, want)
}
}
if len(seen) != 2 {
t.Fatalf("queued checks for %v; want one for each of the two open pull requests", seen)
}
// By the streams' own count: the ready one on its stream, the other on its.
for stream, want := range map[string]uint64{broker.ChecksReadyStream: 1, broker.ChecksOtherStream: 1} {
info, err := js.Context().StreamInfo(stream)
if err != nil {
t.Fatal(err)
}
if info.State.Msgs != want {
t.Errorf("%s holds %d request(s); want %d", stream, info.State.Msgs, want)
}
}
}
+81
View File
@@ -210,3 +210,84 @@ func TestABuildOffTheTrunkIsNeverPublished(t *testing.T) {
t.Errorf("a build seat that said nothing was refused: %v", err)
}
}
// **A verdict names the trunk commit it judged** (novox/hq ADR 0323 §3, issue 341): the controller's `checked`
// carries the head and the trunk commit the build seat merged it into, the commits of the repositories read beside
// it, how long it took, and the kept delivery plan records the trunk commit with its other inputs. A group's
// composed check names, per member, the trunk commit that head was merged into (§6).
func TestAVerdictNamesTheTrunkCommitItJudged(t *testing.T) {
plan := &link.ChangePlan{Repository: "novox/mesh-host", Base: "main", Head: "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
Summary: "builds mesh-host"}
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 66, Base: "main", Plan: plan}
c := checkedOf(link.BuildResult{ID: "b", Ref: "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", Checked: asked,
Check: &link.CheckOutcome{Verdict: "pass", Took: "4m12s", Head: "dfb1415aaaaa", Trunk: "d28ca95bbbbb",
Gate: &link.CheckLayer{Verdict: "pass"}, RepoCheck: &link.CheckLayer{Verdict: "pass"},
Beside: map[string]string{"mesh-catalog": "0123456789ab", "mesh-lab": "ba9876543210"}}})
if c.Trunk != "d28ca95bbbbb" || c.Commit != "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" {
t.Fatalf("the verdict names head %s and trunk %q", c.Commit, c.Trunk)
}
if c.Beside["mesh-catalog"] != "0123456789ab" || c.Beside["mesh-lab"] != "ba9876543210" {
t.Fatalf("the verdict names the repositories read beside it as %v", c.Beside)
}
if c.Took != "4m12s" {
t.Fatalf("the verdict says it took %q", c.Took)
}
if c.Plan == nil || c.Plan.Trunk != "d28ca95bbbbb" || c.Plan.Head != plan.Head {
t.Fatalf("the kept plan records %+v", c.Plan)
}
if plan.Trunk != "" {
t.Fatal("the plan the controller asked with was changed: the verdict's is a copy")
}
group := &link.CheckRequest{Owner: "novox", Repo: "mesh-controller", Number: 7, Base: "main", Group: "feat/x",
Members: []link.GroupMember{{Owner: "novox", Repo: "mesh-host", Number: 8, Ref: "cccccccccccc0000", Base: "main"}}}
c = checkedOf(link.BuildResult{ID: "g", Ref: "aaaaaaaaaaaa0000", Checked: group, Check: &link.CheckOutcome{
Verdict: "pass", Head: "aaaaaaaaaaaa", Trunk: "111111111111", Gate: &link.CheckLayer{Verdict: "pass"},
Members: []link.CheckedMember{{Owner: "novox", Repo: "mesh-host", Commit: "cccccccccccc", Trunk: "222222222222"}}}})
if len(c.Members) != 2 || c.Members[0].Trunk != "111111111111" || c.Members[1].Trunk != "222222222222" ||
c.Members[1].Commit != "cccccccccccc0000" {
t.Fatalf("the composed check names its members %+v", c.Members)
}
}
// **The merge made for a check is never registered** (novox/hq ADR 0323 §2): a build of it — a commit on no branch
// of the forge — is refused, naming the trunk. The rule of ADR 0238 §1 stands unchanged; this holds it for the
// merge commit a check makes.
func TestABuildOfACheckMergeIsRefusedNamingTheTrunk(t *testing.T) {
merge := link.BuildResult{ID: "b", Commit: "4b5c6d7e8f90aabbccddeeff0011223344556677", Trunk: "main"}
err := publishable(merge, "")
if !errors.Is(err, errOffTheTrunk) || !strings.Contains(err.Error(), "not on main") {
t.Fatalf("a build of a check's merge was answered %v", err)
}
}
// **A verdict no build judges names its trunk commit too** (novox/hq ADR 0323 §3): a change touching no module of a
// repository with no merge-check.sh (novox/mesh-tools-go) is answered by the controller itself. Naming no trunk, the
// merge tool refused it, asked again, got the same verdict, and refused for ever.
func TestADirectVerdictNamesTheTrunkTheForgeAnnounced(t *testing.T) {
p := link.PullUpdated{Owner: "novox", Repo: "mesh-tools-go", Number: 3, Base: "main", Commit: "1a2b3c4d5e6f7a8b9c0d",
BaseSHA: "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b"}
c := directVerdict(p, link.ChangePlan{Repository: "novox/mesh-tools-go", Base: "main", Head: p.Commit})
if c.Trunk != "7a8b9c0d1e2f" || c.Plan == nil || c.Plan.Trunk != "7a8b9c0d1e2f" {
t.Fatalf("the direct verdict names trunk %q, its plan %+v", c.Trunk, c.Plan)
}
if c.Verdict != "pass" || c.Commit != p.Commit {
t.Fatalf("the direct verdict is %+v", c)
}
if c := directVerdict(link.PullUpdated{Owner: "novox", Repo: "x", Commit: "abc"}, link.ChangePlan{}); c.Trunk != "" {
t.Fatalf("a forge that said no base head gave the trunk %q", c.Trunk)
}
}
// A head checked again through mesh-delivery (`delivery check --members`, a recheck) reaches the same direct verdict:
// it names the base head the forge last announced, kept on the member, or the merge tool would refuse it for ever.
func TestARecheckedHeadsDirectVerdictNamesItsTrunk(t *testing.T) {
m := orderMember{ID: "novox/mesh-tools-go@1a2b3c4d5e6f", Repository: "novox/mesh-tools-go", Base: "main",
Head: "1a2b3c4d5e6f7a8b9c0d", Number: 3, BaseSHA: "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b"}
if c := directVerdict(m.pull(), link.ChangePlan{}); c.Trunk != "7a8b9c0d1e2f" {
t.Fatalf("a rechecked head's direct verdict names trunk %q", c.Trunk)
}
}
// A check stopped by its pull request's key moving on says no verdict (issue 974, review 2): an error said for it would name the
// live head, and the newer request is the one that judges. A check that failed for any other reason still says its error.
@@ -0,0 +1,68 @@
//go:build integration
package main
import (
"context"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A member of a group whose gate is the composed check's is asked of the build seat with no module: its repository's
// check alone (novox/hq ADR 0403 clause 1). With no word from mesh-delivery, or one that does not say the gate is
// composed, it is checked alone with its gate (clause 6).
func askedForMember(t *testing.T, answer []groupsLine, answerErr error) link.CheckRequest {
t.Helper()
open := aMesh(t)
ctx := t.Context()
js, _ := aBuildQueue(t)
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
was := groupsAsked
groupsAsked = func(context.Context, bool) ([]groupsLine, error) { return answer, answerErr }
t.Cleanup(func() { groupsAsked = was })
p := link.PullUpdated{Owner: "novox", Repo: "mesh-controller", Number: 346, Base: "main", Head: "feat/x", Commit: headController,
Paths: []string{"main.go"}, ModuleDirsSaid: true, MergeCheck: true, MergeCheckSaid: true}
if err := (following{open: open, oneShot: true}).PullUpdated(ctx, p); err != nil {
t.Fatal(err)
}
q, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil || len(q.Asks) != 1 {
t.Fatalf("asks %d: %v", len(q.Asks), err)
}
return *q.Asks[0].Request.Check
}
func TestAComposedMembersCheckIsItsRepositoryChecksAlone(t *testing.T) {
line := group("feat/x", member("novox/mesh-controller", 346, headController, "main", true), member("novox/mesh-catalog", 350, headHost, "main", true))
line.ComposedGate = true
c := askedForMember(t, []groupsLine{line}, nil)
if len(c.Modules) != 0 || len(c.New) != 0 || len(c.Dependents) != 0 || !c.Composed {
t.Fatalf("the member's check is asked as %+v; want no module and composed", c)
}
}
func TestAMemberOfAGroupOfAnOlderMeshDeliveryKeepsItsOwnGate(t *testing.T) {
line := group("feat/x", member("novox/mesh-controller", 346, headController, "main", true), member("novox/mesh-catalog", 350, headHost, "main", true))
c := askedForMember(t, []groupsLine{line}, nil)
if !strings.Contains(strings.Join(c.Modules, ","), "mesh-controller") || c.Composed {
t.Fatalf("an older mesh-delivery's member is asked as %+v; want its own gate", c)
}
}
func TestAMemberWhoseGroupCannotBeLearnedIsCheckedAloneWithItsGate(t *testing.T) {
c := askedForMember(t, nil, context.DeadlineExceeded)
if !strings.Contains(strings.Join(c.Modules, ","), "mesh-controller") || c.Composed ||
!strings.Contains(c.SiblingsUnknown, "reads no siblings") {
t.Fatalf("the member is asked as %+v; want its gate and why its siblings are unknown", c)
}
}
@@ -0,0 +1,136 @@
package main
import (
"context"
"encoding/json"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A delivery group's gate is the composed check's, and its members' own checks run their repository's check alone
// (novox/hq ADR 0403 clauses 1 and 7). Pure: no store, no bus.
// A member of a group whose line says composed_gate is a composed member; one whose group says nothing of it (an older
// mesh-delivery), one of a group of a single pull request and a lone pull request are not.
func TestAMemberOfAComposedGroupIsKnownByTheGroupsLine(t *testing.T) {
two := func(composed bool) groupsLine {
l := group("feat/x", member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))
l.ComposedGate = composed
return l
}
p := link.PullUpdated{Owner: "novox", Repo: "mesh-host", Number: 114, Base: "main", Head: "feat/x", Commit: headHost}
recheck := link.PullUpdated{Owner: "novox", Repo: "mesh-host", Number: 114, Base: "main", Commit: headHost}
for name, c := range map[string]struct {
p link.PullUpdated
lines []groupsLine
want bool
}{
"composed": {p, []groupsLine{two(true)}, true},
"a recheck, by its number": {recheck, []groupsLine{two(true)}, true},
"an older mesh-delivery": {p, []groupsLine{two(false)}, false},
"another group": {p, []groupsLine{group("feat/y", member("novox/mesh-host", 1, headHost, "main", true))}, false},
"no group": {p, nil, false},
} {
if got := composedMember(c.p, c.lines); got != c.want {
t.Errorf("%s: composed %v, want %v", name, got, c.want)
}
}
alone := group("feat/x", member("novox/mesh-host", 114, headHost, "main", true))
alone.ComposedGate = true
if composedMember(p, []groupsLine{alone}) {
t.Error("a group of a single pull request is composed")
}
}
// A composed member's verdict never says its gate passed: the gate layer, and the verdict that is the gate's, read
// "composed"; a failure or an error stays what it was.
func TestAComposedMembersVerdictHasNoPassingGate(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114, Composed: true}
pass := link.BuildResult{ID: "b", Ref: headHost, Checked: asked, Check: &link.CheckOutcome{Verdict: "pass", Summary: "no module touched",
Gate: &link.CheckLayer{Verdict: "pass", Summary: "no module touched"}, RepoCheck: &link.CheckLayer{Verdict: "pass", Summary: "ok"}}}
c := checkedOf(pass)
if c.Gate == nil || c.Gate.Verdict != "composed" || c.Verdict != "composed" || c.RepoCheck == nil || c.RepoCheck.Verdict != "pass" {
t.Fatalf("the member's verdict says %+v gate %+v", c, c.Gate)
}
asked.Composed = false
if c := checkedOf(pass); c.Gate.Verdict != "pass" || c.Verdict != "pass" {
t.Fatalf("a member of no composed group has its own gate: %+v", c.Gate)
}
asked.Composed = true
failing := pass
failing.Check = &link.CheckOutcome{Verdict: "fail", Summary: "conflict", Gate: &link.CheckLayer{Verdict: "fail", Summary: "conflict"},
RepoCheck: &link.CheckLayer{Verdict: "fail", Summary: "conflict"}}
if c := checkedOf(failing); c.Gate.Verdict != "fail" || c.Verdict != "fail" {
t.Fatalf("a failure was read as composed: %+v", c.Gate)
}
broken := link.BuildResult{ID: "b", Ref: headHost, Checked: asked, Failed: "the facts snapshot cannot be read"}
if c := checkedOf(broken); c.Verdict != "error" || c.Gate.Verdict != "error" {
t.Fatalf("an error was read as composed: %+v", c)
}
}
// The composed verdict on the bus carries every member, each with its trunk commit, for the forge's holder to set each
// member's mesh/merge-gate from it.
func TestTheComposedVerdictCarriesEveryMember(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114, Group: "feat/x",
Members: []link.GroupMember{{Owner: "novox", Repo: "mesh-controller", Number: 346, Ref: headController}}}
c := checkedOf(link.BuildResult{ID: "g", Ref: headHost, Checked: asked, Check: &link.CheckOutcome{Verdict: "pass", Summary: "ok",
Trunk: "aaaaaaaaaaaa", Members: []link.CheckedMember{{Owner: "novox", Repo: "mesh-controller", Commit: headController[:12], Trunk: "bbbbbbbbbbbb"}}}})
body, _ := json.Marshal(c)
for _, want := range []string{`"group":"feat/x"`, `"members":[{"owner":"novox","repo":"mesh-host","number":114,"commit":"` + headHost + `","trunk":"aaaaaaaaaaaa"}`,
`{"owner":"novox","repo":"mesh-controller","number":346,"commit":"` + headController + `","trunk":"bbbbbbbbbbbb"}`} {
if !strings.Contains(string(body), want) {
t.Fatalf("the composed verdict lacks %s:\n%s", want, body)
}
}
}
// Members whose manifests order their modules in a cycle fail the composed check, naming every module and edge, and no
// ask is made of the build seat.
func TestAGroupInACycleFailsItsComposedCheckWithoutABuild(t *testing.T) {
members := []orderMember{
{ID: "novox/x@" + headHost[:12], Repository: "novox/x", Base: "main", Head: headHost, Number: 7, Paths: []string{"module.json"}},
{ID: "novox/y@" + headController[:12], Repository: "novox/y", Base: "main", Head: headController, Number: 9, Paths: []string{"module.json"}},
}
reach := map[string]orderReach{members[0].ID: {Moved: []string{"mx"}}, members[1].ID: {Moved: []string{"my"}}}
was, wasParent := manifestAt, parentOf
t.Cleanup(func() { manifestAt, parentOf = was, wasParent })
parentOf = func(context.Context, string, string, string) (string, error) { return "", nil }
manifestAt = func(_ context.Context, _, repo, _, ref string) (catalogue.Manifest, bool, error) {
if ref == "main" {
return catalogue.Manifest{}, false, nil // what the pull requests merge into holds no such manifest
}
if repo == "x" {
return catalogue.Manifest{Module: "mx", DefinesSeats: []catalogue.SeatDeclaration{{Name: "s", Serves: []catalogue.Verb{{Name: "a"}}}},
Invokes: []string{"seat:s.b"}}, true, nil
}
return catalogue.Manifest{Module: "my", DefinesSeats: []catalogue.SeatDeclaration{{Name: "s", Serves: []catalogue.Verb{{Name: "b"}}}},
Invokes: []string{"seat:s.a"}}, true, nil
}
c, cyclic, err := cycleVerdict(context.Background(), "feat/cycle", members, reach, []inventory.Edge(nil), nil)
if err != nil || !cyclic {
t.Fatalf("no cycle found: %v", err)
}
if c.Verdict != "fail" || c.Group != "feat/cycle" || len(c.Members) != 2 || c.Gate == nil || c.Gate.Verdict != "fail" {
t.Fatalf("the verdict is %+v", c)
}
for _, w := range []string{"mx", "my", "s.a", "s.b"} {
if !strings.Contains(c.Summary, w) {
t.Errorf("the cycle does not name %s: %s", w, c.Summary)
}
}
// Members with no edge between them are no cycle.
manifestsReadMu.Lock()
clear(manifestsRead)
manifestsReadMu.Unlock()
manifestAt = func(context.Context, string, string, string, string) (catalogue.Manifest, bool, error) {
return catalogue.Manifest{}, false, nil
}
if _, cyclic, err := cycleVerdict(context.Background(), "feat/cycle", members, reach, nil, nil); err != nil || cyclic {
t.Fatalf("a cycle where there is none: %v %v", cyclic, err)
}
}
+76
View File
@@ -0,0 +1,76 @@
package main
import (
"context"
"encoding/json"
"fmt"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// The looks of the open conditions — how often each was observed, when last, its newest evidence — are
// kept in the controller's store, not the conditions bucket (novox/hq issue 420, ADR 0289 §1): the bucket
// is written when what a condition says changes, never for a look that saw it as it was.
// storeLooks keeps the conditions' looks in the inventory.
type storeLooks struct{ inv *inventory.Inventory }
func (s storeLooks) KeepLooks(ctx context.Context, key string, l conditions.Looks) error {
evidence, err := json.Marshal(l.Evidence)
if err != nil {
return err
}
return s.inv.KeepConditionLooks(ctx, inventory.ConditionLooks{Key: key, Observations: l.Observations,
LastObserved: l.LastObserved, Evidence: evidence})
}
func (s storeLooks) Looked(ctx context.Context, key string) (conditions.Looks, bool, error) {
row, kept, err := s.inv.ConditionLooksOf(ctx, key)
if err != nil || !kept {
return conditions.Looks{}, kept, err
}
l, err := looksOf(row)
return l, err == nil, err
}
func (s storeLooks) AllLooks(ctx context.Context) (map[string]conditions.Looks, error) {
rows, err := s.inv.AllConditionLooks(ctx)
if err != nil {
return nil, err
}
out := make(map[string]conditions.Looks, len(rows))
for key, row := range rows {
l, err := looksOf(row)
if err != nil {
return nil, err
}
out[key] = l
}
return out, nil
}
func (s storeLooks) ForgetLooks(ctx context.Context, key string) error {
return s.inv.ForgetConditionLooks(ctx, key)
}
func looksOf(row inventory.ConditionLooks) (conditions.Looks, error) {
l := conditions.Looks{Observations: row.Observations, LastObserved: row.LastObserved.UTC()}
if len(row.Evidence) > 0 {
if err := json.Unmarshal(row.Evidence, &l.Evidence); err != nil {
return conditions.Looks{}, fmt.Errorf("the evidence kept for the condition %s cannot be read: %w", row.Key, err)
}
}
return l, nil
}
// looksHere is where a process that is not the serving controller reads the conditions' looks: the store,
// opened for it, closed by the function returned. A store that cannot be opened is said on every condition
// read through it, never read as none.
func looksHere(ctx context.Context) (conditions.LookStore, func()) {
inv, err := openInventory(ctx)
if err != nil {
return conditions.NoLooks{Err: err}, func() {}
}
return storeLooks{inv}, inv.Close
}
@@ -0,0 +1,89 @@
//go:build integration
package main
import (
"bytes"
"context"
"encoding/json"
"testing"
"time"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/testbus"
)
// **Looks at an unchanged condition write the bucket once, and `conditions` still says how often and when
// it was seen** (novox/hq issue 420): the looks are in the controller's store, and the verb, run as a
// process of its own, reads them from there. On 2026-10-10 one open condition had rewritten the bucket 7 003
// times in two days.
func TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds(t *testing.T) {
inv := inventory.ForTest(t)
bus := testbus.Start(t)
serving, err := broker.Dial(bus.ClientURL())
if err != nil {
t.Fatal(err)
}
defer serving.Close()
if err := serving.EnsureControllerBuckets(); err != nil {
t.Fatal(err)
}
ctx := t.Context()
store, history, err := conditions.OnTheBus(ctx, serving.Conn())
if err != nil {
t.Fatal(err)
}
k := conditions.NewKeeper(ctx, conditions.Options{Store: store, History: history, Looks: storeLooks{inv}})
defer k.Close(context.Background())
look := conditions.Observation{Scope: conditions.ScopeMachine, ID: "anchor", Kind: "silent", Machine: "anchor",
Severity: conditions.Warning, Summary: "anchor has not been heard from", Source: "S1"}
for range 4 {
if _, err := k.Observe(ctx, look); err != nil {
t.Fatal(err)
}
}
api, err := jetstream.New(serving.Conn())
if err != nil {
t.Fatal(err)
}
stream, err := api.Stream(ctx, "KV_"+broker.ConditionsBucket)
if err != nil {
t.Fatal(err)
}
info, err := stream.Info(ctx)
if err != nil {
t.Fatal(err)
}
if info.State.LastSeq != 1 {
t.Fatalf("four looks at one unchanged condition wrote the bucket %d times, want 1", info.State.LastSeq)
}
// The verb as a process other than the serving controller: it reads the bucket and the store.
keptBefore := conditionsFrom
conditionsFrom = nil
defer func() { conditionsFrom = keptBefore }()
t.Setenv(broker.NATSVar, bus.ClientURL())
t.Setenv(broker.CertificateVar, "")
var shown bytes.Buffer
if err := showCondition(ctx, []string{"machine.anchor.silent", "--json"}, &shown); err != nil {
t.Fatal(err)
}
var c conditions.Condition
if err := json.Unmarshal(shown.Bytes(), &c); err != nil {
t.Fatalf("%v: %s", err, shown.String())
}
if c.Observations != 4 || len(c.Evidence) != 4 || c.LastObserved.IsZero() || c.LooksUnread != "" ||
time.Since(c.LastObserved) > time.Minute {
t.Fatalf("conditions show reads %d observation(s), %d evidence line(s), last %s (%q)", c.Observations,
len(c.Evidence), c.LastObserved, c.LooksUnread)
}
listed, err := openConditions(ctx)
if err != nil || len(listed) != 1 || listed[0].Observations != 4 {
t.Fatalf("conditions lists %+v: %v", listed, err)
}
}
+17 -9
View File
@@ -24,16 +24,18 @@ import (
// **`status` used to be the only place the mesh said it was wrong, and only to whoever asked.** Every
// core failure of research 031 was found by a person looking. A condition is the mesh saying it: raised
// by a watchdog when a signal is late (signals.go), by a probe when an invariant does not hold
// (doctor.go), or by an event a provider sends (standing.go); kept on the bus with since-when and
// evidence; said on the bus as it changes, for the operator's channel to carry; and cleared when an
// (doctor.go), or by an event a provider sends (standing.go); kept on the bus with since-when, and its
// looks — how often, when last, the newest evidence — in the controller's store (condition_looks.go,
// novox/hq issue 420); said on the bus as it changes, for the operator's channel to carry; and cleared when an
// observation says it is resolved. Nobody resolves one by hand. A person who knows silences it, for a
// while, with a reason, and that is recorded as a hand act.
// conditionsFrom is the serving controller's keeper; nil in any other process, which opens its own.
var conditionsFrom *conditions.Keeper
// keeperOn is a keeper over the store on a connection, saying its transitions on that connection.
func keeperOn(ctx context.Context, conn *nats.Conn) (*conditions.Keeper, error) {
// keeperOn is a keeper over the store on a connection, saying its transitions on that connection, with
// the conditions' looks kept in looks (condition_looks.go).
func keeperOn(ctx context.Context, conn *nats.Conn, looks conditions.LookStore) (*conditions.Keeper, error) {
store, history, err := conditions.OnTheBus(ctx, conn)
if err != nil {
return nil, err
@@ -42,12 +44,12 @@ func keeperOn(ctx context.Context, conn *nats.Conn) (*conditions.Keeper, error)
if err != nil {
return nil, err
}
return conditions.NewKeeper(ctx, conditions.Options{Store: store, History: history,
return conditions.NewKeeper(ctx, conditions.Options{Store: store, History: history, Looks: looks,
Teller: link.OverNATS{Conn: conn, JS: js},
Say: func(format string, args ...any) { fmt.Fprintf(os.Stderr, format+"\n", args...) },
// What status leads with changed: composed again soon (a nudge outside the serving controller
// does nothing).
Changed: statusFrom.nudge,
Changed: func() { statusFrom.nudge(); askerFrom.nudge() },
// Written under the lease, carrying its epoch (novox/hq to-be 45 §6).
Epoch: func() (uint64, error) { return theLease.epoch(context.WithoutCancel(ctx)) }}), nil
}
@@ -58,8 +60,10 @@ func withKeeper(ctx context.Context, f func(*conditions.Keeper) error) error {
if conditionsFrom != nil {
return f(conditionsFrom)
}
looks, closeLooks := looksHere(ctx)
defer closeLooks()
return onTheBus(func(conn *nats.Conn) error {
k, err := keeperOn(ctx, conn)
k, err := keeperOn(ctx, conn, looks)
if err != nil {
return err
}
@@ -82,6 +86,8 @@ func openConditions(ctx context.Context) ([]conditions.Condition, error) {
return nil, fmt.Errorf("this process has no bus to read the conditions from: %w", err)
}
var out []conditions.Condition
looks, closeLooks := looksHere(ctx)
defer closeLooks()
err := onTheBus(func(conn *nats.Conn) error {
store, _, err := conditions.OnTheBus(ctx, conn)
if err != nil {
@@ -89,7 +95,7 @@ func openConditions(ctx context.Context) ([]conditions.Condition, error) {
}
reading, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
out, err = conditions.Read(reading, store)
out, err = conditions.ReadWith(reading, store, looks)
return err
})
return out, err
@@ -252,12 +258,14 @@ func showCondition(ctx context.Context, args []string, w io.Writer) error {
if conditionsFrom != nil {
c, found, err = conditionsFrom.Get(ctx, key)
} else {
looks, closeLooks := looksHere(ctx)
defer closeLooks()
err = onTheBus(func(conn *nats.Conn) error {
store, _, err := conditions.OnTheBus(ctx, conn)
if err != nil {
return err
}
c, found, err = conditions.ReadOne(ctx, store, key)
c, found, err = conditions.ReadOneWith(ctx, store, looks, key)
return err
})
}
@@ -0,0 +1,43 @@
//go:build integration
package main
import (
"fmt"
"strings"
"testing"
"time"
)
func TestAVerdictCarriesAProbesFindingsInBrief(t *testing.T) {
var found []string
for i := 0; i < 689; i++ {
found = append(found, fmt.Sprintf("delivery d-%012x has been checking for 13h, past its bound of 45m", i))
}
run := doctorRun{Run: "run-1", Probes: []probeVerdict{{ID: "D14", Verdict: "failed", Found: found},
{ID: "D1", Verdict: "passed"}}}
answer, err := verdictAnswer(run, time.Now(), "")
if err != nil {
t.Fatal(err)
}
brief := answer["run"].(doctorRun).Probes[0]
if len(brief.Found) != overviewFindings || brief.FoundCount != 689 || !strings.Contains(brief.More, "probe=D14") {
t.Fatalf("the verdict carries %d findings of D14, count %d, more %q", len(brief.Found), brief.FoundCount, brief.More)
}
if len(run.Probes[0].Found) != 689 {
t.Fatal("the overview cut the run it was made from")
}
whole, err := verdictAnswer(run, time.Now(), "d14")
if err != nil {
t.Fatal(err)
}
if p := whole["run"].(doctorRun).Probes; len(p) != 1 || len(p[0].Found) != 689 || p[0].More != "" {
t.Fatalf("doctor probe=D14 answered %+v", p)
}
if _, err := verdictAnswer(run, time.Now(), "D99"); err == nil {
t.Fatal("a probe the run does not have was answered")
}
if !strings.Contains(doctorText(answer), "probe=D14` gives them all") {
t.Fatalf("the text does not say how to read them all:\n%s", doctorText(answer))
}
}
@@ -74,35 +74,3 @@ func TestAListOfConditionsCarriesEachInBrief(t *testing.T) {
// The self-check's verdict carries at most overviewFindings of one probe's findings, with their count and
// how to read them all; `doctor probe=` gives that probe whole (novox/hq issue 314).
func TestAVerdictCarriesAProbesFindingsInBrief(t *testing.T) {
var found []string
for i := 0; i < 689; i++ {
found = append(found, fmt.Sprintf("delivery d-%012x has been checking for 13h, past its bound of 45m", i))
}
run := doctorRun{Run: "run-1", Probes: []probeVerdict{{ID: "D14", Verdict: "failed", Found: found},
{ID: "D1", Verdict: "passed"}}}
answer, err := verdictAnswer(run, time.Now(), "")
if err != nil {
t.Fatal(err)
}
brief := answer["run"].(doctorRun).Probes[0]
if len(brief.Found) != overviewFindings || brief.FoundCount != 689 || !strings.Contains(brief.More, "probe=D14") {
t.Fatalf("the verdict carries %d findings of D14, count %d, more %q", len(brief.Found), brief.FoundCount, brief.More)
}
if len(run.Probes[0].Found) != 689 {
t.Fatal("the overview cut the run it was made from")
}
whole, err := verdictAnswer(run, time.Now(), "d14")
if err != nil {
t.Fatal(err)
}
if p := whole["run"].(doctorRun).Probes; len(p) != 1 || len(p[0].Found) != 689 || p[0].More != "" {
t.Fatalf("doctor probe=D14 answered %+v", p)
}
if _, err := verdictAnswer(run, time.Now(), "D99"); err == nil {
t.Fatal("a probe the run does not have was answered")
}
if !strings.Contains(doctorText(answer), "probe=D14` gives them all") {
t.Fatalf("the text does not say how to read them all:\n%s", doctorText(answer))
}
}
@@ -0,0 +1,32 @@
//go:build integration
package main
import (
"errors"
"strings"
"testing"
)
func TestUnreadableConditionsAreNotAWellMesh(t *testing.T) {
open := aMesh(t)
_, store := withConditionsInMemory(t)
store.SetFail(errors.New("the bus is away"))
asked, err := theThreeQuestions(t.Context(), open)
if err != nil {
t.Fatal(err)
}
if asked.well() || asked.conditionsUnread == "" {
t.Fatalf("well with its conditions unread: %+v", asked.conditionsUnread)
}
said := printed(t, func() error { return printStatus(asked) })
if !strings.HasPrefix(said, "the open conditions could NOT be read") || strings.Contains(said, "no open conditions") {
t.Fatalf("%s", said)
}
store.SetFail(nil)
asked, _ = theThreeQuestions(t.Context(), open)
said = printed(t, func() error { return printStatus(asked) })
if asked.well() && !strings.Contains(said, "no open conditions;") {
t.Fatalf("the all-well sentence does not say no conditions are open:\n%s", said)
}
}
-23
View File
@@ -1,7 +1,6 @@
package main
import (
"errors"
"strings"
"testing"
@@ -83,25 +82,3 @@ func TestASilenceThroughTheVerbHoldsAndTheConditionStaysOpen(t *testing.T) {
}
// **Conditions that cannot be read are not none open**: status says so, and is not well.
func TestUnreadableConditionsAreNotAWellMesh(t *testing.T) {
open := aMesh(t)
_, store := withConditionsInMemory(t)
store.SetFail(errors.New("the bus is away"))
asked, err := theThreeQuestions(t.Context(), open)
if err != nil {
t.Fatal(err)
}
if asked.well() || asked.conditionsUnread == "" {
t.Fatalf("well with its conditions unread: %+v", asked.conditionsUnread)
}
said := printed(t, func() error { return printStatus(asked) })
if !strings.HasPrefix(said, "the open conditions could NOT be read") || strings.Contains(said, "no open conditions") {
t.Fatalf("%s", said)
}
store.SetFail(nil)
asked, _ = theThreeQuestions(t.Context(), open)
said = printed(t, func() error { return printStatus(asked) })
if asked.well() && !strings.Contains(said, "no open conditions;") {
t.Fatalf("the all-well sentence does not say no conditions are open:\n%s", said)
}
}
+616
View File
@@ -0,0 +1,616 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"reflect"
"sort"
"strings"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **The move of config** (novox/hq ADR 0348 §9 and its consequences, design 57 K5, issue 664): every config value a
// settings layer holds is set in mesh-config, one of the moves of data a release runs once after its delivery, after
// the move of placement. A module's layer on a node becomes its own value on that node (`module.<node>.<module>.<key>`).
// A whole-mesh layer has no module value for the whole mesh (ADR 0348 §3): its value becomes mesh config for the
// predefined key it falls back to where that module alone falls back to the key (claude-code's managed settings
// become `agent-managed-settings`), and otherwise the module's own value on each node it is assigned to, laid under
// that node's layer as composition lays it. A key a layer still says by its file's name (nodered's `timeZone`) is
// set under its config key (`time-zone`, issue 751).
//
// **Every node composes byte for byte as before.** The move sets the values, reads every module's resolved values
// back through mesh-config and compares them with what the layers lay today; only when they agree does it empty the
// layers it moved (each kept whole in its history first) and turn on the switch (inventory.FinishConfigMove), after
// which the controller reads mesh-config where it read the layers. A module still in the settings form keeps its
// layers and is said so. Anything that cannot move as it composes refuses the whole move, by name, before anything
// is set. Asked again, it sets nothing that is set already.
// configLevel is where mesh-config keeps a value: mesh config, node config, or a module's own value on a node.
type configLevel struct {
Level string `json:"level"`
Node string `json:"node,omitempty"`
Module string `json:"module,omitempty"`
}
func (l configLevel) valueKey(key string) string {
switch l.Level {
case "node":
return "node." + l.Node + "." + key
case "module":
return "module." + l.Node + "." + l.Module + "." + key
}
return "mesh." + key
}
func (l configLevel) String() string {
switch l.Level {
case "node":
return "node config on " + l.Node
case "module":
return l.Module + "'s own value on " + l.Node
}
return "mesh config"
}
// configSet is one value the move sets in mesh-config, and the layers it comes from.
type configSet struct {
configLevel
Key string `json:"key"`
Value any `json:"value"`
Trusted bool `json:"trusted,omitempty"`
// From is the layers it comes from, as `<module> on <where>: <key as the layer says it>`.
From []string `json:"from"`
// Already is true when mesh-config holds this value at this level already: nothing is set.
Already bool `json:"already,omitempty"`
}
// configMoveAnswer is what the move of config says it did, or would do.
type configMoveAnswer struct {
// DryRun is true when nothing was set: asked without --confirm.
DryRun bool `json:"dry-run"`
Sets []configSet `json:"sets"`
// Moved is each layer whose values the move sets, emptied once they read back.
Moved []string `json:"moved"`
// Stays is each layer of a module still in the settings form: kept, and composed from as before.
Stays []string `json:"stays,omitempty"`
// Unread is each layer that composes nothing (its module is not assigned where it says): kept in the history,
// emptied, and set nowhere.
Unread []string `json:"unread,omitempty"`
// Problems is why the move refuses, each by name; nothing is set while there is one.
Problems []string `json:"problems,omitempty"`
// Switched is true once the controller reads mesh-config where it read the layers.
Switched bool `json:"switched"`
}
// configMovePlan is the move worked out from the layers, before anything is asked of mesh-config.
type configMovePlan struct {
answer configMoveAnswer
// moved is the layers the move empties once the values read back.
moved []inventory.LayerToMove
// expected is, for each module the move reaches, keyed `<node>.<module>`, every value composition lays over
// its files today: what mesh-config must resolve for it from mesh config, node config or its own value.
expected map[string]map[string]any
// modules is each module in the config form, by the nodes it is assigned to: whose resolved values are read back.
modules map[string][]string
}
func whereOf(node string) string {
if node == "" {
return "the whole mesh"
}
return node
}
// planConfigMove works out the move from every layer that holds a value, the catalogue, and where each module is
// assigned.
func planConfigMove(layers []inventory.LayerToMove, shelf map[string]catalogue.Manifest,
on map[string][]string) configMovePlan {
p := configMovePlan{answer: configMoveAnswer{Sets: []configSet{}, Moved: []string{}},
expected: map[string]map[string]any{}, modules: map[string][]string{}}
problem := func(format string, a ...any) {
p.answer.Problems = append(p.answer.Problems, fmt.Sprintf(format, a...))
}
for name, m := range shelf {
if m.DeclaresConfig() {
p.modules[name] = on[name]
}
}
byModule := map[string]*layersOfModule{}
var names []string
for _, l := range layers {
said := fmt.Sprintf("%s on %s", l.Module, whereOf(l.Node))
m, known := shelf[l.Module]
if !known || !m.DeclaresConfig() {
p.answer.Stays = append(p.answer.Stays, said+" (its module is in the settings form)")
continue
}
if _, placement := catalogue.SplitPlacement(l.Values); len(placement) > 0 {
problem("%s still holds placement (%s): run move-placement first", said, strings.Join(movedKeys(placement), ", "))
continue
}
assigned := on[l.Module]
if (l.Node == "" && len(assigned) == 0) || (l.Node != "" && !containsString(assigned, l.Node)) {
p.answer.Unread = append(p.answer.Unread, said+" (the module is not assigned there, so it composes nothing)")
p.moved = append(p.moved, l)
continue
}
// The layer's keys as config keys: a key the file names otherwise is set under its config key.
as := catalogue.TakenAs(m)
values := map[string]any{}
from := map[string]string{}
for _, k := range movedKeys(l.Values) {
key := k
if _, declared := m.Settings[k]; !declared {
if renamed, ok := as[k]; ok {
key = renamed
} else {
problem("%s says %q, which %s does not declare: declare it, or clear it from the layer", said, k, l.Module)
continue
}
}
if _, both := values[key]; both {
// The layer says the key and the file's name for it: the config key wins, as composition has it (issue 751).
if key != k {
continue
}
}
values[key] = l.Values[k]
from[key] = said + ": " + k
}
om := byModule[l.Module]
if om == nil {
om = &layersOfModule{nodes: map[string]map[string]any{}, saidAt: map[string]map[string]string{}}
byModule[l.Module] = om
names = append(names, l.Module)
}
om.saidAt[l.Node] = from
if l.Node == "" {
om.mesh = values
} else {
om.nodes[l.Node] = values
}
p.moved = append(p.moved, l)
p.answer.Moved = append(p.answer.Moved, said)
}
sort.Strings(names)
// Which predefined key each module falls back to, so a whole-mesh value goes to mesh config only where no other
// module's value moves with it.
fallingBack := map[string][]string{}
for name, m := range shelf {
for _, d := range m.Settings {
if d.Predefined != "" && !containsString(fallingBack[d.Predefined], name) {
fallingBack[d.Predefined] = append(fallingBack[d.Predefined], name)
}
}
}
for _, name := range names {
m, om := shelf[name], byModule[name]
asked := catalogue.AskedKeys(m)
keys := map[string]bool{}
for k := range om.mesh {
keys[k] = true
}
for _, values := range om.nodes {
for k := range values {
keys[k] = true
}
}
for _, key := range movedKeys(keys) {
d := m.Settings[key]
meshValue, meshSays := om.mesh[key]
alone := d.Predefined != "" && len(fallingBack[d.Predefined]) == 1
if meshSays && alone {
p.set(configLevel{Level: "mesh"}, d.Predefined, meshValue, d, om.said("", key))
}
for _, node := range on[name] {
nodeValue, nodeSays := om.nodes[node][key]
value := meshValue
switch {
case nodeSays && meshSays:
_, under := meshValue.(map[string]any)
_, over := nodeValue.(map[string]any)
if under && over && asked[key] {
problem("%s on %s lays an object for %q over the whole mesh's, and a placeholder takes it whole "+
"where its file merges the two: say the value on %s whole, then move", name, node, key, node)
continue
}
value = catalogue.LaidOver(meshValue, nodeValue)
case nodeSays:
value = nodeValue
case !meshSays:
continue
}
p.expect(node, name, key, value)
if nodeSays || !alone {
saidBy := om.said(node, key)
if meshSays {
saidBy = append(om.said("", key), saidBy...)
}
p.set(configLevel{Level: "module", Node: node, Module: name}, key, value, d, saidBy)
}
}
}
}
sort.Strings(p.answer.Problems)
return p
}
// layersOfModule is one module's layers as the move reads them: its whole-mesh layer's config values, each node's,
// and where each key was said, by node ("" for the whole mesh).
type layersOfModule struct {
mesh map[string]any
nodes map[string]map[string]any
saidAt map[string]map[string]string
}
func (om *layersOfModule) said(node, key string) []string {
if s, ok := om.saidAt[node][key]; ok {
return []string{s}
}
return nil
}
// set is one value the move sets, judged against its declaration as mesh-config will judge it.
func (p *configMovePlan) set(level configLevel, key string, value any, d catalogue.SettingDeclaration, from []string) {
p.answer.Sets = append(p.answer.Sets, configSet{configLevel: level, Key: key, Value: value, Trusted: d.Trusted,
From: from})
for _, wrong := range catalogue.ValueProblems(d, value) {
p.answer.Problems = append(p.answer.Problems, fmt.Sprintf("%s at %s: %s", key, level, wrong))
}
}
func (p *configMovePlan) expect(node, module, key string, value any) {
k := node + "." + module
if p.expected[k] == nil {
p.expected[k] = map[string]any{}
}
p.expected[k][key] = value
}
func movedKeys[V any](m map[string]V) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
// configService is mesh-config as the move asks it: what it holds at each level, what it resolved, and a value set
// as the controller carries it from its terminal.
type configService interface {
// Values is every value set, by its key in mesh-config_values (`mesh.<key>`, `node.<node>.<key>`,
// `module.<node>.<module>.<key>`).
Values(ctx context.Context) (map[string]json.RawMessage, error)
// Resolved is every resolved value.
Resolved(ctx context.Context) ([]resolvedEntry, error)
// Set sets one value at its level, by whom and why.
Set(ctx context.Context, level configLevel, key string, value any, by, why string) error
}
// runConfigMove is the move against a store and mesh-config: a dry run says what it would set.
func runConfigMove(ctx context.Context, inv *inventory.Inventory, config configService, apply bool, by,
why string) (configMoveAnswer, error) {
layers, holding, err := inv.LayersToMove(ctx)
if err != nil {
return configMoveAnswer{}, err
}
entries, err := inv.Catalogued(ctx)
if err != nil {
return configMoveAnswer{}, err
}
shelf := map[string]catalogue.Manifest{}
on := map[string][]string{}
for _, e := range entries {
shelf[e.Manifest.Module] = e.Manifest
on[e.Manifest.Module] = e.On
}
p := planConfigMove(layers, shelf, on)
if len(holding) > 0 && len(p.answer.Problems) == 0 {
p.answer.Problems = append(p.answer.Problems, "a layer still holds placement ("+strings.Join(holding, "; ")+
"): run move-placement first")
}
moved, err := inv.ConfigMoved(ctx)
if err != nil {
return configMoveAnswer{}, err
}
p.answer.Switched = moved.Moved
if config == nil {
return p.answer, errors.New("mesh-config cannot be asked, so nothing was moved")
}
held, err := config.Values(ctx)
if err != nil {
return p.answer, fmt.Errorf("what mesh-config holds cannot be read, so nothing was moved: %w", err)
}
for i := range p.answer.Sets {
s := &p.answer.Sets[i]
if was, ok := held[s.valueKey(s.Key)]; ok && sameJSON(was, s.Value) {
s.Already = true
}
}
p.answer.DryRun = !apply
if !apply || len(p.answer.Problems) > 0 {
if apply {
return p.answer, fmt.Errorf("the move of config refuses, and nothing was set: %s",
strings.Join(p.answer.Problems, "; "))
}
return p.answer, nil
}
if len(p.moved) == 0 && moved.Moved {
return p.answer, nil
}
for _, s := range p.answer.Sets {
if s.Already {
continue
}
said := "moved from the settings layer of " + strings.Join(s.From, " and ") + " (issue 664): " + why
if err := config.Set(ctx, s.configLevel, s.Key, s.Value, by, said); err != nil {
return p.answer, fmt.Errorf("%s at %s was not set, and no layer was emptied; ask the move again: %w",
s.Key, s.configLevel, err)
}
}
// Read back through mesh-config before anything points at it (design 57): every module the move reaches, and
// every other in the config form, must resolve exactly what its layers lay today.
if differs, err := readsBack(ctx, config, p); err != nil {
return p.answer, err
} else if len(differs) > 0 {
return p.answer, fmt.Errorf("mesh-config does not resolve what the layers lay today, so no layer was emptied and "+
"the controller still composes from the layers: %s", strings.Join(differs, "; "))
}
if err := inv.FinishConfigMove(ctx, p.moved, by, why); err != nil {
return p.answer, err
}
p.answer.Switched = true
return p.answer, nil
}
// readsBack compares what mesh-config resolved with what the layers lay, for every module in the config form on
// every node it is assigned to, and says each difference. A value the layers do not lay and mesh-config resolves from
// a level would be laid once the switch is on: a difference too.
func readsBack(ctx context.Context, config configService, p configMovePlan) ([]string, error) {
var resolved []resolvedEntry
var err error
// mesh-config writes the resolved values with each set; a pass that missed one is put right by its next
// resolution, so a difference is read again for a little while before it is said.
for wait := 0; ; wait++ {
if resolved, err = config.Resolved(ctx); err != nil {
return nil, fmt.Errorf("mesh-config's resolved values cannot be read back, so no layer was emptied: %w", err)
}
differs, err := differences(resolved, p)
if err != nil || len(differs) == 0 || wait >= readBackTries {
return differs, err
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(readBackEvery):
}
}
}
// How often, and how many times, the move reads mesh-config's resolved values back before it says they differ.
var (
readBackEvery = 2 * time.Second
readBackTries = 15
)
func differences(resolved []resolvedEntry, p configMovePlan) ([]string, error) {
var differs []string
for _, module := range movedKeys(p.modules) {
for _, node := range p.modules[module] {
laid, err := configLaid(resolved, node, module)
if err != nil {
return nil, err
}
want := p.expected[node+"."+module]
if want == nil {
want = map[string]any{}
}
for _, k := range movedKeys(mergedKeys(laid, want)) {
got, has := laid[k]
w, wants := want[k]
switch {
case !wants:
differs = append(differs, fmt.Sprintf("%s on %s resolves %s from a level no layer says; clear it "+
"there, or set it in a layer first", module, node, k))
case !has:
differs = append(differs, fmt.Sprintf("%s on %s does not resolve %s from the value moved", module, node, k))
case !reflect.DeepEqual(normalJSON(got), normalJSON(w)):
differs = append(differs, fmt.Sprintf("%s on %s resolves %s as %s, and its layers lay %s", module,
node, k, compactJSON(got), compactJSON(w)))
}
}
}
}
return differs, nil
}
func mergedKeys(a, b map[string]any) map[string]bool {
out := map[string]bool{}
for k := range a {
out[k] = true
}
for k := range b {
out[k] = true
}
return out
}
func normalJSON(v any) any {
raw, _ := json.Marshal(v)
var out any
_ = json.Unmarshal(raw, &out)
return out
}
func compactJSON(v any) string {
raw, _ := json.Marshal(v)
return string(raw)
}
func sameJSON(raw json.RawMessage, v any) bool {
var held any
if err := json.Unmarshal(raw, &held); err != nil {
return false
}
return reflect.DeepEqual(held, normalJSON(v))
}
// busConfig is mesh-config on this controller's bus: its buckets read directly, a value set through its verb in
// the controller's own name, so mesh-config takes a trusted value as given at the controller's terminal (ADR 0348 §8).
type busConfig struct{}
func (busConfig) Values(ctx context.Context) (map[string]json.RawMessage, error) {
out := map[string]json.RawMessage{}
err := onTheBus(func(conn *nats.Conn) error {
js, err := jetstream.New(conn)
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
kv, err := js.KeyValue(ctx, broker.BucketName("mesh-config", "values"))
if err != nil {
return err
}
w, err := kv.WatchAll(ctx, jetstream.IgnoreDeletes())
if err != nil {
return err
}
defer func() { _ = w.Stop() }()
for e := range w.Updates() {
if e == nil {
return nil
}
var v struct {
Value json.RawMessage `json:"value"`
}
if err := json.Unmarshal(e.Value(), &v); err != nil {
return fmt.Errorf("mesh-config holds %q, which is not a value: %w", e.Key(), err)
}
out[e.Key()] = v.Value
}
return errors.New("mesh-config's values stopped answering")
})
return out, err
}
func (busConfig) Resolved(ctx context.Context) ([]resolvedEntry, error) { return readResolved(ctx) }
func (busConfig) Set(ctx context.Context, level configLevel, key string, value any, by, why string) error {
args := map[string]any{"action": "set", "key": key, "value": value, "why": why, "by": by}
if level.Node != "" {
args["node"] = level.Node
}
if level.Module != "" {
args["module"] = level.Module
}
return onTheBus(func(conn *nats.Conn) error {
answer, err := link.AskMeshSeatToolAs(ctx, conn, broker.ControllerName, "mesh-config", "config", args, 30*time.Second)
if err != nil {
return err
}
if answer.Error != "" {
return errors.New(answer.Error)
}
return nil
})
}
// moveConfigCommand is `move-config`: the move of config (issue 664).
func moveConfigCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("move-config", flag.ContinueOnError)
asJSON := set.Bool("json", false, "answer as JSON")
confirm := set.Bool("confirm", false, "set the values and switch, rather than only say what would move")
f := addHandActFlags(set)
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 0 {
return errors.New("move-config [--json] [--confirm --why <text>]")
}
if !*confirm && f.given() {
return errors.New("move-config takes --why only with --confirm: without it nothing is moved. Nothing was done")
}
if *confirm {
if err := f.require("move-config"); err != nil {
return err
}
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
if *confirm {
f.record(ctx, "move-config", nil)
}
why := ""
if *confirm {
why = strings.TrimSpace(*f.why)
}
answer, err := runConfigMove(ctx, open.inventory, busConfig{}, *confirm, "the move of config, run by "+link.Caller(), why)
if *asJSON {
encoder := json.NewEncoder(os.Stdout)
encoder.SetIndent("", " ")
if encErr := encoder.Encode(answer); encErr != nil {
return encErr
}
return err
}
fmt.Print(describeConfigMove(answer))
return err
}
// describeConfigMove says the answer as a person reads it.
func describeConfigMove(a configMoveAnswer) string {
var b strings.Builder
verb := "set"
if a.DryRun {
verb = "would set"
}
for _, s := range a.Sets {
state := verb
if s.Already {
state = "holds already"
}
fmt.Fprintf(&b, "%s %s = %s at %s (from %s)\n", state, s.Key, compactJSON(s.Value), s.configLevel,
strings.Join(s.From, " and "))
}
for _, s := range a.Unread {
fmt.Fprintf(&b, "emptied, set nowhere: %s\n", s)
}
for _, s := range a.Stays {
fmt.Fprintf(&b, "stays: %s\n", s)
}
for _, s := range a.Problems {
fmt.Fprintf(&b, "refused: %s\n", s)
}
switch {
case len(a.Problems) > 0:
b.WriteString("nothing was set: the move refuses until each is put right\n")
case a.DryRun:
fmt.Fprintf(&b, "%d layer(s), %d value(s); nothing was set — with --confirm --why <text> they are\n",
len(a.Moved), len(a.Sets))
case a.Switched:
fmt.Fprintf(&b, "%d layer(s) moved into mesh-config, each kept whole in its history first; the controller "+
"reads mesh-config where it read them\n", len(a.Moved))
}
return b.String()
}
@@ -0,0 +1,467 @@
//go:build integration
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"reflect"
"sort"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/overlay"
)
func aConfiguredMesh(t *testing.T) (*stores, configuredMesh, map[string]map[string]any) {
t.Helper()
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
for i, name := range []string{"workstation", "desktop"} {
record, err := inv.AddNode(ctx, name)
if err != nil {
t.Fatal(err)
}
if err := inv.SetPlace(ctx, name, name+".example:51820", "here", false, fmt.Sprintf("10.77.0.%d", i+3)); err != nil {
t.Fatal(err)
}
if err := inv.RecordProfile(ctx, record.ID, map[string]any{"capabilities": []any{
map[string]any{"name": "container-runtime", "present": true},
map[string]any{"name": "systemd", "present": true}}}); err != nil {
t.Fatal(err)
}
if err := inv.RecordSealingKey(ctx, record.ID, aPublicKey(t)); err != nil {
t.Fatal(err)
}
if err := inv.RecordOverlayKey(ctx, record.ID, aPublicKey(t)); err != nil {
t.Fatal(err)
}
if _, err := inv.Assign(ctx, name, overlay.Name); err != nil {
t.Fatal(err)
}
}
raw, err := os.ReadFile(filepath.Join("testdata", "config-moved", "mesh.json"))
if err != nil {
t.Fatal(err)
}
var w configuredMesh
if err := json.Unmarshal(raw, &w); err != nil {
t.Fatal(err)
}
list, err := catalogue.ParsePredefined(w.Predefined)
if err != nil {
t.Fatal(err)
}
catalogue.UsePredefined(list)
served := map[string]map[string]any{}
for _, text := range w.Modules {
var shape map[string]json.RawMessage
if err := json.Unmarshal(text, &shape); err != nil {
t.Fatal(err)
}
var name string
_ = json.Unmarshal(shape["module"], &name)
if s, ok := shape["serves-checked"]; ok {
var serves map[string]any
if err := json.Unmarshal(s, &serves); err != nil {
t.Fatal(err)
}
served[name] = serves
delete(shape, "serves-checked")
}
text, err := json.Marshal(shape)
if err != nil {
t.Fatal(err)
}
m, err := catalogue.ParseManifest(text)
if err != nil {
t.Fatalf("%s: %v", name, err)
}
register(t, open, m)
}
for _, l := range w.Layers {
values, err := json.Marshal(l.Values)
if err != nil {
t.Fatal(err)
}
if l.Node == "" {
inv.ExecForTest(t, `insert into settings (node, module, values) values (null, $1, $2)`, l.Module, string(values))
continue
}
inv.ExecForTest(t, `insert into settings (node, module, values) select id, $2, $3 from node where name = $1`,
l.Node, l.Module, string(values))
}
for _, module := range movedKeys(w.On) {
for _, node := range w.On[module] {
if _, err := assign(ctx, open, node, module); err != nil {
t.Fatalf("assign %s %s: %v", node, module, err)
}
}
}
return open, w, served
}
func TestEveryMachineIsToldTheSameAfterTheMoveOfConfig(t *testing.T) {
open, w, served := aConfiguredMesh(t)
ctx := t.Context()
readBackTries = 0
t.Cleanup(func() { readBackTries = 15 })
inv := open.inventory
config := &standInMeshConfig{open: open, values: map[string]any{}}
inv.ReadsConfigFrom(config.laid)
before := toldEveryMachine(t, open, served, w.On)
listedBefore := effectiveEverywhere(t, open, w)
factsBefore, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
asked, err := runConfigMove(ctx, inv, config, false, "the move of config, run by a test", "")
if err != nil || !asked.DryRun || asked.Switched || len(asked.Problems) > 0 || len(config.sets) > 0 {
t.Fatalf("the move, not confirmed, says %+v (%v) and set %v", asked, err, config.sets)
}
// nodered's live layer says the file's name for its time zone: set under the config key (issue 751).
if s := setAt(asked, "module.anchor.nodered.time-zone"); len(s) != 1 || s[0].Value != "Europe/Brussels" ||
!strings.HasSuffix(s[0].From[0], ": timeZone") {
t.Errorf("nodered's time zone moves as %+v", s)
}
for _, s := range asked.Sets {
if s.Key == "timeZone" {
t.Errorf("a key is set under the file's name: %+v", s)
}
}
// claude-code's whole-mesh managed settings become mesh config for agent-managed-settings (ADR 0348's
// consequences), and its whole-mesh layer sets no module value.
if s := setAt(asked, "mesh.agent-managed-settings"); len(s) != 1 || s[0].From[0] != "claude-code on the whole mesh: managed_settings" {
t.Errorf("claude-code's managed settings move as %+v", s)
}
for _, node := range fourMachines {
if s := setAt(asked, "module."+node+".claude-code.managed_settings"); len(s) > 0 {
t.Errorf("claude-code's managed settings are also set on %s: %+v", node, s)
}
}
// A whole-mesh layer of a value no predefined key holds goes to each machine it is assigned to, under that
// machine's layer: power's on the workstation is the workstation's, its own elsewhere.
if s := setAt(asked, "module.workstation.power.handle-power-key"); len(s) != 1 || s[0].Value != "suspend" {
t.Errorf("power's key on the workstation moves as %+v", s)
}
if s := setAt(asked, "module.anchor.power.handle-power-key"); len(s) != 1 || s[0].Value != "poweroff" {
t.Errorf("power's key on the anchor moves as %+v", s)
}
for _, s := range asked.Sets {
if s.Level == "node" {
t.Errorf("a machine's layer became node config: %+v", s)
}
}
var stays []string
for _, s := range asked.Stays {
stays = append(stays, strings.Fields(s)[0])
}
sort.Strings(stays)
if want := []string{"bazarr", "bazarr", "bookshelf", "jackett", "kernel", "kometa", "lidarr", "nzbget", "ombi",
"plex", "qbittorrent", "radarr", "sonarr", "tautulli"}; len(stays) < 10 || !containsString(stays, "kernel") {
t.Errorf("the layers that stay are %v, want those of %v", stays, want)
}
if after := toldEveryMachine(t, open, served, w.On); !reflect.DeepEqual(after, before) {
t.Fatal("a dry run changed what a machine is told")
}
moved, err := runConfigMove(ctx, inv, config, true, "the move of config, run by a test", "release 57 is delivered")
if err != nil || moved.DryRun || !moved.Switched {
t.Fatalf("the move says %+v (%v)", moved, err)
}
if len(config.sets) != len(asked.Sets) {
t.Fatalf("the move set %d values, and said it would set %d", len(config.sets), len(asked.Sets))
}
if held := inv.QueryForTest(t, `select module from settings s where values <> '{}'
and module in (select name from module where manifest ? 'config')`); len(held) != 0 {
t.Fatalf("a layer of a module in the config form still holds values: %v", held)
}
after := toldEveryMachine(t, open, served, w.On)
for _, node := range fourMachines {
if after[node] != before[node] {
t.Errorf("%s is told something else after the move:\n%s\nwas:\n%s", node, after[node], before[node])
}
}
if listedAfter := effectiveEverywhere(t, open, w); !reflect.DeepEqual(listedAfter, listedBefore) {
t.Errorf("settings lists other values after the move:\n%v\nwas:\n%v", listedAfter, listedBefore)
}
// And config lists what settings listed: each module's resolved value on each machine is the value settings
// said before the move.
resolved, err := config.Resolved(ctx)
if err != nil {
t.Fatal(err)
}
for _, r := range resolved {
said, listed := listedBefore[r.Node+"."+r.Module+"."+r.Key]
if r.Source == "unset" {
if listed && said != nil {
t.Errorf("%s of %s on %s is unset in config, and settings said %v", r.Key, r.Module, r.Node, said)
}
continue
}
var v any
_ = json.Unmarshal(r.Value, &v)
if !listed || !reflect.DeepEqual(v, said) {
t.Errorf("config resolves %s of %s on %s as %v, and settings said %v", r.Key, r.Module, r.Node, v, said)
}
}
// The facts snapshot carries mesh-config's values for the merge gate, as each machine's layer of each module, and
// each machine's declaration composes to the digest it did before the move.
factsAfter, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
// Machines are named in a snapshot by stand-ins of its own; each is found by what it composes.
digests := func(f snapshotFacts) ([]string, string) {
var out []string
zone := ""
for _, m := range f.Machines {
out = append(out, m.Declaration.Digest)
for _, l := range m.Settings {
if l.Module == "nodered" {
zone = compactJSON(l.Values["time-zone"])
}
}
}
sort.Strings(out)
return out, zone
}
was, _ := digests(factsBefore)
now, zone := digests(factsAfter)
if zone != `"Europe/Brussels"` {
t.Errorf("the facts snapshot does not carry nodered's time zone from mesh-config: %s", zone)
}
if len(now) != len(fourMachines) || now[0] == "" || !reflect.DeepEqual(now, was) {
t.Errorf("the declarations in the facts snapshot are %v after the move, %v before", now, was)
}
for _, s := range factsAfter.Settings {
if _, isConfig := map[string]bool{"betterbird": true, "claude-code": true, "power": true, "xdg": true}[s.Module]; isConfig &&
len(s.Values) > 0 {
t.Errorf("the facts snapshot still carries the whole mesh's layer of %s: %v", s.Module, s.Values)
}
}
config.sets = nil
again, err := runConfigMove(ctx, inv, config, true, "the move of config, run by a test", "again")
if err != nil || len(config.sets) != 0 || len(again.Moved) != 0 {
t.Fatalf("asked again, the move set %v and says %+v (%v)", config.sets, again, err)
}
if after := toldEveryMachine(t, open, served, w.On); !reflect.DeepEqual(after, before) {
t.Fatal("the move asked again changed what a machine is told")
}
}
func TestTheMoveOfConfigRefusesWhatWouldChangeAMachine(t *testing.T) {
open, _, _ := aConfiguredMesh(t)
ctx := t.Context()
readBackTries = 0
t.Cleanup(func() { readBackTries = 15 })
inv := open.inventory
config := &standInMeshConfig{open: open, values: map[string]any{}}
inv.ReadsConfigFrom(config.laid)
inv.ExecForTest(t, `update settings set values = values || '{"endpoints": {"web": {"reach": "public"}}}'
where module = 'nodered'`)
inv.ExecForTest(t, `update settings set values = values || '{"colour": "blue"}' where module = 'unifi'`)
inv.ExecForTest(t, `update settings set values = '{"font-size": 900}' where module = 'dunst'`)
answer, err := runConfigMove(ctx, inv, config, true, "a test", "release 57 is delivered")
if err == nil || answer.Switched || len(config.sets) > 0 {
t.Fatalf("the move went ahead: %+v (%v), set %v", answer, err, config.sets)
}
for _, want := range []string{"nodered on anchor still holds placement (endpoints): run move-placement first",
`unifi on anchor says "colour", which unifi does not declare`, "font-size at dunst's own value on workstation"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not say %q: %v", want, err)
}
}
inv.ExecForTest(t, `update settings set values = values - 'endpoints' where module = 'nodered'`)
inv.ExecForTest(t, `update settings set values = values - 'colour' where module = 'unifi'`)
inv.ExecForTest(t, `update settings set values = '{"font-size": 13, "width": 500}' where module = 'dunst'`)
// Set in mesh-config before the move, on a module whose layers say nothing of it: the switch would lay it.
config.values["module.anchor.grafana."+movedKeys(func() map[string]catalogue.SettingDeclaration {
shelf, _ := inv.Catalogue(ctx)
return shelf["grafana"].Settings
}())[0]] = "set before the move"
answer, err = runConfigMove(ctx, inv, config, true, "a test", "release 57 is delivered")
if err == nil || !strings.Contains(err.Error(), "grafana on anchor resolves") || answer.Switched {
t.Fatalf("a value no layer says was laid by the switch: %+v (%v)", answer, err)
}
if moved, _ := inv.ConfigMoved(ctx); moved.Moved {
t.Fatal("the switch is on after a refused move")
}
if held := inv.QueryForTest(t, `select count(*) from settings where values <> '{}'`); held[0][0] == "0" {
t.Fatal("a refused move emptied the layers")
}
}
func TestTheSettingsVerbForwardsToConfigOnceTheValuesMoved(t *testing.T) {
open, w, served := aConfiguredMesh(t)
ctx := t.Context()
inv := open.inventory
readBackTries = 0
t.Cleanup(func() { readBackTries = 15 })
config := &standInMeshConfig{open: open, values: map[string]any{}}
inv.ReadsConfigFrom(config.laid)
savedRead, savedAsk := readConfigValues, configAsk
readConfigValues, configAsk = config.laid, config.ask
t.Cleanup(func() { readConfigValues, configAsk = savedRead, savedAsk })
if _, err := runConfigMove(ctx, inv, config, true, "a test", "release 57 is delivered"); err != nil {
t.Fatal(err)
}
shown := stdoutOf(t, func() error { return settingsCommand(ctx, []string{"show", "nodered", "--node", "anchor"}) })
for _, want := range []string{`"time-zone": "Europe/Brussels"`, `time-zone = "Europe/Brussels" (module)`,
"mesh-config.config", `"mqtt": {`} {
if !strings.Contains(shown, want) {
t.Errorf("settings show does not say %q:\n%s", want, shown)
}
}
if strings.Contains(shown, "timeZone") {
t.Errorf("settings show still says the file's name:\n%s", shown)
}
config.asked = nil
said := stdoutOf(t, func() error {
return settingsCommand(ctx, []string{"set", "dunst", `{"font-size": 12, "width": 500}`, "--node", "workstation"})
})
if config.values["module.workstation.dunst.font-size"] != float64(12) || !strings.Contains(said, "~ font-size") ||
containsString(config.asked, "set module.workstation.dunst.width") {
t.Fatalf("settings set reached mesh-config as %v (%v):\n%s", config.values, config.asked, said)
}
told := toldEveryMachine(t, open, served, w.On)["workstation"]
if !strings.Contains(told, "12") {
t.Error("the workstation is not told the font size set through settings")
}
if err := settingsCommand(ctx, []string{"set", "dunst", `{"font-size": 12}`, "--node", "workstation"}); err == nil ||
!strings.Contains(err.Error(), "would no longer set width") {
t.Fatalf("a set dropping a value without --replace: %v", err)
}
if err := settingsCommand(ctx, []string{"set", "dunst", `{"font-size": 12}`}); err == nil ||
!strings.Contains(err.Error(), "no own value for the whole mesh") {
t.Fatalf("a module value for the whole mesh: %v", err)
}
t.Setenv(verbVar, "settings")
for _, c := range []struct{ module, node, values, key, held, why string }{
// A file root reads: refused as before the move (novox/hq issue 339).
{"hostname", "anchor", `{"hostname": "elsewhere"}`, "module.anchor.hostname.hostname", "Ace", "issue 339"},
{"messenger", "laptop", `{"desktop-machines": ["g14", "shanks"], "time-zone": "UTC"}`,
"module.laptop.messenger.time-zone", "Europe/Brussels", "issue 339"},
} {
err := settingsCommand(ctx, []string{"set", c.module, c.values, "--node", c.node})
if err == nil || !strings.Contains(err.Error(), c.why) || config.values[c.key] != c.held {
t.Fatalf("a trusted value of %s through a verb: %v, holds %v", c.module, err, config.values[c.key])
}
}
// And a trusted value no file makes the terminal's is refused as trusted, since mesh-config takes the controller's
// word for it (ADR 0348 §8).
if err := refuseTrustedThroughAVerb(ctx, inv, "messenger", "laptop", []string{"time-zone"}); err == nil ||
!strings.Contains(err.Error(), "ADR 0348 §8") {
t.Fatalf("a trusted value through a verb: %v", err)
}
if err := refuseTrustedThroughAVerb(ctx, inv, "dunst", "workstation", []string{"width"}); err != nil {
t.Fatalf("a value nobody trusts through a verb: %v", err)
}
stdoutOf(t, func() error {
return settingsCommand(ctx, []string{"set", "dunst", `{"font-size": 12, "width": 600}`, "--node", "workstation"})
})
if config.values["module.workstation.dunst.width"] != float64(600) {
t.Fatalf("a value nobody trusts was not set through a verb: %v", config.values["module.workstation.dunst.width"])
}
t.Setenv(verbVar, "")
stdoutOf(t, func() error { return settingsCommand(ctx, []string{"clear", "dunst", "--node", "workstation"}) })
for _, k := range []string{"font-size", "width"} {
if _, held := config.values["module.workstation.dunst."+k]; held {
t.Errorf("settings clear left dunst's %s on the workstation", k)
}
}
config.asked = nil
proposed := stdoutOf(t, func() error {
return settingsCommand(ctx, []string{"propose", "hostname", `{"hostname": "elsewhere"}`, "--node", "anchor"})
})
if !containsString(config.asked, "propose module.anchor.hostname.hostname") || !strings.Contains(proposed, "p-1") {
t.Fatalf("settings propose reached mesh-config as %v:\n%s", config.asked, proposed)
}
config.asked = nil
kept := stdoutOf(t, func() error { return settingsCommand(ctx, []string{"show", "kernel", "--node", "laptop"}) })
if !strings.Contains(kept, "transparent-huge-pages") || len(config.asked) > 0 {
t.Fatalf("a module in the settings form is shown as %q, asking mesh-config %v", kept, config.asked)
}
}
func TestAnApprovalRaisedBeforeTheMoveOfConfigIsRefusedAfterIt(t *testing.T) {
open, _, _ := aConfiguredMesh(t)
ctx := t.Context()
readBackTries = 0
t.Cleanup(func() { readBackTries = 15 })
inv := open.inventory
config := &standInMeshConfig{open: open, values: map[string]any{}}
inv.ReadsConfigFrom(config.laid)
shelf, err := inv.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
key := movedKeys(func() map[string]catalogue.SettingDeclaration { return shelf["grafana"].Settings }())[0]
values := map[string]any{key: "raised before the move"}
p := settingsProposal{Module: "grafana", Node: "anchor", Values: values, From: "g14/claude-code", At: time.Now(),
Digest: layerDigest(values), BeforeDigest: layerDigest(nil)}
if _, has, err := inv.Layer(ctx, "anchor", "grafana"); err != nil || has {
t.Fatalf("grafana has a layer on anchor before the test: %v %v", has, err)
}
if moved, err := runConfigMove(ctx, inv, config, true, "a test", "release 57 is delivered"); err != nil || !moved.Switched {
t.Fatalf("the move says %+v (%v)", moved, err)
}
if reads, err := inv.ReadsConfig(ctx, "grafana"); err != nil || !reads {
t.Fatalf("grafana does not read config after the move: %v %v", reads, err)
}
_, err = setLayerIn(open)(ctx, p, "s30", "approved by the operator")
if err == nil || !strings.Contains(err.Error(), "moved to mesh-config since the operator was shown the change") ||
!strings.Contains(err.Error(), "propose it again") {
t.Fatalf("the stale approval: %v", err)
}
if layer, has, err := inv.Layer(ctx, "anchor", "grafana"); err != nil || has && len(layer) > 0 {
t.Errorf("the refused approval wrote the layer: %v %v %v", layer, has, err)
}
if reads, err := inv.ReadsConfig(ctx, "grafana"); err != nil || !reads {
t.Errorf("the refused approval turned ReadsConfig off: %v %v", reads, err)
}
}
func TestAPlacementOnlyApprovalRaisedBeforeTheMoveIsSetAfterIt(t *testing.T) {
open, _, _ := aConfiguredMesh(t)
ctx := t.Context()
readBackTries = 0
t.Cleanup(func() { readBackTries = 15 })
inv := open.inventory
config := &standInMeshConfig{open: open, values: map[string]any{}}
inv.ReadsConfigFrom(config.laid)
values := map[string]any{catalogue.PlacesSetting: map[string]any{"data": "/srv/grafana-data"}}
p := settingsProposal{Module: "grafana", Node: "anchor", Values: values, From: "g14/claude-code", At: time.Now(),
Digest: layerDigest(values), BeforeDigest: layerDigest(nil)}
if _, has, err := inv.Layer(ctx, "anchor", "grafana"); err != nil || has {
t.Fatalf("grafana has a layer on anchor before the test: %v %v", has, err)
}
if moved, err := runConfigMove(ctx, inv, config, true, "a test", "release 57 is delivered"); err != nil || !moved.Switched {
t.Fatalf("the move says %+v (%v)", moved, err)
}
if _, err := setLayerIn(open)(ctx, p, "s31", "approved by the operator"); err != nil {
t.Fatalf("the placement-only approval after the move: %v", err)
}
if _, has, err := inv.Layer(ctx, "anchor", "grafana"); err != nil || !has {
t.Errorf("the approval did not set the layer: %v %v", has, err)
}
if reads, err := inv.ReadsConfig(ctx, "grafana"); err != nil || !reads {
t.Errorf("the approval turned ReadsConfig off: %v %v", reads, err)
}
}
+336
View File
@@ -0,0 +1,336 @@
package main
import (
"context"
"encoding/json"
"fmt"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
snapshot "github.com/novox/mesh-controller/internal/facts"
"github.com/novox/mesh-controller/internal/link"
)
// configuredMesh is every live settings layer (issue 664) — read with the settings verb on 2026-10-15, on the four
// machines as anchor, laptop, workstation and desktop, the whole mesh's with no node; personal values stood in for,
// and the placement words left out, which move-placement moves first — with each module in the config form shaped
// as its definition on the feature branch: its config, its files (templates inlined) and directories, placeholders
// of kinds this test's mesh does not hold stood in for. A module still in the settings form (the media catalogue's,
// and kernel) stands in as a mergeable file that takes every key. `on` is where each is assigned, as live.
type configuredMesh struct {
Modules []json.RawMessage `json:"modules"`
Layers []configuredLayer `json:"layers"`
On map[string][]string `json:"on"`
// Predefined is the catalogue's list of predefined keys, which a module falls back to.
Predefined json.RawMessage `json:"predefined"`
}
type configuredLayer struct {
Node string `json:"node"`
Module string `json:"module"`
Values map[string]any `json:"values"`
}
var fourMachines = []string{"anchor", "laptop", "workstation", "desktop"}
// aConfiguredMesh is aMesh with two machines more, every module of the configured mesh registered and assigned, and
// its layers held as the store holds them once move-placement has run.
// standInMeshConfig is mesh-config as this test holds it: each value at its level, and resolved from the catalogue
// the store holds as mesh-config resolves (novox/hq ADR 0348 §2): the module's own value on the node, else node
// config then mesh config for the predefined key it falls back to, else its default; a required value nothing sets is
// unset.
type standInMeshConfig struct {
open *stores
values map[string]any
sets []string
// asked is each call of its verb: the action and the level's key.
asked []string
}
func (c *standInMeshConfig) Values(context.Context) (map[string]json.RawMessage, error) {
out := map[string]json.RawMessage{}
for k, v := range c.values {
raw, err := json.Marshal(v)
if err != nil {
return nil, err
}
out[k] = raw
}
return out, nil
}
func (c *standInMeshConfig) Set(_ context.Context, level configLevel, key string, value any, by, why string) error {
if by == "" || why == "" {
return fmt.Errorf("%s at %s is set without who or why", key, level)
}
c.values[level.valueKey(key)] = normalJSON(value)
c.sets = append(c.sets, level.valueKey(key))
return nil
}
func (c *standInMeshConfig) Resolved(ctx context.Context) ([]resolvedEntry, error) {
entries, err := c.open.inventory.Catalogued(ctx)
if err != nil {
return nil, err
}
var out []resolvedEntry
for _, e := range entries {
m := e.Manifest
if !m.DeclaresConfig() {
continue
}
for _, node := range e.On {
for _, key := range movedKeys(m.Settings) {
d := m.Settings[key]
r := resolvedEntry{Node: node, Module: m.Module, Key: key, Source: "default"}
value, has := c.values["module."+node+"."+m.Module+"."+key]
switch {
case has:
r.Source = "module"
case d.Predefined != "" && c.values["node."+node+"."+d.Predefined] != nil:
value, r.Source = c.values["node."+node+"."+d.Predefined], "node"
case d.Predefined != "" && c.values["mesh."+d.Predefined] != nil:
value, r.Source = c.values["mesh."+d.Predefined], "mesh"
case d.Required:
r.Source = "unset"
default:
value = d.Default
}
if r.Source != "unset" {
if r.Value, err = json.Marshal(value); err != nil {
return nil, err
}
}
out = append(out, r)
}
}
}
return out, nil
}
// laid is the stand-in as composition reads it once the switch is on (inventory.ConfigValues).
func (c *standInMeshConfig) laid(ctx context.Context, node, module string) (map[string]any, error) {
resolved, err := c.Resolved(ctx)
if err != nil {
return nil, err
}
return configLaid(resolved, node, module)
}
// snapshotFacts is the facts snapshot as gatherFacts gives it.
type snapshotFacts = snapshot.Facts
// toldEveryMachine is what each of the four machines is told — every resource and the filter's rules — and what
// each module that serves something serves, settled from its layers, as JSON a person can diff.
func toldEveryMachine(t *testing.T, open *stores, served map[string]map[string]any, on map[string][]string) map[string]string {
t.Helper()
ctx := t.Context()
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
told := map[string]string{}
for _, node := range fourMachines {
plan, settings, err := planFor(ctx, open, node)
if err != nil {
t.Fatalf("%s: %v", node, err)
}
with, _, err := renderingFor(ctx, open, node, plan, settings, gens, Allocating)
if err != nil {
t.Fatalf("%s: %v", node, err)
}
composed, err := plan.Compose(with)
if err != nil {
t.Fatalf("%s: %v", node, err)
}
rules, err := plan.Rules(with)
if err != nil {
t.Fatalf("%s: %v", node, err)
}
settles := map[string]any{}
for module, serves := range served {
if !containsString(on[module], node) {
continue
}
layers, err := open.inventory.SettingsFor(ctx, node, module)
if err != nil {
t.Fatal(err)
}
for name, s := range serves {
fact, _ := s.(map[string]any)
settled, err := catalogue.Settle(fact, catalogue.WithDefaults(shelf[module], layers))
if err != nil {
settles[module+" serves "+name] = "error: " + err.Error()
continue
}
settles[module+" serves "+name] = settled
}
}
got, err := json.MarshalIndent(map[string]any{"resources": composed.Resources, "left-out": composed.LeftOut,
"rules": rules, "served": settles}, "", " ")
if err != nil {
t.Fatal(err)
}
told[node] = string(got)
}
return told
}
// effectiveEverywhere is each module's every value on each machine it is assigned to, as `settings show` and
// `settings preferences` say them (catalogue.Effective): the value, never where it came from, which the move
// changes by design; a key a layer says by its file's name is read as its config key.
func effectiveEverywhere(t *testing.T, open *stores, w configuredMesh) map[string]any {
t.Helper()
ctx := t.Context()
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
out := map[string]any{}
for _, module := range movedKeys(w.On) {
if !shelf[module].DeclaresConfig() {
continue
}
for _, node := range w.On[module] {
layers, err := open.inventory.SettingsFor(ctx, node, module)
if err != nil {
t.Fatal(err)
}
// A layer's old spelling of a key (nodered's timeZone) is listed beside the key it is the file's name
// for, and reaches the file as that key does (issue 751): read as that key, which the move sets.
as := catalogue.TakenAs(shelf[module])
for _, v := range catalogue.Effective(shelf[module], layers) {
if key, old := as[v.Key]; old {
out[node+"."+module+"."+key] = normalJSON(v.Value)
continue
}
if _, said := out[node+"."+module+"."+v.Key]; said && v.FromDefault {
continue
}
out[node+"."+module+"."+v.Key] = normalJSON(v.Value)
}
}
}
return out
}
func setAt(a configMoveAnswer, key string) []configSet {
var out []configSet
for _, s := range a.Sets {
if s.valueKey(s.Key) == key {
out = append(out, s)
}
}
return out
}
// **The settings layers move into mesh-config and no machine is told anything new** (novox/hq ADR 0348 §9, design
// 57 K5, issue 664). Every live layer is seeded as the store holds it once move-placement has run, on modules shaped
// as their definitions, and every machine composed. The move, asked without confirming, says each value it would set
// and where from, and sets nothing; confirmed, it sets them in mesh-config, reads them back, empties the layers it
// moved and switches. Each machine is then told byte for byte what it was told before, every module serves the same,
// and `settings` lists the same values `config` resolves. Asked again, it sets nothing.
// **The move refuses what it cannot move as it composes, and sets nothing** (issue 664): a layer that still holds
// placement (move-placement has not run), a key its module does not declare, and a value its declaration refuses —
// each said by name; and a value mesh-config resolves from a level no layer says (it would be laid once the switch
// is on) leaves every layer and the switch as they were.
// The verb's command line: a dry run unless confirmed, and a confirmed move says why, recorded as a person's step of
// the release, never a repair (issue 664).
func TestTheMoveOfConfigIsADryRunUnlessConfirmed(t *testing.T) {
for _, c := range []struct {
args map[string]any
want string
}{
{map[string]any{}, "move-config --json"},
{map[string]any{"confirm": "true", "why": "release 57 is delivered"}, "move-config --json --confirm --why release 57 is delivered"},
} {
argv, err := argvFor("move-config", c.args)
if err != nil || strings.Join(argv, " ") != c.want {
t.Errorf("%v: %q %v, want %q", c.args, argv, err, c.want)
}
}
for _, args := range []map[string]any{{"confirm": "true"}, {"why": "tidy"}, {"node": "anchor"}} {
if argv, err := argvFor("move-config", args); err == nil {
t.Errorf("%v was composed as %q", args, argv)
}
}
if repairingCommand([]string{"move-config", "--json", "--confirm", "--why", "x"}) != "move-config" {
t.Error("a confirmed move through the generic verb would go unrecorded")
}
if !personsDecision(link.HandAct{Verb: "move-config"}) {
t.Error("a move of the release would count toward a healer the mesh lacks")
}
}
// ask is mesh-config's verb as the stand-in answers it: list, set, clear and propose.
func (c *standInMeshConfig) ask(ctx context.Context, args map[string]any) (json.RawMessage, error) {
str := func(k string) string { s, _ := args[k].(string); return s }
level := configLevel{Level: "mesh"}
switch {
case str("module") != "" && str("node") != "":
level = configLevel{Level: "module", Node: str("node"), Module: str("module")}
case str("node") != "":
level = configLevel{Level: "node", Node: str("node")}
}
c.asked = append(c.asked, str("action")+" "+level.valueKey(str("key")))
switch str("action") {
case "list":
resolved, err := c.Resolved(ctx)
if err != nil {
return nil, err
}
out := map[string]any{"resolved": []resolvedEntry{}, "set": []any{}}
var listed []resolvedEntry
for _, r := range resolved {
if r.Module == str("module") && (str("node") == "" || r.Node == str("node")) {
listed = append(listed, r)
}
}
out["resolved"] = listed
var set []any
for _, k := range movedKeys(c.values) {
parts := strings.SplitN(k, ".", 4)
if len(parts) == 4 && parts[0] == "module" && parts[2] == str("module") {
set = append(set, map[string]any{"level": configLevel{Level: "module", Node: parts[1], Module: parts[2]},
"key": parts[3], "value": c.values[k], "by": "a test"})
}
}
out["set"] = set
return json.Marshal(out)
case "set":
if str("why") == "" || str("by") == "" {
return nil, fmt.Errorf("a set without why or by: %v", args)
}
return nil, c.Set(ctx, level, str("key"), args["value"], str("by"), str("why"))
case "clear":
if _, had := c.values[level.valueKey(str("key"))]; !had {
return nil, fmt.Errorf("nothing is set for %s", str("key"))
}
delete(c.values, level.valueKey(str("key")))
return json.RawMessage(`{}`), nil
case "propose":
return json.RawMessage(`{"proposal":"p-1"}`), nil
}
return nil, fmt.Errorf("the stand-in does not %s", str("action"))
}
// **`settings` answers by forwarding to `config`** (novox/hq ADR 0348 §1, design 57 K5, issue 664): once the move
// has run, a module in the config form is shown from mesh-config, its values set and cleared there as its own on the
// machine (placement still beside the assignment), and proposed through it; a trusted value is refused through a
// verb, a module value for the whole mesh is refused, and a module still in the settings form keeps today's verb.
// **An approval for a module whose config has moved is refused, and writes nothing** (issue 830, novox/hq ADR 0348):
// a proposal raised while grafana read its layer (it had none, so the layer is as the operator was shown it) and
// approved after the move would put its values back into the layer, ReadsConfig would turn false, and the module
// would compose from emptied layers everywhere.
// **A proposal of placement only is set after the move, as before** (issue 830): forwardPropose sends it down the
// old path on purpose, so refusing it would loop the operator (refused, proposed again, refused).
+128
View File
@@ -0,0 +1,128 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
)
// **Where the controller reads config values once the move has run** (novox/hq ADR 0348 §9, design 57 K5, issue 664):
// mesh-config_resolved, the bucket mesh-config alone writes, keyed `<node>.<module>.<key>`, each value with the
// source it came from. Read only for a module the switch reaches (inventory.ReadsConfig), so a controller before the
// move never asks the bus for config. The exception to ADR 0348 §9 ("the controller holds no config value") that
// ADR 0349 clause 17's old way needs, until design 57's R1 hands every module to the provisioner.
// resolvedEntry is one key of mesh-config_resolved as the controller reads it: the value and where it came from.
type resolvedEntry struct {
Node string `json:"node"`
Module string `json:"module"`
Key string `json:"key"`
Value json.RawMessage `json:"value,omitempty"`
Source string `json:"source"`
}
// The sources a resolved value is laid over a module's files from: mesh config, node config and the module's own
// value. Its manifest's default is composition's already, and a taken key's default is never laid over its file;
// a required value nothing sets (`unset`) is refused by composition by name, as it is today.
var laidSources = map[string]bool{"mesh": true, "node": true, "module": true}
// configLaid is the values of one module on one node that are laid over its files, from the resolved entries.
func configLaid(entries []resolvedEntry, node, module string) (map[string]any, error) {
out := map[string]any{}
for _, e := range entries {
if e.Node != node || e.Module != module || !laidSources[e.Source] {
continue
}
var v any
if err := json.Unmarshal(e.Value, &v); err != nil {
return nil, fmt.Errorf("mesh-config's value %s of %s on %s cannot be read: %w", e.Key, module, node, err)
}
out[e.Key] = v
}
return out, nil
}
// resolvedRead is what one read of the whole bucket held, and when it was read.
type resolvedRead struct {
mu sync.Mutex
at time.Time
entries []resolvedEntry
}
// resolvedFresh is how long one read of the bucket answers: a plan reads every module of a machine, and one read
// serves them all; a value set since is read by the next composition.
const resolvedFresh = 5 * time.Second
var theResolved resolvedRead
// readConfigValues is where an opened inventory reads config values once the switch is on; a test replaces it.
var readConfigValues inventory.ConfigValues = busConfigValues
// busConfigValues reads one module's config values on one node from mesh-config_resolved (inventory.ConfigValues).
func busConfigValues(ctx context.Context, node, module string) (map[string]any, error) {
theResolved.mu.Lock()
defer theResolved.mu.Unlock()
if theResolved.entries == nil || time.Since(theResolved.at) > resolvedFresh {
entries, err := readResolved(ctx)
if err != nil {
return nil, err
}
theResolved.entries, theResolved.at = entries, time.Now()
}
return configLaid(theResolved.entries, node, module)
}
// readResolved is every entry of mesh-config_resolved, read on the bus.
func readResolved(ctx context.Context) ([]resolvedEntry, error) {
entries := []resolvedEntry{}
err := onTheBus(func(conn *nats.Conn) error {
js, err := jetstream.New(conn)
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
kv, err := js.KeyValue(ctx, broker.ConfigResolvedBucket)
if err != nil {
return fmt.Errorf("%s cannot be opened: %w", broker.ConfigResolvedBucket, err)
}
w, err := kv.WatchAll(ctx, jetstream.IgnoreDeletes())
if err != nil {
return err
}
defer func() { _ = w.Stop() }()
for {
select {
case <-ctx.Done():
return fmt.Errorf("%s was not read whole: %w", broker.ConfigResolvedBucket, ctx.Err())
case e, open := <-w.Updates():
if !open {
return errors.New(broker.ConfigResolvedBucket + " stopped answering")
}
if e == nil {
return nil
}
var r resolvedEntry
if err := json.Unmarshal(e.Value(), &r); err != nil {
return fmt.Errorf("%s holds %q, which is not a resolved value: %w", broker.ConfigResolvedBucket,
e.Key(), err)
}
if r.Node == "" || r.Module == "" || r.Key == "" || !strings.HasPrefix(e.Key(), r.Node+"."+r.Module+".") {
return fmt.Errorf("%s holds %q, whose value names another key", broker.ConfigResolvedBucket, e.Key())
}
entries = append(entries, r)
}
}
})
return entries, err
}
+305
View File
@@ -0,0 +1,305 @@
package main
import (
"context"
"crypto/ecdh"
"crypto/x509"
"encoding/base64"
"encoding/json"
"encoding/pem"
"errors"
"fmt"
"slices"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The core's own credentials, sealed by the vault (novox/hq ADR 0349 §16, issue 755; the lead's decision on issue 755,
// notes 4 and 5).
//
// For a machine whose engine opens the vault's box (link.VaultBoxContract), each own secret of a module on the core's
// list (catalogue.CoreCredentialModules) travels as the vault sealed it to that machine's key, which the controller
// asks for on the send path (`get`, answered to the controller alone and sealed only to the key that machine's engine
// stated: mesh-vault's core.go) and carries without being able to open it. A machine below that contract is sent the
// controller's own sealed value, exactly as before.
//
// **The vault seals every answer afresh**, so the box last got is kept (inventory.CoreBox) and carried again while the
// vault's generation and the machine's key are the same: a declaration that changed with every answer would never be
// current. Every question but a send reads the kept box and asks nothing.
//
// **Until the vault holds a machine's core credentials, the controller seals them itself** (novox/hq ADR 0396, issue
// 908; it replaces the ADR 0349 §16 rule that the controller never does). When the vault does not answer, a credential
// with a kept box keeps it — the machine keeps the file it has — and one with none has its module composed as for an
// engine below the contract: with the controller's own sealed value, never left out (a core module left out took the
// bus's port from the packet filter and cut the remote machines off). Either way the send raises a condition naming
// the vault and the machine, cleared by the first send the vault answers whole. Only a machine with no sealing key, or
// one whose key cannot be read, leaves its module out.
// engineOpensVaultBox says whether a machine's node-engine opens the vault's box, by its own newest statement.
func engineOpensVaultBox(ctx context.Context, inv *inventory.Inventory, node string) (bool, error) {
stated, had, err := inv.HealthOf(ctx, node)
if err != nil {
return false, err
}
return had && stated.Contract >= link.VaultBoxContract, nil
}
// coreAnswer is the vault's `get` answer as the controller reads it: the name, the generation and the box, never
// opened.
type coreAnswer struct {
Name string `json:"name"`
Generation int `json:"generation"`
Sealed json.RawMessage `json:"sealed"`
}
// coreAsk asks the vault for one secret sealed to the given key.
type coreAsk func(name, publicKey string) (coreAnswer, error)
// coreBoxes is what the vault's boxes make of one machine's composition.
type coreBoxes struct {
// sealed is each core credential's box, by module and name (catalogue.Rendering.VaultSealed).
sealed map[string]map[string]json.RawMessage
// notPlaced is each core module left out, with why (catalogue.Rendering.NotPlaced).
notPlaced map[string]string
// keep is each box newly answered, to keep.
keep []inventory.CoreBox
// byController is each core module the vault did not answer for and no box was kept for, which is composed with
// the controller's own seal until the vault holds it (ADR 0396).
byController []string
// held is each credential the vault did not answer for whose kept box is carried again; silent is why the vault
// did not answer, per credential.
held, silent []string
}
// coreModulesOf is every module of the plan on the core's list whose own secrets this machine's declaration carries,
// with their plain own secret names, sorted: one handed to the provisioner has its secrets from the vault through the
// client, and one issued outside the mesh is never the vault's to seal here.
func coreModulesOf(modules []catalogue.Manifest, handed func(catalogue.Manifest) bool) map[string][]string {
out := map[string][]string{}
for _, m := range modules {
if !catalogue.CarriesVaultSealed(m.Module) || (handed != nil && handed(m)) {
continue
}
var names []string
for name, own := range m.OwnSecrets.Plain() {
if own.IssuedBy != catalogue.IssuedOutside {
names = append(names, name)
}
}
if len(names) > 0 {
sort.Strings(names)
out[m.Module] = names
}
}
return out
}
// sealingKeyPEM is a machine's sealing key, recorded raw in base64, as the SPKI PEM the vault seals to.
func sealingKeyPEM(raw string) (string, error) {
b, err := base64.StdEncoding.DecodeString(strings.TrimSpace(raw))
if err != nil || len(b) != 32 {
return "", errors.New("the machine's sealing key is not a raw X25519 key")
}
pub, err := ecdh.X25519().NewPublicKey(b)
if err != nil {
return "", err
}
der, err := x509.MarshalPKIXPublicKey(pub)
if err != nil {
return "", err
}
return string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: der})), nil
}
// coreBoxesOf decides each core credential of a machine: on a send, the vault's answer (the kept box while its
// generation and key are the same), the kept box where the vault did not answer, or the module left to the controller's
// own seal where nothing was kept; on any other question, the kept box, or a stand-in the next send asks for. Pure: what to keep and what to
// say is answered, not done.
func coreBoxesOf(node, sealingKey string, core map[string][]string, kept map[string]map[string]inventory.CoreBox,
choosing Choosing, ask coreAsk) coreBoxes {
out := coreBoxes{sealed: map[string]map[string]json.RawMessage{}, notPlaced: map[string]string{}}
modules := make([]string, 0, len(core))
for m := range core {
modules = append(modules, m)
}
sort.Strings(modules)
for _, module := range modules {
if sealingKey == "" {
out.notPlaced[module] = fmt.Sprintf("%s has no sealing key, so the vault can seal %s's credentials to "+
"nothing there (novox/hq ADR 0349 §16)", node, module)
continue
}
key, err := sealingKeyPEM(sealingKey)
if err != nil {
out.notPlaced[module] = fmt.Sprintf("%s's credentials cannot be asked of the vault: %v", module, err)
continue
}
boxes := map[string]json.RawMessage{}
for _, name := range core[module] {
k, has := kept[module][name]
valid := has && k.SealedTo == sealingKey && len(k.Box) > 0
if choosing != Allocating {
if valid {
boxes[name] = k.Box
} else {
boxes[name] = catalogue.ForeseenVaultBox
}
continue
}
secret := "own/" + node + "/" + module + "/" + name
got, err := ask(secret, key)
if err == nil && (got.Generation <= 0 || canonicalBox(got.Sealed) == nil) {
err = errors.New("it answered no box")
}
if err != nil {
out.silent = append(out.silent, fmt.Sprintf("%s: %v", secret, err))
if valid {
boxes[name] = k.Box
out.held = append(out.held, module+"/"+name)
continue
}
out.byController = append(out.byController, module)
break
}
if valid && k.Generation == got.Generation {
boxes[name] = k.Box
continue
}
box := canonicalBox(got.Sealed)
boxes[name] = box
out.keep = append(out.keep, inventory.CoreBox{Module: module, Name: name, Generation: got.Generation,
SealedTo: sealingKey, Box: box})
}
if _, left := out.notPlaced[module]; !left && !slices.Contains(out.byController, module) {
out.sealed[module] = boxes
}
}
return out
}
// canonicalBox is the vault's box in the one form the declaration carries — the sealed box's fields, in their order,
// and nothing else — or nil when it is not one.
func canonicalBox(raw json.RawMessage) json.RawMessage {
var b struct {
V int `json:"v"`
Eph string `json:"eph"`
IV string `json:"iv"`
Tag string `json:"tag"`
Ct string `json:"ct"`
}
if err := json.Unmarshal(raw, &b); err != nil || b.V <= 0 || b.Eph == "" || b.Ct == "" {
return nil
}
out, _ := json.Marshal(b)
return out
}
// coreVault is how a send asks the vault for a core credential: its `get`, in the controller's name. A test stands
// in for it.
var coreVault = func(ctx context.Context) coreAsk {
vault := theVault(ctx)
return func(name, publicKey string) (coreAnswer, error) {
raw, err := vault("get", map[string]any{"name": name, "public_key": publicKey})
if err != nil {
return coreAnswer{}, err
}
var a coreAnswer
if err := json.Unmarshal(raw, &a); err != nil {
return coreAnswer{}, fmt.Errorf("the vault's answer is not one: %w", err)
}
return a, nil
}
}
// coreBoxesFor is coreBoxesOf for one machine, read from and kept in the store, and said on a send.
func coreBoxesFor(ctx context.Context, inv *inventory.Inventory, node string, modules []catalogue.Manifest,
handed func(catalogue.Manifest) bool, choosing Choosing) (coreBoxes, error) {
core := coreModulesOf(modules, handed)
if len(core) == 0 {
if choosing == Allocating {
settleVaultSilent(ctx, node)
}
return coreBoxes{}, nil
}
key, err := inv.SealingKeyOf(ctx, node)
if err != nil {
return coreBoxes{}, err
}
kept, err := inv.KeptCoreBoxes(ctx, node)
if err != nil {
return coreBoxes{}, err
}
var ask coreAsk
if choosing == Allocating {
ask = coreVault(ctx)
}
out := coreBoxesOf(node, key, core, kept, choosing, ask)
for _, b := range out.keep {
if err := inv.KeepCoreBox(ctx, node, b); err != nil {
return coreBoxes{}, err
}
}
if choosing == Allocating {
sayVaultSilent(ctx, node, out)
}
return out, nil
}
// kindVaultSilent is a send that did not have the vault's box for a core credential of a machine.
const kindVaultSilent = "vault-silent"
// vaultSilentObservation is that, as a condition: `machine.<machine>.vault-silent`.
func vaultSilentObservation(node string, b coreBoxes) conditions.Observation {
o := conditions.Observation{Scope: conditions.ScopeMachine, ID: node, Kind: kindVaultSilent, Token: kindVaultSilent,
Machine: node, Severity: conditions.Warning, Source: coreDowngradeSource}
var said []string
if len(b.notPlaced) > 0 {
o.Severity = conditions.Urgent
var left []string
for m := range b.notPlaced {
left = append(left, m)
}
sort.Strings(left)
said = append(said, "not placed: "+strings.Join(left, ", "))
}
if len(b.byController) > 0 {
said = append(said, "sealed by the controller until the vault holds it: "+strings.Join(b.byController, ", "))
}
if len(b.held) > 0 {
said = append(said, "kept as last sent: "+strings.Join(b.held, ", "))
}
o.Summary = fmt.Sprintf("the vault did not answer for the core's credentials on %s; %s", node, strings.Join(said, "; "))
o.Said = o.Summary + " — " + strings.Join(b.silent, "; ")
return o
}
// sayVaultSilent raises the condition for a send the vault did not answer whole, and clears it for one it did.
func sayVaultSilent(ctx context.Context, node string, b coreBoxes) {
if len(b.silent) == 0 && len(b.notPlaced) == 0 {
settleVaultSilent(ctx, node)
return
}
fmt.Printf("HELD: %s\n", vaultSilentObservation(node, b).Said)
if conditionsFrom == nil {
return
}
if _, err := conditionsFrom.Observe(ctx, vaultSilentObservation(node, b)); err != nil {
fmt.Printf("the condition for the vault's silence on %s could not be raised: %v\n", node, err)
}
}
// settleVaultSilent clears the condition once a send had every core credential of the machine from the vault.
func settleVaultSilent(ctx context.Context, node string) {
if conditionsFrom == nil {
return
}
key := vaultSilentObservation(node, coreBoxes{}).Key()
if _, err := conditionsFrom.Clear(ctx, key, "the vault answered for every core credential of "+node); err != nil {
fmt.Printf("the condition %s could not be cleared: %v\n", key, err)
}
}
@@ -0,0 +1,129 @@
//go:build integration
package main
import (
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func aCoreMesh(t *testing.T, contract int) (*stores, *coreStandIn) {
t.Helper()
open := aMesh(t)
ctx := t.Context()
client := aClientModule()
register(t, open, client)
if _, err := assign(ctx, open, "laptop", client.Module); err != nil {
t.Fatal(err)
}
if err := issueProvisionerClient(ctx, open.inventory, client, "laptop", broker.Broker{Fingerprint: "ab"},
"10.0.0.1:4222"); err != nil {
t.Fatal(err)
}
if _, err := open.inventory.RecordHealth(ctx, inventory.NodeHealth{Node: "laptop", Contract: contract,
SaidAt: time.Now()}); err != nil {
t.Fatal(err)
}
v := &coreStandIn{generation: 1}
withVault(t, v)
return open, v
}
func TestTheSendCarriesTheVaultsBoxForACoreCredentialAndKeepsIt(t *testing.T) {
open, vault := aCoreMesh(t, link.VaultBoxContract)
key, err := open.inventory.SealingKeyOf(t.Context(), "laptop")
if err != nil {
t.Fatal(err)
}
f := clientsBroker(sentTo(t, open, "laptop"))
if f == nil || f["sealed"] != nil || f[catalogue.VaultSealedField] == nil {
t.Fatalf("the core credential is sent as %v: the vault's box, never the controller's seal", f)
}
if want := []string{"own/laptop/mesh-provisioner-client/" + broker.ProvisionerSecret}; !reflect.DeepEqual(vault.asked, want) {
t.Fatalf("the vault was asked %v", vault.asked)
}
if want, _ := sealingKeyPEM(key); vault.keys[0] != want {
t.Fatalf("the vault was asked to seal to %q, not the machine's key", vault.keys[0])
}
first, _ := json.Marshal(f[catalogue.VaultSealedField])
// The vault answers afresh with the same generation: the kept box is carried again, so the declaration is stable.
again, _ := json.Marshal(clientsBroker(sentTo(t, open, "laptop"))[catalogue.VaultSealedField])
if string(again) != string(first) {
t.Fatalf("a fresh box of the same generation changed the declaration: %s, then %s", first, again)
}
// A rotation in the vault (a newer generation) is carried.
vault.generation = 2
rotated, _ := json.Marshal(clientsBroker(sentTo(t, open, "laptop"))[catalogue.VaultSealedField])
if string(rotated) == string(first) {
t.Fatal("a newer generation was not carried")
}
if _, found := vaultSilence(t, "laptop"); found {
t.Fatal("a send the vault answered raised its silence")
}
}
func TestASilentVaultHoldsTheKeptBoxAndSaysSo(t *testing.T) {
open, vault := aCoreMesh(t, link.VaultBoxContract)
first, _ := json.Marshal(clientsBroker(sentTo(t, open, "laptop"))[catalogue.VaultSealedField])
vault.silent = errors.New("no responders")
held := clientsBroker(sentTo(t, open, "laptop"))
got, _ := json.Marshal(held[catalogue.VaultSealedField])
if held == nil || held["sealed"] != nil || string(got) != string(first) {
t.Fatalf("a silent vault's credential is sent as %v: the box kept, never the controller's seal", held)
}
c, found := vaultSilence(t, "laptop")
if !found || !strings.Contains(c.Summary, "vault") || !strings.Contains(c.Summary, "laptop") {
t.Fatalf("no condition names the vault and the machine: %+v", c)
}
// The vault answers again: the condition clears.
vault.silent = nil
sentTo(t, open, "laptop")
if _, found := vaultSilence(t, "laptop"); found {
t.Fatal("the condition stayed after the vault answered")
}
}
func TestACoreModuleFirstPlacedWhileTheVaultIsSilentIsSealedByTheController(t *testing.T) {
open, vault := aCoreMesh(t, link.VaultBoxContract)
vault.silent = errors.New("no responders")
plan, settings, err := planFor(t.Context(), open, "laptop")
if err != nil {
t.Fatal(err)
}
declared, err := declarationFor(t.Context(), open, "laptop", plan, settings)
if err != nil {
t.Fatal(err)
}
f := clientsBroker(declared.Resources)
if f == nil || f["sealed"] == nil || f[catalogue.VaultSealedField] != nil {
t.Fatalf("the core credential is composed as %v: the controller's own seal, until the vault holds it", f)
}
if why, left := declared.leftOutWhy["mesh-provisioner-client"]; left {
t.Fatalf("the core module was left out: %v", why)
}
c, found := vaultSilence(t, "laptop")
if !found || !strings.Contains(c.Summary, "sealed by the controller until the vault holds it") {
t.Fatalf("the condition does not say why: %+v", c)
}
}
func TestAnEngineBelowTheContractIsSentTheControllersSealAsBefore(t *testing.T) {
open, vault := aCoreMesh(t, link.HandedContract)
f := clientsBroker(sentTo(t, open, "laptop"))
if f == nil || f["sealed"] == nil || f[catalogue.VaultSealedField] != nil {
t.Fatalf("an older engine is sent %v", f)
}
if len(vault.asked) != 0 {
t.Fatalf("the vault was asked %v for an older engine", vault.asked)
}
}
+131
View File
@@ -0,0 +1,131 @@
package main
import (
"context"
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// The core's own credentials travel sealed by the vault to their machine (novox/hq ADR 0349 §16, issue 755; the lead's
// decision, issue 755 notes 4 and 5): to an engine that opens the vault's box, the send carries the box the vault
// answered the controller and never the controller's own seal; when the vault does not answer, a kept box is carried
// again and a module with none is sealed by the controller (ADR 0396, issue 908), with a condition naming the vault and the machine. An engine below the
// contract composes exactly as before, and the vault is not asked.
// coreStandIn answers `get` with a box per name and generation, or its silence.
type coreStandIn struct {
asked []string
keys []string
generation int
silent error
}
func (v *coreStandIn) ask(name, key string) (coreAnswer, error) {
v.asked, v.keys = append(v.asked, name), append(v.keys, key)
if v.silent != nil {
return coreAnswer{}, v.silent
}
box := `{"v":1,"eph":"ZQ==","iv":"aQ==","tag":"dA==","ct":"` + strings.Repeat("c", len(v.asked)) + `"}`
return coreAnswer{Name: name, Generation: v.generation, Sealed: json.RawMessage(box)}, nil
}
func withVault(t *testing.T, v *coreStandIn) {
t.Helper()
before := coreVault
coreVault = func(context.Context) coreAsk { return v.ask }
t.Cleanup(func() { coreVault = before })
}
func clientsBroker(resources []map[string]any) map[string]any {
for _, r := range resources {
if r["id"] == "mesh-provisioner-client.needs-"+broker.ProvisionerSecret {
return r
}
}
return nil
}
func vaultSilence(t *testing.T, node string) (conditions.Condition, bool) {
t.Helper()
c, found, err := conditionsFrom.Get(t.Context(), vaultSilentObservation(node, coreBoxes{}).Key())
if err != nil {
t.Fatal(err)
}
return c, found
}
// Issue 908: a machine at the vault-box contract, the vault silent, no kept box: nats is composed with the
// controller's own seal, never left out, and the vault-silent condition names it.
func TestReplay908(t *testing.T) {
key := "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
core := map[string][]string{"nats": {"password"}}
ask := func(name, publicKey string) (coreAnswer, error) { return coreAnswer{}, errors.New("no responders") }
got := coreBoxesOf("novox", key, core, nil, Allocating, ask)
if _, sealed := got.sealed["nats"]; sealed || len(got.notPlaced) != 0 {
t.Fatalf("nats is vault-sealed %v or not placed %v", got.sealed, got.notPlaced)
}
if !reflect.DeepEqual(got.byController, []string{"nats"}) {
t.Fatalf("nats is not left to the controller's seal: %+v", got)
}
o := vaultSilentObservation("novox", got)
if o.Severity != conditions.Warning || !strings.Contains(o.Summary, "sealed by the controller until the vault holds it: nats") {
t.Fatalf("the condition does not name nats: %+v", o)
}
// A kept box is still carried while the vault is silent.
kept := map[string]map[string]inventory.CoreBox{"nats": {"password": {SealedTo: key, Box: json.RawMessage(`{"v":1}`)}}}
got = coreBoxesOf("novox", key, core, kept, Allocating, ask)
if string(got.sealed["nats"]["password"]) != `{"v":1}` || len(got.byController) != 0 {
t.Fatalf("the kept box was not held: %+v", got)
}
}
// An engine below VaultBoxContract is sent the controller's own seal, as before, and the vault is not asked.
// Every question but a send reads the kept box and asks nothing; with none kept, a stand-in the next send asks for.
func TestOnlyASendAsksTheVault(t *testing.T) {
core := map[string][]string{"nats": {"broker"}}
key := aPublicKey(t)
kept := map[string]map[string]inventory.CoreBox{"nats": {"broker": {Module: "nats", Name: "broker",
Generation: 1, SealedTo: key, Box: json.RawMessage(`{"v":1}`)}}}
never := func(string, string) (coreAnswer, error) {
t.Fatal("the vault was asked by a question that is not a send")
return coreAnswer{}, nil
}
for _, choosing := range []Choosing{Reading, Foreseeing} {
if got := coreBoxesOf("anchor", key, core, kept, choosing, never); string(got.sealed["nats"]["broker"]) != `{"v":1}` {
t.Errorf("read %s", got.sealed["nats"]["broker"])
}
if got := coreBoxesOf("anchor", key, core, nil, choosing, never); string(got.sealed["nats"]["broker"]) !=
string(catalogue.ForeseenVaultBox) {
t.Errorf("with none kept, read %s", got.sealed["nats"]["broker"])
}
}
// A box kept for another key of the machine is never carried.
other := coreBoxesOf("anchor", aPublicKey(t), core, kept, Allocating,
func(string, string) (coreAnswer, error) { return coreAnswer{}, errors.New("silent") })
if _, vaultSealed := other.sealed["nats"]; vaultSealed || !reflect.DeepEqual(other.byController, []string{"nats"}) {
t.Fatalf("a box sealed to the machine's old key was carried: %+v", other)
}
}
// The core's modules on a machine: on the list, not handed, with their own secrets but one issued outside.
func TestTheCoresModulesAreTheListsWithTheirOwnSecrets(t *testing.T) {
modules := []catalogue.Manifest{
{Module: "nats", OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/a"}, "given": {Path: "/b",
IssuedBy: catalogue.IssuedOutside}}},
{Module: "postgres", OwnSecrets: catalogue.OwnSecrets{"superuser": {Path: "/c"}}},
{Module: "mesh-provisioner", OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/d"}}},
}
got := coreModulesOf(modules, func(m catalogue.Manifest) bool { return m.Module == "mesh-provisioner" })
if want := map[string][]string{"nats": {"broker"}}; !reflect.DeepEqual(got, want) {
t.Fatalf("got %v, want %v", got, want)
}
}
+215
View File
@@ -0,0 +1,215 @@
package main
import (
"context"
"fmt"
"sort"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/lease"
)
// **No send but a person's `push <node>` and a gate's own put-back moves the mesh's own core to a build older
// than the one its machine was last sent** (novox/hq issue 201). Every send made through sendToEach keeps it — a
// plan's, a release's, a healer's, a rotation's, and the bus step too, which is a person's act but composes under
// sendKeeps like the rest. On 2026-10-02 two merges a minute apart made two plans;
// the controller's plan rolled the new controller, which widened its seat's row, and the node-engine's plan then
// sent the control node the controller build before it. The older binary read the row it could not serve and the
// mesh lost its voice for ten minutes — the thing that would have corrected it was the thing that was down. ADR
// 0185 took the outage out of it (a controller behind its row serves what it can); the race is closed here,
// without needing its trigger: whatever made the mesh's record older, the send keeps the build.
//
// **Kept, not refused.** The send composes the controller or the node-engine at the build the machine was last
// sent, as a recorded module is kept (ADR 0245), and carries everything else: no other module, secret, healer or
// walk waits on it, and nothing is retried. The kept move is no move for a gate either: ungatedIn and gatedSend
// leave it out (withoutKept), so a gate judges the other moves of its send. A plan whose own build is the older one finds its gate superseded by
// the build kept there (issue 352) and ends. The mesh's record stays older, and a warning condition says so until
// a send carries the record — a person's `push <node>` accepts the older build — or the record moves forward: a
// newer merge, or `rebuild mesh-controller`.
//
// Older is by the builds' registration order — when each was asked (novox/hq issue 219) — never by the commits'
// dates. So a retry or a rebuild of an old commit, asked anew, counts as newer: it is registered as the module's
// current build by the same order, and the guard does not stop that move. A build that failed its gate is never
// ahead: the machine was sent it and the record was put back before it (gateFailed), and if the put-back's own
// send was refused, the next send completes it rather than keeping the failed build there. A gate's own put-back
// carries its modules back whatever the order (sendScope.puttingBack); the controller's is held to the store's
// schema already (issue 352).
// kindCoreDowngrade is a core module kept at the build its machine was last sent, over an older record.
const kindCoreDowngrade = "core-downgrade-kept"
// coreDowngradeSource is the condition's source: the sends, which raise and clear it.
const coreDowngradeSource = "sends"
// downgradeGuarded is the core components a send may not move backwards: the controller, which reads the
// store and its seat's row, and the node-engine, which reads what the controller declares.
var downgradeGuarded = []string{lease.ComponentController, lease.ComponentEngine}
// askedBuild is what the order needs of one build of a module: when it was asked, and whether it failed its gate.
type askedBuild struct {
asked time.Time
failedGate bool
}
// buildAsked is the newest build of a module from a commit that worked — when it was asked (or recorded, when
// the request time is not known), and whether it failed its gate — and false when the records hold none. A
// variable, so a test reads its own order.
var buildAsked = func(ctx context.Context, inv *inventory.Inventory, module, commit string) (askedBuild, bool, error) {
builds, err := inv.Builds(ctx, module, askedLookBack)
if err != nil {
return askedBuild{}, false, err
}
for _, b := range builds {
if b.Worked() && sameCommit(b.Commit, commit) {
failed, err := inv.GateFailed(ctx, b.ID)
if err != nil {
return askedBuild{}, false, err
}
return askedBuild{asked: b.AskedOrAt(), failedGate: failed}, true, nil
}
}
return askedBuild{}, false, nil
}
// askedLookBack is how many of a module's newest builds are read for when a build was asked.
const askedLookBack = 50
// coreDowngrade is one core module a machine was last sent a build of that is newer than the mesh's record.
type coreDowngrade struct {
module, node, component string
// sent is the build the machine was last sent, record the one the mesh holds.
sent, record string
sentAsked, recordAsked time.Time
}
func (d coreDowngrade) String() string {
return fmt.Sprintf("%s was last sent %s %s, and the mesh holds the older %s (asked %s, before %s): a send there "+
"keeps %s; `push %s` sends the older build, and a newer merge or `rebuild %s` moves the record forward "+
"(novox/hq issue 201)", d.node, d.module, short(d.sent), short(d.record),
d.recordAsked.UTC().Format(time.RFC3339), d.sentAsked.UTC().Format(time.RFC3339), short(d.sent), d.node,
d.module)
}
// coreDowngrades is, for one machine, each guarded core module whose record is older than the build the machine
// was last sent, leaving out what a gate's own put-back puts back and a sent build that failed its gate. A move
// whose either build the records do not hold is not proved older and is left to the gate that judges it. Pure
// but for asked.
func coreDowngrades(node string, sent map[string]string, record map[string]string, scope sendScope,
asked func(module, commit string) (askedBuild, bool, error)) ([]coreDowngrade, error) {
var out []coreDowngrade
for module, was := range sent {
component := coreComponent(module)
now := record[module]
if !guarded(component) || was == "" || now == "" || sameCommit(was, now) {
continue
}
if scope.puttingBack && scope.modules[module] {
continue
}
from, known, err := asked(module, was)
if err != nil {
return nil, fmt.Errorf("when %s's build %s was asked cannot be read: %w", module, short(was), err)
}
if !known || from.failedGate {
continue
}
to, known, err := asked(module, now)
if err != nil {
return nil, fmt.Errorf("when %s's build %s was asked cannot be read: %w", module, short(now), err)
}
if known && to.asked.Before(from.asked) {
out = append(out, coreDowngrade{module: module, node: node, component: component, sent: was, record: now,
sentAsked: from.asked, recordAsked: to.asked})
}
}
sort.Slice(out, func(i, j int) bool { return out[i].module < out[j].module })
return out, nil
}
func guarded(component string) bool {
for _, c := range downgradeGuarded {
if c == component {
return true
}
}
return false
}
// coreKept is, for a send that keeps (sendKeeps), each guarded core module of a machine to keep at the build it
// was last sent, module → commit, each said and raised as a condition.
func coreKept(ctx context.Context, inv *inventory.Inventory, node string, sent map[string]string,
current map[string]inventory.CurrentBuild) (map[string]string, error) {
record := map[string]string{}
for m, c := range current {
record[m] = c.Commit
}
found, err := coreDowngrades(node, sent, record, scopeOf(ctx), func(module, commit string) (askedBuild, bool, error) {
return buildAsked(ctx, inv, module, commit)
})
if err != nil || len(found) == 0 {
return nil, err
}
out := map[string]string{}
for _, d := range found {
out[d.module] = d.sent
sayCoreDowngrade(ctx, d)
}
return out, nil
}
// coreDowngradeObservation is a kept core build as a condition: `core.<component>.<machine>.<kind>`.
func coreDowngradeObservation(component, node, summary string) conditions.Observation {
return conditions.Observation{Scope: conditions.ScopeCore, ID: component + "." + node, Kind: kindCoreDowngrade,
Token: kindCoreDowngrade, Machine: node, Severity: conditions.Warning, Source: coreDowngradeSource,
Summary: summary}
}
func sayCoreDowngrade(ctx context.Context, d coreDowngrade) {
fmt.Printf("KEPT: %s\n", d)
keeper := conditionsFrom
if keeper == nil {
return
}
if _, err := keeper.Observe(ctx, coreDowngradeObservation(d.component, d.node, d.String())); err != nil {
fmt.Printf("the condition for %s kept on %s could not be raised: %v\n", d.module, d.node, err)
}
}
// settleCoreDowngrades clears, after a send to a machine is away, the condition of each guarded core module the
// send carried at the mesh's record — a person's push that accepted the older build, or a send after the record
// moved forward — or did not carry at all. A send that kept the newer build leaves it open.
func settleCoreDowngrades(ctx context.Context, inv *inventory.Inventory, node string, carried map[string]string) {
keeper := conditionsFrom
if keeper == nil || carried == nil {
return
}
var open []string
for _, c := range downgradeGuarded {
if _, found, err := keeper.Get(ctx, coreDowngradeObservation(c, node, "").Key()); err == nil && found {
open = append(open, c)
}
}
if len(open) == 0 {
return
}
current, err := inv.CurrentBuilds(ctx)
if err != nil {
fmt.Printf("whether %s still keeps a core build over the mesh's record cannot be read: %v\n", node, err)
return
}
for module, now := range current {
component := coreComponent(module)
if !guarded(component) {
continue
}
if was, has := carried[module]; has && !sameCommit(was, now.Commit) {
continue
}
key := coreDowngradeObservation(component, node, "").Key()
if _, err := keeper.Clear(ctx, key, node+" was sent the "+component+" the mesh holds"); err != nil {
fmt.Printf("the condition %s could not be cleared: %v\n", key, err)
}
}
}
@@ -1,3 +1,5 @@
//go:build integration
package main
import (
@@ -0,0 +1,148 @@
//go:build integration
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/testbus"
)
func TestAMovedPathIsNoShrinkAndAShrinkThereStillIs(t *testing.T) {
inv := inventory.ForTest(t)
ctx := t.Context()
shelf := shelfFor(t, houseManifest)
declared := []inventory.DeclaredData{{Module: "house", Item: "config", Class: "irreplaceable", Owned: true}}
start := time.Now().Add(-6 * time.Hour)
measure := func(at time.Time, path string, size int64) []conditions.Observation {
t.Helper()
if _, err := inv.RecordData(ctx, "home", declared, map[string]map[string]inventory.Measurement{"house": {
"config": {Path: path, Size: bytesOf(size), MeasuredAt: when(at), LastWrite: when(at),
LastBackup: when(at)}}}, "", at); err != nil {
t.Fatal(err)
}
records, err := inv.Data(ctx)
if err != nil {
t.Fatal(err)
}
peaks, err := inv.DataPeaks(ctx, at.Add(-shrinkWindow))
if err != nil {
t.Fatal(err)
}
return dataFindings(records, peaks, shelf, nil, nil, at)
}
measure(start, "/home/operator/.claude", 94_700_000)
if got := findingsByKind(measure(start.Add(10*time.Minute), "/home/agent/.claude", 490)); got[kindDataShrank].Kind != "" {
t.Fatalf("a moved path raised a shrink: %+v", got[kindDataShrank])
}
measure(start.Add(2*time.Hour), "/home/agent/.claude", 300<<20)
got := findingsByKind(measure(start.Add(4*time.Hour), "/home/agent/.claude", 1<<20))[kindDataShrank]
if got.Severity != conditions.Urgent || !strings.Contains(got.Summary, "shrank") {
t.Fatalf("a genuine shrink at the new path was not raised: %+v", got)
}
}
func TestNatsUnassigningIrreplaceableDataRetiresItAndAnEmptyReplacementIsUrgent(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if _, err := inv.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "keeper", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.BackupSeat, Scope: catalogue.ScopeNode, Serves: []string{"backed-up", "now", "restore"}}}})
house, err := catalogue.ParseManifest([]byte(houseManifest))
if err != nil {
t.Fatal(err)
}
register(t, open, house)
if _, err := assign(ctx, open, "laptop", "house"); err == nil {
t.Fatal("irreplaceable data was assigned to a machine with nothing to back it up")
}
for _, node := range []string{"laptop", "anchor"} {
if _, err := assign(ctx, open, node, "keeper"); err != nil {
t.Fatal(err)
}
}
if _, err := assign(ctx, open, "laptop", "house"); err != nil {
t.Fatal(err)
}
// Sent, and applied: a holder is asked only once it has been (novox/hq issue 275).
for _, node := range []string{"laptop", "anchor"} {
pushedAndApplied(t, open, node)
}
conn := onATestBus(t)
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
laptop, anchor := &fakeHolder{}, &fakeHolder{}
laptop.serve(t, conn, "laptop")
anchor.serve(t, conn, "anchor")
now := time.Now()
heard := &watchdogs{last: &signalFacts{now: now, machines: []machineFacts{
{name: "laptop", lastHeard: now}, {name: "anchor", lastHeard: now}}}}
d := &doctor{open: open, js: js, watchdogs: heard}
var d13 probe
for _, p := range probeRegistry {
if p.ID == probeDataID {
d13 = p
}
}
run := func() []conditions.Observation {
t.Helper()
probing := context.WithValue(ctx, probeAsksKey{}, d13)
obs, err := probeData(probing, d)
if err != nil {
t.Fatal(err)
}
return obs
}
laptop.set(measuredHouse("/var/lib/house/config", 2<<30, now))
if obs := run(); len(obs) != 0 {
t.Fatalf("a measured, backed-up item raised %+v", obs)
}
r, err := inv.DataOf(ctx, "laptop", "house", "config")
if err != nil || r.Path != "/var/lib/house/config" || r.Size == nil || *r.Size != 2<<30 || r.LastBackup == nil {
t.Fatalf("what the holder measured was not kept: %+v, %v", r, err)
}
said, err := unassign(ctx, open, "laptop", "house")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(said, "house's config (/var/lib/house/config, 2.0 GB) stays where it is") {
t.Fatalf("the unassignment does not say the data stays:\n%s", said)
}
laptop.set()
run()
r, err = inv.DataOf(ctx, "laptop", "house", "config")
if err != nil || !r.Retired() || r.Path != "/var/lib/house/config" {
t.Fatalf("unassigned, the irreplaceable item is not kept retired: %+v, %v", r, err)
}
records, _ := inv.Data(ctx)
if rows := retiredData(records, time.Now()); len(rows) != 1 || rows[0].Path != "/var/lib/house/config" {
t.Fatalf("cleanup list: %+v", rows)
}
// Assigned on the anchor, onto an empty directory.
if _, err := assign(ctx, open, "anchor", "house"); err != nil {
t.Fatal(err)
}
anchor.set(measuredHouse("/var/lib/house/config", 300<<10, time.Now()))
obs := findingsByKind(run())
o, ok := obs[kindEmptyReplacement]
if !ok || o.Severity != conditions.Urgent || o.Machine != "anchor" || o.Also[0] != "laptop" {
t.Fatalf("an empty replacement of a module's data was not urgent: %+v", obs)
}
}
+5 -101
View File
@@ -1,7 +1,6 @@
package main
import (
"context"
"encoding/json"
"strings"
"sync"
@@ -10,14 +9,11 @@ import (
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/testbus"
)
func bytesOf(n int64) *int64 { return &n }
@@ -169,6 +165,11 @@ func TestAShrinkOfMoreThanHalfIsUrgent(t *testing.T) {
}
}
// THE FALSE ALARM (issue 368), replayed through the store D13 reads: an agent's home moved from the
// operator's own home (94.7 MB) to the agent account's fresh one (490 B), and `data-shrank` was raised
// for data that was never lost. A moved item is read against its new path only, so nothing is raised —
// and a genuine shrink at the new path, a week of history later, still is.
// Data said to be written all the time and not written; data with no backup or an old one — urgent when
// irreplaceable, a warning when valuable; and a new item given its bound before it is said.
func TestQuietDataAndMissingBackupsAreSaidByClass(t *testing.T) {
@@ -275,103 +276,6 @@ func measuredHouse(path string, size int64, at time.Time) map[string]any {
// directory is an empty replacement — through the real stores and a real bus. The unassignment says the
// data stays; the self-check's next run retires it (kept, listed by cleanup), and when the module comes
// up on another machine with an empty directory while the full one waits retired, that is urgent.
func TestNatsUnassigningIrreplaceableDataRetiresItAndAnEmptyReplacementIsUrgent(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if _, err := inv.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "keeper", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.BackupSeat, Scope: catalogue.ScopeNode, Serves: []string{"backed-up", "now", "restore"}}}})
house, err := catalogue.ParseManifest([]byte(houseManifest))
if err != nil {
t.Fatal(err)
}
register(t, open, house)
if _, err := assign(ctx, open, "laptop", "house"); err == nil {
t.Fatal("irreplaceable data was assigned to a machine with nothing to back it up")
}
for _, node := range []string{"laptop", "anchor"} {
if _, err := assign(ctx, open, node, "keeper"); err != nil {
t.Fatal(err)
}
}
if _, err := assign(ctx, open, "laptop", "house"); err != nil {
t.Fatal(err)
}
// Sent, and applied: a holder is asked only once it has been (novox/hq issue 275).
for _, node := range []string{"laptop", "anchor"} {
pushedAndApplied(t, open, node)
}
conn := onATestBus(t)
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
laptop, anchor := &fakeHolder{}, &fakeHolder{}
laptop.serve(t, conn, "laptop")
anchor.serve(t, conn, "anchor")
now := time.Now()
heard := &watchdogs{last: &signalFacts{now: now, machines: []machineFacts{
{name: "laptop", lastHeard: now}, {name: "anchor", lastHeard: now}}}}
d := &doctor{open: open, js: js, watchdogs: heard}
var d13 probe
for _, p := range probeRegistry {
if p.ID == probeDataID {
d13 = p
}
}
run := func() []conditions.Observation {
t.Helper()
probing := context.WithValue(ctx, probeAsksKey{}, d13)
obs, err := probeData(probing, d)
if err != nil {
t.Fatal(err)
}
return obs
}
laptop.set(measuredHouse("/var/lib/house/config", 2<<30, now))
if obs := run(); len(obs) != 0 {
t.Fatalf("a measured, backed-up item raised %+v", obs)
}
r, err := inv.DataOf(ctx, "laptop", "house", "config")
if err != nil || r.Path != "/var/lib/house/config" || r.Size == nil || *r.Size != 2<<30 || r.LastBackup == nil {
t.Fatalf("what the holder measured was not kept: %+v, %v", r, err)
}
said, err := unassign(ctx, open, "laptop", "house")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(said, "house's config (/var/lib/house/config, 2.0 GB) stays where it is") {
t.Fatalf("the unassignment does not say the data stays:\n%s", said)
}
laptop.set()
run()
r, err = inv.DataOf(ctx, "laptop", "house", "config")
if err != nil || !r.Retired() || r.Path != "/var/lib/house/config" {
t.Fatalf("unassigned, the irreplaceable item is not kept retired: %+v, %v", r, err)
}
records, _ := inv.Data(ctx)
if rows := retiredData(records, time.Now()); len(rows) != 1 || rows[0].Path != "/var/lib/house/config" {
t.Fatalf("cleanup list: %+v", rows)
}
// Assigned on the anchor, onto an empty directory.
if _, err := assign(ctx, open, "anchor", "house"); err != nil {
t.Fatal(err)
}
anchor.set(measuredHouse("/var/lib/house/config", 300<<10, time.Now()))
obs := findingsByKind(run())
o, ok := obs[kindEmptyReplacement]
if !ok || o.Severity != conditions.Urgent || o.Machine != "anchor" || o.Also[0] != "laptop" {
t.Fatalf("an empty replacement of a module's data was not urgent: %+v", obs)
}
}
// Data on redundant storage: the array it is on is watched, one condition per array as loud as the most
// precious item on it; an item said to be on redundancy and found on plain storage is said; an item
+5 -1
View File
@@ -144,11 +144,15 @@ func actOnDeadLetter(ctx context.Context, on *busHandles, act string, id uint64,
}
switch act {
case "deliver":
_, to, err := link.DeliverAgain(on.js, id)
delivered, to, err := link.DeliverAgain(on.js, id)
if err != nil {
return nil, err
}
answer["delivered_on"] = to
if delivered.Original != "" {
// What became of the ask in its seat's queue (novox/hq issue 334): removed, or left, and why.
answer["original"] = delivered.Original
}
answer["done"] = fmt.Sprintf("dead letter %d was delivered again to %s, and nobody else; it is no longer kept",
id, consumerWho(d.Stream, d.Consumer))
case "drop":
+117
View File
@@ -0,0 +1,117 @@
package main
import (
"context"
"fmt"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// A dead letter from a restricted stream ages out with its stream (novox/hq ADR 0376 §3, §6).
//
// DEAD_LETTERS lets such a message go when its stream's maximum age runs out, so a consumer's `max-deliveries`
// condition can clear with nobody having delivered it again or dropped it. That is said: the watchdogs remember,
// from one look to the next, when each restricted dead letter ages out, count those gone once their time passed,
// say each in the controller's log as it goes, and clear the condition with words saying how many aged out
// undelivered and from which stream.
// agingTally is what the watchdogs remember of restricted dead letters between looks.
type agingTally struct {
// seen is each consumer's restricted dead letters at the last look, by id, with when each ages out.
seen map[string]map[uint64]time.Time
// agedOut is how many of each consumer's aged out undelivered while its condition was open.
agedOut map[string]int
}
// look compares this look's restricted dead letters with the last: newly is how many of each consumer's aged out
// since, and cleared how many aged out in all for each consumer DEAD_LETTERS no longer holds anything of, when any
// did. held is every consumer's count this look; aging its restricted dead letters.
func (t *agingTally) look(now time.Time, held map[string]int, aging map[string][]link.Aging) (newly, cleared map[string]int) {
if t.seen == nil {
t.seen, t.agedOut = map[string]map[uint64]time.Time{}, map[string]int{}
}
newly, cleared = map[string]int{}, map[string]int{}
for key, before := range t.seen {
still := map[uint64]bool{}
for _, a := range aging[key] {
still[a.ID] = true
}
for id, agesOut := range before {
if !still[id] && !agesOut.After(now) {
newly[key]++
}
}
if newly[key] == 0 {
delete(newly, key)
}
t.agedOut[key] += newly[key]
}
for key, n := range t.agedOut {
if held[key] > 0 {
continue
}
if n > 0 {
cleared[key] = n
}
delete(t.agedOut, key)
}
t.seen = map[string]map[uint64]time.Time{}
for key, letters := range aging {
t.seen[key] = map[uint64]time.Time{}
for _, a := range letters {
t.seen[key][a.ID] = a.AgesOut
}
}
return newly, cleared
}
// agedOutResolved is the line a consumer's condition clears with when messages it gave up on aged out undelivered.
func agedOutResolved(key string, n int) string {
stream, _, _ := strings.Cut(key, ".")
messages := "a message"
if n > 1 {
messages = fmt.Sprintf("%d messages", n)
}
return fmt.Sprintf("Resolved: %s of the stream %s aged out undelivered", messages, stream)
}
// sayAgedOut says, in the log, what aged out since the last look, and clears the condition of every consumer whose
// dead letters are all gone with the words saying how many aged out. Before the rows are reconciled, so the
// condition is cleared with these words and not its own.
func (w *watchdogs) sayAgedOut(ctx context.Context, f *signalFacts) []string {
if f.advisoriesErr != nil || f.deadLettersAging == nil {
return nil
}
newly, cleared := w.aging.look(f.now, f.deadLetters, f.deadLettersAging)
var problems []string
for _, key := range countedKeys(newly) {
stream, consumer, _ := strings.Cut(key, ".")
fmt.Printf("%d message(s) %s gave up on aged out of %s undelivered: %s keeps them only for its maximum age "+
"(novox/hq ADR 0376)\n", newly[key], link.ConsumerInWords(stream, consumer), "DEAD_LETTERS", stream)
}
for _, key := range countedKeys(cleared) {
stream, consumer, _ := strings.Cut(key, ".")
fmt.Printf("%s no longer has a message kept: %d aged out of %s undelivered, so its condition clears "+
"(novox/hq ADR 0376)\n", link.ConsumerInWords(stream, consumer), cleared[key], stream)
if _, err := w.keeper.ClearSaying(ctx, conditions.Key(conditions.ScopeBus, key, link.AdvisoryMaxDeliveries),
fmt.Sprintf("its %d dead letter(s) of %s aged out undelivered", cleared[key], stream),
agedOutResolved(key, cleared[key])); err != nil {
problems = append(problems, err.Error())
}
}
return problems
}
// countedKeys is a count's keys in order.
func countedKeys(m map[string]int) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
@@ -0,0 +1,69 @@
package main
import (
"context"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// **A dead letter that ages out clears its condition and says so** (novox/hq ADR 0376 §6), on a fake clock: while
// DEAD_LETTERS holds a restricted stream's messages for a consumer its max-deliveries condition is open; once they
// have aged out it clears, with words saying how many aged out undelivered and from which stream. One dropped before
// its age is not counted as aged out, and a consumer of EVENTS clears as before.
func TestADeadLetterThatAgesOutClearsItsConditionAndSaysSo(t *testing.T) {
now := time.Date(2026, 10, 15, 12, 0, 0, 0, time.UTC)
store := conditions.NewInMemory()
told := &conditions.Told{}
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store, Teller: told,
Now: func() time.Time { return now }})
w := &watchdogs{keeper: k, started: now.Add(-time.Hour)}
look := func(at time.Time, held map[string]int, aging map[string][]link.Aging) {
f := calm(at)
f.deadLetters, f.deadLettersAging = held, aging
w.see(t.Context(), f)
}
mail := "MAIL.ace_mailer"
look(now, map[string]int{mail: 3, "EVENTS.media_sonarr": 1}, map[string][]link.Aging{mail: {
{ID: 1, AgesOut: now.Add(time.Hour)}, {ID: 2, AgesOut: now.Add(2 * time.Hour)}, {ID: 3, AgesOut: now.Add(3 * time.Hour)}}})
if open, _ := k.Open(t.Context()); len(open) != 2 {
t.Fatalf("open: %+v", open)
}
// One dropped by a person before its age: not aged out.
look(now.Add(30*time.Minute), map[string]int{mail: 2, "EVENTS.media_sonarr": 1}, map[string][]link.Aging{mail: {
{ID: 2, AgesOut: now.Add(2 * time.Hour)}, {ID: 3, AgesOut: now.Add(3 * time.Hour)}}})
look(now.Add(150*time.Minute), map[string]int{mail: 1}, map[string][]link.Aging{mail: {{ID: 3, AgesOut: now.Add(3 * time.Hour)}}})
open, _ := k.Open(t.Context())
if len(open) != 1 || open[0].Key != conditions.Key(conditions.ScopeBus, mail, link.AdvisoryMaxDeliveries) {
t.Fatalf("after one aged out: %+v", open)
}
look(now.Add(4*time.Hour), map[string]int{}, map[string][]link.Aging{})
if open, _ := k.Open(t.Context()); len(open) != 0 {
t.Fatalf("after all aged out: %+v", open)
}
k.Close(context.Background()) // everything said is said once it closes
var mailCleared, sonarrCleared string
for _, e := range told.Said() {
if e.Change != conditions.ChangeCleared {
continue
}
switch e.Condition.Key {
case conditions.Key(conditions.ScopeBus, mail, link.AdvisoryMaxDeliveries):
mailCleared = e.Condition.Resolved
case conditions.Key(conditions.ScopeBus, "EVENTS.media_sonarr", link.AdvisoryMaxDeliveries):
sonarrCleared = e.Condition.Resolved
}
}
if mailCleared != "Resolved: 2 messages of the stream MAIL aged out undelivered" {
t.Fatalf("MAIL's condition cleared with %q", mailCleared)
}
if sonarrCleared != "Resolved: the messages it gave up on were delivered again or dropped" {
t.Fatalf("an EVENTS consumer's condition cleared with %q", sonarrCleared)
}
if why, ok := conditions.Plain(mailCleared); !ok {
t.Fatalf("not plain: %s", why)
}
}
@@ -0,0 +1,88 @@
//go:build integration
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
func servingDeadLetters(t *testing.T) *broker.JetStream {
t.Helper()
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
if err := broker.AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
if err := js.EnsureControllerBuckets(); err != nil {
t.Fatal(err)
}
before := servingBus.Load()
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(before) })
return js
}
func TestDeadLettersListsDropsAndRecordsWhy(t *testing.T) {
js := servingDeadLetters(t)
if _, err := js.Context().Publish("mesh.mod.gitea.event.pull.merged", []byte(`{"n":1}`)); err != nil {
t.Fatal(err)
}
d, err := link.KeepDeadLetter(js.Context(), []byte(`{"stream":"EVENTS","consumer":"media_sonarr","stream_seq":1,"deliveries":5}`))
if err != nil {
t.Fatal(err)
}
answer, err := askDeadLetters(t, map[string]any{})
if err != nil {
t.Fatal(err)
}
listed := answer.(map[string]any)
if listed["held"] != 1 || len(listed["dead_letters"].([]link.DeadLetter)) != 1 {
t.Fatalf("listed %v", listed)
}
// Refused before anything is done: an act without why, an argument the shape passes over, both acts.
for _, args := range []map[string]any{
{"drop": "1"},
{"drop": "1", "why": "x", "limit": "3"},
{"drop": "1", "deliver": "1", "why": "x"},
{"why": "x"},
{"id": "nought"},
} {
if _, err := askDeadLetters(t, args); err == nil {
t.Errorf("%v was done", args)
}
}
if _, total, _ := link.DeadLetters(js.Context(), "", 0); total != 1 {
t.Fatalf("a refused call changed what is kept: %d left", total)
}
done, err := askDeadLetters(t, map[string]any{"drop": "1", "why": "the media server took the download in by hand"})
if err != nil {
t.Fatal(err)
}
if said := done.(map[string]any); said["recorded"] == nil || !strings.Contains(said["done"].(string), "dropped") {
t.Fatalf("answered %v", said)
}
acts, err := link.HandActs(context.Background(), js.Conn(), time.Now().Add(-time.Minute))
if err != nil || len(acts) != 1 || acts[0].Verb != "dead-letters drop" || acts[0].Cause != causeDeadLetter ||
!strings.Contains(acts[0].Condition, "media_sonarr") {
t.Fatalf("recorded %+v (%v)", acts, err)
}
if !personsDecision(acts[0]) {
t.Error("dropping a dead letter is counted as a repair, so S15 would want a healer for it")
}
if _, err := askDeadLetters(t, map[string]any{"id": "1"}); err == nil {
t.Errorf("dead letter %d is still answered after it was dropped", d.ID)
}
}

Some files were not shown because too many files have changed in this diff Show More