Compare commits

..
676 Commits
Author SHA1 Message Date
mesh-tools 56e89d62f1 Merge pull request 'sdk1: the controller's repository is emptied (ADR 0593 §11)' (#491) from feat/sdk1 into main 2026-11-02 20:42:04 +00:00
jschoubben 5970367173 Empty the controller's repository: its jobs moved to the parts ADR 0590 and 0593 name, so nothing builds here against the new mesh-sdk
mesh/delivery merged: source.main moved to novox/mesh-controller@56e89d62f1cb
mesh/delivery-group group feat/sdk1 merged: every member is merged
mesh/check checking 597036717394 on 56e89d62f1cb
Nothing runs the controller (unassigned, out of the catalog) and the group's check builds every member; the repository is archived after the delivery. Issue 1126.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-02 18:16:50 +01:00
jschoubben 4964ad1037 Remove route-proxy from the controller's repository, which it left for mesh-networking
The group's check composed route-proxy twice, here and in mesh-networking; the move is one delivery group
(issue 1126), so it leaves here in the same group. The tests that read its directory go with it.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-02 18:14:41 +01:00
mesh-tools 5c281ad99b Merge pull request 'An unissued bus account holds back only its own module, never the whole node' (#490) from fix/vault-minted-accounts into main 2026-11-01 08:26:02 +00:00
jschoubben 29dfb28f38 Hold only the module whose bus account nobody issued, never the whole node (issue 203)
mesh/delivery merged: source.main moved to novox/mesh-controller@5c281ad99bcc
mesh/check checking 29dfb28f3899 on 5c281ad99bcc
The 1585 group's refusal turned one unissued account into a refusal of the node's whole
composition: novox's mesh-hand-acts, which the vault had minted but the controller's store
does not hold, stopped every take-in on the machine that runs the bus. The module is now held
out of the composition alone, in issue 203's words, and the rest composes; no placeholder is
sealed. A held module of the former form keeps the node's own composition as last written, so
the node-engine never removes it.

Counting the vault's accounts as issued needs the controller to read mesh-vault.bus-accounts,
which mesh-sdk's named readers give to the grants part alone (ADR 0547 clause 9): not built here.

Controller research piece: none moved; this narrows a refusal in the composition the controller
hands over (blocking-fault recovery).

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 09:12:54 +01:00
mesh-tools 88b16e9be2 Merge pull request 'The controller takes the indexer's verdict instead of judging manifests; route-proxy declares its credential form (ADR 0578, 0579)' (#489) from feat/judge-once into main 2026-11-01 07:54:23 +00:00
jschoubben b564c315d0 Test the controller as one that judges no manifest, and reset the new-module waits each test leaves (ADR 0578)
mesh/delivery merged: source.main moved to novox/mesh-controller@88b16e9be20d
mesh/delivery-group group feat/judge-once merged: every member is merged
mesh/check checking b564c315d012 on dc90b3f5bd13 | novox/mesh-controller#489 in group feat/judge-once
Issue 1585: registration publishes the predefined list and judges nothing against it; a definition naming an
installation is the indexer's judge's to refuse; a word the controller does not read is set aside and the module
registered (0578 §6 changes 0501 §7); a refused verdict holds the module and names it. The process-wide list of
new modules waiting is emptied at each take-in test's start and end, so no test reads another's.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 04:14:44 +01:00
jschoubben 5bd03b3984 Refuse a send that would seal a placeholder for a bus credential nobody issued (issue 203)
mesh-sdk #141 (issue 774) stands an unissued bus account in, in the Foreseeing mode, so a proposal's answer can be
asked before the vault mints it; the controller's send then took the stand-in for an own secret to make and would
seal a random value where the broker credential belongs. The send and the question refuse it now, in the words the
composition used to: module issue, then push.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 04:14:44 +01:00
jschoubben e02fcad739 Pin the group's mesh-sdk head after the gofmt fix
mesh/delivery-group group feat/judge-once rejected: composed together: the checker's verdict at the members' heads (0920e5637ce1): integration: github.com/novo…
mesh/delivery closed: a newer head of the same pull request
mesh/check failed (5 tests): integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestABuildHeardIsRecordedAndRegistered: a definition na…
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:27:07 +01:00
jschoubben 4a92ec6102 Pin mesh-sdk at the head after review round 1
mesh/delivery closed: a newer head of the same pull request
mesh/delivery-group group feat/judge-once checking: 4 of 6 member(s) ready
mesh/check checking 4a92ec610227 on dc90b3f5bd13 | novox/mesh-controller#489 in group feat/judge-once
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:13:17 +01:00
jschoubben 909cc25169 Pin mesh-sdk at the judge-less release head in the controller and route-proxy
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:46:24 +01:00
jschoubben 272d5ebd61 Declare credential form none for the provision, which hands consumers no credential
The manifest judge refuses an offer that says no credential form (issue 1585).

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:43:24 +01:00
jschoubben a6386ff23a Decode manifests in tests and pin the mesh-sdk that no longer holds the judge
Issue 1585, S18/S19: the judge's tests live in mesh-delivery now, so an assertion that only tested judging goes
and every test that builds a manifest decodes it.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:40:39 +01:00
jschoubben b51105908a Stop judging manifests at registration; the controller registers what the indexer accepted
Issue 1585, S18 (ADR 0578 §3): registerJudged no longer re-judges the manifest or its predefined keys, and a
standing build is read leniently; module add decodes and judges nothing. Resolve never re-parsed a manifest,
so a key-key fallback needs no list there; a table row holds that.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:09:13 +01:00
jschoubben 8b4d536a11 Take the indexer's verdict on a manifest instead of judging it in the controller
Issue 1585, S18 (ADR 0578 §3, §6): a done key's manifest is decoded leniently and the verdict read from the
indexer's state; a refused one is held and named, an unread word set aside and named, never a wait. The
module check command, which judged files with no mesh and which nothing calls any more, goes with its tests.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:07:44 +01:00
mesh-tools dc90b3f5bd Merge pull request 'route-proxy dials the machine port its plans now carry (issue 774)' (#488) from fix/route-ports into main 2026-10-31 12:56:54 +00:00
jschoubben 0e9c4c8299 Check route-proxy's composed targets over the live shapes in an existing test's place, and pin the mesh-sdk that keeps unassigned bare ports (issue 774, research 072 row 34)
mesh/delivery merged: source.main moved to novox/mesh-controller@dc90b3f5bd13
mesh/delivery-group group fix/route-ports merged: every member is merged
mesh/check failed: integration: mesh-vault/cmd/mesh-vault.TestTheVaultWholeMintsOnTheBusWhoeverHoldsTheLease: never: the account minted under the cont…
The table covers gitea and grafana moved by the placement, influxdb's bare port, mailu's and
photos' fixed mappings and home-assistant on the host's network; it replaces the loopback test it
subsumes, so route-proxy's test count does not grow.

The controller, NATS-native: no piece moves; route-proxy only.
2026-10-31 13:22:00 +01:00
jschoubben 14d10051e5 Dial a route at the machine port its module's placement assigned, from the mesh-sdk whose plans carry it (issue 774, research 072 row 34)
route-proxy from plans dialled every route at the software's own port: gitea at 127.0.0.1:3000
(published at 20000) and grafana on ace at 3000 (published at 20012), and public sites answered
502. The proxy already reaches a module beside it over loopback and one elsewhere at the plan's
address; the plan now carries the assigned port, and a test composes the plan with the SDK and
reads it as the proxy does, so the two cannot drift again.

The controller, NATS-native: no piece moves; route-proxy only.
2026-10-31 13:11:00 +01:00
mesh-tools c58b311ea7 Merge pull request 'route-proxy from the provisioner's plans with its own contract; the controller forwards accounts to the vault (issue 774)' (#487) from feat/072-moves into main 2026-10-31 10:31:07 +00:00
jschoubben 7a169eae76 Keep the controller on main's mesh-sdk pin and mint until the grants part takes over, so its impact stays small and no account is minted that the bus it loads refuses (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-controller@c58b311ea774
mesh/delivery-group group feat/072-moves merged: every member is merged
mesh/check failed: shanks: the check of novox/mesh-controller#487 failed: go mod download in modules/route-proxy: go: git.novox.be/novox/mesh-controll…
The controller is being removed, so its root module goes back to origin/main's mesh-sdk pin (2da0de7) and
testdata/beside's capture; route-proxy keeps its own pins. Its one functional change stays the account issue:

- Before the grants part's take-over (its key user-list.sealed not saying FollowGrants) the bus loads the
  controller's list, whose hashes come only from its store, so the controller mints as before and hands every
  account it mints, covered or not, to an installed vault at once: no account waits on a node reporting its hash.
- From the take-over it forwards the issue to the vault's issue-account (d791052f) and mints nothing.

And the controller no longer declares or asserts the ASSIGNMENTS stream, which the bus module's tools delete at the
lease write and nothing may make again (research 072 row 33); the verified-sender re-issue stops once it is gone.

Research 072 piece: the gap "vault mints after the lease", the minting piece moves to the vault from the take-over.
2026-10-31 10:37:13 +01:00
jschoubben a07621d761 Pin route-proxy at the group's mesh-sdk head acbf80a and its contract at the published tag v0.1.0, so it builds and reads the contributed payload; refresh its manifest digest for the events its contract names (issue 774) 2026-10-31 10:27:05 +01:00
jschoubben d791052f77 Forward module issue and an assignment's issue to the vault once one is installed, so the controller mints no bus account and none waits on a node reporting here (issue 774; research 072, the controller NATS-native: gap "vault mints after the lease", the minting piece moves to the vault) 2026-10-31 10:02:39 +01:00
jschoubben 22935b9202 Serve route-proxy's routes through its own contract: decode each plan entry's opaque contribution with it, say each name served, withdrawn or refused on the bus, and take who is inside from the seat mesh-placement, since the membership went with the assignments stream (issue 774, research 072 rows 33 and 34) 2026-10-31 09:44:07 +01:00
jschoubben efe745f2be Publish route-proxy's contract as a Go module of its own: the route a consumer asks, its schema, the events the proxy says, what it keeps on the bus and its tools' arguments, so other modules use the contract by its version and the provisioner carries a route unread (issue 774, research 072 row 34) 2026-10-31 09:44:00 +01:00
jschoubben ff912d0972 Serve route-proxy's routes from the seat mesh-provisioner's plans alone and drop its received routes file, since a route is a provision and the file is the deprecated path (issue 774, research 072 row 34; needs the mesh-sdk pin carrying plan entries' given and Conn.Read) 2026-10-31 09:24:53 +01:00
mesh-tools 4c9be8efc8 Merge pull request 'The web front's route helper reads the seat's plans, so a route-only module's route is provided (issue 774)' (#486) from fix/route-proxy-reads-plans into main 2026-10-30 23:43:15 +00:00
jschoubben 62c05c681c Start route-proxy's provider helper only once Serve holds the runtime's channel, so its first watch is never refused and the helper never stops until a restart (issue 774, review)
mesh/delivery merged: source.main moved to novox/mesh-controller@4c9be8efc83a
mesh/check checking 62c05c681cfb on 4c9be8efc83a
2026-10-31 00:32:26 +01:00
jschoubben bfb728730f Answer route-proxy's routes from the seat mesh-provisioner's plans and keys, not the controller's assignments, so a route the seat keys for a module not handed is marked provided and its new builds are no longer withheld (issue 774) 2026-10-31 00:27:11 +01:00
mesh-tools 159db37618 Merge pull request 'The web front keeps what it serves, its certificates and what it refuses on the bus, with tools to read them (issue 774)' (#485) from feat/route-proxy-tools into main 2026-10-30 22:11:52 +00:00
jschoubben b7c3868980 Keep a scan out of route-proxy's unrouted state: only names under a served domain, a few a minute, no visitor address (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-controller@159db37618c2
mesh/check being carried to b7c3868980f3 on 159db37618c2 | from a4426190b0c7
Review of ddae6729: a scanner's random Host or SNI names each became a key, bounded only by the
bucket's lifetime and size, and every failed write was retried at each change and logged each time.

A refused name is now kept by name only under the registrable domain of a name the proxy serves, and
at most five new names a minute; every other refusal is counted in memory and written once a minute
under <node>._other (last minute, since start), which route_proxy_unrouted shows. A refused write of
that state is dropped and logged once until a write succeeds; trimming memory to 4096 names moved
from the request to the writer. The visitor's address is no longer kept: the journal has it.

Quick: drop unrouted keys from strangers by a fixed allow-list. Thorough: bound by the proxy's own
domains, by rate, and by never retrying. Chose thorough: the scan is constant on a public machine.
2026-10-30 23:03:00 +01:00
jschoubben ddae6729c1 Give route-proxy its state on the bus and three tools, so a site's route, certificate and last answer are read from the mesh instead of with curl (issue 774)
The web front kept nothing on the bus: whether grafana or drive was routed, where to, with which
certificate, and what it last answered could only be learned with curl or the container's log.

The proxy now keeps `routes` and `unrouted` (announced readable, keyed <node>.<name>), written on
change by one writer: route set changes, certificates stored by autocert's cache or failing at a
handshake, upstream answers that differ from the last written (or are a minute old), and names
refused in a request or a handshake. The provider helper's bundle serves route_proxy_routes,
route_proxy_route and route_proxy_unrouted from those states.

Quick: log the routes at start and grep the journal. Thorough: the state on the bus and the tools
that read it, mesh-wide, in one call. Chose thorough: the operator's question is mesh-wide and
recurs, and the log says nothing about certificates or answers.

Tests: two table tests added (the ledger, the tools), both failing without the change; four
authority tests and four skipped-route tests merged into two, so the module has four fewer.
2026-10-30 22:58:04 +01:00
jschoubben a4426190b0 Merge pull request 'Accept the credential form none in a provisioner's plan, so design 47's release delivers (issue 774)' (#483) from fix/774-none-form into main
Reviewed-on: #483
2026-10-30 11:23:41 +00:00
jschoubben 2154ea8e3c Accept the credential form none in a provisioner's plan, so the release's catalogue, which states it on every offer without a credential, no longer has every plan refused and every module held (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-controller@a4426190b0c7
mesh/check checking 2154ea8e3c4e on a4426190b0c7
2026-10-30 12:18:36 +01:00
mesh-tools 812c528479 Merge pull request 'Pin the release's mesh-sdk in the controller, so it reads design 47's catalog until its lease is handed away (issue 774)' (#482) from feat/774-l3-reland into main 2026-10-30 11:01:57 +00:00
jschoubben 8b32e04e2f Judge the controller's module check against the catalogue it is released with, captured at the group's catalogue head f3a57dcc, so a group that brings a word the trunk's catalogue lacks does not fail on the trunk (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-controller@812c528479e1
mesh/delivery-group group feat/774-l3-reland merged: every member is merged
mesh/check checking 8b32e04e2f8b on 6fe9efa71219 | novox/mesh-controller#482 in group feat/774-l3-reland
The release's fourth check failed TestModuleCheckPassesTheCatalogue with 27 problems. The checker clones the catalogue
beside a controller's tests at its trunk, even in a group holding a catalogue pull request (mesh-delivery checker
internal/check/beside.go besideFor, ADR 0484 clause 4). The release's sdk refuses an offer stating no credential form,
and the trunk's catalogue states none until the release's catalogue merges.

- Quick fix (not chosen): recapture alone, which the test never reads while MESH_CHECK_BESIDE is set.
- Chosen: the test reads the captured copy, now at the group's catalogue f3a57dcc, through beside.CapturedDir. Every other
  beside test still reads the clone. The check itself is unchanged: every rule, every manifest. Moving CAPTURED to the
  catalogue a change is released with is part of that change.

Run as the checker runs it (catalogue at main, mesh-host at main, mesh-sdk at the pin, mesh-controller's trunk
beside): the whole unit and integration suite passes, 1706 tests, 8 skipped. The test also passes with the catalogue at
f3a57dcc beside, and with nothing beside.
2026-10-30 10:38:29 +01:00
jschoubben 79f44386d2 Merge main (delivery 1, the parse-only sdk) into L3's re-land, and pin mesh-sdk 2da0de7, the release's final head (issue 774)
mesh/delivery-group group feat/774-l3-reland rejected: composed together: the checker's verdict at the members' heads (8bad7efb3e7a): unit: github.com/novox/me…
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: github.com/novox/mesh-controller/cmd/mesh-controller.TestModuleCheckPassesTheCatalogue: the catalogue does not pass its own c…
The conflicts were go.mod, go.sum and testdata/beside/CAPTURED alone, taken from the release. 2da0de7 is 6e4a301's code;
its conformance events are unchanged.
2026-10-30 01:13:41 +01:00
mesh-tools 6fe9efa712 Merge pull request 'Pin the parse-only mesh-sdk in the controller, so it reads design 47's manifests (issue 774)' (#481) from fix/774-new-words-first into main 2026-10-30 00:10:12 +00:00
jschoubben d96542cd25 Pin the parse-only mesh-sdk 78c9ec8 in place of 6e4a301, so main's controller reads the release's new words and means nothing new by them (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-controller@6fe9efa71219
mesh/delivery-group group fix/774-new-words-first merged: every member is merged
mesh/check pass d96542cd2503 on e04846e9fa49 | unit 9174 run, 546 kept, 51 skipped | integration 9952 run, 659 kept, 60 skipped | over bound: unit, in…
78c9ec8 is sdk main 307b623 (tree of main's pin dbcaa29) with one parse-only commit: an offer with no form still means
pair. The fixture changes the full sdk forced are reverted to main's; the sdk's conformance events are captured at
78c9ec8 and unchanged. TestModuleCheckPassesTheCatalogue passes, on the captured copy and on mesh-catalog main 96b129e5
cloned beside. Whole unit and integration suites: 1706 tests, 8 skipped for a registry, a container, a refusing bus or a
lab's run.
2026-10-30 00:42:04 +01:00
jschoubben 80dc6980b7 Pin mesh-sdk 6e4a301 on main's controller, so it reads the release's new manifest words before the release uses them (issue 774)
No code of ours changes; the sdk's conformance events are unchanged and captured at 6e4a301. The fixtures of
identity_bound_test and issued_test say the credential form their rows meant (pair, none), since the sdk no longer reads
a missing form as pair (6b8c694).

NOT GREEN: TestModuleCheckPassesTheCatalogue fails. The sdk refuses every offer that states no credential form, and
main's catalogue states none (27 offers). Recapturing cannot fix it: this pin and main's catalogue cannot both stand.
2026-10-30 00:30:05 +01:00
jschoubben 80df1b6361 Pin mesh-sdk 6e4a301, the release's final sdk (issue 774)
The sdk's conformance events are unchanged at 6e4a301 and captured so. The whole integration suite passes at this head
against a local store and bus: 1706 tests, 8 skipped for a registry, a container, a refusing bus or a lab's run.
2026-10-30 00:20:10 +01:00
jschoubben 816cf96459 Pin mesh-sdk 007b6fd, the release's final sdk, so the controller reads the release's manifests and composes as the group does (issue 774)
The sdk now refuses an offer that says no credential form (6b8c694, ADR 0570). So:
- the catalogue copy in testdata/beside is captured again at the release branch's catalogue 8ae8e88d, whose offers each
  say one, and the sdk's conformance events at 007b6fd, as CAPTURED asks;
- the fixtures of identity_bound_test and issued_test say the form their rows meant: pair for a login of each
  consumer's own, none for a keyless provision. No behaviour of the controller changed.
2026-10-30 00:07:04 +01:00
jschoubben 3f05df5d1a Pin mesh-sdk 2e5cc4f, so the controller reads the release's new manifest words (issue 774)
The controller on dbcaa29 parses manifests strictly and refuses `bus` on an artifact and `max-age-days` on a log, both
in the re-land group's catalogue. The SDK's conformance events are captured again at the pinned commit, as
testdata/beside/CAPTURED asks; they are unchanged.
2026-10-29 22:19:39 +01:00
mesh-tools e04846e9fa Merge pull request 'Pin mesh-sdk dbcaa29, so the controller composes with the group fix/1960-no-hold-spread's SDK (issue 1960)' (#480) from fix/1960-no-hold-spread into main 2026-10-29 15:35:31 +00:00
jschoubben 020c6a99b0 Pin mesh-sdk dbcaa29 and capture its conformance events there, so the controller composes with the group fix/1960-no-hold-spread's SDK (issue 1960)
mesh/delivery merged: source.main moved to novox/mesh-controller@e04846e9fa49
mesh/delivery-group group fix/1960-no-hold-spread merged: every member is merged
mesh/check pass 020c6a99b035 on 5a9766d4a47a | unit 3318 run, 1951 kept, 38 skipped | integration 3994 run, 2125 kept, 38 skipped | over bound: unit, …
2026-10-29 16:18:13 +01:00
mesh-tools 5a9766d4a4 Merge pull request 'Pin mesh-sdk 38dbe1d, so the controller knows the seat mesh-placement (issue 1126, delivery 2 of 2)' (#479) from feat/1126-placement-seat into main 2026-10-28 18:33:05 +00:00
jschoubben f46a08772f Pin mesh-sdk 38dbe1d, the group's head, so the controller knows the seat mesh-placement its holder claims (ADR 0567, issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-controller@5a9766d4a47a
mesh/delivery-group group feat/1126-placement-seat merged: every member is merged
mesh/check failed: integration: meshbuildgo/cmd/mesh-build-go.TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus: the compose ask mvsg85b34d25b4 never ende…
2026-10-28 18:49:02 +01:00
jschoubben 1c2adfdca9 Build with the group's mesh-sdk, so the controller knows the seat mesh-placement its holder claims (ADR 0567, issue 1126) 2026-10-28 18:14:58 +01:00
mesh-tools 0fbb5ef359 Merge pull request 'The controller gives its lease back at the stop signal and says a takeover once, so a controller delivery raises no urgent alarm (issue 906, ADR 0558)' (#478) from fix/906-lease-release into main 2026-10-27 00:07:46 +00:00
jschoubben 8f84731896 Name the epoch given back in the lease's delete marker, so only a holder's own release counts as a handover (issue 906, ADR 0558 §3, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-controller@0fbb5ef35991
mesh/check being carried to 8f847318963c on 0fbb5ef35991 | from 2dafbe60e939
A plain delete marker on the lease key is also written by a command's own
lease and by a take given back over the floor; counted as a handover, it
would record a crashed holder's epoch released and say nothing. Release now
writes the marker itself, compare-and-set on the key's revision, with the
header Mesh-Lease-Released naming its epoch, and the next holder ends only
that epoch as released. A takeover whose predecessor cannot be read is now
raised as an expiry, not only printed.
2026-10-27 00:55:30 +01:00
jschoubben fbaa463a38 Give the controller lease back at the stop signal and say a takeover once, so a controller delivery raises no urgent alarm (issue 906, ADR 0558)
The 2026-10-26 replacements showed the lease was given back, but last: the
shutdown took 10 s, the gate had closed at half the key's age, and the release
recorded the epoch released only through the gate, so the next holder found
it expired and S12 said it, urgent, for an hour.

Now the stop signal gives the key back first and records the epoch released
whenever the delete succeeded; a candidate reads the key's delete marker on
the bus to tell a handover from an expiry; and a real takeover (lost or
expired) is raised and cleared once by the controller that took it, instead
of an hour of S12 observation.

Research 072 piece moved: the lease's takeover read from its own key on the
bus and said as an event (03-domains, mesh-controller row; 04-findings 2,
the watchdogs' age reading).
Quick fix weighed: drop the gate check from the release record only; it
leaves the release last in the shutdown and the hour-long alarm for a real
takeover.
2026-10-27 00:49:13 +01:00
mesh-tools 2dafbe60e9 Merge pull request 'Broker tests read a consumer's pending count through pendingOf wherever they expect a count after a publish (issue 1126)' (#477) from fix/1126-tests-wait-for-consumer-counts into main 2026-10-26 22:57:59 +00:00
jschoubben 8d58afea6c Broker tests read a consumer's pending count through pendingOf, which waits up to 10 s, wherever they expect a count after a publish (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-controller@2dafbe60e939
mesh/check being carried to 8d58afea6cb8 on 2dafbe60e939 | from b611f7375a47
The server counts a consumer's pending after it acknowledges the publish, so a read at once failed the try of
main on a loaded ace (TestAResetOfTheEventsConsumerLeavesTheMergesAndTheirConsumerAlone, 2026-10-26 22:24 UTC).
Fixed for the kind, not the one test: all six reads of a positive count go through the package's own pendingOf,
whose deadline was 1 s.

Research 'The controller, NATS-native': none; this changes tests only, not a part of the controller.
2026-10-26 23:28:03 +01:00
mesh-tools b611f7375a Merge pull request 'Pin mesh-sdk main, whose seats promise own and settle, so the catalogue passes the module check again (issue 1126)' (#476) from fix/1126-controller-pins-sdk-main into main 2026-10-26 22:08:19 +00:00
jschoubben 253bbcea2d Pin mesh-sdk main f178323, whose delivery seat promises settle beside the vault's own, so the catalogue check passes against mesh-catalog main f0f7b673 (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-controller@b611f7375a47
mesh/check checking 253bbcea2da8 on b611f7375a47
Moves no piece of research 'The controller, NATS-native': a repin of the seat catalogue, no controller code changed.
2026-10-26 23:03:47 +01:00
mesh-tools 2c6382346f Merge pull request 'Pin the mesh-sdk whose vault seat promises own, so the catalogue passes the module check again (issue 1126)' (#475) from fix/1126-vault-seat-own into main 2026-10-26 22:00:46 +00:00
jschoubben b8cafccaf6 Pin mesh-sdk at e78e881, whose vault seat promises own, so the catalogue check passes against mesh-catalog main (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-controller@2c6382346f48
mesh/delivery-group group fix/1126-vault-seat-own merged: every member is merged
mesh/check failed: unit: github.com/novox/mesh-controller/cmd/mesh-controller.TestModuleCheckPassesTheCatalogue: the catalogue does not pass its own c…
Moves no piece of research 'The controller, NATS-native': a repin for a seat verb list, no controller code changed.
2026-10-26 22:40:34 +01:00
mesh-tools 82d212a2a1 Merge pull request 'The controller answers each node's placement from its own store, the one verb of its exit (issue 1126)' (#474) from feat/1126-nodes-compose into main 2026-10-26 21:11:21 +00:00
jschoubben df8ebf6a77 Pin mesh-sdk 9796444 (l2-next: feat/1126-nodes-compose merged with sdk main #101, the standing-test race fixes) (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-controller@82d212a2a1ff
mesh/delivery-group group feat/1126-nodes-compose merged: every member is merged
mesh/check checking df8ebf6a77c1 on 82d212a2a1ff | novox/mesh-controller#474 in group feat/1126-nodes-compose
2026-10-26 22:05:41 +01:00
jschoubben dc414fbfef Pin mesh-sdk 5f45175 (l2-next: feat/1126-nodes-compose merged with sdk main) after merging main (issue 1126) 2026-10-26 21:59:21 +01:00
jschoubben fd891688ac Pin mesh-sdk feat/1126-nodes-compose 03dce12, the group's head: the checker refuses a pin that is neither trunk nor the group's head (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/delivery-group group feat/1126-nodes-compose checking: 4 of 6 member(s) ready
mesh/check checking fd891688acdf on 2ecfbb3a29cb | novox/mesh-controller#474 in group feat/1126-nodes-compose
2026-10-26 20:35:49 +01:00
jschoubben 9d8a5a7ede Review of L2, round 2: the placement export is checked within an existing integration test over the same fresh mesh, so it adds no test of its own (the test budget; issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 9d8a5a7ede90 on 2ecfbb3a29cb | novox/mesh-controller#474 in group feat/1126-nodes-compose
2026-10-26 20:23:51 +01:00
jschoubben 4e9e5c27d4 Review of L2: the export's test reads each printed placement back as the file the seed writes, at its version (S1); pin mesh-sdk 3968849, CAPTURED taken again (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: ..test: ✔ a transient empty list for four passes retires nothing; five stable passes retire, data kept (290.408814ms) | 4e9e5…
2026-10-26 20:12:49 +01:00
jschoubben 7c8a2e03dd Pin mesh-sdk feat/1126-nodes-compose 9713c30: placement carries a settings-form module's values (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-catalog#505 does not merge into the trunk: files in conflict: .gitignore | 7c8a2e03dd8e on 2ecfbb3a29cb | unit 1245…
2026-10-26 19:40:43 +01:00
jschoubben 89f52a6504 Pin mesh-sdk 53c2def, the group's head, and take the SDK's conformance copy again at it (issue 1126)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: git.novox.be/novox/mesh-catalog/checks/nobinaries.TestTheCatalogueHoldsNoCompiledProgram: modules/mesh-config/cmd/mesh-config…
2026-10-26 19:28:39 +01:00
jschoubben fb2f5a2962 The controller answers each node's placement from its own store, read-only, for the seed of the catalogue's placement files: the one verb of its exit (issue 1126, ADR 0555 §6; the operator 2026-10-26 19:06:40) 2026-10-26 19:16:09 +01:00
mesh-tools 2ecfbb3a29 Merge pull request 'The controller takes mesh-sdk a7a1fbd3, so its composer gives build modules the node's CPU share (issue 1806, ADR 0553/0554 change B)' (#473) from fix/controller-sdk-cpu-share into main 2026-10-26 15:41:03 +00:00
jschoubben 9ad8715871 Bump mesh-sdk to a7a1fbd3 so the build seats' runtimes are composed to yield (issue 1806)
mesh/delivery merged: source.main moved to novox/mesh-controller@2ecfbb3a29cb
mesh/check pass 9ad8715871ea on 2ecfbb3a29cb | carried from e1a55f099b4e | unit 1023 run, 171 kept, 2 skipped | integration 1701 run, 214 kept, 9 skip…
The controller is the only composer, and its pin (10b6e202) predates mesh-sdk #95: without the bump the
build runtimes never get yields {priority idle, cpu-share} or MESH_BUILD_CPU_SHARE (ADR 0553, ADR 0554,
change B). The operator said yes to this one change of the frozen controller on 2026-10-26 15:54.

A composition at both pins over every node's assignment differs only in mesh-build-go, mesh-build-node and
mesh-build-image (the yields and the share word); every other composition is byte-identical. The SDK's
conformance capture moves with the pin, as testdata/beside/CAPTURED asks.

Research 072 (the controller, NATS-native): no piece moved; a pin bump the operator allowed.
2026-10-26 16:34:55 +01:00
mesh-tools e1a55f099b Merge pull request 'Seat users pulled by kind may read their own seat's dead letters (issue 734)' (#472) from fix/734-lost-ask into main 2026-10-26 13:40:43 +00:00
jschoubben ccc7ba334c broker: DeadLetterReads grants only what a pulled ordered consumer needs, as mesh-sdk buscompose (issue 734, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-controller@e1a55f099b4e
mesh/delivery-group group fix/734-lost-ask merged: every member is merged
mesh/check checking ccc7ba334c82 on 08a261ac1a6d | novox/mesh-controller#472 in group fix/734-lost-ask
NATS-native research piece: none moved; the copy stays in step with mesh-sdk until mesh-grants composes the list.
2026-10-26 14:17:00 +01:00
jschoubben 65d5ed3367 broker: a user of a seat pulled by kind reads its seat's dead letters, as mesh-sdk buscompose composes it (issue 734)
The checker hears from DEAD_LETTERS which of its asks a build seat gave up on, and ends the try or asks the check again;
the controller composes today's user list, so its copy of the composer carries the grant too. NATS-native research piece:
none moved; the composition itself moves to mesh-grants (ADR 0547), and this keeps the copy in step with mesh-sdk.
2026-10-26 13:51:35 +01:00
mesh-tools 08a261ac1a Merge pull request 'Declarations test: the indexer stand-in answers the trunk value at the trunk's commit alone (issue 1945)' (#471) from fix/declarations-test-stand-in into main 2026-10-26 12:30:18 +00:00
jschoubben 23226384e5 Answer the indexer's trunk value at the trunk's commit alone in the declarations test, so its take-in records mesh-host's build (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-controller@08a261ac1a6d
mesh/check being carried to 23226384e529 on 08a261ac1a6d | from 350f8ebdfe5f
The stub answered the trunk for every commit, so since registering on the
fingerprint (666d1b68, #465) mesh-host, registered at an older commit, looked
already at the trunk's fingerprint: the take-in only moved its commit, no build
was recorded, and the fingerprints never named the new node-engine. The real
indexer holds no value at a commit it never indexed.

(cherry picked from commit 779533764b)
2026-10-26 13:29:04 +01:00
mesh-tools 350f8ebdfe Merge pull request 'Broker: the bus node may follow the grants part's sealed user list and the grants part may write it, the operator's second one-grant exception (issue 1945)' (#470) from feat/1945-user-list-switch into main 2026-10-26 12:21:44 +00:00
jschoubben 09d2cc5056 Broker: the bus node may follow the grants part's sealed user list and the grants part may write it, the operator's second one-grant exception (issue 1945, ADR 0547 clauses 7-8)
mesh/delivery merged: source.main moved to novox/mesh-controller@350f8ebdfe5f
mesh/delivery-group group feat/1945-user-list-switch merged: every member is merged
mesh/check checking 09d2cc5056ef on 47e64f0284a9 | novox/mesh-controller#470 in group feat/1945-user-list-switch
2026-10-26 13:03:15 +01:00
mesh-tools 47e64f0284 Merge pull request 'Broker: the grants part may create a placed module's declared buckets, logs and streams, creation only (issue 1790; the operator's one-grant exception)' (#469) from fix/1790-grants-create-buckets into main 2026-10-26 08:17:19 +00:00
jschoubben 6927b02d4d Grant the grants part creating the streams manifests declare, so a new module state no longer waits for a controller start
mesh/delivery merged: source.main moved to novox/mesh-controller@47e64f0284a9
mesh/delivery-group group fix/1790-grants-create-buckets merged: every member is merged
mesh/check checking 6927b02d4d58 on fda4859a95b8 | novox/mesh-controller#469 in group fix/1790-grants-create-buckets
The operator's one more grant in the frozen controller's composer (2026-10-26 08:54), beside the
memberships grant of 22:31 (ADR 0547 clause 1 and 5): mesh-grants' principal may publish
$JS.API.STREAM.CREATE.<stream> and $JS.API.STREAM.INFO.<stream> for each bucket
(KV_<module>_<state>), log (LOG_<module>_<log>) and module stream a placed module's manifest
declares, each by its own name, and nothing wider. The writers table lets it create, never update
or delete. Byte-equal with mesh-sdk's buscompose (fix/1790-grants-create-buckets), whose expected
lists this code wrote; the same test file holds both copies. Nothing else changes, and the mesh-sdk
pin stays.

Research 072: moves the grants piece (the grants part split out of the controller) one step, the
creation of a module's declared bus objects; the controller's raise at start stays until it loses
its lease.

Issue: 1790
2026-10-26 09:02:44 +01:00
jschoubben fda4859a95 Merge pull request 'The bus composer grants mesh-grants publishing memberships, nothing more (ADR 0547, the operator's one grant of 2026-10-25 22:31)' (#468) from fix/1945-memberships into main
Reviewed-on: #468
2026-10-25 22:44:04 +00:00
jschoubben a81717e53c Seal the seeded account in the adapter-input test, the second fixture of its kind (issue 1945, PR #467)
mesh/delivery merged: source.main moved to novox/mesh-controller@fda4859a95b8
mesh/delivery-group group fix/1945-memberships merged: every member is merged
mesh/check checking a81717e53cf4 on 7cd4d18bd54e | novox/mesh-controller#468 in group fix/1945-memberships
The kind: a test that seeds a bus account without sealing it, which an account counted as issued only once sealed
(ADR 0543 §2) no longer counts. The whole integration level, run locally, found two: the addresses test (fixed
before) and TestTheAdaptersInputCarriesTheOwnSecretsTheBoxesTheLicencesAndTheBus. Test only.
2026-10-25 23:30:20 +01:00
jschoubben d01839c29c Grant the grants part reading every membership; seal the seeded account in the addresses test (ADR 0547 §5, review; issue 1945)
mesh/delivery-group group fix/1945-memberships rejected: composed together: the checker's verdict at the members' heads (2253ba8bbe9b): integration: github.com…
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestTheAdaptersInputCarriesTheOwnSecretsTheBoxesTheLicencesAndThe…
The grants part reads each module's last membership before it publishes, so it is granted the direct get on every
membership subject beside the publish (review of mesh-grants, B1).

The test TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker failed on main since PR #467: it seeds
anchor.mesh-controller with no seal, and an account counts as issued only once sealed (ADR 0543 §2). The rule is right;
the fixture is fixed, sealing the seeded credential the controller already uses. Test only.
2026-10-25 22:51:25 +01:00
jschoubben f67d15536b Grant the grants part publishing the modules' memberships, and nothing more (ADR 0547 §5, issue 1945)
No membership is published since release 6 (issue 1945 notes 42, 47), so a seat-holding module placed now cannot
serve its seat or pull asks (mesh-build-go on ace, 2026-10-25). The operator decided at 22:31 that the grants part
publishes them from the first milestone: its principal, module mesh-grants, may publish mesh.assignment.*.* and no
stream definition. The SDK's copy of this composer gets the same line, so the two stay byte-equal.

Research 072 (the controller, NATS-native): this moves the memberships of its Bus row to the grants part.
2026-10-25 22:32:35 +01:00
jschoubben 7cd4d18bd5 Merge pull request 'A bus account counts as issued only once sealed, stale accounts are forgotten, and grants name sealed accounts only (issue 1945 note 35)' (#467) from fix/1945-credential-when-row-without-seal into main
Reviewed-on: #467
2026-10-25 19:31:42 +00:00
jschoubben fbf14b68db Forget the accounts of modules no longer on their nodes at the start's look, and grant sealed accounts alone (issue 1945, ADR 0543, review)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/delivery-group group fix/1945-credential-when-row-without-seal held: a member is held: novox/mesh-controller@fbf14b68dbab (merged without a passing check:…
mesh/check checking fbf14b68dbab on 40034be479f8
Migration 0108 marks every standing account sealed, ace's stale mesh-build-go
rows from 17:19 among them, and the forgetting ran only at a later unassign:
the next assignment to ace would have issued nothing again.

- The start's look first forgets, from each node's own resolution (assigned
  and required), the accounts of every module that no longer resolves on its
  node, and says each; only from a reading that holds assignments, and a node
  whose resolution is unknown is left alone.
- Unassign forgets a module's accounts only once it no longer resolves on the
  node (a module another requires keeps running), and never the node's own
  provisioner account; the forgetting moved from the inventory to the act.
- Forgetting holds each account's lock across its delete, so an issue in flight
  never writes a row back after it.
- A module whose accounts are not all sealed is granted its sealed accounts
  alone, and its grant is dropped when none is: an older grant naming ace no
  longer lets the assigner want it.
- An issue's locked section has a deadline of two minutes, since it holds a
  connection of the store's pool across the vault and the bus.
Tests: cases in the existing tables.
Research 072: the grants piece (memberships and credentials heard from placement).
2026-10-25 20:27:30 +01:00
jschoubben 79ac62ffbc Issue an account whose credential is not sealed, and forget an unassigned module's accounts (issue 1945, ADR 0543)
Met a second time live on 2026-10-25 at 20:04 (note 35): mesh-build-go was
assigned to ace again, and its account rows, minted at 17:19, had outlived the
unassignment. The issue minted only when no row stood, so it minted and sealed
nothing; the node had no credential and the bus no user, and the module
crash-looped 20 times. The watch still said "the grants written".

- A bus account now says when its credential was sealed to its node (migration
  0108; every account standing before is taken as sealed). An issue under the
  assign act, the assignments' watch or a registration mints only an account
  that is not issued, a row with no seal counting as none, under the store's
  lock on the account across its look, mint and seal, and marks it sealed once
  sealed. A seal that fails leaves it unsealed for the next issue.
- Unassigning a module forgets its bus accounts and their credentials on that
  node, so an assignment again issues them afresh.
- Only sealed accounts count as minted for a composition and are named in a
  grant, so the assigner, which now wants a node only once the grant names its
  account (mesh-delivery, same branch), holds the node until it is.
- The assignment's log line says which accounts were issued and sealed, or that
  they stood issued and sealed.

The process-only issue drops the atomic insert of the review of 9ff47f6a for
the lock, which also covers the seal. Tests: cases in the existing tables.
Research 072: the grants piece (memberships and credentials heard from placement).
2026-10-25 20:17:46 +01:00
jschoubben 40034be479 Merge pull request 'Kept compositions follow config changes from a durable consumer, so going back to an earlier build keeps today's config (issue 1945, ADR 0545)' (#466) from fix/1945-kept-compositions-follow-config into main
Reviewed-on: #466
2026-10-25 18:37:57 +00:00
jschoubben 045f25bd13 Merge pull request 'Take new builds in from a durable queue on the bus, register on the fingerprint, and issue a node's bus accounts when its assignment stands (issue 1945, ADR 0543, ADR 0544)' (#465) from fix/1945-take-in-consumer into main
Reviewed-on: #465
2026-10-25 18:06:47 +00:00
jschoubben ff3cc47027 Let the kept compositions' pass end at the stop (issue 1945, ADR 0545, review 3)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/delivery-group group fix/1945-kept-compositions-follow-config held: a member is held: novox/mesh-controller@ff3cc47027e0 (merged without a passing check: …
mesh/check failed: integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestTheDeclarationsStreamGoesOnceEveryNodeRunsTheNewEngine: with …
followKept went on composing every kept key, and the config consumer every
touched node, after the controller was told to stop. Both now check the stop
before each key and each node; the writes of an unfinished batch are refused
at once, so the next controller takes them.
Research 072: milestone 2, the composer reacting to the change of an input.
2026-10-25 18:43:38 +01:00
jschoubben 31a2bcdb3c Read each resolved config key once when the consumer is made, so values set before this release reach the kept compositions (issue 1945, ADR 0545, review 3)
The config consumer delivered new writes only, and mesh-config never writes an
unchanged value again: g14's mesh-build-go capacity of 2, set before this
release, would never reach its kept compositions carrying 0. A newly made
durable now reads each key's last value once, through the same gather,
per-write lock and acknowledgement; its deliver policy is set before its first
delivery. Acknowledgements are synchronous and sent past the stop, and one the
bus refuses is said by its key.
Research 072: milestone 2, the composer reacting to the change of an input.
2026-10-25 18:38:45 +01:00
jschoubben 2df47d7b1c Merge branch 'fix/1945-take-in-consumer' into fix/1945-kept-compositions-follow-config 2026-10-25 18:30:53 +01:00
jschoubben 666d1b682e Register on the fingerprint, not the commit; count the take-in's attempts as settled (issue 1945, ADR 0544, review 3)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check checking 666d1b682e7f on 045f25bd1353
registeredAt compared commits, so a catalogue merge that moved only the commit
(2df05030 at 18:02, a85d9037 before it) took every catalogue module in again
and composed it: about 200 take-ins, some 25 minutes, on every mesh-catalog
merge, which held nextcloud for an hour. A module whose trunk fingerprint is
the one it is registered at now has the new commit recorded (built_from and
source_head, as a registration moves them) and composes nothing unless its
composition or grant is missing. Readers of the registered commit: the
fingerprints read the commit's build or, with none, the indexer's value at it,
which names the same fingerprint; status, modules, the board and the facts
read it as the source's commit, which it is; the upgrade path counts it as
built from that commit, which it is.

The stall watch counted only settles the bus accepted, so each refused
acknowledgement at a first start left work held for good: it now counts the
loop's every attempt. The acknowledgement and refusals at the stop are sent
past the stop, and the connection closes only after the readers and the loop
have ended.
Research 072: the take-ins on a bounded pull consumer.
2026-10-25 18:30:07 +01:00
jschoubben e40b1e70db Follow config from a durable consumer, compose again only what changed since, and hold the node only for a write (issue 1945, ADR 0545, review)
The start composed every kept composition of every module on every node, each a
whole node composed in memory under the node's lock: 10 to 50 minutes at a
start, holding take-ins back. Now the writes of mesh-config_resolved are read
from the durable consumer mesh-controller-config-follow, acknowledged once the
change is composed, so a restart composes again only what was written since
the last change handled, and a first start nothing it did not hear. The node's
lock is taken for each write alone. A consumer that cannot be made or read
raises config-not-followed and is tried again a minute later, cleared once it
reads. The controller's account is granted that consumer's acknowledgements
where mesh-config serves, as its verb is.
Research 072: milestone 2, the composer reacting to the change of an input.
2026-10-25 18:27:43 +01:00
jschoubben 6b116586fc Merge branch 'fix/1945-take-in-consumer' into fix/1945-kept-compositions-follow-config 2026-10-25 18:21:58 +01:00
jschoubben bf20d9d8cf Judge the take-in's stall by keys settled with work to do, and clear its alarms after a restart (issue 1945, ADR 0544, review 2)
A refused key stays unacknowledged during its delay, so one waiting key on a
quiet mesh raised take-in-stalled, cleared at the next ack and raised again.
The watch now counts the keys the loop settled (acknowledged or refused, as
the bus took them) and calls it a stall only with work to do, keys not
delivered yet or a batch handed and not settled, and nothing settled for 5
minutes: a hung loop stays loud. Its first look always says what it found, so
a stall raised before a restart is cleared once this controller moves. A
consumer not made raises take-in-not-consumed under its own id per bucket,
cleared when that consumer is made. Acks are synchronous, so only those the
bus took count; on stop the iterator is drained and what it held, and a batch
cut by the stop, are refused with no delay.
Research 072: the take-ins on a bounded pull consumer.
2026-10-25 18:21:52 +01:00
jschoubben 81e335e42d Say why a node with no compositions answers none to compose again (ADR 0545, the empty-on-error rule) 2026-10-25 18:15:22 +01:00
jschoubben 2407bfb9ff Compose kept compositions again when a module's config changes (issue 1945, ADR 0545)
The controller composed a module only at the fingerprint it is registered at,
so each of the up to five compositions kept for going back carried the config
of the moment it was made: a want back to an earlier mesh-build-go on g14 put
its capacity back to 0. Nothing composed a node when mesh-config's resolved
values changed, either.

Now a write of mesh-config_resolved (<node>.<module>.<key>), heard from that
bucket's watch and gathered for a second, drops the controller's 5 s reading
of the values, composes the node, and composes every kept fingerprint of the
touched modules on that node again from its own build's manifest with today's
inputs, writing only the compositions whose digest differs. A starting
controller does it once for every node, for a change no controller heard.
The two replaced-controller tests are one table test with the kept cases.
Research 072: milestone 2, the composer reacting to the change of an input.
2026-10-25 18:14:03 +01:00
jschoubben 850f4b4d26 Keep refused keys from holding the take-in's slots, and judge its stall by acknowledgements (issue 1945, ADR 0544, review)
A refused key stays unacknowledged until its delay ends, so with 8 a consumer
a handful refused for long (a manifest word this controller does not read
yet, a node that keeps failing) stopped that consumer for good, the build that
reads the word among what it held back. Each consumer now leaves up to 1000
unacknowledged; the store's bound is the one loop's batch of 16. A refusal
waits 30 s, doubling each time up to 10 minutes.

The stall watch judged movement by deliveries, which rise with every
redelivery: it now counts the keys this process acknowledged. A consumer that
cannot be made raises take-in-stalled instead of a log line. On stop the
gathered batch and a key in hand are refused with no delay, so the next
controller has them at once. A trunk value with no fingerprint of the new form
is said once, as every wait for a write.
Research 072: the take-ins on a bounded pull consumer.
2026-10-25 18:10:35 +01:00
jschoubben e7673e3bd3 Merge branch 'fix/1945-credential-on-assignment' into fix/1945-take-in-consumer 2026-10-25 18:07:30 +01:00
jschoubben 3e39ccf958 Let only a watch that read every standing assignment be the start's look (issue 1945, ADR 0543, review)
The flag that made the first watch the start's look was cleared after every
watch, even one that failed to open or ended before its Caught mark: the next
watch then handed every standing assignment as fresh, about 300 settles at once
with a catalogue read each, or dropped the keys the cut watch had handed. The
flag is now cleared by the start's look itself, and a watch cut before it opens
the next with nothing heard. Two rows of the existing table test cover it.
Research 072: the grants piece (memberships and credentials heard from placement).
2026-10-25 18:07:21 +01:00
jschoubben 94bb74966f Bound the take-in's two consumers to 16 keys in flight together, 8 each (ADR 0544 §1, its review)
Two consumers of 16 each allowed 32 take-ins on the store at once, twice
issue 1945's bound. A batch takes up to 16. Research 072: the take-ins on a
bounded pull consumer.
2026-10-25 18:01:18 +01:00
jschoubben bfb271a91b Take done keys in from a durable pull consumer, a batch at a time (issue 1945, ADR 0544)
The take-in watched the builder's and the indexer's buckets, started a follower
per module that retried on a 30 s timer, and at start took every catalogued
module in alphabetically: on 2026-10-25 a restart took in about 2 modules a
minute, about 190 followers queued behind 16 slots, and every take-in composed
two or three times even with nothing new, so a new build waited 30 minutes and
more for its credential.

Now two durable consumers named mesh-controller-take-in read the indexer's
modules bucket (every module's last trunk value when the consumer is made, then
each write) and the builder's builds bucket (each write), at most 16 keys
unacknowledged each. The bus keeps what is not taken in: the start's look is the
consumers' own catch-up, and a restart costs only what was left unacknowledged.
A batch reads the catalogue once, takes each module in once, and asks the
composer twice: once for its modules' nodes, once for the bus's node with the
grants. A module registered at its trunk whose compositions and grant exist, or
that this process took in at the same trunk, composes nothing; a key delivered
again is taken in whole. A wait for the builder's or indexer's next write is
acknowledged, a wait on a read or a failure refused with a 30 s delay. The
backlog is in status, and a backlog standing still for 5 minutes raises
take-in-stalled. The controller's account is granted the two consumers' acks.

The follower tests go with the followers: twelve tests become three.
Research 072: the take-ins on a bounded pull consumer (milestone 4, done keys
alone) and the start's look composing only what changed (milestone 2).
2026-10-25 17:59:20 +01:00
jschoubben 9ff47f6aaa Mint an assignment's account once, whoever finds it missing first (issue 1945, ADR 0543, review)
The assign act and the assignments' watch both checked whether an account was
minted and then minted it, and minting overwrites the hash: interleaved, the
bus kept one caller's hash and the node was sealed the other's password, the
crash loop this branch fixes. Both now mint through one statement that inserts
the row only when none stands; the caller whose insert wins seals, the other
seals nothing. issueMissingProcessAccounts mints the same way.

A watch opened again after one ended handed the standing assignments as the
start's look, so one that came to stand in the gap got no bus composition or
grant when its account already stood: only the process's first watch is the
start's look now. The start's look reads the catalogue once and ends with one
composition of the bus's node and the grants; a controller not holding the
lease is told so and tries again rather than dropping the key.

Tests folded to stay within the budget: the assignment tests are three table
tests, the process account and mayIssue tests folded into one of them, and
the concurrent mint is a case of the inventory's rotation test.
Research 072: the grants piece (memberships and credentials heard from placement).
2026-10-25 17:51:06 +01:00
jschoubben ca5f74eadb Issue a node's bus accounts and grant when its assignment comes to stand (issue 1945, ADR 0543)
ADR 0539 clause 7 issues a module's credential and grant only when its build is done.
A module built long before and assigned to one more node never has a build done again:
on 2026-10-25 at 17:19 mesh-build-node and mesh-build-go were assigned to ace and composed
for it, the bus's user list never named ace's accounts, and both runtimes restarted every
5 s (one with no credential, one refused by the bus). The assign act composes the assigned
node alone, never the node holding the bus, and writes no grant.

So a write of the assignments bucket is the event: when a key comes to stand (new, or
ended or pending before), the lease holder issues the module's accounts on that node that
are not minted (never re-issued), composes the node holding the bus and writes the grants,
through the composer the done keys' take-ins use. A rewrite of a standing assignment does
nothing; the start's look settles every standing one, one after another. A step of
research 072's grants (memberships and credentials heard from placement).
2026-10-25 17:37:03 +01:00
jschoubben c3db00ceac Merge pull request 'Compose the packages mirror only on a change, end every serve goroutine before serve returns, and raise no alarm for a cancellation (issue 1945)' (#464) from fix/r6-mirror-on-change-2 into main
Reviewed-on: #464
2026-10-25 16:20:55 +00:00
jschoubben 8e02c4726e Clear a module's fingerprint-not-composed on every successful take-in, and stop serve's goroutines before an error return undoes anything (issue 1945)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check pass 8e02c4726eb4 on c3db00ceac25 | unit 981 run, 171 kept, 2 skipped | integration 1641 run, 232 kept, 7 skipped | over bound: integration…
A success cleared the condition only when the same follower had raised it, so
one raised before a restart or by another follower stayed open after the take-in
worked, and the operator read that a build needed a look. Clearing a condition
that is not open costs one read. On an error return, serve now ends its
goroutines before the keeper closes and before handActConn and the bus are put
back (the review of fix 12).
2026-10-25 17:09:35 +01:00
jschoubben cc5c9ae888 Count the take-ins and every other background work in the serving group, so serve's stop waits for them too (issue 1945)
inBackground ran the take-ins of done keys (now in #463's slots), the start's
look, the given values' replacement and the announced reads beside serve without
serve waiting for them: a take-in could outlive its controller as the loops did.
It now takes the context and is counted in its serving group as well, and a
serve that is stopping starts none.
2026-10-25 16:36:49 +01:00
jschoubben c7e31640cf Raise no condition for a take-in, a layout or a put cut by a cancellation (issue 1945)
Thirty urgent fingerprint-not-composed conditions were live on 2026-10-25, each a
done key's take-in a stop or a restart cut with "context canceled". A
cancellation is no fault of the build, the chain or the bucket: a take-in or a
layout whose controller stops returns and the next one does it; one whose own
step was cancelled is tried again without a condition; a send cut by a
cancellation counts as no failure to put its files.

The put test now starts and leaves the node's failure count at nothing, so a
second run of it (-count=2) finds no count left by the first.
2026-10-25 16:30:58 +01:00
jschoubben c75d726205 End every goroutine serve starts before serve returns, so no loop outlives its controller (issue 1945)
The try of the on-change mirror failed with every test passed: goroutines a
served test started outlived its serve and met the next test's globals and its
closed stores. Under -race the served tests show it as a race on conditionsFrom
(watchTheMesh writing it while settlingPending, already running, read it), and
a take-in cut by the stop raised fingerprint-not-composed, a kind with no plain
words, which fails the package after its PASS line.

- serve counts every goroutine it and its parts start (serving_group.go) and
  ends and waits for them before anything else is undone: the lease's watch,
  the pending settler, the status summary, the facts, the mirrors and the new
  declared-packages keeper with its two bucket watches, the declaration sweep,
  the read-call counter, the watches' closers, the have watches, the vault move
  on a report, and the watchdogs, doctor, healers and asker under their own
  group.
- The condition store opens before anything that reads conditionsFrom starts;
  askerFrom is an atomic pointer, read from the keeper's goroutines.
- A done key's take-in cut by the stop raises nothing, and fingerprint-not-composed
  has plain words.
2026-10-25 16:30:58 +01:00
jschoubben a4cd5ba375 Compose the declared-packages mirror only when what it composes from changed (issue 1945)
The mirror composed every node in full on the record mirrors' 10 s tick; a
pass takes about 45 s on the control node, so the controller never rested
(130-175 % CPU, issue 1945 note 22). It now passes on a watch of the
compositions' digests and of the assignments, one pass at a time with the
wakes of a pass coalesced into one after it (at least 30 s apart), skips the
composition when the stores' Input digest is unchanged, and keeps a safety
pass every 10 minutes that composes nothing when nothing changed.
2026-10-25 16:30:58 +01:00
jschoubben 62802cd966 Merge pull request 'Share the controller's stores and take in at most 16 done keys at once, so a burst of builds cannot fill the store (issue 1277, issue 1945)' (#463) from fix/store-pool-flood into main
Reviewed-on: #463
2026-10-25 15:29:58 +00:00
jschoubben e9a885cda9 Share the controller's stores with its take-ins and cap take-ins at 16
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/check checking e9a885cda925 on 62802cd96602
At 16:07:36 the builder rewrote about 120 done keys at once. Each woke a
take-in of its own with no limit, and each take-in read the rest of the
mesh through restOfTheMeshOf, which opened a licences and an identity
pool per call. novox's store ran out of connections at 16:08 and refused
everyone, the controller's retries keeping it full.

- The serving controller lends its stores by inventory (shareStores);
  restOfTheMeshOf reads on them instead of opening pools per call, and a
  verb run as its own process still opens its own. stores opens each
  store once under a lock, since readers now share it. serve uses the
  shared identity store instead of a second pool.
- The watch's take-ins and the start's look share one cap of 16 take-ins
  at once (takeInSlots); a take-in waiting for its next try holds none.
- store.Ready backs off on a full store (SQLSTATE 53300), doubling its
  wait up to 5 s, and says once that the store is full.

Issues 1945 and 1277.
2026-10-25 16:23:55 +01:00
mesh-tools 2814262ac0 Merge pull request 'Keep the declarations retired once their stream is gone, so the bus is not called unhealthy for a stream release 6 deleted (issue 1945)' (#462) from fix/r6-nodes-stream-gone into main 2026-10-25 14:25:11 +00:00
jschoubben 0a536d7b6d Keep the declarations retired once their stream is gone, so nothing expects, asserts or makes NODES again (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-controller@2814262ac0fa
mesh/check pass 0a536d7b6d79 on 2bcd85d2df4c | unit 966 run, 172 kept, 2 skipped | integration 1622 run, 236 kept, 7 skipped | over bound: integration…
A node-engine newer than the registered one, or a node asleep on an older
have, made the done test false after the start deleted NODES; the bus's
health check, probes D6 and D7 and healer H3 then expected the stream and
every node's consumer again, raised them urgent and made them anew. Release
6 sends no declaration, so a stream made again helps no node: a gone stream
is retired. The advisory watchdog also reads a consumer whose stream is gone
as gone instead of failing.
2026-10-25 15:01:09 +01:00
mesh-tools 2bcd85d2df Merge pull request 'Never put a module whose want names its own composition into the node's own, and read a re-registered module's fingerprint from the indexer (issue 1945)' (#461) from fix/r6-own-composition-dupes into main 2026-10-25 13:37:34 +00:00
jschoubben ac34bce49f Take a module's fingerprint from the indexer when its commit has no build, and never compose a module its want names into the node's own (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-controller@2bcd85d2df4c
mesh/check pass ac34bce49f58 on 2bcd85d2df4c | unit 966 run, 171 kept, 2 skipped | integration 1625 run, 232 kept, 7 skipped | over bound: integration…
A catalogue merge that leaves a module's sources alone registers it at the new
commit from the same done key, whose build record stands at its first commit,
so the commit had no build and the module no fingerprint of the new form: it
went into the node's own composition while its want named its own, and every
node declared its files twice. currentFingerprints now reads the indexer's
trunk value at the commit when no build points at it (1945/AC15), and a module
a want or have names by its own composition is left out of the node's own
(1945/AC16).
2026-10-25 14:10:51 +01:00
mesh-tools af2b0844a1 Merge pull request 'Compose each machine once per pass, so take-ins of done builds stop colliding and every new build gets its composition and credential (issue 1945)' (#460) from fix/r6-compose-once into main 2026-10-25 12:30:32 +00:00
jschoubben 3b4cb69e84 Take again a replacement of given values asked while one ran on the node (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-controller@af2b0844a1fc
mesh/check pass 3b4cb69e84dd on af2b0844a1fc | unit 963 run, 171 kept, 2 skipped | integration 1622 run, 232 kept, 7 skipped | over bound: integration…
2026-10-25 13:02:49 +01:00
jschoubben c47a590463 The start pass reads a module's look-again mark after letting go, and hands it on to takeDoneKey (issue 1945) 2026-10-25 13:00:43 +01:00
jschoubben f35aa35541 Look at a module's mark again after letting go of it, so a wake just before is taken (issue 1945) 2026-10-25 12:55:33 +01:00
jschoubben 8114cee2af Give each caller of a pass its own nodes' errors and write the grants whenever asked; a wake during a take-in makes it look once more (issue 1945)
mesh/delivery closed: a newer head of the same pull request
mesh/check pass 8114cee2af47 on 0fff3efd4e0b | unit 959 run, 172 kept, 2 skipped | integration 1615 run, 236 kept, 7 skipped | over bound: integration…
2026-10-25 12:44:28 +01:00
jschoubben baa8f58473 Compose each node once per pass in one composer for the done keys' take-ins, one composition of a node at a time, and the start's look side by side (issue 1945) 2026-10-25 12:39:59 +01:00
jschoubben 0470869fbb Keep the digest of the composition the bucket holds, following another writer's composition written in between (issue 1945) 2026-10-25 12:36:00 +01:00
jschoubben 8a3c0cd022 Say each wait of a done key's take-in once with its reason and look again every 30 s, one follower per module (issue 1945) 2026-10-25 12:35:42 +01:00
mesh-tools 0fff3efd4e Merge pull request 'Register every done key once at the start, and write a node's own composition last (issue 1945)' (#459) from fix/r6-fp2-seen into main 2026-10-25 04:21:27 +00:00
jschoubben eafd1883f6 Write a node's own composition last, and not in a pass where a module's composition failed (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-controller@0fff3efd4e0b
mesh/check pass eafd1883f65c on 0fff3efd4e0b | unit 953 run, 171 kept, 2 skipped | integration 1612 run, 232 kept, 7 skipped | over bound: integration…
2026-10-25 02:57:24 +02:00
jschoubben e67916bbd8 Take every catalogued module's done key once at the start, beside the updates-only watch, so a key written or interrupted across a restart is registered (issue 1945) 2026-10-25 02:44:50 +02:00
jschoubben 7039c1b4b3 Merge pull request 'Keep moved modules' filled paths in the node's own composition, put every file object, grant access-act and the provisioner's have read (issue 1944)' (#458) from fix/r6-node-composition into main
Reviewed-on: #458
2026-10-25 00:17:42 +00:00
jschoubben 3dadc46ee6 Grant each provisioner client a read of its node's own have, so it refills when the node's own composition moves (issue 1945)
mesh/delivery checking
mesh/delivery-group group fix/r6-node-composition held: a member is held: novox/mesh-sdk@10b6e202320f (merged without a passing check: only a person decides wh…
mesh/check checking 3dadc46ee6cc on ba4d15657343 | novox/mesh-controller#458 in group fix/r6-node-composition
2026-10-25 01:58:27 +02:00
jschoubben d8aa565fee Keep the filled paths of the modules moved into a node's own composition, which the node-engine otherwise wrote empty (issue 1945) 2026-10-25 01:47:57 +02:00
jschoubben 82c89bec65 Split compositions from the inline body so their large files' objects are put, and refuse a composition naming an object the bucket lacks (issue 1945) 2026-10-25 01:42:28 +02:00
jschoubben b1f758ed71 Grant each node-engine its own access-act ask, which it has subscribed to since ADR 0325 and was refused at every connect (issue 1945) 2026-10-25 01:42:28 +02:00
mesh-tools ba4d156573 Merge pull request 'R6: the controller writes signed per-node compositions and their digests; plans, pushes and their verbs go (issue 1945)' (#457) from feat/r6 into main 2026-10-24 23:00:30 +00:00
jschoubben 9c38aed789 Read the declaration a served-key test checks from the composition, since the plan verb is gone (issue 1945)
mesh/delivery delivering: 0 machine step(s) passed
mesh/delivery-group group feat/r6 merged: every member is merged
mesh/check checking 9c38aed78905 on bdb8193456e1 | novox/mesh-controller#457 in group feat/r6
2026-10-25 00:06:06 +02:00
jschoubben a3f1d0460d Wait for the whole start pass before serving a served test, so a rotation test never reads it midway (issue 1945) 2026-10-24 23:48:51 +02:00
jschoubben 0af12b369b Wait for the done key's grant, which the take-in writes after the compositions (issue 1945) 2026-10-24 23:45:33 +02:00
jschoubben e338b1f0f4 Re-capture the converged declaration for release 6: every resource says its module (issue 1945)
readNode sets ReadsModules, since release 6 composes only for node-engines that read a module per resource (issue 1145);
the failure says the bytes on its first line, the one the verdict keeps (issue 1921).
2026-10-24 23:21:28 +02:00
jschoubben e42818a0ab Merge main (the R4g finish: reads-outside markers) into feat/r6 (issue 1945)
The silent mid-tier plan test main changed stays deleted with the plans (ADR 0539); the SDK is pinned to the feat/r6
head that merged mesh-sdk's main, 748c564c, whose hand path tool is now mesh-hand-compose.
2026-10-24 23:04:56 +02:00
jschoubben bdb8193456 Merge pull request 'R4g finish: reads-outside markers (issue 1935)' (#454) from feat/r4g-finish into main
Reviewed-on: #454
2026-10-24 20:57:07 +00:00
jschoubben ca25e4ab45 Give back everything serve sets once the served tests' controller stops (issue 1945)
The served mark in the environment made every later test in the process read as the serving controller's: the
registration test was refused as "never through a verb". The fixture now restores the mark, the hand-act log's
connection, the keeper, the status and the event publishers serve sets.
2026-10-24 22:54:00 +02:00
jschoubben 7a64075702 Let a later test's verb run after the served controller stops (issue 1945)
serve marks the process as handing over once it stops, and refuses every verb from then on; the served tests run it in
the test process, so the connection test's conditions verb was refused without its own process ever starting.
2026-10-24 22:44:12 +02:00
jschoubben 9abe0b59fa Give back the connection name a test's in-process command line set (issue 1945)
runLine runs the controller's dispatch in the test process, which names every later connection after the line's verb:
the proposals tests left "mesh-controller verb settings", and the conditions verb's test later in the same process
read that name instead of its own.
2026-10-24 22:37:46 +02:00
jschoubben 183d7a99ff Assert the controller's buckets in the builder-on-the-bus fixture (issue 1945)
A merge the controller hears is now kept in the bucket of the merges (ADR 0539 §17); the test that sends one passed
only when an earlier test on the shared bus had made that bucket.
2026-10-24 22:23:11 +02:00
jschoubben fad3d7ee71 Let go of the hand-act log's connection when serving ends (issue 1945)
A controller served in the test process left it pointing at a closed connection, so the next test's conditions verb
failed with "connection closed".
2026-10-24 22:13:52 +02:00
jschoubben 3c50e80972 Merge main (#456, the pre-switch grants) into feat/r6 (issue 1945)
Every grant #456 gives was already feat/r6's, by the SDK's names: each conflict takes feat/r6's side, with main's
test of the user list before the switch kept beside feat/r6's, and the duplicate assertion of the digests' bucket
dropped.
2026-10-24 22:02:32 +02:00
jschoubben 6e4bca7fce Run the served rotation's verb on the bus chain the controller keeps, as its child process does (issue 1945) 2026-10-24 21:48:30 +02:00
jschoubben f4998a7266 Merge pull request 'The bus's user list grants what release 6 uses before the switch, so the new controller, node-engines, assigner and the operator's way back are not refused after it (issue 1945)' (#456) from fix/1945-r6-grants-first into main
Reviewed-on: #456
2026-10-24 19:47:40 +00:00
jschoubben 53d9c4de19 Say the rotation test's whole trace on its failure's first line, and each judgement in it; an unreadable last declaration is cannot-judge (issue 1945)
The verdict keeps only a failure's first line (issue 1921), so the trace that names the cause rode on lines it dropped.
2026-10-24 21:41:14 +02:00
jschoubben 9372733861 Judge a have with no composition against the module's slice of the declaration the node last applied: equal is unchanged (issue 1945; ADR 0540 clause 9)
The first want at the switch skips the record hold only where it changes nothing on the node. A slice differing only in
the bus's user list is marked user-list-only, so no yes waits on it at the switch; any other change at the same
fingerprint is judged as any move, against the declaration's digest.
2026-10-24 21:27:07 +02:00
jschoubben d640ef15ec The report's contract names the ordered test 2026-10-24 21:05:52 +02:00
jschoubben 28e1b29e0c Drop the test of the digest-sent rule for an older account; the ordered tests hold it by the node-engine's own order 2026-10-24 21:05:38 +02:00
jschoubben f0815c4f18 A bare alive keeps the last real apply: assert that, not a Current the store no longer holds 2026-10-24 20:54:41 +02:00
jschoubben 72bbdeed35 Re-pin mesh-sdk to 7f9cc0e55c37 (issue 1945) 2026-10-24 20:44:09 +02:00
jschoubben dfd81d8daf Re-capture the catalogue at mesh-catalog feat/r6 3a1017f5 and excuse a derived subject whose module lives in another repository (issue 1945) 2026-10-24 20:37:04 +02:00
jschoubben 8c54139905 Trace a rotation's credential writes and compositions, record the bindings a composition carries, drop SchemaReachOf, and re-pin mesh-sdk to e45bdc0891bb (issue 1945) 2026-10-24 20:23:05 +02:00
jschoubben 6ed5de186d Say the serving controller's lines on the credential while the served rotation lands (issue 1945) 2026-10-24 20:14:30 +02:00
jschoubben 8fab4d8b22 Say the credential's state and what composing the consumer gives when the served rotation test fails (issue 1945) 2026-10-24 20:09:22 +02:00
jschoubben b879128efb Say rotate's output on one line when the served rotation test fails (issue 1945) 2026-10-24 20:03:32 +02:00
jschoubben 9641b88725 Drop the epoch-reading assertions and the two readers B8 left without a writer 2026-10-24 20:00:07 +02:00
jschoubben 36704abc08 Say what rotate printed when the served rotation test fails (issue 1945) 2026-10-24 19:46:34 +02:00
jschoubben 7fccba4658 Say each composition's resources and the credential's holders when a served test waits in vain (issue 1945) 2026-10-24 19:41:41 +02:00
jschoubben 98e948e30e Give the served board its credential in a file, so a rotation shows in its composition (issue 1945) 2026-10-24 19:36:28 +02:00
jschoubben f18bac4533 Give the served done key a fingerprint a key holds, and wait for the store's grants in the rotation (issue 1945) 2026-10-24 19:30:34 +02:00
jschoubben e05481e992 Compose a module that has no resource of its own as an empty composition (issue 1945)
Its want names a composition (ADR 0539 clauses 1 and 6), so a module whose needs others meet must still have one at
its fingerprint, or no want could ever be written for it. Found by the served test.
2026-10-24 19:25:14 +02:00
jschoubben dd58fbcb2c Assign the private network on the chain in the served test, as aMesh does on the store (issue 1945) 2026-10-24 19:19:45 +02:00
jschoubben 1ed5afe679 Say what composing the node answers when a served test waits in vain (issue 1945) 2026-10-24 19:14:39 +02:00
jschoubben 38523c6509 Assert notes among the assigned modules, which an assignment may bring others to 2026-10-24 19:10:32 +02:00
jschoubben a78f244621 Assert the controller's buckets before the served test's fixture writes the chain (issue 1945) 2026-10-24 19:09:54 +02:00
jschoubben 4ab4529440 Read the stored manifest through the record's own reader in the silent-file test 2026-10-24 19:06:05 +02:00
jschoubben 370673195c Put the served test's registrations on the bus's chain, and keep the seat-row test on a verb release 6 keeps (issue 1945)
Assignments and builds live on the bus (ADR 0408), so a fixture written before the chain was attached was not what
the serving controller read; and push, which the row test used, went with the plans.
2026-10-24 19:04:55 +02:00
jschoubben 3ad12f8ec7 Fix the test that has no sent list 2026-10-24 18:55:13 +02:00
jschoubben f6e78c837f Judge a preview and a refused flip from after the performed take, which composes the node; test that a take composes only when performed (release 6, issue 1945) 2026-10-24 18:54:54 +02:00
jschoubben 35e330c7f0 Say what a node's bucket holds when a served composition test waits in vain (issue 1945) 2026-10-24 18:53:32 +02:00
jschoubben aa7e53112b Find a want's have by the have's own key: want.<node>.<module> against <node>.<module> (release 6, issue 1945) 2026-10-24 18:43:22 +02:00
jschoubben af6bcea41e Prove the compositions through serve() whole, on a bus running the composed user list: the start, a done key, a real rotation (issue 1945; the review's G1)
The tests no longer call the writer on a bus without permissions: the controller runs as it serves, the builder and
the indexer write their own keys as their own accounts, and the operator's rotate is the real verb, so a subject the
user list does not grant fails them as it would fail the mesh.
2026-10-24 18:41:23 +02:00
jschoubben 49fc8b6d7d Format the done-key tests (issue 1945) 2026-10-24 18:36:21 +02:00
jschoubben f20d221a60 Return a grant not written and retry the take-in under a condition; drop the plans' events; grant bucket-deleted, restic's backup word and the operator's hand path (issue 1945; the review's D1, F1, H1 and its notes)
With no send left nothing else writes a missed grant, so the take-in of a done key returns it and tries again every
half minute under an urgent condition until it is written. plan-moved and sent go with the plans. The controller's
first-start deletions are said on a granted subject, a node hears only its backup holder, and the operator's account
writes the hand path, an object replaced included (ADR 0539 clauses 2, 7, 10 and 13).
2026-10-24 18:30:24 +02:00
jschoubben bd8ea8ad4f Format the hand path's live test (issue 1945)
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestAPlanPassesATierWhoseMachineNeverReportsAfterTheBoundAndSends…
2026-10-24 18:25:54 +02:00
jschoubben 1350c428ed Let the operator's hand path replace a file over 64 KiB in a node's object bucket (issue 1945; the review of A1, H1)
The object store reads an object's metadata and purges its old chunks when it replaces one, so the hand path is
granted the direct and message gets and the purge of each node's object stream; a test on a bus running the composed
user list walks the hand path end to end.
2026-10-24 18:25:46 +02:00
jschoubben 76c48fa074 Keep the files of every composition a want still names, not only each key's newest (issue 1945; the review's C1)
A node applies exactly the composition its want names (ADR 0539 clause 1), which may be a revision a newer one
replaced: its files over 64 KiB stay until no want names it.
2026-10-24 18:24:19 +02:00
jschoubben df6aab6f44 Judge a revision that changes only the bus's user list as no move, and say so beside it (issue 1945; the review's A2)
The user list is a file in a directory the bus's container mounts, so it read as moving data and held every new
credential for a person's yes, against ADR 0539 clause 7; mesh-sdk's UserListOnly lets the assigner write such a
revision at once.
2026-10-24 18:23:05 +02:00
jschoubben 7f8e5a821c Grant the controller's bucket-deleted, narrow a node's backup word to restic's, and grant the operator release 6's hand path ahead of the switch (issue 1945; the review of A1)
The new controller says its first start's deletions as an event, a node-engine hears only its backup holder, and the
way back right after the switch, a want and a composition written at the terminal, runs on the user list today's
controller composes (ADR 0539 clauses 2, 10 and 13). The operator is the administrator person, the one second
writer ADR 0539 names.
2026-10-24 18:21:58 +02:00
jschoubben 3db472ce56 Ask a backup holder once its module's have is running, not once a send was reported (release 6, issue 1945 B8) 2026-10-24 18:17:30 +02:00
jschoubben 3ae2d6e17c Grant release 6's compositions, wants and backups ahead of its switch (issue 1945)
The switch to release 6 is today's controller's last plan, and the bus runs the user list it composes until the new
controller writes one: so the new controller's writes of each node's compositions and their digests, each new
node-engine's reads of its own compositions and wants and its backup holder's verbs and word, and the assigner's read
of the digests must be granted by this controller already (ADR 0539 clauses 2, 6, 9 and 16; the review's A1).
2026-10-24 18:14:45 +02:00
jschoubben 29ef458d89 Say the send columns are kept for the controller of before and never read (issue 1945 note 8) 2026-10-24 18:12:59 +02:00
jschoubben 9d0798a1c0 Keep every column and table the controller of before reads: no migration 0108-0110, and a test that its queries still run (release 6, issue 1945 review H1) 2026-10-24 18:11:57 +02:00
jschoubben 0e85230769 A node with no want that has reported is current; with the bus out of reach it is not known, and the flip says so; no 'push' hint is left in adoption (release 6, issue 1945 B8) 2026-10-24 18:09:38 +02:00
jschoubben ad0aabc2a9 Merge main (the planner's switch tier, issue 1942) into release 6, which deletes the planner it changed (issue 1945) 2026-10-24 17:53:50 +02:00
jschoubben fe3faaef83 Read what a machine has from its haves: reported-and-current, the bus list, a given value's first good start; drop the digest of a send (migration 0110) (release 6, issue 1945 B8) 2026-10-24 17:51:40 +02:00
mesh-tools 34b105ea88 Merge pull request 'The planner puts the new node-engine and the new controller in one tier for release 6's switch, and orders every other plan as today (issue 1942)' (#455) from fix/1942-switch-one-tier into main 2026-10-24 15:37:02 +00:00
jschoubben 8f90d3ecd6 Drop what a send carried from the node row (migration 0109) and read an image list from the compositions (release 6, issue 1945 B8) 2026-10-24 17:29:03 +02:00
jschoubben b4a62e9e6e Delete the declarations' stream once every node runs the new node-engine, and assert it no more (issue 1945, B13)
A node away during the switch finds its last declaration on the stream; once every node's have names the
node-engine registered, no node reads one, so the stream and the nodes' consumers on it go, said once, and a raise
leaves them out (ADR 0539 clause 11).
2026-10-24 17:28:10 +02:00
jschoubben 03894e5e51 Delete whatTheMergeTouched, IsStaleRefusal and WriterEpoch, unused since the push went (release 6, issue 1945 B8) 2026-10-24 17:25:05 +02:00
jschoubben 565de8c46a Read what a bus machine runs from its have, not from a send kept (issue 1945, B6)
No send is kept any more (ADR 0539 clause 9), so SentBuilds has no last caller and B8 can delete it.
2026-10-24 17:24:51 +02:00
jschoubben c19d4516ef Keep the builds every want names and each composition's provisioner plans; find the replaced controller in the build records (issue 1945)
With no plan left to name a build or a send to keep the plans, a want names what a collection must keep, and the
composition is where the provisioner's plans are composed for the assignments mirror (ADR 0539 clauses 1 and 5);
the assignments no longer name a node's build, so the controller replaced is the newest earlier build of it.
2026-10-24 17:23:55 +02:00
jschoubben a8d63653fb Prove the compositions on a real bus and store: a done key composed and granted, a change rewriting only what reads it, a fresh start composing every node and deleting the plans' buckets once (issue 1945, AC2, AC3, AC6, AC14) 2026-10-24 17:19:07 +02:00
jschoubben aaadb5695c Keep B8's upgrades.go, which the commit before overwrote from an older copy; only the policy's rejudgement is added (issue 1945, B11) 2026-10-24 17:15:31 +02:00
jschoubben 73a31eb205 Judge each move against what the node runs, again when its have or the module's policy changes; carry the operator's keys in the node-engine's composition (issue 1945, B11, B12)
The assigner holds a move of a module of policy record from the judgement beside its digest, so it is judged by ADR
0377 §3's tests against the composition the node's have names, a held one never taking its place, and written again
when the have changes or a person sets the policy (ADR 0539 clauses 2 and 14). The node-engine's own file
operator-keys, from its setting operator-signers, carries the operator's public keys (clause 3).
2026-10-24 17:14:41 +02:00
jschoubben f56458da32 Delete the push and the declaration: the send stub and node holds, link declare and its order, the sent readers, and the sweep judged by compositions (release 6, issue 1945 B8) 2026-10-24 17:12:35 +02:00
jschoubben 66650e315c Take a registered module's new done key in, compose it and grant it; at start delete the plans' buckets and compose every node and the replaced controller (issue 1945, B3, B5, B13)
With no plan left to take a done key in, a write of the builder's keys registers the new fingerprint, composes the
module's nodes and the bus's user list, and writes its grant, so the assigner's rules can name it (ADR 0539 clauses
5 and 7). The new controller composes every node at its start, and the controller it replaced at its own
fingerprint, the operator's way back (clause 10); the merge window's, silent sends' and building plans' buckets are
no longer asserted and are deleted once, each said as the event bucket-deleted (clause 13). The assigner may read
the compositions' digests, and a node-engine hears its backup holder's backed-up.
2026-10-24 17:11:01 +02:00
jschoubben c40e15268e Compose where a declaration was sent: secrets, credentials, acceptances, the hub's peers, adoption, assignments and the bus's user list (issue 1945, B2, B6)
Each act that changed what a node runs now writes that node's compositions, and its wants follow by the assigner's
rules; a bus upgrade composes the bus's machines and answers the want the operator writes, since the assigner never
moves the bus (ADR 0539 clauses 5, 7 and 15). A node-engine may ask its own backup holder before a data module moves.
2026-10-24 16:59:06 +02:00
jschoubben 0f8c47842a Take the removed release 6 verbs out of the schema test's flag checks 2026-10-24 16:55:25 +02:00
jschoubben aee0a2718f Write each module's composition per node, signed, with its digest and the move's judgement beside it (issue 1945, B1)
Every act that sent a declaration now writes compositions instead, by compare-and-set under the lease, and a
module's compositions are kept for its five newest fingerprints and whatever its want or have names (ADR 0539
clauses 1 to 5 and 14). A module the mesh provides itself, which no want can name, rides in the node's own.
2026-10-24 16:54:11 +02:00
jschoubben 4f95be7e08 Answer the verbs release 6 removed by name, and drop the merge window and push settings from the manifest (release 6, issue 1945 B10) 2026-10-24 16:43:40 +02:00
jschoubben 788fb681ec Drop the plan state from the store and the assignments: migration 0108 and placement-only assignments (release 6, issue 1945 B9, B4) 2026-10-24 16:42:02 +02:00
jschoubben 96d7784bce Pin mesh-sdk to release 6's head, with its compositions, wants by digest and removed plan verbs (issue 1945) 2026-10-24 16:41:59 +02:00
jschoubben dc456d5660 Delete the controller's plan machinery: plans, batches, merge window, tiers, gate, retry and healers H1 and H2 (release 6, issue 1945 B7) 2026-10-24 16:27:55 +02:00
jschoubben 44d9d40582 Switch: acts only on a plan moving both engine and controller, and only for value 6 (issue 1942)
mesh/delivery delivered
mesh/check checking 44d9d4058253 on 34b105ea88df
2026-10-24 16:26:35 +02:00
jschoubben 74bed5a800 Switch plan: engine and controller share one tier when a merge names release 6's switch (issue 1942, ADR 0539 §11) 2026-10-24 16:15:52 +02:00
jschoubben ecbb317dc3 Wait for tier 1's module to be marked taken from its key: the tier moves in one look and the mark comes in the next, so a loaded build node read it between them (group check fcb076c5387f)
mesh/delivery delivered
mesh/delivery-group group feat/r4g-finish merged: every member is merged
mesh/check checking ecbb317dc324 on bdb8193456e1 | novox/mesh-controller#454 in group feat/r4g-finish
2026-10-24 14:32:47 +02:00
jschoubben dd2d8b91be Merge main into feat/r4g-finish (#453)
mesh/delivery-group group feat/r4g-finish rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-delivery#35: modules/mesh-delivery-assigner/go.mod names mesh-sdk bc0c25e51052, which is neither on mesh-sdk's t…
2026-10-24 13:26:24 +02:00
jschoubben 1b0fd8b442 Merge pull request 'Answer held and say artifacts-gone; write the grants again; integration tests match R4g (issues 1937, 1938, 1931)' (#453) from fix/1937-done-key-blobs into main
Reviewed-on: #453
2026-10-24 10:35:15 +00:00
jschoubben c06f75d083 Merge remote-tracking branch 'origin/fix/1931-reconnects-verb' into fix/1937-done-key-blobs
mesh/delivery covered after its stop: a later walk that contains it was delivered: plan-1792856223268385098 (novox/mesh-controller at 34b105ea into main,…
mesh/delivery-group group fix/1937-done-key-blobs closed: a member was closed before the group merged
mesh/check failed: forge: go.mod of novox/mesh-controller#453 cannot be read to place the mesh-sdk its tests read: nats: no responders available for r…
2026-10-24 07:16:20 +02:00
jschoubben 5f2da0af69 Have anchor report on tier 1 inside its send in the rollout test, so its report never races the silence bound (issue 1938, met twice)
Under a loaded run the report came 5.7 s and then more than 10 s after the send, and anchor was named silent. Now the
fake send makes anchor's report as soon as tier 1 is recorded, and the test waits for it; laptop never reports and is
named silent.
2026-10-24 07:13:22 +02:00
jschoubben 16ea033124 Merge remote-tracking branch 'origin/fix/1931-reconnects-verb' into fix/1937-done-key-blobs
mesh/delivery superseded: a newer head of the same pull request
mesh/delivery-group group fix/1937-done-key-blobs rejected: composed together: the checker's verdict at the members' heads (dbd9c3771cd8): integration: github.…
mesh/check failed: integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestARolloutAdvancesOnEventsAndAReplacedControllerResumesIt: the …
2026-10-24 06:55:31 +02:00
jschoubben 9088db9a04 Widen the rollout test's silence window to 10 s: under the whole integration level anchor's report came 5.7 s after its send (issue 1938) 2026-10-24 06:47:06 +02:00
jschoubben d0f5ae26de Give the rollout test a silence window anchor's report fits in while the whole integration level runs at once (issue 1938) 2026-10-24 06:31:46 +02:00
jschoubben e8b66a2eb3 Name tier 1's take in the rollout test by b's own repository, novox/b (issue 1938, after issue 1931) 2026-10-24 06:24:42 +02:00
jschoubben f2da7d0540 Have anchor report on tier 1's send in the rollout test only once the plan keeps when it was sent (issue 1938, after issue 1931) 2026-10-24 06:21:35 +02:00
jschoubben 5b03b2e26b Hold the rollout test's tier 1 to a key taken at b's own trunk value: the walk carries only the catalogue's commit, so KeyAt stays empty (issue 1938, after issue 1931) 2026-10-24 06:18:20 +02:00
jschoubben a5b874a2a0 Hold the replay of issue 614 to the plan since R4g: a new SDK plans itself and what stands on its package, no toolchain edge to the bundles (issue 1938) 2026-10-24 06:01:52 +02:00
jschoubben 491cbe2548 Hold the expiry of a pending assignment to the words it has said since R4g: its last build, when, and that it failed (issue 1938)
The test was kept from older runs and ran again only once the inventory changed beneath it; e419d018 changed the words.
2026-10-24 05:42:39 +02:00
jschoubben b2bf14ab01 Read a build's fingerprint back by its id, so the controller writes the grants the assigner reads (issue 1938)
chainBuildByID answered a trimmed build without SourceFingerprint since builds moved to the builds bucket (issue 1090), so
currentFingerprints read every fingerprint empty and grantsFrom skipped every module: mesh-controller_grants has never held
a key, and TestTheGrantOfAnIssuedCredentialIsOnTheBus failed on main.
2026-10-24 05:31:34 +02:00
jschoubben 9db1459cb0 Assert the key takes where a walk asks no build, and hold keys where a walk must stand still (issue 1931) 2026-10-24 05:14:41 +02:00
jschoubben 3cfcf5e3e4 Step the walk once before counting its key takes in the delivery-word test (issue 1931) 2026-10-24 05:06:57 +02:00
jschoubben f99e499288 Format the busobjects test (issue 1931) 2026-10-24 04:57:57 +02:00
jschoubben 3c9a4d56c0 Give the late build-seat holder a broker account so its worker is derived (issue 1931) 2026-10-24 04:57:53 +02:00
jschoubben e57224c25a Seed the seats before the late holder claims a build seat (issue 1931) 2026-10-24 04:55:12 +02:00
jschoubben bb2073b6bb Derive the late holder's worker from a build seat, since the build agent's seat is gone (issue 1931) 2026-10-24 04:48:24 +02:00
jschoubben 9131ef0dbc Hold the keys in the walk-beside test so its walks stay open (issue 1931) 2026-10-24 04:40:00 +02:00
jschoubben 8831c74f70 Mark the packages whose tests read files outside their directory, found by the review (issue 1935) 2026-10-24 04:38:51 +02:00
jschoubben fecfb40ed4 Hold the controller walk's key in the own-path test and read its walk again before ending it (issue 1931) 2026-10-24 04:30:20 +02:00
jschoubben a1969403c6 Read the started walk again before ending it in the let-go test: the steps wrote it (issue 1931) 2026-10-24 04:23:23 +02:00
jschoubben 893aa85349 Format the batches test imports (issue 1931) 2026-10-24 04:12:16 +02:00
jschoubben 2ba9eef58d Keep the started walk started in the let-go test by holding its key, since a walk takes keys at once now (issue 1931) 2026-10-24 04:12:07 +02:00
jschoubben 3b51d3f86e Answer held and say artifacts-gone, so the builder rebuilds a done key whose blob the artifact store no longer holds (issue 1937)
mesh/delivery-group group fix/1937-done-key-blobs rejected: composed together: the checker's verdict at the members' heads (c20fdcb8c623): integration: github.…
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestAWarrantForRetryRetriesAFailedPlanInTheStoreAndEndsItsConditi…
A done key the controller never took in holds no manifest, and the store's nightly collector deleted its bare blob:
node-tools', mesh-host's and the forge bridge's keys name blobs that answer 404. Since issue 1933 the take-in refuses such a
key, and nothing built it again. held (in the serving process, HEADs only) lets the builder ask before it reuses a key; a
refused take-in now names every gone archive and says artifacts-gone with the module, key, fingerprint and references.
2026-10-24 04:12:02 +02:00
jschoubben 5ee08cb5e1 Assert the merge walked alone is taken from its key at its own commit, since a cut records no ask (issue 1931) 2026-10-24 04:09:05 +02:00
jschoubben 443ba6b265 Assert the three-repository walk takes each first-tier module from its key at its commit (issue 1931) 2026-10-24 04:03:53 +02:00
jschoubben a96bd2ff38 Mark the packages whose tests read files outside their directory, so mesh-build-go never keeps their verdict (issue 1935) 2026-10-24 04:00:41 +02:00
jschoubben 64fbef7713 Assert the cut takes its first tier from the key at the carried commit, since the cut asks no build (issue 1931) 2026-10-24 03:59:10 +02:00
jschoubben a77aa8520e Hold tier 1's key in the away test and count one shared key in the backups test, since a walk takes keys at once now (issue 1931) 2026-10-24 03:47:45 +02:00
jschoubben a9de578cb9 End the reconnects test's verb in the serving process: a command verb re-runs the test binary (issue 1931) 2026-10-24 03:41:40 +02:00
jschoubben d951aa954e Assert a Retry waits for the builder's key, since the controller asks no build now (issue 1931) 2026-10-24 03:25:05 +02:00
jschoubben ab9a7771f4 Call a verb that still exists in the reconnects test, since queue was removed (issue 1931) 2026-10-24 03:20:38 +02:00
jschoubben 1c50608246 Merge pull request 'Give route-proxy a build the fixed controller takes in, dropping the incident note #450 added (issue 1933)' (#452) from fix/1933-rebuild-route-proxy-2 into main
Reviewed-on: #452
2026-10-24 01:00:18 +00:00
jschoubben 760351b4ae Merge pull request 'The sweep keeps a fresh build's files, and a take-in holds every file before it records a build (issue 1933)' (#451) from fix/1933-sweep-keeps-fresh-builds into main
Reviewed-on: #451
2026-10-24 00:54:55 +00:00
jschoubben 899389c710 Reword route-provider's stdout line, so route-proxy's tree differs from the 01:52 build whose files are gone (issue 1933)
mesh/delivery delivered
mesh/check checking 899389c7106c on 760351b4ae1b
2026-10-24 02:50:12 +02:00
jschoubben a203f605d4 Give route-proxy a build the fixed controller takes in, dropping the incident note #450 added (issue 1933) 2026-10-24 02:49:56 +02:00
jschoubben 0741880d31 Keep a fresh build's archives from the controller's own sweep (issue 1933)
mesh/delivery delivered
mesh/check checking 0741880d3137 on 760351b4ae1b
A sweep read what it keeps before what was ever recorded, so a take-in's
build recorded between the two reads, by a concurrent take-in off the
plans' hold, was eligible and let go of seconds after the take-in read
it; a new module's build is unkept until it is registered. Now the sweep
reads the records first and keeps what a build recorded within the hour
or an open plan names, and names every reference it lets go of; a take-in
holds every artifact in the store before recording, failing on one the
store lacks; and RecordBuild clears artifact_collected as RecordMirrored
does.
2026-10-24 02:44:20 +02:00
jschoubben fa734d3098 Merge pull request 'Give route-proxy a new build: the controller's sweep removed its last build's files from the store, so novox and ace cannot apply (issue 1933)' (#450) from fix/1933-rebuild-route-proxy into main
Reviewed-on: #450
2026-10-24 00:39:47 +00:00
jschoubben f13d42abb4 Give route-proxy a new build: the controller's sweep removed its last build's files from the store, so novox and ace cannot apply (issue 1933)
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestAWarrantForRetryRetriesAFailedPlanInTheStoreAndEndsItsConditi…
2026-10-24 02:37:06 +02:00
jschoubben fdf26a1fe2 Merge pull request 'A module the catalogue no longer holds ends deleted instead of failing the plan; vendor/ goes; tests name verbs that exist (issues 1922, 1305, 1931)' (#449) from fix/1922-removed-module into main
Reviewed-on: #449
2026-10-23 23:47:05 +00:00
jschoubben 343e271ec7 Integration tests stop naming the verbs R4g removed (issue 1931)
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestAWarrantForRetryRetriesAFailedPlanInTheStoreAndEndsItsConditi…
2026-10-23 23:21:39 +02:00
jschoubben 05c872485a Delete the committed dependencies; builds resolve by go.sum (issue 1305) 2026-10-23 22:16:09 +02:00
jschoubben 12dd2bb3f0 A module the catalogue no longer holds ends deleted instead of failing the plan; the unsent hint names release-backlog (issue 1922) 2026-10-23 22:15:18 +02:00
jschoubben 5831eec770 Merge pull request 'R4g+R5 (mesh-controller): the controller asks no build, build-agent and the merge gate go, register by reference (design 47)' (#448) from feat/r4g into main
Reviewed-on: #448
2026-10-23 17:50:22 +00:00
jschoubben c9a7102bc5 Set aside a retired seat's consume the trunk catalogue still holds, and say it, so a group that retires a seat passes its check
mesh/delivery failed: its walk failed, carrying its own commit: a gate on a first machine (what it carried put back), a build, a machine
mesh/delivery-group group feat/r4g merged: every member is merged
mesh/check checking c9a7102bc53b on 791786d5b87f | novox/mesh-controller#448 in group feat/r4g
2026-10-23 18:05:06 +02:00
jschoubben 67279199e9 Say the trunk rule as ADR 0534 §6 has it in the registration's comment
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: github.com/novox/mesh-controller/internal/broker.TestTheCataloguesDerivedSubjectsMeet: agreement_catalogue_test.go:130: judge…
2026-10-23 17:19:39 +02:00
jschoubben 9d2a9e62d1 Merge remote-tracking branch 'origin/fix/forge-requires-no-status' into feat/r4g 2026-10-23 17:18:58 +02:00
jschoubben b345d10bec Pin mesh-sdk at feat/r4g's head a2c813a6, its test fixture fix, with go.sum tidied
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: git.novox.be/novox/mesh-catalog/checks/config.TestTheUnmovedListHoldsNoModuleTheTrunksDoesNot: realtree_test.go:14: mesh-cata…
2026-10-23 17:06:08 +02:00
jschoubben 13842a7234 Grant the builder the build seats' streams' information and messages by name, so its take-over at start can tell an ask its key names is on its stream (issue 1910)
mesh/delivery-group group feat/r4g rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: g14: novox/mesh-catalog#464: checks/health/go.mod names mesh-sdk a9560890ee26, which is neither on mesh-sdk's trunk nor the group's…
2026-10-23 16:59:03 +02:00
jschoubben 4238318601 Pin mesh-sdk at feat/r4g's head a9560890, re-vendor it and capture its conformance events there (R4g) 2026-10-23 16:54:07 +02:00
jschoubben 758c8e84e5 Bound the build seats' workers' deliveries, so an ask no run ends is offered at most five times even when the holder that says it is poison dies too (issue 1907) 2026-10-23 16:42:13 +02:00
jschoubben 9222575ae9 Vendor the dependencies again for R4g's go.mod (mesh-sdk 9205222a, the mesh-host replace as before), so build-agent compiles the controller offline in the group's first tier (issue 1305) 2026-10-23 16:38:31 +02:00
jschoubben c5b8fca8bd Revert "Delete vendor/ and the route proxy's: the build module fetches the modules, the root image builds with -mod=mod (issue 1305, C6)"
This reverts commit 9c4bf1f08b.
2026-10-23 16:37:40 +02:00
jschoubben ff543d3def Take the watchman's grant on the build seats back: a waiting ask has no pulse, a taken one is read from the steps (issue 1908) 2026-10-23 16:35:28 +02:00
jschoubben ef211faefb Prove the registration by hand reads back as a build and is held by the sweep, and name the store check as the test's limit (issue 1888, C8b) 2026-10-23 16:34:29 +02:00
jschoubben 3c247e80af Fix the view grant test's consumer subjects (issue 1887 A8) 2026-10-23 16:32:43 +02:00
jschoubben bb0d5e17db Grant the watchman the build seats' accept subjects and the view the seat workers' information, each by name, so the backlog and the waiting-ask pulse read live (issue 1887 A8) 2026-10-23 16:32:32 +02:00
jschoubben 24ad859629 Record the checksums of mesh-sdk feat/r4g 9205222a for the group's modules (R4g) 2026-10-23 16:26:11 +02:00
jschoubben 5ccc87f4d0 Capture mesh-catalog at feat/r4g (2246ccc4) with the templates its manifests name and its predefined.json, and lay the copy out as a repository's root
The catalogue's manifests moved their content into template files and fall back to predefined keys, so a copy of
module.json alone no longer passes the catalogue's own check; the bus's Dockerfile is gone from the catalogue.
2026-10-23 16:24:14 +02:00
jschoubben 68c1220e87 Pin mesh-sdk at feat/r4g's head 9205222a, the merge of lanes A, B, S8 and F1, and capture its conformance events there
go.sum gains its lines from go mod tidy once mesh-sdk feat/r4g is pushed.
2026-10-23 16:22:07 +02:00
jschoubben 5b5cd6160a Merge branch 'feat/r4g-asks' into feat/r4g 2026-10-23 16:20:32 +02:00
jschoubben d7d70cd539 Merge branch 'feat/r4g-assigner' into feat/r4g 2026-10-23 16:20:32 +02:00
jschoubben df70ca43a7 Take the controller's publish grants on node-build-agent and mesh-build-machine away, with the cancelled sets and the redelivery of its own asks: it asks no build seat (ADR 0525 §10, C3a) 2026-10-23 16:13:43 +02:00
jschoubben f60c742d93 Prove the grant through the send of the bus's machine (overTheBus.declare), so removing the write from the live path fails the test (issue 1394, review round 1 item 4) 2026-10-23 16:13:42 +02:00
jschoubben 2db5dcdcb4 Write the hand act of register --reference after the registration was made, not before, and test the why and the record (issue 1888, C8b) 2026-10-23 16:12:16 +02:00
jschoubben d9cdb3615d Test that the builder's record names its key and fingerprint after a plan that moves it (issue 1851, C5 criterion) 2026-10-23 16:11:07 +02:00
jschoubben dec3b83e66 Test that the builder, a build module and the controller are taken from the builder's keys and nothing reaches node-build-agent (issue 1851, C2 criterion) 2026-10-23 16:11:07 +02:00
jschoubben 2e36a63096 Pin mesh-sdk at feat/r4g's head c3915ffb, the merge of lane B's layouts, so every r4g module builds against one SDK
go.sum gains its lines from go mod tidy once mesh-sdk feat/r4g is pushed.
2026-10-23 16:01:15 +02:00
jschoubben eb8286fc68 Merge feat/r4g-assigner into feat/r4g: lane B's grants bucket and its writer beside lane C's deletions
The controller's readable states, its bucket lists and the composed golden file keep lane C's removal of the
checks-running and plans-building buckets and the build agent's check subjects, and add lane B's grants bucket.
2026-10-23 15:59:52 +02:00
jschoubben 5540fd4f85 Find the catalogue's repository by its URL or seat source, and fail a plan whose catalogue repository is nowhere, so a take-in from a key no longer waits for ever (issue 1902) 2026-10-23 15:57:30 +02:00
jschoubben e419d0182f Delete the builder, the build asks, the build queue and the merge gate: the builder's keys are the only way a module is built (issues 1851, 1299, 1318; C2b-C5, C10)
The controller asks no build of anybody (ADR 0501 §5, ADR 0525 clause 6, ADR 0503 §3). Every module of a walk is taken from the
builder's done key; a key not done yet waits, named in the plan, and never fails it.

- C2b/C2c: internal/builder and cmd/mesh-controller's asks go (askABuild, onlyTheBuildersOwn, buildOneAsked, takeIn from an
  outcome, buildAgentsOwn, the builder's say and its fallback, the daemon hearing an outcome). The server follows the retired
  seats' built subjects only to let go of them, because the installer's carried user list names them.
- C2d: replay, queue, cancel, clear, kill, pause, resume, delivery-check, check-here and merge-gate are refused by name.
- C3: the check queues (CHECKS_READY, CHECKS_OTHER) and their workers, the checks-running and plans-building buckets and
  their grants, the build queue and its hold (plans_building.go, queue.go, link/holds.go, builds_nats.go, checks_nats.go).
- C4: merge_gate*.go, the pull request's repo-check path, the Checked event and its layers; a pull request asks the
  controller nothing but to tell mesh-config what its head declares.
- C5: the builds bucket is written only by key_takein.
- C10 (help and refusals): the terminal's help lists only what is served.

Integration tests of the plan's tiers, sends, stops and retries now run on a stand-in builder with a done key for every
module (walk_keys_scaffold_integration_test.go); the tests of the asks, the queue, the merge gate and the replays are deleted.
2026-10-23 15:55:15 +02:00
jschoubben 5ca4f14479 Derive a worker per kind for the build seats and grant their users to put asks, so a build node pulls delivery, check and recheck in order and a session cannot ask (issue 1887 A6, ADR 0525 §2) 2026-10-23 15:54:45 +02:00
jschoubben c43883910d Write grants.<module>.<fp> once the bus's user list carrying a module's manifest at that fingerprint is sent, so the assigner's sixth rule reads that its credential exists (issue 1394, subtask C9b; ADR 0528 clause 2, ADR 0429 §6) 2026-10-23 15:39:40 +02:00
jschoubben 548681cc4e Declare the controller's grants bucket, written by the controller alone and readable through mesh-controller.grants, so the assigner can read its sixth rule's key (issue 1394, subtask C9a; ADR 0528 clause 2); the SDK pin moves to the layouts' commit, captured beside 2026-10-23 15:39:40 +02:00
jschoubben 9c4bf1f08b Delete vendor/ and the route proxy's: the build module fetches the modules, the root image builds with -mod=mod (issue 1305, C6) 2026-10-23 15:14:58 +02:00
jschoubben 6300a5b31c Delete merge-check.sh, check-nested-modules.sh, the store-tests list and RELEASE-57: the checker runs the build module's steps, nested modules by convention (issue 1294, C7a and C7b) 2026-10-23 15:14:01 +02:00
jschoubben 505694c424 Register a module at artifacts the store holds, by hand, recorded as a build and a hand act, so the controller can be delivered without the builder (issue 1888, C8a and C8b) 2026-10-23 15:13:17 +02:00
jschoubben 72186815e3 Delete the build-agent module and the builder program: the build module builds, the checker checks (issue 1851, C2a) 2026-10-23 15:07:57 +02:00
jschoubben 526eed99fa Drop the toolchain image from the facts and the dependency edges: no module is compiled in one any more (issue 1851, C1b) 2026-10-23 15:07:29 +02:00
jschoubben b506852388 Move the Go build source reader out of internal/builder so the builder can go (issue 1851, C1a) 2026-10-23 15:06:31 +02:00
mesh-tools 791786d5b8 Merge pull request 'Wait out a resolver that does not answer in a build's registry step, and say it was the resolver (issue 1748)' (#447) from fix/1748-registry-wait-retries-a-lookup into main 2026-10-23 11:20:15 +00:00
jschoubben 0b312218f9 Wait out a failed lookup of the registry's name in a build, and say it was the resolver (issue 1748)
mesh/delivery delivered
mesh/check pass 0b312218f90f on c834e4f82137 | unit 1762 run, 0 kept, 4 skipped | integration 3040 run, 0 kept, 16 skipped | over bound: integration (…
2026-10-23 13:03:17 +02:00
mesh-tools c834e4f821 Merge pull request 'Keep every refused build by its id, so a plan never settles a refused build built and sends the old version (issue 1713)' (#446) from fix/1713-keep-every-refusal into main 2026-10-23 10:58:08 +00:00
jschoubben 149f89e8f9 Keep every refused registration by its build id, so a Retry never settles it built
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792753090890815254 (novox/mesh-controller at c834e4f8 into ma…
mesh/check pass 149f89e8f9bf on 0173d9c70e59 | unit 1759 run, 0 kept, 4 skipped | integration 3037 run, 0 kept, 16 skipped | over bound: integration (…
A build heard and recorded and not registered kept its refusal only when a
plan was open or the plans were held. A tier's first failure fails its plan,
so later refusals were lost; a Retry then settled those builds built from
their records, sent the modules' old versions and reported the plan done.
A build without a readable manifest never reached its plan at all.

Keep the refusal in the store and the process whatever the plans' state, and
fail the asking plan by the build's id when the module is not known.

Issue 1713.
2026-10-23 12:13:33 +02:00
jschoubben b2af3d654f Trust a trunk whose whitelist adds the operator's own login, never an agent's (ADR 0534 §2, §6)
The gitea module keeps the operator's login beside mesh-tools on every main so
the operator can merge by hand; an agent's account there is still refused.
2026-10-23 12:10:11 +02:00
jschoubben 7676e67c5e Trust a module's trunk for who can merge into it, not for a required status (ADR 0534 §6)
The forge requires no status to merge any more (ADR 0534 §1); a trunk is
trusted when it takes no push, its merge whitelist names mesh-tools alone and
no administrator merges past its rule, so only mesh-delivery lands a commit
there, on a passing mesh/check and the operator's Approve. The old rule failed
the 11:51 delivery plan once mesh-catalog's status was unticked.
2026-10-23 12:02:00 +02:00
mesh-tools 0173d9c70e Merge pull request 'Keep the admin address's query in a test database's address, so store tests reach the checker's socket store (issue 1889)' (#445) from fix/1889-identity-store-ready into main 2026-10-23 07:37:06 +00:00
mesh-tools 89abce5df9 Merge pull request 'Judge each machine on a plan's last send and resume a plan lost to silence with plans retry, so a resumed plan is not failed by its own old sends (issue 1886, ADR 0531)' (#444) from fix/1886-resume-times into main 2026-10-23 07:26:31 +00:00
jschoubben 185d0294eb Keep the store address's query in every test database's address, so store tests reach a socket-addressed store (issue 1889)
mesh/merge-gate head 185d0294ebd4 merged into trunk 89abce5df955 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; the ch…
mesh/repo-check head 185d0294ebd4 merged into trunk 89abce5df955 — pass: its merge-check.sh passed
mesh/delivery covered after its stop: a later merge that contains it was delivered: novox/mesh-controller@0b312218f90f (merged as 791786d5 into main, wal…
mesh/check pass 185d0294ebd4 on 89abce5df955 | unit 1756 run, 0 kept, 4 skipped | integration 3034 run, 0 kept, 16 skipped | over bound: integration (…
The checker's integration store answers on a unix socket named in the query of MESH_TEST_POSTGRES
(?host=<socket dir>&sslmode=disable). The identity, inventory and licences fixtures cut the address at
its last slash and appended ?sslmode=disable, dropping host=, so pgx dialled its default socket where no
server is: every store test waited out its 20 s readiness bound, each package ran into go test's
10-minute timeout, and the verdict named only the first failure,
identity.TestACertificateChainsToTheMeshsOwnAuthority.

store.DatabaseForTest now builds that address for every fixture, keeping the query as the template raise
already did; identity's fresh is ForTest rather than a second copy. The readiness bound and its error,
which wraps the driver's cause, are unchanged.
2026-10-23 08:55:50 +02:00
jschoubben 4d4d51eb01 Pin the resume of a lost walk's refusals beside a newer or a related plan, and cite ADR 0360 §7 and ADR 0531 (issue 1886)
mesh/merge-gate head 4d4d51eb017f merged into trunk 219fdcd7f00c — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; the ch…
mesh/repo-check head 4d4d51eb017f merged into trunk 219fdcd7f00c — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-23 08:10:08 +02:00
jschoubben b7d996878b Judge a walk's machine on its last send, enter a resumed tier anew, and resume a walk lost to silence (issue 1886)
R4e, resumed at tier 1 after a stop of five hours, named ace and shanks silent within seconds of tier 1's send: the
walk judged each machine on every send it had made it, tier 0's of 00:23 too, held against tier 1's templates, and
measured the bound from it. A send carries the whole declaration, so the last one is what a machine is judged on.
The resume kept the tier's entry of before the stop, so the plan read waiting for 5h48m. And plans retry refused a
walk failed only by the gate's silence, though its machines had since applied.
2026-10-23 08:02:11 +02:00
mesh-tools 219fdcd7f0 Merge pull request 'Resume a delivery plan stopped by hand with plans retry, so a stop is not the end of what its tier waited on (issue 1886)' (#443) from fix/1886-retry-stopped into main 2026-10-23 05:40:52 +00:00
jschoubben 55ba7f2cb8 Hold every refusal of the stopped path in the test ADR 0523 names: a newer open plan of the branch, a newer plan of a build not sent (issue 1886)
mesh/merge-gate head 55ba7f2cb83c merged into trunk 296e11d499cb — pass: builds mesh-controller → novox; no bus step; the change touches no module of the m…
mesh/repo-check head 55ba7f2cb83c merged into trunk 296e11d499cb — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check checking 55ba7f2cb83c on 219fdcd7f00c
2026-10-23 02:44:58 +02:00
jschoubben 102f8dc267 Judge a resume only by the builds its tier holds, and ask build-agent's own of build-agent (issue 1886, review round 1)
mesh/merge-gate head 102f8dc2674e merged into trunk 296e11d499cb — pass: builds mesh-controller → novox; no bus step; the change touches no module of the m…
mesh/repo-check head 102f8dc2674e merged into trunk 296e11d499cb — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-23 02:34:59 +02:00
jschoubben 296e11d499 Merge pull request 'Ask build-agent until the builder says it writes references, so a plan never waits on a key nothing can reference (issue 1886, ADR 0522)' (#442) from fix/1886-builder-says-references into main
Reviewed-on: #442
2026-10-23 00:27:00 +00:00
jschoubben eed4976483 Resume a walk stopped by hand with plans retry, so a stop is not the end of what its tier waited on (issue 1886) 2026-10-23 02:05:16 +02:00
jschoubben bbadbb2af0 Judge the ask and the settle by one say of the builder (issue 1886, merge check hang)
mesh/merge-gate head bbadbb2af021 merged into trunk 289bbf68e4f9 — pass: builds mesh-controller → novox; no bus step; the change touches no module of the m…
mesh/repo-check head bbadbb2af021 merged into trunk 289bbf68e4f9 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/1886-builder-says-references merged: every member is merged
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
The settle read the say from the builder's bucket while the ask read it through
builderSaysReferences; where they differed (a test's say against a bus with no
bucket) a module was handed back and taken from its key again at every turn,
and the look's loop never ended.
2026-10-23 01:59:18 +02:00
jschoubben 1414ed7f7b Judge an ask by the builder's say the walk read, never a second read (issue 1886, review round 1)
mesh/delivery-group group fix/1886-builder-says-references checking: 0 of 2 member(s) ready
mesh/merge-gate building on ace since 01:38
mesh/delivery superseded: a newer head of the same pull request
A second read that failed, or that turned true in between, refused the ask and
failed the whole plan.
2026-10-23 01:03:19 +02:00
jschoubben f67342f0d8 Ask build-agent for the build modules that write references, whatever the builder says (issue 1886, ADR 0522 §6)
They push and answer each reference, so a key they write themselves can never
carry their own delivery; until R4g build-agent delivers them as it does the
builder.
2026-10-23 00:53:25 +02:00
jschoubben fae7710bed Ask build-agent for every module until a running builder says it writes references (issue 1886)
R4e's own delivery plan waited for ever in tier 1: the controller took every
module but build-agent's own from a referenced builder key, and only R4e's
builder and build modules, delivered by that same plan, write references.
The builder now says so in its bucket of instances; until it does, a walk asks
build-agent as before R4e, and an open walk waiting on a key the running
builder cannot reference is judged again at the next look.
2026-10-23 00:47:15 +02:00
mesh-tools 289bbf68e4 Merge pull request 'R4e (mesh-controller): deliver from the builder's keys, ask build-agent only for its own modules, follow the forge's protection for checks, announce the controller's states' (#441) from feat/r4e into main 2026-10-22 22:12:58 +00:00
mesh-tools f2a4363a1a Merge pull request 'Clone from the forge again when the resolver does not answer, so a passing lookup failure does not fail a build (issue 1773)' (#440) from fix/1773-clone-retries-a-lookup into main 2026-10-22 19:44:36 +00:00
jschoubben d51324df45 Format the help's test (issue 1851)
mesh/repo-check head d51324df4518 merged into trunk f2a4363a1a77 beside mesh-catalog#457@896bd883 — pass: its merge-check.sh passed
mesh/merge-gate head d51324df4518 merged into trunk f2a4363a1a77 — pass: group feat/r4e composed: every machine composes with the change as it did without …
mesh/delivery covered after its stop: a later walk that contains it was delivered: plan-1792707174627079192 (novox/mesh-controller at 289bbf68 into main,…
mesh/delivery-group group feat/r4e closed: a member was closed before the group merged
mesh/check failed: unit: ..test: node:internal/modules/esm/resolve:989 | d51324df4518 on f2a4363a1a77 | unit 1 run, 0 kept | integration 0 run, 0 kept…
2026-10-22 21:20:06 +02:00
jschoubben 58512a9c6d Pin mesh-sdk at the R4e member's final head 104533e8 and vendor it (R4e E)
The SDK's S1 and S2 review fixes landed after 875c680d.
2026-10-22 21:19:46 +02:00
jschoubben 1fd833bfe8 List the store tests R4e adds, and drop those of the hand builds and the stopped checks it deleted (issue 1851, issue 1299)
The merge check refused the eight deleted tests as listed and not run.
2026-10-22 21:18:58 +02:00
jschoubben fd4d26b42c Keep TestReplay867 off the build node's real free space (issue 1834, note 12)
mesh/merge-gate head fd4d26b42cd7 merged into trunk 96441ea28a84 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head fd4d26b42cd7 merged into trunk 96441ea28a84 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
The check's floor of free space reads the disk the test runs on, so
every mesh-controller check failed while its build node had under
20 GiB free, whatever the change (seen in the try of issue 1773's fix).
The replay stubs the free space, as internal/builder's tests do.
2026-10-22 21:17:42 +02:00
jschoubben f9014452d3 List register and switch in the help, and none of the removed commands (issue 1851 A11a, A11b)
The help still offered build, rebuild and replay --register, each now
refused as removed, and did not name the terminal's register.
2026-10-22 21:10:20 +02:00
jschoubben 61449249a8 Name build-agent's own as the builder's module, build-agent and mesh-tools (ADR 0520 §1, issue 1851)
ADR 0520 narrowed after its review: mesh-tools-go builds from its own
directory on a pinned base and has its key from mesh-build-image, and
mesh-tools-python is no module, so the set is named, not derived from
the toolchain bases.
2026-10-22 21:08:47 +02:00
jschoubben 61f7e89ad5 Ask a repository check whenever the trunk requires it, beside mesh/check (issue 1299 C1, ADR 0513 §4)
A trunk requiring both contexts (hq) matched mesh/check first and had
its own merge-check.sh never asked.
2026-10-22 21:07:55 +02:00
jschoubben 9713ceb84d Ask build-agent for itself and its toolchain images, never a builder key (ADR 0520, issue 1851)
The builder writes no key of build-agent or of the toolchain bases its
builds stand on, so a walk moving one would hold its tier for ever.
Until R4g they are build-agent's own, as the builder's module is; the
bases are read from the toolchains, not copied.
2026-10-22 21:06:33 +02:00
jschoubben 93fc2f48a1 Name a retry's modules still waiting for the bus, read as the plan it resumes (issue 1851 A11a, issue 1403)
With the hand rebuild gone, a retry of a plan failed past the bus's bound
read the waiting modules while the plan still said failed, named none,
and answered "asked" with nothing after it.
2026-10-22 21:05:01 +02:00
jschoubben acaf8830ec Try a forge clone again when its name lookup fails (issue 1773)
A build or check failed now and then on 'Could not resolve host:
novox.internal' between clones that resolved the same name, and its
whole delivery failed with it. A clone whose lookup failed reached
nothing, so it is cloned again with a growing pause for up to 60 s,
each retry said in the build's log as the resolver not answering;
past the bound it fails saying so. Any other git failure fails at
once, as before. Covers a module's own clone, an artifact's context
and a check's repositories.
2026-10-22 20:56:46 +02:00
jschoubben d5d53893a4 Send the bus its user list before a starting controller writes (issue 1800)
A new controller's grants reached the bus only after its gate, so the
buckets it newly writes were refused until then. At start it now sends
the bus's machine the user list alone, every build there kept.
2026-10-22 20:51:08 +02:00
jschoubben a0b2d98002 Merge main (upgrade safety's declared packages, issue 1840) and pin the R4e mesh-sdk member's final head 875c680d, re-vendored here and in modules/route-proxy; the controller announces declared-packages readable (issue 1790, D1; E) 2026-10-22 20:44:59 +02:00
jschoubben bc80683b9e Count the work the take-ins start in the background, so a test's mesh closes only once that work has ended (issue 1851, A5-A7; issue 1790, D3) 2026-10-22 20:41:30 +02:00
mesh-tools 96441ea28a Merge pull request 'Keep each node's declared packages in a state the pacman module reads (upgrade safety, issue 1840, ADR 0507 §5)' (#439) from feat/upgrade-safety into main 2026-10-22 18:34:19 +00:00
jschoubben 420ae3d0e7 Test the group check's answer without swapping standard output, and wait for a take-in to forget a new module's wait (issue 1299, C2; issue 1851, A5) 2026-10-22 20:19:57 +02:00
jschoubben 7c0e1d223f Serve a removed verb as its refusal, not as a verb this build is behind on, and fit the tests of the verbs and the stopped checks (issue 1851, A11a; issue 1299, C1; issue 1318, C3) 2026-10-22 19:38:02 +02:00
jschoubben a10ef8995b Vendor what go.mod pins after the rebase on main (issue 1790; the SDK's 1833 fix returns with E's re-pin) 2026-10-22 19:05:21 +02:00
jschoubben dfc2f2aa98 Take a walk's key in again when it was recorded at an earlier merge, so the module is registered at the commit the walk carries before it is built (issue 1851, A6) 2026-10-22 19:05:05 +02:00
jschoubben ab33f015ec Say again what the controller keeps at every push, so a word refused before its grant is kept once a push carries it (issue 1790, D1) 2026-10-22 19:05:05 +02:00
jschoubben 791c2a3d48 Refuse a removed verb by name before reading its arguments, as the SDK member's table now offers none (issue 1851, A11a; issue 1318, C3) 2026-10-22 19:05:05 +02:00
jschoubben d36e0ea278 Announce the controller's states, keep every module's word, and grant a read only where its owner announced the state readable; the copied list of the controller's mirrors goes (issue 1790, D1-D4, ADR 0495, ADR 0513 §3) 2026-10-22 19:05:04 +02:00
jschoubben b6ed18197b Vendor the R4e mesh-sdk member's head with the states announcement (issue 1790; re-pinned to the final head in E) 2026-10-22 19:05:04 +02:00
jschoubben 45d091778c Refuse delivery-plan and forget-checks by name and take a merge's warrant from the teller alone, no longer mesh-delivery's (issue 1318, C3, ADR 0450 §8) 2026-10-22 19:05:04 +02:00
jschoubben 3a1937914e Ask build-agent no composed check of a delivery group, and answer that the checker judges it (issue 1299, C2, ADR 0503 §3) 2026-10-22 19:05:04 +02:00
jschoubben 25fdf9920e Read a commit as the same as itself however short, and fit the pending assignments' tests to a mesh that asks no build (issue 1851, A9, A10) 2026-10-22 19:05:04 +02:00
jschoubben 323e1270cc Ask no pull request check of build-agent but the own merge-check.sh of a repository whose trunk requires mesh/repo-check alone, and say every verdict with no gate layer (issue 1299, C1, ADR 0503 §3, §5) 2026-10-22 19:05:04 +02:00
jschoubben e7e23c57cc Read a module whose every artifact names its paths from those paths and its manifest alone, so a change elsewhere in its directory moves nothing (issue 1758, B1, ADR 0499 §5) 2026-10-22 19:05:04 +02:00
jschoubben a8387ebad8 Name, before the switch, which modules it moves: the held build's digests against the builder's done key, images and packages said as what cannot be compared (issue 1851, A13) 2026-10-22 19:05:04 +02:00
jschoubben 1963d84a7a Refuse, in the one function every ask goes through, an ask of build-agent for any module but the builder's own (issue 1851, A12, ADR 0453 §5) 2026-10-22 19:05:04 +02:00
jschoubben b71425867f Register a module from a repository the catalogue does not build at the controller's terminal alone, from its done trunk key, naming exactly its repository and path (issue 1851, A11b, ADR 0501 §4) 2026-10-22 19:05:04 +02:00
jschoubben c7039d81e1 Refuse build, rebuild and replay --register by name, naming what replaced them, and delete the hand builds (issue 1851, A11a, ADR 0501 §5) 2026-10-22 19:05:04 +02:00
jschoubben f76da2ce9c Ask the checker for a try of a branch head, not build-agent, and read its word in today's answer (issue 1851, A11, ADR 0501 §5) 2026-10-22 19:05:04 +02:00
jschoubben 7f6d418205 Mark the walks the integration tests drive around the one ask left as the builder's own, and expect no ask of a moved module (issue 1851, A5, A6) 2026-10-22 19:05:04 +02:00
jschoubben 0aec975d19 Keep an assignment of a module whose key is coming pending on that key and ask no build; nothing asks a request again at start (issue 1851, A10) 2026-10-22 19:05:04 +02:00
jschoubben 9da79ef8c3 Move no module whose fingerprint is unchanged, and let a key's build stand on the held build-agent build of the same digests at the switch (issue 1851, A9) 2026-10-22 19:05:04 +02:00
jschoubben 7743bd0d4b Delete the walk's reuse of a check's build: a walk takes keys and asks build-agent only for the builder's own module (issue 1851, A8) 2026-10-22 19:05:04 +02:00
jschoubben 62268cf43d Take a walk's modules from the builder's done keys and ask no build but the builder's own: a key building or unreferenced holds the tier and says so, a failed key fails the walk in its words (issue 1851, A6, A7) 2026-10-22 19:05:04 +02:00
jschoubben ea3c8ad7f6 Register a new module from the builder's first done key and ask no first build; its condition reads the key and names a manifest word this controller does not read (issue 1851, A5a, A5b) 2026-10-22 19:05:04 +02:00
jschoubben d984f0a55d Take a module in from the builder's done key of its trunk fingerprint: one record of the key's artifacts, its manifest judged against the catalogue trunk's predefined keys (issue 1851, A3, A4b, A4c) 2026-10-22 19:05:04 +02:00
jschoubben f00b6aff8b Judge a recorded build, not build-agent's outcome, and read a registration's warrant from the merge a key names, so a done key registers under the same rules (issue 1851, A4a) 2026-10-22 19:05:04 +02:00
jschoubben eff0d5cbf4 Vendor the R4e mesh-sdk member's head for its shared launcher rule and the manifest word paths (issue 1851; re-pinned to the final head in E) 2026-10-22 19:05:04 +02:00
jschoubben 1ed7e34155 Read the builder's keys, the indexer's values by commit and its manifests, so the controller can take a module from a done key (issue 1851, A1, A2) 2026-10-22 19:05:04 +02:00
jschoubben abe356a262 Merge main and take the SDK at the milestone's mesh-sdk head 8892a117, which holds main's SDK and the milestone's (issue 1840)
mesh/merge-gate head abe356a262e8 merged into trunk 41c96287578e — pass: group feat/upgrade-safety composed: every machine composes with the change as it d…
mesh/repo-check head abe356a262e8 merged into trunk 41c96287578e beside mesh-catalog#450@86ba54b0 — pass: its merge-check.sh passed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792694058124013462 (novox/mesh-controller at 96441ea2 into ma…
mesh/delivery-group group feat/upgrade-safety merged: every member is merged
mesh/check failed: unit: ..test: node:internal/modules/esm/resolve:241 | abe356a262e8 on 41c96287578e | unit 3372 run, 0 kept, 7 skipped | integration…
2026-10-22 18:14:14 +02:00
mesh-tools 41c9628757 Merge pull request 'Pin mesh-sdk at the fix that refuses an unresolvable machine alone (issue 1833)' (#437) from fix/1833-one-machine-alone into main 2026-10-22 16:10:20 +00:00
jschoubben 3e3400f510 Take the SDK at the milestone's mesh-sdk head f8b8f3a6, which the delivery group's members must pin exactly (issue 1840) 2026-10-22 18:07:38 +02:00
jschoubben e022a3fd6f issue 1840/S3: look the declared packages up by name in the mirror's integration test, since every node also runs the mesh's own modules and their packages 2026-10-22 17:52:40 +02:00
jschoubben 684a8783e5 Merge remote-tracking branch 'origin/main' into feat/upgrade-safety 2026-10-22 17:40:00 +02:00
mesh-tools 11e364bd43 Merge pull request 'Hold a first merge whose new module the running build-agent refuses for an unknown manifest word, name it, and ask it again when build-agent is delivered (issue 1173)' (#438) from fix/1173-held-first-build into main 2026-10-22 15:28:11 +00:00
jschoubben 1e9b87e03a Hold every refused module of a first merge and ask each again by its own path (issue 1173)
mesh/merge-gate head 1e9b87e03acd merged into trunk b8f77577fed2 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 1e9b87e03acd merged into trunk b8f77577fed2 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
A held merge was asked again by the rule of a first merge, which finds nothing once a sibling module of the
same merge registered, so it stayed held forever; and a second refusal in a merge already held fell through to
a failed build. The hold now keeps a list of module, path and word, a refusal in a held waiting merge joins it,
and the delivery of build-agent asks each held path again on the merge's Approve warrant.
2026-10-22 16:50:06 +02:00
jschoubben 45a09f90e3 Carry a held merge's new announcement into its record (issue 1173)
HoldMerge replaced the event on the flag only; the record kept the old one.
2026-10-22 16:33:28 +02:00
jschoubben 4538d05732 Ask a held first build again when a delivery of build-agent ends (issue 1173)
ADR 0482 section 5: the build waiting for a build-agent that reads its manifest is asked once the plan delivering it ends.
2026-10-22 16:19:45 +02:00
jschoubben 980d396d40 Keep a first merge whose manifest word the running build-agent does not read, and name it (issue 1173)
ADR 0482 section 5: a merged manifest the running reader refuses waits for the newer reader, never failed.
2026-10-22 16:18:39 +02:00
mesh-tools b8f77577fe Merge pull request 'Say a delivery's start and end with the machines its merges declare, and word a needed new head as its author's (ADR 0505)' (#436) from feat/quiet-deliveries into main 2026-10-22 14:01:22 +00:00
jschoubben 28a278c146 Take mesh-sdk at bd7731f: a machine that does not resolve keeps its name and its address in the mesh as well as its tunnel (issue 1833)
mesh/repo-check head 28a278c1462a merged into trunk 11e364bd439f — pass: its merge-check.sh passed
mesh/merge-gate head 28a278c1462a merged into trunk 11e364bd439f — pass: group fix/1833-one-machine-alone composed: every machine composes with the change …
mesh/delivery delivered
mesh/delivery-group group fix/1833-one-machine-alone merged: every member is merged
mesh/check failed: g14: go.mod of novox/mesh-controller#437 cannot be read to place the mesh-sdk its tests read: nats: no responders available for req…
2026-10-22 15:27:27 +02:00
jschoubben c34ab0e43a Take mesh-sdk at c154bde: a machine that does not resolve keeps its place on the private network, so it alone is refused (issue 1833)
One machine whose modules could not be resolved left the private network uncomposable when it was the hub, and every
send to every machine was held. The SDK now keeps a refused machine at the place the controller already holds for it.
2026-10-22 15:04:47 +02:00
jschoubben 771cc24fcb Take the SDK whose process can delete a key (issue 1840/S1) 2026-10-22 14:37:18 +02:00
jschoubben b3786cc67e Merge remote-tracking branch 'origin/main' into feat/upgrade-safety
# Conflicts:
#	go.mod
#	go.sum
#	testdata/beside/CAPTURED
#	vendor/modules.txt
2026-10-22 14:25:33 +02:00
jschoubben b7d03b6897 Word a delivery that needs its author's new head as its author's, with no Stop (ADR 0505 §5)
mesh/repo-check head b7d03b68974d merged into trunk 8f2d22db0caa beside mesh-catalog#444@09e749bc — pass: its merge-check.sh passed
mesh/merge-gate head b7d03b68974d merged into trunk 8f2d22db0caa — pass: group feat/quiet-deliveries composed: every machine composes with the change as it…
mesh/delivery delivered
mesh/delivery-group group feat/quiet-deliveries merged: every member is merged
mesh/check checking b7d03b68974d on 8f2d22db0caa
2026-10-22 14:25:26 +02:00
jschoubben 21954ad1bc issue 1840/S3b: prove a process may watch its own state's bucket and no other (AC9) 2026-10-22 14:23:58 +02:00
jschoubben 485eb44de4 issue 1840/S3: keep each node's declared packages on the bus, so the pacman verbs can say what a node declares 2026-10-22 14:23:58 +02:00
jschoubben 8d562f9b22 Say a walk's start and end again with the machines whose declaration its merges change (ADR 0505 §2) 2026-10-22 14:23:54 +02:00
mesh-tools 8f2d22db0c Merge pull request 'Move route-proxy's program into its module and run it on the pinned alpine image (issues 1633, 1634, 1635, ADR 0480 §1-§4, §11)' (#433) from feat/r4d into main 2026-10-22 12:19:07 +00:00
jschoubben cc8a74fce4 issue 1840/S3: take the SDK that names declared-packages 2026-10-22 14:18:39 +02:00
mesh-tools aa9da94260 Merge pull request 'A build node leaves its builds' images behind no more, and takes no work under a floor of free space (issue 1834)' (#435) from fix/1834-build-node-disk into main 2026-10-22 12:05:14 +00:00
jschoubben 01f615aa0a Read the bus release from the nats module's upstream artifact, so the tests' bus is still held to the mesh's once the Dockerfile is gone (issue 1640)
mesh/merge-gate head 01f615aa0ad6 merged into trunk aa9da942606d — pass: group feat/r4d composed: every machine composes with the change as it did without …
mesh/delivery delivered
mesh/repo-check head 01f615aa0ad6 merged into trunk aa9da942606d — pass: its merge-check.sh passed
mesh/delivery-group group feat/r4d merged: every member is merged
mesh/check failed: unit: ..test: node:internal/modules/esm/resolve:241 | 01f615aa0ad6 on aa9da942606d | unit 3345 run, 0 kept, 7 skipped | integration…
mesh-catalog #434 runs the nats server as the upstream image named in module.json (ADR 0497) and
deletes modules/nats/Dockerfile, so the check failed opening it. The manifest's upstream nats
artifact is read first; the Dockerfile only for a catalogue from before the move (the captured copy).
2026-10-22 13:38:00 +02:00
jschoubben f501eaca43 Remove an image's pushed names with its build tag, and take no work under the free-space floor (issue 1834)
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-controller@01f615aa0ad6 (merged as 8f2d22db into main, walk plan-17926707…
mesh/merge-gate head f501eaca4305 merged into trunk 4463991e0609 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head f501eaca4305 merged into trunk 4463991e0609 — pass: its merge-check.sh passed
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
The registry push tags the image a second time, so removing only the build tag freed nothing.
The held image a comparison pulls goes after it is read, and a node under the floor leaves the ask
to another build node instead of taking it and failing.
2026-10-22 13:19:58 +02:00
jschoubben 995f8237ce Free a build node's disk after every build and refuse work under a floor of free space (issue 1834)
A full / made every check on shanks end 'store never became ready' with the reason erased by --rm.
Builds now remove their local image tag, narrowed source and context clone; a build or check
refuses under 20 GiB free on / and the workspace, saying so; the throwaway store is no longer
started with --rm and a store that does not come up reports its exit code and last 20 log lines.
2026-10-22 13:14:00 +02:00
jschoubben 707bfba4ea Check an image's Dockerfile place instead of refusing the image kind (ADR 0487 clause 13)
mesh/delivery-group group feat/r4d rejected: a member's own check failed
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 707bfba4ea36 merged into trunk 4463991e0609 beside mesh-catalog#434@68183ee4 — fail: its merge-check.sh failed: --- FAIL: TestTheBusTh…
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: i3/cmd/i3-tools.TestTheMainFileAndEveryModulesContributionLoadTogether: manifest_test.go:247: expected the launcher, clipboar…
2026-10-22 11:45:06 +02:00
mesh-tools 4463991e06 Merge pull request 'Compose a pull request's check with the config values held for it (issue 1826, ADR 0502)' (#434) from fix/1826-held-config into main 2026-10-22 09:38:35 +00:00
jschoubben 4357b3688f Tell mesh-config what a head declares at every head, so a head that reverts a key drops its held value (issue 1826)
mesh/repo-check head 4357b3688f58 merged into trunk 3d4ad6fefb7c — pass: its merge-check.sh passed
mesh/merge-gate head 4357b3688f58 merged into trunk 3d4ad6fefb7c — pass: group fix/1826-held-config composed: every machine composes with the change as it …
mesh/delivery delivered
mesh/delivery-group group fix/1826-held-config merged: every member is merged
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-22 11:07:34 +02:00
jschoubben 03927a6f6c Carry a config value held for a pull request's new key into its composed check (issue 1826)
The controller tells mesh-config what a pull request declares, asks the approved held values with each check, and the build agent and merge gate lay them over the change's composition only: a key the trunk does not declare yet can then be approved before the merge (ADR 0502).
2026-10-22 11:01:12 +02:00
jschoubben 499ff5c218 Move the route proxy into its module and run it from the trust image, on mesh-sdk's public membership (issues 1633, 1634, 1635)
mesh/repo-check out of date: judged beside mesh-catalog#434 at c948f3a61c74, it is now at 7451e7edf4e5; judged again on a recheck or when it is merged
mesh/merge-gate group feat/r4d: composed again for the group's new heads
mesh/delivery superseded: a newer head of the same pull request
mesh/delivery-group group feat/r4d rejected: a member's own check failed
mesh/check failed: unit: i3/cmd/i3-tools.TestTheMainFileAndEveryModulesContributionLoadTogether: manifest_test.go:247: expected the launcher, clipboar…
A bundle is built only from its module's directory (ADR 0480 clause 4), so the proxy leaves
examples/ and the controller's internal/broker; the broker keeps no second copy of what the
proxy shares with mesh-sdk. The image and its Dockerfile go, and a test holds that only
build-agent declares one (clause 11).
2026-10-22 09:53:56 +02:00
mesh-tools 3d4ad6fefb Merge pull request 'Limit what a build or check starts on a build node: a per-node build slice (issue 1798)' (#432) from fix/1798-limit-build-resources into main 2026-10-21 23:20:57 +00:00
jschoubben 6bf38072ca Say in the build-agent settings that changing one restarts the build slice (issue 1798)
mesh/merge-gate head 6bf38072cacf merged into trunk a929df9b0ec3 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 6bf38072cacf merged into trunk a929df9b0ec3 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-22 01:01:56 +02:00
jschoubben b174d743c0 Name the build slice without a dash so it sits beside user.slice, and restart it when its limits change (issue 1798)
mesh/merge-gate building on ace since 00:57
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-22 00:56:48 +02:00
jschoubben 37119ea31f Fill the build slice's limits as the mesh fills a module's template today (issue 1798)
mesh/merge-gate building on ace since 00:53
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-22 00:38:02 +02:00
jschoubben af2c6b5e94 Run builds and checks in a per-node build slice with CPU, IO and memory limits (issue 1798) 2026-10-22 00:24:45 +02:00
mesh-tools a929df9b0e Merge pull request 'The checker runs beside today's check: checks wait while a delivery builds; a group's SDK member merges; a flaky test fixed' (#430) from feat/the-checker into main 2026-10-21 21:46:26 +00:00
mesh-tools a3c965b9a0 Merge pull request 'Remove a check's throwaway containers with their anonymous volumes, so a build node's disk does not fill (issue 1772)' (#431) from fix/1772-check-volumes into main 2026-10-21 21:15:07 +00:00
jschoubben b9ec1919b8 Name plans-building among the controller's readable states and pin the SDK that does (issue 1687, ADR 0483 clause 2)
mesh/repo-check head b9ec1919b865 merged into trunk a3c965b9a0d5 beside mesh-catalog#423@12ff1cfb — pass: its merge-check.sh passed
mesh/merge-gate head b9ec1919b865 merged into trunk a3c965b9a0d5 — pass: group feat/the-checker composed: every machine composes with the change as it did …
mesh/delivery delivered
mesh/delivery-group group feat/the-checker merged: every member is merged
2026-10-21 22:47:04 +02:00
jschoubben d491356f59 Remove a build's throwaway containers with their anonymous volumes, so a build node's disk does not fill (issue 1772)
mesh/merge-gate head d491356f59dd merged into trunk 1f3fdf311167 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head d491356f59dd merged into trunk 1f3fdf311167 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 22:40:16 +02:00
jschoubben 47e1819101 Merge remote-tracking branch 'origin/fix/1106-sdk-member-merges' into feat/the-checker
mesh/repo-check out of date: judged beside mesh-sdk#66 at 78364e318bdc, it is now at 580da61a9107; judged again on a recheck or when it is merged
mesh/delivery-group group feat/the-checker checking: 4 of 4 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head 47e1819101a1 merged into trunk 1f3fdf311167 — fail: group feat/the-checker composed: 1 problem(s) the change brings; the first: mesh-d…
2026-10-21 22:01:42 +02:00
jschoubben 0e25ed7c5a Merge remote-tracking branch 'origin/fix/1399-person-word-test' into feat/the-checker 2026-10-21 22:01:42 +02:00
jschoubben 79e2923dc8 Merge remote-tracking branch 'origin/fix/1652-checks-wait' into feat/the-checker
# Conflicts:
#	internal/broker/testdata/composed.conf
#	internal/link/builds_nats.go
2026-10-21 22:01:42 +02:00
mesh-tools 1f3fdf3111 Merge pull request 'Judge a duplicate hand-out before marking a build in hand, and never hand every build back over a stale mark (issue 1768)' (#429) from fix/1768-in-hand-marker into main 2026-10-21 19:59:30 +00:00
jschoubben 9902a3f16c Name every group member's trunk in the composed verdict, the announced one when the seat names none (issue 1106) 2026-10-21 21:57:30 +02:00
jschoubben 11815c0fb3 Have the delivery test's view hear a plan a verb let go, so a loaded machine cannot read it still waiting (issue 1399) 2026-10-21 21:45:58 +02:00
mesh-tools f973cefd7f Merge pull request 'Ask the operator for a held move and a failed release, and send no image the store lacks (issues 1732, 1320, 1742)' (#428) from fix/1732-held-moves-asked into main 2026-10-21 19:07:51 +00:00
jschoubben 696300b9e2 Judge a duplicate hand-out before the in-hand claim, and treat a stale mark as stale (issue 1768)
mesh/merge-gate head 696300b9e280 merged into trunk f973cefd7fd3 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 696300b9e280 merged into trunk f973cefd7fd3 — pass: its merge-check.sh passed
mesh/delivery delivered
A dropped duplicate left the in-hand marker, so every later claim failed and every ask spent its deliveries.
2026-10-21 20:57:12 +02:00
jschoubben e7cc315fd0 Guard the taking loop's subscriptions, and update the two tests that held a busy machine's pulls
The loop now lets go of and binds queues around every ask, so Close takes the lock the loop does;
the busy machine's test says it holds no pull, and the stray test keeps its long check alive
within its two-second ack wait.
2026-10-21 20:53:58 +02:00
mesh-tools 98155e327b Merge pull request 'Take mesh-sdk at 07a6592: an item's text places a config value (issue 1658), and fit the unit level in its bound (issue 1763)' (#420) from fix/1656-xterm-config into main 2026-10-21 18:36:39 +00:00
jschoubben 60253c82b7 Fix the review of the checks-wait milestone: redeliveries, deleted modules, the kept queue
A check pending redelivery is read before the let-through fetches (none is handed back);
a deleted module is done for the plan's key; the queue an ask came from is let go of when
it ends. Tries say they run under their own id.
2026-10-21 20:17:37 +02:00
jschoubben d6db334840 Offer no Release on a machine that also holds a person's own hold (issue 1732, review round 1)
mesh/repo-check head d6db334840a5 merged into trunk 98155e327b6d beside mesh-host#129@09c77d57 — pass: its merge-check.sh passed
mesh/merge-gate head d6db334840a5 merged into trunk 98155e327b6d — pass: group fix/1732-held-moves-asked composed: every machine composes with the change a…
mesh/delivery delivered
mesh/delivery-group group fix/1732-held-moves-asked merged: every member is merged
2026-10-21 20:15:58 +02:00
jschoubben 69d016bfdf Merge remote-tracking branch 'origin/main' into fix/1732-held-moves-asked 2026-10-21 20:15:13 +02:00
jschoubben bb8f076cc0 Merge remote-tracking branch 'origin/main' into fix/1652-checks-wait 2026-10-21 20:14:12 +02:00
jschoubben a58b38cee3 Keep the missing image out of the condition's summary (issue 1742) 2026-10-21 20:06:10 +02:00
jschoubben 9d260412b3 Merge remote-tracking branch 'origin/main' into fix/1656-xterm-config
mesh/repo-check head 9d260412b330 merged into trunk 44033f4b6d72 beside mesh-catalog#416@b9dcf19c — pass: its merge-check.sh passed
mesh/merge-gate head 9d260412b330 merged into trunk 44033f4b6d72 — pass: group fix/1656-xterm-config composed: every machine composes with the change as it…
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792609673527907956 (novox/mesh-controller at f973cefd into ma…
mesh/delivery-group group fix/1656-xterm-config merged: every member is merged
2026-10-21 19:59:59 +02:00
jschoubben 99610e9569 Cut the controller's slowest unit tests (a source scan that regexed every file, resolver stand-in timeouts, a wait bound) so mesh-controller's unit level fits its bound (issue 1763) 2026-10-21 19:59:58 +02:00
jschoubben bb4d2e1da4 Let go of the other queues while an ask runs, and read the let-through race right
A message that reaches a builder's subscription outside a fetch was held unheartbeated for
the whole run of another ask, behind which a delivery waited (issue 1747). The queues the
running ask did not come from are let go of and bound again after it.
2026-10-21 19:56:23 +02:00
mesh-tools 44033f4b6d Merge pull request 'Issue a bus process's account when its module is assigned or gains it, and warn of one still missing instead of composing it silently (issue 1226)' (#427) from fix/1226-issue-process-accounts into main 2026-10-21 17:53:47 +00:00
jschoubben dc06af1f5a Leave a module out of a send when the store lacks its image, and say it (issue 1742) 2026-10-21 19:43:36 +02:00
jschoubben b62fbeabbf Ask a held move and the builds held after a failed release, and say them on the plan's line (issue 1732, 1320) 2026-10-21 19:43:36 +02:00
jschoubben 373aefd14e Take mesh-sdk at 07a6592, which counts an item's text as a placement of a config value (issue 1658)
mesh/delivery-group group fix/1656-xterm-config checking: composed check out of date: novox/mesh-sdk@c8e35097e861 was judged against trunk 85b5a2687e6a, the tr…
mesh/merge-gate building on ace since 19:52
mesh/repo-check out of date: judged against trunk 22c9815f6522, the trunk is now 44033f4b6d72; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 19:43:32 +02:00
jschoubben f66458fe25 Merge remote-tracking branch 'origin/main' into fix/1656-xterm-config 2026-10-21 19:43:02 +02:00
mesh-tools 22c9815f65 Merge pull request 'Name the failing line in a failed check's summary, not the warning count printed last (issue 1310)' (#423) from fix/1310-failed-check-summary into main 2026-10-21 17:39:15 +00:00
jschoubben a14dc04b7b Hold the build seat's checks while a delivery builds, and let one check that waited 30 minutes through
ADR 0483: the controller keeps a key per plan in its build phase; holders read it and
take no new check while one is there, but for the oldest check past 30 minutes, one
at a time by a let-through key. The queue says why checks wait.
2026-10-21 19:32:47 +02:00
jschoubben 181b579ac6 Assert the next compose carries the issued account, not a stand-in (issue 1226, review round 2)
mesh/merge-gate head 181b579ac6ea merged into trunk 22c9815f6522 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 181b579ac6ea merged into trunk 22c9815f6522 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 19:27:22 +02:00
jschoubben b08c6372e1 Assert the next compose carries the issued account (issue 1226, review round 2) 2026-10-21 19:26:57 +02:00
jschoubben 4e11965a2a Merge remote-tracking branch 'origin/main' into fix/1226-issue-process-accounts 2026-10-21 19:26:00 +02:00
jschoubben faf8beee84 Warn instead of holding the machine for an unissued process account, and drive the delivery path in the test (issue 1226, review round 1) 2026-10-21 19:25:44 +02:00
mesh-tools 3827c3eb60 Merge pull request 'Say a building plan's refusal in its line, not 'building' with LATE, once its tier is built (issue 920)' (#426) from fix/920-plan-line-says-the-refusal into main 2026-10-21 17:23:01 +00:00
jschoubben 6bcfbeb5b2 Issue a bus process's account when its module is assigned or gains it (issue 1226) 2026-10-21 19:21:11 +02:00
jschoubben deac3e232c Refuse a process account's stand-in, naming issue (issue 1226) 2026-10-21 19:21:11 +02:00
jschoubben 99da867a98 Merge origin/main into fix/1656-xterm-config 2026-10-21 19:16:11 +02:00
jschoubben 612f493c49 Say only a refusal as a plan's hold, not a tier's summary left in its note (issue 920, review round 1)
mesh/merge-gate head 612f493c4926 merged into trunk 5ba2ffe8e608 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 612f493c4926 merged into trunk 5ba2ffe8e608 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 19:01:42 +02:00
jschoubben 2377d75ed8 Say a building plan's refusal in its line, not 'building' with LATE, once its tier is built (issue 920) 2026-10-21 18:57:59 +02:00
mesh-tools 5ba2ffe8e6 Merge pull request 'Send a declaration's large files by reference through a bucket per node, and hold no check twice on the build seat' (#425) from feat/1696-declaration-files into main 2026-10-21 16:51:05 +00:00
jschoubben 8fc0d135bb Merge remote-tracking branch 'origin/main' into feat/1696-declaration-files
mesh/merge-gate head 8fc0d135bbe2 merged into trunk d568d76b9710 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 8fc0d135bbe2 merged into trunk d568d76b9710 — pass: its merge-check.sh passed
mesh/delivery delivered
# Conflicts:
#	go.mod
#	go.sum
#	testdata/beside/CAPTURED
#	vendor/modules.txt
2026-10-21 18:32:56 +02:00
mesh-tools d568d76b97 Merge pull request 'Take mesh-sdk 90b20f5, so node-files' contract names upload and download (issue 1718, design 60 F3 group)' (#424) from feat/files-through-the-store into main 2026-10-21 16:31:27 +00:00
jschoubben 610d88e6bf Merge commit '3217f097d38d9f1c4302ec827ad6b371fd8ba95b' into feat/1696-declaration-files 2026-10-21 18:21:37 +02:00
jschoubben 0b11b6c98d Take mesh-sdk at its merge 725590e0 of the per-node bucket (issue 1694, ADR 0488) 2026-10-21 18:03:47 +02:00
jschoubben 92a5193e31 Merge remote-tracking branch 'origin/main' into feat/1696-declaration-files 2026-10-21 18:03:06 +02:00
jschoubben 550cf37d71 Merge remote-tracking branch 'origin/main' into feat/files-through-the-store-files-verbs
mesh/repo-check head 550cf37d7109 merged into trunk d96c9f56a3f1 beside mesh-catalog#417@aef2a1a9 — pass: its merge-check.sh passed
mesh/merge-gate head 550cf37d7109 merged into trunk d96c9f56a3f1 — pass: group feat/files-through-the-store composed: every machine composes with the chang…
mesh/delivery superseded: its walk was superseded and names no walk that took it over: a walk that took it over answers its merge, and it follows that wa…
mesh/delivery-group group feat/files-through-the-store closed: a member was closed before the group merged
2026-10-21 17:49:29 +02:00
jschoubben 3217f097d3 Make a fetch's pull request end before the fetch does, and drop the hand-back loop that could itself take asks (issue 1649) 2026-10-21 17:28:40 +02:00
mesh-tools d96c9f56a3 Merge pull request 'Pin route-proxy to a mesh-sdk commit on main, so a fresh go mod download resolves it once the feature branch goes (issue 1669)' (#419) from fix/1669-route-proxy-pin into main 2026-10-21 15:12:49 +00:00
jschoubben 69a67876c2 Take mesh-sdk at 6f63275, which counts an item's text as a placement of a config value (issue 1658)
mesh/delivery-group group fix/1656-xterm-config rejected: a member's own check failed
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 69a67876c269 merged into trunk d96c9f56a3f1 beside mesh-catalog#416@f178aab6 — fail: its merge-check.sh failed: --- FAIL: TestModuleCh…
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 17:03:35 +02:00
jschoubben 650274d700 Read a pull's held asks with Fetch, hand them back all through a build, and drop a duplicate only while its builder's key is fresh (issue 1649) 2026-10-21 16:58:39 +02:00
jschoubben 678a517843 Tests first for issue 1649: a held ask really handed back, a stale running key no duplicate 2026-10-21 16:57:40 +02:00
jschoubben bb31f6bc49 Take mesh-sdk d4deecdb, main's merge of node-files' upload and download, over main's container-groups pin (issue 1718)
mesh/delivery-group group feat/files-through-the-store checking: 0 of 2 member(s) ready
mesh/merge-gate building on g14 since 17:08
mesh/repo-check out of date: judged against trunk 12752911b644, the trunk is now d96c9f56a3f1; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 16:57:02 +02:00
jschoubben 516af253c6 Merge main into feat/1696-declaration-files, keeping mesh-sdk at 86f9a1a5, which holds main's pin 85b5a268 2026-10-21 16:46:21 +02:00
mesh-tools 12752911b6 Merge pull request 'Send a container's supplementary groups only to a node that reports container-groups, holding the module elsewhere (issue 1638, R4d)' (#422) from feat/r4d-groups into main 2026-10-21 14:44:55 +00:00
jschoubben 5157ccc3a8 Hand back any ask a pull holds before a build starts, and drop a hand-out of a check another builder runs, so one check is never run twice (issue 1649) 2026-10-21 16:35:09 +02:00
jschoubben 087e9266d1 Test the capability gate, the sweep's mark re-read and the lock use of a send and of the sweep, each failing under its mutation (issues 1696, 1697; review round 2) 2026-10-21 16:28:24 +02:00
jschoubben 3ac1944168 Announce the pull request's head again, so its delivery is made with its base and asks the Approve (issue 1726; tree unchanged)
mesh/merge-gate head 3ac1944168e4 merged into trunk 12752911b644 — pass: builds route-proxy → ace, novox; no bus step; every machine composes with the chan…
mesh/repo-check head 3ac1944168e4 merged into trunk 12752911b644 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 16:17:00 +02:00
jschoubben a3b4e0601d Take mesh-sdk at 86f9a1a5 and name each node's bucket with its FileBucketFor, refusing a node name that could carry a wildcard (issues 1696, 1697; ADR 0488) 2026-10-21 16:05:24 +02:00
jschoubben 227c86ebb3 Take mesh-sdk 90b20f5, so node-files' contract names upload and download (issue 1718)
mesh/delivery-group group feat/files-through-the-store rejected: a member's own check failed
mesh/merge-gate head 227c86ebb388 merged into trunk 12752911b644 — fail: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 227c86ebb388 merged into trunk 12752911b644 — fail: head 227c86ebb388 does not merge cleanly into trunk 12752911b644: in conflict go.m…
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 15:57:18 +02:00
jschoubben da364c2730 Give each node a bucket of its own for the files that travel by digest, and keep the sweep from removing an object a send relies on (issues 1696, 1697; review of ADR 0486 step 3)
A literal * in a published subject matches the grant's *, so in one shared bucket a node could read every node's files. Each node now
has declaration-files-<node>, which its account reads by name and the controller writes; the sweep judges each bucket against that
node's latest declaration, under the node's lock with the send, and an object the sweep marked is put again by the next send. Putting
that fails send after send raises a condition.
2026-10-21 15:43:36 +02:00
mesh-tools 98ffcb667f Merge pull request 'Keep a build's outcome off the controller's event loop when the plans are busy, so a slow send no longer leaves the controller deaf (issue 1708)' (#421) from fix/1708-listening into main 2026-10-21 13:33:59 +00:00
jschoubben 4771ee53d9 Name the failing line in a failed check's summary, not the warning count printed last (issue 1310)
mesh/merge-gate out of date: judged against trunk 5ba2ffe8e608, the trunk is now 22c9815f6522; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 5ba2ffe8e608, the trunk is now 22c9815f6522; judged again on a recheck or when it is merged
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-controller@181b579ac6ea (merged as 44033f4b into main, walk plan-17926043…
2026-10-21 15:21:10 +02:00
jschoubben 2aa291f3b3 Pin mesh-sdk at its merge of the container-groups capability, on main (issue 1638)
mesh/repo-check head 2aa291f3b30b merged into trunk 98ffcb667fb2 — pass: its merge-check.sh passed
mesh/merge-gate head 2aa291f3b30b merged into trunk 98ffcb667fb2 — pass: group feat/r4d-groups composed: every machine composes with the change as it did w…
mesh/delivery delivered
mesh/delivery-group group feat/r4d-groups merged: every member is merged
2026-10-21 15:16:47 +02:00
jschoubben ceaa616d01 Send files over 64 KiB by digest through the bus's object bucket to nodes that read them, grant it, and warn at 75 percent (issues 1696, 1697, 1698; ADR 0486 step 3)
A node that reports file-references gets the digest and size of each file over 64 KiB that may travel so, and the content is put in
the bucket declaration-files before the declaration is sent; others get every content inline. The controller alone writes the
bucket and node-engines read an object by name (the four grants the node-engine's own fetch makes, no list, no write). An object
goes a day after no node's latest declaration names it. The near-limit condition warns at 75 percent, and a test holds the control
node's declaration, in the reference form, under half the bus's largest message.
2026-10-21 15:16:11 +02:00
jschoubben fa5b5e1c11 Take mesh-sdk at 863a981e, which carries the file references of ADR 0486 (issue 1696) 2026-10-21 15:07:36 +02:00
jschoubben 6ef78580ce Take the mesh-sdk that holds the container-groups capability rebased on main (issue 1638) 2026-10-21 15:04:37 +02:00
jschoubben 0e0c775bfa Keep a refused registration left to the build records, in the store and the process, so the look fails the plan rather than settling it built; and forget a deleted module and say what was never sent when the records settle (issue 1708)
mesh/merge-gate head 0e0c775bfa5e merged into trunk f10fbda1f619 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 0e0c775bfa5e merged into trunk f10fbda1f619 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 15:03:23 +02:00
jschoubben 61c8b56107 Leave moving the plans after a build's outcome to the look in the serving controller, so a release's retried send no longer runs on the event loop (issue 1708) 2026-10-21 14:39:17 +02:00
jschoubben c24139a0fd Take a build's outcome on the event loop without waiting for the plans, and leave it to the build records with a look asked, so a holder sending for minutes no longer stops the controller listening (issue 1708) 2026-10-21 14:33:09 +02:00
jschoubben d8d6d196a4 The controller holds a module whose container names groups on a node that does not report container-groups, naming the capability (issue 1638, ADR 0480 clause 7) 2026-10-21 14:30:26 +02:00
jschoubben a474762527 Pin route-proxy to a commit of mesh-sdk main, since the pinned one is on a feature branch only (issue 1669)
mesh/merge-gate head a47476252772 merged into trunk 98ffcb667fb2 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head a47476252772 merged into trunk 98ffcb667fb2 — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 14:25:34 +02:00
mesh-tools f10fbda1f6 Merge pull request 'Send the vault's export compact and measure every declaration against the bus's limit, raising a condition from 90%, so novox's declaration fits again and cannot fail silently (issue 1689)' (#418) from fix/1689-declaration-room into main 2026-10-21 12:22:45 +00:00
jschoubben 09aa063a13 Measure a declaration against the bus's largest message, raise a condition near it, and send the vault's export compact (issue 1689)
mesh/merge-gate out of date: judged against trunk 5be1cfaed19b, the trunk is now f10fbda1f619; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 5be1cfaed19b, the trunk is now f10fbda1f619; judged again on a recheck or when it is merged
mesh/delivery delivered
A declaration is one bus message and the control node's passed the limit by 2 KB. The size is checked at every send
against the connection's own limit, with a condition from 90% and a message naming size and limit when over; the
export, 138 KB compact of 165 KB indented, is sent compact.
2026-10-21 13:54:11 +02:00
mesh-tools 5be1cfaed1 Merge pull request 'Ask a new module's first build when its merge is heard, not when its batch is cut, so it can be assigned right after its delivery (issue 1617)' (#417) from fix/1617-register-new-modules into main 2026-10-21 11:45:56 +00:00
jschoubben 35829d5a9a Keep the test's first batch assembling (issue 1617)
mesh/merge-gate head 35829d5a9a8e merged into trunk cada7e3c8374 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 35829d5a9a8e merged into trunk cada7e3c8374 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 13:18:38 +02:00
jschoubben 57939d6dd4 Ask a new module's first build when its merge is heard, not when the batch is cut, and not twice (issue 1617) 2026-10-21 12:53:16 +02:00
mesh-tools cada7e3c83 Merge pull request 'Take the mesh-sdk that knows shares on an s3-bucket requirement, and prove the controller composes and checks it (issue 1593)' (#416) from feat/files-through-the-store into main 2026-10-21 10:42:04 +00:00
mesh-tools ff42f21839 Merge pull request 'Refuse a build whose module directory, archive or bundle output leads out of the module through a link, so a repository cannot make build-agent pack files from the build node (issue 1609)' (#415) from fix/1609-archive-symlink into main 2026-10-21 10:16:52 +00:00
jschoubben 64c1996a5e Prove the controller composes and checks shares on an s3-bucket requirement (issue 1593)
mesh/merge-gate head 64c1996a5e9d merged into trunk ff42f2183900 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 64c1996a5e9d merged into trunk ff42f2183900 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 11:53:24 +02:00
jschoubben 8a1ef29e36 Take mesh-sdk at 465beb4, which carries shares on an s3-bucket requirement (issue 1593) 2026-10-21 11:51:38 +02:00
mesh-tools 0caa8c6844 Refuse a bundle whose output directory leads out of the module through a link, so a repository cannot make the build write into or pack files it does not hold (issue 1609)
mesh/merge-gate head 0caa8c6844d9 merged into trunk 0205b8f7080a — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 0caa8c6844d9 merged into trunk 0205b8f7080a — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 09:46:00 +00:00
mesh-tools fcca6ba9e0 Refuse an archive or a module directory that leads out of the repository through a link in the tree, so a repository cannot make the build pack files it does not hold (issue 1609) 2026-10-21 09:22:14 +00:00
mesh-tools 0205b8f708 Merge pull request 'Hold a merge for any failure to record it, so the bus hands it over again and no merge is lost (issue 1583)' (#414) from fix/1583-merge-handed-over into main 2026-10-21 09:05:55 +00:00
mesh-tools c8539bf35a Merge pull request 'Do not ask the operator to stop a delivery that is already stopped; say what ends it (issue 1581)' (#413) from fix/1581-stopped-alert into main 2026-10-21 08:52:14 +00:00
mesh-tools 5c94759cc0 Hold a merge for any failure to record it, so the bus handing it over again never loses it (issue 1583)
mesh/merge-gate head 5c94759cc0c2 merged into trunk c8539bf35a8d — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 5c94759cc0c2 merged into trunk c8539bf35a8d — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 08:43:09 +00:00
mesh-tools cddf150871 Spell the dash escape with one backslash, as the reviewed source has it (issue 1581)
mesh/merge-gate head cddf15087178 merged into trunk 0fee5404d95b — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head cddf15087178 merged into trunk 0fee5404d95b — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 08:30:27 +00:00
mesh-tools ef29adbc1a Spell the dash in stoppedNeeds as an escape, as the reviewed source has it (issue 1581) 2026-10-21 08:30:12 +00:00
mesh-tools e962279f1d Do not ask the operator to stop a delivery that is already stopped; say what ends it (issue 1581) 2026-10-21 08:29:57 +00:00
mesh-tools 0fee5404d9 Merge pull request 'Count only the controller's own consumers in the deaf look, not the check workers a builder holds (issues 1063, 1543)' (#412) from fix/1063-deaf-counts-own-consumers into main 2026-10-20 23:46:00 +00:00
mesh-tools 54d8d139ad Record the asked builds under a lock in the merge tests' helper, which the merge window's goroutine and the test call at once (issue 1543)
mesh/merge-gate head 54d8d139ad5b merged into trunk f4584e6fa285 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 54d8d139ad5b merged into trunk f4584e6fa285 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 23:24:37 +00:00
mesh-tools 233db89f29 Count only the controller's own consumers in the deaf look, not the check workers a builder holds (issue 1063) 2026-10-20 23:14:45 +00:00
mesh-tools f4584e6fa2 Merge pull request 'Read the forge's address again before calling a source not on the forge, within one bound (issue 1277)' (#411) from fix/1277-forge-re-serve into main 2026-10-20 22:34:18 +00:00
mesh-tools 2b1eeb7cd5 Keep a registration's two forge waits within one bound, and read an unreadable address once per five seconds (issue 1277)
mesh/merge-gate head 2b1eeb7cd53e merged into trunk f29d193ddbc1 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 2b1eeb7cd53e merged into trunk f29d193ddbc1 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 22:03:55 +00:00
mesh-tools 912178fdf7 Read the forge's address again before calling a source not on the forge (issue 1277) 2026-10-20 21:56:59 +00:00
mesh-tools f29d193ddb Merge pull request 'Read mesh-config's values for a config module that has no settings row (issue 1493)' (#410) from fix/1493-config-without-settings-row into main 2026-10-20 20:27:48 +00:00
jschoubben 7adadf756d Test that a config module with no settings row is read from mesh-config (issue 1493)
mesh/merge-gate head 7adadf756dfe merged into trunk 073ccc29c603 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 7adadf756dfe merged into trunk 073ccc29c603 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 22:09:59 +02:00
jschoubben 6129623778 Read settings for a config-form module that mesh-config holds, not only for one with a settings row (issue 1493) 2026-10-20 22:09:28 +02:00
mesh-tools 073ccc29c6 Merge pull request 'Write the facts snapshot in format 2 with each machine's grants and credential bindings (design 47 R4c-2, issue 1394)' (#409) from feat/r4c2-new-keys into main 2026-10-20 19:31:03 +00:00
mesh-tools e7c6abfd0e Test the grants of the bridge's file read and the indexer's manifests bucket (issues 1448, 1449, design 47 R4c-2)
mesh/repo-check head e7c6abfd0e3c merged into trunk f2d7bc94800d — pass: its merge-check.sh passed
mesh/merge-gate head e7c6abfd0e3c merged into trunk f2d7bc94800d — pass: group feat/r4c2-new-keys composed: every machine composes with the change as it di…
mesh/delivery delivered
mesh/delivery-group group feat/r4c2-new-keys merged: every member is merged
2026-10-20 18:56:37 +00:00
mesh-tools b75fb7899e Write the facts snapshot in format 2, with each machine's grants and credential bindings (design 47 R4c-2, issue 1394)
Every reader takes format 2 since R4b (ADR 0456 clause 1), and the assigner's composition check reads the grants and bindings (ADR 0456 clause 2): the writer moves now, names only, never a sealed value.
2026-10-20 18:26:07 +00:00
mesh-tools f2d7bc9480 Merge pull request 'Let the catch-up read the router, and look only for pull request merges into the trunk since the records began (issue 1336, ADR 0457, ADR 0463)' (#408) from fix/1336-router-read-after-stop into main 2026-10-20 17:45:05 +00:00
mesh-tools 9f0d9098b7 Begin the record of merges at the earliest merge kept on any trunk, not per trunk (ADR 0463 section 2, issue 1336)
mesh/merge-gate head 9f0d9098b735 merged into trunk d163304e3ff1 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 9f0d9098b735 merged into trunk d163304e3ff1 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 14:49:10 +00:00
mesh-tools 3045af4722 A merge from before the controller kept merges of the trunk is history, never a gap (issue 1336) 2026-10-20 14:38:47 +00:00
mesh-tools 0aa588779d Catch up only a pull request's merge into the trunk, not the merges made inside its branches (issue 1336) 2026-10-20 14:35:58 +00:00
mesh-tools 030bdc537e Read the router's ask ids without taking a second stop of the key lister for a failure (issue 1336) 2026-10-20 14:35:32 +00:00
mesh-tools d163304e3f Merge pull request 'Catch up merges the controller did not hear, on the operator's Approve of each (ADR 0457, issue 1336)' (#407) from fix/1336-catch-up-merges-not-heard into main 2026-10-20 14:07:33 +00:00
mesh-tools 1816e8753b Merge pull request 'Wait out a bus that briefly refuses a build's ask instead of failing the delivery (issue 1334)' (#406) from fix/1334-a-bus-reload-fails-a-tier-2 into main 2026-10-20 13:55:34 +00:00
mesh-tools 970622505b Say a retry asked nothing when every module of it still waits for the bus (issue 1403)
mesh/merge-gate head 970622505b30 merged into trunk b718d24f84b9 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 970622505b30 merged into trunk b718d24f84b9 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 13:44:03 +00:00
mesh-tools 3c7048f24e Give the condition of a merge not heard plain words the operator can act on (ADR 0253, ADR 0457)
mesh/merge-gate head 3c7048f24e55 merged into trunk 1816e8753b07 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 3c7048f24e55 merged into trunk 1816e8753b07 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 13:42:37 +00:00
mesh-tools 9b24913656 Test the teller's warrant by the router's real ids, a gap that adds a module, and short pages (ADR 0457, issue 1336, review round 1) 2026-10-20 13:13:48 +00:00
mesh-tools f25481342f Find the teller's warrant by the heads its Approve is bound to, and judge a gap as hearMerge does (ADR 0457, issue 1336, review round 1) 2026-10-20 13:12:58 +00:00
mesh-tools bde1691f0a Judge a merge in one place for hearing it and for the catch-up (ADR 0457, issue 1336, review round 1) 2026-10-20 13:12:23 +00:00
mesh-tools 29bbe4d682 Test the catch-up from ADR 0457's checks (issue 1336) 2026-10-20 12:59:06 +00:00
mesh-tools 815d60a7e4 Run the catch-up at the controller's start and after each merge it hears (ADR 0457, issue 1336) 2026-10-20 12:58:16 +00:00
mesh-tools 4e0a82010c Catch up on merges the controller did not hear: read the trunk's last fifty merges and hear each gap on the router's warrant (ADR 0457, issue 1336) 2026-10-20 12:57:54 +00:00
mesh-tools b76cbf708a Ask a module again at the next look when the bus did not answer, and fail the plan only after ten minutes (issue 1334)
mesh/merge-gate checking again: its repo-check failed at 12:59 UTC only on TestTheCataloguesDerivedSubjectsMeet, a catalogue fault (mesh-issues' dead subsc…
mesh/repo-check checking again: its repo-check failed at 12:59 UTC only on TestTheCataloguesDerivedSubjectsMeet, a catalogue fault (mesh-issues' dead subsc…
mesh/delivery superseded: a newer head of the same pull request
2026-10-20 12:10:05 +00:00
mesh-tools b718d24f84 Merge pull request 'Take the composition, catalogue and validator from mesh-sdk and delete the controller's copies (design 47 R4b, issues 1337, 1381-1391)' (#405) from feat/r4b-sdk-users into main 2026-10-20 12:03:22 +00:00
mesh-tools 8a330afd44 Say a placeholder or replaced mesh-sdk pin as a local mesh-sdk, not as a version
mesh/merge-gate head 8a330afd44ba merged into trunk 6c2a8d62100c — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 8a330afd44ba merged into trunk 6c2a8d62100c — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 11:43:18 +00:00
mesh-tools eb8c313664 Say the walk's state, tier and note when a person's word does not start it, so a flake explains itself 2026-10-20 11:41:29 +00:00
mesh-tools f87590b0f9 Re-make an own secret sealed to a key the machine lost, and name the validator's mesh-sdk pin in a check 2026-10-20 11:33:38 +00:00
mesh-tools 33b285ce09 Make the credentials in four integration tests by a send, not a question, and name the refusal each now meets 2026-10-20 11:30:55 +00:00
mesh-tools 37d61441f8 Remove the rest of the equality test: its comparison of two compositions (issue 1384, 1389)
The controller's own composition it compared mesh-sdk's with is deleted by the commit before; compose_equal_test.go held the comparison (bothWaysDiffer, madeDiffer and their helpers) and is not needed with nothing to compare against. The made type, which compose_perform_integration_test still reads a store through, moves next to madeIn.
2026-10-20 11:19:43 +00:00
mesh-tools 2111b78fc5 Delete the controller's own composition: planFor, declarationWith, renderingFor, grantsFor, theRestOfTheMesh, generators, whoResolves and composedAndValidated, with what only they reached (issue 1389)
Every caller composes through mesh-sdk's go/compose since issues 1386 to 1388; the controller's second copy of the composition is dead code. The parent of this commit is fb556d6b, whose try (build-1792493691274712799, 17m28s, 2026-10-20 11:12 UTC) ran the equality test green on the live facts: every machine was sent and compared both ways (ADR 0453, issue 1384).

Deleted, 40 functions: in plan.go planFor, declarationWith, renderingFor, grantsFor, theRestOfTheMesh and their helpers (boundOfGrant, busCredentialIssued, carriedBuilds, carriedOn, composeBusUsers, engineJudgesRoot, engineReadsHealth, engineTakesHanded, takesHanded, givenUnderAnOldKey, issuedToHolder, keyFor, licencesFor, managerPublicKeyFor, movedProvider, providerMoved, providerModuleOf, sayLeftOut, seatsOn, wireBuilds, zonesInTheMesh); in network.go generators, whoResolves, namesInTheMesh, onThePrivateNetwork; composedAndValidated, adoptionFor, adoptionOf, engineOpensVaultBox, namedModules, localNetworksOf, keepRecorded, passSeatsOn, theMeshPassesSeatsOn, assignmentsOf. Choosing (Reading, Allocating, Foreseeing) is gone: coreBoxesFor and coreBoxesOf take a plain allocating flag. composeForPush no longer takes the private network it ignored. The seat pass of a send is performSeatPasses, over the passes the composition names.

Measured: cmd/mesh-controller non-test code 67,643 -> 65,821 lines (-1,822), its tests 57,691 -> 57,286, with the integration part of the equality test removed here because it calls the deleted functions. Not the issue's 3,000: that counts all of plan.go (2,192 lines), of which the callers, the verbs, the perform step and the sends remain (sendable, recorded builds, the held-back and ready-node logic, the plan verb, the gate's trial); and the 37 functions a name walk lists as unreachable are called through interfaces and method sets and are kept.

Tests dropped, each with the mesh-sdk go/compose test that holds its cases:
- seat_pass_test.go TestACompositionWithTwoClaimantsRecordsNoneAndSaysSo, TestACompositionChangesNothingWhileTheHolderIsAssignedOrNobodyClaims, TestAHolderAssignedButNotOnTheShelfIsNotTakenForGone: rest_test.go TestNoSeatIsPassedForTwoClaimantsAnAssignedHolderOrNoClaimant, TestAHolderAssignedButNotOnTheShelfIsNotTakenForGone, TestASeatWhoseHolderIsAssignedNowhereGoesToItsOnlyClaimant. TestACompositionPassesTheSeatAndWritesOneHandActByTheController and TestALostPassWritesNoActAndAFailedOneLeavesTheRecord stay as tests of the send's step (performSeatPasses); TestAThrowawayMeshPassesTheSeatInMemoryAndWritesNothing stays as TestAThrowawayMeshWritesNoHandActForAPass.
- seat_pass_test.go TestNoSeatIsPassedWhenAMachinesAssignmentsCouldNotBeRead: no counterpart, and none is needed: the adapter reads every machine's assignments for the Input and a machine whose read fails fails the Input, so no seat is passed on a partial read.
- network_test.go TestOnlyAMachineOnThePrivateNetworkIsNamed: network_test.go TestANodeNotOnThePrivateNetworkGetsNoAddressAndNoNetwork.
- held_back_test.go TestASendCarriesTheCurrentBuildsAndALeftOutModuleKeepsItsOwn: node_test.go TestAModuleLeftOutKeepsTheBuildItWasLastSent (and compose_sdk_test.go's table, now literal).
- modules_wire_test.go TestTheBuildOfACarriedCommitIsTheAssignmentsOrTheOneBuiltFromIt: node_test.go TestTheBodyCarriesTheOrderTheInputGivesAndTheBuildsOnlyToAnEngineThatReadsThem.
- compose_equal_integration_test.go: the old path it compares against is deleted; its last run is the parent's try.
Rewritten against mesh-sdk, same cases: issued_test.go (needs now come from compose.Resolve: Needed.IssuedToHolder and ProviderMoved), module_health_test.go and merge_gate_handing tests (the contract in the Input), provisioner_account_integration_test.go (the composition's refusal), one_resolution_integration_test.go (sayFound). Every test of the send, perform, the adapter and the verbs is kept; the tests that used planFor, declarationWith, generators, renderingFor or grantsFor as setup use resolveNode, composeSend, composeQuestion, renderingOf and grantsOf. compose_callers_test.go now fails when any deleted name is declared or called again.

Committed by copying the files of a staging branch (stage/r4b-1389 at 198be8217c, built from fb556d6b by patch commits and compared with the author's worktree), so that this commit's parent is fb556d6b.
2026-10-20 11:18:42 +00:00
mesh-tools fb556d6b74 Make the equality test fail unless every machine of the facts is sent both ways (issue 1384)
A pass printed no line, so it did not show whether idt and ygcyr were compared on the send side or refused alike. A machine either send refuses now fails, naming the machine and both refusals; refusals alike in the Foreseeing comparison (a) stay, said only; facts with no machine fail. A pass proves that every machine was sent and compared.
2026-10-20 10:53:25 +00:00
mesh-tools 11e005fe38 Prepare every send as the controller's composition did: make the own secrets a file names, and ask the vault every time (issues 1385, 1386)
mesh-sdk's composition refuses a module's file that names an own secret not made yet in every mode, where the controller's Allocating composition made it on the way; and it asked the vault on every send, where a send whose kept box was valid composed with no stand-in and so never performed. prepareSend, before the composition of every send, makes the own secrets no row holds (not the handed, not the outside-issued, not one the composition would refuse) and asks the vault for the core's credentials, keeping the boxes it answers and returning the modules the controller seals itself. perform no longer asks the vault.
2026-10-20 10:09:06 +00:00
mesh-tools 157d1599b4 Judge the catalogue's manifests against the predefined list, and put merge-check.sh back as main has it (issue 914)
The seats test I moved to internal/ownmanifest parsed each catalogue manifest alone, so claude-code's managed_settings and dunst's font-size, which fall back to a predefined key, were refused where the check clones the catalogue beside (ADR 0348 section 7). It reads them with ReadTreeManifest from the catalogue's root, as the tests that issue 914 fixed do. merge-check.sh differed from main in a comment only; it is main's again.
2026-10-20 10:08:41 +00:00
mesh-tools 7d949f888f Run the equality test's three stores one after the other, and log where each phase starts (issue 1384)
The live try failed in 32 s: throwawayStores finds its databases by the process environment, and a store opened later finds them there too, so a second store raised at the same time took the first one's database from under it (database gate_inventory_... does not exist). The three stores are raised one after another again, each phase says when it starts, and a failed phase stops the test. The two served-fact tests log one line when they pass their key assertion.
2026-10-20 09:58:49 +00:00
mesh-tools 9f773e0b3c Run the equality test only when named, on three stores at once, with its phases timed (issue 1384)
The live try took 1074 s for it. It now skips unless the tests pattern (-run) names it, which a try sets and a whole-suite merge check does not; its three independent pieces of work (both ways of Foreseeing on a fresh store, the controller's sends and readings, mesh-sdk's sends) run together; mesh-sdk's Input is read once for a phase of questions; and each phase logs its elapsed time, so the next run says where the time goes. A machine refused on either side is left out of the comparison of the send.
2026-10-20 09:54:51 +00:00
mesh-tools b317dfef45 Name only the differing entries of what two sends made, and leave a refused machine out (issue 1384)
The live try cut the lists of the ports allocated and the own secrets made at 2.5 KB, so no difference showed. madeDiffer now says, per kind, the entries made only by the controller, only by mesh-sdk, and by both with a different value; the entries of a machine refused on either side are left out of both, since it sends nothing.
2026-10-20 09:54:06 +00:00
mesh-tools e4d1750015 Seal the stand-in vault box by the resource's id, and set the setting before the machines are assigned in the served-fact tests (issues 1385, 1382)
The live try (build-1792488565282382172) left a stand-in vault box in f35 and yvbpsm: the file was matched by the manifest's path, which the composition fills in for the machine. The seal now matches by the resource's id alone and names any stand-in that is left. The two served-fact tests set the setting before assigning (an assignment resolves the mesh), and the second clears it after.
2026-10-20 09:53:36 +00:00
mesh-tools fb01d514a9 Delete internal/facts: the snapshot's type is mesh-sdk's go/facts (issue 1390, 4 of 4)
Its tests moved with it: mesh-sdk's go/facts holds the same ones, plus the grants of format 2.
2026-10-20 09:41:22 +00:00
mesh-tools b692a95280 Read the facts snapshot with mesh-sdk's go/facts, formats 1 and 2, and keep writing format 1 (issue 1390, 3 of 4)
The merge gate, the build seat's check and the facts writer import the snapshot's type from mesh-sdk (go/facts), whose reader takes formats 1 and 2 and refuses a newer one. The writer keeps writing format 1 (writtenFormat); moving it to 2 is a release of its own (novox/hq ADR 0456). A test shows the writer writes format 1, carries no grants or bindings, and the reader takes both formats and refuses a newer one.
2026-10-20 09:41:16 +00:00
mesh-tools 0e2ab4a96a Vendor mesh-sdk's go/facts and its compose input at ab0c16dd (issue 1390, 2 of 4) 2026-10-20 09:40:34 +00:00
mesh-tools b3e5af0e0c Pin mesh-sdk at the merge of its go/facts format 2 (ab0c16dd, issue 1390, 1 of 4)
go.mod, go.sum, vendor/modules.txt and the capture note name ab0c16ddad26; the vendored files follow in the next commit, and the captured conformance events are the same at the new commit.
2026-10-20 09:40:24 +00:00
mesh-tools 611113fc63 Test that a controller is judged with the validator it vendors and a node-engine member without a validator is judged all the same (issue 1391)
The host member without validate/ failed first against the old judge ("holds no Go files", the validator could not be put in the judge); the controller member is judged by itself whatever the node-engine sibling brings; the group's judge is no longer keyed on the node-engine's validator.
2026-10-20 09:36:41 +00:00
mesh-tools 344b6beb8d Give the judge fixtures mesh-sdk's validator and drop the tests of a validator judge (issue 1391)
The judge fixtures vendor mesh-sdk's validate package, as the controller does; the test of the judge built with a node-engine change's own validator goes with that judge; the replay of issue 867 no longer has the controller change need a validator word of the node-engine's.
2026-10-20 09:36:11 +00:00
mesh-tools 0fc455aa65 Judge a group's controller by itself with the validator it vendors, and copy nothing of mesh-host's (issue 1391, 3 of 3)
judgeFromTheGroup copies nothing into the judge's vendor tree: a mesh-host member whose tree holds no validate/ does not fail the judge, and the verdict names the validator as mesh-sdk's at the pin of the controller that judges. replaceGoFiles and its test are gone.
2026-10-20 09:35:38 +00:00
mesh-tools 52d5e38729 Build no judge with the change's validator for a node-engine change (issue 1391, 2 of 3)
judgeFor judges a node-engine change by the controller the mesh runs, as any module is, and copies nothing of mesh-host into the judge's vendor tree.
2026-10-20 09:35:05 +00:00
mesh-tools 70945de96e Judge a node-engine change by the controller the mesh runs: the scope sets no validator judge (issue 1391, 1 of 2)
pullScope no longer gives a mesh-host change a Judge, and link.JudgeValidator is gone: the validator is mesh-sdk's, at the controller's pin (ADR 0446 clause 5). An older request that says validator is read as the controller the mesh runs.
2026-10-20 09:34:29 +00:00
mesh-tools e98bf1efcb Test the controller's seal under a silent vault, the bus machine's refusal and a served fact that says a setting (issues 1385, 1383, 1382) 2026-10-20 09:28:48 +00:00
mesh-tools b72af5e918 Seal a core credential by the controller where the vault is silent, and refuse the bus machine where no user has a credential (issues 1385, 1383)
Where the vault did not answer for a core credential and no box is kept, ADR 0396 has the controller seal it itself until the vault holds it: perform names those modules, and the send's declaration carries the controller's own seal in place of the stand-in vault box, as the controller's composition sent it. The machine that runs the bus is refused with busUserList's words where no user has a credential, as before; no other machine is. A machine whose recorded module cannot be kept is refused alone.
2026-10-20 09:28:09 +00:00
mesh-tools 09229d85cb Read the settings before the first pass resolves any machine, and refuse one machine alone for what it cannot keep or run (issues 1382, 1383)
The first pass of the adapter resolved every machine with no settings in the Input, so a provider whose served fact says ${setting:domain} could not be settled and the whole Input failed (the equality try on the live facts). The settings of every module that has any are read first; a module's settings that do not read are an error only for a machine that resolves to it. A recorded module that cannot be kept at the build a machine runs refuses that machine alone, as planFor did, and no longer the whole Input. The bus's refusal (no user has a credential) is carried for the machine that runs the bus.
2026-10-20 09:27:31 +00:00
mesh-tools bd17a041de Read the rest of the mesh and the network over mesh-sdk, and fail when a caller reaches for the old composition (issue 1388)
The seats, holdings, source, build-source, delivery group check, check request, backup plan, adoption (take and converge) and the network graph now read the rest of the mesh through restOfTheMesh and compose.Graph. compose_callers_test.go scans the package and names any function other than the old composition's own that still calls planFor, declarationWith, composedAndValidated, generators, renderingFor, theRestOfTheMesh or whoResolves; issue 1389 deletes those.
2026-10-20 09:15:34 +00:00
mesh-tools 7855ea126c Ask over mesh-sdk's composition: a question makes no pair credential and reads the mesh once (issue 1387)
The callers that only ask (status, the doctor's probes, facts, bindings, data, acts, nodes show, images, release, recorded push, consumers) resolve through resolveNode and compose through composeQuestion, over one Input read per operation (withInputCache). A question composes with Reading or Foreseeing and never calls PairCredential's maker: the credential a send would make is named, and perform makes it on a send.
2026-10-20 09:14:25 +00:00
mesh-tools f693d7e171 Send over mesh-sdk's composition: push, the pushes of plans and rollouts, held back, plan and the merge gate's passes (issue 1386)
Every send composes through composeSend (compose over the Input, perform what it named foreseen, read the stores again, compose again, refuse a declaration that still stands in for a value): push.go's two composing sites, composeForPush, declarationFor (the plan verb, which allocates on purpose for one named machine) and the merge gate's two passes and its trial assignment (composeSendValidated, which answers the node-engine's validator beside it). Each machine of a send reads the mesh afresh (freshInput), because the send before it made and recorded things. None calls declarationWith with Allocating any more. The private network is still computed up front, so a send refuses on it before anything is composed. Also here, as they share the helpers: whoResolvesOn, restOfTheMesh and restOfTheMeshOf in compose_sdk.go, and the planFor callers in held_back.go, plan.go and push.go.
2026-10-20 09:12:24 +00:00
mesh-tools 647d20af98 Compose every machine both ways and show the two agree (issue 1384, ADR 0453 clause 3)
TestEveryMachineComposesAlikeBothWays reads the facts snapshot MESH_FACTS names (the live one in every check and try), else testdata/facts-equal.json, else the snapshot of a mesh of its own; raises it into throwaway stores as the merge gate does; and compares, machine by machine: (a) Foreseeing on the fresh store (error or none, the validator's problems, LeftOut and its reasons, Withheld, Unbound, Unplaced, HandedOnly, the own secrets named foreseen, and the ports mesh-sdk names foreseen against the ones the controller's send allocates); (b) the send, on two stores, the controller's Allocating composition and the adapter's compose, perform and compose again (ports, pair credentials and own secrets made, by key); (c) after the send, Reading both ways on one store, byte for byte. A difference names the machine, the comparison and the first resource and field. The comparison itself is tested without a store, and failed first when the body was not read. The test is deleted with the controller's composition (issue 1389).
2026-10-20 09:03:20 +00:00
mesh-tools 8be76b78da Perform what mesh-sdk's composition names foreseen, and compose a send over the SDK: perform, composeSend, composeQuestion, resolveNode (issues 1385, 1386)
A composition over the Input makes nothing; a send performs what it named (the pair credentials no row holds, the seats to pass, the ports and own secrets, the core credentials asked of the vault every send, the certificate), reads the stores again and composes again, and refuses a declaration that still stands in for a value anywhere in its body. A question (composeQuestion) never performs. The operation's Input is read once and kept in its context until a send changes what it read, per what the context keeps (recorded builds, the grants step, a settings walk). A core credential the vault did not answer for and no box is kept for refuses the send, where the controller sealed it itself (ADR 0396): the composition has no such fallback (reported). Callers switch in the next commits.
2026-10-20 09:01:42 +00:00
mesh-tools 7515320b72 Read the credentials and the builds into mesh-sdk's composition input, and make nothing (issue 1383)
How each pair credential a node takes stands (held with its sealed consumer copy, the next send makes it, only a person's to give and nobody gave, or refused in the controller's words), the credentials made for each provider's consumers, the own secrets with their origin and key and value, the vault's kept boxes, the bus's users, minted users and memberships, the mesh's authority and each machine's serving key and kept certificate, the repositories built, the current builds and the build of each commit a machine that reads them is sent. A credential no row holds is not made (planFor made it in every mode): the adapter returns it as would-make for perform (issue 1385). New reads in the inventory (OwnSecretRows, PairCredentialState, BusMinted) and the identity (KnownAuthority, KeptCertificate) write nothing.
2026-10-20 08:58:32 +00:00
mesh-tools 468ecd482e Read the controller's stores once into mesh-sdk's composition input: the nodes and the network (issue 1382)
readComposeInput asks the stores for the Input's mesh-wide and per-node fields (the shelf, each node's place, assignments, capabilities, accounts, pins, bindings, settings, ports, sealing key, taken modules, links, networks, order and builds, the recorded modules it keeps at the build it runs, its node-engine's contract) and writes nothing. A node's kept manifests are its own: the Input's Catalogue stays the current shelf (R4a review round 1, finding 1). Settings are read for the modules each node resolves to, as planFor read them. keepRecorded is split so the adapter reads the kept manifests with its refusals in the same words (applyKept). The credential and build half is issue 1383; vendor/modules.txt now lists go/compose and no longer the mesh-host validator.
2026-10-20 08:55:44 +00:00
mesh-tools 5eaaeb7135 Vendor mesh-sdk's go/compose and drop the node-engine's validator from vendor/ (issues 1381, 1382)
go mod vendor writes go/compose (copied on the forge from mesh-sdk 14fd5cb68fd1, equal to its blobs) now that the adapter imports it, and no longer writes github.com/novox/mesh-host/validate and internal/declaration, which nothing imports since the validator comes from mesh-sdk (issue 1381; the swap commits left those five files behind). vendor/modules.txt follows with the adapter.
2026-10-20 08:38:52 +00:00
mesh-tools b9b5a5e1dc Delete internal/catalogue: mesh-sdk's go/catalogue is the one catalogue package (issue 1381)
Nothing in this repository imports it any more; go/catalogue in mesh-sdk 14fd5cb68fd1 is a copy of it (R4a's review diffed it against 6c2a8d62). The tests that read this repository's own files live on in internal/ownmanifest.
2026-10-20 08:30:50 +00:00
mesh-tools 926444373d Delete the second half of internal/catalogue's tests: mesh-sdk's go/catalogue holds them (issue 1381) 2026-10-20 08:30:25 +00:00
mesh-tools e4364a1c86 Delete the first half of internal/catalogue's tests: mesh-sdk's go/catalogue holds them (issue 1381)
The tests that read this repository's own files (controller_is_a_process, genesis_image, genesis_registry and foundation_manifests' registry test) live on in internal/ownmanifest; engine_guard_test and the rest are in mesh-sdk (issue 1379).
2026-10-20 08:30:14 +00:00
mesh-tools 4d74e06ed8 Delete internal/overlay: mesh-sdk's go/overlay is the one overlay generator (issue 1381)
No Go file imports it any more; go/overlay in mesh-sdk 14fd5cb68fd1 is a copy of it with only import paths and a doc.go changed (R4a's review diffed it).
2026-10-20 08:29:36 +00:00
mesh-tools 928942c78c Take the catalogue from mesh-sdk in internal/recordmirror, finish the own-manifest tests, vendor/modules.txt for the pin, the capture note and the slow-package comment (issue 1381, 6 of 6 import swaps)
With this commit no Go file outside vendor/ and the two packages about to go imports internal/catalogue, internal/overlay or mesh-host/validate, go.mod and vendor/modules.txt agree on mesh-sdk 14fd5cb68fd1, and the tree builds. testdata/beside/CAPTURED names the pinned SDK commit (conformance/events is unchanged since aa68b673). merge-check.sh's comment no longer names internal/catalogue as a slow package.
2026-10-20 08:29:11 +00:00
mesh-tools af267cb557 Take the catalogue from mesh-sdk in internal/inventory and internal/link, and keep the tests that read this repository's own manifest, Dockerfile and Makefile in internal/ownmanifest (issue 1381, 5 of 6 import swaps)
The three tests of the controller's own module.json (process, claimed seat, addresses), the Dockerfile and Makefile pin test, the build-agent registry test and the manifest-in-use seat test move with their old package, as tests of mesh-sdk's exported API; a moved test is exempt from failing first.
2026-10-20 08:27:10 +00:00
mesh-tools bfe7fe1530 Pin mesh-sdk at 14fd5cb6 (R4a's merge) and take the catalogue, overlay and validator from it, s to inventory (issue 1381, 4 of 6 import swaps)
go.mod and go.sum move to mesh-sdk 14fd5cb68fd1, the commit on its main that holds go/catalogue, go/overlay and go/validate (ADR 0453 clause 2); vendor/modules.txt follows in the last of the six commits.
2026-10-20 08:26:01 +00:00
mesh-tools 3208aa0461 Take the catalogue, overlay and validator from mesh-sdk in cmd/mesh-controller, p to s (issue 1381, 3 of 6 import swaps) 2026-10-20 08:24:43 +00:00
mesh-tools 4c84ac992b Take the catalogue, overlay and validator from mesh-sdk in cmd/mesh-controller, g to p (issue 1381, 2 of 6 import swaps) 2026-10-20 08:23:26 +00:00
mesh-tools 013dad4688 Take the catalogue, overlay and validator from mesh-sdk in cmd/mesh-builder and cmd/mesh-controller, a to g (issue 1381, 1 of 6 import swaps)
Swaps three import paths for mesh-sdk's go/catalogue, go/overlay and go/validate (the same packages, copied from this repository at 6c2a8d62) and lets gofmt order the groups. The six swap commits compile together once the pin commit (go.mod, modules.txt) is on the branch; each alone does not, because a file on the SDK's types cannot call one still on the old package.
2026-10-20 08:22:07 +00:00
mesh-tools f64e00e83c Vendor mesh-sdk's catalogue, overlay, validator, declaration and the new delivery parts, copied from its main 14fd5cb6 (issue 1381)
The files go/mod vendor writes for the pin that follows in the next commit, copied on the forge from mesh-sdk at 14fd5cb68fd1 (each equals the blob there; the vendored tree excludes tests). Split from the pin because one commit cannot carry both a patch and copies.
2026-10-20 08:19:32 +00:00
mesh-tools b280b9de10 Wait on the watch having handled the entry, not 2 s twice, in the pull-key watch test (issue 1337)
The unit level ran at 4.4 s of its 5 s bound; this one test took 4.00 s of internal/link's 4.37 s by waiting a fixed 2 s for two things that must not happen. It now waits on the fake watch having handled the entry (two filler entries pass the one-slot channel only after the real one was handled) and on the watches having stopped after a move. Broken in turn (a verdict alone moving the check, a move told twice) it fails.

Unit level at this commit: 2454 tests (6 skipped), 2.84 s (bound 5 s); internal/link 0.36 s. Five slowest tests: TestNoPlanIsWrittenButThroughSavePlansMoving 0.64 s, TestAResolverThatAnswersNothingIsHeldAndSaidInMachineNames 0.48 s, TestNoCodeNamesATableOfTheChain 0.31 s, TestEveryUplinkListsBothResolversItsOwnFirst 0.29 s, TestEveryWordingIsPlain 0.28 s. No test moved to the integration level.
2026-10-20 08:12:38 +00:00
mesh-tools 6c2a8d6210 Merge pull request 'Carry a module on the tool runner only when the runner serves something of it, so a part on its own credential has one writer (issues 1329, 1338)' (#404) from fix/1329-runner-carries-only-what-it-serves into main 2026-10-19 23:39:24 +00:00
mesh-tools c07f604047 Make the rollout tests wait for the plan look to return before their cleanups restore what it reads, which raced (issue 1338)
mesh/merge-gate head c07f604047be merged into trunk 70cb217601f9 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head c07f604047be merged into trunk 70cb217601f9 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 23:23:23 +00:00
mesh-tools b893cc829c Keep carrying a module whose bundle the tool runtime loads, as route-proxy's helper writes its state on the runner's account (issue 1329) 2026-10-19 23:14:51 +00:00
mesh-tools 70cb217601 Merge pull request 'Answer a single pull request's check with its request id on clean standard output (issue 1311)' (#403) from feat/no-terminal into main 2026-10-19 23:11:38 +00:00
mesh-tools 1d0df24a0d Carry a module on the tool runner only when the runner has something of it to serve, so a part on its own credential has one writer (issue 1329) 2026-10-19 23:10:48 +00:00
mesh-tools 202b453de1 Point standard output at standard error while a one-head delivery check runs, so its answer is the only JSON (issue 1311)
mesh/merge-gate out of date: judged against trunk 40350d82fa20, the trunk is now 70cb217601f9; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 40350d82fa20, the trunk is now 70cb217601f9; judged again on a recheck or when it is merged
mesh/delivery delivered
2026-10-19 22:52:00 +00:00
mesh-tools 7a0b275020 Keep delivery check standard output the answer alone and answer the request id asked (issue 1311) 2026-10-19 22:46:30 +00:00
mesh-tools 40350d82fa Merge pull request 'Ask a failed delivery's Retry on the phone, tell a refused answer, and pass a seat to its only claimant (ADR 0449, ADR 0451)' (#402) from feat/no-terminal into main 2026-10-19 22:44:07 +00:00
mesh-tools 1f708e2d8d Merge pull request 'Answer the what-if as data, let the build seat read the gate's pull keys, and prove the parts' and the gate's grants (design 47 R3d, ADR 0450)' (#401) from feat/r3d-sdk-and-controller into main 2026-10-19 22:33:11 +00:00
mesh-tools 7f23c537cc Let a retry see every plan made after the failed one, and pin the 7-day end of plan-failed (issue 1328, ADR 0451)
mesh/merge-gate head 7f23c537cced merged into trunk 1f708e2d8d72 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 7f23c537cced merged into trunk 1f708e2d8d72 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 21:48:41 +00:00
mesh-tools 0aac89f4e2 Test that the gate's buckets are writable by its machine's tool runner account only (issue 1326)
mesh/repo-check head 0aac89f4e22c merged into trunk caa3fc899501 — pass: its merge-check.sh passed
mesh/merge-gate head 0aac89f4e22c merged into trunk caa3fc899501 — pass: group feat/r3d-sdk-and-controller composed: every machine composes with the change…
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792449194411267197 (novox/mesh-controller at 1f708e2d into ma…
mesh/delivery-group group feat/r3d-sdk-and-controller merged: every member is merged
ADR 0437 §3's check: the gate, written in the test as the clause declares it, is composed with the parts and the view
through Users; its seven buckets are writable by its machine's tool runner and by no other account, and each part
reads only the gate buckets its manifest declares.
2026-10-19 21:48:07 +00:00
mesh-tools d8c13d97ca Test that the forge bridge's, indexer's and teller's credentials grant exactly their manifests (issue 1243)
ADR 0437 §4's check: the three manifests, copied from mesh-delivery's main, are composed through Users and
PermissionsFor; each may do what it declares and is refused another module's bucket, another module's name and a seat's
verb subject.
2026-10-19 21:47:47 +00:00
mesh-tools f1775281e2 Refuse a retry of a failed plan that a newer done plan of its branch has passed (issue 1328, ADR 0451) 2026-10-19 21:45:48 +00:00
mesh-tools 4083755ca1 Let the build agent read the gate's pull keys and watch both buckets (issue 1325)
From R3e the gate writes mesh-delivery-gate_pulls and today's bucket stops (ADR 0450 §1). The build agent's manifest
reads both, and a running check's watch listens in every bucket that exists, so a move is heard whichever holds the key.
2026-10-19 21:42:37 +00:00
mesh-tools 7b83aaa949 Tell a refusal as one and an unanswered call as not known; ask a plan that fails again within the hour (issues 1316, 1314, review round 1) 2026-10-19 21:40:27 +00:00
mesh-tools 23a209f3d0 Answer plans what-if as data, the document delivery-plan answers (issue 1324)
The gate reads a delivery's plan as data once delivery-plan goes (ADR 0450 §2). The data form is an explicit
switch, `data: true`, as plans wait asks for its data: a person still gets the text. Both verbs build their
document from one function, deliveryPlanDocument, so they cannot differ; delivery-plan is unchanged (ADR 0450 §8).
2026-10-19 21:40:26 +00:00
mesh-tools 3448f98000 Make a seat pass no repair, read every machine before passing, keep the gate's pass in memory (issue 1317, review round 1) 2026-10-19 21:38:41 +00:00
mesh-tools 12db1d3748 Tell the operator when the answer to a plan's ask is refused (issue 1316, ADR 0449) 2026-10-19 21:30:33 +00:00
mesh-tools 7770d193da Word and perform a failed plan's Retry in the asker (issue 1315, ADR 0449) 2026-10-19 21:28:57 +00:00
mesh-tools 248fe521a3 Raise plan-failed for a failed delivery plan, with Retry when a retry would run (issue 1314, ADR 0449) 2026-10-19 21:26:43 +00:00
mesh-tools 666dcff79b Pass a seat held once to its only claimant when its recorded holder is assigned nowhere (issue 1317, ADR 0449) 2026-10-19 21:21:32 +00:00
mesh-tools caa3fc8995 Merge pull request 'Make a module's consumer from now and refuse a start word in consumes (issue 1279, ADR 0443)' (#400) from fix/1279-consumers-from-now into main 2026-10-19 20:22:43 +00:00
jschoubben a0c971cea9 Make a module's consumer from now and refuse a start word in consumes, so no part replays a week at its first start (issue 1279, ADR 0443)
mesh/repo-check head a0c971cea92e merged into trunk 880207649956 — pass: its merge-check.sh passed
mesh/merge-gate head a0c971cea92e merged into trunk 880207649956 — pass: group fix/1279-consumers-from-now composed: every machine composes with the change…
mesh/delivery delivered
mesh/delivery-group group fix/1279-consumers-from-now merged: every member is merged
2026-10-19 21:53:08 +02:00
mesh-tools 8802076499 Merge pull request 'Raise a delivery stalled line with the severity it says, so a held group reaches the operator as urgent (issue 1278)' (#399) from fix/1278-stalled-line-severity into main 2026-10-19 18:48:08 +00:00
jschoubben 2d2e736b45 Raise a delivery's stalled line with the severity mesh-delivery gives it (issue 1278)
mesh/merge-gate head 2d2e736b4592 merged into trunk fdfda0e839e9 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 2d2e736b4592 merged into trunk fdfda0e839e9 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 20:39:26 +02:00
mesh-tools fdfda0e839 Merge pull request 'Hand a group's members to the delivery verbs in a file when they are long, not as one argument (issue 1272)' (#398) from fix/1272-members-not-an-argument into main 2026-10-19 16:16:57 +00:00
jschoubben 540cda11df Hand a group's long members list to the verb's command in a file, not one argument (issue 1272)
mesh/merge-gate head 540cda11dfd5 merged into trunk ad753c70f6d4 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 540cda11dfd5 merged into trunk ad753c70f6d4 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 18:06:45 +02:00
mesh-tools ad753c70f6 Merge pull request 'Pass a check's changed paths in a file, so a change of thousands of files can start and a check that cannot start says why (issue 1247)' (#396) from fix/1247-changed-paths-file into main 2026-10-19 15:15:31 +00:00
jschoubben da20fea549 Merge remote-tracking branch 'origin/main' into fix/1247-changed-paths-file
mesh/merge-gate head da20fea5495d merged into trunk 2f4f50b9ca8b — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head da20fea5495d merged into trunk 2f4f50b9ca8b — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 16:54:58 +02:00
mesh-tools 2f4f50b9ca Merge pull request 'Announce a plan from a copy that shares nothing with the walk (issue 1260)' (#397) from fix/1260-plan-announce-copy into main 2026-10-19 14:52:39 +00:00
jschoubben 14f405ece5 Announce a plan from a copy that shares nothing with the walk, which goes on writing its steps (issue 1260)
mesh/merge-gate head 14f405ece508 merged into trunk 5c807808e88a — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 14f405ece508 merged into trunk 5c807808e88a — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 16:42:53 +02:00
jschoubben 0d1d780d43 Merge remote-tracking branch 'origin/main' into fix/1247-changed-paths-file
mesh/merge-gate out of date: judged against trunk 5c807808e88a, the trunk is now 2f4f50b9ca8b; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 5c807808e88a, the trunk is now 2f4f50b9ca8b; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
2026-10-19 15:44:18 +02:00
mesh-tools 5c807808e8 Merge pull request 'Count a module's own stream events beside its emits in the catalogue agreement (issue 1249)' (#395) from fix/1249-stream-events-agree into main 2026-10-19 13:42:14 +00:00
jschoubben da5a5c9b67 Give the gate a short changed list as an argument, as before, so an older judge still runs (issue 1247, review round 1) 2026-10-19 15:40:20 +02:00
jschoubben 07a8f04e65 Count a module's own stream events beside its emits in the catalogue agreement (issue 1249)
mesh/merge-gate head 07a8f04e65a3 merged into trunk a5a43c8f31f0 — pass: the change touches no module of the mesh's graph
mesh/repo-check head 07a8f04e65a3 merged into trunk a5a43c8f31f0 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 15:31:13 +02:00
jschoubben dd802c81a7 Pass a check's changed paths in a file so a change of thousands of files can start (issue 1247)
One environment string over 128 KiB made the container fail to start with no words; the gate reads --changed-file, and a command that cannot start now says its error.
2026-10-19 15:22:00 +02:00
mesh-tools a5a43c8f31 Merge pull request 'Find the delivery seat's holder by its claim, and record each build's fingerprint inputs (design 47 R3b: issues 1210-1216)' (#394) from feat/delivery-seat-by-claim into main 2026-10-19 10:28:30 +00:00
jschoubben 3e7f34d041 Test that the check request carries the pulls buckets (issue 1212, review round 1 finding 1)
mesh/repo-check head 3e7f34d041b1 merged into trunk 59c8cf2f2406 — pass: its merge-check.sh passed
mesh/merge-gate head 3e7f34d041b1 merged into trunk 59c8cf2f2406 — pass: group feat/delivery-seat-by-claim composed: every machine composes with the change…
mesh/delivery delivered
mesh/delivery-group group feat/delivery-seat-by-claim merged: every member is merged
2026-10-19 12:09:04 +02:00
jschoubben e29e855bdf Name the pulls buckets of the seat's claimants on the check, so the build agent reads them (issue 1212) 2026-10-19 12:04:26 +02:00
jschoubben da3c01089e Prove a module may say a condition to the operator only under its own name (issue 1216) 2026-10-19 12:01:59 +02:00
jschoubben 253283d68e Read a merge's warrant under the asker its announcement names, the teller or mesh-delivery (issue 1213) 2026-10-19 12:01:12 +02:00
jschoubben 3f038b989e Hear a merge's announcement from whichever module claims the delivery seat (issue 1211) 2026-10-19 11:59:21 +02:00
jschoubben 3834c72747 Find the delivery's owner on the controller's own path by the seat's claim, not its name (issue 1210) 2026-10-19 11:57:07 +02:00
jschoubben 68088ec782 Record the inputs of a build's source fingerprint in its build record (issue 1215) 2026-10-19 11:54:21 +02:00
jschoubben 8da2619920 Compute the source fingerprint with mesh-sdk's one function (issue 1214) 2026-10-19 11:52:17 +02:00
mesh-tools 59c8cf2f24 Merge pull request 'The builds list carries each build's source fingerprint, so builds shows it (issue 1208)' (#393) from fix/1208-builds-list-source into main 2026-10-19 09:00:24 +00:00
jschoubben 3ac8657a49 Carry the source fingerprint in the builds list, so builds can say it (issue 1208)
mesh/merge-gate head 3ac8657a49bf merged into trunk 230f0a1a18f2 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 3ac8657a49bf merged into trunk 230f0a1a18f2 — pass: its merge-check.sh passed
mesh/delivery delivered
Issue 1204's line never showed: chainBuildList copied each build field by field and left the fingerprint out.
2026-10-19 10:46:57 +02:00
mesh-tools 230f0a1a18 Merge pull request 'builds says each build's source fingerprint (issue 1204)' (#392) from fix/1204-builds-source into main 2026-10-19 08:43:30 +00:00
jschoubben 6e487fa01f Say each build's source fingerprint in builds, the one tool that reads what the builder recorded (issue 1204)
mesh/merge-gate head 6e487fa01f63 merged into trunk 7a74cee83f88 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 6e487fa01f63 merged into trunk 7a74cee83f88 — pass: its merge-check.sh passed
mesh/delivery delivered
The builder records a build's source fingerprint, but no tool read it back, so design 47 R3a could not
reproduce a recorded one (ADR 0437 clause 5).
2026-10-19 10:28:45 +02:00
mesh-tools 7a74cee83f Merge pull request 'A delivery does not send to or wait for a machine already away when its tier is sent (issue 1161)' (#391) from fix/1161-away-machine into main 2026-10-19 07:35:28 +00:00
jschoubben 378e65780d Away is presence's verdict alone; a sleeping machine is away once presence says so (issue 1161)
mesh/merge-gate head 378e65780d0b merged into trunk 791b92c875d3 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 378e65780d0b merged into trunk 791b92c875d3 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 09:21:09 +02:00
jschoubben 296e292f55 A machine away when a tier is sent is not sent to or waited for, and the plan says it was away (issue 1161)
ADR 0435 §1: the walk judges away with the silent check's verdict; the machine keeps its want for the backlog send.
2026-10-19 09:15:51 +02:00
mesh-tools 791b92c875 Merge pull request 'A module's process may hold a bus account of its own, and a module stream may be a work queue (ADR 0436, issues 1170, 1177-1181)' (#390) from feat/mail-sender-controller into main 2026-10-19 02:07:51 +00:00
mesh-tools 1490c3b075 Merge pull request 'The controller says when it last heard a machine from presence, and no longer hears the alive message (design 47 R2c, issue 1183)' (#389) from feat/alive-gone into main 2026-10-19 02:01:15 +00:00
jochen 4104d49ec8 mesh-controller.try fault B: derive the test's expected url from what issueProcess is given, not a literal (issue 1181)
mesh/merge-gate head 4104d49ec8b9 merged into trunk 1490c3b0756a — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 4104d49ec8b9 merged into trunk 1490c3b0756a — pass: its merge-check.sh passed
mesh/delivery delivered
The test set MESH_BROKER_ADDRESS to a literal and asserted it, but
issueOnTheNewBus works out the machine's reachable address from the
catalogue's network records, not from this process's environment — the
real try found a different, correct address. Calls issueProcess
directly now, with a known broker and reachable address of its own
choosing, exactly as issued_to_vault_test.go calls issueWith and
issueProvisionerClient directly; the asserted url is that same input.
No production code changed — issueOnTheNewBus reads the right address.
2026-10-19 03:35:57 +02:00
jochen 1991fe931b mesh-controller.try fault A: decide drop-only dead letters by a module's own retention: work-queue, not by the bus's work-queue retention alone (issue 1178)
sourceAge keyed AgainTo's clause-6 refusal on the stream's bus retention
policy, which SEAT_* and other mesh-internal work queues share without
being one of ADR 0436 §6's module streams — so every seat's ask, once
given up on, was refused delivery again. Added a dedicated metadata key
(WorkQueueMetadata), set only by ModuleStream.Stream() when the
catalogue declared retention: work-queue, and read sourceAge from that
instead of the stream's retention policy. New test proves a seat's own
work queue carries neither that metadata nor the restricted one.
2026-10-19 03:35:51 +02:00
jochen d8249dad8b Run messenger on its own account in the process acceptance test, as it holds operator-channel (issue 1181) 2026-10-19 03:24:46 +02:00
jochen df4d1204c7 Review round 2: publish a module stream's event as the tool runner carrying it, not the controller (issue 1181)
Step 3 published mesh.stream.mesh-mail.event.mail.send.approved on the
controller's connection, which the controller is never granted. Added
the runtime module to the fixture's assigned set, gave mesh-mail its
Streams so the carried-module path grants ownStreamGrants, minted the
runtime's password through aMesh like the other principals, and publish
over that connection with its own inbox prefix.
2026-10-19 03:23:16 +02:00
jochen e8c8f0488b Review round 1: check the process's whole publish set, not merely that each wanted subject is in it (issue 1180) 2026-10-19 03:18:26 +02:00
jochen cfb6a8c531 Review round 1: assert the own secret's mode, not merely log it (issue 1170) 2026-10-19 03:18:22 +02:00
jochen 9b7f9a7fd2 Review round 1: a carried module's queue consumer survives PruneHeard, a work-queue stream's restricted metadata is proven on the server config (issue 1178)
ConsumersOf's carried-module path (a module the tool runner carries, no
account of its own — mesh-mail's real shape) was not passing Queues
through, so its work-queue consumer was missing from the composed list
and PruneHeard would have removed what issueProcess made for it. Fixed,
with a test through Users -> ConsumersOf.

Extracted EnsureStream's config building into StreamConfigFor, a pure
function, so the dead-letters test can assert the restricted metadata
EnsureStream would actually send for a work-queue stream instead of
setting the dead letter's header by hand, which passed whether or not
modstreams.go's out.Restricted = true was there at all.
2026-10-19 03:18:18 +02:00
jochen 65632e7999 Review round 1: mesh-mail's real shape has no broker, rollout mint issues its processes, the acceptance test connects with the minted account (issue 1181)
mayIssue no longer requires a module's own broker secret when one of its
processes declares bus: mesh-mail's real manifest has none, its tools
served by the tool runner. rollout mint gained a KindProcess case, so a
process's account rotates the same way every other bus account does. The
release's acceptance test now composes its principals the real way
(inventory.ProcessesOf -> broker.Users -> broker.ComposeAccounts), mints
through the store as module issue does, and sets each connection's own
inbox prefix so replies are not refused; it moved from internal/broker
(which cannot import inventory) to cmd/mesh-controller. Added an
integration test proving the mint, accept and vault hand-off, and that
issuing again rotates the password.
2026-10-19 03:18:08 +02:00
jochen f6a48ac5ef Let module issue mint and deliver a bus process's account (issue 1181) 2026-10-19 02:55:29 +02:00
jochen 93a82cb032 Grant a bus process an account scoped to its own declaration (issue 1180) 2026-10-19 02:44:38 +02:00
jochen de651e0fd1 Give a work-queue stream its one consumer, drop-only dead letters (issue 1178) 2026-10-19 02:33:22 +02:00
jochen 97ee33e8f4 Let a process say what it does on the bus (issue 1179) 2026-10-19 02:17:39 +02:00
jschoubben 19e3eac746 The controller no longer subscribes to the alive message: HostBeats, its handler, the heartbeat-gap duration and a fixed silent bound (issue 1183, AC3, AC4, AC5)
mesh/merge-gate head 19e3eac74658 merged into trunk 89cfa9431d8f — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 19e3eac74658 merged into trunk 89cfa9431d8f — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 02:16:21 +02:00
jschoubben 863db82737 The controller's last-heard word for a machine comes from the watchman's presence in nodes, node, status and the watchdogs (issue 1183, AC1, AC2) 2026-10-19 02:13:56 +02:00
jochen a069898c0c Let a module stream declare work-queue retention (issue 1177) 2026-10-19 02:11:05 +02:00
jochen ffad53aad2 Let an own secret name its owner (issue 1170) 2026-10-19 02:04:00 +02:00
mesh-tools 89cfa9431d Merge pull request 'The controller takes what the watchman says: key-ttl buckets, the checks running readable, stuck keys raised, silence from presence (design 47 R2b)' (#388) from feat/delivery-watch into main 2026-10-18 23:34:07 +00:00
mesh-tools 5939bbdefe Merge pull request 'An approved merge's first build passes the guard of a build through a verb, and a failed one is recorded (issues 1146, 1169)' (#387) from fix/1146-first-build-ask into main 2026-10-18 23:20:17 +00:00
jschoubben ec85cdf70c The system-exports reload test waits for the event naming the user it connected, past the watchman's own (issue 1169)
mesh/merge-gate head ec85cdf70c07 merged into trunk cec26d859695 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head ec85cdf70c07 merged into trunk cec26d859695 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 01:06:56 +02:00
jschoubben cd303f68ed The first-build test reads the asks its stub records under a lock: the merge path asks on its own goroutine (issue 1146) 2026-10-19 01:02:02 +02:00
jschoubben 2cbebbc011 A failed first build of a root module is kept under a key the bus takes, and the real ask's test has the identity store (issue 1146) 2026-10-19 00:58:08 +02:00
jschoubben ac79e29319 Merge remote-tracking branch 'origin/main' into feat/delivery-watch
mesh/repo-check head ac79e29319ee merged into trunk 5939bbdefe5a — pass: its merge-check.sh passed
mesh/merge-gate head ac79e29319ee merged into trunk 5939bbdefe5a — pass: group feat/delivery-watch composed: every machine composes with the change as it d…
mesh/delivery delivered
mesh/delivery-group group feat/delivery-watch merged: every member is merged
# Conflicts:
#	cmd/mesh-controller/doctor.go
2026-10-19 00:53:02 +02:00
jschoubben ef0ca69cb0 The first build of a module a merge adds on the operator's Approve passes the guard of a build through a verb on the warrant it checked (issue 1146, AC9) 2026-10-19 00:48:56 +02:00
mesh-tools cec26d8596 Merge pull request 'An approved merge into a repository nothing is built from asks its modules' first builds (issue 1146)' (#386) from fix/1146-first-build-of-new-repo into main 2026-10-18 22:32:41 +00:00
jschoubben e621e15d2f Its comment names D18 too (issue 1146)
mesh/merge-gate head e621e15d2f5d merged into trunk b1cfb85f86ed — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head e621e15d2f5d merged into trunk b1cfb85f86ed — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 00:20:31 +02:00
jschoubben a08498b240 Name the waiting first merge's probe D18: designs 06 and 54 already name probe D17 (issue 1146) 2026-10-19 00:20:20 +02:00
jschoubben 39ae7de6d3 Say a first merge that waits past ten minutes for mesh-delivery's word, with probe D17 (issue 1146, AC8) 2026-10-19 00:19:03 +02:00
jschoubben 4e12fc19b7 A first merge into a new repository waits kept for mesh-delivery's word, and the word's arrival asks the builds, with no timer; AC7 on the bus in both orders (issue 1146) 2026-10-19 00:16:36 +02:00
jschoubben 19f15d8ea0 A merge on the operator's Approve into a repository the mesh builds nothing from asks the first build of the modules it holds (issue 1146, AC6, AC7) 2026-10-19 00:11:52 +02:00
mesh-tools b1cfb85f86 Merge pull request 'The have bucket takes a node's whole facts, as large as a chain value, never cut (issue 1162)' (#385) from fix/1162-have-node-size into main 2026-10-18 22:07:27 +00:00
jschoubben 75a27ec1a4 The have bucket takes a node's whole facts, as large as a chain value, never cut (issue 1162)
mesh/merge-gate head 75a27ec1a4bc merged into trunk 85af245f1af1 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 75a27ec1a4bc merged into trunk 85af245f1af1 — pass: its merge-check.sh passed
mesh/delivery delivered
A node's own key carries its profile and was refused at 4 KiB on novox and g14
(nats: message size exceeds maximum allowed).
2026-10-18 23:51:56 +02:00
mesh-tools 85af245f1a Merge pull request 'A delivery plan passes a machine silent past its bound at a middle tier, as at the last (issue 1160)' (#384) from fix/1160-silent-mid-tier into main 2026-10-18 20:15:32 +00:00
jschoubben 1ec27e6ba1 Read tier 1's build and the note once the plan has reached them in the silent middle tier test (issue 1160)
mesh/merge-gate head 1ec27e6ba18f merged into trunk 461fa9c7b4ed — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 1ec27e6ba18f merged into trunk 461fa9c7b4ed — pass: its merge-check.sh passed
mesh/delivery covered after its stop: a later merge that contains it was delivered: novox/mesh-controller@75a27ec1a4bc (merged as b1cfb85f into main, wal…
2026-10-18 21:24:07 +02:00
jschoubben 425134784a A machine silent past the bound at a tier between the first and the last is named and left as it was, as at the last tier (issue 1160) 2026-10-18 21:15:48 +02:00
jschoubben fb66608a13 A machine whose presence says online is heard now; last heard is otherwise the later of last_seen and the presence time (issue 1156) 2026-10-18 21:04:59 +02:00
jschoubben 7be2df627f The controller raises what the watchman says stopped moving, and takes a machine's silence from its presence (issue 1156) 2026-10-18 21:00:10 +02:00
jschoubben cb006fc8dd A refused send key raises a condition in place of the timer that stood in for it (issue 1157) 2026-10-18 20:57:10 +02:00
jschoubben 0273609090 The checks running are readable through reads and carry when each started (issue 1155) 2026-10-18 20:55:18 +02:00
jschoubben 3b49d7044f A state entry may say key-ttl and its bucket allows a time to live per key (issue 1154) 2026-10-18 20:53:42 +02:00
jschoubben ba59283b4b Pin mesh-sdk at the stuck bucket's merge (issue 1156) 2026-10-18 20:51:32 +02:00
mesh-tools 461fa9c7b4 Merge pull request 'The have bucket and its grants, a module and build per resource, the bus's connect events for the watchman, and a new module registered at its merge's Approve (issues 1136, 1145, 1147, 1146)' (#383) from feat/delivery-have into main 2026-10-18 18:36:12 +00:00
jschoubben 06bf8cb2fb Merge branch 'feat/delivery-have-broker' into feat/delivery-have
mesh/repo-check head 06bf8cb2fb7a merged into trunk 48e9656c670b beside mesh-host#118@aab65b81 — pass: its merge-check.sh passed
mesh/merge-gate head 06bf8cb2fb7a merged into trunk 48e9656c670b — pass: group feat/delivery-have composed: every machine composes with the change as it di…
mesh/delivery delivered
mesh/delivery-group group feat/delivery-have merged: every member is merged
2026-10-18 19:27:59 +02:00
jschoubben 222c9c1d72 Judge migration 105 alone now that 106 follows it, and list the new store tests (issue 1145) 2026-10-18 19:27:15 +02:00
jschoubben 0520773412 The connections request asks for user names, which the server leaves out otherwise (issue 1147) 2026-10-18 19:27:12 +02:00
jschoubben eba864dc5f Pin mesh-host at 9d9e3a46c4f5 and recapture it beside (issue 1145) 2026-10-18 19:18:38 +02:00
jschoubben 2d3d5ab616 Test that declarationWith sets modules and builds for a node that reads them (issue 1145); the Approve of a head it did not cover refuses a new module, heads keyed by the repository (issue 1146) 2026-10-18 19:18:11 +02:00
jschoubben dc65fe2319 Send each resource's module and each module's build to a node that reads them; pin mesh-host at 0b391f49a25f (issue 1145) 2026-10-18 19:11:10 +02:00
jschoubben fc43238267 Merge branch 'feat/delivery-have-broker' into feat/delivery-have 2026-10-18 19:09:38 +02:00
jschoubben 8c8bc5021e The bus's system account exports MESH's connect and disconnect events and connections request, heard only by the watchman (issue 1147) 2026-10-18 19:07:06 +02:00
jschoubben 557c11af1e A second new module from a repository is judged alongside, with no warrant asked: test (issue 1146, AC5) 2026-10-18 19:06:15 +02:00
jschoubben 661e8eff2f The broker asserts the have bucket and grants each node its own keys of it, the controller none (issue 1136) 2026-10-18 19:04:55 +02:00
jschoubben 8c0fa40e65 A new module from a repository nothing registered is built from registers on the operator's Approve of its merge (issue 1146, ADR 0432) 2026-10-18 19:04:38 +02:00
jschoubben 6746ab6492 Pin the SDK with delivery's keys, from a commit the forge keeps (issue 1136)
The old pin named mesh-sdk #28's head, which no branch holds, so a fresh
download of the controller could not fetch it.
2026-10-18 19:00:44 +02:00
mesh-tools 48e9656c67 Merge pull request 'Lay the builds out before the start reads them, so the controller does not exit on its own start (issue 1121)' (#382) from fix/lay-builds-out-before-serving into main 2026-10-18 12:50:37 +00:00
jschoubben a825a4c904 The first read of a chain not laid out lays it out, so no read of the start is refused (issue 1121)
mesh/merge-gate head a825a4c90446 merged into trunk d3556f452e8c — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head a825a4c90446 merged into trunk d3556f452e8c — pass: its merge-check.sh passed
mesh/delivery delivered
The layout gate is switched on when the stores open (UseChainOpener), before
attachTheChain; a read of the start came first, was refused, and the
controller exited. A read that finds no marker now lays the builds out once,
under a lock, and is refused only when that fails (a process without the
lease).
2026-10-18 14:39:43 +02:00
jschoubben 315c622386 Lay the builds out before the start reads them, so the controller does not exit on its own start (issue 1121)
mesh/merge-gate waiting in the build queue since 14:36
mesh/delivery superseded: a newer head of the same pull request
Delivery 2 (#381) gated every chain read on the layout marker and laid the
builds out beside the start; a read of the start then met the gate, the
controller exited, and systemd started it again every five seconds from
14:34 on 2026-10-18. The layout now runs once inside attachTheChain, before
anything reads; it skips keys laid out before, so it takes seconds, and a
failure is retried beside the serving as before.
2026-10-18 14:36:04 +02:00
mesh-tools d3556f452e Merge pull request 'The chain lives on the bus alone: the store's tables of builds, rollouts and assignments are dropped (design 47 milestone 5, delivery 2: issues 1094, 1108, 1109, 1110, 1096, 1111)' (#381) from feat/chain-on-the-bus-d2 into main 2026-10-18 12:30:42 +00:00
jschoubben 68db18bdfc Make the second merge check pass: the gate's mesh has a chain of its own; the tests follow delivery 2's meaning (issues 1121, 1108, 1109)
mesh/merge-gate head 68db18bdfc1b merged into trunk da6d038e07c0 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 68db18bdfc1b merged into trunk da6d038e07c0 — pass: its merge-check.sh passed
mesh/delivery delivered
The merge gate raises the snapshot's mesh in a throwaway store and now gives it a chain in memory, since the chain lives only on the bus. A module with no commit is carried by no send (a send is read from the commits on the assignments' keys), so the grants step flags only a module with a commit as newly assigned. The tests that sent a module with no build record one, the merge test waits for the merges' consumer before publishing and for the last look of a window before the next test, the pending test follows a chain that deletes nothing, and a wait's settling is a value of its key.
2026-10-18 14:21:35 +02:00
jschoubben cbc6bac8c0 Test chains start laid out; two tests follow delivery 2's meaning (issues 1121, 1108, 1109)
mesh/merge-gate head cbc6bac8c070 merged into trunk da6d038e07c0 — error: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; the m…
mesh/repo-check head cbc6bac8c070 merged into trunk da6d038e07c0 — fail: its merge-check.sh failed: --- FAIL: TestTheGrantsStepSendsTheUserListAndNothingEl…
mesh/delivery superseded: a newer head of the same pull request
The fixtures' fresh chain writes the layout marker, as the controller's start leaves a mesh, so the gate stays for the live start. A pending wait that ended unanswered is carried by the next one, so both are known. A hand send forgets what each assignment runs, so the test writes its build and commit after the send.
2026-10-18 13:51:37 +02:00
jschoubben 0b4cd46b3f Merge remote-tracking branch 'origin/main' into chain-on-the-bus-d2-local
mesh/merge-gate head 0b4cd46b3f04 merged into trunk da6d038e07c0 — error: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; the m…
mesh/repo-check head 0b4cd46b3f04 merged into trunk da6d038e07c0 — fail: its merge-check.sh failed: --- FAIL: TestAnAssignmentThatBlocksAnotherMachineSaysW…
mesh/delivery superseded: a newer head of the same pull request
2026-10-18 13:42:00 +02:00
jschoubben 7f074d9c94 A layout run again sets no pointer back and skips records already moved (issue 1121)
The live start recorded its layout in chain_moved, so the marker is missing and the layout runs once more over the old keys. It wrote newest, worked, held, commit and source over the pointers of every build made since. Pointers are now made only where missing, and a record already under its new key is not read again.
2026-10-18 13:41:25 +02:00
jschoubben e76ba966b3 Merge fix/1121-holds-released: delivery 2 on the builds' new layout (issue 1121)
Delivery 2 deleted the move and chain_moved, so the layout of the builds under their modules is made by the controller's start (LayBuildsOut), recorded by the key layout.done in the builds bucket, retried every half minute with an urgent condition while it fails; the chain on the bus is refused until it exists. Its source readers use the module and pointer reads.
2026-10-18 13:39:24 +02:00
mesh-tools da6d038e07 Merge pull request 'Keep builds on the bus under their module, so no lookup reads the bucket whole (issue 1121, ADR 0427)' (#380) from fix/1121-holds-released into main 2026-10-18 11:39:08 +00:00
jschoubben c795915315 Keep builds on the bus under their module, so no lookup reads the bucket whole (issue 1121, ADR 0427)
mesh/merge-gate head c79591531575 merged into trunk aa02e7bf4e78 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head c79591531575 merged into trunk aa02e7bf4e78 — pass: its merge-check.sh passed
mesh/delivery delivered
Milestone 5 moved 8,599 build records onto the bus under bare build ids, each
carrying its module's whole manifest. Every build lookup then read the whole
bucket, and a delivery plan's send did that per module per machine while it
held the plans and the machine's declaration: the send of build-agent never
finished, and every other act and the status waited behind it.

Builds now live under <module>.build.<id> (requests under req.<id>, id.<id>
names a build's module), with pointer keys newest, worked, held,
commit.<sha> and source.<reuse key> that move only to a newer build and are
read as hints, repaired when wrong. Manifests are kept once in the same
bucket under manifest.sha256-<hex>, so no grant the running bus lacks is
needed. The old keys are laid out once at start, idempotently, and left
unread. The calls a plan look makes on the silent clocks while it holds the
plans are bounded, since the vendored kv.Keys ignores its context.
2026-10-18 13:32:31 +02:00
jschoubben 4eeba8fb20 The copy of sent_builds never revives an ended key; a send empties its node's column (issue 1109)
A found ended or pending key is skipped and a pulled-in key is created only where no key exists. RecordSentWith and RecordSentUnder set sent_builds = null in their own update of the node row, so a machine delivery 2 has sent is never copied afterwards. These two updates and the copy's own null are the only writes to the column.
2026-10-18 05:40:50 +02:00
jschoubben d095a3dcef Copy node.sent_builds onto the assignments' keys once, and keep the column; list the new store tests (issues 1109, 1096)
Delivery 1 wrote no commit on the key of a pulled-in module, so after the upgrade those modules fell out of SentBuilds. At the start of the lease holder a copy fills every key that names no commit (creating a standing pulled-in key where none stands), overwrites nothing, empties each column it copied, and raises an urgent condition while it fails. Migration 0105 no longer drops node.sent_builds (issue 1112 does).
2026-10-18 05:39:28 +02:00
jschoubben 5b08eb09be Integration tests: no table of the chain in the store after a live delivery; migration 0105 refuses unmoved rows and drops the moved (issue 1111) 2026-10-18 05:33:42 +02:00
jschoubben 472744bbaa Migration 0105 drops the tables of the chain behind a guard; a source scan proves no code names them (issue 1096)
The migration raises first unless chain_moved names builds, rollouts and assignments for every kind that holds rows, and unless pending_assignment holds no open or unanswered row; then it drops build_request, build, batched_merge, release_plan, pending_assignment, assignment, raise_assignment_generation(), node.sent_builds and chain_moved. build_gate stays.
2026-10-18 05:32:14 +02:00
jschoubben 9859923664 Delete the dead table paths, the move, the store's assignments mirror and its tick (issue 1110)
Every method of builds, build requests, rollouts, merges and assignments reads and writes the chain alone: the else branches that read the tables, the movers and MoveChain with their tests, chain_moved reads, ChainReads and the flags that switched a kind, the store's assignments mirror (KeepAssignmentPlans needs none of it: it uses only the AssignmentMirrored type, mirrors.go, and AssignmentValues, provisioning.go) and its tick, the move code of the controller's start, and ForTest's chain_moved row. The listed store tests of the move go with their functions; the tests of the readers, requests, assignments and plans that lived beside them stay in chain_store_test.go. Tests that replayed old data migrations over the build, assignment and pin tables are removed with the tables. The static test that no plan is written but through SavePlansMoving now reads the plan's chain path.
2026-10-18 05:30:19 +02:00
jschoubben fdab58ae4f Read what a node was sent from its assignment keys; a pulled-in module keeps a standing key; a hand send is a time (issue 1109)
SentBuilds answers from the keys of the node's assignments that name a commit, known when the node was sent anything and no key says its commit unknown. A send that names its builds writes the commit, build and rollout on each carried module's key (a module nothing assigns gets a standing key marked pulled-in, never a state) and clears what every other key of the node named; a send by hand writes sent-unknown, a time, on every key and clears their commits. A failed write of these fails the record of the send. node.sent_builds is no longer written. The plan pass ends a pulled-in key only when no send carries it.
2026-10-18 05:22:28 +02:00
jschoubben 3b989d8bb3 Callers of the pending assignments on the bucket: a replaced wait keeps its condition; one read of the key through assign (issue 1108)
A wait that ended unmade and whose condition nobody answered is carried by the wait that replaces it, so the operator is never left without the condition for a module still not on its machine, and taking it back, its condition and the orphan sweep read it there. The condition id stays node.module.<id> with the string id. clearOrphaned says a taken-back wait too. TestAnAssignmentWaitingForItsBuildIsOneKeyOfTheAssignmentsBucket enters through the assign verb and reads the key before and after the build's outcome.
2026-10-18 05:19:32 +02:00
jschoubben 88ea361bf6 Fold a pending assignment into the value of its assignment key (issue 1094)
A wait for a build is the value of <node>.<module> with state pending and a pending object (id p-<unix nanoseconds>, a string), so every reader that skips a value with a state skips it, and the controller's own lists skip it through Stands. The transitions are compare-and-set on the key's revision (claim, release, settle, repoint, mark); assigning over a wait carries it into the value that stands, applied when it was waiting; a plan pass never overwrites it; one open wait per machine and module, and a key that holds the module refuses a new one. The table is no longer read or written, ForgetEndedPending is gone (no key is removed), and a wait of a machine the store dropped is none. Callers change type only; their conditions are the next issue.
2026-10-18 05:17:30 +02:00
2694 changed files with 7 additions and 935911 deletions
+3 -1
View File
@@ -2,9 +2,11 @@
# A built binary. The lab writes one here when pointed at the repository root by mistake; # A built binary. The lab writes one here when pointed at the repository root by mistake;
# built artifacts belong in build/, which is already ignored. # built artifacts belong in build/, which is already ignored.
/mesh-builder
/mesh-controller /mesh-controller
/postgres-provisioner /postgres-provisioner
/redis-provisioner /redis-provisioner
# And where `go build` in cmd/mesh-controller leaves it (novox/hq issue 233). # And where `go build` in cmd/mesh-controller leaves it (novox/hq issue 233).
/cmd/mesh-controller/mesh-controller /cmd/mesh-controller/mesh-controller
# No repository commits its dependencies (issue 1305).
vendor/
-45
View File
@@ -1,45 +0,0 @@
# The Go it builds with, pinned here because genesis builds this file with no arguments (novox/hq
# issue 223) — the Makefile passes the same digest. A tag older than go.mod asks for is how
# `make image` broke once before (issue 146).
ARG GO_BASE=golang@sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7236c349a0c
# The control plane's image — for genesis and the lab only. The mesh runs the controller as a Go
# bundle the host starts as a process (module.json; novox/hq issue 213), and builds no image of it.
# Genesis builds this file and raises it as the container the process replaces on the first push
# (mesh-host internal/bootstrap, novox/hq issue 223).
#
# novox/hq ADR 0006: this image is pinned by digest in the bundle the host carries, fetched on a
# machine where no mesh exists yet, and run before there is anything to check it against. So it
# holds the program and nothing else — no shell, no package manager, no libc, nothing with a CVE
# feed of its own. What a person has to audit before trusting a first node is one binary.
#
# There are no CA certificates in here on purpose. Nothing it does today makes an outbound TLS
# connection to a public name: it reaches PostgreSQL on the machine it was raised on, and the
# broker is verified against a fingerprint pinned in a token rather than against a public root
# (novox/hq ADR 0004). Adding them "just in case" would put a trust store in the one image whose
# whole argument is that it contains nothing to reason about.
FROM ${GO_BASE} AS build
WORKDIR /src
# **Nothing is fetched** (novox/hq to-be 45 Phase 1): every dependency is in vendor/, committed, so
# the image builds from this repository alone — the host's validator among them, whose module no
# public proxy is asked for. Dependencies first, so a change to the source does not re-copy them.
COPY go.mod go.sum ./
COPY vendor/ vendor/
# `COPY . .` brings `.git` along, so Go is told not to stamp the commit: the version is said by -X, and a
# rebuild of an unchanged program is the same bytes (novox/hq ADR 0267 rule 6, issue 363).
COPY . .
ARG VERSION=development
RUN CGO_ENABLED=0 GOFLAGS=-mod=vendor GOPROXY=off go build -trimpath -buildvcs=false \
-ldflags "-s -w -X main.version=${VERSION}" \
-o /mesh-controller ./cmd/mesh-controller
FROM scratch
COPY --from=build /mesh-controller /mesh-controller
# Numeric because there is no /etc/passwd to look a name up in. Nothing here needs to be root:
# it opens outbound connections and writes nothing to its own filesystem.
USER 65534:65534
ENTRYPOINT ["/mesh-controller"]
-153
View File
@@ -1,153 +0,0 @@
# novox/hq ADR 0006 — the control plane, in Go.
#
# The image the bundle pins holds the program and nothing else, so the build is static and the
# container is built FROM scratch. That is not a size optimisation: this image is fetched by
# digest and run on a machine where no mesh exists to check anything, and everything in it is
# something a person would have to audit.
VERSION ?= $(shell git describe --tags --always --dirty 2>/dev/null || echo development)
LDFLAGS := -s -w -X main.version=$(VERSION)
# Where `make check` raises PostgreSQL. A high port and a throwaway container: nothing here
# touches a database anybody else is using. Override PG_PORT if this one is taken -- the first
# port chosen was already serving something that had been up for six days.
PG_PORT ?= 55532
PG_CONTAINER ?= mesh-controller-check
PG_IMAGE ?= postgres:17-alpine
export MESH_TEST_POSTGRES ?= postgres://postgres:check@127.0.0.1:$(PG_PORT)/postgres?sslmode=disable
.PHONY: build image check test vet fmt postgres postgres-stop clean
build:
CGO_ENABLED=0 go build -trimpath -ldflags '$(LDFLAGS)' -o build/mesh-controller ./cmd/mesh-controller
# Tagged 'development' as well as by version, because the lab places images by name and a
# scenario naming a version would have to be edited on every build. The version tag is what a
# real bundle pins.
IMAGE ?= mesh-controller:$(VERSION)
DEV_TAG ?= mesh-controller:development
# The Go base the image is built on.
#
# **`make image` was broken and stayed broken**, because the Dockerfile's fallback base was a Go
# older than go.mod asks for: every build died at `go mod download` with "go.mod requires go >=
# 1.26.0", and the pipeline never saw it because the pipeline passes the declared base in. Anybody
# building the image by hand hit it and had to find the digest themselves (novox/hq 04-ISSUES/146,
# what it cost).
#
# **Pinned here since the manifest stopped building an image** (novox/hq issue 213): the mesh builds
# the controller as a Go bundle with its own toolchain, and only `make image` — genesis and the lab —
# still needs a Go base. The digest is the one the manifest declared until then.
GO_BASE ?= golang@sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7236c349a0c
image:
@test -n "$(GO_BASE)" || { echo "no GO_BASE; pass GO_BASE=<image>"; exit 1; }
docker build --build-arg GO_BASE=$(GO_BASE) --build-arg VERSION=$(VERSION) -t $(IMAGE) -t $(DEV_TAG) .
@echo
@docker image inspect $(IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The builder ships as an image too, because it is a module the mesh assigns rather than a program
# somebody starts on a machine by hand.
BUILDER_IMAGE ?= mesh-builder:$(VERSION)
BUILDER_DEV_TAG ?= mesh-builder:development
builder-image:
@test -n "$(GO_BASE)" || { echo "no GO_BASE; pass GO_BASE=<image>"; exit 1; }
docker build --build-arg GO_BASE=$(GO_BASE) -f cmd/mesh-builder/Dockerfile -t $(BUILDER_IMAGE) -t $(BUILDER_DEV_TAG) .
@echo
@docker image inspect $(BUILDER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The provisioner ships as an image too, because it is the thing that makes a sealed credential
# true on a machine -- and the mesh cannot, having discarded the plaintext.
PROVISIONER_IMAGE ?= mesh-provision-postgres:$(VERSION)
PROVISIONER_DEV_TAG ?= mesh-provision-postgres:development
provisioner-image:
docker build --build-arg GO_BASE=$(GO_BASE) -f examples/postgres-provisioner/Dockerfile \
-t $(PROVISIONER_IMAGE) -t $(PROVISIONER_DEV_TAG) .
@echo
@docker image inspect $(PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The object store's provisioner, for the same reason: a bucket and a policy are not files, and
# the mesh cannot make them -- it discarded the credential it would have to use.
OBJECTSTORE_IMAGE ?= mesh-provision-objectstore:$(VERSION)
OBJECTSTORE_DEV_TAG ?= mesh-provision-objectstore:development
objectstore-image:
docker build --build-arg GO_BASE=$(GO_BASE) -f examples/objectstore-provisioner/Dockerfile \
-t $(OBJECTSTORE_IMAGE) -t $(OBJECTSTORE_DEV_TAG) .
@echo
@docker image inspect $(OBJECTSTORE_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The cache's provisioner, for the same reason as the database's: an ACL user is not a file,
# and the mesh cannot make one -- it discarded the credential it would have to use.
REDIS_PROVISIONER_IMAGE ?= mesh-provision-redis:$(VERSION)
REDIS_PROVISIONER_DEV_TAG ?= mesh-provision-redis:development
redis-provisioner-image:
docker build --build-arg GO_BASE=$(GO_BASE) -f examples/redis-provisioner/Dockerfile \
-t $(REDIS_PROVISIONER_IMAGE) -t $(REDIS_PROVISIONER_DEV_TAG) .
@echo
@docker image inspect $(REDIS_PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The proxy that turns a route grant into traffic reaching a workload.
PROXY_IMAGE ?= mesh-route-proxy:$(VERSION)
PROXY_DEV_TAG ?= mesh-route-proxy:development
proxy-image:
docker build --build-arg GO_BASE=$(GO_BASE) -f modules/route-proxy/Dockerfile -t $(PROXY_IMAGE) -t $(PROXY_DEV_TAG) .
@echo
@docker image inspect $(PROXY_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The whole check. Raises a database, runs the repository's own check against it -- merge-check.sh,
# the very script `mesh/repo-check` runs -- and takes the database down again, including when the
# check fails, which is why the teardown is not conditional and the script's exit status is the
# target's.
#
# **It calls the script rather than restating it** (novox/hq issue 431): this target used to list its
# own steps, its formatting step walked vendor/ where the script's does not, and it failed on
# third-party code no change could fix -- so a developer's check and the gate disagreed, and the steps
# after formatting never ran through it. The script is the one list of steps; this target only gives it
# a database (MESH_TEST_POSTGRES, exported above).
#
# **Two levels** (issue 895): the unit level, `go test ./...` untagged, timed and held to a bound; and the integration
# level, `-tags integration`, the tests that start a bus, a store or another process. Each is said with its time.
#
# **Packages in parallel, under the race detector, each test on a bus of its own** (internal/testbus),
# as the script runs them: a bus per test, of the release the mesh runs, makes the suite the same in any
# order, and the timeout bounds a hang to a failure with a stack, never a stalled gate.
check: postgres
@sh merge-check.sh ; status=$$? ; docker rm -f $(PG_CONTAINER) >/dev/null 2>&1 || true ; exit $$status
# The unit level alone (untagged: no bus, no store, no other process), which a developer runs continuously. The tests
# that need a bus or the store carry `//go:build integration` and run in `check`, not here; without a database the
# store's tests would skip rather than fail anyway.
test:
go test -timeout 15m ./...
vet:
go vet ./...
# Quick steps for a developer, not part of `check`. The directories gofmt reads must be the ones
# merge-check.sh lists, never `.`, which walks vendor/ (novox/hq issue 431).
fmt:
@unformatted=$$(gofmt -l cmd internal examples) ; \
if [ -n "$$unformatted" ] ; then echo "not gofmt'd:" ; echo "$$unformatted" ; exit 1 ; fi
postgres:
@docker rm -f $(PG_CONTAINER) >/dev/null 2>&1 || true
@docker run -d --name $(PG_CONTAINER) -e POSTGRES_PASSWORD=check \
-p 127.0.0.1:$(PG_PORT):5432 $(PG_IMAGE) >/dev/null
@printf 'waiting for postgres'
@for i in $$(seq 1 60) ; do \
if docker exec $(PG_CONTAINER) pg_isready -U postgres >/dev/null 2>&1 ; then \
echo ' — ready' ; exit 0 ; fi ; \
printf '.' ; sleep 1 ; \
done ; \
echo ' — never came up' ; docker logs $(PG_CONTAINER) | tail -20 ; exit 1
postgres-stop:
@docker rm -f $(PG_CONTAINER) >/dev/null 2>&1 || true
clean:
rm -rf build/
+4 -215
View File
@@ -1,218 +1,7 @@
# mesh-controller # mesh-controller
**Tier 2 of Novox Mesh — the control plane.** Everything that needs to know about more than one The controller ended under issue 1126. Its jobs moved to the parts that ADR 0590 and ADR 0593 name: the indexer, the
node. assigner, the provisioner, the grants part and the node-engine. Its last module, route-proxy, moved to mesh-networking.
That is the whole test, and it draws the line the host cannot: the host applies and does not This repository holds no code and is archived once the delivery of the group that emptied it (release sdk1) is done.
decide, *because deciding needs knowledge one machine does not have*. Which nodes should run the Its history stays here for reading.
store, which peers belong in an overlay, whether a node has been unreachable for a week — nobody
on a single machine can answer any of them.
The reasoning lives in [novox/hq](https://git.novox.be/novox/hq); this repository carries no
argument that is not settled there.
## What it is not
- **Not the thing that changes machines.** It decides; the host applies. It never reaches into a
node except through the host, over the link, in a bounded vocabulary.
- **Not a database.** There is no mesh database. Each context owns its own store and nothing
outside a context touches it — including nodes, which hold no credential to any of them.
- **Not privileged.** It has no more access to a machine than a declaration can express.
## What exists today
**One context of seven, and one of the things it will do.**
| | |
|---|---|
| `inventory` | node records and enrolment tokens — **built, as far as identity** |
| `identity` | the control plane's signing key, and the keys nodes are known by — **built** |
| `config`, `connectivity`, `provisioning`, `delivery`, `observability` | not built |
| the interface every surface speaks to | not built; its shape is not decided |
```
mesh-controller migrate bring each context's schema up to date
mesh-controller node add <name> create a node record
mesh-controller node list the nodes this mesh knows about
mesh-controller token issue --node <name> a one-time right to join, for an existing record
mesh-controller token issue --new <name> create the record and issue for it
mesh-controller identity show this control plane's signing key
mesh-controller broker show where the broker is, and what to expect there
mesh-controller version what this binary is
```
`migrate` is **step 3 of the foundation bootstrap** — the step the first node cannot get past, run
against a database raised moments earlier from the bundle the host carries.
### Tokens, and what they are missing
A token is **a one-time right to join, issued for a node record** — which is where re-enrolment is
decided, since what an identity binds to is settled when the token is made rather than when it is
presented.
What is built: the secret is 256 bits from the system's random source, shown once, and **stored
only as a hash**, so a copy of this database is not a set of working credentials. It is usable
exactly once and only before it expires, and both are read from the row rather than from a status
something would have had to write. Issuing again for the same node invalidates the outstanding
one — two live tokens are two machines able to join as the same node.
Redemption is a single statement that both finds a live token and spends it, so eight concurrent
attempts on one secret produce exactly one winner. There is a test that runs them.
**A token now carries three of its four parts**, and is one line of base64 a person can copy. The
signing key is real: an Ed25519 key this control plane generates once and keeps, whose public half
travels in every token. A node believes a declaration because it carries a signature that key made
— and pinning only the broker would not do, because it would make the control plane's authority
transitive, so a compromised broker could forge declarations, and since the host applies whatever
the link delivers that is the whole machine.
**All four parts are built.** Given `MESH_BROKER_ADDRESS` and `MESH_BROKER_CERTIFICATE`, a token
carries everything ADR 0004 requires. Without them it carries two, and `token issue` prints it
**and names what is missing** rather than producing something that looks complete and cannot be
used.
**The fingerprint is derived from the certificate, never configured.** A configured one can drift
from the certificate it describes, and a drifted pin is worse than none: every node issued a token
during the drift refuses to connect, and the failure looks like an attack. It is computed over the
DER bytes — what a client actually sees on the wire — so the same certificate re-wrapped with
different line endings still produces the same pin.
### Two contexts, and the rule between them is real
`identity` is the second context and it exists partly to test a claim this repository had made and
never checked: that a context reaches only its own store. It holds `MESH_STORE_IDENTITY`;
`inventory` holds `MESH_STORE_INVENTORY`; there is no setting that reaches both and no way to ask
for one. Run `migrate` with only one and it stops, naming the grant it does not have.
A token needs a node record from one and a signing key from the other. Neither reads the other's
store — the process holding both grants asks each for its part.
### How a node is known
**The node generates a keypair; the mesh records the public half.** The same principle as SSH, and
the same rule `08-connectivity` already applies to the overlay keys — which is where this was
settled all along, though it took being asked directly to notice.
Only the public half is ever stored, and that is the property worth having: **a copy of this
database grants nothing.** It is a list of who to believe, not a set of credentials, which is what
makes *compromise of a node is compromise of that node* literally true.
Exactly one key is live per node. Re-enrolment revokes the one it replaced, in the same
transaction — two live identities for one node record is the stolen-laptop case, with the replaced
machine still believed. The database enforces it as well as the code, and there is a test running
six concurrent enrolments that fails when the constraint is removed.
**Not the machine's SSH host key**, though that was the obvious economy. Host keys are regenerated
by reinstalls and image clones, which would silently un-enrol a node; their lifecycle belongs to
sshd rather than the mesh; and a partial host has no SSH daemon at all.
## Reaching a store
**A context is granted only what it exclusively owns.** No shared writes, no read-only role on
another context's store, and no connection string that reaches more than one.
That is a rule about credentials, so it is built as one. There is no mesh-wide connection setting
and no way to ask for one:
```
MESH_STORE_INVENTORY=postgres://…/inventory
```
A process granted `inventory` holds that variable and no other. Reaching another context's store
is not a matter of restraint — it has no address for it and no credential to present. And it is
how the rule is *checked*: what a context can reach is visible in the declaration that runs it,
as the list of variables it was given.
Each context's database is named after the context. There is deliberately no database named for
the mesh as a whole.
## Migrations
Numbered, embedded in the binary, applied in order, each in a transaction with the row recording
it. The applying is four lines; the rest is refusals, and the refusals are the point:
| it stops when | because |
|---|---|
| a migration that ran has since been edited | the database holds the old version, the repository holds the new one, and nothing holds the difference |
| a migration is numbered below one that already ran | usually two branches taking the same next number — applying it now runs the schema in an order nobody tested |
| two migrations share a number | order is the entire guarantee, and two files with one number have none |
| a file in the directory is not a valid migration name | a misnamed migration would otherwise never run and nothing would say so |
| a migration is empty | it records that something happened and changes nothing, which cannot be told from a mistake |
All of them stop rather than warn. At the moment any of them is true, nobody knows what the
database contains, and there is no correct guess about a schema.
Running it again does nothing. Two copies running at once take an advisory lock, so a restart
during a slow migration does not become two runners racing.
## Building
```
make build the binary
make image the container image
make check gofmt, vet, and every test against a real PostgreSQL
```
`make check` raises a throwaway PostgreSQL in a container and takes it down afterwards, including
when the tests fail. Without one the tests that need a database **skip and say so** rather than
passing quietly — `make test` is the honest subset, not the gate.
Tests run against a real database rather than a fake because what is being tested *is* the
database's behaviour: that DDL is transactional, that an advisory lock serialises, that a
checksum mismatch is caught against a record PostgreSQL actually kept. A fake would assert that
the fake behaves as expected.
Every test here has been confirmed to fail when the behaviour it defends is removed. Two did not,
when first written, and both are now commented with what they were missing.
## Checks that cross into the host's repository
Two things are agreed between this repository and `novox/mesh-host`, and each is a separate struct
on each side. A field renamed on one of them fails **silently** — the crossing succeeds and
something is simply absent — so both are checked by handing one side's real output to the other's
real parser. Neither runs by default; each skips with a reason, because a repository that fails
without its neighbour checked out is a repository nobody can build.
**What this mesh sends, read by the host that receives it:**
```
mesh-controller: ./build/mesh-controller plan <node> --json > /tmp/d.json
mesh-host: MESH_EMITTED=/tmp/d.json go test ./internal/declaration/ -v
```
**What a node says when it joins, read by this mesh:**
```
mesh-host: MESH_ENROL_OUT=/tmp/enrol.json go test ./internal/link/
mesh-controller: MESH_ENROL=/tmp/enrol.json make check
```
The second does more than compare shapes: it seals something to the key that arrived and opens it
with the private half the host kept. A key that is present, correctly named and simply *wrong*
passes every check that only looks at the message.
## The image
**The mesh no longer runs the controller from it** (novox/hq issue 213). The module declares a Go
bundle, `controller`, which the host on the controller's machine unpacks and runs as the process
`mesh-controller` under the account of the same name (ADR 0188 §1, §3). The image stays for what
still runs a container of the controller: genesis, which raises the first controller from it and
installs the module from its manifest (mesh-host `internal/bootstrap`), and the lab. Neither is the
mesh's own build any more — `make image` builds it.
`FROM scratch`, holding one statically linked binary and nothing else — no shell, no package
manager, no libc, no CA certificates.
Not a size optimisation. The bundle a host carries pins this image by digest, and it is fetched
and run on a machine where no mesh exists to check anything and a person is expected to have read
the bundle and believed it. Everything in the image is something that person would have to audit.
## Where the reasoning lives
| | |
|---|---|
| what the control plane is | novox/hq ADR 0006 |
| what it takes to run one, and why Go | novox/hq ADR 0006 |
| a context owns its store, exclusively | novox/hq ADR 0008 |
| schema changes are numbered migrations | novox/hq ADR 0013 |
| a test defends a decision | novox/hq ADR 0017 |
-6
View File
@@ -1,6 +0,0 @@
# Release 57: config and provisioning
This branch carries the work of hq design 57 (ADR 0348, ADR 0349) in this repository. Each package
merges here through its own reviewed pull request; the branch reaches main as one pull request, in one
delivery group with the branch of the same name in mesh-controller, mesh-host, mesh-sdk, mesh-catalog
and hq, after a rehearsal in the lab. This file is deleted by the last commit before that pull request.
-29
View File
@@ -1,29 +0,0 @@
#!/bin/sh
# The unit tests of every Go module of its own below this repository's root, run from the root, for merge-check.sh
# and for the test that tries it on a tree (merge_check_nested_test.go). Its own file so that the test needs no
# particular merge-check.sh: the check that judges a change to merge-check.sh is main's.
set -eu
cgo="${cgo:-0}"
race="${race:-}"
# nested_modules vets and tests every Go module of its own below the root, `modules/<name>/go.mod` (novox/hq ADR 0420
# §5, issue 1025): `./...` stops at a nested go.mod, so a module that lives whole in this repository would have its
# unit tests run by nobody. None is a pass. Each runs in its own directory, outside the vendor directory this
# repository's own packages are built from; a nested module vendors what the forge holds (its own `vendor/`, so the
# check runs offline), as this repository's root module does (issue 1019). A failing module fails the check; the others are still run.
nested_modules() {
nested=0
for gomod in modules/*/go.mod; do
[ -e "$gomod" ] || continue
dir=$(dirname "$gomod")
echo "nested module $dir:"
if ! (cd "$dir" && GOFLAGS= CGO_ENABLED=0 go vet ./... &&
GOFLAGS= CGO_ENABLED="${cgo:-0}" go test ${race:-} -count=1 -timeout 30m ./...); then
echo "nested module $dir FAILED"
nested=1
fi
done
return $nested
}
nested_modules
-22
View File
@@ -1,22 +0,0 @@
ARG ALPINE_BASE=alpine:3
ARG GO_BASE=golang:1.25-alpine
# The builder, as a module ships one.
#
# Not FROM scratch, unlike the control plane: this one runs git and a container client, so it
# needs a filesystem with them on it. That is the honest cost of a machine whose job is to build —
# and it is why building is a MODULE on a machine that has a runtime rather than something the
# control plane does (novox/hq ADR 0005).
FROM ${GO_BASE} AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -buildvcs=false -ldflags '-s -w' -o /mesh-builder ./cmd/mesh-builder
FROM ${ALPINE_BASE}
# git to clone what it is asked to build, and the docker client to build and push it. The daemon
# is the machine's, reached through its socket — a build machine shares the runtime it was given
# rather than running one inside itself.
RUN apk add --no-cache git docker-cli
COPY --from=build /mesh-builder /usr/local/bin/mesh-builder
ENTRYPOINT ["/usr/local/bin/mesh-builder"]
-146
View File
@@ -1,146 +0,0 @@
package main
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"time"
)
// **The build agent says whether it is busy, and being asked drains it** (novox/hq ADR 0335 §1, §5). Its
// container declares `busy` as `mesh-builder busy`: the node-engine runs it inside the container before it
// replaces it. It exits 0 while a build is in hand and 1 when none is; and from the first ask the agent takes
// no new build request, so its answer ends once the build in hand ends — a replaced builder finishes the check
// it runs, instead of the build queue handing it out again from the start (issue 594).
//
// Two marks in the container's own temporary directory, as the ready mark (ready.go): a new container starts
// without them, and a restarted one removes them before it takes work again.
//
// - in hand: written by the agent when it claims a request, before it reads whether it is drained; removed
// when the build ends.
// - drained: written by the ask, before it reads whether a build is in hand.
//
// Each side writes its own mark before it reads the other's, so an ask racing a fetch sees the build in hand,
// or the agent sees the drain and hands the request back; never neither.
func inHandMark() string { return filepath.Join(os.TempDir(), "mesh-builder.in-hand") }
func drainMark() string { return filepath.Join(os.TempDir(), "mesh-builder.drained") }
// The drain lapses when no ask has come for drainLapse: the node-engine asks every 20 seconds for as long as it
// really waits, so an agent whose replacement was withdrawn — the declaration put back, the walk stopped —
// takes work again rather than staying drained for ever (review of ADR 0335). And an agent drained for longer
// than the node-engine's bound and the slack says it is not ready: the replacement it was drained for never
// came, which is a fault to see.
const (
drainLapse = 2 * time.Minute
drainTooLong = 15*time.Minute + 5*time.Minute
)
// drainedAt is when the drain began and when it was last asked, from its mark; false when there is no mark.
func drainedAt() (began, asked time.Time, ok bool) {
info, err := os.Stat(drainMark())
if err != nil {
return time.Time{}, time.Time{}, false
}
asked = info.ModTime()
began = asked
if raw, err := os.ReadFile(drainMark()); err == nil {
if t, err := time.Parse(time.RFC3339, strings.TrimSpace(string(raw))); err == nil && !t.After(asked) {
began = t
}
}
return began, asked, true
}
// drained says the agent was asked whether it is busy within drainLapse: it takes no new build request.
func drained() bool { return drainedSince(time.Now()) }
func drainedSince(now time.Time) bool {
_, asked, ok := drainedAt()
return ok && now.Sub(asked) < drainLapse
}
// drainedTooLong is the health check's second half: nil unless the agent has been drained, asked all along, for
// longer than its node-engine's bound and the slack.
func drainedTooLong(now time.Time) error {
began, asked, ok := drainedAt()
if !ok || now.Sub(asked) >= drainLapse || now.Sub(began) <= drainTooLong {
return nil
}
return fmt.Errorf("drained since %s and still asked: the replacement its node-engine waits for has not come "+
"within %s (novox/hq ADR 0335)", began.UTC().Format(time.RFC3339), drainTooLong)
}
// claim marks a request in hand, unless the agent is drained: then nothing is in hand, and the request goes
// back to the queue.
func claim() bool {
// Made, never written over: one build at a time, and a mark already there is a build in hand.
f, err := os.OpenFile(inHandMark(), os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644)
if err != nil {
// Unmarked, an ask would say idle while this build runs: the build is not taken.
fmt.Fprintf(os.Stderr, "cannot mark a build in hand (%s), so it is handed back: %v\n", inHandMark(), err)
return false
}
_, err = f.WriteString(time.Now().UTC().Format(time.RFC3339) + "\n")
if closeErr := f.Close(); err == nil {
err = closeErr
}
if err != nil {
_ = os.Remove(inHandMark())
fmt.Fprintf(os.Stderr, "cannot mark a build in hand (%s), so it is handed back: %v\n", inHandMark(), err)
return false
}
if drained() {
_ = os.Remove(inHandMark())
fmt.Fprintln(os.Stderr, "asked whether busy, so this agent takes no new build: handed back")
return false
}
return true
}
// holding says what is in hand, for the ask to print: written once the request is read.
func holding(what string) {
_ = os.WriteFile(inHandMark(), []byte(time.Now().UTC().Format(time.RFC3339)+" "+what+"\n"), 0o644)
}
// released ends the build in hand.
func released() {
if err := os.Remove(inHandMark()); err != nil && !errors.Is(err, os.ErrNotExist) {
fmt.Fprintf(os.Stderr, "cannot remove the mark of the build in hand (%s): %v\n", inHandMark(), err)
}
}
// undrained clears both marks at the start of a holder: a container started again is asked afresh.
func undrained() error {
for _, mark := range []string{drainMark(), inHandMark()} {
if err := os.Remove(mark); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove %s: %w", mark, err)
}
}
return nil
}
// busy is the ask: it drains the agent, then says whether a build is in hand. Nil — exit 0 — while one is.
func busy() (string, error) {
// The mark keeps when the drain began while it is asked within drainLapse, and is written again on every
// ask: its time is the last ask.
began := time.Now().UTC()
if was, _, ok := drainedAt(); ok && drained() {
began = was.UTC()
}
if err := os.WriteFile(drainMark(), []byte(began.Format(time.RFC3339)+"\n"), 0o644); err != nil {
// Not drained, it could take the next build: said, and answered busy only for what is in hand now.
fmt.Fprintf(os.Stderr, "cannot mark this agent drained (%s): %v\n", drainMark(), err)
}
held, err := os.ReadFile(inHandMark())
if err != nil {
return "", errors.New("idle: no build in hand, and this agent takes no new one")
}
since, what, _ := strings.Cut(strings.TrimSpace(string(held)), " ")
if what == "" {
what = "a build"
}
return fmt.Sprintf("busy: %s, in hand since %s; taking no new build", what, since), nil
}
-162
View File
@@ -1,162 +0,0 @@
package main
import (
"os"
"strings"
"sync"
"testing"
"time"
)
// `mesh-builder busy` answers 0 while a build is in hand and non-zero when idle, and after its first ask the
// agent takes no new build request (novox/hq ADR 0335 §5).
func TestBusyAnswersWhileABuildIsInHandAndDrainsTheAgent(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
if drained() {
t.Fatal("an agent never asked is drained")
}
// A build is claimed and runs.
if !claim() {
t.Fatal("an agent never asked refused a build")
}
holding("novox/mesh-controller (check-280)")
said, err := busy()
if err != nil || !strings.Contains(said, "novox/mesh-controller (check-280)") {
t.Fatalf("an agent with a build in hand did not answer busy with what it holds: %q %v", said, err)
}
// From that ask on it takes nothing new: the fetch is not made, and one fetched is handed back.
if !drained() {
t.Fatal("the ask did not drain the agent")
}
if claim() {
t.Fatal("a drained agent took a new build")
}
// The build in hand is still in hand after the refused claim, and busy goes on saying so.
if _, err := busy(); err != nil {
t.Fatalf("a refused claim ended the build in hand: %v", err)
}
// It ends: idle, and still drained.
released()
if _, err := busy(); err == nil {
t.Fatal("an agent with nothing in hand answered busy")
}
if claim() {
t.Fatal("a drained agent took a build after its last one ended")
}
// Started again, it is asked afresh.
if err := undrained(); err != nil {
t.Fatal(err)
}
if drained() || !claim() {
t.Fatal("a holder started again is still drained")
}
released()
}
// An ask racing a fetch sees the build in hand, or the agent sees the drain and hands the request back: never
// neither, which would replace the container under a build just taken.
func TestAnAskRacingAFetchNeverMissesTheBuild(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
for i := 0; i < 200; i++ {
if err := undrained(); err != nil {
t.Fatal(err)
}
var claimed bool
var askErr error
var wg sync.WaitGroup
wg.Add(2)
go func() { defer wg.Done(); claimed = claim() }()
go func() { defer wg.Done(); _, askErr = busy() }()
wg.Wait()
if claimed && askErr != nil {
t.Fatalf("round %d: the agent took a build and the ask said idle", i)
}
released()
}
}
// A drain no longer asked lapses: an agent whose replacement was withdrawn after the first ask (a put-back, a
// revert, a stopped walk) takes work again after drainLapse, rather than staying drained with its health green.
func TestADrainNoLongerAskedLapses(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
if !drained() || claim() {
t.Fatal("an agent just asked takes work")
}
// The last ask was three minutes ago.
old := time.Now().Add(-3 * time.Minute)
if err := os.Chtimes(drainMark(), old, old); err != nil {
t.Fatal(err)
}
if drained() {
t.Fatal("a drain no longer asked for three minutes still holds")
}
if !claim() {
t.Fatal("an agent whose drain lapsed takes no work")
}
released()
}
// An agent drained, and asked all along, for longer than its node-engine's bound and the slack is not ready: the
// replacement it was drained for never came. One asked again after a lapse starts a new drain.
func TestAnAgentDrainedTooLongIsNotReady(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
taking("mesh-build-machine")
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
now := time.Now()
if err := drainedTooLong(now); err != nil {
t.Fatalf("an agent drained a moment ago is not ready: %v", err)
}
// The drain began 25 minutes ago, and the node-engine asked a moment ago.
began := now.Add(-25 * time.Minute).UTC().Format(time.RFC3339)
if err := os.WriteFile(drainMark(), []byte(began+"\n"), 0o644); err != nil {
t.Fatal(err)
}
err := drainedTooLong(time.Now())
if err == nil || !strings.Contains(err.Error(), "has not come") {
t.Fatalf("an agent drained for 25 minutes reads ready: %v", err)
}
// Each ask keeps when the drain began.
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
if err := drainedTooLong(time.Now()); err == nil {
t.Fatal("an ask started the drain again")
}
// The asks stopped: the drain lapsed, and so did the fault.
old := time.Now().Add(-3 * time.Minute)
if err := os.Chtimes(drainMark(), old, old); err != nil {
t.Fatal(err)
}
if err := drainedTooLong(time.Now()); err != nil {
t.Fatalf("a lapsed drain still reads not ready: %v", err)
}
}
// An ask this build does not know exits non-zero: an older image asked `busy` reads idle, not busy for the
// node-engine's whole bound.
func TestAnUnknownCommandFails(t *testing.T) {
was := os.Args
t.Cleanup(func() { os.Args = was })
os.Args = []string{"mesh-builder", "some-later-word"}
if err := run(); err == nil {
t.Fatal("an unknown command exited 0")
}
os.Args = []string{"mesh-builder", "help"}
if err := run(); err != nil {
t.Fatalf("help failed: %v", err)
}
}
-38
View File
@@ -1,38 +0,0 @@
package main
import (
"testing"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// The build seat hands the check the judge cache the build agent declares (MESH_JUDGE_CACHE), never a path of its own.
func TestTheBuildSeatHandsTheCheckTheJudgeCacheItDeclares(t *testing.T) {
t.Setenv("MESH_JUDGE_CACHE", "/var/lib/mesh/build-agent/judge-cache")
if got := judgeCache(); got != "/var/lib/mesh/build-agent/judge-cache" {
t.Fatalf("the judge cache the manifest declares is read as %q", got)
}
t.Setenv("MESH_JUDGE_CACHE", "")
if got := judgeCache(); got != "" {
t.Fatalf("with no judge cache declared, %q is used", got)
}
}
// The siblings a check is asked to read reach the build seat's spec and, as read, the outcome (novox/hq ADR 0382).
func TestTheSiblingsAnAskNamesReachTheSpecAndTheOutcome(t *testing.T) {
spec := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main",
Siblings: []link.CheckSibling{{Repository: "novox/mesh-controller", Number: 346, Head: "a", Base: "main", Dir: "mesh-controller"}},
SiblingsUnknown: "why"}})
if len(spec.Siblings) != 1 || spec.Siblings[0] != (builder.Sibling{Repository: "novox/mesh-controller", Number: 346,
Head: "a", Base: "main", Dir: "mesh-controller"}) || spec.SiblingsUnknown != "why" {
t.Fatalf("the spec reads %+v", spec)
}
out := siblingsOut([]builder.JudgedSibling{{Repository: "novox/mesh-controller", Number: 346, Head: "a", Base: "main", BaseCommit: "b"}})
if len(out) != 1 || out[0].BaseCommit != "b" || out[0].Number != 346 {
t.Fatalf("the outcome reads %+v", out)
}
if plain := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "o", Repo: "r"}}); plain.Siblings != nil {
t.Fatalf("a request that names none has siblings %+v", plain.Siblings)
}
}
-433
View File
@@ -1,433 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go/micro"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
// What a holder of the build seat answers for, on its own machine (novox/hq ADR 0219).
//
// **The queue is the controller's; the build running here is this machine's.** The controller can
// see and change what waits in the seat's queue, but an ask a holder already took is a process tree
// on this machine and containers in this machine's runtime, and only this machine can end them. So
// the holder serves four verbs on the seat's subjects for this machine: what it is building, kill
// it, pause, resume.
//
// **One build at a time** (ADR 0190), which is also what builder.Said assumes — a package global
// set per build — so "the build running here" is one or none, and kill names it by id so a call
// that arrives as one build ends and the next begins cannot end the wrong one.
// holder is this machine's state as a holder of the build seat.
type holder struct {
on, seat string
// workspace is where the paused flag is kept, so a holder restarted while paused stays paused
// rather than silently taking work again.
workspace string
// say publishes this machine's state: whether it takes work (link.HolderState).
say func(link.HolderState) error
// remove runs what a kill needs outside the build: the containers left behind.
remove builder.Runner
// scratch is where check builds are kept (novox/hq ADR 0350 §5): what the week's sweep lets go of. Nil where
// this machine publishes to no artifact store.
scratch builder.Scratch
mu sync.Mutex
paused bool
running *running
}
// running is the build this machine is doing.
type running struct {
request link.BuildRequest
step string
started time.Time
cancel context.CancelFunc
killed bool
// returned is the build's own work having ended, before a kill or not; outcome is what was then
// announced, and announced whether it went out.
returned bool
outcome string
announced bool
done chan struct{}
}
// pausedFile is where the flag lives in the workspace.
func pausedFile(workspace string) string { return filepath.Join(workspace, ".mesh-builder-paused") }
// newHolder reads the paused flag the workspace keeps.
func newHolder(on, seat, workspace string, say func(link.HolderState) error) *holder {
h := &holder{on: on, seat: seat, workspace: workspace, say: say, remove: plainRun}
if _, err := os.Stat(pausedFile(workspace)); err == nil {
h.paused = true
}
return h
}
// Paused is asked by the taking loop before every fetch.
func (h *holder) Paused() bool {
h.mu.Lock()
defer h.mu.Unlock()
return h.paused
}
// setPaused records the flag in the workspace first and then in memory, so what this holder says
// it is and what it would be after a restart never differ.
func (h *holder) setPaused(paused bool) error {
path := pausedFile(h.workspace)
if paused {
if err := os.MkdirAll(h.workspace, 0o755); err != nil {
return err
}
if err := os.WriteFile(path, []byte(time.Now().UTC().Format(time.RFC3339)+"\n"), 0o644); err != nil {
return fmt.Errorf("cannot keep the paused flag in %s: %w", path, err)
}
} else if err := os.Remove(path); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove the paused flag %s: %w", path, err)
}
h.mu.Lock()
h.paused = paused
h.mu.Unlock()
h.announce()
return nil
}
// announce says whether this machine takes work. Never fatal: the flag is kept either way, and the
// controller reading an older state is a plan read as late rather than a build lost.
func (h *holder) announce() {
if h.say == nil {
return
}
if err := h.say(link.HolderState{On: h.on, Paused: h.Paused(), At: time.Now().UTC().Format(time.RFC3339Nano)}); err != nil {
fmt.Fprintf(os.Stderr, "cannot say whether this machine takes builds: %v\n", err)
}
}
// stateWhilePaused says this machine's state at start, and again every while it stays paused, so
// a pause outlives the events stream's retention.
func (h *holder) stateWhilePaused(ctx context.Context, every time.Duration) {
h.announce()
tick := time.NewTicker(every)
defer tick.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
if h.Paused() {
h.announce()
}
}
}
}
// expireChecks lets go of every check build older than a week (novox/hq ADR 0350 §5), at start and every while
// after. Every holder sweeps the one store; what one let go of, another finds gone.
func (h *holder) expireChecks(ctx context.Context, every time.Duration, now func() time.Time) {
sweep := func() {
if h.scratch == nil {
return
}
n, err := h.scratch.Expire(ctx, builder.CheckBuildsKept, now())
switch {
case err != nil:
fmt.Fprintf(os.Stderr, "check builds older than %s: %d let go of, and not all: %v\n", builder.CheckBuildsKept, n, err)
case n > 0:
fmt.Fprintf(os.Stderr, "check builds older than %s: %d check(s)' builds let go of\n", builder.CheckBuildsKept, n)
}
}
sweep()
tick := time.NewTicker(every)
defer tick.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
sweep()
}
}
}
// begin records the build this machine took, with the cancel that ends it.
func (h *holder) begin(request link.BuildRequest, cancel context.CancelFunc) *running {
r := &running{request: request, started: time.Now(), cancel: cancel, done: make(chan struct{})}
h.mu.Lock()
h.running = r
h.mu.Unlock()
return r
}
// end clears it, and lets a kill waiting on it know it is over.
func (h *holder) end(r *running) {
h.mu.Lock()
if h.running == r {
h.running = nil
}
h.mu.Unlock()
close(r.done)
}
// stepped records the step a build is at, for `current`.
func (h *holder) stepped(r *running, step string) {
h.mu.Lock()
r.step = step
h.mu.Unlock()
}
// currentBuild is what `current` answers.
type currentBuild struct {
On string `json:"on"`
Paused bool `json:"paused"`
Running *struct {
ID string `json:"id"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
Step string `json:"step,omitempty"`
Started string `json:"started"`
Elapsed string `json:"elapsed"`
} `json:"running,omitempty"`
Said string `json:"said"`
}
func (h *holder) current() currentBuild {
h.mu.Lock()
defer h.mu.Unlock()
out := currentBuild{On: h.on, Paused: h.paused}
taking := "taking builds"
if h.paused {
taking = "paused, taking no new build"
}
if h.running == nil {
out.Said = fmt.Sprintf("%s is building nothing; %s", h.on, taking)
return out
}
r := h.running
elapsed := time.Since(r.started).Round(time.Second)
out.Running = &struct {
ID string `json:"id"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
Step string `json:"step,omitempty"`
Started string `json:"started"`
Elapsed string `json:"elapsed"`
}{r.request.ID, r.request.Repository, r.request.Path, r.request.Ref, r.step,
r.started.UTC().Format(time.RFC3339), elapsed.String()}
out.Said = fmt.Sprintf("%s is building %s (%s) at %s for %s; %s",
h.on, r.request.Repository, r.request.ID, orNothing(r.step), elapsed, taking)
return out
}
// The bounds a kill keeps: each pass removing containers, and the wait for the build to end between
// them. The worst case — 15s, 20s, 15s — is inside what the controller waits for the answer
// (killAnswer in the controller's queue.go, 75s).
var (
killRemoves = 15 * time.Second
killWaits = 20 * time.Second
)
// kill ends the build with this id, if it is the one running here and still working: its context
// cancelled — which kills each command's process group — and the containers it started removed by
// their label, once at once and again after the build has ended, so one created while it was being
// killed is not left. The build's own goroutine announces it failed, killed by hand, and settles the
// ask so it is not redelivered; the answer says whether that happened.
func (h *holder) kill(id string) (string, error) {
h.mu.Lock()
r := h.running
if r == nil || r.request.ID != id {
doing := "nothing"
if r != nil {
doing = r.request.ID
}
h.mu.Unlock()
return "", fmt.Errorf("%s is not building %s; it is building %s. `queue` says where an ask is", h.on, id, doing)
}
if r.returned {
h.mu.Unlock()
return "", fmt.Errorf("%s on %s has already ended on its own and is saying how; `builds` shows it", id, h.on)
}
r.killed = true
cancel, done := r.cancel, r.done
h.mu.Unlock()
cancel()
removed, removeErr := h.removeContainers(id)
ended := false
select {
case <-done:
ended = true
case <-time.After(killWaits):
}
again, againErr := h.removeContainers(id)
removed += again
if removeErr == nil {
removeErr = againErr
}
containers := fmt.Sprintf("%d container(s) it started removed", removed)
if removeErr != nil {
containers = "its containers could not all be listed or removed: " + removeErr.Error()
}
if !ended {
return fmt.Sprintf("killed %s on %s: %s; the build has not finished ending yet — `builds` says when "+
"its outcome is in", id, h.on, containers), nil
}
h.mu.Lock()
outcome, announced := r.outcome, r.announced
h.mu.Unlock()
if !announced {
return fmt.Sprintf("killed %s (%s) on %s: its commands ended and %s, and its outcome could not be "+
"announced — the ask is not settled and will be handed out again", id, r.request.Repository, h.on,
containers), nil
}
return fmt.Sprintf("killed %s (%s) on %s: its commands ended, %s, and its outcome announced as failed, %s — "+
"settled, so it is not handed to another machine", id, r.request.Repository, h.on, containers, outcome), nil
}
// removeContainers is one pass of removing what the build left, bounded.
func (h *holder) removeContainers(id string) (int, error) {
cleanup, stop := context.WithTimeout(context.Background(), killRemoves)
defer stop()
return builder.RemoveContainersOf(cleanup, h.remove, id)
}
// removeLeftBehind removes, once, what earlier holders on this machine left — called before anything is
// taken, so none of it is this holder's — and says what it did. A runtime that cannot be asked is said
// and the holder takes work all the same: what is left is waste, not a reason to stop building.
func (h *holder) removeLeftBehind() int {
cleanup, stop := context.WithTimeout(context.Background(), killRemoves)
defer stop()
n, err := builder.RemoveLeftBehind(cleanup, h.remove)
switch {
case err != nil:
fmt.Fprintf(os.Stderr, "could not look for containers earlier builds left on %s: %v\n", h.on, err)
case n > 0:
fmt.Fprintf(os.Stderr, "removed %d container(s) earlier builds left on %s\n", n, h.on)
}
return n
}
// returned records that the build's work ended, and says whether a kill came first — only then is
// the build killed; an error it ended with on its own is its own outcome.
func (h *holder) returned(r *running) bool {
h.mu.Lock()
defer h.mu.Unlock()
r.returned = true
return r.killed
}
// said records the outcome announced, and whether it went out, for a kill to answer with.
func (h *holder) said(r *running, outcome string, announced bool) {
h.mu.Lock()
r.outcome, r.announced = outcome, announced
h.mu.Unlock()
}
// handlers are the seat's verbs, as this machine answers them.
func (h *holder) handlers() map[string]link.ToolHandler {
return map[string]link.ToolHandler{
"current": func(context.Context, json.RawMessage) (any, error) { return h.current(), nil },
"kill": func(_ context.Context, raw json.RawMessage) (any, error) {
var args struct {
ID string `json:"id"`
}
if err := json.Unmarshal(raw, &args); err != nil || strings.TrimSpace(args.ID) == "" {
return nil, errors.New("kill needs the build's id")
}
said, err := h.kill(strings.TrimSpace(args.ID))
if err != nil {
return nil, err
}
return map[string]any{"said": said}, nil
},
"pause": func(context.Context, json.RawMessage) (any, error) {
if err := h.setPaused(true); err != nil {
return nil, err
}
said := h.on + " is paused: it takes no new build until resumed"
if c := h.current(); c.Running != nil {
said += "; " + c.Running.ID + " runs on and finishes"
}
return map[string]any{"said": said, "paused": true}, nil
},
"resume": func(context.Context, json.RawMessage) (any, error) {
if err := h.setPaused(false); err != nil {
return nil, err
}
return map[string]any{"said": h.on + " takes builds again", "paused": false}, nil
},
}
}
func orNothing(s string) string {
if s == "" {
return "its start"
}
return s
}
// plainRun runs a command outside any build: no line reaches a build's log, because the build whose
// log it would be is the one being ended.
func plainRun(ctx context.Context, dir, name string, args ...string) (string, error) {
cmd := exec.CommandContext(ctx, name, args...)
cmd.Dir = dir
out, err := cmd.CombinedOutput()
if err != nil {
return string(out), fmt.Errorf("%s %s: %w: %s", name, strings.Join(args, " "), err, strings.TrimSpace(string(out)))
}
return string(out), nil
}
// announcement is what this holder answers discovery with (novox/hq ADR 0195, ADR 0197): this
// machine's verbs of the seat, in the shape every tool runtime announces a seat's verb — kind seat,
// the module answering, the seat, scope node, the machine, its description and argument schema — so
// the console finds `<node>/node-build-agent.kill` by searching, as it finds any seat's verb.
//
// One service per machine, named for the seat and identified by the machine, so the answer is this
// machine's four verbs and nothing more. The verbs are the compiled seat row's: a build machine has no
// store, and what it serves is what this binary was built to serve.
func announcement(seat, module, node string) micro.Info {
s, _ := catalogue.SeatNamed(seat)
var endpoints []micro.EndpointInfo
for _, v := range s.Serves {
schema, _ := json.Marshal(v.Input)
endpoints = append(endpoints, micro.EndpointInfo{
Name: seat + "__" + v.Name,
Subject: link.NodeSeatToolSubject(seat, v.Name, node),
// The queue group the verbs are served in, as every runtime announces its own.
QueueGroup: "seat." + seat,
Metadata: map[string]string{
"kind": "seat", "module": module, "tool": v.Name, "seat": seat, "scope": "node",
"node": node, "interchangeable": "false", "description": v.Description, "schema": string(schema),
},
})
}
return micro.Info{
ServiceIdentity: micro.ServiceIdentity{Name: seat, ID: node, Version: "0.1.0",
Metadata: map[string]string{"seat": seat, "scope": "node", "node": node, "module": module}},
Description: "what the build running on " + node + " is, and ending, pausing and resuming it (novox/hq ADR 0219)",
Endpoints: endpoints,
}
}
// moduleOf is the module a credential was issued for: its user is `<node>.<module>`.
func moduleOf(user string) string {
if _, module, ok := strings.Cut(user, "."); ok && module != "" {
return module
}
return "build-agent"
}
-172
View File
@@ -1,172 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// A holder paused stays paused across its restart: the flag is kept in its workspace (novox/hq ADR
// 0219), and what it says about itself follows.
func TestAPausedHolderStaysPausedAcrossARestart(t *testing.T) {
workspace := t.TempDir()
var said []link.HolderState
h := newHolder("ace", link.TheBuildMachine, workspace, func(s link.HolderState) error {
said = append(said, s)
return nil
})
if h.Paused() {
t.Fatal("a new holder starts paused")
}
if _, err := h.handlers()["pause"](context.Background(), json.RawMessage(`{}`)); err != nil {
t.Fatal(err)
}
if !h.Paused() || len(said) != 1 || !said[0].Paused || said[0].On != "ace" {
t.Fatalf("paused: %v, said %+v", h.Paused(), said)
}
again := newHolder("ace", link.TheBuildMachine, workspace, nil)
if !again.Paused() {
t.Fatal("restarted, the holder forgot it was paused")
}
if _, err := again.handlers()["resume"](context.Background(), json.RawMessage(`{}`)); err != nil {
t.Fatal(err)
}
if newHolder("ace", link.TheBuildMachine, workspace, nil).Paused() {
t.Fatal("resumed, the holder came back paused")
}
}
// kill ends the build with that id — its context, which ends its commands — removes what it left by
// label, and refuses an id it is not building.
func TestKillEndsTheBuildRunningHereAndNoOther(t *testing.T) {
h := newHolder("ace", link.TheBuildMachine, t.TempDir(), nil)
var removed []string
h.remove = func(_ context.Context, _ string, name string, args ...string) (string, error) {
removed = append(removed, name+" "+strings.Join(args, " "))
if args[0] == "ps" {
return "c1\n", nil
}
return "", nil
}
kill := h.handlers()["kill"]
if _, err := kill(context.Background(), json.RawMessage(`{"id":"build-1"}`)); err == nil {
t.Fatal("killed a build while none ran")
}
building, cancel := context.WithCancel(context.Background())
r := h.begin(link.BuildRequest{ID: "build-1", Repository: "novox/a"}, cancel)
h.stepped(r, "image")
if c := h.current(); c.Running == nil || c.Running.ID != "build-1" || c.Running.Step != "image" {
t.Fatalf("current says %+v", c)
}
if _, err := kill(context.Background(), json.RawMessage(`{"id":"build-2"}`)); err == nil ||
!strings.Contains(err.Error(), "build-1") {
t.Fatalf("killed another id: %v", err)
}
// The build's own goroutine: it ends when its context does, as a build's commands do, and
// announces what came of it.
go func() {
<-building.Done()
if h.returned(r) {
h.said(r, link.KilledByHand, true)
}
h.end(r)
}()
answer, err := kill(context.Background(), json.RawMessage(`{"id":"build-1"}`))
if err != nil {
t.Fatal(err)
}
if building.Err() == nil {
t.Fatal("the build's context was not cancelled")
}
if !r.killed {
t.Error("the build is not marked killed, so it would be announced as an ordinary failure")
}
said := answer.(map[string]any)["said"].(string)
if !strings.Contains(said, "2 container(s)") || !strings.Contains(said, "announced as failed") || !strings.Contains(said, link.KilledByHand) {
t.Errorf("kill said %q", said)
}
// Removed at the kill and again once the build had ended: a container made in between is caught.
if len(removed) != 4 || !strings.Contains(removed[0], "label=mesh.build=build-1") || !strings.Contains(removed[2], "label=mesh.build=build-1") {
t.Errorf("removed %v", removed)
}
if c := h.current(); c.Running != nil {
t.Errorf("after the kill current says %+v", c)
}
}
// A holder announces this machine's verbs of the seat as the console reads a seat's verb, and no more.
func TestAHolderAnnouncesItsMachinesVerbsForTheConsole(t *testing.T) {
info := announcement(link.TheBuildMachine, moduleOf("ace.build-agent"), "ace")
if info.Name != "node-build-agent" || info.ID != "ace" || len(info.Endpoints) != 4 {
t.Fatalf("announced %s/%s with %d endpoints", info.Name, info.ID, len(info.Endpoints))
}
kill := info.Endpoints[1]
md := kill.Metadata
if kill.Subject != "mesh.seat.node-build-agent.tool.kill.ace" || kill.QueueGroup != "seat.node-build-agent" || md["kind"] != "seat" || md["seat"] != "node-build-agent" ||
md["scope"] != "node" || md["node"] != "ace" || md["tool"] != "kill" || md["module"] != "build-agent" ||
!strings.Contains(md["description"], "killed by hand") || !strings.Contains(md["schema"], `"id"`) {
t.Fatalf("kill is announced as %+v", kill)
}
body, err := json.Marshal(info)
if err != nil || len(body) > 8*1024 {
t.Fatalf("the answer is %d bytes (%v)", len(body), err)
}
}
// A build that ended on its own as the kill arrived says what it did: the kill is refused, and its
// own error is its outcome. One whose outcome could not be announced is not said to be settled.
func TestAKillArrivingAfterTheBuildEndedIsRefusedAndAnUnannouncedKillSaysSo(t *testing.T) {
h := newHolder("ace", link.TheBuildMachine, t.TempDir(), nil)
h.remove = func(context.Context, string, string, ...string) (string, error) { return "", nil }
_, cancel := context.WithCancel(context.Background())
r := h.begin(link.BuildRequest{ID: "build-1"}, cancel)
if killed := h.returned(r); killed {
t.Fatal("a build nobody killed reads as killed")
}
if _, err := h.kill("build-1"); err == nil || !strings.Contains(err.Error(), "ended on its own") {
t.Fatalf("killed a build that had ended: %v", err)
}
h.end(r)
building, cancel := context.WithCancel(context.Background())
r = h.begin(link.BuildRequest{ID: "build-2"}, cancel)
go func() {
<-building.Done()
if h.returned(r) {
h.said(r, link.KilledByHand, false)
}
h.end(r)
}()
said, err := h.kill("build-2")
if err != nil || strings.Contains(said, "announced as failed") || !strings.Contains(said, "could not be announced") {
t.Fatalf("kill said %q (%v)", said, err)
}
}
// **Issue 306**: a holder starting removes the containers earlier builds left on its machine, through
// the runner a kill uses, and a runtime that cannot be asked does not stop it.
func TestAHolderStartingRemovesWhatEarlierBuildsLeftHere(t *testing.T) {
h := newHolder("novox", link.TheBuildMachine, t.TempDir(), nil)
var removed []string
h.remove = func(_ context.Context, _ string, name string, args ...string) (string, error) {
removed = append(removed, name+" "+strings.Join(args, " "))
if args[0] == "ps" {
return "c1 build-1\nc2 build-1\nc3 check-here-2\n", nil
}
return "", nil
}
if n := h.removeLeftBehind(); n != 2 || len(removed) != 2 || removed[1] != "docker rm -f c1 c2" {
t.Fatalf("removed %d: %v", n, removed)
}
h.remove = func(context.Context, string, string, ...string) (string, error) {
return "", errors.New("no runtime")
}
if n := h.removeLeftBehind(); n != 0 {
t.Errorf("a runtime that cannot be asked removed %d", n)
}
}
-935
View File
@@ -1,935 +0,0 @@
// mesh-builder — the thing a build machine runs.
//
// It takes work from the mesh, turns a repository into artifacts, publishes them, and says what
// came out. It is **not** the control plane and it is **not** the host:
//
// - the control plane decides and never touches a machine. Building runs commands on one, and
// what the control plane may send a machine is bounded by the declaration language
// (novox/hq ADR 0005). "Run this build" is not in it, and widening the language so it could
// be would make the control plane able to run anything anywhere.
// - the host applies declarations and holds no opinion about what they contain. A host that
// also built things would need a container runtime and git, on every machine, to do something
// almost none of them will ever do.
//
// So it is a module: a program a machine runs because the mesh told it to, holding its own broker
// credential and nothing else. Compromise of a build machine is compromise of a build machine.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"net/url"
"os"
"os/signal"
"sort"
"strings"
"sync"
"sync/atomic"
"syscall"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// version is set at build time.
var version = "development"
func main() {
if err := run(); err != nil {
fmt.Fprintf(os.Stderr, "mesh-builder: %v\n", err)
os.Exit(1)
}
}
const usage = `mesh-builder — builds modules for the mesh
It consumes build requests and answers with what it made. Nothing is listened on and nothing
is dialled except the broker.
MESH_BROKER_FILE a file the mesh sealed to this machine holding the same
MESH_REGISTRY host:port to publish artifacts to, when the mesh has not said
MESH_BINDING a file the mesh wrote saying where the artifact store is
MESH_PACKAGE_BINDING a file the mesh wrote saying where the package registry is
MESH_NPM_TOKEN_FILE a file the mesh sealed holding the token for it
MESH_GENESIS_PACKAGE_REGISTRY the document the installer sealed at genesis, read while no
binding is present (novox/hq ADR 0336)
MESH_NPM_REGISTRY a package registry URL, when the mesh has not said (a person, the bootstrap)
MESH_NPM_TOKEN the token for it, likewise
MESH_NPM_SCOPE the scope it answers for (default: @novox)
MESH_WORKSPACE where to clone and build (default: a temporary directory)
It also builds one module and stops, which is how a mesh is raised — before there is a
broker to take work from or a registry to publish into:
mesh-builder build <repository> [--path P] [--ref COMMIT] [--registry HOST:PORT]
Without --registry the artifacts stay in this machine's container runtime, named by the
digest of their own configuration. The result is printed as JSON.
mesh-builder ready
exits 0 once the agent running in this container is bound to its build work and pulls it, and
1 before: its container's health check.
mesh-builder busy
exits 0 while the agent running in this container has a build in hand, and 1 when it has none;
from the first ask the agent takes no new build: its container's busy command, asked before the
container is replaced (novox/hq ADR 0335).
`
func run() error {
if len(os.Args) > 1 {
switch os.Args[1] {
case "version":
fmt.Println(version)
return nil
case "build":
return buildOnce(context.Background(), os.Args[2:])
case "ready":
// The container's health check (novox/hq issue 544): exits non-zero until the agent takes work, and
// once it has been drained longer than its node-engine would wait to replace it (ADR 0335).
if err := ready(); err != nil {
return err
}
return drainedTooLong(time.Now())
case "busy":
// The container's busy command (novox/hq ADR 0335): drains the agent, and exits non-zero once
// nothing is in hand.
said, err := busy()
if err != nil {
return err
}
fmt.Println(said)
return nil
case "help", "-h", "--help":
fmt.Print(usage)
return nil
default:
// **An ask this build does not know fails** (novox/hq ADR 0335): an image older than `busy`, asked
// it, must read as idle — exiting 0 would hold its container for the node-engine's whole bound.
fmt.Fprint(os.Stderr, usage)
return fmt.Errorf("%q is not a command this mesh-builder knows", os.Args[1])
}
}
if err := notReady(); err != nil {
return err
}
// **The builder's own version is part of every reuse key** (novox/hq ADR 0350 §1): a builder that builds
// differently never takes a build another one made.
builder.Version = builder.SelfVersion(version)
if err := undrained(); err != nil {
return err
}
credential, err := brokerFrom()
if err != nil {
return err
}
registry, err := whereToPublish()
if err != nil {
return err
}
workspace := os.Getenv("MESH_WORKSPACE")
if workspace == "" {
workspace = os.TempDir() + "/mesh-builder"
}
// **The mesh's name for this machine, not the container's.** A build is reported to the rest
// of the mesh, and a report whose origin reads `104cb10e105b` names something no other module
// can look up. The mesh already knows the answer and has a way to say it — `${machine:name}`
// in the environment file this module is handed — so the hostname is only what is left when
// nobody said.
on := os.Getenv("MESH_NODE")
if on == "" {
hostname, err := os.Hostname()
if err != nil {
return fmt.Errorf("this build machine has no name: nothing said MESH_NODE and the host would not say either: %w", err)
}
on = hostname
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
js, seat, err := dialFor(credential)
if err != nil {
return err
}
defer js.Close()
// **This machine's holder: paused or not, and the build it is running** (novox/hq ADR 0219). The
// paused flag is read from the workspace before anything is taken, so a holder restarted while
// paused takes nothing.
h := newHolder(on, seat, workspace, func(state link.HolderState) error {
body, err := json.Marshal(state)
if err != nil {
return err
}
_, err = js.Context().Publish(link.BuildPausedOf(seat, on), body)
return err
})
if h.Paused() {
fmt.Fprintf(os.Stderr, "paused (kept in %s): taking no build until resumed\n", pausedFile(workspace))
}
// **What an earlier holder here left goes before anything is taken** (novox/hq issue 306): a holder
// recreated mid-check left the check's store and bus running, and the ask went to another machine.
h.removeLeftBehind()
// Paused or drained, it takes nothing new (novox/hq ADR 0219, ADR 0335): a drained agent finishes the build
// in hand and then says idle, and its container is replaced between builds.
machine := link.MachineOverNATSWith(js, on, seat, link.MachineOptions{
Paused: func() bool { return h.Paused() || drained() },
Taking: func() { taking(seat) },
Claim: claim,
})
defer machine.Close()
// The seat's verbs, on this machine's subjects. Only the seat that declares them: the retired
// one serves none, and a subscription its holder has no grant for would be refused for ever.
if seat == link.TheBuildMachine {
stopServing, err := link.OverNATS{Conn: js.Conn()}.ServeNodeSeatTools(seat, on, h.handlers(),
log.New(os.Stderr, "", 0))
if err != nil {
return err
}
defer stopServing()
// And says so, for the console to find (novox/hq ADR 0197).
stopAnnouncing, err := link.OverNATS{Conn: js.Conn()}.Announce(
announcement(seat, moduleOf(credential.User), on), log.New(os.Stderr, "", 0))
if err != nil {
return err
}
defer stopAnnouncing()
go h.stateWhilePaused(ctx, time.Hour)
}
fmt.Fprintf(os.Stderr, "building for the mesh, publishing to %s\n", registry)
publisher := builder.Registry{Address: registry, Run: builder.Command}
// Check builds are let go of after a week (novox/hq ADR 0350 §5).
h.scratch = publisher
go h.expireChecks(ctx, time.Hour, time.Now)
return machine.Take(ctx, func(ctx context.Context, work link.Build) {
defer released()
holding(work.Request().Repository + " (" + work.Request().ID + ")")
answer(ctx, publisher, on, workspace, work, h)
})
}
// dialFor opens this machine's link to whichever bus the mesh is on, and says which build seat it
// holds.
//
// **One place chooses**, as everywhere else the bus change went (novox/hq ADR 0116 step 5): a build
// machine told about both would take work from one and answer on the other, and every log line would
// say it was fine.
func dialFor(credential Credential) (*broker.JetStream, string, error) {
// **The credential names the bus, and there is one** (novox/hq ADR 0131, design 28 task 5.5).
// A credential for the mesh's bus carries user, password and fingerprint beside the address,
// and that is enough to dial it, pinned.
if !credential.onTheNewBus() {
return nil, "", fmt.Errorf("the credential at hand names %q, which is not the mesh's bus", credential.URL)
}
// Named for what it is, not the controller whose code dials it (novox/hq issue 327).
host, _ := os.Hostname()
js, err := broker.DialPinned(credential.natsURL(), credential.Fingerprint, nats.Name("build agent on "+host))
if err != nil {
return nil, "", err
}
// **The seat this machine serves is the one its credential claims** (novox/hq ADR 0190, the
// handover): the mesh issues a build machine's credential naming the seat its module claims,
// and one binary serves the old role as `builder` and the new as `build-agent` from that alone.
seat := link.BuildSeatClaimed(credential.seatsClaimed())
fmt.Fprintf(os.Stderr, "taking build work as a holder of %s\n", seat)
return js, seat, nil
}
// answer does one build and says what happened, whichever way it went.
func answer(ctx context.Context, publisher builder.Publisher, on, workspace string, work link.Build, h *holder) {
request := work.Request()
// **Its own context, so it can be killed alone** (novox/hq ADR 0219): cancelled by `kill`, it ends
// this build's commands and nothing else; the machine's own context ending — a SIGTERM — still
// reaches it through the parent, and that keeps today's meaning below.
building, cancel := context.WithCancel(ctx)
defer cancel()
var mine *running
if h != nil {
mine = h.begin(request, cancel)
defer h.end(mine)
}
// **First thing, and to stdout.** A build request that arrives and produces no visible line until
// it either finishes or fails is indistinguishable from one that never arrived — which cost a long
// diagnosis against a running mesh, chasing "the handler never fired" when the truth was only that
// the handler said nothing until the end.
fmt.Fprintf(os.Stderr, "a build request arrived for %s (%s)\n", request.Repository, request.ID)
// **Everything a build says goes two ways**: to stderr, as always, and onto the bus as the
// role's own events under the build's id (novox/hq ADR 0157) — so whoever asked, and anybody
// watching, reads the same lines this container's log holds, live, and after the fact from the
// stream. Said first, before anything runs, so a build that hangs is one that visibly started.
// One line at a time: a check's builds for its merge say their lines while its tests say theirs (novox/hq
// issue 716).
var saying sync.Mutex
say := func(step, message string) {
saying.Lock()
defer saying.Unlock()
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
work.Say(step, message)
if mine != nil && step != "run" && step != "output" {
h.stepped(mine, step)
}
}
builder.Said = say
defer func() { builder.Said = nil }()
if err := work.Began(ctx); err != nil {
fmt.Fprintf(os.Stderr, "cannot say a build started: %v\n", err)
}
// **A running check watches its pull request's key** (novox/hq ADR 0406 §4, issue 974): when the head or the trunk
// commit in it moves, the check is for a state nobody wants any more, and ends itself — nothing is asked of the forge
// (issue 726). A group's check, a try and a build watch nothing.
var superseded atomic.Bool
if w, ok := work.(interface {
WatchPull(context.Context, func(link.PullKey)) (func(), error)
}); ok && request.Check != nil {
stopWatching, err := w.WatchPull(building, func(k link.PullKey) {
superseded.Store(true)
say("superseded", fmt.Sprintf("the pull request's key moved to %.12s on %.12s while this checked %.12s on %.12s: stopped",
k.Head, k.Trunk, request.Ref, request.Check.Trunk))
cancel()
})
if err != nil {
fmt.Fprintf(os.Stderr, "the key of %s/%s#%d could not be watched, so its check runs on if the key moves: %v\n",
request.Check.Owner, request.Check.Repo, request.Check.Number, err)
} else {
defer stopWatching()
}
}
result := link.BuildResult{
ID: request.ID, Repository: request.Repository, Path: request.Path,
Ref: request.Ref, On: on, Source: request.Source, DryRun: request.DryRun,
}
what := "building " + request.Repository
if request.Path != "" {
what += " at " + request.Path
}
if request.Ref != "" {
what += " on " + request.Ref
}
say("build", what)
npmrc, err := packagesFrom(on)
if err == nil {
say("packages", saysRegistry(npmrc))
}
registryHere, isRegistry := publisher.(builder.Registry)
var built builder.Result
if request.Check != nil {
// **A pull request's merge check, not a build** (novox/hq to-be 45 §9): nothing is built,
// published or registered; the verdict is the outcome.
result.Checked = request.Check
registry := ""
if r, ok := publisher.(builder.Registry); ok {
registry = r.Address
}
spec := checkSpecOf(request)
// The judge cache the build agent declares as its own data, never in the workspace: only judge builds and
// mesh-lab's replays mount it, never a check's containers, so no pull request's script can write what a judge
// is built with or kept as, or what the replays run (novox/hq ADR 0358 §6, issue 709).
spec.JudgeCache = judgeCache()
// What the merge would move, built by the check for the walk to take (novox/hq ADR 0350 §1): into the
// scratch namespace. **Never with the package registry's credential**: this is code nobody has approved
// yet, and a recipe could read the credential it is handed (issue 462). A module that resolves packages
// at build time has no reuse key anyway, so nothing is lost by refusing it here.
if isRegistry && len(spec.Builds) > 0 {
spec.BuildWith = &builder.CheckBuilding{Publish: publisher, Scratch: registryHere, Held: request.Held,
Running: request.Check.Running, Seats: request.Seats, Named: namedOf(request.Check.Named)}
}
var v builder.CheckVerdict
v, err = builder.Check(building, builder.Command, spec, workspace, registry, forgeFrom(), say)
if err == nil {
result.Check = &link.CheckOutcome{Verdict: v.Verdict, Summary: v.Summary, Report: v.Report,
Took: v.Took.Round(time.Second).String(), Gate: layerOf(v.Gate), RepoCheck: layerOf(v.Repo),
Head: v.Head, Trunk: v.Trunk, Beside: v.Beside, Built: checkBuiltOf(v.Built),
Validator: v.Validator, Siblings: siblingsOut(v.Siblings), SiblingsUnknown: v.SiblingsUnknown, JudgeFrom: v.JudgeFrom}
// What each of a group's heads was judged as (novox/hq ADR 0323 §6).
for _, m := range v.Members {
result.Check.Members = append(result.Check.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo,
Commit: m.Head, Trunk: m.Trunk})
}
// A branch tried before its pull request reads its outcome from this log, whole (novox/hq issue 686).
if request.Check.Try {
if line, err := json.Marshal(triedOf(v)); err == nil {
say(link.StepTried, string(line))
}
}
}
} else if err == nil {
// The package-registry credential is a build input, so it is resolved before the clone: a
// build that could not have resolved its dependencies is refused in front of the reason, not
// after a clone that then fails at npm ci.
// Every container it starts is labelled with its id, so a kill finds what outlived the
// docker client (ADR 0219).
if request.Reuse != nil && isRegistry && !request.DryRun {
// **A walk's build takes the check's build of the same key** (novox/hq ADR 0350 §2).
reuse := builder.Reuse{Scratch: registryHere}
for _, p := range request.Reuse.Pulls {
reuse.Pulls = append(reuse.Pulls, builder.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number})
}
// Only what a verdict names is taken (ADR 0350 §2).
reuse.Named = namedOf(request.Reuse.Builds)
built, err = builder.BuildReusing(building, builder.Labelled(builder.Command, request.ID), publisher, reuse,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats)
} else {
built, err = builder.Build(building, builder.Labelled(builder.Command, request.ID), publisher,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats)
}
}
// What it copied into the store, whatever became of the build (novox/hq ADR 0257).
result.Mirrored = built.Mirrored
// Only a build the kill ended: the kill came before its work did. One that finished — built, or
// failed on its own — in the moment the kill arrived says what it did, and the kill is refused.
killed := false
if mine != nil {
killed = h.returned(mine) && err != nil
}
if superseded.Load() && err != nil {
// Ended by its own key moving on (ADR 0406 §4): said as the check's failure, as a kill is.
result.Failed = link.SupersededByKey
say("failed", link.SupersededByKey)
} else if killed {
// **Killed by hand is the outcome, whatever the build was doing** (novox/hq ADR 0219): the
// error it ended with is the kill's consequence, not a fault of the source.
result.Failed = link.KilledByHand
say("failed", link.KilledByHand)
} else if err != nil {
// A failure is a result. A build that fails and says nothing is indistinguishable from a
// builder that is not running, and those want completely different responses.
result.Failed = err.Error()
say("failed", err.Error())
} else if request.Check != nil {
say("checked", result.Check.Verdict+": "+result.Check.Summary)
} else {
manifest, marshalErr := json.Marshal(built.Manifest)
if marshalErr != nil {
result.Failed = marshalErr.Error()
} else {
result.Commit = built.Commit
result.Manifest = manifest
for _, made := range built.Built {
result.Made = append(result.Made, link.MadeArtifact{
Name: made.Name, Kind: made.Kind, Reference: made.Reference,
})
}
result.Against = built.Against
result.SourceFingerprint = built.Source
// What the walk did instead of building, or why it built (novox/hq ADR 0350 §3, §4).
result.ReuseKey = built.ReuseKey
if built.Reused != nil {
f := built.Reused.From
result.Reused = built.Reused.Words()
result.ReusedFrom = &link.ReusedFrom{Pull: link.PullRef{Owner: f.Pull.Owner, Repo: f.Pull.Repo,
Number: f.Pull.Number}, Head: f.Head, Trunk: f.Trunk, Check: f.Check, Key: f.Key}
} else if request.Reuse != nil {
result.BuiltWhy = built.Why
if result.BuiltWhy == "" {
result.BuiltWhy = "it has no artifact to build"
}
}
result.Trunk, result.OnTrunk, result.Branches = built.Trunk, built.OnTrunk, built.Branches
for _, r := range built.Read {
result.Read = append(result.Read, link.ReadRepository{Repository: r.Repository, Ref: r.Ref})
}
for _, s := range built.Sources {
result.Sources = append(result.Sources, link.BuildSource{Repository: s.Repository, Ref: s.Ref, Paths: s.Paths})
}
// The list of predefined keys the manifest was judged against, for the controller to publish
// (novox/hq ADR 0348 §7).
if p := built.Predefined; p != nil {
result.Predefined = &link.PredefinedList{Keys: p.Keys, Generation: p.Generation}
}
say("built", built.Manifest.Module+" from "+short(built.Commit))
}
}
// A killed build is announced on a context of its own: the build's was the one cancelled, and the
// outcome must go out and the ask be settled — acknowledged, never redelivered to another machine
// to be built again. A machine being stopped is the other case and keeps its meaning: the
// parent's context is gone, nothing is announced or settled, and the ask is redelivered.
announcing := ctx
if killed {
fresh, stop := context.WithTimeout(context.Background(), 30*time.Second)
defer stop()
announcing = fresh
}
announceErr := work.Announce(announcing, result)
if mine != nil {
h.said(mine, result.Failed, announceErr == nil)
}
if err := announceErr; err != nil {
// Said, not fatal: the build happened. A build reported as failed because announcing it
// failed is a lie about work that was done — and the request stays unsettled below only if
// nothing was said at all, so another machine can try.
fmt.Fprintf(os.Stderr, "cannot say what came of a build: %v\n", err)
return
}
// Settled only once the outcome is away, so a machine that dies before answering leaves the work
// for another rather than losing it.
if err := work.Done(); err != nil {
fmt.Fprintf(os.Stderr, "the outcome is away and the request could not be settled: %v\n", err)
}
}
// checkSpecOf is a check request as the builder runs it.
func checkSpecOf(request link.BuildRequest) builder.CheckSpec {
c := request.Check
spec := builder.CheckSpec{ID: request.ID, Repository: request.Repository, Ref: request.Ref,
Owner: c.Owner, Repo: c.Repo, Number: c.Number, Paths: c.Paths, Beside: map[string]builder.Beside{},
Modules: c.Modules, New: c.New, Manifests: c.Manifests, Judge: c.Judge, Base: c.Base,
Toolchain: builder.ToolchainOf(request.Held), Toolchains: builder.ToolchainsOf(request.Held),
Tests: c.Tests, StoreOnly: c.StoreOnly, SiblingsUnknown: c.SiblingsUnknown}
for _, s := range c.Siblings {
spec.Siblings = append(spec.Siblings, builder.Sibling{Repository: s.Repository, Number: s.Number, Head: s.Head,
Base: s.Base, Dir: s.Dir})
}
for dir, b := range c.Beside {
spec.Beside[dir] = builder.Beside{Repository: b.Repository, Ref: b.Ref}
}
for _, b := range c.Builds {
spec.Builds = append(spec.Builds, builder.CheckBuildSpec{Module: b.Module, Repository: b.Repository, Path: b.Path,
Ref: b.Ref, Merged: b.Merged, Tier: b.Tier, Skip: b.Skip})
}
for _, m := range c.Members {
spec.Group = append(spec.Group, builder.GroupHead{Owner: m.Owner, Repo: m.Repo, Repository: m.Repository,
Ref: m.Ref, Paths: m.Paths, Base: m.Base})
}
return spec
}
// siblingsOut is the siblings a check read as the outcome carries them (novox/hq ADR 0382 §4).
func siblingsOut(read []builder.JudgedSibling) []link.CheckSibling {
var out []link.CheckSibling
for _, s := range read {
out = append(out, link.CheckSibling{Repository: s.Repository, Number: s.Number, Head: s.Head, Base: s.Base,
BaseCommit: s.BaseCommit})
}
return out
}
// judgeCache is the judge cache the build agent's manifest declares (MESH_JUDGE_CACHE, novox/hq ADR 0358 §6): a
// directory mounted at the same path on the host, so a judge build's mounts name what the agent sees. Empty when the
// agent declares none, and every judge is then built cold under its check's root.
func judgeCache() string { return os.Getenv("MESH_JUDGE_CACHE") }
// triedOf is a try's outcome as its log line says it (novox/hq issue 686): the repository's own check's verdict,
// or the check's where that did not run — a head that does not merge — and the lines naming what failed.
func triedOf(v builder.CheckVerdict) link.Tried {
t := link.Tried{Verdict: v.Verdict, Summary: v.Summary, Took: v.Took.Round(time.Second).String(),
Head: v.Head, Trunk: v.Trunk}
if v.Repo != nil && (v.Gate == nil || v.Gate.Verdict == "pass") {
t.Verdict, t.Summary = v.Repo.Verdict, v.Repo.Summary
}
if v.Repo != nil && v.Repo.Failed != "" {
t.Failed = strings.Split(v.Repo.Failed, "\n")
}
if t.Verdict == "" {
t.Verdict = "error"
}
return t
}
// namedOf are the builds the seat's verdicts name, as the builder believes a scratch record by them (novox/hq ADR
// 0350 §2, issue 703): each module's key with every artifact's digest.
func namedOf(builds []link.CheckedBuild) []builder.NamedBuild {
var out []builder.NamedBuild
for _, b := range builds {
named := builder.NamedBuild{Pull: builder.PullRef{Owner: b.Pull.Owner, Repo: b.Pull.Repo,
Number: b.Pull.Number}, Check: b.Check, Module: b.Built.Module, Key: b.Built.Key,
Digests: map[string]string{}}
for _, m := range b.Built.Made {
named.Digests[m.Name] = builder.DigestOf(m.Reference)
}
out = append(out, named)
}
return out
}
// checkBuiltOf is a check's builds for its merge as the outcome carries them (novox/hq ADR 0350 §1).
func checkBuiltOf(built []builder.CheckBuilt) []link.CheckBuilt {
var out []link.CheckBuilt
for _, b := range built {
c := link.CheckBuilt{Module: b.Module, Key: b.Key, NotBuilt: b.NotBuilt, Took: b.Took.Round(time.Second).String(),
Kept: b.Kept, Same: b.Same, SameAs: b.SameAs}
for _, m := range b.Made {
c.Made = append(c.Made, link.MadeArtifact{Name: m.Name, Kind: m.Kind, Reference: m.Reference})
}
out = append(out, c)
}
return out
}
// layerOf is one layer of a check as the outcome carries it.
func layerOf(l *builder.Layer) *link.CheckLayer {
if l == nil {
return nil
}
return &link.CheckLayer{Verdict: l.Verdict, Summary: l.Summary, Modules: l.Modules}
}
// packagesFrom is where a build resolves the mesh's own published packages — the SDK above all
// (novox/hq ADR 0076, issue 053) — on this machine.
//
// **The binding wins, then the genesis document** (novox/hq ADR 0336 §3, §4). The build agent's
// npm-package-registry requirement is optional: on genesis's node, before the gitea module is assigned,
// nothing provides it, and the installer sealed the forge it raised to this agent as one document
// instead. Once the requirement binds, the binding names the endpoint the way the artifact store's
// does and a sealed token file the credential, and the genesis document is left unread. A binding
// file named and absent is a requirement not bound here, never a fault: the environment names the
// path whether or not the mesh wrote it.
//
// The environment variables remain for a builder run by a person. With none of the three the result
// is disabled and says this machine, so a build that needs the registry is refused naming it (§6),
// and a build that needs none builds anyway.
func packagesFrom(machine string) (builder.Npmrc, error) {
scope := strings.TrimSpace(os.Getenv("MESH_NPM_SCOPE"))
if scope == "" {
scope = "@novox"
}
none := builder.Npmrc{Machine: machine}
if path := strings.TrimSpace(os.Getenv("MESH_PACKAGE_BINDING")); path != "" {
raw, err := os.ReadFile(path)
switch {
case errors.Is(err, os.ErrNotExist):
// Not bound on this machine: the genesis document, or nothing.
case err != nil:
return none, fmt.Errorf("cannot read what the mesh said about the package registry: %w", err)
default:
return fromBinding(path, raw, scope, machine)
}
}
if path := strings.TrimSpace(os.Getenv("MESH_GENESIS_PACKAGE_REGISTRY")); path != "" {
raw, err := os.ReadFile(path)
switch {
case errors.Is(err, os.ErrNotExist):
// Not this mesh's genesis node, or the installer has not sealed it yet.
case err != nil:
return none, fmt.Errorf("cannot read the genesis document for the package registry: %w", err)
default:
return fromGenesis(path, raw, scope, machine)
}
}
// A builder run by a person.
registry := strings.TrimSpace(os.Getenv("MESH_NPM_REGISTRY"))
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
username := strings.TrimSpace(os.Getenv("MESH_NPM_USER"))
if registry == "" && secret == "" {
return none, nil
}
n := builder.Npmrc{Scope: scope, Registry: registry, From: builder.FromEnvironment, Machine: machine}
if username != "" {
n.Username, n.Password = username, secret
} else {
n.Token = secret
}
return n, nil
}
// fromBinding composes the registry from the npm-package-registry binding the mesh wrote and the
// credential it sealed beside it. The mesh authenticates the ordinary way — a generated password the
// provider only applies (novox/hq ADR 0048) — so with a username the secret is a password (basic
// auth); without one it is a bearer token a provider minted.
func fromBinding(path string, raw []byte, scope, machine string) (builder.Npmrc, error) {
none := builder.Npmrc{Machine: machine}
var told struct {
From string `json:"from"`
At string `json:"at"`
As string `json:"as"`
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal(raw, &told); err != nil {
return none, fmt.Errorf("%s is not a binding: %w", path, err)
}
if told.At == "" {
return none, fmt.Errorf(
"%s says the package registry is on %q and gives no address for it", path, told.From)
}
// Composed from what the provider serves, so nothing here knows gitea's URL shape from
// another registry's: it states its port, the path its registry answers on, and the scheme.
scheme := "https"
if s, ok := told.Serves["scheme"]; ok {
scheme = fmt.Sprintf("%v", s)
}
port, ok := told.Serves["port"]
if !ok {
return none, fmt.Errorf("%s says nothing about which port the package registry answers on", path)
}
npmPath, ok := told.Serves["npm-path"]
if !ok {
return none, fmt.Errorf("%s says nothing about the path the package registry answers on", path)
}
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
if file := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN_FILE")); file != "" {
raw, err := os.ReadFile(file)
if err != nil {
return none, fmt.Errorf("cannot read this builder's package-registry credential: %w", err)
}
secret = strings.TrimSpace(string(raw))
}
n := builder.Npmrc{Scope: scope, Registry: fmt.Sprintf("%s://%s:%v%v", scheme, told.At, port, npmPath),
From: builder.FromBinding, Machine: machine}
if told.As != "" {
n.Username, n.Password = told.As, secret
} else {
n.Token = secret
}
return n, nil
}
// GenesisRegistry is the genesis document (novox/hq ADR 0336 §3): what the installer sealed to the
// build agent on its node at genesis's packages step, as the `genesis-package-registry` own-secret —
// the forge it raised, the port this node gave the package registry, its npm path, and the account it
// made with that account's password. mesh-host writes it; the field names are the two repositories'
// agreement.
type GenesisRegistry struct {
Scheme string `json:"scheme"`
At string `json:"at"`
Port int `json:"port"`
NpmPath string `json:"npm-path"`
As string `json:"as"`
Password string `json:"password"`
}
// fromGenesis composes the registry from the genesis document.
func fromGenesis(path string, raw []byte, scope, machine string) (builder.Npmrc, error) {
none := builder.Npmrc{Machine: machine}
var g GenesisRegistry
if err := json.Unmarshal(raw, &g); err != nil {
return none, fmt.Errorf("%s is not the genesis document for the package registry: %w", path, err)
}
var missing []string
for name, value := range map[string]string{"at": g.At, "npm-path": g.NpmPath, "as": g.As, "password": g.Password} {
if strings.TrimSpace(value) == "" {
missing = append(missing, name)
}
}
if g.Port <= 0 {
missing = append(missing, "port")
}
if len(missing) > 0 {
sort.Strings(missing)
return none, fmt.Errorf("%s, the genesis document for the package registry, says nothing of %s",
path, strings.Join(missing, ", "))
}
scheme := g.Scheme
if scheme == "" {
scheme = "http"
}
return builder.Npmrc{Scope: scope, Registry: fmt.Sprintf("%s://%s:%d%s", scheme, g.At, g.Port, g.NpmPath),
Username: g.As, Password: g.Password, From: builder.FromGenesis, Machine: machine}, nil
}
// saysRegistry is the line each build's log carries about the package registry (novox/hq ADR 0336
// §4): which of the binding and the genesis document it resolved from, or that it has neither here.
func saysRegistry(n builder.Npmrc) string {
if n.Enabled() {
return "the package registry is resolved from " + n.From
}
machine := n.Machine
if machine == "" {
machine = "this machine"
}
return fmt.Sprintf("no package registry: %s is unbound on %s and no genesis document is here; "+
"a build needing it is refused", builder.RegistryRequirement, machine)
}
// forgeFrom is the git credential this builder may offer a clone, composed from the same binding
// and sealed secret its package-registry half already reads: the forge that answers npm is the
// forge that hosts the repositories, and its provisioner applies one password to one user for
// both. Anything missing means no credential, and every clone stays anonymous — which is all a
// mesh of public repositories ever needs.
//
// The URL names the binding's own address — the machine the mesh says the forge is on — so a
// private repository is registered and built by that address, and a clone of anything else is
// never shown this credential (git's credential store matches the whole origin).
func forgeFrom() builder.GitCredential {
path := strings.TrimSpace(os.Getenv("MESH_PACKAGE_BINDING"))
if path == "" {
return builder.GitCredential{}
}
raw, err := os.ReadFile(path)
if err != nil {
return builder.GitCredential{}
}
var told struct {
At string `json:"at"`
As string `json:"as"`
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal(raw, &told); err != nil || told.At == "" || told.As == "" {
return builder.GitCredential{}
}
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
if file := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN_FILE")); file != "" {
if raw, err := os.ReadFile(file); err == nil {
secret = strings.TrimSpace(string(raw))
}
}
if secret == "" {
return builder.GitCredential{}
}
scheme := "https"
if s, ok := told.Serves["scheme"]; ok {
scheme = fmt.Sprintf("%v", s)
}
host := told.At
if port, ok := told.Serves["port"]; ok {
host = fmt.Sprintf("%s:%v", told.At, port)
}
made := url.URL{Scheme: scheme, User: url.UserPassword(told.As, secret), Host: host}
return builder.GitCredential{URL: made.String()}
}
func short(commit string) string {
if len(commit) > 8 {
return commit[:8]
}
return commit
}
// whereToPublish is the artifact store this builder uses.
//
// **Preferably from the mesh.** A builder that is a module requires an artifact store, and the
// mesh writes it a file saying which machine answers that and on what port — the same binding any
// consumer of any provision gets. Reading it means the address is not a setting somebody keeps in
// step by hand, and moving the store is an ordinary reassignment rather than an edit on every
// build machine.
//
// The environment variable remains for a builder run by a person, which is how this started and
// how it is still run while being developed.
func whereToPublish() (string, error) {
binding := strings.TrimSpace(os.Getenv("MESH_BINDING"))
if binding == "" {
registry := strings.TrimSpace(os.Getenv("MESH_REGISTRY"))
if registry == "" {
return "", fmt.Errorf("neither MESH_BINDING nor MESH_REGISTRY: a built artifact " +
"nobody can fetch is not built")
}
return registry, nil
}
raw, err := os.ReadFile(binding)
if err != nil {
return "", fmt.Errorf("cannot read what the mesh said about the artifact store: %w", err)
}
var told struct {
From string `json:"from"`
At string `json:"at"`
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal(raw, &told); err != nil {
return "", fmt.Errorf("%s is not a binding: %w", binding, err)
}
if told.At == "" {
// The provider is not on the private network, so there is no name to reach it by. Said
// rather than falling back to the machine's own name, which would publish to a store on
// the wrong machine and be found out much later.
return "", fmt.Errorf(
"%s says the artifact store is on %q and gives no address for it", binding, told.From)
}
port, ok := told.Serves["port"]
if !ok {
return "", fmt.Errorf("%s says nothing about which port the artifact store answers on",
binding)
}
return fmt.Sprintf("%s:%v", told.At, port), nil
}
// brokerFrom is where this builder connects, and with what.
//
// **Preferably from a file the mesh sealed to this machine.** A builder that is a module is given
// its credential the way every other module is given one: generated or accepted centrally, sealed
// to the machine, written by the host. Putting it in an environment variable instead would mean
// the one copy that matters passing through a terminal and a process listing.
//
// The variable remains for a builder run by a person.
func brokerFrom() (Credential, error) {
if path := strings.TrimSpace(os.Getenv("MESH_BROKER_FILE")); path != "" {
raw, err := os.ReadFile(path)
if err != nil {
return Credential{}, fmt.Errorf("cannot read this builder's credential: %w", err)
}
said := strings.TrimSpace(string(raw))
if said == "" {
// An empty credential file is a machine that will connect as nobody and be refused,
// with the reason three layers away.
return Credential{}, fmt.Errorf("%s is empty, so this builder has no credential", path)
}
var held Credential
if err := json.Unmarshal([]byte(said), &held); err == nil && held.URL != "" {
return held, nil
}
// A file holding only a URL, which is what a person writing one by hand produces. The
// broker is then verified against whatever this machine already trusts.
return Credential{URL: said}, nil
}
return Credential{}, fmt.Errorf(
"no MESH_BROKER_FILE: a build machine with no credential for the bus has nothing to build")
}
// Credential is what a build machine is given so it can reach the broker.
//
// Two things, because reaching a broker over TLS needs both: who to connect as, and what to check
// the certificate against. A mesh's broker presents a certificate of the mesh's own, which is in
// no public trust store, so a URL alone can only connect to a broker somebody else vouches for.
//
// **The same shape a node gets, for the same reason** (novox/hq ADR 0004): the fingerprint travels
// out of band — here, sealed with the credential — and the endpoint is verified once at connect.
type Credential struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
// User and Password ride beside the address on the bus being built (design 25): a credential
// embedded in a URL leaks into every log line that prints a connection, so the mesh seals them
// as two fields and this machine joins them once, here, to dial.
User string `json:"user,omitempty"`
Password string `json:"password,omitempty"`
// Claims are the seats the module this credential was issued for claims, as the mesh writes
// them beside the credential (novox/hq ADR 0159). The first is the build role this machine
// serves; a credential naming none is from before claims travelled in it.
Claims []struct {
Seat string `json:"seat"`
} `json:"claims,omitempty"`
}
// seatsClaimed is the seats the credential names, in order.
func (c Credential) seatsClaimed() []string {
out := make([]string, 0, len(c.Claims))
for _, claim := range c.Claims {
out = append(out, claim.Seat)
}
return out
}
// onTheNewBus is whether a credential is for the bus being built: its address says so, and the
// mesh only ever seals such a credential with the user and password beside it.
func (c Credential) onTheNewBus() bool { return strings.HasPrefix(strings.TrimSpace(c.URL), "nats://") }
// natsURL is the address with this machine's credential in it, for the one dial that needs it.
func (c Credential) natsURL() string {
rest := strings.TrimPrefix(strings.TrimSpace(c.URL), "nats://")
if c.User == "" {
return "nats://" + rest
}
return "nats://" + c.User + ":" + c.Password + "@" + rest
}
-188
View File
@@ -1,188 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"strings"
"github.com/novox/mesh-controller/internal/builder"
)
// buildOnce is the builder doing one build and stopping, with no broker and no mesh.
//
// **This is how a mesh is raised** (novox/hq ADR 0073). The installer carries this program and runs
// it once, before anything else exists, to produce the control plane from the same repository and
// path that every later rebuild of it will use. What raises the mesh is therefore the same thing
// that maintains it — not a second mechanism that has to be kept in step with the first and is
// exercised once per new mesh, which is how often enough to rot.
//
// It takes no work from a queue and answers nobody: there is no broker yet, and the only thing
// waiting for the answer is the installer that started it. So the result goes to standard output as
// JSON, which is what the installer reads.
//
// With no --registry it publishes nowhere and the image stays in this machine's container runtime,
// named by the digest of its own configuration (see builder.Local). That is the genesis case. With
// one, it publishes as it always does — the same command is how an operator builds a module by hand
// on a mesh that already exists.
func buildOnce(ctx context.Context, args []string) error {
set := flag.NewFlagSet("build", flag.ContinueOnError)
path := set.String("path", "", "the module's directory inside the repository (default: its root)")
ref := set.String("ref", "", "the commit to build; a branch is a moving target somebody else controls")
registry := set.String("registry", "",
"host:port to publish to. Without it the artifacts stay in this machine's container runtime, which is the genesis case")
workspace := set.String("workspace", "", "where to clone and build (default: a temporary directory)")
on := multiple(set, "on",
"a base this build stands on, as <module>/<artifact>=<reference>. Repeatable")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 1 {
return errors.New("mesh-builder build <repository> [--path P] [--ref COMMIT] [--registry HOST:PORT]")
}
repository := positionals[0]
where := *workspace
if where == "" {
where = os.TempDir() + "/mesh-builder-once"
}
// What this build may stand on, said on the command line because there is no mesh to ask.
//
// **This is the genesis case and it is meant to be awkward.** On a running mesh the control
// plane answers this, because only it knows what this mesh holds. Here nothing has been built
// yet, so whoever runs this says what to use — and for the control plane, which is the one
// module raised before anything else exists, the answer is ordinarily nothing at all.
bases := map[string]string{}
for _, pair := range *on {
key, reference, found := strings.Cut(pair, "=")
if !found || key == "" || reference == "" {
return fmt.Errorf(
"--on takes <module>/<artifact>=<reference>, and %q is not that", pair)
}
bases[key] = reference
}
// Local unless told otherwise, because the moment this exists for has nowhere to publish. A
// default pointing at a registry would mean genesis failing at a push to something that is not
// there yet, one step away from the thing that could explain it.
var publisher builder.Publisher = builder.Local{Run: builder.Command}
if *registry != "" {
publisher = builder.Registry{Address: *registry, Run: builder.Command}
}
fmt.Fprintf(os.Stderr, "building %s", repository)
if *path != "" {
fmt.Fprintf(os.Stderr, " at %s", *path)
}
if *ref != "" {
fmt.Fprintf(os.Stderr, " at %s", *ref)
}
fmt.Fprintln(os.Stderr)
machine := os.Getenv("MESH_NODE")
if machine == "" {
machine, _ = os.Hostname()
}
npmrc, err := packagesFrom(machine)
if err != nil {
return err
}
fmt.Fprintf(os.Stderr, " [packages] %s\n", saysRegistry(npmrc))
built, buildErr := builder.Build(ctx, builder.Command, publisher, repository, *path, *ref, where, bases, npmrc,
forgeFrom(),
func(step, message string) { fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message) })
if buildErr != nil {
return buildErr
}
// To standard output, and everything else to standard error, so the caller can read this
// without having to separate it from progress.
out := onceResult{
Module: built.Manifest.Module,
Commit: built.Commit,
Repository: repository,
Path: *path,
Ref: *ref,
Manifest: built.Manifest,
Against: built.Against,
Source: built.Source,
}
for _, r := range built.Read {
out.Read = append(out.Read, readRepository{Repository: r.Repository, Ref: r.Ref})
}
for _, made := range built.Built {
out.Made = append(out.Made, madeArtifact{Name: made.Name, Kind: made.Kind, Reference: made.Reference})
}
body, err := json.MarshalIndent(out, "", " ")
if err != nil {
return err
}
fmt.Println(string(body))
fmt.Fprintf(os.Stderr, " built %s from %s\n", built.Manifest.Module, short(built.Commit))
return nil
}
// onceResult is what one build reports to whoever started it.
//
// The same fields the mesh records for a build, so a reader comparing a genesis build against an
// ordinary one is comparing the same thing said the same way.
type onceResult struct {
Module string `json:"module"`
Commit string `json:"commit"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
Manifest any `json:"manifest"`
Made []madeArtifact `json:"made"`
Against []string `json:"against,omitempty"`
Read []readRepository `json:"read,omitempty"`
Source string `json:"source-fingerprint,omitempty"`
}
// readRepository is a repository this build read source from besides the module's own.
type readRepository struct {
Repository string `json:"repository"`
Ref string `json:"ref,omitempty"`
}
type madeArtifact struct {
Name string `json:"name"`
Kind string `json:"kind"`
Reference string `json:"reference"`
}
// parseAround lets flags appear on either side of the repository, because a person writing this by
// hand will put them wherever reads best and the standard parser stops at the first thing that is
// not a flag. The same helper the control plane's commands use, for the same reason.
func parseAround(set *flag.FlagSet, args []string) ([]string, error) {
var positionals []string
rest := args
for {
if err := set.Parse(rest); err != nil {
return nil, err
}
rest = set.Args()
if len(rest) == 0 {
return positionals, nil
}
positionals = append(positionals, rest[0])
rest = rest[1:]
}
}
// multiple is a flag that may be given more than once.
type repeated []string
func (r *repeated) String() string { return strings.Join(*r, ", ") }
func (r *repeated) Set(v string) error { *r = append(*r, v); return nil }
func multiple(set *flag.FlagSet, name, usage string) *[]string {
var values repeated
set.Var(&values, name, usage)
return (*[]string)(&values)
}
-102
View File
@@ -1,102 +0,0 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/builder"
)
// The build agent's package registry (novox/hq ADR 0336, issue 121): its npm-package-registry
// requirement is optional, so the binding may be absent, and on genesis's node the installer seals
// the forge it raised as one genesis document instead.
// registryFiles names the three paths the build agent's environment always names, writing only those
// given.
func registryFiles(t *testing.T, binding, token, genesis string) {
t.Helper()
dir := t.TempDir()
for env, content := range map[string]string{
"MESH_PACKAGE_BINDING": binding,
"MESH_NPM_TOKEN_FILE": token,
"MESH_GENESIS_PACKAGE_REGISTRY": genesis,
} {
path := filepath.Join(dir, env)
t.Setenv(env, path)
if content != "" {
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatal(err)
}
}
}
for _, env := range []string{"MESH_NPM_REGISTRY", "MESH_NPM_TOKEN", "MESH_NPM_USER", "MESH_NPM_SCOPE"} {
t.Setenv(env, "")
}
}
const (
aBinding = `{"from":"gitea","at":"anchor.mesh","as":"anchor-agent",
"serves":{"scheme":"http","port":3000,"npm-path":"/api/packages/novox/npm/"}}`
aGenesisDocument = `{"scheme":"http","at":"127.0.0.1","port":3100,"npm-path":"/api/packages/novox/npm/",
"as":"mesh-builder","password":"genesis-secret"}`
)
func TestUntilTheRequirementBindsTheGenesisDocumentIsRead(t *testing.T) {
registryFiles(t, "", "", aGenesisDocument)
n, err := packagesFrom("anchor")
if err != nil {
t.Fatalf("an absent binding beside a genesis document was refused: %v", err)
}
if n.From != builder.FromGenesis || n.Registry != "http://127.0.0.1:3100/api/packages/novox/npm/" ||
n.Username != "mesh-builder" || n.Password != "genesis-secret" || n.Scope != "@novox" {
t.Fatalf("the genesis document was not what the registry came from: %+v", n)
}
file, err := n.File()
if err != nil || !strings.Contains(file, "//127.0.0.1:3100/api/packages/novox/npm/:username=mesh-builder") {
t.Fatalf("the .npmrc is not written from the genesis document: %v\n%s", err, file)
}
if got := saysRegistry(n); !strings.Contains(got, "genesis document") {
t.Errorf("the log does not say the genesis document: %s", got)
}
}
func TestARealBindingWinsOverTheGenesisDocument(t *testing.T) {
registryFiles(t, aBinding, "bound-secret\n", aGenesisDocument)
n, err := packagesFrom("anchor")
if err != nil {
t.Fatal(err)
}
if n.From != builder.FromBinding || n.Registry != "http://anchor.mesh:3000/api/packages/novox/npm/" ||
n.Username != "anchor-agent" || n.Password != "bound-secret" {
t.Fatalf("the binding did not win over the genesis document: %+v", n)
}
if got := saysRegistry(n); !strings.Contains(got, "binding") {
t.Errorf("the log does not say the binding: %s", got)
}
}
func TestWithNeitherTheAgentHasNoRegistryAndSaysWhere(t *testing.T) {
registryFiles(t, "", "", "")
n, err := packagesFrom("workstation")
if err != nil {
t.Fatalf("an unbound optional requirement was a fault: %v", err)
}
if n.Enabled() || n.Machine != "workstation" {
t.Fatalf("a registry came from nowhere: %+v", n)
}
got := saysRegistry(n)
for _, want := range []string{"npm-package-registry", "unbound", "workstation"} {
if !strings.Contains(got, want) {
t.Errorf("the log does not say %q: %s", want, got)
}
}
}
func TestAGenesisDocumentMissingAFieldIsRefusedByName(t *testing.T) {
registryFiles(t, "", "", `{"at":"127.0.0.1","port":3000,"npm-path":"/api/packages/novox/npm/","as":"mesh-builder"}`)
if _, err := packagesFrom("anchor"); err == nil || !strings.Contains(err.Error(), "password") {
t.Fatalf("a genesis document with no password was taken: %v", err)
}
}
-44
View File
@@ -1,44 +0,0 @@
package main
import (
"errors"
"fmt"
"os"
"path/filepath"
)
// **The build agent says it is ready** (novox/hq issue 544). It listens on nothing, so no http or tcp
// check can see it work; it writes a mark once it is bound to its worker and pulls build work, and its
// container's health check runs `mesh-builder ready`, which passes only while the mark is there. Without
// it the node-engine could judge the agent only alive, and every walk that moved it waited out the whole
// grace on each machine.
//
// The mark lives in the container's own temporary directory, never in the workspace the machine keeps:
// a new container starts without it, and a restarted one removes it before it binds again.
func readyMark() string { return filepath.Join(os.TempDir(), "mesh-builder.ready") }
// notReady removes the mark: a holder starting is not taking work until it says so again.
func notReady() error {
if err := os.Remove(readyMark()); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove the mark that this agent takes build work (%s): %w", readyMark(), err)
}
return nil
}
// taking writes the mark. A mark that cannot be written is said, and the agent goes on building: its
// check then says it is not ready, which is the truth the gate should hear.
func taking(seat string) {
if err := os.WriteFile(readyMark(), []byte(seat+"\n"), 0o644); err != nil {
fmt.Fprintf(os.Stderr, "cannot mark that this agent takes build work (%s): %v\n", readyMark(), err)
return
}
fmt.Fprintf(os.Stderr, "taking build work from %s\n", seat)
}
// ready is the health check: nil when this container's agent is bound and pulling build work.
func ready() error {
if _, err := os.Stat(readyMark()); err != nil {
return fmt.Errorf("not taking build work yet: %s is not there", readyMark())
}
return nil
}
-32
View File
@@ -1,32 +0,0 @@
package main
import (
"os"
"testing"
)
// The health check passes only between the agent saying it takes work and its next start (novox/hq
// issue 544).
func TestTheAgentIsReadyOnlyOnceItTakesWork(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := ready(); err == nil {
t.Fatal("an agent that never took work is ready")
}
taking("mesh-build-machine")
if err := ready(); err != nil {
t.Fatalf("an agent taking work is not ready: %v", err)
}
// Started again, it is not ready until it binds again.
if err := notReady(); err != nil {
t.Fatal(err)
}
if err := ready(); err == nil {
t.Fatal("an agent starting again is ready before it takes work")
}
if err := notReady(); err != nil {
t.Fatalf("removing a mark that is not there failed: %v", err)
}
if _, err := os.Stat(readyMark()); err == nil {
t.Fatal("the mark is still there")
}
}
-27
View File
@@ -1,27 +0,0 @@
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// The seat a build machine serves comes from its credential (novox/hq ADR 0190 handover).
func TestTheCredentialSaysWhichBuildRoleThisMachineServes(t *testing.T) {
var held Credential
if err := json.Unmarshal([]byte(`{"url":"nats://bus:4222","user":"anchor.builder","password":"x",
"claims":[{"seat":"mesh-build-machine","scope":"mesh","serves":[]}]}`), &held); err != nil {
t.Fatal(err)
}
if got := link.BuildSeatClaimed(held.seatsClaimed()); got != "mesh-build-machine" {
t.Errorf("the old builder's credential serves %q", got)
}
var bare Credential
if err := json.Unmarshal([]byte(`{"url":"nats://bus:4222","user":"anchor.build-agent","password":"x"}`), &bare); err != nil {
t.Fatal(err)
}
if got := link.BuildSeatClaimed(bare.seatsClaimed()); got != link.TheBuildMachine {
t.Errorf("a credential without claims serves %q, want %s", got, link.TheBuildMachine)
}
}
-45
View File
@@ -1,45 +0,0 @@
package main
import (
"os"
"strings"
"testing"
)
// **The genesis path writes its result to stdout and nothing else there.** The installer captures
// stdout alone and parses it as JSON; one stray log line makes that fail with "invalid character"
// — exactly what a builder fmt.Printf caused, breaking a build that had worked. This keeps
// diagnostics on stderr so a future print cannot repeat it silently.
func TestBuilderDiagnosticsStayOffStdout(t *testing.T) {
allowed := map[string]bool{
"string(body)": true, // once.go: the result JSON, which IS stdout
"version)": true, // --version
`"stopping")`: true, // the loop.s shutdown line
"usage)": true, // --help text, for a human
"said)": true, // busy: the answer its node-engine reads (novox/hq ADR 0335)
}
for _, file := range []string{"once.go", "main.go"} {
src, err := os.ReadFile(file)
if err != nil {
t.Fatal(err)
}
for i, raw := range strings.Split(string(src), "\n") {
line := strings.TrimSpace(raw)
if !strings.HasPrefix(line, "fmt.Printf(") &&
!strings.HasPrefix(line, "fmt.Println(") &&
!strings.HasPrefix(line, "fmt.Print(") {
continue
}
ok := false
for token := range allowed {
if strings.Contains(line, token) {
ok = true
}
}
if !ok {
t.Errorf("%s:%d writes a diagnostic to stdout, which the installer parses as JSON:\n %s",
file, i+1, line)
}
}
}
}
-31
View File
@@ -1,31 +0,0 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/builder"
)
// A try's outcome, as the build seat says it in the build's log (novox/hq issue 686): the repository's own
// check's verdict, how long it took, and each line naming what failed.
func TestATrysOutcomeNamesWhatFailed(t *testing.T) {
tried := triedOf(builder.CheckVerdict{Verdict: "pass", Summary: "the change touches no module", Took: 303 * time.Second,
Head: "c0ffee", Trunk: "ca3f43c", Gate: &builder.Layer{Verdict: "pass"},
Repo: &builder.Layer{Verdict: "fail", Summary: "its merge-check.sh failed: TestX",
Failed: "--- FAIL: TestX (0.01s)\n x_test.go:12: got 1, want 2"}})
if tried.Verdict != "fail" || tried.Took != "5m3s" || len(tried.Failed) != 2 || tried.Trunk != "ca3f43c" ||
tried.Summary != "its merge-check.sh failed: TestX" {
t.Fatalf("a failed try: %+v", tried)
}
// A head that does not merge fails both layers: the check's own word.
conflict := triedOf(builder.CheckVerdict{Verdict: "fail", Summary: "does not merge", Gate: &builder.Layer{Verdict: "fail"},
Repo: &builder.Layer{Verdict: "fail", Summary: "does not merge"}})
if conflict.Verdict != "fail" || conflict.Summary != "does not merge" {
t.Fatalf("a try that does not merge: %+v", conflict)
}
// No verdict is an error, never a pass.
if none := triedOf(builder.CheckVerdict{}); none.Verdict != "error" {
t.Fatalf("a try with no verdict: %+v", none)
}
}
-225
View File
@@ -1,225 +0,0 @@
package main
import (
"crypto/ed25519"
"crypto/rand"
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/hex"
"math/big"
"net"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
)
// Where a builder publishes.
//
// Preferably from the mesh: a builder that is a module requires an artifact store, and the mesh
// writes it a binding saying which machine answers and on what port. Reading it means the address
// is not a setting somebody keeps in step by hand.
func binding(t *testing.T, body string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "artifact-store.json")
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
return path
}
func TestTheMeshSaysWhereToPublish(t *testing.T) {
t.Setenv("MESH_BINDING", binding(t, `{"binding":1,"provision":"artifact-store",
"from":"anchor","at":"anchor.internal","serves":{"port":5000,"scheme":"http"}}`))
where, err := whereToPublish()
if err != nil {
t.Fatal(err)
}
if where != "anchor.internal:5000" {
t.Fatalf("got %q", where)
}
}
func TestABindingWithNoAddressIsRefused(t *testing.T) {
// The provider is not on the private network, so there is no name to reach it by. Falling
// back to anything would publish to a store on the wrong machine and be found out much later.
t.Setenv("MESH_BINDING", binding(t, `{"binding":1,"from":"anchor","serves":{"port":5000}}`))
_, err := whereToPublish()
if err == nil {
t.Fatal("a binding with nowhere to reach was accepted")
}
if !strings.Contains(err.Error(), "anchor") {
t.Fatalf("the failure does not name the machine: %v", err)
}
}
func TestABindingWithNoPortIsRefused(t *testing.T) {
t.Setenv("MESH_BINDING", binding(t, `{"binding":1,"from":"a","at":"a.internal","serves":{}}`))
if _, err := whereToPublish(); err == nil {
t.Fatal("a binding saying nothing about a port was accepted")
}
}
func TestTheVariableStillWorksForABuilderRunByAPerson(t *testing.T) {
// Which is how this started and how it is still run while being developed.
t.Setenv("MESH_BINDING", "")
t.Setenv("MESH_REGISTRY", "127.0.0.1:5000")
where, err := whereToPublish()
if err != nil {
t.Fatal(err)
}
if where != "127.0.0.1:5000" {
t.Fatalf("got %q", where)
}
}
func TestNeitherIsRefusedRatherThanGuessed(t *testing.T) {
t.Setenv("MESH_BINDING", "")
t.Setenv("MESH_REGISTRY", "")
if _, err := whereToPublish(); err == nil {
t.Fatal("a builder with nowhere to publish reported somewhere")
}
}
func TestTheCredentialComesFromAFileTheMeshSealed(t *testing.T) {
// A builder that is a module is given its credential the way every module is: sealed to the
// machine and written by the host. An environment variable instead would put the one copy
// that matters through a terminal and a process listing.
path := filepath.Join(t.TempDir(), "broker")
if err := os.WriteFile(path, []byte("amqps://a-builder:secret@broker.internal:5671/\n"), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_BROKER_FILE", path)
t.Setenv("MESH_BROKER_AMQP", "amqp://should-not-be-used@nowhere/")
got, err := brokerFrom()
if err != nil {
t.Fatal(err)
}
if got.URL != "amqps://a-builder:secret@broker.internal:5671/" {
t.Fatalf("got %q", got.URL)
}
}
// The credential the mesh seals carries what to check the broker's certificate against, because a
// mesh's broker presents a certificate of the mesh's own and no public trust store has it. A URL
// alone can only reach a broker somebody else vouches for.
func TestTheSealedCredentialCarriesWhatVerifiesTheBroker(t *testing.T) {
path := filepath.Join(t.TempDir(), "broker")
if err := os.WriteFile(path, []byte(
`{"url":"amqps://a-builder:secret@broker.internal:5671/","fingerprint":"abc123"}`),
0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_BROKER_FILE", path)
t.Setenv("MESH_BROKER_AMQP", "")
got, err := brokerFrom()
if err != nil {
t.Fatal(err)
}
if got.URL != "amqps://a-builder:secret@broker.internal:5671/" {
t.Fatalf("the url was lost: %q", got.URL)
}
if got.Fingerprint != "abc123" {
t.Fatal("the builder was given nothing to check the broker against, so it can only " +
"connect to a broker some public authority vouches for")
}
}
func TestAnEmptyCredentialFileIsRefused(t *testing.T) {
// Otherwise the builder connects as nobody and is refused, with the reason three layers away.
path := filepath.Join(t.TempDir(), "broker")
if err := os.WriteFile(path, []byte("\n"), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_BROKER_FILE", path)
t.Setenv("MESH_BROKER_AMQP", "")
if _, err := brokerFrom(); err == nil {
t.Fatal("an empty credential was accepted")
}
}
func TestABuilderWithNoCredentialAtAllSaysSo(t *testing.T) {
t.Setenv("MESH_BROKER_FILE", "")
t.Setenv("MESH_BROKER_AMQP", "")
if _, err := brokerFrom(); err == nil {
t.Fatal("a builder with no broker reported one")
}
}
// The pin is compared in the spelling the mesh writes it.
//
// A bare digest against a written fingerprint never matches, and the failure is indistinguishable
// from being pointed at the wrong broker — which is the one thing this check exists to report
// truthfully. It cost a lab run.
func TestThePinIsComparedInTheSpellingTheMeshWritesIt(t *testing.T) {
certificate, key := aServerCertificate(t)
der := certificate.Certificate[0]
sum := sha256.Sum256(der)
written := "sha256:" + hex.EncodeToString(sum[:])
listener, err := tls.Listen("tcp", "127.0.0.1:0", &tls.Config{
Certificates: []tls.Certificate{certificate}, MinVersion: tls.VersionTLS12,
})
if err != nil {
t.Fatal(err)
}
defer listener.Close()
go func() {
for {
conn, err := listener.Accept()
if err != nil {
return
}
_ = conn.(*tls.Conn).Handshake()
conn.Close()
}
}()
_ = key
// The pin the mesh wrote must be accepted.
if err := handshakeWith(listener.Addr().String(), written); err != nil {
t.Fatalf("the broker this builder was told about was refused: %v", err)
}
// And a different one refused, or the check reports nothing.
other := "sha256:" + strings.Repeat("ab", 32)
if err := handshakeWith(listener.Addr().String(), other); err == nil {
t.Fatal("a broker this builder was not told about was accepted")
}
}
// handshakeWith runs the pin check the builder dials with against an address.
func handshakeWith(address, pin string) error {
conn, err := tls.Dial("tcp", address, broker.PinnedToFingerprint(pin))
if err != nil {
return err
}
return conn.Close()
}
func aServerCertificate(t *testing.T) (tls.Certificate, ed25519.PrivateKey) {
t.Helper()
public, private, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
template := &x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{CommonName: "a broker"},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(time.Hour),
IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
}
der, err := x509.CreateCertificate(rand.Reader, template, template, public, private)
if err != nil {
t.Fatal(err)
}
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: private}, private
}
-145
View File
@@ -1,145 +0,0 @@
package main
import (
"context"
"fmt"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// A provision whose credential only a person outside the mesh can give (novox/hq ADR 0337, issue 164, issue 165):
// given once for its provider, at the controller's terminal, and sealed in one act to every consumer that binds the
// provision from that provider now. The value is never read back, printed or kept.
// outsideOffer is the provider module's own name when `secret accept <node> <module> <name>` names a provision that
// module offers with a credential issued outside the mesh, and "" when it names anything else (an own secret, or a
// pair with --provider).
func outsideOffer(ctx context.Context, module, name, provider string) (string, error) {
if provider != "" {
return "", nil
}
open, err := openStores(ctx)
if err != nil {
return "", err
}
defer open.Close()
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return "", err
}
if m, known := shelf[module]; known && m.IssuedOutsideOf(name) {
return module, nil
}
return "", nil
}
// offeredOutsideBy is a module that offers the provision with a credential issued outside the mesh, or "".
func offeredOutsideBy(ctx context.Context, provision string) (string, error) {
open, err := openStores(ctx)
if err != nil {
return "", err
}
defer open.Close()
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return "", err
}
module, _ := catalogue.OffersIssuedOutside(shelf, provision)
return module, nil
}
// consumersFrom is every consumer that binds a provision from one provider now, found by resolving every machine as
// its next send would, and the machines that could not be resolved, with why.
func consumersFrom(ctx context.Context, open *stores, provider, providerModule, provision string) (
[]inventory.PairEnd, []string, error) {
nodes, err := open.inventory.Nodes(ctx)
if err != nil {
return nil, nil, err
}
var out []inventory.PairEnd
var unresolved []string
for _, n := range nodes {
plan, _, err := planFor(ctx, open, n.Name)
if err != nil {
unresolved = append(unresolved, fmt.Sprintf("%s: %s", n.Name, oneLine(err.Error())))
continue
}
out = append(out, bindsFrom(plan, n.Name, provider, providerModule, provision)...)
}
return out, unresolved, nil
}
// bindsFrom is every consumer on one machine whose resolution answers the provision from that provider. Pure.
func bindsFrom(plan catalogue.Resolution, machine, provider, providerModule, provision string) []inventory.PairEnd {
var out []inventory.PairEnd
for _, need := range plan.Needs {
if need.ByRecord || need.Name != provision || need.From != provider {
continue
}
if need.Module != "" && need.Module != providerModule {
continue
}
out = append(out, inventory.PairEnd{Consumer: machine, ConsumerModule: need.For, Local: need.Local})
}
sort.Slice(out, func(i, j int) bool {
return out[i].ConsumerModule+"\x00"+out[i].Local < out[j].ConsumerModule+"\x00"+out[j].Local
})
return out
}
// acceptForProvision gives a provision's credential issued outside the mesh, once for its provider (ADR 0337 §3).
func acceptForProvision(ctx context.Context, provider, providerModule, provision, value string) error {
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
consumers, unresolved, err := consumersFrom(ctx, open, provider, providerModule, provision)
if err != nil {
return err
}
accepted, err := open.inventory.AcceptSecretForProvision(ctx, provision, provider, providerModule, value, consumers)
value = ""
if err != nil {
return err
}
fmt.Print(acceptedSaid(provider, providerModule, provision, accepted, unresolved))
if len(accepted.Sealed) == 0 {
return nil
}
if err := sendTo(ctx, open, accepted.Machines); err != nil {
return fmt.Errorf("%s is sealed and could not be sent: %w; `push %s` sends it", provision, err,
strings.Join(accepted.Machines, "`, `push "))
}
fmt.Printf(" sent %s; each consumer starts again on it\n", strings.Join(accepted.Machines, ", "))
return nil
}
// acceptedSaid is what one accept for a provider did, in words that never carry the value. Pure.
func acceptedSaid(provider, providerModule, provision string, a inventory.ProvisionAccepted, unresolved []string) string {
var b strings.Builder
if len(a.Sealed) == 0 {
fmt.Fprintf(&b, "%s from %s on %s: no consumer binds it now, so nothing was sealed; a consumer that binds "+
"later waits for it, and this line given again seals it to every consumer then\n", provision,
providerModule, provider)
} else {
var names []string
for _, c := range a.Sealed {
names = append(names, fmt.Sprintf("%s on %s%s", c.ConsumerModule, c.Consumer, asLocal(c.Local)))
}
fmt.Fprintf(&b, "%s from %s on %s is given, sealed to %s and to %s\n", provision, providerModule, provider,
strings.Join(names, ", "), provider)
fmt.Fprintf(&b, " the mesh cannot read it back, never makes one of its own for it, and will not rotate it\n")
}
for _, c := range a.Unsealable {
fmt.Fprintf(&b, " not sealed to %s on %s: no sealing key on that machine; give it again once it has one\n",
c.ConsumerModule, c.Consumer)
}
for _, u := range unresolved {
fmt.Fprintf(&b, " not known whether a consumer on %s binds it, which did not resolve; give it again once it does\n", u)
}
return b.String()
}
-378
View File
@@ -1,378 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"os"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/lease"
"github.com/novox/mesh-controller/internal/link"
)
// Acting under the lease (novox/hq to-be 45 §6, ADR 0227 rule 1).
//
// **Only the instance holding the lease acts**: sends a declaration, writes a plan, a condition or a
// call. Every one of those passes theLease.epoch, which answers the epoch the act carries or why it may
// not happen. Three ways a process stands to the lease:
//
// - **The serving controller** takes it before it does anything else — before it asserts the bus's
// objects, which are the controller's to write — waiting while another holds it, and renews it.
// Another holder seen, or the key's age passed with no renewal made, is the lease lost: the gate
// closes at once and the process exits, so its service manager restarts it as a candidate (serve,
// in push.go). A renewal the bus does not answer in time is not a loss (novox/hq issue 477): the
// lease is kept, no new act starts once half the key's age passed unrenewed, and the next renewal
// made opens the gate again.
// - **A command run at a shell** — `push` in the installer, the lab, a person repairing a mesh whose
// controller is down (issue 201) — acts **under the holder's epoch** when a controller holds the
// lease: it is the same mesh's word, composed and sent under the store's hold of each machine like
// the serving controller's, and the epoch it carries is read at the moment it acts, so a handover
// between makes it stale and refused like any other. **When nobody holds the lease, the command
// takes it** for as long as it runs and gives it back; a controller starting meanwhile waits for
// it, as it would for another controller.
// - **A process with no bus** — a test, a command that only reads — acts with no epoch and is
// refused nothing: there is nothing to order against, and nothing it does reaches a machine.
//
// **Unleased, said and temporary.** A serving controller whose bus refuses it the lease's key — the bus's
// user list is older than this build and does not grant the bucket yet — and that sees no other holder
// serves without one, as every controller did before the lease: declarations carry no epoch, which no
// node-engine refuses. Said once, kept as a condition (S12), and tried again every renewal interval; the
// first push that sends the bus its new user list grants it, and the next try takes it. Refusing to act
// instead would be a controller that can never send the user list that lets it act.
// actor is this process's standing to the lease.
type actor struct {
mu sync.Mutex
// held is the lease this process holds: the serving controller's, or a command's own.
held *lease.Lease
// unleased is why a serving controller acts without the lease; empty while it holds it or is not
// serving.
unleased string
// serving is a serving controller, which never borrows another's epoch.
serving bool
// kv is the lease bucket, for a command to read the holder's epoch from.
kv jetstream.KeyValue
close func()
// noBus is a process with no bus configured.
noBus bool
// reset is when the lease bucket was found raised again from nothing and its revisions moved past
// the highest epoch issued, and what was said of it; zero when it was not (S12).
reset time.Time
resetSaid string
// renewEvery and poll, when set, are how often the serving controller renews the lease it holds and
// looks again at one another holds, in place of the lease's own: a test's, so two controllers hand the
// lease over in a second rather than in a renewal of five (novox/hq issue 591). A renewal is given
// the lease's own RenewEvery to answer however often it is made, so a slow bus does not lose it.
renewEvery, poll time.Duration
}
// theLease is this process's standing to the lease.
var theLease = &actor{}
// instance names this process among controller instances: its machine, its process and when it
// started. The lease's holder and every call this process keeps carry it.
var instance = func() string {
host, _ := os.Hostname()
return fmt.Sprintf("controller@%s pid %d since %s", host, os.Getpid(), time.Now().UTC().Format(time.RFC3339))
}()
// epoch is the gate: the epoch an act carries — zero for none — or why it may not happen.
func (a *actor) epoch(ctx context.Context) (uint64, error) {
a.mu.Lock()
held, serving, unleased, noBus := a.held, a.serving, a.unleased, a.noBus
a.mu.Unlock()
switch {
case held != nil:
return held.Epoch()
case serving && unleased != "":
return 0, nil
case serving:
return 0, lease.ErrNotHeld
case noBus:
return 0, nil
}
return a.forACommand(ctx)
}
// forACommand is a command's epoch: the holder's, or a lease of its own when nobody holds one.
func (a *actor) forACommand(ctx context.Context) (uint64, error) {
a.mu.Lock()
defer a.mu.Unlock()
if a.held != nil {
return a.held.Epoch()
}
if a.kv == nil {
address, err := broker.BusAddress()
if err != nil {
// No bus: this process reaches no machine, and has nothing to order against.
a.noBus = true
return 0, nil
}
js, err := broker.Dial(address)
if err != nil {
return 0, fmt.Errorf("the bus cannot be reached, so whether a controller holds the lease cannot be "+
"read and nothing is done: %w", err)
}
api, err := jetstream.New(js.Conn())
if err != nil {
js.Close()
return 0, err
}
reading, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
if err := broker.EnsureLeaseBucket(reading, api); err != nil {
js.Close()
return 0, err
}
kv, err := api.KeyValue(reading, broker.LeaseBucket)
if err != nil {
js.Close()
return 0, err
}
a.kv, a.close = kv, js.Close
if _, found, err := lease.Current(reading, kv); err == nil && !found {
// Nobody: this command takes it for as long as it runs.
l, err := lease.Open(reading, api, broker.LeaseBucket, lease.Options{Holder: holderOf(instance),
Say: func(format string, args ...any) { fmt.Printf(format+"\n", args...) }})
if err != nil {
return 0, err
}
epoch, err := l.TryTake(reading)
if err != nil {
return 0, fmt.Errorf("no controller holds the lease and this command could not take it: %w", err)
}
keeping, stop := context.WithCancel(context.Background())
go l.Keep(keeping)
a.held = l
closeBus := a.close
a.close = func() {
stop()
l.Release(context.Background())
closeBus()
}
return epoch, nil
}
}
reading, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
holder, found, err := lease.Current(reading, a.kv)
if err != nil {
return 0, fmt.Errorf("who holds the controller lease cannot be read, so nothing is done: %w", err)
}
if !found {
return 0, errors.New("the controller that held the lease while this command ran let go of it; nothing " +
"more is done under an epoch nobody holds — run the command again")
}
return holder.Epoch, nil
}
// release gives back what this process holds, at its end.
func (a *actor) release() {
a.mu.Lock()
closing := a.close
a.close = nil
a.mu.Unlock()
if closing != nil {
closing()
}
}
// holderOf is this process as the lease's holder.
func holderOf(instance string) lease.Holder {
host, _ := os.Hostname()
build := runningBuild()
if build == "" {
build = version
}
return lease.Holder{Instance: instance, Host: host, Build: build}
}
// RunningBuildVar is where the declaration tells this process which build it is (module.json, the
// controller process's env): the version its bundle is delivered as, `${version}` composed by the
// catalogue from the bundle's digest. Empty for a process placed by hand.
const RunningBuildVar = "MESH_CONTROLLER_VERSION"
// runningBuild is the version of the build this process is, or empty when the declaration did not say.
func runningBuild() string {
return strings.TrimSpace(os.Getenv(RunningBuildVar))
}
// serveUnderTheLease takes the lease for the serving controller, waiting while another holds it, and
// keeps it until ctx ends. Lost is closed when it is lost; the caller exits on it.
func (a *actor) serveUnderTheLease(ctx context.Context, inv *inventory.Inventory, address string) (lost <-chan struct{}, err error) {
a.mu.Lock()
a.serving = true
a.mu.Unlock()
// Its own connection, held as long as the lease, and named so (novox/hq issue 327).
js, err := broker.Dial(address, nats.Name(broker.ConnectionName+" lease"))
if err != nil {
return nil, fmt.Errorf("the mesh is on the bus at %s and this control plane cannot reach it to take the "+
"lease: %w", broker.BareAddress(address), err)
}
api, err := jetstream.New(js.Conn())
if err != nil {
js.Close()
return nil, err
}
asserting, cancel := context.WithTimeout(ctx, 10*time.Second)
err = broker.EnsureLeaseBucket(asserting, api)
cancel()
if err != nil {
js.Close()
return nil, err
}
say := func(format string, args ...any) { fmt.Printf(format+"\n", args...) }
l, err := lease.Open(ctx, api, broker.LeaseBucket, lease.Options{Holder: holderOf(instance),
RenewEvery: a.renewEvery, RenewWithin: lease.RenewEvery, Poll: a.poll, Floor: inv.HighestEpoch, Say: say, Health: controllerHealth, Moved: func(was, floor uint64) {
a.mu.Lock()
defer a.mu.Unlock()
a.reset = time.Now()
a.resetSaid = fmt.Sprintf("the lease bucket was at revision %d with epoch %d already issued: it was "+
"raised again from nothing (a bus whose data was replaced), and its revisions were moved past %d so "+
"no machine refuses the next epoch", was, floor, floor)
}})
if err != nil {
js.Close()
return nil, err
}
gone := make(chan struct{})
epoch, err := l.Take(ctx)
switch {
case ctx.Err() != nil:
js.Close()
return nil, ctx.Err()
case err != nil && !errors.Is(err, lease.ErrUnwritable):
// Whether another controller acts cannot be told: this one does not act, and exits to try again.
js.Close()
return nil, err
case err != nil:
// Nobody holds it and the bus will not let it be written: unleased, said, tried again (see above).
a.mu.Lock()
a.unleased = err.Error()
a.mu.Unlock()
say("this controller serves WITHOUT the lease: %v. Its declarations carry no epoch; it tries again "+
"every %s, and the first push that sends the bus its user list grants it", err, lease.RenewEvery)
go a.takeWhenGranted(ctx, l, inv, gone)
default:
a.took(ctx, l, inv, epoch, gone)
}
a.mu.Lock()
a.close = func() {
if held, err := l.Epoch(); err == nil {
ending, cancel := context.WithTimeout(context.Background(), 5*time.Second)
if err := inv.EndEpoch(ending, held, inventory.EpochReleased); err != nil {
say("how epoch %d ended could not be recorded: %v", held, err)
}
cancel()
}
l.Release(context.Background())
js.Close()
}
a.mu.Unlock()
return gone, nil
}
// took is the lease taken: recorded, earlier epochs nobody gave back ended as expired, kept.
func (a *actor) took(ctx context.Context, l *lease.Lease, inv *inventory.Inventory, epoch uint64, gone chan struct{}) {
a.mu.Lock()
a.held, a.unleased = l, ""
a.mu.Unlock()
h := holderOf(instance)
recording, cancel := context.WithTimeout(ctx, 10*time.Second)
expired, err := inv.TookEpoch(recording, inventory.Epoch{Epoch: epoch, Instance: h.Instance, Host: h.Host,
Build: h.Build, Taken: time.Now()})
cancel()
if err != nil {
fmt.Printf("epoch %d could not be recorded as taken, so a stale refusal from it will not name it: %v\n", epoch, err)
}
for _, e := range expired {
fmt.Printf("the controller of epoch %d (%s) stopped renewing the lease without giving it back: it is "+
"taken over at epoch %d\n", e.Epoch, e.Instance, epoch)
}
go l.Keep(ctx)
go func() {
<-l.Lost()
if ctx.Err() == nil {
why := l.LostWhy()
ending, cancel := context.WithTimeout(context.Background(), 5*time.Second)
_ = inv.EndEpoch(ending, epoch, inventory.EpochLost)
cancel()
fmt.Printf("the controller lease was lost (epoch %d): %v — this controller stops and exits, to "+
"be started again as a candidate\n", epoch, why)
}
close(gone)
}()
}
// takeWhenGranted tries the lease again every renewal interval while serving unleased, and stops this
// controller if another took it meanwhile: two serving at once is what the lease is for.
func (a *actor) takeWhenGranted(ctx context.Context, l *lease.Lease, inv *inventory.Inventory, gone chan struct{}) {
tick := time.NewTicker(lease.RenewEvery)
defer tick.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
}
epoch, err := l.TryTake(ctx)
if errors.Is(err, lease.ErrTaken) {
fmt.Printf("another controller took the lease while this one served without it: %v — this one "+
"stops and exits\n", err)
close(gone)
return
}
if err != nil {
// Still not written, or not readable this time: unleased, said by S12, tried again.
a.mu.Lock()
a.unleased = err.Error()
a.mu.Unlock()
continue
}
a.took(ctx, l, inv, epoch, gone)
return
}
}
// standing is what `status` and the self-check say of this process and the lease.
type standing struct {
Epoch uint64
Held bool
Renewed time.Time
Unleased string
// Reset is when the lease bucket was found raised again from nothing, and ResetSaid what of it.
Reset time.Time
ResetSaid string
}
func (a *actor) standing() standing {
a.mu.Lock()
held, unleased, reset, resetSaid := a.held, a.unleased, a.reset, a.resetSaid
a.mu.Unlock()
st := standing{Unleased: unleased, Reset: reset, ResetSaid: resetSaid}
if held == nil {
return st
}
epoch, err := held.Epoch()
st.Epoch, st.Held, st.Renewed = epoch, err == nil, held.Renewed()
return st
}
// The gates, given to what acts: a declaration's send (link) and a plan's write (the inventory).
func init() {
link.ActingGate = func(ctx context.Context) error {
_, err := theLease.epoch(ctx)
if err != nil {
return fmt.Errorf("this controller may not send: %w", err)
}
return nil
}
}
-439
View File
@@ -1,439 +0,0 @@
package main
import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/broker"
"io"
"slices"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// The things the mesh can be asked to do, separated from how it was asked.
//
// **A surface is an adapter with no decisions in it** (novox/hq ADR 0035). The command line and
// the command API call the same functions here, so an assignment refused at one is refused at the
// other for the same reason and in the same words. The moment a surface can accept something
// another would reject, the mesh has two answers to one question and people learn which to trust.
//
// Each returns what happened as text a person can read and a caller can pass on. Neither surface
// composes its own explanation, because two explanations of one refusal drift.
// assign puts a module on a node, and says at once what in the mesh no longer works out.
//
// The assignment is kept even when the node does not resolve: it is what a person meant, and
// assignment is not an ordering. A consumer assigned before its provider does not resolve for as
// long as it takes to assign the provider, and refusing the first half of a pair would make the
// order somebody types two commands in part of the mesh's rules.
//
// **What is checked is the mesh, not the one machine** — because that is what the assignment
// changes. novox/hq 04-ISSUES/017 names the shape: an action can succeed into a state its own
// verify rejects, and it happens when the action's test is not the test the verify uses. Here the
// action tested one node and the verify is resolution over all of them, so an assignment could be
// reported as fine while it took a provision away from every other machine — a module offering a
// mesh-scoped provision stops offering it the moment its own node stops resolving, and every
// consumer elsewhere is then told *nothing in this mesh provides it*, with a remedy that names a
// module already assigned. That was found on a four-node raise and read as a version bump breaking
// provider recognition; it was neither the version nor the provider.
//
// It costs a resolution per machine. Assignment is a person typing a command, and being told which
// machines this just blocked is worth more than the milliseconds.
func assign(ctx context.Context, open *stores, node string, modules ...string) (string, error) {
return assignWith(ctx, open, node, assignOptions{}, modules...)
}
// assignWith is assign asked with its options: build, for a module known and not built (novox/hq issue 325).
func assignWith(ctx context.Context, open *stores, node string, opts assignOptions, modules ...string) (string, error) {
if len(modules) == 0 {
return "", fmt.Errorf("assign %s names no module", node)
}
if err := refusedMovingTheController(modules); err != nil {
return "", err
}
// Every line of the answer comes from the one resolution after the write (novox/hq issue 237):
// what the reads before it would say about a machine left out is not the answer's.
ctx = leftOutSaidBy(ctx, io.Discard)
// Held while it is recorded, so it cannot land between a converge's preview and its flip and
// be taken without ever having been previewed (novox/hq ADR 0100).
ctx, release, err := holdNodes(ctx, open, []string{node})
if err != nil {
return "", err
}
defer release()
// **A module the catalogue does not hold is looked for further before it is refused** (novox/hq issue
// 325): a build in flight keeps the assignment pending, a module known and not built is said with how to
// build it, and only a name the mesh never heard of is refused as unknown. Several modules in one act
// are judged together (ADR 0207), so an act naming one not registered is not made in part.
if missing, err := notInCatalogue(ctx, open, modules); err != nil {
return "", err
} else if len(missing) > 0 {
if len(modules) == 1 {
return notRegistered(ctx, open, node, modules[0], opts)
}
var lines []string
for _, m := range missing {
where, err := whereIs(ctx, open.inventory, m, time.Now())
if err != nil {
return "", err
}
lines = append(lines, where.said)
}
return "", fmt.Errorf("%w: nothing was assigned, since modules assigned together are judged together "+
"(ADR 0207) and %s not registered:\n %s\n assign them together once each is registered, or each "+
"alone to keep it pending on its build", inventory.ErrNoSuchModule, strings.Join(missing, ", "),
strings.Join(lines, "\n "))
}
// **The one assignment refused for what the node lacks** (novox/hq ADR 0207). Everything else
// an assignment leaves unresolved is kept, because assignment is not an ordering; a module whose
// resources are applied through a seat nothing on the node holds is refused, because that order
// — the service manager, the package manager and the runtime before anything that installs,
// runs or contains — is the mesh's to keep. Several modules in one act are judged together, so
// holders that depend on each other go on in one command.
shelf, before, err := seatDependenciesOnAssign(ctx, open, node, modules)
if err != nil {
return "", err
}
// **Before the new assignment can unsettle a seat somebody holds only by being alone**
// (novox/hq 04-ISSUES/170): what the mesh derived so far is written down, and then the
// assignment resolves against a record rather than against a coincidence.
settled, err := recordDerivedHolders(ctx, open)
if err != nil {
return "", err
}
var lines []string
var added []string
for _, module := range modules {
fresh, err := open.inventory.Assign(ctx, node, module)
if err != nil {
return strings.Join(lines, "\n"), err
}
if !fresh {
// Nothing changed, and saying "is assigned" would read as an action. One node runs one
// of each — the module's name is the assignment's identity (novox/hq ADR 0115).
lines = append(lines, fmt.Sprintf(
"%s already runs %s — one node runs one of each (ADR 0115); nothing changed", node, module))
continue
}
added = append(added, module)
lines = append(lines, fmt.Sprintf("%s is assigned %s", node, module))
}
if len(added) == 0 {
return strings.Join(lines, "\n"), nil
}
answer := strings.Join(lines, "\n")
for _, line := range settled {
answer += "\n " + line
}
// Its bus credential, in the same act (novox/hq issue 203): an assignment pushed before its
// credential exists delivers a process that cannot authenticate and crash-loops until somebody
// runs a second verb and a second push. Issued here when the module speaks on the bus and has
// no credential yet; kept when it has one, so re-assigning rotates nothing.
var issued []string
for _, module := range added {
if line := issueOnAssign(ctx, open, node, module); line != "" {
issued = append(issued, line)
}
}
// **The one resolution the rest of the answer is said from** (novox/hq issue 237), taken after
// the write: what this act changed about this node's unmet seat dependencies, and nothing else
// (novox/hq ADR 0207) — a dependency of a module just assigned, or one this assignment met; the
// rest of the node's list, and every other node's, is `status`'s — and whether the node resolves.
after := resolvedAfter(ctx, open, shelf, node)
for _, line := range unheldChangeTo(shelf, node, before, after.modules, after.unheld) {
answer += "\n " + line
}
for _, line := range issued {
answer += "\n " + line
}
plan, err := after.plan, after.err
if err != nil {
// Kept, and still refused. Both halves are the answer, and the rest of the mesh is still
// worth reporting: this machine's refusal is rarely the only consequence.
return answer + blockedElsewhere(ctx, open, node), err
}
// Kept, and cannot be hosted here. Said at once rather than discovered at push: a module whose
// capability the machine lacks is on the wrong machine, and the assignment records what a person
// meant while this line says it will not run until it moves. The rest of the node still pushes.
isAdded := map[string]bool{}
for _, m := range added {
isAdded[m] = true
}
for _, u := range plan.Unhostable {
if !isAdded[u.Module] {
continue
}
for _, c := range u.Missing {
answer += "\n but " + catalogue.WrongMachine(u.Module, c, node)
}
}
return answer + fmt.Sprintf("\n run `push %s` to send it", node) +
blockedElsewhere(ctx, open, node), nil
}
// seatDependenciesOnAssign is the refusal ADR 0207 makes at assignment, or nothing, with the
// catalogue and the node's assignments it was judged against. Modules already assigned are not new
// and are not judged again.
func seatDependenciesOnAssign(ctx context.Context, open *stores, node string, modules []string) (
map[string]catalogue.Manifest, []string, error) {
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return nil, nil, err
}
assigned, err := open.inventory.Assigned(ctx, node)
if err != nil {
return nil, nil, err
}
already := map[string]bool{}
for _, a := range assigned {
already[a] = true
}
var adding []string
for _, m := range modules {
if !already[m] {
adding = append(adding, m)
}
}
// The lines AssignRefusal says beside an assignment it lets through are said by unheldChange
// with everything else this act changed, so they are not said twice.
if _, err := catalogue.AssignRefusal(shelf, node, assigned, adding); err != nil {
return nil, nil, err
}
// Two modules declaring one path or unit, or one package present and absent, is refused before anything
// is recorded (novox/hq ADR 0210, ADR 0303, 04-ISSUES/235): kept, the node would not resolve until one came
// off again. Two modules declaring one package present is not a collision (ADR 0303, issue 628).
if err := catalogue.CollisionRefusal(shelf, node, assigned, adding); err != nil {
return nil, nil, err
}
return shelf, assigned, nil
}
// unassign takes modules off a node. What they leave behind is the host's business: a directory
// holding anything the mesh did not put there is kept (novox/hq ADR 0030).
//
// It reports the rest of the mesh for the same reason assign does, and more sharply: taking a
// module off one machine is the ordinary way to stop providing something to another, and nothing
// about the command's own output would ever have said so.
//
// **Refused when it takes away the last holder of a seat a module left on the node depends on**
// (novox/hq ADR 0207) — the other side of refusing that module's assignment without one. Several
// modules in one act are judged together, so a holder and its dependents come off in one command.
func unassign(ctx context.Context, open *stores, node string, modules ...string) (string, error) {
if err := refusedMovingTheController(modules); err != nil {
return "", err
}
if len(modules) == 0 {
return "", fmt.Errorf("unassign %s names no module", node)
}
// One resolution after the write, as assign's answer (novox/hq issue 237).
ctx = leftOutSaidBy(ctx, io.Discard)
ctx, release, err := holdNodes(ctx, open, []string{node})
if err != nil {
return "", err
}
defer release()
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return "", err
}
assigned, err := open.inventory.Assigned(ctx, node)
if err != nil {
return "", err
}
// Every one checked before any is taken off, so a refusal leaves the node as it was.
runs := map[string]bool{}
for _, a := range assigned {
runs[a] = true
}
// A module only pending on the node (novox/hq issue 325) is withdrawn, when it is the act's one module:
// that is the undo of an assignment kept while its build runs.
if len(modules) == 1 && !runs[modules[0]] {
said, withdrawn, err := withdrawPending(ctx, open.inventory, node, modules[0])
if err != nil {
return "", err
}
if withdrawn {
return said, nil
}
}
for _, module := range modules {
if !runs[module] {
return "", fmt.Errorf("%s is not assigned to %s", module, node)
}
}
if err := catalogue.UnassignRefusal(shelf, node, assigned, modules); err != nil {
return "", err
}
for _, module := range modules {
if err := open.inventory.Unassign(ctx, node, module); err != nil {
return "", err
}
}
answer := fmt.Sprintf("%s no longer runs %s — run `push %s` to make it so",
node, strings.Join(modules, ", "), node)
after := resolvedAfter(ctx, open, shelf, node)
for _, line := range unheldChangeTo(shelf, node, assigned, after.modules, after.unheld) {
answer += "\n " + line
}
// Whether the node itself still resolves, from the same resolution: before, only the rest of the
// mesh's reads said it, on standard error and once per other machine.
if after.err != nil {
if unresolvable(after.err) {
answer += fmt.Sprintf("\n but %s does not resolve now, so nothing will be sent to it:", node)
} else {
answer += fmt.Sprintf("\n but whether %s still resolves could not be read:", node)
}
for _, line := range strings.Split(strings.TrimRight(after.err.Error(), "\n"), "\n") {
answer += "\n " + strings.TrimSpace(line)
}
}
// What it leaves behind that is irreplaceable is kept and retired, never removed (novox/hq ADR 0233).
for _, line := range keptOnUnassign(ctx, open.inventory, node, modules) {
answer += "\n " + line
}
return answer + blockedElsewhere(ctx, open, node), nil
}
// afterAct is the one resolution of a node an act's answer is said from, taken after its write
// (novox/hq issue 237): the plan or why there is none, the modules it has on the node, and its unmet
// seat dependencies.
type afterAct struct {
plan catalogue.Resolution
err error
modules []string
unheld []catalogue.Unheld
}
// resolvedAfter resolves the node as `plan` does, once, after the act's write. When the node resolves,
// its modules and unmet seat dependencies are the resolution's — the closure, so a holder pulled in by
// a requirement counts as `plan` counts it. When it does not, they are judged over the assignments read
// after the write, which can only find more unmet than the closure would: a seat said held there is
// held in the closure too.
func resolvedAfter(ctx context.Context, open *stores, shelf map[string]catalogue.Manifest, node string) afterAct {
plan, _, err := planFor(ctx, open, node)
if err == nil {
out := afterAct{plan: plan, unheld: plan.Unheld}
for _, m := range plan.Modules {
out.modules = append(out.modules, m.Module)
}
return out
}
out := afterAct{err: err}
assigned, readErr := open.inventory.Assigned(ctx, node)
if readErr != nil {
return out
}
out.modules, out.unheld = assigned, unheldOver(shelf, node, assigned)
return out
}
// splitModules is a surface's one `module` field as the modules it names: several, comma-separated,
// are one act (novox/hq ADR 0207), so the holders that depend on each other go on together from the
// command API and the controller seat's verbs as they do from the command line.
func splitModules(field string) []string {
var out []string
for _, m := range strings.Split(field, ",") {
if m = strings.TrimSpace(m); m != "" {
out = append(out, m)
}
}
return out
}
// blockedElsewhere is every OTHER machine that cannot be worked out as things now stand.
//
// **The state, not the cause.** Saying "this assignment broke laptop" would mean resolving the
// whole mesh twice and would still be a guess about which of several changes did it; saying
// "laptop cannot be worked out, and here is what it says" is true, is what somebody has to fix,
// and cannot mislead. The machine that was just changed is left out because its own refusal is
// already the answer beside this one.
//
// Nothing here can fail the act it reports on. A mesh that cannot be read is worth saying and is
// not a reason to claim the assignment did not happen — it did.
func blockedElsewhere(ctx context.Context, open *stores, except string) string {
nodes, err := open.inventory.Nodes(ctx)
if err != nil {
return "\n\nThe rest of the mesh could not be checked: " + err.Error()
}
blocked := map[string]string{}
for _, n := range nodes {
if n.Name == except {
continue
}
if _, _, err := planFor(ctx, open, n.Name); err != nil {
blocked[n.Name] = err.Error()
}
}
if len(blocked) == 0 {
return ""
}
names := make([]string, 0, len(blocked))
for name := range blocked {
names = append(names, name)
}
sort.Strings(names)
var out strings.Builder
fmt.Fprintf(&out, "\n\nAND %d other machine(s) cannot be worked out as things stand, so "+
"nothing will be sent to them:\n", len(names))
for _, name := range names {
fmt.Fprintf(&out, " %s\n", name)
for _, line := range strings.Split(strings.TrimRight(blocked[name], "\n"), "\n") {
fmt.Fprintf(&out, " %s\n", strings.TrimSpace(line))
}
}
out.WriteString("\nThis may or may not be what just changed — it is what is true now.")
return out.String()
}
// issueOnAssign gives a newly assigned module its bus credential, the way `module issue` does, and
// says what it did in one line. Nothing for a module that declares no broker secret and runs no
// provisioner client; nothing for one whose user is already minted (a credential is rotated on
// purpose, never by re-assigning); and when the bus cannot be reached from here, the line names the
// verb and the push that would refuse the module until it is run — never a silent placeholder (novox/hq issue 203).
func issueOnAssign(ctx context.Context, open *stores, node, module string) string {
inv := open.inventory
shelf, err := inv.Catalogue(ctx)
if err != nil {
return ""
}
m, known := shelf[module]
if !known || mayIssue(m) != nil {
return ""
}
user := broker.Principal{Kind: broker.KindModule, Node: node, Module: module}.Username()
if _, reads := m.OwnSecrets["broker"]; !reads {
// A module running the provisioner client alone is minted the client's account (issue 917).
user = broker.Principal{Kind: broker.KindProvisioner, Node: node, Module: module}.Username()
}
if _, minted, err := inv.BusUserHash(ctx, user); err != nil || minted {
return ""
}
busAddress, err := broker.BusAddress()
if err == nil {
err = issueOnTheNewBus(ctx, inv, m, node, busAddress)
}
if err != nil {
return fmt.Sprintf("its bus credential is not issued (%v): `module issue %s --node %s` first — "+
"`push %s` refuses to send %s until it is", err, module, node, node, module)
}
return fmt.Sprintf("its bus credential is issued and sealed to %s, and arrives with the push", node)
}
// refusedMovingTheController refuses assigning or unassigning the controller's module through a verb (review of
// novox/hq ADR 0272): the node it is assigned to is the control-node, and the control-node's operator account is
// the controller's terminal, so whoever moves the module chooses the terminal. At the terminal alone, as every
// change that says who may change what.
func refusedMovingTheController(modules []string) error {
verb, through := throughAVerb()
if !through || !slices.Contains(modules, controllerModule) {
return nil
}
return fmt.Errorf("%s is assigned and unassigned at the controller's terminal only (mesh-cli on the control-node), "+
"never through a verb (this came through %q): the node it runs on is the control-node, whose operator is the "+
"terminal (novox/hq ADR 0272). Nothing was assigned", controllerModule, verb)
}
-135
View File
@@ -1,135 +0,0 @@
//go:build integration
package main
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// What an assignment says about the machines it was not about.
// **An assignment that blocks another machine says so.**
//
// The shape found on a four-node raise: a module offering a mesh-scoped provision stops offering it
// the moment its own node stops resolving, so an assignment to the provider's machine silently took
// a provision away from every consumer elsewhere. Those consumers were then told *nothing in this
// mesh provides it*, naming as the remedy a module that was already assigned — which read, on the
// way back, as a version bump breaking provider recognition. It was neither the version nor the
// provider: it was one machine's set of assignments, and nothing said so.
//
// novox/hq 04-ISSUES/017 names the general shape — an action succeeding into a state its own verify
// rejects, because the action's test is not the test the verify uses. `assign` tested one node; the
// verify is resolution over all of them.
func TestAnAssignmentThatBlocksAnotherMachineSaysWhichAndWhy(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
// A provider on the hub and a consumer on the other machine, both resolving.
register(t, open, catalogue.Manifest{Module: "step-ca", Version: "1",
Provides: []catalogue.Offer{{Name: "acme-ca", Scope: catalogue.ScopeMesh}},
Serves: map[string]map[string]any{"acme-ca": {"path": "/acme/directory"}}})
register(t, open, catalogue.Manifest{Module: "route-proxy", Version: "1",
Requires: []string{"acme-ca"}})
if _, err := assign(ctx, open, "anchor", "step-ca"); err != nil {
t.Fatal(err)
}
said, err := assign(ctx, open, "laptop", "route-proxy")
if err != nil {
t.Fatalf("a mesh that should resolve did not: %v\n%s", err, said)
}
if strings.Contains(said, "cannot be worked out") {
t.Fatalf("a well mesh was reported as blocked:\n%s", said)
}
// Now break the hub's own set, with nothing but the command a person has.
one, two := rivals()
register(t, open, one)
register(t, open, two)
if _, err := assign(ctx, open, "anchor", "rival-one"); err != nil {
t.Fatal(err)
}
said, err = assign(ctx, open, "anchor", "rival-two")
if err == nil {
t.Fatal("an assignment that makes its own node incoherent was not reported at all")
}
// The node's own refusal is the error, as it always was. What is new is that the machines this
// just took a provision away from are named in the same breath.
if !strings.Contains(said, "laptop") {
t.Fatalf("the machine this blocked is not named:\n%s\n\n%v", said, err)
}
if !strings.Contains(said, "acme-ca") {
t.Fatalf("what laptop is now missing is not said:\n%s", said)
}
if !strings.Contains(said, "cannot be worked out as things stand") {
t.Fatalf("the report does not say what state the mesh is in:\n%s", said)
}
// And the assignment is kept: it is what a person meant, and assignment is not an ordering.
assigned, err := open.inventory.Assigned(ctx, "anchor")
if err != nil {
t.Fatal(err)
}
if !contains(assigned, "rival-two") {
t.Fatalf("the assignment was not kept: %v", assigned)
}
}
// A consumer assigned before its provider is refused for itself and kept, because assignment is not
// an ordering — refusing the first half of a pair would make the order somebody types two commands
// in part of the mesh's rules.
func TestAConsumerAssignedBeforeItsProviderIsStillAssigned(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, catalogue.Manifest{Module: "route-proxy", Version: "1",
Requires: []string{"acme-ca"}})
said, err := assign(ctx, open, "laptop", "route-proxy")
if err == nil {
t.Fatalf("a consumer with nothing to consume resolved:\n%s", said)
}
assigned, err := open.inventory.Assigned(ctx, "laptop")
if err != nil {
t.Fatal(err)
}
if !contains(assigned, "route-proxy") {
t.Fatalf("assignment became an ordering: %v", assigned)
}
}
// Unassigning is the ordinary way to stop providing something to another machine, and it reports
// the same way for the same reason.
func TestUnassigningAProviderNamesWhoIsNowBlocked(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, catalogue.Manifest{Module: "step-ca", Version: "1",
Provides: []catalogue.Offer{{Name: "acme-ca", Scope: catalogue.ScopeMesh}},
Serves: map[string]map[string]any{"acme-ca": {"path": "/acme/directory"}}})
register(t, open, catalogue.Manifest{Module: "route-proxy", Version: "1",
Requires: []string{"acme-ca"}})
if _, err := assign(ctx, open, "anchor", "step-ca"); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", "route-proxy"); err != nil {
t.Fatal(err)
}
said, err := unassign(ctx, open, "anchor", "step-ca")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(said, "laptop") || !strings.Contains(said, "acme-ca") {
t.Fatalf("taking the provider away said nothing about who was consuming it:\n%s", said)
}
}
func contains(all []string, one string) bool {
for _, s := range all {
if s == one {
return true
}
}
return false
}
@@ -1,228 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"os"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/licences"
)
func TestTheRegistryTrustAndEveryImageFollowThePortTheNodeGaveTheStore(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aStore())
if _, err := assign(ctx, open, "anchor", "distribution"); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetSettings(ctx, "anchor", "distribution",
map[string]any{catalogue.PortsSetting: map[string]any{"5000": 5101}}); err != nil {
t.Fatal(err)
}
// A module the mesh built, recorded by digest and path, running on the other machine.
if err := open.inventory.RecordBuild(ctx, inventory.Build{
ID: "b1", Repository: "r", Module: "app", Commit: "abc",
Made: []inventory.Artifact{{Name: "server", Kind: "image",
Reference: catalogue.ArtifactStoreScheme + "app/server@" + aDigest}},
}); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "app", Version: "1",
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-app",
"image": catalogue.ArtifactStoreScheme + "app/server@" + aDigest}}})
if _, err := assign(ctx, open, "laptop", "app"); err != nil {
t.Fatal(err)
}
// The runtime's trust is the runtime's module's to write (novox/hq ADR 0222): a stand-in for it
// asks where this machine reaches the store, as the docker module does.
register(t, open, catalogue.Manifest{Module: "runtime", Version: "1",
Resources: []map[string]any{{"id": "daemon", "type": "file", "path": "/etc/docker/daemon.json",
"into": "json", "content": `{"insecure-registries": ["${seat:mesh-artifact-store:reach}"]}` + "\n"}}})
if _, err := assign(ctx, open, "laptop", "runtime"); err != nil {
t.Fatal(err)
}
on := map[string]bool{"anchor": true, "laptop": true}
node, port, found, err := artifactStoreOnNetwork(ctx, open.inventory, on)
if err != nil || !found || node != "anchor" || port != "5101" {
t.Fatalf("the store is found on %q:%q (%v, %v); the node put it on 5101", node, port, found, err)
}
var trust string
for _, r := range composed(t, open, "laptop").Resources {
if r["path"] == "/etc/docker/daemon.json" {
trust, _ = r["content"].(string)
}
if r["id"] == "app.server" && r["image"] != "anchor.internal:5101/app/server@"+aDigest {
t.Errorf("the image the mesh built is fetched as %v", r["image"])
}
}
if !strings.Contains(trust, "anchor.internal:5101") || strings.Contains(trust, ":5000") {
t.Fatalf("the runtime is told to trust %q; the node put the store on 5101", trust)
}
// And a replay to the catalogue says where the store is now.
announced, err := following{open: open}.Announceable(ctx)
if err != nil {
t.Fatal(err)
}
if len(announced) != 1 || announced[0].Made[0].Reference != "anchor.internal:5101/app/server@"+aDigest {
t.Fatalf("the replay announces %+v", announced)
}
}
func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
m, err := catalogue.ParseManifest(raw)
if err != nil {
t.Fatal(err)
}
control, err := withSeatPorts(m).Resolve([]catalogue.Built{{Name: "controller", Kind: catalogue.ArtifactBundle,
Reference: "https://registry.example/mesh-controller/controller.tar.gz", Digest: aDigest}})
if err != nil {
t.Fatal(err)
}
register(t, open, control)
register(t, open, catalogue.Manifest{Module: "postgres", Version: "1",
Claims: []catalogue.Claim{{Name: "mesh-store", Scope: catalogue.ScopeMesh}},
Listens: []catalogue.Listening{{Port: 5432, From: catalogue.FromMesh}},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-store",
"ports": []any{"5432:5432"}, "image": "pg@" + aDigest}}})
register(t, open, catalogue.Manifest{Module: "lavinmq", Version: "1",
Claims: []catalogue.Claim{{Name: "mesh-broker", Scope: catalogue.ScopeMesh}},
Listens: []catalogue.Listening{{Port: 5671, From: catalogue.FromMesh},
{Port: 5672, From: catalogue.FromMesh}},
Guards: []int{15672},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-broker",
"ports": []any{"5671:5671", "5672:5672", "127.0.0.1:15672:15672"}, "image": "mq@" + aDigest}}})
// The control plane's own bus user is the installer's, seeded at genesis before the controller
// runs (SeedBusUser); without it a push now refuses the credential nobody issued (issue 203).
if err := open.inventory.SeedBusUser(ctx, inventory.BusUser{Username: "anchor.mesh-controller",
Kind: inventory.BusController, Node: "anchor", Module: "mesh-controller"}, "bootstrap"); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "mesh-controller"); err != nil {
t.Fatal(err)
}
// The store's module is registered and given its port, and NOT assigned: the state genesis
// leaves a given-port node in before the foundation is adopted as modules (04-ISSUES/085).
if err := open.inventory.SetSettings(ctx, "anchor", "postgres",
map[string]any{catalogue.PortsSetting: map[string]any{"5432": 6852}}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "lavinmq"); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetSettings(ctx, "anchor", "lavinmq",
map[string]any{catalogue.PortsSetting: map[string]any{"5672": 5679}}); err != nil {
t.Fatal(err)
}
var env map[string]any
for _, r := range composed(t, open, "anchor").Resources {
if r["id"] == "mesh-controller.controller" {
env, _ = r["env"].(map[string]any)
}
}
if env == nil {
t.Fatal("the control plane's process is not in its own node's declaration")
}
for key, want := range map[string]string{
"MESH_STORE_INVENTORY_PORT": "6852",
"MESH_STORE_IDENTITY_PORT": "6852",
"MESH_STORE_LICENCES_PORT": "6852",
"MESH_BROKER_AMQP_PORT": "5679",
// Neither given nor assigned by the mesh: the manifest's own number is NOT the answer,
// because the sealed value beside it carries the port genesis wrote (finding F3).
"MESH_BROKER_ADDRESS_PORT": "",
"MESH_BROKER_MANAGEMENT_PORT": "",
} {
if env[key] != want {
t.Errorf("the control plane is told %s=%v; the node says %q", key, env[key], want)
}
}
}
func aLoneNode(t *testing.T) *stores {
t.Helper()
inventory.ForTest(t)
licences.ForTest(t)
open, err := openStores(t.Context())
if err != nil {
t.Fatal(err)
}
t.Cleanup(open.Close)
for _, m := range provided {
if err := open.inventory.Provide(t.Context(), m); err != nil {
t.Fatal(err)
}
}
record, err := open.inventory.AddNode(t.Context(), "anchor")
if err != nil {
t.Fatal(err)
}
reported, _ := json.Marshal(map[string]any{"capabilities": []map[string]any{
{"name": "container-runtime", "present": true}}})
var profile map[string]any
_ = json.Unmarshal(reported, &profile)
if err := open.inventory.RecordProfile(t.Context(), record.ID, profile); err != nil {
t.Fatal(err)
}
if err := open.inventory.RecordSealingKey(t.Context(), record.ID, aPublicKey(t)); err != nil {
t.Fatal(err)
}
return open
}
func TestOnANodeWithNoNetworkTheStoreIsReachedByLoopback(t *testing.T) {
open := aLoneNode(t)
ctx := t.Context()
register(t, open, aStore())
register(t, open, catalogue.Manifest{Module: "builder", Version: "1",
Requires: []string{catalogue.ArtifactStoreProvision},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-builder",
"image": "registry.example/mesh-builder@" + aDigest}}})
if err := open.inventory.RecordBuild(ctx, inventory.Build{
ID: "b1", Repository: "r", Module: "postgres", Commit: "abc",
Made: []inventory.Artifact{{Name: "runtime", Kind: "image",
Reference: catalogue.ArtifactStoreScheme + "postgres/runtime@" + aDigest}},
}); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "postgres", Version: "1",
Resources: []map[string]any{{"id": "runtime", "type": "container", "name": "mesh-postgres",
"image": catalogue.ArtifactStoreScheme + "postgres/runtime@" + aDigest}}})
for _, module := range []string{"distribution", "builder", "postgres"} {
if _, err := assign(ctx, open, "anchor", module); err != nil {
t.Fatal(err)
}
}
if err := open.inventory.SetSettings(ctx, "anchor", "distribution",
map[string]any{catalogue.PortsSetting: map[string]any{"5000": 5100}}); err != nil {
t.Fatal(err)
}
var image any
for _, r := range composed(t, open, "anchor").Resources {
if r["id"] == "postgres.runtime" {
image = r["image"]
}
}
if image != "127.0.0.1:5100/postgres/runtime@"+aDigest {
t.Fatalf("on the store's own node, off any network, the image is fetched as %v", image)
}
held := heldBy(ctx)
if got := held["postgres/runtime"]; got != "127.0.0.1:5100/postgres/runtime@"+aDigest {
t.Fatalf("a builder beside the store is handed the base %q", got)
}
}
-139
View File
@@ -1,139 +0,0 @@
package main
import (
"encoding/json"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
// An address is read from the node's settings where it is used, never recorded with a port
// (novox/hq 04-ISSUES/102). Three readers did not follow the setting; each is held to it here.
var aDigest = "sha256:" + strings.Repeat("e", 64)
// **A build is recorded by digest and path**, whatever address the builder pushed to — what it
// made and what it stood on both; an image the module runs from elsewhere is left where it says.
func TestABuildIsRecordedWithoutTheStoresAddress(t *testing.T) {
manifest, _ := json.Marshal(map[string]any{
"module": "gitea", "version": "1",
"resources": []map[string]any{
{"id": "server", "type": "container", "name": "mesh-gitea",
"image": "anchor.internal:5100/gitea/server@" + aDigest},
{"id": "config", "type": "archive", "path": "/etc/gitea", "digest": aDigest,
"source": "http://anchor.internal:5100/v2/gitea/config/blobs/" + aDigest},
{"id": "cache", "type": "container", "name": "mesh-gitea-cache",
"image": "valkey/valkey@" + aDigest},
},
})
kept := buildFrom(link.BuildResult{
ID: "b1", Repository: "https://forge.example/gitea.git", Commit: "abc", On: "laptop",
Manifest: manifest,
Made: []link.MadeArtifact{
{Name: "server", Kind: "image", Reference: "anchor.internal:5100/gitea/server@" + aDigest},
{Name: "config", Kind: "archive", Reference: "http://anchor.internal:5100/v2/gitea/config/blobs/" + aDigest},
},
Against: []string{"anchor.internal:5100/mesh-tools/runtime@" + aDigest},
})
if kept.Module != "gitea" {
t.Fatalf("the module was not read from the recorded manifest: %q", kept.Module)
}
if kept.Made[0].Reference != catalogue.ArtifactStoreScheme+"gitea/server@"+aDigest {
t.Errorf("the image is recorded as %q, address and all", kept.Made[0].Reference)
}
if kept.Made[1].Reference != catalogue.ArtifactStoreScheme+"gitea/config/blobs/"+aDigest {
t.Errorf("the archive is recorded as %q, address and all", kept.Made[1].Reference)
}
recorded, err := catalogue.ParseManifest(kept.Manifest)
if err != nil {
t.Fatal(err)
}
if got := recorded.Resources[0]["image"]; got != catalogue.ArtifactStoreScheme+"gitea/server@"+aDigest {
t.Errorf("the recorded manifest's image is %v", got)
}
if got := recorded.Resources[1]["source"]; got != catalogue.ArtifactStoreScheme+"gitea/config/blobs/"+aDigest {
t.Errorf("the recorded manifest's archive is %v", got)
}
if got := recorded.Resources[2]["image"]; got != "valkey/valkey@"+aDigest {
t.Errorf("an image the build did not make was rewritten: %v", got)
}
if strings.Contains(string(kept.Manifest), "anchor.internal:5100") {
t.Errorf("the recorded manifest still carries the store's address:\n%s", kept.Manifest)
}
// What the build stood on is an edge to another module's artifact, and it is recorded the way
// that artifact is: by path in the store, so the edge still names the same thing when the
// store answers at another address.
if kept.Against[0] != catalogue.ArtifactStoreScheme+"mesh-tools/runtime@"+aDigest {
t.Errorf("what the build stood on was recorded by address: %v", kept.Against)
}
}
// aStore is a module offering the artifact store on 5000, published the long way as the
// distribution module does, so a node may be given another number for it.
func aStore() catalogue.Manifest {
return catalogue.Manifest{Module: "distribution", Version: "1",
Provides: []catalogue.Offer{{Name: catalogue.ArtifactStoreProvision, Scope: catalogue.ScopeMesh}},
Serves: map[string]map[string]any{catalogue.ArtifactStoreProvision: {"port": float64(5000)}},
Listens: []catalogue.Listening{{Port: 5000, From: catalogue.FromMesh}},
Resources: []map[string]any{{"id": "store", "type": "container", "name": "mesh-registry",
"ports": []any{"5000:5000"}, "image": "registry@" + aDigest}}}
}
// **The trust a machine writes for the store, and the address every built image is fetched
// through, say the port the node gave the store** — not the catalogue's number.
// **The control plane's own connections say the port the node gave the store and the broker.**
//
// Composed from the control plane's own manifest against a real inventory: the store's module is
// given 6852 on this node the way genesis or an operator gives it, and the control plane's
// process is told so beside the sealed connection genesis wrote.
// withSeatPorts is the control plane's manifest with the seat placeholders in its environment —
// added here until module.json carries them (the manifest lands one commit after the code that
// fills it, so a control plane one build behind never sees a placeholder it cannot fill).
func withSeatPorts(m catalogue.Manifest) catalogue.Manifest {
seatPorts := map[string]string{
"MESH_STORE_INVENTORY_PORT": "${seat:mesh-store:5432}",
"MESH_STORE_IDENTITY_PORT": "${seat:mesh-store:5432}",
"MESH_STORE_LICENCES_PORT": "${seat:mesh-store:5432}",
"MESH_BROKER_AMQP_PORT": "${seat:mesh-broker:5672}",
"MESH_BROKER_MANAGEMENT_PORT": "${seat:mesh-broker:15672}",
"MESH_BROKER_ADDRESS_PORT": "${seat:mesh-broker:5671}",
}
out := m
out.Resources = nil
for _, r := range m.Resources {
if r["type"] != "container" && r["type"] != "process" {
out.Resources = append(out.Resources, r)
continue
}
copied := map[string]any{}
for k, v := range r {
copied[k] = v
}
env := map[string]any{}
if had, ok := r["env"].(map[string]any); ok {
for k, v := range had {
env[k] = v
}
}
for k, v := range seatPorts {
if _, said := env[k]; !said {
env[k] = v
}
}
copied["env"] = env
out.Resources = append(out.Resources, copied)
}
return out
}
// aLoneNode is one capable machine with nothing placed on any network — the control-node during
// genesis, before the "network" step, which is after the store, the broker, the vault and the
// catalogue have each been built and pushed (finding F2).
// **Before the network exists, the store's own node reaches it by loopback** — never refused,
// never handed the scheme: a genesis pushes the store, the broker, the vault and the catalogue to
// a node on no network, and builds the catalogue on a base it must be able to pull.
-643
View File
@@ -1,643 +0,0 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"errors"
"net/http"
"reflect"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/overlay"
)
// novox/hq ADR 0100: taking a module is its cutover; converging a node is one act, previewed, and
// refused while a found container is held; returning to adopted keeps what was taken.
// anAdoptedAnchor is aMesh with the anchor adopted, running a served module the predecessor also
// runs and a module with only a file, and a filter module in the catalogue.
func anAdoptedAnchor(t *testing.T) (*stores, *[]string) {
t.Helper()
open := aMesh(t)
ctx := t.Context()
register(t, open, catalogue.Manifest{Module: "hello-web", Version: "1",
Listens: []catalogue.Listening{{Port: 8080, From: catalogue.FromEverywhere}},
Resources: []map[string]any{
{"id": "page", "type": "file", "path": "/var/lib/hello-web/index.html", "content": "hi"},
{"id": "server", "type": "container", "name": "hello-web", "ports": []any{"8080:80"},
"image": "registry.example/hello@sha256:" + strings.Repeat("a", 64)},
}})
register(t, open, catalogue.Manifest{Module: "notes", Version: "1",
Resources: []map[string]any{
{"id": "conf", "type": "file", "path": "/etc/notes.conf", "content": "x"},
}})
register(t, open, catalogue.Manifest{Module: "nftables", Version: "1",
Filtering: &catalogue.Filtering{Into: "/etc/nftables.conf"},
Resources: []map[string]any{{"id": "load", "type": "service", "unit": "mesh-filter.service",
"state": "running", "restart-on": []any{"filtering"}}}})
if err := open.inventory.SetAdopted(ctx, "anchor", true); err != nil {
t.Fatal(err)
}
for _, m := range []string{"hello-web", "notes"} {
if _, err := assign(ctx, open, "anchor", m); err != nil {
t.Fatal(err)
}
}
sent := &[]string{}
saved := sendNodes
sendNodes = func(_ context.Context, _ *stores, names []string) error {
*sent = append(*sent, names...)
return nil
}
t.Cleanup(func() { sendNodes = saved })
return open, sent
}
// reportsHolding has the anchor report, on what it was last sent, holding what is given.
func reportsHolding(t *testing.T, open *stores, held ...link.Held) {
t.Helper()
reportsReaching(t, open, []link.Reach{
{Protocol: "tcp", Address: "0.0.0.0", Port: 22, By: "sshd"},
{Protocol: "tcp", Address: "0.0.0.0", Port: 8080, By: "hello-web", Published: true,
ContainerPort: 80},
{Protocol: "tcp", Address: "0.0.0.0", Port: 5000, By: "predecessor-registry",
Published: true, ContainerPort: 5000},
{Protocol: "tcp", Address: "127.0.0.1", Port: 15672, By: "mesh-broker",
Published: true, ContainerPort: 15672},
}, held...)
}
// reportsReaching has the anchor report, on what it was last sent, what is reachable on it and
// holding what is given.
func reportsReaching(t *testing.T, open *stores, reachable []link.Reach, held ...link.Held) {
t.Helper()
ctx := t.Context()
body, err := composed(t, open, "anchor").Body()
if err != nil {
t.Fatal(err)
}
record, err := open.inventory.NodeByName(ctx, "anchor")
if err != nil {
t.Fatal(err)
}
if err := open.inventory.RecordSent(ctx, record.ID, digestOf(body), nil); err != nil {
t.Fatal(err)
}
if _, err := (link.Enrolment{Inventory: open.inventory}).Heard(ctx, link.Report{
Node: "anchor", Applied: []string{"hello-web.x"}, Declared: digestOf(body),
Firewall: "ufw", Held: held, Reachable: reachable,
// A machine says which of its links face outside on every apply (novox/hq ADR 0140), and a
// filter is not sent to one that has not. The anchor reports one, as a real host does; this
// fixture lacked it from 2026-09-28 and nothing ran the test (issue 177).
Outward: []string{"eth0"},
// And what filters it (ADR 0168): its front end, the runtime's own, and a chain a
// predecessor left in the runtime's user chain.
Filters: anchorFilters,
}); err != nil {
t.Fatal(err)
}
}
// anchorFilters is what the adopted anchor says filters it: ufw's chains, the runtime's, and a
// predecessor's chain the mesh did not write.
var anchorFilters = []link.Filter{
{Where: "table ip filter, chain ufw-reject-input", Owner: "found-firewall", Refuses: "reject"},
{Where: "table ip filter, chain DOCKER", Owner: "runtime", Refuses: `iifname != "docker0" oifname "docker0" drop`},
{Where: "table ip filter, chain DOCKER-USER", Owner: "other", Refuses: `iifname "eth0" tcp dport 6000 drop`},
}
var (
heldContainer = link.Held{ID: "hello-web.server", Module: "hello-web", Kind: "container",
Target: "hello-web", Since: time.Now()}
heldFile = link.Held{ID: "notes.conf", Module: "notes", Kind: "file",
Target: "/etc/notes.conf", Since: time.Now(), Kept: "/var/lib/mesh-host/kept/abc-notes.conf"}
)
func TestTakingAModuleNotOnTheNodeIsRefused(t *testing.T) {
open, _ := anAdoptedAnchor(t)
if _, err := take(t.Context(), open, "anchor", "nftables", takeOptions{Yes: true}); !errors.Is(err, inventory.ErrNotAssigned) {
t.Fatalf("taking an unassigned module gave %v", err)
}
if _, err := take(t.Context(), open, "laptop", "network", takeOptions{Yes: true}); !errors.Is(err, inventory.ErrNotAdopted) {
t.Fatalf("taking on a converged node gave %v", err)
}
}
func TestConvergingIsRefusedOnAPreviewThatWouldBeStale(t *testing.T) {
open, sent := anAdoptedAnchor(t)
_, err := converge(t.Context(), open, "anchor", false, "", "")
if err == nil || !strings.Contains(err.Error(), "has not reported") {
t.Fatalf("a node that never reported was previewed: %v", err)
}
if len(*sent) != 0 {
t.Fatal("a refused converge sent something")
}
}
func TestTheFlipIsRefusedWhileAFoundContainerIsHeld(t *testing.T) {
open, sent := anAdoptedAnchor(t)
reportsHolding(t, open, heldContainer, heldFile)
_, err := converge(t.Context(), open, "anchor", true, "", "")
if err == nil || !strings.Contains(err.Error(), "take anchor hello-web once its data has moved") {
t.Fatalf("the flip was not refused while hello-web holds its found container: %v", err)
}
if n, _ := open.inventory.NodeByName(t.Context(), "anchor"); !n.Adopted || len(*sent) != 0 {
t.Fatal("a refused flip changed something")
}
}
func TestTakingNamesWhatItReplaces(t *testing.T) {
open, _ := anAdoptedAnchor(t)
reportsHolding(t, open, heldContainer, heldFile)
ctx := t.Context()
// The machine holds something for the module, so the take acts on the preview the operator
// saw and names its digest (novox/hq ADR 0163).
preview, err := take(ctx, open, "anchor", "hello-web", takeOptions{})
if err != nil {
t.Fatal(err)
}
saw := takeDigestIn(t, preview)
if !strings.Contains(preview, "nothing taken; `take anchor hello-web --yes "+saw+"`") {
t.Fatalf("the preview does not say how to act on it:\n%s", preview)
}
if taken, _ := open.inventory.Taken(ctx, "anchor"); len(taken) != 0 {
t.Fatal("the preview took something")
}
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err == nil || !strings.Contains(err.Error(), "name its digest") {
t.Fatalf("--yes without the digest was not refused: %v", err)
}
said, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(said, "container hello-web (hello-web.server)") ||
!strings.Contains(said, "push anchor") {
t.Fatalf("taking did not say what it replaces and what to run:\n%s", said)
}
}
// takeDigestIn is the digest a take's preview printed.
func takeDigestIn(t *testing.T, preview string) string {
t.Helper()
for _, line := range strings.Split(preview, "\n") {
if fields := strings.Fields(line); len(fields) == 2 && fields[0] == "preview" {
return fields[1]
}
}
t.Fatalf("the preview printed no digest:\n%s", preview)
return ""
}
// A take acts on the preview the operator saw, and on an account of the machine that is still the
// machine: a changed preview and a stale account refuse (novox/hq ADR 0163, rule 1).
func TestATakeIsRefusedOnAChangedPreviewOrAStaleAccount(t *testing.T) {
open, _ := anAdoptedAnchor(t)
ctx := t.Context()
reportsHolding(t, open, heldContainer, heldFile)
preview, err := take(ctx, open, "anchor", "hello-web", takeOptions{})
if err != nil {
t.Fatal(err)
}
saw := takeDigestIn(t, preview)
// The machine reports again, and what it holds has changed: the found container now carries
// facts the preview never showed.
changed := heldContainer
changed.Facts = map[string]any{"image": "hello:2", "declared_image": "registry.example/hello"}
reportsHolding(t, open, changed, heldFile)
_, err = take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw})
if err == nil || !strings.Contains(err.Error(), "has changed since preview "+saw) {
t.Fatalf("a changed preview was acted on: %v", err)
}
if taken, _ := open.inventory.Taken(ctx, "anchor"); len(taken) != 0 {
t.Fatal("a refused take took something")
}
// And an account older than the flip allows.
preview, err = take(ctx, open, "anchor", "hello-web", takeOptions{})
if err != nil {
t.Fatal(err)
}
saw = takeDigestIn(t, preview)
saved := reportFreshFor
reportFreshFor = -time.Second
defer func() { reportFreshFor = saved }()
_, err = take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw})
if err == nil || !strings.Contains(err.Error(), "a take acts only on an account newer than") {
t.Fatalf("a stale account was acted on: %v", err)
}
reportFreshFor = saved
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw}); err != nil {
t.Fatal(err)
}
// A module the machine holds nothing for has nothing to compare: --yes alone suffices.
if _, err := take(ctx, open, "anchor", "notes", takeOptions{Yes: true}); err == nil {
// notes holds a file, so this one needs the digest too.
t.Fatal("notes holds a found file and was taken without a digest")
}
}
func TestConvergingPreviewsThenChangesAndAdoptingKeepsWhatWasTaken(t *testing.T) {
open, sent := anAdoptedAnchor(t)
ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err)
}
reportsHolding(t, open, heldFile)
preview, err := converge(ctx, open, "anchor", false, "", "")
if err != nil {
t.Fatal(err)
}
for _, want := range []string{
"tcp/8080 hello-web (published, container port 80)",
"declared by hello-web (from anywhere)",
"WILL CLOSE — no module assigned here declares it",
// The anchor faces inward and is on the private network: the derived filter admits ssh
// from the mesh only.
"WILL CLOSE to everything outside the private network — ssh stays open from the mesh",
"notes\n replacing the found file /etc/notes.conf (notes.conf), original kept at",
"assigns nftables",
"the found firewall (ufw) is disabled, never flushed",
// What it routes is not a listener: said not to be previewed, and to be dropped.
"not previewed: traffic the machine routes that is not a published port",
"the derived filter drops it unless a module declares it",
} {
if !strings.Contains(preview, want) {
t.Errorf("the preview does not say %q:\n%s", want, preview)
}
}
if strings.Contains(preview, "15672") {
t.Errorf("a loopback listener is in the preview:\n%s", preview)
}
// What filters the machine now, and the fate of each (novox/hq ADR 0168): the predecessor's
// chain is named as not the mesh's and left, so the reader knows before the flip.
for _, want := range []string{
"table ip filter, chain DOCKER-USER",
"NOT THE MESH'S; left in force",
`iifname "eth0" tcp dport 6000 drop`,
"table ip filter, chain ufw-reject-input",
"the found firewall's; retired with it",
"the container runtime's own; left",
} {
if !strings.Contains(preview, want) {
t.Errorf("the preview does not say %q:\n%s", want, preview)
}
}
for _, line := range strings.Split(preview, "\n") {
if strings.Contains(line, "5000") && !strings.Contains(line, "WILL CLOSE") {
t.Errorf("an undeclared published port is not said to close: %s", line)
}
}
if n, _ := open.inventory.NodeByName(ctx, "anchor"); !n.Adopted || len(*sent) != 0 {
t.Fatal("the preview changed something")
}
if _, err := converge(ctx, open, "anchor", true, digestIn(t, preview), ""); err != nil {
t.Fatal(err)
}
n, _ := open.inventory.NodeByName(ctx, "anchor")
if n.Adopted {
t.Fatal("converge --yes left the node adopted")
}
taken, _ := open.inventory.Taken(ctx, "anchor")
if !reflect.DeepEqual(taken, []string{"hello-web", overlay.Name, "nftables", "notes"}) {
t.Fatalf("the flip took %v", taken)
}
if !reflect.DeepEqual(*sent, []string{"anchor"}) {
t.Fatalf("the flip sent %v", *sent)
}
declared := composed(t, open, "anchor")
if declared.Adoption != nil {
t.Fatal("a converged node is still sent an adoption envelope")
}
if !hasID(declared.Resources, "nftables.filtering") {
t.Fatal("the converged node is not declared the mesh's filter")
}
if _, err := adopt(ctx, open, "anchor"); err != nil {
t.Fatal(err)
}
if _, err := adopt(ctx, open, "anchor"); err == nil {
t.Fatal("adopting an adopted node was not refused")
}
again, _ := open.inventory.Taken(ctx, "anchor")
if !reflect.DeepEqual(again, taken) {
t.Fatalf("returning to adopted lost what was taken: %v", again)
}
declared = composed(t, open, "anchor")
if declared.Adoption == nil || len(declared.Adoption.Untaken) != 0 {
t.Fatalf("returned to adopted, the envelope is %+v", declared.Adoption)
}
if hasID(declared.Resources, "nftables.filtering") || hasID(declared.Resources, "nftables.load") {
t.Fatal("returned to adopted, the mesh's filter is still declared")
}
}
func hasID(resources []map[string]any, id string) bool {
for _, r := range resources {
if r["id"] == id {
return true
}
}
return false
}
// The command API refuses a flip exactly as the command line does, in the same words.
func TestTheApiRefusesTheFlipInTheCommandLinesWords(t *testing.T) {
open, _ := anAdoptedAnchor(t)
reportsHolding(t, open, heldContainer)
_, direct := converge(t.Context(), open, "anchor", true, "", "")
if direct == nil {
t.Fatal("the flip was not refused")
}
got := asking(t, letIn{}, "POST", "/converge", `{"node":"anchor","yes":true}`)
if got.Code != http.StatusConflict {
t.Fatalf("got %d: %s", got.Code, got.Body.String())
}
var said map[string]any
if err := json.Unmarshal(got.Body.Bytes(), &said); err != nil {
t.Fatal(err)
}
if said["refused"] != direct.Error() {
t.Fatalf("the API said %q and the command line %q", said["refused"], direct.Error())
}
if got := asking(t, letIn{}, "POST", "/take", `{"node":"anchor"}`); got.Code != http.StatusBadRequest {
t.Fatalf("a take naming no module got %d", got.Code)
}
}
// novox/hq ADR 0100: the preview says what the derived filter does, rendered as it is rendered. On
// a machine that faces inward, ssh is admitted from the private network only, and a port a module
// admits from the mesh only closes to everything outside it: both are said to close.
func TestThePreviewSaysWhatNarrowsToTheMeshCloses(t *testing.T) {
open, _ := anAdoptedAnchor(t)
ctx := t.Context()
register(t, open, catalogue.Manifest{Module: "store", Version: "1",
Listens: []catalogue.Listening{{Port: 5432, From: catalogue.FromMesh}}})
if _, err := assign(ctx, open, "anchor", "store"); err != nil {
t.Fatal(err)
}
reportsReaching(t, open, []link.Reach{
{Protocol: "tcp", Address: "0.0.0.0", Port: 22, By: "sshd"},
{Protocol: "tcp", Address: "0.0.0.0", Port: 5432, By: "postgres"},
{Protocol: "tcp", Address: "10.77.0.1", Port: 5432, By: "postgres"},
{Protocol: "tcp", Address: "0.0.0.0", Port: 8080, By: "hello-web", Published: true,
ContainerPort: 80},
})
preview, err := converge(ctx, open, "anchor", false, "", "")
if err != nil {
t.Fatal(err)
}
lines := map[string]string{}
for _, line := range strings.Split(preview, "\n") {
fields := strings.Fields(line)
if len(fields) > 1 && strings.HasPrefix(fields[0], "tcp/") {
lines[fields[0]+" "+fields[1]] += line + "\n"
}
}
if got := lines["tcp/22 sshd"]; !strings.Contains(got, "WILL CLOSE to everything outside "+
"the private network") {
t.Errorf("ssh on an inward machine is not said to close outside the mesh:\n%s", preview)
}
store := lines["tcp/5432 postgres"]
if strings.Count(store, "WILL CLOSE to everything outside the private network") != 1 ||
!strings.Contains(store, "declared by store (from mesh)") {
t.Errorf("the store's narrowing is not said to close, or its mesh address is:\n%s", preview)
}
if got := lines["tcp/8080 hello-web"]; !strings.Contains(got, "declared by hello-web (from anywhere)") {
t.Errorf("a port open to everywhere is not said to stay:\n%s", preview)
}
}
// digestIn is the digest a converge preview printed.
func digestIn(t *testing.T, preview string) string {
t.Helper()
for _, line := range strings.Split(preview, "\n") {
if fields := strings.Fields(line); len(fields) == 2 && fields[0] == "preview" {
return fields[1]
}
}
t.Fatalf("the preview printed no digest:\n%s", preview)
return ""
}
// The flip acts on the preview the operator saw: it names that preview's digest, and it is refused
// when the digest is missing, when anything the preview says has changed since, or when the node's
// account of itself is too old to be the machine as it is.
func TestTheFlipActsOnlyOnThePreviewTheOperatorSaw(t *testing.T) {
open, sent := anAdoptedAnchor(t)
ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err)
}
reportsHolding(t, open, heldFile)
preview, err := converge(ctx, open, "anchor", false, "", "")
if err != nil {
t.Fatal(err)
}
saw := digestIn(t, preview)
if !strings.Contains(preview, "converge anchor --yes "+saw) {
t.Fatalf("the preview does not say how to act on it:\n%s", preview)
}
unchanged := func() {
t.Helper()
if n, _ := open.inventory.NodeByName(ctx, "anchor"); !n.Adopted || len(*sent) != 0 {
t.Fatal("a refused flip changed something")
}
}
if _, err := converge(ctx, open, "anchor", true, "", ""); err == nil ||
!strings.Contains(err.Error(), "--yes "+saw) {
t.Fatalf("a flip naming no preview was not refused: %v", err)
}
unchanged()
// Something new is reachable: the preview the operator saw is not what would happen.
reportsReaching(t, open, []link.Reach{
{Protocol: "tcp", Address: "0.0.0.0", Port: 22, By: "sshd"},
{Protocol: "tcp", Address: "0.0.0.0", Port: 8080, By: "hello-web", Published: true,
ContainerPort: 80},
{Protocol: "tcp", Address: "0.0.0.0", Port: 6000, By: "something-new"},
}, heldFile)
_, err = converge(ctx, open, "anchor", true, saw, "")
if err == nil || !strings.Contains(err.Error(), "has changed since preview "+saw) {
t.Fatalf("a flip on a changed preview was not refused: %v", err)
}
unchanged()
// An account older than the flip trusts is refused, whatever digest is named.
again, err := converge(ctx, open, "anchor", false, "", "")
if err != nil {
t.Fatal(err)
}
saved := reportFreshFor
reportFreshFor = time.Nanosecond
_, err = converge(ctx, open, "anchor", true, digestIn(t, again), "")
reportFreshFor = saved
if err == nil || !strings.Contains(err.Error(), "wait for its next report") {
t.Fatalf("a flip on an old account was not refused: %v", err)
}
unchanged()
if _, err := converge(ctx, open, "anchor", true, digestIn(t, again), ""); err != nil {
t.Fatalf("the flip on the preview just seen was refused: %v", err)
}
if n, _ := open.inventory.NodeByName(ctx, "anchor"); n.Adopted {
t.Fatal("the flip did not converge the node")
}
}
// The flip holds the node from its checks to its send, so a push composed meanwhile waits and is
// composed after it — never sent after it with the node still adopted. And the send inside the
// flip is not made to wait on the flip's own hold.
func TestTheFlipHoldsTheNodeWhileItSends(t *testing.T) {
open, _ := anAdoptedAnchor(t)
ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err)
}
reportsHolding(t, open, heldFile)
preview, err := converge(ctx, open, "anchor", false, "", "")
if err != nil {
t.Fatal(err)
}
var heldElsewhere, heldHere error
sendNodes = func(inner context.Context, open *stores, names []string) error {
// Another caller cannot hold the node while the flip sends it.
waiting, cancel := context.WithTimeout(ctx, 300*time.Millisecond)
defer cancel()
if release, err := open.inventory.HoldNodes(waiting, names); err == nil {
release()
heldElsewhere = errors.New("another caller held the node while the flip sent it")
}
// The flip's own send holds it without waiting on itself.
_, release, err := holdNodes(inner, open, names)
if err != nil {
heldHere = err
return err
}
release()
return nil
}
if _, err := converge(ctx, open, "anchor", true, digestIn(t, preview), ""); err != nil {
t.Fatal(err)
}
if heldElsewhere != nil || heldHere != nil {
t.Fatalf("%v %v", heldElsewhere, heldHere)
}
// And given back once it is done.
release, err := open.inventory.HoldNodes(ctx, []string{"anchor"})
if err != nil {
t.Fatal(err)
}
release()
}
// novox/hq ADR 0103: the preview names every kind of thing a module the flip takes holds as found,
// not only its files, and the digest changes when any of them does.
func TestThePreviewNamesEveryHeldKind(t *testing.T) {
open, _ := anAdoptedAnchor(t)
ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err)
}
since := time.Now()
held := []link.Held{heldFile,
{ID: "notes.data", Module: "notes", Kind: "directory", Target: "/var/lib/notes", Since: since},
{ID: "notes.daemon", Module: "notes", Kind: "service", Target: "notes.service", Since: since},
{ID: "notes.seed", Module: "notes", Kind: "archive", Target: "/srv/notes", Since: since},
{ID: "notes.worker", Module: "notes", Kind: "process", Target: "notes-worker", Since: since},
{ID: "notes.account", Module: "notes", Kind: "user", Target: "notes", Since: since},
}
reportsHolding(t, open, held...)
preview, err := converge(ctx, open, "anchor", false, "", "")
if err != nil {
t.Fatal(err)
}
for _, want := range []string{
"replacing the found directory /var/lib/notes (notes.data)",
"replacing the found service notes.service (notes.daemon)",
"replacing the found archive /srv/notes (notes.seed)",
"replacing the found process notes-worker (notes.worker)",
"replacing the found user notes (notes.account)",
} {
if !strings.Contains(preview, want) {
t.Errorf("the preview does not say %q:\n%s", want, preview)
}
}
reportsHolding(t, open, held[:len(held)-1]...)
fewer, err := converge(ctx, open, "anchor", false, "", "")
if err != nil {
t.Fatal(err)
}
if digestIn(t, fewer) == digestIn(t, preview) {
t.Fatal("the digest does not change with what is held")
}
}
// novox/hq ADR 0100: the flip acts on what the node said is reachable, so an account naming nothing
// is refused. Every machine that is up answers on ssh; nothing reported means the host's collectors
// did not, and flipping would close ports the preview never named.
func TestTheFlipIsRefusedOnAnAccountNamingNothingReachable(t *testing.T) {
open, sent := anAdoptedAnchor(t)
ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err)
}
// Only a loopback listener: nothing off the machine, which is the same silence.
reportsReaching(t, open, []link.Reach{
{Protocol: "tcp", Address: "127.0.0.1", Port: 15672, By: "mesh-broker"},
}, heldFile)
preview, err := converge(ctx, open, "anchor", false, "", "")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(preview, "this account looks partial") {
t.Errorf("the preview does not mark a partial account:\n%s", preview)
}
_, err = converge(ctx, open, "anchor", true, digestIn(t, preview), "")
if err == nil || !strings.Contains(err.Error(), "says nothing is reachable on it") {
t.Fatalf("the flip was not refused on an account naming nothing: %v", err)
}
if n, _ := open.inventory.NodeByName(ctx, "anchor"); !n.Adopted || len(*sent) != 0 {
t.Fatal("a refused flip changed something")
}
}
// An assignment cannot land between a preview and the flip that takes every module: assigning
// holds the node, so it waits for whatever is converging it.
func TestAssigningWaitsForWhateverIsConvergingTheNode(t *testing.T) {
open, _ := anAdoptedAnchor(t)
ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err)
}
reportsHolding(t, open, heldFile)
preview, err := converge(ctx, open, "anchor", false, "", "")
if err != nil {
t.Fatal(err)
}
saved, savedPoll := inventory.HoldWaitFor, inventory.HoldPoll
inventory.HoldWaitFor, inventory.HoldPoll = time.Second, 50*time.Millisecond
defer func() { inventory.HoldWaitFor, inventory.HoldPoll = saved, savedPoll }()
var whileFlipping error
sendNodes = func(context.Context, *stores, []string) error {
_, whileFlipping = assign(ctx, open, "anchor", "notes")
return nil
}
if _, err := converge(ctx, open, "anchor", true, digestIn(t, preview), ""); err != nil {
t.Fatal(err)
}
if !errors.Is(whileFlipping, inventory.ErrNodeBusy) {
t.Fatalf("an assignment landed while the node was being converged: %v", whileFlipping)
}
}
File diff suppressed because it is too large Load Diff
@@ -1,180 +0,0 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// **A refusal the bus said once is counted once** (novox/hq issue 402). The bus refused the controller
// one publish at 18:52:59 UTC on 2026-10-10 and never again, yet S9 looked at the one remembered
// advisory every half minute for the hour it is kept, and the condition said "observed 15 time(s), last
// 13s ago" with a refusal in its evidence every 30 seconds: two of us read it as a refusal going on and
// went looking for a missing grant. The condition counts what the bus said, with when it last said it,
// never how often the controller looked.
func TestARefusalSaidOnceIsCountedOnceHoweverOftenItIsLookedAt(t *testing.T) {
refused := time.Date(2026, 10, 10, 18, 52, 59, 0, time.UTC)
now := refused.Add(10 * time.Second)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
// The words the live condition bus.controller.refused carried on 2026-10-10.
said := "the bus refused the controller: nats: permissions violation: Permissions Violation for " +
`Publish to "mesh.seat.mesh-delivery.tool.times"`
advisory := link.Advisory{Kind: link.AdvisoryRefused, ID: "controller", Said: said,
First: refused, Last: refused, Count: 1}
look := func() conditions.Condition {
t.Helper()
f := &signalFacts{now: now, host: "novox", advisories: []link.Advisory{advisory}}
if err := k.Reconcile(t.Context(), "S9", watchAdvisories(f)); err != nil {
t.Fatal(err)
}
c, found, err := conditions.ReadOne(t.Context(), store, "bus.controller.refused")
if err != nil || !found {
t.Fatalf("bus.controller.refused: found %v, %v", found, err)
}
return c
}
var c conditions.Condition
for range 15 {
c = look()
now = now.Add(30 * time.Second)
}
if c.Observations != 1 || len(c.Evidence) != 1 {
t.Fatalf("one refusal, looked at 15 times, reads as observed %d time(s) with %d evidence lines: %+v",
c.Observations, len(c.Evidence), c.Evidence)
}
if !c.LastObserved.Equal(refused) || !c.Evidence[0].At.Equal(refused) {
t.Fatalf("last observed %s, evidence at %s: the refusal was at %s", c.LastObserved, c.Evidence[0].At, refused)
}
// The bus refuses it three times more between two looks: the condition counts four refusals, says
// the newest, and adds one line of evidence for the look that saw them.
again := now.Add(-5 * time.Second)
advisory.Last, advisory.Count = again, 4
c = look()
if c.Observations != 4 || len(c.Evidence) != 2 || !c.LastObserved.Equal(again) || !c.Evidence[0].At.Equal(again) {
t.Fatalf("four refusals read as observed %d time(s), last %s, evidence %+v", c.Observations, c.LastObserved, c.Evidence)
}
if !strings.Contains(c.Summary, "(4 times since 18:52 UTC)") {
t.Fatalf("summary %q", c.Summary)
}
now = now.Add(30 * time.Second)
if c = look(); c.Observations != 4 || len(c.Evidence) != 2 {
t.Fatalf("a look with nothing new counted: observed %d time(s), evidence %+v", c.Observations, c.Evidence)
}
}
// **A condition that looks at the present still counts its looks** (novox/hq issue 402 changes only
// what reads a record): a machine silent for three looks was observed three times, and says so.
func TestAConditionOfThePresentCountsItsLooks(t *testing.T) {
now := time.Date(2026, 10, 10, 19, 0, 0, 0, time.UTC)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
var c conditions.Condition
for range 3 {
var err error
if c, err = k.Observe(t.Context(), conditions.Observation{Scope: conditions.ScopeMachine, ID: "ace",
Kind: "silent", Machine: "ace", Severity: conditions.Warning, Summary: "ace has not been heard from",
Source: "S1"}); err != nil {
t.Fatal(err)
}
now = now.Add(30 * time.Second)
}
if c.Observations != 3 || len(c.Evidence) != 3 || !c.LastObserved.Equal(now.Add(-30*time.Second)) {
t.Fatalf("observed %d time(s), %d evidence lines, last %s", c.Observations, len(c.Evidence), c.LastObserved)
}
}
// **One key, one watcher** (novox/hq issue 440). A consumer's max-deliveries condition is said from
// DEAD_LETTERS while it holds a message the consumer gave up on (issue 330). A max-deliveries advisory
// without a token names the same key; read beside it, each look added an observation and a line of
// evidence through the dead-letter half, and the two summaries overwrote each other on every look. The
// dead-letter row alone says that key, and counts the messages given up on, never the looks.
func TestAHeldDeadLetterIsCountedByWhatWasGivenUpNotByTheLooks(t *testing.T) {
gaveUp := time.Date(2026, 10, 10, 21, 4, 0, 0, time.UTC)
now := gaveUp.Add(20 * time.Second)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
const key = "bus.EVENTS.media_sonarr.max-deliveries"
held := map[string]int{"EVENTS.media_sonarr": 1}
newest := map[string]time.Time{"EVENTS.media_sonarr": gaveUp}
advisory := link.Advisory{Kind: link.AdvisoryMaxDeliveries, ID: "EVENTS.media_sonarr", Stream: "EVENTS",
Consumer: "media_sonarr", Said: "sonarr on media handed message 7 over 5 times and gave up on it",
First: gaveUp, Last: gaveUp, Count: 1}
look := func() conditions.Condition {
t.Helper()
f := &signalFacts{now: now, host: "novox", advisories: []link.Advisory{advisory},
deadLetters: held, deadLettersNewest: newest}
if err := k.Reconcile(t.Context(), "S9", watchAdvisories(f)); err != nil {
t.Fatal(err)
}
c, found, err := conditions.ReadOne(t.Context(), store, key)
if err != nil || !found {
t.Fatalf("%s: found %v, %v", key, found, err)
}
if !strings.Contains(c.Summary, "dead-letters verb") {
t.Fatalf("the summary is not the dead-letter row's: %q", c.Summary)
}
return c
}
var c conditions.Condition
for range 5 {
c = look()
now = now.Add(30 * time.Second)
}
if c.Observations != 1 || len(c.Evidence) != 1 || !c.LastObserved.Equal(gaveUp) {
t.Fatalf("one message given up on, looked at five times, reads as observed %d time(s), last %s, "+
"with %d evidence lines: %+v", c.Observations, c.LastObserved, len(c.Evidence), c.Evidence)
}
// The consumer gives up on a second message between two looks: two given up on, one more line.
again := now.Add(-10 * time.Second)
held["EVENTS.media_sonarr"], newest["EVENTS.media_sonarr"] = 2, again
c = look()
if c.Observations != 2 || len(c.Evidence) != 2 || !c.LastObserved.Equal(again) {
t.Fatalf("two given up on read as observed %d time(s), last %s, evidence %+v", c.Observations,
c.LastObserved, c.Evidence)
}
now = now.Add(30 * time.Second)
if c = look(); c.Observations != 2 || len(c.Evidence) != 2 {
t.Fatalf("a look with nothing new counted: observed %d time(s), evidence %+v", c.Observations, c.Evidence)
}
}
// The advisory still says the max-deliveries it alone knows: a message given up on that could not be
// kept, under a key of its own (issue 330), which issue 440's change leaves as it was.
func TestAMessageThatCouldNotBeKeptIsStillSaidFromTheAdvisory(t *testing.T) {
at := time.Date(2026, 10, 10, 21, 4, 0, 0, time.UTC)
f := &signalFacts{now: at, host: "novox", advisories: []link.Advisory{{Kind: link.AdvisoryMaxDeliveries,
ID: "EVENTS.media_sonarr", Stream: "EVENTS", Consumer: "media_sonarr", Token: link.AdvisoryNotKept,
Said: "sonarr on media gave up on message 7, and it could not be kept", First: at, Last: at, Count: 1}}}
said := watchAdvisories(f)
if len(said) != 1 || said[0].Key() != "bus.EVENTS.media_sonarr.not-kept" {
t.Fatalf("%+v", said)
}
}
// A max-deliveries advisory without a token names a consumer's dead-letter key, which only DEAD_LETTERS
// says (novox/hq issues 330 and 440): with nothing held, the advisory alone raises nothing.
func TestAMaxDeliveriesAdvisoryAloneRaisesNothing(t *testing.T) {
at := time.Date(2026, 10, 10, 21, 4, 0, 0, time.UTC)
f := &signalFacts{now: at, host: "novox", advisories: []link.Advisory{{Kind: link.AdvisoryMaxDeliveries,
ID: "EVENTS.media_sonarr", Stream: "EVENTS", Consumer: "media_sonarr",
Said: "sonarr on media handed message 7 over 5 times and gave up on it", First: at, Last: at, Count: 1}}}
if said := watchAdvisories(f); len(said) != 0 {
t.Fatalf("said %+v", said)
}
}
-253
View File
@@ -1,253 +0,0 @@
package main
// The account agents run as (novox/hq ADR 0266).
//
// On the control node every agent session ran as the operator's account, which may become root without a
// password — so any agent there could become root without a person, and ADR 0259 §8 (an answer from the
// operator's phone authorises an act) rests on that being false where the router and its channels run. The
// decision: a node may name an account its agents run as, of their own and without sudo; the operator's
// account keeps its sudo.
//
// - **Named at the controller's terminal only** (`node agent-account`): not a verb, not a setting, so no
// agent can name itself another account. Empty is a real state: agents run as the operator there.
// - **Composed** as `${machine:agent-account}`, `${machine:agent-home}` and `${machine:agent-root}` for the
// agent's module, which declares the account with `root: never`, and as MESH_AGENT_ACCOUNT and
// MESH_AGENT_HOME for its tools (catalogue/machine_into_files.go, runtime.go).
// - **Judged by the machine itself.** The node-engine reads, on every look, whether an account declared
// `root: never` can become root without a person — uid 0, a group that grants root, a sudo rule, a
// secret of the mesh it may read — and says it as the declaring module's account verdict, marked
// Root "never". agentConfined reads that verdict; the self-check (probe DA) raises
// `agent-can-become-root` while it does not hold, and `node show` says it.
//
// A verdict not given is never a pass: an engine older than the judging, an account not yet declared, a
// statement that says nothing of it — each is "not judged", and fails.
import (
"context"
"fmt"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// kindAgentCanBecomeRoot is the condition raised while a machine's agent account can become root without
// a person, or is not judged (ADR 0266).
const kindAgentCanBecomeRoot = "agent-can-become-root"
// agentAccountProbe is the self-check's probe of it.
const agentAccountProbe = "DA"
// agentConfined says whether the agents of a machine that names an agent account are confined: the
// machine's newest statement holds a healthy account verdict, judged for root, on that account. named is
// false for a machine that names none — agents run as the operator account there, which this does not
// judge. why is said either way, in the mesh's words; err is a store that could not be read.
//
// The one judgement: `node show`, the self-check, and ADR 0259's router honouring a verified sender read
// it here.
// now is the judging clock, threaded so a caller judging several things at one instant judges them all at it.
func agentConfined(ctx context.Context, inv *inventory.Inventory, node string, now time.Time) (named, confined bool, why string, err error) {
n, err := inv.NodeByName(ctx, node)
if err != nil {
return false, false, "", err
}
if n.AgentAccount == "" {
return false, false, fmt.Sprintf("%s names no agent account: agents run as the operator account (%s)",
node, orNoneKnown(n.Account)), nil
}
h, had, err := inv.HealthOf(ctx, node)
if err != nil {
return true, false, "", err
}
confined, why = judgedConfined(n.AgentAccount, h, had, now)
return true, confined, why, nil
}
// verdictFreshFor is how old the statement holding the verdict may be, by this controller's clock. A
// node-engine states its health on every change and at least every five minutes (mesh-host's sayAnyway), so
// three statements missed is a node-engine stopped, or a machine away. **A stale verdict is not a pass**: an
// agent that stopped the node-engine must not leave "cannot become root" standing from before.
const verdictFreshFor = 15 * time.Minute
// judgedConfined is the judgement over one statement, without the store, at now.
func judgedConfined(agent string, h inventory.NodeHealth, had bool, now time.Time) (bool, string) {
if !had {
return false, fmt.Sprintf("the agent account %s is not judged: the machine's node-engine has stated "+
"nothing of what it runs", agent)
}
if age := now.Sub(h.HeardAt); age > verdictFreshFor {
return false, fmt.Sprintf("the agent account %s is not judged: the machine's newest statement was heard at "+
"%s, more than %d minutes ago, and a verdict that old is not a verdict on now", agent,
h.HeardAt.Local().Format("2006-01-02 15:04"), int(verdictFreshFor.Minutes()))
}
if h.Contract < link.RootContract {
return false, fmt.Sprintf("the agent account %s is not judged: the machine's node-engine is older than "+
"the judging of an account's root (its statement's contract is %d, the judging is %d)",
agent, h.Contract, link.RootContract)
}
var verdicts []inventory.ResourceHealth
for _, r := range h.Resources {
if r.Kind == link.KindAccount && r.Target == agent && r.Root == link.RootNever {
verdicts = append(verdicts, r)
}
}
if len(verdicts) == 0 {
return false, fmt.Sprintf("the agent account %s is not judged: the machine's newest statement holds no "+
"verdict on it — no module there declares it never to become root, or the declaration naming it "+
"has not been applied", agent)
}
sort.Slice(verdicts, func(i, j int) bool {
return verdicts[i].Module+verdicts[i].Resource < verdicts[j].Module+verdicts[j].Resource
})
for _, v := range verdicts {
switch v.State {
case link.StateHealthy:
case link.StateUnhealthy:
// The engine's own words, which start with link.ReasonRoot when it found a way to root.
return false, fmt.Sprintf("the agent account %s %s (said by %s's %s)", agent,
orNoneKnown(v.Reason), v.Module, v.Resource)
default:
return false, fmt.Sprintf("the agent account %s is not judged: %s (%s's %s, %s)", agent,
orNoneKnown(v.Reason), v.Module, v.Resource, v.State)
}
}
return true, fmt.Sprintf("the agent account %s cannot become root without a person (judged %s)", agent,
h.SaidAt.Local().Format("2006-01-02 15:04"))
}
// searchQuietFor is how long the controller lets an agent account's verdict wait for the node-engine's setuid
// search before that is itself the urgent condition: the longest a search is expected to take (link.RootSearchQuiet,
// the engine's own value; novox/hq issue 361 — the search runs to completion, with no bound of its own), counted
// from when this controller first saw it waiting, never from the engine's start. Quiet raises nothing; it never
// makes the agent account confined, which only a healthy verdict from a complete, fresh search does.
const searchQuietFor = link.RootSearchQuiet
// The kinds of an agent account's verdict, for the quiet a search earns.
const (
verdictOther = iota // anything else: a stale statement, an older engine, no verdict, another unknown
verdictPending // waiting for the search, and otherwise healthy
verdictComplete // judged: healthy, or a way to root found
)
// rootVerdictKind reads a statement for the agent account (novox/hq ADR 0266): pending when every verdict on it
// is healthy or not judged yet because the engine's setuid search runs, at least one of them that; complete when
// every verdict is healthy or one found a way to root. The engine's own "since" is not read: it starts again at
// every restart of the engine.
func rootVerdictKind(agent string, h inventory.NodeHealth, had bool, now time.Time) int {
if !had || now.Sub(h.HeardAt) > verdictFreshFor || h.Contract < link.RootContract {
return verdictOther
}
pending, any := false, false
for _, r := range h.Resources {
if r.Kind != link.KindAccount || r.Target != agent || r.Root != link.RootNever {
continue
}
any = true
switch {
case r.State == link.StateHealthy:
case r.State == link.StateUnhealthy:
return verdictComplete
case r.State == link.StateUnknown && strings.HasPrefix(r.Reason, link.ReasonRootPending):
pending = true
default:
return verdictOther
}
}
switch {
case !any:
return verdictOther
case pending:
return verdictPending
}
return verdictComplete
}
// searchStillRunning says the one thing keeping an agent account from being judged is the node-engine's setuid
// search, and that this controller first saw it waiting less than searchQuietFor ago — kept in the store, so a
// node-engine restarted in a loop does not keep it quiet. A complete verdict forgets when it began.
func searchStillRunning(ctx context.Context, inv *inventory.Inventory, node, agent string, h inventory.NodeHealth,
had bool, now time.Time) (bool, error) {
switch rootVerdictKind(agent, h, had, now) {
case verdictComplete:
return false, inv.RootSearchJudged(ctx, node)
case verdictPending:
since, err := inv.RootSearchPending(ctx, node, now)
if err != nil {
return false, err
}
return now.Sub(since) <= searchQuietFor, nil
}
return false, nil
}
// probeAgentAccounts is DA: every machine that names an agent account has it judged, on its node-engine's
// newest statement, unable to become root without a person (ADR 0266).
func probeAgentAccounts(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
inv := d.open.inventory
nodes, err := inv.Nodes(ctx)
if err != nil {
return nil, err
}
var out []conditions.Observation
for _, n := range nodes {
if n.AgentAccount == "" {
continue
}
h, had, err := inv.HealthOf(ctx, n.Name)
if err != nil {
return nil, err
}
now := time.Now()
quiet, err := searchStillRunning(ctx, inv, n.Name, n.AgentAccount, h, had, now)
if err != nil {
return nil, err
}
confined, why := judgedConfined(n.AgentAccount, h, had, now)
if confined {
continue
}
// Not judged yet only because the node-engine's search runs and no complete, fresh one judges: not the
// urgent condition after every restart. The agent is still not confined — ADR 0259's router reads
// agentConfined, not this — and `node show` still says not judged. Loud again once the search fails,
// waits past searchQuietFor, or the statement goes stale.
if quiet {
continue
}
out = append(out, conditions.Observation{Scope: conditions.ScopeMachine, ID: n.Name, Token: "agent-root",
Machine: n.Name, Severity: conditions.Urgent,
Summary: fmt.Sprintf("on %s, %s (ADR 0266): an agent there may become root without a person, and "+
"no answer from a channel authorises an act there (ADR 0259 §8)", n.Name, why),
Said: why})
}
return sortedFound(out), nil
}
// agentAccountLines is what `node show` says of the account agents run as.
func agentAccountLines(ctx context.Context, inv *inventory.Inventory, n inventory.Node) []string {
if n.AgentAccount == "" {
return []string{fmt.Sprintf(" agents run as the operator account (%s); no agent account is named",
orNoneKnown(n.Account))}
}
_, confined, why, err := agentConfined(ctx, inv, n.Name, time.Now())
if err != nil {
return []string{fmt.Sprintf(" agents run as %s (home %s); whether it can become root could NOT be read: %v",
n.AgentAccount, n.AgentHome(), err)}
}
verdict := "CAN become root, or is not judged: " + why
if confined {
verdict = why
}
return []string{fmt.Sprintf(" agents run as %s (home %s)", n.AgentAccount, n.AgentHome()),
" " + verdict}
}
// orNoneKnown is a value, or that none is known.
func orNoneKnown(s string) string {
if strings.TrimSpace(s) == "" {
return "none known"
}
return s
}
@@ -1,161 +0,0 @@
//go:build integration
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
snapshot "github.com/novox/mesh-controller/internal/facts"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-host/rootsearch"
)
func TestTheSelfCheckSaysAnAgentAccountThatCanBecomeRoot(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
for _, n := range []string{"anchor", "laptop"} {
if _, err := inv.NodeByName(ctx, n); err != nil {
if _, err := inv.AddNode(ctx, n); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAccount(ctx, n, "ops", ""); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAgentAccount(ctx, "anchor", "agent", ""); err != nil {
t.Fatal(err)
}
d := &doctor{open: open}
found, err := probeAgentAccounts(ctx, d)
if err != nil {
t.Fatal(err)
}
found = onlyMachine(found, "anchor")
if len(found) != 1 || found[0].Machine != "anchor" || found[0].Severity != conditions.Urgent ||
!strings.Contains(found[0].Said, "not judged") {
t.Fatalf("a named agent account with no verdict: %+v", found)
}
w := plainWordings[kindAgentCanBecomeRoot](conditions.Observation{Kind: kindAgentCanBecomeRoot, Machine: "anchor"})
if w.Headline == "" || w.Needs == "" || w.Resolved == "" {
t.Errorf("the condition has no plain words: %+v", w)
}
healthy := inventory.ResourceHealth{Module: "claude-code", Resource: "claude-code.agent-account",
Kind: link.KindAccount, Target: "agent", State: link.StateHealthy, Root: link.RootNever, Account: "agent"}
if _, err := inv.RecordHealth(ctx, inventory.NodeHealth{Node: "anchor", Contract: link.RootContract,
SaidAt: time.Now(), HeardAt: time.Now(), Resources: []inventory.ResourceHealth{healthy}}); err != nil {
t.Fatal(err)
}
if found, err = probeAgentAccounts(ctx, d); err != nil || len(onlyMachine(found, "anchor")) != 0 {
t.Fatalf("a judged agent account still fails: %+v %v", found, err)
}
if named, confined, why, err := agentConfined(ctx, inv, "anchor", time.Now()); err != nil || !named || !confined {
t.Fatalf("agentConfined on anchor: %v %v %q %v", named, confined, why, err)
}
if named, _, why, err := agentConfined(ctx, inv, "laptop", time.Now()); err != nil || named ||
!strings.Contains(why, "operator account") {
t.Fatalf("agentConfined on a machine naming none: %v %q %v", named, why, err)
}
}
func TestTheFactsCarryTheAgentAccountAsAPseudonym(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
if err := open.inventory.SetAccount(ctx, "anchor", "keeper", ""); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetAgentAccount(ctx, "anchor", "warden", ""); err != nil {
t.Fatal(err)
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
body, _ := f.Encode()
if strings.Contains(string(body), "warden") {
t.Error("the agent account's name is in the snapshot")
}
m, ok := f.Machine(snapshot.Pseudonym("machine", "anchor"))
if !ok || m.AgentAccount != snapshot.Pseudonym("account", "warden") || m.Account == m.AgentAccount {
t.Fatalf("the anchor's agent account reads as %q (operator %q)", m.AgentAccount, m.Account)
}
}
func TestASearchStillRunningAfterARestartIsNotUrgent(t *testing.T) {
if searchQuietFor != rootsearch.Quiet {
t.Fatalf("the quiet is %s and the node-engine's %s: they are one value", searchQuietFor, rootsearch.Quiet)
}
// A daily search that finishes within the quiet never leaves the account unjudged between two of them.
if rootsearch.FreshFor < rootsearch.Every+rootsearch.Quiet {
t.Fatalf("a complete search judges for %s, less than a day's search (%s) and the quiet (%s)",
rootsearch.FreshFor, rootsearch.Every, rootsearch.Quiet)
}
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if _, err := inv.NodeByName(ctx, "anchor"); err != nil {
if _, err := inv.AddNode(ctx, "anchor"); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAgentAccount(ctx, "anchor", "agent", ""); err != nil {
t.Fatal(err)
}
d := &doctor{open: open}
say := func(state, reason string) []conditions.Observation {
t.Helper()
// The engine's since is always now: it was just restarted.
v := inventory.ResourceHealth{Module: "claude-code", Resource: "claude-code.agent-account",
Kind: link.KindAccount, Target: "agent", State: state, Reason: reason, Root: link.RootNever,
Account: "agent", Since: time.Now()}
if _, err := inv.RecordHealth(ctx, inventory.NodeHealth{Node: "anchor", Contract: link.RootContract,
SaidAt: time.Now(), HeardAt: time.Now(), Resources: []inventory.ResourceHealth{v}}); err != nil {
t.Fatal(err)
}
found, err := probeAgentAccounts(ctx, d)
if err != nil {
t.Fatal(err)
}
return onlyMachine(found, "anchor")
}
running := link.ReasonRootPending + ": the search for setuid programs, started at 19:21, has not finished yet"
healthy := func() {
t.Helper()
if found := say(link.StateHealthy, ""); len(found) != 0 {
t.Fatalf("a healthy verdict raised: %+v", found)
}
}
if found := say(link.StateUnknown, running); len(found) != 0 {
t.Fatalf("a search first seen now was raised: %+v", found)
}
if _, confined, why, _ := agentConfined(ctx, inv, "anchor", time.Now()); confined || !strings.Contains(why, "not judged") {
t.Fatalf("an account whose search runs was read as confined: %q", why)
}
// The engine restarted again and again, each statement's own since fresh: the controller's clock runs on.
if _, err := inv.RootSearchPending(ctx, "anchor", time.Now().Add(-searchQuietFor-time.Minute)); err != nil {
t.Fatal(err)
}
healthy() // a complete verdict forgets when the waiting began …
if _, err := inv.RootSearchPending(ctx, "anchor", time.Now().Add(-searchQuietFor-time.Minute)); err != nil {
t.Fatal(err) // … and this restart loop began past the bound
}
if found := say(link.StateUnknown, running); len(found) != 1 || found[0].Severity != conditions.Urgent {
t.Fatalf("a search pending past the bound, by the controller's clock, was not urgent: %+v", found)
}
healthy()
incomplete := rootsearch.ReasonIncomplete + ": the search for setuid programs did not finish (last tried at " +
"19:23: timeout); it is tried again later"
if found := say(link.StateUnknown, incomplete); len(found) != 1 || found[0].Severity != conditions.Urgent {
t.Fatalf("a search that did not finish was not urgent: %+v", found)
}
if found := say(link.StateUnhealthy, link.ReasonRoot+": in the group docker; "+running); len(found) != 1 ||
found[0].Severity != conditions.Urgent {
t.Fatalf("a way to root found while the search runs was not urgent: %+v", found)
}
}
-134
View File
@@ -1,134 +0,0 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The agent account's judgement (novox/hq ADR 0266): confined only on a healthy account verdict judged for
// root, on the very account; every verdict not given — no statement, an older engine, no verdict on it, a
// verdict of unknown — is "not judged" and fails, never a pass.
func TestAnAgentAccountIsConfinedOnlyOnAHealthyVerdictJudgedForRoot(t *testing.T) {
at := time.Date(2026, 10, 8, 19, 21, 0, 0, time.UTC)
verdict := func(target, root, state, reason string) inventory.ResourceHealth {
return inventory.ResourceHealth{Module: "claude-code", Resource: "claude-code.agent-account",
Kind: link.KindAccount, Target: target, State: state, Reason: reason, Root: root, Account: target}
}
statement := func(contract int, rs ...inventory.ResourceHealth) inventory.NodeHealth {
return inventory.NodeHealth{Node: "anchor", Contract: contract, SaidAt: at, HeardAt: at, Resources: rs}
}
for _, c := range []struct {
name string
h inventory.NodeHealth
had bool
confined bool
says string
}{
{"judged and unable", statement(link.RootContract, verdict("agent", link.RootNever, link.StateHealthy, "")),
true, true, "cannot become root without a person"},
{"judged and able", statement(link.RootContract, verdict("agent", link.RootNever, link.StateUnhealthy,
link.ReasonRoot+": in the group docker, which grants root")), true, false, "in the group docker"},
{"a verdict of unknown", statement(link.RootContract, verdict("agent", link.RootNever, link.StateUnknown,
"sudo could not be read")), true, false, "not judged"},
{"no statement", inventory.NodeHealth{}, false, false, "not judged"},
{"an older engine", statement(link.ReadinessContract, verdict("agent", "", link.StateHealthy, "")),
true, false, "older than the judging"},
{"a verdict on groups only", statement(link.RootContract, verdict("agent", "", link.StateHealthy, "")),
true, false, "no verdict on it"},
{"a verdict on another account", statement(link.RootContract, verdict("ops", link.RootNever, link.StateHealthy, "")),
true, false, "no verdict on it"},
} {
confined, why := judgedConfined("agent", c.h, c.had, at.Add(time.Minute))
if confined != c.confined || !strings.Contains(why, c.says) {
t.Errorf("%s: confined %v, %q; want %v saying %q", c.name, confined, why, c.confined, c.says)
}
}
// A verdict heard longer ago than the bound is no verdict: an agent that stopped the node-engine must not
// leave "healthy" standing.
fresh := statement(link.RootContract, verdict("agent", link.RootNever, link.StateHealthy, ""))
if ok, _ := judgedConfined("agent", fresh, true, at.Add(verdictFreshFor)); !ok {
t.Error("a verdict exactly at the bound is still one")
}
if ok, why := judgedConfined("agent", fresh, true, at.Add(verdictFreshFor+time.Second)); ok ||
!strings.Contains(why, "not judged") {
t.Errorf("a stale healthy verdict passed: %q", why)
}
}
// DA raises an urgent condition, with plain words, on a machine whose agent account is not judged unable to
// become root; a machine that names none is not its to judge.
func TestTheAgentRootWordsArePlain(t *testing.T) {
w := plainWordings[kindAgentCanBecomeRoot](conditions.Observation{Kind: kindAgentCanBecomeRoot, Machine: "anchor"})
if why, ok := conditions.PlainWords(w, "anchor"); !ok {
t.Fatalf("not plain: %s: %+v", why, w)
}
}
func onlyMachine(obs []conditions.Observation, machine string) []conditions.Observation {
var out []conditions.Observation
for _, o := range obs {
if o.Machine == machine {
out = append(out, o)
}
}
return out
}
// The snapshot a merge check composes from carries the agent account as a pseudonym (novox/hq ADR 0266),
// so a change is judged against machines that name one, and the name never leaves.
// No verb runs a `node` command that sets something: through the generic `command` verb, `node account`,
// `node agent-account` and every other `node` subcommand but list and show are refused, naming the terminal.
func TestNoVerbSetsANodesAccounts(t *testing.T) {
for _, line := range []string{
"node agent-account novox --clear",
"node agent-account novox ops",
"node account novox agent",
"node account novox",
"node add intruder",
"node public-domain novox --clear",
"node",
"node frobnicate",
} {
argv, err := argvFor("command", map[string]any{"command": line})
if err == nil || !strings.Contains(err.Error(), "controller's terminal") ||
!strings.Contains(err.Error(), "ADR 0266") {
t.Errorf("%q ran as %v (%v); want a refusal naming the terminal", line, argv, err)
}
}
for _, line := range []string{"node show novox", "node list --json", "status --json"} {
if _, err := argvFor("command", map[string]any{"command": line}); err != nil {
t.Errorf("%q, a read, was refused: %v", line, err)
}
}
if err := terminalOnly([]string{"node", "account", "a", "b"}); err == nil {
t.Error("the refusal is not only the command verb's")
}
}
// Naming the agent account is the controller's terminal's alone: the `node` verb only shows.
func TestTheNodeVerbOnlyShows(t *testing.T) {
argv, err := argvFor("node", map[string]any{"node": "anchor"})
if err != nil || strings.Join(argv, " ") != "node show anchor" {
t.Fatalf("the node verb runs %v (%v)", argv, err)
}
for _, v := range catalogue.ControllerVerbs {
if strings.Contains(v.Name, "agent") {
t.Errorf("a verb %q may name the agent account", v.Name)
}
}
}
// Until a complete search for setuid programs judges — none since the node-engine's state was kept, or the last
// older than the engine lets one judge — the agent account is not judged, and the search runs to its end with no
// bound (novox/hq issue 361). DA does not raise that as urgent within searchQuietFor, counted from when this
// controller first saw it waiting — never from the engine's own "since", which a restart resets, so an engine
// restarted in a loop does not keep it quiet. The account is still not confined, and `node show` still says not
// judged; a search that failed, or a way to root found, is urgent at once.
-186
View File
@@ -1,186 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"net/http"
"strings"
"time"
)
// The command API: what the mesh can be asked to do, over a network.
//
// **An adapter and nothing else** (novox/hq ADR 0035). Every route here calls the same function
// the command line calls, so a refusal is the same refusal in the same words. Nothing is decided
// in this file — the moment it validates something the command line does not, the mesh has two
// answers to one question.
//
// **It refuses everything unless it was told how to know who is asking.** The board is published
// on a public name, and this is what stands behind it: an unauthenticated command surface reachable
// from the internet is authority over the mesh handed to whoever finds it. So there is no
// permissive default and no flag that removes the check — a mesh that has not been told how to
// authenticate serves nothing, loudly.
//
// The intended authenticator is an OAuth2 provider (ADR 0035), which is an ordinary module. Until
// one is configured this refuses, which is the correct behaviour rather than a placeholder: a
// surface that worked without authentication would be one somebody left running.
func apiCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("api", flag.ContinueOnError)
listen := set.String("listen", "127.0.0.1:8081", "where to serve it")
issuer := set.String("issuer", "",
"the OAuth2 issuer whose tokens this accepts; without it, nothing is served")
if _, err := parseAround(set, args); err != nil {
return err
}
if strings.TrimSpace(*issuer) == "" {
// Refused at start rather than per request, so it is discovered by whoever ran it rather
// than by whoever finds it.
return errors.New(
"--issuer is not set, and this serves commands rather than pages: it will not run " +
"without being told whose tokens to believe. An OAuth2 provider is an ordinary " +
"module (novox/hq ADR 0035)")
}
server := &http.Server{
Addr: *listen,
ReadHeaderTimeout: 10 * time.Second,
Handler: commands(mustAuthenticate(*issuer)),
}
fmt.Printf("the command API is on http://%s\n", *listen)
fmt.Printf(" it accepts tokens from %s and refuses everything else\n", *issuer)
fmt.Printf(" every route calls what the command line calls\n")
go func() {
<-ctx.Done()
closing, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
_ = server.Shutdown(closing)
}()
if err := server.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
return err
}
return nil
}
// Authenticator says whether a request may act, and as whom.
//
// An interface so the check can be driven by a test without an identity provider, and so the one
// real implementation is the only thing that has to be right.
type Authenticator interface {
// Who returns the subject a request is acting as, or an error naming why it may not.
Who(r *http.Request) (string, error)
}
// mustAuthenticate is the real one: a bearer token from the configured issuer.
//
// **Not yet verifying the signature**, and it says so rather than pretending. Verification needs
// the issuer's keys, which needs an identity provider to exist — so this refuses every request
// until that is built, which is the same answer as having no API at all and is honest about why.
func mustAuthenticate(issuer string) Authenticator { return notYet{issuer: issuer} }
type notYet struct{ issuer string }
func (n notYet) Who(*http.Request) (string, error) {
return "", fmt.Errorf(
"this mesh has no way to verify a token from %s yet: the identity provider is a module "+
"and none is running. Use the command line, which authenticates through nothing "+
"because it is already behind the machine's own login", n.issuer)
}
// commands is the routing, separate so a test can drive it without a listener.
func commands(who Authenticator) http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("POST /assign", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
return assign(ctx, open, in.Node, splitModules(in.Module)...)
}))
mux.HandleFunc("POST /unassign", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
return unassign(ctx, open, in.Node, splitModules(in.Module)...)
}))
// Adoption (novox/hq ADR 0100): the same acts as `take`, `converge` and `adopt`.
mux.HandleFunc("POST /take", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
return take(ctx, open, in.Node, in.Module, takeOptions{Yes: in.Yes, Digest: in.Digest})
}))
mux.HandleFunc("POST /converge", acting(who, false, func(ctx context.Context, open *stores, in request) (string, error) {
return converge(ctx, open, in.Node, in.Yes, in.Digest, in.Filter)
}))
mux.HandleFunc("POST /adopt", acting(who, false, func(ctx context.Context, open *stores, in request) (string, error) {
return adopt(ctx, open, in.Node)
}))
// Anything else is said plainly, because a command surface answering 404 to a verb somebody
// expected is indistinguishable from one that is down.
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
refuse(w, http.StatusNotFound, fmt.Errorf(
"%s %s is not something this mesh can be asked; it accepts POST /assign, "+
"POST /unassign, POST /take, POST /converge and POST /adopt", r.Method, r.URL.Path))
})
return mux
}
type request struct {
Node string `json:"node"`
Module string `json:"module"`
// Yes, Digest and Filter are converge's and take's: do it rather than preview it, the digest
// of the preview it acts on, and (converge) which module loads the mesh's filter.
Yes bool `json:"yes,omitempty"`
Digest string `json:"digest,omitempty"`
Filter string `json:"filter,omitempty"`
}
// acting is the shape every route shares: authenticate, read, act, answer.
func acting(
who Authenticator,
needsModule bool,
do func(context.Context, *stores, request) (string, error),
) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if _, err := who.Who(r); err != nil {
refuse(w, http.StatusUnauthorized, err)
return
}
var in request
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
refuse(w, http.StatusBadRequest, fmt.Errorf("this is not a request this understands: %w", err))
return
}
if in.Node == "" || (needsModule && in.Module == "") {
if needsModule {
refuse(w, http.StatusBadRequest, errors.New(`both "node" and "module" are needed`))
} else {
refuse(w, http.StatusBadRequest, errors.New(`"node" is needed`))
}
return
}
open, err := openStores(r.Context())
if err != nil {
refuse(w, http.StatusServiceUnavailable, err)
return
}
defer open.Close()
said, err := do(r.Context(), open, in)
if err != nil {
// **The refusal the command line would have given, unchanged.** Carrying `said` with
// it matters: an assignment that was kept and still does not resolve is two facts,
// and dropping either makes the answer wrong.
answer(w, http.StatusConflict, map[string]any{"said": said, "refused": err.Error()})
return
}
answer(w, http.StatusOK, map[string]any{"said": said})
}
}
func answer(w http.ResponseWriter, status int, body map[string]any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
_ = json.NewEncoder(w).Encode(body)
}
func refuse(w http.ResponseWriter, status int, err error) {
answer(w, status, map[string]any{"refused": err.Error()})
}
-79
View File
@@ -1,79 +0,0 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
type letIn struct{}
func (letIn) Who(*http.Request) (string, error) { return "somebody", nil }
func asking(t *testing.T, who Authenticator, method, path, body string) *httptest.ResponseRecorder {
t.Helper()
recorded := httptest.NewRecorder()
commands(who).ServeHTTP(recorded, httptest.NewRequest(method, path, strings.NewReader(body)))
return recorded
}
// **Nothing is served to somebody the mesh cannot identify**, and that is the default rather than
// a setting. The board this stands behind is published on a public name (novox/hq 11-a-board), so
// an unauthenticated command surface is authority over the mesh handed to whoever finds it.
func TestAnUnauthenticatedRequestIsRefused(t *testing.T) {
got := asking(t, mustAuthenticate("https://identity.example/realms/mesh"),
"POST", "/assign", `{"node":"anchor","module":"umami"}`)
if got.Code != http.StatusUnauthorized {
t.Fatalf("an unidentified caller got %d", got.Code)
}
// And it says what to do instead, because the answer is not "give up".
if !strings.Contains(got.Body.String(), "command line") {
t.Errorf("the refusal does not say what still works: %s", got.Body.String())
}
}
// The API refuses to start at all without being told whose tokens to believe.
//
// At start rather than per request, so it is found by whoever ran it rather than by whoever
// finds it.
func TestTheApiWillNotRunWithoutAnIssuer(t *testing.T) {
err := apiCommand(context.Background(), []string{})
if err == nil {
t.Fatal("it served commands without being told who may give them")
}
if !strings.Contains(err.Error(), "issuer") {
t.Errorf("the refusal does not name what is missing: %v", err)
}
}
// A request missing what it acts on is refused before anything is opened.
func TestARequestThatNamesNothingIsRefused(t *testing.T) {
// **The exact refusal, not merely "not accepted".** Asserting non-200 passes even when the
// request got as far as opening a store and failing there, which proves nothing about whether
// anything was checked — that is what the first version of this test did.
for _, body := range []string{`{}`, `{"node":"anchor"}`, `{"module":"umami"}`, `not json`} {
got := asking(t, letIn{}, "POST", "/assign", body)
if got.Code != http.StatusBadRequest {
t.Errorf("%s got %d, and a request naming nothing is a bad request", body, got.Code)
}
}
}
// A verb nobody implemented is said plainly. A command surface answering 404 to something
// somebody expected is indistinguishable from one that is down.
func TestAnUnknownRouteSaysWhatIsAccepted(t *testing.T) {
got := asking(t, letIn{}, "POST", "/rotate", `{}`)
if got.Code != http.StatusNotFound {
t.Fatalf("got %d", got.Code)
}
var said map[string]any
_ = json.Unmarshal(got.Body.Bytes(), &said)
if !strings.Contains(said["refused"].(string), "/assign") {
t.Errorf("it does not say what it does accept: %v", said)
}
}
-164
View File
@@ -1,164 +0,0 @@
package main
import (
"fmt"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// kindApplyFailing is S21's kind: a machine whose declarations have been refused, or applied only in part,
// for longer than one apply interval (novox/hq issue 233).
const kindApplyFailing = "apply-failing"
// applyEvery is how often a node-engine applies what it was last told when nothing new arrives: its
// ReconcileEvery. A failure still there after one more of them is not one the next apply clears.
const applyEvery = 5 * time.Minute
// packageManagerKey is how a node-engine names a package manager that does not answer among what failed
// (mesh-host's probe.go): "the package manager (pacman)".
const packageManagerKey = "the package manager ("
// watchApplyFailing is S21: **a machine that does not apply what it was sent is said by the mesh, not only
// to someone who asks for `status`** (novox/hq issue 233, issue 187's family). A workstation refused every
// declaration for an hour and a half because its package manager did not answer, and the mesh's only word
// on it was a line in `status`. Raised once the machine's first report of this failure is older than one apply
// interval **while the machine is heard** — judged by time, not by how many reports said it: a node-engine
// says a re-apply only when how it went changed, so the same failure is one report however long it lasts.
// Names the refusal's or the failure's words; cleared by the clean apply the node-engine then says. Not while
// the machine is silent, asleep or not known: S1 says silent, and asleep is not lost.
func watchApplyFailing(f *signalFacts) []conditions.Observation {
heard := map[string]bool{}
for _, m := range f.machines {
if !m.asleep() && !m.lastHeard.IsZero() && f.now.Sub(m.lastHeard) <= heartbeatBound(m.every) {
heard[m.name] = true
}
}
var out []conditions.Observation
said := map[string]bool{}
for _, d := range f.doing {
if !d.Wrong() || d.Since == nil || f.now.Sub(*d.Since) <= applyEvery || !heard[d.Node] {
continue
}
out = append(out, applyFailing(d, f.now))
said[d.Node] = true
}
// **A module's files the provisioner failed to place are a failed apply too** (novox/hq ADR 0362 §3): its client's
// newest word, judged by the same bound — the client says a failure once, and fills again on every change —
// on a machine whose node-engine applied what it was sent.
failed := map[string][]inventory.ProvReport{}
for _, r := range f.provFailed {
if r.Status != inventory.ProvFailed || f.now.Sub(r.At) <= applyEvery || !heard[r.Node] || said[r.Node] {
continue
}
failed[r.Node] = append(failed[r.Node], r)
}
for _, node := range sortedNames(failed) {
out = append(out, provisioningFailing(node, failed[node], f.now))
}
return out
}
// provisioningFailing is one machine's S21 condition for the modules whose files its provisioner client failed to
// place.
func provisioningFailing(m string, failed []inventory.ProvReport, now time.Time) conditions.Observation {
sort.Slice(failed, func(a, b int) bool { return failed[a].Module < failed[b].Module })
var names, words []string
since := failed[0].At
for _, r := range failed {
names = append(names, r.Module)
words = append(words, r.Module+": "+oneLine(orNoReason(r.Reason)))
if r.At.Before(since) {
since = r.At
}
}
o := conditions.Observation{Scope: conditions.ScopeMachine, ID: m, Kind: kindApplyFailing, Machine: m,
Severity: conditions.Warning, Said: strings.Join(words, "; "),
Resolved: m + " applies what it is sent again"}
o.Summary = fmt.Sprintf("%s's provisioner has failed to place the files of %s since %s (bound %s)", m,
strings.Join(names, ", "), since.UTC().Format("2006-01-02 15:04 MST"), ago(applyEvery))
o.Headline = m + " is not applying its update"
o.Explanation = fmt.Sprintf("The provisioner on %s has not been able to write the files of %s for %s. The "+
"details say why.", m, strings.Join(names, ", "), humanDuration(now.Sub(since)))
return o
}
func orNoReason(s string) string {
if s == "" {
return "no reason given"
}
return s
}
// applyFailing is one machine's S21 condition.
func applyFailing(d inventory.Doing, now time.Time) conditions.Observation {
m := d.Node
since := d.Since.UTC().Format("2006-01-02 15:04 MST")
o := conditions.Observation{Scope: conditions.ScopeMachine, ID: m, Kind: kindApplyFailing, Machine: m,
Severity: conditions.Warning, Said: doingWords(d),
Resolved: m + " applies what it is sent again"}
how := "refused its declaration whole"
if d.Outcome != inventory.OutcomeRefused {
how = "applied its declaration only in part"
}
o.Summary = fmt.Sprintf("%s has %s since %s (bound %s): %s", m, how, since, ago(applyEvery), failedNames(d))
o.Headline = m + " is not applying its update"
o.Explanation = fmt.Sprintf("%s has %s for %s, and trying again has not helped. The details say what fails.",
m, plainHow(d), humanDuration(now.Sub(*d.Since)))
if tool := unansweringTool(d); tool != "" {
o.Summary = fmt.Sprintf("%s: %s does not answer, so no package reaches it; everything else it was sent "+
"is applied, since %s (bound %s)", m, tool, since, ago(applyEvery))
o.Headline = fmt.Sprintf("%s cannot install packages", m)
o.Explanation = fmt.Sprintf("The package manager on %s (%s) has not answered for %s, so no package reaches "+
"it. Everything else is still applied, so an update that repairs it can land; if none does, it needs a "+
"look on the machine.", m, tool, humanDuration(now.Sub(*d.Since)))
}
return o
}
// plainHow is how a machine failed, as the operator reads it.
func plainHow(d inventory.Doing) string {
if d.Outcome == inventory.OutcomeRefused {
return "refused every update"
}
return "applied its updates only in part"
}
// unansweringTool is the package manager a machine said does not answer, or "".
func unansweringTool(d inventory.Doing) string {
for _, r := range d.Failed {
if rest, ok := strings.CutPrefix(r.ID, packageManagerKey); ok {
return strings.TrimSuffix(rest, ")")
}
}
return ""
}
// failedNames is what failed, by name: the refusal, or the failed resources' ids.
func failedNames(d inventory.Doing) string {
if d.Outcome == inventory.OutcomeRefused {
return "refused"
}
ids := make([]string, 0, len(d.Failed))
for _, r := range d.Failed {
ids = append(ids, r.ID)
}
sort.Strings(ids)
return "failed " + strings.Join(ids, ", ")
}
// doingWords is the machine's own words on what failed: the evidence, kept inside the mesh.
func doingWords(d inventory.Doing) string {
if d.Outcome == inventory.OutcomeRefused {
return "refused: " + oneLine(d.Refused)
}
var parts []string
for _, r := range d.Failed {
parts = append(parts, r.ID+": "+oneLine(r.Error))
}
sort.Strings(parts)
return strings.Join(parts, "; ")
}
-123
View File
@@ -1,123 +0,0 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// unansweringPackageManager is anchor's last report as the workstation's read on 2026-10-04, once its
// node-engine applies the rest (novox/hq issue 233): the package failed and the package manager did not answer,
// the same failure on times reports since for before now.
func unansweringPackageManager(now time.Time, forHowLong time.Duration, times int) inventory.Doing {
since := now.Add(-forHowLong)
return inventory.Doing{Node: "anchor", Outcome: inventory.OutcomeFailed, At: now.Add(-time.Second),
Since: &since, Times: times, Applied: 41, Failed: []inventory.FailedResource{
{ID: "apply", Error: `applying "pacman.base": nothing can be said about "base": this is the arch host and ` +
"pacman does not answer here. Either this machine is not Arch, or its package database is broken: " +
"pacman exited 1: error: config file /etc/pacman.conf could not be read: No such file or directory"},
{ID: "the package manager (pacman)", Error: "pacman does not answer here, so every package in this " +
"declaration fails and the rest of it is applied (novox/hq issue 233): pacman exited 1: error: config " +
"file /etc/pacman.conf could not be read: No such file or directory"}}}
}
// **A machine whose package manager does not answer is raised by the mesh itself, naming the tool and the
// probe's words** (novox/hq issue 233): the words in the evidence, the path kept out of the summary.
func TestAPackageManagerThatDoesNotAnswerIsRaisedNamingItsWords(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
f := calm(now)
f.doing = []inventory.Doing{unansweringPackageManager(now, 90*time.Minute, 18)}
got := watchApplyFailing(f)
if len(got) != 1 {
t.Fatalf("raised %d conditions, want one: %+v", len(got), got)
}
o := got[0]
if o.Key() != "machine.anchor.apply-failing" || o.Machine != "anchor" {
t.Errorf("the condition is not the machine's: %s", o.Key())
}
if !strings.Contains(o.Summary, "pacman does not answer") || !strings.Contains(o.Summary, "since 2026-10-04 12:30") {
t.Errorf("the summary does not name the tool and since when: %s", o.Summary)
}
if strings.Contains(o.Summary, "/etc/") {
t.Errorf("the summary carries a path, which belongs in the evidence: %s", o.Summary)
}
if !strings.Contains(o.Said, "could not be read") {
t.Errorf("the evidence does not carry the probe's words: %s", o.Said)
}
if !strings.Contains(o.Headline, "cannot install packages") || o.Explanation == "" || o.Resolved == "" {
t.Errorf("the plain words do not say it: %+v", o)
}
}
// **A whole refusal is raised too**, with the node-engine's words: what the workstation's report said before the
// node-engine applied the rest.
func TestARefusalThatGoesOnIsRaised(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
since := now.Add(-time.Hour)
f := calm(now)
f.doing = []inventory.Doing{{Node: "anchor", Outcome: inventory.OutcomeRefused, Since: &since, Times: 12,
Refused: "this is the arch host and pacman does not answer here", At: now.Add(-time.Minute)}}
got := watchApplyFailing(f)
if len(got) != 1 || !strings.Contains(got[0].Summary, "refused its declaration whole") ||
!strings.Contains(got[0].Said, "pacman does not answer") {
t.Fatalf("a refusal that went on for an hour was not raised with its words: %+v", got)
}
}
// **Judged by time while the machine is heard, not by how many reports said it** (review of novox/hq issue 233):
// a node-engine says the same failure once, so one report an apply interval old is a fault that goes on; one
// younger is not yet; and a silent or sleeping machine is S1's.
func TestOneReportOlderThanAnApplyIsRaisedAndAQuietMachineIsNot(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
f := calm(now)
f.doing = []inventory.Doing{unansweringPackageManager(now, 6*time.Minute, 1)}
if got := watchApplyFailing(f); len(got) != 1 {
t.Errorf("one report of a failure six minutes old was not raised: %+v", got)
}
f.doing = []inventory.Doing{unansweringPackageManager(now, 4*time.Minute, 1)}
if got := watchApplyFailing(f); len(got) != 0 {
t.Errorf("a failure four minutes old was raised before one apply could clear it: %+v", got)
}
f.doing = []inventory.Doing{unansweringPackageManager(now, time.Hour, 1)}
f.machines[0].lastHeard = now.Add(-time.Hour)
if got := watchApplyFailing(f); len(got) != 0 {
t.Errorf("a silent machine was raised for its last report: %+v", got)
}
f = calm(now)
f.doing = []inventory.Doing{unansweringPackageManager(now, time.Hour, 1)}
f.doing[0].Node = "a-machine-the-mesh-does-not-know"
if got := watchApplyFailing(f); len(got) != 0 {
t.Errorf("a machine not heard of was raised: %+v", got)
}
}
// **A failed re-apply followed by a clean one clears S21** (review of novox/hq issue 233): the node-engine says
// the clean re-apply, the store keeps it as applied with no failure since, and the condition the failure raised
// is cleared through the keeper on the next tick.
func TestACleanReapplyAfterAFailedOneClearsTheCondition(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
defer k.Close(context.Background())
w := &watchdogs{keeper: k, started: now.Add(-time.Hour)}
failing := calm(now)
failing.doing = []inventory.Doing{unansweringPackageManager(now, 20*time.Minute, 1)}
w.see(t.Context(), failing)
open, err := k.Open(t.Context())
if err != nil || len(open) != 1 || open[0].Kind != kindApplyFailing {
t.Fatalf("the failed re-apply raised %+v (%v), want apply-failing", open, err)
}
clean := calm(now.Add(5 * time.Minute))
clean.doing = nil // the clean re-apply recorded: NotDoingWhatTheyWereTold no longer lists the machine
w.see(t.Context(), clean)
if open, _ := k.Open(t.Context()); len(open) != 0 {
t.Fatalf("the condition stayed open after the clean re-apply: %+v", open)
}
}
-62
View File
@@ -1,62 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"time"
"github.com/novox/mesh-controller/internal/link"
)
// ask calls one of a module's tools, through the control plane's own broker connection.
//
// A module serves tools under an account scoped to exactly that (novox/hq ADR 0047), and nothing
// else in the mesh held an account that could ask one — not an operator at a terminal, not an agent
// acting for one (novox/hq 04-ISSUES/049). The control plane does, so it is the way in: one
// process, one connection, one place a question can be seen to have been asked (ADR 0095).
func askCommand(ctx context.Context, args []string) error {
positionals, flags := split(args)
set := flag.NewFlagSet("ask", flag.ContinueOnError)
wait := set.Duration("wait", 60*time.Second, "how long to wait for the module's answer")
if err := set.Parse(flags); err != nil {
return err
}
if len(positionals) < 2 || len(positionals) > 3 {
return errors.New("ask <module> <tool> [json arguments] [--wait 60s]")
}
module, tool := positionals[0], positionals[1]
var arguments json.RawMessage
if len(positionals) == 3 {
if !json.Valid([]byte(positionals[2])) {
return fmt.Errorf("the arguments are not JSON: %s", positionals[2])
}
arguments = json.RawMessage(positionals[2])
}
server, err := connectLink(ctx, nil, nil, nil)
if err != nil {
return err
}
defer server.Close()
answer, err := link.Ask(ctx, server.Bus(), module, tool, arguments, *wait)
if err != nil {
return err
}
// The answer as the module gave it, to standard output, for a person or a program. A tool
// that answered with an error has still answered: printed the same way, and the exit status
// says which.
body, err := json.Marshal(answer)
if err != nil {
return err
}
fmt.Fprintln(os.Stdout, string(body))
if answer.Error != "" {
return fmt.Errorf("%s.%s answered with an error: %s", module, tool, answer.Error)
}
return nil
}
-946
View File
@@ -1,946 +0,0 @@
package main
// The controller asks, and acts on the operator's warrant (novox/hq ADR 0259 §6). It holds no channel, no
// identity and no factor: it asks the router like any other module, and performs the answer chosen with its
// own grant.
//
// - **For every open, unsilenced condition that needs the operator and names its answers**, one ask is
// published on the `operator-channel` seat under the controller's own name: the condition's words, its
// actions as options at their levels (Silence acknowledges; Release, Stop, Start and Restart approve),
// answered by the operator, expiring after a day (a week when every option only acknowledges). A
// condition that clears, is silenced, or changes its answers has its ask cancelled; an ask that expired
// unanswered is asked again while the condition lasts; one the router refused is asked again after a wait
// that grows with each refusal in a row, at most half an hour, so a refusal is never believed longer. Each ask is kept in the controller's bucket
// `asked`, so a restart neither asks twice nor forgets.
// - **On a warrant**, heard on the seat's event under the controller's own name (which only the router may
// say), the controller acts once per ask: only for an ask it holds, only for the option it offered at
// that option's level, and only while the condition is still open. It performs the action as itself —
// a silence through its own conditions, any other through the verb the action names — with the warrant's
// words as its why, and records it in the hand-act log as the operator's decision, naming the channel,
// the ask and the proofs. An ask that ended without a choice is recorded and nothing is done.
// - **A warrant it missed** while away is read from the router's record of its asks, under its own name.
import (
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// The asker's name on the seat: the controller's module.
const askerName = broker.ControllerSeat
// How long an ask lasts: a day when an answer approves, a week when every answer only acknowledges.
const (
// askApproveFor is a day less a margin, so an ask is never refused at the router for lasting a day and
// a moment (the SDK's bound is a day).
askApproveFor = 24*time.Hour - 10*time.Minute
askAcknowledgeFor = 7 * 24 * time.Hour
// askEvery is how often what is open is asked about again, beside every change.
askEvery = time.Minute
// askCatchUpAfter is how old an open ask is before the router's record of it is read: a warrant heard
// on the event needs no reading.
askCatchUpAfter = 2 * time.Minute
// askAgainAfterAnswer is how long a condition the operator answered is not asked about again with the
// same answers: what was chosen takes a while to clear it, and asking again at once would ask twice.
askAgainAfterAnswer = time.Hour
// askMostOpen is how many asks the controller holds open at once (the router refuses a fourth): the
// most urgent conditions first, then the oldest.
askMostOpen = asks.MostOpen
// askRefusedRetryMost is the longest a refusal is believed without asking again (novox/hq issue 369): the
// router refused while its channels had not yet said they could send, the channels could send twenty
// minutes later, and the controller repeated that refusal for eleven hours. A refused ask is asked again
// after askEvery, then twice as long after each refusal in a row, never longer than this — and at once when
// the channels change.
askRefusedRetryMost = 30 * time.Minute
// askRefusedCounted is how far back refusals in a row are counted for that wait.
askRefusedCounted = 6 * time.Hour
// askVerdictWait is how long an ask made again after a refusal waits for the router's word before it is
// taken as taken: the router refuses an ask as it reads it, so a refusal comes within seconds. Meanwhile
// the condition saying it was not delivered stands as it was, neither cleared nor raised again.
askVerdictWait = 2 * time.Minute
)
// refusedRetryAfter is how long a part refused n times in a row waits before it is asked again.
func refusedRetryAfter(n int) time.Duration {
wait := askEvery
for i := 1; i < n && wait < askRefusedRetryMost; i++ {
wait *= 2
}
return min(wait, askRefusedRetryMost)
}
// What became of an ask, as the controller keeps it.
const (
askOpen = "open"
askCancelled = "cancelled"
)
// asked is one ask the controller made, as it keeps it.
type asked struct {
ID string `json:"id"`
Condition string `json:"condition"`
// Channels is what the channels were when it was asked (asker.channels): an ask the router refused is asked
// again at once when the condition's answers or the channels change, and otherwise after a wait that grows
// with each refusal in a row (refusedRetryAfter, novox/hq issue 369).
Channels string `json:"channels,omitempty"`
Ask asks.Ask `json:"ask"`
Actions []conditions.Action `json:"actions"`
// Options are the actions by option id.
Options map[string]int `json:"options"`
State string `json:"state"`
Opened time.Time `json:"opened"`
Ended time.Time `json:"ended,omitempty"`
Warrant *asks.Warrant `json:"warrant,omitempty"`
// Acted is what the controller did on the warrant: empty before it did anything, "acting" while it acts,
// then "done", "failed: …" or "nothing: …". Anything but empty is never acted on again.
Acted string `json:"acted,omitempty"`
// Part is which ask of its condition this is (askPart): empty for the one that carries the condition's
// answers, or the authorising ones where it has both; "acknowledge" for its acknowledging answers asked
// apart (the review of 2026-10-09, M1).
Part string `json:"part,omitempty"`
// Rehearsal is an ask started at the controller's terminal (rehearse.go): about no condition, its answers
// perform nothing, and the reconciling of conditions leaves it alone.
Rehearsal bool `json:"rehearsal,omitempty"`
// Proposal is a settings layer proposed through a verb (proposals.go, novox/hq ADR 0277): about no
// condition, set on Approve by the serving controller itself, and left alone by the reconciling of conditions.
Proposal *settingsProposal `json:"proposal,omitempty"`
}
// ofACondition says an ask is one of a condition's: not a rehearsal, not a proposal.
func (r asked) ofACondition() bool { return !r.Rehearsal && r.Proposal == nil }
// partKey is an ask's place among what is asked: its condition and its part.
func partKey(condition, part string) string { return condition + "#" + part }
// partAcknowledge is the part of a condition asked apart for its acknowledging answers.
const partAcknowledge = "acknowledge"
// askPart is one ask a condition is asked with: its part, what it is about, and its answers.
type askPart struct {
name string
about string
actions []conditions.Action
}
// levelOf is an action's level as an option offers it: one that says none is never taken for less than
// approve.
func levelOf(act conditions.Action) asks.Level {
if act.Level == "" {
return asks.Approve
}
return asks.Level(act.Level)
}
// partsOf is the asks a condition is asked with (the review of 2026-10-09, M1): one, when its answers are all
// of one kind; else its authorising answers (Release, Stop, Restart) in one ask, about the condition, and its
// acknowledging ones (Silence) in another. **An acknowledgement never shares an ask with an approval**: a
// channel that only acknowledges would otherwise answer the ask, and end the approval with it.
func partsOf(c conditions.Condition) []askPart {
var ack, auth []conditions.Action
for _, act := range c.Actions {
if levelOf(act) == asks.Acknowledge {
ack = append(ack, act)
} else {
auth = append(auth, act)
}
}
if len(ack) == 0 || len(auth) == 0 {
return []askPart{{about: c.Key, actions: c.Actions}}
}
return []askPart{{about: c.Key, actions: auth},
{name: partAcknowledge, about: c.Key + "." + partAcknowledge, actions: ack}}
}
// askedStore keeps the asks (broker.AskedBucket). **Every write after the first is a compare-and-set** (the
// review of 2026-10-09, L2): an ask is created once, and changed only over the revision it was read at, the
// change decided again on what is read — so two controllers, or two deliveries of one warrant, never write
// over each other, and of two that would act only the one whose write stands does.
type askedStore interface {
Get(ctx context.Context, id string) (*asked, error)
// Create keeps a new ask, and refuses one already kept under its id.
Create(ctx context.Context, a asked) error
// Change applies change to the ask kept under id, by compare-and-set, and says whether its write stood.
// change says whether to write at all; on a write that came between, it is asked again on what is read.
Change(ctx context.Context, id string, change func(*asked) bool) (bool, error)
All(ctx context.Context) ([]asked, error)
}
// askChangeTries is how often a change is read and tried again when another write came between.
const askChangeTries = 5
// asker is the controller asking the operator and acting on the answer.
type asker struct {
open func(ctx context.Context) ([]conditions.Condition, error)
silence func(ctx context.Context, key string, d time.Duration, by, why string) error
store askedStore
// publish puts a message on a subject's stream, de-duplicated by id.
publish func(ctx context.Context, subject string, body []byte, id string) error
// call performs an action's verb with its arguments, as the controller.
call func(ctx context.Context, a conditions.Action, args map[string]string) error
// setLayer sets a proposed settings layer on the operator's warrant (novox/hq ADR 0277); nil cannot.
setLayer setOnWarrant
// record writes the hand-act log.
record func(ctx context.Context, act link.HandAct) error
// routerRecord reads the router's record of an ask for a warrant missed; nil reads nothing.
routerRecord func(ctx context.Context, id string) (*asks.Warrant, error)
// routerHere says whether a router holds the seat and takes asks under the asker's name; nil is yes.
routerHere func(ctx context.Context) (bool, error)
// grantHeld says whether the bus holds the controller's grant to ask: the user list the bus's machine was
// last sent is the one the mesh composes now (novox/hq issue 353). The grant is composed from the router's
// assignment and reaches the bus only when that machine is next pushed, so between `assign` and `push`
// the record says a router is here and the bus refuses every ask. why says what to do; nil is yes.
grantHeld func(ctx context.Context) (held bool, why string, err error)
// channels is what the channels are now, as a fingerprint: who holds which kind, promising what.
channels func(ctx context.Context) string
// raise keeps the asker's own condition (sourceAsker): which conditions needing the operator could not be
// asked, and why. Nil raises nothing (a test that does not look).
raise func(ctx context.Context, obs []conditions.Observation) error
now func() time.Time
logf func(string, ...any)
saidNoRouter bool
saidNoGrant bool
// saidGrantCaveat is whether a grant held with a caveat was said, so it is said once.
saidGrantCaveat bool
mu sync.Mutex
nudged chan struct{}
}
func (a *asker) nudge() {
if a == nil {
return
}
a.mu.Lock()
if a.nudged == nil {
a.nudged = make(chan struct{}, 1)
}
ch := a.nudged
a.mu.Unlock()
select {
case ch <- struct{}{}:
default:
}
}
// keep asks until ctx ends: now, on every change of a condition, and every askEvery.
func (a *asker) keep(ctx context.Context) {
a.nudge()
tick := time.NewTicker(askEvery)
defer tick.Stop()
a.mu.Lock()
nudged := a.nudged
a.mu.Unlock()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
case <-nudged:
}
if err := a.reconcile(ctx); err != nil {
a.logf("what the operator is asked could not be brought up to date: %v", err)
}
}
}
// wants says whether a condition is one to ask about now.
func wants(c conditions.Condition, now time.Time) bool {
return len(c.Actions) > 0 && c.Needs != "" && !c.SilencedAt(now)
}
func sameAsked(a []conditions.Action, b []conditions.Action) bool {
x, _ := json.Marshal(a)
y, _ := json.Marshal(b)
return string(x) == string(y)
}
// reconcile brings what is asked in line with what is open.
func (a *asker) reconcile(ctx context.Context) error {
now := a.now()
if a.routerHere != nil {
here, err := a.routerHere(ctx)
if err != nil {
return err
}
if !here {
if !a.saidNoRouter {
a.logf("no router takes asks under the controller's name (a module declaring %s with its ask "+
"named by its caller, assigned): the operator is asked nothing until one is", broker.AsksSeat)
a.saidNoRouter = true
}
open, err := a.open(ctx)
if err != nil {
return err
}
var unasked []conditions.Condition
for _, c := range open {
if wants(c, now) {
unasked = append(unasked, c)
}
}
return a.sayUnasked(ctx, unasked, "no router takes the controller's asks: no module holding "+
broker.AsksSeat+" that takes an ask under its asker's name is assigned")
}
a.saidNoRouter = false
}
if a.grantHeld != nil {
held, why, err := a.grantHeld(ctx)
if err != nil {
return err
}
if !held {
if !a.saidNoGrant {
a.logf("the bus does not hold the controller's grant to ask yet: %s; the operator is asked nothing until it does", why)
a.saidNoGrant = true
}
open, err := a.open(ctx)
if err != nil {
return err
}
var unasked []conditions.Condition
for _, c := range open {
if wants(c, now) {
unasked = append(unasked, c)
}
}
return a.sayUnasked(ctx, unasked, "the bus does not hold the controller's grant to ask yet: "+why)
}
if why != "" && !a.saidGrantCaveat {
// Held, with a caveat worth one line (an older bus module cannot confirm the reload).
a.logf("the controller asks: %s", why)
a.saidGrantCaveat = true
} else if why == "" {
a.saidGrantCaveat = false
}
a.saidNoGrant = false
}
channels := ""
if a.channels != nil {
channels = a.channels(ctx)
}
open, err := a.open(ctx)
if err != nil {
return err
}
all, err := a.store.All(ctx)
if err != nil {
return err
}
byCondition := map[string]asked{} // by partKey
// What is open and not a condition's — a rehearsal, a proposal — still counts toward what the router holds
// open for the controller (askMostOpen); expired unanswered, it is kept so, as the router says it too.
otherOpen := 0
for _, r := range all {
if r.State == askOpen && !r.ofACondition() && !now.Before(r.Ask.Expires) {
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.State != askOpen || x.Acted != "" {
return false
}
x.State, x.Ended, x.Acted = string(asks.OutcomeExpired), now, "nothing: the ask expired unanswered"
return true
}); err != nil {
return err
}
continue
}
if r.State == askOpen && !r.ofACondition() {
otherOpen++
}
if r.State == askOpen && r.ofACondition() {
k := partKey(r.Condition, r.Part)
if prior, held := byCondition[k]; !held || r.Opened.After(prior.Opened) {
byCondition[k] = r
}
}
}
// A warrant missed while away, read from the router's record: for a condition's ask, and for a proposal's or a
// rehearsal's alike.
if a.routerRecord != nil {
var lookedUp []asked
for _, r := range byCondition {
lookedUp = append(lookedUp, r)
}
for _, r := range all {
if r.State == askOpen && !r.ofACondition() {
lookedUp = append(lookedUp, r)
}
}
for _, r := range lookedUp {
if now.Sub(r.Opened) < askCatchUpAfter {
continue
}
if w, err := a.routerRecord(ctx, r.ID); err == nil && w != nil {
body, _ := json.Marshal(w)
if err := a.Decided(ctx, body); err != nil {
return err
}
}
}
if all, err = a.store.All(ctx); err != nil {
return err
}
byCondition = map[string]asked{}
for _, r := range all {
if r.State == askOpen && r.ofACondition() {
byCondition[partKey(r.Condition, r.Part)] = r
}
}
}
// What the operator answered lately, by condition: not asked again at once; and what the router refused,
// newest first: not asked again until the answers or the channels change.
answered, refused := map[string]asked{}, map[string]asked{}
for _, r := range all {
k := partKey(r.Condition, r.Part)
if r.State == string(asks.OutcomeChosen) && now.Sub(r.Ended) < askAgainAfterAnswer {
answered[k] = r
}
if r.State == string(asks.OutcomeRefused) {
if prior, has := refused[k]; !has || r.Opened.After(prior.Opened) {
refused[k] = r
}
}
}
// Refusals in a row, by part: those since the last ask that was not refused, within askRefusedCounted.
// superseded: a part asked since its last refusal, by an ask the router did not refuse — open, answered,
// expired or cancelled. Its refusal is history then, never said again (the review of PR 198: an answered
// retry brought the refusal back).
inRow, superseded := map[string]int{}, map[string]asked{}
for k, last := range refused {
for _, r := range all {
if partKey(r.Condition, r.Part) == k && r.State != string(asks.OutcomeRefused) && r.State != askUnsent &&
r.Opened.After(last.Opened) && r.Opened.After(superseded[k].Opened) {
superseded[k] = r
}
}
var since time.Time
for _, r := range all {
if partKey(r.Condition, r.Part) == k && r.State != string(asks.OutcomeRefused) && r.State != askUnsent &&
!r.Opened.After(last.Opened) && r.Opened.After(since) {
since = r.Opened
}
}
for _, r := range all {
if partKey(r.Condition, r.Part) == k && r.State == string(asks.OutcomeRefused) && r.Opened.After(since) &&
now.Sub(r.Opened) < askRefusedCounted {
inRow[k]++
}
}
}
wanted := map[string]bool{}
var unasked []conditions.Condition // refused by the router, and nothing it was refused for changed
var refusedWords []string
// The most urgent first, then the oldest: those are asked when no more than askMostOpen may be.
sort.SliceStable(open, func(i, j int) bool {
ui, uj := open[i].Severity == conditions.Urgent, open[j].Severity == conditions.Urgent
if ui != uj {
return ui
}
if !open[i].Raised.Equal(open[j].Raised) {
return open[i].Raised.Before(open[j].Raised)
}
return open[i].Key < open[j].Key
})
openNow := otherOpen
for _, c := range open {
if !wants(c, now) {
continue
}
for _, p := range partsOf(c) {
if r, held := byCondition[partKey(c.Key, p.name)]; held && sameAsked(r.Actions, p.actions) && now.Before(r.Ask.Expires) {
openNow++
}
}
}
for _, c := range open {
if !wants(c, now) {
continue
}
saidUnasked := false
for _, p := range partsOf(c) {
key := partKey(c.Key, p.name)
wanted[key] = true
if r, was := refused[key]; was && sameAsked(r.Actions, p.actions) {
cur, held := byCondition[key]
ended := r.Ended
if ended.IsZero() {
ended = r.Opened
}
later, asked := superseded[key]
waiting := !held && !asked && r.Channels == channels && now.Sub(ended) < refusedRetryAfter(inRow[key])
// Asked again now (the wait over), or lately and the router's word not in yet: said as it was until
// that word, so the condition neither clears nor is raised again at each try.
retrying := !held && !asked && !waiting
verdictDue := held && asked && later.ID == cur.ID && now.Sub(cur.Opened) < askVerdictWait
if waiting || retrying || verdictDue {
if !saidUnasked {
unasked, saidUnasked = append(unasked, c), true
}
if r.Warrant != nil && r.Warrant.Words != "" {
refusedWords = append(refusedWords, r.Warrant.Words)
}
}
if waiting {
continue // refused lately, and nothing it was refused for has changed: asked again after the wait
}
}
if r, done := answered[key]; done && sameAsked(r.Actions, p.actions) {
if _, held := byCondition[key]; !held {
continue
}
}
if r, held := byCondition[key]; held {
switch {
case !sameAsked(r.Actions, p.actions):
if err := a.cancel(ctx, r, "its answers changed"); err != nil {
return err
}
case !now.Before(r.Ask.Expires):
// Expired unanswered: the router says so too; asked again below while it lasts.
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.State != askOpen {
return false
}
x.State, x.Ended = string(asks.OutcomeExpired), now
return true
}); err != nil {
return err
}
openNow--
default:
continue
}
}
if openNow >= askMostOpen {
continue // asked when one of the open ones ends, most urgent first
}
if err := a.ask(ctx, c, p, channels); err != nil {
a.logf("the operator could not be asked about %s: %v", c.Key, err)
continue
}
openNow++
}
}
stillOpen := map[string]conditions.Condition{}
for _, c := range open {
stillOpen[c.Key] = c
}
for key, r := range byCondition {
if wanted[key] {
continue
}
// **A silence never takes an approval back** (the confirmation review of 2026-10-09, M1). Silence is an
// acknowledgement — anyone at the desk may give it — so a condition silenced while its approval is asked
// keeps that ask open, unchanged, until it is answered on a channel that proves who answered, or expires.
// It is not asked again once it ends, while the silence lasts.
if c, open := stillOpen[r.Condition]; open && c.SilencedAt(now) && r.Ask.Highest() != asks.Acknowledge &&
now.Before(r.Ask.Expires) && keepsItsAnswers(c, r) {
continue
}
if err := a.cancel(ctx, r, "the condition ended, was silenced or needs nothing now"); err != nil {
return err
}
}
why := "the router refused the ask"
if len(refusedWords) > 0 {
why += ": " + refusedWords[0]
}
return a.sayUnasked(ctx, unasked, why)
}
// keepsItsAnswers says a condition still offers the answers an ask kept was asked with.
func keepsItsAnswers(c conditions.Condition, r asked) bool {
for _, p := range partsOf(c) {
if partKey(c.Key, p.name) == partKey(r.Condition, r.Part) {
return sameAsked(r.Actions, p.actions)
}
}
return false
}
// sourceAsker raises the asker's own condition.
const sourceAsker = "asker"
// sayUnasked keeps the asker's one condition: while a condition that needs the operator could not be asked
// on any channel, said loudly (failure must be loud), cleared when every one could be.
func (a *asker) sayUnasked(ctx context.Context, unasked []conditions.Condition, why string) error {
if a.raise == nil {
return nil
}
var obs []conditions.Observation
if len(unasked) > 0 {
keys := make([]string, 0, len(unasked))
severity := conditions.Warning
for _, c := range unasked {
keys = append(keys, c.Key)
if c.Severity == conditions.Urgent {
severity = conditions.Urgent
}
}
sort.Strings(keys)
obs = append(obs, conditions.Observation{Scope: conditions.ScopeSeat, ID: broker.AsksSeat, Token: "unasked",
Kind: "asks-undelivered", Severity: severity, Source: sourceAsker,
Summary: fmt.Sprintf("%d condition(s) that need the operator could not be asked on any channel: %s; %s",
len(keys), strings.Join(keys, ", "), why),
Headline: "Questions for you not delivered",
Explanation: "The mesh could not send you its questions on any channel.",
Needs: "answer them from the mesh MCP server, and check why no channel carries them.",
Resolved: "The mesh can ask you again"})
}
if err := a.raise(ctx, obs); err != nil {
a.logf("whether the operator could be asked could not be kept as a condition: %v", err)
}
return nil
}
// optionID is an action's label as an option's id: "Silence for a week" is silence-for-a-week.
func optionID(label string) string {
var b strings.Builder
dash := false
for _, r := range strings.ToLower(label) {
switch {
case r >= 'a' && r <= 'z', r >= '0' && r <= '9':
b.WriteRune(r)
dash = false
case !dash && b.Len() > 0:
b.WriteByte('-')
dash = true
}
}
return strings.TrimSuffix(b.String(), "-")
}
// doesWords is what an action does, in the words an option says it with.
func doesWords(act conditions.Action) string {
switch {
case act.Arguments["silence"] != "":
return "nothing more is said of it for a week"
case act.Verb == "mesh-delivery.release":
return "the delivery goes on"
case act.Verb == "mesh-delivery.stop":
return "the delivery ends"
case act.Verb == broker.ControllerSeat+".plans" && act.Arguments["go"] != "":
return "the delivery starts"
case act.Verb == broker.ControllerSeat+".plans" && act.Arguments["stop"] != "":
return "the delivery is stopped"
case strings.HasSuffix(act.Verb, ".restart"):
return "its service is restarted on " + act.Machine
}
return strings.ToLower(act.Label)
}
// askText is a condition's words as an ask says them: without where an answer is given when no channel can
// give it (FromMeshMCPServer), since the ask is answered on a channel and the router says where else.
func askText(s string) string {
for _, with := range []string{", " + FromMeshMCPServer, " " + FromMeshMCPServer} {
s = strings.ReplaceAll(s, with, ".")
}
return strings.ReplaceAll(s, "..", ".")
}
// askOf is the ask one part of a condition is asked with.
func askOf(id string, c conditions.Condition, p askPart, now time.Time) (asks.Ask, map[string]int) {
q := asks.Ask{ID: id, Headline: c.Headline, Explanation: askText(c.Explanation), Who: asks.Operator,
OnExpiry: "nothing is done, and you are asked again while it lasts", About: p.about,
Urgent: c.Severity == conditions.Urgent}
options := map[string]int{}
approves := false
for i, act := range p.actions {
level := levelOf(act) // an action that says nothing of its level is never taken for less than approve
approves = approves || level != asks.Acknowledge
oid := optionID(act.Label)
options[oid] = i
// Every option binds the exact act it stands for (novox/hq ADR 0259 §6): the verb, the machine and
// every argument. The warrant then authorises that act and no other.
binds, _ := asks.ActDigest(boundAct(act))
q.Options = append(q.Options, asks.Option{ID: oid, Label: act.Label, Does: doesWords(act), Level: level,
Binds: binds})
}
q.Expires = now.Add(askAcknowledgeFor)
if approves {
q.Expires = now.Add(askApproveFor)
}
return q, options
}
// boundAct is what an option's Binds digests: the act exactly as the controller will perform it — its verb,
// machine, level, and each argument as "arg.<name>" — and never its label or words.
func boundAct(act conditions.Action) asks.Act {
out := asks.Act{"verb": act.Verb, "machine": act.Machine, "level": act.Level}
for k, v := range act.Arguments {
out["arg."+k] = v
}
return out
}
func newAskID() string {
var b [8]byte
_, _ = rand.Read(b[:])
return "c" + hex.EncodeToString(b[:])
}
// askUnsent is an ask kept and never published: asked again at the next look.
const askUnsent = "unsent"
// ask publishes one ask about a part of a condition, kept before it is published (the review of 2026-10-09,
// L3): a warrant for it then always finds it, and one whose publishing failed is marked so and asked again.
func (a *asker) ask(ctx context.Context, c conditions.Condition, p askPart, channels string) error {
now := a.now()
id := newAskID()
q, options := askOf(id, c, p, now)
if err := q.Check(now); err != nil {
return err
}
body, err := json.Marshal(q)
if err != nil {
return err
}
if err := a.store.Create(ctx, asked{ID: id, Condition: c.Key, Part: p.name, Ask: q, Actions: p.actions,
Options: options, State: askOpen, Opened: now, Channels: channels}); err != nil {
return fmt.Errorf("the ask could not be kept, so it was not asked: %w", err)
}
if err := a.publish(ctx, asks.AskSubject(askerName), body, "ask."+id); err != nil {
if _, cerr := a.store.Change(ctx, id, func(x *asked) bool {
if x.State != askOpen || x.Acted != "" {
return false
}
x.State, x.Ended, x.Acted = askUnsent, a.now(), "nothing: it could not be published: "+err.Error()
return true
}); cerr != nil {
a.logf("the ask %s could not be published, and could not be marked so: %v", id, cerr)
}
return err
}
a.logf("asked the operator about %s (%s): %d answer(s)", c.Key, id, len(q.Options))
return nil
}
// cancel takes an ask back: kept cancelled first, so a warrant that comes after is refused, then said to the
// router; a cancel the router did not hear leaves the ask to expire there, and nothing is done on it here.
func (a *asker) cancel(ctx context.Context, r asked, why string) error {
stood, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.State != askOpen {
return false
}
x.State, x.Ended = askCancelled, a.now()
return true
})
if err != nil || !stood {
return err
}
body, _ := json.Marshal(map[string]string{"id": r.ID})
if err := a.publish(ctx, asks.CancelSubject(askerName), body, "cancel."+r.ID); err != nil {
a.logf("the ask %s about %s is taken back here, and the router could not be told (%v): it expires there, "+
"and no answer to it is acted on", r.ID, r.Condition, err)
return nil
}
a.logf("took back the ask %s about %s: %s", r.ID, r.Condition, why)
return nil
}
// Decided takes the router's word on one of the controller's asks (link.Decider). An error is returned only
// when what was decided could not be kept, so the word is held and heard again.
func (a *asker) Decided(ctx context.Context, body []byte) error {
var w asks.Warrant
if err := json.Unmarshal(body, &w); err != nil {
a.logf("the router's word on an ask could not be read; ignored: %v", err)
return nil
}
if w.Asker != askerName {
a.logf("REFUSED a warrant for %s's ask %s: the controller acts only on its own", w.Asker, w.Ask)
return nil
}
r, err := a.store.Get(ctx, w.Ask)
if err != nil {
return err
}
if r == nil {
a.logf("REFUSED a warrant for the ask %s, which the controller does not hold", w.Ask)
return nil
}
if r.Acted != "" {
return nil // heard again: acted on once
}
now := a.now()
if w.Outcome != asks.OutcomeChosen {
acted := "nothing: the ask " + string(w.Outcome)
if w.Words != "" {
acted += ": " + w.Words
}
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.Acted != "" {
return false
}
x.State, x.Ended, x.Warrant, x.Acted = string(w.Outcome), now, &w, acted
return true
}); err != nil {
return err
}
a.logf("the ask %s about %s ended %s; nothing is done", r.ID, r.Condition, w.Outcome)
return nil
}
if r.State != askOpen {
// Cancelled, replaced or expired in the controller's own record: no answer to it is acted on.
a.logf("REFUSED a warrant for the ask %s, which is %s in the controller's own record", r.ID, r.State)
return nil
}
option, err := w.For(askerName, r.Ask)
if err != nil {
a.logf("REFUSED a warrant for the ask %s: %v", r.ID, err)
return nil
}
index, offered := r.Options[option.ID]
if !offered || index >= len(r.Actions) {
a.logf("REFUSED a warrant for the ask %s: it chose %s, which no action stands for", r.ID, option.ID)
return nil
}
act := r.Actions[index]
// The act about to be performed is the one the option bound when the controller asked: a record changed
// since is refused, never performed.
if err := option.Performs(boundAct(act)); err != nil {
a.logf("REFUSED a warrant for the ask %s: %v", r.ID, err)
return nil
}
open, err := a.open(ctx)
if err != nil {
return err
}
stillOpen := r.Rehearsal || r.Proposal != nil // a rehearsal and a proposal are about no condition
for _, c := range open {
stillOpen = stillOpen || c.Key == r.Condition
}
if !stillOpen {
// The asker checks the state is still what it asked about before it acts (to-be 46 §10, step 7).
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.State != askOpen || x.Acted != "" {
return false
}
x.State, x.Warrant, x.Ended, x.Acted = string(asks.OutcomeChosen), &w, now,
"nothing: the condition ended before the answer"
return true
}); err != nil {
return err
}
a.logf("%s, for %s, which ended meanwhile: nothing is done", w.Says(), r.Condition)
return nil
}
// Claimed before acting, by compare-and-set: only the delivery whose write stands acts (security review
// of 2026-10-08, finding 9). Not by the warrant's message id, which another publisher could take first:
// the controller's own record decides.
claimed, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.State != askOpen || x.Acted != "" {
return false
}
x.State, x.Warrant, x.Acted = string(asks.OutcomeChosen), &w, "acting"
return true
})
if err != nil {
return err
}
if !claimed {
a.logf("the warrant for the ask %s was already taken by another delivery; nothing more is done", r.ID)
return nil
}
r.Acted = "acting"
why := fmt.Sprintf("%s (ask %s)", w.Says(), r.ID)
args := map[string]string{}
for k, v := range act.Arguments {
args[k] = v
}
if v, takes := args["why"]; takes && v == "" {
args["why"] = why
}
var acted error
outcome := "done"
switch {
case r.Rehearsal && act.Verb == rehearsalVerb:
// A rehearsal's answer performs nothing: it is recorded below as the operator's decision.
case r.Proposal != nil && act.Verb == proposalVerb:
// A proposed settings layer, set by this controller itself on Approve (novox/hq ADR 0277): the act's
// digest of the values is held to the record's own values before anything is set.
outcome, acted = a.decideProposal(ctx, *r, act, w)
if acted == nil && outcome != "" && !strings.HasPrefix(outcome, "nothing") {
outcome = "done: " + outcome
}
case act.Arguments["silence"] != "":
acted = a.silence(ctx, act.Arguments["silence"], conditions.MaxSilence, byWords(w), why)
default:
acted = a.call(ctx, act, args)
}
ended := a.now()
if acted != nil {
outcome = "failed: " + acted.Error()
}
r.Ended, r.Acted = ended, outcome
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
if x.Acted != "acting" {
return false
}
x.Ended, x.Acted = ended, outcome
return true
}); err != nil {
a.logf("%s was acted on (%s), and how it ended could NOT be kept: %v", r.ID, outcome, err)
}
verbArgs := []string{act.Verb}
if act.Machine != "" {
verbArgs = append(verbArgs, "on "+act.Machine)
}
keys := make([]string, 0, len(args))
for k := range args {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
if k != "why" {
verbArgs = append(verbArgs, k+"="+args[k])
}
}
if err := a.record(ctx, link.HandAct{Verb: handActWarrant, Args: verbArgs, Why: why, By: byWords(w),
Cause: conditions.CauseOperatorAnswer, Condition: r.Condition, Via: viaWords(w), Ask: r.ID,
Proofs: w.Proofs, RequestedBy: r.Condition, Outcome: r.Acted}); err != nil {
a.logf("%s was done, and could NOT be recorded in the hand-act log: %v", why, err)
}
a.logf("%s: %s", why, r.Acted)
return nil
}
// handActWarrant is the verb an act the operator chose on a warrant is recorded under: a person's decision,
// never a repair (handActVerbs).
const handActWarrant = "warrant"
// byWords is who chose, as the hand-act log says it: "the operator, as telegram identity 42".
func byWords(w asks.Warrant) string {
if w.By == nil {
return "the operator"
}
return fmt.Sprintf("the %s, as %s identity %s", w.By.Who, w.By.Kind, w.By.Identity)
}
// viaWords is the channel an answer came through: its module and kind, and how the sender was known.
func viaWords(w asks.Warrant) string {
if w.By == nil {
return w.Channel
}
via := w.Channel + " (" + w.By.Kind + ")"
if w.By.Verified != "" {
via += ", " + w.By.Verified
}
return via
}
// errNotGranted is an action whose verb the controller's grant does not name.
var errNotGranted = errors.New("the controller's grant does not name this verb")
-153
View File
@@ -1,153 +0,0 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// busAsker is an asker on a real bus's `asked` bucket, counting what it performs: two of them are two
// controllers sharing one record.
type busAskerRig struct {
mu sync.Mutex
called int
acts int
open []conditions.Condition
sent [][]byte
}
func (rig *busAskerRig) asker(t *testing.T, conn *nats.Conn, now time.Time) *asker {
return &asker{
open: func(context.Context) ([]conditions.Condition, error) {
rig.mu.Lock()
defer rig.mu.Unlock()
return rig.open, nil
},
silence: func(context.Context, string, time.Duration, string, string) error { return nil },
store: busAsked{conn: conn},
publish: func(_ context.Context, subject string, body []byte, _ string) error {
rig.mu.Lock()
defer rig.mu.Unlock()
if subject == asks.AskSubject(askerName) {
rig.sent = append(rig.sent, body)
}
return nil
},
call: func(context.Context, conditions.Action, map[string]string) error {
time.Sleep(20 * time.Millisecond) // long enough for the other delivery to arrive meanwhile
rig.mu.Lock()
defer rig.mu.Unlock()
rig.called++
return nil
},
record: func(context.Context, link.HandAct) error {
rig.mu.Lock()
defer rig.mu.Unlock()
rig.acts++
return nil
},
now: func() time.Time { return now },
logf: t.Logf,
}
}
func askedBus(t *testing.T) *nats.Conn {
t.Helper()
conn, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(conn.Close)
js, err := jetstream.New(conn)
if err != nil {
t.Fatal(err)
}
if _, err := js.CreateKeyValue(context.Background(), jetstream.KeyValueConfig{Bucket: broker.AskedBucket}); err != nil {
t.Fatal(err)
}
return conn
}
// The review of 2026-10-09 (L7): two deliveries of one warrant, to two controllers at once, perform its act
// exactly once and record it once — the record's compare-and-set decides, never the warrant's message id.
func TestTwoAnswersAtOnceActOnce(t *testing.T) {
conn := askedBus(t)
now := time.Date(2026, 10, 9, 14, 0, 0, 0, time.UTC)
rig := &busAskerRig{open: []conditions.Condition{heldCondition()}}
first, second := rig.asker(t, conn, now), rig.asker(t, conn, now)
if err := first.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
if len(rig.sent) != 1 {
t.Fatalf("asked %d times", len(rig.sent))
}
var q asks.Ask
_ = json.Unmarshal(rig.sent[0], &q)
release, _ := q.Option("release")
w := asks.Warrant{Ask: q.ID, Asker: askerName, About: q.About, Outcome: asks.OutcomeChosen, Option: release.ID,
Label: release.Label, Level: release.Level, Channel: "telegram", Proofs: []string{"P1"}, At: now,
AskDigest: q.Digest(), By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"}}
body, _ := json.Marshal(w)
var wg sync.WaitGroup
for _, a := range []*asker{first, second, first, second} {
wg.Add(1)
go func(a *asker) {
defer wg.Done()
if err := a.Decided(context.Background(), body); err != nil {
t.Error(err)
}
}(a)
}
wg.Wait()
if rig.called != 1 || rig.acts != 1 {
t.Fatalf("performed %d time(s), recorded %d time(s)", rig.called, rig.acts)
}
got, err := busAsked{conn: conn}.Get(context.Background(), q.ID)
if err != nil || got == nil || got.Acted != "done" {
t.Fatalf("kept as %+v (%v)", got, err)
}
}
// The review of 2026-10-09 (L2): a write decided on a record read earlier never lands over one made since. A
// cancel read before the answer was acted on leaves the act's record as it is.
func TestAStaleCancelDoesNotWriteOverAnAct(t *testing.T) {
conn := askedBus(t)
now := time.Date(2026, 10, 9, 14, 0, 0, 0, time.UTC)
rig := &busAskerRig{open: []conditions.Condition{heldCondition()}}
a := rig.asker(t, conn, now)
if err := a.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
var q asks.Ask
_ = json.Unmarshal(rig.sent[0], &q)
stale, _ := busAsked{conn: conn}.Get(context.Background(), q.ID)
release, _ := q.Option("release")
w := asks.Warrant{Ask: q.ID, Asker: askerName, About: q.About, Outcome: asks.OutcomeChosen, Option: release.ID,
Label: release.Label, Level: release.Level, Channel: "telegram", At: now, AskDigest: q.Digest(),
By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"}}
body, _ := json.Marshal(w)
if err := a.Decided(context.Background(), body); err != nil {
t.Fatal(err)
}
if err := a.cancel(context.Background(), *stale, "the condition ended"); err != nil {
t.Fatal(err)
}
got, _ := busAsked{conn: conn}.Get(context.Background(), q.ID)
if got.State != string(asks.OutcomeChosen) || got.Acted != "done" {
t.Errorf("a stale cancel wrote over the act: %+v", got)
}
}
File diff suppressed because it is too large Load Diff
-589
View File
@@ -1,589 +0,0 @@
package main
// The asker on the bus: its asks in the controller's bucket `asked`, its asks and cancels published on the
// seat under the controller's name, the verbs a warrant chooses called with the controller's grant, and the
// router's record of its asks read under its name (novox/hq ADR 0259).
import (
"context"
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// askerFrom is the serving controller's asker; nil in any other process.
var askerFrom *asker
// askWithin is how long a verb a warrant chose is given to answer.
const askWithin = time.Minute
type busAsked struct{ conn *nats.Conn }
func (b busAsked) kv(ctx context.Context) (jetstream.KeyValue, error) {
js, err := jetstream.New(b.conn)
if err != nil {
return nil, err
}
return js.KeyValue(ctx, broker.AskedBucket)
}
func (b busAsked) Get(ctx context.Context, id string) (*asked, error) {
kv, err := b.kv(ctx)
if err != nil {
return nil, err
}
e, err := kv.Get(ctx, id)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, nil
}
if err != nil {
return nil, err
}
var r asked
return &r, json.Unmarshal(e.Value(), &r)
}
// Create keeps a new ask under its id, and only where none is kept: never over another.
func (b busAsked) Create(ctx context.Context, r asked) error {
kv, err := b.kv(ctx)
if err != nil {
return err
}
body, err := json.Marshal(r)
if err != nil {
return err
}
_, err = kv.Create(ctx, r.ID, body)
return err
}
// Change applies change to the ask kept under id by compare-and-set on its key's revision (the review of
// 2026-10-09, L2): read, changed, and written only over the revision read; when another write came between,
// read again and asked again, at most askChangeTries times. change says whether to write at all.
func (b busAsked) Change(ctx context.Context, id string, change func(*asked) bool) (bool, error) {
kv, err := b.kv(ctx)
if err != nil {
return false, err
}
for try := 0; try < askChangeTries; try++ {
e, err := kv.Get(ctx, id)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return false, nil
}
if err != nil {
return false, err
}
var r asked
if err := json.Unmarshal(e.Value(), &r); err != nil {
return false, err
}
if !change(&r) {
return false, nil
}
body, err := json.Marshal(r)
if err != nil {
return false, err
}
if _, err := kv.Update(ctx, id, body, e.Revision()); err != nil {
var api *jetstream.APIError
if errors.Is(err, jetstream.ErrKeyExists) || (errors.As(err, &api) && api.ErrorCode == jetstream.JSErrCodeStreamWrongLastSequence) {
continue
}
return false, err
}
return true, nil
}
return false, fmt.Errorf("the ask %s changed under every one of %d tries", id, askChangeTries)
}
func (b busAsked) All(ctx context.Context) ([]asked, error) {
kv, err := b.kv(ctx)
if err != nil {
return nil, err
}
lister, err := kv.ListKeys(ctx)
if err != nil {
return nil, err
}
defer func() { _ = lister.Stop() }()
var out []asked
for k := range lister.Keys() {
e, err := kv.Get(ctx, k)
if err != nil {
continue
}
var r asked
if json.Unmarshal(e.Value(), &r) == nil {
out = append(out, r)
}
}
return out, nil
}
// callAction performs an action's verb as the controller, through the grant that names it.
func callAction(conn *nats.Conn) func(ctx context.Context, a conditions.Action, args map[string]string) error {
return func(ctx context.Context, a conditions.Action, args map[string]string) error {
seat, verb, ok := strings.Cut(a.Verb, ".")
if !ok {
return fmt.Errorf("%q names no seat and verb", a.Verb)
}
body := map[string]any{}
for k, v := range args {
body[k] = v
}
if seat == catalogue.DeliverySeat {
_, err := askDeliveryOwner(ctx, conn, verb, body)
return err
}
granted := false
for _, v := range broker.VerbsTheControllerActsOnAWarrant {
granted = granted || (v.Seat == seat && v.Verb == verb)
}
if !granted {
return fmt.Errorf("%s: %w", a.Verb, errNotGranted)
}
var answer link.Answer
var err error
if a.Machine != "" {
answer, err = link.AskSeatTool(ctx, conn, seat, verb, a.Machine, body, askWithin)
} else {
answer, err = link.AskMeshSeatTool(ctx, conn, seat, verb, body, askWithin)
}
if err != nil {
return err
}
if answer.Error != "" {
return fmt.Errorf("%s refused: %s", a.Verb, answer.Error)
}
return nil
}
}
// routerRecordOf reads the router's record of one of the controller's asks, under its name, and answers
// how it ended when it did: the bucket is the one the asks seat's declarer names as its records.
func routerRecordOf(conn *nats.Conn, inv *inventory.Inventory) func(ctx context.Context, id string) (*asks.Warrant, error) {
return func(ctx context.Context, id string) (*asks.Warrant, error) {
bucket, err := asksRecords(ctx, inv)
if err != nil || bucket == "" {
return nil, err
}
reply, err := conn.RequestWithContext(ctx, "$JS.API.DIRECT.GET.KV_"+bucket+".$KV."+bucket+"."+askerName+"."+id, nil)
if err != nil {
return nil, err
}
if reply.Header.Get("Status") != "" {
return nil, nil // none, or not readable: the event says it
}
var rec struct {
State string `json:"state"`
Warrant *asks.Warrant `json:"warrant"`
}
if json.Unmarshal(reply.Data, &rec) != nil || rec.State == "open" || rec.Warrant == nil {
return nil, nil
}
return rec.Warrant, nil
}
}
// asksRecords is the bucket the asks seat's declarer keeps its record of asks in.
func asksRecords(ctx context.Context, inv *inventory.Inventory) (string, error) {
declared, err := inv.Catalogue(ctx)
if err != nil {
return "", err
}
for _, m := range declared {
for _, s := range m.DefinesSeats {
if s.Name == broker.AsksSeat && len(s.Records) > 0 {
return broker.BucketName(m.Module, s.Records[0]), nil
}
}
}
return "", nil
}
// routerHereIn says whether a module declaring the asks seat, with its ask named by its caller, is assigned:
// without it nothing takes an ask, and asking would only fill a queue nobody reads.
func routerHereIn(inv *inventory.Inventory) func(ctx context.Context) (bool, error) {
return func(ctx context.Context) (bool, error) {
entries, err := inv.Catalogued(ctx)
if err != nil {
return false, err
}
for _, e := range entries {
for _, s := range e.Manifest.DefinesSeats {
if s.Name == broker.AsksSeat && s.NamedByCaller("ask") && len(e.On) > 0 {
return true, nil
}
}
}
return false, nil
}
}
// grantHeldIn says whether the bus holds the controller's grant to ask (novox/hq issue 353), judged on the two
// facts that make it so, and failing closed on each:
//
// 1. **composed**: the user list the mesh composes now grants the controller itself to publish its ask
// (controllerMayAsk) — the controller's own grant, not whether any user of the list changed;
// 2. **in force**: the bus's own module, on the machine holding the bus, says the user list there allows it
// and the server reloaded after that list was written (the nats module's `nats_user_can`, `in_force`) — a
// list sent is not a list the bus enforces until it reloads.
//
// Anything that cannot be worked out is not held, with why, so the conditions that need the operator are said
// as undelivered rather than asked into a refusal.
func grantHeldIn(open *stores, conn *nats.Conn) func(ctx context.Context) (bool, string, error) {
return grantHeldBy(composedGrantIn(open), busGrantIn(open, conn), time.Now)
}
// grantJudge is one of the two facts: whether it holds, and why not.
type grantJudge func(ctx context.Context) (bool, string, error)
// grantHeldBy is the judgement over the two facts, remembered for grantLookEvery: composing the list resolves
// the bus's machine whole, and asking the bus's module is a call.
func grantHeldBy(composed, inForce grantJudge, now func() time.Time) func(ctx context.Context) (bool, string, error) {
var mu sync.Mutex
var at time.Time
var held bool
var why string
return func(ctx context.Context) (bool, string, error) {
mu.Lock()
defer mu.Unlock()
if !at.IsZero() && now().Sub(at) < grantLookEvery {
return held, why, nil
}
held, why = judgeGrant(ctx, composed, inForce)
at = now()
return held, why, nil
}
}
// judgeGrant is held only when both facts hold; an error is a fact not known, never a yes. A fact that holds
// with a caveat (a reload not confirmed) gives that caveat as the why of a yes.
func judgeGrant(ctx context.Context, composed, inForce grantJudge) (bool, string) {
var caveat []string
for _, fact := range []struct {
judge grantJudge
what string
}{{composed, "whether the mesh composes the controller a grant to ask"}, {inForce, "whether the bus enforces it"}} {
if fact.judge == nil {
return false, fact.what + " is not judged here"
}
ok, why, err := fact.judge(ctx)
if err != nil {
return false, fact.what + " could not be worked out: " + err.Error()
}
if !ok {
return false, why
}
if why != "" {
caveat = append(caveat, why)
}
}
return true, strings.Join(caveat, "; ")
}
// controllerMayAsk says whether the controller's own grant, as composed from these records, lets it publish its
// ask (ADR 0259 §3: composed while a router that takes asks under its asker's name is assigned).
func controllerMayAsk(records broker.Records) (bool, error) {
users, err := broker.Users(records)
if err != nil {
return false, err
}
for _, u := range users {
if u.Kind != broker.KindController {
continue
}
perms, err := broker.PermissionsFor(u)
if err != nil {
return false, err
}
return broker.MayPublish(perms, asks.AskSubject(askerName)), nil
}
return false, errors.New("the composition holds no controller")
}
// composedGrantIn is the first fact, from the mesh's records.
func composedGrantIn(open *stores) grantJudge {
return func(ctx context.Context) (bool, string, error) {
if open == nil || open.inventory == nil {
return false, "", errors.New("the mesh's records are not open")
}
records, err := open.inventory.BusRecords(ctx)
if err != nil {
return false, "", err
}
may, err := controllerMayAsk(records)
if err != nil || may {
return may, "", err
}
return false, "the mesh composes the controller no grant to ask: no router that takes asks under its " +
"asker's name is assigned", nil
}
}
// busGrantIn is the second fact, asked of the bus's own module on the machine holding the bus, with the check
// main made before it (novox/hq issue 353, PR 186: is the user list sent to the bus's machine the one composed
// now) kept for the cases the module's word does not settle.
func busGrantIn(open *stores, conn *nats.Conn) grantJudge {
ask := func(ctx context.Context) (string, json.RawMessage, error) {
if open == nil || open.inventory == nil || conn == nil {
return "", nil, errors.New("the mesh's records or the bus are not open")
}
holders, err := seatHolders(ctx, open.inventory)
if err != nil {
return "", nil, err
}
h, held := holders[theBrokerSeat]
if !held || h.Node == "" || h.Module == "" {
return "", nil, nil
}
a, err := link.AskModuleToolOn(ctx, conn, h.Module, busUserCanTool, h.Node, map[string]any{
"user": broker.ControllerName, "action": "publish", "subject": asks.AskSubject(askerName)}, 15*time.Second)
if err != nil {
return h.Node, nil, err
}
if a.Error != "" {
return h.Node, nil, errors.New(a.Error)
}
return h.Node, a.Result, nil
}
list := func(ctx context.Context) (busListState, error) {
if open == nil || open.inventory == nil {
return busListState{}, errors.New("the mesh's records are not open")
}
s, err := brokerListState(ctx, open, nil)
if err != nil || s.Machine == "" || !s.Behind {
return s, err
}
step, err := busHeld(ctx, open.inventory, []string{s.Machine})
if err != nil {
return busListState{}, err
}
s.BusStep = step[s.Machine]
return s, nil
}
return busGrantBy(ask, list)
}
// busListState is the user list on the bus's machine as the controller's record knows it: whether the list
// sent there is behind the one composed now, why it could not be composed, and the bus step a send there waits
// on (that machine cannot be pushed until a person's `bus upgrade`).
type busListState struct {
Machine string
Behind bool
Uncomposed error
BusStep string
}
// remedy is the one thing that sends the bus's machine the list composed now: the plan's own reason when it
// cannot be composed, the bus upgrade while sends there wait on it, and otherwise a push of that machine.
func (s busListState) remedy(node string) string {
if s.Machine != "" {
node = s.Machine
}
switch {
case s.Uncomposed != nil:
return fmt.Sprintf("the user list for %s cannot be composed (%v); `plan %s` says what to fix first",
node, s.Uncomposed, node)
case s.BusStep != "":
return fmt.Sprintf("sends to %s wait on the bus's planned step, so a push will not carry it: %s", node, s.BusStep)
}
return fmt.Sprintf("`push %s` carries it", node)
}
// busGrantBy judges the bus's word, and names for each way it is not held the one thing that makes it held:
//
// - **in force**: held;
// - **allowed, from a module older than `in_force`**: held, with the caveat that the reload is not confirmed.
// The file allows the publish, and the bus reloads within about 5 s of a change to it, so the worst case is
// one ask refused in those seconds, which is asked again (issue 369). The list check is not consulted:
// while a bus step is pending the bus's machine cannot be pushed, so any change to the composed list would
// read as behind for good and silence every ask until a person's bus upgrade (the review of PR 188);
// - **allowed, written, not reloaded** (the module gives both times): wait about 5 s;
// - **allowed, in force not known** (the module could not read a time): what is not known, said;
// - **not allowed, or the module cannot answer**: the list check decides — the remedy that sends the list
// when it is behind (busListState.remedy), and, for a module that cannot answer, held while the list sent
// is current, as main judged it.
//
// ask answers the machine holding the bus and its module's answer ("" when nothing holds the bus); list is the
// record's word on the list sent to that machine.
func busGrantBy(ask func(ctx context.Context) (string, json.RawMessage, error),
list func(ctx context.Context) (busListState, error)) grantJudge {
return func(ctx context.Context) (bool, string, error) {
node, raw, askErr := ask(ctx)
if askErr == nil && node == "" {
return false, "no module holds the bus, so nothing says what it enforces", nil
}
var answer busAnswer
if askErr == nil {
if answer, askErr = readBusAnswer(raw); askErr != nil {
askErr = fmt.Errorf("its answer could not be read: %w", askErr)
}
}
if askErr != nil {
if node == "" {
node = "the bus's machine"
}
s, err := list(ctx)
if err != nil {
return false, "", fmt.Errorf("the bus module on %s could not answer (%v), and whether the list sent "+
"there is the one composed now could not be worked out: %w", node, askErr, err)
}
if s.Behind {
return false, fmt.Sprintf("the bus module on %s could not answer (%v), and the user list sent there is "+
"behind what the mesh composes; %s", node, askErr, s.remedy(node)), nil
}
return true, fmt.Sprintf("the bus module on %s could not answer (%v); judged by the user list sent there, "+
"which is the one composed now", node, askErr), nil
}
notAllowed := func(older string) (bool, string, error) {
s, err := list(ctx)
if err != nil {
return false, "", fmt.Errorf("the user list on %s does not allow the controller to ask%s, and whether "+
"the list sent there is the one composed now could not be worked out: %w", node, older, err)
}
if s.Behind {
return false, fmt.Sprintf("the user list on %s does not allow the controller to ask%s: it is behind "+
"what the mesh composes; %s", node, older, s.remedy(node)), nil
}
return false, fmt.Sprintf("the user list on %s does not allow the controller to ask%s, though the list "+
"sent there is the one composed now; %s again", node, older, s.remedy(node)), nil
}
switch {
case answer.InForce == nil && answer.Allowed:
return true, fmt.Sprintf("the user list on %s allows the controller to ask, but its reload is not "+
"confirmed: the bus module there is older than the answer whether a grant is in force, and the bus "+
"upgrade (`bus upgrade`) carries one that confirms it", node), nil
case answer.InForce == nil:
return notAllowed(" (the bus module there is older than the answer whether a grant is in force)")
case !answer.Allowed:
return notAllowed("")
case *answer.InForce:
return true, "", nil
case answer.FileWritten != "" && answer.ServerLoaded != "":
return false, fmt.Sprintf("the bus on %s has the controller's grant to ask written (%s) but has not "+
"reloaded since (last load %s); it reloads in place within about 5 s, so nothing is to be done but wait",
node, answer.FileWritten, answer.ServerLoaded), nil
}
why := answer.Why
if why == "" {
why = "it gives no reason"
}
return false, fmt.Sprintf("the user list on %s allows the controller to ask, but whether the bus enforces it "+
"is not known: %s", node, why), nil
}
}
// busUserCanTool is the bus module's answer to whether a user may do something, and whether it is in force.
const busUserCanTool = "nats_user_can"
// busAnswer is the bus module's answer to busUserCanTool; InForce is nil from a module older than that field,
// and FileWritten and ServerLoaded are given only when it could read both times (mesh-catalog's nats InForce).
type busAnswer struct {
Allowed bool `json:"allowed"`
InForce *bool `json:"in_force"`
Why string `json:"why"`
FileWritten string `json:"file_written"`
ServerLoaded string `json:"server_loaded"`
}
func readBusAnswer(raw json.RawMessage) (busAnswer, error) {
var answer busAnswer
err := json.Unmarshal(raw, &answer)
return answer, err
}
// grantLookEvery is how often the bus's user list is judged against the one its machine was last sent.
const grantLookEvery = 30 * time.Second
// channelsIn is what the channels are now, as a fingerprint: each module claiming a kind of the channel
// bench, where, promising what, and whether of its own account. An ask the router refused is asked again
// once this changes.
func channelsIn(inv *inventory.Inventory) func(ctx context.Context) string {
return func(ctx context.Context) string {
entries, err := inv.Catalogued(ctx)
if err != nil {
return ""
}
var parts []string
for _, e := range entries {
for _, c := range e.Manifest.Claims {
if c.Kind == "" || !catalogue.KindedBenches[c.Name] {
continue
}
on := append([]string(nil), e.On...)
sort.Strings(on)
caps := append([]string(nil), c.Capabilities...)
sort.Strings(caps)
parts = append(parts, fmt.Sprintf("%s/%s=%s@%s[%s]own:%t", c.Name, c.Kind, e.Manifest.Module,
strings.Join(on, ","), strings.Join(caps, ","), e.Manifest.RunsAs != ""))
}
}
sort.Strings(parts)
return strings.Join(parts, ";")
}
}
// startAsking makes the serving controller's asker and hands it the router's words.
func startAsking(ctx context.Context, open *stores, server *link.Server, conn *nats.Conn, keeper *conditions.Keeper) {
js, err := jetstream.New(conn)
if err != nil {
fmt.Printf("the operator cannot be asked: %v\n", err)
return
}
a := servingAsker(open, conn, js, keeper)
if err := server.Decides(a); err != nil {
fmt.Printf("the operator's answers cannot be heard, so nothing is asked: %v\n", err)
return
}
askerFrom = a
go a.keep(ctx)
}
// servingAsker is the serving controller's asker, wired to the mesh: what it reads, publishes, performs and
// records, and the judgements it asks by — the bus's grant to ask among them (novox/hq issue 353).
func servingAsker(open *stores, conn *nats.Conn, js jetstream.JetStream, keeper *conditions.Keeper) *asker {
return &asker{
open: keeper.Open,
silence: func(ctx context.Context, key string, d time.Duration, by, why string) error {
_, err := keeper.Silence(ctx, key, d, by, why)
return err
},
store: busAsked{conn: conn},
publish: func(ctx context.Context, subject string, body []byte, id string) error {
_, err := js.Publish(ctx, subject, body, jetstream.WithMsgID(id))
return err
},
call: callAction(conn),
// A proposed settings layer is set by this controller itself on the warrant (novox/hq ADR 0277).
setLayer: setLayerIn(open),
record: func(ctx context.Context, act link.HandAct) error {
_, err := link.RecordHandAct(ctx, conn, act)
return err
},
routerRecord: routerRecordOf(conn, open.inventory),
routerHere: routerHereIn(open.inventory),
grantHeld: grantHeldIn(open, conn),
channels: channelsIn(open.inventory),
raise: func(ctx context.Context, obs []conditions.Observation) error {
return keeper.Reconcile(ctx, sourceAsker, obs)
},
now: time.Now,
logf: func(format string, args ...any) { fmt.Printf(format+"\n", args...) },
}
}
@@ -1,182 +0,0 @@
//go:build integration
package main
import (
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/overlay"
)
func TestAModuleAssignedAndNotPushedIsAwaitingAPushNotUncomposable(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
pushedBy(t, open, "laptop") // the machine was pushed before; then the module is assigned
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
// The read the rest of the mesh asks still refuses it, with the composer's typed error: nothing
// can be compared about a secret that does not exist (status), and nothing here string-matches.
plan, settings, err := planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
_, err = declarationWith(ctx, open, "laptop", plan, settings, gens, Reading)
var notMade *catalogue.NotMadeError
if !errors.As(err, &notMade) || notMade.Module != "keeper" || notMade.Name != "repository" {
t.Fatalf("a read composition did not say which secret is not made, typed: %v", err)
}
// Foreseen, it composes, names what the push makes, and made nothing.
foreseen, err := declarationWith(ctx, open, "laptop", plan, settings, gens, Foreseeing)
if err != nil || len(foreseen.foreseen) != 1 || foreseen.foreseen[0] != "keeper/repository" {
t.Fatalf("foreseen: %v %v", foreseen.foreseen, err)
}
if _, held, err := open.inventory.ModuleSecretIfIssued(ctx, "laptop", "keeper", "repository"); err != nil || held {
t.Fatalf("asking ahead of the push made the secret (held %v, %v)", held, err)
}
// Within the bound: nothing at all — no urgent, no warning, nobody notified.
if got := d1(t, open); len(got) != 0 {
t.Fatalf("a machine waiting for a push raised %+v", got)
}
// Past the bound: a warning, not urgent, saying what the push will make.
before := awaitingPushBound
awaitingPushBound = -time.Minute
t.Cleanup(func() { awaitingPushBound = before })
got := d1(t, open)
if len(got) != 1 || got[0].Key() != "machine.laptop.awaiting-push" || got[0].Severity != conditions.Warning ||
!strings.Contains(got[0].Summary, "keeper/repository") || !strings.Contains(got[0].Summary, "push laptop") {
t.Fatalf("a machine left un-pushed past the bound: %+v", got)
}
// Pushed: the secret is made and D1 says nothing.
pushedBy(t, open, "laptop")
if got := d1(t, open); len(got) != 0 {
t.Fatalf("a pushed machine still raised %+v", got)
}
}
func TestASecretThePushCannotMakeIsStillUncomposable(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aTalker())
if _, err := assign(ctx, open, "laptop", "talker"); err != nil {
t.Fatal(err)
}
got := d1(t, open)
if len(got) != 1 || got[0].Key() != "machine.laptop.uncomposable" || got[0].Severity != conditions.Urgent ||
!strings.Contains(got[0].Summary, "module issue talker --node laptop") {
t.Fatalf("a push that will be refused was not said urgently: %+v", got)
}
// And a machine whose composition fails for anything else, with a secret waiting beside it, is
// uncomposable for that — the stand-in hides nothing.
register(t, open, aKeeper())
if _, err := assign(ctx, open, "anchor", "keeper"); err != nil {
t.Fatal(err)
}
one, two := rivals()
register(t, open, one)
register(t, open, two)
_, _ = assign(ctx, open, "anchor", "rival-one")
_, _ = assign(ctx, open, "anchor", "rival-two")
keys := map[string]conditions.Severity{}
for _, o := range d1(t, open) {
keys[o.Key()] = o.Severity
}
if keys["machine.anchor.uncomposable"] != conditions.Urgent {
t.Fatalf("a real failure beside a waiting secret: %v", keys)
}
}
func TestAGivenSecretUnderAnOldKeyIsNotForeseen(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
if err := open.inventory.AcceptSecretForModule(ctx, "laptop", "keeper", "repository", "given"); err != nil {
t.Fatal(err)
}
record, err := open.inventory.NodeByName(ctx, "laptop")
if err != nil {
t.Fatal(err)
}
if err := open.inventory.RecordSealingKey(ctx, record.ID, aPublicKey(t)); err != nil {
t.Fatal(err)
}
got := d1(t, open)
if len(got) != 1 || got[0].Key() != "machine.laptop.uncomposable" || !strings.Contains(got[0].Summary, "issue it again") {
t.Fatalf("a given secret under an old key: %+v", got)
}
}
func TestAMachineAwaitsAPushSinceItsOldestAssignmentSinceTheLastSend(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
pushedBy(t, open, "laptop")
sent := time.Now()
since, err := open.inventory.AwaitingSince(ctx, "laptop")
if err != nil || since.After(sent) {
t.Fatalf("nothing assigned since the send: waiting since %v (%v), the send was before %v", since, err, sent)
}
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
since, err = open.inventory.AwaitingSince(ctx, "laptop")
if err != nil || since.Before(sent.Add(-time.Second)) {
t.Fatalf("assigned after the send: waiting since %v (%v), not from the assignment", since, err)
}
}
func TestALastSendIsReadFromTheSendAndTheReport(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
pushedBy(t, open, "laptop")
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
sends, err := readDeliveries(ctx, open.inventory)
if err != nil {
t.Fatal(err)
}
if sends["laptop"].settled("keeper", time.Now()) {
t.Fatal("a holder assigned and not pushed is expected to answer")
}
if !sends["laptop"].settled(overlay.Name, time.Now().Add(time.Hour)) {
t.Fatal("a module the machine was sent long ago is not expected to answer")
}
pushedBy(t, open, "laptop")
sends, err = readDeliveries(ctx, open.inventory)
if err != nil {
t.Fatal(err)
}
if sends["laptop"].settled("keeper", time.Now()) {
t.Fatal("pushed a moment ago and not reported, a holder is already expected")
}
if !sends["laptop"].settled("keeper", time.Now().Add(reportGrace+time.Minute)) {
t.Fatal("pushed past the grace, a holder is not expected")
}
if _, ok := sends["laptop"].carried["keeper"]; !ok {
t.Fatalf("the send's builds do not carry keeper: %v", sends["laptop"].carried)
}
pushedAndApplied(t, open, "laptop")
if sends, err = readDeliveries(ctx, open.inventory); err != nil || !sends["laptop"].settled("keeper", time.Now()) {
t.Fatalf("pushed and reported applied, a holder is not expected to answer (%v)", err)
}
}
-115
View File
@@ -1,115 +0,0 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// Between `assign` and `push` a module's own secrets are not made yet: the push makes them. D1 composed
// the machine's declaration without making anything, failed on the missing secret, and raised an urgent
// "nothing can be sent to <machine>" — seen live on 2026-10-06, a desktop notification for a machine
// the next push sent to without a word (novox/hq issue 275). D1 now composes as the push would, with a
// stand-in for what the push makes: pending, not broken, and said only past a bound, as a warning.
// aKeeper is a module with an own secret the mesh makes — a backup repository's password.
func aKeeper() catalogue.Manifest {
return catalogue.Manifest{Module: "keeper", Version: "1",
OwnSecrets: catalogue.OwnSecrets{"repository": {Path: "/var/lib/mesh/keeper/repository"}},
Resources: []map[string]any{
{"id": "state", "type": "directory", "path": "/var/lib/mesh/keeper", "mode": "0700"},
}}
}
// pushedBy records a send to a machine as a push does — composed on the send path, so its own secrets
// are made — without a bus to carry it.
func pushedBy(t *testing.T, open *stores, node string) string {
t.Helper()
ctx := t.Context()
plan, settings, err := planFor(ctx, open, node)
if err != nil {
t.Fatal(err)
}
declared, err := declarationFor(ctx, open, node, plan, settings)
if err != nil {
t.Fatal(err)
}
body, err := declared.Body()
if err != nil {
t.Fatal(err)
}
record, err := open.inventory.NodeByName(ctx, node)
if err != nil {
t.Fatal(err)
}
digest := digestOf(body)
if err := open.inventory.RecordSent(ctx, record.ID, digest, declared.Builds); err != nil {
t.Fatal(err)
}
return digest
}
// pushedAndApplied is pushedBy, and the machine reporting it applied that declaration.
func pushedAndApplied(t *testing.T, open *stores, node string) {
t.Helper()
digest := pushedBy(t, open, node)
record, err := open.inventory.NodeByName(t.Context(), node)
if err != nil {
t.Fatal(err)
}
if _, err := open.inventory.RecordDoing(t.Context(), record.ID, inventory.Doing{
Outcome: inventory.OutcomeApplied, Declared: digest}); err != nil {
t.Fatal(err)
}
}
// d1 runs D1 once.
func d1(t *testing.T, open *stores) []conditions.Observation {
t.Helper()
got, err := probeDeclarations(t.Context(), &doctor{open: open})
if err != nil {
t.Fatal(err)
}
return got
}
// **THE WINDOW, REPRODUCED**: assigned and not pushed, a module whose own secret the push makes is
// waiting, not uncomposable; past the bound it is a warning naming what the push makes; pushed, nothing.
// **A REAL FAILURE IS STILL URGENT**: a secret the push would be refused on is not one it will make.
// A bus credential nobody issued (issue 203) fails the push, so D1 says it — urgent, in the push's words.
// A given secret sealed to a key the machine no longer has is not the mesh's to make again: the push is
// refused on it, so D1 is too.
// The bound is read from when the machine began waiting: the oldest assignment since its last send.
// **D3 AND D13 WAIT FOR THE SEND**: a holder is expected to answer on a machine once the machine was sent
// it and had time to report — never between assign and push.
func TestAHolderIsExpectedOnlyOnceSentAndReported(t *testing.T) {
now := time.Now()
sent := now.Add(-time.Minute)
long := now.Add(-time.Hour)
cases := []struct {
name string
send lastSend
want bool
}{
{"never sent", lastSend{}, false},
{"sent without it", lastSend{sent: &long, current: true, carried: map[string]string{"other": "c"}}, false},
{"sent with it, not reported yet", lastSend{sent: &sent, carried: map[string]string{"keeper": "c"}}, false},
{"sent with it and reported", lastSend{sent: &sent, current: true, carried: map[string]string{"keeper": "c"}}, true},
{"sent with it long ago, never reported", lastSend{sent: &long, carried: map[string]string{"keeper": "c"}}, true},
{"sent before builds were kept", lastSend{sent: &long, current: true}, true},
}
for _, c := range cases {
if got := c.send.settled("keeper", now); got != c.want {
t.Errorf("%s: settled %v, want %v", c.name, got, c.want)
}
}
}
// And through the stores: assigned, not in the last send; pushed and reported, carried and settled.
@@ -1,308 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"os"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// aBackupSeatOnTheBus is one machine's node-backup holder as a test makes it: it counts the `now` asks it hears and
// answers `backed-up` with the last night it is told (novox/hq ADR 0404).
type aBackupSeatOnTheBus struct {
mu sync.Mutex
nows []string
night map[string]any // module → lastNight
}
func (f *aBackupSeatOnTheBus) serve(t *testing.T, conn *nats.Conn, node string) {
t.Helper()
reply := func(m *nats.Msg, result any) {
body, _ := json.Marshal(map[string]any{"result": result, "error": "", "node": node})
_ = m.Respond(body)
}
now, err := conn.Subscribe(link.NodeSeatToolSubject(catalogue.BackupSeat, "now", node), func(m *nats.Msg) {
var a struct {
Module string `json:"module"`
}
_ = json.Unmarshal(m.Data, &a)
f.mu.Lock()
f.nows = append(f.nows, a.Module)
f.mu.Unlock()
reply(m, map[string]any{"started": a.Module})
})
if err != nil {
t.Fatal(err)
}
read, err := conn.Subscribe(link.NodeSeatToolSubject(catalogue.BackupSeat, "backed-up", node), func(m *nats.Msg) {
f.mu.Lock()
defer f.mu.Unlock()
var out []map[string]any
for module, n := range f.night {
out = append(out, map[string]any{"module": module, "lastNight": n})
}
reply(m, out)
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = now.Unsubscribe(); _ = read.Unsubscribe() })
if err := conn.Flush(); err != nil {
t.Fatal(err)
}
}
func (f *aBackupSeatOnTheBus) answers(module string, ok bool, words string) {
f.mu.Lock()
defer f.mu.Unlock()
f.night = map[string]any{module: map[string]any{"ok": ok, "at": time.Now().UTC().Add(time.Second), "error": words}}
}
func (f *aBackupSeatOnTheBus) asks() int {
f.mu.Lock()
defer f.mu.Unlock()
return len(f.nows)
}
// Acceptance test 3 of design 47, on the controller's side: a delivery plan that moves a module keeping data waits
// for its backups on both machines before it asks a build; a backup that fails holds it, with the machine, the
// module and the backup's words, and a condition says the same; the verb the delivery's release calls asks the
// backups again, and the plan goes on. One bus and one store for the scenarios.
func TestADeliveryThatMovesABackedUpModuleWaitsForItsBackupAndAFailedOneHoldsItAndIsSaid(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
asked := asksRecorded(t)
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
before := handActConn
handActConn = js.Conn()
t.Cleanup(func() { handActConn = before })
if _, err := inv.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "keeper", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.BackupSeat, Scope: catalogue.ScopeNode, Serves: []string{"backed-up", "now", "restore"}}}})
notes, err := catalogue.ParseManifest([]byte(`{"module":"notes","version":"1","data":{"own":[` +
`{"id":"files","path":"${dir:files}","class":"valuable","why":"people's notes"}]},` +
`"resources":[{"id":"files","type":"directory"}]}`))
if err != nil {
t.Fatal(err)
}
if err := inv.RegisterModule(ctx, notes, inventory.Source{Repository: "novox/mesh-catalog", Seat: "git",
Path: "modules/notes", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
for _, node := range []string{"anchor", "laptop"} {
if _, err := assign(ctx, open, node, "keeper"); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, node, "notes"); err != nil {
t.Fatal(err)
}
}
seats := map[string]*aBackupSeatOnTheBus{"anchor": {}, "laptop": {}}
for node, s := range seats {
s.serve(t, js.Conn(), node)
}
// Short waits: the bound and the look are variables, so nothing here waits for the minutes of a real backup.
wasWithin, wasEvery := backupsWithin, backupsPollEvery
backupsWithin, backupsPollEvery = time.Minute, 10*time.Millisecond
t.Cleanup(func() { backupsWithin, backupsPollEvery = wasWithin, wasEvery })
newPlan := func(id string) {
t.Helper()
p := inventory.Plan{ID: id, Repository: "novox/mesh-catalog", Branch: "main", Commit: "c1", Created: time.Now().UTC(),
State: inventory.PlanBuilding, Tiers: [][]string{{"notes"}}, Modules: map[string]*inventory.PlanModule{"notes": {}}}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
}
read := func(id string) inventory.Plan {
t.Helper()
p, err := inv.PlanByID(ctx, id)
if err != nil {
t.Fatal(err)
}
return p
}
// 1. Both machines are asked; while one has not answered, no build is asked.
newPlan("plan-one")
advancePlans(ctx, open)
if seats["anchor"].asks() != 1 || seats["laptop"].asks() != 1 || len(*asked) != 0 {
t.Fatalf("asks anchor=%d laptop=%d builds=%v", seats["anchor"].asks(), seats["laptop"].asks(), *asked)
}
seats["anchor"].answers("notes", true, "")
advancePlans(ctx, open)
if len(*asked) != 0 || read("plan-one").Backups.State != inventory.BackupPending {
t.Fatalf("a tier was asked while a backup was pending: %v %+v", *asked, read("plan-one").Backups)
}
seats["laptop"].answers("notes", true, "")
advancePlans(ctx, open)
if len(*asked) != 1 || read("plan-one").Backups.State != inventory.BackupDone {
t.Fatalf("the tier was not asked once every backup was done: %v %+v", *asked, read("plan-one").Backups)
}
finish := read("plan-one")
finish.State = inventory.PlanDone
if err := inv.SavePlan(ctx, &finish); err != nil {
t.Fatal(err)
}
// 2. A backup that fails holds the plan: not failed, nothing asked, the words kept, a condition raised.
seats["anchor"].night, seats["laptop"].night = nil, nil
newPlan("plan-two")
advancePlans(ctx, open)
seats["anchor"].answers("notes", true, "")
seats["laptop"].answers("notes", false, "no space left on device")
advancePlans(ctx, open)
held := read("plan-two")
if held.State != inventory.PlanBuilding || held.Backups.State != inventory.BackupFailed || len(*asked) != 1 {
t.Fatalf("the plan was not held: %s %+v builds=%v", held.State, held.Backups, *asked)
}
var failed *inventory.PlanBackup
for i := range held.Backups.Asked {
if held.Backups.Asked[i].State == inventory.BackupFailed {
failed = &held.Backups.Asked[i]
}
}
if failed == nil || failed.Machine != "laptop" || failed.Module != "notes" || failed.Words != "no space left on device" ||
strings.Contains(held.Note, "tried again") {
t.Fatalf("the failed backup: %+v note=%q", failed, held.Note)
}
advancePlans(ctx, open) // not retried on the next look
if again := read("plan-two"); again.Backups.State != inventory.BackupFailed || len(*asked) != 1 ||
seats["laptop"].asks() != 2 {
t.Fatalf("a held plan was asked again by itself: %+v builds=%v asks=%d", again.Backups, *asked, seats["laptop"].asks())
}
said, err := probeBackupHeld(ctx, &doctor{open: open})
if err != nil || len(said) != 1 || said[0].Kind != kindBackupHeld || !strings.Contains(said[0].Said, "no space left on device") {
t.Fatalf("the condition: %+v %v", said, err)
}
// 3. The verb the release calls: the failed backup is asked again, and only it, and the plan goes on.
seats["laptop"].answers("notes", false, "no space left on device")
if err := deliveryCommand(ctx, []string{"backups-again", "plan-two", "--why", "a person released it", "--by",
"mesh-delivery for jochen"}); err != nil {
t.Fatal(err)
}
if err := deliveryCommand(ctx, []string{"backups-again", "plan-two", "--why", "again"}); err == nil {
t.Fatal("a plan with no failed backup was asked again")
}
seats["laptop"].night = nil
advancePlans(ctx, open)
if seats["laptop"].asks() != 3 || seats["anchor"].asks() != 2 {
t.Fatalf("asked again: laptop=%d anchor=%d", seats["laptop"].asks(), seats["anchor"].asks())
}
seats["laptop"].answers("notes", true, "")
advancePlans(ctx, open)
if got := read("plan-two"); got.Backups.State != inventory.BackupDone || len(*asked) != 2 {
t.Fatalf("the plan did not go on: %+v builds=%v", got.Backups, *asked)
}
if said, _ := probeBackupHeld(ctx, &doctor{open: open}); len(said) != 0 {
t.Fatalf("the condition stays: %+v", said)
}
}
// Issue 1086: a plan cut from a merge asked its first tier at the cut, so a module keeping data was sent without the
// backup the plan promised. The cut now takes the backups first (askFirstTierAtCut, batches.go), and the walk's own
// step asks the tier once they are done.
func TestAPlanCutFromAMergeOfABackedUpModuleAsksItsBackupAndNotItsTierUntilItIsDone(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
settings := t.TempDir() + "/merge-window.json"
if err := os.WriteFile(settings, []byte(`{"merge-window": "90s", "merge-window-at-most": "10m"}`), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv(mergeWindowFileVar, settings)
everyHeadWaits(t)
asked := asksRecorded(t)
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
before := handActConn
handActConn = js.Conn()
t.Cleanup(func() { handActConn = before })
if _, err := inv.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "keeper", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.BackupSeat, Scope: catalogue.ScopeNode, Serves: []string{"backed-up", "now", "restore"}}}})
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
notes, err := catalogue.ParseManifest([]byte(`{"module":"notes","version":"1","data":{"own":[` +
`{"id":"files","path":"${dir:files}","class":"valuable","why":"people's notes"}]},` +
`"resources":[{"id":"files","type":"directory"}]}`))
if err != nil {
t.Fatal(err)
}
if err := inv.RegisterModule(ctx, notes, inventory.Source{Repository: "novox/notes", Seat: "git", Ref: "main",
BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
for _, m := range []string{"keeper", "notes"} {
if _, err := assign(ctx, open, "anchor", m); err != nil {
t.Fatal(err)
}
}
seat := &aBackupSeatOnTheBus{}
seat.serve(t, js.Conn(), "anchor")
wasWithin, wasEvery := backupsWithin, backupsPollEvery
backupsWithin, backupsPollEvery = time.Minute, 10*time.Millisecond
t.Cleanup(func() { backupsWithin, backupsPollEvery = wasWithin, wasEvery })
// 1. A merge of notes is heard and its window closes: the batch is cut through cutBatch.
hear(t, open, repoMerge("notes", "c-notes", t0), t0.Add(time.Second))
cutAt(t, open, t0.Add(time.Second+90*time.Second))
ws, _ := walks(t, open)
if len(ws) != 1 {
t.Fatalf("%d walk(s)", len(ws))
}
// 2. Right after the cut no tier is asked, the record has anchor/notes pending, the seat heard one `now`.
cut, err := inv.PlanByID(ctx, ws[0].ID)
if err != nil {
t.Fatal(err)
}
if len(*asked) != 0 || askedInTier(cut) {
t.Fatalf("the first tier was asked at the cut: %v %+v", *asked, cut.Modules)
}
if cut.Backups == nil || cut.Backups.State != inventory.BackupPending || len(cut.Backups.Asked) != 1 ||
cut.Backups.Asked[0].Machine != "anchor" || cut.Backups.Asked[0].Module != "notes" {
t.Fatalf("the backups of the cut: %+v", cut.Backups)
}
if seat.asks() != 1 {
t.Fatalf("the seat heard %d `now`", seat.asks())
}
// 3. The backup answers done: the walk's own step asks the first tier, once, and asks no second backup.
seat.answers("notes", true, "")
advancePlans(ctx, open)
advancePlans(ctx, open)
after, err := inv.PlanByID(ctx, ws[0].ID)
if err != nil {
t.Fatal(err)
}
if len(*asked) != 1 || after.Backups.State != inventory.BackupDone || seat.asks() != 1 {
t.Fatalf("builds=%v backups=%+v asks=%d", *asked, after.Backups, seat.asks())
}
}
File diff suppressed because it is too large Load Diff
@@ -1,698 +0,0 @@
//go:build integration
package main
import (
"bytes"
"io"
"os"
"slices"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func windowed(t *testing.T) *stores {
t.Helper()
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
settings := t.TempDir() + "/merge-window.json"
if err := os.WriteFile(settings, []byte(`{"merge-window": "90s", "merge-window-at-most": "10m"}`), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv(mergeWindowFileVar, settings)
noGroups(t)
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Seat: "git", Ref: "main", BuiltFrom: "c0",
Head: "c0"}); err != nil {
t.Fatal(err)
}
for _, m := range []struct{ module, repository, path string }{
{"app", "novox/mesh-catalog", "modules/app"},
{"notes", "novox/mesh-catalog", "modules/notes"},
{"one", "novox/one", ""},
{"two", "novox/two", ""},
{"three", "novox/three", ""},
} {
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: m.module, Version: "1"},
inventory.Source{Repository: m.repository, Seat: "git", Path: m.path, Ref: "main", BuiltFrom: "c0",
Head: "c0"}); err != nil {
t.Fatal(err)
}
}
return open
}
func hear(t *testing.T, open *stores, m link.SourceMoved, now time.Time) {
t.Helper()
if err := (following{open: open}).hearMerge(t.Context(), m, now); err != nil {
t.Fatal(err)
}
}
func cutAt(t *testing.T, open *stores, now time.Time) {
t.Helper()
if err := cutBatchesHeld(t.Context(), open, now); err != nil {
t.Fatal(err)
}
}
func TestTwoMergesSecondsApartAreOneWalkAtTheLaterCommit(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksWithPaths(t)
claude := catalogueMerge("553b7191claude", "app", t0)
dunst := catalogueMerge("48bda475dunst", "notes", t0.Add(17*time.Second))
hear(t, open, claude, t0.Add(time.Second))
hear(t, open, dunst, t0.Add(18*time.Second))
if ws, bs := walks(t, open); len(ws) != 0 || len(bs) != 1 {
t.Fatalf("within the window: %d walk(s), %d batch(es)", len(ws), len(bs))
}
cutAt(t, open, t0.Add(18*time.Second+89*time.Second))
if ws, _ := walks(t, open); len(ws) != 0 {
t.Fatalf("cut before the window closed: %+v", ws)
}
cutAt(t, open, t0.Add(18*time.Second+90*time.Second))
ws, bs := walks(t, open)
if len(ws) != 1 || len(bs) != 0 {
t.Fatalf("after the window: %d walk(s), %d batch(es)", len(ws), len(bs))
}
w := ws[0]
if w.Commit != dunst.Commit || len(w.Commits) != 1 || w.Commits[0].Commit != dunst.Commit {
t.Fatalf("the walk is at %s %+v, not the later commit", w.Commit, w.Commits)
}
for _, m := range []string{"app", "notes"} {
if _, in := w.Modules[m]; !in {
t.Fatalf("the walk does not build %s, which one of its merges moved: %v", m, w.Modules)
}
}
want := []inventory.PlanMerge{{Repository: "novox/mesh-catalog", Commit: claude.Commit, Carried: dunst.Commit},
{Repository: "novox/mesh-catalog", Commit: dunst.Commit}}
if w.Delivery == nil || !slices.EqualFunc(w.Delivery.Merges, want, sameMerge) {
t.Fatalf("the walk answers %+v, want %+v", w.Delivery, want)
}
if len(*asked) != 2 || (*asked)[0][2] != "main" || (*asked)[1][2] != "main" {
t.Fatalf("the walk did not ask its modules at the branch, which holds the commit it carries: %v", *asked)
}
// Heard again, as the bus may hand it over twice: never a second merge, nor a second walk.
hear(t, open, claude, t0.Add(5*time.Minute))
if ws, bs := walks(t, open); len(ws) != 1 || len(bs) != 0 {
t.Fatalf("a merge heard twice made %d walk(s) and %d batch(es)", len(ws), len(bs))
}
}
func TestThreeRepositoriesInOneWindowAreOneWalk(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "feat/three")
asksRecorded(t)
for i, r := range []string{"one", "two", "three"} {
hear(t, open, fromHead(repoMerge(r, "c-"+r, t0.Add(time.Duration(i)*20*time.Second)), "feat/three"),
t0.Add(time.Duration(i)*20*time.Second+time.Second))
}
cutAt(t, open, t0.Add(41*time.Second+90*time.Second))
ws, bs := walks(t, open)
if len(ws) != 1 || len(bs) != 0 {
t.Fatalf("%d walk(s), %d batch(es)", len(ws), len(bs))
}
w := ws[0]
if len(w.Commits) != 3 || len(w.Delivery.Merges) != 3 {
t.Fatalf("the walk carries %+v and answers %+v", w.Commits, w.Delivery.Merges)
}
for _, r := range []string{"one", "two", "three"} {
if w.CommitOf("novox/"+r) != "c-"+r {
t.Fatalf("the walk carries %s at %q", r, w.CommitOf("novox/"+r))
}
if _, in := w.Modules[r]; !in {
t.Fatalf("the walk does not build %s: %v", r, w.Modules)
}
}
// Each module's ask is recorded at its own repository's commit.
for _, r := range []string{"one", "two", "three"} {
q, found, err := open.inventory.BuildRequestByID(t.Context(), w.Modules[r].Build)
if err != nil || !found || q.Commit != "c-"+r {
t.Fatalf("%s's build was not recorded at its own commit: %+v %v %v", r, q, found, err)
}
}
}
func TestAMergeRestartsTheWindowAndTheMaximumClosesIt(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "feat/both", "feat/busy")
asksWithPaths(t)
hear(t, open, fromHead(repoMerge("one", "c1", t0), "feat/both"), t0)
hear(t, open, fromHead(repoMerge("two", "c2", t0.Add(80*time.Second)), "feat/both"), t0.Add(80*time.Second))
cutAt(t, open, t0.Add(100*time.Second))
if ws, _ := walks(t, open); len(ws) != 0 {
t.Fatal("the window closed 90 seconds after its first merge, not after its last")
}
cutAt(t, open, t0.Add(170*time.Second))
if ws, _ := walks(t, open); len(ws) != 1 || len(ws[0].Commits) != 2 {
t.Fatalf("the window did not close 90 seconds after its last merge: %+v", ws)
}
busy := windowed(t)
groupsWaiting(t, "feat/both", "feat/busy")
asksWithPaths(t)
var last time.Time
for i := 0; i < 12; i++ {
last = t0.Add(time.Duration(i) * time.Minute)
repo := []string{"one", "two", "three"}[i%3]
hear(t, busy, fromHead(repoMerge(repo, "c"+string(rune('a'+i)), last), "feat/busy"), last)
if ws, _ := walks(t, busy); len(ws) > 0 {
if i != 10 {
t.Fatalf("a busy window closed at merge %d (minute %d), not at its maximum", i, i)
}
break
}
}
ws, _ := walks(t, busy)
if len(ws) != 1 {
t.Fatalf("ten minutes of merges a minute apart were never cut: %d walk(s)", len(ws))
}
}
func TestTheAssemblingBatchIsShown(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/553b7191claude")
asksWithPaths(t)
var said []inventory.Plan
was := inventory.PlanSaved
inventory.PlanSaved = func(_ *inventory.Inventory, p inventory.Plan) { said = append(said, p) }
t.Cleanup(func() { inventory.PlanSaved = was })
now := time.Now().UTC()
claude := catalogueMerge("553b7191claude", "app", now.Add(-20*time.Second))
claude.Number, claude.Title = 174, "claude-code: an agent proposes a section"
dunst := catalogueMerge("48bda475dunst", "notes", now.Add(-2*time.Second))
dunst.Number, dunst.Title = 175, "dunst: the font the operator chose"
hear(t, open, claude, now.Add(-19*time.Second))
hear(t, open, dunst, now.Add(-time.Second))
hear(t, open, fromHead(repoMerge("one", "a6bc0931one", now), dunst.Head), now)
out := captured(t, func() error { return plansCommand(t.Context(), nil) })
lines := strings.Split(out, "\n")
first := lines[0]
for _, want := range []string{"assembling: ", " s left (at the latest ", "grouped: novox/mesh-catalog@48bda475 " +
"(answers 553b7191), novox/one@a6bc0931; plan not yet calculated"} {
if !strings.Contains(first, want) {
t.Fatalf("plans' first line %q does not say %q", first, want)
}
}
// Under it, one line per repository: the pull request, what it answers, what it moves.
if len(lines) < 3 || strings.TrimSpace(lines[1]) != "mesh-catalog #175 dunst: the font the operator chose (answers "+
"#174 claude-code: an agent proposes a section) · app, notes" ||
strings.TrimSpace(lines[2]) != "one a6bc0931 · one" {
t.Fatalf("the grouped list reads %q", lines[1:4])
}
if len(said) == 0 || said[len(said)-1].State != inventory.PlanAssembling || said[len(said)-1].Delivery.Batch == nil ||
len(said[len(said)-1].Delivery.Merges) != 3 {
t.Fatalf("the batch was not said as assembling with its merges: %+v", said)
}
}
func captured(t *testing.T, run func() error) string {
t.Helper()
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
was := os.Stdout
os.Stdout = w
runErr := run()
os.Stdout = was
_ = w.Close()
var b bytes.Buffer
_, _ = io.Copy(&b, r)
if runErr != nil {
t.Fatal(runErr)
}
return b.String()
}
func TestAMergeDuringAWalkJoinsTheNextBatch(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
ws, _ := walks(t, open)
if len(ws) != 1 || !ws[0].Open() {
t.Fatalf("the first batch was not cut: %+v", ws)
}
first := ws[0]
hear(t, open, repoMerge("one", "c2", t0.Add(2*time.Minute)), t0.Add(2*time.Minute))
cutAt(t, open, t0.Add(5*time.Minute))
ws, bs := walks(t, open)
if len(ws) != 1 || len(bs) != 1 || bs[0].State != inventory.PlanQueued || bs[0].Delivery.Batch.Behind != first.ID {
t.Fatalf("the merge during the walk: %d walk(s), batches %+v", len(ws), bs)
}
if got, _ := open.inventory.PlanByID(t.Context(), first.ID); !got.Open() {
t.Fatalf("the started walk was %s by the next merge", got.State)
}
if !strings.HasPrefix(batchWords(bs[0], t0.Add(5*time.Minute)), "queued behind "+first.ID) {
t.Fatalf("a queued batch reads %q", batchWords(bs[0], t0.Add(5*time.Minute)))
}
first.State = inventory.PlanDone
if err := open.inventory.SavePlan(t.Context(), &first); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(6*time.Minute))
ws, bs = walks(t, open)
if len(ws) != 2 || len(bs) != 0 || ws[0].Commit != "c2" {
t.Fatalf("the queued batch was not cut when the walk ended: %+v %+v", ws, bs)
}
}
func TestAWaitingWalkIsFoldedIntoTheNextBatch(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
ctx := t.Context()
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-delivery", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}},
inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/mesh-delivery", Ref: "main",
BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
if _, err := open.inventory.Assign(ctx, "anchor", "mesh-delivery"); err != nil {
t.Fatal(err)
}
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
ws, _ := walks(t, open)
if len(ws) != 1 || !ws[0].Waiting() {
t.Fatalf("the walk does not wait for its word: %+v", ws)
}
waiting := ws[0]
hear(t, open, repoMerge("one", "c2", t0.Add(2*time.Minute)), t0.Add(2*time.Minute))
cutAt(t, open, t0.Add(2*time.Minute+90*time.Second))
ws, bs := walks(t, open)
if len(bs) != 0 || len(ws) != 2 {
t.Fatalf("%d walk(s), %d batch(es)", len(ws), len(bs))
}
folded, err := open.inventory.PlanByID(ctx, waiting.ID)
if err != nil {
t.Fatal(err)
}
newer := ws[0]
if folded.State != inventory.PlanSuperseded || folded.Delivery.TakenOverBy != newer.ID {
t.Fatalf("the waiting walk is %s, taken over by %q", folded.State, folded.Delivery.TakenOverBy)
}
if len(newer.Delivery.Merges) != 2 || newer.CommitOf("novox/one") != "c2" {
t.Fatalf("the newer walk answers %+v", newer.Delivery.Merges)
}
if _, in := newer.Modules["one"]; !in {
t.Fatalf("what the folded walk was to build is not built: %v", newer.Modules)
}
}
func TestALateMergeIsAnsweredByTheWalkOfTheLaterOne(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
later := catalogueMerge("48bda475dunst", "notes", t0.Add(17*time.Second))
hear(t, open, later, t0.Add(18*time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
w := ws[0]
w.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &w); err != nil {
t.Fatal(err)
}
// Moves only notes, which the done walk built from the branch after it: answered there, at once.
hear(t, open, catalogueMerge("553b7191early", "notes", t0), t0.Add(15*time.Minute))
got, _ := open.inventory.PlanByID(ctx, w.ID)
if got.State != inventory.PlanDone || len(got.Delivery.Merges) != 2 ||
!sameMerge(got.Delivery.Merges[0], inventory.PlanMerge{Repository: "novox/mesh-catalog", Commit: "553b7191early",
Carried: later.Commit}) {
t.Fatalf("the late merge is not named by the walk that carried it: %+v", got.Delivery.Merges)
}
// Moves app, which that walk never built: the next batch walks it.
hear(t, open, catalogueMerge("1111aaaaapp", "app", t0.Add(time.Second)), t0.Add(16*time.Minute))
_, bs := walks(t, open)
if len(bs) != 1 || bs[0].Delivery.Merges[0].Commit != "1111aaaaapp" {
t.Fatalf("a late merge moving what the walk never built did not join the next batch: %+v", bs)
}
}
func TestAFailedWalkWalksItsEarlierMergesAlone(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksWithPaths(t)
ctx := t.Context()
// Made before the cut marks their modules seen, as merges are: walked again, they are news all the same.
oldest := catalogueMerge("aaaa0001", "app", t0.Add(-3*time.Minute))
middle := catalogueMerge("bbbb0002", "app", t0.Add(-3*time.Minute+10*time.Second))
newest := catalogueMerge("cccc0003", "notes", t0.Add(-3*time.Minute+20*time.Second))
for i, m := range []link.SourceMoved{oldest, middle, newest} {
hear(t, open, m, t0.Add(time.Duration(i)*10*time.Second+time.Second))
}
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
failed := ws[0]
failed.State, failed.Note = inventory.PlanFailed, "notes failed to build in tier 0"
if err := open.inventory.SavePlan(ctx, &failed); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(3*time.Minute))
got, _ := open.inventory.PlanByID(ctx, failed.ID)
if got.State != inventory.PlanFailed || len(got.Delivery.Merges) != 3 {
t.Fatalf("the failed walk's record changed: %s %+v", got.State, got.Delivery.Merges)
}
ws, _ = walks(t, open)
alone := ws[0]
if alone.ID == failed.ID || alone.Commit != middle.Commit || len(alone.Delivery.Merges) != 1 ||
!sameMerge(alone.Delivery.Merges[0], inventory.PlanMerge{Repository: "novox/mesh-catalog", Commit: middle.Commit}) {
t.Fatalf("the newest earlier merge was not walked alone on its own commit: %+v", alone)
}
if _, in := alone.Modules["app"]; !in || (*asked)[len(*asked)-1] != [3]string{"novox/mesh-catalog", "modules/app",
middle.Commit} {
t.Fatalf("the merge walked alone does not build app at its own commit: %v, asked %v", alone.Modules, *asked)
}
// Retried, the failed walk would name merges the search answers now.
if _, err := retryPlan(ctx, open, failed.ID); err == nil || !strings.Contains(err.Error(), "walked alone") {
t.Fatalf("a failed walk whose merges are searched was retried: %v", err)
}
alone.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &alone); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(4*time.Minute))
ws, bs := walks(t, open)
if ws[0].ID != alone.ID || len(bs) != 0 {
t.Fatalf("the search went on after a merge was delivered: %+v", ws[0])
}
done, _ := open.inventory.PlanByID(ctx, alone.ID)
if len(done.Delivery.Merges) != 2 || !sameMerge(done.Delivery.Merges[0], inventory.PlanMerge{Repository: "novox/mesh-catalog",
Commit: oldest.Commit, Carried: middle.Commit}) {
t.Fatalf("the oldest merge is not answered by the walk that delivered the one after it: %+v", done.Delivery.Merges)
}
// A stopped walk starts no search: a person ended it.
hear(t, open, catalogueMerge("dddd0004", "app", t0.Add(5*time.Minute)), t0.Add(5*time.Minute))
hear(t, open, catalogueMerge("eeee0005", "notes", t0.Add(5*time.Minute+time.Second)), t0.Add(5*time.Minute+time.Second))
cutAt(t, open, t0.Add(8*time.Minute))
ws, _ = walks(t, open)
if _, err := stopWalk(ctx, open.inventory, ws[0].ID, "mesh-delivery for jochen", "not now"); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(9*time.Minute))
if again, _ := walks(t, open); again[0].ID != ws[0].ID {
t.Fatalf("a stopped walk's earlier merge was walked alone: %+v", again[0])
}
}
func TestAGroupsOrderBecomesTiersInsideTheWalk(t *testing.T) {
for _, grouped := range []bool{true, false} {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
for _, m := range []string{"mesh-host"} {
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: m, Version: "1"},
inventory.Source{Repository: "novox/" + m, Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
}
host := repoMerge("mesh-host", "h1", t0)
ctl := repoMerge("mesh-controller", "k1", t0.Add(time.Second))
host.Head, ctl.Head = "feat/together", "feat/together"
if !grouped {
ctl.Head = "feat/alone"
}
hear(t, open, ctl, t0.Add(2*time.Second))
hear(t, open, host, t0.Add(3*time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
if len(ws) != 1 {
t.Fatalf("%d walks", len(ws))
}
tiers := ws[0].Tiers
if grouped {
if len(tiers) != 2 || !slices.Equal(tiers[0], []string{"mesh-host"}) ||
!slices.Equal(tiers[1], []string{"mesh-controller"}) {
t.Fatalf("the group's order is not the walk's tiers: %v", tiers)
}
} else if len(tiers) != 1 {
t.Fatalf("two merges of no group were ordered: %v", tiers)
}
}
}
func TestABatchSurvivesARestart(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "feat/both")
asksWithPaths(t)
hear(t, open, fromHead(repoMerge("one", "c1", t0), "feat/both"), t0)
hear(t, open, fromHead(repoMerge("two", "c2", t0.Add(30*time.Second)), "feat/both"), t0.Add(30*time.Second))
// A new controller: nothing of the first one's but the store.
again, err := openStores(t.Context())
if err != nil {
t.Fatal(err)
}
t.Cleanup(again.Close)
hear(t, again, fromHead(repoMerge("one", "c1", t0), "feat/both"), t0.Add(60*time.Second)) // the bus hands it over again
cutAt(t, again, t0.Add(119*time.Second))
if ws, bs := walks(t, again); len(ws) != 0 || len(bs) != 1 || len(bs[0].Delivery.Merges) != 2 {
t.Fatalf("the restarted controller's batch: %d walk(s), %+v", len(ws), bs)
}
cutAt(t, again, t0.Add(120*time.Second))
ws, bs := walks(t, again)
if len(ws) != 1 || len(bs) != 0 || len(ws[0].Delivery.Merges) != 2 {
t.Fatalf("the restarted controller did not cut the batch at its window: %+v %+v", ws, bs)
}
}
func TestALateMergeOfAOneModuleRepositoryIsNamed(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
c2 := repoMerge("one", "c2", t0.Add(-50*time.Second))
hear(t, open, c2, t0)
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
w := ws[0]
w.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &w); err != nil {
t.Fatal(err)
}
hear(t, open, repoMerge("one", "c1", t0.Add(-60*time.Second)), t0.Add(15*time.Minute))
got, _ := open.inventory.PlanByID(ctx, w.ID)
if len(got.Delivery.Merges) != 2 || !sameMerge(got.Delivery.Merges[0], inventory.PlanMerge{Repository: "novox/one",
Commit: "c1", Carried: "c2"}) {
t.Fatalf("the late merge was not named by the walk that carried it: %+v", got.Delivery.Merges)
}
// A walk of two that built only two: a late merge of one is not its to name.
hear(t, open, repoMerge("two", "d2", t0.Add(16*time.Minute)), t0.Add(16*time.Minute))
cutAt(t, open, t0.Add(18*time.Minute))
ws, _ = walks(t, open)
w = ws[0]
delete(w.Modules, "two")
w.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &w); err != nil {
t.Fatal(err)
}
hear(t, open, repoMerge("two", "d1", t0.Add(15*time.Minute)), t0.Add(20*time.Minute))
if got, _ := open.inventory.PlanByID(ctx, w.ID); len(got.Delivery.Merges) != 1 {
t.Fatalf("a walk that never built what the late merge moves named it: %+v", got.Delivery.Merges)
}
if _, bs := walks(t, open); len(bs) != 1 || bs[0].Delivery.Merges[0].Commit != "d1" {
t.Fatalf("the late merge did not join the next batch: %+v", bs)
}
}
func TestAWalkLetGoBesideAStartedOneStartsOnceItEnded(t *testing.T) {
open := windowed(t)
asked := asksWithPaths(t)
ctx := t.Context()
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(2*time.Minute))
started, _ := walks(t, open)
waiting := inventory.Plan{ID: "plan-waiting", Repository: "novox/one", Branch: "main", Commit: "d1", Created: t0,
State: inventory.PlanBuilding, Tiers: [][]string{{"one"}}, Modules: map[string]*inventory.PlanModule{"one": {}},
Delivery: &inventory.PlanDelivery{Awaits: catalogue.DeliverySeat}}
if err := open.inventory.SavePlan(ctx, &waiting); err != nil {
t.Fatal(err)
}
if _, err := letGo(ctx, open.inventory, waiting.ID, catalogue.DeliverySeat, "its turn"); err != nil {
t.Fatalf("the word was refused beside %s: %v", started[0].ID, err)
}
before := len(*asked)
advanceHeld(ctx, open)
got, _ := open.inventory.PlanByID(ctx, waiting.ID)
if len(*asked) != before || !strings.Contains(got.Note, "starts once "+started[0].ID) {
t.Fatalf("a walk let go beside a started one asked (%d → %d) or does not say it waits: %q", before, len(*asked), got.Note)
}
// Read as a wait, an hour on: its note on the line, never LATE, and no tier of it late for S3.
later := time.Now().Add(time.Hour)
if line := planLine(got, later); !strings.Contains(line, "starts once "+started[0].ID) || strings.Contains(line, "LATE") {
t.Fatalf("a deferred walk reads %q", line)
}
facts, _, err := gatherPlans(ctx, open.inventory, later, nil)
if err != nil {
t.Fatal(err)
}
for _, f := range facts {
if f.id == got.ID {
t.Fatalf("a deferred walk is watched as a tier running late: %+v", f)
}
}
done := started[0]
done.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &done); err != nil {
t.Fatal(err)
}
advanceHeld(ctx, open)
if len(*asked) != before+1 {
t.Fatalf("the walk did not start once the started one ended: asked %v", *asked)
}
}
func TestAMergeOnTheControllersPathNeverSharesABatch(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksWithPaths(t)
ctx := t.Context()
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-delivery", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}},
inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/mesh-delivery", Ref: "main",
BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
if _, err := open.inventory.Assign(ctx, "anchor", "mesh-delivery"); err != nil {
t.Fatal(err)
}
hear(t, open, catalogueMerge("aaaa0001", "app", t0), t0)
hear(t, open, repoMerge("mesh-controller", "k1", t0.Add(10*time.Second)), t0.Add(10*time.Second))
hear(t, open, catalogueMerge("bbbb0002", "notes", t0.Add(20*time.Second)), t0.Add(20*time.Second))
_, bs := walks(t, open)
var ownBatch, catalogueBatch inventory.Plan
for _, b := range bs {
if b.OwnPath() {
ownBatch = b
} else {
catalogueBatch = b
}
}
if len(bs) != 2 || ownBatch.ID == "" || catalogueBatch.ID == "" || len(catalogueBatch.Delivery.Merges) != 2 ||
len(ownBatch.Delivery.Merges) != 1 {
t.Fatalf("a controller merge and two catalogue merges in one window: %+v", bs)
}
if !strings.Contains(batchWords(ownBatch, t0.Add(30*time.Second)), "own path") {
t.Fatalf("the own-path batch does not say so: %q", batchWords(ownBatch, t0.Add(30*time.Second)))
}
// Both windows closed, the controller's batch is cut first and starts; the catalogue batch queues behind it,
// is cut when the controller's walk ended, and waits for its word with both merges.
cutAt(t, open, t0.Add(2*time.Minute))
ws, bs := walks(t, open)
if len(ws) != 1 || ws[0].Waiting() || ws[0].CommitOf("novox/mesh-controller") != "k1" {
t.Fatalf("the controller's batch was not cut first into a started walk: %+v", ws)
}
ownWalk := ws[0]
for _, m := range []string{"app", "notes"} {
if _, in := ownWalk.Modules[m]; in {
t.Fatalf("the controller's walk builds %s, a catalogue module: it skipped its turn", m)
}
}
if len(bs) != 1 || bs[0].State != inventory.PlanQueued || bs[0].Delivery.Batch.Behind != ownWalk.ID || bs[0].OwnPath() {
t.Fatalf("the catalogue batch does not queue behind the controller's walk: %+v", bs)
}
ownWalk.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &ownWalk); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(3*time.Minute))
ws, bs = walks(t, open)
if len(bs) != 0 || ws[0].ID != catalogueBatch.ID || !ws[0].Waiting() || len(ws[0].Delivery.Merges) != 2 {
t.Fatalf("the catalogue batch was not cut into a waiting walk once the controller's ended: %+v %+v", ws, bs)
}
for _, m := range []string{"app", "notes"} {
if _, in := ws[0].Modules[m]; !in {
t.Fatalf("the catalogue walk does not build %s: %v", m, ws[0].Modules)
}
}
for _, a := range *asked {
if a[1] == "modules/app" || a[1] == "modules/notes" {
t.Fatalf("a catalogue module was asked without the word: %v", *asked)
}
}
// A catalogue walk waiting for its word when an own-path batch is cut keeps waiting beside the own-path
// walk, is not folded into it, and its word is taken meanwhile: it starts once the own-path walk ended.
catalogueWalk := ws[0]
hear(t, open, repoMerge("mesh-controller", "k2", t0.Add(4*time.Minute)), t0.Add(4*time.Minute))
cutAt(t, open, t0.Add(6*time.Minute))
ws, _ = walks(t, open)
kept, _ := open.inventory.PlanByID(ctx, catalogueWalk.ID)
if !kept.Waiting() || ws[0].CommitOf("novox/mesh-controller") != "k2" || ws[0].Waiting() {
t.Fatalf("the waiting catalogue walk was not kept waiting beside the controller's walk: %s %+v", kept.State, ws)
}
if _, in := ws[0].Modules["app"]; in {
t.Fatalf("the controller's walk folded the waiting catalogue walk in: %v", ws[0].Modules)
}
if _, err := letGo(ctx, open.inventory, catalogueWalk.ID, catalogue.DeliverySeat, "its turn"); err != nil {
t.Fatal(err)
}
before := len(*asked)
advanceHeld(ctx, open)
if len(*asked) != before {
t.Fatalf("the catalogue walk started beside the controller's: asked %v", (*asked)[before:])
}
own2 := ws[0]
own2.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &own2); err != nil {
t.Fatal(err)
}
advanceHeld(ctx, open)
if len(*asked) < before+1 || (*asked)[before][1] != "modules/app" {
t.Fatalf("the catalogue walk did not start once the controller's ended: %v", (*asked)[before:])
}
}
func TestALateCatalogueMergeIsNotAnsweredByAnOwnPathWalk(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
for _, m := range []struct {
module string
claims []catalogue.Claim
}{
{"nats", nil},
{"mesh-delivery", []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}},
} {
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: m.module, Version: "1", Claims: m.claims},
inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/" + m.module, Ref: "main",
BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
}
if _, err := open.inventory.Assign(ctx, "anchor", "mesh-delivery"); err != nil {
t.Fatal(err)
}
mixed := link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main", Commit: "m1xed", MergedAt: t0.Format(time.RFC3339Nano),
Paths: []string{"modules/nats/module.json", "modules/app/module.json"}, ModuleDirs: []string{"modules/nats", "modules/app"},
ModuleDirsSaid: true}
hear(t, open, mixed, t0.Add(time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
if len(ws) != 1 || ws[0].Waiting() {
t.Fatalf("the mixed merge's walk waited, or was not cut: %+v", ws)
}
done := ws[0]
done.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &done); err != nil {
t.Fatal(err)
}
hear(t, open, catalogueMerge("ear1ier", "app", t0.Add(-30*time.Second)), t0.Add(3*time.Minute))
got, _ := open.inventory.PlanByID(ctx, done.ID)
_, bs := walks(t, open)
if len(got.Delivery.Merges) != 1 || len(bs) != 1 || bs[0].OwnPath() || bs[0].Delivery.Merges[0].Commit != "ear1ier" {
t.Fatalf("the late catalogue merge was answered by the own-path walk (%+v) rather than the next catalogue batch (%+v)",
got.Delivery.Merges, bs)
}
}
-293
View File
@@ -1,293 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"os"
"slices"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The suite's tests of a merge, but for the merge window's own (this file's), plan the merge as it is heard: a
// window of nothing closes at once, so a merge with no walk open is cut into its walk at once, as every merge
// was planned before novox/hq ADR 0276. The window's tests set it through the controller's settings.
func init() {
mergeWindowDefault = 0
// No forge in a test: no manifest is found at any commit, so a merge draws no edge of the manifests' change
// (delivery_edges.go). A test that wants some replaces these (forgeHolding).
manifestAt = func(context.Context, string, string, string, string) (catalogue.Manifest, bool, error) {
return catalogue.Manifest{}, false, nil
}
parentOf = func(context.Context, string, string, string) (string, error) { return "", nil }
}
// windowed is a mesh whose controller's settings give a merge window of 90 seconds and at most 10 minutes, as its
// settings file says them, and
// modules built from the catalogue (app, notes) and from three repositories of their own.
// t0 is the moment a test's clock starts: merges are dated by it, and heard after it.
var t0 = time.Now().UTC().Truncate(time.Second)
// catalogueMerge is a merge of the catalogue changing one module's directory, made at a moment, from a pull
// request's head branch of its own, as the forge announces every merge. Whether its head names a delivery group
// with a member still to merge is mesh-delivery's word, which windowed stubs as no group at all (noGroups); a test
// whose merges wait for the window as a group's says so (groupsWaiting).
func catalogueMerge(commit, module string, made time.Time) link.SourceMoved {
return link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main", Head: "pr/" + commit, Commit: commit,
MergedAt: made.Format(time.RFC3339Nano), Paths: []string{"modules/" + module + "/module.json"},
ModuleDirs: []string{"modules/" + module}, ModuleDirsSaid: true}
}
// repoMerge is a merge of a repository of one module's own, from a head branch of its own (catalogueMerge).
func repoMerge(repo, commit string, made time.Time) link.SourceMoved {
return link.SourceMoved{Owner: "novox", Repo: repo, Base: "main", Head: "pr/" + commit, Commit: commit,
MergedAt: made.Format(time.RFC3339Nano), Paths: []string{"main.go"}}
}
// noGroups stubs mesh-delivery's word on groups for a test: no head names a group.
func noGroups(t *testing.T) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(context.Context, bool, string) ([]groupMember, bool, error) { return nil, false, nil }
t.Cleanup(func() { deliveryGroupOf = was })
}
// groupsWaiting stubs mesh-delivery's word on groups for a test of the merge window's own timing: each head named
// is one whose group mesh-delivery cannot say, so a merge from it waits for the window (ADR 0306 decision 2) and
// nothing holds the batch open; every other head names no group. A group with a member still to merge, which holds
// its batch open however long it takes (ADR 0418 clause 3), is groupStates'.
func groupsWaiting(t *testing.T, heads ...string) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(_ context.Context, _ bool, head string) ([]groupMember, bool, error) {
if slices.Contains(heads, head) {
return nil, false, errors.New("the test's mesh-delivery does not say")
}
return nil, false, nil
}
t.Cleanup(func() { deliveryGroupOf = was })
}
// everyHeadWaits stubs mesh-delivery's word on groups for a test whose merges each wait for the window as they did
// before novox/hq ADR 0304: mesh-delivery cannot say any head's group.
func everyHeadWaits(t *testing.T) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(context.Context, bool, string) ([]groupMember, bool, error) {
return nil, false, errors.New("the test's mesh-delivery does not say")
}
t.Cleanup(func() { deliveryGroupOf = was })
}
// groupStates stubs mesh-delivery's word on one group: its members and their states, which a test changes as the
// members merge.
func groupStates(t *testing.T, head string, states map[string]string) {
t.Helper()
groupStatesFrom(t, head, states, true)
}
// groupStatesFrom is groupStates for a mesh-delivery that does (composed) or does not yet say the group's gate is the
// composed check's: only the first holds the group's batch open for its last member.
func groupStatesFrom(t *testing.T, head string, states map[string]string, composed bool) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(_ context.Context, _ bool, asked string) ([]groupMember, bool, error) {
if asked != head {
return nil, false, nil
}
var members []groupMember
for repository, state := range states {
members = append(members, groupMember{Repository: repository, State: state, Composed: composed})
}
return members, true, nil
}
t.Cleanup(func() { deliveryGroupOf = was })
}
// fromHead is a merge from a head branch a test names: merges from one head are members of one delivery group,
// related by it (novox/hq ADR 0304 decision 1), so they share a batch whatever else they share.
func fromHead(m link.SourceMoved, head string) link.SourceMoved {
m.Head = head
return m
}
// lone is a merge that names no head: known to be no delivery group's, it waits for no window unless it is
// related to a batch assembling (ADR 0304 decision 1).
func lone(m link.SourceMoved) link.SourceMoved {
m.Head = ""
return m
}
// hear is a merge heard at a moment.
// cutAt is the cutter looking at a moment.
// walks is every plan record that is a walk, newest first, and the batches not yet cut.
func walks(t *testing.T, open *stores) (walks, batches []inventory.Plan) {
t.Helper()
recent, err := open.inventory.RecentPlans(t.Context(), 50)
if err != nil {
t.Fatal(err)
}
for _, p := range recent {
if p.Batch() {
batches = append(batches, p)
} else {
walks = append(walks, p)
}
}
return walks, batches
}
// The replay of issue 362: two catalogue merges 18 seconds apart are one batch, cut once into one walk at the
// later commit, which names both merges — the earlier carried by the later.
// Merges of three repositories in one window are one walk, one commit for each.
// Each merge restarts the window: a merge at second 80 keeps the batch open until second 170; the window
// closes at most ten minutes after its first merge however busy.
// `plans` lists the batch being assembled first, in the operator's words: how long is left, when at the latest,
// what is grouped, and that the plan is not yet calculated. It says so on the bus as plan-moved, too.
// captured is what a command printed.
// A merge heard during a walk it is related to (the same repository, issue 973) joins the next batch, which is
// cut when the walk ends; the walk that started is never superseded (ADR 0276 decision 4, ADR 0407 decision 4).
// A walk waiting for its delivery's word is folded into the next batch's walk, which answers its merges; it
// names the walk that took it over.
// A merge heard after a later merge of its branch was walked is answered by that walk when it builds all it
// moves — named at once on a walk done — and joins the next batch when it does not.
// A failed walk marks nothing delivered: its earlier merges are walked alone, on their own commit, the newest
// first; the first delivered answers the older ones, and the search stops.
// A delivery group's order holds inside the walk (ADR 0249 unchanged): merges of two repositories from one head
// branch name, the node-engine's before the controller's, are tiered so; without the group, one tier.
// A restarted controller resumes the window where it stood: the batch, its merges and their times are in the
// store, a merge handed over again is not doubled, and the batch is cut when its window closes.
// The window's settings reach the controller in its settings file, read at every look; one that says no
// duration, or no file, is the default.
func TestTheWindowIsTheControllersSetting(t *testing.T) {
file := t.TempDir() + "/merge-window.json"
t.Setenv(mergeWindowFileVar, file)
for _, c := range []struct {
content string
window, atMost time.Duration
}{
{`{"merge-window": "60s", "merge-window-at-most": "5m"}`, time.Minute, 5 * time.Minute},
{`{"merge-window": 30, "merge-window-at-most": ""}`, 30 * time.Second, mergeWindowAtMostDefault},
{`{"merge-window": "soon"}`, mergeWindowDefault, mergeWindowAtMostDefault},
} {
if err := os.WriteFile(file, []byte(c.content), 0o600); err != nil {
t.Fatal(err)
}
if w, m := mergeWindowOf(); w != c.window || m != c.atMost {
t.Errorf("%s reads as %s and %s, want %s and %s", c.content, w, m, c.window, c.atMost)
}
}
t.Setenv(mergeWindowFileVar, file+".gone")
if w, m := mergeWindowOf(); w != mergeWindowDefault || m != mergeWindowAtMostDefault {
t.Errorf("no file reads as %s and %s", w, m)
}
}
// S16 names every merge a waiting walk answers, never one commit a supersession may have ended (issue 362).
func TestAWaitingWalkNamesTheMergesItAnswers(t *testing.T) {
f := calm(t0)
f.waits = []waitFacts{{id: "plan-2", repository: "novox/mesh-catalog", commit: "48bda475dunst", awaits: "mesh-delivery",
since: t0.Add(-31 * time.Minute), merges: []inventory.PlanMerge{
{Repository: "novox/mesh-catalog", Commit: "553b7191claude", Carried: "48bda475dunst"},
{Repository: "novox/mesh-catalog", Commit: "48bda475dunst"}}}}
got := watchWaits(f)
if len(got) != 1 || !strings.Contains(got[0].Summary, "novox/mesh-catalog@553b7191") ||
!strings.Contains(got[0].Summary, "novox/mesh-catalog@48bda475") {
t.Fatalf("the waiting walk does not name both merges: %+v", got)
}
}
// A merge heard after a later merge of its branch was cut is named by that walk however its modules read since
// the cut, in a repository of one module (review of this change); a walk that never built what it moves names
// nothing, and the merge joins the next batch.
// A related walk waits for the delivery's word too: a waiting walk let go beside a started one it is related to
// (the same repository, issue 973) takes the word and starts once that one ended, asking nothing before.
// A file a merge of the batch removed and a later one brought back is not removed; one removed last is.
func TestARemovedFileIsTheLastMergesWord(t *testing.T) {
ev := func(commit string, paths, removed []string) inventory.BatchedMerge {
m := link.SourceMoved{Owner: "novox", Repo: "one", Base: "main", Commit: commit, Paths: paths, Removed: removed}
b, _ := json.Marshal(m)
return inventory.BatchedMerge{Repository: "novox/one", Branch: "main", Commit: commit, Event: b,
Merged: t0.Add(time.Duration(len(commit)) * time.Second)}
}
got := combinedMerges([]inventory.BatchedMerge{
ev("a", []string{"x/module.json", "y/module.json"}, []string{"x/module.json", "y/module.json"}),
ev("bb", []string{"x/module.json"}, nil),
ev("ccc", []string{"z.go"}, nil)})
if len(got) != 1 || got[0].Commit != "ccc" || !slices.Equal(got[0].Removed, []string{"y/module.json"}) ||
len(got[0].Paths) != 3 {
t.Fatalf("combined as %+v", got)
}
}
// A merge on the controller's own path never shares a batch with one that waits for mesh-delivery's word
// (decided during the build, 2026-10-10): the two are batches of their own, cut one after the other; a walk on
// the own path folds no waiting catalogue walk but batches its merges again behind it; the catalogue's walk
// still waits for the word, so no catalogue delivery skips its turn.
// `plans` names a walk by what it moves (asked by the operator, 2026-10-10): the repository's pull request and
// title, the modules it moves and the machines running them, then the state words; a record naming no pull
// request is named by its commit, and a title is cut at fifty runes.
func TestAPlanLineNamesWhatItMoves(t *testing.T) {
now := time.Date(2026, 10, 10, 12, 0, 0, 0, time.UTC)
p := inventory.Plan{ID: "plan-1", Repository: "novox/mesh-catalog", Branch: "main", Commit: "c1c1c1c1c1",
State: inventory.PlanBuilding, Tiers: [][]string{{"messenger", "telegram"}}, TierEntered: now.Add(-2 * time.Minute),
Modules: map[string]*inventory.PlanModule{"messenger": {State: "asked"}, "telegram": {State: "asked"}},
Commits: []inventory.PlanCommit{{Repository: "novox/mesh-catalog", Branch: "main", Commit: "c1c1c1c1c1"}},
Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{{Repository: "novox/mesh-catalog",
Commit: "c1c1c1c1c1", Number: 175, Title: "a tap shows its outcome", Moves: []string{"telegram", "messenger"}}}}}
running := func(module string) []string {
if module == "messenger" || module == "telegram" {
return []string{"novox"}
}
return nil
}
if got, want := planLineOn(p, now, pauseView{}, tierAtLeast, running),
"mesh-catalog #175 a tap shows its outcome · messenger, telegram → novox · tier 1 of 1, building for 2m0s"; got != want {
t.Fatalf("the line reads %q, want %q", got, want)
}
if got := planLine(p, now); !strings.HasPrefix(got, "mesh-catalog #175 a tap shows its outcome · messenger, telegram · tier") {
t.Fatalf("without the machines the line reads %q", got)
}
old := p
old.Commits, old.Delivery = nil, nil
if got := planLine(old, now); !strings.HasPrefix(got, "mesh-catalog c1c1c1c1 · tier 1 of 1") {
t.Fatalf("a record naming no pull request reads %q", got)
}
long := p
long.Delivery.Merges[0].Title = strings.Repeat("abcdefghij", 6)
if got := planHeadline(long, nil); !strings.Contains(got, "#175 "+strings.Repeat("abcdefghij", 4)+"abcdefghi…") {
t.Fatalf("a long title is not cut at fifty runes: %q", got)
}
}
// sameMerge compares what a walk names of a merge: its repository, commit and the commit carrying it.
func sameMerge(a, b inventory.PlanMerge) bool {
return a.Repository == b.Repository && a.Commit == b.Commit && a.Carried == b.Carried
}
// A catalogue merge heard after a later merge of its branch was walked without the word (a merge that touched
// the bus too, on the controller's own path) is not answered by that walk while the delivery seat has a holder:
// its delivery would skip its turn. It joins the next catalogue batch.
-153
View File
@@ -1,153 +0,0 @@
package main
import (
"context"
"fmt"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// **Every wait of a batch has a bound and a condition** (novox/hq ADR 0276 decision 9): a batch still
// assembling a minute past its maximum while no walk is open is a cut the controller failed to make (S18),
// and a batch waiting behind an open walk longer than that walk's bound waits on a walk gone wrong (S19).
// The kinds S18 and S19 raise.
const (
kindBatchNotCut = "batch-not-cut"
kindBatchBehindWalk = "batch-behind-walk"
// kindGroupNotMerged is raised for a group whose last member has not merged by the merge window's maximum after
// its first (novox/hq ADR 0418 clause 3).
kindGroupNotMerged = "group-not-merged"
)
// batchFacts is one batch not yet cut, as the watchdogs read it.
type batchFacts struct {
id, state, grouped string
// atMost is when its window closes at the latest; closed when it closed.
atMost, closed time.Time
// behind is the open walk it waits behind, and walkBound that walk's bound: a tier's bound for each of its
// tiers.
behind string
walkBound time.Duration
// holds are the groups the batch is held for, by head, with why; created is when the batch was made, the first
// merge of the group heard; atMostWindow is the maximum a window may take (the controller's setting).
holds map[string]string
created time.Time
atMostWindow time.Duration
}
// gatherBatches is every batch not yet cut, with the bound of the walk it waits behind.
func gatherBatches(ctx context.Context, inv *inventory.Inventory, now time.Time) ([]batchFacts, error) {
batches, err := inv.Batches(ctx)
if err != nil || len(batches) == 0 {
return nil, err
}
bounds, err := measuredTierBounds(ctx, inv, now)
if err != nil {
return nil, err
}
var out []batchFacts
for _, b := range batches {
_, atMostWindow := mergeWindowOf()
f := batchFacts{id: b.ID, state: b.State, grouped: groupedWords(b), created: b.Created, atMostWindow: atMostWindow}
if w := b.Delivery; w != nil && w.Batch != nil {
f.holds = w.Batch.Holds
f.atMost, f.closed, f.behind = w.Batch.AtMost, w.Batch.ClosesAt, w.Batch.Behind
if f.atMost.Before(f.closed) {
f.closed = f.atMost
}
if w.Batch.CutAtOnce && !b.Created.IsZero() {
// Its window closed when it was made, which is its latest too (ADR 0304 decision 1).
f.atMost, f.closed = b.Created, b.Created
}
}
if f.behind != "" {
if walk, err := inv.PlanByID(ctx, f.behind); err == nil {
f.walkBound = bounds.of(walk.Repository) * time.Duration(max(1, len(walk.Tiers)))
}
}
out = append(out, f)
}
return out, nil
}
// watchBatchesNotCut is S18: a batch still assembling a minute past its maximum, while no walk is open.
func watchBatchesNotCut(f *signalFacts) []conditions.Observation {
var out []conditions.Observation
for _, b := range f.batches {
if b.state != inventory.PlanAssembling || b.atMost.IsZero() || f.now.Sub(b.atMost) <= batchLateAfter ||
len(b.holds) > 0 { // a batch held for a group waits on a member, which has its own condition
continue
}
late := f.now.Sub(b.atMost)
out = append(out, conditions.Observation{Scope: conditions.ScopePlan, ID: b.id, Kind: kindBatchNotCut,
Severity: conditions.Warning,
Summary: fmt.Sprintf("the batch %s is still assembling %s past its latest close (%s), with no walk open: "+
"the controller did not cut it; grouped: %s", b.id, ago(late), b.atMost.UTC().Format(time.RFC3339), b.grouped),
Said: fmt.Sprintf("assembling since %s past its maximum", ago(late)),
Headline: "Merged changes are not being delivered",
Explanation: fmt.Sprintf("Merges collected for one delivery should have been planned %s ago and were not. "+
"Nothing is lost; the mesh keeps them until it plans them.", humanDuration(late)),
Resolved: "Merged changes are being delivered again"})
}
return out
}
// watchBatchesBehind is S19: a batch waiting behind an open walk longer than that walk's bound, naming it.
func watchBatchesBehind(f *signalFacts) []conditions.Observation {
var out []conditions.Observation
for _, b := range f.batches {
if b.state != inventory.PlanQueued || b.behind == "" || b.walkBound <= 0 || f.now.Sub(b.closed) <= b.walkBound {
continue
}
in := f.now.Sub(b.closed)
out = append(out, conditions.Observation{Scope: conditions.ScopePlan, ID: b.id, Kind: kindBatchBehindWalk,
Severity: conditions.Warning,
Summary: fmt.Sprintf("the batch %s has waited %s behind the walk %s, longer than that walk's bound (%s); "+
"grouped: %s; `plans %s` says where that walk stands", b.id, ago(in), b.behind, ago(b.walkBound),
b.grouped, b.behind),
Said: fmt.Sprintf("queued behind %s for %s", b.behind, ago(in)),
Headline: "Merged changes wait behind a slow delivery",
Explanation: fmt.Sprintf("Merges collected for the next delivery have waited %s for the delivery before "+
"them, which is taking longer than it should. Nothing is lost.", humanDuration(in)),
Resolved: "Merged changes no longer wait behind a slow delivery"})
}
return out
}
// watchGroupsNotMerged is the condition of a batch held for a group whose last member has not merged by the merge
// window's maximum after the first member merged, naming the member and why (novox/hq ADR 0418 clause 3): nothing of
// the group is delivered while it waits.
func watchGroupsNotMerged(f *signalFacts) []conditions.Observation {
var out []conditions.Observation
for _, b := range f.batches {
if len(b.holds) == 0 || b.created.IsZero() || f.now.Sub(b.created) <= b.atMostWindow {
continue
}
heads := make([]string, 0, len(b.holds))
for h := range b.holds {
heads = append(heads, h)
}
sort.Strings(heads)
var why []string
for _, h := range heads {
why = append(why, b.holds[h])
}
waited := f.now.Sub(b.created)
out = append(out, conditions.Observation{Scope: conditions.ScopePlan, ID: b.id,
Kind: kindGroupNotMerged, Severity: conditions.Warning,
Summary: fmt.Sprintf("%s; its first member merged %s ago, past the merge window's maximum (%s), and nothing "+
"of the group is delivered while it waits (batch %s)", strings.Join(why, "; "), ago(waited), ago(b.atMostWindow), b.id),
Said: "a group waits for a member that has not merged",
Headline: "A change waits for the rest of its group to merge",
Explanation: fmt.Sprintf("The changes of a group are delivered together, once the last one is merged. "+
"One has not merged in %s. Nothing is lost; the merged ones are kept and delivered with it.",
humanDuration(waited)),
Resolved: "The group's last change merged"})
}
return out
}
-108
View File
@@ -1,108 +0,0 @@
//go:build integration
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// An unrelated closed batch is cut into a walk beside an open one, and that walk asks its first tier at once
// (issue 973, novox/hq ADR 0407 decision 4): a hotfix is never held by an unrelated delivery.
func TestAnUnrelatedBatchWalksBesideAnOpenWalk(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
ctx := t.Context()
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
if ws, _ := walks(t, open); len(ws) != 1 || !ws[0].Open() {
t.Fatalf("the first walk is not open: %+v", ws)
}
heard := t0.Add(100 * time.Second)
hear(t, open, lone(repoMerge("two", "c2", heard)), heard)
advanceHeld(ctx, open)
ws, bs := walks(t, open)
if len(ws) != 2 || len(bs) != 0 {
t.Fatalf("%d walk(s) and %d batch(es); want the unrelated batch cut beside the open walk", len(ws), len(bs))
}
for _, w := range ws {
if !w.Open() || w.Waiting() || deferred(w) || !askedAny(w) {
t.Errorf("%s (%s) is not walking: open %v, deferred %v, asked %v, note %q", w.ID, w.Named(), w.Open(),
deferred(w), askedAny(w), w.Note)
}
}
}
// A closed batch related to an open walk is queued behind it with the reason, while one that shares nothing is
// cut beside it; the walk that started is never superseded (ADR 0276 decision 4, issue 973).
func TestARelatedBatchWaitsBehindTheOpenWalk(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
ctx := t.Context()
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
first := func() inventory.Plan { ws, _ := walks(t, open); return ws[len(ws)-1] }()
heard := t0.Add(100 * time.Second)
hear(t, open, lone(repoMerge("one", "c2", heard)), heard)
hear(t, open, lone(repoMerge("three", "c3", heard.Add(time.Second))), heard.Add(time.Second))
ws, bs := walks(t, open)
if len(ws) != 2 || len(bs) != 1 {
t.Fatalf("%d walk(s) and %d batch(es); want the unrelated merge walking and the related one queued", len(ws), len(bs))
}
b := bs[0]
words := batchWords(b, heard)
if b.State != inventory.PlanQueued || b.Delivery.Batch.Behind != first.ID || b.CommitOf("novox/one") != "c2" ||
!strings.Contains(words, "queued behind "+first.ID) || !strings.Contains(words, "both change novox/one") {
t.Fatalf("the related batch is %s, behind %q, reading %q", b.State, b.Delivery.Batch.Behind, words)
}
if got, _ := open.inventory.PlanByID(ctx, first.ID); !got.Open() {
t.Fatalf("the started walk was %s by the related merge", got.State)
}
first.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &first); err != nil {
t.Fatal(err)
}
cutAt(t, open, heard.Add(2*time.Second))
if _, bs := walks(t, open); len(bs) != 0 {
t.Fatalf("the queued batch was not cut once the walk it waited for ended: %+v", bs)
}
}
// A batch on the controller's own path never walks beside another walk, and nothing walks beside it, whatever
// the two share (issue 973): it replaces the controller, so it goes alone.
func TestAnOwnPathBatchNeverWalksBeside(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
ctx := t.Context()
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
ws, _ := walks(t, open)
first := ws[0]
heard := t0.Add(100 * time.Second)
hear(t, open, lone(repoMerge("mesh-controller", "k1", heard)), heard)
ws, bs := walks(t, open)
if len(ws) != 1 || len(bs) != 1 || !bs[0].OwnPath() || bs[0].State != inventory.PlanQueued ||
bs[0].Delivery.Batch.Behind != first.ID || !strings.Contains(batchWords(bs[0], heard), "own path") {
t.Fatalf("an own-path batch beside an open walk: %d walk(s), batches %+v", len(ws), bs)
}
first.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &first); err != nil {
t.Fatal(err)
}
cutAt(t, open, heard.Add(time.Second))
ws, _ = walks(t, open)
if len(ws) != 2 || !ws[0].Open() || ws[0].CommitOf("novox/mesh-controller") != "k1" {
t.Fatalf("the own-path batch was not cut once the walk ended: %+v", ws)
}
own := ws[0]
// Unrelated to the own-path walk by every footprint, a merge still waits for it.
later := heard.Add(time.Minute)
hear(t, open, lone(repoMerge("two", "c2", later)), later)
ws, bs = walks(t, open)
if len(ws) != 2 || len(bs) != 1 || bs[0].State != inventory.PlanQueued || bs[0].Delivery.Batch.Behind != own.ID {
t.Fatalf("a merge beside the own-path walk: %d walk(s), batches %+v", len(ws), bs)
}
}
-140
View File
@@ -1,140 +0,0 @@
package main
import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
)
// A binding to data moves only by a person (novox/hq ADR 0232, issue 273).
//
// The resolver keeps each consumer of a provision that keeps its data at the provider it was last
// sent (catalogue/bound.go) and says what it would have moved. This is where that is said: on the
// push that composed it, and by the self-check's D12 every run, as an urgent condition naming the
// consumer, both providers and the pin that confirms the move.
// The condition kinds of D12.
const (
// kindBindingKept is a move the resolver refused: the consumer is still where its data is.
kindBindingKept = "binding-kept"
// kindBindingMoved is a consumer about to be sent another provider than the one on record with
// no pin naming it — what the resolver exists to make impossible, said if it ever is not.
kindBindingMoved = "binding-moved"
// kindBindingMoving is a move a pin asked for, not yet sent: a person's act, said so that the
// data is moved before the push that carries it.
kindBindingMoving = "binding-moving"
)
// probeBindingsID is the self-check's id for this probe, and the source of what it raises.
const probeBindingsID = "D12"
// keptObservation is the urgent condition for one refused move.
func keptObservation(k catalogue.KeptBinding) conditions.Observation {
return conditions.Observation{Scope: conditions.ScopeMachine, ID: bindingID(k.Machine, k.Consumer, k.Provision),
Token: kindBindingKept, Kind: kindBindingKept, Machine: k.Machine, Also: otherMachines(k.Machine, k.Bound.Node, k.Would.Node),
Severity: conditions.Urgent, Resolver: conditions.ResolverOperator,
Summary: fmt.Sprintf("on %s, the mesh %s", k.Machine, k.String())}
}
func bindingID(machine, consumer, provision string) string {
return machine + "." + consumer + "." + provision
}
func otherMachines(machine string, nodes ...string) []string {
var out []string
seen := map[string]bool{machine: true}
for _, n := range nodes {
if n != "" && !seen[n] {
seen[n] = true
out = append(out, n)
}
}
return out
}
// reportKept says every move a machine's resolution refused, on the push composing it, and raises its
// condition at once where this process keeps the conditions: a push is when a person is looking.
func reportKept(ctx context.Context, plan catalogue.Resolution) {
for _, k := range plan.Kept {
fmt.Printf("%s: the mesh %s\n", plan.Node, k)
if conditionsFrom != nil {
o := keptObservation(k)
o.Source = probeBindingsID
if _, err := conditionsFrom.Observe(ctx, o); err != nil {
fmt.Printf("%s: and the condition for it could not be raised: %v\n", plan.Node, err)
}
}
}
}
// probeBindings is D12: every consumer of a provision that keeps its data is bound where it was last
// sent, on every machine — the resolver kept it there (said, urgent, until a person pins), or a pin
// moves it (said, so the data goes first), and never anything else.
func probeBindings(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
inv := d.open.inventory
nodes, err := inv.Nodes(ctx)
if err != nil {
return nil, err
}
var out []conditions.Observation
for _, n := range nodes {
plan, _, err := planFor(ctx, d.open, n.Name)
if err != nil {
if unresolvable(err) {
// D1 says it, with the binding that refused it when that is why.
continue
}
return nil, fmt.Errorf("%s cannot be worked out: %w", n.Name, err)
}
bound, err := inv.BindingsFor(ctx, n.Name)
if err != nil {
return nil, err
}
pins, err := inv.PinsFor(ctx, n.Name)
if err != nil {
return nil, err
}
out = append(out, bindingFindings(plan, bound, pins)...)
}
return out, nil
}
// bindingFindings is what one machine's resolution says against its record.
func bindingFindings(plan catalogue.Resolution, bound map[string]map[string]catalogue.Chosen,
pins map[string]catalogue.Chosen) []conditions.Observation {
var out []conditions.Observation
for _, k := range plan.Kept {
out = append(out, keptObservation(k))
}
said := map[string]bool{}
for _, need := range plan.Needs {
if !need.KeepsData || need.ByRecord {
continue
}
was, recorded := bound[need.For][need.Name]
now := catalogue.Chosen{Node: need.From, Module: need.Module}
if !recorded || was == now || (was.Module == "" && was.Node == now.Node) {
continue
}
id := bindingID(plan.Node, need.For, need.Name)
if said[id] {
continue
}
said[id] = true
o := conditions.Observation{Scope: conditions.ScopeMachine, ID: id, Machine: plan.Node,
Also: otherMachines(plan.Node, was.Node, now.Node), Resolver: conditions.ResolverOperator}
if pin, pinned := pins[need.Name]; pinned && pin.Node == now.Node && (pin.Module == "" || pin.Module == now.Module) {
o.Token, o.Kind, o.Severity = kindBindingMoving, kindBindingMoving, conditions.Warning
o.Summary = fmt.Sprintf("on %s, %s's %s moves from %s to %s at the next push, by the pin — its data "+
"is on %s: move it first", plan.Node, need.For, need.Name, was, now, was)
} else {
o.Token, o.Kind, o.Severity = kindBindingMoved, kindBindingMoved, conditions.Urgent
o.Summary = fmt.Sprintf("on %s, %s's %s would be sent %s, and it is bound to %s, where its data is, "+
"with no pin naming %s — a move nothing asked for", plan.Node, need.For, need.Name, now, was, now)
}
out = append(out, o)
}
return out
}
@@ -1,102 +0,0 @@
//go:build integration
package main
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
)
func TestTheIncidentAConsumerStaysBesideItsDataUntilAPersonPinsIt(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if _, err := inv.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
for _, m := range storeManifests() {
register(t, open, m)
}
assignAll := func(pairs ...[2]string) {
for _, a := range pairs {
if _, err := assign(ctx, open, a[0], a[1]); err != nil {
t.Fatalf("assign %s %s: %v", a[0], a[1], err)
}
}
}
// The anchor's store and resolver hold the mesh's seats, on record; the laptop runs its own of each.
assignAll([2]string{"anchor", "store"}, [2]string{"anchor", "resolver"})
for _, seat := range [][2]string{{"mesh-store", "store"}, {"mesh-dns-resolver", "resolver"}} {
if err := inv.HoldSeat(ctx, seat[0], catalogue.ScopeMesh, "anchor", seat[1]); err != nil {
t.Fatal(err)
}
}
assignAll([2]string{"laptop", "store"}, [2]string{"laptop", "resolver"}, [2]string{"laptop", "network"},
[2]string{"laptop", "board"})
plan, _, err := planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
if n := need(t, plan, "board", "postgres-database"); n.From != "laptop" || n.Module != "store" || !n.KeepsData {
t.Fatalf("the consumer was bound to %s/%s (keeps data: %v); its data is beside it", n.From, n.Module, n.KeepsData)
}
if n := need(t, plan, "network", "wildcard-resolution"); n.From != "anchor" || n.KeepsData {
t.Fatalf("the resolver was bound to %s (keeps data: %v); its seat is held on anchor (issue 258)", n.From, n.KeepsData)
}
// Sent, and recorded: only the binding to data.
bindings := boundToData(plan, nil)
if len(bindings) != 1 || bindings[0].Provider != (catalogue.Chosen{Node: "laptop", Module: "store"}) {
t.Fatalf("recorded %+v", bindings)
}
if err := inv.RecordBindings(ctx, "laptop", bindings); err != nil {
t.Fatal(err)
}
if found, err := probeBindings(ctx, &doctor{open: open}); err != nil || len(found) != 0 {
t.Fatalf("a mesh bound where it was sent: %+v, %v", found, err)
}
// The laptop's store taken away: refused, not moved to the anchor's empty one.
if err := inv.Unassign(ctx, "laptop", "store"); err != nil {
t.Fatal(err)
}
if _, _, err := planFor(ctx, open, "laptop"); err == nil || !unresolvable(err) ||
!strings.Contains(err.Error(), "board on laptop is bound to laptop/store") ||
!strings.Contains(err.Error(), "pin laptop postgres-database anchor store") {
t.Fatalf("the consumer's store went and it was answered elsewhere: %v", err)
}
// A person pins the anchor's: it moves, said before it is sent, and the record keeps where it was.
if err := inv.PinProvision(ctx, "laptop", "postgres-database", "anchor", "store"); err != nil {
t.Fatal(err)
}
plan, _, err = planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
if n := need(t, plan, "board", "postgres-database"); n.From != "anchor" {
t.Fatalf("pinned to the anchor and bound to %s", n.From)
}
found, err := probeBindings(ctx, &doctor{open: open})
if err != nil {
t.Fatal(err)
}
if len(found) != 1 || found[0].Kind != kindBindingMoving || found[0].Severity != conditions.Warning ||
!strings.Contains(found[0].Summary, "board's postgres-database moves from laptop/store to anchor/store") {
t.Fatalf("a pinned move is not said before it is sent: %+v", found)
}
if err := inv.RecordBindings(ctx, "laptop", boundToData(plan, nil)); err != nil {
t.Fatal(err)
}
all, err := inv.Bindings(ctx)
if err != nil || len(all) != 1 || all[0].MovedFrom != "laptop/store" {
t.Fatalf("%+v, %v", all, err)
}
if found, err := probeBindings(ctx, &doctor{open: open}); err != nil || len(found) != 0 {
t.Fatalf("a move sent is still said: %+v, %v", found, err)
}
}
-93
View File
@@ -1,93 +0,0 @@
package main
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
)
// novox/hq issue 273, ADR 0232: a consumer of a provision that keeps its data moves only by a pin.
func storeManifests() []catalogue.Manifest {
return []catalogue.Manifest{
{Module: "store", Version: "1",
Provides: []catalogue.Offer{{Name: "postgres-database", Scope: catalogue.ScopeMesh}},
Claims: []catalogue.Claim{{Name: "mesh-store", Scope: catalogue.ScopeMesh}},
Serves: map[string]map[string]any{"postgres-database": {"port": 5432}},
Grants: map[string]string{"postgres-database": "/var/lib/mesh/store/grants"}},
{Module: "resolver", Version: "1",
Provides: []catalogue.Offer{{Name: "wildcard-resolution", Scope: catalogue.ScopeMesh}},
Claims: []catalogue.Claim{{Name: "mesh-dns-resolver", Scope: catalogue.ScopeMesh}}},
{Module: "network", Version: "1", Requires: []string{"wildcard-resolution"}},
{Module: "board", Version: "1", Requires: []string{"postgres-database"}},
}
}
func need(t *testing.T, plan catalogue.Resolution, consumer, provision string) catalogue.Needed {
t.Helper()
for _, n := range plan.Needs {
if n.For == consumer && n.Name == provision {
return n
}
}
t.Fatalf("no %s for %s: %+v", provision, consumer, plan.Needs)
return catalogue.Needed{}
}
// The incident through the stores: the laptop runs its own store and a consumer of it, the anchor's
// store holds the mesh's seat. The consumer stays beside its data, the resolver follows its seat, the
// binding is recorded as sent, and a pin — only a pin — moves it, said before the push that carries it.
// What one machine's resolution says against its record.
func TestBindingFindingsSayAKeptMoveAndAMoveNothingAskedFor(t *testing.T) {
home, anchor := catalogue.Chosen{Node: "home", Module: "store"}, catalogue.Chosen{Node: "anchor", Module: "store"}
plan := catalogue.Resolution{Node: "laptop",
Kept: []catalogue.KeptBinding{{Machine: "laptop", Consumer: "board", Provision: "postgres-database",
Bound: home, Would: anchor}},
Needs: []catalogue.Needed{
{Name: "postgres-database", For: "board", From: "home", Module: "store", KeepsData: true},
{Name: "postgres-database", For: "game", From: "anchor", Module: "store", KeepsData: true},
{Name: "wildcard-resolution", For: "network", From: "anchor", Module: "resolver"},
}}
bound := map[string]map[string]catalogue.Chosen{
"board": {"postgres-database": home},
"game": {"postgres-database": home},
"network": {"wildcard-resolution": {Node: "home", Module: "resolver"}},
}
found := linted(bindingFindings(plan, bound, nil))
if len(found) != 2 {
t.Fatalf("found %+v", found)
}
kept, moved := found[0], found[1]
if kept.Kind != kindBindingKept || kept.Severity != conditions.Urgent || kept.Machine != "laptop" ||
!strings.Contains(kept.Summary, "would move board's postgres-database from home/store to anchor/store") ||
!strings.Contains(kept.Summary, "its data is on home/store") ||
!strings.Contains(kept.Summary, "`pin laptop postgres-database anchor store` to confirm a move (and move the data first)") {
t.Errorf("kept: %+v", kept)
}
if moved.Kind != kindBindingMoved || moved.Severity != conditions.Urgent ||
!strings.Contains(moved.Summary, "game's postgres-database would be sent anchor/store") {
t.Errorf("moved: %+v", moved)
}
if kept.Key() == moved.Key() {
t.Error("two consumers, one condition")
}
}
// A push says the move it refused, and raises its condition at once.
func TestAPushSaysAKeptMoveAndRaisesItsCondition(t *testing.T) {
k, _ := withConditionsInMemory(t)
reportKept(t.Context(), catalogue.Resolution{Node: "laptop", Kept: []catalogue.KeptBinding{{Machine: "laptop",
Consumer: "board", Provision: "postgres-database", Bound: catalogue.Chosen{Node: "home", Module: "store"},
Would: catalogue.Chosen{Node: "anchor", Module: "store"}}}})
open, err := k.Open(t.Context())
if err != nil {
t.Fatal(err)
}
if len(open) != 1 || open[0].Kind != kindBindingKept || open[0].Severity != conditions.Urgent ||
open[0].Source != probeBindingsID {
t.Fatalf("raised %+v", open)
}
}
-324
View File
@@ -1,324 +0,0 @@
package main
import (
"context"
"errors"
"flag"
"fmt"
"html/template"
"net/http"
"sort"
"strings"
"time"
)
// boardCommand serves the three questions as a page.
//
// **It reads through the same functions everything else does and holds nothing**
// (novox/hq 03-DESIGN/01-to-be/11-a-board.md). The board being replaced is one service that reads
// every context's database directly — [ADR 0008](novox/hq) violated by the one component with a
// reason to violate it, and the cost is that a boundary nothing may cross can move, while one
// thing crossing it is enough to freeze it. A board that reads the provisioning tables is a board
// that breaks when provisioning changes its tables, and the change then gets weighed against the
// board.
//
// **It stores nothing of its own.** No cache that can disagree, no table of what the mesh looked
// like last time. Every request reads the mesh now; if that is slow, the answer belongs in the
// context that owns it, where everything else asking gets it too.
//
// **Reading is the whole of it.** Every action a board could offer already exists as a command,
// and a button that does something no command does is a second implementation of a decision.
func boardCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("board", flag.ContinueOnError)
// The private network, not everything. A board says which machines are broken and what they
// are running, which is exactly the map somebody attacking this would like — and there is no
// reason for it to be reachable from further away than the mesh.
listen := set.String("listen", "127.0.0.1:8080", "where to serve it")
if _, err := parseAround(set, args); err != nil {
return err
}
server := &http.Server{
Addr: *listen,
ReadHeaderTimeout: 10 * time.Second,
Handler: board(),
}
fmt.Printf("the board is on http://%s\n", *listen)
fmt.Printf(" it reads the mesh on every request and keeps nothing\n")
go func() {
<-ctx.Done()
closing, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
_ = server.Shutdown(closing)
}()
if err := server.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
return err
}
return nil
}
// board is the handler, separate so a test can drive it without a listener.
func board() http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
http.NotFound(w, r)
return
}
asked, err := ask(r.Context())
if err != nil {
// **Said, not blank.** A board that cannot reach the mesh and renders an empty page
// says "nothing is wrong" in the one situation where nobody can know that.
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(http.StatusServiceUnavailable)
_ = page.Execute(w, view{Unreachable: err.Error()})
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
if err := page.Execute(w, asked); err != nil {
// The page is half-written by now; there is nothing useful left to say to the
// browser, and saying it here is what stops the failure being silent.
fmt.Printf("the board could not render: %v\n", err)
}
})
// The same answers for something that is not a person, from the same read. A board and a
// script disagreeing about which machine is broken would be worse than either alone.
mux.HandleFunc("/mesh.json", func(w http.ResponseWriter, r *http.Request) {
open, err := openStores(r.Context())
if err != nil {
http.Error(w, err.Error(), http.StatusServiceUnavailable)
return
}
defer open.Close()
asked, err := theThreeQuestions(r.Context(), open)
if err != nil {
http.Error(w, err.Error(), http.StatusServiceUnavailable)
return
}
body, err := statusAsJSON(asked)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write(append(body, '\n'))
})
return mux
}
// ask reads the mesh for one request.
func ask(ctx context.Context) (view, error) {
open, err := openStores(ctx)
if err != nil {
return view{}, err
}
defer open.Close()
asked, err := theThreeQuestions(ctx, open)
if err != nil {
return view{}, err
}
return viewOf(asked), nil
}
// view is what the page is given. Nothing is derived here that the reader could not derive.
type view struct {
Unreachable string
Machines int
Broken []brokenMachine
Quiet []quietMachine
Behind []staleModule
Waiting []waitingMachine
// Unresolved is every machine that cannot be worked out at all. Shown above everything else,
// because a machine here is in none of the other lists: nothing was computed for it, so it is
// not broken, not quiet and not behind — and a page without this said "all well" about a mesh
// where nothing could be sent anywhere.
Unresolved []blockedMachine
// Network is why the private network could not be computed, when it could not.
Network string
// Adopted is every node still adopted (novox/hq ADR 0100). Not broken: nothing forces the
// flip, so a node left adopted is shown rather than read as converged.
Adopted []string
At string
}
type blockedMachine struct {
Node string
// Said is the mesh's own words, a line at a time. Every unmet requirement, not the first:
// a machine is usually blocked by more than one and fixing one of them changes nothing.
Said []string
}
type waitingMachine struct {
Node string
// Never told is not out of date: nobody has ever asked this machine to be anything. Same
// remedy, different situation, and the page says which.
Never bool
}
type brokenMachine struct {
Node string
// Outcome is refused or failed, and stays distinct all the way to the page. **Refused means
// the machine is exactly as it was and what is wrong is in what was sent; failed means it is
// in a state nobody declared and what is wrong is on the machine.** They are fixed in
// different places, so one word for both would send half the readers to the wrong one.
Outcome string
Said []string
When string
}
type quietMachine struct {
Node string
// Heard is "never" or how long ago. Never heard from is not the same as quiet for a while:
// one may be a machine that was never sent anything.
Heard string
}
type staleModule struct {
Module string
Holds string
Source string
Running []string
}
func viewOf(asked answers) view {
out := view{Machines: len(asked.nodes), At: time.Now().Format("15:04:05"),
Network: asked.network, Adopted: adoptedNodes(asked.nodes)}
var blocked []string
for name := range asked.refused {
blocked = append(blocked, name)
}
sort.Strings(blocked)
for _, name := range blocked {
one := blockedMachine{Node: name}
for _, line := range strings.Split(strings.TrimRight(asked.refused[name], "\n"), "\n") {
one.Said = append(one.Said, strings.TrimSpace(line))
}
out.Unresolved = append(out.Unresolved, one)
}
for _, d := range asked.wrong {
one := brokenMachine{Node: d.Node, Outcome: d.Outcome,
When: d.At.Local().Format("2006-01-02 15:04")}
if d.Refused != "" {
// The host's own words. It says exactly what it could not accept, and nothing
// written here would say it better.
one.Said = append(one.Said, firstLine(d.Refused))
}
for _, f := range d.Failed {
one.Said = append(one.Said, f.ID+": "+firstLine(f.Error))
}
out.Broken = append(out.Broken, one)
}
for _, n := range asked.quiet {
out.Quiet = append(out.Quiet, quietMachine{Node: n.Name, Heard: heardFrom(n)})
}
for _, m := range asked.waiting {
out.Waiting = append(out.Waiting, waitingMachine{Node: m.Node, Never: m.Never})
}
for module, on := range asked.behind {
from := asked.sources[module]
out.Behind = append(out.Behind, staleModule{
Module: module, Holds: short(from.BuiltFrom), Source: short(from.Head), Running: on,
})
}
return out
}
// The page. Deliberately one file with no assets: a board that cannot render without fetching
// something is a board that is blank exactly when the mesh is unwell.
var page = template.Must(template.New("board").Parse(`<!doctype html>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>the mesh</title>
<style>
:root { color-scheme: light dark; }
body { font: 15px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace; margin: 2rem auto;
max-width: 52rem; padding: 0 1rem; }
h1 { font-size: 1.1rem; font-weight: 600; margin: 0 0 1.5rem; }
h2 { font-size: 1rem; font-weight: 600; margin: 2rem 0 .5rem; }
.quiet { opacity: .65; }
.said { opacity: .8; padding-left: 1.5rem; }
.outcome { display: inline-block; min-width: 4.5rem; }
.refused { color: #b26b00; }
.failed { color: #c0392b; }
ul { list-style: none; padding: 0; margin: 0; }
li { padding: .15rem 0; }
footer { margin-top: 3rem; opacity: .6; font-size: .85rem; }
</style>
{{if .Unreachable}}
<h1>the mesh cannot be read</h1>
<p class="failed">{{.Unreachable}}</p>
<p class="quiet">This says nothing about whether the mesh is well — only that this page could not
find out.</p>
{{else}}
<h1>{{.Machines}} machine{{if ne .Machines 1}}s{{end}}</h1>
{{if .Unresolved}}
<h2>Can everything be worked out?</h2>
<ul>
{{range .Unresolved}}
<li><span class="outcome failed">blocked</span> <strong>{{.Node}}</strong>
{{range .Said}}<div class="said">{{.}}</div>{{end}}
</li>
{{end}}
</ul>
<p class="quiet">Nothing can be sent to a machine here, and it appears in none of the lists below:
nothing was computed for it, so there is nothing it can be behind.</p>
{{end}}
{{if .Network}}
<p class="failed">The private network could not be computed: {{.Network}}</p>
{{end}}
<h2>Is anything broken?</h2>
{{if .Broken}}
<ul>
{{range .Broken}}
<li>
<span class="outcome {{.Outcome}}">{{.Outcome}}</span>
<strong>{{.Node}}</strong> <span class="quiet">{{.When}}</span>
{{range .Said}}<div class="said">{{.}}</div>{{end}}
</li>
{{end}}
</ul>
{{else}}<p class="quiet">No. Every machine is doing what it was told.</p>{{end}}
<h2>Is anything not answering?</h2>
{{if .Quiet}}
<ul>{{range .Quiet}}<li><strong>{{.Node}}</strong> <span class="quiet">{{.Heard}}</span></li>{{end}}</ul>
<p class="quiet">Not heard from is not the same as tried and could not — a machine here may be
new, switched off, or unreachable.</p>
{{else}}<p class="quiet">No. Every machine has been heard from.</p>{{end}}
<h2>Is anything out of date?</h2>
{{if .Behind}}
<ul>
{{range .Behind}}
<li><strong>{{.Module}}</strong> holds {{.Holds}}, source has {{.Source}}
{{if .Running}}<div class="said">running on {{range $i, $n := .Running}}{{if $i}}, {{end}}{{$n}}{{end}}</div>
{{else}}<div class="said quiet">assigned to nothing</div>{{end}}
</li>
{{end}}
</ul>
{{else}}<p class="quiet">No. Every module is what its source last had.</p>{{end}}
{{if .Waiting}}
<ul>
{{range .Waiting}}
<li><strong>{{.Node}}</strong>
{{if .Never}}<span class="quiet">has never been sent anything</span>
{{else}}<span class="quiet">is not running what the mesh would send it</span>{{end}}
</li>
{{end}}
</ul>
<p class="quiet">Never told is not out of date: nobody has asked that machine to be anything yet.
Both are sent by <code>push --behind</code>.</p>
{{else}}<p class="quiet">Every machine is running what the mesh would send it.</p>{{end}}
{{if .Adopted}}
<h2>Which machines are adopted?</h2>
<ul>{{range .Adopted}}<li><strong>{{.}}</strong> <span class="quiet">adopted — what was found on it is kept until each module is taken</span></li>{{end}}</ul>
{{end}}
{{end}}
<footer>Read at {{.At}}. This page holds nothing and changes nothing.</footer>
`))
-154
View File
@@ -1,154 +0,0 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// Refused and failed stay distinct all the way to the page.
//
// **Refused means the machine is exactly as it was and what is wrong is in what was sent; failed
// means it is in a state nobody declared and what is wrong is on the machine.** They are fixed in
// different places, so a page saying "error" for both sends half its readers to the wrong one.
func TestRefusedAndFailedReachThePageAsDifferentThings(t *testing.T) {
rendered := render(t, viewOf(answers{
nodes: []inventory.Node{{Name: "anchor"}, {Name: "laptop"}},
wrong: []inventory.Doing{
{Node: "anchor", Outcome: "refused", Refused: "not a declaration this host speaks",
At: time.Now()},
{Node: "laptop", Outcome: "failed", At: time.Now(),
Failed: []inventory.FailedResource{{ID: "web.container", Error: "no such image"}}},
},
}))
// The rendered outcome, not the word anywhere on the page: both words appear in the
// stylesheet, so a plain Contains passes whatever the machine actually said. It did.
for _, want := range []string{`<span class="outcome refused">refused</span>`,
`<span class="outcome failed">failed</span>`} {
if !strings.Contains(rendered, want) {
t.Fatalf("the page does not distinguish the two — missing %s:\n%s", want, rendered)
}
}
// And the host's own words, which say exactly what it could not accept.
for _, want := range []string{"not a declaration this host speaks", "web.container", "no such image"} {
if !strings.Contains(rendered, want) {
t.Fatalf("the page does not say %q, so a reader must go and ask:\n%s", want, rendered)
}
}
}
// Nothing wrong is said, not left blank. An empty page and a well mesh must not look alike.
func TestAWellMeshSaysSoRatherThanShowingNothing(t *testing.T) {
rendered := render(t, viewOf(answers{nodes: []inventory.Node{{Name: "anchor"}}}))
for _, want := range []string{
"Every machine is doing what it was told",
"Every machine has been heard from",
"Every module is what its source last had",
} {
if !strings.Contains(rendered, want) {
t.Fatalf("a well mesh does not say %q:\n%s", want, rendered)
}
}
}
// A board that cannot reach the mesh must not render an empty page: that says "nothing is wrong"
// in the one situation where nobody can know it.
func TestABoardThatCannotReadTheMeshSaysSo(t *testing.T) {
rendered := render(t, view{Unreachable: "the store did not answer"})
if !strings.Contains(rendered, "the store did not answer") {
t.Fatalf("the reason is not on the page:\n%s", rendered)
}
if strings.Contains(rendered, "Every machine is doing what it was told") {
t.Fatal("a board that could not read the mesh reported that the mesh is well")
}
}
// Quiet is not broken, and the page says which it is.
func TestQuietIsNotReportedAsBroken(t *testing.T) {
rendered := render(t, viewOf(answers{
nodes: []inventory.Node{{Name: "laptop"}},
quiet: []inventory.Node{{Name: "laptop"}},
}))
if !strings.Contains(rendered, "not the same as tried and could not") {
t.Fatalf("the page does not separate quiet from broken:\n%s", rendered)
}
if !strings.Contains(rendered, "Every machine is doing what it was told") {
t.Fatalf("a quiet machine was counted as broken:\n%s", rendered)
}
}
// The page renders what a machine said, and a hostile string in it is not markup.
//
// What is on this page comes from machines, and a machine's own words are the whole reason the
// page is useful. They are also the one thing here that nobody in this repository wrote.
func TestWhatAMachineSaidIsNotMarkup(t *testing.T) {
rendered := render(t, viewOf(answers{
nodes: []inventory.Node{{Name: "anchor"}},
wrong: []inventory.Doing{{Node: "anchor", Outcome: "refused",
Refused: `<script>alert("from the machine")</script>`, At: time.Now()}},
}))
if strings.Contains(rendered, "<script>alert") {
t.Fatalf("a machine's words were rendered as markup:\n%s", rendered)
}
if !strings.Contains(rendered, "&lt;script&gt;") {
t.Fatalf("the words were not shown at all, so the reader cannot see what it said:\n%s", rendered)
}
}
// A module behind its source names the machines running the old one — the part with consequences.
func TestBeingBehindNamesTheMachinesRunningTheOldOne(t *testing.T) {
rendered := render(t, viewOf(answers{
nodes: []inventory.Node{{Name: "laptop"}},
behind: map[string][]string{"web": {"laptop"}},
sources: map[string]inventory.Source{"web": {BuiltFrom: "aaaaaaaaaa", Head: "bbbbbbbbbb"}},
}))
for _, want := range []string{"web", "aaaaaaaa", "bbbbbbbb", "running on laptop"} {
if !strings.Contains(rendered, want) {
t.Fatalf("the page does not say %q:\n%s", want, rendered)
}
}
}
func render(t *testing.T, v view) string {
t.Helper()
var out strings.Builder
if err := page.Execute(&out, v); err != nil {
t.Fatal(err)
}
return out.String()
}
// The board and the command must answer the same question the same way.
//
// A machine not running what the mesh would send it was added to `status` and would have been
// missing here — which is the one thing this page's design forbids: two answers to the same
// question, disagreeing, with a person in front of each.
func TestTheBoardSaysWhichMachinesHaveNotBeenSentWhatTheyShouldBe(t *testing.T) {
rendered := render(t, viewOf(answers{
nodes: []inventory.Node{{Name: "anchor"}, {Name: "laptop"}},
waiting: []inventory.Machine{
{Node: "anchor"},
{Node: "laptop", Never: true},
},
}))
if !strings.Contains(rendered, "is not running what the mesh would send it") {
t.Fatalf("the page does not say a machine is out of date:\n%s", rendered)
}
// Never told is a different situation with the same remedy, and the page keeps them apart.
if !strings.Contains(rendered, "has never been sent anything") {
t.Fatalf("the page does not separate never told from out of date:\n%s", rendered)
}
if !strings.Contains(rendered, "push --behind") {
t.Fatalf("the page does not say what sends them:\n%s", rendered)
}
}
// And says so when there is nothing waiting, rather than leaving the question unanswered.
func TestAMeshWithNothingWaitingSaysSo(t *testing.T) {
rendered := render(t, viewOf(answers{nodes: []inventory.Node{{Name: "anchor"}}}))
if !strings.Contains(rendered, "Every machine is running what the mesh would send it") {
t.Fatalf("the page leaves the question unanswered:\n%s", rendered)
}
}
-207
View File
@@ -1,207 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"sort"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/conditions"
)
// **A filling bucket or log is said before it is full** (novox/hq ADR 0297 §6, issue 501).
//
// A module's bucket and its log each have a cap on the bus, and when either is full the bus refuses
// the next write: the module stops doing what it writes for, and nothing said so before. On
// 2026-10-11 the issue tracker's bucket held a sixth of its cap after two days, growing toward a stop
// nobody would have heard coming. So the self-check reads every module's bucket and log, and one at
// fillRaiseAt of its cap or more raises a condition naming the module, the bucket or log, and how full
// it is.
//
// **Cleared by observation below fillClearBelow, not below fillRaiseAt**: a bucket or log hovering at
// its threshold would otherwise be raised and cleared on every run. Between the two, an open condition
// is kept and none is raised.
//
// **One that cannot be read is said, and the rest are still judged.** An unanswered question about one
// stream is no reason to know nothing of the others: it is a finding of its own, naming the bucket or
// log and why, and a fill condition already open for it is kept, because not knowing is not a pass.
// The fill thresholds, in percent of a bucket's or log's cap.
const (
fillRaiseAt = 75
fillClearBelow = 70
)
// The conditions the fill probe raises: one filling, and one that could not be read.
const (
kindBucketOrLogFilling = "bucket-or-log-filling"
kindBucketOrLogUnread = "bucket-or-log-unread"
)
// probeFillID is the probe's id in the registry.
const probeFillID = "D-fill"
// bucketOrLog is one module's bucket or log as the fill probe reads it.
type bucketOrLog struct {
Module string
// Kind is `bucket` or `log`; Name its local name; Stream the stream it is on the bus.
Kind, Name, Stream string
}
// filled is how full one bucket or log is, read from the bus.
type filled struct {
Bytes, Max uint64
}
// percent is how full, in whole percent, rounded down.
func (f filled) percent() uint64 {
if f.Max == 0 {
return 0
}
return f.Bytes * 100 / f.Max
}
// fillKey is where a bucket's or log's fill condition is kept.
func fillKey(s bucketOrLog) string { return conditions.Key(conditions.ScopeBus, s.Stream, "filling") }
// fillObservation is what one bucket's or log's fill says: a finding at fillRaiseAt or above, and,
// while its condition is open, at fillClearBelow or above too; nothing otherwise, which is what clears
// it. The operator's words are the kind's (plain_words.go); the summary and the evidence name the
// module, the bucket or log, and the fill.
func fillObservation(s bucketOrLog, f filled, open bool) (conditions.Observation, bool) {
if f.Max == 0 {
return conditions.Observation{}, false // one with no cap cannot fill
}
// Compared in bytes, not rounded percent: 74.9% is not 75%.
atRaise := f.Bytes*100 >= f.Max*fillRaiseAt
aboveClear := f.Bytes*100 >= f.Max*fillClearBelow
if !atRaise && !(open && aboveClear) {
return conditions.Observation{}, false
}
pct := f.percent()
return conditions.Observation{
Scope: conditions.ScopeBus, ID: s.Stream, Token: "filling", Kind: kindBucketOrLogFilling,
Severity: conditions.Warning,
Summary: fmt.Sprintf("%s's %s %s holds %s of %s, %d%%: at its cap the bus refuses the module's next write",
s.Module, s.Kind, s.Name, mibWords(f.Bytes), mibWords(f.Max), pct),
Said: fmt.Sprintf("module %s, %s %s (stream %s): %d of %d bytes, %d%%", s.Module, s.Kind, s.Name,
s.Stream, f.Bytes, f.Max, pct),
}, true
}
// unreadObservation says one bucket or log could not be read, and why. Asked over the network, so one
// look can be wrong: raised on the second look in a row (confirm.go).
func unreadObservation(s bucketOrLog, why error) conditions.Observation {
return conditions.Observation{
Scope: conditions.ScopeBus, ID: s.Stream, Token: "unread", Kind: kindBucketOrLogUnread,
Severity: conditions.Warning, Confirm: true,
Summary: fmt.Sprintf("%s's %s %s could not be read, so how full it is is not known", s.Module, s.Kind, s.Name),
Said: fmt.Sprintf("module %s, %s %s (stream %s): %v", s.Module, s.Kind, s.Name, s.Stream, why),
}
}
// keptFilling is an open fill condition said again for a bucket or log that could not be read this
// time: not knowing how full it is now is no reason to say it has room. Only ever made from a condition
// read back as open — its own summary and severity, never words made up for it.
func keptFilling(s bucketOrLog, open conditions.Condition, why error) conditions.Observation {
return conditions.Observation{
Scope: conditions.ScopeBus, ID: s.Stream, Token: "filling", Kind: kindBucketOrLogFilling,
Severity: open.Severity, Summary: open.Summary,
Said: fmt.Sprintf("module %s, %s %s (stream %s): not read this time (%v); kept open as it was",
s.Module, s.Kind, s.Name, s.Stream, why),
}
}
// mibWords is a size as a person reads it, in MiB with one decimal.
func mibWords(b uint64) string {
return fmt.Sprintf("%.1f MiB", float64(b)/(1024*1024))
}
// readFill is how full one bucket or log is on the bus; found false when it is not on the bus.
type readFill func(ctx context.Context, s bucketOrLog) (f filled, found bool, err error)
// openFill is the fill condition open under a key, if one is, or why it could not be read.
type openFill func(ctx context.Context, key string) (conditions.Condition, bool, error)
// judgeFills judges every bucket and log on its own: one that cannot be read is said as unread, with
// its fill condition kept only when that condition was read back and is open — when it cannot be read
// either, nothing is said of its fill, which the unread finding already covers. Every other is judged
// by its fill; for one of those, a condition that cannot be read is taken as open, so an unknown never
// clears a fill measured between fillClearBelow and fillRaiseAt.
func judgeFills(ctx context.Context, all []bucketOrLog, read readFill, open openFill) []conditions.Observation {
var out []conditions.Observation
for _, s := range all {
f, found, err := read(ctx, s)
if err != nil {
out = append(out, unreadObservation(s, err))
if c, isOpen, cerr := open(ctx, fillKey(s)); cerr == nil && isOpen {
out = append(out, keptFilling(s, c, err))
}
continue
}
if !found {
continue // not on the bus: created on the controller's next raise, and nothing there can fill
}
_, isOpen, cerr := open(ctx, fillKey(s))
isOpen = isOpen || cerr != nil
if o, said := fillObservation(s, f, isOpen); said {
out = append(out, o)
}
}
sort.SliceStable(out, func(i, j int) bool { return out[i].Key() < out[j].Key() })
return out
}
// declaredBucketsAndLogs is every module's bucket and log the catalogue declares, by its stream.
func declaredBucketsAndLogs(ctx context.Context, d *doctor) ([]bucketOrLog, error) {
buckets, err := d.open.inventory.DeclaredBuckets(ctx)
if err != nil {
return nil, err
}
logs, err := d.open.inventory.DeclaredLogs(ctx)
if err != nil {
return nil, err
}
var out []bucketOrLog
for _, b := range buckets {
out = append(out, bucketOrLog{Module: b.Module, Kind: "bucket", Name: b.Name, Stream: "KV_" + b.Bucket()})
}
for _, l := range logs {
out = append(out, bucketOrLog{Module: l.Module, Kind: "log", Name: l.Name, Stream: l.Stream()})
}
return out, nil
}
// probeFill reads every module's bucket and log on the bus and says each one filling toward its cap,
// and each one that could not be read.
func probeFill(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
all, err := declaredBucketsAndLogs(ctx, d)
if err != nil {
return nil, err
}
js := d.js.Context()
read := func(ctx context.Context, s bucketOrLog) (filled, bool, error) {
if err := ctx.Err(); err != nil {
return filled{}, false, err
}
info, err := js.StreamInfo(s.Stream, nats.Context(ctx))
switch {
case errors.Is(err, nats.ErrStreamNotFound):
return filled{}, false, nil
case err != nil:
return filled{}, false, err
case info.Config.MaxBytes <= 0:
return filled{}, true, nil
}
return filled{Bytes: info.State.Bytes, Max: uint64(info.Config.MaxBytes)}, true, nil
}
open := func(ctx context.Context, key string) (conditions.Condition, bool, error) {
if d.keeper == nil {
return conditions.Condition{}, false, nil
}
return d.keeper.Get(ctx, key)
}
return judgeFills(ctx, all, read, open), nil
}
-165
View File
@@ -1,165 +0,0 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/conditions"
)
// **A filling log or bucket is said at three quarters of its cap and cleared below seven tenths**
// (novox/hq ADR 0297 §6): raised at 75% naming the module, the bucket or log and its fill; not at 74.9%;
// kept between 70% and 75% while it is open, and not raised there when it is not; and cleared — said no
// more — once it reads below 70%, open or not.
func TestAFillingBucketOrLogIsRaisedAtThreeQuartersAndClearedBelowSevenTenths(t *testing.T) {
const mib = 1024 * 1024
log := bucketOrLog{Module: "mesh-issues", Kind: "log", Name: "changes", Stream: "LOG_mesh-issues_changes"}
bucket := bucketOrLog{Module: "mesh-issues", Kind: "bucket", Name: "issues", Stream: "KV_mesh-issues_issues"}
for _, c := range []struct {
name string
of bucketOrLog
bytes uint64
max uint64
open bool
said bool
}{
{"a log at exactly 75%", log, 768 * mib, 1024 * mib, false, true},
{"a bucket at exactly 75%", bucket, 48 * mib, 64 * mib, false, true},
{"a bucket full", bucket, 64 * mib, 64 * mib, false, true},
{"a log just under 75%", log, 768*mib - 1, 1024 * mib, false, false},
{"a bucket at 72%, not open", bucket, 64 * mib * 72 / 100, 64 * mib, false, false},
{"a bucket at 72%, open: kept", bucket, 64 * mib * 72 / 100, 64 * mib, true, true},
{"a log at exactly 70%, open: kept", log, 700 * mib, 1000 * mib, true, true},
{"a log just under 70%, open: cleared", log, 700*mib - 1, 1000 * mib, true, false},
{"a bucket at 10%, open: cleared", bucket, 64 * mib / 10, 64 * mib, true, false},
{"a bucket with no cap", bucket, 100, 0, true, false},
} {
o, said := fillObservation(c.of, filled{Bytes: c.bytes, Max: c.max}, c.open)
if said != c.said {
t.Errorf("%s: said %v, want %v", c.name, said, c.said)
continue
}
if !said {
continue
}
if o.Key() != fillKey(c.of) || o.Kind != kindBucketOrLogFilling || o.Severity != conditions.Warning {
t.Errorf("%s: raised as %s (%s, %s)", c.name, o.Key(), o.Kind, o.Severity)
}
for _, part := range []string{c.of.Module, c.of.Kind + " " + c.of.Name, "%", " of "} {
if !strings.Contains(o.Summary, part) {
t.Errorf("%s: does not name %q: %q", c.name, part, o.Summary)
}
}
if !strings.Contains(o.Said, "bytes") {
t.Errorf("%s: the evidence does not say the fill in bytes: %q", c.name, o.Said)
}
}
o, _ := fillObservation(log, filled{Bytes: 800 * mib, Max: 1024 * mib}, false)
if !strings.Contains(o.Summary, "800.0 MiB of 1024.0 MiB, 78%") {
t.Errorf("the fill is said as %q", o.Summary)
}
}
// **One bucket or log that cannot be read does not stop the others being judged** (review of #231): it is
// said as unread with its reason, a fill condition open for it is kept, and every other bucket and log
// is judged by its own fill.
func TestAnUnreadableBucketOrLogIsSaidAndTheRestAreStillJudged(t *testing.T) {
const mib = 1024 * 1024
broken := bucketOrLog{Module: "a", Kind: "bucket", Name: "broken", Stream: "KV_a_broken"}
brokenOpen := bucketOrLog{Module: "a", Kind: "log", Name: "was-filling", Stream: "LOG_a_was-filling"}
full := bucketOrLog{Module: "b", Kind: "log", Name: "changes", Stream: "LOG_b_changes"}
empty := bucketOrLog{Module: "c", Kind: "bucket", Name: "quiet", Stream: "KV_c_quiet"}
absent := bucketOrLog{Module: "d", Kind: "bucket", Name: "not-yet", Stream: "KV_d_not-yet"}
refusal := errors.New("the bus did not answer")
read := func(_ context.Context, s bucketOrLog) (filled, bool, error) {
switch s {
case broken, brokenOpen:
return filled{}, false, refusal
case full:
return filled{Bytes: 900 * mib, Max: 1000 * mib}, true, nil
case empty:
return filled{Bytes: 1, Max: 64 * mib}, true, nil
}
return filled{}, false, nil
}
open := func(_ context.Context, key string) (conditions.Condition, bool, error) {
if key == fillKey(brokenOpen) {
return conditions.Condition{Key: key, Severity: conditions.Warning,
Summary: "a's log was-filling holds 800.0 MiB of 1024.0 MiB, 78%"}, true, nil
}
return conditions.Condition{}, false, nil
}
got := map[string]conditions.Observation{}
for _, o := range judgeFills(context.Background(), []bucketOrLog{broken, brokenOpen, full, empty, absent}, read, open) {
got[o.Key()] = o
}
for _, s := range []bucketOrLog{broken, brokenOpen} {
o, said := got[conditions.Key(conditions.ScopeBus, s.Stream, "unread")]
if !said || o.Kind != kindBucketOrLogUnread || !o.Confirm || !strings.Contains(o.Said, refusal.Error()) {
t.Errorf("%s could not be read and was said as %+v", s.Stream, o)
}
}
if o, kept := got[fillKey(brokenOpen)]; !kept || !strings.Contains(o.Said, "kept open") {
t.Errorf("an open fill condition of a log not read this time was not kept: %+v", o)
}
if _, said := got[fillKey(broken)]; said {
t.Error("a bucket not read and not filling was said filling")
}
if o, said := got[fillKey(full)]; !said || o.Kind != kindBucketOrLogFilling {
t.Errorf("a log at 90%% beside an unreadable one was not judged: %+v", got)
}
if len(got) != 4 {
t.Errorf("said %d findings, want 4 (two unread, one kept, one filling): %v", len(got), got)
}
}
// Both kinds have plain words of their own, which hold to the plain rule (novox/hq ADR 0253).
func TestAFillingOrUnreadBucketOrLogIsSaidInPlainWords(t *testing.T) {
for _, kind := range []string{kindBucketOrLogFilling, kindBucketOrLogUnread} {
wording, has := plainWordings[kind]
if !has {
t.Errorf("%s has no plain words", kind)
continue
}
if why, ok := conditions.PlainWords(wording(conditions.Observation{Kind: kind})); !ok {
t.Errorf("%s: %s", kind, why)
}
}
}
// **When neither the bucket or log nor its condition can be read, nothing is said of its fill** (second
// review of #231): the unread finding covers it, and no fill is invented for it. For one that is read and
// measured between seven tenths and three quarters, a condition that cannot be read is taken as open, so
// an unknown never clears it.
func TestNothingIsSaidOfAFillWhenNeitherItNorItsConditionCanBeRead(t *testing.T) {
const mib = 1024 * 1024
broken := bucketOrLog{Module: "a", Kind: "log", Name: "changes", Stream: "LOG_a_changes"}
between := bucketOrLog{Module: "b", Kind: "bucket", Name: "issues", Stream: "KV_b_issues"}
read := func(_ context.Context, s bucketOrLog) (filled, bool, error) {
if s == broken {
return filled{}, false, errors.New("the bus did not answer")
}
return filled{Bytes: 72 * mib, Max: 100 * mib}, true, nil
}
open := func(context.Context, string) (conditions.Condition, bool, error) {
return conditions.Condition{}, false, errors.New("the conditions bucket did not answer")
}
got := map[string]conditions.Observation{}
for _, o := range judgeFills(context.Background(), []bucketOrLog{broken, between}, read, open) {
got[o.Key()] = o
}
if o, said := got[fillKey(broken)]; said {
t.Fatalf("a fill was said for a log whose fill and condition could not be read: %+v", o)
}
if _, said := got[conditions.Key(conditions.ScopeBus, broken.Stream, "unread")]; !said {
t.Error("the log that could not be read was not said as unread")
}
if _, kept := got[fillKey(between)]; !kept {
t.Error("a bucket at 72% whose condition could not be read was cleared")
}
if len(got) != 2 {
t.Errorf("said %d findings, want 2: %v", len(got), got)
}
}
File diff suppressed because it is too large Load Diff
@@ -1,131 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func TestABuildHeardIsRecordedAndRegistered(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
manifest, _ := json.Marshal(map[string]any{"module": "shop", "version": "3"})
m, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory, link.BuildResult{
ID: "b-1", Repository: "http://forge.internal:20000/novox/shop.git", Path: "modules/shop",
Ref: "main", On: "anchor", Commit: "abcdef0123", Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/shop"},
}))
if err != nil {
t.Fatal(err)
}
if m.Module != "shop" {
t.Fatalf("registered %q", m.Module)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if _, held := shelf["shop"]; !held {
t.Fatal("the module a heard build produced is not in the catalogue")
}
src, err := open.inventory.SourceOf(ctx, "shop")
if err != nil || src.Seat != "git" || src.Repository != "novox/shop" || src.BuiltFrom != "abcdef0123" {
t.Fatalf("the source is the seat and the path, never the URL: %+v %v", src, err)
}
builds, err := open.inventory.Builds(ctx, "shop", 5)
if err != nil || len(builds) != 1 || builds[0].ID != "b-1" {
t.Fatalf("the build is not recorded once: %v %v", builds, err)
}
named, _ := json.Marshal(map[string]any{"module": "idp", "version": "1", "resources": []any{
map[string]any{"id": "server", "type": "container", "image": "x@sha256:aa",
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}}}})
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{
ID: "b-2", Repository: "/r", On: "anchor", Commit: "0123456789", Manifest: named})
if err == nil || !strings.Contains(err.Error(), "does not register it") {
t.Fatalf("a definition naming an installation was taken in: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["idp"].Module != "" {
t.Fatal("the refused module was registered anyway")
}
if builds, _ := open.inventory.Builds(ctx, "idp", 5); len(builds) != 1 {
t.Fatalf("the refused build was not recorded: %v", builds)
}
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{ID: "b-3", Repository: "/r", On: "anchor", Failed: "no compiler"})
if err == nil || !strings.Contains(err.Error(), "no compiler") {
t.Fatalf("a failure is said in the builder's words: %v", err)
}
}
func TestABuildAtACommitKeepsTheBranchTheModuleFollows(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
manifest, _ := json.Marshal(map[string]any{"module": "unifi", "version": "1"})
result := func(id, ref, commit string) link.BuildResult {
return link.BuildResult{ID: id, Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/unifi", Ref: ref, On: "anchor", Commit: commit, Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
}
if _, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory, result("b-1", "main", "1111111aaaa"))); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, result("b-2", "9c97a8a", "9c97a8a1d2c3")); err != nil {
t.Fatal(err)
}
src, err := open.inventory.SourceOf(ctx, "unifi")
if err != nil {
t.Fatal(err)
}
if src.Ref != "main" || src.BuiltFrom != "9c97a8a1d2c3" {
t.Errorf("after a build at a commit the module follows %q, built from %q; want main, 9c97a8a1d2c3", src.Ref, src.BuiltFrom)
}
// One new to the catalogue, first built at a commit, follows the default branch.
other, _ := json.Marshal(map[string]any{"module": "letta", "version": "1"})
r := result("b-3", "deadbeef", "deadbeefcafe")
r.Manifest, r.Path = other, "modules/letta"
if _, _, err := takeIn(ctx, open.inventory, r); err != nil {
t.Fatal(err)
}
if src, _ := open.inventory.SourceOf(ctx, "letta"); src.Ref != "" {
t.Errorf("a module first built at a commit follows %q, want the default branch", src.Ref)
}
}
func TestAnOlderBuildHeardLaterDoesNotReplaceTheNewer(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
older := time.Date(2026, 10, 3, 21, 33, 45, 0, time.UTC)
newer := time.Date(2026, 10, 3, 21, 51, 57, 0, time.UTC)
result := func(asked time.Time, image string) link.BuildResult {
manifest, _ := json.Marshal(map[string]any{"module": "postgres", "version": image})
return link.BuildResult{ID: link.NewBuildID(asked), Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/postgres", Ref: "main", On: "anchor", Commit: "efff5415", Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
}
if _, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory, result(newer, "4bcd5f73"))); err != nil {
t.Fatal(err)
}
_, _, err := takeIn(ctx, open.inventory, result(older, "0ab07fa9"))
if !errors.Is(err, inventory.ErrSuperseded) {
t.Fatalf("the older request's outcome was taken in as current: %v", err)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if got := shelf["postgres"].Version; got != "4bcd5f73" {
t.Errorf("postgres is %q; want the newer request's 4bcd5f73", got)
}
if builds, _ := open.inventory.Builds(ctx, "postgres", 5); len(builds) != 2 {
t.Errorf("the late build was not recorded: %v", builds)
}
}
-43
View File
@@ -1,43 +0,0 @@
package main
import (
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func claiming(module, seat string, on ...string) inventory.Entry {
return inventory.Entry{
Manifest: catalogue.Manifest{Module: module, Claims: []catalogue.Claim{{Name: seat}}},
On: on,
}
}
// The controller asks the build role that has a holder (novox/hq ADR 0190 handover): the retired
// one while only the builder is assigned, the current one from the first build-agent on, and the
// current one when nothing holds either — where every ask goes once the handover is done.
func TestTheControllerAsksTheBuildRoleThatHasAHolder(t *testing.T) {
onlyTheBuilder := []inventory.Entry{
claiming("builder", link.TheBuildMachineBefore, "anchor"),
claiming("build-agent", link.TheBuildMachine), // registered, assigned nowhere yet
}
if got := buildSeatAmong(onlyTheBuilder); got != link.TheBuildMachineBefore {
t.Errorf("with only the builder assigned, asked %q", got)
}
bothHeld := []inventory.Entry{
claiming("builder", link.TheBuildMachineBefore, "anchor"),
claiming("build-agent", link.TheBuildMachine, "home-server"),
}
if got := buildSeatAmong(bothHeld); got != link.TheBuildMachine {
t.Errorf("with a build-agent assigned anywhere, asked %q", got)
}
neither := []inventory.Entry{claiming("builder", link.TheBuildMachineBefore)}
if got := buildSeatAmong(neither); got != link.TheBuildMachine {
t.Errorf("with no holder of either, asked %q, want the current role", got)
}
if got := buildSeatAmong(nil); got != link.TheBuildMachine {
t.Errorf("an empty catalogue asks %q", got)
}
}
-566
View File
@@ -1,566 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"regexp"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// Where a build may register a module from (novox/hq ADR 0266).
//
// A build's outcome registers its module, and a registered module is what the next push sends. Before this,
// an outcome registered whatever its manifest named, from whichever repository it was built from: an agent
// that made a repository of its own, committed `modules/sudo/module.json` granting itself a rule without a
// password, and asked the `build` verb for it had its repository registered as the module `sudo` — whose next
// push made the agent root on every node. A fork of the node-engine did the same everywhere. The trunk rule
// (ADR 0238) did not stop it: the trunk it checked was the trunk of the repository built, which was the
// agent's own.
//
// So **an outcome registers a module only from the repository the catalogue already builds that module
// from**; and a module new to the catalogue only from a repository the catalogue already builds another
// module from — whose trunk takes a reviewed, approved merge, which is how a merge adds a module (novox/hq
// issue 300). Anything else — a module moved to another repository, a module from a repository the
// catalogue has never built — is the operator's, at the controller's terminal: allowed only when the build
// request was kept as asked there. Judged at the take-in, which every outcome reaches whoever hears it and
// whichever verb asked it (`build`, `rebuild`, `replay --register`, `assign` with build), and before the ask
// for a call through a verb, so an agent cannot have a build node run a repository the catalogue does not
// build from at all.
// errNotItsSource is an outcome refused for where it was built from.
var errNotItsSource = errors.New("not built from the repository the catalogue builds it from")
// servedVar marks every process the serving controller starts — each verb's command, each child — and the serving
// process itself, so none of them can read as the operator at the terminal (novox/hq ADR 0266). Set by serve
// before it answers anything, inherited by every child through os.Environ.
const servedVar = "MESH_SERVED_BY_THE_CONTROLLER"
// startedAtTheTerminal says this process was started at the controller's terminal: not the serving controller,
// not anything it started, not a verb's command, not a seat call's. The serving controller marks its own
// environment (servedVar), so a build asked in it, or by any process it starts, never reads as the terminal's;
// runVerb also names the verb and the caller.
//
// **And a line mesh-cli asked as the controller's terminal** (novox/hq ADR 0272 §4): the serving controller runs it
// without the served mark and without a verb, names its caller, and marks it with cliTerminalVar — a mark only the
// mesh-cli path sets and every other command line the controller runs is stripped of (commandEnvironment).
func startedAtTheTerminal() bool {
if os.Getenv(servedVar) != "" || os.Getenv(verbVar) != "" {
return false
}
return os.Getenv(link.CallerVar) == "" || os.Getenv(cliTerminalVar) != ""
}
// cliTerminalVar marks a command line the serving controller runs as the controller's terminal for mesh-cli.
const cliTerminalVar = "MESH_CLI_TERMINAL"
// markServed marks this process, and so everything it starts, as the serving controller's.
func markServed() {
if err := os.Setenv(servedVar, "1"); err != nil {
panic("the serving controller could not mark its environment: " + err.Error())
}
}
// sourceForms compares sources however each is spelled: a path on a seat's holder (ADR 0111) or a URL — a
// build asked of a seat's path is registered with the URL composed from it when its outcome does not echo
// the seat. A seat's path is composed into its URL where the seat's holder is known, so both spellings of
// one repository are one; where it is not, a seat's path matches only the same seat's same path.
type sourceForms struct {
bases map[string]string // the seat's clone base, `scheme://host:port`, by seat; "" where it is not known
base func(seat string) string
}
// newSourceForms reads the seats' bases from the mesh once, when first needed.
func newSourceForms(ctx context.Context, inv *inventory.Inventory) *sourceForms {
f := &sourceForms{bases: map[string]string{}}
var world *catalogue.World
f.base = func(seat string) string {
if b, known := f.bases[seat]; known {
return b
}
if world == nil {
w := catalogue.World{}
if shelf, err := inv.Catalogue(ctx); err == nil {
if read, err := theRestOfTheMesh(ctx, inv, shelf, ""); err == nil {
w = read
}
}
world = &w
}
b, err := seatBase(*world, seat)
if err != nil {
b = ""
}
f.bases[seat] = b
return b
}
return f
}
// canonical is one spelling of a repository: lower case, no `.git`, no trailing slash, and a seat's path as
// the URL its holder serves it at where that is known.
func (f *sourceForms) canonical(repository, seat string) string {
trim := func(s string) string {
s = strings.TrimSpace(strings.ToLower(s))
s = strings.TrimRight(s, "/")
return strings.TrimRight(strings.TrimSuffix(s, ".git"), "/")
}
if seat == "" {
return trim(repository)
}
if b := f.base(seat); b != "" {
return trim(b + "/" + strings.Trim(repository, "/"))
}
return "seat:" + seat + ":" + trim(strings.Trim(repository, "/"))
}
// same says two sources are one repository.
func (f *sourceForms) same(aRepository, aSeat, bRepository, bSeat string) bool {
if aRepository == "" || bRepository == "" {
return false
}
return f.canonical(aRepository, aSeat) == f.canonical(bRepository, bSeat)
}
// buildsFrom says the catalogue builds some module from this repository.
func (f *sourceForms) buildsFrom(entries []inventory.Entry, repository, seat string) bool {
for _, e := range entries {
if e.Provided || e.Source.Repository == "" {
continue
}
if f.same(e.Source.Repository, e.Source.Seat, repository, seat) {
return true
}
}
return false
}
// forgeFacts is what the mesh's forge says of a repository a module is registered from (novox/hq ADR 0266).
type forgeFacts struct {
// ID is the forge's own id of the repository: what tells it from one deleted and made again by its name.
ID int64
// Guarded is whether the branch is protected as a module's trunk must be: no direct push, at least one
// required status, and no administrator merging past one. Why says what is missing when it is not.
Guarded bool
Why string
}
// askTheForge asks the forge, through its module's tools on the bus, for a repository's id and its branch's
// protection, waiting out a forge not served for a moment (novox/hq issue 486). A variable so a test needs no
// forge.
var askTheForge = func(ctx context.Context, owner, repo, branch string) (forgeFacts, error) {
return forgeWaitedOut(ctx, forgeWait, owner+"/"+repo, func(ctx context.Context) (forgeFacts, error) {
return askTheForgeOnce(ctx, owner, repo, branch)
})
}
// waitOut bounds how long a question is asked again while its answer is "not now": a pause of first, doubling
// to most, for at most bound in all. now and sleep are the clock, so a test waits in no real time.
type waitOut struct {
bound, first, most time.Duration
now func() time.Time
sleep func(context.Context, time.Duration) error
}
// forgeWait is the registration's wait for the forge (novox/hq issue 486). On 2026-10-11 a build's registration
// asked the forge while node-tools on the control-node re-served after a rollout of the tool runner, heard
// "nothing serves gitea.gitea_api", and failed its plan; the forge answered again seconds later. A re-serve takes
// about 10 s, so 60 s waits one out with room to spare. It is a cap on the whole wait, an ask in flight included:
// the registration holds the controller's one receive loop while it waits, and a loop held two minutes reads as
// deaf. The pauses are those the build machine waits the registry with (novox/hq issue 457).
var forgeWait = waitOut{bound: 60 * time.Second, first: time.Second, most: 10 * time.Second,
now: time.Now, sleep: sleepOrDone}
// sleepOrDone pauses for d, or less when ctx ends first.
func sleepOrDone(ctx context.Context, d time.Duration) error {
t := time.NewTimer(d)
defer t.Stop()
select {
case <-ctx.Done():
return ctx.Err()
case <-t.C:
return nil
}
}
// forgeNotNow is an answer that says the forge's tool is not served now — the tool runner re-serving — not that
// the forge refused. Only that is asked again: a refusal, an unreadable answer or a repository with no id is the
// forge's word, and a tool that did not answer in time has already held the receive loop its whole wait.
func forgeNotNow(err error) bool { return errors.Is(err, link.ErrNothingServes) }
// forgeWaitedOut asks the forge about repository, asking again while it answers "not served now", all of it
// within w's bound, an ask in flight included (novox/hq issue 486). Past the bound it fails, saying it waited,
// for how long, and what the forge last said; when the caller's own context ends, it says that instead.
func forgeWaitedOut(ctx context.Context, w waitOut, repository string,
ask func(context.Context) (forgeFacts, error)) (forgeFacts, error) {
capped, cancel := context.WithTimeout(ctx, w.bound)
defer cancel()
began := w.now()
pause := w.first
for asked := 1; ; asked++ {
facts, err := ask(capped)
if err == nil {
if asked > 1 {
fmt.Printf("the forge answered about %s again, after %s (novox/hq issue 486)\n", repository,
w.now().Sub(began).Round(time.Second))
}
return facts, nil
}
if ctx.Err() != nil {
return forgeFacts{}, err // the caller's own context ended: no wait of ours to speak of
}
gaveUp := func() (forgeFacts, error) {
return forgeFacts{}, fmt.Errorf("waited %s for the forge to answer about %s, asking %d times, and it "+
"did not (novox/hq issue 486): %w", w.now().Sub(began).Round(time.Second), repository, asked, err)
}
if capped.Err() != nil {
return gaveUp() // the bound ended an ask in flight
}
if !forgeNotNow(err) {
return forgeFacts{}, err
}
if w.now().Sub(began)+pause > w.bound {
return gaveUp()
}
fmt.Printf("the forge cannot be asked about %s now (%v); asking again in %s, for at most %s in all "+
"(novox/hq issue 486)\n", repository, err, pause, w.bound)
if serr := w.sleep(capped, pause); serr != nil {
if ctx.Err() != nil {
return forgeFacts{}, fmt.Errorf("stopped waiting for the forge to answer about %s: %w", repository,
ctx.Err())
}
return gaveUp()
}
pause = min(2*pause, w.most)
}
}
// askTheForgeOnce asks the forge once, through its module's tools on the bus.
func askTheForgeOnce(ctx context.Context, owner, repo, branch string) (forgeFacts, error) {
js, err := aBus()
if err != nil {
return forgeFacts{}, err
}
defer js.Close()
bus := link.OverNATS{Conn: js.Conn()}
ask := func(tool string, args map[string]any, into any) error {
raw, _ := json.Marshal(args)
answer, err := link.Ask(ctx, bus, "gitea", tool, raw, 30*time.Second)
if err != nil {
return err
}
if answer.Error != "" {
return fmt.Errorf("gitea.%s: %s", tool, answer.Error)
}
return json.Unmarshal(answer.Result, into)
}
var found struct {
Result struct {
ID int64 `json:"id"`
FullName string `json:"full_name"`
} `json:"result"`
}
if err := ask("gitea_api", map[string]any{"path": "/repos/" + owner + "/" + repo}, &found); err != nil {
return forgeFacts{}, err
}
if found.Result.ID == 0 {
return forgeFacts{}, fmt.Errorf("the forge named no id for %s/%s", owner, repo)
}
var rules struct {
Rules []protectionRule `json:"rules"`
}
if err := ask("gitea_branch_protection_get", map[string]any{"owner": owner, "repo": repo}, &rules); err != nil {
return forgeFacts{}, err
}
facts := judgedRule(rules.Rules, branch)
facts.ID = found.Result.ID
return facts, nil
}
// protectionRule is a branch protection rule as the forge's tool summarises it.
type protectionRule struct {
Rule string `json:"rule"`
Push bool `json:"push"`
RequiredStatuses []string `json:"required_statuses"`
AdminMayOverride bool `json:"admin_may_override"`
}
// judgedRule judges the one rule the forge applies to a branch, as the forge picks it: the rule named for the
// branch, else the first glob rule, in the forge's order, that covers it (gitea's first matching rule). Never a
// later rule that happens to be stronger: the forge does not read it.
func judgedRule(rules []protectionRule, branch string) forgeFacts {
var applied *protectionRule
for i := range rules {
if rules[i].Rule == branch {
applied = &rules[i]
break
}
}
if applied == nil {
for i := range rules {
if ruleCovers(rules[i].Rule, branch) {
applied = &rules[i]
break
}
}
}
switch r := applied; {
case r == nil:
return forgeFacts{Why: fmt.Sprintf("no protection rule covers %s", branch)}
case r.Push:
return forgeFacts{Why: fmt.Sprintf("the rule %s lets a person push to %s directly", r.Rule, branch)}
case len(r.RequiredStatuses) == 0:
return forgeFacts{Why: fmt.Sprintf("the rule %s requires no status before a merge into %s", r.Rule, branch)}
case r.AdminMayOverride:
return forgeFacts{Why: fmt.Sprintf("the rule %s lets an administrator merge into %s past a status", r.Rule, branch)}
}
return forgeFacts{Guarded: true}
}
// ruleCovers says a protection rule's name — a branch, or a glob of them — covers a branch, as the forge reads it.
func ruleCovers(rule, branch string) bool {
if rule == branch {
return true
}
if !strings.ContainsAny(rule, "*?[") {
return false
}
var re strings.Builder
re.WriteString("^")
for i := 0; i < len(rule); i++ {
switch c := rule[i]; {
case c == '*' && i+1 < len(rule) && rule[i+1] == '*':
re.WriteString(".*")
i++
case c == '*':
re.WriteString("[^/]*")
case c == '?':
re.WriteString("[^/]")
default:
re.WriteString(regexp.QuoteMeta(string(c)))
}
}
re.WriteString("$")
ok, _ := regexp.MatchString(re.String(), branch)
return ok
}
// onTheForge is a source's owner and name on the mesh's own forge (the git seat's holder), and whether it is
// there at all.
func (f *sourceForms) onTheForge(repository, seat string) (owner, name string, ok bool) {
var rest string
switch {
case seat == gitSeat:
rest = strings.Trim(repository, "/")
case seat == "":
base := f.base(gitSeat)
if base == "" {
return "", "", false
}
url, prefix := f.canonical(repository, ""), f.canonical(base, "")+"/"
if !strings.HasPrefix(url, prefix) {
return "", "", false
}
// The case the forge spells it with: the URL as given, past the base.
rest = strings.TrimSuffix(strings.Trim(repository[len(prefix):], "/"), ".git")
default:
return "", "", false
}
parts := strings.Split(rest, "/")
if len(parts) != 2 || parts[0] == "" || parts[1] == "" {
return "", "", false
}
return parts[0], parts[1], true
}
// mayRegisterFrom says whether a build's outcome may register module from the source it was built from, and the
// forge's id of that repository to record (novox/hq ADR 0266). Through any verb, only:
//
// - from the module's registered repository — the same repository by the forge's own id, not only its name; or,
// for a module new to the catalogue, from a repository the catalogue builds another module from;
// - and from a repository on the mesh's forge whose trunk is protected as a trunk must be: no direct push, at
// least one required status, no administrator merging past one.
//
// Anything else only when the build request was kept as asked at the controller's terminal, for this very
// repository and path. path is the module's directory as built; branch the trunk it is registered from.
func mayRegisterFrom(ctx context.Context, inv *inventory.Inventory, module string, built inventory.Source,
buildID, path, branch string) (int64, error) {
forms := newSourceForms(ctx, inv)
was, err := inv.SourceOf(ctx, module)
isNew := errors.Is(err, inventory.ErrNoSuchModule)
if err != nil && !isNew {
return 0, err
}
terminal, err := askedHereFor(ctx, inv, forms, buildID, built, path)
if err != nil {
return 0, err
}
refuse := func(why string) (int64, error) {
return 0, fmt.Errorf("%w: %s. A module is registered from such a source only by a build asked at the "+
"controller's terminal, never through a verb: a registered module is what the next push sends, and whoever "+
"may call a verb includes agents (novox/hq ADR 0266)", errNotItsSource, why)
}
var why string
var alongside []inventory.Entry // the modules a new one's repository already builds
switch {
case !isNew && forms.same(was.Repository, was.Seat, built.Repository, built.Seat):
case !isNew:
registered := was.Repository
if registered == "" {
registered = "no repository (it was handed over by hand)"
}
why = fmt.Sprintf("%s is built from %s, and this build is of %s", module, sourceWords(registered, was.Seat),
sourceWords(built.Repository, built.Seat))
default:
entries, err := inv.Catalogued(ctx)
if err != nil {
return 0, err
}
for _, e := range entries {
if !e.Provided && e.Source.Repository != "" &&
forms.same(e.Source.Repository, e.Source.Seat, built.Repository, built.Seat) {
alongside = append(alongside, e)
}
}
if len(alongside) == 0 {
why = fmt.Sprintf("%s is new to the catalogue, and %s is no repository the catalogue builds a module from",
module, sourceWords(built.Repository, built.Seat))
}
}
// **Or the move the operator approved** (novox/hq ADR 0420 §7, issue 1025): a known module built from another
// repository by a merge, whose pull request mesh-delivery merged on the operator's Approve of that head.
// A verb alone, or a merge with no such warrant, is refused as before.
movedWhy := ""
if why != "" && !terminal && !isNew {
accepted, err := approvedMove(ctx, inv, forms, buildID, built)
switch {
case err == nil:
movedWhy, why = accepted, ""
case !errors.Is(err, errNotAMerge):
fmt.Printf("%s: %s is built from another repository, and no approved move covers this build: %v\n",
buildID, module, err)
}
}
if why != "" && !terminal {
return refuse(why)
}
owner, name, onForge := forms.onTheForge(built.Repository, built.Seat)
if !onForge {
if terminal {
fmt.Printf("%s: %s is not on the mesh's forge — registered, as asked at the controller's terminal\n",
buildID, sourceWords(built.Repository, built.Seat))
return 0, nil
}
return refuse(fmt.Sprintf("%s is not on the mesh's forge, so whether its trunk is protected cannot be read",
sourceWords(built.Repository, built.Seat)))
}
facts, err := askTheForge(ctx, owner, name, branch)
if err != nil {
if terminal {
fmt.Printf("%s: the forge could not be asked about %s/%s (%v) — registered, as asked at the controller's "+
"terminal\n", buildID, owner, name, err)
return 0, nil
}
return refuse(fmt.Sprintf("the forge could not say whether %s/%s's %s is protected: %v", owner, name, branch, err))
}
if terminal {
if why != "" || !facts.Guarded {
fmt.Printf("%s: %s — registered, as asked at the controller's terminal\n", buildID,
strings.Trim(why+"; "+facts.Why, "; "))
}
return facts.ID, nil
}
if !facts.Guarded {
return refuse(fmt.Sprintf("%s/%s's %s is not protected as a module's trunk must be: %s", owner, name, branch,
facts.Why))
}
// The same repository by the forge's id, not only its name: one deleted and made again is another.
recorded := map[string]int64{}
// A moved module is registered from the repository the operator approved, not the one it was registered from.
if !isNew && movedWhy == "" {
id, err := inv.SourceIdentity(ctx, module)
if err != nil {
return 0, err
}
recorded[module] = id
}
for _, e := range alongside {
id, err := inv.SourceIdentity(ctx, e.Manifest.Module)
if err != nil {
return 0, err
}
recorded[e.Manifest.Module] = id
}
for m, id := range recorded {
if id != 0 && id != facts.ID {
return refuse(fmt.Sprintf("%s/%s is not the repository %s was registered from: the forge knows it as "+
"repository %d, and %s was registered from repository %d — one of that name deleted and made again",
owner, name, m, facts.ID, m, id))
}
}
if movedWhy != "" {
fmt.Printf("%s: %s moved from %s to %s/%s — registered at the operator's Approve: %s\n", buildID, module,
sourceWords(was.Repository, was.Seat), owner, name, movedWhy)
}
return facts.ID, nil
}
// askedHereFor says the build was asked at the controller's terminal, for this repository and this path: a kept
// request marked so, whose source is the outcome's (novox/hq ADR 0266).
func askedHereFor(ctx context.Context, inv *inventory.Inventory, forms *sourceForms, buildID string,
built inventory.Source, path string) (bool, error) {
r, found, err := inv.BuildRequestByID(ctx, buildID)
if err != nil || !found || !r.AtTerminal {
return false, err
}
if !forms.same(r.Repository, r.Seat, built.Repository, built.Seat) ||
strings.Trim(r.Path, "/") != strings.Trim(path, "/") {
fmt.Printf("%s was asked at the terminal of %s at %q, and its outcome is of %s at %q: not the terminal's\n",
buildID, sourceWords(r.Repository, r.Seat), r.Path, sourceWords(built.Repository, built.Seat), path)
return false, nil
}
return true, nil
}
// sourceWords is a source as a person reads it.
func sourceWords(repository, seat string) string {
if seat == "" {
return repository
}
return buildSource{Repository: repository, Seat: seat}.String()
}
// verbMayAsk refuses, for a call through a verb, a build of a repository the catalogue builds no module from
// (novox/hq ADR 0266): the build node would run what an agent wrote, and its outcome could never be
// registered anyway. url is the repository as it is cloned. At the terminal anything may be asked.
func verbMayAsk(ctx context.Context, source buildSource, url string) error {
if startedAtTheTerminal() {
return nil
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
entries, err := open.inventory.Catalogued(ctx)
if err != nil {
return err
}
forms := newSourceForms(ctx, open.inventory)
if forms.buildsFrom(entries, source.Repository, source.Seat) || forms.buildsFrom(entries, url, "") {
return nil
}
return terminalRefusal("%s is no repository the catalogue builds a module from, and a build of any other is "+
"asked at the controller's terminal only, never through a verb: a build node runs what the repository "+
"says, and its outcome would register a module the next push sends — whoever may call a verb includes "+
"agents (novox/hq ADR 0266). Nothing was asked", source)
}
@@ -1,353 +0,0 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"os/exec"
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func theCatalogue(t *testing.T) *stores {
t.Helper()
open := aMesh(t)
ctx := t.Context()
for _, b := range []link.BuildResult{
onTrunk("build-sudo", "novox/mesh-catalog", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "1"}),
onTrunk("build-host", "novox/mesh-host", "git", "", map[string]any{"module": "mesh-host", "version": "1"}),
} {
keptAsked(t, open.inventory, b.ID, b.Source.Repository, b.Path, true)
if _, _, err := takeIn(ctx, open.inventory, b); err != nil {
t.Fatal(err)
}
}
return open
}
func TestABuildFromAnAgentsRepositoryIsNotRegisteredAsAModuleTheMeshHolds(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
for _, c := range []struct {
name, repository, path, module string
}{
{"its own repository naming sudo", "agent/sudo", "modules/sudo", "sudo"},
{"a fork of the catalogue", "agent/mesh-catalog", "modules/sudo", "sudo"},
{"a fork of the node-engine", "agent/mesh-host", "", "mesh-host"},
} {
t.Run(c.name, func(t *testing.T) {
id := "build-" + strings.ReplaceAll(c.repository, "/", "-")
keptAsked(t, open.inventory, id, c.repository, c.path, false) // through the build verb
evil := onTrunk(id, c.repository, "git", c.path, map[string]any{"module": c.module, "version": "evil"})
_, _, err := takeIn(ctx, open.inventory, evil)
if !errors.Is(err, errNotItsSource) {
t.Fatalf("a build of %s was taken in as %s: %v", c.repository, c.module, err)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if got := shelf[c.module].Version; got != "1" {
t.Fatalf("%s is now %q, from %s", c.module, got, c.repository)
}
if _, found, _ := open.inventory.BuildByID(ctx, id); !found {
t.Errorf("the refused build %s is not recorded", id)
}
})
}
}
func TestANewModuleFromARepositoryTheCatalogueDoesNotBuildFromIsNotRegistered(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
keptAsked(t, open.inventory, "build-new", "agent/tools", "", false)
_, _, err := takeIn(ctx, open.inventory, onTrunk("build-new", "agent/tools", "git", "",
map[string]any{"module": "agent-tools", "version": "1"}))
if !errors.Is(err, errNotItsSource) {
t.Fatalf("a new module from an agent's repository was taken in: %v", err)
}
// And one asked of nobody here — an outcome on the bus no request was kept for — the same.
_, _, err = takeIn(ctx, open.inventory, onTrunk("build-unasked", "agent/tools", "git", "",
map[string]any{"module": "agent-tools", "version": "1"}))
if !errors.Is(err, errNotItsSource) {
t.Fatalf("an outcome nobody asked for was taken in: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["agent-tools"].Module != "" {
t.Fatal("the refused module is in the catalogue")
}
}
func TestAtTheTerminalAnotherRepositoryIsRegistered(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
keptAsked(t, open.inventory, "build-moved", "novox/sudo", "", true)
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-moved", "novox/sudo", "git", "",
map[string]any{"module": "sudo", "version": "2"})); err != nil {
t.Fatalf("a move the operator asked for at the terminal was refused: %v", err)
}
if src, _ := open.inventory.SourceOf(ctx, "sudo"); src.Repository != "novox/sudo" {
t.Fatalf("sudo is built from %q", src.Repository)
}
if err := open.inventory.RecordBuildRequest(ctx, inventory.BuildRequest{ID: "build-external",
Repository: "http://forge.internal:20000/someone/app.git", For: "build", AtTerminal: true}); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-external", "someone/app", "", "",
map[string]any{"module": "app", "version": "1"})); err != nil {
t.Fatalf("a new module the operator asked for at the terminal was refused: %v", err)
}
}
func TestADeliveryFromTheRegisteredRepositoryIsRegistered(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
rebuilt := onTrunk("build-plan", "novox/mesh-catalog", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "2"})
if err := open.inventory.RecordBuildRequest(ctx, inventory.BuildRequest{ID: rebuilt.ID,
Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/sudo", For: "plan"}); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, rebuilt); err != nil {
t.Fatalf("a plan's build of the module's own repository was refused: %v", err)
}
added := onTrunk("build-merge", "novox/mesh-catalog", "git", "modules/zram", map[string]any{"module": "zram", "version": "1"})
if _, _, err := takeIn(ctx, open.inventory, added); err != nil {
t.Fatalf("a module a merge added to the catalogue repository was refused: %v", err)
}
shelf, _ := open.inventory.Catalogue(ctx)
if shelf["sudo"].Version != "2" || shelf["zram"].Module == "" {
t.Fatalf("not registered: sudo %q, zram %q", shelf["sudo"].Version, shelf["zram"].Module)
}
}
func TestAVerbAsksNoBuildOfARepositoryTheCatalogueDoesNotBuildFrom(t *testing.T) {
theCatalogue(t)
ctx := t.Context()
t.Setenv(verbVar, "build")
t.Setenv(link.CallerVar, "node-tools.anchor, through the mesh-controller seat")
err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, "http://forge.internal:20000/agent/sudo.git")
var policy *heldAtTheTerminal
if !errors.As(err, &policy) {
t.Fatalf("a verb's build of an agent's repository was asked: %v", err)
}
if err := verbMayAsk(ctx, buildSource{Repository: "novox/mesh-catalog", Seat: "git"},
"http://forge.internal:20000/novox/mesh-catalog.git"); err != nil {
t.Fatalf("a verb's build of the catalogue repository was refused: %v", err)
}
t.Setenv(verbVar, "")
t.Setenv(link.CallerVar, "")
if err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, ""); err != nil {
t.Fatalf("the terminal was refused: %v", err)
}
}
func TestARollbackNeverPutsBackABuildFromAnotherRepository(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
inv := open.inventory
fork := onTrunk("build-1791500000000000000", "agent/mesh-catalog", "git", "modules/sudo",
map[string]any{"module": "sudo", "version": "evil"})
fork.Commit = "c0ffee0123456789" // the commit sudo was registered at
keptAsked(t, inv, fork.ID, "agent/mesh-catalog", "modules/sudo", false)
if _, _, err := takeIn(ctx, inv, fork); !errors.Is(err, errNotItsSource) {
t.Fatalf("the fork's build was taken in: %v", err)
}
// Asked after the registered build, whenever the test runs: an id naming a fixed moment read as older
// than the registered build once the clock passed it (2026-10-10 02:40 UTC), and the test failed on main.
failed := onTrunk(fmt.Sprintf("build-%d", time.Now().Add(time.Hour).UnixNano()), "novox/mesh-catalog", "git", "modules/sudo",
map[string]any{"module": "sudo", "version": "2"})
failed.Commit = "badbadbad0123456"
if _, _, err := takeIn(ctx, inv, failed); err != nil {
t.Fatal(err)
}
record, _, err := inv.BuildByID(ctx, failed.ID)
if err != nil {
t.Fatal(err)
}
previous, found, err := inv.PreviousBuild(ctx, "sudo", "c0ffee0123456789", record)
if err != nil {
t.Fatal(err)
}
if found && previous.ID == fork.ID {
t.Fatalf("a rollback would put back the fork's build %s", previous.ID)
}
if !found || previous.ID != "build-sudo" {
t.Fatalf("a rollback puts back %q (found %v), want the registered build-sudo", previous.ID, found)
}
}
func TestATrunkTheForgeDoesNotProtectRegistersNothing(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
theForge.Store("novox/unguarded", forgeFacts{ID: 7, Why: "the rule main lets a person push to main directly"})
t.Cleanup(func() { theForge.Delete("novox/unguarded") })
first := onTrunk("build-unguarded-1", "novox/unguarded", "git", "", map[string]any{"module": "unguarded", "version": "1"})
keptAsked(t, open.inventory, first.ID, "novox/unguarded", "", true)
if _, _, err := takeIn(ctx, open.inventory, first); err != nil {
t.Fatalf("at the terminal: %v", err)
}
again := onTrunk("build-unguarded-2", "novox/unguarded", "git", "", map[string]any{"module": "unguarded", "version": "2"})
if _, _, err := takeIn(ctx, open.inventory, again); !errors.Is(err, errNotItsSource) ||
!strings.Contains(err.Error(), "push to main directly") {
t.Fatalf("a rebuild from an unprotected trunk was taken in: %v", err)
}
theForge.Store("novox/unguarded", errors.New("nothing serves gitea.gitea_api"))
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-unguarded-3", "novox/unguarded", "git", "",
map[string]any{"module": "unguarded", "version": "3"})); !errors.Is(err, errNotItsSource) {
t.Fatalf("a forge that could not say was read as a protected trunk: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["unguarded"].Version != "1" {
t.Fatalf("unguarded is %q", shelf["unguarded"].Version)
}
}
func TestARepositoryMadeAgainUnderItsNameIsNotTheModulesSource(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
theForge.Store("novox/remade", forgeFacts{ID: 100, Guarded: true})
t.Cleanup(func() { theForge.Delete("novox/remade") })
first := onTrunk("build-remade-1", "novox/remade", "git", "", map[string]any{"module": "remade", "version": "1"})
keptAsked(t, open.inventory, first.ID, "novox/remade", "", true)
if _, _, err := takeIn(ctx, open.inventory, first); err != nil {
t.Fatal(err)
}
if id, _ := open.inventory.SourceIdentity(ctx, "remade"); id != 100 {
t.Fatalf("the forge's id was not recorded: %d", id)
}
theForge.Store("novox/remade", forgeFacts{ID: 101, Guarded: true}) // deleted, and made again by an agent
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-remade-2", "novox/remade", "git", "",
map[string]any{"module": "remade", "version": "evil"})); !errors.Is(err, errNotItsSource) ||
!strings.Contains(err.Error(), "made again") {
t.Fatalf("a repository made again under the name was taken in: %v", err)
}
// And a new module from it, beside the one registered from the first, the same.
if _, _, err := takeIn(ctx, open.inventory, onTrunk("build-remade-3", "novox/remade", "git", "modules/other",
map[string]any{"module": "other", "version": "1"})); !errors.Is(err, errNotItsSource) {
t.Fatalf("a new module from a repository made again was taken in: %v", err)
}
}
func TestATerminalRequestCoversOnlyWhatItAsked(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
if err := open.inventory.RecordBuildRequest(ctx, inventory.BuildRequest{ID: "build-asked", Repository: "novox/app",
Seat: "git", Path: "modules/app", For: "build", AtTerminal: true}); err != nil {
t.Fatal(err)
}
other := onTrunk("build-asked", "agent/sudo", "git", "modules/app", map[string]any{"module": "sudo", "version": "evil"})
if _, _, err := takeIn(ctx, open.inventory, other); !errors.Is(err, errNotItsSource) {
t.Fatalf("an outcome of another repository under a terminal request's id was taken in: %v", err)
}
elsewhere := onTrunk("build-asked", "novox/app", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "evil"})
if _, _, err := takeIn(ctx, open.inventory, elsewhere); !errors.Is(err, errNotItsSource) {
t.Fatalf("an outcome of another path under a terminal request's id was taken in: %v", err)
}
}
func TestTheServingControllerIsNeverTheTerminal(t *testing.T) {
t.Setenv(verbVar, "")
t.Setenv(link.CallerVar, "")
t.Setenv(servedVar, "")
if !startedAtTheTerminal() {
t.Fatal("a process started by hand is not the terminal")
}
markServed()
if startedAtTheTerminal() {
t.Fatal("the serving controller reads as the terminal")
}
child := exec.Command(os.Args[0], "-test.run=^$")
child.Env = os.Environ()
if !slices.Contains(child.Env, servedVar+"=1") {
t.Fatal("what the serving controller starts does not carry its mark")
}
}
func TestAnUnknownIdentityKeepsTheRecordedOne(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
if err := open.inventory.SetSourceIdentity(ctx, "sudo", 100); err != nil {
t.Fatal(err)
}
if err := open.inventory.SetSourceIdentity(ctx, "sudo", 0); err != nil {
t.Fatal(err)
}
if id, _ := open.inventory.SourceIdentity(ctx, "sudo"); id != 100 {
t.Fatalf("the recorded id became %d", id)
}
}
func TestAModuleMovesOnTheOperatorsApproveOfTheMergeThatMovedIt(t *testing.T) {
const commit = "c0ffee0123456789"
for _, c := range []struct {
what string
change func(*anApprovedMerge)
unkept bool
request string // who asked: "merge", or "build" through a verb
moved bool
}{
{what: "the operator's Approve of the merged head", request: "merge", moved: true},
{what: "no merge mesh-delivery announced", unkept: true, request: "merge"},
{what: "a warrant for other heads", request: "merge", change: func(a *anApprovedMerge) {
a.merge.Heads = map[string]string{"novox/mesh-host@aaaaaaaaaaaa": "e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5"}
}},
{what: "a Decline", request: "merge", change: func(a *anApprovedMerge) {
a.warrant.Option, a.warrant.Label, a.merge.Option = "decline", "Decline", "decline"
}},
{what: "a person who is not the operator", request: "merge", change: func(a *anApprovedMerge) { a.warrant.By.Who = "agent" }},
{what: "a build asked through a verb", request: "build"},
} {
open := theCatalogue(t)
ctx := t.Context()
inv := open.inventory
a := approvedMerge(t)
a.merge.Owner, a.merge.Repo, a.merge.Number, a.merge.HeadSHA = "novox", "mesh-host", 7, "a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1"
if c.change != nil {
c.change(&a)
}
wasKept, wasAsk := keptDeliveryMerge, routerAskOf
keptDeliveryMerge = func(_ context.Context, owner, repo string, number int, head string) (deliveryMerge, bool, error) {
return a.merge, !c.unkept && owner == "novox" && repo == "mesh-host" && number == 7 && head == a.merge.HeadSHA, nil
}
routerAskOf = func(_ context.Context, _ *inventory.Inventory, id string) (*asks.Ask, *asks.Warrant, error) {
if id != a.merge.AskID {
t.Errorf("%s: the router was asked for %q", c.what, id)
}
return &a.ask, &a.warrant, nil
}
t.Cleanup(func() { keptDeliveryMerge, routerAskOf = wasKept, wasAsk })
event, _ := json.Marshal(map[string]any{"owner": "novox", "repo": "mesh-host", "base": "main", "number": 7,
"head_sha": a.merge.HeadSHA, "merge_commit_sha": commit})
if _, err := inv.AddMerge(ctx, inventory.BatchedMerge{Repository: "novox/mesh-host", Branch: "main", Commit: commit,
Merged: time.Now(), Heard: time.Now(), Event: event}); err != nil {
t.Fatal(err)
}
id := fmt.Sprintf("build-%d", time.Now().Add(time.Hour).UnixNano())
if err := inv.RecordBuildRequest(ctx, inventory.BuildRequest{ID: id, Repository: "novox/mesh-host", Seat: "git",
Path: "modules/sudo", Ref: "main", Commit: commit, For: c.request}); err != nil {
t.Fatal(err)
}
moved := onTrunk(id, "novox/mesh-host", "git", "modules/sudo", map[string]any{"module": "sudo", "version": "2"})
moved.Commit = commit
_, _, err := takeIn(ctx, inv, moved)
src, _ := inv.SourceOf(ctx, "sudo")
switch {
case c.moved && err != nil:
t.Errorf("%s: the move was refused: %v", c.what, err)
case c.moved && src.Repository != "novox/mesh-host":
t.Errorf("%s: sudo is built from %q", c.what, src.Repository)
case !c.moved && !errors.Is(err, errNotItsSource):
t.Errorf("%s: the move was not refused as ADR 0266 has it: %v", c.what, err)
case !c.moved && src.Repository != "novox/mesh-catalog":
t.Errorf("%s: a refused move changed sudo's source to %q", c.what, src.Repository)
}
}
}
-128
View File
@@ -1,128 +0,0 @@
package main
import (
"context"
"encoding/json"
"hash/fnv"
"strings"
"sync"
"testing"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The routes to root a review of ADR 0266 found (novox/hq ADR 0266 §7): an agent makes a repository of its
// own — or forks one the mesh builds from — commits a module.json naming a module the mesh runs everywhere
// (`sudo`, granting itself a rule without a password; `mesh-host`, the node-engine), and asks the `build` verb
// for it. Its outcome was registered under that name from the agent's repository, and the next push sent it.
// onTrunk is an outcome of a commit on its repository's trunk, as the build seat says it.
func onTrunk(id, repository, seat, path string, manifest map[string]any) link.BuildResult {
raw, _ := json.Marshal(manifest)
r := link.BuildResult{ID: id, Repository: "http://forge.internal:20000/" + repository + ".git", Path: path,
Ref: "main", On: "anchor", Commit: "c0ffee0123456789", Manifest: raw,
Trunk: "main", OnTrunk: true, Branches: []string{"main"}}
if seat != "" {
r.Source = &link.SourceOnSeat{Seat: seat, Repository: repository}
}
return r
}
// keptAsked keeps a build request as the asker would: through a verb, or at the terminal.
func keptAsked(t *testing.T, inv *inventory.Inventory, id, repository, path string, atTerminal bool) {
t.Helper()
if err := inv.RecordBuildRequest(t.Context(), inventory.BuildRequest{ID: id, Repository: repository, Seat: "git",
Path: path, For: "build", AtTerminal: atTerminal}); err != nil {
t.Fatal(err)
}
}
// theCatalogue is a mesh whose sudo is built from the catalogue repository and whose node-engine from its own.
// The operator at the terminal may still move a module, or add one from a new repository.
// The delivery's flow is untouched: a merge's rebuild of a module from its own repository, and a merge adding
// a module to a repository the catalogue builds from (novox/hq issue 300), are registered with no terminal.
// Through a verb, a build of a repository the catalogue builds nothing from is not even asked: the build node
// would run what the agent wrote.
// asTheOperator keeps a build as asked at the controller's terminal, as the operator's first build of a module
// from a repository the catalogue does not yet build from is (novox/hq ADR 0266), and hands it back.
func asTheOperator(t *testing.T, inv *inventory.Inventory, b link.BuildResult) link.BuildResult {
t.Helper()
repository, seat := b.Repository, ""
if b.Source != nil {
repository, seat = b.Source.Repository, b.Source.Seat
}
if err := inv.RecordBuildRequest(t.Context(), inventory.BuildRequest{ID: b.ID, Repository: repository, Seat: seat,
Path: b.Path, For: "build", AtTerminal: true}); err != nil {
t.Fatal(err)
}
return b
}
// A rollback puts back only a build of the module's own repository: an agent's build of the module's name,
// recorded and refused, at the very commit the machine ran before (a fork carries it), is never registered by
// the back door of a failed gate.
// theForge is what the forge says in a test, by owner/name: a repository not named here is protected as a
// trunk must be, with an id of its own.
var theForge sync.Map
func init() {
askTheForge = func(_ context.Context, owner, repo, _ string) (forgeFacts, error) {
if said, ok := theForge.Load(owner + "/" + repo); ok {
switch f := said.(type) {
case error:
return forgeFacts{}, f
case forgeFacts:
return f, nil
}
}
h := fnv.New32a()
_, _ = h.Write([]byte(owner + "/" + repo))
return forgeFacts{ID: int64(h.Sum32()), Guarded: true}, nil
}
}
// A module's trunk the forge does not protect — direct pushes, no required status — or a forge that cannot say,
// registers nothing through a verb: the trunk rule means nothing on a branch anyone pushes to.
// A repository deleted and made again under the module's repository's name is another repository: the forge's
// id, recorded at registration, tells them apart.
// The terminal's mark is the operator's for the repository and path they asked: an outcome of another, under that
// build's id, is not theirs.
// The serving controller, and everything it starts, is never the terminal: a build asked in it reads as asked
// through the mesh even when no verb and no caller is named.
// The forge applies one rule to a branch: the rule named for it, else the first glob that covers it. A stronger
// rule later in the list is not what guards the branch, and is not read as if it were (the confirmation review).
func TestTheRuleJudgedIsTheOneTheForgeApplies(t *testing.T) {
guarded := protectionRule{Rule: "*", RequiredStatuses: []string{"mesh/merge-gate"}}
open := protectionRule{Rule: "main", Push: true, RequiredStatuses: []string{"mesh/merge-gate"}}
if f := judgedRule([]protectionRule{guarded, open}, "main"); f.Guarded {
t.Error("an exact rule letting pushes was passed over for a glob that guards")
}
if f := judgedRule([]protectionRule{{Rule: "ma*", RequiredStatuses: nil}, {Rule: "*", RequiredStatuses: []string{"x"}}},
"main"); f.Guarded || !strings.Contains(f.Why, "ma*") {
t.Errorf("the first glob covering the branch was not the one judged: %+v", f)
}
if f := judgedRule([]protectionRule{{Rule: "release/*"}, {Rule: "main", RequiredStatuses: []string{"x"}}}, "main"); !f.Guarded {
t.Errorf("the rule named for the branch was not judged: %+v", f)
}
if f := judgedRule(nil, "main"); f.Guarded {
t.Error("no rule is no protection")
}
}
// An id the forge could not give keeps the id already recorded.
// **The operator's Approve of a reviewed group registers a moved module from its new repository** (novox/hq ADR 0420 §7,
// issue 1025). sudo is built from the catalogue; a merge into the repository of node-engine's module, announced by
// mesh-delivery with the ask it merged on, brings its build from there. Accepted with the operator's Approve of the
// merged head; refused with no kept merge, a warrant for other heads, a Decline, a person who is not the operator, and
// through a verb.
-34
View File
@@ -1,34 +0,0 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/link"
)
// A build's outcome is taken in the same way whoever hears it (novox/hq issue 176): recorded, and
// the module registered with its source as the seat and path when the request said so — never the
// URL. A definition naming an installation is recorded and not registered; a failure is recorded
// and said.
// novox/hq 04-ISSUES/215: a build asked at a commit is recorded as built from that commit, and the
// module keeps following the branch it followed — a new one, the default branch.
// novox/hq 04-ISSUES/219: an older request heard after a newer one is recorded and not registered,
// so a push sends what the newer request built.
func TestABuildIDSaysWhenItWasAsked(t *testing.T) {
at := time.Date(2026, 10, 3, 21, 51, 57, 392539762, time.UTC)
if got, ok := link.BuildAskedAt(link.NewBuildID(at)); !ok || !got.Equal(at) {
t.Errorf("read back %v %v; want %v", got, ok, at)
}
if got, ok := link.BuildAskedAt("build-1791064317392539762"); !ok || got.Format(time.TimeOnly) != "21:51:57" {
t.Errorf("the incident's id reads as %v %v", got, ok)
}
for _, id := range []string{"b-1", "build-2", "build-", "build-x", ""} {
if _, ok := link.BuildAskedAt(id); ok {
t.Errorf("%q read as a request time", id)
}
}
}
-410
View File
@@ -1,410 +0,0 @@
package main
import (
"context"
"errors"
"flag"
"fmt"
"sort"
"strings"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The bus as a planned step (novox/hq to-be 45 §8, ADR 0227 rule 8, ADR 0236).
//
// **A bus upgrade is never rolled out.** The bus carries every declaration, every report and the
// controller's own lease; a new bus build that does not come up is a mesh nobody can tell anything,
// and a version whose data format moved (2.10 → 2.11) cannot be undone by putting the old one back.
// So nothing sends a new bus build on its own: its policy is `record` whatever anyone says (the
// catalogue's DerivedUpgrade, and `upgrade` refuses a roll-out), a plan builds it and sends nothing, a
// cascade holds the machine (ADR 0221), and a push naming that machine is refused while a new bus build
// waits for it (busHeld). The one way is this verb: a person, with why, the streams snapshotted first,
// whether it can be reverted said before it starts, the step said as `bus-maintenance` while it runs,
// and every stream, durable consumer and a round trip checked after (H-bus) — or the step said failed,
// with its snapshot as the way back.
// busStepProbe is the registry's row for the step; its kinds.
const (
busStepProbe = "DB"
kindBusMaintenance = "bus-maintenance"
kindBusUpgradeFailed = "bus-upgrade-failed"
)
// busStepBound is how long after its start a bus upgrade must be followed by a healthy bus.
var busStepBound = 15 * time.Minute
// takeBusSnapshot snapshots every stream before a bus upgrade and answers where the snapshot is: the bus
// machine's backup holder backs the bus module up now, whose dump is the streams' snapshot (novox/hq ADR
// 0235). A variable so a test takes none. A person who took one by hand says where with --snapshot-taken,
// and then none is taken — for a bus whose module does not yet carry the snapshot program.
var takeBusSnapshot = snapshotTheBusNow
// busPending is what a bus upgrade would do: the bus's module, the machines running it, and, per
// machine, the build it was last sent against the build the mesh holds. Empty machines: the mesh holds
// no bus module.
type busPending struct {
module string
machines []string
from map[string]string
to string
// same are the commits whose build was made from the same source as the build the mesh holds, or
// made the same artifacts and manifest (novox/hq issue 280).
same map[string]bool
}
// moves is whether sending the machine would replace its bus: a build it was not last sent, unless the
// two builds were made from the same source, or made the same artifacts from the same manifest — a
// rebuild of the same source for another module's merge changes nothing the machine runs, whatever
// image digest it made (novox/hq issue 280).
func (b busPending) moves(machine string) bool {
from, known := b.from[machine]
if b.module == "" || b.to == "" || (known && sameCommit(from, b.to)) {
return false
}
return !known || !b.same[from]
}
// pendingBus reads what a bus upgrade would do.
func pendingBus(ctx context.Context, inv *inventory.Inventory) (busPending, error) {
var b busPending
shelf, err := inv.Catalogue(ctx)
if err != nil {
return b, err
}
for name, m := range shelf {
if catalogue.ProvidesBus(m) {
b.module = name
}
}
if b.module == "" {
return b, nil
}
current, err := inv.CurrentBuilds(ctx)
if err != nil {
return b, err
}
b.to = current[b.module].Commit
if b.machines, err = inv.Running(ctx, b.module); err != nil {
return b, err
}
b.from, b.same = map[string]string{}, map[string]bool{}
made, err := inv.BuildFingerprints(ctx, b.module)
if err != nil {
return b, err
}
for commit, refs := range made {
b.same[commit] = refs != "" && refs == made[b.to]
}
sources, err := inv.BuildSourceFingerprints(ctx, b.module)
if err != nil {
return b, err
}
for commit, src := range sources {
if src != "" && src == sources[b.to] {
b.same[commit] = true
}
}
for _, n := range b.machines {
sent, known, err := inv.SentBuilds(ctx, n)
if err != nil {
return b, err
}
if known {
if c, carried := sent[b.module]; carried {
b.from[n] = c
}
}
}
return b, nil
}
// busHeld names the machines a push may not send because sending them would replace the bus: the
// planned step's, not a push's (ADR 0236). Said with the remedy.
func busHeld(ctx context.Context, inv *inventory.Inventory, machines []string) (map[string]string, error) {
b, err := pendingBus(ctx, inv)
if err != nil {
return nil, err
}
out := map[string]string{}
for _, n := range machines {
for _, holder := range b.machines {
if n == holder && b.moves(n) {
out[n] = fmt.Sprintf("sending %s would replace the bus (%s %s → %s), which is a planned step: "+
"`bus upgrade --why …` snapshots its streams first and checks them after (novox/hq ADR 0236)",
n, b.module, short(orNotKnown(b.from[n])), short(b.to))
}
}
}
return out, nil
}
// busCommand is `bus` — what a bus upgrade would do and how the last went — and `bus upgrade`.
func busCommand(ctx context.Context, args []string) error {
sub := ""
if len(args) > 0 && !strings.HasPrefix(args[0], "-") {
sub, args = args[0], args[1:]
}
set := flag.NewFlagSet("bus", flag.ContinueOnError)
snapshot := set.String("snapshot-taken", "", "where the streams' snapshot a person took is, while the mesh takes none itself")
reversible := set.Bool("reversible", false, "the new version can be undone by putting the old one back")
irreversible := set.Bool("irreversible", false, "the new version cannot be undone by putting the old one back, "+
"and this is the person's explicit word that it runs anyway")
why := addHandActFlags(set)
if rest, err := parseAround(set, args); err != nil {
return err
} else if len(rest) > 0 {
return errors.New("bus [upgrade --why … --reversible|--irreversible [--snapshot-taken <where>]]")
}
switch sub {
case "":
return busStatus(ctx)
case "upgrade":
default:
return fmt.Errorf("bus says what a bus upgrade would do, or `bus upgrade` — not %q", sub)
}
// Everything refused before anything is done.
if err := why.require("bus upgrade"); err != nil {
return err
}
if *reversible == *irreversible {
return errors.New("bus upgrade says, before it starts, whether the new version can be undone by putting the " +
"old one back: --reversible, or --irreversible as your explicit word that it runs anyway (to-be 45 §8). " +
"Nothing was done")
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
b, err := pendingBus(ctx, inv)
if err != nil {
return err
}
if b.module == "" {
return errors.New("the mesh holds no module that provides its bus: there is nothing to upgrade")
}
var moving []string
for _, n := range b.machines {
if b.moves(n) {
moving = append(moving, n)
}
}
if len(moving) == 0 {
fmt.Printf("every machine running %s runs the build the mesh holds (%s): nothing to upgrade\n", b.module, short(b.to))
return nil
}
where := strings.TrimSpace(*snapshot)
if where == "" {
for _, n := range moving {
fmt.Printf("snapshotting the bus's streams on %s first (its backup holder, ADR 0235)…\n", n)
if where, err = takeBusSnapshot(ctx, b.module, n); err != nil {
return fmt.Errorf("the streams could not be snapshotted, so the bus is not replaced: %w — a snapshot "+
"taken by hand is said with --snapshot-taken <where>", err)
}
}
}
from := map[string]bool{}
for _, n := range moving {
from[orNotKnown(b.from[n])] = true
}
step, err := inv.StartBusStep(ctx, inventory.BusStep{Module: b.module, Machines: moving,
From: strings.Join(sortedKeys(from), ", "), To: b.to, Snapshot: where, Reversible: *reversible,
By: link.Caller(), Why: strings.TrimSpace(*why.why)})
if err != nil {
return err
}
cause := "bus-upgrade"
if strings.TrimSpace(*why.cause) == "" {
why.cause = &cause
}
why.record(ctx, "bus upgrade", append([]string{b.module}, moving...))
fmt.Printf("bus upgrade %d: %s %s → %s on %s; streams snapshotted at %s; %s\n", step.ID, b.module, step.From,
short(b.to), strings.Join(moving, ", "), where, map[bool]string{true: "reversible: putting the old build back undoes it",
false: "NOT reversible: the snapshot is the only way back"}[*reversible])
sent, err := sendRollout(withBusStep(withScope(ctx, sendScope{person: true})), open, moving)
if err != nil {
_ = inv.EndBusStep(ctx, step.ID, "failed", "the send was refused: "+err.Error())
return fmt.Errorf("the bus's machine could not be sent its new build: %w — nothing was replaced", err)
}
fmt.Printf("sent %s; `bus-maintenance` is open until the bus answers healthy again — every stream, every durable "+
"consumer, a round trip to the machines (H-bus) — within %s, or the step is said failed with its snapshot "+
"as the way back. `bus` says how it went\n", strings.Join(sent, ", "), busStepBound)
return nil
}
// busStatus is `bus`: what an upgrade would do, and the last step.
func busStatus(ctx context.Context) error {
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
b, err := pendingBus(ctx, open.inventory)
if err != nil {
return err
}
if b.module == "" {
fmt.Println("the mesh holds no module that provides its bus")
} else {
fmt.Printf("the bus is %s, built %s, on %s; never rolled out — a planned step (`bus upgrade`)\n", b.module,
short(b.to), orNone(strings.Join(b.machines, ", ")))
for _, n := range b.machines {
state := "runs it"
if b.moves(n) {
state = "runs " + short(orNotKnown(b.from[n])) + ": `bus upgrade` replaces it"
}
fmt.Printf(" %-10s %s\n", n, state)
}
}
fmt.Println(" `bus upgrade` has the bus machine's backup holder snapshot the streams first (ADR 0235)")
s, found, err := open.inventory.LatestBusStep(ctx)
if err != nil || !found {
return err
}
state := "running since " + s.Started.Local().Format("2006-01-02 15:04")
if s.Ended != nil {
state = s.Outcome + " at " + s.Ended.Local().Format("2006-01-02 15:04")
}
fmt.Printf("last step %d: %s → %s on %s by %s (%s): %s; snapshot %s\n", s.ID, s.From, short(s.To),
strings.Join(s.Machines, ", "), orNone(s.By), s.Why, state, s.Snapshot)
if s.Found != "" {
fmt.Printf(" %s\n", s.Found)
}
return nil
}
// probeBusStep is DB: a bus upgrade running is said as `bus-maintenance`; the bus healthy again after
// the machines reported the new build ends it done; past its bound, unhealthy, it ends failed and is
// said — urgent, with its snapshot — while the bus is still not healthy.
func probeBusStep(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
inv := d.open.inventory
s, found, err := inv.LatestBusStep(ctx)
if err != nil || !found {
return nil, err
}
if s.Ended != nil && s.Outcome != "failed" {
return nil, nil
}
problems, err := busHealth(ctx, d)
if err != nil {
return nil, err
}
reports, err := inv.LastReports(ctx)
if err != nil {
return nil, err
}
applied := true
for _, r := range reports {
for _, n := range s.Machines {
if r.Node == n && (!r.Current || r.Outcome != inventory.OutcomeApplied || r.At == nil || r.At.Before(s.Started)) {
applied = false
problems = append(problems, n+" has not reported the new bus applied")
}
}
}
id := s.Module
if s.Ended == nil {
switch {
case applied && len(problems) == 0:
return nil, inv.EndBusStep(ctx, s.ID, "done", "the bus answered healthy after the upgrade")
case time.Since(s.Started) > busStepBound:
found := strings.Join(problems, "; ")
if err := inv.EndBusStep(ctx, s.ID, "failed", found); err != nil {
return nil, err
}
s.Found = found
default:
return []conditions.Observation{{Scope: conditions.ScopeBus, ID: id, Token: "maintenance",
Kind: kindBusMaintenance, Severity: conditions.Warning,
Summary: fmt.Sprintf("the bus is being upgraded (step %d, %s → %s on %s, by %s: %s); its snapshot is %s",
s.ID, s.From, short(s.To), strings.Join(s.Machines, ", "), orNone(s.By), s.Why, s.Snapshot),
Said: orNone(strings.Join(problems, "; "))}}, nil
}
}
if len(problems) == 0 {
return nil, nil // failed, and healthy since: nothing wrong now
}
way := "put the old build back"
if !s.Reversible {
way = "restore the snapshot"
}
return []conditions.Observation{{Scope: conditions.ScopeBus, ID: id, Token: "upgrade-failed",
Kind: kindBusUpgradeFailed, Severity: conditions.Urgent, Resolver: conditions.ResolverOperator,
Summary: fmt.Sprintf("the bus upgrade (step %d, %s → %s) did not end healthy within %s: %s — the way back is to %s (%s)",
s.ID, s.From, short(s.To), busStepBound, strings.Join(problems, "; "), way, s.Snapshot)}}, nil
}
func sortedKeys(set map[string]bool) []string {
out := make([]string, 0, len(set))
for k := range set {
out = append(out, k)
}
sort.Strings(out)
return out
}
// busSnapshotWithin is how long the bus machine's backup holder is given to take the bus's snapshot.
var busSnapshotWithin = 15 * time.Minute
// snapshotTheBusNow asks the bus machine's backup holder to back the bus module up now — its dump is
// the streams' snapshot (novox/hq ADR 0235) — and waits until it says a backup newer than the ask:
// where the snapshot is, as a person reads it. The serving controller's connection, or one of its own.
func snapshotTheBusNow(ctx context.Context, module, node string) (string, error) {
var where string
err := onTheBus(func(conn *nats.Conn) error {
asked := time.Now()
answer, err := link.AskSeatTool(ctx, conn, catalogue.BackupSeat, "now", node,
map[string]any{"module": module}, 30*time.Second)
if err != nil {
return fmt.Errorf("%s's backup holder was not asked to take the bus's snapshot: %w", node, err)
}
if answer.Error != "" {
return fmt.Errorf("%s's backup holder would not take the bus's snapshot: %s", node, answer.Error)
}
deadline := time.Now().Add(busSnapshotWithin)
for {
answer, err := link.AskSeatTool(ctx, conn, catalogue.BackupSeat, "backed-up", node, map[string]any{}, 10*time.Second)
if err == nil && answer.Error == "" {
if measured, err := readHolder(answer.Result); err == nil {
for item, m := range measured[module] {
if m.LastBackup != nil && m.LastBackup.After(asked) && m.Error == "" {
where = fmt.Sprintf("%s's restore point of %s (%s) taken %s", node, module, item,
m.LastBackup.UTC().Format(time.RFC3339))
return nil
}
}
}
}
if time.Now().After(deadline) {
return fmt.Errorf("%s's backup holder did not say the bus's snapshot was taken within %s; the bus is "+
"not replaced", node, busSnapshotWithin)
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(10 * time.Second):
}
}
})
return where, err
}
// errBusWaits is a send refused because it would replace the bus outside its planned step.
var errBusWaits = errors.New("a new bus build waits for its planned step")
type busStepKey struct{}
// withBusStep marks a send as the bus's planned step: the one send that may replace the bus.
func withBusStep(ctx context.Context) context.Context {
return context.WithValue(ctx, busStepKey{}, true)
}
func busStepSending(ctx context.Context) bool { on, _ := ctx.Value(busStepKey{}).(bool); return on }
@@ -1,92 +0,0 @@
package main
import (
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// **Every user's slow consumer is S9's, not the controller's alone** (novox/hq ADR 0357, issue 270). On
// 2026-10-07 the bus closed two clients on the laptop as slow consumers and nothing raised a condition:
// the controller heard only its own connection. Said by the bus module and read back, a user the bus
// closes is one condition naming the module and its machine, raised at the first look and cleared an hour
// after the last closure.
func TestAUserTheBusClosedAsASlowConsumerIsOneConditionNamingItsModuleAndMachine(t *testing.T) {
closed := time.Date(2026, 10, 14, 12, 0, 0, 0, time.UTC)
now := closed.Add(20 * time.Second)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
advisories := link.BusClosures([]link.BusClosure{
{User: "laptop.claude-code", CID: 10, Reason: "Slow Consumer (Write Deadline)", Opened: closed.Add(-time.Hour),
Closed: closed.Add(-10 * time.Minute)},
{User: "laptop.claude-code", CID: 44, Reason: "Slow Consumer (Write Deadline)", Opened: closed.Add(-time.Minute),
Closed: closed},
{User: "anchor.desk", CID: 7, Reason: "Maximum Message Payload Exceeded", Opened: closed.Add(-time.Hour),
Closed: closed},
}, []string{link.AdvisorySlowConsumer, link.AdvisorySlowConsumer, link.AdvisoryConnectionClosed}, closed.Add(-time.Hour))
look := func() {
t.Helper()
f := &signalFacts{now: now, host: "anchor", advisories: advisories}
if err := k.Reconcile(t.Context(), "S9", watchAdvisories(f)); err != nil {
t.Fatal(err)
}
}
look()
slow, found, err := conditions.ReadOne(t.Context(), store, "bus.laptop.claude-code.slow-consumer")
if err != nil || !found {
t.Fatalf("no slow-consumer condition for the user: %v %v", found, err)
}
if slow.Subject.Machine != "laptop" || slow.Observations != 2 || !slow.LastObserved.Equal(closed) {
t.Fatalf("the slow consumer is said on %q, %d closure(s), last %s: %+v", slow.Subject.Machine, slow.Observations,
slow.LastObserved, slow)
}
if !strings.Contains(slow.Summary, "claude-code on laptop") || strings.Contains(slow.Summary, "laptop.claude-code") {
t.Fatalf("the summary does not name the module in words, or carries the dotted user: %q", slow.Summary)
}
if !strings.Contains(slow.Headline, "Claude-code on laptop") {
t.Fatalf("the headline is %q", slow.Headline)
}
cut, found, err := conditions.ReadOne(t.Context(), store, "bus.anchor.desk.connection-closed")
if err != nil || !found || cut.Subject.Machine != "anchor" || !strings.Contains(cut.Summary, "maximum message payload exceeded") {
t.Fatalf("the user cut off for breaking the bus's rules: %v %v %+v", found, err, cut)
}
// An hour after the newest closure without another, both clear.
now = closed.Add(time.Hour + time.Minute)
look()
for _, key := range []string{"bus.laptop.claude-code.slow-consumer", "bus.anchor.desk.connection-closed"} {
if _, found, _ := k.Get(t.Context(), key); found {
t.Fatalf("%s is still open an hour after the last closure", key)
}
}
}
// **A read of the bus module's events that fails does not blind S9** (review of #326): the controller's own
// signals are still said, and the failed read is a condition of its own.
func TestABusModuleReadThatFailsKeepsTheControllersOwnSignals(t *testing.T) {
at := time.Date(2026, 10, 14, 12, 0, 0, 0, time.UTC)
f := &signalFacts{now: at.Add(time.Minute), host: "anchor",
advisories: []link.Advisory{{Kind: link.AdvisorySlowConsumer, ID: "controller", Said: "the controller fell behind",
First: at, Last: at, Count: 1}},
busFaultsErr: errors.New("nats: timeout")}
keys := map[string]bool{}
for _, o := range watchAdvisories(f) {
keys[o.Key()] = true
}
if !keys["bus.controller.slow-consumer"] || !keys["bus.bus-module.bus-faults-unread"] {
t.Fatalf("observed %v: the controller's own slow consumer and the unread events, both", keys)
}
for _, row := range signalsTable {
if row.Row == "S9" && row.needs(f) != nil {
t.Fatal("the S9 row went blind for a read of the bus module's events")
}
}
}
-195
View File
@@ -1,195 +0,0 @@
package main
import (
"fmt"
"sort"
"strings"
"sync"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// A send held for the bus's planned step (novox/hq issue 336).
//
// **A wait only a person can end is said to that person at once.** No send replaces the bus but its planned
// step (sendToEach, ADR 0236), and the step is a person's `bus upgrade`. So while a new bus build waits, every
// walk whose tier sends to the bus's machine is refused, and tries again each tick. The refusal was kept only
// as the walk's note; nothing was raised, and the operator found the hold by asking why a walk did not move.
// Row S17 raises it on the first tick after the first refusal, in the bus's scope, for the operator: what
// waits, behind which bus build, since when, and the verb. A walk held only by it is not late, so S3 leaves it
// out, as it leaves out a walk under a paused build seat. It clears once the bus's machine has been sent the
// build the mesh holds — the step was taken, and no send is refused for the bus any more — whatever the walks'
// notes still say. Whether the new bus came up healthy is the step's own condition (probe DB).
// kindBusStepWaiting is S17's kind: bus.<module>.step-waiting.
const kindBusStepWaiting = "bus-step-waiting"
// busFacts is a new bus build waiting for its planned step, and the sends held for it.
type busFacts struct {
module, to string
// from is the build each machine of the bus runs; machines those whose bus the step would replace.
from map[string]string
machines []string
waits []busWaitFacts
}
// busWaitFacts is one walk whose send was refused because it would replace the bus.
type busWaitFacts struct {
plan, repository, commit string
// modules are what its tier sends: what waits.
modules []string
// since is the first refusal: as this controller saw it, or, read back, the save that kept the refusal.
since time.Time
}
// busRefusedFirst is when this controller first saw each walk refused for the bus's step. The walk's note
// keeps the refusal across a restart; this keeps its moment more exactly than the walk's last save.
var busRefusedFirst = &firstSeen{at: map[string]time.Time{}}
type firstSeen struct {
mu sync.Mutex
at map[string]time.Time
}
// mark keeps the first moment an id was seen.
func (s *firstSeen) mark(id string, at time.Time) {
s.mu.Lock()
defer s.mu.Unlock()
if _, seen := s.at[id]; !seen {
s.at[id] = at
}
}
// of is when an id was first seen; zero when it was not.
func (s *firstSeen) of(id string) time.Time {
s.mu.Lock()
defer s.mu.Unlock()
return s.at[id]
}
// keepOnly forgets every id not given: a walk no longer held is not held since then.
func (s *firstSeen) keepOnly(ids map[string]bool) {
s.mu.Lock()
defer s.mu.Unlock()
for id := range s.at {
if !ids[id] {
delete(s.at, id)
}
}
}
// refusedForTheBus is whether an error is the refusal of a send for the bus's planned step.
func refusedForTheBus(err error) bool {
return err != nil && strings.Contains(err.Error(), errBusWaits.Error())
}
// busWaitsOf is the sends held for the bus's step, from the open walks: each walk not waiting for its
// delivery's word whose note keeps a refusal for this very bus build, while the build would still replace
// the bus on a machine. first is when this controller first saw a walk refused, zero when it did not.
func busWaitsOf(plans []inventory.Plan, b busPending, first func(string) time.Time) busFacts {
f := busFacts{module: b.module, to: b.to, from: b.from}
for _, n := range b.machines {
if b.moves(n) {
f.machines = append(f.machines, n)
}
}
if len(f.machines) == 0 {
return f
}
for _, p := range plans {
if !p.Open() || p.Waiting() || !strings.Contains(p.Note, errBusWaits.Error()) || !strings.Contains(p.Note, short(b.to)) {
continue
}
since := p.Updated
if seen := first(p.ID); !seen.IsZero() && (since.IsZero() || seen.Before(since)) {
since = seen
}
var modules []string
if p.Tier >= 0 && p.Tier < len(p.Tiers) {
modules = append(modules, p.Tiers[p.Tier]...)
} else {
modules = planModules(p)
}
sort.Strings(modules)
f.waits = append(f.waits, busWaitFacts{plan: p.ID, repository: p.Repository, commit: p.Commit, modules: modules,
since: since})
}
sort.Slice(f.waits, func(i, j int) bool { return f.waits[i].since.Before(f.waits[j].since) })
return f
}
// heldByTheBus is the walks of a bus's facts, by id: what S3 leaves out.
func (b busFacts) heldByTheBus() map[string]bool {
out := map[string]bool{}
for _, w := range b.waits {
out[w.plan] = true
}
return out
}
// busUpgradeVerb is the call that ends the wait, as the summary names it: through the mesh MCP server, with
// why, saying whether the new version can be undone ("reversible": "true") or not ("irreversible": "true").
const busUpgradeVerb = "`mesh_call mesh-controller.bus {\"upgrade\": \"true\", \"why\": \"…\", \"reversible\" or \"irreversible\": \"true\"}`"
// watchBusWaits is S17: a send held for the bus's planned step, said at once to the operator.
func watchBusWaits(f *signalFacts) []conditions.Observation {
b := f.bus
if len(b.waits) == 0 || len(b.machines) == 0 {
return nil
}
since := b.waits[0].since
var walks, what []string
for _, w := range b.waits {
walks = append(walks, fmt.Sprintf("%s (%s %s, tier: %s)", w.plan, w.repository, short(w.commit),
strings.Join(w.modules, ", ")))
what = append(what, deliveryWhat(w.modules, w.repository))
}
var from []string
for _, n := range b.machines {
from = append(from, short(orNotKnown(b.from[n])))
}
machines := namesWords(b.machines, 3)
in := f.now.Sub(since)
return []conditions.Observation{{Scope: conditions.ScopeBus, ID: b.module, Token: "step-waiting",
Kind: kindBusStepWaiting, Severity: conditions.Warning, Resolver: conditions.ResolverOperator,
Machine: b.machines[0], Also: b.machines[1:],
Summary: fmt.Sprintf("sends to %s wait for the bus's planned step: a new bus build (%s %s → %s) would replace "+
"the bus there, which only a person's %s does; waiting since %s: %s", strings.Join(b.machines, ", "),
b.module, strings.Join(sortedUnique(from), ", "), short(b.to), busUpgradeVerb,
since.UTC().Format(time.RFC3339), strings.Join(walks, "; ")),
Said: fmt.Sprintf("%d walk(s) refused since %s", len(b.waits), since.UTC().Format(time.RFC3339)),
Headline: clipWords("Sends to "+machines+" wait for a bus upgrade", conditions.HeadlineMax),
Explanation: clipWords(fmt.Sprintf("A new version of the mesh's message system is built, and only a person "+
"installs it. Until then nothing else is sent to %s: %s waits, for %s so far.", machines,
namesWords(sortedUnique(what), 3), humanDuration(in)), conditions.ExplanationMax),
Needs: "start the bus upgrade " + FromMeshMCPServer,
Resolved: "Resolved: the bus upgrade started, and sends go on"}}
}
// sortedUnique is a list sorted, each once.
func sortedUnique(xs []string) []string {
seen := map[string]bool{}
var out []string
for _, x := range xs {
if !seen[x] {
seen[x] = true
out = append(out, x)
}
}
sort.Strings(out)
return out
}
// clipWords keeps a plain sentence within a bound, at a word.
func clipWords(s string, n int) string {
if len(s) <= n {
return s
}
cut := strings.LastIndex(s[:n-1], " ")
if cut <= 0 {
cut = n - 1
}
return s[:cut] + "…"
}
@@ -1,70 +0,0 @@
//go:build integration
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
func TestASendHeldForTheBusStepIsFoundFromTheWalksItHolds(t *testing.T) {
now := time.Date(2026, 10, 8, 18, 30, 0, 0, time.UTC)
b := aBusOnAnchor()
walk := func(id, repo, note string, updated time.Time) inventory.Plan {
return inventory.Plan{ID: id, Repository: repo, Commit: "c0ffee001122", State: inventory.PlanRolling, Tier: 0,
Tiers: [][]string{{"mesh-host"}}, Modules: map[string]*inventory.PlanModule{"mesh-host": {}}, Note: note,
Updated: updated}
}
for _, c := range []struct {
name string
bus busPending
plans []inventory.Plan
first map[string]time.Time
want []string
since time.Time
}{
{"a walk refused for the bus", b,
[]inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now.Add(-27*time.Minute))}, nil,
[]string{"plan-1"}, now.Add(-27 * time.Minute)},
{"refused earlier than its last save, as this controller saw it", b,
[]inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now.Add(-5*time.Minute))},
map[string]time.Time{"plan-1": now.Add(-28 * time.Minute)}, []string{"plan-1"}, now.Add(-28 * time.Minute)},
{"a walk refused for another reason", b,
[]inventory.Plan{walk("plan-1", "novox/mesh-host", "tier 0: the build seat is paused — tried again", now)}, nil,
nil, time.Time{}},
{"refused for an older bus build than the one held now", func() busPending { o := b; o.to = "99999999cccc"; return o }(),
[]inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now)}, nil, nil, time.Time{}},
{"the bus already runs the build held: its step started", func() busPending {
o := aBusOnAnchor()
o.from = map[string]string{"anchor": o.to}
return o
}(), []inventory.Plan{walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now)}, nil, nil, time.Time{}},
{"a walk waiting for its delivery's word asks no send", b, func() []inventory.Plan {
p := walk("plan-1", "novox/mesh-host", refusedNote(b, 0), now)
p.Delivery = &inventory.PlanDelivery{Awaits: "mesh-delivery"}
return []inventory.Plan{p}
}(), nil, nil, time.Time{}},
} {
t.Run(c.name, func(t *testing.T) {
got := busWaitsOf(c.plans, c.bus, func(id string) time.Time { return c.first[id] })
var ids []string
for _, w := range got.waits {
ids = append(ids, w.plan)
}
if strings.Join(ids, ",") != strings.Join(c.want, ",") {
t.Fatalf("held for the bus: %v, want %v", ids, c.want)
}
if len(c.want) > 0 {
if !got.waits[0].since.Equal(c.since) {
t.Errorf("waiting since %s, want %s", got.waits[0].since, c.since)
}
if strings.Join(got.machines, ",") != "anchor" || got.to != b.to || got.module != "nats" {
t.Errorf("behind %+v", got)
}
}
})
}
}
-99
View File
@@ -1,99 +0,0 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
)
// novox/hq issue 336: a send held because it would replace the bus outside its planned step waits for a
// person, so a person is told at once — what waits, behind which bus build, since when, and the verb that
// ends it — and the wait is not read as a walk running late.
// aBusOnAnchor is a new bus build waiting for its step on anchor: nats 88135ad0 there, 32307bd1 held.
func aBusOnAnchor() busPending {
return busPending{module: "nats", machines: []string{"anchor"}, from: map[string]string{"anchor": "88135ad0aaaa"},
to: "32307bd1bbbb", same: map[string]bool{}}
}
// refusedNote is the note a walk keeps when its send was refused for the bus, as advanceHeld writes it.
func refusedNote(b busPending, tier int) string {
held := "sending anchor would replace the bus (nats " + short(b.from["anchor"]) + " → " + short(b.to) +
"), which is a planned step: `bus upgrade --why …` snapshots its streams first and checks them after (novox/hq ADR 0236)"
return "tier " + string(rune('0'+tier)) + ": " + errBusWaits.Error() + ": " + held + " — tried again"
}
// **Said at once, not as lateness, and cleared when the step starts**: the walk held only for the bus raises
// the bus's condition on the first tick, before any tier bound, naming what waits, the bus build from and to,
// since when and `bus upgrade`, for the operator; S3 says nothing of it even past its bound; and the
// condition clears once the bus's machine runs the new build.
func TestASendHeldForTheBusStepIsSaidAtOnceAndNotAsLateness(t *testing.T) {
now := time.Date(2026, 10, 8, 18, 30, 0, 0, time.UTC)
b := aBusOnAnchor()
held := func(entered time.Duration) *signalFacts {
f := calm(now)
f.plans = []planFacts{{id: "plan-1", repository: "novox/mesh-host", commit: "c0ffee00", tier: 0, tiers: 1,
entered: now.Add(-entered), bound: 30 * time.Minute, waiting: refusedNote(b, 0), bus: true}}
f.bus = busFacts{module: "nats", to: b.to, from: b.from, machines: []string{"anchor"},
waits: []busWaitFacts{{plan: "plan-1", repository: "novox/mesh-host", commit: "c0ffee001122",
modules: []string{"mesh-host"}, since: now.Add(-time.Minute)}}}
return f
}
f := held(time.Minute)
got := watchBusWaits(f)
if len(got) != 1 {
t.Fatalf("a send held for the bus a minute ago raised %+v", got)
}
o := got[0]
if o.Key() != "bus.nats.step-waiting" || o.Kind != kindBusStepWaiting || o.Resolver != conditions.ResolverOperator {
t.Fatalf("raised %s (%s), resolver %q", o.Key(), o.Kind, o.Resolver)
}
for _, want := range []string{"anchor", "mesh-host", "88135ad0", "32307bd1", "mesh_call mesh-controller.bus",
`"upgrade": "true"`, `"reversible"`, `"irreversible"`, "2026-10-08T18:29:00Z", "plan-1"} {
if !strings.Contains(o.Summary, want) {
t.Errorf("its summary does not say %q: %s", want, o.Summary)
}
}
if !strings.Contains(o.Explanation, "mesh-host") || !strings.Contains(o.Headline, "bus upgrade") || o.Needs == "" {
t.Errorf("its words do not say what waits and what the operator does: %+v", o)
}
if why, ok := conditions.PlainWords(conditions.Words{Headline: o.Headline, Explanation: o.Explanation,
Resolved: o.Resolved, Needs: o.Needs}, "anchor"); !ok {
t.Errorf("its words are not plain: %s", why)
}
// Past S3's bound: still the bus's wait, never a stalled walk.
late := held(45 * time.Minute)
if s3 := watchPlans(late); len(s3) != 0 {
t.Fatalf("a walk held only by the bus step was said stalled: %+v", s3)
}
// A walk held for something else past its bound is still stalled.
other := held(45 * time.Minute)
other.plans[0].bus = false
if s3 := watchPlans(other); len(s3) != 1 {
t.Fatalf("a walk held for something else past its bound raised %+v", s3)
}
// Through the keeper: open at the first tick, cleared when the step started.
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store, Teller: &conditions.Told{},
Now: func() time.Time { return now }})
defer k.Close(context.Background())
w := &watchdogs{keeper: k, started: now.Add(-time.Hour)}
w.see(t.Context(), held(time.Minute))
open, err := k.Open(t.Context())
if err != nil || len(open) != 1 || open[0].Key != "bus.nats.step-waiting" {
t.Fatalf("after the first tick, open: %+v (%v)", open, err)
}
started := held(time.Minute)
started.bus = busFacts{module: "nats", to: b.to}
started.plans[0].bus = false
w.see(t.Context(), started)
if open, _ := k.Open(t.Context()); len(open) != 0 {
t.Fatalf("the step started, and open: %+v", open)
}
}
-258
View File
@@ -1,258 +0,0 @@
package main
import (
"context"
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"errors"
"fmt"
"math/big"
"net"
"os"
"path/filepath"
"strings"
"time"
"github.com/novox/mesh-controller/internal/broker"
)
// The bus's own certificate, made by the mesh rather than borrowed from an image.
//
// **The foundation asked a third-party image for a tool it never said must be there** (novox/hq
// 04-ISSUES/146). The bootstrap made this certificate by running `openssl` inside the broker's
// image, which worked while the broker was one that happened to carry it and stopped the day the
// bus changed: the new one has a shell and no openssl, so the step exited 127 and no mesh could be
// raised. Substituting another image the bundle names does not help — none of them carry it
// either.
//
// So the program that needs a certificate makes one. It is the mesh's own binary, already on the
// machine at this point in the bootstrap (the schema step ran it), and it needs nothing from the
// image it writes into but a mounted directory.
//
// **Self-signed, and that is the design** — a host pins this server's exact certificate and
// authenticates with a password (novox/hq ADR 0004). There is no authority above it to ask, and at
// this moment in a bootstrap there is no mesh to ask one of.
//
// Idempotent, because the step is applied again on every reconcile and a second certificate would
// be one the hosts that pinned the first no longer believe.
// busCertificateNames is what the bus is reached by: the container name on a mesh network, and the
// loopback address the machine's own foundation dials.
var busCertificateNames = []string{"mesh-broker"}
const busCertificateLife = 10 * 365 * 24 * time.Hour
// busCertificate makes the bus's certificate in a directory, or says whether one is there.
//
// broker certificate --into /tls make it if it is not there
// broker certificate --check --into /tls exit non-zero unless a usable pair is
func busCertificate(args []string) error {
into, check := "", false
for i := 0; i < len(args); i++ {
switch args[i] {
case "--check":
check = true
case "--into":
if i+1 >= len(args) {
return errors.New("--into needs a directory")
}
into = args[i+1]
i++
default:
return fmt.Errorf("broker certificate [--check] --into <directory>: %q", args[i])
}
}
if into == "" {
return errors.New("broker certificate [--check] --into <directory>")
}
crt, key := filepath.Join(into, "tls.crt"), filepath.Join(into, "tls.key")
if usable, err := busCertificateUsable(crt, key); err != nil {
return err
} else if usable {
fmt.Printf("the bus already has a certificate at %s, and it was left alone\n", crt)
return nil
}
if check {
// Said as a failure, because that is what the caller asked: a bootstrap's verify runs
// this and a false answer is what makes the step run.
return fmt.Errorf("no usable certificate and key at %s", into)
}
return writeBusCertificate(crt, key)
}
// busCertificateUsable says whether a certificate and its key are both there and parse.
//
// Both, and parsed rather than stat'ed: a half-written pair is the state a bootstrap interrupted
// between the two files leaves behind, and a step that treated it as done would hand the server a
// certificate with no key and report success.
func busCertificateUsable(crt, key string) (bool, error) {
certPEM, err := os.ReadFile(crt)
if errors.Is(err, os.ErrNotExist) {
return false, nil
}
if err != nil {
return false, err
}
keyPEM, err := os.ReadFile(key)
if errors.Is(err, os.ErrNotExist) {
return false, nil
}
if err != nil {
return false, err
}
if _, err := tlsPairParses(certPEM, keyPEM); err != nil {
return false, nil
}
return true, nil
}
func tlsPairParses(certPEM, keyPEM []byte) (*x509.Certificate, error) {
block, _ := pem.Decode(certPEM)
if block == nil || block.Type != "CERTIFICATE" {
return nil, errors.New("not a certificate")
}
certificate, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return nil, err
}
keyBlock, _ := pem.Decode(keyPEM)
if keyBlock == nil {
return nil, errors.New("not a key")
}
if _, err := x509.ParsePKCS8PrivateKey(keyBlock.Bytes); err != nil {
if _, err := x509.ParsePKCS1PrivateKey(keyBlock.Bytes); err != nil {
return nil, err
}
}
return certificate, nil
}
func writeBusCertificate(crt, key string) error {
private, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return err
}
serial, err := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 128))
if err != nil {
return err
}
template := &x509.Certificate{
SerialNumber: serial,
Subject: pkix.Name{CommonName: busCertificateNames[0]},
DNSNames: busCertificateNames,
IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(busCertificateLife),
KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageKeyEncipherment,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
BasicConstraintsValid: true,
}
der, err := x509.CreateCertificate(rand.Reader, template, template, &private.PublicKey, private)
if err != nil {
return err
}
pkcs8, err := x509.MarshalPKCS8PrivateKey(private)
if err != nil {
return err
}
// **The key first, and only then the certificate**, so the pair a reader finds is never a
// certificate whose key has not been written yet — the one order in which an interruption
// leaves something that looks finished (novox/hq 04-ISSUES/014, a key present and unusable).
if err := os.WriteFile(key, pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: pkcs8}), 0o600); err != nil {
return err
}
if err := os.WriteFile(crt, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}), 0o644); err != nil {
return err
}
fmt.Printf("made the bus a certificate for %v, valid until %s\n %s\n %s\n",
busCertificateNames, template.NotAfter.Format(time.RFC3339), crt, key)
return nil
}
// busAccounts writes the mesh's composed user list to a file.
//
// **For genesis, where no declaration can deliver it** (novox/hq 04-ISSUES/146). Everywhere else
// the list reaches the machine running the bus as a resource of the module that holds it — which
// requires that machine to be an enrolled node, and at genesis it is not: the first node cannot
// enrol because the account it would enrol with cannot be composed onto a bus it has no declaration
// for. The installer breaks that circle by placing the file itself, once, and the module takes the
// file over from its first push.
//
// The same composition, not a second one: this asks the store for the same records and renders them
// with the same composer the declaration uses. A genesis that hand-wrote an account would be a
// second statement of who may say what, able to disagree with the first.
//
// **It writes to standard output unless told a file**, and that is the point: the control plane
// composes and says what it composed, and whoever is raising the machine puts it where that
// machine's bus reads it. A control plane that wrote into the bus's own directory would have to
// know where that is and how to make the server re-read it — which is the module's knowledge, and
// the module is what takes this over on the first push.
//
// broker accounts > /var/lib/mesh-bus-conf/accounts.conf
func busAccounts(ctx context.Context, args []string) error {
into := ""
for i := 0; i < len(args); i++ {
switch args[i] {
case "--into":
if i+1 >= len(args) {
return errors.New("--into needs a file")
}
into = args[i+1]
i++
default:
return fmt.Errorf("broker accounts --into <file>: %q", args[i])
}
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
records, err := open.inventory.BusRecords(ctx)
if err != nil {
return err
}
users, err := broker.Users(records)
if err != nil {
return err
}
kept, err := open.inventory.BusUsers(ctx)
if err != nil {
return err
}
hashes := make(map[string]string, len(kept))
for name, u := range kept {
hashes[name] = u.PasswordHash
}
filled, missing := broker.WithPasswords(users, hashes)
if len(missing) > 0 {
// To standard error, always: the composed file may be going to standard output, and a
// remark in the middle of it is a configuration the server refuses to parse.
fmt.Fprintf(os.Stderr, "leaving out %d user(s) the mesh has minted no credential for: %s\n",
len(missing), strings.Join(missing, ", "))
}
if len(filled) == 0 {
return errors.New("not one user has a credential, so this list would refuse every " +
"connection in the mesh")
}
accounts, err := broker.ComposeAccounts(filled)
if err != nil {
return err
}
if into == "" {
fmt.Print(accounts)
return nil
}
if err := os.WriteFile(into, []byte(accounts), 0o600); err != nil {
return err
}
fmt.Printf("wrote %d user(s) to %s\n", len(filled), into)
return nil
}
-121
View File
@@ -1,121 +0,0 @@
package main
import (
"crypto/tls"
"crypto/x509"
"os"
"path/filepath"
"strings"
"testing"
)
// novox/hq 04-ISSUES/146. The bootstrap could not make the bus a certificate: it asked an image for
// `openssl` and the image it asks has none. What replaces it is this command, so what is checked is
// what the bootstrap needs from it — a pair a TLS server can actually load, made once and only once.
func TestTheBusCertificateLoadsAsAServersWould(t *testing.T) {
into := t.TempDir()
if err := busCertificate([]string{"--into", into}); err != nil {
t.Fatalf("the bus could not be given a certificate: %v", err)
}
// The check a cheaper test would not make. The key was present and valid and the server could
// not start, once, because nothing loaded the pair the way a server loads it
// (novox/hq 04-ISSUES/014).
pair, err := tls.LoadX509KeyPair(filepath.Join(into, "tls.crt"), filepath.Join(into, "tls.key"))
if err != nil {
t.Fatalf("a TLS server cannot load what was written: %v", err)
}
leaf := pair.Leaf
if leaf == nil {
if leaf, err = x509.ParseCertificate(pair.Certificate[0]); err != nil {
t.Fatal(err)
}
}
if err := leaf.VerifyHostname("mesh-broker"); err != nil {
t.Errorf("the certificate is not for the name the bus is reached by: %v", err)
}
if len(leaf.IPAddresses) == 0 || leaf.IPAddresses[0].String() != "127.0.0.1" {
t.Errorf("the certificate does not cover the loopback address the foundation dials: %v", leaf.IPAddresses)
}
// The key is not readable by anything else on the machine; the certificate is public and is.
key, err := os.Stat(filepath.Join(into, "tls.key"))
if err != nil {
t.Fatal(err)
}
if key.Mode().Perm() != 0o600 {
t.Errorf("the key is %v", key.Mode().Perm())
}
crt, err := os.Stat(filepath.Join(into, "tls.crt"))
if err != nil {
t.Fatal(err)
}
if crt.Mode().Perm() != 0o644 {
t.Errorf("the certificate is %v, which the server runs as another user cannot read", crt.Mode().Perm())
}
}
// **Made once.** The step is applied again on every reconcile, and a second certificate is one the
// hosts that pinned the first no longer believe (novox/hq ADR 0004).
func TestTheBusCertificateIsMadeOnce(t *testing.T) {
into := t.TempDir()
if err := busCertificate([]string{"--into", into}); err != nil {
t.Fatal(err)
}
first, err := os.ReadFile(filepath.Join(into, "tls.crt"))
if err != nil {
t.Fatal(err)
}
if err := busCertificate([]string{"--into", into}); err != nil {
t.Fatal(err)
}
again, err := os.ReadFile(filepath.Join(into, "tls.crt"))
if err != nil {
t.Fatal(err)
}
if string(first) != string(again) {
t.Fatal("running it twice replaced the certificate every host had pinned")
}
}
// The verify half: false before, true after, which is what makes the bootstrap run the step at all.
func TestTheCheckIsFalseUntilThereIsAPair(t *testing.T) {
into := t.TempDir()
if err := busCertificate([]string{"--check", "--into", into}); err == nil {
t.Fatal("an empty directory reported a usable certificate")
}
if err := busCertificate([]string{"--into", into}); err != nil {
t.Fatal(err)
}
if err := busCertificate([]string{"--check", "--into", into}); err != nil {
t.Fatalf("the certificate it just made does not satisfy its own check: %v", err)
}
}
// A half-written pair is not a pair. An interrupted bootstrap leaves exactly this, and a step that
// called it done would hand the server a certificate with no key and report success.
func TestACertificateWithoutItsKeyIsNotUsable(t *testing.T) {
into := t.TempDir()
if err := busCertificate([]string{"--into", into}); err != nil {
t.Fatal(err)
}
if err := os.Remove(filepath.Join(into, "tls.key")); err != nil {
t.Fatal(err)
}
if err := busCertificate([]string{"--check", "--into", into}); err == nil {
t.Fatal("a certificate with no key passed the check")
}
if err := busCertificate([]string{"--into", into}); err != nil {
t.Fatal(err)
}
if _, err := tls.LoadX509KeyPair(filepath.Join(into, "tls.crt"), filepath.Join(into, "tls.key")); err != nil {
t.Fatalf("it did not replace the unusable pair: %v", err)
}
}
func TestWhereToWriteIsRequired(t *testing.T) {
if err := busCertificate(nil); err == nil || !strings.Contains(err.Error(), "--into") {
t.Fatalf("it did not ask where to write: %v", err)
}
}
-247
View File
@@ -1,247 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// The streams and durable consumers the mesh's own traffic needs, derived once (novox/hq to-be 45 §4,
// D6 and D7).
//
// **What the controller asserts at its start and what the self-check expects to find are one
// derivation**, run against the bus to make them and against a recorder to list them. Two lists would
// drift, and a self-check comparing the bus with a second opinion of what should be there would find
// the drift rather than the fault.
// assertBusObjects brings every stream and consumer into being on r, and answers the machines that
// can now hear a declaration.
func assertBusObjects(ctx context.Context, inv *inventory.Inventory, r broker.Raiser) ([]string, error) {
nodes, err := inv.Nodes(ctx)
if err != nil {
return nil, err
}
names := make([]string, 0, len(nodes))
for _, n := range nodes {
names = append(names, n.Name)
}
if err := broker.Raise(r, names); err != nil {
return nil, err
}
// The work queues of the mesh's own roles (novox/hq ADR 0121). The queue before the holder,
// deliberately: work queues until somebody arrives to do it, so assigning a build machine a week
// after something started asking for builds flushes the backlog instead of having lost it.
// With the seats' holders, so each role's work queue gets the consumer its holder takes
// work from. Passed as nil until the first live raise, which left the build machine bound to a
// consumer nothing had created (2026-09-28).
holders, err := seatHolders(ctx, inv)
if err != nil {
return nil, err
}
if err := broker.RaiseSeats(r, inventory.MeshSeats(), holders); err != nil {
return nil, err
}
// And how every module hears what it consumes. Derived from the same records the user list is
// composed from, so a module the mesh grants a consumer's subjects has that consumer waiting.
// Done on every raise, not only when a credential is issued: every module moved onto this bus
// by the rollout was issued on the old one, and came up with nothing to bind to (2026-09-28).
consumers, records, err := moduleConsumersAndRecords(ctx, inv)
if err != nil {
return nil, err
}
// And a grant of a restricted stream that composes nothing, said on every raise and send (novox/hq ADR 0376 §2).
sayWithheld(records, " ")
// And the work queues of seats that name their caller or their kind, with each holder's worker
// (novox/hq ADR 0259 §3): an ask queues until the router takes it, a channel's work until that kind
// takes it.
trafficStreams, trafficWorkers, err := seatTrafficObjects(ctx, inv)
if err != nil {
return nil, err
}
// And every module's own stream (novox/hq ADR 0363), from the catalogue, before the consumers on it.
modStreams, err := inv.DeclaredStreams(ctx)
if err != nil {
return nil, err
}
// Every one tried, and every failure named: one module's consumer the bus refuses is no reason
// the modules after it in the list hear nothing (novox/hq issue 208, where this runs on each send).
var failed []error
for _, s := range broker.ModuleStreams(modStreams) {
if err := r.EnsureStream(s); err != nil {
failed = append(failed, fmt.Errorf("the module stream %s: %w", s.Name, err))
}
}
for _, s := range trafficStreams {
if err := r.EnsureStream(s); err != nil {
failed = append(failed, fmt.Errorf("the work queue %s: %w", s.Name, err))
}
}
for _, c := range trafficWorkers {
if err := r.EnsureConsumer(c); err != nil {
failed = append(failed, fmt.Errorf("the worker %s on %s: %w", c.Name, c.Stream, err))
}
}
restricted := map[string]bool{}
for _, s := range modStreams {
restricted[s.Name] = s.Restricted
}
for _, c := range consumers {
if err := broker.EnsureHeard(r, c.Consumer, restricted[c.Consumer.Stream]); err != nil {
failed = append(failed, fmt.Errorf("how %s on %s hears what it consumes: %w", c.Module, c.Node, err))
}
}
// **And a consumer no longer granted loses what it reads through** (novox/hq ADR 0363 §6): every consumer
// on a module's stream the composition no longer asks for is removed, on this very send.
if p, prunes := r.(broker.Pruner); prunes && len(modStreams) > 0 {
var wanted []broker.Consumer
for _, c := range consumers {
wanted = append(wanted, c.Consumer)
}
removed, err := broker.PruneHeard(p, modStreams, wanted)
if err != nil {
failed = append(failed, err)
}
for _, gone := range removed {
fmt.Printf(" removed the consumer %s: its module is no longer granted that stream (novox/hq ADR 0363)\n", gone)
}
}
if len(failed) > 0 {
return nil, errors.Join(failed...)
}
return names, nil
}
// What raises the condition a send says when the objects it implies could not be asserted, and its kind.
const (
sourceBusObjects = "bus-objects"
kindBusObjectsUnasserted = "bus-objects-unasserted"
)
// assertOnSend asserts, on the bus a send is about to use, every object assertBusObjects derives —
// **whenever a declaration is sent, not only when the controller starts** (novox/hq issue 208).
//
// A module assigned after the controller started was sent its declaration and found no consumer to
// bind (`consumer not found`, messenger on 2026-10-06), and a seat holder assigned after it found no
// worker: the objects a declaration implies were asserted at start and nowhere else, so they existed
// only for what was assigned before the last restart. The same derivation, not a second list of what
// a send needs: what start asserts, the self-check expects and a send asserts are one answer. Every
// part is idempotent, so asserting the whole of it again is the no-op a restart already relies on.
//
// **A failure is said and raised, and the send goes on.** The objects are the mesh's, not the
// machines' being sent: holding every machine back for one consumer that none of them may use would
// turn one fault into all of them, and the declarations are not what is wrong. It is never silent —
// said in the send's own output and raised as a condition, which the next send that asserts them
// clears — and the start-time raise still refuses to serve without them.
func assertOnSend(ctx context.Context, inv *inventory.Inventory, r broker.Raiser, indent string) error {
_, err := assertBusObjects(ctx, inv, r)
var observed []conditions.Observation
if err != nil {
fmt.Printf("%sTHE BUS DOES NOT HOLD WHAT THIS SEND IMPLIES: %v\n", indent, err)
fmt.Printf("%s a module may find no consumer to bind, or a holder no worker; sent anyway, raised as "+
"condition %s, and asserted again by the next send\n", indent, unassertedObservation(err).Key())
observed = append(observed, unassertedObservation(err))
}
// Observed when it failed, cleared when it did not: a send that asserted everything is the
// observation that the bus holds what it should.
if kerr := withKeeper(ctx, func(k *conditions.Keeper) error {
return k.Reconcile(ctx, sourceBusObjects, observed)
}); kerr != nil {
fmt.Printf("%sand whether the bus holds what this send implies could not be kept as a condition: %v\n",
indent, kerr)
}
return err
}
// unassertedObservation is a send's failure to assert the bus's objects, as a condition.
func unassertedObservation(err error) conditions.Observation {
return conditions.Observation{Scope: conditions.ScopeBus, ID: "objects", Token: "unasserted",
Kind: kindBusObjectsUnasserted, Severity: conditions.Warning, Source: sourceBusObjects,
Summary: "the bus's streams and consumers could not be asserted when a declaration was sent: " +
"a module may find no consumer to bind, or a seat's holder no worker",
Said: err.Error()}
}
// moduleConsumers is every module's durable consumer, from the records the user list is composed from.
func moduleConsumers(ctx context.Context, inv *inventory.Inventory) ([]broker.ModuleConsumer, error) {
consumers, _, err := moduleConsumersAndRecords(ctx, inv)
return consumers, err
}
// moduleConsumersAndRecords is moduleConsumers with the records they were composed from.
func moduleConsumersAndRecords(ctx context.Context, inv *inventory.Inventory) ([]broker.ModuleConsumer, broker.Records, error) {
records, err := inv.BusRecords(ctx)
if err != nil {
return nil, broker.Records{}, err
}
users, err := broker.Users(records)
if err != nil {
return nil, broker.Records{}, err
}
return broker.ConsumersOf(users), records, nil
}
// seatTrafficObjects is the work queues and workers of seats that name their caller or their kind, from
// the records the user list is composed from.
func seatTrafficObjects(ctx context.Context, inv *inventory.Inventory) ([]broker.Stream, []broker.Consumer, error) {
records, err := inv.BusRecords(ctx)
if err != nil {
return nil, nil, err
}
users, err := broker.Users(records)
if err != nil {
return nil, nil, err
}
streams, workers := broker.SeatTrafficObjects(users)
// And the queue of every such seat the catalogue declares, held or not: work queues from registration,
// so what is submitted before a holder is assigned waits for it (the correctness review of 2026-10-08).
declared, err := inv.DeclaredTrafficSeats(ctx)
if err != nil {
return nil, nil, err
}
have := map[string]bool{}
for _, s := range streams {
have[s.Name] = true
}
for _, s := range broker.TrafficQueues(declared) {
if !have[s.Name] {
streams = append(streams, s)
have[s.Name] = true
}
}
return streams, workers, nil
}
// moduleConsumerCount is how many modules hear what they consume, for the raise's one line.
func moduleConsumerCount(ctx context.Context, inv *inventory.Inventory) (int, error) {
consumers, err := moduleConsumers(ctx, inv)
return len(consumers), err
}
// expectedBusObjects is every stream and consumer assertBusObjects would make, made nowhere.
func expectedBusObjects(ctx context.Context, inv *inventory.Inventory) ([]broker.Stream, []broker.Consumer, error) {
var rec recordingRaiser
if _, err := assertBusObjects(ctx, inv, &rec); err != nil {
return nil, nil, fmt.Errorf("what the bus should hold cannot be worked out: %w", err)
}
return rec.streams, rec.consumers, nil
}
// recordingRaiser keeps what it was asked to assert and asserts nothing.
type recordingRaiser struct {
streams []broker.Stream
consumers []broker.Consumer
}
func (r *recordingRaiser) EnsureStream(s broker.Stream) error {
r.streams = append(r.streams, s)
return nil
}
func (r *recordingRaiser) EnsureConsumer(c broker.Consumer) error {
r.consumers = append(r.consumers, c)
return nil
}
-199
View File
@@ -1,199 +0,0 @@
//go:build integration
package main
import (
"errors"
"strings"
"testing"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// novox/hq issue 208: the bus's objects a declaration implies are asserted whenever one is sent, not
// only when the controller starts.
// aCarriedConsumer is the runtime on laptop and, carried by it, a module that consumes an event: the
// shape of messenger on 2026-10-06, assigned after the controller started.
func aCarriedConsumer(t *testing.T, open *stores) broker.Consumer {
t.Helper()
register(t, open, catalogue.Manifest{Module: catalogue.RuntimeModule, Version: "1",
OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/var/lib/mesh/node-tools/broker"}}})
register(t, open, catalogue.Manifest{Module: "messenger", Version: "1",
Consumes: []string{"billing.order.placed"}})
for _, m := range []string{catalogue.RuntimeModule, "messenger"} {
if _, err := open.inventory.Assign(t.Context(), "laptop", m); err != nil {
t.Fatal(err)
}
}
consumers, err := moduleConsumers(t.Context(), open.inventory)
if err != nil {
t.Fatal(err)
}
for _, c := range consumers {
if c.Module == "messenger" && c.Node == "laptop" {
return c.Consumer
}
}
t.Fatalf("messenger on laptop is derived no consumer: %+v", consumers)
return broker.Consumer{}
}
// aLateHolder is a module holding the build agent's seat on anchor, assigned after the controller
// started, and the worker its seat's queue should have for it.
func aLateHolder(t *testing.T, open *stores) broker.Consumer {
t.Helper()
register(t, open, catalogue.Manifest{Module: "late-builder", Version: "1",
Claims: []catalogue.Claim{{Name: "node-build-agent", Scope: catalogue.ScopeNode}}})
if _, err := open.inventory.Assign(t.Context(), "anchor", "late-builder"); err != nil {
t.Fatal(err)
}
for _, s := range inventory.MeshSeats() {
if s.Name == "node-build-agent" {
c, needed := broker.HolderConsumerFor("anchor", "late-builder", s)
if !needed {
t.Fatal("the build agent's seat needs no worker")
}
return c
}
}
t.Fatal("the mesh declares no build agent's seat")
return broker.Consumer{}
}
// asserted says whether a recording holds a consumer by stream and name.
func asserted(rec *recordingRaiser, want broker.Consumer) bool {
for _, c := range rec.consumers {
if c.Stream == want.Stream && c.Name == want.Name {
return true
}
}
return false
}
// What a send asserts includes what was assigned after the start's assertion: a carried module's
// consumer and a late holder's worker. The derivation is the start's own, so this holds without a bus.
func TestASendAssertsWhatWasAssignedAfterStart(t *testing.T) {
open := aMesh(t)
var atStart recordingRaiser
if _, err := assertBusObjects(t.Context(), open.inventory, &atStart); err != nil {
t.Fatal(err)
}
consumer := aCarriedConsumer(t, open)
worker := aLateHolder(t, open)
if asserted(&atStart, consumer) || asserted(&atStart, worker) {
t.Fatal("the start asserted what was not yet assigned; the test proves nothing")
}
var onSend recordingRaiser
if err := assertOnSend(t.Context(), open.inventory, &onSend, ""); err != nil {
t.Fatal(err)
}
if !asserted(&onSend, consumer) {
t.Fatalf("messenger's consumer %s on %s is not asserted by the send: %+v", consumer.Name, consumer.Stream, onSend.consumers)
}
if !asserted(&onSend, worker) {
t.Fatalf("the late holder's worker %s on %s is not asserted by the send: %+v", worker.Name, worker.Stream, onSend.consumers)
}
}
// failingRaiser refuses one consumer by name and records everything it was asked.
type failingRaiser struct {
recordingRaiser
refuse string
}
func (f *failingRaiser) EnsureConsumer(c broker.Consumer) error {
f.consumers = append(f.consumers, c)
if c.Name == f.refuse {
return errors.New("nats: API error: code=503 description=insufficient resources")
}
return nil
}
// A send whose objects cannot be asserted says so in its output and raises a condition — and the next
// send that asserts them clears it. The consumers after the refused one are still asked for.
func TestASendThatCannotAssertTheBusSaysSoAndRaisesACondition(t *testing.T) {
open := aMesh(t)
consumer := aCarriedConsumer(t, open)
worker := aLateHolder(t, open)
failing := &failingRaiser{refuse: consumer.Name}
var sendErr error
out := stdoutOf(t, func() error {
sendErr = assertOnSend(t.Context(), open.inventory, failing, " ")
return nil
})
if sendErr == nil || !strings.Contains(sendErr.Error(), "how messenger on laptop hears what it consumes") {
t.Fatalf("the failure is not answered: %v", sendErr)
}
if !strings.Contains(out, "THE BUS DOES NOT HOLD WHAT THIS SEND IMPLIES") ||
!strings.Contains(out, "insufficient resources") || !strings.Contains(out, "bus.objects.unasserted") {
t.Fatalf("the failure is not said in the send's output:\n%s", out)
}
if !asserted(&failing.recordingRaiser, worker) {
t.Fatal("the seat's worker was not asked for")
}
c, open1, err := conditionsFrom.Get(t.Context(), "bus.objects.unasserted")
if err != nil || !open1 {
t.Fatalf("no condition raised: %v", err)
}
if c.Kind != kindBusObjectsUnasserted || c.Source != sourceBusObjects ||
len(c.Evidence) == 0 || !strings.Contains(c.Evidence[0].Said, "insufficient resources") {
t.Fatalf("the condition does not say what failed: %+v", c)
}
// The next send asserts them, and that observation clears it.
if err := assertOnSend(t.Context(), open.inventory, &recordingRaiser{}, ""); err != nil {
t.Fatal(err)
}
if _, stillOpen, err := conditionsFrom.Get(t.Context(), "bus.objects.unasserted"); err != nil || stillOpen {
t.Fatalf("a send that asserted everything left the condition open: %v", err)
}
}
// Against a real bus, through the send's own grant: a module assigned after start has its consumer
// once a declaration is sent, and a holder assigned after start its seat's worker.
func TestNatsAModuleAssignedAfterStartGetsItsConsumerAtItsFirstSend(t *testing.T) {
url := testbus.URL(t)
open := aMesh(t)
js, err := broker.Dial(url)
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
for _, s := range []string{"CONTROL", "NODES", "ASSIGNMENTS", "EVENTS"} {
_ = js.Context().DeleteStream(s)
}
// The controller starting, before either was assigned.
if _, err := assertBusObjects(t.Context(), open.inventory, js); err != nil {
t.Fatal(err)
}
consumer := aCarriedConsumer(t, open)
worker := aLateHolder(t, open)
_ = js.Context().DeleteConsumer(worker.Stream, worker.Name)
for _, c := range []broker.Consumer{consumer, worker} {
if _, err := js.Context().ConsumerInfo(c.Stream, c.Name); !errors.Is(err, nats.ErrConsumerNotFound) {
t.Fatalf("%s on %s exists before any send; the test proves nothing: %v", c.Name, c.Stream, err)
}
}
server := link.ConnectNats(js, nil, nil)
if err := (overTheBus{open: open, server: server}).grant(t.Context(), nil); err != nil {
t.Fatal(err)
}
for _, c := range []broker.Consumer{consumer, worker} {
if _, err := js.Context().ConsumerInfo(c.Stream, c.Name); err != nil {
t.Fatalf("%s on %s is not on the bus after a send: %v", c.Name, c.Stream, err)
}
}
if _, raised, _ := conditionsFrom.Get(t.Context(), "bus.objects.unasserted"); raised {
t.Fatal("a send that asserted everything raised a condition")
}
}
-103
View File
@@ -1,103 +0,0 @@
package main
import (
"fmt"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// A first-node gate waits for a busy container (novox/hq ADR 0335 §4).
//
// When a machine reports, on what the gate sent it, that a container of a judged module was left running
// because it said busy, the judging has not started: the build is not yet what runs there. The walk says it
// waits for a busy container on that machine, counts no pass and no bound, and starts the judging — its
// Since — at the report that no longer lists the container waiting. The wait is the node-engine's, bounded
// at 15 minutes; the controller's own bound (busyGateBound past the engine's) is for an engine that stopped
// reporting while it waited, so no walk waits for ever. An engine older than the field never lists one, and
// its gate starts as before.
// busyGateSlack is how long past the engine's bound the controller still waits for the report that says the
// container replaced: a pass of the engine (its asks every 20 seconds) and the report's way here.
var busyGateSlack = 5 * time.Minute
// busyEngineBound is the node-engine's bound on a wait (novox/hq ADR 0335 §3), counted again here on this
// controller's clock from when it first read the wait.
const busyEngineBound = 15 * time.Minute
// busyGateEnds is when the gate stops waiting: the engine's own word on its bound, or this controller's count of
// it from when it first read the wait, whichever is sooner, and the slack — so neither a clock ahead on the
// machine nor a wrong word from it holds a walk longer.
func busyGateEnds(b *inventory.GateBusy) time.Time {
until := b.Seen.Add(busyEngineBound)
if !b.Until.IsZero() && b.Until.Before(until) {
until = b.Until
}
return until.Add(busyGateSlack)
}
// busyWait decides, before a judging, whether the gate waits for a busy container on a machine it judges:
// true while it waits, the gate saying so. False when nothing waits, or no longer: a wait that ended moves the
// gate's Since to its end. Pure but for the gate it is handed.
func busyWait(g *inventory.PlanGate, pairs []judged, f gateFacts, now time.Time) bool {
if g.Busy != nil && g.Busy.Ended != nil {
return false
}
for _, j := range pairs {
b, waits := busyOn(j, f)
if !waits {
continue
}
if g.Busy == nil {
g.Busy = &inventory.GateBusy{Machine: j.node, Module: j.module, Container: b.Container, Since: b.Since,
Seen: now}
}
if g.Busy.Seen.IsZero() {
g.Busy.Seen = now
}
g.Busy.Until, g.Busy.Said = b.Until, b.Said
if now.After(busyGateEnds(g.Busy)) {
// The engine said it would have replaced it by now: the judging starts, and finds the module not
// yet healthy while the machine still lists it waiting (busyStill), until the gate's own bound.
end := now
g.Busy.Ended, g.Busy.PastBound = &end, true
g.Since, g.Passes, g.LastPass, g.Healthy, g.HealthyAt, g.Applied = &end, 0, nil, nil, nil, nil
return false
}
g.Passes, g.LastPass, g.Last, g.Failing = 0, nil, g.Busy.Words(), nil
g.Read(now, false, g.Busy.Words())
return true
}
if g.Busy == nil {
return false
}
// Replaced: the judging starts at the report that said so.
end := now
if r, said := f.reports[g.Busy.Machine]; said && r.At != nil && r.At.After(g.Busy.Since) && r.At.Before(now) {
end = r.At.UTC()
}
g.Busy.Ended = &end
g.Since, g.Passes, g.LastPass, g.Last, g.Healthy, g.HealthyAt, g.Applied = &end, 0, nil, "", nil, nil, nil
return false
}
// busyOn is the container of a judged module its machine last reported, on what the gate sent it, as left
// running because it said busy.
func busyOn(j judged, f gateFacts) (inventory.BusyWait, bool) {
r, said := f.reports[j.node]
if !said || r.At == nil || !f.reportedOn(j.node, r) {
return inventory.BusyWait{}, false
}
return inventory.WaitingFor(r.Busy, j.module)
}
// busyStill is a judged module whose container its machine still lists waiting once the gate stopped waiting
// for it: not yet what was sent, so not healthy, whatever its health says of the container still running.
func busyStill(j judged, f gateFacts) (string, bool) {
b, waits := busyOn(j, f)
if !waits {
return "", false
}
return fmt.Sprintf("%s still runs its busy container %s, past its node-engine's bound of %s",
j.node, b.Container, b.Until.UTC().Format(time.RFC3339)), true
}
-148
View File
@@ -1,148 +0,0 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// A walk whose module's container waits as busy on its first machine says so, and starts that machine's
// first-machine gate only once the machine reports the container replaced (novox/hq ADR 0335 §4).
func TestAGateWaitsForABusyContainerAndStartsWhenItIsReplaced(t *testing.T) {
sent := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
g := &inventory.PlanGate{Machines: []string{"novox"}, Since: &sent,
Sent: map[string]inventory.SentDeclaration{"novox": {Digest: "d1", Sequence: 7}}}
pairs := []judged{{module: "build-agent", node: "novox"}, {module: "mesh-controller", node: "novox"}}
reported := sent.Add(20 * time.Second)
waiting := []inventory.BusyWait{{ID: "build-agent.server", Container: "mesh-build-agent",
State: inventory.BusyWaiting, Since: reported, Until: reported.Add(15 * time.Minute),
Said: "busy: novox/mesh-controller (check of #280), in hand since 00:58:00Z"}}
facts := func(at time.Time, busy []inventory.BusyWait) gateFacts {
// As judgeMoves hands it: the report held against the gate's own send.
return gateFacts{sent: g.Sent, reports: map[string]inventory.Reported{"novox": {Node: "novox",
Outcome: inventory.OutcomeApplied, At: &at, Declared: "d1", ReportedSequence: 7, Busy: busy}}}
}
now := reported.Add(time.Minute)
if !busyWait(g, pairs, facts(reported, waiting), now) {
t.Fatal("a gate whose module's container waits as busy judged it")
}
if !g.Since.Equal(sent) || g.Busy == nil || g.Busy.Machine != "novox" || g.Busy.Ended != nil {
t.Fatalf("the gate does not keep the wait: %+v %+v", g.Since, g.Busy)
}
for _, said := range []string{gateLine(g), g.Last} {
if !strings.Contains(said, "waits for a busy container on novox") {
t.Errorf("the walk does not say it waits for a busy container there: %s", said)
}
}
if !strings.Contains(g.Last, "check of #280") {
t.Errorf("the wait does not say what the container holds: %s", g.Last)
}
// Ten minutes on, still busy, within the engine's bound: still waiting, nothing judged, no bound run.
now = reported.Add(10 * time.Minute)
if !busyWait(g, pairs, facts(reported.Add(9*time.Minute), waiting), now) || g.Passes != 0 {
t.Fatalf("the gate stopped waiting within the bound: %+v", g)
}
// The machine reports it replaced: the judging starts at that report.
replacedAt := reported.Add(12 * time.Minute)
now = replacedAt.Add(5 * time.Second)
replaced := []inventory.BusyWait{{ID: "build-agent.server", Container: "mesh-build-agent",
State: "replaced-when-idle", Since: reported, Until: reported.Add(15 * time.Minute)}}
if busyWait(g, pairs, facts(replacedAt, replaced), now) {
t.Fatal("the gate still waits after the container was replaced")
}
if g.Since == nil || !g.Since.Equal(replacedAt) || g.Busy.Ended == nil || !g.Busy.Ended.Equal(replacedAt) {
t.Fatalf("the gate does not start at the report that said it replaced: since %v, %+v", g.Since, g.Busy)
}
if strings.Contains(gateLine(g), "waits for a busy container") {
t.Errorf("a gate judging still says it waits: %s", gateLine(g))
}
// And a later report listing it waiting again does not hold a gate that started.
if busyWait(g, pairs, facts(now, waiting), now.Add(time.Minute)) {
t.Error("a gate that started waits again")
}
}
// An engine that stops reporting while it waits holds no walk for ever: past the engine's bound and the slack,
// the judging starts, and finds the module not yet healthy while its container is still listed waiting.
func TestAGateWaitsNoLongerThanTheEnginesBound(t *testing.T) {
sent := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
g := &inventory.PlanGate{Machines: []string{"novox"}, Since: &sent}
pairs := []judged{{module: "build-agent", node: "novox"}}
waiting := []inventory.BusyWait{{ID: "build-agent.server", Container: "mesh-build-agent",
State: inventory.BusyWaiting, Since: sent, Until: sent.Add(15 * time.Minute)}}
f := gateFacts{reports: map[string]inventory.Reported{"novox": {Node: "novox", Outcome: inventory.OutcomeApplied,
At: &sent, Current: true, Busy: waiting}}}
now := sent.Add(15*time.Minute + busyGateSlack + time.Second)
if busyWait(g, pairs, f, now) {
t.Fatal("a gate waited past the engine's bound and the slack")
}
if !g.Busy.PastBound || !g.Since.Equal(now) {
t.Fatalf("the gate does not start, said past the bound: %+v %v", g.Busy, g.Since)
}
if why, still := busyStill(pairs[0], f); !still || !strings.Contains(why, "past its node-engine's bound") {
t.Fatalf("a module still running its busy container reads as what was sent: %q", why)
}
}
// A report from an engine older than the field lists nothing: the gate starts at the send, as before; and a
// report not about the gate's send says nothing of it.
func TestAGateOfAnEngineThatNeverWaitsStartsAsBefore(t *testing.T) {
sent := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
g := &inventory.PlanGate{Machines: []string{"ace"}, Since: &sent,
Sent: map[string]inventory.SentDeclaration{"ace": {Digest: "d2", Sequence: 9}}}
pairs := []judged{{module: "build-agent", node: "ace"}}
at := sent.Add(time.Minute)
older := gateFacts{sent: g.Sent, reports: map[string]inventory.Reported{"ace": {Node: "ace", Outcome: inventory.OutcomeApplied,
At: &at, Declared: "d2", ReportedSequence: 9}}}
if busyWait(g, pairs, older, at) || g.Busy != nil || !g.Since.Equal(sent) {
t.Fatalf("a gate on an engine that never waits did not start at the send: %+v %v", g.Busy, g.Since)
}
elsewhere := gateFacts{sent: g.Sent, reports: map[string]inventory.Reported{"ace": {Node: "ace", Outcome: inventory.OutcomeApplied,
At: &at, Declared: "d1", ReportedSequence: 8, Busy: []inventory.BusyWait{{ID: "build-agent.server",
State: inventory.BusyWaiting, Until: at.Add(time.Hour)}}}}}
if busyWait(g, pairs, elsewhere, at) {
t.Fatal("a wait reported about an earlier declaration held the gate")
}
}
// The wait is bounded on this controller's clock: an engine whose clock runs an hour ahead, saying it waits until
// then, holds the gate no longer than 15 minutes and the slack from when the controller first read the wait.
func TestAGateWaitIsBoundedOnTheControllersClock(t *testing.T) {
seen := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
g := &inventory.PlanGate{Machines: []string{"novox"}, Since: &seen}
pairs := []judged{{module: "build-agent", node: "novox"}}
ahead := seen.Add(time.Hour)
f := gateFacts{reports: map[string]inventory.Reported{"novox": {Node: "novox", Outcome: inventory.OutcomeApplied,
At: &seen, Current: true, Busy: []inventory.BusyWait{{ID: "build-agent.server", Container: "mesh-build-agent",
State: inventory.BusyWaiting, Since: ahead, Until: ahead.Add(15 * time.Minute)}}}}}
if !busyWait(g, pairs, f, seen) || !g.Busy.Seen.Equal(seen) {
t.Fatalf("the gate did not start waiting, seen now: %+v", g.Busy)
}
if !busyWait(g, pairs, f, seen.Add(15*time.Minute+busyGateSlack-time.Second)) {
t.Fatal("the gate stopped waiting within the bound on its own clock")
}
if busyWait(g, pairs, f, seen.Add(15*time.Minute+busyGateSlack+time.Second)) || !g.Busy.PastBound {
t.Fatalf("an engine's later word held the gate past the controller's bound: %+v", g.Busy)
}
}
// `plan` says, for a machine whose node-engine is older than the field, each container whose busy is not sent.
func TestAPlanSaysABusyAnOlderEngineIsNotSent(t *testing.T) {
modules := []catalogue.Manifest{{Module: "build-agent", Resources: []map[string]any{
{"id": "server", "type": "container", "busy": map[string]any{"command": "mesh-builder busy"}},
{"id": "agent-env", "type": "file"}}}}
lines := busyNotSent(modules, false)
if len(lines) != 1 || !strings.Contains(lines[0], "build-agent.server declares busy") ||
!strings.Contains(lines[0], "replaced at once") {
t.Fatalf("the plan does not say the busy not sent: %v", lines)
}
if lines := busyNotSent(modules, true); len(lines) != 0 {
t.Fatalf("a busy sent is said as not sent: %v", lines)
}
}
@@ -1,36 +0,0 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"strings"
"testing"
)
func TestCallsIsServedAndSaysWhatItKeeps(t *testing.T) {
handlers, behind, err := seatToolHandlers()
if err != nil || len(behind) != 0 {
t.Fatalf("%v %v", behind, err)
}
calls, ok := handlers["calls"]
if !ok {
t.Fatal("calls is not served")
}
if _, err := calls(context.Background(), json.RawMessage(`{"node":"anchor"}`)); err == nil ||
!strings.Contains(err.Error(), `"node"`) {
t.Errorf("calls took an argument it does not declare: %v", err)
}
if _, err := calls(context.Background(), json.RawMessage(`{"call":"call-0-0"}`)); err == nil ||
!strings.Contains(err.Error(), "not across a restart") {
t.Errorf("an unknown call was not said plainly: %v", err)
}
got, err := calls(context.Background(), json.RawMessage(`{}`))
if err != nil {
t.Fatal(err)
}
if _, listed := got.(map[string]any)["calls"]; !listed {
t.Errorf("calls answered %v", got)
}
}
-51
View File
@@ -1,51 +0,0 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
)
// consoleWaits is how long the console waits for an answer (mesh-tools node-tools/internal/bus
// RequestTimeout) — the shortest wait of a caller the mesh ships.
const consoleWaits = 30 * time.Second
// **A call's one answer is never later than its caller or the bus allow** (novox/hq issue 265): a
// holder answers within AnswerWithin, which must be inside both the console's wait and the window the
// bus gives an answer. Before, the console waited 30s, the bus 60s, and a push ran as long as it ran.
func TestAVerbAnswersInsideEveryWaitOnIt(t *testing.T) {
if link.AnswerWithin >= consoleWaits/2 {
t.Errorf("a call answers within %s: not well inside the console's %s", link.AnswerWithin, consoleWaits)
}
if link.AnswerWithin >= broker.ResponseTTL {
t.Errorf("a call answers within %s, after the bus stops permitting an answer at %s", link.AnswerWithin, broker.ResponseTTL)
}
}
// A push — named or through command — answers before it runs: it sends the machine holding the bus
// first, and the broker reloading its user list forgets the answer it was about to permit.
func TestAPushAnswersBeforeItSends(t *testing.T) {
for _, c := range []struct {
verb string
args map[string]any
want bool
}{
{"push", map[string]any{"node": "anchor", "why": "w"}, true},
{"push", map[string]any{"why": "w"}, true},
{"command", map[string]any{"command": "builds"}, false},
{"status", map[string]any{}, false},
{"assign", map[string]any{"node": "anchor", "module": "m"}, false},
} {
argv, err := argvFor(c.verb, c.args)
if err != nil {
t.Fatalf("%s %v: %v", c.verb, c.args, err)
}
if got := answersFirst(argv); got != c.want {
t.Errorf("%s %v answers first: %v, want %v", c.verb, c.args, got, c.want)
}
}
}
// `calls` is served, takes a call's id and nothing else, and says plainly when it holds no such call.
-181
View File
@@ -1,181 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"time"
"github.com/novox/mesh-controller/internal/chain"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The chain on the bus (novox/hq ADR 0408, design 47 milestone 5): builds, rollouts and assignments are keys of the
// bus's key-value store, written by the controller that holds the lease. The tables of the controller's store are
// copied into them once, by the serving controller at its start, and a kind is read from its bucket only once the move
// has recorded it (inventory.ChainReads).
// chainHolds is whether this process may write the chain now: the lease's gate, the one every act passes.
func chainHolds(ctx context.Context) error {
_, err := theLease.epoch(ctx)
return err
}
// openTheChain opens the chain's buckets on the bus: the serving controller's connection, or one dialled for a command.
func openTheChain(ctx context.Context) (*chain.Chain, error) {
js, err := aBus()
if err != nil {
return nil, err
}
// Not closed: a borrowed connection closes nothing, and a dialled one lasts as long as the command.
return chain.OnTheBus(ctx, js.Conn(), chainHolds)
}
// kindChainMoveRefused is the condition of a move that stopped without switching a kind.
const kindChainMoveRefused = "chain-move-refused"
// chainMoveRetry is how long a move that failed for want of the bus waits before it tries again.
const chainMoveRetry = 30 * time.Second
// movingTheChain copies the tables into the buckets and records it, at the serving controller's start: after the buckets
// exist and the lease is held, before anything reads a kind that has not moved. **The first try is made here, before the
// controller serves**, so what it serves is read from one place. A move that fails for want of the bus is tried again
// every half minute while the controller serves from the tables; one that is refused — a count that differs, a value
// over the bus's largest — is raised as an urgent condition, and the tables stay what is read: nothing is half-switched.
func movingTheChain(ctx context.Context, inv *inventory.Inventory) {
say := func(format string, args ...any) { fmt.Printf("chain: "+format+"\n", args...) }
// The first try is made here, before the controller serves; the tries after it, while it serves.
if movedOrStopped(ctx, inv, say) {
return
}
go func() {
for {
select {
case <-ctx.Done():
return
case <-time.After(chainMoveRetry):
}
if movedOrStopped(ctx, inv, say) {
return
}
}
}()
}
// movedOrStopped makes one try at the move and says whether there is nothing more to try: moved, refused, or the controller
// ending. False for a failure of the kind that is tried again (the bus away, a write not granted yet, the lease not held).
func movedOrStopped(ctx context.Context, inv *inventory.Inventory, say func(string, ...any)) bool {
_, err := inv.MoveChain(ctx, say)
var refusal *inventory.Refusal
switch {
case err == nil:
go clearChainMove(ctx)
go askAgainSoon(ctx, inv)
return true
case errors.As(err, &refusal):
say("%v", err)
go raiseChainMove(ctx, refusal)
return true
case ctx.Err() != nil:
return true
}
say("the move to the bus did not finish, and the tables are read until it does: %v", err)
return false
}
// chainMoveObservation is the urgent condition for a refused move.
func chainMoveObservation(r *inventory.Refusal) conditions.Observation {
return conditions.Observation{Scope: conditions.ScopeBus, ID: "chain." + r.Kind, Token: kindChainMoveRefused,
Kind: kindChainMoveRefused, Severity: conditions.Urgent, Resolver: conditions.ResolverOperator,
Summary: "the controller's " + r.Kind + " were not moved to the bus: " + r.Why, Source: "chain move"}
}
// whenKept runs f once the serving controller keeps conditions: the move runs before they are kept.
func whenKept(ctx context.Context, f func(k *conditions.Keeper)) {
for conditionsFrom == nil {
select {
case <-ctx.Done():
return
case <-time.After(time.Second):
}
}
f(conditionsFrom)
}
func raiseChainMove(ctx context.Context, r *inventory.Refusal) {
whenKept(ctx, func(k *conditions.Keeper) {
if _, err := k.Observe(ctx, chainMoveObservation(r)); err != nil {
fmt.Printf("chain: the condition for the refused move could not be raised: %v\n", err)
}
})
}
func clearChainMove(ctx context.Context) {
whenKept(ctx, func(k *conditions.Keeper) {
for _, kind := range []string{inventory.KindBuilds, inventory.KindRollouts, inventory.KindAssignments} {
key := chainMoveObservation(&inventory.Refusal{Kind: kind}).Key()
if _, err := k.Clear(ctx, key, "the "+kind+" are on the bus"); err != nil {
fmt.Printf("chain: the condition for the move of the %s could not be cleared: %v\n", kind, err)
}
}
})
}
// attachTheChain gives the inventory of the serving controller the chain on its connection, then moves what has not moved.
func attachTheChain(ctx context.Context, inv *inventory.Inventory, server *link.Server) error {
bus, ok := server.Bus().(link.OverNATS)
if !ok {
return errors.New("the chain lives on the bus, and this control plane is not on it")
}
c, err := chain.OnTheBus(ctx, bus.Conn, chainHolds)
if err != nil {
return err
}
inv.UseChain(c)
movingTheChain(ctx, inv)
return nil
}
// askAgain asks once more for each build request the move found never handed over (novox/hq issue 981 note 7 Q5): the
// requests a merge could not ask for, which a person's hand `build` would otherwise have to repeat. **At most once**: a
// request is claimed before it is asked and settled after, so a crash between loses the ask, which is where it stood.
func askAgain(ctx context.Context, inv *inventory.Inventory) {
pending, err := inv.RequestsToAskAgain(ctx)
if err != nil {
fmt.Printf("chain: the requests never handed over cannot be read: %v\n", err)
return
}
for _, r := range pending {
if claimed, err := inv.ClaimToAskAgain(ctx, r.ID); err != nil || !claimed {
if err != nil {
fmt.Printf("chain: %s cannot be claimed to be asked again: %v\n", r.ID, err)
}
continue
}
id, err := askABuild(ctx, buildSource{Repository: r.Repository, Seat: r.Seat}, r.Path, r.Ref)
if err != nil {
if serr := inv.AskedAgainAs(ctx, r.ID, "not asked again: "+err.Error()); serr != nil {
fmt.Printf("chain: %s was not asked again, and that could not be kept: %v\n", r.ID, serr)
}
fmt.Printf("chain: %s was never handed over and cannot be asked again: %v\n", r.ID, err)
continue
}
recordBuildRequest(ctx, inv, inventory.BuildRequest{ID: id, Repository: r.Repository, Seat: r.Seat, Path: r.Path,
Ref: r.Ref, Commit: r.Commit, For: r.For})
if err := inv.AskedAgainAs(ctx, r.ID, id); err != nil {
fmt.Printf("chain: %s was asked again as %s, and that could not be kept: %v\n", r.ID, id, err)
}
fmt.Printf("chain: %s, never handed over, is asked again as %s\n", r.ID, id)
}
}
// askAgainSoon waits for the controller to be serving, which an ask needs, and then asks again.
func askAgainSoon(ctx context.Context, inv *inventory.Inventory) {
select {
case <-ctx.Done():
case <-time.After(time.Minute):
askAgain(ctx, inv)
}
}
@@ -1,209 +0,0 @@
//go:build integration
package main
import (
"context"
"fmt"
"slices"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/chain"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/lease"
"github.com/novox/mesh-controller/internal/link"
)
// **Acceptance 10 of design 47: one read of an assignment names the commit, the build and the rollout that put it there,
// and one read of the rollout lists the assignments it changed** (novox/hq ADR 0408, issue 1095). It enters by the live
// trigger — a merge heard, its batch cut into a plan when the window closed, its build answered through the build seat's
// path, the plan advanced — and writes no key and no row by hand. Only the machine's end of the send is the test's: the send
// is recorded as the controller records one it made, and the gate's facts are the store's.
func TestOneReadOfAnAssignmentNamesItsCommitBuildAndRolloutAndOneReadOfTheRolloutListsItsAssignments(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksRecorded(t)
ctx := t.Context()
inv := open.inventory
if _, err := assign(ctx, open, "anchor", "notes"); err != nil {
t.Fatal(err)
}
var sent [][]string
wasSend := sendRollout
sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) {
sent = append(sent, append([]string(nil), names...))
current, err := open.inventory.CurrentBuilds(ctx)
if err != nil {
return nil, err
}
for _, node := range names {
digest := fmt.Sprintf("d-%s-%d", node, len(sent))
if err := open.inventory.RecordSent(ctx, nodeID(t, open, node), digest,
map[string]string{"notes": current["notes"].Commit}); err != nil {
return nil, err
}
}
return names, nil
}
t.Cleanup(func() { sendRollout = wasSend })
wasGather := gatherGateFacts
gatherGateFacts = func(ctx context.Context, open *stores, component string) (gateFacts, error) {
f := gateFacts{now: time.Now(), reports: map[string]inventory.Reported{}, engines: map[string]string{},
rolledBack: map[string][]lease.Rollback{}, served: map[string]served{}}
reports, err := open.inventory.LastReports(ctx)
if err != nil {
return f, err
}
for _, r := range reports {
f.reports[r.Node] = r
}
return f, nil
}
t.Cleanup(func() { gatherGateFacts = wasGather })
// A merge of notes is heard, and its window closes: the batch is cut into the walk.
hear(t, open, catalogueMerge("c-notes", "notes", t0), t0.Add(time.Second))
cutAt(t, open, t0.Add(time.Second+90*time.Second))
ws, _ := walks(t, open)
if len(ws) != 1 {
t.Fatalf("%d walk(s) after the cut", len(ws))
}
rollout := ws[0].ID
// The walk asks its first tier (at the cut, or at its next step); the build seat answers, and the controller takes the
// outcome in as it does.
asking, err := inv.PlanByID(ctx, rollout)
if err != nil {
t.Fatal(err)
}
for try := 0; try < 3 && (asking.Modules["notes"] == nil || asking.Modules["notes"].Build == ""); try++ {
advancePlans(ctx, open)
if asking, err = inv.PlanByID(ctx, rollout); err != nil {
t.Fatal(err)
}
}
buildID := asking.Modules["notes"].Build
if buildID == "" || len(*asked) != 1 || !strings.HasSuffix((*asked)[0], "#"+buildID) {
t.Fatalf("the walk asked %v, and notes' build is %q", *asked, buildID)
}
digest := "sha256:" + strings.Repeat("a", 64)
result := outcome(buildID, "modules/notes", "c-notes", "notes", "")
result.ReuseKey = "rk-notes"
result.Made = []link.MadeArtifact{{Name: "notes", Kind: "image", Reference: "artifact-store://notes/notes@" + digest}}
if err := (builds{inv: inv, open: open}).Built(ctx, result); err != nil {
t.Fatal(err)
}
// The walk sends the tier to its first machine.
for try := 0; try < 3 && len(sent) == 0; try++ {
advancePlans(ctx, open)
}
if !slices.Equal(slices.Concat(sent...), []string{"anchor"}) {
t.Fatalf("sent %v", sent)
}
// One read of the assignment.
on, err := inv.AssignmentsOn(ctx, "anchor")
a, ok := assignmentOf(on, "notes")
if err != nil || !ok {
t.Fatalf("the assignments of anchor: %+v %v", on, err)
}
// Beside it a module pulled in by the assignment, which no send of this walk put anywhere.
if wireguard, pulled := assignmentOf(on, "mesh-wireguard"); pulled && (wireguard.Build != "" || wireguard.Rollout != "") {
t.Fatalf("a module this rollout did not send names a build or a rollout: %+v", wireguard)
}
if a.Module != "notes" || a.Commit != "c-notes" || a.Build != buildID || a.Rollout != rollout || a.Artifact != digest ||
a.Since.IsZero() || a.Assigned.IsZero() || !a.Stands() {
t.Fatalf("the assignment names %+v, not commit c-notes, build %s, artifact %s, rollout %s", a, buildID, digest, rollout)
}
// One read of the rollout.
p, err := inv.PlanByID(ctx, rollout)
if err != nil {
t.Fatal(err)
}
wantChange := chain.Change{Node: "anchor", Module: "notes", After: chain.Ref{Build: buildID, Commit: "c-notes"}}
if !slices.Contains(p.Changed, wantChange) || !slices.Contains(p.Builds, buildID) || len(p.Steps) != 1 ||
p.Steps[0].Node != "anchor" || p.Steps[0].Sent == nil {
t.Fatalf("the rollout lists changed %+v, builds %v, steps %+v; want %+v", p.Changed, p.Builds, p.Steps, wantChange)
}
// And one read of the build names the same commit, with what it published and its reuse key.
built, found, err := inv.BuildByID(ctx, buildID)
if err != nil || !found || built.Commit != "c-notes" || built.Module != "notes" {
t.Fatalf("the build: %+v %v %v", built, found, err)
}
lately, err := inv.Builds(ctx, "notes", 1)
if err != nil || len(lately) != 1 || lately[0].ReuseKey != "rk-notes" || lately[0].Published == nil ||
lately[0].Published.Artifact != digest {
t.Fatalf("what the build published: %+v %v", lately, err)
}
// The verbs say the links.
if lines := changedLines(p); len(lines) != 2 || !strings.Contains(lines[1], "anchor/notes") ||
!strings.Contains(lines[1], buildID) {
t.Fatalf("`plans` says %v", lines)
}
if lines := runLines(on); len(lines) != 2 || !strings.Contains(lines[1], buildID) || !strings.Contains(lines[1], rollout) {
t.Fatalf("`node show` says %v", lines)
}
}
// A machine sent again what it runs says nothing new, and a push by hand with no rollout fills what is missing and blanks
// nothing (issue 1095).
func TestASendOfWhatAMachineRunsAlreadyChangesNoLinkOfItsAssignment(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
register(t, open, catalogue.Manifest{Module: "notes", Version: "1"})
if _, err := assign(ctx, open, "anchor", "notes"); err != nil {
t.Fatal(err)
}
if err := inv.RecordBuild(ctx, inventory.Build{ID: "build-1", Module: "notes", Commit: "c1", Repository: "novox/notes"}); err != nil {
t.Fatal(err)
}
node := nodeID(t, open, "anchor")
inRollout := inventory.WithRollout(ctx, "plan-1", map[string]inventory.RolloutModule{"notes": {Commit: "c1", Build: "build-1"}})
if err := inv.RecordSent(inRollout, node, "d1", map[string]string{"notes": "c1"}); err != nil {
t.Fatal(err)
}
// Sent again by hand, with no rollout: the assignment still names the rollout that put the build there.
if err := inv.RecordSent(ctx, node, "d2", map[string]string{"notes": "c1"}); err != nil {
t.Fatal(err)
}
on, _ := inv.AssignmentsOn(ctx, "anchor")
if a, _ := assignmentOf(on, "notes"); a.Rollout != "plan-1" || a.Build != "build-1" || a.Commit != "c1" {
t.Fatalf("the assignment names %+v", on)
}
// A declaration sent by hand with no builds known says nothing of them.
if err := inv.RecordSent(ctx, node, "d3", nil); err != nil {
t.Fatal(err)
}
if on, _ = inv.AssignmentsOn(ctx, "anchor"); runOf(on, "notes").Commit != "c1" {
t.Fatalf("a send of unknown builds changed %+v", on)
}
// A module sent a commit no build is kept of is named by its commit alone.
if err := inv.RecordSent(ctx, node, "d4", map[string]string{"notes": "c2"}); err != nil {
t.Fatal(err)
}
if on, _ = inv.AssignmentsOn(ctx, "anchor"); runOf(on, "notes").Commit != "c2" || runOf(on, "notes").Build != "" ||
runOf(on, "notes").Rollout != "" {
t.Fatalf("the assignment after a push of c2 by hand names %+v", on)
}
}
// assignmentOf is one module's assignment among those of a machine: assigning a module assigns what it requires beside it.
func assignmentOf(on []chain.Assignment, module string) (chain.Assignment, bool) {
for _, a := range on {
if a.Module == module {
return a, true
}
}
return chain.Assignment{}, false
}
func runOf(on []chain.Assignment, module string) chain.Assignment {
a, _ := assignmentOf(on, module)
return a
}
-121
View File
@@ -1,121 +0,0 @@
package main
import (
"context"
"fmt"
"slices"
"time"
"github.com/novox/mesh-controller/internal/chain"
"github.com/novox/mesh-controller/internal/inventory"
)
// The links a rollout makes (novox/hq ADR 0408 §1, design 47 acceptance 10, issue 1095): the send of a plan's tier puts a
// build on a machine; the assignment of the module there names the build, its commit and the rollout (written by the record of
// the send, inventory.RecordSent, from the context a send is made in), and the rollout lists the assignments it changed with the
// build before and after.
// rolloutModules is what a plan sends of each module: the commit and the build of it.
func rolloutModules(p *inventory.Plan) map[string]inventory.RolloutModule {
out := make(map[string]inventory.RolloutModule, len(p.Modules))
for m, s := range p.Modules {
if s != nil && s.Commit != "" {
out[m] = inventory.RolloutModule{Commit: s.Commit, Build: s.Build}
}
}
return out
}
// sendingAs is the context a plan's send is made in: what is recorded of the send names this rollout.
func sendingAs(ctx context.Context, p *inventory.Plan) context.Context {
return inventory.WithRollout(ctx, p.ID, rolloutModules(p))
}
// ranBefore is what each module on a machine ran before a send, as the chain names it: its assignment's build and commit,
// else the commit the machine was last sent for it. Read before the send, which writes the new ones.
func ranBefore(ctx context.Context, inv *inventory.Inventory, node string) map[string]chain.Ref {
was := map[string]chain.Ref{}
if sent, known, err := inv.SentBuilds(ctx, node); err == nil && known {
for m, commit := range sent {
was[m] = chain.Ref{Commit: commit}
}
}
if ran, err := inv.AssignmentRuns(ctx, node); err == nil {
for m, r := range ran {
was[m] = r
}
}
return was
}
// noteChanges writes into a plan that a send to a machine changed the assignments of the modules it carried: the build and
// commit before and after, the builds the rollout carried, and the step taken. A change noted already for the machine and the
// module is replaced, so a send made again says it once.
func noteChanges(p *inventory.Plan, node string, modules []string, was map[string]chain.Ref, at time.Time) {
for _, m := range modules {
s := p.Modules[m]
if s == nil {
continue
}
change := chain.Change{Node: node, Module: m, Before: was[m], After: chain.Ref{Build: s.Build, Commit: s.Commit}}
if at := slices.IndexFunc(p.Changed, func(c chain.Change) bool { return c.Node == node && c.Module == m }); at >= 0 {
change.Before = p.Changed[at].Before
p.Changed[at] = change
} else {
p.Changed = append(p.Changed, change)
}
if s.Build != "" && !slices.Contains(p.Builds, s.Build) {
p.Builds = append(p.Builds, s.Build)
}
}
if !slices.ContainsFunc(p.Steps, func(s chain.Step) bool { return s.Tier == p.Tier && s.Node == node }) {
sent := at.UTC()
p.Steps = append(p.Steps, chain.Step{Tier: p.Tier, Node: node, Sent: &sent})
}
}
// refSaid is a build and the commit it was made from as a person reads them; "nothing" for a module that ran none.
func refSaid(r chain.Ref) string {
switch {
case r.Build == "" && r.Commit == "":
return "nothing"
case r.Build == "":
return "commit " + short(r.Commit)
}
return r.Build + " (" + short(r.Commit) + ")"
}
// changedLines say the assignments a rollout changed: the machine, the module, and the build before and after.
func changedLines(p inventory.Plan) []string {
if len(p.Changed) == 0 {
return nil
}
out := []string{" changed:"}
for _, c := range p.Changed {
out = append(out, fmt.Sprintf(" %s/%s %s → %s", c.Node, c.Module, refSaid(c.Before), refSaid(c.After)))
}
return out
}
// runLines say what each module assigned to a machine runs there, as its assignment names it: the build, the commit, the
// rollout that put it there and since when.
func runLines(assigned []chain.Assignment) []string {
var out []string
for _, a := range assigned {
if a.Build == "" && a.Commit == "" {
continue
}
line := fmt.Sprintf(" %-22s %s", a.Module, refSaid(chain.Ref{Build: a.Build, Commit: a.Commit}))
if a.Rollout != "" {
line += ", sent by " + a.Rollout
}
if !a.Since.IsZero() {
line += " at " + a.Since.Local().Format("2006-01-02 15:04")
}
out = append(out, line)
}
if len(out) == 0 {
return nil
}
return append([]string{" what they run, as the chain names it:"}, out...)
}
-69
View File
@@ -1,69 +0,0 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/chain"
"github.com/novox/mesh-controller/internal/inventory"
)
// **A rollout lists the assignments it changed, each with the build before and after** (ADR 0408 §1, issue 1095): one
// change for a machine and a module however often the send is noted, the build before kept from the first, the builds the
// rollout carried once each, and one step for a machine and a tier.
func TestARolloutListsTheAssignmentsItChangedWithTheBuildBeforeAndAfter(t *testing.T) {
at := time.Date(2026, 10, 18, 3, 0, 0, 0, time.UTC)
p := inventory.Plan{ID: "plan-1", Tier: 1, Modules: map[string]*inventory.PlanModule{
"notes": {Commit: "c2", Build: "b2"}, "wiki": {Commit: "c2w", Build: "b2w"}, "nothing": nil}}
was := map[string]chain.Ref{"notes": {Build: "b1", Commit: "c1"}, "wiki": {Commit: "c1w"}}
noteChanges(&p, "anchor", []string{"notes", "wiki", "nothing"}, was, at)
noteChanges(&p, "laptop", []string{"notes"}, nil, at)
// The send noted again, after what was sent had become what the machine runs: the first before stands.
noteChanges(&p, "anchor", []string{"notes"}, map[string]chain.Ref{"notes": {Build: "b2", Commit: "c2"}}, at.Add(time.Minute))
want := []chain.Change{
{Node: "anchor", Module: "notes", Before: chain.Ref{Build: "b1", Commit: "c1"}, After: chain.Ref{Build: "b2", Commit: "c2"}},
{Node: "anchor", Module: "wiki", Before: chain.Ref{Commit: "c1w"}, After: chain.Ref{Build: "b2w", Commit: "c2w"}},
{Node: "laptop", Module: "notes", After: chain.Ref{Build: "b2", Commit: "c2"}},
}
if len(p.Changed) != len(want) {
t.Fatalf("changed %+v", p.Changed)
}
for k := range want {
if p.Changed[k] != want[k] {
t.Errorf("change %d is %+v, want %+v", k, p.Changed[k], want[k])
}
}
if len(p.Builds) != 2 || p.Builds[0] != "b2" || p.Builds[1] != "b2w" {
t.Errorf("builds %v", p.Builds)
}
if len(p.Steps) != 2 || p.Steps[0].Node != "anchor" || p.Steps[0].Tier != 1 || p.Steps[0].Sent == nil || p.Steps[1].Node != "laptop" {
t.Errorf("steps %+v", p.Steps)
}
lines := changedLines(p)
if len(lines) != 4 || !strings.Contains(lines[1], "anchor/notes b1 (c1) → b2 (c2)") ||
!strings.Contains(lines[3], "laptop/notes nothing → b2 (c2)") {
t.Errorf("changed lines %q", lines)
}
if got := changedLines(inventory.Plan{}); got != nil {
t.Errorf("a plan that changed nothing says %q", got)
}
}
func TestWhatAMachineRunsIsSaidAsItsAssignmentNamesIt(t *testing.T) {
at := time.Date(2026, 10, 18, 3, 0, 0, 0, time.UTC)
lines := runLines([]chain.Assignment{
{Node: "anchor", Module: "notes", Build: "b2", Commit: "c2c2c2c2c2c2", Rollout: "plan-1", Since: at},
{Node: "anchor", Module: "web", Commit: "c3"},
{Node: "anchor", Module: "never-sent"},
})
if len(lines) != 3 || !strings.Contains(lines[1], "notes") || !strings.Contains(lines[1], "b2 (c2c2c2c2)") ||
!strings.Contains(lines[1], "sent by plan-1") || !strings.Contains(lines[2], "web") || !strings.Contains(lines[2], "commit c3") {
t.Fatalf("lines %q", lines)
}
if runLines([]chain.Assignment{{Node: "a", Module: "m"}}) != nil {
t.Fatal("a module never sent is said to run something")
}
}
-221
View File
@@ -1,221 +0,0 @@
package main
import (
"fmt"
"path"
"slices"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The change plan (novox/hq ADR 0238): **one commit, one plan** — what a change does to the mesh, computed
// from its diffset (a repository, the branch it merges into, the commit at hand) by the planner, never by
// a mapping of its own. reachOfMerge answers what moves and what follows it; this adds where each module
// goes — the deploy plan, machine by machine in the build plan's order — and what is not an ordinary
// send. A pull request's check posts it with its verdict; the release a merge makes follows the same
// planner, so the two can be compared.
// policyOf is a module's upgrade policy, as the controller holds it; false when it cannot be read.
type policyOf func(module string) (inventory.Upgrade, bool)
// changePlanOf is the change plan of a reach: pure, so it is tested without a store.
func changePlanOf(repository, base, head string, r mergeReach, entries []inventory.Entry, policy policyOf) link.ChangePlan {
p := link.ChangePlan{Repository: repository, Base: base, Head: head, Moved: r.Moved(), Dependents: r.Dependents(),
New: r.Added, Unread: r.Unread, Tiers: r.Plan.Tiers}
byName := map[string]inventory.Entry{}
for _, e := range entries {
byName[e.Manifest.Module] = e
}
// The machine holding the bus, whose declaration carries every module's grants (novox/hq issue 490).
holder := ""
for _, e := range entries {
if e.Manifest.BusUsers != "" && e.Manifest.ClaimsSeat(catalogue.BrokerSeat) && len(e.On) > 0 {
holder = e.On[0]
}
}
var granting []string
machines := map[string]*link.MachinePlan{}
machine := func(name string) *link.MachinePlan {
if machines[name] == nil {
machines[name] = &link.MachinePlan{Machine: name}
}
return machines[name]
}
for _, tier := range r.Plan.Tiers {
for _, name := range tier {
e, held := byName[name]
if !held {
continue
}
u, known := policy(name)
waits := known && !u.RollOut
for _, on := range e.On {
if waits {
machine(on).Waits = append(machine(on).Waits, name)
} else {
machine(on).Receives = append(machine(on).Receives, name)
}
}
if !waits && holder != "" && !(len(e.On) == 1 && e.On[0] == holder) && len(e.On) > 0 &&
!slices.Contains(granting, name) {
granting = append(granting, name)
}
switch {
case (name == "nats" || catalogue.ProvidesBus(e.Manifest)) && len(e.On) > 0:
// Said before the merge (novox/hq issue 336): a new bus build holds every send to the bus's machine
// until a person takes the step, so merging it is a promise to take it.
p.Steps = append(p.Steps, fmt.Sprintf("%s: %s is never sent by an ordinary send, and until %s runs, "+
"nothing else is sent to %s either — unless the new build turns out the same as the one running there "+
"(issue 280)", busUpgradeNeeded, name, busUpgradeVerb, strings.Join(e.On, ", ")))
case name == "nats" || catalogue.ProvidesBus(e.Manifest):
p.Steps = append(p.Steps, "a planned bus step: the bus is upgraded by `bus upgrade`, never by an ordinary send")
case waits && len(e.On) > 0 && (u.From == catalogue.FromModule || u.From == catalogue.FromData):
// Judged per machine by the walk, against the build each was last sent (novox/hq ADR 0377).
p.Steps = append(p.Steps, fmt.Sprintf("%s waits for a person: its policy records (%s) — unless on a machine "+
"its move recreates none of its containers and moves none of its data, which the walk judges and then sends",
name, orNone(u.From)))
case waits && len(e.On) > 0:
p.Steps = append(p.Steps, fmt.Sprintf("%s waits for a person: its policy records (%s)", name,
orNone(u.From)))
}
if len(e.Manifest.Provides) > 0 && len(e.On) > 0 {
var offers []string
for _, o := range e.Manifest.Provides {
offers = append(offers, o.Name)
}
p.Steps = append(p.Steps, fmt.Sprintf("%s provides %s: its consumers are sent again after it",
name, strings.Join(offers, ", ")))
}
if e.Manifest.Data != nil && len(e.On) > 0 {
p.Steps = append(p.Steps, fmt.Sprintf("%s keeps data (ADR 0233): what it holds is backed up before it moves", name))
}
}
}
if len(granting) > 0 {
// Said before the merge (novox/hq issue 490): what the walk does when the change alters the bus's
// user list. Whether it does is known only once the builds are registered, so it is said as a rule.
p.Steps = append(p.Steps, fmt.Sprintf("%s: if this changes what the bus's user list says (a new tool, "+
"subject or user), %s is sent that list alone, every build there kept, before %s is judged on its first "+
"machine", grantsStepWord, holder, strings.Join(granting, ", ")))
}
var names []string
for name := range machines {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
p.Machines = append(p.Machines, *machines[name])
}
p.Summary = summaryOf(p)
p.Class = changeClassOf(p, entries)
return p
}
// classNone is the class of a change that moves no module: nothing for a walk to move (novox/hq ADR 0313 §2).
const classNone = "none"
// changeClassOf is a change plan's class (novox/hq ADR 0313 §2): that of the slowest-budgeted module it moves,
// itself or as a dependent — core when one is on the controller's own path or holds the mesh's resolver, leaf
// otherwise — and none when it moves nothing. A module it adds a directory for counts by its own class: the
// catalogue names a module's directory by the module, so the directory's last element is its name. The graph holds
// no manifest of it yet, so a new module is core by its name alone (on the controller's own path), never by a
// resolver claim its manifest makes.
func changeClassOf(p link.ChangePlan, entries []inventory.Entry) string {
if len(p.Moved) == 0 && len(p.Dependents) == 0 && len(p.New) == 0 {
return classNone
}
modules := append(append([]string{}, p.Moved...), p.Dependents...)
for _, dir := range p.New {
if name := path.Base(strings.Trim(dir, "/")); name != "." && name != "/" && name != "" {
modules = append(modules, name)
}
}
return classOfModules(modules, entries)
}
// grantsStepWord names the grants step (novox/hq issue 490): the bus's user list sent alone to the machine
// holding the bus ahead of a walk's gate. Not the bus step, which replaces the bus itself.
const grantsStepWord = "grants step"
// busUpgradeNeeded is how a change plan says that merging it holds the bus's machine for a person's step.
const busUpgradeNeeded = "merging this needs a person's bus upgrade"
// summaryOf is a change plan in one line: what it builds, where it goes, and whether the bus moves.
func summaryOf(p link.ChangePlan) string {
if len(p.Moved) == 0 && len(p.New) == 0 {
return "builds nothing: the change touches no module of the mesh's graph"
}
var parts []string
what := strings.Join(p.Moved, ", ")
if len(p.Dependents) > 0 {
what += fmt.Sprintf(" (+%d dependent(s))", len(p.Dependents))
}
if len(p.New) > 0 {
if what != "" {
what += ", "
}
what += "new: " + strings.Join(p.New, ", ")
}
var to []string
for _, m := range p.Machines {
if len(m.Receives) > 0 {
to = append(to, m.Machine)
}
}
if len(to) > 0 {
parts = append(parts, "builds "+what+" → "+strings.Join(to, ", "))
} else {
parts = append(parts, "builds "+what+", sent nowhere")
}
bus := "no bus step"
for _, s := range p.Steps {
switch {
case strings.HasPrefix(s, busUpgradeNeeded):
bus = busUpgradeNeeded
case strings.HasPrefix(s, "a planned bus step") && bus == "no bus step":
bus = "a bus step"
}
}
parts = append(parts, bus)
waiting := 0
for _, m := range p.Machines {
waiting += len(m.Waits)
}
if waiting > 0 {
parts = append(parts, fmt.Sprintf("%d wait(s) for a person", waiting))
}
return strings.Join(parts, "; ")
}
// planText is a change plan as a person reads it, for the pull request's comment.
func planText(p link.ChangePlan) string {
var b strings.Builder
fmt.Fprintf(&b, "change plan of %s at %.8s into %s: %s\n", p.Repository, p.Head, p.Base, p.Summary)
for i, tier := range p.Tiers {
fmt.Fprintf(&b, " tier %d: %s\n", i, strings.Join(tier, ", "))
}
for _, m := range p.Machines {
line := " " + m.Machine + ": "
if len(m.Receives) > 0 {
line += "receives " + strings.Join(m.Receives, ", ")
}
if len(m.Waits) > 0 {
if len(m.Receives) > 0 {
line += "; "
}
line += "waits for a person: " + strings.Join(m.Waits, ", ")
}
b.WriteString(line + "\n")
}
for _, s := range p.Steps {
b.WriteString(" - " + s + "\n")
}
if len(p.Unread) > 0 {
fmt.Fprintf(&b, " read by no module's build: %s\n", strings.Join(p.Unread, ", "))
}
return b.String()
}
-78
View File
@@ -1,78 +0,0 @@
package main
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **One commit, one change plan** (novox/hq ADR 0238): the planner's reach, laid out machine by machine in
// the build plan's order, with what is not an ordinary send said — the bus step, a module that waits for a
// person, a provider whose consumers follow it.
func TestAChangePlanSaysWhatEachMachineReceives(t *testing.T) {
const catalogue_ = "http://forge.internal:20000/novox/mesh-catalog.git"
entry := func(name, path string, on ...string) inventory.Entry {
e := fromRepo(name, catalogue_, path)
e.Source.BuiltFrom = "old"
e.On = on
return e
}
nats := entry("nats", "modules/nats", "anchor")
nats.Manifest.Provides = []catalogue.Offer{{Name: "mesh-bus"}}
gitea := entry("gitea", "modules/gitea", "anchor")
held := entry("photos", "modules/photos", "anchor", "laptop")
tools := fromRepo("node-tools", "http://forge.internal:20000/novox/mesh-tools.git", "node-tools")
tools.On = []string{"anchor", "laptop"}
entries := []inventory.Entry{nats, gitea, held, tools}
edges := []inventory.Edge{{From: "node-tools", To: "nats", Kind: inventory.EdgeStandsOn}}
policy := func(module string) (inventory.Upgrade, bool) {
if module == "photos" {
return inventory.Upgrade{RollOut: false, From: "a person"}, true
}
return inventory.Upgrade{RollOut: true}, true
}
plan := func(paths ...string) link.ChangePlan {
m := link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main", Commit: "head", Paths: paths}
return changePlanOf("novox/mesh-catalog", "main", "head", reachOfMerge(m, entries, nil, edges), entries, policy)
}
p := plan("modules/gitea/index.ts")
if p.Summary != "builds gitea → anchor; no bus step" {
t.Errorf("one module's change reads %q", p.Summary)
}
p = plan("modules/nats/Dockerfile", "modules/photos/x.js")
if !strings.Contains(p.Summary, "needs a person's bus upgrade") || !strings.Contains(p.Summary, "2 wait(s) for a person") ||
!strings.Contains(p.Summary, "(+1 dependent(s))") {
t.Errorf("the bus and a held module read %q", p.Summary)
}
got := map[string]string{}
for _, m := range p.Machines {
got[m.Machine] = strings.Join(m.Receives, ",") + "|" + strings.Join(m.Waits, ",")
}
// The bus first, what stands on it after: the build plan's order, per machine.
if got["anchor"] != "nats,node-tools|photos" || got["laptop"] != "node-tools|photos" {
t.Errorf("the deploy plan reads %v", got)
}
text := strings.Join(p.Steps, "\n")
// Said before the merge (novox/hq issue 336): merging it holds every send to the bus's machine until a
// person runs the bus's step, and the verb that does.
for _, want := range []string{"merging this needs a person's bus upgrade", "nothing else is sent to anchor",
"mesh_call mesh-controller.bus", "the same as the one running there", "photos waits for a person",
"nats provides mesh-bus"} {
if !strings.Contains(text, want) {
t.Errorf("the steps do not say %q:\n%s", want, text)
}
}
p = plan("merge-check.sh")
if !strings.HasPrefix(p.Summary, "builds nothing") || len(p.Machines) != 0 || strings.Join(p.Unread, ",") != "merge-check.sh" {
t.Errorf("a root file's plan reads %+v", p)
}
if !strings.Contains(planText(p), "read by no module's build: merge-check.sh") {
t.Errorf("the plan's text does not say why nothing is built:\n%s", planText(p))
}
}
-337
View File
@@ -1,337 +0,0 @@
package main
import (
"errors"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
// moduleCheck judges manifests where they are written, with no mesh (novox/hq ADR 0037, issue 148).
//
// **The same functions registration runs, and nothing the command line adds** (ADR 0035): the strict
// parse with every per-manifest problem, then the rules no single manifest can be judged against,
// over exactly the manifests given. Somebody describing their own application in their own
// repository runs this before pushing and finds out there, rather than when a running mesh refuses
// the registration or, later, when a machine applies something that resolved and should not have.
//
// **What it cannot know without a store, it says.** The mesh's own seat set is the store's (ADR
// 0122); this binary carries a compiled copy that the store overrides when loaded, so a claim on a
// mesh seat is judged fully only at registration. A seat another module declares is unknown unless
// that module's manifest is passed too. Both are printed as a note, not as a problem — a check that
// refused what it could not see would teach people to ignore it.
//
// **And every identity against every bound it meets** (novox/hq ADR 0225, issue 263): each module's
// identity, on a machine whose name is `longestMachine` characters, against the bound of every
// provision it wants that a manifest given here offers. An overflow is refused in the pull request
// that introduces it — a new requirement, a lowered bound, a longer slug — instead of on the
// provider's machine when a real machine's name first meets the module's.
// SomeManifestsVar, set by the merge gate, says the manifests given are only some of their repository's.
const SomeManifestsVar = "MESH_MODULE_CHECK_SOME"
func moduleCheck(paths []string, out io.Writer) error {
return moduleCheckFor(paths, catalogue.DefaultLongestMachine, out)
}
func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
if len(paths) == 0 {
return errors.New("module check <manifest.json>... — one file per module; pass every " +
"manifest of a repository together so the rules between them are checked too")
}
shelf := catalogue.Shelf{}
faulted := map[string]bool{}
failed := 0
// The list of predefined keys at the root of the repository above each manifest (novox/hq ADR 0348 §7), read
// once per repository: a manifest naming a key is judged against the list its own tree holds.
lists := map[string]catalogue.PredefinedList{}
listOf := func(path string) (catalogue.PredefinedList, error) {
root := catalogue.RepositoryRoot(filepath.Dir(path))
if root == "" {
return nil, nil
}
if list, read := lists[root]; read {
return list, nil
}
list, err := catalogue.ReadPredefined(root)
if err != nil {
return nil, err
}
lists[root] = list
return list, nil
}
for _, path := range paths {
raw, err := os.ReadFile(path)
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
continue
}
list, err := listOf(path)
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
continue
}
m, err := catalogue.ParseManifestWith(raw, list)
if err == nil {
// Each template's text read from the tree beside the manifest, and judged as inline content is
// (novox/hq ADR 0349 §2): a missing template is refused by name.
m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(path))
}
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
continue
}
if first, twice := shelf[m.Module]; twice {
_ = first
fmt.Fprintf(out, "%s: %s was already given; two manifests name one module\n", path, m.Module)
failed++
continue
}
// A definition names no installation (novox/hq ADR 0112, ADR 0155): judged here, in the
// catalogue-wide test, and at registration, which refuses in the same words.
if wrong := catalogue.TrustProblems(m); len(wrong) > 0 {
for _, p := range wrong {
fmt.Fprintf(out, "%s: %s\n", path, p)
}
failed += len(wrong)
faulted[m.Module] = true
}
// A module that declares a setting names the keys of its mergeable files (novox/hq ADR 0326 §3,
// issue 574): judged where a definition is written, as at registration, never where a stored
// build is read again.
if wrong := catalogue.WrittenSettingsProblems(m); len(wrong) > 0 {
for _, p := range wrong {
fmt.Fprintf(out, "%s: %s\n", path, p)
}
failed += len(wrong)
faulted[m.Module] = true
}
// A module lives whole in one repository (novox/hq ADR 0420 §1, issue 1019): when the check is told the
// repository it runs for (MESH_CHECK_REPOSITORY, set by the build's check), an artifact building from
// another is refused, as the merge gate refuses it. A run by hand names no repository and judges none.
if repo := os.Getenv("MESH_CHECK_REPOSITORY"); repo != "" && m.Build != nil {
for _, a := range m.Build.Artifacts {
if a.Context != nil && a.Context.Repository != "" && !sameRepository(a.Context.Repository,
link.SourceMoved{Owner: ownerOf(repo), Repo: repoOf(repo)}) {
fmt.Fprintf(out, "%s: %s builds artifact %s from %s: a module lives whole in one repository "+
"(novox/hq ADR 0420 §1)\n", path, m.Module, a.Name, a.Context.Repository)
failed++
faulted[m.Module] = true
}
}
}
if named := catalogue.InstallationProblems(m); len(named) > 0 {
for _, p := range named {
fmt.Fprintf(out, "%s: %s\n", path, p)
}
failed += len(named)
faulted[m.Module] = true
}
shelf[m.Module] = m
}
// Between the manifests: a seat declared twice, a use of a seat nothing declares, a claim on
// a seat that does not exist. Run only over what parsed, because a problem inside one manifest
// has already been said and would be said again here in a worse form.
//
// **Over some of a repository's manifests, a seat none of them declares is a note** (novox/hq issue 364), as
// this command's own word says above: the merge gate passes only the manifests a change touches, and says so
// with SomeManifestsVar, so a module that uses or claims a seat another module declares (the operator
// channel's, a channel bench) was refused there for a manifest it was not given. Given every manifest — the
// catalogue's own check, and registration — it stays a refusal.
some := os.Getenv(SomeManifestsVar) != ""
problems := catalogue.CatalogueProblems(shelf)
sort.Strings(problems)
for _, p := range problems {
if some && catalogue.IsUndeclaredSeat(p) {
fmt.Fprintf(out, "note: %s among the manifests given; registration judges it against the whole catalogue, "+
"and passing the declaring module's manifest too judges it here\n", p)
continue
}
fmt.Fprintln(out, p)
failed++
}
// Between the manifests too: an identity against the bounds of the provisions it wants, which
// only the provider's manifest states.
identities := catalogue.IdentityProblems(shelf, longestMachine)
sort.Strings(identities)
for _, p := range identities {
fmt.Fprintln(out, p)
}
failed += len(identities)
// And the data each module keeps (novox/hq ADR 0233): a provider that grants says what it keeps for
// its consumers, a directory a container writes is declared, and no backup line is written by hand.
data := catalogue.DataProblems(shelf)
sort.Strings(data)
for _, p := range data {
fmt.Fprintln(out, p)
}
failed += len(data)
var names []string
for name := range shelf {
names = append(names, name)
}
sort.Strings(names)
// **Every long-running resource says how it is ready** (novox/hq ADR 0240 rule 8): warned until the
// date, refused from it. The count is the catalogue's: its merge check keeps the number and lets a
// change lower it, never raise it.
undeclared := 0
required := !checkNow().Before(catalogue.HealthRequiredFrom)
for _, name := range names {
missing := catalogue.Undeclared(shelf[name])
undeclared += len(missing)
if len(missing) == 0 {
continue
}
if required {
for _, id := range missing {
fmt.Fprintf(out, "%s: %s stays up and does not say how it is ready: a long-running resource declares "+
"health since %s (novox/hq ADR 0240 rule 8)\n", name, id, catalogue.HealthRequiredFrom.Format("2006-01-02"))
}
failed += len(missing)
faulted[name] = true
}
}
// **A file that asks for a setting and does not say whether it is trusted counts as trusted** (novox/hq issue
// 339): listed and counted, never refused, so an author can opt out a file nothing trusts.
unsaid := 0
for _, name := range names {
unsaid += len(catalogue.UnsaidTrust(shelf[name]))
}
for _, name := range names {
m := shelf[name]
if faulted[name] {
continue
}
fmt.Fprintf(out, "%s: ok", name)
if n := len(m.Tools); n > 0 {
fmt.Fprintf(out, ", %d tool(s)", n)
}
if len(m.Invokes) > 0 {
fmt.Fprintf(out, ", invokes %s", joinInvokes(m.Invokes))
}
// The state it keeps and reads (novox/hq ADR 0201), so a reviewer sees what lands on the bus.
if len(m.State) > 0 {
kept := make([]string, 0, len(m.State))
for _, s := range m.State {
kept = append(kept, s.Name)
}
fmt.Fprintf(out, ", keeps state %s", strings.Join(kept, ", "))
}
// And the logs it keeps, with their caps (novox/hq ADR 0297).
if len(m.Logs) > 0 {
kept := make([]string, 0, len(m.Logs))
for _, l := range m.Logs {
kept = append(kept, fmt.Sprintf("%s (%d MiB)", l.Name, l.Cap()))
}
fmt.Fprintf(out, ", keeps log %s", strings.Join(kept, ", "))
}
if len(m.Reads) > 0 {
fmt.Fprintf(out, ", reads %s", strings.Join(m.Reads, ", "))
}
// The data it keeps, by class, so a reviewer sees what the mesh will protect and how.
if items := m.DataItems(); len(items) > 0 {
kept := make([]string, 0, len(items))
for _, it := range items {
kept = append(kept, it.ID+" ("+it.Class+")")
}
fmt.Fprintf(out, ", keeps %s", strings.Join(kept, ", "))
}
// How what it runs is ready (ADR 0240): each declared check, and what is judged by liveness alone.
var checks []string
for _, r := range m.Resources {
if h, has, _ := catalogue.ReadHealth(r); has {
checks = append(checks, fmt.Sprintf("%v by %s", r["id"], catalogue.HealthWords(h)))
}
}
// And its own runtime's, where it runs as its own account (novox/hq ADR 0366).
if own, has := catalogue.OwnRuntimeHealth(m); has {
if h, _, _ := catalogue.ReadHealth(own); h.Kind != "" {
checks = append(checks, fmt.Sprintf("%v by %s", own["id"], catalogue.HealthWords(h)))
}
}
if len(checks) > 0 {
fmt.Fprintf(out, ", ready: %s", strings.Join(checks, "; "))
}
if missing := catalogue.UnsaidTrust(m); len(missing) > 0 {
fmt.Fprintf(out, "; WARNING: %s ask(s) for a setting and do(es) not say whether it is trusted, so it counts as "+
"trusted: set at the terminal alone; say %q false where nothing trusts it (novox/hq issue 339)",
strings.Join(missing, ", "), catalogue.TrustedField)
}
if missing := catalogue.Undeclared(m); len(missing) > 0 {
fmt.Fprintf(out, "; WARNING: %s stay(s) up and say(s) not how it is ready — judged by liveness alone, "+
"refused from %s (ADR 0240 rule 8)", strings.Join(missing, ", "), catalogue.HealthRequiredFrom.Format("2006-01-02"))
}
fmt.Fprintln(out)
}
// The count the catalogue keeps (ADR 0240 rule 8), in a line its merge check reads.
fmt.Fprintf(out, "%s %d\n", UndeclaredHealthLine, undeclared)
fmt.Fprintf(out, "%s %d\n", UnsaidTrustLine, unsaid)
if failed > 0 {
return fmt.Errorf("%d problem(s) in %d manifest(s)", failed, len(paths))
}
fmt.Fprintf(out, "%d manifest(s) checked. Judged against the seats this binary carries; a claim on "+
"one of the mesh's own seats is judged fully at registration, and a seat declared by a "+
"module not given here reads as unknown. Identities judged on a %d-character machine name, "+
"against the bounds of the providers given here\n", len(paths), longestMachine)
return nil
}
// UndeclaredHealthLine starts the line `module check` says the count of long-running resources without
// `health` in, over the manifests given: the catalogue's merge check compares it with the number it keeps.
const UndeclaredHealthLine = "long-running resources without health:"
// UnsaidTrustLine starts the line `module check` says the count of files that ask for a setting and do not say
// whether it is trusted, and so count as trusted (novox/hq issue 339).
const UnsaidTrustLine = "files asking for a setting without saying whether it is trusted:"
// checkNow is the clock `module check` judges the date by; a test sets it.
var checkNow = time.Now
func joinInvokes(invokes []string) string {
if len(invokes) == 1 && invokes[0] == "*" {
return "every tool"
}
s := ""
for i, t := range invokes {
if i > 0 {
s += ", "
}
s += t
}
return s
}
// manifestsUnder lists every module.json below a directory, for `module check <dir>`.
func manifestsUnder(dir string) ([]string, error) {
var found []string
err := filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() && (d.Name() == "node_modules" || d.Name() == ".git" || d.Name() == "dist") {
return filepath.SkipDir
}
if !d.IsDir() && d.Name() == "module.json" {
found = append(found, path)
}
return nil
})
sort.Strings(found)
return found, err
}
-87
View File
@@ -1,87 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"os"
"strconv"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/inventory"
)
// A pull request's check builds go when it closes (novox/hq ADR 0350 §5): the build seat keeps them in the artifact
// store's scratch namespace for the walk of its merge to take. **Let go of where the moment is heard**: the
// controller ends the walk answering a merge — after that walk took what it reuses — and the delivery's owner, which
// hears the forge say a pull request closed unmerged, asks the `forget-checks` verb; both let go through the store,
// as the controller's own collection does. The build seat's holders let go of anything a week old on their own,
// so a word lost here leaves a week's builds, never more.
// checkBuildsIn is the scratch namespace as the controller reaches the artifact store.
func checkBuildsIn(ctx context.Context, inv *inventory.Inventory) (builder.Scratch, error) {
shelf, err := inv.Catalogue(ctx)
if err != nil {
return nil, err
}
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return nil, err
}
if address == "" {
return nil, errors.New("the artifact store is not on the mesh's network")
}
return builder.Registry{Address: address}, nil
}
// forgetChecks lets go of a pull request's check builds and answers what it did.
var forgetChecks = func(ctx context.Context, inv *inventory.Inventory, pull builder.PullRef) (string, error) {
scratch, err := checkBuildsIn(ctx, inv)
if err != nil {
return "", err
}
n, err := scratch.Forget(ctx, pull)
if err != nil {
return "", fmt.Errorf("%s: %d check(s)' builds let go of, and not all: %w", pull, n, err)
}
return fmt.Sprintf("%s: %d check(s)' builds let go of", pull, n), nil
}
// forgetChecksOfWalk lets go, on the save that ends a walk, of the check builds of every pull request whose merge
// it answers: the walk took what it reuses. Said, never fatal.
func forgetChecksOfWalk(ctx context.Context, inv *inventory.Inventory, p inventory.Plan) {
if p.WalkSaid != inventory.WalkEnded || p.Delivery == nil {
return
}
for _, pull := range reuseAskOf(&p).Pulls {
said, err := forgetChecks(ctx, inv, builder.PullRef{Owner: pull.Owner, Repo: pull.Repo, Number: pull.Number})
if err != nil {
fmt.Fprintf(os.Stderr, "%s: its check builds stay until the week's sweep: %v\n", p.ID, err)
continue
}
fmt.Printf("%s: %s\n", p.ID, said)
}
}
// forgetChecksCommand is `forget-checks <owner> <repo> <number>`: the verb the delivery's owner asks when the forge
// says a pull request closed unmerged (novox/hq ADR 0350 §5).
func forgetChecksCommand(ctx context.Context, args []string) error {
if len(args) != 3 {
return errors.New("forget-checks <owner> <repo> <number>")
}
number, err := strconv.Atoi(args[2])
if err != nil || number <= 0 {
return fmt.Errorf("%q is not a pull request's number", args[2])
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
said, err := forgetChecks(ctx, open.inventory, builder.PullRef{Owner: args[0], Repo: args[1], Number: number})
if err != nil {
return err
}
fmt.Println(said)
return nil
}
@@ -1,102 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"reflect"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func TestOnlyAWalksAskTakesAChecksBuildAndADryRunNever(t *testing.T) {
p := &inventory.Plan{Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
{Repository: "novox/shop", Number: 7}, {Repository: "http://forge.invalid/novox/web.git", Number: 3},
{Repository: "novox/shop", Number: 7}, {Repository: "novox/shop"}}}}
asked := reuseAskOf(p)
if want := []link.PullRef{{Owner: "novox", Repo: "shop", Number: 7}, {Owner: "novox", Repo: "web", Number: 3}}; !reflect.DeepEqual(asked.Pulls, want) {
t.Fatalf("the walk names %+v", asked.Pulls)
}
walk := withReuse(t.Context(), asked)
if reuseAsked(walk, false) != asked {
t.Fatal("a walk's ask does not carry its leave")
}
if reuseAsked(walk, true) != nil {
t.Fatal("a dry run carries a leave to take a check's build")
}
if reuseAsked(t.Context(), false) != nil {
t.Fatal("an ask that is not a walk's carries a leave")
}
}
func TestAReusedCheckBuildIsRegisteredByItsDigestAndGatedAsABuiltOne(t *testing.T) {
withTheGate(t) // the first-machine gate's own path (novox/hq ADR 0360 keeps it for earlier walks)
g := aGateMesh(t)
ctx := t.Context()
inv := g.open.inventory
gateEvery = 0
digest := "sha256:" + strings.Repeat("c", 64)
reused := func(id, reference string) link.BuildResult {
manifest := mustJSON(t, catalogue.Manifest{Module: "app", Version: "c3", Resources: []map[string]any{
{"id": "svc", "type": "container", "name": "app", "image": reference}}})
return link.BuildResult{ID: id, Repository: "novox/mesh-catalog", Path: "modules/app", Ref: "main", On: "anchor",
Commit: "c3", Manifest: manifest, Trunk: "main", OnTrunk: true, Branches: []string{"main"},
Made: []link.MadeArtifact{{Name: "server", Kind: "image", Reference: reference}},
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"},
Reused: "reused from the check of novox/mesh-catalog#9 at head abcdefabcdef (merged into trunk 123456123456): server " + digest,
ReusedFrom: &link.ReusedFrom{Pull: link.PullRef{Owner: "novox", Repo: "mesh-catalog", Number: 9},
Head: "abcdefabcdef", Trunk: "123456123456", Key: "r1:" + strings.Repeat("1", 64)}}
}
if _, _, err := takeIn(ctx, inv, reused(link.NewBuildID(time.Now()), "store.internal:5000/scratch/app/server@"+digest)); err == nil ||
!strings.Contains(err.Error(), "scratch") {
t.Fatalf("a build in the scratch namespace was registered: %v", err)
}
id := link.NewBuildID(time.Now().Add(time.Second))
result := reused(id, "store.internal:5000/app/server@"+digest)
if _, _, err := takeIn(ctx, inv, result); err != nil {
t.Fatal(err)
}
current, err := inv.CurrentBuilds(ctx)
if err != nil || current["app"].Commit != "c3" {
t.Fatalf("the reused build is not what app is registered at: %+v %v", current["app"], err)
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(shelf["app"].Resources)
if !strings.Contains(string(raw), "app/server@"+digest) {
t.Fatalf("app is registered as %s, not by the check's digest in its own namespace", raw)
}
// The walk asked it: the walk keeps what its build did, and its gate judges the registered build.
asked := time.Now().UTC()
p := g.plan(t)
p.Commit = "c3"
p.Modules["app"] = &inventory.PlanModule{State: "asked", AskedAt: &asked, Build: id}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
planBuiltSaying(ctx, g.open, "app", "c3", "", asked, id, result.MadeWords())
p = g.plan(t)
if m := p.Modules["app"]; m.State != "built" || !strings.HasPrefix(m.Made, "reused from the check of novox/mesh-catalog#9") {
t.Fatalf("the walk keeps %+v", m)
}
for i := 0; i < 6; i++ {
advancePlans(ctx, g.open)
}
p = g.plan(t)
gate := p.Modules["app"].Gate
if gate == nil || gate.To != "c3" || gate.Verdict != inventory.GatePassed {
t.Fatalf("the gate judged %+v, not the reused build", gate)
}
if sent, _, _ := inv.SentBuilds(ctx, "anchor"); sent["app"] != "c3" {
t.Fatalf("the first machine was sent %v, not the reused build", sent)
}
}
-193
View File
@@ -1,193 +0,0 @@
package main
import (
"context"
"errors"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// One build per change (novox/hq ADR 0350), as the controller asks it and takes it in.
// The check of a change is asked to build what a merge of it would move, in the walk's order: a module of the
// repository checked from the merge, one standing on it from its own repository as the walk builds it, and one
// that reads the repository through a context not at all. A change that moves no module builds nothing.
func TestACheckIsAskedToBuildWhatTheMergeWouldMove(t *testing.T) {
entries := []inventory.Entry{
{Manifest: catalogue.Manifest{Module: "app"}, Source: inventory.Source{Repository: "novox/shop", Seat: "git", Path: "modules/app"}},
{Manifest: catalogue.Manifest{Module: "web"}, Source: inventory.Source{Repository: "novox/web", Seat: "git", Ref: "main"}},
{Manifest: catalogue.Manifest{Module: "docs"}, Source: inventory.Source{Repository: "novox/docs", Seat: "git"}},
}
p := link.PullUpdated{Owner: "novox", Repo: "shop", Number: 7, Base: "main", Commit: "abc"}
scope := checkScope{Reach: mergeReach{Packaging: entries[2:], Plan: inventory.Plan{Tiers: [][]string{{"app", "docs"}, {"web"}}}}}
clone := func(s inventory.Source) (string, error) { return "http://forge.invalid/" + s.Repository + ".git", nil }
got := checkBuildsOf(p, scope, entries, clone)
want := []link.CheckBuild{
{Module: "app", Path: "modules/app", Merged: true, Tier: 0},
{Module: "docs", Tier: 0, Skip: "its build reads novox/shop through a context, which a check cannot hand the merge"},
{Module: "web", Repository: "http://forge.invalid/novox/web.git", Ref: "main", Tier: 1},
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("the check is asked to build\n%+v\nwant\n%+v", got, want)
}
if got := checkBuildsOf(p, checkScope{}, entries, clone); got != nil {
t.Fatalf("a change that moves no module asks for builds: %+v", got)
}
}
// A walk's ask carries its leave to take a check's build, naming the pull requests of the merges it answers; a dry
// run never does, and neither does an ask that is not a walk's.
// A walk ended lets go of its merges' check builds — only once it ended, after it took what it reuses.
func TestAWalkEndedLetsGoOfItsMergesCheckBuilds(t *testing.T) {
var forgot []builder.PullRef
was := forgetChecks
forgetChecks = func(_ context.Context, _ *inventory.Inventory, pull builder.PullRef) (string, error) {
forgot = append(forgot, pull)
return "", nil
}
t.Cleanup(func() { forgetChecks = was })
p := inventory.Plan{ID: "plan-1", Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
{Repository: "novox/shop", Number: 7}}}}
p.WalkSaid = inventory.WalkStarted
forgetChecksOfWalk(t.Context(), nil, p)
p.WalkSaid = ""
forgetChecksOfWalk(t.Context(), nil, p)
if len(forgot) != 0 {
t.Fatalf("a walk not ended let go of %v, which it may still take", forgot)
}
p.WalkSaid = inventory.WalkEnded
forgetChecksOfWalk(t.Context(), nil, p)
if want := []builder.PullRef{{Owner: "novox", Repo: "shop", Number: 7}}; !reflect.DeepEqual(forgot, want) {
t.Fatalf("the walk ended let go of %v", forgot)
}
}
// A build in the scratch namespace is never registered: only the walk's copy in the module's own namespace is.
func TestABuildInTheScratchNamespaceIsNeverRegistered(t *testing.T) {
if got := inScratch([]link.MadeArtifact{{Name: "server", Reference: "store.internal:5000/app/server@sha256:aa"}}); got != "" {
t.Fatalf("a module's own build reads as a scratch build: %s", got)
}
scratch := "store.internal:5000/scratch/app/server@sha256:aa"
if got := inScratch([]link.MadeArtifact{{Name: "server", Reference: scratch}}); got != scratch {
t.Fatalf("a scratch build was not found: %q", got)
}
if got := inScratch([]link.MadeArtifact{{Name: "look", Reference: "http://store.internal:5000/v2/scratch/app/look/blobs/sha256:bb"}}); got == "" {
t.Fatal("a scratch archive was not found")
}
}
// The walk says per module what it did: "reused from …" or "built: <why>" — and a build seat from before says
// neither, which is said.
func TestAWalksBuildSaysWhetherItReusedOrBuilt(t *testing.T) {
if got := (link.BuildResult{Reused: "reused from the check of novox/shop#7"}).MadeWords(); got != "reused from the check of novox/shop#7" {
t.Fatalf("said %q", got)
}
if got := (link.BuildResult{BuiltWhy: "no check built it"}).MadeWords(); got != "built: no check built it" {
t.Fatalf("said %q", got)
}
if got := (link.BuildResult{}).MadeWords(); !strings.Contains(got, "predates ADR 0350") {
t.Fatalf("said %q", got)
}
}
// A walk's module the check's build stood in for is registered by the digest the check built — the copy in the
// module's namespace — and the walk keeps "reused from …" for it; its first-node gate judges that registered
// build as it judges one the walk built. A scratch reference is refused. Needs the store (MESH_TEST_POSTGRES), and
// is named in testdata/store-tests.
// A walk's leave names the builds its own pull requests' verdicts made at the head each merge merged — never an
// earlier head's — read once per walk, and none when they cannot be read: the builder takes nothing a verdict does
// not name (novox/hq ADR 0350 §2, issue 703).
func TestAWalksLeaveNamesTheBuildsItsPullRequestsVerdictsMade(t *testing.T) {
was := checkedBuildsOf
forgetWalkVerdicts()
t.Cleanup(func() { checkedBuildsOf = was; forgetWalkVerdicts() })
var asked [][]link.PullRef
pull := link.PullRef{Owner: "novox", Repo: "shop", Number: 7}
atHead := link.CheckedBuild{Pull: pull, Check: "build-2", Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:2"}}
earlier := link.CheckedBuild{Pull: pull, Check: "build-1", Head: "aaaaaaaaaaaa", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
checkedBuildsOf = func(_ context.Context, pulls []link.PullRef) ([]link.CheckedBuild, error) {
asked = append(asked, pulls)
return []link.CheckedBuild{atHead, earlier}, nil
}
pulls := []link.PullRef{pull}
heads := map[link.PullRef]string{pull: "bbbbbbbbbbbb0123456789"}
for range 3 {
if got := reuseWithVerdicts(t.Context(), "plan-1", &link.ReuseAsk{Pulls: pulls}, heads); !reflect.DeepEqual(got.Builds,
[]link.CheckedBuild{atHead}) {
t.Fatalf("the leave names %+v, not only the merged head's build", got.Builds)
}
}
if !reflect.DeepEqual(asked, [][]link.PullRef{pulls}) {
t.Fatalf("the verdicts were read %d times for one walk: %v", len(asked), asked)
}
// A merge whose head was not said takes nothing.
if got := reuseWithVerdicts(t.Context(), "plan-1", &link.ReuseAsk{Pulls: pulls}, nil); got.Builds != nil {
t.Fatalf("a merge of no said head names %+v", got.Builds)
}
checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) {
return nil, errors.New("no bus")
}
if got := reuseWithVerdicts(t.Context(), "plan-2", &link.ReuseAsk{Pulls: pulls}, heads); got.Builds != nil {
t.Fatalf("verdicts that could not be read named %+v", got.Builds)
}
}
// A check is handed only the builds earlier checks of its own pull request at its own head made: another pull
// request's build, or an earlier head's, is never kept (novox/hq issue 703). None when they cannot be read.
func TestACheckIsHandedOnlyItsOwnHeadsBuilds(t *testing.T) {
was := checkedBuildsOf
t.Cleanup(func() { checkedBuildsOf = was })
pull := link.PullRef{Owner: "novox", Repo: "shop", Number: 7}
mine := link.CheckedBuild{Pull: pull, Check: "build-1", Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
earlier := link.CheckedBuild{Pull: pull, Check: "build-0", Head: "aaaaaaaaaaaa", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
another := link.CheckedBuild{Pull: link.PullRef{Owner: "novox", Repo: "shop", Number: 8}, Check: "build-9",
Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
var asked [][]link.PullRef
checkedBuildsOf = func(_ context.Context, pulls []link.PullRef) ([]link.CheckedBuild, error) {
asked = append(asked, pulls)
return []link.CheckedBuild{mine, earlier, another}, nil
}
p := link.PullUpdated{Owner: "novox", Repo: "shop", Number: 7, Commit: "bbbbbbbbbbbb0123456789"}
builds := []link.CheckBuild{{Module: "app"}}
if got := namedForTheCheck(t.Context(), p, builds); !reflect.DeepEqual(got, []link.CheckedBuild{mine}) ||
!reflect.DeepEqual(asked, [][]link.PullRef{{pull}}) {
t.Fatalf("the check is handed %+v, read for %v", got, asked)
}
checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) {
return nil, errors.New("no bus")
}
if got := namedForTheCheck(t.Context(), p, builds); got != nil {
t.Fatalf("verdicts that could not be read named %+v", got)
}
}
func forgetWalkVerdicts() {
walkVerdicts.Lock()
defer walkVerdicts.Unlock()
walkVerdicts.read = map[string]verdictsRead{}
}
// A walk's merged heads are the forge's word for each merge; a merge announced without its head, or a pull request
// merged at two heads in one walk, has none, and its walk takes no check's build (novox/hq issue 703).
func TestAWalksMergedHeadsAreTheForgesWord(t *testing.T) {
merge := func(number int, head string) inventory.PlanMerge {
return inventory.PlanMerge{Repository: "novox/shop", Number: number, Head: head}
}
p := &inventory.Plan{Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
merge(7, "aaaaaaaaaaaa1"), merge(8, ""), merge(9, "bbbbbbbbbbbb"), merge(9, "cccccccccccc")}}}
heads := mergedHeadsOf(p)
shop := func(n int) link.PullRef { return link.PullRef{Owner: "novox", Repo: "shop", Number: n} }
if heads[shop(7)] != "aaaaaaaaaaaa1" || heads[shop(8)] != "" || heads[shop(9)] != "" {
t.Fatalf("merged heads %v", heads)
}
}
-203
View File
@@ -1,203 +0,0 @@
package main
import (
"context"
"errors"
"flag"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
// `check-here` is a pull request's merge check run on the machine at hand **exactly as the build seat
// runs it** (novox/hq issue 286): the same code (builder.Check), the same ask the controller would make of
// the seat — the gate's modules and judge by the planner's own answer over the facts snapshot, the
// repositories beside it at the refs the snapshot says the seat clones them at — in the toolchain image
// the mesh holds, as the user the build seat runs as, against a throwaway store and bus of the versions
// the mesh runs.
//
// **The build seat is the reference.** A merge-check.sh that passed on an agent's machine failed on the
// seat for three reasons that were each the agent's environment, never the change: a newer Go whose gofmt
// lays a file out differently, a sibling checkout at the agent's feature branch where the seat had the
// commit the mesh runs, and a different user. Run here, a check sees what the seat will.
//
// check-here [--tree <checkout>] [--base main] [--registry <artifact store>] [--forge <url of the owner>]
// [--number <n>] [--user uid:gid] [--facts store|<file>] [--keep]
//
// The checkout's HEAD is what is checked, and it must be committed: the seat checks a commit, never a
// working tree.
func checkHereCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("check-here", flag.ContinueOnError)
tree := set.String("tree", ".", "the change's checkout; its HEAD is checked")
base := set.String("base", "main", "the branch the change would merge into")
registry := set.String("registry", os.Getenv("MESH_REGISTRY"), "the artifact store holding the facts and the toolchains")
forge := set.String("forge", "", "where the repositories beside it are cloned from, as <forge>/<repository>.git; "+
"the checkout's origin without its own name when not given")
number := set.Int("number", 0, "the pull request's number, when there is one")
// The build seat's service runs as root, and its check containers run as the builder does.
user := set.String("user", "0:0", "the user the check's containers run as: the build seat's")
keep := set.Bool("keep", false, "keep the workspace afterwards")
factsFrom := set.String("facts", "store", "the facts snapshot: `store`, the one the artifact store holds — "+
"what the seat reads — or a file")
if _, err := parseAround(set, args); err != nil {
return err
}
if *registry == "" {
return errors.New("check-here reads the facts and the toolchains from the artifact store: --registry <host:port> or MESH_REGISTRY")
}
dir, err := filepath.Abs(*tree)
if err != nil {
return err
}
git := func(args ...string) (string, error) {
cmd := exec.CommandContext(ctx, "git", args...)
cmd.Dir = dir
out, err := cmd.Output()
return strings.TrimSpace(string(out)), err
}
if dirty, err := git("status", "--porcelain", "--untracked-files=no"); err != nil {
return fmt.Errorf("%s is not a checkout: %w", dir, err)
} else if dirty != "" {
return errors.New("the checkout has changes not committed: the build seat checks a commit, so commit first")
}
head, err := git("rev-parse", "HEAD")
if err != nil {
return err
}
origin, err := git("remote", "get-url", "origin")
if err != nil {
return fmt.Errorf("the checkout has no origin to say which repository it is: %w", err)
}
owner, repo, prefix := ownerRepoOf(origin)
if *forge == "" {
*forge = prefix
}
if _, err := git("fetch", "--quiet", "origin", *base); err != nil {
return fmt.Errorf("cannot fetch %s to say what the change touches: %w", *base, err)
}
// Without rename detection, so a file moved out of a build source is said under its old name as well:
// its going is a change to the build that held it (novox/hq ADR 0267).
changedText, err := git("diff", "--name-only", "--no-renames", "origin/"+*base+"...HEAD")
if err != nil {
return err
}
var paths, removed []string
for _, p := range strings.Split(changedText, "\n") {
if p = strings.TrimSpace(p); p == "" {
continue
}
paths = append(paths, p)
if _, err := os.Stat(filepath.Join(dir, p)); os.IsNotExist(err) {
removed = append(removed, p)
}
}
_ = os.Setenv("MESH_REGISTRY", *registry)
f, err := readFacts(ctx, *factsFrom)
if err != nil {
return fmt.Errorf("the facts snapshot cannot be read: %w", err)
}
entries, read, edges, err := graphOfFacts(f)
if err != nil {
return err
}
p := link.PullUpdated{Owner: owner, Repo: repo, Number: *number, Base: *base, Commit: head, Paths: paths,
Removed: removed, ModuleDirs: moduleDirsIn(dir, paths), ModuleDirsSaid: true}
scope := pullScope(p, entries, read, edges)
_, scriptErr := os.Stat(filepath.Join(dir, builder.CheckScript))
if !scope.gated() && !scope.Mesh {
fmt.Printf("%s: %s, and the repository is not the mesh's: the build seat runs nothing for it\n",
owner+"/"+repo, noModuleTouched)
return nil
}
if !scope.gated() && scriptErr != nil {
fmt.Printf("%s: %s; %s\n", owner+"/"+repo, noModuleTouched, noMergeCheck)
return nil
}
toolchains := map[string]string{}
for language, reference := range f.Versions.Toolchains {
toolchains[language] = catalogue.Rerouted(reference, *registry)
}
if len(toolchains) == 0 {
return errors.New("the facts snapshot names no toolchain: it was taken by a controller from before " +
"issue 286, and the seat's toolchain cannot be known here")
}
beside := map[string]builder.Beside{}
for d, ref := range f.Beside {
from := d
if d == "mesh-controller-main" {
from = "mesh-controller"
}
beside[d] = builder.Beside{Repository: strings.TrimSuffix(*forge, "/") + "/" + from + ".git", Ref: ref}
}
id := fmt.Sprintf("check-here-%d", time.Now().UnixNano())
spec := builder.CheckSpec{ID: id, Repository: dir, Ref: head, Owner: owner, Repo: repo, Number: *number,
Paths: paths, Beside: beside, Modules: scope.Modules, New: scope.New, Manifests: scope.Manifests,
Base: *base, Judge: scope.Judge, Toolchain: toolchains["go"], Toolchains: toolchains, User: *user}
workspace, err := os.MkdirTemp("", "mesh-check-here-")
if err != nil {
return err
}
if !*keep {
defer removeWorkspace(spec.Toolchain, workspace, *user)
}
fmt.Fprintf(os.Stderr, "checking %s/%s at %.8s as the build seat would, against the facts of %s, in %s\n",
owner, repo, head, f.Taken.Format(time.RFC3339), workspace)
v, err := builder.Check(ctx, builder.Command, spec, workspace, *registry, builder.GitCredential{},
func(step, message string) {
if step != "output" {
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
}
})
if err != nil {
return fmt.Errorf("the check could not run — on the seat an error, never a pass: %w", err)
}
fmt.Println(v.Report)
fmt.Println()
fmt.Printf("mesh/merge-gate: %s — %s\n", strings.ToUpper(v.Gate.Verdict), v.Gate.Summary)
if v.Repo != nil {
fmt.Printf("mesh/repo-check: %s — %s\n", strings.ToUpper(v.Repo.Verdict), v.Repo.Summary)
}
for _, l := range []*builder.Layer{v.Gate, v.Repo} {
if l != nil && l.Verdict != "pass" && l.Verdict != "warning" {
return errors.New("the build seat would not pass this change")
}
}
return nil
}
// ownerRepoOf reads owner, repository and the owner's URL from a remote: ssh://git@host:222/novox/mesh-host.git
// → novox, mesh-host, ssh://git@host:222/novox.
func ownerRepoOf(remote string) (string, string, string) {
trimmed := strings.TrimSuffix(strings.TrimSuffix(remote, "/"), ".git")
cut := strings.LastIndexAny(trimmed, "/:")
if cut < 0 {
return "", trimmed, ""
}
repo, prefix := trimmed[cut+1:], trimmed[:cut]
owner := prefix
if at := strings.LastIndexAny(prefix, "/:"); at >= 0 {
owner = prefix[at+1:]
}
return owner, repo, prefix
}
// removeWorkspace removes what the check left, written as the seat's user: by a container of that user
// when it is not this one.
func removeWorkspace(image, workspace, user string) {
if image != "" && user != fmt.Sprintf("%d:%d", os.Getuid(), os.Getgid()) {
// Everything in it — the check's HOME is the workspace, so the toolchain's own files are there too.
_ = exec.Command("docker", "run", "--rm", "--user", user, "--volume", workspace+":/workspace", image,
"sh", "-c", "rm -rf /workspace/* /workspace/.[!.]*").Run()
}
_ = os.RemoveAll(workspace)
}
-165
View File
@@ -1,165 +0,0 @@
package main
import (
"context"
"fmt"
"regexp"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A delivery group's member is checked beside the other members' heads** (novox/hq ADR 0382, issue 867).
//
// A mesh-host change and a mesh-controller change that need each other could not both pass their own merge checks:
// each was read beside the other's repository at the commit the mesh runs, which holds neither change. So the
// controller asks mesh-delivery's `groups` once for every pull request's check it asks (§1), and names, in the ask,
// the group's other open pull requests into a base of the same name, in a repository the build seat reads beside the
// check — mesh-lab never — each with its whole head. The build seat reads those repositories at the sibling's merge (§2).
// A pull request for which mesh-delivery answered and named no sibling is asked exactly as before (§7).
// wholeCommit is a commit as the forge says a head: forty hexadecimal characters.
var wholeCommit = regexp.MustCompile(`^[0-9a-f]{40}$`)
// besideRepositories are the repositories (lower case, owner/repo) a check reads beside its own, by the directory each
// is found under — the core's, as besideOf clones them, and mesh-sdk beside the controller; never mesh-lab, which is
// always read at its trunk.
func besideRepositories(entries []inventory.Entry) map[string]string {
read := map[string]string{}
for _, e := range entries {
dir, core := coreModules[e.Manifest.Module]
if !core || e.Provided || e.Source.Repository == "" {
continue
}
read[nameOf(e.Source.Repository)] = dir
if dir == "mesh-controller" && e.Source.Seat != "" {
read[nameOf(siblingOf(e.Source.Repository, "mesh-sdk"))] = "mesh-sdk"
}
}
delete(read, "")
return read
}
// nameOf is a recorded repository as owner/repo in lower case, empty when it names no owner.
func nameOf(repository string) string {
owner, repo := sourceOwner(repository), sourceName(repository)
if owner == "" || repo == "" {
return ""
}
return strings.ToLower(owner + "/" + repo)
}
// siblingsFor is the pull request's siblings, as mesh-delivery's groups say them (ADR 0382 §1): the other members of
// the group its head branch names that are open, unmerged and into a base of its own base's name, in a repository the
// check reads beside it (beside: the directories the ask already names). The second answer is why they could not be
// learned, empty when they could: the group's members carry no head or base — a mesh-delivery from before the rule.
func siblingsFor(p link.PullUpdated, entries []inventory.Entry, beside map[string]link.CheckedOut,
lines []groupsLine) ([]link.CheckSibling, string) {
own := strings.ToLower(p.Owner + "/" + p.Repo)
var group *groupsLine
if p.Head == "" {
// A recheck (mesh-delivery's `recheck` through `delivery check`) carries no head branch name: the group is the one
// whose members hold this very pull request, by repository and number.
learnable := false
for i := range lines {
for _, m := range lines[i].Details {
learnable = true
if p.Number != 0 && m.Number == p.Number && strings.ToLower(m.Repository) == own {
group = &lines[i]
}
}
}
if group == nil {
for _, l := range lines {
if len(l.Details) == 0 && len(l.Members) > 0 && !learnable {
return nil, "mesh-delivery named the groups' members without their pull requests (a mesh-delivery from before novox/hq ADR 0382): " +
"the group of a pull request announced without its head branch cannot be found"
}
}
return nil, "" // a lone pull request
}
} else {
for i := range lines {
if lines[i].ID == p.Head {
group = &lines[i]
}
}
if group == nil {
return nil, ""
}
}
if len(group.Details) == 0 && len(group.Members) > 0 {
return nil, "mesh-delivery named the group's members without their heads and bases (a mesh-delivery from before novox/hq ADR 0382)"
}
read := besideRepositories(entries)
var siblings []link.CheckSibling
for _, m := range group.Details {
repository := strings.ToLower(m.Repository)
if repository == own {
continue
}
if !m.Open {
continue
}
if m.Head == "" || m.Base == "" {
return nil, "mesh-delivery named " + m.ID + " without its head or base (a mesh-delivery from before novox/hq ADR 0382)"
}
dir, readBeside := read[repository]
if m.Base != p.Base || !readBeside || dir == "mesh-lab" {
continue
}
if _, there := beside[dir]; !there {
continue
}
head := strings.ToLower(m.Head)
if !wholeCommit.MatchString(head) {
return nil, fmt.Sprintf("mesh-delivery named %s with a head that is not a whole commit (%q)", m.ID, m.Head)
}
siblings = append(siblings, link.CheckSibling{Repository: m.Repository, Number: m.Number, Head: head, Base: m.Base, Dir: dir})
}
sort.Slice(siblings, func(i, j int) bool { return siblings[i].Dir < siblings[j].Dir })
return siblings, ""
}
// withSiblings asks mesh-delivery's `groups` once and says in the request the pull request's siblings, or why they
// could not be learned (ADR 0382 §1): a check never waits on mesh-delivery past groupAskWithin and never fails for it,
// so a fix to mesh-delivery is still checked without it.
func withSiblings(ctx context.Context, request *link.BuildRequest, p link.PullUpdated, entries []inventory.Entry) {
lines, err := groupsAsked(ctx, deliverySeatHeld(entries))
siblingsFrom(request, p, entries, lines, err)
}
// siblingsFrom is withSiblings from mesh-delivery's answer already had.
func siblingsFrom(request *link.BuildRequest, p link.PullUpdated, entries []inventory.Entry, lines []groupsLine, err error) {
if request.Check == nil {
return
}
if err != nil {
request.Check.SiblingsUnknown = fmt.Sprintf("mesh-delivery did not say the group's pull requests (%v): "+
"the check reads no siblings", err)
return
}
request.Check.Siblings, request.Check.SiblingsUnknown = siblingsFor(p, entries, request.Check.Beside, lines)
}
// composedMember says the pull request is a member of a delivery group whose gate is the composed check's (novox/hq
// ADR 0403 clauses 1 and 7): the group has two members or more and mesh-delivery says so on its line. The group is
// found as siblingsFor finds it, by its head branch name, or, for a recheck that carries none, by the pull request's
// repository and number — not by siblings, which name only what the check reads beside it.
func composedMember(p link.PullUpdated, lines []groupsLine) bool {
own := strings.ToLower(p.Owner + "/" + p.Repo)
for _, l := range lines {
found := p.Head != "" && l.ID == p.Head
for _, m := range l.Details {
if p.Head == "" && p.Number != 0 && m.Number == p.Number && strings.ToLower(m.Repository) == own {
found = true
}
}
if found {
return l.ComposedGate && max(len(l.Details), len(l.Members)) >= 2
}
}
return false
}
-248
View File
@@ -1,248 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A delivery group's member is checked beside the other members' heads** (novox/hq ADR 0382 §1, §7, issue 867): the
// controller asks mesh-delivery's `groups` once for every check it asks, and names in the ask the group's other open
// pull requests into a base of the same name, in a repository the build seat reads beside it — mesh-lab never.
const (
headHost = "1111111111111111111111111111111111111111"
headController = "2222222222222222222222222222222222222222"
headSDK = "3333333333333333333333333333333333333333"
headLab = "4444444444444444444444444444444444444444"
headHq = "5555555555555555555555555555555555555555"
)
func theCoreOnTheGitSeat() []inventory.Entry {
onSeat := func(module, repository string) inventory.Entry {
e := fromRepo(module, repository, "")
e.Source.Seat = "git"
return e
}
nats := onSeat("nats", "novox/mesh-catalog")
nats.Source.Path = "modules/nats"
held := catalogue.Manifest{Module: "mesh-delivery", Claims: []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}}
return []inventory.Entry{onSeat("mesh-controller", "novox/mesh-controller"), onSeat("mesh-host", "novox/mesh-host"),
onSeat("node-tools", "novox/mesh-tools"), nats, {Manifest: held, On: []string{"anchor"}}}
}
func besideTheCore() map[string]link.CheckedOut {
return map[string]link.CheckedOut{"mesh-controller": {Repository: "c"}, "mesh-host": {Repository: "h"},
"mesh-tools": {Repository: "t"}, "mesh-catalog": {Repository: "k"}, "mesh-sdk": {Repository: "s"}, "mesh-lab": {Repository: "l"}}
}
func member(repository string, number int, head, base string, open bool) memberLine {
return memberLine{ID: repository + "@" + head[:12], Repository: repository, State: "checked", Number: number, Head: head,
Base: base, Open: open}
}
func group(id string, members ...memberLine) groupsLine {
line := groupsLine{ID: id, Details: members}
for _, m := range members {
line.Members = append(line.Members, m.ID+" ("+m.State+")")
}
return line
}
func TestASiblingIsTheGroupsOtherOpenPullRequestIntoABaseOfTheSameName(t *testing.T) {
entries := theCoreOnTheGitSeat()
pull := link.PullUpdated{Owner: "novox", Repo: "mesh-host", Number: 114, Base: "main", Head: "feat/x", Commit: headHost}
for _, c := range []struct {
what string
lines []groupsLine
want []string
unknown string
}{
{"the other member, open, same base", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))},
[]string{"mesh-controller=novox/mesh-controller#346@" + headController + " onto main"}, ""},
{"a member into a base of another name is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "release", true))},
nil, ""},
{"a member not open (merged or closed) is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", false))},
nil, ""},
{"mesh-lab is never one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-lab", 5, headLab, "main", true))},
nil, ""},
{"a repository the check reads nothing of is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/hq", 9, headHq, "main", true))},
nil, ""},
{"mesh-sdk, read beside the controller, is one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-sdk", 7, headSDK, "main", true))},
[]string{"mesh-sdk=novox/mesh-sdk#7@" + headSDK + " onto main"}, ""},
{"another group is none of its business", []groupsLine{group("feat/y",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))},
nil, ""},
{"a group of its own alone has none", []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true))},
nil, ""},
{"an older mesh-delivery names no heads", []groupsLine{{ID: "feat/x",
Members: []string{"novox/mesh-host@111111111111 (checked)", "novox/mesh-controller@222222222222 (checked)"}}},
nil, "without their heads and bases"},
{"a member without its base", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "", true))},
nil, "without its head or base"},
{"a head that is no commit", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController[:12]+"zzzzzzzzzzzzzzzzzzzzzzzzzzzz", "main", true))},
nil, "not a whole commit"},
} {
got, unknown := siblingsFor(pull, entries, besideTheCore(), c.lines)
var said []string
for _, s := range got {
said = append(said, s.Dir+"="+s.Repository+"#"+itoa(s.Number)+"@"+s.Head+" onto "+s.Base)
}
if strings.Join(said, ";") != strings.Join(c.want, ";") {
t.Errorf("%s: siblings %v, wanted %v", c.what, said, c.want)
}
if (c.unknown == "") != (unknown == "") || !strings.Contains(unknown, c.unknown) {
t.Errorf("%s: could not learn %q, wanted %q", c.what, unknown, c.unknown)
}
}
// A pull request into a release branch has siblings into the release branch of the same name (ADR 0358 §1).
release := pull
release.Base = "feat/57"
lines := []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "feat/57", true),
member("novox/mesh-controller", 346, headController, "feat/57", true), member("novox/mesh-tools", 8, headLab, "main", true))}
if got, _ := siblingsFor(release, entries, besideTheCore(), lines); len(got) != 1 || got[0].Dir != "mesh-controller" {
t.Errorf("into a release branch: %+v", got)
}
// A recheck announces the pull request without its head branch: its group is the one holding it (repository, number).
pull.Head = ""
got, unknown := siblingsFor(pull, entries, besideTheCore(), lines)
if len(got) != 1 || got[0].Dir != "mesh-tools" || unknown != "" {
t.Errorf("a recheck of a group member: %+v %q", got, unknown)
}
// A pull request in no group, rechecked, is alone: no sibling, and nothing said.
pull.Number = 999
if got, unknown := siblingsFor(pull, entries, besideTheCore(), lines); got != nil || unknown != "" {
t.Errorf("a lone pull request: %+v %q", got, unknown)
}
if got, unknown := siblingsFor(pull, entries, besideTheCore(), nil); got != nil || unknown != "" {
t.Errorf("no groups at all: %+v %q", got, unknown)
}
// A mesh-delivery that names members without their pull requests cannot find the group of a recheck: it says so.
older := []groupsLine{{ID: "feat/x", Members: []string{"novox/mesh-host@111111111111 (checked)"}}}
pull.Number = 114
if got, unknown := siblingsFor(pull, entries, besideTheCore(), older); got != nil || unknown == "" {
t.Errorf("an older mesh-delivery: %+v %q", got, unknown)
}
}
func itoa(n int) string { b, _ := json.Marshal(n); return string(b) }
// Every check asks mesh-delivery's `groups` once; the ask names the sibling's repository, head and base.
func TestEveryCheckAsksMeshDeliveryOnceAndNamesItsSiblings(t *testing.T) {
entries := theCoreOnTheGitSeat()
asked := 0
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
groupsAsked = func(_ context.Context, held bool) ([]groupsLine, error) {
asked++
if !held {
t.Error("the seat is held, yet it was asked as not held")
}
return []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true),
member("novox/mesh-controller", 346, headController, "main", true))}, nil
}
request := link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main", Beside: besideTheCore()}}
withSiblings(t.Context(), &request, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
if asked != 1 {
t.Fatalf("mesh-delivery was asked %d times for one check", asked)
}
want := link.CheckSibling{Repository: "novox/mesh-controller", Number: 346, Head: headController, Base: "main", Dir: "mesh-controller"}
if got := request.Check.Siblings; len(got) != 1 || got[0] != want || request.Check.SiblingsUnknown != "" {
t.Fatalf("the ask names siblings %+v, unknown %q", got, request.Check.SiblingsUnknown)
}
body, _ := json.Marshal(request.Check)
if !strings.Contains(string(body), `"siblings":[{"repository":"novox/mesh-controller","number":346,"head":"`+headController+
`","base":"main","dir":"mesh-controller"}]`) {
t.Fatalf("on the wire: %s", body)
}
}
// When mesh-delivery does not answer in time (or is not held, or answered without the fields), the check reads no
// sibling and its request says it could not learn them, so a fix to mesh-delivery is still checked without it.
func TestWhenMeshDeliveryDoesNotAnswerTheCheckReadsNoSiblingAndSaysSo(t *testing.T) {
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
for _, c := range []struct {
what string
err error
held bool
}{{"it did not answer", context.DeadlineExceeded, true}, {"nothing serves it", errors.New("nothing serves mesh-delivery.groups"), true},
{"no holder on record", errors.New("no holder of mesh-delivery is on record to say it"), false}} {
entries := theCoreOnTheGitSeat()
if !c.held {
entries = entries[:4]
}
groupsAsked = func(_ context.Context, held bool) ([]groupsLine, error) {
if held != c.held {
t.Errorf("%s: held %v", c.what, held)
}
return nil, c.err
}
request := link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main", Beside: besideTheCore()}}
withSiblings(t.Context(), &request, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
if len(request.Check.Siblings) != 0 || !strings.Contains(request.Check.SiblingsUnknown, c.err.Error()) ||
!strings.Contains(request.Check.SiblingsUnknown, "reads no siblings") {
t.Errorf("%s: siblings %+v, unknown %q", c.what, request.Check.Siblings, request.Check.SiblingsUnknown)
}
}
}
// **A pull request for which mesh-delivery answered and named no sibling is asked exactly as before** (ADR 0382 §7):
// byte for byte the request it was without the rule.
func TestAPullRequestWithNoSiblingIsAskedByteForByteAsBefore(t *testing.T) {
entries := theCoreOnTheGitSeat()
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
for _, answer := range [][]groupsLine{nil, {group("feat/other", member("novox/mesh-controller", 346, headController, "main", true))},
{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true), member("novox/hq", 3, headHq, "main", true))}} {
groupsAsked = func(context.Context, bool) ([]groupsLine, error) { return answer, nil }
build := func() link.BuildRequest {
return link.BuildRequest{ID: "build-1", Repository: "r", Ref: headHost, Check: &link.CheckRequest{Owner: "novox",
Repo: "mesh-host", Number: 114, Base: "main", Beside: besideTheCore(), Modules: []string{"mesh-host"}, Judge: "validator"}}
}
before, _ := json.Marshal(build())
after := build()
withSiblings(t.Context(), &after, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
got, _ := json.Marshal(after)
if string(before) != string(got) {
t.Fatalf("the request changed:\nbefore %s\nafter %s", before, got)
}
}
}
// What the build seat's verdict says of its siblings reaches the forge's holder in the controller's `checked`.
func TestTheVerdictCarriesTheSiblingsAndWhereTheJudgeCameFrom(t *testing.T) {
read := []link.CheckSibling{{Repository: "novox/mesh-controller", Number: 346, Head: headController, Base: "main", BaseCommit: headHost}}
c := checkedOf(link.BuildResult{ID: "b", Ref: headHost, Checked: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114},
Check: &link.CheckOutcome{Verdict: "pass", Summary: "ok", Siblings: read, JudgeFrom: "sibling novox/mesh-controller#346 merge abc",
Validator: "the change's own validator"}})
if len(c.Siblings) != 1 || c.Siblings[0] != read[0] || c.JudgeFrom == "" || c.Validator == "" {
t.Fatalf("the verdict says %+v", c)
}
body, _ := json.Marshal(c)
for _, want := range []string{`"siblings":[{"repository":"novox/mesh-controller","number":346,"head":"` + headController +
`","base":"main","base_commit":"` + headHost + `"}]`, `"judge_from":"sibling novox/mesh-controller#346 merge abc"`} {
if !strings.Contains(string(body), want) {
t.Fatalf("the verdict lacks %s:\n%s", want, body)
}
}
unknown := checkedOf(link.BuildResult{ID: "b", Check: &link.CheckOutcome{Verdict: "pass", SiblingsUnknown: "mesh-delivery did not answer"}})
if body, _ := json.Marshal(unknown); !strings.Contains(string(body), `"siblings_unknown":"mesh-delivery did not answer"`) ||
strings.Contains(string(body), `"siblings":`) {
t.Fatalf("the verdict that could not learn them says %s", body)
}
}
-161
View File
@@ -1,161 +0,0 @@
package main
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The check anybody can run is the check registration runs (novox/hq issue 148, ADR 0037): a manifest
// with a known fault is named, and one without passes, with no store opened.
func TestModuleCheckNamesAFaultAndNeedsNoMesh(t *testing.T) {
dir := t.TempDir()
good := filepath.Join(dir, "good.json")
bad := filepath.Join(dir, "bad.json")
os.WriteFile(good, []byte(`{"module":"shop","version":"1","tools":["price"],"invokes":["mesh-catalog.catalog_modules"]}`), 0o600)
os.WriteFile(bad, []byte(`{"module":"till","version":"1","invokes":["shop"]}`), 0o600)
var out bytes.Buffer
if err := moduleCheck([]string{good}, &out); err != nil {
t.Fatalf("a sound manifest was refused: %v\n%s", err, out.String())
}
if !strings.Contains(out.String(), "shop: ok, 1 tool(s), invokes mesh-catalog.catalog_modules") {
t.Fatalf("the report does not say what it checked:\n%s", out.String())
}
out.Reset()
err := moduleCheck([]string{good, bad}, &out)
if err == nil {
t.Fatal("a manifest invoking a module and no tool passed")
}
if !strings.Contains(out.String(), `till invokes "shop", which does not name a tool`) {
t.Fatalf("the fault is not named in the manifest's words:\n%s", out.String())
}
}
// The rules between manifests run over what was given together: a seat two modules declare is
// refused, which no single-manifest check can see.
func TestModuleCheckJudgesBetweenTheManifestsGiven(t *testing.T) {
dir := t.TempDir()
a := filepath.Join(dir, "a.json")
b := filepath.Join(dir, "b.json")
os.WriteFile(a, []byte(`{"module":"a","version":"1","seats":[{"name":"printer","scope":"mesh"}]}`), 0o600)
os.WriteFile(b, []byte(`{"module":"b","version":"1","seats":[{"name":"printer","scope":"mesh"}]}`), 0o600)
var out bytes.Buffer
if err := moduleCheck([]string{a, b}, &out); err == nil {
t.Fatalf("two declarations of one seat passed:\n%s", out.String())
}
if !strings.Contains(out.String(), "a seat name means one protocol") {
t.Fatalf("the cross-manifest rule was not the one named:\n%s", out.String())
}
}
// The real catalogue passes the command, the way it passes the test that used to be the only check.
func TestModuleCheckPassesTheCatalogue(t *testing.T) {
root := beside.Catalogue(t)
paths, err := manifestsUnder(root)
if err != nil || len(paths) == 0 {
t.Fatalf("no manifests under %s: %v", root, err)
}
var out bytes.Buffer
if err := moduleCheck(paths, &out); err != nil {
t.Fatalf("the catalogue does not pass its own check: %v\n%s", err, out.String())
}
}
func TestRegistrationRefusesADefinitionNamingAnInstallation(t *testing.T) {
// novox/hq ADR 0155: the check moves to registration once the catalogue passes it. Both
// ways in — `module add` and a build's result — go through this, and a name declared on
// purpose passes with its reason.
named := catalogue.Manifest{Module: "idp", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "x@sha256:aa",
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}},
}}
err := namesNoInstallation(named)
if err == nil || !strings.Contains(err.Error(), "login.mesh-one.be") ||
!strings.Contains(err.Error(), catalogue.NamesOnPurpose) {
t.Fatalf("a definition naming an installation is refused with the name and the way out; got %v", err)
}
meant := catalogue.Manifest{Module: "site", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "registry.mesh-one.be/org/site@sha256:cc",
catalogue.NamesOnPurpose: map[string]any{
"registry.mesh-one.be": "built outside the mesh until its repository is a build source here"}},
}}
if err := namesNoInstallation(meant); err != nil {
t.Fatalf("a name declared on purpose passes; got %v", err)
}
}
// **The controller's own manifest names every verb of its seat** (novox/hq ADR 0132): its tools lagged
// the seat's verbs for weeks, and `module check` — the gate's first step for a change touching it —
// refused it. Held here, so a verb added to the table without the manifest fails this repository's
// own suite rather than its next pull request's gate.
func TestTheControllersManifestServesEveryVerbOfItsSeat(t *testing.T) {
var out strings.Builder
if err := moduleCheck([]string{"../../module.json"}, &out); err != nil {
t.Fatalf("the controller's own module.json fails module check: %v\n%s", err, out.String())
}
}
// A module that uses a seat another module declares (novox/hq issue 364): refused over the whole catalogue when the
// declarer is missing, a note when the gate says it gives only the manifests a change touches.
func TestASeatAnotherModuleDeclaresIsANoteOverSomeManifests(t *testing.T) {
dir := t.TempDir()
user := filepath.Join(dir, "user.json")
os.WriteFile(user, []byte(`{"module":"asker","version":"1","uses":["operator-channel"]}`), 0o600)
var out bytes.Buffer
if err := moduleCheck([]string{user}, &out); err == nil {
t.Fatalf("a use of a seat nothing given declares passed the whole-catalogue check:\n%s", out.String())
}
t.Setenv(SomeManifestsVar, "1")
out.Reset()
if err := moduleCheck([]string{user}, &out); err != nil {
t.Fatalf("over some manifests the use was refused:\n%s", out.String())
}
if !strings.Contains(out.String(), "note: asker uses the seat \"operator-channel\"") {
t.Fatalf("the note was not said:\n%s", out.String())
}
}
// **A module lives whole in one repository** (novox/hq ADR 0420 §1, issue 1019): `module check`, told the
// repository it runs for, refuses an artifact whose build context names another; run by hand, it does not know.
func TestModuleCheckRefusesAnArtifactContextNamingAnotherRepository(t *testing.T) {
dir := t.TempDir()
write := func(name, context string) string {
path := filepath.Join(dir, name+".json")
raw := `{"module":"` + name + `","version":"1","build":{"artifacts":[{"name":"server","kind":"image","from":"Dockerfile"` + context + `}]}}`
if err := os.WriteFile(path, []byte(raw), 0o600); err != nil {
t.Fatal(err)
}
return path
}
other := write("other", `,"context":{"seat":"git","repository":"novox/mesh-controller","ref":"main"}`)
own := write("own", `,"context":{"seat":"git","repository":"novox/mesh-catalog","ref":"main"}`)
none := write("none", ``)
t.Setenv("MESH_CHECK_REPOSITORY", "novox/mesh-catalog")
var out bytes.Buffer
if err := moduleCheck([]string{other}, &out); err == nil {
t.Fatalf("a context naming another repository passed:\n%s", out.String())
}
if !strings.Contains(out.String(), "other builds artifact server from novox/mesh-controller: a module lives whole in one repository (novox/hq ADR 0420 §1)") {
t.Errorf("the refusal is not in the words of the gate:\n%s", out.String())
}
for _, path := range []string{own, none} {
out.Reset()
if err := moduleCheck([]string{path}, &out); err != nil {
t.Errorf("%s was refused: %v\n%s", path, err, out.String())
}
}
t.Setenv("MESH_CHECK_REPOSITORY", "")
out.Reset()
if err := moduleCheck([]string{other}, &out); err != nil {
t.Errorf("a local run, with no repository named, refused it: %v\n%s", err, out.String())
}
}
-663
View File
@@ -1,663 +0,0 @@
package main
import (
"context"
"encoding/json"
"fmt"
"path"
"regexp"
"runtime/debug"
"slices"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A pull request's merge check (novox/hq to-be 45 §9, ADR 0237 as amended 2026-10-06): the forge
// announces every pull request's new head, the controller decides what is checked, asks the build seat to
// check it, and says the verdict as `checked`, which the forge's holder sets as the pull request's
// statuses. **Before merge, never after**: every check the mesh had ran after a merge, on a machine.
//
// **The mesh's module graph decides, not the repository.** The controller holds the graph — every module,
// the repository and directory it is built from — and maps the pull request's changed paths onto it by
// the planner's own answer (reachOfMerge, touchedBy — the one place a changed file is mapped onto modules,
// for the merge handler, the release planner, the merge gate and this check; issue 280): **a changed file
// touches exactly the modules whose build reads it** — a module's own directory (the whole repository for
// one built from its root), or a repository its recipe packages. A file no build reads — a script at the
// root, a README — touches no module. A directory the change adds a module.json in, which the graph does
// not hold yet (said by the head, issue 278), is a new module and is checked too.
//
// - touches a module: the build seat runs **the gate** — `mesh/merge-gate`, the touched manifests, every
// machine composed with the change, the replays — and the repository's own merge-check.sh beside it;
// - touches none, in a repository that is the mesh's (it sources a module on some branch, or shares the
// core's owner): the gate is a pass that says so — a fact, not a missing check — and the repository's
// own merge-check.sh runs as `mesh/repo-check`, a warning when it has none;
// - touches none, anywhere else: the gate is a pass that says so, and nothing more is said.
//
// What is checked is decided here and run there (internal/builder/check.go).
// checkTimeout is how long one check may run on the build seat. Said here so the ask's watchdog (S6)
// and the builder agree on what late means.
const checkTimeout = 45 * time.Minute
// noModuleTouched is the gate's word for a change that touches nothing of the graph.
const noModuleTouched = "the change touches no module of the mesh's graph"
// noMergeCheck is the repository layer's word for a repository of the mesh with no merge-check.sh.
const noMergeCheck = "the repository declares no merge-check.sh: none of its own tests run before it merges"
// coreModules are the modules whose repositories are the mesh's core, by the directory a check finds
// each beside it — and whose owner is the mesh's own.
var coreModules = map[string]string{"mesh-controller": "mesh-controller", "mesh-host": "mesh-host",
"node-tools": "mesh-tools", "nats": "mesh-catalog"}
// checkScope is what a pull request reaches of the mesh's graph, as the planner reckons it.
type checkScope struct {
// Modules are the modules a merge of the change would move itself — built from the repository into
// the pull request's base and reading a changed file, or packaging the repository's source — and
// Dependents those the plan would build after them; New the directories it adds a module in.
Modules []string
Dependents []string
New []string
// Manifests are the moved modules' and the new ones' manifests in the change's tree.
Manifests []string
// Width is how many modules a merge would build, in how many tiers; Unread the changed files no
// module's build reads.
Width, Tiers int
Unread []string
// Mesh says the repository is the mesh's: modules are built from it on some branch, its owner is the
// core's, or the change adds a module to it.
Mesh bool
// Judge is who judges the gate (link.JudgeSelf, link.JudgeValidator, or the running controller).
Judge string
// From is a module built from the repository, for how the mesh clones it; nil when none is.
From *inventory.Entry
// Reach is the planner's whole answer, which the change plan is made from.
Reach mergeReach
}
func (s checkScope) gated() bool { return len(s.Modules)+len(s.New) > 0 }
// pullScope is what a pull request reaches: **the planner's own answer** (reachOfMerge), asked as if the
// head were merged into the base — never a mapping of its own, so a change to what a merge touches
// changes what is checked with it (novox/hq ADR 0238).
func pullScope(p link.PullUpdated, entries []inventory.Entry, read map[string][]inventory.ReadRepository,
edges []inventory.Edge) checkScope {
m := link.SourceMoved{Owner: p.Owner, Repo: p.Repo, Base: p.Base, CloneURL: p.CloneURL, Commit: p.Commit,
Paths: p.Paths, PathsTruncated: p.PathsTruncated, Removed: p.Removed, ModuleDirs: p.ModuleDirs,
ModuleDirsSaid: p.ModuleDirsSaid}
r := reachOfMerge(m, entries, read, edges)
s := checkScope{Modules: r.Moved(), Dependents: r.Dependents(), New: r.Added, Unread: r.Unread,
Width: len(r.Plan.Modules), Tiers: len(r.Plan.Tiers), Reach: r}
for _, e := range append(append([]inventory.Entry{}, r.Touched...), r.Deleted...) {
s.Manifests = append(s.Manifests, path.Join(strings.Trim(e.Source.Path, "/"), moduleManifestFile))
switch e.Manifest.Module {
case "mesh-controller":
s.Judge = link.JudgeSelf
case "mesh-host":
if s.Judge == "" {
s.Judge = link.JudgeValidator
}
}
}
for _, d := range r.Added {
s.Manifests = append(s.Manifests, path.Join(d, moduleManifestFile))
}
sort.Strings(s.Manifests)
s.Manifests = slices.Compact(s.Manifests)
// Whose repository it is, for how it is cloned and whether its own check is the mesh's to run.
owners := map[string]bool{}
for i, e := range entries {
if e.Provided {
continue
}
if _, core := coreModules[e.Manifest.Module]; core {
if owner := sourceOwner(e.Source.Repository); owner != "" {
owners[owner] = true
}
}
if sameRepository(e.Source.Repository, m) && s.From == nil {
s.From = &entries[i]
}
}
s.Mesh = s.From != nil || owners[strings.ToLower(p.Owner)] || s.gated()
return s
}
// sourceOwner is the owner of a recorded repository, a path on the git seat or a URL: novox/mesh-host → novox.
func sourceOwner(repository string) string {
parts := strings.Split(strings.Trim(strings.TrimSuffix(repository, ".git"), "/"), "/")
if len(parts) < 2 {
return ""
}
return strings.ToLower(parts[len(parts)-2])
}
// PullUpdated decides a pull request's merge check, and asks for it when there is something to run.
func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
inv := f.open.inventory
entries, err := inv.Catalogued(ctx)
if err != nil {
return err
}
read, err := readForPlanning(ctx, inv)
if err != nil {
return err
}
edges, err := inv.Dependencies(ctx)
if err != nil {
return err
}
scope := pullScope(p, entries, read, edges)
// **A cycle of edges fails the check, naming every module and edge** (novox/hq ADR 0419 clause 6), a lone pull
// request's as a group's composed check is: no order exists for what it changes.
alone := orderMember{ID: p.Owner + "/" + p.Repo + "@" + twelve(p.Commit), Repository: p.Owner + "/" + p.Repo, Base: p.Base,
Head: p.Commit, Number: p.Number, Paths: p.Paths, PathsTruncated: p.PathsTruncated, Removed: p.Removed, BaseSHA: p.BaseSHA}
if failing, cyclic, err := cycleVerdict(ctx, "", []orderMember{alone},
map[string]orderReach{alone.ID: {Moved: scope.Modules}}, edges, entries); err != nil {
return err
} else if cyclic {
failing.Commit, failing.Trunk = p.Commit, twelve(p.BaseSHA)
fmt.Printf("%s/%s#%d (%.8s): %s\n", p.Owner, p.Repo, p.Number, p.Commit, failing.Summary)
sayChecked(ctx, failing)
return nil
}
// **A member of a group whose gate is the composed check's runs its repository's check alone** (novox/hq ADR 0403
// clause 1): mesh-delivery is asked once, and says so on the group's line. Told nothing, the pull request is
// checked alone, gate included (clause 6).
lines, groupsErr := groupsAsked(ctx, deliverySeatHeld(entries))
composed := groupsErr == nil && composedMember(p, lines)
// The change plan of the commit at hand (ADR 0238): what a merge of it would build and send, posted
// with the verdict whatever the verdict is.
plan := changePlanOf(p.Owner+"/"+p.Repo, p.Base, p.Commit, scope.Reach, entries, func(module string) (inventory.Upgrade, bool) {
u, err := inv.UpgradeOf(ctx, module)
return u, err == nil
})
fmt.Print(planText(plan))
direct := directVerdict(p, plan)
if composed {
scope.Modules, scope.Dependents, scope.New, scope.Manifests = nil, nil, nil, nil
composedGate(&direct)
}
switch {
case !scope.gated() && !scope.Mesh:
fmt.Printf("%s/%s#%d (%.8s): %s, and the repository is not the mesh's: said, nothing run\n",
p.Owner, p.Repo, p.Number, p.Commit, noModuleTouched)
sayChecked(ctx, direct)
return nil
case !scope.gated() && p.MergeCheckSaid && !p.MergeCheck:
direct.RepoCheck = &link.CheckLayer{Verdict: "warning", Summary: noMergeCheck}
fmt.Printf("%s/%s#%d (%.8s): %s; %s\n", p.Owner, p.Repo, p.Number, p.Commit, noModuleTouched, noMergeCheck)
sayChecked(ctx, direct)
return nil
}
request, err := checkRequestFor(ctx, f.open, p, scope, entries)
if err != nil {
return err
}
request.Check.Plan = &plan
request.Check.Composed = composed
// **The group's other open pull requests are read beside it** (novox/hq ADR 0382 §1).
siblingsFrom(&request, p, entries, lines, groupsErr)
seat := buildSeatHeld(ctx)
ask, err := askOverOn(seat)
if err != nil {
return err
}
defer ask.Close()
if err := ask.Ask(ctx, request); err != nil {
return err
}
// **A newer request replaces the one still waiting, and a newer key stops the one running** (novox/hq ADR 0406 §3, §4,
// issue 974): the check queue keeps one request a pull request, and a running check watches its key. Nothing is asked
// of the forge here (issue 726).
what := "its own merge-check.sh alone: " + noModuleTouched
if composed {
what = "its own merge-check.sh alone: the gate is its group's composed check's"
}
if scope.gated() {
what = fmt.Sprintf("the gate over %s (a merge would build %d module(s) in %d tier(s))",
strings.Join(append(append([]string{}, scope.Modules...), prefixedAll("new:", scope.New)...), ", "),
scope.Width, scope.Tiers)
}
fmt.Printf("%s/%s#%d (%.8s): asked %s to check it before it merges — %s — as %s\n", p.Owner, p.Repo, p.Number,
p.Commit, seat, what, request.ID)
return nil
}
// composedWords is what a group member's own verdict says of its gate layer (novox/hq ADR 0403 clause 1).
const composedWords = "judged by its group's composed check"
// gateComposed is a gate layer's verdict when no gate was run for the member alone: never a pass. Mesh-delivery reads it
// as the composed check's to judge, and the forge's holder sets no mesh/merge-gate from it.
const gateComposed = "composed"
// composedGate says a group member's own verdict has no gate of its own: its gate layer, and the verdict that is
// the gate's, read "composed" in place of a pass.
func composedGate(c *link.Checked) {
c.Verdict, c.Summary = gateComposed, composedWords
c.Gate = &link.CheckLayer{Verdict: gateComposed, Summary: composedWords}
}
// directVerdict is the verdict of a change no build judges — it touches no module, and either the repository is not
// the mesh's or the head holds no merge-check.sh. **It names the trunk commit it holds for** (novox/hq ADR 0323 §3):
// the base branch's head the forge announced with the head, so the merge tool can weigh it against the trunk's head
// and does not refuse it for ever (a verdict naming no trunk is never the trunk's head's). Nothing was built to
// judge: the trunk commit is the one the forge said the change was announced against.
func directVerdict(p link.PullUpdated, plan link.ChangePlan) link.Checked {
trunk := p.BaseSHA
if len(trunk) > 12 {
trunk = trunk[:12]
}
if trunk != "" {
plan.Trunk = trunk
}
return link.Checked{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Commit: p.Commit,
ID: link.NewBuildID(time.Now()), Verdict: "pass", Summary: noModuleTouched,
Gate: &link.CheckLayer{Verdict: "pass", Summary: noModuleTouched}, Plan: &plan, Trunk: trunk}
}
// checkRequestFor is the ask for one pull request's head: the repository as the mesh clones it, the head,
// and what is read beside it.
func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scope checkScope,
entries []inventory.Entry) (link.BuildRequest, error) {
shelf := map[string]catalogue.Manifest{}
for _, e := range entries {
shelf[e.Manifest.Module] = e.Manifest
}
world, err := theRestOfTheMesh(ctx, open.inventory, shelf, "")
if err != nil {
return link.BuildRequest{}, err
}
clone := func(s inventory.Source) (string, error) {
if s.Seat == "" {
return s.Repository, nil
}
return clonedFromSeat(world, s.Seat, s.Repository)
}
// As the mesh clones a module built from it; a repository no module is built from, from the forge.
source := inventory.Source{Seat: gitSeat, Repository: p.Owner + "/" + p.Repo}
if scope.From != nil {
source = scope.From.Source
}
repository, err := clone(source)
if err != nil {
return link.BuildRequest{}, err
}
current, err := open.inventory.CurrentBuilds(ctx)
if err != nil {
return link.BuildRequest{}, err
}
beside, err := besideOf(entries, func(module string) string { return current[module].Commit }, clone)
if err != nil {
return link.BuildRequest{}, err
}
builds := checkBuildsOf(p, scope, entries, clone)
return link.BuildRequest{
ID: link.NewBuildID(time.Now()),
Repository: repository,
Ref: p.Commit,
Held: heldBy(ctx),
Seats: seatBases(ctx),
Source: sourceOnSeat(source),
Check: &link.CheckRequest{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Base: p.Base, Trunk: p.BaseSHA, Ready: p.Ready,
Paths: p.Paths, Beside: beside, Modules: scope.Modules, Dependents: scope.Dependents, New: scope.New,
Manifests: scope.Manifests, Judge: scope.Judge, Builds: builds, Named: namedForTheCheck(ctx, p, builds),
Running: runningBy(ctx)},
}, nil
}
// besideOf is what a check reads beside its own repository, by the directory it is found under: each core repository
// by the module the mesh builds from it, at besideRefs. running is the commit the mesh runs of a module.
func besideOf(entries []inventory.Entry, running func(module string) string,
clone func(inventory.Source) (string, error)) (map[string]link.CheckedOut, error) {
beside := map[string]link.CheckedOut{}
for _, e := range entries {
dir, core := coreModules[e.Manifest.Module]
if !core || e.Provided || e.Source.Repository == "" {
continue
}
url, err := clone(e.Source)
if err != nil {
return nil, err
}
refs := besideRefs(dir, running(e.Manifest.Module))
beside[dir] = link.CheckedOut{Repository: url, Ref: refs[dir]}
if dir == "mesh-controller" {
// The controller's main stays its main: it judges only a check the running controller predates.
beside["mesh-controller-main"] = link.CheckedOut{Repository: url, Ref: refs["mesh-controller-main"]}
if e.Source.Seat != "" {
for _, sibling := range []string{"mesh-lab", "mesh-sdk"} {
if url, err := clone(inventory.Source{Seat: e.Source.Seat, Repository: siblingOf(e.Source.Repository,
sibling)}); err == nil {
beside[sibling] = link.CheckedOut{Repository: url, Ref: refs[sibling]}
}
}
}
}
}
return beside, nil
}
// trunkOf is the branch a module's source follows: main when it records none, or records a commit.
func trunkOf(s inventory.Source) string {
if t := strings.TrimSpace(followedBranch(s.Ref)); t != "" {
return t
}
return "main"
}
// sourceName is the name of a recorded repository, a path on the git seat or a URL: novox/mesh-host → mesh-host.
func sourceName(repository string) string {
parts := strings.Split(strings.Trim(strings.TrimSuffix(repository, ".git"), "/"), "/")
if len(parts) < 2 {
return ""
}
return parts[len(parts)-1]
}
// namedForTheCheck are the builds earlier checks of the same pull request at the same head named, read once for the
// check (novox/hq issue 703): asked again because only the trunk moved, the check keeps what they built of a key
// still the same. Another pull request's build, or one of an earlier head, is never kept. None when they cannot be
// read, and the check builds, as before.
func namedForTheCheck(ctx context.Context, p link.PullUpdated, builds []link.CheckBuild) []link.CheckedBuild {
if len(builds) == 0 || p.Commit == "" {
return nil
}
pull := link.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number}
read, err := checkedBuildsOf(ctx, []link.PullRef{pull})
if err != nil {
fmt.Printf(" the earlier checks' builds of %s/%s#%d could not be read, so the check builds: %v\n",
p.Owner, p.Repo, p.Number, err)
// empty-on-error: said above; a check handed no named build builds every module, as before issue 703
return nil
}
var named []link.CheckedBuild
for _, b := range read {
if b.Pull == pull && link.SameHead(b.Head, p.Commit) {
named = append(named, b)
}
}
return named
}
// checkBuildsOf are the modules a merge of the change would build, in the walk's order, as the check builds them
// for the walk to take (novox/hq ADR 0350 §1): a module built from the repository checked, at the branch it merges
// into, from the merge the check makes; any other — a module standing on one — from its own repository at the
// branch it follows, as the walk builds it. A module whose build reads the repository checked through a context,
// or that the merge deletes, cannot be built as the walk would: said, and nothing after it is built. A change that
// moves no module builds nothing.
func checkBuildsOf(p link.PullUpdated, scope checkScope, entries []inventory.Entry,
clone func(inventory.Source) (string, error)) []link.CheckBuild {
if len(scope.Reach.Plan.Tiers) == 0 {
return nil
}
m := link.SourceMoved{Owner: p.Owner, Repo: p.Repo, Base: p.Base, CloneURL: p.CloneURL}
byName := map[string]inventory.Entry{}
for _, e := range entries {
byName[e.Manifest.Module] = e
}
skip := map[string]string{}
for _, e := range scope.Reach.Packaging {
skip[e.Manifest.Module] = "its build reads " + p.Owner + "/" + p.Repo + " through a context, which a check " +
"cannot hand the merge"
}
for _, e := range scope.Reach.Deleted {
skip[e.Manifest.Module] = "the merge deletes it"
}
var out []link.CheckBuild
for tier, names := range scope.Reach.Plan.Tiers {
for _, name := range names {
e, known := byName[name]
b := link.CheckBuild{Module: name, Tier: tier, Skip: skip[name]}
switch {
case !known:
b.Skip = "the catalogue does not hold it"
case b.Skip != "":
case sourceIs(e.Source, m):
b.Merged, b.Path = true, e.Source.Path
default:
url, err := clone(e.Source)
if err != nil {
b.Skip = "its repository cannot be cloned: " + err.Error()
break
}
b.Repository, b.Path, b.Ref = url, e.Source.Path, followedBranch(e.Source.Ref)
}
out = append(out, b)
}
}
return out
}
// besideRefs is the ref each repository is cloned at beside a check, by the directory it is found under:
// a core repository at the commit the mesh runs of the module built from it (`running`) — but the
// catalogue, whose checkout beside is what tests read its files from, at its main, what the next merge
// builds from; and beside the controller its main, for a judge the running controller predates, and the
// lab's main, whose replays every check runs. **One rule, read by the check the controller asks for and by
// the facts snapshot** (Facts.Beside), so a check run by hand clones what the build seat clones. Beside the
// controller also the SDK, checked out at the commit the running controller's go.mod pins (sdkPinned), not
// one a desktop holds (novox/hq issue 449). The checkout only places the clone: the conformance test reads the
// fixtures at the pin of the tree under check, from the clone's history, so a pull request moving the SDK is
// judged against the SDK it moves to (internal/link/conformance_test.go).
func besideRefs(dir, running string) map[string]string {
switch dir {
case "mesh-catalog":
return map[string]string{dir: "main"}
case "mesh-controller":
return map[string]string{dir: running, "mesh-controller-main": "main", "mesh-lab": "main",
"mesh-sdk": sdkPinned()}
}
return map[string]string{dir: running}
}
// sdkModule is the Go module of the SDK the controller is built against.
const sdkModule = "git.novox.be/novox/mesh-sdk/go"
// pseudoCommit is the commit a Go pseudo-version names: v0.1.11-0.20261009143344-f047d0a4a970 → f047d0a4a970.
var pseudoCommit = regexp.MustCompile(`-([0-9a-f]{12})$`)
// sdkPinned is the ref of the SDK repository this controller was built from, as its go.mod pins it and its
// build records it: a pseudo-version's commit, or a release's tag (the SDK tags its Go module under go/).
// "main" only for a binary that records no SDK version — a local replace — which a running controller is
// not (novox/hq issue 449).
func sdkPinned() string {
info, ok := debug.ReadBuildInfo()
if !ok {
return "main"
}
for _, dep := range info.Deps {
if dep.Path != sdkModule {
continue
}
if dep.Replace != nil {
dep = dep.Replace
}
if m := pseudoCommit.FindStringSubmatch(dep.Version); m != nil {
return m[1]
}
if strings.HasPrefix(dep.Version, "v") {
return "go/" + dep.Version
}
}
return "main"
}
// siblingOf is another repository of the same owner: novox/mesh-controller → novox/mesh-lab.
func siblingOf(repository, name string) string {
if cut := strings.LastIndex(repository, "/"); cut >= 0 {
return repository[:cut+1] + name
}
return name
}
// sourceOnSeat is a source's seat form, nil for one on no seat.
func sourceOnSeat(s inventory.Source) *link.SourceOnSeat {
if s.Seat == "" {
return nil
}
return &link.SourceOnSeat{Seat: s.Seat, Repository: s.Repository}
}
// checkEvents is where the serving controller says a check's verdict; nil in a command.
var checkEvents link.Bus
// maxCheckReport is how much of a check's report travels in its verdict: enough for the failures and
// the machines, never a log.
const maxCheckReport = 60 << 10
// checked says a merge check's verdict as the controller's `checked`. Nothing is recorded or
// registered: a check builds nothing (issue 240's rule for a dry run, kept for a check).
func checked(ctx context.Context, result link.BuildResult) {
// **A check stopped by its key says nothing** (novox/hq ADR 0406 §4, issue 974, review 2): it stopped because the head or
// the trunk commit moved on, and the newer request will judge. Said as an error it would name the live head — red
// statuses on the forge, a delivery refused — and written to the key it would stand in the newer request's way.
if result.Failed == link.SupersededByKey {
fmt.Printf("%s: stopped, its pull request's key moved on; the newer check will judge\n", result.ID)
return
}
sayChecked(ctx, checkedOf(result))
}
// checkedOf is what a check's outcome says: each layer, and an error — never a pass — for a check that
// could not run.
func checkedOf(result link.BuildResult) link.Checked {
c := link.Checked{ID: result.ID, On: result.On, Commit: result.Ref}
if result.Checked != nil {
c.Owner, c.Repo, c.Number = result.Checked.Owner, result.Checked.Repo, result.Checked.Number
}
switch {
case result.Check != nil:
c.Verdict, c.Summary, c.Report = result.Check.Verdict, result.Check.Summary, result.Check.Report
c.Gate, c.RepoCheck = result.Check.Gate, result.Check.RepoCheck
// What was judged (novox/hq ADR 0323 §3): the head merged into the trunk commit, and the commits of the
// repositories read beside it.
c.Trunk, c.Beside, c.Took = result.Check.Trunk, result.Check.Beside, result.Check.Took
c.Built = result.Check.Built
c.Validator = result.Check.Validator
c.Siblings, c.SiblingsUnknown, c.JudgeFrom = result.Check.Siblings, result.Check.SiblingsUnknown, result.Check.JudgeFrom
if c.Gate == nil {
// A build seat from before the layers: its verdict is the gate's.
c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
}
case result.Failed != "":
// The check could not run: an error, never read as a pass — on both layers it was asked for.
c.Verdict, c.Summary = "error", "the check could not run: "+firstLine(result.Failed)
default:
c.Verdict, c.Summary = "error", "the build seat answered the check with no verdict"
}
if c.Verdict == "" {
c.Verdict = "error"
}
if result.Check == nil {
c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
c.RepoCheck = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
}
// **A group member's own check has no gate of its own** (novox/hq ADR 0403 clause 1): what ran was the repository's
// check alone, and its gate layer is "composed", never a pass. A failure or an error stays what it was.
if result.Checked != nil && result.Checked.Composed && result.Check != nil && c.Gate != nil &&
(c.Gate.Verdict == "pass" || c.Gate.Verdict == "warning") {
composedGate(&c)
}
if result.Checked != nil && c.Gate != nil && len(c.Gate.Modules) == 0 {
c.Gate.Modules = append(append([]string{}, result.Checked.Modules...), prefixedAll("new:", result.Checked.New)...)
}
if result.Checked != nil && c.Gate != nil && len(c.Gate.Dependents) == 0 {
c.Gate.Dependents = result.Checked.Dependents
}
if result.Checked != nil {
c.Plan = result.Checked.Plan
if c.Plan != nil && c.Trunk != "" {
// The kept delivery plan records the trunk commit with its other inputs (ADR 0323 §3).
plan := *c.Plan
plan.Trunk = c.Trunk
c.Plan = &plan
}
// A delivery group's composed check (novox/hq ADR 0239): every head it judged, this one first.
if g := result.Checked; g.Group != "" {
c.Group = g.Group
c.Members = append(c.Members, link.CheckedMember{Owner: g.Owner, Repo: g.Repo, Number: g.Number,
Commit: result.Ref, Trunk: c.Trunk})
for _, m := range g.Members {
c.Members = append(c.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo, Number: m.Number,
Commit: m.Ref, Trunk: memberTrunk(result.Check, m)})
}
}
}
for _, l := range []*link.CheckLayer{c.Gate, c.RepoCheck} {
if l != nil && l.Verdict == "" {
l.Verdict = "error"
}
}
// **A check run again says so** (novox/hq issue 594). The summaries are the lines the pull request
// shows, clipped by the forge to a few words, so they get only a short marker after the verdict's own
// words; the report gets the whole sentence. Layers copied, so the build's own outcome is left as it came.
if mark := link.RunMarker(result.Delivery); mark != "" {
c.Summary += " " + mark
for _, l := range []**link.CheckLayer{&c.Gate, &c.RepoCheck} {
if *l != nil {
layer := **l
layer.Summary += " " + mark
*l = &layer
}
}
c.Report = link.Restarted(result.Delivery) + "\n\n" + c.Report
}
if len(c.Report) > maxCheckReport {
c.Report = "…" + c.Report[len(c.Report)-maxCheckReport:]
}
return c
}
// memberTrunk is the trunk commit the build seat merged one of a group's heads into (novox/hq ADR 0323 §6), empty
// when it said none.
func memberTrunk(outcome *link.CheckOutcome, m link.GroupMember) string {
if outcome == nil {
return ""
}
for _, j := range outcome.Members {
if strings.EqualFold(j.Owner, m.Owner) && strings.EqualFold(j.Repo, m.Repo) && strings.HasPrefix(m.Ref, j.Commit) {
return j.Trunk
}
}
return ""
}
func prefixedAll(prefix string, items []string) []string {
out := make([]string, 0, len(items))
for _, i := range items {
out = append(out, prefix+i)
}
return out
}
// sayChecked says a merge check's verdict on the bus, where the forge's holder hears it.
func sayChecked(ctx context.Context, c link.Checked) {
repo := "none"
if c.RepoCheck != nil {
repo = strings.ToUpper(c.RepoCheck.Verdict) + " — " + c.RepoCheck.Summary
}
judged := ""
if c.Trunk != "" {
judged = " merged into trunk " + c.Trunk
}
fmt.Printf("%s: %s/%s#%d at %.8s%s checked on %s: gate %s — %s; repository %s\n", c.ID, c.Owner, c.Repo, c.Number,
c.Commit, judged, orSomewhere(c.On), strings.ToUpper(c.Verdict), c.Summary, repo)
if checkEvents == nil {
return
}
body, err := json.Marshal(c)
if err != nil {
return
}
stating, stop := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
defer stop()
if err := checkEvents.PublishSeatEvent(stating, link.MeshControllerSeat, link.KeyChecked, body); err != nil {
fmt.Printf("%s: the verdict could not be said, so the pull request is not told it: %v\n", c.ID, err)
}
}
@@ -1,56 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
func TestACheckStoppedByItsKeyIsAnnouncedAsNoVerdict(t *testing.T) {
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
if err := broker.AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
legacy, err := js.Conn().JetStream()
if err != nil {
t.Fatal(err)
}
was := checkEvents
checkEvents = link.OverNATS{Conn: js.Conn(), JS: legacy}
t.Cleanup(func() { checkEvents = was })
heard := make(chan *nats.Msg, 4)
sub, err := js.Conn().ChanSubscribe(link.SeatEventSubject(link.MeshControllerSeat, link.KeyChecked), heard)
if err != nil {
t.Fatal(err)
}
defer func() { _ = sub.Unsubscribe() }()
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog", Number: 3}
checked(t.Context(), link.BuildResult{ID: "stopped", Ref: "abc", Checked: asked, Failed: link.SupersededByKey})
checked(t.Context(), link.BuildResult{ID: "broken", Ref: "abc", Checked: asked, Failed: "the facts snapshot cannot be read"})
select {
case m := <-heard:
var c link.Checked
_ = json.Unmarshal(m.Data, &c)
if c.ID != "broken" || c.Verdict != "error" {
t.Fatalf("the first verdict said is %+v; want only the broken check's error", c)
}
case <-time.After(5 * time.Second):
t.Fatal("nothing was said")
}
select {
case m := <-heard:
t.Fatalf("a second verdict was said: %s", m.Data)
case <-time.After(300 * time.Millisecond):
}
}
-97
View File
@@ -1,97 +0,0 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"fmt"
"testing"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// Acceptance 5 of milestone 2 (novox/hq ADR 0406 §3, issue 974), live, with the store and a real bus: two merges into one
// repository within a minute, with two open pull requests, leave each pull request with exactly one queued check, by the
// stream's rule. Each merge moves the trunk, and mesh-delivery asks the controller to check both open pull requests again
// against it (`delivery check`, which is the controller's own path for a head announced); nothing moves a request, nothing
// asks the forge, and the queue is read back whole.
func TestTwoMergesWithinAMinuteLeaveEachOpenPullRequestOneQueuedCheck(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
js, _ := aBuildQueue(t)
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
holdTheDeliverySeat(t, open)
was := askDeliveryOwner
t.Cleanup(func() { askDeliveryOwner = was })
askDeliveryOwner = func(_ context.Context, _ *nats.Conn, verb string, _ map[string]any) (json.RawMessage, error) {
if verb != "groups" {
t.Fatalf("asked the owner %s", verb)
}
return json.RawMessage(`[]`), nil
}
// The repository the two pull requests are into, as a module the mesh builds from it.
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
heads := map[int]string{7: "7777777777777777777777777777777777777777", 8: "8888888888888888888888888888888888888888"}
check := func(number int, trunk string) {
t.Helper()
p := orderMember{ID: fmt.Sprintf("novox/mesh-controller@%.12s", heads[number]), Repository: "novox/mesh-controller",
Base: "main", Head: heads[number], Number: number, Paths: []string{"main.go"}, ModuleDirsSaid: true,
BaseSHA: trunk, Ready: number == 8}.pull()
p.MergeCheck, p.MergeCheckSaid = true, true
if err := (following{open: open, oneShot: true}).PullUpdated(ctx, p); err != nil {
t.Fatal(err)
}
}
// Announced against the trunk as it stands, then asked again after each of two merges.
for _, trunk := range []string{"1111111111111111111111111111111111111111", "2222222222222222222222222222222222222222",
"3333333333333333333333333333333333333333"} {
check(7, trunk)
check(8, trunk)
}
q, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil {
t.Fatal(err)
}
seen := map[int]string{}
for _, a := range q.Asks {
c := a.Request.Check
if c == nil {
t.Fatalf("%s is no check", a.ID)
}
if _, twice := seen[c.Number]; twice {
t.Errorf("pull request %d has two queued checks", c.Number)
}
seen[c.Number] = c.Trunk
if a.State != link.AskWaiting {
t.Errorf("%s is %s", a.ID, a.State)
}
if want := "3333333333333333333333333333333333333333"; c.Trunk != want {
t.Errorf("pull request %d is queued against %.8s; want the newest trunk %.8s", c.Number, c.Trunk, want)
}
}
if len(seen) != 2 {
t.Fatalf("queued checks for %v; want one for each of the two open pull requests", seen)
}
// By the streams' own count: the ready one on its stream, the other on its.
for stream, want := range map[string]uint64{broker.ChecksReadyStream: 1, broker.ChecksOtherStream: 1} {
info, err := js.Context().StreamInfo(stream)
if err != nil {
t.Fatal(err)
}
if info.State.Msgs != want {
t.Errorf("%s holds %d request(s); want %d", stream, info.State.Msgs, want)
}
}
}
-293
View File
@@ -1,293 +0,0 @@
package main
import (
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **The mesh's module graph decides what a pull request's check runs**, not the repository (novox/hq
// ADR 0237 as amended): a change is mapped onto the graph by the rule a merge is (issue 278), the gate
// runs when it touches a module — a new one included — and a repository that is the mesh's and touches
// none still has its own merge-check.sh run.
func TestThePullRequestIsMappedOntoTheModuleGraph(t *testing.T) {
const catalogue = "http://forge.internal:20000/novox/mesh-catalog.git"
const controller = "http://forge.internal:20000/novox/mesh-controller.git"
const host = "http://forge.internal:20000/novox/mesh-host.git"
photos := fromRepo("photos", "http://forge.internal:20000/novox/photos.git", "")
photos.Source.Ref = "nox-mesh"
snake := fromRepo("snake", "jschoubben/snake", "")
snake.Source.Seat = "git"
entries := []inventory.Entry{
fromRepo("gitea", catalogue, "modules/gitea"),
fromRepo("keycloak", catalogue, "modules/keycloak"),
fromRepo("nats", catalogue, "modules/nats"),
fromRepo("mesh-controller", controller, ""),
fromRepo("mesh-host", host, ""),
photos, snake,
}
pull := func(owner, repo, base string, paths []string, dirs ...string) link.PullUpdated {
return link.PullUpdated{Owner: owner, Repo: repo, Base: base, Commit: "abc", Paths: paths,
ModuleDirs: dirs, ModuleDirsSaid: true}
}
for _, c := range []struct {
what string
p link.PullUpdated
modules, new, manifest string
mesh bool
judge string
}{
{"one module's own files", pull("novox", "mesh-catalog", "main", []string{"modules/gitea/index.ts"}, "modules/gitea"),
"gitea", "", "modules/gitea/module.json", true, ""},
{"a file no module's build reads touches no module (issue 280)",
pull("novox", "mesh-catalog", "main", []string{"merge-check.sh", "README.md"}), "", "", "", true, ""},
{"files the announcer could not list: everything built from it",
link.PullUpdated{Owner: "novox", Repo: "mesh-catalog", Base: "main", Commit: "abc", PathsTruncated: true,
Paths: []string{"README.md"}}, "gitea,keycloak,nats", "",
"modules/gitea/module.json,modules/keycloak/module.json,modules/nats/module.json", true, ""},
{"a new module, said by the head", pull("novox", "mesh-catalog", "main",
[]string{"modules/newmod/index.ts", "modules/newmod/module.json"}, "modules/newmod"),
"", "modules/newmod", "modules/newmod/module.json", true, ""},
{"the controller judges itself", pull("novox", "mesh-controller", "main", []string{"cmd/x.go"}),
"mesh-controller", "", "module.json", true, "self"},
{"the node-engine is judged with its validator", pull("novox", "mesh-host", "main", []string{"validate/v.go"}),
"mesh-host", "", "module.json", true, "validator"},
{"the core's owner, no module: the mesh's, its own check alone", pull("novox", "hq", "main", []string{"README.md"}),
"", "", "", true, ""},
{"a branch nothing is built from: the mesh's repository, no module", pull("novox", "photos", "master",
[]string{"server/x.js"}), "", "", "", true, ""},
{"the branch a module is built from", pull("novox", "photos", "nox-mesh", []string{"server/x.js"}),
"photos", "", "module.json", true, ""},
{"a repository on the forge's seat", pull("jschoubben", "snake", "main", []string{"index.html"}),
"snake", "", "module.json", true, ""},
{"a repository of nobody's, touching nothing", pull("someone", "dotfiles", "main", []string{"x"}),
"", "", "", false, ""},
{"a repository adding a module at its root", pull("someone", "newapp", "main", []string{"module.json", "x.js"}),
"", ".", "module.json", true, ""},
} {
s := pullScope(c.p, entries, nil, nil)
got := []string{strings.Join(s.Modules, ","), strings.Join(s.New, ","), strings.Join(s.Manifests, ",")}
want := []string{c.modules, c.new, c.manifest}
for i, what := range []string{"modules", "new", "manifests"} {
if got[i] != want[i] {
t.Errorf("%s: %s %q, wanted %q", c.what, what, got[i], want[i])
}
}
if s.Mesh != c.mesh || s.Judge != c.judge {
t.Errorf("%s: the mesh's %v judged by %q, wanted %v by %q", c.what, s.Mesh, s.Judge, c.mesh, c.judge)
}
if s.gated() != (c.modules != "" || c.new != "") {
t.Errorf("%s: gated %v", c.what, s.gated())
}
}
}
// A module whose build packages another repository's source is touched by a change to it.
func TestAPullRequestTouchesWhatPackagesItsRepository(t *testing.T) {
entries := []inventory.Entry{fromRepo("node-tools", "http://forge.internal:20000/novox/mesh-tools.git", "node-tools")}
read := map[string][]inventory.ReadRepository{"node-tools": {{Repository: "http://forge.internal:20000/novox/mesh-sdk.git"}}}
s := pullScope(link.PullUpdated{Owner: "novox", Repo: "mesh-sdk", Base: "main", Commit: "abc", Paths: []string{"go/x.go"}}, entries, read, nil)
if strings.Join(s.Modules, ",") != "node-tools" || len(s.Manifests) != 0 {
t.Fatalf("a change to what node-tools packages touched %v (manifests %v)", s.Modules, s.Manifests)
}
}
// Each layer is said; a check that could not run is an error on both, never a pass; a build seat from
// before the layers is read as the gate.
func TestAChecksLayersAreEachSaidAndAnErrorIsNeverAPass(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog", Number: 3, Modules: []string{"gitea"}}
c := checkedOf(link.BuildResult{ID: "b", Ref: "abc", Checked: asked, Failed: "the facts snapshot cannot be read"})
if c.Verdict != "error" || c.Gate == nil || c.Gate.Verdict != "error" || c.RepoCheck == nil || c.RepoCheck.Verdict != "error" {
t.Fatalf("a check that could not run said %+v", c)
}
if strings.Join(c.Gate.Modules, ",") != "gitea" {
t.Errorf("the gate names %v", c.Gate.Modules)
}
c = checkedOf(link.BuildResult{ID: "b", Ref: "abc", Checked: asked, Check: &link.CheckOutcome{Verdict: "warning", Summary: "wide"}})
if c.Gate == nil || c.Gate.Verdict != "warning" || c.RepoCheck != nil {
t.Fatalf("an outcome without layers said %+v", c)
}
c = checkedOf(link.BuildResult{ID: "b", Ref: "abc", Checked: asked, Check: &link.CheckOutcome{Verdict: "pass",
Gate: &link.CheckLayer{Verdict: "pass"}, RepoCheck: &link.CheckLayer{Verdict: "fail", Summary: "its merge-check.sh failed"}}})
if c.RepoCheck.Verdict != "fail" || c.Gate.Verdict != "pass" {
t.Fatalf("the layers said %+v / %+v", c.Gate, c.RepoCheck)
}
}
// **The check asks the planner, never a mapping of its own** (novox/hq ADR 0238): what a pull request
// reaches is reachOfMerge's answer — the same that plans a merge — so a file at the root touches no
// module in both, and what stands on a touched module is named as its dependents in both.
func TestAPullRequestReachesWhatAMergeOfItWouldPlan(t *testing.T) {
const catalogue = "http://forge.internal:20000/novox/mesh-catalog.git"
nats := fromRepo("nats", catalogue, "modules/nats")
nats.Source.BuiltFrom = "old"
gitea := fromRepo("gitea", catalogue, "modules/gitea")
gitea.Source.BuiltFrom = "old"
tools := fromRepo("node-tools", "http://forge.internal:20000/novox/mesh-tools.git", "node-tools")
entries := []inventory.Entry{nats, gitea, tools}
// node-tools stands on the bus's image; a code edge, so a plan takes it along.
edges := []inventory.Edge{{From: "node-tools", To: "nats", Kind: inventory.EdgeStandsOn}}
read := map[string][]inventory.ReadRepository{}
for _, c := range []struct {
what string
paths []string
moved, dependents string
width int
unread string
}{
{"a file at the root, read by no build", []string{"merge-check.sh"}, "", "", 0, "merge-check.sh"},
{"the bus's own directory", []string{"modules/nats/Dockerfile"}, "nats", "node-tools", 2, ""},
{"both, and a README", []string{"modules/gitea/index.ts", "modules/nats/x", "README.md"}, "gitea,nats", "node-tools", 3, "README.md"},
} {
p := link.PullUpdated{Owner: "novox", Repo: "mesh-catalog", Base: "main", Commit: "head", Paths: c.paths}
s := pullScope(p, entries, read, edges)
m := link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main", Commit: "head", Paths: c.paths}
r := reachOfMerge(m, entries, read, edges)
if got := strings.Join(s.Modules, ","); got != c.moved || got != strings.Join(r.Moved(), ",") {
t.Errorf("%s: the check reaches %q, the planner %v, wanted %q", c.what, got, r.Moved(), c.moved)
}
if got := strings.Join(s.Dependents, ","); got != c.dependents {
t.Errorf("%s: dependents %q, wanted %q", c.what, got, c.dependents)
}
if s.Width != c.width || s.Width != len(r.Plan.Modules) {
t.Errorf("%s: a merge would build %d, the planner says %d, wanted %d", c.what, s.Width, len(r.Plan.Modules), c.width)
}
if got := strings.Join(s.Unread, ","); got != c.unread {
t.Errorf("%s: unread %q, wanted %q", c.what, got, c.unread)
}
}
// A repository whose build another module's recipe packages: that module is reached through the
// planner's `read`, and what stands on it after it.
read["gitea"] = []inventory.ReadRepository{{Repository: "http://forge.internal:20000/novox/mesh-sdk.git"}}
s := pullScope(link.PullUpdated{Owner: "novox", Repo: "mesh-sdk", Base: "main", Commit: "head",
Paths: []string{"src/index.ts"}}, entries, read, edges)
if strings.Join(s.Modules, ",") != "gitea" || len(s.Manifests) != 0 {
t.Fatalf("a change to the source gitea packages reaches %v (manifests %v)", s.Modules, s.Manifests)
}
}
// **Only a commit on the trunk is published** (novox/hq ADR 0238): a build of a commit off its repository's
// default branch — a pull request's head, a branch built by hand, a rebuild or replay of one — is recorded
// and never registered, so nothing can send it; a check or a dry run never is either.
func TestABuildOffTheTrunkIsNeverPublished(t *testing.T) {
on := link.BuildResult{ID: "b", Commit: "abc", Trunk: "main", OnTrunk: true}
if err := publishable(on, ""); err != nil {
t.Errorf("a build on the trunk was refused: %v", err)
}
off := link.BuildResult{ID: "b", Commit: "abc", Trunk: "main"}
if err := publishable(off, ""); !errors.Is(err, errOffTheTrunk) || !strings.Contains(err.Error(), "main") {
t.Errorf("a build off the trunk was let through: %v", err)
}
for _, r := range []link.BuildResult{
{ID: "c", Trunk: "main", OnTrunk: true, Check: &link.CheckOutcome{Verdict: "pass"}},
{ID: "d", Trunk: "main", OnTrunk: true, Checked: &link.CheckRequest{}},
{ID: "e", Trunk: "main", OnTrunk: true, DryRun: true},
} {
if publishable(r, "") == nil {
t.Errorf("%s, a check or a dry run, was publishable", r.ID)
}
}
// A module that follows a branch other than the default: that branch is its trunk, and the default
// is not.
follows := link.BuildResult{ID: "g", Commit: "abc", Trunk: "master", Branches: []string{"nox-mesh"}}
if err := publishable(follows, "nox-mesh"); err != nil {
t.Errorf("a commit on the branch a module follows was refused: %v", err)
}
if err := publishable(link.BuildResult{ID: "h", Commit: "abc", Trunk: "master", OnTrunk: true,
Branches: []string{"master"}}, "nox-mesh"); err == nil {
t.Error("a commit on the default but not on the branch the module follows was published")
}
if err := publishable(link.BuildResult{ID: "i", Commit: "abc", Trunk: "main", Branches: []string{"feat/x"}}, ""); err == nil {
t.Error("a feature branch's commit was published")
}
// A build seat older than the rule says nothing: let through and said, so the rule can reach the mesh.
if err := publishable(link.BuildResult{ID: "f", Commit: "abc"}, ""); err != nil {
t.Errorf("a build seat that said nothing was refused: %v", err)
}
}
// **A verdict names the trunk commit it judged** (novox/hq ADR 0323 §3, issue 341): the controller's `checked`
// carries the head and the trunk commit the build seat merged it into, the commits of the repositories read beside
// it, how long it took, and the kept delivery plan records the trunk commit with its other inputs. A group's
// composed check names, per member, the trunk commit that head was merged into (§6).
func TestAVerdictNamesTheTrunkCommitItJudged(t *testing.T) {
plan := &link.ChangePlan{Repository: "novox/mesh-host", Base: "main", Head: "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
Summary: "builds mesh-host"}
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 66, Base: "main", Plan: plan}
c := checkedOf(link.BuildResult{ID: "b", Ref: "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", Checked: asked,
Check: &link.CheckOutcome{Verdict: "pass", Took: "4m12s", Head: "dfb1415aaaaa", Trunk: "d28ca95bbbbb",
Gate: &link.CheckLayer{Verdict: "pass"}, RepoCheck: &link.CheckLayer{Verdict: "pass"},
Beside: map[string]string{"mesh-catalog": "0123456789ab", "mesh-lab": "ba9876543210"}}})
if c.Trunk != "d28ca95bbbbb" || c.Commit != "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" {
t.Fatalf("the verdict names head %s and trunk %q", c.Commit, c.Trunk)
}
if c.Beside["mesh-catalog"] != "0123456789ab" || c.Beside["mesh-lab"] != "ba9876543210" {
t.Fatalf("the verdict names the repositories read beside it as %v", c.Beside)
}
if c.Took != "4m12s" {
t.Fatalf("the verdict says it took %q", c.Took)
}
if c.Plan == nil || c.Plan.Trunk != "d28ca95bbbbb" || c.Plan.Head != plan.Head {
t.Fatalf("the kept plan records %+v", c.Plan)
}
if plan.Trunk != "" {
t.Fatal("the plan the controller asked with was changed: the verdict's is a copy")
}
group := &link.CheckRequest{Owner: "novox", Repo: "mesh-controller", Number: 7, Base: "main", Group: "feat/x",
Members: []link.GroupMember{{Owner: "novox", Repo: "mesh-host", Number: 8, Ref: "cccccccccccc0000", Base: "main"}}}
c = checkedOf(link.BuildResult{ID: "g", Ref: "aaaaaaaaaaaa0000", Checked: group, Check: &link.CheckOutcome{
Verdict: "pass", Head: "aaaaaaaaaaaa", Trunk: "111111111111", Gate: &link.CheckLayer{Verdict: "pass"},
Members: []link.CheckedMember{{Owner: "novox", Repo: "mesh-host", Commit: "cccccccccccc", Trunk: "222222222222"}}}})
if len(c.Members) != 2 || c.Members[0].Trunk != "111111111111" || c.Members[1].Trunk != "222222222222" ||
c.Members[1].Commit != "cccccccccccc0000" {
t.Fatalf("the composed check names its members %+v", c.Members)
}
}
// **The merge made for a check is never registered** (novox/hq ADR 0323 §2): a build of it — a commit on no branch
// of the forge — is refused, naming the trunk. The rule of ADR 0238 §1 stands unchanged; this holds it for the
// merge commit a check makes.
func TestABuildOfACheckMergeIsRefusedNamingTheTrunk(t *testing.T) {
merge := link.BuildResult{ID: "b", Commit: "4b5c6d7e8f90aabbccddeeff0011223344556677", Trunk: "main"}
err := publishable(merge, "")
if !errors.Is(err, errOffTheTrunk) || !strings.Contains(err.Error(), "not on main") {
t.Fatalf("a build of a check's merge was answered %v", err)
}
}
// **A verdict no build judges names its trunk commit too** (novox/hq ADR 0323 §3): a change touching no module of a
// repository with no merge-check.sh (novox/mesh-tools-go) is answered by the controller itself. Naming no trunk, the
// merge tool refused it, asked again, got the same verdict, and refused for ever.
func TestADirectVerdictNamesTheTrunkTheForgeAnnounced(t *testing.T) {
p := link.PullUpdated{Owner: "novox", Repo: "mesh-tools-go", Number: 3, Base: "main", Commit: "1a2b3c4d5e6f7a8b9c0d",
BaseSHA: "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b"}
c := directVerdict(p, link.ChangePlan{Repository: "novox/mesh-tools-go", Base: "main", Head: p.Commit})
if c.Trunk != "7a8b9c0d1e2f" || c.Plan == nil || c.Plan.Trunk != "7a8b9c0d1e2f" {
t.Fatalf("the direct verdict names trunk %q, its plan %+v", c.Trunk, c.Plan)
}
if c.Verdict != "pass" || c.Commit != p.Commit {
t.Fatalf("the direct verdict is %+v", c)
}
if c := directVerdict(link.PullUpdated{Owner: "novox", Repo: "x", Commit: "abc"}, link.ChangePlan{}); c.Trunk != "" {
t.Fatalf("a forge that said no base head gave the trunk %q", c.Trunk)
}
}
// A head checked again through mesh-delivery (`delivery check --members`, a recheck) reaches the same direct verdict:
// it names the base head the forge last announced, kept on the member, or the merge tool would refuse it for ever.
func TestARecheckedHeadsDirectVerdictNamesItsTrunk(t *testing.T) {
m := orderMember{ID: "novox/mesh-tools-go@1a2b3c4d5e6f", Repository: "novox/mesh-tools-go", Base: "main",
Head: "1a2b3c4d5e6f7a8b9c0d", Number: 3, BaseSHA: "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b"}
if c := directVerdict(m.pull(), link.ChangePlan{}); c.Trunk != "7a8b9c0d1e2f" {
t.Fatalf("a rechecked head's direct verdict names trunk %q", c.Trunk)
}
}
// A check stopped by its pull request's key moving on says no verdict (issue 974, review 2): an error said for it would name the
// live head, and the newer request is the one that judges. A check that failed for any other reason still says its error.
-276
View File
@@ -1,276 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"os"
"time"
"github.com/novox/mesh-controller/internal/artifacts"
"github.com/novox/mesh-controller/internal/inventory"
)
// Letting the artifact store go of what the mesh no longer keeps (novox/hq ADR 0189, issue 108).
//
// **Run where the records change.** A build is the moment new bytes landed in the store and the
// moment the keep set moved, so it is the moment to say what may go — and it needs no timer of
// its own. Reclaiming the bytes is the store's own nightly step; this only decides.
//
// **And run when a person asks** (novox/hq ADR 0251): the `collect` verb is the same sweep, with
// bounds of its own. One implementation, so what a person runs is exactly what a build runs.
//
// Never fatal to a build. The build succeeded, the module is registered, and a store that could
// not be reached is a thing to say rather than a reason to undo any of that. The next build asks
// again, and the references it could not collect are still uncollected, so nothing is lost by
// having failed.
// sweepBounds is how much one sweep may ask of the store.
type sweepBounds struct {
// most is how many artifacts it lets go of at most.
most int
// budget is the longest it keeps its caller waiting.
budget time.Duration
// platforms is whether an index is let go of with its own platform manifests (novox/hq ADR 0257).
// Only a sweep a person confirmed, after its dry run listed what stays and names a platform: the
// records cannot see an unrecorded index, or a copy in progress, naming the same platform, and the
// store's tools can.
platforms bool
}
// afterBuild are the bounds of the sweep inside somebody's build.
//
// **Bounded, because this runs inside somebody's build.** The first sweep of a mesh that has never
// collected has the whole history to get through, and a person waiting on `build` should not pay for
// it. What is left over is simply offered again next time — builds are frequent, and the point is
// that the store stops growing, not that it empties tonight.
var afterBuild = sweepBounds{most: mostPerSweep, budget: sweepBudget}
// sweepResult is what one sweep did.
type sweepResult struct {
// HoldersWritten is how many kept archives the store now holds by a manifest it did not before.
HoldersWritten int
// Missing is how many kept archives the store does not have at all.
Missing int
// Held is whether every kept archive could be asked to be held; false stops the sweep before it
// lets anything go.
Held bool
// LetGo is what the store let go of, and the record now says collected.
LetGo []string
// Skipped is how many references the sweep will not address (novox/hq issue 226).
Skipped int
// Indexes is how many eligible indexes a sweep a person did not confirm left for one that is: an
// index goes only with its platform manifests (novox/hq ADR 0257 §4).
Indexes int
// Left is how many eligible references were not asked about this time.
Left int
// Bounded is whether it stopped at its bounds rather than at a refusal.
Bounded bool
// Stopped says why the sweep stopped before the end; empty when it did not.
Stopped string
}
// sweep holds every kept archive, then asks the store to let go of each reference in order, records
// what it let go of, and says what it did.
func sweep(ctx context.Context, inv *inventory.Inventory, store artifacts.Store, references, kept []string,
bounds sweepBounds) sweepResult {
within, stop := context.WithTimeout(ctx, bounds.budget)
defer stop()
var r sweepResult
// **Hold before letting go** (novox/hq issue 253). The store's collector keeps only what a
// manifest names, and archives were published as bare blobs, so every kept archive is first
// held by its manifest — which backfills the ones published before holders, a few at a time
// as builds come, and is two HEADs each once done. A kept archive that could not be held stops
// the sweep before it deletes anything: "everything kept is held" is the precondition the
// collector's safety rests on, and a store refusing a hold would refuse the deletes too.
// **An index goes with its platform manifests only when a person confirmed it** (novox/hq ADR
// 0257), and a kept index keeps its own: which platform manifests the kept indexes name is read
// from the store for every repository the sweep will touch, before the first delete. A single
// read that fails stops the sweep before it deletes anything, as a kept archive that cannot be
// held does. The sweep after a build leaves an eligible index for such a collect, below.
store.Spare = nil
if bounds.platforms {
if inv == nil {
r.Stopped = "no records to read which platform manifests a kept index names, so nothing was let go"
r.Left = len(references)
return r
}
images, err := inv.KeptImages(ctx)
if err == nil {
store.Spare, err = store.SpareKeptIndexes(within, images, references)
}
if err != nil {
r.Stopped = fmt.Sprintf("which platform manifests a kept index names could not be read, so nothing was let go: %v", err)
r.Left = len(references)
return r
}
}
wrote, missing, err := holdKept(within, store, kept)
r.HoldersWritten, r.Missing = wrote, missing
if err != nil {
r.Stopped = fmt.Sprintf("not every kept archive could be held, so nothing was let go: %v", err)
r.Left = len(references)
return r
}
r.Held = true
for i, reference := range references {
if i >= bounds.most || within.Err() != nil {
r.Left = len(references) - i
r.Bounded = true
if i >= bounds.most {
r.Stopped = fmt.Sprintf("asked about %d, the most one sweep asks", bounds.most)
} else {
r.Stopped = fmt.Sprintf("ran out of its %s", bounds.budget)
}
break
}
if !bounds.platforms {
// **An index waits for a confirmed collect** (novox/hq ADR 0257 §4). Let go of alone,
// its platform manifests would stay in the store and the record would say collected, so
// no later sweep would offer it again and they would stay for ever. Left eligible, the
// next collect a person confirms takes it with them.
index, err := store.IsIndex(within, reference)
if err != nil && !errors.Is(err, artifacts.Gone) && !errors.Is(err, artifacts.ErrNotOurs) {
r.Stopped = fmt.Sprintf("the artifact store could not say whether %s is an index, so nothing more was asked of it: %v", reference, err)
r.Left = len(references) - i
break
}
if index {
r.Indexes++
continue
}
}
err := store.LetGo(within, reference)
if err == nil || errors.Is(err, artifacts.Gone) {
// Gone is the outcome wanted, already true. Recorded so the next sweep does not ask
// again for ever.
r.LetGo = append(r.LetGo, reference)
continue
}
if errors.Is(err, artifacts.ErrNotOurs) {
// **A fact about this record, so this record is skipped** (novox/hq issue 226). Not
// marked collected — the mesh did not remove it and should not claim to — and not a
// reason to stop, because the store was never asked. One of these at the front of
// the oldest-first order ended every sweep until this.
r.Skipped++
if r.Skipped == 1 {
fmt.Fprintf(os.Stderr, "the sweep will not address %s and went on: %v\n", reference, err)
}
continue
}
// **Stopped at the first refusal by the STORE, not pushed through.** A store that refuses
// one refuses all of them — deletion disabled, the store down, the network gone — so
// going on would be a hundred identical failures and a hundred identical log lines in
// front of whoever was building something.
r.Stopped = fmt.Sprintf("the artifact store kept %s, so nothing more was asked of it: %v", reference, err)
r.Left = len(references) - i
break
}
if len(r.LetGo) > 0 {
// Recorded outside `within`: the deletions happened, and losing the record of them because
// the sweep ran out of budget would mean asking about them again for ever.
if err := inv.MarkCollected(ctx, r.LetGo); err != nil {
r.Stopped = fmt.Sprintf("the store let go of %d artifact(s) and the record of it did not keep: %v",
len(r.LetGo), err)
}
}
return r
}
// collect asks the store to let go of everything the mesh made and no longer keeps, and records
// what it let go of — after a build. Says what it did and what it could not; returns nothing,
// because nothing upstream should branch on it.
func collect(ctx context.Context, inv *inventory.Inventory) {
references, err := inv.ToCollect(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not work out what the artifact store may let go of: %v\n", err)
return
}
kept, err := inv.KeptArchives(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not work out which archives the artifact store keeps: %v\n", err)
return
}
if len(references) == 0 && len(kept) == 0 {
return
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read the catalogue to find the artifact store: %v\n", err)
return
}
// As the mesh reaches it from the network. Empty means the store is not on the network — on a
// mesh being raised it is not yet, and there the store holds one build of anything and has
// nothing to collect.
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil || address == "" {
if err != nil {
fmt.Fprintf(os.Stderr, "could not find the artifact store to collect from: %v\n", err)
}
return
}
r := sweep(ctx, inv, artifacts.Store{Address: address}, references, kept, afterBuild)
if r.HoldersWritten > 0 {
fmt.Fprintf(os.Stderr, "the artifact store now holds %d more kept archive(s) by a manifest\n", r.HoldersWritten)
}
if r.Missing > 0 {
fmt.Fprintf(os.Stderr, "%d archive(s) the mesh keeps are not in the artifact store at all; "+
"`collection` lists them\n", r.Missing)
}
if len(r.LetGo) > 0 {
fmt.Fprintf(os.Stderr, "the artifact store let go of %d artifact(s) the mesh no longer keeps\n", len(r.LetGo))
}
if r.Stopped != "" && !r.Bounded {
fmt.Fprintln(os.Stderr, r.Stopped)
}
if r.Held && r.Left > 0 {
fmt.Fprintf(os.Stderr, "%d more to collect; the next build asks again\n", r.Left)
}
if r.Skipped > 0 {
fmt.Fprintf(os.Stderr, "%d artifact(s) the sweep will not address were skipped\n", r.Skipped)
}
if r.Indexes > 0 {
fmt.Fprintf(os.Stderr, "%d eligible index(es) wait for a collect a person confirms, which takes their platforms too\n", r.Indexes)
}
}
// holdKept holds every kept archive by its manifest, stopping at the first refusal by the store.
// Answers how many holders it wrote and how many kept archives the store does not have.
//
// A missing archive is counted rather than fatal: there is nothing to hold, and that is a fact
// for an operator to read (`collection`), not a reason to stop collecting what is not kept. A
// reference the store cannot be asked about is skipped as the deletion loop skips one
// (novox/hq issue 226).
func holdKept(ctx context.Context, store artifacts.Store, kept []string) (wrote, missing int, err error) {
for _, reference := range kept {
if err := ctx.Err(); err != nil {
return wrote, missing, fmt.Errorf("ran out of time before %s: %w", reference, err)
}
did, err := store.Hold(ctx, reference)
switch {
case err == nil:
if did {
wrote++
}
case errors.Is(err, artifacts.Gone):
missing++
case errors.Is(err, artifacts.ErrNotOurs):
default:
return wrote, missing, fmt.Errorf("holding %s: %w", reference, err)
}
}
return wrote, missing, nil
}
// mostPerSweep is how many artifacts one sweep will ask about. Enough that a mesh building
// several times a day converges within days of this landing; small enough that no single build
// waits on the whole backlog.
const mostPerSweep = 200
// sweepBudget is the longest a sweep will keep a build waiting.
const sweepBudget = 60 * time.Second
-166
View File
@@ -1,166 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"strings"
"github.com/novox/mesh-controller/internal/artifacts"
)
// What the artifact store keeps, whether each kept archive is held, and what the sweep may let go
// (novox/hq issue 253, ADR 0189).
//
// **The question to answer before the store's collector runs for real.** The collector deletes
// every blob no manifest names, and archives were published as bare blobs, so the nightly step
// runs `--dry-run` until every archive the mesh keeps is held by its manifest. The sweep holds
// them as builds come; this says how far that has got — "0 unheld" is the number that lets the
// dry run go.
//
// Reads and changes nothing: each kept archive is asked about with HEADs only. Reached over the
// console through the mesh-controller seat's `command` verb (`collection --json`), which needs no
// new verb in the seat's row.
type collectionReport struct {
// Store is the artifact store as this machine reached it; empty when it is not on the network.
Store string `json:"store"`
// KeptArchives is how many archives the mesh keeps, for either reason.
KeptArchives int `json:"kept_archives"`
// Held is how many of them the store holds by their manifest.
Held int `json:"held"`
// Unheld are the kept archives the collector would delete tonight if it ran for real.
Unheld []string `json:"unheld"`
// Missing are kept archives the store does not have at all.
Missing []string `json:"missing"`
// Unasked is how many could not be asked about, and why the asking stopped.
Unasked int `json:"unasked"`
Stopped string `json:"stopped,omitempty"`
// Eligible is what the sweep may let go of: made by the mesh, kept for no reason, not yet
// collected — split by kind.
Eligible int `json:"eligible"`
EligibleImages int `json:"eligible_images"`
EligibleArchives int `json:"eligible_archives"`
// SafeToCollect is whether every kept archive was asked about and every one is held.
SafeToCollect bool `json:"safe_to_collect"`
}
func collectionCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("collection", flag.ContinueOnError)
asJSON := set.Bool("json", false, "answer as JSON")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 0 {
return errors.New("collection [--json]")
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
kept, err := inv.KeptArchives(ctx)
if err != nil {
return err
}
eligible, err := inv.ToCollect(ctx)
if err != nil {
return err
}
report := collectionReport{KeptArchives: len(kept), Eligible: len(eligible), Unheld: []string{}, Missing: []string{}}
for _, reference := range eligible {
if strings.Contains(reference, "/blobs/") {
report.EligibleArchives++
} else {
report.EligibleImages++
}
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return err
}
report.Store, err = artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return err
}
if report.Store == "" {
report.Unasked = len(kept)
report.Stopped = "this mesh has no artifact store on its network"
} else {
report.Unasked, report.Stopped = askHeld(ctx, artifacts.Store{Address: report.Store}, kept, &report)
}
report.SafeToCollect = report.Unasked == 0 && len(report.Unheld) == 0
if *asJSON {
encoder := json.NewEncoder(os.Stdout)
encoder.SetIndent("", " ")
return encoder.Encode(report)
}
printCollection(report)
return nil
}
// askHeld asks the store about each kept archive, stopping at the first answer that is not about
// the archive: a store that cannot be reached for one cannot be for the next, and a page of
// identical failures says less than one line.
func askHeld(ctx context.Context, store artifacts.Store, kept []string, report *collectionReport) (int, string) {
for i, reference := range kept {
held, err := store.Held(ctx, reference)
switch {
case err == nil && held:
report.Held++
case err == nil:
report.Unheld = append(report.Unheld, reference)
case errors.Is(err, artifacts.Gone):
report.Missing = append(report.Missing, reference)
case errors.Is(err, artifacts.ErrNotOurs):
// KeptArchives names only the mesh's own; counted as unasked if one ever is not.
report.Unasked++
default:
return report.Unasked + len(kept) - i, fmt.Sprintf("asking about %s: %v", reference, err)
}
}
return report.Unasked, ""
}
func printCollection(r collectionReport) {
store := r.Store
if store == "" {
store = "(not on the network)"
}
fmt.Printf("artifact store %s\n", store)
fmt.Printf("kept archives %d\n", r.KeptArchives)
fmt.Printf(" held %d\n", r.Held)
fmt.Printf(" unheld %d\n", len(r.Unheld))
fmt.Printf(" missing %d\n", len(r.Missing))
if r.Unasked > 0 {
fmt.Printf(" not asked %d (%s)\n", r.Unasked, r.Stopped)
}
fmt.Printf("eligible to let go %d (%d images, %d archives)\n", r.Eligible, r.EligibleImages, r.EligibleArchives)
if len(r.Unheld) > 0 {
fmt.Println("\nunheld — the store's collector would delete these; the next build's sweep holds them:")
for _, reference := range r.Unheld {
fmt.Printf(" %s\n", reference)
}
}
if len(r.Missing) > 0 {
fmt.Println("\nmissing — kept by the mesh, not in the store:")
for _, reference := range r.Missing {
fmt.Printf(" %s\n", reference)
}
}
fmt.Println()
if r.SafeToCollect {
fmt.Println("every kept archive is held: the store's collector may run for real")
} else {
fmt.Println("NOT every kept archive is known to be held: keep the store's collector on --dry-run")
}
}
@@ -1,68 +0,0 @@
//go:build integration
package main
import (
"context"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A member of a group whose gate is the composed check's is asked of the build seat with no module: its repository's
// check alone (novox/hq ADR 0403 clause 1). With no word from mesh-delivery, or one that does not say the gate is
// composed, it is checked alone with its gate (clause 6).
func askedForMember(t *testing.T, answer []groupsLine, answerErr error) link.CheckRequest {
t.Helper()
open := aMesh(t)
ctx := t.Context()
js, _ := aBuildQueue(t)
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
was := groupsAsked
groupsAsked = func(context.Context, bool) ([]groupsLine, error) { return answer, answerErr }
t.Cleanup(func() { groupsAsked = was })
p := link.PullUpdated{Owner: "novox", Repo: "mesh-controller", Number: 346, Base: "main", Head: "feat/x", Commit: headController,
Paths: []string{"main.go"}, ModuleDirsSaid: true, MergeCheck: true, MergeCheckSaid: true}
if err := (following{open: open, oneShot: true}).PullUpdated(ctx, p); err != nil {
t.Fatal(err)
}
q, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil || len(q.Asks) != 1 {
t.Fatalf("asks %d: %v", len(q.Asks), err)
}
return *q.Asks[0].Request.Check
}
func TestAComposedMembersCheckIsItsRepositoryChecksAlone(t *testing.T) {
line := group("feat/x", member("novox/mesh-controller", 346, headController, "main", true), member("novox/mesh-catalog", 350, headHost, "main", true))
line.ComposedGate = true
c := askedForMember(t, []groupsLine{line}, nil)
if len(c.Modules) != 0 || len(c.New) != 0 || len(c.Dependents) != 0 || !c.Composed {
t.Fatalf("the member's check is asked as %+v; want no module and composed", c)
}
}
func TestAMemberOfAGroupOfAnOlderMeshDeliveryKeepsItsOwnGate(t *testing.T) {
line := group("feat/x", member("novox/mesh-controller", 346, headController, "main", true), member("novox/mesh-catalog", 350, headHost, "main", true))
c := askedForMember(t, []groupsLine{line}, nil)
if !strings.Contains(strings.Join(c.Modules, ","), "mesh-controller") || c.Composed {
t.Fatalf("an older mesh-delivery's member is asked as %+v; want its own gate", c)
}
}
func TestAMemberWhoseGroupCannotBeLearnedIsCheckedAloneWithItsGate(t *testing.T) {
c := askedForMember(t, nil, context.DeadlineExceeded)
if !strings.Contains(strings.Join(c.Modules, ","), "mesh-controller") || c.Composed ||
!strings.Contains(c.SiblingsUnknown, "reads no siblings") {
t.Fatalf("the member is asked as %+v; want its gate and why its siblings are unknown", c)
}
}
@@ -1,136 +0,0 @@
package main
import (
"context"
"encoding/json"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A delivery group's gate is the composed check's, and its members' own checks run their repository's check alone
// (novox/hq ADR 0403 clauses 1 and 7). Pure: no store, no bus.
// A member of a group whose line says composed_gate is a composed member; one whose group says nothing of it (an older
// mesh-delivery), one of a group of a single pull request and a lone pull request are not.
func TestAMemberOfAComposedGroupIsKnownByTheGroupsLine(t *testing.T) {
two := func(composed bool) groupsLine {
l := group("feat/x", member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))
l.ComposedGate = composed
return l
}
p := link.PullUpdated{Owner: "novox", Repo: "mesh-host", Number: 114, Base: "main", Head: "feat/x", Commit: headHost}
recheck := link.PullUpdated{Owner: "novox", Repo: "mesh-host", Number: 114, Base: "main", Commit: headHost}
for name, c := range map[string]struct {
p link.PullUpdated
lines []groupsLine
want bool
}{
"composed": {p, []groupsLine{two(true)}, true},
"a recheck, by its number": {recheck, []groupsLine{two(true)}, true},
"an older mesh-delivery": {p, []groupsLine{two(false)}, false},
"another group": {p, []groupsLine{group("feat/y", member("novox/mesh-host", 1, headHost, "main", true))}, false},
"no group": {p, nil, false},
} {
if got := composedMember(c.p, c.lines); got != c.want {
t.Errorf("%s: composed %v, want %v", name, got, c.want)
}
}
alone := group("feat/x", member("novox/mesh-host", 114, headHost, "main", true))
alone.ComposedGate = true
if composedMember(p, []groupsLine{alone}) {
t.Error("a group of a single pull request is composed")
}
}
// A composed member's verdict never says its gate passed: the gate layer, and the verdict that is the gate's, read
// "composed"; a failure or an error stays what it was.
func TestAComposedMembersVerdictHasNoPassingGate(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114, Composed: true}
pass := link.BuildResult{ID: "b", Ref: headHost, Checked: asked, Check: &link.CheckOutcome{Verdict: "pass", Summary: "no module touched",
Gate: &link.CheckLayer{Verdict: "pass", Summary: "no module touched"}, RepoCheck: &link.CheckLayer{Verdict: "pass", Summary: "ok"}}}
c := checkedOf(pass)
if c.Gate == nil || c.Gate.Verdict != "composed" || c.Verdict != "composed" || c.RepoCheck == nil || c.RepoCheck.Verdict != "pass" {
t.Fatalf("the member's verdict says %+v gate %+v", c, c.Gate)
}
asked.Composed = false
if c := checkedOf(pass); c.Gate.Verdict != "pass" || c.Verdict != "pass" {
t.Fatalf("a member of no composed group has its own gate: %+v", c.Gate)
}
asked.Composed = true
failing := pass
failing.Check = &link.CheckOutcome{Verdict: "fail", Summary: "conflict", Gate: &link.CheckLayer{Verdict: "fail", Summary: "conflict"},
RepoCheck: &link.CheckLayer{Verdict: "fail", Summary: "conflict"}}
if c := checkedOf(failing); c.Gate.Verdict != "fail" || c.Verdict != "fail" {
t.Fatalf("a failure was read as composed: %+v", c.Gate)
}
broken := link.BuildResult{ID: "b", Ref: headHost, Checked: asked, Failed: "the facts snapshot cannot be read"}
if c := checkedOf(broken); c.Verdict != "error" || c.Gate.Verdict != "error" {
t.Fatalf("an error was read as composed: %+v", c)
}
}
// The composed verdict on the bus carries every member, each with its trunk commit, for the forge's holder to set each
// member's mesh/merge-gate from it.
func TestTheComposedVerdictCarriesEveryMember(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114, Group: "feat/x",
Members: []link.GroupMember{{Owner: "novox", Repo: "mesh-controller", Number: 346, Ref: headController}}}
c := checkedOf(link.BuildResult{ID: "g", Ref: headHost, Checked: asked, Check: &link.CheckOutcome{Verdict: "pass", Summary: "ok",
Trunk: "aaaaaaaaaaaa", Members: []link.CheckedMember{{Owner: "novox", Repo: "mesh-controller", Commit: headController[:12], Trunk: "bbbbbbbbbbbb"}}}})
body, _ := json.Marshal(c)
for _, want := range []string{`"group":"feat/x"`, `"members":[{"owner":"novox","repo":"mesh-host","number":114,"commit":"` + headHost + `","trunk":"aaaaaaaaaaaa"}`,
`{"owner":"novox","repo":"mesh-controller","number":346,"commit":"` + headController + `","trunk":"bbbbbbbbbbbb"}`} {
if !strings.Contains(string(body), want) {
t.Fatalf("the composed verdict lacks %s:\n%s", want, body)
}
}
}
// Members whose manifests order their modules in a cycle fail the composed check, naming every module and edge, and no
// ask is made of the build seat.
func TestAGroupInACycleFailsItsComposedCheckWithoutABuild(t *testing.T) {
members := []orderMember{
{ID: "novox/x@" + headHost[:12], Repository: "novox/x", Base: "main", Head: headHost, Number: 7, Paths: []string{"module.json"}},
{ID: "novox/y@" + headController[:12], Repository: "novox/y", Base: "main", Head: headController, Number: 9, Paths: []string{"module.json"}},
}
reach := map[string]orderReach{members[0].ID: {Moved: []string{"mx"}}, members[1].ID: {Moved: []string{"my"}}}
was, wasParent := manifestAt, parentOf
t.Cleanup(func() { manifestAt, parentOf = was, wasParent })
parentOf = func(context.Context, string, string, string) (string, error) { return "", nil }
manifestAt = func(_ context.Context, _, repo, _, ref string) (catalogue.Manifest, bool, error) {
if ref == "main" {
return catalogue.Manifest{}, false, nil // what the pull requests merge into holds no such manifest
}
if repo == "x" {
return catalogue.Manifest{Module: "mx", DefinesSeats: []catalogue.SeatDeclaration{{Name: "s", Serves: []catalogue.Verb{{Name: "a"}}}},
Invokes: []string{"seat:s.b"}}, true, nil
}
return catalogue.Manifest{Module: "my", DefinesSeats: []catalogue.SeatDeclaration{{Name: "s", Serves: []catalogue.Verb{{Name: "b"}}}},
Invokes: []string{"seat:s.a"}}, true, nil
}
c, cyclic, err := cycleVerdict(context.Background(), "feat/cycle", members, reach, []inventory.Edge(nil), nil)
if err != nil || !cyclic {
t.Fatalf("no cycle found: %v", err)
}
if c.Verdict != "fail" || c.Group != "feat/cycle" || len(c.Members) != 2 || c.Gate == nil || c.Gate.Verdict != "fail" {
t.Fatalf("the verdict is %+v", c)
}
for _, w := range []string{"mx", "my", "s.a", "s.b"} {
if !strings.Contains(c.Summary, w) {
t.Errorf("the cycle does not name %s: %s", w, c.Summary)
}
}
// Members with no edge between them are no cycle.
manifestsReadMu.Lock()
clear(manifestsRead)
manifestsReadMu.Unlock()
manifestAt = func(context.Context, string, string, string, string) (catalogue.Manifest, bool, error) {
return catalogue.Manifest{}, false, nil
}
if _, cyclic, err := cycleVerdict(context.Background(), "feat/cycle", members, reach, nil, nil); err != nil || cyclic {
t.Fatalf("a cycle where there is none: %v %v", cyclic, err)
}
}
-76
View File
@@ -1,76 +0,0 @@
package main
import (
"context"
"encoding/json"
"fmt"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// The looks of the open conditions — how often each was observed, when last, its newest evidence — are
// kept in the controller's store, not the conditions bucket (novox/hq issue 420, ADR 0289 §1): the bucket
// is written when what a condition says changes, never for a look that saw it as it was.
// storeLooks keeps the conditions' looks in the inventory.
type storeLooks struct{ inv *inventory.Inventory }
func (s storeLooks) KeepLooks(ctx context.Context, key string, l conditions.Looks) error {
evidence, err := json.Marshal(l.Evidence)
if err != nil {
return err
}
return s.inv.KeepConditionLooks(ctx, inventory.ConditionLooks{Key: key, Observations: l.Observations,
LastObserved: l.LastObserved, Evidence: evidence})
}
func (s storeLooks) Looked(ctx context.Context, key string) (conditions.Looks, bool, error) {
row, kept, err := s.inv.ConditionLooksOf(ctx, key)
if err != nil || !kept {
return conditions.Looks{}, kept, err
}
l, err := looksOf(row)
return l, err == nil, err
}
func (s storeLooks) AllLooks(ctx context.Context) (map[string]conditions.Looks, error) {
rows, err := s.inv.AllConditionLooks(ctx)
if err != nil {
return nil, err
}
out := make(map[string]conditions.Looks, len(rows))
for key, row := range rows {
l, err := looksOf(row)
if err != nil {
return nil, err
}
out[key] = l
}
return out, nil
}
func (s storeLooks) ForgetLooks(ctx context.Context, key string) error {
return s.inv.ForgetConditionLooks(ctx, key)
}
func looksOf(row inventory.ConditionLooks) (conditions.Looks, error) {
l := conditions.Looks{Observations: row.Observations, LastObserved: row.LastObserved.UTC()}
if len(row.Evidence) > 0 {
if err := json.Unmarshal(row.Evidence, &l.Evidence); err != nil {
return conditions.Looks{}, fmt.Errorf("the evidence kept for the condition %s cannot be read: %w", row.Key, err)
}
}
return l, nil
}
// looksHere is where a process that is not the serving controller reads the conditions' looks: the store,
// opened for it, closed by the function returned. A store that cannot be opened is said on every condition
// read through it, never read as none.
func looksHere(ctx context.Context) (conditions.LookStore, func()) {
inv, err := openInventory(ctx)
if err != nil {
return conditions.NoLooks{Err: err}, func() {}
}
return storeLooks{inv}, inv.Close
}

Some files were not shown because too many files have changed in this diff Show More