Compare commits

...
Author SHA1 Message Date
jschoubben 8831c74f70 Mark the packages whose tests read files outside their directory, found by the review (issue 1935) 2026-10-24 04:38:51 +02:00
jschoubben a96bd2ff38 Mark the packages whose tests read files outside their directory, so mesh-build-go never keeps their verdict (issue 1935) 2026-10-24 04:00:41 +02:00
jschoubben 1c50608246 Merge pull request 'Give route-proxy a build the fixed controller takes in, dropping the incident note #450 added (issue 1933)' (#452) from fix/1933-rebuild-route-proxy-2 into main
Reviewed-on: #452
2026-10-24 01:00:18 +00:00
jschoubben 760351b4ae Merge pull request 'The sweep keeps a fresh build's files, and a take-in holds every file before it records a build (issue 1933)' (#451) from fix/1933-sweep-keeps-fresh-builds into main
Reviewed-on: #451
2026-10-24 00:54:55 +00:00
jschoubben 899389c710 Reword route-provider's stdout line, so route-proxy's tree differs from the 01:52 build whose files are gone (issue 1933)
mesh/delivery delivered
2026-10-24 02:50:12 +02:00
jschoubben a203f605d4 Give route-proxy a build the fixed controller takes in, dropping the incident note #450 added (issue 1933) 2026-10-24 02:49:56 +02:00
jschoubben 0741880d31 Keep a fresh build's archives from the controller's own sweep (issue 1933)
mesh/delivery delivered
A sweep read what it keeps before what was ever recorded, so a take-in's
build recorded between the two reads, by a concurrent take-in off the
plans' hold, was eligible and let go of seconds after the take-in read
it; a new module's build is unkept until it is registered. Now the sweep
reads the records first and keeps what a build recorded within the hour
or an open plan names, and names every reference it lets go of; a take-in
holds every artifact in the store before recording, failing on one the
store lacks; and RecordBuild clears artifact_collected as RecordMirrored
does.
2026-10-24 02:44:20 +02:00
jschoubben fa734d3098 Merge pull request 'Give route-proxy a new build: the controller's sweep removed its last build's files from the store, so novox and ace cannot apply (issue 1933)' (#450) from fix/1933-rebuild-route-proxy into main
Reviewed-on: #450
2026-10-24 00:39:47 +00:00
jschoubben f13d42abb4 Give route-proxy a new build: the controller's sweep removed its last build's files from the store, so novox and ace cannot apply (issue 1933)
mesh/delivery delivered
2026-10-24 02:37:06 +02:00
jschoubben fdf26a1fe2 Merge pull request 'A module the catalogue no longer holds ends deleted instead of failing the plan; vendor/ goes; tests name verbs that exist (issues 1922, 1305, 1931)' (#449) from fix/1922-removed-module into main
Reviewed-on: #449
2026-10-23 23:47:05 +00:00
jschoubben 343e271ec7 Integration tests stop naming the verbs R4g removed (issue 1931)
mesh/delivery delivered
mesh/check checking 343e271ec794 on fdf26a1fe29f
2026-10-23 23:21:39 +02:00
jschoubben 05c872485a Delete the committed dependencies; builds resolve by go.sum (issue 1305) 2026-10-23 22:16:09 +02:00
jschoubben 12dd2bb3f0 A module the catalogue no longer holds ends deleted instead of failing the plan; the unsent hint names release-backlog (issue 1922) 2026-10-23 22:15:18 +02:00
jschoubben 5831eec770 Merge pull request 'R4g+R5 (mesh-controller): the controller asks no build, build-agent and the merge gate go, register by reference (design 47)' (#448) from feat/r4g into main
Reviewed-on: #448
2026-10-23 17:50:22 +00:00
jschoubben c9a7102bc5 Set aside a retired seat's consume the trunk catalogue still holds, and say it, so a group that retires a seat passes its check
mesh/delivery failed: its walk failed, carrying its own commit: a gate on a first machine (what it carried put back), a build, a machine
mesh/delivery-group group feat/r4g failed: a member failed
mesh/check checking c9a7102bc53b on 791786d5b87f | novox/mesh-controller#448 in group feat/r4g
2026-10-23 18:05:06 +02:00
jschoubben 67279199e9 Say the trunk rule as ADR 0534 §6 has it in the registration's comment
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: github.com/novox/mesh-controller/internal/broker.TestTheCataloguesDerivedSubjectsMeet: agreement_catalogue_test.go:130: judge…
2026-10-23 17:19:39 +02:00
jschoubben 9d2a9e62d1 Merge remote-tracking branch 'origin/fix/forge-requires-no-status' into feat/r4g 2026-10-23 17:18:58 +02:00
jschoubben b345d10bec Pin mesh-sdk at feat/r4g's head a2c813a6, its test fixture fix, with go.sum tidied
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: git.novox.be/novox/mesh-catalog/checks/config.TestTheUnmovedListHoldsNoModuleTheTrunksDoesNot: realtree_test.go:14: mesh-cata…
2026-10-23 17:06:08 +02:00
jschoubben 13842a7234 Grant the builder the build seats' streams' information and messages by name, so its take-over at start can tell an ask its key names is on its stream (issue 1910)
mesh/delivery-group group feat/r4g rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: g14: novox/mesh-catalog#464: checks/health/go.mod names mesh-sdk a9560890ee26, which is neither on mesh-sdk's trunk nor the group's…
2026-10-23 16:59:03 +02:00
jschoubben 4238318601 Pin mesh-sdk at feat/r4g's head a9560890, re-vendor it and capture its conformance events there (R4g) 2026-10-23 16:54:07 +02:00
jschoubben 758c8e84e5 Bound the build seats' workers' deliveries, so an ask no run ends is offered at most five times even when the holder that says it is poison dies too (issue 1907) 2026-10-23 16:42:13 +02:00
jschoubben 9222575ae9 Vendor the dependencies again for R4g's go.mod (mesh-sdk 9205222a, the mesh-host replace as before), so build-agent compiles the controller offline in the group's first tier (issue 1305) 2026-10-23 16:38:31 +02:00
jschoubben c5b8fca8bd Revert "Delete vendor/ and the route proxy's: the build module fetches the modules, the root image builds with -mod=mod (issue 1305, C6)"
This reverts commit 9c4bf1f08b.
2026-10-23 16:37:40 +02:00
jschoubben ff543d3def Take the watchman's grant on the build seats back: a waiting ask has no pulse, a taken one is read from the steps (issue 1908) 2026-10-23 16:35:28 +02:00
jschoubben ef211faefb Prove the registration by hand reads back as a build and is held by the sweep, and name the store check as the test's limit (issue 1888, C8b) 2026-10-23 16:34:29 +02:00
jschoubben 3c247e80af Fix the view grant test's consumer subjects (issue 1887 A8) 2026-10-23 16:32:43 +02:00
jschoubben bb0d5e17db Grant the watchman the build seats' accept subjects and the view the seat workers' information, each by name, so the backlog and the waiting-ask pulse read live (issue 1887 A8) 2026-10-23 16:32:32 +02:00
jschoubben 24ad859629 Record the checksums of mesh-sdk feat/r4g 9205222a for the group's modules (R4g) 2026-10-23 16:26:11 +02:00
jschoubben 5ccc87f4d0 Capture mesh-catalog at feat/r4g (2246ccc4) with the templates its manifests name and its predefined.json, and lay the copy out as a repository's root
The catalogue's manifests moved their content into template files and fall back to predefined keys, so a copy of
module.json alone no longer passes the catalogue's own check; the bus's Dockerfile is gone from the catalogue.
2026-10-23 16:24:14 +02:00
jschoubben 68c1220e87 Pin mesh-sdk at feat/r4g's head 9205222a, the merge of lanes A, B, S8 and F1, and capture its conformance events there
go.sum gains its lines from go mod tidy once mesh-sdk feat/r4g is pushed.
2026-10-23 16:22:07 +02:00
jschoubben 5b5cd6160a Merge branch 'feat/r4g-asks' into feat/r4g 2026-10-23 16:20:32 +02:00
jschoubben d7d70cd539 Merge branch 'feat/r4g-assigner' into feat/r4g 2026-10-23 16:20:32 +02:00
jschoubben df70ca43a7 Take the controller's publish grants on node-build-agent and mesh-build-machine away, with the cancelled sets and the redelivery of its own asks: it asks no build seat (ADR 0525 §10, C3a) 2026-10-23 16:13:43 +02:00
jschoubben f60c742d93 Prove the grant through the send of the bus's machine (overTheBus.declare), so removing the write from the live path fails the test (issue 1394, review round 1 item 4) 2026-10-23 16:13:42 +02:00
jschoubben 2db5dcdcb4 Write the hand act of register --reference after the registration was made, not before, and test the why and the record (issue 1888, C8b) 2026-10-23 16:12:16 +02:00
jschoubben d9cdb3615d Test that the builder's record names its key and fingerprint after a plan that moves it (issue 1851, C5 criterion) 2026-10-23 16:11:07 +02:00
jschoubben dec3b83e66 Test that the builder, a build module and the controller are taken from the builder's keys and nothing reaches node-build-agent (issue 1851, C2 criterion) 2026-10-23 16:11:07 +02:00
jschoubben 2e36a63096 Pin mesh-sdk at feat/r4g's head c3915ffb, the merge of lane B's layouts, so every r4g module builds against one SDK
go.sum gains its lines from go mod tidy once mesh-sdk feat/r4g is pushed.
2026-10-23 16:01:15 +02:00
jschoubben eb8286fc68 Merge feat/r4g-assigner into feat/r4g: lane B's grants bucket and its writer beside lane C's deletions
The controller's readable states, its bucket lists and the composed golden file keep lane C's removal of the
checks-running and plans-building buckets and the build agent's check subjects, and add lane B's grants bucket.
2026-10-23 15:59:52 +02:00
jschoubben 5540fd4f85 Find the catalogue's repository by its URL or seat source, and fail a plan whose catalogue repository is nowhere, so a take-in from a key no longer waits for ever (issue 1902) 2026-10-23 15:57:30 +02:00
jschoubben e419d0182f Delete the builder, the build asks, the build queue and the merge gate: the builder's keys are the only way a module is built (issues 1851, 1299, 1318; C2b-C5, C10)
The controller asks no build of anybody (ADR 0501 §5, ADR 0525 clause 6, ADR 0503 §3). Every module of a walk is taken from the
builder's done key; a key not done yet waits, named in the plan, and never fails it.

- C2b/C2c: internal/builder and cmd/mesh-controller's asks go (askABuild, onlyTheBuildersOwn, buildOneAsked, takeIn from an
  outcome, buildAgentsOwn, the builder's say and its fallback, the daemon hearing an outcome). The server follows the retired
  seats' built subjects only to let go of them, because the installer's carried user list names them.
- C2d: replay, queue, cancel, clear, kill, pause, resume, delivery-check, check-here and merge-gate are refused by name.
- C3: the check queues (CHECKS_READY, CHECKS_OTHER) and their workers, the checks-running and plans-building buckets and
  their grants, the build queue and its hold (plans_building.go, queue.go, link/holds.go, builds_nats.go, checks_nats.go).
- C4: merge_gate*.go, the pull request's repo-check path, the Checked event and its layers; a pull request asks the
  controller nothing but to tell mesh-config what its head declares.
- C5: the builds bucket is written only by key_takein.
- C10 (help and refusals): the terminal's help lists only what is served.

Integration tests of the plan's tiers, sends, stops and retries now run on a stand-in builder with a done key for every
module (walk_keys_scaffold_integration_test.go); the tests of the asks, the queue, the merge gate and the replays are deleted.
2026-10-23 15:55:15 +02:00
jschoubben 5ca4f14479 Derive a worker per kind for the build seats and grant their users to put asks, so a build node pulls delivery, check and recheck in order and a session cannot ask (issue 1887 A6, ADR 0525 §2) 2026-10-23 15:54:45 +02:00
jschoubben c43883910d Write grants.<module>.<fp> once the bus's user list carrying a module's manifest at that fingerprint is sent, so the assigner's sixth rule reads that its credential exists (issue 1394, subtask C9b; ADR 0528 clause 2, ADR 0429 §6) 2026-10-23 15:39:40 +02:00
jschoubben 548681cc4e Declare the controller's grants bucket, written by the controller alone and readable through mesh-controller.grants, so the assigner can read its sixth rule's key (issue 1394, subtask C9a; ADR 0528 clause 2); the SDK pin moves to the layouts' commit, captured beside 2026-10-23 15:39:40 +02:00
jschoubben 9c4bf1f08b Delete vendor/ and the route proxy's: the build module fetches the modules, the root image builds with -mod=mod (issue 1305, C6) 2026-10-23 15:14:58 +02:00
jschoubben 6300a5b31c Delete merge-check.sh, check-nested-modules.sh, the store-tests list and RELEASE-57: the checker runs the build module's steps, nested modules by convention (issue 1294, C7a and C7b) 2026-10-23 15:14:01 +02:00
jschoubben 505694c424 Register a module at artifacts the store holds, by hand, recorded as a build and a hand act, so the controller can be delivered without the builder (issue 1888, C8a and C8b) 2026-10-23 15:13:17 +02:00
jschoubben 72186815e3 Delete the build-agent module and the builder program: the build module builds, the checker checks (issue 1851, C2a) 2026-10-23 15:07:57 +02:00
jschoubben 526eed99fa Drop the toolchain image from the facts and the dependency edges: no module is compiled in one any more (issue 1851, C1b) 2026-10-23 15:07:29 +02:00
jschoubben b506852388 Move the Go build source reader out of internal/builder so the builder can go (issue 1851, C1a) 2026-10-23 15:06:31 +02:00
mesh-tools 791786d5b8 Merge pull request 'Wait out a resolver that does not answer in a build's registry step, and say it was the resolver (issue 1748)' (#447) from fix/1748-registry-wait-retries-a-lookup into main 2026-10-23 11:20:15 +00:00
jschoubben 0b312218f9 Wait out a failed lookup of the registry's name in a build, and say it was the resolver (issue 1748)
mesh/delivery delivered
mesh/check pass 0b312218f90f on c834e4f82137 | unit 1762 run, 0 kept, 4 skipped | integration 3040 run, 0 kept, 16 skipped | over bound: integration (…
2026-10-23 13:03:17 +02:00
mesh-tools c834e4f821 Merge pull request 'Keep every refused build by its id, so a plan never settles a refused build built and sends the old version (issue 1713)' (#446) from fix/1713-keep-every-refusal into main 2026-10-23 10:58:08 +00:00
jschoubben 149f89e8f9 Keep every refused registration by its build id, so a Retry never settles it built
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792753090890815254 (novox/mesh-controller at c834e4f8 into ma…
mesh/check pass 149f89e8f9bf on 0173d9c70e59 | unit 1759 run, 0 kept, 4 skipped | integration 3037 run, 0 kept, 16 skipped | over bound: integration (…
A build heard and recorded and not registered kept its refusal only when a
plan was open or the plans were held. A tier's first failure fails its plan,
so later refusals were lost; a Retry then settled those builds built from
their records, sent the modules' old versions and reported the plan done.
A build without a readable manifest never reached its plan at all.

Keep the refusal in the store and the process whatever the plans' state, and
fail the asking plan by the build's id when the module is not known.

Issue 1713.
2026-10-23 12:13:33 +02:00
jschoubben b2af3d654f Trust a trunk whose whitelist adds the operator's own login, never an agent's (ADR 0534 §2, §6)
The gitea module keeps the operator's login beside mesh-tools on every main so
the operator can merge by hand; an agent's account there is still refused.
2026-10-23 12:10:11 +02:00
jschoubben 7676e67c5e Trust a module's trunk for who can merge into it, not for a required status (ADR 0534 §6)
The forge requires no status to merge any more (ADR 0534 §1); a trunk is
trusted when it takes no push, its merge whitelist names mesh-tools alone and
no administrator merges past its rule, so only mesh-delivery lands a commit
there, on a passing mesh/check and the operator's Approve. The old rule failed
the 11:51 delivery plan once mesh-catalog's status was unticked.
2026-10-23 12:02:00 +02:00
mesh-tools 0173d9c70e Merge pull request 'Keep the admin address's query in a test database's address, so store tests reach the checker's socket store (issue 1889)' (#445) from fix/1889-identity-store-ready into main 2026-10-23 07:37:06 +00:00
mesh-tools 89abce5df9 Merge pull request 'Judge each machine on a plan's last send and resume a plan lost to silence with plans retry, so a resumed plan is not failed by its own old sends (issue 1886, ADR 0531)' (#444) from fix/1886-resume-times into main 2026-10-23 07:26:31 +00:00
jschoubben 185d0294eb Keep the store address's query in every test database's address, so store tests reach a socket-addressed store (issue 1889)
mesh/merge-gate head 185d0294ebd4 merged into trunk 89abce5df955 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; the ch…
mesh/repo-check head 185d0294ebd4 merged into trunk 89abce5df955 — pass: its merge-check.sh passed
mesh/delivery covered after its stop: a later merge that contains it was delivered: novox/mesh-controller@0b312218f90f (merged as 791786d5 into main, wal…
mesh/check pass 185d0294ebd4 on 89abce5df955 | unit 1756 run, 0 kept, 4 skipped | integration 3034 run, 0 kept, 16 skipped | over bound: integration (…
The checker's integration store answers on a unix socket named in the query of MESH_TEST_POSTGRES
(?host=<socket dir>&sslmode=disable). The identity, inventory and licences fixtures cut the address at
its last slash and appended ?sslmode=disable, dropping host=, so pgx dialled its default socket where no
server is: every store test waited out its 20 s readiness bound, each package ran into go test's
10-minute timeout, and the verdict named only the first failure,
identity.TestACertificateChainsToTheMeshsOwnAuthority.

store.DatabaseForTest now builds that address for every fixture, keeping the query as the template raise
already did; identity's fresh is ForTest rather than a second copy. The readiness bound and its error,
which wraps the driver's cause, are unchanged.
2026-10-23 08:55:50 +02:00
jschoubben 4d4d51eb01 Pin the resume of a lost walk's refusals beside a newer or a related plan, and cite ADR 0360 §7 and ADR 0531 (issue 1886)
mesh/merge-gate head 4d4d51eb017f merged into trunk 219fdcd7f00c — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; the ch…
mesh/repo-check head 4d4d51eb017f merged into trunk 219fdcd7f00c — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-23 08:10:08 +02:00
jschoubben b7d996878b Judge a walk's machine on its last send, enter a resumed tier anew, and resume a walk lost to silence (issue 1886)
R4e, resumed at tier 1 after a stop of five hours, named ace and shanks silent within seconds of tier 1's send: the
walk judged each machine on every send it had made it, tier 0's of 00:23 too, held against tier 1's templates, and
measured the bound from it. A send carries the whole declaration, so the last one is what a machine is judged on.
The resume kept the tier's entry of before the stop, so the plan read waiting for 5h48m. And plans retry refused a
walk failed only by the gate's silence, though its machines had since applied.
2026-10-23 08:02:11 +02:00
mesh-tools 219fdcd7f0 Merge pull request 'Resume a delivery plan stopped by hand with plans retry, so a stop is not the end of what its tier waited on (issue 1886)' (#443) from fix/1886-retry-stopped into main 2026-10-23 05:40:52 +00:00
jschoubben 55ba7f2cb8 Hold every refusal of the stopped path in the test ADR 0523 names: a newer open plan of the branch, a newer plan of a build not sent (issue 1886)
mesh/merge-gate head 55ba7f2cb83c merged into trunk 296e11d499cb — pass: builds mesh-controller → novox; no bus step; the change touches no module of the m…
mesh/repo-check head 55ba7f2cb83c merged into trunk 296e11d499cb — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check checking 55ba7f2cb83c on 219fdcd7f00c
2026-10-23 02:44:58 +02:00
jschoubben 102f8dc267 Judge a resume only by the builds its tier holds, and ask build-agent's own of build-agent (issue 1886, review round 1)
mesh/merge-gate head 102f8dc2674e merged into trunk 296e11d499cb — pass: builds mesh-controller → novox; no bus step; the change touches no module of the m…
mesh/repo-check head 102f8dc2674e merged into trunk 296e11d499cb — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-23 02:34:59 +02:00
jschoubben 296e11d499 Merge pull request 'Ask build-agent until the builder says it writes references, so a plan never waits on a key nothing can reference (issue 1886, ADR 0522)' (#442) from fix/1886-builder-says-references into main
Reviewed-on: #442
2026-10-23 00:27:00 +00:00
jschoubben eed4976483 Resume a walk stopped by hand with plans retry, so a stop is not the end of what its tier waited on (issue 1886) 2026-10-23 02:05:16 +02:00
jschoubben bbadbb2af0 Judge the ask and the settle by one say of the builder (issue 1886, merge check hang)
mesh/merge-gate head bbadbb2af021 merged into trunk 289bbf68e4f9 — pass: builds mesh-controller → novox; no bus step; the change touches no module of the m…
mesh/repo-check head bbadbb2af021 merged into trunk 289bbf68e4f9 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/1886-builder-says-references delivered: every member is delivered
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
The settle read the say from the builder's bucket while the ask read it through
builderSaysReferences; where they differed (a test's say against a bus with no
bucket) a module was handed back and taken from its key again at every turn,
and the look's loop never ended.
2026-10-23 01:59:18 +02:00
jschoubben 1414ed7f7b Judge an ask by the builder's say the walk read, never a second read (issue 1886, review round 1)
mesh/delivery-group group fix/1886-builder-says-references checking: 0 of 2 member(s) ready
mesh/merge-gate building on ace since 01:38
mesh/delivery superseded: a newer head of the same pull request
A second read that failed, or that turned true in between, refused the ask and
failed the whole plan.
2026-10-23 01:03:19 +02:00
jschoubben f67342f0d8 Ask build-agent for the build modules that write references, whatever the builder says (issue 1886, ADR 0522 §6)
They push and answer each reference, so a key they write themselves can never
carry their own delivery; until R4g build-agent delivers them as it does the
builder.
2026-10-23 00:53:25 +02:00
jschoubben fae7710bed Ask build-agent for every module until a running builder says it writes references (issue 1886)
R4e's own delivery plan waited for ever in tier 1: the controller took every
module but build-agent's own from a referenced builder key, and only R4e's
builder and build modules, delivered by that same plan, write references.
The builder now says so in its bucket of instances; until it does, a walk asks
build-agent as before R4e, and an open walk waiting on a key the running
builder cannot reference is judged again at the next look.
2026-10-23 00:47:15 +02:00
mesh-tools 289bbf68e4 Merge pull request 'R4e (mesh-controller): deliver from the builder's keys, ask build-agent only for its own modules, follow the forge's protection for checks, announce the controller's states' (#441) from feat/r4e into main 2026-10-22 22:12:58 +00:00
mesh-tools f2a4363a1a Merge pull request 'Clone from the forge again when the resolver does not answer, so a passing lookup failure does not fail a build (issue 1773)' (#440) from fix/1773-clone-retries-a-lookup into main 2026-10-22 19:44:36 +00:00
jschoubben d51324df45 Format the help's test (issue 1851)
mesh/repo-check head d51324df4518 merged into trunk f2a4363a1a77 beside mesh-catalog#457@896bd883 — pass: its merge-check.sh passed
mesh/merge-gate head d51324df4518 merged into trunk f2a4363a1a77 — pass: group feat/r4e composed: every machine composes with the change as it did without …
mesh/delivery covered after its stop: a later walk that contains it was delivered: plan-1792707174627079192 (novox/mesh-controller at 289bbf68 into main,…
mesh/delivery-group group feat/r4e delivered: every member is delivered
mesh/check failed: unit: ..test: node:internal/modules/esm/resolve:989 | d51324df4518 on f2a4363a1a77 | unit 1 run, 0 kept | integration 0 run, 0 kept…
2026-10-22 21:20:06 +02:00
jschoubben 58512a9c6d Pin mesh-sdk at the R4e member's final head 104533e8 and vendor it (R4e E)
The SDK's S1 and S2 review fixes landed after 875c680d.
2026-10-22 21:19:46 +02:00
jschoubben 1fd833bfe8 List the store tests R4e adds, and drop those of the hand builds and the stopped checks it deleted (issue 1851, issue 1299)
The merge check refused the eight deleted tests as listed and not run.
2026-10-22 21:18:58 +02:00
jschoubben fd4d26b42c Keep TestReplay867 off the build node's real free space (issue 1834, note 12)
mesh/merge-gate head fd4d26b42cd7 merged into trunk 96441ea28a84 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head fd4d26b42cd7 merged into trunk 96441ea28a84 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
The check's floor of free space reads the disk the test runs on, so
every mesh-controller check failed while its build node had under
20 GiB free, whatever the change (seen in the try of issue 1773's fix).
The replay stubs the free space, as internal/builder's tests do.
2026-10-22 21:17:42 +02:00
jschoubben f9014452d3 List register and switch in the help, and none of the removed commands (issue 1851 A11a, A11b)
The help still offered build, rebuild and replay --register, each now
refused as removed, and did not name the terminal's register.
2026-10-22 21:10:20 +02:00
jschoubben 61449249a8 Name build-agent's own as the builder's module, build-agent and mesh-tools (ADR 0520 §1, issue 1851)
ADR 0520 narrowed after its review: mesh-tools-go builds from its own
directory on a pinned base and has its key from mesh-build-image, and
mesh-tools-python is no module, so the set is named, not derived from
the toolchain bases.
2026-10-22 21:08:47 +02:00
jschoubben 61f7e89ad5 Ask a repository check whenever the trunk requires it, beside mesh/check (issue 1299 C1, ADR 0513 §4)
A trunk requiring both contexts (hq) matched mesh/check first and had
its own merge-check.sh never asked.
2026-10-22 21:07:55 +02:00
jschoubben 9713ceb84d Ask build-agent for itself and its toolchain images, never a builder key (ADR 0520, issue 1851)
The builder writes no key of build-agent or of the toolchain bases its
builds stand on, so a walk moving one would hold its tier for ever.
Until R4g they are build-agent's own, as the builder's module is; the
bases are read from the toolchains, not copied.
2026-10-22 21:06:33 +02:00
jschoubben 93fc2f48a1 Name a retry's modules still waiting for the bus, read as the plan it resumes (issue 1851 A11a, issue 1403)
With the hand rebuild gone, a retry of a plan failed past the bus's bound
read the waiting modules while the plan still said failed, named none,
and answered "asked" with nothing after it.
2026-10-22 21:05:01 +02:00
jschoubben acaf8830ec Try a forge clone again when its name lookup fails (issue 1773)
A build or check failed now and then on 'Could not resolve host:
novox.internal' between clones that resolved the same name, and its
whole delivery failed with it. A clone whose lookup failed reached
nothing, so it is cloned again with a growing pause for up to 60 s,
each retry said in the build's log as the resolver not answering;
past the bound it fails saying so. Any other git failure fails at
once, as before. Covers a module's own clone, an artifact's context
and a check's repositories.
2026-10-22 20:56:46 +02:00
jschoubben d5d53893a4 Send the bus its user list before a starting controller writes (issue 1800)
A new controller's grants reached the bus only after its gate, so the
buckets it newly writes were refused until then. At start it now sends
the bus's machine the user list alone, every build there kept.
2026-10-22 20:51:08 +02:00
jschoubben a0b2d98002 Merge main (upgrade safety's declared packages, issue 1840) and pin the R4e mesh-sdk member's final head 875c680d, re-vendored here and in modules/route-proxy; the controller announces declared-packages readable (issue 1790, D1; E) 2026-10-22 20:44:59 +02:00
jschoubben bc80683b9e Count the work the take-ins start in the background, so a test's mesh closes only once that work has ended (issue 1851, A5-A7; issue 1790, D3) 2026-10-22 20:41:30 +02:00
mesh-tools 96441ea28a Merge pull request 'Keep each node's declared packages in a state the pacman module reads (upgrade safety, issue 1840, ADR 0507 §5)' (#439) from feat/upgrade-safety into main 2026-10-22 18:34:19 +00:00
jschoubben 420ae3d0e7 Test the group check's answer without swapping standard output, and wait for a take-in to forget a new module's wait (issue 1299, C2; issue 1851, A5) 2026-10-22 20:19:57 +02:00
jschoubben 7c0e1d223f Serve a removed verb as its refusal, not as a verb this build is behind on, and fit the tests of the verbs and the stopped checks (issue 1851, A11a; issue 1299, C1; issue 1318, C3) 2026-10-22 19:38:02 +02:00
jschoubben a10ef8995b Vendor what go.mod pins after the rebase on main (issue 1790; the SDK's 1833 fix returns with E's re-pin) 2026-10-22 19:05:21 +02:00
jschoubben dfc2f2aa98 Take a walk's key in again when it was recorded at an earlier merge, so the module is registered at the commit the walk carries before it is built (issue 1851, A6) 2026-10-22 19:05:05 +02:00
jschoubben ab33f015ec Say again what the controller keeps at every push, so a word refused before its grant is kept once a push carries it (issue 1790, D1) 2026-10-22 19:05:05 +02:00
jschoubben 791c2a3d48 Refuse a removed verb by name before reading its arguments, as the SDK member's table now offers none (issue 1851, A11a; issue 1318, C3) 2026-10-22 19:05:05 +02:00
jschoubben d36e0ea278 Announce the controller's states, keep every module's word, and grant a read only where its owner announced the state readable; the copied list of the controller's mirrors goes (issue 1790, D1-D4, ADR 0495, ADR 0513 §3) 2026-10-22 19:05:04 +02:00
jschoubben b6ed18197b Vendor the R4e mesh-sdk member's head with the states announcement (issue 1790; re-pinned to the final head in E) 2026-10-22 19:05:04 +02:00
jschoubben 45d091778c Refuse delivery-plan and forget-checks by name and take a merge's warrant from the teller alone, no longer mesh-delivery's (issue 1318, C3, ADR 0450 §8) 2026-10-22 19:05:04 +02:00
jschoubben 3a1937914e Ask build-agent no composed check of a delivery group, and answer that the checker judges it (issue 1299, C2, ADR 0503 §3) 2026-10-22 19:05:04 +02:00
jschoubben 25fdf9920e Read a commit as the same as itself however short, and fit the pending assignments' tests to a mesh that asks no build (issue 1851, A9, A10) 2026-10-22 19:05:04 +02:00
jschoubben 323e1270cc Ask no pull request check of build-agent but the own merge-check.sh of a repository whose trunk requires mesh/repo-check alone, and say every verdict with no gate layer (issue 1299, C1, ADR 0503 §3, §5) 2026-10-22 19:05:04 +02:00
jschoubben e7e23c57cc Read a module whose every artifact names its paths from those paths and its manifest alone, so a change elsewhere in its directory moves nothing (issue 1758, B1, ADR 0499 §5) 2026-10-22 19:05:04 +02:00
jschoubben a8387ebad8 Name, before the switch, which modules it moves: the held build's digests against the builder's done key, images and packages said as what cannot be compared (issue 1851, A13) 2026-10-22 19:05:04 +02:00
jschoubben 1963d84a7a Refuse, in the one function every ask goes through, an ask of build-agent for any module but the builder's own (issue 1851, A12, ADR 0453 §5) 2026-10-22 19:05:04 +02:00
jschoubben b71425867f Register a module from a repository the catalogue does not build at the controller's terminal alone, from its done trunk key, naming exactly its repository and path (issue 1851, A11b, ADR 0501 §4) 2026-10-22 19:05:04 +02:00
jschoubben c7039d81e1 Refuse build, rebuild and replay --register by name, naming what replaced them, and delete the hand builds (issue 1851, A11a, ADR 0501 §5) 2026-10-22 19:05:04 +02:00
jschoubben f76da2ce9c Ask the checker for a try of a branch head, not build-agent, and read its word in today's answer (issue 1851, A11, ADR 0501 §5) 2026-10-22 19:05:04 +02:00
jschoubben 7f6d418205 Mark the walks the integration tests drive around the one ask left as the builder's own, and expect no ask of a moved module (issue 1851, A5, A6) 2026-10-22 19:05:04 +02:00
jschoubben 0aec975d19 Keep an assignment of a module whose key is coming pending on that key and ask no build; nothing asks a request again at start (issue 1851, A10) 2026-10-22 19:05:04 +02:00
jschoubben 9da79ef8c3 Move no module whose fingerprint is unchanged, and let a key's build stand on the held build-agent build of the same digests at the switch (issue 1851, A9) 2026-10-22 19:05:04 +02:00
jschoubben 7743bd0d4b Delete the walk's reuse of a check's build: a walk takes keys and asks build-agent only for the builder's own module (issue 1851, A8) 2026-10-22 19:05:04 +02:00
jschoubben 62268cf43d Take a walk's modules from the builder's done keys and ask no build but the builder's own: a key building or unreferenced holds the tier and says so, a failed key fails the walk in its words (issue 1851, A6, A7) 2026-10-22 19:05:04 +02:00
jschoubben ea3c8ad7f6 Register a new module from the builder's first done key and ask no first build; its condition reads the key and names a manifest word this controller does not read (issue 1851, A5a, A5b) 2026-10-22 19:05:04 +02:00
jschoubben d984f0a55d Take a module in from the builder's done key of its trunk fingerprint: one record of the key's artifacts, its manifest judged against the catalogue trunk's predefined keys (issue 1851, A3, A4b, A4c) 2026-10-22 19:05:04 +02:00
jschoubben f00b6aff8b Judge a recorded build, not build-agent's outcome, and read a registration's warrant from the merge a key names, so a done key registers under the same rules (issue 1851, A4a) 2026-10-22 19:05:04 +02:00
jschoubben eff0d5cbf4 Vendor the R4e mesh-sdk member's head for its shared launcher rule and the manifest word paths (issue 1851; re-pinned to the final head in E) 2026-10-22 19:05:04 +02:00
jschoubben 1ed7e34155 Read the builder's keys, the indexer's values by commit and its manifests, so the controller can take a module from a done key (issue 1851, A1, A2) 2026-10-22 19:05:04 +02:00
jschoubben abe356a262 Merge main and take the SDK at the milestone's mesh-sdk head 8892a117, which holds main's SDK and the milestone's (issue 1840)
mesh/merge-gate head abe356a262e8 merged into trunk 41c96287578e — pass: group feat/upgrade-safety composed: every machine composes with the change as it d…
mesh/repo-check head abe356a262e8 merged into trunk 41c96287578e beside mesh-catalog#450@86ba54b0 — pass: its merge-check.sh passed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792694058124013462 (novox/mesh-controller at 96441ea2 into ma…
mesh/delivery-group group feat/upgrade-safety delivered: every member is delivered
mesh/check failed: unit: ..test: node:internal/modules/esm/resolve:241 | abe356a262e8 on 41c96287578e | unit 3372 run, 0 kept, 7 skipped | integration…
2026-10-22 18:14:14 +02:00
mesh-tools 41c9628757 Merge pull request 'Pin mesh-sdk at the fix that refuses an unresolvable machine alone (issue 1833)' (#437) from fix/1833-one-machine-alone into main 2026-10-22 16:10:20 +00:00
jschoubben 3e3400f510 Take the SDK at the milestone's mesh-sdk head f8b8f3a6, which the delivery group's members must pin exactly (issue 1840) 2026-10-22 18:07:38 +02:00
jschoubben e022a3fd6f issue 1840/S3: look the declared packages up by name in the mirror's integration test, since every node also runs the mesh's own modules and their packages 2026-10-22 17:52:40 +02:00
jschoubben 684a8783e5 Merge remote-tracking branch 'origin/main' into feat/upgrade-safety 2026-10-22 17:40:00 +02:00
mesh-tools 11e364bd43 Merge pull request 'Hold a first merge whose new module the running build-agent refuses for an unknown manifest word, name it, and ask it again when build-agent is delivered (issue 1173)' (#438) from fix/1173-held-first-build into main 2026-10-22 15:28:11 +00:00
jschoubben 1e9b87e03a Hold every refused module of a first merge and ask each again by its own path (issue 1173)
mesh/merge-gate head 1e9b87e03acd merged into trunk b8f77577fed2 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 1e9b87e03acd merged into trunk b8f77577fed2 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
A held merge was asked again by the rule of a first merge, which finds nothing once a sibling module of the
same merge registered, so it stayed held forever; and a second refusal in a merge already held fell through to
a failed build. The hold now keeps a list of module, path and word, a refusal in a held waiting merge joins it,
and the delivery of build-agent asks each held path again on the merge's Approve warrant.
2026-10-22 16:50:06 +02:00
jschoubben 45a09f90e3 Carry a held merge's new announcement into its record (issue 1173)
HoldMerge replaced the event on the flag only; the record kept the old one.
2026-10-22 16:33:28 +02:00
jschoubben 4538d05732 Ask a held first build again when a delivery of build-agent ends (issue 1173)
ADR 0482 section 5: the build waiting for a build-agent that reads its manifest is asked once the plan delivering it ends.
2026-10-22 16:19:45 +02:00
jschoubben 980d396d40 Keep a first merge whose manifest word the running build-agent does not read, and name it (issue 1173)
ADR 0482 section 5: a merged manifest the running reader refuses waits for the newer reader, never failed.
2026-10-22 16:18:39 +02:00
mesh-tools b8f77577fe Merge pull request 'Say a delivery's start and end with the machines its merges declare, and word a needed new head as its author's (ADR 0505)' (#436) from feat/quiet-deliveries into main 2026-10-22 14:01:22 +00:00
jschoubben 28a278c146 Take mesh-sdk at bd7731f: a machine that does not resolve keeps its name and its address in the mesh as well as its tunnel (issue 1833)
mesh/repo-check head 28a278c1462a merged into trunk 11e364bd439f — pass: its merge-check.sh passed
mesh/merge-gate head 28a278c1462a merged into trunk 11e364bd439f — pass: group fix/1833-one-machine-alone composed: every machine composes with the change …
mesh/delivery delivered
mesh/delivery-group group fix/1833-one-machine-alone delivered: every member is delivered
mesh/check failed: g14: go.mod of novox/mesh-controller#437 cannot be read to place the mesh-sdk its tests read: nats: no responders available for req…
2026-10-22 15:27:27 +02:00
jschoubben c34ab0e43a Take mesh-sdk at c154bde: a machine that does not resolve keeps its place on the private network, so it alone is refused (issue 1833)
One machine whose modules could not be resolved left the private network uncomposable when it was the hub, and every
send to every machine was held. The SDK now keeps a refused machine at the place the controller already holds for it.
2026-10-22 15:04:47 +02:00
jschoubben 771cc24fcb Take the SDK whose process can delete a key (issue 1840/S1) 2026-10-22 14:37:18 +02:00
jschoubben b3786cc67e Merge remote-tracking branch 'origin/main' into feat/upgrade-safety
# Conflicts:
#	go.mod
#	go.sum
#	testdata/beside/CAPTURED
#	vendor/modules.txt
2026-10-22 14:25:33 +02:00
jschoubben b7d03b6897 Word a delivery that needs its author's new head as its author's, with no Stop (ADR 0505 §5)
mesh/repo-check head b7d03b68974d merged into trunk 8f2d22db0caa beside mesh-catalog#444@09e749bc — pass: its merge-check.sh passed
mesh/merge-gate head b7d03b68974d merged into trunk 8f2d22db0caa — pass: group feat/quiet-deliveries composed: every machine composes with the change as it…
mesh/delivery delivered
mesh/delivery-group group feat/quiet-deliveries delivered: every member is delivered
mesh/check checking b7d03b68974d on 8f2d22db0caa
2026-10-22 14:25:26 +02:00
jschoubben 21954ad1bc issue 1840/S3b: prove a process may watch its own state's bucket and no other (AC9) 2026-10-22 14:23:58 +02:00
jschoubben 485eb44de4 issue 1840/S3: keep each node's declared packages on the bus, so the pacman verbs can say what a node declares 2026-10-22 14:23:58 +02:00
jschoubben 8d562f9b22 Say a walk's start and end again with the machines whose declaration its merges change (ADR 0505 §2) 2026-10-22 14:23:54 +02:00
mesh-tools 8f2d22db0c Merge pull request 'Move route-proxy's program into its module and run it on the pinned alpine image (issues 1633, 1634, 1635, ADR 0480 §1-§4, §11)' (#433) from feat/r4d into main 2026-10-22 12:19:07 +00:00
jschoubben cc8a74fce4 issue 1840/S3: take the SDK that names declared-packages 2026-10-22 14:18:39 +02:00
mesh-tools aa9da94260 Merge pull request 'A build node leaves its builds' images behind no more, and takes no work under a floor of free space (issue 1834)' (#435) from fix/1834-build-node-disk into main 2026-10-22 12:05:14 +00:00
jschoubben 01f615aa0a Read the bus release from the nats module's upstream artifact, so the tests' bus is still held to the mesh's once the Dockerfile is gone (issue 1640)
mesh/merge-gate head 01f615aa0ad6 merged into trunk aa9da942606d — pass: group feat/r4d composed: every machine composes with the change as it did without …
mesh/delivery delivered
mesh/repo-check head 01f615aa0ad6 merged into trunk aa9da942606d — pass: its merge-check.sh passed
mesh/delivery-group group feat/r4d delivered: every member is delivered
mesh/check failed: unit: ..test: node:internal/modules/esm/resolve:241 | 01f615aa0ad6 on aa9da942606d | unit 3345 run, 0 kept, 7 skipped | integration…
mesh-catalog #434 runs the nats server as the upstream image named in module.json (ADR 0497) and
deletes modules/nats/Dockerfile, so the check failed opening it. The manifest's upstream nats
artifact is read first; the Dockerfile only for a catalogue from before the move (the captured copy).
2026-10-22 13:38:00 +02:00
jschoubben f501eaca43 Remove an image's pushed names with its build tag, and take no work under the free-space floor (issue 1834)
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-controller@01f615aa0ad6 (merged as 8f2d22db into main, walk plan-17926707…
mesh/merge-gate head f501eaca4305 merged into trunk 4463991e0609 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head f501eaca4305 merged into trunk 4463991e0609 — pass: its merge-check.sh passed
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
The registry push tags the image a second time, so removing only the build tag freed nothing.
The held image a comparison pulls goes after it is read, and a node under the floor leaves the ask
to another build node instead of taking it and failing.
2026-10-22 13:19:58 +02:00
jschoubben 995f8237ce Free a build node's disk after every build and refuse work under a floor of free space (issue 1834)
A full / made every check on shanks end 'store never became ready' with the reason erased by --rm.
Builds now remove their local image tag, narrowed source and context clone; a build or check
refuses under 20 GiB free on / and the workspace, saying so; the throwaway store is no longer
started with --rm and a store that does not come up reports its exit code and last 20 log lines.
2026-10-22 13:14:00 +02:00
jschoubben 707bfba4ea Check an image's Dockerfile place instead of refusing the image kind (ADR 0487 clause 13)
mesh/delivery-group group feat/r4d rejected: a member's own check failed
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 707bfba4ea36 merged into trunk 4463991e0609 beside mesh-catalog#434@68183ee4 — fail: its merge-check.sh failed: --- FAIL: TestTheBusTh…
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: i3/cmd/i3-tools.TestTheMainFileAndEveryModulesContributionLoadTogether: manifest_test.go:247: expected the launcher, clipboar…
2026-10-22 11:45:06 +02:00
mesh-tools 4463991e06 Merge pull request 'Compose a pull request's check with the config values held for it (issue 1826, ADR 0502)' (#434) from fix/1826-held-config into main 2026-10-22 09:38:35 +00:00
jschoubben 4357b3688f Tell mesh-config what a head declares at every head, so a head that reverts a key drops its held value (issue 1826)
mesh/repo-check head 4357b3688f58 merged into trunk 3d4ad6fefb7c — pass: its merge-check.sh passed
mesh/merge-gate head 4357b3688f58 merged into trunk 3d4ad6fefb7c — pass: group fix/1826-held-config composed: every machine composes with the change as it …
mesh/delivery delivered
mesh/delivery-group group fix/1826-held-config delivered: every member is delivered
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-22 11:07:34 +02:00
jschoubben 03927a6f6c Carry a config value held for a pull request's new key into its composed check (issue 1826)
The controller tells mesh-config what a pull request declares, asks the approved held values with each check, and the build agent and merge gate lay them over the change's composition only: a key the trunk does not declare yet can then be approved before the merge (ADR 0502).
2026-10-22 11:01:12 +02:00
jschoubben 499ff5c218 Move the route proxy into its module and run it from the trust image, on mesh-sdk's public membership (issues 1633, 1634, 1635)
mesh/repo-check out of date: judged beside mesh-catalog#434 at c948f3a61c74, it is now at 7451e7edf4e5; judged again on a recheck or when it is merged
mesh/merge-gate group feat/r4d: composed again for the group's new heads
mesh/delivery superseded: a newer head of the same pull request
mesh/delivery-group group feat/r4d rejected: a member's own check failed
mesh/check failed: unit: i3/cmd/i3-tools.TestTheMainFileAndEveryModulesContributionLoadTogether: manifest_test.go:247: expected the launcher, clipboar…
A bundle is built only from its module's directory (ADR 0480 clause 4), so the proxy leaves
examples/ and the controller's internal/broker; the broker keeps no second copy of what the
proxy shares with mesh-sdk. The image and its Dockerfile go, and a test holds that only
build-agent declares one (clause 11).
2026-10-22 09:53:56 +02:00
mesh-tools 3d4ad6fefb Merge pull request 'Limit what a build or check starts on a build node: a per-node build slice (issue 1798)' (#432) from fix/1798-limit-build-resources into main 2026-10-21 23:20:57 +00:00
jschoubben 6bf38072ca Say in the build-agent settings that changing one restarts the build slice (issue 1798)
mesh/merge-gate head 6bf38072cacf merged into trunk a929df9b0ec3 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 6bf38072cacf merged into trunk a929df9b0ec3 — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-22 01:01:56 +02:00
jschoubben b174d743c0 Name the build slice without a dash so it sits beside user.slice, and restart it when its limits change (issue 1798)
mesh/merge-gate building on ace since 00:57
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-22 00:56:48 +02:00
jschoubben 37119ea31f Fill the build slice's limits as the mesh fills a module's template today (issue 1798)
mesh/merge-gate building on ace since 00:53
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: integration: github.com/novox/mesh-controller/internal/identity.TestACertificateChainsToTheMeshsOwnAuthority: authority_test.go:45:…
2026-10-22 00:38:02 +02:00
jschoubben af2c6b5e94 Run builds and checks in a per-node build slice with CPU, IO and memory limits (issue 1798) 2026-10-22 00:24:45 +02:00
mesh-tools a929df9b0e Merge pull request 'The checker runs beside today's check: checks wait while a delivery builds; a group's SDK member merges; a flaky test fixed' (#430) from feat/the-checker into main 2026-10-21 21:46:26 +00:00
mesh-tools a3c965b9a0 Merge pull request 'Remove a check's throwaway containers with their anonymous volumes, so a build node's disk does not fill (issue 1772)' (#431) from fix/1772-check-volumes into main 2026-10-21 21:15:07 +00:00
jschoubben b9ec1919b8 Name plans-building among the controller's readable states and pin the SDK that does (issue 1687, ADR 0483 clause 2)
mesh/repo-check head b9ec1919b865 merged into trunk a3c965b9a0d5 beside mesh-catalog#423@12ff1cfb — pass: its merge-check.sh passed
mesh/merge-gate head b9ec1919b865 merged into trunk a3c965b9a0d5 — pass: group feat/the-checker composed: every machine composes with the change as it did …
mesh/delivery delivered
mesh/delivery-group group feat/the-checker delivered: every member is delivered
2026-10-21 22:47:04 +02:00
jschoubben d491356f59 Remove a build's throwaway containers with their anonymous volumes, so a build node's disk does not fill (issue 1772)
mesh/merge-gate head d491356f59dd merged into trunk 1f3fdf311167 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head d491356f59dd merged into trunk 1f3fdf311167 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-21 22:40:16 +02:00
jschoubben 47e1819101 Merge remote-tracking branch 'origin/fix/1106-sdk-member-merges' into feat/the-checker
mesh/repo-check out of date: judged beside mesh-sdk#66 at 78364e318bdc, it is now at 580da61a9107; judged again on a recheck or when it is merged
mesh/delivery-group group feat/the-checker checking: 4 of 4 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head 47e1819101a1 merged into trunk 1f3fdf311167 — fail: group feat/the-checker composed: 1 problem(s) the change brings; the first: mesh-d…
2026-10-21 22:01:42 +02:00
jschoubben 0e25ed7c5a Merge remote-tracking branch 'origin/fix/1399-person-word-test' into feat/the-checker 2026-10-21 22:01:42 +02:00
jschoubben 79e2923dc8 Merge remote-tracking branch 'origin/fix/1652-checks-wait' into feat/the-checker
# Conflicts:
#	internal/broker/testdata/composed.conf
#	internal/link/builds_nats.go
2026-10-21 22:01:42 +02:00
mesh-tools 1f3fdf3111 Merge pull request 'Judge a duplicate hand-out before marking a build in hand, and never hand every build back over a stale mark (issue 1768)' (#429) from fix/1768-in-hand-marker into main 2026-10-21 19:59:30 +00:00
jschoubben 9902a3f16c Name every group member's trunk in the composed verdict, the announced one when the seat names none (issue 1106) 2026-10-21 21:57:30 +02:00
jschoubben 11815c0fb3 Have the delivery test's view hear a plan a verb let go, so a loaded machine cannot read it still waiting (issue 1399) 2026-10-21 21:45:58 +02:00
mesh-tools f973cefd7f Merge pull request 'Ask the operator for a held move and a failed release, and send no image the store lacks (issues 1732, 1320, 1742)' (#428) from fix/1732-held-moves-asked into main 2026-10-21 19:07:51 +00:00
jschoubben 696300b9e2 Judge a duplicate hand-out before the in-hand claim, and treat a stale mark as stale (issue 1768)
mesh/merge-gate head 696300b9e280 merged into trunk f973cefd7fd3 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 696300b9e280 merged into trunk f973cefd7fd3 — pass: its merge-check.sh passed
mesh/delivery delivered
A dropped duplicate left the in-hand marker, so every later claim failed and every ask spent its deliveries.
2026-10-21 20:57:12 +02:00
mesh-tools 98155e327b Merge pull request 'Take mesh-sdk at 07a6592: an item's text places a config value (issue 1658), and fit the unit level in its bound (issue 1763)' (#420) from fix/1656-xterm-config into main 2026-10-21 18:36:39 +00:00
jschoubben d6db334840 Offer no Release on a machine that also holds a person's own hold (issue 1732, review round 1)
mesh/repo-check head d6db334840a5 merged into trunk 98155e327b6d beside mesh-host#129@09c77d57 — pass: its merge-check.sh passed
mesh/merge-gate head d6db334840a5 merged into trunk 98155e327b6d — pass: group fix/1732-held-moves-asked composed: every machine composes with the change a…
mesh/delivery delivered
mesh/delivery-group group fix/1732-held-moves-asked delivered: every member is delivered
2026-10-21 20:15:58 +02:00
jschoubben 69d016bfdf Merge remote-tracking branch 'origin/main' into fix/1732-held-moves-asked 2026-10-21 20:15:13 +02:00
jschoubben a58b38cee3 Keep the missing image out of the condition's summary (issue 1742) 2026-10-21 20:06:10 +02:00
jschoubben 9d260412b3 Merge remote-tracking branch 'origin/main' into fix/1656-xterm-config
mesh/repo-check head 9d260412b330 merged into trunk 44033f4b6d72 beside mesh-catalog#416@b9dcf19c — pass: its merge-check.sh passed
mesh/merge-gate head 9d260412b330 merged into trunk 44033f4b6d72 — pass: group fix/1656-xterm-config composed: every machine composes with the change as it…
mesh/delivery-group group fix/1656-xterm-config failed: a member failed
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792609673527907956 (novox/mesh-controller at f973cefd into ma…
2026-10-21 19:59:59 +02:00
jschoubben 99610e9569 Cut the controller's slowest unit tests (a source scan that regexed every file, resolver stand-in timeouts, a wait bound) so mesh-controller's unit level fits its bound (issue 1763) 2026-10-21 19:59:58 +02:00
jschoubben dc06af1f5a Leave a module out of a send when the store lacks its image, and say it (issue 1742) 2026-10-21 19:43:36 +02:00
jschoubben b62fbeabbf Ask a held move and the builds held after a failed release, and say them on the plan's line (issue 1732, 1320) 2026-10-21 19:43:36 +02:00
jschoubben 373aefd14e Take mesh-sdk at 07a6592, which counts an item's text as a placement of a config value (issue 1658)
mesh/delivery-group group fix/1656-xterm-config checking: composed check out of date: novox/mesh-sdk@c8e35097e861 was judged against trunk 85b5a2687e6a, the tr…
mesh/merge-gate building on ace since 19:52
mesh/repo-check out of date: judged against trunk 22c9815f6522, the trunk is now 44033f4b6d72; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 19:43:32 +02:00
jschoubben f66458fe25 Merge remote-tracking branch 'origin/main' into fix/1656-xterm-config 2026-10-21 19:43:02 +02:00
jschoubben 99da867a98 Merge origin/main into fix/1656-xterm-config 2026-10-21 19:16:11 +02:00
jschoubben 69a67876c2 Take mesh-sdk at 6f63275, which counts an item's text as a placement of a config value (issue 1658)
mesh/delivery-group group fix/1656-xterm-config rejected: a member's own check failed
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 69a67876c269 merged into trunk d96c9f56a3f1 beside mesh-catalog#416@f178aab6 — fail: its merge-check.sh failed: --- FAIL: TestModuleCh…
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 17:03:35 +02:00
2137 changed files with 23453 additions and 703763 deletions
+3 -1
View File
@@ -2,9 +2,11 @@
# A built binary. The lab writes one here when pointed at the repository root by mistake;
# built artifacts belong in build/, which is already ignored.
/mesh-builder
/mesh-controller
/postgres-provisioner
/redis-provisioner
# And where `go build` in cmd/mesh-controller leaves it (novox/hq issue 233).
/cmd/mesh-controller/mesh-controller
# No repository commits its dependencies (issue 1305).
vendor/
+3 -5
View File
@@ -21,17 +21,15 @@ ARG GO_BASE=golang@sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7
FROM ${GO_BASE} AS build
WORKDIR /src
# **Nothing is fetched** (novox/hq to-be 45 Phase 1): every dependency is in vendor/, committed, so
# the image builds from this repository alone — the host's validator among them, whose module no
# public proxy is asked for. Dependencies first, so a change to the source does not re-copy them.
# **No dependency is committed** (issue 1305): go.sum names each one's version and checksum, and the build
# resolves them with -mod=mod. Dependencies first, so a change to the source does not re-copy them.
COPY go.mod go.sum ./
COPY vendor/ vendor/
# `COPY . .` brings `.git` along, so Go is told not to stamp the commit: the version is said by -X, and a
# rebuild of an unchanged program is the same bytes (novox/hq ADR 0267 rule 6, issue 363).
COPY . .
ARG VERSION=development
RUN CGO_ENABLED=0 GOFLAGS=-mod=vendor GOPROXY=off go build -trimpath -buildvcs=false \
RUN CGO_ENABLED=0 GOFLAGS=-mod=mod go build -trimpath -buildvcs=false \
-ldflags "-s -w -X main.version=${VERSION}" \
-o /mesh-controller ./cmd/mesh-controller
+3 -44
View File
@@ -46,17 +46,6 @@ image:
@echo
@docker image inspect $(IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The builder ships as an image too, because it is a module the mesh assigns rather than a program
# somebody starts on a machine by hand.
BUILDER_IMAGE ?= mesh-builder:$(VERSION)
BUILDER_DEV_TAG ?= mesh-builder:development
builder-image:
@test -n "$(GO_BASE)" || { echo "no GO_BASE; pass GO_BASE=<image>"; exit 1; }
docker build --build-arg GO_BASE=$(GO_BASE) -f cmd/mesh-builder/Dockerfile -t $(BUILDER_IMAGE) -t $(BUILDER_DEV_TAG) .
@echo
@docker image inspect $(BUILDER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The provisioner ships as an image too, because it is the thing that makes a sealed credential
# true on a machine -- and the mesh cannot, having discarded the plaintext.
PROVISIONER_IMAGE ?= mesh-provision-postgres:$(VERSION)
@@ -90,35 +79,6 @@ redis-provisioner-image:
@echo
@docker image inspect $(REDIS_PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The proxy that turns a route grant into traffic reaching a workload.
PROXY_IMAGE ?= mesh-route-proxy:$(VERSION)
PROXY_DEV_TAG ?= mesh-route-proxy:development
proxy-image:
docker build --build-arg GO_BASE=$(GO_BASE) -f modules/route-proxy/Dockerfile -t $(PROXY_IMAGE) -t $(PROXY_DEV_TAG) .
@echo
@docker image inspect $(PROXY_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The whole check. Raises a database, runs the repository's own check against it -- merge-check.sh,
# the very script `mesh/repo-check` runs -- and takes the database down again, including when the
# check fails, which is why the teardown is not conditional and the script's exit status is the
# target's.
#
# **It calls the script rather than restating it** (novox/hq issue 431): this target used to list its
# own steps, its formatting step walked vendor/ where the script's does not, and it failed on
# third-party code no change could fix -- so a developer's check and the gate disagreed, and the steps
# after formatting never ran through it. The script is the one list of steps; this target only gives it
# a database (MESH_TEST_POSTGRES, exported above).
#
# **Two levels** (issue 895): the unit level, `go test ./...` untagged, timed and held to a bound; and the integration
# level, `-tags integration`, the tests that start a bus, a store or another process. Each is said with its time.
#
# **Packages in parallel, under the race detector, each test on a bus of its own** (internal/testbus),
# as the script runs them: a bus per test, of the release the mesh runs, makes the suite the same in any
# order, and the timeout bounds a hang to a failure with a stack, never a stalled gate.
check: postgres
@sh merge-check.sh ; status=$$? ; docker rm -f $(PG_CONTAINER) >/dev/null 2>&1 || true ; exit $$status
# The unit level alone (untagged: no bus, no store, no other process), which a developer runs continuously. The tests
# that need a bus or the store carry `//go:build integration` and run in `check`, not here; without a database the
# store's tests would skip rather than fail anyway.
@@ -128,14 +88,13 @@ test:
vet:
go vet ./...
# Quick steps for a developer, not part of `check`. The directories gofmt reads must be the ones
# merge-check.sh lists, never `.`, which walks vendor/ (novox/hq issue 431).
# A quick step for a developer: the checker runs the build module's own formatting and vet (ADR 0501 §5, D7).
fmt:
@unformatted=$$(gofmt -l cmd internal examples) ; \
if [ -n "$$unformatted" ] ; then echo "not gofmt'd:" ; echo "$$unformatted" ; exit 1 ; fi
postgres:
@docker rm -f $(PG_CONTAINER) >/dev/null 2>&1 || true
@docker rm -f -v $(PG_CONTAINER) >/dev/null 2>&1 || true
@docker run -d --name $(PG_CONTAINER) -e POSTGRES_PASSWORD=check \
-p 127.0.0.1:$(PG_PORT):5432 $(PG_IMAGE) >/dev/null
@printf 'waiting for postgres'
@@ -147,7 +106,7 @@ postgres:
echo ' — never came up' ; docker logs $(PG_CONTAINER) | tail -20 ; exit 1
postgres-stop:
@docker rm -f $(PG_CONTAINER) >/dev/null 2>&1 || true
@docker rm -f -v $(PG_CONTAINER) >/dev/null 2>&1 || true
clean:
rm -rf build/
-6
View File
@@ -1,6 +0,0 @@
# Release 57: config and provisioning
This branch carries the work of hq design 57 (ADR 0348, ADR 0349) in this repository. Each package
merges here through its own reviewed pull request; the branch reaches main as one pull request, in one
delivery group with the branch of the same name in mesh-controller, mesh-host, mesh-sdk, mesh-catalog
and hq, after a rehearsal in the lab. This file is deleted by the last commit before that pull request.
-29
View File
@@ -1,29 +0,0 @@
#!/bin/sh
# The unit tests of every Go module of its own below this repository's root, run from the root, for merge-check.sh
# and for the test that tries it on a tree (merge_check_nested_test.go). Its own file so that the test needs no
# particular merge-check.sh: the check that judges a change to merge-check.sh is main's.
set -eu
cgo="${cgo:-0}"
race="${race:-}"
# nested_modules vets and tests every Go module of its own below the root, `modules/<name>/go.mod` (novox/hq ADR 0420
# §5, issue 1025): `./...` stops at a nested go.mod, so a module that lives whole in this repository would have its
# unit tests run by nobody. None is a pass. Each runs in its own directory, outside the vendor directory this
# repository's own packages are built from; a nested module vendors what the forge holds (its own `vendor/`, so the
# check runs offline), as this repository's root module does (issue 1019). A failing module fails the check; the others are still run.
nested_modules() {
nested=0
for gomod in modules/*/go.mod; do
[ -e "$gomod" ] || continue
dir=$(dirname "$gomod")
echo "nested module $dir:"
if ! (cd "$dir" && GOFLAGS= CGO_ENABLED=0 go vet ./... &&
GOFLAGS= CGO_ENABLED="${cgo:-0}" go test ${race:-} -count=1 -timeout 30m ./...); then
echo "nested module $dir FAILED"
nested=1
fi
done
return $nested
}
nested_modules
-22
View File
@@ -1,22 +0,0 @@
ARG ALPINE_BASE=alpine:3
ARG GO_BASE=golang:1.25-alpine
# The builder, as a module ships one.
#
# Not FROM scratch, unlike the control plane: this one runs git and a container client, so it
# needs a filesystem with them on it. That is the honest cost of a machine whose job is to build —
# and it is why building is a MODULE on a machine that has a runtime rather than something the
# control plane does (novox/hq ADR 0005).
FROM ${GO_BASE} AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -buildvcs=false -ldflags '-s -w' -o /mesh-builder ./cmd/mesh-builder
FROM ${ALPINE_BASE}
# git to clone what it is asked to build, and the docker client to build and push it. The daemon
# is the machine's, reached through its socket — a build machine shares the runtime it was given
# rather than running one inside itself.
RUN apk add --no-cache git docker-cli
COPY --from=build /mesh-builder /usr/local/bin/mesh-builder
ENTRYPOINT ["/usr/local/bin/mesh-builder"]
-146
View File
@@ -1,146 +0,0 @@
package main
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"time"
)
// **The build agent says whether it is busy, and being asked drains it** (novox/hq ADR 0335 §1, §5). Its
// container declares `busy` as `mesh-builder busy`: the node-engine runs it inside the container before it
// replaces it. It exits 0 while a build is in hand and 1 when none is; and from the first ask the agent takes
// no new build request, so its answer ends once the build in hand ends — a replaced builder finishes the check
// it runs, instead of the build queue handing it out again from the start (issue 594).
//
// Two marks in the container's own temporary directory, as the ready mark (ready.go): a new container starts
// without them, and a restarted one removes them before it takes work again.
//
// - in hand: written by the agent when it claims a request, before it reads whether it is drained; removed
// when the build ends.
// - drained: written by the ask, before it reads whether a build is in hand.
//
// Each side writes its own mark before it reads the other's, so an ask racing a fetch sees the build in hand,
// or the agent sees the drain and hands the request back; never neither.
func inHandMark() string { return filepath.Join(os.TempDir(), "mesh-builder.in-hand") }
func drainMark() string { return filepath.Join(os.TempDir(), "mesh-builder.drained") }
// The drain lapses when no ask has come for drainLapse: the node-engine asks every 20 seconds for as long as it
// really waits, so an agent whose replacement was withdrawn — the declaration put back, the walk stopped —
// takes work again rather than staying drained for ever (review of ADR 0335). And an agent drained for longer
// than the node-engine's bound and the slack says it is not ready: the replacement it was drained for never
// came, which is a fault to see.
const (
drainLapse = 2 * time.Minute
drainTooLong = 15*time.Minute + 5*time.Minute
)
// drainedAt is when the drain began and when it was last asked, from its mark; false when there is no mark.
func drainedAt() (began, asked time.Time, ok bool) {
info, err := os.Stat(drainMark())
if err != nil {
return time.Time{}, time.Time{}, false
}
asked = info.ModTime()
began = asked
if raw, err := os.ReadFile(drainMark()); err == nil {
if t, err := time.Parse(time.RFC3339, strings.TrimSpace(string(raw))); err == nil && !t.After(asked) {
began = t
}
}
return began, asked, true
}
// drained says the agent was asked whether it is busy within drainLapse: it takes no new build request.
func drained() bool { return drainedSince(time.Now()) }
func drainedSince(now time.Time) bool {
_, asked, ok := drainedAt()
return ok && now.Sub(asked) < drainLapse
}
// drainedTooLong is the health check's second half: nil unless the agent has been drained, asked all along, for
// longer than its node-engine's bound and the slack.
func drainedTooLong(now time.Time) error {
began, asked, ok := drainedAt()
if !ok || now.Sub(asked) >= drainLapse || now.Sub(began) <= drainTooLong {
return nil
}
return fmt.Errorf("drained since %s and still asked: the replacement its node-engine waits for has not come "+
"within %s (novox/hq ADR 0335)", began.UTC().Format(time.RFC3339), drainTooLong)
}
// claim marks a request in hand, unless the agent is drained: then nothing is in hand, and the request goes
// back to the queue.
func claim() bool {
// Made, never written over: one build at a time, and a mark already there is a build in hand.
f, err := os.OpenFile(inHandMark(), os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644)
if err != nil {
// Unmarked, an ask would say idle while this build runs: the build is not taken.
fmt.Fprintf(os.Stderr, "cannot mark a build in hand (%s), so it is handed back: %v\n", inHandMark(), err)
return false
}
_, err = f.WriteString(time.Now().UTC().Format(time.RFC3339) + "\n")
if closeErr := f.Close(); err == nil {
err = closeErr
}
if err != nil {
_ = os.Remove(inHandMark())
fmt.Fprintf(os.Stderr, "cannot mark a build in hand (%s), so it is handed back: %v\n", inHandMark(), err)
return false
}
if drained() {
_ = os.Remove(inHandMark())
fmt.Fprintln(os.Stderr, "asked whether busy, so this agent takes no new build: handed back")
return false
}
return true
}
// holding says what is in hand, for the ask to print: written once the request is read.
func holding(what string) {
_ = os.WriteFile(inHandMark(), []byte(time.Now().UTC().Format(time.RFC3339)+" "+what+"\n"), 0o644)
}
// released ends the build in hand.
func released() {
if err := os.Remove(inHandMark()); err != nil && !errors.Is(err, os.ErrNotExist) {
fmt.Fprintf(os.Stderr, "cannot remove the mark of the build in hand (%s): %v\n", inHandMark(), err)
}
}
// undrained clears both marks at the start of a holder: a container started again is asked afresh.
func undrained() error {
for _, mark := range []string{drainMark(), inHandMark()} {
if err := os.Remove(mark); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove %s: %w", mark, err)
}
}
return nil
}
// busy is the ask: it drains the agent, then says whether a build is in hand. Nil — exit 0 — while one is.
func busy() (string, error) {
// The mark keeps when the drain began while it is asked within drainLapse, and is written again on every
// ask: its time is the last ask.
began := time.Now().UTC()
if was, _, ok := drainedAt(); ok && drained() {
began = was.UTC()
}
if err := os.WriteFile(drainMark(), []byte(began.Format(time.RFC3339)+"\n"), 0o644); err != nil {
// Not drained, it could take the next build: said, and answered busy only for what is in hand now.
fmt.Fprintf(os.Stderr, "cannot mark this agent drained (%s): %v\n", drainMark(), err)
}
held, err := os.ReadFile(inHandMark())
if err != nil {
return "", errors.New("idle: no build in hand, and this agent takes no new one")
}
since, what, _ := strings.Cut(strings.TrimSpace(string(held)), " ")
if what == "" {
what = "a build"
}
return fmt.Sprintf("busy: %s, in hand since %s; taking no new build", what, since), nil
}
-162
View File
@@ -1,162 +0,0 @@
package main
import (
"os"
"strings"
"sync"
"testing"
"time"
)
// `mesh-builder busy` answers 0 while a build is in hand and non-zero when idle, and after its first ask the
// agent takes no new build request (novox/hq ADR 0335 §5).
func TestBusyAnswersWhileABuildIsInHandAndDrainsTheAgent(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
if drained() {
t.Fatal("an agent never asked is drained")
}
// A build is claimed and runs.
if !claim() {
t.Fatal("an agent never asked refused a build")
}
holding("novox/mesh-controller (check-280)")
said, err := busy()
if err != nil || !strings.Contains(said, "novox/mesh-controller (check-280)") {
t.Fatalf("an agent with a build in hand did not answer busy with what it holds: %q %v", said, err)
}
// From that ask on it takes nothing new: the fetch is not made, and one fetched is handed back.
if !drained() {
t.Fatal("the ask did not drain the agent")
}
if claim() {
t.Fatal("a drained agent took a new build")
}
// The build in hand is still in hand after the refused claim, and busy goes on saying so.
if _, err := busy(); err != nil {
t.Fatalf("a refused claim ended the build in hand: %v", err)
}
// It ends: idle, and still drained.
released()
if _, err := busy(); err == nil {
t.Fatal("an agent with nothing in hand answered busy")
}
if claim() {
t.Fatal("a drained agent took a build after its last one ended")
}
// Started again, it is asked afresh.
if err := undrained(); err != nil {
t.Fatal(err)
}
if drained() || !claim() {
t.Fatal("a holder started again is still drained")
}
released()
}
// An ask racing a fetch sees the build in hand, or the agent sees the drain and hands the request back: never
// neither, which would replace the container under a build just taken.
func TestAnAskRacingAFetchNeverMissesTheBuild(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
for i := 0; i < 200; i++ {
if err := undrained(); err != nil {
t.Fatal(err)
}
var claimed bool
var askErr error
var wg sync.WaitGroup
wg.Add(2)
go func() { defer wg.Done(); claimed = claim() }()
go func() { defer wg.Done(); _, askErr = busy() }()
wg.Wait()
if claimed && askErr != nil {
t.Fatalf("round %d: the agent took a build and the ask said idle", i)
}
released()
}
}
// A drain no longer asked lapses: an agent whose replacement was withdrawn after the first ask (a put-back, a
// revert, a stopped walk) takes work again after drainLapse, rather than staying drained with its health green.
func TestADrainNoLongerAskedLapses(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
if !drained() || claim() {
t.Fatal("an agent just asked takes work")
}
// The last ask was three minutes ago.
old := time.Now().Add(-3 * time.Minute)
if err := os.Chtimes(drainMark(), old, old); err != nil {
t.Fatal(err)
}
if drained() {
t.Fatal("a drain no longer asked for three minutes still holds")
}
if !claim() {
t.Fatal("an agent whose drain lapsed takes no work")
}
released()
}
// An agent drained, and asked all along, for longer than its node-engine's bound and the slack is not ready: the
// replacement it was drained for never came. One asked again after a lapse starts a new drain.
func TestAnAgentDrainedTooLongIsNotReady(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
taking("mesh-build-machine")
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
now := time.Now()
if err := drainedTooLong(now); err != nil {
t.Fatalf("an agent drained a moment ago is not ready: %v", err)
}
// The drain began 25 minutes ago, and the node-engine asked a moment ago.
began := now.Add(-25 * time.Minute).UTC().Format(time.RFC3339)
if err := os.WriteFile(drainMark(), []byte(began+"\n"), 0o644); err != nil {
t.Fatal(err)
}
err := drainedTooLong(time.Now())
if err == nil || !strings.Contains(err.Error(), "has not come") {
t.Fatalf("an agent drained for 25 minutes reads ready: %v", err)
}
// Each ask keeps when the drain began.
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
if err := drainedTooLong(time.Now()); err == nil {
t.Fatal("an ask started the drain again")
}
// The asks stopped: the drain lapsed, and so did the fault.
old := time.Now().Add(-3 * time.Minute)
if err := os.Chtimes(drainMark(), old, old); err != nil {
t.Fatal(err)
}
if err := drainedTooLong(time.Now()); err != nil {
t.Fatalf("a lapsed drain still reads not ready: %v", err)
}
}
// An ask this build does not know exits non-zero: an older image asked `busy` reads idle, not busy for the
// node-engine's whole bound.
func TestAnUnknownCommandFails(t *testing.T) {
was := os.Args
t.Cleanup(func() { os.Args = was })
os.Args = []string{"mesh-builder", "some-later-word"}
if err := run(); err == nil {
t.Fatal("an unknown command exited 0")
}
os.Args = []string{"mesh-builder", "help"}
if err := run(); err != nil {
t.Fatalf("help failed: %v", err)
}
}
-38
View File
@@ -1,38 +0,0 @@
package main
import (
"testing"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// The build seat hands the check the judge cache the build agent declares (MESH_JUDGE_CACHE), never a path of its own.
func TestTheBuildSeatHandsTheCheckTheJudgeCacheItDeclares(t *testing.T) {
t.Setenv("MESH_JUDGE_CACHE", "/var/lib/mesh/build-agent/judge-cache")
if got := judgeCache(); got != "/var/lib/mesh/build-agent/judge-cache" {
t.Fatalf("the judge cache the manifest declares is read as %q", got)
}
t.Setenv("MESH_JUDGE_CACHE", "")
if got := judgeCache(); got != "" {
t.Fatalf("with no judge cache declared, %q is used", got)
}
}
// The siblings a check is asked to read reach the build seat's spec and, as read, the outcome (novox/hq ADR 0382).
func TestTheSiblingsAnAskNamesReachTheSpecAndTheOutcome(t *testing.T) {
spec := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main",
Siblings: []link.CheckSibling{{Repository: "novox/mesh-controller", Number: 346, Head: "a", Base: "main", Dir: "mesh-controller"}},
SiblingsUnknown: "why"}})
if len(spec.Siblings) != 1 || spec.Siblings[0] != (builder.Sibling{Repository: "novox/mesh-controller", Number: 346,
Head: "a", Base: "main", Dir: "mesh-controller"}) || spec.SiblingsUnknown != "why" {
t.Fatalf("the spec reads %+v", spec)
}
out := siblingsOut([]builder.JudgedSibling{{Repository: "novox/mesh-controller", Number: 346, Head: "a", Base: "main", BaseCommit: "b"}})
if len(out) != 1 || out[0].BaseCommit != "b" || out[0].Number != 346 {
t.Fatalf("the outcome reads %+v", out)
}
if plain := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "o", Repo: "r"}}); plain.Siblings != nil {
t.Fatalf("a request that names none has siblings %+v", plain.Siblings)
}
}
-433
View File
@@ -1,433 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go/micro"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// What a holder of the build seat answers for, on its own machine (novox/hq ADR 0219).
//
// **The queue is the controller's; the build running here is this machine's.** The controller can
// see and change what waits in the seat's queue, but an ask a holder already took is a process tree
// on this machine and containers in this machine's runtime, and only this machine can end them. So
// the holder serves four verbs on the seat's subjects for this machine: what it is building, kill
// it, pause, resume.
//
// **One build at a time** (ADR 0190), which is also what builder.Said assumes — a package global
// set per build — so "the build running here" is one or none, and kill names it by id so a call
// that arrives as one build ends and the next begins cannot end the wrong one.
// holder is this machine's state as a holder of the build seat.
type holder struct {
on, seat string
// workspace is where the paused flag is kept, so a holder restarted while paused stays paused
// rather than silently taking work again.
workspace string
// say publishes this machine's state: whether it takes work (link.HolderState).
say func(link.HolderState) error
// remove runs what a kill needs outside the build: the containers left behind.
remove builder.Runner
// scratch is where check builds are kept (novox/hq ADR 0350 §5): what the week's sweep lets go of. Nil where
// this machine publishes to no artifact store.
scratch builder.Scratch
mu sync.Mutex
paused bool
running *running
}
// running is the build this machine is doing.
type running struct {
request link.BuildRequest
step string
started time.Time
cancel context.CancelFunc
killed bool
// returned is the build's own work having ended, before a kill or not; outcome is what was then
// announced, and announced whether it went out.
returned bool
outcome string
announced bool
done chan struct{}
}
// pausedFile is where the flag lives in the workspace.
func pausedFile(workspace string) string { return filepath.Join(workspace, ".mesh-builder-paused") }
// newHolder reads the paused flag the workspace keeps.
func newHolder(on, seat, workspace string, say func(link.HolderState) error) *holder {
h := &holder{on: on, seat: seat, workspace: workspace, say: say, remove: plainRun}
if _, err := os.Stat(pausedFile(workspace)); err == nil {
h.paused = true
}
return h
}
// Paused is asked by the taking loop before every fetch.
func (h *holder) Paused() bool {
h.mu.Lock()
defer h.mu.Unlock()
return h.paused
}
// setPaused records the flag in the workspace first and then in memory, so what this holder says
// it is and what it would be after a restart never differ.
func (h *holder) setPaused(paused bool) error {
path := pausedFile(h.workspace)
if paused {
if err := os.MkdirAll(h.workspace, 0o755); err != nil {
return err
}
if err := os.WriteFile(path, []byte(time.Now().UTC().Format(time.RFC3339)+"\n"), 0o644); err != nil {
return fmt.Errorf("cannot keep the paused flag in %s: %w", path, err)
}
} else if err := os.Remove(path); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove the paused flag %s: %w", path, err)
}
h.mu.Lock()
h.paused = paused
h.mu.Unlock()
h.announce()
return nil
}
// announce says whether this machine takes work. Never fatal: the flag is kept either way, and the
// controller reading an older state is a plan read as late rather than a build lost.
func (h *holder) announce() {
if h.say == nil {
return
}
if err := h.say(link.HolderState{On: h.on, Paused: h.Paused(), At: time.Now().UTC().Format(time.RFC3339Nano)}); err != nil {
fmt.Fprintf(os.Stderr, "cannot say whether this machine takes builds: %v\n", err)
}
}
// stateWhilePaused says this machine's state at start, and again every while it stays paused, so
// a pause outlives the events stream's retention.
func (h *holder) stateWhilePaused(ctx context.Context, every time.Duration) {
h.announce()
tick := time.NewTicker(every)
defer tick.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
if h.Paused() {
h.announce()
}
}
}
}
// expireChecks lets go of every check build older than a week (novox/hq ADR 0350 §5), at start and every while
// after. Every holder sweeps the one store; what one let go of, another finds gone.
func (h *holder) expireChecks(ctx context.Context, every time.Duration, now func() time.Time) {
sweep := func() {
if h.scratch == nil {
return
}
n, err := h.scratch.Expire(ctx, builder.CheckBuildsKept, now())
switch {
case err != nil:
fmt.Fprintf(os.Stderr, "check builds older than %s: %d let go of, and not all: %v\n", builder.CheckBuildsKept, n, err)
case n > 0:
fmt.Fprintf(os.Stderr, "check builds older than %s: %d check(s)' builds let go of\n", builder.CheckBuildsKept, n)
}
}
sweep()
tick := time.NewTicker(every)
defer tick.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
sweep()
}
}
}
// begin records the build this machine took, with the cancel that ends it.
func (h *holder) begin(request link.BuildRequest, cancel context.CancelFunc) *running {
r := &running{request: request, started: time.Now(), cancel: cancel, done: make(chan struct{})}
h.mu.Lock()
h.running = r
h.mu.Unlock()
return r
}
// end clears it, and lets a kill waiting on it know it is over.
func (h *holder) end(r *running) {
h.mu.Lock()
if h.running == r {
h.running = nil
}
h.mu.Unlock()
close(r.done)
}
// stepped records the step a build is at, for `current`.
func (h *holder) stepped(r *running, step string) {
h.mu.Lock()
r.step = step
h.mu.Unlock()
}
// currentBuild is what `current` answers.
type currentBuild struct {
On string `json:"on"`
Paused bool `json:"paused"`
Running *struct {
ID string `json:"id"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
Step string `json:"step,omitempty"`
Started string `json:"started"`
Elapsed string `json:"elapsed"`
} `json:"running,omitempty"`
Said string `json:"said"`
}
func (h *holder) current() currentBuild {
h.mu.Lock()
defer h.mu.Unlock()
out := currentBuild{On: h.on, Paused: h.paused}
taking := "taking builds"
if h.paused {
taking = "paused, taking no new build"
}
if h.running == nil {
out.Said = fmt.Sprintf("%s is building nothing; %s", h.on, taking)
return out
}
r := h.running
elapsed := time.Since(r.started).Round(time.Second)
out.Running = &struct {
ID string `json:"id"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
Step string `json:"step,omitempty"`
Started string `json:"started"`
Elapsed string `json:"elapsed"`
}{r.request.ID, r.request.Repository, r.request.Path, r.request.Ref, r.step,
r.started.UTC().Format(time.RFC3339), elapsed.String()}
out.Said = fmt.Sprintf("%s is building %s (%s) at %s for %s; %s",
h.on, r.request.Repository, r.request.ID, orNothing(r.step), elapsed, taking)
return out
}
// The bounds a kill keeps: each pass removing containers, and the wait for the build to end between
// them. The worst case — 15s, 20s, 15s — is inside what the controller waits for the answer
// (killAnswer in the controller's queue.go, 75s).
var (
killRemoves = 15 * time.Second
killWaits = 20 * time.Second
)
// kill ends the build with this id, if it is the one running here and still working: its context
// cancelled — which kills each command's process group — and the containers it started removed by
// their label, once at once and again after the build has ended, so one created while it was being
// killed is not left. The build's own goroutine announces it failed, killed by hand, and settles the
// ask so it is not redelivered; the answer says whether that happened.
func (h *holder) kill(id string) (string, error) {
h.mu.Lock()
r := h.running
if r == nil || r.request.ID != id {
doing := "nothing"
if r != nil {
doing = r.request.ID
}
h.mu.Unlock()
return "", fmt.Errorf("%s is not building %s; it is building %s. `queue` says where an ask is", h.on, id, doing)
}
if r.returned {
h.mu.Unlock()
return "", fmt.Errorf("%s on %s has already ended on its own and is saying how; `builds` shows it", id, h.on)
}
r.killed = true
cancel, done := r.cancel, r.done
h.mu.Unlock()
cancel()
removed, removeErr := h.removeContainers(id)
ended := false
select {
case <-done:
ended = true
case <-time.After(killWaits):
}
again, againErr := h.removeContainers(id)
removed += again
if removeErr == nil {
removeErr = againErr
}
containers := fmt.Sprintf("%d container(s) it started removed", removed)
if removeErr != nil {
containers = "its containers could not all be listed or removed: " + removeErr.Error()
}
if !ended {
return fmt.Sprintf("killed %s on %s: %s; the build has not finished ending yet — `builds` says when "+
"its outcome is in", id, h.on, containers), nil
}
h.mu.Lock()
outcome, announced := r.outcome, r.announced
h.mu.Unlock()
if !announced {
return fmt.Sprintf("killed %s (%s) on %s: its commands ended and %s, and its outcome could not be "+
"announced — the ask is not settled and will be handed out again", id, r.request.Repository, h.on,
containers), nil
}
return fmt.Sprintf("killed %s (%s) on %s: its commands ended, %s, and its outcome announced as failed, %s — "+
"settled, so it is not handed to another machine", id, r.request.Repository, h.on, containers, outcome), nil
}
// removeContainers is one pass of removing what the build left, bounded.
func (h *holder) removeContainers(id string) (int, error) {
cleanup, stop := context.WithTimeout(context.Background(), killRemoves)
defer stop()
return builder.RemoveContainersOf(cleanup, h.remove, id)
}
// removeLeftBehind removes, once, what earlier holders on this machine left — called before anything is
// taken, so none of it is this holder's — and says what it did. A runtime that cannot be asked is said
// and the holder takes work all the same: what is left is waste, not a reason to stop building.
func (h *holder) removeLeftBehind() int {
cleanup, stop := context.WithTimeout(context.Background(), killRemoves)
defer stop()
n, err := builder.RemoveLeftBehind(cleanup, h.remove)
switch {
case err != nil:
fmt.Fprintf(os.Stderr, "could not look for containers earlier builds left on %s: %v\n", h.on, err)
case n > 0:
fmt.Fprintf(os.Stderr, "removed %d container(s) earlier builds left on %s\n", n, h.on)
}
return n
}
// returned records that the build's work ended, and says whether a kill came first — only then is
// the build killed; an error it ended with on its own is its own outcome.
func (h *holder) returned(r *running) bool {
h.mu.Lock()
defer h.mu.Unlock()
r.returned = true
return r.killed
}
// said records the outcome announced, and whether it went out, for a kill to answer with.
func (h *holder) said(r *running, outcome string, announced bool) {
h.mu.Lock()
r.outcome, r.announced = outcome, announced
h.mu.Unlock()
}
// handlers are the seat's verbs, as this machine answers them.
func (h *holder) handlers() map[string]link.ToolHandler {
return map[string]link.ToolHandler{
"current": func(context.Context, json.RawMessage) (any, error) { return h.current(), nil },
"kill": func(_ context.Context, raw json.RawMessage) (any, error) {
var args struct {
ID string `json:"id"`
}
if err := json.Unmarshal(raw, &args); err != nil || strings.TrimSpace(args.ID) == "" {
return nil, errors.New("kill needs the build's id")
}
said, err := h.kill(strings.TrimSpace(args.ID))
if err != nil {
return nil, err
}
return map[string]any{"said": said}, nil
},
"pause": func(context.Context, json.RawMessage) (any, error) {
if err := h.setPaused(true); err != nil {
return nil, err
}
said := h.on + " is paused: it takes no new build until resumed"
if c := h.current(); c.Running != nil {
said += "; " + c.Running.ID + " runs on and finishes"
}
return map[string]any{"said": said, "paused": true}, nil
},
"resume": func(context.Context, json.RawMessage) (any, error) {
if err := h.setPaused(false); err != nil {
return nil, err
}
return map[string]any{"said": h.on + " takes builds again", "paused": false}, nil
},
}
}
func orNothing(s string) string {
if s == "" {
return "its start"
}
return s
}
// plainRun runs a command outside any build: no line reaches a build's log, because the build whose
// log it would be is the one being ended.
func plainRun(ctx context.Context, dir, name string, args ...string) (string, error) {
cmd := exec.CommandContext(ctx, name, args...)
cmd.Dir = dir
out, err := cmd.CombinedOutput()
if err != nil {
return string(out), fmt.Errorf("%s %s: %w: %s", name, strings.Join(args, " "), err, strings.TrimSpace(string(out)))
}
return string(out), nil
}
// announcement is what this holder answers discovery with (novox/hq ADR 0195, ADR 0197): this
// machine's verbs of the seat, in the shape every tool runtime announces a seat's verb — kind seat,
// the module answering, the seat, scope node, the machine, its description and argument schema — so
// the console finds `<node>/node-build-agent.kill` by searching, as it finds any seat's verb.
//
// One service per machine, named for the seat and identified by the machine, so the answer is this
// machine's four verbs and nothing more. The verbs are the compiled seat row's: a build machine has no
// store, and what it serves is what this binary was built to serve.
func announcement(seat, module, node string) micro.Info {
s, _ := catalogue.SeatNamed(seat)
var endpoints []micro.EndpointInfo
for _, v := range s.Serves {
schema, _ := json.Marshal(v.Input)
endpoints = append(endpoints, micro.EndpointInfo{
Name: seat + "__" + v.Name,
Subject: link.NodeSeatToolSubject(seat, v.Name, node),
// The queue group the verbs are served in, as every runtime announces its own.
QueueGroup: "seat." + seat,
Metadata: map[string]string{
"kind": "seat", "module": module, "tool": v.Name, "seat": seat, "scope": "node",
"node": node, "interchangeable": "false", "description": v.Description, "schema": string(schema),
},
})
}
return micro.Info{
ServiceIdentity: micro.ServiceIdentity{Name: seat, ID: node, Version: "0.1.0",
Metadata: map[string]string{"seat": seat, "scope": "node", "node": node, "module": module}},
Description: "what the build running on " + node + " is, and ending, pausing and resuming it (novox/hq ADR 0219)",
Endpoints: endpoints,
}
}
// moduleOf is the module a credential was issued for: its user is `<node>.<module>`.
func moduleOf(user string) string {
if _, module, ok := strings.Cut(user, "."); ok && module != "" {
return module
}
return "build-agent"
}
-172
View File
@@ -1,172 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// A holder paused stays paused across its restart: the flag is kept in its workspace (novox/hq ADR
// 0219), and what it says about itself follows.
func TestAPausedHolderStaysPausedAcrossARestart(t *testing.T) {
workspace := t.TempDir()
var said []link.HolderState
h := newHolder("ace", link.TheBuildMachine, workspace, func(s link.HolderState) error {
said = append(said, s)
return nil
})
if h.Paused() {
t.Fatal("a new holder starts paused")
}
if _, err := h.handlers()["pause"](context.Background(), json.RawMessage(`{}`)); err != nil {
t.Fatal(err)
}
if !h.Paused() || len(said) != 1 || !said[0].Paused || said[0].On != "ace" {
t.Fatalf("paused: %v, said %+v", h.Paused(), said)
}
again := newHolder("ace", link.TheBuildMachine, workspace, nil)
if !again.Paused() {
t.Fatal("restarted, the holder forgot it was paused")
}
if _, err := again.handlers()["resume"](context.Background(), json.RawMessage(`{}`)); err != nil {
t.Fatal(err)
}
if newHolder("ace", link.TheBuildMachine, workspace, nil).Paused() {
t.Fatal("resumed, the holder came back paused")
}
}
// kill ends the build with that id — its context, which ends its commands — removes what it left by
// label, and refuses an id it is not building.
func TestKillEndsTheBuildRunningHereAndNoOther(t *testing.T) {
h := newHolder("ace", link.TheBuildMachine, t.TempDir(), nil)
var removed []string
h.remove = func(_ context.Context, _ string, name string, args ...string) (string, error) {
removed = append(removed, name+" "+strings.Join(args, " "))
if args[0] == "ps" {
return "c1\n", nil
}
return "", nil
}
kill := h.handlers()["kill"]
if _, err := kill(context.Background(), json.RawMessage(`{"id":"build-1"}`)); err == nil {
t.Fatal("killed a build while none ran")
}
building, cancel := context.WithCancel(context.Background())
r := h.begin(link.BuildRequest{ID: "build-1", Repository: "novox/a"}, cancel)
h.stepped(r, "image")
if c := h.current(); c.Running == nil || c.Running.ID != "build-1" || c.Running.Step != "image" {
t.Fatalf("current says %+v", c)
}
if _, err := kill(context.Background(), json.RawMessage(`{"id":"build-2"}`)); err == nil ||
!strings.Contains(err.Error(), "build-1") {
t.Fatalf("killed another id: %v", err)
}
// The build's own goroutine: it ends when its context does, as a build's commands do, and
// announces what came of it.
go func() {
<-building.Done()
if h.returned(r) {
h.said(r, link.KilledByHand, true)
}
h.end(r)
}()
answer, err := kill(context.Background(), json.RawMessage(`{"id":"build-1"}`))
if err != nil {
t.Fatal(err)
}
if building.Err() == nil {
t.Fatal("the build's context was not cancelled")
}
if !r.killed {
t.Error("the build is not marked killed, so it would be announced as an ordinary failure")
}
said := answer.(map[string]any)["said"].(string)
if !strings.Contains(said, "2 container(s)") || !strings.Contains(said, "announced as failed") || !strings.Contains(said, link.KilledByHand) {
t.Errorf("kill said %q", said)
}
// Removed at the kill and again once the build had ended: a container made in between is caught.
if len(removed) != 4 || !strings.Contains(removed[0], "label=mesh.build=build-1") || !strings.Contains(removed[2], "label=mesh.build=build-1") {
t.Errorf("removed %v", removed)
}
if c := h.current(); c.Running != nil {
t.Errorf("after the kill current says %+v", c)
}
}
// A holder announces this machine's verbs of the seat as the console reads a seat's verb, and no more.
func TestAHolderAnnouncesItsMachinesVerbsForTheConsole(t *testing.T) {
info := announcement(link.TheBuildMachine, moduleOf("ace.build-agent"), "ace")
if info.Name != "node-build-agent" || info.ID != "ace" || len(info.Endpoints) != 4 {
t.Fatalf("announced %s/%s with %d endpoints", info.Name, info.ID, len(info.Endpoints))
}
kill := info.Endpoints[1]
md := kill.Metadata
if kill.Subject != "mesh.seat.node-build-agent.tool.kill.ace" || kill.QueueGroup != "seat.node-build-agent" || md["kind"] != "seat" || md["seat"] != "node-build-agent" ||
md["scope"] != "node" || md["node"] != "ace" || md["tool"] != "kill" || md["module"] != "build-agent" ||
!strings.Contains(md["description"], "killed by hand") || !strings.Contains(md["schema"], `"id"`) {
t.Fatalf("kill is announced as %+v", kill)
}
body, err := json.Marshal(info)
if err != nil || len(body) > 8*1024 {
t.Fatalf("the answer is %d bytes (%v)", len(body), err)
}
}
// A build that ended on its own as the kill arrived says what it did: the kill is refused, and its
// own error is its outcome. One whose outcome could not be announced is not said to be settled.
func TestAKillArrivingAfterTheBuildEndedIsRefusedAndAnUnannouncedKillSaysSo(t *testing.T) {
h := newHolder("ace", link.TheBuildMachine, t.TempDir(), nil)
h.remove = func(context.Context, string, string, ...string) (string, error) { return "", nil }
_, cancel := context.WithCancel(context.Background())
r := h.begin(link.BuildRequest{ID: "build-1"}, cancel)
if killed := h.returned(r); killed {
t.Fatal("a build nobody killed reads as killed")
}
if _, err := h.kill("build-1"); err == nil || !strings.Contains(err.Error(), "ended on its own") {
t.Fatalf("killed a build that had ended: %v", err)
}
h.end(r)
building, cancel := context.WithCancel(context.Background())
r = h.begin(link.BuildRequest{ID: "build-2"}, cancel)
go func() {
<-building.Done()
if h.returned(r) {
h.said(r, link.KilledByHand, false)
}
h.end(r)
}()
said, err := h.kill("build-2")
if err != nil || strings.Contains(said, "announced as failed") || !strings.Contains(said, "could not be announced") {
t.Fatalf("kill said %q (%v)", said, err)
}
}
// **Issue 306**: a holder starting removes the containers earlier builds left on its machine, through
// the runner a kill uses, and a runtime that cannot be asked does not stop it.
func TestAHolderStartingRemovesWhatEarlierBuildsLeftHere(t *testing.T) {
h := newHolder("novox", link.TheBuildMachine, t.TempDir(), nil)
var removed []string
h.remove = func(_ context.Context, _ string, name string, args ...string) (string, error) {
removed = append(removed, name+" "+strings.Join(args, " "))
if args[0] == "ps" {
return "c1 build-1\nc2 build-1\nc3 check-here-2\n", nil
}
return "", nil
}
if n := h.removeLeftBehind(); n != 2 || len(removed) != 2 || removed[1] != "docker rm -f c1 c2" {
t.Fatalf("removed %d: %v", n, removed)
}
h.remove = func(context.Context, string, string, ...string) (string, error) {
return "", errors.New("no runtime")
}
if n := h.removeLeftBehind(); n != 0 {
t.Errorf("a runtime that cannot be asked removed %d", n)
}
}
-936
View File
@@ -1,936 +0,0 @@
// mesh-builder — the thing a build machine runs.
//
// It takes work from the mesh, turns a repository into artifacts, publishes them, and says what
// came out. It is **not** the control plane and it is **not** the host:
//
// - the control plane decides and never touches a machine. Building runs commands on one, and
// what the control plane may send a machine is bounded by the declaration language
// (novox/hq ADR 0005). "Run this build" is not in it, and widening the language so it could
// be would make the control plane able to run anything anywhere.
// - the host applies declarations and holds no opinion about what they contain. A host that
// also built things would need a container runtime and git, on every machine, to do something
// almost none of them will ever do.
//
// So it is a module: a program a machine runs because the mesh told it to, holding its own broker
// credential and nothing else. Compromise of a build machine is compromise of a build machine.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"net/url"
"os"
"os/signal"
"sort"
"strings"
"sync"
"sync/atomic"
"syscall"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// version is set at build time.
var version = "development"
func main() {
if err := run(); err != nil {
fmt.Fprintf(os.Stderr, "mesh-builder: %v\n", err)
os.Exit(1)
}
}
const usage = `mesh-builder — builds modules for the mesh
It consumes build requests and answers with what it made. Nothing is listened on and nothing
is dialled except the broker.
MESH_BROKER_FILE a file the mesh sealed to this machine holding the same
MESH_REGISTRY host:port to publish artifacts to, when the mesh has not said
MESH_BINDING a file the mesh wrote saying where the artifact store is
MESH_PACKAGE_BINDING a file the mesh wrote saying where the package registry is
MESH_NPM_TOKEN_FILE a file the mesh sealed holding the token for it
MESH_GENESIS_PACKAGE_REGISTRY the document the installer sealed at genesis, read while no
binding is present (novox/hq ADR 0336)
MESH_NPM_REGISTRY a package registry URL, when the mesh has not said (a person, the bootstrap)
MESH_NPM_TOKEN the token for it, likewise
MESH_NPM_SCOPE the scope it answers for (default: @novox)
MESH_WORKSPACE where to clone and build (default: a temporary directory)
It also builds one module and stops, which is how a mesh is raised — before there is a
broker to take work from or a registry to publish into:
mesh-builder build <repository> [--path P] [--ref COMMIT] [--registry HOST:PORT]
Without --registry the artifacts stay in this machine's container runtime, named by the
digest of their own configuration. The result is printed as JSON.
mesh-builder ready
exits 0 once the agent running in this container is bound to its build work and pulls it, and
1 before: its container's health check.
mesh-builder busy
exits 0 while the agent running in this container has a build in hand, and 1 when it has none;
from the first ask the agent takes no new build: its container's busy command, asked before the
container is replaced (novox/hq ADR 0335).
`
func run() error {
if len(os.Args) > 1 {
switch os.Args[1] {
case "version":
fmt.Println(version)
return nil
case "build":
return buildOnce(context.Background(), os.Args[2:])
case "ready":
// The container's health check (novox/hq issue 544): exits non-zero until the agent takes work, and
// once it has been drained longer than its node-engine would wait to replace it (ADR 0335).
if err := ready(); err != nil {
return err
}
return drainedTooLong(time.Now())
case "busy":
// The container's busy command (novox/hq ADR 0335): drains the agent, and exits non-zero once
// nothing is in hand.
said, err := busy()
if err != nil {
return err
}
fmt.Println(said)
return nil
case "help", "-h", "--help":
fmt.Print(usage)
return nil
default:
// **An ask this build does not know fails** (novox/hq ADR 0335): an image older than `busy`, asked
// it, must read as idle — exiting 0 would hold its container for the node-engine's whole bound.
fmt.Fprint(os.Stderr, usage)
return fmt.Errorf("%q is not a command this mesh-builder knows", os.Args[1])
}
}
if err := notReady(); err != nil {
return err
}
// **The builder's own version is part of every reuse key** (novox/hq ADR 0350 §1): a builder that builds
// differently never takes a build another one made.
builder.Version = builder.SelfVersion(version)
if err := undrained(); err != nil {
return err
}
credential, err := brokerFrom()
if err != nil {
return err
}
registry, err := whereToPublish()
if err != nil {
return err
}
workspace := os.Getenv("MESH_WORKSPACE")
if workspace == "" {
workspace = os.TempDir() + "/mesh-builder"
}
// **The mesh's name for this machine, not the container's.** A build is reported to the rest
// of the mesh, and a report whose origin reads `104cb10e105b` names something no other module
// can look up. The mesh already knows the answer and has a way to say it — `${machine:name}`
// in the environment file this module is handed — so the hostname is only what is left when
// nobody said.
on := os.Getenv("MESH_NODE")
if on == "" {
hostname, err := os.Hostname()
if err != nil {
return fmt.Errorf("this build machine has no name: nothing said MESH_NODE and the host would not say either: %w", err)
}
on = hostname
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
js, seat, err := dialFor(credential)
if err != nil {
return err
}
defer js.Close()
// **This machine's holder: paused or not, and the build it is running** (novox/hq ADR 0219). The
// paused flag is read from the workspace before anything is taken, so a holder restarted while
// paused takes nothing.
h := newHolder(on, seat, workspace, func(state link.HolderState) error {
body, err := json.Marshal(state)
if err != nil {
return err
}
_, err = js.Context().Publish(link.BuildPausedOf(seat, on), body)
return err
})
if h.Paused() {
fmt.Fprintf(os.Stderr, "paused (kept in %s): taking no build until resumed\n", pausedFile(workspace))
}
// **What an earlier holder here left goes before anything is taken** (novox/hq issue 306): a holder
// recreated mid-check left the check's store and bus running, and the ask went to another machine.
h.removeLeftBehind()
// Paused or drained, it takes nothing new (novox/hq ADR 0219, ADR 0335): a drained agent finishes the build
// in hand and then says idle, and its container is replaced between builds.
machine := link.MachineOverNATSWith(js, on, seat, link.MachineOptions{
Paused: func() bool { return h.Paused() || drained() },
Taking: func() { taking(seat) },
Claim: claim,
})
defer machine.Close()
// The seat's verbs, on this machine's subjects. Only the seat that declares them: the retired
// one serves none, and a subscription its holder has no grant for would be refused for ever.
if seat == link.TheBuildMachine {
stopServing, err := link.OverNATS{Conn: js.Conn()}.ServeNodeSeatTools(seat, on, h.handlers(),
log.New(os.Stderr, "", 0))
if err != nil {
return err
}
defer stopServing()
// And says so, for the console to find (novox/hq ADR 0197).
stopAnnouncing, err := link.OverNATS{Conn: js.Conn()}.Announce(
announcement(seat, moduleOf(credential.User), on), log.New(os.Stderr, "", 0))
if err != nil {
return err
}
defer stopAnnouncing()
go h.stateWhilePaused(ctx, time.Hour)
}
fmt.Fprintf(os.Stderr, "building for the mesh, publishing to %s\n", registry)
publisher := builder.Registry{Address: registry, Run: builder.Command}
// Check builds are let go of after a week (novox/hq ADR 0350 §5).
h.scratch = publisher
go h.expireChecks(ctx, time.Hour, time.Now)
return machine.Take(ctx, func(ctx context.Context, work link.Build) {
defer released()
holding(work.Request().Repository + " (" + work.Request().ID + ")")
answer(ctx, publisher, on, workspace, work, h)
})
}
// dialFor opens this machine's link to whichever bus the mesh is on, and says which build seat it
// holds.
//
// **One place chooses**, as everywhere else the bus change went (novox/hq ADR 0116 step 5): a build
// machine told about both would take work from one and answer on the other, and every log line would
// say it was fine.
func dialFor(credential Credential) (*broker.JetStream, string, error) {
// **The credential names the bus, and there is one** (novox/hq ADR 0131, design 28 task 5.5).
// A credential for the mesh's bus carries user, password and fingerprint beside the address,
// and that is enough to dial it, pinned.
if !credential.onTheNewBus() {
return nil, "", fmt.Errorf("the credential at hand names %q, which is not the mesh's bus", credential.URL)
}
// Named for what it is, not the controller whose code dials it (novox/hq issue 327).
host, _ := os.Hostname()
js, err := broker.DialPinned(credential.natsURL(), credential.Fingerprint, nats.Name("build agent on "+host))
if err != nil {
return nil, "", err
}
// **The seat this machine serves is the one its credential claims** (novox/hq ADR 0190, the
// handover): the mesh issues a build machine's credential naming the seat its module claims,
// and one binary serves the old role as `builder` and the new as `build-agent` from that alone.
seat := link.BuildSeatClaimed(credential.seatsClaimed())
fmt.Fprintf(os.Stderr, "taking build work as a holder of %s\n", seat)
return js, seat, nil
}
// answer does one build and says what happened, whichever way it went.
func answer(ctx context.Context, publisher builder.Publisher, on, workspace string, work link.Build, h *holder) {
request := work.Request()
// **Its own context, so it can be killed alone** (novox/hq ADR 0219): cancelled by `kill`, it ends
// this build's commands and nothing else; the machine's own context ending — a SIGTERM — still
// reaches it through the parent, and that keeps today's meaning below.
building, cancel := context.WithCancel(ctx)
defer cancel()
var mine *running
if h != nil {
mine = h.begin(request, cancel)
defer h.end(mine)
}
// **First thing, and to stdout.** A build request that arrives and produces no visible line until
// it either finishes or fails is indistinguishable from one that never arrived — which cost a long
// diagnosis against a running mesh, chasing "the handler never fired" when the truth was only that
// the handler said nothing until the end.
fmt.Fprintf(os.Stderr, "a build request arrived for %s (%s)\n", request.Repository, request.ID)
// **Everything a build says goes two ways**: to stderr, as always, and onto the bus as the
// role's own events under the build's id (novox/hq ADR 0157) — so whoever asked, and anybody
// watching, reads the same lines this container's log holds, live, and after the fact from the
// stream. Said first, before anything runs, so a build that hangs is one that visibly started.
// One line at a time: a check's builds for its merge say their lines while its tests say theirs (novox/hq
// issue 716).
var saying sync.Mutex
say := func(step, message string) {
saying.Lock()
defer saying.Unlock()
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
work.Say(step, message)
if mine != nil && step != "run" && step != "output" {
h.stepped(mine, step)
}
}
builder.Said = say
defer func() { builder.Said = nil }()
if err := work.Began(ctx); err != nil {
fmt.Fprintf(os.Stderr, "cannot say a build started: %v\n", err)
}
// **A running check watches its pull request's key** (novox/hq ADR 0406 §4, issue 974): when the head or the trunk
// commit in it moves, the check is for a state nobody wants any more, and ends itself — nothing is asked of the forge
// (issue 726). A group's check, a try and a build watch nothing.
var superseded atomic.Bool
if w, ok := work.(interface {
WatchPull(context.Context, func(link.PullKey)) (func(), error)
}); ok && request.Check != nil {
stopWatching, err := w.WatchPull(building, func(k link.PullKey) {
superseded.Store(true)
say("superseded", fmt.Sprintf("the pull request's key moved to %.12s on %.12s while this checked %.12s on %.12s: stopped",
k.Head, k.Trunk, request.Ref, request.Check.Trunk))
cancel()
})
if err != nil {
fmt.Fprintf(os.Stderr, "the key of %s/%s#%d could not be watched, so its check runs on if the key moves: %v\n",
request.Check.Owner, request.Check.Repo, request.Check.Number, err)
} else {
defer stopWatching()
}
}
result := link.BuildResult{
ID: request.ID, Repository: request.Repository, Path: request.Path,
Ref: request.Ref, On: on, Source: request.Source, DryRun: request.DryRun,
}
what := "building " + request.Repository
if request.Path != "" {
what += " at " + request.Path
}
if request.Ref != "" {
what += " on " + request.Ref
}
say("build", what)
npmrc, err := packagesFrom(on)
if err == nil {
say("packages", saysRegistry(npmrc))
}
registryHere, isRegistry := publisher.(builder.Registry)
var built builder.Result
if request.Check != nil {
// **A pull request's merge check, not a build** (novox/hq to-be 45 §9): nothing is built,
// published or registered; the verdict is the outcome.
result.Checked = request.Check
registry := ""
if r, ok := publisher.(builder.Registry); ok {
registry = r.Address
}
spec := checkSpecOf(request)
// The judge cache the build agent declares as its own data, never in the workspace: only judge builds and
// mesh-lab's replays mount it, never a check's containers, so no pull request's script can write what a judge
// is built with or kept as, or what the replays run (novox/hq ADR 0358 §6, issue 709).
spec.JudgeCache = judgeCache()
// What the merge would move, built by the check for the walk to take (novox/hq ADR 0350 §1): into the
// scratch namespace. **Never with the package registry's credential**: this is code nobody has approved
// yet, and a recipe could read the credential it is handed (issue 462). A module that resolves packages
// at build time has no reuse key anyway, so nothing is lost by refusing it here.
if isRegistry && len(spec.Builds) > 0 {
spec.BuildWith = &builder.CheckBuilding{Publish: publisher, Scratch: registryHere, Held: request.Held,
Running: request.Check.Running, Seats: request.Seats, Named: namedOf(request.Check.Named)}
}
var v builder.CheckVerdict
v, err = builder.Check(building, builder.Command, spec, workspace, registry, forgeFrom(), say)
if err == nil {
result.Check = &link.CheckOutcome{Verdict: v.Verdict, Summary: v.Summary, Report: v.Report,
Took: v.Took.Round(time.Second).String(), Gate: layerOf(v.Gate), RepoCheck: layerOf(v.Repo),
Head: v.Head, Trunk: v.Trunk, Beside: v.Beside, Built: checkBuiltOf(v.Built),
Validator: v.Validator, Siblings: siblingsOut(v.Siblings), SiblingsUnknown: v.SiblingsUnknown, JudgeFrom: v.JudgeFrom}
// What each of a group's heads was judged as (novox/hq ADR 0323 §6).
for _, m := range v.Members {
result.Check.Members = append(result.Check.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo,
Commit: m.Head, Trunk: m.Trunk})
}
// A branch tried before its pull request reads its outcome from this log, whole (novox/hq issue 686).
if request.Check.Try {
if line, err := json.Marshal(triedOf(v)); err == nil {
say(link.StepTried, string(line))
}
}
}
} else if err == nil {
// The package-registry credential is a build input, so it is resolved before the clone: a
// build that could not have resolved its dependencies is refused in front of the reason, not
// after a clone that then fails at npm ci.
// Every container it starts is labelled with its id, so a kill finds what outlived the
// docker client (ADR 0219).
if request.Reuse != nil && isRegistry && !request.DryRun {
// **A walk's build takes the check's build of the same key** (novox/hq ADR 0350 §2).
reuse := builder.Reuse{Scratch: registryHere}
for _, p := range request.Reuse.Pulls {
reuse.Pulls = append(reuse.Pulls, builder.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number})
}
// Only what a verdict names is taken (ADR 0350 §2).
reuse.Named = namedOf(request.Reuse.Builds)
built, err = builder.BuildReusing(building, builder.Labelled(builder.Command, request.ID), publisher, reuse,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats)
} else {
built, err = builder.Build(building, builder.Labelled(builder.Command, request.ID), publisher,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats)
}
}
// What it copied into the store, whatever became of the build (novox/hq ADR 0257).
result.Mirrored = built.Mirrored
// Only a build the kill ended: the kill came before its work did. One that finished — built, or
// failed on its own — in the moment the kill arrived says what it did, and the kill is refused.
killed := false
if mine != nil {
killed = h.returned(mine) && err != nil
}
if superseded.Load() && err != nil {
// Ended by its own key moving on (ADR 0406 §4): said as the check's failure, as a kill is.
result.Failed = link.SupersededByKey
say("failed", link.SupersededByKey)
} else if killed {
// **Killed by hand is the outcome, whatever the build was doing** (novox/hq ADR 0219): the
// error it ended with is the kill's consequence, not a fault of the source.
result.Failed = link.KilledByHand
say("failed", link.KilledByHand)
} else if err != nil {
// A failure is a result. A build that fails and says nothing is indistinguishable from a
// builder that is not running, and those want completely different responses.
result.Failed = err.Error()
say("failed", err.Error())
} else if request.Check != nil {
say("checked", result.Check.Verdict+": "+result.Check.Summary)
} else {
manifest, marshalErr := json.Marshal(built.Manifest)
if marshalErr != nil {
result.Failed = marshalErr.Error()
} else {
result.Commit = built.Commit
result.Manifest = manifest
for _, made := range built.Built {
result.Made = append(result.Made, link.MadeArtifact{
Name: made.Name, Kind: made.Kind, Reference: made.Reference,
})
}
result.Against = built.Against
result.SourceFingerprint = built.Source
result.SourceInputs = built.SourceInputs
// What the walk did instead of building, or why it built (novox/hq ADR 0350 §3, §4).
result.ReuseKey = built.ReuseKey
if built.Reused != nil {
f := built.Reused.From
result.Reused = built.Reused.Words()
result.ReusedFrom = &link.ReusedFrom{Pull: link.PullRef{Owner: f.Pull.Owner, Repo: f.Pull.Repo,
Number: f.Pull.Number}, Head: f.Head, Trunk: f.Trunk, Check: f.Check, Key: f.Key}
} else if request.Reuse != nil {
result.BuiltWhy = built.Why
if result.BuiltWhy == "" {
result.BuiltWhy = "it has no artifact to build"
}
}
result.Trunk, result.OnTrunk, result.Branches = built.Trunk, built.OnTrunk, built.Branches
for _, r := range built.Read {
result.Read = append(result.Read, link.ReadRepository{Repository: r.Repository, Ref: r.Ref})
}
for _, s := range built.Sources {
result.Sources = append(result.Sources, link.BuildSource{Repository: s.Repository, Ref: s.Ref, Paths: s.Paths})
}
// The list of predefined keys the manifest was judged against, for the controller to publish
// (novox/hq ADR 0348 §7).
if p := built.Predefined; p != nil {
result.Predefined = &link.PredefinedList{Keys: p.Keys, Generation: p.Generation}
}
say("built", built.Manifest.Module+" from "+short(built.Commit))
}
}
// A killed build is announced on a context of its own: the build's was the one cancelled, and the
// outcome must go out and the ask be settled — acknowledged, never redelivered to another machine
// to be built again. A machine being stopped is the other case and keeps its meaning: the
// parent's context is gone, nothing is announced or settled, and the ask is redelivered.
announcing := ctx
if killed {
fresh, stop := context.WithTimeout(context.Background(), 30*time.Second)
defer stop()
announcing = fresh
}
announceErr := work.Announce(announcing, result)
if mine != nil {
h.said(mine, result.Failed, announceErr == nil)
}
if err := announceErr; err != nil {
// Said, not fatal: the build happened. A build reported as failed because announcing it
// failed is a lie about work that was done — and the request stays unsettled below only if
// nothing was said at all, so another machine can try.
fmt.Fprintf(os.Stderr, "cannot say what came of a build: %v\n", err)
return
}
// Settled only once the outcome is away, so a machine that dies before answering leaves the work
// for another rather than losing it.
if err := work.Done(); err != nil {
fmt.Fprintf(os.Stderr, "the outcome is away and the request could not be settled: %v\n", err)
}
}
// checkSpecOf is a check request as the builder runs it.
func checkSpecOf(request link.BuildRequest) builder.CheckSpec {
c := request.Check
spec := builder.CheckSpec{ID: request.ID, Repository: request.Repository, Ref: request.Ref,
Owner: c.Owner, Repo: c.Repo, Number: c.Number, Paths: c.Paths, Beside: map[string]builder.Beside{},
Modules: c.Modules, New: c.New, Manifests: c.Manifests, Judge: c.Judge, Base: c.Base,
Toolchain: builder.ToolchainOf(request.Held), Toolchains: builder.ToolchainsOf(request.Held),
Tests: c.Tests, StoreOnly: c.StoreOnly, SiblingsUnknown: c.SiblingsUnknown}
for _, s := range c.Siblings {
spec.Siblings = append(spec.Siblings, builder.Sibling{Repository: s.Repository, Number: s.Number, Head: s.Head,
Base: s.Base, Dir: s.Dir})
}
for dir, b := range c.Beside {
spec.Beside[dir] = builder.Beside{Repository: b.Repository, Ref: b.Ref}
}
for _, b := range c.Builds {
spec.Builds = append(spec.Builds, builder.CheckBuildSpec{Module: b.Module, Repository: b.Repository, Path: b.Path,
Ref: b.Ref, Merged: b.Merged, Tier: b.Tier, Skip: b.Skip})
}
for _, m := range c.Members {
spec.Group = append(spec.Group, builder.GroupHead{Owner: m.Owner, Repo: m.Repo, Repository: m.Repository,
Ref: m.Ref, Paths: m.Paths, Base: m.Base})
}
return spec
}
// siblingsOut is the siblings a check read as the outcome carries them (novox/hq ADR 0382 §4).
func siblingsOut(read []builder.JudgedSibling) []link.CheckSibling {
var out []link.CheckSibling
for _, s := range read {
out = append(out, link.CheckSibling{Repository: s.Repository, Number: s.Number, Head: s.Head, Base: s.Base,
BaseCommit: s.BaseCommit})
}
return out
}
// judgeCache is the judge cache the build agent's manifest declares (MESH_JUDGE_CACHE, novox/hq ADR 0358 §6): a
// directory mounted at the same path on the host, so a judge build's mounts name what the agent sees. Empty when the
// agent declares none, and every judge is then built cold under its check's root.
func judgeCache() string { return os.Getenv("MESH_JUDGE_CACHE") }
// triedOf is a try's outcome as its log line says it (novox/hq issue 686): the repository's own check's verdict,
// or the check's where that did not run — a head that does not merge — and the lines naming what failed.
func triedOf(v builder.CheckVerdict) link.Tried {
t := link.Tried{Verdict: v.Verdict, Summary: v.Summary, Took: v.Took.Round(time.Second).String(),
Head: v.Head, Trunk: v.Trunk}
if v.Repo != nil && (v.Gate == nil || v.Gate.Verdict == "pass") {
t.Verdict, t.Summary = v.Repo.Verdict, v.Repo.Summary
}
if v.Repo != nil && v.Repo.Failed != "" {
t.Failed = strings.Split(v.Repo.Failed, "\n")
}
if t.Verdict == "" {
t.Verdict = "error"
}
return t
}
// namedOf are the builds the seat's verdicts name, as the builder believes a scratch record by them (novox/hq ADR
// 0350 §2, issue 703): each module's key with every artifact's digest.
func namedOf(builds []link.CheckedBuild) []builder.NamedBuild {
var out []builder.NamedBuild
for _, b := range builds {
named := builder.NamedBuild{Pull: builder.PullRef{Owner: b.Pull.Owner, Repo: b.Pull.Repo,
Number: b.Pull.Number}, Check: b.Check, Module: b.Built.Module, Key: b.Built.Key,
Digests: map[string]string{}}
for _, m := range b.Built.Made {
named.Digests[m.Name] = builder.DigestOf(m.Reference)
}
out = append(out, named)
}
return out
}
// checkBuiltOf is a check's builds for its merge as the outcome carries them (novox/hq ADR 0350 §1).
func checkBuiltOf(built []builder.CheckBuilt) []link.CheckBuilt {
var out []link.CheckBuilt
for _, b := range built {
c := link.CheckBuilt{Module: b.Module, Key: b.Key, NotBuilt: b.NotBuilt, Took: b.Took.Round(time.Second).String(),
Kept: b.Kept, Same: b.Same, SameAs: b.SameAs}
for _, m := range b.Made {
c.Made = append(c.Made, link.MadeArtifact{Name: m.Name, Kind: m.Kind, Reference: m.Reference})
}
out = append(out, c)
}
return out
}
// layerOf is one layer of a check as the outcome carries it.
func layerOf(l *builder.Layer) *link.CheckLayer {
if l == nil {
return nil
}
return &link.CheckLayer{Verdict: l.Verdict, Summary: l.Summary, Modules: l.Modules}
}
// packagesFrom is where a build resolves the mesh's own published packages — the SDK above all
// (novox/hq ADR 0076, issue 053) — on this machine.
//
// **The binding wins, then the genesis document** (novox/hq ADR 0336 §3, §4). The build agent's
// npm-package-registry requirement is optional: on genesis's node, before the gitea module is assigned,
// nothing provides it, and the installer sealed the forge it raised to this agent as one document
// instead. Once the requirement binds, the binding names the endpoint the way the artifact store's
// does and a sealed token file the credential, and the genesis document is left unread. A binding
// file named and absent is a requirement not bound here, never a fault: the environment names the
// path whether or not the mesh wrote it.
//
// The environment variables remain for a builder run by a person. With none of the three the result
// is disabled and says this machine, so a build that needs the registry is refused naming it (§6),
// and a build that needs none builds anyway.
func packagesFrom(machine string) (builder.Npmrc, error) {
scope := strings.TrimSpace(os.Getenv("MESH_NPM_SCOPE"))
if scope == "" {
scope = "@novox"
}
none := builder.Npmrc{Machine: machine}
if path := strings.TrimSpace(os.Getenv("MESH_PACKAGE_BINDING")); path != "" {
raw, err := os.ReadFile(path)
switch {
case errors.Is(err, os.ErrNotExist):
// Not bound on this machine: the genesis document, or nothing.
case err != nil:
return none, fmt.Errorf("cannot read what the mesh said about the package registry: %w", err)
default:
return fromBinding(path, raw, scope, machine)
}
}
if path := strings.TrimSpace(os.Getenv("MESH_GENESIS_PACKAGE_REGISTRY")); path != "" {
raw, err := os.ReadFile(path)
switch {
case errors.Is(err, os.ErrNotExist):
// Not this mesh's genesis node, or the installer has not sealed it yet.
case err != nil:
return none, fmt.Errorf("cannot read the genesis document for the package registry: %w", err)
default:
return fromGenesis(path, raw, scope, machine)
}
}
// A builder run by a person.
registry := strings.TrimSpace(os.Getenv("MESH_NPM_REGISTRY"))
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
username := strings.TrimSpace(os.Getenv("MESH_NPM_USER"))
if registry == "" && secret == "" {
return none, nil
}
n := builder.Npmrc{Scope: scope, Registry: registry, From: builder.FromEnvironment, Machine: machine}
if username != "" {
n.Username, n.Password = username, secret
} else {
n.Token = secret
}
return n, nil
}
// fromBinding composes the registry from the npm-package-registry binding the mesh wrote and the
// credential it sealed beside it. The mesh authenticates the ordinary way — a generated password the
// provider only applies (novox/hq ADR 0048) — so with a username the secret is a password (basic
// auth); without one it is a bearer token a provider minted.
func fromBinding(path string, raw []byte, scope, machine string) (builder.Npmrc, error) {
none := builder.Npmrc{Machine: machine}
var told struct {
From string `json:"from"`
At string `json:"at"`
As string `json:"as"`
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal(raw, &told); err != nil {
return none, fmt.Errorf("%s is not a binding: %w", path, err)
}
if told.At == "" {
return none, fmt.Errorf(
"%s says the package registry is on %q and gives no address for it", path, told.From)
}
// Composed from what the provider serves, so nothing here knows gitea's URL shape from
// another registry's: it states its port, the path its registry answers on, and the scheme.
scheme := "https"
if s, ok := told.Serves["scheme"]; ok {
scheme = fmt.Sprintf("%v", s)
}
port, ok := told.Serves["port"]
if !ok {
return none, fmt.Errorf("%s says nothing about which port the package registry answers on", path)
}
npmPath, ok := told.Serves["npm-path"]
if !ok {
return none, fmt.Errorf("%s says nothing about the path the package registry answers on", path)
}
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
if file := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN_FILE")); file != "" {
raw, err := os.ReadFile(file)
if err != nil {
return none, fmt.Errorf("cannot read this builder's package-registry credential: %w", err)
}
secret = strings.TrimSpace(string(raw))
}
n := builder.Npmrc{Scope: scope, Registry: fmt.Sprintf("%s://%s:%v%v", scheme, told.At, port, npmPath),
From: builder.FromBinding, Machine: machine}
if told.As != "" {
n.Username, n.Password = told.As, secret
} else {
n.Token = secret
}
return n, nil
}
// GenesisRegistry is the genesis document (novox/hq ADR 0336 §3): what the installer sealed to the
// build agent on its node at genesis's packages step, as the `genesis-package-registry` own-secret —
// the forge it raised, the port this node gave the package registry, its npm path, and the account it
// made with that account's password. mesh-host writes it; the field names are the two repositories'
// agreement.
type GenesisRegistry struct {
Scheme string `json:"scheme"`
At string `json:"at"`
Port int `json:"port"`
NpmPath string `json:"npm-path"`
As string `json:"as"`
Password string `json:"password"`
}
// fromGenesis composes the registry from the genesis document.
func fromGenesis(path string, raw []byte, scope, machine string) (builder.Npmrc, error) {
none := builder.Npmrc{Machine: machine}
var g GenesisRegistry
if err := json.Unmarshal(raw, &g); err != nil {
return none, fmt.Errorf("%s is not the genesis document for the package registry: %w", path, err)
}
var missing []string
for name, value := range map[string]string{"at": g.At, "npm-path": g.NpmPath, "as": g.As, "password": g.Password} {
if strings.TrimSpace(value) == "" {
missing = append(missing, name)
}
}
if g.Port <= 0 {
missing = append(missing, "port")
}
if len(missing) > 0 {
sort.Strings(missing)
return none, fmt.Errorf("%s, the genesis document for the package registry, says nothing of %s",
path, strings.Join(missing, ", "))
}
scheme := g.Scheme
if scheme == "" {
scheme = "http"
}
return builder.Npmrc{Scope: scope, Registry: fmt.Sprintf("%s://%s:%d%s", scheme, g.At, g.Port, g.NpmPath),
Username: g.As, Password: g.Password, From: builder.FromGenesis, Machine: machine}, nil
}
// saysRegistry is the line each build's log carries about the package registry (novox/hq ADR 0336
// §4): which of the binding and the genesis document it resolved from, or that it has neither here.
func saysRegistry(n builder.Npmrc) string {
if n.Enabled() {
return "the package registry is resolved from " + n.From
}
machine := n.Machine
if machine == "" {
machine = "this machine"
}
return fmt.Sprintf("no package registry: %s is unbound on %s and no genesis document is here; "+
"a build needing it is refused", builder.RegistryRequirement, machine)
}
// forgeFrom is the git credential this builder may offer a clone, composed from the same binding
// and sealed secret its package-registry half already reads: the forge that answers npm is the
// forge that hosts the repositories, and its provisioner applies one password to one user for
// both. Anything missing means no credential, and every clone stays anonymous — which is all a
// mesh of public repositories ever needs.
//
// The URL names the binding's own address — the machine the mesh says the forge is on — so a
// private repository is registered and built by that address, and a clone of anything else is
// never shown this credential (git's credential store matches the whole origin).
func forgeFrom() builder.GitCredential {
path := strings.TrimSpace(os.Getenv("MESH_PACKAGE_BINDING"))
if path == "" {
return builder.GitCredential{}
}
raw, err := os.ReadFile(path)
if err != nil {
return builder.GitCredential{}
}
var told struct {
At string `json:"at"`
As string `json:"as"`
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal(raw, &told); err != nil || told.At == "" || told.As == "" {
return builder.GitCredential{}
}
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
if file := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN_FILE")); file != "" {
if raw, err := os.ReadFile(file); err == nil {
secret = strings.TrimSpace(string(raw))
}
}
if secret == "" {
return builder.GitCredential{}
}
scheme := "https"
if s, ok := told.Serves["scheme"]; ok {
scheme = fmt.Sprintf("%v", s)
}
host := told.At
if port, ok := told.Serves["port"]; ok {
host = fmt.Sprintf("%s:%v", told.At, port)
}
made := url.URL{Scheme: scheme, User: url.UserPassword(told.As, secret), Host: host}
return builder.GitCredential{URL: made.String()}
}
func short(commit string) string {
if len(commit) > 8 {
return commit[:8]
}
return commit
}
// whereToPublish is the artifact store this builder uses.
//
// **Preferably from the mesh.** A builder that is a module requires an artifact store, and the
// mesh writes it a file saying which machine answers that and on what port — the same binding any
// consumer of any provision gets. Reading it means the address is not a setting somebody keeps in
// step by hand, and moving the store is an ordinary reassignment rather than an edit on every
// build machine.
//
// The environment variable remains for a builder run by a person, which is how this started and
// how it is still run while being developed.
func whereToPublish() (string, error) {
binding := strings.TrimSpace(os.Getenv("MESH_BINDING"))
if binding == "" {
registry := strings.TrimSpace(os.Getenv("MESH_REGISTRY"))
if registry == "" {
return "", fmt.Errorf("neither MESH_BINDING nor MESH_REGISTRY: a built artifact " +
"nobody can fetch is not built")
}
return registry, nil
}
raw, err := os.ReadFile(binding)
if err != nil {
return "", fmt.Errorf("cannot read what the mesh said about the artifact store: %w", err)
}
var told struct {
From string `json:"from"`
At string `json:"at"`
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal(raw, &told); err != nil {
return "", fmt.Errorf("%s is not a binding: %w", binding, err)
}
if told.At == "" {
// The provider is not on the private network, so there is no name to reach it by. Said
// rather than falling back to the machine's own name, which would publish to a store on
// the wrong machine and be found out much later.
return "", fmt.Errorf(
"%s says the artifact store is on %q and gives no address for it", binding, told.From)
}
port, ok := told.Serves["port"]
if !ok {
return "", fmt.Errorf("%s says nothing about which port the artifact store answers on",
binding)
}
return fmt.Sprintf("%s:%v", told.At, port), nil
}
// brokerFrom is where this builder connects, and with what.
//
// **Preferably from a file the mesh sealed to this machine.** A builder that is a module is given
// its credential the way every other module is given one: generated or accepted centrally, sealed
// to the machine, written by the host. Putting it in an environment variable instead would mean
// the one copy that matters passing through a terminal and a process listing.
//
// The variable remains for a builder run by a person.
func brokerFrom() (Credential, error) {
if path := strings.TrimSpace(os.Getenv("MESH_BROKER_FILE")); path != "" {
raw, err := os.ReadFile(path)
if err != nil {
return Credential{}, fmt.Errorf("cannot read this builder's credential: %w", err)
}
said := strings.TrimSpace(string(raw))
if said == "" {
// An empty credential file is a machine that will connect as nobody and be refused,
// with the reason three layers away.
return Credential{}, fmt.Errorf("%s is empty, so this builder has no credential", path)
}
var held Credential
if err := json.Unmarshal([]byte(said), &held); err == nil && held.URL != "" {
return held, nil
}
// A file holding only a URL, which is what a person writing one by hand produces. The
// broker is then verified against whatever this machine already trusts.
return Credential{URL: said}, nil
}
return Credential{}, fmt.Errorf(
"no MESH_BROKER_FILE: a build machine with no credential for the bus has nothing to build")
}
// Credential is what a build machine is given so it can reach the broker.
//
// Two things, because reaching a broker over TLS needs both: who to connect as, and what to check
// the certificate against. A mesh's broker presents a certificate of the mesh's own, which is in
// no public trust store, so a URL alone can only connect to a broker somebody else vouches for.
//
// **The same shape a node gets, for the same reason** (novox/hq ADR 0004): the fingerprint travels
// out of band — here, sealed with the credential — and the endpoint is verified once at connect.
type Credential struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
// User and Password ride beside the address on the bus being built (design 25): a credential
// embedded in a URL leaks into every log line that prints a connection, so the mesh seals them
// as two fields and this machine joins them once, here, to dial.
User string `json:"user,omitempty"`
Password string `json:"password,omitempty"`
// Claims are the seats the module this credential was issued for claims, as the mesh writes
// them beside the credential (novox/hq ADR 0159). The first is the build role this machine
// serves; a credential naming none is from before claims travelled in it.
Claims []struct {
Seat string `json:"seat"`
} `json:"claims,omitempty"`
}
// seatsClaimed is the seats the credential names, in order.
func (c Credential) seatsClaimed() []string {
out := make([]string, 0, len(c.Claims))
for _, claim := range c.Claims {
out = append(out, claim.Seat)
}
return out
}
// onTheNewBus is whether a credential is for the bus being built: its address says so, and the
// mesh only ever seals such a credential with the user and password beside it.
func (c Credential) onTheNewBus() bool { return strings.HasPrefix(strings.TrimSpace(c.URL), "nats://") }
// natsURL is the address with this machine's credential in it, for the one dial that needs it.
func (c Credential) natsURL() string {
rest := strings.TrimPrefix(strings.TrimSpace(c.URL), "nats://")
if c.User == "" {
return "nats://" + rest
}
return "nats://" + c.User + ":" + c.Password + "@" + rest
}
-188
View File
@@ -1,188 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"strings"
"github.com/novox/mesh-controller/internal/builder"
)
// buildOnce is the builder doing one build and stopping, with no broker and no mesh.
//
// **This is how a mesh is raised** (novox/hq ADR 0073). The installer carries this program and runs
// it once, before anything else exists, to produce the control plane from the same repository and
// path that every later rebuild of it will use. What raises the mesh is therefore the same thing
// that maintains it — not a second mechanism that has to be kept in step with the first and is
// exercised once per new mesh, which is how often enough to rot.
//
// It takes no work from a queue and answers nobody: there is no broker yet, and the only thing
// waiting for the answer is the installer that started it. So the result goes to standard output as
// JSON, which is what the installer reads.
//
// With no --registry it publishes nowhere and the image stays in this machine's container runtime,
// named by the digest of its own configuration (see builder.Local). That is the genesis case. With
// one, it publishes as it always does — the same command is how an operator builds a module by hand
// on a mesh that already exists.
func buildOnce(ctx context.Context, args []string) error {
set := flag.NewFlagSet("build", flag.ContinueOnError)
path := set.String("path", "", "the module's directory inside the repository (default: its root)")
ref := set.String("ref", "", "the commit to build; a branch is a moving target somebody else controls")
registry := set.String("registry", "",
"host:port to publish to. Without it the artifacts stay in this machine's container runtime, which is the genesis case")
workspace := set.String("workspace", "", "where to clone and build (default: a temporary directory)")
on := multiple(set, "on",
"a base this build stands on, as <module>/<artifact>=<reference>. Repeatable")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 1 {
return errors.New("mesh-builder build <repository> [--path P] [--ref COMMIT] [--registry HOST:PORT]")
}
repository := positionals[0]
where := *workspace
if where == "" {
where = os.TempDir() + "/mesh-builder-once"
}
// What this build may stand on, said on the command line because there is no mesh to ask.
//
// **This is the genesis case and it is meant to be awkward.** On a running mesh the control
// plane answers this, because only it knows what this mesh holds. Here nothing has been built
// yet, so whoever runs this says what to use — and for the control plane, which is the one
// module raised before anything else exists, the answer is ordinarily nothing at all.
bases := map[string]string{}
for _, pair := range *on {
key, reference, found := strings.Cut(pair, "=")
if !found || key == "" || reference == "" {
return fmt.Errorf(
"--on takes <module>/<artifact>=<reference>, and %q is not that", pair)
}
bases[key] = reference
}
// Local unless told otherwise, because the moment this exists for has nowhere to publish. A
// default pointing at a registry would mean genesis failing at a push to something that is not
// there yet, one step away from the thing that could explain it.
var publisher builder.Publisher = builder.Local{Run: builder.Command}
if *registry != "" {
publisher = builder.Registry{Address: *registry, Run: builder.Command}
}
fmt.Fprintf(os.Stderr, "building %s", repository)
if *path != "" {
fmt.Fprintf(os.Stderr, " at %s", *path)
}
if *ref != "" {
fmt.Fprintf(os.Stderr, " at %s", *ref)
}
fmt.Fprintln(os.Stderr)
machine := os.Getenv("MESH_NODE")
if machine == "" {
machine, _ = os.Hostname()
}
npmrc, err := packagesFrom(machine)
if err != nil {
return err
}
fmt.Fprintf(os.Stderr, " [packages] %s\n", saysRegistry(npmrc))
built, buildErr := builder.Build(ctx, builder.Command, publisher, repository, *path, *ref, where, bases, npmrc,
forgeFrom(),
func(step, message string) { fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message) })
if buildErr != nil {
return buildErr
}
// To standard output, and everything else to standard error, so the caller can read this
// without having to separate it from progress.
out := onceResult{
Module: built.Manifest.Module,
Commit: built.Commit,
Repository: repository,
Path: *path,
Ref: *ref,
Manifest: built.Manifest,
Against: built.Against,
Source: built.Source,
}
for _, r := range built.Read {
out.Read = append(out.Read, readRepository{Repository: r.Repository, Ref: r.Ref})
}
for _, made := range built.Built {
out.Made = append(out.Made, madeArtifact{Name: made.Name, Kind: made.Kind, Reference: made.Reference})
}
body, err := json.MarshalIndent(out, "", " ")
if err != nil {
return err
}
fmt.Println(string(body))
fmt.Fprintf(os.Stderr, " built %s from %s\n", built.Manifest.Module, short(built.Commit))
return nil
}
// onceResult is what one build reports to whoever started it.
//
// The same fields the mesh records for a build, so a reader comparing a genesis build against an
// ordinary one is comparing the same thing said the same way.
type onceResult struct {
Module string `json:"module"`
Commit string `json:"commit"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
Manifest any `json:"manifest"`
Made []madeArtifact `json:"made"`
Against []string `json:"against,omitempty"`
Read []readRepository `json:"read,omitempty"`
Source string `json:"source-fingerprint,omitempty"`
}
// readRepository is a repository this build read source from besides the module's own.
type readRepository struct {
Repository string `json:"repository"`
Ref string `json:"ref,omitempty"`
}
type madeArtifact struct {
Name string `json:"name"`
Kind string `json:"kind"`
Reference string `json:"reference"`
}
// parseAround lets flags appear on either side of the repository, because a person writing this by
// hand will put them wherever reads best and the standard parser stops at the first thing that is
// not a flag. The same helper the control plane's commands use, for the same reason.
func parseAround(set *flag.FlagSet, args []string) ([]string, error) {
var positionals []string
rest := args
for {
if err := set.Parse(rest); err != nil {
return nil, err
}
rest = set.Args()
if len(rest) == 0 {
return positionals, nil
}
positionals = append(positionals, rest[0])
rest = rest[1:]
}
}
// multiple is a flag that may be given more than once.
type repeated []string
func (r *repeated) String() string { return strings.Join(*r, ", ") }
func (r *repeated) Set(v string) error { *r = append(*r, v); return nil }
func multiple(set *flag.FlagSet, name, usage string) *[]string {
var values repeated
set.Var(&values, name, usage)
return (*[]string)(&values)
}
-102
View File
@@ -1,102 +0,0 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/builder"
)
// The build agent's package registry (novox/hq ADR 0336, issue 121): its npm-package-registry
// requirement is optional, so the binding may be absent, and on genesis's node the installer seals
// the forge it raised as one genesis document instead.
// registryFiles names the three paths the build agent's environment always names, writing only those
// given.
func registryFiles(t *testing.T, binding, token, genesis string) {
t.Helper()
dir := t.TempDir()
for env, content := range map[string]string{
"MESH_PACKAGE_BINDING": binding,
"MESH_NPM_TOKEN_FILE": token,
"MESH_GENESIS_PACKAGE_REGISTRY": genesis,
} {
path := filepath.Join(dir, env)
t.Setenv(env, path)
if content != "" {
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatal(err)
}
}
}
for _, env := range []string{"MESH_NPM_REGISTRY", "MESH_NPM_TOKEN", "MESH_NPM_USER", "MESH_NPM_SCOPE"} {
t.Setenv(env, "")
}
}
const (
aBinding = `{"from":"gitea","at":"anchor.mesh","as":"anchor-agent",
"serves":{"scheme":"http","port":3000,"npm-path":"/api/packages/novox/npm/"}}`
aGenesisDocument = `{"scheme":"http","at":"127.0.0.1","port":3100,"npm-path":"/api/packages/novox/npm/",
"as":"mesh-builder","password":"genesis-secret"}`
)
func TestUntilTheRequirementBindsTheGenesisDocumentIsRead(t *testing.T) {
registryFiles(t, "", "", aGenesisDocument)
n, err := packagesFrom("anchor")
if err != nil {
t.Fatalf("an absent binding beside a genesis document was refused: %v", err)
}
if n.From != builder.FromGenesis || n.Registry != "http://127.0.0.1:3100/api/packages/novox/npm/" ||
n.Username != "mesh-builder" || n.Password != "genesis-secret" || n.Scope != "@novox" {
t.Fatalf("the genesis document was not what the registry came from: %+v", n)
}
file, err := n.File()
if err != nil || !strings.Contains(file, "//127.0.0.1:3100/api/packages/novox/npm/:username=mesh-builder") {
t.Fatalf("the .npmrc is not written from the genesis document: %v\n%s", err, file)
}
if got := saysRegistry(n); !strings.Contains(got, "genesis document") {
t.Errorf("the log does not say the genesis document: %s", got)
}
}
func TestARealBindingWinsOverTheGenesisDocument(t *testing.T) {
registryFiles(t, aBinding, "bound-secret\n", aGenesisDocument)
n, err := packagesFrom("anchor")
if err != nil {
t.Fatal(err)
}
if n.From != builder.FromBinding || n.Registry != "http://anchor.mesh:3000/api/packages/novox/npm/" ||
n.Username != "anchor-agent" || n.Password != "bound-secret" {
t.Fatalf("the binding did not win over the genesis document: %+v", n)
}
if got := saysRegistry(n); !strings.Contains(got, "binding") {
t.Errorf("the log does not say the binding: %s", got)
}
}
func TestWithNeitherTheAgentHasNoRegistryAndSaysWhere(t *testing.T) {
registryFiles(t, "", "", "")
n, err := packagesFrom("workstation")
if err != nil {
t.Fatalf("an unbound optional requirement was a fault: %v", err)
}
if n.Enabled() || n.Machine != "workstation" {
t.Fatalf("a registry came from nowhere: %+v", n)
}
got := saysRegistry(n)
for _, want := range []string{"npm-package-registry", "unbound", "workstation"} {
if !strings.Contains(got, want) {
t.Errorf("the log does not say %q: %s", want, got)
}
}
}
func TestAGenesisDocumentMissingAFieldIsRefusedByName(t *testing.T) {
registryFiles(t, "", "", `{"at":"127.0.0.1","port":3000,"npm-path":"/api/packages/novox/npm/","as":"mesh-builder"}`)
if _, err := packagesFrom("anchor"); err == nil || !strings.Contains(err.Error(), "password") {
t.Fatalf("a genesis document with no password was taken: %v", err)
}
}
-44
View File
@@ -1,44 +0,0 @@
package main
import (
"errors"
"fmt"
"os"
"path/filepath"
)
// **The build agent says it is ready** (novox/hq issue 544). It listens on nothing, so no http or tcp
// check can see it work; it writes a mark once it is bound to its worker and pulls build work, and its
// container's health check runs `mesh-builder ready`, which passes only while the mark is there. Without
// it the node-engine could judge the agent only alive, and every walk that moved it waited out the whole
// grace on each machine.
//
// The mark lives in the container's own temporary directory, never in the workspace the machine keeps:
// a new container starts without it, and a restarted one removes it before it binds again.
func readyMark() string { return filepath.Join(os.TempDir(), "mesh-builder.ready") }
// notReady removes the mark: a holder starting is not taking work until it says so again.
func notReady() error {
if err := os.Remove(readyMark()); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove the mark that this agent takes build work (%s): %w", readyMark(), err)
}
return nil
}
// taking writes the mark. A mark that cannot be written is said, and the agent goes on building: its
// check then says it is not ready, which is the truth the gate should hear.
func taking(seat string) {
if err := os.WriteFile(readyMark(), []byte(seat+"\n"), 0o644); err != nil {
fmt.Fprintf(os.Stderr, "cannot mark that this agent takes build work (%s): %v\n", readyMark(), err)
return
}
fmt.Fprintf(os.Stderr, "taking build work from %s\n", seat)
}
// ready is the health check: nil when this container's agent is bound and pulling build work.
func ready() error {
if _, err := os.Stat(readyMark()); err != nil {
return fmt.Errorf("not taking build work yet: %s is not there", readyMark())
}
return nil
}
-32
View File
@@ -1,32 +0,0 @@
package main
import (
"os"
"testing"
)
// The health check passes only between the agent saying it takes work and its next start (novox/hq
// issue 544).
func TestTheAgentIsReadyOnlyOnceItTakesWork(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := ready(); err == nil {
t.Fatal("an agent that never took work is ready")
}
taking("mesh-build-machine")
if err := ready(); err != nil {
t.Fatalf("an agent taking work is not ready: %v", err)
}
// Started again, it is not ready until it binds again.
if err := notReady(); err != nil {
t.Fatal(err)
}
if err := ready(); err == nil {
t.Fatal("an agent starting again is ready before it takes work")
}
if err := notReady(); err != nil {
t.Fatalf("removing a mark that is not there failed: %v", err)
}
if _, err := os.Stat(readyMark()); err == nil {
t.Fatal("the mark is still there")
}
}
-27
View File
@@ -1,27 +0,0 @@
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// The seat a build machine serves comes from its credential (novox/hq ADR 0190 handover).
func TestTheCredentialSaysWhichBuildRoleThisMachineServes(t *testing.T) {
var held Credential
if err := json.Unmarshal([]byte(`{"url":"nats://bus:4222","user":"anchor.builder","password":"x",
"claims":[{"seat":"mesh-build-machine","scope":"mesh","serves":[]}]}`), &held); err != nil {
t.Fatal(err)
}
if got := link.BuildSeatClaimed(held.seatsClaimed()); got != "mesh-build-machine" {
t.Errorf("the old builder's credential serves %q", got)
}
var bare Credential
if err := json.Unmarshal([]byte(`{"url":"nats://bus:4222","user":"anchor.build-agent","password":"x"}`), &bare); err != nil {
t.Fatal(err)
}
if got := link.BuildSeatClaimed(bare.seatsClaimed()); got != link.TheBuildMachine {
t.Errorf("a credential without claims serves %q, want %s", got, link.TheBuildMachine)
}
}
-45
View File
@@ -1,45 +0,0 @@
package main
import (
"os"
"strings"
"testing"
)
// **The genesis path writes its result to stdout and nothing else there.** The installer captures
// stdout alone and parses it as JSON; one stray log line makes that fail with "invalid character"
// — exactly what a builder fmt.Printf caused, breaking a build that had worked. This keeps
// diagnostics on stderr so a future print cannot repeat it silently.
func TestBuilderDiagnosticsStayOffStdout(t *testing.T) {
allowed := map[string]bool{
"string(body)": true, // once.go: the result JSON, which IS stdout
"version)": true, // --version
`"stopping")`: true, // the loop.s shutdown line
"usage)": true, // --help text, for a human
"said)": true, // busy: the answer its node-engine reads (novox/hq ADR 0335)
}
for _, file := range []string{"once.go", "main.go"} {
src, err := os.ReadFile(file)
if err != nil {
t.Fatal(err)
}
for i, raw := range strings.Split(string(src), "\n") {
line := strings.TrimSpace(raw)
if !strings.HasPrefix(line, "fmt.Printf(") &&
!strings.HasPrefix(line, "fmt.Println(") &&
!strings.HasPrefix(line, "fmt.Print(") {
continue
}
ok := false
for token := range allowed {
if strings.Contains(line, token) {
ok = true
}
}
if !ok {
t.Errorf("%s:%d writes a diagnostic to stdout, which the installer parses as JSON:\n %s",
file, i+1, line)
}
}
}
}
-31
View File
@@ -1,31 +0,0 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/builder"
)
// A try's outcome, as the build seat says it in the build's log (novox/hq issue 686): the repository's own
// check's verdict, how long it took, and each line naming what failed.
func TestATrysOutcomeNamesWhatFailed(t *testing.T) {
tried := triedOf(builder.CheckVerdict{Verdict: "pass", Summary: "the change touches no module", Took: 303 * time.Second,
Head: "c0ffee", Trunk: "ca3f43c", Gate: &builder.Layer{Verdict: "pass"},
Repo: &builder.Layer{Verdict: "fail", Summary: "its merge-check.sh failed: TestX",
Failed: "--- FAIL: TestX (0.01s)\n x_test.go:12: got 1, want 2"}})
if tried.Verdict != "fail" || tried.Took != "5m3s" || len(tried.Failed) != 2 || tried.Trunk != "ca3f43c" ||
tried.Summary != "its merge-check.sh failed: TestX" {
t.Fatalf("a failed try: %+v", tried)
}
// A head that does not merge fails both layers: the check's own word.
conflict := triedOf(builder.CheckVerdict{Verdict: "fail", Summary: "does not merge", Gate: &builder.Layer{Verdict: "fail"},
Repo: &builder.Layer{Verdict: "fail", Summary: "does not merge"}})
if conflict.Verdict != "fail" || conflict.Summary != "does not merge" {
t.Fatalf("a try that does not merge: %+v", conflict)
}
// No verdict is an error, never a pass.
if none := triedOf(builder.CheckVerdict{}); none.Verdict != "error" {
t.Fatalf("a try with no verdict: %+v", none)
}
}
-225
View File
@@ -1,225 +0,0 @@
package main
import (
"crypto/ed25519"
"crypto/rand"
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/hex"
"math/big"
"net"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
)
// Where a builder publishes.
//
// Preferably from the mesh: a builder that is a module requires an artifact store, and the mesh
// writes it a binding saying which machine answers and on what port. Reading it means the address
// is not a setting somebody keeps in step by hand.
func binding(t *testing.T, body string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "artifact-store.json")
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
return path
}
func TestTheMeshSaysWhereToPublish(t *testing.T) {
t.Setenv("MESH_BINDING", binding(t, `{"binding":1,"provision":"artifact-store",
"from":"anchor","at":"anchor.internal","serves":{"port":5000,"scheme":"http"}}`))
where, err := whereToPublish()
if err != nil {
t.Fatal(err)
}
if where != "anchor.internal:5000" {
t.Fatalf("got %q", where)
}
}
func TestABindingWithNoAddressIsRefused(t *testing.T) {
// The provider is not on the private network, so there is no name to reach it by. Falling
// back to anything would publish to a store on the wrong machine and be found out much later.
t.Setenv("MESH_BINDING", binding(t, `{"binding":1,"from":"anchor","serves":{"port":5000}}`))
_, err := whereToPublish()
if err == nil {
t.Fatal("a binding with nowhere to reach was accepted")
}
if !strings.Contains(err.Error(), "anchor") {
t.Fatalf("the failure does not name the machine: %v", err)
}
}
func TestABindingWithNoPortIsRefused(t *testing.T) {
t.Setenv("MESH_BINDING", binding(t, `{"binding":1,"from":"a","at":"a.internal","serves":{}}`))
if _, err := whereToPublish(); err == nil {
t.Fatal("a binding saying nothing about a port was accepted")
}
}
func TestTheVariableStillWorksForABuilderRunByAPerson(t *testing.T) {
// Which is how this started and how it is still run while being developed.
t.Setenv("MESH_BINDING", "")
t.Setenv("MESH_REGISTRY", "127.0.0.1:5000")
where, err := whereToPublish()
if err != nil {
t.Fatal(err)
}
if where != "127.0.0.1:5000" {
t.Fatalf("got %q", where)
}
}
func TestNeitherIsRefusedRatherThanGuessed(t *testing.T) {
t.Setenv("MESH_BINDING", "")
t.Setenv("MESH_REGISTRY", "")
if _, err := whereToPublish(); err == nil {
t.Fatal("a builder with nowhere to publish reported somewhere")
}
}
func TestTheCredentialComesFromAFileTheMeshSealed(t *testing.T) {
// A builder that is a module is given its credential the way every module is: sealed to the
// machine and written by the host. An environment variable instead would put the one copy
// that matters through a terminal and a process listing.
path := filepath.Join(t.TempDir(), "broker")
if err := os.WriteFile(path, []byte("amqps://a-builder:secret@broker.internal:5671/\n"), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_BROKER_FILE", path)
t.Setenv("MESH_BROKER_AMQP", "amqp://should-not-be-used@nowhere/")
got, err := brokerFrom()
if err != nil {
t.Fatal(err)
}
if got.URL != "amqps://a-builder:secret@broker.internal:5671/" {
t.Fatalf("got %q", got.URL)
}
}
// The credential the mesh seals carries what to check the broker's certificate against, because a
// mesh's broker presents a certificate of the mesh's own and no public trust store has it. A URL
// alone can only reach a broker somebody else vouches for.
func TestTheSealedCredentialCarriesWhatVerifiesTheBroker(t *testing.T) {
path := filepath.Join(t.TempDir(), "broker")
if err := os.WriteFile(path, []byte(
`{"url":"amqps://a-builder:secret@broker.internal:5671/","fingerprint":"abc123"}`),
0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_BROKER_FILE", path)
t.Setenv("MESH_BROKER_AMQP", "")
got, err := brokerFrom()
if err != nil {
t.Fatal(err)
}
if got.URL != "amqps://a-builder:secret@broker.internal:5671/" {
t.Fatalf("the url was lost: %q", got.URL)
}
if got.Fingerprint != "abc123" {
t.Fatal("the builder was given nothing to check the broker against, so it can only " +
"connect to a broker some public authority vouches for")
}
}
func TestAnEmptyCredentialFileIsRefused(t *testing.T) {
// Otherwise the builder connects as nobody and is refused, with the reason three layers away.
path := filepath.Join(t.TempDir(), "broker")
if err := os.WriteFile(path, []byte("\n"), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_BROKER_FILE", path)
t.Setenv("MESH_BROKER_AMQP", "")
if _, err := brokerFrom(); err == nil {
t.Fatal("an empty credential was accepted")
}
}
func TestABuilderWithNoCredentialAtAllSaysSo(t *testing.T) {
t.Setenv("MESH_BROKER_FILE", "")
t.Setenv("MESH_BROKER_AMQP", "")
if _, err := brokerFrom(); err == nil {
t.Fatal("a builder with no broker reported one")
}
}
// The pin is compared in the spelling the mesh writes it.
//
// A bare digest against a written fingerprint never matches, and the failure is indistinguishable
// from being pointed at the wrong broker — which is the one thing this check exists to report
// truthfully. It cost a lab run.
func TestThePinIsComparedInTheSpellingTheMeshWritesIt(t *testing.T) {
certificate, key := aServerCertificate(t)
der := certificate.Certificate[0]
sum := sha256.Sum256(der)
written := "sha256:" + hex.EncodeToString(sum[:])
listener, err := tls.Listen("tcp", "127.0.0.1:0", &tls.Config{
Certificates: []tls.Certificate{certificate}, MinVersion: tls.VersionTLS12,
})
if err != nil {
t.Fatal(err)
}
defer listener.Close()
go func() {
for {
conn, err := listener.Accept()
if err != nil {
return
}
_ = conn.(*tls.Conn).Handshake()
conn.Close()
}
}()
_ = key
// The pin the mesh wrote must be accepted.
if err := handshakeWith(listener.Addr().String(), written); err != nil {
t.Fatalf("the broker this builder was told about was refused: %v", err)
}
// And a different one refused, or the check reports nothing.
other := "sha256:" + strings.Repeat("ab", 32)
if err := handshakeWith(listener.Addr().String(), other); err == nil {
t.Fatal("a broker this builder was not told about was accepted")
}
}
// handshakeWith runs the pin check the builder dials with against an address.
func handshakeWith(address, pin string) error {
conn, err := tls.Dial("tcp", address, broker.PinnedToFingerprint(pin))
if err != nil {
return err
}
return conn.Close()
}
func aServerCertificate(t *testing.T) (tls.Certificate, ed25519.PrivateKey) {
t.Helper()
public, private, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
template := &x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{CommonName: "a broker"},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(time.Hour),
IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
}
der, err := x509.CreateCertificate(rand.Reader, template, template, public, private)
if err != nil {
t.Fatal(err)
}
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: private}, private
}
@@ -221,8 +221,4 @@ func TestOnANodeWithNoNetworkTheStoreIsReachedByLoopback(t *testing.T) {
if image != "127.0.0.1:5100/postgres/runtime@"+aDigest {
t.Fatalf("on the store's own node, off any network, the image is fetched as %v", image)
}
held := heldBy(ctx)
if got := held["postgres/runtime"]; got != "127.0.0.1:5100/postgres/runtime@"+aDigest {
t.Fatalf("a builder beside the store is handed the base %q", got)
}
}
-59
View File
@@ -1,12 +1,9 @@
package main
import (
"encoding/json"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
// An address is read from the node's settings where it is used, never recorded with a port
@@ -14,62 +11,6 @@ import (
var aDigest = "sha256:" + strings.Repeat("e", 64)
// **A build is recorded by digest and path**, whatever address the builder pushed to — what it
// made and what it stood on both; an image the module runs from elsewhere is left where it says.
func TestABuildIsRecordedWithoutTheStoresAddress(t *testing.T) {
manifest, _ := json.Marshal(map[string]any{
"module": "gitea", "version": "1",
"resources": []map[string]any{
{"id": "server", "type": "container", "name": "mesh-gitea",
"image": "anchor.internal:5100/gitea/server@" + aDigest},
{"id": "config", "type": "archive", "path": "/etc/gitea", "digest": aDigest,
"source": "http://anchor.internal:5100/v2/gitea/config/blobs/" + aDigest},
{"id": "cache", "type": "container", "name": "mesh-gitea-cache",
"image": "valkey/valkey@" + aDigest},
},
})
kept := buildFrom(link.BuildResult{
ID: "b1", Repository: "https://forge.example/gitea.git", Commit: "abc", On: "laptop",
Manifest: manifest,
Made: []link.MadeArtifact{
{Name: "server", Kind: "image", Reference: "anchor.internal:5100/gitea/server@" + aDigest},
{Name: "config", Kind: "archive", Reference: "http://anchor.internal:5100/v2/gitea/config/blobs/" + aDigest},
},
Against: []string{"anchor.internal:5100/mesh-tools/runtime@" + aDigest},
})
if kept.Module != "gitea" {
t.Fatalf("the module was not read from the recorded manifest: %q", kept.Module)
}
if kept.Made[0].Reference != catalogue.ArtifactStoreScheme+"gitea/server@"+aDigest {
t.Errorf("the image is recorded as %q, address and all", kept.Made[0].Reference)
}
if kept.Made[1].Reference != catalogue.ArtifactStoreScheme+"gitea/config/blobs/"+aDigest {
t.Errorf("the archive is recorded as %q, address and all", kept.Made[1].Reference)
}
recorded, err := catalogue.ParseManifest(kept.Manifest)
if err != nil {
t.Fatal(err)
}
if got := recorded.Resources[0]["image"]; got != catalogue.ArtifactStoreScheme+"gitea/server@"+aDigest {
t.Errorf("the recorded manifest's image is %v", got)
}
if got := recorded.Resources[1]["source"]; got != catalogue.ArtifactStoreScheme+"gitea/config/blobs/"+aDigest {
t.Errorf("the recorded manifest's archive is %v", got)
}
if got := recorded.Resources[2]["image"]; got != "valkey/valkey@"+aDigest {
t.Errorf("an image the build did not make was rewritten: %v", got)
}
if strings.Contains(string(kept.Manifest), "anchor.internal:5100") {
t.Errorf("the recorded manifest still carries the store's address:\n%s", kept.Manifest)
}
// What the build stood on is an edge to another module's artifact, and it is recorded the way
// that artifact is: by path in the store, so the edge still names the same thing when the
// store answers at another address.
if kept.Against[0] != catalogue.ArtifactStoreScheme+"mesh-tools/runtime@"+aDigest {
t.Errorf("what the build stood on was recorded by address: %v", kept.Against)
}
}
// aStore is a module offering the artifact store on 5000, published the long way as the
// distribution module does, so a node may be given another number for it.
func aStore() catalogue.Manifest {
@@ -1,134 +0,0 @@
//go:build integration
package main
import (
"context"
"fmt"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// Issue 1334, the look: when the bus did not answer an ask of any module of a tier, no build is in flight and no event follows,
// so the look itself comes back (it fails, and the looker retries it), asks the modules again, and past the bound fails the plan.
func TestALookComesBackForATierTheBusRefusedWhole(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
for _, name := range []string{"a", "b"} {
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: name, Version: "1"},
inventory.Source{Repository: "novox/" + name, Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
}
p := inventory.Plan{ID: "plan-bus", Repository: "novox/a", Branch: "main", Commit: "c1", Created: time.Now().UTC(),
State: inventory.PlanBuilding, Tiers: [][]string{{"a", "b"}}, Modules: map[string]*inventory.PlanModule{"a": {}, "b": {}}}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
bus := false
was := askABuild
askABuild = func(_ context.Context, source buildSource, _, _ string) (string, error) {
if !bus {
return "", fmt.Errorf("%w: nats: no responders available for request", inventory.ErrNoChain)
}
return "build-" + source.Repository, nil
}
t.Cleanup(func() { askABuild = was })
look := func() bool {
t.Helper()
release, err := inv.HoldPlans(ctx, true)
if err != nil {
t.Fatal(err)
}
defer release()
return advanceHeld(ctx, open)
}
read := func() inventory.Plan {
t.Helper()
got, err := inv.PlanByID(ctx, "plan-bus")
if err != nil {
t.Fatal(err)
}
return got
}
// Every module refused: the look says it failed, which has the looker come back; the plan is building and says why.
if !look() {
t.Fatal("a look that left the whole tier unasked for the bus did not fail, so nothing would come back to it")
}
if got := read(); got.State != inventory.PlanBuilding || !strings.Contains(got.Note, "bus did not answer") ||
got.Modules["a"].State != "" || got.Modules["b"].State != "" {
t.Fatalf("the plan after a refused tier: %s %q %+v %+v", got.State, got.Note, got.Modules["a"], got.Modules["b"])
}
// Again, still refused: the same fact is not saved again, and the look still comes back.
before := read().Revision
if !look() || read().Revision != before {
t.Fatalf("a second refused look failed to come back or saved the plan again (revision %d, was %d)", read().Revision, before)
}
// The bus answers at the next look: both are asked, and nothing waits any more.
bus = true
if look() {
t.Fatal("a look that asked the whole tier failed")
}
if got := read(); got.Modules["a"].State != "asked" || got.Modules["b"].State != "asked" || got.Modules["a"].BusWaitFrom != nil {
t.Fatalf("the tier was not asked once the bus answered: %+v %+v", got.Modules["a"], got.Modules["b"])
}
}
func TestALookFailsTheTierTheBusRefusedPastTheBound(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "a", Version: "1"},
inventory.Source{Repository: "novox/a", Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
long := time.Now().UTC().Add(-busWaitBound - time.Minute)
p := inventory.Plan{ID: "plan-bus", Repository: "novox/a", Branch: "main", Commit: "c1", Created: time.Now().UTC(),
State: inventory.PlanBuilding, Tiers: [][]string{{"a"}}, Modules: map[string]*inventory.PlanModule{"a": {BusWaitFrom: &long}}}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
return "", fmt.Errorf("%w: nats: no responders available for request", inventory.ErrNoChain)
}
t.Cleanup(func() { askABuild = was })
release, err := inv.HoldPlans(ctx, true)
if err != nil {
t.Fatal(err)
}
advanceHeld(ctx, open)
release()
got, err := inv.PlanByID(ctx, "plan-bus")
if err != nil {
t.Fatal(err)
}
if got.State != inventory.PlanFailed || !strings.Contains(got.Note, "the bus did not answer an ask for 10m0s (since ") {
t.Fatalf("a bus silent past its bound did not fail the plan in words: %s %q", got.State, got.Note)
}
// Rebuilt by hand while the bus still reloads: said as waiting, not as a failure and not with an empty reason.
joined, said, err := joinAPlan(ctx, open, "a")
if err != nil || !joined || !strings.Contains(said, "not asked yet: the bus did not answer") {
t.Fatalf("a rebuild left for the bus is said as %v %q %v", joined, said, err)
}
// And retried by hand: the module is named as waiting, not as asked again.
got, _ = inv.PlanByID(ctx, "plan-bus")
if got.State == inventory.PlanFailed {
got.Modules["a"].State, got.Modules["a"].Why = "failed", "x"
if err := inv.SavePlan(ctx, &got); err != nil {
t.Fatal(err)
}
said, err := retryPlan(ctx, open, "plan-bus")
if err != nil || !strings.Contains(said, "not asked yet: the bus did not answer") {
t.Fatalf("a retry left for the bus is said as %q %v", said, err)
}
}
}
-139
View File
@@ -1,139 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// Issue 1334: a bus that does not answer for a moment (it reloads its user list, and JetStream's API has no responder for a few
// seconds) is not a failed build. The module is left unasked and the plan building, and the next look asks it again; only a
// module's own fault, or a bus silent for ten minutes, fails the plan.
func aTierOfThree() (*inventory.Plan, map[string]inventory.Entry) {
p := &inventory.Plan{ID: "plan-1", State: inventory.PlanBuilding, Tiers: [][]string{{"a", "b", "c"}},
Modules: map[string]*inventory.PlanModule{}}
byName := map[string]inventory.Entry{}
for _, m := range []string{"a", "b", "c"} {
byName[m] = inventory.Entry{Source: inventory.Source{Repository: "novox/" + m, Seat: "git", Ref: "main"}}
}
return p, byName
}
func TestABusThatDoesNotAnswerForAMomentDoesNotFailTheTier(t *testing.T) {
p, byName := aTierOfThree()
bus := false
was := askABuild
askABuild = func(_ context.Context, source buildSource, _, _ string) (string, error) {
if source.Repository == "novox/b" && !bus {
return "", fmt.Errorf("%w: the bucket mesh-controller_builds is not on the bus: nats: no responders available for request",
inventory.ErrNoChain)
}
return "build-" + source.Repository, nil
}
t.Cleanup(func() { askABuild = was })
for _, m := range []string{"a", "b", "c"} {
askModule(t.Context(), p, m, byName)
}
if p.State != inventory.PlanBuilding {
t.Fatalf("a bus that did not answer for a moment failed the plan: %s, %q", p.State, p.Note)
}
if p.Modules["a"].State != "asked" || p.Modules["c"].State != "asked" {
t.Fatalf("the modules the bus did answer for were not asked: %+v %+v", p.Modules["a"], p.Modules["c"])
}
if s := p.Modules["b"]; s.State != "" {
t.Fatalf("the module the bus did not answer for is %q, not left unasked", s.State)
}
if got := unaskedOf(*p); len(got) != 1 || got[0] != "b" {
t.Fatalf("the next look would ask %v, not b", got)
}
if !strings.Contains(p.Note, "bus did not answer") {
t.Fatalf("the plan says nothing of the wait: %q", p.Note)
}
// The next look, the bus answering: b is asked and the plan goes on.
bus = true
for _, m := range unaskedOf(*p) {
askModule(t.Context(), p, m, byName)
}
if p.State != inventory.PlanBuilding || p.Modules["b"].State != "asked" || p.Modules["b"].Build == "" {
t.Fatalf("b was not asked once the bus answered: %s %+v", p.State, p.Modules["b"])
}
}
func TestABusSilentForTenMinutesFailsTheTierAsBefore(t *testing.T) {
p, byName := aTierOfThree()
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
return "", fmt.Errorf("%w: nats: no responders available for request", inventory.ErrNoChain)
}
t.Cleanup(func() { askABuild = was })
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanBuilding {
t.Fatalf("the first refusal failed the plan: %q", p.Note)
}
long := time.Now().Add(-busWaitBound - time.Minute)
p.Modules["a"].BusWaitFrom = &long
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanFailed || p.Modules["a"].State != "failed" {
t.Fatalf("a bus silent past its bound did not fail the plan: %s %+v", p.State, p.Modules["a"])
}
// Said in plain words: the bus, the wait, and since when.
want := "a could not be asked for: the bus did not answer an ask for 10m0s (since " + long.UTC().Format(time.RFC3339) +
"): this kind of record lives on the bus"
if !strings.HasPrefix(p.Note, want) || p.Modules["a"].Why != strings.TrimPrefix(want, "a could not be asked for: ")+p.Note[len(want):] {
t.Fatalf("the plan says %q, not words starting %q", p.Note, want)
}
// The wait has ended: a Retry made while the bus still reloads waits again, and does not fail at its first refusal.
if p.Modules["a"].BusWaitFrom != nil {
t.Fatalf("a failed module still holds the start of its wait: %v", p.Modules["a"].BusWaitFrom)
}
p.State = inventory.PlanBuilding
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanBuilding || p.Modules["a"].State != "" || p.Modules["a"].BusWaitFrom == nil {
t.Fatalf("a retry made while the bus is silent did not wait afresh: %s %+v", p.State, p.Modules["a"])
}
}
func TestAPlanWithAModuleWaitingForTheBusIsOneTheLookComesBackTo(t *testing.T) {
p, _ := aTierOfThree()
if busWaiting(*p) {
t.Fatal("a plan whose modules are not asked yet waits for the bus")
}
now := time.Now()
p.Modules["b"] = &inventory.PlanModule{BusWaitFrom: &now}
p.Modules["a"] = &inventory.PlanModule{State: "asked"}
if got := busWaitingIn(*p); len(got) != 1 || got[0] != "b" {
t.Fatalf("the modules waiting for the bus are %v, not b", got)
}
p.State = inventory.PlanFailed
if busWaiting(*p) {
t.Fatal("a plan that is not open is waited for")
}
}
func TestAModulesOwnFaultStillFailsThePlan(t *testing.T) {
p, byName := aTierOfThree()
delete(byName, "a")
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanFailed || !strings.Contains(p.Note, "no longer in the catalogue") {
t.Fatalf("a module gone from the catalogue did not fail the plan: %s %q", p.State, p.Note)
}
p, byName = aTierOfThree()
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
return "", errors.New("git.example/novox/a is no repository the catalogue builds a module from")
}
t.Cleanup(func() { askABuild = was })
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanFailed || p.Modules["a"].State != "failed" {
t.Fatalf("a module's own refusal did not fail the plan: %s %+v", p.State, p.Modules["a"])
}
}
-35
View File
@@ -1,16 +1,12 @@
package main
import (
"errors"
"fmt"
"slices"
"sort"
"strings"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/inventory"
)
@@ -18,37 +14,6 @@ import (
// (issue 1334).
const busWaitBound = 10 * time.Minute
// busDidNotAnswer says an ask failed because the bus's JetStream had no responder at that moment, as it has for a few seconds
// when the bus reloads its user list (novox/hq issue 353), and not because of the module: the same ask is made again.
func busDidNotAnswer(err error) bool {
if err == nil {
return false
}
if errors.Is(err, nats.ErrNoResponders) || errors.Is(err, nats.ErrJetStreamNotEnabled) ||
errors.Is(err, jetstream.ErrJetStreamNotEnabled) || errors.Is(err, inventory.ErrNoChain) {
return true
}
// An error that crossed a boundary as text only.
text := err.Error()
return strings.Contains(text, "no responders available") || strings.Contains(text, "jetstream not enabled")
}
// waitForTheBus leaves a module whose ask the bus did not answer unasked, the plan building, and says so in the plan's note: the
// next look asks the tier's unasked modules again. It says false, so the caller fails the plan as for any refusal, once the bus
// has been silent for busWaitBound.
func waitForTheBus(p *inventory.Plan, state *inventory.PlanModule, name string, err error, now time.Time) bool {
if state.BusWaitFrom == nil {
state.BusWaitFrom = &now
}
if now.Sub(*state.BusWaitFrom) > busWaitBound {
return false
}
state.State, state.Why = "", ""
p.Note = fmt.Sprintf("tier %d: %s not asked: the bus did not answer (%v); asked again at the next look, failing after %s",
p.Tier, name, firstLine(err.Error()), busWaitBound)
return true
}
// busWaiting says an open plan has a module of its current tier that the bus did not answer an ask of: the look must come back
// for it, as nothing else will when no build of the tier is in flight (issue 1334).
func busWaiting(p inventory.Plan) bool {
+23
View File
@@ -0,0 +1,23 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// A retry of a plan failed past the bus's bound, whose module the bus still does not answer, names that module as waiting:
// read while the plan still says failed, it was named nowhere, and the retry said "asked" with nothing after it (issue
// 1403, met again once the hand rebuild no longer resumed the plan first).
func TestARetryNamesTheModulesStillWaitingForTheBus(t *testing.T) {
since := time.Now().UTC()
p := inventory.Plan{ID: "plan-bus", State: inventory.PlanFailed, Tiers: [][]string{{"a"}},
Modules: map[string]*inventory.PlanModule{"a": {BusWaitFrom: &since}}}
waiting := busWaitingOnRetry(p)
_, said := retryWords(p, []string{"a"}, nil, waiting)
if !strings.Contains(said, "a not asked yet: the bus did not answer") || strings.Contains(said, "asked ;") {
t.Fatalf("a retry left for the bus is said as %q", said)
}
}
+4
View File
@@ -645,6 +645,10 @@ func doesWords(act conditions.Action) string {
return "the delivery is stopped"
case act.Verb == broker.ControllerSeat+".plans" && act.Arguments["retry"] != "":
return "the failed builds are asked again"
case act.Verb == broker.ControllerSeat+".push" && act.Arguments["node"] != "":
return "the held builds are sent to " + act.Arguments["node"]
case act.Verb == broker.ControllerSeat+".upgrade" && act.Arguments["release-backlog"] != "":
return "the builds held after the failed delivery are released"
case strings.HasSuffix(act.Verb, ".restart"):
return "its service is restarted on " + act.Machine
}
@@ -33,7 +33,7 @@ func TestAWarrantForRetryRetriesAFailedPlanInTheStoreAndEndsItsCondition(t *test
Note: "a failed to build in tier 0",
Delivery: &inventory.PlanDelivery{},
Modules: map[string]*inventory.PlanModule{"a": {State: "failed", AskedAt: &before, Build: "build-1",
Why: link.KilledByHand}},
Why: "killed by hand"}},
Times: &inventory.PlanTimes{Cut: &before, Started: &before, Ended: &ended, Class: inventory.ClassLeaf}}
if err := open.inventory.SavePlan(ctx, &failed); err != nil {
t.Fatal(err)
+2 -17
View File
@@ -66,17 +66,8 @@ func TestAPlanGoesOnWithoutAMachineAwayWhenItsTierIsSent(t *testing.T) {
// The build machine and the machines, as far as the plan meets them: an ask is counted, a send is recorded and kept.
var mu sync.Mutex
var asks []string
var sends [][]string
wasAsk := askABuild
askABuild = func(_ context.Context, source buildSource, _, _ string) (string, error) {
mu.Lock()
defer mu.Unlock()
id := link.NewBuildID(time.Now().Add(time.Duration(len(asks)) * time.Millisecond))
asks = append(asks, source.Repository+"#"+id)
return id, nil
}
t.Cleanup(func() { askABuild = wasAsk })
everyKeyDone(t) // the builds of the tiers: each module is taken from a done key as soon as the walk reaches it
digests := 0
wasSend := sendRollout
sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) {
@@ -198,13 +189,7 @@ func TestAPlanGoesOnWithoutAMachineAwayWhenItsTierIsSent(t *testing.T) {
if !strings.Contains(moved, "laptop was away: sent when it is heard again") || strings.Contains(moved, "anchor") {
t.Errorf("1161/AC2: the plan says %q; want laptop named away and not anchor", moved)
}
// The ask for tier 1's build is the next step, and records its build id on the module.
eventually(t, "tier 1 asked", func() bool { m := plan().Modules["b"]; return m != nil && m.Build != "" })
b := plan().Modules["b"]
askedAt, _ := link.BuildAskedAt(b.Build)
record(b.Build, "b", "c2", askedAt)
register("b", "c2", askedAt)
planBuilt(ctx, open, "b", "c2", "", askedAt, b.Build)
// Tier 1's module is taken from its key, which the walk's look does by itself.
eventually(t, "tier 1 sent to anchor alone", func() bool {
mu.Lock()
defer mu.Unlock()
+5 -2
View File
@@ -232,8 +232,8 @@ func (f following) hearMerge(ctx context.Context, m link.SourceMoved, now time.T
fmt.Print(nothingReadsIt(repository, m))
return nil
}
// **A module the merge adds is asked for now** (novox/hq issue 1617), not when the batch is cut.
askNewModulesAtHearing(ctx, inv, m, entries, read, facts.from)
// A module the merge adds is registered from the builder's first done key of it, which the builder's write wakes
// (novox/hq ADR 0501 §6): nothing is asked here.
// **Its delivery group's word, before the plans are held** (ADR 0306 decision 2): a slow answer holds up no walk.
own := ownPath(touches)
waits, holding, err := groupWaitHolding(ctx, inv, m, deliverySeatHeld(entries))
@@ -800,6 +800,9 @@ type keptMerge struct {
// FirstBuild says the merge is a first merge into a repository the mesh builds nothing from, kept to ask the first
// build of its modules once mesh-delivery's word on it is kept (novox/hq ADR 0432, issue 1146).
FirstBuild bool `json:"first_build,omitempty"`
// Held says which first builds of the merge were refused for a word the running build-agent does not read: the merge
// waits for a build-agent that reads them, and is never failed (novox/hq ADR 0482 §5, issue 1173).
Held []heldFirstBuild `json:"held,omitempty"`
}
// announcedOf is a kept merge as the forge announced it, and what it moved; empty where the record says none.
+1 -1
View File
@@ -264,7 +264,7 @@ func TestAPlanLineNamesWhatItMoves(t *testing.T) {
}
return nil
}
if got, want := planLineOn(p, now, pauseView{}, tierAtLeast, running),
if got, want := planLineOn(p, now, tierAtLeast, running),
"mesh-catalog #175 a tap shows its outcome · messenger, telegram → novox · tier 1 of 1, building for 2m0s"; got != want {
t.Fatalf("the line reads %q, want %q", got, want)
}
File diff suppressed because it is too large Load Diff
-43
View File
@@ -1,43 +0,0 @@
package main
import (
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func claiming(module, seat string, on ...string) inventory.Entry {
return inventory.Entry{
Manifest: catalogue.Manifest{Module: module, Claims: []catalogue.Claim{{Name: seat}}},
On: on,
}
}
// The controller asks the build role that has a holder (novox/hq ADR 0190 handover): the retired
// one while only the builder is assigned, the current one from the first build-agent on, and the
// current one when nothing holds either — where every ask goes once the handover is done.
func TestTheControllerAsksTheBuildRoleThatHasAHolder(t *testing.T) {
onlyTheBuilder := []inventory.Entry{
claiming("builder", link.TheBuildMachineBefore, "anchor"),
claiming("build-agent", link.TheBuildMachine), // registered, assigned nowhere yet
}
if got := buildSeatAmong(onlyTheBuilder); got != link.TheBuildMachineBefore {
t.Errorf("with only the builder assigned, asked %q", got)
}
bothHeld := []inventory.Entry{
claiming("builder", link.TheBuildMachineBefore, "anchor"),
claiming("build-agent", link.TheBuildMachine, "home-server"),
}
if got := buildSeatAmong(bothHeld); got != link.TheBuildMachine {
t.Errorf("with a build-agent assigned anywhere, asked %q", got)
}
neither := []inventory.Entry{claiming("builder", link.TheBuildMachineBefore)}
if got := buildSeatAmong(neither); got != link.TheBuildMachine {
t.Errorf("with no holder of either, asked %q, want the current role", got)
}
if got := buildSeatAmong(nil); got != link.TheBuildMachine {
t.Errorf("an empty catalogue asks %q", got)
}
}
+86 -105
View File
@@ -7,6 +7,7 @@ import (
"fmt"
"os"
"regexp"
"slices"
"strings"
"time"
@@ -188,25 +189,13 @@ func (f *sourceForms) same(aRepository, aSeat, bRepository, bSeat string) bool {
return f.canonical(aRepository, aSeat) == f.canonical(bRepository, bSeat)
}
// buildsFrom says the catalogue builds some module from this repository.
func (f *sourceForms) buildsFrom(entries []inventory.Entry, repository, seat string) bool {
for _, e := range entries {
if e.Provided || e.Source.Repository == "" {
continue
}
if f.same(e.Source.Repository, e.Source.Seat, repository, seat) {
return true
}
}
return false
}
// forgeFacts is what the mesh's forge says of a repository a module is registered from (novox/hq ADR 0266).
type forgeFacts struct {
// ID is the forge's own id of the repository: what tells it from one deleted and made again by its name.
ID int64
// Guarded is whether the branch is protected as a module's trunk must be: no direct push, at least one
// required status, and no administrator merging past one. Why says what is missing when it is not.
// Guarded is whether the branch is protected as a module's trunk must be: no direct push, a merge whitelist naming
// the tools' account and no agent's account, and no administrator merging past the rule (novox/hq ADR 0534 §6; no
// required status, which the forge no longer asks, §1). Why says what is missing when it is not.
Guarded bool
Why string
}
@@ -361,46 +350,68 @@ func askTheForgeOnce(ctx context.Context, owner, repo, branch string) (forgeFact
return facts, nil
}
// forgeToolsAccount is the forge tools' account, which main's merge whitelist names for mesh-delivery (novox/hq ADR 0398
// §2, the gitea module's tools-account.ts TOOLS_ACCOUNT); beside it the whitelist names only people's own logins, the
// operator's (ADR 0534 §2), never an agent's account.
const forgeToolsAccount = "mesh-tools"
// isAgentAccount is whether a forge login is a node's agent's account, agent-<node> (novox/hq ADR 0398 §1): one that
// may never merge into a module's trunk by itself.
func isAgentAccount(login string) bool { return strings.HasPrefix(login, "agent-") }
// protectionRule is a branch protection rule as the forge's tool summarises it.
type protectionRule struct {
Rule string `json:"rule"`
Push bool `json:"push"`
RequiredStatuses []string `json:"required_statuses"`
AdminMayOverride bool `json:"admin_may_override"`
// MergeWhitelist is who may merge into the branch; nil when the rule keeps no merge whitelist (anyone who writes).
MergeWhitelist []string `json:"merge_whitelist"`
}
// judgedRule judges the one rule the forge applies to a branch, as the forge picks it: the rule named for the
// branch, else the first glob rule, in the forge's order, that covers it (gitea's first matching rule). Never a
// later rule that happens to be stronger: the forge does not read it.
// later rule that happens to be stronger: the forge does not read it. A trunk is trusted for who can land a commit on
// it, not for a status it requires (novox/hq ADR 0534 §6): no push, a merge whitelist naming the tools' account and
// beside it only people's own logins, never an agent's, and no administrator merging past the rule; then a commit lands
// there through mesh-delivery, on a passing mesh/check and the operator's Approve (ADR 0534 §4), or by the operator's
// own hand at the forge (§2).
func judgedRule(rules []protectionRule, branch string) forgeFacts {
var applied *protectionRule
for i := range rules {
if rules[i].Rule == branch {
applied = &rules[i]
break
}
}
if applied == nil {
for i := range rules {
if ruleCovers(rules[i].Rule, branch) {
applied = &rules[i]
break
}
}
}
switch r := applied; {
switch r := appliedRule(rules, branch); {
case r == nil:
return forgeFacts{Why: fmt.Sprintf("no protection rule covers %s", branch)}
case r.Push:
return forgeFacts{Why: fmt.Sprintf("the rule %s lets a person push to %s directly", r.Rule, branch)}
case len(r.RequiredStatuses) == 0:
return forgeFacts{Why: fmt.Sprintf("the rule %s requires no status before a merge into %s", r.Rule, branch)}
case r.MergeWhitelist == nil || len(r.MergeWhitelist) == 0:
return forgeFacts{Why: fmt.Sprintf("the rule %s keeps no merge whitelist naming %s alone for %s", r.Rule, forgeToolsAccount, branch)}
case !slices.Contains(r.MergeWhitelist, forgeToolsAccount):
return forgeFacts{Why: fmt.Sprintf("the rule %s lets %s merge into %s, and not %s", r.Rule,
strings.Join(r.MergeWhitelist, ", "), branch, forgeToolsAccount)}
case slices.ContainsFunc(r.MergeWhitelist, isAgentAccount):
return forgeFacts{Why: fmt.Sprintf("the rule %s lets an agent's account merge into %s: %s", r.Rule, branch,
strings.Join(r.MergeWhitelist, ", "))}
case r.AdminMayOverride:
return forgeFacts{Why: fmt.Sprintf("the rule %s lets an administrator merge into %s past a status", r.Rule, branch)}
return forgeFacts{Why: fmt.Sprintf("the rule %s lets an administrator merge into %s past it", r.Rule, branch)}
}
return forgeFacts{Guarded: true}
}
// appliedRule is the one rule the forge applies to a branch: the rule named for it, else the first glob rule, in the
// forge's order, that covers it; nil when none does.
func appliedRule(rules []protectionRule, branch string) *protectionRule {
for i := range rules {
if rules[i].Rule == branch {
return &rules[i]
}
}
for i := range rules {
if ruleCovers(rules[i].Rule, branch) {
return &rules[i]
}
}
return nil
}
// ruleCovers says a protection rule's name — a branch, or a glob of them — covers a branch, as the forge reads it.
func ruleCovers(rule, branch string) bool {
if rule == branch {
@@ -463,14 +474,42 @@ func (f *sourceForms) onTheForge(repository, seat string) (owner, name string, o
// - from the module's registered repository — the same repository by the forge's own id, not only its name; or,
// for a module new to the catalogue, from a repository the catalogue builds another module from;
// - and from a repository on the mesh's forge whose trunk is protected as a trunk must be: no direct push, at
// least one required status, no administrator merging past one.
// a merge whitelist naming the tools' account and no agent's, no administrator merging past the rule (ADR 0534 §6).
//
// Anything else only when the build request was kept as asked at the controller's terminal, for this very
// repository and path; or, for the build of a merge, when mesh-delivery announced it on the operator's Approve of
// that head (novox/hq ADR 0420 §7 for a moved module, ADR 0432 for a new one), the forge's checks still holding. path is the module's directory as built; branch the trunk it is registered from.
func mayRegisterFrom(ctx context.Context, inv *inventory.Inventory, module string, built inventory.Source,
buildID, path, branch string) (int64, error) {
return mayRegisterIn(ctx, inv, newSourceForms(ctx, inv), module, built, buildID, path, branch)
ask registrationAsk, path, branch string) (int64, error) {
return mayRegisterIn(ctx, inv, newSourceForms(ctx, inv), module, built, ask, path, branch)
}
// registrationAsk is what a registration's warrant is read from (novox/hq ADR 0501 §3): the build an outcome answers,
// whose kept request says whether it was asked at the terminal or for a merge (build-agent's outcome, the builder's own
// module until R4g); or, for a take-in from the builder's done key, the merge the indexer's trunk value names and
// whether the operator asked the registration at the controller's terminal (ADR 0501 §4). A key asks no build, so it
// keeps no request.
type registrationAsk struct {
// Build is the build whose kept request is read; empty for a take-in from a key.
Build string
// Merge is the commit of the merge a key's module is registered from (the indexer's trunk value); empty for an
// outcome, whose request names it.
Merge string
// Terminal is a key registration the operator asked at the controller's terminal for this repository and path.
Terminal bool
}
// String is the ask as the registration's words name it.
func (a registrationAsk) String() string {
switch {
case a.Build != "":
return a.Build
case a.Terminal:
return "the registration asked at the terminal"
case a.Merge != "":
return fmt.Sprintf("the merge %.8s", a.Merge)
}
return "a key"
}
// catalogueReader is the catalogue's manifests, as the router's records are found in them.
@@ -490,15 +529,19 @@ type registrations interface {
// mayRegisterIn is mayRegisterFrom on what it reads of the mesh handed in.
func mayRegisterIn(ctx context.Context, inv registrations, forms *sourceForms, module string, built inventory.Source,
buildID, path, branch string) (int64, error) {
ask registrationAsk, path, branch string) (int64, error) {
buildID := ask.String()
was, err := inv.SourceOf(ctx, module)
isNew := errors.Is(err, inventory.ErrNoSuchModule)
if err != nil && !isNew {
return 0, err
}
terminal, err := askedHereFor(ctx, inv, forms, buildID, built, path)
if err != nil {
return 0, err
terminal := ask.Terminal
if ask.Build != "" {
var err error
if terminal, err = askedHereFor(ctx, inv, forms, ask.Build, built, path); err != nil {
return 0, err
}
}
refuse := func(why string) (int64, error) {
return 0, fmt.Errorf("%w: %s. A module is registered from such a source only by a build asked at the "+
@@ -542,7 +585,7 @@ func mayRegisterIn(ctx context.Context, inv registrations, forms *sourceForms, m
// The forge's protection and the repository's identity are checked below as for every registration.
movedWhy := ""
if why != "" && !terminal {
accepted, err := approvedMove(ctx, inv, forms, buildID, built)
accepted, err := approvedMove(ctx, inv, forms, ask, built)
switch {
case err == nil:
movedWhy, why = accepted, ""
@@ -661,65 +704,3 @@ func sourceWords(repository, seat string) string {
}
return buildSource{Repository: repository, Seat: seat}.String()
}
// firstBuildWarrant is what the controller checked before it asks the first build of the modules a merge adds to a
// repository nothing is built from: the merge was made on the operator's Approve of its head (approvedAnnounced). It
// travels in the context of that ask alone, in this process, so no verb carries one.
type firstBuildWarrant struct {
repository string // owner/name on the git seat
paths map[string]bool
why string
}
type firstBuildWarrantKey struct{}
// withFirstBuildWarrant is ctx carrying the warrant for asking the first builds of paths in repository.
func withFirstBuildWarrant(ctx context.Context, repository string, paths []string, why string) context.Context {
w := &firstBuildWarrant{repository: repository, paths: map[string]bool{}, why: why}
for _, p := range paths {
w.paths[strings.Trim(p, "/")] = true
}
return context.WithValue(ctx, firstBuildWarrantKey{}, w)
}
func firstBuildWarrantIn(ctx context.Context) *firstBuildWarrant {
w, _ := ctx.Value(firstBuildWarrantKey{}).(*firstBuildWarrant)
return w
}
// covers says the warrant is for exactly this ask: the repository on the git seat, and a module path the merge holds.
func (w *firstBuildWarrant) covers(source buildSource, path string) bool {
return w != nil && source.Seat == gitSeat && strings.EqualFold(strings.Trim(source.Repository, "/"), w.repository) &&
w.paths[strings.Trim(path, "/")]
}
// verbMayAsk refuses, for a call through a verb, a build of a repository the catalogue builds no module from
// (novox/hq ADR 0266): the build node would run what an agent wrote, and its outcome could never be
// registered anyway. url is the repository as it is cloned. At the terminal anything may be asked.
func verbMayAsk(ctx context.Context, source buildSource, url, path string) error {
if startedAtTheTerminal() {
return nil
}
// **Or the first build of a module a merge adds on the operator's Approve** (novox/hq ADR 0432, issue 1146): the
// controller's own ask, carrying the warrant it checked, for that repository and the module paths the merge holds.
if firstBuildWarrantIn(ctx).covers(source, path) {
return nil
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
entries, err := open.inventory.Catalogued(ctx)
if err != nil {
return err
}
forms := newSourceForms(ctx, open.inventory)
if forms.buildsFrom(entries, source.Repository, source.Seat) || forms.buildsFrom(entries, url, "") {
return nil
}
return terminalRefusal("%s is no repository the catalogue builds a module from, and a build of any other is "+
"asked at the controller's terminal only, never through a verb: a build node runs what the repository "+
"says, and its outcome would register a module the next push sends — whoever may call a verb includes "+
"agents (novox/hq ADR 0266). Nothing was asked", source)
}
@@ -129,27 +129,6 @@ func TestADeliveryFromTheRegisteredRepositoryIsRegistered(t *testing.T) {
}
}
func TestAVerbAsksNoBuildOfARepositoryTheCatalogueDoesNotBuildFrom(t *testing.T) {
theCatalogue(t)
ctx := t.Context()
t.Setenv(verbVar, "build")
t.Setenv(link.CallerVar, "node-tools.anchor, through the mesh-controller seat")
err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, "http://forge.internal:20000/agent/sudo.git", "")
var policy *heldAtTheTerminal
if !errors.As(err, &policy) {
t.Fatalf("a verb's build of an agent's repository was asked: %v", err)
}
if err := verbMayAsk(ctx, buildSource{Repository: "novox/mesh-catalog", Seat: "git"},
"http://forge.internal:20000/novox/mesh-catalog.git", ""); err != nil {
t.Fatalf("a verb's build of the catalogue repository was refused: %v", err)
}
t.Setenv(verbVar, "")
t.Setenv(link.CallerVar, "")
if err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, "", ""); err != nil {
t.Fatalf("the terminal was refused: %v", err)
}
}
func TestARollbackNeverPutsBackABuildFromAnotherRepository(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
+38 -4
View File
@@ -102,16 +102,17 @@ func init() {
// The forge applies one rule to a branch: the rule named for it, else the first glob that covers it. A stronger
// rule later in the list is not what guards the branch, and is not read as if it were (the confirmation review).
func TestTheRuleJudgedIsTheOneTheForgeApplies(t *testing.T) {
guarded := protectionRule{Rule: "*", RequiredStatuses: []string{"mesh/merge-gate"}}
open := protectionRule{Rule: "main", Push: true, RequiredStatuses: []string{"mesh/merge-gate"}}
tools := []string{forgeToolsAccount}
guarded := protectionRule{Rule: "*", MergeWhitelist: tools}
open := protectionRule{Rule: "main", Push: true, MergeWhitelist: tools}
if f := judgedRule([]protectionRule{guarded, open}, "main"); f.Guarded {
t.Error("an exact rule letting pushes was passed over for a glob that guards")
}
if f := judgedRule([]protectionRule{{Rule: "ma*", RequiredStatuses: nil}, {Rule: "*", RequiredStatuses: []string{"x"}}},
if f := judgedRule([]protectionRule{{Rule: "ma*", MergeWhitelist: nil}, {Rule: "*", MergeWhitelist: tools}},
"main"); f.Guarded || !strings.Contains(f.Why, "ma*") {
t.Errorf("the first glob covering the branch was not the one judged: %+v", f)
}
if f := judgedRule([]protectionRule{{Rule: "release/*"}, {Rule: "main", RequiredStatuses: []string{"x"}}}, "main"); !f.Guarded {
if f := judgedRule([]protectionRule{{Rule: "release/*"}, {Rule: "main", MergeWhitelist: tools}}, "main"); !f.Guarded {
t.Errorf("the rule named for the branch was not judged: %+v", f)
}
if f := judgedRule(nil, "main"); f.Guarded {
@@ -119,6 +120,39 @@ func TestTheRuleJudgedIsTheOneTheForgeApplies(t *testing.T) {
}
}
// **A module's trunk is trusted when it takes no push, only the tools' account and people's own logins merge into it,
// and no administrator merges past its rule** (novox/hq ADR 0534 §2, §6): the forge requires no status to merge (§1), so
// a rule requiring none is trusted; one with no merge whitelist, no tools' account in it, an agent's account in it, a
// push allowed or an administrator's override is not.
func TestATrunkIsTrustedForItsMergersNotForAStatus(t *testing.T) {
tools := []string{forgeToolsAccount}
for _, c := range []struct {
name string
rule protectionRule
want bool
why string
}{
{"no status, no push, the tools alone merge", protectionRule{Rule: "main", MergeWhitelist: tools}, true, ""},
{"a status required as well", protectionRule{Rule: "main", MergeWhitelist: tools, RequiredStatuses: []string{"mesh/check"}}, true, ""},
{"no merge whitelist", protectionRule{Rule: "main"}, false, "merge whitelist"},
{"an empty merge whitelist", protectionRule{Rule: "main", MergeWhitelist: []string{}}, false, "merge whitelist"},
{"the operator's own login beside the tools", protectionRule{Rule: "main", MergeWhitelist: []string{forgeToolsAccount, "jschoubben"}}, true, ""},
{"an agent's account in the whitelist", protectionRule{Rule: "main", MergeWhitelist: []string{forgeToolsAccount, "agent-ace"}}, false, "agent-ace"},
{"an agent's account alone", protectionRule{Rule: "main", MergeWhitelist: []string{"agent-ace"}}, false, "agent-ace"},
{"a person's login without the tools", protectionRule{Rule: "main", MergeWhitelist: []string{"jschoubben"}}, false, forgeToolsAccount},
{"a push allowed", protectionRule{Rule: "main", Push: true, MergeWhitelist: tools}, false, "push"},
{"an administrator's override", protectionRule{Rule: "main", MergeWhitelist: tools, AdminMayOverride: true}, false, "administrator"},
} {
f := judgedRule([]protectionRule{c.rule}, "main")
if f.Guarded != c.want || !strings.Contains(f.Why, c.why) {
t.Errorf("%s: %+v, want guarded %v saying %q", c.name, f, c.want, c.why)
}
if strings.Contains(f.Why, "requires no status") {
t.Errorf("%s: a trunk is still judged on a required status: %s", c.name, f.Why)
}
}
}
// An id the forge could not give keeps the id already recorded.
// **The operator's Approve of a reviewed group registers a moved module from its new repository** (novox/hq ADR 0420 §7,
-11
View File
@@ -2,12 +2,10 @@ package main
import (
"bytes"
"reflect"
"strings"
"testing"
"time"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -68,12 +66,3 @@ func TestBuildsSaysTheSourceFingerprint(t *testing.T) {
t.Errorf("the other lines changed:\n%s\nwant\n%s", plain.String(), want)
}
}
// 1215/AC4: what the builder said of the inputs is what the controller keeps of the build.
func TestTheControllerKeepsTheInputsABuilderSaid(t *testing.T) {
in := &sdkdelivery.SourceInputs{Module: "mailu", Tree: "t", Toolchains: map[string]string{"code": "ts sha256:aa 01"}}
kept := buildFrom(link.BuildResult{ID: "build-1", Repository: "r", Commit: "c", SourceFingerprint: "src1:x", SourceInputs: in})
if !reflect.DeepEqual(kept.SourceInputs, in) {
t.Errorf("the controller keeps %+v, the builder said %+v", kept.SourceInputs, in)
}
}
@@ -0,0 +1,109 @@
//go:build integration
package main
import (
"context"
"testing"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/testbus"
)
// builderOnTheBus is the builder's and the indexer's buckets on a test's own bus, which the controller's connection for
// the moment (servingBus) reaches: what a take-in from a key reads (R4e, issue 1851).
type builderOnTheBus struct {
js *broker.JetStream
builds, modules, manifests jetstream.KeyValue
}
func aBuilderOnTheBus(t *testing.T) builderOnTheBus {
t.Helper()
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
api, err := jetstream.New(js.Conn())
if err != nil {
t.Fatal(err)
}
ctx := t.Context()
b := builderOnTheBus{js: js}
for _, c := range []struct {
into *jetstream.KeyValue
bucket string
history uint8
}{
{&b.builds, sdkdelivery.BuilderBuildsBucket, sdkdelivery.BuilderBuildsHistory},
{&b.modules, sdkdelivery.ModulesBucket, sdkdelivery.ModulesHistory},
{&b.manifests, sdkdelivery.ManifestsBucket, 1},
} {
kv, err := api.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: c.bucket, History: c.history})
if err != nil {
t.Fatal(err)
}
*c.into = kv
}
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
withTrunkList(t, true)
// The indexer on this bus is read, not the "not held" a mesh with no bus reads (aMeshOnTheStore).
wasIndexed := indexedModule
indexedModule = theIndexedModule
t.Cleanup(func() { indexedModule = wasIndexed })
return b
}
// indexed writes the indexer's trunk value of module at commit, naming fp, and its manifest: a module of one archive.
func (b builderOnTheBus) indexed(t *testing.T, module, version, repository, path, commit, fp string) string {
t.Helper()
keys, trunk := aKeyFor(module, version, fp, repository, path, commit)
raw := keys.manifests[trunk.Manifest]
value, err := sdkdelivery.EncodeManifest(sdkdelivery.Manifest{Module: module, JSON: raw})
if err != nil {
t.Fatal(err)
}
if _, err := b.manifests.Put(context.Background(), trunk.Manifest, value); err != nil {
t.Fatal(err)
}
trunk.FP = "src1:" + fp[len("src2:"):]
encoded, err := sdkdelivery.EncodeModule(trunk)
if err != nil {
t.Fatal(err)
}
if _, err := b.modules.Put(context.Background(), module, encoded); err != nil {
t.Fatal(err)
}
return trunk.Manifest
}
// built writes the builder's key of module at fp: done, its one archive named with a reference, or in another state.
func (b builderOnTheBus) built(t *testing.T, module, version, fp, manifest, state string) {
t.Helper()
keys, _ := aKeyFor(module, version, fp, "", "", "")
v := keys.keys[module].Build
v.Manifest = manifest
if state != sdkdelivery.BuildDone {
v = sdkdelivery.BuilderBuild{State: state, By: "ace"}
if state == sdkdelivery.BuildFailed {
v.Error = "digests differ: ace sha256:1, g14 sha256:2"
}
}
value, err := sdkdelivery.EncodeBuilderBuild(v)
if err != nil {
t.Fatal(err)
}
key, err := sdkdelivery.BuilderBuildKey(module, fp)
if err != nil {
t.Fatal(err)
}
if _, err := b.builds.Put(context.Background(), key, value); err != nil {
t.Fatal(err)
}
}
// theIndexedModule is the look at the indexer as the controller makes it, kept before any test replaces it.
var theIndexedModule = indexedModule
+3 -7
View File
@@ -241,10 +241,9 @@ func mergeAskPrefix(number int, head string) string {
return fmt.Sprintf("merge-%d-%s-", number, head)
}
// askersOfMerges are the askers a merge's warrant is read under: the teller, and mesh-delivery for a merge before release 3e
// (ADR 0437 §6). Release 4's controller work stops accepting mesh-delivery (ADR 0450 §8); a merge asked under it earlier is
// still found here until then.
func askersOfMerges() []string { return []string{sdkdelivery.TellerModule, theDelivery} }
// askersOfMerges are the askers a merge's warrant is read under: the teller alone (ADR 0437 §6), since R4e stopped taking
// mesh-delivery's (ADR 0450 §8; issue 1318).
func askersOfMerges() []string { return []string{sdkdelivery.TellerModule} }
// caughtUpWarrant judges a chosen merge ask of the router as the warrant of a merge: the ask is the operator's, the warrant
// answers that very ask as it was published, and it is the operator's Approve proved by the channel it came through (ADR 0437
@@ -391,9 +390,6 @@ var findMergeWarrant = func(ctx context.Context, inv catalogueReader, owner, rep
if asker == sdkdelivery.TellerModule && !strings.HasPrefix(id, sdkdelivery.AskKindMerge+"-") {
continue
}
if asker == theDelivery && !named {
continue
}
ask, w, err := closedRecordOf(ctx, inv, asker, id)
if err != nil {
return "", false, "", err
+6 -34
View File
@@ -69,12 +69,12 @@ func TestAForgeThatDoesNotAnswerIsAnErrorNotAnEmptyTrunk(t *testing.T) {
func TestACaughtUpMergeIsAcceptedOnTheSameWarrantAsAnAnnouncedOne(t *testing.T) {
ok := approvedMerge(t)
for _, asker := range []string{sdkdelivery.TellerModule, theDelivery} {
w := ok.warrant
w.Asker = asker
if _, err := caughtUpWarrant(asker, &ok.ask, &w); err != nil {
t.Errorf("the operator's Approve under %s was refused: %v", asker, err)
}
if _, err := caughtUpWarrant(sdkdelivery.TellerModule, &ok.ask, &ok.warrant); err != nil {
t.Errorf("the operator's Approve under the teller was refused: %v", err)
}
// 1318 (ADR 0450 §8): mesh-delivery's warrant of before the gate is no longer taken.
if _, err := caughtUpWarrant(theDelivery, &ok.ask, &ok.warrant); err == nil {
t.Error("a warrant under mesh-delivery was accepted")
}
if _, err := caughtUpWarrant("keycloak", &ok.ask, &ok.warrant); err == nil {
t.Error("a warrant of an asker the controller takes none from was accepted")
@@ -229,34 +229,6 @@ func TestAGroupsWarrantIsFoundByTheAsksWordsForThePullRequestAndItsHead(t *testi
}
}
// Keycloak's merge was asked as mesh-delivery.merge-378-b9104454f9a5-1, before release 3e: the id names the pull request and
// the first twelve digits of its head, and the warrant is found by them under mesh-delivery.
func TestAWarrantAskedUnderMeshDeliveryIsFoundByThePullRequestAndItsHead(t *testing.T) {
ok := approvedMerge(t)
head := "b9104454f9a5e0c4d3b2a1f0e9d8c7b6a5948372"
askID := "merge-378-b9104454f9a5-1"
ask, w := ok.ask, ok.warrant
ask.ID, w.Ask, w.Asker = askID, askID, theDelivery
w.AskDigest = ask.Digest()
withRouter(t, map[string]map[string]aRouterRecord{theDelivery: {
"merge-377-aaaaaaaaaaaa-1": {},
askID: {ask, w},
}})
why, found, reason, err := findMergeWarrant(t.Context(), nil, "novox", "mesh-catalog", 378, head)
if err != nil || !found || !strings.Contains(why, askID) {
t.Fatalf("the warrant was not found: %q %v %q %v", why, found, reason, err)
}
// Another head names another ask, and none is found.
if _, found, _, _ := findMergeWarrant(t.Context(), nil, "novox", "mesh-catalog", 378, "ffffffffffff0000"); found {
t.Fatal("another head found a warrant")
}
// A router that cannot be read is an error, never "no warrant".
routerAskIDs = func(context.Context, catalogueReader, string) ([]string, error) { return nil, errors.New("no answer") }
if _, found, _, err := findMergeWarrant(t.Context(), nil, "novox", "mesh-catalog", 378, head); found || err == nil {
t.Fatalf("an unreadable router read as no warrant: %v %v", found, err)
}
}
// A trunk shorter than a page, or with none, is read without a fault: the pages end when the forge's do.
func TestAShortTrunkIsReadWholeAndAnEmptyOneIsNoMerges(t *testing.T) {
was := trunkCommitsPage
+2 -44
View File
@@ -51,55 +51,13 @@ func attachTheChain(ctx context.Context, inv *inventory.Inventory, server *link.
fmt.Printf("chain: the builds are not laid out under their modules yet, and none is read until they are: %v\n", err)
go layBuildsOutSoon(ctx, inv)
}
// And the requests the move of delivery 1 found never handed over, asked again once (askAgain), once it is serving.
go askAgainSoon(ctx, inv)
// The requests the move of delivery 1 found never handed over are not asked again: the controller asks no build
// (novox/hq ADR 0501; issue 1851, A10), and the builder builds every fingerprint the indexer writes.
// And what delivery 1 recorded of what each machine was sent, copied onto the keys of the assignments (CopySentBuilds).
go copySentBuildsSoon(ctx, inv)
return nil
}
// askAgain asks once more for each build request the move found never handed over (novox/hq issue 981 note 7 Q5): the
// requests a merge could not ask for, which a person's hand `build` would otherwise have to repeat. **At most once**: a
// request is claimed before it is asked and settled after, so a crash between loses the ask, which is where it stood.
func askAgain(ctx context.Context, inv *inventory.Inventory) {
pending, err := inv.RequestsToAskAgain(ctx)
if err != nil {
fmt.Printf("chain: the requests never handed over cannot be read: %v\n", err)
return
}
for _, r := range pending {
if claimed, err := inv.ClaimToAskAgain(ctx, r.ID); err != nil || !claimed {
if err != nil {
fmt.Printf("chain: %s cannot be claimed to be asked again: %v\n", r.ID, err)
}
continue
}
id, err := askABuild(ctx, buildSource{Repository: r.Repository, Seat: r.Seat}, r.Path, r.Ref)
if err != nil {
if serr := inv.AskedAgainAs(ctx, r.ID, "not asked again: "+err.Error()); serr != nil {
fmt.Printf("chain: %s was not asked again, and that could not be kept: %v\n", r.ID, serr)
}
fmt.Printf("chain: %s was never handed over and cannot be asked again: %v\n", r.ID, err)
continue
}
recordBuildRequest(ctx, inv, inventory.BuildRequest{ID: id, Repository: r.Repository, Seat: r.Seat, Path: r.Path,
Ref: r.Ref, Commit: r.Commit, For: r.For})
if err := inv.AskedAgainAs(ctx, r.ID, id); err != nil {
fmt.Printf("chain: %s was asked again as %s, and that could not be kept: %v\n", r.ID, id, err)
}
fmt.Printf("chain: %s, never handed over, is asked again as %s\n", r.ID, id)
}
}
// askAgainSoon waits for the controller to be serving, which an ask needs, and then asks again.
func askAgainSoon(ctx context.Context, inv *inventory.Inventory) {
select {
case <-ctx.Done():
case <-time.After(time.Minute):
askAgain(ctx, inv)
}
}
// kindSentBuildsNotCopied is the condition of the one-time copy of node.sent_builds onto the assignments' keys that has not
// succeeded: until it has, a module an assignment pulled in may be missing from what a machine is read to run.
const kindSentBuildsNotCopied = "sent-builds-not-copied"
+4 -17
View File
@@ -14,7 +14,6 @@ import (
"github.com/novox/mesh-controller/internal/chain"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/lease"
"github.com/novox/mesh-controller/internal/link"
)
// **Acceptance 10 of design 47: one read of an assignment names the commit, the build and the rollout that put it there,
@@ -80,7 +79,7 @@ func TestOneReadOfAnAssignmentNamesItsCommitBuildAndRolloutAndOneReadOfTheRollou
if err != nil {
t.Fatal(err)
}
for try := 0; try < 3 && (asking.Modules["notes"] == nil || asking.Modules["notes"].Build == ""); try++ {
for try := 0; try < 6 && (asking.Modules["notes"] == nil || asking.Modules["notes"].Build == "" || asking.Modules["notes"].State == "asked"); try++ {
advancePlans(ctx, open)
if asking, err = inv.PlanByID(ctx, rollout); err != nil {
t.Fatal(err)
@@ -90,13 +89,6 @@ func TestOneReadOfAnAssignmentNamesItsCommitBuildAndRolloutAndOneReadOfTheRollou
if buildID == "" || len(*asked) != 1 || !strings.HasSuffix((*asked)[0], "#"+buildID) {
t.Fatalf("the walk asked %v, and notes' build is %q", *asked, buildID)
}
digest := "sha256:" + strings.Repeat("a", 64)
result := outcome(buildID, "modules/notes", "c-notes", "notes", "")
result.ReuseKey = "rk-notes"
result.Made = []link.MadeArtifact{{Name: "notes", Kind: "image", Reference: "artifact-store://notes/notes@" + digest}}
if err := (builds{inv: inv, open: open}).Built(ctx, result); err != nil {
t.Fatal(err)
}
// The walk sends the tier to its first machine.
for try := 0; try < 3 && len(sent) == 0; try++ {
@@ -116,9 +108,9 @@ func TestOneReadOfAnAssignmentNamesItsCommitBuildAndRolloutAndOneReadOfTheRollou
if wireguard, pulled := assignmentOf(on, "mesh-wireguard"); pulled && (wireguard.Build != "" || wireguard.Rollout != "") {
t.Fatalf("a module this rollout did not send names a build or a rollout: %+v", wireguard)
}
if a.Module != "notes" || a.Commit != "c-notes" || a.Build != buildID || a.Rollout != rollout || a.Artifact != digest ||
if a.Module != "notes" || a.Commit != "c-notes" || a.Build != buildID || a.Rollout != rollout ||
a.Since.IsZero() || a.Assigned.IsZero() || !a.Stands() {
t.Fatalf("the assignment names %+v, not commit c-notes, build %s, artifact %s, rollout %s", a, buildID, digest, rollout)
t.Fatalf("the assignment names %+v, not commit c-notes, build %s, rollout %s", a, buildID, rollout)
}
// One read of the rollout.
p, err := inv.PlanByID(ctx, rollout)
@@ -130,16 +122,11 @@ func TestOneReadOfAnAssignmentNamesItsCommitBuildAndRolloutAndOneReadOfTheRollou
p.Steps[0].Node != "anchor" || p.Steps[0].Sent == nil {
t.Fatalf("the rollout lists changed %+v, builds %v, steps %+v; want %+v", p.Changed, p.Builds, p.Steps, wantChange)
}
// And one read of the build names the same commit, with what it published and its reuse key.
// And one read of the build names the same commit.
built, found, err := inv.BuildByID(ctx, buildID)
if err != nil || !found || built.Commit != "c-notes" || built.Module != "notes" {
t.Fatalf("the build: %+v %v %v", built, found, err)
}
lately, err := inv.Builds(ctx, "notes", 1)
if err != nil || len(lately) != 1 || lately[0].ReuseKey != "rk-notes" || lately[0].Published == nil ||
lately[0].Published.Artifact != digest {
t.Fatalf("what the build published: %+v %v", lately, err)
}
// The verbs say the links.
if lines := changedLines(p); len(lines) != 2 || !strings.Contains(lines[1], "anchor/notes") ||
!strings.Contains(lines[1], buildID) {
@@ -6,8 +6,6 @@ import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/link"
)
// **Acceptance 12 of design 47: no table in the controller's store holds builds, rollouts or assignments** (novox/hq ADR
@@ -34,7 +32,7 @@ func TestNoTableOfTheControllersStoreHoldsBuildsRolloutsOrAssignments(t *testing
if err != nil {
t.Fatal(err)
}
for try := 0; try < 3 && (asking.Modules["notes"] == nil || asking.Modules["notes"].Build == ""); try++ {
for try := 0; try < 6 && (asking.Modules["notes"] == nil || asking.Modules["notes"].Build == "" || asking.Modules["notes"].State == "asked"); try++ {
advancePlans(ctx, open)
if asking, err = inv.PlanByID(ctx, rollout); err != nil {
t.Fatal(err)
@@ -44,12 +42,6 @@ func TestNoTableOfTheControllersStoreHoldsBuildsRolloutsOrAssignments(t *testing
if buildID == "" || len(*asked) != 1 {
t.Fatalf("the walk asked %v, and notes' build is %q", *asked, buildID)
}
digest := "sha256:" + strings.Repeat("b", 64)
result := outcome(buildID, "modules/notes", "c-notes", "notes", "")
result.Made = []link.MadeArtifact{{Name: "notes", Kind: "image", Reference: "artifact-store://notes/notes@" + digest}}
if err := (builds{inv: inv, open: open}).Built(ctx, result); err != nil {
t.Fatal(err)
}
// What the live path made is in the chain.
if built, found, err := inv.BuildByID(ctx, buildID); err != nil || !found || built.Commit != "c-notes" {
-29
View File
@@ -190,32 +190,3 @@ func summaryOf(p link.ChangePlan) string {
}
return strings.Join(parts, "; ")
}
// planText is a change plan as a person reads it, for the pull request's comment.
func planText(p link.ChangePlan) string {
var b strings.Builder
fmt.Fprintf(&b, "change plan of %s at %.8s into %s: %s\n", p.Repository, p.Head, p.Base, p.Summary)
for i, tier := range p.Tiers {
fmt.Fprintf(&b, " tier %d: %s\n", i, strings.Join(tier, ", "))
}
for _, m := range p.Machines {
line := " " + m.Machine + ": "
if len(m.Receives) > 0 {
line += "receives " + strings.Join(m.Receives, ", ")
}
if len(m.Waits) > 0 {
if len(m.Receives) > 0 {
line += "; "
}
line += "waits for a person: " + strings.Join(m.Waits, ", ")
}
b.WriteString(line + "\n")
}
for _, s := range p.Steps {
b.WriteString(" - " + s + "\n")
}
if len(p.Unread) > 0 {
fmt.Fprintf(&b, " read by no module's build: %s\n", strings.Join(p.Unread, ", "))
}
return b.String()
}
-3
View File
@@ -72,7 +72,4 @@ func TestAChangePlanSaysWhatEachMachineReceives(t *testing.T) {
if !strings.HasPrefix(p.Summary, "builds nothing") || len(p.Machines) != 0 || strings.Join(p.Unread, ",") != "merge-check.sh" {
t.Errorf("a root file's plan reads %+v", p)
}
if !strings.Contains(planText(p), "read by no module's build: merge-check.sh") {
t.Errorf("the plan's text does not say why nothing is built:\n%s", planText(p))
}
}
-87
View File
@@ -1,87 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"os"
"strconv"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/inventory"
)
// A pull request's check builds go when it closes (novox/hq ADR 0350 §5): the build seat keeps them in the artifact
// store's scratch namespace for the walk of its merge to take. **Let go of where the moment is heard**: the
// controller ends the walk answering a merge — after that walk took what it reuses — and the delivery's owner, which
// hears the forge say a pull request closed unmerged, asks the `forget-checks` verb; both let go through the store,
// as the controller's own collection does. The build seat's holders let go of anything a week old on their own,
// so a word lost here leaves a week's builds, never more.
// checkBuildsIn is the scratch namespace as the controller reaches the artifact store.
func checkBuildsIn(ctx context.Context, inv *inventory.Inventory) (builder.Scratch, error) {
shelf, err := inv.Catalogue(ctx)
if err != nil {
return nil, err
}
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return nil, err
}
if address == "" {
return nil, errors.New("the artifact store is not on the mesh's network")
}
return builder.Registry{Address: address}, nil
}
// forgetChecks lets go of a pull request's check builds and answers what it did.
var forgetChecks = func(ctx context.Context, inv *inventory.Inventory, pull builder.PullRef) (string, error) {
scratch, err := checkBuildsIn(ctx, inv)
if err != nil {
return "", err
}
n, err := scratch.Forget(ctx, pull)
if err != nil {
return "", fmt.Errorf("%s: %d check(s)' builds let go of, and not all: %w", pull, n, err)
}
return fmt.Sprintf("%s: %d check(s)' builds let go of", pull, n), nil
}
// forgetChecksOfWalk lets go, on the save that ends a walk, of the check builds of every pull request whose merge
// it answers: the walk took what it reuses. Said, never fatal.
func forgetChecksOfWalk(ctx context.Context, inv *inventory.Inventory, p inventory.Plan) {
if p.WalkSaid != inventory.WalkEnded || p.Delivery == nil {
return
}
for _, pull := range reuseAskOf(&p).Pulls {
said, err := forgetChecks(ctx, inv, builder.PullRef{Owner: pull.Owner, Repo: pull.Repo, Number: pull.Number})
if err != nil {
fmt.Fprintf(os.Stderr, "%s: its check builds stay until the week's sweep: %v\n", p.ID, err)
continue
}
fmt.Printf("%s: %s\n", p.ID, said)
}
}
// forgetChecksCommand is `forget-checks <owner> <repo> <number>`: the verb the delivery's owner asks when the forge
// says a pull request closed unmerged (novox/hq ADR 0350 §5).
func forgetChecksCommand(ctx context.Context, args []string) error {
if len(args) != 3 {
return errors.New("forget-checks <owner> <repo> <number>")
}
number, err := strconv.Atoi(args[2])
if err != nil || number <= 0 {
return fmt.Errorf("%q is not a pull request's number", args[2])
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
said, err := forgetChecks(ctx, open.inventory, builder.PullRef{Owner: args[0], Repo: args[1], Number: number})
if err != nil {
return err
}
fmt.Println(said)
return nil
}
@@ -1,102 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"reflect"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func TestOnlyAWalksAskTakesAChecksBuildAndADryRunNever(t *testing.T) {
p := &inventory.Plan{Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
{Repository: "novox/shop", Number: 7}, {Repository: "http://forge.invalid/novox/web.git", Number: 3},
{Repository: "novox/shop", Number: 7}, {Repository: "novox/shop"}}}}
asked := reuseAskOf(p)
if want := []link.PullRef{{Owner: "novox", Repo: "shop", Number: 7}, {Owner: "novox", Repo: "web", Number: 3}}; !reflect.DeepEqual(asked.Pulls, want) {
t.Fatalf("the walk names %+v", asked.Pulls)
}
walk := withReuse(t.Context(), asked)
if reuseAsked(walk, false) != asked {
t.Fatal("a walk's ask does not carry its leave")
}
if reuseAsked(walk, true) != nil {
t.Fatal("a dry run carries a leave to take a check's build")
}
if reuseAsked(t.Context(), false) != nil {
t.Fatal("an ask that is not a walk's carries a leave")
}
}
func TestAReusedCheckBuildIsRegisteredByItsDigestAndGatedAsABuiltOne(t *testing.T) {
withTheGate(t) // the first-machine gate's own path (novox/hq ADR 0360 keeps it for earlier walks)
g := aGateMesh(t)
ctx := t.Context()
inv := g.open.inventory
gateEvery = 0
digest := "sha256:" + strings.Repeat("c", 64)
reused := func(id, reference string) link.BuildResult {
manifest := mustJSON(t, catalogue.Manifest{Module: "app", Version: "c3", Resources: []map[string]any{
{"id": "svc", "type": "container", "name": "app", "image": reference}}})
return link.BuildResult{ID: id, Repository: "novox/mesh-catalog", Path: "modules/app", Ref: "main", On: "anchor",
Commit: "c3", Manifest: manifest, Trunk: "main", OnTrunk: true, Branches: []string{"main"},
Made: []link.MadeArtifact{{Name: "server", Kind: "image", Reference: reference}},
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"},
Reused: "reused from the check of novox/mesh-catalog#9 at head abcdefabcdef (merged into trunk 123456123456): server " + digest,
ReusedFrom: &link.ReusedFrom{Pull: link.PullRef{Owner: "novox", Repo: "mesh-catalog", Number: 9},
Head: "abcdefabcdef", Trunk: "123456123456", Key: "r1:" + strings.Repeat("1", 64)}}
}
if _, _, err := takeIn(ctx, inv, reused(link.NewBuildID(time.Now()), "store.internal:5000/scratch/app/server@"+digest)); err == nil ||
!strings.Contains(err.Error(), "scratch") {
t.Fatalf("a build in the scratch namespace was registered: %v", err)
}
id := link.NewBuildID(time.Now().Add(time.Second))
result := reused(id, "store.internal:5000/app/server@"+digest)
if _, _, err := takeIn(ctx, inv, result); err != nil {
t.Fatal(err)
}
current, err := inv.CurrentBuilds(ctx)
if err != nil || current["app"].Commit != "c3" {
t.Fatalf("the reused build is not what app is registered at: %+v %v", current["app"], err)
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(shelf["app"].Resources)
if !strings.Contains(string(raw), "app/server@"+digest) {
t.Fatalf("app is registered as %s, not by the check's digest in its own namespace", raw)
}
// The walk asked it: the walk keeps what its build did, and its gate judges the registered build.
asked := time.Now().UTC()
p := g.plan(t)
p.Commit = "c3"
p.Modules["app"] = &inventory.PlanModule{State: "asked", AskedAt: &asked, Build: id}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
planBuiltSaying(ctx, g.open, "app", "c3", "", asked, id, result.MadeWords())
p = g.plan(t)
if m := p.Modules["app"]; m.State != "built" || !strings.HasPrefix(m.Made, "reused from the check of novox/mesh-catalog#9") {
t.Fatalf("the walk keeps %+v", m)
}
for i := 0; i < 6; i++ {
advancePlans(ctx, g.open)
}
p = g.plan(t)
gate := p.Modules["app"].Gate
if gate == nil || gate.To != "c3" || gate.Verdict != inventory.GatePassed {
t.Fatalf("the gate judged %+v, not the reused build", gate)
}
if sent, _, _ := inv.SentBuilds(ctx, "anchor"); sent["app"] != "c3" {
t.Fatalf("the first machine was sent %v, not the reused build", sent)
}
}
-193
View File
@@ -1,193 +0,0 @@
package main
import (
"context"
"errors"
"reflect"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// One build per change (novox/hq ADR 0350), as the controller asks it and takes it in.
// The check of a change is asked to build what a merge of it would move, in the walk's order: a module of the
// repository checked from the merge, one standing on it from its own repository as the walk builds it, and one
// that reads the repository through a context not at all. A change that moves no module builds nothing.
func TestACheckIsAskedToBuildWhatTheMergeWouldMove(t *testing.T) {
entries := []inventory.Entry{
{Manifest: catalogue.Manifest{Module: "app"}, Source: inventory.Source{Repository: "novox/shop", Seat: "git", Path: "modules/app"}},
{Manifest: catalogue.Manifest{Module: "web"}, Source: inventory.Source{Repository: "novox/web", Seat: "git", Ref: "main"}},
{Manifest: catalogue.Manifest{Module: "docs"}, Source: inventory.Source{Repository: "novox/docs", Seat: "git"}},
}
p := link.PullUpdated{Owner: "novox", Repo: "shop", Number: 7, Base: "main", Commit: "abc"}
scope := checkScope{Reach: mergeReach{Packaging: entries[2:], Plan: inventory.Plan{Tiers: [][]string{{"app", "docs"}, {"web"}}}}}
clone := func(s inventory.Source) (string, error) { return "http://forge.invalid/" + s.Repository + ".git", nil }
got := checkBuildsOf(p, scope, entries, clone)
want := []link.CheckBuild{
{Module: "app", Path: "modules/app", Merged: true, Tier: 0},
{Module: "docs", Tier: 0, Skip: "its build reads novox/shop through a context, which a check cannot hand the merge"},
{Module: "web", Repository: "http://forge.invalid/novox/web.git", Ref: "main", Tier: 1},
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("the check is asked to build\n%+v\nwant\n%+v", got, want)
}
if got := checkBuildsOf(p, checkScope{}, entries, clone); got != nil {
t.Fatalf("a change that moves no module asks for builds: %+v", got)
}
}
// A walk's ask carries its leave to take a check's build, naming the pull requests of the merges it answers; a dry
// run never does, and neither does an ask that is not a walk's.
// A walk ended lets go of its merges' check builds — only once it ended, after it took what it reuses.
func TestAWalkEndedLetsGoOfItsMergesCheckBuilds(t *testing.T) {
var forgot []builder.PullRef
was := forgetChecks
forgetChecks = func(_ context.Context, _ *inventory.Inventory, pull builder.PullRef) (string, error) {
forgot = append(forgot, pull)
return "", nil
}
t.Cleanup(func() { forgetChecks = was })
p := inventory.Plan{ID: "plan-1", Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
{Repository: "novox/shop", Number: 7}}}}
p.WalkSaid = inventory.WalkStarted
forgetChecksOfWalk(t.Context(), nil, p)
p.WalkSaid = ""
forgetChecksOfWalk(t.Context(), nil, p)
if len(forgot) != 0 {
t.Fatalf("a walk not ended let go of %v, which it may still take", forgot)
}
p.WalkSaid = inventory.WalkEnded
forgetChecksOfWalk(t.Context(), nil, p)
if want := []builder.PullRef{{Owner: "novox", Repo: "shop", Number: 7}}; !reflect.DeepEqual(forgot, want) {
t.Fatalf("the walk ended let go of %v", forgot)
}
}
// A build in the scratch namespace is never registered: only the walk's copy in the module's own namespace is.
func TestABuildInTheScratchNamespaceIsNeverRegistered(t *testing.T) {
if got := inScratch([]link.MadeArtifact{{Name: "server", Reference: "store.internal:5000/app/server@sha256:aa"}}); got != "" {
t.Fatalf("a module's own build reads as a scratch build: %s", got)
}
scratch := "store.internal:5000/scratch/app/server@sha256:aa"
if got := inScratch([]link.MadeArtifact{{Name: "server", Reference: scratch}}); got != scratch {
t.Fatalf("a scratch build was not found: %q", got)
}
if got := inScratch([]link.MadeArtifact{{Name: "look", Reference: "http://store.internal:5000/v2/scratch/app/look/blobs/sha256:bb"}}); got == "" {
t.Fatal("a scratch archive was not found")
}
}
// The walk says per module what it did: "reused from …" or "built: <why>" — and a build seat from before says
// neither, which is said.
func TestAWalksBuildSaysWhetherItReusedOrBuilt(t *testing.T) {
if got := (link.BuildResult{Reused: "reused from the check of novox/shop#7"}).MadeWords(); got != "reused from the check of novox/shop#7" {
t.Fatalf("said %q", got)
}
if got := (link.BuildResult{BuiltWhy: "no check built it"}).MadeWords(); got != "built: no check built it" {
t.Fatalf("said %q", got)
}
if got := (link.BuildResult{}).MadeWords(); !strings.Contains(got, "predates ADR 0350") {
t.Fatalf("said %q", got)
}
}
// A walk's module the check's build stood in for is registered by the digest the check built — the copy in the
// module's namespace — and the walk keeps "reused from …" for it; its first-node gate judges that registered
// build as it judges one the walk built. A scratch reference is refused. Needs the store (MESH_TEST_POSTGRES), and
// is named in testdata/store-tests.
// A walk's leave names the builds its own pull requests' verdicts made at the head each merge merged — never an
// earlier head's — read once per walk, and none when they cannot be read: the builder takes nothing a verdict does
// not name (novox/hq ADR 0350 §2, issue 703).
func TestAWalksLeaveNamesTheBuildsItsPullRequestsVerdictsMade(t *testing.T) {
was := checkedBuildsOf
forgetWalkVerdicts()
t.Cleanup(func() { checkedBuildsOf = was; forgetWalkVerdicts() })
var asked [][]link.PullRef
pull := link.PullRef{Owner: "novox", Repo: "shop", Number: 7}
atHead := link.CheckedBuild{Pull: pull, Check: "build-2", Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:2"}}
earlier := link.CheckedBuild{Pull: pull, Check: "build-1", Head: "aaaaaaaaaaaa", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
checkedBuildsOf = func(_ context.Context, pulls []link.PullRef) ([]link.CheckedBuild, error) {
asked = append(asked, pulls)
return []link.CheckedBuild{atHead, earlier}, nil
}
pulls := []link.PullRef{pull}
heads := map[link.PullRef]string{pull: "bbbbbbbbbbbb0123456789"}
for range 3 {
if got := reuseWithVerdicts(t.Context(), "plan-1", &link.ReuseAsk{Pulls: pulls}, heads); !reflect.DeepEqual(got.Builds,
[]link.CheckedBuild{atHead}) {
t.Fatalf("the leave names %+v, not only the merged head's build", got.Builds)
}
}
if !reflect.DeepEqual(asked, [][]link.PullRef{pulls}) {
t.Fatalf("the verdicts were read %d times for one walk: %v", len(asked), asked)
}
// A merge whose head was not said takes nothing.
if got := reuseWithVerdicts(t.Context(), "plan-1", &link.ReuseAsk{Pulls: pulls}, nil); got.Builds != nil {
t.Fatalf("a merge of no said head names %+v", got.Builds)
}
checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) {
return nil, errors.New("no bus")
}
if got := reuseWithVerdicts(t.Context(), "plan-2", &link.ReuseAsk{Pulls: pulls}, heads); got.Builds != nil {
t.Fatalf("verdicts that could not be read named %+v", got.Builds)
}
}
// A check is handed only the builds earlier checks of its own pull request at its own head made: another pull
// request's build, or an earlier head's, is never kept (novox/hq issue 703). None when they cannot be read.
func TestACheckIsHandedOnlyItsOwnHeadsBuilds(t *testing.T) {
was := checkedBuildsOf
t.Cleanup(func() { checkedBuildsOf = was })
pull := link.PullRef{Owner: "novox", Repo: "shop", Number: 7}
mine := link.CheckedBuild{Pull: pull, Check: "build-1", Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
earlier := link.CheckedBuild{Pull: pull, Check: "build-0", Head: "aaaaaaaaaaaa", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
another := link.CheckedBuild{Pull: link.PullRef{Owner: "novox", Repo: "shop", Number: 8}, Check: "build-9",
Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
var asked [][]link.PullRef
checkedBuildsOf = func(_ context.Context, pulls []link.PullRef) ([]link.CheckedBuild, error) {
asked = append(asked, pulls)
return []link.CheckedBuild{mine, earlier, another}, nil
}
p := link.PullUpdated{Owner: "novox", Repo: "shop", Number: 7, Commit: "bbbbbbbbbbbb0123456789"}
builds := []link.CheckBuild{{Module: "app"}}
if got := namedForTheCheck(t.Context(), p, builds); !reflect.DeepEqual(got, []link.CheckedBuild{mine}) ||
!reflect.DeepEqual(asked, [][]link.PullRef{{pull}}) {
t.Fatalf("the check is handed %+v, read for %v", got, asked)
}
checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) {
return nil, errors.New("no bus")
}
if got := namedForTheCheck(t.Context(), p, builds); got != nil {
t.Fatalf("verdicts that could not be read named %+v", got)
}
}
func forgetWalkVerdicts() {
walkVerdicts.Lock()
defer walkVerdicts.Unlock()
walkVerdicts.read = map[string]verdictsRead{}
}
// A walk's merged heads are the forge's word for each merge; a merge announced without its head, or a pull request
// merged at two heads in one walk, has none, and its walk takes no check's build (novox/hq issue 703).
func TestAWalksMergedHeadsAreTheForgesWord(t *testing.T) {
merge := func(number int, head string) inventory.PlanMerge {
return inventory.PlanMerge{Repository: "novox/shop", Number: number, Head: head}
}
p := &inventory.Plan{Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
merge(7, "aaaaaaaaaaaa1"), merge(8, ""), merge(9, "bbbbbbbbbbbb"), merge(9, "cccccccccccc")}}}
heads := mergedHeadsOf(p)
shop := func(n int) link.PullRef { return link.PullRef{Owner: "novox", Repo: "shop", Number: n} }
if heads[shop(7)] != "aaaaaaaaaaaa1" || heads[shop(8)] != "" || heads[shop(9)] != "" {
t.Fatalf("merged heads %v", heads)
}
}
-203
View File
@@ -1,203 +0,0 @@
package main
import (
"context"
"errors"
"flag"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// `check-here` is a pull request's merge check run on the machine at hand **exactly as the build seat
// runs it** (novox/hq issue 286): the same code (builder.Check), the same ask the controller would make of
// the seat — the gate's modules and judge by the planner's own answer over the facts snapshot, the
// repositories beside it at the refs the snapshot says the seat clones them at — in the toolchain image
// the mesh holds, as the user the build seat runs as, against a throwaway store and bus of the versions
// the mesh runs.
//
// **The build seat is the reference.** A merge-check.sh that passed on an agent's machine failed on the
// seat for three reasons that were each the agent's environment, never the change: a newer Go whose gofmt
// lays a file out differently, a sibling checkout at the agent's feature branch where the seat had the
// commit the mesh runs, and a different user. Run here, a check sees what the seat will.
//
// check-here [--tree <checkout>] [--base main] [--registry <artifact store>] [--forge <url of the owner>]
// [--number <n>] [--user uid:gid] [--facts store|<file>] [--keep]
//
// The checkout's HEAD is what is checked, and it must be committed: the seat checks a commit, never a
// working tree.
func checkHereCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("check-here", flag.ContinueOnError)
tree := set.String("tree", ".", "the change's checkout; its HEAD is checked")
base := set.String("base", "main", "the branch the change would merge into")
registry := set.String("registry", os.Getenv("MESH_REGISTRY"), "the artifact store holding the facts and the toolchains")
forge := set.String("forge", "", "where the repositories beside it are cloned from, as <forge>/<repository>.git; "+
"the checkout's origin without its own name when not given")
number := set.Int("number", 0, "the pull request's number, when there is one")
// The build seat's service runs as root, and its check containers run as the builder does.
user := set.String("user", "0:0", "the user the check's containers run as: the build seat's")
keep := set.Bool("keep", false, "keep the workspace afterwards")
factsFrom := set.String("facts", "store", "the facts snapshot: `store`, the one the artifact store holds — "+
"what the seat reads — or a file")
if _, err := parseAround(set, args); err != nil {
return err
}
if *registry == "" {
return errors.New("check-here reads the facts and the toolchains from the artifact store: --registry <host:port> or MESH_REGISTRY")
}
dir, err := filepath.Abs(*tree)
if err != nil {
return err
}
git := func(args ...string) (string, error) {
cmd := exec.CommandContext(ctx, "git", args...)
cmd.Dir = dir
out, err := cmd.Output()
return strings.TrimSpace(string(out)), err
}
if dirty, err := git("status", "--porcelain", "--untracked-files=no"); err != nil {
return fmt.Errorf("%s is not a checkout: %w", dir, err)
} else if dirty != "" {
return errors.New("the checkout has changes not committed: the build seat checks a commit, so commit first")
}
head, err := git("rev-parse", "HEAD")
if err != nil {
return err
}
origin, err := git("remote", "get-url", "origin")
if err != nil {
return fmt.Errorf("the checkout has no origin to say which repository it is: %w", err)
}
owner, repo, prefix := ownerRepoOf(origin)
if *forge == "" {
*forge = prefix
}
if _, err := git("fetch", "--quiet", "origin", *base); err != nil {
return fmt.Errorf("cannot fetch %s to say what the change touches: %w", *base, err)
}
// Without rename detection, so a file moved out of a build source is said under its old name as well:
// its going is a change to the build that held it (novox/hq ADR 0267).
changedText, err := git("diff", "--name-only", "--no-renames", "origin/"+*base+"...HEAD")
if err != nil {
return err
}
var paths, removed []string
for _, p := range strings.Split(changedText, "\n") {
if p = strings.TrimSpace(p); p == "" {
continue
}
paths = append(paths, p)
if _, err := os.Stat(filepath.Join(dir, p)); os.IsNotExist(err) {
removed = append(removed, p)
}
}
_ = os.Setenv("MESH_REGISTRY", *registry)
f, err := readFacts(ctx, *factsFrom)
if err != nil {
return fmt.Errorf("the facts snapshot cannot be read: %w", err)
}
entries, read, edges, err := graphOfFacts(f)
if err != nil {
return err
}
p := link.PullUpdated{Owner: owner, Repo: repo, Number: *number, Base: *base, Commit: head, Paths: paths,
Removed: removed, ModuleDirs: moduleDirsIn(dir, paths), ModuleDirsSaid: true}
scope := pullScope(p, entries, read, edges)
_, scriptErr := os.Stat(filepath.Join(dir, builder.CheckScript))
if !scope.gated() && !scope.Mesh {
fmt.Printf("%s: %s, and the repository is not the mesh's: the build seat runs nothing for it\n",
owner+"/"+repo, noModuleTouched)
return nil
}
if !scope.gated() && scriptErr != nil {
fmt.Printf("%s: %s; %s\n", owner+"/"+repo, noModuleTouched, noMergeCheck)
return nil
}
toolchains := map[string]string{}
for language, reference := range f.Versions.Toolchains {
toolchains[language] = catalogue.Rerouted(reference, *registry)
}
if len(toolchains) == 0 {
return errors.New("the facts snapshot names no toolchain: it was taken by a controller from before " +
"issue 286, and the seat's toolchain cannot be known here")
}
beside := map[string]builder.Beside{}
for d, ref := range f.Beside {
from := d
if d == "mesh-controller-main" {
from = "mesh-controller"
}
beside[d] = builder.Beside{Repository: strings.TrimSuffix(*forge, "/") + "/" + from + ".git", Ref: ref}
}
id := fmt.Sprintf("check-here-%d", time.Now().UnixNano())
spec := builder.CheckSpec{ID: id, Repository: dir, Ref: head, Owner: owner, Repo: repo, Number: *number,
Paths: paths, Beside: beside, Modules: scope.Modules, New: scope.New, Manifests: scope.Manifests,
Base: *base, Judge: scope.Judge, Toolchain: toolchains["go"], Toolchains: toolchains, User: *user}
workspace, err := os.MkdirTemp("", "mesh-check-here-")
if err != nil {
return err
}
if !*keep {
defer removeWorkspace(spec.Toolchain, workspace, *user)
}
fmt.Fprintf(os.Stderr, "checking %s/%s at %.8s as the build seat would, against the facts of %s, in %s\n",
owner, repo, head, f.Taken.Format(time.RFC3339), workspace)
v, err := builder.Check(ctx, builder.Command, spec, workspace, *registry, builder.GitCredential{},
func(step, message string) {
if step != "output" {
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
}
})
if err != nil {
return fmt.Errorf("the check could not run — on the seat an error, never a pass: %w", err)
}
fmt.Println(v.Report)
fmt.Println()
fmt.Printf("mesh/merge-gate: %s — %s\n", strings.ToUpper(v.Gate.Verdict), v.Gate.Summary)
if v.Repo != nil {
fmt.Printf("mesh/repo-check: %s — %s\n", strings.ToUpper(v.Repo.Verdict), v.Repo.Summary)
}
for _, l := range []*builder.Layer{v.Gate, v.Repo} {
if l != nil && l.Verdict != "pass" && l.Verdict != "warning" {
return errors.New("the build seat would not pass this change")
}
}
return nil
}
// ownerRepoOf reads owner, repository and the owner's URL from a remote: ssh://git@host:222/novox/mesh-host.git
// → novox, mesh-host, ssh://git@host:222/novox.
func ownerRepoOf(remote string) (string, string, string) {
trimmed := strings.TrimSuffix(strings.TrimSuffix(remote, "/"), ".git")
cut := strings.LastIndexAny(trimmed, "/:")
if cut < 0 {
return "", trimmed, ""
}
repo, prefix := trimmed[cut+1:], trimmed[:cut]
owner := prefix
if at := strings.LastIndexAny(prefix, "/:"); at >= 0 {
owner = prefix[at+1:]
}
return owner, repo, prefix
}
// removeWorkspace removes what the check left, written as the seat's user: by a container of that user
// when it is not this one.
func removeWorkspace(image, workspace, user string) {
if image != "" && user != fmt.Sprintf("%d:%d", os.Getuid(), os.Getgid()) {
// Everything in it — the check's HOME is the workspace, so the toolchain's own files are there too.
_ = exec.Command("docker", "run", "--rm", "--user", user, "--volume", workspace+":/workspace", image,
"sh", "-c", "rm -rf /workspace/* /workspace/.[!.]*").Run()
}
_ = os.RemoveAll(workspace)
}
-44
View File
@@ -1,44 +0,0 @@
package main
import (
"reflect"
"testing"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// 1212/AC1 and AC2: the check request names the pulls bucket of each module claiming the delivery seat, in the order
// of the claimants' names.
func TestACheckNamesThePullsBucketsOfTheSeatsClaimants(t *testing.T) {
for name, c := range map[string]struct {
entries []inventory.Entry
want []string
}{
"only the gate": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate"), namedOnly("mesh-delivery")}, []string{"mesh-delivery-gate_pulls"}},
"only today's": {[]inventory.Entry{claimingTheSeat("mesh-delivery"), namedOnly("gitea")}, []string{"mesh-delivery_pulls"}},
"both, name order": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate"), claimingTheSeat("mesh-delivery")}, []string{"mesh-delivery_pulls", "mesh-delivery-gate_pulls"}},
"none": {[]inventory.Entry{namedOnly("gitea")}, nil},
} {
if got := pullsBucketsOf(c.entries); !reflect.DeepEqual(got, c.want) {
t.Errorf("%s: %v, want %v", name, got, c.want)
}
}
}
// 1212/AC1 and AC2: the check a request carries names those buckets (the wiring, not only the helper).
func TestTheCheckRequestCarriesThePullsBuckets(t *testing.T) {
for name, c := range map[string]struct {
entries []inventory.Entry
want []string
}{
"only the gate": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate")}, []string{"mesh-delivery-gate_pulls"}},
"only today's": {[]inventory.Entry{claimingTheSeat("mesh-delivery")}, []string{"mesh-delivery_pulls"}},
"both, name order": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate"), claimingTheSeat("mesh-delivery")}, []string{"mesh-delivery_pulls", "mesh-delivery-gate_pulls"}},
} {
got := checkOf(link.PullUpdated{Owner: "novox", Repo: "a", Number: 7}, checkScope{}, c.entries, nil, nil, nil, nil)
if !reflect.DeepEqual(got.Pulls, c.want) || got.Owner != "novox" || got.Number != 7 {
t.Errorf("%s: %+v, want pulls %v", name, got, c.want)
}
}
}
-165
View File
@@ -1,165 +0,0 @@
package main
import (
"context"
"fmt"
"regexp"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A delivery group's member is checked beside the other members' heads** (novox/hq ADR 0382, issue 867).
//
// A mesh-host change and a mesh-controller change that need each other could not both pass their own merge checks:
// each was read beside the other's repository at the commit the mesh runs, which holds neither change. So the
// controller asks mesh-delivery's `groups` once for every pull request's check it asks (§1), and names, in the ask,
// the group's other open pull requests into a base of the same name, in a repository the build seat reads beside the
// check — mesh-lab never — each with its whole head. The build seat reads those repositories at the sibling's merge (§2).
// A pull request for which mesh-delivery answered and named no sibling is asked exactly as before (§7).
// wholeCommit is a commit as the forge says a head: forty hexadecimal characters.
var wholeCommit = regexp.MustCompile(`^[0-9a-f]{40}$`)
// besideRepositories are the repositories (lower case, owner/repo) a check reads beside its own, by the directory each
// is found under — the core's, as besideOf clones them, and mesh-sdk beside the controller; never mesh-lab, which is
// always read at its trunk.
func besideRepositories(entries []inventory.Entry) map[string]string {
read := map[string]string{}
for _, e := range entries {
dir, core := coreModules[e.Manifest.Module]
if !core || e.Provided || e.Source.Repository == "" {
continue
}
read[nameOf(e.Source.Repository)] = dir
if dir == "mesh-controller" && e.Source.Seat != "" {
read[nameOf(siblingOf(e.Source.Repository, "mesh-sdk"))] = "mesh-sdk"
}
}
delete(read, "")
return read
}
// nameOf is a recorded repository as owner/repo in lower case, empty when it names no owner.
func nameOf(repository string) string {
owner, repo := sourceOwner(repository), sourceName(repository)
if owner == "" || repo == "" {
return ""
}
return strings.ToLower(owner + "/" + repo)
}
// siblingsFor is the pull request's siblings, as mesh-delivery's groups say them (ADR 0382 §1): the other members of
// the group its head branch names that are open, unmerged and into a base of its own base's name, in a repository the
// check reads beside it (beside: the directories the ask already names). The second answer is why they could not be
// learned, empty when they could: the group's members carry no head or base — a mesh-delivery from before the rule.
func siblingsFor(p link.PullUpdated, entries []inventory.Entry, beside map[string]link.CheckedOut,
lines []groupsLine) ([]link.CheckSibling, string) {
own := strings.ToLower(p.Owner + "/" + p.Repo)
var group *groupsLine
if p.Head == "" {
// A recheck (mesh-delivery's `recheck` through `delivery check`) carries no head branch name: the group is the one
// whose members hold this very pull request, by repository and number.
learnable := false
for i := range lines {
for _, m := range lines[i].Details {
learnable = true
if p.Number != 0 && m.Number == p.Number && strings.ToLower(m.Repository) == own {
group = &lines[i]
}
}
}
if group == nil {
for _, l := range lines {
if len(l.Details) == 0 && len(l.Members) > 0 && !learnable {
return nil, "mesh-delivery named the groups' members without their pull requests (a mesh-delivery from before novox/hq ADR 0382): " +
"the group of a pull request announced without its head branch cannot be found"
}
}
return nil, "" // a lone pull request
}
} else {
for i := range lines {
if lines[i].ID == p.Head {
group = &lines[i]
}
}
if group == nil {
return nil, ""
}
}
if len(group.Details) == 0 && len(group.Members) > 0 {
return nil, "mesh-delivery named the group's members without their heads and bases (a mesh-delivery from before novox/hq ADR 0382)"
}
read := besideRepositories(entries)
var siblings []link.CheckSibling
for _, m := range group.Details {
repository := strings.ToLower(m.Repository)
if repository == own {
continue
}
if !m.Open {
continue
}
if m.Head == "" || m.Base == "" {
return nil, "mesh-delivery named " + m.ID + " without its head or base (a mesh-delivery from before novox/hq ADR 0382)"
}
dir, readBeside := read[repository]
if m.Base != p.Base || !readBeside || dir == "mesh-lab" {
continue
}
if _, there := beside[dir]; !there {
continue
}
head := strings.ToLower(m.Head)
if !wholeCommit.MatchString(head) {
return nil, fmt.Sprintf("mesh-delivery named %s with a head that is not a whole commit (%q)", m.ID, m.Head)
}
siblings = append(siblings, link.CheckSibling{Repository: m.Repository, Number: m.Number, Head: head, Base: m.Base, Dir: dir})
}
sort.Slice(siblings, func(i, j int) bool { return siblings[i].Dir < siblings[j].Dir })
return siblings, ""
}
// withSiblings asks mesh-delivery's `groups` once and says in the request the pull request's siblings, or why they
// could not be learned (ADR 0382 §1): a check never waits on mesh-delivery past groupAskWithin and never fails for it,
// so a fix to mesh-delivery is still checked without it.
func withSiblings(ctx context.Context, request *link.BuildRequest, p link.PullUpdated, entries []inventory.Entry) {
lines, err := groupsAsked(ctx, deliverySeatHeld(entries))
siblingsFrom(request, p, entries, lines, err)
}
// siblingsFrom is withSiblings from mesh-delivery's answer already had.
func siblingsFrom(request *link.BuildRequest, p link.PullUpdated, entries []inventory.Entry, lines []groupsLine, err error) {
if request.Check == nil {
return
}
if err != nil {
request.Check.SiblingsUnknown = fmt.Sprintf("mesh-delivery did not say the group's pull requests (%v): "+
"the check reads no siblings", err)
return
}
request.Check.Siblings, request.Check.SiblingsUnknown = siblingsFor(p, entries, request.Check.Beside, lines)
}
// composedMember says the pull request is a member of a delivery group whose gate is the composed check's (novox/hq
// ADR 0403 clauses 1 and 7): the group has two members or more and mesh-delivery says so on its line. The group is
// found as siblingsFor finds it, by its head branch name, or, for a recheck that carries none, by the pull request's
// repository and number — not by siblings, which name only what the check reads beside it.
func composedMember(p link.PullUpdated, lines []groupsLine) bool {
own := strings.ToLower(p.Owner + "/" + p.Repo)
for _, l := range lines {
found := p.Head != "" && l.ID == p.Head
for _, m := range l.Details {
if p.Head == "" && p.Number != 0 && m.Number == p.Number && strings.ToLower(m.Repository) == own {
found = true
}
}
if found {
return l.ComposedGate && max(len(l.Details), len(l.Members)) >= 2
}
}
return false
}
-248
View File
@@ -1,248 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A delivery group's member is checked beside the other members' heads** (novox/hq ADR 0382 §1, §7, issue 867): the
// controller asks mesh-delivery's `groups` once for every check it asks, and names in the ask the group's other open
// pull requests into a base of the same name, in a repository the build seat reads beside it — mesh-lab never.
const (
headHost = "1111111111111111111111111111111111111111"
headController = "2222222222222222222222222222222222222222"
headSDK = "3333333333333333333333333333333333333333"
headLab = "4444444444444444444444444444444444444444"
headHq = "5555555555555555555555555555555555555555"
)
func theCoreOnTheGitSeat() []inventory.Entry {
onSeat := func(module, repository string) inventory.Entry {
e := fromRepo(module, repository, "")
e.Source.Seat = "git"
return e
}
nats := onSeat("nats", "novox/mesh-catalog")
nats.Source.Path = "modules/nats"
held := catalogue.Manifest{Module: "mesh-delivery", Claims: []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}}
return []inventory.Entry{onSeat("mesh-controller", "novox/mesh-controller"), onSeat("mesh-host", "novox/mesh-host"),
onSeat("node-tools", "novox/mesh-tools"), nats, {Manifest: held, On: []string{"anchor"}}}
}
func besideTheCore() map[string]link.CheckedOut {
return map[string]link.CheckedOut{"mesh-controller": {Repository: "c"}, "mesh-host": {Repository: "h"},
"mesh-tools": {Repository: "t"}, "mesh-catalog": {Repository: "k"}, "mesh-sdk": {Repository: "s"}, "mesh-lab": {Repository: "l"}}
}
func member(repository string, number int, head, base string, open bool) memberLine {
return memberLine{ID: repository + "@" + head[:12], Repository: repository, State: "checked", Number: number, Head: head,
Base: base, Open: open}
}
func group(id string, members ...memberLine) groupsLine {
line := groupsLine{ID: id, Details: members}
for _, m := range members {
line.Members = append(line.Members, m.ID+" ("+m.State+")")
}
return line
}
func TestASiblingIsTheGroupsOtherOpenPullRequestIntoABaseOfTheSameName(t *testing.T) {
entries := theCoreOnTheGitSeat()
pull := link.PullUpdated{Owner: "novox", Repo: "mesh-host", Number: 114, Base: "main", Head: "feat/x", Commit: headHost}
for _, c := range []struct {
what string
lines []groupsLine
want []string
unknown string
}{
{"the other member, open, same base", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))},
[]string{"mesh-controller=novox/mesh-controller#346@" + headController + " onto main"}, ""},
{"a member into a base of another name is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "release", true))},
nil, ""},
{"a member not open (merged or closed) is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", false))},
nil, ""},
{"mesh-lab is never one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-lab", 5, headLab, "main", true))},
nil, ""},
{"a repository the check reads nothing of is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/hq", 9, headHq, "main", true))},
nil, ""},
{"mesh-sdk, read beside the controller, is one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-sdk", 7, headSDK, "main", true))},
[]string{"mesh-sdk=novox/mesh-sdk#7@" + headSDK + " onto main"}, ""},
{"another group is none of its business", []groupsLine{group("feat/y",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))},
nil, ""},
{"a group of its own alone has none", []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true))},
nil, ""},
{"an older mesh-delivery names no heads", []groupsLine{{ID: "feat/x",
Members: []string{"novox/mesh-host@111111111111 (checked)", "novox/mesh-controller@222222222222 (checked)"}}},
nil, "without their heads and bases"},
{"a member without its base", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "", true))},
nil, "without its head or base"},
{"a head that is no commit", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController[:12]+"zzzzzzzzzzzzzzzzzzzzzzzzzzzz", "main", true))},
nil, "not a whole commit"},
} {
got, unknown := siblingsFor(pull, entries, besideTheCore(), c.lines)
var said []string
for _, s := range got {
said = append(said, s.Dir+"="+s.Repository+"#"+itoa(s.Number)+"@"+s.Head+" onto "+s.Base)
}
if strings.Join(said, ";") != strings.Join(c.want, ";") {
t.Errorf("%s: siblings %v, wanted %v", c.what, said, c.want)
}
if (c.unknown == "") != (unknown == "") || !strings.Contains(unknown, c.unknown) {
t.Errorf("%s: could not learn %q, wanted %q", c.what, unknown, c.unknown)
}
}
// A pull request into a release branch has siblings into the release branch of the same name (ADR 0358 §1).
release := pull
release.Base = "feat/57"
lines := []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "feat/57", true),
member("novox/mesh-controller", 346, headController, "feat/57", true), member("novox/mesh-tools", 8, headLab, "main", true))}
if got, _ := siblingsFor(release, entries, besideTheCore(), lines); len(got) != 1 || got[0].Dir != "mesh-controller" {
t.Errorf("into a release branch: %+v", got)
}
// A recheck announces the pull request without its head branch: its group is the one holding it (repository, number).
pull.Head = ""
got, unknown := siblingsFor(pull, entries, besideTheCore(), lines)
if len(got) != 1 || got[0].Dir != "mesh-tools" || unknown != "" {
t.Errorf("a recheck of a group member: %+v %q", got, unknown)
}
// A pull request in no group, rechecked, is alone: no sibling, and nothing said.
pull.Number = 999
if got, unknown := siblingsFor(pull, entries, besideTheCore(), lines); got != nil || unknown != "" {
t.Errorf("a lone pull request: %+v %q", got, unknown)
}
if got, unknown := siblingsFor(pull, entries, besideTheCore(), nil); got != nil || unknown != "" {
t.Errorf("no groups at all: %+v %q", got, unknown)
}
// A mesh-delivery that names members without their pull requests cannot find the group of a recheck: it says so.
older := []groupsLine{{ID: "feat/x", Members: []string{"novox/mesh-host@111111111111 (checked)"}}}
pull.Number = 114
if got, unknown := siblingsFor(pull, entries, besideTheCore(), older); got != nil || unknown == "" {
t.Errorf("an older mesh-delivery: %+v %q", got, unknown)
}
}
func itoa(n int) string { b, _ := json.Marshal(n); return string(b) }
// Every check asks mesh-delivery's `groups` once; the ask names the sibling's repository, head and base.
func TestEveryCheckAsksMeshDeliveryOnceAndNamesItsSiblings(t *testing.T) {
entries := theCoreOnTheGitSeat()
asked := 0
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
groupsAsked = func(_ context.Context, held bool) ([]groupsLine, error) {
asked++
if !held {
t.Error("the seat is held, yet it was asked as not held")
}
return []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true),
member("novox/mesh-controller", 346, headController, "main", true))}, nil
}
request := link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main", Beside: besideTheCore()}}
withSiblings(t.Context(), &request, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
if asked != 1 {
t.Fatalf("mesh-delivery was asked %d times for one check", asked)
}
want := link.CheckSibling{Repository: "novox/mesh-controller", Number: 346, Head: headController, Base: "main", Dir: "mesh-controller"}
if got := request.Check.Siblings; len(got) != 1 || got[0] != want || request.Check.SiblingsUnknown != "" {
t.Fatalf("the ask names siblings %+v, unknown %q", got, request.Check.SiblingsUnknown)
}
body, _ := json.Marshal(request.Check)
if !strings.Contains(string(body), `"siblings":[{"repository":"novox/mesh-controller","number":346,"head":"`+headController+
`","base":"main","dir":"mesh-controller"}]`) {
t.Fatalf("on the wire: %s", body)
}
}
// When mesh-delivery does not answer in time (or is not held, or answered without the fields), the check reads no
// sibling and its request says it could not learn them, so a fix to mesh-delivery is still checked without it.
func TestWhenMeshDeliveryDoesNotAnswerTheCheckReadsNoSiblingAndSaysSo(t *testing.T) {
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
for _, c := range []struct {
what string
err error
held bool
}{{"it did not answer", context.DeadlineExceeded, true}, {"nothing serves it", errors.New("nothing serves mesh-delivery.groups"), true},
{"no holder on record", errors.New("no holder of mesh-delivery is on record to say it"), false}} {
entries := theCoreOnTheGitSeat()
if !c.held {
entries = entries[:4]
}
groupsAsked = func(_ context.Context, held bool) ([]groupsLine, error) {
if held != c.held {
t.Errorf("%s: held %v", c.what, held)
}
return nil, c.err
}
request := link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main", Beside: besideTheCore()}}
withSiblings(t.Context(), &request, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
if len(request.Check.Siblings) != 0 || !strings.Contains(request.Check.SiblingsUnknown, c.err.Error()) ||
!strings.Contains(request.Check.SiblingsUnknown, "reads no siblings") {
t.Errorf("%s: siblings %+v, unknown %q", c.what, request.Check.Siblings, request.Check.SiblingsUnknown)
}
}
}
// **A pull request for which mesh-delivery answered and named no sibling is asked exactly as before** (ADR 0382 §7):
// byte for byte the request it was without the rule.
func TestAPullRequestWithNoSiblingIsAskedByteForByteAsBefore(t *testing.T) {
entries := theCoreOnTheGitSeat()
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
for _, answer := range [][]groupsLine{nil, {group("feat/other", member("novox/mesh-controller", 346, headController, "main", true))},
{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true), member("novox/hq", 3, headHq, "main", true))}} {
groupsAsked = func(context.Context, bool) ([]groupsLine, error) { return answer, nil }
build := func() link.BuildRequest {
return link.BuildRequest{ID: "build-1", Repository: "r", Ref: headHost, Check: &link.CheckRequest{Owner: "novox",
Repo: "mesh-host", Number: 114, Base: "main", Beside: besideTheCore(), Modules: []string{"mesh-host"}, Judge: "validator"}}
}
before, _ := json.Marshal(build())
after := build()
withSiblings(t.Context(), &after, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
got, _ := json.Marshal(after)
if string(before) != string(got) {
t.Fatalf("the request changed:\nbefore %s\nafter %s", before, got)
}
}
}
// What the build seat's verdict says of its siblings reaches the forge's holder in the controller's `checked`.
func TestTheVerdictCarriesTheSiblingsAndWhereTheJudgeCameFrom(t *testing.T) {
read := []link.CheckSibling{{Repository: "novox/mesh-controller", Number: 346, Head: headController, Base: "main", BaseCommit: headHost}}
c := checkedOf(link.BuildResult{ID: "b", Ref: headHost, Checked: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114},
Check: &link.CheckOutcome{Verdict: "pass", Summary: "ok", Siblings: read, JudgeFrom: "sibling novox/mesh-controller#346 merge abc",
Validator: "the change's own validator"}})
if len(c.Siblings) != 1 || c.Siblings[0] != read[0] || c.JudgeFrom == "" || c.Validator == "" {
t.Fatalf("the verdict says %+v", c)
}
body, _ := json.Marshal(c)
for _, want := range []string{`"siblings":[{"repository":"novox/mesh-controller","number":346,"head":"` + headController +
`","base":"main","base_commit":"` + headHost + `"}]`, `"judge_from":"sibling novox/mesh-controller#346 merge abc"`} {
if !strings.Contains(string(body), want) {
t.Fatalf("the verdict lacks %s:\n%s", want, body)
}
}
unknown := checkedOf(link.BuildResult{ID: "b", Check: &link.CheckOutcome{Verdict: "pass", SiblingsUnknown: "mesh-delivery did not answer"}})
if body, _ := json.Marshal(unknown); !strings.Contains(string(body), `"siblings_unknown":"mesh-delivery did not answer"`) ||
strings.Contains(string(body), `"siblings":`) {
t.Fatalf("the verdict that could not learn them says %s", body)
}
}
+22 -517
View File
@@ -2,8 +2,6 @@ package main
import (
"context"
"encoding/json"
"fmt"
"io"
"os"
"path"
@@ -12,46 +10,18 @@ import (
"slices"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A pull request's merge check (novox/hq to-be 45 §9, ADR 0237 as amended 2026-10-06): the forge
// announces every pull request's new head, the controller decides what is checked, asks the build seat to
// check it, and says the verdict as `checked`, which the forge's holder sets as the pull request's
// statuses. **Before merge, never after**: every check the mesh had ran after a merge, on a machine.
//
// **The mesh's module graph decides, not the repository.** The controller holds the graph — every module,
// the repository and directory it is built from — and maps the pull request's changed paths onto it by
// the planner's own answer (reachOfMerge, touchedBy — the one place a changed file is mapped onto modules,
// for the merge handler, the release planner, the merge gate and this check; issue 280): **a changed file
// touches exactly the modules whose build reads it** — a module's own directory (the whole repository for
// one built from its root), or a repository its recipe packages. A file no build reads — a script at the
// root, a README — touches no module. A directory the change adds a module.json in, which the graph does
// not hold yet (said by the head, issue 278), is a new module and is checked too.
//
// - touches a module: the build seat runs **the gate** — `mesh/merge-gate`, the touched manifests, every
// machine composed with the change, the replays — and the repository's own merge-check.sh beside it;
// - touches none, in a repository that is the mesh's (it sources a module on some branch, or shares the
// core's owner): the gate is a pass that says so — a fact, not a missing check — and the repository's
// own merge-check.sh runs as `mesh/repo-check`, a warning when it has none;
// - touches none, anywhere else: the gate is a pass that says so, and nothing more is said.
//
// What is checked is decided here and run there (internal/builder/check.go).
// checkTimeout is how long one check may run on the build seat. Said here so the ask's watchdog (S6)
// and the builder agree on what late means.
const checkTimeout = 45 * time.Minute
// noModuleTouched is the gate's word for a change that touches nothing of the graph.
const noModuleTouched = "the change touches no module of the mesh's graph"
// noMergeCheck is the repository layer's word for a repository of the mesh with no merge-check.sh.
const noMergeCheck = "the repository declares no merge-check.sh: none of its own tests run before it merges"
// What a pull request reaches of the mesh's graph (novox/hq to-be 45 §9, issue 280). The controller no longer checks a
// pull request or says a verdict on it (R4g, ADR 0503 §3, ADR 0501 §5): the checker judges the heads, and mesh-delivery
// asks the operator's Approve on its verdict. What stays here is the mapping of a pull request's changed paths onto the
// mesh's module graph by the planner's own answer (reachOfMerge, touchedBy), which the delivery verbs (`delivery order`,
// the change plan) and the head's declared config (tellPullDeclares) read: **a changed file touches exactly the modules
// whose build reads it** — a module's own directory (the whole repository for one built from its root), or a repository
// its recipe packages — and a directory the change adds a module.json in is a new module.
// coreModules are the modules whose repositories are the mesh's core, by the directory a check finds
// each beside it — and whose owner is the mesh's own.
@@ -75,8 +45,6 @@ type checkScope struct {
// Mesh says the repository is the mesh's: modules are built from it on some branch, its owner is the
// core's, or the change adds a module to it.
Mesh bool
// Judge is who judges the gate (link.JudgeSelf, or the running controller).
Judge string
// From is a module built from the repository, for how the mesh clones it; nil when none is.
From *inventory.Entry
// Reach is the planner's whole answer, which the change plan is made from.
@@ -98,10 +66,6 @@ func pullScope(p link.PullUpdated, entries []inventory.Entry, read map[string][]
Width: len(r.Plan.Modules), Tiers: len(r.Plan.Tiers), Reach: r}
for _, e := range append(append([]inventory.Entry{}, r.Touched...), r.Deleted...) {
s.Manifests = append(s.Manifests, path.Join(strings.Trim(e.Source.Path, "/"), moduleManifestFile))
switch e.Manifest.Module {
case "mesh-controller":
s.Judge = link.JudgeSelf
}
}
for _, d := range r.Added {
s.Manifests = append(s.Manifests, path.Join(d, moduleManifestFile))
@@ -137,315 +101,37 @@ func sourceOwner(repository string) string {
return strings.ToLower(parts[len(parts)-2])
}
// PullUpdated decides a pull request's merge check, and asks for it when there is something to run.
// PullUpdated hears a pull request's new head. It asks no check and says no verdict (pullUpdated).
func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
_, err := f.pullUpdated(ctx, p, os.Stdout)
return err
return f.pullUpdated(ctx, p, os.Stdout)
}
// pullUpdated is PullUpdated saying what it did to w. It returns the id of the check it asked of the build seat, or
// "" when it said a verdict itself and asked nothing. A command that answers JSON on standard output (`delivery
// check`) gives it standard error, so its output stays the document alone (novox/hq issue 1311).
func (f following) pullUpdated(ctx context.Context, p link.PullUpdated, w io.Writer) (string, error) {
// pullUpdated is PullUpdated saying what it did to w. It asks nothing of anybody and says no verdict (R4g; ADR 0503 §3, §5,
// ADR 0527): the checker judges every repository whose trunk requires `mesh/check`, hq included. All it still does is tell
// mesh-config what the head declares (ADR 0502), whatever judges it.
func (f following) pullUpdated(ctx context.Context, p link.PullUpdated, w io.Writer) error {
inv := f.open.inventory
entries, err := inv.Catalogued(ctx)
if err != nil {
return "", err
return err
}
read, err := readForPlanning(ctx, inv)
if err != nil {
return "", err
return err
}
edges, err := inv.Dependencies(ctx)
if err != nil {
return "", err
return err
}
scope := pullScope(p, entries, read, edges)
// **A cycle of edges fails the check, naming every module and edge** (novox/hq ADR 0419 clause 6), a lone pull
// request's as a group's composed check is: no order exists for what it changes.
alone := orderMember{ID: p.Owner + "/" + p.Repo + "@" + twelve(p.Commit), Repository: p.Owner + "/" + p.Repo, Base: p.Base,
Head: p.Commit, Number: p.Number, Paths: p.Paths, PathsTruncated: p.PathsTruncated, Removed: p.Removed, BaseSHA: p.BaseSHA}
if failing, cyclic, err := cycleVerdict(ctx, "", []orderMember{alone},
map[string]orderReach{alone.ID: {Moved: scope.Modules}}, edges, entries); err != nil {
return "", err
} else if cyclic {
failing.Commit, failing.Trunk = p.Commit, twelve(p.BaseSHA)
fmt.Fprintf(w, "%s/%s#%d (%.8s): %s\n", p.Owner, p.Repo, p.Number, p.Commit, failing.Summary)
sayChecked(ctx, w, failing)
return "", nil
this := pullAtOf(p)
if this.Truncated {
this.Paths, this.Truncated = truncatedPaths(entries, p.Owner, p.Repo, scope.Modules), false
}
// **A member of a group whose gate is the composed check's runs its repository's check alone** (novox/hq ADR 0403
// clause 1): mesh-delivery is asked once, and says so on the group's line. Told nothing, the pull request is
// checked alone, gate included (clause 6).
lines, groupsErr := groupsAsked(ctx, deliverySeatHeld(entries))
composed := groupsErr == nil && composedMember(p, lines)
// The change plan of the commit at hand (ADR 0238): what a merge of it would build and send, posted
// with the verdict whatever the verdict is.
plan := changePlanOf(p.Owner+"/"+p.Repo, p.Base, p.Commit, scope.Reach, entries, func(module string) (inventory.Upgrade, bool) {
u, err := inv.UpgradeOf(ctx, module)
return u, err == nil
})
fmt.Fprint(w, planText(plan))
direct := directVerdict(p, plan)
if composed {
scope.Modules, scope.Dependents, scope.New, scope.Manifests = nil, nil, nil, nil
composedGate(&direct)
if scope.gated() || holdsModules(entries, p.Owner, p.Repo) {
tellPullDeclares(ctx, w, []pullAt{this})
}
switch {
case !scope.gated() && !scope.Mesh:
fmt.Fprintf(w, "%s/%s#%d (%.8s): %s, and the repository is not the mesh's: said, nothing run\n",
p.Owner, p.Repo, p.Number, p.Commit, noModuleTouched)
sayChecked(ctx, w, direct)
return "", nil
case !scope.gated() && p.MergeCheckSaid && !p.MergeCheck:
direct.RepoCheck = &link.CheckLayer{Verdict: "warning", Summary: noMergeCheck}
fmt.Fprintf(w, "%s/%s#%d (%.8s): %s; %s\n", p.Owner, p.Repo, p.Number, p.Commit, noModuleTouched, noMergeCheck)
sayChecked(ctx, w, direct)
return "", nil
}
request, err := checkRequestFor(ctx, f.open, p, scope, entries)
if err != nil {
return "", err
}
request.Check.Plan = &plan
request.Check.Composed = composed
// **The group's other open pull requests are read beside it** (novox/hq ADR 0382 §1).
siblingsFrom(&request, p, entries, lines, groupsErr)
seat := buildSeatHeld(ctx)
ask, err := askOverOn(seat)
if err != nil {
return "", err
}
defer ask.Close()
if err := ask.Ask(ctx, request); err != nil {
return "", err
}
// **A newer request replaces the one still waiting, and a newer key stops the one running** (novox/hq ADR 0406 §3, §4,
// issue 974): the check queue keeps one request a pull request, and a running check watches its key. Nothing is asked
// of the forge here (issue 726).
what := "its own merge-check.sh alone: " + noModuleTouched
if composed {
what = "its own merge-check.sh alone: the gate is its group's composed check's"
}
if scope.gated() {
what = fmt.Sprintf("the gate over %s (a merge would build %d module(s) in %d tier(s))",
strings.Join(append(append([]string{}, scope.Modules...), prefixedAll("new:", scope.New)...), ", "),
scope.Width, scope.Tiers)
}
fmt.Fprintf(w, "%s/%s#%d (%.8s): asked %s to check it before it merges — %s — as %s\n", p.Owner, p.Repo, p.Number,
p.Commit, seat, what, request.ID)
return request.ID, nil
}
// composedWords is what a group member's own verdict says of its gate layer (novox/hq ADR 0403 clause 1).
const composedWords = "judged by its group's composed check"
// gateComposed is a gate layer's verdict when no gate was run for the member alone: never a pass. Mesh-delivery reads it
// as the composed check's to judge, and the forge's holder sets no mesh/merge-gate from it.
const gateComposed = "composed"
// composedGate says a group member's own verdict has no gate of its own: its gate layer, and the verdict that is
// the gate's, read "composed" in place of a pass.
func composedGate(c *link.Checked) {
c.Verdict, c.Summary = gateComposed, composedWords
c.Gate = &link.CheckLayer{Verdict: gateComposed, Summary: composedWords}
}
// directVerdict is the verdict of a change no build judges — it touches no module, and either the repository is not
// the mesh's or the head holds no merge-check.sh. **It names the trunk commit it holds for** (novox/hq ADR 0323 §3):
// the base branch's head the forge announced with the head, so the merge tool can weigh it against the trunk's head
// and does not refuse it for ever (a verdict naming no trunk is never the trunk's head's). Nothing was built to
// judge: the trunk commit is the one the forge said the change was announced against.
func directVerdict(p link.PullUpdated, plan link.ChangePlan) link.Checked {
trunk := p.BaseSHA
if len(trunk) > 12 {
trunk = trunk[:12]
}
if trunk != "" {
plan.Trunk = trunk
}
return link.Checked{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Commit: p.Commit,
ID: link.NewBuildID(time.Now()), Verdict: "pass", Summary: noModuleTouched,
Gate: &link.CheckLayer{Verdict: "pass", Summary: noModuleTouched}, Plan: &plan, Trunk: trunk}
}
// checkRequestFor is the ask for one pull request's head: the repository as the mesh clones it, the head,
// and what is read beside it.
func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scope checkScope,
entries []inventory.Entry) (link.BuildRequest, error) {
shelf := map[string]catalogue.Manifest{}
for _, e := range entries {
shelf[e.Manifest.Module] = e.Manifest
}
world, err := restOfTheMesh(ctx, open, shelf, "")
if err != nil {
return link.BuildRequest{}, err
}
clone := func(s inventory.Source) (string, error) {
if s.Seat == "" {
return s.Repository, nil
}
return clonedFromSeat(world, s.Seat, s.Repository)
}
// As the mesh clones a module built from it; a repository no module is built from, from the forge.
source := inventory.Source{Seat: gitSeat, Repository: p.Owner + "/" + p.Repo}
if scope.From != nil {
source = scope.From.Source
}
repository, err := clone(source)
if err != nil {
return link.BuildRequest{}, err
}
current, err := open.inventory.CurrentBuilds(ctx)
if err != nil {
return link.BuildRequest{}, err
}
beside, err := besideOf(entries, func(module string) string { return current[module].Commit }, clone)
if err != nil {
return link.BuildRequest{}, err
}
builds := checkBuildsOf(p, scope, entries, clone)
return link.BuildRequest{
ID: link.NewBuildID(time.Now()),
Repository: repository,
Ref: p.Commit,
Held: heldBy(ctx),
Seats: seatBases(ctx),
Source: sourceOnSeat(source),
Check: checkOf(p, scope, entries, beside, builds, namedForTheCheck(ctx, p, builds), runningBy(ctx)),
}, nil
}
// checkOf is the check a request carries: the pull request, what the change touches, and the buckets of the pull
// requests' keys of the modules claiming the delivery seat (novox/hq ADR 0437 §3, issue 1212).
func checkOf(p link.PullUpdated, scope checkScope, entries []inventory.Entry, beside map[string]link.CheckedOut,
builds []link.CheckBuild, named []link.CheckedBuild, running map[string]string) *link.CheckRequest {
return &link.CheckRequest{Pulls: pullsBucketsOf(entries), Owner: p.Owner, Repo: p.Repo, Number: p.Number, Base: p.Base,
Trunk: p.BaseSHA, Ready: p.Ready, Paths: p.Paths, Beside: beside, Modules: scope.Modules, Dependents: scope.Dependents,
New: scope.New, Manifests: scope.Manifests, Judge: scope.Judge, Builds: builds, Named: named, Running: running}
}
// besideOf is what a check reads beside its own repository, by the directory it is found under: each core repository
// by the module the mesh builds from it, at besideRefs. running is the commit the mesh runs of a module.
func besideOf(entries []inventory.Entry, running func(module string) string,
clone func(inventory.Source) (string, error)) (map[string]link.CheckedOut, error) {
beside := map[string]link.CheckedOut{}
for _, e := range entries {
dir, core := coreModules[e.Manifest.Module]
if !core || e.Provided || e.Source.Repository == "" {
continue
}
url, err := clone(e.Source)
if err != nil {
return nil, err
}
refs := besideRefs(dir, running(e.Manifest.Module))
beside[dir] = link.CheckedOut{Repository: url, Ref: refs[dir]}
if dir == "mesh-controller" {
// The controller's main stays its main: it judges only a check the running controller predates.
beside["mesh-controller-main"] = link.CheckedOut{Repository: url, Ref: refs["mesh-controller-main"]}
if e.Source.Seat != "" {
for _, sibling := range []string{"mesh-lab", "mesh-sdk"} {
if url, err := clone(inventory.Source{Seat: e.Source.Seat, Repository: siblingOf(e.Source.Repository,
sibling)}); err == nil {
beside[sibling] = link.CheckedOut{Repository: url, Ref: refs[sibling]}
}
}
}
}
}
return beside, nil
}
// trunkOf is the branch a module's source follows: main when it records none, or records a commit.
func trunkOf(s inventory.Source) string {
if t := strings.TrimSpace(followedBranch(s.Ref)); t != "" {
return t
}
return "main"
}
// sourceName is the name of a recorded repository, a path on the git seat or a URL: novox/mesh-host → mesh-host.
func sourceName(repository string) string {
parts := strings.Split(strings.Trim(strings.TrimSuffix(repository, ".git"), "/"), "/")
if len(parts) < 2 {
return ""
}
return parts[len(parts)-1]
}
// namedForTheCheck are the builds earlier checks of the same pull request at the same head named, read once for the
// check (novox/hq issue 703): asked again because only the trunk moved, the check keeps what they built of a key
// still the same. Another pull request's build, or one of an earlier head, is never kept. None when they cannot be
// read, and the check builds, as before.
func namedForTheCheck(ctx context.Context, p link.PullUpdated, builds []link.CheckBuild) []link.CheckedBuild {
if len(builds) == 0 || p.Commit == "" {
return nil
}
pull := link.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number}
read, err := checkedBuildsOf(ctx, []link.PullRef{pull})
if err != nil {
fmt.Printf(" the earlier checks' builds of %s/%s#%d could not be read, so the check builds: %v\n",
p.Owner, p.Repo, p.Number, err)
// empty-on-error: said above; a check handed no named build builds every module, as before issue 703
return nil
}
var named []link.CheckedBuild
for _, b := range read {
if b.Pull == pull && link.SameHead(b.Head, p.Commit) {
named = append(named, b)
}
}
return named
}
// checkBuildsOf are the modules a merge of the change would build, in the walk's order, as the check builds them
// for the walk to take (novox/hq ADR 0350 §1): a module built from the repository checked, at the branch it merges
// into, from the merge the check makes; any other — a module standing on one — from its own repository at the
// branch it follows, as the walk builds it. A module whose build reads the repository checked through a context,
// or that the merge deletes, cannot be built as the walk would: said, and nothing after it is built. A change that
// moves no module builds nothing.
func checkBuildsOf(p link.PullUpdated, scope checkScope, entries []inventory.Entry,
clone func(inventory.Source) (string, error)) []link.CheckBuild {
if len(scope.Reach.Plan.Tiers) == 0 {
return nil
}
m := link.SourceMoved{Owner: p.Owner, Repo: p.Repo, Base: p.Base, CloneURL: p.CloneURL}
byName := map[string]inventory.Entry{}
for _, e := range entries {
byName[e.Manifest.Module] = e
}
skip := map[string]string{}
for _, e := range scope.Reach.Packaging {
skip[e.Manifest.Module] = "its build reads " + p.Owner + "/" + p.Repo + " through a context, which a check " +
"cannot hand the merge"
}
for _, e := range scope.Reach.Deleted {
skip[e.Manifest.Module] = "the merge deletes it"
}
var out []link.CheckBuild
for tier, names := range scope.Reach.Plan.Tiers {
for _, name := range names {
e, known := byName[name]
b := link.CheckBuild{Module: name, Tier: tier, Skip: skip[name]}
switch {
case !known:
b.Skip = "the catalogue does not hold it"
case b.Skip != "":
case sourceIs(e.Source, m):
b.Merged, b.Path = true, e.Source.Path
default:
url, err := clone(e.Source)
if err != nil {
b.Skip = "its repository cannot be cloned: " + err.Error()
break
}
b.Repository, b.Path, b.Ref = url, e.Source.Path, followedBranch(e.Source.Ref)
}
out = append(out, b)
}
}
return out
return nil
}
// besideRefs is the ref each repository is cloned at beside a check, by the directory it is found under:
@@ -501,186 +187,5 @@ func sdkPinned() string {
return "main"
}
// siblingOf is another repository of the same owner: novox/mesh-controller → novox/mesh-lab.
func siblingOf(repository, name string) string {
if cut := strings.LastIndex(repository, "/"); cut >= 0 {
return repository[:cut+1] + name
}
return name
}
// sourceOnSeat is a source's seat form, nil for one on no seat.
func sourceOnSeat(s inventory.Source) *link.SourceOnSeat {
if s.Seat == "" {
return nil
}
return &link.SourceOnSeat{Seat: s.Seat, Repository: s.Repository}
}
// checkEvents is where the serving controller says a check's verdict; nil in a command.
var checkEvents link.Bus
// maxCheckReport is how much of a check's report travels in its verdict: enough for the failures and
// the machines, never a log.
const maxCheckReport = 60 << 10
// checked says a merge check's verdict as the controller's `checked`. Nothing is recorded or
// registered: a check builds nothing (issue 240's rule for a dry run, kept for a check).
func checked(ctx context.Context, result link.BuildResult) {
// **A check stopped by its key says nothing** (novox/hq ADR 0406 §4, issue 974, review 2): it stopped because the head or
// the trunk commit moved on, and the newer request will judge. Said as an error it would name the live head — red
// statuses on the forge, a delivery refused — and written to the key it would stand in the newer request's way.
if result.Failed == link.SupersededByKey {
fmt.Printf("%s: stopped, its pull request's key moved on; the newer check will judge\n", result.ID)
return
}
sayChecked(ctx, os.Stdout, checkedOf(result))
}
// checkedOf is what a check's outcome says: each layer, and an error — never a pass — for a check that
// could not run.
func checkedOf(result link.BuildResult) link.Checked {
c := link.Checked{ID: result.ID, On: result.On, Commit: result.Ref}
if result.Checked != nil {
c.Owner, c.Repo, c.Number = result.Checked.Owner, result.Checked.Repo, result.Checked.Number
}
switch {
case result.Check != nil:
c.Verdict, c.Summary, c.Report = result.Check.Verdict, result.Check.Summary, result.Check.Report
c.Gate, c.RepoCheck = result.Check.Gate, result.Check.RepoCheck
// What was judged (novox/hq ADR 0323 §3): the head merged into the trunk commit, and the commits of the
// repositories read beside it.
c.Trunk, c.Beside, c.Took = result.Check.Trunk, result.Check.Beside, result.Check.Took
c.Built = result.Check.Built
c.Validator = result.Check.Validator
c.Siblings, c.SiblingsUnknown, c.JudgeFrom = result.Check.Siblings, result.Check.SiblingsUnknown, result.Check.JudgeFrom
if c.Gate == nil {
// A build seat from before the layers: its verdict is the gate's.
c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
}
case result.Failed != "":
// The check could not run: an error, never read as a pass — on both layers it was asked for.
c.Verdict, c.Summary = "error", "the check could not run: "+firstLine(result.Failed)
default:
c.Verdict, c.Summary = "error", "the build seat answered the check with no verdict"
}
if c.Verdict == "" {
c.Verdict = "error"
}
if result.Check == nil {
c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
c.RepoCheck = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
}
// **A group member's own check has no gate of its own** (novox/hq ADR 0403 clause 1): what ran was the repository's
// check alone, and its gate layer is "composed", never a pass. A failure or an error stays what it was.
if result.Checked != nil && result.Checked.Composed && result.Check != nil && c.Gate != nil &&
(c.Gate.Verdict == "pass" || c.Gate.Verdict == "warning") {
composedGate(&c)
}
if result.Checked != nil && c.Gate != nil && len(c.Gate.Modules) == 0 {
c.Gate.Modules = append(append([]string{}, result.Checked.Modules...), prefixedAll("new:", result.Checked.New)...)
}
if result.Checked != nil && c.Gate != nil && len(c.Gate.Dependents) == 0 {
c.Gate.Dependents = result.Checked.Dependents
}
if result.Checked != nil {
c.Plan = result.Checked.Plan
if c.Plan != nil && c.Trunk != "" {
// The kept delivery plan records the trunk commit with its other inputs (ADR 0323 §3).
plan := *c.Plan
plan.Trunk = c.Trunk
c.Plan = &plan
}
// A delivery group's composed check (novox/hq ADR 0239): every head it judged, this one first.
if g := result.Checked; g.Group != "" {
c.Group = g.Group
c.Members = append(c.Members, link.CheckedMember{Owner: g.Owner, Repo: g.Repo, Number: g.Number,
Commit: result.Ref, Trunk: c.Trunk})
for _, m := range g.Members {
c.Members = append(c.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo, Number: m.Number,
Commit: m.Ref, Trunk: memberTrunk(result.Check, m)})
}
}
}
for _, l := range []*link.CheckLayer{c.Gate, c.RepoCheck} {
if l != nil && l.Verdict == "" {
l.Verdict = "error"
}
}
// **A check run again says so** (novox/hq issue 594). The summaries are the lines the pull request
// shows, clipped by the forge to a few words, so they get only a short marker after the verdict's own
// words; the report gets the whole sentence. Layers copied, so the build's own outcome is left as it came.
if mark := link.RunMarker(result.Delivery); mark != "" {
c.Summary += " " + mark
for _, l := range []**link.CheckLayer{&c.Gate, &c.RepoCheck} {
if *l != nil {
layer := **l
layer.Summary += " " + mark
*l = &layer
}
}
c.Report = link.Restarted(result.Delivery) + "\n\n" + c.Report
}
if len(c.Report) > maxCheckReport {
c.Report = "…" + c.Report[len(c.Report)-maxCheckReport:]
}
return c
}
// memberTrunk is the trunk commit the build seat merged one of a group's heads into (novox/hq ADR 0323 §6), empty
// when it said none.
func memberTrunk(outcome *link.CheckOutcome, m link.GroupMember) string {
if outcome == nil {
return ""
}
for _, j := range outcome.Members {
if strings.EqualFold(j.Owner, m.Owner) && strings.EqualFold(j.Repo, m.Repo) && strings.HasPrefix(m.Ref, j.Commit) {
return j.Trunk
}
}
return ""
}
func prefixedAll(prefix string, items []string) []string {
out := make([]string, 0, len(items))
for _, i := range items {
out = append(out, prefix+i)
}
return out
}
// sayChecked says a merge check's verdict on the bus, where the forge's holder hears it.
func sayChecked(ctx context.Context, w io.Writer, c link.Checked) {
repo := "none"
if c.RepoCheck != nil {
repo = strings.ToUpper(c.RepoCheck.Verdict) + " — " + c.RepoCheck.Summary
}
judged := ""
if c.Trunk != "" {
judged = " merged into trunk " + c.Trunk
}
fmt.Fprintf(w, "%s: %s/%s#%d at %.8s%s checked on %s: gate %s — %s; repository %s\n", c.ID, c.Owner, c.Repo, c.Number,
c.Commit, judged, orSomewhere(c.On), strings.ToUpper(c.Verdict), c.Summary, repo)
if checkEvents == nil {
return
}
body, err := json.Marshal(c)
if err != nil {
return
}
stating, stop := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
defer stop()
if err := checkEvents.PublishSeatEvent(stating, link.MeshControllerSeat, link.KeyChecked, body); err != nil {
fmt.Fprintf(w, "%s: the verdict could not be said, so the pull request is not told it: %v\n", c.ID, err)
}
}
// pullsBucketsOf are the buckets of the pull requests' keys a check names for the build agent: one per module
// claiming the delivery seat, in the order of their names (novox/hq ADR 0437 §3, issue 1212).
func pullsBucketsOf(entries []inventory.Entry) []string {
var out []string
for _, m := range claimantsOf(entries, catalogue.DeliverySeat) {
out = append(out, broker.BucketName(m, "pulls"))
}
return out
}
@@ -1,56 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
func TestACheckStoppedByItsKeyIsAnnouncedAsNoVerdict(t *testing.T) {
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
if err := broker.AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
legacy, err := js.Conn().JetStream()
if err != nil {
t.Fatal(err)
}
was := checkEvents
checkEvents = link.OverNATS{Conn: js.Conn(), JS: legacy}
t.Cleanup(func() { checkEvents = was })
heard := make(chan *nats.Msg, 4)
sub, err := js.Conn().ChanSubscribe(link.SeatEventSubject(link.MeshControllerSeat, link.KeyChecked), heard)
if err != nil {
t.Fatal(err)
}
defer func() { _ = sub.Unsubscribe() }()
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog", Number: 3}
checked(t.Context(), link.BuildResult{ID: "stopped", Ref: "abc", Checked: asked, Failed: link.SupersededByKey})
checked(t.Context(), link.BuildResult{ID: "broken", Ref: "abc", Checked: asked, Failed: "the facts snapshot cannot be read"})
select {
case m := <-heard:
var c link.Checked
_ = json.Unmarshal(m.Data, &c)
if c.ID != "broken" || c.Verdict != "error" {
t.Fatalf("the first verdict said is %+v; want only the broken check's error", c)
}
case <-time.After(5 * time.Second):
t.Fatal("nothing was said")
}
select {
case m := <-heard:
t.Fatalf("a second verdict was said: %s", m.Data)
case <-time.After(300 * time.Millisecond):
}
}
-97
View File
@@ -1,97 +0,0 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"fmt"
"testing"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// Acceptance 5 of milestone 2 (novox/hq ADR 0406 §3, issue 974), live, with the store and a real bus: two merges into one
// repository within a minute, with two open pull requests, leave each pull request with exactly one queued check, by the
// stream's rule. Each merge moves the trunk, and mesh-delivery asks the controller to check both open pull requests again
// against it (`delivery check`, which is the controller's own path for a head announced); nothing moves a request, nothing
// asks the forge, and the queue is read back whole.
func TestTwoMergesWithinAMinuteLeaveEachOpenPullRequestOneQueuedCheck(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
js, _ := aBuildQueue(t)
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
holdTheDeliverySeat(t, open)
was := askDeliveryOwner
t.Cleanup(func() { askDeliveryOwner = was })
askDeliveryOwner = func(_ context.Context, _ *nats.Conn, verb string, _ map[string]any) (json.RawMessage, error) {
if verb != "groups" {
t.Fatalf("asked the owner %s", verb)
}
return json.RawMessage(`[]`), nil
}
// The repository the two pull requests are into, as a module the mesh builds from it.
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
heads := map[int]string{7: "7777777777777777777777777777777777777777", 8: "8888888888888888888888888888888888888888"}
check := func(number int, trunk string) {
t.Helper()
p := orderMember{ID: fmt.Sprintf("novox/mesh-controller@%.12s", heads[number]), Repository: "novox/mesh-controller",
Base: "main", Head: heads[number], Number: number, Paths: []string{"main.go"}, ModuleDirsSaid: true,
BaseSHA: trunk, Ready: number == 8}.pull()
p.MergeCheck, p.MergeCheckSaid = true, true
if err := (following{open: open, oneShot: true}).PullUpdated(ctx, p); err != nil {
t.Fatal(err)
}
}
// Announced against the trunk as it stands, then asked again after each of two merges.
for _, trunk := range []string{"1111111111111111111111111111111111111111", "2222222222222222222222222222222222222222",
"3333333333333333333333333333333333333333"} {
check(7, trunk)
check(8, trunk)
}
q, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil {
t.Fatal(err)
}
seen := map[int]string{}
for _, a := range q.Asks {
c := a.Request.Check
if c == nil {
t.Fatalf("%s is no check", a.ID)
}
if _, twice := seen[c.Number]; twice {
t.Errorf("pull request %d has two queued checks", c.Number)
}
seen[c.Number] = c.Trunk
if a.State != link.AskWaiting {
t.Errorf("%s is %s", a.ID, a.State)
}
if want := "3333333333333333333333333333333333333333"; c.Trunk != want {
t.Errorf("pull request %d is queued against %.8s; want the newest trunk %.8s", c.Number, c.Trunk, want)
}
}
if len(seen) != 2 {
t.Fatalf("queued checks for %v; want one for each of the two open pull requests", seen)
}
// By the streams' own count: the ready one on its stream, the other on its.
for stream, want := range map[string]uint64{broker.ChecksReadyStream: 1, broker.ChecksOtherStream: 1} {
info, err := js.Context().StreamInfo(stream)
if err != nil {
t.Fatal(err)
}
if info.State.Msgs != want {
t.Errorf("%s holds %d request(s); want %d", stream, info.State.Msgs, want)
}
}
}
+2 -143
View File
@@ -1,7 +1,6 @@
package main
import (
"errors"
"strings"
"testing"
@@ -76,8 +75,8 @@ func TestThePullRequestIsMappedOntoTheModuleGraph(t *testing.T) {
t.Errorf("%s: %s %q, wanted %q", c.what, what, got[i], want[i])
}
}
if s.Mesh != c.mesh || s.Judge != c.judge {
t.Errorf("%s: the mesh's %v judged by %q, wanted %v by %q", c.what, s.Mesh, s.Judge, c.mesh, c.judge)
if s.Mesh != c.mesh {
t.Errorf("%s: the mesh's %v, wanted %v", c.what, s.Mesh, c.mesh)
}
if s.gated() != (c.modules != "" || c.new != "") {
t.Errorf("%s: gated %v", c.what, s.gated())
@@ -95,28 +94,6 @@ func TestAPullRequestTouchesWhatPackagesItsRepository(t *testing.T) {
}
}
// Each layer is said; a check that could not run is an error on both, never a pass; a build seat from
// before the layers is read as the gate.
func TestAChecksLayersAreEachSaidAndAnErrorIsNeverAPass(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog", Number: 3, Modules: []string{"gitea"}}
c := checkedOf(link.BuildResult{ID: "b", Ref: "abc", Checked: asked, Failed: "the facts snapshot cannot be read"})
if c.Verdict != "error" || c.Gate == nil || c.Gate.Verdict != "error" || c.RepoCheck == nil || c.RepoCheck.Verdict != "error" {
t.Fatalf("a check that could not run said %+v", c)
}
if strings.Join(c.Gate.Modules, ",") != "gitea" {
t.Errorf("the gate names %v", c.Gate.Modules)
}
c = checkedOf(link.BuildResult{ID: "b", Ref: "abc", Checked: asked, Check: &link.CheckOutcome{Verdict: "warning", Summary: "wide"}})
if c.Gate == nil || c.Gate.Verdict != "warning" || c.RepoCheck != nil {
t.Fatalf("an outcome without layers said %+v", c)
}
c = checkedOf(link.BuildResult{ID: "b", Ref: "abc", Checked: asked, Check: &link.CheckOutcome{Verdict: "pass",
Gate: &link.CheckLayer{Verdict: "pass"}, RepoCheck: &link.CheckLayer{Verdict: "fail", Summary: "its merge-check.sh failed"}}})
if c.RepoCheck.Verdict != "fail" || c.Gate.Verdict != "pass" {
t.Fatalf("the layers said %+v / %+v", c.Gate, c.RepoCheck)
}
}
// **The check asks the planner, never a mapping of its own** (novox/hq ADR 0238): what a pull request
// reaches is reachOfMerge's answer — the same that plans a merge — so a file at the root touches no
// module in both, and what stands on a touched module is named as its dependents in both.
@@ -171,123 +148,5 @@ func TestAPullRequestReachesWhatAMergeOfItWouldPlan(t *testing.T) {
}
}
// **Only a commit on the trunk is published** (novox/hq ADR 0238): a build of a commit off its repository's
// default branch — a pull request's head, a branch built by hand, a rebuild or replay of one — is recorded
// and never registered, so nothing can send it; a check or a dry run never is either.
func TestABuildOffTheTrunkIsNeverPublished(t *testing.T) {
on := link.BuildResult{ID: "b", Commit: "abc", Trunk: "main", OnTrunk: true}
if err := publishable(on, ""); err != nil {
t.Errorf("a build on the trunk was refused: %v", err)
}
off := link.BuildResult{ID: "b", Commit: "abc", Trunk: "main"}
if err := publishable(off, ""); !errors.Is(err, errOffTheTrunk) || !strings.Contains(err.Error(), "main") {
t.Errorf("a build off the trunk was let through: %v", err)
}
for _, r := range []link.BuildResult{
{ID: "c", Trunk: "main", OnTrunk: true, Check: &link.CheckOutcome{Verdict: "pass"}},
{ID: "d", Trunk: "main", OnTrunk: true, Checked: &link.CheckRequest{}},
{ID: "e", Trunk: "main", OnTrunk: true, DryRun: true},
} {
if publishable(r, "") == nil {
t.Errorf("%s, a check or a dry run, was publishable", r.ID)
}
}
// A module that follows a branch other than the default: that branch is its trunk, and the default
// is not.
follows := link.BuildResult{ID: "g", Commit: "abc", Trunk: "master", Branches: []string{"nox-mesh"}}
if err := publishable(follows, "nox-mesh"); err != nil {
t.Errorf("a commit on the branch a module follows was refused: %v", err)
}
if err := publishable(link.BuildResult{ID: "h", Commit: "abc", Trunk: "master", OnTrunk: true,
Branches: []string{"master"}}, "nox-mesh"); err == nil {
t.Error("a commit on the default but not on the branch the module follows was published")
}
if err := publishable(link.BuildResult{ID: "i", Commit: "abc", Trunk: "main", Branches: []string{"feat/x"}}, ""); err == nil {
t.Error("a feature branch's commit was published")
}
// A build seat older than the rule says nothing: let through and said, so the rule can reach the mesh.
if err := publishable(link.BuildResult{ID: "f", Commit: "abc"}, ""); err != nil {
t.Errorf("a build seat that said nothing was refused: %v", err)
}
}
// **A verdict names the trunk commit it judged** (novox/hq ADR 0323 §3, issue 341): the controller's `checked`
// carries the head and the trunk commit the build seat merged it into, the commits of the repositories read beside
// it, how long it took, and the kept delivery plan records the trunk commit with its other inputs. A group's
// composed check names, per member, the trunk commit that head was merged into (§6).
func TestAVerdictNamesTheTrunkCommitItJudged(t *testing.T) {
plan := &link.ChangePlan{Repository: "novox/mesh-host", Base: "main", Head: "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
Summary: "builds mesh-host"}
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 66, Base: "main", Plan: plan}
c := checkedOf(link.BuildResult{ID: "b", Ref: "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", Checked: asked,
Check: &link.CheckOutcome{Verdict: "pass", Took: "4m12s", Head: "dfb1415aaaaa", Trunk: "d28ca95bbbbb",
Gate: &link.CheckLayer{Verdict: "pass"}, RepoCheck: &link.CheckLayer{Verdict: "pass"},
Beside: map[string]string{"mesh-catalog": "0123456789ab", "mesh-lab": "ba9876543210"}}})
if c.Trunk != "d28ca95bbbbb" || c.Commit != "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" {
t.Fatalf("the verdict names head %s and trunk %q", c.Commit, c.Trunk)
}
if c.Beside["mesh-catalog"] != "0123456789ab" || c.Beside["mesh-lab"] != "ba9876543210" {
t.Fatalf("the verdict names the repositories read beside it as %v", c.Beside)
}
if c.Took != "4m12s" {
t.Fatalf("the verdict says it took %q", c.Took)
}
if c.Plan == nil || c.Plan.Trunk != "d28ca95bbbbb" || c.Plan.Head != plan.Head {
t.Fatalf("the kept plan records %+v", c.Plan)
}
if plan.Trunk != "" {
t.Fatal("the plan the controller asked with was changed: the verdict's is a copy")
}
group := &link.CheckRequest{Owner: "novox", Repo: "mesh-controller", Number: 7, Base: "main", Group: "feat/x",
Members: []link.GroupMember{{Owner: "novox", Repo: "mesh-host", Number: 8, Ref: "cccccccccccc0000", Base: "main"}}}
c = checkedOf(link.BuildResult{ID: "g", Ref: "aaaaaaaaaaaa0000", Checked: group, Check: &link.CheckOutcome{
Verdict: "pass", Head: "aaaaaaaaaaaa", Trunk: "111111111111", Gate: &link.CheckLayer{Verdict: "pass"},
Members: []link.CheckedMember{{Owner: "novox", Repo: "mesh-host", Commit: "cccccccccccc", Trunk: "222222222222"}}}})
if len(c.Members) != 2 || c.Members[0].Trunk != "111111111111" || c.Members[1].Trunk != "222222222222" ||
c.Members[1].Commit != "cccccccccccc0000" {
t.Fatalf("the composed check names its members %+v", c.Members)
}
}
// **The merge made for a check is never registered** (novox/hq ADR 0323 §2): a build of it — a commit on no branch
// of the forge — is refused, naming the trunk. The rule of ADR 0238 §1 stands unchanged; this holds it for the
// merge commit a check makes.
func TestABuildOfACheckMergeIsRefusedNamingTheTrunk(t *testing.T) {
merge := link.BuildResult{ID: "b", Commit: "4b5c6d7e8f90aabbccddeeff0011223344556677", Trunk: "main"}
err := publishable(merge, "")
if !errors.Is(err, errOffTheTrunk) || !strings.Contains(err.Error(), "not on main") {
t.Fatalf("a build of a check's merge was answered %v", err)
}
}
// **A verdict no build judges names its trunk commit too** (novox/hq ADR 0323 §3): a change touching no module of a
// repository with no merge-check.sh (novox/mesh-tools-go) is answered by the controller itself. Naming no trunk, the
// merge tool refused it, asked again, got the same verdict, and refused for ever.
func TestADirectVerdictNamesTheTrunkTheForgeAnnounced(t *testing.T) {
p := link.PullUpdated{Owner: "novox", Repo: "mesh-tools-go", Number: 3, Base: "main", Commit: "1a2b3c4d5e6f7a8b9c0d",
BaseSHA: "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b"}
c := directVerdict(p, link.ChangePlan{Repository: "novox/mesh-tools-go", Base: "main", Head: p.Commit})
if c.Trunk != "7a8b9c0d1e2f" || c.Plan == nil || c.Plan.Trunk != "7a8b9c0d1e2f" {
t.Fatalf("the direct verdict names trunk %q, its plan %+v", c.Trunk, c.Plan)
}
if c.Verdict != "pass" || c.Commit != p.Commit {
t.Fatalf("the direct verdict is %+v", c)
}
if c := directVerdict(link.PullUpdated{Owner: "novox", Repo: "x", Commit: "abc"}, link.ChangePlan{}); c.Trunk != "" {
t.Fatalf("a forge that said no base head gave the trunk %q", c.Trunk)
}
}
// A head checked again through mesh-delivery (`delivery check --members`, a recheck) reaches the same direct verdict:
// it names the base head the forge last announced, kept on the member, or the merge tool would refuse it for ever.
func TestARecheckedHeadsDirectVerdictNamesItsTrunk(t *testing.T) {
m := orderMember{ID: "novox/mesh-tools-go@1a2b3c4d5e6f", Repository: "novox/mesh-tools-go", Base: "main",
Head: "1a2b3c4d5e6f7a8b9c0d", Number: 3, BaseSHA: "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b"}
if c := directVerdict(m.pull(), link.ChangePlan{}); c.Trunk != "7a8b9c0d1e2f" {
t.Fatalf("a rechecked head's direct verdict names trunk %q", c.Trunk)
}
}
// A check stopped by its pull request's key moving on says no verdict (issue 974, review 2): an error said for it would name the
// live head, and the newer request is the one that judges. A check that failed for any other reason still says its error.
+16 -7
View File
@@ -4,7 +4,9 @@ import (
"context"
"errors"
"fmt"
"io"
"os"
"strings"
"time"
"github.com/novox/mesh-controller/internal/artifacts"
@@ -215,27 +217,34 @@ func collect(ctx context.Context, inv *inventory.Inventory) {
}
r := sweep(ctx, inv, artifacts.Store{Address: address}, references, kept, afterBuild)
sayCollected(os.Stderr, r)
}
// sayCollected says what one sweep after a build did.
func sayCollected(w io.Writer, r sweepResult) {
if r.HoldersWritten > 0 {
fmt.Fprintf(os.Stderr, "the artifact store now holds %d more kept archive(s) by a manifest\n", r.HoldersWritten)
fmt.Fprintf(w, "the artifact store now holds %d more kept archive(s) by a manifest\n", r.HoldersWritten)
}
if r.Missing > 0 {
fmt.Fprintf(os.Stderr, "%d archive(s) the mesh keeps are not in the artifact store at all; "+
fmt.Fprintf(w, "%d archive(s) the mesh keeps are not in the artifact store at all; "+
"`collection` lists them\n", r.Missing)
}
if len(r.LetGo) > 0 {
fmt.Fprintf(os.Stderr, "the artifact store let go of %d artifact(s) the mesh no longer keeps\n", len(r.LetGo))
// **Each one named** (novox/hq issue 1933): a count could not say which archive a machine's 404 was let go of.
fmt.Fprintf(w, "the artifact store let go of %d artifact(s) the mesh no longer keeps: %s\n", len(r.LetGo),
strings.Join(r.LetGo, ", "))
}
if r.Stopped != "" && !r.Bounded {
fmt.Fprintln(os.Stderr, r.Stopped)
fmt.Fprintln(w, r.Stopped)
}
if r.Held && r.Left > 0 {
fmt.Fprintf(os.Stderr, "%d more to collect; the next build asks again\n", r.Left)
fmt.Fprintf(w, "%d more to collect; the next build asks again\n", r.Left)
}
if r.Skipped > 0 {
fmt.Fprintf(os.Stderr, "%d artifact(s) the sweep will not address were skipped\n", r.Skipped)
fmt.Fprintf(w, "%d artifact(s) the sweep will not address were skipped\n", r.Skipped)
}
if r.Indexes > 0 {
fmt.Fprintf(os.Stderr, "%d eligible index(es) wait for a collect a person confirms, which takes their platforms too\n", r.Indexes)
fmt.Fprintf(w, "%d eligible index(es) wait for a collect a person confirms, which takes their platforms too\n", r.Indexes)
}
}
+5 -1
View File
@@ -239,7 +239,11 @@ func markUnresolvable(err error) error {
// still names anything foreseen is refused, said, and never sent.
func composeSend(ctx context.Context, open *stores, node string) (sendable, error) {
declared, _, err := composeSendValidated(ctx, open, node)
return declared, err
if err != nil {
return declared, err
}
// Never a declaration naming an image the store does not hold (novox/hq issue 1742): that module is left out.
return withoutMissingImages(ctx, open, node, declared), nil
}
// composeSendValidated is composeSend and what the node-engine's validator refuses of the declaration it composed, for the
@@ -1,68 +0,0 @@
//go:build integration
package main
import (
"context"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A member of a group whose gate is the composed check's is asked of the build seat with no module: its repository's
// check alone (novox/hq ADR 0403 clause 1). With no word from mesh-delivery, or one that does not say the gate is
// composed, it is checked alone with its gate (clause 6).
func askedForMember(t *testing.T, answer []groupsLine, answerErr error) link.CheckRequest {
t.Helper()
open := aMesh(t)
ctx := t.Context()
js, _ := aBuildQueue(t)
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
was := groupsAsked
groupsAsked = func(context.Context, bool) ([]groupsLine, error) { return answer, answerErr }
t.Cleanup(func() { groupsAsked = was })
p := link.PullUpdated{Owner: "novox", Repo: "mesh-controller", Number: 346, Base: "main", Head: "feat/x", Commit: headController,
Paths: []string{"main.go"}, ModuleDirsSaid: true, MergeCheck: true, MergeCheckSaid: true}
if err := (following{open: open, oneShot: true}).PullUpdated(ctx, p); err != nil {
t.Fatal(err)
}
q, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil || len(q.Asks) != 1 {
t.Fatalf("asks %d: %v", len(q.Asks), err)
}
return *q.Asks[0].Request.Check
}
func TestAComposedMembersCheckIsItsRepositoryChecksAlone(t *testing.T) {
line := group("feat/x", member("novox/mesh-controller", 346, headController, "main", true), member("novox/mesh-catalog", 350, headHost, "main", true))
line.ComposedGate = true
c := askedForMember(t, []groupsLine{line}, nil)
if len(c.Modules) != 0 || len(c.New) != 0 || len(c.Dependents) != 0 || !c.Composed {
t.Fatalf("the member's check is asked as %+v; want no module and composed", c)
}
}
func TestAMemberOfAGroupOfAnOlderMeshDeliveryKeepsItsOwnGate(t *testing.T) {
line := group("feat/x", member("novox/mesh-controller", 346, headController, "main", true), member("novox/mesh-catalog", 350, headHost, "main", true))
c := askedForMember(t, []groupsLine{line}, nil)
if !strings.Contains(strings.Join(c.Modules, ","), "mesh-controller") || c.Composed {
t.Fatalf("an older mesh-delivery's member is asked as %+v; want its own gate", c)
}
}
func TestAMemberWhoseGroupCannotBeLearnedIsCheckedAloneWithItsGate(t *testing.T) {
c := askedForMember(t, nil, context.DeadlineExceeded)
if !strings.Contains(strings.Join(c.Modules, ","), "mesh-controller") || c.Composed ||
!strings.Contains(c.SiblingsUnknown, "reads no siblings") {
t.Fatalf("the member is asked as %+v; want its gate and why its siblings are unknown", c)
}
}
@@ -1,136 +0,0 @@
package main
import (
"context"
"encoding/json"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A delivery group's gate is the composed check's, and its members' own checks run their repository's check alone
// (novox/hq ADR 0403 clauses 1 and 7). Pure: no store, no bus.
// A member of a group whose line says composed_gate is a composed member; one whose group says nothing of it (an older
// mesh-delivery), one of a group of a single pull request and a lone pull request are not.
func TestAMemberOfAComposedGroupIsKnownByTheGroupsLine(t *testing.T) {
two := func(composed bool) groupsLine {
l := group("feat/x", member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))
l.ComposedGate = composed
return l
}
p := link.PullUpdated{Owner: "novox", Repo: "mesh-host", Number: 114, Base: "main", Head: "feat/x", Commit: headHost}
recheck := link.PullUpdated{Owner: "novox", Repo: "mesh-host", Number: 114, Base: "main", Commit: headHost}
for name, c := range map[string]struct {
p link.PullUpdated
lines []groupsLine
want bool
}{
"composed": {p, []groupsLine{two(true)}, true},
"a recheck, by its number": {recheck, []groupsLine{two(true)}, true},
"an older mesh-delivery": {p, []groupsLine{two(false)}, false},
"another group": {p, []groupsLine{group("feat/y", member("novox/mesh-host", 1, headHost, "main", true))}, false},
"no group": {p, nil, false},
} {
if got := composedMember(c.p, c.lines); got != c.want {
t.Errorf("%s: composed %v, want %v", name, got, c.want)
}
}
alone := group("feat/x", member("novox/mesh-host", 114, headHost, "main", true))
alone.ComposedGate = true
if composedMember(p, []groupsLine{alone}) {
t.Error("a group of a single pull request is composed")
}
}
// A composed member's verdict never says its gate passed: the gate layer, and the verdict that is the gate's, read
// "composed"; a failure or an error stays what it was.
func TestAComposedMembersVerdictHasNoPassingGate(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114, Composed: true}
pass := link.BuildResult{ID: "b", Ref: headHost, Checked: asked, Check: &link.CheckOutcome{Verdict: "pass", Summary: "no module touched",
Gate: &link.CheckLayer{Verdict: "pass", Summary: "no module touched"}, RepoCheck: &link.CheckLayer{Verdict: "pass", Summary: "ok"}}}
c := checkedOf(pass)
if c.Gate == nil || c.Gate.Verdict != "composed" || c.Verdict != "composed" || c.RepoCheck == nil || c.RepoCheck.Verdict != "pass" {
t.Fatalf("the member's verdict says %+v gate %+v", c, c.Gate)
}
asked.Composed = false
if c := checkedOf(pass); c.Gate.Verdict != "pass" || c.Verdict != "pass" {
t.Fatalf("a member of no composed group has its own gate: %+v", c.Gate)
}
asked.Composed = true
failing := pass
failing.Check = &link.CheckOutcome{Verdict: "fail", Summary: "conflict", Gate: &link.CheckLayer{Verdict: "fail", Summary: "conflict"},
RepoCheck: &link.CheckLayer{Verdict: "fail", Summary: "conflict"}}
if c := checkedOf(failing); c.Gate.Verdict != "fail" || c.Verdict != "fail" {
t.Fatalf("a failure was read as composed: %+v", c.Gate)
}
broken := link.BuildResult{ID: "b", Ref: headHost, Checked: asked, Failed: "the facts snapshot cannot be read"}
if c := checkedOf(broken); c.Verdict != "error" || c.Gate.Verdict != "error" {
t.Fatalf("an error was read as composed: %+v", c)
}
}
// The composed verdict on the bus carries every member, each with its trunk commit, for the forge's holder to set each
// member's mesh/merge-gate from it.
func TestTheComposedVerdictCarriesEveryMember(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114, Group: "feat/x",
Members: []link.GroupMember{{Owner: "novox", Repo: "mesh-controller", Number: 346, Ref: headController}}}
c := checkedOf(link.BuildResult{ID: "g", Ref: headHost, Checked: asked, Check: &link.CheckOutcome{Verdict: "pass", Summary: "ok",
Trunk: "aaaaaaaaaaaa", Members: []link.CheckedMember{{Owner: "novox", Repo: "mesh-controller", Commit: headController[:12], Trunk: "bbbbbbbbbbbb"}}}})
body, _ := json.Marshal(c)
for _, want := range []string{`"group":"feat/x"`, `"members":[{"owner":"novox","repo":"mesh-host","number":114,"commit":"` + headHost + `","trunk":"aaaaaaaaaaaa"}`,
`{"owner":"novox","repo":"mesh-controller","number":346,"commit":"` + headController + `","trunk":"bbbbbbbbbbbb"}`} {
if !strings.Contains(string(body), want) {
t.Fatalf("the composed verdict lacks %s:\n%s", want, body)
}
}
}
// Members whose manifests order their modules in a cycle fail the composed check, naming every module and edge, and no
// ask is made of the build seat.
func TestAGroupInACycleFailsItsComposedCheckWithoutABuild(t *testing.T) {
members := []orderMember{
{ID: "novox/x@" + headHost[:12], Repository: "novox/x", Base: "main", Head: headHost, Number: 7, Paths: []string{"module.json"}},
{ID: "novox/y@" + headController[:12], Repository: "novox/y", Base: "main", Head: headController, Number: 9, Paths: []string{"module.json"}},
}
reach := map[string]orderReach{members[0].ID: {Moved: []string{"mx"}}, members[1].ID: {Moved: []string{"my"}}}
was, wasParent := manifestAt, parentOf
t.Cleanup(func() { manifestAt, parentOf = was, wasParent })
parentOf = func(context.Context, string, string, string) (string, error) { return "", nil }
manifestAt = func(_ context.Context, _, repo, _, ref string) (catalogue.Manifest, bool, error) {
if ref == "main" {
return catalogue.Manifest{}, false, nil // what the pull requests merge into holds no such manifest
}
if repo == "x" {
return catalogue.Manifest{Module: "mx", DefinesSeats: []catalogue.SeatDeclaration{{Name: "s", Serves: []catalogue.Verb{{Name: "a"}}}},
Invokes: []string{"seat:s.b"}}, true, nil
}
return catalogue.Manifest{Module: "my", DefinesSeats: []catalogue.SeatDeclaration{{Name: "s", Serves: []catalogue.Verb{{Name: "b"}}}},
Invokes: []string{"seat:s.a"}}, true, nil
}
c, cyclic, err := cycleVerdict(context.Background(), "feat/cycle", members, reach, []inventory.Edge(nil), nil)
if err != nil || !cyclic {
t.Fatalf("no cycle found: %v", err)
}
if c.Verdict != "fail" || c.Group != "feat/cycle" || len(c.Members) != 2 || c.Gate == nil || c.Gate.Verdict != "fail" {
t.Fatalf("the verdict is %+v", c)
}
for _, w := range []string{"mx", "my", "s.a", "s.b"} {
if !strings.Contains(c.Summary, w) {
t.Errorf("the cycle does not name %s: %s", w, c.Summary)
}
}
// Members with no edge between them are no cycle.
manifestsReadMu.Lock()
clear(manifestsRead)
manifestsReadMu.Unlock()
manifestAt = func(context.Context, string, string, string, string) (catalogue.Manifest, bool, error) {
return catalogue.Manifest{}, false, nil
}
if _, cyclic, err := cycleVerdict(context.Background(), "feat/cycle", members, reach, nil, nil); err != nil || cyclic {
t.Fatalf("a cycle where there is none: %v %v", cyclic, err)
}
}
@@ -0,0 +1,89 @@
//go:build integration
package main
import (
"encoding/json"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// 1840/AC5 (ADR 0507 §5): the source of the declared-packages mirror composes each node and names, for a package two modules
// declare and one declared absent, the modules that declare it; the same records give the same bytes on a second pass, so
// the mirror writes nothing; a node with no package has a key that says so.
func TestTheDeclaredPackagesMirrorNamesWhatEachNodesModulesDeclare(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, catalogue.Manifest{Module: "viewer", Version: "1", Resources: []map[string]any{
{"id": "pkg-curl", "type": "package", "package": "curl"},
{"id": "pkg-nano", "type": "package", "package": "nano", "absent": true}}})
register(t, open, catalogue.Manifest{Module: "fetcher", Version: "1", Resources: []map[string]any{
{"id": "pkg-curl", "type": "package", "package": "curl"}}})
for _, module := range []string{"viewer", "fetcher"} {
if _, err := assign(ctx, open, "anchor", module); err != nil {
t.Fatalf("assign anchor %s: %v", module, err)
}
}
source := declaredPackagesMirror(open)
first, err := source(ctx)
if err != nil {
t.Fatal(err)
}
var got inventory.DeclaredPackagesMirrored
if err := json.Unmarshal(first["anchor"], &got); err != nil {
t.Fatalf("anchor has no value: %v", err)
}
// Every node also runs the mesh's own modules, which declare packages of their own: curl and nano are looked
// up by name, and no package names viewer or fetcher but those they declare.
byName := map[string]inventory.PackageDeclared{}
for _, p := range got.Packages {
byName[p.Package] = p
}
same := func(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
for _, p := range []inventory.PackageDeclared{{Package: "curl", Modules: []string{"fetcher", "viewer"}},
{Package: "nano", Modules: []string{"viewer"}, Absent: true}} {
if g, ok := byName[p.Package]; !ok || g.Absent != p.Absent || !same(g.Modules, p.Modules) {
t.Errorf("anchor declares %s as %+v, want %+v (all: %+v)", p.Package, g, p, got.Packages)
}
}
for _, p := range got.Packages {
if p.Package == "curl" || p.Package == "nano" {
continue
}
for _, m := range p.Modules {
if m == "viewer" || m == "fetcher" {
t.Errorf("%s is said to be declared by %s, which does not declare it", p.Package, m)
}
}
}
var laptop inventory.DeclaredPackagesMirrored
if err := json.Unmarshal(first["laptop"], &laptop); err != nil || laptop.Node != "laptop" || laptop.Packages == nil {
t.Errorf("a node assigned neither module has no value of its own: %s (%v)", first["laptop"], err)
}
for _, p := range laptop.Packages {
if p.Package == "curl" || p.Package == "nano" {
t.Errorf("laptop is assigned neither module and declares %+v", p)
}
}
second, err := source(ctx)
if err != nil {
t.Fatal(err)
}
for node, value := range first {
if string(second[node]) != string(value) {
t.Errorf("the value of %s changed between two passes over the same records:\n%s\n%s", node, value, second[node])
}
}
}
+9 -200
View File
@@ -815,20 +815,21 @@ func deliveryPlanDocument(m orderMember, entries []inventory.Entry, read map[str
// delivery walks [-n 50] [--plan <id>]
func deliveryCommand(ctx context.Context, args []string) error {
if len(args) == 0 {
return errors.New("delivery plan|order|check|go|stop|walks")
return errors.New("delivery order|go|stop|walks")
}
sub, rest := args[0], args[1:]
if sub == "plan" {
// Removed (ADR 0450 §8; issue 1318): refused by name before any flag is read, naming its replacement.
return removedByName("delivery-plan")
}
if sub == "check" {
// Removed (ADR 0503 §3, ADR 0527; issue 1318): the checker judges every head, so the controller asks nothing.
return removedByName("delivery-check")
}
set := flag.NewFlagSet("delivery "+sub, flag.ContinueOnError)
repository := set.String("repository", "", "owner/repository")
base := set.String("base", "main", "the branch it merges into")
head := set.String("head", "", "the commit at hand")
paths := set.String("paths", "", "the files it changes, comma-separated")
moduleDirs := set.String("module-dirs", "", "the directories holding a module.json at the head, comma-separated")
removed := set.String("removed", "", "the files it deletes, comma-separated")
membersJSON := set.String("members", "", "a group's members, as JSON")
membersFile := set.String("members-file", "", "a file of a group's members, as JSON: the same as --members, for a list "+
"too long for an argument (novox/hq issue 1272)")
group := set.String("group", "", "a delivery group's id")
ready := set.Bool("ready", false, "the head's pull request is marked ready: its check is queued before every other check "+
"(novox/hq ADR 0406 §3)")
by := set.String("by", catalogue.DeliverySeat, "who says it")
@@ -872,19 +873,6 @@ func deliveryCommand(ctx context.Context, args []string) error {
return enc.Encode(v)
}
switch sub {
case "plan":
if *repository == "" || *head == "" && *paths == "" {
return errors.New("delivery plan --repository owner/repo --head <commit> --paths a,b")
}
m := changeMember(*repository, *base, *head, splitList(*paths), splitList(*removed), moduleDirsOf(*moduleDirs))
entries, read, edges, err := theGraph(ctx, inv)
if err != nil {
return err
}
return answer(deliveryPlanDocument(m, entries, read, edges, func(module string) (inventory.Upgrade, bool) {
u, err := inv.UpgradeOf(ctx, module)
return u, err == nil
}))
case "order":
if len(members) == 0 {
return errors.New("delivery order --members <json>: a group has members")
@@ -898,27 +886,6 @@ func deliveryCommand(ctx context.Context, args []string) error {
return fmt.Errorf("what the members say of their manifests cannot be read: %w", err)
}
return answer(orderOf(members, reachOfMembers(members, entries, read, edges), edges, changes))
case "check":
if *group == "" && len(members) == 1 {
// One head, checked again as the forge's announcement would have it (a recheck): the controller's
// own path for a pull request, run because the delivery's owner asked.
return recheckOneHead(members[0].ID, answer, func() (string, error) {
var asked string
err := sayingOnTheBus(ctx, inv, func() (err error) {
asked, err = (following{open: open, oneShot: true}).pullUpdated(ctx, members[0].pull(), os.Stderr)
return err
})
return asked, err
})
}
if *group == "" || len(members) < 2 {
return errors.New("delivery check --group <id> --members <json> (two heads or more), or --members <one head>")
}
id, err := askGroupCheck(ctx, open, *group, members)
if err != nil {
return err
}
return answer(map[string]any{"asked": id, "group": *group})
case "go":
if len(positionals) != 1 {
return errors.New("delivery go <plan> [--by <who>] [--why <text>]")
@@ -1093,133 +1060,6 @@ func theGraph(ctx context.Context, inv *inventory.Inventory) ([]inventory.Entry,
return entries, read, edges, nil
}
// cycleVerdict is the failing verdict of a group whose modules are ordered in a cycle, and whether there is one: the
// group's composed check, said without a build. The manifests' change is read from the forge; an answer it cannot give
// is an error, never read as no cycle.
func cycleVerdict(ctx context.Context, group string, members []orderMember, reach map[string]orderReach,
edges []inventory.Edge, entries []inventory.Entry) (link.Checked, bool, error) {
changes, err := changesOf(ctx, changeSourcesOf(members, entries, reach))
if err != nil {
return link.Checked{}, false, fmt.Errorf("what the group's members say of their manifests cannot be read: %w", err)
}
var moved []string
for _, m := range members {
moved = appendNew(moved, reach[m.ID].Moved...)
}
said := orderEdges(moved, changes, edges).CycleSaid()
if said == "" {
return link.Checked{}, false, nil
}
first := members[0].pull()
c := link.Checked{ID: link.NewBuildID(time.Now()), Owner: first.Owner, Repo: first.Repo, Number: first.Number,
Commit: first.Commit, Verdict: "fail", Summary: said, Report: said, Group: group,
Gate: &link.CheckLayer{Verdict: "fail", Summary: said}}
for _, m := range members {
if group == "" {
break // a lone pull request's verdict names no group and no member
}
p := m.pull()
c.Members = append(c.Members, link.CheckedMember{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Commit: p.Commit,
Trunk: twelve(p.BaseSHA)})
}
return c, true, nil
}
// twelve is a commit as twelve characters.
func twelve(c string) string {
if len(c) > 12 {
return c[:12]
}
return c
}
// groupPrimary is the head a group's composed check is run from: the one moving the controller, which then
// judges the group by itself; else the one moving the node-engine, whose validator judges; else the first.
func groupPrimary(members []orderMember, reach map[string]orderReach) int {
for _, want := range []string{"mesh-controller", "mesh-host"} {
for i, m := range members {
if slices.Contains(reach[m.ID].Moved, want) {
return i
}
}
}
return 0
}
// askGroupCheck asks the build seat for a group's composed check: every head laid over the mesh in turn,
// judged as one future state (novox/hq ADR 0239). The verdict comes back as `checked` with the group's id,
// for mesh-delivery; the forge's holder sets no status from it.
func askGroupCheck(ctx context.Context, open *stores, group string, members []orderMember) (string, error) {
entries, read, edges, err := theGraph(ctx, open.inventory)
if err != nil {
return "", err
}
reach := reachOfMembers(members, entries, read, edges)
// **A cycle of edges fails the group's composed check, naming every module and edge** (novox/hq ADR 0419 clause 6),
// before the build seat is asked: no composition of the heads has an order to be sent in.
if failing, cyclic, err := cycleVerdict(ctx, group, members, reach, edges, entries); err != nil {
return "", err
} else if cyclic {
sayChecked(ctx, os.Stderr, failing)
return failing.ID, nil
}
primary := groupPrimary(members, reach)
p := members[primary].pull()
scope := pullScope(p, entries, read, edges)
// The gate runs over what any member moves; a judge from the primary alone.
for _, m := range members {
r := reach[m.ID]
scope.Modules = appendNew(scope.Modules, r.Moved...)
scope.Dependents = appendNew(scope.Dependents, r.Dependents...)
}
request, err := checkRequestFor(ctx, open, p, scope, entries)
if err != nil {
return "", err
}
request.Check.Group = group
// A group's check is queued as ready when any member's pull request is marked ready.
request.Check.Ready = slices.ContainsFunc(members, func(m orderMember) bool { return m.Ready })
world, err := restOfTheMesh(ctx, open, shelfOf(entries), "")
if err != nil {
return "", err
}
for i, m := range members {
if i == primary {
continue
}
mp := m.pull()
// As the mesh clones a module built from it; a repository no module is built from, from the forge.
source := inventory.Source{Seat: gitSeat, Repository: mp.Owner + "/" + mp.Repo}
for _, e := range entries {
if !e.Provided && sameRepository(e.Source.Repository, link.SourceMoved{Owner: mp.Owner, Repo: mp.Repo}) {
source = e.Source
break
}
}
url := source.Repository
if source.Seat != "" {
url, err = clonedFromSeat(world, source.Seat, source.Repository)
}
if err != nil {
return "", fmt.Errorf("%s cannot be cloned for the group's check: %w", m.ID, err)
}
request.Check.Members = append(request.Check.Members, link.GroupMember{Owner: mp.Owner, Repo: mp.Repo,
Number: mp.Number, Repository: url, Ref: mp.Commit, Paths: mp.Paths, Base: mp.Base})
}
seat := buildSeatHeld(ctx)
ask, err := askOverOn(seat)
if err != nil {
return "", err
}
defer ask.Close()
if err := ask.Ask(ctx, request); err != nil {
return "", err
}
fmt.Fprintf(os.Stderr, "group %s: asked %s to check %d head(s) composed together, as %s\n", group, seat,
len(members), request.ID)
return request.ID, nil
}
// appendNew appends what is not there yet.
func appendNew(to []string, items ...string) []string {
for _, i := range items {
@@ -1230,15 +1070,6 @@ func appendNew(to []string, items ...string) []string {
return to
}
// shelfOf is the catalogue's manifests by name.
func shelfOf(entries []inventory.Entry) map[string]catalogue.Manifest {
shelf := map[string]catalogue.Manifest{}
for _, e := range entries {
shelf[e.Manifest.Module] = e.Manifest
}
return shelf
}
// sayPlanMoved says a walk kept in a new state as the controller's `plan-moved`, the plan whole: what the
// delivery it walks reads its steps from. Never in the way of the walk: said beside it, and a save that could
// not be said is logged — the delivery's owner, which also asks for the walks it follows, finds it by
@@ -1380,25 +1211,3 @@ func overNATS(conn *nats.Conn) (link.Bus, error) {
}
return link.OverNATS{Conn: conn, JS: js}, nil
}
// recheckAnswer is what `delivery check` answers for one head: the id of the check asked of the build seat (empty when
// the controller said the verdict itself and asked nothing), which mesh-delivery reads, and the head rechecked.
func recheckAnswer(asked, member string) map[string]any {
return map[string]any{"asked": asked, "rechecked": member}
}
// recheckOneHead runs a one-head `delivery check` and answers it as data. Standard output is the answer's JSON alone,
// or the seat verb reads no answer from it (novox/hq issue 1311): while the check runs, whatever it prints to standard
// output (the controller's own path prints its prose there, as do the stores it reads) goes to standard error.
func recheckOneHead(member string, answer func(any) error, ask func() (string, error)) error {
real := os.Stdout
os.Stdout = os.Stderr
asked, err := func() (string, error) {
defer func() { os.Stdout = real }()
return ask()
}()
if err != nil {
return err
}
return answer(recheckAnswer(asked, member))
}
@@ -1,85 +0,0 @@
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// `delivery check` with one head answers the id of the check it asked, as data (novox/hq issue 1311): the prose
// the check says goes to standard error, so standard output is the document alone and answerOf reads it.
func TestDeliveryCheckOfOneHeadAnswersTheRequestID(t *testing.T) {
argv := []string{"delivery", "check", "--members", "[]"}
doc, err := json.Marshal(recheckAnswer("check-42", "novox/x@abc"))
if err != nil {
t.Fatal(err)
}
// Seen failing first: the prose the command used to print on standard output before the document.
polluted := append([]byte("novox/x#7 (abc): asked the build seat as check-42\n"), doc...)
if got := answerOf(argv, polluted, "", true); got.Answer != nil {
t.Fatalf("prose before the document should leave no answer; got %v", got.Answer)
}
got := answerOf(argv, doc, "novox/x#7 (abc): asked the build seat as check-42\n", true)
m, ok := got.Answer.(map[string]any)
if !ok || m["asked"] != "check-42" || m["rechecked"] != "novox/x@abc" {
t.Fatalf("the answer is %v; want asked check-42 and rechecked novox/x@abc", got.Answer)
}
if !strings.Contains(got.Output, "asked the build seat") {
t.Fatalf("the prose is still said in the text: %q", got.Output)
}
}
// The text of the other callers does not change: sayChecked writes the same line to the writer it is given, and
// the controller's own path still gives PullUpdated standard output.
func TestSayCheckedWritesTheSameLineToItsWriter(t *testing.T) {
var w bytes.Buffer
sayChecked(context.Background(), &w, link.Checked{ID: "c1", Owner: "novox", Repo: "x", Number: 7, Commit: "0123456789abcdef",
Verdict: "pass", Summary: "fine"})
want := "c1: novox/x#7 at 01234567 checked on a machine that did not say which: gate PASS — fine; repository none\n"
if w.String() != want {
t.Fatalf("sayChecked said %q; want %q", w.String(), want)
}
}
// Whatever the check prints to standard output while it runs (checks.go namedForTheCheck, the inventory's pointer
// repair) must not reach the answer's standard output: it is exactly one JSON document (novox/hq issue 1311).
func TestOneHeadRecheckLeavesStandardOutputTheAnswerAlone(t *testing.T) {
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
e, ew, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
realOut, realErr := os.Stdout, os.Stderr
os.Stdout, os.Stderr = w, ew
runErr := recheckOneHead("novox/x@abc", func(v any) error { return json.NewEncoder(os.Stdout).Encode(v) },
func() (string, error) {
fmt.Println(" the earlier checks' builds could not be read, so the check builds")
fmt.Fprintln(os.Stdout, "repaired a pointer")
return "check-42", nil
})
os.Stdout, os.Stderr = realOut, realErr
w.Close()
ew.Close()
if runErr != nil {
t.Fatal(runErr)
}
var out, errOut bytes.Buffer
_, _ = out.ReadFrom(r)
_, _ = errOut.ReadFrom(e)
var doc map[string]any
if err := json.Unmarshal(out.Bytes(), &doc); err != nil || doc["asked"] != "check-42" || doc["rechecked"] != "novox/x@abc" {
t.Fatalf("standard output %q is not the answer alone (%v)", out.String(), err)
}
if !strings.Contains(errOut.String(), "repaired a pointer") {
t.Fatalf("the prose should be on standard error: %q", errOut.String())
}
}
+10 -1
View File
@@ -41,7 +41,7 @@ type stalledLine struct {
// waiting for the owner's word and what holds it (novox/hq issues 362, 529).
Walk string `json:"walk,omitempty"`
// Waiting are the merge checks a line of the state `unanswered` waits on, as mesh-delivery says each:
// "mesh/merge-gate pending since <UTC time>", "mesh/repo-check never set" (novox/hq issue 438).
// "mesh/check pending since <UTC time>", "mesh/check never set" (novox/hq issue 438).
Waiting []string `json:"waiting,omitempty"`
// Checks are the same merge checks as data, which the controller words in the operator's own time (novox/hq
// issue 443): a time inside a finished sentence cannot be said again in another zone. A mesh-delivery from before
@@ -58,8 +58,17 @@ type stalledLine struct {
// Severity is "urgent" for a group held or half merged and a refused member (novox/hq issues 1275, 1278); empty
// or anything else is a warning, as before.
Severity string `json:"severity,omitempty"`
// Needs is "new-head" for a delivery whose cure is its author's new head (novox/hq ADR 0505 §5), Author that author
// as the operator reads it ("the author of feat/x", with the session that pushed it when its pull request names one)
// and Branch its branch. A mesh-delivery from before says none, and the words are as before.
Needs string `json:"needs,omitempty"`
Author string `json:"author,omitempty"`
Branch string `json:"branch,omitempty"`
}
// needsNewHead is what a stalled line needs when its cure is its author's new head (ADR 0505 §5).
const needsNewHead = "new-head"
// severity is the condition's severity a stalled line asks: urgent when it says so, a warning otherwise.
func (l stalledLine) severity() conditions.Severity {
if l.Severity == string(conditions.Urgent) {
@@ -181,32 +181,3 @@ func TestATruncatedChangeIsReadAsChangingTheManifestsOfWhatItMoves(t *testing.T)
t.Fatalf("the member's change is read as %+v", src)
}
}
// A lone pull request's cycle fails its check, naming every module and edge, with no group and no members.
func TestALonePullRequestInACycleFailsWithNoGroup(t *testing.T) {
was, wasParent := manifestAt, parentOf
t.Cleanup(func() { manifestAt, parentOf = was, wasParent })
manifestsReadMu.Lock()
clear(manifestsRead)
manifestsReadMu.Unlock()
const head = "3333333333333333333333333333333333333333"
parentOf = func(context.Context, string, string, string) (string, error) { return "", nil }
manifestAt = func(_ context.Context, _, _, file, ref string) (catalogue.Manifest, bool, error) {
if ref == "main" {
return catalogue.Manifest{}, false, nil
}
if file == "modules/x/module.json" {
return catalogue.Manifest{Module: "mx", DefinesSeats: []catalogue.SeatDeclaration{{Name: "s", Serves: []catalogue.Verb{{Name: "a"}}}},
Invokes: []string{"seat:s.b"}}, true, nil
}
return catalogue.Manifest{Module: "my", DefinesSeats: []catalogue.SeatDeclaration{{Name: "s", Serves: []catalogue.Verb{{Name: "b"}}}},
Invokes: []string{"seat:s.a"}}, true, nil
}
m := orderMember{ID: "novox/r@" + head[:12], Repository: "novox/r", Base: "main", Head: head, Number: 5,
Paths: []string{"modules/x/module.json", "modules/y/module.json"}}
c, cyclic, err := cycleVerdict(context.Background(), "", []orderMember{m}, map[string]orderReach{m.ID: {Moved: []string{"mx", "my"}}}, nil, nil)
if err != nil || !cyclic || c.Verdict != "fail" || c.Group != "" || len(c.Members) != 0 || !strings.Contains(c.Summary, "s.a") ||
!strings.Contains(c.Summary, "my") {
t.Fatalf("the verdict is %+v (cyclic %v, %v)", c, cyclic, err)
}
}
@@ -59,6 +59,18 @@ func TestAMergeWaitsForItsDeliverysWordAndThePersonsWordWorksWithoutIt(t *testin
}
}
// heardOf has this mesh's own view of the plans take in the plan as the bucket holds it. A verb of the
// controller (deliveryCommand, plansCommand) opens an inventory of its own and writes the plan there; this
// mesh's inventory learns of the write from the bucket's watch, a moment later, and a loaded machine makes the
// moment long enough for advanceHeld to read the plan as it was, still waiting for its word, and ask nothing
// (issue 1399). Reading the plan from the bucket puts it into the view at once, with no wait to bound.
heardOf := func(id string) {
t.Helper()
if _, err := inv.PlanByID(ctx, id); err != nil {
t.Fatal(err)
}
}
// No holder on record: the merge starts its walk, as before.
p := merge("c1aaaaaaaa", "modules/app/index.ts")
if p.Waiting() || len(*asked) != 1 {
@@ -90,6 +102,7 @@ func TestAMergeWaitsForItsDeliverysWordAndThePersonsWordWorksWithoutIt(t *testin
if err := deliveryCommand(ctx, []string{"go", p.ID}); err == nil {
t.Fatal("a walk was let go twice")
}
heardOf(p.ID)
advanceHeld(ctx, open)
if len(*asked) != 2 {
t.Fatalf("the word did not start the walk: %v", *asked)
@@ -119,6 +132,7 @@ func TestAMergeWaitsForItsDeliverysWordAndThePersonsWordWorksWithoutIt(t *testin
if err := plansCommand(ctx, []string{"go", p.ID, "--why", "mesh-delivery is down"}); err != nil {
t.Fatal(err)
}
heardOf(p.ID)
advanceHeld(ctx, open)
got, _ = inv.PlanByID(ctx, p.ID)
if got.Waiting() || !strings.HasPrefix(got.Delivery.By, "a person") || len(*asked) < 4 {
-4
View File
@@ -136,10 +136,6 @@ func TestTheDeliveryVerbsComposeTheirCommands(t *testing.T) {
{"delivery-walks", map[string]any{}, []string{"delivery", "walks"}},
{"delivery-walks", map[string]any{"plan": "plan-1"}, []string{"delivery", "walks", "--plan", "plan-1"}},
{"delivery-order", map[string]any{"members": "[]"}, []string{"delivery", "order", "--members", "[]"}},
{"delivery-check", map[string]any{"group": "feat/x", "members": "[]"},
[]string{"delivery", "check", "--group", "feat/x", "--members", "[]"}},
{"delivery-plan", map[string]any{"repository": "novox/a", "paths": "x", "head": "c0"},
[]string{"delivery", "plan", "--repository", "novox/a", "--paths", "x", "--head", "c0"}},
{"plans", map[string]any{"go": "plan-1", "why": "down"}, []string{"plans", "go", "plan-1", "--why", "down"}},
} {
got, err := argvFor(c.verb, c.args)
-10
View File
@@ -35,16 +35,6 @@ func TestAMergeOfABaseTakesWhatStandsOnItAlong(t *testing.T) {
t.Fatalf("%s was rebuilt and stands on nothing that moved (%v)", n, names)
}
}
// The dependents come in base order when the merge orders them: the runtime, then shop, then
// the plugin that stands on shop.
ordered := orderByBases(append([]inventory.Entry{entry("mesh-tools")}, got...), against)
pos := map[string]int{}
for i, e := range ordered {
pos[e.Manifest.Module] = i
}
if !(pos["mesh-tools"] < pos["shop"] && pos["shop"] < pos["shop-plugin"]) {
t.Fatalf("not in base order: %v", ordered)
}
// Nothing moved: nothing follows.
if more := dependentsOf(nil, entries, against); len(more) != 0 {
t.Fatalf("with nothing moved, %d module(s) were rebuilt", len(more))
+1 -1
View File
@@ -669,7 +669,7 @@ func localDoctor(ctx context.Context) (*doctor, func(), error) {
if err != nil {
return nil, nil, err
}
js, err := dialTheBus()
js, err := aBus()
if err != nil {
open.Close()
return nil, nil, err
@@ -1,20 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
func TestADryRunsOutcomeIsTakenInByNothing(t *testing.T) {
err := builds{}.Built(t.Context(), link.BuildResult{
ID: "build-1", Repository: "ssh://forge/app.git", Ref: "unreviewed", Module: "app", DryRun: true,
Manifest: json.RawMessage(`{"module":"app","version":"1"}`),
})
if err != nil {
t.Fatalf("a dry run's outcome was not simply set aside: %v", err)
}
}
-29
View File
@@ -1,29 +0,0 @@
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// A dry run's outcome is looked at, never taken in (novox/hq issue 240). The daemon here holds no
// store at all, so anything that tried to record or register would fail rather than pass quietly.
// The mark survives the wire both ways: asked as a dry run, answered as one.
func TestTheDryRunMarkTravelsWithTheBuild(t *testing.T) {
raw, _ := json.Marshal(link.BuildRequest{ID: "build-1", Repository: "r", DryRun: true})
var asked link.BuildRequest
if err := json.Unmarshal(raw, &asked); err != nil || !asked.DryRun {
t.Fatalf("the request lost its dry-run mark: %s", raw)
}
raw, _ = json.Marshal(link.BuildResult{ID: "build-1", DryRun: true})
var answered link.BuildResult
if err := json.Unmarshal(raw, &answered); err != nil || !answered.DryRun {
t.Fatalf("the outcome lost its dry-run mark: %s", raw)
}
raw, _ = json.Marshal(link.BuildResult{ID: "build-2"})
if string(raw) != `{"id":"build-2","repository":"","on":""}` {
t.Fatalf("an ordinary outcome carries a dry-run mark: %s", raw)
}
}
-20
View File
@@ -12,7 +12,6 @@ import (
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// What the core's bounds are set from (novox/hq to-be 45 Phase 0).
@@ -24,25 +23,6 @@ import (
// controller as it hears each — a send's first report, a machine's next word, a plan leaving a tier, a
// build's outcome — and summarised here per machine, repository or module.
// recordBuildDuration measures one build from its ask to its outcome heard.
func recordBuildDuration(ctx context.Context, inv *inventory.Inventory, result link.BuildResult, asked time.Time) {
if asked.IsZero() || result.ID == "" {
return
}
subject := result.Module
if subject == "" {
subject = result.Repository
}
detail := "built"
if result.Failed != "" {
detail = "failed: " + firstLine(result.Failed)
}
if err := inv.RecordDuration(ctx, inventory.Duration{Kind: inventory.DurationBuild, Subject: subject,
Node: result.On, Ref: result.ID, Started: asked, Took: time.Since(asked), Detail: detail}); err != nil {
fmt.Fprintf(os.Stderr, "%s: how long it took could not be recorded: %v\n", result.ID, err)
}
}
// durationSummary is one subject's measurements of one kind.
type durationSummary struct {
Kind string `json:"kind"`
-12
View File
@@ -19,7 +19,6 @@ import (
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
"github.com/novox/mesh-controller/internal/artifacts"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -241,17 +240,6 @@ func gatherFacts(ctx context.Context, open *stores, busVersion string) (snapshot
scrub.Site(o.Site)
}
// What a merge check runs in and reads beside it, as the build seat would be asked for it.
if held, err := inv.Held(ctx); err == nil {
for language, reference := range builder.ToolchainsOf(held) {
if f.Versions.Toolchains == nil {
f.Versions.Toolchains = map[string]string{}
}
f.Versions.Toolchains[language] = catalogue.Recorded(reference)
}
} else {
return snapshot.Facts{}, err
}
for _, e := range entries {
if dir, core := coreModules[e.Manifest.Module]; core && !e.Provided && e.Source.Repository != "" {
for d, ref := range besideRefs(dir, current[e.Manifest.Module].Commit) {
@@ -1,84 +0,0 @@
//go:build integration
package main
import (
"context"
"errors"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/identity"
"github.com/novox/mesh-controller/internal/link"
)
// aForgeAndABuildSeat gives a test what a real ask of a build needs: the forge holding the git seat, assigned to anchor, so
// the clone URL composes; the identity store; the build seat's queue on a real bus; and the environment of the serving controller, which is
// never the controller's terminal (a verb's, in effect: startedAtTheTerminal is false).
func aForgeAndABuildSeat(t *testing.T, open *stores) *broker.JetStream {
t.Helper()
js, _ := aBuildQueue(t)
// The ask signs what it sends with the controller's identity, as the serving controller does.
identity.ForTest(t)
forge, err := catalogue.ParseManifest([]byte(`{"module":"gitea","version":"1",
"claims":[{"name":"git","scope":"mesh"}],"provides":[{"name":"git","scope":"mesh"}],
"serves":{"git":{"scheme":"http","port":3000}}}`))
if err != nil {
t.Fatal(err)
}
register(t, open, forge)
if _, err := assign(t.Context(), open, "anchor", "gitea"); err != nil {
t.Fatal(err)
}
t.Setenv(servedVar, "1")
return js
}
// 1146/AC9, through the real ask (askABuild, buildOneAsked, verbMayAsk) and a real bus: the first build of a module a
// merge adds on the operator's Approve is queued on the build seat; the same ask without that warrant, and the warrant's
// ask for a path the merge does not hold, are refused by the guard of a build through a verb.
func TestTheApprovedFirstBuildPassesTheGuardOfAVerbAndIsQueued(t *testing.T) {
const head, commit = "a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1", "0177ea7a00000000"
open := aMesh(t)
ctx := t.Context()
js := aForgeAndABuildSeat(t, open)
a := aDeliveryMergeOf(t, head, map[string]string{"novox/mesh-delivery@a1a1a1a1a1a1": head})
wasKept, wasAsk := keptDeliveryMerge, routerAskOf
keptDeliveryMerge = func(context.Context, string, string, int, string) (deliveryMerge, bool, error) {
return a.merge, true, nil
}
routerAskOf = func(context.Context, catalogueReader, string, string) (*asks.Ask, *asks.Warrant, error) {
return &a.ask, &a.warrant, nil
}
t.Cleanup(func() { keptDeliveryMerge, routerAskOf = wasKept, wasAsk })
// Without the warrant the guard refuses, whatever the repository holds.
_, err := askABuild(ctx, buildSource{Repository: "novox/mesh-delivery", Seat: "git"}, "", "main")
var policy *heldAtTheTerminal
if !errors.As(err, &policy) {
t.Fatalf("1146/AC9: a first build asked without the warrant was not refused: %v", err)
}
// And a path the merge does not hold is refused on the warrant.
warranted := withFirstBuildWarrant(ctx, "novox/mesh-delivery", []string{""}, "the Approve")
if _, err := askABuild(warranted, buildSource{Repository: "novox/mesh-delivery", Seat: "git"}, "other", "main"); !errors.As(err, &policy) {
t.Fatalf("1146/AC9: the warrant covered a path the merge does not hold: %v", err)
}
m := link.SourceMoved{Owner: "novox", Repo: "mesh-delivery", Base: "main", HeadSHA: head, Commit: commit, Number: 7,
MergedAt: time.Now().Add(-time.Hour).UTC().Format(time.RFC3339Nano), Paths: []string{"module.json", "main.go"},
ModuleDirs: []string{"."}, ModuleDirsSaid: true}
if err := (following{open: open}).SourceMoved(ctx, m); err != nil {
t.Fatal(err)
}
queue, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil {
t.Fatal(err)
}
if len(queue.Asks) != 1 {
t.Fatalf("1146/AC9: the build queue holds %+v, want the one first build", queue)
}
}
@@ -1,29 +0,0 @@
package main
import "testing"
// 1146/AC9: the warrant the controller checked covers exactly the repository on the git seat and the module paths the
// merge holds; nothing else passes verbMayAsk on it.
func TestTheFirstBuildWarrantCoversOnlyItsRepositoryAndItsModulePaths(t *testing.T) {
ctx := withFirstBuildWarrant(t.Context(), "novox/mesh-delivery", []string{"", "view"}, "the Approve")
w := firstBuildWarrantIn(ctx)
for _, c := range []struct {
source buildSource
path string
covers bool
}{
{buildSource{Repository: "novox/mesh-delivery", Seat: "git"}, "", true},
{buildSource{Repository: "novox/mesh-delivery", Seat: "git"}, "/view/", true},
{buildSource{Repository: "novox/mesh-delivery", Seat: "git"}, "other", false},
{buildSource{Repository: "agent/mesh-delivery", Seat: "git"}, "", false},
{buildSource{Repository: "novox/mesh-delivery"}, "", false},
{buildSource{Repository: "novox/mesh-delivery", Seat: "elsewhere"}, "", false},
} {
if got := w.covers(c.source, c.path); got != c.covers {
t.Errorf("1146/AC9: %v at %q covered = %v, want %v", c.source, c.path, got, c.covers)
}
}
if firstBuildWarrantIn(t.Context()).covers(buildSource{Repository: "novox/mesh-delivery", Seat: "git"}, "") {
t.Error("1146/AC9: a context with no warrant covers an ask")
}
}
+77 -21
View File
@@ -5,6 +5,7 @@ import (
"encoding/json"
"errors"
"fmt"
"regexp"
"strings"
"time"
@@ -46,7 +47,7 @@ func (f following) firstBuilds(ctx context.Context, m link.SourceMoved, entries
case len(added) == 0:
return false, nil
}
if err := askFirstBuilds(ctx, inv, m, now, added, why); err != nil {
if err := askFirstBuilds(ctx, f.open, m, now, added, why); err != nil {
return true, notNow(err)
}
return true, nil
@@ -67,27 +68,35 @@ func keepFirstMerge(ctx context.Context, inv *inventory.Inventory, m link.Source
return err
}
// askFirstBuilds keeps the merge, asks the builds, and marks the merge as no longer waiting for them.
func askFirstBuilds(ctx context.Context, inv *inventory.Inventory, m link.SourceMoved, now time.Time, added []string,
// askFirstBuilds keeps the merge as answered and registers its modules from the builder's done keys (novox/hq ADR 0501
// §3, §6): nothing asks their first builds. The merge kept is the registration's warrant (approvedMove reads it by the
// commit the indexer's trunk value names); a module whose key is not done yet is registered when the key's write wakes
// the take-in (new_modules.go).
func askFirstBuilds(ctx context.Context, open *stores, m link.SourceMoved, now time.Time, added []string,
why string) error {
inv := open.inventory
repository := m.Owner + "/" + m.Repo
if err := keepFirstMerge(ctx, inv, m, now, false); err != nil {
return err
}
fmt.Printf("%s merged into %s (%.8s) on the operator's Approve (%s): the mesh builds nothing from it yet, so the "+
"first build of %s is asked\n", repository, m.Base, m.Commit, why, strings.Join(added, ", "))
// The ask carries the warrant just checked, for this repository and these module paths alone: the guard of a build
// through a verb (verbMayAsk) lets exactly that through.
paths := make([]string, 0, len(added))
for _, dir := range added {
if dir == "." {
dir = ""
}
paths = append(paths, dir)
fmt.Printf("%s merged into %s (%.8s) on the operator's Approve (%s): the mesh builds nothing from it yet, so %s "+
"are registered from the builder's done keys, asking no build\n", repository, m.Base, m.Commit, why,
strings.Join(added, ", "))
// Answered: it no longer waits, so the word's arrival again does nothing twice.
if err := inv.AnswerMerge(ctx, repository, m.Commit, inventory.WaitingRecord, false); err != nil {
return err
}
askBuilds(withFirstBuildWarrant(ctx, repository, paths, why), inv, m, buildSource{Repository: repository, Seat: gitSeat}, added)
// Answered: it no longer waits, so the word's arrival again asks nothing twice.
return inv.AnswerMerge(ctx, repository, m.Commit, inventory.WaitingRecord, false)
modules, err := newModulesOf(ctx, repository)
switch {
case err != nil:
// Said, not failed: the merge is answered, and the look at every module when a controller starts, or the next
// write of the builder or the indexer for one of them, registers them.
fmt.Printf(" the indexer's modules of %s cannot be read now (%v): they are registered when the builder's next "+
"write of one wakes the take-in\n", repository, err)
case len(modules) > 0:
inBackground(func() { takeNewModules(context.WithoutCancel(ctx), open, modules...) })
}
return nil
}
// resumeFirstBuilds runs firstBuilds for the first merges waiting for the word mesh-delivery just announced for a pull
@@ -102,7 +111,7 @@ func (f following) resumeFirstBuilds(ctx context.Context, d deliveryMerge) error
}
for _, w := range waiting {
k := announcedOf(w)
if !k.FirstBuild || !strings.EqualFold(k.Owner+"/"+k.Repo, d.Owner+"/"+d.Repo) || k.Number != d.Number ||
if !k.FirstBuild || len(k.Held) > 0 || !strings.EqualFold(k.Owner+"/"+k.Repo, d.Owner+"/"+d.Repo) || k.Number != d.Number ||
k.HeadSHA != d.HeadSHA {
continue
}
@@ -125,7 +134,7 @@ func (f following) resumeFirstBuilds(ctx context.Context, d deliveryMerge) error
case err != nil:
return notNow(err)
case len(added) > 0:
if err := askFirstBuilds(ctx, inv, k.SourceMoved, time.Now().UTC(), added, why); err != nil {
if err := askFirstBuilds(ctx, f.open, k.SourceMoved, time.Now().UTC(), added, why); err != nil {
return notNow(err)
}
}
@@ -172,6 +181,29 @@ func firstBuildsOf(ctx context.Context, inv catalogueReader, m link.SourceMoved,
return nil, "", err
}
// heldFirstBuild says a first build was refused because its manifest uses a word the running build-agent does not read,
// and its merge is kept waiting for a build-agent that reads it (novox/hq ADR 0482 §5, issue 1173). Read from a merge kept
// before R4e, never written since: nothing asks a first build (ADR 0501 §6), and a controller that starts answers such a
// merge and registers its modules from their keys (answerHeldMerges).
type heldFirstBuild struct {
Module string `json:"module"`
Path string `json:"path"`
Word string `json:"word"`
At time.Time `json:"at"`
}
// unknownWord is how the strict reading of a manifest says a word is not one it knows: encoding/json's own words, which
// the build-agent's answer carries whole. The build's answer has no typed field for it (Failed is the builder's words).
var unknownWord = regexp.MustCompile(`json: unknown field "([^"]+)"`)
// refusedWord is the word a build's failure says the reader did not know; empty when it says none.
func refusedWord(failed string) string {
if m := unknownWord.FindStringSubmatch(failed); m != nil {
return m[1]
}
return ""
}
// probeFirstBuildsID is the probe that reads the first merges waiting for mesh-delivery's word.
const probeFirstBuildsID = "D18"
@@ -203,9 +235,10 @@ func firstBuildObservations(waiting []inventory.BatchedMerge, now time.Time) []c
Headline: fmt.Sprintf("The first build of %s is not asked: no Approve was announced", repository),
Explanation: fmt.Sprintf("The merge %.8s into %s adds modules the mesh builds nothing from yet. It registers them "+
"only when mesh-delivery announces that it merged the pull request on your Approve of that head, and no such "+
"announcement has come in %s. The merge is kept, and the builds are asked the moment the announcement is kept.",
"announcement has come in %s. The merge is kept, and the modules are registered from the builder's keys the moment "+
"the announcement is kept.",
k.Commit, repository, for_),
Resolved: "mesh-delivery announces the merge, or a hand `build` registers the modules",
Resolved: "mesh-delivery announces the merge, or the operator registers the modules at the controller's terminal",
})
}
return out
@@ -217,5 +250,28 @@ func probeFirstBuilds(ctx context.Context, d *doctor) ([]conditions.Observation,
if err != nil {
return nil, err
}
return firstBuildObservations(waiting, time.Now().UTC()), nil
now := time.Now().UTC()
return append(firstBuildObservations(waiting, now), newModuleObservations(now)...), nil
}
// answerHeldMerges answers every merge a controller of before held for a build-agent that reads its new modules'
// manifests (issue 1173): nothing asks their first builds any more, and the look at every module when this controller
// starts registers them from their keys (ADR 0501 §6, §7).
func answerHeldMerges(ctx context.Context, inv *inventory.Inventory) {
waiting, err := inv.WaitingMerges(ctx)
if err != nil {
fmt.Printf("the merges held for a newer build-agent could not be read: %v\n", err)
return
}
for _, w := range waiting {
if k := announcedOf(w); len(k.Held) > 0 {
if err := inv.AnswerMerge(ctx, w.Repository, w.Commit, inventory.WaitingRecord, false); err != nil {
fmt.Printf("the merge %.8s of %s held for a newer build-agent could not be answered: %v\n", w.Commit,
w.Repository, err)
continue
}
fmt.Printf("the merge %.8s of %s was held for a newer build-agent; its modules are registered from the "+
"builder's keys instead\n", w.Commit, w.Repository)
}
}
}
@@ -5,71 +5,48 @@ package main
import (
"context"
"encoding/json"
"slices"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/catalogue"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// 1146/AC7, whole against a real bus and the store, in both orders of the two events (novox/hq ADR 0432, issue 1146):
// mesh-delivery #7 merged on the operator's Approve into a repository the mesh builds nothing from. The forge's merge,
// heard as the controller hears it (SourceMoved), and mesh-delivery's word, published on the subject the controller
// consumes with the router's record of the ask in its bucket, come in either order; each way the first build of the
// module the merge commit holds is asked exactly once, its outcome registers the module, and it is in the catalogue.
// Without the Approve in the record nothing is asked. Store-backed: skips without one.
func TestAMergeOnTheApproveIntoANewRepositoryAsksTheFirstBuildAndItsOutcomeRegistersTheModule(t *testing.T) {
// 1146/AC7 on keys (ADR 0501 §3, §6; issue 1851, A5), whole against a real bus and the store, in both orders of the two
// events: mesh-delivery #7 merged on the operator's Approve into a repository the mesh builds nothing from. The forge's
// merge, heard as the controller hears it (SourceMoved), and mesh-delivery's word, published on the subject the
// controller consumes with the router's record of the ask in its bucket, come in either order; each way the module the
// merge commit holds is registered from the builder's done key of it, and nothing asks a build. Without the Approve in
// the record it is not registered. Store-backed: skips without one.
func TestAMergeOnTheApproveIntoANewRepositoryRegistersItsModuleFromItsKey(t *testing.T) {
const head, commit = "a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1", "0177ea7a00000000"
for _, c := range []struct {
what string
wordFirst bool
approved bool // the router's record holds the operator's Approve of the merged head
wantRegiter bool
what string
wordFirst bool
approved bool // the router's record holds the operator's Approve of the merged head
wantRegister bool
}{
{what: "the forge's merge before mesh-delivery's word", approved: true, wantRegiter: true},
{what: "mesh-delivery's word before the forge's merge", wordFirst: true, approved: true, wantRegiter: true},
{what: "the forge's merge before mesh-delivery's word", approved: true, wantRegister: true},
{what: "mesh-delivery's word before the forge's merge", wordFirst: true, approved: true, wantRegister: true},
{what: "a word whose record holds no Approve", wordFirst: true},
} {
t.Run(c.what, func(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
// The real ask, guard and queue included: only its answers are kept, so the build's outcome can be matched.
aForgeAndABuildSeat(t, open)
// Written by the controller's merge path on its own goroutine, read by the test: under one lock.
var mu sync.Mutex
var asks [][3]string
var ids []string
asked := func() [][3]string { mu.Lock(); defer mu.Unlock(); return slices.Clone(asks) }
realAsk := askABuild
askABuild = func(c context.Context, source buildSource, path, ref string) (string, error) {
id, err := realAsk(c, source, path, ref)
if err == nil {
mu.Lock()
asks = append(asks, [3]string{source.Repository, path, ref})
ids = append(ids, id)
mu.Unlock()
}
return id, err
}
t.Cleanup(func() { askABuild = realAsk })
noBuildAsked(t)
b := aBuilderOnTheBus(t)
js := b.js
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "operator-channel", Version: "1",
DefinesSeats: []catalogue.SeatDeclaration{{Name: broker.AsksSeat, Records: []string{"asks"}}}},
inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/operator-channel", BuiltFrom: "c0ffee"}); err != nil {
t.Fatal(err)
}
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
if err := broker.AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
@@ -87,10 +64,14 @@ func TestAMergeOnTheApproveIntoANewRepositoryAsksTheFirstBuildAndItsOutcomeRegis
a := aDeliveryMergeOf(t, head, map[string]string{"novox/mesh-delivery@a1a1a1a1a1a1": head})
if c.approved {
record, _ := json.Marshal(map[string]any{"state": "closed", "ask": a.ask, "warrant": a.warrant})
if _, err := records.Put(ctx, theDelivery+"."+a.merge.AskID, record); err != nil {
if _, err := records.Put(ctx, sdkdelivery.TellerModule+"."+a.merge.AskID, record); err != nil {
t.Fatal(err)
}
}
// The indexer's value and the builder's done key of the module the merge commit holds.
fp := fpOf('9')
manifest := b.indexed(t, "mesh-delivery-view", "1", "novox/mesh-delivery", ".", commit, fp)
b.built(t, "mesh-delivery-view", "1", fp, manifest, sdkdelivery.BuildDone)
following := following{open: open}
server := link.ConnectNats(js, nil, noListener{})
@@ -104,8 +85,6 @@ func TestAMergeOnTheApproveIntoANewRepositoryAsksTheFirstBuildAndItsOutcomeRegis
done := make(chan struct{})
go func() { defer close(done); _ = server.Serve(serving) }()
t.Cleanup(func() { stop(); <-done })
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
word := func() {
t.Helper()
@@ -133,37 +112,27 @@ func TestAMergeOnTheApproveIntoANewRepositoryAsksTheFirstBuildAndItsOutcomeRegis
forge()
} else {
forge()
if len(asked()) != 0 {
t.Fatalf("1146/AC7: the merge asked %v before mesh-delivery's word was kept", asked())
time.Sleep(300 * time.Millisecond)
if _, known := inventoryHolds(t, inv, "mesh-delivery-view"); known {
t.Fatal("1146/AC7: registered before mesh-delivery's word was kept")
}
word()
}
if !c.wantRegiter {
if len(asked()) != 0 {
t.Fatalf("1146/AC7: a merge whose record holds no Approve asked %v", asked())
if !c.wantRegister {
time.Sleep(time.Second)
if _, known := inventoryHolds(t, inv, "mesh-delivery-view"); known {
t.Fatal("1146/AC7: a merge whose record holds no Approve registered its module")
}
return
}
want := [3]string{"novox/mesh-delivery", "", "main"}
becomes(t, 10*time.Second, "the first build asked", func() bool { return len(asked()) >= 1 })
if got := asked(); len(got) != 1 || got[0] != want {
t.Fatalf("1146/AC7: the merge asked %v, want %v once", got, want)
}
mu.Lock()
first := ids[0]
mu.Unlock()
build := onTrunk(first, "novox/mesh-delivery", "git", "", map[string]any{"module": "mesh-delivery-view", "version": "1"})
build.Commit = commit
if _, _, err := takeIn(ctx, inv, build); err != nil {
t.Fatalf("1146/AC7: the outcome of the first build was refused: %v", err)
}
becomes(t, 10*time.Second, "the module registered", func() bool {
_, known := inventoryHolds(t, inv, "mesh-delivery-view")
return known
})
src, err := inv.SourceOf(ctx, "mesh-delivery-view")
if err != nil || src.Repository != "novox/mesh-delivery" {
t.Fatalf("1146/AC7: the module is built from %q (%v)", src.Repository, err)
}
if _, known := inventoryHolds(t, inv, "mesh-delivery-view"); !known {
t.Fatal("1146/AC7: the module is not in the catalogue")
}
})
}
}
+2 -2
View File
@@ -64,7 +64,7 @@ func registerFrom(t *testing.T, forms *sourceForms) (int64, error) {
mesh := &aMeshRegisteredFromTheURL{anEmptyMesh{asked: true,
request: inventory.BuildRequest{ID: "build-1", Repository: aRepoOnTheForge, For: "build"}}, aRepoOnTheForge}
return mayRegisterIn(t.Context(), mesh, forms, "mesh-delivery-view", inventory.Source{Repository: aRepoOnTheForge},
"build-1", "", "main")
registrationAsk{Build: "build-1"}, "", "main")
}
// 1277/AC2: an address unread twice, then read, is waited out and the forge is asked.
@@ -112,7 +112,7 @@ func TestAKnownAddressAndAnotherHostIsStillNotOnTheForge(t *testing.T) {
const elsewhere = "http://elsewhere.example/x/y.git"
mesh := &aMeshRegisteredFromTheURL{anEmptyMesh{asked: true, request: inventory.BuildRequest{ID: "build-1", For: "build"}}, elsewhere}
_, err := mayRegisterIn(t.Context(), mesh, sourceFormsRead(context.Background(), time.Now, read), "mesh-delivery-view",
inventory.Source{Repository: elsewhere}, "build-1", "", "main")
inventory.Source{Repository: elsewhere}, registrationAsk{Build: "build-1"}, "", "main")
if !errors.Is(err, errNotItsSource) || !strings.Contains(err.Error(), "is not on the mesh's forge") {
t.Fatalf("1277/AC4: the refusal changed: %v", err)
}
+1 -1
View File
@@ -68,7 +68,7 @@ func TestTheForgeNotAnsweringInTimeIsNotAskedAgain(t *testing.T) {
// An ask that hangs past the bound is cut off at the bound, not at its own wait, and the failure still says it
// waited: the receive loop is held no longer than the bound (novox/hq issue 486).
func TestAnAskHangingPastTheBoundIsCutOffAtIt(t *testing.T) {
w := waitOut{bound: 200 * time.Millisecond, first: 10 * time.Millisecond, most: 50 * time.Millisecond,
w := waitOut{bound: 100 * time.Millisecond, first: 10 * time.Millisecond, most: 50 * time.Millisecond,
now: time.Now, sleep: sleepOrDone}
asked := 0
began := time.Now()
+6 -2
View File
@@ -444,8 +444,12 @@ func TestAMergeThatDeletesAModulePlansNothingToBuildForIt(t *testing.T) {
if err := inv.SavePlan(ctx, &plan); err != nil {
t.Fatal(err)
}
planBuilt(ctx, open, "kept", "", "http://forge/novox/mesh-catalog.git has no module.json at modules/kept, so "+
"there is nothing saying what it is: open …/module.json: no such file or directory", asked0, "build-k")
if err := inv.RecordBuild(ctx, inventory.Build{ID: "build-k", Module: "kept", Repository: "novox/mesh-catalog",
At: time.Now().UTC(), Asked: asked0, Failed: "http://forge/novox/mesh-catalog.git has no module.json at modules/kept, so " +
"there is nothing saying what it is: open …/module.json: no such file or directory"}); err != nil {
t.Fatal(err)
}
advancePlans(ctx, open)
p, _ := inv.PlanByID(ctx, "plan-deleted")
if p.State == inventory.PlanFailed || p.Modules["kept"].State != planDeleted {
t.Fatalf("a module deleted at its source failed the plan: %s %+v", p.State, p.Modules["kept"])
+168
View File
@@ -0,0 +1,168 @@
package main
import (
"context"
"errors"
"fmt"
"sort"
"strings"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
)
// The grants issued (novox/hq ADR 0528 clause 2, ADR 0429 §6 rule 6, issue 1394): what the assigner's sixth rule reads,
// that a module's bus credential for a fingerprint exists. A module's permissions on the bus are what the bus's user list
// composes from its registered manifest (inv.BusRecords), and a module's manifest is registered at a fingerprint when the
// controller takes it in from the builder's key; so once the list is sent to the machine that runs the bus, the
// credential of every module it carried is issued at the fingerprint of the build it is registered at. The controller
// writes `<module>.<fingerprint hex>` then, naming the accounts it composed for the module, never removed. A module one
// of whose accounts has no password yet (the list leaves it out) has no grant until a later send carries it.
// grantsFrom is the grant of every module whose accounts all have a password in hashes, at the fingerprint current names
// for it; a module whose current build names no fingerprint of the new form has none. principals are the bus's users
// as broker.Users composes them.
func grantsFrom(principals []broker.Principal, hashes map[string]string, current map[string]string) map[string]sdkdelivery.ControllerGrant {
accounts, missing := map[string][]string{}, map[string]bool{}
for _, p := range principals {
if p.Module == "" || (p.Kind != broker.KindModule && p.Kind != broker.KindProcess) {
continue
}
if hashes[p.Username()] == "" {
missing[p.Module] = true
continue
}
accounts[p.Module] = append(accounts[p.Module], p.Username())
}
out := map[string]sdkdelivery.ControllerGrant{}
for module, fp := range current {
if missing[module] || !strings.HasPrefix(fp, sdkdelivery.SourcePrefix2) {
continue
}
key, err := sdkdelivery.ControllerGrantKey(module, fp)
if err != nil {
continue
}
names := accounts[module]
sort.Strings(names)
out[key] = sdkdelivery.ControllerGrant{Accounts: names}
}
return out
}
// currentFingerprints is, by module, the fingerprint of the build each registered module is registered at: its
// manifest's commit (built_from), that commit's newest successful build, and the fingerprint that build names.
func currentFingerprints(ctx context.Context, inv *inventory.Inventory) (map[string]string, error) {
builds, err := inv.CurrentBuilds(ctx)
if err != nil {
return nil, err
}
out := map[string]string{}
for module, b := range builds {
if b.Commit == "" {
continue
}
id, err := inv.BuildOf(ctx, module, b.Commit)
if err != nil || id == "" {
continue
}
build, found, err := inv.BuildByID(ctx, id)
if err != nil || !found {
continue
}
out[module] = build.SourceFingerprint
}
return out, nil
}
// grantsKV is what writing the grants needs of their bucket.
type grantsKV interface {
Get(ctx context.Context, key string) (jetstream.KeyValueEntry, error)
Create(ctx context.Context, key string, value []byte, opts ...jetstream.KVCreateOpt) (uint64, error)
Update(ctx context.Context, key string, value []byte, revision uint64) (uint64, error)
}
// keepGrants writes every grant the list just sent issued that the bucket does not hold as it is: a create for a new one,
// a compare-and-set for one whose accounts changed. It answers how many it wrote.
func keepGrants(ctx context.Context, kv grantsKV, grants map[string]sdkdelivery.ControllerGrant) (int, error) {
keys := make([]string, 0, len(grants))
for k := range grants {
keys = append(keys, k)
}
sort.Strings(keys)
wrote := 0
var errs []error
for _, key := range keys {
value, err := sdkdelivery.EncodeControllerGrant(grants[key])
if err != nil {
errs = append(errs, fmt.Errorf("%s: %w", key, err))
continue
}
e, err := kv.Get(ctx, key)
switch {
case errors.Is(err, jetstream.ErrKeyNotFound):
_, err = kv.Create(ctx, key, value)
case err != nil:
default:
if before, derr := sdkdelivery.DecodeControllerGrant(e.Value()); derr == nil && sdkdelivery.SameControllerGrant(before, withVersion(grants[key])) {
continue
}
_, err = kv.Update(ctx, key, value, e.Revision())
}
if err != nil {
errs = append(errs, fmt.Errorf("%s: %w", key, err))
continue
}
wrote++
}
return wrote, errors.Join(errs...)
}
func withVersion(g sdkdelivery.ControllerGrant) sdkdelivery.ControllerGrant {
g.V = sdkdelivery.Version
return g
}
// grantsIssued writes the grants the bus's user list just sent to its machine issued (ADR 0528 clause 2). Said, never
// fatal: a grant not written now is written at the next send of the list, and the assigner's wants wait for it.
func grantsIssued(ctx context.Context, inv *inventory.Inventory, js jetstream.JetStream, indent string) {
kv, err := js.KeyValue(ctx, broker.GrantsBucket)
if err != nil {
fmt.Printf("%sthe grants issued are not written: the bucket %s: %v\n", indent, broker.GrantsBucket, err)
return
}
records, err := inv.BusRecords(ctx)
if err != nil {
fmt.Printf("%sthe grants issued are not written: %v\n", indent, err)
return
}
principals, err := broker.Users(records)
if err != nil {
fmt.Printf("%sthe grants issued are not written: %v\n", indent, err)
return
}
kept, err := inv.BusUsers(ctx)
if err != nil {
fmt.Printf("%sthe grants issued are not written: %v\n", indent, err)
return
}
hashes := make(map[string]string, len(kept))
for name, u := range kept {
hashes[name] = u.PasswordHash
}
current, err := currentFingerprints(ctx, inv)
if err != nil {
fmt.Printf("%sthe grants issued are not written: %v\n", indent, err)
return
}
wrote, err := keepGrants(ctx, kv, grantsFrom(principals, hashes, current))
if wrote > 0 {
fmt.Printf("%s%d grant(s) issued written for the assigner\n", indent, wrote)
}
if err != nil {
fmt.Printf("%ssome grants issued are not written: %v\n", indent, err)
}
}
@@ -0,0 +1,91 @@
//go:build integration
package main
import (
"context"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// 1394/AC1 of C9 (ADR 0528 clause 2): when the controller has issued mesh-x's credential for fingerprint F (mesh-x
// registered at a build of F, its account minted, the bus's machine sent its declaration with the user list, through
// overTheBus.declare as a push sends it), then grants.mesh-x.F exists on the bus,
// and a second pass writes it again as nothing new.
func TestTheGrantOfAnIssuedCredentialIsOnTheBus(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
fp := grantFP('d')
m := catalogue.Manifest{Module: "mesh-x", Version: "1", OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/var/lib/mesh-x/broker"}}}
if err := inv.RecordBuild(ctx, inventory.Build{ID: "build-x1", Repository: "novox/mesh-x", Module: "mesh-x", Commit: "c0ffee1",
SourceFingerprint: fp}); err != nil {
t.Fatal(err)
}
if err := inv.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-x", BuiltFrom: "c0ffee1", Head: "c0ffee1"}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "mesh-x"); err != nil {
t.Fatal(err)
}
user := broker.Principal{Kind: broker.KindModule, Node: "anchor", Module: "mesh-x"}.Username()
if _, err := inv.MintBusPassword(ctx, inventory.BusUser{Username: user, Kind: inventory.BusModule, Node: "anchor", Module: "mesh-x"}); err != nil {
t.Fatal(err)
}
conn, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(conn.Close)
js, err := jetstream.New(conn)
if err != nil {
t.Fatal(err)
}
if err := broker.OnConn(conn).EnsureControllerBuckets(); err != nil {
t.Fatal(err)
}
// The live path: a send of the bus's machine's declaration carrying the user list (overTheBus.declare), which records
// the list sent and then writes the grants it issued.
legacy, err := conn.JetStream()
if err != nil {
t.Fatal(err)
}
_ = js.DeleteStream(ctx, "GRANTS_DECLARED")
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "GRANTS_DECLARED", Subjects: []string{link.DeclareSubject("anchor")}}); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = js.DeleteStream(context.Background(), "GRANTS_DECLARED") })
send := overTheBus{open: open, server: link.SendingOn(link.OverNATS{Conn: conn, JS: legacy}), signer: aSignerOfZeros{}}
ready := readyNode{node: "anchor", declared: sendable{BusUsers: "the user list the bus's machine is sent"}}
if _, err := send.declare(ctx, ready, []byte(`{}`)); err != nil {
t.Fatalf("the send of the bus's machine: %v", err)
}
kv, err := js.KeyValue(ctx, broker.GrantsBucket)
if err != nil {
t.Fatal(err)
}
key, _ := sdkdelivery.ControllerGrantKey("mesh-x", fp)
e, err := kv.Get(ctx, key)
if err != nil {
t.Fatalf("no grant %s after the list was issued: %v", key, err)
}
g, err := sdkdelivery.DecodeControllerGrant(e.Value())
if err != nil || len(g.Accounts) != 1 || g.Accounts[0] != user {
t.Fatalf("the grant reads %+v, %v; want the account %s", g, err, user)
}
if _, err := send.declare(ctx, ready, []byte(`{}`)); err != nil {
t.Fatalf("the second send: %v", err)
}
if again, _ := kv.Get(ctx, key); again.Revision() != e.Revision() {
t.Errorf("a second pass wrote the grant again (revision %d, was %d)", again.Revision(), e.Revision())
}
}
-34
View File
@@ -9,9 +9,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The grants reach the bus before the gate (novox/hq issue 490).
@@ -176,35 +174,3 @@ func TestTheGrantsStepKeepsOneMachinesBuilds(t *testing.T) {
t.Error("the grants step keeps the builds of a machine it does not send")
}
}
// The delivery plan says the step before the merge.
func TestAChangePlanSaysTheGrantsStep(t *testing.T) {
const catalogue_ = "http://forge.internal:20000/novox/mesh-catalog.git"
entry := func(name string, on ...string) inventory.Entry {
e := fromRepo(name, catalogue_, "modules/"+name)
e.Source.BuiltFrom = "old"
e.On = on
return e
}
nats := entry("nats", "novox")
nats.Manifest.BusUsers = "/etc/nats/users.conf"
nats.Manifest.Claims = []catalogue.Claim{{Name: catalogue.BrokerSeat, Scope: catalogue.ScopeMesh}}
restic := entry("restic", "ace", "novox", "shanks")
only := entry("bus-tools", "novox")
entries := []inventory.Entry{nats, restic, only}
rollsOut := func(string) (inventory.Upgrade, bool) { return inventory.Upgrade{RollOut: true}, true }
plan := func(paths ...string) link.ChangePlan {
m := link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main", Commit: "head", Paths: paths}
return changePlanOf("novox/mesh-catalog", "main", "head", reachOfMerge(m, entries, nil, nil), entries, rollsOut)
}
text := planText(plan("modules/restic/module.json"))
if !strings.Contains(text, "grants step: if this changes what the bus's user list says (a new tool, subject or "+
"user), novox is sent that list alone, every build there kept, before restic is judged on its first machine") {
t.Errorf("the change plan does not say the grants step:\n%s", text)
}
// A module only on the bus's machine goes there with the list in its own send: no step of its own.
if text := planText(plan("modules/bus-tools/module.json")); strings.Contains(text, "grants step") {
t.Errorf("a module on the bus's machine alone reads:\n%s", text)
}
}
+106
View File
@@ -0,0 +1,106 @@
package main
import (
"context"
"strings"
"testing"
"time"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
)
func grantFP(c byte) string { return sdkdelivery.SourcePrefix2 + strings.Repeat(string(c), 64) }
// 1394/AC (unit half): the list sent issues the credential of every module whose accounts all have a password, at the
// fingerprint its build names; a module with an account the list leaves out, and one registered at a fingerprint not of
// the new form, get no grant; a module with no account of its own is granted with none.
func TestTheGrantsAListIssues(t *testing.T) {
x := broker.Principal{Kind: broker.KindModule, Node: "novox", Module: "mesh-x"}
xp := broker.Principal{Kind: broker.KindProcess, Node: "novox", Module: "mesh-x", Process: "run"}
y := broker.Principal{Kind: broker.KindModule, Node: "ace", Module: "mesh-y"}
node := broker.Principal{Kind: broker.KindNode, Node: "novox"}
hashes := map[string]string{x.Username(): "h1", xp.Username(): "h2", node.Username(): "h3"}
got := grantsFrom([]broker.Principal{x, xp, y, node}, hashes,
map[string]string{"mesh-x": grantFP('a'), "mesh-y": grantFP('b'), "web": grantFP('c'), "old": "src1:abc"})
xKey, _ := sdkdelivery.ControllerGrantKey("mesh-x", grantFP('a'))
webKey, _ := sdkdelivery.ControllerGrantKey("web", grantFP('c'))
if len(got) != 2 {
t.Fatalf("grants = %v, want mesh-x's and web's", got)
}
if g := got[xKey]; len(g.Accounts) != 2 || g.Accounts[0] != x.Username() && g.Accounts[1] != x.Username() {
t.Errorf("mesh-x's grant = %+v, want both its accounts", g)
}
if g, ok := got[webKey]; !ok || len(g.Accounts) != 0 {
t.Errorf("web, with no account of its own, is granted %+v, %v", g, ok)
}
}
// fakeGrants is the grants bucket in memory.
type fakeGrants struct {
values map[string][]byte
creates int
updates int
}
type fakeEntry struct {
key string
value []byte
rev uint64
}
func (e fakeEntry) Bucket() string { return broker.GrantsBucket }
func (e fakeEntry) Key() string { return e.key }
func (e fakeEntry) Value() []byte { return e.value }
func (e fakeEntry) Revision() uint64 { return e.rev }
func (e fakeEntry) Created() time.Time { return time.Time{} }
func (e fakeEntry) Delta() uint64 { return 0 }
func (e fakeEntry) Operation() jetstream.KeyValueOp { return jetstream.KeyValuePut }
func (f *fakeGrants) Get(_ context.Context, key string) (jetstream.KeyValueEntry, error) {
v, ok := f.values[key]
if !ok {
return nil, jetstream.ErrKeyNotFound
}
return fakeEntry{key: key, value: v, rev: 1}, nil
}
func (f *fakeGrants) Create(_ context.Context, key string, value []byte, _ ...jetstream.KVCreateOpt) (uint64, error) {
if _, ok := f.values[key]; ok {
return 0, jetstream.ErrKeyExists
}
f.values[key] = value
f.creates++
return 1, nil
}
func (f *fakeGrants) Update(_ context.Context, key string, value []byte, _ uint64) (uint64, error) {
f.values[key] = value
f.updates++
return 2, nil
}
// 1394/AC (unit half): a grant is created once; a second send of the same list writes nothing; a reissue for other
// accounts is a new revision of the same key.
func TestAGrantIsWrittenOnceAndMovedOnAReissue(t *testing.T) {
kv := &fakeGrants{values: map[string][]byte{}}
key, _ := sdkdelivery.ControllerGrantKey("mesh-x", grantFP('a'))
grants := map[string]sdkdelivery.ControllerGrant{key: {Accounts: []string{"novox.mesh-x"}}}
for range 2 {
if _, err := keepGrants(t.Context(), kv, grants); err != nil {
t.Fatal(err)
}
}
if kv.creates != 1 || kv.updates != 0 {
t.Errorf("creates %d, updates %d, want one create", kv.creates, kv.updates)
}
grants[key] = sdkdelivery.ControllerGrant{Accounts: []string{"ace.mesh-x"}}
if wrote, err := keepGrants(t.Context(), kv, grants); err != nil || wrote != 1 || kv.updates != 1 {
t.Errorf("a reissue wrote %d (%v), updates %d", wrote, err, kv.updates)
}
if g, err := sdkdelivery.DecodeControllerGrant(kv.values[key]); err != nil || g.Accounts[0] != "ace.mesh-x" {
t.Errorf("the grant reads %+v, %v", g, err)
}
}
+3
View File
@@ -64,6 +64,9 @@ var handActVerbs = []handActVerb{
{Verb: "plans stop", Decision: "the operator's answer to an ask is their decision, not a repair (ADR 0259)",
DecidedFor: []string{conditions.CauseOperatorAnswer}},
{Verb: "plans close"},
// A module registered at references by hand, because the builder had no key for it (ADR 0526 clause 3): a repair
// the builder should have made, so it counts.
{Verb: "register"},
// A walk started by a person instead of its delivery's owner (novox/hq ADR 0239): the owner down, or
// not trusted with it — either is a repair the owner should have made. Unless the operator chose it on
// a warrant (ADR 0259).
+198
View File
@@ -0,0 +1,198 @@
package main
import (
"context"
"fmt"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// A move held for a person is asked of the person (novox/hq issue 1732, issue 1320; ADR 0377, ADR 0259).
//
// **A wait only a person can end is said to that person at once.** ADR 0377 holds a recorded module's move that
// recreates a container or touches its data until a person pushes the machine. The walk said so in its note and
// ended "done", `status` named the machine behind, and nobody was asked: on 2026-10-21 mailu's new build waited on
// novox for 40 minutes behind a plan that read "done, 3 tier(s)", and on 2026-10-19 mssql waited on ace the same way.
// Row S24 raises one condition per machine that has such a move, for the operator, with the answer **Release**: the
// push of that machine, which the controller's own `push` verb performs on the operator's warrant (ADR 0259). It ends
// when the machine no longer waits, whoever pushed it.
//
// A hold a person chose is not asked again — a module they set to `record` with a why, and the bus, whose step has
// its own row (S17).
// kindMovesHeld is S24's kind: machine.<node>.moves-held.
const kindMovesHeld = "moves-held"
// heldFacts is one machine and the moves of recorded modules that wait there for a person.
type heldFacts struct {
node string
moves []heldMove
}
// gatherHeldMoves is, per machine that holds at least one move no person chose to hold, all its held moves, by machine.
func gatherHeldMoves(ctx context.Context, inv *inventory.Inventory) ([]heldFacts, error) {
nodes, err := inv.Nodes(ctx)
if err != nil {
return nil, fmt.Errorf("the machines cannot be read: %w", err)
}
f, err := readMoveFacts(ctx, inv)
if err != nil {
return nil, err
}
var out []heldFacts
for _, n := range nodes {
moves, err := heldMovesOn(ctx, inv, f, n.Name)
if err != nil {
return nil, err
}
h := heldFacts{node: n.Name, moves: moves}
if len(h.asked()) > 0 {
out = append(out, h)
}
}
return out, nil
}
// asked are the moves no person chose to hold, and chosen the ones a person did (a module set to record with a why, the bus).
func (h heldFacts) asked() []heldMove { return h.byChoice(false) }
func (h heldFacts) chosen() []heldMove { return h.byChoice(true) }
func (h heldFacts) byChoice(chosen bool) []heldMove {
var out []heldMove
for _, m := range h.moves {
if m.Chosen == chosen {
out = append(out, m)
}
}
return out
}
// modules are the held moves' modules, in order, of all of them.
func (h heldFacts) modules() []string { return moduleNames(h.moves) }
func moduleNames(moves []heldMove) []string {
out := make([]string, 0, len(moves))
for _, m := range moves {
out = append(out, m.Module)
}
return out
}
// releaseAction is the one answer a held move offers: the push of the machine, approved by the operator. The push
// names every module whose move waits there as one that may move its data: the operator's Approve is the word
// ADR 0217 asks for, and the ask says so. Offered only where no person's own hold is on the machine too: a push
// composes every recorded module at its current build, so it would send that move as well, which the ask never named
// (ADR 0259 §6), and hold the whole machine if that module's data would move.
func (h heldFacts) releaseAction() conditions.Action {
return conditions.Action{Label: "Release", Verb: "mesh-controller.push", Level: conditions.LevelApprove,
Arguments: map[string]string{"node": h.node, "move": strings.Join(moduleNames(h.asked()), ","), "why": "",
"cause": conditions.CauseOperatorAnswer}}
}
// heldWords are the plain words of a machine's held moves: which modules wait, why in general, and what an Approve does.
func heldWords(h heldFacts) (headline, explanation, needs string, resolved string) {
names := namesWords(moduleNames(h.asked()), 3)
headline = clipWords(conditions.Capital(names)+" waits for you on "+h.node, conditions.HeadlineMax)
recreates, data := false, false
for _, m := range h.asked() {
switch {
case strings.Contains(m.Why, "recreates"):
recreates = true
default:
data = true
}
}
why := "would change what it runs"
switch {
case recreates && data:
why = "would restart a container or touch its data"
case recreates:
why = "would restart one of its containers"
case data:
why = "would touch its files or data"
}
explanation = clipWords(fmt.Sprintf("The new version of %s %s on %s, so it does not go out with its delivery; "+
"it waits for you. Release sends %s everything it is owed, these among it, and lets their data move where it "+
"has to. Nothing is sent until you answer.", names, why, h.node, h.node), conditions.ExplanationMax)
needs = "release them " + FromMeshMCPServer
if chosen := h.chosen(); len(chosen) > 0 {
// A push of the machine would send the move a person chose to hold as well: no answer is offered here.
explanation = clipWords(fmt.Sprintf("The new version of %s %s on %s, so it does not go out with its delivery; "+
"it waits for you. %s is also held there by your own choice, and a push of %s would send that too, so "+
"nothing is offered here: the push is yours at the controller's terminal.",
names, why, h.node, conditions.Capital(namesWords(moduleNames(chosen), 3)), h.node), conditions.ExplanationMax)
needs = fmt.Sprintf("push %s at the controller's terminal, once you have decided about %s.", h.node,
namesWords(moduleNames(chosen), 3))
}
resolved = "Resolved: " + names + " is no longer waiting on " + h.node
return
}
// watchMovesHeld is S24: a machine that has moves of recorded modules waiting for a person, said to the operator with the
// answer that sends them. The condition ends when the machine no longer waits: it is no longer among the facts.
func watchMovesHeld(f *signalFacts) []conditions.Observation {
var out []conditions.Observation
for _, h := range f.held {
var said []string
for _, m := range h.moves {
said = append(said, fmt.Sprintf("%s %s → %s %s", m.Module, short(m.From), short(m.To), m.Why))
}
headline, explanation, needs, resolved := heldWords(h)
if c := h.chosen(); len(c) > 0 {
said = append(said, "held by a person's own choice: "+strings.Join(moduleNames(c), ", "))
}
var actions []conditions.Action
if len(h.chosen()) == 0 {
actions = []conditions.Action{h.releaseAction()}
}
out = append(out, conditions.Observation{Scope: conditions.ScopeMachine, ID: h.node, Kind: kindMovesHeld,
Machine: h.node, Severity: conditions.Warning, Resolver: conditions.ResolverOperator,
Summary: fmt.Sprintf("%s has %d move(s) of recorded modules waiting for a person's push: %s — `push %s` sends them",
h.node, len(h.moves), strings.Join(h.modules(), ", "), h.node),
Said: strings.Join(said, "; "),
Headline: headline, Explanation: explanation, Needs: needs, Resolved: resolved,
Actions: actions})
}
return out
}
// planHeld is, for a plan that ended or is running, the moves its modules carried that a machine still holds for a
// person: "module on machine", sorted. The plan's own record says the move waited when it was judged
// (PlanModule.Recorded); the machines say whether it still does, and for the build this plan carried, so a plan older
// than the build now held is not named.
func planHeld(p inventory.Plan, held []heldFacts) []string {
var out []string
for _, h := range held {
for _, m := range h.moves {
s := p.Modules[m.Module]
if s == nil || !strings.HasPrefix(s.Recorded[h.node], "waits:") || !sameCommit(s.Commit, m.To) {
continue
}
out = append(out, m.Module+" on "+h.node)
}
}
sort.Strings(out)
return out
}
// heldSaid is what a plan's line adds when it carries a move that a machine holds for a person (novox/hq issue 1732): a
// plan whose module waits for a push is not plainly "done".
func heldSaid(p inventory.Plan, held []heldFacts) string {
waiting := planHeld(p, held)
if len(waiting) == 0 {
return ""
}
return fmt.Sprintf(" — %d held for a person: %s (the operator is asked)", len(waiting), strings.Join(waiting, ", "))
}
// heldNow reads the machines' held moves for a plan's line, nil when they cannot be read: the line then reads as before.
func heldNow(ctx context.Context, inv *inventory.Inventory) []heldFacts {
held, err := gatherHeldMoves(ctx, inv)
if err != nil {
return nil // empty-on-error: a plan's line then reads as it did before; the watchdog (S24) says it cannot read
}
return held
}
@@ -0,0 +1,92 @@
//go:build integration
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
func inventoryUpgrade(why string) inventory.Upgrade { return inventory.Upgrade{Why: why, By: "jochen"} }
func mustMoveFacts(t *testing.T, rm *recordedMesh) moveFacts {
t.Helper()
f, err := readMoveFacts(t.Context(), rm.open.inventory)
if err != nil {
t.Fatal(err)
}
return f
}
// 1732/AC8 (integration): a mesh running mailu at c1 on two machines, whose registered c2 recreates a container, is read
// as two machines each holding mailu's move for a person; a bundle-only c3 on top changes nothing of it; after a machine
// is sent c2 (as a push records it) it is no longer among them, and plans that carried c2 say nothing more of it.
func TestMachinesHoldingAMoveForAPersonAreReadFromTheRecords(t *testing.T) {
rm := aRecordedMesh(t, aMailu(true))
c2 := aMailu(true)
resourceOf(&c2, "imap")["image"] = "registry.invalid/imap@sha256:2"
rm.register(t, c2, "c2", time.Now().Add(-time.Minute))
ctx := t.Context()
held, err := gatherHeldMoves(ctx, rm.open.inventory)
if err != nil {
t.Fatal(err)
}
if len(held) != 2 || held[0].node != "anchor" || held[1].node != "laptop" || len(held[0].moves) != 1 ||
held[0].moves[0].Module != "mailu" || held[0].moves[0].To != "c2" || !strings.Contains(held[0].moves[0].Why, "recreates") {
t.Fatalf("held %+v", held)
}
if err := rm.open.inventory.RecordSent(ctx, nodeID(t, rm.open, "anchor"), "d-anchor-c2", map[string]string{"mailu": "c2"}); err != nil {
t.Fatal(err)
}
held, err = gatherHeldMoves(ctx, rm.open.inventory)
if err != nil || len(held) != 1 || held[0].node != "laptop" {
t.Fatalf("after anchor was sent c2: %+v (%v)", held, err)
}
}
// 1732/AC9 (integration): a module a person set to record is no ask: they chose the hold.
func TestAHoldAPersonChoseIsNotAsked(t *testing.T) {
rm := aRecordedMesh(t, aMailu(false))
c2 := aMailu(false)
c2.Bundles[0].Digest = "sha256:bbb"
rm.register(t, c2, "c2", time.Now().Add(-time.Minute))
if err := rm.open.inventory.SetUpgradeOf(t.Context(), "mailu", inventoryUpgrade("the migration week")); err != nil {
t.Fatal(err)
}
held, err := gatherHeldMoves(t.Context(), rm.open.inventory)
if err != nil || len(held) != 0 {
t.Fatalf("a machine holding only a person's own hold is asked: %+v (%v)", held, err)
}
if moves, _ := heldMovesOn(t.Context(), rm.open.inventory, mustMoveFacts(t, rm), "anchor"); len(moves) != 1 || !moves[0].Chosen {
t.Fatalf("status still names it, marked as chosen: %+v", moves)
}
}
// 1742/AC4 (integration): a registered build naming an image of the store that the store lacks is read as a missing
// image on each machine that runs the module; one the store holds is not; and what the store holds is not asked twice.
func TestARegisteredBuildNamingAnImageTheStoreLacksIsRead(t *testing.T) {
m := aMailu(true)
resourceOf(&m, "imap")["image"] = "artifact-store://mailu/imap@sha256:" + strings.Repeat("e", 64)
rm := aRecordedMesh(t, m)
asked := 0
lacking := func(_ context.Context, reference string) (bool, error) { asked++; return false, nil }
imagesHeld.Lock()
imagesHeld.held = map[string]bool{}
imagesHeld.Unlock()
got, err := gatherMissingImages(t.Context(), rm.open.inventory, lacking)
if err != nil || len(got) != 2 || got[0].Module != "mailu" || got[0].Node != "anchor" || got[1].Node != "laptop" {
t.Fatalf("missing %+v (%v)", got, err)
}
holding := func(_ context.Context, reference string) (bool, error) { asked++; return true, nil }
if got, err := gatherMissingImages(t.Context(), rm.open.inventory, holding); err != nil || len(got) != 0 {
t.Fatalf("an image the store holds is missing: %+v (%v)", got, err)
}
before := asked
if _, err := gatherMissingImages(t.Context(), rm.open.inventory, lacking); err != nil || asked != before {
t.Fatalf("an image the store was found to hold was asked about again (%d, %v)", asked-before, err)
}
}
+237
View File
@@ -0,0 +1,237 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// **A move held for a person is asked of the person** (novox/hq issue 1732, issue 1320; ADR 0377, ADR 0259).
// aHeldMailu is novox holding mailu's new build because it would recreate a container: the walk's own words.
func aHeldMailu() heldFacts {
return heldFacts{node: "novox", moves: []heldMove{{Module: "mailu", From: "b2f5ebb2aaaa", To: "3259765dbbbb",
Why: "waits: recreates 1 of mailu's 12 container(s) (with a new image: automx)"}}}
}
func heldConditionOn(f *signalFacts) conditions.Condition {
o := watchMovesHeld(f)[0]
return conditions.Condition{Key: o.Key(), Kind: o.Kind, Severity: o.Severity, Headline: o.Headline,
Explanation: conditions.Verdict(o.Needs, o.Explanation), Needs: o.Needs, Actions: o.Actions}
}
// 1732/AC1: a machine holding a move of a recorded module for a person raises one condition for the operator, with
// Release as its only answer: the machine's push, approved, naming the modules whose moves wait there.
func TestAHeldMoveIsRaisedWithReleaseAsItsOnlyAction(t *testing.T) {
f := calm(time.Date(2026, 10, 21, 17, 30, 0, 0, time.UTC))
f.held = []heldFacts{aHeldMailu()}
got := watchMovesHeld(f)
if len(got) != 1 {
t.Fatalf("%+v", got)
}
o := got[0]
if o.Kind != "moves-held" || o.Scope != conditions.ScopeMachine || o.ID != "novox" || o.Severity != conditions.Warning ||
o.Resolver != conditions.ResolverOperator || o.Key() != "machine.novox.moves-held" {
t.Fatalf("kind, scope, subject, severity or resolver: %+v (%s)", o, o.Key())
}
if len(o.Actions) != 1 {
t.Fatalf("actions: %+v", o.Actions)
}
a := o.Actions[0]
if a.Label != "Release" || a.Verb != "mesh-controller.push" || a.Level != conditions.LevelApprove ||
a.Arguments["node"] != "novox" || a.Arguments["move"] != "mailu" || a.Arguments["cause"] != conditions.CauseOperatorAnswer {
t.Fatalf("the action is not the push of novox: %+v", a)
}
if _, takes := a.Arguments["why"]; !takes {
t.Fatal("the push takes a why, which the asker fills in with the operator's answer")
}
for _, want := range []string{"mailu", "novox"} {
if !strings.Contains(o.Headline+o.Explanation, want) {
t.Errorf("the words do not name %q: %q / %q", want, o.Headline, o.Explanation)
}
}
if !strings.Contains(o.Said, "recreates 1 of mailu's 12") {
t.Errorf("the evidence does not carry the walk's own words: %q", o.Said)
}
if w := (conditions.Observation{Headline: o.Headline, Explanation: o.Explanation, Needs: o.Needs, Resolved: o.Resolved}); w.Headline == "" || len(o.Headline) > conditions.HeadlineMax {
t.Errorf("headline: %q", o.Headline)
}
}
// 1732/AC2: a machine with nothing held raises nothing, so the condition ends the look after the move is sent.
func TestAMachineHoldingNothingRaisesNoHeldMoveCondition(t *testing.T) {
f := calm(time.Date(2026, 10, 21, 17, 30, 0, 0, time.UTC))
if got := watchMovesHeld(f); len(got) != 0 {
t.Fatalf("%+v", got)
}
}
// 1732/AC3: the condition is asked on a channel at the approve level in words that say what Release does, and the
// operator's warrant calls the controller's own push with the machine and the modules, once.
func TestAHeldMovesReleaseIsAskedAndAWarrantPushesTheMachine(t *testing.T) {
f := calm(time.Date(2026, 10, 21, 17, 30, 0, 0, time.UTC))
f.held = []heldFacts{aHeldMailu()}
c := heldConditionOn(f)
r := newAskerRig(t)
r.open = []conditions.Condition{c}
if err := r.a.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
sent := r.asksSent(t)
if len(sent) != 1 || len(sent[0].Options) != 1 {
t.Fatalf("asked %+v", sent)
}
o := sent[0].Options[0]
if o.Label != "Release" || o.Level != asks.Approve || o.Does != "the held builds are sent to novox" {
t.Errorf("the option is %+v", o)
}
if err := sent[0].Check(r.now); err != nil {
t.Errorf("the ask is refused as made: %v", err)
}
w := r.warrantFor(t, c.Key, "Release")
answerWith(t, r, w)
answerWith(t, r, w)
if len(r.called) != 1 || !strings.HasPrefix(r.called[0], "mesh-controller.push@ ") ||
!strings.Contains(r.called[0], `"node":"novox"`) || !strings.Contains(r.called[0], `"move":"mailu"`) ||
!strings.Contains(r.called[0], `"cause":"`+conditions.CauseOperatorAnswer+`"`) ||
!strings.Contains(r.called[0], "chose Release") {
t.Fatalf("called %v", r.called)
}
}
// 1732/AC4: the controller may call its own push and upgrade verbs on a warrant: the grant names them.
func TestTheControllerMayActOnAWarrantForPushAndUpgrade(t *testing.T) {
for _, verb := range []string{"push", "upgrade"} {
granted := false
for _, v := range broker.VerbsTheControllerActsOnAWarrant {
granted = granted || (v.Seat == broker.ControllerSeat && v.Verb == verb)
}
if !granted {
t.Errorf("the controller may not act on a warrant for %s", verb)
}
}
}
func aDoneMailuPlan() inventory.Plan {
return inventory.Plan{ID: "plan-1792585970446304628", Repository: "novox/mesh-catalog", State: inventory.PlanDone,
Tiers: [][]string{{"a"}, {"b"}, {"mailu"}},
Modules: map[string]*inventory.PlanModule{"mailu": {State: "built", Commit: "3259765dbbbbcccc",
Recorded: map[string]string{"novox": "waits: recreates 1 of mailu's 12 container(s)"}}}}
}
// 1732/AC5: a done plan that carries a move a machine still holds for a person says so on its line; once the machine
// no longer holds it, or holds another build of the module, the line says nothing more.
func TestADonePlanSaysWhatItHoldsForAPerson(t *testing.T) {
p := aDoneMailuPlan()
line := planLine(p, time.Date(2026, 10, 21, 16, 0, 0, 0, time.UTC)) + heldSaid(p, []heldFacts{aHeldMailu()})
if !strings.Contains(line, "done, 3 tier(s)") || !strings.Contains(line, "1 held for a person: mailu on novox") {
t.Errorf("the line: %q", line)
}
if said := heldSaid(p, nil); said != "" {
t.Errorf("a plan whose moves were sent says %q", said)
}
other := aHeldMailu()
other.moves[0].To = "9999999ffff"
if said := heldSaid(p, []heldFacts{other}); said != "" {
t.Errorf("a plan older than the build now held says %q", said)
}
p.Modules["mailu"].Recorded = map[string]string{"novox": recordedRollsWhy}
if said := heldSaid(p, []heldFacts{aHeldMailu()}); said != "" {
t.Errorf("a move the plan sent says %q", said)
}
}
// 1732/AC6: the builds held after a failed release are asked of the operator: the held-release condition offers
// Release, which calls the controller's own `upgrade` with release-backlog and nothing else, once, on a warrant.
func TestTheBuildsHeldAfterAFailedReleaseAreAskedAndReleasedOnAWarrant(t *testing.T) {
saved := backlogNow
t.Cleanup(func() { backlogNow = saved })
backlogNow.held = "release-1792595552499982311 failed (failed on novox); what waits is released again by a person"
backlogNow.waiting = map[string][]inventory.CarriedMove{"novox": {{Module: "indexer"}, {Module: "nextcloud"}}}
got := backlogObservation()
if len(got) != 1 || len(got[0].Actions) != 1 {
t.Fatalf("raised %+v", got)
}
a := got[0].Actions[0]
if a.Label != "Release" || a.Verb != "mesh-controller.upgrade" || a.Level != conditions.LevelApprove ||
a.Arguments["release-backlog"] != "true" || a.Arguments["cause"] != conditions.CauseOperatorAnswer {
t.Fatalf("the action: %+v", a)
}
o := got[0]
c := conditions.Condition{Key: o.Key(), Kind: o.Kind, Severity: o.Severity, Headline: o.Headline,
Explanation: conditions.Verdict(o.Needs, o.Explanation), Needs: o.Needs, Actions: o.Actions}
r := newAskerRig(t)
r.open = []conditions.Condition{c}
if err := r.a.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
sent := r.asksSent(t)
if len(sent) != 1 || len(sent[0].Options) != 1 || sent[0].Options[0].Does != "the builds held after the failed delivery are released" {
t.Fatalf("asked %+v", sent)
}
w := r.warrantFor(t, c.Key, "Release")
answerWith(t, r, w)
answerWith(t, r, w)
if len(r.called) != 1 || !strings.HasPrefix(r.called[0], "mesh-controller.upgrade@ ") ||
!strings.Contains(r.called[0], `"release-backlog":"true"`) || !strings.Contains(r.called[0], "chose Release") {
t.Fatalf("called %v", r.called)
}
}
// 1732/AC7: a plan whose send waits for a gate while the builds are held after a failed release reads as waiting for
// a person to release them, not "building ... LATE"; with nothing held it still reads as the wait it is.
func TestAPlanWaitingForAGateBehindAFailedReleaseIsNotLate(t *testing.T) {
saved := backlogNow
t.Cleanup(func() { backlogNow = saved })
now := time.Date(2026, 10, 21, 18, 50, 0, 0, time.UTC)
p := inventory.Plan{ID: "plan-1", Repository: "novox/mesh-sdk", Commit: "c0ffee00", State: inventory.PlanBuilding,
Tiers: [][]string{{"indexer"}}, TierEntered: now.Add(-100 * time.Minute), Created: now.Add(-110 * time.Minute),
Modules: map[string]*inventory.PlanModule{"indexer": {State: "built"}},
Note: refusalNote(0, errors.New("a build waits there for its gate: novox"))}
backlogNow.held = ""
if line := planLine(p, now); !strings.Contains(line, "a build waits there for its gate: novox") {
t.Errorf("the wait is not said: %q", line)
}
backlogNow.held = "release-1 failed (failed on novox); what waits is released again by a person"
line := planLine(p, now)
if strings.Contains(line, "LATE") || strings.Contains(line, "building") ||
!strings.Contains(line, "until a person releases it") {
t.Errorf("a plan held behind a failed release reads %q", line)
}
}
// 1732/AC10: a machine that also holds a move a person chose to hold gets no Release, since a push would send that move
// too; the condition names both and says the push is the controller's terminal. A machine with only asked moves keeps it.
func TestAMachineAlsoHoldingAPersonsOwnHoldIsOfferedNoRelease(t *testing.T) {
f := calm(time.Date(2026, 10, 21, 17, 30, 0, 0, time.UTC))
h := aHeldMailu()
h.moves = append(h.moves, heldMove{Module: "nftables", From: "n1aaaa", To: "n2bbbb",
Why: "waits: a person set it to record, and holds every move of it until they push", Chosen: true})
f.held = []heldFacts{h}
got := watchMovesHeld(f)
if len(got) != 1 || len(got[0].Actions) != 0 {
t.Fatalf("%+v", got)
}
o := got[0]
text := o.Headline + o.Explanation + o.Needs + o.Summary + o.Said
for _, want := range []string{"mailu", "nftables", "terminal", "push novox"} {
if !strings.Contains(text, want) {
t.Errorf("does not say %q: %q", want, text)
}
}
w := conditions.Words{Headline: o.Headline, Explanation: o.Explanation, Resolved: o.Resolved, Needs: o.Needs}
if why, ok := conditions.PlainWords(w, "novox"); !ok {
t.Errorf("not plain: %s", why)
}
f.held = []heldFacts{aHeldMailu()}
if got := watchMovesHeld(f); len(got) != 1 || len(got[0].Actions) != 1 || got[0].Actions[0].Arguments["move"] != "mailu" {
t.Fatalf("a machine with only asked moves: %+v", got)
}
}
+245
View File
@@ -0,0 +1,245 @@
package main
import (
"context"
"fmt"
"slices"
"sort"
"strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/artifacts"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// No declaration is sent that names an image the artifact store does not hold (novox/hq issue 1742).
//
// On 2026-10-21 at 17:13 a send to novox named `mailu/automx@sha256:c62150b8…`; the build record keeps that digest
// ("recent-build") and the store did not hold it. The node-engine removed the running container, then could not pull
// the replacement, and every later apply on novox failed for the next hour while the mail clients' autoconfiguration
// had no backend. The node-engine's half (pull first, replace after) is its own change; this is the controller's: **a
// module whose image the store lacks is left out of the declaration**, as a module whose setting nobody gave is
// (ADR 0163, rule 6) — the node-engine keeps what it has of it and touches none of its containers — and **it is said**,
// row S25: a condition naming the module, the machine and the image, until the store holds the image again (a new build
// of the module, or the copy put back).
//
// Judged only for an image the store's own address names, from the send's own resources; an image of another registry
// is no business of the store's. A store that cannot be asked judges nothing: a send is not refused for a question
// that did not answer.
// kindImageMissing is S25's kind: module.<module>.<machine>.image-missing.
const kindImageMissing = "image-missing"
// missingImage is a module held out of a machine's declaration because the store lacks an image it names.
type missingImage struct {
Node, Module, Image string
}
// storeHolds is whether the store holds the manifest an image reference at its address names.
type storeHolds func(ctx context.Context, reference string) (bool, error)
// imageInStore is the artifact-store reference of an image the store's address names, "" for any other.
func imageInStore(image, address string) string {
if address == "" || !strings.HasPrefix(image, address+"/") || !strings.Contains(image, "@sha256:") {
return ""
}
reference := catalogue.Recorded(image)
if _, ours := catalogue.InArtifactStore(reference); !ours {
return ""
}
return reference
}
// holdBackMissingImages leaves out of the declaration every module that names an image the store does not hold,
// and answers what it left out. The module's resources go, it joins LeftOut with the reason, and its build stays the
// one the machine was last sent (`was`).
func holdBackMissingImages(ctx context.Context, node string, s *sendable, address string, holds storeHolds,
was map[string]string) []missingImage {
lacks := map[string]bool{} // reference → the store does not hold it; absent: not asked, or no answer
asked := map[string]bool{}
lacking := map[string]string{} // module → the image
for _, r := range s.Resources {
if fmt.Sprint(r["type"]) != "container" {
continue
}
image, _ := r["image"].(string)
reference := imageInStore(image, address)
module := s.moduleOf[fmt.Sprint(r["id"])]
if reference == "" || module == "" || lacking[module] != "" {
continue
}
if !asked[reference] {
asked[reference] = true
held, err := holds(ctx, reference)
// empty-on-error: the store did not answer, so nothing is held back for a question unanswered
lacks[reference] = err == nil && !held
}
if lacks[reference] {
lacking[module] = image
}
}
if len(lacking) == 0 {
return nil
}
kept := s.Resources[:0:0]
for _, r := range s.Resources {
if _, out := lacking[s.moduleOf[fmt.Sprint(r["id"])]]; !out {
kept = append(kept, r)
}
}
s.Resources = kept
if s.leftOutWhy == nil {
s.leftOutWhy = map[string]string{}
}
var found []missingImage
for module, image := range lacking {
s.leftOutWhy[module] = "its image " + image + " is not in the artifact store"
if !slices.Contains(s.LeftOut, module) {
s.LeftOut = append(s.LeftOut, module)
}
if s.Builds != nil {
if prior, sent := was[module]; sent {
s.Builds[module] = prior
} else {
delete(s.Builds, module)
}
}
found = append(found, missingImage{Node: node, Module: module, Image: image})
}
sort.Strings(s.LeftOut)
sort.Slice(found, func(i, j int) bool { return found[i].Module < found[j].Module })
return found
}
// withoutMissingImages is composeSend's last step: the store asked about the images the declaration names, the modules
// whose image it lacks left out and kept in the ledger S25 reads. A store that cannot be found or asked leaves the
// declaration as composed.
func withoutMissingImages(ctx context.Context, open *stores, node string, s sendable) sendable {
inv := open.inventory
shelf, err := inv.Catalogue(ctx)
if err != nil {
return s
}
address, err := artifactStoreAddress(ctx, inv, shelf, node)
if err != nil || address == "" {
return s
}
sent, known, err := inv.SentBuilds(ctx, node)
if err != nil || !known {
sent = nil
}
store := artifacts.Store{Address: address}
found := holdBackMissingImages(ctx, node, &s, address, store.HoldsManifest, sent)
for _, m := range found {
fmt.Printf("%s: %s is left out of the declaration: its image %s is not in the artifact store\n", node, m.Module, m.Image)
}
return s
}
// imageFacts are the images S25 raises: a module whose registered build names an image the store lacks, on each machine
// that runs it.
type imageFacts struct {
missingImage
}
// imagesHeld remembers the images the store was found to hold: a stored image is named by its digest and stays what it
// is, so it is asked about once; one it lacks is asked again at every look.
var imagesHeld = struct {
sync.Mutex
held map[string]bool
}{held: map[string]bool{}}
// gatherMissingImages is every machine's module whose registered build — the one a send would carry — names an image of
// the artifact store that the store does not hold. Read from the catalogue and the store, never from what a send did:
// a push is a command of its own, and the serving controller's watchdog hears nothing of it.
func gatherMissingImages(ctx context.Context, inv *inventory.Inventory, holds storeHolds) ([]imageFacts, error) {
shelf, err := inv.Catalogue(ctx)
if err != nil {
return nil, fmt.Errorf("the catalogue cannot be read: %w", err)
}
var out []imageFacts
for _, module := range sortedKeysOf(modulesOf(shelf)) {
for _, r := range shelf[module].Resources {
if fmt.Sprint(r["type"]) != "container" {
continue
}
image, _ := r["image"].(string)
if _, ours := catalogue.InArtifactStore(image); !ours || !strings.Contains(image, "@sha256:") {
continue
}
imagesHeld.Lock()
known := imagesHeld.held[image]
imagesHeld.Unlock()
if !known {
held, err := holds(ctx, image)
if err != nil {
return nil, fmt.Errorf("the artifact store cannot be asked about %s: %w", image, err)
}
if held {
imagesHeld.Lock()
imagesHeld.held[image] = true
imagesHeld.Unlock()
continue
}
} else {
continue
}
running, err := inv.Running(ctx, module)
if err != nil {
return nil, err
}
for _, node := range running {
out = append(out, imageFacts{missingImage{Node: node, Module: module, Image: image}})
}
}
}
return out, nil
}
// modulesOf is the shelf's modules as a map of names, for sortedKeysOf.
func modulesOf(shelf map[string]catalogue.Manifest) map[string]string {
out := make(map[string]string, len(shelf))
for m := range shelf {
out[m] = m
}
return out
}
// watchMissingImages is S25: a module left out of a machine's declaration because the store lacks its image, said at
// once. The condition ends when the store holds the image, or a send names another.
func watchMissingImages(f *signalFacts) []conditions.Observation {
var out []conditions.Observation
for _, m := range f.images {
out = append(out, conditions.Observation{Scope: conditions.ScopeModule, ID: m.Module + "." + m.Node, Kind: kindImageMissing,
Machine: m.Node, Severity: conditions.Warning, Resolver: conditions.ResolverOperator,
Summary: fmt.Sprintf("%s is left out of every send to %s: its declaration names an image that the artifact store does "+
"not hold (the evidence names it); the running container is untouched", m.Module, m.Node),
Said: "image " + m.Image + " is not in the store",
Headline: clipWords(conditions.Capital(m.Module)+" is not updated on "+m.Node, conditions.HeadlineMax),
Explanation: clipWords(fmt.Sprintf("The new version of %s names a container image that the mesh's store does not "+
"hold, so it was not sent to %s. What runs there is untouched, and the rest of %s is sent as usual.",
m.Module, m.Node, m.Node), conditions.ExplanationMax),
Needs: fmt.Sprintf("build %s again; the details name the image.", m.Module),
Resolved: "Resolved: the store holds the image of " + m.Module + " again"})
}
return out
}
// gatherMissingImagesHere is gatherMissingImages asking the mesh's own artifact store; nothing to ask where the mesh has
// none on its network yet.
func gatherMissingImagesHere(ctx context.Context, inv *inventory.Inventory) ([]imageFacts, error) {
shelf, err := inv.Catalogue(ctx)
if err != nil {
return nil, fmt.Errorf("the catalogue cannot be read: %w", err)
}
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return nil, err
}
if address == "" {
return nil, nil
}
return gatherMissingImages(ctx, inv, artifacts.Store{Address: address}.HoldsManifest)
}
+125
View File
@@ -0,0 +1,125 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
)
// **No declaration is sent that names an image the artifact store does not hold** (novox/hq issue 1742).
const storeAt = "novox.internal:5100"
// automx is the image the send of 2026-10-21 17:13 named and the store lacked; the others are held.
const (
automxImage = storeAt + "/mailu/automx@sha256:c62150b8e2cbcc18cb911b098c2877f928761a1e6251667f39ea18251ea35591"
imapImage = storeAt + "/mailu/imap@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
)
func aMailSend() sendable {
return sendable{
Resources: []map[string]any{
{"id": "mailu.automx", "type": "container", "image": automxImage},
{"id": "mailu.automx-config", "type": "file", "path": "/etc/automx.conf"},
{"id": "mailu.imap", "type": "container", "image": imapImage},
{"id": "other.web", "type": "container", "image": "docker.io/library/nginx@sha256:" + strings.Repeat("b", 64)},
{"id": "dns.dnsmasq", "type": "container", "image": storeAt + "/dnsmasq/main@sha256:" + strings.Repeat("c", 64)},
},
moduleOf: map[string]string{"mailu.automx": "automx", "mailu.automx-config": "automx", "mailu.imap": "mailu",
"other.web": "other", "dns.dnsmasq": "dnsmasq"},
Builds: map[string]string{"automx": "b2f5ebb2new", "mailu": "m2", "other": "o1", "dnsmasq": "d1"},
}
}
func storeHolding(missing ...string) (storeHolds, *[]string) {
var asked []string
return func(_ context.Context, reference string) (bool, error) {
asked = append(asked, reference)
for _, m := range missing {
if strings.Contains(reference, m) {
return false, nil
}
}
return true, nil
}, &asked
}
// 1742/AC1: given a send naming automx's image, which the store does not hold, when the declaration is judged, then
// every resource of automx is out of it, automx is among the modules left out with the reason, its build is the one the
// machine was last sent, and every other module — an image of another registry among them — is as composed.
func TestAModuleWhoseImageTheStoreLacksIsLeftOutOfTheSend(t *testing.T) {
s := aMailSend()
holds, asked := storeHolding("mailu/automx@")
found := holdBackMissingImages(t.Context(), "novox", &s, storeAt, holds, map[string]string{"automx": "b2f5ebb2old"})
if len(found) != 1 || found[0].Module != "automx" || found[0].Node != "novox" || found[0].Image != automxImage {
t.Fatalf("found %+v", found)
}
var ids []string
for _, r := range s.Resources {
ids = append(ids, r["id"].(string))
}
if got := strings.Join(ids, ","); got != "mailu.imap,other.web,dns.dnsmasq" {
t.Errorf("the declaration keeps %s", got)
}
if len(s.LeftOut) != 1 || s.LeftOut[0] != "automx" || !strings.Contains(s.leftOutWhy["automx"], "not in the artifact store") {
t.Errorf("left out %v because %v", s.LeftOut, s.leftOutWhy)
}
if s.Builds["automx"] != "b2f5ebb2old" || s.Builds["mailu"] != "m2" || s.Builds["other"] != "o1" {
t.Errorf("builds %v", s.Builds)
}
for _, a := range *asked {
if strings.Contains(a, "docker.io") {
t.Errorf("the store was asked about an image of another registry: %s", a)
}
}
}
// 1742/AC2: given a module never sent to the machine, when its image is missing, then it has no build in the record of the
// send; given the store does not answer, then nothing is held back for a question unanswered.
func TestAnUnansweredStoreHoldsNothingBackAndANewModuleHasNoBuild(t *testing.T) {
s := aMailSend()
holds, _ := storeHolding("mailu/automx@")
found := holdBackMissingImages(t.Context(), "novox", &s, storeAt, holds, nil)
if len(found) != 1 {
t.Fatalf("%+v", found)
}
if _, has := s.Builds["automx"]; has {
t.Errorf("a module never sent is recorded as sent: %v", s.Builds)
}
s = aMailSend()
silent := func(context.Context, string) (bool, error) { return false, errors.New("registry: connection refused") }
if found := holdBackMissingImages(t.Context(), "novox", &s, storeAt, silent, nil); len(found) != 0 || len(s.LeftOut) != 0 ||
len(s.Resources) != 5 {
t.Errorf("an unanswered store held back %+v, left out %v", found, s.LeftOut)
}
}
// 1742/AC3: a module left out for its image raises a condition for the operator naming the module, the machine and the
// image in the evidence, plain words, and no answer to give; a mesh whose images are all held raises nothing.
func TestAMissingImageIsRaisedForTheOperator(t *testing.T) {
f := calm(time.Date(2026, 10, 21, 17, 40, 0, 0, time.UTC))
if got := watchMissingImages(f); len(got) != 0 {
t.Fatalf("%+v", got)
}
f.images = []imageFacts{{missingImage{Node: "novox", Module: "automx", Image: automxImage}}}
got := watchMissingImages(f)
if len(got) != 1 {
t.Fatalf("%+v", got)
}
o := got[0]
if o.Kind != "image-missing" || o.Key() != "module.automx.novox.image-missing" || o.Resolver != conditions.ResolverOperator ||
o.Severity != conditions.Warning || o.Machine != "novox" {
t.Fatalf("%+v (%s)", o, o.Key())
}
if !strings.Contains(o.Said, "c62150b8") || !strings.Contains(o.Headline+o.Explanation, "automx") {
t.Errorf("words: %q / %q / %q", o.Headline, o.Explanation, o.Said)
}
w := conditions.Words{Headline: o.Headline, Explanation: o.Explanation, Resolved: o.Resolved, Needs: o.Needs}
if why, ok := conditions.PlainWords(w, "novox"); !ok {
t.Errorf("not plain: %s", why)
}
}
+4 -4
View File
@@ -31,7 +31,7 @@ func TestAWalkWaitingOnAHeldDeliveryNamesTheHold(t *testing.T) {
"check did not pass for the heads that merged — a person decides: `mesh-delivery.release` (id " +
"novox/mesh-sdk@d41d255f28ad, why) lets it go, `mesh-delivery.stop` ends it"
waits := walkWaits{p.ID: {{ID: "novox/mesh-sdk@d41d255f28ad", State: "held", Walk: p.ID, Says: held}}}
line := planLineSaid(p, now, pauseView{}, tierAtLeast, nil, waits)
line := planLineSaid(p, now, tierAtLeast, nil, waits)
for _, want := range []string{"held for a person", "composed check did not pass", "mesh-delivery.release"} {
if !strings.Contains(line, want) {
t.Fatalf("the line does not say %q: %q", want, line)
@@ -41,7 +41,7 @@ func TestAWalkWaitingOnAHeldDeliveryNamesTheHold(t *testing.T) {
t.Fatalf("a held walk's line points at a hand start: %q", line)
}
// Told nothing — an older mesh-delivery, or none answering — it reads as before.
if line := planLineSaid(p, now, pauseView{}, tierAtLeast, nil, nil); !strings.Contains(line, "starts it by hand") {
if line := planLineSaid(p, now, tierAtLeast, nil, nil); !strings.Contains(line, "starts it by hand") {
t.Fatalf("told nothing, the line reads %q", line)
}
}
@@ -51,7 +51,7 @@ func TestAWalkWaitingOnADeliveryNotAtItsTurnSaysSo(t *testing.T) {
p := waitingWalk(now)
waits := walkWaits{p.ID: {{ID: "novox/mesh-sdk@9747f3e1b1ac", State: "published", Walk: p.ID,
Says: "not let go yet: novox/mesh-tools@3f0c1a6cec18 (published) carried by it not at its turn"}}}
line := planLineSaid(p, now, pauseView{}, tierAtLeast, nil, waits)
line := planLineSaid(p, now, tierAtLeast, nil, waits)
if !strings.Contains(line, "not at its turn") || !strings.Contains(line, "novox/mesh-tools@3f0c1a6cec18") {
t.Fatalf("the line reads %q", line)
}
@@ -69,7 +69,7 @@ func TestAWalkHeldByNothingSaysWhyAndTheHandStart(t *testing.T) {
waits := walkWaits{p.ID: {{ID: "novox/mesh-media-catalog@cdcdcdcd0000", State: "unheld", Walk: p.ID,
Says: "held by nothing: its merges' deliveries ended as nothing-to-move (novox/mesh-media-catalog@abababab0000), " +
"yet the walk moves nzbgeek, nzbget — nothing gives it its word"}}}
line := planLineSaid(p, now, pauseView{}, tierAtLeast, nil, waits)
line := planLineSaid(p, now, tierAtLeast, nil, waits)
if !strings.Contains(line, "ended as nothing-to-move") || !strings.Contains(line, "plans go "+p.ID) {
t.Fatalf("the line reads %q", line)
}
+1 -1
View File
@@ -35,7 +35,7 @@ func TestAMergeSaysTheModulesWhoseBuildSourceItLeftUnchanged(t *testing.T) {
{Own: true, Paths: []string{"modules/build-agent/Dockerfile", "modules/build-agent/module.json"}},
},
"route-proxy": {
{Repository: "novox/mesh-controller", Ref: "main", Paths: []string{"go.mod", "examples/route-proxy/"}},
{Repository: "novox/mesh-controller", Ref: "main", Paths: []string{"go.mod", "modules/route-proxy/cmd/route-proxy/"}},
{Own: true, Paths: []string{"modules/route-proxy/Dockerfile", "modules/route-proxy/module.json"}},
},
}
+623
View File
@@ -0,0 +1,623 @@
package main
import (
"archive/tar"
"bytes"
"compress/gzip"
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"path"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/catalogue"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/artifacts"
"github.com/novox/mesh-controller/internal/builderkeys"
"github.com/novox/mesh-controller/internal/buildreads"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The take-in from the builder's done key (novox/hq ADR 0501, ADR 0498 §8, ADR 0453 §5; issue 1851).
//
// From R4e the controller asks no build but the builder's own module's: a module of a merge is taken from the builder's
// key for the fingerprint the indexer's trunk value of the module names (ADR 0501 §2). Only a done key whose every
// artifact names a reference is taken (ADR 0498 §8); a key not done yet, one waiting for a reference, a templates
// archive the store does not answer, or a list of predefined keys the forge does not answer is waited for and said,
// never failed and never delivered from. A failed key fails, in the builder's own words. The take-in then records the
// key as a build of today's record layout, so a controller of before reads it on a rollback, and judges it as every
// build is judged (registerJudged).
// errKeyWaits says a take-in from a key waits: for the builder, a reference, the store or the forge. Never a failure.
var errKeyWaits = errors.New("waits")
// errKeyFailed says the module's key failed: the builder's own words follow.
var errKeyFailed = errors.New("the builder's key failed")
// keyReads is what a take-in from a key reads of the builder and the indexer (builderkeys.Reader, or a test's).
type keyReads interface {
Build(ctx context.Context, module, fp string) (builderkeys.Key, error)
Manifest(ctx context.Context, digest string) (sdkdelivery.Manifest, error)
}
// keyBuildID is the id a key's build is recorded under: the key and its revision, so the same revision recorded twice
// is one record, and a rebuild of the same fingerprint (a new revision, ADR 0429 §18) another.
func keyBuildID(k builderkeys.Key) string {
part, err := sdkdelivery.FingerprintKeyPart(k.Fingerprint)
if err != nil {
part = strings.TrimPrefix(k.Fingerprint, sdkdelivery.SourcePrefix2)
}
return fmt.Sprintf("key-%s-%d", part, k.Revision)
}
// readArchive is the archive a key's reference names, read from the artifact store and checked against its digest. A
// variable so a test needs no store.
var readArchive = func(ctx context.Context, inv *inventory.Inventory, reference string) ([]byte, error) {
shelf, err := inv.Catalogue(ctx)
if err != nil {
return nil, err
}
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return nil, err
}
if address == "" {
return nil, errors.New("the artifact store is on no network the controller reaches")
}
return artifacts.Store{Address: address}.GetBlob(ctx, reference)
}
// holdArtifact holds what a key's reference names in the artifact store: Gone when the store does not hold it, an
// archive held by its manifest besides. A variable so a test needs no store.
var holdArtifact = func(ctx context.Context, inv *inventory.Inventory, reference string) error {
shelf, err := inv.Catalogue(ctx)
if err != nil {
return err
}
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return err
}
if address == "" {
// A mesh being raised has no store on its network yet (as the sweep, which collects nothing there): nothing
// can be held, and waiting for the store would keep the store itself from being taken in. Said, never silent.
fmt.Printf("the artifact store is on no network the controller reaches, so %s is taken in without being held\n",
reference)
return nil
}
store := artifacts.Store{Address: address}
if err := store.Holds(ctx, reference); err != nil {
return err
}
_, err = store.Hold(ctx, reference)
return err
}
// predefinedHead is the list of predefined keys at the head of the catalogue repository's trunk, and that head.
type predefinedHead struct {
List catalogue.PredefinedList
Repository string
Commit string
// Generation orders two heads of the trunk: the head's commit time in seconds. A list kept from a build's
// generation (its commit count) is always older than a head read here.
Generation int
}
// predefinedAtTrunk is the list of predefined keys at the head of the catalogue repository's trunk (ADR 0501 §8), read
// from the forge. A variable so a test needs no forge.
var predefinedAtTrunk = func(ctx context.Context, inv *inventory.Inventory) (predefinedHead, error) {
entries, err := inv.Catalogued(ctx)
if err != nil {
return predefinedHead{}, err
}
return readPredefinedHead(ctx, entries, formsOfSources(ctx, inv))
}
// formsOfSources reads how the sources are spelled on this mesh; a variable so a test names the forge's address.
var formsOfSources = newSourceForms
// errNoCatalogueRepository says no module of the mesh is built from the catalogue's repository on the forge: not a wait,
// since no read of the forge makes one appear (issue 1902); the plan fails with these words.
var errNoCatalogueRepository = errors.New("no module of the catalogue names the repository " +
catalogue.CatalogueRepository + " on the forge")
// catalogueRepositoryAmong is the catalogue's repository on the forge, as owner/name, found as catch-up finds the trunks
// of the repositories the catalogue builds from: a source in the URL form or the seat form, `.git` dropped (issue 1902).
func catalogueRepositoryAmong(entries []inventory.Entry, forms *sourceForms) (string, error) {
for _, e := range entries {
if e.Source.Repository == "" {
continue
}
owner, name, onForge := forms.onTheForge(e.Source.Repository, e.Source.Seat)
if onForge && strings.EqualFold(name, catalogue.CatalogueRepository) {
return owner + "/" + name, nil
}
}
return "", errNoCatalogueRepository
}
// readPredefinedHead is the list at the trunk head of the catalogue's repository, found among the modules' sources and
// read from the forge.
func readPredefinedHead(ctx context.Context, entries []inventory.Entry, forms *sourceForms) (predefinedHead, error) {
// A URL source is on the forge only once the forge's address is known; an address that cannot be read is a wait.
if _, err := forms.forgeAddress(ctx, forgeAddressWait); err != nil {
return predefinedHead{}, fmt.Errorf("the forge's address cannot be read: %w", err)
}
repository, err := catalogueRepositoryAmong(entries, forms)
if err != nil {
return predefinedHead{}, err
}
var heads []struct {
SHA string `json:"sha"`
Commit struct {
Committer struct {
Date time.Time `json:"date"`
} `json:"committer"`
} `json:"commit"`
}
if err := askGitea(ctx, "gitea_api", map[string]any{"path": "/repos/" + repository +
"/commits?limit=1&stat=false&verification=false&files=false"}, &heads); err != nil {
return predefinedHead{}, err
}
if len(heads) == 0 || heads[0].SHA == "" {
return predefinedHead{}, fmt.Errorf("the forge named no head of %s's trunk", repository)
}
head := predefinedHead{Repository: repository, Commit: heads[0].SHA,
Generation: int(heads[0].Commit.Committer.Date.Unix())}
var file struct {
Content string `json:"content"`
Encoding string `json:"encoding"`
}
if err := askGitea(ctx, "gitea_api", map[string]any{"path": "/repos/" + repository + "/contents/" +
catalogue.PredefinedFile + "?ref=" + head.Commit}, &file); err != nil {
return predefinedHead{}, err
}
if file.Encoding != "base64" {
return predefinedHead{}, fmt.Errorf("the forge answered %s at %.8s in the encoding %q", catalogue.PredefinedFile,
head.Commit, file.Encoding)
}
raw, err := base64.StdEncoding.DecodeString(file.Content)
if err != nil {
return predefinedHead{}, err
}
if head.List, err = catalogue.ParsePredefined(raw); err != nil {
return predefinedHead{}, fmt.Errorf("%s at %.8s is refused: %w", catalogue.PredefinedFile, head.Commit, err)
}
return head, nil
}
// askGitea asks the forge's module one tool on the bus and reads its result into into.
var askGitea = func(ctx context.Context, tool string, args map[string]any, into any) error {
js, err := aBus()
if err != nil {
return err
}
defer js.Close()
raw, _ := json.Marshal(args)
answer, err := link.Ask(ctx, link.OverNATS{Conn: js.Conn()}, "gitea", tool, raw, 30*time.Second)
if err != nil {
return err
}
if answer.Error != "" {
return fmt.Errorf("gitea.%s: %s", tool, answer.Error)
}
var wrapped struct {
Result json.RawMessage `json:"result"`
}
if err := json.Unmarshal(answer.Result, &wrapped); err != nil {
return err
}
return json.Unmarshal(wrapped.Result, into)
}
// keyTake is one take-in from a key: the module, the indexer's trunk value naming the key's fingerprint, and the
// registration's warrant.
type keyTake struct {
Module string
Trunk sdkdelivery.Module
Ask registrationAsk
// Hand names, by artifact, the references the controller's terminal registers the module at instead of a key's
// (ADR 0526 clause 3, `register --reference`): the key, when the builder has one, supplies the artifacts not named
// here. Nil for every take-in but that one.
Hand map[string]string
}
// takeInKey takes a module in from the builder's key for the fingerprint its trunk value names: records the key as a
// build and registers the module, or says why not. errKeyWaits when it waits, errKeyFailed when the key failed.
func takeInKey(ctx context.Context, inv *inventory.Inventory, keys keyReads, t keyTake) (
catalogue.Manifest, inventory.Build, error) {
head, err := predefinedAtTrunk(ctx, inv)
if errors.Is(err, errNoCatalogueRepository) {
// Fails the plan, loudly, with these words (issue 1902): waiting would never end.
return catalogue.Manifest{}, inventory.Build{}, fmt.Errorf("%s is not judged: %w", t.Module, err)
}
if err != nil {
return catalogue.Manifest{}, inventory.Build{}, fmt.Errorf("%w: the list of predefined keys at the catalogue's "+
"trunk head cannot be read, so %s is not judged: %v", errKeyWaits, t.Module, err)
}
keepPredefinedHead(ctx, inv, head)
previous, err := inv.Builds(ctx, t.Module, 5)
if err != nil {
return catalogue.Manifest{}, inventory.Build{}, err
}
kept, manifest, err := buildOfKey(ctx, keys, t, keyContext{list: head.List, previous: previous,
archive: func(ctx context.Context, reference string) ([]byte, error) { return readArchive(ctx, inv, reference) },
hold: func(ctx context.Context, reference string) error { return holdArtifact(ctx, inv, reference) }})
if err != nil {
return catalogue.Manifest{}, kept, err
}
if err := inv.RecordBuild(ctx, kept); err != nil {
return catalogue.Manifest{}, kept, err
}
recorded := inventory.Source{Repository: t.Trunk.Repository, Seat: gitSeat, Path: keyPath(t.Trunk),
BuiltFrom: t.Trunk.Commit, Head: t.Trunk.Commit, Against: kept.Against}
trunk := "main"
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil {
recorded.Ref = followedBranch(was.Ref)
if recorded.Ref != "" {
trunk = recorded.Ref
}
}
registered, err := registerJudged(ctx, inv, takenBuild{kept: kept, manifest: manifest, recorded: recorded,
ask: t.Ask, path: recorded.Path, trunk: trunk, by: "the builder", what: t.Trunk.Repository,
commit: t.Trunk.Commit})
return registered, kept, err
}
// keyContext is what a key's build is made with besides the key: the list of predefined keys its manifest is judged
// against, the module's recent builds (the bases its last one stood on are carried) and how an archive is read.
type keyContext struct {
list catalogue.PredefinedList
previous []inventory.Build
archive func(ctx context.Context, reference string) ([]byte, error)
// hold holds one artifact the key names in the artifact store, artifacts.Gone when the store does not hold it.
hold func(ctx context.Context, reference string) error
}
// keyPath is a module's directory as the catalogue records it: empty for a module at its repository's root.
func keyPath(m sdkdelivery.Module) string {
if m.Path == sdkdelivery.RootPath {
return ""
}
return strings.Trim(m.Path, "/")
}
// buildOfKey reads the key of the trunk value's fingerprint and makes its build record and the manifest it resolves to
// (ADR 0498 §8, ADR 0501 §1, §8): the module's manifest by the key's digest, judged against the list of predefined keys
// at the catalogue's trunk head, resolved with the key's artifacts.
func buildOfKey(ctx context.Context, keys keyReads, t keyTake, kc keyContext) (
inventory.Build, catalogue.Manifest, error) {
waits := func(format string, a ...any) (inventory.Build, catalogue.Manifest, error) {
return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("%w: "+format, append([]any{errKeyWaits}, a...)...)
}
if t.Trunk.FP2 == "" && t.Hand == nil {
why := t.Trunk.None2
if why == "" {
why = "the indexer wrote no fingerprint of the new form"
}
return waits("%s at %.8s has no fingerprint to take a build by: %s", t.Module, t.Trunk.Commit, why)
}
k, err := keys.Build(ctx, t.Module, t.Trunk.FP2)
if t.Hand != nil {
// **Registered at a reference, no key needed** (ADR 0526 clause 3): the key the builder holds, if any, is the
// start; the references given replace or add its artifacts, and the key counts as done.
k = handKey(t, k)
err = nil
}
if err != nil {
return waits("%s's key cannot be read: %v", t.Module, err)
}
switch {
case k.Failed():
return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("%w: %s", errKeyFailed, k.Says())
case !k.Done():
return waits("%s", k.Says())
case len(k.Unreferenced()) > 0:
return waits("%s", k.Says())
}
src, err := keys.Manifest(ctx, k.Build.Manifest)
if err != nil {
return waits("%s's manifest: %v", t.Module, err)
}
manifest, err := catalogue.ParseManifestWith([]byte(src.JSON), kc.list)
if err != nil {
// **A word the running controller does not read waits for the controller that reads it** (ADR 0501 §7): the
// key stays done, and the controller that reads the word registers it with no second build.
if word := refusedWord(err.Error()); word != "" {
return waits("%s's manifest uses the word %q, which this controller does not read: it is registered "+
"when a controller that reads it runs", t.Module, word)
}
return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("the builder built %s and its manifest is "+
"refused: %w", t.Module, err)
}
if manifest.Module != t.Module {
return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("the key of %s names the manifest of %s",
t.Module, manifest.Module)
}
if t.Hand != nil {
if err := handCoversEvery(manifest, k, t.Hand); err != nil {
return inventory.Build{}, catalogue.Manifest{}, err
}
}
// **Every artifact held in the store before anything is recorded** (novox/hq issue 1933): a definition naming an
// archive the store does not hold is sent, and every machine assigned the module fails its apply on the 404. An
// archive is held by its manifest, so the store's own collector keeps it too.
if kc.hold != nil {
for _, a := range k.Build.Artifacts {
reference := catalogue.Recorded(a.Reference)
if _, ours := catalogue.InArtifactStore(reference); !ours {
continue
}
err := kc.hold(ctx, reference)
switch {
case err == nil, errors.Is(err, artifacts.ErrNotOurs):
case errors.Is(err, artifacts.Gone):
return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("the builder's key of %s names %s for its "+
"artifact %q, which is not in the artifact store: nothing was recorded or registered, and the key "+
"must be built again", t.Module, reference, a.Name)
default:
return waits("the artifact store could not hold %s's artifact %q (%s): %v", t.Module, a.Name, reference, err)
}
}
}
built, err := builtOfKey(ctx, kc.archive, manifest, k)
if err != nil {
return waits("%v", err)
}
resolved, err := manifest.Resolve(built)
if err != nil {
return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("the builder built %s and it does not resolve: %w",
t.Module, err)
}
raw, err := json.Marshal(resolved)
if err != nil {
return inventory.Build{}, catalogue.Manifest{}, err
}
kept := inventory.Build{ID: keyBuildID(k), Repository: t.Trunk.Repository, Module: t.Module,
Commit: t.Trunk.Commit, On: k.Build.By, Path: keyPath(t.Trunk), Manifest: raw,
SourceFingerprint: k.Fingerprint, At: time.Now().UTC()}
// **Read from the manifest, fresh at every key** (the same rule build-agent says, internal/buildreads). Sources are
// left unsaid but for a module whose every artifact names its paths: the planner then reads the module's whole
// directory, over-planning and never under-planning, and a fingerprint that did not change moves nothing (R4e A9).
// **Its build source where its every artifact names its paths** (novox/hq ADR 0499 §1, §5; issue 1758, B1): the
// planner reads a merge's changed files against those paths and the manifest alone, so a change elsewhere in the
// module's directory touches nothing it builds from.
if own := pathsBuildSource(manifest, keyPath(t.Trunk)); own != nil {
kept.Sources = []inventory.BuildSource{{Paths: own}}
}
for _, c := range buildreads.Of(manifest) {
kept.Read = append(kept.Read, inventory.ReadRepository{Repository: c.Repository, Ref: c.Ref})
}
// **The bases its last build stood on, carried** (R4e, lead decision): what the collector keeps a mirrored base by.
// A key names none; the module's newest successful build's are carried so the keep set does not change.
for _, b := range kc.previous {
if b.Worked() && b.Module == t.Module && b.ID != kept.ID {
kept.Against = append([]string(nil), b.Against...)
break
}
}
if t.Hand != nil {
// A build made by hand, said so: the id is its own, so it never reads as the builder's key.
kept.ID, kept.On = "hand-"+keyBuildID(k)+"-"+time.Now().UTC().Format("20060102T150405Z"), "hand"
}
for _, a := range k.Build.Artifacts {
kept.Made = append(kept.Made, inventory.Artifact{Name: a.Name, Kind: kindOf(manifest, a),
Reference: catalogue.Recorded(a.Reference)})
}
return kept, resolved, nil
}
// kindOf is an artifact's kind: the key's, else the manifest's.
func kindOf(m catalogue.Manifest, a sdkdelivery.BuiltArtifact) string {
if a.Kind != "" || m.Build == nil {
return a.Kind
}
for _, d := range m.Build.Artifacts {
if d.Name == a.Name {
return d.Kind
}
}
return ""
}
// builtOfKey is each of a done key's artifacts as Resolve reads it: its reference and digest, a TypeScript bundle's
// launchers by the build modules' one rule (mesh-sdk buildstep.Launchers), and, for the archive a module packs its
// templates into, each template's text read back from the archive the store holds under the key's reference.
func builtOfKey(ctx context.Context, archive func(context.Context, string) ([]byte, error), m catalogue.Manifest,
k builderkeys.Key) (
[]catalogue.Built, error) {
declared := map[string]catalogue.Artifact{}
if m.Build != nil {
for _, a := range m.Build.Artifacts {
declared[a.Name] = a
}
}
templates := ""
if m.Templated() {
templates = catalogue.TemplatesArtifact(m)
}
var out []catalogue.Built
for _, a := range k.Build.Artifacts {
d := declared[a.Name]
b := catalogue.Built{Name: a.Name, Kind: kindOf(m, a), Reference: catalogue.Recorded(a.Reference),
Digest: a.Digest}
if launchers := buildstep.Launchers(d.Language, d.Entrypoints); len(launchers) > 0 {
b.Launchers = launchers
}
if a.Name == templates && templates != "" {
body, err := archive(ctx, b.Reference)
if err != nil {
return nil, fmt.Errorf("the templates archive %q of %s cannot be read from the artifact store: %v",
a.Name, m.Module, err)
}
if b.Texts, err = unpackTexts(body); err != nil {
return nil, fmt.Errorf("the templates archive %q of %s cannot be read back: %v", a.Name, m.Module, err)
}
}
out = append(out, b)
}
return out, nil
}
// unpackTexts is every file of a packed templates archive (gzip over tar), by its path inside it.
func unpackTexts(body []byte) (map[string]string, error) {
zipped, err := gzip.NewReader(bytes.NewReader(body))
if err != nil {
return nil, err
}
defer zipped.Close()
reader := tar.NewReader(zipped)
texts := map[string]string{}
for {
header, err := reader.Next()
if errors.Is(err, io.EOF) {
return texts, nil
}
if err != nil {
return nil, err
}
if header.Typeflag != tar.TypeReg {
continue
}
text, err := io.ReadAll(io.LimitReader(reader, 16<<20))
if err != nil {
return nil, err
}
texts[strings.TrimPrefix(header.Name, "./")] = string(text)
}
}
// keepPredefinedHead keeps the list read at the catalogue's trunk head when it is newer than the one kept, as a build of
// the catalogue's did (ADR 0348 §7), and judges every manifest read again against it. Said, never fatal: the list read
// is the one this take-in judges against either way.
func keepPredefinedHead(ctx context.Context, inv *inventory.Inventory, head predefinedHead) {
kept, err := inv.KeepPredefined(ctx, inventory.PredefinedRecord{Keys: head.List, Repository: head.Repository,
Commit: head.Commit, Generation: head.Generation, Build: "the trunk head " + short(head.Commit)})
if err != nil {
fmt.Printf("the list of predefined keys at %s's trunk head %s is not kept: %v\n", head.Repository,
short(head.Commit), err)
return
}
if kept {
fmt.Printf("the list of predefined keys at %s's trunk head %s is published: %s\n", head.Repository,
short(head.Commit), strings.Join(head.List.Keys(), ", "))
}
if err := loadPredefined(ctx, inv); err != nil {
fmt.Printf("the list of predefined keys kept cannot be read: %v\n", err)
}
}
// openBuilderKeys is the builder's and the indexer's buckets, read on a connection for the moment.
func openBuilderKeys() (builderkeys.Reader, func(), error) {
js, err := aBus()
if err != nil {
return builderkeys.Reader{}, nil, err
}
api, err := jetstream.New(js.Conn())
if err != nil {
js.Close()
return builderkeys.Reader{}, nil, err
}
return builderkeys.Reader{JS: api}, js.Close, nil
}
// pathsBuildSource is the build source of a module whose every artifact names the paths it is built from (ADR 0499
// §1): its manifest and each path, relative to its repository's root, a path read as a file or everything below a
// directory of that name (`<path>/**` holds both). Nil when an artifact names none: the module reads its directory.
func pathsBuildSource(m catalogue.Manifest, dir string) []string {
if m.Build == nil || len(m.Build.Artifacts) == 0 {
return nil
}
in := func(p string) string {
if dir == "" {
return p
}
return dir + "/" + p
}
seen := map[string]bool{}
out := []string{in("module.json")}
for _, a := range m.Build.Artifacts {
if len(a.Paths) == 0 {
return nil
}
for _, p := range a.Paths {
entry := in(strings.Trim(path.Clean(p), "/")) + "/**"
if !seen[entry] {
seen[entry] = true
out = append(out, entry)
}
}
}
return out
}
// handKey is the key a registration at references is made from: the builder's key when it has one (its other artifacts
// stand), with each reference given in place of or beside the key's artifact of that name, and counted done.
func handKey(t keyTake, k builderkeys.Key) builderkeys.Key {
if !k.Found {
k = builderkeys.Key{Module: t.Module, Fingerprint: t.Trunk.FP2, Found: true}
}
build := k.Build
build.State, build.By, build.Error = sdkdelivery.BuildDone, "hand", ""
if build.Manifest == "" {
build.Manifest = t.Trunk.Manifest
}
build.Artifacts = nil
for _, a := range k.Build.Artifacts {
if _, replaced := t.Hand[a.Name]; !replaced {
build.Artifacts = append(build.Artifacts, a)
}
}
names := make([]string, 0, len(t.Hand))
for name := range t.Hand {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
reference := t.Hand[name]
build.Artifacts = append(build.Artifacts, sdkdelivery.BuiltArtifact{Name: name,
Digest: handDigestOf.FindString(reference), Reference: reference})
}
k.Build = build
return k
}
// handCoversEvery refuses a registration at references that names an artifact the manifest does not declare, or leaves
// one it declares without a reference (the key's or a given one).
func handCoversEvery(m catalogue.Manifest, k builderkeys.Key, hand map[string]string) error {
declared := map[string]bool{}
if m.Build != nil {
for _, a := range m.Build.Artifacts {
declared[a.Name] = true
}
}
for name := range hand {
if !declared[name] {
return fmt.Errorf("--reference %s: %s declares no artifact of that name. Nothing was registered", name, m.Module)
}
}
have := map[string]bool{}
for _, a := range k.Build.Artifacts {
have[a.Name] = true
}
var missing []string
for name := range declared {
if !have[name] {
missing = append(missing, name)
}
}
sort.Strings(missing)
if len(missing) > 0 {
return fmt.Errorf("%s declares the artifact(s) %s and no --reference names them, nor does a key of the builder. "+
"Nothing was registered", m.Module, strings.Join(missing, ", "))
}
return nil
}
@@ -0,0 +1,271 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/builderkeys"
"github.com/novox/mesh-controller/internal/inventory"
)
// withTrunkList stands in for the catalogue's trunk head: an empty list of predefined keys, readable, or not.
func withTrunkList(t *testing.T, readable bool) {
t.Helper()
was := predefinedAtTrunk
predefinedAtTrunk = func(context.Context, *inventory.Inventory) (predefinedHead, error) {
if !readable {
return predefinedHead{}, errors.New("the forge did not answer")
}
list, err := catalogue.ParsePredefined([]byte(`{}`))
return predefinedHead{List: list, Repository: "novox/mesh-catalog", Commit: "abcdef0123456789",
Generation: 1 << 40}, err
}
t.Cleanup(func() { predefinedAtTrunk = was })
}
// aKeyFor is a done key of module at fp, of one archive with a reference, and the indexer's trunk value naming it at
// commit in repository/path.
func aKeyFor(module, version, fp, repository, path, commit string) (keyBuckets, sdkdelivery.Module) {
raw := `{"module":"` + module + `","version":"` + version + `","build":{"artifacts":[{"name":"a","kind":"archive","from":"files"}]},` +
`"resources":[{"id":"a","type":"archive","path":"/opt/` + module + `","artifact":"a"}]}`
digest := sdkdelivery.ManifestDigest([]byte(raw))
blob := shaOf(module + version)
k := builderkeys.Key{Module: module, Fingerprint: fp, Found: true, Revision: 1,
Build: sdkdelivery.BuilderBuild{State: sdkdelivery.BuildDone, By: "ace", Manifest: digest,
Artifacts: []sdkdelivery.BuiltArtifact{{Name: "a", Kind: "archive", Digest: blob,
Reference: "artifact-store://" + module + "/a/blobs/" + blob}}}}
return keyBuckets{keys: map[string]builderkeys.Key{module: k}, manifests: map[string]string{digest: raw}},
sdkdelivery.Module{FP2: fp, Manifest: digest, Repository: repository, Commit: commit, Path: path}
}
func fpOf(c byte) string { return "src2:" + strings.Repeat(string(c), 64) }
// A3, A4 (ADR 0498 §8, ADR 0501 §1-§3, §8): a done key of a trunk fingerprint records one build of today's layout,
// whose artifacts are the key's, and registers the module under the same rules of who may register as before.
func TestADoneKeyOfTheTrunkRegistersUnderTheSameRules(t *testing.T) {
open := theCatalogue(t) // sudo from novox/mesh-catalog modules/sudo, mesh-host from novox/mesh-host
ctx := t.Context()
inv := open.inventory
withTrunkList(t, true)
// A known module from the repository and path the catalogue builds it from: registered from its key.
keys, trunk := aKeyFor("sudo", "2", fpOf('a'), "novox/mesh-catalog", "modules/sudo", "c0ffee0123456789")
_, kept, err := takeInKey(ctx, inv, keys, keyTake{Module: "sudo", Trunk: trunk, Ask: registrationAsk{Merge: trunk.Commit}})
if err != nil {
t.Fatalf("0501 §1: a done key of sudo's trunk fingerprint was not taken in: %v", err)
}
if shelf, _ := inv.Catalogue(ctx); shelf["sudo"].Version != "2" {
t.Fatalf("0501 §1: sudo is %q", shelf["sudo"].Version)
}
got, found, err := inv.BuildByID(ctx, kept.ID)
if err != nil || !found || len(got.Made) != 1 || got.Made[0].Reference != kept.Made[0].Reference {
t.Fatalf("0498 §8: the recorded build is %+v (%v)", got, err)
}
if fps, err := inv.BuildSourceFingerprints(ctx, "sudo"); err != nil || fps["c0ffee0123456789"] != fpOf('a') {
t.Fatalf("0498 §8: the recorded build's fingerprint is %v (%v), not the key's", fps, err)
}
// Recording the same key again is one record.
if _, _, err := takeInKey(ctx, inv, keys, keyTake{Module: "sudo", Trunk: trunk, Ask: registrationAsk{Merge: trunk.Commit}}); err != nil {
t.Fatal(err)
}
builds, _ := inv.Builds(ctx, "sudo", 20)
n := 0
for _, b := range builds {
if b.ID == kept.ID {
n++
}
}
if n != 1 {
t.Fatalf("0498 §8: one key taken in twice is %d records", n)
}
// A known module from another repository: refused, as a build of it was (ADR 0266).
other, otherTrunk := aKeyFor("sudo", "evil", fpOf('b'), "agent/sudo", "modules/sudo", "d00d0123456789ab")
if _, _, err := takeInKey(ctx, inv, other, keyTake{Module: "sudo", Trunk: otherTrunk,
Ask: registrationAsk{Merge: otherTrunk.Commit}}); !errors.Is(err, errNotItsSource) {
t.Fatalf("0501 §3: a known module from another repository was taken in: %v", err)
}
// A new module from a repository the catalogue builds from: registered.
zram, zramTrunk := aKeyFor("zram", "1", fpOf('c'), "novox/mesh-catalog", "modules/zram", "c0ffee0123456789")
if _, _, err := takeInKey(ctx, inv, zram, keyTake{Module: "zram", Trunk: zramTrunk, Ask: registrationAsk{Merge: zramTrunk.Commit}}); err != nil {
t.Fatalf("0501 §3: a new module from a built repository was refused: %v", err)
}
// A new module from a repository nothing is built from, with no Approve: refused.
tools, toolsTrunk := aKeyFor("agent-tools", "1", fpOf('d'), "agent/tools", "", "e0e0e0e0e0e0e0e0")
if _, _, err := takeInKey(ctx, inv, tools, keyTake{Module: "agent-tools", Trunk: toolsTrunk,
Ask: registrationAsk{Merge: toolsTrunk.Commit}}); !errors.Is(err, errNotItsSource) {
t.Fatalf("0501 §3: a new repository's module was taken in with no warrant: %v", err)
}
// At the terminal the operator registers it (ADR 0501 §4).
if _, _, err := takeInKey(ctx, inv, tools, keyTake{Module: "agent-tools", Trunk: toolsTrunk,
Ask: registrationAsk{Terminal: true}}); err != nil {
t.Fatalf("0501 §4: the terminal's registration was refused: %v", err)
}
}
// ADR 0501 §2: a key only a candidate names registers nothing: the trunk value names another fingerprint, whose key is
// not there, and the take-in waits.
func TestAKeyOnlyACandidateNamesRegistersNothing(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
withTrunkList(t, true)
keys, trunk := aKeyFor("sudo", "3", fpOf('e'), "novox/mesh-catalog", "modules/sudo", "c0ffee0123456789")
trunk.FP2 = fpOf('f') // the trunk names another fingerprint than the candidate's done key
_, _, err := takeInKey(ctx, open.inventory, keys, keyTake{Module: "sudo", Trunk: trunk, Ask: registrationAsk{Merge: trunk.Commit}})
if !errors.Is(err, errKeyWaits) {
t.Fatalf("0501 §2: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["sudo"].Version != "1" {
t.Fatalf("0501 §2: a candidate's key registered sudo %q", shelf["sudo"].Version)
}
}
// ADR 0501 §8: a take-in that cannot read the list of predefined keys at the catalogue's trunk head waits, says so and
// registers nothing.
func TestATakeInWithoutTheTrunksListWaits(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
withTrunkList(t, false)
keys, trunk := aKeyFor("sudo", "4", fpOf('1'), "novox/mesh-catalog", "modules/sudo", "c0ffee0123456789")
_, _, err := takeInKey(ctx, open.inventory, keys, keyTake{Module: "sudo", Trunk: trunk, Ask: registrationAsk{Merge: trunk.Commit}})
if !errors.Is(err, errKeyWaits) || !strings.Contains(err.Error(), "predefined keys") {
t.Fatalf("0501 §8: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["sudo"].Version != "1" {
t.Fatalf("0501 §8: registered %q", shelf["sudo"].Version)
}
}
// A3 (the lead's decision on Against and Read): a key's record keeps the collector's keep set as it was and its read
// repositories are its manifest's, fresh at every key.
func TestAKeysRecordKeepsTheKeepSetAndReadsItsManifest(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
inv := open.inventory
withTrunkList(t, true)
before, err := inv.KeepSet(ctx)
if err != nil {
t.Fatal(err)
}
raw := `{"module":"sudo","version":"5","build":{"artifacts":[{"name":"img","kind":"image","from":"Dockerfile",` +
`"context":{"repository":"novox/elsewhere","seat":"git"}}]},"resources":[{"id":"run","type":"container","artifact":"img"}]}`
digest := sdkdelivery.ManifestDigest([]byte(raw))
k := builderkeys.Key{Module: "sudo", Fingerprint: fpOf('2'), Found: true, Revision: 1,
Build: sdkdelivery.BuilderBuild{State: sdkdelivery.BuildDone, By: "g14", Manifest: digest,
Artifacts: []sdkdelivery.BuiltArtifact{{Name: "img", Kind: "image", Digest: shaOf("img5"),
Reference: "artifact-store://sudo/img@" + shaOf("img5")}}}}
keys := keyBuckets{keys: map[string]builderkeys.Key{"sudo": k}, manifests: map[string]string{digest: raw}}
trunk := sdkdelivery.Module{FP2: fpOf('2'), Manifest: digest, Repository: "novox/mesh-catalog",
Commit: "c0ffee0123456789", Path: "modules/sudo"}
if _, _, err := takeInKey(ctx, inv, keys, keyTake{Module: "sudo", Trunk: trunk, Ask: registrationAsk{Merge: trunk.Commit}}); err != nil {
t.Fatal(err)
}
after, err := inv.KeepSet(ctx)
if err != nil {
t.Fatal(err)
}
for reference, why := range before {
if !reflect.DeepEqual(after[reference], why) && !strings.Contains(strings.Join(after[reference], ","), strings.Join(why, ",")) {
t.Errorf("the keep set changed for %s: %v, was %v", reference, after[reference], why)
}
}
read, err := inv.ReadRepositories(ctx)
if err != nil {
t.Fatal(err)
}
found := false
for _, r := range read["sudo"] {
if !r.Own && r.Repository == "novox/elsewhere" {
found = true
}
}
if !found {
text, _ := json.Marshal(read["sudo"])
t.Fatalf("the key's record does not read its manifest's context: %s", text)
}
}
// A9 (ADR 0453 §6, ADR 0499 consequences): at the switch, a key's build whose every artifact has the digest of the
// build-agent build the mesh holds (`src1:`) is no move: it is registered with that build's artifacts, and a walk sends
// nothing for it.
func TestAKeyWithTheHeldBuildsDigestsIsNoMove(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
inv := open.inventory
withTrunkList(t, true)
keys, trunk := aKeyFor("sudo", "6", fpOf('6'), "novox/mesh-catalog", "modules/sudo", "c0ffee0123456789")
k := keys.keys["sudo"]
held := inventory.Build{ID: "build-sudo-old", Module: "sudo", Repository: "novox/mesh-catalog", Path: "modules/sudo",
Commit: "0ld0ld0ld0ld0ld0", SourceFingerprint: "src1:" + strings.Repeat("9", 64),
Made: []inventory.Artifact{{Name: "a", Kind: "archive", Reference: k.Build.Artifacts[0].Reference}}}
held.Manifest, _ = json.Marshal(catalogue.Manifest{Module: "sudo", Version: "held"})
if err := inv.RecordBuild(ctx, held); err != nil {
t.Fatal(err)
}
_, kept, err := takeInKey(ctx, inv, keys, keyTake{Module: "sudo", Trunk: trunk, Ask: registrationAsk{Merge: trunk.Commit}})
if err != nil {
t.Fatal(err)
}
same, err := inv.SameSourceCommits(ctx, "sudo", kept.ID)
if err != nil || !same["0ld0ld0ld0ld0ld0"] {
t.Fatalf("A9: the key's build does not stand on the held build of the same digests: %v %v", same, err)
}
if shelf, _ := inv.Catalogue(ctx); shelf["sudo"].Version != "held" {
t.Fatalf("A9: sudo is registered as %q, not with the held build's artifacts", shelf["sudo"].Version)
}
}
// A13 against the store: the switch's list reads the build the mesh holds of each module and the builder's key of its
// trunk fingerprint, and names each module unmoved, moved or not known.
func TestTheSwitchsListReadsTheHeldBuildsAndTheKeys(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
inv := open.inventory
keys, trunk := aKeyFor("sudo", "1", fpOf('b'), "novox/mesh-catalog", "modules/sudo", "c0ffee0123456789")
k := keys.keys["sudo"]
if err := inv.RecordBuild(ctx, inventory.Build{ID: "build-sudo-held", Module: "sudo", Repository: "novox/mesh-catalog",
Commit: "c0ffee0123456789", Made: []inventory.Artifact{{Name: "a", Kind: "archive",
Reference: k.Build.Artifacts[0].Reference}}}); err != nil {
t.Fatal(err)
}
reads := planKeysOf{values: map[string]sdkdelivery.Module{"sudo@": trunk}, keys: map[string]builderkeys.Key{"sudo": k}}
lines, err := switchList(ctx, inv, reads)
if err != nil {
t.Fatal(err)
}
got := map[string]string{}
for _, l := range lines {
got[l.Module] = l.Verdict
}
if got["sudo"] != switchUnmoved || got["mesh-host"] != switchUnknown {
t.Fatalf("A13: the switch's list is %+v", lines)
}
}
// Issue 1902: the take-in with the lookup of the catalogue's repository NOT stubbed - the catalogue's modules are registered
// with a URL source and no seat, as every live row is - reads the list from the forge's wire alone and registers the module.
func TestATakeInFindsTheCataloguesRepositoryInTheURLForm(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
was, wasForms := formsOfSources, askGitea
formsOfSources = func(context.Context, *inventory.Inventory) *sourceForms { return aFormsOnTheForge() }
t.Cleanup(func() { formsOfSources, askGitea = was, wasForms })
aForgeAnsweringTheCatalogue(t)
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "catalogue-anchor", Version: "1"},
inventory.Source{Repository: "http://anchor.internal:20000/novox/mesh-catalog.git", Path: "modules/catalogue-anchor",
Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
keys, trunk := aKeyFor("sudo", "6", fpOf('7'), "novox/mesh-catalog", "modules/sudo", "c0ffee0123456789")
if _, _, err := takeInKey(ctx, open.inventory, keys, keyTake{Module: "sudo", Trunk: trunk, Ask: registrationAsk{Merge: trunk.Commit}}); err != nil {
t.Fatalf("a key was not taken in with the real lookup of the catalogue's repository: %v", err)
}
}
+231
View File
@@ -0,0 +1,231 @@
package main
import (
"archive/tar"
"bytes"
"compress/gzip"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/builderkeys"
"github.com/novox/mesh-controller/internal/inventory"
)
// keyBuckets is the builder's and the indexer's buckets as a take-in reads them, for a test: keys by module, manifests by
// digest.
type keyBuckets struct {
keys map[string]builderkeys.Key
manifests map[string]string
}
func (k keyBuckets) Build(_ context.Context, module, fp string) (builderkeys.Key, error) {
if key, ok := k.keys[module]; ok && key.Fingerprint == fp {
return key, nil
}
return builderkeys.Key{Module: module, Fingerprint: fp}, nil
}
func (k keyBuckets) Manifest(_ context.Context, digest string) (sdkdelivery.Manifest, error) {
raw, ok := k.manifests[digest]
if !ok {
return sdkdelivery.Manifest{}, errors.New("the indexer holds no manifest " + digest)
}
return sdkdelivery.Manifest{Module: "x", JSON: raw}, nil
}
const keyFP = "src2:" + "1111111111111111111111111111111111111111111111111111111111111111"
// aKeyOf is a done key of module x at keyFP for the manifest raw, with these artifacts.
func aKeyOf(raw string, revision uint64, made ...sdkdelivery.BuiltArtifact) (keyBuckets, sdkdelivery.Module) {
digest := sdkdelivery.ManifestDigest([]byte(raw))
k := builderkeys.Key{Module: "x", Fingerprint: keyFP, Found: true, Revision: revision,
Build: sdkdelivery.BuilderBuild{State: sdkdelivery.BuildDone, By: "ace", Manifest: digest, Artifacts: made}}
trunk := sdkdelivery.Module{FP2: keyFP, Manifest: digest, Repository: "novox/mesh-catalog", Commit: "c0ffee0123456789",
Path: "modules/x"}
return keyBuckets{keys: map[string]builderkeys.Key{"x": k}, manifests: map[string]string{digest: raw}}, trunk
}
const tsBundle = `{"module":"x","version":"1","tools":["x_read"],
"build":{"artifacts":[{"name":"tools","kind":"bundle","language":"typescript","entrypoints":["dist/index.js"]},
{"name":"img","kind":"image","from":"Dockerfile","context":{"repository":"novox/other","seat":"git"}}]},
"resources":[{"id":"run","type":"container","artifact":"img"}]}`
func shaOf(b string) string {
sum := sha256.Sum256([]byte(b))
return "sha256:" + hex.EncodeToString(sum[:])
}
// A3 (ADR 0498 §8, ADR 0501 §1): a done key whose every artifact names a reference is one recorded build, its artifacts
// the key's, its launchers by the build modules' rule, its reads from its manifest, the bases its last build stood on
// carried; recorded twice it is one record.
func TestADoneKeyIsOneRecordedBuildOfItsArtifacts(t *testing.T) {
keys, trunk := aKeyOf(tsBundle, 3,
sdkdelivery.BuiltArtifact{Name: "tools", Kind: "bundle", Digest: shaOf("t"),
Reference: "artifact-store://x/tools/blobs/" + shaOf("t")},
sdkdelivery.BuiltArtifact{Name: "img", Kind: "image", Digest: shaOf("i"),
Reference: "artifact-store://x/img@" + shaOf("i")})
previous := []inventory.Build{{ID: "build-1", Module: "x", Against: []string{"artifact-store://base@sha256:aa"}}}
take := keyTake{Module: "x", Trunk: trunk}
kept, resolved, err := buildOfKey(t.Context(), keys, take, keyContext{previous: previous})
if err != nil {
t.Fatal(err)
}
if kept.ID != "key-1111111111111111111111111111111111111111111111111111111111111111-3" {
t.Errorf("the record's id is %s", kept.ID)
}
again, _, _ := buildOfKey(t.Context(), keys, take, keyContext{previous: previous})
if again.ID != kept.ID {
t.Errorf("one key recorded twice is two records: %s and %s", kept.ID, again.ID)
}
if kept.SourceFingerprint != keyFP || kept.Commit != trunk.Commit || kept.Path != "modules/x" || kept.On != "ace" ||
kept.Module != "x" || kept.Repository != "novox/mesh-catalog" {
t.Errorf("the record: %+v", kept)
}
if len(kept.Made) != 2 || kept.Made[0].Reference != "artifact-store://x/tools/blobs/"+shaOf("t") ||
kept.Made[1].Reference != "artifact-store://x/img@"+shaOf("i") || kept.Made[1].Kind != "image" {
t.Errorf("Made is not the key's artifacts: %+v", kept.Made)
}
if len(kept.Read) != 1 || kept.Read[0].Repository != "novox/other" {
t.Errorf("Read is not the manifest's contexts: %+v", kept.Read)
}
if kept.Sources != nil || kept.Mirrored != nil {
t.Errorf("a key's record says build sources or mirrored bases it cannot know: %+v %+v", kept.Sources, kept.Mirrored)
}
if len(kept.Against) != 1 || kept.Against[0] != "artifact-store://base@sha256:aa" {
t.Errorf("the bases the last build stood on are not carried: %v", kept.Against)
}
if len(resolved.Bundles) != 1 || resolved.Bundles[0].Launchers["dist/index.js"] != "dist/index.serve.mjs" {
t.Errorf("the bundle's launchers: %+v", resolved.Bundles)
}
var m map[string]any
if err := json.Unmarshal(kept.Manifest, &m); err != nil || !strings.Contains(string(kept.Manifest), shaOf("i")) {
t.Errorf("the recorded manifest names no image digest: %s", kept.Manifest)
}
}
// A3 (ADR 0498 §8): a key with an artifact naming no reference, a key not done and a missing key are waited for, said;
// a failed key fails in the builder's words; a module with no fingerprint of the new form waits, naming why.
func TestAKeyNotReadyWaitsAndAFailedKeyFails(t *testing.T) {
noRef, trunk := aKeyOf(tsBundle, 1,
sdkdelivery.BuiltArtifact{Name: "tools", Kind: "bundle", Digest: shaOf("t"),
Reference: "artifact-store://x/tools/blobs/" + shaOf("t")},
sdkdelivery.BuiltArtifact{Name: "img", Kind: "image", Digest: shaOf("i")})
take := keyTake{Module: "x", Trunk: trunk}
_, _, err := buildOfKey(t.Context(), noRef, take, keyContext{})
if !errors.Is(err, errKeyWaits) || !strings.Contains(err.Error(), "waits for the reference of img") {
t.Errorf("a key without a reference: %v", err)
}
building := noRef
k := building.keys["x"]
k.Build = sdkdelivery.BuilderBuild{State: sdkdelivery.BuildBuilding, By: "g14"}
building.keys = map[string]builderkeys.Key{"x": k}
if _, _, err := buildOfKey(t.Context(), building, take, keyContext{}); !errors.Is(err, errKeyWaits) ||
!strings.Contains(err.Error(), "building on g14") {
t.Errorf("a building key: %v", err)
}
if _, _, err := buildOfKey(t.Context(), keyBuckets{}, take, keyContext{}); !errors.Is(err, errKeyWaits) ||
!strings.Contains(err.Error(), "no build of x") {
t.Errorf("no key: %v", err)
}
k.Build = sdkdelivery.BuilderBuild{State: sdkdelivery.BuildFailed, Error: "digests differ: ace sha256:1, g14 sha256:2"}
failed := keyBuckets{keys: map[string]builderkeys.Key{"x": k}}
if _, _, err := buildOfKey(t.Context(), failed, take, keyContext{}); !errors.Is(err, errKeyFailed) ||
!strings.Contains(err.Error(), "digests differ: ace sha256:1, g14 sha256:2") {
t.Errorf("a failed key: %v", err)
}
none := take
none.Trunk.FP2, none.Trunk.None2 = "", "no lock"
if _, _, err := buildOfKey(t.Context(), noRef, none, keyContext{}); !errors.Is(err, errKeyWaits) ||
!strings.Contains(err.Error(), "no lock") {
t.Errorf("no fingerprint: %v", err)
}
}
// ADR 0501 §7: a manifest word this controller does not read waits for the controller that reads it; never fails.
func TestAManifestWordTheControllerDoesNotReadWaits(t *testing.T) {
raw := `{"module":"x","version":"1","not-a-word-yet":true,"build":{"artifacts":[{"name":"a","kind":"archive","from":"files"}]},
"resources":[{"id":"a","type":"archive","path":"/opt/x","artifact":"a"}]}`
keys, trunk := aKeyOf(raw, 1, sdkdelivery.BuiltArtifact{Name: "a", Kind: "archive", Digest: shaOf("a"),
Reference: "artifact-store://x/a/blobs/" + shaOf("a")})
_, _, err := buildOfKey(t.Context(), keys, keyTake{Module: "x", Trunk: trunk}, keyContext{})
if !errors.Is(err, errKeyWaits) || !strings.Contains(err.Error(), `"not-a-word-yet"`) {
t.Fatalf("an unread word: %v", err)
}
}
func packed(t *testing.T, files map[string]string) []byte {
t.Helper()
var buf bytes.Buffer
z := gzip.NewWriter(&buf)
w := tar.NewWriter(z)
for name, text := range files {
if err := w.WriteHeader(&tar.Header{Name: name, Mode: 0o644, Size: int64(len(text)), Typeflag: tar.TypeReg}); err != nil {
t.Fatal(err)
}
if _, err := w.Write([]byte(text)); err != nil {
t.Fatal(err)
}
}
if err := w.Close(); err != nil {
t.Fatal(err)
}
if err := z.Close(); err != nil {
t.Fatal(err)
}
return buf.Bytes()
}
// A templated module's texts are read back from the archive the store holds under the key's reference; an archive the
// store does not answer is waited for (the lead's Q1b).
func TestATemplatedModulesTextsAreReadFromItsArchive(t *testing.T) {
raw := `{"module":"x","version":"1","templates":"templates",
"build":{"artifacts":[{"name":"templates","kind":"archive","from":"templates"}]},
"resources":[{"id":"main","type":"file","path":"/etc/x.conf","mode":"0644","template":"templates/x.conf"}]}`
body := packed(t, map[string]string{"x.conf": "listen 1\n"})
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
keys, trunk := aKeyOf(raw, 1, sdkdelivery.BuiltArtifact{Name: "templates", Kind: "archive", Digest: digest,
Reference: "artifact-store://x/templates/blobs/" + digest})
asked := ""
archive := func(_ context.Context, reference string) ([]byte, error) { asked = reference; return body, nil }
_, resolved, err := buildOfKey(t.Context(), keys, keyTake{Module: "x", Trunk: trunk}, keyContext{archive: archive})
if err != nil {
t.Fatal(err)
}
if asked != "artifact-store://x/templates/blobs/"+digest {
t.Errorf("the archive asked: %q", asked)
}
if text, _ := json.Marshal(resolved.Resources); !strings.Contains(string(text), "listen 1") {
t.Errorf("the template's text is not in the file: %s", text)
}
gone := func(context.Context, string) ([]byte, error) { return nil, errors.New("the store does not hold it") }
if _, _, err := buildOfKey(t.Context(), keys, keyTake{Module: "x", Trunk: trunk}, keyContext{archive: gone}); !errors.Is(err, errKeyWaits) ||
!strings.Contains(err.Error(), "cannot be read from the artifact store") {
t.Errorf("an archive the store does not answer: %v", err)
}
}
// The launchers a key's bundle is resolved with are mesh-sdk's one rule (buildstep.Launchers), the names
// mesh-build-node writes: a TypeScript bundle's entrypoint x.js runs as x.serve.mjs, a Go bundle has none.
func TestALaunchersRuleIsTheBuildModules(t *testing.T) {
m, err := catalogue.ParseManifest([]byte(tsBundle))
if err != nil {
t.Fatal(err)
}
k := builderkeys.Key{Build: sdkdelivery.BuilderBuild{Artifacts: []sdkdelivery.BuiltArtifact{{Name: "tools", Digest: "sha256:1",
Reference: "artifact-store://x/tools/blobs/sha256:1"}}}}
built, err := builtOfKey(t.Context(), nil, m, k)
if err != nil {
t.Fatal(err)
}
if built[0].Launchers["dist/index.js"] != "dist/index.serve.mjs" || built[0].Kind != "bundle" {
t.Fatalf("built: %+v", built[0])
}
}
@@ -0,0 +1,317 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"path"
"slices"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// What the tests below take a build in with (R4g): the controller itself registers a module only from the builder's done
// key now (takeInKey), but a great many of these tests only need a module registered with a build recorded, in the form
// the old outcome carried, so that is kept here and nowhere else. It is the controller's take-in of a build-agent outcome
// as it was, less the checks only an outcome needed. Nothing in the controller calls any of it.
// buildFrom turns what a builder said into what the mesh keeps.
//
// **By digest and path, never by where it was pushed** (novox/hq 04-ISSUES/102). The builder
// says `<registry>:<port>/<module>/<artifact>@sha256:…`; the mesh records the artifact-store
// reference and composes the store's address back in where a reference is used. `against` — what
// the build stood on, the catalogue's edge — is recorded the same way, so an edge names a module's
// artifact and not the machine it was pulled from.
func buildFrom(result link.BuildResult) inventory.Build {
kept := inventory.Build{
ID: result.ID, Repository: result.Repository, Ref: result.Ref,
Commit: result.Commit, On: result.On, Failed: result.Failed,
// **What the announcement carries, kept rather than discarded** (novox/hq 04-ISSUES/050).
// The catalogue turns the manifest into requires/provides edges and `against` into build
// edges, and it is not always listening when a build happens — on a fresh mesh it cannot
// be, for exactly the modules it needs most. Keeping them is what makes a replay able to
// rebuild the graph rather than a list of names.
Path: result.Path,
// What it was made from (novox/hq issue 280): two builds with one are one build.
SourceFingerprint: result.SourceFingerprint,
SourceInputs: result.SourceInputs,
}
// When it was asked, which is what orders it against another build of the same module
// (novox/hq 04-ISSUES/219) — not when it was heard.
if asked, ok := link.BuildAskedAt(result.ID); ok {
kept.Asked = asked
}
for _, ref := range result.Against {
kept.Against = append(kept.Against, catalogue.Recorded(ref))
}
for _, ref := range result.Mirrored {
kept.Mirrored = append(kept.Mirrored, catalogue.Recorded(ref))
}
for _, r := range result.Read {
kept.Read = append(kept.Read, inventory.ReadRepository{Repository: r.Repository, Ref: r.Ref})
}
// What it was made from, as files (novox/hq ADR 0267): the planner maps the next merge onto it.
for _, s := range result.Sources {
kept.Sources = append(kept.Sources, inventory.BuildSource{Repository: s.Repository, Ref: s.Ref, Paths: s.Paths})
}
var announced []inventory.Artifact
for _, made := range result.Made {
announced = append(announced, inventory.Artifact{
Name: made.Name, Kind: made.Kind, Reference: made.Reference,
})
kept.Made = append(kept.Made, inventory.Artifact{
Name: made.Name, Kind: made.Kind, Reference: catalogue.Recorded(made.Reference),
})
}
kept.Manifest = recordedManifest(result.Manifest, announced)
// The module name comes from the manifest, which only exists when the build got that far.
if len(kept.Manifest) > 0 {
if list, err := judgedWith(result); err == nil {
if m, err := catalogue.ParseManifestWith(kept.Manifest, list); err == nil {
kept.Module = m.Module
}
}
}
return kept
}
// recordedManifest is a build's manifest with the store's address taken off every reference the
// build itself made. Other references — an image a module runs from a public registry — are what
// they were, which is why this rewrites only what `made` names rather than everything that looks
// like an address.
func recordedManifest(raw json.RawMessage, made []inventory.Artifact) json.RawMessage {
announced := map[string]bool{}
for _, a := range made {
announced[a.Reference] = true
}
return withReferences(raw, func(ref string) (string, bool) {
if !announced[ref] {
return ref, false
}
return catalogue.Recorded(ref), true
})
}
// takeIn is what the mesh does with a build's outcome, whoever hears it: the waiting command and
// the daemon that follows the role's events both come here (novox/hq issue 176), so a build's
// result reaches the catalogue whether or not the asker was still listening.
//
// Kept before it is judged. A failed build that leaves no trace is indistinguishable from one
// nobody asked for, and the difference is the whole of whether somebody should be looking at
// something. Then parsed with the same parser a hand-written manifest goes through — a second path
// would be a second thing to disagree about what a manifest is — and registered with where it came
// from: **for a source on a seat, as the path and the seat, never the URL just cloned** (ADR 0111),
// which the request carried and the outcome echoes. A definition naming an installation is refused
// here, where it would enter the catalogue; the build stays recorded and the refusal says which.
//
// Idempotent: the same outcome taken in twice registers the same module twice, which is one row
// written with the same values.
func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResult) (
catalogue.Manifest, inventory.Build, error) {
// The list of predefined keys as the store keeps it now: another controller process may have kept a newer one.
if err := loadPredefined(ctx, inv); err != nil {
return catalogue.Manifest{}, inventory.Build{}, fmt.Errorf("the list of predefined keys kept cannot be "+
"read: %w", err)
}
kept := buildFrom(result)
if err := inv.RecordBuild(ctx, kept); err != nil {
return catalogue.Manifest{}, kept, err
}
if result.Failed != "" {
// The builder's own words. Wrapping them in something about the control plane would put
// two explanations between a person and a build log.
return catalogue.Manifest{}, kept, fmt.Errorf("%s could not build %s:\n%s",
result.On, result.Repository, result.Failed)
}
// Judged first against the list the build was judged against: the catalogue's at the commit built (novox/hq
// ADR 0348 §7), so a module naming a key its own commit adds is read; and again below against the list the
// mesh publishes, before it is registered.
list, err := judgedWith(result)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: %w", result.On,
result.Repository, short(result.Commit), err)
}
manifest, err := catalogue.ParseManifestWith(kept.Manifest, list)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s and what came back is not a manifest: %w",
result.On, result.Repository, err)
}
recorded := inventory.Source{
Repository: result.Repository, Path: result.Path, Ref: result.Ref,
BuiltFrom: result.Commit, Head: result.Commit,
// What it stood on, so registration can judge a built manifest's base (to-be 38 WP2.4).
Against: kept.Against,
// When it was asked, so an older request heard later does not replace a newer one
// (novox/hq 04-ISSUES/219).
Asked: kept.Asked,
}
if result.Source != nil && result.Source.Seat != "" {
recorded.Repository, recorded.Seat = result.Source.Repository, result.Source.Seat
}
// **A build at a commit does not change the branch a module follows** (novox/hq 04-ISSUES/215):
// the commit is built and recorded as what it was built from, and the module keeps following
// what it followed before — the repository's default branch for one new to the catalogue.
if followedBranch(result.Ref) == "" && result.Ref != "" {
recorded.Ref = ""
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil {
recorded.Ref = followedBranch(was.Ref)
}
}
trunk := result.Trunk
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil && followedBranch(was.Ref) != "" {
trunk = followedBranch(was.Ref)
}
if trunk == "" {
trunk = "main"
}
j := takenBuild{kept: kept, manifest: manifest, recorded: recorded, ask: registrationAsk{Build: result.ID}, path: result.Path,
trunk: trunk, by: result.On, what: result.Repository, commit: result.Commit}
// The checks only an outcome of build-agent needs: a key is a trunk build by construction (ADR 0501 §2), in no
// scratch namespace, and carries no list of its own.
j.published = func() error {
// **Only a commit on the trunk is published** (novox/hq ADR 0238): a commit off its repository's
// default branch — a pull request's head, a feature branch built by hand, a `rebuild` or `replay
// --register` of one — is for checking, and is never a module's version; nothing could then send it.
// The branch the module already follows is its trunk — never the branch a build was asked at, or a
// build of a feature branch by name would make that branch its trunk.
follows := ""
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil {
follows = followedBranch(was.Ref)
}
if err := publishable(result, follows); err != nil {
return err
}
return nil
}
j.carried = func() error {
// **The newest trunk build's list of predefined keys is published** (novox/hq ADR 0348 §7): a build of the
// catalogue that may be registered keeps its list when its commit is newer than the one kept.
carried, err := carriedList(result)
if err != nil {
return err
}
if carried != nil {
if err := publishPredefined(ctx, inv, result, carried); err != nil {
fmt.Printf("%s: the list of predefined keys it carried is not published: %v\n", result.ID, err)
}
}
return nil
}
manifest, err = registerJudged(ctx, inv, j)
return manifest, kept, err
}
// errOffTheTrunk is a build of a commit off its repository's trunk, which is never published.
var errOffTheTrunk = errors.New("the commit is not on its repository's trunk: a commit off the trunk is checked, " +
"never published (ADR 0238) — merge it, and the merge builds it")
// publishable says whether a build's outcome may become a module's version: its commit on the module's
// trunk, as the build seat read the forge at the build — the branch the module follows when its source
// names one, else its repository's default branch. A build seat that could not say — one older than the
// rule, building its own successor — is let through and said, so the rule can reach the mesh.
func publishable(result link.BuildResult, follows string) error {
if result.DryRun {
return errors.New("a dry run is never published")
}
if result.Trunk == "" {
fmt.Printf("%s: the build seat did not say whether %s is on its repository's trunk (it predates ADR 0238); "+
"registered as before\n", result.ID, short(result.Commit))
return nil
}
trunk := result.Trunk
if follows != "" {
trunk = follows
}
on := slices.Contains(result.Branches, trunk) || (trunk == result.Trunk && result.OnTrunk)
if !on {
return fmt.Errorf("%w (%s is not on %s)", errOffTheTrunk, short(result.Commit), trunk)
}
return nil
}
// fromTheCatalogue is whether a build is of the catalogue's repository, whose list alone the mesh publishes: by
// its name, in the URL form or the seat form.
func fromTheCatalogue(result link.BuildResult) bool {
for _, repository := range []string{result.Repository, sourceRepository(result)} {
name := strings.TrimSuffix(path.Base(strings.TrimRight(repository, "/")), ".git")
if name == catalogue.CatalogueRepository {
return true
}
}
return false
}
func sourceRepository(result link.BuildResult) string {
if result.Source == nil {
return ""
}
return result.Source.Repository
}
// carriedList is the list a build of the catalogue carried, read strictly; nil for a build of another
// repository, or one that carried none.
func carriedList(result link.BuildResult) (catalogue.PredefinedList, error) {
if result.Predefined == nil || !fromTheCatalogue(result) {
return nil, nil
}
list, err := catalogue.ParsePredefined(result.Predefined.Keys)
if err != nil {
return nil, fmt.Errorf("%s carried a list of predefined keys that is refused: %w", result.ID, err)
}
return list, nil
}
// judgedWith is the list a build's manifest is first judged against at registration: the catalogue's own at the
// commit built, which its build judged it against, where it carried one; the list kept otherwise.
func judgedWith(result link.BuildResult) (catalogue.PredefinedList, error) {
list, err := carriedList(result)
if err != nil || list != nil {
return list, err
}
return catalogue.PublishedPredefined(), nil
}
// publishPredefined keeps a trunk build's list when it is newer than the one kept, says so, and loads what the
// store keeps then — this build's, or a newer one another controller process kept.
func publishPredefined(ctx context.Context, inv *inventory.Inventory, result link.BuildResult,
list catalogue.PredefinedList) error {
kept, err := inv.KeepPredefined(ctx, inventory.PredefinedRecord{Keys: list, Repository: result.Repository,
Commit: result.Commit, Generation: result.Predefined.Generation, Build: result.ID})
if err != nil {
return err
}
if kept {
fmt.Printf("%s: the list of predefined keys at %s (generation %d) is published: %s\n", result.ID,
short(result.Commit), result.Predefined.Generation, strings.Join(list.Keys(), ", "))
} else {
fmt.Printf("%s: the list of predefined keys at %s (generation %d) is not newer than the one kept, which "+
"stands\n", result.ID, short(result.Commit), result.Predefined.Generation)
}
return loadPredefined(ctx, inv)
}
// builds is what the tests of a pending assignment feed an outcome to: the take-in above, then the tick that settles what
// waited for it (settlePending), as the daemon's hearing of an outcome did before R4g (it also moved the plans, which the
// walk now does from its keys). Nothing in the controller calls it.
type builds struct {
inv *inventory.Inventory
open *stores
}
func (b builds) Built(ctx context.Context, result link.BuildResult) error {
_, _, err := takeIn(ctx, b.inv, result)
if b.open != nil {
settlePending(ctx, b.open, time.Now())
}
if result.Failed != "" {
return nil // the build is recorded; whether it registered is what the tests read
}
return err
}
+4 -23
View File
@@ -34,16 +34,6 @@ func deafObservations(now time.Time, held func(broker.Consumer) (uint64, error))
return obs[0].Said, len(obs)
}
// 1063/AC1: a check in flight on a builder is not the controller's event loop being deaf.
func TestDeafLookIgnoresCheckWorkers(t *testing.T) {
now := time.Now()
for _, w := range broker.CheckWorkers() {
if said, n := deafObservations(now, holdingOne(w.Stream+"/"+w.Name)); n != 0 {
t.Fatalf("%s holding one message raised controller-deaf: %s", w.Name, said)
}
}
}
// 1063/AC2: a message held by one of the controller's own consumers is still deaf, naming its stream.
func TestDeafLookStillCountsOwnConsumers(t *testing.T) {
now := time.Now()
@@ -55,25 +45,16 @@ func TestDeafLookStillCountsOwnConsumers(t *testing.T) {
}
}
// 1063/AC3: the list is the controller's consumers and none of the check workers.
// 1063/AC3: the list is the controller's consumers: the mesh defines no other durable consumer the loop could read (the
// check workers went with the checks' queues, ADR 0525 clause 6).
func TestDeafLookReadsExactlyTheControllersOwnConsumers(t *testing.T) {
workers := map[string]bool{}
for _, w := range broker.CheckWorkers() {
workers[w.Stream+"/"+w.Name] = true
}
want := map[string]bool{}
for _, c := range broker.MeshConsumers() {
if !workers[c.Stream+"/"+c.Name] {
want[c.Stream+"/"+c.Name] = true
}
want[c.Stream+"/"+c.Name] = true
}
got := map[string]bool{}
for _, c := range loopConsumers() {
k := c.Stream + "/" + c.Name
if workers[k] {
t.Errorf("the deaf look reads the check worker %s", k)
}
got[k] = true
got[c.Stream+"/"+c.Name] = true
}
if len(got) != len(want) {
t.Fatalf("deaf look reads %v, want %v", got, want)
+61 -124
View File
@@ -8,7 +8,6 @@ package main
import (
"context"
"errors"
"flag"
"fmt"
"os"
@@ -19,7 +18,6 @@ import (
"github.com/novox/mesh-controller/internal/identity"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/licences"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/store"
)
@@ -54,6 +52,8 @@ func run() error {
return fmt.Errorf("no command given")
}
// The bus's users are composed from what each module announced it keeps (novox/hq ADR 0495, issue 1790).
inventory.Announcements = readAnnouncements
// Every connection this process dials says what it is (novox/hq issue 327).
broker.ConnectionName = connectionName(args[0], os.Getenv(verbVar))
@@ -73,10 +73,13 @@ func run() error {
}
switch args[0] {
case "build":
return buildCommand(ctx, args[1:])
case "builder":
return builderCommand(ctx, args[1:])
case "switch":
return switchCommand(ctx, args[1:])
case "register":
return registerCommand(ctx, args[1:])
case "build", "rebuild":
// **The hand build goes with the asks** (novox/hq ADR 0501 §5): refused by name, naming the replacement.
return removedByName(args[0])
case "board":
return boardCommand(ctx, args[1:])
case "api":
@@ -91,25 +94,10 @@ func run() error {
return rehearseCommand(ctx, args[1:])
case "builds":
return buildsCommand(ctx, args[1:])
// The build queue, controlled by hand (novox/hq ADR 0219).
case "queue":
return queueCommand(ctx, args[1:])
case "cancel":
return cancelCommand(ctx, args[1:])
case "clear":
return clearCommand(ctx, args[1:])
case "rebuild":
return rebuildCommand(ctx, args[1:])
case "replay":
return replayCommand(ctx, args[1:])
case "kill":
return killCommand(ctx, args[1:])
case "forget-checks":
return forgetChecksCommand(ctx, args[1:])
case "forget-checks", "queue", "cancel", "clear", "kill", "replay", "pause", "resume":
return removedByName(args[0])
case "try":
return tryCommand(ctx, args[1:])
case "pause", "resume":
return pauseCommand(ctx, args[0], args[1:])
case "collection":
return collectionCommand(ctx, args[1:])
// What the records say the registries may keep, asked on demand (novox/hq ADR 0251).
@@ -152,12 +140,6 @@ func run() error {
// The facts snapshot a merge check is fed (novox/hq to-be 45 §9).
case "facts":
return factsCommand(ctx, args[1:])
// The merge gate: every machine of the snapshot composed with a change (novox/hq to-be 45 §9).
case "merge-gate":
return mergeGateCommand(ctx, args[1:])
// A pull request's merge check run here exactly as the build seat runs it (novox/hq issue 286).
case "check-here":
return checkHereCommand(ctx, args[1:])
case "upgrade":
return upgradeCommand(ctx, args[1:])
// The bus as a planned step (novox/hq to-be 45 §8, ADR 0236).
@@ -232,8 +214,8 @@ func run() error {
}
}
func usage() {
fmt.Fprint(os.Stderr, `mesh-controller — the control plane
// usageText is the terminal's help: every command it has, and none it refuses as removed (removedByName).
var usageText = `mesh-controller — the control plane
On a node the operator types these as 'mesh-cli <command>' (in zsh, 'nox <command>'): asked
through the node's engine, and run as the controller's terminal only on the control-node
@@ -307,17 +289,12 @@ func usage() {
operator key make [--out <file>] make the operator's sealing key, off the mesh; private half to the file only
operator key set <public> [--replace] tell the mesh which operator key to seal to
operator key show the operator key, and what it can recover
build <repository> [--ref R] have a build machine build it, and record what came out
build --behind build every module the mesh holds older than its source
build --on <module> rebuild every module that stands on this module's artifacts, bases first
builds [<module>] what has been built lately, and what came of it
queue [--json] every ask in the build queue: waiting, in flight (where, how long), dead
cancel <id> drop a waiting or dead ask; recorded failed, cancelled by hand
clear [--dead] cancel every waiting ask (and the dead ones); never one in flight
rebuild <module|build-id> ask the module's source again, or that build's, under a new id
replay <build-id> [--register [--older]] that build's commit again; a dry run unless --register
kill <id> end a build where it runs; recorded failed, killed by hand
pause [<node>] / resume [<node>] the build seat's holder there, or every holder, takes nothing new / again
register <module> <owner>/<repository> <path> register a module from a repository the catalogue does not
build, from the builder's done key of its fingerprint (ADR 0501 §4); with
--reference <artifact>=<reference> (repeatable) and --why, at artifacts the store
holds, recorded as a build by hand (ADR 0526 §3)
switch [--json] the switch's list: each module, the build it runs, and what the builder holds
builds [<module>] [-n N] what has been built lately: the builder's done keys, recorded as builds
plans retry <id> ask a failed plan's failed builds again, and carry the plan on
plans stop|close <id> --why <text> end a plan by hand; recorded in the hand-act log
hand-act record <what> --why <text> --cause <word> [--condition <key>]
@@ -334,8 +311,6 @@ func usage() {
durations [--kind K] [--days N] [--json]
apply, heartbeat, plan-tier, build and walk-phase durations
collection [--json] kept archives held/unheld by a manifest, and what the sweep may let go
builder issue <name> a broker account for a build machine, scoped to build work,
delivered as the builder module's broker secret (module add it first)
licence add|list|use|key model access, under the name a person calls it
licence manager <name> <node> the node that holds a refreshable licence's refresh token
licence refresh <name> mint a new access token and seal it to every holder
@@ -350,10 +325,13 @@ func usage() {
version what this binary is
Each context reaches its own store through its own credential (novox/hq ADR 0008), named
`+store.Variable("<context>")+` — or `+store.FileVariable("<context>")+`, naming a file that holds
` + store.Variable("<context>") + ` — or ` + store.FileVariable("<context>") + `, naming a file that holds
the same thing and keeps the password out of the environment. This process holds:
`)
`
func usage() {
fmt.Fprint(os.Stderr, usageText)
for _, c := range held {
fmt.Fprintf(os.Stderr, " %-12s database %-12s from %s\n",
c.name, store.Database(c.name), store.Variable(c.name))
@@ -383,84 +361,6 @@ func parseAround(set *flag.FlagSet, args []string) ([]string, error) {
}
}
// Built is the daemon hearing a build's outcome on the bus — its own asking, an announcement's, or
// a tool's that did not wait (novox/hq issue 176) — and taking it in: recorded, and the module
// registered, the same as the waiting command does. Said either way, so the daemon's log tells what
// became of a build nobody was watching.
func (b builds) Built(ctx context.Context, result link.BuildResult) error {
// **A merge check builds nothing** (novox/hq to-be 45 §9): its verdict is said, and nothing of it is
// recorded or registered.
if result.Check != nil || result.Checked != nil {
if isTry(result) {
// A branch tried before its pull request (novox/hq issue 686): its asker reads the outcome from the
// build's log. Never said as `checked`, so the forge's holder sets no status.
fmt.Printf("%s: a try of %s at %.8s, read with `try --id`: said to nobody, nothing taken in\n", result.ID,
result.Repository, result.Ref)
return nil
}
checked(ctx, result)
return nil
}
// **A dry run is looked at, never taken in** (novox/hq issue 240). On 2026-10-04 a dry run of an
// unmerged branch was heard here like any build, registered, and its definition reached a machine
// before anyone had reviewed it.
if result.DryRun {
fmt.Printf("%s: a dry run of %s on %s, not taken in\n", result.ID, result.Repository, result.Ref)
return nil
}
manifest, _, err := takeIn(ctx, b.inv, result)
// The assignments pending on this build, made or ended (novox/hq issue 325), said in the daemon's log
// after what became of the build.
if b.open != nil {
defer func() {
for _, line := range settlePendingOnBuild(ctx, b.open, result, manifest.Module, err) {
fmt.Printf("%s: %s\n", result.ID, line)
}
}()
}
// When it was asked, so a plan takes as its outcome only a build asked for it or after it
// (novox/hq 04-ISSUES/219). Zero when the id does not say.
asked, _ := link.BuildAskedAt(result.ID)
// And how long it took, asked to heard, which a build's bound will be set from (novox/hq to-be 45
// Phase 0). Said if lost; never a reason not to take the build in.
recordBuildDuration(ctx, b.inv, result, asked)
switch {
case err != nil && result.Failed != "":
fmt.Printf("%s: %v\n", result.ID, err)
if result.Module != "" {
planBuilt(ctx, b.open, result.Module, result.Commit, result.Failed, asked, result.ID)
} else {
planFailedBuild(ctx, b.open, result)
}
return nil
case errors.Is(err, inventory.ErrSuperseded):
// Not a failure: the module is already at what a later request built. A plan that asked
// before that later request is answered by it; one that asked after it ignores this.
fmt.Printf("%s: %v\n", result.ID, err)
planBuilt(ctx, b.open, manifest.Module, result.Commit, "", asked, result.ID)
return nil
case err != nil:
fmt.Printf("%s: heard and recorded, and not registered: %v\n", result.ID, err)
if manifest.Module != "" {
planBuilt(ctx, b.open, manifest.Module, result.Commit, err.Error(), asked, result.ID)
}
return nil
}
fmt.Printf("%s: %s %s registered, built on %s from %s\n",
result.ID, manifest.Module, manifest.Version, result.On, short(result.Commit))
// What the walk's build did, kept on the walk whose ask it answers (novox/hq ADR 0350 §4); a build seat from
// before it says nothing, and that is what is kept.
made := result.MadeWords()
if result.Reused != "" || result.BuiltWhy != "" {
fmt.Printf("%s: %s %s\n", result.ID, manifest.Module, made)
}
saysWhenThePolicyActs(ctx, b.inv, manifest.Module)
planBuiltSaying(ctx, b.open, manifest.Module, result.Commit, "", asked, result.ID, made)
// A module registered may be one a machine is now behind: `status` is composed again.
statusFrom.nudge()
return nil
}
// verbVar carries the seat verb a command runs for, from the serving controller to the process it starts.
const verbVar = "MESH_VERB"
@@ -475,3 +375,40 @@ func connectionName(command, verb string) string {
}
return "mesh-controller command " + command
}
// removedByName refuses a command the controller no longer has, naming what replaced it (novox/hq ADR 0501 §4, §5;
// ADR 0429 §18; ADR 0503 §3; issue 1851, A11, and issue 1318).
func removedByName(command string) error {
replacement := map[string]string{
"build": "the controller asks no build (ADR 0501 §5): the builder builds every fingerprint the indexer writes, a " +
"module is registered from its done key, and a branch is tried with `try`; a module from a repository the " +
"catalogue does not build is registered at the controller's terminal with `register`",
"rebuild": "the controller asks no build (ADR 0501 §5): a failed key is built again by the builder as a new " +
"revision (ADR 0429 §18), and a walk's Retry waits for it",
"replay": "the controller asks no build, so there is nothing to replay (ADR 0501 §5, ADR 0525 clause 6); a branch is " +
"tried with `try`",
"queue": "the controller keeps no build queue (ADR 0525 clause 6): the builds and checks are asks on the build modules' " +
"streams, and the delivery view shows their backlog and the oldest ask's age",
"cancel": "the controller keeps no build queue (ADR 0525 clause 6): an ask is on a build module's stream, and the delivery " +
"view shows the backlog",
"clear": "the controller keeps no build queue (ADR 0525 clause 6): an ask is on a build module's stream, and the delivery " +
"view shows the backlog",
"kill": "the controller keeps no build queue and starts no build (ADR 0525 clause 6): a build runs on a build module, " +
"which ends it",
"pause": "the build seat is retired (ADR 0525 clause 7): a node's builds are paused with its build module's own pause, " +
"or a capacity of 0",
"resume": "the build seat is retired (ADR 0525 clause 7): a node's builds are resumed with its build module's own " +
"resume, or its capacity",
"forget-checks": "build-agent makes no check build from R4e (ADR 0503 §3), so there is nothing to let go of",
"delivery-plan": "a delivery's plan is read with `delivery-order` and the checker's verdict (ADR 0503 §3)",
"delivery-check": "the checker judges every head and every delivery group (ADR 0503 §3, ADR 0527): the controller asks " +
"nothing and says no verdict of a pull request",
}[command]
return &removedCommand{msg: fmt.Sprintf("`%s` is removed: %s", command, replacement)}
}
// removedCommand is a command or verb the controller no longer has: a refusal that is the verb's whole answer, not a
// verb of a newer build this one cannot run.
type removedCommand struct{ msg string }
func (r *removedCommand) Error() string { return r.msg }
+2 -14
View File
@@ -47,18 +47,6 @@ func readBackFromTheCommand(t *testing.T, argv []string, members string) {
}
}
func TestDeliveryCheckHandsALongMembersListInAFile(t *testing.T) { // 1272/AC1
members := aLongGroup(t)
argv, err := argvFor("delivery-check", map[string]any{"group": "feat/x", "members": members})
if err != nil {
t.Fatal(err)
}
if argv[0] != "delivery" || argv[1] != "check" || !slices.Contains(argv, "--group") {
t.Errorf("the command is %.100v", strings.Join(argv, " "))
}
readBackFromTheCommand(t, argv, members)
}
func TestDeliveryOrderHandsALongMembersListInAFile(t *testing.T) { // 1272/AC2
members := aLongGroup(t)
argv, err := argvFor("delivery-order", map[string]any{"members": members})
@@ -73,8 +61,8 @@ func TestDeliveryOrderHandsALongMembersListInAFile(t *testing.T) { // 1272/AC2
func TestAShortMembersListIsStillTheArgumentItWas(t *testing.T) { // 1272/AC3
members := `[{"id":"novox/a#1"},{"id":"novox/b#2"}]`
argv, err := argvFor("delivery-check", map[string]any{"group": "feat/x", "members": members, "ready": "true"})
want := []string{"delivery", "check", "--group", "feat/x", "--members", members, "--ready"}
argv, err := argvFor("delivery-order", map[string]any{"members": members})
want := []string{"delivery", "order", "--members", members}
if err != nil || !slices.Equal(argv, want) {
t.Errorf("a short list gives %v, %v, want %v", argv, err, want)
}
@@ -1,72 +0,0 @@
//go:build integration
package main
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// **A repository's merge check runs every module's unit tests, a Go module of its own below the root included**
// (novox/hq ADR 0420 §5, issue 1025): `./...` stops at a nested go.mod. This runs the script
// check-nested-modules.sh in a tree with a `go` that only records where it was run and fails for a module named bad.
func TestTheMergeCheckVetsAndTestsEveryNestedGoModule(t *testing.T) {
script, err := filepath.Abs("../../check-nested-modules.sh")
if err != nil {
t.Fatal(err)
}
bin, tree := t.TempDir(), t.TempDir()
log := filepath.Join(t.TempDir(), "go.log")
fake := "#!/bin/sh\necho \"$(basename \"$PWD\") $1 GOFLAGS=[$GOFLAGS]\" >>\"$LOG\"\n" +
"[ \"$(basename \"$PWD\")\" = bad ] && [ \"$1\" = test ] && exit 1\nexit 0\n"
if err := os.WriteFile(filepath.Join(bin, "go"), []byte(fake), 0o755); err != nil {
t.Fatal(err)
}
run := func() (string, error) {
cmd := exec.Command("sh", script)
cmd.Dir = tree
cmd.Env = append(os.Environ(), "PATH="+bin+":"+os.Getenv("PATH"), "LOG="+log, "GOFLAGS=-mod=vendor")
out, err := cmd.CombinedOutput()
return string(out), err
}
// No modules directory: a pass, nothing run.
if out, err := run(); err != nil {
t.Fatalf("a repository with no nested module failed: %v\n%s", err, out)
}
if _, err := os.Stat(log); err == nil {
t.Fatal("go ran with no nested module")
}
for _, m := range []string{"good", "bad", "other"} {
dir := filepath.Join(tree, "modules", m)
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "go.mod"), []byte("module x\n"), 0o644); err != nil {
t.Fatal(err)
}
}
// A directory with no go.mod is no Go module of its own.
if err := os.MkdirAll(filepath.Join(tree, "modules", "typescript"), 0o755); err != nil {
t.Fatal(err)
}
out, err := run()
if err == nil {
t.Fatalf("a nested module whose tests fail passed the check:\n%s", out)
}
raw, _ := os.ReadFile(log)
ran := string(raw)
for _, want := range []string{"good vet GOFLAGS=[]", "good test GOFLAGS=[]", "other test GOFLAGS=[]", "bad test GOFLAGS=[]"} {
if !strings.Contains(ran, want) {
t.Errorf("%q was not run, nor outside the vendor directory:\n%s", want, ran)
}
}
if strings.Contains(ran, "typescript") {
t.Errorf("a directory with no go.mod was run:\n%s", ran)
}
if !strings.Contains(out, "nested module modules/bad FAILED") {
t.Errorf("the failing module is not named:\n%s", out)
}
}

Some files were not shown because too many files have changed in this diff Show More