Compare commits

...
Author SHA1 Message Date
mesh-tools 9f0d9098b7 Begin the record of merges at the earliest merge kept on any trunk, not per trunk (ADR 0463 section 2, issue 1336) 2026-10-20 14:49:10 +00:00
mesh-tools 3045af4722 A merge from before the controller kept merges of the trunk is history, never a gap (issue 1336) 2026-10-20 14:38:47 +00:00
mesh-tools 0aa588779d Catch up only a pull request's merge into the trunk, not the merges made inside its branches (issue 1336) 2026-10-20 14:35:58 +00:00
mesh-tools 030bdc537e Read the router's ask ids without taking a second stop of the key lister for a failure (issue 1336) 2026-10-20 14:35:32 +00:00
mesh-tools d163304e3f Merge pull request 'Catch up merges the controller did not hear, on the operator's Approve of each (ADR 0457, issue 1336)' (#407) from fix/1336-catch-up-merges-not-heard into main 2026-10-20 14:07:33 +00:00
mesh-tools 1816e8753b Merge pull request 'Wait out a bus that briefly refuses a build's ask instead of failing the delivery (issue 1334)' (#406) from fix/1334-a-bus-reload-fails-a-tier-2 into main 2026-10-20 13:55:34 +00:00
mesh-tools 970622505b Say a retry asked nothing when every module of it still waits for the bus (issue 1403)
mesh/merge-gate head 970622505b30 merged into trunk b718d24f84b9 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 970622505b30 merged into trunk b718d24f84b9 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 13:44:03 +00:00
mesh-tools 3c7048f24e Give the condition of a merge not heard plain words the operator can act on (ADR 0253, ADR 0457)
mesh/merge-gate head 3c7048f24e55 merged into trunk 1816e8753b07 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 3c7048f24e55 merged into trunk 1816e8753b07 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 13:42:37 +00:00
mesh-tools 9b24913656 Test the teller's warrant by the router's real ids, a gap that adds a module, and short pages (ADR 0457, issue 1336, review round 1) 2026-10-20 13:13:48 +00:00
mesh-tools f25481342f Find the teller's warrant by the heads its Approve is bound to, and judge a gap as hearMerge does (ADR 0457, issue 1336, review round 1) 2026-10-20 13:12:58 +00:00
mesh-tools bde1691f0a Judge a merge in one place for hearing it and for the catch-up (ADR 0457, issue 1336, review round 1) 2026-10-20 13:12:23 +00:00
mesh-tools 29bbe4d682 Test the catch-up from ADR 0457's checks (issue 1336) 2026-10-20 12:59:06 +00:00
mesh-tools 815d60a7e4 Run the catch-up at the controller's start and after each merge it hears (ADR 0457, issue 1336) 2026-10-20 12:58:16 +00:00
mesh-tools 4e0a82010c Catch up on merges the controller did not hear: read the trunk's last fifty merges and hear each gap on the router's warrant (ADR 0457, issue 1336) 2026-10-20 12:57:54 +00:00
mesh-tools b76cbf708a Ask a module again at the next look when the bus did not answer, and fail the plan only after ten minutes (issue 1334)
mesh/merge-gate checking again: its repo-check failed at 12:59 UTC only on TestTheCataloguesDerivedSubjectsMeet, a catalogue fault (mesh-issues' dead subsc…
mesh/repo-check checking again: its repo-check failed at 12:59 UTC only on TestTheCataloguesDerivedSubjectsMeet, a catalogue fault (mesh-issues' dead subsc…
mesh/delivery superseded: a newer head of the same pull request
2026-10-20 12:10:05 +00:00
mesh-tools b718d24f84 Merge pull request 'Take the composition, catalogue and validator from mesh-sdk and delete the controller's copies (design 47 R4b, issues 1337, 1381-1391)' (#405) from feat/r4b-sdk-users into main 2026-10-20 12:03:22 +00:00
mesh-tools 8a330afd44 Say a placeholder or replaced mesh-sdk pin as a local mesh-sdk, not as a version
mesh/merge-gate head 8a330afd44ba merged into trunk 6c2a8d62100c — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 8a330afd44ba merged into trunk 6c2a8d62100c — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-20 11:43:18 +00:00
mesh-tools eb8c313664 Say the walk's state, tier and note when a person's word does not start it, so a flake explains itself 2026-10-20 11:41:29 +00:00
mesh-tools f87590b0f9 Re-make an own secret sealed to a key the machine lost, and name the validator's mesh-sdk pin in a check 2026-10-20 11:33:38 +00:00
mesh-tools 33b285ce09 Make the credentials in four integration tests by a send, not a question, and name the refusal each now meets 2026-10-20 11:30:55 +00:00
mesh-tools 37d61441f8 Remove the rest of the equality test: its comparison of two compositions (issue 1384, 1389)
The controller's own composition it compared mesh-sdk's with is deleted by the commit before; compose_equal_test.go held the comparison (bothWaysDiffer, madeDiffer and their helpers) and is not needed with nothing to compare against. The made type, which compose_perform_integration_test still reads a store through, moves next to madeIn.
2026-10-20 11:19:43 +00:00
mesh-tools 2111b78fc5 Delete the controller's own composition: planFor, declarationWith, renderingFor, grantsFor, theRestOfTheMesh, generators, whoResolves and composedAndValidated, with what only they reached (issue 1389)
Every caller composes through mesh-sdk's go/compose since issues 1386 to 1388; the controller's second copy of the composition is dead code. The parent of this commit is fb556d6b, whose try (build-1792493691274712799, 17m28s, 2026-10-20 11:12 UTC) ran the equality test green on the live facts: every machine was sent and compared both ways (ADR 0453, issue 1384).

Deleted, 40 functions: in plan.go planFor, declarationWith, renderingFor, grantsFor, theRestOfTheMesh and their helpers (boundOfGrant, busCredentialIssued, carriedBuilds, carriedOn, composeBusUsers, engineJudgesRoot, engineReadsHealth, engineTakesHanded, takesHanded, givenUnderAnOldKey, issuedToHolder, keyFor, licencesFor, managerPublicKeyFor, movedProvider, providerMoved, providerModuleOf, sayLeftOut, seatsOn, wireBuilds, zonesInTheMesh); in network.go generators, whoResolves, namesInTheMesh, onThePrivateNetwork; composedAndValidated, adoptionFor, adoptionOf, engineOpensVaultBox, namedModules, localNetworksOf, keepRecorded, passSeatsOn, theMeshPassesSeatsOn, assignmentsOf. Choosing (Reading, Allocating, Foreseeing) is gone: coreBoxesFor and coreBoxesOf take a plain allocating flag. composeForPush no longer takes the private network it ignored. The seat pass of a send is performSeatPasses, over the passes the composition names.

Measured: cmd/mesh-controller non-test code 67,643 -> 65,821 lines (-1,822), its tests 57,691 -> 57,286, with the integration part of the equality test removed here because it calls the deleted functions. Not the issue's 3,000: that counts all of plan.go (2,192 lines), of which the callers, the verbs, the perform step and the sends remain (sendable, recorded builds, the held-back and ready-node logic, the plan verb, the gate's trial); and the 37 functions a name walk lists as unreachable are called through interfaces and method sets and are kept.

Tests dropped, each with the mesh-sdk go/compose test that holds its cases:
- seat_pass_test.go TestACompositionWithTwoClaimantsRecordsNoneAndSaysSo, TestACompositionChangesNothingWhileTheHolderIsAssignedOrNobodyClaims, TestAHolderAssignedButNotOnTheShelfIsNotTakenForGone: rest_test.go TestNoSeatIsPassedForTwoClaimantsAnAssignedHolderOrNoClaimant, TestAHolderAssignedButNotOnTheShelfIsNotTakenForGone, TestASeatWhoseHolderIsAssignedNowhereGoesToItsOnlyClaimant. TestACompositionPassesTheSeatAndWritesOneHandActByTheController and TestALostPassWritesNoActAndAFailedOneLeavesTheRecord stay as tests of the send's step (performSeatPasses); TestAThrowawayMeshPassesTheSeatInMemoryAndWritesNothing stays as TestAThrowawayMeshWritesNoHandActForAPass.
- seat_pass_test.go TestNoSeatIsPassedWhenAMachinesAssignmentsCouldNotBeRead: no counterpart, and none is needed: the adapter reads every machine's assignments for the Input and a machine whose read fails fails the Input, so no seat is passed on a partial read.
- network_test.go TestOnlyAMachineOnThePrivateNetworkIsNamed: network_test.go TestANodeNotOnThePrivateNetworkGetsNoAddressAndNoNetwork.
- held_back_test.go TestASendCarriesTheCurrentBuildsAndALeftOutModuleKeepsItsOwn: node_test.go TestAModuleLeftOutKeepsTheBuildItWasLastSent (and compose_sdk_test.go's table, now literal).
- modules_wire_test.go TestTheBuildOfACarriedCommitIsTheAssignmentsOrTheOneBuiltFromIt: node_test.go TestTheBodyCarriesTheOrderTheInputGivesAndTheBuildsOnlyToAnEngineThatReadsThem.
- compose_equal_integration_test.go: the old path it compares against is deleted; its last run is the parent's try.
Rewritten against mesh-sdk, same cases: issued_test.go (needs now come from compose.Resolve: Needed.IssuedToHolder and ProviderMoved), module_health_test.go and merge_gate_handing tests (the contract in the Input), provisioner_account_integration_test.go (the composition's refusal), one_resolution_integration_test.go (sayFound). Every test of the send, perform, the adapter and the verbs is kept; the tests that used planFor, declarationWith, generators, renderingFor or grantsFor as setup use resolveNode, composeSend, composeQuestion, renderingOf and grantsOf. compose_callers_test.go now fails when any deleted name is declared or called again.

Committed by copying the files of a staging branch (stage/r4b-1389 at 198be8217c, built from fb556d6b by patch commits and compared with the author's worktree), so that this commit's parent is fb556d6b.
2026-10-20 11:18:42 +00:00
mesh-tools fb556d6b74 Make the equality test fail unless every machine of the facts is sent both ways (issue 1384)
A pass printed no line, so it did not show whether idt and ygcyr were compared on the send side or refused alike. A machine either send refuses now fails, naming the machine and both refusals; refusals alike in the Foreseeing comparison (a) stay, said only; facts with no machine fail. A pass proves that every machine was sent and compared.
2026-10-20 10:53:25 +00:00
mesh-tools 11e005fe38 Prepare every send as the controller's composition did: make the own secrets a file names, and ask the vault every time (issues 1385, 1386)
mesh-sdk's composition refuses a module's file that names an own secret not made yet in every mode, where the controller's Allocating composition made it on the way; and it asked the vault on every send, where a send whose kept box was valid composed with no stand-in and so never performed. prepareSend, before the composition of every send, makes the own secrets no row holds (not the handed, not the outside-issued, not one the composition would refuse) and asks the vault for the core's credentials, keeping the boxes it answers and returning the modules the controller seals itself. perform no longer asks the vault.
2026-10-20 10:09:06 +00:00
mesh-tools 157d1599b4 Judge the catalogue's manifests against the predefined list, and put merge-check.sh back as main has it (issue 914)
The seats test I moved to internal/ownmanifest parsed each catalogue manifest alone, so claude-code's managed_settings and dunst's font-size, which fall back to a predefined key, were refused where the check clones the catalogue beside (ADR 0348 section 7). It reads them with ReadTreeManifest from the catalogue's root, as the tests that issue 914 fixed do. merge-check.sh differed from main in a comment only; it is main's again.
2026-10-20 10:08:41 +00:00
mesh-tools 7d949f888f Run the equality test's three stores one after the other, and log where each phase starts (issue 1384)
The live try failed in 32 s: throwawayStores finds its databases by the process environment, and a store opened later finds them there too, so a second store raised at the same time took the first one's database from under it (database gate_inventory_... does not exist). The three stores are raised one after another again, each phase says when it starts, and a failed phase stops the test. The two served-fact tests log one line when they pass their key assertion.
2026-10-20 09:58:49 +00:00
mesh-tools 9f773e0b3c Run the equality test only when named, on three stores at once, with its phases timed (issue 1384)
The live try took 1074 s for it. It now skips unless the tests pattern (-run) names it, which a try sets and a whole-suite merge check does not; its three independent pieces of work (both ways of Foreseeing on a fresh store, the controller's sends and readings, mesh-sdk's sends) run together; mesh-sdk's Input is read once for a phase of questions; and each phase logs its elapsed time, so the next run says where the time goes. A machine refused on either side is left out of the comparison of the send.
2026-10-20 09:54:51 +00:00
mesh-tools b317dfef45 Name only the differing entries of what two sends made, and leave a refused machine out (issue 1384)
The live try cut the lists of the ports allocated and the own secrets made at 2.5 KB, so no difference showed. madeDiffer now says, per kind, the entries made only by the controller, only by mesh-sdk, and by both with a different value; the entries of a machine refused on either side are left out of both, since it sends nothing.
2026-10-20 09:54:06 +00:00
mesh-tools e4d1750015 Seal the stand-in vault box by the resource's id, and set the setting before the machines are assigned in the served-fact tests (issues 1385, 1382)
The live try (build-1792488565282382172) left a stand-in vault box in f35 and yvbpsm: the file was matched by the manifest's path, which the composition fills in for the machine. The seal now matches by the resource's id alone and names any stand-in that is left. The two served-fact tests set the setting before assigning (an assignment resolves the mesh), and the second clears it after.
2026-10-20 09:53:36 +00:00
mesh-tools fb01d514a9 Delete internal/facts: the snapshot's type is mesh-sdk's go/facts (issue 1390, 4 of 4)
Its tests moved with it: mesh-sdk's go/facts holds the same ones, plus the grants of format 2.
2026-10-20 09:41:22 +00:00
mesh-tools b692a95280 Read the facts snapshot with mesh-sdk's go/facts, formats 1 and 2, and keep writing format 1 (issue 1390, 3 of 4)
The merge gate, the build seat's check and the facts writer import the snapshot's type from mesh-sdk (go/facts), whose reader takes formats 1 and 2 and refuses a newer one. The writer keeps writing format 1 (writtenFormat); moving it to 2 is a release of its own (novox/hq ADR 0456). A test shows the writer writes format 1, carries no grants or bindings, and the reader takes both formats and refuses a newer one.
2026-10-20 09:41:16 +00:00
mesh-tools 0e2ab4a96a Vendor mesh-sdk's go/facts and its compose input at ab0c16dd (issue 1390, 2 of 4) 2026-10-20 09:40:34 +00:00
mesh-tools b3e5af0e0c Pin mesh-sdk at the merge of its go/facts format 2 (ab0c16dd, issue 1390, 1 of 4)
go.mod, go.sum, vendor/modules.txt and the capture note name ab0c16ddad26; the vendored files follow in the next commit, and the captured conformance events are the same at the new commit.
2026-10-20 09:40:24 +00:00
mesh-tools 611113fc63 Test that a controller is judged with the validator it vendors and a node-engine member without a validator is judged all the same (issue 1391)
The host member without validate/ failed first against the old judge ("holds no Go files", the validator could not be put in the judge); the controller member is judged by itself whatever the node-engine sibling brings; the group's judge is no longer keyed on the node-engine's validator.
2026-10-20 09:36:41 +00:00
mesh-tools 344b6beb8d Give the judge fixtures mesh-sdk's validator and drop the tests of a validator judge (issue 1391)
The judge fixtures vendor mesh-sdk's validate package, as the controller does; the test of the judge built with a node-engine change's own validator goes with that judge; the replay of issue 867 no longer has the controller change need a validator word of the node-engine's.
2026-10-20 09:36:11 +00:00
mesh-tools 0fc455aa65 Judge a group's controller by itself with the validator it vendors, and copy nothing of mesh-host's (issue 1391, 3 of 3)
judgeFromTheGroup copies nothing into the judge's vendor tree: a mesh-host member whose tree holds no validate/ does not fail the judge, and the verdict names the validator as mesh-sdk's at the pin of the controller that judges. replaceGoFiles and its test are gone.
2026-10-20 09:35:38 +00:00
mesh-tools 52d5e38729 Build no judge with the change's validator for a node-engine change (issue 1391, 2 of 3)
judgeFor judges a node-engine change by the controller the mesh runs, as any module is, and copies nothing of mesh-host into the judge's vendor tree.
2026-10-20 09:35:05 +00:00
mesh-tools 70945de96e Judge a node-engine change by the controller the mesh runs: the scope sets no validator judge (issue 1391, 1 of 2)
pullScope no longer gives a mesh-host change a Judge, and link.JudgeValidator is gone: the validator is mesh-sdk's, at the controller's pin (ADR 0446 clause 5). An older request that says validator is read as the controller the mesh runs.
2026-10-20 09:34:29 +00:00
mesh-tools e98bf1efcb Test the controller's seal under a silent vault, the bus machine's refusal and a served fact that says a setting (issues 1385, 1383, 1382) 2026-10-20 09:28:48 +00:00
mesh-tools b72af5e918 Seal a core credential by the controller where the vault is silent, and refuse the bus machine where no user has a credential (issues 1385, 1383)
Where the vault did not answer for a core credential and no box is kept, ADR 0396 has the controller seal it itself until the vault holds it: perform names those modules, and the send's declaration carries the controller's own seal in place of the stand-in vault box, as the controller's composition sent it. The machine that runs the bus is refused with busUserList's words where no user has a credential, as before; no other machine is. A machine whose recorded module cannot be kept is refused alone.
2026-10-20 09:28:09 +00:00
mesh-tools 09229d85cb Read the settings before the first pass resolves any machine, and refuse one machine alone for what it cannot keep or run (issues 1382, 1383)
The first pass of the adapter resolved every machine with no settings in the Input, so a provider whose served fact says ${setting:domain} could not be settled and the whole Input failed (the equality try on the live facts). The settings of every module that has any are read first; a module's settings that do not read are an error only for a machine that resolves to it. A recorded module that cannot be kept at the build a machine runs refuses that machine alone, as planFor did, and no longer the whole Input. The bus's refusal (no user has a credential) is carried for the machine that runs the bus.
2026-10-20 09:27:31 +00:00
mesh-tools bd17a041de Read the rest of the mesh and the network over mesh-sdk, and fail when a caller reaches for the old composition (issue 1388)
The seats, holdings, source, build-source, delivery group check, check request, backup plan, adoption (take and converge) and the network graph now read the rest of the mesh through restOfTheMesh and compose.Graph. compose_callers_test.go scans the package and names any function other than the old composition's own that still calls planFor, declarationWith, composedAndValidated, generators, renderingFor, theRestOfTheMesh or whoResolves; issue 1389 deletes those.
2026-10-20 09:15:34 +00:00
mesh-tools 7855ea126c Ask over mesh-sdk's composition: a question makes no pair credential and reads the mesh once (issue 1387)
The callers that only ask (status, the doctor's probes, facts, bindings, data, acts, nodes show, images, release, recorded push, consumers) resolve through resolveNode and compose through composeQuestion, over one Input read per operation (withInputCache). A question composes with Reading or Foreseeing and never calls PairCredential's maker: the credential a send would make is named, and perform makes it on a send.
2026-10-20 09:14:25 +00:00
mesh-tools f693d7e171 Send over mesh-sdk's composition: push, the pushes of plans and rollouts, held back, plan and the merge gate's passes (issue 1386)
Every send composes through composeSend (compose over the Input, perform what it named foreseen, read the stores again, compose again, refuse a declaration that still stands in for a value): push.go's two composing sites, composeForPush, declarationFor (the plan verb, which allocates on purpose for one named machine) and the merge gate's two passes and its trial assignment (composeSendValidated, which answers the node-engine's validator beside it). Each machine of a send reads the mesh afresh (freshInput), because the send before it made and recorded things. None calls declarationWith with Allocating any more. The private network is still computed up front, so a send refuses on it before anything is composed. Also here, as they share the helpers: whoResolvesOn, restOfTheMesh and restOfTheMeshOf in compose_sdk.go, and the planFor callers in held_back.go, plan.go and push.go.
2026-10-20 09:12:24 +00:00
mesh-tools 647d20af98 Compose every machine both ways and show the two agree (issue 1384, ADR 0453 clause 3)
TestEveryMachineComposesAlikeBothWays reads the facts snapshot MESH_FACTS names (the live one in every check and try), else testdata/facts-equal.json, else the snapshot of a mesh of its own; raises it into throwaway stores as the merge gate does; and compares, machine by machine: (a) Foreseeing on the fresh store (error or none, the validator's problems, LeftOut and its reasons, Withheld, Unbound, Unplaced, HandedOnly, the own secrets named foreseen, and the ports mesh-sdk names foreseen against the ones the controller's send allocates); (b) the send, on two stores, the controller's Allocating composition and the adapter's compose, perform and compose again (ports, pair credentials and own secrets made, by key); (c) after the send, Reading both ways on one store, byte for byte. A difference names the machine, the comparison and the first resource and field. The comparison itself is tested without a store, and failed first when the body was not read. The test is deleted with the controller's composition (issue 1389).
2026-10-20 09:03:20 +00:00
mesh-tools 8be76b78da Perform what mesh-sdk's composition names foreseen, and compose a send over the SDK: perform, composeSend, composeQuestion, resolveNode (issues 1385, 1386)
A composition over the Input makes nothing; a send performs what it named (the pair credentials no row holds, the seats to pass, the ports and own secrets, the core credentials asked of the vault every send, the certificate), reads the stores again and composes again, and refuses a declaration that still stands in for a value anywhere in its body. A question (composeQuestion) never performs. The operation's Input is read once and kept in its context until a send changes what it read, per what the context keeps (recorded builds, the grants step, a settings walk). A core credential the vault did not answer for and no box is kept for refuses the send, where the controller sealed it itself (ADR 0396): the composition has no such fallback (reported). Callers switch in the next commits.
2026-10-20 09:01:42 +00:00
mesh-tools 7515320b72 Read the credentials and the builds into mesh-sdk's composition input, and make nothing (issue 1383)
How each pair credential a node takes stands (held with its sealed consumer copy, the next send makes it, only a person's to give and nobody gave, or refused in the controller's words), the credentials made for each provider's consumers, the own secrets with their origin and key and value, the vault's kept boxes, the bus's users, minted users and memberships, the mesh's authority and each machine's serving key and kept certificate, the repositories built, the current builds and the build of each commit a machine that reads them is sent. A credential no row holds is not made (planFor made it in every mode): the adapter returns it as would-make for perform (issue 1385). New reads in the inventory (OwnSecretRows, PairCredentialState, BusMinted) and the identity (KnownAuthority, KeptCertificate) write nothing.
2026-10-20 08:58:32 +00:00
mesh-tools 468ecd482e Read the controller's stores once into mesh-sdk's composition input: the nodes and the network (issue 1382)
readComposeInput asks the stores for the Input's mesh-wide and per-node fields (the shelf, each node's place, assignments, capabilities, accounts, pins, bindings, settings, ports, sealing key, taken modules, links, networks, order and builds, the recorded modules it keeps at the build it runs, its node-engine's contract) and writes nothing. A node's kept manifests are its own: the Input's Catalogue stays the current shelf (R4a review round 1, finding 1). Settings are read for the modules each node resolves to, as planFor read them. keepRecorded is split so the adapter reads the kept manifests with its refusals in the same words (applyKept). The credential and build half is issue 1383; vendor/modules.txt now lists go/compose and no longer the mesh-host validator.
2026-10-20 08:55:44 +00:00
mesh-tools 5eaaeb7135 Vendor mesh-sdk's go/compose and drop the node-engine's validator from vendor/ (issues 1381, 1382)
go mod vendor writes go/compose (copied on the forge from mesh-sdk 14fd5cb68fd1, equal to its blobs) now that the adapter imports it, and no longer writes github.com/novox/mesh-host/validate and internal/declaration, which nothing imports since the validator comes from mesh-sdk (issue 1381; the swap commits left those five files behind). vendor/modules.txt follows with the adapter.
2026-10-20 08:38:52 +00:00
mesh-tools b9b5a5e1dc Delete internal/catalogue: mesh-sdk's go/catalogue is the one catalogue package (issue 1381)
Nothing in this repository imports it any more; go/catalogue in mesh-sdk 14fd5cb68fd1 is a copy of it (R4a's review diffed it against 6c2a8d62). The tests that read this repository's own files live on in internal/ownmanifest.
2026-10-20 08:30:50 +00:00
mesh-tools 926444373d Delete the second half of internal/catalogue's tests: mesh-sdk's go/catalogue holds them (issue 1381) 2026-10-20 08:30:25 +00:00
mesh-tools e4364a1c86 Delete the first half of internal/catalogue's tests: mesh-sdk's go/catalogue holds them (issue 1381)
The tests that read this repository's own files (controller_is_a_process, genesis_image, genesis_registry and foundation_manifests' registry test) live on in internal/ownmanifest; engine_guard_test and the rest are in mesh-sdk (issue 1379).
2026-10-20 08:30:14 +00:00
mesh-tools 4d74e06ed8 Delete internal/overlay: mesh-sdk's go/overlay is the one overlay generator (issue 1381)
No Go file imports it any more; go/overlay in mesh-sdk 14fd5cb68fd1 is a copy of it with only import paths and a doc.go changed (R4a's review diffed it).
2026-10-20 08:29:36 +00:00
mesh-tools 928942c78c Take the catalogue from mesh-sdk in internal/recordmirror, finish the own-manifest tests, vendor/modules.txt for the pin, the capture note and the slow-package comment (issue 1381, 6 of 6 import swaps)
With this commit no Go file outside vendor/ and the two packages about to go imports internal/catalogue, internal/overlay or mesh-host/validate, go.mod and vendor/modules.txt agree on mesh-sdk 14fd5cb68fd1, and the tree builds. testdata/beside/CAPTURED names the pinned SDK commit (conformance/events is unchanged since aa68b673). merge-check.sh's comment no longer names internal/catalogue as a slow package.
2026-10-20 08:29:11 +00:00
mesh-tools af267cb557 Take the catalogue from mesh-sdk in internal/inventory and internal/link, and keep the tests that read this repository's own manifest, Dockerfile and Makefile in internal/ownmanifest (issue 1381, 5 of 6 import swaps)
The three tests of the controller's own module.json (process, claimed seat, addresses), the Dockerfile and Makefile pin test, the build-agent registry test and the manifest-in-use seat test move with their old package, as tests of mesh-sdk's exported API; a moved test is exempt from failing first.
2026-10-20 08:27:10 +00:00
mesh-tools bfe7fe1530 Pin mesh-sdk at 14fd5cb6 (R4a's merge) and take the catalogue, overlay and validator from it, s to inventory (issue 1381, 4 of 6 import swaps)
go.mod and go.sum move to mesh-sdk 14fd5cb68fd1, the commit on its main that holds go/catalogue, go/overlay and go/validate (ADR 0453 clause 2); vendor/modules.txt follows in the last of the six commits.
2026-10-20 08:26:01 +00:00
mesh-tools 3208aa0461 Take the catalogue, overlay and validator from mesh-sdk in cmd/mesh-controller, p to s (issue 1381, 3 of 6 import swaps) 2026-10-20 08:24:43 +00:00
mesh-tools 4c84ac992b Take the catalogue, overlay and validator from mesh-sdk in cmd/mesh-controller, g to p (issue 1381, 2 of 6 import swaps) 2026-10-20 08:23:26 +00:00
mesh-tools 013dad4688 Take the catalogue, overlay and validator from mesh-sdk in cmd/mesh-builder and cmd/mesh-controller, a to g (issue 1381, 1 of 6 import swaps)
Swaps three import paths for mesh-sdk's go/catalogue, go/overlay and go/validate (the same packages, copied from this repository at 6c2a8d62) and lets gofmt order the groups. The six swap commits compile together once the pin commit (go.mod, modules.txt) is on the branch; each alone does not, because a file on the SDK's types cannot call one still on the old package.
2026-10-20 08:22:07 +00:00
mesh-tools f64e00e83c Vendor mesh-sdk's catalogue, overlay, validator, declaration and the new delivery parts, copied from its main 14fd5cb6 (issue 1381)
The files go/mod vendor writes for the pin that follows in the next commit, copied on the forge from mesh-sdk at 14fd5cb68fd1 (each equals the blob there; the vendored tree excludes tests). Split from the pin because one commit cannot carry both a patch and copies.
2026-10-20 08:19:32 +00:00
mesh-tools b280b9de10 Wait on the watch having handled the entry, not 2 s twice, in the pull-key watch test (issue 1337)
The unit level ran at 4.4 s of its 5 s bound; this one test took 4.00 s of internal/link's 4.37 s by waiting a fixed 2 s for two things that must not happen. It now waits on the fake watch having handled the entry (two filler entries pass the one-slot channel only after the real one was handled) and on the watches having stopped after a move. Broken in turn (a verdict alone moving the check, a move told twice) it fails.

Unit level at this commit: 2454 tests (6 skipped), 2.84 s (bound 5 s); internal/link 0.36 s. Five slowest tests: TestNoPlanIsWrittenButThroughSavePlansMoving 0.64 s, TestAResolverThatAnswersNothingIsHeldAndSaidInMachineNames 0.48 s, TestNoCodeNamesATableOfTheChain 0.31 s, TestEveryUplinkListsBothResolversItsOwnFirst 0.29 s, TestEveryWordingIsPlain 0.28 s. No test moved to the integration level.
2026-10-20 08:12:38 +00:00
mesh-tools 6c2a8d6210 Merge pull request 'Carry a module on the tool runner only when the runner serves something of it, so a part on its own credential has one writer (issues 1329, 1338)' (#404) from fix/1329-runner-carries-only-what-it-serves into main 2026-10-19 23:39:24 +00:00
mesh-tools c07f604047 Make the rollout tests wait for the plan look to return before their cleanups restore what it reads, which raced (issue 1338)
mesh/merge-gate head c07f604047be merged into trunk 70cb217601f9 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head c07f604047be merged into trunk 70cb217601f9 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 23:23:23 +00:00
mesh-tools b893cc829c Keep carrying a module whose bundle the tool runtime loads, as route-proxy's helper writes its state on the runner's account (issue 1329) 2026-10-19 23:14:51 +00:00
mesh-tools 70cb217601 Merge pull request 'Answer a single pull request's check with its request id on clean standard output (issue 1311)' (#403) from feat/no-terminal into main 2026-10-19 23:11:38 +00:00
mesh-tools 1d0df24a0d Carry a module on the tool runner only when the runner has something of it to serve, so a part on its own credential has one writer (issue 1329) 2026-10-19 23:10:48 +00:00
mesh-tools 202b453de1 Point standard output at standard error while a one-head delivery check runs, so its answer is the only JSON (issue 1311)
mesh/merge-gate out of date: judged against trunk 40350d82fa20, the trunk is now 70cb217601f9; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 40350d82fa20, the trunk is now 70cb217601f9; judged again on a recheck or when it is merged
mesh/delivery delivered
2026-10-19 22:52:00 +00:00
mesh-tools 7a0b275020 Keep delivery check standard output the answer alone and answer the request id asked (issue 1311) 2026-10-19 22:46:30 +00:00
mesh-tools 40350d82fa Merge pull request 'Ask a failed delivery's Retry on the phone, tell a refused answer, and pass a seat to its only claimant (ADR 0449, ADR 0451)' (#402) from feat/no-terminal into main 2026-10-19 22:44:07 +00:00
mesh-tools 1f708e2d8d Merge pull request 'Answer the what-if as data, let the build seat read the gate's pull keys, and prove the parts' and the gate's grants (design 47 R3d, ADR 0450)' (#401) from feat/r3d-sdk-and-controller into main 2026-10-19 22:33:11 +00:00
mesh-tools 7f23c537cc Let a retry see every plan made after the failed one, and pin the 7-day end of plan-failed (issue 1328, ADR 0451)
mesh/merge-gate head 7f23c537cced merged into trunk 1f708e2d8d72 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 7f23c537cced merged into trunk 1f708e2d8d72 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 21:48:41 +00:00
mesh-tools 0aac89f4e2 Test that the gate's buckets are writable by its machine's tool runner account only (issue 1326)
mesh/repo-check head 0aac89f4e22c merged into trunk caa3fc899501 — pass: its merge-check.sh passed
mesh/merge-gate head 0aac89f4e22c merged into trunk caa3fc899501 — pass: group feat/r3d-sdk-and-controller composed: every machine composes with the change…
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792449194411267197 (novox/mesh-controller at 1f708e2d into ma…
mesh/delivery-group group feat/r3d-sdk-and-controller delivered: every member is delivered
ADR 0437 §3's check: the gate, written in the test as the clause declares it, is composed with the parts and the view
through Users; its seven buckets are writable by its machine's tool runner and by no other account, and each part
reads only the gate buckets its manifest declares.
2026-10-19 21:48:07 +00:00
mesh-tools d8c13d97ca Test that the forge bridge's, indexer's and teller's credentials grant exactly their manifests (issue 1243)
ADR 0437 §4's check: the three manifests, copied from mesh-delivery's main, are composed through Users and
PermissionsFor; each may do what it declares and is refused another module's bucket, another module's name and a seat's
verb subject.
2026-10-19 21:47:47 +00:00
mesh-tools f1775281e2 Refuse a retry of a failed plan that a newer done plan of its branch has passed (issue 1328, ADR 0451) 2026-10-19 21:45:48 +00:00
mesh-tools 4083755ca1 Let the build agent read the gate's pull keys and watch both buckets (issue 1325)
From R3e the gate writes mesh-delivery-gate_pulls and today's bucket stops (ADR 0450 §1). The build agent's manifest
reads both, and a running check's watch listens in every bucket that exists, so a move is heard whichever holds the key.
2026-10-19 21:42:37 +00:00
mesh-tools 7b83aaa949 Tell a refusal as one and an unanswered call as not known; ask a plan that fails again within the hour (issues 1316, 1314, review round 1) 2026-10-19 21:40:27 +00:00
mesh-tools 23a209f3d0 Answer plans what-if as data, the document delivery-plan answers (issue 1324)
The gate reads a delivery's plan as data once delivery-plan goes (ADR 0450 §2). The data form is an explicit
switch, `data: true`, as plans wait asks for its data: a person still gets the text. Both verbs build their
document from one function, deliveryPlanDocument, so they cannot differ; delivery-plan is unchanged (ADR 0450 §8).
2026-10-19 21:40:26 +00:00
mesh-tools 3448f98000 Make a seat pass no repair, read every machine before passing, keep the gate's pass in memory (issue 1317, review round 1) 2026-10-19 21:38:41 +00:00
mesh-tools 12db1d3748 Tell the operator when the answer to a plan's ask is refused (issue 1316, ADR 0449) 2026-10-19 21:30:33 +00:00
mesh-tools 7770d193da Word and perform a failed plan's Retry in the asker (issue 1315, ADR 0449) 2026-10-19 21:28:57 +00:00
mesh-tools 248fe521a3 Raise plan-failed for a failed delivery plan, with Retry when a retry would run (issue 1314, ADR 0449) 2026-10-19 21:26:43 +00:00
mesh-tools 666dcff79b Pass a seat held once to its only claimant when its recorded holder is assigned nowhere (issue 1317, ADR 0449) 2026-10-19 21:21:32 +00:00
mesh-tools caa3fc8995 Merge pull request 'Make a module's consumer from now and refuse a start word in consumes (issue 1279, ADR 0443)' (#400) from fix/1279-consumers-from-now into main 2026-10-19 20:22:43 +00:00
jschoubben a0c971cea9 Make a module's consumer from now and refuse a start word in consumes, so no part replays a week at its first start (issue 1279, ADR 0443)
mesh/repo-check head a0c971cea92e merged into trunk 880207649956 — pass: its merge-check.sh passed
mesh/merge-gate head a0c971cea92e merged into trunk 880207649956 — pass: group fix/1279-consumers-from-now composed: every machine composes with the change…
mesh/delivery delivered
mesh/delivery-group group fix/1279-consumers-from-now delivered: every member is delivered
2026-10-19 21:53:08 +02:00
mesh-tools 8802076499 Merge pull request 'Raise a delivery stalled line with the severity it says, so a held group reaches the operator as urgent (issue 1278)' (#399) from fix/1278-stalled-line-severity into main 2026-10-19 18:48:08 +00:00
jschoubben 2d2e736b45 Raise a delivery's stalled line with the severity mesh-delivery gives it (issue 1278)
mesh/merge-gate head 2d2e736b4592 merged into trunk fdfda0e839e9 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 2d2e736b4592 merged into trunk fdfda0e839e9 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 20:39:26 +02:00
mesh-tools fdfda0e839 Merge pull request 'Hand a group's members to the delivery verbs in a file when they are long, not as one argument (issue 1272)' (#398) from fix/1272-members-not-an-argument into main 2026-10-19 16:16:57 +00:00
jschoubben 540cda11df Hand a group's long members list to the verb's command in a file, not one argument (issue 1272)
mesh/merge-gate head 540cda11dfd5 merged into trunk ad753c70f6d4 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 540cda11dfd5 merged into trunk ad753c70f6d4 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 18:06:45 +02:00
mesh-tools ad753c70f6 Merge pull request 'Pass a check's changed paths in a file, so a change of thousands of files can start and a check that cannot start says why (issue 1247)' (#396) from fix/1247-changed-paths-file into main 2026-10-19 15:15:31 +00:00
jschoubben da20fea549 Merge remote-tracking branch 'origin/main' into fix/1247-changed-paths-file
mesh/merge-gate head da20fea5495d merged into trunk 2f4f50b9ca8b — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head da20fea5495d merged into trunk 2f4f50b9ca8b — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 16:54:58 +02:00
mesh-tools 2f4f50b9ca Merge pull request 'Announce a plan from a copy that shares nothing with the walk (issue 1260)' (#397) from fix/1260-plan-announce-copy into main 2026-10-19 14:52:39 +00:00
jschoubben 14f405ece5 Announce a plan from a copy that shares nothing with the walk, which goes on writing its steps (issue 1260)
mesh/merge-gate head 14f405ece508 merged into trunk 5c807808e88a — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 14f405ece508 merged into trunk 5c807808e88a — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 16:42:53 +02:00
jschoubben 0d1d780d43 Merge remote-tracking branch 'origin/main' into fix/1247-changed-paths-file
mesh/merge-gate out of date: judged against trunk 5c807808e88a, the trunk is now 2f4f50b9ca8b; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 5c807808e88a, the trunk is now 2f4f50b9ca8b; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
2026-10-19 15:44:18 +02:00
mesh-tools 5c807808e8 Merge pull request 'Count a module's own stream events beside its emits in the catalogue agreement (issue 1249)' (#395) from fix/1249-stream-events-agree into main 2026-10-19 13:42:14 +00:00
jschoubben da5a5c9b67 Give the gate a short changed list as an argument, as before, so an older judge still runs (issue 1247, review round 1) 2026-10-19 15:40:20 +02:00
jschoubben 07a8f04e65 Count a module's own stream events beside its emits in the catalogue agreement (issue 1249)
mesh/merge-gate head 07a8f04e65a3 merged into trunk a5a43c8f31f0 — pass: the change touches no module of the mesh's graph
mesh/repo-check head 07a8f04e65a3 merged into trunk a5a43c8f31f0 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 15:31:13 +02:00
jschoubben dd802c81a7 Pass a check's changed paths in a file so a change of thousands of files can start (issue 1247)
One environment string over 128 KiB made the container fail to start with no words; the gate reads --changed-file, and a command that cannot start now says its error.
2026-10-19 15:22:00 +02:00
mesh-tools a5a43c8f31 Merge pull request 'Find the delivery seat's holder by its claim, and record each build's fingerprint inputs (design 47 R3b: issues 1210-1216)' (#394) from feat/delivery-seat-by-claim into main 2026-10-19 10:28:30 +00:00
jschoubben 3e7f34d041 Test that the check request carries the pulls buckets (issue 1212, review round 1 finding 1)
mesh/repo-check head 3e7f34d041b1 merged into trunk 59c8cf2f2406 — pass: its merge-check.sh passed
mesh/merge-gate head 3e7f34d041b1 merged into trunk 59c8cf2f2406 — pass: group feat/delivery-seat-by-claim composed: every machine composes with the change…
mesh/delivery delivered
mesh/delivery-group group feat/delivery-seat-by-claim delivered: every member is delivered
2026-10-19 12:09:04 +02:00
jschoubben e29e855bdf Name the pulls buckets of the seat's claimants on the check, so the build agent reads them (issue 1212) 2026-10-19 12:04:26 +02:00
jschoubben da3c01089e Prove a module may say a condition to the operator only under its own name (issue 1216) 2026-10-19 12:01:59 +02:00
jschoubben 253283d68e Read a merge's warrant under the asker its announcement names, the teller or mesh-delivery (issue 1213) 2026-10-19 12:01:12 +02:00
jschoubben 3f038b989e Hear a merge's announcement from whichever module claims the delivery seat (issue 1211) 2026-10-19 11:59:21 +02:00
jschoubben 3834c72747 Find the delivery's owner on the controller's own path by the seat's claim, not its name (issue 1210) 2026-10-19 11:57:07 +02:00
jschoubben 68088ec782 Record the inputs of a build's source fingerprint in its build record (issue 1215) 2026-10-19 11:54:21 +02:00
jschoubben 8da2619920 Compute the source fingerprint with mesh-sdk's one function (issue 1214) 2026-10-19 11:52:17 +02:00
mesh-tools 59c8cf2f24 Merge pull request 'The builds list carries each build's source fingerprint, so builds shows it (issue 1208)' (#393) from fix/1208-builds-list-source into main 2026-10-19 09:00:24 +00:00
jschoubben 3ac8657a49 Carry the source fingerprint in the builds list, so builds can say it (issue 1208)
mesh/merge-gate head 3ac8657a49bf merged into trunk 230f0a1a18f2 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 3ac8657a49bf merged into trunk 230f0a1a18f2 — pass: its merge-check.sh passed
mesh/delivery delivered
Issue 1204's line never showed: chainBuildList copied each build field by field and left the fingerprint out.
2026-10-19 10:46:57 +02:00
mesh-tools 230f0a1a18 Merge pull request 'builds says each build's source fingerprint (issue 1204)' (#392) from fix/1204-builds-source into main 2026-10-19 08:43:30 +00:00
jschoubben 6e487fa01f Say each build's source fingerprint in builds, the one tool that reads what the builder recorded (issue 1204)
mesh/merge-gate head 6e487fa01f63 merged into trunk 7a74cee83f88 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 6e487fa01f63 merged into trunk 7a74cee83f88 — pass: its merge-check.sh passed
mesh/delivery delivered
The builder records a build's source fingerprint, but no tool read it back, so design 47 R3a could not
reproduce a recorded one (ADR 0437 clause 5).
2026-10-19 10:28:45 +02:00
mesh-tools 7a74cee83f Merge pull request 'A delivery does not send to or wait for a machine already away when its tier is sent (issue 1161)' (#391) from fix/1161-away-machine into main 2026-10-19 07:35:28 +00:00
jschoubben 378e65780d Away is presence's verdict alone; a sleeping machine is away once presence says so (issue 1161)
mesh/merge-gate head 378e65780d0b merged into trunk 791b92c875d3 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 378e65780d0b merged into trunk 791b92c875d3 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 09:21:09 +02:00
jschoubben 296e292f55 A machine away when a tier is sent is not sent to or waited for, and the plan says it was away (issue 1161)
ADR 0435 §1: the walk judges away with the silent check's verdict; the machine keeps its want for the backlog send.
2026-10-19 09:15:51 +02:00
mesh-tools 791b92c875 Merge pull request 'A module's process may hold a bus account of its own, and a module stream may be a work queue (ADR 0436, issues 1170, 1177-1181)' (#390) from feat/mail-sender-controller into main 2026-10-19 02:07:51 +00:00
mesh-tools 1490c3b075 Merge pull request 'The controller says when it last heard a machine from presence, and no longer hears the alive message (design 47 R2c, issue 1183)' (#389) from feat/alive-gone into main 2026-10-19 02:01:15 +00:00
jochen 4104d49ec8 mesh-controller.try fault B: derive the test's expected url from what issueProcess is given, not a literal (issue 1181)
mesh/merge-gate head 4104d49ec8b9 merged into trunk 1490c3b0756a — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 4104d49ec8b9 merged into trunk 1490c3b0756a — pass: its merge-check.sh passed
mesh/delivery delivered
The test set MESH_BROKER_ADDRESS to a literal and asserted it, but
issueOnTheNewBus works out the machine's reachable address from the
catalogue's network records, not from this process's environment — the
real try found a different, correct address. Calls issueProcess
directly now, with a known broker and reachable address of its own
choosing, exactly as issued_to_vault_test.go calls issueWith and
issueProvisionerClient directly; the asserted url is that same input.
No production code changed — issueOnTheNewBus reads the right address.
2026-10-19 03:35:57 +02:00
jochen 1991fe931b mesh-controller.try fault A: decide drop-only dead letters by a module's own retention: work-queue, not by the bus's work-queue retention alone (issue 1178)
sourceAge keyed AgainTo's clause-6 refusal on the stream's bus retention
policy, which SEAT_* and other mesh-internal work queues share without
being one of ADR 0436 §6's module streams — so every seat's ask, once
given up on, was refused delivery again. Added a dedicated metadata key
(WorkQueueMetadata), set only by ModuleStream.Stream() when the
catalogue declared retention: work-queue, and read sourceAge from that
instead of the stream's retention policy. New test proves a seat's own
work queue carries neither that metadata nor the restricted one.
2026-10-19 03:35:51 +02:00
jochen d8249dad8b Run messenger on its own account in the process acceptance test, as it holds operator-channel (issue 1181) 2026-10-19 03:24:46 +02:00
jochen df4d1204c7 Review round 2: publish a module stream's event as the tool runner carrying it, not the controller (issue 1181)
Step 3 published mesh.stream.mesh-mail.event.mail.send.approved on the
controller's connection, which the controller is never granted. Added
the runtime module to the fixture's assigned set, gave mesh-mail its
Streams so the carried-module path grants ownStreamGrants, minted the
runtime's password through aMesh like the other principals, and publish
over that connection with its own inbox prefix.
2026-10-19 03:23:16 +02:00
jochen e8c8f0488b Review round 1: check the process's whole publish set, not merely that each wanted subject is in it (issue 1180) 2026-10-19 03:18:26 +02:00
jochen cfb6a8c531 Review round 1: assert the own secret's mode, not merely log it (issue 1170) 2026-10-19 03:18:22 +02:00
jochen 9b7f9a7fd2 Review round 1: a carried module's queue consumer survives PruneHeard, a work-queue stream's restricted metadata is proven on the server config (issue 1178)
ConsumersOf's carried-module path (a module the tool runner carries, no
account of its own — mesh-mail's real shape) was not passing Queues
through, so its work-queue consumer was missing from the composed list
and PruneHeard would have removed what issueProcess made for it. Fixed,
with a test through Users -> ConsumersOf.

Extracted EnsureStream's config building into StreamConfigFor, a pure
function, so the dead-letters test can assert the restricted metadata
EnsureStream would actually send for a work-queue stream instead of
setting the dead letter's header by hand, which passed whether or not
modstreams.go's out.Restricted = true was there at all.
2026-10-19 03:18:18 +02:00
jochen 65632e7999 Review round 1: mesh-mail's real shape has no broker, rollout mint issues its processes, the acceptance test connects with the minted account (issue 1181)
mayIssue no longer requires a module's own broker secret when one of its
processes declares bus: mesh-mail's real manifest has none, its tools
served by the tool runner. rollout mint gained a KindProcess case, so a
process's account rotates the same way every other bus account does. The
release's acceptance test now composes its principals the real way
(inventory.ProcessesOf -> broker.Users -> broker.ComposeAccounts), mints
through the store as module issue does, and sets each connection's own
inbox prefix so replies are not refused; it moved from internal/broker
(which cannot import inventory) to cmd/mesh-controller. Added an
integration test proving the mint, accept and vault hand-off, and that
issuing again rotates the password.
2026-10-19 03:18:08 +02:00
jochen f6a48ac5ef Let module issue mint and deliver a bus process's account (issue 1181) 2026-10-19 02:55:29 +02:00
jochen 93a82cb032 Grant a bus process an account scoped to its own declaration (issue 1180) 2026-10-19 02:44:38 +02:00
jochen de651e0fd1 Give a work-queue stream its one consumer, drop-only dead letters (issue 1178) 2026-10-19 02:33:22 +02:00
jochen 97ee33e8f4 Let a process say what it does on the bus (issue 1179) 2026-10-19 02:17:39 +02:00
jschoubben 19e3eac746 The controller no longer subscribes to the alive message: HostBeats, its handler, the heartbeat-gap duration and a fixed silent bound (issue 1183, AC3, AC4, AC5)
mesh/merge-gate head 19e3eac74658 merged into trunk 89cfa9431d8f — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 19e3eac74658 merged into trunk 89cfa9431d8f — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 02:16:21 +02:00
jschoubben 863db82737 The controller's last-heard word for a machine comes from the watchman's presence in nodes, node, status and the watchdogs (issue 1183, AC1, AC2) 2026-10-19 02:13:56 +02:00
jochen a069898c0c Let a module stream declare work-queue retention (issue 1177) 2026-10-19 02:11:05 +02:00
jochen ffad53aad2 Let an own secret name its owner (issue 1170) 2026-10-19 02:04:00 +02:00
mesh-tools 89cfa9431d Merge pull request 'The controller takes what the watchman says: key-ttl buckets, the checks running readable, stuck keys raised, silence from presence (design 47 R2b)' (#388) from feat/delivery-watch into main 2026-10-18 23:34:07 +00:00
mesh-tools 5939bbdefe Merge pull request 'An approved merge's first build passes the guard of a build through a verb, and a failed one is recorded (issues 1146, 1169)' (#387) from fix/1146-first-build-ask into main 2026-10-18 23:20:17 +00:00
jschoubben ec85cdf70c The system-exports reload test waits for the event naming the user it connected, past the watchman's own (issue 1169)
mesh/merge-gate head ec85cdf70c07 merged into trunk cec26d859695 — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head ec85cdf70c07 merged into trunk cec26d859695 — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 01:06:56 +02:00
jschoubben cd303f68ed The first-build test reads the asks its stub records under a lock: the merge path asks on its own goroutine (issue 1146) 2026-10-19 01:02:02 +02:00
jschoubben 2cbebbc011 A failed first build of a root module is kept under a key the bus takes, and the real ask's test has the identity store (issue 1146) 2026-10-19 00:58:08 +02:00
jschoubben ac79e29319 Merge remote-tracking branch 'origin/main' into feat/delivery-watch
mesh/repo-check head ac79e29319ee merged into trunk 5939bbdefe5a — pass: its merge-check.sh passed
mesh/merge-gate head ac79e29319ee merged into trunk 5939bbdefe5a — pass: group feat/delivery-watch composed: every machine composes with the change as it d…
mesh/delivery delivered
mesh/delivery-group group feat/delivery-watch delivered: every member is delivered
# Conflicts:
#	cmd/mesh-controller/doctor.go
2026-10-19 00:53:02 +02:00
jschoubben ef0ca69cb0 The first build of a module a merge adds on the operator's Approve passes the guard of a build through a verb on the warrant it checked (issue 1146, AC9) 2026-10-19 00:48:56 +02:00
mesh-tools cec26d8596 Merge pull request 'An approved merge into a repository nothing is built from asks its modules' first builds (issue 1146)' (#386) from fix/1146-first-build-of-new-repo into main 2026-10-18 22:32:41 +00:00
jschoubben e621e15d2f Its comment names D18 too (issue 1146)
mesh/merge-gate head e621e15d2f5d merged into trunk b1cfb85f86ed — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head e621e15d2f5d merged into trunk b1cfb85f86ed — pass: its merge-check.sh passed
mesh/delivery delivered
2026-10-19 00:20:31 +02:00
jschoubben a08498b240 Name the waiting first merge's probe D18: designs 06 and 54 already name probe D17 (issue 1146) 2026-10-19 00:20:20 +02:00
jschoubben 39ae7de6d3 Say a first merge that waits past ten minutes for mesh-delivery's word, with probe D17 (issue 1146, AC8) 2026-10-19 00:19:03 +02:00
jschoubben 4e12fc19b7 A first merge into a new repository waits kept for mesh-delivery's word, and the word's arrival asks the builds, with no timer; AC7 on the bus in both orders (issue 1146) 2026-10-19 00:16:36 +02:00
jschoubben 19f15d8ea0 A merge on the operator's Approve into a repository the mesh builds nothing from asks the first build of the modules it holds (issue 1146, AC6, AC7) 2026-10-19 00:11:52 +02:00
mesh-tools b1cfb85f86 Merge pull request 'The have bucket takes a node's whole facts, as large as a chain value, never cut (issue 1162)' (#385) from fix/1162-have-node-size into main 2026-10-18 22:07:27 +00:00
jschoubben 75a27ec1a4 The have bucket takes a node's whole facts, as large as a chain value, never cut (issue 1162)
mesh/merge-gate head 75a27ec1a4bc merged into trunk 85af245f1af1 — pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus…
mesh/repo-check head 75a27ec1a4bc merged into trunk 85af245f1af1 — pass: its merge-check.sh passed
mesh/delivery delivered
A node's own key carries its profile and was refused at 4 KiB on novox and g14
(nats: message size exceeds maximum allowed).
2026-10-18 23:51:56 +02:00
mesh-tools 85af245f1a Merge pull request 'A delivery plan passes a machine silent past its bound at a middle tier, as at the last (issue 1160)' (#384) from fix/1160-silent-mid-tier into main 2026-10-18 20:15:32 +00:00
jschoubben 1ec27e6ba1 Read tier 1's build and the note once the plan has reached them in the silent middle tier test (issue 1160)
mesh/merge-gate head 1ec27e6ba18f merged into trunk 461fa9c7b4ed — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 1ec27e6ba18f merged into trunk 461fa9c7b4ed — pass: its merge-check.sh passed
mesh/delivery covered after its stop: a later merge that contains it was delivered: novox/mesh-controller@75a27ec1a4bc (merged as b1cfb85f into main, wal…
2026-10-18 21:24:07 +02:00
jschoubben 425134784a A machine silent past the bound at a tier between the first and the last is named and left as it was, as at the last tier (issue 1160) 2026-10-18 21:15:48 +02:00
jschoubben fb66608a13 A machine whose presence says online is heard now; last heard is otherwise the later of last_seen and the presence time (issue 1156) 2026-10-18 21:04:59 +02:00
jschoubben 7be2df627f The controller raises what the watchman says stopped moving, and takes a machine's silence from its presence (issue 1156) 2026-10-18 21:00:10 +02:00
jschoubben cb006fc8dd A refused send key raises a condition in place of the timer that stood in for it (issue 1157) 2026-10-18 20:57:10 +02:00
jschoubben 0273609090 The checks running are readable through reads and carry when each started (issue 1155) 2026-10-18 20:55:18 +02:00
jschoubben 3b49d7044f A state entry may say key-ttl and its bucket allows a time to live per key (issue 1154) 2026-10-18 20:53:42 +02:00
jschoubben ba59283b4b Pin mesh-sdk at the stuck bucket's merge (issue 1156) 2026-10-18 20:51:32 +02:00
mesh-tools 461fa9c7b4 Merge pull request 'The have bucket and its grants, a module and build per resource, the bus's connect events for the watchman, and a new module registered at its merge's Approve (issues 1136, 1145, 1147, 1146)' (#383) from feat/delivery-have into main 2026-10-18 18:36:12 +00:00
jschoubben 06bf8cb2fb Merge branch 'feat/delivery-have-broker' into feat/delivery-have
mesh/repo-check head 06bf8cb2fb7a merged into trunk 48e9656c670b beside mesh-host#118@aab65b81 — pass: its merge-check.sh passed
mesh/merge-gate head 06bf8cb2fb7a merged into trunk 48e9656c670b — pass: group feat/delivery-have composed: every machine composes with the change as it di…
mesh/delivery delivered
mesh/delivery-group group feat/delivery-have delivered: every member is delivered
2026-10-18 19:27:59 +02:00
jschoubben 222c9c1d72 Judge migration 105 alone now that 106 follows it, and list the new store tests (issue 1145) 2026-10-18 19:27:15 +02:00
jschoubben 0520773412 The connections request asks for user names, which the server leaves out otherwise (issue 1147) 2026-10-18 19:27:12 +02:00
jschoubben eba864dc5f Pin mesh-host at 9d9e3a46c4f5 and recapture it beside (issue 1145) 2026-10-18 19:18:38 +02:00
jschoubben 2d3d5ab616 Test that declarationWith sets modules and builds for a node that reads them (issue 1145); the Approve of a head it did not cover refuses a new module, heads keyed by the repository (issue 1146) 2026-10-18 19:18:11 +02:00
jschoubben dc65fe2319 Send each resource's module and each module's build to a node that reads them; pin mesh-host at 0b391f49a25f (issue 1145) 2026-10-18 19:11:10 +02:00
jschoubben fc43238267 Merge branch 'feat/delivery-have-broker' into feat/delivery-have 2026-10-18 19:09:38 +02:00
jschoubben 8c8bc5021e The bus's system account exports MESH's connect and disconnect events and connections request, heard only by the watchman (issue 1147) 2026-10-18 19:07:06 +02:00
jschoubben 557c11af1e A second new module from a repository is judged alongside, with no warrant asked: test (issue 1146, AC5) 2026-10-18 19:06:15 +02:00
jschoubben 661e8eff2f The broker asserts the have bucket and grants each node its own keys of it, the controller none (issue 1136) 2026-10-18 19:04:55 +02:00
jschoubben 8c0fa40e65 A new module from a repository nothing registered is built from registers on the operator's Approve of its merge (issue 1146, ADR 0432) 2026-10-18 19:04:38 +02:00
jschoubben 6746ab6492 Pin the SDK with delivery's keys, from a commit the forge keeps (issue 1136)
The old pin named mesh-sdk #28's head, which no branch holds, so a fresh
download of the controller could not fetch it.
2026-10-18 19:00:44 +02:00
mesh-tools 48e9656c67 Merge pull request 'Lay the builds out before the start reads them, so the controller does not exit on its own start (issue 1121)' (#382) from fix/lay-builds-out-before-serving into main 2026-10-18 12:50:37 +00:00
jschoubben a825a4c904 The first read of a chain not laid out lays it out, so no read of the start is refused (issue 1121)
mesh/merge-gate head a825a4c90446 merged into trunk d3556f452e8c — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head a825a4c90446 merged into trunk d3556f452e8c — pass: its merge-check.sh passed
mesh/delivery delivered
The layout gate is switched on when the stores open (UseChainOpener), before
attachTheChain; a read of the start came first, was refused, and the
controller exited. A read that finds no marker now lays the builds out once,
under a lock, and is refused only when that fails (a process without the
lease).
2026-10-18 14:39:43 +02:00
jschoubben 315c622386 Lay the builds out before the start reads them, so the controller does not exit on its own start (issue 1121)
mesh/merge-gate waiting in the build queue since 14:36
mesh/delivery superseded: a newer head of the same pull request
Delivery 2 (#381) gated every chain read on the layout marker and laid the
builds out beside the start; a read of the start then met the gate, the
controller exited, and systemd started it again every five seconds from
14:34 on 2026-10-18. The layout now runs once inside attachTheChain, before
anything reads; it skips keys laid out before, so it takes seconds, and a
failure is retried beside the serving as before.
2026-10-18 14:36:04 +02:00
mesh-tools d3556f452e Merge pull request 'The chain lives on the bus alone: the store's tables of builds, rollouts and assignments are dropped (design 47 milestone 5, delivery 2: issues 1094, 1108, 1109, 1110, 1096, 1111)' (#381) from feat/chain-on-the-bus-d2 into main 2026-10-18 12:30:42 +00:00
jschoubben 68db18bdfc Make the second merge check pass: the gate's mesh has a chain of its own; the tests follow delivery 2's meaning (issues 1121, 1108, 1109)
mesh/merge-gate head 68db18bdfc1b merged into trunk da6d038e07c0 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 68db18bdfc1b merged into trunk da6d038e07c0 — pass: its merge-check.sh passed
mesh/delivery delivered
The merge gate raises the snapshot's mesh in a throwaway store and now gives it a chain in memory, since the chain lives only on the bus. A module with no commit is carried by no send (a send is read from the commits on the assignments' keys), so the grants step flags only a module with a commit as newly assigned. The tests that sent a module with no build record one, the merge test waits for the merges' consumer before publishing and for the last look of a window before the next test, the pending test follows a chain that deletes nothing, and a wait's settling is a value of its key.
2026-10-18 14:21:35 +02:00
jschoubben cbc6bac8c0 Test chains start laid out; two tests follow delivery 2's meaning (issues 1121, 1108, 1109)
mesh/merge-gate head cbc6bac8c070 merged into trunk da6d038e07c0 — error: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; the m…
mesh/repo-check head cbc6bac8c070 merged into trunk da6d038e07c0 — fail: its merge-check.sh failed: --- FAIL: TestTheGrantsStepSendsTheUserListAndNothingEl…
mesh/delivery superseded: a newer head of the same pull request
The fixtures' fresh chain writes the layout marker, as the controller's start leaves a mesh, so the gate stays for the live start. A pending wait that ended unanswered is carried by the next one, so both are known. A hand send forgets what each assignment runs, so the test writes its build and commit after the send.
2026-10-18 13:51:37 +02:00
jschoubben 0b4cd46b3f Merge remote-tracking branch 'origin/main' into chain-on-the-bus-d2-local
mesh/merge-gate head 0b4cd46b3f04 merged into trunk da6d038e07c0 — error: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; the m…
mesh/repo-check head 0b4cd46b3f04 merged into trunk da6d038e07c0 — fail: its merge-check.sh failed: --- FAIL: TestAnAssignmentThatBlocksAnotherMachineSaysW…
mesh/delivery superseded: a newer head of the same pull request
2026-10-18 13:42:00 +02:00
jschoubben 7f074d9c94 A layout run again sets no pointer back and skips records already moved (issue 1121)
The live start recorded its layout in chain_moved, so the marker is missing and the layout runs once more over the old keys. It wrote newest, worked, held, commit and source over the pointers of every build made since. Pointers are now made only where missing, and a record already under its new key is not read again.
2026-10-18 13:41:25 +02:00
jschoubben e76ba966b3 Merge fix/1121-holds-released: delivery 2 on the builds' new layout (issue 1121)
Delivery 2 deleted the move and chain_moved, so the layout of the builds under their modules is made by the controller's start (LayBuildsOut), recorded by the key layout.done in the builds bucket, retried every half minute with an urgent condition while it fails; the chain on the bus is refused until it exists. Its source readers use the module and pointer reads.
2026-10-18 13:39:24 +02:00
mesh-tools da6d038e07 Merge pull request 'Keep builds on the bus under their module, so no lookup reads the bucket whole (issue 1121, ADR 0427)' (#380) from fix/1121-holds-released into main 2026-10-18 11:39:08 +00:00
jschoubben c795915315 Keep builds on the bus under their module, so no lookup reads the bucket whole (issue 1121, ADR 0427)
mesh/merge-gate head c79591531575 merged into trunk aa02e7bf4e78 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head c79591531575 merged into trunk aa02e7bf4e78 — pass: its merge-check.sh passed
mesh/delivery delivered
Milestone 5 moved 8,599 build records onto the bus under bare build ids, each
carrying its module's whole manifest. Every build lookup then read the whole
bucket, and a delivery plan's send did that per module per machine while it
held the plans and the machine's declaration: the send of build-agent never
finished, and every other act and the status waited behind it.

Builds now live under <module>.build.<id> (requests under req.<id>, id.<id>
names a build's module), with pointer keys newest, worked, held,
commit.<sha> and source.<reuse key> that move only to a newer build and are
read as hints, repaired when wrong. Manifests are kept once in the same
bucket under manifest.sha256-<hex>, so no grant the running bus lacks is
needed. The old keys are laid out once at start, idempotently, and left
unread. The calls a plan look makes on the silent clocks while it holds the
plans are bounded, since the vendored kv.Keys ignores its context.
2026-10-18 13:32:31 +02:00
mesh-tools aa02e7bf4e Merge pull request 'Builds, rollouts and assignments live on the bus, each assignment naming its commit, build and rollout (design 47 milestone 5, issues 1089-1095)' (#379) from feat/chain-on-the-bus-controller into main 2026-10-18 10:40:32 +00:00
jschoubben 4eeba8fb20 The copy of sent_builds never revives an ended key; a send empties its node's column (issue 1109)
A found ended or pending key is skipped and a pulled-in key is created only where no key exists. RecordSentWith and RecordSentUnder set sent_builds = null in their own update of the node row, so a machine delivery 2 has sent is never copied afterwards. These two updates and the copy's own null are the only writes to the column.
2026-10-18 05:40:50 +02:00
jschoubben d095a3dcef Copy node.sent_builds onto the assignments' keys once, and keep the column; list the new store tests (issues 1109, 1096)
Delivery 1 wrote no commit on the key of a pulled-in module, so after the upgrade those modules fell out of SentBuilds. At the start of the lease holder a copy fills every key that names no commit (creating a standing pulled-in key where none stands), overwrites nothing, empties each column it copied, and raises an urgent condition while it fails. Migration 0105 no longer drops node.sent_builds (issue 1112 does).
2026-10-18 05:39:28 +02:00
jschoubben 5b08eb09be Integration tests: no table of the chain in the store after a live delivery; migration 0105 refuses unmoved rows and drops the moved (issue 1111) 2026-10-18 05:33:42 +02:00
jschoubben 472744bbaa Migration 0105 drops the tables of the chain behind a guard; a source scan proves no code names them (issue 1096)
The migration raises first unless chain_moved names builds, rollouts and assignments for every kind that holds rows, and unless pending_assignment holds no open or unanswered row; then it drops build_request, build, batched_merge, release_plan, pending_assignment, assignment, raise_assignment_generation(), node.sent_builds and chain_moved. build_gate stays.
2026-10-18 05:32:14 +02:00
jschoubben 9859923664 Delete the dead table paths, the move, the store's assignments mirror and its tick (issue 1110)
Every method of builds, build requests, rollouts, merges and assignments reads and writes the chain alone: the else branches that read the tables, the movers and MoveChain with their tests, chain_moved reads, ChainReads and the flags that switched a kind, the store's assignments mirror (KeepAssignmentPlans needs none of it: it uses only the AssignmentMirrored type, mirrors.go, and AssignmentValues, provisioning.go) and its tick, the move code of the controller's start, and ForTest's chain_moved row. The listed store tests of the move go with their functions; the tests of the readers, requests, assignments and plans that lived beside them stay in chain_store_test.go. Tests that replayed old data migrations over the build, assignment and pin tables are removed with the tables. The static test that no plan is written but through SavePlansMoving now reads the plan's chain path.
2026-10-18 05:30:19 +02:00
jschoubben fdab58ae4f Read what a node was sent from its assignment keys; a pulled-in module keeps a standing key; a hand send is a time (issue 1109)
SentBuilds answers from the keys of the node's assignments that name a commit, known when the node was sent anything and no key says its commit unknown. A send that names its builds writes the commit, build and rollout on each carried module's key (a module nothing assigns gets a standing key marked pulled-in, never a state) and clears what every other key of the node named; a send by hand writes sent-unknown, a time, on every key and clears their commits. A failed write of these fails the record of the send. node.sent_builds is no longer written. The plan pass ends a pulled-in key only when no send carries it.
2026-10-18 05:22:28 +02:00
jschoubben 3b989d8bb3 Callers of the pending assignments on the bucket: a replaced wait keeps its condition; one read of the key through assign (issue 1108)
A wait that ended unmade and whose condition nobody answered is carried by the wait that replaces it, so the operator is never left without the condition for a module still not on its machine, and taking it back, its condition and the orphan sweep read it there. The condition id stays node.module.<id> with the string id. clearOrphaned says a taken-back wait too. TestAnAssignmentWaitingForItsBuildIsOneKeyOfTheAssignmentsBucket enters through the assign verb and reads the key before and after the build's outcome.
2026-10-18 05:19:32 +02:00
jschoubben 88ea361bf6 Fold a pending assignment into the value of its assignment key (issue 1094)
A wait for a build is the value of <node>.<module> with state pending and a pending object (id p-<unix nanoseconds>, a string), so every reader that skips a value with a state skips it, and the controller's own lists skip it through Stands. The transitions are compare-and-set on the key's revision (claim, release, settle, repoint, mark); assigning over a wait carries it into the value that stands, applied when it was waiting; a plan pass never overwrites it; one open wait per machine and module, and a key that holds the module refuses a new one. The table is no longer read or written, ForgetEndedPending is gone (no key is removed), and a wait of a machine the store dropped is none. Callers change type only; their conditions are the next issue.
2026-10-18 05:17:30 +02:00
870 changed files with 27286 additions and 36723 deletions
+1 -1
View File
@@ -14,8 +14,8 @@ import (
"github.com/nats-io/nats.go/micro"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
+1
View File
@@ -432,6 +432,7 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
}
result.Against = built.Against
result.SourceFingerprint = built.Source
result.SourceInputs = built.SourceInputs
// What the walk did instead of building, or why it built (novox/hq ADR 0350 §3, §4).
result.ReuseKey = built.ReuseKey
if built.Reused != nil {
+4 -2
View File
@@ -6,7 +6,7 @@ import (
"sort"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
@@ -55,6 +55,8 @@ func offeredOutsideBy(ctx context.Context, provision string) (string, error) {
// its next send would, and the machines that could not be resolved, with why.
func consumersFrom(ctx context.Context, open *stores, provider, providerModule, provision string) (
[]inventory.PairEnd, []string, error) {
// One reading of the mesh for every machine this looks at.
ctx = withInputCache(ctx)
nodes, err := open.inventory.Nodes(ctx)
if err != nil {
return nil, nil, err
@@ -62,7 +64,7 @@ func consumersFrom(ctx context.Context, open *stores, provider, providerModule,
var out []inventory.PairEnd
var unresolved []string
for _, n := range nodes {
plan, _, err := planFor(ctx, open, n.Name)
plan, _, err := resolveNode(ctx, open, n.Name)
if err != nil {
unresolved = append(unresolved, fmt.Sprintf("%s: %s", n.Name, oneLine(err.Error())))
continue
+5 -3
View File
@@ -10,7 +10,7 @@ import (
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
@@ -314,7 +314,7 @@ type afterAct struct {
// after the write, which can only find more unmet than the closure would: a seat said held there is
// held in the closure too.
func resolvedAfter(ctx context.Context, open *stores, shelf map[string]catalogue.Manifest, node string) afterAct {
plan, _, err := planFor(ctx, open, node)
plan, _, err := resolveNode(ctx, open, node)
if err == nil {
out := afterAct{plan: plan, unheld: plan.Unheld}
for _, m := range plan.Modules {
@@ -355,6 +355,8 @@ func splitModules(field string) []string {
// Nothing here can fail the act it reports on. A mesh that cannot be read is worth saying and is
// not a reason to claim the assignment did not happen — it did.
func blockedElsewhere(ctx context.Context, open *stores, except string) string {
// One reading of the mesh for every machine this looks at.
ctx = withInputCache(ctx)
nodes, err := open.inventory.Nodes(ctx)
if err != nil {
return "\n\nThe rest of the mesh could not be checked: " + err.Error()
@@ -364,7 +366,7 @@ func blockedElsewhere(ctx context.Context, open *stores, except string) string {
if n.Name == except {
continue
}
if _, _, err := planFor(ctx, open, n.Name); err != nil {
if _, _, err := resolveNode(ctx, open, n.Name); err != nil {
blocked[n.Name] = err.Error()
}
}
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// What an assignment says about the machines it was not about.
@@ -8,7 +8,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/licences"
)
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
+2 -2
View File
@@ -12,10 +12,10 @@ import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/overlay"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/overlay"
)
// novox/hq ADR 0100: taking a module is its cutover; converging a node is one act, previewed, and
+7 -10
View File
@@ -12,9 +12,9 @@ import (
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/overlay"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/overlay"
)
// A node is adopted or converged (novox/hq ADR 0100), and it is said to be adopted wherever the
@@ -243,7 +243,7 @@ func take(ctx context.Context, open *stores, node, module string, opts takeOptio
return "", err
}
if !slices.Contains(assigned, module) {
if plan, _, err := planFor(ctx, open, node); err == nil {
if plan, _, err := resolveNode(ctx, open, node); err == nil {
if why, runs := plan.Because[module]; runs {
return "", fmt.Errorf("%w: %s runs on %s because %s — assign it to %s to take it",
inventory.ErrNotAssigned, module, node, why, node)
@@ -649,7 +649,7 @@ func converge(ctx context.Context, open *stores, node string, yes bool, digest s
if err != nil {
return "", err
}
plan, settings, err := planFor(ctx, open, node)
plan, _, err := resolveNode(ctx, open, node)
if err != nil {
return "", err
}
@@ -706,11 +706,8 @@ func converge(ctx context.Context, open *stores, node string, yes bool, digest s
filter)
}
gens, err := generators(ctx, open)
if err != nil {
return "", err
}
with, _, err := renderingFor(ctx, open, node, plan, settings, gens, Reading)
// What the filter is derived from: the rendering of this machine as a question reads it, with the private network.
with, err := renderingOf(ctx, open, node)
if err != nil {
return "", err
}
@@ -776,7 +773,7 @@ func converge(ctx context.Context, open *stores, node string, yes bool, digest s
if _, err := inv.Assign(ctx, node, filter); err != nil {
return "", err
}
if _, _, err := planFor(ctx, open, node); err != nil {
if _, _, err := resolveNode(ctx, open, node); err != nil {
_ = inv.Unassign(ctx, node, filter)
return "", fmt.Errorf("%s cannot run %s, so it was not converged: %w", node, filter, err)
}
@@ -7,8 +7,8 @@ import (
"testing"
"time"
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
"github.com/novox/mesh-controller/internal/conditions"
snapshot "github.com/novox/mesh-controller/internal/facts"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-host/rootsearch"
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
+1 -1
View File
@@ -33,7 +33,7 @@ const packageManagerKey = "the package manager ("
func watchApplyFailing(f *signalFacts) []conditions.Observation {
heard := map[string]bool{}
for _, m := range f.machines {
if !m.asleep() && !m.lastHeard.IsZero() && f.now.Sub(m.lastHeard) <= heartbeatBound(m.every) {
if !m.asleep() && !m.lastHeard.IsZero() && !m.silent(f.now) {
heard[m.name] = true
}
}
@@ -0,0 +1,134 @@
//go:build integration
package main
import (
"context"
"fmt"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// Issue 1334, the look: when the bus did not answer an ask of any module of a tier, no build is in flight and no event follows,
// so the look itself comes back (it fails, and the looker retries it), asks the modules again, and past the bound fails the plan.
func TestALookComesBackForATierTheBusRefusedWhole(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
for _, name := range []string{"a", "b"} {
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: name, Version: "1"},
inventory.Source{Repository: "novox/" + name, Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
}
p := inventory.Plan{ID: "plan-bus", Repository: "novox/a", Branch: "main", Commit: "c1", Created: time.Now().UTC(),
State: inventory.PlanBuilding, Tiers: [][]string{{"a", "b"}}, Modules: map[string]*inventory.PlanModule{"a": {}, "b": {}}}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
bus := false
was := askABuild
askABuild = func(_ context.Context, source buildSource, _, _ string) (string, error) {
if !bus {
return "", fmt.Errorf("%w: nats: no responders available for request", inventory.ErrNoChain)
}
return "build-" + source.Repository, nil
}
t.Cleanup(func() { askABuild = was })
look := func() bool {
t.Helper()
release, err := inv.HoldPlans(ctx, true)
if err != nil {
t.Fatal(err)
}
defer release()
return advanceHeld(ctx, open)
}
read := func() inventory.Plan {
t.Helper()
got, err := inv.PlanByID(ctx, "plan-bus")
if err != nil {
t.Fatal(err)
}
return got
}
// Every module refused: the look says it failed, which has the looker come back; the plan is building and says why.
if !look() {
t.Fatal("a look that left the whole tier unasked for the bus did not fail, so nothing would come back to it")
}
if got := read(); got.State != inventory.PlanBuilding || !strings.Contains(got.Note, "bus did not answer") ||
got.Modules["a"].State != "" || got.Modules["b"].State != "" {
t.Fatalf("the plan after a refused tier: %s %q %+v %+v", got.State, got.Note, got.Modules["a"], got.Modules["b"])
}
// Again, still refused: the same fact is not saved again, and the look still comes back.
before := read().Revision
if !look() || read().Revision != before {
t.Fatalf("a second refused look failed to come back or saved the plan again (revision %d, was %d)", read().Revision, before)
}
// The bus answers at the next look: both are asked, and nothing waits any more.
bus = true
if look() {
t.Fatal("a look that asked the whole tier failed")
}
if got := read(); got.Modules["a"].State != "asked" || got.Modules["b"].State != "asked" || got.Modules["a"].BusWaitFrom != nil {
t.Fatalf("the tier was not asked once the bus answered: %+v %+v", got.Modules["a"], got.Modules["b"])
}
}
func TestALookFailsTheTierTheBusRefusedPastTheBound(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "a", Version: "1"},
inventory.Source{Repository: "novox/a", Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
long := time.Now().UTC().Add(-busWaitBound - time.Minute)
p := inventory.Plan{ID: "plan-bus", Repository: "novox/a", Branch: "main", Commit: "c1", Created: time.Now().UTC(),
State: inventory.PlanBuilding, Tiers: [][]string{{"a"}}, Modules: map[string]*inventory.PlanModule{"a": {BusWaitFrom: &long}}}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
return "", fmt.Errorf("%w: nats: no responders available for request", inventory.ErrNoChain)
}
t.Cleanup(func() { askABuild = was })
release, err := inv.HoldPlans(ctx, true)
if err != nil {
t.Fatal(err)
}
advanceHeld(ctx, open)
release()
got, err := inv.PlanByID(ctx, "plan-bus")
if err != nil {
t.Fatal(err)
}
if got.State != inventory.PlanFailed || !strings.Contains(got.Note, "the bus did not answer an ask for 10m0s (since ") {
t.Fatalf("a bus silent past its bound did not fail the plan in words: %s %q", got.State, got.Note)
}
// Rebuilt by hand while the bus still reloads: said as waiting, not as a failure and not with an empty reason.
joined, said, err := joinAPlan(ctx, open, "a")
if err != nil || !joined || !strings.Contains(said, "not asked yet: the bus did not answer") {
t.Fatalf("a rebuild left for the bus is said as %v %q %v", joined, said, err)
}
// And retried by hand: the module is named as waiting, not as asked again.
got, _ = inv.PlanByID(ctx, "plan-bus")
if got.State == inventory.PlanFailed {
got.Modules["a"].State, got.Modules["a"].Why = "failed", "x"
if err := inv.SavePlan(ctx, &got); err != nil {
t.Fatal(err)
}
said, err := retryPlan(ctx, open, "plan-bus")
if err != nil || !strings.Contains(said, "not asked yet: the bus did not answer") {
t.Fatalf("a retry left for the bus is said as %q %v", said, err)
}
}
}
+139
View File
@@ -0,0 +1,139 @@
package main
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// Issue 1334: a bus that does not answer for a moment (it reloads its user list, and JetStream's API has no responder for a few
// seconds) is not a failed build. The module is left unasked and the plan building, and the next look asks it again; only a
// module's own fault, or a bus silent for ten minutes, fails the plan.
func aTierOfThree() (*inventory.Plan, map[string]inventory.Entry) {
p := &inventory.Plan{ID: "plan-1", State: inventory.PlanBuilding, Tiers: [][]string{{"a", "b", "c"}},
Modules: map[string]*inventory.PlanModule{}}
byName := map[string]inventory.Entry{}
for _, m := range []string{"a", "b", "c"} {
byName[m] = inventory.Entry{Source: inventory.Source{Repository: "novox/" + m, Seat: "git", Ref: "main"}}
}
return p, byName
}
func TestABusThatDoesNotAnswerForAMomentDoesNotFailTheTier(t *testing.T) {
p, byName := aTierOfThree()
bus := false
was := askABuild
askABuild = func(_ context.Context, source buildSource, _, _ string) (string, error) {
if source.Repository == "novox/b" && !bus {
return "", fmt.Errorf("%w: the bucket mesh-controller_builds is not on the bus: nats: no responders available for request",
inventory.ErrNoChain)
}
return "build-" + source.Repository, nil
}
t.Cleanup(func() { askABuild = was })
for _, m := range []string{"a", "b", "c"} {
askModule(t.Context(), p, m, byName)
}
if p.State != inventory.PlanBuilding {
t.Fatalf("a bus that did not answer for a moment failed the plan: %s, %q", p.State, p.Note)
}
if p.Modules["a"].State != "asked" || p.Modules["c"].State != "asked" {
t.Fatalf("the modules the bus did answer for were not asked: %+v %+v", p.Modules["a"], p.Modules["c"])
}
if s := p.Modules["b"]; s.State != "" {
t.Fatalf("the module the bus did not answer for is %q, not left unasked", s.State)
}
if got := unaskedOf(*p); len(got) != 1 || got[0] != "b" {
t.Fatalf("the next look would ask %v, not b", got)
}
if !strings.Contains(p.Note, "bus did not answer") {
t.Fatalf("the plan says nothing of the wait: %q", p.Note)
}
// The next look, the bus answering: b is asked and the plan goes on.
bus = true
for _, m := range unaskedOf(*p) {
askModule(t.Context(), p, m, byName)
}
if p.State != inventory.PlanBuilding || p.Modules["b"].State != "asked" || p.Modules["b"].Build == "" {
t.Fatalf("b was not asked once the bus answered: %s %+v", p.State, p.Modules["b"])
}
}
func TestABusSilentForTenMinutesFailsTheTierAsBefore(t *testing.T) {
p, byName := aTierOfThree()
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
return "", fmt.Errorf("%w: nats: no responders available for request", inventory.ErrNoChain)
}
t.Cleanup(func() { askABuild = was })
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanBuilding {
t.Fatalf("the first refusal failed the plan: %q", p.Note)
}
long := time.Now().Add(-busWaitBound - time.Minute)
p.Modules["a"].BusWaitFrom = &long
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanFailed || p.Modules["a"].State != "failed" {
t.Fatalf("a bus silent past its bound did not fail the plan: %s %+v", p.State, p.Modules["a"])
}
// Said in plain words: the bus, the wait, and since when.
want := "a could not be asked for: the bus did not answer an ask for 10m0s (since " + long.UTC().Format(time.RFC3339) +
"): this kind of record lives on the bus"
if !strings.HasPrefix(p.Note, want) || p.Modules["a"].Why != strings.TrimPrefix(want, "a could not be asked for: ")+p.Note[len(want):] {
t.Fatalf("the plan says %q, not words starting %q", p.Note, want)
}
// The wait has ended: a Retry made while the bus still reloads waits again, and does not fail at its first refusal.
if p.Modules["a"].BusWaitFrom != nil {
t.Fatalf("a failed module still holds the start of its wait: %v", p.Modules["a"].BusWaitFrom)
}
p.State = inventory.PlanBuilding
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanBuilding || p.Modules["a"].State != "" || p.Modules["a"].BusWaitFrom == nil {
t.Fatalf("a retry made while the bus is silent did not wait afresh: %s %+v", p.State, p.Modules["a"])
}
}
func TestAPlanWithAModuleWaitingForTheBusIsOneTheLookComesBackTo(t *testing.T) {
p, _ := aTierOfThree()
if busWaiting(*p) {
t.Fatal("a plan whose modules are not asked yet waits for the bus")
}
now := time.Now()
p.Modules["b"] = &inventory.PlanModule{BusWaitFrom: &now}
p.Modules["a"] = &inventory.PlanModule{State: "asked"}
if got := busWaitingIn(*p); len(got) != 1 || got[0] != "b" {
t.Fatalf("the modules waiting for the bus are %v, not b", got)
}
p.State = inventory.PlanFailed
if busWaiting(*p) {
t.Fatal("a plan that is not open is waited for")
}
}
func TestAModulesOwnFaultStillFailsThePlan(t *testing.T) {
p, byName := aTierOfThree()
delete(byName, "a")
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanFailed || !strings.Contains(p.Note, "no longer in the catalogue") {
t.Fatalf("a module gone from the catalogue did not fail the plan: %s %q", p.State, p.Note)
}
p, byName = aTierOfThree()
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
return "", errors.New("git.example/novox/a is no repository the catalogue builds a module from")
}
t.Cleanup(func() { askABuild = was })
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanFailed || p.Modules["a"].State != "failed" {
t.Fatalf("a module's own refusal did not fail the plan: %s %+v", p.State, p.Modules["a"])
}
}
+87
View File
@@ -0,0 +1,87 @@
package main
import (
"errors"
"fmt"
"slices"
"sort"
"strings"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/inventory"
)
// busWaitBound is how long a module whose ask the bus did not answer is asked again at every look before the plan fails
// (issue 1334).
const busWaitBound = 10 * time.Minute
// busDidNotAnswer says an ask failed because the bus's JetStream had no responder at that moment, as it has for a few seconds
// when the bus reloads its user list (novox/hq issue 353), and not because of the module: the same ask is made again.
func busDidNotAnswer(err error) bool {
if err == nil {
return false
}
if errors.Is(err, nats.ErrNoResponders) || errors.Is(err, nats.ErrJetStreamNotEnabled) ||
errors.Is(err, jetstream.ErrJetStreamNotEnabled) || errors.Is(err, inventory.ErrNoChain) {
return true
}
// An error that crossed a boundary as text only.
text := err.Error()
return strings.Contains(text, "no responders available") || strings.Contains(text, "jetstream not enabled")
}
// waitForTheBus leaves a module whose ask the bus did not answer unasked, the plan building, and says so in the plan's note: the
// next look asks the tier's unasked modules again. It says false, so the caller fails the plan as for any refusal, once the bus
// has been silent for busWaitBound.
func waitForTheBus(p *inventory.Plan, state *inventory.PlanModule, name string, err error, now time.Time) bool {
if state.BusWaitFrom == nil {
state.BusWaitFrom = &now
}
if now.Sub(*state.BusWaitFrom) > busWaitBound {
return false
}
state.State, state.Why = "", ""
p.Note = fmt.Sprintf("tier %d: %s not asked: the bus did not answer (%v); asked again at the next look, failing after %s",
p.Tier, name, firstLine(err.Error()), busWaitBound)
return true
}
// busWaiting says an open plan has a module of its current tier that the bus did not answer an ask of: the look must come back
// for it, as nothing else will when no build of the tier is in flight (issue 1334).
func busWaiting(p inventory.Plan) bool {
return len(busWaitingIn(p)) > 0
}
// busWaitingIn are the modules of the plan's current tier left unasked because the bus did not answer, sorted.
func busWaitingIn(p inventory.Plan) []string {
if !p.Open() || p.Tier >= len(p.Tiers) {
return nil
}
var out []string
for _, m := range p.Tiers[p.Tier] {
if s := p.Modules[m]; s != nil && s.State == "" && s.BusWaitFrom != nil {
out = append(out, m)
}
}
sort.Strings(out)
return out
}
// busWaitWords is how a verb says modules are waiting for the bus: not asked yet, and asked again by the plan at its next look.
func busWaitWords(p inventory.Plan, modules []string) string {
return fmt.Sprintf("%s not asked yet: the bus did not answer; %s asks again at its next look", strings.Join(modules, ", "), p.ID)
}
// slicesWithout is names without those in drop, in order.
func slicesWithout(names, drop []string) []string {
var out []string
for _, n := range names {
if !slices.Contains(drop, n) {
out = append(out, n)
}
}
return out
}
+63 -4
View File
@@ -208,8 +208,11 @@ type asker struct {
// raise keeps the asker's own condition (sourceAsker): which conditions needing the operator could not be
// asked, and why. Nil raises nothing (a test that does not look).
raise func(ctx context.Context, obs []conditions.Observation) error
now func() time.Time
logf func(string, ...any)
// tell tells the operator one plain notice that needs no answer (operator-channel.tell, novox/hq ADR 0380):
// the refusal of an answer to a plan's ask (ADR 0449 §3). Nil tells nothing (a test that does not look).
tell func(ctx context.Context, headline string, lines []string) error
now func() time.Time
logf func(string, ...any)
saidNoRouter bool
saidNoGrant bool
@@ -406,7 +409,9 @@ func (a *asker) reconcile(ctx context.Context) error {
for _, r := range all {
k := partKey(r.Condition, r.Part)
if r.State == string(asks.OutcomeChosen) && now.Sub(r.Ended) < askAgainAfterAnswer {
answered[k] = r
if prior, has := answered[k]; !has || r.Ended.After(prior.Ended) {
answered[k] = r
}
}
if r.State == string(asks.OutcomeRefused) {
if prior, has := refused[k]; !has || r.Opened.After(prior.Opened) {
@@ -497,7 +502,9 @@ func (a *asker) reconcile(ctx context.Context) error {
continue // refused lately, and nothing it was refused for has changed: asked again after the wait
}
}
if r, done := answered[key]; done && sameAsked(r.Actions, p.actions) {
// **A condition raised after the answered ask ended is a new one** (novox/hq ADR 0449 §2): a retried
// plan that fails again is asked again, not held back for the hour its earlier answer holds.
if r, done := answered[key]; done && sameAsked(r.Actions, p.actions) && !c.Raised.After(r.Ended) {
if _, held := byCondition[key]; !held {
continue
}
@@ -636,6 +643,8 @@ func doesWords(act conditions.Action) string {
return "the delivery starts"
case act.Verb == broker.ControllerSeat+".plans" && act.Arguments["stop"] != "":
return "the delivery is stopped"
case act.Verb == broker.ControllerSeat+".plans" && act.Arguments["retry"] != "":
return "the failed builds are asked again"
case strings.HasSuffix(act.Verb, ".restart"):
return "its service is restarted on " + act.Machine
}
@@ -884,6 +893,7 @@ func (a *asker) Decided(ctx context.Context, body []byte) error {
ended := a.now()
if acted != nil {
outcome = "failed: " + acted.Error()
a.tellRefused(ctx, *r, act, acted)
}
r.Ended, r.Acted = ended, outcome
if _, err := a.store.Change(ctx, r.ID, func(x *asked) bool {
@@ -918,6 +928,55 @@ func (a *asker) Decided(ctx context.Context, body []byte) error {
return nil
}
// tellRefused tells the operator what became of the answer they gave to a plan's ask, when it was not done (novox/hq
// ADR 0449 §3): the controller's `plans` verb refuses a Retry, Start or Stop when the plan moved on between the ask and
// the answer, and the refusal was only recorded, so the operator who pressed the answer heard nothing. **A refusal is
// told as one, and nothing else is**: a verb that said no changed nothing, but a call that got no answer (a timeout, a
// refused grant) may have run, and is told as not known. One plain notice with the ask's headline, beside the hand act
// the caller still records. An answer to any other ask is not told here. A notice that cannot be told is said in the
// log and changes nothing.
func (a *asker) tellRefused(ctx context.Context, r asked, act conditions.Action, failure error) {
if a.tell == nil || act.Verb != broker.ControllerSeat+".plans" {
return
}
var lines []string
var refused verbRefused
if errors.As(failure, &refused) {
lines = []string{
fmt.Sprintf("Your answer, %s, could not be done: %s.", act.Label, strings.TrimRight(refusalWords(refused.words, act), ".")),
"Nothing was changed.",
}
} else {
lines = []string{
fmt.Sprintf("Your answer, %s, was given, and it is not known whether it was done.", act.Label),
"Look at the delivery before you answer again.",
}
}
if err := a.tell(ctx, r.Ask.Headline, lines); err != nil {
a.logf("the operator could NOT be told that %s was refused (ask %s): %v", act.Label, r.ID, err)
}
}
// refusalWords is a verb's refusal as the operator reads it: the plans named as "the plan" (plainRefusal), and, where
// the words still carry an identifier the operator's channel withholds, one sentence of the answer's own.
func refusalWords(words string, act conditions.Action) string {
return plainRefusalOr(words, refusalFallback(act))
}
// refusalFallback is what an answer's refusal says when its own words cannot be said: one sentence for each answer
// to a plan's ask, never another answer's.
func refusalFallback(act conditions.Action) string {
switch {
case act.Arguments["retry"] != "":
return "a retry would not run now; the details say why"
case act.Arguments["go"] != "":
return "the delivery could not be started now; the details say why"
case act.Arguments["stop"] != "":
return "the delivery could not be stopped now; the details say why"
}
return "the controller refused it; the details say why"
}
// handActWarrant is the verb an act the operator chose on a warrant is recorded under: a person's decision,
// never a repair (handActVerbs).
const handActWarrant = "warrant"
@@ -0,0 +1,141 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// 1315/AC3: the operator's Retry on a failed plan, acted on through the real callAction and the controller's own
// `plans` verb, sets the plan building again at its tier and ends its plan-failed condition (novox/hq ADR 0449 §1).
func TestAWarrantForRetryRetriesAFailedPlanInTheStoreAndEndsItsCondition(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
asked := asksRecorded(t)
planMovedHeard(t) // the bus the verb says its moves on, and the hand-act log
twoTiers(t, open)
before := time.Now().UTC().Add(-time.Hour)
ended := before.Add(time.Minute)
failed := inventory.Plan{ID: "plan-1315-retry", Repository: "novox/a", Branch: "main", Commit: "c0ffee",
Created: before, State: inventory.PlanFailed, Tier: 0, Tiers: [][]string{{"a"}, {"b"}},
Note: "a failed to build in tier 0",
Delivery: &inventory.PlanDelivery{},
Modules: map[string]*inventory.PlanModule{"a": {State: "failed", AskedAt: &before, Build: "build-1",
Why: link.KilledByHand}},
Times: &inventory.PlanTimes{Cut: &before, Started: &before, Ended: &ended, Class: inventory.ClassLeaf}}
if err := open.inventory.SavePlan(ctx, &failed); err != nil {
t.Fatal(err)
}
conn, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer conn.Close()
// The controller's seat answers `plans` as it does: by running the verb.
sub, err := conn.Subscribe(link.SeatToolSubject("mesh-controller", "plans"), func(m *nats.Msg) {
var args map[string]string
_ = json.Unmarshal(m.Data, &args)
answer := link.Answer{Result: json.RawMessage(`"ok"`)}
if len(args) != 1 || args["retry"] == "" {
answer = link.Answer{Error: "the retry was asked with more than the plan"}
} else if err := plansCommand(context.Background(), []string{"retry", args["retry"]}); err != nil {
answer = link.Answer{Error: err.Error()}
}
body, _ := json.Marshal(answer)
_ = m.Respond(body)
})
if err != nil {
t.Fatal(err)
}
defer func() { _ = sub.Unsubscribe() }()
// What S23 raises for it, read from the store as the watchdog reads it.
raised := func(context.Context) ([]conditions.Condition, error) {
facts, err := gatherFailedPlans(ctx, open.inventory, time.Now())
if err != nil {
return nil, err
}
var out []conditions.Condition
for _, o := range watchPlansFailed(&signalFacts{now: time.Now(), failed: facts}) {
out = append(out, conditions.Condition{Key: o.Key(), Kind: o.Kind, Severity: o.Severity, Headline: o.Headline,
Explanation: conditions.Verdict(o.Needs, o.Explanation), Needs: o.Needs, Actions: o.Actions})
}
return out, nil
}
now := time.Now()
store := memAskedStore{}
var sent []published
var acts []link.HandAct
a := &asker{open: raised, store: store,
silence: func(context.Context, string, time.Duration, string, string) error { return nil },
publish: func(_ context.Context, subject string, body []byte, id string) error {
sent = append(sent, published{subject, id, body})
return nil
},
call: callAction(conn),
record: func(_ context.Context, act link.HandAct) error { acts = append(acts, act); return nil },
now: func() time.Time { return now },
logf: t.Logf}
cs, _ := raised(ctx)
if len(cs) != 1 || len(cs[0].Actions) != 1 {
t.Fatalf("the failed plan raised %+v", cs)
}
if err := a.reconcile(ctx); err != nil {
t.Fatal(err)
}
var w asks.Warrant
for _, r := range store {
for _, o := range r.Ask.Options {
if o.Label == "Retry" {
w = asks.Warrant{Ask: r.ID, Asker: "mesh-controller", About: r.Condition, Outcome: asks.OutcomeChosen,
Option: o.ID, Label: o.Label, Level: o.Level, Channel: "telegram", Proofs: []string{"P1"}, At: now,
AskDigest: r.Ask.Digest(),
By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"}}
}
}
}
if w.Ask == "" {
t.Fatalf("no ask offers Retry: %+v", store)
}
body, _ := json.Marshal(w)
if err := a.Decided(ctx, body); err != nil {
t.Fatal(err)
}
if err := a.Decided(ctx, body); err != nil { // heard again: nothing more
t.Fatal(err)
}
p, err := open.inventory.PlanByID(ctx, failed.ID)
if err != nil {
t.Fatal(err)
}
m := p.Modules["a"]
if p.State != inventory.PlanBuilding || p.Tier != 0 || m.State != "asked" || m.Build == "" || m.Build == "build-1" {
t.Fatalf("after the warrant the plan is %s at tier %d and a is %+v", p.State, p.Tier, m)
}
if len(*asked) != 1 {
t.Fatalf("the build was asked %d times: %v", len(*asked), *asked)
}
if got := store[w.Ask]; got.Acted != "done" {
t.Fatalf("the ask ended %q", got.Acted)
}
if len(acts) != 1 || acts[0].Verb != handActWarrant || !strings.Contains(strings.Join(acts[0].Args, " "), "retry="+failed.ID) {
t.Fatalf("the warrant was recorded as %+v", acts)
}
if cs, _ = raised(ctx); len(cs) != 0 {
t.Fatalf("the plan-failed condition did not end: %+v", cs)
}
}
+225
View File
@@ -77,6 +77,15 @@ type askerRig struct {
silenced []string
acts []link.HandAct
now time.Time
// refuses, when set, is what every verb the asker calls answers; told is every notice the operator was told.
refuses error
told []told
}
// told is one plain notice the asker told the operator.
type told struct {
headline string
lines []string
}
func newAskerRig(t *testing.T) *askerRig {
@@ -102,6 +111,10 @@ func newAskerRig(t *testing.T) *askerRig {
call: func(_ context.Context, a conditions.Action, args map[string]string) error {
raw, _ := json.Marshal(args)
r.called = append(r.called, a.Verb+"@"+a.Machine+" "+string(raw))
return r.refuses
},
tell: func(_ context.Context, headline string, lines []string) error {
r.told = append(r.told, told{headline, lines})
return nil
},
record: func(_ context.Context, act link.HandAct) error { r.acts = append(r.acts, act); return nil },
@@ -1013,3 +1026,215 @@ func TestTheServingAskerJudgesTheGrantForReal(t *testing.T) {
t.Errorf("held %v (%q, %v) with nothing to read", held, why, err)
}
}
// failedPlanCondition is the condition S23 raises for a failed delivery plan, as the keeper holds it.
func failedPlanCondition(plan string) conditions.Condition {
p := aFailedPlan(plan, nil)
f := calm(time.Date(2026, 10, 19, 18, 30, 0, 0, time.UTC))
f.failed = []failedPlanFacts{failedFactsFor(p)}
o := watchPlansFailed(f)[0]
return conditions.Condition{Key: o.Key(), Kind: o.Kind, Severity: o.Severity, Headline: o.Headline,
Explanation: conditions.Verdict(o.Needs, o.Explanation), Needs: o.Needs, Actions: o.Actions}
}
// 1315/AC1: a failed plan's Retry is asked at the approve level, in the words "the failed builds are asked again".
func TestAFailedPlansRetryIsAskedAtApproveInItsOwnWords(t *testing.T) {
r := newAskerRig(t)
c := failedPlanCondition("plan-1792433168676870614")
r.open = []conditions.Condition{c}
if err := r.a.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
sent := r.asksSent(t)
if len(sent) != 1 || len(sent[0].Options) != 1 {
t.Fatalf("asked %+v", sent)
}
o := sent[0].Options[0]
if o.Label != "Retry" || o.Level != asks.Approve || o.Does != "the failed builds are asked again" {
t.Errorf("the option is %+v", o)
}
if err := sent[0].Check(r.now); err != nil {
t.Errorf("the ask is refused as made: %v", err)
}
if strings.Contains(sent[0].Explanation, "mesh MCP server") {
t.Errorf("the ask still says where else to answer: %q", sent[0].Explanation)
}
}
// 1315/AC2: the warrant calls plans with retry=<plan> and nothing else, once, and is recorded as the hand act;
// the same warrant heard again does nothing.
func TestAWarrantForRetryCallsPlansRetryOnceAndIsRecorded(t *testing.T) {
plan := "plan-1792433168676870614"
r := newAskerRig(t)
c := failedPlanCondition(plan)
r.open = []conditions.Condition{c}
_ = r.a.reconcile(context.Background())
w := r.warrantFor(t, c.Key, "Retry")
answerWith(t, r, w)
answerWith(t, r, w)
if want := `mesh-controller.plans@ {"retry":"` + plan + `"}`; len(r.called) != 1 || r.called[0] != want {
t.Fatalf("called %v, want exactly %s", r.called, want)
}
if len(r.acts) != 1 {
t.Fatalf("hand-acts %+v", r.acts)
}
act := r.acts[0]
if act.Verb != handActWarrant || strings.Join(act.Args, " ") != "mesh-controller.plans retry="+plan ||
act.Ask != w.Ask || act.Condition != c.Key || act.Outcome != "done" || act.Cause != conditions.CauseOperatorAnswer {
t.Errorf("the hand-act is %+v", act)
}
if got := r.store[w.Ask]; got.Acted != "done" {
t.Errorf("kept %+v", got)
}
}
// 1316/AC1 and AC2: a Retry or a Stop on a plan that moved on is refused by `plans`; the operator who pressed it is
// told once, with the ask's headline and the refusal's words, and the hand act records the failure as before.
func TestARefusedAnswerToAPlansAskIsToldOnceAndStillRecorded(t *testing.T) {
plan := "plan-1792433168676870614"
waiting := conditions.Condition{Key: "plan." + plan + ".waiting", Severity: conditions.Urgent,
Headline: "openrazer delivery waiting to start", Needs: "start it, or stop it.",
Explanation: "Needs you: start it, or stop it.", Actions: waitingActions(plan, conditions.Urgent)}
for _, tc := range []struct {
name string
c conditions.Condition
label string
}{
{"retry", failedPlanCondition(plan), "Retry"},
{"stop", waiting, "Stop"},
} {
t.Run(tc.name, func(t *testing.T) {
r := newAskerRig(t)
r.refuses = verbRefused{verb: "mesh-controller.plans", words: plan + " is done; there is nothing to retry"}
r.open = []conditions.Condition{tc.c}
_ = r.a.reconcile(context.Background())
w := r.warrantFor(t, tc.c.Key, tc.label)
answerWith(t, r, w)
answerWith(t, r, w) // heard again: told once
if len(r.told) != 1 {
t.Fatalf("told %+v", r.told)
}
got := r.told[0]
if got.headline != tc.c.Headline {
t.Errorf("the headline is %q, not the ask's %q", got.headline, tc.c.Headline)
}
said := strings.Join(got.lines, " ")
if !strings.Contains(said, "the plan is done; there is nothing to retry") || !strings.Contains(said, tc.label) {
t.Errorf("the lines do not carry the answer and the refusal's words: %q", said)
}
if why, ok := conditions.Plain(said); !ok {
t.Errorf("the notice is not plain, so the channel would withhold it: %s: %q", why, said)
}
if len(r.acts) != 1 || !strings.HasPrefix(r.acts[0].Outcome, "failed: ") {
t.Errorf("the failure is not recorded as before: %+v", r.acts)
}
if kept := r.store[w.Ask]; !strings.HasPrefix(kept.Acted, "failed: ") {
t.Errorf("kept %+v", kept)
}
})
}
}
// 1316/AC3: an answer that is done tells nothing; and a refusal of an answer that is no plan's is not told here.
func TestADoneAnswerTellsNothingAndOnlyAPlansRefusalIsTold(t *testing.T) {
r := newAskerRig(t)
c := failedPlanCondition("plan-1792433168676870614")
r.open = []conditions.Condition{c}
_ = r.a.reconcile(context.Background())
answerWith(t, r, r.warrantFor(t, c.Key, "Retry"))
if len(r.told) != 0 {
t.Fatalf("a retry that was done told %+v", r.told)
}
r = newAskerRig(t)
r.refuses = verbRefused{verb: "node-service-manager.restart", words: "unit not found"}
module := conditions.Condition{Key: "module.openrazer.g14.unhealthy", Severity: conditions.Warning,
Headline: "openrazer not working on g14", Needs: "restart its service openrazer-daemon on g14.",
Explanation: "Needs you: restart it.", Actions: []conditions.Action{{Label: "Restart",
Verb: "node-service-manager.restart", Machine: "g14", Level: conditions.LevelApprove,
Arguments: map[string]string{"unit": "openrazer-daemon.service", "scope": "user"}}}}
r.open = []conditions.Condition{module}
_ = r.a.reconcile(context.Background())
answerWith(t, r, r.warrantFor(t, module.Key, "Restart"))
if len(r.told) != 0 || len(r.acts) != 1 {
t.Fatalf("a refused restart told %+v, acts %+v", r.told, r.acts)
}
}
// 1316, review round 1: a call that got no answer may have run, so it is told as not known, never as "nothing was
// changed"; and the sentence a refusal falls back to fits the answer given.
func TestAnAnswerThatGotNoAnswerIsToldAsNotKnownNotAsRefused(t *testing.T) {
plan := "plan-1792433168676870614"
r := newAskerRig(t)
r.refuses = errors.New("mesh-controller did not answer plans within 30s")
c := failedPlanCondition(plan)
r.open = []conditions.Condition{c}
_ = r.a.reconcile(context.Background())
answerWith(t, r, r.warrantFor(t, c.Key, "Retry"))
if len(r.told) != 1 {
t.Fatalf("told %+v", r.told)
}
said := strings.Join(r.told[0].lines, " ")
if !strings.Contains(said, "not known whether") || strings.Contains(said, "Nothing was changed") {
t.Errorf("a call that may have run is told as %q", said)
}
if why, ok := conditions.Plain(said); !ok {
t.Errorf("not plain: %s: %q", why, said)
}
}
func TestEachAnswersRefusalFallsBackToItsOwnSentence(t *testing.T) {
plan := "plan-1792433168676870614"
for _, tc := range []struct {
act conditions.Action
want string
}{
{retryAction(plan), "a retry would not run now"},
{conditions.Action{Label: "Start", Verb: "mesh-controller.plans", Arguments: map[string]string{"go": plan}}, "could not be started now"},
{conditions.Action{Label: "Stop", Verb: "mesh-controller.plans", Arguments: map[string]string{"stop": plan}}, "could not be stopped now"},
} {
// Words the operator's channel withholds: a hash and a dotted name.
got := refusalWords("build 48bda475dunst at mesh.seat.x refused", tc.act)
if !strings.Contains(got, tc.want) {
t.Errorf("%s: %q, want %q", tc.act.Label, got, tc.want)
}
if _, ok := conditions.Plain(got); !ok {
t.Errorf("%s: the fallback is not plain: %q", tc.act.Label, got)
}
}
}
// 1314/AC6, ADR 0449 §2: a retried plan that fails again within the hour of the answer is asked again, because the
// condition that comes back is a new one, raised after the answered ask ended; the same condition still open is not.
func TestARetriedPlanThatFailsAgainWithinTheHourIsAskedAgain(t *testing.T) {
r := newAskerRig(t)
c := failedPlanCondition("plan-1792433168676870614")
c.Raised = r.now.Add(-time.Minute)
r.open = []conditions.Condition{c}
_ = r.a.reconcile(context.Background())
answerWith(t, r, r.warrantFor(t, c.Key, "Retry"))
// Still open right after the answer: not asked again at once.
r.now = r.now.Add(time.Minute)
_ = r.a.reconcile(context.Background())
if n := len(r.asksSent(t)); n != 1 {
t.Fatalf("asked again while the same condition stayed open: %d", n)
}
// The retry ran, the plan left failed and the condition ended.
r.now = r.now.Add(time.Minute)
r.open = nil
_ = r.a.reconcile(context.Background())
// It fails again, ten minutes after the answer: a condition raised after the answered ask ended.
r.now = r.now.Add(8 * time.Minute)
again := failedPlanCondition("plan-1792433168676870614")
again.Raised = r.now
r.open = []conditions.Condition{again}
if err := r.a.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
if n := len(r.asksSent(t)); n != 2 {
t.Fatalf("the second failure was asked %d times in all, want 2", n)
}
if len(r.called) != 1 {
t.Fatalf("something retried without an answer: %v", r.called)
}
}
+26 -3
View File
@@ -19,8 +19,8 @@ import (
"git.novox.be/novox/mesh-sdk/go/asks"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
@@ -135,6 +135,12 @@ func (b busAsked) All(ctx context.Context) ([]asked, error) {
return out, nil
}
// verbRefused is a verb's own refusal of an act: the seat answered, and said no. Any other error of callAction (a bus
// that did not answer, a grant that is refused) is not one: the verb may have run.
type verbRefused struct{ verb, words string }
func (e verbRefused) Error() string { return e.verb + " refused: " + e.words }
// callAction performs an action's verb as the controller, through the grant that names it.
func callAction(conn *nats.Conn) func(ctx context.Context, a conditions.Action, args map[string]string) error {
return func(ctx context.Context, a conditions.Action, args map[string]string) error {
@@ -168,7 +174,23 @@ func callAction(conn *nats.Conn) func(ctx context.Context, a conditions.Action,
return err
}
if answer.Error != "" {
return fmt.Errorf("%s refused: %s", a.Verb, answer.Error)
return verbRefused{verb: a.Verb, words: answer.Error}
}
return nil
}
}
// tellThrough tells the operator one plain notice through the operator channel's `tell`, over the controller's own
// connection and grant (novox/hq ADR 0380); a holder's refusal is an error naming it.
func tellThrough(conn *nats.Conn) func(ctx context.Context, headline string, lines []string) error {
return func(ctx context.Context, headline string, lines []string) error {
answer, err := link.AskMeshSeatTool(ctx, conn, "operator-channel", "tell",
map[string]any{"headline": headline, "lines": lines}, tellWithin)
if err != nil {
return err
}
if answer.Error != "" {
return fmt.Errorf("operator-channel.tell refused: %s", answer.Error)
}
return nil
}
@@ -201,7 +223,7 @@ func routerRecordOf(conn *nats.Conn, inv *inventory.Inventory) func(ctx context.
}
// asksRecords is the bucket the asks seat's declarer keeps its record of asks in.
func asksRecords(ctx context.Context, inv *inventory.Inventory) (string, error) {
func asksRecords(ctx context.Context, inv catalogueReader) (string, error) {
declared, err := inv.Catalogue(ctx)
if err != nil {
return "", err
@@ -576,6 +598,7 @@ func servingAsker(open *stores, conn *nats.Conn, js jetstream.JetStream, keeper
_, err := link.RecordHandAct(ctx, conn, act)
return err
},
tell: tellThrough(conn),
routerRecord: routerRecordOf(conn, open.inventory),
routerHere: routerHereIn(open.inventory),
grantHeld: grantHeldIn(open, conn),
@@ -8,9 +8,11 @@ import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/overlay"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/overlay"
"github.com/novox/mesh-controller/internal/inventory"
)
func TestAModuleAssignedAndNotPushedIsAwaitingAPushNotUncomposable(t *testing.T) {
@@ -24,21 +26,13 @@ func TestAModuleAssignedAndNotPushedIsAwaitingAPushNotUncomposable(t *testing.T)
// The read the rest of the mesh asks still refuses it, with the composer's typed error: nothing
// can be compared about a secret that does not exist (status), and nothing here string-matches.
plan, settings, err := planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
_, err = declarationWith(ctx, open, "laptop", plan, settings, gens, Reading)
_, _, err := composeQuestion(ctx, open, "laptop", compose.Reading, false)
var notMade *catalogue.NotMadeError
if !errors.As(err, &notMade) || notMade.Module != "keeper" || notMade.Name != "repository" {
t.Fatalf("a read composition did not say which secret is not made, typed: %v", err)
}
// Foreseen, it composes, names what the push makes, and made nothing.
foreseen, err := declarationWith(ctx, open, "laptop", plan, settings, gens, Foreseeing)
foreseen, _, err := composeQuestion(ctx, open, "laptop", compose.Foreseeing, false)
if err != nil || len(foreseen.foreseen) != 1 || foreseen.foreseen[0] != "keeper/repository" {
t.Fatalf("foreseen: %v %v", foreseen.foreseen, err)
}
@@ -146,8 +140,18 @@ func TestAMachineAwaitsAPushSinceItsOldestAssignmentSinceTheLastSend(t *testing.
func TestALastSendIsReadFromTheSendAndTheReport(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aKeeper())
pushedBy(t, open, "laptop")
// What a machine was sent is read from the commits its assignments name (issue 1109): a module is carried by a send only
// when it was built from a commit, and a hand send on a machine with nothing assigned leaves no key to mark unknown.
if err := open.inventory.RegisterModule(ctx, aKeeper(), inventory.Source{BuiltFrom: "c-keeper"}); err != nil {
t.Fatal(err)
}
record, err := open.inventory.NodeByName(ctx, "laptop")
if err != nil {
t.Fatal(err)
}
if err := open.inventory.RecordSent(ctx, record.ID, "carried", map[string]string{overlay.Name: "c-overlay"}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", "keeper"); err != nil {
t.Fatal(err)
}
+16 -3
View File
@@ -4,7 +4,7 @@ import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
@@ -29,7 +29,7 @@ func aKeeper() catalogue.Manifest {
func pushedBy(t *testing.T, open *stores, node string) string {
t.Helper()
ctx := t.Context()
plan, settings, err := planFor(ctx, open, node)
plan, settings, err := resolveNode(ctx, open, node)
if err != nil {
t.Fatal(err)
}
@@ -46,7 +46,20 @@ func pushedBy(t *testing.T, open *stores, node string) string {
t.Fatal(err)
}
digest := digestOf(body)
if err := open.inventory.RecordSent(ctx, record.ID, digest, declared.Builds); err != nil {
// A push records the build of each module it carries (plan.go, Allocating); a module is carried by a send only when it was
// built from a commit, for what a machine was sent is read from the commits its assignments name (issue 1109).
builds := declared.Builds
if builds == nil {
current, err := open.inventory.CurrentBuilds(ctx)
if err != nil {
t.Fatal(err)
}
builds = map[string]string{}
for _, m := range plan.Modules {
builds[m.Module] = current[m.Module].Commit
}
}
if err := open.inventory.RecordSent(ctx, record.ID, digest, builds); err != nil {
t.Fatal(err)
}
return digest
+100
View File
@@ -0,0 +1,100 @@
package main
import (
"context"
"fmt"
"os"
"slices"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
)
// A machine out of touch when a tier is sent is not sent to and not waited for (novox/hq ADR 0435 §1, issue 1161): the
// delivery goes on with the machines heard, says the others were away, and the away ones get the builds by the send
// they are given when heard again (ADR 0353, ADR 0360 §5). Away is the silent check's one verdict, machineFacts.silent
// over the watchman's presence (ADR 0433 §3), never a second rule; a sleeping machine is away once presence says so.
// awayOf is the machines out of touch at a moment: those the silent check calls silent (a sleeping machine
// is away once presence says so). A machine
// never heard from has not joined and is not away.
func awayOf(machines []machineFacts, now time.Time) map[string]bool {
out := map[string]bool{}
for _, m := range machines {
if m.silent(now) {
out[m.name] = true
}
}
return out
}
// judgeAway is the walk's reading of which machines are away as a tier is sent: every node with its last-heard time and
// the watchman's verdict on its presence key. A bus that cannot be asked leaves none away, which is what a walk did
// before: a machine is not passed on a reading that was not made.
var judgeAway = func(ctx context.Context, open *stores, now time.Time) (map[string]bool, error) {
nodes, err := open.inventory.Nodes(ctx)
if err != nil {
return nil, err
}
address, err := broker.BusAddress()
if err != nil {
return nil, err
}
js, err := broker.Dial(address)
if err != nil {
return nil, err
}
defer js.Close()
machines := make([]machineFacts, len(nodes))
for i, n := range nodes {
machines[i] = machineFacts{name: n.Name, lastHeard: n.LastSeen}
}
if err := withPresence(ctx, js.Conn(), machines, now); err != nil {
return nil, err
}
return awayOf(machines, now), nil
}
// rememberAway keeps on the plan the machines of running that are away: sorted, each once.
func rememberAway(p *inventory.Plan, running []string, away map[string]bool) {
for _, n := range running {
if away[n] && !slices.Contains(p.Away, n) {
p.Away = append(p.Away, n)
}
}
sort.Strings(p.Away)
}
// runningHeard is the machines of running that the walk waits on for a module: those a send of the walk reached, and
// those it did not pass as away.
func runningHeard(p inventory.Plan, running []string, s *inventory.PlanModule) []string {
return slices.DeleteFunc(slices.Clone(running), func(n string) bool {
if !slices.Contains(p.Away, n) {
return false
}
_, sent := s.Rest[n]
return !sent
})
}
// awaySaying is what a plan says of the machines it passed as away, empty when it passed none.
func awaySaying(p inventory.Plan) string {
var said []string
for _, n := range p.Away {
said = append(said, n+" was away: sent when it is heard again")
}
return strings.Join(said, "; ")
}
// judgedAway reads which machines are away for a tier about to be sent, and says when it cannot.
func judgedAway(ctx context.Context, open *stores, now time.Time) map[string]bool {
away, err := judgeAway(ctx, open, now)
if err != nil {
fmt.Fprintf(os.Stderr, "which machines are away cannot be read, so none is passed over: %v\n", err)
return nil // empty-on-error: none is passed over on a reading that was not made; the error is said just above
}
return away
}
@@ -0,0 +1,228 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"fmt"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// 1161/AC5, with the store and a real bus and no ticker: a plan of two tiers whose first tier runs on a machine away when it
// is sent finishes that tier without waiting for it and without its bound, sends the second tier to the machine heard
// alone, and says the machine was away (issue 1161). The machine away keeps its want: it was sent nothing of the plan's
// builds, so the send it gets when it is heard again carries them (AC3).
func TestAPlanGoesOnWithoutAMachineAwayWhenItsTierIsSent(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
// laptop is away as every tier is sent; anchor is heard.
wasAway := judgeAway
judgeAway = func(context.Context, *stores, time.Time) (map[string]bool, error) {
return map[string]bool{"laptop": true}, nil
}
t.Cleanup(func() { judgeAway = wasAway })
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
if err := broker.AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
if err := js.EnsureControllerBuckets(); err != nil {
t.Fatal(err)
}
legacy, err := js.Conn().JetStream()
if err != nil {
t.Fatal(err)
}
checkEvents = link.OverNATS{Conn: js.Conn(), JS: legacy}
t.Cleanup(func() { checkEvents = nil })
sentHeard := make(chan *nats.Msg, 32)
sub, err := js.Conn().ChanSubscribe(link.SeatEventSubject(link.MeshControllerSeat, link.KeySent), sentHeard)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = sub.Unsubscribe() })
wasSaved := inventory.PlanSaved
inventory.PlanSaved = servingPlanSaved(ctx, checkEvents, open)
t.Cleanup(func() { inventory.PlanSaved = wasSaved })
// The build machine and the machines, as far as the plan meets them: an ask is counted, a send is recorded and kept.
var mu sync.Mutex
var asks []string
var sends [][]string
wasAsk := askABuild
askABuild = func(_ context.Context, source buildSource, _, _ string) (string, error) {
mu.Lock()
defer mu.Unlock()
id := link.NewBuildID(time.Now().Add(time.Duration(len(asks)) * time.Millisecond))
asks = append(asks, source.Repository+"#"+id)
return id, nil
}
t.Cleanup(func() { askABuild = wasAsk })
digests := 0
wasSend := sendRollout
sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) {
mu.Lock()
defer mu.Unlock()
sends = append(sends, append([]string(nil), names...))
current, err := open.inventory.CurrentBuilds(ctx)
if err != nil {
return nil, err
}
builds := map[string]string{}
for _, m := range []string{"a", "b"} {
builds[m] = current[m].Commit
}
for _, node := range names {
digests++
if err := open.inventory.RecordSent(ctx, nodeID(t, open, node), fmt.Sprintf("d-%s-%d", node, digests), builds); err != nil {
return nil, err
}
}
return names, nil
}
t.Cleanup(func() { sendRollout = wasSend })
register := func(m, commit string, asked time.Time) {
t.Helper()
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: m, Version: commit},
inventory.Source{Repository: "novox/" + m, Seat: "git", Path: "modules/" + m, BuiltFrom: commit, Head: commit,
Asked: asked}); err != nil {
t.Fatal(err)
}
}
record := func(id, m, commit string, at time.Time) {
t.Helper()
b := inventory.Build{ID: id, Module: m, Commit: commit, Repository: "novox/" + m, Path: "modules/" + m, Asked: at, At: at}
b.Manifest, _ = json.Marshal(catalogue.Manifest{Module: m, Version: commit})
if err := inv.RecordBuild(ctx, b); err != nil {
t.Fatal(err)
}
}
old := time.Now().Add(-2 * time.Hour)
for _, m := range []string{"a", "b"} {
record("build-"+m+"-1", m, "c1", old)
register(m, "c1", old)
for _, n := range []string{"anchor", "laptop"} {
if _, err := inv.Assign(ctx, n, m); err != nil {
t.Fatal(err)
}
}
}
for _, n := range []string{"anchor", "laptop"} {
if err := inv.RecordSent(ctx, nodeID(t, open, n), "d-"+n+"-c1", map[string]string{"a": "c1", "b": "c1"}); err != nil {
t.Fatal(err)
}
}
record("build-a-2", "a", "c2", time.Now().Add(-time.Minute))
register("a", "c2", time.Now().Add(-time.Minute))
// A machine's report on what it was sent, heard as the controller hears it.
listener := nudgingListener{Enrolment: link.Enrolment{Inventory: inv}}
seq := int64(0)
reports := func(node string) {
t.Helper()
last, found, err := inv.SentTo(ctx, node)
if err != nil || !found {
t.Fatalf("%s was sent nothing: %v", node, err)
}
seq++
body, _ := json.Marshal(map[string]any{"node": node, "applied": []string{"x.server"}, "declared": last.Digest,
"report_sequence": seq})
var report link.Report
if err := json.Unmarshal(body, &report); err != nil {
t.Fatal(err)
}
if _, err := listener.Heard(ctx, report); err != nil {
t.Fatal(err)
}
}
plan := func() inventory.Plan {
t.Helper()
p, err := inv.PlanByID(ctx, "plan-1160")
if err != nil {
t.Fatal(err)
}
return p
}
built := time.Now().UTC()
p := inventory.Plan{ID: "plan-1160", Repository: "novox/mesh-catalog", Branch: "main", Commit: "c2", Created: time.Now().UTC(),
State: inventory.PlanBuilding, Tiers: [][]string{{"a"}, {"b"}},
Delivery: &inventory.PlanDelivery{Order: []inventory.Edge{{From: "b", To: "a", Kind: edgeChangeOrder}}},
Modules: map[string]*inventory.PlanModule{
"a": {State: "built", BuiltAt: &built, Commit: "c2", Build: "build-a-2"},
"b": {},
}}
first, stopFirst := context.WithCancel(ctx)
defer stopFirst()
waitFirst := startRolloutEvents(first, open, js)
// Ends with the test before the cleanups restore what the look reads, which run after it (issue 1338).
t.Cleanup(func() { stopFirst(); waitFirst() })
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
eventually(t, "tier 0 sent to anchor alone", func() bool {
mu.Lock()
defer mu.Unlock()
return len(sends) == 1 && strings.Join(sends[0], ",") == "anchor"
})
reports("anchor")
// The note of the move to tier 1 is read in the step that sees it: later steps of the tier say their own.
var moved string
eventually(t, "tier 1 reached on anchor's report alone", func() bool {
if q := plan(); q.Tier == 1 {
moved = q.Note
return true
}
return false
})
if !strings.Contains(moved, "laptop was away: sent when it is heard again") || strings.Contains(moved, "anchor") {
t.Errorf("1161/AC2: the plan says %q; want laptop named away and not anchor", moved)
}
// The ask for tier 1's build is the next step, and records its build id on the module.
eventually(t, "tier 1 asked", func() bool { m := plan().Modules["b"]; return m != nil && m.Build != "" })
b := plan().Modules["b"]
askedAt, _ := link.BuildAskedAt(b.Build)
record(b.Build, "b", "c2", askedAt)
register("b", "c2", askedAt)
planBuilt(ctx, open, "b", "c2", "", askedAt, b.Build)
eventually(t, "tier 1 sent to anchor alone", func() bool {
mu.Lock()
defer mu.Unlock()
return len(sends) == 2 && strings.Join(sends[1], ",") == "anchor"
})
// 1161/AC3: laptop was sent nothing of the plan's builds, so it still wants them for the send it gets when heard again.
was, known, err := inv.SentBuilds(ctx, "laptop")
if err != nil || !known {
t.Fatalf("1161/AC3: laptop's last send: %v, known %v", err, known)
}
current, err := inv.CurrentBuilds(ctx)
if err != nil {
t.Fatal(err)
}
if was["a"] == current["a"].Commit || was["a"] != "c1" {
t.Errorf("1161/AC3: laptop was last sent a at %q and a is now %q; want it still at c1, owed c2", was["a"], current["a"].Commit)
}
if got := plan().Away; strings.Join(got, ",") != "laptop" {
t.Errorf("1161/AC2: the plan keeps %v away; want laptop", got)
}
}
+67
View File
@@ -0,0 +1,67 @@
package main
import (
"slices"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// A machine out of touch when a tier is sent is not sent to and not waited for (novox/hq ADR 0435 §1, issue 1161): the
// walk judges it with the silent check's one verdict, machineFacts.silent, and a sleeping machine is away once presence says so.
// 1161/AC1: away is the silent check's verdict; AC4: a machine heard is not away.
func TestAMachineIsAwayAsTheSilentCheckJudgesIt(t *testing.T) {
now := time.Now().UTC()
machines := []machineFacts{
{name: "heard", lastHeard: now.Add(-time.Minute)},
{name: "quiet", lastHeard: now.Add(-silentBound - time.Minute)},
{name: "stuck", lastHeard: now, presence: presenceSilent},
{name: "back", lastHeard: now.Add(-time.Hour), presence: presenceHeard},
{name: "new"},
}
got := awayOf(machines, now)
for node, want := range map[string]bool{"heard": false, "quiet": true, "stuck": true, "back": false,
"new": false} {
if got[node] != want {
t.Errorf("1161/AC1: %s away is %v; want %v (away: %v)", node, got[node], want, got)
}
}
}
// 1161/AC1, AC2: the plan keeps the machines of the tier it passed as away, sorted and once.
func TestAPlanKeepsTheMachinesItPassedAsAway(t *testing.T) {
p := &inventory.Plan{}
away := map[string]bool{"g14": true, "ace": true}
rememberAway(p, []string{"anchor", "g14"}, away)
rememberAway(p, []string{"g14", "ace"}, away)
if !slices.Equal(p.Away, []string{"ace", "g14"}) {
t.Fatalf("1161/AC2: the plan keeps %v away; want ace, g14", p.Away)
}
}
// 1161/AC2: the plan says it, where its other passed machines are said.
func TestAPlanSaysTheMachinesItPassedAsAway(t *testing.T) {
say := awaySaying(inventory.Plan{Away: []string{"g14"}})
if !strings.Contains(say, "g14 was away: sent when it is heard again") {
t.Fatalf("1161/AC2: the plan says %q", say)
}
if got := awaySaying(inventory.Plan{}); got != "" {
t.Fatalf("1161/AC2: a plan that passed no machine says %q", got)
}
}
// 1161/AC1: the gate does not wait for a machine passed as away, unless a send reached it; AC4: a machine heard is waited for.
func TestAGateDoesNotWaitForAMachinePassedAsAway(t *testing.T) {
p := inventory.Plan{Away: []string{"g14"}}
none := &inventory.PlanModule{}
if got := runningHeard(p, []string{"anchor", "g14"}, none); !slices.Equal(got, []string{"anchor"}) {
t.Fatalf("1161/AC1: the gate waits on %v; want anchor alone", got)
}
sent := &inventory.PlanModule{Rest: map[string]inventory.SentDeclaration{"g14": {Digest: "d"}}}
if got := runningHeard(p, []string{"anchor", "g14"}, sent); !slices.Equal(got, []string{"anchor", "g14"}) {
t.Fatalf("1161/AC4: a machine a send reached is waited for: %v", got)
}
}
@@ -10,9 +10,9 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
+68 -23
View File
@@ -124,6 +124,52 @@ func durationSetting(v any) (time.Duration, bool) {
return 0, false
}
// mergeFacts is what hearMerge judges a merge by, kept apart so the catch-up (ADR 0457) judges a merge the way hearing it will.
type mergeFacts struct {
merged time.Time
touches []inventory.Entry
later []inventory.BatchedMerge
owed bool
from, packaging []inventory.Entry
already int
addsModuleToBuilds bool // a first merge into a repository the mesh builds nothing from: firstBuilds acts on it
}
// factsOfMerge reads what hearing a merge turns on: the modules it touches whatever the history, the later merges of its
// branch, whether it is owed late, and the modules it moves.
func factsOfMerge(ctx context.Context, inv *inventory.Inventory, m link.SourceMoved, now time.Time,
entries []inventory.Entry, read map[string][]inventory.ReadRepository) (mergeFacts, error) {
repository := m.Owner + "/" + m.Repo
f := mergeFacts{}
var err error
f.merged, err = time.Parse(time.RFC3339Nano, m.MergedAt)
if err != nil {
f.merged = now
}
// **What it touches, history or not** (review of this change): a merge heard after a later merge of its
// branch was cut reads as history for every module that cut marked seen, and was dropped from the record
// here, never named by the walk that carried it.
raw := m
raw.MergedAt = ""
f.touches = wouldMove(raw, entries, read)
if len(f.touches) > 0 {
if f.later, err = inv.LaterMergesOf(ctx, repository, m.Base, f.merged.UTC()); err != nil {
return f, err
}
}
if f.owed, err = owedLate(ctx, inv, m, f.merged, f.touches); err != nil {
return f, err
}
f.from, f.packaging, f.already = mergeCandidates(m, entries, read)
f.addsModuleToBuilds = f.already == 0 && len(addedDirs(m, entries, read)) > 0
return f, nil
}
// plans says hearing the merge puts it in a batch or a walk.
func (f mergeFacts) plans() bool {
return len(f.from) > 0 || len(f.packaging) > 0 || len(f.later) > 0 || f.owed
}
// hearMerge puts a merge the forge announced into the batch that answers it, at the moment it was heard,
// and cuts what is due.
func (f following) hearMerge(ctx context.Context, m link.SourceMoved, now time.Time) error {
@@ -148,27 +194,13 @@ func (f following) hearMerge(ctx context.Context, m link.SourceMoved, now time.T
if err != nil {
return notNow(err)
}
merged, err := time.Parse(time.RFC3339Nano, m.MergedAt)
if err != nil {
merged = now
}
// **What it touches, history or not** (review of this change): a merge heard after a later merge of its
// branch was cut reads as history for every module that cut marked seen, and was dropped from the record
// here, never named by the walk that carried it.
raw := m
raw.MergedAt = ""
touches := wouldMove(raw, entries, read)
var later []inventory.BatchedMerge
if len(touches) > 0 {
if later, err = inv.LaterMergesOf(ctx, repository, m.Base, merged.UTC()); err != nil {
return notNow(err)
}
}
owed, err := owedLate(ctx, inv, m, merged, touches)
facts, err := factsOfMerge(ctx, inv, m, now, entries, read)
if err != nil {
return notNow(err)
}
if from, packaging, already := mergeCandidates(m, entries, read); len(from) == 0 && len(packaging) == 0 && len(later) == 0 && !owed {
merged, touches := facts.merged, facts.touches
if !facts.plans() {
already := facts.already
// "Already built from it" and "nothing reads it" are different facts, and reading the first
// as the second sends somebody looking for a broken trigger when the mesh is up to date.
if already > 0 {
@@ -176,6 +208,11 @@ func (f following) hearMerge(ctx context.Context, m link.SourceMoved, now time.T
repository, m.Base, m.Commit, already)
return nil
}
if already == 0 {
if done, err := f.firstBuilds(ctx, m, entries, read, now); err != nil || done {
return err
}
}
fmt.Print(nothingReadsIt(repository, m))
return nil
}
@@ -362,7 +399,7 @@ func buildsEvery(p inventory.Plan, modules []inventory.Entry) bool {
// ownPath says a merge touches a module on the controller's own path, whose walk waits for nobody's word.
func ownPath(touches []inventory.Entry) bool {
for _, e := range touches {
if _, own := onTheControllersPath[e.Manifest.Module]; own {
if onTheOwnPath(e.Manifest.Module, e.Manifest) {
return true
}
}
@@ -742,6 +779,9 @@ func carriedOf(merges []inventory.BatchedMerge) ([]inventory.PlanCommit, []inven
type keptMerge struct {
link.SourceMoved
Moves []string `json:"moves,omitempty"`
// FirstBuild says the merge is a first merge into a repository the mesh builds nothing from, kept to ask the first
// build of its modules once mesh-delivery's word on it is kept (novox/hq ADR 0432, issue 1146).
FirstBuild bool `json:"first_build,omitempty"`
}
// announcedOf is a kept merge as the forge announced it, and what it moved; empty where the record says none.
@@ -937,7 +977,11 @@ func cutUnrelated(ctx context.Context, open *stores, batches, started, waiting [
var read map[string][]inventory.ReadRepository
var edges []inventory.Edge
judged := false
openWalks, err := openWalksOf(ctx, inv, started)
onPath, err := ownPathNow(ctx, inv)
if err != nil {
return err
}
openWalks, err := openWalksOf(ctx, inv, started, onPath)
if err != nil {
return err
}
@@ -970,7 +1014,7 @@ func cutUnrelated(ctx context.Context, open *stores, batches, started, waiting [
var fold, keep []inventory.Plan
if !b.OwnPath() {
for _, w := range waiting {
ww, err := openWalkOf(ctx, inv, w)
ww, err := openWalkOf(ctx, inv, w, onPath)
if err != nil {
return err
}
@@ -1005,7 +1049,7 @@ func cutUnrelated(ctx context.Context, open *stores, batches, started, waiting [
if !cut.Open() {
continue
}
w, err := openWalkOf(ctx, inv, cut)
w, err := openWalkOf(ctx, inv, cut, onPath)
if err != nil {
return err
}
@@ -1383,8 +1427,9 @@ func classOfModules(modules []string, entries []inventory.Entry) string {
}
}
}
onPath := ownPathOf(entries)
for _, m := range modules {
if _, own := onTheControllersPath[m]; own || resolvers[m] {
if _, own := onPath[m]; own || resolvers[m] {
return inventory.ClassCore
}
}
@@ -11,7 +11,7 @@ import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+4 -2
View File
@@ -4,7 +4,7 @@ import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
)
@@ -73,6 +73,8 @@ func reportKept(ctx context.Context, plan catalogue.Resolution) {
// sent, on every machine — the resolver kept it there (said, urgent, until a person pins), or a pin
// moves it (said, so the data goes first), and never anything else.
func probeBindings(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
// One reading of the mesh for every machine this looks at.
ctx = withInputCache(ctx)
inv := d.open.inventory
nodes, err := inv.Nodes(ctx)
if err != nil {
@@ -80,7 +82,7 @@ func probeBindings(ctx context.Context, d *doctor) ([]conditions.Observation, er
}
var out []conditions.Observation
for _, n := range nodes {
plan, _, err := planFor(ctx, d.open, n.Name)
plan, _, err := resolveNode(ctx, d.open, n.Name)
if err != nil {
if unresolvable(err) {
// D1 says it, with the binding that refused it when that is why.
@@ -6,7 +6,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
)
@@ -37,7 +37,7 @@ func TestTheIncidentAConsumerStaysBesideItsDataUntilAPersonPinsIt(t *testing.T)
assignAll([2]string{"laptop", "store"}, [2]string{"laptop", "resolver"}, [2]string{"laptop", "network"},
[2]string{"laptop", "board"})
plan, _, err := planFor(ctx, open, "laptop")
plan, _, err := resolveNode(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
@@ -64,7 +64,7 @@ func TestTheIncidentAConsumerStaysBesideItsDataUntilAPersonPinsIt(t *testing.T)
if err := inv.Unassign(ctx, "laptop", "store"); err != nil {
t.Fatal(err)
}
if _, _, err := planFor(ctx, open, "laptop"); err == nil || !unresolvable(err) ||
if _, _, err := resolveNode(ctx, open, "laptop"); err == nil || !unresolvable(err) ||
!strings.Contains(err.Error(), "board on laptop is bound to laptop/store") ||
!strings.Contains(err.Error(), "pin laptop postgres-database anchor store") {
t.Fatalf("the consumer's store went and it was answered elsewhere: %v", err)
@@ -74,7 +74,7 @@ func TestTheIncidentAConsumerStaysBesideItsDataUntilAPersonPinsIt(t *testing.T)
if err := inv.PinProvision(ctx, "laptop", "postgres-database", "anchor", "store"); err != nil {
t.Fatal(err)
}
plan, _, err = planFor(ctx, open, "laptop")
plan, _, err = resolveNode(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
)
+51 -40
View File
@@ -7,6 +7,7 @@ import (
"flag"
"fmt"
"github.com/novox/mesh-controller/internal/conditions"
"io"
"os"
"slices"
"strings"
@@ -14,9 +15,9 @@ import (
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -152,6 +153,7 @@ func buildFrom(result link.BuildResult) inventory.Build {
Path: result.Path,
// What it was made from (novox/hq issue 280): two builds with one are one build.
SourceFingerprint: result.SourceFingerprint,
SourceInputs: result.SourceInputs,
// And its reuse key (novox/hq ADR 0350 §1), which the chain names (ADR 0408 §1).
ReuseKey: result.ReuseKey,
}
@@ -236,47 +238,56 @@ func buildsCommand(ctx context.Context, args []string) error {
}
for _, b := range builds {
what := b.Module
if what == "" {
// It failed before knowing what it was building, which is most of the interesting
// failures. The repository is what a person has to go and look at.
what = "?"
}
outcome := "built " + short(b.Commit)
if !b.Worked() {
outcome = "failed"
}
fmt.Printf("%-18s %-14s %-10s %s %s\n",
what, outcome, b.On, b.At.Local().Format("2006-01-02 15:04"), b.ID)
fmt.Printf(" %s", b.Repository)
if b.Ref != "" {
fmt.Printf(" at %s", b.Ref)
}
fmt.Println()
for _, made := range b.Made {
fmt.Printf(" %-10s %s\n", made.Kind, made.Reference)
}
// What it published and under what reuse key (novox/hq ADR 0408 §1).
if p := b.Published; p != nil {
if p.Artifact != "" {
fmt.Printf(" published %s\n", p.Artifact)
}
if p.Package != nil {
fmt.Printf(" published %s@%s\n", p.Package.Name, p.Package.Version)
}
}
if b.ReuseKey != "" {
fmt.Printf(" reuse key %s\n", b.ReuseKey)
}
if !b.Worked() {
// The builder's own first line. The whole failure is often a build log, and printing
// it here would bury every other row.
fmt.Printf(" %s\n", firstLine(b.Failed))
}
printBuild(os.Stdout, b)
}
return nil
}
// printBuild says one build: what, its outcome, where and when, what it made and published, its reuse key, and its
// source fingerprint (novox/hq issue 1204: the one tool that reads what the builder recorded it was made from).
func printBuild(w io.Writer, b inventory.Build) {
what := b.Module
if what == "" {
// It failed before knowing what it was building, which is most of the interesting
// failures. The repository is what a person has to go and look at.
what = "?"
}
outcome := "built " + short(b.Commit)
if !b.Worked() {
outcome = "failed"
}
fmt.Fprintf(w, "%-18s %-14s %-10s %s %s\n",
what, outcome, b.On, b.At.Local().Format("2006-01-02 15:04"), b.ID)
fmt.Fprintf(w, " %s", b.Repository)
if b.Ref != "" {
fmt.Fprintf(w, " at %s", b.Ref)
}
fmt.Fprintln(w)
for _, made := range b.Made {
fmt.Fprintf(w, " %-10s %s\n", made.Kind, made.Reference)
}
// What it published and under what reuse key (novox/hq ADR 0408 §1).
if p := b.Published; p != nil {
if p.Artifact != "" {
fmt.Fprintf(w, " published %s\n", p.Artifact)
}
if p.Package != nil {
fmt.Fprintf(w, " published %s@%s\n", p.Package.Name, p.Package.Version)
}
}
if b.ReuseKey != "" {
fmt.Fprintf(w, " reuse key %s\n", b.ReuseKey)
}
if b.SourceFingerprint != "" {
fmt.Fprintf(w, " source %s\n", b.SourceFingerprint)
}
if !b.Worked() {
// The builder's own first line. The whole failure is often a build log, and printing
// it here would bury every other row.
fmt.Fprintf(w, " %s\n", firstLine(b.Failed))
}
}
// builderCommand issues a build machine its own broker credential.
//
// **A build machine is not a node**, and giving it a node's account would let it read another
@@ -473,7 +484,7 @@ func buildOneAsked(ctx context.Context, source buildSource, path, ref string, wa
return "", err
}
// Through a verb, only a repository the catalogue builds from (novox/hq ADR 0266).
if err := verbMayAsk(ctx, source, repository); err != nil {
if err := verbMayAsk(ctx, source, repository, path); err != nil {
return "", err
}
@@ -812,7 +823,7 @@ func buildAndShow(ctx context.Context, source buildSource, path, ref string, wai
if err != nil {
return err
}
if err := verbMayAsk(ctx, source, repository); err != nil {
if err := verbMayAsk(ctx, source, repository, path); err != nil {
return err
}
ident, err := openIdentity(ctx)
+1 -1
View File
@@ -3,7 +3,7 @@ package main
import (
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+73 -9
View File
@@ -10,7 +10,7 @@ import (
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -88,7 +88,7 @@ func newSourceForms(ctx context.Context, inv *inventory.Inventory) *sourceForms
if world == nil {
w := catalogue.World{}
if shelf, err := inv.Catalogue(ctx); err == nil {
if read, err := theRestOfTheMesh(ctx, inv, shelf, ""); err == nil {
if read, err := restOfTheMeshOf(ctx, inv, shelf, ""); err == nil {
w = read
}
}
@@ -389,10 +389,31 @@ func (f *sourceForms) onTheForge(repository, seat string) (owner, name string, o
// least one required status, no administrator merging past one.
//
// Anything else only when the build request was kept as asked at the controller's terminal, for this very
// repository and path. path is the module's directory as built; branch the trunk it is registered from.
// repository and path; or, for the build of a merge, when mesh-delivery announced it on the operator's Approve of
// that head (novox/hq ADR 0420 §7 for a moved module, ADR 0432 for a new one), the forge's checks still holding. path is the module's directory as built; branch the trunk it is registered from.
func mayRegisterFrom(ctx context.Context, inv *inventory.Inventory, module string, built inventory.Source,
buildID, path, branch string) (int64, error) {
forms := newSourceForms(ctx, inv)
return mayRegisterIn(ctx, inv, newSourceForms(ctx, inv), module, built, buildID, path, branch)
}
// catalogueReader is the catalogue's manifests, as the router's records are found in them.
type catalogueReader interface {
Catalogue(ctx context.Context) (map[string]catalogue.Manifest, error)
}
// registrations is what a registration's decision reads of the mesh: *inventory.Inventory, or a test's stand-in.
type registrations interface {
catalogueReader
SourceOf(ctx context.Context, module string) (inventory.Source, error)
SourceIdentity(ctx context.Context, module string) (int64, error)
Catalogued(ctx context.Context) ([]inventory.Entry, error)
BuildRequestByID(ctx context.Context, id string) (inventory.BuildRequest, bool, error)
MergeOf(ctx context.Context, repository, commit string) (inventory.BatchedMerge, bool, error)
}
// mayRegisterIn is mayRegisterFrom on what it reads of the mesh handed in.
func mayRegisterIn(ctx context.Context, inv registrations, forms *sourceForms, module string, built inventory.Source,
buildID, path, branch string) (int64, error) {
was, err := inv.SourceOf(ctx, module)
isNew := errors.Is(err, inventory.ErrNoSuchModule)
if err != nil && !isNew {
@@ -404,7 +425,8 @@ func mayRegisterFrom(ctx context.Context, inv *inventory.Inventory, module strin
}
refuse := func(why string) (int64, error) {
return 0, fmt.Errorf("%w: %s. A module is registered from such a source only by a build asked at the "+
"controller's terminal, never through a verb: a registered module is what the next push sends, and whoever "+
"controller's terminal, or by the build of a merge mesh-delivery announced on the operator's Approve of that "+
"head, never through a verb: a registered module is what the next push sends, and whoever "+
"may call a verb includes agents (novox/hq ADR 0266)", errNotItsSource, why)
}
@@ -438,8 +460,11 @@ func mayRegisterFrom(ctx context.Context, inv *inventory.Inventory, module strin
// **Or the move the operator approved** (novox/hq ADR 0420 §7, issue 1025): a known module built from another
// repository by a merge, whose pull request mesh-delivery merged on the operator's Approve of that head.
// A verb alone, or a merge with no such warrant, is refused as before.
// **Or a new module from a repository nothing registered is built from, on the same warrant** (novox/hq ADR 0432,
// issue 1146): the build of a merge mesh-delivery announced on the operator's Approve of that head registers it.
// The forge's protection and the repository's identity are checked below as for every registration.
movedWhy := ""
if why != "" && !terminal && !isNew {
if why != "" && !terminal {
accepted, err := approvedMove(ctx, inv, forms, buildID, built)
switch {
case err == nil:
@@ -507,7 +532,10 @@ func mayRegisterFrom(ctx context.Context, inv *inventory.Inventory, module strin
owner, name, m, facts.ID, m, id))
}
}
if movedWhy != "" {
if movedWhy != "" && isNew {
fmt.Printf("%s: %s is new, from %s/%s — registered at the operator's Approve: %s\n", buildID, module, owner, name,
movedWhy)
} else if movedWhy != "" {
fmt.Printf("%s: %s moved from %s to %s/%s — registered at the operator's Approve: %s\n", buildID, module,
sourceWords(was.Repository, was.Seat), owner, name, movedWhy)
}
@@ -516,7 +544,7 @@ func mayRegisterFrom(ctx context.Context, inv *inventory.Inventory, module strin
// askedHereFor says the build was asked at the controller's terminal, for this repository and this path: a kept
// request marked so, whose source is the outcome's (novox/hq ADR 0266).
func askedHereFor(ctx context.Context, inv *inventory.Inventory, forms *sourceForms, buildID string,
func askedHereFor(ctx context.Context, inv registrations, forms *sourceForms, buildID string,
built inventory.Source, path string) (bool, error) {
r, found, err := inv.BuildRequestByID(ctx, buildID)
if err != nil || !found || !r.AtTerminal {
@@ -539,13 +567,49 @@ func sourceWords(repository, seat string) string {
return buildSource{Repository: repository, Seat: seat}.String()
}
// firstBuildWarrant is what the controller checked before it asks the first build of the modules a merge adds to a
// repository nothing is built from: the merge was made on the operator's Approve of its head (approvedAnnounced). It
// travels in the context of that ask alone, in this process, so no verb carries one.
type firstBuildWarrant struct {
repository string // owner/name on the git seat
paths map[string]bool
why string
}
type firstBuildWarrantKey struct{}
// withFirstBuildWarrant is ctx carrying the warrant for asking the first builds of paths in repository.
func withFirstBuildWarrant(ctx context.Context, repository string, paths []string, why string) context.Context {
w := &firstBuildWarrant{repository: repository, paths: map[string]bool{}, why: why}
for _, p := range paths {
w.paths[strings.Trim(p, "/")] = true
}
return context.WithValue(ctx, firstBuildWarrantKey{}, w)
}
func firstBuildWarrantIn(ctx context.Context) *firstBuildWarrant {
w, _ := ctx.Value(firstBuildWarrantKey{}).(*firstBuildWarrant)
return w
}
// covers says the warrant is for exactly this ask: the repository on the git seat, and a module path the merge holds.
func (w *firstBuildWarrant) covers(source buildSource, path string) bool {
return w != nil && source.Seat == gitSeat && strings.EqualFold(strings.Trim(source.Repository, "/"), w.repository) &&
w.paths[strings.Trim(path, "/")]
}
// verbMayAsk refuses, for a call through a verb, a build of a repository the catalogue builds no module from
// (novox/hq ADR 0266): the build node would run what an agent wrote, and its outcome could never be
// registered anyway. url is the repository as it is cloned. At the terminal anything may be asked.
func verbMayAsk(ctx context.Context, source buildSource, url string) error {
func verbMayAsk(ctx context.Context, source buildSource, url, path string) error {
if startedAtTheTerminal() {
return nil
}
// **Or the first build of a module a merge adds on the operator's Approve** (novox/hq ADR 0432, issue 1146): the
// controller's own ask, carrying the warrant it checked, for that repository and the module paths the merge holds.
if firstBuildWarrantIn(ctx).covers(source, path) {
return nil
}
open, err := openStores(ctx)
if err != nil {
return err
@@ -134,18 +134,18 @@ func TestAVerbAsksNoBuildOfARepositoryTheCatalogueDoesNotBuildFrom(t *testing.T)
ctx := t.Context()
t.Setenv(verbVar, "build")
t.Setenv(link.CallerVar, "node-tools.anchor, through the mesh-controller seat")
err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, "http://forge.internal:20000/agent/sudo.git")
err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, "http://forge.internal:20000/agent/sudo.git", "")
var policy *heldAtTheTerminal
if !errors.As(err, &policy) {
t.Fatalf("a verb's build of an agent's repository was asked: %v", err)
}
if err := verbMayAsk(ctx, buildSource{Repository: "novox/mesh-catalog", Seat: "git"},
"http://forge.internal:20000/novox/mesh-catalog.git"); err != nil {
"http://forge.internal:20000/novox/mesh-catalog.git", ""); err != nil {
t.Fatalf("a verb's build of the catalogue repository was refused: %v", err)
}
t.Setenv(verbVar, "")
t.Setenv(link.CallerVar, "")
if err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, ""); err != nil {
if err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, "", ""); err != nil {
t.Fatalf("the terminal was refused: %v", err)
}
}
@@ -317,7 +317,7 @@ func TestAModuleMovesOnTheOperatorsApproveOfTheMergeThatMovedIt(t *testing.T) {
keptDeliveryMerge = func(_ context.Context, owner, repo string, number int, head string) (deliveryMerge, bool, error) {
return a.merge, !c.unkept && owner == "novox" && repo == "mesh-host" && number == 7 && head == a.merge.HeadSHA, nil
}
routerAskOf = func(_ context.Context, _ *inventory.Inventory, id string) (*asks.Ask, *asks.Warrant, error) {
routerAskOf = func(_ context.Context, _ catalogueReader, _, id string) (*asks.Ask, *asks.Warrant, error) {
if id != a.merge.AskID {
t.Errorf("%s: the router was asked for %q", c.what, id)
}
+45
View File
@@ -1,9 +1,14 @@
package main
import (
"bytes"
"reflect"
"strings"
"testing"
"time"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -32,3 +37,43 @@ func TestABuildIDSaysWhenItWasAsked(t *testing.T) {
}
}
}
// novox/hq issue 1204: builds says the source fingerprint the builder recorded, the one tool that reads it.
func TestBuildsSaysTheSourceFingerprint(t *testing.T) {
at := time.Date(2026, 10, 19, 10, 40, 0, 0, time.UTC)
built := inventory.Build{ID: "build-1", Module: "mailu", Commit: "b4527d9091e972f28cb365623d9280045e7741a3", On: "ace",
Repository: "novox/mesh-catalog", At: at, ReuseKey: "r1:abc",
SourceFingerprint: "src1:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}
// 1204/AC1
var out bytes.Buffer
printBuild(&out, built)
lines := strings.Split(strings.TrimRight(out.String(), "\n"), "\n")
if last := lines[len(lines)-1]; last != " source "+built.SourceFingerprint {
t.Errorf("last line %q; want the source line after the reuse key:\n%s", last, out.String())
}
if !strings.Contains(lines[len(lines)-2], "reuse key r1:abc") {
t.Errorf("the source line does not follow the reuse key:\n%s", out.String())
}
// 1204/AC2
without := built
without.SourceFingerprint = ""
var plain bytes.Buffer
printBuild(&plain, without)
if strings.Contains(plain.String(), "source") {
t.Errorf("a build with no fingerprint says a source line:\n%s", plain.String())
}
if want := strings.Join(lines[:len(lines)-1], "\n") + "\n"; plain.String() != want {
t.Errorf("the other lines changed:\n%s\nwant\n%s", plain.String(), want)
}
}
// 1215/AC4: what the builder said of the inputs is what the controller keeps of the build.
func TestTheControllerKeepsTheInputsABuilderSaid(t *testing.T) {
in := &sdkdelivery.SourceInputs{Module: "mailu", Tree: "t", Toolchains: map[string]string{"code": "ts sha256:aa 01"}}
kept := buildFrom(link.BuildResult{ID: "build-1", Repository: "r", Commit: "c", SourceFingerprint: "src1:x", SourceInputs: in})
if !reflect.DeepEqual(kept.SourceInputs, in) {
t.Errorf("the controller keeps %+v, the builder said %+v", kept.SourceInputs, in)
}
}
+1 -1
View File
@@ -11,7 +11,7 @@ import (
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
+1 -1
View File
@@ -9,8 +9,8 @@ import (
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
+647
View File
@@ -0,0 +1,647 @@
package main
import (
"context"
"errors"
"fmt"
"net/url"
"path"
"regexp"
"slices"
"sort"
"strconv"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/asks"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **The controller catches up on merges it did not hear** (novox/hq ADR 0457, issue 1336). A merge the forge announced while the
// controller was being replaced was never heard, and nothing asked for it again: the trunk held a merge no walk answered. At its
// start, and each time it hears a merge, the controller reads the last catchUpWindow merge commits of a trunk from the forge, and
// hears each one it holds no record of as if the forge had announced it then, on the warrant the router holds for it.
// It lasts until release 6 of design 47 deletes delivery plans and batches (ADR 0429 §7, §8).
// catchUpWindow is how many of a trunk's newest merge commits are read; an older merge is a person's to decide.
const catchUpWindow = 50
// kindMergeNotHeard is the condition of a merge the controller did not hear and holds no warrant to plan.
const kindMergeNotHeard = "merge-not-heard"
// trunkMerge is a merge commit of a trunk as the forge lists it.
type trunkMerge struct {
Commit string
// Head is the merge commit's second parent: the pull request's head that was merged.
Head string
Subject string
At time.Time
}
// trunkCommit is a commit of a trunk as the forge lists it, newest first: a merge commit has a second parent.
type trunkCommit struct {
Commit, SecondParent, Subject string
At time.Time
}
// trunkCommitsPage is one page (fifty) of a trunk's commits, newest first; empty past the last. A variable so a test needs no
// forge.
var trunkCommitsPage = func(ctx context.Context, owner, repo, branch string, page int) ([]trunkCommit, error) {
var found struct {
Result []struct {
SHA string `json:"sha"`
Parents []struct {
SHA string `json:"sha"`
} `json:"parents"`
Commit struct {
Message string `json:"message"`
Committer struct {
Date string `json:"date"`
} `json:"committer"`
} `json:"commit"`
} `json:"result"`
}
apiPath := fmt.Sprintf("/repos/%s/%s/commits?sha=%s&limit=50&page=%d&stat=false&verification=false&files=false",
owner, repo, url.QueryEscape(branch), page)
missing, err := askForgeApi(ctx, apiPath, &found)
if err != nil {
return nil, err
}
if missing {
return nil, fmt.Errorf("the forge holds no branch %s of %s/%s", branch, owner, repo)
}
var out []trunkCommit
for _, c := range found.Result {
at, _ := time.Parse(time.RFC3339, c.Commit.Committer.Date)
tc := trunkCommit{Commit: c.SHA, Subject: firstLine(c.Commit.Message), At: at.UTC()}
if len(c.Parents) >= 2 {
tc.SecondParent = c.Parents[1].SHA
}
out = append(out, tc)
}
return out, nil
}
// readTrunkMerges is the last catchUpWindow merge commits of a repository's trunk, oldest first: pages are read until that many
// merges are held or the trunk ends.
func readTrunkMerges(ctx context.Context, owner, repo, branch string) ([]trunkMerge, error) {
var merges []trunkMerge
for page := 1; page <= 8 && len(merges) < catchUpWindow; page++ {
commits, err := trunkCommitsPage(ctx, owner, repo, branch, page)
if err != nil {
return nil, err
}
if len(commits) == 0 {
break
}
for _, c := range commits {
// A merge of the trunk is a pull request's merge into it. A commit with a second parent that is none is a merge made
// inside some branch ("Merge remote-tracking branch 'origin/main' into fix/..."), which the trunk's history lists too and
// which no Approve ever covered: read as a gap it would raise a condition for nothing a person decides.
if c.SecondParent != "" && mergesIntoTrunk(c.Subject, branch) {
merges = append(merges, trunkMerge{Commit: c.Commit, Head: c.SecondParent, Subject: c.Subject, At: c.At})
}
}
}
return lastMerges(merges), nil
}
// sinceTheRecord are the merges that can be gaps (novox/hq ADR 0463 §2): those made since the controller's record of merges begins on
// the mesh (first, the earliest merge it holds of any repository and branch, less mergeGrace as owedLate judges). A merge before it is
// history on every trunk, and a trunk with no merge kept of its own still has the merges made since as gaps. With no merge kept
// anywhere there is no record to have a gap in.
func sinceTheRecord(merges []trunkMerge, kept bool, first time.Time) []trunkMerge {
if !kept {
return nil
}
var out []trunkMerge
for _, m := range merges {
if !m.At.Before(first.Add(-mergeGrace)) {
out = append(out, m)
}
}
return out
}
// lastMerges is the newest catchUpWindow of merges, given newest first, oldest first.
func lastMerges(newestFirst []trunkMerge) []trunkMerge {
if len(newestFirst) > catchUpWindow {
newestFirst = newestFirst[:catchUpWindow]
}
out := slices.Clone(newestFirst)
slices.Reverse(out)
return out
}
var mergeSubject = regexp.MustCompile(`^Merge pull request '(.*)' \(#(\d+)\) from (\S+) into (\S+)$`)
// pullOfSubject is the pull request a merge commit's subject names: its title and number.
func pullOfSubject(subject string) (title string, number int, ok bool) {
m := mergeSubject.FindStringSubmatch(strings.TrimSpace(subject))
if m == nil {
return "", 0, false
}
n, err := strconv.Atoi(m[2])
if err != nil || n <= 0 {
return "", 0, false
}
return m[1], n, true
}
// mergesIntoTrunk says a commit's subject is a pull request's merge into this trunk.
func mergesIntoTrunk(subject, branch string) bool {
m := mergeSubject.FindStringSubmatch(strings.TrimSpace(subject))
return m != nil && m[4] == branch
}
// headBranchOfSubject is the branch a merge commit's subject says was merged.
func headBranchOfSubject(subject string) string {
if m := mergeSubject.FindStringSubmatch(strings.TrimSpace(subject)); m != nil {
return m[3]
}
return ""
}
// readMergeEvent is the merge as the forge would have announced it: the commit, the pull request, the files it changed and the
// directories among them that hold a module.json. A variable so a test needs no forge.
var readMergeEvent = func(ctx context.Context, owner, repo, branch string, tm trunkMerge) (link.SourceMoved, error) {
m := link.SourceMoved{Owner: owner, Repo: repo, Base: branch, Head: headBranchOfSubject(tm.Subject), HeadSHA: tm.Head,
Commit: tm.Commit, MergedAt: tm.At.UTC().Format(time.RFC3339)}
if title, number, ok := pullOfSubject(tm.Subject); ok {
m.Number, m.Title = number, title
}
var found struct {
Result struct {
Files []struct {
Filename string `json:"filename"`
Status string `json:"status"`
} `json:"files"`
} `json:"result"`
}
missing, err := askForgeApi(ctx, "/repos/"+owner+"/"+repo+"/git/commits/"+url.PathEscape(tm.Commit), &found)
if err != nil {
return m, err
}
if missing || len(found.Result.Files) == 0 {
return m, fmt.Errorf("the forge named no files for the merge %.8s of %s/%s", tm.Commit, owner, repo)
}
for _, f := range found.Result.Files {
m.Paths = append(m.Paths, f.Filename)
if f.Status == "removed" || f.Status == "deleted" {
m.Removed = append(m.Removed, f.Filename)
}
}
// The forge lists a merge's files by the page; a list as long as a page may be a beginning.
m.PathsTruncated = len(m.Paths) >= 300
dirs := map[string]bool{}
for _, p := range m.Paths {
for d := path.Dir(p); ; d = path.Dir(d) {
dirs[d] = true
if d == "." || d == "/" {
break
}
}
}
if len(dirs) > 64 {
return m, nil // too many to look at: the old rule stands, a directory is a module when its manifest changed
}
var holding []string
for d := range dirs {
file := "module.json"
if d != "." {
file = d + "/module.json"
}
_, exists, err := manifestCached(ctx, owner, repo, file, tm.Commit)
if err != nil {
return m, err
}
if exists {
holding = append(holding, d)
}
}
sort.Strings(holding)
m.ModuleDirs, m.ModuleDirsSaid = holding, true
return m, nil
}
// mergeAskPrefix is what the id of a merge ask made by the retired module mesh-delivery begins with: the pull request's number
// and the head's first twelve digits (`merge-<number>-<head>-<try>`). The teller's ids name neither (see findMergeWarrant).
func mergeAskPrefix(number int, head string) string {
if len(head) > 12 {
head = head[:12]
}
return fmt.Sprintf("merge-%d-%s-", number, head)
}
// askersOfMerges are the askers a merge's warrant is read under: the teller, and mesh-delivery for a merge before release 3e
// (ADR 0437 §6). Release 4's controller work stops accepting mesh-delivery (ADR 0450 §8); a merge asked under it earlier is
// still found here until then.
func askersOfMerges() []string { return []string{sdkdelivery.TellerModule, theDelivery} }
// caughtUpWarrant judges a chosen merge ask of the router as the warrant of a merge: the ask is the operator's, the warrant
// answers that very ask as it was published, and it is the operator's Approve proved by the channel it came through (ADR 0437
// §6, as movedOnApprove judges an announced one). The heads the Approve was bound to are checked by the caller, where it can
// name them.
func caughtUpWarrant(asker string, ask *asks.Ask, w *asks.Warrant) (string, error) {
if err := askerAccepted(asker); err != nil {
return "", err
}
switch {
case ask == nil || w == nil:
return "", errors.New("the router holds no closed record of the ask")
case ask.Who != asks.Operator:
return "", fmt.Errorf("the ask %s is for %q to answer, not the operator", ask.ID, ask.Who)
}
option, err := w.For(asker, *ask)
if err != nil {
return "", err
}
if option.ID != optionApprove || w.Option != optionApprove {
return "", fmt.Errorf("the operator chose %q on the ask %s, not an Approve", w.Option, w.Ask)
}
if w.By == nil || w.By.Who != asks.Operator || w.By.Verified == "" {
return "", fmt.Errorf("the ask %s was not answered by the operator on a channel that proves who answered", w.Ask)
}
return w.Says() + " (ask " + w.Ask + ")", nil
}
// routerAskIDs are the ids of the asks an asker has in the router's record. A variable so a test needs no bus.
var routerAskIDs = func(ctx context.Context, inv catalogueReader, asker string) ([]string, error) {
serving := servingBus.Load()
if serving == nil {
return nil, errors.New("only the serving controller reads the router's record")
}
bucket, err := asksRecords(ctx, inv)
if err != nil {
return nil, err
}
if bucket == "" {
return nil, errors.New("no module declares the operator channel's records")
}
api, err := jetstream.New(serving.Conn())
if err != nil {
return nil, err
}
kv, err := api.KeyValue(ctx, bucket)
if err != nil {
return nil, err
}
return askIDsOf(ctx, kv, asker)
}
// askIDsOf are the ids of the asks an asker has in the router's bucket.
func askIDsOf(ctx context.Context, kv jetstream.KeyValue, asker string) ([]string, error) {
lister, err := kv.ListKeysFiltered(ctx, asker+".>")
if err != nil {
return nil, err
}
var ids []string
for key := range lister.Keys() {
ids = append(ids, strings.TrimPrefix(key, asker+"."))
}
// The lister stops its own watch when it has sent the last key, so a stop of ours finds it stopped: nats answers that with
// "invalid subscription", which is no failure of the read (issue 1336). Any other error stays one.
if err := lister.Stop(); err != nil && !errors.Is(err, nats.ErrBadSubscription) {
return nil, err
}
return ids, nil
}
// closedRecord is a router record of an ask that ended: it never changes, so it is read once.
type closedRecord struct {
ask *asks.Ask
w *asks.Warrant
}
var closedRecords sync.Map // asker + "." + id -> closedRecord
// closedRecordOf is the router's record of an ask, kept once it is closed. A nil ask is an ask still open, ended without a
// choice, or not there.
func closedRecordOf(ctx context.Context, inv catalogueReader, asker, id string) (*asks.Ask, *asks.Warrant, error) {
if v, ok := closedRecords.Load(asker + "." + id); ok {
r := v.(closedRecord)
return r.ask, r.w, nil
}
ask, w, err := routerAskOf(ctx, inv, asker, id)
if err != nil {
return nil, nil, err
}
if ask != nil && w != nil {
closedRecords.Store(asker+"."+id, closedRecord{ask, w})
}
return ask, w, nil
}
// deliveryIDOf is the id the gate gives a delivery, and so the key of its head in the heads an Approve binds: the repository as
// owner/name, lower-cased, `@` and the first twelve characters of the head.
func deliveryIDOf(owner, repo, head string) string {
if len(head) > 12 {
head = head[:12]
}
return strings.ToLower(owner+"/"+repo) + "@" + head
}
// coversHead says an ask the operator approved is for this merge: its Approve is bound to the heads of the pull request alone
// (mergeAct over {delivery id: head}, which is how a pull request not in a group is asked), or, for a group's ask, whose other
// members' heads the router's record does not carry in full, the ask as published names this pull request at this head in its
// details ("<repo>#<number> ... at <head>:"), and the digest the warrant answers covers those words.
func coversHead(ask *asks.Ask, owner, repo string, number int, head string) (how string, ok bool) {
option, found := ask.Option(optionApprove)
if !found {
return "", false
}
heads := map[string]string{deliveryIDOf(owner, repo, head): head}
if option.Performs(mergeAct(ask.ID, optionApprove, heads)) == nil {
return "bound to the head of this pull request", true
}
short := head
if len(short) > 12 {
short = short[:12]
}
line := regexp.MustCompile(`(?m)^` + regexp.QuoteMeta(repo) + `#` + strconv.Itoa(number) + ` .* at ` + regexp.QuoteMeta(short) + `:`)
if line.MatchString(ask.Details) {
return "a group's ask naming this pull request at this head", true
}
return "", false
}
// findMergeWarrant finds the warrant of a merge, with no read the controller does not make today: the router's chosen merge
// ask for it. Under the teller (releases 3e on) the id is `merge-<group>-<heads hash>-<tail>`, which names neither the pull
// request nor the head, so the teller's closed asks are read and the one the operator approved whose heads cover the merge's
// second parent is the warrant (coversHead). Under mesh-delivery, which made the earlier merges, the id names the pull request
// and the head and is found by them. Why it holds; or found false with why not, for a person to read; or an error when the
// router could not be read, which is no answer. A variable so a test needs no bus.
var findMergeWarrant = func(ctx context.Context, inv catalogueReader, owner, repo string, number int, head string) (why string, found bool, reason string, err error) {
reason = "the router holds no chosen merge ask of " + strings.Join(askersOfMerges(), " or ") + " that covers the head " + head[:min(12, len(head))]
for _, asker := range askersOfMerges() {
ids, err := routerAskIDs(ctx, inv, asker)
if err != nil {
return "", false, "", err
}
for _, id := range ids {
named := number > 0 && strings.HasPrefix(id, mergeAskPrefix(number, head))
if asker == sdkdelivery.TellerModule && !strings.HasPrefix(id, sdkdelivery.AskKindMerge+"-") {
continue
}
if asker == theDelivery && !named {
continue
}
ask, w, err := closedRecordOf(ctx, inv, asker, id)
if err != nil {
return "", false, "", err
}
if ask == nil || w == nil {
continue
}
warrant, err := caughtUpWarrant(asker, ask, w)
if err != nil {
if named {
reason = fmt.Sprintf("%s's ask %s holds no operator's Approve: %v", asker, id, err)
}
continue
}
how, covers := coversHead(ask, owner, repo, number, head)
if !covers && !named {
continue
}
if !covers {
how = "the ask's id names this pull request and head"
}
return warrant + ", " + how, true, "", nil
}
}
return "", false, reason, nil
}
// caughtTrunk is one trunk the catalogue builds from.
type caughtTrunk struct{ Owner, Repo, Branch string }
func (t caughtTrunk) repository() string { return t.Owner + "/" + t.Repo }
// trunksOfCatalogue are the trunks of the repositories on the mesh's forge the catalogue builds a module from.
func trunksOfCatalogue(forms *sourceForms, entries []inventory.Entry) []caughtTrunk {
var out []caughtTrunk
for _, e := range entries {
if e.Provided || e.Source.Repository == "" {
continue
}
owner, name, onForge := forms.onTheForge(e.Source.Repository, e.Source.Seat)
if !onForge {
continue
}
branch := followedBranch(e.Source.Ref)
if branch == "" {
branch = "main"
}
t := caughtTrunk{owner, name, branch}
if !slices.Contains(out, t) {
out = append(out, t)
}
}
return out
}
// catchUpBeside runs a catch-up beside the hearing of the merge that asked for it, so the bus's acknowledgement of that merge
// waits for no forge. A variable so a test runs it where it stands.
var catchUpBeside = func(run func()) { go run() }
// catchingUp makes one catch-up at a time.
var catchingUp sync.Mutex
// settledMerges are the merges this process found nothing to plan for: heard, and no record kept, because nothing reads them.
// They are not read from the forge again.
var settledMerges sync.Map
// catchUp hears the merges of the catalogue's trunks (only one, when only names it) that the controller holds no record of.
// It says whether a trunk was left for later: the forge, the router or the store did not answer.
func (f following) catchUp(ctx context.Context, only *caughtTrunk) (later bool) {
catchingUp.Lock()
defer catchingUp.Unlock()
inv := f.open.inventory
entries, err := inv.Catalogued(ctx)
if err != nil {
fmt.Printf("catch-up: the catalogue cannot be read, so no merge is caught up now: %v\n", err)
return true
}
kept, first, err := inv.EarliestKept(ctx)
if err != nil {
fmt.Printf("catch-up: what the controller kept cannot be read, so no merge is caught up now: %v\n", err)
return true
}
if kept {
fmt.Printf("catch-up: the controller's record of merges begins %s; a merge made before it, less %s, is history on every trunk\n",
first.UTC().Format(time.RFC3339), mergeGrace)
} else {
fmt.Println("catch-up: the controller holds no merge of any trunk yet, so none can have been missed")
}
trunks := trunksOfCatalogue(newSourceForms(ctx, inv), entries)
if only != nil {
trunks = slices.DeleteFunc(trunks, func(t caughtTrunk) bool {
return !strings.EqualFold(t.repository(), only.repository()) || t.Branch != only.Branch
})
}
for _, t := range trunks {
if f.catchUpTrunk(ctx, t, kept, first) {
later = true
}
}
return later
}
func (f following) catchUpTrunk(ctx context.Context, t caughtTrunk, kept bool, first time.Time) (later bool) {
inv := f.open.inventory
repository := t.repository()
merges, err := readTrunkMerges(ctx, t.Owner, t.Repo, t.Branch)
if err != nil {
// Not nothing to catch up: left for the next start or the next heard merge (ADR 0457 clause 3).
fmt.Printf("catch-up: the forge did not answer for the last merges of %s %s, so they are not caught up now; the next start "+
"or the next merge heard there does it: %v\n", repository, t.Branch, err)
return true
}
// **A merge made before the controller's record of merges begins is history, never a gap** (novox/hq ADR 0463 §2).
read := len(merges)
merges = sinceTheRecord(merges, kept, first)
fmt.Printf("catch-up: %s %s: %d of the last %d pull request merges were made since the record began\n", repository, t.Branch,
len(merges), read)
unwarranted := map[string]bool{}
for _, tm := range merges {
id := fmt.Sprintf("%s_%.8s", repository, tm.Commit)
if _, done := settledMerges.Load(repository + "@" + tm.Commit); done {
continue
}
if _, known, err := inv.MergeOf(ctx, repository, tm.Commit); err != nil {
fmt.Printf("catch-up: the record of %s cannot be read, so %s is not caught up now: %v\n", repository, tm.Commit[:8], err)
return true
} else if known {
continue
}
m, err := readMergeEvent(ctx, t.Owner, t.Repo, t.Branch, tm)
if err != nil {
fmt.Printf("catch-up: the merge %.8s of %s cannot be read from the forge, so it is not caught up now: %v\n", tm.Commit, repository, err)
later = true
continue
}
entries, err := inv.Catalogued(ctx)
if err != nil {
return true
}
read, err := readForPlanning(ctx, inv)
if err != nil {
return true
}
facts, err := factsOfMerge(ctx, inv, m, time.Now().UTC(), entries, read)
if err != nil {
fmt.Printf("catch-up: %s %.8s cannot be judged now: %v\n", repository, tm.Commit, err)
later = true
continue
}
// The very test hearMerge makes: a merge it would plan, or take as a first build, is a gap; any other it would say
// nothing reads, and is no gap for a person to decide.
if !facts.plans() && !facts.addsModuleToBuilds {
settledMerges.Store(repository+"@"+tm.Commit, true)
continue
}
why, found, reason, err := findMergeWarrant(ctx, inv, t.Owner, t.Repo, m.Number, tm.Head)
if err != nil {
fmt.Printf("catch-up: the router cannot be read, so %s %.8s is not caught up now: %v\n", repository, tm.Commit, err)
later = true
continue
}
if !found {
unwarranted[id] = true
f.raiseMergeNotHeard(ctx, repository, t.Branch, tm, m, reason)
continue
}
now := time.Now().UTC()
said := fmt.Sprintf("caught up: merged %s, not heard, planned %s", tm.At.Format(time.RFC3339), now.Format(time.RFC3339))
fmt.Printf("catch-up: %s merged into %s (%.8s) was not heard; %s; on the warrant %s\n", repository, t.Branch, tm.Commit, said, why)
m.Title = strings.TrimSpace(m.Title + " (" + said + ")")
if err := f.hearMerge(ctx, m, now); err != nil {
fmt.Printf("catch-up: %s %.8s could not be heard now: %v\n", repository, tm.Commit, err)
later = true
continue
}
if _, known, err := inv.MergeOf(ctx, repository, tm.Commit); err == nil && !known {
settledMerges.Store(repository+"@"+tm.Commit, true) // heard, and nothing reads it
}
}
// Cleared only when every gap was judged: one left for later may still be open.
if !later {
f.clearMergesHeard(ctx, repository, unwarranted)
}
return later
}
// raiseMergeNotHeard keeps open, and observes again, the condition of a merge with no warrant to plan it. One for each
// repository and commit, however many catch-ups find it.
func (f following) raiseMergeNotHeard(ctx context.Context, repository, branch string, tm trunkMerge, m link.SourceMoved, reason string) {
fmt.Printf("catch-up: %s merged into %s (%.8s) was not heard, and is not planned: %s\n", repository, branch, tm.Commit, reason)
if conditionsFrom == nil {
return
}
o := conditions.Observation{Scope: conditions.ScopeMerge, ID: fmt.Sprintf("%s_%.8s", repository, tm.Commit), Kind: kindMergeNotHeard,
Severity: conditions.Warning, Source: "catch-up", Resolver: conditions.ResolverOperator,
Summary: fmt.Sprintf("the merge %.8s of %s into %s was not heard by the controller, and no operator's Approve of it is on "+
"record to plan it on: a person decides", tm.Commit, repository, branch),
Said: fmt.Sprintf("%s %.8s (pull request %d, head %.8s, merged %s): %s", repository, tm.Commit, m.Number, tm.Head,
tm.At.Format(time.RFC3339), reason),
Headline: fmt.Sprintf("A merge into %s was never delivered", repository),
Explanation: fmt.Sprintf("A merge into %s was made while the controller did not hear it, and no Approve of yours for it is on "+
"record that the controller can plan it on. It stays open while the merge is among the latest %d merges of that repository. "+
"The details say what was looked for.", repository, catchUpWindow),
Needs: "decide whether that merge should be delivered, and say so if it should.",
Resolved: "the merge is planned, or is no longer among the latest merges of its repository"}
if _, err := conditionsFrom.Observe(ctx, o); err != nil {
fmt.Printf("catch-up: the condition of %s %.8s could not be kept: %v\n", repository, tm.Commit, err)
}
}
// clearMergesHeard clears the open conditions of a repository's merges not heard that this catch-up did not find again: the
// gap was planned, or left the window.
func (f following) clearMergesHeard(ctx context.Context, repository string, still map[string]bool) {
if conditionsFrom == nil {
return
}
open, err := conditionsFrom.Open(ctx)
if err != nil {
fmt.Printf("catch-up: the open conditions cannot be read, so none is cleared now: %v\n", err)
return
}
for _, c := range open {
if c.Kind != kindMergeNotHeard || c.Subject.Scope != conditions.ScopeMerge ||
!strings.HasPrefix(c.Subject.ID, repository+"_") || still[c.Subject.ID] {
continue
}
if _, err := conditionsFrom.Clear(ctx, c.Key, "the merge is planned, or left the last merges of its trunk"); err != nil {
fmt.Printf("catch-up: the condition %s could not be cleared: %v\n", c.Key, err)
}
}
}
// catchUpAtStart catches up every trunk once the controller serves, and again every minute while a trunk was left for later.
func catchUpAtStart(ctx context.Context, open *stores) {
select {
case <-ctx.Done():
return
case <-time.After(30 * time.Second):
}
for try := 0; try < 20; try++ {
if !(following{open: open}).catchUp(ctx, nil) {
return
}
select {
case <-ctx.Done():
return
case <-time.After(time.Minute):
}
}
fmt.Println("catch-up: a trunk is still left for later after twenty tries; the next merge heard there does it")
}
@@ -0,0 +1,352 @@
//go:build integration
package main
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// The checks of ADR 0457 (issue 1336): what the controller does about merges it did not hear.
// aTrunk is a forge's trunk as the catch-up reads it: its merges oldest first, the module each moves, and which of them the
// router holds an operator's Approve for.
type aTrunk struct {
merges []trunkMerge
module map[string]string
approved map[string]bool
read []string // the trunks asked of the forge, as owner/repo
events int // merges read in full from the forge
silent bool // the forge does not answer
}
func aTrunkOf(t *testing.T, repo string, modules ...string) *aTrunk {
t.Helper()
tr := &aTrunk{module: map[string]string{}, approved: map[string]bool{}}
for i, m := range modules {
c := fmt.Sprintf("%s-c%02d", repo, i+1)
tr.merges = append(tr.merges, trunkMerge{Commit: c, Head: fmt.Sprintf("%shead%02d", repo, i+1),
Subject: fmt.Sprintf("Merge pull request 'change %d' (#%d) from pr/%s into main", i+1, 100+i, c),
At: t0.Add(time.Duration(i) * time.Minute)})
tr.module[c] = m
tr.approved[c] = true
}
was, wasEvent, wasWarrant := trunkCommitsPage, readMergeEvent, findMergeWarrant
trunkCommitsPage = func(_ context.Context, owner, name, _ string, page int) ([]trunkCommit, error) {
tr.read = append(tr.read, owner+"/"+name)
if tr.silent {
return nil, errors.New("the forge did not answer")
}
if page > 1 || owner+"/"+name != "novox/"+repo {
return nil, nil
}
var newest []trunkCommit
for i := len(tr.merges) - 1; i >= 0; i-- {
m := tr.merges[i]
newest = append(newest, trunkCommit{Commit: m.Commit, SecondParent: m.Head, Subject: m.Subject, At: m.At})
}
return newest, nil
}
readMergeEvent = func(_ context.Context, owner, name, branch string, tm trunkMerge) (link.SourceMoved, error) {
tr.events++
m := catalogueMerge(tm.Commit, tr.module[tm.Commit], tm.At)
m.Owner, m.Repo, m.Base, m.HeadSHA = owner, name, branch, tm.Head
_, m.Number, _ = pullOfSubject(tm.Subject)
return m, nil
}
findMergeWarrant = func(_ context.Context, _ catalogueReader, _, _ string, number int, head string) (string, bool, string, error) {
for _, m := range tr.merges {
if strings.Contains(m.Subject, fmt.Sprintf("(#%d)", number)) && m.Head == head && tr.approved[m.Commit] {
return "the operator chose Approve (ask merge-" + fmt.Sprint(number) + ")", true, "", nil
}
}
return "", false, "the router holds no chosen merge ask for the pull request and head", nil
}
wasBeside := catchUpBeside
catchUpBeside = func(run func()) { run() }
t.Cleanup(func() {
trunkCommitsPage, readMergeEvent, findMergeWarrant, catchUpBeside = was, wasEvent, wasWarrant, wasBeside
settledMerges.Clear()
})
return tr
}
// aKeptMergeBefore is a merge the controller heard before the merges of a test: where its record begins, so what a trunk holds from
// then on can be a gap (novox/hq ADR 0463 §2).
func aKeptMergeBefore(t *testing.T, open *stores) {
t.Helper()
hear(t, open, catalogueMerge("mesh-catalog-seed", "app", t0.Add(-time.Hour)), t0.Add(-time.Hour))
}
func mergeKnown(t *testing.T, open *stores, commit string) bool {
t.Helper()
_, known, err := open.inventory.MergeOf(t.Context(), "novox/mesh-catalog", commit)
if err != nil {
t.Fatal(err)
}
return known
}
// Clause 1: the trunk holds merges 1, 2 and 3 and the controller holds records of 1 and 3 only; at its start it plans 2, said
// as caught up; a second start plans nothing new.
func TestAGapBeforeANewerHeardMergeIsPlannedAtTheControllersStart(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksRecorded(t)
tr := aTrunkOf(t, "mesh-catalog", "app", "notes", "app")
hear(t, open, catalogueMerge("mesh-catalog-c01", "app", t0), t0.Add(time.Second))
hear(t, open, catalogueMerge("mesh-catalog-c03", "app", t0.Add(2*time.Minute)), t0.Add(2*time.Minute+time.Second))
if mergeKnown(t, open, "mesh-catalog-c02") {
t.Fatal("the merge to be caught up is known before the catch-up")
}
if (following{open: open}).catchUp(t.Context(), nil) {
t.Fatal("the catch-up says a trunk was left for later")
}
if !mergeKnown(t, open, "mesh-catalog-c02") {
t.Fatal("the merge the controller did not hear was not planned")
}
kept, _, _ := open.inventory.MergeOf(t.Context(), "novox/mesh-catalog", "mesh-catalog-c02")
if said := announcedOf(kept).Title; !strings.Contains(said, "caught up: merged ") || !strings.Contains(said, "not heard, planned ") {
t.Fatalf("the merge does not say it was caught up: %q", said)
}
cutAt(t, open, t0.Add(10*time.Minute))
ws, _ := walks(t, open)
if len(ws) == 0 {
t.Fatal("no walk answers the caught-up merge")
}
if _, in := ws[0].Modules["notes"]; !in {
t.Fatalf("the walk does not build notes, which the caught-up merge moved: %v", ws[0].Modules)
}
before := tr.events
if (following{open: open}).catchUp(t.Context(), nil) || tr.events != before {
t.Fatalf("a second start read %d merge(s) in full again", tr.events-before)
}
}
// Clause 1: a heard merge of a repository whose trunk holds earlier merges the controller has no record of plans them all, the
// earlier first, and the forge is read for that repository's trunk only.
func TestAHeardMergeCatchesUpTheEarlierOnesOfItsOwnTrunkOnly(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksRecorded(t)
tr := aTrunkOf(t, "mesh-catalog", "app", "notes", "app")
var order []string
inner := readMergeEvent
readMergeEvent = func(ctx context.Context, owner, name, branch string, tm trunkMerge) (link.SourceMoved, error) {
order = append(order, tm.Commit)
return inner(ctx, owner, name, branch, tm)
}
t.Cleanup(func() { readMergeEvent = inner })
tr.read = nil
heard := catalogueMerge("mesh-catalog-c03", "app", t0.Add(2*time.Minute))
if err := (following{open: open}).SourceMoved(t.Context(), heard); err != nil {
t.Fatal(err)
}
for _, c := range []string{"mesh-catalog-c01", "mesh-catalog-c02", "mesh-catalog-c03"} {
if !mergeKnown(t, open, c) {
t.Fatalf("%s is not planned: the heard merge and the earlier ones of its trunk are all wanted", c)
}
}
if len(order) != 2 || order[0] != "mesh-catalog-c01" || order[1] != "mesh-catalog-c02" {
t.Fatalf("the earlier merges were heard in the order %v, not the trunk's", order)
}
if len(tr.read) == 0 {
t.Fatal("the forge was not read for the trunk of the merge heard")
}
for _, r := range tr.read {
if r != "novox/mesh-catalog" {
t.Fatalf("the forge was read for %s, not only for the trunk of the merge heard: %v", r, tr.read)
}
}
}
// Clause 1: a trunk of fifty-one merges whose oldest is unheard plans nothing: the oldest is outside the window.
func TestAMergeOlderThanTheLastFiftyIsNotRead(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksRecorded(t)
modules := make([]string, 51)
for i := range modules {
modules[i] = "app"
}
aTrunkOf(t, "mesh-catalog", modules...)
// Fifty are known; the oldest is not.
for i := 2; i <= 51; i++ {
c := fmt.Sprintf("mesh-catalog-c%02d", i)
hear(t, open, catalogueMerge(c, "app", t0.Add(time.Duration(i)*time.Minute)), t0.Add(time.Duration(i)*time.Minute+time.Second))
}
// Paged as the forge pages: fifty a page, and a trunk of another repository is empty (a short page, or none).
pages := trunkCommitsPage
trunkCommitsPage = func(ctx context.Context, owner, name, branch string, page int) ([]trunkCommit, error) {
all, err := pages(ctx, owner, name, branch, 1)
if err != nil || page < 1 || (page-1)*50 >= len(all) {
return nil, err
}
return all[(page-1)*50 : min(page*50, len(all))], nil
}
t.Cleanup(func() { trunkCommitsPage = pages })
(following{open: open}).catchUp(t.Context(), nil)
if mergeKnown(t, open, "mesh-catalog-c01") {
t.Fatal("the oldest of fifty-one merges was read and planned, outside the window")
}
}
// Clause 2: a gap with no chosen Approve in the router is not planned and raises one condition naming its repository and
// commit; a second start leaves that one condition open, observed twice; planning the gap clears it.
func TestAGapWithNoApproveRaisesOneConditionKeptOpenAndClearedWhenPlanned(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksRecorded(t)
aKeptMergeBefore(t, open)
keeper, _ := withConditionsInMemory(t)
tr := aTrunkOf(t, "mesh-catalog", "notes")
tr.approved["mesh-catalog-c01"] = false
catch := func() { (following{open: open}).catchUp(t.Context(), nil) }
catch()
if mergeKnown(t, open, "mesh-catalog-c01") {
t.Fatal("a merge with no Approve on record was planned")
}
raised := func() []conditions.Condition {
t.Helper()
all, err := keeper.Open(t.Context())
if err != nil {
t.Fatal(err)
}
var out []conditions.Condition
for _, c := range all {
if c.Kind == kindMergeNotHeard {
out = append(out, c)
}
}
return out
}
one := raised()
if len(one) != 1 || !strings.Contains(one[0].Subject.ID, "novox/mesh-catalog") || !strings.Contains(one[0].Subject.ID, "mesh-cata") {
t.Fatalf("the gap raised %+v, not one condition naming its repository and commit", one)
}
catch()
if again := raised(); len(again) != 1 || again[0].Observations < 2 {
t.Fatalf("a second catch-up left %+v: one condition, observed twice, was wanted", again)
}
tr.approved["mesh-catalog-c01"] = true
catch()
if !mergeKnown(t, open, "mesh-catalog-c01") {
t.Fatal("the gap was not planned once its Approve was on record")
}
if left := raised(); len(left) != 0 {
t.Fatalf("planning the gap left its condition open: %+v", left)
}
}
// Clause 3: a forge that does not answer leaves the catch-up undone and says so, and the next start does it.
func TestAForgeThatDoesNotAnswerLeavesTheCatchUpForTheNextStart(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksRecorded(t)
aKeptMergeBefore(t, open)
tr := aTrunkOf(t, "mesh-catalog", "notes")
tr.silent = true
if later := (following{open: open}).catchUp(t.Context(), nil); !later {
t.Fatal("a forge that did not answer counted as nothing to catch up")
}
if mergeKnown(t, open, "mesh-catalog-c01") {
t.Fatal("a merge was planned without the forge's word")
}
tr.silent = false
if later := (following{open: open}).catchUp(t.Context(), nil); later || !mergeKnown(t, open, "mesh-catalog-c01") {
t.Fatalf("the next start did not catch up (left for later: %v)", later)
}
}
// Finding 2 of the review: a gap that adds a module is a gap, judged as hearMerge judges it. With the operator's Approve on record it
// is heard (hearMerge asks the new module in its walk); without, it raises the condition and is not lost in silence.
func TestAGapThatAddsAModuleIsHeardOnItsWarrantAndRaisedWithout(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksRecorded(t)
aKeptMergeBefore(t, open)
keeper, _ := withConditionsInMemory(t)
tr := aTrunkOf(t, "mesh-catalog", "newmodule")
tr.approved["mesh-catalog-c01"] = false
catch := func() { (following{open: open}).catchUp(t.Context(), nil) }
catch()
if mergeKnown(t, open, "mesh-catalog-c01") {
t.Fatal("a merge adding a module was heard with no Approve on record")
}
all, err := keeper.Open(t.Context())
if err != nil {
t.Fatal(err)
}
var raised int
for _, c := range all {
if c.Kind == kindMergeNotHeard {
raised++
}
}
if raised != 1 {
t.Fatalf("a merge adding a module with no Approve raised %d condition(s), not one: it was lost in silence", raised)
}
tr.approved["mesh-catalog-c01"] = true
catch()
if !mergeKnown(t, open, "mesh-catalog-c01") {
t.Fatal("a merge adding a module was not heard once its Approve was on record")
}
}
// A trunk with merges from before the controller began keeping merges, and merges since: only the later ones are gaps.
func TestAMergeFromBeforeTheRecordBeganIsHistoryAndNoCondition(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksRecorded(t)
keeper, _ := withConditionsInMemory(t)
tr := aTrunkOf(t, "mesh-catalog", "notes", "notes", "notes")
tr.approved["mesh-catalog-c01"], tr.approved["mesh-catalog-c02"], tr.approved["mesh-catalog-c03"] = false, false, false
// The record begins at the second merge: the first is two days older.
tr.merges[0].At = t0.Add(-48 * time.Hour)
hear(t, open, catalogueMerge("mesh-catalog-c02", "notes", t0.Add(time.Minute)), t0.Add(time.Minute))
delete(tr.approved, "mesh-catalog-c02")
(following{open: open}).catchUp(t.Context(), nil)
// The first merge (two days before the record) was not even read from the forge: only the third, which is unknown, was.
if tr.events != 1 {
t.Fatalf("%d merge(s) were read from the forge: the one from before the record is history and is not read", tr.events)
}
all, err := keeper.Open(t.Context())
if err != nil {
t.Fatal(err)
}
var raised int
for _, c := range all {
if c.Kind == kindMergeNotHeard {
raised++
}
}
if raised != 1 {
t.Fatalf("%d condition(s) raised: only the merge since the record began (the third) is a gap", raised)
}
}
// novox/hq ADR 0463 §2: the record begins at the earliest merge kept on ANY trunk. A trunk the controller kept no merge of has the
// merges made since as gaps (mesh-delivery #1 on novox), and the merges made before as history.
func TestATrunkWithNoKeptMergeHasItsLaterMergesAsGapsAndItsEarlierOnesAsHistory(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksRecorded(t)
// The record begins on another trunk, at t0 less an hour.
hear(t, open, repoMerge("one", "c-one", t0.Add(-time.Hour)), t0.Add(-time.Hour))
tr := aTrunkOf(t, "mesh-catalog", "notes", "notes")
tr.merges[0].At = t0.Add(-72 * time.Hour) // before the record: history
(following{open: open}).catchUp(t.Context(), nil)
if mergeKnown(t, open, "mesh-catalog-c01") {
t.Fatal("a merge from before the record began was heard")
}
if !mergeKnown(t, open, "mesh-catalog-c02") {
t.Fatal("a merge since the record began, on a trunk with no merge kept, was not caught up")
}
}
+346
View File
@@ -0,0 +1,346 @@
package main
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// The pieces of the catch-up that need no store (novox/hq ADR 0457, issue 1336).
func TestAMergeCommitsSubjectNamesItsPullRequest(t *testing.T) {
title, number, ok := pullOfSubject("Merge pull request 'Fix keycloak's sign-in (issue 1332)' (#378) from fix/1332-sign-in into main")
if !ok || number != 378 || title != "Fix keycloak's sign-in (issue 1332)" {
t.Fatalf("read %q %d %v", title, number, ok)
}
if b := headBranchOfSubject("Merge pull request 'x' (#378) from fix/1332-sign-in into main"); b != "fix/1332-sign-in" {
t.Fatalf("the head branch read as %q", b)
}
if _, _, ok := pullOfSubject("Merge branch 'main' of somewhere"); ok {
t.Fatal("a merge that names no pull request read as one")
}
}
func TestOnlyTheLastFiftyMergesOfATrunkAreRead(t *testing.T) {
pages := map[int][]trunkCommit{}
var all []trunkCommit // newest first: 51 merges, a plain commit between each pair
for i := 51; i >= 1; i-- {
all = append(all, trunkCommit{Commit: fmt.Sprintf("merge-%02d", i), SecondParent: fmt.Sprintf("head-%02d", i),
Subject: fmt.Sprintf("Merge pull request 'change %d' (#%d) from fix/x into main", i, i)},
trunkCommit{Commit: fmt.Sprintf("plain-%02d", i)})
}
for i := 0; i < len(all); i += 50 {
pages[i/50+1] = all[i:min(i+50, len(all))]
}
was := trunkCommitsPage
trunkCommitsPage = func(_ context.Context, _, _, _ string, page int) ([]trunkCommit, error) { return pages[page], nil }
t.Cleanup(func() { trunkCommitsPage = was })
got, err := readTrunkMerges(t.Context(), "novox", "mesh-catalog", "main")
if err != nil {
t.Fatal(err)
}
if len(got) != catchUpWindow || got[0].Commit != "merge-02" || got[len(got)-1].Commit != "merge-51" {
t.Fatalf("read %d merges from %s to %s: the oldest of 51 is outside the window", len(got), got[0].Commit, got[len(got)-1].Commit)
}
if got[0].Head != "head-02" {
t.Fatalf("a merge's head is its second parent, not %q", got[0].Head)
}
}
func TestAForgeThatDoesNotAnswerIsAnErrorNotAnEmptyTrunk(t *testing.T) {
was := trunkCommitsPage
trunkCommitsPage = func(context.Context, string, string, string, int) ([]trunkCommit, error) {
return nil, errors.New("the forge did not answer")
}
t.Cleanup(func() { trunkCommitsPage = was })
if got, err := readTrunkMerges(t.Context(), "novox", "mesh-catalog", "main"); err == nil {
t.Fatalf("a forge that did not answer read as %d merges: nothing to catch up", len(got))
}
}
func TestACaughtUpMergeIsAcceptedOnTheSameWarrantAsAnAnnouncedOne(t *testing.T) {
ok := approvedMerge(t)
for _, asker := range []string{sdkdelivery.TellerModule, theDelivery} {
w := ok.warrant
w.Asker = asker
if _, err := caughtUpWarrant(asker, &ok.ask, &w); err != nil {
t.Errorf("the operator's Approve under %s was refused: %v", asker, err)
}
}
if _, err := caughtUpWarrant("keycloak", &ok.ask, &ok.warrant); err == nil {
t.Error("a warrant of an asker the controller takes none from was accepted")
}
declined := ok.warrant
declined.Option, declined.Label = "decline", "Decline"
if _, err := caughtUpWarrant(theDelivery, &ok.ask, &declined); err == nil {
t.Error("a Decline was accepted as a warrant")
}
unproved := ok.warrant
by := *unproved.By
by.Verified = ""
unproved.By = &by
if _, err := caughtUpWarrant(theDelivery, &ok.ask, &unproved); err == nil {
t.Error("an answer whose channel proves nobody was accepted")
}
if _, err := caughtUpWarrant(theDelivery, nil, nil); err == nil {
t.Error("a router that holds nothing was accepted")
}
}
// aTellersAsk is the ask the delivery gate composes for a pull request, as the router holds it once the operator chose Approve
// (the gate's composeMergeAsk: its id from sdkdelivery.AskingID of the group and the heads, its Approve bound to mergeAct over the
// heads, the pull request and its head's first twelve digits in the details), with the warrant the router wrote.
func aTellersAsk(t *testing.T, group string, members ...sdkdelivery.Head) (asks.Ask, asks.Warrant) {
t.Helper()
key, err := sdkdelivery.MergeAskKey(group, members)
if err != nil {
t.Fatal(err)
}
id := sdkdelivery.AskingID(key, 1)
heads := map[string]string{}
var details []string
for _, m := range members {
owner, repo, _ := strings.Cut(m.Repository, "/")
heads[deliveryIDOf(owner, repo, m.Head)] = m.Head
details = append(details, fmt.Sprintf("%s#%d %q at %s: issues: none named; review: approved; reaches: no delivery plan yet", repo, m.Number, "title", m.Head[:12]))
}
bind := func(option string) string {
d, err := asks.ActDigest(mergeAct(id, option, heads))
if err != nil {
t.Fatal(err)
}
return d
}
at := time.Date(2026, 10, 20, 12, 2, 1, 0, time.UTC)
ask := asks.Ask{ID: id, Headline: "Merge into main", Explanation: "passed its checks.", Who: asks.Operator, Expires: at.Add(24 * time.Hour),
OnExpiry: "nothing is merged; the ask is made again", Details: strings.Join(details, "\n"),
Options: []asks.Option{
{ID: optionApprove, Label: "Approve", Does: "merges", Level: asks.Approve, Binds: bind(optionApprove)},
{ID: "decline", Label: "Decline", Does: "nothing is merged", Level: asks.Approve, Binds: bind("decline")}}}
w := asks.Warrant{Ask: id, Asker: sdkdelivery.TellerModule, Outcome: asks.OutcomeChosen, Option: optionApprove, Label: "Approve",
Level: asks.Approve, By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "1", Verified: "user id"},
At: at.Add(time.Minute), AskDigest: ask.Digest()}
return ask, w
}
// withRouter hands findMergeWarrant these asks as the router's records, by asker.
func withRouter(t *testing.T, records map[string]map[string]struct {
ask asks.Ask
w asks.Warrant
}) {
t.Helper()
wasIDs, wasAsk := routerAskIDs, routerAskOf
closedRecords.Range(func(k, _ any) bool { closedRecords.Delete(k); return true })
routerAskIDs = func(_ context.Context, _ catalogueReader, asker string) ([]string, error) {
var ids []string
for id := range records[asker] {
ids = append(ids, id)
}
return ids, nil
}
routerAskOf = func(_ context.Context, _ catalogueReader, asker, id string) (*asks.Ask, *asks.Warrant, error) {
r, ok := records[asker][id]
if !ok {
return nil, nil, nil
}
return &r.ask, &r.w, nil
}
t.Cleanup(func() {
routerAskIDs, routerAskOf = wasIDs, wasAsk
closedRecords.Range(func(k, _ any) bool { closedRecords.Delete(k); return true })
})
}
type aRouterRecord = struct {
ask asks.Ask
w asks.Warrant
}
// The teller's asks, as the router held them on 2026-10-20: the id is the router's own key, rebuilt here from the group and the
// head the gate composed it for (mesh-controller#405 at 8a330afd, mesh-catalog#381 at 1c702422, mesh-host#124 at 52600ec7). It names
// neither the pull request nor the head, so the warrant is found by the heads the Approve is bound to.
func TestATellersWarrantIsFoundByTheHeadItsApproveIsBoundTo(t *testing.T) {
real := []struct {
group, key, owner, repo string
number int
head string
}{
{"feat/r4b-sdk-users", "merge-feat_r4b_sdk_users-ba59b03220f8-989afe0a3bcf", "novox", "mesh-controller", 405, "8a330afd44ba852873339cf1210e90989dca46e1"},
{"fix/1351-remove-todays-delivery-module", "merge-fix_1351_remove_todays_delivery_module-64a353-d93b97b9697b", "novox", "mesh-catalog", 381, "1c7024224dd77c5b603299378883e0993b2e2fdd"},
{"feat/r4b-sdk-users", "merge-feat_r4b_sdk_users-6a59ba48da14-424e053ebc9c", "novox", "mesh-host", 124, "52600ec7bd61ee271a5cc7f4136bcf0a2a8f00fc"},
}
teller := map[string]aRouterRecord{}
for _, r := range real {
ask, w := aTellersAsk(t, r.group, sdkdelivery.Head{Repository: r.owner + "/" + r.repo, Number: r.number, Head: r.head})
if ask.ID != r.key {
t.Fatalf("the id of %s#%d is %q, and the router held it as %q: the fixture is not the router's", r.repo, r.number, ask.ID, r.key)
}
teller[ask.ID] = aRouterRecord{ask, w}
}
withRouter(t, map[string]map[string]aRouterRecord{sdkdelivery.TellerModule: teller})
for _, r := range real {
why, found, reason, err := findMergeWarrant(t.Context(), nil, r.owner, r.repo, r.number, r.head)
if err != nil || !found || !strings.Contains(why, r.key) || !strings.Contains(why, "bound to the head of this pull request") {
t.Errorf("%s#%d: the warrant was not found: %q %v %q %v", r.repo, r.number, why, found, reason, err)
}
// Another head of the same pull request is another merge: the Approve was not given for it.
if _, found, _, _ := findMergeWarrant(t.Context(), nil, r.owner, r.repo, r.number, "ffffffffffffffffffffffffffffffffffffffff"); found {
t.Errorf("%s#%d: another head found the warrant of this one", r.repo, r.number)
}
}
// One pull request's Approve is none for another repository's merge at the same head digits.
if _, found, _, _ := findMergeWarrant(t.Context(), nil, "novox", "mesh-tools", 405, real[0].head); found {
t.Error("a merge of another repository found mesh-controller#405's warrant")
}
}
// A group's ask binds several heads, and the router's record carries only twelve digits of the other members': the pull request is
// found in the ask as it was published, which the warrant's digest covers.
func TestAGroupsWarrantIsFoundByTheAsksWordsForThePullRequestAndItsHead(t *testing.T) {
a := sdkdelivery.Head{Repository: "novox/mesh-host", Number: 124, Head: "52600ec7bd61ee271a5cc7f4136bcf0a2a8f00fc"}
b := sdkdelivery.Head{Repository: "novox/mesh-controller", Number: 405, Head: "8a330afd44ba852873339cf1210e90989dca46e1"}
ask, w := aTellersAsk(t, "feat/r4b-sdk-users", a, b)
withRouter(t, map[string]map[string]aRouterRecord{sdkdelivery.TellerModule: {ask.ID: {ask, w}}})
for _, h := range []sdkdelivery.Head{a, b} {
owner, repo, _ := strings.Cut(h.Repository, "/")
why, found, reason, err := findMergeWarrant(t.Context(), nil, owner, repo, h.Number, h.Head)
if err != nil || !found || !strings.Contains(why, "a group's ask naming this pull request at this head") {
t.Errorf("%s#%d: not found in the group's ask: %q %v %q %v", repo, h.Number, why, found, reason, err)
}
}
if _, found, _, _ := findMergeWarrant(t.Context(), nil, "novox", "mesh-catalog", 381, a.Head); found {
t.Error("a pull request the group's ask does not name found its warrant")
}
// A Decline is no warrant.
declined := w
declined.Option, declined.Label = "decline", "Decline"
withRouter(t, map[string]map[string]aRouterRecord{sdkdelivery.TellerModule: {ask.ID: {ask, declined}}})
if _, found, _, _ := findMergeWarrant(t.Context(), nil, "novox", "mesh-host", 124, a.Head); found {
t.Error("a Decline found a warrant")
}
}
// Keycloak's merge was asked as mesh-delivery.merge-378-b9104454f9a5-1, before release 3e: the id names the pull request and
// the first twelve digits of its head, and the warrant is found by them under mesh-delivery.
func TestAWarrantAskedUnderMeshDeliveryIsFoundByThePullRequestAndItsHead(t *testing.T) {
ok := approvedMerge(t)
head := "b9104454f9a5e0c4d3b2a1f0e9d8c7b6a5948372"
askID := "merge-378-b9104454f9a5-1"
ask, w := ok.ask, ok.warrant
ask.ID, w.Ask, w.Asker = askID, askID, theDelivery
w.AskDigest = ask.Digest()
withRouter(t, map[string]map[string]aRouterRecord{theDelivery: {
"merge-377-aaaaaaaaaaaa-1": {},
askID: {ask, w},
}})
why, found, reason, err := findMergeWarrant(t.Context(), nil, "novox", "mesh-catalog", 378, head)
if err != nil || !found || !strings.Contains(why, askID) {
t.Fatalf("the warrant was not found: %q %v %q %v", why, found, reason, err)
}
// Another head names another ask, and none is found.
if _, found, _, _ := findMergeWarrant(t.Context(), nil, "novox", "mesh-catalog", 378, "ffffffffffff0000"); found {
t.Fatal("another head found a warrant")
}
// A router that cannot be read is an error, never "no warrant".
routerAskIDs = func(context.Context, catalogueReader, string) ([]string, error) { return nil, errors.New("no answer") }
if _, found, _, err := findMergeWarrant(t.Context(), nil, "novox", "mesh-catalog", 378, head); found || err == nil {
t.Fatalf("an unreadable router read as no warrant: %v %v", found, err)
}
}
// A trunk shorter than a page, or with none, is read without a fault: the pages end when the forge's do.
func TestAShortTrunkIsReadWholeAndAnEmptyOneIsNoMerges(t *testing.T) {
was := trunkCommitsPage
t.Cleanup(func() { trunkCommitsPage = was })
trunkCommitsPage = func(_ context.Context, _, _, _ string, page int) ([]trunkCommit, error) {
if page > 1 {
return nil, nil
}
return []trunkCommit{{Commit: "m2", SecondParent: "h2", Subject: "Merge pull request 'two' (#2) from fix/b into main"},
{Commit: "p"}, {Commit: "m1", SecondParent: "h1", Subject: "Merge pull request 'one' (#1) from fix/a into main"}}, nil
}
got, err := readTrunkMerges(t.Context(), "novox", "x", "main")
if err != nil || len(got) != 2 || got[0].Commit != "m1" || got[1].Commit != "m2" {
t.Fatalf("a short trunk read as %+v %v", got, err)
}
trunkCommitsPage = func(context.Context, string, string, string, int) ([]trunkCommit, error) { return nil, nil }
if got, err := readTrunkMerges(t.Context(), "novox", "x", "main"); err != nil || len(got) != 0 {
t.Fatalf("an empty trunk read as %+v %v", got, err)
}
}
// ADR 0253: what the operator reads of a merge not heard is plain: no hash, no identifier, nothing to do "by hand".
func TestTheConditionOfAMergeNotHeardIsInPlainWords(t *testing.T) {
keeper, _ := withConditionsInMemory(t)
tm := trunkMerge{Commit: "c0880c4d1f2e3a4b5c6d7e8f9a0b1c2d3e4f5a6b", Head: "b9104454f9a5e0c4d3b2a1f0e9d8c7b6a5948372"}
(following{}).raiseMergeNotHeard(t.Context(), "novox/mesh-catalog", "main", tm, link.SourceMoved{Number: 378},
"the router holds no chosen merge ask of mesh-delivery-teller or mesh-delivery that covers the head b9104454f9a5")
open, err := keeper.Open(t.Context())
if err != nil || len(open) != 1 {
t.Fatalf("opened %d conditions, %v", len(open), err)
}
c := open[0]
if why, ok := conditions.PlainWords(conditions.Words{Headline: c.Headline, Explanation: c.Explanation, Needs: c.Needs,
Resolved: c.Resolved}); !ok {
t.Fatalf("the condition's words are not plain: %s\n%q\n%q\n%q", why, c.Headline, c.Explanation, c.Needs)
}
}
// The trunk's history lists the merges made inside a pull request's branch too. On novox (2026-10-20) 15 of the 15 non-pull-request
// "gaps" of mesh-catalog and mesh-controller were such merges; only a pull request's merge into this trunk is a merge to catch up.
func TestAMergeMadeInsideABranchIsNoMergeOfTheTrunk(t *testing.T) {
was := trunkCommitsPage
t.Cleanup(func() { trunkCommitsPage = was })
trunkCommitsPage = func(_ context.Context, _, _, _ string, page int) ([]trunkCommit, error) {
if page > 1 {
return nil, nil
}
return []trunkCommit{
{Commit: "c0880c4d", SecondParent: "b9104454", Subject: "Merge pull request 'Read the grant file' (#378) from fix/1332 into main"},
{Commit: "6394a62a", SecondParent: "x1", Subject: "Merge remote-tracking branch 'origin/main' into fix/1081-withdraw"},
{Commit: "896f1d28", SecondParent: "x2", Subject: "Merge feat/agents-write-git-ssh into feat/agents-write-git-ssh-f (issue 955 F)"},
{Commit: "4d03755c", SecondParent: "x3", Subject: "Bring the branch up to main after #354 (issue 1047)"},
{Commit: "other", SecondParent: "x4", Subject: "Merge pull request 'Elsewhere' (#5) from fix/y into nox-mesh"},
}, nil
}
got, err := readTrunkMerges(t.Context(), "novox", "mesh-catalog", "main")
if err != nil || len(got) != 1 || got[0].Commit != "c0880c4d" {
t.Fatalf("read %+v %v: only keycloak's pull request merge into main is a merge to catch up", got, err)
}
}
// novox/hq ADR 0463 §2: the record of merges begins at the earliest merge the controller holds on any trunk. A merge before it is
// history on every trunk; a trunk with no merge kept of its own still has the merges made since as gaps (mesh-delivery #1 on novox).
func TestOnlyTheMergesSinceTheRecordBeganAreGapsOnEveryTrunk(t *testing.T) {
first := time.Date(2026, 10, 17, 12, 0, 0, 0, time.UTC) // the first merge it heard, on some trunk of the mesh
trunk := func(prefix string) []trunkMerge {
return []trunkMerge{
{Commit: prefix + "-old-1", At: first.Add(-30 * 24 * time.Hour)},
{Commit: prefix + "-old-2", At: first.Add(-2 * time.Hour)},
{Commit: prefix + "-within-grace", At: first.Add(-mergeGrace / 2)},
{Commit: prefix + "-since", At: first.Add(24 * time.Hour)},
}
}
// Two trunks, one of which the controller has kept no merge of: both read by the one starting point.
for _, prefix := range []string{"kept-trunk", "trunk-with-no-kept-merge"} {
var commits []string
for _, m := range sinceTheRecord(trunk(prefix), true, first) {
commits = append(commits, m.Commit)
}
if want := prefix + "-within-grace " + prefix + "-since"; strings.Join(commits, " ") != want {
t.Errorf("%s: the gaps are %v, want %q: earlier ones are history on every trunk, later ones are gaps", prefix, commits, want)
}
}
if none := sinceTheRecord(trunk("x"), false, time.Time{}); len(none) != 0 {
t.Errorf("with no merge kept anywhere there are %d gap(s), but no record to have one in", len(none))
}
}
+91 -95
View File
@@ -13,9 +13,7 @@ import (
)
// The chain on the bus (novox/hq ADR 0408, design 47 milestone 5): builds, rollouts and assignments are keys of the
// bus's key-value store, written by the controller that holds the lease. The tables of the controller's store are
// copied into them once, by the serving controller at its start, and a kind is read from its bucket only once the move
// has recorded it (inventory.ChainReads).
// bus's key-value store, written by the controller that holds the lease, and the controller's store keeps none of them.
// chainHolds is whether this process may write the chain now: the lease's gate, the one every act passes.
func chainHolds(ctx context.Context) error {
@@ -33,97 +31,7 @@ func openTheChain(ctx context.Context) (*chain.Chain, error) {
return chain.OnTheBus(ctx, js.Conn(), chainHolds)
}
// kindChainMoveRefused is the condition of a move that stopped without switching a kind.
const kindChainMoveRefused = "chain-move-refused"
// chainMoveRetry is how long a move that failed for want of the bus waits before it tries again.
const chainMoveRetry = 30 * time.Second
// movingTheChain copies the tables into the buckets and records it, at the serving controller's start: after the buckets
// exist and the lease is held, before anything reads a kind that has not moved. **The first try is made here, before the
// controller serves**, so what it serves is read from one place. A move that fails for want of the bus is tried again
// every half minute while the controller serves from the tables; one that is refused — a count that differs, a value
// over the bus's largest — is raised as an urgent condition, and the tables stay what is read: nothing is half-switched.
func movingTheChain(ctx context.Context, inv *inventory.Inventory) {
say := func(format string, args ...any) { fmt.Printf("chain: "+format+"\n", args...) }
// The first try is made here, before the controller serves; the tries after it, while it serves.
if movedOrStopped(ctx, inv, say) {
return
}
go func() {
for {
select {
case <-ctx.Done():
return
case <-time.After(chainMoveRetry):
}
if movedOrStopped(ctx, inv, say) {
return
}
}
}()
}
// movedOrStopped makes one try at the move and says whether there is nothing more to try: moved, refused, or the controller
// ending. False for a failure of the kind that is tried again (the bus away, a write not granted yet, the lease not held).
func movedOrStopped(ctx context.Context, inv *inventory.Inventory, say func(string, ...any)) bool {
_, err := inv.MoveChain(ctx, say)
var refusal *inventory.Refusal
switch {
case err == nil:
go clearChainMove(ctx)
go askAgainSoon(ctx, inv)
return true
case errors.As(err, &refusal):
say("%v", err)
go raiseChainMove(ctx, refusal)
return true
case ctx.Err() != nil:
return true
}
say("the move to the bus did not finish, and the tables are read until it does: %v", err)
return false
}
// chainMoveObservation is the urgent condition for a refused move.
func chainMoveObservation(r *inventory.Refusal) conditions.Observation {
return conditions.Observation{Scope: conditions.ScopeBus, ID: "chain." + r.Kind, Token: kindChainMoveRefused,
Kind: kindChainMoveRefused, Severity: conditions.Urgent, Resolver: conditions.ResolverOperator,
Summary: "the controller's " + r.Kind + " were not moved to the bus: " + r.Why, Source: "chain move"}
}
// whenKept runs f once the serving controller keeps conditions: the move runs before they are kept.
func whenKept(ctx context.Context, f func(k *conditions.Keeper)) {
for conditionsFrom == nil {
select {
case <-ctx.Done():
return
case <-time.After(time.Second):
}
}
f(conditionsFrom)
}
func raiseChainMove(ctx context.Context, r *inventory.Refusal) {
whenKept(ctx, func(k *conditions.Keeper) {
if _, err := k.Observe(ctx, chainMoveObservation(r)); err != nil {
fmt.Printf("chain: the condition for the refused move could not be raised: %v\n", err)
}
})
}
func clearChainMove(ctx context.Context) {
whenKept(ctx, func(k *conditions.Keeper) {
for _, kind := range []string{inventory.KindBuilds, inventory.KindRollouts, inventory.KindAssignments} {
key := chainMoveObservation(&inventory.Refusal{Kind: kind}).Key()
if _, err := k.Clear(ctx, key, "the "+kind+" are on the bus"); err != nil {
fmt.Printf("chain: the condition for the move of the %s could not be cleared: %v\n", kind, err)
}
}
})
}
// attachTheChain gives the inventory of the serving controller the chain on its connection, then moves what has not moved.
// attachTheChain gives the inventory of the serving controller the chain on its connection.
func attachTheChain(ctx context.Context, inv *inventory.Inventory, server *link.Server) error {
bus, ok := server.Bus().(link.OverNATS)
if !ok {
@@ -134,7 +42,19 @@ func attachTheChain(ctx context.Context, inv *inventory.Inventory, server *link.
return err
}
inv.UseChain(c)
movingTheChain(ctx, inv)
inv.RequireLayout()
// And the builds laid out under their modules, once, before they are read (issue 1121) — **before this returns**, so
// nothing the start goes on to read meets a chain not laid out yet: on 2026-10-18 the layout ran beside the start, a
// read of the start was refused, and the controller exited and was started again every five seconds. The layout of
// keys laid out before skips them, so this takes seconds; one that fails is tried again beside the serving, loudly.
if err := inv.LayBuildsOut(ctx, func(format string, args ...any) { fmt.Printf("chain: "+format+"\n", args...) }); err != nil {
fmt.Printf("chain: the builds are not laid out under their modules yet, and none is read until they are: %v\n", err)
go layBuildsOutSoon(ctx, inv)
}
// And the requests the move of delivery 1 found never handed over, asked again once (askAgain), once it is serving.
go askAgainSoon(ctx, inv)
// And what delivery 1 recorded of what each machine was sent, copied onto the keys of the assignments (CopySentBuilds).
go copySentBuildsSoon(ctx, inv)
return nil
}
@@ -179,3 +99,79 @@ func askAgainSoon(ctx context.Context, inv *inventory.Inventory) {
askAgain(ctx, inv)
}
}
// kindSentBuildsNotCopied is the condition of the one-time copy of node.sent_builds onto the assignments' keys that has not
// succeeded: until it has, a module an assignment pulled in may be missing from what a machine is read to run.
const kindSentBuildsNotCopied = "sent-builds-not-copied"
// copySentBuildsSoon copies what the store recorded of what each machine was sent onto the keys of the assignments, as soon
// as this controller holds the lease (the chain refuses a write before), and tries again every half minute while it fails,
// each time it fails saying so as an urgent condition until it has worked. It is idempotent: CopySentBuilds overwrites no key
// that names a commit and empties each column it has copied.
func copySentBuildsSoon(ctx context.Context, inv *inventory.Inventory) {
failed := false
for {
_, err := inv.CopySentBuilds(ctx)
o := conditions.Observation{Scope: conditions.ScopeBuild, ID: kindSentBuildsNotCopied, Token: kindSentBuildsNotCopied,
Kind: kindSentBuildsNotCopied, Severity: conditions.Urgent, Resolver: conditions.ResolverOperator,
Source: "chain", Summary: "what each machine was sent is not copied onto its assignments: "}
if err != nil {
fmt.Printf("chain: what the machines were sent is not copied onto the assignments: %v\n", err)
if keeper := conditionsFrom; keeper != nil && !failed {
o.Summary += err.Error()
if _, oerr := keeper.Observe(ctx, o); oerr == nil {
failed = true
}
}
} else {
if keeper := conditionsFrom; keeper != nil && failed {
_, _ = keeper.Clear(ctx, o.Key(), "copied")
}
return
}
select {
case <-ctx.Done():
return
case <-time.After(chainMoveRetry):
}
}
}
// chainMoveRetry is how long the copy waits before it tries again.
const chainMoveRetry = 30 * time.Second
// kindBuildsNotLaidOut is the condition of the layout of the builds under their modules that has not succeeded: until it
// has, no build is read, and a send that needs one is refused with the reason.
const kindBuildsNotLaidOut = "builds-not-laid-out"
// layBuildsOutSoon lays the builds out under their modules as soon as this controller holds the lease (the chain refuses a
// write before), and tries again every half minute while it fails, loudly: each failure is printed, and the first raises an
// urgent condition until it has worked. It is idempotent (LayBuildsOut).
func layBuildsOutSoon(ctx context.Context, inv *inventory.Inventory) {
failed := false
for {
err := inv.LayBuildsOut(ctx, func(format string, args ...any) { fmt.Printf("chain: "+format+"\n", args...) })
o := conditions.Observation{Scope: conditions.ScopeBuild, ID: kindBuildsNotLaidOut, Token: kindBuildsNotLaidOut,
Kind: kindBuildsNotLaidOut, Severity: conditions.Urgent, Resolver: conditions.ResolverOperator,
Source: "chain", Summary: "the builds are not laid out under their modules, so none is read: "}
if err != nil {
fmt.Printf("chain: the builds are not laid out under their modules, and none is read until they are: %v\n", err)
if keeper := conditionsFrom; keeper != nil && !failed {
o.Summary += err.Error()
if _, oerr := keeper.Observe(ctx, o); oerr == nil {
failed = true
}
}
} else {
if keeper := conditionsFrom; keeper != nil && failed {
_, _ = keeper.Clear(ctx, o.Key(), "laid out")
}
return
}
select {
case <-ctx.Done():
return
case <-time.After(chainMoveRetry):
}
}
}
+40 -4
View File
@@ -10,7 +10,7 @@ import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/chain"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/lease"
@@ -176,12 +176,12 @@ func TestASendOfWhatAMachineRunsAlreadyChangesNoLinkOfItsAssignment(t *testing.T
if a, _ := assignmentOf(on, "notes"); a.Rollout != "plan-1" || a.Build != "build-1" || a.Commit != "c1" {
t.Fatalf("the assignment names %+v", on)
}
// A declaration sent by hand with no builds known says nothing of them.
// A declaration sent by hand with no builds known makes what the machine runs unknown: a time on the key, no commit.
if err := inv.RecordSent(ctx, node, "d3", nil); err != nil {
t.Fatal(err)
}
if on, _ = inv.AssignmentsOn(ctx, "anchor"); runOf(on, "notes").Commit != "c1" {
t.Fatalf("a send of unknown builds changed %+v", on)
if on, _ = inv.AssignmentsOn(ctx, "anchor"); runOf(on, "notes").Commit != "" || runOf(on, "notes").SentUnknown.IsZero() {
t.Fatalf("a send of unknown builds left %+v", on)
}
// A module sent a commit no build is kept of is named by its commit alone.
if err := inv.RecordSent(ctx, node, "d4", map[string]string{"notes": "c2"}); err != nil {
@@ -207,3 +207,39 @@ func runOf(on []chain.Assignment, module string) chain.Assignment {
a, _ := assignmentOf(on, module)
return a
}
// **An assignment waiting for its build is one key of the assignments bucket** (ADR 0261, ADR 0408, issue 1094): entered
// through the `assign` verb for a module whose build is in flight, one read of `<node>.<module>` says state pending with the
// build it waits for, the build's outcome through the build seat's path makes the key stand with the wait carried as applied,
// and the table of pending assignments is read by nobody. No key is written by hand.
func TestAnAssignmentWaitingForItsBuildIsOneKeyOfTheAssignmentsBucket(t *testing.T) {
open := aCatalogueMesh(t)
ctx := t.Context()
inv := open.inventory
asksWithPaths(t)
mergeAdding(t, open, "modules/sensors", "3da80a4b00aa")
if _, err := assign(ctx, open, "laptop", "sensors"); err != nil {
t.Fatalf("an assignment while its build runs was refused: %v", err)
}
waiting, found, err := inv.AssignmentValue(ctx, "laptop", "sensors")
if err != nil || !found || waiting.State != chain.StatePending || waiting.Stands() || !waiting.Assigned.IsZero() ||
waiting.Pending == nil || waiting.Pending.State != chain.PendingWaiting || waiting.Pending.Build != "b-modules/sensors" ||
!strings.HasPrefix(waiting.Pending.ID, "p-") {
t.Fatalf("one read of the key says %+v (found %v, %v)", waiting, found, err)
}
if got := assignedTo(t, open, "laptop"); slices.Contains(got, "sensors") {
t.Fatalf("a module only waiting is assigned: %v", got)
}
if err := (builds{inv: inv, open: open}).Built(ctx, outcome("b-modules/sensors", "modules/sensors", "3da80a4b00aa", "sensors", "")); err != nil {
t.Fatal(err)
}
made, found, err := inv.AssignmentValue(ctx, "laptop", "sensors")
if err != nil || !found || !made.Stands() || made.State != "" || made.Assigned.IsZero() || made.Pending == nil ||
made.Pending.State != chain.PendingApplied || made.Pending.ID != waiting.Pending.ID {
t.Fatalf("one read of the key after the build says %+v (found %v, %v)", made, found, err)
}
if !slices.Contains(assignedTo(t, open, "laptop"), "sensors") {
t.Fatal("the build registered sensors and it was not assigned")
}
}
@@ -0,0 +1,85 @@
//go:build integration
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/link"
)
// **Acceptance 12 of design 47: no table in the controller's store holds builds, rollouts or assignments** (novox/hq ADR
// 0408, issue 1111). A delivery is driven through the live trigger — a merge heard, its batch cut into a plan, the plan's
// build asked and answered, the assignment made — until a build, a rollout and an assignment exist; they are read from the
// chain, and then the store's own catalog is read: none of the tables of the chain is in it, nor the function that raised the generation from the assignment table; the table that stays, stays.
func TestNoTableOfTheControllersStoreHoldsBuildsRolloutsOrAssignments(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksRecorded(t)
ctx := t.Context()
inv := open.inventory
if _, err := assign(ctx, open, "anchor", "notes"); err != nil {
t.Fatal(err)
}
hear(t, open, catalogueMerge("c-notes", "notes", t0), t0.Add(time.Second))
cutAt(t, open, t0.Add(91*time.Second))
ws, _ := walks(t, open)
if len(ws) != 1 {
t.Fatalf("%d walk(s) after the cut", len(ws))
}
rollout := ws[0].ID
asking, err := inv.PlanByID(ctx, rollout)
if err != nil {
t.Fatal(err)
}
for try := 0; try < 3 && (asking.Modules["notes"] == nil || asking.Modules["notes"].Build == ""); try++ {
advancePlans(ctx, open)
if asking, err = inv.PlanByID(ctx, rollout); err != nil {
t.Fatal(err)
}
}
buildID := asking.Modules["notes"].Build
if buildID == "" || len(*asked) != 1 {
t.Fatalf("the walk asked %v, and notes' build is %q", *asked, buildID)
}
digest := "sha256:" + strings.Repeat("b", 64)
result := outcome(buildID, "modules/notes", "c-notes", "notes", "")
result.Made = []link.MadeArtifact{{Name: "notes", Kind: "image", Reference: "artifact-store://notes/notes@" + digest}}
if err := (builds{inv: inv, open: open}).Built(ctx, result); err != nil {
t.Fatal(err)
}
// What the live path made is in the chain.
if built, found, err := inv.BuildByID(ctx, buildID); err != nil || !found || built.Commit != "c-notes" {
t.Fatalf("the build is not read from the chain: %+v %v %v", built, found, err)
}
if p, err := inv.PlanByID(ctx, rollout); err != nil || p.ID != rollout {
t.Fatalf("the rollout is not read from the chain: %+v %v", p, err)
}
if on, err := inv.AssignmentsOn(ctx, "anchor"); err != nil {
t.Fatal(err)
} else if _, ok := assignmentOf(on, "notes"); !ok {
t.Fatalf("the assignment is not read from the chain: %+v", on)
}
// And the catalog of the store holds none of them.
have := map[string]bool{}
for _, row := range inv.QueryForTest(t, `select table_name::text from information_schema.tables where table_schema = 'public'`) {
have[strings.Trim(row[0], `"`)] = true
}
for _, table := range []string{"build", "build_request", "release_plan", "batched_merge", "assignment", "pending_assignment", "chain_moved"} {
if have[table] {
t.Errorf("the table %s is in the controller's store", table)
}
}
for _, table := range []string{"build_gate", "assignment_generation"} {
if !have[table] {
t.Errorf("the table %s, which stays, is gone", table)
}
}
if rows := inv.QueryForTest(t, `select count(*) from pg_proc where proname = 'raise_assignment_generation'`); rows[0][0] != "0" {
t.Errorf("the function raise_assignment_generation is in the store: %v", rows)
}
}
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"sort"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
@@ -9,7 +9,7 @@ import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+1 -1
View File
@@ -7,8 +7,8 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+1 -1
View File
@@ -11,8 +11,8 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
+44
View File
@@ -0,0 +1,44 @@
package main
import (
"reflect"
"testing"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// 1212/AC1 and AC2: the check request names the pulls bucket of each module claiming the delivery seat, in the order
// of the claimants' names.
func TestACheckNamesThePullsBucketsOfTheSeatsClaimants(t *testing.T) {
for name, c := range map[string]struct {
entries []inventory.Entry
want []string
}{
"only the gate": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate"), namedOnly("mesh-delivery")}, []string{"mesh-delivery-gate_pulls"}},
"only today's": {[]inventory.Entry{claimingTheSeat("mesh-delivery"), namedOnly("gitea")}, []string{"mesh-delivery_pulls"}},
"both, name order": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate"), claimingTheSeat("mesh-delivery")}, []string{"mesh-delivery_pulls", "mesh-delivery-gate_pulls"}},
"none": {[]inventory.Entry{namedOnly("gitea")}, nil},
} {
if got := pullsBucketsOf(c.entries); !reflect.DeepEqual(got, c.want) {
t.Errorf("%s: %v, want %v", name, got, c.want)
}
}
}
// 1212/AC1 and AC2: the check a request carries names those buckets (the wiring, not only the helper).
func TestTheCheckRequestCarriesThePullsBuckets(t *testing.T) {
for name, c := range map[string]struct {
entries []inventory.Entry
want []string
}{
"only the gate": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate")}, []string{"mesh-delivery-gate_pulls"}},
"only today's": {[]inventory.Entry{claimingTheSeat("mesh-delivery")}, []string{"mesh-delivery_pulls"}},
"both, name order": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate"), claimingTheSeat("mesh-delivery")}, []string{"mesh-delivery_pulls", "mesh-delivery-gate_pulls"}},
} {
got := checkOf(link.PullUpdated{Owner: "novox", Repo: "a", Number: 7}, checkScope{}, c.entries, nil, nil, nil, nil)
if !reflect.DeepEqual(got.Pulls, c.want) || got.Owner != "novox" || got.Number != 7 {
t.Errorf("%s: %+v, want pulls %v", name, got, c.want)
}
}
}
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+1 -1
View File
@@ -8,8 +8,8 @@ import (
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/beside"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The check anybody can run is the check registration runs (novox/hq issue 148, ADR 0037): a manifest
+57 -34
View File
@@ -4,6 +4,8 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"os"
"path"
"regexp"
"runtime/debug"
@@ -12,7 +14,8 @@ import (
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -72,7 +75,7 @@ type checkScope struct {
// Mesh says the repository is the mesh's: modules are built from it on some branch, its owner is the
// core's, or the change adds a module to it.
Mesh bool
// Judge is who judges the gate (link.JudgeSelf, link.JudgeValidator, or the running controller).
// Judge is who judges the gate (link.JudgeSelf, or the running controller).
Judge string
// From is a module built from the repository, for how the mesh clones it; nil when none is.
From *inventory.Entry
@@ -98,10 +101,6 @@ func pullScope(p link.PullUpdated, entries []inventory.Entry, read map[string][]
switch e.Manifest.Module {
case "mesh-controller":
s.Judge = link.JudgeSelf
case "mesh-host":
if s.Judge == "" {
s.Judge = link.JudgeValidator
}
}
}
for _, d := range r.Added {
@@ -140,18 +139,26 @@ func sourceOwner(repository string) string {
// PullUpdated decides a pull request's merge check, and asks for it when there is something to run.
func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
_, err := f.pullUpdated(ctx, p, os.Stdout)
return err
}
// pullUpdated is PullUpdated saying what it did to w. It returns the id of the check it asked of the build seat, or
// "" when it said a verdict itself and asked nothing. A command that answers JSON on standard output (`delivery
// check`) gives it standard error, so its output stays the document alone (novox/hq issue 1311).
func (f following) pullUpdated(ctx context.Context, p link.PullUpdated, w io.Writer) (string, error) {
inv := f.open.inventory
entries, err := inv.Catalogued(ctx)
if err != nil {
return err
return "", err
}
read, err := readForPlanning(ctx, inv)
if err != nil {
return err
return "", err
}
edges, err := inv.Dependencies(ctx)
if err != nil {
return err
return "", err
}
scope := pullScope(p, entries, read, edges)
// **A cycle of edges fails the check, naming every module and edge** (novox/hq ADR 0419 clause 6), a lone pull
@@ -160,12 +167,12 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
Head: p.Commit, Number: p.Number, Paths: p.Paths, PathsTruncated: p.PathsTruncated, Removed: p.Removed, BaseSHA: p.BaseSHA}
if failing, cyclic, err := cycleVerdict(ctx, "", []orderMember{alone},
map[string]orderReach{alone.ID: {Moved: scope.Modules}}, edges, entries); err != nil {
return err
return "", err
} else if cyclic {
failing.Commit, failing.Trunk = p.Commit, twelve(p.BaseSHA)
fmt.Printf("%s/%s#%d (%.8s): %s\n", p.Owner, p.Repo, p.Number, p.Commit, failing.Summary)
sayChecked(ctx, failing)
return nil
fmt.Fprintf(w, "%s/%s#%d (%.8s): %s\n", p.Owner, p.Repo, p.Number, p.Commit, failing.Summary)
sayChecked(ctx, w, failing)
return "", nil
}
// **A member of a group whose gate is the composed check's runs its repository's check alone** (novox/hq ADR 0403
// clause 1): mesh-delivery is asked once, and says so on the group's line. Told nothing, the pull request is
@@ -178,7 +185,7 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
u, err := inv.UpgradeOf(ctx, module)
return u, err == nil
})
fmt.Print(planText(plan))
fmt.Fprint(w, planText(plan))
direct := directVerdict(p, plan)
if composed {
scope.Modules, scope.Dependents, scope.New, scope.Manifests = nil, nil, nil, nil
@@ -186,19 +193,19 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
}
switch {
case !scope.gated() && !scope.Mesh:
fmt.Printf("%s/%s#%d (%.8s): %s, and the repository is not the mesh's: said, nothing run\n",
fmt.Fprintf(w, "%s/%s#%d (%.8s): %s, and the repository is not the mesh's: said, nothing run\n",
p.Owner, p.Repo, p.Number, p.Commit, noModuleTouched)
sayChecked(ctx, direct)
return nil
sayChecked(ctx, w, direct)
return "", nil
case !scope.gated() && p.MergeCheckSaid && !p.MergeCheck:
direct.RepoCheck = &link.CheckLayer{Verdict: "warning", Summary: noMergeCheck}
fmt.Printf("%s/%s#%d (%.8s): %s; %s\n", p.Owner, p.Repo, p.Number, p.Commit, noModuleTouched, noMergeCheck)
sayChecked(ctx, direct)
return nil
fmt.Fprintf(w, "%s/%s#%d (%.8s): %s; %s\n", p.Owner, p.Repo, p.Number, p.Commit, noModuleTouched, noMergeCheck)
sayChecked(ctx, w, direct)
return "", nil
}
request, err := checkRequestFor(ctx, f.open, p, scope, entries)
if err != nil {
return err
return "", err
}
request.Check.Plan = &plan
request.Check.Composed = composed
@@ -207,11 +214,11 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
seat := buildSeatHeld(ctx)
ask, err := askOverOn(seat)
if err != nil {
return err
return "", err
}
defer ask.Close()
if err := ask.Ask(ctx, request); err != nil {
return err
return "", err
}
// **A newer request replaces the one still waiting, and a newer key stops the one running** (novox/hq ADR 0406 §3, §4,
// issue 974): the check queue keeps one request a pull request, and a running check watches its key. Nothing is asked
@@ -225,9 +232,9 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
strings.Join(append(append([]string{}, scope.Modules...), prefixedAll("new:", scope.New)...), ", "),
scope.Width, scope.Tiers)
}
fmt.Printf("%s/%s#%d (%.8s): asked %s to check it before it merges — %s — as %s\n", p.Owner, p.Repo, p.Number,
fmt.Fprintf(w, "%s/%s#%d (%.8s): asked %s to check it before it merges — %s — as %s\n", p.Owner, p.Repo, p.Number,
p.Commit, seat, what, request.ID)
return nil
return request.ID, nil
}
// composedWords is what a group member's own verdict says of its gate layer (novox/hq ADR 0403 clause 1).
@@ -270,7 +277,7 @@ func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scop
for _, e := range entries {
shelf[e.Manifest.Module] = e.Manifest
}
world, err := theRestOfTheMesh(ctx, open.inventory, shelf, "")
world, err := restOfTheMesh(ctx, open, shelf, "")
if err != nil {
return link.BuildRequest{}, err
}
@@ -305,13 +312,19 @@ func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scop
Held: heldBy(ctx),
Seats: seatBases(ctx),
Source: sourceOnSeat(source),
Check: &link.CheckRequest{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Base: p.Base, Trunk: p.BaseSHA, Ready: p.Ready,
Paths: p.Paths, Beside: beside, Modules: scope.Modules, Dependents: scope.Dependents, New: scope.New,
Manifests: scope.Manifests, Judge: scope.Judge, Builds: builds, Named: namedForTheCheck(ctx, p, builds),
Running: runningBy(ctx)},
Check: checkOf(p, scope, entries, beside, builds, namedForTheCheck(ctx, p, builds), runningBy(ctx)),
}, nil
}
// checkOf is the check a request carries: the pull request, what the change touches, and the buckets of the pull
// requests' keys of the modules claiming the delivery seat (novox/hq ADR 0437 §3, issue 1212).
func checkOf(p link.PullUpdated, scope checkScope, entries []inventory.Entry, beside map[string]link.CheckedOut,
builds []link.CheckBuild, named []link.CheckedBuild, running map[string]string) *link.CheckRequest {
return &link.CheckRequest{Pulls: pullsBucketsOf(entries), Owner: p.Owner, Repo: p.Repo, Number: p.Number, Base: p.Base,
Trunk: p.BaseSHA, Ready: p.Ready, Paths: p.Paths, Beside: beside, Modules: scope.Modules, Dependents: scope.Dependents,
New: scope.New, Manifests: scope.Manifests, Judge: scope.Judge, Builds: builds, Named: named, Running: running}
}
// besideOf is what a check reads beside its own repository, by the directory it is found under: each core repository
// by the module the mesh builds from it, at besideRefs. running is the commit the mesh runs of a module.
func besideOf(entries []inventory.Entry, running func(module string) string,
@@ -521,7 +534,7 @@ func checked(ctx context.Context, result link.BuildResult) {
fmt.Printf("%s: stopped, its pull request's key moved on; the newer check will judge\n", result.ID)
return
}
sayChecked(ctx, checkedOf(result))
sayChecked(ctx, os.Stdout, checkedOf(result))
}
// checkedOf is what a check's outcome says: each layer, and an error — never a pass — for a check that
@@ -637,7 +650,7 @@ func prefixedAll(prefix string, items []string) []string {
}
// sayChecked says a merge check's verdict on the bus, where the forge's holder hears it.
func sayChecked(ctx context.Context, c link.Checked) {
func sayChecked(ctx context.Context, w io.Writer, c link.Checked) {
repo := "none"
if c.RepoCheck != nil {
repo = strings.ToUpper(c.RepoCheck.Verdict) + " — " + c.RepoCheck.Summary
@@ -646,7 +659,7 @@ func sayChecked(ctx context.Context, c link.Checked) {
if c.Trunk != "" {
judged = " merged into trunk " + c.Trunk
}
fmt.Printf("%s: %s/%s#%d at %.8s%s checked on %s: gate %s — %s; repository %s\n", c.ID, c.Owner, c.Repo, c.Number,
fmt.Fprintf(w, "%s: %s/%s#%d at %.8s%s checked on %s: gate %s — %s; repository %s\n", c.ID, c.Owner, c.Repo, c.Number,
c.Commit, judged, orSomewhere(c.On), strings.ToUpper(c.Verdict), c.Summary, repo)
if checkEvents == nil {
return
@@ -658,6 +671,16 @@ func sayChecked(ctx context.Context, c link.Checked) {
stating, stop := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
defer stop()
if err := checkEvents.PublishSeatEvent(stating, link.MeshControllerSeat, link.KeyChecked, body); err != nil {
fmt.Printf("%s: the verdict could not be said, so the pull request is not told it: %v\n", c.ID, err)
fmt.Fprintf(w, "%s: the verdict could not be said, so the pull request is not told it: %v\n", c.ID, err)
}
}
// pullsBucketsOf are the buckets of the pull requests' keys a check names for the build agent: one per module
// claiming the delivery seat, in the order of their names (novox/hq ADR 0437 §3, issue 1212).
func pullsBucketsOf(entries []inventory.Entry) []string {
var out []string
for _, m := range claimantsOf(entries, catalogue.DeliverySeat) {
out = append(out, broker.BucketName(m, "pulls"))
}
return out
}
+1 -1
View File
@@ -10,8 +10,8 @@ import (
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+2 -2
View File
@@ -53,8 +53,8 @@ func TestThePullRequestIsMappedOntoTheModuleGraph(t *testing.T) {
"", "modules/newmod", "modules/newmod/module.json", true, ""},
{"the controller judges itself", pull("novox", "mesh-controller", "main", []string{"cmd/x.go"}),
"mesh-controller", "", "module.json", true, "self"},
{"the node-engine is judged with its validator", pull("novox", "mesh-host", "main", []string{"validate/v.go"}),
"mesh-host", "", "module.json", true, "validator"},
{"the node-engine is judged by the controller the mesh runs, as any module is (issue 1391)",
pull("novox", "mesh-host", "main", []string{"validate/v.go"}), "mesh-host", "", "module.json", true, ""},
{"the core's owner, no module: the mesh's, its own check alone", pull("novox", "hq", "main", []string{"README.md"}),
"", "", "", true, ""},
{"a branch nothing is built from: the mesh's repository, no module", pull("novox", "photos", "master",
@@ -0,0 +1,59 @@
package main
import (
"go/ast"
"go/parser"
"go/token"
"path/filepath"
"sort"
"strings"
"testing"
)
// 1389: the controller's own composition is gone. No function of the package is named for it any more, and no function
// calls one by that name: a name reappearing is a second copy of what mesh-sdk's go/compose holds (novox/hq ADR 0453).
func TestTheControllersOwnCompositionIsGone(t *testing.T) {
gone := map[string]bool{"planFor": true, "declarationWith": true, "composedAndValidated": true, "generators": true,
"renderingFor": true, "theRestOfTheMesh": true, "whoResolves": true, "grantsFor": true, "namesInTheMesh": true,
"onThePrivateNetwork": true, "passSeatsOn": true, "theMeshPassesSeatsOn": true, "assignmentsOf": true,
"licencesFor": true, "seatsOn": true, "composeBusUsers": true, "adoptionFor": true, "localNetworksOf": true}
files, err := filepath.Glob("*.go")
if err != nil {
t.Fatal(err)
}
fset := token.NewFileSet()
var found []string
for _, f := range files {
if strings.HasSuffix(f, "_test.go") {
continue
}
file, err := parser.ParseFile(fset, f, nil, 0)
if err != nil {
t.Fatal(err)
}
for _, d := range file.Decls {
fn, ok := d.(*ast.FuncDecl)
if !ok {
continue
}
if gone[fn.Name.Name] {
found = append(found, f+": "+fn.Name.Name+" is declared")
}
if fn.Body == nil {
continue
}
ast.Inspect(fn.Body, func(n ast.Node) bool {
if c, ok := n.(*ast.CallExpr); ok {
if id, ok := c.Fun.(*ast.Ident); ok && gone[id.Name] {
found = append(found, f+": "+fn.Name.Name+" calls "+id.Name)
}
}
return true
})
}
}
sort.Strings(found)
for _, one := range found {
t.Errorf("%s: the controller's own composition is back; compose through mesh-sdk (compose_sdk.go)", one)
}
}
@@ -0,0 +1,114 @@
package main
import (
"encoding/base64"
"errors"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"github.com/novox/mesh-controller/internal/broker"
)
// aCoreInput is an Input of one machine that takes a core module whose credential the vault has not boxed: the composition
// stands in a vault box for it, and the machine holds the controller's own seal of it.
func aCoreInput(t *testing.T) (*meshInput, string) {
t.Helper()
raw := make([]byte, 32)
raw[0] = 9
key := base64.StdEncoding.EncodeToString(raw)
client := catalogue.Manifest{Module: "mesh-provisioner-client", Version: "1",
Claims: []catalogue.Claim{{Name: broker.NodeProvisionerSeat, Scope: catalogue.ScopeNode}},
OwnSecrets: catalogue.OwnSecrets{broker.ProvisionerSecret: {Path: "/var/lib/mesh-provisioner-client/broker"}}}
return &meshInput{Input: compose.Input{
Catalogue: map[string]catalogue.Manifest{client.Module: client},
Nodes: []compose.Node{{Name: "laptop", Assigned: []string{client.Module}, SealingKey: key, Stated: true,
Contract: compose.VaultBoxContract}},
OwnSecrets: []compose.OwnSecretRow{{Node: "laptop", Module: client.Module, Name: broker.ProvisionerSecret,
Origin: "made", SealedTo: key, Sealed: "SEALED-BY-THE-CONTROLLER"}},
}}, client.Module
}
// 1385 (the lead's answer 1): where the vault is silent and no box is kept, the declaration carries the controller's own seal
// for the core credential, as ADR 0396 has it, and stands in for nothing. It failed first: the composition's declaration
// carried the stand-in box, and a send was refused.
func TestACoreCredentialTheVaultDidNotBoxIsSealedByTheController(t *testing.T) {
in, module := aCoreInput(t)
d, _, err := compose.ComposeIn(in.Input, "laptop", compose.Foreseeing)
if err != nil {
t.Fatal(err)
}
if len(d.CoreBoxesWanted) != 1 || len(stillForeseen(d)) == 0 {
t.Fatalf("the composition does not stand in for the vault's box: %v", d.CoreBoxesWanted)
}
sealed, _, err := sealedByTheController(in, "laptop", d, []string{module})
if err != nil {
t.Fatal(err)
}
if left := stillForeseen(sealed); len(left) != 0 {
t.Fatalf("the declaration still stands in for %v", left)
}
var file map[string]any
for _, r := range sealed.Resources {
if r["id"] == module+"."+catalogue.NeedID(broker.ProvisionerSecret) {
file = r
}
}
if file == nil || file["sealed"] != "SEALED-BY-THE-CONTROLLER" || file[catalogue.VaultSealedField] != nil {
t.Fatalf("the core credential is carried as %v: the controller's own seal, not the vault's box", file)
}
// And a module the vault did answer for is left as it is.
other, _, err := sealedByTheController(in, "laptop", d, []string{"another-module"})
if err != nil {
t.Fatal(err)
}
if len(stillForeseen(other)) == 0 {
t.Fatal("a module the vault answered for was sealed by the controller")
}
}
// 1383 (the lead's answer 4): the machine that runs the bus is refused where no user has a credential, with busUserList's
// words, and a machine that does not run the bus is not. It failed first: the Input stood a list in and the bus node composed.
func TestTheMachineThatRunsTheBusIsRefusedWhereNoUserHasACredential(t *testing.T) {
bus := catalogue.Manifest{Module: "nats", Version: "1", BusUsers: "/etc/nats/users.conf",
Claims: []catalogue.Claim{{Name: "mesh-broker", Scope: catalogue.ScopeMesh}}}
other := catalogue.Manifest{Module: "files", Version: "1"}
why := errors.New("this machine runs the bus and not one user has a credential, so the composed list would refuse every connection in the mesh")
in := &meshInput{busRefused: why, Input: compose.Input{
Catalogue: map[string]catalogue.Manifest{"nats": bus, "files": other},
Nodes: []compose.Node{{Name: "anchor", Assigned: []string{"nats"}}, {Name: "laptop", Assigned: []string{"files"}}},
}}
if err := refusedBus(in, "anchor"); err == nil || !strings.Contains(err.Error(), "not one user has a credential") {
t.Fatalf("the machine that runs the bus is not refused: %v", err)
}
if err := refusedBus(in, "laptop"); err != nil {
t.Fatalf("a machine that does not run the bus is refused: %v", err)
}
if err := refusedBus(&meshInput{Input: in.Input}, "anchor"); err != nil {
t.Fatalf("a bus with users is refused: %v", err)
}
}
// The controller's seal replaces the stand-in box whatever path the resource carries: the path is the manifest's with the
// machine's words filled in, not the manifest's own. It failed first, with the path compared (the live try: "composed twice and
// still names ... asked of the vault by the next send").
func TestTheControllersSealReplacesTheBoxWhateverPathTheFileCarries(t *testing.T) {
in, module := aCoreInput(t)
d, _, err := compose.ComposeIn(in.Input, "laptop", compose.Foreseeing)
if err != nil {
t.Fatal(err)
}
for _, r := range d.Resources {
if _, box := r[catalogue.VaultSealedField]; box {
r["path"] = "/home/someone/.var/lib/mesh-provisioner-client/broker"
}
}
sealed, _, err := sealedByTheController(in, "laptop", d, []string{module})
if err != nil {
t.Fatal(err)
}
if left := stillForeseen(sealed); len(left) != 0 {
t.Fatalf("the declaration still stands in for %v", left)
}
}
@@ -0,0 +1,33 @@
package main
import (
"context"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
)
// grantsOf is the grants a provider creates for its consumers, the consumers whose identity overflows the provision's bound
// and the credentials bound to another consumer, as mesh-sdk's rendering gives them for the machine (it replaced the
// controller's grantsFor, issue 1389).
func grantsOf(ctx context.Context, open *stores, node string) ([]catalogue.Grant, []catalogue.Overflow, []catalogue.Unbound, error) {
in, err := inputFor(ctx, open)
if err != nil {
return nil, nil, nil, err
}
rendered, err := compose.Render(in.Input, node, compose.Reading)
if err != nil {
return nil, nil, nil, err
}
return rendered.Rendering.Grants, rendered.Withheld, rendered.Unbound, nil
}
// madeAndRendered sends to the machine (so everything a send makes is made) and answers its rendering as a question reads it
// afterwards: what the controller's Allocating rendering was.
func madeAndRendered(ctx context.Context, open *stores, node string) (catalogue.Rendering, error) {
if _, err := composeSend(ctx, open, node); err != nil {
return catalogue.Rendering{}, err
}
forgetInput(ctx)
return renderingOf(ctx, open, node)
}
+436
View File
@@ -0,0 +1,436 @@
package main
import (
"context"
"os"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/overlay"
"github.com/novox/mesh-controller/internal/inventory"
)
// The controller's adapter from its stores to mesh-sdk's composition input (novox/hq ADR 0453 clause 3, design 47 R4b,
// issues 1382 and 1383).
//
// **The adapter reads, and perform writes.** readComposeInput asks the stores once for everything the composition reads
// and returns compose.Input, which the composition never changes; it makes, seals, allocates and records nothing. A
// pair credential, a port, an own secret or a core box that the next send would make is not made here: the composition
// names it foreseen, and compose_perform.go makes it for a send, then reads the stores again and composes again. A
// question (status, facts, probes, the merge gate's reading) never performs.
//
// **The Input is built once per operation.** A verb that composes every node reads the stores once and hands the Input to
// each node's composition of it.
// composeInputBuilt is called once for each Input the adapter builds: the seam a test counts the reads of an operation by
// (issue 1382, criterion 5).
var composeInputBuilt = func() {}
// readComposeInput reads the stores once and returns what the composition needs. ctx carries what a send keeps (the
// recorded modules kept at the build their machine runs, recorded_kept.go): the same contexts every send composes under.
func readComposeInput(ctx context.Context, open *stores) (*meshInput, error) {
in, shape, err := readMeshAndNodes(ctx, open)
if err != nil {
return nil, err
}
// What each node resolves to, with no credential given (the settings are read before it): the modules whose settings are read, and the
// needs whose pair credentials are asked of the stores. Nothing is written by it.
unread, err := readSettings(ctx, open, &in)
if err != nil {
return nil, err
}
plans := map[string]compose.Resolved{}
for _, n := range in.Nodes {
if shape.refused[n.Name] != nil {
continue // refused for what it keeps: nothing of it is read further
}
resolved, err := compose.Resolve(in, n.Name)
switch {
case compose.IsUnresolvable(err):
// Their set does not compose: nothing of theirs is read further, as the controller never read a thing for a
// node that did not resolve.
continue
case err != nil:
return nil, err
}
plans[n.Name] = resolved
}
if err := unreadSettings(unread, plans); err != nil {
return nil, err
}
var busRefused error
wouldMake, err := readCredentialsAndBuilds(ctx, open, &in, plans, shape, &busRefused)
if err != nil {
return nil, err
}
composeInputBuilt()
return &meshInput{Input: in, wouldMake: wouldMake, busRefused: busRefused, refused: shape.refused}, nil
}
// meshInput is the Input an operation composes over, and the pair credentials its reading found none held for: the ones the
// next send makes, which perform makes.
type meshInput struct {
compose.Input
wouldMake []inventory.PairCredential
// busRefused is why the bus's user list cannot be composed (no user has a credential): the machine that runs the bus is
// refused with it, as the controller's own composition refused it, and no other machine is.
busRefused error
// refused is, by node, why that machine alone cannot be composed (nodeRefusal).
refused map[string]error
}
// nodeOf is the node of that name in an Input.
func nodeOf(in compose.Input, name string) (compose.Node, bool) {
for _, n := range in.Nodes {
if n.Name == name {
return n, true
}
}
return compose.Node{}, false
}
// inputShape is what the first half of the reading found that the second half needs.
type inputShape struct {
sent map[string]map[string]string // by node: module to the commit last sent
kept map[string]map[string]string // by node: module to the commit kept
current map[string]inventory.CurrentBuild
// refused is, by node, why a recorded module cannot be kept at the build the machine runs (keepRecorded's refusal, in
// its words): that machine alone is refused with it, as planFor refused it, and nothing else of the mesh is.
refused map[string]error
}
// readMeshAndNodes is the mesh-wide and per-node half of the Input (issue 1382).
func readMeshAndNodes(ctx context.Context, open *stores) (compose.Input, inputShape, error) {
inv := open.inventory
var in compose.Input
shape := inputShape{sent: map[string]map[string]string{}, kept: map[string]map[string]string{}, refused: map[string]error{}}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return in, shape, err
}
in.Catalogue = shelf
if in.Holdings, err = inv.Holdings(ctx); err != nil {
return in, shape, err
}
if in.Licences, err = readLicences(ctx, open); err != nil {
return in, shape, err
}
nodes, err := inv.Nodes(ctx)
if err != nil {
return in, shape, err
}
places, err := inv.Overlays(ctx)
if err != nil {
return in, shape, err
}
placed := map[string]inventory.Overlay{}
for _, p := range places {
placed[p.Name] = p
}
if shape.current, err = inv.CurrentBuilds(ctx); err != nil {
return in, shape, err
}
// The range the mesh allocates from and what its internal names end in, read from the environment here and nowhere else.
in.OverlayCIDR = strings.TrimSpace(os.Getenv(OverlayCIDRVar))
in.InternalSuffix = overlay.Suffix()
if in.Tunnels, err = readTunnels(ctx, inv); err != nil {
return in, shape, err
}
if in.Adopted, err = readAdopted(ctx, inv); err != nil {
return in, shape, err
}
carried, err := inv.CarriedPeers(ctx)
if err != nil {
return in, shape, err
}
for _, c := range carried {
in.CarriedPeers = append(in.CarriedPeers, compose.CarriedPeer{
PublicKey: c.PublicKey, Address: c.Address, EnrolledAs: c.EnrolledAs, Named: c.Named})
}
if in.Joining, err = inv.NodesWithALiveToken(ctx); err != nil {
return in, shape, err
}
keepsSecrets := false
for _, m := range shelf {
keepsSecrets = keepsSecrets || m.Keeps != ""
}
for _, n := range nodes {
var refusal error
node, err := readNode(ctx, open, n, placed[n.Name], &refusal)
if err != nil {
return in, shape, err
}
if refusal != nil {
shape.refused[n.Name] = refusal
}
shape.sent[n.Name], shape.kept[n.Name] = node.SentBuilds, node.KeptBuilds
in.Nodes = append(in.Nodes, node)
}
if keepsSecrets {
if in.OperatorExport, err = inv.OperatorExport(ctx); err != nil {
return in, shape, err
}
}
return in, shape, nil
}
// readNode is one node of the Input: everything the composition reads about it, in the order the controller read it.
func readNode(ctx context.Context, open *stores, n inventory.Node, place inventory.Overlay,
refused *error) (compose.Node, error) {
inv := open.inventory
out := compose.Node{
Name: n.Name,
Place: compose.Place{Key: place.Key, Endpoint: place.Endpoint, Site: place.Site, Hub: place.Hub,
Address: place.Address},
Account: n.Account, AccountHome: n.AccountHome, AgentAccount: n.AgentAccount, AgentAccountHome: n.AgentAccountHome,
Adopted: n.Adopted,
}
var err error
if out.Assigned, err = inv.Assigned(ctx, n.Name); err != nil {
return out, err
}
if out.Capabilities, err = inv.ProfileOf(ctx, n.Name); err != nil {
return out, err
}
if out.PublicDomain, err = inv.PublicDomainOf(ctx, n.Name); err != nil {
return out, err
}
if out.Pins, err = inv.PinsFor(ctx, n.Name); err != nil {
return out, err
}
if out.Bound, err = inv.BindingsFor(ctx, n.Name); err != nil {
return out, err
}
given, err := inv.PortsFor(ctx, n.Name)
if err != nil {
return out, err
}
for _, a := range given {
if out.Ports == nil {
out.Ports = map[string]map[int]int{}
}
if out.Ports[a.Module] == nil {
out.Ports[a.Module] = map[int]int{}
}
out.Ports[a.Module][a.Wanted] = a.Machine
}
if out.SealingKey, err = inv.SealingKeyOf(ctx, n.Name); err != nil {
return out, err
}
if n.Adopted {
if out.Taken, err = inv.Taken(ctx, n.Name); err != nil {
return out, err
}
}
if out.OutwardLinks, err = inv.OutwardLinksOf(ctx, n.Name); err != nil {
return out, err
}
account, had, err := inv.MachineOf(ctx, n.Name)
if err != nil {
return out, err
}
if had && account.Machine != nil {
out.NetworksSaid = true
for _, a := range account.Machine.Networks {
out.Networks = append(out.Networks, catalogue.AttachedNetwork{Link: a.Link, Network: a.Network})
}
}
if out.Sequence, err = inv.Sequence(ctx, n.ID); err != nil {
return out, err
}
if out.Epoch, err = inv.SentEpoch(ctx, n.ID); err != nil {
return out, err
}
if out.Generation, err = inv.SentGeneration(ctx, n.ID); err != nil {
return out, err
}
if out.ReadsModules, err = inv.ReadsModules(ctx, n.ID); err != nil {
return out, err
}
if out.SentBuilds, out.SentKnown, err = inv.SentBuilds(ctx, n.Name); err != nil {
return out, err
}
runs, err := inv.AssignmentRuns(ctx, n.Name)
if err != nil {
return out, err
}
if len(runs) > 0 {
out.Runs = make(map[string]compose.Run, len(runs))
for module, ref := range runs {
out.Runs[module] = compose.Run{Commit: ref.Commit, Build: ref.Build}
}
}
// A recorded module kept at the build this machine runs, in the context a send composes under (recorded_kept.go): its
// manifest and the commit, applied to this node's own resolution only.
// What cannot be kept refuses this machine alone (keepRecorded's refusal, as planFor returned it).
kept, err := recordedKept(ctx, open, n.Name)
if err != nil {
*refused = err
} else if len(kept) > 0 {
out.KeptBuilds = kept
if out.Kept, err = keptManifests(ctx, open, n.Name, kept); err != nil {
*refused = err
out.KeptBuilds, out.Kept = nil, nil
}
}
stated, has, err := inv.HealthOf(ctx, n.Name)
if err != nil {
return out, err
}
out.Stated = has
if has {
out.Contract = stated.Contract
}
return out, nil
}
// keptManifests is the manifest of each module kept at the build a machine runs: keepRecorded's reads, and its refusals in its
// words, without putting any of it into a shelf (that is the node's own Kept, applied by the composition).
func keptManifests(ctx context.Context, open *stores, node string, kept map[string]string) (map[string]catalogue.Manifest, error) {
// applyKept puts each manifest into the shelf it is given and refuses a build that cannot be read; the shelf here is a
// scratch one, whose entries are the kept manifests.
scratch := map[string]catalogue.Manifest{}
if err := applyKept(ctx, open, node, kept, scratch); err != nil {
return nil, err
}
return scratch, nil
}
func readTunnels(ctx context.Context, inv *inventory.Inventory) (map[string]compose.FoundTunnel, error) {
found, err := inv.Tunnels(ctx)
if err != nil {
return nil, err
}
if len(found) == 0 {
return nil, nil
}
out := make(map[string]compose.FoundTunnel, len(found))
for name, t := range found {
out[name] = compose.FoundTunnel{Tunnel: tunnelOf(t.Tunnel), NodeAdopted: t.NodeAdopted}
}
return out, nil
}
func readAdopted(ctx context.Context, inv *inventory.Inventory) (*compose.AdoptedTunnel, error) {
tunnel, hub, adopted, err := inv.AdoptedTunnel(ctx)
if err != nil {
return nil, err
}
if hub == "" {
return nil, nil
}
return &compose.AdoptedTunnel{Tunnel: tunnelOf(tunnel), Hub: hub, Adopted: adopted}, nil
}
func tunnelOf(t inventory.Tunnel) compose.Tunnel {
out := compose.Tunnel{Interface: t.Interface, Unit: t.Unit, Config: t.Config, Port: t.Port, MTU: t.MTU,
Address: t.Address, Range: t.Range, PublicKey: t.PublicKey}
for _, p := range t.Peers {
out.Peers = append(out.Peers, compose.TunnelPeer{PublicKey: p.PublicKey, Address: p.Address})
}
return out
}
// readLicences is every licence the mesh holds, with who uses it and the key sealed to each holder: given even where empty,
// so a holder the mesh holds no key for composes as that, and not as a key that was not asked for.
func readLicences(ctx context.Context, open *stores) ([]compose.Licence, error) {
held, err := open.Licences(ctx)
if err != nil {
return nil, err
}
all, err := held.All(ctx)
if err != nil {
return nil, err
}
var out []compose.Licence
for _, one := range all {
l := compose.Licence{Name: one.Name, Serves: one.Serves, Keys: map[compose.Holder]string{}}
holders, err := held.HoldersOf(ctx, one.Name)
if err != nil {
return nil, err
}
for _, h := range holders {
holder := compose.Holder{Node: h.Node, Module: h.Module}
l.Holders = append(l.Holders, holder)
key, err := held.KeyFor(ctx, one.Name, h.Node, h.Module)
if err != nil {
return nil, err
}
l.Keys[holder] = key
}
node, module, err := held.ManagerOf(ctx, one.Name)
if err != nil {
return nil, err
}
l.Manager = compose.Holder{Node: node, Module: module}
out = append(out, l)
}
return out, nil
}
// readSettings is each node's settings layers for every module that has any, read BEFORE any node is resolved: the first pass
// resolves a node whose set takes a provider's served facts from the node that runs it, with that node's settings on them
// (servedOnNode), so a mesh whose provider says ${setting:domain} resolves only with the settings in the Input. A module's
// settings that fail to read are returned in unread, by node and module, and are an error only for a module a node resolves to
// (unreadSettings): the controller read none for a module nothing runs.
func readSettings(ctx context.Context, open *stores, in *compose.Input) (unread map[string]map[string]error, err error) {
have, err := open.inventory.ModulesWithSettings(ctx)
if err != nil {
return nil, err
}
names := make([]string, 0, len(have))
for module := range have {
if _, onShelf := in.Catalogue[module]; onShelf {
names = append(names, module)
}
}
sort.Strings(names)
for i := range in.Nodes {
n := &in.Nodes[i]
for _, module := range names {
layers, err := open.inventory.SettingsFor(ctx, n.Name, module)
if err != nil {
if unread == nil {
unread = map[string]map[string]error{}
}
if unread[n.Name] == nil {
unread[n.Name] = map[string]error{}
}
unread[n.Name][module] = err
continue
}
if len(layers) == 0 {
continue
}
if n.Settings == nil {
n.Settings = map[string][]catalogue.Layer{}
}
n.Settings[module] = layers
}
}
return unread, nil
}
// unreadSettings is the error of a module's settings that did not read, for a module the node resolves to.
func unreadSettings(unread map[string]map[string]error, plans map[string]compose.Resolved) error {
names := make([]string, 0, len(plans))
for node := range plans {
names = append(names, node)
}
sort.Strings(names)
for _, node := range names {
for _, m := range plans[node].Plan.Modules {
if err := unread[node][m.Module]; err != nil {
return err
}
}
}
return nil
}
@@ -0,0 +1,285 @@
package main
import (
"context"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/overlay"
"github.com/novox/mesh-controller/internal/inventory"
)
// The second half of the adapter (issue 1383): the credential and build fields of the Input, read and never made.
// readCredentialsAndBuilds fills what the first half left: the credentials, the own secrets, the vault's boxes, the bus,
// each node's serving key and certificate, and the builds.
func readCredentialsAndBuilds(ctx context.Context, open *stores, in *compose.Input, plans map[string]compose.Resolved,
shape inputShape, busRefused *error) ([]inventory.PairCredential, error) {
if err := readIdentity(ctx, open, in); err != nil {
return nil, err
}
wouldMake, err := readCredentials(ctx, open, in, plans, busRefused)
if err != nil {
return nil, err
}
return wouldMake, readBuilds(ctx, open, in, plans, shape)
}
// readIdentity is the mesh's authority and each node's serving key and the certificate kept for it, for a mesh in which a
// module asks for a certificate. It establishes and issues nothing: a certificate not kept yet is composed with a stand-in, and
// issued by the send (certificateFor).
func readIdentity(ctx context.Context, open *stores, in *compose.Input) error {
needs := false
for _, m := range in.Catalogue {
needs = needs || m.Certificate != nil
}
if !needs {
return nil
}
ident, err := open.Identity(ctx)
if err != nil {
return err
}
if certificate, known, err := ident.KnownAuthority(ctx); err != nil {
return err
} else if known {
in.Authority = certificate
}
nodes, err := open.inventory.Nodes(ctx)
if err != nil {
return err
}
ids := map[string]string{}
for _, n := range nodes {
ids[n.Name] = n.ID
}
for i := range in.Nodes {
n := &in.Nodes[i]
if n.ServingKey, err = ident.ServingKeyOf(ctx, ids[n.Name]); err != nil {
return err
}
if n.ServingKey == "" {
continue
}
if certificate, known, err := ident.KeptCertificate(ctx, overlay.InternalName(n.Name), n.ServingKey); err != nil {
return err
} else if known {
n.Certificate = certificate
}
}
return nil
}
// readCredentials is how each pair credential a node takes from elsewhere stands, the credentials made for each provider's
// consumers, the own secrets, the vault's boxes and the bus. It reads and never makes: a credential a send would make is
// left out (the composition's would-make), a credential only a person gives and nobody gave is not-given, and one the send
// would be refused is refused with the controller's words.
func readCredentials(ctx context.Context, open *stores, in *compose.Input, plans map[string]compose.Resolved,
busRefused *error) ([]inventory.PairCredential, error) {
var wouldMake []inventory.PairCredential
inv := open.inventory
shelf := in.Catalogue
nodeOf := map[string]compose.Node{}
for _, n := range in.Nodes {
nodeOf[n.Name] = n
}
names := make([]string, 0, len(plans))
for name := range plans {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
resolved := plans[name]
nodeShelf := shelfWith(shelf, nodeOf[name].Kept)
for _, need := range resolved.Plan.Needs {
if need.ByRecord {
continue
}
p := inventory.PairCredential{Name: need.Name, Consumer: name, ConsumerModule: need.For,
Provider: need.From, Local: need.Local}
if module, outside := issuedOutsideBy(nodeShelf, need); outside {
p.Kind, p.ProviderModule = inventory.PairOutside, module
} else if need.SharedOwn != "" {
p.Kind, p.Own = inventory.PairShared, need.SharedOwn
p.ProviderModule = sharedProviderModule(resolved.Plan, shelf, need)
}
state, err := inv.PairCredentialState(ctx, p)
if err != nil {
return nil, err
}
c := compose.Credential{Provision: need.Name, Consumer: name, ConsumerModule: need.For,
Provider: need.From, Local: need.Local}
switch state.State {
case inventory.PairHeld:
c.State, c.Sealed = compose.CredentialHeld, state.Sealed
case inventory.PairNotGiven:
c.State, c.Why = compose.CredentialNotGiven, state.Why
case inventory.PairRefused:
c.State, c.Why = compose.CredentialRefused, state.Why
default:
// Left out: a need with no entry is the credential the next send makes, and perform makes it.
wouldMake = append(wouldMake, p)
continue
}
in.Credentials = append(in.Credentials, c)
}
}
// The credentials made for each provider's consumers, so the grants it must create are read from them.
for _, n := range in.Nodes {
rows, err := inv.SecretsFrom(ctx, n.Name)
if err != nil {
return nil, err
}
for _, s := range rows {
in.Issued = append(in.Issued, compose.Issued{Provision: s.Name, Consumer: s.Consumer,
ConsumerModule: s.ConsumerModule, Local: s.Local, Provider: n.Name, ForProvider: s.ForProvider})
}
}
own, err := inv.OwnSecretRows(ctx)
if err != nil {
return nil, err
}
for _, r := range own {
in.OwnSecrets = append(in.OwnSecrets, compose.OwnSecretRow{Node: r.Node, Module: r.Module, Name: r.Name,
Origin: r.Origin, SealedTo: r.SealedTo, Sealed: r.Sealed})
}
for _, n := range in.Nodes {
boxes, err := inv.KeptCoreBoxes(ctx, n.Name)
if err != nil {
return nil, err
}
for module, byName := range boxes {
for name, b := range byName {
in.KeptCoreBoxes = append(in.KeptCoreBoxes, compose.CoreBox{Node: n.Name, Module: module, Name: name,
Generation: b.Generation, SealedTo: b.SealedTo, Box: b.Box})
}
}
}
sort.Slice(in.KeptCoreBoxes, func(i, j int) bool {
a, b := in.KeptCoreBoxes[i], in.KeptCoreBoxes[j]
if a.Node != b.Node {
return a.Node < b.Node
}
if a.Module != b.Module {
return a.Module < b.Module
}
return a.Name < b.Name
})
// The bus: its user list as the controller composes it, for the machine that runs it, and who the mesh has minted a
// credential for.
var bus []catalogue.Manifest
for _, m := range shelf {
if m.BusUsers != "" && m.ClaimsSeat("mesh-broker") {
bus = append(bus, m)
}
}
if len(bus) > 0 {
list, missing, err := busUserList(ctx, inv, bus)
if err != nil && !strings.Contains(err.Error(), "not one user has a credential") {
return nil, err
}
if err != nil {
// The machine that runs the bus is refused, as the controller's own composition refused it: a list that
// refuses every connection in the mesh is never composed (refusedBus); the other machines compose.
*busRefused = err
}
in.BusUsers, in.BusMissing = list, missing
}
if in.BusMinted, err = inv.BusMinted(ctx); err != nil {
return nil, err
}
if in.BusMemberships, err = inv.BusMemberships(ctx); err != nil {
return nil, err
}
return wouldMake, nil
}
// readBuilds is the build half of the Input: every repository the mesh built, each module's current build, and the build made
// from a commit of a module for the machines whose node-engine reads them.
func readBuilds(ctx context.Context, open *stores, in *compose.Input, plans map[string]compose.Resolved, shape inputShape) error {
inv := open.inventory
held, err := inv.Held(ctx)
if err != nil {
return err
}
for repository := range held {
in.Built = append(in.Built, repository)
}
sort.Strings(in.Built)
in.CurrentBuilds = make(map[string]string, len(shape.current))
for module, b := range shape.current {
in.CurrentBuilds[module] = b.Commit
}
// The build made from each commit a declaration may carry, for the machines that read a build per module.
for _, n := range in.Nodes {
if !n.ReadsModules {
continue
}
resolved, ok := plans[n.Name]
if !ok {
continue
}
for _, m := range resolved.Plan.Modules {
for _, commit := range []string{shape.current[m.Module].Commit, shape.kept[n.Name][m.Module], shape.sent[n.Name][m.Module]} {
if commit == "" {
continue
}
if _, done := in.BuildIDs[m.Module][commit]; done {
continue
}
build, err := inv.BuildOf(ctx, m.Module, commit)
if err != nil {
return err
}
if in.BuildIDs == nil {
in.BuildIDs = map[string]map[string]string{}
}
if in.BuildIDs[m.Module] == nil {
in.BuildIDs[m.Module] = map[string]string{}
}
in.BuildIDs[m.Module][commit] = build
}
}
}
return nil
}
// shelfWith is the shelf with each module a node keeps at the build it runs put in place of the current one: a copy.
func shelfWith(shelf map[string]catalogue.Manifest, kept map[string]catalogue.Manifest) map[string]catalogue.Manifest {
if len(kept) == 0 {
return shelf
}
out := make(map[string]catalogue.Manifest, len(shelf)+len(kept))
for name, m := range shelf {
out[name] = m
}
for name, m := range kept {
out[name] = m
}
return out
}
// sharedProviderModule is which module answers a shared credential on the providing node: the one in the plan, else the first
// the catalogue names (providerModuleOf, in an order that does not move between calls).
func sharedProviderModule(plan catalogue.Resolution, shelf map[string]catalogue.Manifest, n catalogue.Needed) string {
for _, m := range plan.Modules {
if _, shared := m.SharedCredentialOf(n.Name); shared {
return m.Module
}
}
names := make([]string, 0, len(shelf))
for name := range shelf {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
if _, shared := shelf[name].SharedCredentialOf(n.Name); shared {
return name
}
}
return ""
}
@@ -0,0 +1,354 @@
//go:build integration
package main
import (
"encoding/json"
"fmt"
"sort"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"github.com/novox/mesh-controller/internal/inventory"
)
// tableState is every table's rows and every sequence's last value, as the adapter's reads must leave them.
func tableState(t *testing.T, inv *inventory.Inventory) map[string]string {
t.Helper()
out := map[string]string{}
for _, row := range inv.QueryForTest(t, `select table_name from information_schema.tables
where table_schema = 'public' and table_type = 'BASE TABLE' order by table_name`) {
name := strings.Trim(row[0], `"`)
count := inv.QueryForTest(t, fmt.Sprintf(`select count(*)::text from %q`, name))
out["rows:"+name] = fmt.Sprint(count)
digest := inv.QueryForTest(t, fmt.Sprintf(`select coalesce(md5(string_agg(t::text, '|' order by t::text)), '') from %q t`, name))
out["content:"+name] = fmt.Sprint(digest)
}
for _, row := range inv.QueryForTest(t, `select sequencename, coalesce(last_value::text, '') from pg_sequences
where schemaname = 'public' order by sequencename`) {
out["sequence:"+row[0]] = row[1]
}
return out
}
// 1382/AC4 and 1383/AC6: building the Input writes nothing. The store's rows, their contents and its sequences are the same
// before and after, on a mesh whose credentials are not made, whose ports are not given and which keeps a recorded build.
func TestBuildingTheInputWritesNothing(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
inv := open.inventory
// A consumer whose credential nothing has made yet, on a mesh that already composed.
register(t, open, catalogue.Manifest{Module: "reports", Version: "1", Requires: []string{"s3-bucket"},
Resources: []map[string]any{{"id": "config", "type": "file", "path": "/etc/reports/config.json", "mode": "0600",
"content": "{}", "merge": "json"}}})
if _, err := inv.Assign(ctx, "laptop", "reports"); err != nil {
t.Fatal(err)
}
before := tableState(t, inv)
if _, err := readComposeInput(ctx, open); err != nil {
t.Fatal(err)
}
after := tableState(t, inv)
var moved []string
for k, v := range after {
if before[k] != v {
moved = append(moved, k)
}
}
sort.Strings(moved)
if len(moved) > 0 {
t.Fatalf("building the Input wrote: %v", moved)
}
}
// a consumer of the s3-bucket aMeshWithSecrets' provider offers, with a file the credential goes into.
func aConsumerOfTheBucket(name string) catalogue.Manifest {
return catalogue.Manifest{Module: name, Version: "1", Requires: []string{"s3-bucket"},
Secrets: map[string]string{"s3-bucket": "/etc/" + name + "/s3"},
Resources: []map[string]any{{"id": "config", "type": "file", "path": "/etc/" + name + "/config.json", "mode": "0600",
"content": "{}", "merge": "json"}}}
}
// 1383/AC1: of a provider's three consumers, two have their credentials made and one has not: Issued holds the two rows, with
// the copy sealed to the provider, the third is a credential the next send makes, and no row was inserted for it (planFor
// inserted it, plan.go: SecretFor in every mode).
func TestAPairCredentialNoRowHoldsIsAWouldMakeAndNoRowIsInserted(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
inv := open.inventory
register(t, open, aConsumerOfTheBucket("reports"))
if _, err := inv.Assign(ctx, "laptop", "reports"); err != nil {
t.Fatal(err)
}
// A send makes the credentials of files and reports; a question makes none.
if _, err := composeSend(ctx, open, "laptop"); err != nil {
t.Fatal(err)
}
register(t, open, aConsumerOfTheBucket("metrics"))
if _, err := inv.Assign(ctx, "laptop", "metrics"); err != nil {
t.Fatal(err)
}
before := tableState(t, inv)
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
var rows []compose.Issued
for _, r := range in.Issued {
if r.Provider == "anchor" && r.Provision == "s3-bucket" {
rows = append(rows, r)
if r.ForProvider == "" {
t.Errorf("the row of %s carries no copy sealed to the provider", r.ConsumerModule)
}
}
}
if len(rows) != 2 {
t.Fatalf("Issued holds %d row(s) for anchor's s3-bucket, want files and reports: %+v", len(rows), rows)
}
var would []string
for _, p := range in.wouldMake {
would = append(would, p.ConsumerModule)
}
if len(would) != 1 || would[0] != "metrics" {
t.Fatalf("the credentials the next send makes are %v, want metrics", would)
}
for _, c := range in.Credentials {
if c.ConsumerModule == "metrics" {
t.Errorf("a credential nobody holds has an entry: %+v", c)
}
}
after := tableState(t, inv)
for k, v := range after {
if before[k] != v {
t.Errorf("reading the Input moved %s", k)
}
}
}
// 1383/AC2: a credential only a person gives that nobody gave is not given, and so is one given under a key the machine no
// longer has; a credential given by a person for the pair and sealed to a key one end no longer has is refused: in the words
// plan.go and secrets.go use.
func TestACredentialOnlyAPersonGivesIsNotGivenAndAGivenOneUnderAnOldKeyIsSaidSo(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
register(t, open, catalogue.Manifest{Module: "indexers", Version: "1", Provides: []catalogue.Offer{{
Name: "indexer-api", Scope: catalogue.ScopeMesh, Credential: &catalogue.OfferCredential{IssuedBy: catalogue.IssuedOutside}}}})
register(t, open, catalogue.Manifest{Module: "grabber", Version: "1", Requires: []string{"indexer-api"},
Secrets: map[string]string{"indexer-api": "/etc/grabber/key"}})
register(t, open, aConsumerOfTheBucket("ledger"))
register(t, open, catalogue.Manifest{Module: "objects", Version: "1", Provides: catalogue.FromAnywhere("s3-bucket")})
for _, a := range [][2]string{{"anchor", "indexers"}, {"laptop", "grabber"}, {"anchor", "objects"}, {"laptop", "ledger"}} {
if _, err := inv.Assign(ctx, a[0], a[1]); err != nil {
t.Fatalf("assign %s %s: %v", a[0], a[1], err)
}
}
stateOf := func(provision, consumerModule string) (compose.Credential, bool) {
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
for _, c := range in.Credentials {
if c.Provision == provision && c.ConsumerModule == consumerModule {
return c, true
}
}
return compose.Credential{}, false
}
c, found := stateOf("indexer-api", "grabber")
if !found || c.State != compose.CredentialNotGiven || !strings.Contains(c.Why, "nobody gave it") {
t.Fatalf("a credential nobody gave reads as %+v", c)
}
if err := inv.AcceptSecretForPair(ctx, "indexer-api", "laptop", "grabber", "anchor", "", "the-key"); err != nil {
t.Fatal(err)
}
if c, found := stateOf("indexer-api", "grabber"); !found || c.State != compose.CredentialHeld || c.Sealed == "" {
t.Fatalf("a credential a person gave reads as %+v", c)
}
// The consumer's machine generated a key of its own since.
if err := inv.RecordSealingKey(ctx, nodeID(t, open, "laptop"), aPublicKey(t)); err != nil {
t.Fatal(err)
}
if c, found := stateOf("indexer-api", "grabber"); !found || c.State != compose.CredentialNotGiven ||
!strings.Contains(c.Why, "a sealing key one end no longer has") {
t.Fatalf("a credential given under a key the machine no longer has reads as %+v", c)
}
// A credential for a pair, given by a person, under a key one end has since replaced: the mesh cannot make another.
if err := inv.RecordSealingKey(ctx, nodeID(t, open, "laptop"), aPublicKey(t)); err != nil {
t.Fatal(err)
}
if err := inv.AcceptSecretForPair(ctx, "s3-bucket", "laptop", "ledger", "anchor", "", "the-bucket-key"); err != nil {
t.Fatal(err)
}
if err := inv.RecordSealingKey(ctx, nodeID(t, open, "laptop"), aPublicKey(t)); err != nil {
t.Fatal(err)
}
if c, found := stateOf("s3-bucket", "ledger"); !found || c.State != compose.CredentialRefused ||
!strings.Contains(c.Why, "was accepted from a person, and a sealing key at one end has changed since") {
t.Fatalf("a credential given for the pair under an old key reads as %+v", c)
}
}
// 1383/AC3: the own secrets carry their origin, the key they are sealed to and the sealed value; the vault's kept boxes, the
// licences (with their keys given, so an empty key means the mesh holds none), the users the bus has minted and the
// memberships equal what the inventory answers.
func TestTheAdaptersInputCarriesTheOwnSecretsTheBoxesTheLicencesAndTheBus(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
inv := open.inventory
key, err := inv.SealingKeyOf(ctx, "anchor")
if err != nil {
t.Fatal(err)
}
sealed, err := inv.SecretForModule(ctx, "anchor", "objects", "admin")
if err != nil {
t.Fatal(err)
}
box := json.RawMessage(`{"v":1,"eph":"e","iv":"i","tag":"t","ct":"c"}`)
if err := inv.KeepCoreBox(ctx, "anchor", inventory.CoreBox{Module: "objects", Name: "admin", Generation: 4,
SealedTo: key, Box: box}); err != nil {
t.Fatal(err)
}
if err := inv.SeedBusUser(ctx, inventory.BusUser{Username: "anchor.objects", Kind: "module", Node: "anchor",
Module: "objects"}, "a-password"); err != nil {
t.Fatal(err)
}
if err := inv.PutBusMembership(ctx, "laptop", "sealed-membership"); err != nil {
t.Fatal(err)
}
held, err := open.Licences(ctx)
if err != nil {
t.Fatal(err)
}
if err := held.Add(ctx, "claude", "static", map[string]any{"model": "x"}); err != nil {
t.Fatal(err)
}
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
var own *compose.OwnSecretRow
for i := range in.OwnSecrets {
if r := &in.OwnSecrets[i]; r.Node == "anchor" && r.Module == "objects" && r.Name == "admin" {
own = r
}
}
if own == nil || own.Origin != "made" || own.SealedTo != key || own.Sealed != sealed {
t.Errorf("the own secret reads as %+v; the inventory holds %q sealed to %q", own, sealed, key)
}
var kept *compose.CoreBox
for i := range in.KeptCoreBoxes {
if b := &in.KeptCoreBoxes[i]; b.Node == "anchor" && b.Module == "objects" && b.Name == "admin" {
kept = b
}
}
if kept == nil || kept.Generation != 4 || kept.SealedTo != key || string(kept.Box) != string(box) {
t.Errorf("the kept box reads as %+v", kept)
}
if got := strings.Join(in.BusMinted, ","); !strings.Contains(got, "anchor.objects") {
t.Errorf("BusMinted is %v", in.BusMinted)
}
if in.BusMemberships["laptop"] != "sealed-membership" {
t.Errorf("BusMemberships is %v", in.BusMemberships)
}
if len(in.Licences) != 1 || in.Licences[0].Name != "claude" || in.Licences[0].Keys == nil {
t.Errorf("Licences read as %+v", in.Licences)
}
}
// 1383/AC4: the bus's user list is what busUserList gives for the machine that runs the bus, and the Input carries none when
// no module of the shelf runs the bus.
func TestTheBusUsersAreWhatTheControllerComposesForTheMachineThatRunsTheBus(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
if in.BusUsers != "" || len(in.BusMissing) != 0 {
t.Fatalf("a mesh whose shelf runs no bus carries a bus user list: %q %v", in.BusUsers, in.BusMissing)
}
bus := catalogue.Manifest{Module: "nats", Version: "1", BusUsers: "/etc/nats/users.conf",
Claims: []catalogue.Claim{{Name: "mesh-broker", Scope: catalogue.ScopeMesh}}}
register(t, open, bus)
if err := open.inventory.SeedBusUser(ctx, inventory.BusUser{Username: "controller", Kind: "controller"}, "pw"); err != nil {
t.Fatal(err)
}
in, err = readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
list, missing, _ := busUserList(ctx, open.inventory, []catalogue.Manifest{bus})
if in.BusUsers != list || fmt.Sprint(in.BusMissing) != fmt.Sprint(missing) {
t.Errorf("the Input carries %q (%v); busUserList gives %q (%v)", in.BusUsers, in.BusMissing, list, missing)
}
}
// 1383/AC5: Built, CurrentBuilds and BuildIDs equal the inventory's Held, CurrentBuilds and BuildOf, and the serving key and
// the certificate are the machine's.
func TestTheAdaptersInputCarriesTheBuildsAndTheMachinesKey(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
inv := open.inventory
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
held, err := inv.Held(ctx)
if err != nil {
t.Fatal(err)
}
var wantBuilt []string
for repository := range held {
wantBuilt = append(wantBuilt, repository)
}
sort.Strings(wantBuilt)
if fmt.Sprint(in.Built) != fmt.Sprint(wantBuilt) && !(len(in.Built) == 0 && len(wantBuilt) == 0) {
t.Errorf("Built is %v, the inventory's %v", in.Built, wantBuilt)
}
current, err := inv.CurrentBuilds(ctx)
if err != nil {
t.Fatal(err)
}
for module, b := range current {
if in.CurrentBuilds[module] != b.Commit {
t.Errorf("CurrentBuilds[%s] is %q, the inventory's %q", module, in.CurrentBuilds[module], b.Commit)
}
}
for module, byCommit := range in.BuildIDs {
for commit, build := range byCommit {
if want, _ := inv.BuildOf(ctx, module, commit); want != build {
t.Errorf("BuildIDs[%s][%s] is %q, the inventory's %q", module, commit, build, want)
}
}
}
}
// The lead's answer 4: where the mesh holds no credential for any bus user, the machine that runs the bus is refused with
// busUserList's words (the controller's composition refused it), the Input still builds, and another machine composes.
func TestTheBusMachineIsRefusedWhereNoUserHasACredentialAndTheOthersCompose(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
bus := catalogue.Manifest{Module: "nats", Version: "1", BusUsers: "/etc/nats/users.conf",
Claims: []catalogue.Claim{{Name: "mesh-broker", Scope: catalogue.ScopeMesh}}}
register(t, open, bus)
register(t, open, aWebModule())
if _, err := assign(ctx, open, "anchor", "nats"); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", "web"); err != nil {
t.Fatal(err)
}
if _, _, err := composeQuestion(ctx, open, "anchor", compose.Foreseeing, false); err == nil ||
!strings.Contains(err.Error(), "not one user has a credential") {
t.Fatalf("the bus machine is composed: %v", err)
}
if _, _, err := composeQuestion(ctx, open, "laptop", compose.Foreseeing, false); err != nil {
t.Fatalf("a machine that does not run the bus is refused: %v", err)
}
}
@@ -0,0 +1,329 @@
//go:build integration
package main
import (
"reflect"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"github.com/novox/mesh-controller/internal/link"
)
// 1382: the adapter's Input names, for every node, what the inventory answers for it, each compared with the read named
// beside the field in go/compose/input.go (novox/hq ADR 0453 clause 3).
func TestTheAdaptersInputNamesWhatTheInventoryAnswersForEachNode(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
inv := open.inventory
if err := inv.SetPublicDomain(ctx, "anchor", "example.test"); err != nil {
t.Fatal(err)
}
if _, err := inv.PortFor(ctx, "laptop", "files", 8080, false); err != nil {
t.Fatal(err)
}
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(in.Catalogue, shelf) {
t.Error("the Input's Catalogue is not the shelf the inventory holds")
}
nodes, err := inv.Nodes(ctx)
if err != nil {
t.Fatal(err)
}
if len(in.Nodes) != len(nodes) {
t.Fatalf("the Input names %d node(s), the inventory has %d", len(in.Nodes), len(nodes))
}
places, err := inv.Overlays(ctx)
if err != nil {
t.Fatal(err)
}
for _, record := range nodes {
n, ok := nodeOf(in.Input, record.Name)
if !ok {
t.Fatalf("%s is not in the Input", record.Name)
}
for _, p := range places {
if p.Name == record.Name {
if want := (compose.Place{Key: p.Key, Endpoint: p.Endpoint, Site: p.Site, Hub: p.Hub, Address: p.Address}); n.Place != want {
t.Errorf("%s: Place is %+v, the inventory's %+v", record.Name, n.Place, want)
}
}
}
assigned, _ := inv.Assigned(ctx, record.Name)
if !reflect.DeepEqual(n.Assigned, assigned) {
t.Errorf("%s: Assigned is %v, the inventory's %v", record.Name, n.Assigned, assigned)
}
caps, _ := inv.ProfileOf(ctx, record.Name)
if !reflect.DeepEqual(n.Capabilities, caps) {
t.Errorf("%s: Capabilities are %v, the inventory's %v", record.Name, n.Capabilities, caps)
}
domain, _ := inv.PublicDomainOf(ctx, record.Name)
if n.PublicDomain != domain {
t.Errorf("%s: PublicDomain is %q, the inventory's %q", record.Name, n.PublicDomain, domain)
}
if n.Account != record.Account || n.AccountHome != record.AccountHome || n.AgentAccount != record.AgentAccount ||
n.AgentAccountHome != record.AgentAccountHome || n.Adopted != record.Adopted {
t.Errorf("%s: the accounts or the mode differ from the inventory's: %+v", record.Name, n)
}
pins, _ := inv.PinsFor(ctx, record.Name)
if len(pins) != len(n.Pins) || (len(pins) > 0 && !reflect.DeepEqual(n.Pins, pins)) {
t.Errorf("%s: Pins are %v, the inventory's %v", record.Name, n.Pins, pins)
}
bound, _ := inv.BindingsFor(ctx, record.Name)
if len(bound) != len(n.Bound) || (len(bound) > 0 && !reflect.DeepEqual(n.Bound, bound)) {
t.Errorf("%s: Bound is %v, the inventory's %v", record.Name, n.Bound, bound)
}
key, _ := inv.SealingKeyOf(ctx, record.Name)
if n.SealingKey != key {
t.Errorf("%s: SealingKey is %q, the inventory's %q", record.Name, n.SealingKey, key)
}
ports, _ := inv.PortsFor(ctx, record.Name)
want := map[string]map[int]int{}
for _, a := range ports {
if want[a.Module] == nil {
want[a.Module] = map[int]int{}
}
want[a.Module][a.Wanted] = a.Machine
}
if len(want) != len(n.Ports) || (len(want) > 0 && !reflect.DeepEqual(n.Ports, want)) {
t.Errorf("%s: Ports are %v, the inventory's %v", record.Name, n.Ports, want)
}
// The settings of every module it resolves to, layer for layer.
resolved, err := compose.Resolve(in.Input, record.Name)
if err != nil {
continue
}
for _, m := range resolved.Plan.Modules {
layers, err := inv.SettingsFor(ctx, record.Name, m.Module)
if err != nil {
t.Fatal(err)
}
if len(layers) != len(n.Settings[m.Module]) || (len(layers) > 0 && !reflect.DeepEqual(n.Settings[m.Module], layers)) {
t.Errorf("%s: the settings of %s are %v, the inventory's %v", record.Name, m.Module, n.Settings[m.Module], layers)
}
}
}
if domain := nodeDomain(in.Input, "anchor"); domain != "example.test" {
t.Errorf("anchor's PublicDomain is %q", domain)
}
if got := nodePorts(in.Input, "laptop"); got["files"][8080] == 0 {
t.Errorf("laptop's port for files:8080 is not in the Input: %v", got)
}
}
func nodeDomain(in compose.Input, name string) string {
n, _ := nodeOf(in, name)
return n.PublicDomain
}
func nodePorts(in compose.Input, name string) map[string]map[int]int {
n, _ := nodeOf(in, name)
return n.Ports
}
// 1382/AC2: a node keeping a recorded module at the build it runs has that module in Kept and KeptBuilds, and the Input's
// Catalogue is the current shelf, not the kept manifest (R4a review round 1, finding 1).
func TestANodeKeepingARecordedBuildHasItInKeptAndTheCatalogueStaysCurrent(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
start := time.Now().Add(-time.Hour)
image := "registry.invalid:5000/postgres/server@sha256:" + strings.Repeat("a", 64)
for _, b := range []link.BuildResult{
aContainerBuild(t, "postgres", "c1111111", catalogue.PolicyRecord, start, map[string][2]string{"server": {image, ""}}),
} {
if _, _, err := takeIn(ctx, inv, asTheOperator(t, inv, b)); err != nil {
t.Fatal(err)
}
}
if _, err := inv.Assign(ctx, "anchor", "postgres"); err != nil {
t.Fatal(err)
}
if err := inv.RecordSent(ctx, nodeID(t, open, "anchor"), "d-anchor", map[string]string{"postgres": "c1111111"}); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, inv, aContainerBuild(t, "postgres", "c2222222", catalogue.PolicyRecord, start.Add(time.Minute),
map[string][2]string{"server": {image, "runtime"}})); err != nil {
t.Fatal(err)
}
// A send that is not a person's push keeps the build the machine runs.
in, err := readComposeInput(keepingRecorded(ctx), open)
if err != nil {
t.Fatal(err)
}
anchor, _ := nodeOf(in.Input, "anchor")
if anchor.KeptBuilds["postgres"] != "c1111111" {
t.Fatalf("anchor keeps %v, want postgres at c1111111", anchor.KeptBuilds)
}
kept, found := anchor.Kept["postgres"]
if !found {
t.Fatalf("the kept manifest of postgres is not in anchor's Kept: %v", anchor.Kept)
}
current, err := inv.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(in.Catalogue["postgres"], current["postgres"]) {
t.Error("the Input's Catalogue is not the current shelf")
}
if reflect.DeepEqual(kept, current["postgres"]) {
t.Error("the kept manifest is the current one: nothing is kept")
}
if laptop, _ := nodeOf(in.Input, "laptop"); len(laptop.Kept) != 0 || len(laptop.KeptBuilds) != 0 {
t.Errorf("laptop keeps what only anchor runs: %v", laptop.KeptBuilds)
}
// A person's push keeps nothing.
pushed, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
if a, _ := nodeOf(pushed.Input, "anchor"); len(a.Kept) != 0 || len(a.KeptBuilds) != 0 {
t.Errorf("a person's push keeps %v", a.KeptBuilds)
}
}
// 1382/AC3: the seats on record, the tunnels, the adopted tunnel, its carried peers and the nodes joining equal the
// inventory's answers; the range and the suffix are read from the environment once, here.
func TestTheAdaptersInputCarriesTheNetworkAsTheInventoryHoldsIt(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
inv := open.inventory
t.Setenv(OverlayCIDRVar, "10.99.0.0/16")
t.Setenv("MESH_INTERNAL_SUFFIX", "mesh.test")
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
holdings, err := inv.Holdings(ctx)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(in.Holdings, holdings) {
t.Errorf("Holdings are %v, the inventory's %v", in.Holdings, holdings)
}
tunnels, err := inv.Tunnels(ctx)
if err != nil {
t.Fatal(err)
}
if len(tunnels) != len(in.Tunnels) {
t.Errorf("Tunnels are %v, the inventory's %v", in.Tunnels, tunnels)
}
if _, hub, _, err := inv.AdoptedTunnel(ctx); err != nil {
t.Fatal(err)
} else if (hub == "") != (in.Adopted == nil) {
t.Errorf("Adopted is %v, the inventory's hub is %q", in.Adopted, hub)
}
carried, err := inv.CarriedPeers(ctx)
if err != nil {
t.Fatal(err)
}
if len(carried) != len(in.CarriedPeers) {
t.Errorf("CarriedPeers are %v, the inventory's %v", in.CarriedPeers, carried)
}
joining, err := inv.NodesWithALiveToken(ctx)
if err != nil {
t.Fatal(err)
}
if len(joining) != len(in.Joining) {
t.Errorf("Joining is %v, the inventory's %v", in.Joining, joining)
}
if in.OverlayCIDR != "10.99.0.0/16" || in.InternalSuffix != "mesh.test" {
t.Errorf("the range and the suffix are %q and %q", in.OverlayCIDR, in.InternalSuffix)
}
// Read once: what the environment says afterwards moves nothing already read.
t.Setenv(OverlayCIDRVar, "10.1.0.0/16")
if in.OverlayCIDR != "10.99.0.0/16" {
t.Error("the range moved with the environment")
}
}
// The lead's equality finding (build 1792487777757012513): a provider on one machine serves a fact that says ${setting:domain},
// the setting is set for it, and a consumer on another machine takes that fact. The first pass of the adapter resolved every
// node with no settings in the Input, so the served fact could not be settled and the whole Input failed. The settings are in
// the Input before any node is resolved, and the Input builds, with the same plan for the consumer as the controller's own
// composition. It failed first, with the finding's words.
func TestAServedFactThatSaysASettingIsResolvedWithTheSettingInTheInput(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, catalogue.Manifest{Module: "step-ca", Version: "1",
Provides: []catalogue.Offer{{Name: "acme-ca", Scope: catalogue.ScopeMesh}},
Serves: map[string]map[string]any{"acme-ca": {"domain": "${setting:domain}"}}})
register(t, open, catalogue.Manifest{Module: "route-proxy", Version: "1", Requires: []string{"acme-ca"}})
// The setting is there before the machines are assigned: an assignment resolves the mesh, and a served fact that says
// ${setting:domain} resolves only with it.
if err := open.inventory.KeepSettings(ctx, "", "step-ca", map[string]any{"domain": "example.test"}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "step-ca"); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", "route-proxy"); err != nil {
t.Fatal(err)
}
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatalf("the Input cannot be built: %v", err)
}
if anchor, _ := nodeOf(in.Input, "anchor"); len(anchor.Settings["step-ca"]) == 0 {
t.Fatalf("the Input does not carry the setting of the provider: %v", anchor.Settings)
}
oldPlan, _, err := resolveNode(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
newPlan, _, err := resolveNode(withInputCache(ctx), open, "laptop")
if err != nil {
t.Fatalf("the consumer does not resolve: %v", err)
}
if !reflect.DeepEqual(oldPlan.Needs, newPlan.Needs) {
t.Errorf("the consumer's needs are %+v, the controller's own %+v", newPlan.Needs, oldPlan.Needs)
}
t.Logf("a served fact that says a setting resolves with the setting in the Input: the Input builds and the consumer's needs equal planFor's")
}
// A setting that nothing sets, in a fact a provider serves, is not the adapter's to hide: the controller's own composition
// refuses it as well (theRestOfTheMesh returns servedOnNode's error), and the adapter refuses with the same words rather than
// building an Input that composes a placeholder.
func TestAServedFactThatSaysAnUnsetSettingIsRefusedAsTheControllerRefusesIt(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, catalogue.Manifest{Module: "step-ca", Version: "1",
Provides: []catalogue.Offer{{Name: "acme-ca", Scope: catalogue.ScopeMesh}},
Serves: map[string]map[string]any{"acme-ca": {"domain": "${setting:domain}"}}})
register(t, open, catalogue.Manifest{Module: "route-proxy", Version: "1", Requires: []string{"acme-ca"}})
// Assigned with the setting there, then the setting is cleared: the mesh the controller holds when someone clears it.
if err := open.inventory.KeepSettings(ctx, "", "step-ca", map[string]any{"domain": "example.test"}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "step-ca"); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", "route-proxy"); err != nil {
t.Fatal(err)
}
if err := open.inventory.ClearSettings(ctx, "", "step-ca"); err != nil {
t.Fatal(err)
}
_, _, oldErr := resolveNode(ctx, open, "laptop")
_, newErr := readComposeInput(ctx, open)
if (oldErr == nil) != (newErr == nil) {
t.Fatalf("the controller's composition says %v, the adapter says %v", oldErr, newErr)
}
if oldErr != nil && !strings.Contains(newErr.Error(), "nothing sets \"domain\"") {
t.Errorf("the adapter's words are %q, the controller's %q", newErr, oldErr)
}
t.Logf("an unset setting in a served fact is refused alike: the controller says %v, the adapter says %v", oldErr != nil, newErr != nil)
}
@@ -0,0 +1,54 @@
//go:build integration
package main
import (
"context"
"fmt"
"sort"
)
// made is what a send made in a store, by key and never by value: the machine ports given, the pair credentials and the
// own secrets.
type made struct {
Ports []string
Pairs []string
OwnSecrets []string
}
// madeIn is what a store holds that a send makes, by key and never by value.
func madeIn(ctx context.Context, open *stores) (made, error) {
var out made
inv := open.inventory
nodes, err := inv.Nodes(ctx)
if err != nil {
return out, err
}
for _, n := range nodes {
ports, err := inv.PortsFor(ctx, n.Name)
if err != nil {
return out, err
}
for _, p := range ports {
out.Ports = append(out.Ports, fmt.Sprintf("%s %s:%d->%d", n.Name, p.Module, p.Wanted, p.Machine))
}
rows, err := inv.SecretsFrom(ctx, n.Name)
if err != nil {
return out, err
}
for _, s := range rows {
out.Pairs = append(out.Pairs, fmt.Sprintf("%s %s on %s from %s local=%s", s.Name, s.ConsumerModule, s.Consumer, n.Name, s.Local))
}
}
own, err := inv.OwnSecretRows(ctx)
if err != nil {
return out, err
}
for _, r := range own {
out.OwnSecrets = append(out.OwnSecrets, fmt.Sprintf("%s %s/%s (%s)", r.Node, r.Module, r.Name, r.Origin))
}
sort.Strings(out.Ports)
sort.Strings(out.Pairs)
sort.Strings(out.OwnSecrets)
return out, nil
}
@@ -0,0 +1,44 @@
//go:build integration
package main
import (
"testing"
"git.novox.be/novox/mesh-sdk/go/compose"
)
// 1382/AC5: composing every node of one operation builds the Input once; without an operation's cache, once for each.
func TestComposingEveryNodeOfAnOperationBuildsTheInputOnce(t *testing.T) {
open, _ := aMeshWithSecrets(t)
built := 0
was := composeInputBuilt
composeInputBuilt = func() { built++ }
t.Cleanup(func() { composeInputBuilt = was })
ctx := withInputCache(t.Context())
nodes, err := open.inventory.Nodes(ctx)
if err != nil {
t.Fatal(err)
}
for _, n := range nodes {
if _, _, err := resolveNode(ctx, open, n.Name); err != nil && !unresolvable(err) {
t.Fatal(err)
}
if _, _, err := composeQuestion(ctx, open, n.Name, compose.Reading, false); err != nil && !unresolvable(err) {
t.Fatal(err)
}
}
if _, err := generatorsOf(ctx, open); err != nil {
t.Fatal(err)
}
if built != 1 {
t.Fatalf("an operation over %d node(s) built the Input %d time(s)", len(nodes), built)
}
built = 0
if _, _, err := resolveNode(t.Context(), open, nodes[0].Name); err != nil && !unresolvable(err) {
t.Fatal(err)
}
if built != 1 {
t.Fatalf("a composition with no cache built the Input %d time(s)", built)
}
}
+259
View File
@@ -0,0 +1,259 @@
package main
import (
"context"
"errors"
"fmt"
"io"
"os"
"slices"
"sort"
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"github.com/novox/mesh-controller/internal/inventory"
)
// perform makes what a composition named foreseen (novox/hq ADR 0453 clause 3, design 47 R4b, issue 1385).
//
// **The composition makes nothing; a send does.** mesh-sdk's composition over an Input names what the next send would make
// (the ports it would assign, the credentials it would seal, the core boxes it would ask the vault for, the seats it would
// pass on, the certificate it would issue) and stands in for each. A send composes, performs, reads the stores again and
// composes again (send in compose_sdk.go), and refuses to send a declaration that still names anything foreseen. A question
// (status, facts, probes, the merge gate's reading) never reaches this file.
//
// What is made is exactly what the controller's own composition made on the way to a declaration it was going to send:
// - every pair credential no row holds, for every machine that resolves, since planning one node resolved them all;
// - the seats the mesh passes on, with the hand act (not on a throwaway mesh, whose hand act has nowhere to go);
// - the machine ports the node's modules were given none for;
// - the node's own secrets, by the controller's own rules (a bus credential nobody issued was refused by the composition);
// - the core credentials, asked of the vault every send, so a rotated one reaches the machine;
// - the certificate of a machine that was never certified.
//
// perform returns whether it made or recorded anything: the reason the caller reads the stores again.
func perform(ctx context.Context, open *stores, in *meshInput, node string, d compose.Declared) (bool, error) {
did := false
inv := open.inventory
// The seats passed on: recorded in the store with the controller's own hand act (ADR 0449 §4, issue 1317).
if len(d.Passes.Passes) > 0 {
var w io.Writer = os.Stderr
if lw, ok := ctx.Value(leftOutKey{}).(io.Writer); ok {
w = lw
}
for _, line := range d.Passes.Said {
fmt.Fprintln(w, line)
}
if performSeatPasses(ctx, seatPassingOn(inv, w), d.Passes.Passes) {
did = true
}
}
// Every pair credential the stores hold none for: made once and kept, and sealed to both ends before it was ever written
// down. The error is the one planFor met: the machine that needs it, and why none could be made.
for _, p := range in.wouldMake {
if err := makePair(ctx, inv, p); err != nil {
return did, fmt.Errorf("%s on %s needs %s from %s and no credential could be made for it: %w",
p.ConsumerModule, p.Consumer, p.Name, p.Provider, err)
}
did = true
}
// The ports this node's modules were given none for.
if len(d.ForeseenPorts) > 0 {
shelf, err := inv.Catalogue(ctx)
if err != nil {
return did, err
}
for _, one := range d.ForeseenPorts {
module, port, ok := strings.Cut(one, ":")
wanted, perr := strconv.Atoi(port)
if !ok || perr != nil {
return did, fmt.Errorf("the composition named a port the controller cannot read: %q", one)
}
fixed := false
for _, l := range nodeShelf(in, shelf, node)[module].Listens {
if l.Port == wanted {
fixed = l.Fixed
}
}
if _, err := inv.PortFor(ctx, node, module, wanted, fixed); err != nil {
return did, fmt.Errorf("%s needs %d reachable on %s and it could not be assigned: %w", module, wanted, node, err)
}
did = true
}
}
// The node's own secrets: a module's superuser password, an administrator, an account. The composition refused what the
// send refuses (a bus credential nobody issued, a given value under an old key).
for _, one := range d.StoodIn {
module, name, ok := strings.Cut(one, "/")
if !ok || strings.Contains(one, " ") {
continue // a grant's sealed copy, the certificate or the bus's list: made below or by nobody
}
if _, err := inv.SecretForModule(ctx, node, module, name); err != nil {
return did, err
}
did = true
}
// The certificate of a machine that was never certified: issued and kept, with the authority when there is none yet.
for _, one := range d.StoodIn {
if strings.HasPrefix(one, "the certificate and the authority of ") {
if _, _, err := certificateFor(ctx, open, node); err != nil {
return did, err
}
did = true
break
}
}
return did, nil
}
// makePair makes one pair credential by the controller's own read of it: the credential for the pair, or the provider's one
// credential sealed to this consumer too. A credential only a person gives is never made (it is not a would-make).
func makePair(ctx context.Context, inv *inventory.Inventory, p inventory.PairCredential) error {
var err error
switch p.Kind {
case inventory.PairShared:
_, err = inv.SharedSecretFor(ctx, p.Name, p.Consumer, p.ConsumerModule, p.Provider, p.ProviderModule, p.Local, p.Own)
case inventory.PairPlain:
_, err = inv.SecretFor(ctx, p.Name, p.Consumer, p.ConsumerModule, p.Provider, p.Local)
default:
return errors.New("a credential only a person gives is not made")
}
if inventory.IsNotGiven(err) {
return nil
}
return err
}
// stillForeseen says what a composed declaration still stands in for: the lists the composition names, and the stand-in itself
// anywhere in the body (a pair credential's stand-in is not in any list). A declaration that has any is never sent.
func stillForeseen(d compose.Declared) []string {
var left []string
left = append(left, d.Foreseen...)
left = append(left, d.StoodIn...)
for _, p := range d.ForeseenPorts {
left = append(left, "the port "+p)
}
for _, c := range d.CoreBoxesWanted {
left = append(left, "the vault's box for "+c.Module+"/"+c.Name)
}
for _, p := range d.Passes.Passes {
left = append(left, "the pass of "+p.Seat.Name+" to "+p.To.Module+" on "+p.To.Node)
}
if body, err := d.Body(); err == nil {
for _, marker := range []string{catalogue.ForeseenSealed, "asked of the vault by the next send", compose.ForeseenCertificate,
compose.ForeseenBusUsers} {
if strings.Contains(string(body), marker) {
left = append(left, "a stand-in: "+marker)
}
}
}
sort.Strings(left)
return slicesCompact(left)
}
func slicesCompact(in []string) []string {
var out []string
for _, s := range in {
if len(out) == 0 || out[len(out)-1] != s {
out = append(out, s)
}
}
return out
}
// nodeShelf is the shelf as one node composes its own set: the current one with each module it keeps at the build it runs put
// in place of the current.
func nodeShelf(in *meshInput, shelf map[string]catalogue.Manifest, node string) map[string]catalogue.Manifest {
n, _ := nodeOf(in.Input, node)
return shelfWith(shelf, n.Kept)
}
// prepareSend is what a send does before it composes, every send and whatever the composition would have named: the
// controller's own composition did it on the way, so a module's file that names an own secret composes (mesh-sdk's
// composition can stand in for none: a file naming a secret not made yet is refused in every mode), and the core's credentials
// are asked of the vault on every send, so a rotated one reaches the machine and a silent vault is said (core_box.go).
//
// - Each own secret of a module the node runs (not one handed to the provisioner, not one issued outside the mesh, not one a
// person gives) that no row holds is made, unless the composition would refuse it (a bus credential nobody issued): that
// refusal is the composition's to say, in its words.
// - The core's credentials are asked of the vault, the boxes newly answered are kept, and the modules the vault did not answer
// for with no box kept are returned: the controller seals those itself until the vault holds them (ADR 0396).
func prepareSend(ctx context.Context, open *stores, node string) ([]string, error) {
in, err := inputFor(ctx, open)
if err != nil {
return nil, err
}
if in.refused[node] != nil {
return nil, nil // empty-on-error: refused alone for what it keeps; the composition that follows says so, in its words
}
resolved, err := compose.Resolve(in.Input, node)
if err != nil {
return nil, nil // empty-on-error: it does not compose for its own reasons; the composition that follows says them
}
n, _ := nodeOf(in.Input, node)
hands := n.Stated && n.Contract >= compose.HandedContract
handing := catalogue.Rendering{Hands: hands, Provisioner: provisionerAccount(resolved.Plan)}
inv := open.inventory
did := false
if n.SealingKey != "" {
held := map[string]bool{}
for _, r := range in.OwnSecrets {
// Held only when sealed to the key the machine has now, or given by a person (which is not the mesh's
// to make again): a made row under an old key is re-made, as SecretForModule did.
if r.Node == node && (r.SealedTo == n.SealingKey || r.Origin == inventory.OriginAccepted) {
held[r.Module+"/"+r.Name] = true
}
}
for _, m := range resolved.Plan.Modules {
if handing.Handing(m) {
continue
}
plain := m.OwnSecrets.Plain()
names := make([]string, 0, len(plain))
for name := range plain {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
if plain[name].IssuedBy == catalogue.IssuedOutside || held[m.Module+"/"+name] {
continue
}
switch name {
case "broker":
if !slices.Contains(in.BusMinted, node+"."+m.Module) {
continue
}
case compose.ProvisionerSecret:
if !slices.Contains(in.BusMinted, "provisioner."+node) {
continue
}
}
if _, err := inv.SecretForModule(ctx, node, m.Module, name); err == nil {
did = true
}
}
}
}
var byController []string
if n.Stated && n.Contract >= compose.VaultBoxContract {
boxes, err := coreBoxesFor(ctx, inv, node, resolved.Plan.Modules, func(m catalogue.Manifest) bool {
return handing.Handing(m)
}, true)
if err != nil {
return nil, err
}
did = did || len(boxes.keep) > 0
byController = boxes.byController
}
if did {
// What was made is in the stores now: the composition reads them afresh.
forgetInput(ctx)
}
return byController, nil
}
@@ -0,0 +1,235 @@
//go:build integration
package main
import (
"errors"
"fmt"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"github.com/novox/mesh-controller/internal/link"
)
func aWebModule() catalogue.Manifest {
return catalogue.Manifest{Module: "web", Version: "1",
Listens: []catalogue.Listening{{Port: 8080, From: catalogue.FromEverywhere}},
OwnSecrets: catalogue.OwnSecrets{"admin": {Path: "/etc/web/admin"}},
Resources: []map[string]any{
{"id": "server", "type": "container", "name": "web", "ports": []any{"8080"},
"image": "registry.example/web@sha256:" + strings.Repeat("a", 64)},
}}
}
// 1385/AC1: a port the composition names foreseen is allocated with the inventory's PortFor for that module and port, a second
// composition names it no more, and the number is the one the controller's own Allocating composition gives in the same store.
func TestPerformAllocatesAForeseenPortAndTheNumberIsTheControllers(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aWebModule())
if _, err := assign(ctx, open, "laptop", "web"); err != nil {
t.Fatal(err)
}
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
d, _, err := compose.ComposeIn(in.Input, "laptop", compose.Foreseeing)
if err != nil {
t.Fatal(err)
}
if len(d.ForeseenPorts) != 1 || d.ForeseenPorts[0] != "web:8080" {
t.Fatalf("the composition names the ports %v foreseen, want web:8080", d.ForeseenPorts)
}
did, err := perform(ctx, open, in, "laptop", d)
if err != nil || !did {
t.Fatalf("perform did %v: %v", did, err)
}
held, err := open.inventory.PortsFor(ctx, "laptop")
if err != nil || len(held) != 1 || held[0].Module != "web" || held[0].Wanted != 8080 {
t.Fatalf("the port perform allocated reads %+v: %v", held, err)
}
in, err = readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
if again, _, err := compose.ComposeIn(in.Input, "laptop", compose.Foreseeing); err != nil || len(again.ForeseenPorts) != 0 {
t.Fatalf("a second composition names %v foreseen: %v", again.ForeseenPorts, err)
}
// The number is the one the controller allocates in the same store.
plan, settings, err := resolveNode(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
old, err := declarationFor(ctx, open, "laptop", plan, settings)
if err != nil {
t.Fatal(err)
}
after, _, err := compose.ComposeIn(in.Input, "laptop", compose.Foreseeing)
if err != nil {
t.Fatal(err)
}
if got, want := portsOf(after.Resources), portsOf(old.Resources); got != want {
t.Fatalf("the declaration publishes %s; the controller's allocation publishes %s", got, want)
}
}
func portsOf(resources []map[string]any) string {
var out []string
for _, r := range resources {
if r["type"] == "container" {
out = append(out, fmt.Sprint(r["name"], r["ports"]))
}
}
return strings.Join(out, ";")
}
// 1385/AC2: an own secret named foreseen is made with SecretForModule, and a pair credential the Input names would-make is
// made as planFor chose; both exist after the send's composition and a second composition names neither.
func TestPerformMakesTheOwnSecretsAndThePairCredentialsTheCompositionNamed(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
inv := open.inventory
register(t, open, aWebModule())
register(t, open, aConsumerOfTheBucket("reports"))
for _, m := range []string{"web", "reports"} {
if _, err := assign(ctx, open, "laptop", m); err != nil {
t.Fatal(err)
}
}
before, err := madeIn(ctx, open)
if err != nil {
t.Fatal(err)
}
declared, err := composeSend(ctx, open, "laptop")
if err != nil {
t.Fatalf("the send is refused: %v", err)
}
if len(declared.foreseen) != 0 {
t.Fatalf("the declaration still names %v foreseen", declared.foreseen)
}
after, err := madeIn(ctx, open)
if err != nil {
t.Fatal(err)
}
madeNow := func(have, was []string) []string {
seen := map[string]bool{}
for _, s := range was {
seen[s] = true
}
var out []string
for _, s := range have {
if !seen[s] {
out = append(out, s)
}
}
return out
}
if got := madeNow(after.OwnSecrets, before.OwnSecrets); len(got) != 1 || !strings.Contains(got[0], "web/admin") {
t.Errorf("the own secrets made are %v, want laptop web/admin", got)
}
if got := madeNow(after.Pairs, before.Pairs); len(got) != 1 || !strings.Contains(got[0], "reports") {
t.Errorf("the pair credentials made are %v, want the one of reports", got)
}
// And the credential is the one a second composition reads as held.
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
if len(in.wouldMake) != 0 {
t.Errorf("after the send the next send would still make %v", in.wouldMake)
}
if _, err := inv.SecretsFrom(ctx, "anchor"); err != nil {
t.Fatal(err)
}
}
// 1385/AC3 (the lead's answer 1): with the vault unreachable and no box of the credential kept, the send carries the
// controller's own seal for it (ADR 0396), says so as a condition, and keeps no box.
func TestAVaultThatIsSilentSealsTheSendByTheControllerWhereNoBoxIsKept(t *testing.T) {
open, vault := aCoreMesh(t, link.VaultBoxContract)
vault.silent = errors.New("no responders")
declared, err := composeSend(t.Context(), open, "laptop")
if err != nil {
t.Fatalf("the send is refused: %v", err)
}
f := clientsBroker(declared.Resources)
if f == nil || f["sealed"] == nil || f[catalogue.VaultSealedField] != nil {
t.Fatalf("the core credential is composed as %v: the controller's own seal, until the vault holds it", f)
}
if len(declared.foreseen) != 0 {
t.Fatalf("the declaration still names %v foreseen", declared.foreseen)
}
if c, found := vaultSilence(t, "laptop"); !found || !strings.Contains(c.Summary, "sealed by the controller until the vault holds it") {
t.Fatalf("no condition says why: %+v", c)
}
kept, err := open.inventory.KeptCoreBoxes(t.Context(), "laptop")
if err != nil {
t.Fatal(err)
}
if len(kept["mesh-provisioner-client"]) != 0 {
t.Fatalf("a box was kept for a credential the vault never boxed: %v", kept)
}
}
// 1385/AC6: perform is idempotent: a second send's composition over the stores it left makes and records nothing.
func TestAFinishedSendMakesNothingTheSecondTime(t *testing.T) {
open, _ := aMeshWithSecrets(t)
ctx := t.Context()
register(t, open, aWebModule())
if _, err := assign(ctx, open, "laptop", "web"); err != nil {
t.Fatal(err)
}
if _, err := composeSend(ctx, open, "laptop"); err != nil {
t.Fatal(err)
}
before := tableState(t, open.inventory)
if _, err := composeSend(ctx, open, "laptop"); err != nil {
t.Fatal(err)
}
for k, v := range tableState(t, open.inventory) {
if before[k] != v {
t.Errorf("a second send of the same machine moved %s", k)
}
}
}
// Review round 1, finding 1: a machine that rejoined with a new sealing key keeps its made rows sealed to the old key. A
// send re-makes them for the new key, as SecretForModule did on main, and the file naming the secret is not refused.
func TestASendReMakesAnOwnSecretSealedToAKeyTheMachineNoLongerHas(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
register(t, open, aBot("bot", false))
if _, err := assign(ctx, open, "laptop", "bot"); err != nil {
t.Fatal(err)
}
if _, err := composeSend(ctx, open, "laptop"); err != nil {
t.Fatalf("the first send is refused: %v", err)
}
if err := inv.RecordSealingKey(ctx, nodeID(t, open, "laptop"), aPublicKey(t)); err != nil {
t.Fatal(err)
}
if _, err := composeSend(ctx, open, "laptop"); err != nil {
t.Fatalf("a machine that rejoined cannot be sent to: %v", err)
}
in, err := readComposeInput(ctx, open)
if err != nil {
t.Fatal(err)
}
n, _ := nodeOf(in.Input, "laptop")
found := false
for _, r := range in.OwnSecrets {
if r.Node == "laptop" && r.Module == "bot" && r.Name == "token" {
found = true
if r.SealedTo != n.SealingKey {
t.Fatalf("the own secret is still sealed to a key the machine no longer has")
}
}
}
if !found {
t.Fatal("the own secret has no row")
}
}
+500
View File
@@ -0,0 +1,500 @@
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"os"
"slices"
"sort"
"strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/validate"
"github.com/novox/mesh-controller/internal/inventory"
)
// The controller's composition on mesh-sdk's (novox/hq ADR 0453 clause 3, design 47 R4b, issues 1386 to 1388): every
// caller that planned or composed a node reads the Input once for its operation (compose_input.go), asks the SDK to
// compose over it, and, when it sends, performs what the composition names foreseen (compose_perform.go) and composes again.
// inputCacheKey carries an operation's Input in its context, so the planning of every node of one verb reads the stores once.
type inputCacheKey struct{}
type inputCache struct {
mu sync.Mutex
by map[string]*meshInput
}
// withInputCache is ctx for an operation: the Input read for it is kept until a send changes what it read. A context that
// carries none reads the stores for each composition, which is correct and slow.
func withInputCache(ctx context.Context) context.Context {
if _, ok := ctx.Value(inputCacheKey{}).(*inputCache); ok {
return ctx
}
return context.WithValue(ctx, inputCacheKey{}, &inputCache{by: map[string]*meshInput{}})
}
// freshInput is ctx for one composition whose Input is read afresh and kept only until the composition makes something: a
// send that is composed after another one made and recorded things (a push's machines, one after the other).
func freshInput(ctx context.Context) context.Context {
return context.WithValue(ctx, inputCacheKey{}, &inputCache{by: map[string]*meshInput{}})
}
// keptSignature is what the Input read depends on in ctx: which recorded modules each machine keeps at the build it runs
// (recorded_kept.go), so an Input read for a person's push is never taken for one read for a gated send.
func keptSignature(ctx context.Context) string {
skip, on := keptExcept(ctx)
var names []string
for m := range skip {
names = append(names, m)
}
sort.Strings(names)
every, _ := ctx.Value(keepEveryBuildKey{}).(string)
settings, _ := ctx.Value(keepForSettingsKey{}).(string)
return fmt.Sprintf("kept=%v;except=%v;every=%s;settings=%s", on, names, every, settings)
}
// inputFor is the Input for ctx's operation.
func inputFor(ctx context.Context, open *stores) (*meshInput, error) {
cache, ok := ctx.Value(inputCacheKey{}).(*inputCache)
if !ok {
return readComposeInput(ctx, open)
}
sig := keptSignature(ctx)
cache.mu.Lock()
defer cache.mu.Unlock()
if in := cache.by[sig]; in != nil {
return in, nil
}
in, err := readComposeInput(ctx, open)
if err != nil {
return nil, err
}
cache.by[sig] = in
return in, nil
}
// forgetInput drops what the operation read, because a send made or recorded something it read.
func forgetInput(ctx context.Context) {
if cache, ok := ctx.Value(inputCacheKey{}).(*inputCache); ok {
cache.mu.Lock()
cache.by = map[string]*meshInput{}
cache.mu.Unlock()
}
}
// resolveNode works out everything a node should run, from what was assigned to it (planFor on the SDK): over the Input,
// writing nothing. A failure to compose the node's own modules is marked notResolvable, as planFor marked it; every other
// failure is returned as it is.
func resolveNode(ctx context.Context, open *stores, node string) (catalogue.Resolution, catalogue.SettingsBy, error) {
in, err := inputFor(ctx, open)
if err != nil {
return catalogue.Resolution{}, nil, err
}
if err := in.refused[node]; err != nil {
return catalogue.Resolution{}, nil, err
}
resolved, err := compose.Resolve(in.Input, node)
if err != nil {
if compose.IsUnresolvable(err) {
return catalogue.Resolution{}, nil, notResolvable{err}
}
return catalogue.Resolution{}, nil, err
}
logUnheld(node, resolved.Plan.Unheld)
return resolved.Plan, resolved.Settings, nil
}
// generatorsOf is what every node's declaration is composed with, over the whole mesh (generators on the SDK).
func generatorsOf(ctx context.Context, open *stores) (map[string]catalogue.Generator, error) {
in, err := inputFor(ctx, open)
if err != nil {
return nil, err
}
return compose.Generators(in.Input)
}
// renderingOf is what a machine is rendered with as a question reads it (nothing made), with the private network: what the
// filter's rules, the roster and the like are derived from.
func renderingOf(ctx context.Context, open *stores, node string) (catalogue.Rendering, error) {
in, err := inputFor(ctx, open)
if err != nil {
return catalogue.Rendering{}, err
}
rendered, err := compose.Render(in.Input, node, compose.Reading)
if err != nil {
return catalogue.Rendering{}, markUnresolvable(err)
}
with := rendered.Rendering
if with.Generators, err = compose.Generators(in.Input); err != nil {
return catalogue.Rendering{}, err
}
return with, nil
}
// whoResolvesOn is the machines whose resolution answers a requirement, and why the others did not (whoResolves over the
// Input): by what a module provides, not by its name. A machine whose set does not compose is in refused, said; one the mesh
// could not be asked about is an error, never a refusal (novox/hq 04-ISSUES/152).
func whoResolvesOn(ctx context.Context, open *stores, requirement string) (map[string]bool, map[string]string, error) {
in, err := inputFor(ctx, open)
if err != nil {
return nil, nil, err
}
on := map[string]bool{}
refused := map[string]string{}
for _, n := range in.sortedNodes() {
if err := in.refused[n.Name]; err != nil {
return nil, nil, fmt.Errorf("whether %s answers %q cannot be read: %w", n.Name, requirement, err)
}
resolved, err := compose.Resolve(in.Input, n.Name)
switch {
case compose.IsUnresolvable(err):
refused[n.Name] = err.Error()
continue
case err != nil:
return nil, nil, fmt.Errorf("whether %s answers %q cannot be read: %w", n.Name, requirement, err)
}
for _, m := range resolved.Plan.Modules {
for _, offered := range m.Offers() {
if offered == requirement {
on[n.Name] = true
}
}
}
}
return on, refused, nil
}
// sortedNodes is the Input's nodes by name, the order the controller reads them in.
func (in *meshInput) sortedNodes() []compose.Node {
out := append([]compose.Node(nil), in.Nodes...)
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out
}
// composeQuestion composes a node's declaration as a question asks it: nothing is made, and what a send would make is a
// stand-in (Foreseeing) or left out (Reading). It also answers what the node-engine's validator refuses of it. The lines the
// composition found are said where the controller says its own (standard error, or the writer an act gave).
//
// withOrder says the declaration carries the order it was last sent under (composedAndValidated's, which a comparison with the
// last send needs); without it the order is none, as declarationWith left it for the sender to stamp.
func composeQuestion(ctx context.Context, open *stores, node string, mode compose.Mode, withOrder bool) (sendable, []string, error) {
in, err := inputFor(ctx, open)
if err != nil {
return sendable{}, nil, err
}
if err := in.refused[node]; err != nil {
return sendable{}, nil, err
}
d, problems, err := compose.ComposeIn(in.Input, node, mode)
if err != nil {
return sendable{}, nil, markUnresolvable(err)
}
if err := refusedBus(in, node); err != nil {
return sendable{}, nil, err
}
sayFound(ctx, d)
out := sendableOf(d)
if !withOrder {
out.Sequence, out.Epoch, out.Generation = 0, 0, 0
}
return out, problems, nil
}
// refusedBus is why the node cannot be composed because it runs the bus and the bus's user list cannot be: no user has a
// credential, so the list would refuse every connection in the mesh (busUserList). The controller's own composition refused that
// machine with those words; no other machine is refused for it. nil when nothing refuses the node.
func refusedBus(in *meshInput, node string) error {
if in.busRefused == nil {
return nil
}
resolved, err := compose.Resolve(in.Input, node)
if err != nil {
return nil // it does not compose for its own reasons, which are said
}
for _, m := range resolved.Plan.Modules {
if m.BusUsers != "" && m.ClaimsSeat("mesh-broker") {
return in.busRefused
}
}
return nil
}
// markUnresolvable is an error of a node whose own set does not compose, as planFor marked it.
func markUnresolvable(err error) error {
if compose.IsUnresolvable(err) {
return notResolvable{err}
}
return err
}
// composeSend composes a node's declaration to send it, as the Allocating composition did: composed over what is read,
// then what it named foreseen is made (perform), the stores are read again and it is composed again. A declaration that
// still names anything foreseen is refused, said, and never sent.
func composeSend(ctx context.Context, open *stores, node string) (sendable, error) {
declared, _, err := composeSendValidated(ctx, open, node)
return declared, err
}
// composeSendValidated is composeSend and what the node-engine's validator refuses of the declaration it composed, for the
// merge gate's two passes (composedAndValidated in its Allocating mode).
func composeSendValidated(ctx context.Context, open *stores, node string) (sendable, []string, error) {
// The core modules whose credentials the controller sealed itself because the vault was silent and no box was kept
// (ADR 0396): the composition stands in a vault box for each, and a send carries the controller's own seal instead.
byController, err := prepareSend(ctx, open, node)
if err != nil {
return sendable{}, nil, err
}
for attempt := 0; ; attempt++ {
in, err := inputFor(ctx, open)
if err != nil {
return sendable{}, nil, err
}
if err := in.refused[node]; err != nil {
return sendable{}, nil, err
}
d, problems, err := compose.ComposeIn(in.Input, node, compose.Foreseeing)
if err != nil {
return sendable{}, nil, markUnresolvable(err)
}
if err := refusedBus(in, node); err != nil {
return sendable{}, nil, err
}
if len(byController) > 0 {
if d, problems, err = sealedByTheController(in, node, d, byController); err != nil {
return sendable{}, nil, err
}
}
left := stillForeseen(d)
resolved, rerr := compose.Resolve(in.Input, node)
if rerr != nil {
return sendable{}, nil, markUnresolvable(rerr)
}
if len(resolved.Foreseen) > 0 {
left = append(left, resolved.Foreseen...)
}
if len(left) == 0 && len(in.wouldMake) == 0 {
sayFound(ctx, d)
out := sendableOf(d)
// The sender stamps the order it allots (order.stamp); a composition for a send carries none of its own.
out.Sequence, out.Epoch, out.Generation = 0, 0, 0
here, _ := nodeOf(in.Input, node)
out.Builds = buildsOf(in.Input, here, resolved.Plan, d.LeftOutWhy)
out.Bindings = boundToData(resolved.Plan, d.LeftOutWhy)
if len(d.Filled) > 0 {
if out.plans, err = handedPlans(in, node, resolved); err != nil {
return sendable{}, nil, err
}
}
return out, problems, nil
}
if attempt > 0 {
return sendable{}, nil, fmt.Errorf("%s was composed twice and still names what only a send makes (%s): nothing is sent "+
"that stands in for a value", node, strings.Join(left, "; "))
}
if _, err := perform(ctx, open, in, node, d); err != nil {
return sendable{}, nil, err
}
// What was made is in the stores now: the next read is a fresh one.
forgetInput(ctx)
}
}
// restOfTheMesh is what every other machine holds and offers (theRestOfTheMesh over the Input), for a question: the lines it
// finds are said where the controller says its own, and a seat it would pass on is passed by the next send (perform), not by
// the question. shelf, when given, replaces the Input's catalogue: a caller composing a change's shelf.
func restOfTheMesh(ctx context.Context, open *stores, shelf map[string]catalogue.Manifest, exclude string) (catalogue.World, error) {
in, err := inputFor(ctx, open)
if err != nil {
return catalogue.World{}, err
}
input := in.Input
if shelf != nil {
input.Catalogue = shelf
}
world, passes, left, err := compose.TheRestOfTheMesh(input, exclude)
if err != nil {
return catalogue.World{}, err
}
var w io.Writer = os.Stderr
if lw, ok := ctx.Value(leftOutKey{}).(io.Writer); ok {
w = lw
}
for _, l := range left {
fmt.Fprintln(w, l.String())
}
for _, line := range passes.Said {
fmt.Fprintln(w, line)
}
return world, nil
}
// restOfTheMeshOf is restOfTheMesh for a caller that holds the inventory alone: the other stores the Input reads are opened for
// the call and let go of again.
func restOfTheMeshOf(ctx context.Context, inv *inventory.Inventory, shelf map[string]catalogue.Manifest, exclude string) (catalogue.World, error) {
open := &stores{inventory: inv}
defer func() {
if open.licences != nil {
open.licences.Close()
}
if open.identity != nil {
open.identity.Close()
}
}()
return restOfTheMesh(ctx, open, shelf, exclude)
}
// sayFound says the lines a composition found while composing, once each: a machine left out of the rest of the mesh, a seat
// that two claimants could take, the bus's users left out for want of a credential.
func sayFound(ctx context.Context, d compose.Declared) {
if len(d.Said) == 0 {
return
}
var w io.Writer = os.Stderr
if lw, ok := ctx.Value(leftOutKey{}).(io.Writer); ok {
w = lw
}
for _, line := range d.Said {
fmt.Fprintln(w, line)
}
}
// sendableOf is a composed declaration as the controller's sends know it.
func sendableOf(d compose.Declared) sendable {
out := sendable{Resources: d.Resources, Sequence: d.Sequence, Epoch: d.Epoch, Generation: d.Generation,
ReadsModules: d.ReadsModules, moduleOf: d.ModuleOf, buildIDs: d.BuildIDs, modules: d.Modules,
localNetwork: d.LocalNetwork, Received: d.Received, Mesh: d.Mesh, BusUsers: d.BusUsers,
LeftOut: d.LeftOut, leftOutWhy: d.LeftOutWhy, withheld: d.Withheld, unbound: d.Unbound, foreseen: d.Foreseen,
unplaced: d.Unplaced, handedOnly: d.HandedOnly, Filled: d.Filled, Provisioner: d.Provisioner,
requiredBy: d.RequiredBy}
if d.Adoption != nil {
out.Adoption = &adoptionEnvelope{Taken: d.Adoption.Taken, Untaken: d.Adoption.Untaken}
}
return out
}
// buildsOf is the build of each module a declaration for the node carries, as a send records it (novox/hq issue 259): the
// module's current build for each module in it, the build it was last sent for one left out of it, and a module kept at the
// build the machine runs carried at that one (ADR 0245). Never nil.
func buildsOf(in compose.Input, node compose.Node, plan catalogue.Resolution, leftOut map[string]string) map[string]string {
before := node.SentBuilds
if !node.SentKnown {
before = nil
}
out := map[string]string{}
for _, m := range plan.Modules {
if _, left := leftOut[m.Module]; left {
if was, kept := before[m.Module]; kept {
out[m.Module] = was
}
continue
}
out[m.Module] = in.CurrentBuilds[m.Module]
}
for m, was := range node.KeptBuilds {
if _, carried := out[m]; carried {
out[m] = was
}
}
return out
}
// handedPlans is the plan of each module handed to the provisioner, which the assignments mirror keeps with the send (design
// 57 P6): the composition's declaration does not carry it, so it is composed from the same rendering again.
func handedPlans(in *meshInput, node string, resolved compose.Resolved) (map[string]catalogue.Plan, error) {
rendered, err := compose.Render(in.Input, node, compose.Foreseeing)
if err != nil {
return nil, err
}
gens, err := compose.Generators(in.Input)
if err != nil {
return nil, err
}
with := rendered.Rendering
with.Generators = gens
composed, err := resolved.Plan.Compose(with)
if err != nil {
return nil, err
}
return composed.Plans, nil
}
// sealedByTheController is the declaration with the controller's own seal in place of the vault's box for each core module the
// vault did not answer for and no box was kept for (core_box.go, ADR 0396): the file carries "sealed", as the controller's
// composition sent it, until the vault holds the credential. What the composition stood in for the box is replaced and no longer
// wanted; the validator is run again over the body.
func sealedByTheController(in *meshInput, node string, d compose.Declared, modules []string) (compose.Declared, []string, error) {
rendered, err := compose.Render(in.Input, node, compose.Foreseeing)
if err != nil {
return d, nil, err
}
stood, _ := compactBox(catalogue.ForeseenVaultBox)
var wanted []compose.CoreBoxWanted
var foreseen []string
for _, w := range d.CoreBoxesWanted {
if !slices.Contains(modules, w.Module) {
wanted = append(wanted, w)
continue
}
sealed := rendered.Rendering.Needed[w.Module][w.Name]
if sealed == "" {
return d, nil, fmt.Errorf("%s/%s on %s has no seal of the controller's to carry until the vault holds it", w.Module, w.Name, node)
}
for _, r := range d.Resources {
box, isBox := r[catalogue.VaultSealedField]
// By the resource's id alone: its path is the manifest's with the machine's words filled in, and so is not the
// manifest's own.
if !isBox || r["id"] != w.Module+"."+catalogue.NeedID(w.Name) {
continue
}
if had, err := compactBox(box); err != nil || had != stood {
continue
}
delete(r, catalogue.VaultSealedField)
r["sealed"] = sealed
}
}
// A stand-in box that is left is said by its resource, never sent and never guessed at.
for _, r := range d.Resources {
if box, has := r[catalogue.VaultSealedField]; has {
if had, err := compactBox(box); err == nil && had == stood {
if id, _ := r["id"].(string); slices.ContainsFunc(modules, func(m string) bool { return strings.HasPrefix(id, m+".") }) {
return d, nil, fmt.Errorf("%s on %s still carries a stand-in for the vault's box, and the controller has "+
"no seal of its own for it", id, node)
}
}
}
}
d.CoreBoxesWanted = wanted
for _, f := range d.Foreseen {
module, _, _ := strings.Cut(f, "/")
if !slices.Contains(modules, module) {
foreseen = append(foreseen, f)
}
}
d.Foreseen = foreseen
body, err := d.Body()
if err != nil {
return d, nil, err
}
return d, validate.Declaration(body), nil
}
// compactBox is a value as the compact JSON it marshals to, so a box is compared by what it says and not by the type that
// holds it.
func compactBox(v any) (string, error) {
raw, err := json.Marshal(v)
if err != nil {
return "", err
}
var out bytes.Buffer
if err := json.Compact(&out, raw); err != nil {
return "", err
}
return out.String(), nil
}
+111
View File
@@ -0,0 +1,111 @@
package main
import (
"context"
"reflect"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
)
// A declaration as the SDK composes it is the one the controller sends, byte for byte, whatever it carries: the body is the
// envelope's only marshaller on either side (design 47 R4b, issue 1386).
func TestASendableIsTheComposedDeclarationByteForByte(t *testing.T) {
d := compose.Declared{
Resources: []map[string]any{{"id": "files.config", "type": "file"}, {"id": "mesh.hosts", "type": "file"}},
Sequence: 7, Epoch: 3, Generation: 9, ReadsModules: true,
ModuleOf: map[string]string{"files.config": "files"}, BuildIDs: map[string]string{"files": "b-1"},
Adoption: &compose.Adoption{Taken: []string{"files"}, Untaken: map[string][]string{"nats": {"nats.server"}}},
Filled: []string{"/etc/files/filled.json"}, Provisioner: "mesh-provisioner",
LeftOut: []string{"cache"}, LeftOutWhy: map[string]string{"cache": "a setting cannot compose"},
}
want, err := d.Body()
if err != nil {
t.Fatal(err)
}
got, err := sendableOf(d).Body()
if err != nil {
t.Fatal(err)
}
if string(got) != string(want) {
t.Fatalf("the controller would send\n%s\nand the composition made\n%s", got, want)
}
if empty, err := sendableOf(compose.Declared{}).Body(); err != nil || !strings.Contains(string(empty), `"owns_nothing":true`) {
t.Fatalf("an empty declaration is not deliberate: %s %v", empty, err)
}
}
// 1386: what a send records as carried: the current build of each module it carries, the build a module left out of it was last
// sent (when that is known), and a module kept at the build the machine runs at that one. (The controller's carriedOn recorded
// the same; mesh-sdk's node_test TestAModuleLeftOutKeepsTheBuildItWasLastSent holds the left-out case.)
func TestWhatASendRecordsAsCarriedIsWhatTheControllerRecorded(t *testing.T) {
plan := catalogue.Resolution{Modules: []catalogue.Manifest{{Module: "postgres"}, {Module: "files"}, {Module: "cache"}}}
leftOut := map[string]string{"cache": "a setting cannot compose"}
for name, c := range map[string]struct {
sent map[string]string
known bool
kept map[string]string
want map[string]string
}{
"nothing sent": {want: map[string]string{"postgres": "p2", "files": "f2"}},
"sent and kept": {sent: map[string]string{"postgres": "p1", "cache": "c1", "files": "f1"}, known: true,
kept: map[string]string{"postgres": "p1"}, want: map[string]string{"postgres": "p1", "files": "f2", "cache": "c1"}},
"sent but not known": {sent: map[string]string{"cache": "c1"}, want: map[string]string{"postgres": "p2", "files": "f2"}},
"kept a module not in": {known: true, kept: map[string]string{"mail": "m1"}, want: map[string]string{"postgres": "p2", "files": "f2"}},
} {
in := compose.Input{CurrentBuilds: map[string]string{"postgres": "p2", "files": "f2", "cache": "c2"}}
node := compose.Node{SentBuilds: c.sent, SentKnown: c.known, KeptBuilds: c.kept}
if got := buildsOf(in, node, plan, leftOut); !reflect.DeepEqual(got, c.want) {
t.Errorf("%s: a send records %v, want %v", name, got, c.want)
}
}
}
// 1385: a declaration that stands in for a value anywhere is still foreseen, in a list the composition names or in the body
// alone (a pair credential's stand-in is in no list); one that stands in for nothing is not. It failed first, before the body
// was scanned.
func TestADeclarationThatStandsInForAValueIsStillForeseen(t *testing.T) {
clean := compose.Declared{Resources: []map[string]any{{"id": "files.config", "type": "file", "content": "ok"}}}
if left := stillForeseen(clean); len(left) != 0 {
t.Fatalf("a declaration that stands in for nothing is foreseen: %v", left)
}
for name, d := range map[string]compose.Declared{
"an own secret": {StoodIn: []string{"files/admin"}},
"a port": {ForeseenPorts: []string{"files:80"}},
"a core box": {CoreBoxesWanted: []compose.CoreBoxWanted{{Node: "anchor", Module: "mesh-vault", Name: "root"}}},
"a pass of a seat": {Passes: compose.Passes{Passes: []catalogue.SeatPass{{Seat: catalogue.Seat{Name: "mesh-store"}}}}},
"a pair credential": {Resources: []map[string]any{{"id": "files.needs-db", "sealed": catalogue.ForeseenSealed}}},
"the certificate": {Resources: []map[string]any{{"id": "proxy.cert", "content": compose.ForeseenCertificate}}},
"the bus's list": {Resources: []map[string]any{{"id": "nats.users", "content": compose.ForeseenBusUsers}}},
"the vault's box": {Resources: []map[string]any{{"id": "x", "box": catalogue.ForeseenVaultBox}}},
} {
if left := stillForeseen(d); len(left) == 0 {
t.Errorf("%s: a declaration that stands in for it is not foreseen", name)
}
}
}
// 1386: an Input read for a person's push is never the one read for a send that keeps recorded builds, and the cache is
// emptied by what a send made.
func TestAnInputIsCachedPerWhatItKeepsAndForgottenAfterASend(t *testing.T) {
ctx := withInputCache(context.Background())
plain := keptSignature(ctx)
kept := keptSignature(keepingRecorded(ctx))
bus := keptSignature(keepingRecorded(ctx, "nats"))
every := keptSignature(keepingEveryBuild(ctx, "anchor"))
for name, sig := range map[string]string{"a push": plain, "a kept send": kept, "the bus step": bus, "the grants step": every} {
for other, o := range map[string]string{"a push": plain, "a kept send": kept, "the bus step": bus, "the grants step": every} {
if name != other && sig == o {
t.Errorf("%s and %s read the same Input", name, other)
}
}
}
cache := ctx.Value(inputCacheKey{}).(*inputCache)
cache.by[plain] = &meshInput{}
forgetInput(ctx)
if len(cache.by) != 0 {
t.Fatal("what a send made left the Input it read in the cache")
}
}
@@ -7,7 +7,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
+1 -1
View File
@@ -15,8 +15,8 @@ import (
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -13,8 +13,8 @@ import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/overlay"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/overlay"
)
func aConfiguredMesh(t *testing.T) (*stores, configuredMesh, map[string]map[string]any) {
+4 -8
View File
@@ -7,8 +7,8 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
snapshot "github.com/novox/mesh-controller/internal/facts"
"git.novox.be/novox/mesh-sdk/go/catalogue"
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
"github.com/novox/mesh-controller/internal/link"
)
@@ -127,21 +127,17 @@ type snapshotFacts = snapshot.Facts
func toldEveryMachine(t *testing.T, open *stores, served map[string]map[string]any, on map[string][]string) map[string]string {
t.Helper()
ctx := t.Context()
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
told := map[string]string{}
for _, node := range fourMachines {
plan, settings, err := planFor(ctx, open, node)
plan, _, err := resolveNode(ctx, open, node)
if err != nil {
t.Fatalf("%s: %v", node, err)
}
with, _, err := renderingFor(ctx, open, node, plan, settings, gens, Allocating)
with, err := madeAndRendered(ctx, open, node)
if err != nil {
t.Fatalf("%s: %v", node, err)
}
+8 -18
View File
@@ -13,10 +13,9 @@ import (
"sort"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The core's own credentials, sealed by the vault (novox/hq ADR 0349 §16, issue 755; the lead's decision on issue 755,
@@ -40,15 +39,6 @@ import (
// the vault and the machine, cleared by the first send the vault answers whole. Only a machine with no sealing key, or
// one whose key cannot be read, leaves its module out.
// engineOpensVaultBox says whether a machine's node-engine opens the vault's box, by its own newest statement.
func engineOpensVaultBox(ctx context.Context, inv *inventory.Inventory, node string) (bool, error) {
stated, had, err := inv.HealthOf(ctx, node)
if err != nil {
return false, err
}
return had && stated.Contract >= link.VaultBoxContract, nil
}
// coreAnswer is the vault's `get` answer as the controller reads it: the name, the generation and the box, never
// opened.
type coreAnswer struct {
@@ -121,7 +111,7 @@ func sealingKeyPEM(raw string) (string, error) {
// own seal where nothing was kept; on any other question, the kept box, or a stand-in the next send asks for. Pure: what to keep and what to
// say is answered, not done.
func coreBoxesOf(node, sealingKey string, core map[string][]string, kept map[string]map[string]inventory.CoreBox,
choosing Choosing, ask coreAsk) coreBoxes {
allocating bool, ask coreAsk) coreBoxes {
out := coreBoxes{sealed: map[string]map[string]json.RawMessage{}, notPlaced: map[string]string{}}
modules := make([]string, 0, len(core))
for m := range core {
@@ -143,7 +133,7 @@ func coreBoxesOf(node, sealingKey string, core map[string][]string, kept map[str
for _, name := range core[module] {
k, has := kept[module][name]
valid := has && k.SealedTo == sealingKey && len(k.Box) > 0
if choosing != Allocating {
if !allocating {
if valid {
boxes[name] = k.Box
} else {
@@ -218,10 +208,10 @@ var coreVault = func(ctx context.Context) coreAsk {
// coreBoxesFor is coreBoxesOf for one machine, read from and kept in the store, and said on a send.
func coreBoxesFor(ctx context.Context, inv *inventory.Inventory, node string, modules []catalogue.Manifest,
handed func(catalogue.Manifest) bool, choosing Choosing) (coreBoxes, error) {
handed func(catalogue.Manifest) bool, allocating bool) (coreBoxes, error) {
core := coreModulesOf(modules, handed)
if len(core) == 0 {
if choosing == Allocating {
if allocating {
settleVaultSilent(ctx, node)
}
return coreBoxes{}, nil
@@ -235,16 +225,16 @@ func coreBoxesFor(ctx context.Context, inv *inventory.Inventory, node string, mo
return coreBoxes{}, err
}
var ask coreAsk
if choosing == Allocating {
if allocating {
ask = coreVault(ctx)
}
out := coreBoxesOf(node, key, core, kept, choosing, ask)
out := coreBoxesOf(node, key, core, kept, allocating, ask)
for _, b := range out.keep {
if err := inv.KeepCoreBox(ctx, node, b); err != nil {
return coreBoxes{}, err
}
}
if choosing == Allocating {
if allocating {
sayVaultSilent(ctx, node, out)
}
return out, nil
@@ -10,8 +10,8 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -96,7 +96,7 @@ func TestASilentVaultHoldsTheKeptBoxAndSaysSo(t *testing.T) {
func TestACoreModuleFirstPlacedWhileTheVaultIsSilentIsSealedByTheController(t *testing.T) {
open, vault := aCoreMesh(t, link.VaultBoxContract)
vault.silent = errors.New("no responders")
plan, settings, err := planFor(t.Context(), open, "laptop")
plan, settings, err := resolveNode(t.Context(), open, "laptop")
if err != nil {
t.Fatal(err)
}
+7 -7
View File
@@ -8,8 +8,8 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
@@ -68,7 +68,7 @@ func TestReplay908(t *testing.T) {
key := "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
core := map[string][]string{"nats": {"password"}}
ask := func(name, publicKey string) (coreAnswer, error) { return coreAnswer{}, errors.New("no responders") }
got := coreBoxesOf("novox", key, core, nil, Allocating, ask)
got := coreBoxesOf("novox", key, core, nil, true, ask)
if _, sealed := got.sealed["nats"]; sealed || len(got.notPlaced) != 0 {
t.Fatalf("nats is vault-sealed %v or not placed %v", got.sealed, got.notPlaced)
}
@@ -81,7 +81,7 @@ func TestReplay908(t *testing.T) {
}
// A kept box is still carried while the vault is silent.
kept := map[string]map[string]inventory.CoreBox{"nats": {"password": {SealedTo: key, Box: json.RawMessage(`{"v":1}`)}}}
got = coreBoxesOf("novox", key, core, kept, Allocating, ask)
got = coreBoxesOf("novox", key, core, kept, true, ask)
if string(got.sealed["nats"]["password"]) != `{"v":1}` || len(got.byController) != 0 {
t.Fatalf("the kept box was not held: %+v", got)
}
@@ -99,17 +99,17 @@ func TestOnlyASendAsksTheVault(t *testing.T) {
t.Fatal("the vault was asked by a question that is not a send")
return coreAnswer{}, nil
}
for _, choosing := range []Choosing{Reading, Foreseeing} {
if got := coreBoxesOf("anchor", key, core, kept, choosing, never); string(got.sealed["nats"]["broker"]) != `{"v":1}` {
for _, allocating := range []bool{false} {
if got := coreBoxesOf("anchor", key, core, kept, allocating, never); string(got.sealed["nats"]["broker"]) != `{"v":1}` {
t.Errorf("read %s", got.sealed["nats"]["broker"])
}
if got := coreBoxesOf("anchor", key, core, nil, choosing, never); string(got.sealed["nats"]["broker"]) !=
if got := coreBoxesOf("anchor", key, core, nil, allocating, never); string(got.sealed["nats"]["broker"]) !=
string(catalogue.ForeseenVaultBox) {
t.Errorf("with none kept, read %s", got.sealed["nats"]["broker"])
}
}
// A box kept for another key of the machine is never carried.
other := coreBoxesOf("anchor", aPublicKey(t), core, kept, Allocating,
other := coreBoxesOf("anchor", aPublicKey(t), core, kept, true,
func(string, string) (coreAnswer, error) { return coreAnswer{}, errors.New("silent") })
if _, vaultSealed := other.sealed["nats"]; vaultSealed || !reflect.DeepEqual(other.byController, []string{"nats"}) {
t.Fatalf("a box sealed to the machine's old key was carried: %+v", other)
@@ -6,7 +6,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
@@ -40,7 +40,7 @@ func TestAssigningAModuleThatSpeaksOnTheBusNamesItsCredential(t *testing.T) {
}
// And the push refuses to send it, naming the same verb, rather than sealing a placeholder.
plan, settings, err := planFor(ctx, open, "laptop")
plan, settings, err := resolveNode(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
+5 -3
View File
@@ -14,7 +14,7 @@ import (
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
@@ -152,6 +152,8 @@ type consumerCopy struct {
// probeData is D13.
func probeData(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
// One reading of the mesh for every machine this looks at.
ctx = withInputCache(ctx)
if d.js == nil {
return nil, errors.New("no bus to ask the machines over")
}
@@ -180,7 +182,7 @@ func probeData(ctx context.Context, d *doctor) ([]conditions.Observation, error)
}
var machines []*machine
for _, n := range nodes {
plan, _, err := planFor(ctx, open, n.Name)
plan, _, err := resolveNode(ctx, open, n.Name)
if err != nil {
if ctx.Err() != nil {
return nil, ctx.Err()
@@ -849,7 +851,7 @@ func deleteRetiredData(ctx context.Context, conn *nats.Conn, open *stores, r inv
return fmt.Errorf("%s of %s on %s was never measured, so where it is was never said; nothing was deleted",
r.Item, r.Module, r.Machine)
}
if plan, _, err := planFor(ctx, open, r.Machine); err == nil {
if plan, _, err := resolveNode(ctx, open, r.Machine); err == nil {
for _, m := range plan.Modules {
if _, still := m.DataItem(r.Item); still && m.Module == r.Module {
return fmt.Errorf("%s runs on %s again and declares %s: it is not retired any more. Nothing was done",
+10 -4
View File
@@ -8,8 +8,8 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/testbus"
@@ -56,13 +56,19 @@ func TestNatsUnassigningIrreplaceableDataRetiresItAndAnEmptyReplacementIsUrgent(
if _, err := inv.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
register(t, open, catalogue.Manifest{Module: "keeper", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.BackupSeat, Scope: catalogue.ScopeNode, Serves: []string{"backed-up", "now", "restore"}}}})
// Built from a commit, so that a send carries them: a holder is asked only of what the machine was sent (issue 1109).
built := inventory.Source{BuiltFrom: "c1"}
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "keeper", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.BackupSeat, Scope: catalogue.ScopeNode, Serves: []string{"backed-up", "now", "restore"}}}}, built); err != nil {
t.Fatal(err)
}
house, err := catalogue.ParseManifest([]byte(houseManifest))
if err != nil {
t.Fatal(err)
}
register(t, open, house)
if err := inv.RegisterModule(ctx, house, built); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", "house"); err == nil {
t.Fatal("irreplaceable data was assigned to a machine with nothing to back it up")
}
+1 -1
View File
@@ -11,7 +11,7 @@ import (
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
+1 -1
View File
@@ -9,8 +9,8 @@ import (
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
+147 -35
View File
@@ -15,7 +15,7 @@ import (
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -42,13 +42,53 @@ import (
// `delivery order` (a group's order from the graph), `delivery check` (a group's heads composed), `delivery
// go`, `delivery stop` and `delivery walks`.
// onTheControllersPath are the modules whose walk never waits for the delivery's owner, and what each is.
// onTheControllersPath are the modules, by name, whose walk never waits for the delivery's owner, and what each is.
// The delivery's owner is not among them: it is whichever module claims the seat (ownPathOf, novox/hq ADR 0437 §3).
var onTheControllersPath = map[string]string{
"mesh-controller": "the controller",
"mesh-host": "the node-engine",
"node-tools": "the node tools",
"nats": "the bus",
catalogue.DeliverySeat: "the delivery's owner",
"mesh-controller": "the controller",
"mesh-host": "the node-engine",
"node-tools": "the node tools",
"nats": "the bus",
}
// claimantsOf are the modules among the catalogued entries that claim a seat, each once, in name order.
func claimantsOf(entries []inventory.Entry, seat string) []string {
var out []string
for _, e := range entries {
if e.Manifest.ClaimsSeat(seat) && !slices.Contains(out, e.Manifest.Module) {
out = append(out, e.Manifest.Module)
}
}
sort.Strings(out)
return out
}
// ownPathOf is the controller's own path among these entries: the modules by name, and the modules that claim
// the delivery seat, which is how the delivery's owner is found (novox/hq ADR 0437 §3).
func ownPathOf(entries []inventory.Entry) map[string]string {
own := make(map[string]string, len(onTheControllersPath)+1)
for m, what := range onTheControllersPath {
own[m] = what
}
for _, m := range claimantsOf(entries, catalogue.DeliverySeat) {
own[m] = "the delivery's owner"
}
return own
}
// onTheOwnPath is whether one module, with its manifest at hand, is on the controller's own path.
func onTheOwnPath(module string, m catalogue.Manifest) bool {
_, named := onTheControllersPath[module]
return named || m.ClaimsSeat(catalogue.DeliverySeat)
}
// ownPathNow is ownPathOf the catalogue as it is.
func ownPathNow(ctx context.Context, inv *inventory.Inventory) (map[string]string, error) {
entries, err := inv.Catalogued(ctx)
if err != nil {
return nil, err
}
return ownPathOf(entries), nil
}
// deliverySeatHeld is whether a module claiming the delivery seat is assigned somewhere: the seat has a
@@ -69,8 +109,9 @@ func awaitsFor(entries []inventory.Entry, modules []string) *inventory.PlanDeliv
if !deliverySeatHeld(entries) {
return nil
}
onPath := ownPathOf(entries)
for _, m := range modules {
if _, own := onTheControllersPath[m]; own {
if _, own := onPath[m]; own {
return nil
}
}
@@ -327,16 +368,17 @@ func stopInBatch(ctx context.Context, inv *inventory.Inventory, b *inventory.Pla
// the stop's one act. The kind is read from what the merges moved; the stopped walk itself is no walk a cut folds.
func batchToWalkAgain(ctx context.Context, inv *inventory.Inventory, stopping string, again []inventory.BatchedMerge,
now time.Time) (inventory.Plan, []*inventory.Plan, []inventory.MergeMove, error) {
own := false
for _, n := range movesOf(again) {
if _, on := onTheControllersPath[n]; on {
own = true
}
}
entries, read, edges, err := theGraph(ctx, inv)
if err != nil {
return inventory.Plan{}, nil, nil, err
}
own := false
onPath := ownPathOf(entries)
for _, n := range movesOf(again) {
if _, on := onPath[n]; on {
own = true
}
}
joins, _, _, err := relatedBatches(ctx, inv, again, own, stopping, entries, read, edges)
if err != nil {
return inventory.Plan{}, nil, nil, err
@@ -742,6 +784,27 @@ func reachOfMembers(members []orderMember, entries []inventory.Entry, read map[s
return out
}
// changeMember is one change as the planner is asked about it, the same for `delivery plan` and `plans --what-if`.
func changeMember(repository, base, head string, paths, removed []string, moduleDirs *[]string) orderMember {
m := orderMember{ID: repository + "@" + head, Repository: repository, Base: base, Head: head,
Paths: paths, Removed: removed}
if moduleDirs != nil {
m.ModuleDirs, m.ModuleDirsSaid = *moduleDirs, true
}
return m
}
// deliveryPlanDocument is the answer of `delivery plan` and of `plans --what-if … --data`: one change's
// change plan with its reach, the mesh's own part in it and whether the gate would run. One function, so
// the two verbs cannot differ (novox/hq ADR 0450 §2, issue 1324).
func deliveryPlanDocument(m orderMember, entries []inventory.Entry, read map[string][]inventory.ReadRepository,
edges []inventory.Edge, policy policyOf) map[string]any {
s := pullScope(m.pull(), entries, read, edges)
plan := changePlanOf(m.Repository, m.Base, m.Head, s.Reach, entries, policy)
return map[string]any{"plan": plan, "reach": orderReach{Moved: s.Modules, Dependents: s.Dependents,
New: s.New, Manifests: s.Manifests}, "mesh": s.Mesh, "gated": s.gated()}
}
// deliveryCommand is `delivery`, the controller's verbs for the delivery's owner:
//
// delivery plan --repository owner/repo --head <commit> [--base main] --paths a,b [--module-dirs …] [--removed …]
@@ -763,6 +826,8 @@ func deliveryCommand(ctx context.Context, args []string) error {
moduleDirs := set.String("module-dirs", "", "the directories holding a module.json at the head, comma-separated")
removed := set.String("removed", "", "the files it deletes, comma-separated")
membersJSON := set.String("members", "", "a group's members, as JSON")
membersFile := set.String("members-file", "", "a file of a group's members, as JSON: the same as --members, for a list "+
"too long for an argument (novox/hq issue 1272)")
group := set.String("group", "", "a delivery group's id")
ready := set.Bool("ready", false, "the head's pull request is marked ready: its check is queued before every other check "+
"(novox/hq ADR 0406 §3)")
@@ -777,6 +842,13 @@ func deliveryCommand(ctx context.Context, args []string) error {
return err
}
var members []orderMember
if *membersFile != "" {
raw, err := os.ReadFile(*membersFile)
if err != nil {
return fmt.Errorf("--members-file: %w", err)
}
*membersJSON = string(raw)
}
if *membersJSON != "" {
if err := json.Unmarshal([]byte(*membersJSON), &members); err != nil {
return fmt.Errorf("the members are not readable: %w", err)
@@ -804,22 +876,15 @@ func deliveryCommand(ctx context.Context, args []string) error {
if *repository == "" || *head == "" && *paths == "" {
return errors.New("delivery plan --repository owner/repo --head <commit> --paths a,b")
}
m := orderMember{ID: *repository + "@" + *head, Repository: *repository, Base: *base, Head: *head,
Paths: splitList(*paths), Removed: splitList(*removed)}
if d := moduleDirsOf(*moduleDirs); d != nil {
m.ModuleDirs, m.ModuleDirsSaid = *d, true
}
m := changeMember(*repository, *base, *head, splitList(*paths), splitList(*removed), moduleDirsOf(*moduleDirs))
entries, read, edges, err := theGraph(ctx, inv)
if err != nil {
return err
}
s := pullScope(m.pull(), entries, read, edges)
plan := changePlanOf(*repository, *base, *head, s.Reach, entries, func(module string) (inventory.Upgrade, bool) {
return answer(deliveryPlanDocument(m, entries, read, edges, func(module string) (inventory.Upgrade, bool) {
u, err := inv.UpgradeOf(ctx, module)
return u, err == nil
})
return answer(map[string]any{"plan": plan, "reach": orderReach{Moved: s.Modules, Dependents: s.Dependents,
New: s.New, Manifests: s.Manifests}, "mesh": s.Mesh, "gated": s.gated()})
}))
case "order":
if len(members) == 0 {
return errors.New("delivery order --members <json>: a group has members")
@@ -837,10 +902,14 @@ func deliveryCommand(ctx context.Context, args []string) error {
if *group == "" && len(members) == 1 {
// One head, checked again as the forge's announcement would have it (a recheck): the controller's
// own path for a pull request, run because the delivery's owner asked.
if err := sayingOnTheBus(ctx, inv, func() error { return (following{open: open, oneShot: true}).PullUpdated(ctx, members[0].pull()) }); err != nil {
return err
}
return answer(map[string]any{"rechecked": members[0].ID})
return recheckOneHead(members[0].ID, answer, func() (string, error) {
var asked string
err := sayingOnTheBus(ctx, inv, func() (err error) {
asked, err = (following{open: open, oneShot: true}).pullUpdated(ctx, members[0].pull(), os.Stderr)
return err
})
return asked, err
})
}
if *group == "" || len(members) < 2 {
return errors.New("delivery check --group <id> --members <json> (two heads or more), or --members <one head>")
@@ -935,7 +1004,7 @@ func deliveryCommand(ctx context.Context, args []string) error {
walks[i].Phases = walks[i].WalkPhases(now, appliedFrom(ctx, inv))
}
return answer(map[string]any{"held": deliverySeatHeld(entries), "walks": walks,
"own-path": sortedKeysOf(ownPathWords())})
"own-path": sortedKeysOf(ownPathOf(entries))})
}
return fmt.Errorf("delivery %s: plan, order, check, go, stop, backups-again or walks", sub)
}
@@ -1006,9 +1075,6 @@ func recordWalkPhases(ctx context.Context, inv *inventory.Inventory, now time.Ti
return nil
}
// ownPathWords is the controller's own path as words, for an answer.
func ownPathWords() map[string]string { return onTheControllersPath }
// theGraph is what the planner reads.
func theGraph(ctx context.Context, inv *inventory.Inventory) ([]inventory.Entry, map[string][]inventory.ReadRepository,
[]inventory.Edge, error) {
@@ -1094,7 +1160,7 @@ func askGroupCheck(ctx context.Context, open *stores, group string, members []or
if failing, cyclic, err := cycleVerdict(ctx, group, members, reach, edges, entries); err != nil {
return "", err
} else if cyclic {
sayChecked(ctx, failing)
sayChecked(ctx, os.Stderr, failing)
return failing.ID, nil
}
primary := groupPrimary(members, reach)
@@ -1113,7 +1179,7 @@ func askGroupCheck(ctx context.Context, open *stores, group string, members []or
request.Check.Group = group
// A group's check is queued as ready when any member's pull request is marked ready.
request.Check.Ready = slices.ContainsFunc(members, func(m orderMember) bool { return m.Ready })
world, err := theRestOfTheMesh(ctx, open.inventory, shelfOf(entries), "")
world, err := restOfTheMesh(ctx, open, shelfOf(entries), "")
if err != nil {
return "", err
}
@@ -1181,8 +1247,32 @@ func shelfOf(entries []inventory.Entry) map[string]catalogue.Manifest {
// On the save that starts the walk and the one that ends it, the walk also says what it touches, its bound and,
// at the end, its outcome (novox/hq ADR 0308 decision 1, walk_said.go), read from facts; nil facts say only what
// the walk recorded.
//
// The plan the announcement is made from is a copy that shares nothing with the walk, taken before the goroutine
// starts: the walk goes on writing its steps' maps and pointers while the announcement is marshalled and its
// touches read (novox/hq issue 1260), and it says the plan as it was when this was called.
func sayPlanMoved(ctx context.Context, bus link.Bus, facts walkFacts, p inventory.Plan) {
go publishPlanMoved(ctx, bus, facts, p)
kept, err := copyOfPlan(p)
if err != nil {
fmt.Fprintf(os.Stderr, "%s: kept, and could not be said as plan-moved: %v\n", p.ID, err)
return
}
go publishPlanMoved(ctx, bus, facts, kept)
}
// copyOfPlan is the plan with nothing shared with the original: its JSON round trip, which is all of it but what
// the plan never keeps (WalkSaid), carried over.
func copyOfPlan(p inventory.Plan) (inventory.Plan, error) {
raw, err := json.Marshal(p)
if err != nil {
return inventory.Plan{}, err
}
var kept inventory.Plan
if err := json.Unmarshal(raw, &kept); err != nil {
return inventory.Plan{}, err
}
kept.WalkSaid = p.WalkSaid
return kept, nil
}
func publishPlanMoved(ctx context.Context, bus link.Bus, facts walkFacts, p inventory.Plan) {
@@ -1290,3 +1380,25 @@ func overNATS(conn *nats.Conn) (link.Bus, error) {
}
return link.OverNATS{Conn: conn, JS: js}, nil
}
// recheckAnswer is what `delivery check` answers for one head: the id of the check asked of the build seat (empty when
// the controller said the verdict itself and asked nothing), which mesh-delivery reads, and the head rechecked.
func recheckAnswer(asked, member string) map[string]any {
return map[string]any{"asked": asked, "rechecked": member}
}
// recheckOneHead runs a one-head `delivery check` and answers it as data. Standard output is the answer's JSON alone,
// or the seat verb reads no answer from it (novox/hq issue 1311): while the check runs, whatever it prints to standard
// output (the controller's own path prints its prose there, as do the stores it reads) goes to standard error.
func recheckOneHead(member string, answer func(any) error, ask func() (string, error)) error {
real := os.Stdout
os.Stdout = os.Stderr
asked, err := func() (string, error) {
defer func() { os.Stdout = real }()
return ask()
}()
if err != nil {
return err
}
return answer(recheckAnswer(asked, member))
}
+1 -1
View File
@@ -12,7 +12,7 @@ import (
"sync"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -0,0 +1,85 @@
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// `delivery check` with one head answers the id of the check it asked, as data (novox/hq issue 1311): the prose
// the check says goes to standard error, so standard output is the document alone and answerOf reads it.
func TestDeliveryCheckOfOneHeadAnswersTheRequestID(t *testing.T) {
argv := []string{"delivery", "check", "--members", "[]"}
doc, err := json.Marshal(recheckAnswer("check-42", "novox/x@abc"))
if err != nil {
t.Fatal(err)
}
// Seen failing first: the prose the command used to print on standard output before the document.
polluted := append([]byte("novox/x#7 (abc): asked the build seat as check-42\n"), doc...)
if got := answerOf(argv, polluted, "", true); got.Answer != nil {
t.Fatalf("prose before the document should leave no answer; got %v", got.Answer)
}
got := answerOf(argv, doc, "novox/x#7 (abc): asked the build seat as check-42\n", true)
m, ok := got.Answer.(map[string]any)
if !ok || m["asked"] != "check-42" || m["rechecked"] != "novox/x@abc" {
t.Fatalf("the answer is %v; want asked check-42 and rechecked novox/x@abc", got.Answer)
}
if !strings.Contains(got.Output, "asked the build seat") {
t.Fatalf("the prose is still said in the text: %q", got.Output)
}
}
// The text of the other callers does not change: sayChecked writes the same line to the writer it is given, and
// the controller's own path still gives PullUpdated standard output.
func TestSayCheckedWritesTheSameLineToItsWriter(t *testing.T) {
var w bytes.Buffer
sayChecked(context.Background(), &w, link.Checked{ID: "c1", Owner: "novox", Repo: "x", Number: 7, Commit: "0123456789abcdef",
Verdict: "pass", Summary: "fine"})
want := "c1: novox/x#7 at 01234567 checked on a machine that did not say which: gate PASS — fine; repository none\n"
if w.String() != want {
t.Fatalf("sayChecked said %q; want %q", w.String(), want)
}
}
// Whatever the check prints to standard output while it runs (checks.go namedForTheCheck, the inventory's pointer
// repair) must not reach the answer's standard output: it is exactly one JSON document (novox/hq issue 1311).
func TestOneHeadRecheckLeavesStandardOutputTheAnswerAlone(t *testing.T) {
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
e, ew, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
realOut, realErr := os.Stdout, os.Stderr
os.Stdout, os.Stderr = w, ew
runErr := recheckOneHead("novox/x@abc", func(v any) error { return json.NewEncoder(os.Stdout).Encode(v) },
func() (string, error) {
fmt.Println(" the earlier checks' builds could not be read, so the check builds")
fmt.Fprintln(os.Stdout, "repaired a pointer")
return "check-42", nil
})
os.Stdout, os.Stderr = realOut, realErr
w.Close()
ew.Close()
if runErr != nil {
t.Fatal(runErr)
}
var out, errOut bytes.Buffer
_, _ = out.ReadFrom(r)
_, _ = errOut.ReadFrom(e)
var doc map[string]any
if err := json.Unmarshal(out.Bytes(), &doc); err != nil || doc["asked"] != "check-42" || doc["rechecked"] != "novox/x@abc" {
t.Fatalf("standard output %q is not the answer alone (%v)", out.String(), err)
}
if !strings.Contains(errOut.String(), "repaired a pointer") {
t.Fatalf("the prose should be on standard error: %q", errOut.String())
}
}
+13 -2
View File
@@ -10,8 +10,8 @@ import (
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
@@ -55,6 +55,17 @@ type stalledLine struct {
Bound string `json:"bound"`
H2 string `json:"h2"`
Says string `json:"says"`
// Severity is "urgent" for a group held or half merged and a refused member (novox/hq issues 1275, 1278); empty
// or anything else is a warning, as before.
Severity string `json:"severity,omitempty"`
}
// severity is the condition's severity a stalled line asks: urgent when it says so, a warning otherwise.
func (l stalledLine) severity() conditions.Severity {
if l.Severity == string(conditions.Urgent) {
return conditions.Urgent
}
return conditions.Warning
}
// waitingCheck is one merge check a stalled line waits on, as mesh-delivery gives it: its context, its state —
@@ -165,7 +176,7 @@ func stalledObservations(lines []stalledLine) []conditions.Observation {
out := make([]conditions.Observation, 0, len(lines))
for _, l := range lines {
o := conditions.Observation{Scope: conditions.ScopeDelivery, ID: l.ID, Kind: kindDeliveryStalled,
Severity: conditions.Warning,
Severity: l.severity(),
Summary: fmt.Sprintf("the delivery %s has been %s for %s, past its bound of %s (%s): healer H2 may %s — "+
"`mesh-delivery.show %s`", l.ID, l.State, l.For, l.Bound, l.Says, l.H2, l.ID),
Said: fmt.Sprintf("%s for %s", l.State, l.For)}
@@ -13,8 +13,8 @@ import (
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
@@ -8,9 +8,9 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
@@ -122,7 +122,7 @@ func TestAMergeWaitsForItsDeliverysWordAndThePersonsWordWorksWithoutIt(t *testin
advanceHeld(ctx, open)
got, _ = inv.PlanByID(ctx, p.ID)
if got.Waiting() || !strings.HasPrefix(got.Delivery.By, "a person") || len(*asked) < 4 {
t.Fatalf("a person's word did not start the walk: %+v, asked %v", got.Delivery, *asked)
t.Fatalf("a person's word did not start the walk: %+v, asked %v, state %q, tier %d, note %q", got.Delivery, *asked, got.State, got.Tier, got.Note)
}
// The owner's stop: failed, said as stopped by it.
@@ -0,0 +1,37 @@
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/conditions"
)
func severityRaisedFor(t *testing.T, line string) conditions.Severity {
t.Helper()
var l stalledLine
if err := json.Unmarshal([]byte(line), &l); err != nil {
t.Fatal(err)
}
got := stalledObservations([]stalledLine{l})
if len(got) != 1 {
t.Fatalf("%d conditions", len(got))
}
return got[0].Severity
}
func TestAnUrgentStalledLineRaisesAnUrgentCondition(t *testing.T) { // 1278/AC1
line := `{"id":"novox/mesh-delivery#5","state":"held","for":"3m0s","bound":"0s","h2":"close","says":"held","severity":"urgent"}`
if got := severityRaisedFor(t, line); got != conditions.Urgent {
t.Errorf("an urgent line raised %q", got)
}
}
func TestAStalledLineWithoutOrWithAnUnknownSeverityRaisesAWarning(t *testing.T) { // 1278/AC2
for _, extra := range []string{``, `,"severity":""`, `,"severity":"critical"`, `,"severity":"warning"`} {
line := `{"id":"novox/a#1","state":"unanswered","for":"3m0s","bound":"2m0s","h2":"close","says":"x"` + extra + `}`
if got := severityRaisedFor(t, line); got != conditions.Warning {
t.Errorf("the line %s raised %q", extra, got)
}
}
}
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
+1 -1
View File
@@ -3,7 +3,7 @@ package main
import (
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
+14
View File
@@ -120,6 +120,16 @@ var probeRegistry = []probe{
{ID: probeDeliveriesID, Asserts: "no delivery is held past its state's bound unsaid: mesh-delivery's " +
"`stalled`, each with the transition its table lets healer H2 take", From: "ADR 0239",
Kind: kindDeliveryStalled, Phase: 3, run: probeDeliveries},
// What the watchman says stopped moving (ADR 0433 §2, issue 1156): each key of its stuck bucket not moved, but a
// node's presence, which the silent check takes.
{ID: probeStuckID, Asserts: "no key the watchman says stopped moving is left unsaid: each key of its stuck bucket " +
"not moved is a condition of the delivery scope until it says moved", From: "ADR 0433 §2, issue 1156",
Kind: kindStuck, Phase: 3, run: probeStuck},
// A first merge into a repository the mesh builds nothing from, kept waiting for mesh-delivery's word on the
// operator's Approve (novox/hq ADR 0432, issue 1146): said past a bound, never only in a log line (ADR 0426 §3, §8).
{ID: probeFirstBuildsID, Asserts: "no first merge into a repository the mesh builds nothing from waits more than ten " +
"minutes for mesh-delivery's announcement of the operator's Approve of its head", From: "ADR 0432, issue 1146",
Kind: kindFirstBuildUnasked, Phase: 3, run: probeFirstBuilds},
// The delivery budgets (novox/hq ADR 0282 decision 7): the newest delivery of each class within its budget,
// read from mesh-delivery's `times`; a measurement said, never a delivery held.
{ID: probeBudgetsID, Asserts: "the newest delivery of a leaf module ran on every machine within three minutes of " +
@@ -178,6 +188,10 @@ var probeRegistry = []probe{
{ID: backupHeldProbe, Asserts: "no delivery plan is held for a backup before its send that failed or did not " +
"answer in ten minutes", From: "ADR 0404 decision 4, issue 1066", Kind: kindBackupHeld, Phase: 1,
run: probeBackupHeld},
// A plan's send key the bus refused (ADR 0433 §4, issue 1157): in place of the timer that stood in for it.
{ID: sendKeyRefusedID, Asserts: "no delivery plan's key that watches a machine for its report was refused by the " +
"bus at the newest look", From: "ADR 0433 §4, issue 1157", Kind: kindSendKeyRefused, Phase: 1,
run: probeSendKeyRefused},
}
// probeVerdict is one probe's outcome in a run.
+1 -1
View File
@@ -10,8 +10,8 @@ import (
"golang.org/x/net/dns/dnsmessage"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
+1 -3
View File
@@ -72,8 +72,6 @@ func summarise(ds []inventory.Duration) []durationSummary {
switch k.kind {
case inventory.DurationApply:
s.Suggests = "S2 bound max(2m, 3×last apply) ≈ " + round(max(2*time.Minute, 3*at(0.9))) + " at the p90"
case inventory.DurationHeartbeatGap:
s.Suggests = "S1 bound 3×interval ≈ " + round(3*at(0.5))
}
out = append(out, s)
}
@@ -138,7 +136,7 @@ func durationsCommand(ctx context.Context, args []string) error {
return nil
}
if len(summary) == 0 {
fmt.Printf("nothing measured in the last %d day(s): the serving controller records apply, heartbeat-gap, "+
fmt.Printf("nothing measured in the last %d day(s): the serving controller records apply, "+
"plan-tier and build durations as it hears them\n", *days)
return nil
}
+2 -10
View File
@@ -42,12 +42,8 @@ func TestAMachineIsSentTheEpochOnlyOnceItSaysItReadsOne(t *testing.T) {
if err := inv.RecordReadsEpoch(ctx, anchor.ID, true); err != nil {
t.Fatal(err)
}
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
d := &bodiesDelivery{recordedDelivery: recordedDelivery{inv: inv}, bodies: map[string][]byte{}}
if _, err := sendRound(ctx, open, []string{"anchor", "laptop"}, composeForPush(open, gens), d, "", nil); err != nil {
if _, err := sendRound(ctx, open, []string{"anchor", "laptop"}, composeForPush(open), d, "", nil); err != nil {
t.Fatal(err)
}
carried := func(node string) (epoch float64, has bool) {
@@ -89,12 +85,8 @@ func TestNothingIsComposedOrSentWithoutTheLease(t *testing.T) {
was := epochForActs
epochForActs = func(context.Context) (uint64, error) { return 0, errors.New("this controller lost the lease") }
t.Cleanup(func() { epochForActs = was })
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
d := &bodiesDelivery{recordedDelivery: recordedDelivery{inv: open.inventory}, bodies: map[string][]byte{}}
refused, err := sendRound(ctx, open, []string{"anchor"}, composeForPush(open, gens), d, "", nil)
refused, err := sendRound(ctx, open, []string{"anchor"}, composeForPush(open), d, "", nil)
if err != nil {
t.Fatal(err)
}
+16 -45
View File
@@ -14,13 +14,12 @@ import (
"sync"
"time"
"github.com/novox/mesh-host/validate"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
"github.com/novox/mesh-controller/internal/artifacts"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
snapshot "github.com/novox/mesh-controller/internal/facts"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -170,10 +169,15 @@ func factsStore(ctx context.Context, open *stores) (string, error) {
return address, nil
}
// writtenFormat is the format of the snapshot this controller writes: 1, the format its own reader knew before mesh-sdk's
// go/facts took the type over (issue 1390, novox/hq ADR 0456). The readers (the merge gate, the check) read formats 1 and
// 2; the writer moves to 2, which also carries each machine's grants and bindings, in a later release of its own.
const writtenFormat = 1
// gatherFacts composes one snapshot from the store: read only, nothing made, nothing sent.
func gatherFacts(ctx context.Context, open *stores, busVersion string) (snapshot.Facts, error) {
inv := open.inventory
f := snapshot.Facts{Format: snapshot.Format, Taken: time.Now().UTC(), Controller: snapshot.Build{Version: version}}
f := snapshot.Facts{Format: writtenFormat, Taken: time.Now().UTC(), Controller: snapshot.Build{Version: version}}
f.Versions.Bus = busVersion
storeVersion, err := inv.ServerVersion(ctx)
if err != nil {
@@ -263,7 +267,9 @@ func gatherFacts(ctx context.Context, open *stores, busVersion string) (snapshot
f.Controller.Commit = c.Commit
}
gens, gensErr := generators(ctx, open)
// The Input every machine's declaration below is composed over is read once for the whole snapshot.
ctx = withInputCache(ctx)
_, gensErr := generatorsOf(ctx, open)
hostShelf := shelf[hostModule]
meshWide := map[string]bool{}
engines := map[string]bool{}
@@ -361,7 +367,7 @@ func gatherFacts(ctx context.Context, open *stores, busVersion string) (snapshot
m.Settings = append(m.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Whole())})
}
}
m.Declaration = declarationFacts(ctx, open, n.Name, gens, gensErr, scrub)
m.Declaration = declarationFacts(ctx, open, n.Name, gensErr, scrub)
if ctx.Err() != nil {
return snapshot.Facts{}, ctx.Err()
}
@@ -456,14 +462,14 @@ func gatherFacts(ctx context.Context, open *stores, busVersion string) (snapshot
// declarationFacts is how one machine's declaration composes now, as the next push would compose it and
// without making anything (D1's composition), and whether the node-engine's validator takes it.
func declarationFacts(ctx context.Context, open *stores, node string, gens map[string]catalogue.Generator,
gensErr error, scrub *snapshot.Scrubber) snapshot.Declaration {
func declarationFacts(ctx context.Context, open *stores, node string, gensErr error,
scrub *snapshot.Scrubber) snapshot.Declaration {
var d snapshot.Declaration
if gensErr != nil {
d.Problems = []string{scrub.Text("the private network cannot be computed: " + oneLine(gensErr.Error()))}
return d
}
declared, problems, err := composedAndValidated(ctx, open, node, gens, Foreseeing)
declared, problems, err := composeQuestion(ctx, open, node, compose.Foreseeing, true)
if err != nil {
d.Problems = []string{scrub.Text(oneLine(err.Error()))}
return d
@@ -496,41 +502,6 @@ func declarationFacts(ctx context.Context, open *stores, node string, gens map[s
return d
}
// composedAndValidated composes one machine's declaration — as a push would (Allocating) or as the next
// push will without making anything (Foreseeing) — with the order it was last sent, and runs the
// node-engine's own validator over the body. An error is that it did not compose; problems are what the
// validator refuses.
func composedAndValidated(ctx context.Context, open *stores, node string, gens map[string]catalogue.Generator,
choosing Choosing) (sendable, []string, error) {
plan, settings, err := planFor(ctx, open, node)
if err != nil {
return sendable{}, nil, err
}
declared, err := declarationWith(ctx, open, node, plan, settings, gens, choosing)
if err != nil {
return sendable{}, nil, err
}
record, err := open.inventory.NodeByName(ctx, node)
if err != nil {
return sendable{}, nil, err
}
if declared.Sequence, err = open.inventory.Sequence(ctx, record.ID); err != nil {
return sendable{}, nil, err
}
if declared.Epoch, err = open.inventory.SentEpoch(ctx, record.ID); err != nil {
return sendable{}, nil, err
}
// And the generation it was last sent, as the would-send is (novox/hq issue 234).
if declared.Generation, err = open.inventory.SentGeneration(ctx, record.ID); err != nil {
return sendable{}, nil, err
}
body, err := declared.Body()
if err != nil {
return sendable{}, nil, err
}
return declared, validate.Declaration(body), nil
}
// resourceNames are a declaration's resources as `type:id`, sorted.
func resourceNames(resources []map[string]any) []string {
out := make([]string, 0, len(resources))
+58
View File
@@ -0,0 +1,58 @@
package main
import (
"encoding/json"
"strings"
"testing"
"time"
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
)
// 1390 (novox/hq ADR 0456): the snapshot this controller writes is format 1, and the reader it shares with the build seat
// (mesh-sdk's go/facts) takes formats 1 and 2 and refuses one newer than it knows. The writer moves to format 2 in a release
// of its own. It failed first with the controller's own reader, which refused format 2 as newer than it knew.
func TestTheWriterWritesFormatOneAndTheReaderTakesBothFormats(t *testing.T) {
if writtenFormat != 1 {
t.Fatalf("the writer writes format %d, not 1", writtenFormat)
}
taken := time.Date(2026, 10, 20, 9, 0, 0, 0, time.UTC)
for format, want := range map[int]bool{1: true, 2: true, 3: false} {
body, err := (snapshot.Facts{Format: format, Taken: taken}).Encode()
if err != nil {
t.Fatal(err)
}
f, err := snapshot.Decode(body)
switch {
case want && err != nil:
t.Errorf("format %d is refused: %v", format, err)
case want && f.Format != format:
t.Errorf("format %d is read as %d", format, f.Format)
case !want && (err == nil || !strings.Contains(err.Error(), "newer")):
t.Errorf("format %d is not refused as newer: %v", format, err)
}
}
// What the writer composes carries no field of format 2: a reader of 1 and of 2 read it alike.
body, err := (snapshot.Facts{Format: writtenFormat, Taken: taken, Machines: []snapshot.Machine{{Name: "abcdef", Length: 6}}}).Encode()
if err != nil {
t.Fatal(err)
}
var raw struct {
Format int `json:"facts"`
Machines []map[string]json.RawMessage `json:"machines"`
}
if err := json.Unmarshal(body, &raw); err != nil {
t.Fatal(err)
}
if raw.Format != 1 {
t.Errorf("the snapshot says format %d", raw.Format)
}
for _, m := range raw.Machines {
if _, has := m["grants"]; has {
t.Error("a format 1 snapshot carries grants")
}
if _, has := m["bindings"]; has {
t.Error("a format 1 snapshot carries bindings")
}
}
}
+6 -7
View File
@@ -7,8 +7,8 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
snapshot "github.com/novox/mesh-controller/internal/facts"
"git.novox.be/novox/mesh-sdk/go/catalogue"
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
)
// The facts snapshot (novox/hq to-be 45 §9): composed from the store, every machine under a pseudonym
@@ -42,12 +42,8 @@ func aMeshWithSecrets(t *testing.T) (*stores, []string) {
t.Fatal(err)
}
// A push's worth of composition, which makes the pair credential the consumer is sent.
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
for _, node := range []string{"laptop", "anchor"} {
if _, _, err := composedAndValidated(ctx, open, node, gens, Allocating); err != nil {
if _, err := composeSend(ctx, open, node); err != nil {
t.Fatalf("%s does not compose: %v", node, err)
}
}
@@ -68,6 +64,9 @@ func TestTheFactsCarryNoSecretNoAddressAndNoName(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if f.Format != 1 {
t.Fatalf("the writer wrote format %d: it keeps format 1 until its own release moves it (issue 1390)", f.Format)
}
body, err := f.Encode()
if err != nil {
t.Fatal(err)

Some files were not shown because too many files have changed in this diff Show More