Compare commits

...
Author SHA1 Message Date
jschoubben 945906ac97 The assigner waits for a requirement nothing offers, and for a provision the provisioner keys whoever is placed yet; a requirement something offers (a module's own name, a module the mesh provides itself) is left to Awaited as before (ADR 0603 §5)
mesh/delivery ready: it is merged on the Approve
mesh/check pass 945906ac976c on c4470f42dba9 | unit 1186 run, 5555 kept, 27 skipped | integration 1205 run, 5717 kept, 35 skipped | over bound: unit (…
mesh/delivery-group group feat/l3b-bus ready: every member ready, and composed together they pass
2026-11-03 19:45:15 +01:00
jschoubben 1c66d58bce The assigner waits for a provision whose provider is not known yet: an unknown provider has provided nothing, so a consumer never moves ahead of a provider the indexer has not taken in (ADR 0603 §5); the integration test's write order is unordered again 2026-11-03 19:36:18 +01:00
jschoubben 34ef185ba5 The assigner's integration test puts the provider's module before the consumer's, so the consumer's requirement always has a provider to be judged against (the group check's flaky failure, 5 of 40 runs) 2026-11-03 19:33:43 +01:00
jschoubben f12823681c Merge the indexer's integration test: a field its judge does not know waits while the running build is unknown (ADR 0601 §3; the group check's failure)
mesh/check failed: integration: mesh-delivery-assigner/cmd/mesh-delivery-assigner.TestWantsAreWrittenAndNoDifferenceIs:… | f12823681c06 on c4470f42dba9
mesh/delivery-group group feat/l3b-bus rejected: composed together: the checker's verdict at the members' heads (62ca67fc6b58): integration: mesh-delivery-assi…
mesh/delivery closed: a newer head of the same pull request
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 18:51:14 +01:00
jschoubben b96a4f1d99 Pin mesh-sdk b7abe203a929, the group's head of feat/l3b-bus: it keeps, deprecated, the exported names the running parts still use
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 18:50:12 +01:00
jschoubben 70ac4ff79a The indexer's integration test expects a field from later to wait, not be refused, while no engine says which build runs (ADR 0601 §3)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 18:41:10 +01:00
jschoubben 51deed51e3 Merge ADR 0603 §10: the check waits on the builder's builds of its candidate before it composes, asks itself again when a failed build is done again, and names what it composes from manifests (the operator, 2026-11-03: build §10 in L3)
mesh/check checking 51deed51e3b4 on c4470f42dba9 | novox/mesh-delivery#147 in group feat/l3b-bus
mesh/delivery-group group feat/l3b-bus checking: 4 of 4 member(s) ready
mesh/delivery closed: a newer head of the same pull request
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:56:03 +01:00
jschoubben 3842a464ed A check failed on a build asks itself again once the build is done again, and a failed check never waits on builds (ADR 0603 §10)
A failed candidate build no longer needs mesh-delivery.recheck, an exception verb (ADR 0421 §12): the checker hears the
builder's keys the failed verdict names and, once each is written done (builder_build_again), retakes the check of the
same heads as it does after a failure of the nodes, at its start too. When the tests have already failed the check
composes at once and names the builds it did not wait on, so its red status is never held back by a build.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:55:41 +01:00
jschoubben 877b09e92b checker_status names the builds a running check waits on (ADR 0603 §10)
Worked out from the builder's builds state at each read, stored nowhere: of each running check, the candidate's builds
that are neither done nor failed.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:51:09 +01:00
jschoubben 453cdfa607 The check waits on the builder's builds of its candidate before it composes, so it composes with them (ADR 0603 §10)
The builder already builds every candidate fingerprint into its own key. The check now watches exactly those keys on the
bus and composes once each is done or failed, within the new setting build-wait-bound (1800 s). A failed build fails the
check, naming it; a build not ended within the bound is named and the check composes without it. A module built by a
build module the change alters (mesh-build-go or mesh-build-node), one standing on a base the change alters, and one the
trunk holds no record of are not waited on, and the composition's warnings name each with why.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:51:09 +01:00
jschoubben 8f7945a109 Merge round 2 finding C and the check's true words: the new rows fold into existing tests, the check names what it composes from manifests (ADR 0603 §10; test budget)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:36:49 +01:00
jschoubben 2773885600 Merge round 2 findings A and B: only the newest manifest's provisions hold a want, and a wait keeps its first start while it stands (ADR 0603 §8, §9)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:36:49 +01:00
jschoubben 45bd63427f A wait the watchman mirrors keeps its earliest start while the same mark stands, so a key written again with new words does not start the bound again (ADR 0603 §8) 2026-11-03 17:26:45 +01:00
jschoubben effa12c4b4 Fold the three new tests into existing ones (test budget); the base note says what the check does
The check builds no artifacts and no changed base before its dependents; build_change.go now says only that they are composed from their manifests.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:26:31 +01:00
jschoubben 9da14440e8 The assigner waits only on the keys of provisions the newest manifest still requires: an older edge's asked or retiring key holds nothing (ADR 0603 §9) 2026-11-03 17:26:30 +01:00
jschoubben fd638b99d5 Merge D9, D13b, D15 and review round 1 findings 3 and 4: the indexer runs without its graph state, a package inside a module's directory has its publisher (ADR 0603 §3, §4, §10; ADR 0604 §1)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:21:28 +01:00
jschoubben 0d1edef846 Merge D11, D12, D14: delivered over the nodes present, the watchman's waits with a bound, the newest provider edge (ADR 0603 §7, §8, §9; L3 review round 1, finding 2)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:21:28 +01:00
jschoubben 5c4d770aa1 The check names the modules built with a changed build module or base; rows for D9 and the assigner's integration test (ADR 0603 §3, §4, §10)
D13b: the composition's warnings, and so the check's line, name the changed modules built with the Go build module the change alters and those standing on a changed base. D9: builder and verdict rows. D15: the assigner's integration test gets a source revision changing a provider, its new consumer and a build module.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:21:07 +01:00
jschoubben b4855f21c3 Test rows: a missing graph state fails no recompute (review finding 3); a recipe change builds a Go module again (D9)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:21:07 +01:00
jschoubben 1387b10e78 Take a missing graph state as no graph yet and a Publisher's directory as the nearest module above it
The indexer's recompute and merge failed on listing a graph bucket not yet on the bus, and a package published from inside a module's directory had no publisher, so the dependents of a build module missed what imports its packages (review findings 3 and 4 of L3).

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 17:21:01 +01:00
jschoubben a1814c6571 Assigner row: the newest source revision's provider edge replaces the older one, and only it is waited on (ADR 0603 §9) 2026-11-03 17:19:05 +01:00
jschoubben 92b4307922 The watchman mirrors a want held by a rule, a provision asked and a build queued behind a package as waits with a bound, and raises a finding past it (ADR 0603 §8) 2026-11-03 17:18:36 +01:00
jschoubben 2c37f48f9d A source revision is delivered over the nodes present: the view leaves out a node the watchman says is away and names it, and the node counts once it runs its wants (ADR 0603 §7) 2026-11-03 17:16:30 +01:00
jschoubben 4a736858b6 Pin mesh-sdk cf5812101b0b, the group's head of feat/l3b-bus with the module graph taken out 2026-11-03 16:56:37 +01:00
jschoubben 3411b91571 The module graph is mesh-delivery's own: a Go module sdk/ with package graph, used by path by the indexer, the view and the checker, and the seat mesh-catalog's answers built from mesh-sdk's contract types (the operator, 2026-11-03: no graph-related code in mesh-sdk, only contracts; research 081 D11, D13) 2026-11-03 16:56:37 +01:00
jschoubben 0ce16ad773 Pin mesh-sdk 18b3eac47c8e, the group's head of feat/l3b-bus rebased onto main 2341e8e9 2026-11-03 16:46:40 +01:00
jschoubben 875d42fe3e On B1: the ledger test names its tool runner by its manifest's tool-runner, and ShelfWaiting takes B1's old-section counts (rebase onto main c4470f42) 2026-11-03 16:46:40 +01:00
jschoubben aacdba44e3 The judge's copy of the mesh-catalog seat is the SDK's entry: four optional verbs, so the old module's claim is not outgrown (ADR 0604 §1; F3) 2026-11-03 16:45:06 +01:00
jschoubben 39b3ce3c73 builder_builds_since names the operator's command nox builds as its reader, not mesh-catalog (ADR 0604) 2026-11-03 16:45:06 +01:00
jschoubben e1d0c9fe03 A merge that leaves a repository no module never deletes a module another trunk holds, and indexer_take_in takes a repository in (issue 1367)
dropRemoved hands the module over from the other trunk, read from the forge at the head the source names, so no wait in memory is needed. indexer_take_in with a repository takes in a head with no module.json as a merge that leaves none, and answers by name for a head with modules.
2026-11-03 16:45:06 +01:00
jschoubben 6bc16d74db Write and take in a merge that leaves no module.json, so a module-less repository's reach is not held for ever and a removed last module is dropped (issue 1126)
The forge bridge wrote no merged message for such a merge, so taken.<repo> never moved
and the view held 'the indexer has not yet taken in <head>' (hq #490-#492). The bridge
now writes it with no trees; the indexer drops every module of the repository on it,
after the same order check as any merged message. The view needed no change: with the
taken key moving it reaches the merge, which its new row proves.

Needs mesh-sdk fix/1126-no-module-reach (a merged message may carry no trees).
2026-11-03 16:45:06 +01:00
jschoubben 9f49bc92ad The seat mesh-catalog's four verbs answered from the indexer's modules, manifests and graph, with the old catalog's field names (captured answers as fixtures); served once the indexer claims the seat, in the release after L3's (ADR 0604 §1, §2) 2026-11-03 16:45:06 +01:00
jschoubben 4f6e1021c6 view_tiers: a source revision's tiers worked out from the module graph at each call, and per module what it waits on now, its build, each node's want and have and each provision it awaits, the nodes offline named away (ADR 0603 §6) 2026-11-03 16:45:06 +01:00
jschoubben 027c445bae Pin mesh-sdk 182d8757dba9, the group's head of feat/l3b-bus with the graph's ordering edges (ADR 0603 §1, §6) 2026-11-03 16:45:06 +01:00
jschoubben 7ba2668aa8 The check refuses a change whose module graph has a cycle and names it, worked out from the catalogue as the change leaves it; a cycle the trunk already has among the same modules is said, not the change's failure (ADR 0603 §2) 2026-11-03 16:45:06 +01:00
jschoubben a901b840a0 Pin mesh-sdk acad60547c84, and the indexer's graph nodes take their bases and provision edges from the contract's GraphEdgesOf (ADR 0603 §2) 2026-11-03 16:45:06 +01:00
jschoubben f70faed5b6 The indexer keeps the module graph as its state graph: each module's build and provision edges written at every recompute when they change, at most 2000 modules, a module's key deleted once the modules state no longer holds it (ADR 0603 §2) 2026-11-03 16:45:06 +01:00
jschoubben 5ee3b4c15f Pin mesh-sdk 25b371bc8397, the group's head of feat/l3b-bus with the module graph and the seat mesh-catalog's contract (ADR 0603 §2, ADR 0604 §1) 2026-11-03 16:44:47 +01:00
jschoubben 2f387a3f4c Pin mesh-sdk e7ce86ff62b5, the group's head of feat/l3b-bus (review round 2) 2026-11-03 16:44:47 +01:00
jschoubben 22aedbe97f A recompute with no merge writes the fingerprint's source revision as not known, so no release's Approve answers a build it never carried, though the source revision is still that release's (ADR 0601 §1; review round 2) 2026-11-03 16:44:47 +01:00
jschoubben 478f0110e9 Pin mesh-sdk 825483043549, the group's head of feat/l3b-bus (review round 1) 2026-11-03 16:44:47 +01:00
jschoubben 6d24922172 The node-engine's witness ping is granted by the tool runner's name its manifest gives, any service's instance on its own node while none says so, never a name in mesh-grants (ADR 0601 §4; review round 1, finding 4) 2026-11-03 16:44:47 +01:00
jschoubben 0fa00d5e6c The watchman raises what a node keeps of a module as it runs, for a manifest that does not yet say it, from the module's have, and clears it once the have says none (ADR 0601 §4; review round 1, finding 3) 2026-11-03 16:44:47 +01:00
jschoubben a41d2851c7 A release answers only the build whose fingerprint it moved: the indexer stamps the source revision at which a fingerprint last changed, a recompute with no merge the revision of its moment, and the assigner reads that stamp, never the Src a recompute keeps (ADR 0601 §1; review round 1, finding 1) 2026-11-03 16:44:47 +01:00
jschoubben ea5be10532 Pin mesh-sdk 09ec82cb3551, the group's head of feat/l3b-bus (ADR 0601) 2026-11-03 16:44:47 +01:00
jschoubben 6b9d211f72 The checker decides a node the sandbox did not judge by the contract's field (composer.Machine.NotJudged, verdict not-judged) and names it with the composer's words, never by a phrase in its reason (ADR 0593 §9; the prepare's seams finding on "not judged") 2026-11-03 16:44:47 +01:00
jschoubben b342c67b19 The grants part knows the tool runner by what its manifest says, never by its name, and keeps a node's tool runner user as the list in force holds it while no manifest there says it yet, said in the held condition (ADR 0601 §4; the prepare's seams finding on mesh-grants's own name) 2026-11-03 16:44:47 +01:00
jschoubben daf178c4ca A release's Approve answers the record ask of a move of the build alone to the build it carries: the assigner reads the release by the build's source revision from the gate's source and merges, and names it in the want; a move that changes a setting or a secret too is asked as before (ADR 0601 §1) 2026-11-03 16:44:37 +01:00
jschoubben a9aa6d29ea The assigner moves the bus module by its rules as any module, with no bar and no snapshot of its own, and a person may choose its policy as any module's: its node backs its streams up before it applies (ADR 0601 §1, §2) 2026-11-03 16:44:37 +01:00
jschoubben ecf96be8ce The judge's row for a field it does not know names a field mesh-sdk does not know now: core-credentials is one of its manifest words (ADR 0601 §4) 2026-11-03 16:44:37 +01:00
jschoubben ab3cb1ca9e The judge knows the tool runner by what its manifest says and keeps the kept streams itself, and this repository's check that no manifest names the ended controller reads them as text: mesh-sdk names no module (ADR 0601 §4) 2026-11-03 16:44:37 +01:00
jschoubben 054f089614 The bus has the upgrade policy its manifest says, as any module: the judge and the assigner rows read it from the manifest (ADR 0601 §1) 2026-11-03 16:44:37 +01:00
jschoubben 6301e49c73 Review round 2: a held module's object reads come from its running manifest; one with none keeps the list in force, said; indexer_judge asks the engine for the running build again (issue 774) 2026-11-03 16:44:37 +01:00
jschoubben 78fe1ce960 Review round 1: the indexer learns its build from the node-engine's seat; a held module's seats and hearings come from its running manifest; indexer_judge; one judge pass at a time (issue 774) 2026-11-03 16:44:37 +01:00
jschoubben 6645ac58c2 A manifest with a field the judge does not know waits instead of being refused; a held module keeps its bus users; tools list and redo the verdicts (ADR 0578 amended, issue 774) 2026-11-03 16:44:19 +01:00
mesh-tools c4470f42db Merge pull request 'Bus state by template: the judge's rules, manifest-accepted, the old-section count and the ledger account (issue 1966, B1)' (#146) from feat/bus-state into main 2026-11-03 15:24:36 +00:00
jschoubben 4bb790bedf Pin the group's mesh-sdk head f8bd12e in place of the local replace (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-delivery@c4470f42dba9
mesh/delivery-group group feat/bus-state merged: every member is merged
mesh/check pass 4bb790bedffe on 8f83608503da | unit 6460 run, 262 kept, 27 skipped | integration 6639 run, 264 kept, 35 skipped | over bound: unit, in…
2026-11-03 15:35:28 +01:00
jschoubben aa98729457 The zero rule takes the count of the accepted set the last pass held, and is not applied when it cannot be read (issue 1966 D4) 2026-11-03 15:28:58 +01:00
jschoubben 2fdb0516cd The checker's judge counts the old sections from the trunk case as the indexer does, so a change adding a section at 0 is refused at the merge check (issue 1966 D4) 2026-11-03 15:14:28 +01:00
jschoubben 28a6858863 The ledger account's subjects come from mesh-sdk's one naming of them (issue 1966 D5) 2026-11-03 15:13:38 +01:00
jschoubben 1ab0274547 The wanted accounts include the ledger kind, held in the tool runner's ledgers, in step with mesh-sdk's buscompose (issue 1966 D5b) 2026-11-03 13:39:21 +01:00
jschoubben 07af86b153 The grants part composes a ledger account per assignment of a module asking for a ledger, publishing only on its ledger histories and snapshots (issue 1966 D5) 2026-11-03 13:38:20 +01:00
jschoubben dd2e7c8e64 The indexer counts the modules on each old section, a tool reads it, and the judge refuses a section at 0 outside the foundation (issue 1966 D4) 2026-11-03 13:34:54 +01:00
jschoubben 756a3813e2 The indexer says manifest-accepted, with the bus-state entries whole, for a newly accepted verdict (issue 1966 D3) 2026-11-03 13:32:41 +01:00
jschoubben 9a508701ff One local name is one form, and the old state says what it accepts: 0 is taken as 1 (issue 1966 D2) 2026-11-03 13:31:42 +01:00
jschoubben 7ee7248fad The judge refuses bus state in requires that breaks a template's bounds, naming the entry and the field (issue 1966 D1) 2026-11-03 13:30:42 +01:00
144 changed files with 6109 additions and 419 deletions
@@ -27,6 +27,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/health"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/provisions"
"mesh-delivery-assigner/internal/judge"
"mesh-delivery-assigner/internal/assign"
@@ -434,6 +435,61 @@ func TestWantsAreWrittenAndNoDifferenceIs(t *testing.T) {
if !self {
t.Error("ace's own want does not say it composes its own once placed")
}
// ADR 0603 (L3, D15): one source revision changes a provider (db), its new consumer (app) and a build module (tool, whose
// recipe moved: ace runs its first build and the second, under the agreed toolchain, is the one the source revision names).
// The keys are written as their writers write them: the indexer's modules and manifests, the builder's builds, the
// controller's grants, the placement seat's placement and the provisioner's provisions. The build module's want moves at
// once and the provider's with it; the consumer's waits for its provision, with the words naming it, and moves once the
// provisioner provides it (rule 8, ADR 0570 §4): nothing waits for a turn or an order, only for a key.
offer := `"provides":[{"name":"database","scope":"mesh","answered-by":"helper"}]`
dbDigest := b.manifest(t, "db", `{"module":"db","version":"1",`+offer+`}`)
appDigest := b.manifest(t, "app", `{"module":"app","version":"1","requires":["database"]}`)
toolDigest := b.manifest(t, "tool", `{"module":"tool","version":"1"}`)
asked := func(state, why string) {
raw, err := provisions.Encode(provisions.Provision{V: provisions.Version, Provision: "database", Consumer: provisions.End{Module: "app", Node: "ace"},
Provider: provisions.End{Module: "db", Node: "ace"}, State: state, Why: why, At: time.Now()})
if err != nil {
t.Fatal(err)
}
b.putJSON(t, provisions.ProvisionsBucket, "ace.app.database", raw)
}
// The provisioner keys it from the manifests, as soon as they are known and whether the provider is placed yet or not.
asked(provisions.Asked, "the provider's build is not done")
fpOf := func(c byte) string { return delivery.SourcePrefix2 + strings.Repeat(string(c), 64) }
toolOld := fpOf('1')
for module, row := range map[string]struct {
fp, digest string
}{"db": {fpOf('2'), dbDigest}, "app": {fpOf('3'), appDigest}, "tool": {fpOf('4'), toolDigest}} {
mod, _ := delivery.EncodeModule(delivery.Module{FP2: row.fp, Manifest: row.digest, Repository: "novox/" + module, Commit: strings.Repeat("2", 40), Path: delivery.RootPath})
b.putJSON(t, delivery.ModulesBucket, module, mod)
fps := []string{row.fp}
if module == "tool" {
fps = append(fps, toolOld)
}
for _, fp := range fps {
built, _ := delivery.EncodeBuilderBuild(delivery.BuilderBuild{State: delivery.BuildDone, By: "builder.novox.1", Step: "pushed", Manifest: row.digest})
bkey, _ := delivery.BuilderBuildKey(module, fp)
b.putJSON(t, delivery.BuilderBuildsBucket, bkey, built)
gkey, _ := delivery.PartGrantKey(module, fp)
grant, _ := delivery.EncodeGrant(delivery.Grant{})
b.putJSON(t, delivery.PartGrantsBucket, gkey, grant)
}
b.place(t, "ace", module)
}
runsOld, _ := delivery.EncodeHave(delivery.Have{FP: toolOld, Phase: delivery.PhaseRunning, Health: delivery.HealthOK})
b.putJSON(t, delivery.HaveBucket, "ace.tool", runsOld)
if w := b.wantNow(t, "ace", "tool", "the build module's want at the second build", func(w delivery.Want) bool { return w.FP == fpOf('4') }); w.FP != fpOf('4') {
t.Errorf("tool's want = %+v", w)
}
b.wantNow(t, "ace", "db", "the provider's want at its fingerprint", func(w delivery.Want) bool { return w.FP == fpOf('2') })
if e, err := b.buckets[delivery.AssignerWantsBucket].Get(b.ctx, "want.ace.app"); err == nil {
t.Errorf("app was wanted while its provision was asked: %s", e.Value())
}
asked(provisions.Provided, "")
if w := b.wantNow(t, "ace", "app", "the consumer's want once its provision is provided", func(w delivery.Want) bool { return w.FP == fpOf('3') }); w.FP != fpOf('3') {
t.Errorf("app's want = %+v", w)
}
}
// 1297/B9 (the assigner's half, ADR 0528 clause 6): started whole, the assigner holds a fingerprint whose health check
@@ -60,6 +60,8 @@ var readBuckets = []string{
delivery.PresenceBucket,
delivery.HaveBucket,
provisions.ProvisionsBucket,
delivery.SourceBucket,
delivery.MergesBucket,
}
// readFamilies are the states kept one bucket per node the assigner reads, each named in its manifest's reads: every
@@ -140,6 +142,12 @@ func open(ctx context.Context, js process.BucketOpener, logf func(string, ...any
logf("the bucket %s cannot be opened (%v): the assigner's conditions are said straight to the router until it starts again", name, err)
continue
}
if (name == delivery.SourceBucket || name == delivery.MergesBucket) && err != nil {
// The gate's source and merges (ADR 0601 §1): without them no release is known to carry a build, so every move of
// a record module is asked of a person, as before.
logf("the bucket %s cannot be opened (%v): no release's Approve answers a record ask until it is bound", name, err)
continue
}
if name == delivery.PartGrantsBucket && err != nil {
// The grants part's grants (ADR 0571 clause 9): without them no module counts as granted until it is bound.
logf("the bucket %s cannot be opened (%v): no grant is read until it is bound once the bus says it was made", name, err)
@@ -158,7 +166,7 @@ func open(ctx context.Context, js process.BucketOpener, logf func(string, ...any
func startsWithout(name string) bool {
switch name {
case delivery.SeatHoldersBucket, provisions.ProvisionsBucket, assign.PoliciesBucket, assign.FindingsBucket,
delivery.AssignerPutsBucket:
delivery.AssignerPutsBucket, delivery.SourceBucket, delivery.MergesBucket:
return true
}
return slices.Contains(seatBuckets, name) || name == delivery.PartGrantsBucket
@@ -241,6 +249,8 @@ var watched = []string{
assign.PoliciesBucket,
provisions.ProvisionsBucket,
delivery.SeatHoldersBucket,
delivery.SourceBucket,
delivery.MergesBucket,
}
// follow watches every bucket a pass reads and makes a pass at start, once every watch handed what stands, and after
@@ -336,8 +346,9 @@ func follow(ctx context.Context, js jetstream.JetStream, nc process.Subscriber,
}
started++
var opts []jetstream.WatchOpt
if name == delivery.HaveBucket {
opts = append(opts, jetstream.IncludeHistory()) // what each node ran healthy before, for a set-back
if name == delivery.HaveBucket || name == delivery.SourceBucket {
// what each node ran healthy before, for a set-back; and the release behind each source revision (ADR 0601 §1)
opts = append(opts, jetstream.IncludeHistory())
}
if err := watch(name, opts...); err != nil {
return err
@@ -158,7 +158,7 @@ func (p *passer) wants(ctx context.Context) {
for name := range names {
fp := trunk[name].FP2
name := name
m := assign.Module{Name: name, FP: fp, Manifest: manifests[name], Chosen: chosenOf(chosen, name), Provisions: provisionsOf(name), Answering: answering(manifests, nodes), Nodes: nodesOf[name], Now: time.Now(), Puts: puts[name],
m := assign.Module{Name: name, FP: fp, Manifest: manifests[name], Chosen: chosenOf(chosen, name), Provisions: provisionsOf(name), Answering: answering(manifests, nodes), Offered: offered(manifests), Nodes: nodesOf[name], Now: time.Now(), Puts: puts[name],
Self: func(node string) bool { return placed[node] },
Composed: func(node, fp string) (delivery.ComposedDigest, bool) {
return p.w.composed(node, name, fp) // a placed node's own: the digest its own have names (ADR 0571 §6)
@@ -174,7 +174,8 @@ func (p *passer) wants(ctx context.Context) {
Allowed: func(node, digest string) bool {
key, err := delivery.RecordAskKey(node, name, digest)
return err == nil && p.answeredWith(key, assign.OptionYes)
}}
},
Release: func(fp string) (string, bool) { return p.w.release(name, fp) }}
if fp != "" {
held, _ := delivery.HeldKey(name, fp)
grant, _ := delivery.PartGrantKey(name, fp)
@@ -819,7 +820,9 @@ func (p *passer) backupAsk(ctx context.Context, key, node, module string, want d
// yeses writes a placed node's wants naming the build alone, and sees to the moves a node that composes its own judged a
// person must allow (ADR 0547 §3): for a have of phase
// waiting-yes it raises the record ask once, naming the composition waiting and the node's judgement, and on a yes writes
// the want again naming it; a yes once written is kept while the want stays at its build, so the node is not asked again.
// the want again naming it; a yes once written is kept while the want stays at its build, so the node is not asked again. A
// move of the build alone (the have says so) to a build an approved release carries is given its yes by that Approve, the
// want naming the release, with no ask (ADR 0601 §1).
func (p *passer) yeses(ctx context.Context, module string, m assign.Module, before map[string]delivery.Want, out *assign.Outcome) {
for _, node := range m.Nodes {
if m.Self == nil || !m.Self(node) {
@@ -836,25 +839,37 @@ func (p *passer) yeses(ctx context.Context, module string, m assign.Module, befo
if w.Removed {
continue
}
yes := ""
yes, release := "", ""
if had && was.FP == w.FP {
yes = was.Yes
yes, release = was.Yes, was.Release
}
key, err := delivery.HaveKey(node, module)
if err != nil {
continue
}
if h, err := delivery.DecodeHave(p.w.value(delivery.HaveBucket, key)); err == nil && h.Phase == delivery.PhaseWaitingYes && h.Waits != yes {
if m.Allowed != nil && m.Allowed(node, h.Waits) {
yes = h.Waits
} else {
switch {
case m.Allowed != nil && m.Allowed(node, h.Waits):
yes, release = h.Waits, ""
case h.BuildAlone && h.FP != w.FP && m.Release != nil:
// A move of the build alone, to the build an approved release carries: that Approve is the yes (ADR 0601 §1),
// named in the want. A move that changes a setting or a secret too, or one the node could not tell, is asked.
if r, ok := m.Release(w.FP); ok {
yes, release = h.Waits, r
break
}
p.recordAsk(ctx, module, w.FP, assign.RecordNeed{Node: node, Digest: h.Waits, Judgement: h.Judgement, Words: h.Detail})
default:
p.recordAsk(ctx, module, w.FP, assign.RecordNeed{Node: node, Digest: h.Waits, Judgement: h.Judgement, Words: h.Detail})
}
}
// A want carried over from before the node composed its own may still name the controller's composition; a placed
// node's want names the build alone (ADR 0547 §2).
if yes != w.Yes || w.Composed != "" || !w.Self {
w.Yes, w.Composed, w.Self = yes, "", true
if yes == "" {
release = ""
}
if yes != w.Yes || release != w.Release || w.Composed != "" || !w.Self {
w.Yes, w.Release, w.Composed, w.Self = yes, release, "", true
out.Wants[node] = w
}
}
@@ -1071,6 +1086,26 @@ func (p *passer) seatHolders(ctx context.Context) {
}
}
// offered says whether some manifest of the trunk, or a module the mesh provides itself, offers a provision under its name
// (a module offers its own name too, catalogue.Manifest.Offers): what a requirement nobody offers is told from.
func offered(manifests map[string]catalogue.Manifest) func(string) bool {
names := map[string]bool{}
add := func(m catalogue.Manifest) {
for _, o := range m.Offers() {
names[o] = true
}
}
for _, m := range manifests {
add(m)
}
for _, module := range delivery.ProvidedModules() {
if m, _, err := delivery.ProvidedManifest(module); err == nil {
add(m)
}
}
return func(provision string) bool { return names[provision] }
}
// answering is, by provision, the manifests of the modules placed on some node that provide it: what a want's
// provisions are judged by (catalogue.Awaited, issue 774). A provider in the catalogue placed nowhere answers nobody.
func answering(manifests map[string]catalogue.Manifest, n assign.Nodes) func(string) []catalogue.Manifest {
@@ -111,6 +111,14 @@ func TestAHoldAskedAgainAfterALiftAndARestartHasANewID(t *testing.T) {
// 1906/H4: an answer counts only for the asking it answered: an answer to an earlier hold of the same fingerprint, or to an ask
// taken away, is not taken for the hold that stands now.
func TestAnAnswerCountsOnlyForTheAskingItAnswered(t *testing.T) {
// ADR 0603 §5: a provision is offered by a manifest's provides, by a module's own name, and by a module the mesh provides
// itself (mesh-addressing, mesh-wireguard's); nothing else.
off := offered(map[string]catalogue.Manifest{"node-tools": {Module: "node-tools"}, "db": {Module: "db", Provides: []catalogue.Offer{{Name: "database"}}}})
for provision, want := range map[string]bool{"node-tools": true, "database": true, "mesh-addressing": true, "queue": false} {
if off(provision) != want {
t.Errorf("offered(%s) = %v, want %v", provision, !want, want)
}
}
key, _ := delivery.HoldAskKey("web", fpTest)
p := newPasser(buckets{}, time.Now())
put := func(asking int) string {
@@ -414,10 +422,11 @@ func TestAModuleOnNeitherListIsRaisedOnceAndClearedOnce(t *testing.T) {
if last := logged[len(logged)-1]; last != "screen-lock on g14 has a want again" {
t.Errorf("the clear said %q", last)
}
// 1945 (the bus bar): a module that leaves neither list with no want is cleared in words saying so, never "has a want again".
bus := map[string]neitherOne{"novox.nats": {node: "novox", module: "nats", holding: assign.Holding{Bus: true, Words: "the bus moves only by the operator's put"}}}
// 1945: a module that leaves neither list with no want is cleared in words saying so, never "has a want again". The bus is
// on neither list by a rule as any module (ADR 0601 §1), nothing for the operator to do.
bus := map[string]neitherOne{"novox.nats": {node: "novox", module: "nats", holding: assign.Holding{Rule: delivery.RuleGrant, Words: "no credential"}}}
p.sayNeither(bus, nil, nil)
if c := said[len(said)-1]; c.State != asks.StateRaised || !strings.HasPrefix(c.Explanation, asks.NeedsYou) || !strings.Contains(c.Explanation, "the bus moves only by the operator's put") {
if c := said[len(said)-1]; c.State != asks.StateRaised || !strings.HasPrefix(c.Explanation, asks.NothingToDo) || !strings.Contains(c.Explanation, "rule 6 holds its first want back: no credential") {
t.Errorf("the bus's neither condition = %+v", c)
}
p.sayNeither(map[string]neitherOne{}, nil, map[string]string{"novox.nats": "it is held, and its held condition says so"})
@@ -474,7 +483,6 @@ func testThePassGathersTheModulesOnNeitherList(t *testing.T) {
{nil, assign.Outcome{Rules: map[string]assign.Holding{"shanks": {Rule: delivery.RuleGrant, Words: "no credential"}}}, "rofi on shanks gets no want: rule 6: no credential"},
{nil, wanted, "rofi on shanks has a want"},
{map[string]delivery.Want{"shanks": wanted.Wants["shanks"]}, assign.Outcome{}, ""},
{nil, assign.Outcome{Rules: map[string]assign.Holding{"shanks": {Bus: true, Words: "the bus moves only by the operator's put"}}}, "rofi on shanks gets no want: the bus moves only by the operator's put"},
} {
logged = nil
p.sayNoWant("rofi", []string{"shanks"}, c.before, c.out)
@@ -788,6 +796,74 @@ func mustKey(key string, err error) string {
func TestARecordAskOfAHeldBackNodeSurvivesPassesAndItsYesMovesTheWant(t *testing.T) {
// A placed node's yes comes from its have (issue 1126): folded here to hold the count.
aPlacedNodesWantsNameTheBuildAloneAndItsYesComesFromItsHave(t)
aReleasesApproveAnswersTheMoveOfItsBuildAlone(t)
}
// ADR 0601 §1: a placed node's have waiting for a yes to a move of the build alone, to a build an approved release carries, is
// given its yes by that Approve, the want naming the release, and no record ask is raised; a move that changes a setting or a
// secret too, one at the build it runs, and one no approved release carries are asked of a person as before.
func aReleasesApproveAnswersTheMoveOfItsBuildAlone(t *testing.T) {
fpOld := delivery.SourcePrefix2 + strings.Repeat("a", 64)
waiting, was := "sha256:"+strings.Repeat("e", 64), "sha256:"+strings.Repeat("d", 64)
commit := strings.Repeat("1", 40)
for _, c := range []struct {
name string
alone bool
at string // the fingerprint the have runs
mergedAt string // the commit the release merged web at; "" for no merge of the release
fpSrc uint64 // the source revision at which web's fingerprint last changed
yes, asked bool
}{
{"the build alone, its release approved", true, fpOld, commit, 7, true, false},
{"a setting changed too", false, fpOld, commit, 7, false, true},
{"a setting changed at the build it runs", true, fpTest, commit, 7, false, true},
{"no approved release carries it", true, fpOld, "", 7, false, true},
{"the release merged another commit", true, fpOld, strings.Repeat("2", 40), 7, false, true},
// Its fingerprint last moved at another release's revision, which merged nothing of web: that Approve answers nothing.
{"a fingerprint another release moved", true, fpOld, commit, 8, false, true},
// Review rounds 1 and 2: moved by a recompute after the release (a toolchain, a package, a base) with no merge, the
// indexer writes fp-src 0, not known, though Src is still the release's: asked of a person.
{"a fingerprint moved by a recompute after the release", true, fpOld, commit, 0, false, true},
} {
asksKV := &memKV{values: map[string][]byte{}}
p := newPasser(buckets{delivery.AssignerAsksBucket: asksKV}, time.Now())
p.logf = t.Logf
mod, _ := delivery.EncodeModule(delivery.Module{FP2: fpTest, Repository: "novox/web", Commit: commit, Src: 7, FPSrc: c.fpSrc, Path: delivery.RootPath})
p.w.put(delivery.ModulesBucket, "web", mod)
src, _ := delivery.EncodeSource(delivery.Source{Group: "feat/web", Heads: map[string]string{"novox/web": commit}})
p.w.putEntry(delivery.SourceBucket, delivery.SourceMainKey, src, 7, time.Now())
later, _ := delivery.EncodeSource(delivery.Source{Group: "feat/other", Heads: map[string]string{"novox/web": commit, "novox/x": commit}})
p.w.putEntry(delivery.SourceBucket, delivery.SourceMainKey, later, 8, time.Now()) // a later merge leaves revision 7's release
if c.mergedAt != "" {
merge, err := delivery.EncodeMerge(delivery.Merge{Group: "feat/web", Ask: "merge.feat_web.0123456789ab", Pulls: []delivery.MergePull{
{Head: delivery.Head{Repository: "novox/web", Number: 3, Head: strings.Repeat("3", 40)}, State: delivery.MergeMerged, Commit: c.mergedAt}}})
if err != nil {
t.Fatal(err)
}
p.w.put(delivery.MergesBucket, "feat/web", merge)
other, err := delivery.EncodeMerge(delivery.Merge{Group: "feat/other", Ask: "merge.feat_other.0123456789ab", Pulls: []delivery.MergePull{
{Head: delivery.Head{Repository: "novox/x", Number: 4, Head: strings.Repeat("4", 40)}, State: delivery.MergeMerged, Commit: commit}}})
if err != nil {
t.Fatal(err)
}
p.w.put(delivery.MergesBucket, "feat/other", other)
}
have, _ := delivery.EncodeHave(delivery.Have{FP: c.at, Composed: was, Phase: delivery.PhaseWaitingYes, Health: delivery.HealthOK,
Judgement: delivery.JudgedRecreates, Waits: waiting, Detail: "it recreates web.server", BuildAlone: c.alone})
p.w.put(delivery.HaveBucket, "g14.web", have)
m := assign.Module{Name: "web", FP: fpTest, Nodes: []string{"g14"}, Self: func(string) bool { return true },
Release: func(fp string) (string, bool) { return p.w.release("web", fp) }}
out := assign.Outcome{Wants: map[string]delivery.Want{"g14": {FP: fpTest, Self: true}}}
p.yeses(context.Background(), "web", m, map[string]delivery.Want{"g14": {FP: fpOld, Self: true}}, &out)
w := out.Wants["g14"]
if got := w.Yes == waiting && w.Release == "feat/web"; got != c.yes || (!c.yes && (w.Yes != "" || w.Release != "")) {
t.Errorf("%s: the want is %+v", c.name, w)
}
akey, _ := delivery.RecordAskKey("g14", "web", waiting)
if _, there := asksKV.values[akey]; there != c.asked {
t.Errorf("%s: a record ask raised %v, want %v", c.name, there, c.asked)
}
}
}
// 774/S26/AC1 (ADR 0549 §5, ADR 0567 §8): the seat's placement alone names where modules run: no controller state is read, so a
@@ -385,7 +385,7 @@ func wantsVerbs(t *testing.T) {
}
manifest("web", "")
manifest("photos", `,"upgrade":{"policy":"record","why":"a person checks the albums"}`)
manifest("nats", `,"provides":[{"name":"mesh-bus","scope":"mesh"}]`)
manifest("nats", `,"provides":[{"name":"mesh-bus","scope":"mesh"}],"upgrade":{"policy":"record","why":"its streams"}`)
for node, fp := range map[string]string{"anchor": fpOld, "laptop": fpNew} {
st[haveState].values[node+".nats"], _ = delivery.EncodeHave(delivery.Have{FP: fp, Phase: delivery.PhaseRunning, Health: delivery.HealthOK})
}
@@ -406,7 +406,7 @@ func wantsVerbs(t *testing.T) {
if p := policyOf("photos"); p.Policy != catalogue.PolicyRecord || p.From != catalogue.FromModule {
t.Errorf("photos = %+v", p)
}
if p := policyOf("nats"); p.Policy != catalogue.PolicyRecord || p.From != catalogue.FromBus {
if p := policyOf("nats"); p.Policy != catalogue.PolicyRecord || p.From != catalogue.FromModule { // the policy its manifest says (issue 774)
t.Errorf("nats = %+v", p)
}
// compare: each node's own composition is the digest its have names, and the assigner's findings; nothing splits, the
@@ -471,8 +471,12 @@ func wantsVerbs(t *testing.T) {
len(said) != 1 || !strings.HasPrefix(said[0], "policy-changed policy-changed.web.") {
t.Errorf("chosen %+v, asked %v, said %v", c, asked, said)
}
if _, err := w.doUpgrade(map[string]any{"module": "nats", "policy": "roll", "why": "x"}, "op"); err == nil {
t.Error("the bus was given a policy")
// A person may choose the bus's policy as any module's (ADR 0601 §1: no part treats the module that provides the bus apart).
if _, err := w.doUpgrade(map[string]any{"module": "nats", "policy": "roll", "why": "x"}, "op"); err != nil {
t.Errorf("a person's policy for the bus was refused: %v", err)
}
if _, err := w.doUpgrade(map[string]any{"module": "nats", "policy": "default", "why": "back to the module"}, "op"); err != nil {
t.Fatal(err)
}
if _, err := w.doUpgrade(map[string]any{"module": "web", "policy": "roll"}, "op"); err == nil {
t.Error("a policy without why was set")
@@ -197,18 +197,14 @@ func (s *wantsTools) doUpgrade(args map[string]any, by string) (any, error) {
}
// setPolicy writes a person's choice of a module's policy, or deletes it for default, records the act in the hand-act log
// and says policy-changed. The bus is never rolled: a choice for it is refused.
// and says policy-changed. The bus's choice is a person's as any module's (ADR 0601 §1).
func (s *wantsTools) setPolicy(module, policy, why, by string, manifests map[string]catalogue.Manifest) (any, error) {
if module == "" {
return nil, errors.New("a policy is set for one module: name it")
}
m, ok := manifests[module]
if !ok {
if _, ok := manifests[module]; !ok {
return nil, fmt.Errorf("the indexer holds no manifest of %s", module)
}
if catalogue.ProvidesBus(m) {
return nil, fmt.Errorf("%s is the bus, which is never rolled whoever says so (novox/hq ADR 0236): its step is the put on the operator's warrant", module)
}
if why == "" {
return nil, errors.New("a person's policy says why")
}
@@ -37,6 +37,9 @@ type world struct {
// placement is the seat's state placement assembled node by node from its keys in either form (mesh-sdk
// delivery.PlacementKeys; research 072 row 37): the one reader of its keys in the assigner.
placement *delivery.PlacementKeys
// sourceGroups is, by revision of the gate's source.main, the release (group) that moved it, from every revision its
// history keeps (watched with history): a build's source revision names the release that carries it (ADR 0601 §1).
sourceGroups map[uint64]string
}
// entryMeta is a value's revision in its bucket and when it was written.
@@ -54,7 +57,8 @@ type parsed struct {
func newWorld() *world {
return &world{values: map[string]map[string][]byte{}, manifests: map[string]parsed{}, healthy: map[string][]string{}, changes: map[string]uint64{},
meta: map[string]entryMeta{}, trunkBuilt: map[string]delivery.BuildLogEntry{}, placement: delivery.NewPlacementKeys()}
meta: map[string]entryMeta{}, trunkBuilt: map[string]delivery.BuildLogEntry{}, placement: delivery.NewPlacementKeys(),
sourceGroups: map[uint64]string{}}
}
// putEntry is put, keeping the value's revision and the time it was written.
@@ -70,6 +74,44 @@ func (w *world) putEntry(bucket, key string, value []byte, revision uint64, at t
if bucket == delivery.PlacementBucket {
_, _ = w.placement.Put(key, value, revision, false) // put took it already; this is its revision
}
if bucket == delivery.SourceBucket && key == delivery.SourceMainKey {
if s, err := delivery.DecodeSource(value); err == nil && s.Group != "" {
w.sourceGroups[revision] = s.Group
}
}
}
// release is the release whose Approve carries module's build fp (ADR 0601 §1), and whether one does: the trunk's value of
// the module names fp, the source revision at which that fingerprint last changed (FPSrc, never the Src of its last merge,
// which a recompute with no merge keeps) names the release that moved it, and the gate's merge of that release, which it
// writes on the operator's Approve, holds the module's own trunk merge among its merged pull requests. Any of them
// unknown: none, and the move is asked of a person.
func (w *world) release(module, fp string) (string, bool) {
t, ok := w.trunk()[module]
if !ok || fp == "" || t.FP2 != fp || t.FPSrc == 0 || t.Commit == "" {
return "", false
}
w.mu.Lock()
group := w.sourceGroups[t.FPSrc]
w.mu.Unlock()
key, err := delivery.MergesKey(group)
if group == "" || err != nil {
return "", false
}
raw := w.value(delivery.MergesBucket, key)
if raw == nil {
return "", false
}
merge, err := delivery.DecodeMerge(raw)
if err != nil {
return "", false
}
for _, pull := range merge.Pulls {
if pull.State == delivery.MergeMerged && pull.Commit == t.Commit && delivery.SameRepository(pull.Repository, t.Repository) {
return group, true
}
}
return "", false
}
// metaOf is the revision and time of the value a bucket holds at key, as the watch said them.
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-assigner
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929 h1:RtAFbUJRP1qO9AIMM2ue+cXNzqj7WZ7IOlSqNvttwuU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -119,14 +119,6 @@ func NeitherCondition(node, module string, h Holding, raised bool) asks.Conditio
if n := len([]rune(headline)); n > asks.HeadlineLength {
headline = string([]rune(headline)[:asks.HeadlineLength])
}
if h.Bus {
needs := fmt.Sprintf("put %s on %s with assigner_put", module, node)
return asks.Condition{Key: key, Severity: asks.SeverityUrgent, Subject: subject, State: asks.StateRaised,
Summary: headline, Headline: headline, Needs: needs,
Explanation: fmt.Sprintf("%s%s. %s is assigned to %s, but the node holds no want for it and the node's own composition does not "+
"hold it either, so nothing applies it there. The assigner never moves the bus: %s. This clears once the node has a want for it.",
asks.NeedsYou, needs, module, node, h.Words)}
}
return asks.Condition{Key: key, Severity: asks.SeverityUrgent, Subject: subject, State: asks.StateRaised,
Summary: headline, Headline: headline,
Explanation: fmt.Sprintf("%s %s is assigned to %s, but the node holds no want for it and the node's own composition does not "+
@@ -16,7 +16,7 @@ import (
// A person's upgrade policy is the assigner's (novox/hq ADR 0565 §7, §8, issue 1953 D4, D5): kept in its state `policies`, one
// key per module, set and taken back through `mesh-wants.upgrade`. Every reader takes one derivation, mesh-sdk's
// catalogue.Upgrade: a person's choice over the module's word, its irreplaceable data and the default, the bus never rolled.
// catalogue.Upgrade: a person's choice over the module's word, its irreplaceable data and the default, the bus as any module (ADR 0601 §1).
// PoliciesBucket is the assigner's state `policies`.
var PoliciesBucket = delivery.AssignerModule + "_" + delivery.AssignerPoliciesState
@@ -36,6 +36,10 @@ type Module struct {
// Answering is the manifests of the modules placed somewhere that provide a provision: what catalogue.Awaited
// judges a want's provisions by (issue 774). Nil: none known, and no key is waited for that is not there.
Answering func(provision string) []catalogue.Manifest
// Offered says whether any module offers a provision by name or by its own name (catalogue.Manifest.Offers) among every
// manifest the trunk holds and the modules the mesh provides itself, placed or not. Nil: not known, and no requirement
// is waited on for want of an offer.
Offered func(provision string) bool
// Chosen is a person's choice of the module's upgrade policy (the assigner's `policies`, ADR 0565 §7); nil: none.
Chosen *catalogue.PersonPolicy
Nodes []string
@@ -45,6 +49,10 @@ type Module struct {
// Allowed says whether the operator answered yes to the record ask about the move of this module on node to the composition
// with this digest (ADR 0539 §14). Nil: none did.
Allowed func(node, digest string) bool
// Release names the release whose Approve carries the build fp, as mesh-delivery's gate holds it for the release's heads,
// read by the build's source revision (ADR 0601 §1): that Approve is the yes to a move of the build alone. False when no
// approved release is known to carry it. Nil: none is read.
Release func(fp string) (string, bool)
// Released is a hold on FP the operator released (ADR 0539 §19): FP is not held again by a health check that failed.
Released bool
// Restored says whether the operator answered "restore" to the restore ask of this data module on node: it is then set back
@@ -149,8 +157,36 @@ func (m Module) provisionWaiting(node string) (string, bool) {
return fmt.Sprintf("provision %s has no key yet for %s on %s", missing[0], m.Name, node), true
}
keyed := m.Provisions(node)
// Only a provision the newest manifest still requires is waited on (ADR 0603 §9): a key of an older revision's edge,
// asked or retiring for the operator's answer (ADR 0570 §5), holds nothing. A requirement catalogue.Awaited leaves out is
// not waited on when something offers it (a module's own name, a module the mesh provides itself, a provision the
// composition does not key: optional, a toolchain on the module's own machine, a login the controller makes). One that
// nothing offers is waited on (ADR 0603 §5: an unknown provider has provided nothing), so a consumer never moves ahead of
// a provider whose module the indexer has not yet taken in; it holds loudly, since the watchman raises the hold past its
// bound (ADR 0603 §8).
needed := m.Manifest.Requires
if m.Answering != nil {
needed = catalogue.Awaited(m.Manifest, m.Answering)
have := map[string]bool{}
for _, p := range keyed {
have[p.Provision] = true
}
for _, r := range slices.Sorted(slices.Values(m.Manifest.Requires)) {
if have[r] && !m.Manifest.OptionalRequires[r] && !slices.Contains(needed, r) {
needed = append(needed, r) // the provisioner keys it from the manifests, whoever is placed yet
continue
}
if m.Offered == nil || m.Manifest.OptionalRequires[r] || len(m.Answering(r)) > 0 || m.Offered(r) || slices.Contains(needed, r) {
continue
}
needed = append(needed, r)
if !have[r] {
return fmt.Sprintf("no module offers %s yet for %s on %s: an unknown provider has provided nothing, so the want waits for one, and the watchman raises it past its bound", r, m.Name, node), true
}
}
}
for _, key := range slices.Sorted(maps.Keys(keyed)) {
if p := keyed[key]; p.State != provisions.Provided {
if p := keyed[key]; p.State != provisions.Provided && slices.Contains(needed, p.Provision) {
words := fmt.Sprintf("the provision %s of %s on %s is %s, not provided", key, m.Name, node, p.State)
if p.Why != "" {
words += ": " + p.Why
@@ -268,29 +304,18 @@ type RecordNeed struct {
Words string
}
// Holding is a rule holding a node back, and why. Bus marks the bar on the bus instead of one of the seven rules (Rule is then
// 0): the bus moves only by the operator's put (ADR 0539 §15, ADR 0546 §8), never by the assigner (issue 1945: nats on novox,
// running an old-form build, got no want and the pass said nothing).
// Holding is a rule holding a node back, and why. The bus module is held back by the rules alone, as every module (ADR 0601 §1,
// §2): no bar of its own.
type Holding struct {
Rule int
Words string
Bus bool
}
// Reason is the holding in words: the rule and its words, or for the bus bar its words alone.
// Reason is the holding in words: the rule and its words.
func (h Holding) Reason() string {
if h.Bus {
return h.Words
}
return fmt.Sprintf("rule %d: %s", h.Rule, h.Words)
}
// BusBarWords say why the assigner writes no want moving the bus on node: it runs prev, the trunk builds fp, and only the
// operator's put moves it (ADR 0539 §15, ADR 0546 §8). The words reach the operator, so they name no record.
func BusBarWords(node, prev, fp string) string {
return fmt.Sprintf("only your put moves the bus: %s runs %s, the trunk builds %s; put it with assigner_put", node, short(prev), short(fp))
}
// Wants is the pass of one module m: for each of its nodes, the want by the six rules, starting from the want written
// before (wants, by node); lastHealthy is, by node, the newest fingerprint other than m.FP the node ran healthy. A failed
// health check of m.FP on a node holds it: every node wanting m.FP is set back to its last healthy fingerprint, but a
@@ -425,14 +450,6 @@ func Wants(m Module, n Nodes, wants map[string]delivery.Want, lastHealthy map[st
if need, wait := at.mustAsk(node, asked, hadLive, ran, running.FP, running.Composed == "", d); wait {
if need != nil {
out.Record = append(out.Record, *need)
} else if prev, barred := at.busBar(asked, hadLive, running.FP); barred {
// The bus is barred: no want moves it but the operator's put. Said, never silent (issue 1945): its holding is named,
// and a node that runs it with no live want has it on neither list, so its condition stays raised.
h := Holding{Bus: true, Words: BusBarWords(node, prev, at.FP)}
out.Rules[node] = h
if !hadLive && running.Phase != delivery.PhaseRemoved && !held {
out.Neither[node] = h
}
}
continue
}
@@ -659,10 +676,10 @@ func short(fp string) string {
return part[:12]
}
// mustAsk says whether the move to the composition d on node waits: for the operator's yes (need names the ask) or, for the bus,
// for the operator's hand (ADR 0539 §14, §15). A module the node never ran (it has no want and no have), a move judged neither, a module that is not of
// policy record and a want that stays are written as any want; so is a revision of the bus module's composition at the
// same fingerprint judged neither (a user list only, §7).
// mustAsk says whether the move to the composition d on node waits for the operator's yes (need names the ask, ADR 0539 §14). A
// module the node never ran (it has no want and no have), a move judged neither, a module that is not of policy record and a want
// that stays are written as any want. The bus module is no exception: it moves by the rules (ADR 0601 §1), its node backing up its
// streams before it applies (§2); the assigner asks no snapshot for it.
func (m Module) mustAsk(node string, before delivery.Want, had, ran bool, running string, haveNamesNoComposition bool, d delivery.ComposedDigest) (*RecordNeed, bool) {
if (!had && !ran) || before.Removed || (before.FP == m.FP && before.Composed == d.Digest) {
return nil, false
@@ -674,9 +691,6 @@ func (m Module) mustAsk(node string, before delivery.Want, had, ran bool, runnin
if !had && ran && running == m.FP && haveNamesNoComposition && d.Judgement == delivery.JudgedUnchanged {
return nil, false
}
if _, barred := m.busBar(before, had, running); barred {
return nil, true // the bus is never moved by the assigner; the operator writes its want (§15)
}
if !m.recorded(d) || d.Judgement == delivery.JudgedNeither || d.Judgement == delivery.JudgedUnchanged {
return nil, false
}
@@ -686,17 +700,7 @@ func (m Module) mustAsk(node string, before delivery.Want, had, ran bool, runnin
return &RecordNeed{Node: node, Digest: d.Digest, Judgement: d.Judgement, Words: d.Words}, true
}
// busBar says whether m is the bus and the move would take it off the fingerprint it ran before, prev: its want's, or with no
// readable want the one its have names (ADR 0539 §15 bars a new one).
func (m Module) busBar(before delivery.Want, had bool, running string) (prev string, barred bool) {
prev = before.FP
if !had {
prev = running
}
return prev, catalogue.ProvidesBus(m.Manifest) && prev != m.FP
}
// recorded says whether the module is of policy record: a person's choice at the assigner over everything but the bus (ADR
// recorded says whether the module is of policy record: a person's choice at the assigner over the manifest's word (ADR
// 0565 §7, catalogue.Upgrade); for a module with none, as the controller holds it beside the composition's digest until the
// lease write (ADR 0539 §14); on a composition made by hand, which carries none, as the manifest derives it.
func (m Module) recorded(d delivery.ComposedDigest) bool {
@@ -104,6 +104,27 @@ func TestEachRuleHoldsTheWantBack(t *testing.T) {
t.Errorf("provided provisions held g14's want: %+v (%s)", out.Wants["g14"], out.Rules["g14"].Words)
}
}
// ADR 0603 §9: the newest source revision governs. The module now at fNew no longer requires the database an older revision
// asked for and requires oidc instead; the seat's keys follow the newest manifest, so the older edge is no wait: a provided
// oidc moves the want, an asked one holds it and names oidc and never database.
for _, c := range []struct {
name string
state, older string
moves bool
}{{"the replaced edge provided, the older edge retiring", provisions.Provided, provisions.Retiring, true},
{"the replaced edge provided, the older edge asked", provisions.Provided, provisions.Asked, true},
{"the replaced edge asked, the older edge retiring", provisions.Asked, provisions.Retiring, false}} {
m, n := ready()
m.Manifest.Requires = []string{"oidc"}
m.Provisions = func(node string) map[string]provisions.Provision {
return map[string]provisions.Provision{node + ".mail.oidc": {Provision: "oidc", State: c.state},
node + ".mail.database": {Provision: "database", State: c.older, Why: "the operator's answer"}}
}
out := Wants(m, n, wantedOld(), nil)
if got := out.Wants["g14"].FP == fNew; got != c.moves || (!c.moves && (!strings.Contains(out.Rules["g14"].Words, "oidc") || strings.Contains(out.Rules["g14"].Words, "database"))) {
t.Errorf("%s: want %+v, held by %q", c.name, out.Wants["g14"], out.Rules["g14"].Words)
}
}
// 1954/D1 (the lead, 2026-10-29): a required provision with no key yet holds the want, named; and only one the
// composition keys (catalogue.Awaited, issue 774), replaying 2026-10-30: grafana (not handed) waited on ace for
// oidc-client, which the controller still makes, and mesh-build-go (handed) for go-toolchain on its own machine.
@@ -155,6 +176,24 @@ func TestEachRuleHoldsTheWantBack(t *testing.T) {
}
}
}
// ADR 0603 §5: a requirement nobody offers is an unknown provider and waits, loudly; one something offers (a module's own
// name, a module the mesh provides itself) is left to catalogue.Awaited as before and does not.
for _, c := range []struct {
name, requires string
offered bool
}{{"a module's own name (node-tools)", "node-tools", true}, {"a module the mesh provides itself (mesh-addressing)", "mesh-addressing", true},
{"a provision nobody offers", "database", false}} {
m, n := ready()
m.Manifest.Requires = []string{c.requires}
m.Answering = func(string) []catalogue.Manifest { return nil }
m.Offered = func(p string) bool { return c.offered && p == c.requires }
m.Provisions = func(string) map[string]provisions.Provision { return map[string]provisions.Provision{} }
out := Wants(m, n, wantedOld(), nil)
if got := out.Wants["g14"].FP == fNew; got != c.offered || (!c.offered && (out.Rules["g14"].Rule != delivery.RuleProvisions ||
!strings.Contains(out.Rules["g14"].Words, "no module offers database yet") || !strings.Contains(out.Rules["g14"].Words, "past its bound"))) {
t.Errorf("%s: want %+v, held by %+v", c.name, out.Wants["g14"], out.Rules["g14"])
}
}
// 1960: rule 5 holds no want, and no module is left without a first want for a provider down or a node offline; the
// provider down is said (Unreached). novox on 2026-10-29: route-proxy down held de-spiegel and link2pay, minio not
// healthy held photos, and novox offline left a module with no canary.
@@ -639,11 +678,13 @@ func TestAMoveJudgedNeitherOrNeverRunOrNotRecordedIsWrittenAtOnce(t *testing.T)
// 1906/D4 (ADR 0539 §7, §15): the bus module's new fingerprint is never written by the assigner, but a revision of its
// composition at the same fingerprint that only changes the user list (judged neither) is written at once.
func TestTheBusIsNotMovedButItsUserListRevisionIsWrittenAtOnce(t *testing.T) {
func TestTheBusMovesByTheRulesAndItsUserListRevisionIsWrittenAtOnce(t *testing.T) {
m, n, _ := movingTo(delivery.JudgedNeither)
m.Manifest.Provides = []catalogue.Offer{{Name: "mesh-bus"}}
if out := Wants(m, n, wantedOld(), nil); out.Wants["ace"].FP == fNew {
t.Errorf("the bus moved to a new fingerprint: %+v", out.Wants["ace"])
// ADR 0601 §1, §2: the bus module's want is written by the rules, as any module's, and no snapshot is asked for it: its node
// backs its streams up before it applies.
if out := Wants(m, n, wantedOld(), nil); out.Wants["ace"].FP != fNew || len(out.Record) != 0 || len(out.Neither) != 0 {
t.Errorf("the bus did not move by the rules: %+v", out)
}
running := map[string]delivery.Want{"ace": {FP: fNew, Composed: dig(fOld)}, "g14": {FP: fNew, Composed: dig(fOld)}}
out := Wants(m, n, running, nil)
@@ -696,23 +737,6 @@ func TestTheBusIsNotMovedButItsUserListRevisionIsWrittenAtOnce(t *testing.T) {
t.Errorf("a held bus did not take its user-list revision as a put: %+v", w)
}
m.Puts = nil
// 1945 (the bus bar made loud): a node that runs the bus at an old-form build and holds no want for it gets none, since only
// the operator's put moves the bus, and the pass says so: a holding naming the bar, and the node on neither list so its
// condition stays raised. A node with a live want keeps it and is not on neither list.
m.Composed = composedEverywhere
n.Haves["ace"]["mail"] = delivery.Have{FP: "b-1700000000-oldform", Phase: delivery.PhaseRunning, Health: delivery.HealthOK}
out = Wants(m, n, map[string]delivery.Want{"g14": {FP: fOld, Composed: dig(fOld)}}, nil)
if w, wrote := out.Wants["ace"]; wrote {
t.Errorf("the bus bar wrote ace's want %+v", w)
}
h, on := out.Neither["ace"]
if !on || !h.Bus || out.Rules["ace"] != h || !strings.Contains(h.Words, "only your put moves the bus") || !strings.Contains(h.Words, "ace runs b-1700000000-oldform") ||
!strings.Contains(h.Words, "assigner_put") {
t.Errorf("the barred bus on ace: neither %+v (%v), rules %+v", h, on, out.Rules["ace"])
}
if _, on := out.Neither["g14"]; on || out.Wants["g14"].FP == fNew {
t.Errorf("g14 has a live want: neither %+v, its want %+v", out.Neither, out.Wants["g14"])
}
stepsOfAPut(t)
}
@@ -889,8 +913,8 @@ func TestTheRecordPolicyIsTheOneTheControllerHolds(t *testing.T) {
}
}
// Review round 1, item 2 (ADR 0539 §15): a node that runs the bus with no readable want compares the new fingerprint with the one
// it runs, from its have: the same fingerprint is not a new one (written by the rules), another is barred.
// A node that runs the bus with no readable want gets its want by the rules, at the fingerprint it runs or at a new one alike: the
// bus has no bar of its own (ADR 0601 §1).
func TestTheBusWithNoWantIsComparedWithWhatItRuns(t *testing.T) {
m, n, _ := movingTo(delivery.JudgedNeither)
m.Manifest.Provides = []catalogue.Offer{{Name: "mesh-bus"}}
@@ -899,8 +923,8 @@ func TestTheBusWithNoWantIsComparedWithWhatItRuns(t *testing.T) {
t.Errorf("the bus runs F with no want and was not given one: %+v %v", w, ok)
}
n.Haves["ace"]["mail"] = delivery.Have{FP: fOld, Phase: delivery.PhaseRunning, Health: delivery.HealthOK}
if w, ok := Wants(m, n, nil, nil).Wants["ace"]; ok {
t.Errorf("the bus was moved to a new fingerprint with no want: %+v", w)
if w, ok := Wants(m, n, nil, nil).Wants["ace"]; !ok || w.FP != fNew {
t.Errorf("the bus running an older fingerprint with no want was not given its want by the rules: %+v %v", w, ok)
}
}
@@ -34,8 +34,8 @@ func TestManifest(t *testing.T) {
reads := slices.Clone(m.Reads)
slices.Sort(reads)
// ADR 0593 §8: it composes nothing, so it reads no candidate, held or resolved value, compose step or build log; it reads
// every node's published compositions to compare them.
want := []string{"mesh-delivery-builder.builds", "mesh-delivery-indexer.manifests", "mesh-delivery-indexer.modules",
// every node's published compositions to compare them, and the gate's source and merges, for the release that carries a build (ADR 0601 §1).
want := []string{"mesh-delivery-builder.builds", "mesh-delivery-gate.merges", "mesh-delivery-gate.source", "mesh-delivery-indexer.manifests", "mesh-delivery-indexer.modules",
"mesh-delivery-teller.answers", "mesh-delivery-watchman.presence", "mesh-grants.grants", "mesh-host.composed",
"mesh-provisioner.provisions", "node-engine.have"}
if !reflect.DeepEqual(reads, want) {
@@ -25,6 +25,8 @@
],
"reads": [
"mesh-delivery-builder.builds",
"mesh-delivery-gate.merges",
"mesh-delivery-gate.source",
"mesh-delivery-indexer.manifests",
"mesh-delivery-indexer.modules",
"mesh-delivery-teller.answers",
@@ -3,7 +3,7 @@ module git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929
github.com/nats-io/nats.go v1.54.0
)
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929 h1:RtAFbUJRP1qO9AIMM2ue+cXNzqj7WZ7IOlSqNvttwuU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -98,7 +98,7 @@ func Tools(r build.StuckReader, askAgain func(key string, value []byte) error, q
Description: "Every build that ended since a time, oldest first, from the builder's log build-log (kept 30 days): per build its module, " +
"fingerprint, source revision, repository, directory, commit and ref, the build node, when it ended, its outcome (done, failed or " +
"queued), the manifest built, the bases it was built against and each artifact made with its digest and reference. more says the " +
"log held more than one answer carries: ask again from the last build's time. What mesh-catalog registers a build from. Read-only.",
"log held more than one answer carries: ask again from the last build's time. What the operator's command nox builds reads. Read-only.",
Input: map[string]any{"type": "object", "required": []string{"since"}, "properties": map[string]any{
"since": map[string]any{"type": "string", "description": "RFC 3339 time; the builds that ended at or after it"}}},
Run: func(args map[string]any) (any, error) {
@@ -131,7 +131,7 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
trunks.start()
b.Trunk = trunks.head
// The builder's log (ADR 0571 §1): each build's start and end in its module log `build-log`, read by the view, the
// indexer and the assigner through their reads, and by mesh-catalog through the tool builder_builds_since.
// indexer and the assigner through their reads, and by the operator's command nox builds through the tool builder_builds_since.
b.Log = func(e delivery.BuildLogEntry) {
if err := appendLog(context.WithoutCancel(ctx), js, cred, e); err != nil {
logger.Printf("%v", err)
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-builder
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,7 +1,7 @@
git.novox.be/novox/mesh-catalog/modules/mesh-build-go/buildstep v0.0.0-20261102193506-d221c633799a h1:oxgksCiq272/afXbUION/dxtAS5MPGnmyq6+AZQxmXg=
git.novox.be/novox/mesh-catalog/modules/mesh-build-go/buildstep v0.0.0-20261102193506-d221c633799a/go.mod h1:kpsX4Pgss5EeuBvel1zWa2ogj7MkR3jZso6oa/J0YUQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929 h1:RtAFbUJRP1qO9AIMM2ue+cXNzqj7WZ7IOlSqNvttwuU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -944,7 +944,7 @@ func LogStarted(module string, m delivery.Module, ref, by string, at time.Time)
}
// LogEnded is the log's entry of that build ending: the outcome (done, failed or queued), the manifest built, the bases it
// was built against and each artifact made, as the done key has them: what mesh-catalog registers a build from.
// was built against and each artifact made, as the done key has them: what the operator's command nox builds shows of a build.
func LogEnded(module string, m delivery.Module, ref, by string, at time.Time, result delivery.BuilderBuild) delivery.BuildLogEntry {
e := LogStarted(module, m, ref, by, at)
e.Event, e.Outcome, e.Manifest = delivery.BuildEndedEvent, result.State, m.Manifest
@@ -565,6 +565,7 @@ func TestAKeyIsBuiltOnlyWithTheToolchainItsFingerprintNamesAndNothingIsWrittenOt
r.b.Module(context.Background(), "m", moduleValueWith(t, "modules/m", old))
r.b.Wait()
if len(r.st.vals) != 0 || len(r.bm.calls) != 0 {
t.Run("a recipe that moves builds the module again under its new fingerprint", aRecipeThatMovesBuildsTheModuleAgainUnderItsNewFingerprint)
t.Errorf("keys %v and %d calls for a fingerprint of go1.26.0 while the nodes agree on go1.27.1, want neither", r.st.vals, len(r.bm.calls))
}
// A recipe that differs is a different toolchain too, and a value that names none (written before) is not built.
@@ -589,6 +590,45 @@ func TestAKeyIsBuiltOnlyWithTheToolchainItsFingerprintNamesAndNothingIsWrittenOt
}
}
// ADR 0603 §3 (D9): a source revision changing the Go build module's recipe and a Go module builds the module at once with
// the recipe the nodes run and again, a second key under the new fingerprint, once the agreement moves to the new recipe;
// the first key is neither rebuilt nor waited on.
func aRecipeThatMovesBuildsTheModuleAgainUnderItsNewFingerprint(t *testing.T) {
r := newRig(t, goBundle)
r.consider(t, "modules/m")
calls := len(r.bm.calls)
fp2 := "src2:" + hex64('2')
for _, n := range []string{"ace", "novox"} {
b, err := delivery.EncodeToolchain(delivery.Toolchain{Language: "go", Version: "go1.27.1", Recipe: "2", At: time.Now()})
if err != nil {
t.Fatal(err)
}
r.tools.Put(buildrun.GoBuildModule, n+".go", b, false)
}
r.b.Reconsider(context.Background())
r.b.Wait()
if len(r.bm.calls) != calls {
t.Errorf("%d calls, was %d: the first fingerprint was built again for the new recipe", len(r.bm.calls), calls)
}
data, err := delivery.EncodeModule(delivery.Module{Repository: "novox/mesh-catalog", Commit: "7a74cee83f88e6585a5511c3cc53bdf0f5303d20", FP: "src1:" + hex64('f'), FP2: fp2,
Manifest: mdigest('a'), Path: "modules/m", Toolchains: map[string]delivery.ToolchainUsed{"go": {Version: "go1.27.1", Recipe: "2"}}})
if err != nil {
t.Fatal(err)
}
r.b.Module(context.Background(), "m", data)
r.b.Wait()
second, err := delivery.BuilderBuildKey("m", fp2)
if err != nil {
t.Fatal(err)
}
if v := valueOf(t, r.st, second); v.State != delivery.BuildDone || v.Toolchains["go"].Recipe != "2" {
t.Errorf("second build = %+v, want done with recipe 2", v)
}
if v := valueOf(t, r.st, r.key); v.State != delivery.BuildDone {
t.Errorf("first build = %+v, want it left done", v)
}
}
func TestNodesTheyDisagreeOnLeaveTheKeyUnwritten(t *testing.T) { // 1615/AC4
r := newRig(t, goBundle)
r.tools.Put(buildrun.GoBuildModule, "ace.go", tcValue(t, "go", "go1.26.0"), false)
@@ -244,7 +244,9 @@ func Tools(r check.Reader) []stdio.Tool {
"build seats' streams, whose backlog the delivery view shows (novox/hq ADR 0525). held_by_turn names the candidates whose check an approved " +
"merge's turn holds off (novox/hq ADR 0548 §3), with the repository and the merge that waits; the gate's turns tool reads every turn and its " +
"hand-now tool steers one. dead_letters says whether the checker hears the asks the build seats give up on (issue 734), and why not: " +
"deaf, a try or check whose ask is given up waits until its bound. Read-only.",
"deaf, a try or check whose ask is given up waits until its bound. builds_not_ended names, of each running check, the builder's builds of " +
"its candidate's changed modules that are neither done nor failed: the check waits on them before it composes, within its build-wait-bound " +
"(novox/hq ADR 0603 §10); a failed one fails the check: rebuild it with the builder's builder_build_again, and the check asks itself again when the build is done. Read-only.",
Input: map[string]any{"type": "object", "properties": map[string]any{}},
Run: func(map[string]any) (any, error) { return r.Status() }},
}
@@ -283,7 +285,7 @@ func main() {
StepStates: map[string]check.StateRead{"mesh-build-go": read(stdio.State("mesh-build-go.steps")), "mesh-build-node": read(stdio.State("mesh-build-node.steps"))},
HaveState: read(stdio.State("node-engine.have")), StepBound: stepBound(),
TurnState: read(stdio.State("mesh-delivery-gate.turns")), HearingState: read(stdio.State(buildrun.HearingState)),
AskState: read(stdio.State(check.AsksState))}
AskState: read(stdio.State(check.AsksState)), BuildState: read(stdio.State("mesh-delivery-builder.builds"))}
r.Runs = runsOf{read(stdio.State("mesh-delivery-indexer.modules"))} // the node-engine at the commit the mesh runs, as the checker reads it
tries := stdio.State(check.TriesState)
source := stdio.State("mesh-delivery-gate.source")
@@ -128,7 +128,9 @@ func (m *manifestVerdicts) put(module string, v *delivery.ManifestVerdict) {
delete(m.by, module)
return
}
m.by[module] = *v
// The sandbox's composer may be the node-engine of an older delivery, which does not know the verdict waits: it is handed a
// wait as the refusal it is held as.
m.by[module] = v.AsRefused()
}
func (m *manifestVerdicts) all() map[string]delivery.ManifestVerdict {
@@ -398,7 +398,7 @@ func startBed(t *testing.T) *bed {
delivery.CheckerOwedBucket: delivery.CheckerOwedHistory, delivery.PreviewsBucket: delivery.PreviewsHistory, delivery.CandidatesBucket: delivery.CandidatesHistory,
delivery.ModulesBucket: delivery.ModulesHistory, delivery.ManifestsBucket: delivery.ManifestsHistory, delivery.SourceBucket: delivery.SourceHistory,
delivery.PresenceBucket: delivery.PresenceHistory, delivery.HaveBucket: delivery.HaveHistory, check.TriesBucket: 1,
delivery.TurnsBucket: delivery.TurnsHistory} {
delivery.TurnsBucket: delivery.TurnsHistory, delivery.BuilderBuildsBucket: 1} {
kv, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: name, History: uint8(history)})
if err != nil {
t.Fatal(err)
@@ -578,20 +578,72 @@ func (b *bed) owed(t *testing.T, number int, head, state string) delivery.Owed {
// TestCheckerJudgesACandidateAndOwesItsStatus: a candidate the indexer wrote, one pull request of mesh-delivery, is fetched
// merged onto the trunk the gate names, digested and tested at both levels on the node, kept, previewed; the verdict passes
// with the counts of both levels and the composition not run, and the status mesh/check is owed on the head as success.
func TestCheckerJudgesACandidateAndOwesItsStatus(t *testing.T) { // N6/AC5, 1300/AC5, 1445/AC1, 1929/AC1
func TestCheckerJudgesACandidateAndOwesItsStatus(t *testing.T) { // N6/AC5, 1300/AC5, 1445/AC1, 1929/AC1, ADR 0603 §10
b := startBed(t)
key := b.candidate(t, "feat/x", pull(repo, 7, head1))
// ADR 0603 §10: the candidate changes the module m, whose manifest the trunk holds; the builder has not built m's
// candidate fingerprint yet, so the check waits on the builder's key before it composes.
manifest := `{"module":"m"}`
digest := delivery.ManifestDigest([]byte(manifest))
stored, err := delivery.EncodeManifest(delivery.Manifest{Module: "m", JSON: manifest})
if err != nil {
t.Fatal(err)
}
mkey, _ := delivery.ManifestsKey(digest)
if _, err := b.buckets[delivery.ManifestsBucket].Put(b.ctx, mkey, stored); err != nil {
t.Fatal(err)
}
known, err := delivery.EncodeModule(delivery.Module{Repository: repo, Commit: trunk, Path: "modules/m", Manifest: digest, None2: "the fingerprint is computed with its chain"})
if err != nil {
t.Fatal(err)
}
if _, err := b.buckets[delivery.ModulesBucket].Put(b.ctx, "m", known); err != nil {
t.Fatal(err)
}
fp := "src2:" + strings.Repeat("f", 64)
p := pull(repo, 7, head1)
p.Modules, p.Fps = []string{"m"}, map[string]delivery.CandidateModule{"m": {FP2: fp, Manifest: digest}}
key := b.candidate(t, "feat/x", p)
// ADR 0593 §7, §9 (D12): n1 and n2 are placed; n1's engine answers compose-candidate on the bus, n2's never does.
asked := b.engine(t, "n1", "n2")
b.start(t, nil)
v := b.verdict(t, key, ended)
if v.State != delivery.VerdictPass || v.Levels[delivery.LevelUnit].Run != 2 || v.Levels[delivery.LevelIntegration].Run != 2 || v.Trunk[repo] != trunk {
b.said(t, "waits on the builder's builds of m before it composes")
if got := asked(); len(got) != 0 {
t.Fatalf("the engines were asked %+v before m's build ended", got)
}
bkey, _ := delivery.BuilderBuildKey("m", fp)
building, _ := json.Marshal(delivery.BuilderBuild{V: 1, State: delivery.BuildBuilding})
if _, err := b.buckets[delivery.BuilderBuildsBucket].Put(b.ctx, bkey, building); err != nil {
t.Fatal(err)
}
if v := b.verdict(t, key, func(delivery.Verdict) bool { return true }); v.State != delivery.VerdictRunning || len(asked()) != 0 {
t.Fatalf("verdict %s and %d asks while m builds: the check composes only once its build ended", v.State, len(asked()))
}
// m's build fails: the check fails naming it. Built again (builder_build_again writes the key done), the check asks
// itself again for the same heads, with no recheck, and passes.
failedBuild, _ := json.Marshal(delivery.BuilderBuild{V: 1, State: delivery.BuildFailed, Error: "go build: undefined: x"})
if _, err := b.buckets[delivery.BuilderBuildsBucket].Put(b.ctx, bkey, failedBuild); err != nil {
t.Fatal(err)
}
if v := b.verdict(t, key, ended); v.State != delivery.VerdictFail || v.Failed.Test != check.BuildFailedTest+"m" {
t.Fatalf("verdict %+v, want it failed on m's build", v)
}
built, _ := json.Marshal(delivery.BuilderBuild{V: 1, State: delivery.BuildDone, Manifest: digest})
if _, err := b.buckets[delivery.BuilderBuildsBucket].Put(b.ctx, bkey, built); err != nil {
t.Fatal(err)
}
b.said(t, "the builds of m it failed on are done again, so it is checked again")
v := b.verdict(t, key, func(v delivery.Verdict) bool { return v.State == delivery.VerdictPass })
// The check asked again takes the packages the failed check ran from their kept verdicts.
if v.State != delivery.VerdictPass || v.Levels[delivery.LevelUnit].Kept != 2 || v.Levels[delivery.LevelIntegration].Kept != 2 || v.Trunk[repo] != trunk {
t.Fatalf("verdict = %+v", v)
}
if c := v.Composition; c == nil || c.State != delivery.CompositionPass || !slices.Equal(c.Machines, []string{"n1"}) ||
!strings.Contains(c.Words, "n2: "+delivery.ComposeNoAnswer) || len(v.Machines) != 1 || v.Machines[0] != "n1" {
t.Errorf("composition %+v machines %v", v.Composition, v.Machines)
}
if v.Composition != nil && strings.Contains(v.Composition.Words, "did not end") {
t.Errorf("the composition says %q: m's build ended within the bound", v.Composition.Words)
}
if got := asked(); len(got) == 0 || got[0].Candidate != key {
t.Errorf("n1's engine was asked %+v, want the check's candidate %s", got, key)
}
@@ -602,10 +654,11 @@ func TestCheckerJudgesACandidateAndOwesItsStatus(t *testing.T) { // N6/AC5, 1300
t.Errorf("the check ask named the nodes %v, want [n1] from the have's node record", names)
}
o := b.owed(t, 7, head1, "success")
if o.Context != delivery.StatusContextCheck || !strings.HasPrefix(o.Text, "pass 111111111111 on aaaaaaaaaaaa") || !strings.Contains(o.Text, "unit 2 run") {
if o.Context != delivery.StatusContextCheck || !strings.HasPrefix(o.Text, "pass 111111111111 on aaaaaaaaaaaa") || !strings.Contains(o.Text, "unit 0 run, 2 kept") {
t.Errorf("owed = %+v", o)
}
for tool, want := range map[string]int{"mesh-build-go.build_go_fetch": 1, "mesh-build-go.build_go_digest": 2, "mesh-build-go.build_go_test": 2} {
// Two checks of the same heads: the one failed on m's build and the one it asked itself again; the second runs no test.
for tool, want := range map[string]int{"mesh-build-go.build_go_fetch": 2, "mesh-build-go.build_go_digest": 4, "mesh-build-go.build_go_test": 2} {
if got := b.fake.count(tool); got != want {
t.Errorf("%s called %d times, want %d", tool, got, want)
}
@@ -8,10 +8,12 @@ import (
"sort"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/process"
@@ -31,6 +33,10 @@ type judgeDeps struct {
forge check.Forge
announce check.Announcer
builds check.BuildStates
// ends waits on the builder's builds of a candidate before the check composes (ADR 0603 §10); nil composes without waiting.
ends check.BuildEnds
// buildWait is the setting build-wait-bound; check.DefaultBuildWait when 0.
buildWait time.Duration
// compositions makes a check's composition from the engines' answers (ADR 0593 §7); nil leaves it not run.
compositions check.CompositionReader
// nodes names every node with a record; nil hands no names.
@@ -97,7 +103,7 @@ func newJudge(d judgeDeps) check.Judge {
nodes = d.nodes
}
return &check.Examiner{Nodes: nodes, Compositions: d.compositions, Asks: d.asks, Toolchains: d.tools,
Kept: check.Kept{Store: d.kept}, Previews: d.previews, Modules: d.repos, Packages: d.repos, Published: d.published, Forge: d.forge, Runs: d.repos, Builds: d.builds, Announce: d.announce, Trunks: d.trunks, Manifests: d.repos, Predefined: d.predefined, Logf: d.logf}
Kept: check.Kept{Store: d.kept}, Previews: d.previews, Modules: d.repos, Packages: d.repos, Published: d.published, Forge: d.forge, Runs: d.repos, Builds: d.builds, BuildEnds: d.ends, BuildWait: d.buildWait, Announce: d.announce, Trunks: d.trunks, Manifests: d.repos, Read: d.repos, Predefined: d.predefined, Logf: d.logf}
}
// manifestKV reads a manifest by its digest.
@@ -122,6 +128,42 @@ type repoModules struct {
repoOf map[string]string // by module: its repository
commit map[string]string // by module: the commit its record was computed at (the commit the mesh runs)
typed map[string]string // by module: its manifest digest
parsed map[string]catalogue.Manifest // by manifest digest: the manifest as read, once (a digest's manifest never changes)
}
// Manifest is the kept manifest of a digest, read from the indexer's manifests state once and then from memory.
func (r *repoModules) Manifest(ctx context.Context, digest string) (catalogue.Manifest, error) {
r.mu.Lock()
m, known := r.parsed[digest]
r.mu.Unlock()
if known {
return m, nil
}
key, err := delivery.ManifestsKey(digest)
if err != nil {
return catalogue.Manifest{}, err
}
raw, _, found, err := r.manifests.Get(ctx, key)
if err != nil {
return catalogue.Manifest{}, err
}
if !found {
return catalogue.Manifest{}, fmt.Errorf("the indexer keeps no manifest %s", digest)
}
kept, err := delivery.DecodeManifest(raw)
if err != nil {
return catalogue.Manifest{}, err
}
if m, err = parseManifest(kept.JSON); err != nil {
return catalogue.Manifest{}, err
}
r.mu.Lock()
if r.parsed == nil {
r.parsed = map[string]catalogue.Manifest{}
}
r.parsed[digest] = m
r.mu.Unlock()
return m, nil
}
// Put takes a change of the indexer's modules bucket in: a module's repository and the languages its manifest names.
@@ -257,6 +299,69 @@ func (r buildReader) Build(ctx context.Context, module, fp2 string) (delivery.Bu
return b, err == nil, err
}
// buildEnds hears the builder's build keys of a candidate on the bus (mesh-delivery-builder_builds): a watch of exactly
// those keys gives each one's value now and every write the builder makes after, so the check waits on the part that owns
// the fact and polls nothing (novox/hq ADR 0603 §10).
type buildEnds struct{ kv jetstream.KeyValue }
func (e buildEnds) Ended(ctx context.Context, builds map[string]string) (map[string]delivery.BuilderBuild, error) {
return e.until(ctx, builds, func(b delivery.BuilderBuild) bool {
return b.State == delivery.BuildDone || b.State == delivery.BuildFailed
})
}
func (e buildEnds) Done(ctx context.Context, builds map[string]string) bool {
done, err := e.until(ctx, builds, func(b delivery.BuilderBuild) bool { return b.State == delivery.BuildDone })
return err == nil && len(done) == len(builds)
}
// until hears the builds' keys until each holds a build that ended holds, or ctx ends, and answers those that do, by module.
func (e buildEnds) until(ctx context.Context, builds map[string]string, ended func(delivery.BuilderBuild) bool) (map[string]delivery.BuilderBuild, error) {
byKey := map[string]string{}
keys := make([]string, 0, len(builds))
for module, fp2 := range builds {
key, err := delivery.BuilderBuildKey(module, fp2)
if err != nil {
return nil, err
}
byKey[key] = module
keys = append(keys, key)
}
out := map[string]delivery.BuilderBuild{}
if len(keys) == 0 {
return out, nil
}
w, err := e.kv.WatchFiltered(ctx, keys)
if err != nil {
return nil, err
}
defer func() { _ = w.Stop() }()
for len(out) < len(keys) {
select {
case <-ctx.Done():
return out, nil // the bound: the builds not ended are left out
case entry, open := <-w.Updates():
if !open {
return out, nil
}
if entry == nil || entry.Operation() != jetstream.KeyValuePut {
continue // the end of the values now, or a key deleted: it has not ended
}
module, ours := byKey[entry.Key()]
if !ours {
continue
}
b, err := delivery.DecodeBuilderBuild(entry.Value())
if err == nil && ended(b) {
out[module] = b
} else {
delete(out, module) // asked again (builder_build_again), or not a build the check can read: not ended
}
}
}
return out, nil
}
// ManifestDigests is the manifest digest of every module the indexer holds, by module: the catalogue the indexer's judge
// judges a change's manifests in (novox/hq ADR 0578 §2).
func (r *repoModules) ManifestDigests() map[string]string {
@@ -170,4 +170,10 @@ func TestAManifestFallingBackToAPredefinedKeyIsRead(t *testing.T) {
if _, err := parseManifest(manifest); err != nil {
t.Errorf("manifestReader: %v", err)
}
// The sandbox's composer may be an older node-engine, which knows no verdict waits: it is handed a wait as a refusal.
var held manifestVerdicts
held.put("nats", &delivery.ManifestVerdict{Module: "nats", State: delivery.VerdictManifestWaits, Unknown: []string{"core-credentials"}, Judge: "j"})
if v := held.all()["nats"]; v.State != delivery.VerdictRefused || len(v.Problems) != 1 || len(v.Unknown) != 0 {
t.Errorf("a waiting verdict is handed to the sandbox as %+v", v)
}
}
@@ -166,14 +166,19 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
// The builder's builds say how the candidates' fingerprints stand (issue 1684); where its bucket is not there yet the line
// says nothing of the builds, until the checker starts again.
var builds check.BuildStates
if kv := process.OptionalBucket(ctx, js, delivery.BuilderBuildsBucket, "the line says nothing of the builds until the checker starts again", logger.Printf); kv != nil {
builds = buildReader{kvStore{kv}}
var ends check.BuildEnds
if kv := process.OptionalBucket(ctx, js, delivery.BuilderBuildsBucket, "the line says nothing of the builds, and a check composes without waiting on them, until the checker starts again", logger.Printf); kv != nil {
builds, ends = buildReader{kvStore{kv}}, buildEnds{kv}
}
if ends != nil {
ch.Rebuilds = ends // a check failed on a build is asked again once that build is done again (ADR 0603 §10)
}
// The check's composition is the checker's, from what each placed node's engine answers (ADR 0593 §7, §9).
answerBound, err := composeAnswerBound(getenv("MESH_CHECKER_SETTINGS"))
set, err := readSettings(getenv("MESH_CHECKER_SETTINGS"))
if err != nil {
return err
}
answerBound := set.answer
holdCallsFor(answerBound)
verdicts := &manifestVerdicts{}
compositions := check.EngineCompositions{
@@ -212,7 +217,7 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
ch.Judge = newJudge(judgeDeps{
asks: buildrun.Asker{Publisher: asks, Waiter: waiter, Bound: stepBound}, tools: tools, trunks: trunks,
kept: kvStore{buckets[delivery.CheckerKeptBucket]}, previews: kvStore{buckets[delivery.PreviewsBucket]},
repos: repos, published: published, builds: builds, forge: forgeReads{call: caller.Call}, logf: logger.Printf,
repos: repos, published: published, builds: builds, ends: ends, buildWait: set.buildWait, forge: forgeReads{call: caller.Call}, logf: logger.Printf,
announce: announce, compositions: compositions,
nodes: openNodeNames(ctx, js, logger.Printf),
predefined: keptPredefined{process.AwaitBucket(ctx, js, conn, delivery.IndexerModule+"_"+delivery.IndexerPredefinedState,
@@ -10,38 +10,57 @@ import (
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/process"
"mesh-delivery-checker/internal/check"
)
// The checker's own settings (its config, written by the node-engine into ${dir:state}/settings.json, named to the
// process by MESH_CHECKER_SETTINGS): compose-answer-bound, the seconds a check waits for a node's engine to answer
// compose-candidate (novox/hq ADR 0593 §7, §9: 60 by default; the checker owns it).
// compose-candidate (novox/hq ADR 0593 §7, §9: 60 by default), and build-wait-bound, the seconds a check waits for the
// builder's builds of its candidate before it composes (ADR 0603 §10: 1800 by default). The checker owns both.
// composeAnswerBound reads compose-answer-bound from the settings file at path: delivery.ComposeAnswerBound where no
// file or no value is there; a value that is no number of seconds from 1 to 600 is refused, naming it.
func composeAnswerBound(path string) (time.Duration, error) {
// settings are the checker's settings as read.
type settings struct {
// answer is compose-answer-bound; buildWait is build-wait-bound.
answer, buildWait time.Duration
}
// readSettings reads the settings file at path: each value's default where no file or no value is there; a value that is
// no whole number of seconds within its range is refused, naming it.
func readSettings(path string) (settings, error) {
out := settings{answer: delivery.ComposeAnswerBound, buildWait: check.DefaultBuildWait}
if path == "" {
return delivery.ComposeAnswerBound, nil
return out, nil
}
raw, err := os.ReadFile(path)
if errors.Is(err, fs.ErrNotExist) {
return delivery.ComposeAnswerBound, nil
return out, nil
}
if err != nil {
return 0, err
return settings{}, err
}
var s struct {
Bound *float64 `json:"compose-answer-bound"`
Answer *float64 `json:"compose-answer-bound"`
BuildWait *float64 `json:"build-wait-bound"`
}
if err := json.Unmarshal(raw, &s); err != nil {
return 0, fmt.Errorf("%s is no settings file: %w", path, err)
return settings{}, fmt.Errorf("%s is no settings file: %w", path, err)
}
if s.Bound == nil {
return delivery.ComposeAnswerBound, nil
for _, v := range []struct {
name string
value *float64
min, max float64
into *time.Duration
}{{"compose-answer-bound", s.Answer, 1, 600, &out.answer}, {"build-wait-bound", s.BuildWait, 60, 7200, &out.buildWait}} {
if v.value == nil {
continue
}
if *v.value < v.min || *v.value > v.max || *v.value != float64(int(*v.value)) {
return settings{}, fmt.Errorf("%s is %v: a whole number of seconds from %v to %v", v.name, *v.value, v.min, v.max)
}
*v.into = time.Duration(*v.value) * time.Second
}
if *s.Bound < 1 || *s.Bound > 600 || *s.Bound != float64(int(*s.Bound)) {
return 0, fmt.Errorf("compose-answer-bound is %v: a whole number of seconds from 1 to 600", *s.Bound)
}
return time.Duration(*s.Bound) * time.Second, nil
return out, nil
}
// holdCallsFor makes a bus call wait the compose-answer-bound (novox/hq ADR 0593 §7), not the SDK's CallTimeout of 30 s:
@@ -8,19 +8,25 @@ import (
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/process"
"mesh-delivery-checker/internal/check"
)
// ADR 0593 §7: the answer bound is the checker's setting compose-answer-bound, 60 s when the file or the value is
// absent; a value that is no whole number of seconds from 1 to 600 stops the start, naming it.
// absent; a value that is no whole number of seconds from 1 to 600 stops the start, naming it. ADR 0603 §10: the wait for
// the candidate's builds is build-wait-bound, 30 minutes when absent, from 60 to 7200 seconds.
func TestTheAnswerBoundIsTheCheckersSetting(t *testing.T) {
dir := t.TempDir()
for _, row := range []struct {
body string
want time.Duration
err bool
}{{"", delivery.ComposeAnswerBound, false}, {`{}`, delivery.ComposeAnswerBound, false},
{`{"compose-answer-bound": 90}`, 90 * time.Second, false}, {`{"compose-answer-bound": 0}`, 0, true},
{`{"compose-answer-bound": 1.5}`, 0, true}} {
body string
want time.Duration
err bool
buildWait time.Duration
}{{"", delivery.ComposeAnswerBound, false, check.DefaultBuildWait}, {`{}`, delivery.ComposeAnswerBound, false, check.DefaultBuildWait},
{`{"compose-answer-bound": 90}`, 90 * time.Second, false, check.DefaultBuildWait}, {`{"compose-answer-bound": 0}`, 0, true, 0},
{`{"compose-answer-bound": 1.5}`, 0, true, 0},
{`{"compose-answer-bound": 60, "build-wait-bound": 600}`, 60 * time.Second, false, 10 * time.Minute},
{`{"build-wait-bound": 30}`, 0, true, 0}} {
path := filepath.Join(dir, "settings.json")
os.Remove(path)
if row.body != "" {
@@ -28,9 +34,10 @@ func TestTheAnswerBoundIsTheCheckersSetting(t *testing.T) {
t.Fatal(err)
}
}
got, err := composeAnswerBound(path)
if (err != nil) != row.err || got != row.want {
t.Errorf("%q: %s, %v", row.body, got, err)
set, err := readSettings(path)
got := set.answer
if (err != nil) != row.err || got != row.want || set.buildWait != row.buildWait {
t.Errorf("%q: %s, %s, %v", row.body, got, set.buildWait, err)
}
// Issue 1126: the SDK's call gives up at CallTimeout (30 s), so the checker's calls must wait the bound read
// (an answer at 31 s is heard under 60 s), and the bound never shortens the SDK's own.
@@ -1,3 +1,4 @@
{
"compose-answer-bound": 60
"compose-answer-bound": 60,
"build-wait-bound": 1800
}
+5 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-checker
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
@@ -25,3 +25,7 @@ require (
)
replace git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun => ../mesh-delivery-builder/buildrun
require git.novox.be/novox/mesh-delivery/sdk v0.0.0-00010101000000-000000000000
replace git.novox.be/novox/mesh-delivery/sdk => ../../sdk
+2 -2
View File
@@ -1,7 +1,7 @@
git.novox.be/novox/mesh-catalog/modules/mesh-build-go/buildstep v0.0.0-20261102193506-d221c633799a h1:oxgksCiq272/afXbUION/dxtAS5MPGnmyq6+AZQxmXg=
git.novox.be/novox/mesh-catalog/modules/mesh-build-go/buildstep v0.0.0-20261102193506-d221c633799a/go.mod h1:kpsX4Pgss5EeuBvel1zWa2ogj7MkR3jZso6oa/J0YUQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929 h1:RtAFbUJRP1qO9AIMM2ue+cXNzqj7WZ7IOlSqNvttwuU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -0,0 +1,111 @@
package check
import (
"context"
"fmt"
"maps"
"slices"
"strings"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-delivery/sdk/graph"
)
// What the check builds of a candidate before it composes (novox/hq ADR 0603 §10). The builder builds every fingerprint the
// indexer's candidate carries, as a key of its own (builds.<module>.<fingerprint>), with the build modules that run; the
// check adds no build path of its own and waits on those keys (awaitBuilds), so the node-engines compose the candidate's
// modules with their builds. It does not wait on two kinds of changed module, and names them in its composition's warnings:
//
// - one built by a build module the change alters (the recipe of mesh-build-go or mesh-build-node enters the fingerprint
// of every module it builds, ADR 0530 §1): its candidate build is the running build module's, and the check cannot run
// the candidate's build module to build it (it runs it only for its own digest and tests, ADR 0535 §1), so it is not
// composed with a build of the changed build module; it is built again once the agreed toolchain moves (ADR 0603 §3).
// The build module's own build is waited on, as any other: the running build module builds its new source.
// - one that stands on a module the change alters (build.on): its candidate fingerprint names the base's trunk
// fingerprint, and no build module builds on another module's artifact (ADR 0487 §1), so its build stands on the
// trunk's base; the base's own build is waited on.
//
// A module the trunk holds no record of yet is not built by the builder until it is merged, so it is not waited on either.
// buildsOfTheCheck are the builds the check waits on before it composes (module to fingerprint) and the words for the
// changed modules it composes without waiting on their build, one line each, sorted. A manifest that cannot be read leaves
// every build waited on and nothing said of build modules or bases (nothing is said that was not read).
func (st *state) buildsOfTheCheck(ctx context.Context) (map[string]string, []string) {
wait := map[string]string{}
changed := map[string]catalogue.Manifest{}
readable := true
for _, p := range st.c.Pulls {
for module, cm := range p.Fps {
if cm.FP2 == "" || cm.Manifest == "" {
continue // the builder builds no fingerprint of it (candidates.go: none of the new form, or no manifest)
}
wait[module] = cm.FP2
if !readable || st.j.Read == nil {
readable = false
continue
}
m, err := st.j.Read.Manifest(ctx, cm.Manifest)
if err != nil {
st.j.logf("%s: the manifest of %s cannot be read, so the modules built with a changed build module or base are not named and every build is waited on: %v", st.c.Key, module, err)
readable = false
continue
}
changed[module] = m
}
}
var words []string
if st.j.Runs != nil {
var unheld []string
for _, module := range slices.Sorted(maps.Keys(wait)) {
if st.j.Runs.CommitOf(module) == "" {
unheld = append(unheld, module)
delete(wait, module)
}
}
if len(unheld) > 0 {
words = append(words, fmt.Sprintf("%s: the trunk holds no record of it yet, so the builder builds it once it is merged and the check composes it without a build (ADR 0603 §10)",
strings.Join(unheld, ", ")))
}
}
if !readable {
return wait, words
}
for _, bm := range slices.Sorted(maps.Keys(buildrun.LanguageOf)) {
if _, alters := changed[bm]; !alters {
continue
}
var built []string
for module, m := range changed {
if _, isBuildModule := buildrun.LanguageOf[module]; !isBuildModule && buildsWith(m, buildrun.LanguageOf[bm]) {
built = append(built, module)
delete(wait, module)
}
}
slices.Sort(built)
if len(built) > 0 {
words = append(words, fmt.Sprintf("the change alters the build module %s: %s, which it builds, are built by the running %s, not the changed one, which the check cannot run, so the check does not wait on their builds; they are built again once the agreed toolchain moves (ADR 0603 §3, §10)",
bm, strings.Join(built, ", "), bm))
}
}
g := graph.OfManifests(changed)
for _, base := range slices.Sorted(maps.Keys(changed)) {
if on := g.Dependents(base); len(on) > 0 {
for _, module := range on {
delete(wait, module)
}
words = append(words, fmt.Sprintf("the change alters %s, a base of %s: their builds stand on the trunk's %s, since no build module builds on a candidate's base, so the check waits on the build of %s alone (ADR 0603 §10)",
base, strings.Join(on, ", "), base, base))
}
}
return wait, words
}
// buildsWith says whether a manifest has a bundle in the language: the artifacts whose fingerprint names that language's
// toolchain and its build module's recipe (an archive is packed with no toolchain, ADR 0478 clause 1).
func buildsWith(m catalogue.Manifest, language string) bool {
return m.Build != nil && slices.ContainsFunc(m.Build.Artifacts, func(a catalogue.Artifact) bool {
return a.Kind == catalogue.ArtifactBundle && a.Language == language
})
}
@@ -2,7 +2,12 @@ package check
import (
"context"
"fmt"
"maps"
"slices"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
@@ -64,3 +69,102 @@ func (j *Examiner) buildNotes(ctx context.Context, c *Case) []BuildNote {
}
return notes
}
// DefaultBuildWait is how long a check waits for the candidate's builds to end before it composes (the setting
// build-wait-bound, novox/hq ADR 0603 §10): past it the check composes without the builds not ended and names them.
const DefaultBuildWait = 30 * time.Minute
// BuildEnds waits on the builder's build keys (builds.<module>.<fingerprint>), hearing each key's changes on the bus as the
// builder writes them: the builder owns the fact, and nothing is polled.
type BuildEnds interface {
// Ended waits until the build of every module at its fingerprint (module to fingerprint) is done or failed, or ctx ends,
// and answers the builds that ended, by module. An error is only builds that cannot be heard at all.
Ended(ctx context.Context, builds map[string]string) (map[string]delivery.BuilderBuild, error)
// Done waits until the build of every module at its fingerprint is done (a failed one built again, builder_build_again),
// and says whether they all are; false when ctx ends first or they cannot be heard.
Done(ctx context.Context, builds map[string]string) bool
}
// BuildFailedTest is what a failure of the check names a failed build of the candidate by: the test `build: <module>`.
// A verdict failed on builds is asked again by the checker itself once each is built again (Checker.Rebuilt).
const BuildFailedTest = "build: "
// failedBuilds are the candidate's builds a failed verdict names as failed (BuildFailedTest), by module, at the fingerprint
// the case carries for it; none for a verdict that names no failed build.
func failedBuilds(v delivery.Verdict, cs *Case) map[string]string {
if v.State != delivery.VerdictFail {
return nil
}
out := map[string]string{}
for _, f := range append([]delivery.FailedFirst{v.Failed}, v.Failures...) {
module, named := strings.CutPrefix(f.Test, BuildFailedTest)
if !named {
continue
}
for _, p := range cs.Pulls {
if cm, ok := p.Fps[module]; ok && cm.FP2 != "" {
out[module] = cm.FP2
}
}
}
if len(out) == 0 {
return nil
}
return out
}
func (j *Examiner) buildWait() time.Duration {
if j.BuildWait > 0 {
return j.BuildWait
}
return DefaultBuildWait
}
// awaitBuilds waits for the candidate's builds before the check composes (ADR 0603 §10): the check builds what the builder
// builds of the candidate and composes with it. A failed build fails the check, naming the module and the builder's words;
// a build not ended within the bound is named, and the check composes without it. It answers the words of the composition's
// warnings: what the check composes without waiting on its build, and why.
func (st *state) awaitBuilds(ctx context.Context, v *delivery.Verdict) []string {
wait, words := st.buildsOfTheCheck(ctx)
if len(wait) == 0 {
return words
}
modules := slices.Sorted(maps.Keys(wait))
if v.State == delivery.VerdictFail {
// The check has failed already: its red status is never held back by a build (the lead, 2026-11-03).
return append(words, fmt.Sprintf("the check failed before its builds were waited on, so it composes at once, without waiting on the builds of %s (ADR 0603 §10)", strings.Join(modules, ", ")))
}
if st.j.BuildEnds == nil {
return append(words, fmt.Sprintf("the builds of %s cannot be heard, so the check composes without waiting on them (ADR 0603 §10)", strings.Join(modules, ", ")))
}
bound := st.j.buildWait()
waits := fmt.Sprintf("waits on the builder's builds of %s before it composes, at most %s (ADR 0603 §10)", strings.Join(modules, ", "), bound)
st.c.note(waits)
st.j.logf("%s %s", st.c.Key, waits)
at, cancel := context.WithTimeout(ctx, bound)
ended, err := st.j.BuildEnds.Ended(at, wait)
cancel()
if err != nil {
st.j.logf("%s: the builds of %s cannot be heard: %v", st.c.Key, strings.Join(modules, ", "), err)
return append(words, fmt.Sprintf("the builds of %s cannot be heard (%v), so the check composes without waiting on them (ADR 0603 §10)", strings.Join(modules, ", "), err))
}
var late []string
for _, module := range modules {
b, done := ended[module]
switch {
case !done:
late = append(late, module)
case b.State == delivery.BuildFailed:
f := delivery.FailedFirst{Test: BuildFailedTest + module, Words: fmt.Sprintf("the candidate's build at %s failed (rebuild with builder_build_again; the check asks itself again when the build is done): %s", wait[module], b.Error)}
if v.State != delivery.VerdictFail {
v.State, v.Failed = delivery.VerdictFail, f
continue
}
addFailure(v, f)
}
}
if len(late) > 0 && ctx.Err() == nil {
words = append(words, fmt.Sprintf("the builds of %s did not end within the build-wait-bound of %s, so the check composes without them (ADR 0603 §10)", strings.Join(late, ", "), bound))
}
return words
}
@@ -4,6 +4,8 @@ import (
"context"
"errors"
"fmt"
"maps"
"slices"
"sort"
"strings"
"sync"
@@ -79,6 +81,9 @@ type Checker struct {
}
// Bound is how long one check may run; none when 0 (DefaultBound).
Bound time.Duration
// Rebuilds hears the builder's build keys a failed verdict names (failedBuilds): once each is written done, the check of
// those heads is asked again by the checker itself (ADR 0603 §10), as a failure of the nodes is; nil asks none again.
Rebuilds BuildEnds
// Asks keeps the asks each running check put, by verdict key (AsksState), for checker_ask_again; nil keeps none.
Asks AsksKV
// Link is the base of the address a status points at; the verdict key follows.
@@ -104,6 +109,8 @@ type Checker struct {
held map[string]string // by verdict key: the turn that holds its check off, as last said
live map[string]liveCheck // by verdict key: the check running in this process, ended when its heads go
settling map[string]*window // by candidates key: the window its heads settle in
rebuilt map[string]bool // by verdict key: every failed build its verdict names is done again
awaiting map[string]bool // by verdict key: its failed builds are heard (awaitRebuilt)
// outsideSaid are the candidates outside the mesh said in the log, by group and heads.
outsideSaid map[string]bool
}
@@ -162,6 +169,7 @@ func (ch *Checker) init() {
ch.known, ch.inflight, ch.again, ch.tried = map[string]delivery.Candidate{}, map[string]bool{}, map[string]bool{}, map[string]bool{}
ch.nodesFail, ch.held = map[string]uint64{}, map[string]string{}
ch.live, ch.settling = map[string]liveCheck{}, map[string]*window{}
ch.rebuilt, ch.awaiting = map[string]bool{}, map[string]bool{}
ch.outsideSaid = map[string]bool{}
})
}
@@ -380,8 +388,9 @@ func (ch *Checker) consider(ctx context.Context, key string, c delivery.Candidat
// Every check is asked at once: the build seats' streams hold the backlog, a new head's check and a check after a
// failure of the nodes on check, a re-judge after a trunk move on recheck, pulled after them (ADR 0525 §4, §9;
// ADR 0536 §4). The checker holds no place and no capacity of its own.
ch.prepare(ctx, cs)
ch.prepare(ctx, key, cs)
ch.try(ctx, run, key, cs)
ch.hearFailedBuilds(ctx, key, cs) // a check it failed on a build is asked again once that build is done again
ch.mu.Lock()
more := ch.again[vkey]
delete(ch.again, vkey)
@@ -518,7 +527,7 @@ const (
// or sibling commits that moved since is carried, its packages run only where the move changed their digest, and a pass's
// commits are named as the carry's CarriedFrom. A verdict that stands is ranked with the carries: its claim finds nothing
// to do.
func (ch *Checker) prepare(ctx context.Context, cs *Case) {
func (ch *Checker) prepare(ctx context.Context, group string, cs *Case) {
cs.Kind, cs.CarriedFrom, cs.Composition = KindNew, nil, nil
value, _, found, err := ch.Verdicts.Get(ctx, cs.Key)
if err != nil || !found {
@@ -531,6 +540,10 @@ func (ch *Checker) prepare(ctx context.Context, cs *Case) {
case IsNodesFailure(v):
cs.Kind = KindNodes
return
case failedBuilds(v, cs) != nil:
cs.Kind = KindNodes // asked again on check once its builds are done again (awaitRebuilt), never by a recheck
ch.awaitRebuilt(ctx, group, cs.Key, failedBuilds(v, cs))
return
case IsConflictFailure(v):
return // a head git refused: its next check is whole (ADR 0548 §7)
case IsEnded(v), IsUntrunked(v):
@@ -687,6 +700,8 @@ func (ch *Checker) claim(ctx context.Context, cs *Case, first bool, epoch uint64
return nil, nil
case IsEnded(cur), ch.stale(cur, cs):
retake = true // heads held again after their check was ended unjudged (issue 1890), or a trunk that moved
case cur.State == delivery.VerdictFail && ch.takeRebuilt(cs.Key):
retake = true // every failed build it names is done again (ADR 0603 §10)
case cur.State == delivery.VerdictFail && IsNodesFailure(cur):
ch.mu.Lock()
at, ok := ch.nodesFail[cs.Key]
@@ -1070,3 +1085,61 @@ func (ch *Checker) movedSince(cs *Case) bool {
}
return false
}
// awaitRebuilt hears, once per verdict key, the builder's keys of the builds a failed verdict names: when each is written
// done again (builder_build_again), the check of those heads is asked again (claim retakes it), as a failure of the nodes
// is after the nodes changed, so no recheck by hand is needed (ADR 0603 §10). Heard on the builder's own writes, never
// polled; it ends with the checker.
func (ch *Checker) awaitRebuilt(ctx context.Context, group, vkey string, builds map[string]string) {
if ch.Rebuilds == nil {
return
}
ch.mu.Lock()
if ch.awaiting[vkey] {
ch.mu.Unlock()
return
}
ch.awaiting[vkey] = true
ch.mu.Unlock()
ch.wg.Add(1)
go func() {
defer ch.wg.Done()
done := ch.Rebuilds.Done(ctx, builds)
ch.mu.Lock()
delete(ch.awaiting, vkey)
if done {
ch.rebuilt[vkey] = true
}
c, known := ch.known[group]
ch.mu.Unlock()
if done && known && ctx.Err() == nil {
ch.logf("%s: the builds of %s it failed on are done again, so it is checked again (ADR 0603 §10)", vkey, strings.Join(slices.Sorted(maps.Keys(builds)), ", "))
ch.consider(ctx, group, c)
}
}()
}
// hearFailedBuilds starts awaitRebuilt for the case's verdict when it names failed builds (failedBuilds).
func (ch *Checker) hearFailedBuilds(ctx context.Context, group string, cs *Case) {
if ch.Rebuilds == nil || ctx.Err() != nil {
return
}
value, _, found, err := ch.Verdicts.Get(ctx, cs.Key)
if err != nil || !found {
return
}
if v, err := delivery.DecodeVerdict(value); err == nil {
if builds := failedBuilds(v, cs); builds != nil {
ch.awaitRebuilt(ctx, group, cs.Key, builds)
}
}
}
// takeRebuilt says whether every failed build the verdict of vkey names is done again, and forgets it: one retake each.
func (ch *Checker) takeRebuilt(vkey string) bool {
ch.mu.Lock()
defer ch.mu.Unlock()
was := ch.rebuilt[vkey]
delete(ch.rebuilt, vkey)
return was
}
@@ -3,9 +3,12 @@ package check
import (
"context"
"errors"
"maps"
"slices"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
@@ -30,6 +33,7 @@ func (f *fakeCompositions) Compose(ctx context.Context, c *Case) (delivery.Compo
// 1297/AC3 (its unit half): a check whose tests pass takes the assigner's composition of it, by the check's own key,
// into its verdict: a pass says it passed with the mesh-sdk and machines it composed, and is no longer "not run".
func TestAPassingCheckTakesTheAssignersComposition(t *testing.T) {
t.Run("a change to a build module or base names the modules built on it", aChangeToABuildModuleOrBaseNamesTheModulesBuiltOnIt)
b := newBench(t)
f := &fakeCompositions{v: delivery.CompositionVerdict{Group: "feat/x", Heads: "0123456789ab", State: delivery.CompositionPass, SDK: "v0.1.14-x", Machines: []string{"g14"}, Took: 40}}
b.ex.Compositions = f
@@ -99,3 +103,148 @@ func TestACompositionNotHadIsNotRun(t *testing.T) {
type fixedTrunks struct{ heads map[string]string }
func (f fixedTrunks) Heads() map[string]string { return f.heads }
// fakeEnds answers the builder's builds of a candidate as fakeBuilds holds them, recording the builds waited on; the
// composition must not have been asked yet when it is (ADR 0603 §10: the check composes with the builds).
type fakeEnds struct {
builds fakeBuilds
asked map[string]string
composed *fakeCompositions
early bool
}
func (f *fakeEnds) Ended(_ context.Context, builds map[string]string) (map[string]delivery.BuilderBuild, error) {
f.asked = builds
f.early = f.composed != nil && len(f.composed.asked) > 0
out := map[string]delivery.BuilderBuild{}
for module, fp := range builds {
if b, ok := f.builds[module+" "+fp]; ok && (b.State == delivery.BuildDone || b.State == delivery.BuildFailed) {
out[module] = b
}
}
return out, nil
}
func (f *fakeEnds) Done(context.Context, map[string]string) bool { return false }
// knownRuns are the modules the trunk holds a record of (the indexer's modules): any commit.
type knownRuns []string
func (k knownRuns) CommitOf(module string) string {
if slices.Contains(k, module) {
return trunkA
}
return ""
}
// ADR 0603 §10: before it composes, the check waits on the builder's builds of the candidate's changed modules and composes
// with them; a failed build fails the check naming it, and one not ended within the bound is named. It does not wait on a
// module built by a build module the change alters, nor on one standing on a base the change alters, nor on one the trunk
// holds no record of, and names each in the composition's warnings (and so the check's line); a change that alters none of
// these says nothing.
func aChangeToABuildModuleOrBaseNamesTheModulesBuiltOnIt(t *testing.T) {
dig := func(c string) string { return strings.Repeat(c, 64) }
fp := "src2:" + dig("f")
goBuild := &catalogue.Build{Artifacts: []catalogue.Artifact{{Name: "bin", Kind: catalogue.ArtifactBundle, Language: "go"}}}
tsBuild := &catalogue.Build{Artifacts: []catalogue.Artifact{{Name: "app", Kind: catalogue.ArtifactBundle, Language: "typescript"}}}
done := delivery.BuilderBuild{State: delivery.BuildDone}
for _, row := range []struct {
name string
manifest map[string]catalogue.Manifest
builds map[string]delivery.BuilderBuild // by module, at fp; nil leaves the check without the builds' ends
runs knownRuns // nil: the trunk holds every module
want []string
waited string // the modules waited on, sorted
failed string // the failure's test, when the check fails on a build
tests bool // the tests fail: the check fails before it would wait
}{
{name: "the Go build module changes",
manifest: map[string]catalogue.Manifest{"mesh-build-go": {Module: "mesh-build-go", Build: goBuild}, "web": {Module: "web", Build: goBuild},
"mail": {Module: "mail", Build: goBuild}, "docs": {Module: "docs"}},
builds: map[string]delivery.BuilderBuild{"mesh-build-go": done, "docs": done},
want: []string{"the build module mesh-build-go: mail, web, which it builds"}, waited: "docs, mesh-build-go"},
{name: "the Node build module changes",
manifest: map[string]catalogue.Manifest{"mesh-build-node": {Module: "mesh-build-node", Build: goBuild}, "panel": {Module: "panel", Build: tsBuild},
"web": {Module: "web", Build: goBuild}},
builds: map[string]delivery.BuilderBuild{"mesh-build-node": done, "web": done},
want: []string{"the build module mesh-build-node: panel, which it builds"}, waited: "mesh-build-node, web"},
{name: "a base changes",
manifest: map[string]catalogue.Manifest{"image": {Module: "image"},
"web": {Module: "web", Build: &catalogue.Build{On: []catalogue.BuildsOn{{Arg: "BASE", Module: "image", Artifact: "img"}}}}},
builds: map[string]delivery.BuilderBuild{"image": done},
want: []string{"alters image, a base of web", "waits on the build of image alone"}, waited: "image"},
{name: "neither changes", manifest: map[string]catalogue.Manifest{"web": {Module: "web", Build: goBuild}},
builds: map[string]delivery.BuilderBuild{"web": done}, waited: "web"},
{name: "a build fails", manifest: map[string]catalogue.Manifest{"web": {Module: "web", Build: goBuild}, "mail": {Module: "mail", Build: goBuild}},
builds: map[string]delivery.BuilderBuild{"web": {State: delivery.BuildFailed, Error: "go build: undefined: x"}, "mail": done},
waited: "mail, web", failed: "build: web"},
{name: "the tests fail", manifest: map[string]catalogue.Manifest{"web": {Module: "web", Build: goBuild}},
builds: map[string]delivery.BuilderBuild{"web": {State: delivery.BuildBuilding}}, tests: true,
want: []string{"composes at once, without waiting on the builds of web"}},
{name: "a build does not end within the bound", manifest: map[string]catalogue.Manifest{"web": {Module: "web", Build: goBuild}},
builds: map[string]delivery.BuilderBuild{"web": {State: delivery.BuildBuilding}},
want: []string{"the builds of web did not end within the build-wait-bound"}, waited: "web"},
{name: "a module the trunk does not hold", manifest: map[string]catalogue.Manifest{"web": {Module: "web", Build: goBuild}, "new": {Module: "new", Build: goBuild}},
builds: map[string]delivery.BuilderBuild{"web": done}, runs: knownRuns{"web"},
want: []string{"new: the trunk holds no record of it yet"}, waited: "web"},
{name: "the builds cannot be heard", manifest: map[string]catalogue.Manifest{"web": {Module: "web", Build: goBuild}},
want: []string{"the builds of web cannot be heard"}},
} {
b := candidateBench(t, true)
if row.tests {
b.test = func(c stepCall) (any, error) {
r := captured[Tested](t, "go-test-failing")
r.Level, _ = c.args["level"].(string)
return r, nil
}
}
comps := &fakeCompositions{v: delivery.CompositionVerdict{Group: "feat/x", Heads: "0123456789ab", State: delivery.CompositionPass, SDK: "x", Machines: []string{"g14"}}}
b.ex.Compositions = comps
var ends *fakeEnds
if row.builds != nil {
ends = &fakeEnds{builds: fakeBuilds{}, composed: comps}
for module, build := range row.builds {
ends.builds[module+" "+fp] = build
}
b.ex.BuildEnds = ends
}
if row.runs != nil {
b.ex.Runs = row.runs
}
read, fps := fixedManifests{}, map[string]delivery.CandidateModule{}
for module, m := range row.manifest {
d := dig(string(rune('a' + len(read))))
read[d], fps[module] = m, delivery.CandidateModule{FP2: fp, Manifest: d}
}
b.ex.Read = read
p := pullOf(catalogueRepository, 9, head1)
p.Fps, p.Modules = fps, slices.Sorted(maps.Keys(fps))
v := b.judge(t, caseOf(t, p)).Verdict
if v.Composition == nil || len(comps.asked) != 1 {
t.Fatalf("%s: composed %d times: %+v", row.name, len(comps.asked), v)
}
if ends != nil {
if got := strings.Join(slices.Sorted(maps.Keys(ends.asked)), ", "); got != row.waited || ends.early {
t.Errorf("%s: waited on %q (after the composition: %v), want %q before it composes", row.name, got, ends.early, row.waited)
}
}
switch {
case row.failed != "" && (v.State != delivery.VerdictFail || v.Failed.Test != row.failed || !strings.Contains(v.Failed.Words, "undefined: x") ||
!strings.Contains(v.Failed.Words, "the check asks itself again when the build is done")):
t.Errorf("%s: the verdict is %s failing %+v, want it failed naming %s, the builder's words and how it is asked again", row.name, v.State, v.Failed, row.failed)
case row.tests && v.State != delivery.VerdictFail:
t.Errorf("%s: %+v, want the tests' failure", row.name, v)
case row.failed == "" && !row.tests && v.State != delivery.VerdictPass:
t.Errorf("%s: %+v", row.name, v)
}
words := compositionWords(v.Composition, v.Took.Composition)
for _, w := range row.want {
if !strings.Contains(words, w) {
t.Errorf("%s: the line says %q, want it to name %q", row.name, words, w)
}
}
if row.want == nil && v.Composition.Words != "" {
t.Errorf("%s: warned %q though the check waited on every build of the change", row.name, v.Composition.Words)
}
}
}
@@ -147,6 +147,12 @@ func JudgeEngines(group, heads, composerName string, nodes []string, answers map
v.Warnings = append(v.Warnings, node+": "+delivery.ComposeNoAnswer)
continue
}
if a.Verdict == delivery.ComposeCandidateNotJudged {
// Composed in the sandbox and not judged (ADR 0593 §9): named as no answer, with the composer's words; decided by the
// verdict, never by the words (ADR 0601 §4's kind).
v.Warnings = append(v.Warnings, node+": "+delivery.ComposeNoAnswer+" ("+a.Refused+")")
continue
}
v.Machines = append(v.Machines, node)
if a.Verdict != delivery.ComposeCandidateRefused {
continue
@@ -221,8 +227,11 @@ func (e EngineCompositions) inSandbox(ctx context.Context, c *Case, head, node s
return a, fmt.Errorf("the composer at the node-engine's head %s did not answer for %s: %w", head, node, err)
}
out := delivery.ComposeCandidateAnswer{V: a.V, Node: node, Candidate: a.Candidate, Verdict: delivery.ComposeCandidateComposed}
notJudged := ""
for _, cv := range verdict.Cases {
if m, ok := cv.Machines[node]; ok && !m.Composes {
if m, ok := cv.Machines[node]; ok && m.NotJudged != "" {
notJudged = m.NotJudged // the composer did not judge the node: neither composed nor refused (ADR 0593 §9)
} else if ok && !m.Composes {
out.Verdict = delivery.ComposeCandidateRefused
out.Refused = strings.Join(append(append([]string{m.Reason}, m.Problems...), m.Refusals...), "; ")
}
@@ -234,5 +243,8 @@ func (e EngineCompositions) inSandbox(ctx context.Context, c *Case, head, node s
}
}
sort.Slice(out.Modules, func(i, j int) bool { return out.Modules[i].Module < out.Modules[j].Module })
if notJudged != "" && out.Verdict == delivery.ComposeCandidateComposed {
out.Verdict, out.Refused = delivery.ComposeCandidateNotJudged, notJudged
}
return out, nil
}
@@ -168,6 +168,25 @@ func TestTheCompositionIsMadeFromTheEnginesAnswers(t *testing.T) {
if v.State != delivery.CompositionFail || v.SDK != head2 || !strings.Contains(v.Words, "the new composer refuses it") {
t.Errorf("verdict = %+v", v)
}
// ADR 0593 §9 (ADR 0601 §4's kind): a node the composer did not judge is named as no answer with its words, by the
// contract's field, and the check passes; words in Reason alone are no such mark.
sb.err = nil
for _, row := range []struct {
m composer.Machine
pass bool
}{
{composer.Machine{NotJudged: "its node-engine sends no running state for web"}, true},
{composer.Machine{Reason: "not judged: its node-engine sends no running state for web"}, false},
} {
sb.v = func(node string) composer.Verdict {
return composer.Verdict{V: composer.Version, Cases: []composer.CaseVerdict{{Name: "change", Machines: map[string]composer.Machine{node: row.m}}}}
}
v, err = ec.Compose(context.Background(), host)
named := len(v.Warnings) == 1 && strings.Contains(v.Warnings[0], "anchor: "+delivery.ComposeNoAnswer+" (its node-engine sends no running state for web)")
if err != nil || (v.State == delivery.CompositionPass) != row.pass || named != row.pass {
t.Errorf("%+v: verdict %+v, %v", row.m, v, err)
}
}
// Review finding 3 (ADR 0593 §9 lets only a node that does not answer pass): a composer at the head that cannot
// run fails the composition, naming the head, never passing as a silent node.
sb.err = errors.New("the build of mesh-compose failed")
@@ -20,6 +20,7 @@ import (
gobuild "git.novox.be/novox/mesh-catalog/modules/mesh-build-go/buildstep"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/composer"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
@@ -1000,6 +1001,17 @@ func (k keptList) PredefinedList(context.Context) (json.RawMessage, error) {
return json.RawMessage(k), nil
}
// fixedManifests are the indexer's kept manifests by digest as a test gives them.
type fixedManifests map[string]catalogue.Manifest
func (f fixedManifests) Manifest(_ context.Context, digest string) (catalogue.Manifest, error) {
m, ok := f[digest]
if !ok {
return catalogue.Manifest{}, errors.New("no manifest " + digest)
}
return m, nil
}
// fixedDigests are the indexer's manifest digests as a test gives them.
type fixedDigests map[string]string
@@ -1017,17 +1029,33 @@ func TestTheIndexersJudgeJudgesTheChangesManifests(t *testing.T) {
// untouched is the judge's refusal of mail, which the change leaves as it is, and trunkMail its refusal on the trunk:
// a problem the trunk had is a warning, one the change causes fails (ADR 0579 §2)
untouched, trunkMail string
// cycle reads the manifests, so the change's module graph is worked out (ADR 0603 §2): the change's web awaits
// mail's site while mail awaits web's database; trunkCycle has the trunk's web await it too.
cycle, trunkCycle bool
}{
{name: "accepted"},
{name: "an untouched manifest's problem the trunk had", untouched: "mail says no credential form", trunkMail: "mail says no credential form"},
{name: "a problem the change causes in an untouched manifest", untouched: "mail reads a state web no longer keeps",
fails: "refuses the manifest of mail: mail reads a state web no longer keeps"},
{name: "a cycle the change makes", cycle: true, fails: "the change's module graph has the cycle mail -> web -> mail"},
{name: "a cycle the trunk has", cycle: true, trunkCycle: true},
{name: "refused", refused: "web requires \"amqp\"\nweb has no id", fails: "the indexer's judge at mesh-delivery's trunk " + strings.Repeat("d", 40) + ` refuses the manifest of web: web requires "amqp"; web has no id`},
} {
b := newBench(t)
b.ex.Manifests = fixedDigests{"mail": strings.Repeat("a", 64), "web": strings.Repeat("b", 64)}
b.ex.Trunks = fixedTrunks{heads: map[string]string{DeliveryRepository: trunk}}
b.ex.Predefined = keptList(`{"font-size": {"type": "number", "description": "the size"}}`)
if row.cycle {
offer := func(name string) []catalogue.Offer {
return []catalogue.Offer{{Name: name, Scope: catalogue.ScopeMesh, AnsweredBy: catalogue.AnsweredByHelper}}
}
trunkWeb := catalogue.Manifest{Module: "web", Provides: offer("database")}
if row.trunkCycle {
trunkWeb.Requires = []string{"site"}
}
b.ex.Read = fixedManifests{strings.Repeat("a", 64): {Module: "mail", Provides: offer("site"), Requires: []string{"database"}},
strings.Repeat("b", 64): trunkWeb, web: {Module: "web", Provides: offer("database"), Requires: []string{"site"}}}
}
var judged []delivery.ComposeJob
answer := b.asks.Answer
b.asks.Answer = func(a delivery.StepAsk) delivery.Step {
@@ -0,0 +1,74 @@
package check
import (
"context"
"fmt"
"maps"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/composer"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-delivery/sdk/graph"
)
// The check refuses a change whose module graph has a cycle and names it (novox/hq ADR 0603 §2): modules that would each
// wait for the other for good, through their bases or the provisions they await. The graph is worked out from the
// catalogue as the change leaves it (every trunk manifest, each one a pull request touches replaced by its candidate's),
// with the contract's ManifestGraph, the way the indexer works out its graph's edges. A cycle the trunk already has among
// the same modules is the trunk's: said in the verdict's notes, not the change's failure.
// ManifestReader reads a kept manifest by its digest (the indexer's manifests state).
type ManifestReader interface {
Manifest(ctx context.Context, digest string) (catalogue.Manifest, error)
}
// refuseCycle fails the check for a cycle of the change's module graph, naming it; nil when the case touches no manifest,
// when nothing reads manifests, or when there is none. A manifest that cannot be read fails the check, said, never passes it.
func (st *state) refuseCycle(ctx context.Context) *delivery.Verdict {
if st.j.Manifests == nil || st.j.Read == nil || st.c.Try != nil {
return nil
}
trunk := st.j.Manifests.ManifestDigests()
req, touched := judgeRequest(trunk, st.c.Pulls)
if len(touched) == 0 {
return nil
}
read := func(digests map[string]composer.Manifest) (map[string]catalogue.Manifest, error) {
out := map[string]catalogue.Manifest{}
for _, module := range slices.Sorted(maps.Keys(digests)) {
m, err := st.j.Read.Manifest(ctx, digests[module].Digest)
if err != nil {
return nil, fmt.Errorf("the manifest %s of %s: %w", digests[module].Digest, module, err)
}
out[module] = m
}
return out, nil
}
change, err := read(req.Cases[0].Manifests)
if err != nil {
return fail(goModule, "the change's module graph cannot be worked out, so no cycle can be ruled out: "+err.Error())
}
cycle := graph.OfManifests(change).Cycle()
if cycle == nil {
st.c.note("the change's module graph has no cycle")
return nil
}
words := strings.Join(cycle, " -> ")
if before, err := read(req.Cases[1].Manifests); err == nil {
among := graph.Graph{}
for name, n := range graph.OfManifests(before) {
if slices.Contains(cycle, name) {
among[name] = n
}
}
if among.Cycle() != nil {
st.c.note("the module graph has the cycle " + words + ", which the trunk already has among the same modules (ADR 0603 §2)")
return nil
}
}
return fail(goModule, fmt.Sprintf("the change's module graph has the cycle %s: these modules would each wait for the other for "+
"good, through their bases or the provisions they await (ADR 0603 §2)", words))
}
@@ -102,6 +102,11 @@ type Examiner struct {
// Builds says how the build of a fingerprint the candidate carries stands (the builder's builds bucket); nil says nothing
// of the builds.
Builds BuildStates
// BuildEnds waits on the builder's builds of the candidate before the check composes (ADR 0603 §10); nil composes
// without waiting, said in the composition's warnings.
BuildEnds BuildEnds
// BuildWait is how long the check waits for those builds (the setting build-wait-bound); DefaultBuildWait when 0.
BuildWait time.Duration
// Announce says a preview was written under the key (EventPreviewWritten); nil says nothing.
Announce Announcer
// Packages names, of a repository, the paths (repository-relative) its TypeScript package artifacts are built from, from
@@ -114,6 +119,9 @@ type Examiner struct {
// Manifests are the indexer's manifest digests by module: the catalogue a change's manifests are judged in by the
// indexer's judge (manifests_judge.go, novox/hq ADR 0578 §2); nil judges no manifest.
Manifests TrunkManifests
// Read reads a kept manifest by its digest: the change's module graph is worked out from them (graph_cycle.go, novox/hq
// ADR 0603 §2); nil works out no graph.
Read ManifestReader
// Predefined reads the catalogue's predefined keys the indexer keeps (manifests_judge.go); nil reads none.
Predefined PredefinedList
// Compositions makes a check's composition (ADR 0593 §7): the engines' answers (EngineCompositions); nil leaves the
@@ -242,7 +250,7 @@ func (j *Examiner) Judge(ctx context.Context, c *Case) (Outcome, error) {
// Every member and level runs to its end, and every manifest the change touches is judged by the indexer's judge (ADR
// 0578 §2), also after a failure: the verdict names every failure, not only the first (the operator, 2026-11-02).
tests := st.judgeAll(ctx)
if v := settled(joinFailures(tests, st.judgeManifests(ctx))); v != nil {
if v := settled(joinFailures(tests, st.judgeManifests(ctx), st.refuseCycle(ctx))); v != nil {
return st.finish(ctx, *v), nil
}
return st.finish(ctx, delivery.Verdict{State: delivery.VerdictPass}), nil
@@ -277,6 +285,7 @@ func (st *state) composed(ctx context.Context, v *delivery.Verdict) {
strings.Join(st.unpreviewed, ", ") + " (its head does not merge or its fetch did not answer), so the change cannot be composed"}
return
}
words := st.awaitBuilds(ctx, v) // ADR 0603 §10: the check composes with the candidate's builds
comp, err := st.j.Compositions.Compose(ctx, st.c)
if err != nil {
// Said in the verdict, never left as a composition not yet run (review finding 4).
@@ -284,6 +293,7 @@ func (st *state) composed(ctx context.Context, v *delivery.Verdict) {
v.Composition = &delivery.Composition{State: delivery.CompositionNotRun, Words: "not made: " + err.Error()}
return
}
comp.Warnings = append(comp.Warnings, words...) // ADR 0603 §10
withComposition(v, comp)
if v.Composition != nil && v.Composition.State != delivery.CompositionNotRun {
v.Composition.FPSet = st.c.FPSet // a carry keeps it while the fingerprints stay the same (ADR 0548 §5)
@@ -3,6 +3,7 @@ package check
import (
"encoding/json"
"fmt"
"slices"
"sort"
"strings"
"time"
@@ -47,6 +48,9 @@ type Reader struct {
Runs interface{ CommitOf(module string) string }
// HearingState is the checker's state hearing: whether it hears DEAD_LETTERS (issue 734).
HearingState StateRead
// BuildState is the builder's builds (mesh-delivery-builder.builds): the builds of a running check's candidate that have
// not ended, which the check waits on before it composes (ADR 0603 §10); without it none is named.
BuildState StateRead
// AskState is the checker's state asks: the asks each running check put (AsksState), which checker_ask_again reads.
AskState StateRead
Now func() time.Time
@@ -344,6 +348,10 @@ type Status struct {
// Lost are the candidates whose verdict is running while no step of theirs can still run and their newest step ended, by
// group, with that step: nothing will judge them (the lost check of 2026-10-28); checker_ask_again asks one again.
Lost map[string]string `json:"lost"`
// BuildsNotEnded are, of each candidate whose check runs, the builder's builds of its changed modules that are neither
// done nor failed, by group: `<module> <state>`, sorted (no key yet is `not begun`). A check waits on them before it
// composes (ADR 0603 §10), but for the modules its line names as composed without them; builder_build_again builds a failed one again, and the check asks itself again when it is done.
BuildsNotEnded map[string]string `json:"builds_not_ended,omitempty"`
// DeadLetters says whether the checker hears the asks the build seats give up on (issue 734): deaf, a try or check
// whose ask is given up waits until its bound.
DeadLetters DeadLettersHeard `json:"dead_letters"`
@@ -500,6 +508,12 @@ func (r Reader) Status() (Status, error) {
if why := read.lostWhy(c); why != "" {
st.Lost[c.Group] = why
}
if words := r.buildsNotEnded(c); words != "" {
if st.BuildsNotEnded == nil {
st.BuildsNotEnded = map[string]string{}
}
st.BuildsNotEnded[c.Group] = words
}
case IsNodesFailure(v):
st.Waiting[KindNodes] = append(st.Waiting[KindNodes], c.Group)
case IsUntrunked(v):
@@ -663,3 +677,38 @@ func (r Reader) leftWhy(module string, step delivery.Step, now time.Time, bound
}
return ""
}
// buildsNotEnded are the builder's builds of the candidate's changed modules that are neither done nor failed, as
// Status.BuildsNotEnded says them; empty when none, or when the builder's builds are not readable here.
func (r Reader) buildsNotEnded(c delivery.Candidate) string {
if r.BuildState.Get == nil {
return ""
}
var out []string
for _, p := range c.Pulls {
for module, cm := range p.Fps {
if cm.FP2 == "" || cm.Manifest == "" {
continue
}
key, err := delivery.BuilderBuildKey(module, cm.FP2)
if err != nil {
continue
}
raw, err := r.BuildState.Get(key)
if err != nil {
continue
}
state := "not begun"
if len(raw) > 0 {
b, err := delivery.DecodeBuilderBuild(raw)
if err != nil || b.State == delivery.BuildDone || b.State == delivery.BuildFailed {
continue
}
state = b.State
}
out = append(out, module+" "+state)
}
}
sort.Strings(out)
return strings.Join(slices.Compact(out), ", ")
}
@@ -147,10 +147,24 @@ func TestTheStatusToolReadsRunningSteps(t *testing.T) {
passed, _ := delivery.EncodeStep(delivery.Step{ID: "abc2", Tool: "build_go_test", Node: "ace", State: delivery.StepPassed})
cand := candidateOf(t, "feat/x", pullOf(repoA, 7, head1))
ckey, _ := delivery.CandidatesKey("feat/x")
// ADR 0603 §10: feat/y's check runs and waits on the builds of its changed modules: web building, mail not begun; api
// is done and named nowhere.
fp, man := "src2:"+strings.Repeat("f", 64), strings.Repeat("e", 64)
py := pullOf(repoA, 8, head2)
py.Modules, py.Fps = []string{"api", "mail", "web"}, map[string]delivery.CandidateModule{"web": {FP2: fp, Manifest: man}, "mail": {FP2: fp, Manifest: man}, "api": {FP2: fp, Manifest: man}}
candY := candidateOf(t, "feat/y", py)
ykey, _ := delivery.CandidatesKey("feat/y")
runningV, _ := delivery.EncodeVerdict(delivery.Verdict{Group: "feat/y", Heads: "0123456789ab", State: delivery.VerdictRunning})
builds := fakeState{}
for module, state := range map[string]string{"web": delivery.BuildBuilding, "api": delivery.BuildDone} {
k, _ := delivery.BuilderBuildKey(module, fp)
builds[k], _ = json.Marshal(delivery.BuilderBuild{V: 1, State: state})
}
rd := Reader{
StepStates: map[string]StateRead{"mesh-build-go": fakeState{"shanks.abc1": running, "ace.abc2": passed}.read()},
CandidateState: fakeState{ckey: cand}.read(),
VerdictState: fakeState{}.read(),
CandidateState: fakeState{ckey: cand, ykey: candY}.read(),
VerdictState: fakeState{keyOf(t, "feat/y", candY): runningV}.read(),
BuildState: builds.read(),
HaveState: fakeState{"shanks.mesh-build-go": haveOf(t, delivery.PhaseRunning)}.read(),
HearingState: fakeState{"dead-letters": []byte(`{"heard":false,"streams":["SEAT_NODE_BUILD_GO"],` +
`"why":"DEAD_LETTERS is not heard, so an ask a build seat gives up on is waited for until its bound: SEAT_NODE_BUILD_GO: permissions violation",` +
@@ -167,6 +181,9 @@ func TestTheStatusToolReadsRunningSteps(t *testing.T) {
if len(st.Unjudged) != 1 || st.Unjudged[0] != "feat/x" {
t.Errorf("unjudged = %v", st.Unjudged)
}
if got := st.BuildsNotEnded; len(got) != 1 || got["feat/y"] != "mail not begun, web building" {
t.Errorf("builds not ended = %v, want feat/y's mail not begun and web building", got)
}
// Issue 734, review round 1: a checker that does not hear DEAD_LETTERS says so in its status, with why.
if st.DeadLetters.Heard || !strings.Contains(st.DeadLetters.Why, "permissions violation") {
t.Errorf("dead letters = %+v, want not heard, saying why", st.DeadLetters)
@@ -174,13 +174,16 @@ func TestManifestProcess(t *testing.T) { // 1678/AC1
if len(env) != 2 || env["MESH_BROKER_FILE"] != "${dir:state}/broker" || env["MESH_CHECKER_SETTINGS"] != "${dir:state}/settings.json" {
t.Errorf("env = %v, want the broker file and the settings file", p["env"])
}
// Its settings hold its answer bound alone (ADR 0593 §7: compose-answer-bound, 60 s, the checker's), never a capacity
// of its own (ADR 0525 §4).
// Its settings hold its answer bound (ADR 0593 §7: compose-answer-bound, 60 s, the checker's) and its wait for the
// candidate's builds (ADR 0603 §10: build-wait-bound, 1800 s), never a capacity of its own (ADR 0525 §4).
for _, r := range m.Resources {
if r["id"] == "settings" && !reflect.DeepEqual(r["takes-config"], []any{"compose-answer-bound"}) {
t.Errorf("the settings resource %v takes more than compose-answer-bound", r)
if r["id"] == "settings" && !reflect.DeepEqual(r["takes-config"], []any{"compose-answer-bound", "build-wait-bound"}) {
t.Errorf("the settings resource %v takes other than compose-answer-bound and build-wait-bound", r)
}
}
if c, ok := m.Config["build-wait-bound"]; !ok || c.Default != float64(1800) || c.Trusted {
t.Errorf("build-wait-bound = %+v, want 1800 by default, not trusted", c)
}
if c, ok := m.Config["compose-answer-bound"]; !ok || c.Default != float64(60) || c.Trusted {
t.Errorf("compose-answer-bound = %+v, want 60 by default, not trusted", c)
}
+10 -1
View File
@@ -81,6 +81,14 @@
"maximum": 600,
"trusted": false,
"default": 60
},
"build-wait-bound": {
"type": "number",
"description": "seconds a check waits for the builder's builds of its candidate to end before it composes with them (novox/hq ADR 0603 §10); a build failed fails the check, naming it, and a build not ended by then is named in the verdict and the check composes without it.",
"minimum": 60,
"maximum": 7200,
"trusted": false,
"default": 1800
}
},
"resources": [
@@ -106,7 +114,8 @@
"mode": "0600",
"owner": "mesh-delivery-checker",
"takes-config": [
"compose-answer-bound"
"compose-answer-bound",
"build-wait-bound"
],
"merge": "json",
"template": "files/settings.json"
@@ -74,10 +74,6 @@ func handle(ctx context.Context, m delivered, pub publisher, trees bridge.Trees,
logger.Printf("%v; heard again in %s", err, retryDelay)
_ = m.NakWithDelay(retryDelay)
return
case errors.Is(err, bridge.ErrNoModules):
logger.Printf("%v; no merged message", err)
_ = m.Ack()
return
case errors.Is(err, bridge.ErrNotHandled):
logger.Printf("%v; nothing written", err)
_ = m.Ack()
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-forge-bridge
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929 h1:RtAFbUJRP1qO9AIMM2ue+cXNzqj7WZ7IOlSqNvttwuU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -29,9 +29,6 @@ var ErrNotHandled = errors.New("event not handled")
// heard again later.
var ErrRetry = errors.New("the forge could not say")
// ErrNoModules says a merge into the trunk left a repository with no module.json anywhere: no merged message.
var ErrNoModules = errors.New("no module.json in the merge")
// Trunk is the branch a merged message is written for.
const Trunk = "main"
@@ -100,7 +97,9 @@ func unreadable(subject, why string, args ...any) error {
// Convert turns one event of the forge module, by its bus subject and body, into a message. An event it cannot read
// is an ErrUnreadable error naming the subject and why; an event it writes nothing for is ErrNotHandled. A merge into
// the trunk asks trees once and becomes a merged message; a merge into any other branch is a closed pull message, as the
// pull request left its group. A forge that cannot say is ErrRetry; a merge with no module.json is ErrNoModules.
// pull request left its group. A forge that cannot say is ErrRetry. A merge that leaves no module.json anywhere is a merged message with no
// trees all the same (issue 1126): the indexer takes it in and drops the repository's modules, and the view reads the
// repository as reached at it.
func Convert(subject string, data []byte, trees Trees, files Files) (Message, error) {
return ConvertKept(subject, data, trees, files, nil)
}
@@ -294,9 +293,6 @@ func merged(subject string, p pull, trees Trees, files Files, kept Kept) (Messag
if err != nil {
return Message{}, fmt.Errorf("%w: %s: the trees of %s at the merge commit %s: %v", ErrRetry, subject, repository, p.MergeSHA, err)
}
if len(dirs) == 0 {
return Message{}, fmt.Errorf("%w: %s: %s at the merge commit %s", ErrNoModules, subject, repository, p.MergeSHA)
}
names := make([]string, 0, len(dirs))
for dir := range dirs {
names = append(names, dir)
@@ -339,7 +335,11 @@ func merged(subject string, p pull, trees Trees, files Files, kept Kept) (Messag
if err != nil {
return Message{}, unreadable(subject, "%v", err)
}
return Message{Event: event, Data: body}, nil
msg := Message{Event: event, Data: body}
if len(dirs) == 0 {
msg.Said = fmt.Sprintf("%s at the merge commit %s holds no module.json: a merged message with no module directories", repository, p.MergeSHA)
}
return msg, nil
}
// openPull is an entry of gitea_open_pulls's answer (mesh-catalog modules/gitea/client.ts:369-381 openPulls, with the
@@ -269,11 +269,19 @@ func TestTreesRefusedWritesNothingAndSaysWhy(t *testing.T) { // 1236/AC3 (the co
}
}
func TestNoModuleJSONAnywhereWritesNoMergedMessage(t *testing.T) { // 1236 (notes: a repository with no module.json)
// 1126 (note 94): a merge that leaves no module.json anywhere (hq, or a repository whose last module was removed) is a merged
// message with no trees, so the indexer takes it in and the repository's reach is not held for ever.
func TestNoModuleJSONAnywhereWritesAMergedMessageWithNoTrees(t *testing.T) {
var calls []call
msg, err := Convert(mergedSubject, mergedBody(nil), treesOf(&calls, "r0", map[string]string{}, nil), anyFiles)
if !errors.Is(err, ErrNoModules) || msg.Data != nil {
t.Errorf("err = %v data = %s", err, msg.Data)
if err != nil {
t.Fatal(err)
}
f, err := delivery.DecodeForge(msg.Data)
want := delivery.Forge{V: delivery.Version, Kind: delivery.ForgeMerged, Repository: "novox/mesh-delivery", Branch: "feat/x", Number: 12,
Head: head, Merge: merge, Root: "r0"}
if err != nil || msg.Event != "forge.novox.mesh-delivery.merged" || !delivery.SameForge(f, want) || !strings.Contains(msg.Said, "no module.json") {
t.Errorf("event %q message %+v said %q, %v; want a merged message with no trees", msg.Event, f, msg.Said, err)
}
}
@@ -0,0 +1,103 @@
package main
import (
"fmt"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-delivery/sdk/graph"
"mesh-delivery-indexer/internal/index"
)
// CatalogTools are the seat mesh-catalog's four verbs as tools named <seat>.<verb>, with the descriptions and inputs the
// seat's contract gives them (novox/hq ADR 0604 §1, §2), answered from the indexer's own states. They are served once the
// indexer's manifest claims the seat, in the release after L3's (the lead's decision): until then main serves them not.
func CatalogTools(c index.CatalogRead) []stdio.Tool {
seat, ok := catalogue.SeatNamed(catalogue.CatalogSeat)
if !ok {
return nil
}
str := func(args map[string]any, k string) string { s, _ := args[k].(string); return s }
runs := map[string]func(map[string]any) (any, error){
"modules": func(map[string]any) (any, error) { return c.CatalogModulesAnswer() },
"module": func(a map[string]any) (any, error) { return c.CatalogModuleAnswer(str(a, "module")) },
"provides": func(a map[string]any) (any, error) { return c.CatalogProvidesAnswer(str(a, "provision")) },
"dependents": func(a map[string]any) (any, error) { return c.CatalogDependentsAnswer(str(a, "module")) },
}
var out []stdio.Tool
for _, v := range seat.Serves {
if run, ok := runs[v.Name]; ok {
out = append(out, stdio.Tool{Name: catalogue.CatalogSeat + "." + v.Name, Description: v.Description, Input: v.Input, Run: run})
}
}
return out
}
// catalogRead reads the indexer's modules, manifests and graph states for the seat's verbs.
func catalogRead(modules, manifests, nodes stdio.KeptState) index.CatalogRead {
return index.CatalogRead{
Modules: func() (map[string]delivery.Module, error) {
keys, err := modules.Keys()
if err != nil {
return nil, err
}
out := map[string]delivery.Module{}
for _, k := range keys {
e, err := modules.Get(k)
if err != nil {
return nil, fmt.Errorf("modules.%s: %w", k, err)
}
if e == nil {
continue
}
if m, err := delivery.DecodeModule(e.Value); err == nil {
out[k] = m
}
}
return out, nil
},
Manifest: func(digest string) (string, error) {
key, err := delivery.ManifestsKey(digest)
if err != nil {
return "", err
}
e, err := manifests.Get(key)
if err != nil {
return "", err
}
if e == nil {
return "", fmt.Errorf("the manifests state holds no %s", digest)
}
kept, err := delivery.DecodeManifest(e.Value)
return kept.JSON, err
},
Graph: func() (graph.Graph, error) {
keys, err := nodes.Keys()
if err != nil {
return nil, err
}
g := graph.Graph{}
for _, k := range keys {
e, err := nodes.Get(k)
if err != nil {
return nil, fmt.Errorf("graph.%s: %w", k, err)
}
if e == nil {
continue
}
if n, err := graph.Decode(e.Value); err == nil {
g[k] = n
}
}
return g, nil
},
}
}
// catalogTools are the seat's verbs over the indexer's own states, as main serves them once the indexer claims the seat.
func catalogTools() []stdio.Tool {
return CatalogTools(catalogRead(stdio.State("modules"), stdio.State("manifests"), stdio.State(graph.State)))
}
@@ -1,6 +1,7 @@
// Command mesh-delivery-indexer-tools serves the indexer's doing tool (novox/hq issue 1945): the runtime launches it for a
// call. indexer_take_in writes one value of the indexer's takeins state and waits for the indexer's answer there. The
// indexer is the other program: it serves no request subject (ADR 0437 §4). The read tools of the indexer's states are
// indexer is the other program: it serves no request subject (ADR 0437 §4); indexer_judge asks the same way, and
// indexer_verdicts reads the indexer's verdicts state. The read tools of the indexer's states are
// mesh-delivery-view's (indexer_module, indexer_modules, indexer_candidate, indexer_candidates).
package main
@@ -21,10 +22,23 @@ import (
// asker writes a takeins value and waits for the indexer's answer to it.
type asker func(key string, value []byte) (index.TakeInAsk, error)
// Tools are the indexer's tools.
func Tools(ask asker, now func() time.Time) []stdio.Tool {
// Tools are the indexer's tools. read is the indexer's state of verdicts, which indexer_verdicts reads; counts the old sections' count.
func Tools(ask asker, now func() time.Time, read stdio.KeptState, counts func() (index.OldSectionsCount, error)) []stdio.Tool {
str := func(args map[string]any, k string) string { s, _ := args[k].(string); return strings.TrimSpace(s) }
return []stdio.Tool{
{Name: index.OldSectionsTool,
Description: "How many accepted modules use each old section of a manifest (state, logs, streams, objects), the foundation " +
"(nats, mesh-grants, the indexer) left out: the count the judge refuses a use of a section at 0 by, as modules move " +
"their bus state into `requires` by template. Read from the indexer's oldsections state, kept as the indexer judges " +
"the manifests; an empty answer means the indexer has not judged since it started.",
Input: map[string]any{"type": "object", "properties": map[string]any{}},
Run: func(map[string]any) (any, error) {
c, err := counts()
if err != nil {
return nil, err
}
return c.Counts, nil
}},
{Name: index.TakeInTool,
Description: "Take a registered module in now: the indexer reads its manifest from the repository and commit the controller's " +
"build records register it from (at the head the gate's source names, when it names one), whatever it tried before, and " +
@@ -34,12 +48,68 @@ func Tools(ask asker, now func() time.Time) []stdio.Tool {
"indexer) is read again at that head and written when it changed. Answers stored, from which " +
"repository, commit and path, or why not (held already, no registration, the forge's error, no manifest there). " +
"The indexer reads such a module again by itself on the next source revision and merged message; this asks now. " +
"indexer_module (mesh-delivery-view) reads the result.",
"indexer_module (mesh-delivery-view) reads the result. With repository instead of module it takes in the head the gate's " +
"source names for that repository, for a merge whose merged message the indexer never heard (the forge bridge has no " +
"tool): a head whose trunk holds no module.json is taken in as a merge that leaves none (every module of the repository " +
"is dropped, one another repository's trunk holds handed over from there first, listed in handed and dropped); a head " +
"that holds modules is answered \"take each module in by name\".",
Input: map[string]any{"type": "object", "properties": map[string]any{
"module": map[string]any{"type": "string", "description": "the module's name, as the catalogue registers it"},
"repository": map[string]any{"type": "string", "description": "instead of module: the repository (owner/name) whose head is taken in"},
"why": map[string]any{"type": "string", "description": "why it is taken in now, in plain words"}}},
Run: func(args map[string]any) (any, error) {
var key string
var value []byte
var err error
if str(args, "repository") != "" && str(args, "module") == "" {
key, value, err = index.NewTakeInRepository(str(args, "repository"), str(args, "why"), stdio.CallerOf(args), now())
} else {
key, value, err = index.NewTakeIn(str(args, "module"), str(args, "why"), stdio.CallerOf(args), now())
}
if err != nil {
return nil, err
}
a, err := ask(key, value)
if err != nil {
return nil, err
}
return a.Result, nil
}},
{Name: index.VerdictsTool,
Description: "The indexer's verdicts on manifests that wait or are refused (all: every verdict), each as its verdicts state holds it: " +
"the module, the digest, the state (waits: the manifest carries a field the indexer's judge does not know yet, named in " +
"unknown, and every reader holds the module as a refused one and keeps what runs; refused: the judge's problems), the " +
"judge that made it and when. A module's name narrows it to that module. A waiting manifest is judged again at each " +
"new indexer and refused only once the newest build of the indexer on the trunk still does not know the field; " +
"indexer_judge judges now.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"module": map[string]any{"type": "string", "description": "only this module's verdicts"},
"all": map[string]any{"type": "boolean", "description": "list accepted verdicts too"}}},
Run: func(args map[string]any) (any, error) {
keys, err := read.Keys()
if err != nil {
return nil, fmt.Errorf("the verdicts state cannot be listed: %w", err)
}
all, _ := args["all"].(bool)
rows, unread := index.ListVerdicts(keys, func(key string) ([]byte, error) {
e, err := read.Get(key)
if err != nil || e == nil {
return nil, err
}
return e.Value, nil
}, str(args, "module"), all)
return map[string]any{"verdicts": rows, "unread": unread}, nil
}},
{Name: index.JudgeAgainTool,
Description: "Judge the manifests again now, as the indexer does at its start and after each merge, and answer how the module's " +
"verdict stands (waits, refused or accepted, and why). For a manifest that waits for a judge that knows its fields, or " +
"after a manifest was mended where the indexer has not heard it. A waiting manifest becomes refused here only when the " +
"running indexer is the newest build of itself the trunk holds. indexer_verdicts reads the result.",
Input: map[string]any{"type": "object", "required": []string{"module"}, "properties": map[string]any{
"module": map[string]any{"type": "string", "description": "the module's name, as the catalogue registers it"},
"why": map[string]any{"type": "string", "description": "why it is taken in now, in plain words"}}},
"why": map[string]any{"type": "string", "description": "why it is judged again now, in plain words"}}},
Run: func(args map[string]any) (any, error) {
key, value, err := index.NewTakeIn(str(args, "module"), str(args, "why"), stdio.CallerOf(args), now())
key, value, err := index.NewJudgeAgain(str(args, "module"), str(args, "why"), stdio.CallerOf(args), now())
if err != nil {
return nil, err
}
@@ -105,8 +175,22 @@ func askTakeIn(s stdio.KeptState, bound time.Duration) asker {
}
}
// heldCounts reads the count of the old sections the indexer keeps.
func heldCounts(s stdio.KeptState) func() (index.OldSectionsCount, error) {
return func() (index.OldSectionsCount, error) {
e, err := s.Get(index.OldSectionsKey)
if err != nil {
return index.OldSectionsCount{}, fmt.Errorf("reading %s.%s: %w", index.OldSectionsBucket, index.OldSectionsKey, err)
}
if e == nil {
return index.OldSectionsCount{}, fmt.Errorf("the indexer has not counted the old sections yet: %s holds no %s", index.OldSectionsBucket, index.OldSectionsKey)
}
return index.DecodeOldSections(e.Value)
}
}
func main() {
if err := stdio.Serve("", Tools(askTakeIn(stdio.State(index.TakeInsState), 25*time.Second), time.Now)); err != nil {
if err := stdio.Serve("", Tools(askTakeIn(stdio.State(index.TakeInsState), 25*time.Second), time.Now, stdio.State(index.VerdictsState), heldCounts(stdio.State(index.OldSectionsState)))); err != nil {
fmt.Fprintf(os.Stderr, "[mesh-delivery-indexer-tools] %v\n", err)
os.Exit(1)
}
@@ -229,7 +229,7 @@ func TestIndexerWritesModulesAtAMerge(t *testing.T) { // 1239/AC9; I1, I3: the m
// its inputs and the reason it has no fingerprint of the new form, and manifests.<digest> with the manifest byte for byte.
//
// And it judges them (novox/hq ADR 0578, issue 1585): with three manifests more, one valid, one refused and one with a
// field the decoder does not know, mesh-sdk's bus reader read before the indexer judged composes all three, the field set
// field the decoder does not know (it waits, ADR 0601 §3), mesh-sdk's bus reader read before the indexer judged composes all three, the field set
// aside and named (a missing verdict is accepted, §3); the indexer writes the three verdicts, says manifest-refused for
// each refusal, and the reader then composes the valid one and holds the others, named, never ending its read.
func TestIndexerKeepsManifestsAtAMerge(t *testing.T) { // 1444/AC4, 1449/AC4, 1585
@@ -300,7 +300,7 @@ func TestIndexerKeepsManifestsAtAMerge(t *testing.T) { // 1444/AC4, 1449/AC4, 15
if err != nil {
t.Fatal(err)
}
want := map[string]string{"good": delivery.VerdictAccepted, "bad": delivery.VerdictRefused, "later": delivery.VerdictRefused}
want := map[string]string{"good": delivery.VerdictAccepted, "bad": delivery.VerdictRefused, "later": delivery.VerdictManifestWaits}
got := map[string]delivery.ManifestVerdict{}
for deadline := time.Now().Add(10 * time.Second); time.Now().Before(deadline) && len(got) < len(want); time.Sleep(50 * time.Millisecond) {
for name := range want {
@@ -316,10 +316,12 @@ func TestIndexerKeepsManifestsAtAMerge(t *testing.T) { // 1444/AC4, 1449/AC4, 15
t.Errorf("the verdict on %s = %+v within 10 s, want %s by the running judge", name, v, state)
}
}
if !strings.Contains(strings.Join(got["bad"].Problems, " "), `provides "amqp"`) || !strings.Contains(strings.Join(got["later"].Problems, " "), "a-field-from-later") {
t.Errorf("the refusals name no problem: %+v", got)
// ADR 0601 §3: the field this judge does not know waits and is named, since no node-engine answers which build of the
// indexer runs here (the unit tests of the index package prove it refused once that build is the trunk's).
if !strings.Contains(strings.Join(got["bad"].Problems, " "), `provides "amqp"`) || strings.Join(got["later"].Unknown, " ") != "a-field-from-later" {
t.Errorf("the refusal names no problem or the wait no field: %+v", got)
}
// manifest-refused, said once for each refusal.
// manifest-refused, said once for each refusal (a wait is no refusal).
events, err := js.Stream(ctx, "INDEXER_EVENTS")
if err != nil {
t.Fatal(err)
@@ -335,8 +337,8 @@ func TestIndexerKeepsManifestsAtAMerge(t *testing.T) { // 1444/AC4, 1449/AC4, 15
said[r.Module] = true
}
}
if !said["bad"] || !said["later"] || said["good"] {
t.Errorf("manifest-refused said for %v, want bad and later", said)
if !said["bad"] || said["later"] || said["good"] {
t.Errorf("manifest-refused said for %v, want bad only", said)
}
// How a composer holds a refused module by these verdicts is the node-engine's to test (ADR 0593 §7: the composer
// and its bus reader are its own).
@@ -26,6 +26,8 @@ import (
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-delivery/sdk/graph"
"git.novox.be/novox/mesh-sdk/go/process"
"mesh-delivery-indexer/internal/index"
@@ -296,11 +298,17 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
// The published packages (ADR 0593 §2): a state this delivery brings, bound when it appears; until then every
// fingerprint takes the version line.
packages := process.AwaitBucket(ctx, js, conn, delivery.PackagesBucket, "no package is published, so every fingerprint takes the version line", logger.Printf)
ix := &index.Indexer{Store: kvStore{candidates}, Modules: kvStore{modules}, Manifests: kvStore{manifests}, Taken: awaitedStore{taken}, Builds: known,
Verdicts: awaitedStore{verdicts}, SayRefused: sayRefused(js, cred, logger.Printf),
// How many modules use each old section (issue 1966 D4): a state this delivery brings, bound when it appears; until then
// the judge refuses no use of an old section.
oldSections := process.AwaitBucket(ctx, js, conn, index.OldSectionsBucket, "the count of the old sections is not kept, so the judge refuses no use of one", logger.Printf)
// The module graph (ADR 0603 §2): a state this delivery brings, bound when it appears; until then no graph is written.
graph := process.AwaitBucket(ctx, js, conn, graph.Bucket, "no module graph is written", logger.Printf)
ix := &index.Indexer{OldSections: awaitedStore{oldSections}, Store: kvStore{candidates}, Modules: kvStore{modules}, Manifests: kvStore{manifests}, Taken: awaitedStore{taken}, Builds: known,
Verdicts: awaitedStore{verdicts}, SayRefused: sayRefused(js, cred, logger.Printf), SayAccepted: sayAccepted(js, cred, logger.Printf),
Predefined: awaitedStore{predefined}, ConfigCandidates: awaitedStore{configCandidates}, Logf: logger.Printf,
Named: namedVia(calls.Call), Trunk: trunkVia(calls.Call), Contains: containsVia(calls.Call), Read: readVia(calls.Call),
ReadOpen: openVia(calls.Call), Packages: awaitedStore{packages}, Files: filesVia(calls.Call), GoSum: goSumVia(calls.Call), GoMod: goModVia(calls.Call)}
ReadOpen: openVia(calls.Call), Packages: awaitedStore{packages}, Files: filesVia(calls.Call), GoSum: goSumVia(calls.Call), GoMod: goModVia(calls.Call),
OwnBuild: ownBuildVia(calls.Call, cred.Node), Graph: awaitedStore{graph}}
// The log's entries at the start are read before any fingerprint is computed or registered module indexed (below); each
// new one indexes the module it names when that module is registered and the bucket holds none (registered.go).
var started atomic.Bool
@@ -421,8 +429,8 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
logger.Printf("judging the manifests at start: %v; judged again at the next message heard", err)
return
}
logger.Printf("judged %d manifests at start: %d verdicts written, %d with problems, of which outgrown and kept accepted: %v",
j.Judged, j.Written, len(j.Refused), j.Outgrown)
logger.Printf("judged %d manifests at start: %d verdicts written, %d with problems, of which outgrown and kept accepted: %v, waiting for a judge that knows their fields: %v",
j.Judged, j.Written, len(j.Refused), j.Outgrown, j.Waiting)
})
// A build module says each time it wrote its toolchains key: its stream, and the indexer's durable consumer on it, are
// made by the controller as the forge's are. Where the stream is not there yet the indexer says so and is started again
@@ -50,6 +50,19 @@ func TestTheTrunkIsTheDirectoriesTheForgeListsTrees(t *testing.T) { // 1900/AC3
if _, has := gotArgs.(map[string]any)["paths"]; has {
t.Error("the trunk asked for paths")
}
// The build the indexer runs is the engine's word (Runs.FP of its composition verb), asked of its own machine.
var asked string
own := func(key string, body any) (json.RawMessage, error) {
asked = key
return json.RawMessage(`{"runs":{"fp":"src2:abc"}}`), nil
}
if fp, err := ownBuildVia(own, "n1")(context.Background()); err != nil || fp != "src2:abc" || asked != "seat:node-engine.composition@n1" {
t.Errorf("own build = %q %v asked %q", fp, err, asked)
}
none := func(string, any) (json.RawMessage, error) { return json.RawMessage(`{}`), nil }
if _, err := ownBuildVia(none, "n1")(context.Background()); err == nil {
t.Error("an engine that says nothing runs gave a build")
}
}
func TestReadViaReadsManifestsAndLocks(t *testing.T) {
@@ -0,0 +1,37 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// compositionVerb is the node-engine seat's read verb that says what runs of a module on a machine (its Runs.FP), asked of
// that machine's own engine as `seat:node-engine.composition@<machine>`. The indexer learns the build it runs from it, never
// from the trunk's value of itself or the haves bucket.
const compositionVerb = "seat:node-engine.composition"
// ownBuildVia asks the engine of node which fingerprint of the indexer runs there.
func ownBuildVia(call toolCaller, node string) func(ctx context.Context) (string, error) {
return func(context.Context) (string, error) {
raw, err := call(compositionVerb+"@"+node, map[string]any{"module": delivery.IndexerModule})
if err != nil {
return "", fmt.Errorf("%s: %w", compositionVerb, err)
}
var a struct {
Runs *struct {
FP string `json:"fp"`
} `json:"runs"`
}
if err := json.Unmarshal(raw, &a); err != nil {
return "", fmt.Errorf("the answer of %s is not a composition: %w", compositionVerb, err)
}
if a.Runs == nil || a.Runs.FP == "" {
return "", errors.New("the engine says no fingerprint of the indexer runs")
}
return a.Runs.FP, nil
}
}
@@ -14,6 +14,8 @@ import (
// takerIn is the part of the indexer a takeins value asks through.
type takerIn interface {
TakeIn(ctx context.Context, module string) index.TakenIn
TakeInRepository(ctx context.Context, repository string) index.TakenIn
JudgeAgain(ctx context.Context, module string) index.TakenIn
}
// takenIn answers one value of the takeins state (the tool indexer_take_in, issue 1945): a value not yet answered takes the
@@ -31,9 +33,17 @@ func takenIn(ctx context.Context, ix takerIn, kv jetstream.KeyValue, e jetstream
if !a.Answered.IsZero() {
return
}
r := ix.TakeIn(ctx, a.Module)
asked, r := "take-in", index.TakenIn{}
if a.Repository != "" {
asked, r = "take-in of a repository", ix.TakeInRepository(ctx, a.Repository)
a.Module = a.Repository
} else if a.Judge {
asked, r = "judging again", ix.JudgeAgain(ctx, a.Module)
} else {
r = ix.TakeIn(ctx, a.Module)
}
a.Result, a.Answered = &r, time.Now().UTC()
logf("%s: a take-in asked by %s (%s): %s", a.Module, a.By, a.Why, r.Answer)
logf("%s: a %s asked by %s (%s): %s", a.Module, asked, a.By, a.Why, r.Answer)
data, err := json.Marshal(a)
if err != nil {
return
@@ -36,3 +36,29 @@ func sayRefused(js jetstream.JetStream, cred process.Account, logf func(string,
}
}
}
// sayAccepted says the indexer's event manifest-accepted (issue 1966 D3, novox/hq ADR 0595 §4):
// mesh.mod.<module>.event.manifest-accepted, carrying the module's bus-state entries whole, with the digest and judge as
// the bus's dedup id (delivery.ManifestAccepted.EventID), so one acceptance is said once.
func sayAccepted(js jetstream.JetStream, cred process.Account, logf func(string, ...any)) func(context.Context, delivery.ManifestAccepted) {
return func(ctx context.Context, a delivery.ManifestAccepted) {
data, err := json.Marshal(a)
if err != nil {
logf("judge: %s for %s is not said: %v", delivery.ManifestAcceptedEvent, a.Module, err)
return
}
msg := nats.NewMsg(cred.EventSubject(delivery.ManifestAcceptedEvent))
id := a.EventID()
msg.Header.Set("x-event-id", id)
msg.Header.Set("x-source", cred.Module)
msg.Header.Set("x-node", cred.Node)
msg.Header.Set("x-time", time.Now().UTC().Format(time.RFC3339Nano))
msg.Header.Set("content-type", "application/json")
msg.Data = data
pub, cancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
defer cancel()
if _, err := js.PublishMsg(pub, msg, jetstream.WithMsgID(id)); err != nil {
logf("judge: %s", fmt.Errorf("saying %s for %s: %w", delivery.ManifestAcceptedEvent, a.Module, err))
}
}
}
+5 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-indexer
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
@@ -23,3 +23,7 @@ require (
)
replace git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/buildrun => ../mesh-delivery-builder/buildrun
require git.novox.be/novox/mesh-delivery/sdk v0.0.0-00010101000000-000000000000
replace git.novox.be/novox/mesh-delivery/sdk => ../../sdk
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929 h1:RtAFbUJRP1qO9AIMM2ue+cXNzqj7WZ7IOlSqNvttwuU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -54,13 +54,21 @@ type Indexer struct {
Taken Store
// Verdicts is the verdicts state (verdicts.go, novox/hq ADR 0578 §1); nil judges nothing.
Verdicts Store
// OldSections is the state of how many accepted modules use each old section (oldsections.go, issue 1966 D4); nil keeps no
// count and the judge refuses no use of an old section.
OldSections Store
// JudgeID names the judge a verdict says made it; empty is judge.Identity, the running program's.
JudgeID string
// SayRefused says the event manifest-refused for a refusal newly written; nil says nothing.
SayRefused func(ctx context.Context, r delivery.ManifestRefused)
// SayAccepted says the event manifest-accepted for an accepted verdict newly written whose manifest asks for bus state,
// carrying the entries whole (verdicts.go; issue 1966 D3); nil says nothing.
SayAccepted func(ctx context.Context, a delivery.ManifestAccepted)
// Packages is the packages state (delivery.PackagesBucket, packages.go; novox/hq ADR 0593 §2); nil publishes none,
// and every fingerprint takes the version line.
Packages Store
// Graph is the module graph's state (graph.Bucket, graph.go; novox/hq ADR 0603 §2); nil writes no graph.
Graph Store
// Files asks the forge for every blob under each directory at a commit, by directory then path relative to it, each
// with its git object id (gitea.gitea_commit_trees with files): the listing a package's digest is computed from.
// Nil publishes no package.
@@ -92,22 +100,27 @@ type Indexer struct {
// closed message the indexer missed heals at the next one (issue 1126); nil asks none.
ReadOpen func(ctx context.Context) ([]delivery.OpenPull, error)
mu sync.Mutex // guards source, toolchains and previews, which the watches and the forge messages all reach
source sourceNow
previews map[string]delivery.Preview // by previews key (previews.go)
toolchains map[string]map[string]delivery.Toolchain // language, then node (toolchains.go)
tried map[string]bool // repository, commit and directory a registered module's manifest was tried at (registered.go)
unread map[string]bool // registered modules whose last read the forge could not answer, read again by RetryUnread
moving map[string]delivery.Forge // by modules key: a merge naming a module another repository holds, waiting for that trunk to drop it (moves.go)
retrying atomic.Bool // a RetryUnread runs in the background (kickRetry)
retryAgain atomic.Bool // an event asked for one more RetryUnread after the running one
retryWG sync.WaitGroup // the background RetryUnread, for tests to wait on
recompute sync.Mutex // one Recompute at a time
untaken map[string]string // repository to the latest merge applied while the taken state was missing (taken.go)
filling atomic.Bool // a FillTaken runs in the background (KickFill)
fillAgain atomic.Bool // a kick arrived while it ran
atHeads map[string]map[string]delivery.PublishedPackages // a member head's published Go modules, by repository@head (candidatePublished)
judgePending atomic.Bool // the start's judge pass failed: judged again at the next message (verdicts.go)
mu sync.Mutex // guards source, toolchains and previews, which the watches and the forge messages all reach
source sourceNow
previews map[string]delivery.Preview // by previews key (previews.go)
toolchains map[string]map[string]delivery.Toolchain // language, then node (toolchains.go)
tried map[string]bool // repository, commit and directory a registered module's manifest was tried at (registered.go)
unread map[string]bool // registered modules whose last read the forge could not answer, read again by RetryUnread
moving map[string]delivery.Forge // by modules key: a merge naming a module another repository holds, waiting for that trunk to drop it (moves.go)
retrying atomic.Bool // a RetryUnread runs in the background (kickRetry)
retryAgain atomic.Bool // an event asked for one more RetryUnread after the running one
retryWG sync.WaitGroup // the background RetryUnread, for tests to wait on
recompute sync.Mutex // one Recompute at a time
untaken map[string]string // repository to the latest merge applied while the taken state was missing (taken.go)
filling atomic.Bool // a FillTaken runs in the background (KickFill)
fillAgain atomic.Bool // a kick arrived while it ran
atHeads map[string]map[string]delivery.PublishedPackages // a member head's published Go modules, by repository@head (candidatePublished)
// OwnBuild answers the fingerprint of the build this program runs, from the node-engine's seat; nil knows none.
OwnBuild func(ctx context.Context) (string, error)
judgeMu sync.Mutex // one judge pass at a time (verdicts.go)
ownMu sync.Mutex // guards ownFP
ownFP string // the fingerprint of the build this program runs, asked at the start pass (learnOwnBuild)
judgePending atomic.Bool // the start's judge pass failed: judged again at the next message (verdicts.go)
}
func (ix *Indexer) logf(format string, args ...any) {
@@ -0,0 +1,149 @@
package index
import (
"encoding/json"
"fmt"
"maps"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-delivery/sdk/graph"
)
// The seat mesh-catalog's verbs, answered from the indexer's own states (novox/hq ADR 0604 §1, §2): modules, module,
// provides and dependents read the modules state, the manifests state and the module graph, and write nothing. The
// indexer serves no request subject (ADR 0437 §4): its tools bundle serves them as <seat>.<verb>. Each answer keeps the
// field names of the old catalog module's tool it replaces (catalog_modules, catalog_module, catalog_provides,
// catalog_dependents): the answers are the seat's contract (catalogue.CatalogModules and the rest), built here.
// CatalogRead reads the states the seat's verbs answer from.
type CatalogRead struct {
// Modules is every value of the modules state, by module.
Modules func() (map[string]delivery.Module, error)
// Manifest is the manifest kept under a digest in the manifests state, as its JSON.
Manifest func(digest string) (string, error)
// Graph is the module graph, by module.
Graph func() (graph.Graph, error)
}
func entryOf(name string, m delivery.Module) catalogue.CatalogEntry {
return catalogue.CatalogEntry{Module: name, Repository: m.Repository, Path: m.Path, Commit: m.Commit, FP: m.FP2, None: m.None2}
}
// CatalogModulesAnswer is the verb modules: every module the modules state holds, by name.
func (c CatalogRead) CatalogModulesAnswer() (catalogue.CatalogModules, error) {
mods, err := c.Modules()
if err != nil {
return catalogue.CatalogModules{}, fmt.Errorf("the indexer's modules cannot be read: %w", err)
}
out := []catalogue.CatalogEntry{}
for _, name := range slices.Sorted(maps.Keys(mods)) {
out = append(out, entryOf(name, mods[name]))
}
return catalogue.CatalogModules{Modules: out}, nil
}
// CatalogModuleAnswer is the verb module: one module whole, its manifest included.
func (c CatalogRead) CatalogModuleAnswer(name string) (catalogue.CatalogOne, error) {
name = strings.TrimSpace(name)
if name == "" {
return catalogue.CatalogOne{}, fmt.Errorf("name the module")
}
mods, err := c.Modules()
if err != nil {
return catalogue.CatalogOne{}, fmt.Errorf("the indexer's modules cannot be read: %w", err)
}
m, ok := mods[name]
if !ok {
return catalogue.CatalogOne{}, fmt.Errorf("the trunk holds no module %s", name)
}
one := catalogue.CatalogModule{CatalogEntry: entryOf(name, m)}
if m.Manifest == "" {
return catalogue.CatalogOne{}, fmt.Errorf("the indexer keeps no manifest of %s", name)
}
raw, err := c.Manifest(m.Manifest)
if err != nil {
return catalogue.CatalogOne{}, fmt.Errorf("the manifest %s of %s cannot be read: %w", m.Manifest, name, err)
}
one.Manifest = json.RawMessage(raw)
return catalogue.CatalogOne{Module: one}, nil
}
// CatalogProvidesAnswer is the verb provides: the modules that provide a provision, each with its scope (a module provides
// its own name, at node scope), and the modules that require it, from the module graph's provision edges.
func (c CatalogRead) CatalogProvidesAnswer(provision string) (catalogue.CatalogProvides, error) {
provision = strings.TrimSpace(provision)
if provision == "" {
return catalogue.CatalogProvides{}, fmt.Errorf("name the provision")
}
g, mods, err := c.graphAndModules()
if err != nil {
return catalogue.CatalogProvides{}, err
}
provides, requires := []catalogue.CatalogEnd{}, []catalogue.CatalogEnd{}
for _, name := range g.Providers(provision) {
end := catalogue.CatalogEnd{Module: name, Commit: mods[name].Commit, Scope: catalogue.ScopeNode}
if name != provision {
end.Scope = c.scopeOf(mods[name], provision)
}
provides = append(provides, end)
}
for _, name := range g.Requirers(provision) {
requires = append(requires, catalogue.CatalogEnd{Module: name, Commit: mods[name].Commit})
}
return catalogue.CatalogProvides{Provision: provision, Provides: provides, Requires: requires}, nil
}
// scopeOf is the scope the module's manifest offers provision at, "" when its manifest cannot be read.
func (c CatalogRead) scopeOf(m delivery.Module, provision string) string {
if m.Manifest == "" {
return ""
}
raw, err := c.Manifest(m.Manifest)
if err != nil {
return ""
}
man, err := catalogue.Decode([]byte(raw))
if err != nil {
return ""
}
for _, o := range man.Provides {
if o.Name == provision {
return o.At()
}
}
return ""
}
// CatalogDependentsAnswer is the verb dependents: the modules whose build edges reach the module, directly or through
// others, from the module graph.
func (c CatalogRead) CatalogDependentsAnswer(module string) (catalogue.CatalogDependents, error) {
module = strings.TrimSpace(module)
if module == "" {
return catalogue.CatalogDependents{}, fmt.Errorf("name the module")
}
g, mods, err := c.graphAndModules()
if err != nil {
return catalogue.CatalogDependents{}, err
}
out := []catalogue.CatalogEnd{}
for _, name := range g.Dependents(module) {
out = append(out, catalogue.CatalogEnd{Module: name, Commit: mods[name].Commit})
}
return catalogue.CatalogDependents{Module: module, Dependents: out}, nil
}
func (c CatalogRead) graphAndModules() (graph.Graph, map[string]delivery.Module, error) {
g, err := c.Graph()
if err != nil {
return nil, nil, fmt.Errorf("the module graph cannot be read: %w", err)
}
mods, err := c.Modules()
if err != nil {
return nil, nil, fmt.Errorf("the indexer's modules cannot be read: %w", err)
}
return g, mods, nil
}
@@ -0,0 +1,127 @@
package index
import (
"context"
"encoding/json"
"os"
"path/filepath"
"reflect"
"slices"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-delivery/sdk/graph"
)
// capturedKeys are the field names of the first object at path (a list's first element when the value is a list) in the
// old catalog module's answer captured with mesh_call on 2026-11-03 (testdata/catalog; trimmed to a few modules, and the
// manifest of catalog_module to three of its keys).
func capturedKeys(t *testing.T, file string, path ...string) []string {
t.Helper()
raw, err := os.ReadFile(filepath.Join("testdata", "catalog", file))
if err != nil {
t.Fatal(err)
}
return keysAt(t, raw, path...)
}
func keysAt(t *testing.T, raw []byte, path ...string) []string {
t.Helper()
var v any
if err := json.Unmarshal(raw, &v); err != nil {
t.Fatal(err)
}
for _, p := range path {
v = v.(map[string]any)[p]
if l, ok := v.([]any); ok {
v = l[0]
}
}
var out []string
for k := range v.(map[string]any) {
out = append(out, k)
}
slices.Sort(out)
return out
}
// catalogVerbsAnswerFromTheStates: the seat mesh-catalog's four verbs answer from the indexer's modules, manifests and
// graph, with the field names of the old catalog module's answers (ADR 0604 §1, §2).
func catalogVerbsAnswerFromTheStates(t *testing.T) {
withAcme := strings.Replace(renamed(t, "audit-logger", "acme"), `"module": "acme",`,
`"module": "acme", "provides": [{"name": "internal-acme-ca", "scope": "mesh", "answered-by": "helper"}],`, 1)
ix, mods := newFormIndexer(t, mod{"mesh-tools", renamed(t, "avahi", "mesh-tools")},
mod{"avahi", standingOn(t, catalogueManifest(t, "avahi"), "mesh-tools")},
mod{"ca-trust", catalogueManifest(t, "ca-trust")}, mod{"acme", withAcme})
reportToolchain(t, ix, "mesh-build-go", "novox", "go", "go1.27.1")
c := CatalogRead{
Modules: func() (map[string]delivery.Module, error) {
out := map[string]delivery.Module{}
for k := range mods.vals {
m, _ := moduleOf(t, mods, k)
out[k] = m
}
return out, nil
},
Manifest: func(digest string) (string, error) {
raw, _, err := ix.manifestRaw(context.Background(), digest)
return raw, err
},
Graph: func() (graph.Graph, error) {
g := graph.Graph{}
for k := range ix.Graph.(*memStore).vals {
g[k] = graphOf(t, ix, k)
}
return g, nil
},
}
answer := func(v any, err error) []byte {
t.Helper()
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(v)
return raw
}
contains := func(got, want []string) bool {
for _, w := range want {
if !slices.Contains(got, w) {
return false
}
}
return true
}
raw := answer(c.CatalogModulesAnswer())
if got, want := keysAt(t, raw, "modules"), capturedKeys(t, "catalog_modules.json", "modules"); !contains(got, want) {
t.Errorf("modules' fields %v, want the old catalog's %v", got, want)
}
if !strings.Contains(string(raw), `"module":"avahi"`) || !strings.Contains(string(raw), `"fp":"src2:`) {
t.Errorf("modules = %s", raw)
}
raw = answer(c.CatalogModuleAnswer("ca-trust"))
want := slices.DeleteFunc(capturedKeys(t, "catalog_module.json", "module"), func(k string) bool { return k == "ref" }) // the trunk's, no ref
if got := keysAt(t, raw, "module"); !contains(got, want) || !strings.Contains(string(raw), `"internal-acme-ca"`) {
t.Errorf("module ca-trust = %s, want the fields %v and its manifest", raw, want)
}
if _, err := c.CatalogModuleAnswer("gone"); err == nil || !strings.Contains(err.Error(), "no module gone") {
t.Errorf("a module the trunk does not hold: %v", err)
}
raw = answer(c.CatalogProvidesAnswer("internal-acme-ca"))
if got, want := keysAt(t, raw, "provides"), capturedKeys(t, "catalog_provides.json", "provides"); !reflect.DeepEqual(got, want) ||
!reflect.DeepEqual(keysAt(t, raw, "requires"), capturedKeys(t, "catalog_provides.json", "requires")) ||
!strings.Contains(string(raw), `"provides":[{"module":"acme","commit":"`+merge+`","scope":"mesh"}]`) ||
!strings.Contains(string(raw), `"requires":[{"module":"ca-trust"`) {
t.Errorf("provides internal-acme-ca = %s", raw)
}
raw = answer(c.CatalogDependentsAnswer("mesh-tools"))
if got, want := keysAt(t, raw), capturedKeys(t, "catalog_dependents.json"); !reflect.DeepEqual(got, want) ||
!strings.Contains(string(raw), `"dependents":[{"module":"avahi"`) {
t.Errorf("dependents of mesh-tools = %s", raw)
}
raw = answer(c.CatalogDependentsAnswer("mesh-build-go")) // the build module reaches every Go module it builds
if !strings.Contains(string(raw), `"avahi"`) || !strings.Contains(string(raw), `"mesh-tools"`) {
t.Errorf("dependents of mesh-build-go = %s", raw)
}
}
@@ -0,0 +1,135 @@
package index
import (
"context"
"errors"
"fmt"
"maps"
"slices"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-delivery/sdk/graph"
)
// The module graph (novox/hq ADR 0603 §2): at every recompute the indexer writes, in its state graph
// (graph.Bucket), one key per module it holds the inputs of, its build and provision edges as that recompute
// computed them, and deletes the key of a module the modules state no longer holds. A key is written only when its value
// changes, so its history of 5 is five changes of the module's edges. The delivery view, the checker and the seat
// mesh-catalog's verbs read it; nothing stores an order (ADR 0603 §1).
// GraphMost is the most modules the graph holds (ADR 0603 §2): past it, the modules after the first GraphMost in name
// order are not written, and it is said in the log. The cap is the indexer's code, not a manifest field, until a judge
// that knows such a field runs (the lead's decision of L3).
const GraphMost = 2000
// graphEdges is what a recompute knows of one module for its graph node.
type graphEdges struct {
module string
cur delivery.Module
manifest catalogue.Manifest
fp, none string
tcs map[string]delivery.ToolchainUsed
pkgs delivery.PackageVersions
}
// graphNode is the graph node of e: its edges, the awaited provisions judged against providers (catalogue.Awaited, as
// the assigner judges them).
func graphNode(e graphEdges, providers func(string) []catalogue.Manifest) graph.Node {
if e.fp == "" && e.none == "" {
e.none = "its fingerprint was not computed"
}
n := graph.EdgesOf(e.manifest, providers)
n.FP, n.None, n.Repository, n.Path = e.fp, e.none, e.cur.Repository, e.cur.Path
n.Packages = slices.Sorted(maps.Keys(e.pkgs))
tcs := e.tcs
if len(tcs) == 0 {
tcs = e.cur.Toolchains // no fingerprint now: the toolchain it was last computed with still builds it
}
for language, t := range tcs {
builder, known := ToolchainModules[language]
if !known {
continue
}
if n.Toolchains == nil {
n.Toolchains = map[string]graph.Toolchain{}
}
n.Toolchains[language] = graph.Toolchain{Module: builder, Version: t.Version, Recipe: t.Recipe}
}
return n
}
// writeGraph writes the graph node of every module of edges whose value changed, at most GraphMost of them, then
// deletes each graph key whose module the modules state no longer holds. Nil Graph writes nothing, and neither does a
// graph state that is not on the bus yet (process.ErrNotYet; main.go: until then no graph is written), said in the log;
// a listing that fails otherwise skips the deletion pass, said, and never fails the merge or the recompute that wrote.
func (ix *Indexer) writeGraph(ctx context.Context, edges map[string]graphEdges) error {
if ix.Graph == nil {
return nil
}
keys, listErr := ix.Graph.Keys(ctx)
if errors.Is(listErr, process.ErrNotYet) {
ix.logf("the module graph is not written: its state is not on the bus yet")
return nil
}
names := slices.Sorted(maps.Keys(edges))
if len(names) > GraphMost {
ix.logf("the module graph holds at most %d modules: %d are known, so %s and the %d after it are not written",
GraphMost, len(names), names[GraphMost], len(names)-GraphMost-1)
names = names[:GraphMost]
}
providers := func(provision string) []catalogue.Manifest {
var out []catalogue.Manifest
for _, name := range slices.Sorted(maps.Keys(edges)) {
if slices.Contains(edges[name].manifest.Offers(), provision) {
out = append(out, edges[name].manifest)
}
}
return out
}
for _, name := range names {
key, err := graph.Key(name)
if err != nil {
ix.logf("graph.%s is not written: %v", name, err)
continue
}
next := graphNode(edges[name], providers)
err = write(ctx, ix.Graph, key, func(value []byte, found bool) ([]byte, bool, error) {
if found {
if cur, err := graph.Decode(value); err == nil && graph.Same(cur, next) {
return nil, false, nil
}
}
data, err := graph.Encode(next)
if err != nil {
return nil, false, fmt.Errorf("graph.%s: %w", key, err)
}
return data, false, nil
})
if err != nil {
ix.logf("graph.%s is not written: %v", key, err)
}
}
held, err := ix.Modules.Keys(ctx)
switch {
case listErr != nil:
ix.logf("the graph keys of modules the modules state no longer holds are not deleted: listing the graph: %v", listErr)
return nil
case err != nil:
ix.logf("the graph keys of modules the modules state no longer holds are not deleted: listing the modules: %v", err)
return nil
}
for _, key := range keys {
if slices.Contains(held, key) {
continue
}
if err := write(ctx, ix.Graph, key, func(_ []byte, found bool) ([]byte, bool, error) { return nil, found, nil }); err != nil {
ix.logf("graph.%s is not deleted: %v", key, err)
continue
}
ix.logf("graph.%s deleted: the modules state holds no module %s", key, key)
}
return nil
}
@@ -19,6 +19,7 @@ const (
// key under the indexer; unbound, when set, is every read's error (a state not yet on the bus).
type memStore struct {
unbound error
keysErr error // the listing's error
vals map[string][]byte
revs map[string]uint64
next uint64
@@ -36,6 +37,9 @@ func (m *memStore) Get(_ context.Context, key string) ([]byte, uint64, bool, err
return v, m.revs[key], ok, nil
}
func (m *memStore) Keys(context.Context) ([]string, error) {
if m.keysErr != nil {
return nil, m.keysErr
}
var keys []string
for k := range m.vals {
keys = append(keys, k)
@@ -33,7 +33,7 @@ func manifestsIndexer(t *testing.T, recs ...[2]any) (*Indexer, *memStore, *memSt
t.Helper()
ix, mods := modulesIndexer(t, recs...)
manifests := newMem()
ix.Manifests = manifests
ix.Manifests, ix.Graph = manifests, newMem()
return ix, mods, manifests
}
@@ -45,6 +45,7 @@ func withManifests(f delivery.Forge, manifests map[string]string, locks map[stri
func TestAMergedManifestIsKeptOnceByItsDigestAndNamedByTheModule(t *testing.T) {
t.Run("kept by its digest and named by the module", mergedManifestIsKeptByItsDigestAndNamedByTheModule)
t.Run("the same manifest applied again is written once", sameManifestAppliedAgainIsWrittenOnce)
t.Run("the seat mesh-catalog's verbs answer from the states", catalogVerbsAnswerFromTheStates)
}
func mergedManifestIsKeptByItsDigestAndNamedByTheModule(t *testing.T) { // 1449/AC1
@@ -117,8 +117,10 @@ func (ix *Indexer) applyMoving(ctx context.Context, f delivery.Forge, drop bool,
overlay[key] = next
}
// A module of this repository the merge commit holds no module.json for is gone from the trunk, and so is its key
// (issue 1900). A message that carries no manifests (an older writer) or one that cannot be read whole says nothing.
if drop && len(f.Manifests) > 0 && every {
// (issue 1900). A message with no trees says the merge commit holds no module.json at all, so every module of the
// repository goes (issue 1126 note 94: a repository that removed its last module). A message with trees but no
// manifests (an older writer) or one that cannot be read whole says nothing.
if drop && (len(f.Manifests) > 0 || len(f.Trees) == 0) && every {
if err := ix.dropRemoved(ctx, f.Repository, held, func(key string, _ delivery.Module) bool { return named[key] }); err != nil {
return w, err
}
@@ -214,6 +216,7 @@ func (ix *Indexer) namedObjects(ctx context.Context, f delivery.Forge) (map[stri
// held(key, module) says whether the trunk holds it. A module of another repository, or of none known, is never touched.
func (ix *Indexer) dropRemoved(ctx context.Context, repository string, held map[string]delivery.Module, trunkHolds func(key string, m delivery.Module) bool) error {
var handed []string
elsewhere := map[string]delivery.Forge{}
for _, key := range slices.Sorted(maps.Keys(held)) {
cur := held[key]
if cur.Repository == "" || !delivery.SameRepository(cur.Repository, repository) || trunkHolds(key, cur) {
@@ -223,6 +226,12 @@ func (ix *Indexer) dropRemoved(ctx context.Context, repository string, held map[
handed = append(handed, key) // written in place by the repository it moves to, never deleted first
continue
}
// The wait of moves.go is memory, which a restart forgets (issue 1367): another repository's trunk that holds the
// module is asked, so the move survives it.
if f, ok := ix.heldByAnotherTrunk(ctx, key, repository); ok {
elsewhere[key] = f
continue
}
err := write(ctx, ix.Modules, key, func(value []byte, found bool) ([]byte, bool, error) {
if !found {
return nil, false, nil
@@ -242,6 +251,11 @@ func (ix *Indexer) dropRemoved(ctx context.Context, repository string, held map[
return err
}
}
for _, key := range slices.Sorted(maps.Keys(elsewhere)) {
if err := ix.handOverFrom(ctx, key, elsewhere[key]); err != nil {
return err
}
}
return nil
}
@@ -219,6 +219,32 @@ func TestAnUnchangedModuleWritesNothing(t *testing.T) { // 1239 (unchanged value
}
return ix.Apply(context.Background(), mergedMsg(repo, three))
}},
// Issue 1126 note 94: a merge that leaves no module.json anywhere (hq, or a repository whose last module was removed) is a
// merged message with no trees; it drops the repository's modules and moves its taken key, and an older one, heard late,
// goes through the same order check and drops nothing.
{name: "a merge that leaves no module.json drops every module and is taken in", writes: 3, commit: map[string]string{"a": "", "b": "", "c": ""}, taken: next,
said: "modules written: none",
then: func(ix *Indexer) error {
m := mergedMsg(repo, nil)
m.Merge = next
return ix.Apply(context.Background(), m)
}},
{name: "an earlier merge with no module.json heard after a later one drops nothing", writes: 0, commit: map[string]string{"a": merge, "b": merge, "c": merge}, taken: next,
said: "the merged message is older than taken.novox/mesh-delivery at cccccccccccc",
then: func(ix *Indexer) error {
const earlier = "dddddddddddddddddddddddddddddddddddddddd"
ix.Contains = func(_ context.Context, _, commit, in string) (bool, error) {
return commit == in || commit == earlier && in == next, nil
}
later := mergedMsg(repo, three)
later.Merge = next
if err := ix.Apply(context.Background(), later); err != nil {
return err
}
gone := mergedMsg(repo, nil)
gone.Merge = earlier
return ix.Apply(context.Background(), gone)
}},
// Review round 2: a 502 whose words carry "404" inside a sha is no not-found; the late message is left for redelivery.
{name: "a late message whose order the forge fails with a 502 writes nothing and is left for redelivery", writes: 1, commit: map[string]string{"a": next, "b": merge, "c": merge}, taken: next, err: true,
said: "502 Bad Gateway",
@@ -305,8 +331,8 @@ func TestAnUnchangedModuleWritesNothing(t *testing.T) { // 1239 (unchanged value
if got := mods.writes - writes; got != tc.writes {
t.Errorf("%d values written, want %d", got, tc.writes)
}
for module, want := range tc.commit {
if m, ok := moduleOf(t, mods, module); !ok || m.Commit != want {
for module, want := range tc.commit { // "" is a module the merge deleted
if m, ok := moduleOf(t, mods, module); ok != (want != "") || m.Commit != want {
t.Errorf("%s = %+v (%v), want commit %.8s", module, m, ok, want)
}
}
@@ -419,31 +445,76 @@ func (d *deletesSeen) Delete(ctx context.Context, key string, rev uint64) error
// holding repository's merge that drops it hands it over in place, never deleting its key; merged the other way round,
// the module is deleted and then written new.
func TestAModuleMovesBetweenRepositoriesOnlyWhenItsTrunkDropsIt(t *testing.T) {
const from, to = "novox/mesh-controller", "novox/mesh-networking"
const from, to, toHead = "novox/mesh-controller", "novox/mesh-networking", "bbbbbbbbbbbb"
held := mergedMsg(from, map[string]string{"modules/a": "t", "modules/b": "t"})
named := mergedMsg(to, map[string]string{"modules/a": "t2"})
dropped := mergedMsg(from, map[string]string{"modules/b": "t"})
gone := mergedMsg(from, nil) // issue 1367: a merge that leaves no module.json
gone.Trees, gone.Manifests = nil, nil
for _, tc := range []struct {
name string
merges []delivery.Forge
repository string // modules.a's repository at the end, "" for none
deletedKeyA bool
// Issue 1367: a fresh indexer, no wait in memory, the source naming heads; trunk is the module directories the forge
// reads at the head of `to`; takeIn is a repository taken in after the merges (indexer_take_in), whose answer must
// hold answer and be stored or not.
heads bool
trunk []string
takeIn string
answer string
stored bool
bGoneAtEnd bool
}{
{"named while its trunk holds it: left", []delivery.Forge{held, named}, from, false},
{"named, then dropped by its trunk: handed over in place", []delivery.Forge{held, named, dropped}, to, false},
{"dropped by its trunk, then named: deleted, then written new", []delivery.Forge{held, dropped, named}, to, true},
{"dropped by its trunk with no merge naming it: deleted", []delivery.Forge{held, dropped}, "", true},
{name: "named while its trunk holds it: left", merges: []delivery.Forge{held, named}, repository: from},
{name: "named, then dropped by its trunk: handed over in place", merges: []delivery.Forge{held, named, dropped}, repository: to},
{name: "dropped by its trunk, then named: deleted, then written new", merges: []delivery.Forge{held, dropped, named}, repository: to, deletedKeyA: true},
{name: "dropped by its trunk with no merge naming it: deleted", merges: []delivery.Forge{held, dropped}, deletedKeyA: true},
{name: "no module.json left, another trunk holds it, nothing waits in memory: written from that trunk, never deleted",
merges: []delivery.Forge{held, gone}, heads: true, trunk: []string{"modules/a", "modules/other"}, repository: to, bGoneAtEnd: true},
{name: "no module.json left, no other trunk holds it: deleted",
merges: []delivery.Forge{held, gone}, heads: true, deletedKeyA: true, bGoneAtEnd: true},
{name: "taken in by repository, a head with no module.json: handed over from the other trunk",
merges: []delivery.Forge{held}, heads: true, trunk: []string{"modules/a"}, takeIn: from, answer: "handed over", stored: true, repository: to, bGoneAtEnd: true},
{name: "taken in by repository, a head with modules: answered by name",
merges: []delivery.Forge{held}, heads: true, trunk: []string{"modules/a"}, takeIn: to, answer: "by name", repository: from},
{name: "taken in by repository, no head named: answered so",
merges: []delivery.Forge{held}, heads: true, takeIn: "novox/nobody", answer: "names no head", repository: from},
} {
t.Run(tc.name, func(t *testing.T) {
mods := &deletesSeen{memStore: newMem()}
ix := &Indexer{Store: newMem(), Modules: mods, Builds: NewBuilds()}
var logs []string
ix.Logf = func(format string, args ...any) { logs = append(logs, format) }
if tc.heads {
ix.source = sourceNow{rev: 7, heads: map[string]string{from: merge, to: toHead}}
ix.Trunk = func(_ context.Context, repository, sha string) ([]string, error) {
if repository == to && sha == toHead {
return tc.trunk, nil
}
if repository == from && tc.takeIn == from {
return nil, nil
}
return nil, nil
}
ix.Read = func(_ context.Context, repository, sha, dir string) (RepoRead, error) {
if repository != to || sha != toHead || dir != "modules/a" {
t.Errorf("read %s at %s %q, want only %s at %s modules/a", repository, sha, dir, to, toHead)
}
return RepoRead{Root: "r2", Trees: map[string]string{dir: "t2"}, Manifests: map[string]string{dir: `{"module":"a","version":"1"}`}}, nil
}
}
for _, f := range tc.merges {
if err := ix.Apply(context.Background(), f); err != nil {
t.Fatal(err)
}
}
if tc.takeIn != "" {
got := ix.TakeInRepository(context.Background(), tc.takeIn)
if got.Stored != tc.stored || !strings.Contains(got.Answer, tc.answer) {
t.Errorf("take-in answered %+v, want stored %v and %q", got, tc.stored, tc.answer)
}
}
m, ok := moduleOf(t, mods.memStore, "a")
if got := map[bool]string{true: m.Repository}[ok]; got != tc.repository {
t.Errorf("modules.a of %q, want %q (logs %v)", got, tc.repository, logs)
@@ -451,11 +522,14 @@ func TestAModuleMovesBetweenRepositoriesOnlyWhenItsTrunkDropsIt(t *testing.T) {
if ok && m.Path != "modules/a" {
t.Errorf("modules.a at %q, want modules/a", m.Path)
}
if ok && m.Repository == to && tc.heads && m.Commit != toHead {
t.Errorf("modules.a at %s, want the other trunk's head %s", m.Commit, toHead)
}
if deleted := slices.Contains(mods.deleted, "a"); deleted != tc.deletedKeyA {
t.Errorf("modules.a deleted %v, want %v", deleted, tc.deletedKeyA)
}
if _, ok := mods.vals["b"]; !ok {
t.Error("modules.b, which its trunk still holds, is gone")
if _, ok := mods.vals["b"]; ok == tc.bGoneAtEnd {
t.Errorf("modules.b held %v, want %v", ok, !tc.bGoneAtEnd)
}
if len(logs) == 0 {
t.Error("nothing was said")
@@ -3,10 +3,66 @@ package index
import (
"context"
"fmt"
"maps"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// heldByAnotherTrunk reads, for a module the trunk of repository no longer holds, whether the trunk of another repository
// holds it at the head the latest source names: the module directories there are asked of the forge (Trunk) and the manifest of
// each directory named like the module is read (Read). It answers the merge to hand the module over with, or none: a forge
// that cannot say leaves the module to be dropped as it was, said (issue 1367).
func (ix *Indexer) heldByAnotherTrunk(ctx context.Context, module, repository string) (delivery.Forge, bool) {
if ix.Trunk == nil || ix.Read == nil {
return delivery.Forge{}, false
}
ix.mu.Lock()
heads := maps.Clone(ix.source.heads)
ix.mu.Unlock()
for _, other := range slices.Sorted(maps.Keys(heads)) {
head := heads[other]
if head == "" || delivery.SameRepository(other, repository) {
continue
}
dirs, err := ix.Trunk(ctx, other, head)
if err != nil {
ix.logf("%s at %s: the modules of the trunk cannot be read, so %s is not looked for there: %v", other, head, module, err)
continue
}
for _, d := range dirs {
dir := strings.Trim(d, "/")
if nameOfDirectory(dir) != module {
continue
}
read, err := ix.Read(ctx, other, head, dir)
if err != nil {
ix.logf("%s at %s: %q cannot be read, so %s is not handed over from it: %v", other, head, dir, module, err)
continue
}
raw := read.Manifests[dir]
if read.Trees[dir] == "" || manifestModule(raw) != module {
continue
}
f := delivery.Forge{V: delivery.Version, Kind: delivery.ForgeMerged, Repository: other, Merge: head, Root: read.Root,
Trees: map[string]string{dir: read.Trees[dir]}, Manifests: map[string]string{dir: raw}}
if l := read.Locks[dir]; len(l) > 0 {
f.Locks = map[string]map[string]string{dir: l}
}
if p := read.Packages[dir]; len(p) > 0 {
f.Packages = map[string][]string{dir: p}
}
ix.logf("%s at %s: the trunk holds the module %s at %q, so it is handed over from there and not deleted (issue 1367)", other, head, module, dir)
return f, true
}
}
return delivery.Forge{}, false
}
// nameOfDirectory is the last element of a module directory.
func nameOfDirectory(dir string) string { return dir[strings.LastIndex(dir, "/")+1:] }
// A module moves between repositories by the merges of the delivery group that moves it (novox/hq ADR 0420 clause 7, ADR
// 0501 clause 3; issue 1126: route-proxy leaves the controller's repository, which is archived). The merge of the
// repository it moves to names it while the indexer holds it of another: that merge leaves it, says so, and waits here.
@@ -57,6 +113,11 @@ func (ix *Indexer) handOver(ctx context.Context, key string) error {
if !ok {
return nil
}
return ix.handOverFrom(ctx, key, f)
}
// handOverFrom writes the module at key from the merge f, which names it, in place.
func (ix *Indexer) handOverFrom(ctx context.Context, key string, f delivery.Forge) error {
w, err := ix.applyMoving(ctx, f, false, map[string]bool{key: true})
if err != nil {
return fmt.Errorf("modules.%s, moving to %s: %w", key, f.Repository, err)
@@ -0,0 +1,76 @@
package index
import (
"context"
"encoding/json"
"fmt"
"maps"
"git.novox.be/novox/mesh-sdk/go/delivery"
"mesh-delivery-indexer/judge"
)
// How many accepted modules use each old section (issue 1966 D4; novox/hq ADR 0595 §11): the indexer counts them, the
// foundation left out, as it judges, keeps the count in its state oldsections under one key, and the tool
// indexer_old_sections reads it. The judge refuses a manifest of another module that uses a section whose count is 0.
const (
// OldSectionsTool is the tool that reads the count.
OldSectionsTool = "indexer_old_sections"
// OldSectionsState is the indexer's state of the count: the bucket mesh-delivery-indexer_oldsections.
OldSectionsState = "oldsections"
OldSectionsBucket = delivery.IndexerModule + "_" + OldSectionsState
// OldSectionsKey is the one key it holds.
OldSectionsKey = "counts"
)
// OldSectionsCount is the value of the oldsections state: per old section, the accepted modules outside the foundation using it.
type OldSectionsCount struct {
V int `json:"v"`
Counts map[string]int `json:"counts"`
}
// DecodeOldSections reads the oldsections value.
func DecodeOldSections(value []byte) (OldSectionsCount, error) {
var c OldSectionsCount
if err := json.Unmarshal(value, &c); err != nil {
return c, fmt.Errorf("the count of the old sections cannot be read: %w", err)
}
if c.V != 1 {
return c, fmt.Errorf("a count of the old sections of version %d, not 1", c.V)
}
return c, nil
}
// heldOldSections is the count the last pass wrote (the accepted set it held): a module that keeps using a section was
// accepted with it, so it is counted, and a fresh manifest changes nothing in it. Nil when none is held yet or it cannot be
// read; a failed read is said, never taken as 0 (issue 1966 D4).
func (ix *Indexer) heldOldSections(ctx context.Context) judge.OldSectionCounts {
value, _, found, err := ix.OldSections.Get(ctx, OldSectionsKey)
if err == nil && found {
var c OldSectionsCount
if c, err = DecodeOldSections(value); err == nil {
return c.Counts
}
}
if err != nil {
ix.logf("judge: the count of the old sections cannot be read (%v), so the zero rule is not applied in this pass", err)
}
return nil
}
// keepOldSections writes the counts under OldSectionsKey when they differ from the ones held.
func (ix *Indexer) keepOldSections(ctx context.Context, counts judge.OldSectionCounts) error {
if ix.OldSections == nil {
return nil
}
return write(ctx, ix.OldSections, OldSectionsKey, func(value []byte, found bool) ([]byte, bool, error) {
if found {
if cur, err := DecodeOldSections(value); err == nil && maps.Equal(cur.Counts, counts) {
return nil, false, nil
}
}
next, err := json.Marshal(OldSectionsCount{V: 1, Counts: counts})
return next, false, err
})
}
@@ -213,6 +213,10 @@ type TakenIn struct {
Commit string `json:"commit,omitempty"`
Path string `json:"path,omitempty"`
Answer string `json:"answer"`
// Handed and Dropped are the modules a take-in of a repository (TakeInRepository) moved to another repository's trunk
// and deleted.
Handed []string `json:"handed,omitempty"`
Dropped []string `json:"dropped,omitempty"`
}
// TakeIn reads a registered module's manifest now, whatever was tried before, and writes it as a merge would (the tool
@@ -358,3 +362,61 @@ func (ix *Indexer) takeInHeld(ctx context.Context, module string) (TakenIn, bool
}
return out, true
}
// TakeInRepository takes in the head the gate's source names for a repository whose merged message the indexer never
// heard (the tool indexer_take_in with repository, issue 1367). A head whose trunk holds no module.json is applied as
// the merged message of a merge that leaves none: every module of the repository goes, one another trunk holds handed over
// from there first. A head whose trunk holds modules is answered "by name": each is taken in with its own name.
func (ix *Indexer) TakeInRepository(ctx context.Context, repository string) TakenIn {
out := TakenIn{Repository: repository}
ix.mu.Lock()
head := ""
for repo, h := range ix.source.heads {
if delivery.SameRepository(repo, repository) && h != "" {
head, out.Repository = h, repo
}
}
ix.mu.Unlock()
if head == "" || ix.Modules == nil || ix.Trunk == nil {
out.Answer = "the gate's source names no head of " + repository + ", or the indexer reads no forge or writes no modules"
return out
}
out.Commit = head
dirs, err := ix.Trunk(ctx, out.Repository, head)
if err != nil {
out.Answer = fmt.Sprintf("the modules of %s at %.12s cannot be read, so nothing is taken in: %v", repository, head, err)
return out
}
if len(dirs) > 0 {
out.Answer = fmt.Sprintf("%s at %.12s holds modules (%s): take each module in by name (module)", repository, head, strings.Join(dirs, ", "))
return out
}
before, err := ix.heldModules(ctx)
if err != nil {
out.Answer = "the modules state cannot be read: " + err.Error()
return out
}
f := delivery.Forge{V: delivery.Version, Kind: delivery.ForgeMerged, Repository: out.Repository, Merge: head}
if _, err := ix.ApplyMessage(ctx, f); err != nil {
out.Answer = fmt.Sprintf("the merge of %s at %.12s could not be taken in: %v", repository, head, err)
return out
}
after, err := ix.heldModules(ctx)
if err != nil {
out.Answer = "the modules state cannot be read: " + err.Error()
return out
}
for _, key := range slices.Sorted(maps.Keys(before)) {
if !delivery.SameRepository(before[key].Repository, repository) {
continue
}
if now, ok := after[key]; ok {
out.Handed = append(out.Handed, key+" to "+now.Repository+" at "+now.Commit)
} else {
out.Dropped = append(out.Dropped, key)
}
}
out.Stored = true
out.Answer = fmt.Sprintf("%s at %.12s holds no module.json: taken in as a merge that leaves none; handed over %v, dropped %v", repository, head, out.Handed, out.Dropped)
return out
}
@@ -82,6 +82,9 @@ func (ix *Indexer) ApplySource(ctx context.Context, rev uint64, s delivery.Sourc
return nil, false, nil
}
cur.Src = rev
if cur.FPSrc == 0 {
cur.FPSrc = rev // its fingerprint moved at this merge, whose revision is heard only now (ADR 0601 §1)
}
data, err := delivery.EncodeModule(cur)
return data, false, err
})
@@ -31,7 +31,7 @@ func TestASourceRevisionStampsTheModulesOfTheRepositoriesHeadItNames(t *testing.
}
m, ok := moduleOf(t, mods, "a")
want := before
want.Src = 7
want.Src, want.FPSrc = 7, 7 // its fingerprint moved at that merge (ADR 0601 §1)
if !ok || !reflect.DeepEqual(m, want) {
t.Errorf("module a = %+v (%v), want %+v: the fingerprint and the commit as the merge wrote them, with Src 7", m, ok, want)
}
@@ -25,9 +25,13 @@ const (
// TakeInAsk is a value of the takeins state: which module a person asked to be taken in, why, who and when, and, once the
// indexer read it, its answer.
type TakeInAsk struct {
V int `json:"v"`
Module string `json:"module,omitempty"`
Why string `json:"why,omitempty"`
V int `json:"v"`
Module string `json:"module,omitempty"`
// Repository asks the head the gate's source names for a repository to be taken in (TakeInRepository) instead of a module.
Repository string `json:"repository,omitempty"`
Why string `json:"why,omitempty"`
// Judge asks the manifests to be judged again now (indexer_judge_again) instead of the module to be taken in.
Judge bool `json:"judge,omitempty"`
By string `json:"by,omitempty"`
At time.Time `json:"at"`
Result *TakenIn `json:"result,omitempty"`
@@ -46,8 +50,8 @@ func DecodeTakeIn(data []byte) (TakeInAsk, error) {
if a.V != 1 {
return a, fmt.Errorf("a takeins value of version %d, not 1", a.V)
}
if a.Module == "" {
return a, errors.New("a takeins value naming no module")
if a.Module == "" && a.Repository == "" {
return a, errors.New("a takeins value naming no module or repository")
}
return a, nil
}
@@ -65,3 +69,14 @@ func NewTakeIn(module, why, by string, at time.Time) (string, []byte, error) {
data, err := json.Marshal(TakeInAsk{V: 1, Module: module, Why: strings.TrimSpace(why), By: by, At: at.UTC()})
return key, data, err
}
// NewTakeInRepository is the value indexer_take_in writes for a repository: its key and the value.
func NewTakeInRepository(repository, why, by string, at time.Time) (string, []byte, error) {
repository = strings.TrimSpace(repository)
key, err := delivery.RepositoryKey(repository)
if err != nil {
return "", nil, err
}
data, err := json.Marshal(TakeInAsk{V: 1, Repository: repository, Why: strings.TrimSpace(why), By: by, At: at.UTC()})
return "repository." + key, data, err
}
@@ -0,0 +1,4 @@
{
"dependents": [],
"module": "mesh-build-go"
}
@@ -0,0 +1,18 @@
{
"module": {
"commit": "b5475cfa5c3daffd52b5b25eb7063e745b976026",
"manifest": {
"module": "gitea",
"requires": [
"postgres-database",
"route",
"secret"
],
"version": "1"
},
"module": "gitea",
"path": "modules/gitea",
"ref": "refs/pull/571/head",
"repository": "novox/mesh-catalog"
}
}
@@ -0,0 +1,22 @@
{
"modules": [
{
"commit": "b5475cfa5c3daffd52b5b25eb7063e745b976026",
"module": "gitea",
"path": "modules/gitea",
"repository": "novox/mesh-catalog"
},
{
"commit": "dc90b3f5bd1386747ac70f2792169babf8eeed73",
"module": "mesh-controller",
"path": ".",
"repository": "novox/mesh-controller"
},
{
"commit": "432326264f6c34ceec1fb67f5ca9d450b4caaa1d",
"module": "postgres",
"path": "modules/postgres",
"repository": "novox/mesh-catalog"
}
]
}
@@ -0,0 +1,15 @@
{
"provides": [
{
"commit": "432326264f6c34ceec1fb67f5ca9d450b4caaa1d",
"module": "postgres",
"scope": "mesh"
}
],
"requires": [
{
"commit": "b5475cfa5c3daffd52b5b25eb7063e745b976026",
"module": "gitea"
}
]
}
@@ -111,6 +111,7 @@ type toolchainOrNone struct {
type known struct {
key string
cur delivery.Module
manifest catalogue.Manifest
languages map[string]string
bases []string
contexts bool
@@ -196,7 +197,7 @@ func (ix *Indexer) recomputeModules(ctx context.Context, overlay map[string]deli
cur, stale[key] = again, true
held[key] = cur
}
all[key] = &known{key: key, cur: cur, languages: languagesOf(m), bases: basesOf(m), contexts: hasContext(m), held: before}
all[key] = &known{key: key, cur: cur, manifest: m, languages: languagesOf(m), bases: basesOf(m), contexts: hasContext(m), held: before}
}
tools := ix.agreed()
published := ix.publishedAll(ctx)
@@ -278,9 +279,11 @@ func (ix *Indexer) recomputeModules(ctx context.Context, overlay map[string]deli
return r
}
finals := map[string]delivery.Module{}
edges := map[string]graphEdges{} // the module graph's nodes of this recompute (graph.go)
for _, key := range slices.Sorted(maps.Keys(all)) {
k := all[key]
r := of(key, nil)
edges[key] = graphEdges{module: key, cur: k.cur, manifest: k.manifest, fp: r.fp, none: r.none, tcs: r.tcs, pkgs: r.pkgs}
if _, merged := overlay[key]; merged {
final := overlay[key]
final.FP2, final.None2, final.Inputs2, final.Toolchains, final.Packages = r.fp, r.none, k.cur.Inputs2, r.tcs, r.pkgs
@@ -312,6 +315,13 @@ func (ix *Indexer) recomputeModules(ctx context.Context, overlay map[string]deli
if cur.FP2 == r.fp && cur.None2 == r.none && reflect.DeepEqual(cur.Toolchains, r.tcs) && reflect.DeepEqual(cur.Packages, r.pkgs) {
return nil, false, nil
}
if cur.FP2 != r.fp {
// Moved with no merge (a toolchain, a package, a base): no release carries this build, and the source revision
// moves only at a merge, so it is not known (0) and the move is asked of a person; never the revision of now,
// which may be the very release that last merged the module (ADR 0601 §1). The source stamp leaves it, since
// Src is set.
cur.FPSrc = 0
}
cur.FP2, cur.None2, cur.Toolchains, cur.Packages = r.fp, r.none, r.tcs, r.pkgs
cur.Inputs2, cur.ArtifactPaths = inputs, paths
data, err := delivery.EncodeModule(cur)
@@ -325,7 +335,10 @@ func (ix *Indexer) recomputeModules(ctx context.Context, overlay map[string]deli
w.wrote(key)
}
}
return ix.writeOverlay(ctx, overlay, finals, w)
if err := ix.writeOverlay(ctx, overlay, finals, w); err != nil {
return err
}
return ix.writeGraph(ctx, edges)
}
// writeOverlay writes the records of a merge, each once and only if it differs from what the bucket holds; finals has
@@ -353,7 +366,14 @@ func (ix *Indexer) writeOverlay(ctx context.Context, overlay, finals map[string]
if held && cur.Manifest == next.Manifest {
// The judge's verdict stands while the digest does (ADR 0578 §3, ADR 0579 §1): a recompute never writes a
// refused or outgrown module as accepted; only the judge changes it.
next.Verdict, next.Problems, next.Warnings = cur.Verdict, cur.Problems, cur.Warnings
next.Verdict, next.Problems, next.Unknown, next.Warnings = cur.Verdict, cur.Problems, cur.Unknown, cur.Warnings
}
// The source revision at which FP2 last changed (ADR 0601 §1): this merge's when it moved it (0 until the
// source revision naming the merge is heard, which stamps it), else the one held.
if held && cur.FP2 == next.FP2 {
next.FPSrc = cur.FPSrc
} else {
next.FPSrc = next.Src
}
if held && sameButWhere(cur, next) {
return nil, false, nil
@@ -379,8 +399,8 @@ func (ix *Indexer) writeOverlay(ctx context.Context, overlay, finals map[string]
// which only the judge writes: the module is unchanged.
func sameButWhere(cur, next delivery.Module) bool {
next = withV(next)
next.Commit, next.Src = cur.Commit, cur.Src
next.Verdict, next.Problems, next.Warnings = cur.Verdict, cur.Problems, cur.Warnings
next.Commit, next.Src, next.FPSrc = cur.Commit, cur.Src, cur.FPSrc
next.Verdict, next.Problems, next.Unknown, next.Warnings = cur.Verdict, cur.Problems, cur.Unknown, cur.Warnings
return reflect.DeepEqual(cur, next)
}
@@ -2,12 +2,17 @@ package index
import (
"context"
"errors"
"fmt"
"reflect"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-delivery/sdk/graph"
"mesh-delivery-indexer/internal/fingerprint"
)
@@ -60,6 +65,20 @@ type mod struct {
// merged applies one merge of the catalogue carrying these modules (every one a Go bundle's go.sum, a TypeScript bundle's
// package-lock.json) and returns the indexer.
// graphOf is the graph node the indexer holds of module.
func graphOf(t *testing.T, ix *Indexer, module string) graph.Node {
t.Helper()
v, ok := ix.Graph.(*memStore).vals[module]
if !ok {
t.Fatalf("graph.%s is not held", module)
}
g, err := graph.Decode(v)
if err != nil {
t.Fatal(err)
}
return g
}
func newFormIndexer(t *testing.T, mods ...mod) (*Indexer, *memStore) {
t.Helper()
ix, store, f := newFormIndexerUnmerged(t, mods...)
@@ -98,12 +117,17 @@ func toolchainValue(t *testing.T, language, version, recipe string) []byte {
}
func reportToolchain(t *testing.T, ix *Indexer, module, node, language, version string) {
t.Helper()
reportToolchainRecipe(t, ix, module, node, language, version, "1")
}
func reportToolchainRecipe(t *testing.T, ix *Indexer, module, node, language, version, recipe string) {
t.Helper()
key, err := delivery.ToolchainKey(node, language)
if err != nil {
t.Fatal(err)
}
if err := ix.ApplyToolchain(context.Background(), module, key, toolchainValue(t, language, version, "1"), false); err != nil {
if err := ix.ApplyToolchain(context.Background(), module, key, toolchainValue(t, language, version, recipe), false); err != nil {
t.Fatal(err)
}
}
@@ -111,14 +135,75 @@ func reportToolchain(t *testing.T, ix *Indexer, module, node, language, version
func TestAgreedToolchainChangesAGoModulesReasonWithNoMerge(t *testing.T) { // 1444/AC5
ix, mods := newFormIndexer(t, mod{"avahi", catalogueManifest(t, "avahi")})
if m, _ := moduleOf(t, mods, "avahi"); m.None2 != delivery.ToolchainNoReport {
t.Run("a recipe change builds a Go module at once and again once the agreement moves", aRecipeChangeBuildsAGoModuleAtOnceAndAgainOnceTheAgreementMoves)
t.Fatalf("before: none2 = %q", m.None2)
}
// The release that last merged avahi: its source revision names avahi's commit, so Src (and FPSrc) are 7.
merged, _ := moduleOf(t, mods, "avahi")
src := delivery.Source{Group: "g", Heads: map[string]string{merged.Repository: merged.Commit}}
if err := ix.ApplySource(context.Background(), 7, src); err != nil {
t.Fatal(err)
}
if m, _ := moduleOf(t, mods, "avahi"); m.Src != 7 || m.FPSrc != 7 {
t.Fatalf("before: src %d fp-src %d, want 7 and 7", m.Src, m.FPSrc)
}
if g := graphOf(t, ix, "avahi"); g.None != delivery.ToolchainNoReport || g.Toolchains != nil {
t.Errorf("before: graph.avahi = %+v, want no toolchain edge and the reason", g)
}
reportToolchain(t, ix, "mesh-build-go", "novox", "go", "go1.27.1")
reportToolchain(t, ix, "mesh-build-go", "ace", "go", "go1.27.1")
m, _ := moduleOf(t, mods, "avahi")
// L3 (ADR 0603 §2, §3): the agreement moving rewrites the module's toolchain edge, and again when it moves on.
if g := graphOf(t, ix, "avahi"); g.FP != m.FP2 || g.Toolchains["go"].Version != "go1.27.1" || g.Toolchains["go"].Module != "mesh-build-go" {
t.Errorf("after: graph.avahi = %+v, want the toolchain edge of go1.27.1", g)
}
reportToolchain(t, ix, "mesh-build-go", "novox", "go", "go1.28.0")
reportToolchain(t, ix, "mesh-build-go", "ace", "go", "go1.28.0")
if g := graphOf(t, ix, "avahi"); g.Toolchains["go"].Version != "go1.28.0" {
t.Errorf("moved on: graph.avahi = %+v, want the toolchain edge of go1.28.0", g)
}
reportToolchain(t, ix, "mesh-build-go", "novox", "go", "go1.27.1")
reportToolchain(t, ix, "mesh-build-go", "ace", "go", "go1.27.1")
m, _ = moduleOf(t, mods, "avahi")
if m.None2 != "" || m.FP2 == "" {
t.Errorf("none2 = %q fp2 = %q, want a fingerprint: no builder input is waited for (ADR 0530 §1)", m.None2, m.FP2)
}
// ADR 0601 §1 (review round 2): a fingerprint moved with no merge has no release that carries it: fp-src 0, not known,
// though the source revision is still the release's at the module's own Src; a source revision heard again stamps nothing.
if m.FPSrc != 0 {
t.Errorf("a fingerprint moved by a recompute: fp-src %d, want 0", m.FPSrc)
}
if err := ix.ApplySource(context.Background(), 8, src); err != nil {
t.Fatal(err)
}
if again, _ := moduleOf(t, mods, "avahi"); again.FPSrc != 0 {
t.Errorf("a source revision stamped the recomputed fingerprint: fp-src %d", again.FPSrc)
}
}
// ADR 0603 §3 (D9): a source revision that changes the Go build module's recipe and a Go module builds the module at once, with
// the recipe the nodes run, and again once the agreement moves to the new one: the module never waits and never loses its
// fingerprint between the two, and its toolchain edge names the recipe each was computed with.
func aRecipeChangeBuildsAGoModuleAtOnceAndAgainOnceTheAgreementMoves(t *testing.T) {
ix, mods := newFormIndexer(t, mod{"avahi", catalogueManifest(t, "avahi")})
for _, node := range []string{"novox", "ace"} {
reportToolchain(t, ix, "mesh-build-go", node, "go", "go1.27.1")
}
first, _ := moduleOf(t, mods, "avahi")
if first.FP2 == "" || graphOf(t, ix, "avahi").Toolchains["go"].Recipe != "1" {
t.Fatalf("with the running recipe: fp2 %q, graph %+v", first.FP2, graphOf(t, ix, "avahi"))
}
// The release carrying the new recipe is merged, but the nodes still run the old one: one node reporting the new recipe is
// no strict majority, so the fingerprint stays, nothing waits.
reportToolchainRecipe(t, ix, "mesh-build-go", "novox", "go", "go1.27.1", "2")
if held, _ := moduleOf(t, mods, "avahi"); held.FP2 != first.FP2 || held.None2 != "" {
t.Errorf("one node on the new recipe: fp2 %q none2 %q, want the fingerprint of the running recipe, %q", held.FP2, held.None2, first.FP2)
}
reportToolchainRecipe(t, ix, "mesh-build-go", "ace", "go", "go1.27.1", "2")
again, _ := moduleOf(t, mods, "avahi")
if again.FP2 == "" || again.FP2 == first.FP2 || again.None2 != "" || graphOf(t, ix, "avahi").Toolchains["go"].Recipe != "2" {
t.Errorf("the agreement moved: fp2 %q (was %q) none2 %q graph %+v, want a second fingerprint at recipe 2", again.FP2, first.FP2, again.None2, graphOf(t, ix, "avahi"))
}
}
func TestNodesReportingDifferentVersionsLeaveTheReasonNamingBoth(t *testing.T) { // 1444/AC6
@@ -182,6 +267,59 @@ func TestBasesAreComputedFirstAndNoFingerprintTakesTheBuilder(t *testing.T) { //
if m, _ := moduleOf(t, mods, "other"); m.FP2 != "" || !strings.Contains(m.None2, "ts-base") || !strings.Contains(m.None2, "base") {
t.Errorf("other: fp2 = %q none2 = %q, want none2 naming the base ts-base", m.FP2, m.None2)
}
// L3 (ADR 0603 §2): the module graph holds each module's edges as this recompute computed them.
goTool := map[string]graph.Toolchain{"go": {Module: "mesh-build-go", Version: "go1.27.1", Recipe: "1"}}
if g := graphOf(t, ix, "avahi"); g.FP != want["avahi"] || !reflect.DeepEqual(g.Bases, []string{"mesh-tools"}) ||
!reflect.DeepEqual(g.Toolchains, goTool) || g.Repository != catalogueRepo || g.Path != "modules/avahi" {
t.Errorf("graph.avahi = %+v", g)
}
if g := graphOf(t, ix, "other"); g.FP != "" || !strings.Contains(g.None, "ts-base") || !reflect.DeepEqual(g.Bases, []string{"ts-base"}) {
t.Errorf("graph.other = %+v, want its base and its reason", g)
}
graph := ix.Graph.(*memStore)
written := graph.writes
if err := ix.Recompute(context.Background()); err != nil || graph.writes != written {
t.Errorf("a recompute that moves nothing wrote the graph %d times (%v)", graph.writes-written, err)
}
// A module the modules state no longer holds (a merge dropped it, issue 1900) leaves the graph at the next recompute.
delete(mods.vals, "other")
if err := ix.Recompute(context.Background()); err != nil {
t.Fatal(err)
}
if _, held := graph.vals["other"]; held {
t.Error("graph.other is still held after its module went")
}
if _, held := graph.vals["avahi"]; !held {
t.Error("graph.avahi went with another module")
}
// Review finding 3 (ADR 0603 §2): a graph state not on the bus yet (process.ErrNotYet), or one whose listing fails, never
// fails a recompute; the first writes nothing and says so, the second skips the deletion pass and says so.
graph.vals["ghost"], graph.revs["ghost"] = []byte("x"), 1
var said []string
ix.Logf = func(format string, args ...any) { said = append(said, fmt.Sprintf(format, args...)) }
for _, row := range []struct {
name string
err error
say string
}{
{"not on the bus yet", process.ErrNotYet, "not on the bus yet"},
{"a listing that fails", errors.New("the bus is slow"), "are not deleted"},
} {
said, graph.keysErr = nil, row.err
if err := ix.Recompute(context.Background()); err != nil {
t.Errorf("%s: the recompute failed: %v", row.name, err)
}
if _, held := graph.vals["ghost"]; !held || !strings.Contains(strings.Join(said, "\n"), row.say) {
t.Errorf("%s: ghost held %v, log %q, want it kept and %q said", row.name, held, said, row.say)
}
}
graph.keysErr = nil
if err := ix.Recompute(context.Background()); err != nil {
t.Fatal(err)
}
if _, held := graph.vals["ghost"]; held {
t.Error("graph.ghost is still held once the graph can be listed")
}
a := agentBuild("build-1792587792086548572", agentSrcY, "2026-10-21T15:03:00Z")
_ = ix.Builds.Put(a[1].([]byte))
if err := ix.Recompute(context.Background()); err != nil {
@@ -7,6 +7,8 @@ import (
"fmt"
"maps"
"slices"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
@@ -18,19 +20,23 @@ import (
// The indexer judges every manifest it holds, once, with its own judge (novox/hq ADR 0578 §1): it writes each verdict in
// its state verdicts (delivery.VerdictsBucket), one key per manifest digest, only when the verdict differs from the one
// held; a module's value in the modules state carries a refusal (Verdict, Problems), so a reader of modules takes it
// without a second read; a new refusal is said as the event manifest-refused. It judges at its start, so a judge that
// without a second read; a new refusal is said as the event manifest-refused; a manifest with a field the judge does not know waits instead (ownBuildIsNewest). It judges at its start, so a judge that
// changed (a new indexer) judges every manifest again, and after every merge it takes in, since a rule between manifests
// can turn on any of them. Nothing polls.
// VerdictsBucket is the indexer's state of verdicts.
// VerdictsBucket is the indexer's state of verdicts; VerdictsState is its local name, as the tools bundle reads it.
var VerdictsBucket = delivery.VerdictsBucket
const VerdictsState = "verdicts"
// Judged is what one pass of the judge did: the manifests judged, the verdicts written, and the modules refused.
type Judged struct {
Judged, Written int
Refused map[string][]string
// Outgrown are the modules accepted before whose problems are now warnings (ADR 0579 §1).
Outgrown []string
// Waiting are the modules whose manifest carries a field this judge does not know and waits.
Waiting []string
}
// JudgeAll judges every manifest the modules state names, as one catalogue with the predefined list the indexer keeps,
@@ -43,6 +49,10 @@ type Judged struct {
// watchman raises as manifest-outgrown. A manifest in fresh (the digests a merge just took in) with no verdict yet, and
// one already refused, is judged in full.
func (ix *Indexer) JudgeAll(ctx context.Context, fresh map[string]bool) (Judged, error) {
// A merge, the start pass and indexer_judge may all judge at once: one pass at a time, so none writes a verdict from
// modules another is rewriting.
ix.judgeMu.Lock()
defer ix.judgeMu.Unlock()
var out Judged
if ix.Verdicts == nil || ix.Modules == nil || ix.Manifests == nil {
return out, nil
@@ -74,19 +84,44 @@ func (ix *Indexer) JudgeAll(ctx context.Context, fresh map[string]bool) (Judged,
raws[name] = []byte(kept.JSON)
digests[name] = digest
}
refused, unnamed := judge.Shelf(raws, ix.predefinedList(ctx), 0)
predefined := ix.predefinedList(ctx)
// The old sections end when no module is left in them (issue 1966 D4): the count is the accepted set the last pass held.
var counts judge.OldSectionCounts
if ix.OldSections != nil {
counts = ix.heldOldSections(ctx)
}
refused, waits, unnamed := judge.ShelfWaiting(raws, predefined, 0, counts)
for _, p := range unnamed {
ix.logf("judge: a problem between the manifests that no one module's removal ends, so it refuses none: %s", p)
}
out.Refused = refused
// A manifest with a field this judge does not know waits for a judge that may know it (clause 3 of the record on a
// release's Approve; ADR 0578 §1): it is refused only when this program is the newest build of the indexer the trunk
// holds, so no judge that could know the field is still to come.
newest := ix.ownBuildIsNewest(held)
id := ix.JudgeID
if id == "" {
id = judge.Identity()
}
var accepted []catalogue.Manifest
for _, name := range slices.Sorted(maps.Keys(digests)) {
out.Judged++
v := delivery.ManifestVerdict{Module: name, State: delivery.VerdictAccepted, Judge: id, At: ix.now()}
if problems := refused[name]; len(problems) > 0 {
if fields := waits[name]; len(fields) > 0 {
accepted := ix.acceptedBefore(ctx, digests[name], fresh)
if !newest && !accepted {
v.State, v.Unknown = delivery.VerdictManifestWaits, fields
out.Waiting = append(out.Waiting, name)
} else {
_, problems := judge.Manifest(raws[name], ix.predefinedList(ctx))
if accepted {
v.Warnings = problems
out.Outgrown = append(out.Outgrown, name)
} else {
v.State, v.Problems = delivery.VerdictRefused, problems
}
}
} else if problems := refused[name]; len(problems) > 0 {
if ix.acceptedBefore(ctx, digests[name], fresh) {
v.Warnings = problems
out.Outgrown = append(out.Outgrown, name)
@@ -94,6 +129,12 @@ func (ix *Indexer) JudgeAll(ctx context.Context, fresh map[string]bool) (Judged,
v.State, v.Problems = delivery.VerdictRefused, problems
}
}
if !v.Refused() {
if m, err := catalogue.Decode(raws[name]); err == nil {
m.Module = name
accepted = append(accepted, m)
}
}
wrote, err := ix.writeVerdict(ctx, digests[name], v)
if errors.Is(err, process.ErrNotYet) {
return out, fmt.Errorf("the state %s is not on the bus yet: %w", VerdictsBucket, err)
@@ -104,23 +145,35 @@ func (ix *Indexer) JudgeAll(ctx context.Context, fresh map[string]bool) (Judged,
}
if wrote {
out.Written++
if v.Waits() {
ix.logf("judge: %s", v.Why())
}
if v.Refused() {
ix.logf("judge: %s", v.Why())
if ix.SayRefused != nil {
ix.SayRefused(ctx, delivery.RefusedOf(digests[name], v))
}
} else if ix.SayAccepted != nil {
// A manifest asking for bus state, newly accepted, is said whole to the bus module (issue 1966 D3, ADR 0595 §4).
if m, problems := judge.Manifest(raws[name], predefined); len(problems) == 0 && len(m.BusState) > 0 {
ix.SayAccepted(ctx, delivery.AcceptedOf(digests[name], v, m))
}
}
}
if err := ix.carryVerdict(ctx, name, digests[name], v); err != nil {
ix.logf("judge: modules.%s does not carry its verdict: %v", name, err)
}
}
if err := ix.keepOldSections(ctx, judge.CountOldSections(accepted)); err != nil {
ix.logf("judge: the count of the old sections is not kept: %v", err)
}
return out, nil
}
// JudgeAtStart is JudgeAll of every manifest held, none fresh, as the indexer starts (a judge that changed judges every
// one again); one that fails is judged again at the next message or source revision heard (kickJudge).
func (ix *Indexer) JudgeAtStart(ctx context.Context) (Judged, error) {
ix.learnOwnBuild(ctx)
j, err := ix.JudgeAll(ctx, nil)
ix.judgePending.Store(err != nil)
return j, err
@@ -149,7 +202,7 @@ func (ix *Indexer) acceptedBefore(ctx context.Context, digest string, fresh map[
return !fresh[digest]
}
prev, err := delivery.DecodeManifestVerdict(value)
return err == nil && !prev.Refused()
return err == nil && prev.State == delivery.VerdictAccepted
}
// writeVerdict writes v under the digest's key unless the verdict held there says the same (ManifestVerdict.Same); wrote
@@ -173,6 +226,36 @@ func (ix *Indexer) writeVerdict(ctx context.Context, digest string, v delivery.M
return wrote, err
}
// learnOwnBuild asks where the indexer's program is told which build it runs (OwnBuild: the node-engine's seat, never the
// trunk's value of the indexer, which names a build that may be merged and not delivered). Asked at the start pass; where
// it is not answered the build stays unknown and every unknown field waits, which is the safe side.
func (ix *Indexer) learnOwnBuild(ctx context.Context) {
if ix.OwnBuild == nil {
return
}
fp, err := ix.OwnBuild(ctx)
if err != nil || fp == "" {
ix.logf("judge: the build this program runs is not known (%v), so a manifest with a field it does not know waits and is not refused", err)
return
}
ix.ownMu.Lock()
ix.ownFP = fp
ix.ownMu.Unlock()
}
// ownBuildIsNewest says whether the build of the indexer this program runs (ownFP, from the node-engine) is the newest the
// trunk holds: the indexer's own modules value names it. A newer build merged and not yet delivered ends it at the trunk's
// merge, as one that pins a changed mesh-sdk always is; false when either is unknown.
func (ix *Indexer) ownBuildIsNewest(held map[string]delivery.Module) bool {
fp := held[delivery.IndexerModule].FP2
if fp == "" {
return false
}
ix.ownMu.Lock()
defer ix.ownMu.Unlock()
return ix.ownFP != "" && ix.ownFP == fp
}
// carryVerdict puts a refusal into the module's value, and takes one out once the manifest is accepted, while the value
// still names the digest judged; an accepted module's value carries nothing, so judging writes no value of a module the
// judge accepts (a reader takes a value without a verdict as accepted, delivery.TakeManifestVerdict).
@@ -189,14 +272,18 @@ func (ix *Indexer) carryVerdict(ctx context.Context, name, digest string, v deli
if err != nil || cur.Manifest != digest {
return nil, false, nil // gone, unreadable, or moved to another manifest since: the next pass judges it
}
state := ""
if v.Refused() {
// A wait is written as a refusal naming the fields, so a reader pinned before the verdict waits holds it as refused.
state, problems := "", v.Problems
switch {
case v.Waits():
state, problems = delivery.VerdictRefused, []string{v.WaitsOn()}
case v.Refused():
state = delivery.VerdictRefused
}
if cur.Verdict == state && slices.Equal(cur.Problems, v.Problems) && slices.Equal(cur.Warnings, v.Warnings) {
if cur.Verdict == state && slices.Equal(cur.Problems, problems) && slices.Equal(cur.Unknown, v.Unknown) && slices.Equal(cur.Warnings, v.Warnings) {
return nil, false, nil
}
cur.Verdict, cur.Problems, cur.Warnings = state, v.Problems, v.Warnings
cur.Verdict, cur.Problems, cur.Unknown, cur.Warnings = state, problems, v.Unknown, v.Warnings
if state == "" {
cur.Problems = nil
}
@@ -226,3 +313,111 @@ func (ix *Indexer) predefinedList(ctx context.Context) catalogue.PredefinedList
}
return list
}
// The tools that read and steer the verdicts (the instruction "tools are how a module is spoken to"): indexer_verdicts
// lists the manifests that wait or are refused, and indexer_judge judges one module again now.
const (
VerdictsTool = "indexer_verdicts"
JudgeAgainTool = "indexer_judge"
judgeAgainPrefix = "judge."
)
// VerdictRow is one verdict as indexer_verdicts lists it: the manifest's digest, the module, its state, the fields it waits
// on or the problems that refuse it, and the judge that made it.
type VerdictRow struct {
Module string `json:"module"`
Digest string `json:"digest"`
State string `json:"state"`
Words string `json:"words,omitempty"`
Unknown []string `json:"unknown,omitempty"`
Problems []string `json:"problems,omitempty"`
Warnings []string `json:"warnings,omitempty"`
Judge string `json:"judge"`
At time.Time `json:"at"`
}
// ListVerdicts reads the verdicts under keys with get and lists those of module (every module when empty) that wait or are
// refused, or every one with all, sorted by module and digest. A value that does not read is said in unread by its key.
func ListVerdicts(keys []string, get func(key string) ([]byte, error), module string, all bool) (rows []VerdictRow, unread map[string]string) {
unread = map[string]string{}
for _, key := range keys {
value, err := get(key)
if err != nil || value == nil {
if err != nil {
unread[key] = err.Error()
}
continue
}
v, err := delivery.DecodeManifestVerdict(value)
if err != nil {
unread[key] = err.Error()
continue
}
if (module != "" && v.Module != module) || (!all && !v.Held()) {
continue
}
rows = append(rows, VerdictRow{Module: v.Module, Digest: key, State: v.State, Words: v.Why(), Unknown: v.Unknown,
Problems: v.Problems, Warnings: v.Warnings, Judge: v.Judge, At: v.At})
}
slices.SortFunc(rows, func(a, b VerdictRow) int {
if c := strings.Compare(a.Module, b.Module); c != 0 {
return c
}
return strings.Compare(a.Digest, b.Digest)
})
return rows, unread
}
// JudgeAgain judges the manifests again now, as at the start, and answers how the module's stands: the tool
// indexer_judge. A module the modules state does not hold, or whose manifest is not in the manifests state, is
// answered so.
func (ix *Indexer) JudgeAgain(ctx context.Context, module string) TakenIn {
out := TakenIn{Module: module}
held, err := ix.heldModules(ctx)
if err != nil {
out.Answer = "the modules state cannot be read: " + err.Error()
return out
}
m, ok := held[module]
if !ok || m.Manifest == "" {
out.Answer = "the indexer holds no manifest of " + module
return out
}
// The build this program runs is asked again, so a start pass that missed the engine's answer is recovered without a restart.
ix.learnOwnBuild(ctx)
if _, err := ix.JudgeAll(ctx, nil); err != nil {
out.Answer = "the manifests could not be judged: " + err.Error()
return out
}
value, _, found, err := ix.Verdicts.Get(ctx, m.Manifest)
if err != nil || !found {
out.Answer = "judged, but the verdict on " + module + " cannot be read"
return out
}
v, err := delivery.DecodeManifestVerdict(value)
if err != nil {
out.Answer = "judged, but the verdict on " + module + " cannot be read: " + err.Error()
return out
}
out.Stored = true
out.Answer = "judged again by " + v.Judge + ": " + v.State
if v.Held() {
out.Answer += " (" + v.Why() + ")"
}
return out
}
// NewJudgeAgain is the value indexer_judge writes for a module: its key in the takeins state and the value.
func NewJudgeAgain(module, why, by string, at time.Time) (string, []byte, error) {
key, data, err := NewTakeIn(module, why, by, at)
if err != nil {
return "", nil, err
}
var a TakeInAsk
if err := json.Unmarshal(data, &a); err != nil {
return "", nil, err
}
a.Judge = true
data, err = json.Marshal(a)
return judgeAgainPrefix + key, data, err
}
@@ -2,6 +2,8 @@ package index
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"time"
@@ -74,6 +76,11 @@ func TestTheIndexerJudgesEveryManifestOnce(t *testing.T) {
if j, err := ix.JudgeAll(ctx, nil); err != nil || j.Written != 0 || len(said) != 1 {
t.Fatalf("a judge that judges alike: %+v %v, said %d", j, err, len(said))
}
// ADR 0603 §4 (D9): a source revision that changes the indexer builds the new indexer, which at its start judges every
// manifest it holds again, none of them fresh: all of them judged, nothing written for a verdict that stands.
if j, err := ix.JudgeAtStart(ctx); err != nil || j.Judged != 2 || j.Written != 0 || len(said) != 1 {
t.Fatalf("the new indexer at its start: %+v %v, said %d, want both manifests judged and nothing written", j, err, len(said))
}
// The manifest mended: its new digest is accepted and the module's refusal goes.
mended := hold("bad", `{"module":"bad","version":"2"}`)
if j, err := ix.JudgeAll(ctx, map[string]bool{mended: true}); err != nil || j.Written != 1 || len(j.Refused) != 0 {
@@ -118,9 +125,171 @@ func TestTheIndexerJudgesEveryManifestOnce(t *testing.T) {
if err := ix.writeOverlay(ctx, map[string]delivery.Module{"old": same}, nil, w); err != nil || !w.fresh[other] {
t.Fatalf("a changed digest is not fresh: %v %v", w.fresh, err)
}
// 1966/AC10: a newly accepted manifest asking for bus state is said once as manifest-accepted, carrying both entries
// whole; a verdict already held, and a manifest asking for none, say nothing.
var accepted []delivery.ManifestAccepted
ix.SayAccepted = func(_ context.Context, a delivery.ManifestAccepted) { accepted = append(accepted, a) }
moved := hold("moved", `{"module":"moved","version":"1","requires":[`+
`{"name":"cache","storage":"key-value","template":"state","options":{"history":8}},`+
`{"name":"audit","storage":"stream","template":"log","options":{"max-mib":64}}]}`)
for pass := 0; pass < 2; pass++ {
if _, err := ix.JudgeAll(ctx, map[string]bool{moved: true}); err != nil {
t.Fatal(err)
}
}
if len(accepted) != 1 || accepted[0].Module != "moved" || accepted[0].Digest != moved || accepted[0].Judge != "judge-2" ||
len(accepted[0].BusState) != 2 || accepted[0].BusState[0].Name != "cache" || accepted[0].BusState[1].Template != "log" ||
string(accepted[0].BusState[0].Options["history"]) != "8" {
t.Fatalf("said accepted %+v", accepted)
}
// 1966/AC11: modules using state and none using logs but the foundation: logs counts 0, so a manifest of another
// module using logs is refused and the foundation's is accepted.
ix.OldSections = newMem()
hold("user", `{"module":"user","version":"1","state":["x"]}`)
hold("mesh-grants", `{"module":"mesh-grants","version":"1","logs":[{"name":"l"}]}`)
if _, err := ix.JudgeAll(ctx, nil); err != nil {
t.Fatal(err)
}
counts := func() map[string]int {
value, _, found, err := ix.OldSections.Get(ctx, OldSectionsKey)
if err != nil || !found {
t.Fatalf("no count of the old sections: %v", err)
}
c, err := DecodeOldSections(value)
if err != nil {
t.Fatal(err)
}
return c.Counts
}
if c := counts(); c["state"] != 1 || c["logs"] != 0 || c["streams"] != 0 || c["objects"] != 0 {
t.Fatalf("the old sections count %v, want state 1 and the rest 0", c)
}
late := hold("late", `{"module":"late","version":"1","logs":[{"name":"l"}]}`)
grants := hold("mesh-grants", `{"module":"mesh-grants","version":"2","logs":[{"name":"l"},{"name":"m"}]}`)
if _, err := ix.JudgeAll(ctx, map[string]bool{late: true, grants: true}); err != nil {
t.Fatal(err)
}
if v := verdict(late); !v.Refused() || !strings.Contains(v.Problems[0], "late uses the old section logs, which no other module uses any more") {
t.Fatalf("late: %+v", v)
}
if v := verdict(grants); v.Refused() {
t.Fatalf("the foundation's manifest was refused: %+v", v)
}
// 1966/AC11: the sole user of an old section changes its manifest and keeps the section: accepted, since it was accepted
// with it (the count is the accepted set held from the last pass, not recounted from fresh digests).
hold("solo", `{"module":"solo","version":"1","objects":[{"name":"o","max-mib":1}]}`)
if _, err := ix.JudgeAll(ctx, nil); err != nil {
t.Fatal(err)
}
solo2 := hold("solo", `{"module":"solo","version":"2","objects":[{"name":"o","max-mib":2}]}`)
if _, err := ix.JudgeAll(ctx, map[string]bool{solo2: true}); err != nil {
t.Fatal(err)
}
if v := verdict(solo2); v.Refused() {
t.Fatalf("the sole user of objects, changing its manifest and keeping objects, was refused: %+v", v)
}
// A count that cannot be read is never taken as 0: the zero rule is not applied in that pass and the log says so.
var logged []string
ix.Logf = func(f string, a ...any) { logged = append(logged, fmt.Sprintf(f, a...)) }
held := ix.OldSections
ix.OldSections = unreadable{held}
retry := hold("late", `{"module":"late","version":"2","logs":[{"name":"l"}]}`)
if _, err := ix.JudgeAll(ctx, map[string]bool{retry: true}); err != nil {
t.Fatal(err)
}
if v := verdict(retry); v.Refused() || !strings.Contains(strings.Join(logged, "\n"), "the zero rule is not applied") {
t.Fatalf("a count that could not be read: %+v, logged %q", v, logged)
}
ix.OldSections = held
// Clause 3 of the record on a release's Approve: a manifest with a field this judge does not know waits (no refusal, no
// event) while a newer build of the indexer is on the trunk, and is judged again at the next judge; once this program's
// build is the trunk's newest it is refused; a wait is written into the module's value as a refusal naming the fields.
ownFP := "src2:" + strings.Repeat("e", 64)
hold(delivery.IndexerModule, `{"module":"mesh-delivery-indexer","version":"1"}`)
ix.ownFP = ownFP // this program started when the trunk held another build of the indexer than the one it holds now
later := hold("later", `{"module":"later","version":"1","a-field-from-later":1,"another":{"x":1}}`)
said = nil
before := len(said)
j, err = ix.JudgeAll(ctx, map[string]bool{later: true})
if err != nil || len(j.Waiting) != 1 || j.Waiting[0] != "later" || len(said) != before {
t.Fatalf("a manifest with fields from later: %+v %v, said %d", j, err, len(said))
}
if v := verdict(later); !v.Waits() || v.Refused() || strings.Join(v.Unknown, ",") != "a-field-from-later,another" {
t.Fatalf("later: %+v", v)
}
if m := module("later"); m.Verdict != delivery.VerdictRefused || len(m.Unknown) != 2 || !m.VerdictOf("later").Waits() ||
!strings.Contains(m.Problems[0], "does not know yet") {
t.Fatalf("modules.later carries %+v", m)
}
// The build this program runs is asked of the engine, and the trunk moving before the start pass changes nothing: the
// trunk's value of the indexer names a newer build than the one the engine says runs, so the field still waits.
ix.ownFP = ""
ix.OwnBuild = func(context.Context) (string, error) { return ownFP, nil }
if _, err := ix.JudgeAtStart(ctx); err != nil || !verdict(later).Waits() {
t.Fatalf("the trunk moved before the start pass: %v %+v", err, verdict(later))
}
ix.OwnBuild = func(context.Context) (string, error) { return "", errors.New("the engine does not answer") }
ix.ownFP = ""
if _, err := ix.JudgeAtStart(ctx); err != nil || !verdict(later).Waits() {
t.Fatalf("an unknown build refused: %v %+v", err, verdict(later))
}
ix.ownFP = ownFP
// A renamed field is a known mistake: refused at once.
renamed := hold("renamed", `{"module":"renamed","version":"1","needs":{"x":"y"}}`)
if j, err := ix.JudgeAll(ctx, map[string]bool{renamed: true}); err != nil || len(j.Waiting) != 1 || !verdict(renamed).Refused() {
t.Fatalf("a renamed field: %+v %v", j, err)
}
// The trunk's newest build of the indexer is this program's: the field is one no judge to come knows.
ix.ownFP = "src2:" + strings.Repeat("a", 64)
if j, err := ix.JudgeAll(ctx, nil); err != nil || len(j.Waiting) != 0 || !verdict(later).Refused() || !strings.Contains(verdict(later).Problems[0], "a-field-from-later") {
t.Fatalf("the newest build refuses: %+v %v %+v", j, err, verdict(later))
}
if m := module("later"); m.Verdict != delivery.VerdictRefused || m.VerdictOf("later").Waits() || len(m.Unknown) != 0 {
t.Fatalf("modules.later after the refusal carries %+v", m)
}
// The tools: indexer_verdicts lists the verdicts that wait or are refused (and every one with all), indexer_judge
// judges now and says how the module's verdict stands.
keys, _ := ix.Verdicts.Keys(ctx)
get := func(key string) ([]byte, error) { v, _, _, err := ix.Verdicts.Get(ctx, key); return v, err }
if rows, unread := ListVerdicts(keys, get, "later", false); len(rows) != 1 || rows[0].State != delivery.VerdictRefused || len(unread) != 0 {
t.Fatalf("indexer_verdicts of later: %+v %v", rows, unread)
}
if rows, _ := ListVerdicts(keys, get, "", true); len(rows) <= len(keys)-3 || rows[0].Module > rows[len(rows)-1].Module {
t.Fatalf("indexer_verdicts, all: %d rows of %d verdicts", len(rows), len(keys))
}
// indexer_judge asks the engine for the running build again: a start pass that missed it is recovered (the trunk's value of
// the indexer is the newer one, so the field waits; asked and answered with that same build it is refused).
ix.ownFP = ""
ix.OwnBuild = func(context.Context) (string, error) { return "src2:" + strings.Repeat("a", 64), nil }
if got := ix.JudgeAgain(ctx, "later"); !got.Stored || !strings.Contains(got.Answer, "refused") || ix.ownFP == "" {
t.Fatalf("indexer_judge did not learn the running build: %+v", got)
}
ix.ownFP = ownFP
ix.OwnBuild = nil
if got := ix.JudgeAgain(ctx, "later"); !got.Stored || !strings.Contains(got.Answer, "waits") || !strings.Contains(got.Answer, "a-field-from-later") {
t.Fatalf("indexer_judge: %+v", got)
}
if rows, _ := ListVerdicts(keys, get, "later", false); len(rows) != 1 {
t.Fatalf("indexer_verdicts of later after: %+v", rows)
}
if got := ix.JudgeAgain(ctx, "nobody"); got.Stored || !strings.Contains(got.Answer, "holds no manifest") {
t.Fatalf("indexer_judge of nobody: %+v", got)
}
if key, value, err := NewJudgeAgain("later", "why", "me", time.Now()); err != nil || key != "judge.later" {
t.Fatalf("the ask: %q %v", key, err)
} else if a, err := DecodeTakeIn(value); err != nil || !a.Judge || a.Module != "later" {
t.Fatalf("the ask's value: %+v %v", a, err)
}
// Without the verdicts state the indexer judges nothing.
ix.Verdicts = nil
if j, err := ix.JudgeAll(ctx, nil); err != nil || j.Judged != 0 {
t.Fatalf("without the state: %+v %v", j, err)
}
}
// unreadable is a store whose reads fail.
type unreadable struct{ Store }
func (unreadable) Get(context.Context, string) ([]byte, uint64, bool, error) {
return nil, 0, false, errors.New("the bus did not answer")
}
@@ -0,0 +1,420 @@
package judge
import (
"bytes"
"encoding/json"
"fmt"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// The rules for bus state asked for in `requires` (issue 1966 D1; novox/hq ADR 0595 §1 to §3, ADR 0596 §2, ADR 0597 §2,
// design 25 § 12.1). mesh-sdk keeps only the entry's shape (ADR 0578 §7); every bound below is this judge's. An entry is
// judged once, here, and a refusal names the module, the entry and the field.
// optionKind is what an option's value is.
type optionKind int
const (
optionInt optionKind = iota
optionFlag
optionText
optionEvents
)
// optionRule is one option a template takes: its kind, its bound where it is a number, and whether it must be given.
type optionRule struct {
kind optionKind
least, most int
required bool
}
// busTemplate is a template: the storage type it is on, the options it takes, and what it fixes in plain words.
type busTemplate struct {
storage string
options map[string]optionRule
fixed string
}
// busTemplates are the six templates of design 25 § 12.1 and the bounds of its table.
var busTemplates = map[string]busTemplate{
catalogue.TemplateState: {
storage: catalogue.StorageKeyValue,
fixed: "what happens when full, the size of a value and of the bucket, and the copies",
options: map[string]optionRule{
catalogue.OptionPastValues: {kind: optionInt, least: 1, most: catalogue.StateMostHistory},
catalogue.OptionTimeToLive: {kind: optionInt, least: 1, most: 315360000},
catalogue.OptionLifetimePerKey: {kind: optionFlag},
catalogue.OptionPerMachine: {kind: optionFlag},
},
},
catalogue.TemplateEvents: {
storage: catalogue.StorageStream,
fixed: "what happens when full (the oldest is dropped) and the module's own subjects",
options: map[string]optionRule{
"events": {kind: optionEvents, required: true},
catalogue.OptionAgeDays: {kind: optionInt, least: catalogue.StreamLeastDays, most: catalogue.StreamMostDays},
catalogue.OptionCapMiB: {kind: optionInt, least: catalogue.StreamLeastMiB, most: catalogue.StreamMostMiB},
catalogue.OptionGrant: {kind: optionText},
},
},
catalogue.TemplateWorkQueue: {
storage: catalogue.StorageStream,
fixed: "what happens when full (a new message is refused), one taker, and no grant",
options: map[string]optionRule{
"events": {kind: optionEvents, required: true},
catalogue.OptionDeliveries: {kind: optionInt, least: catalogue.StreamLeastDeliveries, most: catalogue.StreamMostDeliveries},
catalogue.OptionDuplicates: {kind: optionInt, least: 1, most: catalogue.StreamMostDays * 86400},
catalogue.OptionAgeDays: {kind: optionInt, least: catalogue.StreamLeastDays, most: catalogue.StreamMostDays},
catalogue.OptionCapMiB: {kind: optionInt, least: catalogue.StreamLeastMiB, most: catalogue.StreamMostMiB},
},
},
catalogue.TemplateLog: {
storage: catalogue.StorageStream,
fixed: "what happens when full (a new entry is refused), that an entry is never rewritten, and its size",
options: map[string]optionRule{
catalogue.OptionCapMiB: {kind: optionInt, least: catalogue.StreamLeastMiB, most: catalogue.StreamMostMiB},
catalogue.OptionAgeDays: {kind: optionInt, least: 1, most: 3650},
},
},
catalogue.TemplateObjects: {
storage: catalogue.StorageObjectStore,
fixed: "what happens when full (a new object is refused) and that objects are kept for good",
options: map[string]optionRule{
catalogue.OptionCapMiB: {kind: optionInt, least: catalogue.ObjectsLeastMiB, most: catalogue.ObjectsMostMiB, required: true},
catalogue.OptionGrant: {kind: optionText},
},
},
catalogue.TemplateLedger: {
storage: catalogue.StorageLedger,
fixed: "what happens when full (a new entry is refused) and its state and index buckets",
options: map[string]optionRule{
catalogue.OptionCapMiB: {kind: optionInt, least: catalogue.StreamLeastMiB, most: catalogue.StreamMostMiB},
},
},
}
// bareStream is a stream of a module's own configuration, where no template fits: the three settings the judge
// refuses one for missing (design 25 § 12.1), and those it may also give.
var bareStream = map[string]optionRule{
catalogue.OptionCapMiB: {kind: optionInt, least: catalogue.StreamLeastMiB, most: catalogue.StreamMostMiB, required: true},
catalogue.OptionAgeDays: {kind: optionInt, least: 1, most: 3650, required: true},
catalogue.OptionGrant: {kind: optionText},
"events": {kind: optionEvents},
}
// ledgerConfiguration is what a ledger's own configuration may give: how long its history is kept (design 25 § 12.1).
var ledgerConfiguration = map[string]optionRule{
catalogue.OptionAgeDays: {kind: optionInt, least: 1, most: 3650},
}
// noAge is how a bare stream says it keeps its messages for good.
const noAge = "none"
// ledgerEffects are the four effects an operation may have (ADR 0597 §2).
var ledgerEffects = []string{catalogue.EffectSetFields, catalogue.EffectSetEntry, catalogue.EffectAppend, catalogue.EffectRemove}
// BusStateProblems is what is wrong with one manifest's bus state, judged on the manifest alone (issue 1966 D1).
func BusStateProblems(m catalogue.Manifest) []string {
if len(m.BusState) == 0 {
return nil
}
var problems []string
if !stateName.MatchString(m.Module) {
problems = append(problems, fmt.Sprintf(
"%s keeps bus state, and a module's name is part of its names on the bus, which take one plain name — "+
"no dot (novox/hq ADR 0201, ADR 0595 §1)", m.Module))
}
storageOf := map[string]string{}
oldSection := map[string]string{} // a local name an old section keeps, to that section (issue 1966 D2)
for _, st := range m.State {
oldSection[st.Name] = "state"
}
for _, l := range m.Logs {
oldSection[l.Name] = "logs"
}
for _, st := range m.Streams {
oldSection[st.Name] = "streams"
}
for _, o := range m.Objects {
oldSection[o.Name] = "objects"
}
terminal := catalogue.TerminalKeys(m)
for _, e := range m.BusState {
at := fmt.Sprintf("%s's bus state %q", m.Module, e.Name)
switch {
case strings.Contains(e.Name, "_"):
problems = append(problems, fmt.Sprintf(
"%s: its name holds an underscore, which separates the module from the local name on the bus; use a "+
"hyphen (novox/hq design 25 § 12.1)", at))
case !stateName.MatchString(e.Name):
problems = append(problems, fmt.Sprintf(
"%s: its name is a local name — lower-case letters, digits and hyphens, no dot and no underscore; the "+
"mesh derives the name on the bus (novox/hq ADR 0201)", at))
}
if first, taken := storageOf[e.Name]; taken {
problems = append(problems, fmt.Sprintf(
"%s: the name is used twice (as %s and as %s); each name is one, whatever its storage type (novox/hq "+
"ADR 0595 §1)", at, first, e.Storage))
}
storageOf[e.Name] = e.Storage
if section, kept := oldSection[e.Name]; kept {
problems = append(problems, fmt.Sprintf(
"%s: the name is also kept in the old section %s; one local name is one form, so a module that moves an "+
"entry removes it from the old section (novox/hq ADR 0595 §11)", at, section))
}
problems = append(problems, busEntryProblems(at, e, terminal)...)
}
return problems
}
// busEntryProblems judges one entry's storage type, template, options and configuration.
func busEntryProblems(at string, e catalogue.BusStateEntry, terminal []string) []string {
var problems []string
switch e.Storage {
case catalogue.StorageStream, catalogue.StorageKeyValue, catalogue.StorageObjectStore, catalogue.StorageLedger:
default:
return []string{fmt.Sprintf("%s: the storage type %q is unknown; it is %s", at, e.Storage,
strings.Join([]string{catalogue.StorageStream, catalogue.StorageKeyValue, catalogue.StorageObjectStore, catalogue.StorageLedger}, ", "))}
}
if e.Storage != catalogue.StorageLedger && len(e.Types) > 0 {
problems = append(problems, fmt.Sprintf("%s: types are a ledger's; storage type %s has none", at, e.Storage))
}
if e.Template == "" {
if e.Storage != catalogue.StorageStream {
return append(problems, fmt.Sprintf(
"%s: the storage type %s is asked for by its template; only a stream is asked for bare, with a "+
"configuration of its own (novox/hq design 25 § 12.1)", at, e.Storage))
}
if len(e.Options) > 0 {
problems = append(problems, fmt.Sprintf(
"%s: a bare stream gives its settings in its configuration, not in options", at))
}
return append(problems, settingsProblems(at, "configuration", e.Configuration, bareStream, "a bare stream", terminal, true)...)
}
tpl, known := busTemplates[e.Template]
switch {
case !known:
names := make([]string, 0, len(busTemplates))
for n := range busTemplates {
names = append(names, n)
}
slices.Sort(names)
return append(problems, fmt.Sprintf("%s: the template %q is unknown; it is %s", at, e.Template, strings.Join(names, ", ")))
case tpl.storage != e.Storage:
return append(problems, fmt.Sprintf(
"%s: the template %s is on the storage type %s, not %s", at, e.Template, tpl.storage, e.Storage))
}
problems = append(problems, settingsProblems(at, "options", e.Options, tpl.options, "the template "+e.Template+" (it fixes "+tpl.fixed+")", terminal, false)...)
if e.Template == catalogue.TemplateLedger {
problems = append(problems, settingsProblems(at, "configuration", e.Configuration, ledgerConfiguration, "a ledger's own configuration", terminal, false)...)
problems = append(problems, ledgerTypeProblems(at, e)...)
} else if len(e.Configuration) > 0 {
problems = append(problems, fmt.Sprintf(
"%s: the template %s is given a configuration; a configuration of its own is for a bare stream or a "+
"ledger whose history expires, and a template's settings are its options (novox/hq ADR 0595 §2)", at, e.Template))
}
if e.Template == catalogue.TemplateWorkQueue {
problems = append(problems, workQueueProblems(at, e)...)
}
return problems
}
// settingsProblems judges the given settings against the rules: an unknown one is refused (a fixed setting among them),
// a number outside its bound, a kind mismatch, and a required one missing. A bare stream's when-full and age are read
// here too.
func settingsProblems(at, field string, given map[string]json.RawMessage, rules map[string]optionRule, of string, terminal []string, bare bool) []string {
var problems []string
for _, k := range sortedKeys(given) {
raw := given[k]
if bare && k == catalogue.ConfigWhenFull {
var v string
if json.Unmarshal(raw, &v) != nil || (v != catalogue.WhenFullRefuse && v != catalogue.WhenFullDropOldest) {
problems = append(problems, fmt.Sprintf(
"%s: %s %q is %s; a bare stream says %q or %q", at, field, k, string(raw), catalogue.WhenFullRefuse, catalogue.WhenFullDropOldest))
}
continue
}
rule, ok := rules[k]
if !ok {
if k == catalogue.ConfigWhenFull {
problems = append(problems, fmt.Sprintf(
"%s: %s %q is given to %s; what happens when full is fixed and is never an option (novox/hq ADR 0595 §3)", at, field, k, of))
continue
}
problems = append(problems, fmt.Sprintf(
"%s: %s %q is not a setting of %s (novox/hq ADR 0595 §3)", at, field, k, of))
continue
}
problems = append(problems, valueProblems(at, field, k, raw, rule, terminal, bare)...)
}
for _, k := range sortedKeys(rules) {
if _, given := given[k]; !given && rules[k].required {
problems = append(problems, fmt.Sprintf("%s: %s %q is missing; %s gives it", at, field, k, of))
}
}
if bare {
if _, given := given[catalogue.ConfigWhenFull]; !given {
problems = append(problems, fmt.Sprintf(
"%s: %s %q is missing; a bare stream says %q or %q", at, field, catalogue.ConfigWhenFull, catalogue.WhenFullRefuse, catalogue.WhenFullDropOldest))
}
}
return problems
}
// valueProblems judges one setting's value against its rule.
func valueProblems(at, field, key string, raw json.RawMessage, rule optionRule, terminal []string, bare bool) []string {
switch rule.kind {
case optionInt:
if bare && key == catalogue.OptionAgeDays {
var word string
if json.Unmarshal(raw, &word) == nil && word == noAge {
return nil
}
}
n, ok := wholeNumber(raw)
if !ok {
return []string{fmt.Sprintf("%s: %s %q is %s, which is not a whole number", at, field, key, string(raw))}
}
if n < rule.least || n > rule.most {
extra := ""
if key == catalogue.OptionPastValues && n == 0 {
extra = "; 0 is refused here"
}
return []string{fmt.Sprintf("%s: %s %q is %d, outside its bound of %d to %d%s", at, field, key, n, rule.least, rule.most, extra)}
}
case optionFlag:
var b bool
if json.Unmarshal(raw, &b) != nil {
return []string{fmt.Sprintf("%s: %s %q is %s, which is not true or false", at, field, key, string(raw))}
}
case optionText:
var config string
if json.Unmarshal(raw, &config) != nil || !slices.Contains(terminal, config) {
return []string{fmt.Sprintf(
"%s: %s %q names %s, which is not a trusted config value of the module, set only at the controller's "+
"terminal or on the operator's Approve (novox/hq ADR 0348 §8)", at, field, key, string(raw))}
}
case optionEvents:
var events []string
if json.Unmarshal(raw, &events) != nil || len(events) == 0 {
return []string{fmt.Sprintf("%s: %s %q is %s; it is a list of at least one event", at, field, key, string(raw))}
}
var out []string
for _, ev := range events {
if err := eventPattern(ev, false); err != nil {
out = append(out, fmt.Sprintf("%s: %s %q carries %q: %v", at, field, key, ev, err))
}
}
return out
}
return nil
}
// wholeNumber reads a JSON number that is a whole number.
func wholeNumber(raw json.RawMessage) (int, bool) {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var n json.Number
if dec.Decode(&n) != nil {
return 0, false
}
i, err := n.Int64()
return int(i), err == nil
}
// workQueueProblems holds a work queue's duplicate window to its age (design 25 § 12.1: up to its age).
func workQueueProblems(at string, e catalogue.BusStateEntry) []string {
raw, given := e.Options[catalogue.OptionDuplicates]
if !given {
return nil
}
window, ok := wholeNumber(raw)
if !ok {
return nil // valueProblems said so
}
days := catalogue.StreamDefaultDays
if age, ok := wholeNumber(e.Options[catalogue.OptionAgeDays]); ok && age > 0 {
days = age
}
if window > days*86400 {
return []string{fmt.Sprintf(
"%s: options %q is %d seconds, longer than the queue's age of %d days (novox/hq design 25 § 12.1)",
at, catalogue.OptionDuplicates, window, days)}
}
return nil
}
// ledgerTypeProblems judges a ledger's types: named locally and once, and every operation's effect one of the four.
func ledgerTypeProblems(at string, e catalogue.BusStateEntry) []string {
var problems []string
if len(e.Types) == 0 {
problems = append(problems, fmt.Sprintf("%s: a ledger names at least one type (novox/hq ADR 0597 §1)", at))
}
seen := map[string]bool{}
for _, t := range e.Types {
where := fmt.Sprintf("%s: type %q", at, t.Name)
switch {
case !stateName.MatchString(t.Name):
problems = append(problems, fmt.Sprintf("%s: its name is lower-case letters, digits and hyphens", where))
case seen[t.Name]:
problems = append(problems, fmt.Sprintf("%s: the type is named twice", where))
}
seen[t.Name] = true
operations := map[string]bool{}
for _, op := range t.Operations {
if operations[op.Name] {
problems = append(problems, fmt.Sprintf("%s: the operation %q is named twice", where, op.Name))
}
operations[op.Name] = true
if !slices.Contains(ledgerEffects, op.Effect) {
problems = append(problems, fmt.Sprintf(
"%s: the operation %q has the effect %q, outside the four (%s) (novox/hq ADR 0597 §2)",
where, op.Name, op.Effect, strings.Join(ledgerEffects, ", ")))
}
}
}
return problems
}
// BusStateCatalogueProblems are the rules of bus state no single manifest can be judged against: two modules' names on
// the bus alike, and a `reads` naming bus state in the new form, which has no readers (novox/hq ADR 0596 §2).
func BusStateCatalogueProblems(shelf catalogue.Shelf) []string {
var problems []string
owner := map[string]string{} // a name on the bus, upper-cased, to the module asking for it
for _, module := range shelfOrder(shelf) {
for _, o := range shelf[module].Objects {
owner[strings.ToUpper(o.Bucket(module))] = module
}
}
for _, module := range shelfOrder(shelf) {
for _, e := range shelf[module].BusState {
name := catalogue.ObjectsBucket(module, e.Name)
if first, taken := owner[strings.ToUpper(name)]; taken && first != module {
problems = append(problems, fmt.Sprintf(
"%s's bus state %q has the name %s on the bus, which %s's already has or is alike; a name on the bus "+
"has one owner (novox/hq ADR 0201 §4)", module, e.Name, name, first))
continue
}
owner[strings.ToUpper(name)] = module
}
}
for _, module := range shelfOrder(shelf) {
for _, r := range shelf[module].Reads {
from, local, err := catalogue.ReadState(r)
if err != nil || from == module {
continue // StateProblems says why
}
for _, e := range shelf[from].BusState {
if e.Name == local {
problems = append(problems, fmt.Sprintf(
"%s reads %q, which %s has moved to bus state in `requires`; the new form has no readers: ask %s's "+
"tools, or hear its events (novox/hq ADR 0596 §2)", module, r, from, from))
}
}
}
}
return problems
}
@@ -0,0 +1,95 @@
package judge
import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// A manifest's bus state, asked for in `requires` by template, is judged once and each refusal names the module, the entry
// and the field (issue 1966 D1, D2; novox/hq design 25 § 12.1, ADR 0595, ADR 0596 §2, ADR 0597 §2).
func TestBusStateIsJudgedOnce(t *testing.T) {
const ledger = `{"name":"records","storage":"ledger","template":"ledger","options":{"max-mib":512},"types":[` +
`{"name":"decision","schema":{"title":{"type":"text"}},"operations":[{"name":"propose","effect":"set-fields","fields":["title"]}]}]}`
manifest := func(entries string) string {
return `{"module":"a","version":"1","requires":[` + entries + `]}`
}
for _, c := range []struct {
name string
raw string
refused []string // words the first problem holds, in order: absent is accepted
}{
// 1966/AC8: a state, a log, objects and a ledger within bounds
{"within bounds", manifest(`{"name":"cache","storage":"key-value","template":"state","options":{"history":8,"key-ttl":true}},` +
`{"name":"audit","storage":"stream","template":"log","options":{"max-mib":64,"max-age-days":365}},` +
`{"name":"files","storage":"object-store","template":"objects","options":{"max-mib":2048}},` + ledger), nil},
// 1966/AC7: each refused case, naming the module, the entry and the field
{"unknown storage type", manifest(`{"name":"x","storage":"tape"}`), []string{"a's bus state \"x\"", "storage type \"tape\""}},
{"unknown template", manifest(`{"name":"x","storage":"stream","template":"pile"}`), []string{"\"x\"", "template \"pile\""}},
{"template on another storage type", manifest(`{"name":"x","storage":"stream","template":"state"}`), []string{"\"x\"", "template state is on the storage type key-value"}},
{"past values 0", manifest(`{"name":"x","storage":"key-value","template":"state","options":{"history":0}}`), []string{"\"x\"", "options \"history\" is 0", "0 is refused"}},
{"past values 65", manifest(`{"name":"x","storage":"key-value","template":"state","options":{"history":65}}`), []string{"\"x\"", "\"history\" is 65", "1 to 64"}},
{"a lifetime that is no number", manifest(`{"name":"x","storage":"key-value","template":"state","options":{"ttl-seconds":"soon"}}`), []string{"\"x\"", "\"ttl-seconds\""}},
{"events age over 30 days", manifest(`{"name":"x","storage":"stream","template":"events","options":{"events":["seen"],"max-age-days":31}}`), []string{"\"x\"", "\"max-age-days\" is 31", "1 to 30"}},
{"events cap over 8192", manifest(`{"name":"x","storage":"stream","template":"events","options":{"events":["seen"],"max-mib":8193}}`), []string{"\"x\"", "\"max-mib\" is 8193"}},
{"events naming none", manifest(`{"name":"x","storage":"stream","template":"events","options":{}}`), []string{"\"x\"", "\"events\" is missing"}},
{"a grant that is no trusted value", manifest(`{"name":"x","storage":"stream","template":"events","options":{"events":["seen"],"grant":"nobody"}}`), []string{"\"x\"", "\"grant\"", "trusted config value"}},
{"deliveries 11", manifest(`{"name":"x","storage":"stream","template":"work-queue","options":{"events":["job"],"max-deliveries":11}}`), []string{"\"x\"", "\"max-deliveries\" is 11", "1 to 10"}},
{"duplicates past the age", manifest(`{"name":"x","storage":"stream","template":"work-queue","options":{"events":["job"],"max-age-days":1,"duplicates":86401}}`), []string{"\"x\"", "\"duplicates\"", "longer than the queue's age of 1 days"}},
{"log age 3651", manifest(`{"name":"x","storage":"stream","template":"log","options":{"max-age-days":3651}}`), []string{"\"x\"", "\"max-age-days\" is 3651"}},
{"objects without a cap", manifest(`{"name":"x","storage":"object-store","template":"objects"}`), []string{"\"x\"", "\"max-mib\" is missing"}},
{"when full given as an option", manifest(`{"name":"x","storage":"stream","template":"log","options":{"when-full":"drop-oldest"}}`), []string{"\"x\"", "\"when-full\"", "fixed"}},
{"another fixed setting given", manifest(`{"name":"x","storage":"key-value","template":"state","options":{"max-value-kib":1024}}`), []string{"\"x\"", "\"max-value-kib\" is not a setting of the template state"}},
{"a configuration beside a template", manifest(`{"name":"x","storage":"key-value","template":"state","configuration":{"max-mib":1}}`), []string{"\"x\"", "configuration"}},
{"a name used twice across storage types", manifest(
`{"name":"x","storage":"key-value","template":"state"},{"name":"x","storage":"stream","template":"log"}`), []string{"\"x\"", "used twice (as key-value and as stream)"}},
{"a local name holding an underscore", manifest(`{"name":"my_x","storage":"key-value","template":"state"}`), []string{"\"my_x\"", "underscore"}},
{"a bare stream without when full", manifest(`{"name":"x","storage":"stream","configuration":{"max-mib":10,"max-age-days":"none"}}`), []string{"\"x\"", "\"when-full\" is missing"}},
{"a bare stream without a cap", manifest(`{"name":"x","storage":"stream","configuration":{"when-full":"refuse","max-age-days":3}}`), []string{"\"x\"", "\"max-mib\" is missing"}},
{"a bare stream without an age", manifest(`{"name":"x","storage":"stream","configuration":{"when-full":"refuse","max-mib":10}}`), []string{"\"x\"", "\"max-age-days\" is missing"}},
{"a bare stream, complete", manifest(`{"name":"x","storage":"stream","configuration":{"when-full":"drop-oldest","max-mib":10,"max-age-days":"none"}}`), nil},
{"a bare stream with a wrong when full", manifest(`{"name":"x","storage":"stream","configuration":{"when-full":"crash","max-mib":10,"max-age-days":9}}`), []string{"\"x\"", "\"when-full\"", "\"refuse\" or \"drop-oldest\""}},
{"a ledger effect outside the four", manifest(strings.Replace(ledger, `"set-fields"`, `"upsert"`, 1)), []string{"\"records\"", "effect \"upsert\"", "outside the four"}},
{"a ledger whose history expires", manifest(strings.Replace(ledger, `"options"`, `"configuration":{"max-age-days":30},"options"`, 1)), nil},
} {
_, problems := Manifest([]byte(c.raw), nil)
switch {
case c.refused == nil && len(problems) > 0:
t.Errorf("%s: refused: %v", c.name, problems)
case c.refused != nil && len(problems) == 0:
t.Errorf("%s: accepted", c.name)
case c.refused != nil:
text := strings.Join(problems, "\n")
at := 0
for _, word := range c.refused {
i := strings.Index(text[at:], word)
if i < 0 {
t.Errorf("%s: problems %q lack %q", c.name, text, word)
break
}
at += i
}
}
}
// Between the manifests: two modules' names on the bus alike, and a read of bus state that moved (ADR 0596 §2).
entry := func(name string) []catalogue.BusStateEntry {
return []catalogue.BusStateEntry{{Name: name, Storage: catalogue.StorageKeyValue, Template: catalogue.TemplateState}}
}
for _, c := range []struct {
name string
shelf catalogue.Shelf
want string
}{
{"names alike", catalogue.Shelf{"a_b": {Module: "a_b", BusState: entry("c")}, "a": {Module: "a", BusState: entry("b_c")}}, "has one owner"},
{"a read of moved state", catalogue.Shelf{"owner": {Module: "owner", BusState: entry("cache")},
"reader": {Module: "reader", Reads: []string{"owner.cache"}}}, "reader reads \"owner.cache\", which owner has moved"},
{"a read of state still kept", catalogue.Shelf{"owner": {Module: "owner", State: []catalogue.StateDeclaration{{Name: "cache"}}},
"reader": {Module: "reader", Reads: []string{"owner.cache"}}}, ""},
} {
got := strings.Join(BusStateCatalogueProblems(c.shelf), "\n")
if (c.want == "") != (got == "") || !strings.Contains(got, c.want) {
t.Errorf("%s: between the manifests: %q, want %q", c.name, got, c.want)
}
}
}
+65 -3
View File
@@ -69,9 +69,33 @@ func problemsOf(err error) []string {
// whose removal from the catalogue ends it (a rule two modules break together refuses both); one no removal ends is in
// unnamed, for the caller to say. A module not in the answer is accepted.
func Shelf(raws map[string][]byte, predefined catalogue.PredefinedList, longestMachine int) (refused map[string][]string, unnamed []string) {
refused = map[string][]string{}
return ShelfCounting(raws, predefined, longestMachine, nil)
}
// ShelfCounting is Shelf also refusing a manifest of a module outside the foundation that uses an old section whose count
// is 0 (OldSectionProblems; issue 1966 D4); nil counts refuse none. A field this judge does not know is refused, strictly, as
// the checker judges a change: the judge that runs there is the one built at the change's head, which knows the fields the
// change brings, so a field it does not know is a mistake.
func ShelfCounting(raws map[string][]byte, predefined catalogue.PredefinedList, longestMachine int, counts OldSectionCounts) (refused map[string][]string, unnamed []string) {
refused, waits, unnamed := ShelfWaiting(raws, predefined, longestMachine, counts)
for name := range waits {
_, refused[name] = Manifest(raws[name], predefined)
}
return refused, unnamed
}
// ShelfWaiting is ShelfCounting where a manifest that carries a field this judge does not know (UnknownFields) is not
// refused: it is in waits, by module, naming each field, and judged no further, because a newer judge may know the field
// (novox/hq ADR 0578 §1, amended). The indexer judges so; the checker, which runs the judge built at the head of the change,
// judges with Shelf.
func ShelfWaiting(raws map[string][]byte, predefined catalogue.PredefinedList, longestMachine int, counts OldSectionCounts) (refused, waits map[string][]string, unnamed []string) {
refused, waits = map[string][]string{}, map[string][]string{}
shelf := catalogue.Shelf{}
for name, raw := range raws {
if unknown := UnknownFields(raw); len(unknown) > 0 {
waits[name] = unknown
continue
}
m, problems := Manifest(raw, predefined)
if len(problems) > 0 {
refused[name] = problems
@@ -81,6 +105,10 @@ func Shelf(raws map[string][]byte, predefined catalogue.PredefinedList, longestM
refused[name] = []string{fmt.Sprintf("the manifest is of module %q, not %q", m.Module, name)}
continue
}
if problems := OldSectionProblems(m, counts); len(problems) > 0 {
refused[name] = problems
continue
}
shelf[name] = m
}
between := Between(shelf, longestMachine)
@@ -122,7 +150,7 @@ func Shelf(raws map[string][]byte, predefined catalogue.PredefinedList, longestM
for name := range refused {
sort.Strings(refused[name])
}
return refused, unnamed
return refused, waits, unnamed
}
// Between is every problem between the manifests of shelf, sorted: catalogue.CatalogueProblems, IdentityProblems over
@@ -179,6 +207,7 @@ const Unnamed = "(between manifests)"
func Request(r composer.Request) (composer.Verdict, error) {
v := composer.Verdict{V: composer.Version, SDK: Identity()}
seen := map[string]bool{}
counts := trunkOldSections(r)
for _, c := range r.Cases {
if seen[c.Name] {
return composer.Verdict{}, fmt.Errorf("the case %q is named twice", c.Name)
@@ -191,7 +220,7 @@ func Request(r composer.Request) (composer.Verdict, error) {
}
raws[name] = m.Raw
}
refused, unnamed := Shelf(raws, r.Predefined, 0)
refused, unnamed := ShelfCounting(raws, r.Predefined, 0, counts)
cv := composer.CaseVerdict{Name: c.Name, Manifests: map[string]composer.ManifestRead{}}
for name := range raws {
if problems := refused[name]; len(problems) > 0 {
@@ -208,6 +237,39 @@ func Request(r composer.Request) (composer.Verdict, error) {
return v, nil
}
// TrunkCase is the case of a request holding the catalogue as the trunk holds it (the checker's JudgeTrunk).
const TrunkCase = "trunk"
// trunkOldSections counts the old sections the modules accepted at the trunk use, as the indexer counts them from its
// verdicts (issue 1966 D4, ADR 0595 §11): a change is refused at the merge check exactly as at take-in (judged once, ADR
// 0578). Nil where the request holds no trunk case, which refuses no use of a section.
func trunkOldSections(r composer.Request) OldSectionCounts {
for _, c := range r.Cases {
if c.Name != TrunkCase {
continue
}
raws := map[string][]byte{}
for name, m := range c.Manifests {
if len(m.Raw) > 0 {
raws[name] = m.Raw
}
}
refused, _ := Shelf(raws, r.Predefined, 0)
var accepted []catalogue.Manifest
for name, raw := range raws {
if len(refused[name]) > 0 {
continue
}
if m, err := catalogue.Decode(raw); err == nil {
m.Module = name
accepted = append(accepted, m)
}
}
return CountOldSections(accepted)
}
return nil
}
var (
identityOnce sync.Once
identity string
@@ -54,6 +54,21 @@ func TestTheJudge(t *testing.T) {
}
}
}
// A field the judge does not know waits in the indexer's judge and stays a refusal in the checker's (clause 3 of the
// record on a release's Approve); a renamed field and any other problem are refused by both.
waitRaws := map[string][]byte{
"later": []byte(`{"module":"later","version":"1","later-field":true,"other":1}`),
"renamed": []byte(`{"module":"renamed","version":"1","needs":{"x":"y"}}`),
"bad": []byte(`{"module":"bad","version":"1","provides":[{"name":"amqp"}]}`),
"ok": []byte(`{"module":"ok","version":"1"}`),
}
refused, waits, _ := ShelfWaiting(waitRaws, nil, 0, nil)
if !reflect.DeepEqual(waits, map[string][]string{"later": {"later-field", "other"}}) || len(refused) != 2 || refused["renamed"] == nil || refused["bad"] == nil {
t.Errorf("ShelfWaiting: waits %v, refused %v", waits, refused)
}
if strict, _ := Shelf(waitRaws, nil, 0); len(strict) != 3 || !strings.Contains(strings.Join(strict["later"], ""), "later-field") {
t.Errorf("Shelf refuses as before: %v", strict)
}
// The private network's manifest, which the mesh provides itself, is accepted (moved from mesh-sdk overlay's test).
if raw, err := json.Marshal(overlay.Manifest()); err != nil {
t.Fatal(err)
@@ -86,6 +101,27 @@ func TestTheJudge(t *testing.T) {
v.Cases[0].Manifests["b"].Accepted || !strings.Contains(v.Cases[0].Manifests["b"].Refused, `requires "amqp"`) {
t.Fatalf("request: %+v %v", v, err)
}
// 1966/AC11: the checker's judge counts the old sections from the trunk case as the indexer does from its verdicts (the
// foundation left out): a change adding logs where no trunk module uses them is refused, the foundation's accepted, and
// state, which a trunk module uses, is accepted.
old := composer.Request{V: composer.Version, Cases: []composer.Case{
{Name: "change", Manifests: map[string]composer.Manifest{
"user": {Raw: json.RawMessage(`{"module":"user","version":"1","state":["x"]}`)},
"late": {Raw: json.RawMessage(`{"module":"late","version":"1","logs":[{"name":"l"}]}`)},
"newer": {Raw: json.RawMessage(`{"module":"newer","version":"1","state":["y"]}`)},
"mesh-grants": {Raw: json.RawMessage(`{"module":"mesh-grants","version":"1","logs":[{"name":"l"}]}`)}}},
{Name: "trunk", Manifests: map[string]composer.Manifest{
"user": {Raw: json.RawMessage(`{"module":"user","version":"1","state":["x"]}`)},
"mesh-grants": {Raw: json.RawMessage(`{"module":"mesh-grants","version":"1","logs":[{"name":"l"}]}`)}}}}}
ov, err := Request(old)
if err != nil {
t.Fatal(err)
}
got := ov.Cases[0].Manifests
if got["late"].Accepted || !strings.Contains(got["late"].Refused, "late uses the old section logs") ||
!got["newer"].Accepted || !got["user"].Accepted || !got["mesh-grants"].Accepted {
t.Errorf("the change's old sections: %+v", got)
}
r.Cases[0].Manifests["c"] = composer.Manifest{Digest: strings.Repeat("a", 64)}
if _, err := Request(r); err == nil {
t.Fatal("a manifest left as a digest was judged")
@@ -0,0 +1,69 @@
package judge
import (
"fmt"
"slices"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// The old sections end when no module is left in them (issue 1966 D4; novox/hq ADR 0595 §11): the indexer counts, per old
// section, the accepted modules using it, the foundation left out, and the judge refuses a manifest of another module
// that uses a section whose count is 0. The foundation (the bus module, the grants part and the indexer itself) is
// created before the bus module can create anything, so it keeps the old sections for good and is always accepted.
// OldSections are the sections of a manifest that keep bus state in the old forms.
var OldSections = []string{"state", "logs", "streams", "objects"}
// Foundation are the modules that keep using the old sections for good.
var Foundation = []string{"nats", "mesh-grants", "mesh-delivery-indexer"}
// OldSectionCounts is, per old section, how many accepted modules outside the foundation use it. Nil is no count: the
// judge then refuses no use of a section.
type OldSectionCounts map[string]int
// SectionsUsed are the old sections a manifest uses, in OldSections' order.
func SectionsUsed(m catalogue.Manifest) []string {
var used []string
for _, s := range OldSections {
if (s == "state" && len(m.State) > 0) || (s == "logs" && len(m.Logs) > 0) ||
(s == "streams" && len(m.Streams) > 0) || (s == "objects" && len(m.Objects) > 0) {
used = append(used, s)
}
}
return used
}
// CountOldSections counts, per old section, the manifests using it, the foundation's left out; every section is in the
// answer, with 0 where none uses it.
func CountOldSections(manifests []catalogue.Manifest) OldSectionCounts {
counts := OldSectionCounts{}
for _, s := range OldSections {
counts[s] = 0
}
for _, m := range manifests {
if slices.Contains(Foundation, m.Module) {
continue
}
for _, s := range SectionsUsed(m) {
counts[s]++
}
}
return counts
}
// OldSectionProblems are the old sections a manifest uses that no accepted module outside the foundation uses any more.
func OldSectionProblems(m catalogue.Manifest, counts OldSectionCounts) []string {
if counts == nil || slices.Contains(Foundation, m.Module) {
return nil
}
var problems []string
for _, s := range SectionsUsed(m) {
if counts[s] == 0 {
problems = append(problems, fmt.Sprintf(
"%s uses the old section %s, which no other module uses any more (0 modules count); ask for it in "+
"`requires` by template (novox/hq ADR 0595 §11)", m.Module, s))
}
}
return problems
}
+88 -8
View File
@@ -136,7 +136,7 @@ func spokenSystems() string {
// and was simply absent — seven modules' tools went missing that way on 2026-10-03. Refused here,
// naming the field that would deliver it.
func undeliveredBundles(m catalogue.Manifest) []string {
if m.Build == nil || m.Module == catalogue.RuntimeModule {
if m.Build == nil || catalogue.IsToolRunner(m) {
return nil
}
named := runByAResource(m)
@@ -2388,6 +2388,44 @@ func ownSecretPromptProblems(m catalogue.Manifest, name, prompt string, family b
return problems
}
// unknownFieldText is how the lenient decoder words the first field its shape does not have, and readWithout the list of
// every field it read the manifest without (catalogue.Manifest.SetAside).
var (
unknownFieldText = regexp.MustCompile(`json: unknown field "([^"]+)"`)
readWithout = regexp.MustCompile(`\(read without ([^)]*)\)`)
)
// UnknownFields are the fields of a manifest the judge does not know, as the lenient decoder sets them aside (catalogue.Decode,
// each once, the path it names them with), or none: a manifest this judge reads whole has none, and a field that used to
// mean something (renamed) is a known mistake, not a field a later judge may know, so a manifest with one has none
// either.
func UnknownFields(raw []byte) []string {
m, err := catalogue.Decode(raw)
aside := m.SetAside()
if err != nil || aside == "" {
return nil
}
first := unknownFieldText.FindStringSubmatch(aside)
if first == nil {
return nil // set aside for another reason: the judge's refusal as it is
}
fields := []string{first[1]}
if list := readWithout.FindStringSubmatch(aside); list != nil {
fields = fields[:0]
for _, f := range strings.Split(list[1], ", ") {
if f = strings.TrimSpace(f); f != "" && !slices.Contains(fields, f) {
fields = append(fields, f)
}
}
}
for _, f := range fields {
if _, was := renamed[f[strings.LastIndex(f, ".")+1:]]; was {
return nil
}
}
return fields
}
func parseManifest(raw []byte, predefined catalogue.PredefinedList) (catalogue.Manifest, error) {
var m catalogue.Manifest
// Strictly. **An unknown key is refused**, which is the discipline the host's declaration
@@ -2626,6 +2664,8 @@ func parseManifest(raw []byte, predefined catalogue.PredefinedList) (catalogue.M
// And the object buckets it owns (novox/hq ADR 0592 §5, §9).
problems = append(problems, ObjectsProblems(m)...)
problems = append(problems, WorkQueueStreamProblems(m)...)
// And the bus state it asks for in `requires`, by template (busstate.go, novox/hq ADR 0595; issue 1966 D1).
problems = append(problems, BusStateProblems(m)...)
// And the defaults it gives its settings (setting_defaults.go, novox/hq ADR 0262), or, written in the
// config form, the whole declaration of each value (config.go, novox/hq ADR 0348).
if m.DeclaresConfig() {
@@ -3266,9 +3306,9 @@ func waitProblems(m catalogue.Manifest) []string {
if len(m.Waits) == 0 {
return nil
}
if m.Module != catalogue.RuntimeModule {
return []string{fmt.Sprintf("%s says what a session waits on; only %s, the tool runner's module, "+
"declares waits (novox/hq ADR 0367 §2)", m.Module, catalogue.RuntimeModule)}
if !catalogue.IsToolRunner(m) {
return []string{fmt.Sprintf("%s says what a session waits on; only the tool runner's module (it says "+
"\"tool-runner\": true) declares waits (novox/hq ADR 0367 §2)", m.Module)}
}
var problems []string
seen := map[string]bool{}
@@ -4554,6 +4594,8 @@ func CatalogueProblems(shelf catalogue.Shelf) []string {
problems = append(problems, StreamNameProblems(shelf)...)
// An object bucket's name has one owner, and another module reads it only through its owner's grant (ADR 0592 §5, §9).
problems = append(problems, ObjectsCatalogueProblems(shelf)...)
// Bus state in `requires`: a name on the bus has one owner, and the new form has no readers (ADR 0595, ADR 0596 §2).
problems = append(problems, BusStateCatalogueProblems(shelf)...)
// A module that hears a restricted stream runs as an account of its own (novox/hq ADR 0376 §2).
problems = append(problems, RestrictedHearingProblems(shelf)...)
// A work-queue stream is never asked for by another module's consumes (novox/hq ADR 0436 §6).
@@ -5358,7 +5400,7 @@ func StateProblems(m catalogue.Manifest) []string {
seen[s.Name] = true
if s.History < 0 || s.History > catalogue.StateMostHistory {
problems = append(problems, fmt.Sprintf(
"%s keeps %d values of %q; a key keeps between 1 and %d", m.Module, s.History, s.Name, catalogue.StateMostHistory))
"%s keeps %d values of %q; a key keeps 0 or 1 to %d (0 is taken as 1)", m.Module, s.History, s.Name, catalogue.StateMostHistory))
}
if s.TTLSeconds < 0 {
problems = append(problems, fmt.Sprintf("%s gives %q a negative lifetime", m.Module, s.Name))
@@ -5378,12 +5420,29 @@ func StateProblems(m catalogue.Manifest) []string {
return problems
}
// PredefinedFormProblems refuses each stream a module declares that StreamsKeptUntilMoved does not name for it.
// **A module keeps its data in the bus's predefined forms** (the lead and the operator, 2026-10-29; the operator's rule of
// 2026-10-22, "data storage is done using our predefined nats bus options"): a state, a log or an object store, which the
// runtime reads and writes for its tools and other modules read by a declared `reads`. A stream of its own is outside
// them and refused, naming it. streamsKeptUntilMoved are the streams declared before this check, each allowed by name
// until its module moves it; the list only shrinks. The judge's own exception, kept here and no longer in mesh-sdk,
// which names no module (issue 774, the operator 2026-11-03).
var streamsKeptUntilMoved = map[string][]string{
"brother": {"device-asks"},
"mesh-build-go": {"toolchains-go"},
"mesh-build-node": {"toolchains-node"},
"mesh-mail": {"mail", "mail-sends"},
"only-office": {"convert-asks"},
"mesh-delivery-assigner": {"placement"},
"mesh-delivery-checker": {"checks"},
"mesh-delivery-forge-bridge": {"forge"},
}
// PredefinedFormProblems refuses each stream a module declares that streamsKeptUntilMoved does not name for it.
func PredefinedFormProblems(m catalogue.Manifest) []string {
var out []string
for _, s := range m.Streams {
// A stream that keeps its history is one of the bus's predefined forms (novox/hq ADR 0592 §6).
if !s.KeepsHistory() && !slices.Contains(catalogue.StreamsKeptUntilMoved[m.Module], s.Name) {
if !s.KeepsHistory() && !slices.Contains(streamsKeptUntilMoved[m.Module], s.Name) {
out = append(out, fmt.Sprintf("%s keeps the stream %q, outside the bus's predefined forms: keep it as a state, "+
"a log or an object store, which the runtime reads for the module's tools and others read by `reads`", m.Module, s.Name))
}
@@ -7403,7 +7462,8 @@ var defaultSeats = append([]catalogue.Seat{
// images and archives, by digest — which is why the provision is the artifact store and not an
// image registry.
{Name: "mesh-artifact-store", Scope: catalogue.ScopeMesh, Delivers: "artifact-store", Decision: "novox/hq ADR 0075"},
{Name: "mesh-catalog", Scope: catalogue.ScopeMesh, Decision: "novox/hq ADR 0121"},
// The catalog (novox/hq ADR 0604): the SDK's entry, its four verbs optional while the module that claimed it before stands.
{Name: "mesh-catalog", Scope: catalogue.ScopeMesh, Serves: catalogVerbs(), Decision: "novox/hq ADR 0604"},
// The delivery's owner (novox/hq ADR 0239): one commit's journey from its pull request's head to every
// machine, and a group of them, recorded and ordered by its holder, which asks the controller for each act.
// What it says it says as its module (`mesh-delivery.transition`), through the runtime that launches it.
@@ -7822,6 +7882,26 @@ func configInput() map[string]any {
return in
}
// catalogVerbs are the catalog seat's four verbs, as the SDK's seat serves them (catalogue.CatalogSeat).
func catalogVerbs() []catalogue.Verb {
return []catalogue.Verb{
{Name: "modules", Optional: true, Description: "Every module the trunk holds, each with its repository, its " +
"directory there, the merge commit it was last changed at and its fingerprint (or why it has none): the " +
"catalog of the mesh, read from its holder's state. Reads only.",
Input: schema(map[string]string{}, nil)},
{Name: "module", Optional: true, Description: "One module whole: its repository, directory, commit and " +
"fingerprint, and its manifest as the trunk holds it. Older versions are in its repository's history. Reads only.",
Input: schema(map[string]string{"module": "the module's name"}, []string{"module"})},
{Name: "provides", Optional: true, Description: "Which modules provide a provision (a module provides its own " +
"name too) and which require it, from the module graph's provision edges. Reads only.",
Input: schema(map[string]string{"provision": "the provision, such as postgres-database"}, []string{"provision"})},
{Name: "dependents", Optional: true, Description: "The modules whose build edges reach a module, directly or " +
"through others (a base, a mesh package it publishes, the build module that builds them): what a change of it " +
"builds again, from the module graph. Reads only.",
Input: schema(map[string]string{"module": "the module's name"}, []string{"module"})},
}
}
// deliveryVerbs are the delivery seat's tools: what reads the merges and checks, and the acts of a person and healer H2
// on a merge (ADR 0539 clause 17).
func deliveryVerbs() []catalogue.Verb {
@@ -54,12 +54,6 @@ func TestABinaryMustSayWhichSystemItIsFor(t *testing.T) {
}
}
func TestABinaryThatNamesASystemIsAccepted(t *testing.T) {
if got := buildProblemsOf(t, bundleFor("go", "arch")); got != "" {
t.Fatalf("a compiled bundle naming a system was refused:\n%s", got)
}
}
func TestABlockOutsideTheShapeIsRefused(t *testing.T) {
cases := map[string]string{
`{"seat":"node-bar","kind":"block","content":"[[block]]"}`: "has no data",
@@ -163,12 +157,18 @@ func TestABundleNothingDeliversIsRefused(t *testing.T) {
run := base()
run.Resources = []map[string]any{{"id": "daemon", "type": "process", "artifact": "tools", "run": []any{"node", "tools/index.js"}}}
runtime := base()
runtime.Module = catalogue.RuntimeModule
runtime.Module, runtime.ToolRunner = "node-tools", true
for name, m := range map[string]catalogue.Manifest{"loads": loads, "tools": tools, "a process": run, "the runtime": runtime} {
if p := undeliveredBundles(m); len(p) != 0 {
t.Errorf("a bundle delivered by %s was refused: %v", name, p)
}
}
// The tool runner is what says it is one (issue 774): its name alone exempts nothing.
named := base()
named.Module = "node-tools"
if p := undeliveredBundles(named); len(p) != 1 {
t.Errorf("a bundle of a module named as the tool runner, which does not say it is one: %v", p)
}
}
func TestABundlesWordsAreRefusedWhenTheyAreNotPathsOrConstants(t *testing.T) {
@@ -913,6 +913,12 @@ func TestALogIsNamedLocallyAndCapped(t *testing.T) {
{`{"module":"a.b","version":"1","logs":["c"]}`, `no dot`},
} {
_, err := ParseManifest([]byte(c.manifest))
if c.says == "" {
if err != nil {
t.Errorf("%s was refused: %v", c.manifest, err)
}
continue
}
if err == nil {
t.Errorf("%s was accepted", c.manifest)
continue
@@ -2356,18 +2362,19 @@ func TestAPlaceholderOutsideTheHolderIsRefused(t *testing.T) {
}
}
// **And a language that runs anywhere must not name one.** A system that decides nothing reads as
// though it did, which is the same fault as a restriction that restricts nothing (novox/hq ADR 0045).
func TestAPortableBundleMayNotNameASystem(t *testing.T) {
got := buildProblemsOf(t, bundleFor("typescript", "arch"))
if !strings.Contains(got, "runs anywhere") {
t.Fatalf("a portable bundle was allowed to name a system:\n%s", got)
}
}
func TestAPortableBundleNamingNoSystemIsAccepted(t *testing.T) {
if got := buildProblemsOf(t, bundleFor("typescript", "")); got != "" {
t.Fatalf("an ordinary bundle was refused:\n%s", got)
// **A language that runs anywhere must not name a system.** A system that decides nothing reads as
// though it did, which is the same fault as a restriction that restricts nothing (novox/hq ADR 0045); a compiled
// bundle names one, and a portable bundle naming none is an ordinary one.
func TestABundleNamesASystemOnlyWhereItCompiles(t *testing.T) {
for _, c := range []struct{ language, system, refused string }{
{"go", "arch", ""},
{"typescript", "", ""},
{"typescript", "arch", "runs anywhere"},
} {
got := buildProblemsOf(t, bundleFor(c.language, c.system))
if (c.refused == "") != (got == "") || !strings.Contains(got, c.refused) {
t.Errorf("a %s bundle naming %q: %q, want %q", c.language, c.system, got, c.refused)
}
}
}
@@ -2882,13 +2889,23 @@ func TestAStateNameIsLocalAndOneToken(t *testing.T) {
{`{"module":"a","version":"1","state":["mesh.servers"]}`, `keeps state "mesh.servers": a state is named locally`},
{`{"module":"a","version":"1","state":["my_servers"]}`, `keeps state "my_servers"`},
{`{"module":"a","version":"1","state":["s","s"]}`, `keeps state "s" twice`},
{`{"module":"a","version":"1","state":[{"name":"s","history":65}]}`, `a key keeps between 1 and 64`},
{`{"module":"a","version":"1","state":[{"name":"s","history":65}]}`, `a key keeps 0 or 1 to 64 (0 is taken as 1)`},
{`{"module":"a","version":"1","state":[{"name":"s","history":0}]}`, ``}, // 0 passes, as before (ADR 0579 §1)
// 1966/AC9: one name, one form
{`{"module":"a","version":"1","state":["x"],"requires":[{"name":"x","storage":"key-value","template":"state"}]}`,
`a's bus state "x": the name is also kept in the old section state`},
{`{"module":"a.b","version":"1","state":["s"]}`, `no dot`},
{`{"module":"a","version":"1","reads":["bindings"]}`, `a read is <module>.<name>`},
{`{"module":"a","version":"1","reads":["a.s"]}`, `which is its own state`},
{`{"module":"a","version":"1","state":[{"name":"s","shared":true}]}`, `{name, history, ttl-seconds, per-machine, key-ttl}`},
} {
_, err := ParseManifest([]byte(c.manifest))
if c.says == "" {
if err != nil {
t.Errorf("%s was refused: %v", c.manifest, err)
}
continue
}
if err == nil {
t.Errorf("%s was accepted", c.manifest)
continue
@@ -3097,15 +3114,16 @@ func TestAVerbWithAnUnknownKeyIsRefused(t *testing.T) {
// each refusal is said in the manifest's words, at parse, rather than at the composition of the bus's users.
func TestAWaitIsRefusedUnlessItNamesOneFamilyOfTheToolRunner(t *testing.T) {
for manifest, want := range map[string]string{
`{"module":"shop","version":"1","waits":["mesh-knowledge.reviewed"]}`: "only node-tools",
`{"module":"node-tools","version":"1","waits":["mesh-knowledge.*"]}`: "never a wildcard",
`{"module":"node-tools","version":"1","waits":["**"]}`: "never a wildcard",
`{"module":"node-tools","version":"1","waits":["mesh-knowledge.>"]}`: "never a wildcard",
`{"module":"node-tools","version":"1","waits":["*.reviewed"]}`: "never a wildcard",
`{"module":"node-tools","version":"1","waits":["reviewed"]}`: "does not name an emitter and an event",
`{"module":"node-tools","version":"1","waits":["mesh-knowledge..reviewed"]}`: "does not name an emitter and an event",
`{"module":"node-tools","version":"1","waits":["Mesh-Knowledge.reviewed x"]}`: "does not name an emitter and an event",
`{"module":"node-tools","version":"1","waits":["a.b","a.b"]}`: "twice",
`{"module":"shop","version":"1","waits":["mesh-knowledge.reviewed"]}`: "only the tool runner's module",
`{"module":"node-tools","version":"1","waits":["mesh-knowledge.reviewed"]}`: "only the tool runner's module", // its name says nothing (issue 774)
`{"module":"node-tools","version":"1","tool-runner":true,"waits":["mesh-knowledge.*"]}`: "never a wildcard",
`{"module":"node-tools","version":"1","tool-runner":true,"waits":["**"]}`: "never a wildcard",
`{"module":"node-tools","version":"1","tool-runner":true,"waits":["mesh-knowledge.>"]}`: "never a wildcard",
`{"module":"node-tools","version":"1","tool-runner":true,"waits":["*.reviewed"]}`: "never a wildcard",
`{"module":"node-tools","version":"1","tool-runner":true,"waits":["reviewed"]}`: "does not name an emitter and an event",
`{"module":"node-tools","version":"1","tool-runner":true,"waits":["mesh-knowledge..reviewed"]}`: "does not name an emitter and an event",
`{"module":"node-tools","version":"1","tool-runner":true,"waits":["Mesh-Knowledge.reviewed x"]}`: "does not name an emitter and an event",
`{"module":"node-tools","version":"1","tool-runner":true,"waits":["a.b","a.b"]}`: "twice",
} {
_, err := ParseManifest([]byte(manifest))
if err == nil {
@@ -4807,6 +4825,17 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
[]string{"placement", "proposals", "propose", "withdraw", "again"}) {
t.Errorf("the placement seat is %v, %v", ok, s)
}
// The judge's copy of the catalog seat is the SDK's entry exactly (ADR 0604 §1): four optional verbs, so the claim of the
// module that held the seat before is not outgrown while both stand.
if sdk, ok := catalogue.SeatNamed(catalogue.CatalogSeat); !ok || len(sdk.Serves) != 4 {
t.Errorf("the SDK's catalog seat is %v, %+v", ok, sdk)
} else {
for _, s := range defaultSeats {
if s.Name == catalogue.CatalogSeat && !reflect.DeepEqual(s, sdk) {
t.Errorf("the judge's copy of the catalog seat is %+v, want the SDK's %+v", s, sdk)
}
}
}
// The checker's seat serves the try, which replaces the shell's merge check (ADR 0547 §10, ADR 0245).
var checkerVerbs []string
for _, s := range catalogue.Seats() {
@@ -5160,22 +5189,20 @@ func TestWhereAModulesUpgradePolicyComesFrom(t *testing.T) {
catalogue.PolicyRoll, catalogue.FromDefault},
{"irreplaceable but says roll", catalogue.Manifest{Module: "photos", Upgrade: &catalogue.UpgradePolicy{Policy: catalogue.PolicyRoll},
Data: &catalogue.Data{Own: []catalogue.DataItem{{ID: "originals", Class: catalogue.ClassIrreplaceable}}}}, catalogue.PolicyRoll, catalogue.FromModule},
{"the bus, whatever it says", catalogue.Manifest{Module: "nats", Provides: []catalogue.Offer{{Name: "mesh-bus"}},
Upgrade: &catalogue.UpgradePolicy{Policy: catalogue.PolicyRoll}}, catalogue.PolicyRecord, catalogue.FromBus},
// Issue 774 (the operator, 2026-11-03): the bus has the policy its manifest says, as any module.
{"the bus says record", catalogue.Manifest{Module: "nats", Provides: []catalogue.Offer{{Name: "mesh-bus"}},
Upgrade: &catalogue.UpgradePolicy{Policy: catalogue.PolicyRecord, Why: "its streams"}}, catalogue.PolicyRecord, catalogue.FromModule},
}
for _, c := range cases {
policy, from, _ := catalogue.DerivedUpgrade(c.m)
if policy != c.policy || from != c.from {
t.Errorf("%s: %s from %s, want %s from %s", c.name, policy, from, c.policy, c.from)
}
// ADR 0565 clause 7 (issue 1953 L5-S3): a person's choice wins over each source but the bus; none leaves the
// derivation as it was.
// ADR 0565 clause 7 (issue 1953 L5-S3): a person's choice wins over each source; none leaves the derivation as it
// was. The bus is no exception (ADR 0601 §1).
person := &catalogue.PersonPolicy{V: catalogue.PersonPolicyVersion, Module: c.m.Module, Policy: catalogue.PolicyTogether, Why: "one zone", By: "person.jochen"}
policy, from, _, by := catalogue.Upgrade(c.m, person)
wantPolicy, wantFrom, wantBy := catalogue.PolicyTogether, catalogue.FromPerson, "person.jochen"
if c.from == catalogue.FromBus {
wantPolicy, wantFrom, wantBy = c.policy, catalogue.FromBus, ""
}
if policy != wantPolicy || from != wantFrom || by != wantBy {
t.Errorf("%s with a person's choice: %s from %s by %q", c.name, policy, from, by)
}
+13 -7
View File
@@ -13,6 +13,8 @@ import (
"testing"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-delivery/sdk/graph"
)
// The controller's manifest reader is mesh-controller internal/catalogue (Manifest, manifest.go: Consumes :521,
@@ -84,12 +86,14 @@ func TestManifestDeclarations(t *testing.T) { // 1238/AC1, 1444 (reads the build
t.Errorf("module = %q", m.Module)
}
// 1126: placement is the seat mesh-placement's (ADR 0567): the indexer holds no placement state. ADR 0578 §1: it says
// a refused manifest as manifest-refused, its one event.
if !reflect.DeepEqual(m.Emits, []string{"manifest-refused"}) {
t.Errorf("emits = %v, want manifest-refused alone", m.Emits)
// a refused manifest as manifest-refused, and (issue 1966 D3, 1966/AC10) a newly accepted one asking for bus state as
// manifest-accepted.
if !reflect.DeepEqual(m.Emits, []string{"manifest-refused", "manifest-accepted"}) {
t.Errorf("emits = %v, want manifest-refused and manifest-accepted", m.Emits)
}
// 1126 note 76: heard keeps the last 64 FORGE messages the indexer handled, for indexer_heard.
if want := []state{{Name: "candidates", History: 10}, {Name: "heard", History: 64}, {Name: "manifests", History: 1}, {Name: "modules", History: 20}, {Name: "predefined", History: 5}, {Name: "config-candidates", History: 5}, {Name: "taken", History: 1}, {Name: "takeins", History: 1}, {Name: "verdicts", History: 5}, {Name: delivery.PackagesState, History: delivery.PackagesHistory}}; !reflect.DeepEqual(m.State, want) {
if want := []state{{Name: "candidates", History: 10}, {Name: "heard", History: 64}, {Name: "manifests", History: 1}, {Name: "modules", History: 20}, {Name: "predefined", History: 5}, {Name: "config-candidates", History: 5}, {Name: "taken", History: 1}, {Name: "takeins", History: 1}, {Name: "verdicts", History: 5}, {Name: "oldsections", History: 1}, {Name: delivery.PackagesState, History: delivery.PackagesHistory},
{Name: graph.State, History: graph.History}}; !reflect.DeepEqual(m.State, want) {
t.Errorf("state = %+v, want %+v", m.State, want)
}
if want := []string{"mesh-build-go.toolchains", "mesh-build-node.toolchains", "mesh-delivery-assigner.placement", "mesh-delivery-builder.build-log", "mesh-delivery-checker.previews", "mesh-delivery-gate.source"}; !reflect.DeepEqual(sorted(m.Reads), want) {
@@ -109,11 +113,13 @@ func TestManifestDeclarations(t *testing.T) { // 1238/AC1, 1444 (reads the build
// Three invokes: the file of a registered module's manifest and lock files (registered.go), the forge's trees (the named paths' objects, since the indexer is the one reader of a manifest's paths:
// ADR 0499 clauses 2 and 3, ADR 0495 clause 1; and the module directories of the trunk, issue 1900) and whether a
// commit is in another (issue 1900).
if want := []string{"gitea.gitea_commit_trees", "gitea.gitea_contains", "gitea.gitea_get_file", "gitea.gitea_open_pulls"}; !reflect.DeepEqual(m.Invokes, want) {
if want := []string{"gitea.gitea_commit_trees", "gitea.gitea_contains", "gitea.gitea_get_file", "gitea.gitea_open_pulls", "seat:node-engine.composition"}; !reflect.DeepEqual(m.Invokes, want) {
t.Errorf("invokes = %v, want %v", m.Invokes, want)
}
// 1945: indexer_take_in, served by its tools bundle through the takeins state; the process serves none.
if want := []string{"indexer_take_in"}; !reflect.DeepEqual(m.Tools, want) {
// 1945: indexer_take_in, served by its tools bundle through the takeins state; the process serves none. The same bundle
// reads the verdicts state (indexer_verdicts) and asks the indexer to judge again through the takeins state
// (indexer_judge; issue 774, clause 3). 1966/AC11: indexer_old_sections reads the count of the old sections (state oldsections).
if want := []string{"indexer_take_in", "indexer_old_sections", "indexer_verdicts", "indexer_judge"}; !reflect.DeepEqual(m.Tools, want) {
t.Errorf("tools = %v, want %v", m.Tools, want)
}
// No streams, no bus, no seat (ADR 0437 §4); it says no event.
+11 -4
View File
@@ -3,7 +3,10 @@
"version": "1",
"slug": "indexer",
"tools": [
"indexer_take_in"
"indexer_take_in",
"indexer_old_sections",
"indexer_verdicts",
"indexer_judge"
],
"consumes": [
"mesh-build-go.toolchain.reported",
@@ -21,10 +24,13 @@
{ "name": "taken", "history": 1 },
{ "name": "takeins", "history": 1 },
{ "name": "verdicts", "history": 5 },
{ "name": "packages", "history": 5 }
{ "name": "oldsections", "history": 1 },
{ "name": "packages", "history": 5 },
{ "name": "graph", "history": 5 }
],
"emits": [
"manifest-refused"
"manifest-refused",
"manifest-accepted"
],
"reads": [
"mesh-build-go.toolchains",
@@ -38,7 +44,8 @@
"gitea.gitea_commit_trees",
"gitea.gitea_contains",
"gitea.gitea_get_file",
"gitea.gitea_open_pulls"
"gitea.gitea_open_pulls",
"seat:node-engine.composition"
],
"own-secrets": {
"broker": "${dir:state}/broker"
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-teller
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929 h1:RtAFbUJRP1qO9AIMM2ue+cXNzqj7WZ7IOlSqNvttwuU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -14,6 +14,9 @@ import (
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-delivery/sdk/graph"
"git.novox.be/novox/mesh-sdk/go/provisions"
stdio "git.novox.be/novox/mesh-sdk/go"
)
@@ -43,6 +46,16 @@ func Tools(r view.Reader, u view.UsageReader) []stdio.Tool {
module, _ := args["module"].(string)
return r.Reach(repository, module)
}},
{Name: view.TiersTool,
Description: "A source revision's tiers, worked out now and never stored (novox/hq ADR 0603 §6): the modules the revision moves " +
"(each whose fingerprint last changed at it), ordered by the module graph (a module after the bases and the providers " +
"of the provisions it awaits among them), and for each module what it waits on now and why: its build key, on each node " +
"that wants it its want (the assigner's rule and words when held back), its have, and each provision it awaits. Nodes " +
"the watchman says are offline are named away and left out. At most 500 modules, said when cut short; said when the " +
"module graph cannot be read. Give revision for an older one, none for the gate's newest. Read-only.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"revision": map[string]any{"type": "integer", "description": "the source revision; none for the gate's newest"}}},
Run: func(args map[string]any) (any, error) { return r.TiersOf(args) }},
{Name: view.ModuleHistoryTool,
Description: "The history of one module's value in the indexer's modules state (modules.<module>), with times: each value it " +
"still keeps (up to 20), newest first, with its revision, when it was written, put or del, the repository and commit " +
@@ -307,7 +320,9 @@ func main() {
PredefinedState: read(stdio.State(delivery.IndexerModule + "." + delivery.IndexerPredefinedState)),
ConfigCandidatesState: read(stdio.State(delivery.IndexerModule + "." + delivery.IndexerConfigCandidatesState)),
PackagesState: read(stdio.State(delivery.IndexerModule + "." + delivery.PackagesState)),
BuildLog: stdio.Log(delivery.BuildLogRead).Since}
GraphState: read(stdio.State(graph.Read)), SourceState: read(stdio.State(delivery.GateModule + ".source")),
ProvisionsState: read(stdio.State(provisions.ProvisionsRead)),
BuildLog: stdio.Log(delivery.BuildLogRead).Since}
u := view.UsageReader{Haves: r.HavesState, Resolved: read(stdio.State("mesh-config.resolved")), Tries: read(stdio.State("mesh-delivery-checker.tries")),
Steps: map[string]view.StateRead{}, Pauses: map[string]view.StateRead{}, Backlog: backlogOf(socket())}
for _, lang := range delivery.AskLanguages {
@@ -55,6 +55,7 @@ const (
// 1562/AC1: indexer_module answers the key's value as stored: FP, FP2, None2, Inputs2, Path and Toolchains are all there.
func TestModuleToolAnswersTheStoredValue(t *testing.T) {
t.Run("indexer_packages answers the published modules", packagesToolAnswersThePublishedModules)
t.Run("view_tiers names each module's wait", tiersToolSaysEachModulesWait)
r := fake{"a": withFP2, "b": withNone}.reader()
for key, want := range map[string]string{"a": withFP2, "b": withNone} {
out, err := run(t, r, view.ModuleReadTool, map[string]any{"module": key})
@@ -0,0 +1,122 @@
package main
import (
"encoding/json"
"errors"
"fmt"
"reflect"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-delivery/sdk/graph"
"git.novox.be/novox/mesh-sdk/go/provisions"
"mesh-delivery-view/internal/view"
)
// encoded is v as its contract's encoder writes it, failing the test when it refuses.
func encoded(t *testing.T, v []byte, err error) string {
t.Helper()
if err != nil {
t.Fatal(err)
}
return string(v)
}
// tiersToolSaysEachModulesWait: view_tiers orders a source revision's modules by the module graph and names each one's
// wait (ADR 0603 §6): a consumer after its provider, held by the assigner's provision rule and its provision not provided;
// a module whose build waits for a package; a provider with no wait; a node offline named away; older modules left out.
func tiersToolSaysEachModulesWait(t *testing.T) {
fp := func(c string) string { return "src2:" + strings.Repeat(c, 64) }
manifest := strings.Repeat("d", 64)
mod := func(f string, src uint64) string {
b, err := delivery.EncodeModule(delivery.Module{FP2: f, Manifest: manifest, Src: src, FPSrc: src, Repository: "novox/mesh-catalog", Commit: "c"})
return encoded(t, b, err)
}
node := func(g graph.Node) string {
b, err := graph.Encode(g)
return encoded(t, b, err)
}
build := func(b delivery.BuilderBuild) string {
v, err := delivery.EncodeBuilderBuild(b)
return encoded(t, v, err)
}
want := func(w delivery.Want) string {
v, err := delivery.EncodeWant(w)
return encoded(t, v, err)
}
have := func(h delivery.Have) string {
v, err := delivery.EncodeHave(h)
return encoded(t, v, err)
}
key := func(k string, err error) string {
if err != nil {
t.Fatal(err)
}
return k
}
since := time.Date(2026, 11, 3, 16, 40, 0, 0, time.UTC)
r := fake{"db": mod(fp("a"), 9), "web": mod(fp("b"), 9), "lib": mod(fp("c"), 9), "old": mod(fp("e"), 5)}.reader()
r.GraphState = fake{
"db": node(graph.Node{FP: fp("a"), Repository: "novox/mesh-catalog", Provides: []string{"database"}}),
"web": node(graph.Node{FP: fp("b"), Repository: "novox/mesh-catalog", Requires: []string{"database"}, Awaits: []string{"database"}}),
"lib": node(graph.Node{FP: fp("c"), Repository: "novox/mesh-catalog"}),
}.reader().State
r.Builds = fake{
key(delivery.BuilderBuildKey("db", fp("a"))): build(delivery.BuilderBuild{State: delivery.BuildDone, Manifest: manifest}),
key(delivery.BuilderBuildKey("web", fp("b"))): build(delivery.BuilderBuild{State: delivery.BuildDone, Manifest: manifest}),
key(delivery.BuilderBuildKey("lib", fp("c"))): build(delivery.BuilderBuild{State: delivery.BuildQueued, Waits: "@novox/x@1.2.0"}),
}.reader().State
r.WantsState = fake{
key(delivery.WantKey("ace", "db")): want(delivery.Want{FP: fp("a"), Self: true}),
key(delivery.WantKey("ace", "web")): want(delivery.Want{Self: true, FP: fp("f"), Newest: fp("b"), Rule: delivery.RuleProvisions, Words: "the provision ace.web.database of web on ace is asked, not provided"}),
key(delivery.WantKey("g14", "web")): want(delivery.Want{Self: true, FP: fp("f"), Newest: fp("b"), Rule: delivery.RuleCanary, Words: "the canary"}),
}.reader().State
r.WantsState.History = func(string) ([]view.Revision, error) { return []view.Revision{{At: since}}, nil }
r.HavesState = fake{key(delivery.HaveKey("ace", "db")): have(delivery.Have{FP: fp("a"), Phase: delivery.PhaseRunning, Health: delivery.HealthOK})}.reader().State
p, err := provisions.Encode(provisions.Provision{Provision: "database", Consumer: provisions.End{Module: "web", Node: "ace"},
Provider: provisions.End{Module: "db", Node: "ace"}, State: provisions.Asked, Why: "the build db runs does not offer it", At: since})
r.ProvisionsState = fake{key(provisions.Key("ace", "web", "database", "")): encoded(t, p, err)}.reader().State
off, err := delivery.EncodePresence(delivery.Presence{Online: false})
r.PresenceState = fake{key(delivery.PresenceNodeKey("g14")): encoded(t, off, err)}.reader().State
out, err := run(t, r, view.TiersTool, map[string]any{"revision": float64(9)})
got, _ := out.(view.Tiers)
if err != nil || got.Revision != 9 || got.Modules != 3 || len(got.Tiers) != 2 || !reflect.DeepEqual(got.Away, []string{"g14"}) || got.Graph != "" {
t.Fatalf("view_tiers 9 = %+v, %v", out, err)
}
first := fmt.Sprint(got.Tiers[0])
if len(got.Tiers[0]) != 2 || got.Tiers[0][0].Module != "db" || len(got.Tiers[0][0].Waits) != 0 || got.Tiers[0][1].Module != "lib" ||
len(got.Tiers[0][1].Waits) != 1 || !strings.Contains(got.Tiers[0][1].Waits[0].Words, "waiting for the package @novox/x@1.2.0") {
t.Errorf("the first tier = %s, want db with no wait and lib's build waiting for its package", first)
}
web := got.Tiers[1][0]
if web.Module != "web" || !reflect.DeepEqual(web.After, []string{"db"}) || len(web.Waits) != 2 ||
web.Waits[0].Edge != "want" || web.Waits[0].Node != "ace" || web.Waits[0].Rule != delivery.RuleProvisions || !web.Waits[0].Since.Equal(since) ||
web.Waits[1].Edge != "provision database" || !strings.Contains(web.Waits[1].Words, "asked: the build db runs does not offer it") {
t.Errorf("web = %+v, want after db, held on ace by the provision rule and its provision asked (g14 is away)", web)
}
// No revision named: the gate's newest, its source key's revision.
r.SourceState = view.StateRead{GetEntry: func(string) (json.RawMessage, uint64, error) { return json.RawMessage(`{}`), 9, nil }}
if out, err := run(t, r, view.TiersTool, map[string]any{}); err != nil || out.(view.Tiers).Revision != 9 || out.(view.Tiers).Modules != 3 {
t.Errorf("view_tiers of the gate's newest = %+v, %v", out, err)
}
// The graph not readable: every module in the first tier, said; past 500 modules the answer says it is cut short.
r.GraphState = view.StateRead{Keys: func() ([]string, error) { return nil, errors.New("no bucket") }, Get: r.GraphState.Get}
if out, err := run(t, r, view.TiersTool, map[string]any{"revision": float64(9)}); err != nil || len(out.(view.Tiers).Tiers) != 1 ||
!strings.Contains(out.(view.Tiers).Graph, "cannot be read") {
t.Errorf("view_tiers with no graph = %+v, %v", out, err)
}
many := fake{}
for i := 0; i < view.TiersMost+2; i++ {
many[fmt.Sprintf("m%03d", i)] = mod(fp("a"), 9)
}
r.State = many.reader().State
if out, err := run(t, r, view.TiersTool, map[string]any{"revision": "9"}); err != nil || out.(view.Tiers).Modules != view.TiersMost+2 ||
!strings.Contains(out.(view.Tiers).Cut, "the first 500") {
t.Errorf("view_tiers past its bound = %+v, %v", out.(view.Tiers).Cut, err)
}
}
+5 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-view
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
@@ -20,3 +20,7 @@ require (
golang.org/x/sys v0.48.0 // indirect
golang.org/x/time v0.15.0 // indirect
)
require git.novox.be/novox/mesh-delivery/sdk v0.0.0-00010101000000-000000000000
replace git.novox.be/novox/mesh-delivery/sdk => ../../sdk
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929 h1:RtAFbUJRP1qO9AIMM2ue+cXNzqj7WZ7IOlSqNvttwuU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261103174208-b7abe203a929/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=

Some files were not shown because too many files have changed in this diff Show More