Compare commits

..
35 Commits
Author SHA1 Message Date
mesh-tools bcb3478462 Pin the feature's SDK and node-engine so the release controller builds mesh-host's new code (issue 804)
mesh/merge-gate head bcb3478462ca merged into trunk fcdecb436117 — pass: [release feat/57-config-and-provisioning, judged by its controller 8e285bcd134f] t…
mesh/repo-check head bcb3478462ca merged into trunk fcdecb436117 — fail: [release feat/57-config-and-provisioning, judged by its controller 8e285bcd134f] i…
mesh/delivery stopped: the pull request closed unmerged
mesh-host's declaration code now imports mesh-sdk's seal package (issue 755);
the release branch's controller vendors mesh-host, so it needs both pins
before mesh-host's own pull request can be judged by it.
2026-10-15 21:58:25 +00:00
mesh-admin fcdecb4361 Merge pull request '57/K2: The controller reads, judges against and keeps the catalogue's list of predefined keys (issue 661)' (#323) from pkg/57-k2-predefined-keys into feat/57-config-and-provisioning 2026-10-14 06:52:24 +00:00
mesh-adminandClaude Opus 5.5 197e60c97b List the K2 store tests again, apart from the integration branch's new lines
mesh/merge-gate head 197e60c97bcc merged into trunk 9932adfa00fe — pass: the change touches no module of the mesh's graph
mesh/repo-check head 197e60c97bcc merged into trunk 9932adfa00fe — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group pkg/57-k2-predefined-keys delivered: every member is delivered
Placed above the lines both branches added at the end of the file, so the pull request merges into feat/57-config-and-provisioning without a conflict and the merged list is the union of both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 06:42:51 +00:00
mesh-adminandClaude Opus 5.5 e75c376a7d Take the integration branch's store-test list before merging it
mesh/delivery-group group pkg/57-k2-predefined-keys checking: 1 of 2 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head e75c376a7dd9 merged into trunk 9932adfa00fe — pass: the change touches no module of the mesh's graph
mesh/repo-check head e75c376a7dd9 merged into trunk 9932adfa00fe — pass: its merge-check.sh passed
The two lists added lines at the same place; taking the integration branch's here lets it merge cleanly, and the K2 store tests are listed again right after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 06:41:52 +00:00
mesh-admin 9932adfa00 Merge pull request 'Release 57: bring main into the integration branch (conflicts resolved)' (#324) from pkg/57-sync-main-1 into feat/57-config-and-provisioning 2026-10-14 06:31:43 +00:00
mesh-adminandClaude Opus 5.5 4b2eac702a Bring main into release 57's branch, conflicts resolved
mesh/merge-gate head 4b2eac702ab1 merged into trunk 221fe5a60d1f — pass: the change touches no module of the mesh's graph
mesh/repo-check head 4b2eac702ab1 merged into trunk 221fe5a60d1f — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): feat/57-config-and-provisioning'…
mesh/delivery delivered
So the release branch carries what main gained, and the store-tests drop check passes again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 06:22:29 +00:00
mesh-adminandClaude Opus 5.5 c88474a127 Bring main into release 57's branch, part 2 of 3: the rest at once and the try verb
So the release branch carries what main gained; part 3 follows on this branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 06:20:08 +00:00
mesh-adminandClaude Opus 5.5 3d3ca6e500 Bring main into release 57's branch, part 1 of 3: the builder's try outcome and the release walk
So the release branch carries what main gained; parts 2 and 3 follow on this branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 06:16:47 +00:00
mesh-adminandClaude Opus 5.5 fbcf44702c Prove the list is read strictly and judges module check, the gate and registration
mesh/delivery-group group pkg/57-k2-predefined-keys rejected: a member's own check failed
mesh/merge-gate head fbcf44702cd6 merged into trunk 9932adfa00fe — fail: head fbcf44702cd6 does not merge cleanly into trunk 9932adfa00fe: in conflict test…
mesh/repo-check head fbcf44702cd6 merged into trunk 9932adfa00fe — fail: head fbcf44702cd6 does not merge cleanly into trunk 9932adfa00fe: in conflict test…
mesh/delivery superseded: a newer head of the same pull request
Part 4: the tests of the list's reading and of the controller's commands on it (hq ADR 0348 clause 7, issue 661).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 05:48:49 +00:00
mesh-adminandClaude Opus 5.5 2941ef3695 Read the list of predefined keys strictly, and prove the build, check, gate and registration on it
Part 3: the list's own reading (catalogue/predefined.go) and the tests of the build, module check, merge gate and registration (hq ADR 0348 clause 7, issue 661).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 05:47:44 +00:00
mesh-adminandClaude Opus 5.5 9f2e16ff70 Keep the newest trunk build's list of predefined keys, and prove each refusal
The list itself, read strictly (catalogue/predefined.go); the controller keeps a catalogue trunk build's list in its store when its commit's generation is larger than the one kept (migration 0098), so an older build's list heard later never replaces a newer one, and loads it with its store (hq ADR 0348 clause 7, issue 661). Part 2 of 2: the new files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 05:46:49 +00:00
mesh-adminandClaude Opus 5.5 c0bee5d042 Judge every manifest against the list of predefined keys of its tree, and carry the list from a build
Mesh and node config may hold values only for keys on a fixed, reviewed list (hq ADR 0348 clause 7, design 57 package K2, issue 661). A build reads predefined.json at the root of the repository at the commit it builds and carries it with the commit's generation; module check and the merge gate read it from the tree. (Part 1 of 2: the changes to existing files.)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 05:45:48 +00:00
mesh-admin 221fe5a60d Merge pull request '57/K1: A module declares its config with type, description and validation (issue 660)' (#320) from pkg/57-k1-config-declaration into feat/57-config-and-provisioning 2026-10-14 05:25:35 +00:00
mesh-adminandClaude Opus 5.5 9aa0332694 Refuse an undeclared takes-config key, and say an unset required value in the config form's words
mesh/merge-gate head 9aa033269495 merged into trunk 60d390adeddc — pass: the change touches no module of the mesh's graph
mesh/repo-check head 9aa033269495 merged into trunk 60d390adeddc — pass: its merge-check.sh passed
mesh/delivery delivered
A key a mergeable file took had no type, validation or trusted mark, and the config verb could never set it (ADR 0348 §4). A required config value nothing sets was refused naming ${setting:} and settings set, words the module does not use. Review of mesh-controller#320 (issue 660).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 05:17:26 +00:00
mesh-admin 60d390aded Merge pull request '57/F2b: A settings layer no longer replaces what a provider serves (issue 696)' (#321) from pkg/57-f2b-served-keys into feat/57-config-and-provisioning 2026-10-14 05:16:03 +00:00
mesh-adminandClaude Opus 5.5 c5fe7d4795 Stop a settings layer replacing what a provider serves (issue 696)
mesh/merge-gate head c5fe7d4795d7 merged into trunk dc36536e199e — pass: the change touches no module of the mesh's graph
mesh/repo-check head c5fe7d4795d7 merged into trunk dc36536e199e — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group pkg/57-f2b-served-keys delivered: every member is delivered
What a provider serves is its definition's (ADR 0348 clause 11): a layer
fills only the keys it asks for as ${setting:<key>}. Settle replaced any
served key a layer named, so a setting for one of a provider's files could
change what every consumer was told. influxdb on ace was the one live layer
relying on it and now serves its settings (mesh-catalog, merged first), so
grafana's data source composes byte for byte as before. UnusedSettings no
longer counts a literal served key as a place a setting lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 05:03:08 +00:00
mesh-adminandClaude Opus 5.5 ae071e2b59 Prove each config refusal and the byte-identical composition of a module moved to config (issue 660)
mesh/merge-gate out of date: judged against trunk dc36536e199e, the trunk is now 60d390adeddc; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk dc36536e199e, the trunk is now 60d390adeddc; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
One registration refusal test per rule of design 57 package K1, the forms kept apart and written back as read, and dunst rewritten from settings to config composing to the same bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 01:39:13 +00:00
mesh-adminandClaude Opus 5.5 233956ef09 Let a module declare its config with type, description and validation (issue 660)
The config form's declaration and its registration refusals, each by name with what to do (ADR 0348, design 57 package K1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 01:38:04 +00:00
mesh-adminandClaude Opus 5.5 2f40ae5f76 Read the config form beside the settings form in a manifest (part of issue 660)
The manifest's hooks for the config form: read into the fields the settings form fills, written back as read, and judged by ConfigProblems; the rules are in config.go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 01:36:45 +00:00
mesh-admin dc36536e19 Merge pull request '57/V1: The vault's seat promises get (issue 674)' (#317) from pkg/57-v1-vault-in-go into feat/57-config-and-provisioning 2026-10-14 01:19:23 +00:00
mesh-admin 71362c196d Merge pull request '57/F2: Placement leaves the settings layers, held beside the assignment (issue 651)' (#316) from pkg/57-f2-placement-facts into feat/57-config-and-provisioning 2026-10-14 01:09:55 +00:00
mesh-admin 617881c309 Merge pull request '57/T1: A file resource may name a template file in its module's repository (issue 654)' (#313) from pkg/57-t1-template-files into feat/57-config-and-provisioning 2026-10-14 01:00:47 +00:00
mesh-adminandClaude Opus 5.5 661918f4e4 Promise get on the vault's seat, so the Go vault can serve it (issue 674)
mesh/merge-gate head 661918f4e4e0 merged into trunk 71362c196dbc — pass: the change touches no module of the mesh's graph
mesh/repo-check head 661918f4e4e0 merged into trunk 71362c196dbc — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group pkg/57-v1-vault-in-go delivered: every member is delivered
The vault hands a secret only on request and reply, sealed to the asker's key
(hq ADR 0349 §11). The seat set is the controller's, so the verb is promised
here; optional until the vault that runs serves it (design 33 §7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:51:59 +00:00
mesh-adminandClaude Opus 5.5 def3ccefe3 Refuse templates read through a symlink anywhere under files/
mesh/delivery delivered
mesh/merge-gate head def3ccefe3c8 merged into trunk bfc79c64d989 — pass: the change touches no module of the mesh's graph
mesh/repo-check head def3ccefe3c8 merged into trunk bfc79c64d989 — pass: its merge-check.sh passed
A symlinked files/ or a symlinked directory under it let module check, the merge
gate and module add read a template from outside the repository tree, and a
symlink no template names passed them while the build's packing refuses it
(review of mesh-controller#313). Every entry under files/ must now be a plain
directory or file, and templates are read through an os.Root of the module's
directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:51:48 +00:00
mesh-adminandClaude Opus 5.5 912d76df25 Move every layer's placement out with migration 0097, and prove no machine is told anything new (issue 651)
mesh/delivery delivered
mesh/merge-gate head 912d76df25f8 merged into trunk 617881c3093d — pass: the change touches no module of the mesh's graph
mesh/repo-check head 912d76df25f8 merged into trunk 617881c3093d — pass: its merge-check.sh passed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:45:58 +00:00
mesh-adminandClaude Opus 5.5 3048dccd0e Keep the declarations the controller composed before placement moved (issue 651)
Written by the integration branch's controller at bfc79c6 from the seed of
TestADeclarationIsTheSameAfterPlacementMoves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:44:51 +00:00
mesh-adminandClaude Opus 5.5 f0af09cfc7 Test placement read only from the placement, and raise a store before a migration (issue 651)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:44:20 +00:00
mesh-adminandClaude Opus 5.5 85b85a3fa9 Say a layer whole where the mesh reports it, and give tests their placement (issue 651)
The facts snapshot, the take preview and the settings history read a layer as it
was set; tests that place give the layer through LayerOf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:43:40 +00:00
mesh-adminandClaude Opus 5.5 02b542a059 Keep a layer's placement beside it, and say the layer whole (issue 651)
The settings verb's path splits a layer it is given into its config values and
its placement, so what a person types still works; a layer reads back and goes
into the history whole.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:42:37 +00:00
mesh-adminandClaude Opus 5.5 7530c2aeac Read placement from a layer's Placement, never from its config values (issue 651)
ADR 0348 clause 11: places, accesses and kept networks are placement, not config.
Today's mergeable files still take them as said, so nothing composes differently.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:41:06 +00:00
mesh-adminandClaude Opus 5.5 f45e0a1de5 Let a file resource name a template file in its module's repository: composition, check and gate tests
mesh/merge-gate head f45e0a1de508 merged into trunk bfc79c64d989 — pass: the change touches no module of the mesh's graph
mesh/repo-check head f45e0a1de508 merged into trunk bfc79c64d989 — pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:59:22 +00:00
mesh-adminandClaude Opus 5.5 276f6755ab Let a file resource name a template file in its module's repository: the catalogue's reading of templates
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:58:16 +00:00
mesh-adminandClaude Opus 5.5 1ce4ab8c2a Let a file resource name a template file in its module's repository: the templates and their tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:57:33 +00:00
mesh-adminandClaude Opus 5.5 d43fbc202c Let a file resource name a template file in its module's repository
A file's text may live in files/<name> beside the manifest instead of inline,
so modules can move their files out of module.json (ADR 0349 clause 2, issue
654). The build packs files/ into the module's declared archive and reads each
template back from the packed bytes, proven by the digest, into content: the
manifest the mesh holds, composition and the node-engine stay as they are.
module check, the merge gate and module add read templates from the tree beside
the manifest, so a change to a template alone is judged. A file naming both
template and content, a template outside files/, a missing one, and a held
manifest that still names one are refused by name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:56:43 +00:00
mesh-admin bfc79c64d9 Open the integration branch of release 57 (config and provisioning)
So the packages of hq design 57 meet here, reviewed one by one, and reach main as one delivery group.
2026-10-13 23:39:22 +00:00
214 changed files with 5464 additions and 13779 deletions
+6
View File
@@ -0,0 +1,6 @@
# Release 57: config and provisioning
This branch carries the work of hq design 57 (ADR 0348, ADR 0349) in this repository. Each package
merges here through its own reviewed pull request; the branch reaches main as one pull request, in one
delivery group with the branch of the same name in mesh-controller, mesh-host, mesh-sdk, mesh-catalog
and hq, after a rehearsal in the lab. This file is deleted by the last commit before that pull request.
+19 -54
View File
@@ -26,7 +26,6 @@ import (
"os/signal" "os/signal"
"sort" "sort"
"strings" "strings"
"sync"
"syscall" "syscall"
"time" "time"
@@ -273,12 +272,7 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
// role's own events under the build's id (novox/hq ADR 0157) — so whoever asked, and anybody // role's own events under the build's id (novox/hq ADR 0157) — so whoever asked, and anybody
// watching, reads the same lines this container's log holds, live, and after the fact from the // watching, reads the same lines this container's log holds, live, and after the fact from the
// stream. Said first, before anything runs, so a build that hangs is one that visibly started. // stream. Said first, before anything runs, so a build that hangs is one that visibly started.
// One line at a time: a check's builds for its merge say their lines while its tests say theirs (novox/hq
// issue 716).
var saying sync.Mutex
say := func(step, message string) { say := func(step, message string) {
saying.Lock()
defer saying.Unlock()
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message) fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
work.Say(step, message) work.Say(step, message)
if mine != nil && step != "run" && step != "output" { if mine != nil && step != "run" && step != "output" {
@@ -319,26 +313,20 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
registry = r.Address registry = r.Address
} }
spec := checkSpecOf(request) spec := checkSpecOf(request)
// The judge cache the build agent declares as its own data, never in the workspace: only judge builds and
// mesh-lab's replays mount it, never a check's containers, so no pull request's script can write what a judge
// is built with or kept as, or what the replays run (novox/hq ADR 0358 §6, issue 709).
spec.JudgeCache = judgeCache()
// What the merge would move, built by the check for the walk to take (novox/hq ADR 0350 §1): into the // What the merge would move, built by the check for the walk to take (novox/hq ADR 0350 §1): into the
// scratch namespace. **Never with the package registry's credential**: this is code nobody has approved // scratch namespace. **Never with the package registry's credential**: this is code nobody has approved
// yet, and a recipe could read the credential it is handed (issue 462). A module that resolves packages // yet, and a recipe could read the credential it is handed (issue 462). A module that resolves packages
// at build time has no reuse key anyway, so nothing is lost by refusing it here. // at build time has no reuse key anyway, so nothing is lost by refusing it here.
if isRegistry && len(spec.Builds) > 0 { if isRegistry && len(spec.Builds) > 0 {
spec.BuildWith = &builder.CheckBuilding{Publish: publisher, Scratch: registryHere, Held: request.Held, spec.BuildWith = &builder.CheckBuilding{Publish: publisher, Scratch: registryHere, Held: request.Held,
Running: request.Check.Running, Seats: request.Seats, Named: namedOf(request.Check.Named)} Seats: request.Seats}
} }
var v builder.CheckVerdict var v builder.CheckVerdict
v, err = builder.Check(building, builder.Command, spec, workspace, registry, forgeFrom(), say) v, err = builder.Check(building, builder.Command, spec, workspace, registry, forgeFrom(), say)
if err == nil { if err == nil {
result.Check = &link.CheckOutcome{Verdict: v.Verdict, Summary: v.Summary, Report: v.Report, result.Check = &link.CheckOutcome{Verdict: v.Verdict, Summary: v.Summary, Report: v.Report,
Took: v.Took.Round(time.Second).String(), Gate: layerOf(v.Gate), RepoCheck: layerOf(v.Repo), Took: v.Took.Round(time.Second).String(), Gate: layerOf(v.Gate), RepoCheck: layerOf(v.Repo),
Head: v.Head, Trunk: v.Trunk, Beside: v.Beside, Built: checkBuiltOf(v.Built), Head: v.Head, Trunk: v.Trunk, Beside: v.Beside, Built: checkBuiltOf(v.Built)}
Release: v.Release, JudgedBy: v.JudgedBy, BesideAt: v.BesideAt, Validator: v.Validator,
Siblings: siblingsOut(v.Siblings), SiblingsUnknown: v.SiblingsUnknown, JudgeFrom: v.JudgeFrom}
// What each of a group's heads was judged as (novox/hq ADR 0323 §6). // What each of a group's heads was judged as (novox/hq ADR 0323 §6).
for _, m := range v.Members { for _, m := range v.Members {
result.Check.Members = append(result.Check.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo, result.Check.Members = append(result.Check.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo,
@@ -364,7 +352,15 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
reuse.Pulls = append(reuse.Pulls, builder.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number}) reuse.Pulls = append(reuse.Pulls, builder.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number})
} }
// Only what a verdict names is taken (ADR 0350 §2). // Only what a verdict names is taken (ADR 0350 §2).
reuse.Named = namedOf(request.Reuse.Builds) for _, b := range request.Reuse.Builds {
named := builder.NamedBuild{Pull: builder.PullRef{Owner: b.Pull.Owner, Repo: b.Pull.Repo,
Number: b.Pull.Number}, Check: b.Check, Module: b.Built.Module, Key: b.Built.Key,
Digests: map[string]string{}}
for _, m := range b.Built.Made {
named.Digests[m.Name] = builder.DigestOf(m.Reference)
}
reuse.Named = append(reuse.Named, named)
}
built, err = builder.BuildReusing(building, builder.Labelled(builder.Command, request.ID), publisher, reuse, built, err = builder.BuildReusing(building, builder.Labelled(builder.Command, request.ID), publisher, reuse,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc, request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats) forgeFrom(), say, request.Seats)
@@ -428,6 +424,11 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
for _, s := range built.Sources { for _, s := range built.Sources {
result.Sources = append(result.Sources, link.BuildSource{Repository: s.Repository, Ref: s.Ref, Paths: s.Paths}) result.Sources = append(result.Sources, link.BuildSource{Repository: s.Repository, Ref: s.Ref, Paths: s.Paths})
} }
// The list of predefined keys the manifest was judged against, for the controller to publish
// (novox/hq ADR 0348 §7).
if p := built.Predefined; p != nil {
result.Predefined = &link.PredefinedList{Keys: p.Keys, Generation: p.Generation}
}
say("built", built.Manifest.Module+" from "+short(built.Commit)) say("built", built.Manifest.Module+" from "+short(built.Commit))
} }
} }
@@ -468,13 +469,9 @@ func checkSpecOf(request link.BuildRequest) builder.CheckSpec {
Owner: c.Owner, Repo: c.Repo, Number: c.Number, Paths: c.Paths, Beside: map[string]builder.Beside{}, Owner: c.Owner, Repo: c.Repo, Number: c.Number, Paths: c.Paths, Beside: map[string]builder.Beside{},
Modules: c.Modules, New: c.New, Manifests: c.Manifests, Judge: c.Judge, Base: c.Base, Modules: c.Modules, New: c.New, Manifests: c.Manifests, Judge: c.Judge, Base: c.Base,
Toolchain: builder.ToolchainOf(request.Held), Toolchains: builder.ToolchainsOf(request.Held), Toolchain: builder.ToolchainOf(request.Held), Toolchains: builder.ToolchainsOf(request.Held),
Tests: c.Tests, StoreOnly: c.StoreOnly, Release: c.Release, SiblingsUnknown: c.SiblingsUnknown} Tests: c.Tests, StoreOnly: c.StoreOnly}
for _, s := range c.Siblings {
spec.Siblings = append(spec.Siblings, builder.Sibling{Repository: s.Repository, Number: s.Number, Head: s.Head,
Base: s.Base, Dir: s.Dir})
}
for dir, b := range c.Beside { for dir, b := range c.Beside {
spec.Beside[dir] = builder.Beside{Repository: b.Repository, Ref: b.Ref, Else: b.Else} spec.Beside[dir] = builder.Beside{Repository: b.Repository, Ref: b.Ref}
} }
for _, b := range c.Builds { for _, b := range c.Builds {
spec.Builds = append(spec.Builds, builder.CheckBuildSpec{Module: b.Module, Repository: b.Repository, Path: b.Path, spec.Builds = append(spec.Builds, builder.CheckBuildSpec{Module: b.Module, Repository: b.Repository, Path: b.Path,
@@ -487,21 +484,6 @@ func checkSpecOf(request link.BuildRequest) builder.CheckSpec {
return spec return spec
} }
// siblingsOut is the siblings a check read as the outcome carries them (novox/hq ADR 0382 §4).
func siblingsOut(read []builder.JudgedSibling) []link.CheckSibling {
var out []link.CheckSibling
for _, s := range read {
out = append(out, link.CheckSibling{Repository: s.Repository, Number: s.Number, Head: s.Head, Base: s.Base,
BaseCommit: s.BaseCommit})
}
return out
}
// judgeCache is the judge cache the build agent's manifest declares (MESH_JUDGE_CACHE, novox/hq ADR 0358 §6): a
// directory mounted at the same path on the host, so a judge build's mounts name what the agent sees. Empty when the
// agent declares none, and every judge is then built cold under its check's root.
func judgeCache() string { return os.Getenv("MESH_JUDGE_CACHE") }
// triedOf is a try's outcome as its log line says it (novox/hq issue 686): the repository's own check's verdict, // triedOf is a try's outcome as its log line says it (novox/hq issue 686): the repository's own check's verdict,
// or the check's where that did not run — a head that does not merge — and the lines naming what failed. // or the check's where that did not run — a head that does not merge — and the lines naming what failed.
func triedOf(v builder.CheckVerdict) link.Tried { func triedOf(v builder.CheckVerdict) link.Tried {
@@ -519,28 +501,11 @@ func triedOf(v builder.CheckVerdict) link.Tried {
return t return t
} }
// namedOf are the builds the seat's verdicts name, as the builder believes a scratch record by them (novox/hq ADR
// 0350 §2, issue 703): each module's key with every artifact's digest.
func namedOf(builds []link.CheckedBuild) []builder.NamedBuild {
var out []builder.NamedBuild
for _, b := range builds {
named := builder.NamedBuild{Pull: builder.PullRef{Owner: b.Pull.Owner, Repo: b.Pull.Repo,
Number: b.Pull.Number}, Check: b.Check, Module: b.Built.Module, Key: b.Built.Key,
Digests: map[string]string{}}
for _, m := range b.Built.Made {
named.Digests[m.Name] = builder.DigestOf(m.Reference)
}
out = append(out, named)
}
return out
}
// checkBuiltOf is a check's builds for its merge as the outcome carries them (novox/hq ADR 0350 §1). // checkBuiltOf is a check's builds for its merge as the outcome carries them (novox/hq ADR 0350 §1).
func checkBuiltOf(built []builder.CheckBuilt) []link.CheckBuilt { func checkBuiltOf(built []builder.CheckBuilt) []link.CheckBuilt {
var out []link.CheckBuilt var out []link.CheckBuilt
for _, b := range built { for _, b := range built {
c := link.CheckBuilt{Module: b.Module, Key: b.Key, NotBuilt: b.NotBuilt, Took: b.Took.Round(time.Second).String(), c := link.CheckBuilt{Module: b.Module, Key: b.Key, NotBuilt: b.NotBuilt, Took: b.Took.Round(time.Second).String()}
Kept: b.Kept, Same: b.Same, SameAs: b.SameAs}
for _, m := range b.Made { for _, m := range b.Made {
c.Made = append(c.Made, link.MadeArtifact{Name: m.Name, Kind: m.Kind, Reference: m.Reference}) c.Made = append(c.Made, link.MadeArtifact{Name: m.Name, Kind: m.Kind, Reference: m.Reference})
} }
-46
View File
@@ -1,46 +0,0 @@
package main
import (
"testing"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// **A pull request into a release branch reaches the check as one** (novox/hq ADR 0358): the build seat hands the
// check the release branch and, beside it, the trunk each repository is read at when it has no branch of that name;
// and it hands the check the judge cache the build agent declares (MESH_JUDGE_CACHE), never a path of its own.
func TestTheBuildSeatHandsTheCheckItsReleaseBranch(t *testing.T) {
spec := checkSpecOf(link.BuildRequest{ID: "b", Repository: "r", Ref: "abc", Check: &link.CheckRequest{
Owner: "novox", Repo: "mesh-catalog", Base: "feat/57", Release: "feat/57",
Beside: map[string]link.CheckedOut{"mesh-host": {Repository: "h", Ref: "feat/57", Else: "main"}}}})
if spec.Release != "feat/57" || spec.Beside["mesh-host"].Ref != "feat/57" || spec.Beside["mesh-host"].Else != "main" {
t.Fatalf("the check was handed release %q and mesh-host %+v", spec.Release, spec.Beside["mesh-host"])
}
t.Setenv("MESH_JUDGE_CACHE", "/var/lib/mesh/build-agent/judge-cache")
if got := judgeCache(); got != "/var/lib/mesh/build-agent/judge-cache" {
t.Fatalf("the judge cache the manifest declares is read as %q", got)
}
t.Setenv("MESH_JUDGE_CACHE", "")
if got := judgeCache(); got != "" {
t.Fatalf("with no judge cache declared, %q is used", got)
}
}
// The siblings a check is asked to read reach the build seat's spec and, as read, the outcome (novox/hq ADR 0382).
func TestTheSiblingsAnAskNamesReachTheSpecAndTheOutcome(t *testing.T) {
spec := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main",
Siblings: []link.CheckSibling{{Repository: "novox/mesh-controller", Number: 346, Head: "a", Base: "main", Dir: "mesh-controller"}},
SiblingsUnknown: "why"}})
if len(spec.Siblings) != 1 || spec.Siblings[0] != (builder.Sibling{Repository: "novox/mesh-controller", Number: 346,
Head: "a", Base: "main", Dir: "mesh-controller"}) || spec.SiblingsUnknown != "why" {
t.Fatalf("the spec reads %+v", spec)
}
out := siblingsOut([]builder.JudgedSibling{{Repository: "novox/mesh-controller", Number: 346, Head: "a", Base: "main", BaseCommit: "b"}})
if len(out) != 1 || out[0].BaseCommit != "b" || out[0].Number != 346 {
t.Fatalf("the outcome reads %+v", out)
}
if plain := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "o", Repo: "r"}}); plain.Siblings != nil {
t.Fatalf("a request that names none has siblings %+v", plain.Siblings)
}
}
+2 -5
View File
@@ -26,11 +26,8 @@ import (
// //
// - **The serving controller** takes it before it does anything else — before it asserts the bus's // - **The serving controller** takes it before it does anything else — before it asserts the bus's
// objects, which are the controller's to write — waiting while another holds it, and renews it. // objects, which are the controller's to write — waiting while another holds it, and renews it.
// Another holder seen, or the key's age passed with no renewal made, is the lease lost: the gate // A renewal refused or failed is the lease lost: the gate closes at once and the process exits, so
// closes at once and the process exits, so its service manager restarts it as a candidate (serve, // its service manager restarts it as a candidate (serve, in push.go).
// in push.go). A renewal the bus does not answer in time is not a loss (novox/hq issue 477): the
// lease is kept, no new act starts once half the key's age passed unrenewed, and the next renewal
// made opens the gate again.
// - **A command run at a shell** — `push` in the installer, the lab, a person repairing a mesh whose // - **A command run at a shell** — `push` in the installer, the lab, a person repairing a mesh whose
// controller is down (issue 201) — acts **under the holder's epoch** when a controller holds the // controller is down (issue 201) — acts **under the holder's epoch** when a controller holds the
// lease: it is the same mesh's word, composed and sent under the store's hold of each machine like // lease: it is the same mesh's word, composed and sent under the store's hold of each machine like
+1 -1
View File
@@ -140,7 +140,7 @@ func TestTheRegistryTrustAndEveryImageFollowThePortTheNodeGaveTheStore(t *testin
} }
// And a replay to the catalogue says where the store is now. // And a replay to the catalogue says where the store is now.
announced, err := following{open: open}.Announceable(ctx) announced, err := following{open}.Announceable(ctx)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
+3 -2
View File
@@ -424,8 +424,9 @@ func comparisonOf(module string, c comparison, opts takeOptions) (preview string
} }
// And its settings on this machine, composed against its definition (rule 1, rule 6). // And its settings on this machine, composed against its definition (rule 1, rule 6).
for _, layer := range c.layers { for _, layer := range c.layers {
keys := make([]string, 0, len(layer.Values)) said := layer.Whole()
for k := range layer.Values { keys := make([]string, 0, len(said))
for k := range said {
keys = append(keys, k) keys = append(keys, k)
} }
sort.Strings(keys) sort.Strings(keys)
+3 -8
View File
@@ -688,7 +688,7 @@ func spelledOf(m inventory.BatchedMerge) string {
func namedOf(m inventory.BatchedMerge, repository string) inventory.PlanMerge { func namedOf(m inventory.BatchedMerge, repository string) inventory.PlanMerge {
k := announcedOf(m) k := announcedOf(m)
return inventory.PlanMerge{Repository: repository, Commit: m.Commit, Number: k.Number, Title: k.Title, Moves: k.Moves, return inventory.PlanMerge{Repository: repository, Commit: m.Commit, Number: k.Number, Title: k.Title, Moves: k.Moves,
Head: k.HeadSHA, Merged: m.Merged, Heard: m.Heard} Merged: m.Merged, Heard: m.Heard}
} }
// laterMerge says a was merged after b: by the forge's merge time, then by when each was heard. // laterMerge says a was merged after b: by the forge's merge time, then by when each was heard.
@@ -1000,15 +1000,10 @@ func planBatch(ctx context.Context, open *stores, batch *inventory.Plan, carry [
plan.Delivery.Merges = named plan.Delivery.Merges = named
plan.Delivery.Alone = batch.Delivery != nil && batch.Delivery.Alone plan.Delivery.Alone = batch.Delivery != nil && batch.Delivery.Alone
plan.Delivery.Withheld = withheld plan.Delivery.Withheld = withheld
// **Left where it is, a module whose program did not change** (novox/hq issue 721): its check found it the same as
// the build the mesh holds.
for n, why := range leaveSameAsRunning(ctx, inv, &plan, moved, append(append([]inventory.Edge{}, edges...), ordered...)) {
unchanged[n] = why
}
plan.Delivery.Unchanged = unchangedLeft(unchanged, plan.Modules) plan.Delivery.Unchanged = unchangedLeft(unchanged, plan.Modules)
// **Its moments and its class** (novox/hq ADR 0282 decision 6): measured, never acted on. // **Its moments and its class** (novox/hq ADR 0282 decision 6): measured, never acted on.
plan.Times = walkTimesAtCut(*batch, plan, entries, now) plan.Times = walkTimesAtCut(*batch, plan, entries, now)
if len(moved) == 0 || len(plan.Modules) == 0 { if len(moved) == 0 {
plan.State = inventory.PlanDone plan.State = inventory.PlanDone
plan.Tiers = [][]string{} plan.Tiers = [][]string{}
plan.Note = "moved nothing the mesh holds: nothing to walk" plan.Note = "moved nothing the mesh holds: nothing to walk"
@@ -1049,7 +1044,7 @@ func planBatch(ctx context.Context, open *stores, batch *inventory.Plan, carry [
return nil return nil
} }
// Written and its first tier asked as one act on the plans (novox/hq issue 213), under the caller's hold. // Written and its first tier asked as one act on the plans (novox/hq issue 213), under the caller's hold.
if err := askTier(ctx, inv, &plan, plan.Tiers[plan.Tier]); err != nil { if err := askTier(ctx, inv, &plan); err != nil {
return err return err
} }
if err := inv.SavePlan(ctx, &plan); err != nil { if err := inv.SavePlan(ctx, &plan); err != nil {
+1 -1
View File
@@ -758,7 +758,7 @@ func TestTheCatchUpKeepsWhatACutMadeHistory(t *testing.T) {
return entries, read, err return entries, read, err
} }
if err := catchUpOnMerges(ctx, time.Now(), unheardMerges{announcedList(lost), open.inventory}, catalogued, if err := catchUpOnMerges(ctx, time.Now(), unheardMerges{announcedList(lost), open.inventory}, catalogued,
(following{open: open}).SourceMoved, t.Logf); err != nil { (following{open}).SourceMoved, t.Logf); err != nil {
t.Fatal(err) t.Fatal(err)
} }
got, _ := open.inventory.PlanByID(ctx, w.ID) got, _ := open.inventory.PlanByID(ctx, w.ID)
+34 -33
View File
@@ -183,10 +183,12 @@ func buildFrom(result link.BuildResult) inventory.Build {
kept.Manifest = recordedManifest(result.Manifest, announced) kept.Manifest = recordedManifest(result.Manifest, announced)
// The module name comes from the manifest, which only exists when the build got that far. // The module name comes from the manifest, which only exists when the build got that far.
if len(kept.Manifest) > 0 { if len(kept.Manifest) > 0 {
if m, err := catalogue.ParseManifest(kept.Manifest); err == nil { if list, err := judgedWith(result); err == nil {
if m, err := catalogue.ParseManifestWith(kept.Manifest, list); err == nil {
kept.Module = m.Module kept.Module = m.Module
} }
} }
}
return kept return kept
} }
@@ -603,6 +605,11 @@ func saysWhenThePolicyActs(ctx context.Context, inv *inventory.Inventory, module
// written with the same values. // written with the same values.
func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResult) ( func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResult) (
catalogue.Manifest, inventory.Build, error) { catalogue.Manifest, inventory.Build, error) {
// The list of predefined keys as the store keeps it now: another controller process may have kept a newer one.
if err := loadPredefined(ctx, inv); err != nil {
return catalogue.Manifest{}, inventory.Build{}, fmt.Errorf("the list of predefined keys kept cannot be "+
"read: %w", err)
}
kept := buildFrom(result) kept := buildFrom(result)
if err := inv.RecordBuild(ctx, kept); err != nil { if err := inv.RecordBuild(ctx, kept); err != nil {
return catalogue.Manifest{}, kept, err return catalogue.Manifest{}, kept, err
@@ -613,7 +620,15 @@ func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResu
return catalogue.Manifest{}, kept, fmt.Errorf("%s could not build %s:\n%s", return catalogue.Manifest{}, kept, fmt.Errorf("%s could not build %s:\n%s",
result.On, result.Repository, result.Failed) result.On, result.Repository, result.Failed)
} }
manifest, err := catalogue.ParseManifest(kept.Manifest) // Judged first against the list the build was judged against: the catalogue's at the commit built (novox/hq
// ADR 0348 §7), so a module naming a key its own commit adds is read; and again below against the list the
// mesh publishes, before it is registered.
list, err := judgedWith(result)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: %w", result.On,
result.Repository, short(result.Commit), err)
}
manifest, err := catalogue.ParseManifestWith(kept.Manifest, list)
if err != nil { if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s and what came back is not a manifest: %w", return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s and what came back is not a manifest: %w",
result.On, result.Repository, err) result.On, result.Repository, err)
@@ -687,6 +702,23 @@ func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResu
"back: it is recorded and not registered again — a newer build is", result.On, manifest.Module, "back: it is recorded and not registered again — a newer build is", result.On, manifest.Module,
short(result.Commit)) short(result.Commit))
} }
// **The newest trunk build's list of predefined keys is published** (novox/hq ADR 0348 §7): a build of the
// catalogue that may be registered keeps its list when its commit is newer than the one kept. Then the module
// is judged against the list the mesh publishes, as every manifest read again is: one naming a key that list
// does not hold, or of another type, is refused here and never enters the catalogue.
if carried, err := carriedList(result); err != nil {
return manifest, kept, err
} else if carried != nil {
if err := publishPredefined(ctx, inv, result, carried); err != nil {
fmt.Printf("%s: the list of predefined keys it carried is not published: %v\n", result.ID, err)
}
}
if manifest.NamesPredefined() {
if _, err := catalogue.ParseManifest(kept.Manifest); err != nil {
return manifest, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: judged against the "+
"list of predefined keys the mesh publishes, %w", result.On, manifest.Module, short(result.Commit), err)
}
}
// **A build whose source is unchanged is never a move** (novox/hq issue 280): a rebuild made from // **A build whose source is unchanged is never a move** (novox/hq issue 280): a rebuild made from
// what the build the mesh stands on was made from registers that build's artifacts at the new // what the build the mesh stands on was made from registers that build's artifacts at the new
// commit, so no machine is sent a new digest for a source nobody changed — an image is not // commit, so no machine is sent a new digest for a source nobody changed — an image is not
@@ -820,9 +852,6 @@ type answers struct {
sources map[string]inventory.Source sources map[string]inventory.Source
// waiting is every machine not running what the mesh would send it. // waiting is every machine not running what the mesh would send it.
waiting []inventory.Machine waiting []inventory.Machine
// recordedWaits is, per machine, every move of a recorded module that waits there for a person (novox/hq ADR
// 0377 §6): the machine is behind for these, and `push <node>` sends them.
recordedWaits map[string][]string
// reported is every machine's last word beside when it was last sent a declaration — the // reported is every machine's last word beside when it was last sent a declaration — the
// pair that answers "has it caught up", which waiting alone cannot (the sent digest is // pair that answers "has it caught up", which waiting alone cannot (the sent digest is
// recorded at send, not at apply). // recorded at send, not at apply).
@@ -902,34 +931,6 @@ type answers struct {
// //
// Routed through the artifact store as the network reaches it now (novox/hq 04-ISSUES/102): a // Routed through the artifact store as the network reaches it now (novox/hq 04-ISSUES/102): a
// base is recorded by digest and path, and a build machine needs something it can pull. // base is recorded by digest and path, and a build machine needs something it can pull.
// runningBy is, keyed "<module>/<artifact>", each artifact of the build the mesh has registered for its module, as a
// builder reaches the artifact store (novox/hq issue 721): what a check compares its builds with. Empty when it cannot
// be read, said, and then nothing is said unchanged.
func runningBy(ctx context.Context) map[string]string {
open, err := openStores(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read what this mesh runs, so the check compares nothing: %v\n", err)
// empty-on-error: said above; a check given nothing to compare with says no module unchanged, and all move
return nil
}
defer open.Close()
running, err := open.inventory.RegisteredHeld(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read what this mesh runs, so the check compares nothing: %v\n", err)
// empty-on-error: said above; a check given nothing to compare with says no module unchanged, and all move
return nil
}
address, err := whereABuilderReachesTheStore(ctx, open.inventory)
if err != nil || address == "" {
return running
}
routed := make(map[string]string, len(running))
for key, reference := range running {
routed[key] = catalogue.Rerouted(reference, address)
}
return routed
}
func heldBy(ctx context.Context) map[string]string { func heldBy(ctx context.Context) map[string]string {
open, err := openStores(ctx) open, err := openStores(ctx)
if err != nil { if err != nil {
@@ -1,92 +0,0 @@
package main
import (
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// **Every user's slow consumer is S9's, not the controller's alone** (novox/hq ADR 0357, issue 270). On
// 2026-10-07 the bus closed two clients on the laptop as slow consumers and nothing raised a condition:
// the controller heard only its own connection. Said by the bus module and read back, a user the bus
// closes is one condition naming the module and its machine, raised at the first look and cleared an hour
// after the last closure.
func TestAUserTheBusClosedAsASlowConsumerIsOneConditionNamingItsModuleAndMachine(t *testing.T) {
closed := time.Date(2026, 10, 14, 12, 0, 0, 0, time.UTC)
now := closed.Add(20 * time.Second)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
advisories := link.BusClosures([]link.BusClosure{
{User: "laptop.claude-code", CID: 10, Reason: "Slow Consumer (Write Deadline)", Opened: closed.Add(-time.Hour),
Closed: closed.Add(-10 * time.Minute)},
{User: "laptop.claude-code", CID: 44, Reason: "Slow Consumer (Write Deadline)", Opened: closed.Add(-time.Minute),
Closed: closed},
{User: "anchor.desk", CID: 7, Reason: "Maximum Message Payload Exceeded", Opened: closed.Add(-time.Hour),
Closed: closed},
}, []string{link.AdvisorySlowConsumer, link.AdvisorySlowConsumer, link.AdvisoryConnectionClosed}, closed.Add(-time.Hour))
look := func() {
t.Helper()
f := &signalFacts{now: now, host: "anchor", advisories: advisories}
if err := k.Reconcile(t.Context(), "S9", watchAdvisories(f)); err != nil {
t.Fatal(err)
}
}
look()
slow, found, err := conditions.ReadOne(t.Context(), store, "bus.laptop.claude-code.slow-consumer")
if err != nil || !found {
t.Fatalf("no slow-consumer condition for the user: %v %v", found, err)
}
if slow.Subject.Machine != "laptop" || slow.Observations != 2 || !slow.LastObserved.Equal(closed) {
t.Fatalf("the slow consumer is said on %q, %d closure(s), last %s: %+v", slow.Subject.Machine, slow.Observations,
slow.LastObserved, slow)
}
if !strings.Contains(slow.Summary, "claude-code on laptop") || strings.Contains(slow.Summary, "laptop.claude-code") {
t.Fatalf("the summary does not name the module in words, or carries the dotted user: %q", slow.Summary)
}
if !strings.Contains(slow.Headline, "Claude-code on laptop") {
t.Fatalf("the headline is %q", slow.Headline)
}
cut, found, err := conditions.ReadOne(t.Context(), store, "bus.anchor.desk.connection-closed")
if err != nil || !found || cut.Subject.Machine != "anchor" || !strings.Contains(cut.Summary, "maximum message payload exceeded") {
t.Fatalf("the user cut off for breaking the bus's rules: %v %v %+v", found, err, cut)
}
// An hour after the newest closure without another, both clear.
now = closed.Add(time.Hour + time.Minute)
look()
for _, key := range []string{"bus.laptop.claude-code.slow-consumer", "bus.anchor.desk.connection-closed"} {
if _, found, _ := k.Get(t.Context(), key); found {
t.Fatalf("%s is still open an hour after the last closure", key)
}
}
}
// **A read of the bus module's events that fails does not blind S9** (review of #326): the controller's own
// signals are still said, and the failed read is a condition of its own.
func TestABusModuleReadThatFailsKeepsTheControllersOwnSignals(t *testing.T) {
at := time.Date(2026, 10, 14, 12, 0, 0, 0, time.UTC)
f := &signalFacts{now: at.Add(time.Minute), host: "anchor",
advisories: []link.Advisory{{Kind: link.AdvisorySlowConsumer, ID: "controller", Said: "the controller fell behind",
First: at, Last: at, Count: 1}},
busFaultsErr: errors.New("nats: timeout")}
keys := map[string]bool{}
for _, o := range watchAdvisories(f) {
keys[o.Key()] = true
}
if !keys["bus.controller.slow-consumer"] || !keys["bus.bus-module.bus-faults-unread"] {
t.Fatalf("observed %v: the controller's own slow consumer and the unread events, both", keys)
}
for _, row := range signalsTable {
if row.Row == "S9" && row.needs(f) != nil {
t.Fatal("the S9 row went blind for a read of the bus module's events")
}
}
}
+5 -42
View File
@@ -49,12 +49,10 @@ func assertBusObjects(ctx context.Context, inv *inventory.Inventory, r broker.Ra
// composed from, so a module the mesh grants a consumer's subjects has that consumer waiting. // composed from, so a module the mesh grants a consumer's subjects has that consumer waiting.
// Done on every raise, not only when a credential is issued: every module moved onto this bus // Done on every raise, not only when a credential is issued: every module moved onto this bus
// by the rollout was issued on the old one, and came up with nothing to bind to (2026-09-28). // by the rollout was issued on the old one, and came up with nothing to bind to (2026-09-28).
consumers, records, err := moduleConsumersAndRecords(ctx, inv) consumers, err := moduleConsumers(ctx, inv)
if err != nil { if err != nil {
return nil, err return nil, err
} }
// And a grant of a restricted stream that composes nothing, said on every raise and send (novox/hq ADR 0376 §2).
sayWithheld(records, " ")
// And the work queues of seats that name their caller or their kind, with each holder's worker // And the work queues of seats that name their caller or their kind, with each holder's worker
// (novox/hq ADR 0259 §3): an ask queues until the router takes it, a channel's work until that kind // (novox/hq ADR 0259 §3): an ask queues until the router takes it, a channel's work until that kind
// takes it. // takes it.
@@ -62,19 +60,9 @@ func assertBusObjects(ctx context.Context, inv *inventory.Inventory, r broker.Ra
if err != nil { if err != nil {
return nil, err return nil, err
} }
// And every module's own stream (novox/hq ADR 0363), from the catalogue, before the consumers on it.
modStreams, err := inv.DeclaredStreams(ctx)
if err != nil {
return nil, err
}
// Every one tried, and every failure named: one module's consumer the bus refuses is no reason // Every one tried, and every failure named: one module's consumer the bus refuses is no reason
// the modules after it in the list hear nothing (novox/hq issue 208, where this runs on each send). // the modules after it in the list hear nothing (novox/hq issue 208, where this runs on each send).
var failed []error var failed []error
for _, s := range broker.ModuleStreams(modStreams) {
if err := r.EnsureStream(s); err != nil {
failed = append(failed, fmt.Errorf("the module stream %s: %w", s.Name, err))
}
}
for _, s := range trafficStreams { for _, s := range trafficStreams {
if err := r.EnsureStream(s); err != nil { if err := r.EnsureStream(s); err != nil {
failed = append(failed, fmt.Errorf("the work queue %s: %w", s.Name, err)) failed = append(failed, fmt.Errorf("the work queue %s: %w", s.Name, err))
@@ -85,30 +73,11 @@ func assertBusObjects(ctx context.Context, inv *inventory.Inventory, r broker.Ra
failed = append(failed, fmt.Errorf("the worker %s on %s: %w", c.Name, c.Stream, err)) failed = append(failed, fmt.Errorf("the worker %s on %s: %w", c.Name, c.Stream, err))
} }
} }
restricted := map[string]bool{}
for _, s := range modStreams {
restricted[s.Name] = s.Restricted
}
for _, c := range consumers { for _, c := range consumers {
if err := broker.EnsureHeard(r, c.Consumer, restricted[c.Consumer.Stream]); err != nil { if err := r.EnsureConsumer(c.Consumer); err != nil {
failed = append(failed, fmt.Errorf("how %s on %s hears what it consumes: %w", c.Module, c.Node, err)) failed = append(failed, fmt.Errorf("how %s on %s hears what it consumes: %w", c.Module, c.Node, err))
} }
} }
// **And a consumer no longer granted loses what it reads through** (novox/hq ADR 0363 §6): every consumer
// on a module's stream the composition no longer asks for is removed, on this very send.
if p, prunes := r.(broker.Pruner); prunes && len(modStreams) > 0 {
var wanted []broker.Consumer
for _, c := range consumers {
wanted = append(wanted, c.Consumer)
}
removed, err := broker.PruneHeard(p, modStreams, wanted)
if err != nil {
failed = append(failed, err)
}
for _, gone := range removed {
fmt.Printf(" removed the consumer %s: its module is no longer granted that stream (novox/hq ADR 0363)\n", gone)
}
}
if len(failed) > 0 { if len(failed) > 0 {
return nil, errors.Join(failed...) return nil, errors.Join(failed...)
} }
@@ -167,21 +136,15 @@ func unassertedObservation(err error) conditions.Observation {
// moduleConsumers is every module's durable consumer, from the records the user list is composed from. // moduleConsumers is every module's durable consumer, from the records the user list is composed from.
func moduleConsumers(ctx context.Context, inv *inventory.Inventory) ([]broker.ModuleConsumer, error) { func moduleConsumers(ctx context.Context, inv *inventory.Inventory) ([]broker.ModuleConsumer, error) {
consumers, _, err := moduleConsumersAndRecords(ctx, inv)
return consumers, err
}
// moduleConsumersAndRecords is moduleConsumers with the records they were composed from.
func moduleConsumersAndRecords(ctx context.Context, inv *inventory.Inventory) ([]broker.ModuleConsumer, broker.Records, error) {
records, err := inv.BusRecords(ctx) records, err := inv.BusRecords(ctx)
if err != nil { if err != nil {
return nil, broker.Records{}, err return nil, err
} }
users, err := broker.Users(records) users, err := broker.Users(records)
if err != nil { if err != nil {
return nil, broker.Records{}, err return nil, err
} }
return broker.ConsumersOf(users), records, nil return broker.ConsumersOf(users), nil
} }
// seatTrafficObjects is the work queues and workers of seats that name their caller or their kind, from // seatTrafficObjects is the work queues and workers of seats that name their caller or their kind, from
-5
View File
@@ -73,11 +73,6 @@ func changePlanOf(repository, base, head string, r mergeReach, entries []invento
"(issue 280)", busUpgradeNeeded, name, busUpgradeVerb, strings.Join(e.On, ", "))) "(issue 280)", busUpgradeNeeded, name, busUpgradeVerb, strings.Join(e.On, ", ")))
case name == "nats" || catalogue.ProvidesBus(e.Manifest): case name == "nats" || catalogue.ProvidesBus(e.Manifest):
p.Steps = append(p.Steps, "a planned bus step: the bus is upgraded by `bus upgrade`, never by an ordinary send") p.Steps = append(p.Steps, "a planned bus step: the bus is upgraded by `bus upgrade`, never by an ordinary send")
case waits && len(e.On) > 0 && (u.From == catalogue.FromModule || u.From == catalogue.FromData):
// Judged per machine by the walk, against the build each was last sent (novox/hq ADR 0377).
p.Steps = append(p.Steps, fmt.Sprintf("%s waits for a person: its policy records (%s) — unless on a machine "+
"its move recreates none of its containers and moves none of its data, which the walk judges and then sends",
name, orNone(u.From)))
case waits && len(e.On) > 0: case waits && len(e.On) > 0:
p.Steps = append(p.Steps, fmt.Sprintf("%s waits for a person: its policy records (%s)", name, p.Steps = append(p.Steps, fmt.Sprintf("%s waits for a person: its policy records (%s)", name,
orNone(u.From))) orNone(u.From)))
+30 -1
View File
@@ -47,6 +47,24 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
shelf := catalogue.Shelf{} shelf := catalogue.Shelf{}
faulted := map[string]bool{} faulted := map[string]bool{}
failed := 0 failed := 0
// The list of predefined keys at the root of the repository above each manifest (novox/hq ADR 0348 §7), read
// once per repository: a manifest naming a key is judged against the list its own tree holds.
lists := map[string]catalogue.PredefinedList{}
listOf := func(path string) (catalogue.PredefinedList, error) {
root := catalogue.RepositoryRoot(filepath.Dir(path))
if root == "" {
return nil, nil
}
if list, read := lists[root]; read {
return list, nil
}
list, err := catalogue.ReadPredefined(root)
if err != nil {
return nil, err
}
lists[root] = list
return list, nil
}
for _, path := range paths { for _, path := range paths {
raw, err := os.ReadFile(path) raw, err := os.ReadFile(path)
if err != nil { if err != nil {
@@ -54,7 +72,18 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
failed++ failed++
continue continue
} }
m, err := catalogue.ParseManifest(raw) list, err := listOf(path)
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
continue
}
m, err := catalogue.ParseManifestWith(raw, list)
if err == nil {
// Each template's text read from the tree beside the manifest, and judged as inline content is
// (novox/hq ADR 0349 §2): a missing template is refused by name.
m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(path))
}
if err != nil { if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err) fmt.Fprintf(out, "%s: %v\n", path, err)
failed++ failed++
+13 -77
View File
@@ -125,7 +125,6 @@ func TestAWalksBuildSaysWhetherItReusedOrBuilt(t *testing.T) {
// build as it judges one the walk built. A scratch reference is refused. Needs the store (MESH_TEST_POSTGRES), and // build as it judges one the walk built. A scratch reference is refused. Needs the store (MESH_TEST_POSTGRES), and
// is named in testdata/store-tests. // is named in testdata/store-tests.
func TestAReusedCheckBuildIsRegisteredByItsDigestAndGatedAsABuiltOne(t *testing.T) { func TestAReusedCheckBuildIsRegisteredByItsDigestAndGatedAsABuiltOne(t *testing.T) {
withTheGate(t) // the first-machine gate's own path (novox/hq ADR 0360 keeps it for earlier walks)
g := aGateMesh(t) g := aGateMesh(t)
ctx := t.Context() ctx := t.Context()
inv := g.open.inventory inv := g.open.inventory
@@ -191,90 +190,27 @@ func TestAReusedCheckBuildIsRegisteredByItsDigestAndGatedAsABuiltOne(t *testing.
} }
} }
// A walk's leave names the builds its own pull requests' verdicts made at the head each merge merged — never an // A walk's leave names the builds its own pull requests' verdicts made, and none when they cannot be read: the
// earlier head's — read once per walk, and none when they cannot be read: the builder takes nothing a verdict does // builder takes nothing a verdict does not name (novox/hq ADR 0350 §2).
// not name (novox/hq ADR 0350 §2, issue 703).
func TestAWalksLeaveNamesTheBuildsItsPullRequestsVerdictsMade(t *testing.T) { func TestAWalksLeaveNamesTheBuildsItsPullRequestsVerdictsMade(t *testing.T) {
was := checkedBuildsOf was := checkedBuildsOf
forgetWalkVerdicts() t.Cleanup(func() { checkedBuildsOf = was })
t.Cleanup(func() { checkedBuildsOf = was; forgetWalkVerdicts() }) var asked []link.PullRef
var asked [][]link.PullRef named := []link.CheckedBuild{{Pull: link.PullRef{Owner: "novox", Repo: "shop", Number: 7}, Check: "build-1",
pull := link.PullRef{Owner: "novox", Repo: "shop", Number: 7} Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}}
atHead := link.CheckedBuild{Pull: pull, Check: "build-2", Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:2"}}
earlier := link.CheckedBuild{Pull: pull, Check: "build-1", Head: "aaaaaaaaaaaa", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
checkedBuildsOf = func(_ context.Context, pulls []link.PullRef) ([]link.CheckedBuild, error) { checkedBuildsOf = func(_ context.Context, pulls []link.PullRef) ([]link.CheckedBuild, error) {
asked = append(asked, pulls) asked = pulls
return []link.CheckedBuild{atHead, earlier}, nil return named, nil
} }
pulls := []link.PullRef{pull} pulls := []link.PullRef{{Owner: "novox", Repo: "shop", Number: 7}}
heads := map[link.PullRef]string{pull: "bbbbbbbbbbbb0123456789"} if got := reuseWithVerdicts(t.Context(), &link.ReuseAsk{Pulls: pulls}); !reflect.DeepEqual(got.Builds, named) ||
for range 3 { !reflect.DeepEqual(asked, pulls) {
if got := reuseWithVerdicts(t.Context(), "plan-1", &link.ReuseAsk{Pulls: pulls}, heads); !reflect.DeepEqual(got.Builds, t.Fatalf("the leave names %+v, read for %+v", got.Builds, asked)
[]link.CheckedBuild{atHead}) {
t.Fatalf("the leave names %+v, not only the merged head's build", got.Builds)
}
}
if !reflect.DeepEqual(asked, [][]link.PullRef{pulls}) {
t.Fatalf("the verdicts were read %d times for one walk: %v", len(asked), asked)
}
// A merge whose head was not said takes nothing.
if got := reuseWithVerdicts(t.Context(), "plan-1", &link.ReuseAsk{Pulls: pulls}, nil); got.Builds != nil {
t.Fatalf("a merge of no said head names %+v", got.Builds)
} }
checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) { checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) {
return nil, errors.New("no bus") return nil, errors.New("no bus")
} }
if got := reuseWithVerdicts(t.Context(), "plan-2", &link.ReuseAsk{Pulls: pulls}, heads); got.Builds != nil { if got := reuseWithVerdicts(t.Context(), &link.ReuseAsk{Pulls: pulls}); got.Builds != nil {
t.Fatalf("verdicts that could not be read named %+v", got.Builds) t.Fatalf("verdicts that could not be read named %+v", got.Builds)
} }
} }
// A check is handed only the builds earlier checks of its own pull request at its own head made: another pull
// request's build, or an earlier head's, is never kept (novox/hq issue 703). None when they cannot be read.
func TestACheckIsHandedOnlyItsOwnHeadsBuilds(t *testing.T) {
was := checkedBuildsOf
t.Cleanup(func() { checkedBuildsOf = was })
pull := link.PullRef{Owner: "novox", Repo: "shop", Number: 7}
mine := link.CheckedBuild{Pull: pull, Check: "build-1", Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
earlier := link.CheckedBuild{Pull: pull, Check: "build-0", Head: "aaaaaaaaaaaa", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
another := link.CheckedBuild{Pull: link.PullRef{Owner: "novox", Repo: "shop", Number: 8}, Check: "build-9",
Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
var asked [][]link.PullRef
checkedBuildsOf = func(_ context.Context, pulls []link.PullRef) ([]link.CheckedBuild, error) {
asked = append(asked, pulls)
return []link.CheckedBuild{mine, earlier, another}, nil
}
p := link.PullUpdated{Owner: "novox", Repo: "shop", Number: 7, Commit: "bbbbbbbbbbbb0123456789"}
builds := []link.CheckBuild{{Module: "app"}}
if got := namedForTheCheck(t.Context(), p, builds); !reflect.DeepEqual(got, []link.CheckedBuild{mine}) ||
!reflect.DeepEqual(asked, [][]link.PullRef{{pull}}) {
t.Fatalf("the check is handed %+v, read for %v", got, asked)
}
checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) {
return nil, errors.New("no bus")
}
if got := namedForTheCheck(t.Context(), p, builds); got != nil {
t.Fatalf("verdicts that could not be read named %+v", got)
}
}
func forgetWalkVerdicts() {
walkVerdicts.Lock()
defer walkVerdicts.Unlock()
walkVerdicts.read = map[string]verdictsRead{}
}
// A walk's merged heads are the forge's word for each merge; a merge announced without its head, or a pull request
// merged at two heads in one walk, has none, and its walk takes no check's build (novox/hq issue 703).
func TestAWalksMergedHeadsAreTheForgesWord(t *testing.T) {
merge := func(number int, head string) inventory.PlanMerge {
return inventory.PlanMerge{Repository: "novox/shop", Number: number, Head: head}
}
p := &inventory.Plan{Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
merge(7, "aaaaaaaaaaaa1"), merge(8, ""), merge(9, "bbbbbbbbbbbb"), merge(9, "cccccccccccc")}}}
heads := mergedHeadsOf(p)
shop := func(n int) link.PullRef { return link.PullRef{Owner: "novox", Repo: "shop", Number: n} }
if heads[shop(7)] != "aaaaaaaaaaaa1" || heads[shop(8)] != "" || heads[shop(9)] != "" {
t.Fatalf("merged heads %v", heads)
}
}
-140
View File
@@ -1,140 +0,0 @@
package main
import (
"context"
"fmt"
"regexp"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A delivery group's member is checked beside the other members' heads** (novox/hq ADR 0382, issue 867).
//
// A mesh-host change and a mesh-controller change that need each other could not both pass their own merge checks:
// each was read beside the other's repository at the commit the mesh runs, which holds neither change. So the
// controller asks mesh-delivery's `groups` once for every pull request's check it asks (§1), and names, in the ask,
// the group's other open pull requests into a base of the same name, in a repository the build seat reads beside the
// check — mesh-lab never — each with its whole head. The build seat reads those repositories at the sibling's merge (§2).
// A pull request for which mesh-delivery answered and named no sibling is asked exactly as before (§7).
// wholeCommit is a commit as the forge says a head: forty hexadecimal characters.
var wholeCommit = regexp.MustCompile(`^[0-9a-f]{40}$`)
// besideRepositories are the repositories (lower case, owner/repo) a check reads beside its own, by the directory each
// is found under — the core's, as besideOf clones them, and mesh-sdk beside the controller; never mesh-lab, which is
// always read at its trunk.
func besideRepositories(entries []inventory.Entry) map[string]string {
read := map[string]string{}
for _, e := range entries {
dir, core := coreModules[e.Manifest.Module]
if !core || e.Provided || e.Source.Repository == "" {
continue
}
read[nameOf(e.Source.Repository)] = dir
if dir == "mesh-controller" && e.Source.Seat != "" {
read[nameOf(siblingOf(e.Source.Repository, "mesh-sdk"))] = "mesh-sdk"
}
}
delete(read, "")
return read
}
// nameOf is a recorded repository as owner/repo in lower case, empty when it names no owner.
func nameOf(repository string) string {
owner, repo := sourceOwner(repository), sourceName(repository)
if owner == "" || repo == "" {
return ""
}
return strings.ToLower(owner + "/" + repo)
}
// siblingsFor is the pull request's siblings, as mesh-delivery's groups say them (ADR 0382 §1): the other members of
// the group its head branch names that are open, unmerged and into a base of its own base's name, in a repository the
// check reads beside it (beside: the directories the ask already names). The second answer is why they could not be
// learned, empty when they could: the group's members carry no head or base — a mesh-delivery from before the rule.
func siblingsFor(p link.PullUpdated, entries []inventory.Entry, beside map[string]link.CheckedOut,
lines []groupsLine) ([]link.CheckSibling, string) {
own := strings.ToLower(p.Owner + "/" + p.Repo)
var group *groupsLine
if p.Head == "" {
// A recheck (mesh-delivery's `recheck` through `delivery check`) carries no head branch name: the group is the one
// whose members hold this very pull request, by repository and number.
learnable := false
for i := range lines {
for _, m := range lines[i].Details {
learnable = true
if p.Number != 0 && m.Number == p.Number && strings.ToLower(m.Repository) == own {
group = &lines[i]
}
}
}
if group == nil {
for _, l := range lines {
if len(l.Details) == 0 && len(l.Members) > 0 && !learnable {
return nil, "mesh-delivery named the groups' members without their pull requests (a mesh-delivery from before novox/hq ADR 0382): " +
"the group of a pull request announced without its head branch cannot be found"
}
}
return nil, "" // a lone pull request
}
} else {
for i := range lines {
if lines[i].ID == p.Head {
group = &lines[i]
}
}
if group == nil {
return nil, ""
}
}
if len(group.Details) == 0 && len(group.Members) > 0 {
return nil, "mesh-delivery named the group's members without their heads and bases (a mesh-delivery from before novox/hq ADR 0382)"
}
read := besideRepositories(entries)
var siblings []link.CheckSibling
for _, m := range group.Details {
repository := strings.ToLower(m.Repository)
if repository == own {
continue
}
if !m.Open {
continue
}
if m.Head == "" || m.Base == "" {
return nil, "mesh-delivery named " + m.ID + " without its head or base (a mesh-delivery from before novox/hq ADR 0382)"
}
dir, readBeside := read[repository]
if m.Base != p.Base || !readBeside || dir == "mesh-lab" {
continue
}
if _, there := beside[dir]; !there {
continue
}
head := strings.ToLower(m.Head)
if !wholeCommit.MatchString(head) {
return nil, fmt.Sprintf("mesh-delivery named %s with a head that is not a whole commit (%q)", m.ID, m.Head)
}
siblings = append(siblings, link.CheckSibling{Repository: m.Repository, Number: m.Number, Head: head, Base: m.Base, Dir: dir})
}
sort.Slice(siblings, func(i, j int) bool { return siblings[i].Dir < siblings[j].Dir })
return siblings, ""
}
// withSiblings asks mesh-delivery's `groups` once and says in the request the pull request's siblings, or why they
// could not be learned (ADR 0382 §1): a check never waits on mesh-delivery past groupAskWithin and never fails for it,
// so a fix to mesh-delivery is still checked without it.
func withSiblings(ctx context.Context, request *link.BuildRequest, p link.PullUpdated, entries []inventory.Entry) {
if request.Check == nil {
return
}
lines, err := groupsAsked(ctx, deliverySeatHeld(entries))
if err != nil {
request.Check.SiblingsUnknown = fmt.Sprintf("mesh-delivery did not say the group's pull requests (%v): "+
"the check reads no siblings", err)
return
}
request.Check.Siblings, request.Check.SiblingsUnknown = siblingsFor(p, entries, request.Check.Beside, lines)
}
-248
View File
@@ -1,248 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A delivery group's member is checked beside the other members' heads** (novox/hq ADR 0382 §1, §7, issue 867): the
// controller asks mesh-delivery's `groups` once for every check it asks, and names in the ask the group's other open
// pull requests into a base of the same name, in a repository the build seat reads beside it — mesh-lab never.
const (
headHost = "1111111111111111111111111111111111111111"
headController = "2222222222222222222222222222222222222222"
headSDK = "3333333333333333333333333333333333333333"
headLab = "4444444444444444444444444444444444444444"
headHq = "5555555555555555555555555555555555555555"
)
func theCoreOnTheGitSeat() []inventory.Entry {
onSeat := func(module, repository string) inventory.Entry {
e := fromRepo(module, repository, "")
e.Source.Seat = "git"
return e
}
nats := onSeat("nats", "novox/mesh-catalog")
nats.Source.Path = "modules/nats"
held := catalogue.Manifest{Module: "mesh-delivery", Claims: []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}}
return []inventory.Entry{onSeat("mesh-controller", "novox/mesh-controller"), onSeat("mesh-host", "novox/mesh-host"),
onSeat("node-tools", "novox/mesh-tools"), nats, {Manifest: held, On: []string{"anchor"}}}
}
func besideTheCore() map[string]link.CheckedOut {
return map[string]link.CheckedOut{"mesh-controller": {Repository: "c"}, "mesh-host": {Repository: "h"},
"mesh-tools": {Repository: "t"}, "mesh-catalog": {Repository: "k"}, "mesh-sdk": {Repository: "s"}, "mesh-lab": {Repository: "l"}}
}
func member(repository string, number int, head, base string, open bool) memberLine {
return memberLine{ID: repository + "@" + head[:12], Repository: repository, State: "checked", Number: number, Head: head,
Base: base, Open: open}
}
func group(id string, members ...memberLine) groupsLine {
line := groupsLine{ID: id, Details: members}
for _, m := range members {
line.Members = append(line.Members, m.ID+" ("+m.State+")")
}
return line
}
func TestASiblingIsTheGroupsOtherOpenPullRequestIntoABaseOfTheSameName(t *testing.T) {
entries := theCoreOnTheGitSeat()
pull := link.PullUpdated{Owner: "novox", Repo: "mesh-host", Number: 114, Base: "main", Head: "feat/x", Commit: headHost}
for _, c := range []struct {
what string
lines []groupsLine
want []string
unknown string
}{
{"the other member, open, same base", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))},
[]string{"mesh-controller=novox/mesh-controller#346@" + headController + " onto main"}, ""},
{"a member into a base of another name is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "release", true))},
nil, ""},
{"a member not open (merged or closed) is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", false))},
nil, ""},
{"mesh-lab is never one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-lab", 5, headLab, "main", true))},
nil, ""},
{"a repository the check reads nothing of is not one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/hq", 9, headHq, "main", true))},
nil, ""},
{"mesh-sdk, read beside the controller, is one", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-sdk", 7, headSDK, "main", true))},
[]string{"mesh-sdk=novox/mesh-sdk#7@" + headSDK + " onto main"}, ""},
{"another group is none of its business", []groupsLine{group("feat/y",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "main", true))},
nil, ""},
{"a group of its own alone has none", []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true))},
nil, ""},
{"an older mesh-delivery names no heads", []groupsLine{{ID: "feat/x",
Members: []string{"novox/mesh-host@111111111111 (checked)", "novox/mesh-controller@222222222222 (checked)"}}},
nil, "without their heads and bases"},
{"a member without its base", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController, "", true))},
nil, "without its head or base"},
{"a head that is no commit", []groupsLine{group("feat/x",
member("novox/mesh-host", 114, headHost, "main", true), member("novox/mesh-controller", 346, headController[:12]+"zzzzzzzzzzzzzzzzzzzzzzzzzzzz", "main", true))},
nil, "not a whole commit"},
} {
got, unknown := siblingsFor(pull, entries, besideTheCore(), c.lines)
var said []string
for _, s := range got {
said = append(said, s.Dir+"="+s.Repository+"#"+itoa(s.Number)+"@"+s.Head+" onto "+s.Base)
}
if strings.Join(said, ";") != strings.Join(c.want, ";") {
t.Errorf("%s: siblings %v, wanted %v", c.what, said, c.want)
}
if (c.unknown == "") != (unknown == "") || !strings.Contains(unknown, c.unknown) {
t.Errorf("%s: could not learn %q, wanted %q", c.what, unknown, c.unknown)
}
}
// A pull request into a release branch has siblings into the release branch of the same name (ADR 0358 §1).
release := pull
release.Base = "feat/57"
lines := []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "feat/57", true),
member("novox/mesh-controller", 346, headController, "feat/57", true), member("novox/mesh-tools", 8, headLab, "main", true))}
if got, _ := siblingsFor(release, entries, besideTheCore(), lines); len(got) != 1 || got[0].Dir != "mesh-controller" {
t.Errorf("into a release branch: %+v", got)
}
// A recheck announces the pull request without its head branch: its group is the one holding it (repository, number).
pull.Head = ""
got, unknown := siblingsFor(pull, entries, besideTheCore(), lines)
if len(got) != 1 || got[0].Dir != "mesh-tools" || unknown != "" {
t.Errorf("a recheck of a group member: %+v %q", got, unknown)
}
// A pull request in no group, rechecked, is alone: no sibling, and nothing said.
pull.Number = 999
if got, unknown := siblingsFor(pull, entries, besideTheCore(), lines); got != nil || unknown != "" {
t.Errorf("a lone pull request: %+v %q", got, unknown)
}
if got, unknown := siblingsFor(pull, entries, besideTheCore(), nil); got != nil || unknown != "" {
t.Errorf("no groups at all: %+v %q", got, unknown)
}
// A mesh-delivery that names members without their pull requests cannot find the group of a recheck: it says so.
older := []groupsLine{{ID: "feat/x", Members: []string{"novox/mesh-host@111111111111 (checked)"}}}
pull.Number = 114
if got, unknown := siblingsFor(pull, entries, besideTheCore(), older); got != nil || unknown == "" {
t.Errorf("an older mesh-delivery: %+v %q", got, unknown)
}
}
func itoa(n int) string { b, _ := json.Marshal(n); return string(b) }
// Every check asks mesh-delivery's `groups` once; the ask names the sibling's repository, head and base.
func TestEveryCheckAsksMeshDeliveryOnceAndNamesItsSiblings(t *testing.T) {
entries := theCoreOnTheGitSeat()
asked := 0
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
groupsAsked = func(_ context.Context, held bool) ([]groupsLine, error) {
asked++
if !held {
t.Error("the seat is held, yet it was asked as not held")
}
return []groupsLine{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true),
member("novox/mesh-controller", 346, headController, "main", true))}, nil
}
request := link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main", Beside: besideTheCore()}}
withSiblings(t.Context(), &request, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
if asked != 1 {
t.Fatalf("mesh-delivery was asked %d times for one check", asked)
}
want := link.CheckSibling{Repository: "novox/mesh-controller", Number: 346, Head: headController, Base: "main", Dir: "mesh-controller"}
if got := request.Check.Siblings; len(got) != 1 || got[0] != want || request.Check.SiblingsUnknown != "" {
t.Fatalf("the ask names siblings %+v, unknown %q", got, request.Check.SiblingsUnknown)
}
body, _ := json.Marshal(request.Check)
if !strings.Contains(string(body), `"siblings":[{"repository":"novox/mesh-controller","number":346,"head":"`+headController+
`","base":"main","dir":"mesh-controller"}]`) {
t.Fatalf("on the wire: %s", body)
}
}
// When mesh-delivery does not answer in time (or is not held, or answered without the fields), the check reads no
// sibling and its request says it could not learn them, so a fix to mesh-delivery is still checked without it.
func TestWhenMeshDeliveryDoesNotAnswerTheCheckReadsNoSiblingAndSaysSo(t *testing.T) {
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
for _, c := range []struct {
what string
err error
held bool
}{{"it did not answer", context.DeadlineExceeded, true}, {"nothing serves it", errors.New("nothing serves mesh-delivery.groups"), true},
{"no holder on record", errors.New("no holder of mesh-delivery is on record to say it"), false}} {
entries := theCoreOnTheGitSeat()
if !c.held {
entries = entries[:4]
}
groupsAsked = func(_ context.Context, held bool) ([]groupsLine, error) {
if held != c.held {
t.Errorf("%s: held %v", c.what, held)
}
return nil, c.err
}
request := link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main", Beside: besideTheCore()}}
withSiblings(t.Context(), &request, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
if len(request.Check.Siblings) != 0 || !strings.Contains(request.Check.SiblingsUnknown, c.err.Error()) ||
!strings.Contains(request.Check.SiblingsUnknown, "reads no siblings") {
t.Errorf("%s: siblings %+v, unknown %q", c.what, request.Check.Siblings, request.Check.SiblingsUnknown)
}
}
}
// **A pull request for which mesh-delivery answered and named no sibling is asked exactly as before** (ADR 0382 §7):
// byte for byte the request it was without the rule.
func TestAPullRequestWithNoSiblingIsAskedByteForByteAsBefore(t *testing.T) {
entries := theCoreOnTheGitSeat()
was := groupsAsked
t.Cleanup(func() { groupsAsked = was })
for _, answer := range [][]groupsLine{nil, {group("feat/other", member("novox/mesh-controller", 346, headController, "main", true))},
{group("feat/x", member("novox/mesh-host", 114, headHost, "main", true), member("novox/hq", 3, headHq, "main", true))}} {
groupsAsked = func(context.Context, bool) ([]groupsLine, error) { return answer, nil }
build := func() link.BuildRequest {
return link.BuildRequest{ID: "build-1", Repository: "r", Ref: headHost, Check: &link.CheckRequest{Owner: "novox",
Repo: "mesh-host", Number: 114, Base: "main", Beside: besideTheCore(), Modules: []string{"mesh-host"}, Judge: "validator"}}
}
before, _ := json.Marshal(build())
after := build()
withSiblings(t.Context(), &after, link.PullUpdated{Owner: "novox", Repo: "mesh-host", Base: "main", Head: "feat/x"}, entries)
got, _ := json.Marshal(after)
if string(before) != string(got) {
t.Fatalf("the request changed:\nbefore %s\nafter %s", before, got)
}
}
}
// What the build seat's verdict says of its siblings reaches the forge's holder in the controller's `checked`.
func TestTheVerdictCarriesTheSiblingsAndWhereTheJudgeCameFrom(t *testing.T) {
read := []link.CheckSibling{{Repository: "novox/mesh-controller", Number: 346, Head: headController, Base: "main", BaseCommit: headHost}}
c := checkedOf(link.BuildResult{ID: "b", Ref: headHost, Checked: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114},
Check: &link.CheckOutcome{Verdict: "pass", Summary: "ok", Siblings: read, JudgeFrom: "sibling novox/mesh-controller#346 merge abc",
Validator: "the change's own validator"}})
if len(c.Siblings) != 1 || c.Siblings[0] != read[0] || c.JudgeFrom == "" || c.Validator == "" {
t.Fatalf("the verdict says %+v", c)
}
body, _ := json.Marshal(c)
for _, want := range []string{`"siblings":[{"repository":"novox/mesh-controller","number":346,"head":"` + headController +
`","base":"main","base_commit":"` + headHost + `"}]`, `"judge_from":"sibling novox/mesh-controller#346 merge abc"`} {
if !strings.Contains(string(body), want) {
t.Fatalf("the verdict lacks %s:\n%s", want, body)
}
}
unknown := checkedOf(link.BuildResult{ID: "b", Check: &link.CheckOutcome{Verdict: "pass", SiblingsUnknown: "mesh-delivery did not answer"}})
if body, _ := json.Marshal(unknown); !strings.Contains(string(body), `"siblings_unknown":"mesh-delivery did not answer"`) ||
strings.Contains(string(body), `"siblings":`) {
t.Fatalf("the verdict that could not learn them says %s", body)
}
}
+22 -193
View File
@@ -3,7 +3,6 @@ package main
import ( import (
"context" "context"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"path" "path"
"regexp" "regexp"
@@ -180,8 +179,6 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
return err return err
} }
request.Check.Plan = &plan request.Check.Plan = &plan
// **The group's other open pull requests are read beside it** (novox/hq ADR 0382 §1).
withSiblings(ctx, &request, p, entries)
seat := buildSeatHeld(ctx) seat := buildSeatHeld(ctx)
ask, err := askOverOn(seat) ask, err := askOverOn(seat)
if err != nil { if err != nil {
@@ -191,8 +188,6 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
if err := ask.Ask(ctx, request); err != nil { if err := ask.Ask(ctx, request); err != nil {
return err return err
} }
// A new head ends the check of the old one (novox/hq issue 726).
endingSupersededChecks(ctx, seat, p, request.ID, f.oneShot)
what := "its own merge-check.sh alone: " + noModuleTouched what := "its own merge-check.sh alone: " + noModuleTouched
if scope.gated() { if scope.gated() {
what = fmt.Sprintf("the gate over %s (a merge would build %d module(s) in %d tier(s))", what = fmt.Sprintf("the gate over %s (a merge would build %d module(s) in %d tier(s))",
@@ -253,16 +248,31 @@ func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scop
if err != nil { if err != nil {
return link.BuildRequest{}, err return link.BuildRequest{}, err
} }
// **A pull request into a release branch is judged by that branch** (novox/hq ADR 0358): asked of the forge once. // Beside it, at what the mesh runs: each core repository by the module the mesh builds from it.
release, err := releaseBranchOf(ctx, p, scope, entries) beside := map[string]link.CheckedOut{}
for _, e := range entries {
dir, core := coreModules[e.Manifest.Module]
if !core || e.Provided || e.Source.Repository == "" {
continue
}
url, err := clone(e.Source)
if err != nil { if err != nil {
return link.BuildRequest{}, err return link.BuildRequest{}, err
} }
beside, err := besideOf(entries, func(module string) string { return current[module].Commit }, clone, release) refs := besideRefs(dir, current[e.Manifest.Module].Commit)
if err != nil { beside[dir] = link.CheckedOut{Repository: url, Ref: refs[dir]}
return link.BuildRequest{}, err if dir == "mesh-controller" {
beside["mesh-controller-main"] = link.CheckedOut{Repository: url, Ref: refs["mesh-controller-main"]}
if e.Source.Seat != "" {
for _, sibling := range []string{"mesh-lab", "mesh-sdk"} {
if url, err := clone(inventory.Source{Seat: e.Source.Seat, Repository: siblingOf(e.Source.Repository,
sibling)}); err == nil {
beside[sibling] = link.CheckedOut{Repository: url, Ref: refs[sibling]}
}
}
}
}
} }
builds := checkBuildsOf(p, scope, entries, clone)
return link.BuildRequest{ return link.BuildRequest{
ID: link.NewBuildID(time.Now()), ID: link.NewBuildID(time.Now()),
Repository: repository, Repository: repository,
@@ -272,188 +282,10 @@ func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scop
Source: sourceOnSeat(source), Source: sourceOnSeat(source),
Check: &link.CheckRequest{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Base: p.Base, Check: &link.CheckRequest{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Base: p.Base,
Paths: p.Paths, Beside: beside, Modules: scope.Modules, Dependents: scope.Dependents, New: scope.New, Paths: p.Paths, Beside: beside, Modules: scope.Modules, Dependents: scope.Dependents, New: scope.New,
Manifests: scope.Manifests, Judge: scope.Judge, Builds: builds, Named: namedForTheCheck(ctx, p, builds), Manifests: scope.Manifests, Judge: scope.Judge, Builds: checkBuildsOf(p, scope, entries, clone)},
Release: release, Running: runningBy(ctx)},
}, nil }, nil
} }
// besideOf is what a check reads beside its own repository, by the directory it is found under: each core repository
// by the module the mesh builds from it, at besideRefs — or, for a pull request into a release branch (novox/hq ADR
// 0358 §3), at its branch of that name where it has one and at its trunk otherwise, the controller's main kept for
// its judge, and mesh-lab at its trunk whatever the base: its replays run with the container runtime's socket. running
// is the commit the mesh runs of a module.
func besideOf(entries []inventory.Entry, running func(module string) string,
clone func(inventory.Source) (string, error), release string) (map[string]link.CheckedOut, error) {
beside := map[string]link.CheckedOut{}
for _, e := range entries {
dir, core := coreModules[e.Manifest.Module]
if !core || e.Provided || e.Source.Repository == "" {
continue
}
url, err := clone(e.Source)
if err != nil {
return nil, err
}
refs := besideRefs(dir, running(e.Manifest.Module))
at := func(dir, trunk string) link.CheckedOut {
if release == "" {
return link.CheckedOut{Repository: url, Ref: refs[dir]}
}
if trunk == "" {
trunk = "main"
}
return link.CheckedOut{Repository: url, Ref: release, Else: trunk}
}
beside[dir] = at(dir, followedBranch(e.Source.Ref))
if dir == "mesh-controller" {
// The controller's main stays its main: it judges only a check the running controller predates.
beside["mesh-controller-main"] = link.CheckedOut{Repository: url, Ref: refs["mesh-controller-main"]}
if release != "" {
// mesh-controller has the release branch: that is what makes it one (ADR 0358 §1).
beside[dir] = link.CheckedOut{Repository: url, Ref: release}
}
if e.Source.Seat != "" {
for _, sibling := range []string{"mesh-lab", "mesh-sdk"} {
if url, err := clone(inventory.Source{Seat: e.Source.Seat, Repository: siblingOf(e.Source.Repository,
sibling)}); err == nil {
b := link.CheckedOut{Repository: url, Ref: refs[sibling]}
// **mesh-lab is read at its trunk, always** (ADR 0358): its replays run with the container
// runtime's socket, so they are only ever reviewed code — never a branch anyone may create.
if release != "" && sibling != "mesh-lab" {
b = link.CheckedOut{Repository: url, Ref: release, Else: "main"}
}
beside[sibling] = b
}
}
}
}
}
return beside, nil
}
// releaseBranchOf is the release branch a pull request merges into (novox/hq ADR 0358 §1): its base, when that is not
// its repository's trunk and mesh-controller has a branch of that name; empty otherwise, and the pull request is
// checked as before it. **A base that is a trunk asks nothing**: the branch a module built from the repository follows
// (main when it records none), or mesh-controller's own trunk. Any other base is asked of the forge once, and an
// answer it cannot give is an error, never read as "no release branch".
func releaseBranchOf(ctx context.Context, p link.PullUpdated, scope checkScope, entries []inventory.Entry) (string, error) {
base := p.Base
if base == "" {
return "", nil
}
if scope.From != nil && base == trunkOf(scope.From.Source) {
return "", nil
}
var controller *inventory.Entry
for i, e := range entries {
if e.Manifest.Module == "mesh-controller" && !e.Provided && e.Source.Repository != "" {
controller = &entries[i]
break
}
}
if controller == nil || base == trunkOf(controller.Source) {
return "", nil
}
if !tryRef.MatchString(base) || strings.Contains(base, "..") {
return "", nil // no branch the forge could hold: checked as before
}
owner, repo := sourceOwner(controller.Source.Repository), sourceName(controller.Source.Repository)
if owner == "" || repo == "" {
return "", nil
}
has, err := forgeHasBranch(ctx, owner, repo, base)
if err != nil {
return "", fmt.Errorf("whether %s/%s#%d merges into a release branch cannot be told — the forge did not say "+
"whether %s/%s has a branch %s (novox/hq ADR 0358): %w", p.Owner, p.Repo, p.Number, owner, repo, base, err)
}
if !has {
return "", nil
}
return base, nil
}
// trunkOf is the branch a module's source follows: main when it records none, or records a commit.
func trunkOf(s inventory.Source) string {
if t := strings.TrimSpace(followedBranch(s.Ref)); t != "" {
return t
}
return "main"
}
// sourceName is the name of a recorded repository, a path on the git seat or a URL: novox/mesh-host → mesh-host.
func sourceName(repository string) string {
parts := strings.Split(strings.Trim(strings.TrimSuffix(repository, ".git"), "/"), "/")
if len(parts) < 2 {
return ""
}
return parts[len(parts)-1]
}
// forgeHasBranch asks the forge, through its module's tools on the bus, whether a repository has a branch. A
// variable so a test needs no forge.
var forgeHasBranch = func(ctx context.Context, owner, repo, branch string) (bool, error) {
js, err := aBus()
if err != nil {
return false, err
}
defer js.Close()
raw, _ := json.Marshal(map[string]any{"path": "/repos/" + owner + "/" + repo + "/branches/" + branch})
answer, err := link.Ask(ctx, link.OverNATS{Conn: js.Conn()}, "gitea", "gitea_api", raw, 30*time.Second)
if err != nil {
return false, err
}
return branchAnswered(answer, branch)
}
// branchAnswered reads the forge's answer about a branch: a 404 is "no such branch"; an answer naming the branch is
// "it has it"; anything else — another error, an unreadable answer, one naming no branch or another — is an error,
// never read as "no release branch".
func branchAnswered(answer link.Answer, branch string) (bool, error) {
if answer.Error != "" {
if strings.Contains(answer.Error, " 404 ") {
return false, nil
}
return false, errors.New(answer.Error)
}
var found struct {
Result struct {
Name string `json:"name"`
} `json:"result"`
}
if err := json.Unmarshal(answer.Result, &found); err != nil {
return false, fmt.Errorf("the forge's answer about branch %s cannot be read: %w", branch, err)
}
if found.Result.Name != branch {
return false, fmt.Errorf("the forge answered about branch %s naming %q", branch, found.Result.Name)
}
return true, nil
}
// namedForTheCheck are the builds earlier checks of the same pull request at the same head named, read once for the
// check (novox/hq issue 703): asked again because only the trunk moved, the check keeps what they built of a key
// still the same. Another pull request's build, or one of an earlier head, is never kept. None when they cannot be
// read, and the check builds, as before.
func namedForTheCheck(ctx context.Context, p link.PullUpdated, builds []link.CheckBuild) []link.CheckedBuild {
if len(builds) == 0 || p.Commit == "" {
return nil
}
pull := link.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number}
read, err := checkedBuildsOf(ctx, []link.PullRef{pull})
if err != nil {
fmt.Printf(" the earlier checks' builds of %s/%s#%d could not be read, so the check builds: %v\n",
p.Owner, p.Repo, p.Number, err)
// empty-on-error: said above; a check handed no named build builds every module, as before issue 703
return nil
}
var named []link.CheckedBuild
for _, b := range read {
if b.Pull == pull && link.SameHead(b.Head, p.Commit) {
named = append(named, b)
}
}
return named
}
// checkBuildsOf are the modules a merge of the change would build, in the walk's order, as the check builds them // checkBuildsOf are the modules a merge of the change would build, in the walk's order, as the check builds them
// for the walk to take (novox/hq ADR 0350 §1): a module built from the repository checked, at the branch it merges // for the walk to take (novox/hq ADR 0350 §1): a module built from the repository checked, at the branch it merges
// into, from the merge the check makes; any other — a module standing on one — from its own repository at the // into, from the merge the check makes; any other — a module standing on one — from its own repository at the
@@ -600,9 +432,6 @@ func checkedOf(result link.BuildResult) link.Checked {
// repositories read beside it. // repositories read beside it.
c.Trunk, c.Beside, c.Took = result.Check.Trunk, result.Check.Beside, result.Check.Took c.Trunk, c.Beside, c.Took = result.Check.Trunk, result.Check.Beside, result.Check.Took
c.Built = result.Check.Built c.Built = result.Check.Built
c.Release, c.JudgedBy, c.BesideAt = result.Check.Release, result.Check.JudgedBy, result.Check.BesideAt
c.Validator = result.Check.Validator
c.Siblings, c.SiblingsUnknown, c.JudgeFrom = result.Check.Siblings, result.Check.SiblingsUnknown, result.Check.JudgeFrom
if c.Gate == nil { if c.Gate == nil {
// A build seat from before the layers: its verdict is the gate's. // A build seat from before the layers: its verdict is the gate's.
c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary} c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
-117
View File
@@ -1,117 +0,0 @@
package main
import (
"context"
"fmt"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// A dead letter from a restricted stream ages out with its stream (novox/hq ADR 0376 §3, §6).
//
// DEAD_LETTERS lets such a message go when its stream's maximum age runs out, so a consumer's `max-deliveries`
// condition can clear with nobody having delivered it again or dropped it. That is said: the watchdogs remember,
// from one look to the next, when each restricted dead letter ages out, count those gone once their time passed,
// say each in the controller's log as it goes, and clear the condition with words saying how many aged out
// undelivered and from which stream.
// agingTally is what the watchdogs remember of restricted dead letters between looks.
type agingTally struct {
// seen is each consumer's restricted dead letters at the last look, by id, with when each ages out.
seen map[string]map[uint64]time.Time
// agedOut is how many of each consumer's aged out undelivered while its condition was open.
agedOut map[string]int
}
// look compares this look's restricted dead letters with the last: newly is how many of each consumer's aged out
// since, and cleared how many aged out in all for each consumer DEAD_LETTERS no longer holds anything of, when any
// did. held is every consumer's count this look; aging its restricted dead letters.
func (t *agingTally) look(now time.Time, held map[string]int, aging map[string][]link.Aging) (newly, cleared map[string]int) {
if t.seen == nil {
t.seen, t.agedOut = map[string]map[uint64]time.Time{}, map[string]int{}
}
newly, cleared = map[string]int{}, map[string]int{}
for key, before := range t.seen {
still := map[uint64]bool{}
for _, a := range aging[key] {
still[a.ID] = true
}
for id, agesOut := range before {
if !still[id] && !agesOut.After(now) {
newly[key]++
}
}
if newly[key] == 0 {
delete(newly, key)
}
t.agedOut[key] += newly[key]
}
for key, n := range t.agedOut {
if held[key] > 0 {
continue
}
if n > 0 {
cleared[key] = n
}
delete(t.agedOut, key)
}
t.seen = map[string]map[uint64]time.Time{}
for key, letters := range aging {
t.seen[key] = map[uint64]time.Time{}
for _, a := range letters {
t.seen[key][a.ID] = a.AgesOut
}
}
return newly, cleared
}
// agedOutResolved is the line a consumer's condition clears with when messages it gave up on aged out undelivered.
func agedOutResolved(key string, n int) string {
stream, _, _ := strings.Cut(key, ".")
messages := "a message"
if n > 1 {
messages = fmt.Sprintf("%d messages", n)
}
return fmt.Sprintf("Resolved: %s of the stream %s aged out undelivered", messages, stream)
}
// sayAgedOut says, in the log, what aged out since the last look, and clears the condition of every consumer whose
// dead letters are all gone with the words saying how many aged out. Before the rows are reconciled, so the
// condition is cleared with these words and not its own.
func (w *watchdogs) sayAgedOut(ctx context.Context, f *signalFacts) []string {
if f.advisoriesErr != nil || f.deadLettersAging == nil {
return nil
}
newly, cleared := w.aging.look(f.now, f.deadLetters, f.deadLettersAging)
var problems []string
for _, key := range countedKeys(newly) {
stream, consumer, _ := strings.Cut(key, ".")
fmt.Printf("%d message(s) %s gave up on aged out of %s undelivered: %s keeps them only for its maximum age "+
"(novox/hq ADR 0376)\n", newly[key], link.ConsumerInWords(stream, consumer), "DEAD_LETTERS", stream)
}
for _, key := range countedKeys(cleared) {
stream, consumer, _ := strings.Cut(key, ".")
fmt.Printf("%s no longer has a message kept: %d aged out of %s undelivered, so its condition clears "+
"(novox/hq ADR 0376)\n", link.ConsumerInWords(stream, consumer), cleared[key], stream)
if _, err := w.keeper.ClearSaying(ctx, conditions.Key(conditions.ScopeBus, key, link.AdvisoryMaxDeliveries),
fmt.Sprintf("its %d dead letter(s) of %s aged out undelivered", cleared[key], stream),
agedOutResolved(key, cleared[key])); err != nil {
problems = append(problems, err.Error())
}
}
return problems
}
// countedKeys is a count's keys in order.
func countedKeys(m map[string]int) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
@@ -1,69 +0,0 @@
package main
import (
"context"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// **A dead letter that ages out clears its condition and says so** (novox/hq ADR 0376 §6), on a fake clock: while
// DEAD_LETTERS holds a restricted stream's messages for a consumer its max-deliveries condition is open; once they
// have aged out it clears, with words saying how many aged out undelivered and from which stream. One dropped before
// its age is not counted as aged out, and a consumer of EVENTS clears as before.
func TestADeadLetterThatAgesOutClearsItsConditionAndSaysSo(t *testing.T) {
now := time.Date(2026, 10, 15, 12, 0, 0, 0, time.UTC)
store := conditions.NewInMemory()
told := &conditions.Told{}
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store, Teller: told,
Now: func() time.Time { return now }})
w := &watchdogs{keeper: k, started: now.Add(-time.Hour)}
look := func(at time.Time, held map[string]int, aging map[string][]link.Aging) {
f := calm(at)
f.deadLetters, f.deadLettersAging = held, aging
w.see(t.Context(), f)
}
mail := "MAIL.ace_mailer"
look(now, map[string]int{mail: 3, "EVENTS.media_sonarr": 1}, map[string][]link.Aging{mail: {
{ID: 1, AgesOut: now.Add(time.Hour)}, {ID: 2, AgesOut: now.Add(2 * time.Hour)}, {ID: 3, AgesOut: now.Add(3 * time.Hour)}}})
if open, _ := k.Open(t.Context()); len(open) != 2 {
t.Fatalf("open: %+v", open)
}
// One dropped by a person before its age: not aged out.
look(now.Add(30*time.Minute), map[string]int{mail: 2, "EVENTS.media_sonarr": 1}, map[string][]link.Aging{mail: {
{ID: 2, AgesOut: now.Add(2 * time.Hour)}, {ID: 3, AgesOut: now.Add(3 * time.Hour)}}})
look(now.Add(150*time.Minute), map[string]int{mail: 1}, map[string][]link.Aging{mail: {{ID: 3, AgesOut: now.Add(3 * time.Hour)}}})
open, _ := k.Open(t.Context())
if len(open) != 1 || open[0].Key != conditions.Key(conditions.ScopeBus, mail, link.AdvisoryMaxDeliveries) {
t.Fatalf("after one aged out: %+v", open)
}
look(now.Add(4*time.Hour), map[string]int{}, map[string][]link.Aging{})
if open, _ := k.Open(t.Context()); len(open) != 0 {
t.Fatalf("after all aged out: %+v", open)
}
k.Close(context.Background()) // everything said is said once it closes
var mailCleared, sonarrCleared string
for _, e := range told.Said() {
if e.Change != conditions.ChangeCleared {
continue
}
switch e.Condition.Key {
case conditions.Key(conditions.ScopeBus, mail, link.AdvisoryMaxDeliveries):
mailCleared = e.Condition.Resolved
case conditions.Key(conditions.ScopeBus, "EVENTS.media_sonarr", link.AdvisoryMaxDeliveries):
sonarrCleared = e.Condition.Resolved
}
}
if mailCleared != "Resolved: 2 messages of the stream MAIL aged out undelivered" {
t.Fatalf("MAIL's condition cleared with %q", mailCleared)
}
if sonarrCleared != "Resolved: the messages it gave up on were delivered again or dropped" {
t.Fatalf("an EVENTS consumer's condition cleared with %q", sonarrCleared)
}
if why, ok := conditions.Plain(mailCleared); !ok {
t.Fatalf("not plain: %s", why)
}
}
+1 -1
View File
@@ -952,7 +952,7 @@ func deliveryCommand(ctx context.Context, args []string) error {
if *group == "" && len(members) == 1 { if *group == "" && len(members) == 1 {
// One head, checked again as the forge's announcement would have it (a recheck): the controller's // One head, checked again as the forge's announcement would have it (a recheck): the controller's
// own path for a pull request, run because the delivery's owner asked. // own path for a pull request, run because the delivery's owner asked.
if err := sayingOnTheBus(ctx, inv, func() error { return (following{open: open, oneShot: true}).PullUpdated(ctx, members[0].pull()) }); err != nil { if err := sayingOnTheBus(ctx, inv, func() error { return (following{open}).PullUpdated(ctx, members[0].pull()) }); err != nil {
return err return err
} }
return answer(map[string]any{"rechecked": members[0].ID}) return answer(map[string]any{"rechecked": members[0].ID})
+8 -105
View File
@@ -5,7 +5,6 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"log"
"strings" "strings"
"time" "time"
@@ -55,10 +54,6 @@ type deskGive struct {
trusted func(module string) (bool, error) trusted func(module string) (bool, error)
// ask asks one machine's node-launcher.secret and answers its result, or the holder's refusal. // ask asks one machine's node-launcher.secret and answers its result, or the holder's refusal.
ask func(machine string, args map[string]any) (json.RawMessage, error) ask func(machine string, args map[string]any) (json.RawMessage, error)
// promptFor is the one line the secret's declaration asks for it as (novox/hq issue 789): read from the
// controller's own catalogue, never from the call or the module's runtime. Nil, or an empty answer, asks by
// name alone, as the desk does today.
promptFor func(module, name string) (string, error)
// accept seals the value as `secret accept` does, and says whether it lives until the module's start. // accept seals the value as `secret accept` does, and says whether it lives until the module's start.
accept func(value string) (untilStart bool, err error) accept func(value string) (untilStart bool, err error)
// record writes the act in the hand-act log. // record writes the act in the hand-act log.
@@ -158,28 +153,15 @@ func (d deskGive) give(node, module, name, desk string) (string, error) {
} }
// By name, never by words: the holder writes the prompt from these, and says the controller asks, which // By name, never by words: the holder writes the prompt from these, and says the controller asks, which
// the bus alone makes true (broker.ControllerOnly). Who asked is the bus's word on the caller, cut to a // the bus alone makes true (broker.ControllerOnly). Who asked is the bus's word on the caller, cut to a
// name's characters — never an argument of the call. `what` is the catalogue's own words for the secret // name's characters — never an argument of the call.
// (novox/hq issue 789), never a word the caller chose; omitted when the module declares none, so an old raw, err := d.ask(desk, map[string]any{
// desk and an old module behave as today.
args := map[string]any{
"module": module, "module": module,
"secret": name, "secret": name,
"node": node, "node": node,
"asked_by": askedByName(d.askedBy), "asked_by": askedByName(d.askedBy),
"seal_to": public, "seal_to": public,
"timeout_seconds": deskPromptWithin, "timeout_seconds": deskPromptWithin,
} })
if d.promptFor != nil {
what, err := d.promptFor(module, name)
if err != nil {
outcome = "refused"
return "", fmt.Errorf("nobody was asked to type anything: %w", err)
}
if what != "" {
args["what"] = what
}
}
raw, err := d.ask(desk, args)
if err != nil { if err != nil {
outcome = "refused" outcome = "refused"
return "", fmt.Errorf("the desk on %s could not be asked: %w", desk, err) return "", fmt.Errorf("the desk on %s could not be asked: %w", desk, err)
@@ -254,9 +236,8 @@ func askAtDesk(ctx context.Context, node, module, name, desk string) error {
return err return err
} }
defer open.Close() defer open.Close()
askedBy := link.Caller()
d := deskGive{ d := deskGive{
askedBy: askedBy, askedBy: link.Caller(),
open: func(node, module, name, desk string) (int64, error) { open: func(node, module, name, desk string) (int64, error) {
return open.inventory.OpenSecretAsk(ctx, node, module, name, askedByName(link.Caller()), desk) return open.inventory.OpenSecretAsk(ctx, node, module, name, askedByName(link.Caller()), desk)
}, },
@@ -267,7 +248,6 @@ func askAtDesk(ctx context.Context, node, module, name, desk string) error {
return err return err
}, },
trusted: func(module string) (bool, error) { return open.inventory.RunsAsItsOwnAccount(ctx, module) }, trusted: func(module string) (bool, error) { return open.inventory.RunsAsItsOwnAccount(ctx, module) },
promptFor: func(module, name string) (string, error) { return open.inventory.OwnSecretPrompt(ctx, module, name) },
ask: func(machine string, args map[string]any) (json.RawMessage, error) { ask: func(machine string, args map[string]any) (json.RawMessage, error) {
var result json.RawMessage var result json.RawMessage
err := onTheBus(func(conn *nats.Conn) error { err := onTheBus(func(conn *nats.Conn) error {
@@ -293,9 +273,7 @@ func askAtDesk(ctx context.Context, node, module, name, desk string) error {
return err return err
}) })
}, },
announce: func(node, module, name, how string) error { announce: func(node, module, name, how string) error { return announceSecretGiven(ctx, node, module, name, how) },
return announceSecretGiven(ctx, node, module, name, how, askedBy)
},
} }
words, err := d.give(node, module, name, desk) words, err := d.give(node, module, name, desk)
if err != nil { if err != nil {
@@ -346,85 +324,10 @@ func secretNameWords(name string) string {
return strings.Join(strings.FieldsFunc(name, func(r rune) bool { return r == '-' || r == '_' || r == '.' }), " ") return strings.Join(strings.FieldsFunc(name, func(r rune) bool { return r == '-' || r == '_' || r == '.' }), " ")
} }
// tellWithin is how long the operator channel's `tell` is given to answer: well under a human's patience, // announceSecretGiven raises it on this controller's keeper.
// and short because nothing in the call waits on the operator — it asks nothing (novox/hq ADR 0380). func announceSecretGiven(ctx context.Context, node, module, name, how string) error {
const tellWithin = 10 * time.Second
// askOperatorChannel asks the operator channel's seat (the holder issue 797 gives it) its `tell` verb,
// through the controller's grant (novox/hq ADR 0380 §8), over this controller's own bus connection; a seam
// a test replaces whole. The holder's own refusal is an error naming it; an absent holder or a refused
// grant answer as link.ErrNothingServes or link.ErrGrantRefused (checked with errors.Is), so the caller can
// tell the two apart from any other failure.
var askOperatorChannel = func(ctx context.Context, args map[string]any) (json.RawMessage, error) {
var result json.RawMessage
err := onTheBus(func(conn *nats.Conn) error {
answer, err := link.AskMeshSeatTool(ctx, conn, "operator-channel", "tell", args, tellWithin)
if err != nil {
return err
}
if answer.Error != "" {
return fmt.Errorf("operator-channel.tell refused: %s", answer.Error)
}
result = answer.Result
return nil
})
return result, err
}
// announceSecretGiven tells the operator, once, that a module's own secret was given (novox/hq ADR 0380):
// one plain message through the operator channel's `tell`, naming the secret, the module, the machine,
// where and when it was given and who asked — asking nothing, and opening no condition. **The controller's
// change waits for that verb at run time, not at merge** (clause 9): while nothing on the bus yet serves it,
// or the controller's own grant is refused, the secret-given condition is raised exactly as before ADR 0380
// — any other failure (the holder itself refusing, or answering something unreadable) is the announcement's
// own failure, said to the caller as it always was.
func announceSecretGiven(ctx context.Context, node, module, name, how, askedBy string) error {
at := time.Now()
body := map[string]any{
"headline": fmt.Sprintf("A password was entered for %s's %s on %s", module, secretNameWords(name), node),
"subject": node,
"lines": []string{
fmt.Sprintf("%s at %s, asked by %s.", how, at.Local().Format("15:04"), askedByName(askedBy)),
"If that was not you, give it again yourself at the controller's terminal.",
},
}
_, told := askOperatorChannel(ctx, body)
switch {
case told == nil:
clearSecretGivenConditions(ctx)
return nil
case errors.Is(told, link.ErrNothingServes), errors.Is(told, link.ErrGrantRefused):
log.Printf("secret-given: %s's %s on %s could not be told through the operator channel (%v); the "+
"secret-given condition is raised as it was before novox/hq ADR 0380 (its clause 9)", module, name,
node, told)
return withKeeper(ctx, func(k *conditions.Keeper) error { return withKeeper(ctx, func(k *conditions.Keeper) error {
_, err := k.Observe(ctx, secretGivenObservation(node, module, name, how, at)) _, err := k.Observe(ctx, secretGivenObservation(node, module, name, how, time.Now()))
return err return err
}) })
default:
return told
}
}
// clearSecretGivenConditions retires every open secret-given condition once a plain message has replaced
// it (novox/hq ADR 0380 §10): called after the first time a secret-given message is told successfully. No
// resolved line is sent — the condition is simply gone — and why is kept in the keeper's own history.
func clearSecretGivenConditions(ctx context.Context) {
if err := withKeeper(ctx, func(k *conditions.Keeper) error {
open, err := k.Open(ctx)
if err != nil {
return err
}
for _, c := range open {
if c.Kind != kindSecretGiven {
continue
}
if _, err := k.ClearSaying(ctx, c.Key, "retired by novox/hq ADR 0380", ""); err != nil {
return err
}
}
return nil
}); err != nil {
log.Printf("secret-given: the open secret-given conditions, retired by ADR 0380, could not be cleared: %v", err)
}
} }
-215
View File
@@ -456,52 +456,6 @@ func TestThePromptNamesWhoAskedFromTheBussWordAlone(t *testing.T) {
} }
} }
// The desk's prompt names which account of a family is asked for, from the catalogue's own words (novox/hq
// issue 789): a family declaring `prompt` with `{member}` sends `what` with the member's suffix filled in.
func TestThePromptNamesWhichAccountOfAFamilyIsAsked(t *testing.T) {
d, _, _, asked := aDesk(t, sealedTo(t, typed))
d.declares = func(module, name string) error {
if module != "mesh-mail" || name != "mail-credential-gmail" {
return errors.New(module + " does not declare " + name + " as an own secret")
}
return nil
}
d.promptFor = func(module, name string) (string, error) {
if module == "mesh-mail" && name == "mail-credential-gmail" {
return "the password of the mail account gmail", nil
}
return "", nil
}
if _, err := d.give("anchor", "mesh-mail", "mail-credential-gmail", "laptop"); err != nil {
t.Fatal(err)
}
if got := (*asked)[0]["what"]; got != "the password of the mail account gmail" {
t.Errorf(`"what" = %v`, got)
}
}
// Without a declared prompt, no `what` is sent: an old module, or one that declares none, is asked by name alone,
// as the desk does today (novox/hq issue 789).
func TestWithNoDeclaredPromptNoWhatIsSent(t *testing.T) {
d, _, _, asked := aDesk(t, sealedTo(t, typed))
d.promptFor = func(module, name string) (string, error) { return "", nil }
if _, err := d.give("anchor", "telegram", "telegram-token", "laptop"); err != nil {
t.Fatal(err)
}
if _, there := (*asked)[0]["what"]; there {
t.Errorf(`"what" was sent with no declared prompt: %v`, (*asked)[0])
}
// And with no promptFor wired at all (an older caller of deskGive), same thing: `give` still works, asking
// nothing more than it does today.
d2, _, _, asked2 := aDesk(t, sealedTo(t, typed))
if _, err := d2.give("anchor", "telegram", "telegram-token", "laptop"); err != nil {
t.Fatal(err)
}
if _, there := (*asked2)[0]["what"]; there {
t.Errorf(`"what" was sent with promptFor nil: %v`, (*asked2)[0])
}
}
// An ask for a secret is bounded before anybody is asked to type (ADR 0277): the record refuses it, nothing is // An ask for a secret is bounded before anybody is asked to type (ADR 0277): the record refuses it, nothing is
// asked; and how every ask ends is recorded. // asked; and how every ask ends is recorded.
func TestASecretAskIsBoundedAndItsEndRecorded(t *testing.T) { func TestASecretAskIsBoundedAndItsEndRecorded(t *testing.T) {
@@ -556,172 +510,3 @@ func TestASecretAskIsNeverASecretRead(t *testing.T) {
} }
} }
} }
// askOperatorChannelFake replaces askOperatorChannel for the test and records every call it was given.
func askOperatorChannelFake(t *testing.T, answer func(args map[string]any) (json.RawMessage, error)) *[]map[string]any {
t.Helper()
var calls []map[string]any
was := askOperatorChannel
askOperatorChannel = func(_ context.Context, args map[string]any) (json.RawMessage, error) {
calls = append(calls, args)
return answer(args)
}
t.Cleanup(func() { askOperatorChannel = was })
return &calls
}
// announceWithRealTell wires a deskGive's announce to the real announceSecretGiven, as askAtDesk does,
// rather than the fakes the tests above use to look at what `give` does around it.
func announceWithRealTell(ctx context.Context, askedBy string) func(node, module, name, how string) error {
return func(node, module, name, how string) error {
return announceSecretGiven(ctx, node, module, name, how, askedBy)
}
}
// novox/hq ADR 0380 clauses 1-2: a secret given at the desk is told to the operator once, in a plain
// message through the operator channel's `tell`, naming the secret, the module, where and when and who
// asked, and opens no condition — the kind the review of 2026-10-09 (M4) raised is retired.
func TestASecretGivenAtTheDeskIsToldOnceAndOpensNoCondition(t *testing.T) {
ctx := context.Background()
k, _ := withConditionsInMemory(t)
calls := askOperatorChannelFake(t, func(map[string]any) (json.RawMessage, error) { return nil, nil })
d, _, _, _ := aDesk(t, sealedTo(t, typed))
d.announce = announceWithRealTell(ctx, "mail-seats-835@novox")
if _, err := d.give("anchor", "telegram", "telegram-token", "laptop"); err != nil {
t.Fatal(err)
}
if len(*calls) != 1 {
t.Fatalf("told the operator %d times, want exactly 1: %v", len(*calls), *calls)
}
call := (*calls)[0]
if call["headline"] != "A password was entered for telegram's telegram token on anchor" {
t.Errorf("headline %v", call["headline"])
}
if call["subject"] != "anchor" {
t.Errorf("subject %v", call["subject"])
}
lines, _ := call["lines"].([]string)
if len(lines) != 2 || !strings.Contains(lines[0], "at the desk on laptop at") ||
!strings.Contains(lines[0], "asked by mail-seats-835@novox") ||
lines[1] != "If that was not you, give it again yourself at the controller's terminal." {
t.Errorf("lines %v", lines)
}
open, err := k.Open(ctx)
if err != nil {
t.Fatal(err)
}
if len(open) != 0 {
t.Errorf("a condition was raised though the operator was told: %+v", open)
}
}
// novox/hq ADR 0380 §9: until the operator channel's `tell` is delivered — nothing yet serves it, or the
// controller's own grant is refused — the secret-given condition is raised exactly as before.
func TestASecretGivenAtTheDeskRaisesTheConditionWhileNothingServesTell(t *testing.T) {
ctx := context.Background()
for _, failure := range []error{link.ErrNothingServes, link.ErrGrantRefused} {
k, _ := withConditionsInMemory(t)
askOperatorChannelFake(t, func(map[string]any) (json.RawMessage, error) { return nil, failure })
d, _, _, _ := aDesk(t, sealedTo(t, typed))
d.announce = announceWithRealTell(ctx, "mesh-mail-835, novox")
if _, err := d.give("anchor", "telegram", "telegram-token", "laptop"); err != nil {
t.Fatal(err)
}
open, err := k.Open(ctx)
if err != nil {
t.Fatal(err)
}
if len(open) != 1 || open[0].Kind != kindSecretGiven {
t.Fatalf("%v: the condition was not raised as before: %+v", failure, open)
}
}
}
// novox/hq ADR 0380 §10: once the operator channel is told successfully, every secret-given condition
// still open from before this change is retired, without a resolved line of its own.
func TestASuccessfulTellRetiresEveryOpenSecretGivenCondition(t *testing.T) {
ctx := context.Background()
k, _ := withConditionsInMemory(t)
for _, module := range []string{"telegram", "mesh-mail"} {
if _, err := k.Observe(ctx, secretGivenObservation("anchor", module, "a-secret", "at the desk on laptop",
time.Now())); err != nil {
t.Fatal(err)
}
}
askOperatorChannelFake(t, func(map[string]any) (json.RawMessage, error) { return nil, nil })
if err := announceSecretGiven(ctx, "anchor", "telegram", "telegram-token", "at the desk on laptop", "me"); err != nil {
t.Fatal(err)
}
open, err := k.Open(ctx)
if err != nil {
t.Fatal(err)
}
if len(open) != 0 {
t.Fatalf("the open secret-given conditions were not retired: %+v", open)
}
}
// novox/hq ADR 0380 clause 7: a trusted party's secret given at the controller's terminal is announced
// before it is kept, and not kept when the announcement fails. It still fails when telling the operator
// AND raising the fallback condition both fail — nothing silently keeps a trusted party's value unheard.
func TestATrustedPartysSecretAtTheTerminalIsNotKeptWhenTellAndTheConditionBothFail(t *testing.T) {
ctx := context.Background()
askOperatorChannelFake(t, func(map[string]any) (json.RawMessage, error) { return nil, link.ErrNothingServes })
// conditionsFrom is left nil and there is no bus to dial here, so the fallback condition cannot be
// raised either: both of ADR 0380's ways to tell the operator fail.
kept := false
if _, _, err := keepGiven(true,
func() error {
return announceSecretGiven(ctx, "anchor", "telegram", "telegram-token", "at the controller's terminal", "me")
},
func() (bool, error) { kept = true; return false, nil }); err == nil {
t.Error("a trusted party's secret was kept though telling the operator and raising the condition both failed")
}
if kept {
t.Error("the value was kept despite the failure")
}
}
// Review round 2 of mesh-controller (feat/mail-seats, head 5f24a47d): ADR 0380 §9's fallback — raising the
// secret-given condition — answers only link.ErrNothingServes and link.ErrGrantRefused. Any other failure of
// the operator channel's `tell` (the holder itself refusing, or answering something unreadable) is the
// announcement's own failure, surfaced to the caller, and opens no condition.
func TestAnOperatorChannelFailureOtherThanNothingServesOrGrantRefusedSurfacesAndRaisesNoCondition(t *testing.T) {
ctx := context.Background()
k, _ := withConditionsInMemory(t)
refused := errors.New("operator-channel.tell refused: too many lines")
askOperatorChannelFake(t, func(map[string]any) (json.RawMessage, error) { return nil, refused })
if err := announceSecretGiven(ctx, "anchor", "telegram", "telegram-token", "at the desk on laptop", "me"); err != refused {
t.Fatalf("announceSecretGiven = %v, want the refusal itself surfaced", err)
}
open, err := k.Open(ctx)
if err != nil {
t.Fatal(err)
}
if len(open) != 0 {
t.Errorf("a condition was raised though the failure was neither ErrNothingServes nor ErrGrantRefused: %+v", open)
}
}
// Review round 2 of mesh-controller: a secret accepted at the controller's terminal writes a hand act naming
// who asked, where and when it was given — and never the value (novox/hq ADR 0380 §6).
func TestASecretAcceptedAtTheTerminalWritesAHandActNamingWhoWhereAndNeverTheValue(t *testing.T) {
act := terminalGiveHandAct("anchor", "telegram", "telegram-token", "mail-seats-835@novox")
if act.Verb != "secret accept" || act.Cause != "given-at-the-terminal" {
t.Fatalf("act %+v", act)
}
if !strings.Contains(act.Why, "at the controller's terminal") {
t.Errorf("the act does not name where: %q", act.Why)
}
if !strings.Contains(act.Why, "asked by mail-seats-835@novox") {
t.Errorf("the act does not name who: %q", act.Why)
}
if strings.Contains(act.Why, typed) {
t.Error("the act carries the value")
}
// When is RecordHandAct's own job (it stamps At with time.Now when the act leaves none, and By with the
// caller when the act leaves that empty too): this act leaves both for it to fill.
if act.By != "" || !act.At.IsZero() {
t.Errorf("the act sets its own who/when instead of leaving them to RecordHandAct: %+v", act)
}
}
-5
View File
@@ -146,11 +146,6 @@ var probeRegistry = []probe{
{ID: probeFillID, Asserts: "every module's bucket and log holds less than three quarters of its cap; one " + {ID: probeFillID, Asserts: "every module's bucket and log holds less than three quarters of its cap; one " +
"said filling is cleared once it holds less than seven tenths", From: "ADR 0297, issue 501", "said filling is cleared once it holds less than seven tenths", From: "ADR 0297, issue 501",
Kind: kindBucketOrLogFilling, Raises: []string{kindBucketOrLogUnread}, Phase: 1, run: probeFill}, Kind: kindBucketOrLogFilling, Raises: []string{kindBucketOrLogUnread}, Phase: 1, run: probeFill},
// A grant of a restricted stream that composes nothing (novox/hq ADR 0376 §2): the module is carried by the
// machine's tool runner, which never hears a restricted stream, so it hears nothing it was granted.
{ID: probeWithheldID, Asserts: "no grant of a restricted stream names a module that composes nothing for it: every " +
"module granted one runs as an account of its own", From: "ADR 0376 §2", Kind: kindGrantComposesNothing,
Phase: 1, run: probeWithheldHearing},
{ID: "DW", Asserts: "the watchdogs of the signals table ran within three of their intervals", {ID: "DW", Asserts: "the watchdogs of the signals table ran within three of their intervals",
From: "ADR 0227 rule 6: the watchers are watched", Kind: "watchdogs-silent", Phase: 1, run: probeWatchdogs}, From: "ADR 0227 rule 6: the watchers are watched", Kind: "watchdogs-silent", Phase: 1, run: probeWatchdogs},
// The core's health definitions (novox/hq to-be 45 §8, ADR 0236): what a core component's new build is // The core's health definitions (novox/hq to-be 45 §8, ADR 0236): what a core component's new build is
+2 -2
View File
@@ -348,11 +348,11 @@ func gatherFacts(ctx context.Context, open *stores, busVersion string) (snapshot
if layer.From == catalogue.MeshWideLayer { if layer.From == catalogue.MeshWideLayer {
if !meshWide[module] { if !meshWide[module] {
meshWide[module] = true meshWide[module] = true
f.Settings = append(f.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Values)}) f.Settings = append(f.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Whole())})
} }
continue continue
} }
m.Settings = append(m.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Values)}) m.Settings = append(m.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Whole())})
} }
} }
m.Declaration = declarationFacts(ctx, open, n.Name, gens, gensErr, scrub) m.Declaration = declarationFacts(ctx, open, n.Name, gens, gensErr, scrub)
-3
View File
@@ -22,7 +22,6 @@ func foundDirectory(module string, since time.Time) inventory.ResourceHealth {
} }
func TestADirectoryFoundBeforeTheSendIsAWaitForAPerson(t *testing.T) { func TestADirectoryFoundBeforeTheSendIsAWaitForAPerson(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
now := time.Now() now := time.Now()
sent := now.Add(-time.Minute) sent := now.Add(-time.Minute)
f := gateFacts{now: now, health: map[string]inventory.NodeHealth{"laptop": {Node: "laptop", HeardAt: now, f := gateFacts{now: now, health: map[string]inventory.NodeHealth{"laptop": {Node: "laptop", HeardAt: now,
@@ -49,7 +48,6 @@ func TestADirectoryFoundBeforeTheSendIsAWaitForAPerson(t *testing.T) {
// The whole walk: a module whose directory was used as found long before still gets its fix to every machine, // The whole walk: a module whose directory was used as found long before still gets its fix to every machine,
// its pass kept and the wait said; a directory this very send found holds it and puts it back. // its pass kept and the wait said; a directory this very send found holds it and puts it back.
func TestAFixGoesThroughPastADirectoryFoundBefore(t *testing.T) { func TestAFixGoesThroughPastADirectoryFoundBefore(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
for _, c := range []struct { for _, c := range []struct {
name string name string
found time.Duration // when the directory was found, against now found time.Duration // when the directory was found, against now
@@ -118,7 +116,6 @@ func TestAFixGoesThroughPastADirectoryFoundBefore(t *testing.T) {
// The condition says the wait in its own kind: the operator's, never urgent, and cleared once handed over. // The condition says the wait in its own kind: the operator's, never urgent, and cleared once handed over.
func TestADirectoryUsedAsFoundIsItsOwnCondition(t *testing.T) { func TestADirectoryUsedAsFoundIsItsOwnCondition(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
k, _ := withConditionsInMemory(t) k, _ := withConditionsInMemory(t)
ctx := t.Context() ctx := t.Context()
rs := map[string][]inventory.ResourceHealth{"notes": {foundDirectory("notes", time.Now().Add(-time.Hour))}} rs := map[string][]inventory.ResourceHealth{"notes": {foundDirectory("notes", time.Now().Add(-time.Hour))}}
@@ -57,7 +57,6 @@ func healthyContainer(module string) inventory.ResourceHealth {
// tools' witness put its build back at once, and the send waits for the healthy module's own verdict — a pass, // tools' witness put its build back at once, and the send waits for the healthy module's own verdict — a pass,
// counted over the gate's spacing — before it says its own. The broken one alone is put back. // counted over the gate's spacing — before it says its own. The broken one alone is put back.
func TestAFaultBeforeTheSettleTimeWaitsForTheModuleBesideIt(t *testing.T) { func TestAFaultBeforeTheSettleTimeWaitsForTheModuleBesideIt(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
@@ -100,7 +99,6 @@ const broker_runtime = "node-tools"
// **A pass is counted only the gate's spacing after the one before** (review (c)): a send judged every tick // **A pass is counted only the gate's spacing after the one before** (review (c)): a send judged every tick
// while a module beside it is not yet healthy counts the healthy one once. // while a module beside it is not yet healthy counts the healthy one once.
func TestAPassIsCountedNoFasterThanTheGatesSpacing(t *testing.T) { func TestAPassIsCountedNoFasterThanTheGatesSpacing(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
withGateBounds(t, time.Hour, time.Hour, time.Hour) withGateBounds(t, time.Hour, time.Hour, time.Hour)
@@ -125,7 +123,6 @@ func TestAPassIsCountedNoFasterThanTheGatesSpacing(t *testing.T) {
// **A carried build's pass carries its own wait, never another's** (review (e)). // **A carried build's pass carries its own wait, never another's** (review (e)).
func TestACarriedPassCarriesOnlyItsOwnWait(t *testing.T) { func TestACarriedPassCarriesOnlyItsOwnWait(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
b := aBacklog(t) b := aBacklog(t)
ctx := t.Context() ctx := t.Context()
inv := b.open.inventory inv := b.open.inventory
@@ -150,7 +147,6 @@ func TestACarriedPassCarriesOnlyItsOwnWait(t *testing.T) {
// carried its own module, healthy, and a build waiting on that machine that never became healthy. At the // carried its own module, healthy, and a build waiting on that machine that never became healthy. At the
// bound the waiting one is put back and marked; the plan's own module keeps its pass, so no walk waits on it. // bound the waiting one is put back and marked; the plan's own module keeps its pass, so no walk waits on it.
func TestThePlansOwnModuleKeepsItsPassWhenItsSendFails(t *testing.T) { func TestThePlansOwnModuleKeepsItsPassWhenItsSendFails(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
g := aGateMesh(t) g := aGateMesh(t)
ctx := t.Context() ctx := t.Context()
inv := g.open.inventory inv := g.open.inventory
@@ -231,7 +227,6 @@ func TestThePlansOwnModuleKeepsItsPassWhenItsSendFails(t *testing.T) {
// **A provider waiting for a login says who waits on it** (review (g)): its consumer, failing a check that // **A provider waiting for a login says who waits on it** (review (g)): its consumer, failing a check that
// needs it, is held under it, and the provider's relogin-needed condition lists it and is urgent. // needs it, is held under it, and the provider's relogin-needed condition lists it and is urgent.
func TestAProviderWaitingForALoginSaysWhoWaitsOnIt(t *testing.T) { func TestAProviderWaitingForALoginSaysWhoWaitsOnIt(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
inv := open.inventory inv := open.inventory
@@ -294,7 +289,6 @@ func TestAProviderWaitingForALoginSaysWhoWaitsOnIt(t *testing.T) {
// tools are marked and put back in the very judging that found them broken — their registered build is the // tools are marked and put back in the very judging that found them broken — their registered build is the
// one before, and the laptop is sent it — while `app` goes on being judged, recovers, and keeps its own pass. // one before, and the laptop is sent it — while `app` goes on being judged, recovers, and keeps its own pass.
func TestABrokenModuleIsPutBackAtOnceAndTheOneBesideItGetsItsOwnVerdict(t *testing.T) { func TestABrokenModuleIsPutBackAtOnceAndTheOneBesideItGetsItsOwnVerdict(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
@@ -437,7 +431,6 @@ func TestABrokenModuleIsPutBackAtOnceAndTheOneBesideItGetsItsOwnVerdict(t *testi
// **A move from a build not known excuses no wait** (issue 318 review): a plan's own module whose previous // **A move from a build not known excuses no wait** (issue 318 review): a plan's own module whose previous
// build was not recorded, or whose previous manifest is not kept, cannot be shown to have added the group. // build was not recorded, or whose previous manifest is not kept, cannot be shown to have added the group.
func TestAMoveFromAnUnknownBuildExcusesNoWait(t *testing.T) { func TestAMoveFromAnUnknownBuildExcusesNoWait(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
inv := open.inventory inv := open.inventory
-32
View File
@@ -36,10 +36,6 @@ func aGateMesh(t *testing.T) *gateMesh {
ctx := t.Context() ctx := t.Context()
inv := open.inventory inv := open.inventory
g := &gateMesh{open: open, health: map[string]health{}} g := &gateMesh{open: open, health: map[string]health{}}
// The gate's own tests judge a first machine, as walks a controller before novox/hq ADR 0360 started still are.
wasGated := gateFirstMachine
gateFirstMachine = true
t.Cleanup(func() { gateFirstMachine = wasGated })
g.keeper, _ = withConditionsInMemory(t) g.keeper, _ = withConditionsInMemory(t)
g.told = &conditions.Told{} g.told = &conditions.Told{}
was := doctorFrom was := doctorFrom
@@ -275,34 +271,6 @@ func TestABuildThatPassesItsGateRollsEverywhereUnattended(t *testing.T) {
} }
} }
// A build rolls everywhere with no hand and no gate (novox/hq ADR 0360 §1): both machines in one send, no health
// judged, and the walk done once both reported on the send. Before, anchor was judged first, then laptop sent.
func TestABuildRollsToEveryMachineAtOnceWithNoGate(t *testing.T) {
g := aGateMesh(t)
gateFirstMachine = false
ctx := t.Context()
judged := false
wasGather := gatherGateFacts
gatherGateFacts = func(ctx context.Context, open *stores, component string) (gateFacts, error) {
judged = true
return wasGather(ctx, open, component)
}
t.Cleanup(func() { gatherGateFacts = wasGather })
for i := 0; i < 6; i++ {
advancePlans(ctx, g.open)
}
p := g.plan(t)
if !reflect.DeepEqual(g.sent, [][]string{{"anchor", "laptop"}}) {
t.Fatalf("sent %v, not both machines in one send", g.sent)
}
if s := p.Modules["app"]; p.State != inventory.PlanDone || s.Gate != nil || s.FirstAt != nil || judged {
t.Fatalf("the plan is %s (%s), app %+v, judged %v", p.State, p.Note, s, judged)
}
if obs, err := probeGates(ctx, doctorFrom); err != nil || len(obs) != 0 {
t.Fatalf("a rolled build left a condition: %+v %v", obs, err)
}
}
// A leaf module passes its gate on one reading, as soon as its first machine is healthy, without waiting out the // A leaf module passes its gate on one reading, as soon as its first machine is healthy, without waiting out the
// core's settle time (novox/hq ADR 0282 decision 4 as ADR 0304 decision 3 changed it, issue 524). // core's settle time (novox/hq ADR 0282 decision 4 as ADR 0304 decision 3 changed it, issue 524).
func TestALeafModulePassesOnOneHealthyReading(t *testing.T) { func TestALeafModulePassesOnOneHealthyReading(t *testing.T) {
+20 -52
View File
@@ -8,8 +8,6 @@ import (
"strings" "strings"
"time" "time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/catalogue" "github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link" "github.com/novox/mesh-controller/internal/link"
@@ -36,62 +34,32 @@ type groupMember struct {
Repository, State string Repository, State string
} }
// groupsLine is one group as mesh-delivery's `groups` says it: its id (the head branch name), its members as text and,
// from a mesh-delivery that knows novox/hq ADR 0382, each member again as an object (`member_details`) naming its pull
// request's number, whole head, base branch and whether it is open.
type groupsLine struct {
ID string `json:"id"`
Members []string `json:"members"`
Details []memberLine `json:"member_details,omitempty"`
}
// memberLine is one member of a group, as `groups` says it beside its text.
type memberLine struct {
ID string `json:"id"`
Repository string `json:"repository"`
State string `json:"state"`
Number int `json:"number"`
Head string `json:"head"`
Base string `json:"base"`
Open bool `json:"open"`
}
// groupsAsked is mesh-delivery's `groups`: every group it keeps, or an error when it cannot be asked or did not
// answer within groupAskWithin. A seam a test replaces.
var groupsAsked = func(ctx context.Context, held bool) ([]groupsLine, error) {
if !held {
return nil, fmt.Errorf("no holder of %s is on record to say it", catalogue.DeliverySeat)
}
// On the bus as this process is: the serving controller's own connection, lent, or — in a verb that runs as a command
// of its own, such as a recheck's `delivery check` — one dialled for the moment (onTheBus).
var raw json.RawMessage
err := onTheBus(func(conn *nats.Conn) error {
asking, cancel := context.WithTimeout(ctx, groupAskWithin)
defer cancel()
var err error
raw, err = askDeliveryOwner(asking, conn, "groups", map[string]any{})
return err
})
if err != nil {
if errors.Is(err, link.ErrNothingServes) {
return nil, fmt.Errorf("nothing serves %s.groups", catalogue.DeliverySeat)
}
return nil, err
}
var lines []groupsLine
if err := json.Unmarshal(raw, &lines); err != nil {
return nil, fmt.Errorf("%s.groups answered something unreadable: %w", catalogue.DeliverySeat, err)
}
return lines, nil
}
// deliveryGroupOf is what mesh-delivery says of the group a head names: its members, and whether the head names a // deliveryGroupOf is what mesh-delivery says of the group a head names: its members, and whether the head names a
// group at all; an error when it cannot be asked or did not answer. A seam a test replaces. // group at all; an error when it cannot be asked or did not answer. A seam a test replaces.
var deliveryGroupOf = func(ctx context.Context, held bool, head string) ([]groupMember, bool, error) { var deliveryGroupOf = func(ctx context.Context, held bool, head string) ([]groupMember, bool, error) {
lines, err := groupsAsked(ctx, held) if !held {
return nil, false, fmt.Errorf("no holder of %s is on record to say it", catalogue.DeliverySeat)
}
serving := servingBus.Load()
if serving == nil {
return nil, false, errors.New("this controller is not on the bus")
}
asking, cancel := context.WithTimeout(ctx, groupAskWithin)
defer cancel()
raw, err := askDeliveryOwner(asking, serving.Conn(), "groups", map[string]any{})
if err != nil { if err != nil {
if errors.Is(err, link.ErrNothingServes) {
return nil, false, fmt.Errorf("nothing serves %s.groups", catalogue.DeliverySeat)
}
return nil, false, err return nil, false, err
} }
var lines []struct {
ID string `json:"id"`
Members []string `json:"members"`
}
if err := json.Unmarshal(raw, &lines); err != nil {
return nil, false, fmt.Errorf("%s.groups answered something unreadable: %w", catalogue.DeliverySeat, err)
}
for _, l := range lines { for _, l := range lines {
if l.ID != head { if l.ID != head {
continue continue
+1 -2
View File
@@ -642,8 +642,7 @@ func planStale(ctx context.Context, inv *inventory.Inventory, p inventory.Plan)
if err != nil { if err != nil {
return "", "", err return "", "", err
} }
// A recorded module whose move goes out on a machine is still the walk's to send (novox/hq ADR 0377). if u.RollOut {
if u.RollOut || len(wentOut(s.Recorded)) > 0 {
return "", "", nil return "", "", nil
} }
} }
-66
View File
@@ -1,66 +0,0 @@
package main
import (
"context"
"fmt"
"strings"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
)
// A grant of a restricted stream that composes nothing (novox/hq ADR 0376 §2).
//
// A restricted stream is heard only by a module that runs as an account of its own (`runs-as`, ADR 0259): the
// tool runner's bus account is the one every agent session on its machine reaches the bus as, so the controller
// composes no consumer, grant or bind word for a module it carries (broker.ComposedHears). Where a grant still
// names such a module, the module hears nothing it was granted, and that is said — in `doctor` and in the
// controller's log on every raise and send — naming the module, the stream and the remedy. Never silent.
const (
// probeWithheldID is the self-check's probe of it, and kindGrantComposesNothing the condition it raises.
probeWithheldID = "D-heard"
kindGrantComposesNothing = "grant-composes-nothing"
)
// probeWithheldHearing says every granted hearing of a restricted stream composed for nothing.
func probeWithheldHearing(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
records, err := d.open.inventory.BusRecords(ctx)
if err != nil {
return nil, err
}
return withheldObservations(broker.WithheldOf(records)), nil
}
// withheldObservations is each withheld hearing as a condition of its module on its machine.
func withheldObservations(withheld []broker.Withheld) []conditions.Observation {
var out []conditions.Observation
for _, w := range withheld {
out = append(out, conditions.Observation{Scope: conditions.ScopeModule,
ID: w.Module + "." + strings.ToLower(w.Stream) + "." + w.Node,
Kind: kindGrantComposesNothing, Machine: w.Node, Severity: conditions.Warning,
Summary: w.Said(), Said: fmt.Sprintf("%s on %s granted %s; composed nothing", w.Module, w.Node, w.Stream)})
}
return sortedFound(out)
}
// sayWithheld puts every withheld hearing in the controller's log, once per raise or send that composes it.
func sayWithheld(records broker.Records, indent string) {
for _, w := range broker.WithheldOf(records) {
fmt.Printf("%sA GRANT COMPOSES NOTHING: %s\n", indent, w.Said())
}
}
// withheldWords are the plain words of a withheld hearing: the module, the machine and what to do.
func withheldWords(o conditions.Observation) words {
module, m := "a module", machineOr(o, "a machine")
if o.Scope == conditions.ScopeModule && idPart(o, 0) != "" {
module = idPart(o, 0)
}
return words{Headline: clip(conditions.Capital(fmt.Sprintf("%s on %s hears nothing it was granted", module, m)), 60),
Needs: fmt.Sprintf("give %s an account of its own in its manifest; the details say which stream.", module),
Explanation: fmt.Sprintf("You granted %s part of a stream that only modules running under an account of "+
"their own may hear. It runs inside the shared tool runner on %s, which every assistant session there "+
"also uses, so the mesh gives it none of that stream.", module, m),
Resolved: fmt.Sprintf("%s on %s hears what it was granted, or is no longer granted it", module, m)}
}
@@ -1,43 +0,0 @@
package main
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
)
// A grant naming a module the tool runner carries composes nothing, and doctor says so in plain words, naming the
// module, the stream and the remedy (novox/hq ADR 0376 §2).
func TestAGrantThatComposesNothingIsSaid(t *testing.T) {
records := broker.Records{Nodes: []string{"ace"}, Assigned: map[string][]broker.Declared{"ace": {
{Module: broker.RuntimeModule},
{Module: "reader", Hears: []broker.Heard{{Stream: "MAIL", Emitter: "mesh-mail", Restricted: true,
Filters: []string{"mesh.stream.mesh-mail.event.mail.mesh.>"}}}},
{Module: "mailer", RunsAs: "mailer", Hears: []broker.Heard{{Stream: "MAIL", Emitter: "mesh-mail", Restricted: true,
Filters: []string{"mesh.stream.mesh-mail.event.mail.mesh.>"}}}},
}}}
found := withheldObservations(broker.WithheldOf(records))
if len(found) != 1 {
t.Fatalf("said %+v", found)
}
o := found[0]
if o.Kind != kindGrantComposesNothing || o.Machine != "ace" || !strings.Contains(o.Summary, "reader on ace") ||
!strings.Contains(o.Summary, "MAIL") || !strings.Contains(o.Summary, "runs-as") {
t.Fatalf("%+v", o)
}
w := plainWordings[kindGrantComposesNothing](o)
if why, ok := conditions.PlainWords(w, "ace"); !ok {
t.Fatalf("not plain: %s (%+v)", why, w)
}
registered := false
for _, p := range probeRegistry {
if p.ID == probeWithheldID && p.Kind == kindGrantComposesNothing && p.run != nil {
registered = true
}
}
if !registered {
t.Fatal("doctor does not run the probe of a grant that composes nothing")
}
}
+1 -17
View File
@@ -146,23 +146,7 @@ func heldMachines(ctx context.Context, open *stores, names []string) (map[string
same[m] = current[m].Commit same[m] = current[m].Commit
} }
} }
// A move under a module's own `record` that recreates nothing and moves no data goes as a module that rolls why := heldBack(node, modules, same, known, current, plans)
// out goes (novox/hq ADR 0377): a walk's to send, not a person's.
here, copied := current, false
for m, was := range same {
if now, held := current[m]; held && now.OwnRecord() && !f.identical(m, was, now.Commit) && f.rolls(m, was) {
if !copied {
copied = true
here = make(map[string]inventory.CurrentBuild, len(current))
for k, v := range current {
here[k] = v
}
}
now.RollOut = true
here[m] = now
}
}
why := heldBack(node, modules, same, known, here, plans)
// And a build no gate has seen is not carried by a send that does not judge it (ADR 0236). // And a build no gate has seen is not carried by a send that does not judge it (ADR 0236).
for _, mv := range f.moves(node, modules, same, known, false) { for _, mv := range f.moves(node, modules, same, known, false) {
if planStillToSend(plans, mv.Module, node) == "" { if planStillToSend(plans, mv.Module, node) == "" {
-8
View File
@@ -25,7 +25,6 @@ import (
// another send reaches anchor, unreported; the gate still passes. And a send that moves a judged module // another send reaches anchor, unreported; the gate still passes. And a send that moves a judged module
// to another build supersedes the judging: no verdict, nothing put back. // to another build supersedes the judging: no verdict, nothing put back.
func TestReplay352(t *testing.T) { func TestReplay352(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
t.Run("a newer send to the judged machine does not unreport the gate's", testANewerSendDoesNotUnreportTheGatesOwn) t.Run("a newer send to the judged machine does not unreport the gate's", testANewerSendDoesNotUnreportTheGatesOwn)
t.Run("a send that moves the module supersedes the judging", testASendThatMovesTheModuleSupersedesTheJudging) t.Run("a send that moves the module supersedes the judging", testASendThatMovesTheModuleSupersedesTheJudging)
@@ -68,7 +67,6 @@ func testANewerSendDoesNotUnreportTheGatesOwn(t *testing.T) {
// A plan's own first send waits while a release judges the same module on that machine with another build. // A plan's own first send waits while a release judges the same module on that machine with another build.
func TestAPlansFirstSendWaitsForAReleaseJudgingTheModuleThere(t *testing.T) { func TestAPlansFirstSendWaitsForAReleaseJudgingTheModuleThere(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
b := aBacklog(t) b := aBacklog(t)
ctx := t.Context() ctx := t.Context()
@@ -84,7 +82,6 @@ func TestAPlansFirstSendWaitsForAReleaseJudgingTheModuleThere(t *testing.T) {
// A merge plan's judging is superseded the same way: another send moved its module on the first machine. // A merge plan's judging is superseded the same way: another send moved its module on the first machine.
func TestAPlansJudgingIsSupersededByASendThatMovesItsModule(t *testing.T) { func TestAPlansJudgingIsSupersededByASendThatMovesItsModule(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
g := aGateMesh(t) g := aGateMesh(t)
ctx := t.Context() ctx := t.Context()
@@ -143,7 +140,6 @@ func testASendThatMovesTheModuleSupersedesTheJudging(t *testing.T) {
// A report is on the gate's own send: the declaration itself, or one sequenced after it; a gate kept // A report is on the gate's own send: the declaration itself, or one sequenced after it; a gate kept
// without its send reads the report against the send made last, as before. Pure. // without its send reads the report against the send made last, as before. Pure.
func TestAReportIsHeldAgainstTheGatesOwnSend(t *testing.T) { func TestAReportIsHeldAgainstTheGatesOwnSend(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
sent := inventory.SentDeclaration{Digest: "d-490", Sequence: 490} sent := inventory.SentDeclaration{Digest: "d-490", Sequence: 490}
for _, c := range []struct { for _, c := range []struct {
r inventory.Reported r inventory.Reported
@@ -189,7 +185,6 @@ func TestAReportIsHeldAgainstTheGatesOwnSend(t *testing.T) {
// A gate judges only the build the machine was last sent: last sent another build of the module, the // A gate judges only the build the machine was last sent: last sent another build of the module, the
// judging is superseded, whatever the machine reports. Pure. // judging is superseded, whatever the machine reports. Pure.
func TestAGateJudgesOnlyTheBuildTheMachineWasLastSent(t *testing.T) { func TestAGateJudgesOnlyTheBuildTheMachineWasLastSent(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
at := time.Now() at := time.Now()
f := gateFacts{now: at, reports: map[string]inventory.Reported{"anchor": {Node: "anchor", Outcome: inventory.OutcomeApplied, f := gateFacts{now: at, reports: map[string]inventory.Reported{"anchor": {Node: "anchor", Outcome: inventory.OutcomeApplied,
At: &at, Current: true}}, engines: map[string]string{}, served: map[string]served{}, rolledBack: map[string][]lease.Rollback{}, At: &at, Current: true}}, engines: map[string]string{}, served: map[string]served{}, rolledBack: map[string][]lease.Rollback{},
@@ -217,7 +212,6 @@ func TestAGateJudgesOnlyTheBuildTheMachineWasLastSent(t *testing.T) {
// A move of another build of a module to a machine where a release or a plan is judging that module // A move of another build of a module to a machine where a release or a plan is judging that module
// waits for that judging; the same build to that machine is already there. Pure. // waits for that judging; the same build to that machine is already there. Pure.
func TestAWalkWaitsForAJudgingOfTheSameModuleOnThatMachine(t *testing.T) { func TestAWalkWaitsForAJudgingOfTheSameModuleOnThatMachine(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
at := time.Now() at := time.Now()
release := inventory.Plan{ID: "release-1", State: inventory.PlanRolling, Release: &inventory.PlanRelease{ release := inventory.Plan{ID: "release-1", State: inventory.PlanRolling, Release: &inventory.PlanRelease{
Gate: &inventory.PlanGate{Machines: []string{"novox"}, Carried: []inventory.CarriedMove{ Gate: &inventory.PlanGate{Machines: []string{"novox"}, Carried: []inventory.CarriedMove{
@@ -255,7 +249,6 @@ func TestAWalkWaitsForAJudgingOfTheSameModuleOnThatMachine(t *testing.T) {
// The controller is never put back to a build that reaches less of the store's schema than the store // The controller is never put back to a build that reaches less of the store's schema than the store
// has, or to one that never said what it reaches: the current build is kept, and the condition says so. // has, or to one that never said what it reaches: the current build is kept, and the condition says so.
func TestTheControllerIsNotPutBackToABuildOlderThanTheStoresSchema(t *testing.T) { func TestTheControllerIsNotPutBackToABuildOlderThanTheStoresSchema(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
inv := open.inventory inv := open.inventory
@@ -352,7 +345,6 @@ func TestTheControllerIsNotPutBackToABuildOlderThanTheStoresSchema(t *testing.T)
// The lease's holder names the build the declaration told it it is, and the version stamp only without one. // The lease's holder names the build the declaration told it it is, and the version stamp only without one.
func TestTheHolderNamesTheBuildTheDeclarationToldIt(t *testing.T) { func TestTheHolderNamesTheBuildTheDeclarationToldIt(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
t.Setenv(RunningBuildVar, " ad62528c47c7 ") t.Setenv(RunningBuildVar, " ad62528c47c7 ")
if h := holderOf("x"); h.Build != "ad62528c47c7" { if h := holderOf("x"); h.Build != "ad62528c47c7" {
t.Fatalf("the holder's build is %q", h.Build) t.Fatalf("the holder's build is %q", h.Build)
+5 -6
View File
@@ -20,8 +20,8 @@ func TestANoMoveTierWaitsForNoNewReport(t *testing.T) {
ctx := t.Context() ctx := t.Context()
inv := g.open.inventory inv := g.open.inventory
// app holds the build seat, so a module of the next tier is built by it; since novox/hq ADR 0360 §3 that gates // app is the build machine, so a module of the next tier is built by it: the tier gates the next on app
// nothing. // running on its machines.
for _, m := range []catalogue.Manifest{ for _, m := range []catalogue.Manifest{
{Module: "app", Version: "c2", Claims: []catalogue.Claim{{Name: "node-build-agent", Scope: catalogue.ScopeNode}}}, {Module: "app", Version: "c2", Claims: []catalogue.Claim{{Name: "node-build-agent", Scope: catalogue.ScopeNode}}},
{Module: "web", Version: "w1"}, {Module: "web", Version: "w1"},
@@ -71,8 +71,7 @@ func TestANoMoveTierWaitsForNoNewReport(t *testing.T) {
t.Fatalf("the walk waits on a no-move tier for a report nothing will cause: tier %d, %s %q", p.Tier, p.State, p.Note) t.Fatalf("the walk waits on a no-move tier for a report nothing will cause: tier %d, %s %q", p.Tier, p.State, p.Note)
} }
// A machine that has not answered its last send is not waited on either: a built-by edge gates no tier // A machine that has not answered its last send is still waited on: no move is not "never sent".
// (novox/hq ADR 0360 §3). Before, the walk waited for app on laptop.
p.Tier, p.State, p.Note = 0, inventory.PlanRolling, "" p.Tier, p.State, p.Note = 0, inventory.PlanRolling, ""
if err := inv.RecordSent(ctx, nodeID(t, g.open, "laptop"), "d-laptop-later", map[string]string{"app": "c1"}); err != nil { if err := inv.RecordSent(ctx, nodeID(t, g.open, "laptop"), "d-laptop-later", map[string]string{"app": "c1"}); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -82,7 +81,7 @@ func TestANoMoveTierWaitsForNoNewReport(t *testing.T) {
} }
advancePlans(ctx, g.open) advancePlans(ctx, g.open)
p = g.plan(t) p = g.plan(t)
if p.Tier < 1 || strings.Contains(p.Note, "waiting for app") { if p.Tier != 0 || !strings.Contains(p.Note, "waiting for app on laptop") {
t.Fatalf("the walk waits on a built-by edge: tier %d, %q", p.Tier, p.Note) t.Fatalf("a machine that has not answered its last send was not waited on: tier %d, %q", p.Tier, p.Note)
} }
} }
-188
View File
@@ -1,188 +0,0 @@
package main
import (
"context"
"slices"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// novox/hq ADR 0360 §4, §5: a walk ends on its machines' reports — every machine sent has reported on its send,
// or is silent past ApplySilentAfter — and names a failed or refused apply; it never waits for a health reading.
func TestAWalkEndsOnItsMachinesReports(t *testing.T) {
now := time.Date(2026, 10, 14, 12, 0, 0, 0, time.UTC)
sent := now.Add(-time.Minute)
long := now.Add(-inventory.ApplySilentAfter - time.Minute)
at := now.Add(-10 * time.Second)
p := inventory.Plan{Modules: map[string]*inventory.PlanModule{
"app": {Rest: map[string]inventory.SentDeclaration{
"ace": {Digest: "d-ace", At: sent},
"novox": {Digest: "d-novox", At: sent},
"shanks": {Digest: "d-shanks", At: sent},
"laptop": {Digest: "d-laptop", At: long},
}},
"proxy": {Rest: map[string]inventory.SentDeclaration{"ace": {Digest: "d-ace", At: sent}}},
}}
reports := []inventory.Reported{
{Node: "ace", Outcome: inventory.OutcomeApplied, At: &at, Declared: "d-ace"},
{Node: "novox", Outcome: inventory.OutcomeFailed, At: &at, Declared: "d-novox"},
{Node: "shanks", Outcome: inventory.OutcomeApplied, At: &at, Declared: "d-before"},
}
waiting, failed, silent := walkReports(p, reports, now)
if !slices.Equal(waiting, []string{"shanks"}) || !slices.Equal(failed, []string{"novox"}) ||
!slices.Equal(silent, []string{"laptop"}) {
t.Fatalf("waiting %v, failed %v, silent %v", waiting, failed, silent)
}
reports[2].Declared = "d-shanks"
if waiting, _, _ := walkReports(p, reports, now); len(waiting) != 0 {
t.Fatalf("the walk still waits once every machine reported: %v", waiting)
}
}
// withTheGate turns the first-machine gate on for a test of its own path: kept for the walks a controller before
// novox/hq ADR 0360 started.
func withTheGate(t *testing.T) {
t.Helper()
was := gateFirstMachine
gateFirstMachine = true
t.Cleanup(func() { gateFirstMachine = was })
}
// Every machine heard from is sent at once and passed on its applied report, with no judging (novox/hq ADR 0360 §7);
// before, the first machine was judged before the rest were sent (ADR 0353 §1).
func TestAReleaseSendsEveryMachineAtOnce(t *testing.T) {
judgeLeavesAsTheCore(t)
b := aBacklogOnThree(t)
ctx := t.Context()
said := events(t)
for i := 0; i < 12; i++ {
advancePlans(ctx, b.open)
}
p := b.release(t)
if p.State != inventory.PlanDone || len(p.Release.Done) != 3 {
t.Fatalf("the release is %s: %s %+v", p.State, p.Note, p.Release)
}
for _, n := range []string{"desk", "laptop", "anchor"} {
if !slices.Contains(*said, "send "+n) {
t.Fatalf("%s was not sent: %v", n, *said)
}
}
if slices.Contains(*said, "judge") {
t.Fatalf("a machine was judged by its health: %v", *said)
}
if v, found, err := b.open.inventory.GateOf(ctx, "build-late-c2"); err != nil || !found || v.Verdict != inventory.GatePassed {
t.Fatalf("late's pass on laptop was not kept: %+v %v %v", v, found, err)
}
}
// A machine that reports its apply failed is named and fails the release, with nothing put back, and every other
// machine goes on to its own report (novox/hq ADR 0360 §5, §7); before, it was put back (ADR 0353 §3).
func TestAMachineThatFailsItsApplyIsNamedAndNothingIsPutBack(t *testing.T) {
judgeLeavesAsTheCore(t)
b := aBacklogOnThree(t)
ctx := t.Context()
inv := b.open.inventory
b.broken["desk"] = true // desk fails what it is sent
advancePlans(ctx, b.open)
for i := 0; i < 12 && b.release(t).Open(); i++ {
advancePlans(ctx, b.open)
}
p := b.release(t)
if p.State != inventory.PlanFailed || !slices.Equal(p.Release.Failed, []string{"desk"}) {
t.Fatalf("the release is %s: %s %+v", p.State, p.Note, p.Release)
}
if !slices.Contains(p.Release.Done, "laptop") || !slices.Contains(p.Release.Done, "anchor") {
t.Fatalf("the others have no verdict of their own: %+v", p.Release)
}
if v, found, err := inv.GateOf(ctx, "build-late-c2"); err != nil || !found || v.Verdict != inventory.GatePassed {
t.Fatalf("late's pass on laptop was not kept: %+v %v %v", v, found, err)
}
if sent, _, _ := inv.SentBuilds(ctx, "desk"); sent["app"] != "c2" {
t.Fatalf("desk was put back: %v", sent)
}
}
// A release's send kept before it was made, and made, is judged by the controller that takes over without being
// made again (ADR 0353 §8), with every machine sent at once (ADR 0360 §7).
func TestAReleaseSendKeptAndMadeIsNotMadeAgainAtOnce(t *testing.T) {
judgeLeavesAsTheCore(t)
b := aBacklog(t)
ctx := t.Context()
inv := b.open.inventory
p, err := releaseBacklog(ctx, b.open, "")
if err != nil || p == nil {
t.Fatalf("no release opened: %v", err)
}
if _, err := advanceRelease(ctx, b.open, p); err != nil || p.Release.Gates["anchor"] == nil || !p.Release.Gates["anchor"].Pending {
t.Fatalf("the send was not kept before it was made: %+v %v", p.Release, err)
}
if err := inv.SavePlan(ctx, p); err != nil {
t.Fatal(err)
}
// The send is made, and the controller is replaced before it keeps it.
if _, err := sendRollout(ctx, b.open, []string{"anchor"}); err != nil {
t.Fatal(err)
}
advancePlans(ctx, b.open)
if n := len(slices.DeleteFunc(slices.Clone(b.sent), func(s []string) bool { return !slices.Equal(s, []string{"anchor"}) })); n != 1 {
t.Fatalf("anchor was sent again: %v", b.sent)
}
g := b.release(t).Release.Gates["anchor"]
if g == nil || g.Pending || g.Since == nil || len(g.Carried) == 0 {
t.Fatalf("anchor is not judged on the send made: %+v", g)
}
}
// A machine of the rest that says nothing on its send is left silent after fifteen minutes: not failed, not put
// back; a rest that went silent whole ends failed, said as a condition (ADR 0353 §4); its step also sent anchor, so the release is done, not failed (ADR 0360 §7).
func TestAMachineSilentPastItsBoundIsLeftAndSaidAtOnce(t *testing.T) {
judgeLeavesAsTheCore(t)
b := aBacklog(t)
ctx := t.Context()
inv := b.open.inventory
reporting := sendRollout
sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) {
if slices.Equal(names, []string{"laptop"}) { // laptop sleeps: sent, and says nothing
b.sent = append(b.sent, names)
current, _ := open.inventory.CurrentBuilds(ctx)
return names, open.inventory.RecordSent(ctx, nodeID(t, open, "laptop"), "d-laptop-asleep",
map[string]string{"app": current["app"].Commit, "late": current["late"].Commit, "same": current["same"].Commit})
}
return reporting(ctx, open, names)
}
t.Cleanup(func() { sendRollout = reporting })
for i := 0; i < 8; i++ {
advancePlans(ctx, b.open)
}
p := b.release(t)
g := p.Release.Gates["laptop"]
if !p.Open() || g == nil || g.Verdict != "" || g.Silent {
t.Fatalf("laptop is judged before fifteen minutes: %s %s %+v", p.State, p.Note, g)
}
since := time.Now().Add(-inventory.ApplySilentAfter - time.Minute)
g.Since = &since
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
advancePlans(ctx, b.open)
p = b.release(t)
// anchor applied in the same step, so the release is done, laptop named silent (novox/hq ADR 0360 §7).
if p.State != inventory.PlanDone || !slices.Equal(p.Release.Silent, []string{"laptop"}) || len(p.Release.Failed) > 0 {
t.Fatalf("the release is %s: %s %+v", p.State, p.Note, p.Release)
}
if failed, _ := inv.GateFailed(ctx, "build-late-c2"); failed {
t.Fatal("a silent machine's build was marked failed")
}
if sent, _, _ := inv.SentBuilds(ctx, "laptop"); sent["late"] != "c2" {
t.Fatalf("laptop was put back: %v", sent)
}
obs, err := silentObservation(ctx, inv)
if err != nil {
t.Fatal(err)
}
if len(obs) != 1 || obs[0].Kind != "release-silent" || !slices.Equal(obs[0].Also, []string{"laptop"}) {
t.Fatalf("the silence is not said: %+v", obs)
}
}
+17 -3
View File
@@ -550,8 +550,17 @@ func shelfWithChange(base map[string]catalogue.Manifest, sources map[string]inve
out[k] = m out[k] = m
} }
var failures []string var failures []string
// The list of predefined keys at the tree's root (novox/hq ADR 0348 §7): what the build of this tree judges each
// manifest against and registration publishes, so what the gate judges each against too. A list that is
// wrong is refused by name, and every manifest is then judged against none.
predefined, err := catalogue.ReadPredefined(tree)
if err != nil {
failures = append(failures, fmt.Sprintf("%s: the controller judging this (%s) refuses it — %s",
catalogue.PredefinedFile, version, oneLine(err.Error())))
predefined = nil
}
found := map[string]string{} // module → its directory in the tree found := map[string]string{} // module → its directory in the tree
err := filepath.WalkDir(tree, func(path string, d fs.DirEntry, err error) error { err = filepath.WalkDir(tree, func(path string, d fs.DirEntry, err error) error {
if err != nil { if err != nil {
return err return err
} }
@@ -575,7 +584,12 @@ func shelfWithChange(base map[string]catalogue.Manifest, sources map[string]inve
if err != nil { if err != nil {
return err return err
} }
m, err := catalogue.ParseManifest(raw) m, err := catalogue.ParseManifestWith(raw, predefined)
if err == nil {
// Each template's text from the tree beside the manifest, as the build reads it back from the archive it
// packs (novox/hq ADR 0349 §2): so a change to a template alone is a change to its module.
m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(path))
}
if err != nil { if err != nil {
if inScope(dir) { if inScope(dir) {
failures = append(failures, fmt.Sprintf("%s: the controller judging this (%s) refuses it — %s", failures = append(failures, fmt.Sprintf("%s: the controller judging this (%s) refuses it — %s",
@@ -605,7 +619,7 @@ func shelfWithChange(base map[string]catalogue.Manifest, sources map[string]inve
// As registration reads a built manifest: strictly, again (takeIn). // As registration reads a built manifest: strictly, again (takeIn).
var raw []byte var raw []byte
if raw, err = json.Marshal(resolved); err == nil { if raw, err = json.Marshal(resolved); err == nil {
resolved, err = catalogue.ParseManifest(raw) resolved, err = catalogue.ParseManifestWith(raw, predefined)
} }
} }
if err == nil { if err == nil {
+14
View File
@@ -33,6 +33,20 @@ import (
func aMesh(t *testing.T) *stores { func aMesh(t *testing.T) *stores {
t.Helper() t.Helper()
inventory.ForTest(t) // raises the store, migrates it, and points the environment at it inventory.ForTest(t) // raises the store, migrates it, and points the environment at it
return aMeshOnTheStore(t)
}
// aMeshBefore is aMesh on a store migrated only below migration `number`, with the function that runs the
// rest: for a test of what a migration that moves data does to what the mesh composes (novox/hq issue 651).
func aMeshBefore(t *testing.T, number int) (*stores, func()) {
t.Helper()
_, migrate := inventory.ForTestBefore(t, number)
return aMeshOnTheStore(t), migrate
}
// aMeshOnTheStore is the mesh aMesh makes, on the store the environment points at.
func aMeshOnTheStore(t *testing.T) *stores {
t.Helper()
licences.ForTest(t) // planning reaches this one too, by name and never by connection licences.ForTest(t) // planning reaches this one too, by name and never by connection
open, err := openStores(t.Context()) open, err := openStores(t.Context())
+3 -96
View File
@@ -7,10 +7,7 @@ import (
"fmt" "fmt"
"os/exec" "os/exec"
"slices" "slices"
"sort"
"strings" "strings"
"sync"
"time"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link" "github.com/novox/mesh-controller/internal/link"
@@ -126,82 +123,8 @@ func answerMeshCLI(inv *inventory.Inventory) link.CLIHandler {
// cliJournal says one line in the controller's journal (its standard output); a variable so a test can read it. // cliJournal says one line in the controller's journal (its standard output); a variable so a test can read it.
var cliJournal = func(line string) { fmt.Println(line) } var cliJournal = func(line string) { fmt.Println(line) }
// onlyReads says a line only reads, judged by the generic `command` verb's reading forms (novox/hq ADR 0266) — the // runForMeshCLI runs a judged line and answers what it said. Every line is said in the journal first: its call,
// same allow list that decides what an ordinary call may run, never a second list kept here — judged on the line's // who asked on which node, its command word only, and how it runs (review of ADR 0272).
// every word, so `plans go`, `conditions silence` and `settings set` are not reads while `plans`, `conditions` and
// `settings show` are. A line those forms do not name is not judged a read, and is said (novox/hq issue 762).
func onlyReads(asked link.CLIAsked, v cliVerdict) bool {
if v.refused != "" || len(asked.Line) == 0 || len(asked.Stdin) > 0 || cliServers[asked.Line[0]] {
return false
}
return commandReads(asked.Line) == nil && terminalOnly(asked.Line) == nil
}
// readCalls counts the read calls said in no line, per caller, until the next hourly line says them.
type readCalls struct {
mu sync.Mutex
counts map[string]int
}
// cliReads are this controller's read calls not yet said.
var cliReads = &readCalls{}
func (r *readCalls) add(caller string) {
r.mu.Lock()
defer r.mu.Unlock()
if r.counts == nil {
r.counts = map[string]int{}
}
r.counts[caller]++
}
// said is the line that says the read calls counted since the last, and starts counting again; empty when there
// were none, so an idle hour says nothing.
func (r *readCalls) said(over time.Duration) string {
r.mu.Lock()
counts := r.counts
r.counts = nil
r.mu.Unlock()
if len(counts) == 0 {
return ""
}
callers := make([]string, 0, len(counts))
for c := range counts {
callers = append(callers, c)
}
sort.Strings(callers)
parts := make([]string, len(callers))
for i, c := range callers {
parts[i] = fmt.Sprintf("%s %d", c, counts[c])
}
return fmt.Sprintf("mesh-cli read calls in the last %s, said in no line of their own: %s", over,
strings.Join(parts, ", "))
}
// readCallsEvery is how often the count of read calls is said.
const readCallsEvery = time.Hour
// sayReadCalls says, every readCallsEvery until ctx ends, how many read calls each caller made — when any did — so
// a runaway poller stays visible while its calls write no line each (novox/hq issue 762).
func sayReadCalls(ctx context.Context) {
tick := time.NewTicker(readCallsEvery)
defer tick.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
if line := cliReads.said(readCallsEvery); line != "" {
cliJournal(line)
}
}
}
}
// runForMeshCLI runs a judged line and answers what it said. Every line that changes something, is refused or fails
// is said in the journal: its call, who asked on which node, its command word only, and how it runs (review of ADR
// 0272). A line that only reads and succeeds is counted instead, and the count said hourly (novox/hq issue 762): a
// page polling `plans` every few seconds would otherwise write thousands of lines a day.
func runForMeshCLI(ctx context.Context, node string, asked link.CLIAsked, v cliVerdict) link.CLIAnswer { func runForMeshCLI(ctx context.Context, node string, asked link.CLIAsked, v cliVerdict) link.CLIAnswer {
how := "as an ordinary call" how := "as an ordinary call"
switch { switch {
@@ -214,23 +137,7 @@ func runForMeshCLI(ctx context.Context, node string, asked link.CLIAsked, v cliV
if call == "" { if call == "" {
call = "(no call)" call = "(no call)"
} }
line := fmt.Sprintf("mesh-cli %s: %s on %s asked %q, %s", call, asked.Account, node, asked.Line[0], how) cliJournal(fmt.Sprintf("mesh-cli %s: %s on %s asked %q, %s", call, asked.Account, node, asked.Line[0], how))
if !onlyReads(asked, v) {
cliJournal(line)
return runJudgedLine(ctx, node, asked, v)
}
answer := runJudgedLine(ctx, node, asked, v)
if answer.Exit != 0 || answer.Refused != "" {
// A read that was refused or failed is said, as a change is.
cliJournal(fmt.Sprintf("%s, and it failed (exit %d)", line, answer.Exit))
} else {
cliReads.add(fmt.Sprintf("%s on %s", asked.Account, node))
}
return answer
}
// runJudgedLine runs a line runForMeshCLI has said or counted.
func runJudgedLine(ctx context.Context, node string, asked link.CLIAsked, v cliVerdict) link.CLIAnswer {
if v.refused != "" { if v.refused != "" {
return link.CLIRefusal(v.refused) return link.CLIRefusal(v.refused)
} }
-123
View File
@@ -1,123 +0,0 @@
package main
import (
"context"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// echoFails makes the echoing test binary end with a failure, as a read that failed would.
const echoFails = "MESH_TEST_ECHO_FAILS"
// journalled runs lines through runForMeshCLI and answers what the journal said, with a fresh count of read calls.
func journalled(t *testing.T) *[]string {
t.Helper()
t.Setenv(echoEnvironment, "1")
said := &[]string{}
was, wasReads := cliJournal, cliReads
cliJournal = func(line string) { *said = append(*said, line) }
cliReads = &readCalls{}
t.Cleanup(func() { cliJournal, cliReads = was, wasReads })
return said
}
// **A call that only reads writes no journal line; one that changes something does** (novox/hq issue 762).
func TestAReadCallWritesNoJournalLine(t *testing.T) {
said := journalled(t)
ctx := link.WithCallID(context.Background(), "call-1")
ordinary := cliVerdict{why: "not the terminal"}
for _, line := range [][]string{{"plans"}, {"status"}, {"conditions"}, {"node", "list"}, {"builds"}, {"queue"},
{"settings", "show", "zsh"}, {"plans", "--json"}} {
if a := runForMeshCLI(ctx, "g14", cliAsked("jochen", 1000, line...), ordinary); a.Exit != 0 || a.Refused != "" {
t.Fatalf("%q did not run: %+v", line, a)
}
}
runForMeshCLI(ctx, "control", cliAsked("jochen", 1000, "plans"), cliVerdict{terminal: true, why: "the terminal"})
if len(*said) != 0 {
t.Fatalf("read calls wrote journal lines: %q", *said)
}
runForMeshCLI(ctx, "control", cliAsked("jochen", 1000, "assign", "g14", "zsh"), cliVerdict{terminal: true, why: "the terminal"})
if len(*said) != 1 || !strings.Contains((*said)[0], `asked "assign"`) {
t.Fatalf("a call that changes something was not said: %q", *said)
}
}
// **A refused or failed read keeps its line** (novox/hq issue 762).
func TestAFailedOrRefusedReadIsSaid(t *testing.T) {
said := journalled(t)
runForMeshCLI(context.Background(), "g14", cliAsked("root", 0, "plans"), cliVerdict{refused: "root is not answered"})
t.Setenv(echoFails, "1")
a := runForMeshCLI(context.Background(), "g14", cliAsked("jochen", 1000, "plans"), cliVerdict{why: "not the terminal"})
if a.Exit == 0 {
t.Fatalf("the read did not fail: %+v", a)
}
if len(*said) != 2 || !strings.Contains((*said)[0], "refused") || !strings.Contains((*said)[1], "failed (exit 3)") {
t.Fatalf("the journal said %q", *said)
}
if line := cliReads.said(readCallsEvery); line != "" {
t.Fatalf("a failed read was counted as a read: %q", line)
}
}
// **A verb is classed by what the call does, not its name** (novox/hq issue 762): the same command word reads with
// some arguments and writes with others, and a line no reading form names is said.
func TestAVerbIsClassedByItsArguments(t *testing.T) {
ordinary := cliVerdict{why: "not the terminal"}
for _, c := range []struct {
line []string
reads bool
}{
{[]string{"settings", "show", "zsh"}, true},
{[]string{"settings", "preferences"}, true},
{[]string{"settings", "set", "zsh", `{"a":1}`}, false},
{[]string{"settings", "clear", "zsh"}, false},
{[]string{"plans"}, true},
{[]string{"plans", "p-1"}, true},
{[]string{"plans", "go", "p-1"}, false},
{[]string{"plans", "--json", "stop", "p-1"}, false},
{[]string{"plans", "close", "p-1"}, false},
{[]string{"plans", "retry", "p-1"}, false},
{[]string{"conditions"}, true},
{[]string{"conditions", "show", "x"}, true},
{[]string{"conditions", "silence", "x"}, false},
{[]string{"doctor", "run"}, false},
{[]string{"node", "account", "g14", "x"}, false},
{[]string{"assign", "g14", "zsh"}, false},
{[]string{"a-command-added-later"}, false},
{[]string{"serve"}, false},
} {
if got := onlyReads(cliAsked("jochen", 1000, c.line...), ordinary); got != c.reads {
t.Errorf("%q judged a read: %v, want %v", c.line, got, c.reads)
}
}
if onlyReads(cliAsked("jochen", 1000, "plans"), cliVerdict{refused: "no"}) {
t.Error("a refused line judged a read")
}
stdin := cliAsked("jochen", 1000, "status")
stdin.Stdin = []byte("x")
if onlyReads(stdin, cliVerdict{terminal: true}) {
t.Error("a line carrying standard input judged a read")
}
}
// **Once an hour the read calls are counted per caller, only if there were any** (novox/hq issue 762).
func TestTheReadCallsAreCountedPerCaller(t *testing.T) {
said := journalled(t)
ordinary := cliVerdict{why: "not the terminal"}
for range 3 {
runForMeshCLI(context.Background(), "g14", cliAsked("jochen", 1000, "plans"), ordinary)
}
runForMeshCLI(context.Background(), "ace", cliAsked("jochen", 1000, "status"), ordinary)
if len(*said) != 0 {
t.Fatalf("read calls wrote lines: %q", *said)
}
line := cliReads.said(readCallsEvery)
if !strings.Contains(line, "jochen on ace 1, jochen on g14 3") || !strings.Contains(line, "1h0m0s") {
t.Fatalf("the hourly line said %q", line)
}
if again := cliReads.said(readCallsEvery); again != "" {
t.Fatalf("an hour with no read calls said %q", again)
}
}
+1 -1
View File
@@ -86,7 +86,7 @@ func owedToTheRecord(ctx context.Context, announced merges, m link.SourceMoved,
// catchingUpOnMerges reads back the forge's announcements on a timer, until the context ends. // catchingUpOnMerges reads back the forge's announcements on a timer, until the context ends.
func catchingUpOnMerges(ctx context.Context, open *stores, announced merges) { func catchingUpOnMerges(ctx context.Context, open *stores, announced merges) {
f := following{open: open} f := following{open}
catalogued := func(ctx context.Context) ([]inventory.Entry, map[string][]inventory.ReadRepository, error) { catalogued := func(ctx context.Context) ([]inventory.Entry, map[string][]inventory.ReadRepository, error) {
entries, err := open.inventory.Catalogued(ctx) entries, err := open.inventory.Catalogued(ctx)
if err != nil { if err != nil {
-15
View File
@@ -240,7 +240,6 @@ func judgeModuleHealth(ctx context.Context, inv *inventory.Inventory, k *conditi
continue continue
} }
o := moduleUnhealthyObservation(m, node, unhealthy[m]) o := moduleUnhealthyObservation(m, node, unhealthy[m])
declaredSeverity(&o, hold, m)
if hold != nil { if hold != nil {
if by, held := hold.heldWith(node, m, unhealthy[m]); held { if by, held := hold.heldWith(node, m, unhealthy[m]); held {
// Held under the provider's condition: nothing of its own, and the provider's says it waits. The // Held under the provider's condition: nothing of its own, and the provider's says it waits. The
@@ -348,7 +347,6 @@ func sayWaiters(ctx context.Context, k *conditions.Keeper, hold *holding, p cata
} else { } else {
_, rest := besideAWait(p.Module, rs) _, rest := besideAWait(p.Module, rs)
o = moduleUnhealthyObservation(p.Module, p.Node, rest) o = moduleUnhealthyObservation(p.Module, p.Node, rest)
declaredSeverity(&o, hold, p.Module)
if said, waits := personWait(p.Module, p.Node, rest); waits { if said, waits := personWait(p.Module, p.Node, rest); waits {
o = reloginObservation(p.Module, p.Node, said, operatorOn(ctx, hold.inv, p.Node), rest) o = reloginObservation(p.Module, p.Node, said, operatorOn(ctx, hold.inv, p.Node), rest)
} }
@@ -483,19 +481,6 @@ func sayWaitingOn(o *conditions.Observation, waiters []string) {
o.Explanation += fmt.Sprintf(" %d module(s) that depend on it wait for it.", len(waiters)) o.Explanation += fmt.Sprintf(" %d module(s) that depend on it wait for it.", len(waiters))
} }
// declaredSeverity raises a module's unhealthy condition urgent when its manifest declares `unhealthy` urgent
// (novox/hq ADR 0355 §1). Only that condition changes (ADR 0355 §4): a wait for a new login, a module that needs
// the operator and a directory used as found keep their severities, and without the field, or without a manifest
// to read it from, it stays the warning it was.
func declaredSeverity(o *conditions.Observation, hold *holding, module string) {
if o.Kind != kindModuleUnhealthy || hold == nil {
return
}
if m, ok := hold.manifestOf(module); ok && m.UnhealthyIsUrgent() {
o.Severity = conditions.Urgent
}
}
// moduleUnhealthyObservation is a module unhealthy on a machine, in words: the summary names the module, // moduleUnhealthyObservation is a module unhealthy on a machine, in words: the summary names the module,
// the machine and what is wrong with each resource; the detail — targets, streaks, since — is evidence. // the machine and what is wrong with each resource; the detail — targets, streaks, since — is evidence.
func moduleUnhealthyObservation(module, node string, rs []inventory.ResourceHealth) conditions.Observation { func moduleUnhealthyObservation(module, node string, rs []inventory.ResourceHealth) conditions.Observation {
+9
View File
@@ -8,6 +8,7 @@ import (
"fmt" "fmt"
"net" "net"
"os" "os"
"path/filepath"
"sort" "sort"
"strings" "strings"
"time" "time"
@@ -127,6 +128,10 @@ func moduleCommand(ctx context.Context, args []string) error {
if err != nil { if err != nil {
return err return err
} }
// A manifest handed over by hand takes each template's text from the tree beside it (novox/hq ADR 0349 §2).
if m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(positionals[0])); err != nil {
return err
}
from, err := whereItComesFrom(*repo, *ref, *commit, *path, *self) from, err := whereItComesFrom(*repo, *ref, *commit, *path, *self)
if err != nil { if err != nil {
return err return err
@@ -537,6 +542,10 @@ func settingsCommand(ctx context.Context, args []string) error {
if p.SetBy != "" { if p.SetBy != "" {
by = "; " + p.SetBy by = "; " + p.SetBy
} }
if p.Why != "" {
// A layer whose placement moved out of it (migration 0097): still the layer, as it stands.
by += "; " + p.Why
}
fmt.Printf("%s on %s, until %s (%s%s):\n %s\n", positionals[0], where, fmt.Printf("%s on %s, until %s (%s%s):\n %s\n", positionals[0], where,
p.ReplacedAt.Local().Format("2006-01-02 15:04:05"), p.ReplacedBy, by, shown) p.ReplacedAt.Local().Format("2006-01-02 15:04:05"), p.ReplacedBy, by, shown)
} }
-7
View File
@@ -30,7 +30,6 @@ func userUnit(module, account string) inventory.ResourceHealth {
// Only what depends on the new login alone is a wait; anything else unhealthy of the module is judged as // Only what depends on the new login alone is a wait; anything else unhealthy of the module is judged as
// before, so that a fault is never excused. // before, so that a fault is never excused.
func TestOnlyWhatDependsOnTheNewLoginIsAWait(t *testing.T) { func TestOnlyWhatDependsOnTheNewLoginIsAWait(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
notInGroup := relogin("lights", "operator") notInGroup := relogin("lights", "operator")
notInGroup.Reason = "not in the group lights: the apply has not put operator there; its outcome says why" notInGroup.Reason = "not in the group lights: the apply has not put operator there; its outcome says why"
systemUnit := userUnit("lights", "operator") systemUnit := userUnit("lights", "operator")
@@ -75,7 +74,6 @@ func TestOnlyWhatDependsOnTheNewLoginIsAWait(t *testing.T) {
// The gate reads a wait for a person as its own reading, never a fault, and passes with it; the same module // The gate reads a wait for a person as its own reading, never a fault, and passes with it; the same module
// with a container down beside it is not yet healthy, as before. // with a container down beside it is not yet healthy, as before.
func TestAWaitForAPersonIsAPassCarriedAlong(t *testing.T) { func TestAWaitForAPersonIsAPassCarriedAlong(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
now := time.Now() now := time.Now()
since := now.Add(-time.Minute) since := now.Add(-time.Minute)
account := relogin("lights", "operator") account := relogin("lights", "operator")
@@ -104,7 +102,6 @@ func TestAWaitForAPersonIsAPassCarriedAlong(t *testing.T) {
// The module's condition says the wait in one sentence for a person, and clears on the first statement that // The module's condition says the wait in one sentence for a person, and clears on the first statement that
// no longer says it — said as the module's own fault when its unit still fails after the new login. // no longer says it — said as the module's own fault when its unit still fails after the new login.
func TestTheReloginConditionSaysTheWaitAndClearsAfterTheLogin(t *testing.T) { func TestTheReloginConditionSaysTheWaitAndClearsAfterTheLogin(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
k, _ := withConditionsInMemory(t) k, _ := withConditionsInMemory(t)
@@ -161,7 +158,6 @@ func TestTheReloginConditionSaysTheWaitAndClearsAfterTheLogin(t *testing.T) {
// A module that is healthy on its own keeps its pass when another module of the same send fails at the // A module that is healthy on its own keeps its pass when another module of the same send fails at the
// bound: it is neither put back nor left without a verdict for every other walk to wait on. // bound: it is neither put back nor left without a verdict for every other walk to wait on.
func TestAModuleHealthyOnItsOwnKeepsItsPassWhenItsSendFails(t *testing.T) { func TestAModuleHealthyOnItsOwnKeepsItsPassWhenItsSendFails(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
b := aBacklog(t) b := aBacklog(t)
ctx := t.Context() ctx := t.Context()
inv := b.open.inventory inv := b.open.inventory
@@ -211,7 +207,6 @@ func TestAModuleHealthyOnItsOwnKeepsItsPassWhenItsSendFails(t *testing.T) {
// What the operator reads of the wait (ADR 0253, ADR 0254): it needs them, so it is never quiet, and it offers // What the operator reads of the wait (ADR 0253, ADR 0254): it needs them, so it is never quiet, and it offers
// no button, since nothing but their own new login can do it. // no button, since nothing but their own new login can do it.
func TestTheReloginConditionNeedsTheOperatorAndOffersNoButton(t *testing.T) { func TestTheReloginConditionNeedsTheOperatorAndOffersNoButton(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
o := reloginObservation("openrazer", "g14", "relogin needed on g14: …", "operator", []inventory.ResourceHealth{ o := reloginObservation("openrazer", "g14", "relogin needed on g14: …", "operator", []inventory.ResourceHealth{
relogin("openrazer", "operator"), userUnit("openrazer", "operator")}) relogin("openrazer", "operator"), userUnit("openrazer", "operator")})
plainExample(t, o, "openrazer waits for a new login on g14", plainExample(t, o, "openrazer waits for a new login on g14",
@@ -243,7 +238,6 @@ func TestTheReloginConditionNeedsTheOperatorAndOffersNoButton(t *testing.T) {
// When a module not working turns into one waiting for a new login, and back, the clearing line says what it // When a module not working turns into one waiting for a new login, and back, the clearing line says what it
// became, never that it works again (issue 318 review). // became, never that it works again (issue 318 review).
func TestAConditionThatBecameTheOtherKindSaysSoWhenItClears(t *testing.T) { func TestAConditionThatBecameTheOtherKindSaysSoWhenItClears(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
store := conditions.NewInMemory() store := conditions.NewInMemory()
@@ -292,7 +286,6 @@ func TestAConditionThatBecameTheOtherKindSaysSoWhenItClears(t *testing.T) {
// Which moves add an account group, read from the builds' manifests (issue 318 review). // Which moves add an account group, read from the builds' manifests (issue 318 review).
func TestOnlyAMoveThatAddsAnAccountGroupCanBringAWait(t *testing.T) { func TestOnlyAMoveThatAddsAnAccountGroupCanBringAWait(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
user := func(groups ...any) catalogue.Manifest { user := func(groups ...any) catalogue.Manifest {
return catalogue.Manifest{Module: "lights", Resources: []map[string]any{{"id": "operator", "type": "user", return catalogue.Manifest{Module: "lights", Resources: []map[string]any{{"id": "operator", "type": "user",
"name": "operator", "groups": groups}}} "name": "operator", "groups": groups}}}
+113
View File
@@ -0,0 +1,113 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// **Placement leaves the settings layers and no machine is told anything new** (novox/hq ADR 0348 clause
// 11, issue 651). A mesh is seeded as the store holds it before migration 0097 — `places`, `accesses` and a
// kept network among the settings, at the mesh's level and a machine's, one layer of placement only — then
// migrated, and every machine's declaration is composed. What each machine is told of the seeded modules is
// byte for byte what the controller before this change composed from the same seed: the golden files were
// written by the integration branch's controller at bfc79c6 running this same test (MESH_WRITE_GOLDEN=1),
// where there was no migration 0097 to run.
func TestADeclarationIsTheSameAfterPlacementMoves(t *testing.T) {
open, migrate := aMeshBefore(t, 97)
ctx := t.Context()
inv := open.inventory
image := "registry.invalid:5000/library@sha256:" + strings.Repeat("a", 64)
register(t, open, catalogue.Manifest{Module: "library", Version: "1",
Accesses: []catalogue.Access{{ID: "series"}, {ID: "downloads", Mode: catalogue.AccessRead}},
Resources: []map[string]any{
{"id": "config", "type": "directory", "mode": "0750"},
{"id": "server", "type": "container", "image": image,
"volumes": []any{"${dir:config}:/config", "${access:series}:/series", "${access:downloads}:/downloads:ro"}},
// Today's form: a mergeable file naming no keys takes every key of the layer (ADR 0326 §3).
{"id": "runtime-config", "type": "file", "path": "${dir:config}/runtime.json", "mode": "0640",
"content": `{"log":"info"}`, "merge": "json"},
}})
register(t, open, catalogue.Manifest{Module: "found", Version: "1",
Resources: []map[string]any{{"id": "server", "type": "container", "image": image}}})
if err := inv.SetAdopted(ctx, "anchor", true); err != nil {
t.Fatal(err)
}
for _, l := range []struct{ node, module, values string }{
{"", "library", `{"accesses": {"downloads": "/storage/downloads"}, "log": "warn"}`},
{"anchor", "library", `{"places": {"config": {"path": "/var/lib/library/config", "owner": "1001:2000"}},
"accesses": {"series": "/storage/media/series"}, "workers": 4}`},
{"laptop", "library", `{"accesses": {"series": "/home/media/series"}}`},
{"anchor", "found", `{"networks": {"server": ["predecessor_default"]}}`},
} {
if l.node == "" {
inv.ExecForTest(t, `insert into settings (node, module, values) values (null, $1, $2)`, l.module, l.values)
continue
}
inv.ExecForTest(t, `insert into settings (node, module, values) select id, $2, $3 from node where name = $1`,
l.node, l.module, l.values)
}
migrate()
// Assigned once the store is migrated: an assignment reads the layers, as this controller holds them.
for _, a := range [][]string{{"anchor", "library", "found"}, {"laptop", "library"}} {
if _, err := assign(ctx, open, a[0], a[1:]...); err != nil {
t.Fatalf("assign %v: %v", a, err)
}
}
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
for _, node := range []string{"anchor", "laptop"} {
plan, settings, err := planFor(ctx, open, node)
if err != nil {
t.Fatal(err)
}
with, _, err := renderingFor(ctx, open, node, plan, settings, gens, Allocating)
if err != nil {
t.Fatal(err)
}
composed, err := plan.Compose(with)
if err != nil {
t.Fatal(err)
}
if len(composed.LeftOut) > 0 {
t.Fatalf("%s leaves out %v", node, composed.LeftOut)
}
var told []map[string]any
for _, r := range composed.Resources {
if owner := composed.Owner[r["id"].(string)]; owner == "library" || owner == "found" {
told = append(told, r)
}
}
sort.Slice(told, func(i, j int) bool { return told[i]["id"].(string) < told[j]["id"].(string) })
got, err := json.MarshalIndent(told, "", " ")
if err != nil {
t.Fatal(err)
}
golden := filepath.Join("testdata", "placement-moved", node+".json")
if os.Getenv("MESH_WRITE_GOLDEN") != "" {
if err := os.MkdirAll(filepath.Dir(golden), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(golden, append(got, '\n'), 0o644); err != nil {
t.Fatal(err)
}
continue
}
want, err := os.ReadFile(golden)
if err != nil {
t.Fatal(err)
}
if string(want) != string(got)+"\n" {
t.Errorf("%s is told something else after placement moved:\n%s\nwas:\n%s", node, got, want)
}
}
}
-14
View File
@@ -117,8 +117,6 @@ var plainWordings = map[string]func(conditions.Observation) words{
"reaches it. It keeps running what it has.", m), "reaches it. It keeps running what it has.", m),
Resolved: m + " can get new instructions again"} Resolved: m + " can get new instructions again"}
}), }),
// A grant of a restricted stream that composes nothing (novox/hq ADR 0376 §2).
kindGrantComposesNothing: worded(withheldWords),
kindAgentCanBecomeRoot: worded(func(o conditions.Observation) words { kindAgentCanBecomeRoot: worded(func(o conditions.Observation) words {
m := machineOr(o, "a machine") m := machineOr(o, "a machine")
return words{Headline: "Sessions on " + m + " could become root", return words{Headline: "Sessions on " + m + " could become root",
@@ -622,18 +620,6 @@ var plainWordings = map[string]func(conditions.Observation) words{
Explanation: "The bus reports a listener too slow to keep up, so messages to it are late.", Explanation: "The bus reports a listener too slow to keep up, so messages to it are late.",
Resolved: "The listener keeps up again"} Resolved: "The listener keeps up again"}
}), }),
kindBusFaultsUnread: worded(func(o conditions.Observation) words {
return words{Headline: "Slow clients of the bus cannot be checked",
Explanation: "The controller could not read what the bus said about its clients' connections, so a " +
"client the bus cut off would go unnoticed. It tries again at every check.",
Resolved: "Resolved: the bus's clients are checked again"}
}),
"connection-closed": worded(func(o conditions.Observation) words {
return words{Headline: "The bus cut a client off",
Explanation: "The bus closed a client's connection because it broke the bus's rules, so what it was " +
"sending was lost.",
Resolved: "Resolved: the client keeps to the bus's rules"}
}),
kindBusReconnects: worded(func(o conditions.Observation) words { kindBusReconnects: worded(func(o conditions.Observation) words {
return words{Headline: "A client keeps losing the bus", return words{Headline: "A client keeps losing the bus",
Explanation: "One of the mesh's clients lost its connection to the bus again and again in the last hour. " + Explanation: "One of the mesh's clients lost its connection to the bus again and again in the last hour. " +
-7
View File
@@ -372,13 +372,6 @@ func joinAPlan(ctx context.Context, open *stores, module string) (bool, string,
if !found { if !found {
return false, "", nil return false, "", nil
} }
// **A walk that waits for its delivery's word asks nothing** (novox/hq ADR 0239, issue 754): a rebuild does not
// join it, and its outcome is not the walk's. The walk asks its own tier once it has its word.
if p.Waiting() {
return false, fmt.Sprintf("not joining %s (%s at %s): it waits for %s's word, so this build is not its "+
"outcome; once let go, the walk asks %s itself", p.ID, p.Repository, short(p.Commit), p.Delivery.Awaits,
module), nil
}
entries, err := inv.Catalogued(ctx) entries, err := inv.Catalogued(ctx)
if err != nil { if err != nil {
return false, "", err return false, "", err
+1 -1
View File
@@ -133,7 +133,7 @@ func TestASharedRepositoryIsPlannedFromTheRecordedBuildSources(t *testing.T) {
{"the route proxy's program: the route proxy alone", "mesh-controller", {"the route proxy's program: the route proxy alone", "mesh-controller",
[]string{"examples/route-proxy/main.go"}, "route-proxy", true}, []string{"examples/route-proxy/main.go"}, "route-proxy", true},
{"a package all three build from: all three", "mesh-controller", {"a package all three build from: all three", "mesh-controller",
[]string{"internal/broker/broker.go"}, "mesh-controller,route-proxy | build-agent", false}, []string{"internal/broker/broker.go"}, "mesh-controller | build-agent | route-proxy", false},
{"the route proxy's directory in the catalogue: it alone", "mesh-catalog", {"the route proxy's directory in the catalogue: it alone", "mesh-catalog",
[]string{"modules/route-proxy/module.json"}, "route-proxy", false}, []string{"modules/route-proxy/module.json"}, "route-proxy", false},
{"the build agent's directory: it alone, nothing it builds", "mesh-catalog", {"the build agent's directory: it alone, nothing it builds", "mesh-catalog",
+9 -11
View File
@@ -145,9 +145,8 @@ func TestAPlanIsWhatTheChangeTouchedAndWhatIsBuiltOnIt(t *testing.T) {
repo: "one", paths: []string{"modules/a/x", "modules/x/y"}, want: "a,x"}, repo: "one", paths: []string{"modules/a/x", "modules/x/y"}, want: "a,x"},
{what: "built-by never widens", edges: []inventory.Edge{dep("x", builtBy, "a")}, {what: "built-by never widens", edges: []inventory.Edge{dep("x", builtBy, "a")},
repo: "one", paths: []string{"modules/a/x"}, want: "a"}, repo: "one", paths: []string{"modules/a/x"}, want: "a"},
// Whichever builder runs builds it (novox/hq ADR 0360 §3): built-by orders nothing, as before it ordered. {what: "built-by orders", edges: []inventory.Edge{dep("x", builtBy, "a")},
{what: "built-by does not order", edges: []inventory.Edge{dep("x", builtBy, "a")}, repo: "one", paths: []string{"modules/a/x", "modules/x/y"}, want: "a | x"},
repo: "one", paths: []string{"modules/a/x", "modules/x/y"}, want: "a,x"},
{what: "worker-of never widens", edges: []inventory.Edge{dep("x", workerOf, "a")}, {what: "worker-of never widens", edges: []inventory.Edge{dep("x", workerOf, "a")},
repo: "one", paths: []string{"modules/a/x"}, want: "a"}, repo: "one", paths: []string{"modules/a/x"}, want: "a"},
{what: "worker-of orders", edges: []inventory.Edge{dep("x", workerOf, "a")}, {what: "worker-of orders", edges: []inventory.Edge{dep("x", workerOf, "a")},
@@ -297,18 +296,18 @@ func TestASharedRepositoryMovesOnlyWhatItsBuildSourceHolds(t *testing.T) {
[]string{"internal/inventory/migrations/0088-a-build-says-its-build-source.sql"}, read, []string{"internal/inventory/migrations/0088-a-build-says-its-build-source.sql"}, read,
"mesh-controller | build-agent", ""}, "mesh-controller | build-agent", ""},
// A package all three build from: all three; the build seat's holder after the controller whose worker // A package all three build from: all three; the build seat's holder after the controller whose worker
// it binds (worker-of, issue 206), the proxy beside the controller: built-by orders nothing (ADR 0360 §3). // it binds (worker-of, issue 206), the proxy after the holder that builds it (built-by).
{"a package all three build from: all three", "mesh-controller", {"a package all three build from: all three", "mesh-controller",
[]string{"internal/broker/broker.go"}, read, "mesh-controller,route-proxy | build-agent", ""}, []string{"internal/broker/broker.go"}, read, "mesh-controller | build-agent | route-proxy", ""},
{"a vendored package all three build from: all three", "mesh-controller", {"a vendored package all three build from: all three", "mesh-controller",
[]string{"vendor/github.com/nats-io/nats.go/nats.go"}, read, "mesh-controller,route-proxy | build-agent", ""}, []string{"vendor/github.com/nats-io/nats.go/nats.go"}, read, "mesh-controller | build-agent | route-proxy", ""},
{"go.sum: all three", "mesh-controller", {"go.sum: all three", "mesh-controller",
[]string{"go.sum"}, read, "mesh-controller,route-proxy | build-agent", ""}, []string{"go.sum"}, read, "mesh-controller | build-agent | route-proxy", ""},
{"a file added to the proxy's package: the proxy", "mesh-controller", {"a file added to the proxy's package: the proxy", "mesh-controller",
[]string{"examples/route-proxy/new.go"}, read, "route-proxy", ""}, []string{"examples/route-proxy/new.go"}, read, "route-proxy", ""},
// Before any build source is said: as before. // Before any build source is said: as before.
{"no build source said: a README moves all three, as before", "mesh-controller", {"no build source said: a README moves all three, as before", "mesh-controller",
[]string{"README.md"}, unsaid, "mesh-controller,route-proxy | build-agent", ""}, []string{"README.md"}, unsaid, "mesh-controller | build-agent | route-proxy", ""},
// In the catalogue, where they live, each by its own build source. // In the catalogue, where they live, each by its own build source.
{"the route proxy's recipe: it alone", "mesh-catalog", {"the route proxy's recipe: it alone", "mesh-catalog",
[]string{"modules/route-proxy/Dockerfile"}, read, "route-proxy", ""}, []string{"modules/route-proxy/Dockerfile"}, read, "route-proxy", ""},
@@ -332,7 +331,7 @@ func TestASharedRepositoryMovesOnlyWhatItsBuildSourceHolds(t *testing.T) {
// Files not all said: everything the repository builds, as before. // Files not all said: everything the repository builds, as before.
m := link.SourceMoved{Owner: "novox", Repo: "mesh-controller", Base: "main", Commit: "head", m := link.SourceMoved{Owner: "novox", Repo: "mesh-controller", Base: "main", Commit: "head",
Paths: []string{"README.md"}, PathsTruncated: true} Paths: []string{"README.md"}, PathsTruncated: true}
if got := tiered(reachOfMerge(m, entries, read, edges).Plan.Tiers); got != "mesh-controller,route-proxy | build-agent" { if got := tiered(reachOfMerge(m, entries, read, edges).Plan.Tiers); got != "mesh-controller | build-agent | route-proxy" {
t.Errorf("a merge whose files were not all said: planned %q, wanted all three", got) t.Errorf("a merge whose files were not all said: planned %q, wanted all three", got)
} }
} }
@@ -453,9 +452,8 @@ func TestAPlanIsTheTouchedModulesAndWhatIsReachableAlongTheWideningEdges(t *test
kinds := []string{inventory.EdgeStandsOn, inventory.EdgeDeclared, inventory.EdgePackages, kinds := []string{inventory.EdgeStandsOn, inventory.EdgeDeclared, inventory.EdgePackages,
inventory.EdgeBuiltBy, inventory.EdgeWorkerOf} inventory.EdgeBuiltBy, inventory.EdgeWorkerOf}
widens := map[string]bool{inventory.EdgeStandsOn: true, inventory.EdgeDeclared: true} widens := map[string]bool{inventory.EdgeStandsOn: true, inventory.EdgeDeclared: true}
// Built-by orders nothing since novox/hq ADR 0360 §3.
orders := map[string]bool{inventory.EdgeStandsOn: true, inventory.EdgeDeclared: true, orders := map[string]bool{inventory.EdgeStandsOn: true, inventory.EdgeDeclared: true,
inventory.EdgeWorkerOf: true} inventory.EdgeBuiltBy: true, inventory.EdgeWorkerOf: true}
// Directory names drawn from one pool, so two repositories hold directories of the same name, and one // Directory names drawn from one pool, so two repositories hold directories of the same name, and one
// is a prefix of another. // is a prefix of another.
dirs := []string{"a", "ab", "b", "c", "lib/x", "lib/y", "modules/a", "modules/a/sub"} dirs := []string{"a", "ab", "b", "c", "lib/x", "lib/y", "modules/a", "modules/a/sub"}
+93
View File
@@ -0,0 +1,93 @@
package main
import (
"context"
"fmt"
"path"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The list of predefined keys the mesh publishes (novox/hq ADR 0348 §7, design 57 package K2, issue 661).
//
// **The newest trunk build of the catalogue wins.** A build of the catalogue reads predefined.json at the commit
// it builds and carries it with the commit's generation (link.PredefinedList). Registration keeps it in the
// controller's store when that generation is larger than the one kept (inventory.KeepPredefined), so an older
// build's list heard later never replaces a newer one, and every manifest the controller reads again is judged
// against the list kept (catalogue.UsePredefined). mesh-config publishes it on the bus from there (design 57
// package K3); the controller keeps no config value, only the keys.
// fromTheCatalogue is whether a build is of the catalogue's repository, whose list alone the mesh publishes: by
// its name, in the URL form or the seat form.
func fromTheCatalogue(result link.BuildResult) bool {
for _, repository := range []string{result.Repository, sourceRepository(result)} {
name := strings.TrimSuffix(path.Base(strings.TrimRight(repository, "/")), ".git")
if name == catalogue.CatalogueRepository {
return true
}
}
return false
}
func sourceRepository(result link.BuildResult) string {
if result.Source == nil {
return ""
}
return result.Source.Repository
}
// carriedList is the list a build of the catalogue carried, read strictly; nil for a build of another
// repository, or one that carried none.
func carriedList(result link.BuildResult) (catalogue.PredefinedList, error) {
if result.Predefined == nil || !fromTheCatalogue(result) {
return nil, nil
}
list, err := catalogue.ParsePredefined(result.Predefined.Keys)
if err != nil {
return nil, fmt.Errorf("%s carried a list of predefined keys that is refused: %w", result.ID, err)
}
return list, nil
}
// judgedWith is the list a build's manifest is first judged against at registration: the catalogue's own at the
// commit built, which its build judged it against, where it carried one; the list kept otherwise.
func judgedWith(result link.BuildResult) (catalogue.PredefinedList, error) {
list, err := carriedList(result)
if err != nil || list != nil {
return list, err
}
return catalogue.PublishedPredefined(), nil
}
// loadPredefined makes the list kept in the store the one this controller judges manifests against. A store
// keeping none leaves the list it has.
func loadPredefined(ctx context.Context, inv *inventory.Inventory) error {
kept, has, err := inv.PredefinedKept(ctx)
if err != nil || !has {
return err
}
catalogue.UsePredefined(kept.Keys)
return nil
}
// publishPredefined keeps a trunk build's list when it is newer than the one kept, says so, and loads what the
// store keeps then — this build's, or a newer one another controller process kept.
func publishPredefined(ctx context.Context, inv *inventory.Inventory, result link.BuildResult,
list catalogue.PredefinedList) error {
kept, err := inv.KeepPredefined(ctx, inventory.PredefinedRecord{Keys: list, Repository: result.Repository,
Commit: result.Commit, Generation: result.Predefined.Generation, Build: result.ID})
if err != nil {
return err
}
if kept {
fmt.Printf("%s: the list of predefined keys at %s (generation %d) is published: %s\n", result.ID,
short(result.Commit), result.Predefined.Generation, strings.Join(list.Keys(), ", "))
} else {
fmt.Printf("%s: the list of predefined keys at %s (generation %d) is not newer than the one kept, which "+
"stands\n", result.ID, short(result.Commit), result.Predefined.Generation)
}
return loadPredefined(ctx, inv)
}
+197
View File
@@ -0,0 +1,197 @@
package main
import (
"bytes"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
// The list of predefined keys (novox/hq ADR 0348 §7, design 57 package K2, issue 661): `module check` and the
// merge gate judge each manifest against the list at the root of its tree; registration publishes the newest
// trunk build's list of the catalogue and judges every module against the list the mesh publishes.
const theFirstKeys = `{
"font-size": {"type": "number", "description": "the size text is shown in, in points", "minimum": 6, "maximum": 72},
"agent-managed-settings": {"type": "object", "description": "the managed settings of every agent session, a JSON object", "trusted": true}
}`
// notifierFallingBackTo is the notifier in the config form, its font size falling back to the key named.
func notifierFallingBackTo(key, kind string) string {
def := "10"
if kind == "string" {
def = `"10"`
}
return `{"module":"notifier","version":"1",
"config":{"font-size":{"type":"` + kind + `","description":"the notification's font size, in points",
"default":` + def + `,"predefined":"` + key + `"}},
"resources":[{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644",
"content":"font = ${config:font-size}\n"}]}`
}
// withoutAPublishedList judges the test against no list the mesh published, and puts the process's back after.
func withoutAPublishedList(t *testing.T) {
t.Helper()
was := catalogue.ReplacePublished(nil)
t.Cleanup(func() { catalogue.ReplacePublished(was) })
}
// aRepositoryTree is a repository's checkout holding one module at modules/notifier, and the list when given.
func aRepositoryTree(t *testing.T, manifest string, list *string) string {
t.Helper()
root := t.TempDir()
if err := os.Mkdir(filepath.Join(root, ".git"), 0o755); err != nil {
t.Fatal(err)
}
if list != nil {
if err := os.WriteFile(filepath.Join(root, catalogue.PredefinedFile), []byte(*list), 0o644); err != nil {
t.Fatal(err)
}
}
return aTemplatedModule(t, filepath.Join(root, "modules", "notifier"), manifest, nil)
}
func TestModuleCheckJudgesAManifestAgainstTheListAtItsRepositoryRoot(t *testing.T) {
withoutAPublishedList(t)
list := theFirstKeys
check := func(path string) (string, error) {
var out bytes.Buffer
err := moduleCheck([]string{path}, &out)
return out.String(), err
}
if out, err := check(aRepositoryTree(t, notifierFallingBackTo("font-size", "number"), &list)); err != nil ||
!strings.Contains(out, "notifier: ok") {
t.Fatalf("a module naming font-size was refused: %v\n%s", err, out)
}
for _, c := range []struct {
manifest string
list *string
says string
}{
{notifierFallingBackTo("font-scale", "number"), &list, `"font-scale", which is not on the list`},
{notifierFallingBackTo("font-size", "string"), &list, "which is a number, and this value is a string"},
{notifierFallingBackTo("font-size", "number"), nil, "is not present"},
{notifierFallingBackTo("font-size", "number"), ptr(`{"font-size": {"type": "number"}}`), "no description"},
} {
if out, err := check(aRepositoryTree(t, c.manifest, c.list)); err == nil || !strings.Contains(out, c.says) {
t.Errorf("wanted the check refused saying %q, got %v:\n%s", c.says, err, out)
}
}
}
func ptr(s string) *string { return &s }
// writeTheList puts the list at the root of a tree the gate judges.
func writeTheList(t *testing.T, tree, list string) string {
t.Helper()
if err := os.WriteFile(filepath.Join(tree, catalogue.PredefinedFile), []byte(list), 0o644); err != nil {
t.Fatal(err)
}
return tree
}
func TestTheMergeGateJudgesEachManifestAgainstTheListOfTheTree(t *testing.T) {
withoutAPublishedList(t)
f, manifests := templatedMesh(t)
v := gateJudged(t, f, writeTheList(t, aTree(t, withEdit(manifests, "notifier",
notifierFallingBackTo("font-size", "number"))), theFirstKeys), "modules/notifier/module.json")
if v.Verdict == "fail" || v.Modules["notifier"] != "changed" {
t.Fatalf("a module naming font-size on the tree's list failed the gate (%v):\n%s", v.Modules, v.Report())
}
for _, c := range []struct{ manifest, list, says string }{
{notifierFallingBackTo("font-scale", "number"), theFirstKeys, `"font-scale", which is not on the list`},
{notifierFallingBackTo("font-size", "string"), theFirstKeys, "which is a number"},
{inlineNotifier, `{"font-size": {"type": "number", "description": "points", "unit": "pt"}}`, `unknown field "unit"`},
} {
v := gateJudged(t, f, writeTheList(t, aTree(t, withEdit(manifests, "notifier", c.manifest)), c.list),
"modules/notifier/module.json", catalogue.PredefinedFile)
if v.Verdict != "fail" || !strings.Contains(v.Report(), c.says) {
t.Errorf("wanted the gate to fail saying %q:\n%s", c.says, v.Report())
}
}
}
// catalogueBuild is a trunk build of the catalogue carrying the list at a generation.
func catalogueBuild(id, manifest, list string, generation int) link.BuildResult {
r := link.BuildResult{ID: id, Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/notifier", Ref: "main", On: "anchor", Commit: strings.Repeat(id[len(id)-1:], 12),
Manifest: json.RawMessage(manifest), Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
if list != "" {
r.Predefined = &link.PredefinedList{Keys: json.RawMessage(list), Generation: generation}
}
return r
}
func TestRegistrationPublishesTheNewestTrunkListAndJudgesEveryModuleAgainstIt(t *testing.T) {
withoutAPublishedList(t)
open := aMesh(t)
ctx := t.Context()
// A manifest naming font-size registers, and its build's list is published.
if _, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory,
catalogueBuild("b-1", notifierFallingBackTo("font-size", "number"), theFirstKeys, 10))); err != nil {
t.Fatalf("a module naming font-size was not registered: %v", err)
}
kept, has, err := open.inventory.PredefinedKept(ctx)
if err != nil || !has || kept.Generation != 10 || kept.Build != "b-1" {
t.Fatalf("the list was not published: %+v %v %v", kept, has, err)
}
if got := catalogue.PublishedPredefined(); len(got) != 2 {
t.Fatalf("the controller judges against %v, not the list published", got)
}
// One naming a key not on the list, or of another type, is refused and never enters the catalogue.
for i, c := range []struct{ manifest, says string }{
{notifierFallingBackTo("font-scale", "number"), "not on the list"},
{notifierFallingBackTo("font-size", "string"), "which is a number"},
} {
id := []string{"b-2", "b-3"}[i]
if _, _, err := takeIn(ctx, open.inventory, catalogueBuild(id, c.manifest, theFirstKeys, 10)); err == nil ||
!strings.Contains(err.Error(), c.says) {
t.Errorf("wanted the registration refused saying %q, got %v", c.says, err)
}
}
src, err := open.inventory.SourceOf(ctx, "notifier")
if err != nil || src.BuiltFrom != strings.Repeat("1", 12) {
t.Fatalf("a refused build replaced the registered one: %+v %v", src, err)
}
// A newer build's list, which drops agent-managed-settings, replaces the published one.
smaller := `{"font-size": {"type": "number", "description": "points", "minimum": 6, "maximum": 72}}`
if _, _, err := takeIn(ctx, open.inventory, catalogueBuild("b-4",
notifierFallingBackTo("font-size", "number"), smaller, 12)); err != nil {
t.Fatal(err)
}
// The older build's outcome heard later registers nothing new about the list: the newer stands.
if _, _, err := takeIn(ctx, open.inventory, catalogueBuild("b-5",
notifierFallingBackTo("font-size", "number"), theFirstKeys, 11)); err != nil {
t.Fatal(err)
}
kept, _, _ = open.inventory.PredefinedKept(ctx)
if kept.Build != "b-4" || len(kept.Keys) != 1 || len(catalogue.PublishedPredefined()) != 1 {
t.Fatalf("an older build's list heard later replaced the newer one: %+v", kept)
}
// A build of another repository publishes nothing, and its manifest is judged against the published list.
other := catalogueBuild("b-6", notifierFallingBackTo("font-size", "number"), theFirstKeys, 99)
other.Repository, other.Source.Repository = "http://forge.internal:20000/novox/notifier.git", "novox/notifier"
other.Manifest = json.RawMessage(strings.Replace(notifierFallingBackTo("font-size", "number"), `"notifier"`,
`"chimes"`, 1))
if _, _, err := takeIn(ctx, open.inventory, asTheOperator(t, open.inventory, other)); err != nil {
t.Fatalf("a module of another repository naming a published key: %v", err)
}
if kept, _, _ = open.inventory.PredefinedKept(ctx); kept.Build != "b-4" {
t.Fatalf("a build of another repository published its list: %+v", kept)
}
other.ID, other.Manifest = "b-7", json.RawMessage(strings.Replace(notifierFallingBackTo("agent-managed-settings",
"number"), `"notifier"`, `"chimes"`, 1))
if _, _, err := takeIn(ctx, open.inventory, other); err == nil || !strings.Contains(err.Error(), "not on the list") {
t.Fatalf("a key only another repository's list holds was registered: %v", err)
}
}
-6
View File
@@ -929,12 +929,6 @@ func streamDiffers(want broker.Stream, have nats.StreamConfig) string {
differs = append(differs, fmt.Sprintf("keeps one of a message id for %s, defined %s", have.Duplicates, differs = append(differs, fmt.Sprintf("keeps one of a message id for %s, defined %s", have.Duplicates,
time.Duration(want.DuplicatesSeconds)*time.Second)) time.Duration(want.DuplicatesSeconds)*time.Second))
} }
if want.AllowMsgTTL && !have.AllowMsgTTL {
differs = append(differs, "takes no time to live of a message's own, defined to (novox/hq ADR 0376)")
}
if want.Restricted && have.Metadata[broker.RestrictedMetadata] != "true" {
differs = append(differs, "does not say it is restricted, so a message kept from it would not age out with it")
}
if want.DiscardNew && have.Discard != nats.DiscardNew { if want.DiscardNew && have.Discard != nats.DiscardNew {
differs = append(differs, "drops what it holds when full, defined to refuse what comes next") differs = append(differs, "drops what it holds when full, defined to refuse what comes next")
} }
+3 -88
View File
@@ -17,7 +17,6 @@ import (
"strings" "strings"
"time" "time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker" "github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue" "github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
@@ -195,7 +194,7 @@ func serve(ctx context.Context) (err error) {
// And what the catalogue decided a build meant. The builder's own result is already handled // And what the catalogue decided a build meant. The builder's own result is already handled
// above; this is the other half — the control plane is the only one of the three that knows // above; this is the other half — the control plane is the only one of the three that knows
// which machines run the thing, so it is the one that acts (novox/hq ADR 0072). // which machines run the thing, so it is the one that acts (novox/hq ADR 0072).
if err := server.Follows(following{open: open}); err != nil { if err := server.Follows(following{open}); err != nil {
return err return err
} }
// And the merges the bus announced and never handed over, read back on a timer and acted on late // And the merges the bus announced and never handed over, read back on a timer and acted on late
@@ -203,7 +202,7 @@ func serve(ctx context.Context) (err error) {
go catchingUpOnMerges(ctx, open, server) go catchingUpOnMerges(ctx, open, server)
// And a catalogue that has just started, asking for what it missed. The same type answers // And a catalogue that has just started, asking for what it missed. The same type answers
// both: what a build meant and what the builds were are two questions about one record. // both: what a build meant and what the builds were are two questions about one record.
if err := server.Answers(following{open: open}); err != nil { if err := server.Answers(following{open}); err != nil {
return err return err
} }
// And what providers say about consumers they keep failing, kept for `status` (novox/hq ADR // And what providers say about consumers they keep failing, kept for `status` (novox/hq ADR
@@ -257,7 +256,7 @@ func serve(ctx context.Context) (err error) {
go forgetChecksOfWalk(context.WithoutCancel(ctx), open.inventory, p) go forgetChecksOfWalk(context.WithoutCancel(ctx), open.inventory, p)
} }
} }
if err := server.Checks(following{open: open}); err != nil { if err := server.Checks(following{open}); err != nil {
return err return err
} }
// Composed now and kept current, before the verb that answers from it is served. // Composed now and kept current, before the verb that answers from it is served.
@@ -297,8 +296,6 @@ func serve(ctx context.Context) (err error) {
return err return err
} }
defer stopCLI() defer stopCLI()
// Its read calls write no line each; how many each caller made is said once an hour (novox/hq issue 762).
go sayReadCalls(ctx)
// And says so on the bus (novox/hq ADR 0197): what it serves, as the NATS services protocol asks. // And says so on the bus (novox/hq ADR 0197): what it serves, as the NATS services protocol asks.
stopAnnouncing, err := bus.Announce(seatAnnouncement(handlers), log.New(os.Stdout, "", log.LstdFlags)) stopAnnouncing, err := bus.Announce(seatAnnouncement(handlers), log.New(os.Stdout, "", log.LstdFlags))
if err != nil { if err != nil {
@@ -1389,13 +1386,6 @@ func issueMemberships(ctx context.Context, open *stores, server *link.Server, se
if issued > 0 { if issued > 0 {
fmt.Printf(" issued %d membership(s)\n", issued) fmt.Printf(" issued %d membership(s)\n", issued)
} }
// **And what every other machine hears of a module's stream** (novox/hq ADR 0363 §6): a grant the provider's
// setting changed reaches a consumer on a machine this send does not reach. Its membership is issued again
// as last issued, with only what it hears changed, so nothing else of it moves before its own send.
reheard := reissueHearing(ctx, bus, records, where, sent)
if reheard > 0 {
fmt.Printf(" issued %d membership(s) again for what they hear of a module's stream\n", reheard)
}
if len(refused) > 0 { if len(refused) > 0 {
// Returned, never passed over (novox/hq issue 249): the declarations of the machines they are // Returned, never passed over (novox/hq issue 249): the declarations of the machines they are
// for are not sent, and the rollout that asked is tried again rather than waiting for a push. // for are not sent, and the rollout that asked is tried again rather than waiting for a push.
@@ -1754,78 +1744,3 @@ func schemaReach() (int, error) {
} }
return reach, nil return reach, nil
} }
// reissueHearing issues again, on every machine not just sent, the membership of each assignment whose
// `consumes` names a module owning a stream, when what it hears changed: read as last issued from the bus,
// its `hears` replaced, and published. One that was never issued is left to its own send. Failures are said
// and passed over: the send that carries the grant is not the consumer's, and its own next send issues it.
func reissueHearing(ctx context.Context, bus link.OverNATS, records broker.Records, where broker.Placements,
sent []readyNode) int {
reached := map[string]bool{}
for _, s := range sent {
reached[s.node] = true
}
issued := 0
for node, declared := range records.Assigned {
if reached[node] {
continue
}
for _, d := range declared {
if !d.HearsAStream {
continue
}
subject := broker.MembershipSubject(node, d.Module)
last, err := bus.JS.GetLastMsg(broker.AssignmentsStream, subject)
if err != nil {
if !errors.Is(err, nats.ErrMsgNotFound) {
fmt.Printf(" %s on %s: its membership could not be read to issue what it hears again: %v\n",
d.Module, node, err)
}
continue
}
body, changed, err := withHearing(last.Data, broker.HeardIssuedFor(node, d))
if err != nil {
fmt.Printf(" %s on %s: its membership could not be read to issue what it hears again: %v\n",
d.Module, node, err)
continue
}
if !changed {
continue
}
if err := bus.PublishMembership(ctx, node, d.Module, body); err != nil {
fmt.Printf(" %s on %s: what it hears could not be issued again: %v\n", d.Module, node, err)
continue
}
issued++
}
}
return issued
}
// withHearing is a membership as issued with its `hears` replaced, and whether that changed it. Pure.
func withHearing(issued []byte, hears []broker.HeardIssued) ([]byte, bool, error) {
var fields map[string]json.RawMessage
if err := json.Unmarshal(issued, &fields); err != nil {
return nil, false, err
}
var had []broker.HeardIssued
if raw, ok := fields["hears"]; ok {
if err := json.Unmarshal(raw, &had); err != nil {
return nil, false, err
}
}
if slices.Equal(had, hears) {
return issued, false, nil
}
if len(hears) == 0 {
delete(fields, "hears")
} else {
raw, err := json.Marshal(hears)
if err != nil {
return nil, false, err
}
fields["hears"] = raw
}
body, err := json.Marshal(fields)
return body, true, err
}
+1 -11
View File
@@ -340,10 +340,7 @@ func cancelAsk(ctx context.Context, js *broker.JetStream, open *stores, seat str
return "", fmt.Errorf("%s is marked cancelled and could not be deleted from the queue (seq %d): %w — a "+ return "", fmt.Errorf("%s is marked cancelled and could not be deleted from the queue (seq %d): %w — a "+
"holder taking it ends it as cancelled", ask.ID, ask.Seq, err) "holder taking it ends it as cancelled", ask.ID, ask.Seq, err)
} }
// A check ended because its pull request has a newer head records nothing (novox/hq issue 726): open is nil.
if open != nil {
recordCancelled(ctx, open, ask) recordCancelled(ctx, open, ask)
}
return cancelSaid(ask, seat, removed), nil return cancelSaid(ask, seat, removed), nil
} }
@@ -521,9 +518,7 @@ func rebuildCommand(ctx context.Context, args []string) error {
} }
// **A module a plan holds unbuilt, or failed, joins that plan** rather than running beside it: the // **A module a plan holds unbuilt, or failed, joins that plan** rather than running beside it: the
// plan would ask it again, or stay failed on an outcome a rebuild has replaced. A walk waiting for its word // plan would ask it again, or stay failed on an outcome a rebuild has replaced.
// is not joined (novox/hq issue 754): the rebuild runs beside it, and says so.
var beside string
if module != "" { if module != "" {
joined, said, err := joinAPlan(ctx, open, module) joined, said, err := joinAPlan(ctx, open, module)
if err != nil { if err != nil {
@@ -533,7 +528,6 @@ func rebuildCommand(ctx context.Context, args []string) error {
fmt.Println(said) fmt.Println(said)
return nil return nil
} }
beside = said
} }
id, err := askABuild(ctx, source, path, ref) id, err := askABuild(ctx, source, path, ref)
if err != nil { if err != nil {
@@ -541,10 +535,6 @@ func rebuildCommand(ctx context.Context, args []string) error {
} }
recordAsked(ctx, inventory.BuildRequest{ID: id, Repository: source.Repository, Seat: source.Seat, Path: path, recordAsked(ctx, inventory.BuildRequest{ID: id, Repository: source.Repository, Seat: source.Seat, Path: path,
Ref: ref, For: "rebuild"}) Ref: ref, For: "rebuild"})
if beside != "" {
fmt.Printf("rebuild asked as %s, beside the walk: %s\n", id, beside)
return nil
}
fmt.Printf("rebuild asked as %s\n", id) fmt.Printf("rebuild asked as %s\n", id)
return nil return nil
} }
+2 -4
View File
@@ -616,7 +616,6 @@ func TestKillAndPauseAskTheHolderOnTheMachine(t *testing.T) {
// A plan that stopped at its first machine is retried: the module sent to that machine again, the // A plan that stopped at its first machine is retried: the module sent to that machine again, the
// send recorded as the first anew, and the plan goes on — unless a newer plan holds the module. // send recorded as the first anew, and the plan goes on — unless a newer plan holds the module.
func TestAPlanStoppedAtItsFirstMachineIsRetried(t *testing.T) { func TestAPlanStoppedAtItsFirstMachineIsRetried(t *testing.T) {
withTheGate(t) // the first-machine gate's own path (novox/hq ADR 0360 keeps it for earlier walks)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
asked := asksRecorded(t) asked := asksRecorded(t)
@@ -637,9 +636,8 @@ func TestAPlanStoppedAtItsFirstMachineIsRetried(t *testing.T) {
stopped := inventory.Plan{ID: "plan-rollout", Repository: "novox/a", Branch: "main", Commit: "c0ffee", stopped := inventory.Plan{ID: "plan-rollout", Repository: "novox/a", Branch: "main", Commit: "c0ffee",
Created: long, State: inventory.PlanFailed, Tiers: [][]string{{"a"}, {"b"}}, Created: long, State: inventory.PlanFailed, Tiers: [][]string{{"a"}, {"b"}},
Note: "a stopped at its first machine in tier 0: laptop refused what it was sent", Note: "a stopped at its first machine in tier 0: laptop refused what it was sent",
// Asked by the plan, as every module a plan holds built is (novox/hq issue 754). Modules: map[string]*inventory.PlanModule{"a": {State: "built", BuiltAt: &long, Commit: "c0ffee",
Modules: map[string]*inventory.PlanModule{"a": {State: "built", AskedAt: &long, Build: "build-a", BuiltAt: &long, First: []string{"laptop"}, FirstAt: &long, Why: "laptop refused what it was sent"}}}
Commit: "c0ffee", First: []string{"laptop"}, FirstAt: &long, Why: "laptop refused what it was sent"}}}
if err := open.inventory.SavePlan(ctx, &stopped); err != nil { if err := open.inventory.SavePlan(ctx, &stopped); err != nil {
t.Fatal(err) t.Fatal(err)
} }
-6
View File
@@ -158,12 +158,6 @@ func recordedKept(ctx context.Context, open *stores, node string) (map[string]st
if f.identical(m, was, now.Commit) { if f.identical(m, was, now.Commit) {
continue continue
} }
// **Under its own `record`, a move that recreates nothing and moves no data is not kept** (novox/hq ADR 0377
// §1): it goes as a module that rolls out goes. A person's `record`, the bus, and a move that cannot be
// judged are kept as before (§4, §5).
if !every && f.rolls(m, was) {
continue
}
out[m] = was out[m] = was
} }
// **And the controller and the node-engine at the build the machine was last sent, over an older record** // **And the controller and the node-engine at the build the machine was last sent, over an older record**
-213
View File
@@ -1,213 +0,0 @@
package main
import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// Under a module's own `record`, a move that recreates no container and moves no data rolls out (novox/hq ADR
// 0377, changing ADR 0242 §1 and ADR 0360 §1).
//
// A module records because its manifest says so, because it keeps irreplaceable data, because a person set it, or
// because it provides the bus. The first two hold for a reason the manifest shows — a change that recreates mail's
// containers interrupts everyone's mail, a new database image may upgrade its databases in place — and that hold
// also held the moves that interrupt nothing: on 2026-10-15 the database server moved with only its tools bundle
// changed, its walk said it "recreates none of its containers", and ace stayed behind until a person pushed.
//
// So **a move under a module's own `record` is judged per machine, against the build that machine was last sent**
// (§2): when it recreates none of the module's containers and moves none of its data (catalogue.Holds, §3) it
// goes as a module that rolls out goes — a walk sends it, and every other send treats it as a move waiting for its
// walk — and otherwise it is kept at the build the machine runs, as before. The module's policy stays `record`. A
// move that cannot be judged waits (§4); a person's `record` and the bus hold every move (§5); and each walk says
// per machine which way a recorded module went (§6).
// recordedRollsWhy is what a walk says of a recorded module's move that went out (ADR 0377 §6).
const recordedRollsWhy = "went out: it recreates none of its containers and moves none of its data (novox/hq ADR 0377)"
// recordVerdict is one machine's move of a recorded module: whether it rolls out, and what the walk says of it.
type recordVerdict struct {
rolls bool
// said is what the walk says: recordedRollsWhy, or "waits: " and why — the recreate words, the data change
// named, the person's or the bus's hold, or that it could not be judged.
said string
}
// recordedMove judges a module's move on a machine from the build it was last sent (`was`) to the one it would be
// sent now. A module that rolls out rolls; one whose policy is a person's or the bus's waits, said; one under its
// own `record` rolls when the move recreates nothing and moves no data, and waits when it does or cannot be judged.
func (f moveFacts) recordedMove(module, was string) recordVerdict {
now := f.current[module]
switch {
case now.RollOut:
return recordVerdict{rolls: true}
case now.From == catalogue.FromPerson:
return recordVerdict{said: "waits: a person set it to record, and holds every move of it until they push (novox/hq ADR 0377 §5)"}
case now.From == catalogue.FromBus:
return recordVerdict{said: "waits: it provides the bus, which moves only by the bus step a person starts (novox/hq ADR 0377 §5)"}
case !now.OwnRecord():
return recordVerdict{said: "waits: its policy records"}
case f.judge == nil:
return recordVerdict{said: "waits: it could not be judged — the build records were not read (novox/hq ADR 0377 §4)"}
}
return f.judge(module, was, now.Commit)
}
// rolls is whether a module's move on a machine from `was` goes as a move that rolls out (ADR 0377 §1).
func (f moveFacts) rolls(module, was string) bool {
return f.recordedMove(module, was).rolls
}
// recordJudge is the judging of moves under a module's own `record`, reading the build records once per move and
// the catalogue once (ADR 0377 §2–§4).
func recordJudge(ctx context.Context, inv *inventory.Inventory) func(module, was, now string) recordVerdict {
judged := map[string]recordVerdict{}
var shelf map[string]catalogue.Manifest
var shelfErr error
shelfRead := false
return func(module, was, now string) recordVerdict {
key := module + "\x00" + was + "\x00" + now
if v, seen := judged[key]; seen {
return v
}
v := func() recordVerdict {
cannot := func(why string) recordVerdict {
return recordVerdict{said: "waits: it could not be judged — " + why + " (novox/hq ADR 0377 §4)"}
}
if was == "" {
return cannot("what the machine was last sent of it is not known")
}
from, found, err := inv.ManifestAt(ctx, module, was)
if err != nil {
return cannot(fmt.Sprintf("its build %s, which the machine was last sent, cannot be read: %v", short(was), err))
}
if !found {
return cannot(fmt.Sprintf("its build %s, which the machine was last sent, is not in the build records", short(was)))
}
if !shelfRead {
shelf, shelfErr = inv.Catalogue(ctx)
shelfRead = true
}
if shelfErr != nil {
return cannot(fmt.Sprintf("the build it would be sent cannot be read: %v", shelfErr))
}
to, held := shelf[module]
if !held {
return cannot("the build it would be sent is not in the catalogue")
}
if why := catalogue.Holds(from, to); why != "" {
return recordVerdict{said: "waits: " + why}
}
return recordVerdict{rolls: true, said: recordedRollsWhy}
}()
judged[key] = v
return v
}
}
// judgeRecordedInWalk judges, for a walk carrying a module whose policy records, each machine running it whose
// build would move (ADR 0377 §2), keeps on the plan which way each went (§6), says it on the controller's log, and
// answers the machines its move goes out to. Judged once per walk: a walk read again answers what it kept.
func judgeRecordedInWalk(ctx context.Context, inv *inventory.Inventory, p *inventory.Plan, module string,
state *inventory.PlanModule) ([]string, error) {
if state.Recorded != nil {
return wentOut(state.Recorded), nil
}
running, err := inv.Running(ctx, module)
if err != nil {
return nil, err
}
f, err := readMoveFacts(ctx, inv)
if err != nil {
return nil, err
}
now := f.current[module]
said := map[string]string{}
for _, n := range running {
sent, known, err := inv.SentBuilds(ctx, n)
if err != nil {
return nil, err
}
was, carried := sent[module]
if known && carried && f.identical(module, was, now.Commit) {
continue // no move there
}
v := f.recordedMove(module, was)
if !known {
v = recordVerdict{said: "waits: it could not be judged — what " + n + " was last sent is not known (novox/hq ADR 0377 §4)"}
}
said[n] = v.said
fmt.Printf("%s: %s on %s %s\n", p.ID, module, n, v.said)
}
if len(said) > 0 {
state.Recorded = said
}
return wentOut(said), nil
}
// wentOut is the machines a recorded module's move went out to, sorted.
func wentOut(recorded map[string]string) []string {
var out []string
for _, n := range sortedKeysOf(recorded) {
if recorded[n] == recordedRollsWhy {
out = append(out, n)
}
}
return out
}
// recordedSaid is, for a walk's note at its end, which way each recorded module it carried went on each machine
// (ADR 0377 §6).
func recordedSaid(p inventory.Plan) []string {
var out []string
for _, m := range sortedKeysOf(planModuleNames(p)) {
s := p.Modules[m]
for _, n := range sortedKeysOf(s.Recorded) {
out = append(out, fmt.Sprintf("%s on %s %s", m, n, s.Recorded[n]))
}
}
return out
}
func planModuleNames(p inventory.Plan) map[string]string {
out := map[string]string{}
for m, s := range p.Modules {
if s != nil && len(s.Recorded) > 0 {
out[m] = m
}
}
return out
}
// recordedWaiting is, on one machine, every move of a recorded module that waits for a person, each said with why
// (ADR 0377 §6): what `status` names the machine behind for, with `push <node>` as the remedy. A move under a
// module's own `record` that recreates nothing and moves no data is not among them: it goes with its walk.
func recordedWaiting(ctx context.Context, inv *inventory.Inventory, f moveFacts, node string) ([]string, error) {
sent, known, err := inv.SentBuilds(ctx, node)
if err != nil || !known {
return nil, err
}
var out []string
for _, m := range sortedKeysOf(sent) {
was := sent[m]
now, held := f.current[m]
if !held || now.RollOut || now.Commit == "" || was == "" || f.identical(m, was, now.Commit) {
continue
}
if v := f.recordedMove(m, was); !v.rolls {
out = append(out, fmt.Sprintf("%s %s → %s %s", m, short(was), short(now.Commit), v.said))
}
}
return out, nil
}
// stringsKeys is a map of lists as a map of its keys, for sortedKeysOf.
func stringsKeys(m map[string][]string) map[string]string {
out := map[string]string{}
for k := range m {
out[k] = k
}
return out
}
-361
View File
@@ -1,361 +0,0 @@
package main
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// Under a module's own `record`, a move that recreates no container and moves no data rolls out (novox/hq ADR
// 0377). Seen 2026-10-15: the database server moved with only its tools bundle changed, its walk said it
// "recreates none of its containers", and ace stayed behind until a person pushed.
// aMailu is a mail module as the catalogue declares it, small: it records because its manifest says so, keeps its
// mailboxes as data, mounts its environment file, and serves its tools from a bundle.
func aMailu(policy bool) catalogue.Manifest {
raw := `{"module":"mailu","version":"1","tools":["mailu_status"],
"data":{"own":[{"id":"mail","path":"${dir:data-mail}","class":"valuable","why":"every mailbox"}]},
"bundles":[{"name":"tools","source":"store/mailu-tools@sha256:aaa","digest":"sha256:aaa"}],
"resources":[
{"id":"state","type":"directory"},
{"id":"mesh-state","type":"directory"},
{"id":"data-mail","type":"directory"},
{"id":"config-env","type":"file","path":"${dir:state}/mailu.env","content":"A=1"},
{"id":"runtime-config","type":"file","path":"${dir:mesh-state}/config.json","content":"{}"},
{"id":"imap","type":"container","name":"mailu-imap","image":"registry.invalid/imap@sha256:1",
"volumes":["${dir:data-mail}:/mail"],"env-file":["${dir:state}/mailu.env"]}]}`
var m catalogue.Manifest
if err := json.Unmarshal([]byte(raw), &m); err != nil {
panic(err)
}
if policy {
m.Upgrade = &catalogue.UpgradePolicy{Policy: catalogue.PolicyRecord, Why: "people's mail: a person chooses the moment"}
}
return m
}
func resourceOf(m *catalogue.Manifest, id string) map[string]any {
for _, r := range m.Resources {
if r["id"] == id {
return r
}
}
panic("no resource " + id)
}
// recordedMesh is a mesh running mailu at build c1 on anchor and laptop, as each was last sent.
type recordedMesh struct {
open *stores
sent [][]string
}
func aRecordedMesh(t *testing.T, c1 catalogue.Manifest) *recordedMesh {
t.Helper()
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
rm := &recordedMesh{open: open}
rm.register(t, c1, "c1", time.Now().Add(-2*time.Hour))
for _, n := range []string{"anchor", "laptop"} {
if _, err := inv.Assign(ctx, n, c1.Module); err != nil {
t.Fatal(err)
}
if err := inv.RecordSent(ctx, nodeID(t, open, n), "d-"+n+"-c1", map[string]string{c1.Module: "c1"}); err != nil {
t.Fatal(err)
}
}
return rm
}
// register keeps a build of a module and makes it the mesh's.
func (rm *recordedMesh) register(t *testing.T, m catalogue.Manifest, commit string, at time.Time) {
t.Helper()
ctx := t.Context()
raw, _ := json.Marshal(m)
if err := rm.open.inventory.RecordBuild(ctx, inventory.Build{ID: "build-" + m.Module + "-" + commit, Module: m.Module,
Commit: commit, Repository: "novox/mesh-catalog", Path: "modules/" + m.Module, Manifest: raw, Asked: at, At: at}); err != nil {
t.Fatal(err)
}
if err := rm.open.inventory.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-catalog", Seat: "git",
Path: "modules/" + m.Module, BuiltFrom: commit, Head: commit, Asked: at}); err != nil {
t.Fatal(err)
}
}
// keptOn is what an ordinary send — a walk's, a healer's — keeps on a machine: module → the build it keeps.
func (rm *recordedMesh) keptOn(t *testing.T, node string) map[string]string {
t.Helper()
ctx := t.Context()
under, err := sendKeeps(ctx, rm.open.inventory)
if err != nil {
t.Fatal(err)
}
kept, err := recordedKept(under, rm.open, node)
if err != nil {
t.Fatal(err)
}
return kept
}
// walk opens a merge's walk of mailu's c2, built, and walks it with every send composed as an ordinary send
// composes it — a recorded module kept where its move waits — and applied at once.
func (rm *recordedMesh) walk(t *testing.T) inventory.Plan {
t.Helper()
ctx := t.Context()
inv := rm.open.inventory
wasSend := sendRollout
n := 0
sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) {
rm.sent = append(rm.sent, append([]string(nil), names...))
current, err := open.inventory.CurrentBuilds(ctx)
if err != nil {
return nil, err
}
for _, node := range names {
kept := rm.keptOn(t, node)
builds := map[string]string{"mailu": current["mailu"].Commit}
if was, held := kept["mailu"]; held {
builds["mailu"] = was
}
n++
digest := fmt.Sprintf("d-%s-%d", node, n)
if err := inv.RecordSent(ctx, nodeID(t, open, node), digest, builds); err != nil {
return nil, err
}
if _, err := inv.RecordDoing(ctx, nodeID(t, open, node), inventory.Doing{Node: node,
Outcome: inventory.OutcomeApplied, Declared: digest, Applied: 1, At: time.Now()}); err != nil {
return nil, err
}
}
return names, nil
}
t.Cleanup(func() { sendRollout = wasSend })
built := time.Now().UTC()
plan := inventory.Plan{ID: "plan-mailu", Repository: "novox/mesh-catalog", Branch: "main", Commit: "c2",
Created: built, State: inventory.PlanBuilding, Tiers: [][]string{{"mailu"}},
Modules: map[string]*inventory.PlanModule{"mailu": {State: "built", BuiltAt: &built, Commit: "c2", Build: "build-mailu-c2"}}}
if err := inv.SavePlan(ctx, &plan); err != nil {
t.Fatal(err)
}
for i := 0; i < 8; i++ {
advancePlans(ctx, rm.open)
}
p, err := inv.PlanByID(ctx, "plan-mailu")
if err != nil {
t.Fatal(err)
}
return p
}
func (rm *recordedMesh) sentBuild(t *testing.T, node string) string {
t.Helper()
sent, _, err := rm.open.inventory.SentBuilds(t.Context(), node)
if err != nil {
t.Fatal(err)
}
return sent["mailu"]
}
// recordedWaitsOn is what `status` names a machine behind for, waiting for a person (ADR 0377 §6).
func (rm *recordedMesh) recordedWaitsOn(t *testing.T, node string) []string {
t.Helper()
f, err := readMoveFacts(t.Context(), rm.open.inventory)
if err != nil {
t.Fatal(err)
}
held, err := recordedWaiting(t.Context(), rm.open.inventory, f, node)
if err != nil {
t.Fatal(err)
}
return held
}
// §1, §6: a module that records because its manifest says so, whose move changes only its tools bundle, goes out
// with its walk to every machine, without a push; the walk says per machine that it went out and why; the policy
// still reads record; and status names no machine behind for it.
func TestARecordedModulesBundleOnlyMoveRollsOutInItsWalk(t *testing.T) {
c1 := aMailu(true)
rm := aRecordedMesh(t, c1)
c2 := aMailu(true)
c2.Bundles[0].Digest, c2.Bundles[0].Source = "sha256:bbb", "store/mailu-tools@sha256:bbb"
c2.Tools = append(c2.Tools, "mailu_queue")
rm.register(t, c2, "c2", time.Now().Add(-time.Minute))
for _, n := range []string{"anchor", "laptop"} {
if _, kept := rm.keptOn(t, n)["mailu"]; kept {
t.Fatalf("an ordinary send to %s keeps mailu at c1 for a move that recreates nothing", n)
}
}
p := rm.walk(t)
if p.State != inventory.PlanDone {
t.Fatalf("the walk is %s: %s", p.State, p.Note)
}
for _, n := range []string{"anchor", "laptop"} {
if got := rm.sentBuild(t, n); got != "c2" {
t.Fatalf("%s was sent mailu %s; want c2, with its walk", n, got)
}
if said := p.Modules["mailu"].Recorded[n]; said != recordedRollsWhy {
t.Fatalf("the walk says of mailu on %s %q", n, said)
}
if held := rm.recordedWaitsOn(t, n); len(held) != 0 {
t.Fatalf("status names %s behind for %v", n, held)
}
}
if !strings.Contains(p.Note, "mailu on anchor went out") || !strings.Contains(p.Note, "mailu on laptop went out") {
t.Fatalf("the walk's note does not say which way mailu went: %q", p.Note)
}
if u, err := rm.open.inventory.UpgradeOf(t.Context(), "mailu"); err != nil || u.Policy() != catalogue.PolicyRecord {
t.Fatalf("mailu's policy is %q (%v); want record still", u.Policy(), err)
}
}
// §1, §2, §6: a move that recreates a container — a new image, or a changed declaration — still waits for a push;
// the walk says per machine that it waits, with the recreate words; and status names the machine behind, with
// `push <node>` as the remedy. A bundle-only change on top of it stays held: it is judged against what the machine
// was last sent.
func TestARecordedModulesMoveThatRecreatesAContainerWaits(t *testing.T) {
for name, change := range map[string]func(m *catalogue.Manifest){
"image": func(m *catalogue.Manifest) { resourceOf(m, "imap")["image"] = "registry.invalid/imap@sha256:2" },
"spec": func(m *catalogue.Manifest) { resourceOf(m, "imap")["health"] = map[string]any{"kind": "runtime"} },
} {
t.Run(name, func(t *testing.T) {
rm := aRecordedMesh(t, aMailu(true))
c2 := aMailu(true)
change(&c2)
c2.Bundles[0].Digest = "sha256:bbb"
rm.register(t, c2, "c2", time.Now().Add(-time.Minute))
p := rm.walk(t)
for _, n := range []string{"anchor", "laptop"} {
if got := rm.sentBuild(t, n); got != "c1" {
t.Fatalf("%s was sent mailu %s; want c1 kept, waiting for a push", n, got)
}
if said := p.Modules["mailu"].Recorded[n]; !strings.HasPrefix(said, "waits: recreates 1 of mailu's 1") {
t.Fatalf("the walk says of mailu on %s %q", n, said)
}
if held := rm.recordedWaitsOn(t, n); len(held) != 1 || !strings.Contains(held[0], "recreates") {
t.Fatalf("status names %s behind for %v", n, held)
}
}
// §2: the next build changes only the bundle, and the machines were last sent c1: still held.
c3 := c2
c3.Bundles = []catalogue.Bundle{{Name: "tools", Source: "store/mailu-tools@sha256:ccc", Digest: "sha256:ccc"}}
rm.register(t, c3, "c3", time.Now())
if kept := rm.keptOn(t, "anchor"); kept["mailu"] != "c1" {
t.Fatalf("a bundle-only build on top of a held container change is not held: %v", kept)
}
})
}
}
// §3: a move touching a file a container mounts, a directory holding declared data, the data declaration or a
// provider waits.
func TestARecordedModulesMoveThatTouchesDataOrAProviderWaits(t *testing.T) {
for name, c := range map[string]struct {
change func(m *catalogue.Manifest)
says string
}{
"a mounted file": {func(m *catalogue.Manifest) { resourceOf(m, "config-env")["content"] = "A=2" }, "which its container imap mounts"},
"a data directory": {func(m *catalogue.Manifest) { resourceOf(m, "data-mail")["mode"] = "0700" }, "data-mail"},
"the data declaration": {func(m *catalogue.Manifest) {
m.Data = &catalogue.Data{Own: []catalogue.DataItem{{ID: "mail", Path: "${dir:data-mail}", Class: "irreplaceable", Why: "x"}}}
}, "data declaration"},
"a provider": {func(m *catalogue.Manifest) { m.Requires = append(m.Requires, "postgres-database") }, "what it requires"},
} {
t.Run(name, func(t *testing.T) {
rm := aRecordedMesh(t, aMailu(true))
c2 := aMailu(true)
c.change(&c2)
rm.register(t, c2, "c2", time.Now().Add(-time.Minute))
kept := rm.keptOn(t, "anchor")
if kept["mailu"] != "c1" {
t.Fatalf("an ordinary send does not keep mailu at c1: %v", kept)
}
held := rm.recordedWaitsOn(t, "anchor")
if len(held) != 1 || !strings.Contains(held[0], c.says) {
t.Fatalf("status says %v; want it to name %q", held, c.says)
}
})
}
}
// §5: a module a person set to record, and the bus, wait on a bundle-only move.
func TestAPersonsRecordAndTheBusHoldEveryMove(t *testing.T) {
t.Run("a person sets record", func(t *testing.T) {
rm := aRecordedMesh(t, aMailu(false))
c2 := aMailu(false)
c2.Bundles[0].Digest = "sha256:bbb"
rm.register(t, c2, "c2", time.Now().Add(-time.Minute))
if err := rm.open.inventory.SetUpgradeOf(t.Context(), "mailu", inventory.Upgrade{Why: "the migration week", By: "jochen"}); err != nil {
t.Fatal(err)
}
p := rm.walk(t)
if got := rm.sentBuild(t, "anchor"); got != "c1" {
t.Fatalf("a person's record was walked past: anchor was sent %s", got)
}
if said := p.Modules["mailu"].Recorded["anchor"]; !strings.Contains(said, "a person set it to record") {
t.Fatalf("the walk says %q", said)
}
if held := rm.recordedWaitsOn(t, "anchor"); len(held) != 1 {
t.Fatalf("status names anchor behind for %v", held)
}
})
t.Run("the bus", func(t *testing.T) {
bus := aMailu(false)
bus.Module = "nats"
bus.Provides = []catalogue.Offer{{Name: "mesh-bus"}}
bus.Data = nil
rm := aRecordedMesh(t, bus)
c2 := bus
c2.Bundles = []catalogue.Bundle{{Name: "tools", Source: "store/nats-tools@sha256:bbb", Digest: "sha256:bbb"}}
rm.register(t, c2, "c2", time.Now().Add(-time.Minute))
if kept := rm.keptOn(t, "anchor"); kept["nats"] != "c1" {
t.Fatalf("the bus moved by an ordinary send: %v", kept)
}
f, err := readMoveFacts(t.Context(), rm.open.inventory)
if err != nil {
t.Fatal(err)
}
if v := f.recordedMove("nats", "c1"); v.rolls || !strings.Contains(v.said, "bus") {
t.Fatalf("the bus's move: %+v", v)
}
})
}
// §4: a move whose earlier build is not in the build records cannot be judged, waits, and is said so.
func TestARecordedMoveThatCannotBeJudgedWaits(t *testing.T) {
rm := aRecordedMesh(t, aMailu(true))
c2 := aMailu(true)
c2.Bundles[0].Digest = "sha256:bbb"
rm.register(t, c2, "c2", time.Now().Add(-time.Minute))
// anchor was last sent a build the records do not hold.
if err := rm.open.inventory.RecordSent(t.Context(), nodeID(t, rm.open, "anchor"), "d-anchor-c0",
map[string]string{"mailu": "c0"}); err != nil {
t.Fatal(err)
}
f, err := readMoveFacts(t.Context(), rm.open.inventory)
if err != nil {
t.Fatal(err)
}
if v := f.recordedMove("mailu", "c0"); v.rolls || !strings.Contains(v.said, "could not be judged") ||
!strings.Contains(v.said, "not in the build records") {
t.Fatalf("an unjudgeable move: %+v", v)
}
if _, _, err := planFor(keepingRecorded(t.Context()), rm.open, "anchor"); err == nil || !strings.Contains(err.Error(), "push anchor") {
t.Fatalf("a send that cannot keep the unjudged build did not refuse with its remedy: %v", err)
}
// laptop, judged from c1, goes; anchor waits.
p := rm.walk(t)
if said := p.Modules["mailu"].Recorded["anchor"]; !strings.Contains(said, "could not be judged") {
t.Fatalf("the walk says of anchor %q", said)
}
if !slices.Contains(wentOut(p.Modules["mailu"].Recorded), "laptop") || rm.sentBuild(t, "laptop") != "c2" {
t.Fatalf("laptop, judged from c1, did not go: %+v", p.Modules["mailu"].Recorded)
}
}
@@ -1,39 +0,0 @@
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/broker"
)
// A membership issued again for what it hears changes nothing else of it (novox/hq ADR 0363 §6).
func TestAMembershipIssuedAgainChangesOnlyWhatItHears(t *testing.T) {
issued := []byte(`{"node":"ace","module":"reader","receives":{"x":[1]},"emits":"mesh.mod.reader.event.{event}"}`)
hears := []broker.HeardIssued{{Stream: "MAIL", Consumer: "ace_reader"}}
body, changed, err := withHearing(issued, hears)
if err != nil || !changed {
t.Fatalf("changed %v, %v", changed, err)
}
var got map[string]any
if err := json.Unmarshal(body, &got); err != nil {
t.Fatal(err)
}
if got["receives"] == nil || got["emits"] != "mesh.mod.reader.event.{event}" || got["hears"] == nil {
t.Fatalf("issued again as %s", body)
}
if _, again, _ := withHearing(body, hears); again {
t.Fatal("the same hearing issued again counts as a change")
}
gone, changed, err := withHearing(body, nil)
if err != nil || !changed {
t.Fatalf("a withdrawal: %v, %v", changed, err)
}
got = nil
if err := json.Unmarshal(gone, &got); err != nil {
t.Fatal(err)
}
if _, still := got["hears"]; still {
t.Fatalf("a withdrawn hearing is still issued: %s", gone)
}
}
+3 -39
View File
@@ -83,9 +83,6 @@ type moveFacts struct {
plans []inventory.Plan plans []inventory.Plan
// self is the release walking these moves, whose own gates are no other walk's (novox/hq ADR 0353). // self is the release walking these moves, whose own gates are no other walk's (novox/hq ADR 0353).
self string self string
// judge judges a move under a module's own `record` (novox/hq ADR 0377): nil when the facts were not read,
// and every such move then waits.
judge func(module, was, now string) recordVerdict
} }
type walkingKey struct{} type walkingKey struct{}
@@ -112,7 +109,6 @@ func readMoveFacts(ctx context.Context, inv *inventory.Inventory) (moveFacts, er
} }
f.plans, err = inv.OpenPlans(ctx) f.plans, err = inv.OpenPlans(ctx)
f.self, _ = ctx.Value(walkingKey{}).(string) f.self, _ = ctx.Value(walkingKey{}).(string)
f.judge = recordJudge(ctx, inv)
return f, err return f, err
} }
@@ -176,8 +172,7 @@ func inRun(run map[string]bool, commit string) bool {
} }
// moves is what a machine's next send would move that no gate has seen: modules whose policy rolls out // moves is what a machine's next send would move that no gate has seen: modules whose policy rolls out
// (a recorded one is a person's push, unless its move under its own record recreates nothing and moves no data — // (a recorded one is a person's push), that the machine was sent before, moving to a build not identical
// novox/hq ADR 0377), that the machine was sent before, moving to a build not identical
// to the one it runs and that has passed no gate. A machine whose last send's builds are not known names // to the one it runs and that has passed no gate. A machine whose last send's builds are not known names
// none: the cascade holds it whole (ADR 0221). // none: the cascade holds it whole (ADR 0221).
// //
@@ -190,7 +185,7 @@ func (f moveFacts) moves(node string, modules []string, sent map[string]string,
for _, m := range modules { for _, m := range modules {
now := f.current[m] now := f.current[m]
was, carried := sent[m] was, carried := sent[m]
if !carried || f.identical(m, was, now.Commit) || (!all && f.gated(m, now.Commit)) || !f.rolls(m, was) { if !now.RollOut || !carried || f.identical(m, was, now.Commit) || (!all && f.gated(m, now.Commit)) {
continue continue
} }
out = append(out, inventory.CarriedMove{Module: m, Node: node, From: was, To: now.Commit}) out = append(out, inventory.CarriedMove{Module: m, Node: node, From: was, To: now.Commit})
@@ -755,11 +750,6 @@ func advanceRelease(ctx context.Context, open *stores, p *inventory.Plan) (bool,
if r.Rest { if r.Rest {
return advanceRest(ctx, open, p, now) return advanceRest(ctx, open, p, now)
} }
if r.Gate == nil && !gateFirstMachine {
// **No first machine** (novox/hq ADR 0360 §7): every machine heard from is sent at once, in one step.
r.Rest = true
return advanceRest(ctx, open, p, now)
}
if r.Gate == nil { if r.Gate == nil {
heard, err := releaseHeard(ctx, open) heard, err := releaseHeard(ctx, open)
if err != nil { if err != nil {
@@ -1025,25 +1015,6 @@ func advanceRest(ctx context.Context, open *stores, p *inventory.Plan, now time.
judging = append(judging, node+": not yet reported on its send") judging = append(judging, node+": not yet reported on its send")
continue continue
} }
if !gateFirstMachine {
// **Judged by its report alone** (novox/hq ADR 0360 §7): applied passes; failed or refused is named and
// fails the walk, with nothing put back — apply-failing says it.
switch rep := byNode[node]; rep.Outcome {
case inventory.OutcomeApplied:
decide(g, inventory.GatePassed, node+" reported it applied its send (novox/hq ADR 0360)", now)
passCarried(ctx, open, p, g, "")
r.Done = append(r.Done, node)
changed = true
case inventory.OutcomeFailed, inventory.OutcomeRefused:
decide(g, inventory.GateFailed, fmt.Sprintf("%s %s its send", node, rep.Outcome), now)
r.Failed = append(r.Failed, node)
changed = true
notes = append(notes, fmt.Sprintf("%s %s its send: nothing put back, apply-failing says it", node, rep.Outcome))
default:
judging = append(judging, node+": not yet reported on its send")
}
continue
}
verdict, err := judgeMoves(ctx, open, g, nil, now) verdict, err := judgeMoves(ctx, open, g, nil, now)
if err != nil { if err != nil {
return changed, err return changed, err
@@ -1128,9 +1099,6 @@ func advanceRest(ctx context.Context, open *stores, p *inventory.Plan, now time.
// keptStepGrew says why a step's kept sends would now carry a build no gate passed that they were not kept with; // keptStepGrew says why a step's kept sends would now carry a build no gate passed that they were not kept with;
// empty when none would. // empty when none would.
func keptStepGrew(ctx context.Context, open *stores, r *inventory.PlanRelease, pending []string) (string, error) { func keptStepGrew(ctx context.Context, open *stores, r *inventory.PlanRelease, pending []string) (string, error) {
if !gateFirstMachine {
return "", nil // every machine goes at once (novox/hq ADR 0360 §7): a send carries what waits when it is made
}
f, err := readMoveFacts(ctx, open.inventory) f, err := readMoveFacts(ctx, open.inventory)
if err != nil { if err != nil {
return "", err return "", err
@@ -1239,11 +1207,7 @@ func openRestStep(ctx context.Context, open *stores, p *inventory.Plan) (bool, s
if err != nil { if err != nil {
return false, "", err return false, "", err
} }
// **Every machine at once** (novox/hq ADR 0360 §2, §7): no build waits for one machine to pass first. step := restStep(nodes, waiting, f.gated, func(module string) bool { return oneAtATime(module, shelf[module]) })
step := nodes
if gateFirstMachine {
step = restStep(nodes, waiting, f.gated, func(module string) bool { return oneAtATime(module, shelf[module]) })
}
if len(step) == 0 { if len(step) == 0 {
if len(walked) > 0 { if len(walked) > 0 {
return false, "waiting before " + strings.Join(walked, "; "), nil return false, "waiting before " + strings.Join(walked, "; "), nil
-273
View File
@@ -1,273 +0,0 @@
package main
import (
"context"
"errors"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A pull request into a release branch is judged by that branch** (novox/hq ADR 0358, issue 704): a base that is
// not its repository's trunk, where mesh-controller has a branch of that name, is a release branch; the check then
// reads every repository beside it at its branch of that name where it has one, else at its trunk. Any other base is
// checked exactly as before.
const aRelease = "feat/57-config-and-provisioning"
// theCore is the mesh's core as a check reads it beside: the controller, the node-engine, the tool runner and the
// catalogue's bus, on the git seat.
func theCore() []inventory.Entry {
onSeat := func(module, repository string) inventory.Entry {
e := fromRepo(module, repository, "")
e.Source.Seat = "git"
return e
}
nats := onSeat("nats", "novox/mesh-catalog")
nats.Source.Path = "modules/nats"
return []inventory.Entry{onSeat("mesh-controller", "novox/mesh-controller"), onSeat("mesh-host", "novox/mesh-host"),
onSeat("node-tools", "novox/mesh-tools"), nats, onSeat("gitea", "novox/mesh-catalog")}
}
// aForgeWithBranches stands in for the forge: which branches each repository has, and how often it was asked.
func aForgeWithBranches(t *testing.T, branches map[string][]string, failing error) *int {
t.Helper()
asked := 0
was := forgeHasBranch
t.Cleanup(func() { forgeHasBranch = was })
forgeHasBranch = func(_ context.Context, owner, repo, branch string) (bool, error) {
asked++
if failing != nil {
return false, failing
}
for _, b := range branches[owner+"/"+repo] {
if b == branch {
return true, nil
}
}
return false, nil
}
return &asked
}
func TestABaseIsAReleaseBranchWhenTheControllerHasABranchOfThatName(t *testing.T) {
entries := theCore()
scopeOf := func(p link.PullUpdated) checkScope { return pullScope(p, entries, nil, nil) }
pull := func(repo, base string) link.PullUpdated {
return link.PullUpdated{Owner: "novox", Repo: repo, Number: 286, Base: base, Commit: "abc",
Paths: []string{"modules/gitea/index.ts"}}
}
asked := aForgeWithBranches(t, map[string][]string{"novox/mesh-controller": {"main", aRelease}}, nil)
for _, c := range []struct {
what string
p link.PullUpdated
release string
asks int
}{
{"a pull request into the trunk asks nothing and is no release", pull("mesh-catalog", "main"), "", 0},
{"a catalogue pull request into the release branch", pull("mesh-catalog", aRelease), aRelease, 1},
{"a controller pull request into the release branch", pull("mesh-controller", aRelease), aRelease, 1},
{"a base mesh-controller has no branch of is checked as today", pull("mesh-catalog", "feat/other"), "", 1},
{"no base: as today", pull("mesh-catalog", ""), "", 0},
} {
*asked = 0
got, err := releaseBranchOf(t.Context(), c.p, scopeOf(c.p), entries)
if err != nil {
t.Fatalf("%s: %v", c.what, err)
}
if got != c.release || *asked != c.asks {
t.Errorf("%s: release %q after %d question(s) to the forge, not %q after %d", c.what, got, *asked,
c.release, c.asks)
}
}
// The branch a module built from the repository follows is its trunk: never asked about.
photos := fromRepo("photos", "novox/photos", "")
photos.Source.Ref = "nox-mesh"
withPhotos := append(theCore(), photos)
p := link.PullUpdated{Owner: "novox", Repo: "photos", Base: "nox-mesh", Commit: "abc", Paths: []string{"x.js"}}
*asked = 0
if got, err := releaseBranchOf(t.Context(), p, pullScope(p, withPhotos, nil, nil), withPhotos); err != nil ||
got != "" || *asked != 0 {
t.Fatalf("a module's own trunk read as release %q (%v) after %d question(s)", got, err, *asked)
}
// A forge that cannot answer is an error, never "no release branch".
aForgeWithBranches(t, nil, errors.New("nothing serves gitea.gitea_api"))
q := pull("mesh-catalog", aRelease)
if _, err := releaseBranchOf(t.Context(), q, scopeOf(q), entries); err == nil ||
!strings.Contains(err.Error(), "cannot be told") {
t.Fatalf("an unanswered forge was read as an answer: %v", err)
}
}
func TestBesideAReleaseBranchCheckEachRepositoryIsReadAtThatBranchElseItsTrunk(t *testing.T) {
entries := theCore()
clone := func(s inventory.Source) (string, error) { return "http://git/" + s.Repository + ".git", nil }
running := func(module string) string { return "running-" + module }
beside, err := besideOf(entries, running, clone, aRelease)
if err != nil {
t.Fatal(err)
}
at := func(repo string, ref, orElse string) link.CheckedOut {
return link.CheckedOut{Repository: "http://git/novox/" + repo + ".git", Ref: ref, Else: orElse}
}
want := map[string]link.CheckedOut{
// The judge: mesh-controller's release branch, which it has.
"mesh-controller": at("mesh-controller", aRelease, ""),
"mesh-controller-main": at("mesh-controller", "main", ""),
// A controller pull request into the release branch reads mesh-host at that branch where it has one.
"mesh-host": at("mesh-host", aRelease, "main"),
"mesh-tools": at("mesh-tools", aRelease, "main"),
"mesh-catalog": at("mesh-catalog", aRelease, "main"),
// mesh-lab at its trunk, always: its replays run with the container runtime's socket.
"mesh-lab": at("mesh-lab", "main", ""),
"mesh-sdk": at("mesh-sdk", aRelease, "main"),
}
if !reflect.DeepEqual(beside, want) {
t.Fatalf("beside a release-branch check:\n got %v\nwant %v", beside, want)
}
// Any other base: exactly as before — what the mesh runs, the catalogue's and the lab's main.
beside, err = besideOf(entries, running, clone, "")
if err != nil {
t.Fatal(err)
}
want = map[string]link.CheckedOut{
"mesh-controller": at("mesh-controller", "running-mesh-controller", ""),
"mesh-controller-main": at("mesh-controller", "main", ""),
"mesh-host": at("mesh-host", "running-mesh-host", ""),
"mesh-tools": at("mesh-tools", "running-node-tools", ""),
"mesh-catalog": at("mesh-catalog", "main", ""),
"mesh-lab": at("mesh-lab", "main", ""),
"mesh-sdk": at("mesh-sdk", sdkPinned(), ""),
}
if !reflect.DeepEqual(beside, want) {
t.Fatalf("beside a trunk check:\n got %v\nwant %v", beside, want)
}
}
func TestAReleaseBranchVerdictNamesTheBranchTheJudgeAndWhatItRead(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog", Number: 286, Base: aRelease, Release: aRelease}
c := checkedOf(link.BuildResult{ID: "b", Ref: "aaaaaaaaaaaa0000", Checked: asked, Check: &link.CheckOutcome{
Verdict: "pass", Head: "aaaaaaaaaaaa", Trunk: "111111111111", Gate: &link.CheckLayer{Verdict: "pass"},
RepoCheck: &link.CheckLayer{Verdict: "pass"}, Release: aRelease, JudgedBy: "fcdecb436117",
Beside: map[string]string{"mesh-host": "222222222222", "mesh-lab": "333333333333"},
BesideAt: map[string]string{"mesh-host": aRelease, "mesh-lab": "main"},
Validator: "mesh-host's validator of " + aRelease + " at 222222222222"}})
if c.Release != aRelease || c.JudgedBy != "fcdecb436117" {
t.Fatalf("the verdict names release %q judged by %q", c.Release, c.JudgedBy)
}
if c.BesideAt["mesh-host"] != aRelease || c.BesideAt["mesh-lab"] != "main" || c.Beside["mesh-host"] != "222222222222" ||
c.Validator != "mesh-host's validator of "+aRelease+" at 222222222222" {
t.Fatalf("the verdict names what it read beside it as %v at %v", c.Beside, c.BesideAt)
}
}
// **Clause 5** (ADR 0358 §5): a release branch's own pull request into its trunk is no release-branch check: the forge
// is asked nothing, the request names no release branch, and the repositories beside it are read as for any trunk
// pull request. Its delivery group's composed check is asked as for any other (askGroupCheck, unchanged).
func TestAReleaseBranchsPullRequestIntoItsTrunkIsCheckedAsAnyTrunkPullRequest(t *testing.T) {
entries := theCore()
asked := aForgeWithBranches(t, map[string][]string{"novox/mesh-controller": {"main", aRelease}}, nil)
for _, repo := range []string{"mesh-catalog", "mesh-controller", "mesh-host"} {
p := link.PullUpdated{Owner: "novox", Repo: repo, Number: 400, Head: aRelease, Base: "main", Commit: "abc",
Paths: []string{"modules/gitea/index.ts"}}
got, err := releaseBranchOf(t.Context(), p, pullScope(p, entries, nil, nil), entries)
if err != nil || got != "" || *asked != 0 {
t.Fatalf("%s's release branch into main read as release %q (%v) after %d question(s)", repo, got, err, *asked)
}
}
clone := func(s inventory.Source) (string, error) { return "http://git/" + s.Repository + ".git", nil }
beside, err := besideOf(entries, func(module string) string { return "running-" + module }, clone, "")
if err != nil {
t.Fatal(err)
}
for dir, b := range beside {
if b.Ref == aRelease || b.Else != "" {
t.Fatalf("beside a trunk pull request %s is read at %+v", dir, b)
}
}
}
// **A forge answer that names no branch is an error**, never "no release branch"; a 404 is "no such branch".
func TestTheForgesAnswerAboutABranchIsReadStrictly(t *testing.T) {
for _, c := range []struct {
what string
answer link.Answer
has bool
fails bool
}{
{"the branch named", link.Answer{Result: []byte(`{"result":{"name":"` + aRelease + `"}}`)}, true, false},
{"not found", link.Answer{Error: "Gitea API /repos/novox/mesh-controller/branches/x: 404 {\"message\":\"not found\"}"},
false, false},
{"an empty branch name", link.Answer{Result: []byte(`{"result":{"name":""}}`)}, false, true},
{"no result", link.Answer{Result: []byte(`{}`)}, false, true},
{"another branch named", link.Answer{Result: []byte(`{"result":{"name":"main"}}`)}, false, true},
{"unreadable", link.Answer{Result: []byte(`not json`)}, false, true},
{"another error", link.Answer{Error: "Gitea API: 500 internal"}, false, true},
} {
has, err := branchAnswered(c.answer, aRelease)
if has != c.has || (err != nil) != c.fails {
t.Errorf("%s: has %v, error %v", c.what, has, err)
}
}
}
// **The trunk is the repository's own, not a name**: a repository whose trunk is not main, one that follows main, and
// a controller whose trunk is not main.
func TestATrunkIsTheBranchTheRepositoryFollows(t *testing.T) {
asked := aForgeWithBranches(t, map[string][]string{"novox/mesh-controller": {"main", aRelease, "master"}}, nil)
legacy := fromRepo("legacy", "novox/legacy", "")
legacy.Source.Ref = "master"
onMain := fromRepo("photos", "novox/photos", "")
onMain.Source.Ref = "main"
entries := append(theCore(), legacy, onMain)
for _, c := range []struct {
what, repo, base, release string
asks int
}{
{"a repository whose trunk is master, into master", "legacy", "master", "", 0},
{"a repository whose trunk is master, into the release branch", "legacy", aRelease, aRelease, 1},
{"a repository whose module follows main, into main", "photos", "main", "", 0},
{"a repository whose module follows main, into the release branch", "photos", aRelease, aRelease, 1},
} {
*asked = 0
p := link.PullUpdated{Owner: "novox", Repo: c.repo, Base: c.base, Commit: "abc", Paths: []string{"x"}}
got, err := releaseBranchOf(t.Context(), p, pullScope(p, entries, nil, nil), entries)
if err != nil || got != c.release || *asked != c.asks {
t.Errorf("%s: release %q (%v) after %d question(s), not %q after %d", c.what, got, err, *asked, c.release, c.asks)
}
}
// A controller whose trunk is not main: main may be a release branch there, and its own trunk never is.
core := theCore()
core[0].Source.Ref = "trunk"
for _, c := range []struct {
base, release string
asks int
}{{"trunk", "", 0}, {"main", "main", 1}} {
*asked = 0
p := link.PullUpdated{Owner: "novox", Repo: "hq", Base: c.base, Commit: "abc", Paths: []string{"x"}}
got, err := releaseBranchOf(t.Context(), p, pullScope(p, core, nil, nil), core)
if err != nil || got != c.release || *asked != c.asks {
t.Errorf("hq into %s, the controller following trunk: release %q (%v) after %d question(s)", c.base, got, err,
*asked)
}
}
}
// trunkOf: the branch a source follows, and main when it records none or records a commit (issue 215's rule).
func TestATrunkIsMainWhenTheSourceRecordsNone(t *testing.T) {
for ref, want := range map[string]string{"": "main", "main": "main", "master": "master", "nox-mesh": "nox-mesh",
"0123456789abcdef0123456789abcdef01234567": "main", " ": "main"} {
if got := trunkOf(inventory.Source{Ref: ref}); got != want {
t.Errorf("a source recording %q follows %q, not %q", ref, got, want)
}
}
}
+81 -253
View File
@@ -9,13 +9,11 @@ import (
"slices" "slices"
"sort" "sort"
"strings" "strings"
"sync"
"time" "time"
"github.com/nats-io/nats.go" "github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/builder" "github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link" "github.com/novox/mesh-controller/internal/link"
) )
@@ -28,10 +26,6 @@ import (
// alone cannot (a tier waiting for machines to report); a controller replaced mid-plan finds the // alone cannot (a tier waiting for machines to report); a controller replaced mid-plan finds the
// plan where it left it. // plan where it left it.
// gateFirstMachine sends a module that rolls out to one machine first and judges it there (ADR 0236). Off since
// novox/hq ADR 0360: the gate's code is kept for the walks a controller before it started, and its tests turn it on.
var gateFirstMachine = false
// planWaitBound is how long a plan may wait on one thing before `status` names it red. // planWaitBound is how long a plan may wait on one thing before `status` names it red.
const planWaitBound = 30 * time.Minute const planWaitBound = 30 * time.Minute
@@ -48,15 +42,29 @@ func tiersOf(set []string, edges []inventory.Edge) [][]string {
for _, m := range set { for _, m := range set {
deps[m] = map[string]bool{} deps[m] = map[string]bool{}
} }
// The build seat's holders follow the controller that defines their worker (EdgeWorkerOf,
// novox/hq issue 206), so the built-by edge from that controller to such a holder yields: the
// controller is built by whichever build machine is running, as the runtime image always was.
worker := map[string]map[string]bool{}
for _, e := range edges {
if e.Kind == inventory.EdgeWorkerOf && in[e.From] && in[e.To] {
if worker[e.To] == nil {
worker[e.To] = map[string]bool{}
}
worker[e.To][e.From] = true
}
}
for _, e := range edges { for _, e := range edges {
// A code dependency — B packages A's source — rebuilds B with A, in the same tier: B's // A code dependency — B packages A's source — rebuilds B with A, in the same tier: B's
// build needs nothing of A's first. Stands-on and declared order after the base is built, and the // build needs nothing of A's first. The other kinds order: stands-on and declared after
// build seat's holder follows the controller that defines its worker (EdgeWorkerOf, issue 206). // the base is built, built-by after the build machine is built and running — except for
// what the build machine itself stands on, and for the controller whose worker the build
// machine binds. The runtime image is built by the builder and the builder is built on the
// runtime image; the image comes first, built by the builder that is running.
if !in[e.From] || !in[e.To] || e.From == e.To || e.Kind == inventory.EdgePackages { if !in[e.From] || !in[e.To] || e.From == e.To || e.Kind == inventory.EdgePackages {
continue continue
} }
// **Being built by the builder orders nothing** (novox/hq ADR 0360 §3): whichever builder runs builds it. if e.Kind == inventory.EdgeBuiltBy && (isBaseOf(e.From, e.To, edges, in) || worker[e.From][e.To]) {
if e.Kind == inventory.EdgeBuiltBy {
continue continue
} }
deps[e.From][e.To] = true deps[e.From][e.To] = true
@@ -97,6 +105,29 @@ func tiersOf(set []string, edges []inventory.Edge) [][]string {
return tiers return tiers
} }
// isBaseOf says whether `to` stands on `base`, directly or through other bases in the set, along
// the build edges alone.
func isBaseOf(base, to string, edges []inventory.Edge, in map[string]bool) bool {
seen := map[string]bool{}
var walk func(string) bool
walk = func(m string) bool {
if m == base {
return true
}
if seen[m] {
return false
}
seen[m] = true
for _, e := range edges {
if e.From == m && in[e.To] && (e.Kind == inventory.EdgeStandsOn || e.Kind == inventory.EdgeDeclared) && walk(e.To) {
return true
}
}
return false
}
return walk(to)
}
// reachableFrom is the moved modules plus everything that depends on them, through every layer: // reachableFrom is the moved modules plus everything that depends on them, through every layer:
// what a merge rebuilds. Along the code and build edges only: a module *built by* the build machine // what a merge rebuilds. Along the code and build edges only: a module *built by* the build machine
// is not changed by a new build machine, so a built-by edge orders and gates a plan and never // is not changed by a new build machine, so a built-by edge orders and gates a plan and never
@@ -132,7 +163,7 @@ func reachableFrom(moved []string, edges []inventory.Edge) []string {
} }
// hasCycle says whether the tiers' last tier holds modules that still depend on each other. A packages // hasCycle says whether the tiers' last tier holds modules that still depend on each other. A packages
// or built-by edge orders nothing (tiersOf), so a module and what packages its source share a tier by rule: that is // edge orders nothing (tiersOf), so a module and what packages its source share a tier by rule: that is
// no cycle, and saying one was is a false report in every such plan's log. // no cycle, and saying one was is a false report in every such plan's log.
func hasCycle(tiers [][]string, edges []inventory.Edge) bool { func hasCycle(tiers [][]string, edges []inventory.Edge) bool {
if len(tiers) == 0 { if len(tiers) == 0 {
@@ -143,7 +174,7 @@ func hasCycle(tiers [][]string, edges []inventory.Edge) bool {
last[m] = true last[m] = true
} }
for _, e := range edges { for _, e := range edges {
if e.Kind == inventory.EdgePackages || e.Kind == inventory.EdgeBuiltBy { if e.Kind == inventory.EdgePackages {
continue continue
} }
if last[e.From] && last[e.To] { if last[e.From] && last[e.To] {
@@ -189,6 +220,12 @@ func gates(p inventory.Plan, edges []inventory.Edge, rollsOut func(string) bool)
return nil return nil
} }
inTier := map[string]bool{} inTier := map[string]bool{}
all := map[string]bool{}
for _, tier := range p.Tiers {
for _, m := range tier {
all[m] = true
}
}
for _, m := range p.Tiers[p.Tier] { for _, m := range p.Tiers[p.Tier] {
inTier[m] = true inTier[m] = true
} }
@@ -203,8 +240,8 @@ func gates(p inventory.Plan, edges []inventory.Edge, rollsOut func(string) bool)
for _, e := range edges { for _, e := range edges {
// A delivery group's order inside a walk (novox/hq ADR 0276 decision 5) gates as built-by does: the // A delivery group's order inside a walk (novox/hq ADR 0276 decision 5) gates as built-by does: the
// member before is running on its machines before the member after is asked. // member before is running on its machines before the member after is asked.
// A built-by edge waits for nothing since novox/hq ADR 0360 §3. ordered := e.Kind == edgeGroupOrder ||
ordered := e.Kind == edgeGroupOrder e.Kind == inventory.EdgeBuiltBy && !isBaseOf(e.From, e.To, edges, all)
if later[e.From] && inTier[e.To] && ordered && !seen[e.To] && rollsOut(e.To) { if later[e.From] && inTier[e.To] && ordered && !seen[e.To] && rollsOut(e.To) {
seen[e.To] = true seen[e.To] = true
out = append(out, e.To) out = append(out, e.To)
@@ -259,11 +296,10 @@ func applied(module string, builtAt time.Time, running []string, reports []inven
return len(waiting) == 0, waiting return len(waiting) == 0, waiting
} }
// askTier asks the build machine for the named modules of the plan's tier — all of it, or what of it is still // askTier asks the build machine for every module of the tier, and marks each asked. A module
// unasked (novox/hq issue 754) — and marks each asked. A module
// the catalogue no longer holds, or whose ask could not be made, is a failure of the plan: a tier // the catalogue no longer holds, or whose ask could not be made, is a failure of the plan: a tier
// half asked is a tier that will never complete. // half asked is a tier that will never complete.
func askTier(ctx context.Context, inv *inventory.Inventory, p *inventory.Plan, names []string) error { func askTier(ctx context.Context, inv *inventory.Inventory, p *inventory.Plan) error {
entries, err := inv.Catalogued(ctx) entries, err := inv.Catalogued(ctx)
if err != nil { if err != nil {
return err return err
@@ -272,62 +308,12 @@ func askTier(ctx context.Context, inv *inventory.Inventory, p *inventory.Plan, n
for _, e := range entries { for _, e := range entries {
byName[e.Manifest.Module] = e byName[e.Manifest.Module] = e
} }
for _, name := range names { for _, name := range p.Tiers[p.Tier] {
askModule(ctx, p, name, byName) askModule(ctx, p, name, byName)
} }
return nil return nil
} }
// unaskedOf are the modules of a plan's current tier it has still to ask: those it never asked, and one it holds
// built that it never asked for (novox/hq issue 754) — a build taken in by a controller from before a walk took
// only its own asks, somebody else's, which the walk would otherwise send as its own (ADR 0219). A tier partly asked is asked the rest, not
// waited on for ever: a walk that waits for every build of a tier it never asked whole never moves again.
func unaskedOf(p inventory.Plan) []string {
if p.Tier >= len(p.Tiers) {
return nil
}
var out []string
for _, m := range p.Tiers[p.Tier] {
s := p.Modules[m]
switch {
case s == nil || s.State == "":
out = append(out, m)
case s.State == "built" && s.Build == "" && s.AskedAt == nil:
out = append(out, m)
}
}
return out
}
// askedInTier says the plan asked a module of its current tier itself: under an id, or at a time it kept.
func askedInTier(p inventory.Plan) bool {
if p.Tier >= len(p.Tiers) {
return false
}
for _, m := range p.Tiers[p.Tier] {
if s := p.Modules[m]; s != nil && (s.Build != "" || s.AskedAt != nil) {
return true
}
}
return false
}
// takesOutcome says whether a build's outcome is the plan's for a module of its current tier (novox/hq ADR 0219,
// issues 219 and 754). **A walk that waits for its delivery's word asks nothing, and so is answered by nothing**
// (ADR 0239): a rebuild that lands in the wait is somebody else's build. **Nor is a module the plan never asked
// answered**: taken, the plan held a module built that it never asked, and waited for the rest of the tier for
// ever. Asked under an id, the plan takes that id's outcome and no other; asked before ids were kept, a build
// asked at or after the plan's ask.
func takesOutcome(p inventory.Plan, state *inventory.PlanModule, asked time.Time, id string) bool {
if p.Waiting() || state == nil || (state.Build == "" && state.AskedAt == nil) {
return false
}
if state.Build != "" {
return state.Build == id
}
return !askedBefore(asked, state.AskedAt)
}
// askABuild is how a plan asks for one build, not waited for, and learns the id it asked under. A // askABuild is how a plan asks for one build, not waited for, and learns the id it asked under. A
// variable so a test of what a plan does around an ask needs no build machine. // variable so a test of what a plan does around an ask needs no build machine.
// //
@@ -400,73 +386,15 @@ var checkedBuildsOf = func(ctx context.Context, pulls []link.PullRef) ([]link.Ch
return link.ReadCheckBuilds(ctx, js, buildSeatHeld(ctx), pulls, time.Now().Add(-builder.CheckBuildsKept)) return link.ReadCheckBuilds(ctx, js, buildSeatHeld(ctx), pulls, time.Now().Add(-builder.CheckBuildsKept))
} }
// mergedHeadsOf are the head commits the walk's merges merged, by pull request (novox/hq issue 703): a pull request
// merged twice in one walk, or whose head the announcement did not say, has none, and takes no check's build.
func mergedHeadsOf(p *inventory.Plan) map[link.PullRef]string {
heads := map[link.PullRef]string{}
if p.Delivery == nil {
return heads
}
seen := map[link.PullRef]bool{}
for _, m := range p.Delivery.Merges {
pull, ok := pullOf(m)
if !ok {
continue
}
if seen[pull] && heads[pull] != m.Head {
heads[pull] = ""
continue
}
seen[pull] = true
heads[pull] = m.Head
}
return heads
}
// walkVerdicts are the checks' builds a walk read, by the walk's plan: read once per walk (novox/hq issue 649), not
// once per module it asks for. Kept a day; a walk lasts minutes.
var walkVerdicts = struct {
sync.Mutex
read map[string]verdictsRead
}{read: map[string]verdictsRead{}}
type verdictsRead struct {
at time.Time
builds []link.CheckedBuild
}
// reuseWithVerdicts is a walk's leave with the builds its pull requests' verdicts name: the only builds the builder // reuseWithVerdicts is a walk's leave with the builds its pull requests' verdicts name: the only builds the builder
// may take (ADR 0350 §2), and **only those of a check of the head each merge merged** (novox/hq issue 703): a build // may take (ADR 0350 §2). When they cannot be read the leave names none, and the walk builds, saying why.
// a check made at an earlier head of the same pull request is never registered. Read once per walk. When they func reuseWithVerdicts(ctx context.Context, r *link.ReuseAsk) *link.ReuseAsk {
// cannot be read the leave names none, and the walk builds, saying why.
func reuseWithVerdicts(ctx context.Context, walk string, r *link.ReuseAsk, heads map[link.PullRef]string) *link.ReuseAsk {
walkVerdicts.Lock()
read, known := walkVerdicts.read[walk]
walkVerdicts.Unlock()
if !known || walk == "" {
builds, err := checkedBuildsOf(ctx, r.Pulls) builds, err := checkedBuildsOf(ctx, r.Pulls)
if err != nil { if err != nil {
fmt.Printf(" the checks' builds of %d pull request(s) could not be read, so the walk builds: %v\n", len(r.Pulls), err) fmt.Printf(" the checks' builds of %d pull request(s) could not be read, so the walk builds: %v\n", len(r.Pulls), err)
return r return r
} }
read = verdictsRead{at: time.Now(), builds: builds} r.Builds = builds
if walk != "" {
walkVerdicts.Lock()
for id, w := range walkVerdicts.read {
if time.Since(w.at) > 24*time.Hour {
delete(walkVerdicts.read, id)
}
}
walkVerdicts.read[walk] = read
walkVerdicts.Unlock()
}
}
r.Builds = nil
for _, b := range read.builds {
if link.SameHead(b.Head, heads[b.Pull]) {
r.Builds = append(r.Builds, b)
}
}
return r return r
} }
@@ -509,8 +437,7 @@ func askModule(ctx context.Context, p *inventory.Plan, name string, byName map[s
} }
// **The walk takes the check's build of the same key** (novox/hq ADR 0350 §2): its ask says so, with the pull // **The walk takes the check's build of the same key** (novox/hq ADR 0350 §2): its ask says so, with the pull
// requests it answers, whose checks name the part of a differing key. // requests it answers, whose checks name the part of a differing key.
id, err := askABuild(withReuse(ctx, reuseWithVerdicts(ctx, p.ID, reuseAskOf(p), mergedHeadsOf(p))), source, id, err := askABuild(withReuse(ctx, reuseWithVerdicts(ctx, reuseAskOf(p))), source, e.Source.Path, ref)
e.Source.Path, ref)
if err != nil { if err != nil {
state.State = "failed" state.State = "failed"
state.Why = err.Error() state.Why = err.Error()
@@ -577,15 +504,22 @@ func planBuiltSaying(ctx context.Context, open *stores, module, commit, failed s
if !inTier { if !inTier {
continue continue
} }
state := p.Modules[module]
if state == nil {
state = &inventory.PlanModule{}
p.Modules[module] = state
}
// **The plan's own ask is its outcome, by id** (novox/hq ADR 0219); another build of the module // **The plan's own ask is its outcome, by id** (novox/hq ADR 0219); another build of the module
// is, as before, when it was asked at or after the plan's ask (issue 219). // is, as before, when it was asked at or after the plan's ask (issue 219).
// **A module asked under an id is answered by that id's outcome and no other** (novox/hq ADR // **A module asked under an id is answered by that id's outcome and no other** (novox/hq ADR
// 0219): a replay, a rebuild beside the plan, or an older ask finishing late is somebody else's // 0219): a replay, a rebuild beside the plan, or an older ask finishing late is somebody else's
// build, made from other source, and settling the plan with it would send that. A plan from // build, made from other source, and settling the plan with it would send that. A plan from
// before ids were kept is matched as it was: by when the build was asked (issue 219). A walk // before ids were kept is matched as it was: by when the build was asked (issue 219).
// waiting for its word, or a module it never asked, takes no outcome at all (issue 754). if state.Build != "" {
state := p.Modules[module] if state.Build != id {
if !takesOutcome(*p, state, asked, id) { continue
}
} else if askedBefore(asked, state.AskedAt) {
continue continue
} }
if failed != "" && deletedAtSource(failed) { if failed != "" && deletedAtSource(failed) {
@@ -791,11 +725,17 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
return true, nil return true, nil
} }
tier := p.Tiers[p.Tier] tier := p.Tiers[p.Tier]
// Not yet asked: ask — the whole tier, or what of it is still unasked (novox/hq issue 754). // Not yet asked: ask.
if unasked := unaskedOf(*p); len(unasked) > 0 { unasked := 0
for _, m := range tier {
if s := p.Modules[m]; s == nil || s.State == "" {
unasked++
}
}
if unasked == len(tier) {
// **One walk at a time** (novox/hq ADR 0276): a walk about to ask its first tier while another started // **One walk at a time** (novox/hq ADR 0276): a walk about to ask its first tier while another started
// walk is open waits for that one to end, let go or not, and says so. // walk is open waits for that one to end, let go or not, and says so.
if p.Tier == 0 && !askedInTier(*p) { if p.Tier == 0 {
if behind, err := startedBeside(ctx, inv, p.ID, p.Created); err != nil { if behind, err := startedBeside(ctx, inv, p.ID, p.Created); err != nil {
return false, err return false, err
} else if behind != "" { } else if behind != "" {
@@ -805,10 +745,7 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
return changed, nil return changed, nil
} }
} }
if len(unasked) < len(tier) { if err := askTier(ctx, inv, p); err != nil {
fmt.Printf("%s: tier %d was asked in part; asking the rest: %s\n", p.ID, p.Tier, strings.Join(unasked, ", "))
}
if err := askTier(ctx, inv, p, unasked); err != nil {
return false, err return false, err
} }
return true, nil return true, nil
@@ -865,32 +802,13 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
together := map[string]bool{} together := map[string]bool{}
for _, m := range tier { for _, m := range tier {
state := p.Modules[m] state := p.Modules[m]
if state == nil || state.SentAt != nil || state.State == planDeleted { if state == nil || state.SentAt != nil || state.State == planDeleted || !rollsOut(m) {
continue continue
} }
// **A recorded module goes where its move recreates nothing and moves no data** (novox/hq ADR 0377): judged
// per machine against the build it was last sent, said per machine, and sent only there.
var goes map[string]bool
if !rollsOut(m) {
out, err := judgeRecordedInWalk(ctx, inv, p, m, state)
if err != nil {
return false, err
}
if len(out) == 0 {
continue
}
goes = map[string]bool{}
for _, n := range out {
goes[n] = true
}
}
running, err := inv.Running(ctx, m) running, err := inv.Running(ctx, m)
if err != nil { if err != nil {
return false, err return false, err
} }
if goes != nil {
running = slices.DeleteFunc(running, func(n string) bool { return !goes[n] })
}
runningOf[m] = running runningOf[m] = running
// **A build whose source is unchanged is never a move** (novox/hq issue 280): made from the source // **A build whose source is unchanged is never a move** (novox/hq issue 280): made from the source
// of the build every machine running it was sent, it is registered with that build's artifacts — // of the build every machine running it was sent, it is registered with that build's artifacts —
@@ -910,12 +828,6 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
if err != nil { if err != nil {
return false, err return false, err
} }
// **Every machine at once** (novox/hq ADR 0360 §1, §2): a module that rolls out is sent to every machine
// running it in one step, with no first machine and no gate, whatever module it is. A module a controller
// before ADR 0360 already sent to its first machine finishes under its gate.
if !gateFirstMachine && state.FirstAt == nil {
policy.Together = true
}
var reports []inventory.Reported var reports []inventory.Reported
if !policy.Together { if !policy.Together {
// Read for the choice of the first machine as well as for its report. // Read for the choice of the first machine as well as for its report.
@@ -1145,93 +1057,15 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
return changed, nil return changed, nil
} }
} }
// **A walk ends on its machines' reports** (novox/hq ADR 0360 §4, §5): after its last tier, every machine it sent
// has reported on that send, or is silent past ApplySilentAfter; never a health reading.
end := ""
if p.Tier == len(p.Tiers)-1 {
reports, err := inv.LastReports(ctx)
if err != nil {
return false, err
}
waiting, failed, silent := walkReports(*p, reports, time.Now().UTC())
if len(waiting) > 0 {
note := "sent; waiting for " + strings.Join(waiting, ", ") + " to report on their send"
changed := p.State != inventory.PlanRolling || p.Note != note
p.State = inventory.PlanRolling
p.Note = note
return changed, nil
}
var said []string
if len(failed) > 0 {
said = append(said, "failed or refused its send (apply-failing says it): "+strings.Join(failed, ", "))
}
if len(silent) > 0 {
said = append(said, fmt.Sprintf("said nothing on its send within %s: %s", inventory.ApplySilentAfter,
strings.Join(silent, ", ")))
}
said = append(said, recordedSaid(*p)...)
end = strings.Join(said, "; ")
// **The new controller's user list follows its walk** (ADR 0236 §34, read without the gate since ADR 0360):
// once a walk that moved the controller ends, the bus's machine is sent the list it composes, if behind.
if s := p.Modules[catalogue.ControllerSeatName]; s != nil && s.SentAt != nil && s.Gate == nil && !noMove(s) {
carryUserList(ctx, open, p)
}
}
p.Tier++ p.Tier++
p.State = inventory.PlanBuilding p.State = inventory.PlanBuilding
p.Note = end p.Note = ""
if p.Tier < len(p.Tiers) { if p.Tier < len(p.Tiers) {
fmt.Printf("%s: tier %d done; asking tier %d: %s\n", p.ID, p.Tier-1, p.Tier, strings.Join(p.Tiers[p.Tier], ", ")) fmt.Printf("%s: tier %d done; asking tier %d: %s\n", p.ID, p.Tier-1, p.Tier, strings.Join(p.Tiers[p.Tier], ", "))
} }
return true, nil return true, nil
} }
// walkReports is, for every machine a walk sent its rest, whether it reported on that send (novox/hq ADR 0360 §4):
// those still waiting, those that reported a failed or refused apply, and those silent past ApplySilentAfter,
// each sorted. Pure.
func walkReports(p inventory.Plan, reports []inventory.Reported, now time.Time) (waiting, failed, silent []string) {
byNode := map[string]inventory.Reported{}
for _, r := range reports {
byNode[r.Node] = r
}
seen := map[string]bool{}
for _, s := range p.Modules {
if s == nil {
continue
}
for node, sent := range s.Rest {
if seen[node] {
continue
}
r := byNode[node]
at := sent.At
if at.IsZero() && s.SentAt != nil {
at = *s.SentAt
}
switch {
case r.At != nil && sent.ReportsOn(r):
seen[node] = true
if r.Outcome == inventory.OutcomeFailed || r.Outcome == inventory.OutcomeRefused {
failed = append(failed, node)
}
case !at.IsZero() && now.Sub(at) > inventory.ApplySilentAfter:
seen[node] = true
silent = append(silent, node)
default:
if !slices.Contains(waiting, node) {
waiting = append(waiting, node)
}
}
}
}
// A machine one module's send was reported on and another's not yet is the same send: reported.
waiting = slices.DeleteFunc(waiting, func(n string) bool { return seen[n] })
sort.Strings(waiting)
sort.Strings(failed)
sort.Strings(silent)
return waiting, failed, silent
}
// restOf is, for one module, every machine of its rest that the send reached and what it carried there. // restOf is, for one module, every machine of its rest that the send reached and what it carried there.
func restOf(to, sent []string, what map[string]inventory.SentDeclaration) map[string]inventory.SentDeclaration { func restOf(to, sent []string, what map[string]inventory.SentDeclaration) map[string]inventory.SentDeclaration {
out := map[string]inventory.SentDeclaration{} out := map[string]inventory.SentDeclaration{}
@@ -1975,12 +1809,6 @@ func plansCommand(ctx context.Context, args []string) error {
case s != nil && s.Gate != nil: case s != nil && s.Gate != nil:
fmt.Printf(" %-22s %s\n", "", gateLine(s.Gate)) fmt.Printf(" %-22s %s\n", "", gateLine(s.Gate))
} }
// Which way a recorded module went on each machine (novox/hq ADR 0377 §6).
if s != nil {
for _, n := range sortedKeysOf(s.Recorded) {
fmt.Printf(" %-22s on %s %s\n", "", n, s.Recorded[n])
}
}
// What the send to its first machine did to its containers (ADR 0245). // What the send to its first machine did to its containers (ADR 0245).
if s != nil && s.Gate != nil { if s != nil && s.Gate != nil {
for _, c := range s.Gate.Carried { for _, c := range s.Gate.Carried {
+12 -14
View File
@@ -1,7 +1,6 @@
package main package main
import ( import (
"slices"
"testing" "testing"
"time" "time"
@@ -50,10 +49,8 @@ func TestAMergeIsPlannedInTiersAlongTheThreeKindsOfDependency(t *testing.T) {
if pos["mesh-tools"] != 0 || pos["builder"] != 1 { if pos["mesh-tools"] != 0 || pos["builder"] != 1 {
t.Fatalf("the runtime then the builder: %v", tiers) t.Fatalf("the runtime then the builder: %v", tiers)
} }
// Built-by orders nothing (novox/hq ADR 0360 §3): what the builder builds is beside it, after the runtime it if !(pos["shop"] > pos["builder"] && pos["postgres"] > pos["builder"]) {
// stands on. t.Fatalf("what the builder builds comes after the builder: %v", tiers)
if pos["shop"] != 1 || pos["postgres"] != 1 {
t.Fatalf("what the builder builds waits for the builder: %v", tiers)
} }
if pos["shop-plugin"] <= pos["shop"] { if pos["shop-plugin"] <= pos["shop"] {
t.Fatalf("a plugin after what it is declared on: %v", tiers) t.Fatalf("a plugin after what it is declared on: %v", tiers)
@@ -70,23 +67,24 @@ func TestAMergeIsPlannedInTiersAlongTheThreeKindsOfDependency(t *testing.T) {
// A change to a package all three build from moves all three. The builder holds // A change to a package all three build from moves all three. The builder holds
// the build seat, whose worker the controller defines, so it follows the controller (worker-of, // the build seat, whose worker the controller defines, so it follows the controller (worker-of,
// novox/hq issue 206), and the controller's built-by edge to it yields: the controller is built by the // novox/hq issue 206), and the controller's built-by edge to it yields: the controller is built by the
// build machine that is running. The proxy is built by whichever builder runs, so it goes beside the // build machine that is running. The proxy is built by the new builder: the controller, the builder,
// controller (novox/hq ADR 0360 §3; it read "the controller, the builder, the proxy" before). // the proxy — the live plan of every controller merge. (This read "the builder, then the controller
// and the proxy together" before issue 206, and the fixture had no worker-of edge.)
small := reachableFrom([]string{"mesh-controller", "builder", "route-proxy"}, edges) small := reachableFrom([]string{"mesh-controller", "builder", "route-proxy"}, edges)
smallTiers := tiersOf(small, edges) smallTiers := tiersOf(small, edges)
if got := tiered(smallTiers); got != "mesh-controller,route-proxy | builder" { if got := tiered(smallTiers); got != "mesh-controller | builder | route-proxy" {
t.Fatalf("the controller and the proxy, then the builder: %v", smallTiers) t.Fatalf("the controller, then the builder, then the proxy: %v", smallTiers)
} }
// The builder alone moved: the builder, and nothing it builds. // The builder alone moved: the builder, and nothing it builds.
if only := reachableFrom([]string{"builder"}, edges); len(only) != 1 { if only := reachableFrom([]string{"builder"}, edges); len(only) != 1 {
t.Fatalf("a build machine change rebuilds the build machine alone: %v", only) t.Fatalf("a build machine change rebuilds the build machine alone: %v", only)
} }
// Nothing waits on deployment for a built-by edge (novox/hq ADR 0360 §3): the builder gates no tier after it. // Only a runtime dependency gates on deployment, and only when the module rolls out.
p := planOfMerge(link.SourceMoved{Owner: "novox", Repo: "mesh-tools", Commit: "abc"}, []string{"mesh-tools"}, edges) p := planOfMerge(link.SourceMoved{Owner: "novox", Repo: "mesh-tools", Commit: "abc"}, []string{"mesh-tools"}, edges)
p.Tier = pos["builder"] p.Tier = pos["builder"]
rollsOut := func(m string) bool { return m == "builder" } rollsOut := func(m string) bool { return m == "builder" }
if g := gates(p, edges, rollsOut); len(g) != 0 { if g := gates(p, edges, rollsOut); len(g) != 1 || g[0] != "builder" {
t.Fatalf("the builder gates the tier after it: %v", g) t.Fatalf("the builder gates the tier after it: %v", g)
} }
p.Tier = 0 p.Tier = 0
@@ -255,10 +253,10 @@ func TestAChangeToTheBuildAgentRebuildsTheBuildAgentAlone(t *testing.T) {
t.Fatalf("%v planned %v: the build agent alone", paths, p.Tiers) t.Fatalf("%v planned %v: the build agent alone", paths, p.Tiers)
} }
} }
// With what it builds moved beside it, both go in one tier (novox/hq ADR 0360 §3): built-by orders nothing. // With what it builds moved beside it, the agent comes first and they follow: an order, not a widening.
p := planOfMerge(link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Commit: "abc"}, p := planOfMerge(link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Commit: "abc"},
[]string{"build-agent", "redis"}, edges) []string{"build-agent", "redis"}, edges)
if len(p.Tiers) != 1 || !slices.Equal(p.Tiers[0], []string{"build-agent", "redis"}) || len(p.Modules) != 2 { if len(p.Tiers) != 2 || p.Tiers[0][0] != "build-agent" || p.Tiers[1][0] != "redis" || len(p.Modules) != 2 {
t.Fatalf("the agent and what moved beside it, in one tier: %v", p.Tiers) t.Fatalf("the agent, then what moved beside it: %v", p.Tiers)
} }
} }
+1 -8
View File
@@ -111,13 +111,8 @@ func aBacklog(t *testing.T) *backlogMesh {
if err := open.inventory.RecordSent(ctx, nodeID(t, open, node), digest, carried); err != nil { if err := open.inventory.RecordSent(ctx, nodeID(t, open, node), digest, carried); err != nil {
return nil, err return nil, err
} }
// A broken machine reports its apply failed (novox/hq ADR 0360 §7: the report is the verdict).
outcome := inventory.OutcomeApplied
if b.broken[node] {
outcome = inventory.OutcomeFailed
}
if _, err := open.inventory.RecordDoing(ctx, nodeID(t, open, node), inventory.Doing{Node: node, if _, err := open.inventory.RecordDoing(ctx, nodeID(t, open, node), inventory.Doing{Node: node,
Outcome: outcome, Declared: digest, Applied: 1, At: time.Now()}); err != nil { Outcome: inventory.OutcomeApplied, Declared: digest, Applied: 1, At: time.Now()}); err != nil {
return nil, err return nil, err
} }
} }
@@ -169,7 +164,6 @@ func (b *backlogMesh) release(t *testing.T) inventory.Plan {
// The backlog goes out one machine at a time: the first judged before the second is sent, each build's // The backlog goes out one machine at a time: the first judged before the second is sent, each build's
// pass kept; a rebuild with the same bytes is no move at all; and a send outside a gate is refused. // pass kept; a rebuild with the same bytes is no move at all; and a send outside a gate is refused.
func TestTheBacklogIsReleasedOneMachineAtATimeEachJudged(t *testing.T) { func TestTheBacklogIsReleasedOneMachineAtATimeEachJudged(t *testing.T) {
withTheGate(t) // the first-machine gate's own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
b := aBacklog(t) b := aBacklog(t)
ctx := t.Context() ctx := t.Context()
@@ -225,7 +219,6 @@ func TestTheBacklogIsReleasedOneMachineAtATimeEachJudged(t *testing.T) {
// A release that fails on its first machine puts back what it carried there, goes no further, and the // A release that fails on its first machine puts back what it carried there, goes no further, and the
// next one waits for a person, said as a condition; a person releases it with why. // next one waits for a person, said as a condition; a person releases it with why.
func TestAFailedReleaseIsPutBackAndTheNextWaitsForAPerson(t *testing.T) { func TestAFailedReleaseIsPutBackAndTheNextWaitsForAPerson(t *testing.T) {
withTheGate(t) // the first-machine gate's own path (novox/hq ADR 0360 keeps it for earlier walks)
b := aBacklog(t) b := aBacklog(t)
ctx := t.Context() ctx := t.Context()
inv := b.open.inventory inv := b.open.inventory
-1
View File
@@ -31,7 +31,6 @@ import (
// nothing is put back, the walk goes on to the workstation, and openrazer's condition says, in one sentence // nothing is put back, the walk goes on to the workstation, and openrazer's condition says, in one sentence
// for the operator, that a new login is needed on the laptop. // for the operator, that a new login is needed on the laptop.
func TestReplay318(t *testing.T) { func TestReplay318(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
inv := open.inventory inv := open.inventory
-2
View File
@@ -16,7 +16,6 @@ import (
// at its gate with the pass's own words, and put it back. Written with only what the controller had before // at its gate with the pass's own words, and put it back. Written with only what the controller had before
// its fix, so it is laid over the commit before. // its fix, so it is laid over the commit before.
func TestReplay335(t *testing.T) { func TestReplay335(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
t.Run("the first machine's later reports", testAPassedGateIsNotJudgedAgainFromTheFirstMachinesLaterReports) t.Run("the first machine's later reports", testAPassedGateIsNotJudgedAgainFromTheFirstMachinesLaterReports)
t.Run("a walk stopped after the pass", testAWalkStoppedAfterItsGatePassedKeepsThePass) t.Run("a walk stopped after the pass", testAWalkStoppedAfterItsGatePassedKeepsThePass)
} }
@@ -78,7 +77,6 @@ func testAWalkStoppedAfterItsGatePassedKeepsThePass(t *testing.T) {
// the first send is older than the wait for a first machine's report. Read before the pass is taken over, the // the first send is older than the wait for a first machine's report. Read before the pass is taken over, the
// carried module's step said the first machine never applied it, and the passed build was put back. // carried module's step said the first machine never applied it, and the passed build was put back.
func TestACarriedModuleTakesItsLeadsPassBeforeItsStepIsRead(t *testing.T) { func TestACarriedModuleTakesItsLeadsPassBeforeItsStepIsRead(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
tm := aTierMesh(t, "m01", "m02") tm := aTierMesh(t, "m01", "m02")
ctx := t.Context() ctx := t.Context()
-1
View File
@@ -19,7 +19,6 @@ import (
// had before its fix — the stores, register, assign, a plan saved and advanced, the watchdogs' gathering and // had before its fix — the stores, register, assign, a plan saved and advanced, the watchdogs' gathering and
// seeing — so it is laid over the commit before. // seeing — so it is laid over the commit before.
func TestReplay336(t *testing.T) { func TestReplay336(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
inv := open.inventory inv := open.inventory
-469
View File
@@ -1,469 +0,0 @@
package main
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"net"
"net/http"
"net/http/httptest"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/catalogue"
meshfacts "github.com/novox/mesh-controller/internal/facts"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// TestReplay867 is the replay of novox/hq issue 867, written with only what the controller and the build seat had
// before the fix, so mesh-lab's prover can lay it over the commit before. A mesh-host change (#114) and a
// mesh-controller change (#346) of one delivery group need each other: the node-engine change's own check reads the
// controller's lock beside it, which only the controller change brings; the controller change builds only with the
// validator word only the node-engine change brings. Each was read beside the other's repository at the commit the mesh
// runs, which holds neither change, so neither could pass its own merge check — and the pair could not merge.
//
// It runs the whole path: the forge announces each pull request (following.PullUpdated), the controller asks
// mesh-delivery which pull requests the group holds, the ask is queued for the build seat, and the build seat's check
// (builder.Check, with a container runtime that runs each container's command on this machine) judges it.
func TestReplay867(t *testing.T) { replay867(t, false) }
// TestReplay867Recheck is the same pair re-checked through mesh-delivery (`delivery check` with one member): the forge's
// announcement is not repeated, so the pull request reaches the controller without its head branch name.
func TestReplay867Recheck(t *testing.T) { replay867(t, true) }
func replay867(t *testing.T, recheck bool) {
if _, err := exec.LookPath("go"); err != nil {
t.Skip("no Go toolchain on this machine to build a judge with")
}
r := theGroupOfIssue867(t)
for _, c := range []struct {
what, repo string
number int
head string
module string
}{{"the node-engine change", "mesh-host", 114, r.hostHead, "mesh-host"},
{"the controller change", "mesh-controller", 346, r.controllerHead, "mesh-controller"}} {
v := r.checked(t, c.repo, c.number, c.head, recheck)
if v.Gate == nil || v.Gate.Verdict != "pass" || v.Repo == nil || v.Repo.Verdict != "pass" {
t.Errorf("%s (%s#%d) did not pass its own merge check beside its group: gate %+v, repository %+v\n%s",
c.what, c.repo, c.number, v.Gate, v.Repo, v.Report)
}
}
}
// theIssue867 is the two repositories as the trunk holds them and each pull request's head.
type theIssue867 struct {
open *stores
hostHead, controllerHead string
js interface{ Conn() *nats.Conn }
judgeCache string
registry string
hostRepo, controllerRepo string
groups string // what mesh-delivery's groups answers
}
func r867Git(t *testing.T, dir string, args ...string) string {
t.Helper()
cmd := exec.Command("git", args...)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@example.org",
"GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@example.org")
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("git %v: %v\n%s", args, err, out)
}
return strings.TrimSpace(string(out))
}
func r867Commit(t *testing.T, dir string, files map[string]string) string {
t.Helper()
for name, body := range files {
if err := os.MkdirAll(filepath.Dir(filepath.Join(dir, name)), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o755); err != nil {
t.Fatal(err)
}
}
r867Git(t, dir, "add", "-A")
r867Git(t, dir, "commit", "--quiet", "-m", "x")
return r867Git(t, dir, "rev-parse", "HEAD")
}
// r867Controller is a controller whose judge says who it is and the validator word it was built with.
func r867Controller(who, extra string) map[string]string {
return map[string]string{
"go.mod": "module example.org/judge\n\ngo 1.22\n\nrequire github.com/novox/mesh-host v0.0.0\n",
"vendor/modules.txt": "# github.com/novox/mesh-host v0.0.0\n## explicit; go 1.22\ngithub.com/novox/mesh-host/validate\n",
"vendor/github.com/novox/mesh-host/validate/v.go": "package validate\n\nconst Word = \"the vendored validator\"\n",
"vendor/github.com/novox/mesh-host/internal/declaration/d.go": "package declaration\n",
"cmd/mesh-controller/main.go": `package main
import (
"fmt"
"os"
"github.com/novox/mesh-host/validate"
)
func main() {
if len(os.Args) == 2 && os.Args[1] == "merge-gate" {
fmt.Println("usage: merge-gate --facts <file> --store <postgres>")
os.Exit(2)
}
fmt.Printf("{\"verdict\":\"pass\",\"summary\":\"%s with %s\"}\n", "` + who + `", validate.Word` + extra + `)
}
`,
}
}
func theGroupOfIssue867(t *testing.T) *theIssue867 { return theGroupOfIssue867Into(t, "main") }
// theGroupOfIssue867Into is the group with both pull requests into base: main, or a release branch both repositories
// hold, cut from their trunks.
func theGroupOfIssue867Into(t *testing.T, base string) *theIssue867 {
t.Helper()
open := aMesh(t)
r := &theIssue867{open: open, judgeCache: t.TempDir()}
root := filepath.Join(t.TempDir(), "novox")
controller, host := filepath.Join(root, "mesh-controller"), filepath.Join(root, "mesh-host")
r.hostRepo, r.controllerRepo = host, controller
for _, d := range []string{controller, host} {
if err := os.MkdirAll(d, 0o755); err != nil {
t.Fatal(err)
}
r867Git(t, d, "init", "--quiet", "-b", "main")
}
// The trunks.
files := r867Controller("the controller", "")
files["lock.txt"] = "foundation: old\n"
files["module.json"] = `{"module":"mesh-controller"}`
files[builder.CheckScript] = "true\n"
r867Commit(t, controller, files)
r867Commit(t, host, map[string]string{"validate/v.go": "package validate\n\nconst Word = \"the trunk's validator\"\n",
"internal/declaration/d.go": "package declaration\n", "module.json": `{"module":"mesh-host"}`,
builder.CheckScript: "set -e\ngrep -q 'foundation: new' \"$MESH_CHECK_BESIDE/mesh-controller/lock.txt\"\n"})
if base != "main" {
for _, d := range []string{controller, host} {
r867Git(t, d, "branch", base, "main")
}
}
// The two pull requests of one group, feat/867.
r867Git(t, controller, "checkout", "--quiet", "-b", "feat/867", base)
change := r867Controller("the controller", "+validate.More")
change["lock.txt"] = "foundation: new\n"
r.controllerHead = r867Commit(t, controller, change)
r867Git(t, controller, "checkout", "--quiet", "main")
r867Git(t, host, "checkout", "--quiet", "-b", "feat/867", base)
r.hostHead = r867Commit(t, host, map[string]string{"validate/more.go": "package validate\n\nconst More = \" and more\"\n"})
r867Git(t, host, "checkout", "--quiet", "main")
followed := func(string) string { return "main" }
for _, m := range []struct{ module, dir string }{{"mesh-controller", controller}, {"mesh-host", host}} {
// The mesh runs each at its trunk's head, the commit a check reads it at beside another repository's change.
running := r867Git(t, m.dir, "rev-parse", "main")
if err := open.inventory.RegisterModule(t.Context(), catalogue.Manifest{Module: m.module, Version: "1"},
inventory.Source{Repository: m.dir, Ref: followed(m.module), BuiltFrom: running, Head: running}); err != nil {
t.Fatal(err)
}
}
if base != "main" {
// A module built from the controller's repository follows the release branch, as the mesh runs it there: a pull
// request into the release branch moves it, so the composed check's gate runs.
running := r867Git(t, controller, "rev-parse", "main")
if err := open.inventory.RegisterModule(t.Context(), catalogue.Manifest{Module: "zz-agent", Version: "1"},
inventory.Source{Repository: controller, Ref: base, BuiltFrom: running, Head: running}); err != nil {
t.Fatal(err)
}
}
holdTheDeliverySeat(t, open)
// mesh-delivery says the group: two open pull requests into main, each with its head.
was := askDeliveryOwner
t.Cleanup(func() { askDeliveryOwner = was })
askDeliveryOwner = func(_ context.Context, _ *nats.Conn, verb string, _ map[string]any) (json.RawMessage, error) {
if verb != "groups" {
t.Fatalf("asked the owner %s", verb)
}
return json.RawMessage(r.groups), nil
}
r.groups = (`[{"id":"feat/867","members":["` + root + `/mesh-host@` + r.hostHead[:12] + ` (checked)","` +
root + `/mesh-controller@` + r.controllerHead[:12] + ` (checked)"],"member_details":[` +
`{"id":"novox/mesh-host@` + r.hostHead[:12] + `","repository":"novox/mesh-host","state":"checked","number":114,"head":"` +
r.hostHead + `","base":"` + base + `","open":true},` +
`{"id":"novox/mesh-controller@` + r.controllerHead[:12] + `","repository":"novox/mesh-controller","state":"checked","number":346,"head":"` +
r.controllerHead + `","base":"` + base + `","open":true}]}]`)
r.registry = r867Registry(t)
r867Runtime(t)
return r
}
// checked is a pull request announced by the forge, asked of the build seat by the controller, and checked there.
func (r *theIssue867) checked(t *testing.T, repo string, number int, head string, recheck bool) builderVerdict {
t.Helper()
js, ask := aBuildQueue(t)
_ = ask
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
p := link.PullUpdated{Owner: "novox", Repo: repo, Number: number, Base: "main", Head: "feat/867", Commit: head,
Paths: []string{"lock.txt", "validate/more.go", "cmd/mesh-controller/main.go"}, ModuleDirsSaid: true,
MergeCheck: true, MergeCheckSaid: true}
if recheck {
// As `delivery check --members <one head>` builds it from mesh-delivery's member: no head branch name.
p = orderMember{ID: "novox/" + repo + "@" + head[:12], Repository: "novox/" + repo, Base: "main", Head: head,
Number: number, Paths: p.Paths, ModuleDirsSaid: true}.pull()
p.MergeCheck, p.MergeCheckSaid = true, true
}
if err := (following{open: r.open, oneShot: true}).PullUpdated(t.Context(), p); err != nil {
t.Fatal(err)
}
q, err := link.ReadQueue(t.Context(), js, link.TheBuildMachine)
if err != nil {
t.Fatal(err)
}
if len(q.Asks) == 0 {
t.Fatalf("%s#%d: nothing was asked of the build seat", repo, number)
}
request := q.Asks[len(q.Asks)-1].Request
// What the build seat makes of the ask: the fields the two share, by name.
var spec builder.CheckSpec
body, _ := json.Marshal(request.Check)
if err := json.Unmarshal(body, &spec); err != nil {
t.Fatal(err)
}
spec.ID, spec.Repository, spec.Ref = request.ID, request.Repository, request.Ref
spec.Toolchain, spec.JudgeCache = "golang", r.judgeCache
v, err := builder.Check(t.Context(), builder.Command, spec, t.TempDir(), r.registry, builder.GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
return v
}
type builderVerdict = builder.CheckVerdict
func r867Registry(t *testing.T) string {
t.Helper()
body, err := json.Marshal(meshfacts.Facts{Format: meshfacts.Format, Taken: time.Now().UTC(),
Versions: meshfacts.Versions{Bus: "2.11.17", Store: "17.11"}, Machines: []meshfacts.Machine{{Name: "abcdef", Length: 6}}})
if err != nil {
t.Fatal(err)
}
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
manifest, _ := json.Marshal(map[string]any{"schemaVersion": 2, "layers": []map[string]any{
{"mediaType": meshfacts.MediaType, "digest": digest, "size": len(body)}}})
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
switch {
case strings.Contains(req.URL.Path, "/manifests/"):
w.Write(manifest)
case strings.HasSuffix(req.URL.Path, "/blobs/"+digest):
w.Write(body)
default:
http.NotFound(w, req)
}
}))
t.Cleanup(srv.Close)
return strings.TrimPrefix(srv.URL, "http://")
}
// r867Runtime puts a container runtime on PATH that runs every container's command on this machine, in the directory
// and with the environment the check gives it, and a store and a bus that answer.
func r867Runtime(t *testing.T) {
t.Helper()
bus, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { bus.Close() })
go func() {
for {
c, err := bus.Accept()
if err != nil {
return
}
c.Close()
}
}()
dir := t.TempDir()
script := `#!/bin/sh
cmd="$1"; shift
case "$cmd" in
ps|rm|exec) exit 0 ;;
port) echo "` + bus.Addr().String() + `"; exit 0 ;;
run)
workdir=""; detached=""
while [ $# -gt 0 ]; do
case "$1" in
-d) detached=1; shift ;;
--rm) shift ;;
--workdir) workdir="$2"; shift 2 ;;
--env) export "$2"; shift 2 ;;
--network|--volume|--user|--label|--name|-p|-e) shift 2 ;;
*) break ;;
esac
done
if [ -n "$detached" ]; then echo a-container; exit 0; fi
shift
cd "$workdir" && exec "$@" ;;
esac
exit 1
`
if err := os.WriteFile(filepath.Join(dir, "docker"), []byte(script), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", dir+string(os.PathListSeparator)+os.Getenv("PATH"))
t.Setenv("GOTOOLCHAIN", "local")
}
// A recheck runs as a command of its own (`delivery check` with one member), where the controller is not the serving
// one and holds no connection: it dials the bus for the moment and asks mesh-delivery's `groups` as the daemon does.
// This drives the verb on a test bus with a fake mesh-delivery answering `groups`, with no stub of the seam between.
func TestTheRecheckVerbAsksMeshDeliveryOnABusOfItsOwn(t *testing.T) {
realAsk := askDeliveryOwner
r := theGroupOfIssue867(t)
askDeliveryOwner = realAsk
js, _ := aBuildQueue(t)
if servingBus.Load() != nil {
t.Fatal("this test is of a process that is not the serving controller")
}
asked := 0
wrapped, _ := json.Marshal(map[string]any{"content": []map[string]any{{"type": "text", "text": r.groups}}})
sub, err := js.Conn().Subscribe(link.SeatToolSubject(catalogue.DeliverySeat, "groups"), func(m *nats.Msg) {
asked++
body, _ := json.Marshal(link.Answer{Result: wrapped})
_ = m.Respond(body)
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = sub.Unsubscribe() })
member := `[{"id":"novox/mesh-host@` + r.hostHead[:12] + `","repository":"novox/mesh-host","base":"main","head":"` +
r.hostHead + `","number":114,"paths":["validate/more.go"],"module_dirs_said":true}]`
if err := deliveryCommand(t.Context(), []string{"check", "--members", member}); err != nil {
t.Fatal(err)
}
q, err := link.ReadQueue(t.Context(), js, link.TheBuildMachine)
if err != nil || len(q.Asks) == 0 {
t.Fatalf("nothing was asked of the build seat: %v", err)
}
c := q.Asks[len(q.Asks)-1].Request.Check
if c.SiblingsUnknown != "" || len(c.Siblings) != 1 || c.Siblings[0].Repository != "novox/mesh-controller" ||
c.Siblings[0].Number != 346 || c.Siblings[0].Head != r.controllerHead {
t.Fatalf("the recheck learned siblings %+v, unknown %q", c.Siblings, c.SiblingsUnknown)
}
if asked != 1 {
t.Fatalf("mesh-delivery was asked %d times", asked)
}
}
// **A delivery group's composed check is judged by the group's own controller** (novox/hq ADR 0239 §14), into a release
// branch too: the controller member and the mesh-host member of the pair are consistent only together (the controller
// calls what only the node-engine change brings), and both merge into a release branch. The release judge (the
// controller of the release branch with the validator of its own) cannot build it; the group's own controller, as
// merged, with the node-engine member's validator, can. It drives askGroupCheck and the build seat's Check for real.
func TestAGroupsCompositionIntoAReleaseBranchIsJudgedByItsOwnController(t *testing.T) {
groupIntoARelease(t, false)
}
// The shape the live check had: the node-engine member is the group's primary, so the composed request is asked with
// Judge "validator" (a union of modules that includes mesh-host's) and the release branch set; the release judge then
// put this change's validator into the old controller and "judge-with-this-validator" did not build.
func TestAGroupsCompositionJudgedWithTheValidatorIntoAReleaseBranchIsJudgedByItsOwnController(t *testing.T) {
groupIntoARelease(t, true)
}
func groupIntoARelease(t *testing.T, hostPrimary bool) {
if _, err := exec.LookPath("go"); err != nil {
t.Skip("no Go toolchain on this machine to build a judge with")
}
const release = "feat/r"
r := theGroupOfIssue867Into(t, release)
aForgeWithBranches(t, map[string][]string{"novox/mesh-controller": {release}}, nil)
// The pair renames what the validator says: the node-engine change drops Word for Word2, the controller change
// calls Word2. The controller of the release branch calls Word, so it does not build with the new validator; the
// controller change does not build with the validator it vendors. Only together are they consistent.
r867Git(t, r.hostRepo, "checkout", "--quiet", "-b", "swap", release)
r.hostHead = r867Commit(t, r.hostRepo, map[string]string{"validate/v.go": "package validate\n\nconst Word2 = \"the new validator\"\n"})
r867Git(t, r.hostRepo, "checkout", "--quiet", "main")
r867Git(t, r.controllerRepo, "checkout", "--quiet", "-b", "swap", release)
swapped := strings.Replace(r867Controller("the controller", "")["cmd/mesh-controller/main.go"], "validate.Word)", "validate.Word2)", 1)
r.controllerHead = r867Commit(t, r.controllerRepo, map[string]string{"cmd/mesh-controller/main.go": swapped})
r867Git(t, r.controllerRepo, "checkout", "--quiet", "main")
js, _ := aBuildQueue(t)
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
members := []orderMember{
{ID: "novox/mesh-controller@" + r.controllerHead[:12], Repository: "novox/mesh-controller", Base: release,
Head: r.controllerHead, Number: 346, Paths: []string{"lock.txt", "cmd/mesh-controller/main.go"}, ModuleDirsSaid: true},
{ID: "novox/mesh-host@" + r.hostHead[:12], Repository: "novox/mesh-host", Base: release, Head: r.hostHead, Number: 114,
Paths: []string{"validate/v.go"}, ModuleDirsSaid: true}}
if hostPrimary {
members[0], members[1] = members[1], members[0]
}
if _, err := askGroupCheck(t.Context(), r.open, "feat/867", members); err != nil {
t.Fatal(err)
}
q, err := link.ReadQueue(t.Context(), js, link.TheBuildMachine)
if err != nil || len(q.Asks) == 0 {
t.Fatalf("nothing was asked of the build seat: %v", err)
}
request := q.Asks[len(q.Asks)-1].Request
if request.Check.Release != release || request.Check.Group != "feat/867" || len(request.Check.Members) != 1 || len(request.Check.Modules) == 0 {
t.Fatalf("the composed check is asked as release %q, group %q, %d other head(s)", request.Check.Release,
request.Check.Group, len(request.Check.Members))
}
if hostPrimary {
if request.Check.Repo != "mesh-host" {
t.Fatalf("the primary is %s, not the node-engine", request.Check.Repo)
}
// As the live ask had it (the mesh-host module among the union's): the node-engine's validator judges.
request.Check.Judge = link.JudgeValidator
}
// What the build seat makes of the ask.
var wire map[string]json.RawMessage
body, _ := json.Marshal(request.Check)
_ = json.Unmarshal(body, &wire)
delete(wire, "group")
delete(wire, "members")
body, _ = json.Marshal(wire)
var spec builder.CheckSpec
if err := json.Unmarshal(body, &spec); err != nil {
t.Fatal(err)
}
for _, m := range request.Check.Members {
spec.Group = append(spec.Group, builder.GroupHead{Owner: m.Owner, Repo: m.Repo, Repository: m.Repository, Ref: m.Ref,
Paths: m.Paths, Base: m.Base})
}
spec.ID, spec.Repository, spec.Ref = request.ID, request.Repository, request.Ref
spec.Toolchain, spec.JudgeCache = "golang", r.judgeCache
v, err := builder.Check(t.Context(), builder.Command, spec, t.TempDir(), r.registry, builder.GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if v.Gate == nil || v.Gate.Verdict != "pass" {
t.Fatalf("the group's composition into %s was not judged by its own controller: %+v\n%s", release, v.Gate, v.Report)
}
if mark := "[release " + release + ", judged by the group's controller " + v.JudgedBy + "] "; !strings.HasPrefix(v.Gate.Summary, mark) ||
!strings.Contains(v.Report, "by the group's controller at "+v.JudgedBy) {
t.Fatalf("the statuses and the report do not say the group's controller judged:\n%s\n%s", v.Gate.Summary, v.Report)
}
if !strings.Contains(v.Gate.Summary, "the new validator") {
t.Fatalf("the gate was judged by %q, not by the group's controller with the node-engine member's validator", v.Gate.Summary)
}
if !strings.HasPrefix(v.JudgeFrom, "the group's own controller") || !strings.Contains(v.Validator, "the group's mesh-host merge") {
t.Fatalf("the verdict does not say where the judge and the validator came from: %q / %q", v.JudgeFrom, v.Validator)
}
if v.Release != release || v.JudgedBy == "" {
t.Fatalf("the verdict names release %q judged by %q", v.Release, v.JudgedBy)
}
}
-1
View File
@@ -139,7 +139,6 @@ func TestReplay273AConsumerBesideItsStoreStaysBoundToIt(t *testing.T) {
// one, kept below. Heard as bytes, so an older controller reads them as it reads any report — this file is // one, kept below. Heard as bytes, so an older controller reads them as it reads any report — this file is
// written only with what the controller had before the judging, for the prover to lay over that commit. // written only with what the controller had before the judging, for the prover to lay over that commit.
func TestReplayCrashLoopFailsItsGateOnTheFirstMachine(t *testing.T) { func TestReplayCrashLoopFailsItsGateOnTheFirstMachine(t *testing.T) {
withTheGate(t) // the first-machine gate's own path (novox/hq ADR 0360 keeps it for earlier walks)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
inv := open.inventory inv := open.inventory
+1 -11
View File
@@ -11,7 +11,7 @@ import (
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
) )
// novox/hq ADR 0353, kept behind gateFirstMachine for walks a controller before ADR 0360 started: after its first machine passes, a build goes to every other machine at once, each judged by // novox/hq ADR 0353: after its first machine passes, a build goes to every other machine at once, each judged by
// its own gate and a failing one put back by itself; and a gate's readings outlive the controller that took them — // its own gate and a failing one put back by itself; and a gate's readings outlive the controller that took them —
// a machine already sent what a send is for is not sent again, and its judging goes on from what was read. // a machine already sent what a send is for is not sent again, and its judging goes on from what was read.
// //
@@ -62,7 +62,6 @@ func events(t *testing.T) *[]string {
// Once the first machine passed, the rest are both sent before either is judged (ADR 0353 §1); before, the second // Once the first machine passed, the rest are both sent before either is judged (ADR 0353 §1); before, the second
// was judged to its verdict before the third was sent. // was judged to its verdict before the third was sent.
func TestAReleaseSendsTheRestAtOnceAfterTheFirstPasses(t *testing.T) { func TestAReleaseSendsTheRestAtOnceAfterTheFirstPasses(t *testing.T) {
withTheGate(t)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
b := aBacklogOnThree(t) b := aBacklogOnThree(t)
ctx := t.Context() ctx := t.Context()
@@ -90,7 +89,6 @@ func TestAReleaseSendsTheRestAtOnceAfterTheFirstPasses(t *testing.T) {
// A machine of the rest whose gate fails is put back at once, and the other machine of the step goes on to its own // A machine of the rest whose gate fails is put back at once, and the other machine of the step goes on to its own
// verdict (ADR 0353 §3); before, the walk stopped at the failure and the machine after it was never sent. // verdict (ADR 0353 §3); before, the walk stopped at the failure and the machine after it was never sent.
func TestARestMachineThatFailsIsPutBackAloneAndTheOthersAreJudgedOn(t *testing.T) { func TestARestMachineThatFailsIsPutBackAloneAndTheOthersAreJudgedOn(t *testing.T) {
withTheGate(t)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
b := aBacklogOnThree(t) b := aBacklogOnThree(t)
ctx := t.Context() ctx := t.Context()
@@ -127,7 +125,6 @@ func TestARestMachineThatFailsIsPutBackAloneAndTheOthersAreJudgedOn(t *testing.T
// A walk whose first machine already runs its build under another walk's gate sends nothing and goes on from that // A walk whose first machine already runs its build under another walk's gate sends nothing and goes on from that
// gate's readings (ADR 0353 §7); before, it sent the machine again and judged from its first reading. // gate's readings (ADR 0353 §7); before, it sent the machine again and judged from its first reading.
func TestAWalkTakesOverTheReadingsOfAGateJudgingTheSameBuildThere(t *testing.T) { func TestAWalkTakesOverTheReadingsOfAGateJudgingTheSameBuildThere(t *testing.T) {
withTheGate(t)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
b := aBacklog(t) b := aBacklog(t)
ctx := t.Context() ctx := t.Context()
@@ -166,7 +163,6 @@ func TestAWalkTakesOverTheReadingsOfAGateJudgingTheSameBuildThere(t *testing.T)
// A walk's first send that was made and not kept — the controller replaced between the two — is not made again // A walk's first send that was made and not kept — the controller replaced between the two — is not made again
// by the controller that takes over: the gate counts from that send (ADR 0353 §5, §6). // by the controller that takes over: the gate counts from that send (ADR 0353 §5, §6).
func TestAFirstSendMadeAndNotKeptIsNotSentAgain(t *testing.T) { func TestAFirstSendMadeAndNotKeptIsNotSentAgain(t *testing.T) {
withTheGate(t)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
g := aGateMesh(t) g := aGateMesh(t)
ctx := t.Context() ctx := t.Context()
@@ -204,7 +200,6 @@ func TestAFirstSendMadeAndNotKeptIsNotSentAgain(t *testing.T) {
// A release's send kept before it was made, and made, is judged by the controller that takes over without being // A release's send kept before it was made, and made, is judged by the controller that takes over without being
// made again (ADR 0353 §8). // made again (ADR 0353 §8).
func TestAReleaseSendKeptAndMadeIsNotMadeAgain(t *testing.T) { func TestAReleaseSendKeptAndMadeIsNotMadeAgain(t *testing.T) {
withTheGate(t)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
b := aBacklog(t) b := aBacklog(t)
ctx := t.Context() ctx := t.Context()
@@ -236,7 +231,6 @@ func TestAReleaseSendKeptAndMadeIsNotMadeAgain(t *testing.T) {
// A build no gate has passed goes to one machine of a step, its first; a machine that would be its second waits // A build no gate has passed goes to one machine of a step, its first; a machine that would be its second waits
// for the next step, and one whose every build passed goes now (ADR 0353 §2). // for the next step, and one whose every build passed goes now (ADR 0353 §2).
func TestABuildThatPassedNoGateGoesToOneMachineOfAStep(t *testing.T) { func TestABuildThatPassedNoGateGoesToOneMachineOfAStep(t *testing.T) {
withTheGate(t)
waiting := map[string][]inventory.CarriedMove{ waiting := map[string][]inventory.CarriedMove{
"desk": {{Module: "app", To: "c2"}, {Module: "new", To: "n1"}}, "desk": {{Module: "app", To: "c2"}, {Module: "new", To: "n1"}},
"laptop": {{Module: "app", To: "c2"}, {Module: "new", To: "n1"}}, "laptop": {{Module: "app", To: "c2"}, {Module: "new", To: "n1"}},
@@ -257,7 +251,6 @@ func TestABuildThatPassedNoGateGoesToOneMachineOfAStep(t *testing.T) {
// The node-engine and the mesh's resolver go to the rest one machine at a time, even passed (ADR 0353 §9): a step // The node-engine and the mesh's resolver go to the rest one machine at a time, even passed (ADR 0353 §9): a step
// carries either to one machine only, while what else passed goes everywhere at once. // carries either to one machine only, while what else passed goes everywhere at once.
func TestTheNodeEngineAndTheResolverGoToTheRestOneAtATime(t *testing.T) { func TestTheNodeEngineAndTheResolverGoToTheRestOneAtATime(t *testing.T) {
withTheGate(t)
waiting := map[string][]inventory.CarriedMove{ waiting := map[string][]inventory.CarriedMove{
"desk": {{Module: hostModule, To: "h2"}}, "desk": {{Module: hostModule, To: "h2"}},
"laptop": {{Module: hostModule, To: "h2"}, {Module: "app", To: "c2"}}, "laptop": {{Module: hostModule, To: "h2"}, {Module: "app", To: "c2"}},
@@ -284,7 +277,6 @@ func TestTheNodeEngineAndTheResolverGoToTheRestOneAtATime(t *testing.T) {
// A first machine that fails still stops the walk (ADR 0236 §5, kept by ADR 0353 §3): the rest are never sent. // A first machine that fails still stops the walk (ADR 0236 §5, kept by ADR 0353 §3): the rest are never sent.
func TestAFirstMachineThatFailsStillStopsTheWalk(t *testing.T) { func TestAFirstMachineThatFailsStillStopsTheWalk(t *testing.T) {
withTheGate(t)
b := aBacklogOnThree(t) b := aBacklogOnThree(t)
ctx := t.Context() ctx := t.Context()
gateEvery = 0 gateEvery = 0
@@ -306,7 +298,6 @@ func TestAFirstMachineThatFailsStillStopsTheWalk(t *testing.T) {
// The control node is sent last in a step of the rest (ADR 0353 §1): the machine holding the controller is // The control node is sent last in a step of the rest (ADR 0353 §1): the machine holding the controller is
// ordered after every other. // ordered after every other.
func TestTheControlNodeIsSentLastInAStep(t *testing.T) { func TestTheControlNodeIsSentLastInAStep(t *testing.T) {
withTheGate(t)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
b := aBacklogOnThree(t) b := aBacklogOnThree(t)
ctx := t.Context() ctx := t.Context()
@@ -331,7 +322,6 @@ func TestTheControlNodeIsSentLastInAStep(t *testing.T) {
// A machine of the rest that says nothing on its send is left silent after fifteen minutes: not failed, not put // A machine of the rest that says nothing on its send is left silent after fifteen minutes: not failed, not put
// back; a rest that went silent whole ends failed, said as a condition (ADR 0353 §4). // back; a rest that went silent whole ends failed, said as a condition (ADR 0353 §4).
func TestARestMachineSilentPastItsBoundIsLeftAndSaid(t *testing.T) { func TestARestMachineSilentPastItsBoundIsLeftAndSaid(t *testing.T) {
withTheGate(t)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
b := aBacklog(t) b := aBacklog(t)
ctx := t.Context() ctx := t.Context()
-176
View File
@@ -1,176 +0,0 @@
package main
import (
"context"
"fmt"
"slices"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A module moves only when its program changes** (novox/hq issue 721). A merge check builds every module the
// delivery plan moves and compares each with the build the mesh holds of it; a walk is cut without a module whose
// check found its whole image the same as that build, while the mesh still holds it, and says so among the modules
// it left unchanged. A merge of one sentence in a package a program imports but never reaches moves no build of it.
// sameAsRunning are the modules of a walk its checks found the same as the build the mesh holds, each with what its
// check said. **Only when every pull request the walk answers was checked at the head it merged, and every one of
// those checks built the module and found it the same as the build the mesh still holds**: a merge the walk answers
// without a pull request, a check of another head, a check that built it differently, or a held build that moved
// since, leaves the module in the walk.
func sameAsRunning(plan *inventory.Plan, builds []link.CheckedBuild, held map[string]string) map[string]string {
if plan.Delivery == nil || len(plan.Delivery.Merges) == 0 {
return nil
}
for _, m := range plan.Delivery.Merges {
if _, ok := pullOf(m); !ok {
return nil
}
}
pulls := reuseAskOf(plan).Pulls
heads := mergedHeadsOf(plan)
out := map[string]string{}
for name := range plan.Modules {
said := ""
for _, pull := range pulls {
b, ok := checkedBuildOf(builds, pull, heads[pull], name)
if !ok || !stillHeld(b.Built, name, held) {
said = ""
break
}
if said == "" {
said = b.Built.Same
}
}
if said != "" {
out[name] = said
}
}
if len(out) == 0 {
return nil
}
return out
}
// checkedBuildOf is the newest build a check of the pull request at head made of the module; false when none did.
func checkedBuildOf(builds []link.CheckedBuild, pull link.PullRef, head, module string) (link.CheckedBuild, bool) {
if head == "" {
return link.CheckedBuild{}, false
}
for _, b := range builds {
if b.Pull == pull && b.Built.Module == module && link.SameHead(b.Head, head) {
return b, true
}
}
return link.CheckedBuild{}, false
}
// stillHeld is whether a check found every artifact of the module the same as a build the mesh still holds.
func stillHeld(b link.CheckBuilt, module string, held map[string]string) bool {
if b.Same == "" || b.NotBuilt != "" || len(b.Made) == 0 || len(b.SameAs) != len(b.Made) {
return false
}
for _, m := range b.Made {
was := pinnedDigest(b.SameAs[m.Name])
if was == "" || was != pinnedDigest(held[module+"/"+m.Name]) {
return false
}
}
return true
}
// pinnedDigest is a reference's pinned digest, empty when it is not pinned.
func pinnedDigest(ref string) string {
if i := strings.LastIndex(ref, "@"); i >= 0 {
return ref[i+1:]
}
return ""
}
// leaveSameAsRunning takes from a walk being cut every module its checks found the same as the build the mesh holds
// (sameAsRunning), and every dependent in the walk only for one of those (withDependentsOf): what was taken, with why. **Anything not read leaves the walk as it was**, said.
func leaveSameAsRunning(ctx context.Context, inv *inventory.Inventory, plan *inventory.Plan, moved []string,
edges []inventory.Edge) map[string]string {
if plan.Delivery == nil || len(plan.Modules) == 0 {
return nil
}
pulls := reuseAskOf(plan).Pulls
if len(pulls) == 0 {
return nil
}
builds, err := checkedBuildsOf(ctx, pulls)
if err != nil {
fmt.Printf(" the checks' builds could not be read, so every module moves: %v\n", err)
// empty-on-error: said above; nothing left out of the walk is the safe answer, every module moves as before
return nil
}
held, err := inv.RegisteredHeld(ctx)
if err != nil {
fmt.Printf(" what the mesh holds could not be read, so every module moves: %v\n", err)
// empty-on-error: said above; nothing left out of the walk is the safe answer, every module moves as before
return nil
}
same := withDependentsOf(sameAsRunning(plan, builds, held), plan, moved, edges)
if len(same) == 0 {
return nil
}
names := make([]string, 0, len(same))
for n := range same {
names = append(names, n)
}
sort.Strings(names)
withhold(plan, names, nil)
for _, n := range names {
fmt.Printf(" %s left out: %s\n", n, same[n])
}
return same
}
// withDependentsOf adds to the modules found the same every module of the walk that is in it only as a dependent of
// one of them (novox/hq ADR 0267 rule 3): **a dependent is left out when its base comes out the same**, since the
// base it stands on stays the build the mesh holds. A module a merge moves itself, or a dependent of a base that
// still moves, stays. moved are the modules the walk's merges move; edges the graph the walk was planned along.
func withDependentsOf(same map[string]string, plan *inventory.Plan, moved []string, edges []inventory.Edge) map[string]string {
if len(same) == 0 {
return same
}
var still []string
for _, n := range moved {
if same[n] == "" {
still = append(still, n)
}
}
keep := map[string]bool{}
for _, n := range reachableFrom(still, edges) {
keep[n] = true
}
// Until nothing more is found: a dependent of a dependent left out is left out too.
for grew := true; grew; {
grew = false
for n := range plan.Modules {
if keep[n] || same[n] != "" {
continue
}
var bases []string
for _, e := range edges {
ordering := e.Kind == inventory.EdgeBuiltBy || e.Kind == inventory.EdgeWorkerOf ||
e.Kind == inventory.EdgePackages || e.Kind == edgeGroupOrder
if !ordering && e.From == n && same[e.To] != "" && !slices.Contains(bases, e.To) {
bases = append(bases, e.To)
}
}
sort.Strings(bases)
if len(bases) == 0 {
// In the walk for no base found the same: whatever brought it in is not said here, so it moves.
continue
}
same[n] = "unchanged: it is in the walk only for " + strings.Join(bases, ", ") +
", which came out the same as the build the mesh holds (ADR 0267 rule 3)"
grew = true
}
}
return same
}
-141
View File
@@ -1,141 +0,0 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// TestAWalkLeavesOutAModuleWhoseProgramDidNotChange is novox/hq issue 721: mesh-controller #335 changed one sentence
// that mesh-builder never reaches; its check found build-agent the same as the build the mesh holds, so the walk moves
// mesh-controller alone and says build-agent is unchanged.
func TestAWalkLeavesOutAModuleWhoseProgramDidNotChange(t *testing.T) {
pull := link.PullRef{Owner: "novox", Repo: "mesh-controller", Number: 335}
head := "34050dc6aaaabbbbccccdddd"
aPlan := func() *inventory.Plan {
return &inventory.Plan{Tiers: [][]string{{"mesh-controller", "build-agent"}},
Modules: map[string]*inventory.PlanModule{"mesh-controller": {}, "build-agent": {}},
Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{{Repository: "novox/mesh-controller",
Commit: "c2ec0a60", Number: 335, Head: head}}}}
}
held := map[string]string{"build-agent/agent": "store/build-agent@sha256:aaaa",
"mesh-controller/controller": "store/mesh-controller@sha256:cccc"}
builds := []link.CheckedBuild{
{Pull: pull, Head: head[:12], Built: link.CheckBuilt{Module: "build-agent", Key: "k1",
Made: []link.MadeArtifact{{Name: "agent", Kind: "image", Reference: "scratch/build-agent@sha256:bbbb"}},
Same: "unchanged: its program is the same as running build agent aaaa",
SameAs: map[string]string{"agent": "10.0.0.1:5000/build-agent@sha256:aaaa"}}},
{Pull: pull, Head: head[:12], Built: link.CheckBuilt{Module: "mesh-controller", Key: "k2",
Made: []link.MadeArtifact{{Name: "controller", Kind: "image", Reference: "scratch/mesh-controller@sha256:dddd"}}}},
}
p := aPlan()
same := sameAsRunning(p, builds, held)
if len(same) != 1 || same["build-agent"] == "" {
t.Fatalf("the walk leaves out %v, want build-agent alone", same)
}
withhold(p, []string{"build-agent"}, nil)
if len(p.Tiers) != 1 || len(p.Tiers[0]) != 1 || p.Tiers[0][0] != "mesh-controller" {
t.Fatalf("the walk moves %v, want mesh-controller alone", p.Tiers)
}
// **The held build moved since the check**: what it was compared with no longer runs.
moved := map[string]string{"build-agent/agent": "store/build-agent@sha256:eeee"}
if same := sameAsRunning(aPlan(), builds, moved); len(same) != 0 {
t.Fatalf("a module compared with a build the mesh no longer holds is left out: %v", same)
}
// **A check of another head** says nothing of this merge.
other := aPlan()
other.Delivery.Merges[0].Head = "ffffffffffffffff"
if same := sameAsRunning(other, builds, held); len(same) != 0 {
t.Fatalf("a check of another head left out %v", same)
}
// **Another pull request the walk answers** that did not find it the same leaves it in.
two := aPlan()
two.Delivery.Merges = append(two.Delivery.Merges, inventory.PlanMerge{Repository: "novox/mesh-catalog",
Commit: "1234", Number: 9, Head: "abcdefabcdefabcdef"})
if same := sameAsRunning(two, builds, held); len(same) != 0 {
t.Fatalf("a module another merge moves is left out: %v", same)
}
// **A merge with no pull request** leaves the walk as it is.
direct := aPlan()
direct.Delivery.Merges[0].Number = 0
if same := sameAsRunning(direct, builds, held); len(same) != 0 {
t.Fatalf("a walk answering a merge without a pull request left out %v", same)
}
}
// TestADependentOfABaseThatCameOutTheSameIsLeftOut is ADR 0267 rule 3 (novox/hq issue 607): a dependent is in a walk
// for its base; when the base comes out the same as the build the mesh holds, the dependent stands on that build
// still, and leaves the walk with it — and so does a dependent of that dependent. One a merge moves itself stays.
func TestADependentOfABaseThatCameOutTheSameIsLeftOut(t *testing.T) {
edges := []inventory.Edge{{From: "app", To: "base", Kind: inventory.EdgeDeclared},
{From: "app2", To: "app", Kind: inventory.EdgeDeclared}, {From: "own", To: "base", Kind: inventory.EdgeDeclared}}
plan := &inventory.Plan{Modules: map[string]*inventory.PlanModule{"base": {}, "app": {}, "app2": {}, "own": {}}}
same := withDependentsOf(map[string]string{"base": "unchanged: its program is the same"}, plan,
[]string{"base", "own"}, edges)
if !strings.Contains(same["app"], "only for base") || !strings.Contains(same["app2"], "only for app") {
t.Fatalf("the dependents of a base that came out the same are not left out: %v", same)
}
if _, left := same["own"]; left {
t.Fatalf("a module its merge moves itself is left out for its base: %v", same)
}
// A dependent of a base that still moves stays.
edges = append(edges, inventory.Edge{From: "app", To: "own", Kind: inventory.EdgeDeclared})
same = withDependentsOf(map[string]string{"base": "unchanged"}, plan, []string{"base", "own"}, edges)
if _, left := same["app"]; left {
t.Fatalf("a dependent of a base that still moves is left out: %v", same)
}
}
// Against the store, the path a real merge takes (novox/hq issues 721 and 611): a merge whose check built its one
// module the same as the build the mesh holds is cut into a walk done with nothing to walk, whose record names the
// module unchanged and why — the walk mesh-delivery reads as delivered, never stranded.
func TestAWalkWhoseEveryBuildCameOutTheSameIsDone(t *testing.T) {
open := windowed(t)
endsItsLine(t)
asksRecorded(t)
ctx := t.Context()
inv := open.inventory
if err := inv.RecordBuild(ctx, inventory.Build{ID: "build-one", Repository: "novox/one", Ref: "main",
Module: "one", Commit: "c0", On: "builder", Asked: t0.Add(-time.Hour),
Made: []inventory.Artifact{{Name: "one", Kind: "image", Reference: "store/one@sha256:aaaa"}},
Sources: []inventory.BuildSource{{Paths: []string{"main.go", "go.mod"}}}}); err != nil {
t.Fatal(err)
}
// What runs: the module registered with that build's image (RegisteredHeld reads the module table).
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "one", Version: "1", Resources: []map[string]any{
{"id": "app", "type": "container", "name": "one", "image": "store/one@sha256:aaaa"}}},
inventory.Source{Repository: "novox/one", Ref: "main", BuiltFrom: "c0", Asked: time.Now()}); err != nil {
t.Fatal(err)
}
pull := link.PullRef{Owner: "novox", Repo: "one", Number: 7}
was := checkedBuildsOf
checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) {
return []link.CheckedBuild{{Pull: pull, Head: "headaaaaaaaa", Built: link.CheckBuilt{Module: "one", Key: "k",
Made: []link.MadeArtifact{{Name: "one", Kind: "image", Reference: "scratch/one@sha256:bbbb"}},
Same: "unchanged: its program is the same as running build one aaaa",
SameAs: map[string]string{"one": "10.0.0.1:5000/one@sha256:aaaa"}}}}, nil
}
t.Cleanup(func() { checkedBuildsOf = was })
code := repoMerge("one", "one1aaaaaaaa", t0)
code.Number, code.HeadSHA = 7, "headaaaaaaaa1111"
hear(t, open, code, t0.Add(time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
if len(ws) != 1 {
t.Fatalf("walks: %+v", ws)
}
w := ws[0]
if w.State != inventory.PlanDone || len(w.Tiers) != 0 || len(w.Modules) != 0 {
t.Fatalf("the walk reads %s, tiers %v, modules %v; want done with nothing to walk", w.State, w.Tiers, w.Modules)
}
if w.Delivery == nil || !strings.Contains(w.Delivery.Unchanged["one"], "its program is the same as running build") {
t.Fatalf("the walk does not say one matched what runs: %+v", w.Delivery)
}
}
-3
View File
@@ -43,9 +43,6 @@ func TestMain(m *testing.M) {
} }
if os.Getenv(echoEnvironment) != "" { if os.Getenv(echoEnvironment) != "" {
fmt.Printf("verb=%q caller=%q terminal=%v\n", os.Getenv("MESH_VERB"), os.Getenv("MESH_CALLER"), startedAtTheTerminal()) fmt.Printf("verb=%q caller=%q terminal=%v\n", os.Getenv("MESH_VERB"), os.Getenv("MESH_CALLER"), startedAtTheTerminal())
if os.Getenv(echoFails) != "" {
os.Exit(3)
}
os.Exit(0) os.Exit(0)
} }
conditions.Unsayable = func(o conditions.Observation, field string, r outward.Refusal) { conditions.Unsayable = func(o conditions.Observation, field string, r outward.Refusal) {
+1 -26
View File
@@ -12,11 +12,8 @@ import (
"strings" "strings"
"time" "time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/catalogue" "github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/secrets" "github.com/novox/mesh-controller/internal/secrets"
) )
@@ -154,11 +151,8 @@ func secretCommand(ctx context.Context, args []string) error {
if err != nil { if err != nil {
return err return err
} }
askedBy := link.Caller()
untilStart, unannounced, err := keepGiven(trusted, untilStart, unannounced, err := keepGiven(trusted,
func() error { func() error { return announceSecretGiven(ctx, node, module, name, "at the controller's terminal") },
return announceSecretGiven(ctx, node, module, name, "at the controller's terminal", askedBy)
},
func() (bool, error) { return open.inventory.AcceptGivenSecret(ctx, node, module, name, value) }) func() (bool, error) { return open.inventory.AcceptGivenSecret(ctx, node, module, name, value) })
if err != nil { if err != nil {
if trusted && unannounced != nil { if trusted && unannounced != nil {
@@ -170,15 +164,6 @@ func secretCommand(ctx context.Context, args []string) error {
if unannounced != nil { if unannounced != nil {
fmt.Printf(" this change could NOT be announced on the operator's channels: %v\n", unannounced) fmt.Printf(" this change could NOT be announced on the operator's channels: %v\n", unannounced)
} }
// The giving is recorded, never the value (novox/hq ADR 0380 §6): the desk already logs its own act; the
// terminal's own secret accept did not until now.
act := terminalGiveHandAct(node, module, name, askedBy)
if err := onTheBus(func(conn *nats.Conn) error {
_, err := link.RecordHandAct(ctx, conn, act)
return err
}); err != nil {
fmt.Printf(" this act could NOT be recorded in the hand-act log, and is done anyway: %v\n", err)
}
// Not printed back, and there is nowhere it could be printed from: it is sealed to that // Not printed back, and there is nowhere it could be printed from: it is sealed to that
// machine and the mesh cannot read it again. // machine and the mesh cannot read it again.
fmt.Printf("%s on %s now holds %q, sealed to that machine.\n", module, node, name) fmt.Printf("%s on %s now holds %q, sealed to that machine.\n", module, node, name)
@@ -556,16 +541,6 @@ func whoAsked() string {
return "the mesh" return "the mesh"
} }
// terminalGiveHandAct is the hand-act log's entry for a secret accepted at the controller's terminal
// (novox/hq ADR 0380 §6): who asked, where and that it happened — never the value, which the act never
// carries a field for.
func terminalGiveHandAct(node, module, name, askedBy string) link.HandAct {
return link.HandAct{Verb: "secret accept", Args: []string{node, module, name},
Why: fmt.Sprintf("the operator gave %s for %s on %s at the controller's terminal, asked by %s", name, module,
node, askedByName(askedBy)),
Cause: "given-at-the-terminal"}
}
// keepGiven keeps a value given at the controller's terminal, and announces it on the operator's channels // keepGiven keeps a value given at the controller's terminal, and announces it on the operator's channels
// (the confirmation review of 2026-10-09, N1-give). **A trusted party's — a module running as an account of its // (the confirmation review of 2026-10-09, N1-give). **A trusted party's — a module running as an account of its
// own: the router, a verified channel — is announced before it is kept, and not kept when the announcement // own: the router, a verified channel — is announced before it is kept, and not kept when the announcement
+3 -59
View File
@@ -153,15 +153,11 @@ var signalsTable = []signalRow{
return newestOf(f.standings, func(c conditions.Condition) time.Time { return c.LastObserved }) return newestOf(f.standings, func(c conditions.Condition) time.Time { return c.LastObserved })
}}, }},
{Row: "S9", Signal: "bus advisories: maximum deliveries, consumer deleted; the controller's own slow " + {Row: "S9", Signal: "bus advisories: maximum deliveries, consumer deleted; the controller's own slow " +
"consumer and refused subjects; every user's connection the server closed as a slow consumer or for " + "consumer and refused subjects", Emitter: "bus server's advisory subjects; the controller's connection",
"breaking the bus's rules (ADR 0357)",
Emitter: "bus server's advisory subjects; the controller's connection; the bus module's watch of its " +
"server (bus.slow-consumer, bus.connection-closed)",
Trigger: "any", Bound: "any occurrence; clears after an hour without another, a deleted consumer " + Trigger: "any", Bound: "any occurrence; clears after an hour without another, a deleted consumer " +
"once it exists again or the mesh no longer expects it, and a message given up on once DEAD_LETTERS " + "once it exists again or the mesh no longer expects it, and a message given up on once DEAD_LETTERS " +
"no longer holds it (novox/hq issue 330)", "no longer holds it (novox/hq issue 330)",
Kind: "slow-consumer, connection-closed, max-deliveries, refused, consumer-lost", Severity: conditions.Warning, Kind: "slow-consumer, max-deliveries, refused, consumer-lost", Severity: conditions.Warning, Phase: 1,
Phase: 1,
needs: func(f *signalFacts) error { return f.advisoriesErr }, watch: watchAdvisories, needs: func(f *signalFacts) error { return f.advisoriesErr }, watch: watchAdvisories,
newest: func(f *signalFacts) time.Time { newest: func(f *signalFacts) time.Time {
return newestOf(f.advisories, func(a link.Advisory) time.Time { return a.Last }) return newestOf(f.advisories, func(a link.Advisory) time.Time { return a.Last })
@@ -573,9 +569,6 @@ func watchAdvisories(f *signalFacts) []conditions.Observation {
// The advisory is remembered and read again on every look for an hour: the condition counts // The advisory is remembered and read again on every look for an hour: the condition counts
// what the bus said and when, not the looks (novox/hq issue 402). // what the bus said and when, not the looks (novox/hq issue 402).
Happened: a.Last, Times: a.Count} Happened: a.Last, Times: a.Count}
if a.User != "" && a.Reason != "" {
busUserClosed(&o, a, f.host)
}
if a.Kind == link.AdvisoryMaxDeliveries && a.Token == link.AdvisoryNotKept { if a.Kind == link.AdvisoryMaxDeliveries && a.Token == link.AdvisoryNotKept {
o.Machine = consumerMachine(a.Stream, a.Consumer) o.Machine = consumerMachine(a.Stream, a.Consumer)
o.Headline = clip(conditions.Capital(fmt.Sprintf("%s gave up on a message, not kept", o.Headline = clip(conditions.Capital(fmt.Sprintf("%s gave up on a message, not kept",
@@ -586,58 +579,9 @@ func watchAdvisories(f *signalFacts) []conditions.Observation {
} }
out = append(out, o) out = append(out, o)
} }
if f.busFaultsErr != nil {
out = append(out, busFaultsUnread(f.busFaultsErr))
}
return append(out, watchDeadLetters(f)...) return append(out, watchDeadLetters(f)...)
} }
// kindBusFaultsUnread is the bus module's events not read back (novox/hq ADR 0357): every other user's
// slow consumer is unheard while it lasts, and the controller's own signals are still said.
const kindBusFaultsUnread = "bus-faults-unread"
func busFaultsUnread(err error) conditions.Observation {
return conditions.Observation{Scope: conditions.ScopeBus, ID: "bus-module", Kind: kindBusFaultsUnread,
Severity: conditions.Warning, Confirm: true,
Summary: "what the bus module said of its users' connections could not be read back, so a slow consumer " +
"or a connection the bus closed is not said until it can",
Said: firstLine(err.Error())}
}
// busUserClosed words an advisory about a bus user's connection the server closed (novox/hq ADR 0357): the
// user as the operator says it and the machine it is on, the dotted user name kept in the evidence only.
func busUserClosed(o *conditions.Observation, a link.Advisory, host string) {
who, machine := busUserWords(a.User)
if machine == "" && a.User == "controller" {
machine = host
}
o.Machine = machine
times, count := "once", "once"
if a.Count > 1 {
times = fmt.Sprintf("%d times since %s", a.Count, a.First.UTC().Format("15:04 MST"))
count = fmt.Sprintf("%d times", a.Count)
}
switch a.Kind {
case link.AdvisorySlowConsumer:
o.Summary = fmt.Sprintf("the bus closed the connection of %s as a slow consumer %s in the last hour, "+
"newest at %s: it could not take what it was sent fast enough, and what was on its way to it was lost",
who, times, a.Last.UTC().Format("15:04 MST"))
o.Headline = clip(conditions.Capital(who)+" was too slow for the bus", 60)
o.Explanation = conditions.Capital(fmt.Sprintf("%s could not keep up with what the bus sent it, so the bus "+
"closed its connection (%s in the last hour) and what was on its way was lost. It connects again "+
"by itself; this clears an hour after the last time.", who, count))
o.Resolved = "Resolved: " + who + " kept up for an hour"
default:
o.Summary = fmt.Sprintf("the bus closed the connection of %s for breaking its rules %s in the last hour, "+
"newest at %s: %s", who, times, a.Last.UTC().Format("15:04 MST"), strings.ToLower(a.Reason))
o.Headline = clip(conditions.Capital(who)+" was cut off by the bus", 60)
o.Explanation = conditions.Capital(fmt.Sprintf("The bus closed the connection of %s because it broke "+
"the bus's rules (%s, %s in the last hour), so what it was sending was lost. This clears an hour "+
"after the last time.", who, strings.ToLower(a.Reason), count))
o.Resolved = "Resolved: " + who + " kept to the bus's rules for an hour"
}
}
// watchDeadLetters says each consumer that DEAD_LETTERS holds a message for (novox/hq issue 330): open // watchDeadLetters says each consumer that DEAD_LETTERS holds a message for (novox/hq issue 330): open
// while it holds any, so it clears when they are delivered again or dropped, never because the server // while it holds any, so it clears when they are delivered again or dropped, never because the server
// stopped saying it. // stopped saying it.
@@ -862,7 +806,7 @@ func watchLease(f *signalFacts) []conditions.Observation {
} }
} }
if len(lost) > 0 { if len(lost) > 0 {
how := "lost it: another holder wrote its key, or no renewal was made within the key's age" how := "lost it: its renewal was refused or could not be made"
if newest.How == inventory.EpochExpired { if newest.How == inventory.EpochExpired {
how = "stopped renewing it without giving it back, and was taken over" how = "stopped renewing it without giving it back, and was taken over"
} }
-27
View File
@@ -205,10 +205,6 @@ func printStatus(asked answers) error {
fmt.Printf("%d machine(s) have never been sent anything:\n %s\n", fmt.Printf("%d machine(s) have never been sent anything:\n %s\n",
len(never), strings.Join(never, ", ")) len(never), strings.Join(never, ", "))
} }
for _, n := range sortedKeysOf(stringsKeys(asked.recordedWaits)) {
fmt.Printf(" %s is behind, waiting for a person: %s — `push %s` sends it\n", n,
strings.Join(asked.recordedWaits[n], "; "), n)
}
fmt.Printf("\n `push --behind` sends them\n\n") fmt.Printf("\n `push --behind` sends them\n\n")
} }
@@ -544,29 +540,6 @@ func theThreeQuestions(ctx context.Context, open *stores) (answers, error) {
if err != nil { if err != nil {
return answers{}, err return answers{}, err
} }
// And, of those, what waits for a person's push (novox/hq ADR 0377 §6): a recorded module's move that went out
// with its walk raises nothing here.
if len(out.waiting) > 0 {
f, err := readMoveFacts(ctx, inv)
if err != nil {
return answers{}, err
}
for _, m := range out.waiting {
if m.Never {
continue
}
held, err := recordedWaiting(ctx, inv, f, m.Node)
if err != nil {
return answers{}, err
}
if len(held) > 0 {
if out.recordedWaits == nil {
out.recordedWaits = map[string][]string{}
}
out.recordedWaits[m.Node] = held
}
}
}
out.reported, err = inv.LastReports(ctx) out.reported, err = inv.LastReports(ctx)
if err != nil { if err != nil {
return answers{}, err return answers{}, err
+6
View File
@@ -3,6 +3,7 @@ package main
import ( import (
"context" "context"
"fmt" "fmt"
"os"
"time" "time"
"github.com/novox/mesh-controller/internal/catalogue" "github.com/novox/mesh-controller/internal/catalogue"
@@ -124,6 +125,11 @@ func openInventory(ctx context.Context) (*inventory.Inventory, error) {
if aliases, err := inv.Aliases(ctx); err == nil { if aliases, err := inv.Aliases(ctx); err == nil {
catalogue.UseAliases(aliases) catalogue.UseAliases(aliases)
} }
// And the list of predefined keys the mesh publishes (novox/hq ADR 0348 §7), which every manifest read again
// is judged against. A store a migration has not reached yet keeps none, and nothing names a key then.
if err := loadPredefined(ctx, inv); err != nil {
fmt.Fprintf(os.Stderr, "the list of predefined keys kept cannot be read: %v\n", err)
}
return inv, nil return inv, nil
} }
-157
View File
@@ -1,157 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"strings"
"time"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
)
// A new push to a pull request ends the check of its old head (novox/hq issue 726).
//
// A check judges one head; once the pull request has another, its verdict answers nobody and its run holds a build
// machine for minutes the new head waits behind. So when a check is asked for pull request N of a repository at a
// head, **every check of that pull request at another head ends**: a waiting one is dropped from the queue as
// `cancel` drops it, one in flight is ended on its machine as `kill` ends it. Each is said in the log.
//
// **Never anything else**: a try (`try`, asked for no pull request), a walk's build (no check), a delivery group's
// composed check (it answers several pull requests), another pull request's check, a dead ask (left for `clear
// --dead`), and a check of the same head — a recheck of that head is not another head.
// **Only for the pull request's current head.** Announcements of a pull request are not ordered, one is heard again
// after a restart, and a recheck asks for the head it recorded: a check asked for a head the pull request no longer
// has must never end the check of the head it has. So the forge is asked first which head the pull request has; only
// when it names this one does anything end, and when it cannot be asked, or names another, nothing ends and that is
// said.
// forgePullHead asks the forge, through its module's tools on the bus, which commit a pull request's head is. A
// variable so a test needs no forge.
var forgePullHead = func(ctx context.Context, js *broker.JetStream, owner, repo string, number int) (string, error) {
raw, _ := json.Marshal(map[string]any{"path": fmt.Sprintf("/repos/%s/%s/pulls/%d", owner, repo, number)})
answer, err := link.Ask(ctx, link.OverNATS{Conn: js.Conn()}, "gitea", "gitea_api", raw, 30*time.Second)
if err != nil {
return "", err
}
if answer.Error != "" {
return "", errors.New(answer.Error)
}
var pull struct {
Result struct {
Head struct {
SHA string `json:"sha"`
} `json:"head"`
} `json:"result"`
}
if err := json.Unmarshal(answer.Result, &pull); err != nil {
return "", fmt.Errorf("the forge's answer about the pull request cannot be read: %w", err)
}
if pull.Result.Head.SHA == "" {
return "", errors.New("the forge named no head for the pull request")
}
return pull.Result.Head.SHA, nil
}
// supersededChecks are the checks in the queue that a check of owner/repo#number at head ends: of the same pull
// request, at another head, waiting or in flight, and not the ask keep.
func supersededChecks(q link.Queue, owner, repo string, number int, head, keep string) []link.QueuedAsk {
if number <= 0 || head == "" {
return nil
}
var out []link.QueuedAsk
for _, a := range q.Asks {
c := a.Request.Check
switch {
case c == nil, link.IsTry(a.Request), c.Group != "", c.Number != number, a.ID == keep,
!strings.EqualFold(c.Owner, owner), !strings.EqualFold(c.Repo, repo),
a.State != link.AskWaiting && a.State != link.AskInFlight:
continue
}
ref := a.Request.Ref
if ref == "" {
ref = a.Ref
}
if ref == "" || ref == head || link.SameHead(ref, head) {
continue
}
out = append(out, a)
}
return out
}
// endSupersededChecks ends the checks a check of p at its head supersedes, keep being the new check's own ask, and
// says each in the log. A waiting one is cancelled — nothing recorded: a check builds nothing, and the forge's status
// of the old head is no longer the pull request's — and one a machine builds is killed there.
func endSupersededChecks(ctx context.Context, js *broker.JetStream, seat string, p link.PullUpdated, keep string) []string {
if p.Number <= 0 || p.Commit == "" {
return nil
}
current, err := forgePullHead(ctx, js, p.Owner, p.Repo, p.Number)
if err != nil {
return []string{fmt.Sprintf("%s/%s#%d (%.8s): nothing ended — the forge could not say the pull request's "+
"current head, so checks of its other heads keep running: %v", p.Owner, p.Repo, p.Number, p.Commit, err)}
}
if current != p.Commit && !link.SameHead(current, p.Commit) {
return []string{fmt.Sprintf("%s/%s#%d (%.8s): nothing ended — the pull request's head is %.12s now, not "+
"this one", p.Owner, p.Repo, p.Number, p.Commit, current)}
}
q, err := link.ReadQueue(ctx, js, seat)
if err != nil {
return []string{fmt.Sprintf("%s/%s#%d (%.8s): could not read the %s queue for checks of its earlier heads: %v",
p.Owner, p.Repo, p.Number, p.Commit, seat, err)}
}
var said []string
for _, a := range supersededChecks(q, p.Owner, p.Repo, p.Number, p.Commit, keep) {
old := a.Request.Ref
if old == "" {
old = a.Ref
}
what := fmt.Sprintf("check of %s/%s#%d at %.12s", p.Owner, p.Repo, p.Number, old)
var err error
if a.State == link.AskInFlight && a.On != "" {
var answer link.Answer
answer, err = link.AskSeatTool(ctx, js.Conn(), seat, "kill", a.On, map[string]string{"id": a.ID}, killAnswer)
if err == nil && answer.Error != "" {
err = fmt.Errorf("%s refused: %s", a.On, answer.Error)
}
} else {
_, err = cancelAsk(ctx, js, nil, seat, a)
}
if err != nil {
said = append(said, fmt.Sprintf("not ended: %s (%s) — superseded by %.12s, but: %v", what, a.ID, p.Commit, err))
continue
}
said = append(said, fmt.Sprintf("ended: %s — superseded by %.12s (%s, %s)", what, p.Commit, a.ID, a.State))
}
return said
}
// endingSupersededChecks ends them beside the announcement, so a kill waiting on its machine never holds the
// handler: on its own connection to the bus, within twice a kill's wait. A one-shot process (oneShot) ends them
// within the call instead, as its exit would cut them off.
func endingSupersededChecks(ctx context.Context, seat string, p link.PullUpdated, keep string, oneShot bool) {
end := func() {
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 2*killAnswer)
defer cancel()
js, err := dialTheBus()
if err != nil {
fmt.Fprintf(os.Stderr, "%s/%s#%d (%.8s): could not reach the bus to end checks of its earlier heads: %v\n",
p.Owner, p.Repo, p.Number, p.Commit, err)
return
}
defer js.Close()
for _, line := range endSupersededChecks(ctx, js, seat, p, keep) {
fmt.Println(line)
}
}
if oneShot {
end()
return
}
go end()
}
-192
View File
@@ -1,192 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
)
// A new push to a pull request ends the check of its old head (novox/hq issue 726): the waiting one is dropped from
// the queue, the one in flight is killed on its machine, and each is said. A try, a walk's build, another pull
// request's check, a group's check and a recheck of the new head are left alone. On the bus alone: no store.
func TestANewHeadEndsTheChecksOfTheOldOnes(t *testing.T) {
js, _ := aBuildQueue(t)
ctx := t.Context()
const (
oldHead = "1111111111111111111111111111111111111111"
olderHead = "2222222222222222222222222222222222222222"
newHead = "3333333333333333333333333333333333333333"
)
publish := func(r link.BuildRequest) {
t.Helper()
body, _ := json.Marshal(r)
if _, err := js.Context().Publish(link.BuildWork(), body); err != nil {
t.Fatal(err)
}
}
check := func(id string, number int, head string) link.BuildRequest {
return link.BuildRequest{ID: id, Repository: "https://forge.example/novox/a.git", Ref: head,
Check: &link.CheckRequest{Owner: "novox", Repo: "a", Number: number}}
}
// In flight first: taken by a holder, started on ace.
publish(check("check-running", 7, olderHead))
worker, _ := broker.HolderConsumerFor("", "", broker.DeclaredSeat{Name: link.TheBuildMachine, Accepts: []string{"build"}})
sub, err := js.Context().PullSubscribe(worker.Filters[0], worker.Name, nats.Bind(worker.Stream, worker.Name), nats.ManualAck())
if err != nil {
t.Fatal(err)
}
defer func() { _ = sub.Unsubscribe() }()
if _, err := sub.Fetch(1, nats.MaxWait(3*time.Second)); err != nil {
t.Fatal(err)
}
started, _ := json.Marshal(link.BuildStart{ID: "check-running", On: "ace", At: time.Now().UTC().Format(time.RFC3339Nano)})
if _, err := js.Context().Publish(link.BuildStarted(), started); err != nil {
t.Fatal(err)
}
publish(check("check-old", 7, oldHead))
publish(check("check-other-pull", 8, oldHead))
publish(check("check-recheck", 7, newHead))
publish(check("check-new", 7, newHead))
try := check("try-old", 7, oldHead)
try.Check.Number, try.Check.Try = 0, true
publish(try)
group := check("group-old", 7, oldHead)
group.Check.Group = "g1"
publish(group)
publish(link.BuildRequest{ID: "walk-build", Repository: "https://forge.example/novox/a.git", Ref: oldHead})
var mu sync.Mutex
var killed []string
stop, err := link.OverNATS{Conn: js.Conn()}.ServeNodeSeatTools(link.TheBuildMachine, "ace", map[string]link.ToolHandler{
"kill": func(_ context.Context, raw json.RawMessage) (any, error) {
var args struct{ ID string }
_ = json.Unmarshal(raw, &args)
mu.Lock()
killed = append(killed, args.ID)
mu.Unlock()
return map[string]any{"said": "killed " + args.ID}, nil
}}, nil)
if err != nil {
t.Fatal(err)
}
defer stop()
forgeSays(t, newHead, nil)
said := endSupersededChecks(ctx, js, link.TheBuildMachine,
link.PullUpdated{Owner: "novox", Repo: "a", Number: 7, Commit: newHead}, "check-new")
mu.Lock()
if len(killed) != 1 || killed[0] != "check-running" {
t.Errorf("the holder was asked to kill %v, want only the in-flight check of the older head", killed)
}
mu.Unlock()
q, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil {
t.Fatal(err)
}
if _, found := q.Find("check-old"); found {
t.Error("the waiting check of the old head is still in the queue")
}
for _, id := range []string{"check-other-pull", "check-recheck", "check-new", "try-old", "group-old", "walk-build"} {
if _, found := q.Find(id); !found {
t.Errorf("%s was ended; only checks of the same pull request at another head end", id)
}
}
text := strings.Join(said, "\n")
for _, want := range []string{
"ended: check of novox/a#7 at " + oldHead[:12] + " — superseded by " + newHead[:12],
"ended: check of novox/a#7 at " + olderHead[:12] + " — superseded by " + newHead[:12],
} {
if !strings.Contains(text, want) {
t.Errorf("the log does not say %q:\n%s", want, text)
}
}
if len(said) != 2 {
t.Errorf("said %d lines, want 2:\n%s", len(said), text)
}
}
// Which asks a check supersedes, read from the queue alone.
func TestSupersededChecksAreOnlyThePullRequestsOtherHeads(t *testing.T) {
ask := func(id, state, ref string, c *link.CheckRequest) link.QueuedAsk {
return link.QueuedAsk{ID: id, State: state, Ref: ref, Request: link.BuildRequest{ID: id, Ref: ref, Check: c}}
}
pr := func(n int) *link.CheckRequest { return &link.CheckRequest{Owner: "Novox", Repo: "A", Number: n} }
head := "abcdefabcdefabcdef0123456789abcdef012345"
q := link.Queue{Asks: []link.QueuedAsk{
ask("old", link.AskWaiting, "0123456789ab0123456789ab0123456789ab0123", pr(7)),
ask("short-same", link.AskWaiting, head[:12], pr(7)),
ask("dead", link.AskDead, "0123456789ab0123456789ab0123456789ab0123", pr(7)),
ask("other", link.AskWaiting, "0123456789ab0123456789ab0123456789ab0123", pr(8)),
ask("build", link.AskWaiting, "0123456789ab0123456789ab0123456789ab0123", nil),
ask("keep", link.AskWaiting, "0123456789ab0123456789ab0123456789ab0123", pr(7)),
}}
got := supersededChecks(q, "novox", "a", 7, head, "keep")
if len(got) != 1 || got[0].ID != "old" {
t.Fatalf("superseded %+v, want only old", got)
}
if got := supersededChecks(q, "novox", "a", 0, head, ""); got != nil {
t.Errorf("a check of no pull request superseded %+v", got)
}
}
// forgeSays makes the forge name head as every pull request's current head, or fail with err.
func forgeSays(t *testing.T, head string, err error) {
t.Helper()
was := forgePullHead
forgePullHead = func(context.Context, *broker.JetStream, string, string, int) (string, error) { return head, err }
t.Cleanup(func() { forgePullHead = was })
}
// An older head heard after a newer one — an announcement out of order, one heard again after a restart, a recheck
// of a recorded head — ends nothing: the forge says the pull request's head is the newer one. Nor does a check whose
// pull request's head the forge cannot say.
func TestAnOlderHeadHeardLateEndsNothing(t *testing.T) {
js, _ := aBuildQueue(t)
ctx := t.Context()
const (
oldHead = "1111111111111111111111111111111111111111"
newHead = "3333333333333333333333333333333333333333"
)
for _, id := range []string{"check-new", "check-old-late"} {
head := newHead
if id == "check-old-late" {
head = oldHead
}
body, _ := json.Marshal(link.BuildRequest{ID: id, Repository: "https://forge.example/novox/a.git", Ref: head,
Check: &link.CheckRequest{Owner: "novox", Repo: "a", Number: 7}})
if _, err := js.Context().Publish(link.BuildWork(), body); err != nil {
t.Fatal(err)
}
}
late := link.PullUpdated{Owner: "novox", Repo: "a", Number: 7, Commit: oldHead}
forgeSays(t, newHead, nil)
said := endSupersededChecks(ctx, js, link.TheBuildMachine, late, "check-old-late")
if len(said) != 1 || !strings.Contains(said[0], "nothing ended") {
t.Errorf("an older head heard late said %q", said)
}
forgeSays(t, "", errors.New("nothing serves gitea.gitea_api"))
said = endSupersededChecks(ctx, js, link.TheBuildMachine,
link.PullUpdated{Owner: "novox", Repo: "a", Number: 7, Commit: newHead}, "check-new")
if len(said) != 1 || !strings.Contains(said[0], "nothing ended") {
t.Errorf("a head the forge could not confirm said %q", said)
}
q, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil {
t.Fatal(err)
}
for _, id := range []string{"check-new", "check-old-late"} {
if _, found := q.Find(id); !found {
t.Errorf("%s was ended", id)
}
}
}
+204
View File
@@ -0,0 +1,204 @@
package main
import (
"bytes"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
snapshot "github.com/novox/mesh-controller/internal/facts"
"github.com/novox/mesh-controller/internal/inventory"
)
// A file's text kept as a template file in its module's repository (novox/hq ADR 0349 §2, design 57 package T1,
// issue 654): `module check` and the merge gate read it from the tree beside the manifest, so a change to a
// template alone is judged as a change to its module.
const templatedNotifier = `{"module":"notifier","version":"1",
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
"resources":[{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644","template":"files/notifier.conf"}]}`
// inlineNotifier is the same module as the mesh holds it: built, its text inline.
const inlineNotifier = `{"module":"notifier","version":"1",
"resources":[{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644","content":"volume = 3\n"}]}`
// aTemplatedModule writes a module's directory: its manifest and its files/.
func aTemplatedModule(t *testing.T, dir, manifest string, files map[string]string) string {
t.Helper()
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, "module.json")
if err := os.WriteFile(path, []byte(manifest), 0o644); err != nil {
t.Fatal(err)
}
for name, text := range files {
full := filepath.Join(dir, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte(text), 0o644); err != nil {
t.Fatal(err)
}
}
return path
}
func TestModuleCheckReadsTemplatesFromTheTreeBesideTheManifest(t *testing.T) {
good := aTemplatedModule(t, t.TempDir(), templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
var out bytes.Buffer
if err := moduleCheck([]string{good}, &out); err != nil || !strings.Contains(out.String(), "notifier: ok") {
t.Fatalf("a module with its text in files/ was refused: %v\n%s", err, out.String())
}
// A change to the template alone, to text the controller refuses inline, is refused the same.
bad := aTemplatedModule(t, t.TempDir(), templatedNotifier,
map[string]string{"files/notifier.conf": "volume = 3\nstore = ${dir:nowhere}\n"})
out.Reset()
if err := moduleCheck([]string{bad}, &out); err == nil || !strings.Contains(out.String(), "nowhere") {
t.Fatalf("a template whose text is refused inline passed:\n%s", out.String())
}
// A symlink under files/ that no template names: the build's packing refuses it, so the check does too.
linked := t.TempDir()
aTemplatedModule(t, linked, templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
if err := os.Symlink("/etc/hostname", filepath.Join(linked, "files", "elsewhere")); err != nil {
t.Fatal(err)
}
out.Reset()
if err := moduleCheck([]string{filepath.Join(linked, "module.json")}, &out); err == nil ||
!strings.Contains(out.String(), "files/elsewhere is a symlink") {
t.Fatalf("a symlink under files/ passed the check:\n%s", out.String())
}
// And a template the tree lacks, by name.
missing := aTemplatedModule(t, t.TempDir(), templatedNotifier, nil)
out.Reset()
if err := moduleCheck([]string{missing}, &out); err == nil || !strings.Contains(out.String(), "files/notifier.conf") {
t.Fatalf("a missing template was not refused by name:\n%s", out.String())
}
}
// templatedMesh is catalogueMesh with the notifier on the laptop, registered as built: its text inline.
func templatedMesh(t *testing.T) (snapshot.Facts, map[string]string) {
t.Helper()
open := aMesh(t)
ctx := t.Context()
manifests := map[string]string{
"lemurs": `{"module":"lemurs","version":"1","capabilities":["systemd"],
"resources":[{"id":"service","type":"service","unit":"lemurs.service","state":"running","boot":"enabled"}]}`,
"notifier": inlineNotifier,
}
for name, raw := range manifests {
m, err := catalogue.ParseManifest([]byte(raw))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if err := open.inventory.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-catalog",
Path: "modules/" + name, BuiltFrom: "c0ffee"}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", name); err != nil {
t.Fatalf("assign %s: %v", name, err)
}
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
return f, manifests
}
// aTemplatedTree is the catalogue's checkout with the notifier's text in files/.
func aTemplatedTree(t *testing.T, manifests map[string]string, text string, has bool) string {
t.Helper()
tree := aTree(t, withEdit(manifests, "notifier", ""))
files := map[string]string{}
if has {
files["files/notifier.conf"] = text
}
aTemplatedModule(t, filepath.Join(tree, "modules", "notifier"), templatedNotifier, files)
return tree
}
const notifierTemplate = "modules/notifier/files/notifier.conf"
func TestTheMergeGateJudgesAChangeToATemplateAlone(t *testing.T) {
f, manifests := templatedMesh(t)
// The module moved to its template, the same text: nothing changes on any machine.
v := gateJudged(t, f, aTemplatedTree(t, manifests, "volume = 3\n", true), "modules/notifier/module.json",
notifierTemplate)
if v.Verdict == "fail" || v.Modules["notifier"] != "" {
t.Fatalf("the module moved to its template with the same text reads as a change (%v):\n%s", v.Modules, v.Report())
}
// The template alone changed: its module changed, and composes.
v = gateJudged(t, f, aTemplatedTree(t, manifests, "volume = 4\n", true), notifierTemplate)
if v.Verdict == "fail" || v.Modules["notifier"] != "changed" {
t.Fatalf("a change to the template alone is not a change to its module (%v):\n%s", v.Modules, v.Report())
}
if v.Width == nil || strings.Join(v.Width.Modules, ",") != "notifier" {
t.Fatalf("a change to the template rebuilds %+v, not its module", v.Width)
}
// The template alone changed to text the controller refuses: the pull request fails, naming the module.
v = gateJudged(t, f, aTemplatedTree(t, manifests, "store = ${dir:nowhere}\n", true), notifierTemplate)
if v.Verdict != "fail" || !strings.Contains(v.Report(), "notifier") {
t.Fatalf("a template the controller refuses passed the gate:\n%s", v.Report())
}
// A symlinked files/ fails: nothing is read from outside the tree.
tree := aTree(t, withEdit(manifests, "notifier", ""))
elsewhere := t.TempDir()
if err := os.WriteFile(filepath.Join(elsewhere, "notifier.conf"), []byte("volume = 3\n"), 0o644); err != nil {
t.Fatal(err)
}
aTemplatedModule(t, filepath.Join(tree, "modules", "notifier"), templatedNotifier, nil)
if err := os.Symlink(elsewhere, filepath.Join(tree, "modules", "notifier", "files")); err != nil {
t.Fatal(err)
}
v = gateJudged(t, f, tree, notifierTemplate)
if v.Verdict != "fail" || !strings.Contains(v.Report(), "files/ is a symlink") {
t.Fatalf("a symlinked files/ passed the gate:\n%s", v.Report())
}
// And the template removed from the tree fails, naming it.
v = gateJudged(t, f, aTemplatedTree(t, manifests, "", false), notifierTemplate)
if v.Verdict != "fail" || !strings.Contains(v.Report(), "files/notifier.conf") {
t.Fatalf("a missing template passed the gate:\n%s", v.Report())
}
}
// What the gate composes for the module in its template form is what the mesh composes for it inline.
func TestTheMergeGateResolvesATemplatedModuleToItsInlineTwin(t *testing.T) {
dir := t.TempDir()
path := aTemplatedModule(t, dir, templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
m, err := catalogue.ParseManifest(raw)
if err != nil {
t.Fatal(err)
}
m, err = catalogue.WithTemplatesFrom(m, dir)
if err != nil {
t.Fatal(err)
}
resolved, err := standInBuild(m)
if err != nil {
t.Fatal(err)
}
inline, err := catalogue.ParseManifest([]byte(inlineNotifier))
if err != nil {
t.Fatal(err)
}
if builtAlike(resolved) != builtAlike(inline) {
got, _ := json.Marshal(resolved)
t.Fatalf("the templated module resolves to %s, not its inline twin", got)
}
}
@@ -0,0 +1,46 @@
[
{
"id": "found.server",
"image": "registry.invalid:5000/library@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"networks": [
"predecessor_default"
],
"type": "container"
},
{
"id": "library.access-storage/downloads",
"mode": "read",
"path": "/storage/downloads",
"type": "access"
},
{
"id": "library.access-storage/media/series",
"mode": "read",
"path": "/storage/media/series",
"type": "access"
},
{
"id": "library.config",
"mode": "0750",
"owner": "1001:2000",
"path": "/var/lib/library/config",
"type": "directory"
},
{
"content": "{\n \"accesses\": {\n \"downloads\": \"/storage/downloads\",\n \"series\": \"/storage/media/series\"\n },\n \"log\": \"warn\",\n \"places\": {\n \"config\": {\n \"owner\": \"1001:2000\",\n \"path\": \"/var/lib/library/config\"\n }\n },\n \"workers\": 4\n}\n",
"id": "library.runtime-config",
"mode": "0640",
"path": "/var/lib/library/config/runtime.json",
"type": "file"
},
{
"id": "library.server",
"image": "registry.invalid:5000/library@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"type": "container",
"volumes": [
"/var/lib/library/config:/config",
"/storage/media/series:/series",
"/storage/downloads:/downloads:ro"
]
}
]
@@ -0,0 +1,37 @@
[
{
"id": "library.access-home/media/series",
"mode": "read",
"path": "/home/media/series",
"type": "access"
},
{
"id": "library.access-storage/downloads",
"mode": "read",
"path": "/storage/downloads",
"type": "access"
},
{
"id": "library.config",
"mode": "0750",
"path": "/var/lib/library/config",
"type": "directory"
},
{
"content": "{\n \"accesses\": {\n \"downloads\": \"/storage/downloads\",\n \"series\": \"/home/media/series\"\n },\n \"log\": \"warn\"\n}\n",
"id": "library.runtime-config",
"mode": "0640",
"path": "/var/lib/library/config/runtime.json",
"type": "file"
},
{
"id": "library.server",
"image": "registry.invalid:5000/library@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"type": "container",
"volumes": [
"/var/lib/library/config:/config",
"/home/media/series:/series",
"/storage/downloads:/downloads:ro"
]
}
]
-7
View File
@@ -171,7 +171,6 @@ func (tm *tierMesh) raise(machine, kind, summary string) {
// The incident's tier, small: a dozen modules whose first machine is the same are sent there in one // The incident's tier, small: a dozen modules whose first machine is the same are sent there in one
// send, judged by one gate, and the rest of their machines are sent once — two sends, not two dozen. // send, judged by one gate, and the rest of their machines are sent once — two sends, not two dozen.
func TestATierOfManyModulesIsSentToEachMachineOnce(t *testing.T) { func TestATierOfManyModulesIsSentToEachMachineOnce(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
var modules []string var modules []string
for i := 1; i <= 12; i++ { for i := 1; i <= 12; i++ {
@@ -224,7 +223,6 @@ func TestATierOfManyModulesIsSentToEachMachineOnce(t *testing.T) {
// settling, is the node tools' — inside their settle window, the gate's bound — and no failure of the // settling, is the node tools' — inside their settle window, the gate's bound — and no failure of the
// modules sent with them: at the bound only the node tools are put back. // modules sent with them: at the bound only the node tools are put back.
func TestACoreBehindWhileTheNodeToolsSettleFailsNoOtherModule(t *testing.T) { func TestACoreBehindWhileTheNodeToolsSettleFailsNoOtherModule(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
tm := aTierMesh(t, "app1", "app2", broker.RuntimeModule) tm := aTierMesh(t, "app1", "app2", broker.RuntimeModule)
ctx := t.Context() ctx := t.Context()
inv := tm.open.inventory inv := tm.open.inventory
@@ -278,7 +276,6 @@ func TestACoreBehindWhileTheNodeToolsSettleFailsNoOtherModule(t *testing.T) {
// A machine-level condition about anything else holds the machine back as a whole: everything the send // A machine-level condition about anything else holds the machine back as a whole: everything the send
// moved there fails together, once, with that reason — not one module for it. // moved there fails together, once, with that reason — not one module for it.
func TestAMachineUnhealthyAsAWholeFailsTheWholeSendTogether(t *testing.T) { func TestAMachineUnhealthyAsAWholeFailsTheWholeSendTogether(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
tm := aTierMesh(t, "app1", "app2", "app3") tm := aTierMesh(t, "app1", "app2", "app3")
ctx := t.Context() ctx := t.Context()
@@ -310,7 +307,6 @@ func TestAMachineUnhealthyAsAWholeFailsTheWholeSendTogether(t *testing.T) {
// The machine-level conditions, sorted: one naming a module is that module's; the core behind is the // The machine-level conditions, sorted: one naming a module is that module's; the core behind is the
// core's when the send moved it and nobody's when it did not; anything else is the machine's as a whole. // core's when the send moved it and nobody's when it did not; anything else is the machine's as a whole.
func TestTheConditionsAboutAMachineItself(t *testing.T) { func TestTheConditionsAboutAMachineItself(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
since := time.Now().Add(-time.Minute) since := time.Now().Add(-time.Minute)
about := func(kind, id string) conditions.Condition { about := func(kind, id string) conditions.Condition {
return conditions.Condition{Key: "machine." + id + "." + kind, Kind: kind, Raised: time.Now(), return conditions.Condition{Key: "machine." + id + "." + kind, Kind: kind, Raised: time.Now(),
@@ -343,7 +339,6 @@ func TestTheConditionsAboutAMachineItself(t *testing.T) {
// rollback that failed; and `plans retry` asks it again rather than refusing a build that was never // rollback that failed; and `plans retry` asks it again rather than refusing a build that was never
// judged (the incident's "NOT put back: no build kept"). // judged (the incident's "NOT put back: no build kept").
func TestAModuleItsSendChangedNothingOfIsLeftAsItWasAndRetried(t *testing.T) { func TestAModuleItsSendChangedNothingOfIsLeftAsItWasAndRetried(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
g := aGateMesh(t) g := aGateMesh(t)
ctx := t.Context() ctx := t.Context()
@@ -399,7 +394,6 @@ func TestAModuleItsSendChangedNothingOfIsLeftAsItWasAndRetried(t *testing.T) {
// them, they are registered back and marked in the judging that finds them broken, the plan goes on judging // them, they are registered back and marked in the judging that finds them broken, the plan goes on judging
// the other module to its own pass, and only then fails. // the other module to its own pass, and only then fails.
func TestATierPutsABrokenModuleBackAtOnce(t *testing.T) { func TestATierPutsABrokenModuleBackAtOnce(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
judgeLeavesAsTheCore(t) judgeLeavesAsTheCore(t)
for _, order := range [][]string{{broker.RuntimeModule, "app1"}, {"app1", broker.RuntimeModule}} { for _, order := range [][]string{{broker.RuntimeModule, "app1"}, {"app1", broker.RuntimeModule}} {
t.Run("gate kept on "+order[0], func(t *testing.T) { t.Run("gate kept on "+order[0], func(t *testing.T) {
@@ -460,7 +454,6 @@ func TestATierPutsABrokenModuleBackAtOnce(t *testing.T) {
// decision 4): the leaf beside the node tools has its one reading at once, but the send is judged by the tool // decision 4): the leaf beside the node tools has its one reading at once, but the send is judged by the tool
// runner's rule: one reading, not before the node-engine's witness window for it has ended. // runner's rule: one reading, not before the node-engine's witness window for it has ended.
func TestASendCarryingTheToolRunnerWaitsForItsWitnessWindow(t *testing.T) { func TestASendCarryingTheToolRunnerWaitsForItsWitnessWindow(t *testing.T) {
withTheGate(t) // the first-machine gate\'s own path (novox/hq ADR 0360 keeps it for earlier walks)
tm := aTierMesh(t, broker.RuntimeModule, "app1") tm := aTierMesh(t, broker.RuntimeModule, "app1")
ctx := t.Context() ctx := t.Context()
// One reading counted per advance, as every test with a gate left open does: a gate judged with no time // One reading counted per advance, as every test with a gate left open does: a gate judged with no time
+1 -6
View File
@@ -23,12 +23,7 @@ import (
// **It holds the stores, not a copy of the decision.** What to do about an upgrade is read when // **It holds the stores, not a copy of the decision.** What to do about an upgrade is read when
// one arrives, so changing it takes effect on the next upgrade rather than on the next restart of // one arrives, so changing it takes effect on the next upgrade rather than on the next restart of
// the control plane. // the control plane.
type following struct { type following struct{ open *stores }
open *stores
// oneShot is a process that ends after the call (a recheck through `delivery check`): what it would do beside
// the call it does within it, or the process's exit would cut it off (novox/hq issue 726).
oneShot bool
}
// Upgraded sends the machines running a module the version the catalogue now considers current — // Upgraded sends the machines running a module the version the catalogue now considers current —
// or records that they are behind, which is the default and needs no record. // or records that they are behind, which is the default and needs no record.
@@ -1,91 +0,0 @@
package main
import (
"context"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A module may declare its unhealthy urgent (novox/hq ADR 0355): its module-unhealthy condition is then raised
// urgent; without the field it stays a warning; a wait for a new login stays a warning whatever the module says.
// judgedTwice judges one machine's statement twice, over a catalogue in which mail declares its unhealthy
// urgent and notes says nothing, and returns what is open.
func judgedTwice(t *testing.T, unhealthy map[string][]inventory.ResourceHealth) []conditions.Condition {
t.Helper()
ctx := t.Context()
k, _ := withConditionsInMemory(t)
was := readHoldingFor
t.Cleanup(func() { readHoldingFor = was })
readHoldingFor = func(_ context.Context, _ *inventory.Inventory, open []conditions.Condition) (*holding, error) {
return &holding{ctx: context.Background(), open: open, providers: map[string]providerLookup{},
shelf: map[string]catalogue.Manifest{
"mail": {Module: "mail", Version: "1", Unhealthy: &catalogue.UnhealthyPolicy{
Severity: catalogue.UnhealthyUrgent, Why: "mail is refused or stuck for everyone with a mailbox"}},
"notes": {Module: "notes", Version: "1"},
}}, nil
}
streaks := map[string]int{}
for look := 1; look <= 2; look++ {
for m := range unhealthy {
streaks[m] = look
}
if err := judgeModuleHealth(ctx, nil, k, "anchor", unhealthy, streaks, time.Now()); err != nil {
t.Fatal(err)
}
}
open, err := k.Open(ctx)
if err != nil {
t.Fatal(err)
}
return open
}
func containerDown(module string) inventory.ResourceHealth {
return inventory.ResourceHealth{Module: module, Resource: module + ".web", Kind: "container", Target: module,
State: link.StateUnhealthy, Reason: "down"}
}
func TestAModuleThatDeclaresItsUnhealthyUrgentIsRaisedUrgent(t *testing.T) {
open := judgedTwice(t, map[string][]inventory.ResourceHealth{"mail": {containerDown("mail")}})
c := conditionOf(open, "module.mail.anchor.unhealthy")
if c == nil {
t.Fatalf("mail's unhealthy condition was not raised: %+v", open)
}
if c.Severity != conditions.Urgent {
t.Errorf("mail declares its unhealthy urgent, and it was raised %s", c.Severity)
}
}
func TestAModuleThatSaysNothingOfItsUnhealthyStaysAWarning(t *testing.T) {
open := judgedTwice(t, map[string][]inventory.ResourceHealth{"notes": {containerDown("notes")}})
c := conditionOf(open, "module.notes.anchor.unhealthy")
if c == nil {
t.Fatalf("notes' unhealthy condition was not raised: %+v", open)
}
if c.Severity != conditions.Warning {
t.Errorf("notes says nothing of its unhealthy, and it was raised %s", c.Severity)
}
}
// Only the module-unhealthy condition changes (ADR 0355 §4): a wait for a new login (ADR 0254) of a module that
// declares its unhealthy urgent stays a warning.
func TestAWaitForANewLoginStaysAWarningWhenTheModuleDeclaresItsUnhealthyUrgent(t *testing.T) {
open := judgedTwice(t, map[string][]inventory.ResourceHealth{"mail": {relogin("mail", "operator"),
userUnit("mail", "operator")}})
c := conditionOf(open, "module.mail.anchor.relogin-needed")
if c == nil {
t.Fatalf("mail's relogin-needed condition was not raised: %+v", open)
}
if c.Severity != conditions.Warning {
t.Errorf("a wait for a new login was raised %s; want a warning", c.Severity)
}
if conditionOf(open, "module.mail.anchor.unhealthy") != nil {
t.Errorf("a wait for a new login raised mail's unhealthy condition beside it: %+v", open)
}
}
@@ -1,122 +0,0 @@
package main
import (
"reflect"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// novox/hq issue 754: a walk waiting for its delivery's word takes no build's outcome, nor does a walk take one for
// a module it never asked; asked under an id, it takes that id's alone.
func TestAWalkTakesOnlyTheOutcomesOfWhatItAsked(t *testing.T) {
asked := time.Date(2026, 10, 14, 23, 30, 0, 0, time.UTC)
before := asked.Add(-time.Minute)
waiting := inventory.Plan{Delivery: &inventory.PlanDelivery{Awaits: "mesh-delivery"}}
letGo := inventory.Plan{Delivery: &inventory.PlanDelivery{Awaits: "mesh-delivery", Go: &asked}}
own := &inventory.PlanModule{State: "asked", AskedAt: &asked, Build: "build-own"}
old := &inventory.PlanModule{State: "asked", AskedAt: &asked}
for _, c := range []struct {
what string
p inventory.Plan
state *inventory.PlanModule
at time.Time
id string
takes bool
}{
{"a rebuild while the walk waits", waiting, nil, asked, "build-rebuild", false},
{"its own id while it waits", waiting, own, asked, "build-own", false},
{"a module it never asked", letGo, nil, asked, "build-rebuild", false},
{"a module it never asked, state kept empty", letGo, &inventory.PlanModule{}, asked, "build-rebuild", false},
{"another id", letGo, own, asked, "build-rebuild", false},
{"its own id", letGo, own, asked, "build-own", true},
{"a plan from before ids, asked after it", letGo, old, asked.Add(time.Second), "x", true},
{"a plan from before ids, asked before it", letGo, old, before, "x", false},
{"a plan with no delivery, its own id", inventory.Plan{}, own, asked, "build-own", true},
} {
if got := takesOutcome(c.p, c.state, c.at, c.id); got != c.takes {
t.Errorf("%s: takes %v, want %v", c.what, got, c.takes)
}
}
}
// novox/hq issue 754: a tier partly asked asks the rest; a module held built that the plan never asked — the live
// shape, a hand rebuild taken in during the wait, built after the plan was made — is asked again; one asked is kept.
func TestATierPartlyAskedAsksTheRest(t *testing.T) {
created := time.Date(2026, 10, 14, 23, 30, 0, 0, time.UTC)
earlier, later := created.Add(-time.Hour), created.Add(7*time.Minute)
p := inventory.Plan{Created: created, Tiers: [][]string{{"build-agent", "gitea", "route-proxy", "stale", "kept", "own"}},
Modules: map[string]*inventory.PlanModule{
"gitea": {},
"route-proxy": {State: "built", BuiltAt: &later, Commit: "c51357f8"},
"stale": {State: "built", BuiltAt: &earlier},
"kept": {State: "asked", AskedAt: &created, Build: "build-kept"},
"own": {State: "built", AskedAt: &created, BuiltAt: &later, Build: "build-own"},
}}
if got, want := unaskedOf(p), []string{"build-agent", "gitea", "route-proxy", "stale"}; !reflect.DeepEqual(got, want) {
t.Fatalf("unasked %v, want %v", got, want)
}
if !askedInTier(p) {
t.Fatal("a tier with a module asked under an id reads as never asked")
}
p.Modules["kept"], p.Modules["own"] = nil, nil
if askedInTier(p) {
t.Fatal("a tier the plan never asked reads as asked")
}
}
// novox/hq issue 754, as seen on 2026-10-14: a rebuild lands while the walk waits for its word; the walk is let go;
// it asks every module of its tier, and completes it once those builds come in.
func TestARebuildInTheWaitIsNotTheWalksBuild(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
asked := asksRecorded(t)
twoTiers(t, open)
created := time.Now().UTC().Add(-10 * time.Minute)
plan := inventory.Plan{ID: "plan-754", Repository: "novox/a", Branch: "main", Commit: "c0ffee", Created: created,
State: inventory.PlanBuilding, Tiers: [][]string{{"a", "b"}}, Modules: map[string]*inventory.PlanModule{},
Delivery: &inventory.PlanDelivery{Awaits: "mesh-delivery"}}
if err := open.inventory.SavePlan(ctx, &plan); err != nil {
t.Fatal(err)
}
// A rebuild of a, beside the waiting walk: not joined, and its outcome not taken.
joined, said, err := joinAPlan(ctx, open, "a")
if err != nil || joined || said == "" {
t.Fatalf("a rebuild joined a waiting walk: %v %q %v", joined, said, err)
}
planBuilt(ctx, open, "a", "c0ffee", "", time.Now().UTC(), "build-rebuild")
p, _ := open.inventory.PlanByID(ctx, plan.ID)
if s := p.Modules["a"]; s != nil && s.State != "" {
t.Fatalf("the waiting walk took a rebuild as its build: %+v", s)
}
if len(*asked) != 0 {
t.Fatalf("a waiting walk asked %v", *asked)
}
// Let go: both are asked, and their outcomes complete the tier.
if _, err := letGo(ctx, open.inventory, plan.ID, "operator", "test"); err != nil {
t.Fatal(err)
}
advancePlans(ctx, open)
p, _ = open.inventory.PlanByID(ctx, plan.ID)
for _, m := range []string{"a", "b"} {
if s := p.Modules[m]; s == nil || s.State != "asked" || s.Build == "" {
t.Fatalf("let go, %s is %+v", m, s)
}
}
if len(*asked) != 2 {
t.Fatalf("asked %v", *asked)
}
for _, m := range []string{"a", "b"} {
id := p.Modules[m].Build
at, _ := link.BuildAskedAt(id)
planBuilt(ctx, open, m, "c0ffee", "", at, id)
}
advancePlans(ctx, open)
if p, _ = open.inventory.PlanByID(ctx, plan.ID); p.Open() && p.Tier == 0 {
t.Fatalf("the walk did not get past its tier: %s at %d, %q", p.State, p.Tier, p.Note)
}
}
-22
View File
@@ -91,12 +91,7 @@ type signalFacts struct {
// deadLettersNewest is when DEAD_LETTERS kept the newest message it holds for each of those // deadLettersNewest is when DEAD_LETTERS kept the newest message it holds for each of those
// consumers: the condition counts the messages given up on, not the looks (novox/hq issue 440). // consumers: the condition counts the messages given up on, not the looks (novox/hq issue 440).
deadLettersNewest map[string]time.Time deadLettersNewest map[string]time.Time
// deadLettersAging is, for each consumer whose dead letters came from a restricted stream, when each ages out
// (novox/hq ADR 0376 §6); nil when it was not read.
deadLettersAging map[string][]link.Aging
advisoriesErr error advisoriesErr error
// busFaultsErr is why the bus module's events could not be read this tick (novox/hq ADR 0357).
busFaultsErr error
selfCheck selfCheckFacts selfCheck selfCheckFacts
@@ -214,9 +209,6 @@ type watchdogs struct {
// confirm holds back a row blind for one tick: a read that timed out once on a loaded store. // confirm holds back a row blind for one tick: a read that timed out once on a loaded store.
confirm confirming confirm confirming
// aging is what is remembered of restricted dead letters between looks (novox/hq ADR 0376 §6).
aging agingTally
mu sync.Mutex mu sync.Mutex
ticked time.Time ticked time.Time
last *signalFacts last *signalFacts
@@ -270,8 +262,6 @@ func (w *watchdogs) tick(ctx context.Context) {
func (w *watchdogs) see(running context.Context, f *signalFacts) { func (w *watchdogs) see(running context.Context, f *signalFacts) {
var problems []string var problems []string
var blind []conditions.Observation var blind []conditions.Observation
// What aged out of DEAD_LETTERS, said and cleared in its own words before the rows clear it in theirs.
problems = append(problems, w.sayAgedOut(running, f)...)
for _, row := range watchedRows() { for _, row := range watchedRows() {
if err := row.needs(f); err != nil { if err := row.needs(f); err != nil {
blind = append(blind, blindRow(row, err)) blind = append(blind, blindRow(row, err))
@@ -368,15 +358,6 @@ func (w *watchdogs) gather(ctx context.Context) *signalFacts {
} }
f.advisories = link.Advisories.Since(now.Add(-advisoryQuiet)) f.advisories = link.Advisories.Since(now.Add(-advisoryQuiet))
f.lostConsumers, f.advisoriesErr = w.lostConsumers(ctx, f.advisories) f.lostConsumers, f.advisoriesErr = w.lostConsumers(ctx, f.advisories)
// And what the bus did to every other user's connection, as the bus module said it (novox/hq ADR 0357):
// read back for the same hour, and one advisory with the controller's own where both are its.
// A read that fails is its own condition, never S9 blind: the controller's own signals are still said.
if w.server != nil && w.js != nil {
var faults []link.Advisory
if faults, f.busFaultsErr = w.server.BusFaults(ctx, now.Add(-advisoryQuiet)); f.busFaultsErr == nil {
f.advisories = link.MergeAdvisories(f.advisories, faults)
}
}
if f.advisoriesErr == nil && w.js != nil { if f.advisoriesErr == nil && w.js != nil {
f.deadLetters, f.advisoriesErr = link.HeldDeadLetters(w.js.Context()) f.deadLetters, f.advisoriesErr = link.HeldDeadLetters(w.js.Context())
} }
@@ -390,9 +371,6 @@ func (w *watchdogs) gather(ctx context.Context) *signalFacts {
} }
} }
} }
if f.advisoriesErr == nil && w.js != nil {
f.deadLettersAging, f.advisoriesErr = link.AgingDeadLetters(w.js.Context(), f.deadLetters)
}
f.handActs, f.handActsErr = w.gatherHandActs(ctx, now) f.handActs, f.handActsErr = w.gatherHandActs(ctx, now)
f.refusedSends, f.refusedSendsErr = inv.RefusedSendsSince(ctx, now.Add(-generationRefusalsSaid)) f.refusedSends, f.refusedSendsErr = inv.RefusedSendsSince(ctx, now.Add(-generationRefusalsSaid))
f.doing, f.doingErr = inv.NotDoingWhatTheyWereTold(ctx) f.doing, f.doingErr = inv.NotDoingWhatTheyWereTold(ctx)
+2 -3
View File
@@ -38,9 +38,8 @@ func TestTheBuildSeatsHolderFollowsTheControllerThatDefinesItsWorker(t *testing.
if pos["build-agent"] <= pos["mesh-controller"] { if pos["build-agent"] <= pos["mesh-controller"] {
t.Fatalf("the build machine after the controller that defines its worker: %v", tiers) t.Fatalf("the build machine after the controller that defines its worker: %v", tiers)
} }
// Built-by orders nothing (novox/hq ADR 0360 §3): whichever builder runs builds the proxy, beside the controller. if pos["route-proxy"] <= pos["build-agent"] {
if pos["route-proxy"] != 0 { t.Fatalf("what the build machine builds comes after it: %v", tiers)
t.Fatalf("what the builder builds waits for a new one: %v", tiers)
} }
if hasCycle(tiers, edges) { if hasCycle(tiers, edges) {
t.Fatalf("no cycle here: %v", tiers) t.Fatalf("no cycle here: %v", tiers)
+2 -2
View File
@@ -3,7 +3,7 @@ module github.com/novox/mesh-controller
go 1.26.0 go 1.26.0
require ( require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013182924-5933eb757d44 git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261015210729-d3e182503c19
github.com/jackc/pgx/v5 v5.10.0 github.com/jackc/pgx/v5 v5.10.0
github.com/nats-io/nats-server/v2 v2.11.17 github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0 github.com/nats-io/nats.go v1.54.0
@@ -36,4 +36,4 @@ require (
// committed. Every build (the build agent's `go build`, the Dockerfile) compiles from vendor/ and // committed. Every build (the build agent's `go build`, the Dockerfile) compiles from vendor/ and
// fetches nothing; go refuses to build when vendor/ and this file disagree, so a pin moved without // fetches nothing; go refuses to build when vendor/ and this file disagree, so a pin moved without
// `go mod vendor` fails loudly, at once, everywhere. // `go mod vendor` fails loudly, at once, everywhere.
replace github.com/novox/mesh-host => git.novox.be/novox/mesh-host v0.0.0-20261013220009-b9a293f8c8bc replace github.com/novox/mesh-host => git.novox.be/novox/mesh-host v0.0.0-20261015211328-5a7ec3775b98
+4 -10
View File
@@ -1,13 +1,7 @@
git.novox.be/novox/mesh-host v0.0.0-20261013220009-b9a293f8c8bc h1:h5v2AbdjM/clMY1YxS9p7VuyYtIjkcw85w9Oz1cOYQc= git.novox.be/novox/mesh-host v0.0.0-20261015211328-5a7ec3775b98 h1:27nPn3XO9yfrHoAOuTuuJPVZTA3Bdnbqlc2wRI7S/LY=
git.novox.be/novox/mesh-host v0.0.0-20261013220009-b9a293f8c8bc/go.mod h1:tcTK4LMs1d6JpZUwy3hSHMFG/MIuDr/XFs7/nbeaW/c= git.novox.be/novox/mesh-host v0.0.0-20261015211328-5a7ec3775b98/go.mod h1:5QOm3cIfu/k9fN6QqgDbKGCcDst7Ul1UlcfsiMVm9j4=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c h1:l5onJwoIeH8yE/PjVlEeoPyXBsHp2NQiWLllz2fwX7s= git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261015210729-d3e182503c19 h1:mAMt3KFmrHzDiBv4MLL94opWcG9y73ISMrBUyisapQE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY= git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261015210729-d3e182503c19/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013141520-9588871d6629 h1:7TYFV1Q6Mcz+lsLP/sds87tysET4rKCn3rRIQV8gNmA=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013141520-9588871d6629/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013145622-f4745447ba6d h1:HEJyqorKktrkJPeeJdwEo0kdrJjH3fm/YVS9MGftCqY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013145622-f4745447ba6d/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013182924-5933eb757d44 h1:r5xIo5qWOXQdCSMxvqJy8kV4U+P/QWXFK6971Ha6wU8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013182924-5933eb757d44/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-34
View File
@@ -1,34 +0,0 @@
package broker
import (
"slices"
"testing"
)
// **The bus module's watch speaks through the runtime that carries it** (novox/hq ADR 0357): the node's
// runtime is granted the events the bus module declares, under the module's name, while the bus's own
// account keeps the snapshot API and nothing else (ADR 0235).
func TestTheRuntimeCarryingTheBusModuleMayPublishItsEventsAndTheBussAccountMayNot(t *testing.T) {
events := []string{"bus.slow-consumer", "bus.connection-closed"}
runtime := Principal{Kind: KindNodeTools, Node: "anchor", Module: RuntimeModule,
Carries: []Declared{{Module: "nats", Emits: events}}}
p, err := PermissionsFor(runtime)
if err != nil {
t.Fatal(err)
}
for _, e := range events {
if !slices.Contains(p.Publish, "mesh.mod.nats.event."+e) {
t.Errorf("the runtime carrying the bus module may not publish %s: %v", e, p.Publish)
}
}
account := Principal{Kind: KindModule, Node: "anchor", Module: "nats", Emits: events, SnapshotsTheBus: true}
a, err := PermissionsFor(account)
if err != nil {
t.Fatal(err)
}
for _, s := range a.Publish {
if s == "mesh.mod.nats.event.bus.slow-consumer" || s == "mesh.mod.nats.event.bus.connection-closed" {
t.Errorf("the bus's own account is granted %s", s)
}
}
}
+4 -21
View File
@@ -128,13 +128,7 @@ func ConsumerFor(p Principal) (Consumer, bool) {
// connected, and its graph stayed empty with nothing anywhere reporting why. // connected, and its graph stayed empty with nothing anywhere reporting why.
// And one that hears its own answers on a seat it uses (novox/hq ADR 0259 §3): an asker's warrants. // And one that hears its own answers on a seat it uses (novox/hq ADR 0259 §3): an asker's warrants.
hearsItsOwn := len(SeatTrafficOf(p.Module, nil, p.Uses, nil).Subscribe) > 0 hearsItsOwn := len(SeatTrafficOf(p.Module, nil, p.Uses, nil).Subscribe) > 0
hearsAStream := false if p.Kind != KindModule || (len(p.Consumes) == 0 && len(p.Watches) == 0 && !hearsItsOwn) {
for _, h := range p.Hears {
if len(h.Filters) > 0 {
hearsAStream = true
}
}
if p.Kind != KindModule || (len(p.Consumes) == 0 && len(p.Watches) == 0 && !hearsItsOwn && !hearsAStream) {
return Consumer{}, false return Consumer{}, false
} }
perms, err := PermissionsFor(p) perms, err := PermissionsFor(p)
@@ -146,14 +140,11 @@ func ConsumerFor(p Principal) (Consumer, bool) {
// business, which is why this is a filter and not the whole list. // business, which is why this is a filter and not the whole list.
var filters []string var filters []string
for _, s := range perms.Subscribe { for _, s := range perms.Subscribe {
// Not a module stream's, which its own consumer on that stream hears (novox/hq ADR 0363). if strings.Contains(s, ".event.") {
if strings.Contains(s, ".event.") && !strings.HasPrefix(s, moduleStreamPrefix) {
filters = append(filters, s) filters = append(filters, s)
} }
} }
// **A module that hears only a module's stream still has one here** (novox/hq ADR 0363 §7): what it if len(filters) == 0 {
// gives up on there is delivered again under `mesh.again.<consumer>.` on EVENTS, and only this filters it.
if len(filters) == 0 && !hearsAStream {
return Consumer{}, false return Consumer{}, false
} }
sort.Strings(filters) sort.Strings(filters)
@@ -190,13 +181,6 @@ func ConsumersOf(users []Principal) []ModuleConsumer {
} }
seen[p.Node+"/"+p.Module] = true seen[p.Node+"/"+p.Module] = true
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: c}) out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: c})
// And one on each module stream it hears (novox/hq ADR 0363 §5).
for _, h := range p.Hears {
if len(h.Filters) == 0 {
continue
}
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: heardConsumer(p, h)})
}
} }
for _, p := range users { for _, p := range users {
if p.Kind == KindModule { if p.Kind == KindModule {
@@ -209,8 +193,7 @@ func ConsumersOf(users []Principal) []ModuleConsumer {
} }
for _, d := range p.Carries { for _, d := range p.Carries {
add(Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits, add(Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits,
Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches})
Streams: d.Streams, Hears: ComposedHears(d)})
} }
} }
return out return out
+1 -54
View File
@@ -35,7 +35,7 @@ func TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose(t *testing.T)
carriedPub := subjectsIn(accounts, "publish") carriedPub := subjectsIn(accounts, "publish")
carriedSub := subjectsIn(accounts, "subscribe") carriedSub := subjectsIn(accounts, "subscribe")
if diff := requiredPublishGap(t, want.Publish, carriedPub); len(diff) > 0 { if diff := missing(want.Publish, carriedPub); len(diff) > 0 {
t.Errorf("the installer's user list does not let the controller publish %v — it would come up "+ t.Errorf("the installer's user list does not let the controller publish %v — it would come up "+
"and be refused on the first thing it tried", diff) "and be refused on the first thing it tried", diff)
} }
@@ -62,59 +62,6 @@ func TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose(t *testing.T)
} }
} }
// requiredPublishGap is what in want is missing from carried, leaving out a grant DegradableGrants names
// and saying why in the test log instead (novox/hq issue 835, the forge check of mesh-controller#345): the
// merge check reads mesh-host at the commit the machines run, which a change to mesh-host's examples alone
// never moves, so the carried list can lag the controller's derived one without the controller refusing to
// come up.
func requiredPublishGap(t *testing.T, want, carried []string) []string {
t.Helper()
diff := missing(want, carried)
var required []string
for _, subject := range diff {
if reason, ok := degradableReason(subject); ok {
t.Logf("the installer's user list does not let the controller publish %s, but that falls back "+
"rather than refuses: %s", subject, reason)
continue
}
required = append(required, subject)
}
return required
}
// degradableReason says whether a subject is one of DegradableGrants, and why.
func degradableReason(subject string) (string, bool) {
for _, g := range DegradableGrants {
if g.Subject == subject {
return g.Reason, true
}
}
return "", false
}
// TestADegradableGrantMissingFromTheInstallersListPasses is the forge check's own case (mesh-controller#345,
// novox/hq issue 835): an installer list that lacks operator-channel.tell alone, because mesh-host has not
// yet built with the grant, still passes — the controller falls back to the secret-given condition rather
// than coming up refused.
func TestADegradableGrantMissingFromTheInstallersListPasses(t *testing.T) {
want := []string{"mesh.seat.operator-channel.tool.tell", "mesh.seat.node-service-manager.tool.restart.*"}
carried := []string{"mesh.seat.node-service-manager.tool.restart.*"}
if diff := requiredPublishGap(t, want, carried); len(diff) > 0 {
t.Errorf("a degradable grant missing from the installer's list must not fail the genesis test, got %v", diff)
}
}
// TestANonDegradableGrantMissingFromTheInstallersListStillFails proves the exemption is named, not general:
// every other grant the controller derives stays required.
func TestANonDegradableGrantMissingFromTheInstallersListStillFails(t *testing.T) {
want := []string{"mesh.seat.operator-channel.tool.tell", "mesh.seat.node-service-manager.tool.restart.*"}
carried := []string{"mesh.seat.operator-channel.tool.tell"}
diff := requiredPublishGap(t, want, carried)
if len(diff) != 1 || diff[0] != "mesh.seat.node-service-manager.tool.restart.*" {
t.Errorf("a missing grant that is not degradable must still fail the genesis test, got %v", diff)
}
}
// theCarriedAccounts is the accounts file the installer's template writes at genesis. // theCarriedAccounts is the accounts file the installer's template writes at genesis.
func theCarriedAccounts(t *testing.T) string { func theCarriedAccounts(t *testing.T) string {
t.Helper() t.Helper()
-26
View File
@@ -173,10 +173,6 @@ func (j *JetStream) EnsureStream(s Stream) error {
Description: s.Why, Description: s.Why,
} }
want.AllowDirect = s.Direct want.AllowDirect = s.Direct
want.AllowMsgTTL = s.AllowMsgTTL
if s.Restricted {
want.Metadata = map[string]string{RestrictedMetadata: "true"}
}
if s.MaxBytes > 0 { if s.MaxBytes > 0 {
want.MaxBytes = s.MaxBytes want.MaxBytes = s.MaxBytes
} }
@@ -490,25 +486,3 @@ func (j *JetStream) LogStreams() ([]string, error) {
} }
return out, lister.Err() return out, lister.Err()
} }
// ConsumerNames is every durable consumer on a stream; none for a stream that does not exist yet.
func (j *JetStream) ConsumerNames(stream string) ([]string, error) {
if _, err := j.js.StreamInfo(stream); errors.Is(err, nats.ErrStreamNotFound) {
return nil, nil
} else if err != nil {
return nil, err
}
var out []string
for name := range j.js.ConsumerNames(stream) {
out = append(out, name)
}
return out, nil
}
// RemoveConsumer deletes one durable consumer (novox/hq ADR 0363 §6): one the composition no longer asks for.
func (j *JetStream) RemoveConsumer(stream, name string) error {
if err := j.js.DeleteConsumer(stream, name); err != nil && !errors.Is(err, nats.ErrConsumerNotFound) {
return err
}
return nil
}
-12
View File
@@ -61,15 +61,6 @@ type Membership struct {
// over every module on the machine, so one module's code reaching another's name or kind through // over every module on the machine, so one module's code reaching another's name or kind through
// the runtime is the runtime's to refuse. // the runtime is the runtime's to refuse.
SeatTraffic *SeatTraffic `json:"seat-traffic,omitempty"` SeatTraffic *SeatTraffic `json:"seat-traffic,omitempty"`
// Streams are this module's own streams (novox/hq ADR 0363 §2): an event one of them carries is published
// on that stream's subject, never on Emits. Hears are the module streams it hears, each through a durable
// consumer on that stream the runtime binds beside the one on EVENTS (§8).
Streams []StreamIssued `json:"streams,omitempty"`
Hears []HeardIssued `json:"hears,omitempty"`
// Waits are the event families a session may wait on through the mesh MCP server, the tool runner's
// alone (novox/hq ADR 0367 §2): the subjects its account is granted, under the names a session asks for.
// The tool runner refuses any family not listed here.
Waits []WaitIssued `json:"waits,omitempty"`
} }
// Served is one address a tool is answered on. // Served is one address a tool is answered on.
@@ -135,9 +126,6 @@ func MembershipFor(node string, d Declared, where Placements) Membership {
} }
m.State = stateIssuedFor(d, node) m.State = stateIssuedFor(d, node)
m.Logs = logsIssuedFor(d) m.Logs = logsIssuedFor(d)
m.Streams = streamsIssuedFor(d)
m.Hears = HeardIssuedFor(node, d)
m.Waits = waitsIssuedFor(d)
t := SeatTrafficOf(d.Module, d.Holds, d.Uses, d.Watches) t := SeatTrafficOf(d.Module, d.Holds, d.Uses, d.Watches)
for _, s := range append(append([]Seat{}, d.Uses...), d.Watches...) { for _, s := range append(append([]Seat{}, d.Uses...), d.Watches...) {
if !s.Kinded { if !s.Kinded {
-302
View File
@@ -1,302 +0,0 @@
package broker
import (
"fmt"
"sort"
"strings"
)
// A module's own stream on the bus (novox/hq ADR 0363).
//
// A module declares a stream as it declares a log: named locally, created by the controller from the
// catalogue on every raise and every send, never removed by the mesh. Its events are published on
// `mesh.stream.<module>.event.<event>`, a subject space the shared EVENTS stream (`mesh.mod.*.event.>`)
// does not take — the bus refuses two streams claiming one subject, so a module's stream cannot share
// EVENTS' — and a consumer hears it through a durable consumer of its own on that stream, named as on
// EVENTS. Pure; the controller's raise asserts what is composed here.
// moduleStreamPrefix starts every subject a module's own stream takes.
const moduleStreamPrefix = "mesh.stream."
// ModuleStreamSubject is where a module's event on its own stream lands: `mesh.stream.<module>.event.<event>`.
// The token before `.event.` is the emitter, as on EVENTS, so a runtime reads the same key from it.
func ModuleStreamSubject(module, event string) string {
return moduleStreamPrefix + module + ".event." + event
}
// OnAModuleStream says a subject is one a module's own stream takes.
func OnAModuleStream(subject string) bool { return strings.HasPrefix(subject, moduleStreamPrefix) }
// WirePattern is a local event pattern as the bus spells it: `**` is `>`.
func WirePattern(local string) string {
parts := strings.Split(local, ".")
for i, p := range parts {
if p == "**" {
parts[i] = ">"
}
}
return strings.Join(parts, ".")
}
// A ModuleStream is one module's declared stream as the bus holds it.
type ModuleStream struct {
Module string
// Name is its name on the bus, `MAIL`.
Name string
// Events are the events it carries, as the bus spells them (WirePattern).
Events []string
MaxAgeDays int
MaxMiB int
// Restricted says a trusted config value of the module grants who hears it (ADR 0363 §4).
Restricted bool
}
// Subjects are the subjects the stream takes.
func (s ModuleStream) Subjects() []string {
var out []string
for _, e := range s.Events {
out = append(out, ModuleStreamSubject(s.Module, e))
}
sort.Strings(out)
return out
}
// Stream is the stream the controller asserts: kept by limits, its age and size the module's, dropping
// its oldest when full — a consumer that was away catches up within the age, and the module's own record
// answers for anything older.
func (s ModuleStream) Stream() Stream {
why := fmt.Sprintf("%s's own stream (novox/hq ADR 0363): its events, kept %d days and at most %d MiB so a "+
"consumer that was away catches up", s.Module, s.MaxAgeDays, s.MaxMiB)
if s.Restricted {
why += "; heard only by the consumers its trusted config value grants"
}
return Stream{
Name: s.Name,
Subjects: s.Subjects(),
Retention: RetentionLimits,
MaxAge: s.MaxAgeDays * 24 * 60 * 60,
MaxBytes: int64(s.MaxMiB) * 1024 * 1024,
Why: why,
// Said on the bus, so a dead letter kept from it ages out with it (novox/hq ADR 0376 §3).
Restricted: s.Restricted,
}
}
// ModuleStreams is every declared module stream as the controller asserts it, in name order.
func ModuleStreams(declared []ModuleStream) []Stream {
sorted := append([]ModuleStream(nil), declared...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Name < sorted[j].Name })
out := make([]Stream, 0, len(sorted))
for _, s := range sorted {
out = append(out, s.Stream())
}
return out
}
// Heard is a module's stream another module hears, and the subjects it hears of it: its `consumes`
// within the stream and, for a restricted stream, within what the provider's trusted config value grants it
// (ADR 0363 §5). Composed only with at least one subject.
type Heard struct {
Stream string
Emitter string
Filters []string
// Restricted says the stream is restricted by its provider's trusted config value (ADR 0363 §4): then it
// is heard only by a module on an account of its own (ADR 0376 §1, ComposedHears).
Restricted bool
}
// ComposedHears is what of a module's hearing the controller composes (novox/hq ADR 0376 §1): every stream it
// hears, but a restricted one only for a module that runs in a runtime of its own on a bus account of its own
// (`runs-as`, ADR 0259). **The tool runner never hears a restricted stream**: a module it carries reaches the bus
// through the tool runner's account, which every agent session on that machine reaches the bus as too, so a
// consumer, its grants or its bind word composed there would hand the stream to any session. A module that
// declares no `runs-as` and keeps its own container credential gets none either: only `runs-as` is an account
// the mesh keeps from agents.
func ComposedHears(d Declared) []Heard {
if d.RunsAs != "" {
return d.Hears
}
var out []Heard
for _, h := range d.Hears {
if !h.Restricted {
out = append(out, h)
}
}
return out
}
// Withheld is a restricted stream a grant gives a module that the controller composes nothing for, because the
// module has no account of its own (novox/hq ADR 0376 §2): said in `doctor` and in the controller's log, never
// silent.
type Withheld struct {
Node, Module, Stream, Emitter string
}
// Said is a withheld hearing in the operator's words: the module, the stream and the remedy.
func (w Withheld) Said() string {
return fmt.Sprintf("%s on %s is granted part of %s's restricted stream %s, and hears none of it: it is carried "+
"by the machine's tool runner, whose bus account every agent session there shares, so nothing is composed "+
"for it. Declare runs-as in %s's manifest, so it runs as an account of its own (novox/hq ADR 0376)",
w.Module, w.Node, w.Emitter, w.Stream, w.Module)
}
// WithheldOf is every granted hearing of a restricted stream composed for nothing, by machine and module.
func WithheldOf(r Records) []Withheld {
var out []Withheld
for _, node := range sortedCopy(r.Nodes) {
for _, d := range r.Assigned[node] {
if d.RunsAs != "" {
continue
}
for _, h := range d.Hears {
if h.Restricted && len(h.Filters) > 0 {
out = append(out, Withheld{Node: node, Module: d.Module, Stream: h.Stream, Emitter: h.Emitter})
}
}
}
}
return out
}
// heardConsumer is the durable consumer a module hears a module's stream through: named as its consumer
// on EVENTS, on that stream, with the subjects it was granted as its filters.
func heardConsumer(p Principal, h Heard) Consumer {
filters := append([]string(nil), h.Filters...)
sort.Strings(filters)
return Consumer{
Name: consumerDurable(p),
Stream: h.Stream,
Filters: filters,
AckWaitSeconds: 30,
MaxDeliver: 5,
Why: "what " + p.Module + " consumes of " + h.Emitter + "'s stream " + h.Stream + " and is granted " +
"(novox/hq ADR 0363); after max-deliver it gives an event up, and the controller keeps it in " +
"DEAD_LETTERS until a person delivers it again or drops it",
}
}
// heardGrants are what a principal publishes to read and acknowledge the consumers it hears module streams
// through: asking about each, pulling from it, acknowledging it — its own name on each stream, no other's.
func heardGrants(p Principal, hears []Heard) []string {
var out []string
durable := consumerDurable(p)
for _, h := range hears {
if !safeSubject.MatchString(h.Stream) || len(h.Filters) == 0 {
continue
}
out = append(out,
"$JS.API.CONSUMER.INFO."+h.Stream+"."+durable,
"$JS.API.CONSUMER.MSG.NEXT."+h.Stream+"."+durable,
"$JS.ACK."+h.Stream+"."+durable+".>")
}
return out
}
// ownStreamGrants are the subjects a module publishes its own streams' events on, and no other's.
func ownStreamGrants(module string, streams []ModuleStream) []string {
var out []string
for _, s := range streams {
if s.Module != module {
continue
}
out = append(out, s.Subjects()...)
}
return out
}
// StreamIssued is one of a module's own streams as its membership names it (ADR 0363 §2): the events it
// carries, as the bus spells them, and the subject an event of them is published on, `{event}` the event.
type StreamIssued struct {
Stream string `json:"stream"`
Events []string `json:"events"`
Subject string `json:"subject"`
}
// HeardIssued is one module stream an assignment hears, as its membership names it (ADR 0363 §8): the
// stream and the durable consumer on it the runtime binds.
type HeardIssued struct {
Stream string `json:"stream"`
Consumer string `json:"consumer"`
}
// streamsIssuedFor is every stream of its own a module's membership names.
func streamsIssuedFor(d Declared) []StreamIssued {
var out []StreamIssued
for _, s := range d.Streams {
if s.Module != d.Module || !safeSubject.MatchString(d.Module) {
continue
}
out = append(out, StreamIssued{Stream: s.Name, Events: append([]string(nil), s.Events...),
Subject: ModuleStreamSubject(d.Module, "{event}")})
}
return out
}
// HeardIssuedFor is every module stream an assignment hears, as its membership names it.
func HeardIssuedFor(node string, d Declared) []HeardIssued {
var out []HeardIssued
// A restricted stream's bind word only for a module of its own account (novox/hq ADR 0376 §1).
for _, h := range ComposedHears(d) {
if len(h.Filters) == 0 {
continue
}
out = append(out, HeardIssued{Stream: h.Stream, Consumer: node + "_" + d.Module})
}
sort.Slice(out, func(i, j int) bool { return out[i].Stream < out[j].Stream })
return out
}
// Pruner is the part of a JetStream connection that removes a module's consumer on a module's stream it
// may no longer hear (ADR 0363 §6). Optional: what only records what it is asked to assert has none.
type Pruner interface {
ConsumerNames(stream string) ([]string, error)
RemoveConsumer(stream, name string) error
}
// PruneHeard removes, from each module stream, every durable consumer the composition no longer asks for,
// and answers what it removed as `<stream>/<consumer>`. A consumer that loses its grant loses what it reads
// through, whatever its runtime still holds; a stream that does not exist yet has nothing to remove.
func PruneHeard(p Pruner, streams []ModuleStream, wanted []Consumer) ([]string, error) {
keep := map[string]bool{}
for _, c := range wanted {
keep[c.Stream+"/"+c.Name] = true
}
var removed []string
for _, s := range streams {
names, err := p.ConsumerNames(s.Name)
if err != nil {
return removed, fmt.Errorf("listing the consumers on %s: %w", s.Name, err)
}
for _, n := range names {
if keep[s.Name+"/"+n] {
continue
}
if err := p.RemoveConsumer(s.Name, n); err != nil {
return removed, fmt.Errorf("removing %s's consumer on %s, which is no longer granted: %w", n, s.Name, err)
}
removed = append(removed, s.Name+"/"+n)
}
}
return removed, nil
}
// EnsureHeard asserts a consumer on a module's stream (ADR 0363 §5, §6). **On a restricted stream, a consumer
// the bus will not bring to match is removed**: a pull consumer delivers to whoever pulls it, so one left with
// its wider filters after a narrowing was refused would keep handing over what the grant no longer names,
// whatever the user list now says. The next assertion makes it again with the granted filters. The refusal is
// returned either way, saying what was done, so the send says it and raises it.
func EnsureHeard(r Raiser, c Consumer, restricted bool) error {
err := r.EnsureConsumer(c)
if err == nil || !restricted {
return err
}
p, prunes := r.(Pruner)
if !prunes {
return fmt.Errorf("%w; it could not be removed here, and may still deliver more than it is granted", err)
}
if rerr := p.RemoveConsumer(c.Stream, c.Name); rerr != nil {
return fmt.Errorf("%w; REMOVING IT FAILED TOO, so it may still deliver more than it is granted: %v", err, rerr)
}
return fmt.Errorf("%w; REMOVED, so it delivers nothing it is no longer granted: the next assertion makes it "+
"again with its granted filters (novox/hq ADR 0363 §6)", err)
}
-68
View File
@@ -1,68 +0,0 @@
package broker
import (
"slices"
"testing"
"time"
"github.com/novox/mesh-controller/internal/testbus"
)
// On a real bus: a module's stream is made beside EVENTS without a clash, keeps what the module publishes on
// its own subjects, a granted consumer hears only its part, a narrowed grant narrows it, and a withdrawn one
// is removed (novox/hq ADR 0363).
//
// go test ./internal/broker/ -run TestAModuleStreamOnARealBus
func TestAModuleStreamOnARealBus(t *testing.T) {
js, err := Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer js.Close()
for _, s := range []Stream{MeshStreams()[3], mailStream.Stream()} {
_ = js.js.DeleteStream(s.Name)
if err := js.EnsureStream(s); err != nil {
t.Fatalf("%s: %v", s.Name, err)
}
defer func(name string) { _ = js.js.DeleteStream(name) }(s.Name)
}
for _, account := range []string{"mesh", "work"} {
if _, err := js.js.Publish(ModuleStreamSubject("mesh-mail", "mail."+account+".message.added"), []byte("{}")); err != nil {
t.Fatalf("publishing %s's mail: %v", account, err)
}
}
if info, _ := js.js.StreamInfo(EventsStream); info.State.Msgs != 0 {
t.Fatalf("EVENTS kept %d of the module stream's events", info.State.Msgs)
}
reader := Principal{Kind: KindModule, Node: "ace", Module: "reader", Hears: []Heard{mailHeard}}
c := heardConsumer(reader, mailHeard)
if err := js.EnsureConsumer(c); err != nil {
t.Fatal(err)
}
time.Sleep(50 * time.Millisecond)
if info, _ := js.js.ConsumerInfo("MAIL", c.Name); info.NumPending != 1 {
t.Fatalf("the reader granted `mesh` has %d pending, want 1", info.NumPending)
}
narrowed := c
narrowed.Filters = []string{"mesh.stream.mesh-mail.event.mail.mesh.message.changed"}
if err := EnsureHeard(js, narrowed, true); err != nil {
t.Fatalf("narrowing the grant: %v", err)
}
if info, _ := js.js.ConsumerInfo("MAIL", c.Name); info.Config.FilterSubject != narrowed.Filters[0] || len(info.Config.FilterSubjects) != 0 {
t.Fatalf("narrowed to %q %v, want only %s", info.Config.FilterSubject, info.Config.FilterSubjects, narrowed.Filters[0])
}
if info, _ := js.js.ConsumerInfo("MAIL", c.Name); info.NumPending != 0 {
t.Fatalf("after narrowing, %d of the reader's former grant are still pending", info.NumPending)
}
removed, err := PruneHeard(js, []ModuleStream{mailStream}, nil)
if err != nil {
t.Fatal(err)
}
if !slices.Equal(removed, []string{"MAIL/ace_reader"}) {
t.Fatalf("removed %v", removed)
}
if names, _ := js.ConsumerNames("MAIL"); len(names) != 0 {
t.Fatalf("still on MAIL: %v", names)
}
}
-307
View File
@@ -1,307 +0,0 @@
package broker
import (
"errors"
"reflect"
"slices"
"strings"
"testing"
)
// mailStream is mesh-mail's own stream as the bus holds it (novox/hq ADR 0363).
var mailStream = ModuleStream{Module: "mesh-mail", Name: "MAIL",
Events: []string{"mail.*.message.added", "mail.*.message.changed", "mail.*.message.removed"},
MaxAgeDays: 7, MaxMiB: 1024, Restricted: true}
// mailHeard is what a reader granted the account `mesh` hears of it.
var mailHeard = Heard{Stream: "MAIL", Emitter: "mesh-mail", Restricted: true, Filters: []string{
"mesh.stream.mesh-mail.event.mail.mesh.message.added", "mesh.stream.mesh-mail.event.mail.mesh.message.changed"}}
// newsStream is a module's stream without a grant: heard by whoever asks (ADR 0363 §5), the tool runner included.
var newsStream = ModuleStream{Module: "news", Name: "NEWS", Events: []string{"story.**"}, MaxAgeDays: 7, MaxMiB: 16}
var newsHeard = Heard{Stream: "NEWS", Emitter: "news", Filters: []string{"mesh.stream.news.event.story.>"}}
// A module's stream is its own: subjects EVENTS does not claim, its age and size, kept by limits (ADR 0363 §1, §2).
func TestAModuleStreamTakesSubjectsEventsDoesNot(t *testing.T) {
s := mailStream.Stream()
if s.Name != "MAIL" || s.Retention != RetentionLimits || s.MaxAge != 7*24*60*60 || s.MaxBytes != 1024<<20 || s.DiscardNew {
t.Fatalf("composed as %+v", s)
}
want := []string{"mesh.stream.mesh-mail.event.mail.*.message.added", "mesh.stream.mesh-mail.event.mail.*.message.changed",
"mesh.stream.mesh-mail.event.mail.*.message.removed"}
if !reflect.DeepEqual(s.Subjects, want) {
t.Fatalf("subjects %v", s.Subjects)
}
for _, own := range MeshStreams() {
for _, a := range own.Subjects {
for _, b := range s.Subjects {
if SubjectsOverlap(a, b) {
t.Fatalf("%s claims %s, which overlaps %s of %s: the bus would refuse the stream", s.Name, b, a, own.Name)
}
}
}
}
if WirePattern("mail.**") != "mail.>" {
t.Fatal("`**` is spelled `>` on the wire")
}
}
// The provider publishes its stream's events on their subjects and nowhere on EVENTS for them; a consumer
// granted a part hears it through its own consumer on the stream, and one not granted has nothing there
// (ADR 0363 §2, §5).
func TestTheProviderPublishesItsStreamAndOnlyAGrantedConsumerHearsIt(t *testing.T) {
provider, err := PermissionsFor(Principal{Kind: KindModule, Node: "novox", Module: "mesh-mail",
Streams: []ModuleStream{mailStream}})
if err != nil {
t.Fatal(err)
}
if !MayPublish(provider, "mesh.stream.mesh-mail.event.mail.work.message.added") {
t.Fatal("the provider may not publish its own stream's event")
}
if MayPublish(provider, "mesh.mod.mesh-mail.event.mail.work.message.added") {
t.Fatal("the provider may publish its stream's event on EVENTS")
}
if MayPublish(provider, "mesh.stream.other.event.mail.work.message.added") {
t.Fatal("the provider may publish on another module's stream")
}
reader := Principal{Kind: KindModule, Node: "ace", Module: "reader", Hears: []Heard{mailHeard}}
perms, err := PermissionsFor(reader)
if err != nil {
t.Fatal(err)
}
for _, s := range []string{"$JS.API.CONSUMER.MSG.NEXT.MAIL.ace_reader", "$JS.API.CONSUMER.INFO.MAIL.ace_reader",
"$JS.ACK.MAIL.ace_reader.1"} {
if !MayPublish(perms, s) {
t.Errorf("the granted reader may not publish %s", s)
}
}
if !MaySubscribe(perms, "mesh.stream.mesh-mail.event.mail.mesh.message.added") ||
MaySubscribe(perms, "mesh.stream.mesh-mail.event.mail.work.message.added") {
t.Fatal("the reader's subscription is not what it was granted")
}
if MayPublish(perms, "$JS.API.CONSUMER.MSG.NEXT.MAIL.ace_other") {
t.Fatal("the reader may pull another's consumer")
}
stranger, err := PermissionsFor(Principal{Kind: KindModule, Node: "ace", Module: "stranger",
Consumes: []string{"billing.paid"}})
if err != nil {
t.Fatal(err)
}
if MayPublish(stranger, "$JS.API.CONSUMER.MSG.NEXT.MAIL.ace_stranger") ||
MaySubscribe(stranger, "mesh.stream.mesh-mail.event.mail.mesh.message.added") {
t.Fatal("a module not granted reaches the stream")
}
}
// A granted consumer of its own account has a durable consumer on the stream, with its granted subjects as
// filters, and an EVENTS consumer that hears what it gives up on delivered again (ADR 0363 §5, §7); a module of its
// own account is no module the runtime carries, so the runtime reads neither (ADR 0376 §1).
func TestAGrantedConsumerHasAConsumerOnTheStream(t *testing.T) {
records := Records{Nodes: []string{"ace", "novox"}, Assigned: map[string][]Declared{
"novox": {{Module: "mesh-mail", Streams: []ModuleStream{mailStream}}},
"ace": {{Module: RuntimeModule}, {Module: "reader", RunsAs: "reader", Hears: []Heard{mailHeard}, HearsAStream: true},
{Module: "stranger", HearsAStream: true}},
}}
users, err := Users(records)
if err != nil {
t.Fatal(err)
}
var onMail, onEvents []Consumer
for _, c := range ConsumersOf(users) {
switch c.Consumer.Stream {
case "MAIL":
onMail = append(onMail, c.Consumer)
case EventsStream:
onEvents = append(onEvents, c.Consumer)
}
}
if len(onMail) != 1 || onMail[0].Name != "ace_reader" || !slices.Equal(onMail[0].Filters, mailHeard.Filters) {
t.Fatalf("on MAIL: %+v", onMail)
}
if len(onEvents) != 1 || onEvents[0].Name != "ace_reader" || !slices.Equal(onEvents[0].Filters, []string{AgainFilter("ace_reader")}) {
t.Fatalf("on EVENTS: %+v", onEvents)
}
for _, u := range users {
if u.Node != "ace" || (u.Kind != KindNodeTools && u.Module != "reader") {
continue
}
perms, err := PermissionsFor(u)
if err != nil {
t.Fatal(err)
}
reads := MayPublish(perms, "$JS.API.CONSUMER.MSG.NEXT.MAIL.ace_reader") && MayPublish(perms, "$JS.ACK.MAIL.ace_reader.1")
if u.Kind == KindNodeTools && reads {
t.Fatal("the runtime may read the consumer on MAIL of a module of its own account")
}
if u.Kind == KindModule && !reads {
t.Fatal("the reader's own account may not read its consumer on MAIL")
}
if MayPublish(perms, "$JS.API.CONSUMER.MSG.NEXT.MAIL.ace_stranger") {
t.Fatal("a consumer no module is granted may be read")
}
}
}
// **The tool runner never hears a restricted stream** (ADR 0376 §1, §2): a module it carries, granted part of
// MAIL, gets no consumer on MAIL, the tool runner's account no grant to ask about, pull or acknowledge one there,
// and its membership no word to bind one; a module of its own account granted the same part gets all three, on
// its own account. The grant that composes nothing is said. A stream without a grant is heard through the tool
// runner as before.
func TestTheToolRunnerNeverHearsARestrictedStream(t *testing.T) {
records := Records{Nodes: []string{"ace", "novox"}, Assigned: map[string][]Declared{
"novox": {{Module: "mesh-mail", Streams: []ModuleStream{mailStream}}, {Module: "news", Streams: []ModuleStream{newsStream}}},
"ace": {{Module: RuntimeModule},
{Module: "carried", Hears: []Heard{mailHeard, newsHeard}, HearsAStream: true},
{Module: "mailer", RunsAs: "mailer", Hears: []Heard{mailHeard}, HearsAStream: true}},
}}
users, err := Users(records)
if err != nil {
t.Fatal(err)
}
var onMail, onNews []string
for _, c := range ConsumersOf(users) {
switch c.Consumer.Stream {
case "MAIL":
onMail = append(onMail, c.Consumer.Name)
case "NEWS":
onNews = append(onNews, c.Consumer.Name)
}
}
if !slices.Equal(onMail, []string{"ace_mailer"}) {
t.Fatalf("consumers on MAIL: %v, want only the mailer's", onMail)
}
if !slices.Equal(onNews, []string{"ace_carried"}) {
t.Fatalf("consumers on NEWS: %v, want the carried module's as before", onNews)
}
for _, u := range users {
if u.Node != "ace" {
continue
}
perms, err := PermissionsFor(u)
if err != nil {
t.Fatal(err)
}
switch {
case u.Kind == KindNodeTools:
for _, s := range perms.Publish {
if strings.Contains(s, ".MAIL.") {
t.Errorf("the tool runner's account may publish %s", s)
}
}
for _, s := range perms.Subscribe {
if strings.HasPrefix(s, "mesh.stream.mesh-mail.") {
t.Errorf("the tool runner's account may subscribe %s", s)
}
}
if !MayPublish(perms, "$JS.API.CONSUMER.MSG.NEXT.NEWS.ace_carried") {
t.Error("the tool runner may no longer read the carried module's consumer on a stream without a grant")
}
case u.Kind == KindModule && u.Module == "carried":
if MayPublish(perms, "$JS.API.CONSUMER.MSG.NEXT.MAIL.ace_carried") {
t.Error("a module without an account of its own may pull a consumer on MAIL")
}
case u.Kind == KindModule && u.Module == "mailer":
for _, s := range []string{"$JS.API.CONSUMER.INFO.MAIL.ace_mailer", "$JS.API.CONSUMER.MSG.NEXT.MAIL.ace_mailer",
"$JS.ACK.MAIL.ace_mailer.1"} {
if !MayPublish(perms, s) {
t.Errorf("the mailer's own account may not publish %s", s)
}
}
}
}
where := Placements{Nodes: map[string][]string{}}
carried := MembershipFor("ace", records.Assigned["ace"][1], where)
if !reflect.DeepEqual(carried.Hears, []HeardIssued{{Stream: "NEWS", Consumer: "ace_carried"}}) {
t.Fatalf("the carried module is told to bind %+v", carried.Hears)
}
mailer := MembershipFor("ace", records.Assigned["ace"][2], where)
if !reflect.DeepEqual(mailer.Hears, []HeardIssued{{Stream: "MAIL", Consumer: "ace_mailer"}}) {
t.Fatalf("the mailer is told to bind %+v", mailer.Hears)
}
withheld := WithheldOf(records)
if len(withheld) != 1 || withheld[0] != (Withheld{Node: "ace", Module: "carried", Stream: "MAIL", Emitter: "mesh-mail"}) {
t.Fatalf("withheld %+v", withheld)
}
if said := withheld[0].Said(); !strings.Contains(said, "carried on ace") || !strings.Contains(said, "MAIL") ||
!strings.Contains(said, "runs-as") {
t.Fatalf("said %q", said)
}
}
// The membership says where a module's stream events go and which consumers on module streams to bind
// (ADR 0363 §2, §8).
func TestTheMembershipNamesTheStreamAndWhatIsHeard(t *testing.T) {
where := Placements{Nodes: map[string][]string{}}
provider := MembershipFor("novox", Declared{Module: "mesh-mail", Streams: []ModuleStream{mailStream}}, where)
if len(provider.Streams) != 1 || provider.Streams[0].Stream != "MAIL" ||
provider.Streams[0].Subject != "mesh.stream.mesh-mail.event.{event}" ||
!slices.Equal(provider.Streams[0].Events, mailStream.Events) {
t.Fatalf("the provider's membership: %+v", provider.Streams)
}
reader := MembershipFor("ace", Declared{Module: "reader", RunsAs: "reader", Hears: []Heard{mailHeard}}, where)
if !reflect.DeepEqual(reader.Hears, []HeardIssued{{Stream: "MAIL", Consumer: "ace_reader"}}) {
t.Fatalf("the reader's membership: %+v", reader.Hears)
}
if none := MembershipFor("ace", Declared{Module: "stranger", HearsAStream: true}, where); none.Hears != nil {
t.Fatalf("a module not granted is told to bind %+v", none.Hears)
}
}
type fakePruner struct {
on map[string][]string
removed []string
}
func (f *fakePruner) ConsumerNames(stream string) ([]string, error) { return f.on[stream], nil }
func (f *fakePruner) RemoveConsumer(stream, name string) error {
f.removed = append(f.removed, stream+"/"+name)
return nil
}
// A consumer that lost its grant has its consumer on the stream removed by the next assertion (ADR 0363 §6).
func TestAConsumerThatLostItsGrantIsRemoved(t *testing.T) {
f := &fakePruner{on: map[string][]string{"MAIL": {"ace_reader", "ace_former"}}}
wanted := []Consumer{{Name: "ace_reader", Stream: "MAIL"}, {Name: "ace_former", Stream: EventsStream}}
removed, err := PruneHeard(f, []ModuleStream{mailStream}, wanted)
if err != nil {
t.Fatal(err)
}
if !slices.Equal(removed, []string{"MAIL/ace_former"}) || !slices.Equal(f.removed, removed) {
t.Fatalf("removed %v", removed)
}
}
type refusingRaiser struct {
fakePruner
refuse bool
}
func (r *refusingRaiser) EnsureStream(Stream) error { return nil }
func (r *refusingRaiser) EnsureConsumer(Consumer) error {
if r.refuse {
return errors.New("consumer update refused")
}
return nil
}
// A narrowing the bus refuses on a restricted stream removes the consumer, so its wider filters deliver nothing
// more; on an unrestricted stream the refusal is only said (review of PR 338, ADR 0363 §6).
func TestARefusedNarrowingOnARestrictedStreamRemovesTheConsumer(t *testing.T) {
c := Consumer{Name: "ace_reader", Stream: "MAIL", Filters: mailHeard.Filters[:1]}
r := &refusingRaiser{refuse: true}
err := EnsureHeard(r, c, true)
if err == nil || !strings.Contains(err.Error(), "REMOVED") || !slices.Equal(r.removed, []string{"MAIL/ace_reader"}) {
t.Fatalf("restricted: %v, removed %v", err, r.removed)
}
r = &refusingRaiser{refuse: true}
if err := EnsureHeard(r, c, false); err == nil || len(r.removed) != 0 {
t.Fatalf("unrestricted: %v, removed %v", err, r.removed)
}
r = &refusingRaiser{}
if err := EnsureHeard(r, c, true); err != nil || len(r.removed) != 0 {
t.Fatalf("accepted: %v, removed %v", err, r.removed)
}
}
-86
View File
@@ -1,86 +0,0 @@
package broker
import (
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/testbus"
)
// **DEAD_LETTERS and EVENTS take a message's own time to live once updated in place** (novox/hq ADR 0376 §3, §5),
// on the pinned server: each made as it was before, without one, is brought to its definition by the controller's
// assertion, keeps what it held, and removes a message once its time to live passes. A restricted module stream says
// so in its metadata; one without a grant does not.
//
// go test ./internal/broker/ -run TestTheStreamsTakeAMessagesTimeToLive
func TestTheStreamsTakeAMessagesTimeToLive(t *testing.T) {
js, err := Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer js.Close()
for _, s := range MeshStreams() {
if s.Name != EventsStream && s.Name != DeadLettersStream {
if s.AllowMsgTTL {
t.Errorf("%s takes a time to live of a message's own, which nothing asks of it", s.Name)
}
continue
}
if !s.AllowMsgTTL {
t.Fatalf("%s is not defined to take a message's time to live", s.Name)
}
_ = js.js.DeleteStream(s.Name)
before := s
before.AllowMsgTTL = false
if err := js.EnsureStream(before); err != nil {
t.Fatalf("%s as it was: %v", s.Name, err)
}
subject := DeadLetterSubject("EVENTS", "ace_reader")
if s.Name == EventsStream {
subject = "mesh.mod.gitea.event.pull.merged"
}
if _, err := js.js.Publish(subject, []byte("held")); err != nil {
t.Fatal(err)
}
if err := js.EnsureStream(s); err != nil {
t.Fatalf("%s updated in place: %v", s.Name, err)
}
info, err := js.js.StreamInfo(s.Name)
if err != nil || !info.Config.AllowMsgTTL || info.State.Msgs != 1 {
t.Fatalf("%s after the update: %+v (%v)", s.Name, info, err)
}
msg := &nats.Msg{Subject: subject, Data: []byte("brief"), Header: nats.Header{}}
msg.Header.Set(nats.MsgTTLHdr, "1s")
ack, err := js.js.PublishMsg(msg)
if err != nil {
t.Fatalf("%s refused a message's time to live: %v", s.Name, err)
}
deadline := time.Now().Add(8 * time.Second)
for {
if _, err := js.js.GetMsg(s.Name, ack.Sequence); err != nil {
break
}
if time.Now().After(deadline) {
t.Fatalf("%s still holds a message past its time to live", s.Name)
}
time.Sleep(200 * time.Millisecond)
}
if info, _ := js.js.StreamInfo(s.Name); info.State.Msgs != 1 {
t.Fatalf("%s holds %d, want only the message without an age", s.Name, info.State.Msgs)
}
_ = js.js.PurgeStream(s.Name)
}
if !mailStream.Stream().Restricted || newsStream.Stream().Restricted {
t.Fatal("a module stream does not say whether it is restricted")
}
_ = js.js.DeleteStream("MAIL")
if err := js.EnsureStream(mailStream.Stream()); err != nil {
t.Fatal(err)
}
defer func() { _ = js.js.DeleteStream("MAIL") }()
if info, _ := js.js.StreamInfo("MAIL"); info.Config.Metadata[RestrictedMetadata] != "true" {
t.Fatalf("MAIL's metadata: %v", info.Config.Metadata)
}
}
+1 -63
View File
@@ -141,10 +141,6 @@ type Principal struct {
KeyedReads []KeyedRead KeyedReads []KeyedRead
// Logs is the local names of the logs this principal's module keeps (novox/hq ADR 0297). // Logs is the local names of the logs this principal's module keeps (novox/hq ADR 0297).
Logs []string Logs []string
// Streams are the streams this principal's module owns, and Hears the module streams it hears with the
// subjects it was granted (novox/hq ADR 0363).
Streams []ModuleStream
Hears []Heard
// SnapshotsTheBus is the bus's own module, the one holding mesh-broker (novox/hq ADR 0235). Its // SnapshotsTheBus is the bus's own module, the one holding mesh-broker (novox/hq ADR 0235). Its
// whole authority is BusSnapshotGrants: it copies the streams for the night's backup and nothing // whole authority is BusSnapshotGrants: it copies the streams for the night's backup and nothing
@@ -279,34 +275,6 @@ var VerbsTheControllerAsksTheDeliveryOwner = []SeatVerb{{Seat: "mesh-delivery",
var VerbsTheControllerActsOnAWarrant = []SeatVerb{{Seat: "node-service-manager", Verb: "restart"}, var VerbsTheControllerActsOnAWarrant = []SeatVerb{{Seat: "node-service-manager", Verb: "restart"},
{Seat: ControllerSeat, Verb: "plans"}} {Seat: ControllerSeat, Verb: "plans"}}
// VerbsTheControllerAsksTheOperatorChannel are the operator channel's verbs the controller calls (novox/hq
// ADR 0380): `tell`, one plain message with no condition behind it — a module's own secret given, named,
// where and when and by whom, asking nothing. A mesh seat's verb is flat: no machine in the subject.
var VerbsTheControllerAsksTheOperatorChannel = []SeatVerb{{Seat: "operator-channel", Verb: "tell"}}
// DegradableGrant is a publish grant whose absence from the installer's carried first-user list is not a
// bug in the template: a fallback covers the refusal.
type DegradableGrant struct {
Subject string
Reason string
}
// DegradableGrants are the only publish grants TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose
// may find missing from the installer's carried list without failing (novox/hq issue 835, the forge check
// of mesh-controller#345): a change that only edits mesh-host's examples can never satisfy a check that
// reads the mesh-host commit the machines actually run, so the carried list lags behind the controller's
// derived one until mesh-host's own module next builds. operator-channel.tell is the one grant the
// controller already has a fallback for: refused (no holder, or the bus refuses it), the controller raises
// the secret-given condition exactly as before (ADR 0380 §9), so an older carried list still comes up and
// works. Every other grant stays required — missing one of those is a controller that comes up and is
// refused on the first thing it tries, with no fallback that covers it.
var DegradableGrants = []DegradableGrant{
{
Subject: "mesh.seat." + VerbsTheControllerAsksTheOperatorChannel[0].Seat + ".tool." + VerbsTheControllerAsksTheOperatorChannel[0].Verb,
Reason: "refused: the controller raises the secret-given condition as before (ADR 0380 §9)",
},
}
// perMachineEvents are a node-scoped seat's events about the holder itself, whose last token is the // perMachineEvents are a node-scoped seat's events about the holder itself, whose last token is the
// holder's machine (novox/hq ADR 0219): `paused.<node>`, the build agent saying whether it takes work. // holder's machine (novox/hq ADR 0219): `paused.<node>`, the build agent saying whether it takes work.
var perMachineEvents = map[string]bool{"paused.*": true} var perMachineEvents = map[string]bool{"paused.*": true}
@@ -528,10 +496,6 @@ func PermissionsFor(p Principal) (Permissions, error) {
for _, v := range VerbsTheControllerAsksForASecret { for _, v := range VerbsTheControllerAsksForASecret {
pub = append(pub, "mesh.seat."+v.Seat+".tool."+v.Verb+".*") pub = append(pub, "mesh.seat."+v.Seat+".tool."+v.Verb+".*")
} }
// And the operator channel, a mesh seat, asked flat: one message when a secret is given (ADR 0380).
for _, v := range VerbsTheControllerAsksTheOperatorChannel {
pub = append(pub, "mesh.seat."+v.Seat+".tool."+v.Verb)
}
// And the delivery's owner, a mesh seat, asked on its flat subjects (ADR 0239). // And the delivery's owner, a mesh seat, asked on its flat subjects (ADR 0239).
for _, v := range VerbsTheControllerAsksTheDeliveryOwner { for _, v := range VerbsTheControllerAsksTheDeliveryOwner {
pub = append(pub, "mesh.seat."+v.Seat+".tool."+v.Verb) pub = append(pub, "mesh.seat."+v.Seat+".tool."+v.Verb)
@@ -673,8 +637,6 @@ func PermissionsFor(p Principal) (Permissions, error) {
for _, e := range p.Emits { for _, e := range p.Emits {
pub = append(pub, own+".event."+e) pub = append(pub, own+".event."+e)
} }
// And its own streams' events, on their own subjects (novox/hq ADR 0363 §2).
pub = append(pub, ownStreamGrants(p.Module, p.Streams)...)
// Every tool under its own name, not a list: the tools a module serves are what its code // Every tool under its own name, not a list: the tools a module serves are what its code
// answers, and a second copy of that list in the manifest would be a second source of // answers, and a second copy of that list in the manifest would be a second source of
// truth for the mesh to keep in step (2026-09-28: every module that served a tool was // truth for the mesh to keep in step (2026-09-28: every module that served a tool was
@@ -716,16 +678,6 @@ func PermissionsFor(p Principal) (Permissions, error) {
sub = append(sub, subject) sub = append(sub, subject)
} }
// 2a. What it hears of a module's own stream, as it was granted (novox/hq ADR 0363 §5): the subjects,
// and its own consumer on that stream, asked about, pulled from and acknowledged.
for _, h := range p.Hears {
if len(h.Filters) == 0 {
continue
}
sub = append(sub, h.Filters...)
}
pub = append(pub, heardGrants(p, p.Hears)...)
// 2b. Events of a role it watches, under the seat's own namespace. Subscribe only: watching a // 2b. Events of a role it watches, under the seat's own namespace. Subscribe only: watching a
// role is hearing what it announced, not taking part in it. // role is hearing what it announced, not taking part in it.
for _, w := range p.Watches { for _, w := range p.Watches {
@@ -848,8 +800,6 @@ func PermissionsFor(p Principal) (Permissions, error) {
for _, e := range d.Emits { for _, e := range d.Emits {
pub = append(pub, own+".event."+e) pub = append(pub, own+".event."+e)
} }
// And its own streams' events (novox/hq ADR 0363 §2).
pub = append(pub, ownStreamGrants(d.Module, d.Streams)...)
for _, s := range d.Holds { for _, s := range d.Holds {
if servedOnlyByTheController(s) { if servedOnlyByTheController(s) {
continue continue
@@ -888,11 +838,7 @@ func PermissionsFor(p Principal) (Permissions, error) {
// consumer is still the controller's to make, from the module's own principal. // consumer is still the controller's to make, from the module's own principal.
for _, d := range p.Carries { for _, d := range p.Carries {
own := Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits, own := Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits,
Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches}
Streams: d.Streams, Hears: ComposedHears(d)}
// And the module streams it hears, each through its own consumer there (novox/hq ADR 0363 §8) — never
// a restricted one, which only a module on an account of its own hears (ADR 0376 §1).
pub = append(pub, heardGrants(own, own.Hears)...)
if _, consumes := ConsumerFor(own); !consumes { if _, consumes := ConsumerFor(own); !consumes {
continue continue
} }
@@ -916,14 +862,6 @@ func PermissionsFor(p Principal) (Permissions, error) {
for _, d := range p.Carries { for _, d := range p.Carries {
pub = append(pub, logGrants(d.Module, logNames(d.Logs))...) pub = append(pub, logGrants(d.Module, logNames(d.Logs))...)
} }
// **And the event families a session may wait on** (novox/hq ADR 0367 §2): the tool runner's module's own,
// named one by one in its manifest, as a plain subscription — no consumer is made for them, and no
// call or setting adds one.
waits, err := waitGrants(p.Carries)
if err != nil {
return Permissions{}, err
}
sub = append(sub, waits...)
// **Never the traffic of a trusted holder** (novox/hq ADR 0259 §8): the machine's runtime runs as the // **Never the traffic of a trusted holder** (novox/hq ADR 0259 §8): the machine's runtime runs as the
// operator's account, which every agent runs as, so a module saying warrants or speaking for a kind // operator's account, which every agent runs as, so a module saying warrants or speaking for a kind
// that proves its sender is never composed into it — refused here, naming it, whatever registration // that proves its sender is never composed into it — refused here, naming it, whatever registration
@@ -1,23 +0,0 @@
package broker
import (
"slices"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The catalogue's list of the mesh's own stream names, which a module's stream may not take, is the broker's
// own list (novox/hq ADR 0363 §1): read from MeshStreams, so a stream added there cannot be forgotten there.
func TestTheCataloguesReservedStreamNamesAreTheMeshsOwn(t *testing.T) {
var own []string
for _, s := range MeshStreams() {
own = append(own, s.Name)
}
reserved := append([]string(nil), catalogue.MeshStreamNames...)
slices.Sort(own)
slices.Sort(reserved)
if !slices.Equal(own, reserved) {
t.Fatalf("the broker's own streams are %v, the catalogue reserves %v", own, reserved)
}
}

Some files were not shown because too many files have changed in this diff Show More