Compare commits

..
Author SHA1 Message Date
mesh-admin 2959c81435 Merge pull request 'Link a build's bases into its repository before pushing it (issue 694)' (#318) from fix/694-a-push-holds-its-bases into main 2026-10-14 01:19:24 +00:00
jochen 8933864a49 Link a build's bases into its repository before pushing it (issue 694)
mesh/merge-gate head 8933864a49d0 merged into trunk df9e5728a925 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 8933864a49d0 merged into trunk df9e5728a925 — pass: its merge-check.sh passed
mesh/delivery delivered
A runtime on containerd's image store, the default on a fresh docker 29,
need not hold the compressed layers of a base it built FROM: a layer it
had unpacked already is never fetched. Its push then fails on that layer
although the registry holds it under the base's mirror. A push reads
locally only what the target repository lacks, so mount the base's blobs
there first.
2026-10-14 02:59:53 +02:00
mesh-admin df9e5728a9 Merge pull request 'Try a pushed branch on the build seat before its pull request (hq issue 686)' (#314) from fix/686-a-branch-is-tested-before-its-pull-request into main 2026-10-14 00:57:46 +00:00
mesh-admin 2d18f438c5 Merge pull request 'A no-move tier waits for no new report, so its walk asks the next tier (issue 648)' (#309) from fix/648-a-no-move-tier-waits-for-nothing into main 2026-10-14 00:32:47 +00:00
jschoubben 3ce5a0239b Keep a try off the forge and bound how many run (hq issue 686)
mesh/merge-gate head 3ce5a0239bb1 merged into trunk 2d18f438c516 — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 3ce5a0239bb1 merged into trunk 2d18f438c516 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery delivered
Review of #314: a try's start named its commit, which the forge's holder
matches to an open pull request's head and marks pending. Its start now
names no repository or commit. One try per commit and four in all, a try
never ages ahead of deliveries or pull request checks, and its id is
checked before it is a bus subject.
2026-10-14 02:29:28 +02:00
jochen 70885ab3c4 Share one prefix between the no-move Why's writer and its reader (review of #309, issue 648)
mesh/delivery delivered
mesh/merge-gate head 70885ab3c4a3 merged into trunk ca3f43c400ba — pass: builds mesh-controller → novox; no bus step; every machine composes with the chang…
mesh/repo-check head 70885ab3c4a3 merged into trunk ca3f43c400ba — pass: its merge-check.sh passed
2026-10-14 02:22:25 +02:00
jschoubben 65508d9df9 Try a pushed branch on the build seat before its pull request (hq issue 686)
mesh/merge-gate head 65508d9df9f4 merged into trunk ca3f43c400ba — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 65508d9df9f4 merged into trunk ca3f43c400ba — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery superseded: a newer head of the same pull request
Store tests skip without a store and no agent may start one, so each failure
cost a 15-30 minute review round. The try verb asks the build seat for the
repository's merge-check.sh on the ref merged into the trunk, as a check in
the build queue, and reads its outcome from the build's log: no status on
the forge, nothing registered, nothing built for a merge.
2026-10-14 02:20:37 +02:00
jschoubben 8123ca5654 Test that a branch is tried before its pull request (hq issue 686)
A builder's store tests first ran in the pull request's merge check, a review
round late. These fail on main: a try asks the repository's own check alone,
its outcome sets no status and registers nothing, an unknown ref is refused
before anything is asked, and the answer carries the failing lines.
2026-10-14 02:20:37 +02:00
jochen d9b86376d3 A no-move tier waits for no new report, so its walk asks the next tier (issue 648)
A build made from the source every machine running it runs is marked sent
and sent nothing, yet the tier's wait asked each machine for a report after
that marking. None follows, and the walk waited for ever. Each machine is
now judged by whether it answered its last send, the one whose builds made
the build no move.
2026-10-14 02:18:53 +02:00
mesh-admin ca3f43c400 Merge pull request 'The rest go at once after the first passes, and a gate survives a new controller (hq ADR 0353)' (#312) from fix/the-rest-in-parallel-and-gates-survive-a-new-controller into main 2026-10-13 23:43:02 +00:00
jschoubben b482122d66 Say a read error of a silent rest, and keep the node-engine one at a time in a kept step (hq ADR 0353)
mesh/delivery delivered
mesh/merge-gate head b482122d6646 merged into trunk 1601cb17432e — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head b482122d6646 merged into trunk 1601cb17432e — pass: its merge-check.sh passed
An empty answer on a read error would clear release-silent unseen.
2026-10-14 01:26:00 +02:00
jschoubben 2f5cc146fe Keep a walk's sends to what was kept, and say a silent rest (hq ADR 0353 review)
mesh/merge-gate head 2f5cc146fe46 merged into trunk 1601cb17432e — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
mesh/repo-check head 2f5cc146fe46 merged into trunk 1601cb17432e — fail: its merge-check.sh failed: --- FAIL: TestAFirstMachineThatFailsStillStopsTheWalk (…
mesh/delivery superseded: a newer head of the same pull request
A release's own gates held its other sends back for ever; an adopted send
lost what the machine ran before; a kept send could grow a build no gate
passed; a take-over matched by commit alone; a rest that went silent ended
done. The node-engine and the resolver still go one machine at a time, since
a fault that cuts a machine off shows only as silence.
2026-10-14 01:18:51 +02:00
jschoubben c2ea9b4c51 Send the rest at once after the first passes, and resume a stored gate (hq ADR 0353)
mesh/repo-check head c2ea9b4c5187 merged into trunk 1601cb17432e — fail: its merge-check.sh failed: --- FAIL: TestAReleaseSendsTheRestAtOnceAfterTheFirstPa…
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head c2ea9b4c5187 merged into trunk 1601cb17432e — pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every …
A release now sends every other machine in one step once its first machine
passed, each judged by its own gate; a failing one is put back without
stopping the others, a silent one is left. A send is kept before it is made,
and a machine already sent what a gate is for is not sent again: the gate
counts from that send and takes over another walk's readings of the same
build there.
2026-10-14 01:06:49 +02:00
jschoubben b03d257f81 Test that the rest go at once after the first passes and a gate survives a new controller (hq ADR 0353)
The release walked four machines one after another in eleven minutes, and a
new controller re-sent and re-judged a build its machine already ran.
2026-10-14 01:06:49 +02:00
66 changed files with 2528 additions and 2126 deletions
-6
View File
@@ -1,6 +0,0 @@
# Release 57: config and provisioning
This branch carries the work of hq design 57 (ADR 0348, ADR 0349) in this repository. Each package
merges here through its own reviewed pull request; the branch reaches main as one pull request, in one
delivery group with the branch of the same name in mesh-controller, mesh-host, mesh-sdk, mesh-catalog
and hq, after a rehearsal in the lab. This file is deleted by the last commit before that pull request.
+25 -1
View File
@@ -332,6 +332,12 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
result.Check.Members = append(result.Check.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo,
Commit: m.Head, Trunk: m.Trunk})
}
// A branch tried before its pull request reads its outcome from this log, whole (novox/hq issue 686).
if request.Check.Try {
if line, err := json.Marshal(triedOf(v)); err == nil {
say(link.StepTried, string(line))
}
}
}
} else if err == nil {
// The package-registry credential is a build input, so it is resolved before the clone: a
@@ -457,7 +463,8 @@ func checkSpecOf(request link.BuildRequest) builder.CheckSpec {
spec := builder.CheckSpec{ID: request.ID, Repository: request.Repository, Ref: request.Ref,
Owner: c.Owner, Repo: c.Repo, Number: c.Number, Paths: c.Paths, Beside: map[string]builder.Beside{},
Modules: c.Modules, New: c.New, Manifests: c.Manifests, Judge: c.Judge, Base: c.Base,
Toolchain: builder.ToolchainOf(request.Held), Toolchains: builder.ToolchainsOf(request.Held)}
Toolchain: builder.ToolchainOf(request.Held), Toolchains: builder.ToolchainsOf(request.Held),
Tests: c.Tests, StoreOnly: c.StoreOnly}
for dir, b := range c.Beside {
spec.Beside[dir] = builder.Beside{Repository: b.Repository, Ref: b.Ref}
}
@@ -472,6 +479,23 @@ func checkSpecOf(request link.BuildRequest) builder.CheckSpec {
return spec
}
// triedOf is a try's outcome as its log line says it (novox/hq issue 686): the repository's own check's verdict,
// or the check's where that did not run — a head that does not merge — and the lines naming what failed.
func triedOf(v builder.CheckVerdict) link.Tried {
t := link.Tried{Verdict: v.Verdict, Summary: v.Summary, Took: v.Took.Round(time.Second).String(),
Head: v.Head, Trunk: v.Trunk}
if v.Repo != nil && (v.Gate == nil || v.Gate.Verdict == "pass") {
t.Verdict, t.Summary = v.Repo.Verdict, v.Repo.Summary
}
if v.Repo != nil && v.Repo.Failed != "" {
t.Failed = strings.Split(v.Repo.Failed, "\n")
}
if t.Verdict == "" {
t.Verdict = "error"
}
return t
}
// checkBuiltOf is a check's builds for its merge as the outcome carries them (novox/hq ADR 0350 §1).
func checkBuiltOf(built []builder.CheckBuilt) []link.CheckBuilt {
var out []link.CheckBuilt
+31
View File
@@ -0,0 +1,31 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/builder"
)
// A try's outcome, as the build seat says it in the build's log (novox/hq issue 686): the repository's own
// check's verdict, how long it took, and each line naming what failed.
func TestATrysOutcomeNamesWhatFailed(t *testing.T) {
tried := triedOf(builder.CheckVerdict{Verdict: "pass", Summary: "the change touches no module", Took: 303 * time.Second,
Head: "c0ffee", Trunk: "ca3f43c", Gate: &builder.Layer{Verdict: "pass"},
Repo: &builder.Layer{Verdict: "fail", Summary: "its merge-check.sh failed: TestX",
Failed: "--- FAIL: TestX (0.01s)\n x_test.go:12: got 1, want 2"}})
if tried.Verdict != "fail" || tried.Took != "5m3s" || len(tried.Failed) != 2 || tried.Trunk != "ca3f43c" ||
tried.Summary != "its merge-check.sh failed: TestX" {
t.Fatalf("a failed try: %+v", tried)
}
// A head that does not merge fails both layers: the check's own word.
conflict := triedOf(builder.CheckVerdict{Verdict: "fail", Summary: "does not merge", Gate: &builder.Layer{Verdict: "fail"},
Repo: &builder.Layer{Verdict: "fail", Summary: "does not merge"}})
if conflict.Verdict != "fail" || conflict.Summary != "does not merge" {
t.Fatalf("a try that does not merge: %+v", conflict)
}
// No verdict is an error, never a pass.
if none := triedOf(builder.CheckVerdict{}); none.Verdict != "error" {
t.Fatalf("a try with no verdict: %+v", none)
}
}
+2 -3
View File
@@ -424,9 +424,8 @@ func comparisonOf(module string, c comparison, opts takeOptions) (preview string
}
// And its settings on this machine, composed against its definition (rule 1, rule 6).
for _, layer := range c.layers {
said := layer.Whole()
keys := make([]string, 0, len(said))
for k := range said {
keys := make([]string, 0, len(layer.Values))
for k := range layer.Values {
keys = append(keys, k)
}
sort.Strings(keys)
-5
View File
@@ -55,11 +55,6 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
continue
}
m, err := catalogue.ParseManifest(raw)
if err == nil {
// Each template's text read from the tree beside the manifest, and judged as inline content is
// (novox/hq ADR 0349 §2): a missing template is refused by name.
m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(path))
}
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
+2 -2
View File
@@ -348,11 +348,11 @@ func gatherFacts(ctx context.Context, open *stores, busVersion string) (snapshot
if layer.From == catalogue.MeshWideLayer {
if !meshWide[module] {
meshWide[module] = true
f.Settings = append(f.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Whole())})
f.Settings = append(f.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Values)})
}
continue
}
m.Settings = append(m.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Whole())})
m.Settings = append(m.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Values)})
}
}
m.Declaration = declarationFacts(ctx, open, n.Name, gens, gensErr, scrub)
+12
View File
@@ -1443,6 +1443,12 @@ func probeGates(ctx context.Context, d *doctor) ([]conditions.Observation, error
}
// A release held after a failed one, while builds still wait for a gate (ADR 0236).
out = append(out, backlogObservation()...)
// And a rest that went silent (novox/hq ADR 0353 §4).
silent, err := silentObservation(ctx, d.open.inventory)
if err != nil {
return nil, err
}
out = append(out, silent...)
return sortedFound(dedupeObservations(out)), nil
}
@@ -1532,6 +1538,12 @@ func gateLine(g *inventory.PlanGate) string {
case inventory.GateFailed:
what = "FAILED"
}
switch {
case g.Verdict == "" && g.Pending:
what = "kept, not yet sent" // novox/hq ADR 0353 §8
case g.Verdict == "" && g.Silent:
what = "silent on its send, left" // ADR 0353 §4
}
if g.Verdict == "" && g.Waiting() {
what = "waits for a busy container on " + g.Busy.Machine
}
+1 -1
View File
@@ -151,7 +151,7 @@ func heldMachines(ctx context.Context, open *stores, names []string) (map[string
for _, mv := range f.moves(node, modules, same, known, false) {
if planStillToSend(plans, mv.Module, node) == "" {
why = append(why, fmt.Sprintf("%s would move from %s to %s, which has passed no gate yet — a release "+
"plan sends it, one machine at a time, judged", mv.Module, buildName(mv.From), buildName(mv.To)))
"plan sends it, one machine first and then the rest at once, judged", mv.Module, buildName(mv.From), buildName(mv.To)))
}
}
if len(why) > 0 {
+87
View File
@@ -0,0 +1,87 @@
package main
import (
"encoding/json"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// A tier whose module is no move — made from the source every machine running it runs, so nothing is
// sent — is not waited on for a report after its marking (novox/hq issue 648). Its machines answered the
// send that last carried that build before the walk got there, and nothing will make them report again:
// the walk asks the next tier in the same step, rather than "waiting for app on anchor, laptop" for ever.
func TestANoMoveTierWaitsForNoNewReport(t *testing.T) {
judgeLeavesAsTheCore(t)
g := aGateMesh(t)
ctx := t.Context()
inv := g.open.inventory
// app is the build machine, so a module of the next tier is built by it: the tier gates the next on app
// running on its machines.
for _, m := range []catalogue.Manifest{
{Module: "app", Version: "c2", Claims: []catalogue.Claim{{Name: "node-build-agent", Scope: catalogue.ScopeNode}}},
{Module: "web", Version: "w1"},
} {
if err := inv.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-catalog", Seat: "git",
Path: "modules/" + m.Module, BuiltFrom: m.Version, Head: m.Version, Asked: time.Now().Add(-time.Minute)}); err != nil {
t.Fatal(err)
}
}
// What anchor and laptop were last sent (c1) was made from the source the plan's build of c2 is.
for _, b := range []inventory.Build{
{ID: "build-1s", Module: "app", Commit: "c1", SourceFingerprint: "src1:app",
Asked: time.Now().Add(-30 * time.Second), At: time.Now().Add(-30 * time.Second)},
{ID: "build-2s", Module: "app", Commit: "c2", SourceFingerprint: "src1:app", Asked: time.Now(), At: time.Now()},
} {
b.Manifest, _ = json.Marshal(catalogue.Manifest{Module: "app", Version: "1"})
if err := inv.RecordBuild(ctx, b); err != nil {
t.Fatal(err)
}
}
// Each machine answered that send, applied, before the walk's tier was built.
for _, n := range []string{"anchor", "laptop"} {
if _, err := inv.RecordDoing(ctx, nodeID(t, g.open, n), inventory.Doing{Node: n,
Outcome: inventory.OutcomeApplied, Declared: "d-" + n + "-c1", Applied: 1,
At: time.Now().Add(-10 * time.Second)}); err != nil {
t.Fatal(err)
}
}
built := time.Now().UTC()
p := g.plan(t)
p.Tiers = [][]string{{"app"}, {"web"}}
p.Modules["app"].Build, p.Modules["app"].BuiltAt = "build-2s", &built
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
advancePlans(ctx, g.open)
if len(g.sent) != 0 {
t.Fatalf("a no-move build was sent: %v", g.sent)
}
p = g.plan(t)
if s := p.Modules["app"]; s == nil || !strings.HasPrefix(s.Why, "no move") {
t.Fatalf("the build was not marked no move: %+v", s)
}
if p.Tier < 1 || strings.Contains(p.Note, "waiting for app") {
t.Fatalf("the walk waits on a no-move tier for a report nothing will cause: tier %d, %s %q", p.Tier, p.State, p.Note)
}
// A machine that has not answered its last send is still waited on: no move is not "never sent".
p.Tier, p.State, p.Note = 0, inventory.PlanRolling, ""
if err := inv.RecordSent(ctx, nodeID(t, g.open, "laptop"), "d-laptop-later", map[string]string{"app": "c1"}); err != nil {
t.Fatal(err)
}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
advancePlans(ctx, g.open)
p = g.plan(t)
if p.Tier != 0 || !strings.Contains(p.Note, "waiting for app on laptop") {
t.Fatalf("a machine that has not answered its last send was not waited on: tier %d, %q", p.Tier, p.Note)
}
}
+9
View File
@@ -106,6 +106,8 @@ func run() error {
return killCommand(ctx, args[1:])
case "forget-checks":
return forgetChecksCommand(ctx, args[1:])
case "try":
return tryCommand(ctx, args[1:])
case "pause", "resume":
return pauseCommand(ctx, args[0], args[1:])
case "collection":
@@ -381,6 +383,13 @@ func (b builds) Built(ctx context.Context, result link.BuildResult) error {
// **A merge check builds nothing** (novox/hq to-be 45 §9): its verdict is said, and nothing of it is
// recorded or registered.
if result.Check != nil || result.Checked != nil {
if isTry(result) {
// A branch tried before its pull request (novox/hq issue 686): its asker reads the outcome from the
// build's log. Never said as `checked`, so the forge's holder sets no status.
fmt.Printf("%s: a try of %s at %.8s, read with `try --id`: said to nobody, nothing taken in\n", result.ID,
result.Repository, result.Ref)
return nil
}
checked(ctx, result)
return nil
}
-5
View File
@@ -576,11 +576,6 @@ func shelfWithChange(base map[string]catalogue.Manifest, sources map[string]inve
return err
}
m, err := catalogue.ParseManifest(raw)
if err == nil {
// Each template's text from the tree beside the manifest, as the build reads it back from the archive it
// packs (novox/hq ADR 0349 §2): so a change to a template alone is a change to its module.
m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(path))
}
if err != nil {
if inScope(dir) {
failures = append(failures, fmt.Sprintf("%s: the controller judging this (%s) refuses it — %s",
+1 -15
View File
@@ -33,21 +33,7 @@ import (
func aMesh(t *testing.T) *stores {
t.Helper()
inventory.ForTest(t) // raises the store, migrates it, and points the environment at it
return aMeshOnTheStore(t)
}
// aMeshBefore is aMesh on a store migrated only below migration `number`, with the function that runs the
// rest: for a test of what a migration that moves data does to what the mesh composes (novox/hq issue 651).
func aMeshBefore(t *testing.T, number int) (*stores, func()) {
t.Helper()
_, migrate := inventory.ForTestBefore(t, number)
return aMeshOnTheStore(t), migrate
}
// aMeshOnTheStore is the mesh aMesh makes, on the store the environment points at.
func aMeshOnTheStore(t *testing.T) *stores {
t.Helper()
licences.ForTest(t) // planning reaches this one too, by name and never by connection
licences.ForTest(t) // planning reaches this one too, by name and never by connection
open, err := openStores(t.Context())
if err != nil {
-9
View File
@@ -8,7 +8,6 @@ import (
"fmt"
"net"
"os"
"path/filepath"
"sort"
"strings"
"time"
@@ -128,10 +127,6 @@ func moduleCommand(ctx context.Context, args []string) error {
if err != nil {
return err
}
// A manifest handed over by hand takes each template's text from the tree beside it (novox/hq ADR 0349 §2).
if m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(positionals[0])); err != nil {
return err
}
from, err := whereItComesFrom(*repo, *ref, *commit, *path, *self)
if err != nil {
return err
@@ -542,10 +537,6 @@ func settingsCommand(ctx context.Context, args []string) error {
if p.SetBy != "" {
by = "; " + p.SetBy
}
if p.Why != "" {
// A layer whose placement moved out of it (migration 0097): still the layer, as it stands.
by += "; " + p.Why
}
fmt.Printf("%s on %s, until %s (%s%s):\n %s\n", positionals[0], where,
p.ReplacedAt.Local().Format("2006-01-02 15:04:05"), p.ReplacedBy, by, shown)
}
-113
View File
@@ -1,113 +0,0 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// **Placement leaves the settings layers and no machine is told anything new** (novox/hq ADR 0348 clause
// 11, issue 651). A mesh is seeded as the store holds it before migration 0097 — `places`, `accesses` and a
// kept network among the settings, at the mesh's level and a machine's, one layer of placement only — then
// migrated, and every machine's declaration is composed. What each machine is told of the seeded modules is
// byte for byte what the controller before this change composed from the same seed: the golden files were
// written by the integration branch's controller at bfc79c6 running this same test (MESH_WRITE_GOLDEN=1),
// where there was no migration 0097 to run.
func TestADeclarationIsTheSameAfterPlacementMoves(t *testing.T) {
open, migrate := aMeshBefore(t, 97)
ctx := t.Context()
inv := open.inventory
image := "registry.invalid:5000/library@sha256:" + strings.Repeat("a", 64)
register(t, open, catalogue.Manifest{Module: "library", Version: "1",
Accesses: []catalogue.Access{{ID: "series"}, {ID: "downloads", Mode: catalogue.AccessRead}},
Resources: []map[string]any{
{"id": "config", "type": "directory", "mode": "0750"},
{"id": "server", "type": "container", "image": image,
"volumes": []any{"${dir:config}:/config", "${access:series}:/series", "${access:downloads}:/downloads:ro"}},
// Today's form: a mergeable file naming no keys takes every key of the layer (ADR 0326 §3).
{"id": "runtime-config", "type": "file", "path": "${dir:config}/runtime.json", "mode": "0640",
"content": `{"log":"info"}`, "merge": "json"},
}})
register(t, open, catalogue.Manifest{Module: "found", Version: "1",
Resources: []map[string]any{{"id": "server", "type": "container", "image": image}}})
if err := inv.SetAdopted(ctx, "anchor", true); err != nil {
t.Fatal(err)
}
for _, l := range []struct{ node, module, values string }{
{"", "library", `{"accesses": {"downloads": "/storage/downloads"}, "log": "warn"}`},
{"anchor", "library", `{"places": {"config": {"path": "/var/lib/library/config", "owner": "1001:2000"}},
"accesses": {"series": "/storage/media/series"}, "workers": 4}`},
{"laptop", "library", `{"accesses": {"series": "/home/media/series"}}`},
{"anchor", "found", `{"networks": {"server": ["predecessor_default"]}}`},
} {
if l.node == "" {
inv.ExecForTest(t, `insert into settings (node, module, values) values (null, $1, $2)`, l.module, l.values)
continue
}
inv.ExecForTest(t, `insert into settings (node, module, values) select id, $2, $3 from node where name = $1`,
l.node, l.module, l.values)
}
migrate()
// Assigned once the store is migrated: an assignment reads the layers, as this controller holds them.
for _, a := range [][]string{{"anchor", "library", "found"}, {"laptop", "library"}} {
if _, err := assign(ctx, open, a[0], a[1:]...); err != nil {
t.Fatalf("assign %v: %v", a, err)
}
}
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
for _, node := range []string{"anchor", "laptop"} {
plan, settings, err := planFor(ctx, open, node)
if err != nil {
t.Fatal(err)
}
with, _, err := renderingFor(ctx, open, node, plan, settings, gens, Allocating)
if err != nil {
t.Fatal(err)
}
composed, err := plan.Compose(with)
if err != nil {
t.Fatal(err)
}
if len(composed.LeftOut) > 0 {
t.Fatalf("%s leaves out %v", node, composed.LeftOut)
}
var told []map[string]any
for _, r := range composed.Resources {
if owner := composed.Owner[r["id"].(string)]; owner == "library" || owner == "found" {
told = append(told, r)
}
}
sort.Slice(told, func(i, j int) bool { return told[i]["id"].(string) < told[j]["id"].(string) })
got, err := json.MarshalIndent(told, "", " ")
if err != nil {
t.Fatal(err)
}
golden := filepath.Join("testdata", "placement-moved", node+".json")
if os.Getenv("MESH_WRITE_GOLDEN") != "" {
if err := os.MkdirAll(filepath.Dir(golden), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(golden, append(got, '\n'), 0o644); err != nil {
t.Fatal(err)
}
continue
}
want, err := os.ReadFile(golden)
if err != nil {
t.Fatal(err)
}
if string(want) != string(got)+"\n" {
t.Errorf("%s is told something else after placement moved:\n%s\nwas:\n%s", node, got, want)
}
}
}
+16
View File
@@ -456,6 +456,9 @@ var plainWordings = map[string]func(conditions.Observation) words{
"release-held": worded(func(o conditions.Observation) words {
return releaseHeldWords(nil, o.Also)
}),
"release-silent": worded(func(o conditions.Observation) words {
return releaseSilentWords(o.Also)
}),
"facts-stale": worded(func(o conditions.Observation) words {
return words{Headline: "Merge checks use outdated facts",
Explanation: "The facts a merge check judges a change against are out of date; they are gathered again " +
@@ -841,6 +844,19 @@ func releaseHeldWords(modules, machines []string) words {
Resolved: "Resolved: the held updates are released"}
}
// releaseSilentWords are the plain words of machines that said nothing on an update they were sent (ADR 0353 §4).
func releaseSilentWords(machines []string) words {
where := "Some machines"
if len(machines) > 0 {
where = conditions.Capital(namesWords(machines, 4))
}
return words{Headline: "Machines did not answer an update",
Needs: "check that they are awake and on the mesh; the next walk sends them what waits.",
Explanation: where + " were sent module updates and said nothing about them within fifteen minutes, so " +
"they were left as they were. What they run is not known to be the update.",
Resolved: "Resolved: the next walk reached them"}
}
// reloginNeeds is what an account waiting for its groups needs (ADR 0252).
func reloginNeeds(node string) string {
return fmt.Sprintf("log out of %s completely and log in again, or restart it.", node)
+537 -42
View File
@@ -39,8 +39,8 @@ import (
// from the same manifest is no move.
//
// **The release plan** is what walks the waiting moves through: whenever moves wait for a gate that no
// started plan is walking, the mesh opens one — every such machine, one at a time, the control node last,
// each sent and judged before the next. A release plan that fails stops the next one opening on its own:
// started plan is walking, the mesh opens one — every such machine, the first sent and judged, then the rest
// at once, each judged by its own gate (novox/hq ADR 0353, in place of one at a time). A release plan that fails stops the next one opening on its own:
// a person releases it again (`upgrade release-backlog --why`), after the condition says why.
// sendScope is what a send may carry that no gate has seen.
@@ -81,6 +81,15 @@ type moveFacts struct {
srcs map[string]map[string]string
passed map[string]map[string]bool
plans []inventory.Plan
// self is the release walking these moves, whose own gates are no other walk's (novox/hq ADR 0353).
self string
}
type walkingKey struct{}
// withWalking says a release plan is the one sending: its own gates never hold its own sends back.
func withWalking(ctx context.Context, id string) context.Context {
return context.WithValue(ctx, walkingKey{}, id)
}
func readMoveFacts(ctx context.Context, inv *inventory.Inventory) (moveFacts, error) {
@@ -99,6 +108,7 @@ func readMoveFacts(ctx context.Context, inv *inventory.Inventory) (moveFacts, er
return f, err
}
f.plans, err = inv.OpenPlans(ctx)
f.self, _ = ctx.Value(walkingKey{}).(string)
return f, err
}
@@ -193,17 +203,24 @@ func (f moveFacts) moves(node string, modules []string, sent map[string]string,
// feet. A release keeps no module records: its open gate's carried moves are its walk.
func (f moveFacts) walkedBy(module, node, to string) string {
for _, p := range f.plans {
if !p.Open() {
if !p.Open() || (p.Release != nil && p.ID == f.self) {
continue
}
if p.Release != nil {
g := p.Release.Gate
if g == nil || g.Verdict != "" {
continue
// The first machine's gate, and every gate of the rest still judging (novox/hq ADR 0353).
gates := []*inventory.PlanGate{p.Release.Gate}
for _, g := range p.Release.Gates {
gates = append(gates, g)
}
for _, c := range g.Carried {
if c.Module == module && !(slices.Contains(g.Machines, node) && sameCommit(c.To, to)) {
return p.ID
for _, g := range gates {
// A send kept and not yet made walks nothing yet (ADR 0353 §8).
if g == nil || g.Verdict != "" || g.Silent || g.Pending {
continue
}
for _, c := range g.Carried {
if c.Module == module && !(slices.Contains(g.Machines, node) && sameCommit(c.To, to)) {
return p.ID
}
}
}
continue
@@ -303,8 +320,8 @@ func ungatedIn(ctx context.Context, open *stores, names []string, addedHolder st
}
}
if len(refused) > 0 {
return nil, fmt.Errorf("%w: %s — a walk sends them, one machine at a time, each judged (novox/hq ADR "+
"0236); `upgrade backlog` lists them", errUngated, strings.Join(refused, "; "))
return nil, fmt.Errorf("%w: %s — a walk sends them, one machine first and then the rest at once, each judged (novox/hq ADR "+
"0236, ADR 0353); `upgrade backlog` lists them", errUngated, strings.Join(refused, "; "))
}
return kept, nil
}
@@ -320,19 +337,35 @@ func ungatedIn(ctx context.Context, open *stores, names []string, addedHolder st
// And the grants step first, when the send changes what the bus's user list must say (novox/hq issue 490):
// answered for the gate to keep, nil when there was none.
func gatedSend(ctx context.Context, open *stores, node string, owns []inventory.CarriedMove) ([]inventory.CarriedMove, []string, *inventory.GrantsStep, error) {
inv := open.inventory
f, err := readMoveFacts(ctx, inv)
moves, err := gatedMoves(ctx, open, node, owns)
if err != nil || len(moves) == 0 {
return nil, nil, nil, err
}
sayRecreations(ctx, open, moves)
sent, grants, err := sendJudged(ctx, open, node)
if err != nil {
return nil, nil, nil, err
}
return moves, sent, grants, nil
}
// gatedMoves is what a gated send to one machine would carry, with the build each moves to, made of nothing:
// gatedSend's reading, which a walk also keeps before it sends (novox/hq ADR 0353 §8). Refused the same way
// while a plan that has started walks one of those builds elsewhere.
func gatedMoves(ctx context.Context, open *stores, node string, owns []inventory.CarriedMove) ([]inventory.CarriedMove, error) {
inv := open.inventory
f, err := readMoveFacts(ctx, inv)
if err != nil {
return nil, err
}
moves, err := machineMoves(ctx, open, f, node, true)
if err != nil {
return nil, nil, nil, err
return nil, err
}
// A move the send keeps is not made, and is no move for the gate to judge (novox/hq issue 201): a core module
// kept over an older record judged as moving back to it would read the gate as superseded by its own send.
if moves, err = withoutKept(ctx, open, node, moves); err != nil {
return nil, nil, nil, err
return nil, err
}
own := func(module string) bool {
return slices.ContainsFunc(owns, func(o inventory.CarriedMove) bool { return o.Module == module })
@@ -342,7 +375,7 @@ func gatedSend(ctx context.Context, open *stores, node string, owns []inventory.
continue
}
if id := f.walkedBy(mv.Module, node, mv.To); id != "" {
return nil, nil, nil, fmt.Errorf("%w: %s's build %s waits on %s, which %s is walking", errWalkedElsewhere,
return nil, fmt.Errorf("%w: %s's build %s waits on %s, which %s is walking", errWalkedElsewhere,
mv.Module, short(mv.To), node, id)
}
}
@@ -350,7 +383,7 @@ func gatedSend(ctx context.Context, open *stores, node string, owns []inventory.
// plan, or a release, on this machine is not crossed with a newer build; this send waits for its gate.
for _, o := range owns {
if id := f.walkedBy(o.Module, node, o.To); id != "" {
return nil, nil, nil, fmt.Errorf("%w: %s's build %s waits on %s, which %s is walking", errWalkedElsewhere,
return nil, fmt.Errorf("%w: %s's build %s waits on %s, which %s is walking", errWalkedElsewhere,
o.Module, short(o.To), node, id)
}
}
@@ -363,30 +396,26 @@ func gatedSend(ctx context.Context, open *stores, node string, owns []inventory.
moves[i].Build = o.Build
}
}
if len(moves) == 0 && len(owns) == 0 {
return nil, nil, nil, nil
if len(moves) == 0 {
return nil, nil
}
for i := range moves {
if moves[i].Build == "" {
if moves[i].Build, err = inv.BuildOf(ctx, moves[i].Module, moves[i].To); err != nil {
return nil, nil, nil, err
return nil, err
}
}
}
sort.Slice(moves, func(i, j int) bool { return moves[i].Module < moves[j].Module })
sayRecreations(ctx, open, moves)
sent, grants, err := sendJudged(ctx, open, node)
if err != nil {
return nil, nil, nil, err
}
return moves, sent, grants, nil
return moves, nil
}
// passCarried keeps a pass as the verdict of every build the gate judged beside its own module.
func passCarried(ctx context.Context, open *stores, p *inventory.Plan, g *inventory.PlanGate, except string) {
seen := map[string]bool{}
for _, c := range g.Carried {
if c.Module == except || c.Build == "" || seen[c.Build] {
// A build put back with another machine's failure is not this gate's to pass (novox/hq ADR 0353 §3).
if c.Module == except || c.Build == "" || seen[c.Build] || slices.Contains(g.Returned, c.Module) {
continue
}
seen[c.Build] = true
@@ -695,7 +724,7 @@ func releaseBacklog(ctx context.Context, open *stores, by string) (*inventory.Pl
p := inventory.Plan{ID: fmt.Sprintf("release-%d", now.UnixNano()), Repository: releaseRepository,
Created: now, State: inventory.PlanRolling, Tiers: [][]string{names}, Modules: map[string]*inventory.PlanModule{},
Release: &inventory.PlanRelease{Order: order, By: by},
Note: fmt.Sprintf("%d build(s) wait for a gate on %s; one machine at a time, each judged", len(names),
Note: fmt.Sprintf("%d build(s) wait for a gate on %s; one machine first, then the rest at once, each judged", len(names),
strings.Join(order, ", "))}
if err := inv.SavePlan(ctx, &p); err != nil {
return nil, err
@@ -704,14 +733,23 @@ func releaseBacklog(ctx context.Context, open *stores, by string) (*inventory.Pl
return &p, nil
}
// advanceRelease takes one step of a release plan: the next machine sent everything waiting there under a
// gate, or the machine being judged judged once more; the plan done when every machine is.
// advanceRelease takes one step of a release plan: the first machine sent everything waiting there under a
// gate and judged; once it passed, every other machine sent at once, each under its own gate (novox/hq ADR
// 0353 §1), and judged; the plan done when every machine is.
//
// **A send is kept before it is made** (ADR 0353 §8): the step that chooses a machine keeps its gate pending with
// what it will carry, and the next step makes the send. A controller replaced between the send and keeping it
// finds the gate pending, reads the machine's last send, and neither sends again nor leaves the machine unjudged.
func advanceRelease(ctx context.Context, open *stores, p *inventory.Plan) (bool, error) {
r := p.Release
if r.Settings != nil {
return advanceSettingsWalk(ctx, open, p)
}
ctx = withWalking(ctx, p.ID)
now := time.Now().UTC()
if r.Rest {
return advanceRest(ctx, open, p, now)
}
if r.Gate == nil {
heard, err := releaseHeard(ctx, open)
if err != nil {
@@ -724,7 +762,7 @@ func advanceRelease(ctx context.Context, open *stores, p *inventory.Plan) (bool,
r.Next++
continue
}
moves, sent, grants, err := gatedSend(ctx, open, node, nil)
moves, err := gatedMoves(ctx, open, node, nil)
if errors.Is(err, errWalkedElsewhere) {
note := fmt.Sprintf("waiting before %s: %v", node, err)
changed := p.Note != note
@@ -732,23 +770,15 @@ func advanceRelease(ctx context.Context, open *stores, p *inventory.Plan) (bool,
return changed, nil
}
if err != nil {
return false, fmt.Errorf("sending %s what waits there: %w", node, err)
return false, fmt.Errorf("reading what waits on %s: %w", node, err)
}
if len(moves) == 0 {
r.Done = append(r.Done, node)
r.Next++
continue
}
r.Gate = &inventory.PlanGate{Machines: sent, Since: &now, Carried: moves, Sent: sentNow(ctx, open.inventory, sent),
Grants: grants}
p.Note = fmt.Sprintf("sent %s %d build(s) that waited for a gate; judging them there", node, len(moves))
if said := grantsSaid(grants); said != "" {
p.Note += "; " + said
}
if said := recreationsSaid(moves); said != "" {
p.Note += "; " + said
}
fmt.Printf("%s: %s\n", p.ID, p.Note)
r.Gate = &inventory.PlanGate{Machines: []string{node}, Carried: moves, Pending: true}
p.Note = fmt.Sprintf("sending %s %d build(s) that wait for a gate, first", node, len(moves))
return true, nil
}
p.State, p.Tier = inventory.PlanDone, len(p.Tiers)
@@ -759,6 +789,19 @@ func advanceRelease(ctx context.Context, open *stores, p *inventory.Plan) (bool,
return true, nil
}
g := r.Gate
if g.Pending {
node := firstOf(g.Machines)
out, err := sendKept(ctx, open, p, node, g, now)
if err != nil || out == keptWaiting {
return out == keptWaiting, err
}
if out == keptNothing {
r.Gate = nil
r.Done = append(r.Done, node)
r.Next++
}
return true, nil
}
verdict, err := judgeMoves(ctx, open, g, nil, now)
if err != nil {
return false, err
@@ -777,6 +820,11 @@ func advanceRelease(ctx context.Context, open *stores, p *inventory.Plan) (bool,
r.Done = append(r.Done, firstOf(g.Machines))
r.Next++
r.Gate = nil
// **Then the rest at once** (novox/hq ADR 0353 §1): every other machine heard from, each judged by its own
// gate, in place of one machine at a time (ADR 0236 §22).
r.Rest = true
p.Note = fmt.Sprintf("passed on %s; the rest are sent at once, each judged", firstOf(g.Machines))
fmt.Printf("%s: %s\n", p.ID, p.Note)
return true, nil
case inventory.GateSuperseded:
// Another send moved a judged module on the judged machine (novox/hq issue 352): no verdict on what
@@ -797,6 +845,453 @@ func advanceRelease(ctx context.Context, open *stores, p *inventory.Plan) (bool,
return true, nil
}
// What sendKept did with a kept send.
const (
keptSent = iota // sent now
keptAlready // the machine was already sent it: judged from that send, nothing sent
keptNothing // nothing waits there any more: no gate
keptWaiting // another walk judges a build it would carry: it waits
)
// sendKept makes a send a gate kept before it was made (novox/hq ADR 0353 §8) — unless the machine's last send
// already carries what the gate is for, made before a controller was replaced and before it was kept: then
// nothing is sent, and the gate judges from that send (§6).
func sendKept(ctx context.Context, open *stores, p *inventory.Plan, node string, g *inventory.PlanGate,
now time.Time) (int, error) {
inv := open.inventory
moves, sent, grants, err := gatedSend(ctx, open, node, nil)
if errors.Is(err, errWalkedElsewhere) {
p.Note = fmt.Sprintf("waiting before %s: %v", node, err)
return keptWaiting, nil
}
if err != nil {
return keptSent, fmt.Errorf("sending %s what waits there: %w", node, err)
}
if len(moves) > 0 {
g.Machines, g.Carried, g.Since, g.Grants, g.Pending = sent, moves, &now, grants, false
g.Sent = sentNow(ctx, inv, sent)
p.Note = fmt.Sprintf("sent %s %d build(s) that waited for a gate; judging them there", node, len(moves))
if said := grantsSaid(grants); said != "" {
p.Note += "; " + said
}
if said := recreationsSaid(moves); said != "" {
p.Note += "; " + said
}
fmt.Printf("%s: %s\n", p.ID, p.Note)
return keptSent, nil
}
if !alreadySent(ctx, inv, node, g.Carried) {
return keptNothing, nil
}
since := now
if s, found, err := inv.SentTo(ctx, node); err == nil && found {
if !s.At.IsZero() {
since = s.At
}
g.Sent = map[string]inventory.SentDeclaration{node: s}
}
g.Since, g.Pending = &since, false
p.Note = fmt.Sprintf("%s was already sent the %d build(s) kept for it, at %s: judged from that send, nothing sent again",
node, len(g.Carried), since.Local().Format("15:04:05"))
fmt.Printf("%s: %s\n", p.ID, p.Note)
return keptAlready, nil
}
// alreadySent says a machine's last send carries every build of the moves: a send made and not kept.
func alreadySent(ctx context.Context, inv *inventory.Inventory, node string, moves []inventory.CarriedMove) bool {
if len(moves) == 0 {
return false
}
sent, known, err := inv.SentBuilds(ctx, node)
if err != nil || !known {
return false
}
for _, mv := range moves {
if !sameCommit(sent[mv.Module], mv.To) {
return false
}
}
return true
}
// advanceRest takes one step of a release plan's rest (novox/hq ADR 0353): the sends kept for a step made, every
// machine of the step judged by its own gate, one that fails put back by itself while the others go on to their
// verdicts, and the next step — or the end — once every gate of the step has its verdict or is silent.
func advanceRest(ctx context.Context, open *stores, p *inventory.Plan, now time.Time) (bool, error) {
r := p.Release
// The sends kept for this step, in the plan's order: the control node, last in it, last (§1).
var pending []string
for _, n := range r.Order {
if g := r.Gates[n]; g != nil && g.Pending {
pending = append(pending, n)
}
}
if len(pending) > 0 {
// **A kept send carries only what was kept** (ADR 0353 §2): a build that passed no gate and came to wait on
// a machine since its step was chosen could reach a second machine in it; the step is chosen again.
if grew, err := keptStepGrew(ctx, open, r, pending); err != nil {
return false, err
} else if grew != "" {
for _, n := range pending {
delete(r.Gates, n)
}
opened, waiting, err := openRestStep(ctx, open, p)
if err != nil {
return true, err
}
p.Note = "chosen again: " + grew
if !opened && waiting != "" {
p.Note += "; " + waiting
}
return true, nil
}
var sent, already, waiting []string
for _, node := range pending {
out, err := sendKept(ctx, open, p, node, r.Gates[node], now)
if err != nil {
return true, err
}
switch out {
case keptSent:
sent = append(sent, node)
case keptAlready:
already = append(already, node)
case keptNothing:
delete(r.Gates, node)
r.Done = append(r.Done, node)
case keptWaiting:
waiting = append(waiting, node)
}
}
var said []string
if len(sent) > 0 {
said = append(said, "sent "+strings.Join(sent, ", ")+" at once, each judged there")
}
if len(already) > 0 {
said = append(said, strings.Join(already, ", ")+" already sent it: judged from that send")
}
if len(waiting) > 0 {
said = append(said, "waiting before "+strings.Join(waiting, ", ")+": another walk judges a build it carries")
}
if len(said) > 0 {
p.Note = strings.Join(said, "; ")
}
return len(sent)+len(already) > 0 || len(waiting) < len(pending), nil
}
reports, err := open.inventory.LastReports(ctx)
if err != nil {
return false, err
}
byNode := map[string]inventory.Reported{}
for _, rep := range reports {
byNode[rep.Node] = rep
}
changed, noteWas := false, p.Note
var notes, judging []string
batched, back := batchingRollbacks(ctx)
for _, node := range r.Order {
g := r.Gates[node]
if g == nil || g.Verdict != "" || g.Silent {
continue
}
// What another machine's failure already put back is not this gate's to judge (ADR 0353 §3).
if len(g.Carried) > 0 && len(g.Returned) >= len(g.Carried) {
decide(g, inventory.GateFailed, "what it was sent was put back with another machine's failure", now)
r.Failed = append(r.Failed, node)
changed = true
continue
}
// **A machine that says nothing on its send is left silent, not failed** (ADR 0353 §4): it is not judged
// until it reports, and past ApplySilentAfter it is listed and left.
if !reportedOnSend(byNode[node], node, g) {
if now.Sub(*g.Since) > inventory.ApplySilentAfter {
g.Silent = true
r.Silent = append(r.Silent, node)
changed = true
notes = append(notes, fmt.Sprintf("%s said nothing on its send within %s: left silent", node,
inventory.ApplySilentAfter))
continue
}
judging = append(judging, node+": not yet reported on its send")
continue
}
verdict, err := judgeMoves(ctx, open, g, nil, now)
if err != nil {
return changed, err
}
switch verdict {
case "":
if putBackBroken(ctx, open, p, g, "", nil) {
changed = true
notes = append(notes, p.Note)
}
judging = append(judging, node+": "+gateLine(g))
case inventory.GatePassed:
passCarried(ctx, open, p, g, "")
r.Done = append(r.Done, node)
changed = true
case inventory.GateSuperseded:
// Another send moved a judged module there (novox/hq issue 352): nothing judged, nothing put back on
// it; what still waits is released again.
r.Done = append(r.Done, node)
changed = true
notes = append(notes, fmt.Sprintf("superseded on %s: %s", node, g.Why))
default:
// **Put back by itself** (ADR 0353 §3): what was found wanting goes back at once, and every other
// machine of the step goes on to its own verdict.
stateWas := p.State
p.Note = ""
failCarried(batched, open, p, g, "")
p.State = stateWas
r.Failed = append(r.Failed, node)
changed = true
notes = append(notes, fmt.Sprintf("failed its gate on %s: %s — %s", node, g.Why, p.Note))
returnedElsewhere(ctx, open, r, node, g)
}
}
p.Note = ""
sendRollbacks(ctx, open, p, back)
if p.Note != "" {
notes = append(notes, strings.TrimPrefix(p.Note, "; "))
}
if len(judging) > 0 {
note := strings.Join(append(notes, "judging "+strings.Join(judging, "; ")), "; ")
p.Note = note
return changed || noteWas != note, nil
}
// Every gate of the step has its verdict or is silent: the next step, unless a machine failed (§3).
if len(r.Failed) == 0 {
opened, waiting, err := openRestStep(ctx, open, p)
if err != nil {
return changed, err
}
if opened {
return true, nil
}
if waiting != "" {
note := strings.Join(append(notes, waiting), "; ")
p.Note = note
return changed || noteWas != note, nil
}
}
p.Tier = len(p.Tiers)
end := fmt.Sprintf("released on %s", orNone(strings.Join(r.Done, ", ")))
if len(r.Silent) > 0 {
end += "; silent on their send, left: " + strings.Join(r.Silent, ", ")
}
if len(r.Skipped) > 0 {
end += "; not heard from, left as they were: " + strings.Join(r.Skipped, ", ")
}
p.State = inventory.PlanDone
if len(r.Failed) > 0 {
p.State = inventory.PlanFailed
end = "failed on " + strings.Join(r.Failed, ", ") + "; " + end
} else if len(r.Silent) > 0 && len(r.Gates) == len(r.Silent) {
// **A rest that said nothing is no success** (ADR 0353 §4): failure is loud; the condition says it.
p.State = inventory.PlanFailed
end = "no machine of the rest reported on its send; " + end
}
p.Note = strings.Join(append(notes, end), "; ")
fmt.Printf("%s: %s\n", p.ID, p.Note)
return true, nil
}
// keptStepGrew says why a step's kept sends would now carry a build no gate passed that they were not kept with;
// empty when none would.
func keptStepGrew(ctx context.Context, open *stores, r *inventory.PlanRelease, pending []string) (string, error) {
f, err := readMoveFacts(ctx, open.inventory)
if err != nil {
return "", err
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return "", err
}
for _, node := range pending {
moves, err := gatedMoves(ctx, open, node, nil)
if errors.Is(err, errWalkedElsewhere) {
continue // sendKept says it waits
}
if err != nil {
return "", err
}
kept := r.Gates[node].Carried
for _, mv := range moves {
// The node-engine and the resolver keep one machine at a time, passed or not (ADR 0353 §9).
if (f.gated(mv.Module, mv.To) && !oneAtATime(mv.Module, shelf[mv.Module])) || slices.ContainsFunc(kept, func(c inventory.CarriedMove) bool {
return c.Module == mv.Module && sameCommit(c.To, mv.To)
}) {
continue
}
return fmt.Sprintf("%s's build %s, which this step did not keep for it, came to wait on %s", mv.Module, short(mv.To), node), nil
}
}
return "", nil
}
// reportedOnSend says a machine's last report is about the send its gate judges (novox/hq issue 352): the
// declaration kept on the gate, or one sequenced after it; a gate that kept none reads the send made last.
func reportedOnSend(r inventory.Reported, node string, g *inventory.PlanGate) bool {
if r.At == nil {
return false
}
if s, kept := g.Sent[node]; kept {
return s.ReportsOn(r)
}
return r.Current
}
// returnedElsewhere marks, on every other gate of the rest still judging, the builds a failed machine's
// rollback put back wherever they were sent (ADR 0236 §5): no longer theirs to judge (ADR 0353 §3).
func returnedElsewhere(ctx context.Context, open *stores, r *inventory.PlanRelease, failed string, g *inventory.PlanGate) {
for _, c := range g.Carried {
if c.Build == "" {
continue
}
if marked, err := open.inventory.GateFailed(ctx, c.Build); err != nil || !marked {
continue
}
for node, other := range r.Gates {
if node == failed || other.Verdict != "" || slices.Contains(other.Returned, c.Module) {
continue
}
for _, oc := range other.Carried {
if oc.Module == c.Module && sameCommit(oc.To, c.To) {
other.Returned = append(other.Returned, c.Module)
}
}
}
}
}
// openRestStep chooses the next step of a release plan's rest and keeps its sends (novox/hq ADR 0353 §1, §2):
// every machine left that is heard from and has something waiting, except that a build no gate has passed goes
// to one machine of the step only. Answers whether a step was kept, or why the walk waits.
func openRestStep(ctx context.Context, open *stores, p *inventory.Plan) (bool, string, error) {
r := p.Release
heard, err := releaseHeard(ctx, open)
if err != nil {
return false, "", err
}
f, err := readMoveFacts(ctx, open.inventory)
if err != nil {
return false, "", err
}
waiting := map[string][]inventory.CarriedMove{}
var nodes, walked []string
for _, node := range r.Order[min(r.Next, len(r.Order)):] {
if r.Gates[node] != nil || slices.Contains(r.Done, node) || slices.Contains(r.Skipped, node) ||
slices.Contains(r.Failed, node) || slices.Contains(r.Silent, node) {
continue
}
if !heard[node] {
r.Skipped = append(r.Skipped, node)
continue
}
moves, err := gatedMoves(ctx, open, node, nil)
if errors.Is(err, errWalkedElsewhere) {
walked = append(walked, fmt.Sprintf("%s (%v)", node, err))
continue
}
if err != nil {
return false, "", fmt.Errorf("reading what waits on %s: %w", node, err)
}
if len(moves) == 0 {
r.Done = append(r.Done, node)
continue
}
waiting[node] = moves
nodes = append(nodes, node)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return false, "", err
}
step := restStep(nodes, waiting, f.gated, func(module string) bool { return oneAtATime(module, shelf[module]) })
if len(step) == 0 {
if len(walked) > 0 {
return false, "waiting before " + strings.Join(walked, "; "), nil
}
return false, "", nil
}
if r.Gates == nil {
r.Gates = map[string]*inventory.PlanGate{}
}
for _, node := range step {
r.Gates[node] = &inventory.PlanGate{Machines: []string{node}, Carried: waiting[node], Pending: true}
}
p.Note = fmt.Sprintf("sending %s at once, each judged there", strings.Join(step, ", "))
if left := len(nodes) - len(step); left > 0 {
p.Note += fmt.Sprintf("; %d machine(s) wait for a build that passed no gate yet to pass on one", left)
}
return true, "", nil
}
// oneAtATime is a module whose builds go to the rest one machine at a time (novox/hq ADR 0353 §9): the
// node-engine, which applies every send, and a module holding the mesh's resolver, which names the bus.
func oneAtATime(module string, m catalogue.Manifest) bool {
if module == hostModule {
return true
}
return slices.ContainsFunc(m.Claims, func(c catalogue.Claim) bool { return c.Name == resolverSeat })
}
// restStep is the machines of one step of the rest (novox/hq ADR 0353 §2), in the order given: each machine, unless
// a build it would carry has passed no gate and an earlier machine of the step already carries it — that build's
// first machine. Pure.
//
// **The node-engine and the mesh's resolver still go one machine at a time** (ADR 0353 §9): a step carries a
// build of either to one machine only, judged before the next, since a fault that cuts a machine off the bus
// shows only as its silence and no rollback reaches it.
func restStep(nodes []string, waiting map[string][]inventory.CarriedMove, passed func(module, commit string) bool,
alone func(module string) bool) []string {
first := map[string]string{}
var out []string
for _, n := range nodes {
var unpassed []string
for _, mv := range waiting[n] {
if !passed(mv.Module, mv.To) || alone(mv.Module) {
unpassed = append(unpassed, mv.Module+"@"+mv.To)
}
}
if slices.ContainsFunc(unpassed, func(k string) bool { _, taken := first[k]; return taken }) {
continue
}
for _, k := range unpassed {
first[k] = n
}
out = append(out, n)
}
return out
}
// silentObservation is what the gate's probe says of the newest release whose rest went silent (novox/hq ADR 0353
// §4): machines that never reported on their send, left as they were — loud, never a success said by nothing.
func silentObservation(ctx context.Context, inv *inventory.Inventory) ([]conditions.Observation, error) {
recent, err := inv.RecentPlans(ctx, 20)
if err != nil {
return nil, err
}
for _, p := range recent {
if p.Release == nil || p.Release.Settings != nil || p.Open() {
continue
}
if len(p.Release.Silent) == 0 {
return nil, nil
}
machines := append([]string(nil), p.Release.Silent...)
sort.Strings(machines)
o := conditions.Observation{Scope: conditions.ScopeMesh, ID: "release", Token: "silent", Kind: "release-silent",
Severity: conditions.Warning, Resolver: conditions.ResolverOperator, Also: machines,
Summary: fmt.Sprintf("%s: %s said nothing on the send of the builds that waited for a gate within %s, and were "+
"left as they were", p.ID, strings.Join(machines, ", "), inventory.ApplySilentAfter)}
w := releaseSilentWords(machines)
o.Headline, o.Explanation, o.Resolved, o.Needs = w.Headline, w.Explanation, w.Resolved, w.Needs
return []conditions.Observation{o}, nil
}
return nil, nil
}
// backlogObservation is what the gate's probe says of a release held after a failure.
func backlogObservation() []conditions.Observation {
if backlogNow.held == "" || len(backlogNow.waiting) == 0 {
+183 -10
View File
@@ -251,6 +251,20 @@ func gates(p inventory.Plan, edges []inventory.Edge, rollsOut func(string) bool)
return out
}
// noMoveWhy is what a plan says of a module of its tier whose build was made from the source every machine
// running it runs (novox/hq issue 280): nothing is sent for it.
// noMovePrefix opens it, and is what noMove reads: one prefix for the writer and the reader.
const (
noMovePrefix = "no move"
noMoveWhy = noMovePrefix + ": made from the source every machine running it runs"
)
// noMove is whether the plan marked the module no move and sent it nothing. Read from what the plan says,
// so a plan kept before the mark had a name of its own is read the same (novox/hq issue 648).
func noMove(s *inventory.PlanModule) bool {
return s != nil && s.FirstAt == nil && s.Gate == nil && strings.HasPrefix(s.Why, noMovePrefix)
}
// applied says whether every machine running the module has reported since the module was built.
// appliedEach is applied with a moment of its own for each machine: the reports that count are the ones
// after that machine was sent the build (novox/hq issue 256).
@@ -595,7 +609,7 @@ func advancePlans(ctx context.Context, open *stores) {
return
}
defer release()
// What waits for a gate, released one machine at a time (ADR 0236).
// What waits for a gate, released one machine first, then the rest at once (ADR 0236, ADR 0353).
if _, err := releaseBacklog(ctx, open, ""); err != nil {
fmt.Printf("plans: what waits for a gate could not be looked at: %v\n", err)
}
@@ -805,7 +819,7 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
} else if same {
now := time.Now().UTC()
state.SentAt = &now
state.Why = "no move: made from the source every machine running it runs"
state.Why = noMoveWhy
fmt.Printf("%s: %s built from the source every machine running it runs: no move, nothing sent\n", p.ID, m)
continue
}
@@ -995,6 +1009,25 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
if state.BuiltAt != nil {
since = *state.BuiltAt
}
// **A build that is no move is waited on by the send that last carried it** (novox/hq issue 648).
// Nothing was sent for it, so no report follows its marking or its build: a machine that does not
// happen to report again held the walk for ever. Each machine runs it once it answered its last
// send — the one whose builds made it no move — whenever that was.
if noMove(state) {
byNode := map[string]inventory.Reported{}
for _, r := range reports {
byNode[r.Node] = r
}
if ok, on := appliedEach(m, func(node string) time.Time {
if s := byNode[node].Sent; s != nil {
return *s
}
return time.Time{}
}, running, reports); !ok {
waiting = append(waiting, fmt.Sprintf("%s on %s", m, strings.Join(on, ", ")))
}
continue
}
// **Each machine from its own send** (novox/hq issue 256). With one machine first (ADR 0218)
// a module is sent twice — the first machine, then the rest — and SentAt is the second.
// Asked of every machine, the first machine's report, made between the two sends, read as
@@ -1056,10 +1089,36 @@ func firstSend(ctx context.Context, open *stores, p *inventory.Plan, node string
runningOf map[string][]string) error {
inv := open.inventory
before, beforeKnown, _ := inv.SentBuilds(ctx, node)
// **A machine already sent what this send is for is not sent again** (novox/hq ADR 0353 §6): sent by this plan
// before a controller was replaced and before it was kept, or by another walk. Judged from that send, and from
// the readings of a gate already judging it there.
// Only a send the machine reported it applied is one it has; one it never reported on is sent, as before
// (issue 280: what it ran before is then that build, and a failure leaves it as it was).
if beforeKnown && slices.IndexFunc(modules, func(m string) bool { return !sameCommit(before[m], p.Modules[m].Commit) }) < 0 &&
appliedLastSend(ctx, inv, node) {
if others, err := gatedMoves(ctx, open, node, nil); err == nil && len(others) == 0 {
adoptSend(ctx, open, p, node, modules, before, runningOf)
return nil
}
}
// **What the machine ran before is kept before the send is made** (novox/hq ADR 0353 §6): a controller replaced
// between the send and keeping it reads it here, and a failure still puts the machine back.
kept := false
for _, m := range modules {
s := p.Modules[m]
if beforeKnown && (s.Previous == "" || !sameCommit(before[m], s.Commit)) && s.Previous != before[m] {
s.Previous, kept = before[m], true
}
}
if kept {
if err := inv.SavePlan(ctx, p); err != nil {
return fmt.Errorf("what %s ran before is not kept, so nothing is sent: %w", node, err)
}
}
owns := make([]inventory.CarriedMove, 0, len(modules))
for _, m := range modules {
s := p.Modules[m]
owns = append(owns, inventory.CarriedMove{Module: m, Node: node, From: before[m], To: s.Commit, Build: s.Build})
owns = append(owns, inventory.CarriedMove{Module: m, Node: node, From: s.Previous, To: s.Commit, Build: s.Build})
}
// A gated send (ADR 0236): everything waiting on the machine goes with the tier, and the gate judges
// all of it there.
@@ -1074,10 +1133,7 @@ func firstSend(ctx context.Context, open *stores, p *inventory.Plan, node string
sentWhat := sentNow(ctx, inv, sent)
for _, m := range modules {
s := p.Modules[m]
// What the first machine ran before: what a failed gate puts back (ADR 0236).
if beforeKnown {
s.Previous = before[m]
}
// What the first machine ran before (ADR 0236), kept above before the send.
s.First, s.FirstAt = sent, &now
s.Gate = &inventory.PlanGate{Component: coreComponent(m), Machines: firstRunning(sent, runningOf[m]),
From: s.Previous, To: s.Commit, Since: &now, Sent: sentWhat}
@@ -1106,6 +1162,113 @@ func firstSend(ctx context.Context, open *stores, p *inventory.Plan, node string
return nil
}
// adoptSend records a first send the machine already had (novox/hq ADR 0353 §6, §7): nothing is sent; the gate counts
// from the machine's last send, and takes over the readings of an open gate of another walk judging the same build
// there — whose carried move also says what the machine ran before, for a rollback.
func adoptSend(ctx context.Context, open *stores, p *inventory.Plan, node string, modules []string,
before map[string]string, runningOf map[string][]string) {
inv := open.inventory
since := time.Now().UTC()
last, found, err := inv.SentTo(ctx, node)
if err == nil && found && !last.At.IsZero() {
since = last.At
}
lead := modules[0]
other, id, from := judgingThere(ctx, inv, p.ID, lead, node, p.Modules[lead].Commit, p.Modules[lead].Build)
for _, m := range modules {
s := p.Modules[m]
// What it ran before: kept by this walk before its send was made (§6); else what the other walk's send
// moved it from; else the build itself, so a failure leaves it as it was (issue 280).
switch {
case s.Previous != "":
case m == lead && from != "":
s.Previous = from
default:
s.Previous = before[m]
}
at := since
s.First, s.FirstAt = []string{node}, &at
s.Gate = &inventory.PlanGate{Component: coreComponent(m), Machines: firstRunning([]string{node}, runningOf[m]),
From: s.Previous, To: s.Commit, Since: &at}
if found {
s.Gate.Sent = map[string]inventory.SentDeclaration{node: last}
}
s.GatedBy = ""
if m == lead {
s.Gate.TakeReadings(other, modules, id)
} else {
s.GatedBy = lead
}
}
p.State = inventory.PlanRolling
p.Note = fmt.Sprintf("tier %d built; %s already runs %s's build: nothing sent again, judged from that send",
p.Tier, node, strings.Join(modules, ", "))
if id != "" {
p.Note += ", going on from the readings of " + id
}
fmt.Printf("%s: %s\n", p.ID, p.Note)
}
// appliedLastSend says a machine reported it applied the declaration it was sent last.
func appliedLastSend(ctx context.Context, inv *inventory.Inventory, node string) bool {
reports, err := inv.LastReports(ctx)
if err != nil {
return false
}
for _, r := range reports {
if r.Node == node {
return r.Current && r.At != nil && r.Outcome == inventory.OutcomeApplied
}
}
return false
}
// judgingThere is an open gate of another walk judging a module's build on a machine (novox/hq ADR 0353 §7), the
// walk's id, and what the machine ran before; nil when none does.
func judgingThere(ctx context.Context, inv *inventory.Inventory, self, module, node, commit, build string) (*inventory.PlanGate, string, string) {
plans, err := inv.OpenPlans(ctx)
if err != nil {
return nil, "", ""
}
// The same build, by its id where both name one, else by its commit.
same := func(b, c string) bool {
if b != "" && build != "" {
return b == build
}
return sameCommit(c, commit)
}
usable := func(g *inventory.PlanGate) bool {
return g != nil && !g.Pending && g.Since != nil && (g.Verdict == "" || g.Verdict == inventory.GatePassed) &&
slices.Contains(g.Machines, node)
}
for _, q := range plans {
if q.ID == self {
continue
}
if r := q.Release; r != nil {
gates := []*inventory.PlanGate{r.Gate}
for _, g := range r.Gates {
gates = append(gates, g)
}
for _, g := range gates {
if !usable(g) {
continue
}
for _, c := range g.Carried {
if c.Module == module && c.Node == node && same(c.Build, c.To) {
return g, q.ID, c.From
}
}
}
continue
}
if s := q.Modules[module]; s != nil && usable(s.Gate) && same(s.Build, s.Commit) {
return s.Gate, q.ID, s.Previous
}
}
return nil, "", ""
}
// passedWith keeps, on a module sent with others, the verdict of the gate that judged the send: the gate
// on the first of them passed, and its pass was kept for every build it carried (passCarried).
func passedWith(module string, s *inventory.PlanModule, lead string, g *inventory.PlanGate) {
@@ -1594,9 +1757,19 @@ func plansCommand(ctx context.Context, args []string) error {
// A release plan's walk (ADR 0236): machines done, the one judged, those to come.
fmt.Printf(" machines in order: %s; done: %s; skipped: %s\n", strings.Join(r.Order, ", "),
orNone(strings.Join(r.Done, ", ")), orNone(strings.Join(r.Skipped, ", ")))
if r.Gate != nil {
fmt.Printf(" %s\n", gateLine(r.Gate))
for _, c := range r.Gate.Carried {
if len(r.Failed)+len(r.Silent) > 0 {
fmt.Printf(" failed: %s; silent: %s\n", orNone(strings.Join(r.Failed, ", ")), orNone(strings.Join(r.Silent, ", ")))
}
gates := []*inventory.PlanGate{r.Gate}
for _, n := range r.Order {
gates = append(gates, r.Gates[n])
}
for _, g := range gates {
if g == nil {
continue
}
fmt.Printf(" %s\n", gateLine(g))
for _, c := range g.Carried {
fmt.Printf(" %-22s %s → %s\n", c.Module, short(c.From), short(c.To))
if c.Recreates != "" {
fmt.Printf(" %-22s %s\n", "", c.Recreates)
+372
View File
@@ -0,0 +1,372 @@
package main
import (
"context"
"slices"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// novox/hq ADR 0353: after its first machine passes, a build goes to every other machine at once, each judged by
// its own gate and a failing one put back by itself; and a gate's readings outlive the controller that took them —
// a machine already sent what a send is for is not sent again, and its judging goes on from what was read.
//
// Seen 2026-10-14: release-1791930194184991192 sent ace, g14, shanks and the control node one after another in
// eleven minutes; the new controller its last send started sent the control node again the controller build it
// had just applied, and judged it from its first reading.
// aBacklogOnThree is the backlog fixture with a third machine, desk, running app at c1 like the others.
func aBacklogOnThree(t *testing.T) *backlogMesh {
t.Helper()
b := aBacklog(t)
ctx := t.Context()
inv := b.open.inventory
if _, err := inv.AddNode(ctx, "desk"); err != nil {
t.Fatal(err)
}
if _, err := inv.Assign(ctx, "desk", "app"); err != nil {
t.Fatal(err)
}
if err := inv.RecordSent(ctx, nodeID(t, b.open, "desk"), "d-desk", map[string]string{"app": "c1"}); err != nil {
t.Fatal(err)
}
wasHeard := releaseHeard
releaseHeard = func(context.Context, *stores) (map[string]bool, error) {
return map[string]bool{"anchor": true, "laptop": true, "desk": true}, nil
}
t.Cleanup(func() { releaseHeard = wasHeard })
return b
}
// events records every send and every judging, in order.
func events(t *testing.T) *[]string {
t.Helper()
var said []string
wasSend, wasGather := sendRollout, gatherGateFacts
sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) {
said = append(said, "send "+strings.Join(names, ","))
return wasSend(ctx, open, names)
}
gatherGateFacts = func(ctx context.Context, open *stores, component string) (gateFacts, error) {
said = append(said, "judge")
return wasGather(ctx, open, component)
}
t.Cleanup(func() { sendRollout, gatherGateFacts = wasSend, wasGather })
return &said
}
// Once the first machine passed, the rest are both sent before either is judged (ADR 0353 §1); before, the second
// was judged to its verdict before the third was sent.
func TestAReleaseSendsTheRestAtOnceAfterTheFirstPasses(t *testing.T) {
judgeLeavesAsTheCore(t)
b := aBacklogOnThree(t)
ctx := t.Context()
said := events(t)
gateEvery = 0
for i := 0; i < 12; i++ {
advancePlans(ctx, b.open)
}
p := b.release(t)
if p.State != inventory.PlanDone || len(p.Release.Done) != 3 {
t.Fatalf("the release is %s: %s %+v", p.State, p.Note, p.Release)
}
desk, laptop := slices.Index(*said, "send desk"), slices.Index(*said, "send laptop")
if desk < 0 || laptop < 0 || slices.Index(*said, "send anchor") > min(desk, laptop) {
t.Fatalf("sends and judgings: %v", *said)
}
if slices.Contains((*said)[min(desk, laptop):max(desk, laptop)], "judge") {
t.Fatalf("the rest were sent one at a time, each judged before the next: %v", *said)
}
if v, found, err := b.open.inventory.GateOf(ctx, "build-late-c2"); err != nil || !found || v.Verdict != inventory.GatePassed {
t.Fatalf("late's pass on laptop was not kept: %+v %v %v", v, found, err)
}
}
// A machine of the rest whose gate fails is put back at once, and the other machine of the step goes on to its own
// verdict (ADR 0353 §3); before, the walk stopped at the failure and the machine after it was never sent.
func TestARestMachineThatFailsIsPutBackAloneAndTheOthersAreJudgedOn(t *testing.T) {
judgeLeavesAsTheCore(t)
b := aBacklogOnThree(t)
ctx := t.Context()
inv := b.open.inventory
gateEvery = 0
b.broken["desk"] = true // desk fails what it is sent in the rest's step
advancePlans(ctx, b.open)
for i := 0; i < 12 && b.release(t).Open(); i++ {
advancePlans(ctx, b.open)
}
p := b.release(t)
if p.State != inventory.PlanFailed || !slices.Equal(p.Release.Failed, []string{"desk"}) {
t.Fatalf("the release is %s: %s %+v", p.State, p.Note, p.Release)
}
if !slices.ContainsFunc(b.sent, func(s []string) bool { return slices.Equal(s, []string{"laptop"}) }) {
t.Fatalf("laptop was never sent: %v", b.sent)
}
if !slices.Contains(p.Release.Done, "laptop") {
t.Fatalf("laptop has no verdict of its own: %+v", p.Release)
}
// late waited on laptop alone and was healthy there: its pass is kept, whatever desk did.
if v, found, err := inv.GateOf(ctx, "build-late-c2"); err != nil || !found || v.Verdict != inventory.GatePassed {
t.Fatalf("late's pass on laptop was not kept: %+v %v %v", v, found, err)
}
// app failed on desk: marked, and put back wherever it was sent (ADR 0236).
if failed, _ := inv.GateFailed(ctx, "build-app-c2"); !failed {
t.Fatal("app's build is not marked failed")
}
if sent, _, _ := inv.SentBuilds(ctx, "desk"); sent["app"] != "c1" {
t.Fatalf("desk was not put back: %v", sent)
}
}
// A walk whose first machine already runs its build under another walk's gate sends nothing and goes on from that
// gate's readings (ADR 0353 §7); before, it sent the machine again and judged from its first reading.
func TestAWalkTakesOverTheReadingsOfAGateJudgingTheSameBuildThere(t *testing.T) {
judgeLeavesAsTheCore(t)
b := aBacklog(t)
ctx := t.Context()
inv := b.open.inventory
advancePlans(ctx, b.open) // the release sends anchor app c2, and reads it once
release := b.release(t)
judging := release.Release.Gate
if judging == nil || judging.Pending || judging.Passes == 0 {
t.Fatalf("the release is not judging anchor: %+v", judging)
}
built := time.Now().UTC()
plan := inventory.Plan{ID: "plan-app", Repository: "novox/mesh-catalog", Branch: "main", Commit: "c2",
Created: built, State: inventory.PlanBuilding, Tiers: [][]string{{"app"}},
Modules: map[string]*inventory.PlanModule{"app": {State: "built", BuiltAt: &built, Commit: "c2",
Build: "build-app-c2"}}}
if err := inv.SavePlan(ctx, &plan); err != nil {
t.Fatal(err)
}
advancePlans(ctx, b.open)
if len(b.sent) != 1 {
t.Fatalf("anchor was sent again what it runs: %v", b.sent)
}
p, err := inv.PlanByID(ctx, "plan-app")
if err != nil {
t.Fatal(err)
}
g := p.Modules["app"].Gate
if g == nil || g.Since == nil || !g.Since.Equal(*judging.Since) || g.Passes < judging.Passes || g.TookOver != release.ID {
t.Fatalf("the walk's gate starts from zero: %+v (the release's: %+v)", g, judging)
}
if p.Modules["app"].Previous != "c1" {
t.Fatalf("what anchor ran before is %q, not c1", p.Modules["app"].Previous)
}
}
// A walk's first send that was made and not kept — the controller replaced between the two — is not made again
// by the controller that takes over: the gate counts from that send (ADR 0353 §5, §6).
func TestAFirstSendMadeAndNotKeptIsNotSentAgain(t *testing.T) {
judgeLeavesAsTheCore(t)
g := aGateMesh(t)
ctx := t.Context()
// The old controller sends anchor app c2 and is replaced before it keeps the send.
made := sendRollout
sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) {
if _, err := made(ctx, open, names); err != nil {
return nil, err
}
return nil, context.Canceled
}
advancePlans(ctx, g.open)
sendRollout = made
if len(g.sent) != 1 || g.plan(t).Modules["app"].FirstAt != nil {
t.Fatalf("the fixture's send was not made, or was kept: %v", g.sent)
}
last, _, err := g.open.inventory.SentTo(ctx, "anchor")
if err != nil {
t.Fatal(err)
}
advancePlans(ctx, g.open)
if len(g.sent) != 1 {
t.Fatalf("the send was made again: %v", g.sent)
}
s := g.plan(t).Modules["app"]
if s.FirstAt == nil || s.Gate == nil || s.Gate.Since == nil || !s.Gate.Since.Equal(last.At) {
t.Fatalf("the gate does not count from the send made: %+v %+v", s, s.Gate)
}
// What anchor ran before was kept before the send, so a failure still puts it back (§5).
if s.Previous != "c1" || s.Gate.From != "c1" {
t.Fatalf("what anchor ran before is %q (gate from %q), not c1", s.Previous, s.Gate.From)
}
}
// A release's send kept before it was made, and made, is judged by the controller that takes over without being
// made again (ADR 0353 §8).
func TestAReleaseSendKeptAndMadeIsNotMadeAgain(t *testing.T) {
judgeLeavesAsTheCore(t)
b := aBacklog(t)
ctx := t.Context()
inv := b.open.inventory
p, err := releaseBacklog(ctx, b.open, "")
if err != nil || p == nil {
t.Fatalf("no release opened: %v", err)
}
if _, err := advanceRelease(ctx, b.open, p); err != nil || p.Release.Gate == nil || !p.Release.Gate.Pending {
t.Fatalf("the send was not kept before it was made: %+v %v", p.Release, err)
}
if err := inv.SavePlan(ctx, p); err != nil {
t.Fatal(err)
}
// The send is made, and the controller is replaced before it keeps it.
if _, err := sendRollout(ctx, b.open, []string{"anchor"}); err != nil {
t.Fatal(err)
}
advancePlans(ctx, b.open)
if len(b.sent) != 1 {
t.Fatalf("anchor was sent again: %v", b.sent)
}
g := b.release(t).Release.Gate
if g == nil || g.Pending || g.Since == nil || len(g.Carried) == 0 {
t.Fatalf("anchor is not judged on the send made: %+v", g)
}
}
// A build no gate has passed goes to one machine of a step, its first; a machine that would be its second waits
// for the next step, and one whose every build passed goes now (ADR 0353 §2).
func TestABuildThatPassedNoGateGoesToOneMachineOfAStep(t *testing.T) {
waiting := map[string][]inventory.CarriedMove{
"desk": {{Module: "app", To: "c2"}, {Module: "new", To: "n1"}},
"laptop": {{Module: "app", To: "c2"}, {Module: "new", To: "n1"}},
"shanks": {{Module: "app", To: "c2"}},
"novox": {{Module: "app", To: "c2"}, {Module: "ctl", To: "k1"}},
}
passed := func(module, commit string) bool { return module == "app" && commit == "c2" }
never := func(string) bool { return false }
step := restStep([]string{"desk", "laptop", "shanks", "novox"}, waiting, passed, never)
if !slices.Equal(step, []string{"desk", "shanks", "novox"}) {
t.Fatalf("the step is %v: laptop would be new's second machine before it passed", step)
}
if step := restStep([]string{"laptop"}, waiting, func(string, string) bool { return true }, never); !slices.Equal(step, []string{"laptop"}) {
t.Fatalf("a machine whose every build passed waits: %v", step)
}
}
// The node-engine and the mesh's resolver go to the rest one machine at a time, even passed (ADR 0353 §9): a step
// carries either to one machine only, while what else passed goes everywhere at once.
func TestTheNodeEngineAndTheResolverGoToTheRestOneAtATime(t *testing.T) {
waiting := map[string][]inventory.CarriedMove{
"desk": {{Module: hostModule, To: "h2"}},
"laptop": {{Module: hostModule, To: "h2"}, {Module: "app", To: "c2"}},
"shanks": {{Module: "app", To: "c2"}},
"novox": {{Module: "dns", To: "d2"}},
"ace": {{Module: "dns", To: "d2"}},
}
resolver := catalogue.Manifest{Module: "dns", Claims: []catalogue.Claim{{Name: resolverSeat}}}
alone := func(m string) bool {
if m == "dns" {
return oneAtATime(m, resolver)
}
return oneAtATime(m, catalogue.Manifest{Module: m})
}
step := restStep([]string{"desk", "laptop", "shanks", "novox", "ace"}, waiting,
func(string, string) bool { return true }, alone)
if !slices.Equal(step, []string{"desk", "shanks", "novox"}) {
t.Fatalf("the step is %v: the node-engine and the resolver each reach one machine of it", step)
}
if oneAtATime("app", catalogue.Manifest{Module: "app"}) {
t.Fatal("a leaf module is held to one machine at a time")
}
}
// A first machine that fails still stops the walk (ADR 0236 §5, kept by ADR 0353 §3): the rest are never sent.
func TestAFirstMachineThatFailsStillStopsTheWalk(t *testing.T) {
b := aBacklogOnThree(t)
ctx := t.Context()
gateEvery = 0
b.broken["anchor"] = true
advancePlans(ctx, b.open)
for i := 0; i < 6 && b.release(t).Open(); i++ {
advancePlans(ctx, b.open)
}
if p := b.release(t); p.State != inventory.PlanFailed || p.Release.Rest {
t.Fatalf("the release is %s: %s %+v", p.State, p.Note, p.Release)
}
for _, s := range b.sent {
if !slices.Equal(s, []string{"anchor"}) {
t.Fatalf("sent %v: the rest after a failed first machine", b.sent)
}
}
}
// The control node is sent last in a step of the rest (ADR 0353 §1): the machine holding the controller is
// ordered after every other.
func TestTheControlNodeIsSentLastInAStep(t *testing.T) {
judgeLeavesAsTheCore(t)
b := aBacklogOnThree(t)
ctx := t.Context()
register(t, b.open, catalogue.Manifest{Module: catalogue.ControllerSeatName, Version: "1"})
if _, err := b.open.inventory.Assign(ctx, "anchor", catalogue.ControllerSeatName); err != nil {
t.Fatal(err)
}
said := events(t)
gateEvery = 0
for i := 0; i < 12 && (len(b.sent) == 0 || b.release(t).Open()); i++ {
advancePlans(ctx, b.open)
}
if p := b.release(t); !slices.Equal(p.Release.Order, []string{"desk", "laptop", "anchor"}) {
t.Fatalf("the order is %v", p.Release.Order)
}
laptop, anchor := slices.Index(*said, "send laptop"), slices.Index(*said, "send anchor")
if laptop < 0 || anchor < laptop || slices.Contains((*said)[laptop:anchor], "judge") {
t.Fatalf("the control node is not sent last in the rest's step: %v", *said)
}
}
// A machine of the rest that says nothing on its send is left silent after fifteen minutes: not failed, not put
// back; a rest that went silent whole ends failed, said as a condition (ADR 0353 §4).
func TestARestMachineSilentPastItsBoundIsLeftAndSaid(t *testing.T) {
judgeLeavesAsTheCore(t)
b := aBacklog(t)
ctx := t.Context()
inv := b.open.inventory
reporting := sendRollout
sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) {
if slices.Equal(names, []string{"laptop"}) { // laptop sleeps: sent, and says nothing
b.sent = append(b.sent, names)
current, _ := open.inventory.CurrentBuilds(ctx)
return names, open.inventory.RecordSent(ctx, nodeID(t, open, "laptop"), "d-laptop-asleep",
map[string]string{"app": current["app"].Commit, "late": current["late"].Commit, "same": current["same"].Commit})
}
return reporting(ctx, open, names)
}
t.Cleanup(func() { sendRollout = reporting })
gateEvery = 0
for i := 0; i < 8; i++ {
advancePlans(ctx, b.open)
}
p := b.release(t)
g := p.Release.Gates["laptop"]
if !p.Open() || g == nil || g.Verdict != "" || g.Silent {
t.Fatalf("laptop is judged before fifteen minutes: %s %s %+v", p.State, p.Note, g)
}
since := time.Now().Add(-inventory.ApplySilentAfter - time.Minute)
g.Since = &since
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
advancePlans(ctx, b.open)
p = b.release(t)
if p.State != inventory.PlanFailed || !slices.Equal(p.Release.Silent, []string{"laptop"}) || len(p.Release.Failed) > 0 {
t.Fatalf("the release is %s: %s %+v", p.State, p.Note, p.Release)
}
if failed, _ := inv.GateFailed(ctx, "build-late-c2"); failed {
t.Fatal("a silent machine's build was marked failed")
}
if sent, _, _ := inv.SentBuilds(ctx, "laptop"); sent["late"] != "c2" {
t.Fatalf("laptop was put back: %v", sent)
}
obs, err := silentObservation(ctx, inv)
if err != nil {
t.Fatal(err)
}
if len(obs) != 1 || obs[0].Kind != "release-silent" || !slices.Equal(obs[0].Also, []string{"laptop"}) {
t.Fatalf("the silence is not said: %+v", obs)
}
}
+28
View File
@@ -457,6 +457,28 @@ func (a *verbArguments) commandLine() ([]string, error) {
return nil, err
}
return []string{"forget-checks", str("owner"), str("repo"), str("number")}, nil
case "try":
// A branch tried before its pull request (novox/hq issue 686): asked, or read by the id it answered.
if id := str("id"); id != "" {
return []string{"try", "--id", id}, nil
}
if a.given["repository"] == "" && a.given["ref"] == "" {
// Neither shape named: the command says its usage, which names both, as `build` does.
return []string{"try"}, nil
}
if err := need("repository", "ref"); err != nil {
return nil, err
}
argv := []string{"try", str("repository"), str("ref")}
for _, flag := range []string{"base", "tests"} {
if v := str(flag); v != "" {
argv = append(argv, "--"+flag, v)
}
}
if on("store-only") {
argv = append(argv, "--store-only")
}
return argv, nil
case "cancel", "kill":
if err := need("id"); err != nil {
return nil, err
@@ -1018,6 +1040,8 @@ var jsonVerbs = map[string]bool{"status": true, "seats": true, "plan": true, "co
"delivery": true,
// The build queue, each ask with its position among the waiting (novox/hq issue 455).
"queue": true,
// A branch tried before its pull request (novox/hq issue 686).
"try": true,
// A plan followed (novox/hq issue 229): `plans wait` answers JSON; its other forms say text, and so answer no data.
"plans": true}
@@ -1308,6 +1332,10 @@ func seatToolHandlers() (map[string]link.ToolHandler, []string, error) {
link.AnswersWithin(ctx, bound+planWaitStart)
}
}
if verb == "try" && len(argv) > 1 && argv[1] == "--id" {
// Reading a try holds its answer for the verdict, up to its bound (novox/hq issue 686, 229).
link.AnswersWithin(ctx, tryWait+planWaitStart)
}
if answer, read := readHere(ctx, argv); read {
return answer, nil
}
-204
View File
@@ -1,204 +0,0 @@
package main
import (
"bytes"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
snapshot "github.com/novox/mesh-controller/internal/facts"
"github.com/novox/mesh-controller/internal/inventory"
)
// A file's text kept as a template file in its module's repository (novox/hq ADR 0349 §2, design 57 package T1,
// issue 654): `module check` and the merge gate read it from the tree beside the manifest, so a change to a
// template alone is judged as a change to its module.
const templatedNotifier = `{"module":"notifier","version":"1",
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
"resources":[{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644","template":"files/notifier.conf"}]}`
// inlineNotifier is the same module as the mesh holds it: built, its text inline.
const inlineNotifier = `{"module":"notifier","version":"1",
"resources":[{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644","content":"volume = 3\n"}]}`
// aTemplatedModule writes a module's directory: its manifest and its files/.
func aTemplatedModule(t *testing.T, dir, manifest string, files map[string]string) string {
t.Helper()
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, "module.json")
if err := os.WriteFile(path, []byte(manifest), 0o644); err != nil {
t.Fatal(err)
}
for name, text := range files {
full := filepath.Join(dir, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte(text), 0o644); err != nil {
t.Fatal(err)
}
}
return path
}
func TestModuleCheckReadsTemplatesFromTheTreeBesideTheManifest(t *testing.T) {
good := aTemplatedModule(t, t.TempDir(), templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
var out bytes.Buffer
if err := moduleCheck([]string{good}, &out); err != nil || !strings.Contains(out.String(), "notifier: ok") {
t.Fatalf("a module with its text in files/ was refused: %v\n%s", err, out.String())
}
// A change to the template alone, to text the controller refuses inline, is refused the same.
bad := aTemplatedModule(t, t.TempDir(), templatedNotifier,
map[string]string{"files/notifier.conf": "volume = 3\nstore = ${dir:nowhere}\n"})
out.Reset()
if err := moduleCheck([]string{bad}, &out); err == nil || !strings.Contains(out.String(), "nowhere") {
t.Fatalf("a template whose text is refused inline passed:\n%s", out.String())
}
// A symlink under files/ that no template names: the build's packing refuses it, so the check does too.
linked := t.TempDir()
aTemplatedModule(t, linked, templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
if err := os.Symlink("/etc/hostname", filepath.Join(linked, "files", "elsewhere")); err != nil {
t.Fatal(err)
}
out.Reset()
if err := moduleCheck([]string{filepath.Join(linked, "module.json")}, &out); err == nil ||
!strings.Contains(out.String(), "files/elsewhere is a symlink") {
t.Fatalf("a symlink under files/ passed the check:\n%s", out.String())
}
// And a template the tree lacks, by name.
missing := aTemplatedModule(t, t.TempDir(), templatedNotifier, nil)
out.Reset()
if err := moduleCheck([]string{missing}, &out); err == nil || !strings.Contains(out.String(), "files/notifier.conf") {
t.Fatalf("a missing template was not refused by name:\n%s", out.String())
}
}
// templatedMesh is catalogueMesh with the notifier on the laptop, registered as built: its text inline.
func templatedMesh(t *testing.T) (snapshot.Facts, map[string]string) {
t.Helper()
open := aMesh(t)
ctx := t.Context()
manifests := map[string]string{
"lemurs": `{"module":"lemurs","version":"1","capabilities":["systemd"],
"resources":[{"id":"service","type":"service","unit":"lemurs.service","state":"running","boot":"enabled"}]}`,
"notifier": inlineNotifier,
}
for name, raw := range manifests {
m, err := catalogue.ParseManifest([]byte(raw))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if err := open.inventory.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-catalog",
Path: "modules/" + name, BuiltFrom: "c0ffee"}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", name); err != nil {
t.Fatalf("assign %s: %v", name, err)
}
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
return f, manifests
}
// aTemplatedTree is the catalogue's checkout with the notifier's text in files/.
func aTemplatedTree(t *testing.T, manifests map[string]string, text string, has bool) string {
t.Helper()
tree := aTree(t, withEdit(manifests, "notifier", ""))
files := map[string]string{}
if has {
files["files/notifier.conf"] = text
}
aTemplatedModule(t, filepath.Join(tree, "modules", "notifier"), templatedNotifier, files)
return tree
}
const notifierTemplate = "modules/notifier/files/notifier.conf"
func TestTheMergeGateJudgesAChangeToATemplateAlone(t *testing.T) {
f, manifests := templatedMesh(t)
// The module moved to its template, the same text: nothing changes on any machine.
v := gateJudged(t, f, aTemplatedTree(t, manifests, "volume = 3\n", true), "modules/notifier/module.json",
notifierTemplate)
if v.Verdict == "fail" || v.Modules["notifier"] != "" {
t.Fatalf("the module moved to its template with the same text reads as a change (%v):\n%s", v.Modules, v.Report())
}
// The template alone changed: its module changed, and composes.
v = gateJudged(t, f, aTemplatedTree(t, manifests, "volume = 4\n", true), notifierTemplate)
if v.Verdict == "fail" || v.Modules["notifier"] != "changed" {
t.Fatalf("a change to the template alone is not a change to its module (%v):\n%s", v.Modules, v.Report())
}
if v.Width == nil || strings.Join(v.Width.Modules, ",") != "notifier" {
t.Fatalf("a change to the template rebuilds %+v, not its module", v.Width)
}
// The template alone changed to text the controller refuses: the pull request fails, naming the module.
v = gateJudged(t, f, aTemplatedTree(t, manifests, "store = ${dir:nowhere}\n", true), notifierTemplate)
if v.Verdict != "fail" || !strings.Contains(v.Report(), "notifier") {
t.Fatalf("a template the controller refuses passed the gate:\n%s", v.Report())
}
// A symlinked files/ fails: nothing is read from outside the tree.
tree := aTree(t, withEdit(manifests, "notifier", ""))
elsewhere := t.TempDir()
if err := os.WriteFile(filepath.Join(elsewhere, "notifier.conf"), []byte("volume = 3\n"), 0o644); err != nil {
t.Fatal(err)
}
aTemplatedModule(t, filepath.Join(tree, "modules", "notifier"), templatedNotifier, nil)
if err := os.Symlink(elsewhere, filepath.Join(tree, "modules", "notifier", "files")); err != nil {
t.Fatal(err)
}
v = gateJudged(t, f, tree, notifierTemplate)
if v.Verdict != "fail" || !strings.Contains(v.Report(), "files/ is a symlink") {
t.Fatalf("a symlinked files/ passed the gate:\n%s", v.Report())
}
// And the template removed from the tree fails, naming it.
v = gateJudged(t, f, aTemplatedTree(t, manifests, "", false), notifierTemplate)
if v.Verdict != "fail" || !strings.Contains(v.Report(), "files/notifier.conf") {
t.Fatalf("a missing template passed the gate:\n%s", v.Report())
}
}
// What the gate composes for the module in its template form is what the mesh composes for it inline.
func TestTheMergeGateResolvesATemplatedModuleToItsInlineTwin(t *testing.T) {
dir := t.TempDir()
path := aTemplatedModule(t, dir, templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
m, err := catalogue.ParseManifest(raw)
if err != nil {
t.Fatal(err)
}
m, err = catalogue.WithTemplatesFrom(m, dir)
if err != nil {
t.Fatal(err)
}
resolved, err := standInBuild(m)
if err != nil {
t.Fatal(err)
}
inline, err := catalogue.ParseManifest([]byte(inlineNotifier))
if err != nil {
t.Fatal(err)
}
if builtAlike(resolved) != builtAlike(inline) {
got, _ := json.Marshal(resolved)
t.Fatalf("the templated module resolves to %s, not its inline twin", got)
}
}
@@ -1,46 +0,0 @@
[
{
"id": "found.server",
"image": "registry.invalid:5000/library@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"networks": [
"predecessor_default"
],
"type": "container"
},
{
"id": "library.access-storage/downloads",
"mode": "read",
"path": "/storage/downloads",
"type": "access"
},
{
"id": "library.access-storage/media/series",
"mode": "read",
"path": "/storage/media/series",
"type": "access"
},
{
"id": "library.config",
"mode": "0750",
"owner": "1001:2000",
"path": "/var/lib/library/config",
"type": "directory"
},
{
"content": "{\n \"accesses\": {\n \"downloads\": \"/storage/downloads\",\n \"series\": \"/storage/media/series\"\n },\n \"log\": \"warn\",\n \"places\": {\n \"config\": {\n \"owner\": \"1001:2000\",\n \"path\": \"/var/lib/library/config\"\n }\n },\n \"workers\": 4\n}\n",
"id": "library.runtime-config",
"mode": "0640",
"path": "/var/lib/library/config/runtime.json",
"type": "file"
},
{
"id": "library.server",
"image": "registry.invalid:5000/library@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"type": "container",
"volumes": [
"/var/lib/library/config:/config",
"/storage/media/series:/series",
"/storage/downloads:/downloads:ro"
]
}
]
@@ -1,37 +0,0 @@
[
{
"id": "library.access-home/media/series",
"mode": "read",
"path": "/home/media/series",
"type": "access"
},
{
"id": "library.access-storage/downloads",
"mode": "read",
"path": "/storage/downloads",
"type": "access"
},
{
"id": "library.config",
"mode": "0750",
"path": "/var/lib/library/config",
"type": "directory"
},
{
"content": "{\n \"accesses\": {\n \"downloads\": \"/storage/downloads\",\n \"series\": \"/home/media/series\"\n },\n \"log\": \"warn\"\n}\n",
"id": "library.runtime-config",
"mode": "0640",
"path": "/var/lib/library/config/runtime.json",
"type": "file"
},
{
"id": "library.server",
"image": "registry.invalid:5000/library@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"type": "container",
"volumes": [
"/var/lib/library/config:/config",
"/home/media/series:/series",
"/storage/downloads:/downloads:ro"
]
}
]
+353
View File
@@ -0,0 +1,353 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"net/url"
"os"
"regexp"
"strings"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
)
// A branch is tried before its pull request (novox/hq issue 686).
//
// A repository's store tests skip without a store, and no agent may start a database on its machine. So a
// builder pushed, a reviewer read, and only the pull request's merge check ran those tests for the first time:
// each failure there cost a round of fifteen to thirty minutes (mesh-controller #283, #310, #312).
//
// `try <owner>/<repository> <ref>` asks the build seat for the repository's own check of a pushed branch or
// commit, as the merge check runs it (internal/builder/check.go): the ref merged into the trunk's head, the
// trunk's merge-check.sh, each part in its toolchain's container, against a throwaway store and bus. It is a
// check and nothing more:
//
// - **no pull request, so no status**: its outcome is never said as `checked`, which is what the forge's
// holder sets statuses from (isTry, in Built);
// - **nothing registered, nothing built**: it names no module of the graph, so the merge gate does not run and
// no build is made for a merge (novox/hq ADR 0350's scratch builds are a pull request's check's alone);
// - **a check in the build queue**: after deliveries (link.KindOf, novox/hq issue 558), and bounded by the
// check's own bound (checkTimeout);
// - **the code a pull request's check would run anyway**: only a repository of the mesh, whose every pull
// request the build seat already checks, in containers that hold no runtime socket.
//
// A check outlasts a seat verb's one answer, so the ask answers its id at once and `try --id <id>` reads what
// came of it from the build's log, where the build seat says it whole (link.StepTried): holding its answer up
// to tryWait for a verdict, like `plans wait` (novox/hq issue 229). Nothing is kept beside the log.
// tryWait is how long reading a try holds its answer for the verdict. A variable so a test need not wait.
var tryWait = 20 * time.Second
// A ref as the forge is asked for it — a branch or a commit, never an option — and a `go test -run` pattern.
var (
tryRef = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._/-]*$`)
tryTests = regexp.MustCompile(`^[A-Za-z0-9_^$|().*+?/\[\]-]+$`)
)
// tryAnswer is what `try` answers, asking or reading.
type tryAnswer struct {
ID string `json:"id"`
// State is asked (just now), waiting (nothing said of it yet: in the build queue), running, or the verdict:
// pass, warning (the repository declares no merge-check.sh), fail, or error (it could not run).
State string `json:"state"`
Said string `json:"said"`
Repository string `json:"repository,omitempty"`
Ref string `json:"ref,omitempty"`
Commit string `json:"commit,omitempty"`
Base string `json:"base,omitempty"`
Trunk string `json:"trunk,omitempty"`
On string `json:"on,omitempty"`
Took string `json:"took,omitempty"`
Failed []string `json:"failed,omitempty"`
// Last is the last thing the build seat said of one still running.
Last string `json:"last,omitempty"`
}
// isTry says a check's outcome is a try's: marked so, or — echoed by a build seat from before the mark — a
// check of no pull request and no group, which is nobody's status either.
func isTry(result link.BuildResult) bool {
c := result.Checked
return c != nil && (c.Try || (c.Number == 0 && c.Group == ""))
}
// asATry makes a check's ask a try's: the repository's own script alone, for no pull request.
func asATry(request link.BuildRequest, tests string, storeOnly bool) link.BuildRequest {
c := *request.Check
c.Number, c.Try, c.Tests, c.StoreOnly = 0, true, tests, storeOnly
c.Paths, c.Modules, c.New, c.Dependents, c.Manifests, c.Builds = nil, nil, nil, nil, nil, nil
c.Judge, c.Plan, c.Group, c.Members = "", nil, "", nil
request.Check = &c
return request
}
// maxTries is how many tries may wait or run at once, mesh-wide: each holds a build machine for minutes.
const maxTries = 4
// triesIn reads the build queue for a new try of one commit: the id of a try of that commit already waiting or
// running — the answer to a second ask — and whether there is room for another.
func triesIn(q link.Queue, owner, repo, commit string) (same string, room bool) {
n := 0
for _, a := range q.Asks {
if !link.IsTry(a.Request) || (a.State != link.AskWaiting && a.State != link.AskInFlight) {
continue
}
n++
c := a.Request.Check
if same == "" && strings.EqualFold(c.Owner, owner) && strings.EqualFold(c.Repo, repo) && a.Request.Ref == commit {
same = a.ID
}
}
return same, n < maxTries
}
// forgeCommit asks the forge, through its module's tools on the bus, which commit a branch or commit of a
// repository is, and the repository's default branch. A variable so a test needs no forge.
var forgeCommit = func(ctx context.Context, owner, repo, ref string) (commit, trunk string, err error) {
js, err := aBus()
if err != nil {
return "", "", err
}
defer js.Close()
bus := link.OverNATS{Conn: js.Conn()}
ask := func(path string, into any) error {
raw, _ := json.Marshal(map[string]any{"path": path})
answer, err := link.Ask(ctx, bus, "gitea", "gitea_api", raw, 30*time.Second)
if err != nil {
return err
}
if answer.Error != "" {
return errors.New(answer.Error)
}
return json.Unmarshal(answer.Result, into)
}
var commits struct {
Result []struct {
SHA string `json:"sha"`
} `json:"result"`
}
if err := ask("/repos/"+owner+"/"+repo+"/commits?limit=1&stat=false&verification=false&files=false&sha="+
url.QueryEscape(ref), &commits); err != nil {
return "", "", err
}
if len(commits.Result) == 0 || commits.Result[0].SHA == "" {
return "", "", errors.New("the forge named no commit for it")
}
var found struct {
Result struct {
DefaultBranch string `json:"default_branch"`
} `json:"result"`
}
if err := ask("/repos/"+owner+"/"+repo, &found); err != nil {
return "", "", err
}
return commits.Result[0].SHA, found.Result.DefaultBranch, nil
}
// tryCommand is `try <owner>/<repository> <ref> [--base <branch>] [--tests <pattern>] [--store-only]`, and
// `try --id <id>` to read what came of one.
func tryCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("try", flag.ContinueOnError)
base := set.String("base", "", "the branch it is merged into (the repository's default branch when none)")
tests := set.String("tests", "", "only the tests this `go test -run` pattern names")
storeOnly := set.Bool("store-only", false, "only the repository's store tests")
id := set.String("id", "", "a try's id: what came of it")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
usage := errors.New("try <owner>/<repository> <branch or commit> [--base <branch>] [--tests <pattern>] " +
"[--store-only], or try --id <id>")
if *id != "" {
if len(positionals) != 0 {
return usage
}
// Before it is a token of a bus subject: an id, never a pattern.
if !link.SafeBuildID(*id) {
return fmt.Errorf("%q is not a try's id", *id)
}
return tryRead(ctx, *id)
}
if len(positionals) != 2 {
return usage
}
owner, repo, isPath := strings.Cut(positionals[0], "/")
ref := positionals[1]
switch {
case !isPath || !tryRef.MatchString(owner) || !tryRef.MatchString(repo) || strings.Contains(repo, "/"):
return fmt.Errorf("%q is not a repository on the forge, owner/name; nothing was asked", positionals[0])
case !tryRef.MatchString(ref):
return fmt.Errorf("%q is not a branch or a commit; nothing was asked", ref)
case *base != "" && !tryRef.MatchString(*base):
return fmt.Errorf("%q is not a branch; nothing was asked", *base)
case *tests != "" && !tryTests.MatchString(*tests):
return fmt.Errorf("%q is not a `go test -run` pattern of test names; nothing was asked", *tests)
}
// **An unknown ref is refused here, loudly** — not sent to a machine to fail at its clone minutes later.
commit, trunk, err := forgeCommit(ctx, owner, repo, ref)
if err != nil {
return fmt.Errorf("the forge holds no branch or commit %s in %s/%s — is it pushed? — so nothing was asked: %w",
ref, owner, repo, err)
}
if *base != "" {
trunk = *base
}
if trunk == "" {
return fmt.Errorf("the forge named no default branch for %s/%s, and no --base says what %s is merged into; "+
"nothing was asked", owner, repo, ref)
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
entries, read, edges, err := theGraph(ctx, open.inventory)
if err != nil {
return err
}
p := link.PullUpdated{Owner: owner, Repo: repo, Base: trunk, Commit: commit}
// Whose repository it is, as a pull request's check reckons it: only the mesh's is tried, the code the
// build seat runs for every pull request already.
mesh := pullScope(p, entries, read, edges)
if !mesh.Mesh {
return fmt.Errorf("%s/%s is no repository of the mesh — no module is built from it and its owner is not "+
"the core's — and only the mesh's are tried on the build seat; nothing was asked", owner, repo)
}
request, err := checkRequestFor(ctx, open, p, checkScope{Mesh: true, From: mesh.From}, entries)
if err != nil {
return err
}
request = asATry(request, *tests, *storeOnly)
seat := buildSeatHeld(ctx)
// **Bounded in number** (novox/hq issue 686): one try of a commit at a time, and maxTries in all.
js, err := dialTheBus()
if err != nil {
return err
}
queue, err := link.ReadQueue(ctx, js, seat)
js.Close()
if err != nil {
return err
}
same, room := triesIn(queue, owner, repo, commit)
if same != "" {
return printTry(tryAnswer{ID: same, State: "asked", Repository: owner + "/" + repo, Ref: ref, Commit: commit,
On: seat, Said: fmt.Sprintf("a try of %s/%s at %.12s already waits or runs, as %s: nothing new was asked. "+
"Read it with try id=%s", owner, repo, commit, same, same)})
}
if !room {
return fmt.Errorf("%d tries already wait or run on the build seat, the most at once (`queue` lists them); "+
"nothing was asked — read one of yours, or ask again when one has ended", maxTries)
}
ask, err := askOverOn(seat)
if err != nil {
return err
}
defer ask.Close()
if err := ask.Ask(ctx, request); err != nil {
return err
}
return printTry(tryAnswer{ID: request.ID, State: "asked", Repository: owner + "/" + repo, Ref: ref, Commit: commit,
Base: trunk, On: seat, Said: fmt.Sprintf("asked %s to try %s/%s at %.12s (%s) merged into %s: its own "+
"merge-check.sh, as a check in the build queue, bounded at %s; no status is set and nothing is registered. "+
"Read it with try id=%s — a full run of this kind takes minutes", seat, owner, repo, commit, ref, trunk,
checkTimeout, request.ID)})
}
// tryRead answers what came of a try, holding the answer up to tryWait for its verdict.
func tryRead(ctx context.Context, id string) error {
until := time.Now().Add(tryWait)
for {
lines, err := buildLines(ctx, id)
if err != nil {
return err
}
a := tryAnswerOf(id, lines)
if (a.State != "waiting" && a.State != "running") || !time.Now().Add(2*time.Second).Before(until) {
return printTry(a)
}
if err := sleepOrDone(ctx, 2*time.Second); err != nil {
return err
}
}
}
// tryAnswerOf reads a try's answer from what the build seat said of it: its outcome under StepTried; `failed`
// for one that could not run — an error, never a pass; else how far it is.
func tryAnswerOf(id string, lines []link.BuildLine) tryAnswer {
a := tryAnswer{ID: id, State: "waiting", Said: "nothing is said of " + id + " yet: it waits in the build queue " +
"(`queue` says where), or no try has that id; ask again"}
for _, l := range lines {
switch l.Step {
case link.StepTried:
var t link.Tried
if err := json.Unmarshal([]byte(l.Message), &t); err != nil || t.Verdict == "" {
return tryAnswer{ID: id, State: "error", Said: "the build seat's word on it cannot be read: " + l.Message}
}
return tryAnswer{ID: id, State: t.Verdict, Said: t.Summary, Took: t.Took, Commit: t.Head, Trunk: t.Trunk,
Failed: t.Failed}
case "failed":
return tryAnswer{ID: id, State: "error", Said: "it could not run: " + l.Message}
case "checked":
// A check that is no try, or a build seat from before the verb: its verdict is not a try's.
return tryAnswer{ID: id, State: "error", Said: "the build seat checked it and said no try's outcome — " +
"it predates the verb (novox/hq issue 686), or " + id + " is no try: " + l.Message + "; `builds` with " +
"the id says every line"}
}
a.State, a.Last = "running", l.Message
a.Said = "still running; ask again. `builds` with the id says every line so far"
}
return a
}
func printTry(a tryAnswer) error {
fmt.Fprintln(os.Stderr, a.Said)
for _, l := range a.Failed {
fmt.Fprintln(os.Stderr, " "+l)
}
out, err := json.MarshalIndent(a, "", " ")
if err != nil {
return err
}
fmt.Println(string(out))
return nil
}
// buildLines is everything the build seat said of one build, as the bus keeps it, in order.
func buildLines(ctx context.Context, id string) ([]link.BuildLine, error) {
js, err := aBus()
if err != nil {
return nil, fmt.Errorf("cannot reach the bus to read a build's log: %w", err)
}
defer js.Close()
subject := link.BuildLogOf("*", id)
sub, err := js.Context().PullSubscribe(subject, "",
nats.BindStream(broker.EventsStream), nats.DeliverAll(), nats.AckNone())
if err != nil {
return nil, fmt.Errorf("cannot read %s from the bus: %w", subject, err)
}
defer func() { _ = sub.Unsubscribe() }()
var lines []link.BuildLine
for {
batch, err := sub.Fetch(200, nats.MaxWait(2*time.Second))
if err != nil && !errors.Is(err, nats.ErrTimeout) && !errors.Is(err, context.DeadlineExceeded) {
return nil, fmt.Errorf("reading a build's log: %w", err)
}
for _, msg := range batch {
var line link.BuildLine
if json.Unmarshal(msg.Data, &line) == nil {
lines = append(lines, line)
}
}
if len(batch) < 200 {
return lines, nil
}
}
}
+194
View File
@@ -0,0 +1,194 @@
package main
import (
"context"
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
// A branch is tried before its pull request (novox/hq issue 686): the repository's own check asked for a pushed
// branch or commit, with no pull request.
// What a try asks is a check — after deliveries in the build queue, bounded as one — of the repository's own
// script alone: no module of the graph, so no gate; nothing built for a merge, so no reusable image.
func TestATryAsksACheckOfTheRepositorysOwnScriptAlone(t *testing.T) {
request := link.BuildRequest{ID: "build-1", Ref: "c0ffee", Check: &link.CheckRequest{Owner: "novox", Repo: "a",
Number: 7, Base: "main", Modules: []string{"a"}, New: []string{"b"}, Manifests: []string{"module.json"},
Dependents: []string{"c"}, Builds: []link.CheckBuild{{Module: "a"}}, Judge: link.JudgeSelf,
Plan: &link.ChangePlan{}}}
tried := asATry(request, "^TestStore", true)
c := tried.Check
if !c.Try || c.Number != 0 || c.Tests != "^TestStore" || !c.StoreOnly {
t.Fatalf("not marked a try, or it kept a pull request's number: %+v", c)
}
if len(c.Modules)+len(c.New)+len(c.Manifests)+len(c.Dependents)+len(c.Builds) != 0 || c.Judge != "" || c.Plan != nil {
t.Fatalf("a try runs the gate or builds for a merge: %+v", c)
}
if c.Base != "main" || c.Owner != "novox" || tried.Ref != "c0ffee" {
t.Fatalf("a try lost what it merges into what: %+v", tried)
}
if link.KindOf(tried) != link.KindCheck {
t.Fatal("a try does not count as a check in the build queue")
}
}
// A try's outcome is heard and left there: no `checked` is said, so the forge's holder sets no status, and
// nothing is recorded or registered (the handler holds no store here, and would fail reaching for one).
func TestATrysOutcomeSetsNoStatusAndRegistersNothing(t *testing.T) {
said := &saidOnTheBus{}
checkEvents = said
defer func() { checkEvents = nil }()
for name, result := range map[string]link.BuildResult{
"a try": {ID: "build-1", Ref: "c0ffee", Checked: &link.CheckRequest{Owner: "novox", Repo: "a", Try: true},
Check: &link.CheckOutcome{Verdict: "pass", RepoCheck: &link.CheckLayer{Verdict: "fail", Summary: "failed"}}},
"a try that could not run": {ID: "build-2", Checked: &link.CheckRequest{Owner: "novox", Repo: "a", Try: true},
Failed: "cannot clone"},
// A build seat from before the mark echoes the ask without it: a check of no pull request and no
// group is still nobody's status.
"a try echoed by an older build seat": {ID: "build-3", Checked: &link.CheckRequest{Owner: "novox", Repo: "a"},
Check: &link.CheckOutcome{Verdict: "pass"}},
} {
if err := (builds{}).Built(context.Background(), result); err != nil {
t.Fatalf("%s: %v", name, err)
}
}
if len(said.events) != 0 {
t.Fatalf("a try's outcome was said for the forge: %v", said.events)
}
// A pull request's check is still said.
_ = (builds{}).Built(context.Background(), link.BuildResult{ID: "build-4", Checked: &link.CheckRequest{
Owner: "novox", Repo: "a", Number: 7}, Check: &link.CheckOutcome{Verdict: "pass"}})
if len(said.events) != 1 {
t.Fatalf("a pull request's check was said %d time(s)", len(said.events))
}
}
// saidOnTheBus keeps the seat events said, and nothing else.
type saidOnTheBus struct {
link.Bus
events []string
}
func (s *saidOnTheBus) PublishSeatEvent(_ context.Context, seat, event string, _ []byte) error {
s.events = append(s.events, seat+" "+event)
return nil
}
// A ref the forge does not hold is refused before anything is asked, naming it.
func TestATryOfAnUnknownRefIsRefusedLoudly(t *testing.T) {
was := forgeCommit
defer func() { forgeCommit = was }()
forgeCommit = func(_ context.Context, owner, repo, ref string) (string, string, error) {
return "", "", errors.New("404 object does not exist [id: " + ref + "]")
}
err := tryCommand(context.Background(), []string{"novox/mesh-controller", "fix/no-such-branch"})
if err == nil {
t.Fatal("a try of a ref the forge does not hold was asked")
}
for _, want := range []string{"fix/no-such-branch", "novox/mesh-controller", "nothing was asked", "404"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not say %q: %v", want, err)
}
}
for _, args := range [][]string{{"mesh-controller", "main"}, {"novox/mesh-controller"}, {"novox/a", "--upload-pack=x"},
{"novox/a", "main", "--tests", "x; rm -rf /"}, {"novox/a", "main", "--id", "build-1"}} {
if err := tryCommand(context.Background(), args); err == nil {
t.Errorf("try %v was taken", args)
}
}
}
// The answer of a try carries the lines that name what failed (novox/hq issue 460's shape) and how long it took.
func TestATrysAnswerCarriesTheFailingLines(t *testing.T) {
tried, _ := json.Marshal(link.Tried{Verdict: "fail", Summary: "its merge-check.sh failed: TestX", Took: "5m3s",
Head: "c0ffee", Trunk: "ca3f43c", Failed: []string{"--- FAIL: TestX (0.01s)", " x_test.go:12: got 1, want 2",
"FAIL\tgithub.com/novox/a/cmd\t3.1s"}})
lines := []link.BuildLine{{Seq: 1, Step: "check", Message: "novox/a#0 at c0ffee"},
{Seq: 2, Step: "output", Message: "ok github.com/novox/a/internal 1.2s"},
{Seq: 3, Step: link.StepTried, Message: string(tried)}, {Seq: 4, Step: "checked", Message: "pass: x"}}
a := tryAnswerOf("build-1", lines)
if a.State != "fail" || a.Took != "5m3s" || len(a.Failed) != 3 || a.Failed[0] != "--- FAIL: TestX (0.01s)" ||
a.Trunk != "ca3f43c" || a.ID != "build-1" {
t.Fatalf("the answer lost the verdict, the time or the failing lines: %+v", a)
}
// Not answered yet: said as running, with how far it is — never as a pass.
if a := tryAnswerOf("build-1", lines[:2]); a.State != "running" || a.Last == "" {
t.Fatalf("a try still running: %+v", a)
}
if a := tryAnswerOf("build-1", nil); a.State != "waiting" {
t.Fatalf("a try nothing was said of: %+v", a)
}
// One that could not run is an error with the build seat's word, never a pass.
if a := tryAnswerOf("build-1", append(lines[:1:1], link.BuildLine{Seq: 2, Step: "failed",
Message: "cannot clone"})); a.State != "error" || !strings.Contains(a.Said, "cannot clone") {
t.Fatalf("a try that could not run: %+v", a)
}
}
// One try of a commit at a time, and four in all (novox/hq issue 686): a second ask is answered the first's id,
// a fifth is refused; a try that ended, and a pull request's check, count for nothing.
func TestTriesAreBoundedInNumber(t *testing.T) {
ask := func(id, repo, ref, state string, try bool) link.QueuedAsk {
return link.QueuedAsk{ID: id, State: state, Request: link.BuildRequest{ID: id, Ref: ref,
Check: &link.CheckRequest{Owner: "novox", Repo: repo, Try: try}}}
}
q := link.Queue{Asks: []link.QueuedAsk{
ask("t1", "a", "c1", link.AskInFlight, true), ask("t2", "a", "c2", link.AskWaiting, true),
ask("t3", "b", "c1", link.AskDead, true), ask("p1", "a", "c9", link.AskWaiting, false)}}
if same, room := triesIn(q, "novox", "a", "c2"); same != "t2" || !room {
t.Fatalf("a second try of a commit: %q %v", same, room)
}
if same, room := triesIn(q, "novox", "b", "c1"); same != "" || !room {
t.Fatalf("a dead try, or another repository's, was taken for this one: %q %v", same, room)
}
if same, _ := triesIn(q, "novox", "a", "c9"); same != "" {
t.Fatalf("a pull request's check was taken for a try: %q", same)
}
q.Asks = append(q.Asks, ask("t4", "c", "c3", link.AskWaiting, true), ask("t5", "d", "c4", link.AskWaiting, true))
if _, room := triesIn(q, "novox", "e", "c5"); room {
t.Fatal("a fifth try was let in")
}
// An id is checked before it is a token of a bus subject.
for _, id := range []string{"x.>", "*", "a b"} {
if err := tryCommand(context.Background(), []string{"--id", id}); err == nil ||
!strings.Contains(err.Error(), "not a try's id") {
t.Errorf("try --id %q: %v", id, err)
}
}
}
// The verb runs its command, and is declared with the shell habit it replaces (novox/hq ADR 0245).
func TestTheTryVerbRunsItsCommand(t *testing.T) {
for _, c := range []struct {
args map[string]any
want []string
}{
{map[string]any{"repository": "novox/a", "ref": "fix/x"}, []string{"try", "novox/a", "fix/x"}},
{map[string]any{"repository": "novox/a", "ref": "fix/x", "base": "main", "tests": "^TestX$", "store-only": "true"},
[]string{"try", "novox/a", "fix/x", "--base", "main", "--tests", "^TestX$", "--store-only"}},
{map[string]any{"id": "build-1"}, []string{"try", "--id", "build-1"}},
} {
got, err := argvFor("try", c.args)
if err != nil || !reflect.DeepEqual(got, c.want) {
t.Errorf("try %v: %v %v, want %v", c.args, got, err, c.want)
}
}
if _, err := argvFor("try", map[string]any{"repository": "novox/a"}); err == nil {
t.Error("a try of no ref was taken")
}
for _, v := range catalogue.ControllerVerbs {
if v.Name == "try" {
if len(v.Replaces) == 0 {
t.Error("try names no shell habit it replaces")
}
return
}
}
t.Error("try is not a verb of the controller seat")
}
+1 -1
View File
@@ -79,7 +79,7 @@ func (f following) Upgraded(ctx context.Context, u link.Upgraded) error {
// `rebuild`, a `replay` registered — was sent here one machine after another without any judging; it
// now waits for a gate like any other build, and the release plan walks it through the machines, one
// at a time, each judged.
fmt.Printf("%s moved to %s outside a plan; a release plan sends it to %s, one machine at a time, each judged "+
fmt.Printf("%s moved to %s outside a plan; a release plan sends it to %s, one machine first and then the rest at once, each judged "+
"(`upgrade backlog` lists what waits)\n", u.Module, shortCommit(u.Commit), readableList(on))
return nil
}
+6
View File
@@ -177,6 +177,12 @@ func walkReached(p inventory.Plan, running func(string) []string) map[string][]s
reached = appendNew(reached, r.Gate.Machines...)
carried(r.Gate)
}
for _, g := range r.Gates {
if g != nil && !g.Pending {
reached = appendNew(reached, g.Machines...)
carried(g)
}
}
for _, tier := range p.Tiers {
for _, m := range tier {
for _, n := range running(m) {
+24 -13
View File
@@ -268,14 +268,7 @@ func build(ctx context.Context, run Runner, publish Publisher,
}
// What is judged only where a definition is written (novox/hq ADR 0326 §3, issue 574): a build is
// where a changed definition is first read, so it is refused here before anything is made of it.
// Judged with each template's text in its file, as written in the tree (novox/hq ADR 0349 §2): a template the
// module's directory lacks is refused here, before anything is built.
judged, err := catalogue.WithTemplatesFrom(manifest, within)
if err != nil {
say("manifest", "INVALID: %v", err)
return Result{}, err
}
if problems := catalogue.WrittenSettingsProblems(judged); len(problems) > 0 {
if problems := catalogue.WrittenSettingsProblems(manifest); len(problems) > 0 {
err := fmt.Errorf("%s", strings.Join(problems, "; "))
say("manifest", "INVALID: %v", err)
return Result{}, err
@@ -417,11 +410,6 @@ func build(ctx context.Context, run Runner, publish Publisher,
}
}
// Each template's text, read back from the archive the build made or took (novox/hq ADR 0349 §2).
if built, err = withTemplateTexts(manifest, within, built); err != nil {
say("templates", "FAILED: %v", err)
return Result{}, err
}
resolved, err := manifest.Resolve(built)
if err != nil {
say("resolve", "FAILED: %v", err)
@@ -1088,6 +1076,16 @@ func one(ctx context.Context, run Runner, publish Publisher,
return catalogue.Built{}, fmt.Errorf("%s: building %s failed: %w", module, a.Name, err)
}
say("image", "built, publishing")
// Its bases linked into its repository first, so the push needs none of their blobs from this
// machine's store, which may not hold them (novox/hq issue 694). Said and not fatal: a push
// from a store that holds them works without, and one that fails says which blob it lacked.
if lender, can := publish.(BaseLender); can {
if n, err := lender.LendBases(ctx, module+"/"+a.Name, buildArgValues(args)); err != nil {
say("image", "its bases could not all be linked into %s/%s before the push: %v", module, a.Name, err)
} else if n > 0 {
say("image", "%d blob(s) of its bases linked into %s/%s before the push", n, module, a.Name)
}
}
reference, err := publish.PublishImage(ctx, local, module+"/"+a.Name)
if err != nil {
return catalogue.Built{}, err
@@ -1439,6 +1437,19 @@ func standingOn(ctx context.Context, manifest catalogue.Manifest, held map[strin
return args, resolved, nil
}
// buildArgValues is what each --build-arg of a build was handed: the bases, as this mesh holds them.
func buildArgValues(args []string) []string {
var values []string
for i := 0; i+1 < len(args); i += 2 {
if args[i] == "--build-arg" {
if _, value, has := strings.Cut(args[i+1], "="); has {
values = append(values, value)
}
}
}
return values
}
// compile runs a module's own code through its toolchain, and says where the result is.
//
// **In the module's own directory, under the path the toolchain expects.** A module is compiled
+23
View File
@@ -103,6 +103,23 @@ type CheckSpec struct {
// for the walk to take. None for a group's composed check, whose members' own checks build.
Builds []CheckBuildSpec
BuildWith *CheckBuilding
// Tests and StoreOnly narrow what the repository's own script runs, for a branch tried before its pull
// request (novox/hq issue 686): handed to the script as EnvTests and EnvStoreOnly, which it may read.
Tests string
StoreOnly bool
}
// selection is what a check hands the repository's script to narrow what it runs (novox/hq issue 686): nothing
// for a pull request's check, which always runs whole.
func selection(spec CheckSpec) []string {
var env []string
if spec.Tests != "" {
env = append(env, EnvTests+"="+spec.Tests)
}
if spec.StoreOnly {
env = append(env, EnvStoreOnly+"=1")
}
return env
}
// GroupHead is one other head of a delivery group's composed check.
@@ -193,6 +210,11 @@ const (
// EnvGroup is a delivery group's other heads, as JSON, for the gate (novox/hq ADR 0239); empty for a
// pull request checked alone.
EnvGroup = "MESH_CHECK_GROUP"
// EnvTests is a `go test -run` pattern and EnvStoreOnly is 1 when a branch tried before its pull request
// asks for only those tests, or only the store tests (novox/hq issue 686). Never set for a pull request's
// check. A script that reads neither runs whole.
EnvTests = "MESH_CHECK_TESTS"
EnvStoreOnly = "MESH_CHECK_STORE_ONLY"
)
// CheckTimeout bounds one check; a check that runs past it is an error, not a pass.
@@ -515,6 +537,7 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
EnvModules + "=" + strings.Join(append(append([]string{}, spec.Modules...), spec.New...), ","),
EnvGroup + "=" + groupFile,
"GOCACHE=" + filepath.Join(workspace, "go-cache"), "GOMODCACHE=" + filepath.Join(workspace, "go-modules")}
env = append(env, selection(spec)...)
v := CheckVerdict{}
modules := append(append([]string{}, spec.Modules...), prefixed("new:", spec.New)...)
+12
View File
@@ -590,3 +590,15 @@ func TestARepositoryInTwoLanguagesIsCheckedInBoth(t *testing.T) {
t.Errorf("a part in a toolchain the mesh does not hold answered %+v — never a pass", layer)
}
}
// A branch tried before its pull request may narrow what the repository's script runs (novox/hq issue 686); a
// pull request's check is handed nothing, and runs whole.
func TestOnlyATrysSelectionReachesTheScript(t *testing.T) {
if env := selection(CheckSpec{Number: 7}); len(env) != 0 {
t.Fatalf("a pull request's check was handed a selection: %v", env)
}
env := selection(CheckSpec{Tests: "^TestX$", StoreOnly: true})
if len(env) != 2 || env[0] != "MESH_CHECK_TESTS=^TestX$" || env[1] != "MESH_CHECK_STORE_ONLY=1" {
t.Fatalf("a try's selection: %v", env)
}
}
+178
View File
@@ -0,0 +1,178 @@
package builder
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"runtime"
"strings"
)
// **A built image's bases are linked into its repository before it is pushed** (novox/hq issue 694).
//
// A push reads every blob of the image out of this machine's store, unless the repository it goes to
// already holds that blob: the runtime asks the registry first, blob by blob, and reads locally only
// what the registry lacks. A runtime keeping its images in containerd's store — the default for a
// fresh install from docker 29 on — does not always hold the compressed layers of the base an image
// was built FROM: a layer whose unpacked form it already had is never fetched. The push of such an
// image then fails on a layer this machine never had ("not all of them are available locally"), while
// the very bytes sit in this registry, under the repository the base was mirrored into.
//
// So the base's blobs are mounted into the image's repository first. A mount moves no bytes, and once
// the repository holds a blob the push asks this machine for nothing of it — whichever store the
// runtime keeps. A runtime that holds every layer (the classic store) is not changed by this: its push
// finds the same blobs already there.
// BaseLender links what a build's bases hold into the repository its image is about to be pushed to.
type BaseLender interface {
// LendBases mounts every blob of each base this registry holds — for the platform this machine
// builds — into `repository`, and answers how many it mounted.
LendBases(ctx context.Context, repository string, bases []string) (int, error)
}
// platformed is an index's entry: a manifest and the platform it is for.
type platformed struct {
descriptor
Platform *struct {
OS string `json:"os"`
Architecture string `json:"architecture"`
} `json:"platform"`
}
// LendBases mounts each base's blobs into the repository. A base this registry does not hold under its
// repository — named by tag, or held somewhere else — lends nothing and is no error: the push reads it
// from this machine's store as before.
func (r Registry) LendBases(ctx context.Context, repository string, bases []string) (int, error) {
mounted := 0
for _, base := range bases {
name, digest, pinned := strings.Cut(base, "@")
if !pinned || !strings.HasPrefix(digest, "sha256:") {
continue
}
where, err := parseReference(name + "@" + digest)
if err != nil {
continue
}
from := where.repository
if from == repository {
continue
}
blobs, held, err := r.blobsOf(ctx, from, digest)
if err != nil {
return mounted, fmt.Errorf("reading %s: %w", base, err)
}
if !held {
continue
}
for _, blob := range blobs {
did, err := r.mount(ctx, repository, from, blob)
if err != nil {
return mounted, fmt.Errorf("linking %s of %s into %s: %w", blob, base, repository, err)
}
if did {
mounted++
}
}
}
return mounted, nil
}
// blobsOf is the config and layers of the manifest under repository@digest — for an index, of each
// manifest it names for this machine's platform. False when the registry holds no such manifest there.
func (r Registry) blobsOf(ctx context.Context, repository, digest string) ([]string, bool, error) {
body, held, err := r.manifest(ctx, repository, digest)
if err != nil || !held {
return nil, held, err
}
var document struct {
Manifests []platformed `json:"manifests"`
Config *descriptor `json:"config"`
Layers []descriptor `json:"layers"`
}
if err := json.Unmarshal(body, &document); err != nil {
return nil, false, fmt.Errorf("%s@%s is not a manifest: %w", repository, digest, err)
}
var blobs []string
if document.Config != nil {
blobs = append(blobs, document.Config.Digest)
}
for _, l := range document.Layers {
blobs = append(blobs, l.Digest)
}
for _, m := range document.Manifests {
if m.Platform == nil || m.Platform.OS != runtime.GOOS || m.Platform.Architecture != runtime.GOARCH {
continue
}
more, held, err := r.blobsOf(ctx, repository, m.Digest)
if err != nil {
return nil, false, err
}
if held {
blobs = append(blobs, more...)
}
}
return blobs, true, nil
}
// manifest reads one manifest document from this registry; false when it is not there.
func (r Registry) manifest(ctx context.Context, repository, digest string) ([]byte, bool, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodGet,
"http://"+r.Address+"/v2/"+repository+"/manifests/"+digest, nil)
if err != nil {
return nil, false, err
}
request.Header.Set("Accept", manifestAccept)
response, err := r.client().Do(request)
if err != nil {
return nil, false, fmt.Errorf("cannot reach the registry at %s: %w", r.Address, err)
}
defer response.Body.Close()
if response.StatusCode == http.StatusNotFound {
return nil, false, nil
}
if response.StatusCode != http.StatusOK {
return nil, false, fmt.Errorf("%s/%s@%s: %s", r.Address, repository, digest, response.Status)
}
body, err := io.ReadAll(io.LimitReader(response.Body, 4<<20))
return body, err == nil, err
}
// mount links one blob from one repository into another; false when the repository already held it.
func (r Registry) mount(ctx context.Context, repository, from, digest string) (bool, error) {
base := "http://" + r.Address + "/v2/" + repository
if there, err := r.has(ctx, base+"/blobs/"+digest); err != nil {
return false, err
} else if there {
return false, nil
}
start, err := http.NewRequestWithContext(ctx, http.MethodPost,
base+"/blobs/uploads/?mount="+digest+"&from="+from, nil)
if err != nil {
return false, err
}
begun, err := r.client().Do(start)
if err != nil {
return false, fmt.Errorf("cannot reach the registry at %s: %w", r.Address, err)
}
begun.Body.Close()
if begun.StatusCode == http.StatusCreated {
return true, nil
}
if begun.StatusCode == http.StatusAccepted {
// Not mounted, and an upload begun in its place: let go of it, so nothing is left half-open.
if location := begun.Header.Get("Location"); location != "" {
if strings.HasPrefix(location, "/") {
location = "http://" + r.Address + location
}
if cancel, err := http.NewRequestWithContext(ctx, http.MethodDelete, location, nil); err == nil {
if done, err := r.client().Do(cancel); err == nil {
done.Body.Close()
}
}
}
return false, fmt.Errorf("%s did not mount it from %s", r.Address, from)
}
return false, fmt.Errorf("%s answered %s when asked to mount it from %s", r.Address, begun.Status, from)
}
+140
View File
@@ -0,0 +1,140 @@
package builder
import (
"context"
"encoding/json"
"net/http/httptest"
"runtime"
"strings"
"testing"
)
// A built image's bases are linked into its repository before it is pushed (novox/hq issue 694): a
// runtime keeping containerd's image store may not hold a base layer it never had to fetch, and its push
// asks this machine only for what the repository lacks.
// aBaseForThisMachine puts an index into the registry under `repository`: one manifest for this
// machine's platform, one for another, and answers the index's digest and each manifest's blobs.
func aBaseForThisMachine(t *testing.T, dst *aRegistryOfRepositories, repository string) (string, []string, []string) {
t.Helper()
put := func(arch string) (string, []string) {
config := []byte(`{"architecture":"` + arch + `"}`)
layer := []byte(arch + " layer bytes")
for _, b := range [][]byte{config, layer} {
dst.blobs[digestOf(b)] = b
dst.link(repository, digestOf(b))
}
m, _ := json.Marshal(map[string]any{
"schemaVersion": 2, "mediaType": mediaManifestOCI,
"config": map[string]any{"mediaType": "application/vnd.oci.image.config.v1+json", "digest": digestOf(config), "size": len(config)},
"layers": []map[string]any{{"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip", "digest": digestOf(layer), "size": len(layer)}},
})
dst.manifests[repository+"@"+digestOf(m)] = m
return digestOf(m), []string{digestOf(config), digestOf(layer)}
}
other := "s390x"
if runtime.GOARCH == other {
other = "amd64"
}
mine, mineBlobs := put(runtime.GOARCH)
theirs, theirBlobs := put(other)
index, _ := json.Marshal(map[string]any{
"schemaVersion": 2, "mediaType": mediaIndexOCI,
"manifests": []map[string]any{
{"mediaType": mediaManifestOCI, "digest": mine, "size": 1, "platform": map[string]string{"os": runtime.GOOS, "architecture": runtime.GOARCH}},
{"mediaType": mediaManifestOCI, "digest": theirs, "size": 1, "platform": map[string]string{"os": "linux", "architecture": other}},
},
})
dst.manifests[repository+"@"+digestOf(index)] = index
return digestOf(index), mineBlobs, theirBlobs
}
func TestABuildsBaseIsLinkedIntoItsRepositoryBeforeThePush(t *testing.T) {
dst := newRegistryOfRepositories()
server := httptest.NewServer(dst.handler())
defer server.Close()
address := strings.TrimPrefix(server.URL, "http://")
r := Registry{Address: address}
const from = "upstream/docker.io/library/golang"
index, mine, theirs := aBaseForThisMachine(t, dst, from)
bases := []string{
address + "/" + from + "@" + index,
// Not bases this registry holds: a package version, a tag, an image held nowhere here.
"@novox/mesh-sdk@1.2.3",
"golang:1-alpine",
"elsewhere.invalid/thing@sha256:" + strings.Repeat("e", 64),
}
n, err := r.LendBases(context.Background(), "mesh-tools-go/build", bases)
if err != nil {
t.Fatal(err)
}
// Every blob of this machine's platform is in the image's repository, so a push asks this machine's
// store for none of them; another platform's are not, since nothing built here stands on them.
for _, blob := range mine {
if !dst.links["mesh-tools-go/build"][blob] {
t.Fatalf("the base's blob %s is not in the image's repository", blob)
}
}
for _, blob := range theirs {
if dst.links["mesh-tools-go/build"][blob] {
t.Fatalf("another platform's blob %s was linked", blob)
}
}
if n != len(mine) || dst.mounts != len(mine) || dst.uploads != 0 {
t.Fatalf("%d linked, %d mounts, %d uploads; want %d mounts and no bytes moved", n, dst.mounts, dst.uploads, len(mine))
}
// Asked again, nothing is mounted twice.
if n, err := r.LendBases(context.Background(), "mesh-tools-go/build", bases); err != nil || n != 0 {
t.Fatalf("a second lending mounted %d: %v", n, err)
}
}
// A registry that will not mount is said, not hidden: the push is still tried, from the store.
func TestABaseTheRegistryWillNotMountIsAnError(t *testing.T) {
dst := newRegistryOfRepositories()
dst.noMount = true
server := httptest.NewServer(dst.handler())
defer server.Close()
address := strings.TrimPrefix(server.URL, "http://")
index, _, _ := aBaseForThisMachine(t, dst, "upstream/thing")
_, err := Registry{Address: address}.LendBases(context.Background(), "hello/server",
[]string{address + "/upstream/thing@" + index})
if err == nil || !strings.Contains(err.Error(), "did not mount") {
t.Fatalf("a refused mount passed silently: %v", err)
}
}
// lending records what a build lent, and when, against the push.
type lending struct {
*mirroring
order []string
}
func (l *lending) LendBases(_ context.Context, repository string, bases []string) (int, error) {
l.order = append(l.order, "lend "+repository+" "+strings.Join(bases, " "))
return len(bases), nil
}
func (l *lending) PublishImage(ctx context.Context, localTag, repository string) (string, error) {
l.order = append(l.order, "push "+repository)
return l.mirroring.PublishImage(ctx, localTag, repository)
}
func TestAnImageIsPushedOnlyAfterItsBasesAreLent(t *testing.T) {
r, workspace := aRepository(t, anImage, map[string]string{"modules/bus/Dockerfile": "ARG BASE\nFROM ${BASE}"})
r.contents["modules/bus/module.json"] = anImage
l := &lending{mirroring: &mirroring{recorded: r, at: "127.0.0.1:5000"}}
if _, err := Build(context.Background(), r.run, l, "https://forge.invalid/catalogue.git", "modules/bus", "", workspace,
nil, Npmrc{}, GitCredential{}, nil); err != nil {
t.Fatal(err)
}
want := []string{
"lend bus/server 127.0.0.1:5000/upstream/docker.io/vendor/server@sha256:" + strings.Repeat("1", 64),
"push bus/server",
}
if strings.Join(l.order, "\n") != strings.Join(want, "\n") {
t.Fatalf("the build did:\n%s\nwant:\n%s", strings.Join(l.order, "\n"), strings.Join(want, "\n"))
}
}
+7
View File
@@ -257,6 +257,13 @@ func (s scratchPublisher) PublishArchive(ctx context.Context, repository string,
return s.to.PublishArchive(ctx, ScratchPrefix+repository, body, digest)
}
func (s scratchPublisher) LendBases(ctx context.Context, repository string, bases []string) (int, error) {
if l, can := s.to.(BaseLender); can {
return l.LendBases(ctx, ScratchPrefix+repository, bases)
}
return 0, nil
}
func (s scratchPublisher) MirrorImage(ctx context.Context, from, repository string) (string, error) {
m, can := s.to.(Mirrorer)
if !can {
+2 -14
View File
@@ -73,14 +73,8 @@ const reusedModule = `{"module":"meshboard","version":"1",
// aModuleAt is a repository holding the module under mod/, whose directory git names as tree, on its trunk.
func aModuleAt(t *testing.T, tree string) (*recorded, Runner, string) {
t.Helper()
return aModuleWith(t, tree, map[string]string{"mod/" + ManifestName: reusedModule,
"mod/Dockerfile": "FROM scratch", "mod/files/theme.conf": "dark"})
}
// aModuleWith is aModuleAt with the repository's files given.
func aModuleWith(t *testing.T, tree string, contents map[string]string) (*recorded, Runner, string) {
t.Helper()
r := &recorded{tree: tree, contents: contents}
r := &recorded{tree: tree, contents: map[string]string{"mod/" + ManifestName: reusedModule,
"mod/Dockerfile": "FROM scratch", "mod/files/theme.conf": "dark"}}
run := func(ctx context.Context, dir, name string, args ...string) (string, error) {
if name == "git" && len(args) > 0 && args[0] == "symbolic-ref" {
r.ran = append(r.ran, name+" "+strings.Join(args, " "))
@@ -106,12 +100,6 @@ var aPull = PullRef{Owner: "novox", Repo: "meshboard", Number: 7}
func aCheckBuilt(t *testing.T, scratch *aScratch, tree string) CheckBuilt {
t.Helper()
r, run, workspace := aModuleAt(t, tree)
return aCheckBuiltBy(t, scratch, r, run, workspace)
}
// aCheckBuiltBy is aCheckBuilt of the repository given.
func aCheckBuiltBy(t *testing.T, scratch *aScratch, r *recorded, run Runner, workspace string) CheckBuilt {
t.Helper()
spec := CheckSpec{ID: "build-1", Owner: aPull.Owner, Repo: aPull.Repo, Number: aPull.Number,
Builds: []CheckBuildSpec{{Module: "meshboard", Repository: "https://forge.invalid/meshboard.git", Path: "mod",
Ref: "main"}}}
-94
View File
@@ -1,94 +0,0 @@
package builder
import (
"archive/tar"
"bytes"
"compress/gzip"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"io"
"path/filepath"
"github.com/novox/mesh-controller/internal/catalogue"
)
// withTemplateTexts is what was built with each template's text, for a module whose files name templates
// (novox/hq ADR 0349 §2): its templates archive gains the text of every file in it, read back from the packed
// bytes, by its path inside the archive. Resolve reads them into the files' content.
//
// **Read from the archive, proven by its digest.** The module's files/ is packed again, as the build packs it,
// and the bytes are read only when they hash to the digest the build made or took: a fresh build's own archive,
// or a check's build a walk copied by digest without building (novox/hq ADR 0350 §2). Packing is deterministic,
// so one source packs to one digest; a tree that packs to another is not the archive, and its text is refused.
func withTemplateTexts(manifest catalogue.Manifest, within string, built []catalogue.Built) ([]catalogue.Built, error) {
if !manifest.Templated() {
return built, nil
}
name := catalogue.TemplatesArtifact(manifest)
at := -1
for i, b := range built {
if b.Name == name {
at = i
}
}
if name == "" || at < 0 {
return nil, fmt.Errorf("%s names templates and built no archive of %s/ to read them from",
manifest.Module, catalogue.TemplatesDir)
}
body, err := pack(filepath.Join(within, catalogue.TemplatesDir))
if err != nil {
return nil, fmt.Errorf("%s: packing %s/ to read its templates back failed: %w", manifest.Module,
catalogue.TemplatesDir, err)
}
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
if built[at].Digest != digest {
return nil, fmt.Errorf("%s: %s/ packs to the digest %s, and the archive %q built or taken is %s: the "+
"templates are read only from the archive the mesh records", manifest.Module, catalogue.TemplatesDir,
digest, name, orNone(built[at].Digest))
}
texts, err := unpackTexts(body)
if err != nil {
return nil, fmt.Errorf("%s: reading the archive %q back failed: %w", manifest.Module, name, err)
}
out := append([]catalogue.Built(nil), built...)
out[at].Texts = texts
return out, nil
}
// unpackTexts is every file of a packed archive (pack), by its path inside it.
func unpackTexts(body []byte) (map[string]string, error) {
zipped, err := gzip.NewReader(bytes.NewReader(body))
if err != nil {
return nil, err
}
defer zipped.Close()
reader := tar.NewReader(zipped)
texts := map[string]string{}
for {
header, err := reader.Next()
if errors.Is(err, io.EOF) {
return texts, nil
}
if err != nil {
return nil, err
}
if header.Typeflag != tar.TypeReg {
continue
}
text, err := io.ReadAll(reader)
if err != nil {
return nil, err
}
texts[header.Name] = string(text)
}
}
func orNone(digest string) string {
if digest == "" {
return "no digest"
}
return digest
}
-136
View File
@@ -1,136 +0,0 @@
package builder
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// A module's files kept as template files in its repository (novox/hq ADR 0349 §2, design 57 package T1, issue
// 654): the build packs files/ into the module's templates archive and reads each template's text back from the
// packed bytes into the file's content, so the manifest the mesh holds is the one it would hold for the text inline.
const templatedModule = `{"module":"meshboard","version":"1",
"build":{"artifacts":[
{"name":"server","kind":"image","from":"Dockerfile"},
{"name":"files","kind":"archive","from":"files"}]},
"resources":[
{"id":"svc","type":"container","name":"meshboard","artifact":"server"},
{"id":"conf","type":"file","path":"/etc/meshboard/meshboard.conf","template":"files/meshboard.conf"},
{"id":"deeper","type":"file","path":"/etc/meshboard/deeper.conf","template":"files/a/deeper.conf"}]}`
var templatedFiles = map[string]string{
"Dockerfile": "FROM scratch", "files/meshboard.conf": "listen = 8080\n\"quoted\" é\n",
"files/a/deeper.conf": "deeper\n", "files/unused.conf": "packed, named by nothing",
}
func contentOf(t *testing.T, m catalogue.Manifest, id string) string {
t.Helper()
for _, r := range m.Resources {
if r["id"] == id {
if _, has := r[catalogue.TemplateField]; has {
t.Fatalf("%s still names its template in the built manifest: %v", id, r)
}
text, ok := r["content"].(string)
if !ok {
t.Fatalf("%s carries no content: %v", id, r)
}
return text
}
}
t.Fatalf("no resource %s in %v", id, m.Resources)
return ""
}
func TestABuildReadsEachTemplateBackFromItsArchive(t *testing.T) {
r, workspace := aRepository(t, templatedModule, templatedFiles)
got, err := Build(context.Background(), r.run, r, "https://forge.invalid/meshboard.git", "", "", workspace, nil,
Npmrc{}, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if text := contentOf(t, got.Manifest, "conf"); text != templatedFiles["files/meshboard.conf"] {
t.Fatalf("the template's text is %q, not the file's", text)
}
if text := contentOf(t, got.Manifest, "deeper"); text != templatedFiles["files/a/deeper.conf"] {
t.Fatalf("the template's text is %q, not the file's", text)
}
if r.archives["meshboard/files"] == "" {
t.Fatalf("the templates archive was not published: %v", r.archives)
}
if problem := catalogue.HeldTemplateProblem(got.Manifest); problem != "" {
t.Fatal(problem)
}
}
func TestABuildWhoseTemplateIsMissingFailsNamingIt(t *testing.T) {
files := map[string]string{}
for k, v := range templatedFiles {
files[k] = v
}
delete(files, "files/a/deeper.conf")
r, workspace := aRepository(t, templatedModule, files)
_, err := Build(context.Background(), r.run, r, "https://forge.invalid/meshboard.git", "", "", workspace, nil,
Npmrc{}, GitCredential{}, nil)
if err == nil || !strings.Contains(err.Error(), "files/a/deeper.conf") {
t.Fatalf("a build whose template is missing did not fail naming it: %v", err)
}
}
// A walk that takes its check's build copies the archive by digest and builds nothing: the templates are read back
// from the tree packed again, which is the archive only when it packs to the digest taken.
func TestAWalkTakingAChecksBuildReadsTheTemplatesBackToo(t *testing.T) {
withVersion(t, "builder-1")
contents := map[string]string{"mod/" + ManifestName: templatedModule}
for k, v := range templatedFiles {
contents["mod/"+k] = v
}
scratch := newScratch()
r, run, workspace := aModuleWith(t, "tree-one", contents)
checked := aCheckBuiltBy(t, scratch, r, run, workspace)
r, run, workspace = aModuleWith(t, "tree-one", contents)
got, err := BuildReusing(t.Context(), run, r, Reuse{Scratch: scratch, Pulls: []PullRef{aPull}, Named: named(checked)},
"https://forge.invalid/meshboard.git", "mod", "main", workspace, nil, Npmrc{}, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if got.Reused == nil || len(r.archives) > 0 {
t.Fatalf("the walk built again: reused %+v, why %q, published %v", got.Reused, got.Why, r.archives)
}
if text := contentOf(t, got.Manifest, "conf"); text != templatedFiles["files/meshboard.conf"] {
t.Fatalf("the template's text is %q, not the file's", text)
}
}
// The tree packed again must be the archive taken; text read from anything else is not the archive's.
func TestTemplateTextsAreRefusedWhenTheTreeDoesNotPackToTheDigestTaken(t *testing.T) {
m, err := catalogue.ParseManifest([]byte(templatedModule))
if err != nil {
t.Fatal(err)
}
within := writeTree(t, t.TempDir(), templatedFiles)
built := []catalogue.Built{{Name: "files", Kind: catalogue.ArtifactArchive, Digest: "sha256:" + strings.Repeat("0", 64)}}
if _, err := withTemplateTexts(m, within, built); err == nil || !strings.Contains(err.Error(), "digest") {
t.Fatalf("text was read from a tree that does not pack to the archive taken: %v", err)
}
}
// writeTree writes files under dir and answers dir.
func writeTree(t *testing.T, dir string, files map[string]string) string {
t.Helper()
for name, body := range files {
full := filepath.Join(dir, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
return dir
}
-24
View File
@@ -35,10 +35,6 @@ type Built struct {
// Launchers are, for a bundle in an interpreted language, the executable the build wrote beside
// each entrypoint, by entrypoint (novox/hq ADR 0193): what the node's runtime starts to serve it.
Launchers map[string]string
// Texts are, for the archive a module packs its templates into (TemplatesArtifact), each file's text as the
// build read it back from the packed archive, by its path inside it (novox/hq ADR 0349 §2). Never carried on
// the wire or kept: Resolve reads them into the files' content, which is what the mesh holds.
Texts map[string]string `json:"-"`
}
// Resolve fills a manifest's resources in from what was built.
@@ -75,26 +71,6 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
m.Module, strings.Join(missing, " and "), oneOrOther(len(missing)))
}
// **Each template's text, as the build read it back from its templates archive** (novox/hq ADR 0349 §2): into
// the file's content, so what the mesh holds and composes is what the same text inline would be.
if m.Templated() {
name := TemplatesArtifact(m)
texts := by[name].Texts
if name == "" || texts == nil {
return Manifest{}, fmt.Errorf("%s names templates, and the build carried no text read back from the "+
"archive it packs %s/ into (%q): the builder that built it does not read templates",
m.Module, TemplatesDir, name)
}
filled, err := fillTemplates(m.Module, m.Resources, func(template string) (string, bool) {
t, ok := texts[strings.TrimPrefix(template, TemplatesDir+"/")]
return t, ok
}, fmt.Sprintf("the archive %q", name))
if err != nil {
return Manifest{}, err
}
m.Resources = filled
}
out := m
out.Build = nil
out.Resources = nil
+1 -1
View File
@@ -173,7 +173,7 @@ func TestTheBackupHoldersLinesAreDerivedFromTheData(t *testing.T) {
{"id":"meta","path":"${dir:meta}","class":"rebuildable"}]},
"resources":[{"id":"meta","type":"directory","mode":"0700"}]}`)
facts := map[string]string{"account-home": "/home/op"}
with := Rendering{Settings: SettingsBy{"media": {LayerOf("node", map[string]any{AccessesSetting: map[string]any{"films": "/tank/films"}})}}}
with := Rendering{Settings: SettingsBy{"media": {{From: "node", Values: map[string]any{AccessesSetting: map[string]any{"films": "/tank/films"}}}}}}
backup, _, err := seatContributions([]Manifest{pg, agent, media}, Manifest{}, BackupSeat+":"+BackupKindBackup, with, facts, nil)
if err != nil {
t.Fatal(err)
-3
View File
@@ -1853,9 +1853,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
}
}
problems = append(problems, m.Build.problems(m.Module)...)
// A file's text in a template file of the module's repository (novox/hq ADR 0349 §2): only a file, never beside
// content, only under files/, and packed by the module's build.
problems = append(problems, templateProblems(m)...)
problems = append(problems, undeliveredBundles(m)...)
// **What provides the artifact store cannot be delivered through it** (novox/hq 04-ISSUES/029).
//
+2 -65
View File
@@ -34,69 +34,6 @@ const PlacesSetting = "places"
// AccessesSetting is the settings key that says where a module's accesses are on this node, by id.
const AccessesSetting = "accesses"
// **Placement is not config** (novox/hq ADR 0348 clause 11, issue 651). `places`, `accesses` and
// `networks` are facts of an assignment the controller decides, held beside the assignment and apart
// from its settings: a Layer carries them in Placement, never in Values, and only Places, AccessPlaces
// and KeptNetworks read them. A person still says them as keys of a layer through `settings`; LayerOf
// splits such a layer, so what they type is routed to the placement and nothing they do changes.
// placementKeys are the keys of a layer that are placement, not config.
var placementKeys = map[string]bool{PlacesSetting: true, AccessesSetting: true, NetworksSetting: true}
// SplitPlacement divides a layer as a person says it into its config values and its placement. Each is
// nil when it holds nothing; the values themselves are not copied.
func SplitPlacement(values map[string]any) (config, placement map[string]any) {
for key, value := range values {
if placementKeys[key] {
if placement == nil {
placement = map[string]any{}
}
placement[key] = value
continue
}
if config == nil {
config = map[string]any{}
}
config[key] = value
}
return config, placement
}
// LayerOf is a layer as a person says it — config values and placement in one map — split into the two.
func LayerOf(from string, values map[string]any) Layer {
config, placement := SplitPlacement(values)
if config == nil {
config = map[string]any{}
}
return Layer{From: from, Values: config, Placement: placement}
}
// Whole is the layer as a person says it: its values and its placement in one map, a new map.
func (l Layer) Whole() map[string]any {
out := make(map[string]any, len(l.Values)+len(l.Placement))
for k, v := range l.Values {
out[k] = v
}
for k, v := range l.Placement {
out[k] = v
}
return out
}
// asSaid is the layers with placement folded back into the values, for what reads a layer whole as it
// did while placement was kept in it: today's mergeable file, which takes every key (ADR 0326 §3), and
// the judgement that no value holds a line end.
func asSaid(layers []Layer) []Layer {
out := make([]Layer, len(layers))
for i, l := range layers {
out[i] = l
if len(l.Placement) > 0 {
out[i] = Layer{From: l.From, Values: l.Whole(), Default: l.Default}
}
}
return out
}
// Placement is what an assignment says about one of a module's directories.
type Placement struct {
// Path is where the directory is on this machine. Absolute.
@@ -271,7 +208,7 @@ func Places(m Manifest, layers []Layer) (map[string]Placement, error) {
}
out := map[string]Placement{}
for _, layer := range layers {
raw, ok := layer.Placement[PlacesSetting]
raw, ok := layer.Values[PlacesSetting]
if !ok {
continue
}
@@ -341,7 +278,7 @@ func AccessPlaces(m Manifest, layers []Layer) (map[string]string, error) {
}
out := map[string]string{}
for _, layer := range layers {
raw, ok := layer.Placement[AccessesSetting]
raw, ok := layer.Values[AccessesSetting]
if !ok {
continue
}
+1 -41
View File
@@ -23,7 +23,7 @@ func placeable() Manifest {
}
func placedBy(values map[string]any) Rendering {
return Rendering{Settings: SettingsBy{"arr": {LayerOf("node anchor", values)}}}
return Rendering{Settings: SettingsBy{"arr": {{From: "node anchor", Values: values}}}}
}
func TestAnAssignmentPlacesDirectoriesAndAccesses(t *testing.T) {
@@ -179,43 +179,3 @@ func TestAnUnknownAccessReferenceIsRefusedAtParse(t *testing.T) {
t.Fatalf("an access with no id and no path was accepted: %v", err)
}
}
// **Placement is read from the placement, never from a layer's values** (novox/hq ADR 0348 clause 11, issue
// 651). A `places` or `accesses` among the config values places nothing; the same in Placement does. A
// mergeable file of today's form still takes every key of the layer as it is said, placement too, so it
// composes as it did while placement was kept in the layer.
func TestPlacementIsReadFromThePlacementOnly(t *testing.T) {
m := placeable()
said := map[string]any{
PlacesSetting: map[string]any{"config": "/services/arr/config"},
AccessesSetting: map[string]any{"series": "/storage/media/series", "spool": "/storage/downloads"},
"log": "warn",
}
among := []Layer{{From: "anchor", Values: said}}
if placed, err := Places(m, among); err != nil || placed != nil {
t.Fatalf("a place among the config values placed %v (%v)", placed, err)
}
if _, _, err := accessesFor(m, among); err == nil {
t.Fatal("an access among the config values placed it")
}
beside := []Layer{LayerOf("anchor", said)}
if beside[0].Values["log"] != "warn" || beside[0].Values[PlacesSetting] != nil || beside[0].Placement[PlacesSetting] == nil {
t.Fatalf("LayerOf split %+v", beside[0])
}
if placed, err := Places(m, beside); err != nil || placed["config"].Path != "/services/arr/config" {
t.Fatalf("the placement did not place: %v (%v)", placed, err)
}
file := map[string]any{"id": "runtime", "type": "file", "path": "/x", "content": `{"a":1}`, "merge": "json"}
whole, err := ApplySettings(file, among)
if err != nil {
t.Fatal(err)
}
split, err := ApplySettings(file, beside)
if err != nil {
t.Fatal(err)
}
if whole["content"] != split["content"] || !strings.Contains(split["content"].(string), `"places"`) {
t.Fatalf("today's mergeable file composes %q, was %q", split["content"], whole["content"])
}
}
+1 -13
View File
@@ -128,19 +128,7 @@ var defaultSeats = append([]Seat{
// The vault: the controller seals every minted credential with what it provides, which is the
// test for a seat of the mesh's own (novox/hq ADR 0161) — a second provider of `secret` is a
// second claimant, refused by name, rather than a candidate for a pin.
//
// Its one verb, `get`, hands a secret on request and reply, sealed to the key the asker sends and only
// to an asker the vault allows (novox/hq ADR 0349 §11, issue 674). Optional until the vault that runs
// serves it (design 33 §7).
{Name: "mesh-vault", Scope: ScopeMesh, Delivers: "secret", Decision: "novox/hq ADR 0161",
Serves: []Verb{
{Name: "get", Optional: true, Description: "One secret, sealed to the public key the asker sends (the " +
"sealed box of ADR 0206), with its generation; answered only to an asker the vault allows, as the " +
"bus proved it. Never the value in the clear, and never to a private key.",
Input: schema(map[string]string{"name": "the secret's name",
"public_key": "the asker's X25519 public key as SPKI PEM — never a private key"},
[]string{"name", "public_key"})},
}},
{Name: "mesh-vault", Scope: ScopeMesh, Delivers: "secret", Decision: "novox/hq ADR 0161"},
// Named for its scope since 2026-09-30 (novox/hq ADR 0156); `the-artifact-store` resolves to it as
// an alias on a mesh that predates the rename. It serves artifacts of every kind a build makes —
// images and archives, by digest — which is why the provision is the artifact store and not an
+1 -2
View File
@@ -207,8 +207,7 @@ func Effective(m Manifest, layers []Layer) []SettingSource {
Default: d.Default, HasDefault: true}
}
for _, layer := range layers {
// As the layer is said, placement too: `settings` answers it as a person set it.
for key, v := range layer.Whole() {
for key, v := range layer.Values {
s, ok := byKey[key]
if !ok {
s = &SettingSource{Key: key}
+1 -1
View File
@@ -427,7 +427,7 @@ func TestALeftOutModulesUnplaceableBackupLineCostsOnlyThatLine(t *testing.T) {
r := anAdoptedAnchor()
r.Modules = append(r.Modules, holder, media, placedBadly)
with := anchorRendering(false)
with.Settings["notes"] = []Layer{LayerOf("anchor", map[string]any{PlacesSetting: "not a map"})}
with.Settings["notes"] = []Layer{{From: "anchor", Values: map[string]any{PlacesSetting: "not a map"}}}
composed, err := r.Compose(with)
if err != nil {
t.Fatalf("an unplaceable line of a left-out module failed the machine: %v", err)
+3 -13
View File
@@ -37,11 +37,6 @@ type Layer struct {
// Default marks the layer a module's own defaults make (novox/hq ADR 0262). Marked rather than
// recognised by From, which is a node's name for a node's layer, and a node may be called anything.
Default bool
// Placement is the assignment's placement at this layer's scope — `places`, `accesses`, `networks`
// — which the controller holds apart from settings (novox/hq ADR 0348 clause 11, issue 651): read
// by Places, AccessPlaces and KeptNetworks, and never from Values. LayerOf splits a layer as a person
// says it into the two.
Placement map[string]any
}
// ApplySettings produces a resource's final content from the module's own and the layers over it.
@@ -50,10 +45,6 @@ type Layer struct {
// module first, then this node's. A node that differs from the rest is expressed by differing,
// rather than by repeating everything the rest already say.
func ApplySettings(resource map[string]any, layers []Layer) (map[string]any, error) {
// Today's form reaches a mergeable file with every key of the layer as it was said, placement too
// (novox/hq ADR 0326 §3, ADR 0348 clause 11: a module not yet moved keeps it), so a file composes
// byte for byte as it did while placement was kept in the layer.
layers = asSaid(layers)
how, _ := resource["merge"].(string)
taken, names := takenKeys(resource)
if how == "" {
@@ -404,7 +395,7 @@ func UnusedSettings(m Manifest, layers []Layer) []string {
}
var unused []string
for _, layer := range layers {
for key := range layer.Whole() {
for key := range layer.Values {
if lands[key] {
continue
}
@@ -492,7 +483,7 @@ func KeptNetworks(m Manifest, layers []Layer, adopted bool) (map[string][]string
}
out := map[string][]string{}
for _, layer := range layers {
raw, ok := layer.Placement[NetworksSetting]
raw, ok := layer.Values[NetworksSetting]
if !ok {
continue
}
@@ -552,8 +543,7 @@ var networkName = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.-]*$`)
// refused where it is stored (SetSettings, with UnusedSettings) and said where a plan is read,
// and never costs a module its place.
func JudgeSettings(m Manifest, layers []Layer, adopted bool) error {
// Placement is judged for line ends as it always was: a path is written into files as well.
if err := settingsHoldOneLine(m.Module, asSaid(layers)); err != nil {
if err := settingsHoldOneLine(m.Module, layers); err != nil {
return err
}
// Which keys each mergeable file takes is the definition's to say right (novox/hq ADR 0326);
+7 -7
View File
@@ -27,7 +27,7 @@ func TestASettingThatCannotComposeIsRefusedByNameAndLeavesOnlyItsModuleOut(t *te
if c.name == "a mesh-wide port" {
from = MeshWideLayer
}
err := JudgeSettings(web, []Layer{LayerOf(from, c.layer)}, true)
err := JudgeSettings(web, []Layer{{From: from, Values: c.layer}}, true)
if err == nil || !strings.Contains(err.Error(), c.refuse) {
t.Errorf("%s: judged %v, want %q", c.name, err, c.refuse)
}
@@ -69,8 +69,8 @@ func TestASettingThatCannotComposeIsRefusedByNameAndLeavesOnlyItsModuleOut(t *te
// declaration as the networks it also joins.
func TestAKeptNetworkReachesTheContainerOnAnAdoptedMachineOnly(t *testing.T) {
r := anAdoptedAnchor()
keep := SettingsBy{"hello-web": {LayerOf("anchor",
map[string]any{NetworksSetting: map[string]any{"server": []any{"predecessor_default"}}})}}
keep := SettingsBy{"hello-web": {{From: "anchor",
Values: map[string]any{NetworksSetting: map[string]any{"server": []any{"predecessor_default"}}}}}}
with := anchorRendering(true)
with.Settings = keep
@@ -107,13 +107,13 @@ func TestAKeptNetworkReachesTheContainerOnAnAdoptedMachineOnly(t *testing.T) {
layer Layer
want string
}{
{"mesh-wide", LayerOf(MeshWideLayer, map[string]any{NetworksSetting: map[string]any{"server": []any{"x"}}}),
{"mesh-wide", Layer{From: MeshWideLayer, Values: map[string]any{NetworksSetting: map[string]any{"server": []any{"x"}}}},
"a found network is a fact about one machine"},
{"an unknown container", LayerOf("anchor", map[string]any{NetworksSetting: map[string]any{"db": []any{"x"}}}),
{"an unknown container", Layer{From: "anchor", Values: map[string]any{NetworksSetting: map[string]any{"db": []any{"x"}}}},
`names the container "db", which it does not declare`},
{"not a list", LayerOf("anchor", map[string]any{NetworksSetting: map[string]any{"server": "x"}}),
{"not a list", Layer{From: "anchor", Values: map[string]any{NetworksSetting: map[string]any{"server": "x"}}},
"is a list of network names"},
{"not a network name", LayerOf("anchor", map[string]any{NetworksSetting: map[string]any{"server": []any{"a/b"}}}),
{"not a network name", Layer{From: "anchor", Values: map[string]any{NetworksSetting: map[string]any{"server": []any{"a/b"}}}},
"which is not a network name"},
} {
_, err := KeptNetworks(web, []Layer{c.layer}, true)
-334
View File
@@ -1,334 +0,0 @@
package catalogue
import (
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"path"
"path/filepath"
"sort"
"strings"
"unicode/utf8"
)
// A file's text kept as a template file in its module's repository (novox/hq ADR 0349 §2, design 57 package T1,
// issue 654).
//
// **Two forms, one composition.** A file resource says its text inline (`content`) or names a template file
// beside its manifest (`"template": "files/<name>"`), never both. The module's build packs `files/` into an
// archive artifact of its own — an `archive` artifact built `from: "files"` — and resolving the build reads each
// template's text back from that packed archive into `content` and drops `template`. So the manifest the mesh
// holds, composition and the node-engine are exactly as they were: a module with its text in `files/` composes
// byte-identically to the same module with it inline. Where there is no build — `module check`, the merge gate,
// `module add` — the text is read from the repository tree beside the manifest (WithTemplatesFrom).
//
// **Read back by the builder, carried in the resolved manifest.** The builder reads each template from the very
// bytes it hashed into the archive's digest, so the text the mesh holds is the archive's. The controller
// registers the resolved manifest as the build result carries it and fetches nothing from the artifact store; a
// manifest that still names a template when it would be held — built by a builder that predates templates — is
// refused at registration (HeldTemplateProblem).
// TemplateField is the key a file resource names its template file by.
const TemplateField = "template"
// TemplatesDir is the module's directory its template files are kept in, and the directory its templates archive
// is built from.
const TemplatesDir = "files"
// Templated is whether any of the manifest's resources names a template.
func (m Manifest) Templated() bool {
for _, r := range m.Resources {
if _, has := r[TemplateField]; has {
return true
}
}
return false
}
// TemplatesArtifact is the name of the archive artifact the module packs its templates into — the one built from
// files/ — or "" when it declares none, or more than one (both refused by templateProblems for a module naming a
// template).
func TemplatesArtifact(m Manifest) string {
found := templatesArtifacts(m)
if len(found) != 1 {
return ""
}
return found[0]
}
func templatesArtifacts(m Manifest) []string {
if m.Build == nil {
return nil
}
var found []string
for _, a := range m.Build.Artifacts {
if a.Kind == ArtifactArchive && a.URL == "" && cleanFrom(a.From) == TemplatesDir {
found = append(found, a.Name)
}
}
return found
}
// cleanFrom is an artifact's directory as one relative path: "./files/" is "files".
func cleanFrom(from string) string {
c := path.Clean(strings.TrimSpace(from))
return strings.TrimPrefix(c, "./")
}
// templateOf is the template a resource names, whether it names one, and what is wrong with it ("" when nothing).
func templateOf(module string, r map[string]any) (string, bool, string) {
raw, has := r[TemplateField]
if !has {
return "", false, ""
}
id := fmt.Sprint(r["id"])
if t := fmt.Sprint(r["type"]); t != "file" {
return "", true, fmt.Sprintf("%s: %s is a %s and names a template; only a file takes its text from one "+
"(novox/hq ADR 0349 §2)", module, id, t)
}
name, ok := raw.(string)
if !ok || strings.TrimSpace(name) == "" {
return "", true, fmt.Sprintf("%s: %s names a template that is not a path: %v; a template is %q",
module, id, raw, TemplatesDir+"/<name>")
}
if _, both := r["content"]; both {
return name, true, fmt.Sprintf("%s: %s names the template %q and says its content too; a file's text is "+
"in one place, the template or the content, never both (novox/hq ADR 0349 §2)", module, id, name)
}
if strings.HasPrefix(name, "/") || path.Clean(name) != name || !strings.HasPrefix(name, TemplatesDir+"/") ||
strings.Contains(name, "\\") {
return name, true, fmt.Sprintf("%s: %s names the template %q, which is not a file under %s/ of its own "+
"repository: a template is %q, a plain relative path, and the build packs only %s/",
module, id, name, TemplatesDir, TemplatesDir+"/<name>", TemplatesDir)
}
return name, true, ""
}
// templateProblems is what is wrong with the manifest's templates, as written: a template on something not a file,
// beside content, outside files/, or a module naming templates whose build packs no archive from files/. Whether
// each template exists is judged where a tree is (ReadTemplates) or the archive's text is (Resolve).
func templateProblems(m Manifest) []string {
var problems []string
templated := false
for _, r := range m.Resources {
_, has, wrong := templateOf(m.Module, r)
templated = templated || has
if wrong != "" {
problems = append(problems, wrong)
}
}
if !templated {
return problems
}
switch found := templatesArtifacts(m); len(found) {
case 0:
problems = append(problems, fmt.Sprintf("%s names templates and its build packs no archive from %s/: "+
"declare an artifact {\"name\": \"files\", \"kind\": \"archive\", \"from\": %q} (novox/hq ADR 0349 §2)",
m.Module, TemplatesDir, TemplatesDir))
case 1:
default:
sort.Strings(found)
problems = append(problems, fmt.Sprintf("%s packs %s/ into %s; its templates are read from one archive",
m.Module, TemplatesDir, strings.Join(found, " and ")))
}
return problems
}
// fillTemplates is resources with each template's text as its content and the template key dropped: text says the
// text of a template by its path ("files/<name>"), and whether it has one. A template it has no text for, or text
// that is not UTF-8, is refused by name. Every resource is copied; the manifest's own are left as they were.
func fillTemplates(module string, resources []map[string]any, text func(string) (string, bool),
where string) ([]map[string]any, error) {
out := make([]map[string]any, 0, len(resources))
var problems []string
for _, r := range resources {
name, has, wrong := templateOf(module, r)
if !has {
out = append(out, r)
continue
}
if wrong != "" {
problems = append(problems, wrong)
continue
}
body, ok := text(name)
if !ok {
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, and %s holds no such file",
module, r["id"], name, where))
continue
}
if !utf8.ValidString(body) {
// A manifest is JSON: text that is not UTF-8 would be changed on the way, silently.
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, which is not UTF-8 text; a "+
"file's text is carried as JSON and would not reach the machine as written", module, r["id"], name))
continue
}
filled := make(map[string]any, len(r))
for k, v := range r {
filled[k] = v
}
delete(filled, TemplateField)
filled["content"] = body
out = append(out, filled)
}
if len(problems) > 0 {
return nil, errors.New(strings.Join(problems, "; "))
}
return out, nil
}
// ReadTemplates is the text of each template the manifest names, read from the module's directory in a repository
// tree, by template path. Refused by name: a template the directory lacks, and anything under files/ the build's
// packing refuses — files/ itself, or any directory or file under it, that is a symlink or not a plain file — so
// nothing judged here can be read from outside the tree, and no tree passes here that the build would refuse. Each
// template is read through an os.Root of the module's directory besides, which no path element can leave.
func ReadTemplates(m Manifest, dir string) (map[string]string, error) {
problems := templatesTreeProblems(m.Module, dir)
root, err := os.OpenRoot(dir)
if err != nil {
return nil, fmt.Errorf("%s: the module's directory cannot be opened to read its templates: %w", m.Module, err)
}
defer root.Close()
texts := map[string]string{}
for _, r := range m.Resources {
name, has, wrong := templateOf(m.Module, r)
if !has {
continue
}
if wrong != "" {
problems = append(problems, wrong)
continue
}
if _, read := texts[name]; read {
continue
}
info, err := root.Lstat(name)
switch {
case errors.Is(err, fs.ErrNotExist):
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, and the module's directory "+
"has no such file", m.Module, r["id"], name))
continue
case err != nil:
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, which cannot be read: %v",
m.Module, r["id"], name, err))
continue
case !info.Mode().IsRegular():
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, which is not a plain file; "+
"the build packs only files", m.Module, r["id"], name))
continue
}
body, err := root.ReadFile(name)
if err != nil {
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, which cannot be read: %v",
m.Module, r["id"], name, err))
continue
}
texts[name] = string(body)
}
if len(problems) > 0 {
return nil, errors.New(strings.Join(problems, "; "))
}
return texts, nil
}
// templatesTreeProblems is everything under the module's files/ that the build's packing refuses: files/ itself not
// a plain directory, and any entry under it that is neither a directory nor a plain file — a symlink, to a file or a
// directory, a device, a pipe. Each said by its path. A files/ that does not exist is no problem here: each template
// it would have held is refused as missing.
func templatesTreeProblems(module, dir string) []string {
top := filepath.Join(dir, TemplatesDir)
info, err := os.Lstat(top)
if errors.Is(err, fs.ErrNotExist) {
return nil
}
if err != nil {
return []string{fmt.Sprintf("%s: %s/ cannot be read: %v", module, TemplatesDir, err)}
}
if !info.IsDir() {
return []string{fmt.Sprintf("%s: %s/ is %s, not a directory of the module's own; its templates are read "+
"only from inside the repository, and the build packs only directories and files", module, TemplatesDir,
kindOf(info.Mode()))}
}
var problems []string
err = filepath.WalkDir(top, func(at string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() || d.Type().IsRegular() {
return nil
}
rel, _ := filepath.Rel(dir, at)
problems = append(problems, fmt.Sprintf("%s: %s is %s; the build packs only directories and files, so "+
"%s/ holds nothing else", module, filepath.ToSlash(rel), kindOf(d.Type()), TemplatesDir))
return nil
})
if err != nil {
problems = append(problems, fmt.Sprintf("%s: %s/ cannot be read: %v", module, TemplatesDir, err))
}
return problems
}
// kindOf says what a file that is not a plain file is.
func kindOf(mode fs.FileMode) string {
switch {
case mode&fs.ModeSymlink != 0:
return "a symlink"
case mode&fs.ModeDevice != 0:
return "a device"
case mode&fs.ModeNamedPipe != 0:
return "a named pipe"
case mode&fs.ModeSocket != 0:
return "a socket"
case mode.IsDir():
return "a directory"
}
return "not a plain file"
}
// WithTemplatesFrom is the manifest with each template's text read from the module's directory in a repository
// tree as its content: what `module check`, the merge gate and `module add` judge, where nothing was built. Read
// again strictly afterwards, so every rule about a file's text judges the template's text exactly as it judges
// inline content. A manifest naming no template is returned as it is.
func WithTemplatesFrom(m Manifest, dir string) (Manifest, error) {
if !m.Templated() {
return m, nil
}
texts, err := ReadTemplates(m, dir)
if err != nil {
return Manifest{}, err
}
filled, err := fillTemplates(m.Module, m.Resources, func(name string) (string, bool) {
t, ok := texts[name]
return t, ok
}, "the module's directory")
if err != nil {
return Manifest{}, err
}
out := m
out.Resources = filled
raw, err := json.Marshal(out)
if err != nil {
return Manifest{}, err
}
return ParseManifest(raw)
}
// HeldTemplateProblem is why a manifest cannot be held by the mesh because it still names a template, or "": the
// manifest the mesh holds carries every file's text, which a build reads back from its templates archive. One
// naming a template was resolved by a builder that does not read templates, and would compose files with no text.
func HeldTemplateProblem(m Manifest) string {
var named []string
for _, r := range m.Resources {
if t, has := r[TemplateField]; has {
named = append(named, fmt.Sprintf("%v (%v)", r["id"], t))
}
}
if len(named) == 0 {
return ""
}
return fmt.Sprintf("%s still names the template(s) of %s: the manifest the mesh holds carries each file's "+
"text, read back from the build's templates archive — this one was resolved by a builder that does not "+
"read templates, or not built at all (novox/hq ADR 0349 §2)", m.Module, strings.Join(named, ", "))
}
-323
View File
@@ -1,323 +0,0 @@
package catalogue
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
)
// A file resource may name a template file in its module's repository (novox/hq ADR 0349 §2, design 57 package
// T1, issue 654): the build packs files/ into an archive, and resolving the build reads each template's text back
// into content, so the mesh holds, composes and sends exactly what it would for the same text inline.
// The two forms of one module: its text inline, and its text in files/.
const (
inlineTwin = `{"module":"tidy","version":"1",
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
"resources":[
{"id":"conf","type":"directory","path":"${dir:conf}"},
{"id":"main","type":"file","path":"${dir:conf}/tidy.conf","mode":"0644","content":"` + mainText + `"},
{"id":"extra","type":"file","path":"${dir:conf}/extra.conf","content":"` + extraText + `"}]}`
templatedTwin = `{"module":"tidy","version":"1",
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
"resources":[
{"id":"conf","type":"directory","path":"${dir:conf}"},
{"id":"main","type":"file","path":"${dir:conf}/tidy.conf","mode":"0644","template":"files/tidy.conf"},
{"id":"extra","type":"file","path":"${dir:conf}/extra.conf","template":"files/deeper/extra.conf"}]}`
mainText = `# tidy\nroot = ${dir:conf}\nquoted = \"yes\" é\n`
extraText = `one\ntwo\n`
)
// templateTexts is what a build reads back from the packed files/ archive: by path inside it.
func templateTexts(t *testing.T) map[string]string {
t.Helper()
var m struct{ Main, Extra string }
if err := json.Unmarshal([]byte(`{"Main":"`+mainText+`","Extra":"`+extraText+`"}`), &m); err != nil {
t.Fatal(err)
}
return map[string]string{"tidy.conf": m.Main, "deeper/extra.conf": m.Extra, "unused.conf": "not named"}
}
func builtFiles(texts map[string]string) []Built {
return []Built{{Name: "files", Kind: ArtifactArchive,
Reference: ArtifactStoreScheme + "tidy/files/blobs/sha256:" + strings.Repeat("c", 64),
Digest: "sha256:" + strings.Repeat("c", 64), Texts: texts}}
}
// held is a manifest as registration holds it: resolved with what was built, then read again strictly.
func held(t *testing.T, raw string, built []Built) Manifest {
t.Helper()
m, err := ParseManifest([]byte(raw))
if err != nil {
t.Fatal(err)
}
resolved, err := m.Resolve(built)
if err != nil {
t.Fatal(err)
}
again, err := json.Marshal(resolved)
if err != nil {
t.Fatal(err)
}
out, err := ParseManifest(again)
if err != nil {
t.Fatal(err)
}
if problem := HeldTemplateProblem(out); problem != "" {
t.Fatal(problem)
}
return out
}
func composed(t *testing.T, m Manifest) string {
t.Helper()
got, err := Resolve(shelf(m), []string{m.Module}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
out, err := json.Marshal(mustDeclare(t, got))
if err != nil {
t.Fatal(err)
}
return string(out)
}
func TestATemplatedModuleComposesByteIdenticallyToItsInlineTwin(t *testing.T) {
inline := composed(t, held(t, inlineTwin, builtFiles(nil)))
templated := composed(t, held(t, templatedTwin, builtFiles(templateTexts(t))))
if inline != templated {
t.Fatalf("the two forms compose differently:\ninline: %s\ntemplated: %s", inline, templated)
}
if !strings.Contains(templated, "quoted = \\\"yes\\\"") || strings.Contains(templated, `"template"`) {
t.Fatalf("the composition does not carry the template's text as content: %s", templated)
}
}
// What `module check`, the merge gate and `module add` judge: the text read from the tree beside the manifest.
func TestATemplatedModuleReadFromItsTreeComposesByteIdenticallyToItsInlineTwin(t *testing.T) {
dir := t.TempDir()
for name, text := range templateTexts(t) {
full := filepath.Join(dir, "files", filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte(text), 0o644); err != nil {
t.Fatal(err)
}
}
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
filled, err := WithTemplatesFrom(m, dir)
if err != nil {
t.Fatal(err)
}
if filled.Templated() {
t.Fatal("a manifest read from its tree still names a template")
}
resolved, err := filled.Resolve(builtFiles(nil))
if err != nil {
t.Fatal(err)
}
if got, want := composed(t, resolved), composed(t, held(t, inlineTwin, builtFiles(nil))); got != want {
t.Fatalf("read from the tree, the module composes differently:\ninline: %s\ntemplated: %s", want, got)
}
}
func TestAFileNamingBothATemplateAndContentIsRefused(t *testing.T) {
raw := strings.Replace(templatedTwin, `"template":"files/tidy.conf"`,
`"template":"files/tidy.conf","content":"x"`, 1)
_, err := ParseManifest([]byte(raw))
if err == nil || !strings.Contains(err.Error(), "main") || !strings.Contains(err.Error(), "never both") {
t.Fatalf("a file with both a template and content was not refused by name: %v", err)
}
}
func TestATemplateOutsideFilesIsRefusedByName(t *testing.T) {
for _, bad := range []string{"../tidy.conf", "/etc/tidy.conf", "files", "files/", "files/../tidy.conf",
"files/./tidy.conf", "other/tidy.conf", "tidy.conf", "files//tidy.conf", `files\\tidy.conf`} {
raw := strings.Replace(templatedTwin, `"files/tidy.conf"`, `"`+bad+`"`, 1)
_, err := ParseManifest([]byte(raw))
if err == nil || !strings.Contains(err.Error(), "main") || !strings.Contains(err.Error(), "not a file under files/") {
t.Errorf("the template %q was not refused by name: %v", bad, err)
}
}
}
func TestATemplateOnAResourceThatIsNotAFileIsRefused(t *testing.T) {
raw := strings.Replace(templatedTwin, `{"id":"conf","type":"directory","path":"${dir:conf}"}`,
`{"id":"conf","type":"directory","path":"${dir:conf}","template":"files/tidy.conf"}`, 1)
_, err := ParseManifest([]byte(raw))
if err == nil || !strings.Contains(err.Error(), "conf is a directory and names a template") {
t.Fatalf("a directory naming a template was not refused: %v", err)
}
}
func TestATemplatedModuleWhoseBuildPacksNoFilesIsRefused(t *testing.T) {
for _, build := range []string{
``,
`"build":{"artifacts":[{"name":"look","kind":"archive","from":"theme"}]},`,
`"build":{"artifacts":[{"name":"a","kind":"archive","from":"files"},{"name":"b","kind":"archive","from":"./files/"}]},`,
} {
raw := strings.Replace(templatedTwin,
`"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},`, build, 1)
_, err := ParseManifest([]byte(raw))
if err == nil || !strings.Contains(err.Error(), "files/") {
t.Errorf("a templated module with the build %q was not refused: %v", build, err)
}
}
}
func TestAMissingTemplateIsRefusedByName(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
// In the tree beside the manifest.
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "files"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "files", "tidy.conf"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
_, err = WithTemplatesFrom(m, dir)
if err == nil || !strings.Contains(err.Error(), "extra") || !strings.Contains(err.Error(), "files/deeper/extra.conf") {
t.Fatalf("a template missing from the tree was not refused by name: %v", err)
}
// And in what the build read back from its archive.
texts := templateTexts(t)
delete(texts, "deeper/extra.conf")
_, err = m.Resolve(builtFiles(texts))
if err == nil || !strings.Contains(err.Error(), "extra") || !strings.Contains(err.Error(), "files/deeper/extra.conf") {
t.Fatalf("a template missing from the archive was not refused by name: %v", err)
}
}
func TestATemplateThatIsASymlinkIsRefused(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "files", "deeper"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "elsewhere"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "files", "tidy.conf"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(filepath.Join(dir, "elsewhere"), filepath.Join(dir, "files", "deeper", "extra.conf")); err != nil {
t.Fatal(err)
}
if _, err := WithTemplatesFrom(m, dir); err == nil || !strings.Contains(err.Error(), "not a plain file") {
t.Fatalf("a template that is a symlink was not refused: %v", err)
}
}
func TestAResolveWithoutTheArchivesTextIsRefused(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
if _, err := m.Resolve(builtFiles(nil)); err == nil || !strings.Contains(err.Error(), "files") {
t.Fatalf("a templated module resolved with no text read back from its archive: %v", err)
}
}
func TestATemplateThatIsNotUTF8IsRefused(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
texts := templateTexts(t)
texts["tidy.conf"] = "\xff\xfe"
if _, err := m.Resolve(builtFiles(texts)); err == nil || !strings.Contains(err.Error(), "UTF-8") {
t.Fatalf("a template that is not UTF-8 was resolved: %v", err)
}
}
// What registration refuses: a manifest to be held that still names a template — one resolved by a builder that
// predates templates would otherwise compose files with no text.
func TestAManifestStillNamingATemplateIsNotHeld(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
if problem := HeldTemplateProblem(m); !strings.Contains(problem, "main") || !strings.Contains(problem, "files/tidy.conf") {
t.Fatalf("a manifest naming a template would be held: %q", problem)
}
if problem := HeldTemplateProblem(held(t, inlineTwin, builtFiles(nil))); problem != "" {
t.Fatalf("an inline manifest is refused: %s", problem)
}
}
// A symlinked files/, or a symlinked directory under it, would read a file outside the tree: refused by name,
// as is any symlink or special file under files/ that no template names, which the build's packing refuses.
func TestASymlinkAnywhereUnderFilesIsRefusedByName(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
outside := t.TempDir()
if err := os.MkdirAll(filepath.Join(outside, "deeper"), 0o755); err != nil {
t.Fatal(err)
}
for _, name := range []string{"tidy.conf", "deeper/extra.conf", "secret"} {
if err := os.WriteFile(filepath.Join(outside, filepath.FromSlash(name)), []byte("outside"), 0o644); err != nil {
t.Fatal(err)
}
}
write := func(t *testing.T, dir, name string) {
t.Helper()
full := filepath.Join(dir, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte("inside"), 0o644); err != nil {
t.Fatal(err)
}
}
link := func(t *testing.T, target, at string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(at), 0o755); err != nil {
t.Fatal(err)
}
if err := os.Symlink(target, at); err != nil {
t.Fatal(err)
}
}
for _, c := range []struct {
name string
make func(t *testing.T, dir string)
names string
}{
{"files/ itself", func(t *testing.T, dir string) {
link(t, outside, filepath.Join(dir, "files"))
}, "files/ is a symlink"},
{"a directory under files/", func(t *testing.T, dir string) {
write(t, dir, "files/tidy.conf")
link(t, filepath.Join(outside, "deeper"), filepath.Join(dir, "files", "deeper"))
}, "files/deeper is a symlink"},
{"a file no template names", func(t *testing.T, dir string) {
write(t, dir, "files/tidy.conf")
write(t, dir, "files/deeper/extra.conf")
link(t, filepath.Join(outside, "secret"), filepath.Join(dir, "files", "unnamed"))
}, "files/unnamed is a symlink"},
} {
dir := t.TempDir()
c.make(t, dir)
_, err := WithTemplatesFrom(m, dir)
if err == nil || !strings.Contains(err.Error(), c.names) {
t.Errorf("a symlinked %s was not refused by name: %v", c.name, err)
}
if err != nil && strings.Contains(err.Error(), "outside") {
t.Errorf("a symlinked %s was read: %v", c.name, err)
}
}
}
+7 -7
View File
@@ -13,22 +13,22 @@ func TestAPlacementOrAnAccessAtTheMachinesOwnIsRefused(t *testing.T) {
for _, path := range []string{"/", "/etc", "/etc/", "/etc/sudoers.d", "/usr/bin", "/root", "/var", "/var/lib",
"/home", "/home/", "/run", "/run/user/1000", "/var/lib/mesh/x", "/var/lib/mesh-host", "/var/lib/mesh-host/identity",
"/srv/../etc", "//etc", "/proc/1", "/sys", "/dev", "/boot/efi", "/bin", "/sbin", "/lib", "/lib64"} {
layers := []Layer{LayerOf("laptop", map[string]any{PlacesSetting: map[string]any{"data": path}})}
layers := []Layer{{From: "laptop", Values: map[string]any{PlacesSetting: map[string]any{"data": path}}}}
if _, err := Places(m, layers); err == nil || !strings.Contains(err.Error(), "issue 339") {
t.Errorf("a place at %s: %v", path, err)
}
layers = []Layer{LayerOf("laptop", map[string]any{AccessesSetting: map[string]any{"media": path}})}
layers = []Layer{{From: "laptop", Values: map[string]any{AccessesSetting: map[string]any{"media": path}}}}
if _, err := AccessPlaces(m, layers); err == nil || !strings.Contains(err.Error(), "issue 339") {
t.Errorf("an access at %s: %v", path, err)
}
}
for _, path := range []string{"/srv/notes", "/mnt/plex/data", "/storage/media", "/services/media/movies",
"/var/lib/notes/data", "/home/restic/repo", "/var/lib/mesh-store"} {
layers := []Layer{LayerOf("laptop", map[string]any{PlacesSetting: map[string]any{"data": path}})}
layers := []Layer{{From: "laptop", Values: map[string]any{PlacesSetting: map[string]any{"data": path}}}}
if got, err := Places(m, layers); err != nil || got["data"].Path != path {
t.Errorf("a place at %s: %v %v", path, got, err)
}
layers = []Layer{LayerOf("laptop", map[string]any{AccessesSetting: map[string]any{"media": path}})}
layers = []Layer{{From: "laptop", Values: map[string]any{AccessesSetting: map[string]any{"media": path}}}}
if got, err := AccessPlaces(m, layers); err != nil || got["media"] != path {
t.Errorf("an access at %s: %v %v", path, got, err)
}
@@ -111,17 +111,17 @@ func TestTheRuntimesDataAndAnyHomesSSHAreTheMachinesOwn(t *testing.T) {
for _, path := range []string{"/var/spool", "/var/spool/cron", "/var/lib/docker", "/var/lib/docker/volumes",
"/var/lib/containers/storage", "/var/lib/containerd", "/var/lib/containerd/io.containerd.snapshotter.v1", "/opt", "/opt/app", "/home/alice/.ssh", "/home/alice/.ssh/keys",
"/srv/backup/.ssh", "/root/.ssh"} {
layers := []Layer{LayerOf("laptop", map[string]any{PlacesSetting: map[string]any{"data": path}})}
layers := []Layer{{From: "laptop", Values: map[string]any{PlacesSetting: map[string]any{"data": path}}}}
if _, err := Places(m, layers); err == nil || !strings.Contains(err.Error(), "issue 339") {
t.Errorf("a place at %s: %v", path, err)
}
layers = []Layer{LayerOf("laptop", map[string]any{AccessesSetting: map[string]any{"media": path}})}
layers = []Layer{{From: "laptop", Values: map[string]any{AccessesSetting: map[string]any{"media": path}}}}
if _, err := AccessPlaces(m, layers); err == nil || !strings.Contains(err.Error(), "issue 339") {
t.Errorf("an access at %s: %v", path, err)
}
}
for _, path := range []string{"/var/lib/dockerish", "/home/alice/ssh", "/home/alice/.sshd-notes", "/storage/media"} {
layers := []Layer{LayerOf("laptop", map[string]any{PlacesSetting: map[string]any{"data": path}})}
layers := []Layer{{From: "laptop", Values: map[string]any{PlacesSetting: map[string]any{"data": path}}}}
if _, err := Places(m, layers); err != nil {
t.Errorf("a place at %s: %v", path, err)
}
-31
View File
@@ -23,34 +23,3 @@ func TestTheVaultsSeatDeliversSecret(t *testing.T) {
t.Fatal("a provider of secret that does not claim the seat was allowed to hold it")
}
}
// The vault answers `get` on request and reply (novox/hq ADR 0349 §11, issue 674): the seat promises the
// verb, so a holder serving it is accepted — and, optional while the vault that runs is the TypeScript one,
// so is a holder that does not yet.
func TestTheVaultsSeatPromisesGet(t *testing.T) {
seat, _ := SeatNamed("mesh-vault")
var get *Verb
for i := range seat.Serves {
if seat.Serves[i].Name == "get" {
get = &seat.Serves[i]
}
}
if get == nil {
t.Fatal("the vault's seat does not promise get")
}
for _, arg := range []string{"name", "public_key"} {
if props, _ := get.Input["properties"].(map[string]any); props[arg] == nil {
t.Errorf("get takes no %s", arg)
}
}
serving := Manifest{Module: "mesh-vault", Provides: []Offer{{Name: "secret", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-vault", Scope: ScopeMesh, Serves: []string{"get"}}}}
if err := CanHold(serving, seat); err != nil {
t.Fatalf("the vault serving get was refused: %v", err)
}
notYet := Manifest{Module: "mesh-vault", Provides: []Offer{{Name: "secret", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-vault", Scope: ScopeMesh}}}
if err := CanHold(notYet, seat); err != nil {
t.Fatalf("the vault that does not serve get yet was refused: %v", err)
}
}
+22
View File
@@ -362,6 +362,28 @@ var ControllerVerbs = []Verb{
"register": "\"true\" to register what it builds",
"older": "\"true\", with register: even though a newer build of the module is registered",
}, []string{"id"}, "register", "older")},
// novox/hq issue 686: a builder ran a branch's store tests for the first time in its pull request's merge check.
{Name: "try", Description: "Try a pushed branch or commit before its pull request (novox/hq issue 686): the build " +
"seat merges it into the repository's trunk and runs the repository's own merge-check.sh on the merge, in the " +
"toolchain containers and against the throwaway store and bus a pull request's merge check gets. Answers at " +
"once with an id; asked again with that id it waits up to 20 s and answers pass, fail or error, how long it " +
"took and the lines naming what failed — or that it is still waiting or running, to ask again. Sets no " +
"status on the forge, registers and builds nothing, runs no merge gate, counts as a check in the build queue " +
"(after deliveries and after every pull request's check, however long it waits) and is bounded as one. One " +
"try of a commit at a time — a second ask answers the first's id — and at most four waiting or running in " +
"all: a fifth is refused. Only for a repository of the mesh; the ref must be on the forge.",
Input: schema(map[string]string{
"repository": "the repository on the forge, owner/name; with ref",
"ref": "the branch or commit to try, as pushed to the forge",
"base": "the branch it is merged into (the repository's default branch when none is named). The " +
"merge-check.sh that runs is that branch's, never the ref's own: the same trust as a pull request's base",
"tests": "only the tests this `go test -run` pattern names, where the repository's merge-check.sh reads " +
"MESH_CHECK_TESTS; a script that does not runs whole",
"store-only": "\"true\": only the repository's store tests, where its merge-check.sh reads " +
"MESH_CHECK_STORE_ONLY",
"id": "instead of a repository: the id a try answered, to read what came of it",
}, nil, "store-only"),
Replaces: []string{"sh merge-check.sh", "docker run postgres"}},
// novox/hq ADR 0350 §5: asked by the delivery's owner when the forge says a pull request closed unmerged.
{Name: "forget-checks", Description: "Let go of every build a pull request's merge checks made for its merge in " +
"the artifact store's scratch namespace (novox/hq ADR 0350 §5): asked when it closed unmerged; a merged one's go " +
+19 -83
View File
@@ -117,11 +117,6 @@ func (i *Inventory) RegisterModule(ctx context.Context, m catalogue.Manifest, fr
}
}
}
// **The manifest the mesh holds carries each file's text** (novox/hq ADR 0349 §2): one still naming a template
// would compose a file with no text, so it is refused by name — a build by a builder that predates templates.
if why := catalogue.HeldTemplateProblem(m); why != "" {
return fmt.Errorf("%s is not registered: %s", m.Module, why)
}
raw, err := json.Marshal(m)
if err != nil {
return err
@@ -844,18 +839,7 @@ func (i *Inventory) KeepSettings(ctx context.Context, nodeName, module string, v
}
func (i *Inventory) setSettings(ctx context.Context, nodeName, module string, values map[string]any, judge bool, setBy string) error {
// **Placement is not config** (novox/hq ADR 0348 clause 11, issue 651): the layer is said whole, as
// it always was, and its placement keys are kept beside the assignment, never in the settings row.
// Both are written in one transaction, and the layer a set replaces is the whole of both.
config, placement := catalogue.SplitPlacement(values)
if config == nil {
config = map[string]any{}
}
raw, err := json.Marshal(config)
if err != nil {
return err
}
placed, err := placementJSON(placement)
raw, err := json.Marshal(values)
if err != nil {
return err
}
@@ -897,9 +881,6 @@ func (i *Inventory) setSettings(ctx context.Context, nodeName, module string, va
do update set values = excluded.values, set_at = now(), set_by = excluded.set_by`, module, raw, nullable(setBy)); err != nil {
return wrapModule(err, module)
}
if err := keepPlacement(ctx, tx, nil, module, placed, setBy); err != nil {
return err
}
return tx.Commit(ctx)
}
node, err := i.NodeByName(ctx, nodeName)
@@ -931,9 +912,6 @@ func (i *Inventory) setSettings(ctx context.Context, nodeName, module string, va
if err != nil {
return wrapModule(err, module)
}
if err := keepPlacement(ctx, tx, &node.ID, module, placed, setBy); err != nil {
return err
}
// A given port replaces what the mesh assigned for that port: the assignment is given back,
// so the number is free for the next module rather than held for ever in the name of a port
// that now lives elsewhere.
@@ -964,15 +942,21 @@ func (i *Inventory) judgeSettings(ctx context.Context, nodeName, module string,
return err
}
where, from, adopted = nodeName, nodeName, node.Adopted
under, has, err := i.Layer(ctx, "", module)
if err != nil {
var meshWide []byte
err = i.store.Pool().QueryRow(ctx,
`select values from settings where module = $1 and node is null`, module).Scan(&meshWide)
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
return err
}
if has {
layers = append(layers, catalogue.LayerOf(catalogue.MeshWideLayer, under))
if len(meshWide) > 0 {
var under map[string]any
if err := json.Unmarshal(meshWide, &under); err != nil {
return err
}
layers = append(layers, catalogue.Layer{From: catalogue.MeshWideLayer, Values: under})
}
}
layers = append(layers, catalogue.LayerOf(from, values))
layers = append(layers, catalogue.Layer{From: from, Values: values})
if err := catalogue.JudgeSettings(m, layers, adopted); err != nil {
return fmt.Errorf("refused: %s on %s cannot compose with the layer %q — %w", module, where, from, err)
}
@@ -1105,8 +1089,7 @@ func refuseGivenCollisions(ctx context.Context, tx pgx.Tx, nodeID any, node, mod
}
func wrapModule(err error, module string) error {
if err != nil && (strings.Contains(err.Error(), "settings_module_fkey") ||
strings.Contains(err.Error(), "placement_module_fkey")) {
if err != nil && strings.Contains(err.Error(), "settings_module_fkey") {
return fmt.Errorf("%w: %s", ErrNoSuchModule, module)
}
return err
@@ -1147,11 +1130,6 @@ func (i *Inventory) ClearSettingsBy(ctx context.Context, nodeName, module, clear
`delete from settings where module = $1 and node is not distinct from $2::uuid`, module, nodeID); err != nil {
return err
}
// And its placement, which is the same layer as a person says it (novox/hq ADR 0348 clause 11).
if _, err := tx.Exec(ctx,
`delete from placement where module = $1 and node is not distinct from $2::uuid`, module, nodeID); err != nil {
return err
}
return tx.Commit(ctx)
}
@@ -1164,13 +1142,10 @@ func (i *Inventory) SettingsFor(ctx context.Context, nodeName, module string) ([
if err != nil {
return nil, err
}
// Each layer's placement beside it, from the placement the controller holds (novox/hq ADR 0348
// clause 11): a layer's Values hold its config values only, its Placement where things are.
rows, err := i.store.Pool().Query(ctx,
`select s.node is null, s.values, p.values from settings s
left join placement p on p.module = s.module and p.node is not distinct from s.node
where s.module = $1 and (s.node is null or s.node = $2)
order by s.node is null desc`, module, node.ID)
`select node is null, values from settings
where module = $1 and (node is null or node = $2)
order by node is null desc`, module, node.ID)
if err != nil {
return nil, err
}
@@ -1179,62 +1154,23 @@ func (i *Inventory) SettingsFor(ctx context.Context, nodeName, module string) ([
var layers []catalogue.Layer
for rows.Next() {
var meshWide bool
var raw, placed []byte
if err := rows.Scan(&meshWide, &raw, &placed); err != nil {
var raw []byte
if err := rows.Scan(&meshWide, &raw); err != nil {
return nil, err
}
values := map[string]any{}
if err := json.Unmarshal(raw, &values); err != nil {
return nil, err
}
var placement map[string]any
if len(placed) > 0 {
if err := json.Unmarshal(placed, &placement); err != nil {
return nil, err
}
}
from := nodeName
if meshWide {
from = catalogue.MeshWideLayer
}
layers = append(layers, catalogue.Layer{From: from, Values: values, Placement: placement})
layers = append(layers, catalogue.Layer{From: from, Values: values})
}
return layers, rows.Err()
}
// placementJSON is a layer's placement as the placement table keeps it, or nil when it has none.
func placementJSON(placement map[string]any) ([]byte, error) {
if len(placement) == 0 {
return nil, nil
}
return json.Marshal(placement)
}
// keepPlacement writes one layer's placement beside its settings row, in the write's transaction: the
// placement the layer says, or none when it says none — a set replaces the layer whole, placement too.
func keepPlacement(ctx context.Context, tx pgx.Tx, nodeID *string, module string, placed []byte, setBy string) error {
if placed == nil {
_, err := tx.Exec(ctx,
`delete from placement where module = $1 and node is not distinct from $2::uuid`, module, nodeID)
return err
}
var err error
if nodeID == nil {
_, err = tx.Exec(ctx,
`insert into placement (node, module, values, set_by) values (null, $1, $2, $3)
on conflict (module) where node is null
do update set values = excluded.values, set_at = now(), set_by = excluded.set_by`,
module, placed, nullable(setBy))
} else {
_, err = tx.Exec(ctx,
`insert into placement (node, module, values, set_by) values ($1, $2, $3, $4)
on conflict (node, module) where node is not null
do update set values = excluded.values, set_at = now(), set_by = excluded.set_by`,
*nodeID, module, placed, nullable(setBy))
}
return wrapModule(err, module)
}
// PinProvision records which provider a machine gets a provision from: a module, and the node it
// runs on — both, always (novox/hq #258). A provision comes from a module, and the same module on
// two machines is two answers, so neither half alone says which.
-92
View File
@@ -2,7 +2,6 @@ package inventory
import (
"context"
"encoding/json"
"fmt"
"os"
"strings"
@@ -76,94 +75,3 @@ func (i *Inventory) RemoveNodeForTest(ctx context.Context, name string) (bool, e
}
return tag.RowsAffected() == 1, nil
}
// ForTestBefore is ForTest with the store migrated only below migration `number`, and a function that runs the
// rest: for a test of a migration that moves data, which seeds the store as the mesh holds it before the
// migration, migrates, and reads what the migration made of it (novox/hq issue 651).
func ForTestBefore(t *testing.T, number int) (*Inventory, func()) {
t.Helper()
admin := os.Getenv("MESH_TEST_POSTGRES")
if admin == "" {
t.Skip("no MESH_TEST_POSTGRES; run `make check` to raise one")
}
name := fmt.Sprintf("invb_%d_%s", time.Now().UnixNano()%1_000_000,
strings.ToLower(strings.NewReplacer("/", "", "-", "").Replace(t.Name())))
if len(name) > 60 {
name = name[:60]
}
all, err := Migrations()
if err != nil {
t.Fatal(err)
}
var before []store.Migration
for _, m := range all {
if m.Number < number {
before = append(before, m)
}
}
store.CreateMigratedForTest(t, admin, name, before)
busPasswordCostForTest.Store(int32(bcrypt.MinCost))
cut := strings.LastIndex(admin, "/")
t.Setenv(store.Variable(Name), admin[:cut]+"/"+name+"?sslmode=disable")
inv, err := Open(t.Context())
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
inv.Close()
c, err := pgx.Connect(context.Background(), admin)
if err != nil {
return
}
defer c.Close(context.Background())
_, _ = c.Exec(context.Background(), "drop database if exists "+name+" with (force)")
})
if err := inv.Ready(t.Context(), 20*time.Second); err != nil {
t.Fatal(err)
}
return inv, func() {
t.Helper()
if _, err := inv.store.Migrate(t.Context(), all); err != nil {
t.Fatal(err)
}
}
}
// ExecForTest runs one statement against a test's store, for a fixture no verb of the mesh writes: a row as an
// older migration left it.
func (i *Inventory) ExecForTest(t *testing.T, sql string, args ...any) {
t.Helper()
if _, err := i.store.Pool().Exec(t.Context(), sql, args...); err != nil {
t.Fatalf("%s: %v", sql, err)
}
}
// QueryForTest reads rows of a test's store, each row's columns as JSON text, for a test that checks what a
// migration left.
func (i *Inventory) QueryForTest(t *testing.T, sql string, args ...any) [][]string {
t.Helper()
rows, err := i.store.Pool().Query(t.Context(), sql, args...)
if err != nil {
t.Fatalf("%s: %v", sql, err)
}
defer rows.Close()
var out [][]string
for rows.Next() {
values, err := rows.Values()
if err != nil {
t.Fatal(err)
}
row := make([]string, len(values))
for k, v := range values {
row[k] = fmt.Sprint(v)
if b, err := json.Marshal(v); err == nil {
row[k] = string(b)
}
}
out = append(out, row)
}
if err := rows.Err(); err != nil {
t.Fatal(err)
}
return out
}
@@ -1,46 +0,0 @@
-- Placement is not config (novox/hq ADR 0348 clause 11, issue 651).
--
-- Where a module's directories are on a machine (`places`), where the operator's data it reads is
-- (`accesses`) and which found networks a taken container keeps (`networks`) were kept as keys of a
-- settings layer. They are facts of an assignment that the controller decides, never config values, and
-- the config values are leaving for the mesh-config module (ADR 0348 clause 9) while placement stays with
-- the controller. So they get a table of their own beside the assignment, and every settings layer's
-- placement keys move into it, unchanged.
--
-- Scoped exactly as the layer they came from: one machine's, or the whole mesh's (node null) for a
-- module's every assignment, which is how a mesh-wide `places` or `accesses` layer composes today. The
-- composer reads placement from here only; the `settings` verb still takes the three keys and routes
-- them here, so nothing a person types changes.
create table placement (
node uuid references node(id) on delete cascade,
module text not null references module(name) on delete cascade,
-- Only `places`, `accesses` and `networks`, as the settings layer held them.
values jsonb not null,
set_at timestamptz not null default now(),
set_by text
);
create unique index placement_for_the_mesh on placement (module) where node is null;
create unique index placement_for_a_node on placement (node, module) where node is not null;
-- Why a layer went into the history, beside how (`replaced_by`): said for a layer this migration moved.
alter table settings_history add column why text;
-- Each layer that held a placement key is kept whole in the history first, as a `settings history` reads it.
insert into settings_history (node, module, values, set_at, replaced_by, set_by, why)
select node, module, values, set_at, 'moved', set_by, 'moved to placement, ADR 0348 clause 11'
from settings
where values ?| array['places', 'accesses', 'networks'];
insert into placement (node, module, values, set_at, set_by)
select node, module,
(select jsonb_object_agg(e.key, e.value) from jsonb_each(values) as e
where e.key in ('places', 'accesses', 'networks')),
set_at, set_by
from settings
where values ?| array['places', 'accesses', 'networks'];
-- The layer keeps its other keys, when it was set and who set it; a layer that held only placement stays,
-- empty, so a layer that existed still exists.
update settings
set values = values - 'places' - 'accesses' - 'networks'
where values ?| array['places', 'accesses', 'networks'];
-222
View File
@@ -1,222 +0,0 @@
package inventory
import (
"encoding/json"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// placedLibrary is a module with each kind of placement: a directory to place, two accesses, a container that
// may keep a found network.
func placedLibrary() catalogue.Manifest {
return catalogue.Manifest{Module: "library", Version: "1",
Accesses: []catalogue.Access{{ID: "series"}, {ID: "downloads"}},
Resources: []map[string]any{
{"id": "config", "type": "directory", "mode": "0750"},
{"id": "server", "type": "container",
"image": "registry.invalid:5000/library@sha256:" + strings.Repeat("a", 64),
"volumes": []any{"${dir:config}:/config", "${access:series}:/series", "${access:downloads}:/downloads"}},
{"id": "runtime-config", "type": "file", "path": "${dir:config}/runtime.json", "mode": "0640",
"content": `{"log":"info"}`, "merge": "json"},
}}
}
// The layers of the issue (651): a mesh-wide layer with an access and a config value, a node's layer with a
// place, an access and a kept network beside a config value, a node's layer of placement only, and one with
// none.
var placementLayers = []struct {
node string
values string
}{
{"", `{"accesses": {"downloads": "/storage/downloads"}, "log": "warn"}`},
{"anchor", `{"places": {"config": {"path": "/var/lib/library/config", "owner": "1001:2000"}},
"accesses": {"series": "/storage/media/series"}, "networks": {"server": ["predecessor_default"]}, "workers": 4}`},
{"laptop", `{"accesses": {"series": "/home/media/series"}}`},
{"spare", `{"log": "debug"}`},
}
// **Placement moves out of the settings layers, and every layer reads as it did** (novox/hq ADR 0348 clause
// 11, issue 651). Migration 0097 moves each layer's places, accesses and kept networks beside it: no settings
// row holds a placement key after, every layer the settings verb and composition read is what it was, each
// moved layer is kept whole in the history with why, and a layer without placement is not touched.
func TestPlacementMovesOutOfTheSettingsLayers(t *testing.T) {
inv, migrate := ForTestBefore(t, 97)
ctx := t.Context()
if err := inv.RegisterModule(ctx, placedLibrary(), Source{}); err != nil {
t.Fatal(err)
}
for _, n := range []string{"anchor", "laptop", "spare"} {
if _, err := inv.AddNode(ctx, n); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAdopted(ctx, "anchor", true); err != nil {
t.Fatal(err)
}
// As the mesh holds them before the migration: placement among the settings.
for _, l := range placementLayers {
if l.node == "" {
inv.ExecForTest(t, `insert into settings (node, module, values, set_by) values (null, 'library', $1, 'the operator')`, l.values)
continue
}
inv.ExecForTest(t, `insert into settings (node, module, values) select id, 'library', $2 from node where name = $1`,
l.node, l.values)
}
setAt := inv.QueryForTest(t, `select coalesce(n.name, ''), s.set_at from settings s left join node n on n.id = s.node order by 1`)
migrate()
if held := inv.QueryForTest(t, `select module from settings where values ?| array['places', 'accesses', 'networks']`); len(held) != 0 {
t.Fatalf("a settings layer still holds a placement key: %v", held)
}
if after := inv.QueryForTest(t, `select coalesce(n.name, ''), s.set_at from settings s left join node n on n.id = s.node order by 1`); !reflect.DeepEqual(after, setAt) {
t.Fatalf("the layers or when they were set changed: %v, was %v", after, setAt)
}
for _, l := range placementLayers {
var want map[string]any
if err := json.Unmarshal([]byte(l.values), &want); err != nil {
t.Fatal(err)
}
got, has, err := inv.Layer(ctx, l.node, "library")
if err != nil || !has || !reflect.DeepEqual(roundTrip(t, got), want) {
t.Errorf("the layer on %q reads %v (%v, %v), was %v", l.node, got, has, err, want)
}
}
// Composition's view: the config values in Values, the placement in Placement, together the layer.
layers, err := inv.SettingsFor(ctx, "anchor", "library")
if err != nil || len(layers) != 2 {
t.Fatalf("anchor's layers: %v, %v", layers, err)
}
if layers[1].Values["workers"] == nil || layers[1].Values["places"] != nil || layers[1].Placement["places"] == nil ||
layers[1].Placement["networks"] == nil || layers[0].Placement["accesses"] == nil || layers[0].Values["log"] != "warn" {
t.Fatalf("anchor's layers split wrong: %+v", layers)
}
for i, l := range layers {
var want map[string]any
_ = json.Unmarshal([]byte(placementLayers[i].values), &want)
if !reflect.DeepEqual(roundTrip(t, l.Whole()), want) {
t.Errorf("anchor's layer %s is %v whole, was %v", l.From, l.Whole(), want)
}
}
// Judged as composition judges them, as before: anchor is adopted, so its kept network composes.
if err := catalogue.JudgeSettings(placedLibrary(), layers, true); err != nil {
t.Fatalf("the moved layers no longer compose: %v", err)
}
moved := inv.QueryForTest(t, `select coalesce(n.name, ''), h.replaced_by, h.why, h.set_by from settings_history h
left join node n on n.id = h.node order by 1`)
want := [][]string{
{`""`, `"moved"`, `"moved to placement, ADR 0348 clause 11"`, `"the operator"`},
{`"anchor"`, `"moved"`, `"moved to placement, ADR 0348 clause 11"`, `null`},
{`"laptop"`, `"moved"`, `"moved to placement, ADR 0348 clause 11"`, `null`},
}
if !reflect.DeepEqual(moved, want) {
t.Fatalf("the history of what moved: %v, want %v", moved, want)
}
past, err := inv.SettingsHistory(ctx, "anchor", "library")
if err != nil || len(past) != 1 || past[0].Why != "moved to placement, ADR 0348 clause 11" || past[0].Values["places"] == nil {
t.Fatalf("settings history on anchor: %+v, %v", past, err)
}
}
// **The settings verb's path still takes a placement key, and routes it** (issue 651): a set keeps the
// placement beside the layer and the rest in it, reads back whole, is refused as before when the placement
// cannot compose (nothing written), and a set or a clear replaces the placement with the layer.
func TestASettingRoutesPlacementBesideTheLayer(t *testing.T) {
inv := ForTest(t)
ctx := t.Context()
if err := inv.RegisterModule(ctx, placedLibrary(), Source{}); err != nil {
t.Fatal(err)
}
if _, err := inv.AddNode(ctx, "anchor"); err != nil {
t.Fatal(err)
}
// Placement alone is a layer as well: the whole mesh's says where both accesses are.
meshWide := map[string]any{"accesses": map[string]any{
"downloads": "/storage/downloads", "series": "/storage/series"}}
if err := inv.SetSettings(ctx, "", "library", meshWide); err != nil {
t.Fatal(err)
}
if got, has, _ := inv.Layer(ctx, "", "library"); !has || !reflect.DeepEqual(roundTrip(t, got), roundTrip(t, meshWide)) {
t.Fatalf("the mesh-wide layer of placement only reads %v (%v)", got, has)
}
set := map[string]any{"log": "warn",
"places": map[string]any{"config": map[string]any{"path": "/var/lib/library/config"}},
"accesses": map[string]any{"series": "/storage/media/series"}}
if err := inv.SetSettings(ctx, "anchor", "library", set); err != nil {
t.Fatal(err)
}
stored := inv.QueryForTest(t, `select s.values, p.values from settings s join placement p using (module, node)
where s.node is not null`)
if len(stored) != 1 || stored[0][0] != `{"log":"warn"}` ||
!strings.Contains(stored[0][1], `"places"`) || !strings.Contains(stored[0][1], `"accesses"`) {
t.Fatalf("the layer and its placement are kept as %v", stored)
}
if got, _, _ := inv.Layer(ctx, "anchor", "library"); !reflect.DeepEqual(roundTrip(t, got), roundTrip(t, set)) {
t.Fatalf("the layer reads %v, was set %v", got, set)
}
// Refused as before, by the same judgement, and nothing changes.
for name, bad := range map[string]map[string]any{
"an undeclared directory": {"places": map[string]any{"cache": "/var/lib/library/cache"}},
"the machine's own": {"accesses": map[string]any{"series": "/etc"}},
"a kept network on a converged machine": {
"networks": map[string]any{"server": []any{"predecessor_default"}}},
} {
if err := inv.SetSettings(ctx, "anchor", "library", bad); err == nil {
t.Errorf("%s was kept", name)
}
}
if err := inv.SetSettings(ctx, "", "library", map[string]any{"accesses": meshWide["accesses"],
"networks": map[string]any{"server": []any{"x"}}}); err == nil || !strings.Contains(err.Error(), "per node") {
t.Errorf("a mesh-wide kept network: %v", err)
}
if got, _, _ := inv.Layer(ctx, "anchor", "library"); !reflect.DeepEqual(roundTrip(t, got), roundTrip(t, set)) {
t.Fatalf("a refused set changed the layer to %v", got)
}
// A set without placement replaces it too, and the history keeps the layer whole.
if err := inv.SetSettings(ctx, "anchor", "library", map[string]any{"log": "debug"}); err != nil {
t.Fatal(err)
}
if rows := inv.QueryForTest(t, `select module from placement where node is not null`); len(rows) != 0 {
t.Fatalf("a set without placement left it: %v", rows)
}
past, err := inv.SettingsHistory(ctx, "anchor", "library")
if err != nil || len(past) != 1 || past[0].Values["places"] == nil || past[0].Values["log"] != "warn" {
t.Fatalf("the replaced layer is not kept whole: %+v, %v", past, err)
}
// The merge gate's way in keeps a layer as the facts carry it, whole: split the same way.
if err := inv.KeepSettings(ctx, "anchor", "library", map[string]any{"log": "info",
"places": map[string]any{"config": "/var/lib/library/config"}}); err != nil {
t.Fatal(err)
}
if rows := inv.QueryForTest(t, `select values from settings where values ? 'places'`); len(rows) != 0 {
t.Fatalf("a kept layer put placement among the settings: %v", rows)
}
for _, node := range []string{"", "anchor"} {
if err := inv.ClearSettings(ctx, node, "library"); err != nil {
t.Fatal(err)
}
}
if rows := inv.QueryForTest(t, `select module from placement`); len(rows) != 0 {
t.Fatalf("a clear left the placement: %v", rows)
}
}
func roundTrip(t *testing.T, v map[string]any) map[string]any {
t.Helper()
b, err := json.Marshal(v)
if err != nil {
t.Fatal(err)
}
out := map[string]any{}
if err := json.Unmarshal(b, &out); err != nil {
t.Fatal(err)
}
return out
}
+70 -2
View File
@@ -5,6 +5,7 @@ import (
"encoding/json"
"errors"
"fmt"
"slices"
"strings"
"time"
@@ -391,6 +392,65 @@ type PlanGate struct {
// reports its module's container left running because it said busy, the judging waits and has not
// started; once it reports the container replaced, Since is moved to then. Nil when nothing waited.
Busy *GateBusy `json:"busy,omitempty"`
// Pending says the send this gate judges is kept and not yet made (novox/hq ADR 0353 §8): a controller that
// takes over reads the machine's last send before making it, and never sends it twice.
Pending bool `json:"pending,omitempty"`
// Silent says the machine reported nothing on its send within ApplySilentAfter (novox/hq ADR 0353 §4): left,
// neither failed nor put back.
Silent bool `json:"silent,omitempty"`
// TookOver names the gate whose readings this one took over (novox/hq ADR 0353 §7): another walk's judging of
// the same build on the same machine, which this gate goes on from instead of starting from zero.
TookOver string `json:"took_over,omitempty"`
}
// TakeReadings makes this gate go on from another's readings of the same build on the same machine (novox/hq ADR
// 0353 §7): when the judging began, when each machine applied, the passes counted and when, and what each found.
// A module's own count is taken only for the modules this gate judges.
func (g *PlanGate) TakeReadings(from *PlanGate, modules []string, id string) {
if from == nil {
return
}
if from.Since != nil {
since := *from.Since
g.Since = &since
}
for n, s := range from.Sent {
if slices.Contains(g.Machines, n) {
if g.Sent == nil {
g.Sent = map[string]SentDeclaration{}
}
g.Sent[n] = s
}
}
for n, at := range from.Applied {
if slices.Contains(g.Machines, n) {
if g.Applied == nil {
g.Applied = map[string]time.Time{}
}
g.Applied[n] = at
}
}
g.Passes, g.Faulted, g.HealthySeen, g.Last = from.Passes, from.Faulted, from.HealthySeen, from.Last
if from.LastPass != nil {
at := *from.LastPass
g.LastPass = &at
}
for _, m := range modules {
if n, counted := from.Healthy[m]; counted {
if g.Healthy == nil {
g.Healthy = map[string]int{}
}
g.Healthy[m] = n
}
if at, counted := from.HealthyAt[m]; counted {
if g.HealthyAt == nil {
g.HealthyAt = map[string]time.Time{}
}
g.HealthyAt[m] = at
}
}
g.Readings = append([]GateReading(nil), from.Readings...)
g.TookOver = id
}
// GateBusy is a first-node gate's wait for a busy container (novox/hq ADR 0335 §4).
@@ -477,9 +537,17 @@ type CarriedMove struct {
type PlanRelease struct {
Order []string `json:"order"`
Next int `json:"next"`
// Gate is the machine being judged; nil between machines.
// Gate is the first machine being judged; nil between machines.
Gate *PlanGate `json:"gate,omitempty"`
Done []string `json:"done,omitempty"`
// Rest says the first machine passed (novox/hq ADR 0353 §1): every other machine is sent at once, each judged by
// its own gate in Gates, by machine — a build no gate passed going to one machine of a step only (§2).
Rest bool `json:"rest,omitempty"`
Gates map[string]*PlanGate `json:"gates,omitempty"`
// Failed are the machines of the rest whose gate failed, and Silent those that said nothing on their send
// within ApplySilentAfter (ADR 0353 §3 and §4).
Failed []string `json:"failed,omitempty"`
Silent []string `json:"silent,omitempty"`
Done []string `json:"done,omitempty"`
// Skipped are the machines not heard from when their turn came, left as they were.
Skipped []string `json:"skipped,omitempty"`
// By is the person who released it, empty when the mesh did.
-25
View File
@@ -1,25 +0,0 @@
package inventory
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The manifest the mesh holds carries each file's text (novox/hq ADR 0349 §2): one still naming a template — a
// build by a builder that predates templates, or a manifest handed over unbuilt — would compose a file with no
// text, and is refused by name.
func TestAManifestStillNamingATemplateIsNotRegistered(t *testing.T) {
inv := ForTest(t)
m := catalogue.Manifest{Module: "notifier", Version: "1", Resources: []map[string]any{
{"id": "conf", "type": "file", "path": "/etc/notifier.conf", "template": "files/notifier.conf"}}}
err := inv.RegisterModule(t.Context(), m, Source{})
if err == nil || !strings.Contains(err.Error(), "files/notifier.conf") {
t.Fatalf("a manifest naming a template was registered: %v", err)
}
m.Resources[0] = map[string]any{"id": "conf", "type": "file", "path": "/etc/notifier.conf", "content": "x"}
if err := inv.RegisterModule(t.Context(), m, Source{}); err != nil {
t.Fatalf("the same manifest with its text was refused: %v", err)
}
}
+7 -21
View File
@@ -20,11 +20,9 @@ func (i *Inventory) Layer(ctx context.Context, nodeName, module string) (map[str
if err != nil {
return nil, false, err
}
// Said whole, as it was set: its config values and the placement kept beside them (novox/hq ADR 0348
// clause 11), so a person reading it before setting it sees every key they may keep or remove.
var raw []byte
err = i.store.Pool().QueryRow(ctx, `select `+layerWhole+` from settings s
where s.module = $1 and s.node is not distinct from $2::uuid`,
err = i.store.Pool().QueryRow(ctx,
`select values from settings where module = $1 and node is not distinct from $2::uuid`,
module, nodeID).Scan(&raw)
if errors.Is(err, pgx.ErrNoRows) {
return nil, false, nil
@@ -48,9 +46,6 @@ type PastLayer struct {
ReplacedBy string
// SetBy is who had set the layer, when it was kept (novox/hq ADR 0277); empty for one set before that was kept.
SetBy string
// Why is why it went, where that was said: "moved to placement, ADR 0348 clause 11" for a layer whose
// placement keys migration 0097 moved out of it (ReplacedBy "moved").
Why string
}
// LayerOrigin is who set a layer and when (novox/hq ADR 0277): empty words for a layer set before who set it was
@@ -84,7 +79,7 @@ func (i *Inventory) SettingsHistory(ctx context.Context, nodeName, module string
return nil, err
}
rows, err := i.store.Pool().Query(ctx,
`select values, set_at, replaced_at, replaced_by, set_by, why from settings_history
`select values, set_at, replaced_at, replaced_by, set_by from settings_history
where module = $1 and node is not distinct from $2::uuid order by replaced_at desc`,
module, nodeID)
if err != nil {
@@ -95,16 +90,13 @@ func (i *Inventory) SettingsHistory(ctx context.Context, nodeName, module string
for rows.Next() {
var raw []byte
var p PastLayer
var by, why *string
if err := rows.Scan(&raw, &p.SetAt, &p.ReplacedAt, &p.ReplacedBy, &by, &why); err != nil {
var by *string
if err := rows.Scan(&raw, &p.SetAt, &p.ReplacedAt, &p.ReplacedBy, &by); err != nil {
return nil, err
}
if by != nil {
p.SetBy = *by
}
if why != nil {
p.Why = *why
}
if err := json.Unmarshal(raw, &p.Values); err != nil {
return nil, err
}
@@ -115,15 +107,9 @@ func (i *Inventory) SettingsHistory(ctx context.Context, nodeName, module string
// keepReplacedSQL copies a layer into the history before it is replaced or cleared; run in the same
// transaction as the write where there is one, so a write that fails leaves no history of it.
//
// The layer is kept whole, as it was set: its config values and its placement (novox/hq ADR 0348 clause 11).
const keepReplacedSQL = `insert into settings_history (node, module, values, set_at, replaced_by, set_by)
select s.node, s.module, ` + layerWhole + `, s.set_at, $3, s.set_by from settings s
where s.module = $1 and s.node is not distinct from $2::uuid`
// layerWhole is a settings row `s` with the placement kept beside it folded back in: the layer as it was set.
const layerWhole = `s.values || coalesce((select p.values from placement p
where p.module = s.module and p.node is not distinct from s.node), '{}'::jsonb)`
select node, module, values, set_at, $3, set_by from settings
where module = $1 and node is not distinct from $2::uuid`
// layerNode is the node id of a layer, nil for the whole mesh's.
func (i *Inventory) layerNode(ctx context.Context, nodeName string) (*string, error) {
+24
View File
@@ -164,6 +164,30 @@ type CheckRequest struct {
// each, from the merge it judged, into the scratch namespace once it has judged and not failed, for the walk to
// take. A build seat from before it builds nothing, and the walk builds as before.
Builds []CheckBuild `json:"builds,omitempty"`
// Try marks a branch tried before its pull request (novox/hq issue 686): the repository's own check for a
// pushed branch or commit, asked by a builder through the controller's `try` verb. Its outcome is read from
// the build's log (StepTried) and said to nobody: no status on the forge, nothing recorded or registered,
// nothing built for a merge. Tests narrows what the repository's script runs to the tests a pattern names,
// and StoreOnly to its store tests; a script that reads neither runs whole.
Try bool `json:"try,omitempty"`
Tests string `json:"tests,omitempty"`
StoreOnly bool `json:"store-only,omitempty"`
}
// StepTried is the step a build seat says a try's outcome under, in the build's log: one line, a Tried as JSON
// (novox/hq issue 686).
const StepTried = "tried"
// Tried is what came of a branch tried before its pull request (novox/hq issue 686): the verdict of the
// repository's own check for the ref merged into the trunk, how long it took, and the lines that name what
// failed, picked from the whole of its output (issue 460).
type Tried struct {
Verdict string `json:"verdict"`
Summary string `json:"summary"`
Took string `json:"took"`
Head string `json:"head,omitempty"`
Trunk string `json:"trunk,omitempty"`
Failed []string `json:"failed,omitempty"`
}
// CheckBuild is one module a check builds for its merge: from the merge it made when Merged — a module built from
+19
View File
@@ -96,8 +96,27 @@ type BuildStart struct {
// an earlier taker left unfinished — most often a build agent replaced mid-build — and started
// again from the beginning (novox/hq issue 594). 0 from a machine that predates it.
Delivery int `json:"delivery,omitempty"`
// Try marks the start of a branch tried before its pull request (novox/hq issue 686), which names no
// repository and no ref (startOf).
Try bool `json:"try,omitempty"`
}
// startOf is what a build machine says of the ask it took. **A try's start names no repository and no commit**
// (novox/hq issue 686): the forge's holder writes mesh/merge-gate pending on an open pull request whose head is
// the commit a start names, and a try of that head is no check of the pull request — nothing on the forge moves
// for a try. Its id, machine and time are all the queue reads of a start.
func startOf(r BuildRequest, on string, delivery int, at time.Time) BuildStart {
s := BuildStart{ID: r.ID, Repository: r.Repository, Path: r.Path, Ref: r.Ref, On: on,
At: at.UTC().Format(time.RFC3339Nano), Delivery: delivery}
if IsTry(r) {
s.Repository, s.Path, s.Ref, s.Try = "", "", "", true
}
return s
}
// IsTry says an ask is a branch tried before its pull request (novox/hq issue 686).
func IsTry(r BuildRequest) bool { return r.Check != nil && r.Check.Try }
// Restarted is how a build handed out again says so (novox/hq issue 594), in its log and in its check's
// report, so a second run never reads as the first. Neutral on why: a holder replaced mid-build, a paused
// holder handing an ask back as it arrived, and a bus that held its acknowledgements past the wait all
+1 -4
View File
@@ -401,10 +401,7 @@ func (b *natsBuild) Done() error { return b.msg.Ack() }
// Began publishes that this machine has taken the build, into the stream like the outcome, so a
// reader that asks afterwards sees when it started as well as how it ended.
func (b *natsBuild) Began(ctx context.Context) error {
body, err := json.Marshal(BuildStart{
ID: b.request.ID, Repository: b.request.Repository, Path: b.request.Path,
Ref: b.request.Ref, On: b.on, At: time.Now().UTC().Format(time.RFC3339Nano), Delivery: b.delivery,
})
body, err := json.Marshal(startOf(b.request, b.on, b.delivery, time.Now()))
if err != nil {
return err
}
+3
View File
@@ -427,6 +427,9 @@ func ReadCheckBuilds(ctx context.Context, js *broker.JetStream, seat string, pul
// safeKey is an id that can be a key, and a subject token, as it is: a build id, never a pattern.
var safeKey = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
// SafeBuildID says an id can be a subject token as it is: a build's id, never a pattern.
func SafeBuildID(id string) bool { return safeKey.MatchString(id) }
// Cancelled is what the controller writes for an ask it cancelled.
type Cancelled struct {
At string `json:"at"`
+9 -2
View File
@@ -26,6 +26,8 @@ import (
// - a check that has waited CheckWaitsAtMost goes before every delivery asked after it, so no check
// waits for ever behind a stream of deliveries;
// - within a kind, first come, first served, by when it was asked;
// - a branch tried before its pull request goes after everything else, however long it has waited: it is
// never promoted ahead of a delivery, and never ahead of a pull request's check (novox/hq issue 686);
// - an ask whose kind cannot be told counts as a check: a delivery taken late breaks a budget, a check
// taken late breaks nothing.
//
@@ -64,8 +66,12 @@ func (a QueuedAsk) askedOrStored() time.Time {
// QueueOrder is the order the waiting asks are to be handed out in, now: the deliveries and every check
// that has waited CheckWaitsAtMost, by when each was asked, then the other checks, by when each was asked.
func QueueOrder(waiting []QueuedAsk, now time.Time) []QueuedAsk {
var first, then []QueuedAsk
var first, then, tries []QueuedAsk
for _, a := range waiting {
if IsTry(a.Request) {
tries = append(tries, a)
continue
}
if KindOf(a.Request) == KindCheck && now.Sub(a.askedOrStored()) < CheckWaitsAtMost {
then = append(then, a)
continue
@@ -82,7 +88,8 @@ func QueueOrder(waiting []QueuedAsk, now time.Time) []QueuedAsk {
}
byAsking(first)
byAsking(then)
return append(first, then...)
byAsking(tries)
return append(append(first, then...), tries...)
}
// toMove is the asks to move to the tail, in this order, for the queue to stand as order says: everything
+55
View File
@@ -0,0 +1,55 @@
package link
import (
"encoding/json"
"reflect"
"strings"
"testing"
"time"
)
// The start of a try names no repository and no commit (novox/hq issue 686): the forge's holder writes
// mesh/merge-gate pending on the open pull request whose head a start names, and nothing on the forge moves
// for a try. A pull request's check still names both.
func TestATrysStartNamesNoCommitTheForgeCouldMatch(t *testing.T) {
sha := strings.Repeat("c0ffee", 7)[:40]
at := time.Date(2026, 10, 14, 0, 0, 0, 0, time.UTC)
try := BuildRequest{ID: "build-1", Repository: "https://forge/novox/a.git", Ref: sha,
Check: &CheckRequest{Owner: "novox", Repo: "a", Try: true}}
body, _ := json.Marshal(startOf(try, "ace", 1, at))
if strings.Contains(string(body), sha) || strings.Contains(string(body), "novox/a") {
t.Fatalf("a try's start names what the forge matches a pull request by: %s", body)
}
if s := startOf(try, "ace", 2, at); s.ID != "build-1" || s.On != "ace" || !s.Try || s.Delivery != 2 || s.At == "" {
t.Fatalf("a try's start lost what the queue reads: %+v", s)
}
pull := try
pull.Check = &CheckRequest{Owner: "novox", Repo: "a", Number: 7}
if s := startOf(pull, "ace", 1, at); s.Ref != sha || s.Repository == "" || s.Try {
t.Fatalf("a pull request's check's start: %+v", s)
}
}
// A try stays behind every delivery and every pull request's check, however long it has waited: the ten
// minutes that put a check ahead of later deliveries never promote a try (novox/hq issue 686).
func TestATryIsNeverHandedOutBeforeADeliveryOrAPullRequestsCheck(t *testing.T) {
now := time.Date(2026, 10, 12, 12, 0, 0, 0, time.UTC)
try := func(seq uint64, id string, asked time.Time) QueuedAsk {
a := anAsk(seq, id, asked, true)
a.Request.Check.Try = true
return a
}
waiting := []QueuedAsk{
try(1, "try-old", now.Add(-40*time.Minute)),
try(2, "try-young", now.Add(-time.Minute)),
anAsk(3, "check-young", now.Add(-2*time.Minute), true),
anAsk(4, "delivery", now.Add(-time.Minute), false),
}
want := []string{"delivery", "check-young", "try-old", "try-young"}
if got := idsOf(QueueOrder(waiting, now)); !reflect.DeepEqual(got, want) {
t.Fatalf("handed out as %v, want %v", got, want)
}
if !SafeBuildID("build-20261014T000000-1a2b") || SafeBuildID("x.>") || SafeBuildID("*") || SafeBuildID("") {
t.Fatal("an id that is a pattern passes for a build's id")
}
}
+19 -4
View File
@@ -54,11 +54,26 @@ if [ -n "$unformatted" ]; then
fi
CGO_ENABLED=0 go vet ./...
if command -v gcc >/dev/null 2>&1; then
CGO_ENABLED=1 go test -race -count=1 -timeout 30m ./...
suite() {
if command -v gcc >/dev/null 2>&1; then
CGO_ENABLED=1 go test -race -count=1 -timeout 30m "$@"
else
echo "NOT RACE-CHECKED: the toolchain holds no C compiler; the suite runs without the race detector"
CGO_ENABLED=0 go test -count=1 -timeout 30m "$@"
fi
}
# **A branch tried before its pull request may ask for a part** (novox/hq issue 686): the controller's `try` verb
# hands MESH_CHECK_TESTS, a `go test -run` pattern, or MESH_CHECK_STORE_ONLY=1. A pull request's merge check is
# handed neither and always runs whole; a part that passes is said as a part, never as the check.
if [ -n "${MESH_CHECK_TESTS:-}" ]; then
echo "TRIED IN PART: only the tests matching $MESH_CHECK_TESTS, in every package"
suite -run "$MESH_CHECK_TESTS" ./...
exit 0
fi
if [ "${MESH_CHECK_STORE_ONLY:-}" = 1 ]; then
echo "TRIED IN PART: only the store tests listed in testdata/store-tests"
else
echo "NOT RACE-CHECKED: the toolchain holds no C compiler; the suite runs without the race detector"
CGO_ENABLED=0 go test -count=1 -timeout 30m ./...
suite ./...
fi
# **The store tests listed ran** (novox/hq issue 459): a test that needs the store skips without one, and a skip
+1
View File
@@ -77,6 +77,7 @@
"replay",
"kill",
"forget-checks",
"try",
"pause",
"resume",
"hand-act",
+9 -3
View File
@@ -61,6 +61,12 @@
./cmd/mesh-controller TestAProviderElsewhereServesItsOwnComposedName
./cmd/mesh-controller TestAReusedCheckBuildIsRegisteredByItsDigestAndGatedAsABuiltOne
./internal/inventory TestAReportOfABusyContainerIsKeptAndReadBack
./internal/inventory TestPlacementMovesOutOfTheSettingsLayers
./internal/inventory TestASettingRoutesPlacementBesideTheLayer
./cmd/mesh-controller TestADeclarationIsTheSameAfterPlacementMoves
./cmd/mesh-controller TestAReleaseSendsTheRestAtOnceAfterTheFirstPasses
./cmd/mesh-controller TestARestMachineThatFailsIsPutBackAloneAndTheOthersAreJudgedOn
./cmd/mesh-controller TestAWalkTakesOverTheReadingsOfAGateJudgingTheSameBuildThere
./cmd/mesh-controller TestAFirstSendMadeAndNotKeptIsNotSentAgain
./cmd/mesh-controller TestAReleaseSendKeptAndMadeIsNotMadeAgain
./cmd/mesh-controller TestAFirstMachineThatFailsStillStopsTheWalk
./cmd/mesh-controller TestTheControlNodeIsSentLastInAStep
./cmd/mesh-controller TestARestMachineSilentPastItsBoundIsLeftAndSaid
./cmd/mesh-controller TestANoMoveTierWaitsForNoNewReport