Compare commits

...
Author SHA1 Message Date
jschoubben 83accdbb4e build modules: cite A-489c on the runs-counted kit tests and failurekit's refusal
Builds: design 47 G §24; lead answer A-489c

Claude-Session: https://claude.ai/code/session_01JG73KHUnBze48rdua8u2sy
2026-11-19 15:45:49 +01:00
jschoubben 7c64c17e65 build modules: every one runs the runs-counted kit case; failurekit refuses one that does not (A-489a)
Builds: design 47 G §24; lead answers A-489a, A-489b

Claude-Session: https://claude.ai/code/session_01JG73KHUnBze48rdua8u2sy
2026-11-19 15:42:44 +01:00
jschoubben 0a446c7572 build modules: cite A-489b on the step-ends kit case, its check and the node's need
Builds: design 47 G §24; lead answer A-489b

Claude-Session: https://claude.ai/code/session_01JG73KHUnBze48rdua8u2sy
2026-11-19 15:38:16 +01:00
jschoubben e1521a2ecf build modules: every one runs the step-ends kit case; failurekit refuses one that does not; image and go count runs that started
mesh-build-image's steps now carry Node and SayStep (an image has no step, §328, but the kit holds all three). Prior wired in go and image.

Builds: design 47 G §24, §328

Claude-Session: https://claude.ai/code/session_01JG73KHUnBze48rdua8u2sy
2026-11-19 15:25:38 +01:00
jschoubben 3a94021766 mesh-build-node: its steps say a step's end, lint is the type-check, it has its own need
A pipeline step ask (the sdk's lib lint and unit steps) was refused with 'this build module says no step's end': its
AskSteps set no SayStep, Node, Lint or Has. They are set as mesh-build-go sets them; lint runs tsc --noEmit over each TypeScript
package; Puller.Prior counts only runs that started toward poison.

Builds: design 47 G §328 (Lint per language), §22, §24, §25

Claude-Session: https://claude.ai/code/session_01JG73KHUnBze48rdua8u2sy
2026-11-19 15:25:38 +01:00
mesh-tools 282758456c Merge pull request 'The build modules declare what their seat-list reader uses (A-488a, A-488b)' (#705) from fix/build-node-consumes into main 2026-11-19 12:51:11 +00:00
jschoubben 0e9f916806 The three build modules consume the seat list's events and invoke its verb
mesh-build-go and mesh-build-image follow the seat list as mesh-build-node does, so each consumes mesh-catalog.seat-definition-changed and
-removed, and all three invoke seat:mesh-catalog.seat-definitions, which the reader calls.

Builds: design 47 G A-488a, A-488b (hq 57c14047)
Claude-Session: https://claude.ai/code/session_01JG73KHUnBze48rdua8u2sy
2026-11-19 13:49:07 +01:00
jschoubben daa5d3ea5f mesh-build-node consumes the seat list's events, so its runtime's consumer exists
Its runtime follows the seat list (seatlist.Start subscribes to mesh-catalog.seat-definition-changed and -removed); with no consumes
there was no consumer shanks_mesh-build-node_tools, the seat list was never read and the asks of a named asker were given back.

Builds: design 47 G A-488a (hq 25df48ad)
Claude-Session: https://claude.ai/code/session_01JG73KHUnBze48rdua8u2sy
2026-11-19 13:41:12 +01:00
mesh-tools be993cb2c5 Merge pull request 'restic: its secrets belong to the account its processes run as (ADR 0680)' (#703) from fix/restic-secrets-owner into main 2026-11-18 01:18:35 +00:00
jschoubben cd3b8fe8e9 restic: its secrets belong to the account its processes run as
Its processes run as the operator account, so its own secrets are owned by it (design 20 "Whose a process's own secrets are").

Builds: ADR 0680 §1, §4; design 20 "Whose a process's own secrets are"
2026-11-18 01:21:54 +01:00
mesh-tools f0c21ca033 Merge pull request 'feh: the second single-module proof (one comment line)' (#702) from proof/feh-two into main 2026-11-17 19:49:31 +00:00
jschoubben c6ec42bc2e feh: mark the wallpaper file with the second single-module proof
The single-module proof of design 47 G (the operator, 2026-11-17), after pack49:
one change to one leaf module, followed to delivered on g14 and shanks with no hand act.
2026-11-17 20:49:20 +01:00
mesh-tools f65d789052 Merge pull request 'feh: the wallpaper file is replaced at every delivery (single-module proof)' (#701) from proof/feh-one-line into main 2026-11-17 16:02:47 +00:00
jschoubben 62b8251b99 feh: say the wallpaper file is replaced at every delivery, the glossary's word
The single-module proof of design 47 G (the operator, 2026-11-17): one change to one module,
followed through its pipeline to delivered on g14 and shanks.
2026-11-17 17:01:16 +01:00
mesh-tools 90c88fc7b5 Merge pull request 'A-471: the build modules' tests expect the language version in a toolchain report' (#700) from fix/a471-toolchain-language-version into main 2026-11-17 14:56:47 +00:00
jschoubben 740c218139 A-471: mesh-build-go start test expects the language version
Builds: design 47 G A-471 (G-22-lead-answers-2.md)
2026-11-17 15:53:05 +01:00
jschoubben 1481465ec7 A-471: the build modules' tests expect the language version in a toolchain report
The toolchain a build module reports (guard key and toolchain-reported event) names the
language version alone (mesh-delivery-sdk steprun Guard); the build module's own
Toolchain func keeps the installed text. Delivered with or after the sdk change.

Builds: design 47 G A-471 (G-22-lead-answers-2.md)
2026-11-17 15:53:05 +01:00
mesh-tools d2b8f64c94 Merge pull request 'G-24 tests on main: mesh-build-go keeps no step-code bridge, uses a stand-in composer, and runs within its level bounds' (#699) from fix/g24-tests-on-main into main 2026-11-17 14:35:32 +00:00
jschoubben eed95cd63b mesh-build-go: the integration level runs its slowest independent tests side by side, inside its 60 s bound
The level took 67 s locally and 74 s in the pipeline, over IntegrationRun's 60 s. The eight slowest tests share no state and run in
parallel: 31 s locally with the fixture's store and bus. The real-store test makes its work directory under the test temp
directory, not /tmp.

Builds: ADR 0445 §3 (the integration level's 60 s bound)
2026-11-17 15:31:00 +01:00
jschoubben 84106a9338 mesh-build-go: the compose ask's test builds a stand-in composer from its own testdata, reading no other repository
A pipeline's test step places no repository beside the tree and a module's tests read nothing outside their module (lead answer
A-474), so the test no longer needs MESH_CHECK_BESIDE or the node-engine's tree: it fetches a small program of the composer's
contract (testdata/composer) and still proves the build once, the kept composer on the second ask, and the verdict by case.

Builds: design 47 G lead answer A-474 (principles 10 and 9)
2026-11-17 15:31:00 +01:00
jschoubben 9ed13cd9dc mesh-build-go: the empty buildstep bridge goes (G's indexer runs), and the unit level runs its digest tests side by side
The bridge kept a directory for the pre-G indexer's package publishing; its own commit says it goes once G's indexer runs, and
the test of the delivery SDK's step code refuses it. The unit level took 6.3 s over its bound of 5 s in the take-in's pipeline:
the digest tests ran their go commands one after another; the marker table is a subtest per case in a fixture of its own, the
list half a test of its own, and the six slowest independent tests run in parallel (4.6 s locally).

Builds: design 47 G §464, §680 (the build modules keep no copy of the delivery SDK step code); ADR 0445 §3 (the unit level's 5 s bound)
2026-11-17 15:31:00 +01:00
mesh-tools d76c3588e3 Merge pull request 'gate: the start comparison ends only when every head was compared (A-464e)' (#698) from fix/47-G-start-compare into main 2026-11-17 08:22:03 +00:00
jschoubben 377d42f5ea gate: the start comparison ends only when every head was compared (A-464e)
Builds: design 47 G-22 A-464e, A-464
2026-11-17 09:03:29 +01:00
mesh-tools 9ade421e12 Merge pull request 'gate: the source of a watched branch follows every merge, forward only (A-464..A-464d)' (#697) from fix/47-G-untaken-merge into main 2026-11-17 01:13:07 +00:00
jschoubben 46f3dc8cd7 gate: take-in.taken moves a head only forward too (A-464d)
Also corrects the manifest test's comment, which said the gate does not hear merged (A-464).

Builds: design 47 G-22 A-464d, A-464
2026-11-17 02:10:36 +01:00
jschoubben 7da6dd20a1 gate: a head moves only forward and only when it changes (A-464c)
Builds: design 47 G-22 A-464c, A-464
2026-11-17 02:02:07 +01:00
jschoubben c87cf1ace5 gate: the source of a watched branch follows every merge into it (A-464, A-464b)
The forge bridge's merged moves the repository's head and says source-moved; at its start each head is compared with the forge's branch head; gate_source_follow does it for one repository by hand.

Builds: design 47 G-22 A-464, A-464b
2026-11-17 01:59:09 +01:00
mesh-tools 3f3f6789dd Merge pull request 'nats: the tools bundle answers initialize again' (#696) from fix/nats-starts-again into main 2026-11-16 21:29:12 +00:00
mesh-tools 7c6aed9d69 Merge pull request 'nats: a tool purges a stream's messages below a sequence' (#695) from fix/nats-stream-purge into main 2026-11-16 21:07:49 +00:00
55 changed files with 953 additions and 135 deletions
+20 -1
View File
@@ -5,7 +5,7 @@
// exempt from that, and said so. Every build module, test step or not, must also hold a test that calls
// conformance.LeftRunning with its own start sweep, so a step a previous process left running is ended at its start
// (issue 1890), and one that calls conformance.Passing, so a step whose network did not answer is said as a transport fault
// (issue 1748). Its manifest asks the capability seat-settle-delay (ADR 0664 §5).
// (issue 1748), and one that calls conformance.StepEnds, so its steps can say a pipeline step's end (design 47 G §24). Its manifest asks the capability seat-settle-delay (ADR 0664 §5).
package failurekit
import (
@@ -91,6 +91,25 @@ func Judge(root string) ([]Judged, error) {
refused = append(refused, fmt.Sprintf("it claims %s, and no test of it calls conformance.LeftRunning "+
"(mesh-build-go buildstep/conformance): run the kit's left-running case with the module's own start sweep (issue 1890)", seat))
}
// A-489b: every build module is asked pipeline steps, so every one runs the case that its steps can say a step's end: without a
// SayStep every step ask is refused live ("this build module says no step's end"), as mesh-build-node's was.
ends, err := runsTheKit(dir, "conformance.StepEnds(")
if err != nil {
return nil, err
}
counts, err := runsTheKit(dir, "conformance.RunsCounted(")
if err != nil {
return nil, err
}
if !counts {
// A-489c
refused = append(refused, fmt.Sprintf("it claims %s, and no test of it calls conformance.RunsCounted "+
"(mesh-delivery-sdk steprun/conformance): run the kit's case that its puller counts the runs that started, not the bus's deliveries (A-489a)", seat))
}
if !ends {
refused = append(refused, fmt.Sprintf("it claims %s, and no test of it calls conformance.StepEnds "+
"(mesh-delivery-sdk steprun/conformance): run the kit's step-ends case with the module's own steps (design 47 G §24)", seat))
}
// Every build module meets the network (a fetch, a download, an install, a publish), so every one runs the passing case: its
// steps say a network that did not answer as a transport fault the bus gives the ask again for (ADR 0664, issue 1748).
says, err := runsTheKit(dir, "conformance.Passing(")
+7 -3
View File
@@ -13,7 +13,7 @@ import (
func TestABuildModuleWithATestStepMustRunTheKit(t *testing.T) {
root := t.TempDir()
const imp = "package main\n\nimport \"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/steprun/conformance\"\n\n"
const sweep = "func TestSweep(t *testing.T) { conformance.LeftRunning(t, nil) }\nfunc TestPassing(t *testing.T) { conformance.Passing(t, nil, nil) }\n"
const sweep = "func TestSweep(t *testing.T) { conformance.LeftRunning(t, nil) }\nfunc TestPassing(t *testing.T) { conformance.Passing(t, nil, nil) }\nfunc TestEnds(t *testing.T) { conformance.StepEnds(t, steprun.AskSteps{}) }\nfunc TestCounted(t *testing.T) { conformance.RunsCounted(t, nil) }\n"
kit := imp + "func TestKit(t *testing.T) { conformance.Run(t, conformance.GoTest, nil) }\n" + sweep
for path, body := range map[string]string{
"modules/a/module.json": `{"claims":[{"name":"node-build-a"}],"tools":["build_a_fetch","build_a_test"]}`,
@@ -30,6 +30,10 @@ func TestABuildModuleWithATestStepMustRunTheKit(t *testing.T) {
"modules/h/cmd/h/kit_test.go": imp + "func TestSweep(t *testing.T) { conformance.LeftRunning(t, nil) }\n",
"modules/i/module.json": `{"claims":[{"name":"node-build-i"}],"capabilities":["other"],"tools":["build_i_build"]}`, // the whole kit, and no seat-settle-delay
"modules/i/cmd/i/kit_test.go": imp + sweep,
"modules/j/module.json": `{"claims":[{"name":"node-build-j"}],"capabilities":["seat-settle-delay"],"tools":["build_j_build"]}`, // the rest of the kit, and no step-ends case
"modules/j/cmd/j/kit_test.go": imp + "func TestSweep(t *testing.T) { conformance.LeftRunning(t, nil) }\nfunc TestPassing(t *testing.T) { conformance.Passing(t, nil, nil) }\nfunc TestCounted(t *testing.T) { conformance.RunsCounted(t, nil) }\n",
"modules/k/module.json": `{"claims":[{"name":"node-build-k"}],"capabilities":["seat-settle-delay"],"tools":["build_k_build"]}`, // no runs-counted case
"modules/k/cmd/k/kit_test.go": imp + "func TestSweep(t *testing.T) { conformance.LeftRunning(t, nil) }\nfunc TestPassing(t *testing.T) { conformance.Passing(t, nil, nil) }\nfunc TestEnds(t *testing.T) { conformance.StepEnds(t, steprun.AskSteps{}) }\n",
"modules/e/module.json": `{"claims":[{"name":"node-build-e"}],"tools":["build_e_test"]}`,
"modules/e/cmd/e/kit.go": kit, // not a test file
} {
@@ -45,9 +49,9 @@ func TestABuildModuleWithATestStepMustRunTheKit(t *testing.T) {
if err != nil {
t.Fatal(err)
}
want := map[string]string{"a": "refused", "b": "passes", "c": "exempt", "e": "refused", "f": "refused", "g": "refused", "h": "refused", "i": "refused"}
want := map[string]string{"a": "refused", "b": "passes", "c": "exempt", "e": "refused", "f": "refused", "g": "refused", "h": "refused", "i": "refused", "j": "refused", "k": "refused"}
if len(got) != len(want) {
t.Fatalf("judged %+v, want a, b, c, e, f, g, h and i", got)
t.Fatalf("judged %+v, want a, b, c, e, f, g, h, i, j and k", got)
}
for _, j := range got {
is := "passes"
+2 -2
View File
@@ -1,6 +1,6 @@
#!/bin/sh
# The wallpaper (module feh, novox/hq ADR 0208). Owned by the mesh: replaced at every push. The
# The wallpaper (module feh, novox/hq ADR 0208). Owned by the mesh: replaced at every delivery. The
# session's start runs it, and so may anything that wants the declared wallpaper back. The image is
# the module's own, in ~/.local/share/feh/wallpapers. feh_set changes the wallpaper for a session
# without touching this file.
# without touching this file (the second single-module proof, 2026-11-17).
feh --no-fehbg --bg-fill "$HOME/.local/share/feh/wallpapers/default.jpg"
-3
View File
@@ -1,3 +0,0 @@
// Package buildstep is kept empty as a bridge: the running pre-G indexer publishes mesh-catalog's packages only while
// every package directory it published before is in the tree. It goes once G's indexer runs.
package buildstep
@@ -91,7 +91,7 @@ func (m *Module) Puller(steps []StepTool, source steprun.AskSource) *steprun.Pul
if m.SayEvent != nil {
say = steprun.SayOn(sdkstep.AskGo, m.Node, m.SayEvent)
}
return &steprun.Puller{KeepOutput: steprun.KeepOutputIn(m.Outputs), Source: source, Language: sdkstep.AskGo, Say: say, Callers: m.Callers, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load, ForgetPause: m.ForgetPause, WatchPauses: m.WatchPauses, PauseKeys: m.ReadPauses.Keys,
return &steprun.Puller{KeepOutput: steprun.KeepOutputIn(m.Outputs), Source: source, Language: sdkstep.AskGo, Say: say, Callers: m.Callers, Prior: steprun.PriorRuns(m.ReadSteps.Get), Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load, ForgetPause: m.ForgetPause, WatchPauses: m.WatchPauses, PauseKeys: m.ReadPauses.Keys,
Capacity: capacity, SettingsFile: settings, WatchConfig: watch, CoresPerStep: cores, BuildHours: hours, Logf: m.Say, Has: m.Has,
Run: func(ctx context.Context, a sdkstep.StepAsk) (any, error) {
if err := m.Check(ctx); err != nil {
@@ -58,6 +58,7 @@ func main() {
// compiles that program and runs it shut off, answers its mark judged by the candidate at the tree's commit, and the
// program reached no key, no login, no name and no socket of the machine.
func TestTheUnitLevelUnderTheWorkspaceRunnerRunsTheTreesProgramShutOff(t *testing.T) {
t.Parallel()
needTools(t, "the candidate's program shut off", "go", "git", "bwrap")
home := t.TempDir()
key, login := filepath.Join(home, ".ssh", "mesh_ed25519"), filepath.Join(home, "broker")
@@ -195,6 +195,7 @@ func TestACandidateThatDoesNotCompileFailsTheStepWithTheCompilersFirstErrorAndRu
}
func TestTheInstalledProgramSaysItJudged(t *testing.T) { // 1898 K1, ADR 0535 §5
t.Parallel()
m, states, ws, _ := workspaceFor(t, "slow")
for _, runner := range []any{nil, RunnerInstalled} {
args := map[string]any{"workspace": ws, "level": LevelUnit}
@@ -223,6 +224,7 @@ func TestAnUnknownRunnerIsRefused(t *testing.T) { // 1898 K1
}
func TestTheShutOffProcessRunsTheStepItIsAsked(t *testing.T) { // 1898 K2: the candidate's side of the spec
t.Parallel()
m, _, ws, calls := workspaceFor(t, "slow")
m.Inside = true
a := m.answer(context.Background(), levelSpec{Step: stepUnit, Workspace: ws, Packages: []string{"./..."}, Bound: UnitBound})
@@ -24,70 +24,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/testbus"
)
// nodeEngineTree is the node-engine's repository as the checker places it beside this check (MESH_CHECK_BESIDE, the
// repository mesh-host), and with it every Go module of the other repositories placed there (mesh-sdk, mesh-contracts,
// mesh-delivery, mesh-networking, the catalogue's trunk) and of the catalogue under check, first, as candidates: the workspace lays those the
// node-engine's composer requires at the zero version, transitively (workspaceOf), so they resolve as the group lays
// them, wherever the composer lives. The test
// builds the node-engine's real composer command from that tree; it does not stand one in. With no beside directory it
// fails, and with the node-engine beside but no composer command in it it fails: a composer test that could not build its
// composer proves nothing. With a beside directory that holds no node-engine at all it skips aloud (NOT JUDGED), since a
// checker from before this release places none for the catalogue.
func nodeEngineTree(t *testing.T) (tree string, modules []string) {
t.Helper()
beside := os.Getenv("MESH_CHECK_BESIDE")
if beside == "" {
t.Fatal("NOT CHECKED: MESH_CHECK_BESIDE is not set, so the node-engine's repository (mesh-host) is not beside this check")
}
tree = filepath.Join(beside, "mesh-host")
if !dirExists(tree) {
// A checker from before novox/hq ADR 0593 places only mesh-sdk beside a catalogue check, so the node-engine's
// repository is absent, not wrong: said aloud, since nothing here could build a composer to judge.
t.Skipf("NOT JUDGED: the node-engine's repository (mesh-host) is not beside this check in MESH_CHECK_BESIDE=%s; the running checker places none for the catalogue, so the composer is judged only once it does", beside)
}
if _, err := os.Stat(filepath.Join(tree, "cmd", "mesh-compose")); err != nil {
t.Fatalf("NOT CHECKED: the node-engine's composer command is not beside this check in MESH_CHECK_BESIDE=%s: %v", beside, err)
}
entries, err := os.ReadDir(beside)
if err != nil {
t.Fatal(err)
}
var roots []string
for _, e := range entries {
if e.Name() == "mesh-host" || !dirExists(filepath.Join(beside, e.Name())) {
continue
}
root, err := filepath.EvalSymlinks(filepath.Join(beside, e.Name()))
if err != nil {
t.Fatal(err)
}
roots = append(roots, root)
}
// The catalogue under check, first: beside lies its trunk (issue 1295), whose modules a head's must win over.
catalogue, err := filepath.Abs(filepath.Join("..", "..", "..", ".."))
if err != nil {
t.Fatal(err)
}
for _, root := range append([]string{catalogue}, roots...) {
filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return nil
}
if d.IsDir() && (d.Name() == ".git" || d.Name() == "testdata" || d.Name() == "node_modules") {
return filepath.SkipDir
}
if !d.IsDir() && d.Name() == "go.mod" {
modules = append(modules, filepath.Dir(p))
}
return nil
})
}
return tree, modules
}
func dirExists(p string) bool { st, err := os.Stat(p); return err == nil && st.IsDir() }
// copyTree copies the files of a source tree into dst, leaving out its .git.
// copyHostTree copies the files of a source tree into dst, leaving out its .git.
func copyHostTree(t *testing.T, src, dst string) {
t.Helper()
src, err := filepath.EvalSymlinks(src)
@@ -130,7 +67,7 @@ func goEnvOf(t *testing.T, name string) string {
}
// 1622/AC1, AC3 against the real bus (ADR 0593 §7, C9): a compose ask put on the Go build seat's check stream is pulled by
// the module started whole on that bus, which fetches the commit (the node-engine's repository beside the check, its Go module at the root), builds the composer command the ask names there with nothing fetched, fills the
// the module started whole on that bus, which fetches the commit (a stand-in composer of this module's testdata, its Go module at the root), builds the composer command the ask names there with nothing fetched, fills the
// request's manifests from the indexer's state and runs the composer; the step key <node>.<ask id> passes with the
// composer's verdict. A second ask of the same commit runs the kept composer: fetched and built once.
func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
@@ -157,14 +94,11 @@ func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
m.Run = StepRunner(t.TempDir())
m.GoModCache = goEnvOf(t, "GOMODCACHE") // the module cache this module's own build filled, read and not fetched
run, gocache := m.Run, goEnvOf(t, "GOCACHE") // this machine's build cache, so the composer's packages compile once
hostTree, beside := nodeEngineTree(t)
workFile := ""
// The composer is a stand-in of this module's own testdata (design 47 G A-474: a module's tests read nothing outside it, and a
// pipeline's step places no other repository beside the tree): a program of the composer's contract that imports nothing.
hostTree := filepath.Join("testdata", "composer")
m.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
env = append(env, "GOCACHE="+gocache)
if workFile != "" {
env = append(env, "GOWORK="+workFile, "GOFLAGS=-mod=readonly") // a workspace takes no -mod=mod
}
return run(ctx, env, name, args...)
return run(ctx, append(env, "GOCACHE="+gocache), name, args...)
}
read, digests := catalogHolds(t, `{"module":"demo","version":"1"}`)
m.ReadEntry = read
@@ -181,18 +115,6 @@ func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
return nil, err
}
copyHostTree(t, hostTree, tree)
if len(beside) > 0 { // the group's other repositories, laid beside the node-engine's tree as one workspace
work, err := workspaceOf(tree, beside)
if err != nil {
return nil, err
}
if err := os.WriteFile(filepath.Join(m.Work, ws, "go.work"), []byte(work), 0o600); err != nil {
return nil, err
}
workFile = filepath.Join(m.Work, ws, "go.work")
// Nothing is downloaded here: this level has no network. The check's own fetch downloaded this workspace's
// graph (downloadBeside), and a module it lacks fails the build naming it.
}
return map[string]any{"workspace": ws}, nil
}
}
@@ -203,7 +125,7 @@ func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
ask := func() sdkstep.Step {
a := composeAskOf(t, digests[0])
a.Compose.Fetch.Repository, a.Compose.Dir, a.Compose.Package = "novox/mesh-host", catalogclient.RootPath, "github.com/novox/mesh-host/cmd/mesh-compose"
a.Compose.Fetch.Repository, a.Compose.Dir, a.Compose.Package = "novox/mesh-catalog", catalogclient.RootPath, "standin/cmd/mesh-compose"
raw, err := sdkstep.EncodeStepAsk(a)
if err != nil {
t.Fatal(err)
@@ -273,6 +273,7 @@ func TestAPackageWithTheReadsOutsideMarkerIsKeptOnlyByWhatItLists(t *testing.T)
}
edit(t, tree, "docs/one.md", "one\n")
edit(t, tree, "other.md", "other\n")
// Each marker is judged in a fixture of its own, side by side: the digests are go commands, and the table is the test's longest part.
for name, marker := range map[string]string{
"an empty marker": "# nothing listed\n\n",
"a sentence": "the tests read ../docs\n",
@@ -282,19 +283,44 @@ func TestAPackageWithTheReadsOutsideMarkerIsKeptOnlyByWhatItLists(t *testing.T)
"a path that is not clean": "docs/./one.md\n",
"a good line and a bad line": "docs\nnope/*.md\n",
} {
edit(t, tree, "c/"+ReadsOutsideMarker, marker)
first, second := digests(t, m, "unit"), digests(t, m, "unit")
if len(first.Packages[c]) != 64 || first.Packages[c] == second.Packages[c] || first.Packages[c] == plain.Packages[c] {
t.Errorf("%s: c's digest is stable: %s, %s, before %s", name, first.Packages[c], second.Packages[c], plain.Packages[c])
}
if strings.Join(first.ReadsOutside, ",") != c {
t.Errorf("%s: the answer does not name the package that reads outside: %v", name, first.ReadsOutside)
}
if first.Packages[a] != plain.Packages[a] || first.Packages[a] != second.Packages[a] {
t.Errorf("%s: a has no marker and its digest moved: %s, %s, before %s", name, first.Packages[a], second.Packages[a], plain.Packages[a])
}
t.Run(name, func(t *testing.T) {
t.Parallel()
m, tree := digestFixture(t)
plain := digests(t, m, "unit")
if err := os.MkdirAll(filepath.Join(tree, "docs"), 0o755); err != nil {
t.Fatal(err)
}
edit(t, tree, "docs/one.md", "one\n")
edit(t, tree, "other.md", "other\n")
edit(t, tree, "c/"+ReadsOutsideMarker, marker)
first, second := digests(t, m, "unit"), digests(t, m, "unit")
if len(first.Packages[c]) != 64 || first.Packages[c] == second.Packages[c] || first.Packages[c] == plain.Packages[c] {
t.Errorf("%s: c's digest is stable: %s, %s, before %s", name, first.Packages[c], second.Packages[c], plain.Packages[c])
}
if strings.Join(first.ReadsOutside, ",") != c {
t.Errorf("%s: the answer does not name the package that reads outside: %v", name, first.ReadsOutside)
}
if first.Packages[a] != plain.Packages[a] || first.Packages[a] != second.Packages[a] {
t.Errorf("%s: a has no marker and its digest moved: %s, %s, before %s", name, first.Packages[a], second.Packages[a], plain.Packages[a])
}
})
}
}
// The marker that lists what the package's tests read, and the marker of what the package needs (issue 890/AC3): in a fixture of its
// own, so it runs beside the table above.
func TestAListedReadsOutsideMarkerKeepsTheDigestOfOnlyWhatItLists(t *testing.T) {
t.Parallel()
m, tree := digestFixture(t)
plain := digests(t, m, "unit")
c, a := "fixture/digest/c", "fixture/digest/a"
if err := os.MkdirAll(filepath.Join(tree, "docs"), 0o755); err != nil {
t.Fatal(err)
}
edit(t, tree, "docs/one.md", "one\n")
edit(t, tree, "other.md", "other\n")
// A list: a glob, a directory read whole, and a path whose absence is read.
edit(t, tree, "c/"+ReadsOutsideMarker, "# what c's tests read\ndocs/*.md\ndocs\n?gone\n")
before := digests(t, m, "unit")
@@ -34,7 +34,7 @@ func TestTheGuardSaysToolchainReportedOnAChangeWithTheProgramDigest(t *testing.T
t.Fatal(err)
}
if len(got) != 1 || got[0].subject != "mesh.seat.node-build-go.event.toolchain-reported.ace" || got[0].body["node"] != "ace" || got[0].body["language"] != "go" ||
got[0].body["version"] != "go1.27.1" || got[0].body["recipe"] != m.Recipe.Version() || len(got[0].body["digest"]) != len("sha256:")+64 {
got[0].body["version"] != "go1.27" || got[0].body["recipe"] != m.Recipe.Version() || len(got[0].body["digest"]) != len("sha256:")+64 {
t.Fatalf("events = %+v, want one toolchain-reported for ace, go with the program's digest", got)
}
if tc, _ := states.toolchain("ace.go"); tc.Digest != got[0].body["digest"] {
@@ -51,7 +51,7 @@ func TestTheGuardSaysToolchainReportedOnAChangeWithTheProgramDigest(t *testing.T
os.WriteFile(prog, []byte("another go program"), 0o755)
m.Probe = func(context.Context) (string, error) { return "go1.27.2", nil }
m.Check(context.Background())
if len(got) != 2 || got[1].body["version"] != "go1.27.2" || got[1].body["digest"] == got[0].body["digest"] {
if len(got) != 2 || got[1].body["version"] != "go1.27" || got[1].body["digest"] == got[0].body["digest"] {
t.Errorf("a new version said %+v, want a second event with another digest", got)
}
}
@@ -149,8 +149,8 @@ func TestAnyInstalledGoIsHealthyNamedAndReported(t *testing.T) {
t.Errorf("health = %+v, want healthy naming %s", a, installed)
}
tc, ok := states.toolchain("ace.go")
if !ok || tc.Version != installed || tc.Recipe != m.Recipe.Version() {
t.Errorf("ace.go = %+v, %v; want version %s and recipe %s", tc, ok, installed, m.Recipe.Version())
if !ok || tc.Version != sdkstep.LanguageVersion(installed) || tc.Recipe != m.Recipe.Version() {
t.Errorf("ace.go = %+v, %v; want version %s and recipe %s", tc, ok, sdkstep.LanguageVersion(installed), m.Recipe.Version())
}
ran := false
run := func(context.Context, map[string]any) (any, error) { ran = true; return "done", nil }
@@ -186,6 +186,7 @@ func TestABuildRunsShutOffAndOnlyATrunkBuildWritesTheCaches(t *testing.T) {
// 1309-S3: the digest's go list runs shut off.
func TestTheDigestListsPackagesShutOff(t *testing.T) {
t.Parallel()
m, _ := digestFixture(t)
var seen []shutOff
m.Run = throughBubblewrap(t, m.Run, &seen)
@@ -15,6 +15,7 @@ import (
// resolves no name, reaches no socket of the machine (a TCP port on its loopback, a unix socket) and leaves the shared
// caches unchanged. The probe is testdata/repos/probe.
func TestARunStepReachesNoKeyLoginNameOrSocketAndLeavesTheCachesUnchanged(t *testing.T) {
t.Parallel()
needTools(t, "the shut-off unit level", "go", "git", "bwrap")
tree := repository(t, "probe")
work, err := os.MkdirTemp("", "w")
@@ -56,3 +56,19 @@ func TestTheStepsSayAFailedNetworkAsATransportFault(t *testing.T) {
}
})
}
// A-489b, design 47 G §24: the module's own steps can say a pipeline step's end, or
// every step ask the runner sends it is refused ("this build module says no step's end"), as the first live pipeline met.
func TestTheStepsCanSayAStepsEnd(t *testing.T) {
conformance.StepEnds(t, (&Module{Node: "kitnode"}).askSteps(nil))
}
// A-489c (A-489a): the module's puller counts the runs its steps state says started, not the bus's deliveries (an ask given back before
// it ran is no run), so the poison rule never ends an ask that never ran.
func TestThePullerCountsRunsThatStarted(t *testing.T) {
conformance.RunsCounted(t, func(read func(string) (json.RawMessage, error)) *steprun.Puller {
m := &Module{Node: "kitnode", Capacity: func() int { return 1 }}
m.ReadSteps = StateRead{Get: read}
return m.Puller(nil, steprun.NewFakeAsks())
})
}
@@ -195,6 +195,16 @@ func TestTheModuleInvokesTheArtifactSeatsVerbsAndWritesNoStore(t *testing.T) {
if !slices.Contains(m.Consumes, "mesh-conditions.findings-asked") {
t.Errorf("consumes %v lack mesh-conditions.findings-asked", m.Consumes)
}
// Design 47 G A-488a, A-488b: its runtime follows the seat list, so the module consumes the seat's two events (which makes its
// durable consumer) and invokes the verb that reads the list.
for _, ev := range []string{"mesh-catalog.seat-definition-changed", "mesh-catalog.seat-definition-removed"} {
if !slices.Contains(m.Consumes, ev) {
t.Errorf("consumes %v lack %s", m.Consumes, ev)
}
}
if !slices.Contains(m.Invokes, "seat:mesh-catalog.seat-definitions") {
t.Errorf("invokes %v lack seat:mesh-catalog.seat-definitions", m.Invokes)
}
// Not a foundation part, it asks its finding stream in requires, by the template events (design 47 G §529, A-30), and
// declares no stream of that name.
for _, st := range m.Streams {
@@ -158,6 +158,7 @@ func sleepABit() { time.Sleep(30 * time.Millisecond) }
// The workspace is self-contained after the fetch: no alternates file, and git finds the commit with the mirror gone,
// as a step shut off with the workspace alone needs. Real git, a local repository as the forge.
func TestTheWorkspaceIsSelfContainedAfterAFetchThroughTheMirror(t *testing.T) {
t.Parallel()
if _, err := exec.LookPath("git"); err != nil {
t.Skip("no git")
}
@@ -93,8 +93,8 @@ func TestStartWritesTheToolchainAndHealthIsHealthy(t *testing.T) {
break
}
}
if tc.Version != "go1.27.1" || tc.Recipe != m.Recipe.Version() || tc.Language != "go" {
t.Fatalf("ace.go = %+v, want go1.27.1 and recipe %s", tc, m.Recipe.Version())
if tc.Version != "go1.27" || tc.Recipe != m.Recipe.Version() || tc.Language != "go" {
t.Fatalf("ace.go = %+v, want go1.27 and recipe %s", tc, m.Recipe.Version())
}
if a := m.Health(ctx); a.State != health.StateHealthy || !a.Healthy {
t.Errorf("health = %+v, want healthy", a)
@@ -21,6 +21,7 @@ import (
// build_go_fetch is asked that head as the package's version and build_go_build runs on the member, then both pass, the
// build with GOPROXY=off against the head: the older commit holds no Go package at all.
func TestRealFetchAndBuildOfAMemberAskedAPullHeadItsGoModDoesNotName(t *testing.T) {
t.Parallel()
// Not skipped when a tool is missing: a pass of the check must mean this test ran (the check prints no test log).
for _, tool := range []string{"go", "git"} {
if _, err := exec.LookPath(tool); err != nil {
@@ -304,6 +304,7 @@ func TestARelativeReplaceStillResolvesInTheTemporaryModfile(t *testing.T) {
// the asked versions, else from the trunk's latest published version, and never looked for as the commit 000000000000
// among the pull request heads.
func TestARequirementAtTheZeroVersionIsTakenFromTheAskOrTheTrunk(t *testing.T) {
t.Parallel()
const sdk = "git.novox.be/novox/mesh-sdk/go"
zeroMod := "module fixture/member\n\ngo 1.26\n\nrequire (\n\t" + sdk + " v0.0.0-00010101000000-" + zeroShort + "\n)\n"
trunk := strings.Repeat("a", 40)
@@ -171,6 +171,7 @@ func TestRealFetchMergesConflictsAndPlainlyFetches(t *testing.T) {
// sibling and passes with it; the sibling is a plain directory with no symlink in it; and a sibling that holds a
// symlink fails the fetch naming it.
func TestRealTestsReadTheSiblingBesideTheTree(t *testing.T) {
t.Parallel()
needTools(t, "the shut-off unit level", "go", "git", "bwrap") // the tests run in bubblewrap (1309)
f := newForge(t)
files := map[string]string{}
@@ -118,6 +118,7 @@ func resolve(t *testing.T, addr string, a builds.BuiltArtifact) []byte {
// reference resolves in a real artifact store to bytes of that digest. The build is the real go's, shut off, of the
// fixture testdata/repos/build, with an archive beside its bundle.
func TestADoneBuildsArtifactsResolveInARealArtifactStore(t *testing.T) {
t.Parallel()
needTools(t, "the build", "go", "bwrap")
work := t.TempDir()
tree := filepath.Join(work, "ws1", "tree")
@@ -62,6 +62,7 @@ func repository(t *testing.T, name string) string {
// test_test.go read the output captured from them, and this one proves the capture is what the real go says today. It
// starts go test, so it carries the tag.
func TestUnitLevelRunsTheRealGoOnTheFixtureRepositories(t *testing.T) {
t.Parallel()
needTools(t, "the unit level on fixtures", "go", "git", "bwrap")
want := map[string]struct {
passed, noTests bool
@@ -104,6 +105,7 @@ func TestUnitLevelRunsTheRealGoOnTheFixtureRepositories(t *testing.T) {
// 086 A-C3 against the real go (design 47 line 571 at b229f5c4): a tree holding an unformatted file passes its unit
// level, its two tests run, and the lint step's body fails naming the file.
func TestAnUnformattedFileFailsTheRealLintStepAndNotTheUnitLevel(t *testing.T) {
t.Parallel()
needTools(t, "the unit level's gofmt and vet", "go", "git", "bwrap")
home := t.TempDir()
m := &Module{Home: home, Work: t.TempDir(), Cache: t.TempDir()}
@@ -150,7 +152,7 @@ func TestIntegrationLevelRunsAgainstARealStoreAndBusAndLeavesNothing(t *testing.
needTools(t, "the integration level", "go", "git", "initdb", "pg_ctl", "postgres", "bwrap")
tree := repository(t, "store")
// A short work directory: a unix socket's path is bounded at 108 bytes.
work, err := os.MkdirTemp("/tmp", "w")
work, err := os.MkdirTemp(os.TempDir(), "w")
if err != nil {
t.Fatal(err)
}
@@ -185,6 +187,7 @@ func TestIntegrationLevelRunsAgainstARealStoreAndBusAndLeavesNothing(t *testing.
// 1487/AC2: given a fixture Go module, when it is built twice, both digests are equal and the archive holds one
// executable named as binaryName says. It runs the real go, so it carries the tag.
func TestBuildTwiceGivesOneDigestAndOneExecutable(t *testing.T) {
t.Parallel()
needTools(t, "the build", "go", "bwrap")
work := t.TempDir()
tree := filepath.Join(work, "ws1", "tree")
@@ -0,0 +1,44 @@
// Command mesh-compose is the stand-in composer the compose ask's test builds: a program of the contract the Go build module
// runs (`-request <file>`, the last line of its standard output a composer verdict of version 1), which judges nothing and
// answers each case of the request by its name, with no manifest and no machine. It imports nothing of the mesh.
package main
import (
"encoding/json"
"flag"
"fmt"
"os"
)
func main() {
file := flag.String("request", "", "the request file")
flag.Parse()
raw, err := os.ReadFile(*file)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
var req struct {
Cases []struct {
Name string `json:"name"`
} `json:"cases"`
}
if err := json.Unmarshal(raw, &req); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
type verdict struct {
Name string `json:"name"`
Manifests map[string]any `json:"manifests"`
}
out := struct {
V int `json:"v"`
SDK string `json:"sdk"`
Cases []verdict `json:"cases"`
}{V: 1, SDK: "stand-in"}
for _, c := range req.Cases {
out.Cases = append(out.Cases, verdict{Name: c.Name, Manifests: map[string]any{}})
}
line, _ := json.Marshal(out)
fmt.Println(string(line))
}
@@ -0,0 +1,3 @@
module standin
go 1.26
@@ -19,6 +19,7 @@ import (
// group's head Go cannot fetch is placed from the forge's pull request heads; a go.mod that does not require the package
// takes nothing.
func TestAnAskedVersionIsWrittenIntoTheModfileWhateverGoModNames(t *testing.T) {
t.Parallel()
const sdk = "git.novox.be/novox/mesh-sdk/go"
trunk := strings.Repeat("a", 40)
offHead := strings.Repeat("b", 40) // a commit no pull request's head carries
+4 -1
View File
@@ -33,7 +33,9 @@
}
],
"consumes": [
"mesh-conditions.findings-asked"
"mesh-conditions.findings-asked",
"mesh-catalog.seat-definition-changed",
"mesh-catalog.seat-definition-removed"
],
"invokes": [
"seat:mesh-hand-acts.hand-act",
@@ -46,6 +48,7 @@
"seat:mesh-wants.input",
"seat:mesh-module-assigner.assignments",
"seat:mesh-catalog.module",
"seat:mesh-catalog.seat-definitions",
"seat:mesh-catalog.entry"
],
"config": {
@@ -31,9 +31,20 @@ func (m *Module) askSteps(steps []StepTool) steprun.AskSteps {
Fetch: byName["build_image_fetch"], Build: byName["build_image_build"], Copy: byName["build_image_copy"],
Release: func(ws string) error { return m.spaces().Release(ws) },
Logf: m.Say,
Node: m.Node,
SayStep: m.sayStep(),
}
}
// sayStep says a step ask's end on this node's subject of the seat (design 47 G §24): an image has no lint, unit or integration
// step (§328), so none is asked of this seat, but the steps of every build module can say their ends, which the kit holds.
func (m *Module) sayStep() func(sdkstep.StepEnded) error {
if m.SayStep != nil {
return m.SayStep
}
return steprun.SayStepOn(sdkstep.AskImage, m.Node, stdio.SeatPublish)
}
// Puller is the module's pull of its seat's asks, on source (the runtime's seat pull when nil).
func (m *Module) Puller(steps []StepTool, source steprun.AskSource) *steprun.Puller {
if source == nil {
@@ -69,7 +80,7 @@ func (m *Module) Puller(steps []StepTool, source steprun.AskSource) *steprun.Pul
if m.SayEvent != nil {
say = steprun.SayOn(sdkstep.AskImage, m.Node, m.SayEvent)
}
return &steprun.Puller{KeepOutput: steprun.KeepOutputIn(m.Outputs), Source: source, Language: sdkstep.AskImage, Say: say, Callers: m.Callers, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load, ForgetPause: m.ForgetPause, WatchPauses: m.WatchPauses, PauseKeys: m.ReadPauses.Keys,
return &steprun.Puller{KeepOutput: steprun.KeepOutputIn(m.Outputs), Source: source, Language: sdkstep.AskImage, Say: say, Callers: m.Callers, Prior: steprun.PriorRuns(m.ReadSteps.Get), Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load, ForgetPause: m.ForgetPause, WatchPauses: m.WatchPauses, PauseKeys: m.ReadPauses.Keys,
Capacity: capacity, SettingsFile: settings, WatchConfig: watch, CoresPerStep: cores, BuildHours: hours, Eligible: m.Eligible, Logf: m.Say,
Run: func(ctx context.Context, a sdkstep.StepAsk) (any, error) {
if err := m.Check(ctx); err != nil {
@@ -24,7 +24,7 @@ func TestTheModuleInvokesTheImageSeatAndHoldsNoRegistry(t *testing.T) {
t.Fatal(err)
}
if want := []string{"seat:mesh-hand-acts.hand-act", "seat:container-image-registry.publish", "seat:container-image-registry.publish-part",
"seat:container-image-registry.mirror", "seat:mesh-module-assigner.assignments", "seat:mesh-catalog.module"}; !slices.Equal(m.Invokes, want) {
"seat:container-image-registry.mirror", "seat:mesh-module-assigner.assignments", "seat:mesh-catalog.module", "seat:mesh-catalog.seat-definitions"}; !slices.Equal(m.Invokes, want) {
t.Errorf("invokes = %v, want %v", m.Invokes, want)
}
// The seat's outcomes a copy waits on; the version comes in the ask, so version.numbered is not heard (design 47 G §413).
@@ -32,6 +32,7 @@ func TestTheModuleInvokesTheImageSeatAndHoldsNoRegistry(t *testing.T) {
for _, e := range OutcomeEvents {
consumes = append(consumes, "container-image-registry."+e)
}
consumes = append(consumes, "mesh-catalog.seat-definition-changed", "mesh-catalog.seat-definition-removed") // the seat list, A-488a
if !slices.Equal(m.Consumes, consumes) {
t.Errorf("consumes = %v, want %v", m.Consumes, consumes)
}
@@ -2,6 +2,7 @@ package main
import (
"context"
"encoding/json"
"errors"
sdkstep "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/buildstep"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/steprun"
@@ -44,3 +45,19 @@ func TestTheStepsSayAFailedNetworkAsATransportFault(t *testing.T) {
}
})
}
// A-489b, design 47 G §24: the module's own steps can say a pipeline step's end, or
// every step ask the runner sends it is refused ("this build module says no step's end"), as the first live pipeline met.
func TestTheStepsCanSayAStepsEnd(t *testing.T) {
conformance.StepEnds(t, (&Module{Node: "kitnode"}).askSteps(nil))
}
// A-489c (A-489a): the module's puller counts the runs its steps state says started, not the bus's deliveries (an ask given back before
// it ran is no run), so the poison rule never ends an ask that never ran.
func TestThePullerCountsRunsThatStarted(t *testing.T) {
conformance.RunsCounted(t, func(read func(string) (json.RawMessage, error)) *steprun.Puller {
m := &Module{Node: "kitnode", Capacity: func() int { return 1 }}
m.ReadSteps = StateRead{Get: read}
return m.Puller(nil, steprun.NewFakeAsks())
})
}
@@ -113,6 +113,16 @@ func TestWorkAndStoreDirectoriesLieOutsideTheHome(t *testing.T) {
// and the registry's address is bound into a file of the home that the program reads.
func TestManifestKeepsItsStatesClaimsItsBuildSeatAndDeclaresItsHealthTool(t *testing.T) {
m := readManifest(t)
// Design 47 G A-488a, A-488b: its runtime follows the seat list, so the module consumes the seat's two events (which makes its
// durable consumer) and invokes the verb that reads the list.
for _, ev := range []string{"mesh-catalog.seat-definition-changed", "mesh-catalog.seat-definition-removed"} {
if !slices.Contains(m.Consumes, ev) {
t.Errorf("consumes %v lack %s", m.Consumes, ev)
}
}
if !slices.Contains(m.Invokes, "seat:mesh-catalog.seat-definitions") {
t.Errorf("invokes %v lack seat:mesh-catalog.seat-definitions", m.Invokes)
}
states := []string{}
for _, s := range m.State {
states = append(states, s.Name)
@@ -28,6 +28,8 @@ type Module struct {
// SayEvent publishes an event of the image build seat on a seat subject (stdio.SeatPublish; nil says none, a test's): the
// puller's ask-taken, step-progress and step-ended (design 47 G §496, §602).
SayEvent func(subject string, body any, id string) error
// SayStep says a step ask's end (steprun.AskSteps.SayStep); the seat's publish on this node when nil (a test's own).
SayStep func(sdkstep.StepEnded) error
Node string
// RecordAct writes an act in the hand-act log of the seat mesh-hand-acts (§463, §480): a drop writes it first.
+5 -2
View File
@@ -68,11 +68,14 @@
"seat:container-image-registry.publish-part",
"seat:container-image-registry.mirror",
"seat:mesh-module-assigner.assignments",
"seat:mesh-catalog.module"
"seat:mesh-catalog.module",
"seat:mesh-catalog.seat-definitions"
],
"consumes": [
"container-image-registry.published",
"container-image-registry.refused"
"container-image-registry.refused",
"mesh-catalog.seat-definition-changed",
"mesh-catalog.seat-definition-removed"
],
"own-secrets": {
"broker": "${dir:state}/broker"
@@ -41,9 +41,22 @@ func (m *Module) askSteps(steps []StepTool) steprun.AskSteps {
},
Release: func(ws string) error { return m.spaces().Release(ws) },
Logf: m.Say,
Has: m.Has,
Lint: m.lintStep,
Node: m.Node,
SayStep: m.sayStep(),
}
}
// sayStep says a pipeline step's end on this node's subject of the seat (design 47 G §24, *Asked on a work queue, ended by an
// event*): SayStep when a test sets it, else the runtime's seat publish.
func (m *Module) sayStep() func(sdkstep.StepEnded) error {
if m.SayStep != nil {
return m.SayStep
}
return steprun.SayStepOn(sdkstep.AskNode, m.Node, stdio.SeatPublish)
}
// Puller is the module's pull of its seat's asks, on source (the runtime's seat pull when nil).
func (m *Module) Puller(steps []StepTool, source steprun.AskSource) *steprun.Puller {
if source == nil {
@@ -79,7 +92,7 @@ func (m *Module) Puller(steps []StepTool, source steprun.AskSource) *steprun.Pul
if m.SayEvent != nil {
say = steprun.SayOn(sdkstep.AskNode, m.Node, m.SayEvent)
}
return &steprun.Puller{KeepOutput: steprun.KeepOutputIn(m.Outputs), Source: source, Language: sdkstep.AskNode, Say: say, Callers: m.Callers, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load, ForgetPause: m.ForgetPause, WatchPauses: m.WatchPauses, PauseKeys: m.ReadPauses.Keys,
return &steprun.Puller{KeepOutput: steprun.KeepOutputIn(m.Outputs), Source: source, Language: sdkstep.AskNode, Say: say, Callers: m.Callers, Prior: steprun.PriorRuns(m.ReadSteps.Get), Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load, ForgetPause: m.ForgetPause, WatchPauses: m.WatchPauses, PauseKeys: m.ReadPauses.Keys,
Capacity: capacity, SettingsFile: settings, WatchConfig: watch, CoresPerStep: cores, BuildHours: hours, Logf: m.Say,
Run: func(ctx context.Context, a sdkstep.StepAsk) (any, error) {
if err := m.Prepare(ctx); err != nil {
@@ -9,6 +9,7 @@ import (
"testing"
"time"
sdkstep "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/buildstep"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/steprun"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
@@ -226,3 +227,54 @@ func TestTheTestScriptFindsTscAndEsbuildOnItsPath(t *testing.T) {
t.Errorf("npm ran with %v, want NODE_OPTIONS naming the TAP reporter and its file", npm.env)
}
}
// Design 47 G §328 (*Lint per language*: TypeScript's lint is the type-check) and §24: the pipeline step lint of a TypeScript
// artifact runs tsc without emitting over the module's directory; a type error fails it naming the file, a clean package passes,
// a tree with no .ts file has nothing to lint, and a tsc that cannot start is an error, not a verdict.
func TestTheLintStepIsTheTypeCheck(t *testing.T) {
for _, c := range []struct {
name string
files map[string]string
ran forgekey.Ran
failed int
nothing bool
err bool
}{
{"a type error", map[string]string{"tools/bad.ts": `export const n: number = "text";`}, forgekey.Ran{Status: 2, Stdout: tscTypeError}, 1, false, false},
{"a clean package", map[string]string{"index.ts": `export const a = 1;`}, forgekey.Ran{}, 0, false, false},
{"no TypeScript", map[string]string{"package.json": `{"name":"x"}`}, forgekey.Ran{}, 0, true, false},
{"tsc cannot start", map[string]string{"index.ts": `export const a = 1;`}, forgekey.Ran{Status: 127, Err: "no such file"}, 0, false, true},
} {
t.Run(c.name, func(t *testing.T) {
m, f, work := newMachine(t)
workspaceWith(t, work, c.files)
f.answer["tsc"] = c.ran
out, err := m.lintStep(context.Background(), map[string]any{"workspace": "w1", "dir": "."})
if (err != nil) != c.err {
t.Fatalf("error = %v, want error %v", err, c.err)
}
if err != nil {
return
}
la := out.(steprun.LintAnswer)
if len(la.Failed) != c.failed || la.Nothing != c.nothing {
t.Fatalf("answer = %+v, want %d failed, nothing %v", la, c.failed, c.nothing)
}
if c.failed > 0 && !strings.Contains(la.Failed[0].Cause+la.Failed[0].Output, "tools/bad.ts") {
t.Errorf("the failure %+v does not name tools/bad.ts", la.Failed[0])
}
if !c.nothing && !strings.Contains(joined(f.calls[0]), "--noEmit") {
t.Errorf("the lint ran %s, want tsc --noEmit", joined(f.calls[0]))
}
})
}
}
// Design 47 G §24: the Node build module's steps carry what a step ask needs (its node, the lint body, the end's publisher and
// its own need), so the ask the first live pipeline sent is no longer refused as "says no step's end".
func TestTheNodeStepsCarryWhatAStepAskNeeds(t *testing.T) {
s := (&Module{Node: "shanks"}).askSteps(nil)
if s.Lint == nil || s.Node != "shanks" || s.SayStep == nil || s.Has == nil || !s.Has(sdkstep.NeedNodeBuildNode) || s.Has("other") {
t.Errorf("steps = %+v, want Lint, Node, SayStep and Has set", s)
}
}
@@ -34,7 +34,7 @@ func TestTheGuardSaysToolchainReportedOnAChangeWithTheProgramDigest(t *testing.T
t.Fatal(err)
}
if len(got) != 1 || got[0].subject != "mesh.seat.node-build-node.event.toolchain-reported.ace" || got[0].body["node"] != "ace" || got[0].body["language"] != "typescript" ||
got[0].body["version"] != "v26.10.0" || got[0].body["recipe"] != m.Recipe.Version() || len(got[0].body["digest"]) != len("sha256:")+64 {
got[0].body["version"] != "v26.10" || got[0].body["recipe"] != m.Recipe.Version() || len(got[0].body["digest"]) != len("sha256:")+64 {
t.Fatalf("events = %+v, want one toolchain-reported for ace, go with the program's digest", got)
}
if tc, _ := states.toolchain("ace.typescript"); tc.Digest != got[0].body["digest"] {
@@ -51,7 +51,7 @@ func TestTheGuardSaysToolchainReportedOnAChangeWithTheProgramDigest(t *testing.T
os.WriteFile(prog, []byte("another node program"), 0o755)
m.Probe = func(context.Context) (string, error) { return "v26.11.0", nil }
m.Check(context.Background())
if len(got) != 2 || got[1].body["version"] != "v26.11.0" || got[1].body["digest"] == got[0].body["digest"] {
if len(got) != 2 || got[1].body["version"] != "v26.11" || got[1].body["digest"] == got[0].body["digest"] {
t.Errorf("a new version said %+v, want a second event with another digest", got)
}
}
@@ -8,6 +8,7 @@ import (
"strings"
"testing"
sdkstep "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/buildstep"
"git.novox.be/novox/mesh-sdk/go/health"
)
@@ -36,8 +37,8 @@ func TestAnyInstalledNodeIsHealthyNamedAndReported(t *testing.T) {
t.Errorf("health = %+v, want healthy naming %s", a, installed)
}
tc, ok := states.toolchain("ace.typescript")
if !ok || tc.Version != installed || tc.Recipe != m.Recipe.Version() {
t.Errorf("ace.typescript = %+v, %v; want version %s and recipe %s", tc, ok, installed, m.Recipe.Version())
if !ok || tc.Version != sdkstep.LanguageVersion(installed) || tc.Recipe != m.Recipe.Version() {
t.Errorf("ace.typescript = %+v, %v; want version %s and recipe %s", tc, ok, sdkstep.LanguageVersion(installed), m.Recipe.Version())
}
if err := m.Prepare(context.Background()); err != nil {
t.Errorf("no step can run on node %s: %v", installed, err)
@@ -2,6 +2,7 @@ package main
import (
"context"
"encoding/json"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/steprun"
"strings"
"testing"
@@ -49,3 +50,19 @@ func TestTheStepsSayAFailedNetworkAsATransportFault(t *testing.T) {
}
})
}
// A-489b, design 47 G §24: the module's own steps can say a pipeline step's end, or
// every step ask the runner sends it is refused ("this build module says no step's end"), as the first live pipeline met.
func TestTheStepsCanSayAStepsEnd(t *testing.T) {
conformance.StepEnds(t, (&Module{Node: "kitnode"}).askSteps(nil))
}
// A-489c (A-489a): the module's puller counts the runs its steps state says started, not the bus's deliveries (an ask given back before
// it ran is no run), so the poison rule never ends an ask that never ran.
func TestThePullerCountsRunsThatStarted(t *testing.T) {
conformance.RunsCounted(t, func(read func(string) (json.RawMessage, error)) *steprun.Puller {
m := &Module{Node: "kitnode", Capacity: func() int { return 1 }}
m.ReadSteps = StateRead{Get: read}
return m.Puller(nil, steprun.NewFakeAsks())
})
}
@@ -0,0 +1,56 @@
package main
import (
"context"
"fmt"
"path/filepath"
sdkstep "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/buildstep"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/steprun"
)
// **The type-check is a TypeScript artifact's pipeline step lint** (design 47 G §328 *Lint per language*: Go's lint is the Go
// build module's linter, TypeScript's the type-check; §22: lint is a step the build seats serve; §24: asked on a work queue,
// its end said as an event). It takes {"workspace", "dir"} as the lint step of mesh-build-go does and runs tsc over each
// TypeScript package of the module's directory (testTargets, the way the unit level finds them), without emitting: a package
// with no .ts file has nothing to lint, and nothing at all ends the step `nothing to run`. A failure is the step's, naming
// the tsc output as the unit level names it; a tsc that could not start is an error, never a verdict.
func (m *Module) lintStep(ctx context.Context, args map[string]any) (any, error) {
workspace, _ := args["workspace"].(string)
dir, _ := args["dir"].(string)
root, mdir, err := m.moduleDirOf(workspace, dir)
if err != nil {
return nil, err
}
targets, err := testTargets(mdir, filepath.ToSlash(filepath.Clean(dir)))
if err != nil {
return nil, err
}
var failed []Failure
linted := false
for _, tg := range targets {
sources, err := typeScriptFiles(tg.dir)
if err != nil {
return nil, err
}
if len(sources) == 0 {
continue
}
linted = true
if err := m.provide(root); err != nil {
return nil, fmt.Errorf("providing the toolchain's packages to the workspace: %w", err)
}
r := m.runIn(ctx, root, false, tg.dir, nil, "node", m.typeCheck(tg.dir, sources)...)
if r.Err != "" {
return nil, fmt.Errorf("type-check could not be run: status %d %s: %s", r.Status, r.Err, firstLine(r.Stderr))
}
if r.Status != 0 {
failed = append(failed, testFailures(tg.where, "type-check", tg.dir, r, "")...)
}
}
return steprun.LintAnswer{Failed: failed, Nothing: !linted}, nil
}
// A-489b: Has says whether this node has a need a package's tests name (sdkstep.AskNeeds): the need node-build-node is this module's own
// (a TypeScript package's tests run through it); any other it lacks.
func (m *Module) Has(need string) bool { return need == sdkstep.NeedNodeBuildNode }
@@ -277,6 +277,17 @@ func TestTheModuleRequiresNodeToolsItPullsThrough(t *testing.T) {
if !slices.Contains(m.Requires, "node-tools") {
t.Errorf("requires %v lack node-tools", m.Requires)
}
// Design 47 G A-488a: its runtime follows the seat list (seatlist.Start subscribes to the seat mesh-catalog's events), so the
// module consumes both, which makes its durable consumer <node>_mesh-build-node_tools: with none, the seat list is never
// read and the asks of a named asker are given back for ever.
for _, ev := range []string{"mesh-catalog.seat-definition-changed", "mesh-catalog.seat-definition-removed"} {
if !slices.Contains(m.Consumes, ev) {
t.Errorf("consumes %v lack %s", m.Consumes, ev)
}
}
if !slices.Contains(m.Invokes, "seat:mesh-catalog.seat-definitions") {
t.Errorf("invokes %v lack seat:mesh-catalog.seat-definitions (A-488b)", m.Invokes)
}
}
// Design 47 G A-254, A-313, A-314: the module keeps a run's answer over what step-ended carries inline in the steps state's own
@@ -34,6 +34,9 @@ type Module struct {
// puller's ask-taken, step-progress and step-ended (design 47 G §496, §602).
SayEvent func(subject string, body any, id string) error
// SayStep says a pipeline step's end (steprun.AskSteps.SayStep); the seat's publish on this node when nil (a test's own).
SayStep func(sdkstep.StepEnded) error
Node string
// Recipe is the recipe in force. Reload, when set, reads it again at every check (the lock file the mesh
// writes can change under a running module); a failure to read it is a refusal.
@@ -94,8 +94,8 @@ func TestStartWritesTheToolchainAndHealthIsHealthy(t *testing.T) {
break
}
}
if tc.Version != "v26.10.0" || tc.Recipe != m.Recipe.Version() || tc.Language != "typescript" {
t.Fatalf("ace.typescript = %+v, want v26.10.0 and recipe %s", tc, m.Recipe.Version())
if tc.Version != "v26.10" || tc.Recipe != m.Recipe.Version() || tc.Language != "typescript" {
t.Fatalf("ace.typescript = %+v, want v26.10 and recipe %s", tc, m.Recipe.Version())
}
if a := m.Health(ctx); a.State != health.StateHealthy || !a.Healthy {
t.Errorf("health = %+v, want healthy", a)
+6 -1
View File
@@ -24,6 +24,10 @@
"serves": ["current", "drop", "pause", "resume", "steps", "kept-artifact", "output"]
}
],
"consumes": [
"mesh-catalog.seat-definition-changed",
"mesh-catalog.seat-definition-removed"
],
"config": {
"capacity": {
"type": "number",
@@ -76,7 +80,8 @@
"seat:npm-package-registry.publish",
"seat:npm-package-registry.publish-part",
"seat:mesh-module-assigner.assignments",
"seat:mesh-catalog.module"
"seat:mesh-catalog.module",
"seat:mesh-catalog.seat-definitions"
],
"own-secrets": {
"broker": "${dir:state}/broker"
@@ -71,6 +71,8 @@ type ForgeReads interface {
Pull(repository string, number int64) (gitseat.PullAnswer, error)
// Branch is a repository branch's head (the seat git's verb branch).
Branch(repository, branch string) (gitseat.BranchAnswer, error)
// Contains is whether commit is in (or is) the commit in (the seat git's verb contains).
Contains(repository, commit, in string) (bool, error)
}
// kvKept is the store over the runtime.
@@ -42,7 +42,9 @@ type fakeForge struct {
open []gitseat.OpenPull
pulls map[string]gitseat.Pull
heads map[string]string // "<repository>@<branch>" -> head
fail bool
// contains says whether commit is in `in`; nil means every commit is in every other (a straight line forward)
contains func(commit, in string) bool
fail bool
}
func (f *fakeForge) OpenPulls() (gitseat.OpenPullsAnswer, error) {
@@ -61,6 +63,15 @@ func (f *fakeForge) Branch(repository, branch string) (gitseat.BranchAnswer, err
}
return gitseat.BranchAnswer{Branch: branch, Repository: repository, Head: head}, nil
}
func (f *fakeForge) Contains(repository, commit, in string) (bool, error) {
if f.fail {
return false, errors.New("the forge is away")
}
if f.contains == nil {
return true, nil
}
return f.contains(commit, in), nil
}
func (f *fakeForge) Pull(repository string, number int64) (gitseat.PullAnswer, error) {
p, ok := f.pulls[fmt.Sprintf("%s#%d", repository, number)]
if !ok {
@@ -44,7 +44,7 @@ type FindingWriter interface {
// generated contract of the seat that defines it (A-17). It is started once the runtime serves (A-40).
func NewFindingWriter() (*views.HeldWriter[meshconditions.Finding, meshconditions.FindingCleared, meshconditions.FindingsStated, meshconditions.FindingsAskedEvent], error) {
return views.NewStdioHeldWriter(views.WriterConfig[meshconditions.Finding, meshconditions.FindingCleared, meshconditions.FindingsStated, meshconditions.FindingsAskedEvent]{
Asked: meshconditions.SubjectFindingsAsked, Module: Module, State: stdio.State("findings"), Kinds: []string{ApprovalCarriedOpen}})
Asked: meshconditions.SubjectFindingsAsked, Module: Module, State: stdio.State("findings"), Kinds: []string{ApprovalCarriedOpen, SourceUncompared}})
}
// approvalID is the id of a pull request's finding: its repository and number, each escaped as a key's name is.
@@ -32,7 +32,7 @@ func (l *listening) get() string { l.mu.Lock(); defer l.mu.Unlock(); return l.n
// The events this owner follows: the forge bridge's word on an act owed, and the step events the delivery records are built
// from (consumer.go).
var followed = append(append([]string{owedEndedEvent}, stepPatterns()...), pullUpdatedEvent, pullMergedEvent, pullClosedEvent, registrationChangedEvent)
var followed = append(append([]string{owedEndedEvent}, stepPatterns()...), pullUpdatedEvent, pullMergedEvent, pullClosedEvent, registrationChangedEvent, forgeMergedEvent)
func main() {
settingsFile := os.Getenv("MESH_DELIVERY_GATE_SETTINGS")
@@ -118,7 +118,13 @@ func run(h *Holder, l *listening) {
l.set("listening")
logf("[mesh-delivery] listening: %s", strings.Join(followed, ", "))
// A watched branch no source key holds starts from the forge, once at each start and at each registration change (A-401).
go startSourcesAtStart(h)
// then each head is compared with the forge's, and a merge made outside the gate is followed (A-464).
go func() {
startSourcesAtStart(h)
if _, err := h.FollowAtStart(); err != nil {
logf("[mesh-delivery] the sources were not all compared with the forge's heads: %v", err)
}
}()
// A missed pull event is put right from the forge's own record, once at each start (§716).
if err := h.ReadForge(); err != nil {
logf("[mesh-delivery] the open pull requests were not read from the forge, they are read again at the next start: %v", err)
@@ -179,6 +185,9 @@ func take(h *Holder, e stdio.Envelope) error {
_, err := h.StartSources()
return err
}
if forgeMergedEvent != "" && stdio.TopicMatches(forgeMergedEvent, e.Key) {
return h.ForgeMerged(e.Body)
}
if e.Key == pullClosedEvent {
return errors.Join(h.DeliveryClosed(e.Body), h.pullEnded(e.Body, "closed"))
}
@@ -134,13 +134,16 @@ func TestItClaimsTheDeliverySeatAndFollowsWhatItHandles(t *testing.T) {
}
}
}
// From G's delivery the gate hears neither the bridge's merged and pushed messages nor the teller's answers nor the view's
// reach (design 47 G §592, §593): it consumes the indexer's take-in.taken and the bridge's owed.ended, and the git events that clear
// From G's delivery the gate hears neither the bridge's pushed messages nor the teller's answers nor the view's reach
// (design 47 G §592, §593); it hears the bridge's merged message alone, to follow every merge (A-464): it consumes the indexer's take-in.taken and the bridge's owed.ended, and the git events that clear
// an approval carried.
if want := append(slices.Clone(followed), meshconditions.Seat+"."+meshconditions.EventFindingsAsked); !reflect.DeepEqual(m.Consumes, want) {
t.Fatalf("consumes %v, follows %v", m.Consumes, followed)
}
for _, c := range m.Consumes {
if c == forgeMergedEvent {
continue // A-464: the source follows every merge, so the gate hears the forge bridge's merged message, and it alone
}
for _, gone := range []string{".forge.", "teller", "view", "forge-pull", "forge-status"} {
if strings.Contains(c, gone) {
t.Errorf("the manifest still consumes %s, which G's gate does not hear (§431, §592, §593)", c)
@@ -0,0 +1,278 @@
package main
// The gate's source of a watched branch follows every merge into that branch, whoever made it (design 47 G A-464, A-464b, amending
// A-463): on the forge bridge's `merged` of a repository the source names, at the branch the source watches, the gate moves that
// repository's head to the merge and says source-moved, as a take-in moves it; at its start it compares each head with the
// forge's branch head, asked of the seat git, and moves every one behind it the same way, so a merge whose message was lost is
// followed too; gate_source_follow does the same for one repository now. A head the source does not name, or a branch it does
// not watch, is left: only a start or a registration change adds them (sourcestart.go).
import (
"context"
"encoding/json"
"errors"
"fmt"
"sort"
"strconv"
"strings"
"time"
"git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshhandactsclient"
"git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-forge-bridge/forgemsg"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/trunk"
"git.novox.be/novox/mesh-sdk/go/keys"
)
// forgeMergedEvent is the forge bridge's merged message of any repository, as the manifest consumes it.
var forgeMergedEvent = declared("forge.*.*.merged")
// moveHead moves the head the branch's source names for repository to head, and says source-moved with the key's revision. It
// answers false, and writes nothing, when the branch has no source key, the key does not name the repository or names the head
// already. group is the group that made the merge, empty for a merge no group of the gate made.
func (h *Holder) moveHead(branch, repository, head, group string) (moved bool, err error) {
if h.Source == nil || head == "" {
return false, nil
}
held, _, err := h.Source.Read(branch)
if err != nil || held == nil {
return false, err
}
name := ""
for r := range held.Heads {
if trunk.SameRepository(r, repository) {
name = r
}
}
if name == "" || held.Heads[name] == head {
return false, nil // not named, or no change: no revision and no source-moved (A-464c)
}
was := held.Heads[name]
// Only forward (A-464c): the head held must be contained in the merge, asked of the seat git.
if h.Forge == nil {
return false, errors.New("this gate reads no forge, so it cannot tell whether the merge is ahead of the head")
}
ahead, err := h.Forge.Contains(name, was, head)
if err != nil {
return false, fmt.Errorf("whether %.12s is in %.12s of %s cannot be read, so the head stays: %w", was, head, name, err)
}
if !ahead {
h.logf("[mesh-delivery] source.%s keeps %s at %.12s: the merge %.12s does not contain it (heard again, late or out of order), left without effect", branch, name, was, head)
return false, nil
}
rev, err := h.Source.Set(branch, func(prev *Source) (Source, bool) {
moved = false
if prev == nil || prev.Heads[name] != was {
return Source{}, false // another writer moved it first
}
next := Source{Group: group, Heads: map[string]string{}}
for r, c := range prev.Heads {
next.Heads[r] = c
}
next.Heads[name] = head
moved = true
return next, true
})
if err != nil {
return false, fmt.Errorf("source.%s cannot be moved: %w", branch, err)
}
if !moved || h.SaySeat == nil {
return moved, nil
}
at := h.Now().UTC()
return true, h.SaySeat(seat.SubjectSourceMoved(keys.EscapeName(branch)),
seat.SourceMovedEvent{Branch: branch, Group: group, Revision: int64(rev), At: &at},
"source-moved."+keys.EscapeName(branch)+"."+strconv.FormatUint(rev, 10))
}
// ForgeMerged takes one merged message of the forge bridge: the head of its repository in the source of the branch it landed on
// moves to the merge. A merge no delivery group made (the message names no group) is said so. A message that cannot be read, or
// names no branch or merge, is said and taken: heard again it would be refused again.
func (h *Holder) ForgeMerged(body json.RawMessage) error {
f, err := forgemsg.DecodeForge(body)
if err != nil {
h.logf("[mesh-delivery] a merged message of the forge bridge is not readable, the source does not follow it: %v", err)
return nil
}
if f.Kind != forgemsg.ForgeMerged {
return nil
}
if f.Base == "" || f.Merge == "" {
h.logf("[mesh-delivery] the merged message of %s names no branch or no merge, the source does not follow it", f.Repository)
return nil
}
moved, err := h.moveHead(f.Base, f.Repository, f.Merge, f.Group)
if err != nil {
return err
}
if moved {
if f.Group == "" { // A-464b: a merge whose message names no delivery group is one the gate did not make
h.logf("[mesh-delivery] source.%s follows the merge %.12s of %s: not made by a delivery group of the gate", f.Base, f.Merge, f.Repository)
} else {
h.logf("[mesh-delivery] source.%s follows the merge %.12s of %s, made by the delivery group %s", f.Base, f.Merge, f.Repository, f.Group)
}
}
return nil
}
// SourceUncompared is the finding of a start comparison that left a head uncompared (A-464e).
const SourceUncompared = "source-uncompared"
// FollowAtStart compares each head of every source key with the forge's head of that branch and moves every one behind it, at
// each start. It ends only when every head was compared (A-464e): a head the forge could not be asked for (the seat git not
// answering yet, as when its bundle starts in the same tool runner) is compared again after a growing pause of at most a
// minute, and the finding source-uncompared names the repositories and the last error while any stands. It answers the heads
// moved as "<repository>@<branch>".
func (h *Holder) FollowAtStart() ([]string, error) {
if h.Source == nil || h.Forge == nil {
return nil, errors.New("this gate keeps no source or reads no forge")
}
var moved []string
var only map[string]bool
for wait := 2 * time.Second; ; wait = min(wait*2, time.Minute) {
got, left := h.followPass(only)
moved = append(moved, got...)
if len(left) == 0 {
h.clearUncompared()
h.logf("[mesh-delivery] the sources compared with the forge's heads at the start: moved %v", moved)
return moved, nil
}
only = map[string]bool{}
var names []string
var last error
for n, err := range left {
only[n] = true
names = append(names, n)
last = err
}
sort.Strings(names)
h.sayUncompared(names, last)
h.logf("[mesh-delivery] %d heads are not compared with the forge's yet (%s; last: %v); asking again in %s", len(names), strings.Join(names, ", "), last, wait)
startSleep(wait)
}
}
// followPass compares the heads named in only (every head when nil) once; it answers the heads moved and, by "<repository>@<branch>",
// each head it could not compare with why. A source it cannot list or read is one entry named for itself.
func (h *Holder) followPass(only map[string]bool) (moved []string, left map[string]error) {
left = map[string]error{}
branches, err := h.Source.Branches()
if err != nil {
left["the branches of the source"] = err
return nil, left
}
for _, branch := range branches {
held, _, err := h.Source.Read(branch)
if err != nil {
left["source."+branch] = err
continue
}
if held == nil {
continue
}
for repository, head := range held.Heads {
name := repository + "@" + branch
if only != nil && !only[name] {
continue
}
ok, err := h.followForge(branch, repository, head)
if err != nil {
left[name] = err
continue
}
if ok {
moved = append(moved, name)
}
}
}
return moved, left
}
func (h *Holder) sayUncompared(names []string, last error) {
if h.Findings == nil {
return
}
at := h.Now().UTC()
err := h.Findings.Write(context.Background(), meshconditions.Finding{Scope: FindingScope, ID: SourceUncompared, Kind: SourceUncompared,
Subject: meshconditions.Subject{Scope: FindingScope, ID: SourceUncompared},
Severity: meshconditions.FindingSeverityWarning, Resolver: "operator",
Summary: fmt.Sprintf("the gate's source is not compared with the forge's head for %s; the last error: %v", strings.Join(names, ", "), last),
Headline: "The gate's source is not compared with the forge",
Explanation: "At its start the gate compares each head of its source with the forge's head of the branch. A head it could not compare is asked again after a growing pause, and this clears once every head is compared.",
Needs: "the seat git answering; gate_source_follow moves one repository's head by hand.",
Source: SourceUncompared, Observer: Module, At: at, Since: at})
if err != nil {
h.logf("[mesh-delivery] the finding %s is not written: %v", SourceUncompared, err)
}
}
func (h *Holder) clearUncompared() {
if h.Findings == nil {
return
}
for _, f := range h.Findings.Held() {
if f.Scope == FindingScope && f.ID == SourceUncompared && f.Kind == SourceUncompared {
if err := h.Findings.Clear(context.Background(), FindingScope, SourceUncompared, SourceUncompared, "every head is compared with the forge's"); err != nil {
h.logf("[mesh-delivery] the finding %s is not cleared: %v", SourceUncompared, err)
}
}
}
}
// followForge moves one repository's head in a branch's source to the forge's head of the branch when they differ.
func (h *Holder) followForge(branch, repository, held string) (bool, error) {
a, err := h.Forge.Branch(repository, branch)
if err == nil && a.Error != "" {
err = errors.New(a.Error)
}
if err == nil && a.Head == "" {
err = errors.New("the forge named no head")
}
if err != nil {
return false, err
}
if sameCommit(held, a.Head) && len(held) == len(a.Head) {
return false, nil
}
moved, err := h.moveHead(branch, repository, a.Head, "")
if moved {
h.logf("[mesh-delivery] source.%s follows the forge: %s from %.12s to %.12s", branch, repository, held, a.Head)
}
return moved, err
}
// SourceFollow moves one repository's head to the forge's head of the branch by hand, after the hand-act log took it.
func (h *Holder) SourceFollow(branch, repository, why, by string, acts handActs) (map[string]any, error) {
if strings.TrimSpace(branch) == "" || strings.TrimSpace(repository) == "" || strings.TrimSpace(why) == "" {
return nil, errors.New("say the branch, the repository and why its head follows the forge")
}
if h.Source == nil || h.Forge == nil {
return nil, errors.New("this gate keeps no source or reads no forge")
}
if acts == nil {
return nil, errors.New("the hand-act log cannot be reached here, so nothing was moved")
}
held, _, err := h.Source.Read(branch)
if err != nil {
return nil, err
}
if held == nil {
return nil, fmt.Errorf("the gate keeps no source of the branch %s", branch)
}
name := ""
for r := range held.Heads {
if trunk.SameRepository(r, repository) {
name = r
}
}
if name == "" {
return nil, fmt.Errorf("the source of %s names no head of %s", branch, repository)
}
if _, err := acts.Record(meshhandactsclient.Request{What: "moved the head of " + repository + " in the source of the branch " + branch + " to the forge's head",
Why: why, Cause: "gate_source_follow", Verb: "source-follow"}); err != nil {
return nil, fmt.Errorf("the hand act was not recorded through mesh-hand-acts, so nothing was moved: %w", err)
}
moved, err := h.followForge(branch, name, held.Heads[name])
return map[string]any{"branch": branch, "repository": name, "moved": moved, "by": by}, err
}
@@ -0,0 +1,193 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
meshcatalog "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-forge-bridge/forgemsg"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// Design 47 G A-464, A-464b: the forge bridge's merged message moves the head of its repository in the source of the branch it
// landed on and says source-moved; a merge naming no delivery group is said not made by one; a branch no source key holds, a
// repository the key does not name, and a head the key holds already move nothing.
func TestAMergedMessageMovesTheSourceOfItsBranchWhoeverMadeTheMerge(t *testing.T) {
old, merge := strings.Repeat("a", 40), strings.Repeat("b", 40)
body := func(repository, base, group string) []byte {
f := forgemsg.Forge{V: 1, Kind: forgemsg.ForgeMerged, Repository: repository, Base: base, Merge: merge, Head: merge, Number: 14, Root: strings.Repeat("d", 40)}
data, err := forgemsg.EncodeForge(f)
if err != nil {
t.Fatal(err)
}
return data
}
for _, c := range []struct {
name, repository, base string
moved bool
}{
{"watched branch, named repository", "novox/mesh-contracts", "main", true},
{"a branch no source key holds", "novox/mesh-contracts", "other", false},
{"a repository the key does not name", "novox/other", "main", false},
} {
t.Run(c.name, func(t *testing.T) {
src := &memSource{vals: map[string]Source{"main": {Group: "g", Heads: map[string]string{"novox/mesh-contracts": old}}}}
var said []string
var logged []string
h := &Holder{Now: time.Now, Source: src, Forge: &fakeForge{},
Logf: func(f string, a ...any) { logged = append(logged, f) },
SaySeat: func(subject string, _ any, id string) error { said = append(said, id); return nil }}
if err := h.ForgeMerged(body(c.repository, c.base, "")); err != nil {
t.Fatal(err)
}
got := src.vals["main"]
if c.moved != (got.Heads["novox/mesh-contracts"] == merge) || c.moved != (len(said) == 1) || (c.moved && got.Group != "") {
t.Fatalf("source %+v said %v: moved wanted %v", got, said, c.moved)
}
if c.moved && !strings.Contains(strings.Join(logged, "\n"), "not made by a delivery group of the gate") {
t.Errorf("a merge no group made is not said so: %v", logged)
}
// Heard again, the key names the head already: nothing is moved or said twice.
if err := h.ForgeMerged(body(c.repository, c.base, "")); err != nil || len(said) > 1 {
t.Errorf("heard twice: %v, said %v", err, said)
}
})
}
}
// Design 47 G A-464: at its start the gate compares each head with the forge's branch head and moves every one behind it; a head
// the forge cannot read is said and left; gate_source_follow does it for one repository after the hand-act log took it.
func TestTheStartAndTheToolMoveAHeadBehindTheForge(t *testing.T) {
old, now := strings.Repeat("a", 40), strings.Repeat("c", 40)
mk := func() (*Holder, *branchSource, *[]string) {
src := &branchSource{memSource{vals: map[string]Source{"main": {Heads: map[string]string{"o/f1": old, "o/f2": old, "o/f3": now}}}}}
var said []string
return &Holder{Now: time.Now, Source: src, Logf: func(string, ...any) {},
Forge: &fakeForge{heads: map[string]string{"o/f1@main": now, "o/f3@main": now}}, // o/f2 cannot be read
SaySeat: func(_ string, _ any, id string) error { said = append(said, id); return nil }}, src, &said
}
h, src, said := mk()
fw := &fakeFindings{}
h.Findings = fw
forge := h.Forge.(*fakeForge)
var pauses []time.Duration
var heldWhileWaiting int
prevSleep := startSleep
startSleep = func(d time.Duration) {
pauses = append(pauses, d)
heldWhileWaiting = len(fw.held) // the finding stands while a head is uncompared
forge.heads["o/f2@main"] = now // the seat git answers by the second pass
}
defer func() { startSleep = prevSleep }()
moved, err := h.FollowAtStart()
if err != nil || len(moved) != 2 || src.vals["main"].Heads["o/f1"] != now || src.vals["main"].Heads["o/f2"] != now || len(*said) != 2 {
t.Fatalf("start: moved %v err %v heads %v said %v: want o/f1 and, after the second pass, o/f2 moved", moved, err, src.vals["main"].Heads, *said)
}
if len(pauses) != 1 || heldWhileWaiting != 1 || len(fw.held) != 0 {
t.Fatalf("pauses %v, finding held while waiting %d and after %d: want one pause, the finding raised while a head stood uncompared and cleared after", pauses, heldWhileWaiting, len(fw.held))
}
h, src, said = mk()
acts := &handActsFake{refuse: errors.New("log down")}
if _, err := h.SourceFollow("main", "o/f1", "missed a merge", "me", acts); err == nil || src.vals["main"].Heads["o/f1"] != old {
t.Fatalf("moved without the log: %v", err)
}
acts.refuse = nil
if _, err := h.SourceFollow("main", "o/f1", "", "me", acts); err == nil {
t.Fatal("moved with no reason")
}
out, err := h.SourceFollow("main", "o/f1", "missed a merge", "me", acts)
if err != nil || out["moved"] != true || len(acts.took) != 1 || acts.took[0].Cause != "gate_source_follow" || src.vals["main"].Heads["o/f1"] != now {
t.Fatalf("tool: %v %v log %+v heads %v", out, err, acts.took, src.vals["main"].Heads)
}
}
// Design 47 G A-464: the gate takes the forge bridge's merged event of any repository, by the manifest's own pattern.
func TestTheGateTakesTheForgeBridgesMergedEventOfAnyRepository(t *testing.T) {
merge := strings.Repeat("b", 40)
data, err := forgemsg.EncodeForge(forgemsg.Forge{V: 1, Kind: forgemsg.ForgeMerged, Repository: "novox/mesh-contracts", Base: "main", Merge: merge, Head: merge, Number: 14, Root: strings.Repeat("d", 40)})
if err != nil {
t.Fatal(err)
}
src := &memSource{vals: map[string]Source{"main": {Heads: map[string]string{"novox/mesh-contracts": strings.Repeat("a", 40)}}}}
h := &Holder{Now: time.Now, Source: src, Forge: &fakeForge{}, Logf: func(string, ...any) {}}
key := strings.Replace(strings.Replace(forgeMergedEvent, "*", "novox", 1), "*", "mesh-contracts", 1)
if err := take(h, stdio.Envelope{Key: key, Body: data}); err != nil || src.vals["main"].Heads["novox/mesh-contracts"] != merge {
t.Fatalf("event %q: %v, source %+v", key, err, src.vals["main"])
}
}
// Design 47 G A-464c: a head moves only forward and only when it changes. A merged message for a commit that does not contain
// the held head (replayed, late, out of order) moves nothing and is said; a second message for the same head writes no
// revision; and a take-in.taken for a head a merged already moved writes none and says no source-moved.
func TestAHeadMovesOnlyForwardAndOnlyWhenItChanges(t *testing.T) {
oldC, newC := strings.Repeat("a", 40), strings.Repeat("b", 40)
msg := func(merge string) []byte {
data, err := forgemsg.EncodeForge(forgemsg.Forge{V: 1, Kind: forgemsg.ForgeMerged, Repository: "novox/app", Base: "main", Merge: merge, Head: merge, Number: 3, Root: strings.Repeat("d", 40)})
if err != nil {
t.Fatal(err)
}
return data
}
src := &memSource{vals: map[string]Source{"main": {Heads: map[string]string{"novox/app": newC}}}}
var said, logged []string
// newC contains oldC, not the other way round.
forge := &fakeForge{contains: func(commit, in string) bool { return commit == in || (commit == oldC && in == newC) }}
h := &Holder{Now: time.Now, Source: src, Forge: forge, Logf: func(f string, a ...any) { logged = append(logged, f) },
SaySeat: func(_ string, _ any, id string) error { said = append(said, id); return nil }}
if err := h.ForgeMerged(msg(oldC)); err != nil || src.vals["main"].Heads["novox/app"] != newC || src.rev != 0 || len(said) != 0 || len(logged) == 0 {
t.Fatalf("a replayed old merge: %v, source %+v, revisions %d, said %v, logged %v: want nothing moved and a log line", err, src.vals["main"], src.rev, said, logged)
}
if err := h.ForgeMerged(msg(newC)); err != nil || src.rev != 0 || len(said) != 0 {
t.Fatalf("a second message for the same head wrote revision %d, said %v", src.rev, said)
}
// A take-in.taken naming the head a merged moved already writes no revision and says nothing.
body := []byte(`{"v":1,"kind":"taken","key":"g.feat","what":"group","group":"feat","branch":"main","source":7,` +
`"merges":[{"repository":"novox/app","number":3,"merge":"` + newC + `"}],"entries":[]}`)
if err := h.moveSource(mustTaken(t, body)); err != nil || src.rev != 0 || len(said) != 0 {
t.Fatalf("take-in for a head already moved: %v, revision %d, said %v", err, src.rev, said)
}
// A real forward move still writes one revision.
src.vals["main"] = Source{Heads: map[string]string{"novox/app": oldC}}
if err := h.ForgeMerged(msg(newC)); err != nil || src.rev != 1 || len(said) != 1 {
t.Fatalf("a forward move: %v, revision %d, said %v", err, src.rev, said)
}
}
func mustTaken(t *testing.T, body []byte) meshcatalog.TakeInTaken {
t.Helper()
var v meshcatalog.TakeInTaken
if err := json.Unmarshal(body, &v); err != nil {
t.Fatal(err)
}
return v
}
// Design 47 G A-464d: a take-in.taken arriving after a later merge moved the head leaves the head where it is.
func TestATakeInTakenAfterALaterMergeLeavesTheHead(t *testing.T) {
oldC, newC := strings.Repeat("a", 40), strings.Repeat("b", 40)
src := &memSource{vals: map[string]Source{"main": {Heads: map[string]string{"novox/app": newC}}}}
var said []string
h := &Holder{Now: time.Now, Source: src, Logf: func(string, ...any) {},
Forge: &fakeForge{contains: func(commit, in string) bool { return commit == in || (commit == oldC && in == newC) }},
SaySeat: func(_ string, _ any, id string) error { said = append(said, id); return nil }}
body := []byte(`{"v":1,"kind":"taken","key":"g.feat","what":"group","group":"feat","branch":"main","source":7,` +
`"merges":[{"repository":"novox/app","number":3,"merge":"` + oldC + `"}],"entries":[]}`)
if err := h.moveSource(mustTaken(t, body)); err != nil || src.vals["main"].Heads["novox/app"] != newC || src.rev != 0 || len(said) != 0 {
t.Fatalf("take-in of an older merge: %v, source %+v, revision %d, said %v: want the head left at the later merge", err, src.vals["main"], src.rev, said)
}
}
// fakeFindings is the gate's finding writer in memory.
type fakeFindings struct{ held []meshconditions.Finding }
func (f *fakeFindings) Write(_ context.Context, x meshconditions.Finding) error {
f.held = append(f.held, x)
return nil
}
func (f *fakeFindings) Clear(_ context.Context, _, _, _, _ string) error { f.held = nil; return nil }
func (f *fakeFindings) Held() []meshconditions.Finding { return f.held }
@@ -55,6 +55,18 @@ func (s seatGit) Branch(repository, branch string) (gitseat.BranchAnswer, error)
return a, s.call(gitseat.VerbBranch, gitseat.BranchInput{Repository: repository, Branch: branch}, &a)
}
// Contains is whether commit is the commit in or an ancestor of it.
func (s seatGit) Contains(repository, commit, in string) (bool, error) {
var a gitseat.ContainsAnswer
if err := s.call(gitseat.VerbContains, gitseat.ContainsInput{Repository: repository, Commit: commit, In: in}, &a); err != nil {
return false, err
}
if a.Error != "" {
return false, errors.New(a.Error)
}
return a.Contains, nil
}
// watchedRepositories are the repositories each watched branch is registered for, each once, sorted.
func watchedRepositories(regs []meshcatalog.Registration) map[string][]string {
seen := map[string]map[string]bool{}
@@ -89,5 +89,16 @@ func sourceTools(h *Holder, acts handActs) []stdio.Tool {
Run: func(a map[string]any) (any, error) {
return h.SourceStart(strArg(a, "branch"), strArg(a, "why"), stdio.CallerOf(a), acts)
}},
{Name: "gate_source_follow",
Description: "Move the head one watched branch's source names for one repository to the forge's head of that branch now, as " +
"the gate does when it hears a merge and at its start (design 47 G A-464); recorded in the hand-act log of the seat " +
"mesh-hand-acts first. Nothing is moved when the source already names the forge's head.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"branch": str("the watched branch"), "repository": str("the repository, owner/name"),
"why": str("why its head follows the forge by hand, in plain words")},
"required": []string{"branch", "repository", "why"}},
Run: func(a map[string]any) (any, error) {
return h.SourceFollow(strArg(a, "branch"), strArg(a, "repository"), strArg(a, "why"), stdio.CallerOf(a), acts)
}},
}
}
@@ -9,6 +9,7 @@ import (
mdc "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshdeliveryclient"
meshcatalog "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/trunk"
"git.novox.be/novox/mesh-sdk/go/keys"
)
@@ -107,9 +108,43 @@ func (h *Holder) moveSource(t meshcatalog.TakeInTaken) error {
for _, m := range t.Merges {
heads[m.Repository] = m.Merge
}
// A head moves only forward (A-464d): a repository whose held head is not contained in the merge is left where it is.
if held, _, err := h.Source.Read(t.Branch); err != nil {
return err
} else if held != nil && h.Forge != nil {
for r, c := range heads {
for name, was := range held.Heads {
if !trunk.SameRepository(name, r) || was == c {
continue
}
ahead, err := h.Forge.Contains(name, was, c)
if err != nil {
return fmt.Errorf("whether %.12s is in %.12s of %s cannot be read, so the head stays: %w", was, c, name, err)
}
if !ahead {
h.logf("[mesh-delivery] source.%s keeps %s at %.12s: the take-in's merge %.12s does not contain it, left without effect", t.Branch, name, was, c)
delete(heads, r)
}
}
}
if len(heads) == 0 {
return nil
}
}
wrote := false
rev, err := h.Source.Set(t.Branch, func(prev *Source) (Source, bool) {
wrote = false
next := Source{Group: t.Group, Heads: heads}
if prev != nil {
same := true
for r, c := range heads {
if prev.Heads[r] != c {
same = false
}
}
if same {
return Source{}, false // a merged message moved these heads already: no revision (A-464c)
}
next.Heads = map[string]string{}
for r, c := range prev.Heads {
next.Heads[r] = c
@@ -118,12 +153,13 @@ func (h *Holder) moveSource(t meshcatalog.TakeInTaken) error {
next.Heads[r] = c
}
}
wrote = true
return next, true
})
if err != nil {
return fmt.Errorf("source.%s cannot be moved: %w", t.Branch, err)
}
if h.SaySeat == nil {
if h.SaySeat == nil || !wrote {
return nil
}
at := h.Now().UTC()
@@ -84,7 +84,7 @@ func TestATakeInTakenOpensOneRecordAndMovesItsBranchsSource(t *testing.T) {
if s := src.vals["main"]; s.Group != "feat" || s.Heads["novox/app"] != merge {
t.Fatalf("source.main %+v", s)
}
if len(events) != 2 || !strings.HasSuffix(events[0].subject, ".event.source-moved.main") {
if len(events) != 1 || !strings.HasSuffix(events[0].subject, ".event.source-moved.main") {
t.Fatalf("said %+v", events)
}
drop := json.RawMessage(`{"v":1,"kind":"taken","key":"m.00000000000000000002","what":"revision","dropped":true,"branch":"main","source":8,` +
@@ -92,7 +92,7 @@ func TestATakeInTakenOpensOneRecordAndMovesItsBranchsSource(t *testing.T) {
if err := c.Hear(stdio.Envelope{Key: "mesh-delivery-indexer.take-in.taken", Body: drop}); err != nil {
t.Fatal(err)
}
if kept, _ := h.Records.Kept(); len(kept) != 1 || len(events) != 2 {
if kept, _ := h.Records.Kept(); len(kept) != 1 || len(events) != 1 {
t.Fatalf("a hand drop opened %d records and said %d events", len(kept), len(events))
}
}
+2
View File
@@ -23,6 +23,7 @@
"git.pull-merged",
"git.pull-closed",
"mesh-catalog.registration-changed",
"mesh-delivery-forge-bridge.forge.*.*.merged",
"mesh-conditions.findings-asked"
],
"emits": [
@@ -383,6 +384,7 @@
"gate_records",
"gate_source",
"gate_source_again",
"gate_source_follow",
"gate_source_start"
],
"config": {
+1
View File
@@ -33,6 +33,7 @@
"backup_delete_retired",
"backup_deleted"
],
"secrets-owner": "${machine:account}",
"own-secrets": {
"repository": "${dir:state}/repository.secret",
"broker": "${dir:state}/broker"