Compare commits

..
Author SHA1 Message Date
jschoubben e0d1de0475 process: a part opens what it cannot work without as essential, the rest as optional (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-sdk@7ac4493ce9ab
mesh/delivery-group group fix/1945-ask-again-and-why merged: every member is merged
mesh/check pass e0d1de047585 on 3db051be57ca | unit 1642 run, 752 kept, 2 skipped | integration 1710 run, 803 kept, 2 skipped | over bound: unit, inte…
The delivery builder exited twice on 2026-10-25 for a bucket the bus
refused at start, though it could build without it: a grant reaches the
bus with a delivery, so a refusal at start is often only a grant not
there yet. EssentialBucket stops a start naming the bucket and its grant;
OptionalBucket and NotWatched say what is not heard and go on. Here so
the seven delivery parts take one copy, not seven.
2026-10-25 22:12:20 +01:00
mesh-tools db26f7e8b6 Merge pull request 'The put's words: delivery.Put, a want's put and hold marks, a grant that says which node's accounts it names (ADR 0546)' (#82) from feat/put-version into main 2026-10-25 20:35:28 +00:00
jschoubben d4ae4b6790 Say the put's tools, not a seat's verbs, in the put's words (ADR 0546)
mesh/delivery merged: source.main moved to novox/mesh-sdk@db26f7e8b6f5
mesh/check pass d4ae4b6790b8 on db26f7e8b6f5 | unit 1102 run, 680 kept, 2 skipped | integration 1103 run, 722 kept, 2 skipped | over bound: unit (slow…
2026-10-25 21:08:10 +01:00
jschoubben 2f6741b7f8 No seat and no compose verb for the put: the controller takes no change, so the assigner serves the put as its own tools from compositions the controller already writes (ADR 0546) 2026-10-25 21:07:58 +01:00
jschoubben 20a6dc4920 A grant says whether it names a module's accounts on a node, for the assigner and the compose verb (ADR 0546) 2026-10-25 20:50:51 +01:00
jschoubben c4f37c279a The assigner's seat with put, release and puts, the controller's compose verb, a put's key and value, and the view answers no option that authorises (ADR 0546) 2026-10-25 20:49:29 +01:00
jschoubben 7f89cb51de Merge pull request 'Keep a build node's pause in a state of its own, and pull as not paused when the pause cannot be read (steps bucket full, 2026-10-25)' (#81) from fix/steps-bucket-ages into main
Reviewed-on: #81
2026-10-25 18:06:32 +00:00
jschoubben 518dae5ec9 Pull as not paused when the pause cannot be read, and say so once, instead of ending the puller silently
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/delivery-group group fix/steps-bucket-ages held: a member is held: novox/mesh-sdk@518dae5ec9c8 (merged without a passing check: only a person decides what…
mesh/check checking 518dae5ec9c8 on 7f89cb51de15 | novox/mesh-sdk#81 in group fix/steps-bucket-ages
A Load error ended Pull, and the modules run it in a goroutine whose error nobody reads: every
build node would stop pulling when the new pauses state could not be read (or when Pull reached
restore before Serve). A node kept from building is held by capacity 0, not by a pause.
2026-10-25 18:56:17 +01:00
jschoubben e8f8ef63fa Keep a build node's pause in a pauses state of its own, so the steps state can drop its old keys without ending a pause
The steps state of the build modules filled to the bus's 64 MiB cap (27,370 keys of mesh-build-go,
none ever removed) and refused a passed check's end. Its keys get an age in the catalogue; the
pause key lived among them and would then have ended by itself. Puller.PausePut writes it to
delivery.PausesState instead; Put stays the fallback for a module not yet moved.
2026-10-25 18:51:04 +01:00
jschoubben 4f4d5ca9c9 Merge pull request 'Wake a parked build puller when its capacity changes, so a node raised from capacity 0 takes asks (issue 1887)' (#80) from fix/puller-wakes-on-capacity into main
Reviewed-on: #80
2026-10-25 17:10:33 +00:00
jschoubben a8fbabbfc3 Fold the puller-wake and Served tests into existing tests to stay within the test budget (issue 1887)
mesh/delivery merged: source.main moved to novox/mesh-sdk@4f4d5ca9c956
mesh/delivery-group group fix/puller-wakes-on-capacity merged: every member is merged
mesh/check pass a8fbabbfc373 on 4f4d5ca9c956 | unit 1102 run, 679 kept, 2 skipped | integration 1103 run, 721 kept, 2 skipped | over bound: unit, inte…
2026-10-25 17:44:03 +01:00
jschoubben b9911dc1c1 Start the puller's capacity watch once Serve has the runtime's channel, since a build module pulls before it serves (issue 1887)
A watch asked before Serve is refused, which would have left every build
module on its settings file again. stdio.Served closes when Serve has the
channel; WatchConfigServed waits on it, and Pull starts the watch beside its
loop.
2026-10-25 17:26:10 +01:00
jschoubben e6b2df6506 Wake a parked build puller on its capacity's own change, so a node started at capacity 0 takes asks once raised (issue 1887)
g14's Go builder sat idle at capacity 2 while delivery builds waited: a puller
with no room waited on its wake alone, which only a pause, a resume or a step's
end gives, and it read its capacity from the settings file only when woken.
The puller now follows the module's config value capacity through the
runtime's watch of what mesh-config resolved (stdio.WatchConfig), uses the
value it hands, and wakes on each change: no timer, no file watch, and no wait
for the node-engine to fill the file again. It says when it parks and when it
wakes, with its capacity.
2026-10-25 17:23:28 +01:00
29 changed files with 671 additions and 639 deletions
+19 -39
View File
@@ -8,7 +8,6 @@ import (
"slices"
"sort"
"strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
@@ -171,36 +170,16 @@ func (s AskSteps) check(ctx context.Context, job delivery.CheckJob) (any, error)
if job.Runner != "" && (job.RunnerWhen == "" || changesUnder(f.Changed, job.RunnerWhen)) {
runner = job.Runner
}
// The levels run side by side (issue 1921, try speed S10; the operator's yes to D1, ADR 0541): the integration level
// does not wait for the unit level. They are answered in their order, so the first level that did not pass is still
// the first failure the asker reads (ADR 0484 clause 13); the levels after it are kept in the answer for the log.
type ran struct {
la delivery.LevelAnswer
passed bool
err error
}
runs := make([]ran, len(job.Levels))
var wg sync.WaitGroup
for i, level := range job.Levels {
wg.Add(1)
go func() {
defer wg.Done()
runs[i].la = delivery.LevelAnswer{Level: level}
runs[i].passed, runs[i].err = s.level(ctx, job, level, f, runner, &runs[i].la)
}()
}
wg.Wait()
failed := false
for _, r := range runs {
if r.err != nil {
if failed {
continue // a level after the first failure that could not run: the failure is the answer
}
answer.Levels = append(answer.Levels, r.la)
return nil, r.err
for _, level := range job.Levels {
la := delivery.LevelAnswer{Level: level}
passed, err := s.level(ctx, job, level, f, runner, &la)
answer.Levels = append(answer.Levels, la)
if err != nil {
return nil, err
}
if !passed {
return answer, nil // the first level that did not pass ends the check (ADR 0484 clause 13)
}
answer.Levels = append(answer.Levels, r.la)
failed = failed || !r.passed
}
return answer, nil
}
@@ -229,9 +208,6 @@ func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string
_ = json.Unmarshal(r, &t)
return t.Passed, nil
}
// Every test step of the level runs, also after one that did not pass, so one check names every failing test of the
// level and not only the first (issue 1921); the level as a whole passes only when each of its steps did.
all := true
if s.Digest == nil {
// A module with no digest (Node) tests each module directory of the tree, those whose tests changed first.
for _, priority := range []bool{true, false} {
@@ -243,12 +219,14 @@ func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string
if err != nil {
return false, fmt.Errorf("the %s level of %s: %w", level, dir, err)
}
all = all && passed
if !passed {
return false, nil
}
}
}
return all, nil
return true, nil
}
todo, whole, keepRunner := s.plan(ctx, job, level, f, runner, withRunner, la)
todo, all, keepRunner := s.plan(ctx, job, level, f, runner, withRunner, la)
if !keepRunner {
runner = "" // an installed module that ignores the runner judges the level as it judges any other (ADR 0535 §2)
}
@@ -257,7 +235,7 @@ func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string
if job.Run != "" {
args["run"] = job.Run
}
if whole {
if all {
if priority {
continue // nothing to put first: the level runs whole after
}
@@ -277,9 +255,11 @@ func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string
if err != nil {
return false, fmt.Errorf("the %s level: %w", level, err)
}
all = all && passed // the packages left still run (issue 1921)
if !passed {
return false, nil
}
}
return all, nil
return true, nil
}
// release removes an ask's workspace, saying a failure: the module's sweep removes it later.
-60
View File
@@ -1,60 +0,0 @@
package buildstep
import (
"context"
"encoding/json"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// 1921/S10 (the operator's yes to D1, ADR 0541): a check's levels run side by side: the integration level has started
// before the unit level ends. The answer holds the levels in their order, so with the unit level failing it still names
// the unit level first; the integration level's answer is kept after it for the log.
func TestACheckRunsItsLevelsSideBySideAndAnswersThemInOrder(t *testing.T) {
integrationStarted := make(chan struct{})
var once sync.Once
var mu sync.Mutex
unitWaited := false
steps := AskSteps{
Fetch: func(context.Context, map[string]any) (any, error) {
return map[string]any{"workspace": "/w", "modules": []string{"."}}, nil
},
Test: func(ctx context.Context, args map[string]any) (any, error) {
switch args["level"] {
case delivery.LevelUnit:
select {
case <-integrationStarted:
mu.Lock()
unitWaited = true
mu.Unlock()
case <-time.After(2 * time.Second):
}
return map[string]any{"passed": false, "level": "unit"}, nil
default:
once.Do(func() { close(integrationStarted) })
return map[string]any{"passed": true, "level": "integration"}, nil
}
},
}
out, err := RunAsk(context.Background(), steps, ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit, delivery.LevelIntegration},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
mu.Lock()
defer mu.Unlock()
if !unitWaited {
t.Error("the integration level did not start while the unit level ran")
}
a := out.(delivery.CheckAnswer)
if len(a.Levels) != 2 || a.Levels[0].Level != delivery.LevelUnit || a.Levels[1].Level != delivery.LevelIntegration {
t.Fatalf("levels %+v, want unit then integration", a.Levels)
}
var first struct{ Passed bool }
if err := json.Unmarshal(a.Levels[0].Tests[0], &first); err != nil || first.Passed {
t.Errorf("the first level's result %s, want the unit level's failure", a.Levels[0].Tests[0])
}
}
+8 -58
View File
@@ -6,7 +6,6 @@ import (
"errors"
"fmt"
"strings"
"sync"
"testing"
"time"
@@ -17,7 +16,6 @@ const runCommit = "0123456789abcdef0123456789abcdef01234567"
// recorder is a build module's step tools that record each call.
type recorder struct {
mu sync.Mutex // a check's levels run side by side (issue 1921, S10)
calls []string
fetch map[string]any
tests map[string]bool // package or module directory -> passed
@@ -28,9 +26,7 @@ func (r *recorder) steps() AskSteps {
call := func(name string, answer func(args map[string]any) any) StepFunc {
return func(ctx context.Context, args map[string]any) (any, error) {
b, _ := json.Marshal(args)
r.mu.Lock()
r.calls = append(r.calls, name+" "+string(b))
r.mu.Unlock()
return answer(args), nil
}
}
@@ -113,9 +109,9 @@ func TestRunAskRefusesAnotherToolchain(t *testing.T) {
}
}
// 1887/A3, A5a: a check ask runs the levels of one check after one fetch: it digests the unit level, takes the kept
// verdicts, tests the packages whose tests changed first, then the rest also after a failure (issue 1921); the levels run
// side by side and are answered in order, the first that did not pass first (ADR 0484 clause 13; issue 1921 S10).
// 1887/A3, A5a, issue 1887 design limit 1: a check ask runs the levels of one check after one fetch: it digests the unit
// level, takes the kept verdicts, tests the packages whose tests changed first, and stops at the first failure, running
// no later level.
func TestRunAskChecksTheLevelsOfOneCheck(t *testing.T) {
r := &recorder{
fetch: map[string]any{"workspace": "/w", "changed": []string{"pkg/b/b_test.go"}},
@@ -127,26 +123,19 @@ func TestRunAskChecksTheLevelsOfOneCheck(t *testing.T) {
if err != nil {
t.Fatal(err)
}
var unit []string
for _, c := range r.calls {
if !strings.Contains(c, `"level":"integration"`) {
unit = append(unit, c)
}
}
got := strings.Join(unit, "\n")
got := strings.Join(r.calls, "\n")
want := `fetch {"base":"` + runCommit + `","commit":"` + runCommit + `","repository":"novox/x"}
digest {"level":"unit","workspace":"/w"}
test {"level":"unit","packages":["x/pkg/b"],"workspace":"/w"}
test {"level":"unit","packages":["x/pkg/a"],"workspace":"/w"}`
test {"level":"unit","packages":["x/pkg/b"],"workspace":"/w"}`
if got != want {
t.Fatalf("calls:\n%s\nwant:\n%s", got, want)
}
a := out.(delivery.CheckAnswer)
if len(a.Levels) != 2 || a.Levels[1].Level != delivery.LevelIntegration {
t.Fatalf("levels %+v, want the unit level first, the integration level after it", a.Levels)
if len(a.Levels) != 1 {
t.Fatalf("levels %+v, want the unit level alone", a.Levels)
}
u := a.Levels[0]
if u.Level != delivery.LevelUnit || len(u.Kept) != 1 || u.Kept[0].Package != "x/pkg/k" || u.Kept[0].Tests != 7 || len(u.Tests) != 2 || u.Digest == nil {
if u.Level != delivery.LevelUnit || len(u.Kept) != 1 || u.Kept[0].Package != "x/pkg/k" || u.Kept[0].Tests != 7 || len(u.Tests) != 1 || u.Digest == nil {
t.Fatalf("the unit level %+v", u)
}
}
@@ -214,42 +203,3 @@ func TestRunAskChecksNodeModulesAndConflicts(t *testing.T) {
t.Fatalf("a conflict: %v %v", err, r.calls)
}
}
// Issue 1921: a failure in the packages whose tests changed does not stop the level: the other packages still run, so two
// failing packages, one changed and one not, are two test results of the one level, and the level does not pass. A Node
// check runs every module directory after one fails too.
func TestRunAskRunsEveryPackageAfterAFailure(t *testing.T) {
r := &recorder{
fetch: map[string]any{"workspace": "/w", "changed": []string{"pkg/b/b_test.go"}},
digests: map[string]string{"x/pkg/a": "d-a", "x/pkg/b": "d-b"},
tests: map[string]bool{"x/pkg/a": false, "x/pkg/b": false},
}
out, err := RunAsk(context.Background(), r.steps(), ask(delivery.CheckJob{Levels: []string{delivery.LevelIntegration, delivery.LevelUnit},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
a := out.(delivery.CheckAnswer)
if len(a.Levels) != 2 || len(a.Levels[0].Tests) != 2 || a.Levels[0].Level != delivery.LevelIntegration {
t.Fatalf("levels %+v, want the first level with both packages' results, in the order asked", a.Levels)
}
for i, raw := range a.Levels[0].Tests {
var res struct{ Passed bool }
if err := json.Unmarshal(raw, &res); err != nil || res.Passed {
t.Errorf("result %d %s, want a failure", i, raw)
}
}
r = &recorder{fetch: map[string]any{"workspace": "/w", "modules": []string{"a", "b"}, "changed": []string{"b/test/x.test.ts"}},
tests: map[string]bool{"a": true, "b": false}}
s := r.steps()
s.Digest = nil
out, err = RunAsk(context.Background(), s, ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit, delivery.LevelIntegration},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
if a := out.(delivery.CheckAnswer); len(r.calls) != 5 || len(a.Levels) != 2 || len(a.Levels[0].Tests) != 2 || a.Levels[0].Level != delivery.LevelUnit {
t.Fatalf("calls %v, levels %+v: want both directories of each level, the unit level first", r.calls, a.Levels)
}
}
+12 -3
View File
@@ -42,11 +42,20 @@ func NodeCapacity(path, fallback, node string, logf func(string, ...any)) func()
}
setting, _ = s[CapacitySetting].(string)
}
caps, err := buildrun.ParseCapacity(setting)
n, err := capacityOf(setting, node)
if err != nil {
say("%v, so %s pulls nothing", fmt.Errorf("the build capacity: %w", err), node)
say("%v, so %s pulls nothing", err, node)
return 0
}
return (&buildrun.Places{Capacity: caps}).CapacityOf(node)
return n
}
}
// capacityOf is node's capacity in a setting's `<node>=<asks>` pairs: DefaultCapacity for a node it does not name.
func capacityOf(setting, node string) (int, error) {
caps, err := buildrun.ParseCapacity(setting)
if err != nil {
return 0, fmt.Errorf("the build capacity: %w", err)
}
return (&buildrun.Places{Capacity: caps}).CapacityOf(node), nil
}
+10 -114
View File
@@ -3,13 +3,10 @@ package buildstep
import (
"context"
"fmt"
"io/fs"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
@@ -32,25 +29,16 @@ const Bubblewrap = "bwrap"
// - the Workspace and Writable directories read-write, the ReadOnly ones (the toolchain, the modules the fetch step
// downloaded) read-only;
// - the shared warm Caches read-only under a throwaway writable layer (S18), or read-write for a trunk build alone
// (WriteCaches), so a candidate's code never changes what a later build reads;
// - with a CacheLayer, what one command writes into the caches kept for the later commands of the same ask (issue 1921,
// the try speed-up): the vet, the unit tests and the integration compile of one ask compile each package once.
// (WriteCaches), so a candidate's code never changes what a later build reads.
type Isolation struct {
Workspace string
Writable []string
ReadOnly []string
Caches []string
WriteCaches bool
// CacheLayer is a directory of the ask's own, in its workspace and gone with it: each command writes the caches into
// a layer of its own beneath it (overlayfs refuses an upper layer two mounts share, and an ask's commands run side by
// side), which Isolated moves into the ask's kept layer when the command ends; every later command reads the kept
// layer above the shared cache. The shared cache stays unchanged. Empty: what a command writes is thrown away.
CacheLayer string
Env []string
Env []string
// Hosts is the file bound as /etc/hosts; Isolated writes one naming only the loopback.
Hosts string
// run is the directory of this command's own writable layers under CacheLayer; Isolated makes one per command.
run string
}
// etcNeeded is what of /etc a program in the step may read: who the users are, where the libraries are, the
@@ -103,7 +91,7 @@ func (iso Isolation) command(env []string, name string, args ...string) ([]strin
for _, group := range []struct {
what string
dirs []string
}{{"writable directory", iso.Writable}, {"read-only directory", iso.ReadOnly}, {"cache", iso.Caches}, {"cache layer", layerDirs(iso.CacheLayer)}} {
}{{"writable directory", iso.Writable}, {"read-only directory", iso.ReadOnly}, {"cache", iso.Caches}} {
for _, d := range group.dirs {
if err := checkDir(group.what, d); err != nil {
return nil, err
@@ -125,7 +113,7 @@ func (iso Isolation) command(env []string, name string, args ...string) ([]strin
for _, d := range iso.ReadOnly {
argv = append(argv, "--ro-bind", d, d)
}
argv = append(argv, iso.cacheArgs()...)
argv = append(argv, cacheArgs(iso.Caches, iso.WriteCaches)...)
argv = append(argv, "--bind", iso.Workspace, iso.Workspace)
for _, d := range iso.Writable {
argv = append(argv, "--bind", d, d)
@@ -146,107 +134,19 @@ func (iso Isolation) command(env []string, name string, args ...string) ([]strin
// cacheArgs binds the shared warm caches (S18): for a trunk build read-write, so the next build is warm; for any other
// read-only beneath an overlay whose writes go to a tmpfs of the step's own, so the step writes as a build does and
// nothing it wrote outlives it (bubblewrap's --tmp-overlay, an overlay mount in the step's user namespace). With a
// CacheLayer the ask's kept layer lies above the shared cache and the command's writes go to its own layer on disk
// (--overlay), which Isolated keeps for the ask when the command ends.
func (iso Isolation) cacheArgs() []string {
// nothing it wrote outlives it (bubblewrap's --tmp-overlay, an overlay mount in the step's user namespace).
func cacheArgs(caches []string, write bool) []string {
var argv []string
for i, c := range iso.Caches {
switch {
case iso.WriteCaches:
for _, c := range caches {
if write {
argv = append(argv, "--bind", c, c)
case iso.CacheLayer != "":
upper, work := iso.runLayer(i)
argv = append(argv, "--overlay-src", c, "--overlay-src", keptLayer(iso.CacheLayer, i), "--overlay", upper, work, c)
default:
argv = append(argv, "--overlay-src", c, "--tmp-overlay", c)
continue
}
argv = append(argv, "--overlay-src", c, "--tmp-overlay", c)
}
return argv
}
func layerDirs(layer string) []string {
if layer == "" {
return nil
}
return []string{layer}
}
// keptLayer is where the ask keeps what its commands wrote into the cache i.
func keptLayer(layer string, i int) string { return filepath.Join(layer, "kept", strconv.Itoa(i)) }
// runLayer is the upper and work directory of this command's writes into the cache i. Command, which makes no
// directory, names the layer "run"; Isolated gives each command one of its own.
func (iso Isolation) runLayer(i int) (upper, work string) {
run := iso.run
if run == "" {
run = filepath.Join(iso.CacheLayer, "run")
}
return filepath.Join(run, strconv.Itoa(i), "upper"), filepath.Join(run, strconv.Itoa(i), "work")
}
// layerLocks hold one mutex per cache layer, so two commands ending at once keep their writes one after the other.
var layerLocks sync.Map
// prepareLayer makes the kept layers and a run directory of the command's own under the cache layer.
func (iso *Isolation) prepareLayer() error {
if iso.CacheLayer == "" || iso.WriteCaches || len(iso.Caches) == 0 {
return nil
}
if err := os.MkdirAll(filepath.Join(iso.CacheLayer, "runs"), 0o700); err != nil {
return err
}
run, err := os.MkdirTemp(filepath.Join(iso.CacheLayer, "runs"), "run-")
if err != nil {
return err
}
iso.run = run
for i := range iso.Caches {
upper, work := iso.runLayer(i)
for _, d := range []string{keptLayer(iso.CacheLayer, i), upper, work} {
if err := os.MkdirAll(d, 0o700); err != nil {
return err
}
}
}
return nil
}
// keepLayer moves what the command wrote into the caches into the ask's kept layers, and removes its run directory.
// A file the command wrote replaces the kept one (a cache's newest write wins); a whiteout (a file the command
// deleted) is not kept, since a cache entry deleted by one command is no fault of a later one. A write that cannot be
// kept only means a later command builds it again.
func (iso Isolation) keepLayer() {
if iso.run == "" {
return
}
defer os.RemoveAll(iso.run)
mu, _ := layerLocks.LoadOrStore(iso.CacheLayer, &sync.Mutex{})
mu.(*sync.Mutex).Lock()
defer mu.(*sync.Mutex).Unlock()
for i := range iso.Caches {
upper, _ := iso.runLayer(i)
kept := keptLayer(iso.CacheLayer, i)
filepath.WalkDir(upper, func(p string, d fs.DirEntry, err error) error {
if err != nil || p == upper {
return nil
}
rel, _ := filepath.Rel(upper, p)
to := filepath.Join(kept, rel)
switch {
case d.IsDir():
if fi, err := os.Lstat(to); err == nil && !fi.IsDir() {
os.Remove(to)
}
os.MkdirAll(to, 0o700)
case d.Type().IsRegular():
os.Rename(p, to)
}
return nil
})
}
}
// loopbackHosts is the hosts file of a step: the loopback and nothing else.
const loopbackHosts = "127.0.0.1 localhost\n::1 localhost\n"
@@ -266,10 +166,6 @@ func Isolated(run forgekey.Runner, iso Isolation) forgekey.Runner {
}
iso := iso
iso.Hosts = hosts
if err := iso.prepareLayer(); err != nil {
return forgekey.Ran{Status: 1, Err: "the step's cache layer: " + err.Error()}
}
defer iso.keepLayer()
argv, err := iso.command(env, name, args...)
if err != nil {
return forgekey.Ran{Status: 1, Err: err.Error()}
-36
View File
@@ -64,39 +64,3 @@ func TestARunStepReachesNothingAndLeavesTheCachesUnchanged(t *testing.T) {
t.Fatalf("the step's write to its workspace did not stay (status %d, %s%s): %q %v", r.Status, r.Stderr, r.Err, b, err)
}
}
// 1921/S2: in one ask (one cache layer), a second go build of the same package does not compile it again, and the
// shared cache is unchanged afterwards.
func TestASecondBuildOfTheAskFindsTheFirstsCompiledAndTheSharedCacheIsUnchanged(t *testing.T) {
if _, err := exec.LookPath(Bubblewrap); err != nil {
t.Fatalf("bubblewrap is not installed here (issue 1309): %v", err)
}
goBin, err := exec.LookPath("go")
if err != nil {
t.Fatal(err)
}
root := t.TempDir()
shared := filepath.Join(root, "gobuild")
if err := os.MkdirAll(shared, 0o700); err != nil {
t.Fatal(err)
}
workspace := filepath.Join(root, "ws")
tree := filepath.Join(workspace, "tree")
writeFile(t, tree, "go.mod", "module probe\n\ngo 1.22\n")
writeFile(t, tree, "probe.go", "package probe\n\nfunc Probe() int { return 42 }\n")
run := Isolated(forgekey.ExecRunnerFor(0), Isolation{Workspace: workspace, Caches: []string{shared},
CacheLayer: filepath.Join(workspace, "cachelayer"), ReadOnly: []string{filepath.Dir(filepath.Dir(goBin))}})
env := []string{"PATH=" + filepath.Dir(goBin) + ":/usr/bin:/bin", "GOCACHE=" + shared, "GOTOOLCHAIN=local", "GOFLAGS=-mod=mod"}
for i, want := range []bool{true, false} {
r := run(context.Background(), env, "go", "-C", tree, "build", "-x", "./...")
if r.Status != 0 || r.Err != "" {
t.Fatalf("build %d: status %d %s\n%s", i+1, r.Status, r.Err, r.Stderr)
}
if compiled := strings.Contains(r.Stderr, "/compile "); compiled != want {
t.Errorf("build %d compiled the package: %v, want %v\n%s", i+1, compiled, want, r.Stderr)
}
}
if entries, _ := os.ReadDir(shared); len(entries) != 0 {
t.Errorf("the shared cache was written: %d entries", len(entries))
}
}
-110
View File
@@ -1,110 +0,0 @@
package buildstep
import (
"context"
"os"
"path/filepath"
"sync"
"testing"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// overlayOf answers the upper directory a command line gives the cache c, "" when it gives none.
func overlayOf(argv []string, c string) string {
for i := 0; i+3 < len(argv); i++ {
if argv[i] == "--overlay" && argv[i+3] == c {
return argv[i+1]
}
}
return ""
}
// 1921/S2: with a cache layer, a command reads the ask's kept layer above the shared cache and writes into a layer on
// disk of its own, never into the shared cache and never into a throwaway tmpfs.
func TestACommandWithACacheLayerReadsTheKeptLayerAndWritesItsOwn(t *testing.T) {
fixedSystem(t)
iso := Isolation{Workspace: "/w", Caches: []string{"/cache/gomod", "/cache/gobuild"}, CacheLayer: "/w/cachelayer"}
argv, err := iso.Command("go", "vet")
if err != nil {
t.Fatal(err)
}
for i, c := range iso.Caches {
if !holds(argv, "--overlay-src", c, "--overlay-src", keptLayer(iso.CacheLayer, i), "--overlay") {
t.Errorf("the cache %s is not read beneath the ask's kept layer: %v", c, argv)
}
if up := overlayOf(argv, c); up == "" || filepath.Dir(filepath.Dir(filepath.Dir(up))) != "/w/cachelayer" {
t.Errorf("the cache %s is written into %q, not a layer of the ask's own: %v", c, up, argv)
}
if holds(argv, "--tmp-overlay", c) || holds(argv, "--bind", c, c) {
t.Errorf("the cache %s is thrown away or written: %v", c, argv)
}
}
if _, err := (Isolation{Workspace: "/w", CacheLayer: "cachelayer"}).Command("go"); err == nil {
t.Error("a relative cache layer was taken")
}
}
// 1921/S2: what one command wrote into the caches is kept for the ask's later commands, the shared cache stays
// unchanged, two commands at once write into layers of their own (overlayfs refuses an upper layer two mounts share),
// and a command's own layer is gone when it ends.
func TestWhatOneCommandCompiledIsReadByTheNextOfTheSameAsk(t *testing.T) {
fixedSystem(t)
root := t.TempDir()
shared := filepath.Join(root, "gobuild")
if err := os.MkdirAll(shared, 0o700); err != nil {
t.Fatal(err)
}
layer := filepath.Join(root, "ws", "cachelayer")
var mu sync.Mutex
uppers := map[string]bool{}
gate := make(chan struct{})
var started sync.WaitGroup
started.Add(2)
inner := func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
up := overlayOf(args, shared)
mu.Lock()
uppers[up] = true
mu.Unlock()
if holds(args, "--", "go", "build") {
started.Done()
<-gate // both commands are running before either ends
os.MkdirAll(filepath.Join(up, "ab"), 0o700)
os.WriteFile(filepath.Join(up, "ab", args[len(args)-1]+"-d"), []byte("compiled"), 0o600)
}
return forgekey.Ran{}
}
run := Isolated(inner, Isolation{Workspace: filepath.Join(root, "ws"), Caches: []string{shared}, CacheLayer: layer})
var wg sync.WaitGroup
for _, pkg := range []string{"one", "two"} {
wg.Add(1)
go func() { defer wg.Done(); run(context.Background(), nil, "go", "build", pkg) }()
}
started.Wait()
close(gate)
wg.Wait()
if len(uppers) != 2 {
t.Fatalf("two commands at once wrote into %d layers, want 2 of their own: %v", len(uppers), uppers)
}
for _, pkg := range []string{"one", "two"} {
if b, err := os.ReadFile(filepath.Join(keptLayer(layer, 0), "ab", pkg+"-d")); err != nil || string(b) != "compiled" {
t.Errorf("what the build of %s compiled is not kept for the ask: %q %v", pkg, b, err)
}
}
for up := range uppers {
if _, err := os.Stat(up); !os.IsNotExist(err) {
t.Errorf("the command's own layer %s outlived it: %v", up, err)
}
}
var next []string
Isolated(func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
next = args
return forgekey.Ran{}
}, Isolation{Workspace: filepath.Join(root, "ws"), Caches: []string{shared}, CacheLayer: layer})(context.Background(), nil, "go", "test")
if !holds(next, "--overlay-src", shared, "--overlay-src", keptLayer(layer, 0)) {
t.Errorf("the next command does not read the kept layer: %v", next)
}
if entries, _ := os.ReadDir(shared); len(entries) != 0 {
t.Errorf("the shared cache was written: %v", entries)
}
}
+6 -15
View File
@@ -24,8 +24,7 @@ type MergeResult struct {
// signature, no hook, nothing of the machine's git configuration that changes a merge. Nothing is pushed.
//
// A merge needs both histories, so it makes the fetch whole (the shallow clone FetchCommit made is deepened) and
// fetches base by its sha: from the mirror WithMirror names, when it holds base (UpdateMirror to base first), else
// from remote. A head that does not merge cleanly is an answer, not an error: the files in conflict, the
// fetches base by its sha. A head that does not merge cleanly is an answer, not an error: the files in conflict, the
// merge undone and the workspace left at base. A head already in base answers a Merge equal to Base and no changed
// paths. An error is that the merge could not be tried: a base or commit that is not a full sha, no base, a workspace
// not at commit, or a fetch or git command that failed. remote and the options are those of FetchCommit; the runner's
@@ -68,20 +67,12 @@ func MergeInto(ctx context.Context, run forgekey.Runner, dir, remote, base, comm
if err != nil {
return res, err
}
fetchFrom := func(source string) error {
fetch := []string{"fetch", "--no-tags", source, base}
if strings.TrimSpace(shallow) == "true" {
fetch = []string{"fetch", "--no-tags", "--unshallow", source, base}
}
_, err := git(fetch...)
return err
fetch := []string{"fetch", "--no-tags", "origin", base}
if strings.TrimSpace(shallow) == "true" {
fetch = []string{"fetch", "--no-tags", "--unshallow", "origin", base}
}
// With a mirror brought to the trunk commit, the history is deepened from it, a local fetch (issue 1921, try speed
// S6); a mirror that cannot answer leaves it to the forge, as before.
if o.mirror == "" || fetchFrom(o.mirror) != nil {
if err := fetchFrom("origin"); err != nil {
return res, fmt.Errorf("the trunk commit %s: %w", base, err)
}
if _, err := git(fetch...); err != nil {
return res, fmt.Errorf("the trunk commit %s: %w", base, err)
}
if _, err := git("checkout", "-q", "--detach", base); err != nil {
return res, fmt.Errorf("the trunk commit %s cannot be checked out: %w", base, err)
-30
View File
@@ -170,33 +170,3 @@ func TestMergeIntoAHeadAlreadyInTheTrunkChangesNothing(t *testing.T) {
t.Errorf("result = %+v, %v", res, err)
}
}
// 1921/S6: with a mirror brought to the head and to the trunk commit, the merge asks the forge for nothing: it succeeds
// with the forge gone.
func TestMergeIntoWithAMirrorAtTheTrunkNeedsNoForge(t *testing.T) {
r := newMergeRig(t)
head := r.head(map[string]string{"b.txt": "head\n"})
base := r.advance(map[string]string{"a.txt": "one\ntwo\nthree\nfour\n"})
mirror := filepath.Join(t.TempDir(), "mirror.git")
if err := os.MkdirAll(mirror, 0o755); err != nil {
t.Fatal(err)
}
for _, c := range []string{head, base} {
if err := UpdateMirror(context.Background(), forgekey.ExecRunner, mirror, r.remote, c, "", AllowFileRemote()); err != nil {
t.Fatal(err)
}
}
if err := FetchCommit(context.Background(), forgekey.ExecRunner, r.ws, r.remote, head, "", AllowFileRemote(), WithMirror(mirror)); err != nil {
t.Fatal(err)
}
if err := os.RemoveAll(r.bare); err != nil { // the forge is gone
t.Fatal(err)
}
res, err := MergeInto(context.Background(), forgekey.ExecRunner, r.ws, r.remote, base, head, AllowFileRemote(), WithMirror(mirror))
if err != nil {
t.Fatalf("the merge asked the forge: %v", err)
}
if len(res.Merge) != 40 || !reflect.DeepEqual(res.Changed, []string{"b.txt"}) {
t.Errorf("result = %+v", res)
}
}
-36
View File
@@ -72,39 +72,3 @@ func TestMergeIntoRunsTheCheckMergesCommands(t *testing.T) {
}
}
}
// 1921/S6: with a mirror that holds the trunk commit, the merge deepens the workspace from the mirror, a local fetch,
// and asks the forge nothing; a mirror that cannot answer falls back to the forge.
func TestMergeIntoWithAMirrorFetchesTheTrunkFromTheMirror(t *testing.T) {
base := strings.Repeat("b", 40)
mirror := t.TempDir()
answer := func(failMirror bool) *fakeGit {
return &fakeGit{rule: func(l string) forgekey.Ran {
switch {
case strings.HasSuffix(l, "rev-parse HEAD"):
return forgekey.Ran{Stdout: sha + "\n"}
case strings.Contains(l, "--is-shallow-repository"):
return forgekey.Ran{Stdout: "true\n"}
case failMirror && strings.Contains(l, " fetch ") && strings.Contains(l, mirror):
return forgekey.Ran{Status: 128, Stderr: "fatal: not our ref"}
}
return forgekey.Ran{}
}}
}
f := answer(false)
dir := t.TempDir()
if _, err := MergeInto(context.Background(), f.run, dir, "ssh://mesh-forge/o/r.git", base, sha, WithMirror(mirror)); err != nil {
t.Fatal(err)
}
all := strings.Join(f.calls, "\n")
if !strings.Contains(all, "fetch --no-tags --unshallow "+mirror+" "+base) || strings.Contains(all, "--unshallow origin") {
t.Errorf("the trunk was not fetched from the mirror alone:\n%s", all)
}
g := answer(true)
if _, err := MergeInto(context.Background(), g.run, t.TempDir(), "ssh://mesh-forge/o/r.git", base, sha, WithMirror(mirror)); err != nil {
t.Fatal(err)
}
if all := strings.Join(g.calls, "\n"); !strings.Contains(all, "fetch --no-tags --unshallow origin "+base) {
t.Errorf("a mirror that could not answer did not fall back to the forge:\n%s", all)
}
}
+104 -9
View File
@@ -9,6 +9,7 @@ import (
"sync"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
@@ -64,10 +65,17 @@ type Puller struct {
PullWait time.Duration
Heartbeat time.Duration
Logf func(format string, args ...any)
// Load reads one key of the module's steps state (nil and no error when it is absent). When set, Pull reads the
// node's pause key (delivery.PauseKey) before it pulls, and Pause and Resume write it with Put, so a pause outlives
// a restart.
// Load reads one key of the state the node's pause is kept in (nil and no error when it is absent). When set, Pull
// reads the node's pause key (delivery.PauseKey) before it pulls, and Pause and Resume write it with PausePut, so a
// pause outlives a restart.
Load func(key string) ([]byte, error)
// PausePut writes the node's pause key: the module's pauses state (delivery.PausesState), which has no age, so a
// pause never ends by itself while the steps state drops its old keys. Put when nil.
PausePut Put
// WatchConfig, when set, follows the module's config value `capacity` (CapacitySetting) as mesh-config resolves it
// for this node, and wakes the puller on each change (watchCapacity): WatchConfigServed in a module, a test's own
// double. Until it hands a value the puller judges by Capacity.
WatchConfig func(key string, handle func(stdio.ProvisionChange) error) error
mu sync.Mutex
inHand int
@@ -75,6 +83,9 @@ type Puller struct {
paused bool
wake chan struct{}
once sync.Once
// watched is the capacity the config watch handed last, used in place of Capacity once hasWatched.
watched int
hasWatched bool
}
func (p *Puller) init() {
@@ -113,7 +124,11 @@ func (p *Puller) setPaused(paused bool, by string) error {
if err != nil {
return err
}
if _, err := p.Put(key, json.RawMessage(raw)); err != nil {
put := p.PausePut
if put == nil {
put = p.Put
}
if _, err := put(key, json.RawMessage(raw)); err != nil {
return fmt.Errorf("buildstep: writing the pause of %s: %w", p.Node, err)
}
}
@@ -186,13 +201,29 @@ func (p *Puller) Running() []RunningAsk {
// room says whether the puller may take one more ask now.
func (p *Puller) room() bool {
capacity := 0
if p.Capacity != nil {
ok, _, _ := p.roomNow()
return ok
}
// roomNow is room, with the capacity and the asks in hand it was judged on.
func (p *Puller) roomNow() (ok bool, capacity, inHand int) {
p.mu.Lock()
watched, has := p.watched, p.hasWatched
p.mu.Unlock()
if has {
capacity = watched
} else if p.Capacity != nil {
capacity = p.Capacity()
}
p.mu.Lock()
defer p.mu.Unlock()
return !p.paused && p.inHand < capacity
return !p.paused && p.inHand < capacity, capacity, p.inHand
}
// CapacityNow is the capacity the puller judges by: the config watch's last value, else Capacity.
func (p *Puller) CapacityNow() int {
_, capacity, _ := p.roomNow()
return capacity
}
// Pull takes and runs asks until ctx ends. An ask still running then is neither ended nor acknowledged: the stream
@@ -203,8 +234,11 @@ func (p *Puller) Pull(ctx context.Context) error {
return errors.New("buildstep: a puller needs its source, its steps' put and its run")
}
if err := p.restore(); err != nil {
return err // pulling on a pause that cannot be read could build on a node paused on purpose
// An unreadable pause degrades to not paused, said once: a puller that stops silently is the worse failure, and a
// node kept from building is held by its capacity (0), not by a pause.
p.logf("the pause of %s cannot be read: %v; pulling as not paused", p.Node, err)
}
go p.watchCapacity() // WatchConfigServed waits for Serve, which a module starts after its pull
wait := p.PullWait
if wait <= 0 {
wait = DefaultPullWait
@@ -213,12 +247,17 @@ func (p *Puller) Pull(ctx context.Context) error {
if err := ctx.Err(); err != nil {
return err
}
if !p.room() {
if ok, capacity, inHand := p.roomNow(); !ok {
// Parked until something wakes it: a capacity change (the config watch), a resume, or a step's end. Every
// park logs once, so a puller that sits idle says why (issue 1887).
p.logf("the %s puller on %s parks: %d of capacity %d in hand, paused %v", p.Language, p.Node, inHand, capacity, p.Paused())
select {
case <-ctx.Done():
return ctx.Err()
case <-p.wake:
}
_, capacity, inHand = p.roomNow()
p.logf("the %s puller on %s wakes: %d of capacity %d in hand, paused %v", p.Language, p.Node, inHand, capacity, p.Paused())
continue
}
msg, err := p.next(ctx, wait)
@@ -348,3 +387,59 @@ func (p *Puller) run(ctx context.Context, msg *Pulled, ask delivery.StepAsk, key
p.logf("the ask %s ended on %s and its acknowledgement failed: %v", ask.ID, p.Node, aerr)
}
}
// watchCapacity follows the module's config value `capacity` when WatchConfig is set (issue 1887, 2026-10-25: g14's
// puller started at capacity 0 sat parked with capacity 2 set, because nothing woke it to read its settings file
// again). The value comes as mesh-config resolved it for this node, the event of the change itself: the runtime hands
// the value now and then each change its machine's provisioner client says moved, so nothing polls, and the puller does
// not wait for the node-engine to fill its settings file again. Each value handed is kept, used in place of Capacity,
// and wakes the puller. A watch that cannot start leaves the puller on Capacity, said once.
func (p *Puller) watchCapacity() {
if p.WatchConfig == nil {
return
}
err := p.WatchConfig(CapacitySetting, func(c stdio.ProvisionChange) error {
n, why := p.capacityFrom(c)
p.mu.Lock()
before, had := p.watched, p.hasWatched
p.watched, p.hasWatched = n, true
p.mu.Unlock()
switch {
case why != "":
p.logf("the %s puller on %s pulls nothing: %s", p.Language, p.Node, why)
case !had || before != n:
p.logf("the %s build capacity of %s is %d, as mesh-config resolved it", p.Language, p.Node, n)
}
p.Wake()
return nil
})
if err != nil {
p.logf("the %s build capacity of %s cannot be followed on the bus, so it is read from the settings file when the "+
"puller wakes: %v", p.Language, p.Node, err)
}
}
// WatchConfigServed is stdio.WatchConfig once Serve has the runtime's channel (stdio.Served): a build module starts its
// pull before it serves, and a watch asked before that would be refused.
func WatchConfigServed(key string, handle func(stdio.ProvisionChange) error) error {
<-stdio.Served()
return stdio.WatchConfig(key, handle)
}
// capacityFrom is the node's capacity in one value the watch handed, and why it is 0 when it cannot be read.
func (p *Puller) capacityFrom(c stdio.ProvisionChange) (int, string) {
if c.Refused != "" {
return 0, "its capacity is no longer answered: " + c.Refused
}
setting := ""
if len(c.Value) > 0 && string(c.Value) != "null" {
if err := json.Unmarshal(c.Value, &setting); err != nil {
return 0, fmt.Sprintf("its capacity %s is not a string", c.Value)
}
}
n, err := capacityOf(setting, p.Node)
if err != nil {
return 0, err.Error()
}
return n, ""
}
+39
View File
@@ -0,0 +1,39 @@
package buildstep
import (
"encoding/json"
"sync"
"testing"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// configWatch is a double of stdio.WatchConfig: it hands the value now at the start, as the runtime does, and a test
// hands each change after with set.
type configWatch struct {
mu sync.Mutex
value string
handle func(stdio.ProvisionChange) error
asked string
}
func (w *configWatch) watch(key string, handle func(stdio.ProvisionChange) error) error {
w.mu.Lock()
w.asked, w.handle = key, handle
value := w.value
w.mu.Unlock()
return handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`"` + value + `"`)})
}
func (w *configWatch) set(t *testing.T, value string) {
t.Helper()
w.mu.Lock()
handle, key := w.handle, w.asked
w.mu.Unlock()
if handle == nil {
t.Fatal("the puller watches no config value")
}
if err := handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`"` + value + `"`)}); err != nil {
t.Fatal(err)
}
}
+101 -24
View File
@@ -3,11 +3,15 @@ package buildstep
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
@@ -175,36 +179,85 @@ func (k *kv) load(key string) ([]byte, error) {
return k.vals[key], nil
}
// 1887/A2b, 1887/AC-zero: at capacity 0 the puller takes nothing; a capacity raised takes effect at the next pull.
// 1887/A2b, 1887/AC-zero, 1887/AC-wake: at capacity 0 the puller takes nothing; a capacity raised takes effect at the
// next pull, or, with the config watch (g14 idle at capacity 2, 2026-10-25), at once when the value moves, with no
// pause, resume or step end to wake it; a refused value pulls nothing, and a watch that cannot start leaves the
// settings file's capacity.
func TestPullAtCapacityZeroTakesNothing(t *testing.T) {
src, st := NewFakeAsks(), &kv{}
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
var mu sync.Mutex
capacity := 0
p := &Puller{Source: src, Language: delivery.AskGo, Node: "novox", Put: st.put, PullWait: 10 * time.Millisecond,
Capacity: func() int { mu.Lock(); defer mu.Unlock(); return capacity },
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { return nil, nil }}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
time.Sleep(50 * time.Millisecond)
if src.Waiting(delivery.AskDelivery) != 1 {
t.Fatal("a node at capacity 0 took an ask")
t.Run("settings file", func(t *testing.T) {
src, st := NewFakeAsks(), &kv{}
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
var mu sync.Mutex
capacity := 0
p := &Puller{Source: src, Language: delivery.AskGo, Node: "novox", Put: st.put, PullWait: 10 * time.Millisecond,
Capacity: func() int { mu.Lock(); defer mu.Unlock(); return capacity },
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { return nil, nil }}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
time.Sleep(50 * time.Millisecond)
if src.Waiting(delivery.AskDelivery) != 1 {
t.Fatal("a node at capacity 0 took an ask")
}
mu.Lock()
capacity = 1
mu.Unlock()
p.Wake()
waitFor(t, "the ask taken at capacity 1", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
})
t.Run("config watch wakes", func(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
w := &configWatch{value: "shanks=3,g14=0"}
p := &Puller{Source: src, Language: delivery.AskGo, Node: "g14", Put: st.put, PullWait: 10 * time.Millisecond,
Capacity: func() int { return 0 }, // the settings file, not refilled yet
WatchConfig: w.watch,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { return nil, nil }}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "the watch of the capacity", func() bool { w.mu.Lock(); defer w.mu.Unlock(); return w.handle != nil })
time.Sleep(30 * time.Millisecond)
if src.Waiting(delivery.AskDelivery) != 1 {
t.Fatal("a node at capacity 0 took an ask")
}
if w.asked != CapacitySetting {
t.Fatalf("the puller watches %q, want %q", w.asked, CapacitySetting)
}
w.set(t, "shanks=3,g14=2")
waitFor(t, "the ask taken once the capacity moved to 2", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
})
for _, c := range []struct {
name string
watch func(string, func(stdio.ProvisionChange) error) error
room bool
}{
{"refused value leaves no room", func(key string, handle func(stdio.ProvisionChange) error) error {
return handle(stdio.ProvisionChange{Kind: "config", Key: key, Refused: "the module is no longer on g14"})
}, false},
{"watch that cannot start keeps the settings file", func(string, func(stdio.ProvisionChange) error) error {
return errors.New("no runtime")
}, true},
} {
t.Run(c.name, func(t *testing.T) {
p := &Puller{Source: NewFakeAsks(), Language: delivery.AskGo, Node: "g14", Put: newSteps().put,
Capacity: func() int { return 1 }, WatchConfig: c.watch,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { return nil, nil }}
p.watchCapacity()
if p.room() != c.room {
t.Fatalf("room = %v, want %v", p.room(), c.room)
}
})
}
mu.Lock()
capacity = 1
mu.Unlock()
p.Wake()
waitFor(t, "the ask taken at capacity 1", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
}
// 1887/A2b: a paused puller takes nothing, its pause is kept on the pause key, and a puller started again on that state
// stays paused until resumed.
func TestPullPauseOutlivesARestart(t *testing.T) {
src, st := NewFakeAsks(), &kv{}
src, st, steps := NewFakeAsks(), &kv{}, &kv{}
run := func(ctx context.Context, a delivery.StepAsk) (any, error) { return nil, nil }
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Load: st.load, PullWait: 10 * time.Millisecond,
Capacity: func() int { return 2 }, Run: run}
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: steps.put, PausePut: st.put, Load: st.load,
PullWait: 10 * time.Millisecond, Capacity: func() int { return 2 }, Run: run}
if err := p.Pause("the operator"); err != nil {
t.Fatal(err)
}
@@ -213,9 +266,12 @@ func TestPullPauseOutlivesARestart(t *testing.T) {
if v, err := delivery.DecodePause(raw); err != nil || !v.Paused || v.By != "the operator" {
t.Fatalf("the pause key holds %s (%v)", raw, err)
}
if raw, _ := steps.load(key); raw != nil {
t.Fatalf("the pause was written in the steps state, whose keys age out: %s", raw)
}
// The restart: a new puller on the same state.
again := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Load: st.load, PullWait: 10 * time.Millisecond,
Capacity: func() int { return 2 }, Run: run}
again := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: steps.put, PausePut: st.put, Load: st.load,
PullWait: 10 * time.Millisecond, Capacity: func() int { return 2 }, Run: run}
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
@@ -228,6 +284,27 @@ func TestPullPauseOutlivesARestart(t *testing.T) {
t.Fatal(err)
}
waitFor(t, "the ask taken after the resume", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
t.Run("a pause that cannot be read is said and pulled as not paused", func(t *testing.T) {
src := NewFakeAsks()
var said atomic.Value
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: steps.put, PausePut: st.put,
Load: func(string) ([]byte, error) { return nil, errors.New("no runtime yet") },
PullWait: 10 * time.Millisecond, Capacity: func() int { return 2 }, Run: run,
Logf: func(f string, a ...any) {
if m := fmt.Sprintf(f, a...); strings.Contains(m, "cannot be read") {
said.Store(m)
}
}}
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "the ask taken", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
if m, _ := said.Load().(string); !strings.Contains(m, "pulling as not paused") {
t.Fatalf("the unreadable pause was not said: %q", m)
}
})
}
// 1887/A2c: a long step tells the bus it is still being worked; a holder that stops mid-step neither ends nor
+1 -5
View File
@@ -14,11 +14,7 @@ func SeatTools(seat, what string, p *Puller) []stdio.Tool {
Run: func(map[string]any) (any, error) { return run() }}
}
current := func() (any, error) {
capacity := 0
if p.Capacity != nil {
capacity = p.Capacity()
}
return map[string]any{"node": p.Node, "capacity": capacity, "running": p.Running(), "paused": p.Paused()}, nil
return map[string]any{"node": p.Node, "capacity": p.CapacityNow(), "running": p.Running(), "paused": p.Paused()}, nil
}
return []stdio.Tool{
verb("current", "The "+what+" asks this node runs now (each one's id, kind and when it started), its build capacity, "+
+12 -2
View File
@@ -181,15 +181,25 @@ type Want struct {
Rule int `json:"rule,omitempty"`
Backup string `json:"backup,omitempty"`
Words string `json:"words,omitempty"`
// Put says the want was written by a put (ADR 0546), and Hold that the put holds it: the rules pass over it until it
// is released. A put's want names no newest fingerprint or rule.
Put bool `json:"put,omitempty"`
Hold bool `json:"hold,omitempty"`
}
func (w Want) check() error {
if w.Removed {
if w.FP != "" || w.Composed != "" || w.Newest != "" || w.Rule != 0 || w.Backup != "" {
return fmt.Errorf("delivery: a removed want names no fingerprint, composition, rule or backup answer")
if w.FP != "" || w.Composed != "" || w.Newest != "" || w.Rule != 0 || w.Backup != "" || w.Put || w.Hold {
return fmt.Errorf("delivery: a removed want names no fingerprint, composition, rule, backup answer or put")
}
return nil
}
if w.Hold && !w.Put {
return fmt.Errorf("delivery: only a put holds a want")
}
if w.Put && (w.Newest != "" || w.Rule != 0) {
return fmt.Errorf("delivery: a put's want is held back by no rule")
}
if err := checkFingerprint(w.FP, "a want's fingerprint"); err != nil {
return err
}
+52
View File
@@ -4,6 +4,7 @@ import (
"errors"
"strings"
"testing"
"time"
)
// 1297/B1a: a want key round-trips through its parser; a held key, a have key and a node's own key are refused,
@@ -77,6 +78,7 @@ func TestAssignerValuesRoundTrip(t *testing.T) {
{FP: fp2of('a'), Composed: hashOf('1')},
{FP: fp2of('a'), Composed: hashOf('1'), Newest: fp2of('b'), Rule: RuleCanary, Words: "the canary ace does not run it healthy yet"},
{Removed: true, Words: "the module left g14"},
{FP: fp2of('a'), Composed: hashOf('1'), Put: true, Hold: true, Words: "put by the operator: the bus's user list"},
}
for _, w := range wants {
data, err := EncodeWant(w)
@@ -89,6 +91,24 @@ func TestAssignerValuesRoundTrip(t *testing.T) {
t.Errorf("want %+v came back %+v, %v", w, got, err)
}
}
at := time.Date(2026, 10, 25, 21, 0, 0, 0, time.UTC)
p := Put{FP: fp2of('a'), Hold: true, Why: "the bus's user list", By: "shanks.node-tools", At: at}.Said(at, PutAsked, "put asked")
p = p.Said(at, PutAsked, "put asked").Said(at.Add(time.Second), PutWanted, "want written")
p.Composed = hashOf('1')
if len(p.Steps) != 2 || p.Step != PutWanted || PutEnded(p.Step) || !PutEnded(PutReleased) {
t.Errorf("a put's steps are %+v at %s", p.Steps, p.Step)
}
pkey, _ := PutKey("novox", "nats")
if n, m, err := ParsePutKey(pkey); err != nil || n != "novox" || m != "nats" {
t.Errorf("the put key %s reads %s %s, %v", pkey, n, m, err)
}
pdata, err := EncodePut(p)
if err != nil {
t.Fatal(err)
}
if gotP, err := DecodePut(pdata); err != nil || gotP.FP != p.FP || gotP.Step != PutWanted || len(gotP.Steps) != 2 || !gotP.Hold {
t.Errorf("the put came back %+v, %v", gotP, err)
}
h := Held{Node: "ace", Words: "its health check failed: connection refused"}
data, err := EncodeHeld(h)
if err != nil {
@@ -128,6 +148,9 @@ func TestAssignerValuesRefused(t *testing.T) {
{FP: fp2of('a'), Newest: fp2of('b')},
{FP: fp2of('a'), Newest: fp2of('a'), Rule: 3},
{FP: fp2of('a'), Composed: hashOf('1'), Newest: fp2of('b'), Rule: 8},
{FP: fp2of('a'), Composed: hashOf('1'), Hold: true},
{FP: fp2of('a'), Composed: hashOf('1'), Put: true, Newest: fp2of('b'), Rule: 3},
{Removed: true, Put: true},
} {
if _, err := EncodeWant(w); err == nil {
t.Errorf("EncodeWant(%+v) accepted", w)
@@ -148,6 +171,19 @@ func TestAssignerValuesRefused(t *testing.T) {
t.Errorf("EncodeDifference(%+v) accepted", d)
}
}
for _, p := range []Put{
{FP: fp2of('a'), Why: "w", Step: "walking"},
{FP: fp2of('a'), Step: PutAsked},
{FP: "src2:abc", Why: "w", Step: PutAsked},
{FP: fp2of('a'), Why: "w", Step: PutWanted, Composed: "sha256:x"},
} {
if _, err := EncodePut(p); err == nil {
t.Errorf("EncodePut(%+v) accepted", p)
}
}
if _, _, err := ParsePutKey("ace.node"); err == nil {
t.Errorf("a node's own key was read as a put")
}
if _, err := DecodeWant([]byte(`{"v":2,"fp":"` + fp2of('a') + `"}`)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("a want of version 2 was read: %v", err)
}
@@ -262,4 +298,20 @@ func TestControllerGrant(t *testing.T) {
if _, err := EncodeControllerGrant(ControllerGrant{Accounts: []string{""}}); err == nil {
t.Errorf("a grant naming an empty account was accepted")
}
g := ControllerGrant{Accounts: []string{"ace.mesh-build-go", "shanks.mesh-mail.reader"}}
for _, c := range []struct {
g ControllerGrant
node, module string
names bool
}{
{g, "ace", "mesh-build-go", true},
{g, "shanks", "mesh-mail", true},
{g, "g14", "mesh-build-go", false},
{g, "ace", "mesh-build", false},
{ControllerGrant{}, "novox", "nats", true},
} {
if got := c.g.NamesNode(c.node, c.module); got != c.names {
t.Errorf("%v names %s.%s: %v, want %v", c.g.Accounts, c.node, c.module, got, c.names)
}
}
}
-27
View File
@@ -1,27 +0,0 @@
package delivery
import (
"strings"
"testing"
)
// 1791/1921 S9: a member of a group that merged stays in the candidate, marked merged, with its head, so the heads hash
// the Approve covers holds until every member merged; a candidate whose every member merged is refused.
func TestACandidateKeepsAMergedMemberAndRefusesOneWhoseEveryMemberMerged(t *testing.T) {
a := Head{Repository: "novox/a", Number: 1, Head: strings.Repeat("1", 40)}
b := Head{Repository: "novox/b", Number: 2, Head: strings.Repeat("2", 40)}
hash, _ := HeadsHash([]Head{a, b})
c := Candidate{Group: "feat/x", HeadsHash: hash, Pulls: []CandidatePull{{Head: a, Modules: []string{}, Merged: true}, {Head: b, Modules: []string{}}}}
raw, err := EncodeCandidate(c)
if err != nil {
t.Fatal(err)
}
back, err := DecodeCandidate(raw)
if err != nil || !back.Pulls[0].Merged || back.Pulls[1].Merged || back.HeadsHash != hash {
t.Fatalf("read back %+v, %v", back, err)
}
c.Pulls[1].Merged = true
if _, err := EncodeCandidate(c); err == nil {
t.Error("a candidate whose every member merged was written")
}
}
+6 -21
View File
@@ -139,25 +139,16 @@ type FailedFirst struct {
Words string `json:"words,omitempty"`
}
// MaxFailures is the most entries a verdict's Failures holds (issue 1921): fifty tests named one by one, and an entry
// after them saying how many more failed.
const MaxFailures = 50
// Verdict is the value of a verdict key: the group and its heads hash, the trunk head of each repository it was judged
// against (repository to full sha), the State (running, pass or fail), what failed first, the tests run and kept per
// level, each part's time, the machines composed and the mesh-sdk version the composition ran with (ADR 0446 clause 5).
type Verdict struct {
V int `json:"v"`
Group string `json:"group"`
Heads string `json:"heads"`
Trunk map[string]string `json:"trunk,omitempty"`
State string `json:"state"`
Failed FailedFirst `json:"failed,omitzero"`
// Failures names every test that failed, in the order judged, each with the first line of its words (issue 1921):
// Failed keeps the first one's words whole, and a check that ran every test says all it found here. At most
// MaxFailures entries, the last of a longer list saying how many more failed. Absent in a value written before it,
// and from a verdict whose failure is no test.
Failures []FailedFirst `json:"failures,omitempty"`
V int `json:"v"`
Group string `json:"group"`
Heads string `json:"heads"`
Trunk map[string]string `json:"trunk,omitempty"`
State string `json:"state"`
Failed FailedFirst `json:"failed,omitzero"`
Levels map[string]LevelResult `json:"levels,omitempty"`
Took Took `json:"took,omitzero"`
Machines []string `json:"machines,omitempty"`
@@ -231,12 +222,6 @@ func (v Verdict) check() error {
}
}
named := v.Failed.Test != "" || v.Failed.Machine != ""
if len(v.Failures) > 0 && v.State != VerdictFail {
return fmt.Errorf("delivery: a %s verdict names no failures", v.State)
}
if len(v.Failures) > MaxFailures+1 {
return fmt.Errorf("delivery: the verdict names %d failures, more than %d and the count of the rest", len(v.Failures), MaxFailures)
}
switch v.State {
case VerdictRunning:
if v.Failed != (FailedFirst{}) {
-36
View File
@@ -3,7 +3,6 @@ package delivery
import (
"encoding/json"
"errors"
"strconv"
"strings"
"testing"
"time"
@@ -601,38 +600,3 @@ func TestPreviewOfAHeadInConflict(t *testing.T) {
t.Errorf("the old reader: %+v %v", old, err)
}
}
// Issue 1921: a failed verdict names every failed test in Failures and reads back the same; an older reader's value
// without it reads too; Failures on a verdict that is no fail, and more than MaxFailures and the count of the rest, are
// refused.
func TestVerdictNamesEveryFailure(t *testing.T) {
v := passVerdict()
v.State = VerdictFail
v.Failed = FailedFirst{Test: "integration: x/a.TestA", Words: "a_test.go:9: want 1, got 2\nmore"}
v.Failures = []FailedFirst{{Test: "integration: x/a.TestA", Words: "a_test.go:9: want 1, got 2"}, {Test: "integration: x/b.TestB", Words: "b_test.go:3: boom"}}
data, err := EncodeVerdict(v)
if err != nil {
t.Fatal(err)
}
got, err := DecodeVerdict(data)
v.V = Version
if err != nil || !SameVerdict(got, v) || !strings.Contains(string(data), `"failures":[{"test":"integration: x/a.TestA"`) {
t.Fatalf("%s: %+v %v", data, got, err)
}
old := strings.Replace(string(data), string(data[strings.Index(string(data), `,"failures":`):strings.Index(string(data), `,"levels":`)]), "", 1)
if got, err := DecodeVerdict([]byte(old)); err != nil || len(got.Failures) != 0 || got.Failed != v.Failed {
t.Fatalf("a value without failures: %+v %v (%s)", got, err, old)
}
p := passVerdict()
p.Failures = v.Failures
if _, err := EncodeVerdict(p); err == nil {
t.Error("a pass with failures was written")
}
v.Failures = make([]FailedFirst, MaxFailures+2)
for i := range v.Failures {
v.Failures[i] = FailedFirst{Test: "unit: x.T" + strconv.Itoa(i)}
}
if _, err := EncodeVerdict(v); err == nil {
t.Error("a verdict naming more than MaxFailures and the rest was written")
}
}
+17
View File
@@ -73,3 +73,20 @@ func DecodeControllerGrant(data []byte) (ControllerGrant, error) {
// SameControllerGrant says whether two grants are the same value.
func SameControllerGrant(a, b ControllerGrant) bool { return reflect.DeepEqual(a, b) }
// NamesNode says whether the grant names module's accounts on node: its own `<node>.<module>` or a process's
// `<node>.<module>.<process>`. A grant that names no account (a module that does not speak on the bus) names every
// node. The assigner wants a node only once the grant names its account (issue 1945), and a put waits for it (ADR
// 0546 clause 4).
func (g ControllerGrant) NamesNode(node, module string) bool {
if len(g.Accounts) == 0 {
return true
}
own := node + "." + module
for _, a := range g.Accounts {
if a == own || strings.HasPrefix(a, own+".") {
return true
}
}
return false
}
+148
View File
@@ -0,0 +1,148 @@
package delivery
import (
"encoding/json"
"fmt"
"time"
)
// A put (novox/hq ADR 0546): one version of a module put on one node by the assigner's tool `assigner_put`. Each put is the
// key `<node>.<module>` of the assigner's state puts, which holds the version asked, the hold, why, and each step the
// assigner took or waits on, so a reader watches the key to follow it. The want the put writes carries Put (and Hold),
// so the assigner's rules pass over that key while the put holds it.
// AssignerPutsBucket is the assigner's bucket of puts, `<owner module>_puts`.
const AssignerPutsBucket = AssignerModule + "_puts"
// AssignerPutsHistory is the revisions the puts bucket keeps per key: each step of a put is one.
const AssignerPutsHistory = 20
// A put's steps, in the order they are taken (ADR 0546 clauses 3 to 9). A put stands at the last step written.
const (
// PutAsked is a put written by the tool, not taken up by a pass yet.
PutAsked = "asked"
// PutBuild waits for the builder's build at the fingerprint.
PutBuild = "build"
// PutCompose waits for the controller's composition (and grant) at the fingerprint.
PutCompose = "compose"
// PutPerson waits for the operator's answer to the ask about the move.
PutPerson = "person"
// PutSnapshot waits for the bus's streams to be snapshotted after the operator's yes.
PutSnapshot = "snapshot"
// PutWanted is the want written: the node's have has not reached it yet.
PutWanted = "wanted"
// PutHolding is a held put whose have runs it healthy: the want stays until released.
PutHolding = "holding"
// PutDone is a put without hold whose have ran it healthy: the rules write the want again.
PutDone = "done"
// PutReleased is a put released by the tool `assigner_release`.
PutReleased = "released"
// PutRefused is a put that cannot go on: a build that failed, an answer no, a snapshot not taken. Its words say why.
PutRefused = "refused"
)
// PutEnded says a put at this step is over: nothing more is done for it and the rules own the want.
func PutEnded(step string) bool {
return step == PutDone || step == PutReleased || step == PutRefused
}
// PutStep is one step a put took: when, which, and what was done or waits, in a sentence.
type PutStep struct {
At time.Time `json:"at"`
Step string `json:"step"`
Words string `json:"words"`
}
// PutStepsKept is how many steps a put keeps, newest last.
const PutStepsKept = 16
// Put is the value of a key of AssignerPutsBucket.
type Put struct {
V int `json:"v"`
// FP is the fingerprint put, of the new form.
FP string `json:"fp"`
// Hold keeps the want at FP until released.
Hold bool `json:"hold,omitempty"`
Why string `json:"why"`
// By is who asked, as the bus named the caller; "unproven" when it named none.
By string `json:"by,omitempty"`
At time.Time `json:"at"`
// Step is where the put stands now, one of the Put* steps; Words say it in a sentence.
Step string `json:"step"`
Words string `json:"words,omitempty"`
// Composed is the digest of the composition the want names, once known.
Composed string `json:"composed,omitempty"`
// Ask is the key, in AssignerAsksBucket, of the ask about the move, once asked.
Ask string `json:"ask,omitempty"`
// Snapshot says where the bus's streams were snapshotted, once they were.
Snapshot string `json:"snapshot,omitempty"`
// Steps are the steps taken, oldest first, at most PutStepsKept.
Steps []PutStep `json:"steps,omitempty"`
}
// Said is p at step with words, the step added to its steps when it differs from the last one.
func (p Put) Said(at time.Time, step, words string) Put {
if p.Step == step && p.Words == words {
return p
}
p.Step, p.Words = step, words
p.Steps = append(append([]PutStep(nil), p.Steps...), PutStep{At: at.UTC(), Step: step, Words: words})
if len(p.Steps) > PutStepsKept {
p.Steps = p.Steps[len(p.Steps)-PutStepsKept:]
}
return p
}
func (p Put) check() error {
if err := checkFingerprint(p.FP, "a put's fingerprint"); err != nil {
return err
}
if p.Why == "" {
return fmt.Errorf("delivery: a put says why")
}
switch p.Step {
case PutAsked, PutBuild, PutCompose, PutPerson, PutSnapshot, PutWanted, PutHolding, PutDone, PutReleased, PutRefused:
default:
return fmt.Errorf("delivery: %q is not a step of a put", p.Step)
}
if p.Composed != "" {
return checkDigest(p.Composed, "a put's composition digest")
}
return nil
}
// PutKey is the key, in AssignerPutsBucket, of the put of module on node: `<node>.<module>`.
func PutKey(node, module string) (string, error) {
return HaveKey(node, module)
}
// ParsePutKey reads a PutKey.
func ParsePutKey(key string) (node, module string, err error) {
node, module, isNode, err := ParseHaveKey(key)
if err == nil && isNode {
err = fmt.Errorf("delivery: %q is a node's own key, and no put", key)
}
return node, module, err
}
// EncodePut is p as JSON at Version; it refuses a fingerprint not of the new form, a put that says no why and an
// unknown step.
func EncodePut(p Put) ([]byte, error) {
if err := p.check(); err != nil {
return nil, err
}
p.V = Version
return json.Marshal(p)
}
// DecodePut reads a put; it refuses an unknown version (ErrUnknownVersion) and what EncodePut refuses.
func DecodePut(data []byte) (Put, error) {
var p Put
if err := decode(data, &p); err != nil {
return Put{}, err
}
if err := p.check(); err != nil {
return Put{}, err
}
return p, nil
}
-11
View File
@@ -269,10 +269,6 @@ type CandidatePull struct {
Head
Modules []string `json:"modules"`
Fps map[string]CandidateModule `json:"fps,omitempty"`
// Merged says the pull request merged while another member of its group is still open: it stays in the candidate
// with the head it merged at, so the heads hash an Approve covers holds until every member merged, and the checker
// judges the members still open by the verdict the group had (issue 1791). A reader of the older shape ignores it.
Merged bool `json:"merged,omitempty"`
}
// CandidateNoneNoModule is the None2 of a candidate's module whose directory the merge preview names no manifest for:
@@ -350,13 +346,6 @@ func (c Candidate) check() error {
if len(c.Pulls) == 0 {
return fmt.Errorf("delivery: a candidate names no pull request")
}
open := false
for _, p := range c.Pulls {
open = open || !p.Merged
}
if !open {
return fmt.Errorf("delivery: every pull request of the candidate merged: the group is no candidate any more")
}
heads := make([]Head, len(c.Pulls))
for i, p := range c.Pulls {
heads[i] = p.Head
+6 -2
View File
@@ -535,9 +535,13 @@ func ParseAskStepKey(key string) (node, askID string, err error) {
return node, askID, nil
}
// PauseKey is the key, in a build module's StepsBucket, that keeps whether the module on node takes asks:
// PausesState is the state a build module keeps its nodes' pause keys in. It has no age, unlike StepsState, whose keys
// are per step and dropped after their age: a pause kept among the steps would end by itself.
const PausesState = "pauses"
// PauseKey is the key, in a build module's PausesState, that keeps whether the module on node takes asks:
// `paused.<node>` (issue 1887's design limit 2: every holder pulls from one shared worker, so a node's pause is its holder
// not fetching, kept here so a restart keeps it). A pause key is never an ask's step key: its second part is a node.
// not fetching, kept so a restart keeps it). A pause key is never an ask's step key: its second part is a node.
func PauseKey(node string) (string, error) {
if err := checkName("node", node); err != nil {
return "", err
+7 -1
View File
@@ -283,9 +283,15 @@ func ViewAnswerFits(a ViewAnswer, ask AssignerAsk) error {
if a.Asking != ask.Ask.ID {
return fmt.Errorf("delivery: the view's answer is to the asking %s, and the ask stands at the asking %s", a.Asking, ask.Ask.ID)
}
if _, ok := ask.Ask.Option(a.Option); !ok {
o, ok := ask.Ask.Option(a.Option)
if !ok {
return fmt.Errorf("delivery: the view's answer chose %q, which the asking %s does not offer", a.Option, ask.Ask.ID)
}
if o.Level != "" && o.Level != asks.Acknowledge {
// An option that authorises is answered on a channel that proves who answered (ADR 0259 §5, ADR 0546 clause 8),
// never in the view, which proves nobody.
return fmt.Errorf("delivery: the view's answer chose %q, an option of level %s, which only a channel that proves who answered takes", a.Option, o.Level)
}
return nil
}
+7
View File
@@ -158,4 +158,11 @@ func TestViewAnswerFits(t *testing.T) {
if err := ViewAnswerFits(ViewAnswer{Asking: first.Ask.ID, Option: "restore"}, first); err == nil {
t.Errorf("an option the ask does not offer fits")
}
// An option that authorises is never answered in the view (ADR 0546 clause 8).
approving := first
approving.Ask.Options = append([]asks.Option(nil), first.Ask.Options...)
approving.Ask.Options[0].Level = asks.Approve
if err := ViewAnswerFits(answer, approving); err == nil || !strings.Contains(err.Error(), "approve") {
t.Errorf("the view answered an option of level approve: %v", err)
}
}
+32
View File
@@ -1,11 +1,16 @@
package process
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/nats-io/nats.go/jetstream"
)
// 1171/AC2: given an account file broker-<process id> in the controller's form {url, fingerprint, node,
@@ -46,6 +51,28 @@ func TestReadAccountToleratesNoClaimsAtAll(t *testing.T) {
// 1171/AC2: a file missing url, user or password is refused, naming the field; the password never appears
// in the error even when it is the field present and another is missing.
func TestReadAccountRefusesAFileMissingAField(t *testing.T) {
t.Run("1945: a refused bucket stops a start only when the part cannot work without it", func(t *testing.T) {
refusal := errors.New("nats: permissions violation for publish to $JS.API.STREAM.INFO.KV_mesh-delivery-gate_source")
for _, c := range []struct {
name string
err error
}{{"refused", refusal}, {"not answered", context.DeadlineExceeded}} {
var said []string
logf := func(f string, a ...any) { said = append(said, fmt.Sprintf(f, a...)) }
if kv := OptionalBucket(context.Background(), refusingBus{c.err}, "b", "b is not heard", logf); kv != nil {
t.Errorf("%s: an optional bucket answered %v", c.name, kv)
}
NotWatched(logf, "b", c.err, "b is not heard")
if len(said) != 2 || !strings.Contains(said[0], c.err.Error()) || !strings.Contains(said[1], "cannot be watched") {
t.Errorf("%s: said %q", c.name, said)
}
if _, err := EssentialBucket(context.Background(), refusingBus{c.err}, "builder", "b", "build"); err == nil ||
!strings.Contains(err.Error(), "grant of b") || !strings.Contains(err.Error(), "the builder cannot build") {
t.Errorf("%s: an essential bucket: %v", c.name, err)
}
}
})
for _, tc := range []struct {
field string
values map[string]any
@@ -82,3 +109,8 @@ func writeAccount(t *testing.T, fields map[string]any) string {
}
return path
}
// refusingBus refuses every bucket with err.
type refusingBus struct{ err error }
func (r refusingBus) KeyValue(context.Context, string) (jetstream.KeyValue, error) { return nil, r.err }
+55
View File
@@ -0,0 +1,55 @@
package process
import (
"context"
"fmt"
"github.com/nats-io/nats.go/jetstream"
)
// **A part does not stop for a bucket it can work without** (novox/hq issue 1945, met twice on 2026-10-25: the delivery
// builder exited on a refused read of the gate's source and crash-looped). A grant reaches the bus with a delivery, so a
// bucket, a watch or a consumer the bus refuses, or times out on, at start is often only not granted yet: exiting for it
// stops the part until a person's hand. A part opens what it cannot do its job at all without with EssentialBucket, which
// stops its start with an error naming the bucket and its grant, and everything else with OptionalBucket, which says what
// is not heard and goes on; a watch the bus refuses on an optional bucket is said with NotWatched. Here, beside State,
// because this is where a process opens what it keeps and reads on the bus; the parts that open buckets with their own
// JetStream (the delivery parts) take the same three.
// BucketOpener is the part of the bus a start opens buckets with: a jetstream.JetStream, or a test's fake.
type BucketOpener interface {
KeyValue(ctx context.Context, bucket string) (jetstream.KeyValue, error)
}
// EssentialBucket opens a bucket part cannot work without (it cannot `does` without it); its error is EssentialErr's.
func EssentialBucket(ctx context.Context, js BucketOpener, part, name, does string) (jetstream.KeyValue, error) {
kv, err := js.KeyValue(ctx, name)
if err != nil {
return nil, EssentialErr(part, name, does, err)
}
return kv, nil
}
// EssentialErr is the error of a bucket, consumer or stream named name that part cannot work without: it says what part
// cannot do, that it stops, and the grant to look at.
func EssentialErr(part, name, does string, err error) error {
return fmt.Errorf("%s cannot be read (%v): the %s cannot %s without it, so it stops; where the bus refused it or did not "+
"answer, this module's grant of %s (its manifest's reads, state or consumes, issued with its credential) has not "+
"reached the bus yet", name, err, part, does, name)
}
// OptionalBucket opens a bucket the part works without: any error is said with what is not heard (without), and nil
// answered.
func OptionalBucket(ctx context.Context, js BucketOpener, name, without string, logf func(string, ...any)) jetstream.KeyValue {
kv, err := js.KeyValue(ctx, name)
if err != nil {
logf("the bucket %s cannot be read (%v): %s", name, err, without)
return nil
}
return kv
}
// NotWatched says that an optional bucket opened but its watch was refused or not answered: what is not heard.
func NotWatched(logf func(string, ...any), name string, err error, without string) {
logf("the bucket %s cannot be watched (%v): %s", name, err, without)
}
+8
View File
@@ -154,8 +154,15 @@ const CodeChanged = -32010
var (
current *channel
currentMu sync.Mutex
// served is closed once Serve has the runtime's channel (Served).
served = make(chan struct{})
servedOnce sync.Once
)
// Served is closed once Serve has the runtime's channel, so code started before Serve can wait for it instead of
// asking again until the runtime takes it: a watch of a config value, a subscribe.
func Served() <-chan struct{} { return served }
// Serve answers the runtime on stdin and stdout until stdin closes. An empty name serves as the
// module the runtime named in MESH_SERVED_MODULE.
func Serve(name string, tools []Tool) error {
@@ -180,6 +187,7 @@ func ServeOn(name string, tools []Tool, in io.Reader, out io.Writer) error {
currentMu.Lock()
current = c
currentMu.Unlock()
servedOnce.Do(func() { close(served) })
// What the bundle said it keeps is announced once it serves: the runtime answers the publish on this channel, so
// it is asked from beside the loop that reads the answer.
go func() { _ = announce1() }()
+21
View File
@@ -16,6 +16,22 @@ func TestABundleEmitsThroughTheRuntimeAndServesAsItsModule(t *testing.T) {
t.Setenv(ServedAs, "chime")
t.Setenv("MESH_MODULE", "chime")
t.Setenv("MESH_NODE", "anchor")
// 1887/AC-wake: Served closes once Serve has the runtime's channel, not before; the globals are put back after.
oldServed := served
served, servedOnce = make(chan struct{}), sync.Once{}
t.Cleanup(func() {
served, servedOnce = oldServed, sync.Once{}
select {
case <-oldServed: // it was closed: its Once is spent too
servedOnce.Do(func() {})
default:
}
})
select {
case <-Served():
t.Fatal("Served closed before Serve")
default:
}
inR, inW := io.Pipe()
outR, outW := io.Pipe()
go func() {
@@ -50,6 +66,11 @@ func TestABundleEmitsThroughTheRuntimeAndServesAsItsModule(t *testing.T) {
if got := next()["result"].(map[string]any)["serverInfo"].(map[string]any)["name"]; got != "chime" {
t.Fatalf("served as %v, want the module the runtime named", got)
}
select {
case <-Served():
case <-time.After(2 * time.Second):
t.Fatal("Served did not close once serving")
}
send(map[string]any{"jsonrpc": "2.0", "id": 2, "method": "tools/call", "params": map[string]any{"name": "ring", "arguments": map[string]any{}}})
asked := next()
if asked["method"] != "mesh/publish" {