Compare commits

..
Author SHA1 Message Date
jschoubben 22c9ddd32b A runtime that already dropped a carried module's tools serves them again, read from what runs and its build (issue 1126, review)
rev-held2: ToolsRunning took a carried module's bundles from the runtime's MESH_TOOL_MODULES alone, and shanks had
already applied the drop of systemd-resolved, so its runs stayed empty and the drop permanent. Now every bundle
archive that runs at its module's BundlePath is a running bundle; one the runtime loads nothing of takes its loads and
words from the module's build (its built bundles, else its build's artifacts, by catalogue.BundleLoads, the rule a
build's resolution keeps, now one function). The runtime process and each archive are matched by their owning module
too, not only by id and path. A row added: the runtime already dropped it.
2026-10-29 21:03:16 +01:00
jschoubben 94cee5d199 A data flag that differs alone moves no data: the judgement reads the data items first, and says the flag loudly (issue 1126)
novox's mongodb and mssql asked a yes for moving their data on a toolchain move: the running side, read back from the
kept declaration, carries no data flag, and judge.go returned moves-data on the flag before it compared the data
directories and files, so a flag mismatch both raised a false ask and hid a real move. Now moves-data only when a
data item (a directory or file within the data places, or a process of a module with data) changes; a flag-only
difference is neither, with a WARNING in its words. One row changed and one added in TestJudgeMove.
2026-10-29 21:03:16 +01:00
jschoubben 52056010fe A module composed as what runs of it serves its tools as they run, so the runtime no longer drops it (issue 1126)
After the take-over of placement on 2026-10-29, shanks's runtime stopped serving the folded systemd-resolved (the
node-resolver seat's verbs answer "nothing serves") and novox's the folded nats. Neither has a want: each runs inside
its node's own composition at a build of the old form (Scope.InOwn), and is composed as Carried of the indexer's
manifest. That manifest is the catalogue's, which no build resolved, so it has no bundles (catalogue/build.go: Bundles
are set only by a build's resolution), and the runtime's MESH_TOOL_MODULES and MESH_TOOL_ENV are made from bundles
(catalogue/runtime.go). The controller composed them from the resolved manifest its store kept.

The kind: a module composed as what runs of it (Stub, Carried: held, or folded at an old build) contributes to the
others from its manifest, not from what runs. For a held module with a new build it is the other way round: its new
bundles are served, which were never delivered.

- The quick fix: read the old-form build's resolved manifest for a folded module. One instance; a held module still
  hands its new bundles.
- The thorough fix, chosen: compose.ToolsRunning reads what the node's runtime serves as it runs (its process's
  MESH_TOOL_MODULES and MESH_TOOL_ENV, each bundle's archive), frombus gives it as Node.RunsTools, and every stubbed
  module of the machine's runtime (held or carried) contributes those bundles, whatever its manifest says.

Tests: two rows (folded, held) added to TestCompositionsAreTheDeclarationSplitByModule; no new test function.
2026-10-29 21:03:16 +01:00
mesh-tools 307b6234e6 Merge pull request 'A held module offers what it runs, and its hold no longer spreads to the modules that read from it (issue 1960)' (#131) from fix/1960-no-hold-spread into main 2026-10-29 15:35:24 +00:00
jschoubben dbcaa29a1a Never hold a module because a module it reads from is held: it composes with what the held one last offered (issue 1960)
mesh/delivery merged: source.main moved to novox/mesh-sdk@307b6234e6bb
mesh/delivery-group group fix/1960-no-hold-spread merged: every member is merged
mesh/check pass dbcaa29a1a38 on 394fec8a63c1 | unit 3318 run, 1951 kept, 38 skipped | integration 3994 run, 2125 kept, 38 skipped | over bound: unit, …
On novox on 2026-10-29 de-spiegel, held for an unset config value, held mailu, route-proxy and nftables ("it reads
what de-spiegel contributes or listens on"), and mailu held postgres. mesh-sdk #128 (issue 1126) stopped the hold
spreading to consumers and collectors but kept dependents(): the provider of what a held module contributes to, and
the filter where it listens, were held behind it, transitively.

- Quick fix: drop dependents() alone. The filter and the proxy would then read a held module's new manifest, which
  may not be what runs (review of the apply, round 2).
- Thorough fix, chosen: no module is held behind another, in the apply and in the answer (dependents, heldBehind and
  the held-behind path of heldAsToday removed). A held module offers what it last offered: its contributions and
  listens from its manifest at the build its own composition runs at (ranOf), else the manifest it is composed from.
  Only the part of its offer that does not compose is left out, by provision (culprit): de-spiegel's routes leave
  route-proxy without them, while mailu keeps its smtp account; the hold names it.

Tests: rows changed in the existing answer test (de-spiegel held, mailu and proxy composed; site held, the filter
composed with what runs; leave-out by provision). No new test function.
2026-10-29 15:54:21 +01:00
mesh-tools 394fec8a63 Merge pull request 'A held module composes only as what runs of it, so its hold no longer spreads to the modules that use it (issue 1126)' (#128) from fix/1126-held-as-runs into main 2026-10-29 00:07:03 +00:00
jschoubben c2999a4e13 A held module is no dependence of its neighbours: its hold stops spreading (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@394fec8a63c1
mesh/delivery-group group fix/1126-held-as-runs merged: every member is merged
mesh/check pass c2999a4e13cc on d4c61ee8ffd0 | unit 3054 run, 234 kept, 36 skipped | integration 3075 run, 266 kept, 29 skipped | over bound: unit (sl…
novox's seed p-20261028t231141-ec84b4 refused 11 modules "what it composes of ..., held, is not what runs of them":
de-spiegel's hold spread through mailu to postgres, nftables and route-proxy, and from postgres to every consumer, and
gitea, held so, refused the node: "its credential of postgres-database is handed to the provisioner and the module is
not: nothing would fill it". Three rounds each moved the error. The kind: a held module still took part in the
composition as a dependence, so its hold spread.

- The quick fix: let heldAsToday also follow dependsOnHeld's words, and skip the handed refusal for gitea. It moves the
  error once more and leaves every node's applying path holding consumers.
- The thorough fix, chosen: a held module composes only as what runs of it. composeEach no longer compares the
  composition with each held module as its stub (dependsOnHeld removed): a consumer or collector composes against what
  runs of the held module like any other module, and a real difference is still a named difference in the answer, a
  module's own refusal still its refusal. A stub (compose.Stub, catalogue.Manifest.Stubbed) leaves the credentials and
  own secrets it would hand the provisioner out of its plan as they run, never refused for want of a plan. Only what a
  held module's new manifest says to others in their own files is still held behind it (dependents: the provider of
  what it contributes to, the filter where it listens), which heldAsToday already follows.

Tests: rows added to existing tests (a held provider with a binding consumer, which the old code held in the seed's
words; a held feeder of a collector; a held module with a handed credential and own secret). No new test function.
2026-10-29 00:49:00 +01:00
17 changed files with 555 additions and 518 deletions
+10 -114
View File
@@ -3,13 +3,10 @@ package buildstep
import ( import (
"context" "context"
"fmt" "fmt"
"io/fs"
"os" "os"
"path" "path"
"path/filepath" "path/filepath"
"strconv"
"strings" "strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/forgekey" "git.novox.be/novox/mesh-sdk/go/forgekey"
) )
@@ -32,25 +29,16 @@ const Bubblewrap = "bwrap"
// - the Workspace and Writable directories read-write, the ReadOnly ones (the toolchain, the modules the fetch step // - the Workspace and Writable directories read-write, the ReadOnly ones (the toolchain, the modules the fetch step
// downloaded) read-only; // downloaded) read-only;
// - the shared warm Caches read-only under a throwaway writable layer (S18), or read-write for a trunk build alone // - the shared warm Caches read-only under a throwaway writable layer (S18), or read-write for a trunk build alone
// (WriteCaches), so a candidate's code never changes what a later build reads; // (WriteCaches), so a candidate's code never changes what a later build reads.
// - with a CacheLayer, what one command writes into the caches kept for the later commands of the same ask (issue 1921,
// the try speed-up): the vet, the unit tests and the integration compile of one ask compile each package once.
type Isolation struct { type Isolation struct {
Workspace string Workspace string
Writable []string Writable []string
ReadOnly []string ReadOnly []string
Caches []string Caches []string
WriteCaches bool WriteCaches bool
// CacheLayer is a directory of the ask's own, in its workspace and gone with it: each command writes the caches into Env []string
// a layer of its own beneath it (overlayfs refuses an upper layer two mounts share, and an ask's commands run side by
// side), which Isolated moves into the ask's kept layer when the command ends; every later command reads the kept
// layer above the shared cache. The shared cache stays unchanged. Empty: what a command writes is thrown away.
CacheLayer string
Env []string
// Hosts is the file bound as /etc/hosts; Isolated writes one naming only the loopback. // Hosts is the file bound as /etc/hosts; Isolated writes one naming only the loopback.
Hosts string Hosts string
// run is the directory of this command's own writable layers under CacheLayer; Isolated makes one per command.
run string
} }
// etcNeeded is what of /etc a program in the step may read: who the users are, where the libraries are, the // etcNeeded is what of /etc a program in the step may read: who the users are, where the libraries are, the
@@ -103,7 +91,7 @@ func (iso Isolation) command(env []string, name string, args ...string) ([]strin
for _, group := range []struct { for _, group := range []struct {
what string what string
dirs []string dirs []string
}{{"writable directory", iso.Writable}, {"read-only directory", iso.ReadOnly}, {"cache", iso.Caches}, {"cache layer", layerDirs(iso.CacheLayer)}} { }{{"writable directory", iso.Writable}, {"read-only directory", iso.ReadOnly}, {"cache", iso.Caches}} {
for _, d := range group.dirs { for _, d := range group.dirs {
if err := checkDir(group.what, d); err != nil { if err := checkDir(group.what, d); err != nil {
return nil, err return nil, err
@@ -125,7 +113,7 @@ func (iso Isolation) command(env []string, name string, args ...string) ([]strin
for _, d := range iso.ReadOnly { for _, d := range iso.ReadOnly {
argv = append(argv, "--ro-bind", d, d) argv = append(argv, "--ro-bind", d, d)
} }
argv = append(argv, iso.cacheArgs()...) argv = append(argv, cacheArgs(iso.Caches, iso.WriteCaches)...)
argv = append(argv, "--bind", iso.Workspace, iso.Workspace) argv = append(argv, "--bind", iso.Workspace, iso.Workspace)
for _, d := range iso.Writable { for _, d := range iso.Writable {
argv = append(argv, "--bind", d, d) argv = append(argv, "--bind", d, d)
@@ -146,107 +134,19 @@ func (iso Isolation) command(env []string, name string, args ...string) ([]strin
// cacheArgs binds the shared warm caches (S18): for a trunk build read-write, so the next build is warm; for any other // cacheArgs binds the shared warm caches (S18): for a trunk build read-write, so the next build is warm; for any other
// read-only beneath an overlay whose writes go to a tmpfs of the step's own, so the step writes as a build does and // read-only beneath an overlay whose writes go to a tmpfs of the step's own, so the step writes as a build does and
// nothing it wrote outlives it (bubblewrap's --tmp-overlay, an overlay mount in the step's user namespace). With a // nothing it wrote outlives it (bubblewrap's --tmp-overlay, an overlay mount in the step's user namespace).
// CacheLayer the ask's kept layer lies above the shared cache and the command's writes go to its own layer on disk func cacheArgs(caches []string, write bool) []string {
// (--overlay), which Isolated keeps for the ask when the command ends.
func (iso Isolation) cacheArgs() []string {
var argv []string var argv []string
for i, c := range iso.Caches { for _, c := range caches {
switch { if write {
case iso.WriteCaches:
argv = append(argv, "--bind", c, c) argv = append(argv, "--bind", c, c)
case iso.CacheLayer != "": continue
upper, work := iso.runLayer(i)
argv = append(argv, "--overlay-src", c, "--overlay-src", keptLayer(iso.CacheLayer, i), "--overlay", upper, work, c)
default:
argv = append(argv, "--overlay-src", c, "--tmp-overlay", c)
} }
argv = append(argv, "--overlay-src", c, "--tmp-overlay", c)
} }
return argv return argv
} }
func layerDirs(layer string) []string {
if layer == "" {
return nil
}
return []string{layer}
}
// keptLayer is where the ask keeps what its commands wrote into the cache i.
func keptLayer(layer string, i int) string { return filepath.Join(layer, "kept", strconv.Itoa(i)) }
// runLayer is the upper and work directory of this command's writes into the cache i. Command, which makes no
// directory, names the layer "run"; Isolated gives each command one of its own.
func (iso Isolation) runLayer(i int) (upper, work string) {
run := iso.run
if run == "" {
run = filepath.Join(iso.CacheLayer, "run")
}
return filepath.Join(run, strconv.Itoa(i), "upper"), filepath.Join(run, strconv.Itoa(i), "work")
}
// layerLocks hold one mutex per cache layer, so two commands ending at once keep their writes one after the other.
var layerLocks sync.Map
// prepareLayer makes the kept layers and a run directory of the command's own under the cache layer.
func (iso *Isolation) prepareLayer() error {
if iso.CacheLayer == "" || iso.WriteCaches || len(iso.Caches) == 0 {
return nil
}
if err := os.MkdirAll(filepath.Join(iso.CacheLayer, "runs"), 0o700); err != nil {
return err
}
run, err := os.MkdirTemp(filepath.Join(iso.CacheLayer, "runs"), "run-")
if err != nil {
return err
}
iso.run = run
for i := range iso.Caches {
upper, work := iso.runLayer(i)
for _, d := range []string{keptLayer(iso.CacheLayer, i), upper, work} {
if err := os.MkdirAll(d, 0o700); err != nil {
return err
}
}
}
return nil
}
// keepLayer moves what the command wrote into the caches into the ask's kept layers, and removes its run directory.
// A file the command wrote replaces the kept one (a cache's newest write wins); a whiteout (a file the command
// deleted) is not kept, since a cache entry deleted by one command is no fault of a later one. A write that cannot be
// kept only means a later command builds it again.
func (iso Isolation) keepLayer() {
if iso.run == "" {
return
}
defer os.RemoveAll(iso.run)
mu, _ := layerLocks.LoadOrStore(iso.CacheLayer, &sync.Mutex{})
mu.(*sync.Mutex).Lock()
defer mu.(*sync.Mutex).Unlock()
for i := range iso.Caches {
upper, _ := iso.runLayer(i)
kept := keptLayer(iso.CacheLayer, i)
filepath.WalkDir(upper, func(p string, d fs.DirEntry, err error) error {
if err != nil || p == upper {
return nil
}
rel, _ := filepath.Rel(upper, p)
to := filepath.Join(kept, rel)
switch {
case d.IsDir():
if fi, err := os.Lstat(to); err == nil && !fi.IsDir() {
os.Remove(to)
}
os.MkdirAll(to, 0o700)
case d.Type().IsRegular():
os.Rename(p, to)
}
return nil
})
}
}
// loopbackHosts is the hosts file of a step: the loopback and nothing else. // loopbackHosts is the hosts file of a step: the loopback and nothing else.
const loopbackHosts = "127.0.0.1 localhost\n::1 localhost\n" const loopbackHosts = "127.0.0.1 localhost\n::1 localhost\n"
@@ -266,10 +166,6 @@ func Isolated(run forgekey.Runner, iso Isolation) forgekey.Runner {
} }
iso := iso iso := iso
iso.Hosts = hosts iso.Hosts = hosts
if err := iso.prepareLayer(); err != nil {
return forgekey.Ran{Status: 1, Err: "the step's cache layer: " + err.Error()}
}
defer iso.keepLayer()
argv, err := iso.command(env, name, args...) argv, err := iso.command(env, name, args...)
if err != nil { if err != nil {
return forgekey.Ran{Status: 1, Err: err.Error()} return forgekey.Ran{Status: 1, Err: err.Error()}
+1 -35
View File
@@ -1,9 +1,6 @@
package buildstep package buildstep
import ( import "testing"
"strings"
"testing"
)
// The shared warm caches are read under a throwaway writable layer (novox/hq ADR 0448; issue 1309): a candidate's step // The shared warm caches are read under a throwaway writable layer (novox/hq ADR 0448; issue 1309): a candidate's step
// writes into them as a build does, and nothing it writes outlives it or reaches a later build; only a trunk build // writes into them as a build does, and nothing it writes outlives it or reaches a later build; only a trunk build
@@ -30,35 +27,4 @@ func TestACandidatesStepWritesTheCachesOnlyIntoAThrowawayLayer(t *testing.T) {
t.Errorf("a trunk build does not write the cache %s: %v", c, argv) t.Errorf("a trunk build does not write the cache %s: %v", c, argv)
} }
} }
// 1921/S2: with a cache layer, a command reads the ask's kept layer above the shared cache and writes into a layer
// on disk of the ask's own, never into the shared cache and never into a throwaway tmpfs.
iso = Isolation{Workspace: "/w", Caches: []string{"/cache/gomod", "/cache/gobuild"}, CacheLayer: "/w/cachelayer"}
argv, err = iso.Command("go", "vet")
if err != nil {
t.Fatal(err)
}
for i, c := range iso.Caches {
if !holds(argv, "--overlay-src", c, "--overlay-src", keptLayer(iso.CacheLayer, i), "--overlay") {
t.Errorf("the cache %s is not read beneath the ask's kept layer: %v", c, argv)
}
if up := overlayOf(argv, c); up == "" || !strings.HasPrefix(up, "/w/cachelayer/") {
t.Errorf("the cache %s is written into %q, not a layer of the ask's own: %v", c, up, argv)
}
if holds(argv, "--tmp-overlay", c) || holds(argv, "--bind", c, c) {
t.Errorf("the cache %s is thrown away or written: %v", c, argv)
}
}
if _, err := (Isolation{Workspace: "/w", CacheLayer: "cachelayer"}).Command("go"); err == nil {
t.Error("a relative cache layer was taken")
}
}
// overlayOf answers the upper directory a command line gives the cache c, "" when it gives none.
func overlayOf(argv []string, c string) string {
for i := 0; i+3 < len(argv); i++ {
if argv[i] == "--overlay" && argv[i+3] == c {
return argv[i+1]
}
}
return ""
} }
-19
View File
@@ -63,23 +63,4 @@ func TestARunStepReachesNothingAndLeavesTheCachesUnchanged(t *testing.T) {
if b, err := os.ReadFile(filepath.Join(workspace, "out")); r.Status != 0 || err != nil || string(b) != "built\n" { if b, err := os.ReadFile(filepath.Join(workspace, "out")); r.Status != 0 || err != nil || string(b) != "built\n" {
t.Fatalf("the step's write to its workspace did not stay (status %d, %s%s): %q %v", r.Status, r.Stderr, r.Err, b, err) t.Fatalf("the step's write to its workspace did not stay (status %d, %s%s): %q %v", r.Status, r.Stderr, r.Err, b, err)
} }
// 1921/S2: with the ask's cache layer, what one command writes into the cache the next command of the ask reads,
// above what the shared cache holds, and the shared cache stays unchanged.
asked := Isolated(forgekey.ExecRunnerFor(0), Isolation{Workspace: workspace, Caches: []string{cacheDir},
CacheLayer: filepath.Join(workspace, "cachelayer")})
r = asked(context.Background(), []string{"PATH=/usr/bin:/bin"}, "/bin/sh", "-c", "cat "+filepath.Join(cacheDir, "warm")+" && mkdir -p "+filepath.Join(cacheDir, "ab")+" && echo compiled > "+filepath.Join(cacheDir, "ab", "one-d"))
if r.Status != 0 || r.Err != "" || strings.TrimSpace(r.Stdout) != "warm" {
t.Fatalf("the first command of the ask did not read the shared cache and write its own (status %d, %s): %s%s", r.Status, r.Err, r.Stdout, r.Stderr)
}
r = asked(context.Background(), []string{"PATH=/usr/bin:/bin"}, "/bin/cat", filepath.Join(cacheDir, "ab", "one-d"), filepath.Join(cacheDir, "warm"))
if r.Status != 0 || r.Err != "" || r.Stdout != "compiled\nwarm\n" {
t.Fatalf("the next command of the ask did not find what the first wrote (status %d, %s): %q %s", r.Status, r.Err, r.Stdout, r.Stderr)
}
if _, err := os.Stat(filepath.Join(cacheDir, "ab")); !os.IsNotExist(err) {
t.Fatalf("the ask's commands wrote the shared cache: %v", err)
}
if err := os.RemoveAll(filepath.Join(workspace, "cachelayer")); err != nil {
t.Fatalf("the ask's cache layer cannot be removed with its workspace: %v", err)
}
} }
-54
View File
@@ -2,10 +2,7 @@ package buildstep
import ( import (
"context" "context"
"os"
"path/filepath"
"strings" "strings"
"sync"
"testing" "testing"
"git.novox.be/novox/mesh-sdk/go/forgekey" "git.novox.be/novox/mesh-sdk/go/forgekey"
@@ -115,55 +112,4 @@ func TestIsolatedRunsEachCommandThroughBubblewrap(t *testing.T) {
if hosts == "" { if hosts == "" {
t.Fatalf("no hosts file was bound: %v", ran) t.Fatalf("no hosts file was bound: %v", ran)
} }
// 1921/S2: with a cache layer, what one command wrote into the caches is kept for the ask's later commands, the
// shared cache stays unchanged, two commands at once write into layers of their own (overlayfs refuses an upper
// layer two mounts share), and a command's own layer is gone when it ends.
root := t.TempDir()
shared := filepath.Join(root, "gobuild")
if err := os.MkdirAll(shared, 0o700); err != nil {
t.Fatal(err)
}
layer := filepath.Join(root, "ws", "cachelayer")
var mu sync.Mutex
uppers := map[string]bool{}
gate := make(chan struct{})
var started sync.WaitGroup
started.Add(2)
writer := func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
up := overlayOf(args, shared)
mu.Lock()
uppers[up] = true
mu.Unlock()
started.Done()
<-gate // both commands are running before either ends
os.MkdirAll(filepath.Join(up, "ab"), 0o700)
os.WriteFile(filepath.Join(up, "ab", args[len(args)-1]+"-d"), []byte("compiled"), 0o600)
return forgekey.Ran{}
}
run := Isolated(writer, Isolation{Workspace: filepath.Join(root, "ws"), Caches: []string{shared}, CacheLayer: layer})
var wg sync.WaitGroup
for _, pkg := range []string{"one", "two"} {
wg.Add(1)
go func() { defer wg.Done(); run(context.Background(), nil, "go", "build", pkg) }()
}
started.Wait()
close(gate)
wg.Wait()
if len(uppers) != 2 {
t.Fatalf("two commands at once wrote into %d layers, want 2 of their own: %v", len(uppers), uppers)
}
for _, pkg := range []string{"one", "two"} {
if b, err := os.ReadFile(filepath.Join(keptLayer(layer, 0), "ab", pkg+"-d")); err != nil || string(b) != "compiled" {
t.Errorf("what the build of %s compiled is not kept for the ask: %q %v", pkg, b, err)
}
}
for up := range uppers {
if _, err := os.Stat(up); !os.IsNotExist(err) {
t.Errorf("the command's own layer %s outlived it: %v", up, err)
}
}
if entries, _ := os.ReadDir(shared); len(entries) != 0 {
t.Errorf("the shared cache was written: %v", entries)
}
} }
+29 -17
View File
@@ -126,23 +126,7 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
continue continue
} }
made := by[a.Name] made := by[a.Name]
// What the runtime loads: what the artifact said, else every entrypoint of a module loads := BundleLoads(m, a, run)
// that declares tools, else nothing (the field's own rule; see Artifact.Loads).
//
// **Never, unasked, a bundle one of the module's own resources runs** (novox/hq issue 213).
// A process the host runs is the module's service, not its tools: the controller declares
// the verbs it answers as `tools` and serves them itself, and its bundle would otherwise
// have been launched a second time by the node's runtime, as an MCP child it is not.
loads := append([]string(nil), a.Loads...)
if a.Loads == nil && len(m.Tools) > 0 && !run[a.Name] {
loads = append([]string(nil), a.Entrypoints...)
// A bundle compiled to a binary has no entrypoints: the binary is what it is, and what
// the runtime starts to serve it (novox/hq ADR 0193). So a Go tools bundle is served
// as Go — the runtime execs it — exactly as a TypeScript one is through its launcher.
if bin := BinaryOf(a); bin != "" {
loads = []string{bin}
}
}
// **Kept, never routed** (ADR 0155): the builder publishes to the store at the address // **Kept, never routed** (ADR 0155): the builder publishes to the store at the address
// it reached it by, and a manifest carrying that address names an installation — // it reached it by, and a manifest carrying that address names an installation —
// registration refused node-tools for exactly this on 2026-10-02. The build record // registration refused node-tools for exactly this on 2026-10-02. The build record
@@ -715,3 +699,31 @@ func undeliveredBundles(m Manifest) []string {
} }
return problems return problems
} }
// BundleLoads is what the runtime loads of the bundle artifact a of m (run: the bundles one of m's own resources runs,
// runByAResource): what the artifact said, else every entrypoint of a module that declares tools, else nothing; a
// bundle compiled to a binary is served by its binary. The rule a build's resolution keeps on Bundle.Loads, and what a
// module composed as what runs of it reads again when the runtime already dropped it (issue 1126).
func BundleLoads(m Manifest, a Artifact, run map[string]bool) []string {
// What the runtime loads: what the artifact said, else every entrypoint of a module
// that declares tools, else nothing (the field's own rule; see Artifact.Loads).
//
// **Never, unasked, a bundle one of the module's own resources runs** (novox/hq issue 213).
// A process the host runs is the module's service, not its tools: the controller declares
// the verbs it answers as `tools` and serves them itself, and its bundle would otherwise
// have been launched a second time by the node's runtime, as an MCP child it is not.
loads := append([]string(nil), a.Loads...)
if a.Loads == nil && len(m.Tools) > 0 && !run[a.Name] {
loads = append([]string(nil), a.Entrypoints...)
// A bundle compiled to a binary has no entrypoints: the binary is what it is, and what
// the runtime starts to serve it (novox/hq ADR 0193). So a Go tools bundle is served
// as Go — the runtime execs it — exactly as a TypeScript one is through its launcher.
if bin := BinaryOf(a); bin != "" {
loads = []string{bin}
}
}
return loads
}
// RunByAResource is the bundles of m one of its own resources runs (its service, not its tools).
func RunByAResource(m Manifest) map[string]bool { return runByAResource(m) }
+6
View File
@@ -1103,6 +1103,9 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
} }
for _, unsettled := range resources { for _, unsettled := range resources {
if pr, ok := unsettled[HandedProvisionField].(PlanProvision); ok { if pr, ok := unsettled[HandedProvisionField].(PlanProvision); ok {
if handing == nil && m.Stubbed {
continue // held: its credential file is what runs of it, left out (issue 1126)
}
if handing == nil { if handing == nil {
return nil, fmt.Errorf("%s: its credential of %s is handed to the provisioner and the module is "+ return nil, fmt.Errorf("%s: its credential of %s is handed to the provisioner and the module is "+
"not: nothing would fill it", m.Module, pr.Provision) "not: nothing would fill it", m.Module, pr.Provision)
@@ -1127,6 +1130,9 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
continue continue
} }
if unsettled[HandedSecretField] != nil { if unsettled[HandedSecretField] != nil {
if handing == nil && m.Stubbed {
continue // held: its own secret's file is what runs of it, left out (issue 1126)
}
if handing == nil { if handing == nil {
return nil, fmt.Errorf("%s: its own secret %v is handed to the provisioner and the module is "+ return nil, fmt.Errorf("%s: its own secret %v is handed to the provisioner and the module is "+
"not: nothing would fill it", m.Module, unsettled[HandedSecretField]) "not: nothing would fill it", m.Module, unsettled[HandedSecretField])
+16
View File
@@ -185,6 +185,22 @@ func TestAHandedModulesCredentialIssuedForItIsNamedInItsPlanAndNoValueIsSealed(t
if !reflect.DeepEqual(c.Plans["bot"].Provisions, want) { if !reflect.DeepEqual(c.Plans["bot"].Provisions, want) {
t.Fatalf("the plan names %+v, want %+v", c.Plans["bot"].Provisions, want) t.Fatalf("the plan names %+v, want %+v", c.Plans["bot"].Provisions, want)
} }
// Issue 1126 (novox's seed: "gitea: its credential of postgres-database is handed to the provisioner and the module
// is not"): held, the module is composed as its stub, with no template and no item, so nothing is handed. Its
// credential and its own secret are what runs of it, left out, and the composition is not refused.
r := forgeReader(t, true, true)
for i := range r.Modules {
if r.Modules[i].Module == "bot" {
h := *r.Modules[i].Handed
h.Templates = nil
r.Modules[i].Handed, r.Modules[i].Stubbed, r.Modules[i].Build = &h, true, nil
r.Modules[i].Resources = slices.DeleteFunc(r.Modules[i].Resources, func(x map[string]any) bool { return x["type"] != "directory" })
}
}
c, got = composeWith(t, r, handingTo("store:5000"))
if strings.Contains(got, SecretID("forge-read")) || strings.Contains(got, NeedID("token")) || c.Plans["bot"].Provisions != nil {
t.Fatalf("a held module's handed credential or own secret is composed: %s", got)
}
} }
func TestACredentialIssuedForAHolderIsSealedAsBeforeWhereItsModuleIsNotHanded(t *testing.T) { func TestACredentialIssuedForAHolderIsSealedAsBeforeWhereItsModuleIsNotHanded(t *testing.T) {
+5
View File
@@ -870,6 +870,11 @@ type Manifest struct {
// path its manifest leaves to the mesh, a binding). Nil for any other module. Never in a manifest's JSON. // path its manifest leaves to the mesh, a binding). Nil for any other module. Never in a manifest's JSON.
OwnsPaths []string `json:"-"` OwnsPaths []string `json:"-"`
// Stubbed marks a module composed as its stub (compose.Stub: held on its node, issue 1126): none of its own
// composition is applied, so what it hands the provisioner (a credential the vault keeps for it, its own secrets)
// is left out of its plan as it runs, never refused for want of a plan. Never in a manifest's JSON.
Stubbed bool `json:"-"`
// Jails are the fail2ban jails this module declares for its own service (novox/hq to-be 31). // Jails are the fail2ban jails this module declares for its own service (novox/hq to-be 31).
// Written into whichever node runs the module, the same way `listens` become that node's rules. // Written into whichever node runs the module, the same way `listens` become that node's rules.
Jails []Jail `json:"jails,omitempty"` Jails []Jail `json:"jails,omitempty"`
+118
View File
@@ -130,6 +130,22 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
n.Kept = kept n.Kept = kept
} }
} }
// A module composed as what runs of it contributes to the runtime what the runtime serves of it as it runs
// (Node.RunsTools); one of its own account is served by a runtime of its own, from its manifest as before.
for i := range reading.Nodes {
n := &reading.Nodes[i]
if n.Name != node || n.RunsTools == nil {
continue
}
kept := make(map[string]catalogue.Manifest, len(n.Kept))
for m, man := range n.Kept {
if man.Stubbed && man.RunsAs == "" {
man.Bundles = runningBundles(n.RunsTools[m], reading.Catalogue[m])
}
kept[m] = man
}
n.Kept = kept
}
isCarried := func(module, id string) bool { isCarried := func(module, id string) bool {
if module == "" { if module == "" {
module, _ = declaration.ModuleGuess(id) module, _ = declaration.ModuleGuess(id)
@@ -293,6 +309,106 @@ func RunsAnyOf(runs *delivery.Composed, module string) bool {
return false return false
} }
// ToolsRunning is, by module, the tools bundles that run on the node (Node.RunsTools): each bundle archive a module's
// resource unpacks at its BundlePath, with what the runtime's own process loads of it and the words it hands it
// (catalogue.RuntimeToolModules, catalogue.RuntimeToolEnv, the mesh's own words left out). A bundle the runtime loads
// nothing of has no Loads: the runtime dropped it, and the composition reads its loads again from the module's build
// (issue 1126, review: shanks had already applied the drop of systemd-resolved). Nil when running holds no runtime process.
func ToolsRunning(running []delivery.Composed) map[string][]catalogue.Bundle {
runtimeID := catalogue.RuntimeModule + "." + catalogue.RuntimeProcessID()
type resource struct {
ID string `json:"id"`
Module string `json:"module"`
Type string `json:"type"`
Path string `json:"path"`
Source string `json:"source"`
Digest string `json:"digest"`
Env map[string]string `json:"env"`
}
var env map[string]string
out := map[string][]catalogue.Bundle{}
for _, c := range running {
for _, raw := range c.Resources {
var r resource
if json.Unmarshal(raw, &r) != nil {
continue
}
owner := r.Module
if owner == "" {
owner, _ = declaration.ModuleGuess(r.ID)
}
switch {
case r.Type == "process" && r.ID == runtimeID && owner == catalogue.RuntimeModule:
env = r.Env
case r.Type == "archive" && owner != "" && strings.HasPrefix(r.ID, owner+"."+catalogue.BundleID("")):
name := strings.TrimPrefix(r.ID, owner+"."+catalogue.BundleID(""))
if r.Path == catalogue.BundlePath(owner, name) {
out[owner] = append(out[owner], catalogue.Bundle{Name: name, Source: catalogue.Recorded(r.Source), Digest: r.Digest})
}
}
}
}
if env == nil {
return nil
}
var words map[string]map[string]string
_ = json.Unmarshal([]byte(env[catalogue.RuntimeToolEnv]), &words)
for _, entry := range strings.Split(env[catalogue.RuntimeToolModules], ",") {
module, path, ok := strings.Cut(entry, "=")
if !ok || !strings.HasPrefix(path, catalogue.BundleRoot+"/"+module+"/") {
continue
}
name, load, ok := strings.Cut(strings.TrimPrefix(path, catalogue.BundleRoot+"/"+module+"/"), "/")
bundles := out[module]
i := slices.IndexFunc(bundles, func(b catalogue.Bundle) bool { return b.Name == name })
if !ok || i < 0 {
continue // no archive of it runs: nothing to load
}
var own map[string]string
for w, v := range words[module] {
if w == catalogue.FilesOwnedPaths || w == catalogue.ReadsSecretPlaces {
continue // the mesh's to say, composed again
}
if own == nil {
own = map[string]string{}
}
own[w] = v
}
bundles[i].Loads = append(bundles[i].Loads, load)
bundles[i].Env = own
}
for _, bundles := range out {
sort.Slice(bundles, func(i, j int) bool { return bundles[i].Name < bundles[j].Name })
}
return out
}
// runningBundles is the bundles of a module composed as what runs of it (Node.RunsTools): each that runs, with what the
// runtime loads of it and its words; one the runtime loads nothing of takes its loads and words from the module's build
// (man: its built bundles, else its build's artifacts, catalogue.BundleLoads), so a drop already applied is undone.
func runningBundles(runs []catalogue.Bundle, man catalogue.Manifest) []catalogue.Bundle {
var out []catalogue.Bundle
for _, b := range runs {
if len(b.Loads) == 0 {
for _, built := range man.Bundles {
if built.Name == b.Name {
b.Loads, b.Env, b.Launchers, b.Binary = built.Loads, built.Env, built.Launchers, built.Binary
}
}
if man.Build != nil {
run := catalogue.RunByAResource(man)
for _, a := range man.Build.Artifacts {
if a.Kind == catalogue.ArtifactBundle && a.Name == b.Name {
b.Loads, b.Env, b.Binary = catalogue.BundleLoads(man, a, run), a.Env, catalogue.BinaryOf(a)
}
}
}
}
out = append(out, b)
}
return out
}
// NamesBuild says a manifest names what a build makes: its build, or a resource naming an artifact. A node cannot // NamesBuild says a manifest names what a build makes: its build, or a resource naming an artifact. A node cannot
// compose such a module folded into its own composition: the build of the old form it runs is held by the controller // compose such a module folded into its own composition: the build of the old form it runs is held by the controller
// alone (issue 1126). // alone (issue 1126).
@@ -318,6 +434,8 @@ func Stub(m catalogue.Manifest) catalogue.Manifest {
m.OwnsPaths = catalogue.DeclaredPaths(m) m.OwnsPaths = catalogue.DeclaredPaths(m)
} }
m.Build = nil m.Build = nil
// What it hands the provisioner is left out as it runs, not refused for want of a plan (catalogue.Manifest.Stubbed).
m.Stubbed = true
// Nor is a template its build kept for its own files handed: a stub carries none of them (issue 1126: novox's node // Nor is a template its build kept for its own files handed: a stub carries none of them (issue 1126: novox's node
// refused "postgres: its build kept a template for consumers, which it does not declare" with postgres held). Its // refused "postgres: its build kept a template for consumers, which it does not declare" with postgres held). Its
// items stay, and the templates they render through: a held module's contributions to other modules' collections // items stay, and the templates they render through: a held module's contributions to other modules' collections
+86
View File
@@ -8,6 +8,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery" "git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/overlay"
) )
func fpFor(c byte) string { return delivery.SourcePrefix2 + strings.Repeat(string(c), 64) } func fpFor(c byte) string { return delivery.SourcePrefix2 + strings.Repeat(string(c), 64) }
@@ -96,6 +97,91 @@ func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) {
if _, _, err := Compositions(in, "nowhere", fps, Foreseeing); err == nil { if _, _, err := Compositions(in, "nowhere", fps, Foreseeing); err == nil {
t.Errorf("a node the mesh does not have was composed") t.Errorf("a node the mesh does not have was composed")
} }
// Issue 1126, 2026-10-29: a module composed as what runs of it (folded at a build of the old form, or held) still
// contributes to the runtime what the runtime serves of it as it runs (Node.RunsTools): its manifest at the
// indexer's commit has no bundles (shanks's runtime lost the folded systemd-resolved, novox's the folded nats), and a
// held module's new build's bundles are not delivered.
dg := func(c byte) string { return "sha256:" + strings.Repeat(string(c), 64) }
runtime := catalogue.Manifest{Module: catalogue.RuntimeModule, Version: "1",
OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/var/lib/mesh/node-tools/broker"}},
Bundles: []catalogue.Bundle{{Name: "runtime", Source: catalogue.ArtifactStoreScheme + "node-tools/runtime/blobs/" + dg('a'),
Digest: dg('a'), Language: "go", Binary: "node-tools"}}}
store := catalogue.Manifest{Module: "artifacts", Version: "1",
Serves: map[string]map[string]any{catalogue.ArtifactStoreProvision: {"port": 5100}}}
lamp := catalogue.Manifest{Module: "lamp", Version: "1", Tools: []string{"on"},
Resources: []map[string]any{{"id": "state", "type": "directory", "path": "/var/lib/lamp", "mode": "0700"}},
Build: &catalogue.Build{Artifacts: []catalogue.Artifact{{Name: "tools", Kind: catalogue.ArtifactBundle,
Language: "go", System: "arch", From: "cmd/lamp-tools", Env: map[string]string{"LAMP_CONF": "/var/lib/lamp/conf"}}}}}
newLamp, err := lamp.Resolve([]catalogue.Built{{Name: "tools", Kind: catalogue.ArtifactBundle,
Reference: catalogue.ArtifactStoreScheme + "lamp/tools/blobs/" + dg('c'), Digest: dg('c')}})
if err != nil {
t.Fatal(err)
}
served := "lamp=" + catalogue.BundlePath("lamp", "tools") + "/lamp-tools"
runtimeRuns := func(dropped bool) delivery.Composed {
env := `"MESH_TOOL_MODULES":"` + served + `","MESH_TOOL_ENV":"{\"lamp\":{\"LAMP_CONF\":\"/var/lib/lamp/conf\"}}"`
if dropped {
env = `"MESH_TOOL_MODULES":""` // the drop already applied (shanks, 2026-10-29)
}
return delivery.Composed{Node: "anchor", Module: catalogue.RuntimeModule, Resources: []json.RawMessage{json.RawMessage(
`{"id":"node-tools.runtime","module":"node-tools","type":"process","name":"node-tools","env":{` + env + `}}`)}}
}
lampRuns := `{"id":"lamp.bundle-tools","type":"archive","source":"` + catalogue.ArtifactStoreScheme + `lamp/tools/blobs/` + dg('b') +
`","digest":"` + dg('b') + `","path":"` + catalogue.BundlePath("lamp", "tools") + `"}`
for _, row := range []struct {
name string
set func(n *Node) delivery.Composed // and what runs of lamp
fps map[string]string
dropped bool
}{
{"folded at a build of the old form", func(n *Node) delivery.Composed {
n.InOwn = map[string]bool{"lamp": true}
n.RunsOwn = &delivery.Composed{Node: "anchor", Resources: []json.RawMessage{json.RawMessage(lampRuns)}}
return *n.RunsOwn
}, map[string]string{}, false},
{"folded, the runtime already dropped it", func(n *Node) delivery.Composed {
n.InOwn = map[string]bool{"lamp": true}
n.RunsOwn = &delivery.Composed{Node: "anchor", Resources: []json.RawMessage{json.RawMessage(lampRuns)}}
return *n.RunsOwn
}, map[string]string{}, true},
{"held, its new build not delivered", func(n *Node) delivery.Composed {
n.Held = map[string]bool{"lamp": true}
lr := delivery.Composed{Node: "anchor", Module: "lamp", Resources: []json.RawMessage{json.RawMessage(
strings.Replace(lampRuns, `"type"`, `"module":"lamp","type"`, 1))}}
n.Kept = map[string]catalogue.Manifest{"lamp": Carried(newLamp, "lamp", &lr)}
return lr
}, map[string]string{"lamp": fpFor('1')}, false},
} {
tools := aMesh(t)
tools.Catalogue[catalogue.RuntimeModule], tools.Catalogue["artifacts"], tools.Catalogue["lamp"] = runtime, store, lamp
anchor := nodeOf(t, &tools, "anchor")
anchor.Assigned = append(anchor.Assigned, "artifacts", catalogue.RuntimeModule, "lamp")
tools.BusMinted = []string{"anchor." + catalogue.RuntimeModule}
anchor.SealingKey = keyOf(t, "anchor")
anchor.RunsTools = ToolsRunning([]delivery.Composed{runtimeRuns(row.dropped), row.set(anchor)})
row.fps[overlay.Name], row.fps["artifacts"], row.fps[catalogue.RuntimeModule] = fpFor('a'), fpFor('5'), fpFor('e')
out, _, _, err := CompositionsIn(tools, "anchor", row.fps, Foreseeing)
if err != nil {
t.Fatalf("%s: %v", row.name, err)
}
var env map[string]any
for _, c := range out {
for _, raw := range c.Resources {
var r struct {
ID string `json:"id"`
Env map[string]any `json:"env"`
}
if json.Unmarshal(raw, &r) == nil && r.ID == "node-tools.runtime" {
env = r.Env
}
}
}
if env[catalogue.RuntimeToolModules] != served || env[catalogue.RuntimeToolEnv] != `{"lamp":{"LAMP_CONF":"/var/lib/lamp/conf"}}` {
t.Errorf("%s: the runtime serves %v with %v, want what runs of lamp, %s", row.name,
env[catalogue.RuntimeToolModules], env[catalogue.RuntimeToolEnv], served)
}
}
} }
// 1943/AC1: the upgrade policies and sources a composition's value may name are the catalogue's own words. // 1943/AC1: the upgrade policies and sources a composition's value may name are the catalogue's own words.
+116 -234
View File
@@ -2,8 +2,8 @@ package frombus
import ( import (
"context" "context"
"encoding/json"
"fmt" "fmt"
"maps"
"regexp" "regexp"
"slices" "slices"
"sort" "sort"
@@ -11,7 +11,6 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose" "git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery" "git.novox.be/novox/mesh-sdk/go/delivery"
) )
@@ -72,7 +71,7 @@ func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]st
return refuse(fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err)) return refuse(fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err))
} }
} }
composed, skipped, stood, err := composeEach(in, s.Node, fps, false, s.Running, failed) composed, skipped, stood, err := composeEach(in, s.Node, fps, false, s.Running, failed, ranOf(ctx, r, mesh, s.Running, fps))
if err != nil { if err != nil {
// Not one module's: the node's own (a value served elsewhere, the mesh's network), said once as the node's. // Not one module's: the node's own (a value served elsewhere, the mesh's network), said once as the node's.
if len(failed) == 0 { if len(failed) == 0 {
@@ -209,59 +208,24 @@ func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]st
return a return a
} }
// heldBehind is why a module is held with a held one it reads (composeEach's dependents).
func heldBehind(m string) string {
return "it reads what " + m + " contributes or listens on, and " + m + " is held"
}
// behind is the held module a refusal's words say it is held behind, or "".
func behind(why string) string {
rest, ok := strings.CutPrefix(why, "it reads what ")
if !ok {
return ""
}
m, _, ok := strings.Cut(rest, " contributes or listens on, and ")
if !ok || why != heldBehind(m) {
return ""
}
return m
}
// heldAsToday moves into Held each refusal of a module the node keeps today as it runs (Scope.HeldToday, with how: // heldAsToday moves into Held each refusal of a module the node keeps today as it runs (Scope.HeldToday, with how:
// held, failed, running a build of the old form), and of each module held only behind one of those: the candidate keeps // held, failed, running a build of the old form): the candidate keeps it as the node does, which is no change (issue
// them as the node does, which is no change (issue 1126: a seed reproduces today, and today keeps them so). Each says // 1126: a seed reproduces today, and today keeps them so). Each says how it stands today (ModuleRefused.Today). No
// how it stands today (ModuleRefused.Today). // module is held behind another (issue 1960): a module that reads from a held one composes with what it last offered.
func heldAsToday(a *delivery.CandidateAnswer, today map[string]string) { func heldAsToday(a *delivery.CandidateAnswer, today map[string]string) {
if len(today) == 0 { if len(today) == 0 {
return return
} }
how := map[string]string{} kept := a.Refusals[:0]
for m, w := range today { for _, r := range a.Refusals {
how[m] = w if w, keep := today[r.Module]; keep {
} r.Today = w
for moved := true; moved; { a.Held = append(a.Held, r)
moved = false } else {
kept := a.Refusals[:0] kept = append(kept, r)
for _, r := range a.Refusals {
w, keep := how[r.Module]
if !keep {
if m := behind(r.Why); m != "" {
if _, behindKept := how[m]; behindKept {
w, keep = "held behind "+m, true
}
}
}
if keep {
r.Today = w
how[r.Module] = w
a.Held = append(a.Held, r)
moved = true
} else {
kept = append(kept, r)
}
} }
a.Refusals = kept
} }
a.Refusals = kept
if len(a.Refusals) == 0 { if len(a.Refusals) == 0 {
a.Refusals = nil a.Refusals = nil
} }
@@ -312,12 +276,18 @@ func standsInFor(x, node string) (string, string) {
// **A held module is what runs of it** (review of the apply, round 1): composed from what running holds of it // **A held module is what runs of it** (review of the apply, round 1): composed from what running holds of it
// (compose.Carried of its own composition there; the node's own carries it wherever it runs there), so what the others // (compose.Carried of its own composition there; the node's own carries it wherever it runs there), so what the others
// read of it (the machine ports its containers publish, the routes reaching it) is what runs, and none of its own // read of it (the machine ports its containers publish, the routes reaching it) is what runs, and none of its own
// composition is in out: the node keeps what it last applied. **What depends on how it is held is held too:** a module // composition is in out: the node keeps what it last applied.
// whose composition differs when the held ones are composed as their stubs instead is held with them, and a node's own //
// composition that differs (but for the held modules' own resources) refuses the node. And since a held module is // **A hold never spreads** (issue 1126, issue 1960: de-spiegel's hold for an unset value held mailu, which held
// composed from its new manifest, which may not be the build's that runs, what reads that manifest is held with it // route-proxy, nftables and postgres on novox). No module is held because a module it reads from is held: a consumer,
// (dependents): the providers of what it contributes to (the proxy its routes), and, when it listens, the filter. // a collector, the provider of what the held one contributes to and the filter where it listens each compose with what
func composeEach(in compose.Input, node string, fps map[string]string, applied bool, running []delivery.Composed, failed map[string]string) (out map[string]delivery.Composed, skipped, stood []string, err error) { // the held module last offered. That offer (what it contributes, contributes many, listens on) is its manifest at the
// build it runs where one is known (ran: a composition of its own running at another build than it is composed at),
// else the manifest it is composed from. Only the part of the offer that cannot compose is left out, by provision
// (culprit): de-spiegel's routes, its names unset, leave route-proxy without them while mailu keeps its smtp account,
// and the hold says so.
func composeEach(in compose.Input, node string, fps map[string]string, applied bool, running []delivery.Composed, failed map[string]string,
ran func(module string) (catalogue.Manifest, bool)) (out map[string]delivery.Composed, skipped, stood []string, err error) {
mode := compose.Foreseeing mode := compose.Foreseeing
if applied { if applied {
mode = compose.Reading mode = compose.Reading
@@ -328,43 +298,46 @@ func composeEach(in compose.Input, node string, fps map[string]string, applied b
runs[m] = &running[i] runs[m] = &running[i]
} }
} }
src := map[string]catalogue.Manifest{} // each held module's manifest before it was held src := map[string]catalogue.Manifest{} // each held module's manifest before it was held, offering what it last offered
bareOf := map[string]bool{} left := map[string]map[string]bool{} // by held module, what of its offer is left out (leaveOut)
var hold func(m, why string, bare bool) hold := func(m, why, leave string) {
hold = func(m, why string, bare bool) {
if _, done := failed[m]; !done { if _, done := failed[m]; !done {
failed[m] = why // a held module tried bare keeps the words it was first held for failed[m] = why // a held module tried again keeps the words it was first held for
} }
first := false
if _, ok := src[m]; !ok { if _, ok := src[m]; !ok {
man, ok := manifestOn(in, node, m) man, ok := manifestOn(in, node, m)
if !ok { if !ok {
return return
} }
src[m], first = man, true if ran != nil {
if was, ok := ran(m); ok {
man.Contributes, man.ContributesMany, man.Listens = was.Contributes, was.ContributesMany, was.Listens
}
}
src[m] = man
} }
bareOf[m] = bareOf[m] || bare if leave != "" {
in = heldAs(in, node, m, src[m], bareOf[m], runs[m]) if left[m] == nil {
if !first { left[m] = map[string]bool{}
return }
} left[m][leave] = true
// What the others read of its manifest (what it contributes, what it listens on) is its new manifest's, not if leave != leaveAll {
// necessarily the build's that runs: what reads it is held with it (review of the apply, round 2). failed[m] += "; composed without what it contributes to " + leave + ", which does not compose"
for _, r := range dependents(in, node, src[m], failed) { }
hold(r, heldBehind(m), false)
} }
in = heldAs(in, node, m, src[m], left[m], runs[m])
} }
for _, m := range sortedKeys(failed) { for _, m := range sortedKeys(failed) {
hold(m, failed[m], false) hold(m, failed[m], "")
} }
for { for {
out, skipped, stood, err = compose.CompositionsIn(in, node, fps, mode) out, skipped, stood, err = compose.CompositionsIn(in, node, fps, mode)
if err != nil { if err != nil {
module, bare := culprit(in, node, fps, mode, failed, err) module, leave := culprit(in, node, fps, mode, failed, err)
if module == "" { if module == "" {
return nil, nil, nil, err return nil, nil, nil, err
} }
hold(module, err.Error(), bare) hold(module, err.Error(), leave)
continue continue
} }
again := false again := false
@@ -384,7 +357,7 @@ func composeEach(in compose.Input, node string, fps map[string]string, applied b
} }
for _, m := range sortedKeys(of) { for _, m := range sortedKeys(of) {
if _, done := failed[m]; !done { if _, done := failed[m]; !done {
hold(m, "stands in for "+strings.Join(of[m], ", "), false) hold(m, "stands in for "+strings.Join(of[m], ", "), "")
again = true again = true
} }
} }
@@ -392,17 +365,6 @@ func composeEach(in compose.Input, node string, fps map[string]string, applied b
return nil, nil, nil, fmt.Errorf("%w on %s: %s", compose.ErrStoodIn, node, strings.Join(nodes, ", ")) return nil, nil, nil, fmt.Errorf("%w on %s: %s", compose.ErrStoodIn, node, strings.Join(nodes, ", "))
} }
} }
if !again && len(failed) > 0 {
dependent, own := dependsOnHeld(in, node, fps, mode, out, src, bareOf, failed)
if own {
return nil, nil, nil, fmt.Errorf("frombus: %s's own composition depends on how %s, held, is composed, and is not applied",
node, strings.Join(sortedKeys(failed), ", "))
}
for _, m := range dependent {
hold(m, "what it composes of "+strings.Join(sortedKeys(failed), ", ")+", held, is not what runs of them", false)
again = true
}
}
if !again { if !again {
break break
} }
@@ -411,117 +373,9 @@ func composeEach(in compose.Input, node string, fps map[string]string, applied b
return out, skipped, stood, nil return out, skipped, stood, nil
} }
// dependsOnHeld composes in again with each held module as its stub rather than as what runs of it, and answers the // leaveAll leaves out the whole of a held module's offer and its own secrets (a bare stub); any other word leaves out
// modules, not held, whose composition differs between the two (sorted), and own when the node's own composition does // what it contributes to that provision alone.
// but for the held modules' own resources. A composition of stubs that fails says nothing more: what runs composed. const leaveAll = "*"
func dependsOnHeld(in compose.Input, node string, fps map[string]string, mode compose.Mode, asRuns map[string]delivery.Composed,
src map[string]catalogue.Manifest, bareOf map[string]bool, failed map[string]string) (dependent []string, own bool) {
stubs := in
for _, m := range sortedKeys(failed) {
if man, ok := src[m]; ok {
stubs = heldAs(stubs, node, m, man, bareOf[m], nil)
}
}
asStubs, _, _, err := compose.CompositionsIn(stubs, node, fps, mode)
if err != nil {
return nil, false
}
isHeld := func(m string) bool { _, ok := failed[m]; return ok }
keys := map[string]bool{}
for k := range asRuns {
keys[k] = true
}
for k := range asStubs {
keys[k] = true
}
for _, k := range sortedKeys(keys) {
a, inA := asRuns[k]
b, inB := asStubs[k]
m := a.Module
if !inA {
m = b.Module
}
if m != "" && isHeld(m) {
continue
}
if m == "" {
a, b = withoutHeld(a, isHeld), withoutHeld(b, isHeld)
}
da, errA := delivery.ComposedDigestOf(a)
db, errB := delivery.ComposedDigestOf(b)
if inA == inB && errA == nil && errB == nil && da == db {
continue
}
if m == "" {
own = true
continue
}
dependent = append(dependent, m)
}
return dependent, own
}
// withoutHeld is the node's own composition c without the resources of a held module (its id naming it) and without
// its filled paths, for comparing the rest.
func withoutHeld(c delivery.Composed, isHeld func(string) bool) delivery.Composed {
out := c
out.Filled = nil
out.Resources = nil
for _, raw := range c.Resources {
var head struct {
ID string `json:"id"`
Module string `json:"module"`
}
_ = json.Unmarshal(raw, &head)
owner := head.Module
if owner == "" {
owner, _ = declaration.ModuleGuess(head.ID)
}
if owner != "" && isHeld(owner) {
continue
}
out.Resources = append(out.Resources, raw)
}
return out
}
// dependents is, of the modules placed on node and not held, those that read what man says to them: each that provides
// what man contributes to (the proxy its routes), and, when man listens on a port, each that holds the filter (the
// openings).
func dependents(in compose.Input, node string, man catalogue.Manifest, failed map[string]string) []string {
to := map[string]bool{}
for p := range man.Contributes {
to[p] = true
}
for p := range man.ContributesMany {
to[p] = true
}
listens := len(man.Listens) > 0
if len(to) == 0 && !listens {
return nil
}
var out []string
for _, m := range assignedOn(in, node) {
if _, held := failed[m]; held || m == man.Module {
continue
}
pm, ok := manifestOn(in, node, m)
if !ok {
continue
}
if listens && pm.Filtering != nil {
out = append(out, m)
continue
}
for _, o := range pm.Provides {
if to[o.Name] {
out = append(out, m)
break
}
}
}
return out
}
// assignedOn is the modules placed on node, in placement order. // assignedOn is the modules placed on node, in placement order.
func assignedOn(in compose.Input, node string) []string { func assignedOn(in compose.Input, node string) []string {
@@ -546,18 +400,14 @@ func manifestOn(in compose.Input, node, module string) (catalogue.Manifest, bool
return man, ok return man, ok
} }
// culprit is the module placed on node whose composition the error is of: the first, in placement order, whose stub // culprit is the module placed on node whose composition the error is of, and what of its offer to leave out: the
// (compose.Stub) makes the composition in mode succeed or fail otherwise; empty when none does (the error is not one // first, in placement order, whose stub (compose.Stub) makes the composition in mode succeed or fail otherwise (leave
// module's). Modules already refused are passed over. A module whose stub still fails the same is tried bare (its stub // ""); else the first whose stub without what it contributes to one provision does (leave that provision: de-spiegel's
// contributing nothing, bare true): a refusal of what it contributes is its own, which the stub keeps (issue 1126, // routes, its names unset, and not its smtp account, issue 1960); else the first whose bare stub does (leaveAll: its
// seed-8: de-spiegel's route names on novox, an unset required value, refused the whole node). // stub contributing nothing, its own secret never issued, issue 1126 seed-8). Empty when none does (the error is not
func culprit(in compose.Input, node string, fps map[string]string, mode compose.Mode, failed map[string]string, err error) (string, bool) { // one module's). Modules already refused are passed over for their stub.
var assigned []string func culprit(in compose.Input, node string, fps map[string]string, mode compose.Mode, failed map[string]string, err error) (string, string) {
for _, n := range in.Nodes { assigned := assignedOn(in, node)
if n.Name == node {
assigned = n.Assigned
}
}
// Only the modules the error names: the words of a module's refusal name it. Each try is a whole composition, so a // Only the modules the error names: the words of a module's refusal name it. Each try is a whole composition, so a
// refusal naming none is no module's, answered at once rather than after a composition per module (issue 1126: ace's // refusal naming none is no module's, answered at once rather than after a composition per module (issue 1126: ace's
// answer took 7m46s). // answer took 7m46s).
@@ -567,65 +417,97 @@ func culprit(in compose.Input, node string, fps map[string]string, mode compose.
order = append(order, m) order = append(order, m)
} }
} }
changes := func(m string, bare bool) bool { changes := func(m, leave string) bool {
_, _, _, again := compose.CompositionsIn(stubbed(in, node, m, bare), node, fps, mode) _, _, _, again := compose.CompositionsIn(stubbed(in, node, m, leave), node, fps, mode)
return again == nil || again.Error() != err.Error() return again == nil || again.Error() != err.Error()
} }
for _, m := range order { for _, m := range order {
if _, done := failed[m]; done { if _, done := failed[m]; done {
continue continue
} }
if changes(m, false) { if changes(m, "") {
return m, false return m, ""
}
}
var kept map[string]catalogue.Manifest
for _, n := range in.Nodes {
if n.Name == node {
kept = n.Kept
} }
} }
for _, m := range order { for _, m := range order {
// One already held is tried bare too, once: held for its build, its stub may still refuse (its own secret man, ok := manifestOn(in, node, m)
// never issued, issue 1126). if !ok {
if _, done := failed[m]; done && isBare(kept[m]) {
continue continue
} }
if changes(m, true) { // One already held is tried again too: held for its build, its stub may still refuse (its own secret never
return m, true // issued, issue 1126). A provision named in the error is tried first, each at one more composition.
var to []string
for p := range man.Contributes {
to = append(to, p)
}
for p := range man.ContributesMany {
if _, both := man.Contributes[p]; !both {
to = append(to, p)
}
}
sort.SliceStable(to, func(i, j int) bool {
return strings.Contains(err.Error(), " "+to[i]) && !strings.Contains(err.Error(), " "+to[j])
})
for _, p := range to {
if changes(m, p) {
return m, p
}
}
if isBare(man) {
continue
}
if changes(m, leaveAll) {
return m, leaveAll
} }
} }
return "", false return "", ""
} }
// isBare says a stub contributes nothing and declares no own secret (stubbed, bare). // isBare says a stub contributes nothing and declares no own secret (stubbed, leaveAll).
func isBare(m catalogue.Manifest) bool { func isBare(m catalogue.Manifest) bool {
return len(m.Contributes) == 0 && len(m.ContributesMany) == 0 && len(m.OwnSecrets) == 0 return len(m.Contributes) == 0 && len(m.ContributesMany) == 0 && len(m.OwnSecrets) == 0
} }
// stubbed is in with module held on node as its stub alone (heldAs, nothing of it running), from the manifest node // stubbed is in with module held on node as its stub alone (heldAs, nothing of it running), from the manifest node
// composes it from: a culprit's try. // composes it from, leave left out of its offer: a culprit's try.
func stubbed(in compose.Input, node, module string, bare bool) compose.Input { func stubbed(in compose.Input, node, module, leave string) compose.Input {
man, ok := manifestOn(in, node, module) man, ok := manifestOn(in, node, module)
if !ok { if !ok {
return in return in
} }
return heldAs(in, node, module, man, bare, nil) var left map[string]bool
if leave != "" {
left = map[string]bool{leave: true}
}
return heldAs(in, node, module, man, left, nil)
} }
// heldAs is in with module held on node (compose.Node.Held): composed from man as what runs of it (compose.Carried of // heldAs is in with module held on node (compose.Node.Held): composed from man as what runs of it (compose.Carried of
// runs; its stub, compose.Stub, when runs is nil or holds nothing of it), as node's Kept; bare, contributing nothing to // runs; its stub, compose.Stub, when runs is nil or holds nothing of it), as node's Kept; with what left names out of
// what it requires and declaring no own secret: none of a held module's own composition is applied, so what it has not // its offer: a provision's contributions, or with leaveAll everything it contributes and its own secrets, none of a
// been issued (restic's repository on a node it never ran on) is no refusal of the rest. // held module's own composition being applied, so what it has not been issued (restic's repository on a node it never
func heldAs(in compose.Input, node, module string, man catalogue.Manifest, bare bool, runs *delivery.Composed) compose.Input { // ran on) is no refusal of the rest.
func heldAs(in compose.Input, node, module string, man catalogue.Manifest, left map[string]bool, runs *delivery.Composed) compose.Input {
out := in out := in
out.Nodes = append([]compose.Node(nil), in.Nodes...) out.Nodes = append([]compose.Node(nil), in.Nodes...)
for i := range out.Nodes { for i := range out.Nodes {
if out.Nodes[i].Name != node { if out.Nodes[i].Name != node {
continue continue
} }
if bare { if left[leaveAll] {
man.Contributes, man.ContributesMany, man.OwnSecrets = nil, nil, nil man.Contributes, man.ContributesMany, man.OwnSecrets = nil, nil, nil
} else if len(left) > 0 {
contributes := map[string]map[string]any{}
for p, v := range man.Contributes {
if !left[p] {
contributes[p] = v
}
}
many := maps.Clone(man.ContributesMany)
for p := range left {
delete(many, p)
}
man.Contributes, man.ContributesMany = contributes, many
} }
kept := make(map[string]catalogue.Manifest, len(out.Nodes[i].Kept)+1) kept := make(map[string]catalogue.Manifest, len(out.Nodes[i].Kept)+1)
for k, v := range out.Nodes[i].Kept { for k, v := range out.Nodes[i].Kept {
+89 -31
View File
@@ -104,12 +104,12 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
if part.Verdict != delivery.CandidateRefused || len(part.Refusals) != 2 || len(part.Held) != 1 || part.Held[0].Module != "api" { if part.Verdict != delivery.CandidateRefused || len(part.Refusals) != 2 || len(part.Held) != 1 || part.Held[0].Module != "api" {
t.Errorf("held as today in part: %+v", part) t.Errorf("held as today in part: %+v", part)
} }
// A module held only behind one kept today is kept as today too, saying behind which (novox's mailu behind de-spiegel). // No module is held behind one kept today (issue 1960): mailu's own refusal stays a refusal beside de-spiegel's hold.
if a := (delivery.CandidateAnswer{Refusals: []delivery.ModuleRefused{{Module: "de-spiegel", Why: "a value is unset"}, if a := (delivery.CandidateAnswer{Refusals: []delivery.ModuleRefused{{Module: "de-spiegel", Why: "a value is unset"},
{Module: "mailu", Why: heldBehind("de-spiegel")}, {Module: "proxy", Why: heldBehind("mailu")}, {Module: "web", Why: "no build"}}}); true { {Module: "mailu", Why: "no build"}}}); true {
heldAsToday(&a, map[string]string{"de-spiegel": "runs a build of the old form"}) heldAsToday(&a, map[string]string{"de-spiegel": "runs a build of the old form"})
if len(a.Refusals) != 1 || a.Refusals[0].Module != "web" || len(a.Held) != 3 || a.Held[2].Today != "held behind mailu" { if len(a.Refusals) != 1 || a.Refusals[0].Module != "mailu" || len(a.Held) != 1 {
t.Errorf("held behind a module kept today: %+v", a) t.Errorf("held as today spreads: %+v", a)
} }
} }
@@ -330,23 +330,26 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
// de-spiegel folded, and its manifest at the indexer's commit (e25e1beb, captured) asks four required config values // de-spiegel folded, and its manifest at the indexer's commit (e25e1beb, captured) asks four required config values
// for its route names that nothing sets. novox answered the whole candidate refused ("de-spiegel contributing // for its route names that nothing sets. novox answered the whole candidate refused ("de-spiegel contributing
// praktijk to route says ${setting:praktijk-name} ...": its stub still contributed the route). Only de-spiegel is // praktijk to route says ${setting:praktijk-name} ...": its stub still contributed the route). Only de-spiegel is
// refused, in the config form's words, and proxy, which receives its routes (what it would compose lacks them: review // refused, in the config form's words; the rest is judged. Issue 1960, novox's chain on 2026-10-29: the hold spread
// of the apply, round 1); the rest is judged. // to mailu (its smtp provider) and proxy (its route provider), and from them on. Neither is held now: proxy composes
// without de-spiegel's routes, which do not compose, and mailu with its smtp account, which does.
spiegel, err := os.ReadFile("testdata/de-spiegel-e25e1beb.module.json") spiegel, err := os.ReadFile("testdata/de-spiegel-e25e1beb.module.json")
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
proxy := []byte(`{"module":"proxy","version":"1","capabilities":["systemd"],"provides":[{"name":"route","scope":"mesh"},{"name":"smtp","scope":"mesh"}],` + proxy := []byte(`{"module":"proxy","version":"1","capabilities":["systemd"],"provides":[{"name":"route","scope":"mesh"}],` +
`"resources":[{"id":"service","type":"service","unit":"proxy.service","state":"running","boot":"enabled"}]}`) `"resources":[{"id":"service","type":"service","unit":"proxy.service","state":"running","boot":"enabled"}]}`)
b = mk("lemurs", overlay.Name, "proxy", "de-spiegel") mailu := []byte(`{"module":"mailu","version":"1","capabilities":["systemd"],"provides":[{"name":"smtp","scope":"mesh"}],` +
for m, raw := range map[string][]byte{"de-spiegel": spiegel, "proxy": proxy} { `"resources":[{"id":"service","type":"service","unit":"mailu.service","state":"running","boot":"enabled"}]}`)
b = mk("lemurs", overlay.Name, "mailu", "proxy", "de-spiegel")
for m, raw := range map[string][]byte{"de-spiegel": spiegel, "proxy": proxy, "mailu": mailu} {
b.manifests[delivery.ManifestDigest(raw)] = raw b.manifests[delivery.ManifestDigest(raw)] = raw
b.modules[m] = delivery.Module{Manifest: delivery.ManifestDigest(raw)} b.modules[m] = delivery.Module{Manifest: delivery.ManifestDigest(raw)}
} }
p = b.placements["n1"] p = b.placements["n1"]
p.Decided.Endpoint = "n1.example:51820" p.Decided.Endpoint = "n1.example:51820"
b.placements["n1"] = p b.placements["n1"] = p
fps = map[string]string{"lemurs": fp("e"), "proxy": fp("f")} fps = map[string]string{"lemurs": fp("e"), "proxy": fp("f"), "mailu": fp("d")}
a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b), InOwn: []string{"de-spiegel"}}, fps, nil, "p-h1", 3) a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b), InOwn: []string{"de-spiegel"}}, fps, nil, "p-h1", 3)
refused = nil refused = nil
for _, r := range a.Refusals { for _, r := range a.Refusals {
@@ -355,11 +358,13 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
t.Errorf("de-spiegel is refused in other words than its config form's: %s", r.Why) t.Errorf("de-spiegel is refused in other words than its config form's: %s", r.Why)
} }
} }
if a.Refused != "" || !reflect.DeepEqual(refused, []string{"de-spiegel", "proxy"}) { if a.Refused != "" || !reflect.DeepEqual(refused, []string{"de-spiegel"}) {
t.Errorf("a module's own refusal: %+v, want de-spiegel refused with proxy, which receives its routes, the node not", a) t.Errorf("a module's own refusal: %+v, want de-spiegel refused alone, the node, mailu and proxy not", a)
} }
if !slices.ContainsFunc(a.Added, func(m delivery.ModuleAdded) bool { return m.Module == "lemurs" }) { for _, m := range []string{"lemurs", "mailu", "proxy"} {
t.Errorf("lemurs, which composes, is not judged beside de-spiegel's refusal: %+v", a) if !slices.ContainsFunc(a.Added, func(x delivery.ModuleAdded) bool { return x.Module == m }) {
t.Errorf("%s, which composes, is not judged beside de-spiegel's refusal: %+v", m, a)
}
} }
// (E) Issue 1126, the applying path (review of the seed): the composition novox applies follows the same rule. It // (E) Issue 1126, the applying path (review of the seed): the composition novox applies follows the same rule. It
@@ -393,16 +398,13 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("de-spiegel's own refusal refuses the node's whole composition: %v", err) t.Fatalf("de-spiegel's own refusal refuses the node's whole composition: %v", err)
} }
if len(held) != 2 || !strings.Contains(held["de-spiegel"], "${config:praktijk-name}") || !strings.Contains(held["proxy"], "de-spiegel") { if len(held) != 1 || !strings.Contains(held["de-spiegel"], "${config:praktijk-name}") || !strings.Contains(held["de-spiegel"], "without what it contributes to route") {
t.Errorf("held %v, want de-spiegel in its config form's words, and proxy for its routes", held) t.Errorf("held %v, want de-spiegel alone, in its config form's words, composed without its routes", held)
} }
lemursKey, _ := delivery.ComposedKey("n1", "lemurs", fps["lemurs"]) for _, m := range []string{"lemurs", "mailu", "proxy"} {
proxyKey, _ := delivery.ComposedKey("n1", "proxy", fps["proxy"]) if key, _ := delivery.ComposedKey("n1", m, fps[m]); got[key].Module != m {
if _, ok := got[lemursKey]; !ok { t.Errorf("%s is not composed beside de-spiegel's hold: %v", m, sortedKeys(got))
t.Errorf("lemurs, which composes, is not composed beside de-spiegel's refusal: %v", sortedKeys(got)) }
}
if _, ok := got[proxyKey]; ok {
t.Errorf("proxy is composed without the routes of held de-spiegel: %v", sortedKeys(got))
} }
carries := func(c delivery.Composed, what string) { carries := func(c delivery.Composed, what string) {
t.Helper() t.Helper()
@@ -433,8 +435,8 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
// (G) A held module that runs a composition of its own (review of the apply, round 1, finding 2): what the others // (G) A held module that runs a composition of its own (review of the apply, round 1, finding 2): what the others
// compose of it is what runs, never its stub's. site listens on 8080, placed at the machine's 20005, and its own // compose of it is what runs, never its stub's. site listens on 8080, placed at the machine's 20005, and its own
// secret was never issued here: held, the filter must not open 8080 and drop the 20005 that runs. A module whose // secret was never issued here: held, the filter must not open 8080 and drop the 20005 that runs. The filter, which
// composition depends on how site is held (here the filter) is held with it. // reads what site listens on, composes with it (issue 1960: a hold never spreads).
site := []byte(`{"module":"site","version":"1","capabilities":["container-runtime"],` + site := []byte(`{"module":"site","version":"1","capabilities":["container-runtime"],` +
`"listens":[{"port":8080,"from":"mesh","why":"the site"}],"own-secrets":{"key":"${dir:state}/key"},` + `"listens":[{"port":8080,"from":"mesh","why":"the site"}],"own-secrets":{"key":"${dir:state}/key"},` +
`"resources":[{"id":"state","type":"directory","path":"/var/lib/site","mode":"0700"},` + `"resources":[{"id":"state","type":"directory","path":"/var/lib/site","mode":"0700"},` +
@@ -460,8 +462,12 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
json.RawMessage(`{"id":"site.state","module":"site","type":"directory","path":"/var/lib/site","mode":"0700"}`), json.RawMessage(`{"id":"site.state","module":"site","type":"directory","path":"/var/lib/site","mode":"0700"}`),
json.RawMessage(`{"id":"site.server","module":"site","type":"container","name":"site","image":"registry.example/site@sha256:` + strings.Repeat("c", 64) + `","ports":["10.77.0.1:20005:8080"]}`)}} json.RawMessage(`{"id":"site.server","module":"site","type":"container","name":"site","image":"registry.example/site@sha256:` + strings.Repeat("c", 64) + `","ports":["10.77.0.1:20005:8080"]}`)}}
got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", Running: []delivery.Composed{siteRuns}}, fps) got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", Running: []delivery.Composed{siteRuns}}, fps)
if err != nil || held["site"] == "" || held["nft"] == "" { nftKey, _ := delivery.ComposedKey("n1", "nft", fps["nft"])
t.Fatalf("site, its own secret never issued: %v, held %v; want site held, and the filter, which depends on it", err, held) if err != nil || held["site"] == "" || held["nft"] != "" || got[nftKey].Module != "nft" {
t.Fatalf("site, its own secret never issued: %v, held %v; want site held alone, and the filter composed", err, held)
}
if rules, _ := json.Marshal(got[nftKey].Resources); !strings.Contains(string(rules), "dport 20005") {
t.Errorf("the filter drops held site's opening where it runs (20005): %s", rules)
} }
for key, c := range got { for key, c := range got {
for _, raw := range c.Resources { for _, raw := range c.Resources {
@@ -470,10 +476,56 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
} }
} }
} }
// (G2) Issue 1126, the spreading hold: a held module is no dependence of its neighbours. site, held, provides page, and
// app requires and binds it: app composes against what runs of site and is not held (novox's seed held every postgres
// consumer behind de-spiegel's hold, "what it composes of …, held, is not what runs of them").
provider := []byte(strings.Replace(string(site), `"listens":`, `"provides":[{"name":"page","scope":"node"}],"serves":{"page":{"port":8080}},"listens":`, 1))
app := []byte(`{"module":"app","version":"1","capabilities":["systemd"],"requires":["page"],"binds":{"page":"/etc/app/page.json"},` +
`"resources":[{"id":"service","type":"service","unit":"app.service","state":"running","boot":"enabled"}]}`)
b2 := mk("lemurs", overlay.Name, "site", "nft", "app")
for m, raw := range map[string][]byte{"site": provider, "nft": nft, "app": app} {
b2.manifests[delivery.ManifestDigest(raw)] = raw
b2.modules[m] = delivery.Module{Manifest: delivery.ManifestDigest(raw)}
}
p2 := b2.placements["n1"]
p2.Decided.Endpoint = "n1.example:51820"
for i := range p2.Modules {
if p2.Modules[i].Module == "site" {
p2.Modules[i].Ports = map[string]int{"8080": 20005}
}
}
b2.placements["n1"] = p2
b2.records["n1"] = rec
b2.held = true
fps["app"] = fp("9")
got, _, held, err = Compose(ctx, b2, Mesh{}, Scope{Node: "n1", Running: []delivery.Composed{siteRuns}}, fps)
appKey, _ := delivery.ComposedKey("n1", "app", fps["app"])
if _, composed := got[appKey]; err != nil || held["site"] == "" || held["app"] != "" || !composed {
t.Errorf("a consumer of held site: %v, held %v; want site held, and app composed against what runs of it", err, held)
}
if bound, _ := json.Marshal(got[appKey].Resources); !strings.Contains(string(bound), "20005") {
t.Errorf("app's binding does not reach held site where it runs (20005): %s", bound)
}
delete(fps, "app")
// A held module's offer is left out by provision, never whole when one part composes (issue 1960): de-spiegel's
// routes, its names unset, leave the proxy without them, and its smtp account stays with mailu.
withProxy := compose.Input{Catalogue: map[string]catalogue.Manifest{},
Nodes: []compose.Node{{Name: "n1", Assigned: []string{"feeder"}}}}
feeder := catalogue.Manifest{Module: "feeder", Contributes: map[string]map[string]any{"route": {"name": "x"}, "smtp": {"account": "feeder"}},
OwnSecrets: catalogue.OwnSecrets{"key": {Path: "/k"}}}
for _, row := range []struct {
left map[string]bool
route, smtp, own bool
}{{nil, true, true, true}, {map[string]bool{"route": true}, false, true, true}, {map[string]bool{leaveAll: true}, false, false, false}} {
k := heldAs(withProxy, "n1", "feeder", feeder, row.left, nil).Nodes[0].Kept["feeder"]
if _, r := k.Contributes["route"]; r != row.route || (k.Contributes["smtp"] != nil) != row.smtp || (len(k.OwnSecrets) > 0) != row.own {
t.Errorf("held leaving out %v offers %+v, own secrets %v", row.left, k.Contributes, k.OwnSecrets)
}
}
// (H) Review of the apply, round 2: a held module is composed from its new manifest, so what the others read of its // (H) Review of the apply, round 2: a held module is composed from its new manifest, so what the others read of its
// manifest (its listens, what it contributes) may not be what runs. site's new manifest moves it to 9090 while the // manifest (its listens, what it contributes) may not be what runs. site's new manifest moves it to 9090 while the
// build that runs listens on 8080: the filter, the holder of what site listens on, is held with it, and nothing // build that runs (7) listens on 8080. The filter composes with what site last offered, its manifest at the build
// applied opens 9090. // that runs (issue 1960: a hold never spreads), so it opens 8080 and nothing applied opens 9090.
moved := []byte(strings.Replace(string(site), `"port":8080`, `"port":9090`, 1)) moved := []byte(strings.Replace(string(site), `"port":8080`, `"port":9090`, 1))
b.manifests[delivery.ManifestDigest(moved)] = moved b.manifests[delivery.ManifestDigest(moved)] = moved
b.modules["site"] = delivery.Module{Manifest: delivery.ManifestDigest(moved)} b.modules["site"] = delivery.Module{Manifest: delivery.ManifestDigest(moved)}
@@ -482,9 +534,15 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
} }
b.placements["n1"] = p b.placements["n1"] = p
siteRuns.Resources[1] = json.RawMessage(`{"id":"site.server","module":"site","type":"container","name":"site","image":"registry.example/site@sha256:` + strings.Repeat("c", 64) + `","ports":["8080"]}`) siteRuns.Resources[1] = json.RawMessage(`{"id":"site.server","module":"site","type":"container","name":"site","image":"registry.example/site@sha256:` + strings.Repeat("c", 64) + `","ports":["8080"]}`)
siteRuns.FP = fp("7")
b.manifests[delivery.ManifestDigest(site)] = site
b.builds["site "+fp("7")] = &delivery.BuilderBuild{State: delivery.BuildDone, Manifest: delivery.ManifestDigest(site)}
got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", Running: []delivery.Composed{siteRuns}}, fps) got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", Running: []delivery.Composed{siteRuns}}, fps)
if err != nil || held["site"] == "" || held["nft"] == "" { if err != nil || held["site"] == "" || held["nft"] != "" {
t.Errorf("site moved to 9090 in its new manifest: %v, held %v; want site held, and the filter with it", err, held) t.Errorf("site moved to 9090 in its new manifest: %v, held %v; want site held alone", err, held)
}
if rules, _ := json.Marshal(got[nftKey].Resources); !strings.Contains(string(rules), "dport 8080") {
t.Errorf("the filter drops what held site last offered (8080): %s", rules)
} }
for key, c := range got { for key, c := range got {
for _, raw := range c.Resources { for _, raw := range c.Resources {
+41 -7
View File
@@ -240,12 +240,7 @@ func input(ctx context.Context, r Reader, mesh Mesh, s Scope, refuse func(module
in.Authority = delivery.AuthorityBundle(in.Authority, vault) in.Authority = delivery.AuthorityBundle(in.Authority, vault)
} }
} }
parse := func(raw []byte) (catalogue.Manifest, error) { parse := parser(mesh)
if mesh.Predefined != nil {
return catalogue.ParseManifestWith(raw, mesh.Predefined)
}
return catalogue.ParseManifestForInputs(raw)
}
read := func(module, digest string) (catalogue.Manifest, error) { read := func(module, digest string) (catalogue.Manifest, error) {
raw, err := r.Manifest(ctx, digest) raw, err := r.Manifest(ctx, digest)
if err != nil { if err != nil {
@@ -313,6 +308,8 @@ func input(ctx context.Context, r Reader, mesh Mesh, s Scope, refuse func(module
} }
} }
if node == s.Node { if node == s.Node {
// What its runtime serves as it runs: what a module composed as what runs of it contributes there (issue 1126).
n.RunsTools = compose.ToolsRunning(s.Running)
for _, module := range s.InOwn { for _, module := range s.InOwn {
if n.InOwn == nil { if n.InOwn == nil {
n.InOwn = map[string]bool{} n.InOwn = map[string]bool{}
@@ -425,10 +422,47 @@ func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]s
return nil, nil, held, fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err) return nil, nil, held, fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err)
} }
} }
out, skipped, _, err = composeEach(in, s.Node, fps, true, s.Running, held) out, skipped, _, err = composeEach(in, s.Node, fps, true, s.Running, held, ranOf(ctx, r, mesh, s.Running, fps))
return out, skipped, held, err return out, skipped, held, err
} }
// parser reads a manifest as the indexer does, against the mesh's predefined keys when given.
func parser(mesh Mesh) func([]byte) (catalogue.Manifest, error) {
return func(raw []byte) (catalogue.Manifest, error) {
if mesh.Predefined != nil {
return catalogue.ParseManifestWith(raw, mesh.Predefined)
}
return catalogue.ParseManifestForInputs(raw)
}
}
// ranOf is what a module last offered the others, read when it is held (composeEach, issue 1960): its manifest at the
// build its own composition runs at (running), where that is another build than the one it is composed at (fps) and
// the builder holds it done. False where nothing is known of it (it runs no composition of its own, it runs the build
// it is composed at, or its build is unread): the manifest it is composed from is its offer then. Only what it says to
// others is taken of it, which needs no take-in of its artifacts.
func ranOf(ctx context.Context, r Reader, mesh Mesh, running []delivery.Composed, fps map[string]string) func(string) (catalogue.Manifest, bool) {
parse := parser(mesh)
return func(module string) (catalogue.Manifest, bool) {
for _, c := range running {
if c.Module != module || c.FP == "" || c.FP == fps[module] {
continue
}
b, err := r.Build(ctx, module, c.FP)
if err != nil || b == nil || b.State != delivery.BuildDone || b.Manifest == "" {
return catalogue.Manifest{}, false
}
raw, err := r.Manifest(ctx, b.Manifest)
if err != nil {
return catalogue.Manifest{}, false
}
man, err := parse(raw)
return man, err == nil
}
return catalogue.Manifest{}, false
}
}
// fromBuilds puts in the built manifest of each module placed on the composing node, at the trunk's FP2, and of each // fromBuilds puts in the built manifest of each module placed on the composing node, at the trunk's FP2, and of each
// module it keeps at another build (Scope.KeptAt, into its node's Kept), read from the builder's done key and resolved // module it keeps at another build (Scope.KeptAt, into its node's Kept), read from the builder's done key and resolved
// by the one take-in rule (takein.Manifest), a templates archive read in the artifact store as the node reaches it // by the one take-in rule (takein.Manifest), a templates archive read in the artifact store as the node reaches it
+14 -2
View File
@@ -45,6 +45,8 @@ type bus struct {
served map[string]map[string]any served map[string]map[string]any
servedRead []string servedRead []string
buildsRead []string buildsRead []string
// held answers every credential asked as held, sealed to the consumer.
held bool
} }
func (b *bus) Build(_ context.Context, module, fp string) (*delivery.BuilderBuild, error) { func (b *bus) Build(_ context.Context, module, fp string) (*delivery.BuilderBuild, error) {
@@ -89,7 +91,7 @@ func (b *bus) OwnSecrets(context.Context, string) ([]compose.OwnSecretRow, error
return b.own, b.ownErr return b.own, b.ownErr
} }
func (b *bus) Credentials(_ context.Context, node string, asConsumer, _ []compose.CredentialKey) ([]compose.Credential, []compose.Issued, error) { func (b *bus) Credentials(_ context.Context, node string, asConsumer, asProvider []compose.CredentialKey) ([]compose.Credential, []compose.Issued, error) {
b.asked = append(b.asked, asConsumer...) b.asked = append(b.asked, asConsumer...)
for _, k := range asConsumer { for _, k := range asConsumer {
placed := false placed := false
@@ -100,7 +102,17 @@ func (b *bus) Credentials(_ context.Context, node string, asConsumer, _ []compos
return nil, nil, fmt.Errorf("%s is not on %s: neither assigned there nor required by a module there", k.ConsumerModule, node) return nil, nil, fmt.Errorf("%s is not on %s: neither assigned there nor required by a module there", k.ConsumerModule, node)
} }
} }
return nil, nil, nil var out []compose.Credential
var issued []compose.Issued
for _, k := range append(append([]compose.CredentialKey{}, asConsumer...), asProvider...) {
if b.held {
out = append(out, compose.Credential{Provision: k.Provision, Consumer: k.Consumer, ConsumerModule: k.ConsumerModule,
Provider: k.Provider, Local: k.Local, State: compose.CredentialHeld, Sealed: `"box"`})
issued = append(issued, compose.Issued{Provision: k.Provision, Consumer: k.Consumer, ConsumerModule: k.ConsumerModule,
Provider: k.Provider, Local: k.Local, ForProvider: `"box"`})
}
}
return out, issued, nil
} }
func (b *bus) Values(_ context.Context, node, module string) (map[string]any, error) { func (b *bus) Values(_ context.Context, node, module string) (map[string]any, error) {
+6
View File
@@ -210,6 +210,12 @@ type Node struct {
// (RunsOwn), folded or leaving the fold, is carried there as it runs, whatever its manifest names, so the node's own // (RunsOwn), folded or leaving the fold, is carried there as it runs, whatever its manifest names, so the node's own
// never drops it. // never drops it.
Held map[string]bool Held map[string]bool
// RunsTools is, by module, the tools bundles the node's runtime serves of it as it runs (ToolsRunning of what the
// node runs); nil when not known. A module composed as what runs of it (its Stub or Carried: held, or folded at a
// build of the old form) contributes these to the runtime, never its manifest's: a manifest no build resolved has no
// bundles, and a held module's new build's bundles are not delivered (issue 1126, 2026-10-29: shanks's runtime lost
// the folded systemd-resolved, novox's the folded nats).
RunsTools map[string][]catalogue.Bundle
} }
// Place is where a node is, as its placement says (inv.Overlay). // Place is where a node is, as its placement says (inv.Overlay).
+13 -4
View File
@@ -3,6 +3,7 @@ package delivery
import ( import (
"bytes" "bytes"
"encoding/json" "encoding/json"
"fmt"
"path" "path"
"sort" "sort"
"strings" "strings"
@@ -84,9 +85,6 @@ func judged(from, to Composed) (Judgement, string) {
if ids := changedOf(before, after, func(kind string) bool { return kind == string(declaration.TypeContainer) }); len(ids) > 0 { if ids := changedOf(before, after, func(kind string) bool { return kind == string(declaration.TypeContainer) }); len(ids) > 0 {
return JudgedRecreates, "it recreates " + strings.Join(ids, ", ") return JudgedRecreates, "it recreates " + strings.Join(ids, ", ")
} }
if from.Data != to.Data {
return JudgedMovesData, "its data declaration changes"
}
places := dataPlaces(before, after) places := dataPlaces(before, after)
held := func(kind string) bool { held := func(kind string) bool {
return kind == string(declaration.TypeDirectory) || kind == string(declaration.TypeFile) return kind == string(declaration.TypeDirectory) || kind == string(declaration.TypeFile)
@@ -104,9 +102,20 @@ func judged(from, to Composed) (Judgement, string) {
if to.Data || from.Data { if to.Data || from.Data {
moved = append(moved, changedOf(before, after, func(kind string) bool { return kind == string(declaration.TypeProcess) })...) moved = append(moved, changedOf(before, after, func(kind string) bool { return kind == string(declaration.TypeProcess) })...)
} }
// A data flag that differs is no move of data by itself: data moves only when a data item does (issue 1126, 2026-10-29:
// a running composition read back from the kept declaration carries no flag, and mongodb's toolchain move asked a yes for
// moving its data). It is said loudly in the words, and never hides an item that moved.
if len(moved) > 0 { if len(moved) > 0 {
sort.Strings(moved) sort.Strings(moved)
return JudgedMovesData, "it changes where its data is kept: " + strings.Join(moved, ", ") words := "it changes where its data is kept: " + strings.Join(moved, ", ")
if from.Data != to.Data {
words += fmt.Sprintf("; its data flag differs too (running %v, composed %v)", from.Data, to.Data)
}
return JudgedMovesData, words
}
if from.Data != to.Data {
return JudgedNeither, fmt.Sprintf("it recreates no container and moves no data; WARNING: its data flag differs "+
"(running %v, composed %v) while no data item does, so one side's flag is wrong", from.Data, to.Data)
} }
return JudgedNeither, "it recreates no container and moves no data" return JudgedNeither, "it recreates no container and moves no data"
} }
+5 -1
View File
@@ -48,7 +48,8 @@ func TestJudgeMove(t *testing.T) {
{"a container gone", mail('a', true, container, spool), mail('b', true, spool), JudgedRecreates, false}, {"a container gone", mail('a', true, container, spool), mail('b', true, spool), JudgedRecreates, false},
{"its spool moved", mail('a', true, container, spool), mail('b', true, container, spoolMoved), JudgedMovesData, false}, {"its spool moved", mail('a', true, container, spool), mail('b', true, container, spoolMoved), JudgedMovesData, false},
{"a file in its spool", mail('a', true, container, spool, config), mail('b', true, container, spool, config2), JudgedMovesData, false}, {"a file in its spool", mail('a', true, container, spool, config), mail('b', true, container, spool, config2), JudgedMovesData, false},
{"its data declaration", mail('a', false, container, spool), mail('b', true, container, spool), JudgedMovesData, false}, {"its data flag alone", mail('a', false, container, spool), mail('b', true, container, spool), JudgedNeither, false},
{"its data flag and its spool", mail('a', false, container, spool), mail('b', true, container, spoolMoved), JudgedMovesData, false},
{"a process of a module with data", mail('a', true, spool, process), mail('b', true, spool, process2), JudgedMovesData, false}, {"a process of a module with data", mail('a', true, spool, process), mail('b', true, spool, process2), JudgedMovesData, false},
{"a process of a module without", mail('a', false, process), mail('b', false, process2), JudgedNeither, false}, {"a process of a module without", mail('a', false, process), mail('b', false, process2), JudgedNeither, false},
{"its tools only", mail('a', true, container, spool, tools), mail('b', true, container, spool, tools2), JudgedNeither, false}, {"its tools only", mail('a', true, container, spool, tools), mail('b', true, container, spool, tools2), JudgedNeither, false},
@@ -61,6 +62,9 @@ func TestJudgeMove(t *testing.T) {
if got.Judgement != c.want || got.UserListOnly != c.userList || got.Words == "" { if got.Judgement != c.want || got.UserListOnly != c.userList || got.Words == "" {
t.Errorf("%s: %+v, want %s (user list only %v)", c.name, got, c.want, c.userList) t.Errorf("%s: %+v, want %s (user list only %v)", c.name, got, c.want, c.userList)
} }
if strings.Contains(c.name, "data flag alone") && !strings.Contains(got.Words, "WARNING: its data flag differs") {
t.Errorf("%s: the flag's difference is not said: %q", c.name, got.Words)
}
} }
if got := JudgeMove(nil, Composed{Node: "ace"}); got.Judgement != JudgedNeither { if got := JudgeMove(nil, Composed{Node: "ace"}); got.Judgement != JudgedNeither {
t.Errorf("the node's own composition: %+v", got) t.Errorf("the node's own composition: %+v", got)