Compare commits

...
Author SHA1 Message Date
jschoubben 2a86701891 buildstep: keep what one command of an ask writes into the caches for its later commands
Every command of a check ran over the shared Go build cache beneath a throwaway
tmpfs layer of its own, so the digests, the vet, the unit tests and the integration
compile of one ask each compiled every package again. With Isolation.CacheLayer a
command writes into a layer on disk of its own and Isolated moves it into the ask's
kept layer when it ends; later commands read the kept layer above the shared cache,
which stays unchanged (ADR 0541 §2, ADR 0448 §3). Empty CacheLayer keeps today's
throwaway layer. Ported from feat/try-speed S2 (issue 1921).
2026-10-29 00:51:26 +01:00
mesh-tools d4c61ee8ff Merge pull request 'The seed leaves a module as it runs today when it is held, failed or at an old-form build, and the modules held only behind it too (issue 1126)' (#127) from fix/1126-seed-oldbuild into main 2026-10-28 23:05:04 +00:00
jschoubben 76f7c4d223 A stub keeps its items and their templates, so a held module still contributes to other modules' collections; it drops only the templates for its own files (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@d4c61ee8ffd0
mesh/delivery-group group fix/1126-seed-oldbuild merged: every member is merged
mesh/check pass 76f7c4d2238f on d747c1128e42 | unit 3018 run, 270 kept, 36 skipped | integration 3039 run, 302 kept, 29 skipped | over bound: unit (sl…
2026-10-28 23:56:33 +01:00
jschoubben f609be7864 Keep as today what a node keeps today and what is held only behind it, saying how; a stub hands and collects nothing of its own (issue 1126) 2026-10-28 23:49:22 +01:00
mesh-tools d747c1128e Merge pull request 'L3: the vault mints bus accounts, the grants part and the node seat node-engine (issue 774, ADR 0561), rebased onto L2' (#126) from feat/774-vault-mints-sdk into main 2026-10-28 22:27:36 +00:00
mesh-tools f9afe32888 Merge pull request 'A node's answer holds as today what it holds today, and the seed carries its nodes' differences, named (issue 1126)' (#125) from fix/1126-seed-sdk into main 2026-10-28 21:59:48 +00:00
jschoubben 8eafc0c5ce The placement reader names the seat mesh-placement's state, held by the assigner, as L2 moved it (issue 774, ADR 0567)
mesh/delivery-group group feat/774-vault-mints-l2 checking: the checker's verdict is out of date: novox/mesh-delivery@f6df07605dd0 was judged on trunk 5b2e5bc5…
mesh/delivery merged: source.main moved to novox/mesh-sdk@d747c1128e42
mesh/check checking 8eafc0c5ce3c on f9afe328881f | novox/mesh-sdk#123 in group feat/774-vault-mints-l2
2026-10-28 21:54:04 +01:00
jschoubben 50de0a5a71 The node seat's acts reads one act by its id (issue 774, ADR 0563 §9) 2026-10-28 21:54:04 +01:00
jschoubben 2323ba19db The node-engine's declared state acts: created by the grants part as a placed module's, its manifest placed on every node, and written by each node's engine under its own keys alone (issue 774, ADR 0563 §2, §3) 2026-10-28 21:54:04 +01:00
jschoubben 63607ba169 The vault's secret-ask is said to answer the callers its manifest names (issue 774, ADR 0563 §8) 2026-10-28 21:54:04 +01:00
jschoubben 82c54870ed A module's trusted list of callers in the SDK, so the vault's secret-ask answers the callers its manifest names as the runtime proved them (issue 774, ADR 0563 §8) 2026-10-28 21:54:04 +01:00
jschoubben 79231a9e6c A node's engine answers its own node's node-engine verbs, says the seat's events and asks the operator as node-engine-<node>, and no other account answers them (issue 774, ADR 0563 §1, §3, §6) 2026-10-28 21:54:04 +01:00
jschoubben 222d605f6f The node seat node-engine: held by every node's engine, its six verbs and three events, and no module may claim it (issue 774, ADR 0563 §1) 2026-10-28 21:54:04 +01:00
jschoubben 3a8997c58a Bundle the mesh's authorities by certificate, and check a node's certificate against every one a bundle holds (issue 774, ADR 0550 §2) 2026-10-28 21:54:04 +01:00
jschoubben d6db3f38e6 The vault signs: the contract of its state signers, its verb sign and its event signed, a node's grants to ask it and to read what it trusts, and the authority composed as a bundle (issue 774, ADR 0550 §2) 2026-10-28 21:54:04 +01:00
jschoubben 18a47ac3c7 Placed: a module in the first revision heard of a node's key stands since before it, so the seed's placement does not leave out what already ran (issue 774) 2026-10-28 21:54:04 +01:00
jschoubben 1386219d4d One reader of the catalogue's placement and the toolchain gate over it, in place of the three copies over the controller's assignments (issue 774, ADR 0549 §3, issue 1940); the assigner may read the lease (ADR 0549 §5) 2026-10-28 21:54:04 +01:00
jschoubben 10d4d2f845 Checker seat test: the try replaces both shell commands (issue 774) 2026-10-28 21:54:04 +01:00
jschoubben b3100c55a1 Checker seat: the try's verb also replaces docker run postgres, as the controller's try did (ADR 0547 §10, issue 774) 2026-10-28 21:54:04 +01:00
jschoubben 0062bdc4a6 buscompose: the assigner may read the controller's lease, to choose where grants are read from (issue 774) 2026-10-28 21:54:04 +01:00
jschoubben a1a688b19e delivery: the grants part's grant keys and the source the assigner reads (a1; issue 774)
Why: the grants part writes grants.<module>.<fp> in shadow beside the controller's; the assigner switches source only when the lease is handed away.
2026-10-28 21:54:04 +01:00
jschoubben 54b17164ef Checker seat test: fold into the closed-set seats test, no new test function (issue 774) 2026-10-28 21:54:04 +01:00
jschoubben 3ed9a0e342 Checker seat: the try's end by hand is a verb too (issue 774) 2026-10-28 21:54:04 +01:00
jschoubben 9b5657a3d6 Seat mesh-delivery-checker: the try and its read are verbs of the checker's own seat, replacing sh merge-check.sh (ADR 0547 §10, issue 774) 2026-10-28 21:54:04 +01:00
jschoubben 1d4637e46b Name the vault's verb issue-account on its seat, so its holder may serve it (issue 774, ADR 0561 §9) 2026-10-28 21:54:04 +01:00
jschoubben fefa8b88ea Say when the lease is handed away, let the vault and the grants part read it, and let the grants part create the consumers an account needs, so both turn on at the lease write alone (issue 774, ADR 0561 §3, §6) 2026-10-28 21:54:04 +01:00
jschoubben 8d9b3f779d Build a minted account's value as the controller does, byte for byte, so a runtime reads the vault's unchanged (issue 774, ADR 0561 §2) 2026-10-28 21:54:04 +01:00
jschoubben da506bb3b8 Say which bus accounts the grants part wants, in a state only the vault reads, and what the vault says when it mints one, so the vault can mint from the grants part's word (issue 774, ADR 0561 §2) 2026-10-28 21:54:04 +01:00
41 changed files with 2409 additions and 53 deletions
+5
View File
@@ -571,3 +571,8 @@ type ProofAnswer struct {
Accepted bool `json:"accepted"` Accepted bool `json:"accepted"`
Words string `json:"words"` Words string `json:"words"`
} }
// EngineAsker is the asker name a node's engine asks the operator under (novox/hq ADR 0563 §3, which changes ADR 0259
// §2 to admit a node's engine as an asker): `node-engine-<node>`, the last token of the by-caller subjects granted to that
// node's engine account alone, so no engine acts on another node's warrant.
func EngineAsker(node string) string { return "node-engine-" + node }
+114 -10
View File
@@ -3,10 +3,13 @@ package buildstep
import ( import (
"context" "context"
"fmt" "fmt"
"io/fs"
"os" "os"
"path" "path"
"path/filepath" "path/filepath"
"strconv"
"strings" "strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/forgekey" "git.novox.be/novox/mesh-sdk/go/forgekey"
) )
@@ -29,16 +32,25 @@ const Bubblewrap = "bwrap"
// - the Workspace and Writable directories read-write, the ReadOnly ones (the toolchain, the modules the fetch step // - the Workspace and Writable directories read-write, the ReadOnly ones (the toolchain, the modules the fetch step
// downloaded) read-only; // downloaded) read-only;
// - the shared warm Caches read-only under a throwaway writable layer (S18), or read-write for a trunk build alone // - the shared warm Caches read-only under a throwaway writable layer (S18), or read-write for a trunk build alone
// (WriteCaches), so a candidate's code never changes what a later build reads. // (WriteCaches), so a candidate's code never changes what a later build reads;
// - with a CacheLayer, what one command writes into the caches kept for the later commands of the same ask (issue 1921,
// the try speed-up): the vet, the unit tests and the integration compile of one ask compile each package once.
type Isolation struct { type Isolation struct {
Workspace string Workspace string
Writable []string Writable []string
ReadOnly []string ReadOnly []string
Caches []string Caches []string
WriteCaches bool WriteCaches bool
Env []string // CacheLayer is a directory of the ask's own, in its workspace and gone with it: each command writes the caches into
// a layer of its own beneath it (overlayfs refuses an upper layer two mounts share, and an ask's commands run side by
// side), which Isolated moves into the ask's kept layer when the command ends; every later command reads the kept
// layer above the shared cache. The shared cache stays unchanged. Empty: what a command writes is thrown away.
CacheLayer string
Env []string
// Hosts is the file bound as /etc/hosts; Isolated writes one naming only the loopback. // Hosts is the file bound as /etc/hosts; Isolated writes one naming only the loopback.
Hosts string Hosts string
// run is the directory of this command's own writable layers under CacheLayer; Isolated makes one per command.
run string
} }
// etcNeeded is what of /etc a program in the step may read: who the users are, where the libraries are, the // etcNeeded is what of /etc a program in the step may read: who the users are, where the libraries are, the
@@ -91,7 +103,7 @@ func (iso Isolation) command(env []string, name string, args ...string) ([]strin
for _, group := range []struct { for _, group := range []struct {
what string what string
dirs []string dirs []string
}{{"writable directory", iso.Writable}, {"read-only directory", iso.ReadOnly}, {"cache", iso.Caches}} { }{{"writable directory", iso.Writable}, {"read-only directory", iso.ReadOnly}, {"cache", iso.Caches}, {"cache layer", layerDirs(iso.CacheLayer)}} {
for _, d := range group.dirs { for _, d := range group.dirs {
if err := checkDir(group.what, d); err != nil { if err := checkDir(group.what, d); err != nil {
return nil, err return nil, err
@@ -113,7 +125,7 @@ func (iso Isolation) command(env []string, name string, args ...string) ([]strin
for _, d := range iso.ReadOnly { for _, d := range iso.ReadOnly {
argv = append(argv, "--ro-bind", d, d) argv = append(argv, "--ro-bind", d, d)
} }
argv = append(argv, cacheArgs(iso.Caches, iso.WriteCaches)...) argv = append(argv, iso.cacheArgs()...)
argv = append(argv, "--bind", iso.Workspace, iso.Workspace) argv = append(argv, "--bind", iso.Workspace, iso.Workspace)
for _, d := range iso.Writable { for _, d := range iso.Writable {
argv = append(argv, "--bind", d, d) argv = append(argv, "--bind", d, d)
@@ -134,19 +146,107 @@ func (iso Isolation) command(env []string, name string, args ...string) ([]strin
// cacheArgs binds the shared warm caches (S18): for a trunk build read-write, so the next build is warm; for any other // cacheArgs binds the shared warm caches (S18): for a trunk build read-write, so the next build is warm; for any other
// read-only beneath an overlay whose writes go to a tmpfs of the step's own, so the step writes as a build does and // read-only beneath an overlay whose writes go to a tmpfs of the step's own, so the step writes as a build does and
// nothing it wrote outlives it (bubblewrap's --tmp-overlay, an overlay mount in the step's user namespace). // nothing it wrote outlives it (bubblewrap's --tmp-overlay, an overlay mount in the step's user namespace). With a
func cacheArgs(caches []string, write bool) []string { // CacheLayer the ask's kept layer lies above the shared cache and the command's writes go to its own layer on disk
// (--overlay), which Isolated keeps for the ask when the command ends.
func (iso Isolation) cacheArgs() []string {
var argv []string var argv []string
for _, c := range caches { for i, c := range iso.Caches {
if write { switch {
case iso.WriteCaches:
argv = append(argv, "--bind", c, c) argv = append(argv, "--bind", c, c)
continue case iso.CacheLayer != "":
upper, work := iso.runLayer(i)
argv = append(argv, "--overlay-src", c, "--overlay-src", keptLayer(iso.CacheLayer, i), "--overlay", upper, work, c)
default:
argv = append(argv, "--overlay-src", c, "--tmp-overlay", c)
} }
argv = append(argv, "--overlay-src", c, "--tmp-overlay", c)
} }
return argv return argv
} }
func layerDirs(layer string) []string {
if layer == "" {
return nil
}
return []string{layer}
}
// keptLayer is where the ask keeps what its commands wrote into the cache i.
func keptLayer(layer string, i int) string { return filepath.Join(layer, "kept", strconv.Itoa(i)) }
// runLayer is the upper and work directory of this command's writes into the cache i. Command, which makes no
// directory, names the layer "run"; Isolated gives each command one of its own.
func (iso Isolation) runLayer(i int) (upper, work string) {
run := iso.run
if run == "" {
run = filepath.Join(iso.CacheLayer, "run")
}
return filepath.Join(run, strconv.Itoa(i), "upper"), filepath.Join(run, strconv.Itoa(i), "work")
}
// layerLocks hold one mutex per cache layer, so two commands ending at once keep their writes one after the other.
var layerLocks sync.Map
// prepareLayer makes the kept layers and a run directory of the command's own under the cache layer.
func (iso *Isolation) prepareLayer() error {
if iso.CacheLayer == "" || iso.WriteCaches || len(iso.Caches) == 0 {
return nil
}
if err := os.MkdirAll(filepath.Join(iso.CacheLayer, "runs"), 0o700); err != nil {
return err
}
run, err := os.MkdirTemp(filepath.Join(iso.CacheLayer, "runs"), "run-")
if err != nil {
return err
}
iso.run = run
for i := range iso.Caches {
upper, work := iso.runLayer(i)
for _, d := range []string{keptLayer(iso.CacheLayer, i), upper, work} {
if err := os.MkdirAll(d, 0o700); err != nil {
return err
}
}
}
return nil
}
// keepLayer moves what the command wrote into the caches into the ask's kept layers, and removes its run directory.
// A file the command wrote replaces the kept one (a cache's newest write wins); a whiteout (a file the command
// deleted) is not kept, since a cache entry deleted by one command is no fault of a later one. A write that cannot be
// kept only means a later command builds it again.
func (iso Isolation) keepLayer() {
if iso.run == "" {
return
}
defer os.RemoveAll(iso.run)
mu, _ := layerLocks.LoadOrStore(iso.CacheLayer, &sync.Mutex{})
mu.(*sync.Mutex).Lock()
defer mu.(*sync.Mutex).Unlock()
for i := range iso.Caches {
upper, _ := iso.runLayer(i)
kept := keptLayer(iso.CacheLayer, i)
filepath.WalkDir(upper, func(p string, d fs.DirEntry, err error) error {
if err != nil || p == upper {
return nil
}
rel, _ := filepath.Rel(upper, p)
to := filepath.Join(kept, rel)
switch {
case d.IsDir():
if fi, err := os.Lstat(to); err == nil && !fi.IsDir() {
os.Remove(to)
}
os.MkdirAll(to, 0o700)
case d.Type().IsRegular():
os.Rename(p, to)
}
return nil
})
}
}
// loopbackHosts is the hosts file of a step: the loopback and nothing else. // loopbackHosts is the hosts file of a step: the loopback and nothing else.
const loopbackHosts = "127.0.0.1 localhost\n::1 localhost\n" const loopbackHosts = "127.0.0.1 localhost\n::1 localhost\n"
@@ -166,6 +266,10 @@ func Isolated(run forgekey.Runner, iso Isolation) forgekey.Runner {
} }
iso := iso iso := iso
iso.Hosts = hosts iso.Hosts = hosts
if err := iso.prepareLayer(); err != nil {
return forgekey.Ran{Status: 1, Err: "the step's cache layer: " + err.Error()}
}
defer iso.keepLayer()
argv, err := iso.command(env, name, args...) argv, err := iso.command(env, name, args...)
if err != nil { if err != nil {
return forgekey.Ran{Status: 1, Err: err.Error()} return forgekey.Ran{Status: 1, Err: err.Error()}
+35 -1
View File
@@ -1,6 +1,9 @@
package buildstep package buildstep
import "testing" import (
"strings"
"testing"
)
// The shared warm caches are read under a throwaway writable layer (novox/hq ADR 0448; issue 1309): a candidate's step // The shared warm caches are read under a throwaway writable layer (novox/hq ADR 0448; issue 1309): a candidate's step
// writes into them as a build does, and nothing it writes outlives it or reaches a later build; only a trunk build // writes into them as a build does, and nothing it writes outlives it or reaches a later build; only a trunk build
@@ -27,4 +30,35 @@ func TestACandidatesStepWritesTheCachesOnlyIntoAThrowawayLayer(t *testing.T) {
t.Errorf("a trunk build does not write the cache %s: %v", c, argv) t.Errorf("a trunk build does not write the cache %s: %v", c, argv)
} }
} }
// 1921/S2: with a cache layer, a command reads the ask's kept layer above the shared cache and writes into a layer
// on disk of the ask's own, never into the shared cache and never into a throwaway tmpfs.
iso = Isolation{Workspace: "/w", Caches: []string{"/cache/gomod", "/cache/gobuild"}, CacheLayer: "/w/cachelayer"}
argv, err = iso.Command("go", "vet")
if err != nil {
t.Fatal(err)
}
for i, c := range iso.Caches {
if !holds(argv, "--overlay-src", c, "--overlay-src", keptLayer(iso.CacheLayer, i), "--overlay") {
t.Errorf("the cache %s is not read beneath the ask's kept layer: %v", c, argv)
}
if up := overlayOf(argv, c); up == "" || !strings.HasPrefix(up, "/w/cachelayer/") {
t.Errorf("the cache %s is written into %q, not a layer of the ask's own: %v", c, up, argv)
}
if holds(argv, "--tmp-overlay", c) || holds(argv, "--bind", c, c) {
t.Errorf("the cache %s is thrown away or written: %v", c, argv)
}
}
if _, err := (Isolation{Workspace: "/w", CacheLayer: "cachelayer"}).Command("go"); err == nil {
t.Error("a relative cache layer was taken")
}
}
// overlayOf answers the upper directory a command line gives the cache c, "" when it gives none.
func overlayOf(argv []string, c string) string {
for i := 0; i+3 < len(argv); i++ {
if argv[i] == "--overlay" && argv[i+3] == c {
return argv[i+1]
}
}
return ""
} }
+19
View File
@@ -63,4 +63,23 @@ func TestARunStepReachesNothingAndLeavesTheCachesUnchanged(t *testing.T) {
if b, err := os.ReadFile(filepath.Join(workspace, "out")); r.Status != 0 || err != nil || string(b) != "built\n" { if b, err := os.ReadFile(filepath.Join(workspace, "out")); r.Status != 0 || err != nil || string(b) != "built\n" {
t.Fatalf("the step's write to its workspace did not stay (status %d, %s%s): %q %v", r.Status, r.Stderr, r.Err, b, err) t.Fatalf("the step's write to its workspace did not stay (status %d, %s%s): %q %v", r.Status, r.Stderr, r.Err, b, err)
} }
// 1921/S2: with the ask's cache layer, what one command writes into the cache the next command of the ask reads,
// above what the shared cache holds, and the shared cache stays unchanged.
asked := Isolated(forgekey.ExecRunnerFor(0), Isolation{Workspace: workspace, Caches: []string{cacheDir},
CacheLayer: filepath.Join(workspace, "cachelayer")})
r = asked(context.Background(), []string{"PATH=/usr/bin:/bin"}, "/bin/sh", "-c", "cat "+filepath.Join(cacheDir, "warm")+" && mkdir -p "+filepath.Join(cacheDir, "ab")+" && echo compiled > "+filepath.Join(cacheDir, "ab", "one-d"))
if r.Status != 0 || r.Err != "" || strings.TrimSpace(r.Stdout) != "warm" {
t.Fatalf("the first command of the ask did not read the shared cache and write its own (status %d, %s): %s%s", r.Status, r.Err, r.Stdout, r.Stderr)
}
r = asked(context.Background(), []string{"PATH=/usr/bin:/bin"}, "/bin/cat", filepath.Join(cacheDir, "ab", "one-d"), filepath.Join(cacheDir, "warm"))
if r.Status != 0 || r.Err != "" || r.Stdout != "compiled\nwarm\n" {
t.Fatalf("the next command of the ask did not find what the first wrote (status %d, %s): %q %s", r.Status, r.Err, r.Stdout, r.Stderr)
}
if _, err := os.Stat(filepath.Join(cacheDir, "ab")); !os.IsNotExist(err) {
t.Fatalf("the ask's commands wrote the shared cache: %v", err)
}
if err := os.RemoveAll(filepath.Join(workspace, "cachelayer")); err != nil {
t.Fatalf("the ask's cache layer cannot be removed with its workspace: %v", err)
}
} }
+54
View File
@@ -2,7 +2,10 @@ package buildstep
import ( import (
"context" "context"
"os"
"path/filepath"
"strings" "strings"
"sync"
"testing" "testing"
"git.novox.be/novox/mesh-sdk/go/forgekey" "git.novox.be/novox/mesh-sdk/go/forgekey"
@@ -112,4 +115,55 @@ func TestIsolatedRunsEachCommandThroughBubblewrap(t *testing.T) {
if hosts == "" { if hosts == "" {
t.Fatalf("no hosts file was bound: %v", ran) t.Fatalf("no hosts file was bound: %v", ran)
} }
// 1921/S2: with a cache layer, what one command wrote into the caches is kept for the ask's later commands, the
// shared cache stays unchanged, two commands at once write into layers of their own (overlayfs refuses an upper
// layer two mounts share), and a command's own layer is gone when it ends.
root := t.TempDir()
shared := filepath.Join(root, "gobuild")
if err := os.MkdirAll(shared, 0o700); err != nil {
t.Fatal(err)
}
layer := filepath.Join(root, "ws", "cachelayer")
var mu sync.Mutex
uppers := map[string]bool{}
gate := make(chan struct{})
var started sync.WaitGroup
started.Add(2)
writer := func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
up := overlayOf(args, shared)
mu.Lock()
uppers[up] = true
mu.Unlock()
started.Done()
<-gate // both commands are running before either ends
os.MkdirAll(filepath.Join(up, "ab"), 0o700)
os.WriteFile(filepath.Join(up, "ab", args[len(args)-1]+"-d"), []byte("compiled"), 0o600)
return forgekey.Ran{}
}
run := Isolated(writer, Isolation{Workspace: filepath.Join(root, "ws"), Caches: []string{shared}, CacheLayer: layer})
var wg sync.WaitGroup
for _, pkg := range []string{"one", "two"} {
wg.Add(1)
go func() { defer wg.Done(); run(context.Background(), nil, "go", "build", pkg) }()
}
started.Wait()
close(gate)
wg.Wait()
if len(uppers) != 2 {
t.Fatalf("two commands at once wrote into %d layers, want 2 of their own: %v", len(uppers), uppers)
}
for _, pkg := range []string{"one", "two"} {
if b, err := os.ReadFile(filepath.Join(keptLayer(layer, 0), "ab", pkg+"-d")); err != nil || string(b) != "compiled" {
t.Errorf("what the build of %s compiled is not kept for the ask: %q %v", pkg, b, err)
}
}
for up := range uppers {
if _, err := os.Stat(up); !os.IsNotExist(err) {
t.Errorf("the command's own layer %s outlived it: %v", up, err)
}
}
if entries, _ := os.ReadDir(shared); len(entries) != 0 {
t.Errorf("the shared cache was written: %v", entries)
}
} }
+244
View File
@@ -0,0 +1,244 @@
package buscompose
import (
"encoding/json"
"fmt"
"slices"
"strings"
"time"
)
// **The accounts the grants part wants, and the one shape a held account takes** (novox/hq ADR 0561 §1, §2; issue 774).
//
// The bus composer derives every user from placement, the manifests and the wants. For each account the mesh mints (a
// module's, a module's process's, a machine's provisioner client's), the grants part writes one key of its state
// `accounts`, keyed by the bus user, holding the account's fields that are not secret: what the controller's credential
// carries today beside the password. The vault watches that state, mints the user it holds no value for, and keeps the
// value as the module's own secret on that node, built by HeldCredential, which gives byte for byte the controller's
// shape, so a runtime reads it unchanged. The state is read by the vault alone (NamedReaders), as `bus-accounts` is read
// by the grants part alone.
//
// The node accounts, the controller's own, an enrolment user and a person are never wanted here: the vault mints none
// of them (ADR 0561 §4, §5).
// AccountsState is the grants part's state of wanted accounts, by bus user.
const AccountsState = "accounts"
// GrantsAccounts is that state as a reader names it (`<owner>.<name>`).
const GrantsAccounts = GrantsModule + "." + AccountsState
// AccountsBucket is the bucket the state lives in.
var AccountsBucket = BucketName(GrantsModule, AccountsState)
// AccountsSubject is the subject of one wanted account's key: `$KV.mesh-grants_accounts.<user>`.
func AccountsSubject(user string) string { return "$KV." + AccountsBucket + "." + user }
// AccountIssuedEvent is what the vault says when it minted an account: AccountIssued, never a value or a hash.
const AccountIssuedEvent = "account-issued"
// AccountIssuedSubject is where it says it, as the runtime addresses mesh-vault's events.
const AccountIssuedSubject = "mesh.mod." + VaultSeat + ".event." + AccountIssuedEvent
// ClaimedSeat is one seat a module claims, as its runtime needs it (the controller's seatClaimed): the name, the scope (a
// node-scoped seat's verb carries the machine) and the verbs the runtime serves for it.
type ClaimedSeat struct {
Seat string `json:"seat"`
Scope string `json:"scope"`
Serves []string `json:"serves,omitempty"`
}
// WantedAccount is one key of the grants part's state `accounts`: an account the mesh needs, without its password.
type WantedAccount struct {
User string `json:"user"`
// Kind is KindModule (KindNodeTools for the runtime, named as a module is), KindProcess or KindProvisioner.
Kind Kind `json:"kind"`
Node string `json:"node"`
Module string `json:"module"`
// Process is a KindProcess account's process id.
Process string `json:"process,omitempty"`
// URL is the bus's address as that node reaches it, `nats://<host>:<port>`; Fingerprint the bus certificate's.
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
// Claims are a module's claims with the verbs each seat promises: none for a process or a provisioner client.
Claims []ClaimedSeat `json:"claims,omitempty"`
// At is when the grants part wrote the key: not compared (Same).
At time.Time `json:"at"`
}
// mintedKinds are the kinds of account the vault mints (ADR 0561 §1).
var mintedKinds = []Kind{KindModule, KindNodeTools, KindProcess, KindProvisioner}
// principal is the account as the composer names it.
func (w WantedAccount) principal() Principal {
return Principal{Kind: w.Kind, Node: w.Node, Module: w.Module, Process: w.Process}
}
// Valid says why w is not an account the vault may mint, or nil: a kind it mints, every name a subject token, the user
// the one the composer derives from them, and an address to reach the bus at.
func (w WantedAccount) Valid() error {
if !slices.Contains(mintedKinds, w.Kind) {
return fmt.Errorf("%s is a %s account, which the vault never mints (ADR 0561 §4)", w.User, w.Kind)
}
for what, v := range map[string]string{"node": w.Node, "module": w.Module} {
if !safeSubject.MatchString(v) {
return fmt.Errorf("%s: the %s %q is not a name the mesh writes", w.User, what, v)
}
}
if w.Kind == KindProcess && !safeSubject.MatchString(w.Process) {
return fmt.Errorf("%s: the process %q is not a name the mesh writes", w.User, w.Process)
}
if w.Kind != KindProcess && w.Process != "" {
return fmt.Errorf("%s: only a process's account names a process", w.User)
}
if want := w.principal().Username(); w.User != want {
return fmt.Errorf("%s is not the user of a %s account of %s on %s (that is %s)", w.User, w.Kind, w.Module, w.Node, want)
}
if !strings.HasPrefix(w.URL, "nats://") || len(w.URL) == len("nats://") {
return fmt.Errorf("%s names no bus address (url, nats://<host>:<port>)", w.User)
}
if len(w.Claims) > 0 && w.Kind != KindModule && w.Kind != KindNodeTools {
return fmt.Errorf("%s: a %s account holds no seat, so it carries no claims", w.User, w.Kind)
}
return nil
}
// Same says whether two wanted accounts want the same account: every field but when it was written.
func (w WantedAccount) Same(o WantedAccount) bool {
a, b := w, o
a.At, b.At = time.Time{}, time.Time{}
ra, _ := json.Marshal(a)
rb, _ := json.Marshal(b)
return string(ra) == string(rb)
}
// OwnSecret is the module's own secret the account is held in: `broker`, `broker-<process id>` or `provisioner-broker`
// (the controller's names, ADR 0436 §2, ADR 0349 §11).
func (w WantedAccount) OwnSecret() string {
switch w.Kind {
case KindProcess:
return "broker-" + w.Process
case KindProvisioner:
return "provisioner-broker"
}
return "broker"
}
// ReadWantedAccount reads the key user of the state: refused when it is not a valid wanted account of that user.
func ReadWantedAccount(user string, raw []byte) (WantedAccount, error) {
var w WantedAccount
if err := json.Unmarshal(raw, &w); err != nil {
return WantedAccount{}, fmt.Errorf("%s %s is not a wanted account: %w", GrantsAccounts, user, err)
}
if w.User != user {
return WantedAccount{}, fmt.Errorf("%s %s names the user %q", GrantsAccounts, user, w.User)
}
return w, w.Valid()
}
// AccountIssued is the vault's account-issued: which account it minted, and its generation. Never a value or a hash.
type AccountIssued struct {
User string `json:"user"`
Kind Kind `json:"kind"`
Node string `json:"node"`
Module string `json:"module"`
Process string `json:"process,omitempty"`
Generation int `json:"generation"`
At time.Time `json:"at"`
}
// HeldCredential is the value the account's holder reads, exactly as the controller builds it today (issueWith,
// sealProcess and sealProvisionerClient): a module's `{url, fingerprint, node, module, user, password, claims}`, a
// process's without claims, a provisioner client's without module or claims; an empty fingerprint or claims left out.
func HeldCredential(w WantedAccount, password string) ([]byte, error) {
if err := w.Valid(); err != nil {
return nil, err
}
if password == "" {
return nil, fmt.Errorf("%s: an account is held with its password", w.User)
}
switch w.Kind {
case KindProcess:
return json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
Module string `json:"module"`
User string `json:"user"`
Password string `json:"password"`
}{URL: w.URL, Fingerprint: w.Fingerprint, Node: w.Node, Module: w.Module, User: w.User, Password: password})
case KindProvisioner:
return json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
User string `json:"user"`
Password string `json:"password"`
}{URL: w.URL, Fingerprint: w.Fingerprint, Node: w.Node, User: w.User, Password: password})
}
return json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
Module string `json:"module"`
User string `json:"user"`
Password string `json:"password"`
Claims []ClaimedSeat `json:"claims,omitempty"`
}{URL: w.URL, Fingerprint: w.Fingerprint, Node: w.Node, Module: w.Module, User: w.User, Password: password,
Claims: w.Claims})
}
// **One issuer at every moment: minting turns on at the lease write** (novox/hq ADR 0561 §3, §6). The controller's lease
// is the one key `holder` of its bucket (LeaseBucket). The operator's credential hands it away by writing a value that
// names its new holder in `handed-to`; while the key holds that, the vault mints and the grants part creates the durable
// consumers an account needs. A controller's value, a key deleted or expired, or one that cannot be read or parsed, turn
// both off: when in doubt nothing is minted, so the controller and the vault never both mint one account. No clock and
// no setting turns it.
// ControllerLease is the lease's bucket as a reader names it; the vault and the grants part alone read it
// (NamedReaders), the controller's announcement marking it readable by nobody.
const ControllerLease = ControllerSeat + ".lease"
// ControllerLeaseKey is the lease's one key.
const ControllerLeaseKey = leaseKey
// LeaseHolder is the lease key's value: a controller instance's (mesh-controller internal/lease Holder), or, once handed
// away, one naming its holder in HandedTo.
type LeaseHolder struct {
Instance string `json:"instance,omitempty"`
Host string `json:"host,omitempty"`
Epoch uint64 `json:"epoch,omitempty"`
Taken time.Time `json:"taken,omitzero"`
Renewed time.Time `json:"renewed,omitzero"`
// HandedTo is who holds the lease once the controller handed it away: never a controller instance.
HandedTo string `json:"handed-to,omitempty"`
// HandedBy is the credential that wrote it (the operator's), and Why its reason.
HandedBy string `json:"handed-by,omitempty"`
Why string `json:"why,omitempty"`
}
// LeaseHandedAway reads the lease key as a getter answered it (its value, whether it was there, the read's error) and
// says whether it is handed to a holder that is no controller, with why either way.
func LeaseHandedAway(raw []byte, found bool, readErr error) (bool, string) {
switch {
case readErr != nil:
return false, "the controller's lease cannot be read (" + readErr.Error() + "), so nothing is minted"
case !found || len(raw) == 0 || string(raw) == "null":
return false, "no lease is written (expired, deleted, or never taken): a controller may take it, so nothing is minted"
}
var h LeaseHolder
if err := json.Unmarshal(raw, &h); err != nil {
return false, "the controller's lease is not readable as a holder (" + err.Error() + "), so nothing is minted"
}
if strings.TrimSpace(h.HandedTo) == "" {
who := h.Instance
if who == "" {
who = "an instance that names itself not"
}
return false, "the controller holds the lease (" + who + "): it mints, so the vault mints nothing (ADR 0561 §3)"
}
if h.Instance != "" {
return false, "the lease names both a controller instance (" + h.Instance + ") and a holder it is handed to (" +
h.HandedTo + "), so nothing is minted"
}
return true, "the lease is handed to " + h.HandedTo + ": the vault mints every new account (ADR 0561 §1, §3)"
}
+10
View File
@@ -1,6 +1,7 @@
package buscompose package buscompose
import ( import (
"slices"
"sort" "sort"
"strings" "strings"
@@ -58,6 +59,10 @@ func ReadsAnnouncedBy(reader string, reads []string, announced map[string]announ
switch { switch {
case reader != "" && !ReaderMay(reader, r): case reader != "" && !ReaderMay(reader, r):
refused = append(refused, r) refused = append(refused, r)
case reader != "" && namedReader(reader, r):
// A state the composer names its readers for is theirs whatever its owner announced (ADR 0561 §3: the
// controller's lease, which its announcement marks readable by nobody).
granted = append(granted, r)
case reader != "" && ok && said && s.Keeps(name): case reader != "" && ok && said && s.Keeps(name):
if s.ReadableBy(name, reader) { if s.ReadableBy(name, reader) {
granted = append(granted, r) granted = append(granted, r)
@@ -95,3 +100,8 @@ func NotAnnounced(r Records) []string {
sort.Strings(out) sort.Strings(out)
return out return out
} }
// namedReader says the composer names module among the readers of read (NamedReaders).
func namedReader(module, read string) bool {
return slices.Contains(NamedReaders[strings.TrimSpace(read)], module)
}
+262 -1
View File
@@ -2,6 +2,7 @@ package buscompose
import ( import (
"encoding/json" "encoding/json"
"errors"
"os" "os"
"path/filepath" "path/filepath"
"reflect" "reflect"
@@ -9,8 +10,11 @@ import (
"slices" "slices"
"strings" "strings"
"testing" "testing"
"time"
"git.novox.be/novox/mesh-sdk/go/announce"
"git.novox.be/novox/mesh-sdk/go/delivery" "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
) )
// Defends issue 1377 and novox/hq ADR 0547 §5: this is the controller's bus composer copied, and it composes byte for // Defends issue 1377 and novox/hq ADR 0547 §5: this is the controller's bus composer copied, and it composes byte for
@@ -51,6 +55,26 @@ func mayBeyondTheControllers(user string) map[string]bool {
VaultSecretChanged: true} VaultSecretChanged: true}
} }
// engineBeyond is a node's engine holding its node seat and asking the operator in its own name (novox/hq ADR 0563 §1,
// §3), beyond the controller's on purpose: its own node's verbs, events and discovery, the bus's discovery asked, and
// the router's seat traffic under node-engine-<node>.
func engineBeyond(user, g string) bool {
node, ok := strings.CutPrefix(user, "node.")
if !ok {
return false
}
return strings.HasPrefix(g, "mesh.seat."+EngineSeat+".") && strings.Contains(g, "."+node) ||
strings.HasPrefix(g, "$SRV.") || strings.Contains(g, ".node-engine-"+node) ||
strings.Contains(g, "KV_"+EngineActsBucket) || strings.HasPrefix(g, "$KV."+EngineActsBucket+"."+node+".")
}
// createsAnAccountsConsumer is the grants part's create and read of a durable consumer an account needs, by its stream
// and name (novox/hq ADR 0561 §6): beyond the controller's on purpose, since the controller made them alone.
func createsAnAccountsConsumer(user, g string) bool {
return strings.HasSuffix(user, "."+GrantsModule) &&
(strings.HasPrefix(g, "$JS.API.CONSUMER.CREATE.") || strings.HasPrefix(g, "$JS.API.CONSUMER.INFO."))
}
// quotedToken is a quoted token of a composed list. // quotedToken is a quoted token of a composed list.
var quotedToken = regexp.MustCompile(`"[^"]*"`) var quotedToken = regexp.MustCompile(`"[^"]*"`)
@@ -102,7 +126,8 @@ func grantsAsTheControllers(t *testing.T, got, want string) {
} }
} }
for g := range now { for g := range now {
if !was[g] && !intended[g] && !mayBeyondTheControllers(user)[g] && !createdAdvisory.MatchString(`"`+g+`", `) { if !was[g] && !intended[g] && !mayBeyondTheControllers(user)[g] && !createdAdvisory.MatchString(`"`+g+`", `) &&
!createsAnAccountsConsumer(user, g) && !engineBeyond(user, g) {
t.Errorf("%s: the copy grants %s beyond the controller's, and no decision names it", user, g) t.Errorf("%s: the copy grants %s beyond the controller's, and no decision names it", user, g)
} }
} }
@@ -119,6 +144,8 @@ func grantsAsTheControllers(t *testing.T, got, want string) {
} }
} }
const SignersBucketName = "mesh-vault_signers"
func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) { func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
// ADR 0567 §3, §9: a node reads every key of the seat's placement and its candidates (it composes with the whole // ADR 0567 §3, §9: a node reads every key of the seat's placement and its candidates (it composes with the whole
// proposal laid over), and none of its open proposals; it writes neither. // proposal laid over), and none of its open proposals; it writes neither.
@@ -221,6 +248,55 @@ func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
t.Error("a user of a seat not pulled by kind reads DEAD_LETTERS") t.Error("a user of a seat not pulled by kind reads DEAD_LETTERS")
} }
}) })
// ADR 0563 §1, §3, §6: each node's engine alone answers its own node's node-engine verbs and says its events, any
// runtime may call them, and it asks the operator under node-engine-<node> alone.
t.Run("a node's engine holds its node seat and asks in its own name", func(t *testing.T) {
router := Seat{Name: AsksSeat, Accepts: []string{"ask", "cancel"}, Emits: []string{"decided"},
ByCaller: []string{"ask", "cancel", "decided"}, Records: []string{"messenger_asks"}}
users, err := Users(Records{Nodes: []string{"one", "two"}, Assigned: map[string][]Declared{
"one": {{Module: "messenger", Holds: []Seat{router}, Invokes: []string{"seat:" + EngineSeat + ".root-free"}}},
"two": {{Module: RuntimeModule}, {Module: "shop", Invokes: []string{"*"}}}}})
if err != nil {
t.Fatal(err)
}
for _, u := range users {
perms, err := PermissionsFor(u)
if err != nil {
t.Fatal(err)
}
answers := MaySubscribe(perms, "mesh.seat."+EngineSeat+".tool.root-free.one") ||
MaySubscribe(perms, "mesh.seat."+EngineSeat+".call.root-free.one.messenger~one")
if engineOne := u.Kind == KindNode && u.Node == "one"; answers != engineOne {
t.Errorf("%s answers one's root-free: %v", u.Username(), answers)
}
switch {
case u.Kind == KindNode && u.Node == "one":
for subject, want := range map[string]bool{
"mesh.seat.operator-channel.accept.ask.node-engine-one": true,
"mesh.seat.operator-channel.accept.cancel.node-engine-one": true,
"mesh.seat.operator-channel.accept.ask.node-engine-two": false,
"$JS.API.DIRECT.GET.KV_messenger_asks.$KV.messenger_asks.node-engine-one.hand-over-1": true,
"$JS.API.DIRECT.GET.KV_messenger_asks.$KV.messenger_asks.node-engine-two.hand-over-1": false,
"mesh.seat." + EngineSeat + ".event.done.one": true,
"mesh.seat." + EngineSeat + ".event.done.two": false,
"$SRV.INFO": true,
} {
if got := MayPublish(perms, subject); got != want {
t.Errorf("node one may publish %s: %v, want %v", subject, got, want)
}
}
if !MaySubscribe(perms, "mesh.seat.operator-channel.event.decided.node-engine-one") ||
MaySubscribe(perms, "mesh.seat.operator-channel.event.decided.node-engine-two") ||
!MaySubscribe(perms, "$SRV.INFO."+EngineSeat) || MaySubscribe(perms, "mesh.seat."+EngineSeat+".tool.hand-over.two") {
t.Errorf("node one hears %v", perms.Subscribe)
}
case u.Kind == KindNodeTools || u.Module == "messenger":
if !MayPublish(perms, "mesh.seat."+EngineSeat+".tool.root-free.one") {
t.Errorf("%s cannot call one's root-free", u.Username())
}
}
}
})
// Issue 1126 (ADR 0550 §5): a node's engine asks the vault's `own` in its own name while the vault serves it, and no // Issue 1126 (ADR 0550 §5): a node's engine asks the vault's `own` in its own name while the vault serves it, and no
// other node's. // other node's.
t.Run("a node asks the vault its own secrets", func(t *testing.T) { t.Run("a node asks the vault its own secrets", func(t *testing.T) {
@@ -239,6 +315,21 @@ func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
if !slices.Contains(perms.Subscribe, VaultSecretChanged) { if !slices.Contains(perms.Subscribe, VaultSecretChanged) {
t.Errorf("node one does not hear the vault's secret-changed: %v", perms.Subscribe) t.Errorf("node one does not hear the vault's secret-changed: %v", perms.Subscribe)
} }
// L3 s6 (ADR 0550 §2): while the vault serves `sign`, a node asks it in its own name alone, and reads the vault's
// `signers` and the lease key, never writing either.
signs, _ := PermissionsFor(Principal{Kind: KindNode, Node: "one", AsksTheVaultToSign: true})
signersGet := "$JS.API.DIRECT.GET.KV_" + SignersBucketName + ".$KV." + SignersBucketName + ".key"
leaseGet := "$JS.API.DIRECT.GET.KV_" + LeaseBucket + ".$KV." + LeaseBucket + ".holder"
for subject, want := range map[string]bool{VaultSignCall(one): true, signersGet: true, leaseGet: true,
VaultSignCall(Principal{Kind: KindNode, Node: "two"}): false, "$KV." + SignersBucketName + ".key": false,
"$KV." + LeaseBucket + ".holder": false} {
if MayPublish(signs, subject) != want {
t.Errorf("a node asking sign may publish %s: %v, want %v", subject, !want, want)
}
}
if MayPublish(perms, VaultSignCall(one)) || MayPublish(perms, signersGet) {
t.Error("a node asks sign or reads signers while the vault does not serve sign")
}
// And what a module on another node serves from its settings, that key alone (mailu's smtp domain, issue 1126). // And what a module on another node serves from its settings, that key alone (mailu's smtp domain, issue 1126).
users, err := Users(Records{Nodes: []string{"one", "two"}, Assigned: map[string][]Declared{ users, err := Users(Records{Nodes: []string{"one", "two"}, Assigned: map[string][]Declared{
"two": {{Module: "mailer", Served: []string{"domain", "not.a.token"}}}}}) "two": {{Module: "mailer", Served: []string{"domain", "not.a.token"}}}}})
@@ -293,4 +384,174 @@ func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
} }
grantsAsTheControllers(t, got, string(want)) grantsAsTheControllers(t, got, string(want))
}) })
// 774/AC2 (novox/hq ADR 0561 §2): a wanted account round-trips through the grants part's state `accounts`, keyed by
// its user, at the subject `$KV.mesh-grants_accounts.<user>`, read by the vault alone; and the vault's account-issued
// names the account, never a value.
t.Run("a wanted account is kept by its user and read by the vault alone", func(t *testing.T) {
for _, w := range []WantedAccount{
{User: "ace.mesh-build-go", Kind: KindModule, Node: "ace", Module: "mesh-build-go", URL: "nats://novox.mesh:4222",
Fingerprint: "ab", Claims: []ClaimedSeat{{Seat: "node-build-go", Scope: "node", Serves: []string{"build"}}}},
{User: "novox.mesh-mail.send", Kind: KindProcess, Node: "novox", Module: "mesh-mail", Process: "send",
URL: "nats://novox.mesh:4222"},
{User: "provisioner.g14", Kind: KindProvisioner, Node: "g14", Module: "mesh-provisioner-client",
URL: "nats://novox.mesh:4222", Fingerprint: "ab"},
} {
if err := w.Valid(); err != nil {
t.Fatalf("%s: %v", w.User, err)
}
raw, err := json.Marshal(w)
if err != nil {
t.Fatal(err)
}
back, err := ReadWantedAccount(w.User, raw)
if err != nil || !back.Same(w) {
t.Errorf("%s does not round-trip: %+v, %v", w.User, back, err)
}
if got := AccountsSubject(w.User); got != "$KV.mesh-grants_accounts."+w.User {
t.Errorf("%s is kept at %s", w.User, got)
}
if _, err := ReadWantedAccount("someone.else", raw); err == nil {
t.Errorf("%s was read under another user's key", w.User)
}
}
for _, bad := range []WantedAccount{
{User: "ace.x", Kind: KindModule, Node: "ace", Module: "y", URL: "nats://a:1"},
{User: "ace.x.p", Kind: KindProcess, Node: "ace", Module: "x", URL: "nats://a:1"},
{User: "node.ace", Kind: KindNode, Node: "ace", URL: "nats://a:1"},
{User: "controller", Kind: KindController, URL: "nats://a:1"},
{User: "ace.x", Kind: KindModule, Node: "ace", Module: "x"},
} {
if bad.Valid() == nil {
t.Errorf("%+v is taken as a wanted account", bad)
}
}
if ReaderMay("mesh-delivery-assigner", GrantsAccounts) || !ReaderMay(VaultSeat, GrantsAccounts) {
t.Errorf("%s is read by %v alone, want the vault's", GrantsAccounts, NamedReaders[GrantsAccounts])
}
e := AccountIssued{User: "ace.x", Kind: KindModule, Node: "ace", Module: "x", Generation: 1}
raw, _ := json.Marshal(e)
if strings.Contains(string(raw), "password") || strings.Contains(string(raw), "hash") ||
AccountIssuedSubject != "mesh.mod.mesh-vault.event.account-issued" {
t.Errorf("account-issued says %s at %s", raw, AccountIssuedSubject)
}
})
// 774/AC3 (novox/hq ADR 0561 §2): the account the vault seals is, byte for byte, what the controller hands for each of
// its three kinds: testdata/controller-credentials.json was written by the controller's own code (its "how").
t.Run("a held account is the controller's shape byte for byte", func(t *testing.T) {
raw, err := os.ReadFile(filepath.Join("testdata", "controller-credentials.json"))
if err != nil {
t.Fatal(err)
}
var captured struct {
Rows []struct{ Name, Held string }
}
if err := json.Unmarshal(raw, &captured); err != nil {
t.Fatal(err)
}
vault := ClaimedSeat{Seat: "mesh-vault", Scope: "mesh", Serves: []string{"get", "accept", "secret-ask", "issued",
"rotate", "move-in", "node-key", "own", "move-asks"}}
controller := ClaimedSeat{Seat: "mesh-controller", Scope: "mesh", Serves: strings.Split("tools,calls,status,nodes,"+
"node,placement-export,modules,seats,builds,plans,delivery-plan,delivery-order,delivery-check,deliver,"+
"delivery-stop,delivery-backups-again,delivery-walks,plan,assign,unassign,pin,unpin,push,rotate,give,issued,"+
"secret-move,secret-ask,issue,token,settings,command,queue,cancel,clear,replay,try,kill,pause,resume,hand-act,"+
"drill,warranted,hand-acts,durations,conditions,healers,doctor,upgrade,bus,retire,cleanup,judge-directories,"+
"root-free,dead-letters,data,build,artifacts,held,collect,move-placement,move-config,mirrors,images", ",")}
at := "nats://10.0.0.1:4222"
wanted := map[string]struct {
w WantedAccount
password string
}{
"module": {WantedAccount{User: "laptop.cap-bot", Kind: KindModule, Node: "laptop", Module: "cap-bot", URL: at,
Fingerprint: "ab:cd", Claims: []ClaimedSeat{vault, {Seat: "node-service-manager", Scope: "node",
Serves: []string{"status"}}, controller, {Seat: "no-such-seat", Scope: "node"}}}, "pw-module"},
"module-no-claims-no-fingerprint": {WantedAccount{User: "laptop.cap-plain", Kind: KindModule, Node: "laptop",
Module: "cap-plain", URL: at}, "pw-plain"},
"provisioner": {WantedAccount{User: "provisioner.laptop", Kind: KindProvisioner, Node: "laptop", Module: "nats",
URL: at, Fingerprint: "ab:cd"}, "pw-prov"},
"process": {WantedAccount{User: "laptop.mesh-mail.send", Kind: KindProcess, Node: "laptop", Module: "mesh-mail",
Process: "send", URL: at, Fingerprint: "ab:cd"}, "pw-proc"},
}
if len(captured.Rows) != len(wanted) {
t.Fatalf("%d rows captured, %d wanted", len(captured.Rows), len(wanted))
}
for _, row := range captured.Rows {
c, ok := wanted[row.Name]
if !ok {
t.Fatalf("no wanted account for the captured %s", row.Name)
}
got, err := HeldCredential(c.w, c.password)
if err != nil || string(got) != row.Held {
t.Errorf("%s:\n got %s (%v)\nwant %s", row.Name, got, err, row.Held)
}
}
if _, err := HeldCredential(wanted["module"].w, ""); err == nil {
t.Error("an account was held with no password")
}
})
// 774 subtasks 6 and 8 (novox/hq ADR 0561 §3, §6): minting and the grants part's consumers turn on only while the
// lease is handed to a holder that is no controller; the vault and the grants part alone read the lease, whatever the
// controller announced; a module's consumer is created as the controller's EnsureConsumer creates it.
t.Run("the lease turns minting on only when handed away; a consumer is created as the controller's", func(t *testing.T) {
for _, row := range []struct {
name string
raw string
found bool
err error
on bool
}{
{"a controller holds it", `{"instance":"novox/1234/2026-10-27T20:00:00Z","epoch":57}`, true, nil, false},
{"handed away", `{"handed-to":"the operator's hand-over","handed-by":"person.jochen"}`, true, nil, true},
{"expired or deleted", ``, false, nil, false},
{"unreadable", ``, false, errors.New("refused"), false},
{"not a holder", `[1]`, true, nil, false},
{"both a controller and a handed holder", `{"instance":"novox/1","handed-to":"x"}`, true, nil, false},
} {
if on, why := LeaseHandedAway([]byte(row.raw), row.found, row.err); on != row.on || why == "" {
t.Errorf("%s: on %v (%s), want %v", row.name, on, why, row.on)
}
// The assigner's grants source follows it: the grants part's bucket only for a handed-away lease (issue 774).
on, _ := LeaseHandedAway([]byte(row.raw), row.found, row.err)
if bucket, _ := delivery.GrantSource(on); (bucket == delivery.PartGrantsBucket) != row.on {
t.Errorf("%s: the grants source is %s", row.name, bucket)
}
}
announced := map[string]announce.States{ControllerSeat: ControllerAnnouncement("novox")}
for reader, want := range map[string]bool{VaultSeat: true, GrantsModule: true, "mesh-delivery-assigner": true, "mesh-delivery-view": false} {
granted, _ := ReadsAnnouncedBy(reader, []string{ControllerLease}, announced)
if (len(granted) == 1) != want {
t.Errorf("%s reads the lease: %v, want %v", reader, granted, want)
}
}
c, _ := ConsumerFor(Principal{Kind: KindModule, Node: "a", Module: "m", Consumes: []string{"x.happened"}})
cfg := ConsumerConfig(c)
if cfg.Durable != "a_m" || cfg.AckPolicy != jetstream.AckExplicitPolicy || cfg.AckWait != 30*time.Second ||
cfg.MaxDeliver != 5 || cfg.DeliverPolicy != jetstream.DeliverNewPolicy || len(cfg.FilterSubjects) != 2 ||
cfg.Description == "" {
t.Errorf("a module's consumer is created as %+v", cfg)
}
// The grants part is granted the create of each account's consumer by its stream and name, and no other.
users, err := Users(Records{Nodes: []string{"a"}, Assigned: map[string][]Declared{"a": {
{Module: GrantsModule}, {Module: "m", Consumes: []string{"x.happened"}}, {Module: "quiet"}}}})
if err != nil {
t.Fatal(err)
}
for _, u := range users {
if u.Module != GrantsModule || u.Kind != KindModule {
continue
}
perms, err := PermissionsFor(u)
if err != nil {
t.Fatal(err)
}
if !MayPublish(perms, "$JS.API.CONSUMER.CREATE.EVENTS.a_m") || MayPublish(perms, "$JS.API.CONSUMER.CREATE.EVENTS.a_quiet") ||
MayPublish(perms, "$JS.API.CONSUMER.DELETE.EVENTS.a_m") || MayPublish(perms, "$JS.API.CONSUMER.CREATE.EVENTS.a_m.mesh.mod.x.event.happened") {
t.Errorf("the grants part creates consumers as %v", u.CreatesConsumers)
}
}
q := ConsumerConfig(QueueConsumer(Principal{Kind: KindProcess, Node: "a", Module: "m", Process: "p"},
QueueOwner{Process: "p", Stream: "M_WORK", MaxDeliver: 3}))
if q.Durable != "a_m_p" || q.FilterSubject != "" || q.MaxDeliver != 3 || q.DeliverPolicy != jetstream.DeliverAllPolicy {
t.Errorf("a process's queue consumer is created as %+v", q)
}
})
} }
+27
View File
@@ -86,3 +86,30 @@ func ModuleStreamConfig(m ModuleStream) jetstream.StreamConfig {
} }
return cfg return cfg
} }
// ConsumerConfig is a pull consumer as the controller's EnsureConsumer creates it (mesh-controller internal/broker
// JetStream.EnsureConsumer, a consumer not there yet): its durable name, explicit acks, its ack wait, max deliver and max
// ack pending, its description, one filter or several, and from the stream's end when it is made FromNow. The grants part
// creates a module's and a process's consumers with it from the lease write (novox/hq ADR 0561 §6): a create alone,
// never an update of one the bus holds. A push consumer or a queue group is the controller's own and never made here.
func ConsumerConfig(c Consumer) jetstream.ConsumerConfig {
cfg := jetstream.ConsumerConfig{
Durable: c.Name,
AckPolicy: jetstream.AckExplicitPolicy,
AckWait: time.Duration(c.AckWaitSeconds) * time.Second,
MaxDeliver: c.MaxDeliver,
MaxAckPending: c.MaxAckPending,
Description: c.Why,
}
switch len(c.Filters) {
case 0:
case 1:
cfg.FilterSubject = c.Filters[0]
default:
cfg.FilterSubjects = c.Filters
}
if c.FromNow {
cfg.DeliverPolicy = jetstream.DeliverNewPolicy
}
return cfg
}
+11
View File
@@ -23,6 +23,8 @@ func TestTheGrantsPartCreatesOnlyDeclaredStreams(t *testing.T) {
{Module: GrantsModule, State: []Bucket{{Module: GrantsModule, Name: "assertions"}}, Reads: []string{VaultBusAccounts}}, {Module: GrantsModule, State: []Bucket{{Module: GrantsModule, Name: "assertions"}}, Reads: []string{VaultBusAccounts}},
{Module: RuntimeModule}, {Module: RuntimeModule},
}}, }},
// The node-engine's own manifest, placed on every node though no node is assigned it (novox/hq ADR 0563 §2, §3).
Engine: &Declared{Module: EngineModule, State: []Bucket{{Module: EngineModule, Name: EngineActsState}}},
} }
users, err := Users(records) users, err := Users(records)
if err != nil { if err != nil {
@@ -64,6 +66,15 @@ func TestTheGrantsPartCreatesOnlyDeclaredStreams(t *testing.T) {
{GrantsModule, "$JS.API.STREAM.CREATE.ASSIGNMENTS", false}, {GrantsModule, "$JS.API.STREAM.CREATE.ASSIGNMENTS", false},
{GrantsModule, "$JS.API.STREAM.INFO.EVENTS", false}, {GrantsModule, "$JS.API.STREAM.INFO.EVENTS", false},
{"mesh-delivery-builder", "$JS.API.STREAM.CREATE.KV_mesh-delivery-builder_steps", false}, {"mesh-delivery-builder", "$JS.API.STREAM.CREATE.KV_mesh-delivery-builder_steps", false},
// The node-engine's acts: created by the grants part, written by each node's engine under its own keys alone.
{GrantsModule, "$JS.API.STREAM.CREATE.KV_mesh-host_acts", true},
{"node.ace", "$JS.API.STREAM.CREATE.KV_mesh-host_acts", false},
{"node.ace", "$KV.mesh-host_acts.ace.act.hand-over-1", true},
{"node.ace", "$KV.mesh-host_acts.ace.open.hand-over.abc", true},
{"node.ace", "$KV.mesh-host_acts.novox.act.hand-over-1", false},
{"node.ace", "$JS.API.DIRECT.GET.KV_mesh-host_acts.$KV.mesh-host_acts.ace.ask.hand-over-1", true},
{"node.ace", "$JS.API.DIRECT.GET.KV_mesh-host_acts.$KV.mesh-host_acts.novox.ask.hand-over-1", false},
{"novox.node-tools", "$KV.mesh-host_acts.novox.act.hand-over-1", false},
// The sealed user list (ADR 0547 clause 7): the grants part writes its own bucket and no other. // The sealed user list (ADR 0547 clause 7): the grants part writes its own bucket and no other.
{GrantsModule, "$JS.API.STREAM.CREATE.OBJ_mesh-grants-user-list", true}, {GrantsModule, "$JS.API.STREAM.CREATE.OBJ_mesh-grants-user-list", true},
{GrantsModule, "$O.mesh-grants-user-list.C.abc", true}, {GrantsModule, "$O.mesh-grants-user-list.C.abc", true},
+24
View File
@@ -247,6 +247,30 @@ func ConsumersOf(users []Principal) []ModuleConsumer {
return out return out
} }
// AccountConsumers are the durable consumers an account needs (novox/hq ADR 0561 §6): of ConsumersOf, each module's on
// EVENTS and each process's on its own work queue, never one on a module stream it hears (those the controller makes with
// the stream's grant). The grants part creates them from the lease write, with these configurations.
func AccountConsumers(users []Principal) []ModuleConsumer {
queues := map[string]bool{}
for _, p := range users {
for _, q := range p.Queues {
queues[q.Stream] = true
}
for _, d := range p.Carries {
for _, q := range d.Queues {
queues[q.Stream] = true
}
}
}
var out []ModuleConsumer
for _, mc := range ConsumersOf(users) {
if mc.Consumer.Stream == consumerStream(Principal{Kind: KindModule}) || queues[mc.Consumer.Stream] {
out = append(out, mc)
}
}
return out
}
// HolderConsumerFor is the worker a seat's holders share on that seat's work queue. // HolderConsumerFor is the worker a seat's holders share on that seat's work queue.
// //
// **One worker for every holder, and each holder pulls one ask when it is idle** (novox/hq ADR // **One worker for every holder, and each holder pulls one ask when it is idle** (novox/hq ADR
+45
View File
@@ -0,0 +1,45 @@
package buscompose
import (
"git.novox.be/novox/mesh-sdk/go/asks"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// The node seat node-engine (novox/hq ADR 0563 §1): every node's engine holds it on its own bus account, and no module
// claims it. Its verbs and events are the catalogue's (catalogue.EngineVerbs), read here rather than copied.
// EngineSeat is the seat.
const EngineSeat = catalogue.EngineSeat
// EngineModule is the node-engine's manifest's module name, and EngineActsState its state the acts asked through the
// node seat and its own asks are kept in (ADR 0563 §2, §3: "the engine's own state"): the bucket mesh-host_acts, keys
// `<node>.act.<id>`, `<node>.open.<act>.<digest>` and `<node>.ask.<id>`, each written by that node's engine alone.
const (
EngineModule = "mesh-host"
EngineActsState = "acts"
)
// EngineActsBucket is that state's bucket.
var EngineActsBucket = BucketName(EngineModule, EngineActsState)
// engineGrants is what a node's engine is granted to hold its node seat and to ask the operator (ADR 0563 §1, §3): it
// answers its own node's verbs (the caller-named subjects are added beside them with every principal's), says the
// seat's events about its node, answers discovery for the seat, asks who answers what (the login shell's execute, ADR
// 0563 §6), and asks the operator under asks.EngineAsker(node) on the router's seat as its holder declares it, as any
// user of that seat is granted.
func engineGrants(p Principal) (pub, sub []string) {
for _, v := range catalogue.EngineVerbs() {
sub = append(sub, "mesh.seat."+EngineSeat+".tool."+v.Name+"."+p.Node)
}
for _, e := range catalogue.EngineEvents {
pub = append(pub, "mesh.seat."+EngineSeat+".event."+e+"."+p.Node)
}
sub = append(sub, announcing(EngineSeat)...)
pub = append(pub, discovering()...)
tp, ts := seatTrafficGrants(SeatTrafficOf(asks.EngineAsker(p.Node), nil, p.Uses, nil))
// And its acts and asks, under its own node's keys alone; never the bucket's creation, which is the grants part's.
if own, err := NodeKeysGrants(EngineActsBucket, p.Node); err == nil {
pub = append(pub, own...)
}
return append(pub, tp...), append(sub, ts...)
}
+47
View File
@@ -217,6 +217,11 @@ type Principal struct {
// ADR 0547 §2, ADR 0550 §5, issue 1126): its modules' own secrets sealed to its key, for the compositions it makes // ADR 0547 §2, ADR 0550 §5, issue 1126): its modules' own secrets sealed to its key, for the compositions it makes
// itself. Set while the vault's holder serves it. // itself. Set while the vault's holder serves it.
AsksItsOwnSecrets bool AsksItsOwnSecrets bool
// AsksTheVaultToSign is a machine principal whose node-engine may ask the vault's `sign` in its own name, read the
// vault's public signing halves (`signers`) and the controller's lease key (novox/hq ADR 0550 §2): its own node's
// certificate, the keys it trusts, and the lease write that drops the controller's key. Set while the vault's holder
// serves `sign`.
AsksTheVaultToSign bool
// ServedReads are, for a machine principal, the keys of mesh-config's resolved values that other nodes' modules serve // ServedReads are, for a machine principal, the keys of mesh-config's resolved values that other nodes' modules serve
// from (`<node>.<module>.<key>`, catalogue.ServedSettings): what every consumer is told, and so what a node composing // from (`<node>.<module>.<key>`, catalogue.ServedSettings): what every consumer is told, and so what a node composing
// its own modules reads of a provider elsewhere (issue 1126), each key alone. Sorted, none twice. // its own modules reads of a provider elsewhere (issue 1126), each key alone. Sorted, none twice.
@@ -232,6 +237,10 @@ type Principal struct {
// by name. Created only, never updated or deleted: a new module state is on the bus when the grants part hears its // by name. Created only, never updated or deleted: a new module state is on the bus when the grants part hears its
// manifest, not at the controller's next start (issue 1790). // manifest, not at the controller's next start (issue 1790).
CreatesStreams []string CreatesStreams []string
// CreatesConsumers are the durable consumers the grants part's principal alone may create beside the controller
// (novox/hq ADR 0561 §6): each placed module's on EVENTS and each process's on its work queue (AccountConsumers), by
// stream and name. Created only, never updated or deleted.
CreatesConsumers []ModuleConsumer
// PasswordHash is the bcrypt hash the mesh minted. The plaintext is sealed to the principal // PasswordHash is the bcrypt hash the mesh minted. The plaintext is sealed to the principal
// and never appears here: this file is written to a node's disk and read by a server, and a // and never appears here: this file is written to a node's disk and read by a server, and a
@@ -356,6 +365,10 @@ var VaultNodeKey = SeatVerb{Seat: "mesh-vault", Verb: "node-key"}
// key (issue 1126, ADR 0550 §5); granted as Principal.AsksItsOwnSecrets says. // key (issue 1126, ADR 0550 §5); granted as Principal.AsksItsOwnSecrets says.
var VaultOwn = SeatVerb{Seat: "mesh-vault", Verb: "own"} var VaultOwn = SeatVerb{Seat: "mesh-vault", Verb: "own"}
// VaultSign is the vault's verb a machine's node-engine asks in its own name for its own node's certificate (novox/hq
// ADR 0550 §2); granted as Principal.AsksTheVaultToSign says.
var VaultSign = SeatVerb{Seat: "mesh-vault", Verb: delivery.SignVerb}
// VerbsTheControllerActsOnAWarrant are the other seat verbs the controller calls when the operator's warrant // VerbsTheControllerActsOnAWarrant are the other seat verbs the controller calls when the operator's warrant
// chooses them (novox/hq ADR 0259): a machine's service restarted, and a walk started or stopped through the // chooses them (novox/hq ADR 0259): a machine's service restarted, and a walk started or stopped through the
// controller's own `plans`, a machine's held builds sent by its own `push` and the builds held after a failed release // controller's own `plans`, a machine's held builds sent by its own `push` and the builds held after a failed release
@@ -952,6 +965,18 @@ func PermissionsFor(p Principal) (Permissions, error) {
// on it rather than at the next unrelated change (ADR 0547 §2). // on it rather than at the next unrelated change (ADR 0547 §2).
sub = append(sub, VaultSecretChanged) sub = append(sub, VaultSecretChanged)
} }
// And, while the vault serves `sign`, its own node's certificate in its own name, the vault's public signing
// halves, read and watched, and the controller's lease key, read and watched, which says when the controller's key
// is no longer trusted (ADR 0550 §2). Read only.
if p.AsksTheVaultToSign {
pub = append(pub, VaultSignCall(p))
pub = append(pub, readGrants(delivery.SignersBucket, "")...)
pub = append(pub, readGrants(LeaseBucket, leaseKey)...)
}
// And its node seat, node-engine, and its asks to the operator in its own name (novox/hq ADR 0563 §1, §3).
ep, es := engineGrants(p)
pub = append(pub, ep...)
sub = append(sub, es...)
// And, on the node holding mesh-broker, the grants part's sealed user list it follows from take-over (novox/hq ADR // And, on the node holding mesh-broker, the grants part's sealed user list it follows from take-over (novox/hq ADR
// 0547 clause 7): one key, and an object by its name. // 0547 clause 7): one key, and an object by its name.
if p.FollowsGrantsUserList { if p.FollowsGrantsUserList {
@@ -972,6 +997,8 @@ func PermissionsFor(p Principal) (Permissions, error) {
// And creating each stream a manifest declares, by its own name (the operator, 2026-10-26 08:54; issue // And creating each stream a manifest declares, by its own name (the operator, 2026-10-26 08:54; issue
// 1790): the create and the read of that one stream, never an update, a delete or a stream not declared. // 1790): the create and the read of that one stream, never an update, a delete or a stream not declared.
pub = append(pub, CreateStreamGrants(p.CreatesStreams)...) pub = append(pub, CreateStreamGrants(p.CreatesStreams)...)
// And creating each durable consumer an account needs, by its stream and name (ADR 0561 §6).
pub = append(pub, CreateConsumerGrants(p.CreatesConsumers)...)
// And its own object bucket, where it writes the user list sealed to the bus node (ADR 0547 clause 7). // And its own object bucket, where it writes the user list sealed to the bus node (ADR 0547 clause 7).
pub = append(pub, GrantsUserListWrites()...) pub = append(pub, GrantsUserListWrites()...)
} }
@@ -1659,6 +1686,21 @@ func CreateStreamGrants(streams []string) []string {
return out return out
} }
// CreateConsumerGrants are the subjects that create each consumer by its stream and name, and read its info back: the
// create of the client library's CreateConsumer for a consumer with no filter or several, never one filter's form (a
// wildcard in a filter would widen the grant), an update or a delete.
func CreateConsumerGrants(consumers []ModuleConsumer) []string {
var out []string
for _, mc := range consumers {
c := mc.Consumer
if !safeSubject.MatchString(c.Stream) || !safeSubject.MatchString(c.Name) || len(c.Filters) == 1 {
continue
}
out = append(out, "$JS.API.CONSUMER.CREATE."+c.Stream+"."+c.Name, "$JS.API.CONSUMER.INFO."+c.Stream+"."+c.Name)
}
return out
}
// GrantsModule is the one module whose principal publishes the modules' memberships (novox/hq ADR 0547 §5): the grants // GrantsModule is the one module whose principal publishes the modules' memberships (novox/hq ADR 0547 §5): the grants
// part, which composes them from the manifests and placement. It defines no stream and is granted no other subject here. // part, which composes them from the manifests and placement. It defines no stream and is granted no other subject here.
const GrantsModule = "mesh-grants" const GrantsModule = "mesh-grants"
@@ -1827,6 +1869,11 @@ func VaultOwnCall(p Principal) string {
return "mesh.seat." + VaultOwn.Seat + "." + CallKind + "." + VaultOwn.Verb + "." + CallerToken(p.Username()) return "mesh.seat." + VaultOwn.Seat + "." + CallKind + "." + VaultOwn.Verb + "." + CallerToken(p.Username())
} }
// VaultSignCall is the subject a machine principal's node-engine asks the vault's `sign` on, in its own name.
func VaultSignCall(p Principal) string {
return "mesh.seat." + VaultSign.Seat + "." + CallKind + "." + VaultSign.Verb + "." + CallerToken(p.Username())
}
// VaultSecretChanged is the vault's event that a secret changed (a name and its generation, never a value), which a // VaultSecretChanged is the vault's event that a secret changed (a name and its generation, never a value), which a
// node composing its own hears to compose again (issue 1126, ADR 0547 §2). // node composing its own hears to compose again (issue 1126, ADR 0547 §2).
var VaultSecretChanged = "mesh.mod." + VaultOwn.Seat + ".event.secret-changed" var VaultSecretChanged = "mesh.mod." + VaultOwn.Seat + ".event.secret-changed"
+21
View File
@@ -0,0 +1,21 @@
{
"how": "Written by mesh-controller 0fbb5ef's own issueWith, sealProvisionerClient and heldProcessAccount (cmd/mesh-controller), run unchanged in a scratch integration test against a local store with the default seats seeded, on 2026-10-27 (issue 774): each row is the exact value the controller handed the vault. Never edit a row by hand: capture it again.",
"rows": [
{
"name": "module",
"held": "{\"url\":\"nats://10.0.0.1:4222\",\"fingerprint\":\"ab:cd\",\"node\":\"laptop\",\"module\":\"cap-bot\",\"user\":\"laptop.cap-bot\",\"password\":\"pw-module\",\"claims\":[{\"seat\":\"mesh-vault\",\"scope\":\"mesh\",\"serves\":[\"get\",\"accept\",\"secret-ask\",\"issued\",\"rotate\",\"move-in\",\"node-key\",\"own\",\"move-asks\"]},{\"seat\":\"node-service-manager\",\"scope\":\"node\",\"serves\":[\"status\"]},{\"seat\":\"mesh-controller\",\"scope\":\"mesh\",\"serves\":[\"tools\",\"calls\",\"status\",\"nodes\",\"node\",\"placement-export\",\"modules\",\"seats\",\"builds\",\"plans\",\"delivery-plan\",\"delivery-order\",\"delivery-check\",\"deliver\",\"delivery-stop\",\"delivery-backups-again\",\"delivery-walks\",\"plan\",\"assign\",\"unassign\",\"pin\",\"unpin\",\"push\",\"rotate\",\"give\",\"issued\",\"secret-move\",\"secret-ask\",\"issue\",\"token\",\"settings\",\"command\",\"queue\",\"cancel\",\"clear\",\"replay\",\"try\",\"kill\",\"pause\",\"resume\",\"hand-act\",\"drill\",\"warranted\",\"hand-acts\",\"durations\",\"conditions\",\"healers\",\"doctor\",\"upgrade\",\"bus\",\"retire\",\"cleanup\",\"judge-directories\",\"root-free\",\"dead-letters\",\"data\",\"build\",\"artifacts\",\"held\",\"collect\",\"move-placement\",\"move-config\",\"mirrors\",\"images\"]},{\"seat\":\"no-such-seat\",\"scope\":\"node\"}]}"
},
{
"name": "module-no-claims-no-fingerprint",
"held": "{\"url\":\"nats://10.0.0.1:4222\",\"node\":\"laptop\",\"module\":\"cap-plain\",\"user\":\"laptop.cap-plain\",\"password\":\"pw-plain\"}"
},
{
"name": "provisioner",
"held": "{\"url\":\"nats://10.0.0.1:4222\",\"fingerprint\":\"ab:cd\",\"node\":\"laptop\",\"user\":\"provisioner.laptop\",\"password\":\"pw-prov\"}"
},
{
"name": "process",
"held": "{\"url\":\"nats://10.0.0.1:4222\",\"fingerprint\":\"ab:cd\",\"node\":\"laptop\",\"module\":\"mesh-mail\",\"user\":\"laptop.mesh-mail.send\",\"password\":\"pw-proc\"}"
}
]
}
+6 -2
View File
@@ -77,8 +77,12 @@ func grantsOwnObjects(p Principal, pattern string) bool {
// readers too (announce.State.Readers); both are honoured. // readers too (announce.State.Readers); both are honoured.
const VaultBusAccounts = "mesh-vault.bus-accounts" const VaultBusAccounts = "mesh-vault.bus-accounts"
// NamedReaders are the states only the modules named may read, by `<owner>.<name>`. // NamedReaders are the states only the modules named may read, by `<owner>.<name>`: the grants part's wanted accounts
var NamedReaders = map[string][]string{VaultBusAccounts: {GrantsModule}} // are read by the vault alone (novox/hq ADR 0561 §2).
// The controller's lease is read by the parts that turn on at its write (ADR 0561 §3, §6), and by the assigner, which
// reads the catalogue's placement alone from then (ADR 0549 §5, issue 774).
var NamedReaders = map[string][]string{VaultBusAccounts: {GrantsModule}, GrantsAccounts: {VaultSeat},
ControllerLease: {VaultSeat, GrantsModule, delivery.AssignerModule}}
// ReaderMay says whether module may read the state read (`<owner>.<name>`) by the named readers: always, for a state // ReaderMay says whether module may read the state read (`<owner>.<name>`) by the named readers: always, for a state
// that names none. // that names none.
+21 -1
View File
@@ -134,6 +134,11 @@ type Records struct {
// account agents run as, judged unable to become root by its node-engine, and serves no login shell // account agents run as, judged unable to become root by its node-engine, and serves no login shell
// execute. A machine absent is not free: no record of a pass is no pass. // execute. A machine absent is not free: no record of a pass is no pass.
RootFree map[string]bool RootFree map[string]bool
// Engine is the node-engine's own declaration, from its manifest (EngineModule): placed on every node though no node is
// assigned it, since every node runs it (novox/hq ADR 0563 §2, §3). Its state, logs and streams are created by the
// grants part as a placed module's are; it adds no principal, since each node's engine speaks as the node's account.
// Nil where its manifest was not read.
Engine *Declared
// Announced is each module's last announcement of the states it keeps (novox/hq ADR 0495, issue 1790), as the // Announced is each module's last announcement of the states it keeps (novox/hq ADR 0495, issue 1790), as the
// controller keeps it; a module absent has announced nothing yet. Nil where none was read. // controller keeps it; a module absent has announced nothing yet. Nil where none was read.
Announced map[string]announce.States Announced map[string]announce.States
@@ -149,6 +154,7 @@ func Users(r Records) ([]Principal, error) {
AsksTheConfig: servedOf(r, VerbsTheControllerAsksTheConfig), FilesFor: sortedCopy(r.Nodes)}} AsksTheConfig: servedOf(r, VerbsTheControllerAsksTheConfig), FilesFor: sortedCopy(r.Nodes)}}
served := servedReadsOf(r) served := servedReadsOf(r)
grants := -1
for _, node := range sortedCopy(r.Nodes) { for _, node := range sortedCopy(r.Nodes) {
witness := false witness := false
var checks []string var checks []string
@@ -164,7 +170,7 @@ func Users(r Records) ([]Principal, error) {
} }
out = append(out, Principal{Kind: KindNode, Node: node, WitnessesController: witness, Checks: checks, out = append(out, Principal{Kind: KindNode, Node: node, WitnessesController: witness, Checks: checks,
MovesIntoTheVault: servesMoveIn(r), StatesItsKeyToTheVault: servesVerb(r, VaultNodeKey), MovesIntoTheVault: servesMoveIn(r), StatesItsKeyToTheVault: servesVerb(r, VaultNodeKey),
AsksItsOwnSecrets: servesVerb(r, VaultOwn), AsksItsOwnSecrets: servesVerb(r, VaultOwn), AsksTheVaultToSign: servesVerb(r, VaultSign), Uses: asksSeatOf(r),
FollowsGrantsUserList: bus, ServedReads: servedOn(served, node)}) FollowsGrantsUserList: bus, ServedReads: servedOn(served, node)})
// **And the machine's provisioner client, on an account of its own** (novox/hq ADR 0349 clause 11): one // **And the machine's provisioner client, on an account of its own** (novox/hq ADR 0349 clause 11): one
// per machine that runs it, delivered to the module running it. // per machine that runs it, delivered to the module running it.
@@ -225,6 +231,7 @@ func Users(r Records) ([]Principal, error) {
// 08:54): every placed module's buckets, logs and streams, each by its stream's name. // 08:54): every placed module's buckets, logs and streams, each by its stream's name.
if d.Module == GrantsModule { if d.Module == GrantsModule {
out[len(out)-1].CreatesStreams = declaredStreamsOf(r) out[len(out)-1].CreatesStreams = declaredStreamsOf(r)
grants = len(out) - 1
} }
// **The vault's holder asks the desks and the issuing providers itself** (issue 723), under the // **The vault's holder asks the desks and the issuing providers itself** (issue 723), under the
// account its own runtime runs on (ADR 0259 §8's runs-as): the one module granted it. // account its own runtime runs on (ADR 0259 §8's runs-as): the one module granted it.
@@ -258,6 +265,11 @@ func Users(r Records) ([]Principal, error) {
out = append(out, p) out = append(out, p)
} }
// The grants part creates the durable consumers the accounts need (novox/hq ADR 0561 §6), by stream and name.
if grants >= 0 {
out[grants].CreatesConsumers = AccountConsumers(out)
}
// Refused here rather than discovered by the server. Two users with one name is a file the // Refused here rather than discovered by the server. Two users with one name is a file the
// server reads as one of them, and which one depends on the order — so a module assigned to a // server reads as one of them, and which one depends on the order — so a module assigned to a
// node twice, or a person named after nothing, is a composition that must not be written. // node twice, or a person named after nothing, is a composition that must not be written.
@@ -444,7 +456,15 @@ func runnerServes(d Declared) bool {
// name. What the grants part may create, and nothing beside. // name. What the grants part may create, and nothing beside.
func declaredStreamsOf(r Records) []string { func declaredStreamsOf(r Records) []string {
seen := map[string]bool{} seen := map[string]bool{}
placed := [][]Declared{}
for _, mods := range r.Assigned { for _, mods := range r.Assigned {
placed = append(placed, mods)
}
// The node-engine's manifest is placed on every node (ADR 0563): its declared state is created as a placed module's.
if r.Engine != nil {
placed = append(placed, []Declared{*r.Engine})
}
for _, mods := range placed {
for _, d := range mods { for _, d := range mods {
for _, b := range d.State { for _, b := range d.State {
seen["KV_"+b.Bucket()] = true seen["KV_"+b.Bucket()] = true
+5
View File
@@ -245,6 +245,11 @@ var WritersTable = []WriterRow{
{State: "a machine's have", Writer: "the node-engine of the machine, under its own keys `<node>.>`", {State: "a machine's have", Writer: "the node-engine of the machine, under its own keys `<node>.>`",
KeptIn: "key-value " + delivery.HaveBucket + ", twenty revisions of each key", KeptIn: "key-value " + delivery.HaveBucket + ", twenty revisions of each key",
Others: "read through `reads` as node-engine.have", Subjects: kvOf(delivery.HaveBucket), Writes: ownNodeKeys}, Others: "read through `reads` as node-engine.have", Subjects: kvOf(delivery.HaveBucket), Writes: ownNodeKeys},
// The acts a node's engine was asked through its node seat, and its own asks (novox/hq ADR 0563 §2, §3): the
// node-engine's declared state, created by the grants part, written by each node's engine under its own keys.
{State: "a machine's hand acts", Writer: "the node-engine of the machine, under its own keys `<node>.>`, compare-and-set",
KeptIn: "key-value " + EngineActsBucket + ", the node-engine's declared state acts",
Others: "read through the node seat's verb acts", Subjects: kvOf(EngineActsBucket), Writes: ownNodeKeys},
// The contents of a declaration's files over 64 KiB (novox/hq ADR 0486 §2): the controller alone writes the bucket, each // The contents of a declaration's files over 64 KiB (novox/hq ADR 0486 §2): the controller alone writes the bucket, each
// object under the digest of its content; a node-engine reads one by its name, with no list and no write. // object under the digest of its content; a node-engine reads one by its name, with no list and no write.
{State: "a declaration's large files", Writer: "controller, each object under the sha256 digest of its content; the " + {State: "a declaration's large files", Writer: "controller, each object under the sha256 digest of its content; the " +
+15
View File
@@ -75,6 +75,21 @@ func TestACallerInTheArgumentsIsNeverTheCaller(t *testing.T) {
if got != "shanks.node-tools" { if got != "shanks.node-tools" {
t.Fatalf("with a caller in the arguments the tool read %q, want the runtime's", got) t.Fatalf("with a caller in the arguments the tool read %q, want the runtime's", got)
} }
// A module's trusted list of callers (issue 774, ADR 0563 §8) grants the callers it names as the runtime proved them:
// a module on any machine, <machine>.<module> exactly, person.<name> exactly; no wildcard, never one unproven.
callers, err := ParseCallers([]byte(`{"callers":["node-tools","ace.claude-code","person.jo","*"]}`))
if err != nil {
t.Fatal(err)
}
for caller, want := range map[string]bool{"shanks.node-tools": true, "ace.claude-code": true, "g14.claude-code": false,
"person.jo": true, "person.node-tools": false, Unproven: false, "": false, "novox.mesh-vault": false} {
if callers.Allows(caller) != want {
t.Errorf("%q allowed: %v, want %v", caller, !want, want)
}
}
if _, err := ParseCallers([]byte(`{"callers":["x"],"more":1}`)); err == nil {
t.Error("a list with a field it does not know was read")
}
} }
// A runtime that names no caller — one from before issue 365, or a call on the subject that names none — is // A runtime that names no caller — one from before issue 365, or a call on the subject that names none — is
+48
View File
@@ -0,0 +1,48 @@
package stdio
import (
"bytes"
"encoding/json"
"fmt"
"strings"
)
// Callers is a module's trusted list of who may call a verb of its (novox/hq ADR 0459 §2, ADR 0563 §8), as the mesh
// writes a list setting into a file the module reads: `{"callers": [...]}`. An entry is a module's name (that module on
// any machine), exactly <machine>.<module>, or exactly person.<name>. Each caller is the runtime's proof (CallerOf), never
// an argument; nobody beyond the list is granted, a wildcard is no entry, and an unproven caller never matches.
type Callers []string
// ParseCallers reads a callers file: the list under `callers`, and nothing else.
func ParseCallers(raw []byte) (Callers, error) {
var f struct {
Callers Callers `json:"callers"`
}
dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields()
if err := dec.Decode(&f); err != nil {
return nil, fmt.Errorf("the callers setting is not a list of callers: %v", err)
}
return f.Callers, nil
}
// Allows says whether caller, as the runtime proved it, is one the list names.
func (c Callers) Allows(caller string) bool {
if caller == "" || caller == Unproven {
return false
}
for _, entry := range c {
if entry == "" || strings.ContainsAny(entry, "*> ") {
continue
}
if entry == caller {
return true
}
if !strings.Contains(entry, ".") {
if _, module, ok := strings.Cut(caller, "."); ok && module == entry && !strings.HasPrefix(caller, "person.") {
return true
}
}
}
return false
}
+37
View File
@@ -0,0 +1,37 @@
package catalogue
// The delivery's checker (novox/hq ADR 0547 §10, issue 774): one holder for the mesh, the module
// mesh-delivery-checker, which judges a pull request's head and a pushed head the same way. The try the
// controller forwarded to it is a verb of this seat, answered by the checker with the verdict it gives a pull
// request, so the try outlives the controller.
// CheckerSeat is the seat mesh-delivery-checker holds.
const CheckerSeat = "mesh-delivery-checker"
// checkerVerbs are the checker seat's verbs: ask a try, read one, end one by hand. Optional until the checker serves them
// everywhere.
func checkerVerbs() []Verb {
return []Verb{
{Name: "try", Optional: true, Description: "Try a pushed head before its pull request is opened: the checker " +
"judges it as it judges a pull request's head, merged onto its trunk's head, its unit and integration " +
"levels through the build modules, and owes the forge nothing. Answers the try's id at once; try-read " +
"reads where it stands. A try of the same head that waits or runs is the answer to a second ask; at most " +
"4 wait or run at once, mesh-wide.",
Input: schema(map[string]string{"repository": "<owner>/<name>",
"head": "the full commit to try, 40 or 64 hex digits",
"trunk": "the trunk commit to merge it onto, or the trunk's branch for its head now; the trunk's head when absent",
"tests": "a go test -run pattern: only those tests run, and nothing is kept",
"level": "unit or integration: only this level; both when absent"},
[]string{"repository", "head"}),
Replaces: []string{"sh merge-check.sh", "docker run postgres"}},
{Name: "try-read", Optional: true, Description: "Where a try stands, as stored: queued, running, or its end " +
"(pass, fail, or error when it could not run), with what was said of it, the head and the trunk commit " +
"it was merged onto, the time it took and what failed first. Read-only.",
Input: schema(map[string]string{"id": "the try's id, try-<digits>"}, []string{"id"})},
{Name: "try-end", Optional: true, Description: "End a try that waits or runs, as error, by hand: say why (kept in " +
"what it says). Its place among the tries is free at once and the next try of its head asks anew with a new " +
"id. A try that ended already is refused.",
Input: schema(map[string]string{"id": "the try's id, try-<digits>", "why": "why it is ended, in words"},
[]string{"id", "why"})},
}
}
+57
View File
@@ -0,0 +1,57 @@
package catalogue
// The node seat node-engine (novox/hq ADR 0563 §1): held on every node by that node's engine (mesh-host), which serves
// its verbs on its own bus account. No module claims it and no assignment holds it: what the mesh knows of its holder
// is the node's record and the engine's haves (ADR 0563 §1 changes ADR 0110 §6 and §7 for this one seat).
//
// Its verbs carry no signature (§7): the runtime proves the caller, which matters only for the record. The read-only
// ones change nothing or are bounded in the engine's own state (§2); hand-over and access-act perform nothing on the
// call, and act only on the operator's warrant to the engine's own ask (§3).
// EngineSeat is the node seat every node's engine holds.
const EngineSeat = "node-engine"
// EngineVerbs are the node-engine seat's verbs, in the order ADR 0563 §1 names them.
func EngineVerbs() []Verb {
return []Verb{
{Name: "hand-over", Description: "Hand a directory this machine uses as found to the mesh (novox/hq ADR 0563 §3, " +
"§5): the engine judges the path as it does every hand-over (absolute, as it states it, used as found, " +
"reached through no link) and refuses in words, or asks the operator on their channel at the level approve, " +
"binding the owner, group and mode it will set. Nothing changes on the call: only the operator's yes hands " +
"it over. One ask is open per path; acts shows where it stands.",
Input: schema(map[string]string{"path": "the directory's absolute path, as the engine states it",
"why": "why it is asked, in words the operator reads"}, []string{"path"})},
{Name: "access-act", Description: "Ask the operator for the take act on a library whose root holds data outside its " +
"group (novox/hq ADR 0328 §7, ADR 0563 §3, §4), on the library's home engine: the access as declared and its " +
"root as last declared, judged and refused in words or asked at the level approve. Only the operator's yes " +
"sets the library's group, group write and setgid under its root. One ask is open per access.",
Input: schema(map[string]string{"act": "the act: take", "access": "the access, library:<name>",
"why": "why it is asked, in words the operator reads"}, []string{"act", "access"})},
{Name: "judge-directories", Description: "Which of this machine's declared directories the engine would refuse to " +
"make or keep, one line each (novox/hq ADR 0563 §2). Read-only: changes no owner or mode.",
Input: schema(map[string]string{}, nil)},
{Name: "setuid-search", Description: "Throw away the engine's last search for setuid programs and start a full one " +
"(novox/hq ADR 0563 §2); until it completes root-free answers that the machine is not judged yet. Refused while " +
"one runs, and when one was started within the hour, naming when the next may start.",
Input: schema(map[string]string{"why": "why it is asked, in words"}, nil)},
{Name: "root-free", Description: "Whether this machine is root-free (novox/hq ADR 0259 §8, ADR 0563 §6): free only " +
"on the engine's own verdict, at most fifteen minutes old, that the account agents run as cannot become root " +
"without a person, and the login shell's execute not served here. Anything else is no pass, and says why. " +
"Answered by this machine's engine alone.",
Input: schema(map[string]string{}, nil)},
{Name: "acts", Description: "Every act this machine's engine was asked through this seat (novox/hq ADR 0563 §9): " +
"each hand-over, access act and setuid search, open, done or refused, with who asked, the ask and the " +
"operator's warrant. Read-only.",
Input: schema(map[string]string{"limit": "at most this many, newest", "id": "one act, by its id"}, nil)},
}
}
// EngineEvents are what the seat says of an act, each naming the act, the path or access, the caller as the runtime
// proved it and the ask (ADR 0563 §1).
var EngineEvents = []string{"asked", "done", "refused"}
// engineSeat is the seat's row in the closed set.
func engineSeat() Seat {
return Seat{Name: EngineSeat, Scope: ScopeNode, Serves: EngineVerbs(), Emits: append([]string(nil), EngineEvents...),
HeldByTheEngine: true, Decision: "novox/hq ADR 0563"}
}
+31 -2
View File
@@ -67,6 +67,10 @@ type Seat struct {
// names paths. Compiled, never stored, like Receives: adding or taking away a path is a decision, // names paths. Compiled, never stored, like Receives: adding or taking away a path is a decision,
// recorded, and the store's rows carry no column for it. // recorded, and the store's rows carry no column for it.
Paths []SeatPath Paths []SeatPath
// HeldByTheEngine says the seat is held on every node by that node's engine, which serves its verbs on its own bus
// account (novox/hq ADR 0563 §1): no module claims it and no assignment holds it. Compiled, never stored, like
// Receives.
HeldByTheEngine bool
// Decision is the record that made it a seat. // Decision is the record that made it a seat.
Decision string Decision string
} }
@@ -171,7 +175,8 @@ var defaultSeats = append([]Seat{
"origin": "issued for a bus account the controller minted for a core or moved module and hands " + "origin": "issued for a bus account the controller minted for a core or moved module and hands " +
"at once (ADR 0374, issue 787); left out for a value given at the terminal"}), nil)}, "at once (ADR 0374, issue 787); left out for a value given at the terminal"}), nil)},
{Name: "secret-ask", Optional: true, Description: "Ask the operator for a module's own secret at a desk, " + {Name: "secret-ask", Optional: true, Description: "Ask the operator for a module's own secret at a desk, " +
"from the controller alone: the ask is recorded in the vault's store and bounded before the prompt " + "from the controller or a caller the vault's manifest names, named as the bus proved it (novox/hq ADR " +
"0563 §8): the ask is recorded in the vault's store and bounded before the prompt " +
"opens, and what is typed is sealed to a key the vault made for the call and opens there alone.", "opens, and what is typed is sealed to a key the vault made for the call and opens there alone.",
Input: schema(vaultIntake(map[string]string{"node": "the machine the secret is for", Input: schema(vaultIntake(map[string]string{"node": "the machine the secret is for",
"module": "the module", "name": "the secret's name in the module", "module": "the module", "name": "the secret's name in the module",
@@ -224,6 +229,18 @@ var defaultSeats = append([]Seat{
Input: map[string]any{"type": "object", "properties": map[string]any{"asks": map[string]any{ Input: map[string]any{"type": "object", "properties": map[string]any{"asks": map[string]any{
"type": "array", "items": map[string]any{"type": "object"}, "type": "array", "items": map[string]any{"type": "object"},
"description": "the controller's rows: id, node, module, name, desk, asked_by, opened_at, ended_at, outcome"}}}}, "description": "the controller's rows: id, node, module, name, desk, asked_by, opened_at, ended_at, outcome"}}}},
{Name: "issue-account", Optional: true, Description: "Mint one bus account the grants part wants, now (novox/hq " +
"ADR 0561 §9): only while the controller's lease is handed away, never a node's account or the " +
"controller's, never anew for one the vault holds a value or an adopted hash for. Never the value.",
Input: schema(map[string]string{"user": "the bus user, e.g. ace.mesh-build-go"}, []string{"user"})},
{Name: "sign", Optional: true, Description: "Sign with the vault's key or authority (novox/hq ADR 0550 §2): " +
"kind node-certificate signs the certificate of the calling machine's internal name for the serving key " +
"its record states, built from bus state alone and asked by that machine's node-engine only; kind " +
"enrolment signs an enrolment reply under its own context, for a caller the adoption record names. " +
"Answers the certificate or the signature and the signer's fingerprint, and says signed; never a key.",
Input: schema(map[string]string{"kind": "node-certificate or enrolment",
"node": "the machine, for a node certificate: the caller's own",
"reply": "the enrolment reply's bytes in base64, for an enrolment"}, []string{"kind"})},
}}, }},
// The owner of every config value (novox/hq ADR 0348 §9, design 57 K3): its holder alone writes the buckets // The owner of every config value (novox/hq ADR 0348 §9, design 57 K3): its holder alone writes the buckets
// mesh-config_values, _resolved, _predefined and _proposals, and serves the one verb `config` (§1, §10). // mesh-config_values, _resolved, _predefined and _proposals, and serves the one verb `config` (§1, §10).
@@ -261,6 +278,8 @@ var defaultSeats = append([]Seat{
// Placement (novox/hq ADR 0567): which module runs on which node, state on the bus held by the assigner, changed only by // Placement (novox/hq ADR 0567): which module runs on which node, state on the bus held by the assigner, changed only by
// a proposal the nodes answer and the operator approves. Its states and events are the holder module's. // a proposal the nodes answer and the operator approves. Its states and events are the holder module's.
{Name: PlacementSeat, Scope: ScopeMesh, Serves: placementVerbs(), Decision: "novox/hq ADR 0567"}, {Name: PlacementSeat, Scope: ScopeMesh, Serves: placementVerbs(), Decision: "novox/hq ADR 0567"},
// The delivery's checker (novox/hq ADR 0547 §10, issue 774): the try is its verb, not the controller's.
{Name: CheckerSeat, Scope: ScopeMesh, Serves: checkerVerbs(), Decision: "novox/hq ADR 0547"},
// Deferred renames (novox/hq ADR 0121): these deliver a provision, so renaming them is a // Deferred renames (novox/hq ADR 0121): these deliver a provision, so renaming them is a
// delivering-seat migration with a mesh-wide cascade if a holder stops resolving mid-flight. // delivering-seat migration with a mesh-wide cascade if a holder stops resolving mid-flight.
// They keep their names until that migration is done deliberately, apart from the node-* pass. // They keep their names until that migration is done deliberately, apart from the node-* pass.
@@ -442,6 +461,8 @@ var defaultSeats = append([]Seat{
// A node's files, read and managed (novox/hq ADR 0394, files.go): beside node-reads until mesh-files is assigned // A node's files, read and managed (novox/hq ADR 0394, files.go): beside node-reads until mesh-files is assigned
// on every node, then node-reads goes (ADR 0394 §9). // on every node, then node-reads goes (ADR 0394 §9).
{Name: FilesSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0394", Serves: filesVerbs()}, {Name: FilesSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0394", Serves: filesVerbs()},
// The node's engine's own verbs and asks (novox/hq ADR 0563 §1): held by the engine on every node, by no module.
engineSeat(),
}, },
// The graphical session's roles (novox/hq ADR 0208), last because they are a workstation's. // The graphical session's roles (novox/hq ADR 0208), last because they are a workstation's.
graphicalSessionSeats()...) graphicalSessionSeats()...)
@@ -515,7 +536,8 @@ func UseSeats(s []Seat) {
row.Paths = d.Paths row.Paths = d.Paths
row.Replicated = d.Replicated row.Replicated = d.Replicated
row.PulledByKind = d.PulledByKind row.PulledByKind = d.PulledByKind
row.Yields = d.Yields // never stored (ADR 0553) row.Yields = d.Yields // never stored (ADR 0553)
row.HeldByTheEngine = d.HeldByTheEngine // never stored (ADR 0563)
row.Serves = optionalAsCompiled(row.Serves, d.Serves) row.Serves = optionalAsCompiled(row.Serves, d.Serves)
} }
merged = append(merged, row) merged = append(merged, row)
@@ -628,6 +650,13 @@ func claimProblems(m Manifest) []string {
} }
for _, c := range m.Claims { for _, c := range m.Claims {
if s, known := SeatNamed(c.Name); known && s.HeldByTheEngine {
// The node's engine holds it on every node, and no module may (novox/hq ADR 0563 §1): a module answering
// root-free or a hand-over would be an agent answering it.
problems = append(problems, fmt.Sprintf("%s claims %s, which every node's engine holds itself and no "+
"module may (novox/hq ADR 0563 §1)", m.Module, c.Name))
continue
}
if _, known := SeatNamed(c.Name); known { if _, known := SeatNamed(c.Name); known {
// **A seat's scope and what it delivers are not judged here** (novox/hq ADR 0122). // **A seat's scope and what it delivers are not judged here** (novox/hq ADR 0122).
// This function runs wherever a manifest is parsed, and one of those places is the // This function runs wherever a manifest is parsed, and one of those places is the
+40 -2
View File
@@ -49,6 +49,43 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
[]string{"placement", "proposals", "propose", "withdraw", "again", "adopt", "hand-back"}) { []string{"placement", "proposals", "propose", "withdraw", "again", "adopt", "hand-back"}) {
t.Errorf("the placement seat is %v, %v", ok, s) t.Errorf("the placement seat is %v, %v", ok, s)
} }
// The checker's seat serves the try, which replaces the shell's merge check (ADR 0547 §10, ADR 0245).
var checkerVerbs []string
for _, s := range Seats() {
if s.Name == CheckerSeat {
for _, v := range s.Serves {
checkerVerbs = append(checkerVerbs, v.Name)
if v.Name == "try" && strings.Join(v.Replaces, "|") != "sh merge-check.sh|docker run postgres" {
t.Errorf("try replaces %v", v.Replaces)
}
}
}
}
if strings.Join(checkerVerbs, ",") != "try,try-read,try-end" {
t.Errorf("the checker seat serves %v", checkerVerbs)
}
// The node seat node-engine is the node's engine's (novox/hq ADR 0563 §1): at node scope, delivering nothing, its six
// verbs and three events, held by no assignment, and a module's claim of it refused.
if engine, ok := SeatNamed(EngineSeat); !ok || engine.Scope != ScopeNode || engine.Delivers != "" || !engine.HeldByTheEngine {
t.Errorf("node-engine is %+v, not a node seat delivering nothing that the engine holds", engine)
} else {
var verbs []string
for _, v := range engine.Serves {
verbs = append(verbs, v.Name)
}
// acts reads one act by its id too (ADR 0563 §9: the seat's tools are its command line).
if props, _ := engine.Serves[len(engine.Serves)-1].Input["properties"].(map[string]any); props["id"] == nil {
t.Errorf("acts takes no id: %v", engine.Serves[len(engine.Serves)-1].Input)
}
if strings.Join(verbs, ",") != "hand-over,access-act,judge-directories,setuid-search,root-free,acts" ||
strings.Join(engine.Emits, ",") != "asked,done,refused" {
t.Errorf("node-engine serves %v and emits %v", verbs, engine.Emits)
}
}
if _, err := ParseManifest([]byte(`{"module":"thing","version":"1","claims":[{"name":"node-engine","scope":"node"}]}`)); err == nil ||
!strings.Contains(err.Error(), "node-engine") {
t.Errorf("a module's claim of node-engine was not refused: %v", err)
}
// Forty-six with the three build seats replacing node-build-agent and the retired mesh-build-machine (novox/hq // Forty-six with the three build seats replacing node-build-agent and the retired mesh-build-machine (novox/hq
// ADR 0525 §2, §10); forty-five with node-files (novox/hq ADR 0394); forty-four again once node-reads goes. // ADR 0525 §2, §10); forty-five with node-files (novox/hq ADR 0394); forty-four again once node-reads goes.
// Forty-four with node-provisioner (novox/hq ADR 0349 clause 3); forty-three with mesh-provisioner (novox/hq ADR 0349 clause 3); forty-two with mesh-config (novox/hq ADR 0348 §9); forty-one with node-reads (novox/hq ADR 0310); forty with node-nfs-server and node-mounts (novox/hq ADR 0263); thirty-eight with node-resolver (novox/hq ADR 0247); thirty-seven with mesh-delivery (novox/hq ADR 0239); thirty-six since node-resolver-config retired // Forty-four with node-provisioner (novox/hq ADR 0349 clause 3); forty-three with mesh-provisioner (novox/hq ADR 0349 clause 3); forty-two with mesh-config (novox/hq ADR 0348 §9); forty-one with node-reads (novox/hq ADR 0310); forty with node-nfs-server and node-mounts (novox/hq ADR 0263); thirty-eight with node-resolver (novox/hq ADR 0247); thirty-seven with mesh-delivery (novox/hq ADR 0239); thirty-six since node-resolver-config retired
@@ -62,8 +99,9 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
// node-container-runtime (ADR 0207); nineteen with node-environment and node-login-shell (ADR 0203, // node-container-runtime (ADR 0207); nineteen with node-environment and node-login-shell (ADR 0203,
// ADR 0204); seventeen with node-build-agent (ADR 0190). One fewer once the retired // ADR 0204); seventeen with node-build-agent (ADR 0190). One fewer once the retired
// mesh-build-machine row goes, when no registered manifest claims it. // mesh-build-machine row goes, when no registered manifest claims it.
if len(Seats()) != 47 { // Forty-eight with mesh-delivery-checker (novox/hq ADR 0547 §10, issue 774); forty-nine with node-engine (novox/hq ADR 0563 §1).
t.Errorf("the mesh defines %d seats rather than 47; the set is closed, so a change here is "+ if len(Seats()) != 49 {
t.Errorf("the mesh defines %d seats rather than 49; the set is closed, so a change here is "+
"a decision (novox/hq ADR 0110): %s", len(Seats()), seatNames()) "a decision (novox/hq ADR 0110): %s", len(Seats()), seatNames())
} }
} }
+9
View File
@@ -318,6 +318,15 @@ func Stub(m catalogue.Manifest) catalogue.Manifest {
m.OwnsPaths = catalogue.DeclaredPaths(m) m.OwnsPaths = catalogue.DeclaredPaths(m)
} }
m.Build = nil m.Build = nil
// Nor is a template its build kept for its own files handed: a stub carries none of them (issue 1126: novox's node
// refused "postgres: its build kept a template for consumers, which it does not declare" with postgres held). Its
// items stay, and the templates they render through: a held module's contributions to other modules' collections
// (seat contributions, shell blocks, seat data) still reach their collectors.
if m.Handed != nil {
h := *m.Handed
h.Templates = nil
m.Handed = &h
}
var dirs []map[string]any var dirs []map[string]any
for _, r := range m.Resources { for _, r := range m.Resources {
if r["type"] == "directory" { if r["type"] == "directory" {
+49 -11
View File
@@ -209,21 +209,59 @@ func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]st
return a return a
} }
// heldAsToday moves each refusal of a module the node holds today (Scope.HeldToday) into Held: the candidate holds it as // heldBehind is why a module is held with a held one it reads (composeEach's dependents).
// the node does, which is no change (issue 1126: a seed reproduces today, and today holds it too). func heldBehind(m string) string {
func heldAsToday(a *delivery.CandidateAnswer, today []string) { return "it reads what " + m + " contributes or listens on, and " + m + " is held"
}
// behind is the held module a refusal's words say it is held behind, or "".
func behind(why string) string {
rest, ok := strings.CutPrefix(why, "it reads what ")
if !ok {
return ""
}
m, _, ok := strings.Cut(rest, " contributes or listens on, and ")
if !ok || why != heldBehind(m) {
return ""
}
return m
}
// heldAsToday moves into Held each refusal of a module the node keeps today as it runs (Scope.HeldToday, with how:
// held, failed, running a build of the old form), and of each module held only behind one of those: the candidate keeps
// them as the node does, which is no change (issue 1126: a seed reproduces today, and today keeps them so). Each says
// how it stands today (ModuleRefused.Today).
func heldAsToday(a *delivery.CandidateAnswer, today map[string]string) {
if len(today) == 0 { if len(today) == 0 {
return return
} }
kept := a.Refusals[:0] how := map[string]string{}
for _, r := range a.Refusals { for m, w := range today {
if slices.Contains(today, r.Module) { how[m] = w
a.Held = append(a.Held, r) }
} else { for moved := true; moved; {
kept = append(kept, r) moved = false
kept := a.Refusals[:0]
for _, r := range a.Refusals {
w, keep := how[r.Module]
if !keep {
if m := behind(r.Why); m != "" {
if _, behindKept := how[m]; behindKept {
w, keep = "held behind "+m, true
}
}
}
if keep {
r.Today = w
how[r.Module] = w
a.Held = append(a.Held, r)
moved = true
} else {
kept = append(kept, r)
}
} }
a.Refusals = kept
} }
a.Refusals = kept
if len(a.Refusals) == 0 { if len(a.Refusals) == 0 {
a.Refusals = nil a.Refusals = nil
} }
@@ -313,7 +351,7 @@ func composeEach(in compose.Input, node string, fps map[string]string, applied b
// What the others read of its manifest (what it contributes, what it listens on) is its new manifest's, not // What the others read of its manifest (what it contributes, what it listens on) is its new manifest's, not
// necessarily the build's that runs: what reads it is held with it (review of the apply, round 2). // necessarily the build's that runs: what reads it is held with it (review of the apply, round 2).
for _, r := range dependents(in, node, src[m], failed) { for _, r := range dependents(in, node, src[m], failed) {
hold(r, "it reads what "+m+" contributes or listens on, and "+m+" is held", false) hold(r, heldBehind(m), false)
} }
} }
for _, m := range sortedKeys(failed) { for _, m := range sortedKeys(failed) {
+35 -2
View File
@@ -93,17 +93,25 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
} }
// Issue 1126: a module the node holds today that the candidate holds too is held as today, no refusal: with every // Issue 1126: a module the node holds today that the candidate holds too is held as today, no refusal: with every
// refused module held today the verdict is no refusal, and each is named under Held. // refused module held today the verdict is no refusal, and each is named under Held.
today := Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b), HeldToday: []string{"api", "spiegel", "web"}}, today := Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b), HeldToday: map[string]string{"api": "held", "spiegel": "held", "web": "held"}},
map[string]string{"lemurs": fp("e"), "web": fp("1"), "api": fp("2"), "spiegel": fp("3")}, nil, "p-h1", 3) map[string]string{"lemurs": fp("e"), "web": fp("1"), "api": fp("2"), "spiegel": fp("3")}, nil, "p-h1", 3)
if today.Verdict == delivery.CandidateRefused || len(today.Refusals) != 0 || len(today.Held) != 3 || today.Held[0].Why == "" { if today.Verdict == delivery.CandidateRefused || len(today.Refusals) != 0 || len(today.Held) != 3 || today.Held[0].Why == "" {
t.Errorf("held as today: %+v", today) t.Errorf("held as today: %+v", today)
} }
// A refused module the node does not hold today stays a refusal: holding only api today leaves spiegel and web refused. // A refused module the node does not hold today stays a refusal: holding only api today leaves spiegel and web refused.
part := Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b), HeldToday: []string{"api"}}, part := Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b), HeldToday: map[string]string{"api": "held"}},
map[string]string{"lemurs": fp("e"), "web": fp("1"), "api": fp("2"), "spiegel": fp("3")}, nil, "p-h1", 3) map[string]string{"lemurs": fp("e"), "web": fp("1"), "api": fp("2"), "spiegel": fp("3")}, nil, "p-h1", 3)
if part.Verdict != delivery.CandidateRefused || len(part.Refusals) != 2 || len(part.Held) != 1 || part.Held[0].Module != "api" { if part.Verdict != delivery.CandidateRefused || len(part.Refusals) != 2 || len(part.Held) != 1 || part.Held[0].Module != "api" {
t.Errorf("held as today in part: %+v", part) t.Errorf("held as today in part: %+v", part)
} }
// A module held only behind one kept today is kept as today too, saying behind which (novox's mailu behind de-spiegel).
if a := (delivery.CandidateAnswer{Refusals: []delivery.ModuleRefused{{Module: "de-spiegel", Why: "a value is unset"},
{Module: "mailu", Why: heldBehind("de-spiegel")}, {Module: "proxy", Why: heldBehind("mailu")}, {Module: "web", Why: "no build"}}}); true {
heldAsToday(&a, map[string]string{"de-spiegel": "runs a build of the old form"})
if len(a.Refusals) != 1 || a.Refusals[0].Module != "web" || len(a.Held) != 3 || a.Held[2].Today != "held behind mailu" {
t.Errorf("held behind a module kept today: %+v", a)
}
}
// The fold: spiegel runs inside the node's own composition at a build of the old form. Its resources are carried as // The fold: spiegel runs inside the node's own composition at a build of the old form. Its resources are carried as
// they run, each after the one it runs after: the node's own is the one it runs, and the node composes itself. // they run, each after the one it runs after: the node's own is the one it runs, and the node composes itself.
@@ -286,6 +294,31 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
if got, want := catalogue.DeclaredPaths(compose.Carried(f1, "f1-circuits", &f1runs)), catalogue.DeclaredPaths(f1); !reflect.DeepEqual(got, want) { if got, want := catalogue.DeclaredPaths(compose.Carried(f1, "f1-circuits", &f1runs)), catalogue.DeclaredPaths(f1); !reflect.DeepEqual(got, want) {
t.Errorf("f1-circuits carried owns %v, want what the controller composes from its manifest, %v", got, want) t.Errorf("f1-circuits carried owns %v, want what the controller composes from its manifest, %v", got, want)
} }
// Issue 1126: a held module hands no template its build kept for a file it no longer carries (novox's node:
// "postgres: its build kept a template for consumers"), but keeps its items and their templates, so what it
// contributes to other modules' collections still reaches them.
for _, row := range []struct {
name string
handed *catalogue.Handed
items []catalogue.Item
}{
{"templates and items", &catalogue.Handed{Templates: map[string]string{"consumers": "files/consumers.json"},
Items: map[string]string{"routes": "files/routes.tmpl"}}, []catalogue.Item{{Collection: "routes", Content: "x"}}},
{"items alone", nil, []catalogue.Item{{Collection: "routes", Content: "x"}}},
} {
kept := f1
kept.Handed, kept.Items = row.handed, row.items
st := compose.Stub(kept)
if st.Handed != nil && st.Handed.Templates != nil {
t.Errorf("%s: a stub hands templates %v", row.name, st.Handed.Templates)
}
if !reflect.DeepEqual(st.Items, row.items) {
t.Errorf("%s: a stub collects %+v, want its items %+v", row.name, st.Items, row.items)
}
if row.handed != nil && (st.Handed == nil || !reflect.DeepEqual(st.Handed.Items, row.handed.Items) || row.handed.Templates == nil) {
t.Errorf("%s: a stub hands item templates %+v, want %v and its manifest untouched", row.name, st.Handed, row.handed.Items)
}
}
var k1, k2 any var k1, k2 any
_ = json.Unmarshal(keptComposed, &k1) _ = json.Unmarshal(keptComposed, &k1)
_ = json.Unmarshal(keptFile, &k2) _ = json.Unmarshal(keptFile, &k2)
+21 -4
View File
@@ -83,6 +83,16 @@ type Reader interface {
Served(ctx context.Context, node, module string, keys []string) (map[string]any, error) Served(ctx context.Context, node, module string, keys []string) (map[string]any, error)
} }
// SignersReader is a Reader that also reads the vault's public authority from its state `signers` (delivery.SignersBucket,
// key delivery.SignersAuthorityKey; ADR 0550 §2). A Reader that has it composes the mesh's authority as a bundle, the
// controller's and the vault's; one that has not, or that answers none, composes the controller's alone, as before.
type SignersReader interface {
// VaultAuthority is the vault's authority certificate as PEM, checked (delivery.DecodeSignerAuthority); empty when the
// vault has published none. A read that fails is said by the reader and answered empty: a node that may not read the
// state yet composes as before rather than not at all.
VaultAuthority(ctx context.Context) string
}
// Mesh is what is true of the whole mesh and is no part's state on the bus yet: the private network's range and the // Mesh is what is true of the whole mesh and is no part's state on the bus yet: the private network's range and the
// internal names' suffix as genesis was told them, and the mesh's authority certificate (public). Empty fields take // internal names' suffix as genesis was told them, and the mesh's authority certificate (public). Empty fields take
// compose's defaults. // compose's defaults.
@@ -124,10 +134,10 @@ type Scope struct {
Placements map[string]*delivery.Placement Placements map[string]*delivery.Placement
InOwn []string InOwn []string
Running []delivery.Composed Running []delivery.Composed
// HeldToday is, in an answer, the modules the node holds today ("node" for its own composition): their have's phase // HeldToday is, in an answer, the modules the node keeps today as they run ("node" for its own composition), each with
// is held, or they are held inside its own composition. A candidate that holds one too holds it as today, which is // how ("held", "failed", "runs a build of the old form"): a candidate that cannot compose one, or holds one only
// said under CandidateAnswer.Held and is no refusal (issue 1126). // behind one of them, keeps it as today, which is said under CandidateAnswer.Held and is no refusal (issue 1126).
HeldToday []string HeldToday map[string]string
// SameValue, in an answer, says whether two sealed values open to the same value (the node opens them with its own // SameValue, in an answer, says whether two sealed values open to the same value (the node opens them with its own
// key), so a differing resource shows a value sealed again as such (delivery.CompareComposedWith). Nil shows them // key), so a differing resource shows a value sealed again as such (delivery.CompareComposedWith). Nil shows them
// as differing. // as differing.
@@ -223,6 +233,13 @@ func input(ctx context.Context, r Reader, mesh Mesh, s Scope, refuse func(module
} }
} }
} }
// And the vault's authority beside it, once the vault publishes one (ADR 0550 §2): a certificate either signed checks
// against the bundle. The controller's stays while a certificate it issued is in a node's record.
if sr, ok := r.(SignersReader); ok {
if vault := sr.VaultAuthority(ctx); vault != "" {
in.Authority = delivery.AuthorityBundle(in.Authority, vault)
}
}
parse := func(raw []byte) (catalogue.Manifest, error) { parse := func(raw []byte) (catalogue.Manifest, error) {
if mesh.Predefined != nil { if mesh.Predefined != nil {
return catalogue.ParseManifestWith(raw, mesh.Predefined) return catalogue.ParseManifestWith(raw, mesh.Predefined)
+18 -8
View File
@@ -25,14 +25,15 @@ import (
// bus is a Reader over values held in memory, counting whose config values were read. // bus is a Reader over values held in memory, counting whose config values were read.
type bus struct { type bus struct {
placements map[string]delivery.Placement vaultAuthority string
records map[string]delivery.NodeRecord placements map[string]delivery.Placement
modules map[string]delivery.Module records map[string]delivery.NodeRecord
manifests map[string][]byte modules map[string]delivery.Module
values map[string]map[string]any // by node.module manifests map[string][]byte
valuesRead []string values map[string]map[string]any // by node.module
own []compose.OwnSecretRow valuesRead []string
ownErr error own []compose.OwnSecretRow
ownErr error
// asked is every consumer credential asked; the fake refuses the whole ask, as the vault's own does, for a consuming // asked is every consumer credential asked; the fake refuses the whole ask, as the vault's own does, for a consuming
// module its placement does not put on the node. // module its placement does not put on the node.
asked []compose.CredentialKey asked []compose.CredentialKey
@@ -74,6 +75,7 @@ func (b *bus) Served(_ context.Context, node, module string, keys []string) (map
func (b *bus) Placements(context.Context) (map[string]delivery.Placement, error) { func (b *bus) Placements(context.Context) (map[string]delivery.Placement, error) {
return b.placements, nil return b.placements, nil
} }
func (b *bus) VaultAuthority(context.Context) string { return b.vaultAuthority }
func (b *bus) Records(context.Context) (map[string]delivery.NodeRecord, error) { return b.records, nil } func (b *bus) Records(context.Context) (map[string]delivery.NodeRecord, error) { return b.records, nil }
func (b *bus) Modules(context.Context) (map[string]delivery.Module, error) { return b.modules, nil } func (b *bus) Modules(context.Context) (map[string]delivery.Module, error) { return b.modules, nil }
func (b *bus) Manifest(_ context.Context, digest string) ([]byte, error) { func (b *bus) Manifest(_ context.Context, digest string) ([]byte, error) {
@@ -216,6 +218,14 @@ func TestInput(t *testing.T) {
wj, _ := json.MarshalIndent(want, "", " ") wj, _ := json.MarshalIndent(want, "", " ")
t.Fatalf("Input from the bus differs from the store's:\n got %s\nwant %s", gj, wj) t.Fatalf("Input from the bus differs from the store's:\n got %s\nwant %s", gj, wj)
} }
// L3 s8 (ADR 0550 §2): with the vault's authority published, the mesh's authority is the bundle of the controller's
// and the vault's; with none, the controller's alone, as above.
signing := mk()
signing.vaultAuthority = "the vault's certificate"
if bundled, err := Input(ctx, signing, Mesh{}, Scope{Node: "laptop"}); err != nil ||
bundled.Authority != "the mesh's certificate\nthe vault's certificate\n" {
t.Errorf("the authority with the vault's published: %q, %v", bundled.Authority, err)
}
if !reflect.DeepEqual(b.valuesRead, []string{"laptop." + overlay.Name, "laptop.xterm", "laptop.media"}) { if !reflect.DeepEqual(b.valuesRead, []string{"laptop." + overlay.Name, "laptop.xterm", "laptop.media"}) {
t.Errorf("laptop read config values %v; a node reads its own slice alone", b.valuesRead) t.Errorf("laptop read config values %v; a node reads its own slice alone", b.valuesRead)
} }
+6 -2
View File
@@ -258,9 +258,13 @@ func Brief(p delivery.PlacementProposal, v delivery.PlacementVerdict) string {
if held := v.Answers[n].Held; len(held) > 0 { if held := v.Answers[n].Held; len(held) > 0 {
var names []string var names []string
for _, h := range held { for _, h := range held {
names = append(names, h.Module) today := h.Today
if today == "" {
today = "held"
}
names = append(names, h.Module+" ("+today+")")
} }
fmt.Fprintf(&b, "; stays held as today: %s", strings.Join(names, ", ")) fmt.Fprintf(&b, "; stays as today: %s", strings.Join(names, ", "))
} }
} }
return b.String() return b.String()
+1 -1
View File
@@ -79,7 +79,7 @@ func TestJudge(t *testing.T) {
t.Errorf("the difference is named %q", got) t.Errorf("the difference is named %q", got)
} }
} }
if strings.HasPrefix(row.name, "a module held as today") && !strings.Contains(Brief(q, v), "stays held as today: spiegel") { if strings.HasPrefix(row.name, "a module held as today") && !strings.Contains(Brief(q, v), "stays as today: spiegel (held)") {
t.Errorf("%s: the brief does not show it: %s", row.name, Brief(q, v)) t.Errorf("%s: the brief does not show it: %s", row.name, Brief(q, v))
} }
if row.marked != "" && (!slices.Contains(v.Unintended["anchor"], row.marked) || !strings.Contains(Brief(q, v), "not computed: "+row.marked)) { if row.marked != "" && (!slices.Contains(v.Unintended["anchor"], row.marked) || !strings.Contains(Brief(q, v), "not computed: "+row.marked)) {
+19
View File
@@ -298,6 +298,25 @@ func TestControllerGrant(t *testing.T) {
if _, err := EncodeControllerGrant(ControllerGrant{Accounts: []string{""}}); err == nil { if _, err := EncodeControllerGrant(ControllerGrant{Accounts: []string{""}}); err == nil {
t.Errorf("a grant naming an empty account was accepted") t.Errorf("a grant naming an empty account was accepted")
} }
// The grants part's keys are the controller's: the same key round-trips; the source is the part's bucket only for a
// handed-away lease (issue 774).
if pk, err := PartGrantKey("mesh-delivery-assigner", fp); err != nil || pk != key {
t.Errorf("PartGrantKey = %q, %v, want the controller's %q", pk, err, key)
}
if m, f, err := ParsePartGrantKey(key); err != nil || m != "mesh-delivery-assigner" || f != fp {
t.Errorf("ParsePartGrantKey = %q, %q, %v", m, f, err)
}
for _, c := range []struct {
away bool
bucket, read string
}{
{false, "mesh-controller_grants", "mesh-controller.grants"},
{true, "mesh-grants_grants", "mesh-grants.grants"},
} {
if b, r := GrantSource(c.away); b != c.bucket || r != c.read {
t.Errorf("GrantSource(%v) = %q, %q, want %q, %q", c.away, b, r, c.bucket, c.read)
}
}
g := ControllerGrant{Accounts: []string{"ace.mesh-build-go", "shanks.mesh-mail.reader"}} g := ControllerGrant{Accounts: []string{"ace.mesh-build-go", "shanks.mesh-mail.reader"}}
for _, c := range []struct { for _, c := range []struct {
g ControllerGrant g ControllerGrant
+225
View File
@@ -0,0 +1,225 @@
package buildrun
import (
"context"
"errors"
"fmt"
"maps"
"slices"
"sync"
"time"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/process"
)
// A build module writes <node>.<language> of its toolchains state on every node it runs on, and nothing deletes the key
// when the module leaves a node: no part writes another module's state, and the module is gone. A report of a node the
// build module no longer runs on is therefore no builder's (issue 1940: mesh-build-node taken off ace on 2026-10-23 left
// ace.typescript at v26.12.0, and every Node fingerprint said "builders disagree").
//
// ToolchainGate stands between the toolchains buckets and what takes their entries in (the indexer's PutToolchain, a
// ToolchainSet): an entry is passed on only while the placement (the seat mesh-placement's state `placement`, held by
// the assigner, delivery.PlacementBucket, read by delivery.Placed; ADR 0567, ADR 0549 §3) places its build module on
// its node, and only once it was written since that placement; it is passed on as deleted otherwise, said once, and
// passed on when its node writes it again. A report written before the placement is one an earlier run of the module
// left (issue 1940, 2026-10-25 20:56: mesh-build-node put on ace again brought ace.typescript v26.12.0 of 2026-10-23
// back into the agreement before the module ran there). Until the placement is settled, and while no node's placement
// is held (before the assigner's adopt writes the first one, or where the state cannot be read), every entry is passed
// on. Builders that run and report different versions still disagree: that refusal is the guard of novox/hq ADR 0465,
// and it stays.
//
// This replaces the copies of the gate the indexer, the builder and the checker each kept over the controller's
// assignments (issue 1940's subtask, issue 774: those go with the controller's lease, ADR 0549 §6).
type ToolchainGate struct {
mu sync.Mutex
modules map[string]bool // the build modules whose toolchains pass the gate
forward func(module, key string, value []byte, deleted bool) bool // answers whether it changed what it holds
logf func(format string, a ...any)
placed *delivery.Placed
known bool // the placement is settled
raw map[string]map[string][]byte // module -> toolchains key -> value, as the bucket holds it
left map[string]string // <module>/<key> -> why it was said left out, so each reason is said once
}
// NewToolchainGate is a gate over the toolchains of modules, passing what stands on to forward and saying with logf what
// it leaves out.
func NewToolchainGate(modules []string, forward func(module, key string, value []byte, deleted bool) bool, logf func(string, ...any)) *ToolchainGate {
g := &ToolchainGate{modules: map[string]bool{}, forward: forward, logf: logf, placed: delivery.NewPlaced(),
raw: map[string]map[string][]byte{}, left: map[string]string{}}
for _, m := range modules {
g.modules[m] = true
}
return g
}
// Toolchain takes one change of a build module's toolchains bucket in and answers whether what it passed on changed.
func (g *ToolchainGate) Toolchain(module, key string, value []byte, deleted bool) bool {
g.mu.Lock()
defer g.mu.Unlock()
if g.raw[module] == nil {
g.raw[module] = map[string][]byte{}
}
if deleted {
delete(g.raw[module], key)
} else {
g.raw[module][key] = value
}
return g.pass(module, key)
}
// Placement takes one revision of the seat mesh-placement's placement in (at is when the bus wrote it) and answers whether what it
// passed on changed. A value that does not read is said, and places nothing on its node.
func (g *ToolchainGate) Placement(key string, value []byte, deleted bool, at time.Time) bool {
g.mu.Lock()
defer g.mu.Unlock()
was := g.placed.Seeded()
node, changed, err := g.placed.Put(key, value, deleted, at)
if err != nil {
g.logf("%s: %v (issue 1940)", delivery.PlacementBucket, err)
}
if !g.known {
return false
}
if was != g.placed.Seeded() {
// The first placement came, or the last went: every report's standing changed.
return g.all()
}
return g.again(node, changed)
}
// Settled says the placement's history is read: from now on only the reports of placed nodes pass, once a node's
// placement is held. It answers whether what it passed on changed.
func (g *ToolchainGate) Settled() bool {
g.mu.Lock()
defer g.mu.Unlock()
g.known = true
return g.all()
}
// all passes on every report again. The caller holds mu.
func (g *ToolchainGate) all() bool {
changed := false
for _, module := range slices.Sorted(maps.Keys(g.raw)) {
for _, k := range slices.Sorted(maps.Keys(g.raw[module])) {
changed = g.pass(module, k) || changed
}
}
return changed
}
// again passes on every report of node by a build module among modules. The caller holds mu.
func (g *ToolchainGate) again(node string, modules []string) bool {
changed := false
for _, module := range modules {
if !g.modules[module] {
continue
}
for _, k := range slices.Sorted(maps.Keys(g.raw[module])) {
if n, _, err := delivery.ParseToolchainKey(k); err == nil && n == node {
changed = g.pass(module, k) || changed
}
}
}
return changed
}
// pass hands one key on as it stands: its value when its node is placed and wrote it since (or the placement is not
// known), deleted otherwise. The caller holds mu.
func (g *ToolchainGate) pass(module, key string) bool {
value, held := g.raw[module][key]
if !held {
return g.forward(module, key, nil, true)
}
node, _, err := delivery.ParseToolchainKey(key)
why := ""
if err == nil && g.known && g.placed.Seeded() {
why = g.leftOut(module, node, value)
}
if why == "" {
delete(g.left, module+"/"+key)
return g.forward(module, key, value, false)
}
if g.left[module+"/"+key] != why {
g.left[module+"/"+key] = why
g.logf("%s.%s is left out of the agreed toolchain: %s (issue 1940)", delivery.ToolchainsBucket(module), key, why)
}
return g.forward(module, key, nil, true)
}
// leftOut is why a node's report is no builder's, or empty when it counts: the catalogue does not place the module on its
// node, or the report was written before it did. A value that is no toolchain is passed on for its reader to refuse.
// The caller holds mu.
func (g *ToolchainGate) leftOut(module, node string, value []byte) string {
since, placed := g.placed.Since(node, module)
if !placed {
return fmt.Sprintf("the catalogue does not place %s on %s, so it is no builder", module, node)
}
tc, err := delivery.DecodeToolchain(value)
if err == nil && tc.At.Before(since) {
return fmt.Sprintf("it was written at %s, before %s was placed on %s at %s: it counts once %s writes it again",
tc.At.UTC().Format(time.RFC3339), module, node, since.UTC().Format(time.RFC3339), node)
}
return ""
}
// FollowPlacement reads the seat mesh-placement's placement with its history into gate, says it settled, and follows it; changed
// is told whenever what the gate passes on changed. Where the state cannot be read or watched (not there, or this part
// may not read it) it says so and the gate passes every report on: not fatal, since an indexer, builder or checker that
// stops for it stops every delivery (issue 1945). An error of the watch after that ends on the channel.
func FollowPlacement(ctx context.Context, js process.BucketOpener, gate *ToolchainGate, changed func(context.Context), logf func(string, ...any)) (<-chan error, error) {
errs := make(chan error, 1)
kv, err := js.KeyValue(ctx, delivery.PlacementBucket)
if err != nil {
logf("the state %s cannot be read, so every node's toolchain report counts, a build node's the catalogue does not place too (issue 1940): %v", delivery.PlacementBucket, err)
return errs, nil
}
w, err := kv.WatchAll(ctx, jetstream.IncludeHistory())
if err != nil {
logf("the state %s cannot be watched, so every node's toolchain report counts, a build node's the catalogue does not place too (issue 1940): %v", delivery.PlacementBucket, err)
return errs, nil
}
put := func(e jetstream.KeyValueEntry) bool {
return gate.Placement(e.Key(), e.Value(), e.Operation() != jetstream.KeyValuePut, e.Created())
}
for settled := false; !settled; {
select {
case <-ctx.Done():
_ = w.Stop()
return errs, ctx.Err()
case e, ok := <-w.Updates():
if !ok {
return errs, errors.New("the watch of " + delivery.PlacementBucket + " ended before it settled")
}
if e == nil {
settled = true
continue
}
put(e)
}
}
if gate.Settled() {
changed(ctx)
}
go func() {
defer func() { _ = w.Stop() }()
for {
select {
case <-ctx.Done():
errs <- ctx.Err()
return
case e, ok := <-w.Updates():
if !ok {
errs <- errors.New("the watch of " + delivery.PlacementBucket + " ended")
return
}
if e != nil && put(e) {
changed(ctx)
}
}
}
}()
return errs, nil
}
+238
View File
@@ -0,0 +1,238 @@
package buildrun
import (
"bytes"
"context"
"errors"
"fmt"
"log"
"strings"
"testing"
"time"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
func gateReport(t *testing.T, node, version string, at time.Time) (string, []byte) {
t.Helper()
key, err := delivery.ToolchainKey(node, "typescript")
if err != nil {
t.Fatal(err)
}
value, err := delivery.EncodeToolchain(delivery.Toolchain{Language: "typescript", Version: version, Recipe: "f59571cf4be75adf", At: at})
if err != nil {
t.Fatal(err)
}
return key, value
}
// placing is a value of the state `placement` placing modules on node.
func placing(t *testing.T, node string, modules ...string) (string, []byte) {
t.Helper()
p := delivery.Placement{Node: node}
for _, m := range modules {
p.Modules = append(p.Modules, delivery.PlacedModule{Module: m})
}
key, err := delivery.PlacementKey(node)
if err != nil {
t.Fatal(err)
}
value, err := delivery.EncodePlacement(p)
if err != nil {
t.Fatal(err)
}
return key, value
}
func agreedOf(set *ToolchainSet) (string, string) {
tc, none := delivery.AgreedToolchain(set.Entries("typescript"))
return tc.Version, none
}
// Issue 1940, the gate over the catalogue's placement (issue 774, ADR 0549 §3): a build node's report counts only while
// the placement places its build module on its node, and only once written since; before the placement is settled, and
// while no node's placement is held, every report counts. The live cases: mesh-build-node taken off ace left
// ace.typescript at v26.12.0 against v26.12.1 on the three others; put on ace again, its report of 2026-10-23 06:29 came
// back before the module ran there.
func TestToolchainGate(t *testing.T) {
old := time.Date(2026, 10, 23, 6, 29, 18, 0, time.UTC) // ace's report from its earlier run
placed := time.Date(2026, 10, 25, 19, 56, 45, 0, time.UTC) // mesh-build-node placed on ace again
three := []string{"g14", "novox", "shanks"}
start := func(t *testing.T, seeded bool) (*ToolchainGate, *ToolchainSet, *[]string) {
set := &ToolchainSet{}
said := &[]string{}
gate := NewToolchainGate([]string{GoBuildModule, NodeBuildModule}, set.Put, func(f string, a ...any) { *said = append(*said, fmt.Sprintf(f, a...)) })
for _, node := range three {
if seeded {
key, value := placing(t, node, NodeBuildModule)
gate.Placement(key, value, false, old.Add(-time.Hour))
}
key, value := gateReport(t, node, "v26.12.1", placed.Add(time.Minute))
gate.Toolchain(NodeBuildModule, key, value, false)
}
return gate, set, said
}
t.Run("not placed on ace: its report is no builder's", func(t *testing.T) {
gate, set, said := start(t, true)
key, value := gateReport(t, "ace", "v26.12.0", old)
gate.Toolchain(NodeBuildModule, key, value, false)
if _, none := agreedOf(set); !strings.HasPrefix(none, "builders disagree") {
t.Fatalf("before the placement is settled every report counts; got %q", none)
}
k, v := placing(t, "ace", GoBuildModule) // another build module on ace: no matter
gate.Placement(k, v, false, old)
if !gate.Settled() {
t.Fatal("settling left ace's report out, so what was passed on changed")
}
if v, none := agreedOf(set); v != "v26.12.1" || none != "" {
t.Fatalf("the placed nodes agree on v26.12.1; got %q, %q", v, none)
}
if len(*said) != 1 || !strings.Contains((*said)[0], "does not place mesh-build-node on ace") {
t.Fatalf("ace's report is said left out once; said %q", *said)
}
if gate.Toolchain(NodeBuildModule, key, value, false) {
t.Fatal("a report of a node not placed changed what was passed on")
}
// Placed on ace with a report written since: it counts, and the builders really disagree.
k, v = placing(t, "ace", GoBuildModule, NodeBuildModule)
gate.Placement(k, v, false, old.Add(-time.Minute))
if _, none := agreedOf(set); !strings.HasPrefix(none, "builders disagree: v26.12.0 recipe f59571cf4be75adf on ace") {
t.Fatalf("builders that run different versions still disagree; got %q", none)
}
// Taken off ace again (its key deleted): its report goes again.
if !gate.Placement(k, nil, true, placed) {
t.Fatal("taking ace's placement away passed nothing on")
}
if v, none := agreedOf(set); v != "v26.12.1" || none != "" {
t.Fatalf("got %q, %q", v, none)
}
// A report deleted from the bucket goes whether its node is placed or not.
key, _ = gateReport(t, "g14", "v26.12.1", placed)
if !gate.Toolchain(NodeBuildModule, key, nil, true) || len(set.Entries("typescript")) != 2 {
t.Fatalf("deleting g14's report: %d entries left, want novox and shanks", len(set.Entries("typescript")))
}
})
for _, c := range []struct {
name string
at time.Time // when ace wrote its report
counts bool
}{
{"a report left before the placement", old, false},
{"a report written since the placement", placed.Add(30 * time.Second), true},
} {
t.Run(c.name, func(t *testing.T) {
gate, set, said := start(t, true)
k, v := placing(t, "ace") // ace's key, first heard placing nothing
gate.Placement(k, v, false, old.Add(-time.Hour))
gate.Settled()
k, v = placing(t, "ace", NodeBuildModule)
if gate.Placement(k, v, false, placed) {
t.Fatal("a placement with no report passed something on")
}
key, value := gateReport(t, "ace", "v26.12.0", c.at)
if got := gate.Toolchain(NodeBuildModule, key, value, false); got != c.counts {
t.Fatalf("ace's report passed on: %v, want %v", got, c.counts)
}
// ace's placement written again with another module beside: its standing does not move, nothing is said again.
k, v = placing(t, "ace", GoBuildModule, NodeBuildModule)
if gate.Placement(k, v, false, placed.Add(time.Minute)) {
t.Fatal("the same placement of mesh-build-node changed what was passed on")
}
if c.counts {
return
}
if v, none := agreedOf(set); v != "v26.12.1" || none != "" {
t.Fatalf("ace's left report is no builder's; got %q, %q", v, none)
}
if len(*said) != 1 || !strings.Contains((*said)[0], "before mesh-build-node was placed on ace at 2026-10-25T19:56:45Z") {
t.Fatalf("said %q, want once why ace's report is left out", *said)
}
key, value = gateReport(t, "ace", "v26.12.1", placed.Add(time.Minute))
if !gate.Toolchain(NodeBuildModule, key, value, false) || len(set.Entries("typescript")) != 4 {
t.Fatal("ace's new report was not passed on")
}
})
}
t.Run("no placement held: every report counts", func(t *testing.T) {
gate, set, _ := start(t, false)
key, value := gateReport(t, "ace", "v26.12.0", old)
gate.Toolchain(NodeBuildModule, key, value, false)
gate.Settled()
if n := len(set.Entries("typescript")); n != 4 {
t.Fatalf("before the first placement file every report counts; got %d", n)
}
k, v := placing(t, "g14", NodeBuildModule)
if !gate.Placement(k, v, false, old.Add(-time.Hour)) || len(set.Entries("typescript")) != 1 {
t.Fatalf("the first placement leaves every node it does not place out; got %d", len(set.Entries("typescript")))
}
if !gate.Placement(k, nil, true, placed) || len(set.Entries("typescript")) != 4 {
t.Fatal("the last placement gone: every report counts again")
}
})
t.Run("the history: a module in the first revision heard of a key stands since before it", func(t *testing.T) {
p := delivery.NewPlaced()
k, v := placing(t, "ace", NodeBuildModule)
p.Put(k, v, false, old)
k2, v2 := placing(t, "g14")
p.Put(k2, v2, false, old)
k2, v2 = placing(t, "g14", NodeBuildModule)
if _, changed, _ := p.Put(k2, v2, false, placed); strings.Join(changed, ",") != NodeBuildModule {
t.Errorf("a module placed says changed %v", changed)
}
if _, _, err := p.Put("placement.novox", []byte("not json"), false, old); err == nil {
t.Error("a value that does not read was not said")
}
if s, ok := p.Since("ace", NodeBuildModule); !ok || !s.IsZero() {
t.Errorf("ace's module stands since %v, %v; want the zero time", s, ok)
}
if s, ok := p.Since("g14", NodeBuildModule); !ok || !s.Equal(placed) {
t.Errorf("g14's module stands since %v, %v; want the revision that placed it", s, ok)
}
if got := p.PlacedNodes(NodeBuildModule); strings.Join(got, ",") != "ace,g14" || len(p.ByNode()) != 2 || !p.Seeded() {
t.Errorf("PlacedNodes = %v, ByNode = %v", got, p.ByNode())
}
})
t.Run("a placement the bus refuses is said, and the part goes on", func(t *testing.T) {
var logs bytes.Buffer
logger := log.New(&logs, "", 0)
refused := errors.New("nats: permissions violation for publish to $JS.API.CONSUMER.CREATE.KV_mesh-delivery-assigner_placement")
gate, _, _ := start(t, false)
for _, bus := range []refusingBus{{err: refused}, {refuseOpen: true, err: refused}} {
if _, err := FollowPlacement(context.Background(), bus, gate, func(context.Context) {}, logger.Printf); err != nil {
t.Errorf("a refused placement stopped the start: %v", err)
}
}
if !strings.Contains(logs.String(), "cannot be watched") || !strings.Contains(logs.String(), "cannot be read") {
t.Errorf("the refusals were not said: %q", logs.String())
}
})
}
// refusingBus is a bus that refuses: every open (refuseOpen), or every watch of a bucket it opens, with err.
type refusingBus struct {
refuseOpen bool
err error
}
func (r refusingBus) KeyValue(context.Context, string) (jetstream.KeyValue, error) {
if r.refuseOpen {
return nil, r.err
}
return refusedWatch{err: r.err}, nil
}
type refusedWatch struct {
jetstream.KeyValue
err error
}
func (r refusedWatch) WatchAll(context.Context, ...jetstream.WatchOpt) (jetstream.KeyWatcher, error) {
return nil, r.err
}
+2 -4
View File
@@ -35,10 +35,8 @@ func TestToolchainSetKeepsEachNodesEntryByLanguageAndSaysWhenOneChanged(t *testi
if !s.Put(GoBuildModule, "ace.go", nil, true) || len(s.Entries("go")) != 1 { if !s.Put(GoBuildModule, "ace.go", nil, true) || len(s.Entries("go")) != 1 {
t.Error("a delete did not remove the node's entry") t.Error("a delete did not remove the node's entry")
} }
} // What is not a toolchain of the module's language is left out.
s = ToolchainSet{}
func TestToolchainSetLeavesOutWhatIsNotAToolchainOfTheModulesLanguage(t *testing.T) { // 1607/AC6
var s ToolchainSet
if s.Put(GoBuildModule, "novox.typescript", tcValue(t, "typescript", "x"), false) { if s.Put(GoBuildModule, "novox.typescript", tcValue(t, "typescript", "x"), false) {
t.Error("a typescript toolchain in the go build module's bucket counted") t.Error("a typescript toolchain in the go build module's bucket counted")
} }
+87
View File
@@ -3,9 +3,12 @@ package delivery
import ( import (
"crypto/ed25519" "crypto/ed25519"
"crypto/rand" "crypto/rand"
"crypto/x509"
"encoding/json" "encoding/json"
"encoding/pem"
"strings" "strings"
"testing" "testing"
"time"
) )
func aSigningKey(t *testing.T) (ed25519.PublicKey, ed25519.PrivateKey) { func aSigningKey(t *testing.T) (ed25519.PublicKey, ed25519.PrivateKey) {
@@ -100,4 +103,88 @@ func TestACompositionIsSignedByAKeptKey(t *testing.T) {
if _, err := SignComposedBy(c, pub, func(m []byte) ([]byte, error) { return ed25519.Sign(other, m), nil }); err == nil { if _, err := SignComposedBy(c, pub, func(m []byte) ([]byte, error) { return ed25519.Sign(other, m), nil }); err == nil {
t.Errorf("a signature by another key than the one named was accepted") t.Errorf("a signature by another key than the one named was accepted")
} }
// L3 s1 (ADR 0550 §2): a message the vault or the controller signed under a kind's context verifies against the
// signers a node trusts and names which; an unknown key and another kind's context are refused.
vaultPub, vault := aSigningKey(t)
controllerPub, controller := aSigningKey(t)
trusted := []Signer{{Name: SignerController, Key: controllerPub}, {Name: SignerVault, Key: vaultPub}}
enrol, _ := SignContext(KindEnrolment)
certCtx, _ := SignContext(KindNodeCertificate)
msg := []byte("the reply")
for _, row := range []struct {
name string
key ed25519.PrivateKey
signed string
checked string
want string
}{
{"the vault's key verifies", vault, enrol, enrol, SignerVault},
{"the controller's key verifies", controller, enrol, enrol, SignerController},
{"an unknown key is refused", other, enrol, enrol, ""},
{"the wrong context is refused", vault, certCtx, enrol, ""},
} {
got, err := VerifyBy(trusted, row.checked, msg, SignMessage(row.key, row.signed, msg))
if row.want == "" && err == nil {
t.Errorf("%s: verified as %s", row.name, got.Name)
}
if row.want != "" && (err != nil || got.Name != row.want) {
t.Errorf("%s: %+v, %v", row.name, got.Name, err)
}
}
if _, err := SignContext("hand-over"); err == nil {
t.Error("a kind the vault does not sign has a context")
}
// The signers' key reads back only when its fingerprint is its key's, and what a node trusts names both holders.
k := NewSignerKey(vaultPub, time.Unix(0, 0))
raw, _ := json.Marshal(k)
if back, err := DecodeSignerKey(raw); err != nil || back.Fingerprint != SignerFingerprint(vaultPub) {
t.Errorf("the signers' key: %+v, %v", back, err)
}
k.Fingerprint = SignerFingerprint(controllerPub)
if raw, _ := json.Marshal(k); true {
if _, err := DecodeSignerKey(raw); err == nil {
t.Error("a signers' key naming another fingerprint was taken")
}
}
if got := Trusts(trusted, nil); len(got) != 2 || !TrustsKey(got, SignerVault, SignerFingerprint(vaultPub)) ||
!TrustsKey(got, SignerController, SignerFingerprint(controllerPub)) {
t.Errorf("trusts: %v", got)
}
// A certificate the vault's authority signs for a node's name and serving key checks against a bundle holding that
// authority, and not for another name, another key, or against the controller's alone; a bundle given again adds no
// certificate twice.
authority := func(pub ed25519.PublicKey, priv ed25519.PrivateKey) string {
tpl := AuthorityTemplate(pub, time.Now().Add(-time.Hour))
der, err := x509.CreateCertificate(nil, tpl, tpl, pub, priv)
if err != nil {
t.Fatal(err)
}
return string(pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}))
}
vaultCA, controllerCA := authority(vaultPub, vault), authority(controllerPub, controller)
parent, _ := ParseCertificate(vaultCA)
serving, _ := aSigningKey(t)
der, err := x509.CreateCertificate(nil, NodeCertificateTemplate("ace.internal", serving, time.Now().Add(-time.Hour)), parent, serving, vault)
if err != nil {
t.Fatal(err)
}
leaf := string(pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}))
bundle := AuthorityBundle(controllerCA, AuthorityBundle(controllerCA, vaultCA))
if n := strings.Count(bundle, "BEGIN CERTIFICATE"); n != 2 {
t.Errorf("the bundle holds %d certificates, want 2", n)
}
now := time.Now()
if err := CheckNodeCertificate(leaf, []string{bundle}, "ace.internal", serving, now); err != nil {
t.Errorf("the vault's certificate does not check: %v", err)
}
for name, err := range map[string]error{
"another name": CheckNodeCertificate(leaf, []string{vaultCA}, "g14.internal", serving, now),
"another key": CheckNodeCertificate(leaf, []string{vaultCA}, "ace.internal", controllerPub, now),
"the controller's alone": CheckNodeCertificate(leaf, []string{controllerCA}, "ace.internal", serving, now),
} {
if err == nil {
t.Errorf("%s: checked", name)
}
}
} }
+36
View File
@@ -90,3 +90,39 @@ func (g ControllerGrant) NamesNode(node, module string) bool {
} }
return false return false
} }
// **The grants part's grants** (novox/hq ADR 0547 §4, §5, ADR 0561 §2, §6; issue 774). The grants part writes the same
// keys, with the same value, into its own state `grants` (the bucket PartGrantsBucket), beside the controller's: first in
// shadow, which nobody reads, so a tool can compare the two before anything switches. The assigner reads the part's keys
// only once the controller's lease is handed away (GrantSource), and the controller's until then.
const (
// PartGrantsState is the grants part's state of issued credentials by fingerprint, as its manifest names it.
PartGrantsState = "grants"
// PartGrantsBucket is that state's bucket, `<owner module>_<name>`.
PartGrantsBucket = "mesh-grants_" + PartGrantsState
// PartGrantsRead is that state as a reader names it in `reads`.
PartGrantsRead = "mesh-grants." + PartGrantsState
// PartGrantsHistory is the revisions the bucket keeps per key.
PartGrantsHistory = ControllerGrantsHistory
)
// PartGrantKey is the key, in PartGrantsBucket, of a module's credential issued for a fingerprint: the layout of the
// controller's key (ControllerGrantKey), so a key of one is the key of the other and a compare is by key.
func PartGrantKey(module, fingerprint string) (string, error) {
return ControllerGrantKey(module, fingerprint)
}
// ParsePartGrantKey is the inverse of PartGrantKey.
func ParsePartGrantKey(key string) (module, fingerprint string, err error) {
return ParseControllerGrantKey(key)
}
// GrantSource names the bucket the assigner reads grants from, and the `reads` name that grants it: the grants part's
// once the controller's lease is handed away (buscompose.LeaseHandedAway, which refuses every other value of the lease,
// an unreadable one included), the controller's in every other case.
func GrantSource(leaseHandedAway bool) (bucket, read string) {
if leaseHandedAway {
return PartGrantsBucket, PartGrantsRead
}
return ControllerGrantsBucket, "mesh-controller.grants"
}
+4
View File
@@ -55,6 +55,10 @@ type NodeRecord struct {
// Authority is the mesh's own certificate as this machine holds it (public): the one its declaration last wrote where // Authority is the mesh's own certificate as this machine holds it (public): the one its declaration last wrote where
// a module asked for it. A node composing its own composes a certificate with it (compose.Input.Authority). // a module asked for it. A node composing its own composes a certificate with it (compose.Input.Authority).
Authority string `json:"authority,omitempty"` Authority string `json:"authority,omitempty"`
// Trusts is what this machine trusts a mesh signature from now (Trusts in signers.go, ADR 0550 §2): the controller's
// key until the lease is handed away, and the vault's key and authority once it reads them from `signers`. The gate
// before the lease write reads every node's from the bus.
Trusts []string `json:"trusts,omitempty"`
} }
func (r NodeRecord) check() error { func (r NodeRecord) check() error {
+1 -1
View File
@@ -18,7 +18,7 @@ var secretNamed = regexp.MustCompile(`(?i)(secret|passw|token|api[-_]?key|privat
func TestNodeRecord(t *testing.T) { func TestNodeRecord(t *testing.T) {
hub := NodeRecord{Node: "novox", TunnelKey: "k", Capabilities: []string{"docker"}, SealingKey: "age1x", ServingKey: "pub", hub := NodeRecord{Node: "novox", TunnelKey: "k", Capabilities: []string{"docker"}, SealingKey: "age1x", ServingKey: "pub",
Certificate: "cert", OutwardLinks: []string{"eth0"}, Networks: []catalogue.AttachedNetwork{{Link: "eth0", Network: "192.0.2.0/24"}}, Certificate: "cert", OutwardLinks: []string{"eth0"}, Networks: []catalogue.AttachedNetwork{{Link: "eth0", Network: "192.0.2.0/24"}},
NetworksSaid: true, Contract: 3} NetworksSaid: true, Contract: 3, Trusts: []string{"controller key sha256:aa", "vault key sha256:bb"}}
raw, err := EncodeNodeRecord(hub) raw, err := EncodeNodeRecord(hub)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
+126
View File
@@ -0,0 +1,126 @@
package delivery
import (
"fmt"
"maps"
"slices"
"sync"
"time"
)
// Placed is where the placement (the seat mesh-placement's state `placement`, held by the assigner; ADR 0567, ADR 0549 §3) places each module, and since when:
// the one reader of that state for every part that asks "does module stand on node" (issue 774, design 47's third
// milestone), in place of each part reading the controller's assignments (`mesh-controller_assignments`, which goes
// with the controller's lease, ADR 0549 §6).
//
// Its input is the state's revisions in the order the bus wrote them, as a watch with the history (jetstream's
// IncludeHistory) hands them: Put takes each. A module stands since the revision of its node's key that first named it
// in an unbroken run. A module in the first revision heard of a key stands since the zero time: that revision is the
// key's first (the seed of ADR 0549 §4 placed what already ran there) or the oldest the bus still keeps
// (PlacementHistory), and either way the module may have run there before it. A key's value that does not read places
// nothing on its node, and Put says why.
//
// Safe for use from a watch and from a reader at once.
type Placed struct {
mu sync.Mutex
since map[string]map[string]time.Time // node -> module -> since when it stands
heard map[string]bool // the nodes whose key was heard at all
}
// NewPlaced is a Placed that holds no placement yet.
func NewPlaced() *Placed {
return &Placed{since: map[string]map[string]time.Time{}, heard: map[string]bool{}}
}
// Put takes one revision of a key of the state `placement` (its value, whether it is a delete, and when the bus wrote
// it). It answers the node the key is of and the modules whose standing on it changed (placed or removed), in name
// order, and an error for a key or a value that does not read (the node then places nothing).
func (p *Placed) Put(key string, value []byte, deleted bool, at time.Time) (node string, changed []string, err error) {
node, err = ParsePlacementKey(key)
if err != nil {
return "", nil, err
}
now := map[string]bool{}
if !deleted {
pl, derr := DecodePlacement(value)
switch {
case derr != nil:
err = fmt.Errorf("the placement of %s does not read, so it places nothing there: %w", node, derr)
case pl.Node != node:
err = fmt.Errorf("the placement under %s names the node %q, so it places nothing there", key, pl.Node)
default:
for _, m := range pl.Assigned() {
now[m] = true
}
}
}
p.mu.Lock()
defer p.mu.Unlock()
if !p.heard[node] {
p.heard[node] = true
at = time.Time{}
}
was := p.since[node]
next := map[string]time.Time{}
for m := range now {
if s, ok := was[m]; ok {
next[m] = s
} else {
next[m] = at
changed = append(changed, m)
}
}
for m := range was {
if !now[m] {
changed = append(changed, m)
}
}
if len(next) == 0 {
delete(p.since, node)
} else {
p.since[node] = next
}
slices.Sort(changed)
return node, changed, err
}
// Seeded says some node's placement is held: before the assigner's adopt writes the first placement nothing is placed,
// and a reader keeps what it did before (ADR 0549 §5).
func (p *Placed) Seeded() bool {
p.mu.Lock()
defer p.mu.Unlock()
return len(p.since) > 0
}
// Since says whether module stands on node, and since when (the zero time: since before the revisions kept).
func (p *Placed) Since(node, module string) (time.Time, bool) {
p.mu.Lock()
defer p.mu.Unlock()
s, ok := p.since[node][module]
return s, ok
}
// PlacedNodes is every node module is placed on, in name order.
func (p *Placed) PlacedNodes(module string) []string {
p.mu.Lock()
defer p.mu.Unlock()
var out []string
for node, mods := range p.since {
if _, ok := mods[module]; ok {
out = append(out, node)
}
}
slices.Sort(out)
return out
}
// ByNode is, by node, the modules placed on it in name order.
func (p *Placed) ByNode() map[string][]string {
p.mu.Lock()
defer p.mu.Unlock()
out := make(map[string][]string, len(p.since))
for node, mods := range p.since {
out[node] = slices.Sorted(maps.Keys(mods))
}
return out
}
+4 -1
View File
@@ -190,10 +190,13 @@ type ModuleAdded struct {
Unissued []string `json:"unissued,omitempty"` Unissued []string `json:"unissued,omitempty"`
} }
// ModuleRefused is a value missing for a module the node runs, in the words the composition stood in for it with. // ModuleRefused is a value missing for a module the node runs, in the words the composition stood in for it with. Today,
// in CandidateAnswer.Held, is how the node keeps it today (held, failed, runs a build of the old form, held behind
// another), which the candidate keeps.
type ModuleRefused struct { type ModuleRefused struct {
Module string `json:"module"` Module string `json:"module"`
Why string `json:"why"` Why string `json:"why"`
Today string `json:"today,omitempty"`
} }
// VerdictOf is the verdict of an answer's lists: refused on any refusal, differs on any difference, addition or // VerdictOf is the verdict of an answer's lists: refused on any refusal, differs on any difference, addition or
+350
View File
@@ -0,0 +1,350 @@
package delivery
import (
"crypto/ed25519"
"crypto/sha256"
"crypto/x509"
"crypto/x509/pkix"
"encoding/base64"
"encoding/hex"
"encoding/json"
"encoding/pem"
"errors"
"fmt"
"math/big"
"slices"
"strings"
"time"
)
// **The vault signs** (novox/hq ADR 0550 §2; design 47's L3 item d). The vault makes an ed25519 signing key and an x509
// authority once, keeps both as its own secrets with their operator copies, and publishes their public halves in its
// state `signers`, which only the vault writes and every node principal and composing part reads. Its verb `sign` signs a
// node's certificate for that node's own node-engine, from bus state alone (the name from the node's placement, the
// serving key from its record), and says `signed` naming the caller, the kind, the node and the digest, never the key.
//
// A node trusts the vault's key through the bus grants alone (the lead's design limit, 2026-10-28): only the vault may
// write `signers`, the same trust the wants and the placement rest on. Until the controller's lease is handed away the
// node trusts the controller's key beside it; from then on the vault's alone (LeaseHandedAway). What a node trusts now
// it states in its record, NodeRecord.Trusts, so the gate before the lease write reads it from the bus.
const (
// VaultModule is the module holding the seat mesh-vault, whose state `signers` is.
VaultModule = "mesh-vault"
// SignersState is the vault's state of its public signing halves, and SignersBucket its bucket.
SignersState = "signers"
SignersBucket = VaultModule + "_" + SignersState
// SignersRead is the state as a manifest's `reads` names it.
SignersRead = VaultModule + "." + SignersState
// SignersKeyKey holds the signing key's public half (SignerKey); SignersAuthorityKey the authority (SignerAuthority).
SignersKeyKey = "key"
SignersAuthorityKey = "authority"
// SignVerb is the vault's verb that signs.
SignVerb = "sign"
// SignedEvent is what the vault says when it signed (Signed), on SignedSubject.
SignedEvent = "signed"
SignedSubject = "mesh.mod." + VaultModule + ".event." + SignedEvent
// KindNodeCertificate is a node's certificate for its internal name, asked by that node's node-engine alone.
KindNodeCertificate = "node-certificate"
// KindEnrolment is an enrolment reply, signed under its own context; no caller is named until adoption's record
// (ADR 0561 §5).
KindEnrolment = "enrolment"
// KindAuthority is the vault's own authority certificate, which the vault signs for itself.
KindAuthority = "authority"
// SignerController and SignerVault name the two signers a node may trust.
SignerController = "controller"
SignerVault = "vault"
// CertificateLife is how long a certificate the vault signs for a node lasts: the controller's ten years
// (identity.forever), renewed by asking again, never by a clock running out unseen.
CertificateLife = 10 * 365 * 24 * time.Hour
)
// SignKinds are the kinds `sign` knows.
var SignKinds = []string{KindNodeCertificate, KindEnrolment, KindAuthority}
// SignContext is the line a message of kind is signed after, so a signature made for one kind never verifies as
// another's (as the composition's and the asks' contexts are kept apart).
func SignContext(kind string) (string, error) {
if !slices.Contains(SignKinds, kind) {
return "", fmt.Errorf("delivery: %q is not a kind the vault signs (%s)", kind, strings.Join(SignKinds, ", "))
}
return "novox " + kind + " v1\n", nil
}
// SignerKey is the key `key` of `signers`: the vault's signing key's public half in standard base64, its fingerprint,
// and when it was made.
type SignerKey struct {
Key string `json:"key"`
Fingerprint string `json:"fingerprint"`
Made time.Time `json:"made"`
}
// SignerAuthority is the key `authority` of `signers`: the vault's authority certificate as PEM, its fingerprint, and
// when it was made.
type SignerAuthority struct {
Certificate string `json:"certificate"`
Fingerprint string `json:"fingerprint"`
Made time.Time `json:"made"`
}
// SignerFingerprint is an ed25519 public key's fingerprint, `sha256:<hex>` of its 32 bytes.
func SignerFingerprint(key ed25519.PublicKey) string {
sum := sha256.Sum256(key)
return "sha256:" + hex.EncodeToString(sum[:])
}
// AuthorityFingerprint is an authority certificate's fingerprint, `sha256:<hex>` of its DER; refused when certificate is
// not one PEM certificate.
func AuthorityFingerprint(certificate string) (string, error) {
c, err := ParseCertificate(certificate)
if err != nil {
return "", err
}
sum := sha256.Sum256(c.Raw)
return "sha256:" + hex.EncodeToString(sum[:]), nil
}
// ParseCertificate is the first certificate of a PEM text.
func ParseCertificate(text string) (*x509.Certificate, error) {
block, _ := pem.Decode([]byte(strings.TrimSpace(text)))
if block == nil || block.Type != "CERTIFICATE" {
return nil, errors.New("delivery: not a PEM certificate")
}
return x509.ParseCertificate(block.Bytes)
}
// NewSignerKey is the value of `key` for pub.
func NewSignerKey(pub ed25519.PublicKey, made time.Time) SignerKey {
return SignerKey{Key: base64.StdEncoding.EncodeToString(pub), Fingerprint: SignerFingerprint(pub), Made: made.UTC()}
}
// PublicKey is k's key, checked: an ed25519 public key whose fingerprint is the one k names.
func (k SignerKey) PublicKey() (ed25519.PublicKey, error) {
raw, err := base64.StdEncoding.DecodeString(k.Key)
if err != nil || len(raw) != ed25519.PublicKeySize {
return nil, errors.New("delivery: the signers' key is not an ed25519 public key in base64")
}
if fp := SignerFingerprint(raw); fp != k.Fingerprint {
return nil, fmt.Errorf("delivery: the signers' key has the fingerprint %s, and names %s", fp, k.Fingerprint)
}
return ed25519.PublicKey(raw), nil
}
// DecodeSignerKey reads the key `key` of `signers`, refused when it does not check.
func DecodeSignerKey(raw []byte) (SignerKey, error) {
var k SignerKey
if err := json.Unmarshal(raw, &k); err != nil {
return SignerKey{}, fmt.Errorf("delivery: %s %s: %w", SignersRead, SignersKeyKey, err)
}
if _, err := k.PublicKey(); err != nil {
return SignerKey{}, err
}
return k, nil
}
// DecodeSignerAuthority reads the key `authority` of `signers`: one authority certificate (a CA that signs leaves
// alone) whose fingerprint is the one named.
func DecodeSignerAuthority(raw []byte) (SignerAuthority, error) {
var a SignerAuthority
if err := json.Unmarshal(raw, &a); err != nil {
return SignerAuthority{}, fmt.Errorf("delivery: %s %s: %w", SignersRead, SignersAuthorityKey, err)
}
c, err := ParseCertificate(a.Certificate)
if err != nil {
return SignerAuthority{}, err
}
if !c.IsCA {
return SignerAuthority{}, errors.New("delivery: the signers' authority is not an authority certificate")
}
if fp, _ := AuthorityFingerprint(a.Certificate); fp != a.Fingerprint {
return SignerAuthority{}, fmt.Errorf("delivery: the signers' authority has the fingerprint %s, and names %s", fp, a.Fingerprint)
}
return a, nil
}
// Signer is one key a node trusts, by who holds it (SignerController, SignerVault, or the operator's).
type Signer struct {
Name string
Key ed25519.PublicKey
}
// Fingerprint is the signer's key's fingerprint.
func (s Signer) Fingerprint() string { return SignerFingerprint(s.Key) }
// VerifyBy is the trusted signer whose key verifies sig over context and msg: refused when no trusted key does, saying
// so, and when a key is not an ed25519 key. A signature made under another context never verifies.
func VerifyBy(trusted []Signer, context string, msg, sig []byte) (Signer, error) {
if len(sig) != ed25519.SignatureSize {
return Signer{}, errors.New("delivery: not an ed25519 signature")
}
message := append([]byte(context), msg...)
for _, s := range trusted {
if len(s.Key) != ed25519.PublicKeySize {
continue
}
if ed25519.Verify(s.Key, message, sig) {
return s, nil
}
}
names := make([]string, 0, len(trusted))
for _, s := range trusted {
names = append(names, s.Name)
}
return Signer{}, fmt.Errorf("delivery: the signature verifies against no trusted key (%s)", strings.Join(names, ", "))
}
// SignMessage is sig over context and msg by key: what the vault's `sign` does for a kind whose bytes are given.
func SignMessage(key ed25519.PrivateKey, context string, msg []byte) []byte {
return ed25519.Sign(key, append([]byte(context), msg...))
}
// Signed is the vault's `signed`: who asked, the kind, the node, and the sha256 digest of what was signed. Never a key.
type Signed struct {
Caller string `json:"caller"`
Kind string `json:"kind"`
Node string `json:"node,omitempty"`
Digest string `json:"digest"`
Signer string `json:"signer"`
At time.Time `json:"at"`
}
// SignAnswer is what `sign` answers: for a node certificate the certificate as PEM and the authority that signed it,
// for an enrolment the signature in base64; always the signer's fingerprint and the digest. Never a key's private half.
type SignAnswer struct {
Kind string `json:"kind"`
Node string `json:"node,omitempty"`
Certificate string `json:"certificate,omitempty"`
Authority string `json:"authority,omitempty"`
Signature string `json:"signature,omitempty"`
Signer string `json:"signer"`
Digest string `json:"digest"`
}
// DigestOf is the digest a `signed` names: `sha256:<hex>` of the bytes signed.
func DigestOf(b []byte) string {
sum := sha256.Sum256(b)
return "sha256:" + hex.EncodeToString(sum[:])
}
// NodeCertificateTemplate is the certificate the vault signs for node's internal name, as the controller's authority
// shaped it (identity.Certify): the name in the subject and the subject alternative names, server and client use, the
// serial derived from the name and the key so the same ask gives the same certificate, and its life from notBefore.
func NodeCertificateTemplate(name string, servingKey ed25519.PublicKey, notBefore time.Time) *x509.Certificate {
sum := sha256.Sum256(append([]byte(name+"\n"), servingKey...))
return &x509.Certificate{
SerialNumber: new(big.Int).SetBytes(sum[:16]),
Subject: pkix.Name{CommonName: name},
DNSNames: []string{name},
NotBefore: notBefore.UTC().Truncate(time.Second),
NotAfter: notBefore.UTC().Truncate(time.Second).Add(CertificateLife),
KeyUsage: x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth, x509.ExtKeyUsageClientAuth},
}
}
// AuthorityTemplate is the vault's authority certificate for its key, made once: an authority that signs leaves and no
// other authority, as the controller's.
func AuthorityTemplate(key ed25519.PublicKey, made time.Time) *x509.Certificate {
sum := sha256.Sum256(append([]byte("authority\n"), key...))
return &x509.Certificate{
SerialNumber: new(big.Int).SetBytes(sum[:16]),
Subject: pkix.Name{CommonName: "the mesh's vault"},
NotBefore: made.UTC().Truncate(time.Second),
NotAfter: made.UTC().Truncate(time.Second).Add(2 * CertificateLife),
IsCA: true,
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageCRLSign,
BasicConstraintsValid: true,
MaxPathLen: 0,
MaxPathLenZero: true,
}
}
// CheckNodeCertificate says why certificate is not a certificate of name for servingKey by one of the authorities, or
// nil: a node uses it to tell whether its record holds a certificate from an authority it trusts, and another node to
// check one the vault signed (ADR 0550 §2's live check).
func CheckNodeCertificate(certificate string, authorities []string, name string, servingKey ed25519.PublicKey, at time.Time) error {
c, err := ParseCertificate(certificate)
if err != nil {
return err
}
pool := x509.NewCertPool()
for _, a := range authorities {
pool.AppendCertsFromPEM([]byte(a)) // a bundle is every certificate it holds
}
if _, err := c.Verify(x509.VerifyOptions{Roots: pool, DNSName: name, CurrentTime: at,
KeyUsages: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}}); err != nil {
return fmt.Errorf("delivery: the certificate of %s is not one a trusted authority signed for it: %w", name, err)
}
held, ok := c.PublicKey.(ed25519.PublicKey)
if !ok || !held.Equal(servingKey) {
return fmt.Errorf("delivery: the certificate of %s binds another key than the node serves with", name)
}
return nil
}
// AuthorityBundle is the authorities given as one PEM text, each certificate once, in the order given, blanks left out:
// what a module asking for the mesh's authority is given once the vault signs beside the controller (ADR 0550 §2). An
// authority given is split into its certificates, so a bundle given again (the one a node kept from its last
// composition) adds nothing twice; a text that holds no PEM block is taken whole.
func AuthorityBundle(authorities ...string) string {
var out []string
for _, a := range authorities {
for _, piece := range pemPieces(a) {
if !slices.Contains(out, piece) {
out = append(out, piece)
}
}
}
if len(out) == 0 {
return ""
}
return strings.Join(out, "\n") + "\n"
}
// pemPieces is each PEM block of text re-encoded, or the trimmed text whole when it holds none.
func pemPieces(text string) []string {
text = strings.TrimSpace(text)
if text == "" {
return nil
}
var out []string
rest := []byte(text)
for {
block, more := pem.Decode(rest)
if block == nil {
break
}
out = append(out, strings.TrimSpace(string(pem.EncodeToMemory(block))))
rest = more
}
if len(out) == 0 {
return []string{text}
}
return out
}
// Trusts is what a node states it trusts now (NodeRecord.Trusts): `<who> key <fingerprint>` for each signer and
// `<who> authority <fingerprint>` for each authority certificate, by who holds it, each once, sorted. The gate before the
// lease write reads `vault key <fingerprint>` in every node's record.
func Trusts(signers []Signer, authorities map[string]string) []string {
var out []string
for _, s := range signers {
if len(s.Key) == ed25519.PublicKeySize {
out = append(out, s.Name+" key "+s.Fingerprint())
}
}
for who, a := range authorities {
if fp, err := AuthorityFingerprint(a); err == nil {
out = append(out, who+" authority "+fp)
}
}
slices.Sort(out)
return slices.Compact(out)
}
// TrustsKey says whether trusts names the key with fingerprint fp held by who.
func TrustsKey(trusts []string, who, fp string) bool { return slices.Contains(trusts, who+" key "+fp) }