Compare commits

..
8 Commits
Author SHA1 Message Date
mesh-adminandClaude Opus 5.5 7a691edd3a Find a bare or aliased mirror reader, and say what the reader test misses
mesh/merge-gate head 7a691edd3a6c merged into trunk dc36536e199e — pass: the change touches no module of the mesh's graph
mesh/repo-check head 7a691edd3a6c merged into trunk dc36536e199e — fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheContro…
mesh/delivery rejected: the gate failed or could not run, or the repository's own check failed
Review 257 of mesh-controller#315 (hq issue 652): the test that only the audit
and a mirror's writer read a mirror (ADR 0289 §19, ADR 0356 §2) matched only
names qualified as broker. or recordmirror., so a reader inside package broker,
or one importing broker under another name, passed. It now matches by import
path and bare names in the defining packages, proves it on planted readers,
and names what it cannot see.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 01:40:33 +00:00
mesh-adminandClaude Opus 5.5 35c7662f89 Check the mirrors as ADR 0356 says, and prove a removal asks for the lease
mesh/merge-gate head 35c7662f895f merged into trunk dc36536e199e — pass: the change touches no module of the mesh's graph
mesh/repo-check head 35c7662f895f merged into trunk dc36536e199e — fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheContro…
mesh/delivery superseded: a newer head of the same pull request
Reviews 251 of mesh-controller#315 and ADR 0356 v4 (hq issue 652): a removal
without the lease removes nothing; a module no longer registered loses its
declared-config key with its event; a listed mirror is never reported as
undeclared state; only the audit and a mirror's own writer name a mirror's
bucket; and the mirrors' list and the writers table agree. The store test's
line moves where it merges with the integration branch's own new lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 01:26:29 +00:00
mesh-adminandClaude Opus 5.5 69367aa786 Correct a mirror written behind the writer's back, and write none without the lease
mesh/merge-gate out of date: judged against trunk 617881c3093d, the trunk is now 71362c196dbc; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 617881c3093d, the trunk is now 71362c196dbc; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
Review of mesh-controller#315 (hq issue 652): the writer trusted the bucket as
it read it once, so a former lease holder's write stayed for good; it wrote
without the lease, asking only after a refused compare-and-set; and the
declared config carried a manifest's declarations whole, so a field the
manifest gains would reach the bus unreviewed (ADR 0289 §9, §10, §12). Every
pass now reads the bucket and writes back the store's value where it differs,
asks for the lease before the pass and before every write, and a declaration
is mirrored as its listed fields (kind, default, why), with a test that fails
on a declaration field the list does not name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 01:01:00 +00:00
mesh-adminandClaude Opus 5.5 8563b90918 Capture the genesis template from mesh-host's branch of this package
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate out of date: judged against trunk bfc79c64d989, the trunk is now 617881c3093d; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk bfc79c64d989, the trunk is now 617881c3093d; judged again on a recheck or when it is merged
The template at mesh-host c8920622 carries the controller's grant to write the
two record mirrors and say their events; the genesis template test holds the
controller's composition to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:34:24 +00:00
mesh-adminandClaude Opus 5.5 3218fc3c1b Hold the composed grant and the captured genesis template to the mirrors
The controller's composition gains the two mirror buckets and their events, so
the golden composition moves, and the genesis template is captured again from
mesh-host's branch of this package, whose installer user list carries them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:33:05 +00:00
mesh-adminandClaude Opus 5.5 ea5cc6fc35 Prove a record mirror against a real bus and the store
An assignment and a registration each change exactly one key and raise exactly
one event; the same records written again change nothing (hq issue 652, the
package's done-when). Listed as a store test, so the merge check runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:32:29 +00:00
mesh-adminandClaude Opus 5.5 176a3d4fdf Write a record mirror only where its store differs, one event per write
The writer behind the assignments and declared-config mirrors (hq issue 652,
ADR 0289, design 54 §2-3): filled from the bucket on its first pass, it
compares each value byte for byte, writes by compare-and-set, purges a key
whose record is gone with a marker that lives a day, and says each write once
after it succeeded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:31:17 +00:00
mesh-adminandClaude Opus 5.5 8f8f7dff23 Keep the assignments and each module's declared config on the bus
mesh-config resolves a module's config values per node from which module runs
where and what each registered module declares, and both lived only in the
controller's store (hq issue 652, design 57 F3). The controller now keeps two
record mirrors, mesh-controller_assignments (<node>.<module>) and
mesh-controller_declared-config (<module>), built from the store by the lease
holder every ten seconds and written only where a value differs, by
compare-and-set, with one assignment-changed or declared-config-changed per
write and a purge said as removed (ADR 0289, design 54 §2-3). A module reads
them through `reads` as mesh-controller.<mirror>; the writers table keeps every
other writer out. Neither holds a secret: an assignment names its node and
module, a declaration is what the public manifest declares.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:00:25 +00:00
56 changed files with 1676 additions and 2091 deletions
+2 -3
View File
@@ -424,9 +424,8 @@ func comparisonOf(module string, c comparison, opts takeOptions) (preview string
}
// And its settings on this machine, composed against its definition (rule 1, rule 6).
for _, layer := range c.layers {
said := layer.Whole()
keys := make([]string, 0, len(said))
for k := range said {
keys := make([]string, 0, len(layer.Values))
for k := range layer.Values {
keys = append(keys, k)
}
sort.Strings(keys)
-5
View File
@@ -55,11 +55,6 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
continue
}
m, err := catalogue.ParseManifest(raw)
if err == nil {
// Each template's text read from the tree beside the manifest, and judged as inline content is
// (novox/hq ADR 0349 §2): a missing template is refused by name.
m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(path))
}
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
+2 -2
View File
@@ -348,11 +348,11 @@ func gatherFacts(ctx context.Context, open *stores, busVersion string) (snapshot
if layer.From == catalogue.MeshWideLayer {
if !meshWide[module] {
meshWide[module] = true
f.Settings = append(f.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Whole())})
f.Settings = append(f.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Values)})
}
continue
}
m.Settings = append(m.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Whole())})
m.Settings = append(m.Settings, snapshot.Settings{Module: module, Values: scrub.Values(layer.Values)})
}
}
m.Declaration = declarationFacts(ctx, open, n.Name, gens, gensErr, scrub)
-5
View File
@@ -576,11 +576,6 @@ func shelfWithChange(base map[string]catalogue.Manifest, sources map[string]inve
return err
}
m, err := catalogue.ParseManifest(raw)
if err == nil {
// Each template's text from the tree beside the manifest, as the build reads it back from the archive it
// packs (novox/hq ADR 0349 §2): so a change to a template alone is a change to its module.
m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(path))
}
if err != nil {
if inScope(dir) {
failures = append(failures, fmt.Sprintf("%s: the controller judging this (%s) refuses it — %s",
+1 -15
View File
@@ -33,21 +33,7 @@ import (
func aMesh(t *testing.T) *stores {
t.Helper()
inventory.ForTest(t) // raises the store, migrates it, and points the environment at it
return aMeshOnTheStore(t)
}
// aMeshBefore is aMesh on a store migrated only below migration `number`, with the function that runs the
// rest: for a test of what a migration that moves data does to what the mesh composes (novox/hq issue 651).
func aMeshBefore(t *testing.T, number int) (*stores, func()) {
t.Helper()
_, migrate := inventory.ForTestBefore(t, number)
return aMeshOnTheStore(t), migrate
}
// aMeshOnTheStore is the mesh aMesh makes, on the store the environment points at.
func aMeshOnTheStore(t *testing.T) *stores {
t.Helper()
licences.ForTest(t) // planning reaches this one too, by name and never by connection
licences.ForTest(t) // planning reaches this one too, by name and never by connection
open, err := openStores(t.Context())
if err != nil {
-9
View File
@@ -8,7 +8,6 @@ import (
"fmt"
"net"
"os"
"path/filepath"
"sort"
"strings"
"time"
@@ -128,10 +127,6 @@ func moduleCommand(ctx context.Context, args []string) error {
if err != nil {
return err
}
// A manifest handed over by hand takes each template's text from the tree beside it (novox/hq ADR 0349 §2).
if m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(positionals[0])); err != nil {
return err
}
from, err := whereItComesFrom(*repo, *ref, *commit, *path, *self)
if err != nil {
return err
@@ -542,10 +537,6 @@ func settingsCommand(ctx context.Context, args []string) error {
if p.SetBy != "" {
by = "; " + p.SetBy
}
if p.Why != "" {
// A layer whose placement moved out of it (migration 0097): still the layer, as it stands.
by += "; " + p.Why
}
fmt.Printf("%s on %s, until %s (%s%s):\n %s\n", positionals[0], where,
p.ReplacedAt.Local().Format("2006-01-02 15:04:05"), p.ReplacedBy, by, shown)
}
-113
View File
@@ -1,113 +0,0 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// **Placement leaves the settings layers and no machine is told anything new** (novox/hq ADR 0348 clause
// 11, issue 651). A mesh is seeded as the store holds it before migration 0097 — `places`, `accesses` and a
// kept network among the settings, at the mesh's level and a machine's, one layer of placement only — then
// migrated, and every machine's declaration is composed. What each machine is told of the seeded modules is
// byte for byte what the controller before this change composed from the same seed: the golden files were
// written by the integration branch's controller at bfc79c6 running this same test (MESH_WRITE_GOLDEN=1),
// where there was no migration 0097 to run.
func TestADeclarationIsTheSameAfterPlacementMoves(t *testing.T) {
open, migrate := aMeshBefore(t, 97)
ctx := t.Context()
inv := open.inventory
image := "registry.invalid:5000/library@sha256:" + strings.Repeat("a", 64)
register(t, open, catalogue.Manifest{Module: "library", Version: "1",
Accesses: []catalogue.Access{{ID: "series"}, {ID: "downloads", Mode: catalogue.AccessRead}},
Resources: []map[string]any{
{"id": "config", "type": "directory", "mode": "0750"},
{"id": "server", "type": "container", "image": image,
"volumes": []any{"${dir:config}:/config", "${access:series}:/series", "${access:downloads}:/downloads:ro"}},
// Today's form: a mergeable file naming no keys takes every key of the layer (ADR 0326 §3).
{"id": "runtime-config", "type": "file", "path": "${dir:config}/runtime.json", "mode": "0640",
"content": `{"log":"info"}`, "merge": "json"},
}})
register(t, open, catalogue.Manifest{Module: "found", Version: "1",
Resources: []map[string]any{{"id": "server", "type": "container", "image": image}}})
if err := inv.SetAdopted(ctx, "anchor", true); err != nil {
t.Fatal(err)
}
for _, l := range []struct{ node, module, values string }{
{"", "library", `{"accesses": {"downloads": "/storage/downloads"}, "log": "warn"}`},
{"anchor", "library", `{"places": {"config": {"path": "/var/lib/library/config", "owner": "1001:2000"}},
"accesses": {"series": "/storage/media/series"}, "workers": 4}`},
{"laptop", "library", `{"accesses": {"series": "/home/media/series"}}`},
{"anchor", "found", `{"networks": {"server": ["predecessor_default"]}}`},
} {
if l.node == "" {
inv.ExecForTest(t, `insert into settings (node, module, values) values (null, $1, $2)`, l.module, l.values)
continue
}
inv.ExecForTest(t, `insert into settings (node, module, values) select id, $2, $3 from node where name = $1`,
l.node, l.module, l.values)
}
migrate()
// Assigned once the store is migrated: an assignment reads the layers, as this controller holds them.
for _, a := range [][]string{{"anchor", "library", "found"}, {"laptop", "library"}} {
if _, err := assign(ctx, open, a[0], a[1:]...); err != nil {
t.Fatalf("assign %v: %v", a, err)
}
}
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
for _, node := range []string{"anchor", "laptop"} {
plan, settings, err := planFor(ctx, open, node)
if err != nil {
t.Fatal(err)
}
with, _, err := renderingFor(ctx, open, node, plan, settings, gens, Allocating)
if err != nil {
t.Fatal(err)
}
composed, err := plan.Compose(with)
if err != nil {
t.Fatal(err)
}
if len(composed.LeftOut) > 0 {
t.Fatalf("%s leaves out %v", node, composed.LeftOut)
}
var told []map[string]any
for _, r := range composed.Resources {
if owner := composed.Owner[r["id"].(string)]; owner == "library" || owner == "found" {
told = append(told, r)
}
}
sort.Slice(told, func(i, j int) bool { return told[i]["id"].(string) < told[j]["id"].(string) })
got, err := json.MarshalIndent(told, "", " ")
if err != nil {
t.Fatal(err)
}
golden := filepath.Join("testdata", "placement-moved", node+".json")
if os.Getenv("MESH_WRITE_GOLDEN") != "" {
if err := os.MkdirAll(filepath.Dir(golden), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(golden, append(got, '\n'), 0o644); err != nil {
t.Fatal(err)
}
continue
}
want, err := os.ReadFile(golden)
if err != nil {
t.Fatal(err)
}
if string(want) != string(got)+"\n" {
t.Errorf("%s is told something else after placement moved:\n%s\nwas:\n%s", node, got, want)
}
}
}
+3
View File
@@ -264,6 +264,9 @@ func serve(ctx context.Context) (err error) {
// The facts snapshot a merge check is fed (novox/hq to-be 45 §9): kept current while this
// controller holds the lease, read by the build seat from the artifact store.
go exportingFacts(ctx, open, bus.Conn.ConnectedServerVersion)
// The record mirrors (novox/hq ADR 0289, design 57 F3): the assignments and each registered module's
// declared config on the bus, written from the store only when a record changed, by the lease holder.
go mirroringRecords(ctx, inv, bus.Conn, bus)
// Every call carries the lease's epoch, and its record is written only under the lease (novox/hq
// to-be 45 §6).
link.Calls.UnderLease(func() (uint64, error) { return theLease.epoch(ctx) })
+105
View File
@@ -0,0 +1,105 @@
package main
import (
"context"
"fmt"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/recordmirror"
)
// The controller's record mirrors on the bus (novox/hq ADR 0289, design 54, design 57 F3): the assignments,
// and each registered module's declared config, which mesh-config resolves a module's config values from.
//
// **Built from the store, on a tick of their own, by the lease holder alone.** A record changes through a
// verb that runs as a command of its own, a reconcile, a merge's registration or a pending assignment made,
// so no one call site sees every change; what every change has in common is that it is committed to the
// store. So the serving controller reads the store every mirrorEvery and writes only what differs from what
// the bucket holds (ADR 0289 §1): a quiet mesh writes nothing, and a change rolled back is never read. A
// mirror is therefore at most mirrorEvery behind its store.
// mirrorEvery is how often the mirrors are compared with the store: two small reads of the store.
const mirrorEvery = 10 * time.Second
// mirroringRecords keeps the mirrors current for as long as this controller serves: ctx ends when it stops.
func mirroringRecords(ctx context.Context, inv *inventory.Inventory, conn *nats.Conn, announce recordmirror.Announcer) {
api, err := jetstream.New(conn)
if err != nil {
fmt.Printf("the record mirrors are not kept: %v\n", err)
return
}
holds := func() bool {
epoch, err := theLease.epoch(ctx)
return err == nil && epoch > 0
}
epoch := func() uint64 {
e, _ := theLease.epoch(ctx)
return e
}
sources := map[string]func(context.Context) (map[string][]byte, error){
recordmirror.Assignments.Name: inv.AssignmentsMirror,
recordmirror.DeclaredConfig.Name: inv.DeclaredConfigMirror,
}
writers := map[string]*recordmirror.Writer{}
failing := map[string]string{}
tick := time.NewTicker(mirrorEvery)
defer tick.Stop()
for {
if holds() {
for _, m := range recordmirror.Mirrors() {
why := mirrorOnce(ctx, api, m, writers, sources[m.Name], announce, epoch, holds)
if why != failing[m.Name] {
if why != "" {
fmt.Printf("the %s mirror cannot be kept: %s\n", m.Name, why)
} else {
fmt.Printf("the %s mirror is kept again\n", m.Name)
}
failing[m.Name] = why
}
}
}
select {
case <-ctx.Done():
return
case <-tick.C:
}
}
}
// mirrorOnce brings one mirror to what the store holds, and answers why it could not; empty when it could.
func mirrorOnce(ctx context.Context, api jetstream.JetStream, m recordmirror.Mirror, writers map[string]*recordmirror.Writer,
source func(context.Context) (map[string][]byte, error), announce recordmirror.Announcer, epoch func() uint64,
holds func() bool) string {
reading, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
w := writers[m.Name]
if w == nil {
kv, err := api.KeyValue(reading, m.Bucket)
if err != nil {
return fmt.Sprintf("the bucket %s cannot be read: %v", m.Bucket, err)
}
w = &recordmirror.Writer{Mirror: m, KV: kv, Announce: announce, Epoch: epoch, Holds: holds}
writers[m.Name] = w
}
want, err := source(reading)
if err != nil {
return fmt.Sprintf("the store cannot be read: %v", err)
}
lost := w.LostEvents
r, err := w.Sync(reading, want)
if r.Written+r.Removed > 0 {
fmt.Printf("the %s mirror moved: %d written, %d removed\n", m.Name, r.Written, r.Removed)
}
if w.LostEvents > lost {
fmt.Printf("the %s mirror lost %d event(s): the bucket holds the change, its notice was refused twice\n",
m.Name, w.LostEvents-lost)
}
if err != nil {
return err.Error()
}
return ""
}
-204
View File
@@ -1,204 +0,0 @@
package main
import (
"bytes"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
snapshot "github.com/novox/mesh-controller/internal/facts"
"github.com/novox/mesh-controller/internal/inventory"
)
// A file's text kept as a template file in its module's repository (novox/hq ADR 0349 §2, design 57 package T1,
// issue 654): `module check` and the merge gate read it from the tree beside the manifest, so a change to a
// template alone is judged as a change to its module.
const templatedNotifier = `{"module":"notifier","version":"1",
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
"resources":[{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644","template":"files/notifier.conf"}]}`
// inlineNotifier is the same module as the mesh holds it: built, its text inline.
const inlineNotifier = `{"module":"notifier","version":"1",
"resources":[{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644","content":"volume = 3\n"}]}`
// aTemplatedModule writes a module's directory: its manifest and its files/.
func aTemplatedModule(t *testing.T, dir, manifest string, files map[string]string) string {
t.Helper()
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, "module.json")
if err := os.WriteFile(path, []byte(manifest), 0o644); err != nil {
t.Fatal(err)
}
for name, text := range files {
full := filepath.Join(dir, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte(text), 0o644); err != nil {
t.Fatal(err)
}
}
return path
}
func TestModuleCheckReadsTemplatesFromTheTreeBesideTheManifest(t *testing.T) {
good := aTemplatedModule(t, t.TempDir(), templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
var out bytes.Buffer
if err := moduleCheck([]string{good}, &out); err != nil || !strings.Contains(out.String(), "notifier: ok") {
t.Fatalf("a module with its text in files/ was refused: %v\n%s", err, out.String())
}
// A change to the template alone, to text the controller refuses inline, is refused the same.
bad := aTemplatedModule(t, t.TempDir(), templatedNotifier,
map[string]string{"files/notifier.conf": "volume = 3\nstore = ${dir:nowhere}\n"})
out.Reset()
if err := moduleCheck([]string{bad}, &out); err == nil || !strings.Contains(out.String(), "nowhere") {
t.Fatalf("a template whose text is refused inline passed:\n%s", out.String())
}
// A symlink under files/ that no template names: the build's packing refuses it, so the check does too.
linked := t.TempDir()
aTemplatedModule(t, linked, templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
if err := os.Symlink("/etc/hostname", filepath.Join(linked, "files", "elsewhere")); err != nil {
t.Fatal(err)
}
out.Reset()
if err := moduleCheck([]string{filepath.Join(linked, "module.json")}, &out); err == nil ||
!strings.Contains(out.String(), "files/elsewhere is a symlink") {
t.Fatalf("a symlink under files/ passed the check:\n%s", out.String())
}
// And a template the tree lacks, by name.
missing := aTemplatedModule(t, t.TempDir(), templatedNotifier, nil)
out.Reset()
if err := moduleCheck([]string{missing}, &out); err == nil || !strings.Contains(out.String(), "files/notifier.conf") {
t.Fatalf("a missing template was not refused by name:\n%s", out.String())
}
}
// templatedMesh is catalogueMesh with the notifier on the laptop, registered as built: its text inline.
func templatedMesh(t *testing.T) (snapshot.Facts, map[string]string) {
t.Helper()
open := aMesh(t)
ctx := t.Context()
manifests := map[string]string{
"lemurs": `{"module":"lemurs","version":"1","capabilities":["systemd"],
"resources":[{"id":"service","type":"service","unit":"lemurs.service","state":"running","boot":"enabled"}]}`,
"notifier": inlineNotifier,
}
for name, raw := range manifests {
m, err := catalogue.ParseManifest([]byte(raw))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if err := open.inventory.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-catalog",
Path: "modules/" + name, BuiltFrom: "c0ffee"}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", name); err != nil {
t.Fatalf("assign %s: %v", name, err)
}
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
return f, manifests
}
// aTemplatedTree is the catalogue's checkout with the notifier's text in files/.
func aTemplatedTree(t *testing.T, manifests map[string]string, text string, has bool) string {
t.Helper()
tree := aTree(t, withEdit(manifests, "notifier", ""))
files := map[string]string{}
if has {
files["files/notifier.conf"] = text
}
aTemplatedModule(t, filepath.Join(tree, "modules", "notifier"), templatedNotifier, files)
return tree
}
const notifierTemplate = "modules/notifier/files/notifier.conf"
func TestTheMergeGateJudgesAChangeToATemplateAlone(t *testing.T) {
f, manifests := templatedMesh(t)
// The module moved to its template, the same text: nothing changes on any machine.
v := gateJudged(t, f, aTemplatedTree(t, manifests, "volume = 3\n", true), "modules/notifier/module.json",
notifierTemplate)
if v.Verdict == "fail" || v.Modules["notifier"] != "" {
t.Fatalf("the module moved to its template with the same text reads as a change (%v):\n%s", v.Modules, v.Report())
}
// The template alone changed: its module changed, and composes.
v = gateJudged(t, f, aTemplatedTree(t, manifests, "volume = 4\n", true), notifierTemplate)
if v.Verdict == "fail" || v.Modules["notifier"] != "changed" {
t.Fatalf("a change to the template alone is not a change to its module (%v):\n%s", v.Modules, v.Report())
}
if v.Width == nil || strings.Join(v.Width.Modules, ",") != "notifier" {
t.Fatalf("a change to the template rebuilds %+v, not its module", v.Width)
}
// The template alone changed to text the controller refuses: the pull request fails, naming the module.
v = gateJudged(t, f, aTemplatedTree(t, manifests, "store = ${dir:nowhere}\n", true), notifierTemplate)
if v.Verdict != "fail" || !strings.Contains(v.Report(), "notifier") {
t.Fatalf("a template the controller refuses passed the gate:\n%s", v.Report())
}
// A symlinked files/ fails: nothing is read from outside the tree.
tree := aTree(t, withEdit(manifests, "notifier", ""))
elsewhere := t.TempDir()
if err := os.WriteFile(filepath.Join(elsewhere, "notifier.conf"), []byte("volume = 3\n"), 0o644); err != nil {
t.Fatal(err)
}
aTemplatedModule(t, filepath.Join(tree, "modules", "notifier"), templatedNotifier, nil)
if err := os.Symlink(elsewhere, filepath.Join(tree, "modules", "notifier", "files")); err != nil {
t.Fatal(err)
}
v = gateJudged(t, f, tree, notifierTemplate)
if v.Verdict != "fail" || !strings.Contains(v.Report(), "files/ is a symlink") {
t.Fatalf("a symlinked files/ passed the gate:\n%s", v.Report())
}
// And the template removed from the tree fails, naming it.
v = gateJudged(t, f, aTemplatedTree(t, manifests, "", false), notifierTemplate)
if v.Verdict != "fail" || !strings.Contains(v.Report(), "files/notifier.conf") {
t.Fatalf("a missing template passed the gate:\n%s", v.Report())
}
}
// What the gate composes for the module in its template form is what the mesh composes for it inline.
func TestTheMergeGateResolvesATemplatedModuleToItsInlineTwin(t *testing.T) {
dir := t.TempDir()
path := aTemplatedModule(t, dir, templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
m, err := catalogue.ParseManifest(raw)
if err != nil {
t.Fatal(err)
}
m, err = catalogue.WithTemplatesFrom(m, dir)
if err != nil {
t.Fatal(err)
}
resolved, err := standInBuild(m)
if err != nil {
t.Fatal(err)
}
inline, err := catalogue.ParseManifest([]byte(inlineNotifier))
if err != nil {
t.Fatal(err)
}
if builtAlike(resolved) != builtAlike(inline) {
got, _ := json.Marshal(resolved)
t.Fatalf("the templated module resolves to %s, not its inline twin", got)
}
}
@@ -1,46 +0,0 @@
[
{
"id": "found.server",
"image": "registry.invalid:5000/library@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"networks": [
"predecessor_default"
],
"type": "container"
},
{
"id": "library.access-storage/downloads",
"mode": "read",
"path": "/storage/downloads",
"type": "access"
},
{
"id": "library.access-storage/media/series",
"mode": "read",
"path": "/storage/media/series",
"type": "access"
},
{
"id": "library.config",
"mode": "0750",
"owner": "1001:2000",
"path": "/var/lib/library/config",
"type": "directory"
},
{
"content": "{\n \"accesses\": {\n \"downloads\": \"/storage/downloads\",\n \"series\": \"/storage/media/series\"\n },\n \"log\": \"warn\",\n \"places\": {\n \"config\": {\n \"owner\": \"1001:2000\",\n \"path\": \"/var/lib/library/config\"\n }\n },\n \"workers\": 4\n}\n",
"id": "library.runtime-config",
"mode": "0640",
"path": "/var/lib/library/config/runtime.json",
"type": "file"
},
{
"id": "library.server",
"image": "registry.invalid:5000/library@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"type": "container",
"volumes": [
"/var/lib/library/config:/config",
"/storage/media/series:/series",
"/storage/downloads:/downloads:ro"
]
}
]
@@ -1,37 +0,0 @@
[
{
"id": "library.access-home/media/series",
"mode": "read",
"path": "/home/media/series",
"type": "access"
},
{
"id": "library.access-storage/downloads",
"mode": "read",
"path": "/storage/downloads",
"type": "access"
},
{
"id": "library.config",
"mode": "0750",
"path": "/var/lib/library/config",
"type": "directory"
},
{
"content": "{\n \"accesses\": {\n \"downloads\": \"/storage/downloads\",\n \"series\": \"/home/media/series\"\n },\n \"log\": \"warn\"\n}\n",
"id": "library.runtime-config",
"mode": "0640",
"path": "/var/lib/library/config/runtime.json",
"type": "file"
},
{
"id": "library.server",
"image": "registry.invalid:5000/library@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"type": "container",
"volumes": [
"/var/lib/library/config:/config",
"/home/media/series:/series",
"/storage/downloads:/downloads:ro"
]
}
]
+58 -2
View File
@@ -40,6 +40,39 @@ var (
AskedBucket = BucketName(ControllerSeat, "asked")
)
// The controller's record mirrors (novox/hq ADR 0289, design 54, design 57 package F3): a bucket per kind of
// record, one key per record, written from the store by the lease holder only when the record changed, with
// one event on the controller's seat per write. Read by whatever names them in `reads` as
// `mesh-controller.<mirror>` (ADR 0201 §1), written by nobody else. **No secret in either** (ADR 0201): an
// assignment's value names its module and node; a module's declared config is what its manifest, which is
// public, declares.
//
// AssignmentsBucket holds which module is assigned to which node, keyed `<node>.<module>`; and
// DeclaredConfigBucket each registered module's config declarations as its manifest holds them, keyed by
// the module: what mesh-config resolves a config value from (ADR 0348 clauses 2 and 4).
var (
AssignmentsBucket = BucketName(ControllerSeat, MirrorAssignments)
DeclaredConfigBucket = BucketName(ControllerSeat, MirrorDeclaredConfig)
)
// The mirrors' local names, as a reader names them in `reads` after `mesh-controller.`.
const (
MirrorAssignments = "assignments"
MirrorDeclaredConfig = "declared-config"
)
// ControllerMirrors are the local names of the controller's record mirrors, which a module may read.
func ControllerMirrors() []string { return []string{MirrorAssignments, MirrorDeclaredConfig} }
// The mirrors' caps (design 54 §5): one value per key, no age, 64 KiB a value, 64 MiB a bucket.
const (
MirrorValueBytes = 64 << 10
MirrorBytes = 64 << 20
// MirrorMarkerTTL is the longest a purge's marker is kept (design 54 §2): the bucket allows a
// per-message age for it, and the writer gives each marker a day.
MirrorMarkerTTL = 24 * time.Hour
)
// AskedKeptFor is how long an ask is kept after it was made: a month, as the router keeps its own.
const AskedKeptFor = 30 * 24 * time.Hour
@@ -66,12 +99,14 @@ const (
// IsControllerBucket says a bucket is the controller's own, not a module's state nothing declares.
func IsControllerBucket(bucket string) bool {
return bucket == CallsBucket || bucket == HandActsBucket || bucket == ConditionsBucket ||
bucket == ConditionHistoryBucket || bucket == LeaseBucket || bucket == AskedBucket
bucket == ConditionHistoryBucket || bucket == LeaseBucket || bucket == AskedBucket ||
bucket == AssignmentsBucket || bucket == DeclaredConfigBucket
}
// ControllerBuckets are the controller's own buckets, in the order they are asserted.
func ControllerBuckets() []string {
return []string{LeaseBucket, CallsBucket, HandActsBucket, ConditionsBucket, ConditionHistoryBucket, AskedBucket}
return []string{LeaseBucket, CallsBucket, HandActsBucket, ConditionsBucket, ConditionHistoryBucket, AskedBucket,
AssignmentsBucket, DeclaredConfigBucket}
}
// ControllerBucketsAsserter is what raising the controller's buckets needs of a connection.
@@ -168,6 +203,27 @@ func (j *JetStream) EnsureControllerBuckets() error {
}); err != nil {
return fmt.Errorf("asserting bucket %s: %w", AskedBucket, err)
}
// **The record mirrors: no age** (design 54 §5). A key is removed when its record is, by a purge whose
// marker lives a day; a key that expired would be a record the bus forgot while the store still holds it.
for _, m := range []struct{ bucket, why string }{
{AssignmentsBucket, "which module is assigned to which node, keyed <node>.<module> (novox/hq ADR 0289, " +
"design 54): written by the controller alone from its store when an assignment changes, read through `reads`"},
{DeclaredConfigBucket, "each registered module's config declarations as its manifest holds them, keyed by " +
"the module (novox/hq ADR 0348, design 57 F3): written by the controller alone from its store when a " +
"registration changes them, read through `reads`"},
} {
if _, err := js.CreateOrUpdateKeyValue(ctx, jetstream.KeyValueConfig{
Bucket: m.bucket,
Description: m.why,
History: 1,
MaxValueSize: MirrorValueBytes,
MaxBytes: MirrorBytes,
Storage: jetstream.FileStorage,
LimitMarkerTTL: MirrorMarkerTTL,
}); err != nil {
return fmt.Errorf("asserting bucket %s: %w", m.bucket, err)
}
}
return nil
}
+83 -1
View File
@@ -21,7 +21,8 @@ func TestTheControllerMayWriteEveryBucketItWrites(t *testing.T) {
if err != nil {
t.Fatal(err)
}
want := []string{"$KV." + CallsBucket + ".>", "$KV." + HandActsBucket + ".>"}
want := []string{"$KV." + CallsBucket + ".>", "$KV." + HandActsBucket + ".>",
"$KV." + AssignmentsBucket + ".>", "$KV." + DeclaredConfigBucket + ".>"}
for _, seat := range seatsTheControllerAsks {
if hasCancelledSet(seat) {
want = append(want, "$KV."+CancelledSetName(seat)+".>")
@@ -37,6 +38,34 @@ func TestTheControllerMayWriteEveryBucketItWrites(t *testing.T) {
}
}
// **A module reads a record mirror through `reads`, and never writes it** (ADR 0201 §1, design 54 §5,
// design 57 F3): granted the direct reads and watches of the bucket, and refused at composition a grant to
// write it, whose writer is the controller.
func TestAModuleReadsTheControllersMirrorsAndWritesNone(t *testing.T) {
reader := Principal{Kind: KindModule, Node: "one", Module: "mesh-config", PasswordHash: "x",
Reads: []string{"mesh-controller." + MirrorAssignments, "mesh-controller." + MirrorDeclaredConfig}}
p, err := PermissionsFor(reader)
if err != nil {
t.Fatal(err)
}
for _, bucket := range []string{AssignmentsBucket, DeclaredConfigBucket} {
for _, subject := range []string{"$JS.API.STREAM.INFO.KV_" + bucket, "$JS.API.DIRECT.GET.KV_" + bucket + ".>",
"$JS.API.CONSUMER.CREATE.KV_" + bucket + ".>"} {
if !slices.Contains(p.Publish, subject) {
t.Errorf("a module reading %s may not publish %s", bucket, subject)
}
}
for _, subject := range p.Publish {
if SubjectsOverlap(subject, "$KV."+bucket+".>") {
t.Errorf("a module reading %s may write it through %s", bucket, subject)
}
}
if err := CheckWriters(reader, []string{"$KV." + bucket + ".>"}); err == nil {
t.Errorf("a module granted a write of %s would compose", bucket)
}
}
}
// **A machine's node tools may say they are there, as that machine and no other** (novox/hq to-be 45
// S11), and the controller may hear the bus's advisories and ask who answers — read-only, named.
func TestTheWatchedSignalsMayBeSaidAndHeard(t *testing.T) {
@@ -96,3 +125,56 @@ func TestWhatTheControllerAskedIsBounded(t *testing.T) {
t.Errorf("history %d, age %s, bytes %d", status.History(), status.TTL(), info.Config.MaxBytes)
}
}
// **A listed mirror is no undeclared bucket** (ADR 0356 §3, ADR 0201 §16): the controller's list of its mirrors
// stands for their declaration, so the report of state nothing declares never names one.
func TestAListedMirrorIsNotReportedAsUndeclared(t *testing.T) {
names := []string{"gone_state"}
for _, m := range ControllerMirrors() {
names = append(names, BucketName(ControllerSeat, m))
}
undeclared, err := RaiseBuckets(fakeBuckets{names: names}, nil)
if err != nil {
t.Fatal(err)
}
if !slices.Equal(undeclared, []string{"gone_state"}) {
t.Fatalf("the report of undeclared state names %v", undeclared)
}
}
// **The mirrors' list and the writers table agree** (ADR 0356 §3, to-be 45 §1): every listed mirror is a row of
// the table whose one writer is the controller, on its bucket alone; and every row that writes a bucket of the
// controller's named as a mirror is on the list.
func TestTheMirrorsListAndTheWritersTableAgree(t *testing.T) {
controller := Principal{Kind: KindController}
module := Principal{Kind: KindModule, Module: ControllerSeat, Node: "one"}
rowed := map[string]bool{}
for _, row := range WritersTable {
if !strings.HasSuffix(row.State, " mirror") {
continue
}
for _, subject := range row.Subjects {
bucket := strings.TrimSuffix(strings.TrimPrefix(subject, "$KV."), ".>")
rowed[bucket] = true
if row.Writes == nil || !row.Writes(controller, subject) || row.Writes(module, subject) {
t.Errorf("%q is not written by the controller alone", row.State)
}
}
}
listed := map[string]bool{}
for _, m := range ControllerMirrors() {
bucket := BucketName(ControllerSeat, m)
listed[bucket] = true
if !rowed[bucket] {
t.Errorf("the mirror %s is listed and no row of the writers table writes it", bucket)
}
if !IsControllerBucket(bucket) || !slices.Contains(ControllerBuckets(), bucket) {
t.Errorf("the mirror %s is listed and not asserted as the controller's own", bucket)
}
}
for bucket := range rowed {
if !listed[bucket] {
t.Errorf("the writers table writes the mirror %s, which the list does not name", bucket)
}
}
}
+6 -2
View File
@@ -172,7 +172,9 @@ func TestRaisingLogsReportsWhatNothingDeclares(t *testing.T) {
// **No path of the mesh deletes or purges a log or a bucket, or recreates one** (novox/hq ADR 0297 §2,
// ADR 0201 §15–16): no code of this repository outside its tests calls the client's stream or bucket
// delete, or purges a stream, but for the controller lease's own stream, which purges its own history
// below a sequence (internal/lease). A new call is a decision, not a refactor.
// below a sequence (internal/lease), and the controller's record mirrors, which purge the key of a record
// the store no longer holds (internal/recordmirror; novox/hq ADR 0289, design 54 §2). A new call is a decision,
// not a refactor.
func TestNoPathDeletesALogOrABucket(t *testing.T) {
removers := map[string]bool{"DeleteStream": true, "DeleteKeyValue": true, "PurgeStream": true,
"DeleteObjectStore": true}
@@ -197,7 +199,9 @@ func TestNoPathDeletesALogOrABucket(t *testing.T) {
return true
}
name := sel.Sel.Name
if removers[name] || (name == "Purge" && !strings.Contains(filepath.ToSlash(path), "internal/lease/")) {
decided := strings.Contains(filepath.ToSlash(path), "internal/lease/") ||
strings.Contains(filepath.ToSlash(path), "internal/recordmirror/")
if removers[name] || (name == "Purge" && !decided) {
found = append(found, path+": "+name)
}
return true
+7
View File
@@ -34,6 +34,8 @@ func TestTheFactsTheGrantPermitsAreTheFactsTheMeshStates(t *testing.T) {
states = append(states, link.KeyChecked)
// And a walk kept in a new state, for the delivery it walks (novox/hq ADR 0239).
states = append(states, link.KeyPlanMoved)
// And a key of a record mirror written or removed (novox/hq ADR 0289, design 54 §3; design 57 F3).
states = append(states, link.KeyAssignmentChanged, link.KeyDeclaredConfigChanged)
for _, event := range states {
if !slices.Contains(broker.ControllerStates, event) {
t.Errorf("the mesh states %q and its account may not publish it", event)
@@ -42,6 +44,11 @@ func TestTheFactsTheGrantPermitsAreTheFactsTheMeshStates(t *testing.T) {
if len(broker.ControllerStates) != len(states) {
t.Errorf("the grant permits %v, which is more than the mesh states", broker.ControllerStates)
}
// The record mirrors a module may read are the ones the bus holds (design 54 §5, design 57 F3).
if !slices.Equal(catalogue.ControllerMirrors, broker.ControllerMirrors()) {
t.Errorf("the catalogue lets a module read the controller's mirrors %v and the bus holds %v",
catalogue.ControllerMirrors, broker.ControllerMirrors())
}
// **And the seat says it.** A seat carries the protocol of its role (novox/hq ADR 0129), so the
// facts the control plane states are the seat's `emits` — which is what lets anything else declare
// that it consumes them, and what the subject-agreement check reads to know they have an owner.
+4 -1
View File
@@ -313,7 +313,10 @@ var ControllerStates = []string{"applied", "refused", "built-before",
// the pull request's status, and anybody else may read.
"checked",
// And a walk kept in a new state (novox/hq ADR 0239): what the delivery it walks reads its steps from.
"plan-moved"}
"plan-moved",
// And a key of a record mirror written or removed (novox/hq ADR 0289, design 54 §3): the assignments and
// each registered module's declared config (design 57 F3).
"assignment-changed", "declared-config-changed"}
// BusAdvisories are what the bus server says about the mesh's own account that the controller
// reads (novox/hq to-be 45 §3, S9): a durable consumer that handed a message over as often as it
+1 -1
View File
@@ -24,7 +24,7 @@ accounts {
jetstream: enabled
users = [
{ user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.DEAD_LETTER_NOTICES.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>", "$KV.SEAT_NODE_BUILD_AGENT_cancelled.>", "$KV.mesh-controller_asked.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.again.>", "mesh.assignment.>", "mesh.events.dead.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-build-machine.tool.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.checked", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.plan-moved", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.rolled-back", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.mesh-controller.tool.plans", "mesh.seat.mesh-delivery.tool.close", "mesh.seat.mesh-delivery.tool.groups", "mesh.seat.mesh-delivery.tool.release", "mesh.seat.mesh-delivery.tool.stalled", "mesh.seat.mesh-delivery.tool.stop", "mesh.seat.mesh-delivery.tool.times", "mesh.seat.node-backup.tool.backed-up.*", "mesh.seat.node-backup.tool.now.*", "mesh.seat.node-build-agent.accept.>", "mesh.seat.node-build-agent.tool.>", "mesh.seat.node-intrusion-prevention.tool.banned.*", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-service-manager.tool.restart.*"] }
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.DEAD_LETTER_NOTICES.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>", "$KV.SEAT_NODE_BUILD_AGENT_cancelled.>", "$KV.mesh-controller_asked.>", "$KV.mesh-controller_assignments.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_declared-config.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.again.>", "mesh.assignment.>", "mesh.events.dead.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-build-machine.tool.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.assignment-changed", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.checked", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.declared-config-changed", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.plan-moved", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.rolled-back", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.mesh-controller.tool.plans", "mesh.seat.mesh-delivery.tool.close", "mesh.seat.mesh-delivery.tool.groups", "mesh.seat.mesh-delivery.tool.release", "mesh.seat.mesh-delivery.tool.stalled", "mesh.seat.mesh-delivery.tool.stop", "mesh.seat.mesh-delivery.tool.times", "mesh.seat.node-backup.tool.backed-up.*", "mesh.seat.node-backup.tool.now.*", "mesh.seat.node-build-agent.accept.>", "mesh.seat.node-build-agent.tool.>", "mesh.seat.node-intrusion-prevention.tool.banned.*", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-service-manager.tool.restart.*"] }
subscribe: { allow: ["$JS.API.>", "$JS.EVENT.ADVISORY.CONSUMER.DELETED.>", "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "$SRV.STATS", "$SRV.STATS.mesh-controller", "$SRV.STATS.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered", "mesh.mod.*.event.provisioner.retirement", "mesh.mod.gitea.event.pull.merged", "mesh.mod.gitea.event.pull.updated", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built", "mesh.seat.operator-channel.event.decided.mesh-controller"] }
allow_responses: { max: 1, ttl: "1m" }
} }
+9
View File
@@ -158,6 +158,15 @@ var WritersTable = []WriterRow{
{State: "a message a consumer gave up on", Writer: "controller", KeptIn: "the bus, the stream " + DeadLettersStream,
Others: "read, delivered again or dropped through the controller's dead-letters verb",
Subjects: []string{deadLetterPrefix + ">", againPrefix + ">"}, Writes: isController},
// The record mirrors (novox/hq ADR 0289 §9, design 54 §2): rows of this table, so a grant that lets
// anything else write them is refused at composition. Built so far: the assignments and each registered
// module's declared config (design 57 F3).
{State: "the assignments mirror", Writer: "controller (lease holder), from its store, only on a change",
KeptIn: "key-value " + AssignmentsBucket, Others: "read through `reads` as mesh-controller." + MirrorAssignments,
Subjects: kvOf(AssignmentsBucket), Writes: isController},
{State: "the declared-config mirror", Writer: "controller (lease holder), from its store, only on a change",
KeptIn: "key-value " + DeclaredConfigBucket, Others: "read through `reads` as mesh-controller." + MirrorDeclaredConfig,
Subjects: kvOf(DeclaredConfigBucket), Writes: isController},
}
// CheckWriters refuses a grant that lets a principal publish on a subject the writers table gives
+3
View File
@@ -34,6 +34,9 @@ var designRows = []string{
"the operator-channel's open messages",
"the facts snapshot",
"a message a consumer gave up on",
// The record mirrors, rows of to-be 45 §1's table by design 54 §2 (ADR 0289 §9).
"the assignments mirror",
"the declared-config mirror",
}
func TestTheWritersTableIsTheDesigns(t *testing.T) {
+1 -13
View File
@@ -268,14 +268,7 @@ func build(ctx context.Context, run Runner, publish Publisher,
}
// What is judged only where a definition is written (novox/hq ADR 0326 §3, issue 574): a build is
// where a changed definition is first read, so it is refused here before anything is made of it.
// Judged with each template's text in its file, as written in the tree (novox/hq ADR 0349 §2): a template the
// module's directory lacks is refused here, before anything is built.
judged, err := catalogue.WithTemplatesFrom(manifest, within)
if err != nil {
say("manifest", "INVALID: %v", err)
return Result{}, err
}
if problems := catalogue.WrittenSettingsProblems(judged); len(problems) > 0 {
if problems := catalogue.WrittenSettingsProblems(manifest); len(problems) > 0 {
err := fmt.Errorf("%s", strings.Join(problems, "; "))
say("manifest", "INVALID: %v", err)
return Result{}, err
@@ -417,11 +410,6 @@ func build(ctx context.Context, run Runner, publish Publisher,
}
}
// Each template's text, read back from the archive the build made or took (novox/hq ADR 0349 §2).
if built, err = withTemplateTexts(manifest, within, built); err != nil {
say("templates", "FAILED: %v", err)
return Result{}, err
}
resolved, err := manifest.Resolve(built)
if err != nil {
say("resolve", "FAILED: %v", err)
+2 -14
View File
@@ -73,14 +73,8 @@ const reusedModule = `{"module":"meshboard","version":"1",
// aModuleAt is a repository holding the module under mod/, whose directory git names as tree, on its trunk.
func aModuleAt(t *testing.T, tree string) (*recorded, Runner, string) {
t.Helper()
return aModuleWith(t, tree, map[string]string{"mod/" + ManifestName: reusedModule,
"mod/Dockerfile": "FROM scratch", "mod/files/theme.conf": "dark"})
}
// aModuleWith is aModuleAt with the repository's files given.
func aModuleWith(t *testing.T, tree string, contents map[string]string) (*recorded, Runner, string) {
t.Helper()
r := &recorded{tree: tree, contents: contents}
r := &recorded{tree: tree, contents: map[string]string{"mod/" + ManifestName: reusedModule,
"mod/Dockerfile": "FROM scratch", "mod/files/theme.conf": "dark"}}
run := func(ctx context.Context, dir, name string, args ...string) (string, error) {
if name == "git" && len(args) > 0 && args[0] == "symbolic-ref" {
r.ran = append(r.ran, name+" "+strings.Join(args, " "))
@@ -106,12 +100,6 @@ var aPull = PullRef{Owner: "novox", Repo: "meshboard", Number: 7}
func aCheckBuilt(t *testing.T, scratch *aScratch, tree string) CheckBuilt {
t.Helper()
r, run, workspace := aModuleAt(t, tree)
return aCheckBuiltBy(t, scratch, r, run, workspace)
}
// aCheckBuiltBy is aCheckBuilt of the repository given.
func aCheckBuiltBy(t *testing.T, scratch *aScratch, r *recorded, run Runner, workspace string) CheckBuilt {
t.Helper()
spec := CheckSpec{ID: "build-1", Owner: aPull.Owner, Repo: aPull.Repo, Number: aPull.Number,
Builds: []CheckBuildSpec{{Module: "meshboard", Repository: "https://forge.invalid/meshboard.git", Path: "mod",
Ref: "main"}}}
-94
View File
@@ -1,94 +0,0 @@
package builder
import (
"archive/tar"
"bytes"
"compress/gzip"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"io"
"path/filepath"
"github.com/novox/mesh-controller/internal/catalogue"
)
// withTemplateTexts is what was built with each template's text, for a module whose files name templates
// (novox/hq ADR 0349 §2): its templates archive gains the text of every file in it, read back from the packed
// bytes, by its path inside the archive. Resolve reads them into the files' content.
//
// **Read from the archive, proven by its digest.** The module's files/ is packed again, as the build packs it,
// and the bytes are read only when they hash to the digest the build made or took: a fresh build's own archive,
// or a check's build a walk copied by digest without building (novox/hq ADR 0350 §2). Packing is deterministic,
// so one source packs to one digest; a tree that packs to another is not the archive, and its text is refused.
func withTemplateTexts(manifest catalogue.Manifest, within string, built []catalogue.Built) ([]catalogue.Built, error) {
if !manifest.Templated() {
return built, nil
}
name := catalogue.TemplatesArtifact(manifest)
at := -1
for i, b := range built {
if b.Name == name {
at = i
}
}
if name == "" || at < 0 {
return nil, fmt.Errorf("%s names templates and built no archive of %s/ to read them from",
manifest.Module, catalogue.TemplatesDir)
}
body, err := pack(filepath.Join(within, catalogue.TemplatesDir))
if err != nil {
return nil, fmt.Errorf("%s: packing %s/ to read its templates back failed: %w", manifest.Module,
catalogue.TemplatesDir, err)
}
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
if built[at].Digest != digest {
return nil, fmt.Errorf("%s: %s/ packs to the digest %s, and the archive %q built or taken is %s: the "+
"templates are read only from the archive the mesh records", manifest.Module, catalogue.TemplatesDir,
digest, name, orNone(built[at].Digest))
}
texts, err := unpackTexts(body)
if err != nil {
return nil, fmt.Errorf("%s: reading the archive %q back failed: %w", manifest.Module, name, err)
}
out := append([]catalogue.Built(nil), built...)
out[at].Texts = texts
return out, nil
}
// unpackTexts is every file of a packed archive (pack), by its path inside it.
func unpackTexts(body []byte) (map[string]string, error) {
zipped, err := gzip.NewReader(bytes.NewReader(body))
if err != nil {
return nil, err
}
defer zipped.Close()
reader := tar.NewReader(zipped)
texts := map[string]string{}
for {
header, err := reader.Next()
if errors.Is(err, io.EOF) {
return texts, nil
}
if err != nil {
return nil, err
}
if header.Typeflag != tar.TypeReg {
continue
}
text, err := io.ReadAll(reader)
if err != nil {
return nil, err
}
texts[header.Name] = string(text)
}
}
func orNone(digest string) string {
if digest == "" {
return "no digest"
}
return digest
}
-136
View File
@@ -1,136 +0,0 @@
package builder
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// A module's files kept as template files in its repository (novox/hq ADR 0349 §2, design 57 package T1, issue
// 654): the build packs files/ into the module's templates archive and reads each template's text back from the
// packed bytes into the file's content, so the manifest the mesh holds is the one it would hold for the text inline.
const templatedModule = `{"module":"meshboard","version":"1",
"build":{"artifacts":[
{"name":"server","kind":"image","from":"Dockerfile"},
{"name":"files","kind":"archive","from":"files"}]},
"resources":[
{"id":"svc","type":"container","name":"meshboard","artifact":"server"},
{"id":"conf","type":"file","path":"/etc/meshboard/meshboard.conf","template":"files/meshboard.conf"},
{"id":"deeper","type":"file","path":"/etc/meshboard/deeper.conf","template":"files/a/deeper.conf"}]}`
var templatedFiles = map[string]string{
"Dockerfile": "FROM scratch", "files/meshboard.conf": "listen = 8080\n\"quoted\" é\n",
"files/a/deeper.conf": "deeper\n", "files/unused.conf": "packed, named by nothing",
}
func contentOf(t *testing.T, m catalogue.Manifest, id string) string {
t.Helper()
for _, r := range m.Resources {
if r["id"] == id {
if _, has := r[catalogue.TemplateField]; has {
t.Fatalf("%s still names its template in the built manifest: %v", id, r)
}
text, ok := r["content"].(string)
if !ok {
t.Fatalf("%s carries no content: %v", id, r)
}
return text
}
}
t.Fatalf("no resource %s in %v", id, m.Resources)
return ""
}
func TestABuildReadsEachTemplateBackFromItsArchive(t *testing.T) {
r, workspace := aRepository(t, templatedModule, templatedFiles)
got, err := Build(context.Background(), r.run, r, "https://forge.invalid/meshboard.git", "", "", workspace, nil,
Npmrc{}, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if text := contentOf(t, got.Manifest, "conf"); text != templatedFiles["files/meshboard.conf"] {
t.Fatalf("the template's text is %q, not the file's", text)
}
if text := contentOf(t, got.Manifest, "deeper"); text != templatedFiles["files/a/deeper.conf"] {
t.Fatalf("the template's text is %q, not the file's", text)
}
if r.archives["meshboard/files"] == "" {
t.Fatalf("the templates archive was not published: %v", r.archives)
}
if problem := catalogue.HeldTemplateProblem(got.Manifest); problem != "" {
t.Fatal(problem)
}
}
func TestABuildWhoseTemplateIsMissingFailsNamingIt(t *testing.T) {
files := map[string]string{}
for k, v := range templatedFiles {
files[k] = v
}
delete(files, "files/a/deeper.conf")
r, workspace := aRepository(t, templatedModule, files)
_, err := Build(context.Background(), r.run, r, "https://forge.invalid/meshboard.git", "", "", workspace, nil,
Npmrc{}, GitCredential{}, nil)
if err == nil || !strings.Contains(err.Error(), "files/a/deeper.conf") {
t.Fatalf("a build whose template is missing did not fail naming it: %v", err)
}
}
// A walk that takes its check's build copies the archive by digest and builds nothing: the templates are read back
// from the tree packed again, which is the archive only when it packs to the digest taken.
func TestAWalkTakingAChecksBuildReadsTheTemplatesBackToo(t *testing.T) {
withVersion(t, "builder-1")
contents := map[string]string{"mod/" + ManifestName: templatedModule}
for k, v := range templatedFiles {
contents["mod/"+k] = v
}
scratch := newScratch()
r, run, workspace := aModuleWith(t, "tree-one", contents)
checked := aCheckBuiltBy(t, scratch, r, run, workspace)
r, run, workspace = aModuleWith(t, "tree-one", contents)
got, err := BuildReusing(t.Context(), run, r, Reuse{Scratch: scratch, Pulls: []PullRef{aPull}, Named: named(checked)},
"https://forge.invalid/meshboard.git", "mod", "main", workspace, nil, Npmrc{}, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if got.Reused == nil || len(r.archives) > 0 {
t.Fatalf("the walk built again: reused %+v, why %q, published %v", got.Reused, got.Why, r.archives)
}
if text := contentOf(t, got.Manifest, "conf"); text != templatedFiles["files/meshboard.conf"] {
t.Fatalf("the template's text is %q, not the file's", text)
}
}
// The tree packed again must be the archive taken; text read from anything else is not the archive's.
func TestTemplateTextsAreRefusedWhenTheTreeDoesNotPackToTheDigestTaken(t *testing.T) {
m, err := catalogue.ParseManifest([]byte(templatedModule))
if err != nil {
t.Fatal(err)
}
within := writeTree(t, t.TempDir(), templatedFiles)
built := []catalogue.Built{{Name: "files", Kind: catalogue.ArtifactArchive, Digest: "sha256:" + strings.Repeat("0", 64)}}
if _, err := withTemplateTexts(m, within, built); err == nil || !strings.Contains(err.Error(), "digest") {
t.Fatalf("text was read from a tree that does not pack to the archive taken: %v", err)
}
}
// writeTree writes files under dir and answers dir.
func writeTree(t *testing.T, dir string, files map[string]string) string {
t.Helper()
for name, body := range files {
full := filepath.Join(dir, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
return dir
}
-24
View File
@@ -35,10 +35,6 @@ type Built struct {
// Launchers are, for a bundle in an interpreted language, the executable the build wrote beside
// each entrypoint, by entrypoint (novox/hq ADR 0193): what the node's runtime starts to serve it.
Launchers map[string]string
// Texts are, for the archive a module packs its templates into (TemplatesArtifact), each file's text as the
// build read it back from the packed archive, by its path inside it (novox/hq ADR 0349 §2). Never carried on
// the wire or kept: Resolve reads them into the files' content, which is what the mesh holds.
Texts map[string]string `json:"-"`
}
// Resolve fills a manifest's resources in from what was built.
@@ -75,26 +71,6 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
m.Module, strings.Join(missing, " and "), oneOrOther(len(missing)))
}
// **Each template's text, as the build read it back from its templates archive** (novox/hq ADR 0349 §2): into
// the file's content, so what the mesh holds and composes is what the same text inline would be.
if m.Templated() {
name := TemplatesArtifact(m)
texts := by[name].Texts
if name == "" || texts == nil {
return Manifest{}, fmt.Errorf("%s names templates, and the build carried no text read back from the "+
"archive it packs %s/ into (%q): the builder that built it does not read templates",
m.Module, TemplatesDir, name)
}
filled, err := fillTemplates(m.Module, m.Resources, func(template string) (string, bool) {
t, ok := texts[strings.TrimPrefix(template, TemplatesDir+"/")]
return t, ok
}, fmt.Sprintf("the archive %q", name))
if err != nil {
return Manifest{}, err
}
m.Resources = filled
}
out := m
out.Build = nil
out.Resources = nil
+1 -1
View File
@@ -173,7 +173,7 @@ func TestTheBackupHoldersLinesAreDerivedFromTheData(t *testing.T) {
{"id":"meta","path":"${dir:meta}","class":"rebuildable"}]},
"resources":[{"id":"meta","type":"directory","mode":"0700"}]}`)
facts := map[string]string{"account-home": "/home/op"}
with := Rendering{Settings: SettingsBy{"media": {LayerOf("node", map[string]any{AccessesSetting: map[string]any{"films": "/tank/films"}})}}}
with := Rendering{Settings: SettingsBy{"media": {{From: "node", Values: map[string]any{AccessesSetting: map[string]any{"films": "/tank/films"}}}}}}
backup, _, err := seatContributions([]Manifest{pg, agent, media}, Manifest{}, BackupSeat+":"+BackupKindBackup, with, facts, nil)
if err != nil {
t.Fatal(err)
-3
View File
@@ -1853,9 +1853,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
}
}
problems = append(problems, m.Build.problems(m.Module)...)
// A file's text in a template file of the module's repository (novox/hq ADR 0349 §2): only a file, never beside
// content, only under files/, and packed by the module's build.
problems = append(problems, templateProblems(m)...)
problems = append(problems, undeliveredBundles(m)...)
// **What provides the artifact store cannot be delivered through it** (novox/hq 04-ISSUES/029).
//
+2 -65
View File
@@ -34,69 +34,6 @@ const PlacesSetting = "places"
// AccessesSetting is the settings key that says where a module's accesses are on this node, by id.
const AccessesSetting = "accesses"
// **Placement is not config** (novox/hq ADR 0348 clause 11, issue 651). `places`, `accesses` and
// `networks` are facts of an assignment the controller decides, held beside the assignment and apart
// from its settings: a Layer carries them in Placement, never in Values, and only Places, AccessPlaces
// and KeptNetworks read them. A person still says them as keys of a layer through `settings`; LayerOf
// splits such a layer, so what they type is routed to the placement and nothing they do changes.
// placementKeys are the keys of a layer that are placement, not config.
var placementKeys = map[string]bool{PlacesSetting: true, AccessesSetting: true, NetworksSetting: true}
// SplitPlacement divides a layer as a person says it into its config values and its placement. Each is
// nil when it holds nothing; the values themselves are not copied.
func SplitPlacement(values map[string]any) (config, placement map[string]any) {
for key, value := range values {
if placementKeys[key] {
if placement == nil {
placement = map[string]any{}
}
placement[key] = value
continue
}
if config == nil {
config = map[string]any{}
}
config[key] = value
}
return config, placement
}
// LayerOf is a layer as a person says it — config values and placement in one map — split into the two.
func LayerOf(from string, values map[string]any) Layer {
config, placement := SplitPlacement(values)
if config == nil {
config = map[string]any{}
}
return Layer{From: from, Values: config, Placement: placement}
}
// Whole is the layer as a person says it: its values and its placement in one map, a new map.
func (l Layer) Whole() map[string]any {
out := make(map[string]any, len(l.Values)+len(l.Placement))
for k, v := range l.Values {
out[k] = v
}
for k, v := range l.Placement {
out[k] = v
}
return out
}
// asSaid is the layers with placement folded back into the values, for what reads a layer whole as it
// did while placement was kept in it: today's mergeable file, which takes every key (ADR 0326 §3), and
// the judgement that no value holds a line end.
func asSaid(layers []Layer) []Layer {
out := make([]Layer, len(layers))
for i, l := range layers {
out[i] = l
if len(l.Placement) > 0 {
out[i] = Layer{From: l.From, Values: l.Whole(), Default: l.Default}
}
}
return out
}
// Placement is what an assignment says about one of a module's directories.
type Placement struct {
// Path is where the directory is on this machine. Absolute.
@@ -271,7 +208,7 @@ func Places(m Manifest, layers []Layer) (map[string]Placement, error) {
}
out := map[string]Placement{}
for _, layer := range layers {
raw, ok := layer.Placement[PlacesSetting]
raw, ok := layer.Values[PlacesSetting]
if !ok {
continue
}
@@ -341,7 +278,7 @@ func AccessPlaces(m Manifest, layers []Layer) (map[string]string, error) {
}
out := map[string]string{}
for _, layer := range layers {
raw, ok := layer.Placement[AccessesSetting]
raw, ok := layer.Values[AccessesSetting]
if !ok {
continue
}
+1 -41
View File
@@ -23,7 +23,7 @@ func placeable() Manifest {
}
func placedBy(values map[string]any) Rendering {
return Rendering{Settings: SettingsBy{"arr": {LayerOf("node anchor", values)}}}
return Rendering{Settings: SettingsBy{"arr": {{From: "node anchor", Values: values}}}}
}
func TestAnAssignmentPlacesDirectoriesAndAccesses(t *testing.T) {
@@ -179,43 +179,3 @@ func TestAnUnknownAccessReferenceIsRefusedAtParse(t *testing.T) {
t.Fatalf("an access with no id and no path was accepted: %v", err)
}
}
// **Placement is read from the placement, never from a layer's values** (novox/hq ADR 0348 clause 11, issue
// 651). A `places` or `accesses` among the config values places nothing; the same in Placement does. A
// mergeable file of today's form still takes every key of the layer as it is said, placement too, so it
// composes as it did while placement was kept in the layer.
func TestPlacementIsReadFromThePlacementOnly(t *testing.T) {
m := placeable()
said := map[string]any{
PlacesSetting: map[string]any{"config": "/services/arr/config"},
AccessesSetting: map[string]any{"series": "/storage/media/series", "spool": "/storage/downloads"},
"log": "warn",
}
among := []Layer{{From: "anchor", Values: said}}
if placed, err := Places(m, among); err != nil || placed != nil {
t.Fatalf("a place among the config values placed %v (%v)", placed, err)
}
if _, _, err := accessesFor(m, among); err == nil {
t.Fatal("an access among the config values placed it")
}
beside := []Layer{LayerOf("anchor", said)}
if beside[0].Values["log"] != "warn" || beside[0].Values[PlacesSetting] != nil || beside[0].Placement[PlacesSetting] == nil {
t.Fatalf("LayerOf split %+v", beside[0])
}
if placed, err := Places(m, beside); err != nil || placed["config"].Path != "/services/arr/config" {
t.Fatalf("the placement did not place: %v (%v)", placed, err)
}
file := map[string]any{"id": "runtime", "type": "file", "path": "/x", "content": `{"a":1}`, "merge": "json"}
whole, err := ApplySettings(file, among)
if err != nil {
t.Fatal(err)
}
split, err := ApplySettings(file, beside)
if err != nil {
t.Fatal(err)
}
if whole["content"] != split["content"] || !strings.Contains(split["content"].(string), `"places"`) {
t.Fatalf("today's mergeable file composes %q, was %q", split["content"], whole["content"])
}
}
+4 -14
View File
@@ -107,7 +107,9 @@ var defaultSeats = append([]Seat{
// A pull request's merge check, judged (novox/hq to-be 45 §9).
"checked",
// A walk kept in a new state, for the delivery it walks (novox/hq ADR 0239).
"plan-moved"},
"plan-moved",
// A key of a record mirror written or removed (novox/hq ADR 0289, design 54 §3; design 57 F3).
"assignment-changed", "declared-config-changed"},
Serves: ControllerVerbs},
// The store's first verbs (novox/hq ADR 0159): the smallest set that makes the store askable,
// served by whichever module holds the seat with tools of these names.
@@ -128,19 +130,7 @@ var defaultSeats = append([]Seat{
// The vault: the controller seals every minted credential with what it provides, which is the
// test for a seat of the mesh's own (novox/hq ADR 0161) — a second provider of `secret` is a
// second claimant, refused by name, rather than a candidate for a pin.
//
// Its one verb, `get`, hands a secret on request and reply, sealed to the key the asker sends and only
// to an asker the vault allows (novox/hq ADR 0349 §11, issue 674). Optional until the vault that runs
// serves it (design 33 §7).
{Name: "mesh-vault", Scope: ScopeMesh, Delivers: "secret", Decision: "novox/hq ADR 0161",
Serves: []Verb{
{Name: "get", Optional: true, Description: "One secret, sealed to the public key the asker sends (the " +
"sealed box of ADR 0206), with its generation; answered only to an asker the vault allows, as the " +
"bus proved it. Never the value in the clear, and never to a private key.",
Input: schema(map[string]string{"name": "the secret's name",
"public_key": "the asker's X25519 public key as SPKI PEM — never a private key"},
[]string{"name", "public_key"})},
}},
{Name: "mesh-vault", Scope: ScopeMesh, Delivers: "secret", Decision: "novox/hq ADR 0161"},
// Named for its scope since 2026-09-30 (novox/hq ADR 0156); `the-artifact-store` resolves to it as
// an alias on a mesh that predates the rename. It serves artifacts of every kind a build makes —
// images and archives, by digest — which is why the provision is the artifact store and not an
+1 -2
View File
@@ -207,8 +207,7 @@ func Effective(m Manifest, layers []Layer) []SettingSource {
Default: d.Default, HasDefault: true}
}
for _, layer := range layers {
// As the layer is said, placement too: `settings` answers it as a person set it.
for key, v := range layer.Whole() {
for key, v := range layer.Values {
s, ok := byKey[key]
if !ok {
s = &SettingSource{Key: key}
+1 -1
View File
@@ -427,7 +427,7 @@ func TestALeftOutModulesUnplaceableBackupLineCostsOnlyThatLine(t *testing.T) {
r := anAdoptedAnchor()
r.Modules = append(r.Modules, holder, media, placedBadly)
with := anchorRendering(false)
with.Settings["notes"] = []Layer{LayerOf("anchor", map[string]any{PlacesSetting: "not a map"})}
with.Settings["notes"] = []Layer{{From: "anchor", Values: map[string]any{PlacesSetting: "not a map"}}}
composed, err := r.Compose(with)
if err != nil {
t.Fatalf("an unplaceable line of a left-out module failed the machine: %v", err)
+3 -13
View File
@@ -37,11 +37,6 @@ type Layer struct {
// Default marks the layer a module's own defaults make (novox/hq ADR 0262). Marked rather than
// recognised by From, which is a node's name for a node's layer, and a node may be called anything.
Default bool
// Placement is the assignment's placement at this layer's scope — `places`, `accesses`, `networks`
// — which the controller holds apart from settings (novox/hq ADR 0348 clause 11, issue 651): read
// by Places, AccessPlaces and KeptNetworks, and never from Values. LayerOf splits a layer as a person
// says it into the two.
Placement map[string]any
}
// ApplySettings produces a resource's final content from the module's own and the layers over it.
@@ -50,10 +45,6 @@ type Layer struct {
// module first, then this node's. A node that differs from the rest is expressed by differing,
// rather than by repeating everything the rest already say.
func ApplySettings(resource map[string]any, layers []Layer) (map[string]any, error) {
// Today's form reaches a mergeable file with every key of the layer as it was said, placement too
// (novox/hq ADR 0326 §3, ADR 0348 clause 11: a module not yet moved keeps it), so a file composes
// byte for byte as it did while placement was kept in the layer.
layers = asSaid(layers)
how, _ := resource["merge"].(string)
taken, names := takenKeys(resource)
if how == "" {
@@ -404,7 +395,7 @@ func UnusedSettings(m Manifest, layers []Layer) []string {
}
var unused []string
for _, layer := range layers {
for key := range layer.Whole() {
for key := range layer.Values {
if lands[key] {
continue
}
@@ -492,7 +483,7 @@ func KeptNetworks(m Manifest, layers []Layer, adopted bool) (map[string][]string
}
out := map[string][]string{}
for _, layer := range layers {
raw, ok := layer.Placement[NetworksSetting]
raw, ok := layer.Values[NetworksSetting]
if !ok {
continue
}
@@ -552,8 +543,7 @@ var networkName = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.-]*$`)
// refused where it is stored (SetSettings, with UnusedSettings) and said where a plan is read,
// and never costs a module its place.
func JudgeSettings(m Manifest, layers []Layer, adopted bool) error {
// Placement is judged for line ends as it always was: a path is written into files as well.
if err := settingsHoldOneLine(m.Module, asSaid(layers)); err != nil {
if err := settingsHoldOneLine(m.Module, layers); err != nil {
return err
}
// Which keys each mergeable file takes is the definition's to say right (novox/hq ADR 0326);
+7 -7
View File
@@ -27,7 +27,7 @@ func TestASettingThatCannotComposeIsRefusedByNameAndLeavesOnlyItsModuleOut(t *te
if c.name == "a mesh-wide port" {
from = MeshWideLayer
}
err := JudgeSettings(web, []Layer{LayerOf(from, c.layer)}, true)
err := JudgeSettings(web, []Layer{{From: from, Values: c.layer}}, true)
if err == nil || !strings.Contains(err.Error(), c.refuse) {
t.Errorf("%s: judged %v, want %q", c.name, err, c.refuse)
}
@@ -69,8 +69,8 @@ func TestASettingThatCannotComposeIsRefusedByNameAndLeavesOnlyItsModuleOut(t *te
// declaration as the networks it also joins.
func TestAKeptNetworkReachesTheContainerOnAnAdoptedMachineOnly(t *testing.T) {
r := anAdoptedAnchor()
keep := SettingsBy{"hello-web": {LayerOf("anchor",
map[string]any{NetworksSetting: map[string]any{"server": []any{"predecessor_default"}}})}}
keep := SettingsBy{"hello-web": {{From: "anchor",
Values: map[string]any{NetworksSetting: map[string]any{"server": []any{"predecessor_default"}}}}}}
with := anchorRendering(true)
with.Settings = keep
@@ -107,13 +107,13 @@ func TestAKeptNetworkReachesTheContainerOnAnAdoptedMachineOnly(t *testing.T) {
layer Layer
want string
}{
{"mesh-wide", LayerOf(MeshWideLayer, map[string]any{NetworksSetting: map[string]any{"server": []any{"x"}}}),
{"mesh-wide", Layer{From: MeshWideLayer, Values: map[string]any{NetworksSetting: map[string]any{"server": []any{"x"}}}},
"a found network is a fact about one machine"},
{"an unknown container", LayerOf("anchor", map[string]any{NetworksSetting: map[string]any{"db": []any{"x"}}}),
{"an unknown container", Layer{From: "anchor", Values: map[string]any{NetworksSetting: map[string]any{"db": []any{"x"}}}},
`names the container "db", which it does not declare`},
{"not a list", LayerOf("anchor", map[string]any{NetworksSetting: map[string]any{"server": "x"}}),
{"not a list", Layer{From: "anchor", Values: map[string]any{NetworksSetting: map[string]any{"server": "x"}}},
"is a list of network names"},
{"not a network name", LayerOf("anchor", map[string]any{NetworksSetting: map[string]any{"server": []any{"a/b"}}}),
{"not a network name", Layer{From: "anchor", Values: map[string]any{NetworksSetting: map[string]any{"server": []any{"a/b"}}}},
"which is not a network name"},
} {
_, err := KeptNetworks(web, []Layer{c.layer}, true)
+13
View File
@@ -124,6 +124,14 @@ func StateProblems(m Manifest) []string {
return problems
}
// ControllerMirrors are the controller's record mirrors a module may read, each `mesh-controller.<name>`
// (novox/hq ADR 0289, design 54 §5, design 57 F3). **The controller keeps them, not its manifest's `state`**:
// they are asserted with the controller's own buckets, with options of their own, and written by the
// controller's account alone (the writers table); declared as a module's state, they would be asserted a
// second way and granted to the module's own principal as well. Named here so a read of one is a read of
// kept state. The bus's list is broker.ControllerMirrors; a test keeps the two agreeing.
var ControllerMirrors = []string{"assignments", "declared-config"}
// StateReadsNothingDeclares is every read across a catalogue whose owner is present and declares no
// such state. An absent owner says nothing — a module may be installed long before the one whose
// state it reads, as a consumer may before its emitter (design 32 §1).
@@ -136,6 +144,11 @@ func StateReadsNothingDeclares(manifests []Manifest) []string {
}
declared[m.Module] = own
}
if own, present := declared[ControllerSeatName]; present {
for _, name := range ControllerMirrors {
own[name] = true
}
}
var problems []string
for _, m := range manifests {
for _, r := range m.Reads {
+15
View File
@@ -67,6 +67,21 @@ func TestAReadNamesStateItsOwnerKeeps(t *testing.T) {
}
}
// **The controller's record mirrors are kept state** (design 54 §5, design 57 F3): a module may read
// `mesh-controller.assignments` and `mesh-controller.declared-config` although the controller's manifest
// declares no `state`; a read of anything else of the controller's is still named.
func TestAReadOfTheControllersMirrorsIsAReadOfKeptState(t *testing.T) {
controller := Manifest{Module: ControllerSeatName}
reader := Manifest{Module: "mesh-config", Reads: []string{"mesh-controller.assignments", "mesh-controller.declared-config"}}
if p := StateReadsNothingDeclares([]Manifest{controller, reader}); len(p) != 0 {
t.Fatalf("a read of the controller's mirrors was refused: %v", p)
}
other := Manifest{Module: "other", Reads: []string{"mesh-controller.calls"}}
if p := StateReadsNothingDeclares([]Manifest{controller, other}); len(p) != 1 {
t.Fatalf("a read of the controller's own calls was not named: %v", p)
}
}
// **Across the whole catalogue**: every state name is local, and every read whose owner is present
// names state that owner keeps.
func TestEveryManifestsStateIsLocalAndEveryReadIsKept(t *testing.T) {
-334
View File
@@ -1,334 +0,0 @@
package catalogue
import (
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"path"
"path/filepath"
"sort"
"strings"
"unicode/utf8"
)
// A file's text kept as a template file in its module's repository (novox/hq ADR 0349 §2, design 57 package T1,
// issue 654).
//
// **Two forms, one composition.** A file resource says its text inline (`content`) or names a template file
// beside its manifest (`"template": "files/<name>"`), never both. The module's build packs `files/` into an
// archive artifact of its own — an `archive` artifact built `from: "files"` — and resolving the build reads each
// template's text back from that packed archive into `content` and drops `template`. So the manifest the mesh
// holds, composition and the node-engine are exactly as they were: a module with its text in `files/` composes
// byte-identically to the same module with it inline. Where there is no build — `module check`, the merge gate,
// `module add` — the text is read from the repository tree beside the manifest (WithTemplatesFrom).
//
// **Read back by the builder, carried in the resolved manifest.** The builder reads each template from the very
// bytes it hashed into the archive's digest, so the text the mesh holds is the archive's. The controller
// registers the resolved manifest as the build result carries it and fetches nothing from the artifact store; a
// manifest that still names a template when it would be held — built by a builder that predates templates — is
// refused at registration (HeldTemplateProblem).
// TemplateField is the key a file resource names its template file by.
const TemplateField = "template"
// TemplatesDir is the module's directory its template files are kept in, and the directory its templates archive
// is built from.
const TemplatesDir = "files"
// Templated is whether any of the manifest's resources names a template.
func (m Manifest) Templated() bool {
for _, r := range m.Resources {
if _, has := r[TemplateField]; has {
return true
}
}
return false
}
// TemplatesArtifact is the name of the archive artifact the module packs its templates into — the one built from
// files/ — or "" when it declares none, or more than one (both refused by templateProblems for a module naming a
// template).
func TemplatesArtifact(m Manifest) string {
found := templatesArtifacts(m)
if len(found) != 1 {
return ""
}
return found[0]
}
func templatesArtifacts(m Manifest) []string {
if m.Build == nil {
return nil
}
var found []string
for _, a := range m.Build.Artifacts {
if a.Kind == ArtifactArchive && a.URL == "" && cleanFrom(a.From) == TemplatesDir {
found = append(found, a.Name)
}
}
return found
}
// cleanFrom is an artifact's directory as one relative path: "./files/" is "files".
func cleanFrom(from string) string {
c := path.Clean(strings.TrimSpace(from))
return strings.TrimPrefix(c, "./")
}
// templateOf is the template a resource names, whether it names one, and what is wrong with it ("" when nothing).
func templateOf(module string, r map[string]any) (string, bool, string) {
raw, has := r[TemplateField]
if !has {
return "", false, ""
}
id := fmt.Sprint(r["id"])
if t := fmt.Sprint(r["type"]); t != "file" {
return "", true, fmt.Sprintf("%s: %s is a %s and names a template; only a file takes its text from one "+
"(novox/hq ADR 0349 §2)", module, id, t)
}
name, ok := raw.(string)
if !ok || strings.TrimSpace(name) == "" {
return "", true, fmt.Sprintf("%s: %s names a template that is not a path: %v; a template is %q",
module, id, raw, TemplatesDir+"/<name>")
}
if _, both := r["content"]; both {
return name, true, fmt.Sprintf("%s: %s names the template %q and says its content too; a file's text is "+
"in one place, the template or the content, never both (novox/hq ADR 0349 §2)", module, id, name)
}
if strings.HasPrefix(name, "/") || path.Clean(name) != name || !strings.HasPrefix(name, TemplatesDir+"/") ||
strings.Contains(name, "\\") {
return name, true, fmt.Sprintf("%s: %s names the template %q, which is not a file under %s/ of its own "+
"repository: a template is %q, a plain relative path, and the build packs only %s/",
module, id, name, TemplatesDir, TemplatesDir+"/<name>", TemplatesDir)
}
return name, true, ""
}
// templateProblems is what is wrong with the manifest's templates, as written: a template on something not a file,
// beside content, outside files/, or a module naming templates whose build packs no archive from files/. Whether
// each template exists is judged where a tree is (ReadTemplates) or the archive's text is (Resolve).
func templateProblems(m Manifest) []string {
var problems []string
templated := false
for _, r := range m.Resources {
_, has, wrong := templateOf(m.Module, r)
templated = templated || has
if wrong != "" {
problems = append(problems, wrong)
}
}
if !templated {
return problems
}
switch found := templatesArtifacts(m); len(found) {
case 0:
problems = append(problems, fmt.Sprintf("%s names templates and its build packs no archive from %s/: "+
"declare an artifact {\"name\": \"files\", \"kind\": \"archive\", \"from\": %q} (novox/hq ADR 0349 §2)",
m.Module, TemplatesDir, TemplatesDir))
case 1:
default:
sort.Strings(found)
problems = append(problems, fmt.Sprintf("%s packs %s/ into %s; its templates are read from one archive",
m.Module, TemplatesDir, strings.Join(found, " and ")))
}
return problems
}
// fillTemplates is resources with each template's text as its content and the template key dropped: text says the
// text of a template by its path ("files/<name>"), and whether it has one. A template it has no text for, or text
// that is not UTF-8, is refused by name. Every resource is copied; the manifest's own are left as they were.
func fillTemplates(module string, resources []map[string]any, text func(string) (string, bool),
where string) ([]map[string]any, error) {
out := make([]map[string]any, 0, len(resources))
var problems []string
for _, r := range resources {
name, has, wrong := templateOf(module, r)
if !has {
out = append(out, r)
continue
}
if wrong != "" {
problems = append(problems, wrong)
continue
}
body, ok := text(name)
if !ok {
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, and %s holds no such file",
module, r["id"], name, where))
continue
}
if !utf8.ValidString(body) {
// A manifest is JSON: text that is not UTF-8 would be changed on the way, silently.
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, which is not UTF-8 text; a "+
"file's text is carried as JSON and would not reach the machine as written", module, r["id"], name))
continue
}
filled := make(map[string]any, len(r))
for k, v := range r {
filled[k] = v
}
delete(filled, TemplateField)
filled["content"] = body
out = append(out, filled)
}
if len(problems) > 0 {
return nil, errors.New(strings.Join(problems, "; "))
}
return out, nil
}
// ReadTemplates is the text of each template the manifest names, read from the module's directory in a repository
// tree, by template path. Refused by name: a template the directory lacks, and anything under files/ the build's
// packing refuses — files/ itself, or any directory or file under it, that is a symlink or not a plain file — so
// nothing judged here can be read from outside the tree, and no tree passes here that the build would refuse. Each
// template is read through an os.Root of the module's directory besides, which no path element can leave.
func ReadTemplates(m Manifest, dir string) (map[string]string, error) {
problems := templatesTreeProblems(m.Module, dir)
root, err := os.OpenRoot(dir)
if err != nil {
return nil, fmt.Errorf("%s: the module's directory cannot be opened to read its templates: %w", m.Module, err)
}
defer root.Close()
texts := map[string]string{}
for _, r := range m.Resources {
name, has, wrong := templateOf(m.Module, r)
if !has {
continue
}
if wrong != "" {
problems = append(problems, wrong)
continue
}
if _, read := texts[name]; read {
continue
}
info, err := root.Lstat(name)
switch {
case errors.Is(err, fs.ErrNotExist):
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, and the module's directory "+
"has no such file", m.Module, r["id"], name))
continue
case err != nil:
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, which cannot be read: %v",
m.Module, r["id"], name, err))
continue
case !info.Mode().IsRegular():
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, which is not a plain file; "+
"the build packs only files", m.Module, r["id"], name))
continue
}
body, err := root.ReadFile(name)
if err != nil {
problems = append(problems, fmt.Sprintf("%s: %v names the template %q, which cannot be read: %v",
m.Module, r["id"], name, err))
continue
}
texts[name] = string(body)
}
if len(problems) > 0 {
return nil, errors.New(strings.Join(problems, "; "))
}
return texts, nil
}
// templatesTreeProblems is everything under the module's files/ that the build's packing refuses: files/ itself not
// a plain directory, and any entry under it that is neither a directory nor a plain file — a symlink, to a file or a
// directory, a device, a pipe. Each said by its path. A files/ that does not exist is no problem here: each template
// it would have held is refused as missing.
func templatesTreeProblems(module, dir string) []string {
top := filepath.Join(dir, TemplatesDir)
info, err := os.Lstat(top)
if errors.Is(err, fs.ErrNotExist) {
return nil
}
if err != nil {
return []string{fmt.Sprintf("%s: %s/ cannot be read: %v", module, TemplatesDir, err)}
}
if !info.IsDir() {
return []string{fmt.Sprintf("%s: %s/ is %s, not a directory of the module's own; its templates are read "+
"only from inside the repository, and the build packs only directories and files", module, TemplatesDir,
kindOf(info.Mode()))}
}
var problems []string
err = filepath.WalkDir(top, func(at string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() || d.Type().IsRegular() {
return nil
}
rel, _ := filepath.Rel(dir, at)
problems = append(problems, fmt.Sprintf("%s: %s is %s; the build packs only directories and files, so "+
"%s/ holds nothing else", module, filepath.ToSlash(rel), kindOf(d.Type()), TemplatesDir))
return nil
})
if err != nil {
problems = append(problems, fmt.Sprintf("%s: %s/ cannot be read: %v", module, TemplatesDir, err))
}
return problems
}
// kindOf says what a file that is not a plain file is.
func kindOf(mode fs.FileMode) string {
switch {
case mode&fs.ModeSymlink != 0:
return "a symlink"
case mode&fs.ModeDevice != 0:
return "a device"
case mode&fs.ModeNamedPipe != 0:
return "a named pipe"
case mode&fs.ModeSocket != 0:
return "a socket"
case mode.IsDir():
return "a directory"
}
return "not a plain file"
}
// WithTemplatesFrom is the manifest with each template's text read from the module's directory in a repository
// tree as its content: what `module check`, the merge gate and `module add` judge, where nothing was built. Read
// again strictly afterwards, so every rule about a file's text judges the template's text exactly as it judges
// inline content. A manifest naming no template is returned as it is.
func WithTemplatesFrom(m Manifest, dir string) (Manifest, error) {
if !m.Templated() {
return m, nil
}
texts, err := ReadTemplates(m, dir)
if err != nil {
return Manifest{}, err
}
filled, err := fillTemplates(m.Module, m.Resources, func(name string) (string, bool) {
t, ok := texts[name]
return t, ok
}, "the module's directory")
if err != nil {
return Manifest{}, err
}
out := m
out.Resources = filled
raw, err := json.Marshal(out)
if err != nil {
return Manifest{}, err
}
return ParseManifest(raw)
}
// HeldTemplateProblem is why a manifest cannot be held by the mesh because it still names a template, or "": the
// manifest the mesh holds carries every file's text, which a build reads back from its templates archive. One
// naming a template was resolved by a builder that does not read templates, and would compose files with no text.
func HeldTemplateProblem(m Manifest) string {
var named []string
for _, r := range m.Resources {
if t, has := r[TemplateField]; has {
named = append(named, fmt.Sprintf("%v (%v)", r["id"], t))
}
}
if len(named) == 0 {
return ""
}
return fmt.Sprintf("%s still names the template(s) of %s: the manifest the mesh holds carries each file's "+
"text, read back from the build's templates archive — this one was resolved by a builder that does not "+
"read templates, or not built at all (novox/hq ADR 0349 §2)", m.Module, strings.Join(named, ", "))
}
-323
View File
@@ -1,323 +0,0 @@
package catalogue
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
)
// A file resource may name a template file in its module's repository (novox/hq ADR 0349 §2, design 57 package
// T1, issue 654): the build packs files/ into an archive, and resolving the build reads each template's text back
// into content, so the mesh holds, composes and sends exactly what it would for the same text inline.
// The two forms of one module: its text inline, and its text in files/.
const (
inlineTwin = `{"module":"tidy","version":"1",
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
"resources":[
{"id":"conf","type":"directory","path":"${dir:conf}"},
{"id":"main","type":"file","path":"${dir:conf}/tidy.conf","mode":"0644","content":"` + mainText + `"},
{"id":"extra","type":"file","path":"${dir:conf}/extra.conf","content":"` + extraText + `"}]}`
templatedTwin = `{"module":"tidy","version":"1",
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
"resources":[
{"id":"conf","type":"directory","path":"${dir:conf}"},
{"id":"main","type":"file","path":"${dir:conf}/tidy.conf","mode":"0644","template":"files/tidy.conf"},
{"id":"extra","type":"file","path":"${dir:conf}/extra.conf","template":"files/deeper/extra.conf"}]}`
mainText = `# tidy\nroot = ${dir:conf}\nquoted = \"yes\" é\n`
extraText = `one\ntwo\n`
)
// templateTexts is what a build reads back from the packed files/ archive: by path inside it.
func templateTexts(t *testing.T) map[string]string {
t.Helper()
var m struct{ Main, Extra string }
if err := json.Unmarshal([]byte(`{"Main":"`+mainText+`","Extra":"`+extraText+`"}`), &m); err != nil {
t.Fatal(err)
}
return map[string]string{"tidy.conf": m.Main, "deeper/extra.conf": m.Extra, "unused.conf": "not named"}
}
func builtFiles(texts map[string]string) []Built {
return []Built{{Name: "files", Kind: ArtifactArchive,
Reference: ArtifactStoreScheme + "tidy/files/blobs/sha256:" + strings.Repeat("c", 64),
Digest: "sha256:" + strings.Repeat("c", 64), Texts: texts}}
}
// held is a manifest as registration holds it: resolved with what was built, then read again strictly.
func held(t *testing.T, raw string, built []Built) Manifest {
t.Helper()
m, err := ParseManifest([]byte(raw))
if err != nil {
t.Fatal(err)
}
resolved, err := m.Resolve(built)
if err != nil {
t.Fatal(err)
}
again, err := json.Marshal(resolved)
if err != nil {
t.Fatal(err)
}
out, err := ParseManifest(again)
if err != nil {
t.Fatal(err)
}
if problem := HeldTemplateProblem(out); problem != "" {
t.Fatal(problem)
}
return out
}
func composed(t *testing.T, m Manifest) string {
t.Helper()
got, err := Resolve(shelf(m), []string{m.Module}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
out, err := json.Marshal(mustDeclare(t, got))
if err != nil {
t.Fatal(err)
}
return string(out)
}
func TestATemplatedModuleComposesByteIdenticallyToItsInlineTwin(t *testing.T) {
inline := composed(t, held(t, inlineTwin, builtFiles(nil)))
templated := composed(t, held(t, templatedTwin, builtFiles(templateTexts(t))))
if inline != templated {
t.Fatalf("the two forms compose differently:\ninline: %s\ntemplated: %s", inline, templated)
}
if !strings.Contains(templated, "quoted = \\\"yes\\\"") || strings.Contains(templated, `"template"`) {
t.Fatalf("the composition does not carry the template's text as content: %s", templated)
}
}
// What `module check`, the merge gate and `module add` judge: the text read from the tree beside the manifest.
func TestATemplatedModuleReadFromItsTreeComposesByteIdenticallyToItsInlineTwin(t *testing.T) {
dir := t.TempDir()
for name, text := range templateTexts(t) {
full := filepath.Join(dir, "files", filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte(text), 0o644); err != nil {
t.Fatal(err)
}
}
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
filled, err := WithTemplatesFrom(m, dir)
if err != nil {
t.Fatal(err)
}
if filled.Templated() {
t.Fatal("a manifest read from its tree still names a template")
}
resolved, err := filled.Resolve(builtFiles(nil))
if err != nil {
t.Fatal(err)
}
if got, want := composed(t, resolved), composed(t, held(t, inlineTwin, builtFiles(nil))); got != want {
t.Fatalf("read from the tree, the module composes differently:\ninline: %s\ntemplated: %s", want, got)
}
}
func TestAFileNamingBothATemplateAndContentIsRefused(t *testing.T) {
raw := strings.Replace(templatedTwin, `"template":"files/tidy.conf"`,
`"template":"files/tidy.conf","content":"x"`, 1)
_, err := ParseManifest([]byte(raw))
if err == nil || !strings.Contains(err.Error(), "main") || !strings.Contains(err.Error(), "never both") {
t.Fatalf("a file with both a template and content was not refused by name: %v", err)
}
}
func TestATemplateOutsideFilesIsRefusedByName(t *testing.T) {
for _, bad := range []string{"../tidy.conf", "/etc/tidy.conf", "files", "files/", "files/../tidy.conf",
"files/./tidy.conf", "other/tidy.conf", "tidy.conf", "files//tidy.conf", `files\\tidy.conf`} {
raw := strings.Replace(templatedTwin, `"files/tidy.conf"`, `"`+bad+`"`, 1)
_, err := ParseManifest([]byte(raw))
if err == nil || !strings.Contains(err.Error(), "main") || !strings.Contains(err.Error(), "not a file under files/") {
t.Errorf("the template %q was not refused by name: %v", bad, err)
}
}
}
func TestATemplateOnAResourceThatIsNotAFileIsRefused(t *testing.T) {
raw := strings.Replace(templatedTwin, `{"id":"conf","type":"directory","path":"${dir:conf}"}`,
`{"id":"conf","type":"directory","path":"${dir:conf}","template":"files/tidy.conf"}`, 1)
_, err := ParseManifest([]byte(raw))
if err == nil || !strings.Contains(err.Error(), "conf is a directory and names a template") {
t.Fatalf("a directory naming a template was not refused: %v", err)
}
}
func TestATemplatedModuleWhoseBuildPacksNoFilesIsRefused(t *testing.T) {
for _, build := range []string{
``,
`"build":{"artifacts":[{"name":"look","kind":"archive","from":"theme"}]},`,
`"build":{"artifacts":[{"name":"a","kind":"archive","from":"files"},{"name":"b","kind":"archive","from":"./files/"}]},`,
} {
raw := strings.Replace(templatedTwin,
`"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},`, build, 1)
_, err := ParseManifest([]byte(raw))
if err == nil || !strings.Contains(err.Error(), "files/") {
t.Errorf("a templated module with the build %q was not refused: %v", build, err)
}
}
}
func TestAMissingTemplateIsRefusedByName(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
// In the tree beside the manifest.
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "files"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "files", "tidy.conf"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
_, err = WithTemplatesFrom(m, dir)
if err == nil || !strings.Contains(err.Error(), "extra") || !strings.Contains(err.Error(), "files/deeper/extra.conf") {
t.Fatalf("a template missing from the tree was not refused by name: %v", err)
}
// And in what the build read back from its archive.
texts := templateTexts(t)
delete(texts, "deeper/extra.conf")
_, err = m.Resolve(builtFiles(texts))
if err == nil || !strings.Contains(err.Error(), "extra") || !strings.Contains(err.Error(), "files/deeper/extra.conf") {
t.Fatalf("a template missing from the archive was not refused by name: %v", err)
}
}
func TestATemplateThatIsASymlinkIsRefused(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "files", "deeper"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "elsewhere"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "files", "tidy.conf"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(filepath.Join(dir, "elsewhere"), filepath.Join(dir, "files", "deeper", "extra.conf")); err != nil {
t.Fatal(err)
}
if _, err := WithTemplatesFrom(m, dir); err == nil || !strings.Contains(err.Error(), "not a plain file") {
t.Fatalf("a template that is a symlink was not refused: %v", err)
}
}
func TestAResolveWithoutTheArchivesTextIsRefused(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
if _, err := m.Resolve(builtFiles(nil)); err == nil || !strings.Contains(err.Error(), "files") {
t.Fatalf("a templated module resolved with no text read back from its archive: %v", err)
}
}
func TestATemplateThatIsNotUTF8IsRefused(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
texts := templateTexts(t)
texts["tidy.conf"] = "\xff\xfe"
if _, err := m.Resolve(builtFiles(texts)); err == nil || !strings.Contains(err.Error(), "UTF-8") {
t.Fatalf("a template that is not UTF-8 was resolved: %v", err)
}
}
// What registration refuses: a manifest to be held that still names a template — one resolved by a builder that
// predates templates would otherwise compose files with no text.
func TestAManifestStillNamingATemplateIsNotHeld(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
if problem := HeldTemplateProblem(m); !strings.Contains(problem, "main") || !strings.Contains(problem, "files/tidy.conf") {
t.Fatalf("a manifest naming a template would be held: %q", problem)
}
if problem := HeldTemplateProblem(held(t, inlineTwin, builtFiles(nil))); problem != "" {
t.Fatalf("an inline manifest is refused: %s", problem)
}
}
// A symlinked files/, or a symlinked directory under it, would read a file outside the tree: refused by name,
// as is any symlink or special file under files/ that no template names, which the build's packing refuses.
func TestASymlinkAnywhereUnderFilesIsRefusedByName(t *testing.T) {
m, err := ParseManifest([]byte(templatedTwin))
if err != nil {
t.Fatal(err)
}
outside := t.TempDir()
if err := os.MkdirAll(filepath.Join(outside, "deeper"), 0o755); err != nil {
t.Fatal(err)
}
for _, name := range []string{"tidy.conf", "deeper/extra.conf", "secret"} {
if err := os.WriteFile(filepath.Join(outside, filepath.FromSlash(name)), []byte("outside"), 0o644); err != nil {
t.Fatal(err)
}
}
write := func(t *testing.T, dir, name string) {
t.Helper()
full := filepath.Join(dir, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte("inside"), 0o644); err != nil {
t.Fatal(err)
}
}
link := func(t *testing.T, target, at string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(at), 0o755); err != nil {
t.Fatal(err)
}
if err := os.Symlink(target, at); err != nil {
t.Fatal(err)
}
}
for _, c := range []struct {
name string
make func(t *testing.T, dir string)
names string
}{
{"files/ itself", func(t *testing.T, dir string) {
link(t, outside, filepath.Join(dir, "files"))
}, "files/ is a symlink"},
{"a directory under files/", func(t *testing.T, dir string) {
write(t, dir, "files/tidy.conf")
link(t, filepath.Join(outside, "deeper"), filepath.Join(dir, "files", "deeper"))
}, "files/deeper is a symlink"},
{"a file no template names", func(t *testing.T, dir string) {
write(t, dir, "files/tidy.conf")
write(t, dir, "files/deeper/extra.conf")
link(t, filepath.Join(outside, "secret"), filepath.Join(dir, "files", "unnamed"))
}, "files/unnamed is a symlink"},
} {
dir := t.TempDir()
c.make(t, dir)
_, err := WithTemplatesFrom(m, dir)
if err == nil || !strings.Contains(err.Error(), c.names) {
t.Errorf("a symlinked %s was not refused by name: %v", c.name, err)
}
if err != nil && strings.Contains(err.Error(), "outside") {
t.Errorf("a symlinked %s was read: %v", c.name, err)
}
}
}
+7 -7
View File
@@ -13,22 +13,22 @@ func TestAPlacementOrAnAccessAtTheMachinesOwnIsRefused(t *testing.T) {
for _, path := range []string{"/", "/etc", "/etc/", "/etc/sudoers.d", "/usr/bin", "/root", "/var", "/var/lib",
"/home", "/home/", "/run", "/run/user/1000", "/var/lib/mesh/x", "/var/lib/mesh-host", "/var/lib/mesh-host/identity",
"/srv/../etc", "//etc", "/proc/1", "/sys", "/dev", "/boot/efi", "/bin", "/sbin", "/lib", "/lib64"} {
layers := []Layer{LayerOf("laptop", map[string]any{PlacesSetting: map[string]any{"data": path}})}
layers := []Layer{{From: "laptop", Values: map[string]any{PlacesSetting: map[string]any{"data": path}}}}
if _, err := Places(m, layers); err == nil || !strings.Contains(err.Error(), "issue 339") {
t.Errorf("a place at %s: %v", path, err)
}
layers = []Layer{LayerOf("laptop", map[string]any{AccessesSetting: map[string]any{"media": path}})}
layers = []Layer{{From: "laptop", Values: map[string]any{AccessesSetting: map[string]any{"media": path}}}}
if _, err := AccessPlaces(m, layers); err == nil || !strings.Contains(err.Error(), "issue 339") {
t.Errorf("an access at %s: %v", path, err)
}
}
for _, path := range []string{"/srv/notes", "/mnt/plex/data", "/storage/media", "/services/media/movies",
"/var/lib/notes/data", "/home/restic/repo", "/var/lib/mesh-store"} {
layers := []Layer{LayerOf("laptop", map[string]any{PlacesSetting: map[string]any{"data": path}})}
layers := []Layer{{From: "laptop", Values: map[string]any{PlacesSetting: map[string]any{"data": path}}}}
if got, err := Places(m, layers); err != nil || got["data"].Path != path {
t.Errorf("a place at %s: %v %v", path, got, err)
}
layers = []Layer{LayerOf("laptop", map[string]any{AccessesSetting: map[string]any{"media": path}})}
layers = []Layer{{From: "laptop", Values: map[string]any{AccessesSetting: map[string]any{"media": path}}}}
if got, err := AccessPlaces(m, layers); err != nil || got["media"] != path {
t.Errorf("an access at %s: %v %v", path, got, err)
}
@@ -111,17 +111,17 @@ func TestTheRuntimesDataAndAnyHomesSSHAreTheMachinesOwn(t *testing.T) {
for _, path := range []string{"/var/spool", "/var/spool/cron", "/var/lib/docker", "/var/lib/docker/volumes",
"/var/lib/containers/storage", "/var/lib/containerd", "/var/lib/containerd/io.containerd.snapshotter.v1", "/opt", "/opt/app", "/home/alice/.ssh", "/home/alice/.ssh/keys",
"/srv/backup/.ssh", "/root/.ssh"} {
layers := []Layer{LayerOf("laptop", map[string]any{PlacesSetting: map[string]any{"data": path}})}
layers := []Layer{{From: "laptop", Values: map[string]any{PlacesSetting: map[string]any{"data": path}}}}
if _, err := Places(m, layers); err == nil || !strings.Contains(err.Error(), "issue 339") {
t.Errorf("a place at %s: %v", path, err)
}
layers = []Layer{LayerOf("laptop", map[string]any{AccessesSetting: map[string]any{"media": path}})}
layers = []Layer{{From: "laptop", Values: map[string]any{AccessesSetting: map[string]any{"media": path}}}}
if _, err := AccessPlaces(m, layers); err == nil || !strings.Contains(err.Error(), "issue 339") {
t.Errorf("an access at %s: %v", path, err)
}
}
for _, path := range []string{"/var/lib/dockerish", "/home/alice/ssh", "/home/alice/.sshd-notes", "/storage/media"} {
layers := []Layer{LayerOf("laptop", map[string]any{PlacesSetting: map[string]any{"data": path}})}
layers := []Layer{{From: "laptop", Values: map[string]any{PlacesSetting: map[string]any{"data": path}}}}
if _, err := Places(m, layers); err != nil {
t.Errorf("a place at %s: %v", path, err)
}
-31
View File
@@ -23,34 +23,3 @@ func TestTheVaultsSeatDeliversSecret(t *testing.T) {
t.Fatal("a provider of secret that does not claim the seat was allowed to hold it")
}
}
// The vault answers `get` on request and reply (novox/hq ADR 0349 §11, issue 674): the seat promises the
// verb, so a holder serving it is accepted — and, optional while the vault that runs is the TypeScript one,
// so is a holder that does not yet.
func TestTheVaultsSeatPromisesGet(t *testing.T) {
seat, _ := SeatNamed("mesh-vault")
var get *Verb
for i := range seat.Serves {
if seat.Serves[i].Name == "get" {
get = &seat.Serves[i]
}
}
if get == nil {
t.Fatal("the vault's seat does not promise get")
}
for _, arg := range []string{"name", "public_key"} {
if props, _ := get.Input["properties"].(map[string]any); props[arg] == nil {
t.Errorf("get takes no %s", arg)
}
}
serving := Manifest{Module: "mesh-vault", Provides: []Offer{{Name: "secret", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-vault", Scope: ScopeMesh, Serves: []string{"get"}}}}
if err := CanHold(serving, seat); err != nil {
t.Fatalf("the vault serving get was refused: %v", err)
}
notYet := Manifest{Module: "mesh-vault", Provides: []Offer{{Name: "secret", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-vault", Scope: ScopeMesh}}}
if err := CanHold(notYet, seat); err != nil {
t.Fatalf("the vault that does not serve get yet was refused: %v", err)
}
}
+19 -83
View File
@@ -117,11 +117,6 @@ func (i *Inventory) RegisterModule(ctx context.Context, m catalogue.Manifest, fr
}
}
}
// **The manifest the mesh holds carries each file's text** (novox/hq ADR 0349 §2): one still naming a template
// would compose a file with no text, so it is refused by name — a build by a builder that predates templates.
if why := catalogue.HeldTemplateProblem(m); why != "" {
return fmt.Errorf("%s is not registered: %s", m.Module, why)
}
raw, err := json.Marshal(m)
if err != nil {
return err
@@ -844,18 +839,7 @@ func (i *Inventory) KeepSettings(ctx context.Context, nodeName, module string, v
}
func (i *Inventory) setSettings(ctx context.Context, nodeName, module string, values map[string]any, judge bool, setBy string) error {
// **Placement is not config** (novox/hq ADR 0348 clause 11, issue 651): the layer is said whole, as
// it always was, and its placement keys are kept beside the assignment, never in the settings row.
// Both are written in one transaction, and the layer a set replaces is the whole of both.
config, placement := catalogue.SplitPlacement(values)
if config == nil {
config = map[string]any{}
}
raw, err := json.Marshal(config)
if err != nil {
return err
}
placed, err := placementJSON(placement)
raw, err := json.Marshal(values)
if err != nil {
return err
}
@@ -897,9 +881,6 @@ func (i *Inventory) setSettings(ctx context.Context, nodeName, module string, va
do update set values = excluded.values, set_at = now(), set_by = excluded.set_by`, module, raw, nullable(setBy)); err != nil {
return wrapModule(err, module)
}
if err := keepPlacement(ctx, tx, nil, module, placed, setBy); err != nil {
return err
}
return tx.Commit(ctx)
}
node, err := i.NodeByName(ctx, nodeName)
@@ -931,9 +912,6 @@ func (i *Inventory) setSettings(ctx context.Context, nodeName, module string, va
if err != nil {
return wrapModule(err, module)
}
if err := keepPlacement(ctx, tx, &node.ID, module, placed, setBy); err != nil {
return err
}
// A given port replaces what the mesh assigned for that port: the assignment is given back,
// so the number is free for the next module rather than held for ever in the name of a port
// that now lives elsewhere.
@@ -964,15 +942,21 @@ func (i *Inventory) judgeSettings(ctx context.Context, nodeName, module string,
return err
}
where, from, adopted = nodeName, nodeName, node.Adopted
under, has, err := i.Layer(ctx, "", module)
if err != nil {
var meshWide []byte
err = i.store.Pool().QueryRow(ctx,
`select values from settings where module = $1 and node is null`, module).Scan(&meshWide)
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
return err
}
if has {
layers = append(layers, catalogue.LayerOf(catalogue.MeshWideLayer, under))
if len(meshWide) > 0 {
var under map[string]any
if err := json.Unmarshal(meshWide, &under); err != nil {
return err
}
layers = append(layers, catalogue.Layer{From: catalogue.MeshWideLayer, Values: under})
}
}
layers = append(layers, catalogue.LayerOf(from, values))
layers = append(layers, catalogue.Layer{From: from, Values: values})
if err := catalogue.JudgeSettings(m, layers, adopted); err != nil {
return fmt.Errorf("refused: %s on %s cannot compose with the layer %q — %w", module, where, from, err)
}
@@ -1105,8 +1089,7 @@ func refuseGivenCollisions(ctx context.Context, tx pgx.Tx, nodeID any, node, mod
}
func wrapModule(err error, module string) error {
if err != nil && (strings.Contains(err.Error(), "settings_module_fkey") ||
strings.Contains(err.Error(), "placement_module_fkey")) {
if err != nil && strings.Contains(err.Error(), "settings_module_fkey") {
return fmt.Errorf("%w: %s", ErrNoSuchModule, module)
}
return err
@@ -1147,11 +1130,6 @@ func (i *Inventory) ClearSettingsBy(ctx context.Context, nodeName, module, clear
`delete from settings where module = $1 and node is not distinct from $2::uuid`, module, nodeID); err != nil {
return err
}
// And its placement, which is the same layer as a person says it (novox/hq ADR 0348 clause 11).
if _, err := tx.Exec(ctx,
`delete from placement where module = $1 and node is not distinct from $2::uuid`, module, nodeID); err != nil {
return err
}
return tx.Commit(ctx)
}
@@ -1164,13 +1142,10 @@ func (i *Inventory) SettingsFor(ctx context.Context, nodeName, module string) ([
if err != nil {
return nil, err
}
// Each layer's placement beside it, from the placement the controller holds (novox/hq ADR 0348
// clause 11): a layer's Values hold its config values only, its Placement where things are.
rows, err := i.store.Pool().Query(ctx,
`select s.node is null, s.values, p.values from settings s
left join placement p on p.module = s.module and p.node is not distinct from s.node
where s.module = $1 and (s.node is null or s.node = $2)
order by s.node is null desc`, module, node.ID)
`select node is null, values from settings
where module = $1 and (node is null or node = $2)
order by node is null desc`, module, node.ID)
if err != nil {
return nil, err
}
@@ -1179,62 +1154,23 @@ func (i *Inventory) SettingsFor(ctx context.Context, nodeName, module string) ([
var layers []catalogue.Layer
for rows.Next() {
var meshWide bool
var raw, placed []byte
if err := rows.Scan(&meshWide, &raw, &placed); err != nil {
var raw []byte
if err := rows.Scan(&meshWide, &raw); err != nil {
return nil, err
}
values := map[string]any{}
if err := json.Unmarshal(raw, &values); err != nil {
return nil, err
}
var placement map[string]any
if len(placed) > 0 {
if err := json.Unmarshal(placed, &placement); err != nil {
return nil, err
}
}
from := nodeName
if meshWide {
from = catalogue.MeshWideLayer
}
layers = append(layers, catalogue.Layer{From: from, Values: values, Placement: placement})
layers = append(layers, catalogue.Layer{From: from, Values: values})
}
return layers, rows.Err()
}
// placementJSON is a layer's placement as the placement table keeps it, or nil when it has none.
func placementJSON(placement map[string]any) ([]byte, error) {
if len(placement) == 0 {
return nil, nil
}
return json.Marshal(placement)
}
// keepPlacement writes one layer's placement beside its settings row, in the write's transaction: the
// placement the layer says, or none when it says none — a set replaces the layer whole, placement too.
func keepPlacement(ctx context.Context, tx pgx.Tx, nodeID *string, module string, placed []byte, setBy string) error {
if placed == nil {
_, err := tx.Exec(ctx,
`delete from placement where module = $1 and node is not distinct from $2::uuid`, module, nodeID)
return err
}
var err error
if nodeID == nil {
_, err = tx.Exec(ctx,
`insert into placement (node, module, values, set_by) values (null, $1, $2, $3)
on conflict (module) where node is null
do update set values = excluded.values, set_at = now(), set_by = excluded.set_by`,
module, placed, nullable(setBy))
} else {
_, err = tx.Exec(ctx,
`insert into placement (node, module, values, set_by) values ($1, $2, $3, $4)
on conflict (node, module) where node is not null
do update set values = excluded.values, set_at = now(), set_by = excluded.set_by`,
*nodeID, module, placed, nullable(setBy))
}
return wrapModule(err, module)
}
// PinProvision records which provider a machine gets a provision from: a module, and the node it
// runs on — both, always (novox/hq #258). A provision comes from a module, and the same module on
// two machines is two answers, so neither half alone says which.
-92
View File
@@ -2,7 +2,6 @@ package inventory
import (
"context"
"encoding/json"
"fmt"
"os"
"strings"
@@ -76,94 +75,3 @@ func (i *Inventory) RemoveNodeForTest(ctx context.Context, name string) (bool, e
}
return tag.RowsAffected() == 1, nil
}
// ForTestBefore is ForTest with the store migrated only below migration `number`, and a function that runs the
// rest: for a test of a migration that moves data, which seeds the store as the mesh holds it before the
// migration, migrates, and reads what the migration made of it (novox/hq issue 651).
func ForTestBefore(t *testing.T, number int) (*Inventory, func()) {
t.Helper()
admin := os.Getenv("MESH_TEST_POSTGRES")
if admin == "" {
t.Skip("no MESH_TEST_POSTGRES; run `make check` to raise one")
}
name := fmt.Sprintf("invb_%d_%s", time.Now().UnixNano()%1_000_000,
strings.ToLower(strings.NewReplacer("/", "", "-", "").Replace(t.Name())))
if len(name) > 60 {
name = name[:60]
}
all, err := Migrations()
if err != nil {
t.Fatal(err)
}
var before []store.Migration
for _, m := range all {
if m.Number < number {
before = append(before, m)
}
}
store.CreateMigratedForTest(t, admin, name, before)
busPasswordCostForTest.Store(int32(bcrypt.MinCost))
cut := strings.LastIndex(admin, "/")
t.Setenv(store.Variable(Name), admin[:cut]+"/"+name+"?sslmode=disable")
inv, err := Open(t.Context())
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
inv.Close()
c, err := pgx.Connect(context.Background(), admin)
if err != nil {
return
}
defer c.Close(context.Background())
_, _ = c.Exec(context.Background(), "drop database if exists "+name+" with (force)")
})
if err := inv.Ready(t.Context(), 20*time.Second); err != nil {
t.Fatal(err)
}
return inv, func() {
t.Helper()
if _, err := inv.store.Migrate(t.Context(), all); err != nil {
t.Fatal(err)
}
}
}
// ExecForTest runs one statement against a test's store, for a fixture no verb of the mesh writes: a row as an
// older migration left it.
func (i *Inventory) ExecForTest(t *testing.T, sql string, args ...any) {
t.Helper()
if _, err := i.store.Pool().Exec(t.Context(), sql, args...); err != nil {
t.Fatalf("%s: %v", sql, err)
}
}
// QueryForTest reads rows of a test's store, each row's columns as JSON text, for a test that checks what a
// migration left.
func (i *Inventory) QueryForTest(t *testing.T, sql string, args ...any) [][]string {
t.Helper()
rows, err := i.store.Pool().Query(t.Context(), sql, args...)
if err != nil {
t.Fatalf("%s: %v", sql, err)
}
defer rows.Close()
var out [][]string
for rows.Next() {
values, err := rows.Values()
if err != nil {
t.Fatal(err)
}
row := make([]string, len(values))
for k, v := range values {
row[k] = fmt.Sprint(v)
if b, err := json.Marshal(v); err == nil {
row[k] = string(b)
}
}
out = append(out, row)
}
if err := rows.Err(); err != nil {
t.Fatal(err)
}
return out
}
@@ -1,46 +0,0 @@
-- Placement is not config (novox/hq ADR 0348 clause 11, issue 651).
--
-- Where a module's directories are on a machine (`places`), where the operator's data it reads is
-- (`accesses`) and which found networks a taken container keeps (`networks`) were kept as keys of a
-- settings layer. They are facts of an assignment that the controller decides, never config values, and
-- the config values are leaving for the mesh-config module (ADR 0348 clause 9) while placement stays with
-- the controller. So they get a table of their own beside the assignment, and every settings layer's
-- placement keys move into it, unchanged.
--
-- Scoped exactly as the layer they came from: one machine's, or the whole mesh's (node null) for a
-- module's every assignment, which is how a mesh-wide `places` or `accesses` layer composes today. The
-- composer reads placement from here only; the `settings` verb still takes the three keys and routes
-- them here, so nothing a person types changes.
create table placement (
node uuid references node(id) on delete cascade,
module text not null references module(name) on delete cascade,
-- Only `places`, `accesses` and `networks`, as the settings layer held them.
values jsonb not null,
set_at timestamptz not null default now(),
set_by text
);
create unique index placement_for_the_mesh on placement (module) where node is null;
create unique index placement_for_a_node on placement (node, module) where node is not null;
-- Why a layer went into the history, beside how (`replaced_by`): said for a layer this migration moved.
alter table settings_history add column why text;
-- Each layer that held a placement key is kept whole in the history first, as a `settings history` reads it.
insert into settings_history (node, module, values, set_at, replaced_by, set_by, why)
select node, module, values, set_at, 'moved', set_by, 'moved to placement, ADR 0348 clause 11'
from settings
where values ?| array['places', 'accesses', 'networks'];
insert into placement (node, module, values, set_at, set_by)
select node, module,
(select jsonb_object_agg(e.key, e.value) from jsonb_each(values) as e
where e.key in ('places', 'accesses', 'networks')),
set_at, set_by
from settings
where values ?| array['places', 'accesses', 'networks'];
-- The layer keeps its other keys, when it was set and who set it; a layer that held only placement stays,
-- empty, so a layer that existed still exists.
update settings
set values = values - 'places' - 'accesses' - 'networks'
where values ?| array['places', 'accesses', 'networks'];
+122
View File
@@ -0,0 +1,122 @@
package inventory
import (
"bytes"
"context"
"encoding/json"
"fmt"
"time"
)
// The values of the controller's record mirrors, built from the store (novox/hq ADR 0289, design 54 §1,
// design 57 package F3).
//
// **Each value is built from a named list of fields, and nothing else** (design 54 §1): no epoch, no time
// of writing and no time of the last look, so any lease holder builds the same bytes from the same record
// and a pass over unchanged records writes nothing (ADR 0289 §1, §4). The encoding is fixed: the fields in
// their struct's order, a declaration's maps sorted.
// AssignmentMirrored is one assignment as the assignments mirror holds it: the module, the node, and since
// when it is assigned there. Design 54 §1 also lists whether it is pending, the builds sent and running and
// the seats and provisions it holds there; those join with issue 419, which builds the rest of design 54.
type AssignmentMirrored struct {
Node string `json:"node"`
Module string `json:"module"`
Assigned time.Time `json:"assigned"`
}
// DeclaredConfigMirrored is one registered module's config declarations as the declared-config mirror holds
// them: the module, and each key its manifest's `settings` declares with that declaration's listed fields.
// An empty object when the module declares none: registered, declaring nothing.
type DeclaredConfigMirrored struct {
Module string `json:"module"`
Settings map[string]DeclaredMirrored `json:"settings"`
}
// DeclaredMirrored is one key's declaration as the mirror holds it: **its listed fields and no other**
// (ADR 0289 §12). Today a declaration is kind, default and why (catalogue.SettingDeclaration); a field the
// manifest gains reaches the bus only once it is added here, deliberately, and a test fails until it is
// listed or refused (ADR 0348 clause 4's fields arrive with package K1). A default is never a secret: the
// catalogue refuses a default for a key whose name says secret.
type DeclaredMirrored struct {
Kind string `json:"kind"`
Default json.RawMessage `json:"default"`
Why string `json:"why"`
}
// DeclaredMirroredFields are the fields of a declaration the mirror carries, by their manifest names.
var DeclaredMirroredFields = []string{"kind", "default", "why"}
// AssignmentsMirror is every assignment's mirrored value, keyed `<node>.<module>`.
func (i *Inventory) AssignmentsMirror(ctx context.Context) (map[string][]byte, error) {
rows, err := i.store.Pool().Query(ctx,
`select n.name, a.module, a.assigned from assignment a join node n on n.id = a.node`)
if err != nil {
return nil, err
}
defer rows.Close()
out := map[string][]byte{}
for rows.Next() {
var a AssignmentMirrored
if err := rows.Scan(&a.Node, &a.Module, &a.Assigned); err != nil {
return nil, err
}
a.Assigned = a.Assigned.UTC()
value, err := json.Marshal(a)
if err != nil {
return nil, err
}
out[a.Node+"."+a.Module] = value
}
return out, rows.Err()
}
// DeclaredConfigMirror is every registered module's mirrored config declarations, keyed by the module.
func (i *Inventory) DeclaredConfigMirror(ctx context.Context) (map[string][]byte, error) {
rows, err := i.store.Pool().Query(ctx,
`select name, coalesce(manifest->'settings', '{}'::jsonb) from module`)
if err != nil {
return nil, err
}
defer rows.Close()
out := map[string][]byte{}
for rows.Next() {
var (
d DeclaredConfigMirrored
raw []byte
)
if err := rows.Scan(&d.Module, &raw); err != nil {
return nil, err
}
var declared map[string]DeclaredMirrored
if err := json.Unmarshal(raw, &declared); err != nil {
return nil, fmt.Errorf("the declarations of %s cannot be read: %w", d.Module, err)
}
d.Settings = map[string]DeclaredMirrored{}
for key, decl := range declared {
if len(decl.Default) > 0 {
if decl.Default, err = canonical(decl.Default); err != nil {
return nil, err
}
}
d.Settings[key] = decl
}
value, err := json.Marshal(d)
if err != nil {
return nil, err
}
out[d.Module] = value
}
return out, rows.Err()
}
// canonical is a JSON document encoded the one way: objects' keys sorted, numbers as written.
func canonical(raw []byte) ([]byte, error) {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var v any
if err := dec.Decode(&v); err != nil {
return nil, err
}
return json.Marshal(v)
}
-222
View File
@@ -1,222 +0,0 @@
package inventory
import (
"encoding/json"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// placedLibrary is a module with each kind of placement: a directory to place, two accesses, a container that
// may keep a found network.
func placedLibrary() catalogue.Manifest {
return catalogue.Manifest{Module: "library", Version: "1",
Accesses: []catalogue.Access{{ID: "series"}, {ID: "downloads"}},
Resources: []map[string]any{
{"id": "config", "type": "directory", "mode": "0750"},
{"id": "server", "type": "container",
"image": "registry.invalid:5000/library@sha256:" + strings.Repeat("a", 64),
"volumes": []any{"${dir:config}:/config", "${access:series}:/series", "${access:downloads}:/downloads"}},
{"id": "runtime-config", "type": "file", "path": "${dir:config}/runtime.json", "mode": "0640",
"content": `{"log":"info"}`, "merge": "json"},
}}
}
// The layers of the issue (651): a mesh-wide layer with an access and a config value, a node's layer with a
// place, an access and a kept network beside a config value, a node's layer of placement only, and one with
// none.
var placementLayers = []struct {
node string
values string
}{
{"", `{"accesses": {"downloads": "/storage/downloads"}, "log": "warn"}`},
{"anchor", `{"places": {"config": {"path": "/var/lib/library/config", "owner": "1001:2000"}},
"accesses": {"series": "/storage/media/series"}, "networks": {"server": ["predecessor_default"]}, "workers": 4}`},
{"laptop", `{"accesses": {"series": "/home/media/series"}}`},
{"spare", `{"log": "debug"}`},
}
// **Placement moves out of the settings layers, and every layer reads as it did** (novox/hq ADR 0348 clause
// 11, issue 651). Migration 0097 moves each layer's places, accesses and kept networks beside it: no settings
// row holds a placement key after, every layer the settings verb and composition read is what it was, each
// moved layer is kept whole in the history with why, and a layer without placement is not touched.
func TestPlacementMovesOutOfTheSettingsLayers(t *testing.T) {
inv, migrate := ForTestBefore(t, 97)
ctx := t.Context()
if err := inv.RegisterModule(ctx, placedLibrary(), Source{}); err != nil {
t.Fatal(err)
}
for _, n := range []string{"anchor", "laptop", "spare"} {
if _, err := inv.AddNode(ctx, n); err != nil {
t.Fatal(err)
}
}
if err := inv.SetAdopted(ctx, "anchor", true); err != nil {
t.Fatal(err)
}
// As the mesh holds them before the migration: placement among the settings.
for _, l := range placementLayers {
if l.node == "" {
inv.ExecForTest(t, `insert into settings (node, module, values, set_by) values (null, 'library', $1, 'the operator')`, l.values)
continue
}
inv.ExecForTest(t, `insert into settings (node, module, values) select id, 'library', $2 from node where name = $1`,
l.node, l.values)
}
setAt := inv.QueryForTest(t, `select coalesce(n.name, ''), s.set_at from settings s left join node n on n.id = s.node order by 1`)
migrate()
if held := inv.QueryForTest(t, `select module from settings where values ?| array['places', 'accesses', 'networks']`); len(held) != 0 {
t.Fatalf("a settings layer still holds a placement key: %v", held)
}
if after := inv.QueryForTest(t, `select coalesce(n.name, ''), s.set_at from settings s left join node n on n.id = s.node order by 1`); !reflect.DeepEqual(after, setAt) {
t.Fatalf("the layers or when they were set changed: %v, was %v", after, setAt)
}
for _, l := range placementLayers {
var want map[string]any
if err := json.Unmarshal([]byte(l.values), &want); err != nil {
t.Fatal(err)
}
got, has, err := inv.Layer(ctx, l.node, "library")
if err != nil || !has || !reflect.DeepEqual(roundTrip(t, got), want) {
t.Errorf("the layer on %q reads %v (%v, %v), was %v", l.node, got, has, err, want)
}
}
// Composition's view: the config values in Values, the placement in Placement, together the layer.
layers, err := inv.SettingsFor(ctx, "anchor", "library")
if err != nil || len(layers) != 2 {
t.Fatalf("anchor's layers: %v, %v", layers, err)
}
if layers[1].Values["workers"] == nil || layers[1].Values["places"] != nil || layers[1].Placement["places"] == nil ||
layers[1].Placement["networks"] == nil || layers[0].Placement["accesses"] == nil || layers[0].Values["log"] != "warn" {
t.Fatalf("anchor's layers split wrong: %+v", layers)
}
for i, l := range layers {
var want map[string]any
_ = json.Unmarshal([]byte(placementLayers[i].values), &want)
if !reflect.DeepEqual(roundTrip(t, l.Whole()), want) {
t.Errorf("anchor's layer %s is %v whole, was %v", l.From, l.Whole(), want)
}
}
// Judged as composition judges them, as before: anchor is adopted, so its kept network composes.
if err := catalogue.JudgeSettings(placedLibrary(), layers, true); err != nil {
t.Fatalf("the moved layers no longer compose: %v", err)
}
moved := inv.QueryForTest(t, `select coalesce(n.name, ''), h.replaced_by, h.why, h.set_by from settings_history h
left join node n on n.id = h.node order by 1`)
want := [][]string{
{`""`, `"moved"`, `"moved to placement, ADR 0348 clause 11"`, `"the operator"`},
{`"anchor"`, `"moved"`, `"moved to placement, ADR 0348 clause 11"`, `null`},
{`"laptop"`, `"moved"`, `"moved to placement, ADR 0348 clause 11"`, `null`},
}
if !reflect.DeepEqual(moved, want) {
t.Fatalf("the history of what moved: %v, want %v", moved, want)
}
past, err := inv.SettingsHistory(ctx, "anchor", "library")
if err != nil || len(past) != 1 || past[0].Why != "moved to placement, ADR 0348 clause 11" || past[0].Values["places"] == nil {
t.Fatalf("settings history on anchor: %+v, %v", past, err)
}
}
// **The settings verb's path still takes a placement key, and routes it** (issue 651): a set keeps the
// placement beside the layer and the rest in it, reads back whole, is refused as before when the placement
// cannot compose (nothing written), and a set or a clear replaces the placement with the layer.
func TestASettingRoutesPlacementBesideTheLayer(t *testing.T) {
inv := ForTest(t)
ctx := t.Context()
if err := inv.RegisterModule(ctx, placedLibrary(), Source{}); err != nil {
t.Fatal(err)
}
if _, err := inv.AddNode(ctx, "anchor"); err != nil {
t.Fatal(err)
}
// Placement alone is a layer as well: the whole mesh's says where both accesses are.
meshWide := map[string]any{"accesses": map[string]any{
"downloads": "/storage/downloads", "series": "/storage/series"}}
if err := inv.SetSettings(ctx, "", "library", meshWide); err != nil {
t.Fatal(err)
}
if got, has, _ := inv.Layer(ctx, "", "library"); !has || !reflect.DeepEqual(roundTrip(t, got), roundTrip(t, meshWide)) {
t.Fatalf("the mesh-wide layer of placement only reads %v (%v)", got, has)
}
set := map[string]any{"log": "warn",
"places": map[string]any{"config": map[string]any{"path": "/var/lib/library/config"}},
"accesses": map[string]any{"series": "/storage/media/series"}}
if err := inv.SetSettings(ctx, "anchor", "library", set); err != nil {
t.Fatal(err)
}
stored := inv.QueryForTest(t, `select s.values, p.values from settings s join placement p using (module, node)
where s.node is not null`)
if len(stored) != 1 || stored[0][0] != `{"log":"warn"}` ||
!strings.Contains(stored[0][1], `"places"`) || !strings.Contains(stored[0][1], `"accesses"`) {
t.Fatalf("the layer and its placement are kept as %v", stored)
}
if got, _, _ := inv.Layer(ctx, "anchor", "library"); !reflect.DeepEqual(roundTrip(t, got), roundTrip(t, set)) {
t.Fatalf("the layer reads %v, was set %v", got, set)
}
// Refused as before, by the same judgement, and nothing changes.
for name, bad := range map[string]map[string]any{
"an undeclared directory": {"places": map[string]any{"cache": "/var/lib/library/cache"}},
"the machine's own": {"accesses": map[string]any{"series": "/etc"}},
"a kept network on a converged machine": {
"networks": map[string]any{"server": []any{"predecessor_default"}}},
} {
if err := inv.SetSettings(ctx, "anchor", "library", bad); err == nil {
t.Errorf("%s was kept", name)
}
}
if err := inv.SetSettings(ctx, "", "library", map[string]any{"accesses": meshWide["accesses"],
"networks": map[string]any{"server": []any{"x"}}}); err == nil || !strings.Contains(err.Error(), "per node") {
t.Errorf("a mesh-wide kept network: %v", err)
}
if got, _, _ := inv.Layer(ctx, "anchor", "library"); !reflect.DeepEqual(roundTrip(t, got), roundTrip(t, set)) {
t.Fatalf("a refused set changed the layer to %v", got)
}
// A set without placement replaces it too, and the history keeps the layer whole.
if err := inv.SetSettings(ctx, "anchor", "library", map[string]any{"log": "debug"}); err != nil {
t.Fatal(err)
}
if rows := inv.QueryForTest(t, `select module from placement where node is not null`); len(rows) != 0 {
t.Fatalf("a set without placement left it: %v", rows)
}
past, err := inv.SettingsHistory(ctx, "anchor", "library")
if err != nil || len(past) != 1 || past[0].Values["places"] == nil || past[0].Values["log"] != "warn" {
t.Fatalf("the replaced layer is not kept whole: %+v, %v", past, err)
}
// The merge gate's way in keeps a layer as the facts carry it, whole: split the same way.
if err := inv.KeepSettings(ctx, "anchor", "library", map[string]any{"log": "info",
"places": map[string]any{"config": "/var/lib/library/config"}}); err != nil {
t.Fatal(err)
}
if rows := inv.QueryForTest(t, `select values from settings where values ? 'places'`); len(rows) != 0 {
t.Fatalf("a kept layer put placement among the settings: %v", rows)
}
for _, node := range []string{"", "anchor"} {
if err := inv.ClearSettings(ctx, node, "library"); err != nil {
t.Fatal(err)
}
}
if rows := inv.QueryForTest(t, `select module from placement`); len(rows) != 0 {
t.Fatalf("a clear left the placement: %v", rows)
}
}
func roundTrip(t *testing.T, v map[string]any) map[string]any {
t.Helper()
b, err := json.Marshal(v)
if err != nil {
t.Fatal(err)
}
out := map[string]any{}
if err := json.Unmarshal(b, &out); err != nil {
t.Fatal(err)
}
return out
}
+41
View File
@@ -0,0 +1,41 @@
package inventory
import (
"reflect"
"slices"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// jsonFields is the names a struct's fields are encoded under.
func jsonFields(t reflect.Type) []string {
var out []string
for i := 0; i < t.NumField(); i++ {
name, _, _ := strings.Cut(t.Field(i).Tag.Get("json"), ",")
if name != "" && name != "-" {
out = append(out, name)
}
}
return out
}
// **A declaration reaches the bus with its listed fields and no other** (ADR 0289 §12): the mirror's value
// carries exactly DeclaredMirroredFields, and every field a manifest's declaration has is on that list. A
// field a declaration gains (package K1 widens it, ADR 0348 clause 4) fails here until it is listed — read,
// judged not a secret and carried — so nothing reaches the declared-config mirror unreviewed.
func TestTheDeclaredConfigMirrorCarriesOnlyItsListedFields(t *testing.T) {
if got := jsonFields(reflect.TypeOf(DeclaredMirrored{})); !slices.Equal(got, DeclaredMirroredFields) {
t.Errorf("the mirrored declaration carries %v, its list is %v", got, DeclaredMirroredFields)
}
for _, field := range jsonFields(reflect.TypeOf(catalogue.SettingDeclaration{})) {
if !slices.Contains(DeclaredMirroredFields, field) {
t.Errorf("a declaration has the field %q, which the declared-config mirror does not list: list it "+
"in DeclaredMirrored (and say why it is no secret), or say here why it stays off the bus", field)
}
}
if got := jsonFields(reflect.TypeOf(DeclaredConfigMirrored{})); !slices.Equal(got, []string{"module", "settings"}) {
t.Errorf("the mirrored module carries %v", got)
}
}
-25
View File
@@ -1,25 +0,0 @@
package inventory
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The manifest the mesh holds carries each file's text (novox/hq ADR 0349 §2): one still naming a template — a
// build by a builder that predates templates, or a manifest handed over unbuilt — would compose a file with no
// text, and is refused by name.
func TestAManifestStillNamingATemplateIsNotRegistered(t *testing.T) {
inv := ForTest(t)
m := catalogue.Manifest{Module: "notifier", Version: "1", Resources: []map[string]any{
{"id": "conf", "type": "file", "path": "/etc/notifier.conf", "template": "files/notifier.conf"}}}
err := inv.RegisterModule(t.Context(), m, Source{})
if err == nil || !strings.Contains(err.Error(), "files/notifier.conf") {
t.Fatalf("a manifest naming a template was registered: %v", err)
}
m.Resources[0] = map[string]any{"id": "conf", "type": "file", "path": "/etc/notifier.conf", "content": "x"}
if err := inv.RegisterModule(t.Context(), m, Source{}); err != nil {
t.Fatalf("the same manifest with its text was refused: %v", err)
}
}
+7 -21
View File
@@ -20,11 +20,9 @@ func (i *Inventory) Layer(ctx context.Context, nodeName, module string) (map[str
if err != nil {
return nil, false, err
}
// Said whole, as it was set: its config values and the placement kept beside them (novox/hq ADR 0348
// clause 11), so a person reading it before setting it sees every key they may keep or remove.
var raw []byte
err = i.store.Pool().QueryRow(ctx, `select `+layerWhole+` from settings s
where s.module = $1 and s.node is not distinct from $2::uuid`,
err = i.store.Pool().QueryRow(ctx,
`select values from settings where module = $1 and node is not distinct from $2::uuid`,
module, nodeID).Scan(&raw)
if errors.Is(err, pgx.ErrNoRows) {
return nil, false, nil
@@ -48,9 +46,6 @@ type PastLayer struct {
ReplacedBy string
// SetBy is who had set the layer, when it was kept (novox/hq ADR 0277); empty for one set before that was kept.
SetBy string
// Why is why it went, where that was said: "moved to placement, ADR 0348 clause 11" for a layer whose
// placement keys migration 0097 moved out of it (ReplacedBy "moved").
Why string
}
// LayerOrigin is who set a layer and when (novox/hq ADR 0277): empty words for a layer set before who set it was
@@ -84,7 +79,7 @@ func (i *Inventory) SettingsHistory(ctx context.Context, nodeName, module string
return nil, err
}
rows, err := i.store.Pool().Query(ctx,
`select values, set_at, replaced_at, replaced_by, set_by, why from settings_history
`select values, set_at, replaced_at, replaced_by, set_by from settings_history
where module = $1 and node is not distinct from $2::uuid order by replaced_at desc`,
module, nodeID)
if err != nil {
@@ -95,16 +90,13 @@ func (i *Inventory) SettingsHistory(ctx context.Context, nodeName, module string
for rows.Next() {
var raw []byte
var p PastLayer
var by, why *string
if err := rows.Scan(&raw, &p.SetAt, &p.ReplacedAt, &p.ReplacedBy, &by, &why); err != nil {
var by *string
if err := rows.Scan(&raw, &p.SetAt, &p.ReplacedAt, &p.ReplacedBy, &by); err != nil {
return nil, err
}
if by != nil {
p.SetBy = *by
}
if why != nil {
p.Why = *why
}
if err := json.Unmarshal(raw, &p.Values); err != nil {
return nil, err
}
@@ -115,15 +107,9 @@ func (i *Inventory) SettingsHistory(ctx context.Context, nodeName, module string
// keepReplacedSQL copies a layer into the history before it is replaced or cleared; run in the same
// transaction as the write where there is one, so a write that fails leaves no history of it.
//
// The layer is kept whole, as it was set: its config values and its placement (novox/hq ADR 0348 clause 11).
const keepReplacedSQL = `insert into settings_history (node, module, values, set_at, replaced_by, set_by)
select s.node, s.module, ` + layerWhole + `, s.set_at, $3, s.set_by from settings s
where s.module = $1 and s.node is not distinct from $2::uuid`
// layerWhole is a settings row `s` with the placement kept beside it folded back in: the layer as it was set.
const layerWhole = `s.values || coalesce((select p.values from placement p
where p.module = s.module and p.node is not distinct from s.node), '{}'::jsonb)`
select node, module, values, set_at, $3, set_by from settings
where module = $1 and node is not distinct from $2::uuid`
// layerNode is the node id of a layer, nil for the whole mesh's.
func (i *Inventory) layerNode(ctx context.Context, nodeName string) (*string, error) {
+5
View File
@@ -81,6 +81,11 @@ const (
// steps from. Said after every save of the serving controller; a delivery's holder also asks for the
// walks it follows, so a save made elsewhere is found by comparison, never lost.
KeyPlanMoved = "plan-moved"
// KeyAssignmentChanged and KeyDeclaredConfigChanged: a key of the controller's assignments or declared-config
// mirror was written or removed (novox/hq ADR 0289, design 54 §3, design 57 F3): which key, how and at which
// revision of the bucket — never the value, which the bucket holds.
KeyAssignmentChanged = "assignment-changed"
KeyDeclaredConfigChanged = "declared-config-changed"
)
// Applied is what a machine now runs, as the mesh states it.
+344
View File
@@ -0,0 +1,344 @@
// Package recordmirror writes the controller's record mirrors (novox/hq ADR 0289, design 54): a bucket per kind
// of record, one key per record, written from the store only when the record changed, and one event on the
// controller's seat per write.
//
// **The store stays the record.** A mirror is built from what the store committed, so a change rolled back
// is never written. Every pass reads what the bucket holds and compares it byte for byte with what the store
// says: a pass over unchanged records writes nothing and says nothing (ADR 0289 §1), and a value somebody
// else wrote behind the writer's back (a former lease holder) differs from the store and is put right on
// the next pass. Each write names the revision it read (compare-and-set), so a write in between is noticed,
// never overwritten blind.
//
// **Only the lease holder writes** (ADR 0289 §9, design 54 §2): a pass begins by asking whether this
// controller holds the lease, and asks again before every write; without it, it writes nothing.
//
// **The bucket is the authority, the event a notice** (design 54 §3): the event is sent after the write
// succeeded, so there is no event for a value the bucket does not hold. A refused event is tried once more
// and then counted as lost; a watcher that must not miss a change watches the bucket.
//
// What this package does not do yet is design 54 §4's audit (probe D17 and its healer H6), which compares a
// mirror with its store behind the writer's back; novox/hq issue 419 carries it.
package recordmirror
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"regexp"
"sort"
"time"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
)
// A Mirror is one kind of record the controller keeps on the bus: its local name, its bucket and the event
// each write raises on the controller's seat.
type Mirror struct {
Name string
Bucket string
Event string
}
// The mirrors this package writes (novox/hq design 57, package F3). The assignments mirror is design 54's,
// keyed `<node>.<module>`; the declared config is each registered module's config declarations as its
// manifest holds them, keyed by the module, which mesh-config resolves a module's config values from
// (ADR 0348 clauses 2 and 4).
var (
Assignments = Mirror{Name: broker.MirrorAssignments, Bucket: broker.AssignmentsBucket,
Event: link.KeyAssignmentChanged}
DeclaredConfig = Mirror{Name: broker.MirrorDeclaredConfig, Bucket: broker.DeclaredConfigBucket,
Event: link.KeyDeclaredConfigChanged}
)
// Mirrors is every mirror this package writes, in the order they are written.
func Mirrors() []Mirror { return []Mirror{Assignments, DeclaredConfig} }
// The changes an event says.
const (
Added = "added"
Changed = "changed"
Removed = "removed"
)
// Event is what one write says on the controller's seat (design 54 §3): which mirror and key, what kind of
// change, the names of the top-level fields that changed, the bucket's revision of the write, the lease's
// epoch and when. Never the value: a watcher reads it from the bucket at that revision or later.
type Event struct {
Mirror string `json:"mirror"`
Key string `json:"key"`
Change string `json:"change"`
Fields []string `json:"fields,omitempty"`
Revision uint64 `json:"revision"`
Epoch uint64 `json:"epoch,omitempty"`
At time.Time `json:"at"`
}
// Announcer says an event on a seat; link.OverNATS is one, and so is link.Bus.
type Announcer interface {
PublishSeatEvent(ctx context.Context, seat, event string, body []byte) error
}
// MarkerTTL is how long the marker a purge leaves is kept (design 54 §2): a day, so the markers of removed
// records do not pile up in a bucket that has no age.
const MarkerTTL = 24 * time.Hour
// ErrNotHeld is a pass refused because this controller does not hold the lease: it wrote nothing.
var ErrNotHeld = errors.New("this controller does not hold the lease, so it writes no mirror")
// Writer writes one mirror. Not safe for use by two goroutines at once: one controller's one loop owns it.
type Writer struct {
Mirror Mirror
KV jetstream.KeyValue
Announce Announcer
// Epoch is the lease's epoch the events carry; nil carries none.
Epoch func() uint64
// Holds says this controller holds the lease. Asked before every pass and every write; a writer
// without it writes nothing (design 54 §2). Required: nil holds nothing.
Holds func() bool
// Now is the clock the events are dated by; nil is time.Now.
Now func() time.Time
known map[string]kept
// LostEvents counts the events refused twice since the writer was made.
LostEvents int
}
type kept struct {
value []byte
revision uint64
}
// Result is what one pass did: keys written, keys removed, and keys refused with why.
type Result struct {
Written int
Removed int
Refused int
}
// Sync makes the bucket hold want, one key per record: writes each value that differs from what the key
// holds, purges each key no record is left for, and raises one event per write. A key the bus cannot hold
// is refused alone, and the others are written; the error then names it.
func (w *Writer) Sync(ctx context.Context, want map[string][]byte) (Result, error) {
var r Result
if !w.holds() {
return r, ErrNotHeld
}
// What the bucket holds now, every pass: a value written behind this writer's back differs from the
// store's and is written again below.
if err := w.fill(ctx); err != nil {
return r, fmt.Errorf("reading the %s mirror: %w", w.Mirror.Name, err)
}
var problems []error
keys := make([]string, 0, len(want))
for k := range want {
keys = append(keys, k)
}
sort.Strings(keys)
for _, key := range keys {
value := want[key]
before, had := w.known[key]
if had && bytes.Equal(before.value, value) {
continue
}
if err := validKey(key); err != nil {
r.Refused++
problems = append(problems, err)
continue
}
if !w.holds() {
return r, errors.Join(append(problems, ErrNotHeld)...)
}
revision, wrote, err := w.put(ctx, key, value, before, had)
if err != nil {
problems = append(problems, fmt.Errorf("writing %s: %w", key, err))
continue
}
w.known[key] = kept{value: append([]byte(nil), value...), revision: revision}
if !wrote {
continue
}
r.Written++
change, fields := Added, []string(nil)
if had {
change, fields = Changed, changedFields(before.value, value)
}
w.say(ctx, key, change, fields, revision)
}
gone := make([]string, 0)
for key := range w.known {
if _, ok := want[key]; !ok {
gone = append(gone, key)
}
}
sort.Strings(gone)
for _, key := range gone {
k := w.known[key]
if !w.holds() {
return r, errors.Join(append(problems, ErrNotHeld)...)
}
err := w.KV.Purge(ctx, key, jetstream.LastRevision(k.revision), jetstream.PurgeTTL(MarkerTTL))
if refusedByRevision(err) {
// Read it again: purged already, or moved by another writer.
if e, gerr := w.KV.Get(ctx, key); errors.Is(gerr, jetstream.ErrKeyNotFound) {
delete(w.known, key)
continue
} else if gerr == nil && w.holds() {
err = w.KV.Purge(ctx, key, jetstream.LastRevision(e.Revision()), jetstream.PurgeTTL(MarkerTTL))
}
}
if err != nil {
problems = append(problems, fmt.Errorf("removing %s: %w", key, err))
continue
}
delete(w.known, key)
r.Removed++
revision := uint64(0)
if status, err := w.KV.Status(ctx); err == nil {
// The purge's own sequence is the stream's last one only when nothing wrote in between, which no
// other writer may; said as the bucket's revision after it.
if s, ok := status.(interface{ StreamInfo() *jetstream.StreamInfo }); ok {
revision = s.StreamInfo().State.LastSeq
}
}
w.say(ctx, key, Removed, nil, revision)
}
return r, errors.Join(problems...)
}
// put writes one value by compare-and-set and answers its revision, and whether this call wrote it (false
// when the bucket already held it).
func (w *Writer) put(ctx context.Context, key string, value []byte, before kept, had bool) (uint64, bool, error) {
var (
revision uint64
err error
)
if had {
revision, err = w.KV.Update(ctx, key, value, before.revision)
} else {
revision, err = w.KV.Create(ctx, key, value)
}
if !refusedByRevision(err) {
return revision, err == nil, err
}
// Refused: another writer in between, or this writer's own write whose acknowledgement was lost.
e, gerr := w.KV.Get(ctx, key)
switch {
case errors.Is(gerr, jetstream.ErrKeyNotFound):
if !w.holds() {
return 0, false, errors.New("the key moved and this controller no longer holds the lease")
}
revision, err = w.KV.Create(ctx, key, value)
return revision, err == nil, err
case gerr != nil:
return 0, false, gerr
case bytes.Equal(e.Value(), value):
return e.Revision(), false, nil
case !w.holds():
return 0, false, errors.New("the key moved and this controller no longer holds the lease")
}
revision, err = w.KV.Update(ctx, key, value, e.Revision())
return revision, err == nil, err
}
// say raises one write's event, tried once more when refused and then counted as lost.
func (w *Writer) say(ctx context.Context, key, change string, fields []string, revision uint64) {
if w.Announce == nil {
return
}
e := Event{Mirror: w.Mirror.Name, Key: key, Change: change, Fields: fields, Revision: revision, At: w.now().UTC()}
if w.Epoch != nil {
e.Epoch = w.Epoch()
}
body, err := json.Marshal(e)
if err != nil {
w.LostEvents++
return
}
for try := 0; try < 2; try++ {
if err = w.Announce.PublishSeatEvent(ctx, broker.ControllerSeat, w.Mirror.Event, body); err == nil {
return
}
}
w.LostEvents++
}
// fill reads what the bucket holds now, so a controller started again writes only what differs.
func (w *Writer) fill(ctx context.Context) error {
w.known = map[string]kept{}
watch, err := w.KV.WatchAll(ctx, jetstream.IgnoreDeletes())
if err != nil {
return err
}
defer func() { _ = watch.Stop() }()
for {
select {
case <-ctx.Done():
return ctx.Err()
case e := <-watch.Updates():
if e == nil {
return nil
}
w.known[e.Key()] = kept{value: append([]byte(nil), e.Value()...), revision: e.Revision()}
}
}
}
func (w *Writer) holds() bool { return w.Holds != nil && w.Holds() }
func (w *Writer) now() time.Time {
if w.Now == nil {
return time.Now()
}
return w.Now()
}
// refusedByRevision is the server refusing a compare-and-set: the key is not at the revision named.
func refusedByRevision(err error) bool {
if err == nil {
return false
}
if errors.Is(err, jetstream.ErrKeyExists) {
return true
}
var apiErr *jetstream.APIError
return errors.As(err, &apiErr) && apiErr.ErrorCode == jetstream.JSErrCodeStreamWrongLastSequence
}
// keyChars is what a bucket's key may be made of.
var keyChars = regexp.MustCompile(`^[-/_=.a-zA-Z0-9]+$`)
// validKey refuses a key the bus cannot hold, naming it.
func validKey(key string) error {
if key == "" || !keyChars.MatchString(key) || key[0] == '.' || key[len(key)-1] == '.' {
return fmt.Errorf("%q cannot be a key of a bucket: letters, digits, '-', '_', '=', '/' and '.', not at "+
"either end", key)
}
return nil
}
// changedFields names the top-level fields whose values differ between two encoded objects, sorted; nil
// when either is not an object.
func changedFields(before, after []byte) []string {
var a, b map[string]json.RawMessage
if json.Unmarshal(before, &a) != nil || json.Unmarshal(after, &b) != nil || a == nil || b == nil {
return nil
}
var out []string
for k, v := range b {
if old, ok := a[k]; !ok || !bytes.Equal(old, v) {
out = append(out, k)
}
}
for k := range a {
if _, ok := b[k]; !ok {
out = append(out, k)
}
}
sort.Strings(out)
return out
}
+178
View File
@@ -0,0 +1,178 @@
package recordmirror
import (
"encoding/json"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// **Design 57's F3, end to end** (novox/hq issue 652): from the controller's store to a real bus. An
// assignment changes exactly one key of the assignments mirror and raises exactly one `assignment-changed`;
// a registration changes exactly one key of the declared-config mirror and raises exactly one
// `declared-config-changed`; and the same records written again change nothing and say nothing — a module
// registered again unchanged included, although the store's registration time moved.
func TestNatsAnAssignmentAndARegistrationEachChangeOneKeyAndRaiseOneEvent(t *testing.T) {
inv := inventory.ForTest(t)
ctx := t.Context()
js, api := aMirrorBus(t)
assignments := aWriter(t, js, api, Assignments)
declared := aWriter(t, js, api, DeclaredConfig)
heardAssignments := listen(t, js.Conn(), "mesh.seat."+broker.ControllerSeat+".event."+Assignments.Event)
heardDeclared := listen(t, js.Conn(), "mesh.seat."+broker.ControllerSeat+".event."+DeclaredConfig.Event)
pass := func() (Result, Result) {
t.Helper()
a, err := inv.AssignmentsMirror(ctx)
if err != nil {
t.Fatal(err)
}
ra, err := assignments.Sync(ctx, a)
if err != nil {
t.Fatal(err)
}
d, err := inv.DeclaredConfigMirror(ctx)
if err != nil {
t.Fatal(err)
}
rd, err := declared.Sync(ctx, d)
if err != nil {
t.Fatal(err)
}
return ra, rd
}
if _, err := inv.AddNode(ctx, "anchor"); err != nil {
t.Fatal(err)
}
ui := catalogue.Manifest{Module: "ui", Version: "1", Settings: map[string]catalogue.SettingDeclaration{
"font-size": {Kind: catalogue.KindPreference, Default: 14, Why: "readable on the desktop"}}}
if err := inv.RegisterModule(ctx, ui, inventory.Source{}); err != nil {
t.Fatal(err)
}
pass()
assignmentWrites, declaredWrites := writes(t, api, Assignments.Bucket), writes(t, api, DeclaredConfig.Bucket)
assignmentsSaid, declaredSaid := len(settle(heardAssignments, 0)), len(settle(heardDeclared, 1))
// **A registration**: a module new to the catalogue.
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "notes", Version: "1"}, inventory.Source{}); err != nil {
t.Fatal(err)
}
ra, rd := pass()
if ra != (Result{}) || rd != (Result{Written: 1}) {
t.Errorf("a registration wrote assignments %+v and declared config %+v", ra, rd)
}
if got := writes(t, api, DeclaredConfig.Bucket) - declaredWrites; got != 1 {
t.Errorf("a registration changed %d keys of the declared config", got)
}
if got := writes(t, api, Assignments.Bucket) - assignmentWrites; got != 0 {
t.Errorf("a registration changed %d keys of the assignments", got)
}
said := settle(heardDeclared, declaredSaid+1)[declaredSaid:]
if len(said) != 1 || said[0].Key != "notes" || said[0].Change != Added {
t.Errorf("a registration raised %+v", said)
}
declaredWrites, declaredSaid = writes(t, api, DeclaredConfig.Bucket), declaredSaid+len(said)
// **An assignment.**
if _, err := inv.Assign(ctx, "anchor", "ui"); err != nil {
t.Fatal(err)
}
ra, rd = pass()
if ra != (Result{Written: 1}) || rd != (Result{}) {
t.Errorf("an assignment wrote assignments %+v and declared config %+v", ra, rd)
}
if got := writes(t, api, Assignments.Bucket) - assignmentWrites; got != 1 {
t.Errorf("an assignment changed %d keys of the assignments", got)
}
said = settle(heardAssignments, assignmentsSaid+1)[assignmentsSaid:]
if len(said) != 1 || said[0].Key != "anchor.ui" || said[0].Change != Added {
t.Errorf("an assignment raised %+v", said)
}
assignmentWrites, assignmentsSaid = writes(t, api, Assignments.Bucket), assignmentsSaid+len(said)
e, err := assignments.KV.Get(ctx, "anchor.ui")
if err != nil {
t.Fatal(err)
}
var value inventory.AssignmentMirrored
if err := json.Unmarshal(e.Value(), &value); err != nil || value.Node != "anchor" || value.Module != "ui" ||
value.Assigned.IsZero() {
t.Errorf("the assignment is mirrored as %s (%v)", e.Value(), err)
}
// **The same records written again change nothing**: an assignment made again, and a module registered
// again with the manifest it had.
if _, err := inv.Assign(ctx, "anchor", "ui"); err != nil {
t.Fatal(err)
}
if err := inv.RegisterModule(ctx, ui, inventory.Source{}); err != nil {
t.Fatal(err)
}
if ra, rd := pass(); ra != (Result{}) || rd != (Result{}) {
t.Errorf("unchanged records wrote assignments %+v and declared config %+v", ra, rd)
}
if writes(t, api, Assignments.Bucket) != assignmentWrites || writes(t, api, DeclaredConfig.Bucket) != declaredWrites {
t.Error("unchanged records changed a key")
}
if a, d := settle(heardAssignments, assignmentsSaid+1), settle(heardDeclared, declaredSaid+1); len(a) != assignmentsSaid ||
len(d) != declaredSaid {
t.Errorf("unchanged records raised %+v and %+v", a[assignmentsSaid:], d[declaredSaid:])
}
// **A declaration changed by a registration**: one key, one event, naming the field that moved.
ui.Version = "2"
ui.Settings["font-size"] = catalogue.SettingDeclaration{Kind: catalogue.KindPreference, Default: 16,
Why: "readable on the desktop"}
if err := inv.RegisterModule(ctx, ui, inventory.Source{}); err != nil {
t.Fatal(err)
}
if ra, rd := pass(); ra != (Result{}) || rd != (Result{Written: 1}) {
t.Errorf("a changed declaration wrote assignments %+v and declared config %+v", ra, rd)
}
said = settle(heardDeclared, declaredSaid+1)[declaredSaid:]
if len(said) != 1 || said[0].Key != "ui" || said[0].Change != Changed || strings.Join(said[0].Fields, ",") != "settings" {
t.Errorf("a changed declaration raised %+v", said)
}
got, err := declared.KV.Get(ctx, "ui")
if err != nil {
t.Fatal(err)
}
if want := `{"module":"ui","settings":{"font-size":{"kind":"preference","default":16,"why":"readable on the desktop"}}}`; string(got.Value()) != want {
t.Errorf("the declaration is mirrored as %s, want %s", got.Value(), want)
}
// **An unassignment** removes its key, said once.
if err := inv.Unassign(ctx, "anchor", "ui"); err != nil {
t.Fatal(err)
}
if ra, _ := pass(); ra != (Result{Removed: 1}) {
t.Errorf("an unassignment did %+v", ra)
}
said = settle(heardAssignments, assignmentsSaid+1)[assignmentsSaid:]
if len(said) != 1 || said[0].Key != "anchor.ui" || said[0].Change != Removed {
t.Errorf("an unassignment raised %+v", said)
}
// **A module no longer registered** has its key removed, said once as `removed` (ADR 0356 §4).
if err := inv.ForgetModule(ctx, "notes"); err != nil {
t.Fatal(err)
}
declaredWrites = writes(t, api, DeclaredConfig.Bucket)
declaredSaid = len(heardDeclared.all())
if _, rd := pass(); rd != (Result{Removed: 1}) {
t.Errorf("a module forgotten did %+v", rd)
}
if got := writes(t, api, DeclaredConfig.Bucket) - declaredWrites; got != 1 {
t.Errorf("a module forgotten changed %d keys of the declared config", got)
}
if _, err := declared.KV.Get(ctx, "notes"); err == nil {
t.Error("a module forgotten keeps its key")
}
said = settle(heardDeclared, declaredSaid+1)[declaredSaid:]
if len(said) != 1 || said[0].Key != "notes" || said[0].Change != Removed {
t.Errorf("a module forgotten raised %+v", said)
}
}
+356
View File
@@ -0,0 +1,356 @@
package recordmirror
import (
"encoding/json"
"errors"
"slices"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// A record mirror against a real bus (novox/hq design 54 §2, §3; ADR 0289): a change of a record changes
// exactly one key and raises exactly one event; a pass over unchanged records writes nothing and says
// nothing; a record gone is a purge said as `removed`.
// aMirrorBus is a bus of this test alone with the mesh's streams and the controller's buckets, as the
// controller asserts them at its start, and a counter of the writes made to one bucket.
func aMirrorBus(t *testing.T) (*broker.JetStream, jetstream.JetStream) {
t.Helper()
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
if err := broker.AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
if err := js.EnsureControllerBuckets(); err != nil {
t.Fatal(err)
}
api, err := jetstream.New(js.Conn())
if err != nil {
t.Fatal(err)
}
return js, api
}
// heard counts what is said on one subject, each body kept.
type heard struct {
mu sync.Mutex
bodies []Event
}
func (h *heard) all() []Event {
h.mu.Lock()
defer h.mu.Unlock()
return append([]Event(nil), h.bodies...)
}
func listen(t *testing.T, conn *nats.Conn, subject string) *heard {
t.Helper()
h := &heard{}
sub, err := conn.Subscribe(subject, func(m *nats.Msg) {
var e Event
if err := json.Unmarshal(m.Data, &e); err != nil {
t.Errorf("an event that is not one: %s", m.Data)
return
}
h.mu.Lock()
h.bodies = append(h.bodies, e)
h.mu.Unlock()
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = sub.Unsubscribe() })
if err := conn.Flush(); err != nil {
t.Fatal(err)
}
return h
}
// writes is how many messages the bucket's stream has taken: every put and every purge is one.
func writes(t *testing.T, api jetstream.JetStream, bucket string) uint64 {
t.Helper()
s, err := api.Stream(t.Context(), "KV_"+bucket)
if err != nil {
t.Fatal(err)
}
info, err := s.Info(t.Context())
if err != nil {
t.Fatal(err)
}
return info.State.LastSeq
}
// settle waits until n events are heard or a second passed, and answers what was heard.
func settle(h *heard, n int) []Event {
deadline := time.Now().Add(time.Second)
for time.Now().Before(deadline) && len(h.all()) < n {
time.Sleep(10 * time.Millisecond)
}
// And a moment more, so an event too many is heard too.
time.Sleep(100 * time.Millisecond)
return h.all()
}
func aWriter(t *testing.T, js *broker.JetStream, api jetstream.JetStream, m Mirror) *Writer {
t.Helper()
kv, err := api.KeyValue(t.Context(), m.Bucket)
if err != nil {
t.Fatal(err)
}
return &Writer{Mirror: m, KV: kv, Announce: link.OverNATS{Conn: js.Conn(), JS: js.Context()},
Epoch: func() uint64 { return 7 }, Holds: func() bool { return true }}
}
func TestNatsAMirrorChangesOneKeyAndRaisesOneEventPerChange(t *testing.T) {
for _, m := range []Mirror{Assignments, DeclaredConfig} {
t.Run(m.Name, func(t *testing.T) {
js, api := aMirrorBus(t)
events := listen(t, js.Conn(), "mesh.seat."+broker.ControllerSeat+".event."+m.Event)
w := aWriter(t, js, api, m)
ctx := t.Context()
first := map[string][]byte{"anchor.gitea": []byte(`{"a":1}`)}
if _, err := w.Sync(ctx, first); err != nil {
t.Fatal(err)
}
before := writes(t, api, m.Bucket)
heardBefore := len(settle(events, 1))
// One record changes, one is added: two keys, two events, nothing else.
second := map[string][]byte{"anchor.gitea": []byte(`{"a":2}`), "anchor.forgejo": []byte(`{"a":1}`)}
r, err := w.Sync(ctx, second)
if err != nil {
t.Fatal(err)
}
if r.Written != 2 || r.Removed != 0 {
t.Errorf("a pass over one changed and one new record wrote %+v", r)
}
if got := writes(t, api, m.Bucket) - before; got != 2 {
t.Errorf("the bucket took %d writes for two changed records", got)
}
said := settle(events, heardBefore+2)[heardBefore:]
if len(said) != 2 {
t.Fatalf("two changes raised %d events: %+v", len(said), said)
}
changes := map[string]Event{}
for _, e := range said {
changes[e.Key] = e
}
if e := changes["anchor.gitea"]; e.Change != Changed || !slices.Equal(e.Fields, []string{"a"}) ||
e.Mirror != m.Name || e.Epoch != 7 || e.Revision == 0 {
t.Errorf("the change of anchor.gitea was said as %+v", e)
}
if e := changes["anchor.forgejo"]; e.Change != Added {
t.Errorf("the new anchor.forgejo was said as %+v", e)
}
// **An unchanged pass writes nothing**, and says nothing (ADR 0289 §1).
heardBefore = len(events.all())
before = writes(t, api, m.Bucket)
r, err = w.Sync(ctx, second)
if err != nil {
t.Fatal(err)
}
if r != (Result{}) {
t.Errorf("a pass over unchanged records did %+v", r)
}
if got := writes(t, api, m.Bucket) - before; got != 0 {
t.Errorf("a pass over unchanged records wrote the bucket %d times", got)
}
if said := settle(events, heardBefore+1)[heardBefore:]; len(said) != 0 {
t.Errorf("a pass over unchanged records raised %+v", said)
}
// **A record gone is a purge**, one write and one `removed`.
before = writes(t, api, m.Bucket)
r, err = w.Sync(ctx, map[string][]byte{"anchor.gitea": []byte(`{"a":2}`)})
if err != nil {
t.Fatal(err)
}
if r.Removed != 1 || r.Written != 0 {
t.Errorf("a record gone did %+v", r)
}
if got := writes(t, api, m.Bucket) - before; got != 1 {
t.Errorf("a record gone wrote the bucket %d times", got)
}
if _, err := w.KV.Get(ctx, "anchor.forgejo"); !errors.Is(err, jetstream.ErrKeyNotFound) {
t.Errorf("the key of a record gone is still read: %v", err)
}
said = settle(events, heardBefore+1)[heardBefore:]
if len(said) != 1 || said[0].Change != Removed || said[0].Key != "anchor.forgejo" {
t.Errorf("a record gone was said as %+v", said)
}
})
}
}
// **A controller started again writes nothing for what the bucket already holds** (design 54 §2: the writer's
// memory is filled from the bucket), and a key nobody holds a record for is removed.
func TestNatsAWriterStartedAgainWritesOnlyWhatDiffers(t *testing.T) {
js, api := aMirrorBus(t)
ctx := t.Context()
first := aWriter(t, js, api, Assignments)
held := map[string][]byte{"anchor.gitea": []byte(`{"a":1}`), "anchor.forgejo": []byte(`{"a":1}`)}
if _, err := first.Sync(ctx, held); err != nil {
t.Fatal(err)
}
before := writes(t, api, Assignments.Bucket)
again := aWriter(t, js, api, Assignments)
r, err := again.Sync(ctx, held)
if err != nil {
t.Fatal(err)
}
if r != (Result{}) || writes(t, api, Assignments.Bucket) != before {
t.Errorf("a writer started again over unchanged records did %+v", r)
}
}
// **Another writer in between is no lost value** (design 54 §2, compare-and-set): a write refused because the
// key moved reads it again, takes the revision when it already holds the value, and otherwise writes again.
func TestNatsAWriteRefusedByCompareAndSetReadsAgain(t *testing.T) {
js, api := aMirrorBus(t)
ctx := t.Context()
w := aWriter(t, js, api, Assignments)
if _, err := w.Sync(ctx, map[string][]byte{"anchor.gitea": []byte(`{"a":1}`)}); err != nil {
t.Fatal(err)
}
// Behind its back, as a former lease holder would.
if _, err := w.KV.Put(ctx, "anchor.gitea", []byte(`{"a":9}`)); err != nil {
t.Fatal(err)
}
r, err := w.Sync(ctx, map[string][]byte{"anchor.gitea": []byte(`{"a":2}`)})
if err != nil {
t.Fatal(err)
}
if r.Written != 1 {
t.Errorf("a refused write was not made again: %+v", r)
}
e, err := w.KV.Get(ctx, "anchor.gitea")
if err != nil || string(e.Value()) != `{"a":2}` {
t.Fatalf("the bucket holds %v, %v", e, err)
}
}
// **A value written behind the writer's back is put right on the next pass** (ADR 0289 §10): a former lease
// holder's write differs from the store, and the store's value is written again, said as one change.
func TestNatsAForeignWriteIsCorrectedOnTheNextPass(t *testing.T) {
js, api := aMirrorBus(t)
ctx := t.Context()
events := listen(t, js.Conn(), "mesh.seat."+broker.ControllerSeat+".event."+Assignments.Event)
w := aWriter(t, js, api, Assignments)
store := map[string][]byte{"anchor.gitea": []byte(`{"a":1}`)}
if _, err := w.Sync(ctx, store); err != nil {
t.Fatal(err)
}
heardBefore := len(settle(events, 1))
if _, err := w.KV.Put(ctx, "anchor.gitea", []byte(`{"a":9}`)); err != nil {
t.Fatal(err)
}
r, err := w.Sync(ctx, store)
if err != nil {
t.Fatal(err)
}
if r != (Result{Written: 1}) {
t.Errorf("a pass after a foreign write did %+v", r)
}
e, err := w.KV.Get(ctx, "anchor.gitea")
if err != nil || string(e.Value()) != `{"a":1}` {
t.Fatalf("the bucket still holds %v, %v", e, err)
}
if said := settle(events, heardBefore+1)[heardBefore:]; len(said) != 1 || said[0].Change != Changed {
t.Errorf("the correction was said as %+v", said)
}
// And put right, it stays: the next pass writes nothing.
if r, err := w.Sync(ctx, store); err != nil || r != (Result{}) {
t.Errorf("a pass after the correction did %+v, %v", r, err)
}
}
// **A writer without the lease writes nothing** (ADR 0289 §9, design 54 §2): not a new key, not a change, not
// a removal, and it says so.
func TestNatsAWriterWithoutTheLeaseWritesNothing(t *testing.T) {
js, api := aMirrorBus(t)
ctx := t.Context()
held := aWriter(t, js, api, Assignments)
if _, err := held.Sync(ctx, map[string][]byte{"anchor.gitea": []byte(`{"a":1}`)}); err != nil {
t.Fatal(err)
}
before := writes(t, api, Assignments.Bucket)
for _, holds := range []func() bool{nil, func() bool { return false }} {
w := aWriter(t, js, api, Assignments)
w.Holds = holds
r, err := w.Sync(ctx, map[string][]byte{"anchor.gitea": []byte(`{"a":2}`), "anchor.forgejo": []byte(`{"a":1}`)})
if !errors.Is(err, ErrNotHeld) || r != (Result{}) {
t.Errorf("a writer without the lease did %+v, %v", r, err)
}
r, err = w.Sync(ctx, map[string][]byte{})
if !errors.Is(err, ErrNotHeld) || r != (Result{}) {
t.Errorf("a writer without the lease removed: %+v, %v", r, err)
}
}
// And one that loses it mid-pass stops before its next write.
asked := 0
w := aWriter(t, js, api, Assignments)
w.Holds = func() bool { asked++; return asked <= 2 }
r, err := w.Sync(ctx, map[string][]byte{"anchor.a": []byte(`{}`), "anchor.b": []byte(`{}`), "anchor.gitea": []byte(`{"a":1}`)})
if !errors.Is(err, ErrNotHeld) || r.Written != 1 {
t.Errorf("a writer that lost the lease mid-pass did %+v, %v", r, err)
}
if got := writes(t, api, Assignments.Bucket) - before; got != 1 {
t.Errorf("writers without the lease wrote the bucket %d times", got)
}
}
// **A removal asks for the lease too** (ADR 0289 §9): a writer that held the lease when its pass began and lost
// it before its first removal purges nothing, and says it does not hold the lease.
func TestNatsARemovalWithoutTheLeaseRemovesNothing(t *testing.T) {
js, api := aMirrorBus(t)
ctx := t.Context()
held := aWriter(t, js, api, Assignments)
if _, err := held.Sync(ctx, map[string][]byte{"anchor.gitea": []byte(`{"a":1}`)}); err != nil {
t.Fatal(err)
}
before := writes(t, api, Assignments.Bucket)
asked := 0
w := aWriter(t, js, api, Assignments)
w.Holds = func() bool { asked++; return asked == 1 }
r, err := w.Sync(ctx, map[string][]byte{})
if !errors.Is(err, ErrNotHeld) || r != (Result{}) || asked != 2 {
t.Errorf("a removal without the lease did %+v, %v (the lease asked %d times)", r, err, asked)
}
if got := writes(t, api, Assignments.Bucket) - before; got != 0 {
t.Errorf("a removal without the lease wrote the bucket %d times", got)
}
if _, err := w.KV.Get(ctx, "anchor.gitea"); err != nil {
t.Errorf("the key was removed without the lease: %v", err)
}
}
// **A key the bus cannot hold is refused alone**, and the others are written.
func TestAKeyTheBusCannotHoldIsRefusedAlone(t *testing.T) {
if validKey("anchor.gitea") != nil || validKey("bad key") == nil || validKey(".lead") == nil ||
validKey("") == nil || validKey("trail.") == nil {
t.Error("the keys a bucket may hold are not told apart")
}
}
func TestTheFieldsThatChangedAreNamed(t *testing.T) {
got := changedFields([]byte(`{"a":1,"b":{"x":1},"c":"same"}`), []byte(`{"a":2,"b":{"x":1},"d":true,"c":"same"}`))
if !slices.Equal(got, []string{"a", "d"}) {
t.Errorf("the fields that changed are %v", got)
}
if got := changedFields(nil, []byte(`{"a":1}`)); got != nil {
t.Errorf("an added key names fields %v", got)
}
}
+181
View File
@@ -0,0 +1,181 @@
package recordmirror
import (
"go/ast"
"go/parser"
"go/token"
"io/fs"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/broker"
)
// mayNameAMirror are the files of the controller that may name a record mirror's bucket outside their tests:
// where the buckets are defined, asserted and granted (internal/broker), the list a reader names in `reads`
// (internal/catalogue/state.go), and each mirror's own writer (this package and the loop that runs it). The
// audit of a mirror against its store (ADR 0289 §4, issue 419) joins this list when it is built.
var mayNameAMirror = []string{
"cmd/mesh-controller/record_mirrors.go",
"internal/broker/controller_buckets.go",
"internal/broker/writers.go",
"internal/catalogue/state.go",
}
// The packages whose names stand for a mirror, by import path, and the names in each.
const (
brokerPath = "github.com/novox/mesh-controller/internal/broker"
recordmirrorPath = "github.com/novox/mesh-controller/internal/recordmirror"
)
var mirrorNames = map[string][]string{
brokerPath: {"AssignmentsBucket", "DeclaredConfigBucket", "MirrorAssignments", "MirrorDeclaredConfig",
"ControllerMirrors"},
recordmirrorPath: {"Assignments", "DeclaredConfig", "Mirrors"},
}
// **Nothing in the controller reads a mirror to decide anything** (ADR 0289 §19 as ADR 0356 §2 changes it): no
// file of the controller's packages outside its tests names a mirror's bucket but the audit and a mirror's own
// writer. A reader elsewhere fails here, naming the file and what it named.
//
// **What it catches**: a name that stands for a mirror, qualified by the package it comes from under any
// local name the file imports it as (`broker.AssignmentsBucket`, `b.AssignmentsBucket`); the same names bare
// in a file of package broker or package recordmirror; and a string literal holding a mirror's bucket name.
// **What it does not catch**: a bucket name built at run time, such as `BucketName(ControllerSeat,
// "assignments")` or literals concatenated into one, and a mirror reached through a value passed in from an
// allowed file. Those are deliberate evasions rather than ordinary code; a reviewer judges them.
func TestOnlyTheAuditAndAMirrorsWriterReadAMirror(t *testing.T) {
var buckets []string
for _, m := range broker.ControllerMirrors() {
buckets = append(buckets, broker.BucketName(broker.ControllerSeat, m))
}
root := filepath.Join("..", "..")
found := mirrorReaders(t, root, buckets)
if len(found) > 0 {
t.Fatalf("a package of the controller reads a record mirror, which only the audit and the mirror's "+
"writer may (ADR 0289 §19, ADR 0356 §2):\n %s", strings.Join(found, "\n "))
}
}
// **The reader test sees what it says it sees**: in a planted tree, each planted reader is named, and an allowed
// file and a selector on a value are not.
func TestTheReaderTestFindsPlantedReaders(t *testing.T) {
root := t.TempDir()
plant := func(path, src string) {
full := filepath.Join(root, path)
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte(src), 0o644); err != nil {
t.Fatal(err)
}
}
plant("cmd/mesh-controller/qualified.go", "package main\n\nimport \""+brokerPath+"\"\n\nvar a = broker.AssignmentsBucket\n")
plant("internal/x/aliased.go", "package x\n\nimport b \""+brokerPath+"\"\n\nvar a = b.DeclaredConfigBucket\n")
plant("internal/broker/zz_reader.go", "package broker\n\nfunc zzR() string { return AssignmentsBucket }\n")
plant("internal/inventory/literal.go", "package inventory\n\nvar a = \"mesh-controller_declared-config\"\n")
plant("internal/y/writer.go", "package y\n\nimport \""+recordmirrorPath+"\"\n\nvar a = recordmirror.Assignments\n")
// Allowed: a mirror's definition and its writer.
plant("internal/broker/controller_buckets.go", "package broker\n\nvar AssignmentsBucket = \"x\"\n")
plant("internal/broker/other.go", "package broker\n\nfunc other() string { return \"nothing\" }\n")
// A selector on a value (v.AssignmentsBucket) is no use of the mirror, but declaring a field of a mirror's
// name in package broker is named, on the safe side; and a bare use inside a call under a selector is found.
plant("internal/broker/field.go", "package broker\n\ntype s struct{ AssignmentsBucket string }\n\nfunc f(v s) string { return v.AssignmentsBucket }\n")
plant("internal/broker/nested.go", "package broker\n\nfunc g() int { return len(h(DeclaredConfigBucket).x) }\n")
found := mirrorReaders(t, root, []string{"mesh-controller_assignments", "mesh-controller_declared-config"})
want := []string{
"cmd/mesh-controller/qualified.go: broker.AssignmentsBucket",
"internal/broker/field.go: AssignmentsBucket",
"internal/broker/nested.go: DeclaredConfigBucket",
"internal/broker/zz_reader.go: AssignmentsBucket",
"internal/inventory/literal.go: \"mesh-controller_declared-config\"",
"internal/x/aliased.go: b.DeclaredConfigBucket",
"internal/y/writer.go: recordmirror.Assignments",
}
slices.Sort(found)
if !slices.Equal(found, want) {
t.Errorf("the planted readers were found as\n %s\nwant\n %s", strings.Join(found, "\n "),
strings.Join(want, "\n "))
}
}
// mirrorReaders is every place under root's cmd and internal, outside tests and the allowed files, that names a
// mirror.
func mirrorReaders(t *testing.T, root string, buckets []string) []string {
t.Helper()
var found []string
for _, dir := range []string{"cmd", "internal"} {
err := filepath.WalkDir(filepath.Join(root, dir), func(path string, e fs.DirEntry, err error) error {
if err != nil || e.IsDir() || !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") {
return err
}
rel, err := filepath.Rel(root, path)
if err != nil {
return err
}
rel = filepath.ToSlash(rel)
if strings.HasPrefix(rel, "internal/recordmirror/") || slices.Contains(mayNameAMirror, rel) {
return nil
}
f, err := parser.ParseFile(token.NewFileSet(), path, nil, 0)
if err != nil {
return err
}
// Each import's local name, mapped to the package it stands for.
imported := map[string]string{}
for _, imp := range f.Imports {
p, _ := strconv.Unquote(imp.Path.Value)
if _, ok := mirrorNames[p]; !ok {
continue
}
local := filepath.Base(p)
if imp.Name != nil {
local = imp.Name.Name
}
imported[local] = p
}
// A file of the package that defines the names may use them bare.
var bare []string
switch f.Name.Name {
case "broker":
bare = mirrorNames[brokerPath]
case "recordmirror":
bare = mirrorNames[recordmirrorPath]
}
ast.Inspect(f, func(n ast.Node) bool {
switch n := n.(type) {
case *ast.SelectorExpr:
pkg, ok := n.X.(*ast.Ident)
if !ok {
return true
}
if p, imp := imported[pkg.Name]; imp && slices.Contains(mirrorNames[p], n.Sel.Name) {
found = append(found, rel+": "+pkg.Name+"."+n.Sel.Name)
}
// Neither a selector's own name nor its package is a bare use.
return false
case *ast.Ident:
if slices.Contains(bare, n.Name) {
found = append(found, rel+": "+n.Name)
}
case *ast.BasicLit:
for _, b := range buckets {
if n.Kind == token.STRING && strings.Contains(n.Value, b) {
found = append(found, rel+": "+n.Value)
}
}
}
return true
})
return nil
})
if err != nil {
t.Fatal(err)
}
}
return found
}
+1 -1
View File
@@ -8,7 +8,7 @@ private addresses. That is acceptable while mesh-controller is private; if it ev
a second place to clean besides the catalogue itself.
mesh-catalog e0bc919d4945bb5f13f971f74643a0b714cf45da modules/*/module.json, modules/nats/Dockerfile
mesh-host 7a41e120635b3f7b6cb94cca8b3fc4093e4e6110 examples/foundation-first-node.lock
mesh-host c8920622516548978d92e734f3991a262f98a742 examples/foundation-first-node.lock
mesh-sdk 5933eb757d443293631be2bed27a96288c2348d8 conformance/events/module-event.json
To move them, from this repository's root, with the two repositories checked out beside it:
+79 -29
View File
@@ -28,11 +28,12 @@
// image, and this file is applied on a machine with no mesh to ask about anything (ADR 0006). The
// registry prefix is the lab's; the digests are upstream's.
//
// The store's data is a NAMED VOLUME, not a directory on the machine. A directory the host
// creates is owned by root, and the database runs as somebody else inside the container — so it
// could not write, and the container crash-looped. A named volume lets the image set up its own
// ownership, and outlives the container, which is what you want for the thing holding the mesh's
// state.
// **The store is raised where the `postgres` module keeps it** (novox/hq ADR 0338): the container
// `postgres`, on the module's pgvector image, its data in the directory /var/lib/mesh-store owned by
// the image's postgres user, its superuser password read from the module's own-secret file — so the
// store step adopts it as it runs. Pinned upstream by the module's own digest, not through the lab's
// registry prefix, because the two must be the same reference (mesh-host internal/bootstrap holds
// them to each other against the module's manifest).
{
"declaration": 1,
@@ -83,17 +84,28 @@
"boot": "enabled",
"restart-on": ["base-filter"]
},
// The store's data is a directory owned by the image's postgres user (999), made before the
// container starts: a directory the runtime made would be root's, and the database could not
// write it. Measured on the module's image 2026-10-13: it initialises a new store on a directory
// made so (novox/hq ADR 0338 §5).
{
"id": "store-data",
"type": "directory",
"path": "/var/lib/mesh-store",
"mode": "0700",
"owner": "999:70"
},
{
"id": "store",
"type": "container",
"name": "mesh-store",
"image": "192.0.2.250:5000/postgres@sha256:7abf537131b66ed5af448d90653abf1679b0c7e9a1f07efdd4c3108a401b259a",
"name": "postgres",
"image": "pgvector/pgvector@sha256:cf134a767f474095eeba57e0117be8e568e011a63f33fbf252f14c9b760f8e6f",
"env": {
"POSTGRES_PASSWORD": "bootstrap",
"PGDATA": "/var/lib/postgresql/data/pgdata"
},
"ports": ["5432:5432"],
"volumes": ["mesh-store-data:/var/lib/postgresql/data"]
"volumes": ["/var/lib/mesh-store:/var/lib/postgresql/data"]
},
// Over TCP, not the socket. While the store initialises it runs a temporary server on the
// socket ONLY, then stops it and starts the real one — so a socket check passes, the action
@@ -102,21 +114,21 @@
{
"id": "store-ready",
"type": "action",
"in": "mesh-store",
"in": "postgres",
"command": ["sh", "-c", "for i in $(seq 1 180); do pg_isready -h 127.0.0.1 -U postgres >/dev/null 2>&1 && exit 0; sleep 1; done; echo 'the store did not answer within 180s; its own last words follow'; pg_isready -h 127.0.0.1 -U postgres; tail -n 20 /var/lib/postgresql/data/log/*.log 2>/dev/null; exit 1"],
"verify": ["pg_isready", "-h", "127.0.0.1", "-U", "postgres"]
},
{
"id": "inventory-database",
"type": "action",
"in": "mesh-store",
"in": "postgres",
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE inventory'"],
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw inventory"]
},
{
"id": "identity-database",
"type": "action",
"in": "mesh-store",
"in": "postgres",
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE identity'"],
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw identity"]
},
@@ -126,20 +138,44 @@
{
"id": "licences-database",
"type": "action",
"in": "mesh-store",
"in": "postgres",
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE licences'"],
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw licences"]
},
{
"id": "context-schemas",
"type": "action",
"command": ["docker", "run", "--rm", "--network", "container:mesh-store",
"command": ["docker", "run", "--rm", "--network", "container:postgres",
"-e", "MESH_STORE_INVENTORY=postgres://postgres:bootstrap@127.0.0.1:5432/inventory?sslmode=disable",
"-e", "MESH_STORE_IDENTITY=postgres://postgres:bootstrap@127.0.0.1:5432/identity?sslmode=disable",
"-e", "MESH_STORE_LICENCES=postgres://postgres:bootstrap@127.0.0.1:5432/licences?sslmode=disable",
"192.0.2.250:5000/mesh-controller@sha256:c67db38439ff0aee242b467486765467bb95801f52175fc5727cc4e437338ace",
"migrate"],
"verify": ["sh", "-c", "docker exec mesh-store psql -U postgres -d inventory -tAc \"select to_regclass('public.node')\" | grep -qx node && docker exec mesh-store psql -U postgres -d identity -tAc \"select to_regclass('public.signing_key')\" | grep -qx signing_key && docker exec mesh-store psql -U postgres -d licences -tAc \"select to_regclass('public.licence')\" | grep -qx licence"]
"verify": ["sh", "-c", "docker exec postgres psql -U postgres -d inventory -tAc \"select to_regclass('public.node')\" | grep -qx node && docker exec postgres psql -U postgres -d identity -tAc \"select to_regclass('public.signing_key')\" | grep -qx signing_key && docker exec postgres psql -U postgres -d licences -tAc \"select to_regclass('public.licence')\" | grep -qx licence"]
},
// **The bus is raised where the `nats` module keeps it** (novox/hq ADR 0333 §1): the container
// `mesh-broker-nats`, its configuration and user list in /var/lib/nats-module/conf, its streams in
// /var/lib/mesh-broker-nats and its certificate in /var/lib/mesh-broker-tls — the names and paths the
// catalogue's module declares. So adopting it as that module at the broker step keeps it: the
// module's built image takes the place of the upstream one below, once, on the same streams, users
// and certificate, and nothing else moves. mesh-host's internal/bootstrap holds the two to each other
// against the module's own manifest (busLayout), at the broker step and in its tests.
//
// Directories on the machine, not volumes the runtime names: the module mounts these paths, and the
// control plane, once it runs as the mesh's process, reads the certificate at its path on the machine.
// The certificate's directory is readable by everyone because the certificate is public and the control
// plane runs as nobody; its key is written 0600 by the program that makes it.
{
"id": "bus-tls-dir",
"type": "directory",
"path": "/var/lib/mesh-broker-tls",
"mode": "0755"
},
{
"id": "bus-data-dir",
"type": "directory",
"path": "/var/lib/mesh-broker-nats",
"mode": "0700"
},
{
"id": "bus-certificate",
@@ -151,49 +187,63 @@
// it, and asking nothing of the image it writes into. Self-signed on purpose: a host pins
// this server's exact certificate (novox/hq ADR 0004), and at this moment there is no mesh
// to ask an authority of.
// `--user 0:0` because the volume is root's and this image runs as nobody, which is right
// for the long-running control plane and wrong for a one-shot writing into a fresh volume.
"command": ["docker", "run", "--rm", "--user", "0:0", "-v", "mesh-broker-tls:/tls",
// `--user 0:0` because the directory is root's and this image runs as nobody, which is right
// for the long-running control plane and wrong for a one-shot writing into a fresh directory.
"command": ["docker", "run", "--rm", "--user", "0:0", "-v", "/var/lib/mesh-broker-tls:/tls",
"192.0.2.250:5000/mesh-controller@sha256:c67db38439ff0aee242b467486765467bb95801f52175fc5727cc4e437338ace",
"broker", "certificate", "--into", "/tls"],
"verify": ["docker", "run", "--rm", "--user", "0:0", "-v", "mesh-broker-tls:/tls",
"verify": ["docker", "run", "--rm", "--user", "0:0", "-v", "/var/lib/mesh-broker-tls:/tls",
"192.0.2.250:5000/mesh-controller@sha256:c67db38439ff0aee242b467486765467bb95801f52175fc5727cc4e437338ace",
"broker", "certificate", "--check", "--into", "/tls"]
},
// **Seeds, written only where nothing is** (create-once). The bus's configuration and user list are
// the bundle's until the bus is adopted, and the module's from then on (ADR 0333 §4): an installer
// run again after the adoption applies this bundle again, and an ordinary file would put the
// bundle's one user back over every user the mesh composed, and leave the node unable to reach its
// bus. Until the adoption the installer places the composed list over the seed (enrol.go).
{
"id": "bus-conf-dir",
"type": "directory",
"path": "/var/lib/mesh-bus-conf",
"path": "/var/lib/nats-module/conf",
"mode": "0700"
},
{
"id": "bus-conf",
"type": "file",
"path": "/var/lib/mesh-bus-conf/nats.conf",
"path": "/var/lib/nats-module/conf/nats.conf",
"mode": "0644",
"content": "port: 4222\nhttp: 127.0.0.1:8222\n\ntls {\n cert_file: \"/tls/tls.crt\"\n key_file: \"/tls/tls.key\"\n}\n\njetstream {\n store_dir: \"/data\"\n}\n\ninclude accounts.conf\n"
"create-once": true,
"content": "port: 4222\nhttp: 0.0.0.0:8222\n\ntls {\n cert_file: \"/tls/tls.crt\"\n key_file: \"/tls/tls.key\"\n}\n\njetstream {\n store_dir: \"/data\"\n}\n\ninclude accounts.conf\n"
},
{
"id": "bus-accounts",
"type": "file",
"path": "/var/lib/mesh-bus-conf/accounts.conf",
"path": "/var/lib/nats-module/conf/accounts.conf",
"mode": "0600",
"content": "// The first user list, carried by the installer because at genesis there is no mesh to\n// compose one. A bootstrap credential, rotated with the store's and replaced by the\n// controller's own composition from its first start onward.\naccounts {\n MESH {\n jetstream: enabled\n users = [\n { user: \"controller\", password: \"$2a$10$AHqJgOifIVbU41KmATiMhuXFs8xa7Wl2HuN4UVBCXdN2jIQzjqApy\", permissions: {\n publish: { allow: [\"$JS.ACK.CONTROL.controller.>\", \"$JS.ACK.EVENTS.controller.>\", \"$JS.ACK.DEAD_LETTER_NOTICES.controller.>\", \"$JS.API.>\", \"$KV.mesh-controller_asked.>\", \"$KV.mesh-controller_calls.>\", \"$KV.mesh-controller_hand-acts.>\", \"$KV.mesh-controller_conditions.>\", \"$KV.mesh-controller_condition-history.>\", \"$KV.mesh-controller_lease.>\", \"$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>\", \"$KV.SEAT_NODE_BUILD_AGENT_cancelled.>\", \"_INBOX.enrol.>\", \"mesh.assignment.>\", \"mesh.mod.*.tool.>\", \"mesh.node.>\", \"mesh.seat.mesh-build-machine.accept.>\", \"mesh.seat.node-build-agent.accept.>\", \"mesh.seat.mesh-build-machine.tool.>\", \"mesh.seat.node-build-agent.tool.>\", \"mesh.seat.mesh-controller.event.applied\", \"mesh.seat.mesh-controller.event.built-before\", \"mesh.seat.mesh-controller.event.refused\", \"mesh.seat.mesh-controller.event.rolled-back\", \"mesh.seat.mesh-controller.event.checked\", \"mesh.seat.mesh-controller.event.plan-moved\", \"mesh.seat.mesh-controller.event.condition-raised\", \"mesh.seat.mesh-controller.event.condition-changed\", \"mesh.seat.mesh-controller.event.condition-cleared\", \"mesh.seat.mesh-controller.event.doctor-heartbeat\", \"mesh.seat.mesh-controller.event.secret-replaced\", \"mesh.seat.mesh-controller.event.healer-acted\", \"$SRV.INFO\", \"mesh.seat.node-intrusion-prevention.tool.banned.*\", \"mesh.seat.node-backup.tool.backed-up.*\", \"mesh.seat.node-backup.tool.now.*\", \"mesh.seat.mesh-delivery.tool.stalled\", \"mesh.seat.mesh-delivery.tool.close\", \"mesh.seat.node-launcher.tool.secret.*\", \"mesh.seat.mesh-delivery.tool.release\", \"mesh.seat.mesh-delivery.tool.stop\", \"mesh.seat.mesh-delivery.tool.times\", \"mesh.seat.mesh-delivery.tool.groups\", \"mesh.seat.node-service-manager.tool.restart.*\", \"mesh.seat.mesh-controller.tool.plans\", \"mesh.events.dead.>\", \"mesh.again.>\"] }\n subscribe: { allow: [\"$JS.API.>\", \"$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>\", \"$JS.EVENT.ADVISORY.CONSUMER.DELETED.>\", \"_DELIVER.controller\", \"_DELIVER.controller.>\", \"_INBOX.controller.>\", \"mesh.control.>\", \"mesh.mod.*.event.provisioner.failing\", \"mesh.mod.*.event.provisioner.recovered\", \"mesh.mod.*.event.provisioner.retirement\", \"mesh.mod.gitea.event.pull.merged\", \"mesh.mod.gitea.event.pull.updated\", \"mesh.mod.mesh-catalog.event.catching-up\", \"mesh.mod.mesh-catalog.event.upgraded\", \"mesh.seat.mesh-build-machine.event.built\", \"mesh.seat.node-build-agent.event.built\", \"mesh.seat.operator-channel.event.decided.mesh-controller\", \"mesh.seat.mesh-controller.tool.>\", \"$SRV.PING\", \"$SRV.INFO\", \"$SRV.PING.mesh-controller\", \"$SRV.PING.mesh-controller.>\", \"$SRV.INFO.mesh-controller\", \"$SRV.INFO.mesh-controller.>\", \"$SRV.STATS\", \"$SRV.STATS.mesh-controller\", \"$SRV.STATS.mesh-controller.>\"] }\n allow_responses: { max: 1, ttl: \"1m\" }\n } }\n ]\n }\n}\n"
"create-once": true,
"content": "// The first user list, carried by the installer because at genesis there is no mesh to\n// compose one. A bootstrap credential, rotated with the store's and replaced by the\n// controller's own composition from its first start onward.\naccounts {\n MESH {\n jetstream: enabled\n users = [\n { user: \"controller\", password: \"$2a$10$AHqJgOifIVbU41KmATiMhuXFs8xa7Wl2HuN4UVBCXdN2jIQzjqApy\", permissions: {\n publish: { allow: [\"$JS.ACK.CONTROL.controller.>\", \"$JS.ACK.EVENTS.controller.>\", \"$JS.ACK.DEAD_LETTER_NOTICES.controller.>\", \"$JS.API.>\", \"$KV.mesh-controller_asked.>\", \"$KV.mesh-controller_assignments.>\", \"$KV.mesh-controller_calls.>\", \"$KV.mesh-controller_hand-acts.>\", \"$KV.mesh-controller_conditions.>\", \"$KV.mesh-controller_condition-history.>\", \"$KV.mesh-controller_declared-config.>\", \"$KV.mesh-controller_lease.>\", \"$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>\", \"$KV.SEAT_NODE_BUILD_AGENT_cancelled.>\", \"_INBOX.enrol.>\", \"mesh.assignment.>\", \"mesh.mod.*.tool.>\", \"mesh.node.>\", \"mesh.seat.mesh-build-machine.accept.>\", \"mesh.seat.node-build-agent.accept.>\", \"mesh.seat.mesh-build-machine.tool.>\", \"mesh.seat.node-build-agent.tool.>\", \"mesh.seat.mesh-controller.event.applied\", \"mesh.seat.mesh-controller.event.built-before\", \"mesh.seat.mesh-controller.event.refused\", \"mesh.seat.mesh-controller.event.rolled-back\", \"mesh.seat.mesh-controller.event.checked\", \"mesh.seat.mesh-controller.event.plan-moved\", \"mesh.seat.mesh-controller.event.assignment-changed\", \"mesh.seat.mesh-controller.event.declared-config-changed\", \"mesh.seat.mesh-controller.event.condition-raised\", \"mesh.seat.mesh-controller.event.condition-changed\", \"mesh.seat.mesh-controller.event.condition-cleared\", \"mesh.seat.mesh-controller.event.doctor-heartbeat\", \"mesh.seat.mesh-controller.event.secret-replaced\", \"mesh.seat.mesh-controller.event.healer-acted\", \"$SRV.INFO\", \"mesh.seat.node-intrusion-prevention.tool.banned.*\", \"mesh.seat.node-backup.tool.backed-up.*\", \"mesh.seat.node-backup.tool.now.*\", \"mesh.seat.mesh-delivery.tool.stalled\", \"mesh.seat.mesh-delivery.tool.close\", \"mesh.seat.node-launcher.tool.secret.*\", \"mesh.seat.mesh-delivery.tool.release\", \"mesh.seat.mesh-delivery.tool.stop\", \"mesh.seat.mesh-delivery.tool.times\", \"mesh.seat.mesh-delivery.tool.groups\", \"mesh.seat.node-service-manager.tool.restart.*\", \"mesh.seat.mesh-controller.tool.plans\", \"mesh.events.dead.>\", \"mesh.again.>\"] }\n subscribe: { allow: [\"$JS.API.>\", \"$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>\", \"$JS.EVENT.ADVISORY.CONSUMER.DELETED.>\", \"_DELIVER.controller\", \"_DELIVER.controller.>\", \"_INBOX.controller.>\", \"mesh.control.>\", \"mesh.mod.*.event.provisioner.failing\", \"mesh.mod.*.event.provisioner.recovered\", \"mesh.mod.*.event.provisioner.retirement\", \"mesh.mod.gitea.event.pull.merged\", \"mesh.mod.gitea.event.pull.updated\", \"mesh.mod.mesh-catalog.event.catching-up\", \"mesh.mod.mesh-catalog.event.upgraded\", \"mesh.seat.mesh-build-machine.event.built\", \"mesh.seat.node-build-agent.event.built\", \"mesh.seat.operator-channel.event.decided.mesh-controller\", \"mesh.seat.mesh-controller.tool.>\", \"$SRV.PING\", \"$SRV.INFO\", \"$SRV.PING.mesh-controller\", \"$SRV.PING.mesh-controller.>\", \"$SRV.INFO.mesh-controller\", \"$SRV.INFO.mesh-controller.>\", \"$SRV.STATS\", \"$SRV.STATS.mesh-controller\", \"$SRV.STATS.mesh-controller.>\"] }\n allow_responses: { max: 1, ttl: \"1m\" }\n } }\n ]\n }\n}\n"
},
// The upstream server, until the broker step adopts it: the `nats` module's own image, built by the
// mesh, takes its place there once, on these same directories (ADR 0333 §3), as the bus's planned
// step (ADR 0236). The release is the one the module's image is built on (its NATS_BASE, 2.11.17),
// pinned upstream by that same digest: the adoption changes no server release, so it can be undone
// by putting the old image back and is said reversible. The published port is the
// node's bus port, 5671 unless given, which every enrolment token names; the installer gives the
// module the setting that maps its declared 4222 to it (ADR 0333 §2).
{
"id": "broker",
"type": "container",
"name": "mesh-broker",
"image": "192.0.2.250:5000/nats@sha256:b83efabe3e7def1e0a4a31ec6e078999bb17c80363f881df35edc70fcb6bb927",
"name": "mesh-broker-nats",
"image": "nats@sha256:e4bf19f15fd3218814a4e3c9e0064e1334bd8aa20d5984b9f1a0afd084f8cc00",
"ports": ["5671:4222", "127.0.0.1:8222:8222"],
"volumes": ["mesh-broker-data:/data", "mesh-broker-tls:/tls:ro", "/var/lib/mesh-bus-conf:/etc/nats:ro"],
"volumes": ["/var/lib/mesh-broker-nats:/data", "/var/lib/nats-module/conf:/etc/nats:ro", "/var/lib/mesh-broker-tls:/tls:ro"],
"args": ["-c", "/etc/nats/nats.conf", "-js"]
},
{
"id": "broker-ready",
"type": "action",
"command": ["sh", "-c", "for i in $(seq 1 60); do docker run --rm --network host --entrypoint sh 192.0.2.250:5000/nats@sha256:b83efabe3e7def1e0a4a31ec6e078999bb17c80363f881df35edc70fcb6bb927 -c 'nc -z 127.0.0.1 5671' >/dev/null 2>&1 && exit 0; sleep 1; done; exit 1"],
"verify": ["sh", "-c", "docker run --rm --network host --entrypoint sh 192.0.2.250:5000/nats@sha256:b83efabe3e7def1e0a4a31ec6e078999bb17c80363f881df35edc70fcb6bb927 -c 'nc -z 127.0.0.1 5671'"]
"command": ["sh", "-c", "for i in $(seq 1 60); do docker run --rm --network host --entrypoint sh nats@sha256:e4bf19f15fd3218814a4e3c9e0064e1334bd8aa20d5984b9f1a0afd084f8cc00 -c 'nc -z 127.0.0.1 5671' >/dev/null 2>&1 && exit 0; sleep 1; done; exit 1"],
"verify": ["sh", "-c", "docker run --rm --network host --entrypoint sh nats@sha256:e4bf19f15fd3218814a4e3c9e0064e1334bd8aa20d5984b9f1a0afd084f8cc00 -c 'nc -z 127.0.0.1 5671'"]
},
{
"id": "control-plane",
@@ -202,7 +252,7 @@
"image": "192.0.2.250:5000/mesh-controller@sha256:c67db38439ff0aee242b467486765467bb95801f52175fc5727cc4e437338ace",
"network": "host",
"args": ["serve"],
"volumes": ["mesh-broker-tls:/broker-tls:ro"],
"volumes": ["/var/lib/mesh-broker-tls:/broker-tls:ro"],
"env": {
"MESH_STORE_INVENTORY": "postgres://postgres:bootstrap@127.0.0.1:5432/inventory?sslmode=disable",
"MESH_STORE_IDENTITY": "postgres://postgres:bootstrap@127.0.0.1:5432/identity?sslmode=disable",
+1 -3
View File
@@ -49,6 +49,7 @@
./internal/inventory TestACredentialIssuedOutsideIsNeverMade
./internal/inventory TestOneAcceptForAProviderIsSealedToEveryConsumer
./internal/inventory TestASharedOwnSecretIssuedOutsideIsNeverMade
./internal/recordmirror TestNatsAnAssignmentAndARegistrationEachChangeOneKeyAndRaiseOneEvent
./cmd/mesh-controller TestOneAcceptForAProviderIsPlacedForEveryConsumer
./cmd/mesh-controller TestReplay164
./cmd/mesh-controller TestAnApprovedLayerOpensTheWalkThatSendsIt
@@ -61,6 +62,3 @@
./cmd/mesh-controller TestAProviderElsewhereServesItsOwnComposedName
./cmd/mesh-controller TestAReusedCheckBuildIsRegisteredByItsDigestAndGatedAsABuiltOne
./internal/inventory TestAReportOfABusyContainerIsKeptAndReadBack
./internal/inventory TestPlacementMovesOutOfTheSettingsLayers
./internal/inventory TestASettingRoutesPlacementBesideTheLayer
./cmd/mesh-controller TestADeclarationIsTheSameAfterPlacementMoves