novox/mesh-controller
6
0
Fork 0
Code Issues Pull Requests 3 Releases Wiki Activity

Compare commits

merge into: novox/mesh-controller:feat/no-change-takes-effect-unseen
Branches Tags
novox/mesh-controller:main novox/mesh-controller:fix/missed-merges-are-caught-up novox/mesh-controller:fix/a-verb-answers-before-its-caller-gives-up novox/mesh-controller:fix/the-registry-trust-test-reads-the-runtimes-module novox/mesh-controller:fix/the-runtime-file-is-the-runtimes novox/mesh-controller:feat/no-change-takes-effect-unseen novox/mesh-controller:feat/build-edges-are-recorded novox/mesh-controller:fix/several-contributions-collide-in-grants-v2
..
pull from: novox/mesh-controller:fix/issue-211-a-bundle-stands-on-its-toolchain
Branches Tags
novox/mesh-controller:fix/missed-merges-are-caught-up novox/mesh-controller:fix/a-verb-answers-before-its-caller-gives-up novox/mesh-controller:main novox/mesh-controller:fix/the-registry-trust-test-reads-the-runtimes-module novox/mesh-controller:fix/the-runtime-file-is-the-runtimes novox/mesh-controller:feat/no-change-takes-effect-unseen novox/mesh-controller:feat/build-edges-are-recorded novox/mesh-controller:fix/several-contributions-collide-in-grants-v2
1 Commits
feat/no-change-takes-effect-unseen .. fix/issue-211-a-bundle-stands-on-its-toolchain
Author SHA1 Message Date
jochen c72f6ca8cb A bundle stands on the toolchain it is compiled in (hq issue 211) 
A manifest names its toolchain by language, not in build.on, so the planner did not know a bundle
depends on the module that publishes its toolchain and built the two in one tier: the bundle
against the old toolchain, recorded as built from the new commit. The edge is read from the
manifest, so it holds before any build recorded it, and a toolchain that moves rebuilds every
bundle compiled in it.
 2026-10-03 22:19:50 +02:00
138 changed files with 513 additions and 12908 deletions
Expand all files Collapse all files
Dockerfile
+2 -8
View File
@@ -1,11 +1,5 @@
# The Go it builds with, pinned here because genesis builds this file with no arguments (novox/hq
# issue 223) — the Makefile passes the same digest. A tag older than go.mod asks for is how
# `make image` broke once before (issue 146).
ARG GO_BASE=golang@sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7236c349a0c
# The control plane's image — for genesis and the lab only. The mesh runs the controller as a Go
# bundle the host starts as a process (module.json; novox/hq issue 213), and builds no image of it.
# Genesis builds this file and raises it as the container the process replaces on the first push
# (mesh-host internal/bootstrap, novox/hq issue 223).
ARG GO_BASE=golang:1.25-alpine
# The control plane's image.
#
# novox/hq ADR 0006: this image is pinned by digest in the bundle the host carries, fetched on a
# machine where no mesh exists yet, and run before there is anything to check it against. So it
Makefile
+4 -8
View File
@@ -27,21 +27,17 @@ build:
IMAGE ?= mesh-controller:$(VERSION)
DEV_TAG ?= mesh-controller:development
# The Go base the image is built on.
# The base the module declares, read from the manifest rather than written here twice.
#
# **`make image` was broken and stayed broken**, because the Dockerfile's fallback base was a Go
# older than go.mod asks for: every build died at `go mod download` with "go.mod requires go >=
# 1.26.0", and the pipeline never saw it because the pipeline passes the declared base in. Anybody
# building the image by hand hit it and had to find the digest themselves (novox/hq 04-ISSUES/146,
# what it cost).
#
# **Pinned here since the manifest stopped building an image** (novox/hq issue 213): the mesh builds
# the controller as a Go bundle with its own toolchain, and only `make image` — genesis and the lab —
# still needs a Go base. The digest is the one the manifest declared until then.
GO_BASE ?= golang@sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7236c349a0c
GO_BASE ?= $(shell python3 -c "import json;print(next(o['image'] for o in json.load(open('module.json'))['build']['on'] if o['arg']=='GO_BASE'))" 2>/dev/null)
image:
@test -n "$(GO_BASE)" || { echo "no GO_BASE; pass GO_BASE=<image>"; exit 1; }
@test -n "$(GO_BASE)" || { echo "module.json declares no GO_BASE; pass GO_BASE=<image> or fix the manifest"; exit 1; }
docker build --build-arg GO_BASE=$(GO_BASE) --build-arg VERSION=$(VERSION) -t $(IMAGE) -t $(DEV_TAG) .
@echo
@docker image inspect $(IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
@@ -52,7 +48,7 @@ BUILDER_IMAGE ?= mesh-builder:$(VERSION)
BUILDER_DEV_TAG ?= mesh-builder:development
builder-image:
@test -n "$(GO_BASE)" || { echo "no GO_BASE; pass GO_BASE=<image>"; exit 1; }
@test -n "$(GO_BASE)" || { echo "module.json declares no GO_BASE; pass GO_BASE=<image> or fix the manifest"; exit 1; }
docker build --build-arg GO_BASE=$(GO_BASE) -f cmd/mesh-builder/Dockerfile -t $(BUILDER_IMAGE) -t $(BUILDER_DEV_TAG) .
@echo
@docker image inspect $(BUILDER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
README.md
-7
View File
@@ -193,13 +193,6 @@ passes every check that only looks at the message.
## The image
**The mesh no longer runs the controller from it** (novox/hq issue 213). The module declares a Go
bundle, `controller`, which the host on the controller's machine unpacks and runs as the process
`mesh-controller` under the account of the same name (ADR 0188 §1, §3). The image stays for what
still runs a container of the controller: genesis, which raises the first controller from it and
installs the module from its manifest (mesh-host `internal/bootstrap`), and the lab. Neither is the
mesh's own build any more — `make image` builds it.
`FROM scratch`, holding one statically linked binary and nothing else — no shell, no package
manager, no libc, no CA certificates.
cmd/mesh-builder/main.go
+1 -1
View File
@@ -171,7 +171,7 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
result := link.BuildResult{
ID: request.ID, Repository: request.Repository, Path: request.Path,
Ref: request.Ref, On: on, Source: request.Source, DryRun: request.DryRun,
Ref: request.Ref, On: on, Source: request.Source,
}
what := "building " + request.Repository
if request.Path != "" {
cmd/mesh-controller/acts.go
+22 -147
View File
@@ -4,7 +4,6 @@ import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/broker"
"slices"
"sort"
"strings"
@@ -40,10 +39,7 @@ import (
//
// It costs a resolution per machine. Assignment is a person typing a command, and being told which
// machines this just blocked is worth more than the milliseconds.
func assign(ctx context.Context, open *stores, node string, modules ...string) (string, error) {
if len(modules) == 0 {
return "", fmt.Errorf("assign %s names no module", node)
}
func assign(ctx context.Context, open *stores, node, module string) (string, error) {
// Held while it is recorded, so it cannot land between a converge's preview and its flip and
// be taken without ever having been previewed (novox/hq ADR 0100).
ctx, release, err := holdNodes(ctx, open, []string{node})
@@ -51,16 +47,6 @@ func assign(ctx context.Context, open *stores, node string, modules ...string) (
return "", err
}
defer release()
// **The one assignment refused for what the node lacks** (novox/hq ADR 0207). Everything else
// an assignment leaves unresolved is kept, because assignment is not an ordering; a module whose
// resources are applied through a seat nothing on the node holds is refused, because that order
// — the service manager, the package manager and the runtime before anything that installs,
// runs or contains — is the mesh's to keep. Several modules in one act are judged together, so
// holders that depend on each other go on in one command.
shelf, before, err := seatDependenciesOnAssign(ctx, open, node, modules)
if err != nil {
return "", err
}
// **Before the new assignment can unsettle a seat somebody holds only by being alone**
// (novox/hq 04-ISSUES/170): what the mesh derived so far is written down, and then the
// assignment resolves against a record rather than against a coincidence.
@@ -68,176 +54,65 @@ func assign(ctx context.Context, open *stores, node string, modules ...string) (
if err != nil {
return "", err
}
var lines []string
var added []string
for _, module := range modules {
fresh, err := open.inventory.Assign(ctx, node, module)
if err != nil {
return strings.Join(lines, "\n"), err
}
if !fresh {
// Nothing changed, and saying "is assigned" would read as an action. One node runs one
// of each — the module's name is the assignment's identity (novox/hq ADR 0115).
lines = append(lines, fmt.Sprintf(
"%s already runs %s — one node runs one of each (ADR 0115); nothing changed", node, module))
continue
}
added = append(added, module)
lines = append(lines, fmt.Sprintf("%s is assigned %s", node, module))
fresh, err := open.inventory.Assign(ctx, node, module)
if err != nil {
return "", err
}
if len(added) == 0 {
return strings.Join(lines, "\n"), nil
if !fresh {
// Nothing changed, and saying "is assigned" would read as an action. One node runs one
// of each — the module's name is the assignment's identity (novox/hq ADR 0115).
return fmt.Sprintf("%s already runs %s — one node runs one of each (ADR 0115); nothing changed",
node, module), nil
}
answer := strings.Join(lines, "\n")
said := fmt.Sprintf("%s is assigned %s", node, module)
for _, line := range settled {
answer += "\n " + line
}
// What this act changed about this node's unmet seat dependencies, and nothing else (novox/hq
// ADR 0207): a dependency of a module just assigned, or one this assignment met. The rest of the
// node's list, and every other node's, is `status`'s.
for _, line := range unheldChange(shelf, node, before, append(append([]string(nil), before...), added...)) {
answer += "\n " + line
said += "\n " + line
}
// Its bus credential, in the same act (novox/hq issue 203): an assignment pushed before its
// credential exists delivers a process that cannot authenticate and crash-loops until somebody
// runs a second verb and a second push. Issued here when the module speaks on the bus and has
// no credential yet; kept when it has one, so re-assigning rotates nothing.
for _, module := range added {
if line := issueOnAssign(ctx, open, node, module); line != "" {
answer += "\n " + line
}
if line := issueOnAssign(ctx, open, node, module); line != "" {
said += "\n " + line
}
plan, _, err := planFor(ctx, open, node)
if err != nil {
// Kept, and still refused. Both halves are the answer, and the rest of the mesh is still
// worth reporting: this machine's refusal is rarely the only consequence.
return answer + blockedElsewhere(ctx, open, node), err
return said + blockedElsewhere(ctx, open, node), err
}
// Kept, and cannot be hosted here. Said at once rather than discovered at push: a module whose
// capability the machine lacks is on the wrong machine, and the assignment records what a person
// meant while this line says it will not run until it moves. The rest of the node still pushes.
isAdded := map[string]bool{}
for _, m := range added {
isAdded[m] = true
}
for _, u := range plan.Unhostable {
if !isAdded[u.Module] {
if u.Module != module {
continue
}
for _, c := range u.Missing {
answer += "\n but " + catalogue.WrongMachine(u.Module, c, node)
said += "\n but " + catalogue.WrongMachine(u.Module, c, node)
}
}
return answer + fmt.Sprintf("\n run `push %s` to send it", node) +
return said + fmt.Sprintf("\n run `push %s` to send it", node) +
blockedElsewhere(ctx, open, node), nil
}
// seatDependenciesOnAssign is the refusal ADR 0207 makes at assignment, or nothing, with the
// catalogue and the node's assignments it was judged against. Modules already assigned are not new
// and are not judged again.
func seatDependenciesOnAssign(ctx context.Context, open *stores, node string, modules []string) (
map[string]catalogue.Manifest, []string, error) {
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return nil, nil, err
}
assigned, err := open.inventory.Assigned(ctx, node)
if err != nil {
return nil, nil, err
}
already := map[string]bool{}
for _, a := range assigned {
already[a] = true
}
var adding []string
for _, m := range modules {
if !already[m] {
adding = append(adding, m)
}
}
// The lines AssignRefusal says beside an assignment it lets through are said by unheldChange
// with everything else this act changed, so they are not said twice.
if _, err := catalogue.AssignRefusal(shelf, node, assigned, adding); err != nil {
return nil, nil, err
}
// Two modules declaring one package, path or unit is refused before anything is recorded
// (novox/hq ADR 0210, 04-ISSUES/235): kept, the node would not resolve until one came off again.
if err := catalogue.CollisionRefusal(shelf, node, assigned, adding); err != nil {
return nil, nil, err
}
return shelf, assigned, nil
}
// unassign takes modules off a node. What they leave behind is the host's business: a directory
// unassign takes a module off a node. What it leaves behind is the host's business: a directory
// holding anything the mesh did not put there is kept (novox/hq ADR 0030).
//
// It reports the rest of the mesh for the same reason assign does, and more sharply: taking a
// module off one machine is the ordinary way to stop providing something to another, and nothing
// about the command's own output would ever have said so.
//
// **Refused when it takes away the last holder of a seat a module left on the node depends on**
// (novox/hq ADR 0207) — the other side of refusing that module's assignment without one. Several
// modules in one act are judged together, so a holder and its dependents come off in one command.
func unassign(ctx context.Context, open *stores, node string, modules ...string) (string, error) {
if len(modules) == 0 {
return "", fmt.Errorf("unassign %s names no module", node)
}
func unassign(ctx context.Context, open *stores, node, module string) (string, error) {
ctx, release, err := holdNodes(ctx, open, []string{node})
if err != nil {
return "", err
}
defer release()
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
if err := open.inventory.Unassign(ctx, node, module); err != nil {
return "", err
}
assigned, err := open.inventory.Assigned(ctx, node)
if err != nil {
return "", err
}
// Every one checked before any is taken off, so a refusal leaves the node as it was.
runs := map[string]bool{}
for _, a := range assigned {
runs[a] = true
}
for _, module := range modules {
if !runs[module] {
return "", fmt.Errorf("%s is not assigned to %s", module, node)
}
}
if err := catalogue.UnassignRefusal(shelf, node, assigned, modules); err != nil {
return "", err
}
for _, module := range modules {
if err := open.inventory.Unassign(ctx, node, module); err != nil {
return "", err
}
}
answer := fmt.Sprintf("%s no longer runs %s — run `push %s` to make it so",
node, strings.Join(modules, ", "), node)
var left []string
for _, a := range assigned {
if !slices.Contains(modules, a) {
left = append(left, a)
}
}
for _, line := range unheldChange(shelf, node, assigned, left) {
answer += "\n " + line
}
return answer + blockedElsewhere(ctx, open, node), nil
}
// splitModules is a surface's one `module` field as the modules it names: several, comma-separated,
// are one act (novox/hq ADR 0207), so the holders that depend on each other go on together from the
// command API and the controller seat's verbs as they do from the command line.
func splitModules(field string) []string {
var out []string
for _, m := range strings.Split(field, ",") {
if m = strings.TrimSpace(m); m != "" {
out = append(out, m)
}
}
return out
return fmt.Sprintf("%s no longer runs %s — run `push %s` to make it so",
node, module, node) + blockedElsewhere(ctx, open, node), nil
}
// blockedElsewhere is every OTHER machine that cannot be worked out as things now stand.
cmd/mesh-controller/addresses_test.go
+6 -6
View File
@@ -145,7 +145,7 @@ func TestTheRegistryTrustAndEveryImageFollowThePortTheNodeGaveTheStore(t *testin
//
// Composed from the control plane's own manifest against a real inventory: the store's module is
// given 6852 on this node the way genesis or an operator gives it, and the control plane's
// process is told so beside the sealed connection genesis wrote.
// container is told so beside the sealed connection genesis wrote.
func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
@@ -157,8 +157,8 @@ func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T)
if err != nil {
t.Fatal(err)
}
control, err := withSeatPorts(m).Resolve([]catalogue.Built{{Name: "controller", Kind: catalogue.ArtifactBundle,
Reference: "https://registry.example/mesh-controller/controller.tar.gz", Digest: aDigest}})
control, err := withSeatPorts(m).Resolve([]catalogue.Built{{Name: "server", Kind: catalogue.ArtifactImage,
Reference: "registry.example/control@" + aDigest}})
if err != nil {
t.Fatal(err)
}
@@ -200,12 +200,12 @@ func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T)
var env map[string]any
for _, r := range composed(t, open, "anchor").Resources {
if r["id"] == "mesh-controller.controller" {
if r["id"] == "mesh-controller.server" {
env, _ = r["env"].(map[string]any)
}
}
if env == nil {
t.Fatal("the control plane's process is not in its own node's declaration")
t.Fatal("the control plane's container is not in its own node's declaration")
}
for key, want := range map[string]string{
"MESH_STORE_INVENTORY_PORT": "6852",
@@ -238,7 +238,7 @@ func withSeatPorts(m catalogue.Manifest) catalogue.Manifest {
out := m
out.Resources = nil
for _, r := range m.Resources {
if r["type"] != "container" && r["type"] != "process" {
if r["type"] != "container" {
out.Resources = append(out.Resources, r)
continue
}
cmd/mesh-controller/api.go
+2 -2
View File
@@ -95,10 +95,10 @@ func commands(who Authenticator) http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("POST /assign", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
return assign(ctx, open, in.Node, splitModules(in.Module)...)
return assign(ctx, open, in.Node, in.Module)
}))
mux.HandleFunc("POST /unassign", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
return unassign(ctx, open, in.Node, splitModules(in.Module)...)
return unassign(ctx, open, in.Node, in.Module)
}))
// Adoption (novox/hq ADR 0100): the same acts as `take`, `converge` and `adopt`.
mux.HandleFunc("POST /take", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
cmd/mesh-controller/build.go
+2 -33
View File
@@ -148,11 +148,6 @@ func buildFrom(result link.BuildResult) inventory.Build {
// rebuild the graph rather than a list of names.
Path: result.Path,
}
// When it was asked, which is what orders it against another build of the same module
// (novox/hq 04-ISSUES/219) — not when it was heard.
if asked, ok := link.BuildAskedAt(result.ID); ok {
kept.Asked = asked
}
for _, ref := range result.Against {
kept.Against = append(kept.Against, catalogue.Recorded(ref))
}
@@ -414,7 +409,7 @@ func buildOne(ctx context.Context, source buildSource, path, ref string, wait ti
// Correlated by something the control plane makes, not by the module's name: two builds of one
// module can be in flight, and the second answer is not the first one's.
request := link.BuildRequest{
ID: link.NewBuildID(time.Now()),
ID: fmt.Sprintf("%s-%d", "build", time.Now().UnixNano()),
Repository: repository,
Path: path,
Ref: ref,
@@ -531,37 +526,17 @@ func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResu
BuiltFrom: result.Commit, Head: result.Commit,
// What it stood on, so registration can judge a built manifest's base (to-be 38 WP2.4).
Against: kept.Against,
// When it was asked, so an older request heard later does not replace a newer one
// (novox/hq 04-ISSUES/219).
Asked: kept.Asked,
}
if result.Source != nil && result.Source.Seat != "" {
recorded.Repository, recorded.Seat = result.Source.Repository, result.Source.Seat
}
// **A build at a commit does not change the branch a module follows** (novox/hq 04-ISSUES/215):
// the commit is built and recorded as what it was built from, and the module keeps following
// what it followed before — the repository's default branch for one new to the catalogue.
if followedBranch(result.Ref) == "" && result.Ref != "" {
recorded.Ref = ""
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil {
recorded.Ref = followedBranch(was.Ref)
}
}
if err := namesNoInstallation(manifest); err != nil {
return manifest, kept, fmt.Errorf("%s built %s (%s), and the mesh does not register it: %w",
result.On, result.Repository, short(result.Commit), err)
}
if err := inv.RegisterModule(ctx, manifest, recorded); err != nil {
if errors.Is(err, inventory.ErrSuperseded) {
return manifest, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: %w",
result.On, manifest.Module, short(result.Commit), err)
}
return manifest, kept, err
}
// The keep set just moved, and new bytes just landed (novox/hq ADR 0189). Asked here rather
// than on a timer of its own: this is the only moment either is true. Never fatal — the build
// worked and the module is registered.
collect(ctx, inv)
return manifest, kept, nil
}
@@ -589,10 +564,9 @@ func buildAndShow(ctx context.Context, source buildSource, path, ref string, wai
defer ask.Close()
result, err := ask.Submit(ctx, link.BuildRequest{
ID: link.NewBuildID(time.Now()),
ID: fmt.Sprintf("%s-%d", "build", time.Now().UnixNano()),
Repository: repository, Path: path, Ref: ref,
Held: heldBy(ctx), Seats: seatBases(ctx),
DryRun: true,
}, wait)
if err != nil {
return err
@@ -651,11 +625,6 @@ type answers struct {
// public name on the machine went dark. The holds were correct; they were recorded only in the
// machine's own state file, and the one visible symptom was a count that did not add up.
untaken map[string]map[string]int
// unheld is every module on a machine whose resources are applied through a seat nothing on
// that machine holds (novox/hq ADR 0207), with the modules that could hold it. Reported, not
// refused, until the switch — and while there is any, the mesh is not all well: the order the
// machines' modules are built in is the mesh's to keep, and this is where it says it is not kept.
unheld []catalogue.Unheld
}
// heldBy is every artifact this mesh has built, for a build that may need one as its base.
cmd/mesh-controller/build_test.go
-87
View File
@@ -2,12 +2,9 @@ package main
import (
"encoding/json"
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -66,87 +63,3 @@ func TestABuildHeardIsRecordedAndRegistered(t *testing.T) {
t.Fatalf("a failure is said in the builder's words: %v", err)
}
}
// novox/hq 04-ISSUES/215: a build asked at a commit is recorded as built from that commit, and the
// module keeps following the branch it followed — a new one, the default branch.
func TestABuildAtACommitKeepsTheBranchTheModuleFollows(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
manifest, _ := json.Marshal(map[string]any{"module": "unifi", "version": "1"})
result := func(id, ref, commit string) link.BuildResult {
return link.BuildResult{ID: id, Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/unifi", Ref: ref, On: "anchor", Commit: commit, Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
}
if _, _, err := takeIn(ctx, open.inventory, result("b-1", "main", "1111111aaaa")); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, result("b-2", "9c97a8a", "9c97a8a1d2c3")); err != nil {
t.Fatal(err)
}
src, err := open.inventory.SourceOf(ctx, "unifi")
if err != nil {
t.Fatal(err)
}
if src.Ref != "main" || src.BuiltFrom != "9c97a8a1d2c3" {
t.Errorf("after a build at a commit the module follows %q, built from %q; want main, 9c97a8a1d2c3", src.Ref, src.BuiltFrom)
}
// One new to the catalogue, first built at a commit, follows the default branch.
other, _ := json.Marshal(map[string]any{"module": "letta", "version": "1"})
r := result("b-3", "deadbeef", "deadbeefcafe")
r.Manifest, r.Path = other, "modules/letta"
if _, _, err := takeIn(ctx, open.inventory, r); err != nil {
t.Fatal(err)
}
if src, _ := open.inventory.SourceOf(ctx, "letta"); src.Ref != "" {
t.Errorf("a module first built at a commit follows %q, want the default branch", src.Ref)
}
}
// novox/hq 04-ISSUES/219: an older request heard after a newer one is recorded and not registered,
// so a push sends what the newer request built.
func TestAnOlderBuildHeardLaterDoesNotReplaceTheNewer(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
older := time.Date(2026, 10, 3, 21, 33, 45, 0, time.UTC)
newer := time.Date(2026, 10, 3, 21, 51, 57, 0, time.UTC)
result := func(asked time.Time, image string) link.BuildResult {
manifest, _ := json.Marshal(map[string]any{"module": "postgres", "version": image})
return link.BuildResult{ID: link.NewBuildID(asked), Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/postgres", Ref: "main", On: "anchor", Commit: "efff5415", Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
}
if _, _, err := takeIn(ctx, open.inventory, result(newer, "4bcd5f73")); err != nil {
t.Fatal(err)
}
_, _, err := takeIn(ctx, open.inventory, result(older, "0ab07fa9"))
if !errors.Is(err, inventory.ErrSuperseded) {
t.Fatalf("the older request's outcome was taken in as current: %v", err)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if got := shelf["postgres"].Version; got != "4bcd5f73" {
t.Errorf("postgres is %q; want the newer request's 4bcd5f73", got)
}
if builds, _ := open.inventory.Builds(ctx, "postgres", 5); len(builds) != 2 {
t.Errorf("the late build was not recorded: %v", builds)
}
}
func TestABuildIDSaysWhenItWasAsked(t *testing.T) {
at := time.Date(2026, 10, 3, 21, 51, 57, 392539762, time.UTC)
if got, ok := link.BuildAskedAt(link.NewBuildID(at)); !ok || !got.Equal(at) {
t.Errorf("read back %v %v; want %v", got, ok, at)
}
if got, ok := link.BuildAskedAt("build-1791064317392539762"); !ok || got.Format(time.TimeOnly) != "21:51:57" {
t.Errorf("the incident's id reads as %v %v", got, ok)
}
for _, id := range []string{"b-1", "build-2", "build-", "build-x", ""} {
if _, ok := link.BuildAskedAt(id); ok {
t.Errorf("%q read as a request time", id)
}
}
}
cmd/mesh-controller/check.go
-12
View File
@@ -7,7 +7,6 @@ import (
"os"
"path/filepath"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
)
@@ -91,17 +90,6 @@ func moduleCheck(paths []string, out io.Writer) error {
if len(m.Invokes) > 0 {
fmt.Fprintf(out, ", invokes %s", joinInvokes(m.Invokes))
}
// The state it keeps and reads (novox/hq ADR 0201), so a reviewer sees what lands on the bus.
if len(m.State) > 0 {
kept := make([]string, 0, len(m.State))
for _, s := range m.State {
kept = append(kept, s.Name)
}
fmt.Fprintf(out, ", keeps state %s", strings.Join(kept, ", "))
}
if len(m.Reads) > 0 {
fmt.Fprintf(out, ", reads %s", strings.Join(m.Reads, ", "))
}
fmt.Fprintln(out)
}
if failed > 0 {
cmd/mesh-controller/collect.go
-175
View File
@@ -1,175 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"os"
"time"
"github.com/novox/mesh-controller/internal/artifacts"
"github.com/novox/mesh-controller/internal/inventory"
)
// Letting the artifact store go of what the mesh no longer keeps (novox/hq ADR 0189, issue 108).
//
// **Run where the records change.** A build is the moment new bytes landed in the store and the
// moment the keep set moved, so it is the moment to say what may go — and it needs no timer of
// its own. Reclaiming the bytes is the store's own nightly step; this only decides.
//
// Never fatal to a build. The build succeeded, the module is registered, and a store that could
// not be reached is a thing to say rather than a reason to undo any of that. The next build asks
// again, and the references it could not collect are still uncollected, so nothing is lost by
// having failed.
// collect asks the store to let go of everything the mesh made and no longer keeps, and records
// what it let go of. Says what it did and what it could not; returns nothing, because nothing
// upstream should branch on it.
func collect(ctx context.Context, inv *inventory.Inventory) {
references, err := inv.ToCollect(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not work out what the artifact store may let go of: %v\n", err)
return
}
kept, err := inv.KeptArchives(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not work out which archives the artifact store keeps: %v\n", err)
return
}
if len(references) == 0 && len(kept) == 0 {
return
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read the catalogue to find the artifact store: %v\n", err)
return
}
// As the mesh reaches it from the network. Empty means the store is not on the network — on a
// mesh being raised it is not yet, and there the store holds one build of anything and has
// nothing to collect.
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil || address == "" {
if err != nil {
fmt.Fprintf(os.Stderr, "could not find the artifact store to collect from: %v\n", err)
}
return
}
// **Bounded, because this runs inside somebody's build.** The first sweep of a mesh that has
// never collected has the whole history to get through, and a person waiting on `build` should
// not pay for it. Two bounds, and what is left over is simply offered again next time —
// builds are frequent, and the point is that the store stops growing, not that it empties
// tonight.
within, stop := context.WithTimeout(ctx, sweepBudget)
defer stop()
store := artifacts.Store{Address: address}
// **Hold before letting go** (novox/hq issue 253). The store's collector keeps only what a
// manifest names, and archives were published as bare blobs, so every kept archive is first
// held by its manifest — which backfills the ones published before holders, a few at a time
// as builds come, and is two HEADs each once done. A kept archive that could not be held stops
// the sweep before it deletes anything: "everything kept is held" is the precondition the
// collector's safety rests on, and a store refusing a hold would refuse the deletes too.
wrote, missing, err := holdKept(within, store, kept)
if wrote > 0 {
fmt.Fprintf(os.Stderr, "the artifact store now holds %d more kept archive(s) by a manifest\n", wrote)
}
if missing > 0 {
fmt.Fprintf(os.Stderr, "%d archive(s) the mesh keeps are not in the artifact store at all; "+
"`collection` lists them\n", missing)
}
if err != nil {
fmt.Fprintf(os.Stderr, "not every kept archive could be held, so nothing was let go: %v\n", err)
return
}
var done []string
var left, skipped int
for i, reference := range references {
if i >= mostPerSweep || within.Err() != nil {
left = len(references) - i
break
}
err := store.LetGo(within, reference)
if err == nil || errors.Is(err, artifacts.Gone) {
// Gone is the outcome wanted, already true. Recorded so the next sweep does not ask
// again for ever.
done = append(done, reference)
continue
}
if errors.Is(err, artifacts.ErrNotOurs) {
// **A fact about this record, so this record is skipped** (novox/hq issue 226). Not
// marked collected — the mesh did not remove it and should not claim to — and not a
// reason to stop, because the store was never asked. One of these at the front of
// the oldest-first order ended every sweep until this.
skipped++
if skipped == 1 {
fmt.Fprintf(os.Stderr,
"the sweep will not address %s and went on: %v\n", reference, err)
}
continue
}
// **Stopped at the first refusal by the STORE, not pushed through.** A store that refuses
// one refuses all of them — deletion disabled, the store down, the network gone — so
// going on would be a hundred identical failures and a hundred identical log lines in
// front of whoever was building something.
fmt.Fprintf(os.Stderr, "the artifact store kept %s, so nothing more was asked of it: %v\n",
reference, err)
left = len(references) - i
break
}
if len(done) > 0 {
// Recorded outside `within`: the deletions happened, and losing the record of them because
// the sweep ran out of budget would mean asking about them again for ever.
if err := inv.MarkCollected(ctx, done); err != nil {
fmt.Fprintf(os.Stderr, "the store let go of %d artifact(s) and the record of it did not keep: %v\n",
len(done), err)
return
}
fmt.Fprintf(os.Stderr, "the artifact store let go of %d artifact(s) the mesh no longer keeps\n",
len(done))
}
if left > 0 {
fmt.Fprintf(os.Stderr, "%d more to collect; the next build asks again\n", left)
}
if skipped > 0 {
fmt.Fprintf(os.Stderr, "%d artifact(s) the sweep will not address were skipped\n", skipped)
}
}
// holdKept holds every kept archive by its manifest, stopping at the first refusal by the store.
// Answers how many holders it wrote and how many kept archives the store does not have.
//
// A missing archive is counted rather than fatal: there is nothing to hold, and that is a fact
// for an operator to read (`collection`), not a reason to stop collecting what is not kept. A
// reference the store cannot be asked about is skipped as the deletion loop skips one
// (novox/hq issue 226).
func holdKept(ctx context.Context, store artifacts.Store, kept []string) (wrote, missing int, err error) {
for _, reference := range kept {
if err := ctx.Err(); err != nil {
return wrote, missing, fmt.Errorf("ran out of time before %s: %w", reference, err)
}
did, err := store.Hold(ctx, reference)
switch {
case err == nil:
if did {
wrote++
}
case errors.Is(err, artifacts.Gone):
missing++
case errors.Is(err, artifacts.ErrNotOurs):
default:
return wrote, missing, fmt.Errorf("holding %s: %w", reference, err)
}
}
return wrote, missing, nil
}
// mostPerSweep is how many artifacts one sweep will ask about. Enough that a mesh building
// several times a day converges within days of this landing; small enough that no single build
// waits on the whole backlog.
const mostPerSweep = 200
// sweepBudget is the longest a sweep will keep a build waiting.
const sweepBudget = 60 * time.Second
cmd/mesh-controller/collection.go
-166
View File
@@ -1,166 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"strings"
"github.com/novox/mesh-controller/internal/artifacts"
)
// What the artifact store keeps, whether each kept archive is held, and what the sweep may let go
// (novox/hq issue 253, ADR 0189).
//
// **The question to answer before the store's collector runs for real.** The collector deletes
// every blob no manifest names, and archives were published as bare blobs, so the nightly step
// runs `--dry-run` until every archive the mesh keeps is held by its manifest. The sweep holds
// them as builds come; this says how far that has got — "0 unheld" is the number that lets the
// dry run go.
//
// Reads and changes nothing: each kept archive is asked about with HEADs only. Reached over the
// console through the mesh-controller seat's `command` verb (`collection --json`), which needs no
// new verb in the seat's row.
type collectionReport struct {
// Store is the artifact store as this machine reached it; empty when it is not on the network.
Store string `json:"store"`
// KeptArchives is how many archives the mesh keeps, for either reason.
KeptArchives int `json:"kept_archives"`
// Held is how many of them the store holds by their manifest.
Held int `json:"held"`
// Unheld are the kept archives the collector would delete tonight if it ran for real.
Unheld []string `json:"unheld"`
// Missing are kept archives the store does not have at all.
Missing []string `json:"missing"`
// Unasked is how many could not be asked about, and why the asking stopped.
Unasked int `json:"unasked"`
Stopped string `json:"stopped,omitempty"`
// Eligible is what the sweep may let go of: made by the mesh, kept for no reason, not yet
// collected — split by kind.
Eligible int `json:"eligible"`
EligibleImages int `json:"eligible_images"`
EligibleArchives int `json:"eligible_archives"`
// SafeToCollect is whether every kept archive was asked about and every one is held.
SafeToCollect bool `json:"safe_to_collect"`
}
func collectionCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("collection", flag.ContinueOnError)
asJSON := set.Bool("json", false, "answer as JSON")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 0 {
return errors.New("collection [--json]")
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
kept, err := inv.KeptArchives(ctx)
if err != nil {
return err
}
eligible, err := inv.ToCollect(ctx)
if err != nil {
return err
}
report := collectionReport{KeptArchives: len(kept), Eligible: len(eligible), Unheld: []string{}, Missing: []string{}}
for _, reference := range eligible {
if strings.Contains(reference, "/blobs/") {
report.EligibleArchives++
} else {
report.EligibleImages++
}
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return err
}
report.Store, err = artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return err
}
if report.Store == "" {
report.Unasked = len(kept)
report.Stopped = "this mesh has no artifact store on its network"
} else {
report.Unasked, report.Stopped = askHeld(ctx, artifacts.Store{Address: report.Store}, kept, &report)
}
report.SafeToCollect = report.Unasked == 0 && len(report.Unheld) == 0
if *asJSON {
encoder := json.NewEncoder(os.Stdout)
encoder.SetIndent("", " ")
return encoder.Encode(report)
}
printCollection(report)
return nil
}
// askHeld asks the store about each kept archive, stopping at the first answer that is not about
// the archive: a store that cannot be reached for one cannot be for the next, and a page of
// identical failures says less than one line.
func askHeld(ctx context.Context, store artifacts.Store, kept []string, report *collectionReport) (int, string) {
for i, reference := range kept {
held, err := store.Held(ctx, reference)
switch {
case err == nil && held:
report.Held++
case err == nil:
report.Unheld = append(report.Unheld, reference)
case errors.Is(err, artifacts.Gone):
report.Missing = append(report.Missing, reference)
case errors.Is(err, artifacts.ErrNotOurs):
// KeptArchives names only the mesh's own; counted as unasked if one ever is not.
report.Unasked++
default:
return report.Unasked + len(kept) - i, fmt.Sprintf("asking about %s: %v", reference, err)
}
}
return report.Unasked, ""
}
func printCollection(r collectionReport) {
store := r.Store
if store == "" {
store = "(not on the network)"
}
fmt.Printf("artifact store %s\n", store)
fmt.Printf("kept archives %d\n", r.KeptArchives)
fmt.Printf(" held %d\n", r.Held)
fmt.Printf(" unheld %d\n", len(r.Unheld))
fmt.Printf(" missing %d\n", len(r.Missing))
if r.Unasked > 0 {
fmt.Printf(" not asked %d (%s)\n", r.Unasked, r.Stopped)
}
fmt.Printf("eligible to let go %d (%d images, %d archives)\n", r.Eligible, r.EligibleImages, r.EligibleArchives)
if len(r.Unheld) > 0 {
fmt.Println("\nunheld — the store's collector would delete these; the next build's sweep holds them:")
for _, reference := range r.Unheld {
fmt.Printf(" %s\n", reference)
}
}
if len(r.Missing) > 0 {
fmt.Println("\nmissing — kept by the mesh, not in the store:")
for _, reference := range r.Missing {
fmt.Printf(" %s\n", reference)
}
}
fmt.Println()
if r.SafeToCollect {
fmt.Println("every kept archive is held: the store's collector may run for real")
} else {
fmt.Println("NOT every kept archive is known to be held: keep the store's collector on --dry-run")
}
}
cmd/mesh-controller/delivery_order_test.go
-145
View File
@@ -1,145 +0,0 @@
package main
import (
"context"
"errors"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// recordingDelivery is a delivery that writes down what was done, in order, and fails where told.
type recordingDelivery struct {
did []string
grantErr error
declareErr error
}
func (r *recordingDelivery) grant(_ context.Context, sending []readyNode) error {
for _, s := range sending {
r.did = append(r.did, "grant "+s.node)
}
return r.grantErr
}
func (r *recordingDelivery) declare(_ context.Context, s readyNode, _ []byte) (string, error) {
if r.declareErr != nil {
return "", r.declareErr
}
r.did = append(r.did, "declare "+s.node)
return "digest-" + s.node, nil
}
func ready(names ...string) []readyNode {
var out []readyNode
for _, n := range names {
out = append(out, readyNode{node: n, declared: sendable{Resources: []map[string]any{{"id": "x"}}}})
}
return out
}
// novox/hq issue 249: the grants that come with a module's new declarations are issued before any
// machine is sent the code that uses them — except the machine holding the bus, whose declaration
// carries the controller's own right to issue them, and goes first.
func TestGrantsAreIssuedBeforeTheDeclarations(t *testing.T) {
d := &recordingDelivery{}
digests, err := deliver(t.Context(), d, "", ready("anchor", "laptop"))
if err != nil {
t.Fatal(err)
}
want := []string{"grant anchor", "grant laptop", "declare anchor", "declare laptop"}
if !reflect.DeepEqual(d.did, want) {
t.Fatalf("delivered in the order %v, wanted %v", d.did, want)
}
if digests["anchor"] != "digest-anchor" || digests["laptop"] != "digest-laptop" {
t.Fatalf("the digests sent were not answered: %v", digests)
}
held := &recordingDelivery{}
if _, err := deliver(t.Context(), held, "broker", ready("broker", "anchor")); err != nil {
t.Fatal(err)
}
want = []string{"declare broker", "grant broker", "grant anchor", "declare anchor"}
if !reflect.DeepEqual(held.did, want) {
t.Fatalf("with the bus's machine in the send: %v, wanted %v", held.did, want)
}
}
// A grant that cannot be issued holds back the machines it concerns and is an error the caller
// retries on — never "until the next push" — and the bus's own machine is sent regardless, so the
// grant that would let the controller issue memberships is never held behind them.
func TestAGrantThatFailsHoldsBackWhatItConcerns(t *testing.T) {
// A failure naming no machine (the buckets): everything but the bus's machine.
d := &recordingDelivery{grantErr: errors.New("the bus refused the bucket")}
_, err := deliver(t.Context(), d, "broker", ready("broker", "anchor", "laptop"))
if err == nil || !errors.Is(err, errGrants) || !strings.Contains(err.Error(), "the bus refused the bucket") ||
!strings.Contains(err.Error(), "anchor, laptop not sent") {
t.Fatalf("a failed grant was not said as the send's failure: %v", err)
}
if want := []string{"declare broker", "grant broker", "grant anchor", "grant laptop"}; !reflect.DeepEqual(d.did, want) {
t.Fatalf("delivered %v, wanted the bus's machine alone", d.did)
}
// A membership that failed for one machine: that machine alone.
one := &recordingDelivery{grantErr: &grantsRefused{nodes: map[string]error{"laptop": errors.New("no")}}}
_, err = deliver(t.Context(), one, "", ready("anchor", "laptop"))
if !errors.Is(err, errGrants) || !strings.Contains(err.Error(), "laptop not sent") {
t.Fatalf("one machine's refused membership was not said: %v", err)
}
if want := []string{"grant anchor", "grant laptop", "declare anchor"}; !reflect.DeepEqual(one.did, want) {
t.Fatalf("delivered %v, wanted anchor sent and laptop held back", one.did)
}
// The announced upgrade that hit it is asked again.
if !errors.Is(askAgainOnGrants(err), link.ErrTryAgain) {
t.Fatal("an announcement whose send stopped at its grants is not asked again")
}
if other := errors.New("laptop could not be resolved"); errors.Is(askAgainOnGrants(other), link.ErrTryAgain) {
t.Fatal("any failure is asked again, not only a grant's")
}
// Nothing to send is nothing granted either.
none := &recordingDelivery{grantErr: errors.New("never asked")}
if _, err := deliver(t.Context(), none, "", nil); err != nil || len(none.did) != 0 {
t.Fatalf("an empty send granted or failed: %v %v", none.did, err)
}
}
// Whether the bus's machine goes first is read from the user list alone, by its digest.
func TestTheBusMachineIsBehindByItsUserListAlone(t *testing.T) {
list := "users: [a, b]"
if userListBehind(list, digestOf([]byte(list))) {
t.Fatal("the list it was sent reads as behind")
}
if !userListBehind(list, digestOf([]byte("users: [a]"))) || !userListBehind(list, "") {
t.Fatal("a changed or never-sent list reads as current")
}
if userListBehind("", "") {
t.Fatal("a machine sent no list reads as behind")
}
}
// The machine holding the bus goes first: its declaration carries the user list the new grants are
// checked against. Among the machines it is moved to the front; not among them it is added only
// when it is behind.
func TestTheMachineHoldingTheBusIsSentFirst(t *testing.T) {
for _, c := range []struct {
what string
names []string
holder string
behind bool
want []string
}{
{"among them", []string{"ace", "g14", "novox"}, "novox", false, []string{"novox", "ace", "g14"}},
{"not among them, behind", []string{"ace", "g14"}, "novox", true, []string{"novox", "ace", "g14"}},
{"not among them, current", []string{"ace", "g14"}, "novox", false, []string{"ace", "g14"}},
{"nothing holds the bus", []string{"ace", "g14"}, "", true, []string{"ace", "g14"}},
{"only it", []string{"novox"}, "novox", false, []string{"novox"}},
} {
if got := brokerFirst(c.names, c.holder, c.behind); !reflect.DeepEqual(got, c.want) {
t.Errorf("%s: sent in the order %v, wanted %v", c.what, got, c.want)
}
}
}
cmd/mesh-controller/dry_run_test.go
-38
View File
@@ -1,38 +0,0 @@
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// A dry run's outcome is looked at, never taken in (novox/hq issue 240). The daemon here holds no
// store at all, so anything that tried to record or register would fail rather than pass quietly.
func TestADryRunsOutcomeIsTakenInByNothing(t *testing.T) {
err := builds{}.Built(t.Context(), link.BuildResult{
ID: "build-1", Repository: "ssh://forge/app.git", Ref: "unreviewed", Module: "app", DryRun: true,
Manifest: json.RawMessage(`{"module":"app","version":"1"}`),
})
if err != nil {
t.Fatalf("a dry run's outcome was not simply set aside: %v", err)
}
}
// The mark survives the wire both ways: asked as a dry run, answered as one.
func TestTheDryRunMarkTravelsWithTheBuild(t *testing.T) {
raw, _ := json.Marshal(link.BuildRequest{ID: "build-1", Repository: "r", DryRun: true})
var asked link.BuildRequest
if err := json.Unmarshal(raw, &asked); err != nil || !asked.DryRun {
t.Fatalf("the request lost its dry-run mark: %s", raw)
}
raw, _ = json.Marshal(link.BuildResult{ID: "build-1", DryRun: true})
var answered link.BuildResult
if err := json.Unmarshal(raw, &answered); err != nil || !answered.DryRun {
t.Fatalf("the outcome lost its dry-run mark: %s", raw)
}
raw, _ = json.Marshal(link.BuildResult{ID: "build-2"})
if string(raw) != `{"id":"build-2","repository":"","on":""}` {
t.Fatalf("an ordinary outcome carries a dry-run mark: %s", raw)
}
}
cmd/mesh-controller/hold_test.go
+4 -4
View File
@@ -18,16 +18,16 @@ func TestASendRoundGivesItsHoldBackOnEveryWayOut(t *testing.T) {
plain := func(context.Context, string) (sendable, error) {
return sendable{Resources: []map[string]any{{"id": "x"}}}, nil
}
failing := &recordingDelivery{declareErr: errors.New("the broker went away")}
fine := &recordingDelivery{}
failing := func(readyNode, []byte) error { return errors.New("the broker went away") }
fine := func(readyNode, []byte) error { return nil }
for name, round := range map[string]func() error{
"a body that cannot be marshalled": func() error {
_, err := sendRound(ctx, open, []string{"anchor"}, unmarshallable, fine, "", nil)
_, err := sendRound(ctx, open, []string{"anchor"}, unmarshallable, fine)
return err
},
"a send that fails": func() error {
_, err := sendRound(ctx, open, []string{"anchor"}, plain, failing, "", nil)
_, err := sendRound(ctx, open, []string{"anchor"}, plain, failing)
return err
},
} {
cmd/mesh-controller/licence.go
+4 -4
View File
@@ -232,7 +232,7 @@ func licenceKey(ctx context.Context, args []string) error {
// and printing the value here would put the one copy that matters on a terminal.
fmt.Printf("sealed to %d holder(s). The mesh has discarded the key and cannot read it back\n",
sealed)
fmt.Printf(" run `push --behind` to deliver it\n")
fmt.Printf(" run `push` to deliver it\n")
return nil
}
@@ -340,7 +340,7 @@ func licenceSetGrant(ctx context.Context, args []string) error {
return err
}
fmt.Printf("%s now holds a refresh token for %s, sealed to that node's key and readable by it "+
"alone.\n the control plane stored the box without opening it; run `push --behind` to deliver it\n",
"alone.\n the control plane stored the box without opening it; run `push` to deliver it\n",
"the manager", name)
return nil
}
@@ -423,7 +423,7 @@ func licenceSubmitRefresh(ctx context.Context, args []string) error {
"manager node alone"
}
fmt.Printf("submitted a refresh for %s: a new access token sealed to %d holder(s), and %s.\n"+
" run `push --behind` to deliver it\n", name, sealed, rotatedNote)
" run `push` to deliver it\n", name, sealed, rotatedNote)
return nil
}
@@ -456,7 +456,7 @@ func licenceRefresh(ctx context.Context, args []string) error {
return err
}
fmt.Printf("refreshed %s: a new access token sealed to %d holder(s), and the refresh token left "+
"with its manager.\n run `push --behind` to deliver it\n", name, sealed)
"with its manager.\n run `push` to deliver it\n", name, sealed)
return nil
}
cmd/mesh-controller/main.go
+5 -25
View File
@@ -8,7 +8,6 @@ package main
import (
"context"
"errors"
"flag"
"fmt"
"os"
@@ -73,8 +72,6 @@ func run() error {
return askCommand(ctx, args[1:])
case "builds":
return buildsCommand(ctx, args[1:])
case "collection":
return collectionCommand(ctx, args[1:])
case "plans":
return plansCommand(ctx, args[1:])
case "pin":
@@ -181,8 +178,8 @@ func usage() {
seat <name> --to <node>/<module> hand a seat to that assignment as one act; never empty in between (ADR 0131)
board [--listen ADDR] the same three questions, as a page that holds nothing
api --issuer URL [--listen A] assign and unassign over http, for a surface that is not here
assign <node> <module>... put modules on a node, judged together (ADR 0207)
unassign <node> <module>... take them off
assign <node> <module> put a module on a node
unassign <node> <module> take it off
take <node> <module> preview a module's cutover on an adopted node: what runs beside
what it declares; --yes <digest> cuts it over as previewed
converge <node> [--yes <digest>] [--filter nftables] preview, then make, an adopted node converged
@@ -202,7 +199,6 @@ func usage() {
build --behind build every module the mesh holds older than its source
build --on <module> rebuild every module that stands on this module's artifacts, bases first
builds [<module>] what has been built lately, and what came of it
collection [--json] kept archives held/unheld by a manifest, and what the sweep may let go
builder issue <name> a broker account for a build machine, scoped to build work,
delivered as the builder module's broker secret (module add it first)
licence add|list|use|key model access, under the name a person calls it
@@ -256,42 +252,26 @@ func parseAround(set *flag.FlagSet, args []string) ([]string, error) {
// registered, the same as the waiting command does. Said either way, so the daemon's log tells what
// became of a build nobody was watching.
func (b builds) Built(ctx context.Context, result link.BuildResult) error {
// **A dry run is looked at, never taken in** (novox/hq issue 240). On 2026-10-04 a dry run of an
// unmerged branch was heard here like any build, registered, and its definition reached a machine
// before anyone had reviewed it.
if result.DryRun {
fmt.Printf("%s: a dry run of %s on %s, not taken in\n", result.ID, result.Repository, result.Ref)
return nil
}
manifest, _, err := takeIn(ctx, b.inv, result)
// When it was asked, so a plan takes as its outcome only a build asked for it or after it
// (novox/hq 04-ISSUES/219). Zero when the id does not say.
asked, _ := link.BuildAskedAt(result.ID)
switch {
case err != nil && result.Failed != "":
fmt.Printf("%s: %v\n", result.ID, err)
if result.Module != "" {
planBuilt(ctx, b.open, result.Module, result.Commit, result.Failed, asked)
planBuilt(ctx, b.open, result.Module, result.Commit, result.Failed)
} else {
planFailedBuild(ctx, b.open, result)
}
return nil
case errors.Is(err, inventory.ErrSuperseded):
// Not a failure: the module is already at what a later request built. A plan that asked
// before that later request is answered by it; one that asked after it ignores this.
fmt.Printf("%s: %v\n", result.ID, err)
planBuilt(ctx, b.open, manifest.Module, result.Commit, "", asked)
return nil
case err != nil:
fmt.Printf("%s: heard and recorded, and not registered: %v\n", result.ID, err)
if manifest.Module != "" {
planBuilt(ctx, b.open, manifest.Module, result.Commit, err.Error(), asked)
planBuilt(ctx, b.open, manifest.Module, result.Commit, err.Error())
}
return nil
}
fmt.Printf("%s: %s %s registered, built on %s from %s\n",
result.ID, manifest.Module, manifest.Version, result.On, short(result.Commit))
saysWhenThePolicyActs(ctx, b.inv, manifest.Module)
planBuilt(ctx, b.open, manifest.Module, result.Commit, "", asked)
planBuilt(ctx, b.open, manifest.Module, result.Commit, "")
return nil
}
cmd/mesh-controller/modules.go
+12 -86
View File
@@ -8,6 +8,7 @@ import (
"fmt"
"net"
"os"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/broker"
@@ -333,10 +334,8 @@ func moduleCommand(ctx context.Context, args []string) error {
}
func assignCommand(ctx context.Context, verb string, args []string) error {
// Several modules in one act (novox/hq ADR 0207): holders that depend on each other — the
// service manager and the package manager — can only go on, or come off, together.
if len(args) < 2 {
return fmt.Errorf("%s <node> <module> [<module>…]", verb)
if len(args) != 2 {
return fmt.Errorf("%s <node> <module>", verb)
}
open, err := openStores(ctx)
if err != nil {
@@ -350,7 +349,7 @@ func assignCommand(ctx context.Context, verb string, args []string) error {
if verb == "unassign" {
act = unassign
}
said, err := act(ctx, open, args[0], args[1:]...)
said, err := act(ctx, open, args[0], args[1])
if said != "" {
fmt.Println(said)
}
@@ -363,8 +362,7 @@ func assignCommand(ctx context.Context, verb string, args []string) error {
func settingsCommand(ctx context.Context, args []string) error {
if len(args) == 0 {
return errors.New("settings show <module> [--node <node>] [--history], settings set <module> <file> " +
"[--node <node>] [--replace], or settings clear <module> [--node <node>]")
return errors.New("settings set <module> <file> [--node <node>], or settings clear <module> [--node <node>]")
}
open, err := openStores(ctx)
if err != nil {
@@ -375,11 +373,6 @@ func settingsCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("settings", flag.ContinueOnError)
node := set.String("node", "", "one machine, rather than the whole mesh")
// **What a set removes is refused unless meant** (novox/hq ADR 0217). A layer is replaced whole,
// and on 2026-10-05 setting one placement dropped a machine's whole layer for a module without a
// word (issue 246). Adding and changing keys needs nothing; removing one needs this.
replace := set.Bool("replace", false, "for set: remove the keys the new layer does not name")
history := set.Bool("history", false, "for show: the layers this one replaced, the latest first")
positionals, err := parseAround(set, args[1:])
if err != nil {
return err
@@ -407,76 +400,17 @@ func settingsCommand(ctx context.Context, args []string) error {
if err := json.Unmarshal(raw, &values); err != nil {
return fmt.Errorf("%s is not a settings file: %w", positionals[1], err)
}
before, _, err := inv.Layer(ctx, *node, positionals[0])
if err != nil {
return err
}
added, changed, removed := settingsChange(before, values)
if len(removed) > 0 && !*replace {
return fmt.Errorf("%s on %s: this layer would no longer set %s. A layer is replaced whole; "+
"read it with `settings show %s%s` and include what should stay, or add --replace if the "+
"removal is meant (novox/hq ADR 0217). Nothing was changed",
positionals[0], where, strings.Join(removed, ", "), positionals[0], nodeFlag(*node))
}
if err := inv.SetSettings(ctx, *node, positionals[0], values); err != nil {
return err
}
fmt.Printf("%s on %s:\n", positionals[0], where)
if len(added)+len(changed)+len(removed) == 0 {
fmt.Println(" nothing changed")
}
for _, p := range added {
fmt.Printf(" + %s\n", p)
}
for _, p := range changed {
fmt.Printf(" ~ %s\n", p)
}
for _, p := range removed {
fmt.Printf(" - %s\n", p)
}
if before != nil {
fmt.Printf(" the layer it replaced is kept: settings show %s%s --history\n", positionals[0], nodeFlag(*node))
}
if *node != "" {
fmt.Printf(" run `plan %s --diff` to see what it changes, `push %s` to send it\n", *node, *node)
} else {
fmt.Println(" run `push --behind` to send it to the machines running it")
}
return nil
case "show":
if len(positionals) != 1 {
return errors.New("settings show <module> [--node <node>] [--history]")
var keys []string
for k := range values {
keys = append(keys, k)
}
if *history {
past, err := inv.SettingsHistory(ctx, *node, positionals[0])
if err != nil {
return err
}
if len(past) == 0 {
fmt.Printf("%s on %s: no layer has been replaced\n", positionals[0], where)
return nil
}
for _, p := range past {
shown, _ := json.MarshalIndent(p.Values, " ", " ")
fmt.Printf("%s on %s, until %s (%s):\n %s\n", positionals[0], where,
p.ReplacedAt.Local().Format("2006-01-02 15:04:05"), p.ReplacedBy, shown)
}
return nil
}
values, has, err := inv.Layer(ctx, *node, positionals[0])
if err != nil {
return err
}
if !has {
fmt.Printf("%s on %s: no layer — the module's definition says\n", positionals[0], where)
return nil
}
shown, err := json.MarshalIndent(values, "", " ")
if err != nil {
return err
}
fmt.Println(string(shown))
sort.Strings(keys)
fmt.Printf("%s on %s: %s\n", positionals[0], where, strings.Join(keys, ", "))
fmt.Println(" run `push` to send it")
return nil
case "clear":
@@ -490,18 +424,10 @@ func settingsCommand(ctx context.Context, args []string) error {
return nil
default:
return fmt.Errorf("settings has no %q; it has show, set and clear", args[0])
return fmt.Errorf("settings has no %q; it has set and clear", args[0])
}
}
// nodeFlag is ` --node <node>` for a machine's layer, nothing for the whole mesh's.
func nodeFlag(node string) string {
if node == "" {
return ""
}
return " --node " + node
}
// describeOffers says what a module provides, and marks the ones answered from anywhere in the
// mesh — because "provides a database" and "provides a shell" are read the same way and mean
// entirely different things about where the answer has to be.
cmd/mesh-controller/network_test.go
+8 -14
View File
@@ -254,10 +254,11 @@ func theResolver(t *testing.T) catalogue.Manifest {
return m
}
// The resolver is handed every machine on the private network as a wildcard, and is handed it again
// when a machine leaves — through the module's own manifest asking for the fact, with no module of the
// mesh's own in between (hal dnsmasq-app conversion, novox/hq 08-connectivity). It is the mesh's one
// resolver (ADR 0194), and the container runtime is given no resolver of its own (ADR 0196).
// The resolver is handed every machine on the private network as a wildcard, the same set and the
// same source as the hosts file, and is handed it again when a machine leaves — through the
// module's own manifest asking for the fact, with no module of the mesh's own in between (hal
// dnsmasq-app conversion, novox/hq 08-connectivity). The runtime on that machine is pointed at the
// machine's own address, where the resolver answers for its containers.
func TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
@@ -265,12 +266,6 @@ func TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves(t *t
if _, err := assign(ctx, open, "anchor", "dnsmasq"); err != nil {
t.Fatal(err)
}
// Its bus credential, as assigning issues it where the bus is reachable (novox/hq issue 203):
// no bus is known to this test, so it is minted here, or composing refuses the placeholder.
if _, err := open.inventory.MintBusPassword(ctx, inventory.BusUser{
Username: "anchor.dnsmasq", Kind: inventory.BusModule, Node: "anchor", Module: "dnsmasq"}); err != nil {
t.Fatal(err)
}
zones := func() string {
t.Helper()
for _, r := range composed(t, open, "anchor").Resources {
@@ -292,12 +287,11 @@ func TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves(t *t
t.Errorf("the resolver's machines lack %q:\n%s", want, first)
}
}
// The container runtime is given no resolver of its own (novox/hq ADR 0196): it copies its
// machine's, which name the mesh's resolver first. A `dns` key would be a second account of where a
// container asks, read only when the runtime starts.
for _, r := range composed(t, open, "anchor").Resources {
if r["id"] == "dnsmasq.runtime-dns" {
t.Errorf("the resolver still writes the runtime's own dns: %v", r)
if !strings.Contains(r["content"].(string), `"10.77.0.1"`) || r["into"] != "json" {
t.Errorf("the runtime is not pointed at this machine's own address, written into its file: %v", r)
}
}
}
cmd/mesh-controller/nodes.go
+1 -33
View File
@@ -386,12 +386,8 @@ func brokerCommand(ctx context.Context, args []string) error {
if len(args) > 0 && args[0] == "accounts" {
return busAccounts(ctx, args[1:])
}
if len(args) > 0 && args[0] == "consumer-reset" {
return consumerReset(args[1:])
}
if len(args) == 0 || args[0] != "show" {
return errors.New("broker show | broker certificate [--check] --into <directory> | broker accounts --into <file> | " +
"broker consumer-reset <stream> <consumer>")
return errors.New("broker show | broker certificate [--check] --into <directory> | broker accounts --into <file>")
}
known, err := broker.FromEnvironment()
if errors.Is(err, broker.ErrNotConfigured) {
@@ -411,34 +407,6 @@ func brokerCommand(ctx context.Context, args []string) error {
return nil
}
// consumerReset re-makes one consumer on a stream that keeps history to start from now (novox/hq issue
// 248): the way out of a consumer replaying a week of announcements, said rather than done by hand. A
// person's act — what was pending is dropped — so it is a command, and nothing calls it on its own.
func consumerReset(args []string) error {
if len(args) != 2 {
return errors.New("broker consumer-reset <stream> <consumer>, e.g. broker consumer-reset EVENTS controller")
}
address, err := broker.BusAddress()
if err != nil {
return err
}
js, err := broker.Dial(address)
if err != nil {
return fmt.Errorf("cannot reach the bus: %w", err)
}
defer js.Close()
before, after, err := js.ResetConsumer(args[0], args[1])
if err != nil {
return err
}
fmt.Printf("consumer %s on %s re-made to deliver from now\n", args[1], args[0])
fmt.Printf(" before: delivers %s, delivered to %d, acknowledged to %d, %d pending, %d unacknowledged\n",
before.DeliverPolicy, before.Delivered, before.AckFloor, before.Pending, before.AckPending)
fmt.Printf(" after: delivers %s, %d pending; what was pending is dropped. A holder bound to it may need its "+
"process restarted to bind again\n", after.DeliverPolicy, after.Pending)
return nil
}
// heardFrom says when a node was last heard from, in a form somebody can act on.
//
// "never" and "an hour ago" are different answers and are kept different. A node that has never
cmd/mesh-controller/order_test.go
-32
View File
@@ -147,14 +147,6 @@ func TestAMergeRebuildsTheModulesItChanged(t *testing.T) {
{"a module the mesh does not hold", merge([]string{"modules/plex/index.ts"}, false), ""},
{"nothing said about the files", merge(nil, false), "gitea,keycloak"},
{"more files than were listed", merge([]string{"modules/gitea/index.ts"}, true), "gitea,keycloak"},
// novox/hq issue 252: a module the mesh has never registered is still a module, when the merge
// shows it is one — and a directory that may be shared code is still shared.
{"a new module beside a held one", merge([]string{"modules/gitea/x", "modules/newmod/module.json"}, false), "gitea"},
{"a new module's other files", merge([]string{"modules/newmod/index.ts", "modules/newmod/module.json"}, false), ""},
{"a module removed", merge([]string{"modules/gone/module.json"}, false), ""},
{"a directory with no manifest", merge([]string{"modules/lib/x.go"}, false), "gitea,keycloak"},
{"a file directly among the modules", merge([]string{"modules/README.md"}, false), "gitea,keycloak"},
{"the root's files still", merge([]string{"tsconfig.json"}, false), "gitea,keycloak"},
} {
if got := named(whatTheMergeTouched(candidates, known, c.m)); got != c.want {
t.Errorf("%s: rebuilt %q, wanted %q", c.what, got, c.want)
@@ -219,27 +211,3 @@ func TestWhatAHandedOverModuleRecordsAboutItsSource(t *testing.T) {
}
}
}
// novox/hq 04-ISSUES/215: a module once built at a commit still follows its branch — a merge into it
// matches the module, and a plan re-asks the branch, not the old commit.
func TestAModuleBuiltAtACommitStillFollowsItsBranch(t *testing.T) {
m := link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main"}
pinned := inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Ref: "9c97a8a"}
if !sourceIs(pinned, m) {
t.Error("a module whose record names a commit is left out of a merge into its branch")
}
full := inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Ref: "9c97a8a1d2c3b4a5f60718293a4b5c6d7e8f9012"}
if !sourceIs(full, m) {
t.Error("a full commit hash is read as a branch")
}
if got := followedBranch("9c97a8a"); got != "" {
t.Errorf("a plan would re-ask the old commit %q", got)
}
if got := followedBranch("release"); got != "release" {
t.Errorf("a branch is not followed as named: %q", got)
}
// A module that follows another branch is still not this merge's.
if sourceIs(inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Ref: "release"}, m) {
t.Error("a module following another branch was matched")
}
}
cmd/mesh-controller/plan.go
+15 -201
View File
@@ -8,10 +8,8 @@ import (
"flag"
"fmt"
"os"
"slices"
"sort"
"strings"
"sync"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
@@ -129,7 +127,6 @@ func planFor(ctx context.Context, open *stores, nodeName string) (catalogue.Reso
// a mesh-wide gatherer may pass over — see notResolvable.
return catalogue.Resolution{}, nil, notResolvable{err}
}
logUnheld(nodeName, resolved.Unheld)
// The credential for each thing this node takes from elsewhere. Made once and kept, so the
// password a provider is told to create is the one its consumer was given — and sealed to
@@ -398,7 +395,7 @@ func declarationWith(ctx context.Context, open *stores, node string,
return sendable{}, err
}
return sendable{Resources: composed.Resources, Adoption: adoption,
Received: composed.Received, Mesh: with.Mesh, BusUsers: with.BusUsers,
Received: composed.Received, Mesh: with.Mesh,
LeftOut: sortedKeysOf(composed.LeftOut), leftOutWhy: composed.LeftOut}, nil
}
@@ -660,13 +657,6 @@ func renderingFor(ctx context.Context, open *stores, node string,
machines[name] = at
}
// And every zone a module in the mesh answers itself (novox/hq ADR 0199), for the mesh's resolver
// to forward.
zones, err := zonesInTheMesh(ctx, open)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
// **The bus is never public** (novox/hq ADR 0169). It was a foundation port — widened from the
// broker's own `from: mesh` to from-anywhere on the broker's host, so a machine could enrol
// before it had an address on the private network. A machine joins through the tunnel now, and
@@ -736,79 +726,14 @@ func renderingFor(ctx context.Context, open *stores, node string,
BusMembership: memberships[node],
Settings: settings, Generators: gens, Grants: grants, Needed: needed, Ports: ports,
Certificate: certificate, Authority: authority, Mesh: private, Names: names,
Machines: machines, Zones: zones,
Suffix: overlay.Suffix(), MeshRange: meshRange, TunnelInterface: overlay.Interface, Accounts: accounts, Foundation: foundation,
Machines: machines,
Suffix: overlay.Suffix(), MeshRange: meshRange, TunnelInterface: overlay.Interface, Accounts: accounts, Foundation: foundation,
Kept: kept, Adopted: record.Adopted, OutwardLinks: outwardLinks,
Given: given, Taken: taken, Seats: seats, ArtifactStore: artifactStore, Built: built,
BusUsers: busUsers,
}, record, nil
}
// zonesInTheMesh is every zone a module in the mesh declares, where the mesh placed it (novox/hq ADR
// 0199): the zone settled from that node's settings, the node's private address, the port the
// answering listen is published on there.
//
// Read across every machine's resolution, as the roster once read routed names: a node whose set does
// not compose declares nothing and is passed over, so one broken machine does not cost the rest their
// zones; a store that cannot be read is raised, naming the machine, because returning the zones
// without it would withdraw them from the resolver as if the operator had (novox/hq 04-ISSUES/152).
// What the mesh refuses about the zones together — one declared twice, one shadowing the mesh's
// suffix or a node's public domain — is refused here, by name.
func zonesInTheMesh(ctx context.Context, open *stores) ([]catalogue.ZoneAt, error) {
inv := open.inventory
places, err := inv.Overlays(ctx)
if err != nil {
return nil, fmt.Errorf("where the machines are cannot be read: %w", err)
}
address := map[string]string{}
for _, p := range places {
if strings.TrimSpace(p.Address) != "" {
address[p.Name] = p.Address
}
}
nodes, err := inv.Nodes(ctx)
if err != nil {
return nil, fmt.Errorf("which machines the mesh has cannot be read: %w", err)
}
var zones []catalogue.ZoneAt
var public []string
for _, n := range nodes {
plan, _, err := planFor(ctx, open, n.Name)
switch {
case unresolvable(err):
continue
case err != nil:
return nil, fmt.Errorf("the zones %s answers cannot be read: %w", n.Name, err)
}
if plan.PublicDomain != "" {
public = append(public, plan.PublicDomain)
}
for _, m := range plan.Modules {
if m.Zone == nil {
continue
}
at := address[n.Name]
if at == "" {
// Not on the private network yet: nothing could reach its answerer.
continue
}
published, layers, err := portsGivenOn(ctx, inv, n.Name, m)
if err != nil {
return nil, fmt.Errorf("the zone %s declares on %s cannot be read: %w", m.Module, n.Name, err)
}
z, err := catalogue.ZoneOn(m, layers, published, n.Name, at)
if err != nil {
return nil, err
}
zones = append(zones, *z)
}
}
if problems := catalogue.ZonesProblems(zones, overlay.Suffix(), public); len(problems) > 0 {
return nil, fmt.Errorf("the mesh's zones cannot be forwarded:\n - %s", strings.Join(problems, "\n - "))
}
return zones, nil
}
// certificateFor is what the mesh certifies about one machine's internal name.
//
// It reaches across two contexts and reads neither one's store from the other: `inventory` knows
@@ -980,15 +905,12 @@ func planCommand(ctx context.Context, args []string) error {
// checking it against the host's own parser, most usefully, which is the only way to know
// that what the control plane emits is what the host accepts.
asJSON := set.Bool("json", false, "print the declaration this node would be sent")
// What a push would change, against what the machine was last sent (novox/hq ADR 0217): read
// before sending, so a change that removes, moves or recreates something is seen first.
asDiff := set.Bool("diff", false, "what a push would change on this node, against what it was last sent")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 1 {
return errors.New("plan <node> [--files] [--json] [--diff]")
return errors.New("plan <node> [--files] [--json]")
}
args = positionals
open, err := openStores(ctx)
@@ -1005,17 +927,6 @@ func planCommand(ctx context.Context, args []string) error {
fmt.Printf("%s is assigned nothing\n", args[0])
return nil
}
if *asDiff {
declared, err := declarationFor(ctx, open, args[0], plan, settings)
if err != nil {
return err
}
body, err := declared.Body()
if err != nil {
return err
}
return writePlanDiff(ctx, open.inventory, args[0], body)
}
if *asJSON {
declared, err := declarationFor(ctx, open, args[0], plan, settings)
if err != nil {
@@ -1355,20 +1266,6 @@ func composeBusUsers(ctx context.Context, inv *inventory.Inventory,
//
// Asked of what this push resolves to rather than of the seat's holder mesh-wide: the file is a
// resource of that module, so the question is whether it is here.
list, missing, err := busUserList(ctx, inv, onThisNode)
if len(missing) > 0 {
fmt.Printf("the bus's user list leaves out %d user(s) the mesh has minted no credential "+
"for: %s. Each is a user that cannot connect until one is issued\n",
len(missing), strings.Join(missing, ", "))
}
return list, err
}
// busUserList is composeBusUsers without saying anything: the list, and the users left out of it
// for want of a credential. Asked on every send to decide whether the machine holding the bus must
// go first (novox/hq issue 249), where saying the same missing users each time would bury them.
func busUserList(ctx context.Context, inv *inventory.Inventory,
onThisNode []catalogue.Manifest) (string, []string, error) {
holdsTheBus := false
for _, m := range onThisNode {
if m.BusUsers != "" && m.ClaimsSeat("mesh-broker") {
@@ -1376,33 +1273,37 @@ func busUserList(ctx context.Context, inv *inventory.Inventory,
}
}
if !holdsTheBus {
return "", nil, nil
return "", nil
}
records, err := inv.BusRecords(ctx)
if err != nil {
return "", nil, err
return "", err
}
users, err := broker.Users(records)
if err != nil {
return "", nil, err
return "", err
}
kept, err := inv.BusUsers(ctx)
if err != nil {
return "", nil, err
return "", err
}
hashes := make(map[string]string, len(kept))
for name, u := range kept {
hashes[name] = u.PasswordHash
}
filled, missing := broker.WithPasswords(users, hashes)
if len(missing) > 0 {
fmt.Printf("the bus's user list leaves out %d user(s) the mesh has minted no credential "+
"for: %s. Each is a user that cannot connect until one is issued\n",
len(missing), strings.Join(missing, ", "))
}
if len(filled) == 0 {
return "", missing, fmt.Errorf(
return "", fmt.Errorf(
"this machine runs the bus and not one user has a credential, so the composed list " +
"would refuse every connection in the mesh")
}
list, err := broker.ComposeAccounts(filled)
return list, missing, err
return broker.ComposeAccounts(filled)
}
// providerModuleOf is which module answers a need on the providing node: the one in this node's
@@ -1424,90 +1325,3 @@ func providerModuleOf(resolved catalogue.Resolution, open *stores, ctx context.C
}
return ""
}
// unheldLogged is what was last logged about each node's unmet seat dependencies, so the log says
// each change once (novox/hq ADR 0207), on stderr so `status --json` stays a document.
//
// **The serving controller's log only.** planFor runs for every node on every push, assignment and
// status — `blockedElsewhere` alone resolves the whole mesh — and a one-shot command starts with an
// empty memory, so every node's report was "a change" and a push printed the whole mesh's list,
// burying the line about the node it acted on. A command says what concerns its own act instead
// (unheldChange, reportUnheldPushed); the full list is `status`'s.
var (
unheldLogged = map[string]string{}
unheldLoggedMu sync.Mutex
logUnheldChanges bool
)
// logUnheld logs a node's unmet seat dependencies when they differ from what was last logged for
// it, including when they become none — in the serving controller, and nowhere else.
func logUnheld(node string, unheld []catalogue.Unheld) {
if !logUnheldChanges {
return
}
lines := make([]string, 0, len(unheld))
for _, u := range unheld {
lines = append(lines, u.String())
}
now := strings.Join(lines, "\n")
unheldLoggedMu.Lock()
before, seen := unheldLogged[node]
unheldLogged[node] = now
unheldLoggedMu.Unlock()
if (seen && before == now) || (!seen && now == "") {
return
}
if now == "" {
fmt.Fprintf(os.Stderr, "%s: every seat its modules depend on is held (novox/hq ADR 0207)\n", node)
return
}
fmt.Fprintf(os.Stderr, "%s: %d unmet seat dependenc(ies), reported and not refused (novox/hq ADR 0207):\n %s\n",
node, len(lines), strings.Join(lines, "\n "))
}
// unheldChange is what an act on one node changed about its unmet seat dependencies, judged over
// its assignments before and after (novox/hq ADR 0207): each dependency now unmet that was not —
// which includes every one of a module just assigned — and each now met that was not. Nothing about
// any other node, and nothing that was already true before the act.
func unheldChange(shelf map[string]catalogue.Manifest, node string, before, after []string) []string {
judge := func(names []string) map[string]catalogue.Unheld {
var set []catalogue.Manifest
for _, n := range names {
if m, known := shelf[n]; known {
set = append(set, m)
}
}
out := map[string]catalogue.Unheld{}
for _, u := range catalogue.UnheldDependencies(shelf, node, set, nil) {
out[u.Module+" "+u.Seat] = u
}
return out
}
was, now := judge(before), judge(after)
var lines []string
for _, k := range sortedNames(now) {
if _, already := was[k]; !already {
lines = append(lines, "but "+now[k].String())
}
}
for _, k := range sortedNames(was) {
if _, still := now[k]; still {
continue
}
u := was[k]
if !slices.Contains(after, u.Module) {
continue // went with its module, which says nothing about the seat
}
lines = append(lines, fmt.Sprintf("and %s on %s now has %s held", u.Module, node, u.Seat))
}
return lines
}
func sortedNames[V any](m map[string]V) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
cmd/mesh-controller/plans_one_at_a_time_test.go
-47
View File
@@ -1,47 +0,0 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// novox/hq issue 213: for the moment a machine hands its controller over, the container and the
// process both run the plan timer on one store. Only the one holding the plans moves them; the other
// leaves them alone, and moves them once they are let go.
func TestAControllerLeavesThePlansToTheOneHoldingThem(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
now := time.Now().UTC()
// Every tier done: the next step is the plan's last, and needs nothing but the store.
plan := inventory.Plan{ID: "plan-213", Repository: "r", Commit: "abc", Created: now, Updated: now,
State: inventory.PlanRolling, Tier: 1, Tiers: [][]string{{"app"}},
Modules: map[string]*inventory.PlanModule{"app": {State: "built"}}}
if err := open.inventory.SavePlan(ctx, plan); err != nil {
t.Fatal(err)
}
// The other controller: its own connections to the same store, holding the plans.
other, err := inventory.Open(ctx)
if err != nil {
t.Fatal(err)
}
t.Cleanup(other.Close)
release, err := other.HoldPlans(ctx, false)
if err != nil {
t.Fatal(err)
}
t.Cleanup(release) // before the close above: a pool waits for a connection still held
advancePlans(ctx, open)
if p, err := open.inventory.PlanByID(ctx, "plan-213"); err != nil || !p.Open() {
t.Fatalf("a controller moved a plan another held: %+v %v", p, err)
}
release()
advancePlans(ctx, open)
if p, err := open.inventory.PlanByID(ctx, "plan-213"); err != nil || p.State != inventory.PlanDone {
t.Fatalf("the plan did not move once it was let go: %+v %v", p, err)
}
}
cmd/mesh-controller/push.go
+83 -429
View File
@@ -8,7 +8,6 @@ import (
"errors"
"flag"
"fmt"
"io"
"log"
"os"
"sort"
@@ -63,9 +62,6 @@ func connectLink(ctx context.Context, inv *inventory.Inventory, enroller link.En
}
func serve(ctx context.Context) error {
// The one process whose log is read over time, so the one that says each change to a node's
// unmet seat dependencies once (novox/hq ADR 0207).
logUnheldChanges = true
open, err := openStores(ctx)
if err != nil {
return err
@@ -249,31 +245,14 @@ func pushCommand(ctx context.Context, args []string) error {
// (novox/hq ADR 0010). 0 waits for nothing, which is the old fire-and-forget.
wait := set.Duration("wait", 0,
"for a named node, how long to wait for it to report applying what it was sent (0: do not wait)")
// Every machine at once is said, not defaulted to (novox/hq ADR 0217): a bare `push` sent the
// whole mesh on 2026-10-05 when its usage was wanted.
all := set.Bool("all", false, "every machine")
// A running module's data moving is sent only when said, per module (novox/hq ADR 0217).
move := set.String("move", "", "modules whose data may move with this push, comma-separated")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
args = positionals
if len(args) > 1 {
return errors.New("push <node> | push --behind | push --all — one machine, the ones behind, or all of them")
return errors.New("push [<node>] [--behind] — one node, or all of them")
}
if len(args) == 0 && !*behind && !*all {
return errors.New("push names a machine, or says --behind (the ones not running what they " +
"should) or --all (every machine) — a push to the whole mesh is not the default (novox/hq ADR 0217)")
}
if *all && (*behind || len(args) == 1) {
return errors.New("push --all is every machine; it takes no machine and no --behind")
}
movable := map[string]bool{}
for _, m := range splitModules(*move) {
movable[m] = true
}
var heldBack []string
if len(args) == 1 && *behind {
// Naming a machine and asking for the ones that need it are two different requests, and
// guessing which was meant would sometimes push to a machine somebody did not name.
@@ -375,26 +354,12 @@ func pushCommand(ctx context.Context, args []string) error {
asked = append(asked, n.Name)
}
// **The machine holding the bus first** (novox/hq issue 249): its declaration carries the bus's
// user list, and a module's new grants are refused by the bus until that list says them. Among
// the machines asked it goes first; not among them and behind, it is added — a named push whose
// module gained a state would otherwise send the code and leave the right to use it for the
// cascade below, after.
holder, holderBehind, err := brokerBehind(ctx, open, asked)
if err != nil {
return err
}
asked = brokerFirst(asked, holder, holderBehind)
// Held from composing to sending, so a converge on one of them cannot send between the two
// and be overtaken by what was composed before it (novox/hq ADR 0100).
held, release, err := holdNodes(ctx, open, asked)
if err != nil {
return err
}
// Each machine's unmet seat dependencies (novox/hq ADR 0207), said after the sends: in full
// for a machine named, as a count for each of many — the full list is `status`'s.
unheld := map[string][]catalogue.Unheld{}
sending, refusals := composeEach(asked, allotting(held, inv), func(node string) (sendable, error) {
plan, settings, err := planFor(held, open, node)
if err != nil {
@@ -404,7 +369,6 @@ func pushCommand(ctx context.Context, args []string) error {
// healthy modules beside it are still resolved and sent. Reported so it is not silently
// dropped — the remedy is to move it, and until then the rest of the node converges.
reportUnhostable(node, plan)
unheld[node] = plan.Unheld
// The private network is in here with everything else. It used to be composed separately
// and prepended, which meant every machine with an address was on it and no machine could
// be kept off. It is a module now, so it arrives the way a module does.
@@ -415,22 +379,34 @@ func pushCommand(ctx context.Context, args []string) error {
return declared, err
})
sentDigest := map[string]string{}
defer release()
// A machine whose declaration would move a running module's data is held, and only it (novox/hq
// ADR 0217); every other machine is delivered, its memberships first, then its declaration
// (novox/hq issue 249, ADR 0218).
bus := overTheBus{open: open, server: server, signer: ident}
toSend, err := holdingBack(ctx, inv, sending, movable, &heldBack)
if err != nil {
return err
}
sentDigest, err := deliver(ctx, bus, holder, toSend)
if err != nil {
return err
for _, s := range sending {
// The number is inside the signed bytes, so a replayed older declaration cannot borrow a
// newer one's (novox/hq 04-ISSUES/107); it was taken when the composition began (issue 204).
body, err := s.declared.Body()
if err != nil {
return err
}
if err := link.Declare(ctx, server.Bus(), ident, s.node, body, 15*time.Second); err != nil {
return err
}
// After it is away, not before. A digest recorded for something that failed to send would
// make the machine look current for a declaration it never received.
digest, err := recordSent(ctx, inv, s.node, body)
if err != nil {
return err
}
sentDigest[s.node] = digest
fmt.Printf("sent %s %d resource(s)\n", s.node, len(s.declared.Resources))
}
release()
fmt.Printf("\n%d node(s) told\n", len(sending))
reportUnheldPushed(os.Stdout, len(args) == 1, asked, unheld)
// And each machine's memberships, as every other send does (ADR 0160): a push is the one most
// operators run, and on 2026-10-01 it was the one path that issued none.
if err := issueMemberships(ctx, open, server, sending); err != nil {
return err
}
// **A named push leaves the mesh consistent, not just the machine it named** (novox/hq
// issue 057, ADR 0083). Assigning a cross-node consumer mints a provision, and the PROVIDER's
@@ -499,9 +475,16 @@ func pushCommand(ctx context.Context, args []string) error {
}
return declared, err
},
bus, holder, func(held context.Context, sending []readyNode) ([]readyNode, error) {
// The cascade is a push too, and held the same way (novox/hq ADR 0217).
return holdingBack(held, inv, sending, movable, &heldBack)
func(s readyNode, body []byte) error {
if err := link.Declare(ctx, server.Bus(), ident, s.node, body,
15*time.Second); err != nil {
return err
}
if _, err := recordSent(ctx, inv, s.node, body); err != nil {
return err
}
fmt.Printf("sent %s %d resource(s)\n", s.node, len(s.declared.Resources))
return nil
})
refusals = append(refusals, refused...)
if err != nil {
@@ -524,11 +507,6 @@ func pushCommand(ctx context.Context, args []string) error {
return err
}
}
if len(heldBack) > 0 {
sort.Strings(heldBack)
return fmt.Errorf("held %s: a running module's data would move — see above; nothing was sent "+
"there (novox/hq ADR 0217)", strings.Join(heldBack, ", "))
}
return couldNotBeResolved(refusals, len(sending))
}
@@ -634,10 +612,10 @@ func composeEach(names []string, allot func(node string) (int64, error),
// sendRound holds the named nodes, composes each and sends each that composed, and gives the hold
// back on every way out — a body that cannot be marshalled and a send that fails included
// (novox/hq ADR 0100). A node that cannot be composed is a refusal, not an error: the others are
// still sent. Their memberships go before their declarations, as every send's do (issue 249).
// still sent.
func sendRound(ctx context.Context, open *stores, names []string,
compose func(held context.Context, node string) (sendable, error),
d delivery, holder string, keep func(context.Context, []readyNode) ([]readyNode, error)) ([]string, error) {
send func(s readyNode, body []byte) error) ([]string, error) {
held, release, err := holdNodes(ctx, open, names)
if err != nil {
return nil, err
@@ -646,276 +624,18 @@ func sendRound(ctx context.Context, open *stores, names []string,
sending, refused := composeEach(names, allotting(held, open.inventory), func(node string) (sendable, error) {
return compose(held, node)
})
if keep != nil {
if sending, err = keep(held, sending); err != nil {
for _, s := range sending {
body, err := s.declared.Body()
if err != nil {
return refused, err
}
if err := send(s, body); err != nil {
return refused, err
}
}
if _, err := deliver(held, d, holder, sending); err != nil {
return refused, err
}
return refused, nil
}
// holdingBack leaves out each machine whose declaration would move a running module's data, says so,
// and names it among those held back (novox/hq ADR 0217); the rest go on to be delivered, grants first
// (ADR 0218). A declaration's body is the same bytes however often it is read.
func holdingBack(ctx context.Context, inv *inventory.Inventory, sending []readyNode, movable map[string]bool,
heldBack *[]string) ([]readyNode, error) {
var out []readyNode
for _, s := range sending {
body, err := s.declared.Body()
if err != nil {
return nil, err
}
moves, err := heldMoves(ctx, inv, s.node, body, movable)
if err != nil {
return nil, err
}
if len(moves) > 0 {
sayHeld(os.Stdout, s.node, moves)
*heldBack = append(*heldBack, s.node)
continue
}
out = append(out, s)
}
return out, nil
}
// delivery is the two acts of sending machines what they should be, apart, so the order between
// them is one function's and can be read and tested there (novox/hq issue 249).
type delivery interface {
// grant issues what the machines' modules may do — each module's state raised and its
// membership issued — for every machine about to be sent.
grant(ctx context.Context, sending []readyNode) error
// declare sends one machine its declaration and records it sent, answering the digest.
declare(ctx context.Context, s readyNode, body []byte) (string, error)
}
// errGrants marks a send that stopped because what the machines' modules may do could not be issued
// (novox/hq issue 249). Nothing about the machines is wrong; asked again, it is likely to work, so an
// announcement that hits it is held and asked again.
var errGrants = errors.New("what the machines' modules may do on the bus could not be issued, and code " +
"sent before its grants is refused there")
// grantsRefused is a grant that failed for some machines and not others: their memberships could not
// be issued, by machine, and only those machines are held back.
type grantsRefused struct{ nodes map[string]error }
func (g *grantsRefused) Error() string {
names := make([]string, 0, len(g.nodes))
for n := range g.nodes {
names = append(names, n)
}
sort.Strings(names)
return fmt.Sprintf("the memberships of %s could not be issued; the first: %v",
strings.Join(names, ", "), g.nodes[names[0]])
}
// deliver sends the machines their declarations: **the machine holding the bus, then the grants,
// then the rest** (novox/hq issue 249).
//
// A merge gave a module a new state; its bundle reached every machine within a minute, and the
// machines' permissions on the bus did not include the state until somebody pushed by hand: the code
// arrived before the right to use it. A module that read its new state on start failed its start; the
// one that was there retried for two minutes. The memberships were issued after the declarations —
// "because the runtime it is for arrives with it" — and a membership is retained last-per-subject on
// the bus (internal/link/bus.go), so issued first it waits for the runtime that arrives after it. A
// runtime still on the old code merely holds a grant it does not use yet.
//
// **The holder's declaration before the grants, though.** The bus's user list travels in it, and the
// controller's own right to publish memberships and raise buckets is in that list (the precedent of
// issue 183): grants first, and a grant the controller is not yet allowed to make would hold the very
// declaration that allows it — a lock only a hand on the broker could open. A runtime already running
// on that machine follows a membership issued after its declaration, as it always has.
//
// **A grant that cannot be issued holds back what it concerns, and says so as an error.** It used to
// be said and passed over — "the machines keep what they derive until the next push" — which reported
// a rollout done that had delivered code its machines could not run. A membership that failed holds
// back its own machine; a failure that names no machine (the buckets) holds back every machine but the
// holder, already sent. The error carries errGrants, so the caller's rollout is not marked sent and is
// tried again.
//
// The grants are issued, not waited on: a membership is a retained message the runtime reads when it
// comes, and the bus answers its publication; nothing here waits for a runtime to have read one.
func deliver(ctx context.Context, d delivery, holder string, sending []readyNode) (map[string]string, error) {
digests := map[string]string{}
if len(sending) == 0 {
return digests, nil
}
send := func(s readyNode) error {
// The number is inside the signed bytes, so a replayed older declaration cannot borrow a
// newer one's (novox/hq 04-ISSUES/107); it was taken when the composition began (issue 204).
body, err := s.declared.Body()
if err != nil {
return err
}
digest, err := d.declare(ctx, s, body)
if err != nil {
return err
}
digests[s.node] = digest
return nil
}
var rest []readyNode
for _, s := range sending {
if holder != "" && s.node == holder {
if err := send(s); err != nil {
return digests, err
}
continue
}
rest = append(rest, s)
}
held := map[string]error{}
if err := d.grant(ctx, sending); err != nil {
var some *grantsRefused
if !errors.As(err, &some) {
var names []string
for _, s := range rest {
names = append(names, s.node)
}
if len(names) == 0 {
return digests, fmt.Errorf("%w: %w", errGrants, err)
}
return digests, fmt.Errorf("%w; %s not sent: %w", errGrants, strings.Join(names, ", "), err)
}
held = some.nodes
}
var notSent []string
for _, s := range rest {
if _, refused := held[s.node]; refused {
notSent = append(notSent, s.node)
continue
}
if err := send(s); err != nil {
return digests, err
}
}
if len(notSent) > 0 {
return digests, fmt.Errorf("%w; %s not sent: %w", errGrants, strings.Join(notSent, ", "),
&grantsRefused{nodes: held})
}
if len(held) > 0 {
// Only the holder's own memberships failed, and it was sent before them.
return digests, fmt.Errorf("%w: %w", errGrants, &grantsRefused{nodes: held})
}
return digests, nil
}
// overTheBus is delivery as the mesh does it: memberships on the bus, declarations signed.
type overTheBus struct {
open *stores
server *link.Server
signer link.Signer
// indent is put before each "sent" line, for the callers whose output is nested.
indent string
}
func (b overTheBus) grant(ctx context.Context, sending []readyNode) error {
return issueMemberships(ctx, b.open, b.server, sending)
}
func (b overTheBus) declare(ctx context.Context, s readyNode, body []byte) (string, error) {
if err := link.Declare(ctx, b.server.Bus(), b.signer, s.node, body, 15*time.Second); err != nil {
return "", err
}
// After it is away, not before. A digest recorded for something that failed to send would make
// the machine look current for a declaration it never received.
digest, err := recordSent(ctx, b.open.inventory, s.node, body)
if err != nil {
return "", err
}
if s.declared.BusUsers != "" {
// And the user list it carried, so the next send reads whether it must go first from the
// list alone (novox/hq issue 249). On the same outliving context as the send's record.
kept, cancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
err := b.open.inventory.RecordSentBusUsers(kept, s.node, digestOf([]byte(s.declared.BusUsers)))
cancel()
if err != nil {
return "", err
}
}
fmt.Printf("%ssent %s %d resource(s)\n", b.indent, s.node, len(s.declared.Resources))
return digest, nil
}
// brokerFirst is the machines to send in the order a grant needs (novox/hq issue 249): the machine
// holding the bus first — its declaration carries the bus's user list (composeBusUsers), and a
// module's new permissions are refused by the bus until that list says them. Among the machines it
// is moved to the front; not among them, it is added only when it is behind.
func brokerFirst(names []string, holder string, behind bool) []string {
if holder == "" {
return names
}
present := false
rest := make([]string, 0, len(names))
for _, n := range names {
if n == holder {
present = true
continue
}
rest = append(rest, n)
}
if !present && !behind {
return names
}
return append([]string{holder}, rest...)
}
// brokerBehind is the machine holding the bus — the one whose declaration carries the user list —
// and, when it is not among the machines named, whether the user list it would be sent now differs
// from the one it was last sent (novox/hq issue 249).
//
// **The user list alone, not the whole declaration.** Read from the whole declaration, any change
// pending on that machine — an upgrade its policy records rather than rolls out — went with every
// send anywhere, and a module running there always put it in its first wave. A digest of the list
// last sent is kept for this (ADR 0043: the list is composed on each push, never kept itself).
func brokerBehind(ctx context.Context, open *stores, names []string) (string, bool, error) {
inv := open.inventory
holders, err := seatHolders(ctx, inv)
if err != nil {
return "", false, err
}
h, held := holders[theBrokerSeat]
if !held || h.Node == "" {
return "", false, nil
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return "", false, err
}
if m, known := shelf[h.Module]; !known || m.BusUsers == "" {
// A holder that is sent no user list carries no grant: nothing to send first.
return "", false, nil
}
for _, n := range names {
if n == h.Node {
return h.Node, false, nil
}
}
plan, _, err := planFor(ctx, open, h.Node)
if err != nil {
// It cannot be worked out: sending it would refuse the whole send, and `plan` says why.
return h.Node, false, nil
}
list, _, err := busUserList(ctx, inv, plan.Modules)
if err != nil {
return h.Node, false, nil
}
sent, err := inv.SentBusUsers(ctx, h.Node)
if err != nil {
return "", false, err
}
return h.Node, userListBehind(list, sent), nil
}
// userListBehind is whether the user list composed now is not the one last sent, by its digest. An
// empty list composed is never behind: there is nothing for it to carry.
func userListBehind(now, sentDigest string) bool {
return now != "" && digestOf([]byte(now)) != sentDigest
}
// couldNotBeResolved is what a push ends with when some machines could not be worked out.
//
// **After the rest have been sent, never instead of sending them.** It is still an error, because
@@ -938,36 +658,23 @@ func couldNotBeResolved(refusals []string, sent int) error {
// consumer and refused on the provider would leave one end holding a credential the other has
// never heard of — which is the state this whole mechanism exists to make impossible.
func sendTo(ctx context.Context, open *stores, names []string) error {
_, err := sendToEach(ctx, open, names)
return err
}
// sendToEach is sendTo, answering the machines it sent: those named, and before them the machine
// holding the bus when its user list must go first (novox/hq issue 249) — so a caller that waits for
// the machines it sent waits for that one too.
func sendToEach(ctx context.Context, open *stores, names []string) ([]string, error) {
inv := open.inventory
ident, err := openIdentity(ctx)
if err != nil {
return nil, err
return err
}
defer ident.Close()
gens, err := generators(ctx, open)
if err != nil {
return nil, err
return err
}
holder, behind, err := brokerBehind(ctx, open, names)
if err != nil {
return nil, err
}
names = brokerFirst(names, holder, behind)
// Held from composing to sending (novox/hq ADR 0100); a caller that holds them already —
// converge, which flips the node and then sends it — is not made to wait on itself.
ctx, release, err := holdNodes(ctx, open, names)
if err != nil {
return nil, err
return err
}
defer release()
@@ -996,29 +703,33 @@ func sendToEach(ctx context.Context, open *stores, names []string) ([]string, er
sending = append(sending, readyNode{name, declared})
}
if len(refusals) > 0 {
return nil, fmt.Errorf("nothing was sent. %d machine(s) could not be resolved:\n\n%s",
return fmt.Errorf("nothing was sent. %d machine(s) could not be resolved:\n\n%s",
len(refusals), strings.Join(refusals, "\n\n"))
}
server, err := connectLink(ctx, nil, nil, nil)
if err != nil {
return nil, err
return err
}
defer server.Close()
for _, s := range sending {
body, err := s.declared.Body()
if err != nil {
return err
}
if err := link.Declare(ctx, server.Bus(), ident, s.node, body, 15*time.Second); err != nil {
return err
}
if _, err := recordSent(ctx, inv, s.node, body); err != nil {
return err
}
fmt.Printf(" sent %s %d resource(s)\n", s.node, len(s.declared.Resources))
}
// And every assignment on those machines its membership (novox/hq ADR 0160): composed from the
// same records the bus's accounts are, so what a runtime serves and what its account may are one
// composition. **Issued before the declarations** (novox/hq issue 249): the runtime the
// membership is for arrives with the declaration, and a membership waits for it on the bus; the
// code arriving first was refused its own state until somebody pushed.
if _, err := deliver(ctx, overTheBus{open: open, server: server, signer: ident, indent: " "}, holder, sending); err != nil {
return nil, err
}
sent := make([]string, 0, len(sending))
for _, s := range sending {
sent = append(sent, s.node)
}
return sent, nil
// composition. Issued after the declaration, because the runtime it is for arrives with it.
return issueMemberships(ctx, open, server, sending)
}
// issueMemberships publishes the membership of every module on the machines just sent.
@@ -1036,26 +747,11 @@ func issueMemberships(ctx context.Context, open *stores, server *link.Server, se
if !ok {
return nil
}
// **Every declared state's bucket, before the memberships that name it** (novox/hq ADR 0201). The
// raise at start asserts them too, but a module registered and assigned since would otherwise have
// its bucket only after the control plane next restarts — found the first time a module declared
// state: its bundle asked for a bucket that did not exist. Idempotent and cheap.
//
// **A failure here is the send's failure** (novox/hq issue 249). It was said and the push stood,
// because the declarations were already away; they are sent after this now — all but the bus's
// own machine, sent before it (deliver) — and a module whose state does not exist is a module
// that fails its start, so they are not sent and the caller tries again rather than reporting the
// rollout done.
buckets, err := open.inventory.DeclaredBuckets(ctx)
if err != nil {
return fmt.Errorf("the modules' state could not be read, so no bucket was asserted: %w", err)
}
if _, err := broker.RaiseBuckets(broker.OnConn(bus.Conn), buckets); err != nil {
return fmt.Errorf("the modules' state could not be asserted on the bus: %w", err)
}
// Every membership is tried, and the first failure named once.
issued := 0
refused := map[string]error{}
// The declarations are sent and recorded by now; a membership that cannot be issued is said
// and does not unsay them. Every runtime without one serves the shape it derives (ADR 0160), so
// the push stands, the first failure is named once, and the next push tries again.
issued, failed := 0, 0
var first error
for _, s := range sent {
node := s.node
for _, d := range records.Assigned[node] {
@@ -1076,9 +772,10 @@ func issueMemberships(ctx context.Context, open *stores, server *link.Server, se
return err
}
if err := bus.PublishMembership(ctx, node, d.Module, body); err != nil {
if refused[node] == nil {
refused[node] = fmt.Errorf("%s: %w", d.Module, err)
if first == nil {
first = err
}
failed++
continue
}
issued++
@@ -1087,10 +784,9 @@ func issueMemberships(ctx context.Context, open *stores, server *link.Server, se
if issued > 0 {
fmt.Printf(" issued %d membership(s)\n", issued)
}
if len(refused) > 0 {
// Returned, never passed over (novox/hq issue 249): the declarations of the machines they are
// for are not sent, and the rollout that asked is tried again rather than waiting for a push.
return &grantsRefused{nodes: refused}
if failed > 0 {
fmt.Printf(" %d membership(s) could not be issued; the first: %v — the machines keep what "+
"they derive until the next push\n", failed, first)
}
return nil
}
@@ -1202,21 +898,6 @@ func raiseTheBus(ctx context.Context, inv *inventory.Inventory, address string)
if err := broker.RaiseSeats(js, inventory.MeshSeats(), holders); err != nil {
return err
}
// Every module's state (novox/hq ADR 0201), from the catalogue: a bucket exists from
// registration, so a module reading one may watch it before its owner runs anywhere. One that
// nothing declares any more is said and kept — what it holds is data.
buckets, err := inv.DeclaredBuckets(ctx)
if err != nil {
return err
}
undeclared, err := broker.RaiseBuckets(js, buckets)
if err != nil {
return err
}
if len(undeclared) > 0 {
fmt.Printf("the bus holds state nothing declares any more, kept because it is data: %s — "+
"removing it is a person's act\n", strings.Join(undeclared, ", "))
}
// And how every module hears what it consumes. Derived from the same records the user list is
// composed from, so a module the mesh grants a consumer's subjects has that consumer waiting.
// Done on every raise, not only when a credential is issued: every module moved onto this bus
@@ -1230,14 +911,18 @@ func raiseTheBus(ctx context.Context, inv *inventory.Inventory, address string)
return err
}
hearing := 0
for _, c := range broker.ConsumersOf(users) {
if err := js.EnsureConsumer(c.Consumer); err != nil {
return fmt.Errorf("how %s on %s hears what it consumes: %w", c.Module, c.Node, err)
for _, p := range users {
consumer, needed := broker.ConsumerFor(p)
if !needed {
continue
}
if err := js.EnsureConsumer(consumer); err != nil {
return fmt.Errorf("how %s on %s hears what it consumes: %w", p.Module, p.Node, err)
}
hearing++
}
fmt.Printf("the bus at %s has its streams, %d machine(s) can hear a declaration, %d module(s) "+
"can hear what they consume, and %d bucket(s) of state\n", broker.BareAddress(address), len(names), hearing, len(buckets))
fmt.Printf("the bus at %s has its streams, %d machine(s) can hear a declaration, and %d module(s) "+
"can hear what they consume\n", broker.BareAddress(address), len(names), hearing)
return nil
}
@@ -1310,36 +995,5 @@ func recordSent(ctx context.Context, inv *inventory.Inventory, node string, body
if err := inv.RecordSent(kept, record.ID, digest); err != nil {
return "", err
}
// And what it was, summarised, so the next push can be compared with it (novox/hq ADR 0217).
// Never a reason to fail a send that is already away: a summary that cannot be kept means the
// next comparison has nothing to hold against, which is how every machine starts.
if summary, err := summarize(body); err == nil {
if raw, err := json.Marshal(summary); err == nil {
if err := inv.RecordSentSummary(kept, record.ID, raw); err != nil {
fmt.Fprintf(os.Stderr, "%s: what it was sent is not kept for comparison: %v\n", node, err)
}
}
}
return digest, nil
}
// reportUnheldPushed says what a push's machines lack of the seats their modules depend on
// (novox/hq ADR 0207): every line for a machine the push named, since that is the machine somebody
// is looking at, and one line per machine otherwise — a list per machine across the mesh is the
// hundred lines that buried the one that mattered. Nothing for a machine that lacks nothing.
func reportUnheldPushed(w io.Writer, named bool, asked []string, unheld map[string][]catalogue.Unheld) {
for _, node := range asked {
lines := unheld[node]
if len(lines) == 0 {
continue
}
if named {
fmt.Fprintf(w, "\n%s has %d unmet seat dependenc(ies) (novox/hq ADR 0207):\n", node, len(lines))
for _, u := range lines {
fmt.Fprintf(w, " %s\n", u)
}
continue
}
fmt.Fprintf(w, "%s: %d unmet seat dependenc(ies) — see `status`\n", node, len(lines))
}
}
cmd/mesh-controller/readable.go
-6
View File
@@ -3,7 +3,6 @@ package main
import (
"encoding/json"
"fmt"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"sort"
"time"
@@ -74,10 +73,6 @@ type meshStatus struct {
// alone. A document without this called a machine well while a predecessor's chain refused
// what the mesh declared open.
Filtered []machineFiltered `json:"filtered,omitempty"`
// Unheld is every module on a machine whose resources are applied through a seat nothing on
// that machine holds, with the modules that could hold it (novox/hq ADR 0207). Absent when every
// dependency is met. Reported, not refused, until the switch.
Unheld []catalogue.Unheld `json:"unheld,omitempty"`
}
// machineFiltered is one rule set on a converged machine that the mesh did not write and that
@@ -209,7 +204,6 @@ func statusAsJSON(asked answers) ([]byte, error) {
out.Filtered = append(out.Filtered, machineFiltered{Node: name, Where: x.Where, Owner: x.Owner, Refuses: x.Refuses})
}
}
out.Unheld = asked.unheld
for name := range asked.refused {
out.Unresolved = append(out.Unresolved, machineUnresolved{
Node: name, Problem: asked.refused[name]})
cmd/mesh-controller/release_plan.go
+19 -403
View File
@@ -2,7 +2,6 @@ package main
import (
"context"
"errors"
"flag"
"fmt"
"sort"
@@ -184,7 +183,6 @@ func planOfMerge(m link.SourceMoved, moved []string, edges []inventory.Edge) inv
return inventory.Plan{
ID: fmt.Sprintf("plan-%d", time.Now().UnixNano()),
Repository: m.Owner + "/" + m.Repo,
Branch: m.Base,
Commit: m.Commit,
Created: time.Now().UTC(),
State: inventory.PlanBuilding,
@@ -193,57 +191,6 @@ func planOfMerge(m link.SourceMoved, moved []string, edges []inventory.Edge) inv
}
}
// supersededBy is what a newer plan takes over from the open plans it supersedes (novox/hq issue
// 254, ADR 0218): the modules they had not finished, and those plans closed as superseded.
//
// **A merge looked at no plan but its own.** Two merges of one repository a few minutes apart were
// two open plans asking for the same modules, each sending machines what it built; and a plan that
// would never move again — waiting on a report that could not come, at 97b1b2b — stayed open for
// ever beside the newer ones, read as work in progress by everyone who looked. The newer merge is the
// newer intent for that repository and branch, so its plan takes over: every open plan of the same
// repository and branch **created before it** — by the time the plans were made, never by comparing
// commits, which have no order of their own — gives up the modules it had not built, and those are
// planned again in the newer plan beside what the newer merge moved.
//
// "Not built" is a module not yet asked, or asked and not answered; **and a module built and not
// yet sent to its machines**, where its policy rolls it out: closed, the older plan would never send
// it, and the catalogue announces no move for a rebuild (issue 189), so the newer plan builds and
// sends it. A build the older plan asked still finishes and registers as any build does — ordered by
// when it was asked (issue 219), so the newer plan's ask, made later, is the one that stands.
//
// A plan with no branch recorded is from before branches were kept, and is superseded by the next
// plan of its repository: what it had not built is folded in, so nothing is lost by it.
func supersededBy(newer inventory.Plan, open []inventory.Plan, rollsOut func(string) bool) ([]string, []inventory.Plan) {
folded := map[string]bool{}
var closed []inventory.Plan
for _, old := range open {
if old.ID == newer.ID || !old.Open() || !strings.EqualFold(old.Repository, newer.Repository) ||
(old.Branch != "" && old.Branch != newer.Branch) || !old.Created.Before(newer.Created) {
continue
}
var took []string
for name, s := range old.Modules {
if s == nil || s.State != "built" || (s.SentAt == nil && rollsOut(name)) {
folded[name] = true
took = append(took, name)
}
}
sort.Strings(took)
old.State = inventory.PlanSuperseded
old.Note = fmt.Sprintf("superseded at tier %d by %s (%s at %s)", old.Tier, newer.ID, newer.Repository, short(newer.Commit))
if len(took) > 0 {
old.Note += "; " + strings.Join(took, ", ") + " planned there again"
}
closed = append(closed, old)
}
out := make([]string, 0, len(folded))
for name := range folded {
out = append(out, name)
}
sort.Strings(out)
return out, closed
}
// gates is what the next tier needs running from this one: a module of the tier that a later
// tier is built by — the runtime dependency — and whose policy rolls it out, must be applied by
// the machines running it before the next tier is asked. A base an image stands on need only be
@@ -282,22 +229,6 @@ func gates(p inventory.Plan, edges []inventory.Edge, rollsOut func(string) bool)
}
// applied says whether every machine running the module has reported since the module was built.
// appliedEach is applied with a moment of its own for each machine: the reports that count are the ones
// after that machine was sent the build (novox/hq issue 256).
func appliedEach(module string, since func(node string) time.Time, running []string, reports []inventory.Reported) (bool, []string) {
at := map[string]*time.Time{}
for _, r := range reports {
at[r.Node] = r.At
}
var waiting []string
for _, n := range running {
if t := at[n]; t == nil || t.Before(since(n)) {
waiting = append(waiting, n)
}
}
return len(waiting) == 0, waiting
}
func applied(module string, builtAt time.Time, running []string, reports []inventory.Reported) (bool, []string) {
at := map[string]*time.Time{}
for _, r := range reports {
@@ -341,8 +272,7 @@ func askTier(ctx context.Context, inv *inventory.Inventory, p *inventory.Plan) e
}
source := buildSource{Repository: e.Source.Repository, Seat: e.Source.Seat}
fmt.Printf(" tier %d: ", p.Tier)
// The branch it follows, never a commit a build once named (novox/hq 04-ISSUES/215).
if err := buildOne(ctx, source, e.Source.Path, followedBranch(e.Source.Ref), 0); err != nil {
if err := buildOne(ctx, source, e.Source.Path, e.Source.Ref, 0); err != nil {
state.State = "failed"
state.Why = err.Error()
p.State = inventory.PlanFailed
@@ -357,23 +287,8 @@ func askTier(ctx context.Context, inv *inventory.Inventory, p *inventory.Plan) e
// planBuilt marks a module built (or failed) in every open plan whose current tier holds it, and
// advances what that completes. Called from the daemon's take-in of every outcome.
//
// **Only a build asked at or after the plan's ask is its outcome** (novox/hq 04-ISSUES/219). Two
// plans a few minutes apart both ask for a module; the earlier plan's build, finishing late, is not
// the later plan's answer — it stood on the bases from before the later plan's merge, and taking it
// would send machines, and the next tier, what the later merge replaced. asked is zero when the
// build's request time is not known, and such an outcome is taken as before.
func planBuilt(ctx context.Context, open *stores, module, commit, failed string, asked time.Time) {
func planBuilt(ctx context.Context, open *stores, module, commit, failed string) {
inv := open.inventory
// One controller works the plans at a time (novox/hq issue 213); an outcome waits its turn rather
// than write over what the holder is about to save. Not taken, it is still in the build records,
// which the holder settles the plan from (issue 214).
release, err := inv.HoldPlans(ctx, true)
if err != nil {
fmt.Printf("plans: %s's outcome is left to the build records: %v\n", module, err)
return
}
defer release()
plans, err := inv.OpenPlans(ctx)
if err != nil {
fmt.Printf("plans: cannot read them: %v\n", err)
@@ -399,18 +314,11 @@ func planBuilt(ctx context.Context, open *stores, module, commit, failed string,
state = &inventory.PlanModule{}
p.Modules[module] = state
}
if askedBefore(asked, state.AskedAt) {
continue
}
if failed != "" {
state.State = "failed"
state.Why = failed
p.State = inventory.PlanFailed
p.Note = fmt.Sprintf("%s failed to build in tier %d", module, p.Tier)
sayUnsent(p, func(m string) bool {
u, err := inv.UpgradeOf(ctx, m)
return err == nil && u.RollOut
})
} else {
state.State = "built"
state.BuiltAt = &now
@@ -424,30 +332,13 @@ func planBuilt(ctx context.Context, open *stores, module, commit, failed string,
fmt.Printf("%s: %s; the tiers after it are not asked\n", p.ID, p.Note)
}
}
advanceHeld(ctx, open)
advancePlans(ctx, open)
}
// advancePlans moves every open plan as far as the facts allow: a tier whose modules are all built
// and whose gates are applied gives way to the next; the last tier done is the plan done. Called
// after every outcome and on a timer, so a plan waiting on a machine's report moves when it comes.
//
// **One controller at a time** (novox/hq issue 213). A plan is read, changed and saved whole; two
// controllers — the old and the new while a machine hands its controller over — would each ask a
// tier the other had just asked. Taken without waiting: whoever holds the plans is moving them.
func advancePlans(ctx context.Context, open *stores) {
release, err := open.inventory.HoldPlans(ctx, false)
if err != nil {
if !errors.Is(err, inventory.ErrPlansBusy) {
fmt.Printf("plans: cannot hold them: %v\n", err)
}
return
}
defer release()
advanceHeld(ctx, open)
}
// advanceHeld is advancePlans for a caller already holding the plans.
func advanceHeld(ctx context.Context, open *stores) {
inv := open.inventory
plans, err := inv.OpenPlans(ctx)
if err != nil {
@@ -472,17 +363,8 @@ func advanceHeld(ctx context.Context, open *stores) {
moved, err := advanceOnce(ctx, open, p, edges, rollsOut)
if err != nil {
fmt.Printf("%s: %v\n", p.ID, err)
// Kept in the plan, so `plans` says why it has not moved rather than the log alone;
// the state is left as it was and the step is tried again on the next tick.
p.Note = "tier " + fmt.Sprint(p.Tier) + ": " + err.Error() + " — tried again"
if err := inv.SavePlan(ctx, *p); err != nil {
fmt.Printf("%s: cannot keep the plan: %v\n", p.ID, err)
}
break
}
if p.State == inventory.PlanFailed {
sayUnsent(p, rollsOut)
}
if err := inv.SavePlan(ctx, *p); err != nil {
fmt.Printf("%s: cannot keep the plan: %v\n", p.ID, err)
break
@@ -517,24 +399,6 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
}
return true, nil
}
// **Asked: settle from the build records first** (novox/hq 04-ISSUES/214). An outcome is taken
// in by whichever controller hears it, and a merge to the controller's own repository replaces
// the controller in its first tier: the build that produced the new one is recorded, and the
// plan never hears it. The record is the fact; a build recorded after the ask is that tier's
// outcome, whoever was listening.
recorded := map[string][]inventory.Build{}
for _, m := range tier {
if s := p.Modules[m]; s != nil && s.State == "asked" {
builds, err := inv.Builds(ctx, m, 5)
if err != nil {
return false, err
}
recorded[m] = builds
}
}
if settleFromRecords(p, tier, recorded) {
return true, nil
}
// Asked: wait for every build.
var latest time.Time
for _, m := range tier {
@@ -551,15 +415,6 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
// a module that packages another repository's source, keeps its commit; the catalogue announces
// no move for it and its machines would keep the old image until somebody pushed (novox/hq
// issue 189). A module whose policy records is built and left, as its policy says.
//
// **One machine first, unless the module's policy says together** (novox/hq issue 249, ADR
// 0218). The plan sent every machine running the module at once, and the operator's policy —
// one at a time, stopping at the first that fails, which an announced upgrade honours — was not
// read here at all: a module whose new declarations broke it broke everywhere in the same
// minute. Now the first machine is sent, the plan records it and waits for that machine's report
// after the send to say it applied what it was sent; only then are the rest sent. A first machine
// that fails or refuses stops the module's rollout and the plan with it, the rest untouched.
var pending []string
for _, m := range tier {
state := p.Modules[m]
if state == nil || state.SentAt != nil || !rollsOut(m) {
@@ -569,64 +424,17 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
if err != nil {
return false, err
}
policy, err := inv.UpgradeOf(ctx, m)
if err != nil {
return false, err
}
var reports []inventory.Reported
if !policy.Together {
// Read for the choice of the first machine as well as for its report.
if reports, err = inv.LastReports(ctx); err != nil {
return false, err
}
}
now := time.Now().UTC()
step := nextRollout(*state, running, policy.Together, reports, now, planWaitBound)
switch {
case step.failed != "":
state.Why = step.failed
p.State = inventory.PlanFailed
p.Note = fmt.Sprintf("%s stopped at its first machine in tier %d: %s; %s left as it was",
m, p.Tier, step.failed, orNone(strings.Join(step.rest, ", ")))
fmt.Printf("%s: %s\n", p.ID, p.Note)
return true, nil
case step.waiting != "":
pending = append(pending, fmt.Sprintf("%s on %s, sent first at %s", m, step.waiting, state.FirstAt.Local().Format("15:04")))
continue
case len(step.send) == 0:
// No machine runs it: nothing to send, and nothing to wait for.
state.SentAt = &now
continue
}
// What sendToEach answers, not what was asked: the machine holding the bus is sent before
// the first when its user list must change (issue 249), and the plan waits for it too.
sent, err := sendToEach(ctx, open, step.send)
if err != nil {
// Not marked sent, so the next step tries again (issue 249): a grant that could not be
// issued is a send that did not happen.
return false, fmt.Errorf("sending %s to %s after tier %d: %w", m, strings.Join(step.send, ", "), p.Tier, err)
}
if step.first {
state.First = sent
state.FirstAt = &now
p.State = inventory.PlanRolling
p.Note = fmt.Sprintf("tier %d built; sent %s to %s first", p.Tier, m, strings.Join(sent, ", "))
fmt.Printf("%s: tier %d built; sent %s to %s first, the rest once it reports it applied\n",
p.ID, p.Tier, m, strings.Join(sent, ", "))
return true, nil
}
state.SentAt = &now
fmt.Printf("%s: tier %d built; sent %s to %s\n", p.ID, p.Tier, m, strings.Join(sent, ", "))
if len(running) == 0 {
continue
}
if err := sendTo(ctx, open, running); err != nil {
return false, fmt.Errorf("sending %s to %s after tier %d: %w", m, strings.Join(running, ", "), p.Tier, err)
}
fmt.Printf("%s: tier %d built; sent %s to %s\n", p.ID, p.Tier, m, strings.Join(running, ", "))
return true, nil
}
if len(pending) > 0 {
note := "tier " + fmt.Sprint(p.Tier) + " built; waiting for " + strings.Join(pending, "; ") +
" to report it applied before the rest are sent"
changed := p.State != inventory.PlanRolling || p.Note != note
p.State = inventory.PlanRolling
p.Note = note
return changed, nil
}
// And wait for what the next tier needs running.
needed := gates(*p, edges, rollsOut)
if len(needed) > 0 {
@@ -653,24 +461,10 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
if state.BuiltAt != nil {
since = *state.BuiltAt
}
// **Each machine from its own send** (novox/hq issue 256). With one machine first (ADR 0218)
// a module is sent twice — the first machine, then the rest — and SentAt is the second.
// Asked of every machine, the first machine's report, made between the two sends, read as
// older than the build, and the gate waited for a report it already had, for ever.
sinceFor := func(node string) time.Time {
at := since
sent := state.SentAt
for _, n := range state.First {
if n == node {
sent = state.FirstAt
}
}
if sent != nil && sent.After(at) {
at = *sent
}
return at
if state.SentAt != nil && state.SentAt.After(since) {
since = *state.SentAt
}
if ok, on := appliedEach(m, sinceFor, running, reports); !ok {
if ok, on := applied(m, since, running, reports); !ok {
waiting = append(waiting, fmt.Sprintf("%s on %s", m, strings.Join(on, ", ")))
}
}
@@ -691,110 +485,6 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
return true, nil
}
// rolloutStep is what a plan does next with one built module's machines (novox/hq issue 249).
type rolloutStep struct {
// send is the machines to send now; first, whether they are the first machine's send.
send []string
first bool
// waiting names the first machines whose report the rest wait for.
waiting string
// failed says how a first machine did not take it; rest is what is then left alone.
failed string
rest []string
}
// nextRollout is the next step of one module's rollout in a plan (novox/hq issue 249, ADR 0218).
//
// Together, every machine running it at once, as the policy says. Otherwise one machine first — the
// first by name among those that have reported within the bound, so a laptop that is away is not
// the one the rest wait on; the first by name when none has; the same choice on every controller and
// every resume — and the rest once each machine the first send reached reports, about the
// declaration it was last sent, that it applied it. Compared by the store's own record of what was
// sent (Reported.Current), never by this controller's clock against the machine's.
//
// **A first machine that fails, refuses, or does not report within the bound stops the rollout
// there** (ADR 0218 §2), naming the machine; the rest are not sent. A wait with no end is not a
// rollout: it held the plan open for ever, read as work in progress (issue 254).
func nextRollout(s inventory.PlanModule, running []string, together bool, reports []inventory.Reported,
now time.Time, bound time.Duration) rolloutStep {
if len(running) == 0 {
return rolloutStep{}
}
if together {
return rolloutStep{send: running}
}
byNode := map[string]inventory.Reported{}
for _, r := range reports {
byNode[r.Node] = r
}
if s.FirstAt == nil {
sorted := append([]string{}, running...)
sort.Strings(sorted)
for _, n := range sorted {
if r, said := byNode[n]; said && r.At != nil && now.Sub(*r.At) <= bound {
return rolloutStep{send: []string{n}, first: true}
}
}
return rolloutStep{send: sorted[:1], first: true}
}
sentFirst := map[string]bool{}
for _, n := range s.First {
sentFirst[n] = true
}
var rest []string
for _, n := range running {
if !sentFirst[n] {
rest = append(rest, n)
}
}
var waiting, failed []string
for _, n := range s.First {
r, said := byNode[n]
// Only a report about what it was last sent says anything about this build.
if !said || r.At == nil || !r.Current {
waiting = append(waiting, n)
continue
}
switch r.Outcome {
case inventory.OutcomeApplied:
case inventory.OutcomeFailed, inventory.OutcomeRefused:
failed = append(failed, n+" "+r.Outcome+" what it was sent")
default:
waiting = append(waiting, n)
}
}
if len(failed) > 0 {
return rolloutStep{failed: strings.Join(failed, "; "), rest: rest}
}
if len(waiting) > 0 {
if now.Sub(*s.FirstAt) > bound {
return rolloutStep{failed: fmt.Sprintf("%s did not report it applied within %s",
strings.Join(waiting, ", "), bound), rest: rest}
}
return rolloutStep{waiting: strings.Join(waiting, ", ")}
}
return rolloutStep{send: rest}
}
// sayUnsent adds to an ended plan's note the modules it built and never sent (novox/hq issue 249).
// An announced move of a module a plan held was left to that plan; a plan that ends without
// sending it — failed elsewhere, or closed by hand — would leave its machines behind with nothing
// saying so. A module whose rollout stopped at its first machine is not among them: that stop was
// the point. Said once.
func sayUnsent(p *inventory.Plan, rollsOut func(string) bool) {
var unsent []string
for name, s := range p.Modules {
if s != nil && s.State == "built" && s.SentAt == nil && s.FirstAt == nil && rollsOut(name) {
unsent = append(unsent, name)
}
}
if len(unsent) == 0 || strings.Contains(p.Note, "built and never sent") {
return
}
sort.Strings(unsent)
p.Note += "; built and never sent: " + strings.Join(unsent, ", ") + " — `push --behind` sends them"
}
// planTicker advances open plans on a timer, for the steps outcomes alone cannot take.
func planTicker(ctx context.Context, open *stores) {
advancePlans(ctx, open)
@@ -818,10 +508,8 @@ func planLine(p inventory.Plan, now time.Time) string {
return fmt.Sprintf("%s %s done, %d tier(s)", p.Repository, short(p.Commit), len(p.Tiers))
case inventory.PlanFailed:
return fmt.Sprintf("%s %s FAILED at %s: %s", p.Repository, short(p.Commit), where, p.Note)
case inventory.PlanSuperseded:
return fmt.Sprintf("%s %s %s", p.Repository, short(p.Commit), p.Note)
}
since := now.Sub(p.Updated).Round(time.Second)
since := now.Sub(p.Updated).Round(time.Minute)
late := ""
if since > planWaitBound {
late = " — LATE"
@@ -842,8 +530,7 @@ func planFailedBuild(ctx context.Context, open *stores, result link.BuildResult)
}
for _, e := range entries {
if repositoryMatches(e.Source.Repository, result.Repository) && e.Source.Path == result.Path {
asked, _ := link.BuildAskedAt(result.ID)
planBuilt(ctx, open, e.Manifest.Module, result.Commit, result.Failed, asked)
planBuilt(ctx, open, e.Manifest.Module, result.Commit, result.Failed)
return
}
}
@@ -950,19 +637,7 @@ func plansCommand(ctx context.Context, args []string) error {
if *whatIf != "" {
return planWhatIf(ctx, inv, *whatIf, splitList(*paths), splitList(*modules))
}
// `stop`, or `close` (novox/hq issue 254): a person ending a plan that will not move again — one
// waiting on a report that cannot come — so it stops reading as work in progress. Marked failed
// with who ended it; what it asked still builds and registers.
if len(positionals) == 2 && (positionals[0] == "stop" || positionals[0] == "close") {
how := "stopped"
if positionals[0] == "close" {
how = "closed"
}
release, err := inv.HoldPlans(ctx, true)
if err != nil {
return err
}
defer release()
if len(positionals) == 2 && positionals[0] == "stop" {
p, err := inv.PlanByID(ctx, positionals[1])
if err != nil {
return err
@@ -971,16 +646,12 @@ func plansCommand(ctx context.Context, args []string) error {
return fmt.Errorf("%s is already %s", p.ID, p.State)
}
p.State = inventory.PlanFailed
p.Note = how + " by hand at tier " + fmt.Sprint(p.Tier)
sayUnsent(&p, func(m string) bool {
u, err := inv.UpgradeOf(ctx, m)
return err == nil && u.RollOut
})
p.Note = "stopped by hand at tier " + fmt.Sprint(p.Tier)
if err := inv.SavePlan(ctx, p); err != nil {
return err
}
fmt.Printf("%s %s at tier %d of %d; what was asked still builds and registers, nothing further is asked\n",
p.ID, how, p.Tier, len(p.Tiers))
fmt.Printf("%s stopped at tier %d of %d; what was asked still builds and registers, nothing further is asked\n",
p.ID, p.Tier, len(p.Tiers))
return nil
}
plans, err := inv.RecentPlans(ctx, *limit)
@@ -1055,13 +726,7 @@ func planWhatIf(ctx context.Context, inv *inventory.Inventory, repository string
how := "built; its policy records, so nothing is sent"
if u, err := inv.UpgradeOf(ctx, name); err == nil && u.RollOut {
running, _ := inv.Running(ctx, name)
reports, _ := inv.LastReports(ctx)
how = "built, then sent to " + orNone(strings.Join(running, ", "))
// One machine first unless the policy says together (novox/hq issue 249).
if first := nextRollout(inventory.PlanModule{}, running, u.Together, reports, time.Now(), planWaitBound); first.first && len(running) > 1 {
how = fmt.Sprintf("built, then sent to %s first and to the rest once it has applied it",
first.send[0])
}
rolls[name] = how
}
fmt.Printf(" %-22s %s\n", name, how)
@@ -1090,52 +755,3 @@ func splitList(s string) []string {
}
return out
}
// settleFromRecords marks every module of the tier still `asked` built — or failed — from a build
// recorded after it was asked, and says whether it changed anything (novox/hq 04-ISSUES/214).
// Newest first, as Builds answers: the first record after the ask is the outcome of that ask.
func settleFromRecords(p *inventory.Plan, tier []string, recorded map[string][]inventory.Build) bool {
changed := false
for _, m := range tier {
s := p.Modules[m]
if s == nil || s.State != "asked" || s.AskedAt == nil {
continue
}
var outcome *inventory.Build
for i := range recorded[m] {
b := recorded[m][i]
if b.At.Before(*s.AskedAt) {
break
}
// Recorded after the ask and asked before it: an earlier ask's late outcome, not this
// one's (novox/hq 04-ISSUES/219).
if askedBefore(b.Asked, s.AskedAt) {
continue
}
outcome = &b
}
if outcome == nil {
continue
}
at := outcome.At
if outcome.Worked() {
s.State = "built"
s.BuiltAt = &at
s.Commit = outcome.Commit
} else {
s.State = "failed"
s.Why = outcome.Failed
p.State = inventory.PlanFailed
p.Note = fmt.Sprintf("%s failed to build in tier %d", m, p.Tier)
}
fmt.Printf("%s: %s settled from the build records as %s (%s)\n", p.ID, m, s.State, outcome.ID)
changed = true
}
return changed
}
// askedBefore is whether a build asked at asked was asked before a plan asked for its module — and
// so is not that plan's outcome (novox/hq 04-ISSUES/219). False when either time is not known.
func askedBefore(asked time.Time, planAsked *time.Time) bool {
return !asked.IsZero() && planAsked != nil && asked.Before(*planAsked)
}
cmd/mesh-controller/release_plan_test.go
-86
View File
@@ -126,89 +126,3 @@ func TestABundleIsPlannedAfterTheToolchainItIsCompiledIn(t *testing.T) {
t.Fatalf("the toolchain, then the bundle: %v", p.Tiers)
}
}
// novox/hq 04-ISSUES/214: a plan whose build outcome was recorded while no controller followed it —
// the controller rebuilding itself — settles from the build records instead of waiting for ever.
func TestAPlanSettlesAnAskedBuildFromTheRecords(t *testing.T) {
asked := time.Date(2026, 10, 3, 19, 20, 0, 0, time.UTC)
p := inventory.Plan{ID: "plan-1", Tiers: [][]string{{"mesh-controller", "builder"}, {"route-proxy"}},
Modules: map[string]*inventory.PlanModule{
"mesh-controller": {State: "asked", AskedAt: &asked},
"builder": {State: "asked", AskedAt: &asked},
}}
records := map[string][]inventory.Build{
// Newest first, as Builds answers: the build after the ask is the outcome.
"mesh-controller": {
{ID: "build-2", Commit: "2ebbb799", At: asked.Add(4 * time.Minute)},
{ID: "build-1", Commit: "06ea2168", At: asked.Add(-10 * time.Minute)},
},
// Only a build from before the ask: not this ask's outcome.
"builder": {{ID: "build-0", Commit: "06ea2168", At: asked.Add(-time.Hour)}},
}
if !settleFromRecords(&p, p.Tiers[0], records) {
t.Fatal("nothing settled, though the controller's build is recorded after the ask")
}
if s := p.Modules["mesh-controller"]; s.State != "built" || s.Commit != "2ebbb799" || s.BuiltAt == nil {
t.Errorf("the controller's ask is %+v, want built from 2ebbb799", s)
}
if s := p.Modules["builder"]; s.State != "asked" {
t.Errorf("an ask with no record after it was settled: %+v", s)
}
// novox/hq 04-ISSUES/219: a build recorded after the ask but asked before it — an earlier
// plan's late outcome — is not this ask's, built or failed.
r := inventory.Plan{ID: "plan-3", Tiers: [][]string{{"postgres"}},
Modules: map[string]*inventory.PlanModule{"postgres": {State: "asked", AskedAt: &asked}}}
late := map[string][]inventory.Build{"postgres": {
{ID: "build-old", Commit: "efff5415", Asked: asked.Add(-18 * time.Minute), At: asked.Add(12 * time.Minute)},
}}
if settleFromRecords(&r, r.Tiers[0], late) || r.Modules["postgres"].State != "asked" {
t.Errorf("an earlier ask's late outcome settled this ask: %+v", r.Modules["postgres"])
}
// Newest heard first: the earlier ask's late outcome, then this ask's own, heard before it.
late["postgres"] = append(late["postgres"], inventory.Build{ID: "build-mine", Commit: "4bcd5f73",
Asked: asked.Add(time.Second), At: asked.Add(5 * time.Minute)})
if !settleFromRecords(&r, r.Tiers[0], late) || r.Modules["postgres"].State != "built" ||
r.Modules["postgres"].Commit != "4bcd5f73" {
t.Errorf("this ask's own outcome, heard before the earlier ask's, did not settle it: %+v", r.Modules["postgres"])
}
// A failure recorded after the ask fails the plan, as hearing it would have.
q := inventory.Plan{ID: "plan-2", Tiers: [][]string{{"x"}},
Modules: map[string]*inventory.PlanModule{"x": {State: "asked", AskedAt: &asked}}}
settleFromRecords(&q, q.Tiers[0], map[string][]inventory.Build{"x": {{ID: "b", Failed: "no", At: asked.Add(time.Minute)}}})
if q.State != inventory.PlanFailed || q.Modules["x"].State != "failed" {
t.Errorf("a recorded failure did not fail the plan: %+v %+v", q, q.Modules["x"])
}
}
// The first machine's report, made between the send to it and the send to the rest, opens the gate for it:
// each machine is judged from its own send, not from the last one (novox/hq issue 256).
func TestTheGateJudgesEachMachineFromItsOwnSend(t *testing.T) {
built := time.Date(2026, 10, 5, 18, 22, 0, 0, time.UTC)
firstSent := built.Add(31 * time.Second)
firstReported := built.Add(43 * time.Second)
restSent := built.Add(58 * time.Second)
restReported := built.Add(74 * time.Second)
reports := []inventory.Reported{
{Node: "ace", At: &firstReported},
{Node: "g14", At: &restReported},
}
since := func(node string) time.Time {
if node == "ace" {
return firstSent
}
return restSent
}
if ok, waiting := appliedEach("build-agent", since, []string{"ace", "g14"}, reports); !ok {
t.Fatalf("the gate still waits on %v, though each reported after its own send", waiting)
}
// The old reading, every machine from the last send, is what held the plan.
if ok, _ := applied("build-agent", restSent, []string{"ace", "g14"}, reports); ok {
t.Fatal("the single-moment reading should hold the first machine back")
}
early := built.Add(10 * time.Second)
if ok, waiting := appliedEach("build-agent", since, []string{"ace"}, []inventory.Reported{{Node: "ace", At: &early}}); ok || waiting[0] != "ace" {
t.Fatal("a report from before the machine was sent opened the gate")
}
}
cmd/mesh-controller/rollout_first_test.go
-152
View File
@@ -1,152 +0,0 @@
package main
import (
"reflect"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// novox/hq issue 249, ADR 0218: a plan rolls a module out to one machine first and the rest only
// once that machine has reported it applied; a module whose policy says together goes everywhere at
// once, as before.
func TestAPlanSendsOneMachineFirstAndTheRestAfterItsReport(t *testing.T) {
running := []string{"novox", "ace", "g14"}
sentAt := time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC)
now := sentAt.Add(5 * time.Minute)
bound := 30 * time.Minute
after := sentAt.Add(time.Minute)
next := func(s inventory.PlanModule, running []string, together bool, reports []inventory.Reported) rolloutStep {
return nextRollout(s, running, together, reports, now, bound)
}
// Together: every machine at once.
if step := next(inventory.PlanModule{}, running, true, nil); !reflect.DeepEqual(step.send, running) || step.first {
t.Fatalf("a together policy did not send every machine at once: %+v", step)
}
// Otherwise the first by name, alone, when none has reported lately.
step := next(inventory.PlanModule{}, running, false, nil)
if !step.first || !reflect.DeepEqual(step.send, []string{"ace"}) {
t.Fatalf("the first send was %+v, wanted ace alone", step)
}
state := inventory.PlanModule{First: []string{"ace"}, FirstAt: &sentAt}
report := func(outcome string, current bool) []inventory.Reported {
return []inventory.Reported{{Node: "ace", At: &after, Outcome: outcome, Current: current},
{Node: "g14", At: &after, Outcome: inventory.OutcomeApplied, Current: true}}
}
// No report yet, or one about an older declaration than it was last sent: wait.
for what, reports := range map[string][]inventory.Reported{
"no report": nil,
"a report about older": report(inventory.OutcomeApplied, false),
} {
step := next(state, running, false, reports)
if len(step.send) != 0 || step.waiting != "ace" || step.failed != "" {
t.Errorf("%s: %+v, wanted to wait for ace", what, step)
}
}
// Applied what it was last sent: the rest, and only the rest.
step = next(state, running, false, report(inventory.OutcomeApplied, true))
if step.first || !reflect.DeepEqual(step.send, []string{"novox", "g14"}) {
t.Fatalf("after ace applied it the plan sent %+v, wanted novox and g14", step)
}
// Failed or refused: stop, the rest untouched.
for _, outcome := range []string{inventory.OutcomeFailed, inventory.OutcomeRefused} {
step := next(state, running, false, report(outcome, true))
if len(step.send) != 0 || !strings.Contains(step.failed, "ace "+outcome) ||
!reflect.DeepEqual(step.rest, []string{"novox", "g14"}) {
t.Errorf("a first machine that %s it: %+v", outcome, step)
}
}
// The machine holding the bus went with the first send: the rest wait for it too, and it is
// not sent again.
both := inventory.PlanModule{First: []string{"novox", "ace"}, FirstAt: &sentAt}
half := report(inventory.OutcomeApplied, true)
if step := next(both, running, false, half); step.waiting != "novox" {
t.Fatalf("the plan did not wait for the bus's machine sent first: %+v", step)
}
all := append(half, inventory.Reported{Node: "novox", At: &after, Outcome: inventory.OutcomeApplied, Current: true})
if step := next(both, running, false, all); !reflect.DeepEqual(step.send, []string{"g14"}) {
t.Fatalf("after both applied it the plan sent %+v, wanted g14 alone", step)
}
// One machine, or none: nothing is waited for that cannot come.
if step := next(inventory.PlanModule{}, nil, false, nil); len(step.send) != 0 || step.first {
t.Fatalf("a module nothing runs was sent: %+v", step)
}
if step := next(state, []string{"ace"}, false, report(inventory.OutcomeApplied, true)); len(step.send) != 0 || step.waiting != "" {
t.Fatalf("a module on one machine waited for more: %+v", step)
}
}
// ADR 0218 §2: a first machine that does not report within the bound stops the rollout there,
// naming the machine and the bound; the rest are left alone.
func TestAFirstMachineThatDoesNotReportStopsTheRollout(t *testing.T) {
sentAt := time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC)
state := inventory.PlanModule{First: []string{"ace"}, FirstAt: &sentAt}
step := nextRollout(state, []string{"ace", "g14"}, false, nil, sentAt.Add(31*time.Minute), 30*time.Minute)
if !strings.Contains(step.failed, "ace did not report it applied within 30m") ||
!reflect.DeepEqual(step.rest, []string{"g14"}) || len(step.send) != 0 {
t.Fatalf("a silent first machine: %+v", step)
}
}
// The first machine is the first by name among those heard from lately: a laptop that is away is
// not the one the rest wait on. When none has been heard from, the first by name.
func TestTheFirstMachineIsOneThatHasReportedLately(t *testing.T) {
now := time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC)
lately, long := now.Add(-time.Minute), now.Add(-3*time.Hour)
reports := []inventory.Reported{
{Node: "ace", At: &long}, {Node: "g14", At: &lately}, {Node: "novox", At: &lately},
}
step := nextRollout(inventory.PlanModule{}, []string{"novox", "ace", "g14"}, false, reports, now, 30*time.Minute)
if !step.first || !reflect.DeepEqual(step.send, []string{"g14"}) {
t.Fatalf("the first send was %+v, wanted g14, the first heard from lately", step)
}
}
// An announced move of a module an open plan is still rolling out is left to the plan: sending it
// here as well put the bundle on every machine at once (novox/hq issue 249).
func TestAnAnnouncedMoveIsLeftToThePlanRollingItOut(t *testing.T) {
sent := time.Now()
plans := []inventory.Plan{
{ID: "plan-done", State: inventory.PlanDone, Modules: map[string]*inventory.PlanModule{"agent": {}}},
{ID: "plan-1", State: inventory.PlanRolling, Modules: map[string]*inventory.PlanModule{
"agent": {State: "built", First: []string{"ace"}, FirstAt: &sent}, "gitea": {State: "built", SentAt: &sent}}},
}
if got := rolledOutByAPlan(plans, "agent"); got != "plan-1" {
t.Fatalf("a module the plan is rolling out was not left to it: %q", got)
}
for _, m := range []string{"gitea", "keycloak"} {
if got := rolledOutByAPlan(plans, m); got != "" {
t.Errorf("%s, which no plan will send, was left to %s", m, got)
}
}
}
// A plan that ends without sending what it built says so, with the remedy; a module whose rollout
// stopped at its first machine is not among them.
func TestAnEndedPlanSaysWhatItBuiltAndNeverSent(t *testing.T) {
at := time.Now()
p := inventory.Plan{State: inventory.PlanFailed, Note: "closed by hand at tier 1",
Modules: map[string]*inventory.PlanModule{
"agent": {State: "built"},
"stopped": {State: "built", First: []string{"ace"}, FirstAt: &at},
"sent": {State: "built", SentAt: &at},
"notes": {State: "built"},
"later": {},
}}
rollsOut := func(m string) bool { return m != "notes" }
sayUnsent(&p, rollsOut)
sayUnsent(&p, rollsOut)
if p.Note != "closed by hand at tier 1; built and never sent: agent — `push --behind` sends them" {
t.Fatalf("the note reads %q", p.Note)
}
}
cmd/mesh-controller/seat_dependencies_test.go
-149
View File
@@ -1,149 +0,0 @@
package main
import (
"encoding/json"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Defends novox/hq ADR 0207 at the controller's acts: `assign` refuses a module whose resources a
// seat nothing on the node holds applies, `unassign` refuses taking the last holder from under its
// dependents, and `status` reports what composition does not yet refuse.
func serviceManagerHolder() catalogue.Manifest {
return catalogue.Manifest{Module: "systemd", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.ServiceManagerSeat, Scope: catalogue.ScopeNode,
Serves: []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal"}}},
Resources: []map[string]any{{"id": "systemd", "type": "package", "package": "systemd"}}}
}
func packageManagerHolder() catalogue.Manifest {
return catalogue.Manifest{Module: "pacman", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.PackageManagerSeat, Scope: catalogue.ScopeNode}},
Resources: []map[string]any{{"id": "refresh", "type": "service", "unit": "pacman-refresh.timer"}}}
}
func aDaemon() catalogue.Manifest {
return catalogue.Manifest{Module: "sshd", Version: "1",
Resources: []map[string]any{{"id": "sshd", "type": "service", "unit": "sshd.service"}}}
}
func TestAnAssignmentWithoutItsHolderIsRefusedAndNotKept(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, serviceManagerHolder())
register(t, open, packageManagerHolder())
register(t, open, aDaemon())
_, err := assign(ctx, open, "laptop", "sshd")
if err == nil {
t.Fatal("sshd went onto a machine nothing holds the service manager of")
}
for _, want := range []string{catalogue.ServiceManagerSeat, "systemd", "ADR 0207"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not say %q:\n%v", want, err)
}
}
assigned, err := open.inventory.Assigned(ctx, "laptop")
if err != nil {
t.Fatal(err)
}
if contains(assigned, "sshd") {
t.Fatalf("a refused assignment was kept: %v", assigned)
}
// The holders depend on each other, so neither goes on alone — and both go on in one act.
if _, err := assign(ctx, open, "laptop", "systemd"); err == nil {
t.Fatal("systemd went on alone though its package needs a package manager")
}
if said, err := assign(ctx, open, "laptop", "systemd", "pacman"); err != nil {
t.Fatalf("the two holders assigned together were refused: %v\n%s", err, said)
}
if said, err := assign(ctx, open, "laptop", "sshd"); err != nil {
t.Fatalf("sshd beside its holder was refused: %v\n%s", err, said)
}
}
func TestTheControllerSeatsAssignTakesSeveralModulesAsOneAct(t *testing.T) {
argv, err := argvFor("assign", map[string]any{"node": "laptop", "module": "systemd, pacman"})
if err != nil {
t.Fatal(err)
}
if strings.Join(argv, " ") != "assign laptop systemd pacman" {
t.Errorf("the seat's assign became %v", argv)
}
}
func TestUnassigningTheLastHolderUnderItsDependentsIsRefused(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, serviceManagerHolder())
register(t, open, packageManagerHolder())
register(t, open, aDaemon())
if _, err := assign(ctx, open, "laptop", "systemd", "pacman", "sshd"); err != nil {
t.Fatal(err)
}
_, err := unassign(ctx, open, "laptop", "systemd")
if err == nil {
t.Fatal("the service manager came off a machine still running services")
}
for _, want := range []string{catalogue.ServiceManagerSeat, "sshd", "pacman"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not name %q:\n%v", want, err)
}
}
assigned, _ := open.inventory.Assigned(ctx, "laptop")
if !contains(assigned, "systemd") {
t.Fatalf("a refused unassignment took the module off anyway: %v", assigned)
}
if _, err := unassign(ctx, open, "laptop", "sshd"); err != nil {
t.Fatalf("a dependent could not come off: %v", err)
}
}
func TestStatusReportsAnUnheldDependencyWithoutRefusingTheMachine(t *testing.T) {
// The mesh as it ran before the switch (novox/hq ADR 0207 §4).
defer catalogue.EnforcingSeatDependencies(false)()
open := aMesh(t)
ctx := t.Context()
register(t, open, serviceManagerHolder())
register(t, open, aDaemon())
// Assigned straight into the store: a machine whose modules predate the rule, which is every
// machine on the day it ships.
if _, err := open.inventory.Assign(ctx, "laptop", "sshd"); err != nil {
t.Fatal(err)
}
asked, err := theThreeQuestions(ctx, open)
if err != nil {
t.Fatal(err)
}
if _, refused := asked.refused["laptop"]; refused {
t.Fatalf("an unmet dependency refused the machine before the switch: %s", asked.refused["laptop"])
}
if asked.well() {
t.Error("a mesh with an unheld dependency reads as all well")
}
got := printed(t, func() error { return printStatus(asked) })
for _, want := range []string{"unheld", "laptop", "sshd", catalogue.ServiceManagerSeat, "systemd"} {
if !strings.Contains(got, want) {
t.Errorf("status does not say %q:\n%s", want, got)
}
}
body, err := statusAsJSON(asked)
if err != nil {
t.Fatal(err)
}
var doc struct {
Unheld []catalogue.Unheld `json:"unheld"`
}
if err := json.Unmarshal(body, &doc); err != nil {
t.Fatal(err)
}
if len(doc.Unheld) != 1 || doc.Unheld[0].Module != "sshd" || doc.Unheld[0].Seat != catalogue.ServiceManagerSeat {
t.Errorf("the document's unheld is %+v", doc.Unheld)
}
}
cmd/mesh-controller/seatverbs.go
+6 -30
View File
@@ -100,9 +100,6 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
if id := str("stop"); id != "" {
return []string{"plans", "stop", id}, nil
}
if id := str("close"); id != "" {
return []string{"plans", "close", id}, nil
}
if id := str("id"); id != "" {
return []string{"plans", id}, nil
}
@@ -116,9 +113,7 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
if err := need("node", "module"); err != nil {
return nil, err
}
// Several modules comma-separated, judged as one act (novox/hq ADR 0207): the holders of
// the seats that apply resources depend on each other and go on together.
return append([]string{verb, str("node")}, splitModules(str("module"))...), nil
return []string{verb, str("node"), str("module")}, nil
case "pin":
if err := need("node", "provision", "from", "module"); err != nil {
return nil, err
@@ -134,12 +129,7 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
// what a person at a shell does too. A tool call that blocked for a push's whole apply would
// time out on every machine that takes a minute, and say nothing about the ones that did not.
if n := str("node"); n != "" {
argv := []string{"push", n, "--wait", "0"}
// A running module's data moving is sent only when said (novox/hq ADR 0217).
if m := str("move"); m != "" {
argv = append(argv, "--move", m)
}
return argv, nil
return []string{"push", n, "--wait", "0"}, nil
}
return []string{"push", "--behind", "--wait", "0"}, nil
case "rotate":
@@ -168,17 +158,6 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
argv = []string{"settings", "clear", str("module")}
case str("values") != "":
argv = append(argv, str("values"))
// What a set removes is refused unless meant (novox/hq ADR 0217).
if str("replace") == "true" {
argv = append(argv, "--replace")
}
default:
// Neither values nor clear: the layer as it stands, which is what a caller reads before
// replacing it (novox/hq ADR 0217) — and with history, the layers it replaced.
argv = []string{"settings", "show", str("module")}
if str("history") == "true" {
argv = append(argv, "--history")
}
}
// Neither values nor clear: the command says its usage, which names both, and that is the
// answer the caller needs — the same as `rotate` given half of either shape.
@@ -217,7 +196,7 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
}
// jsonVerbs are the verbs whose command speaks JSON, so the answer carries it as data as well.
var jsonVerbs = map[string]bool{"status": true, "seats": true, "plan": true, "collection": true}
var jsonVerbs = map[string]bool{"status": true, "seats": true, "plan": true}
// runVerb runs this binary with the given command line and gathers what it said.
func runVerb(ctx context.Context, argv []string) (verbAnswer, error) {
@@ -423,22 +402,19 @@ func seatAnnouncement(handlers map[string]link.ToolHandler) micro.Info {
var endpoints []micro.EndpointInfo
for _, verb := range verbs {
schema, _ := json.Marshal(about[verb].Input)
// The same shape every tool runtime announces in (node-tools' announce package): the name is
// `<seat>__<verb>`, as the protocol's characters allow; the metadata is what identifies it.
endpoints = append(endpoints, micro.EndpointInfo{
Name: catalogue.ControllerSeatName + "__" + verb,
Name: verb,
Subject: link.SeatToolSubject(catalogue.ControllerSeatName, verb),
QueueGroup: "seat." + catalogue.ControllerSeatName,
Metadata: map[string]string{
"kind": "seat", "module": catalogue.ControllerSeatName, "tool": verb,
"seat": catalogue.ControllerSeatName, "scope": "mesh", "interchangeable": "false",
"description": about[verb].Description, "schema": string(schema),
"seat": catalogue.ControllerSeatName, "scope": "mesh",
},
})
}
return micro.Info{
ServiceIdentity: micro.ServiceIdentity{
Name: catalogue.ControllerSeatName, ID: "controller", Version: "0.1.0",
Name: catalogue.ControllerSeatName, ID: "controller", Version: "1.0.0",
Metadata: map[string]string{"seat": catalogue.ControllerSeatName, "scope": "mesh"},
},
Description: "the mesh's own verbs, answered by the holder of the mesh-controller seat",
cmd/mesh-controller/seatverbs_test.go
+5 -11
View File
@@ -86,11 +86,9 @@ func TestSettingsSetsOrClearsALayer(t *testing.T) {
if strings.Join(argv, " ") != "settings clear dnsmasq" {
t.Fatalf("clear for the mesh: %v", argv)
}
// Neither values nor clear reads the layer as it stands (novox/hq ADR 0217): what a caller reads
// before replacing it, where it used to fall to the command's usage.
argv, _ = argvFor("settings", map[string]any{"module": "dnsmasq"})
if strings.Join(argv, " ") != "settings show dnsmasq" {
t.Fatalf("a call with no values reads the layer: %v", argv)
if strings.Join(argv, " ") != "settings set dnsmasq" {
t.Fatalf("a set with no values falls to the command's usage: %v", argv)
}
}
@@ -283,14 +281,10 @@ func TestTheControllerAnnouncesTheVerbsItServes(t *testing.T) {
t.Fatalf("%d endpoints announced for %d verbs served", len(info.Endpoints), len(handlers))
}
for _, e := range info.Endpoints {
verb := e.Metadata["tool"]
if _, served := handlers[verb]; !served || e.Name != catalogue.ControllerSeatName+"__"+verb {
t.Errorf("%s (%s) is announced and not served under that name", e.Name, verb)
if _, served := handlers[e.Name]; !served {
t.Errorf("%s is announced and not served", e.Name)
}
if e.Metadata["kind"] != "seat" || e.Metadata["seat"] != catalogue.ControllerSeatName {
t.Errorf("%s is not announced as the seat's verb: %v", e.Name, e.Metadata)
}
if e.Subject != link.SeatToolSubject(catalogue.ControllerSeatName, verb) || e.QueueGroup != "seat."+catalogue.ControllerSeatName {
if e.Subject != link.SeatToolSubject(catalogue.ControllerSeatName, e.Name) || e.QueueGroup != "seat."+catalogue.ControllerSeatName {
t.Errorf("%s is announced on %s/%s, not where it is served", e.Name, e.Subject, e.QueueGroup)
}
if e.Metadata["description"] == "" || e.Metadata["schema"] == "" || e.Metadata["scope"] != "mesh" {
cmd/mesh-controller/sendable.go
-4
View File
@@ -31,10 +31,6 @@ type sendable struct {
// the same composition as its received files, and every machine's private-network address.
Received map[string]map[string][]catalogue.Contribution
Mesh []string
// BusUsers is the bus's user list this declaration carries, empty for every machine but the one
// holding the bus; not sent apart from the file it is in. Its digest is recorded once sent, so
// whether that machine must go first is read from the list alone (novox/hq issue 249).
BusUsers string
// LeftOut is every module of the machine's set left out of this declaration because a stored
// setting cannot compose with its definition (novox/hq ADR 0163, rule 6), sorted. The host
// keeps that module's held things and touches none of its containers; a machine is told
cmd/mesh-controller/status.go
+1 -31
View File
@@ -282,22 +282,6 @@ func printStatus(asked answers) error {
fmt.Printf("\n `take <node> <module>` compares what runs against what it declares, and runs it\n\n")
}
if len(asked.unheld) > 0 {
// **Reported, and not refused yet** (novox/hq ADR 0207 §4). Each machine still resolves and
// is sent what it would be; this says which of its modules depend on a seat nothing there
// holds, until every machine has its holders and the switch makes it a refusal.
fmt.Printf("%d module dependenc(ies) on a seat nothing on the machine holds (unheld, ADR 0207):\n",
len(asked.unheld))
for _, u := range asked.unheld {
holders := "no module in the catalogue claims it yet"
if len(u.Holders) > 0 {
holders = "could be held by " + strings.Join(u.Holders, ", ")
}
fmt.Printf(" %-12s %-24s %-24s %s\n", u.Node, u.Module, u.Seat, holders)
}
fmt.Printf("\n `assign <node> <holder>` meets it; reported until every machine has its holders, then refused\n\n")
}
if adopted := adoptedNodes(nodes); len(adopted) > 0 {
// Said, because nothing forces the flip: a node left adopted is visible here rather than
// read as converged (novox/hq ADR 0100). Not a fault, so it does not break "all well".
@@ -402,20 +386,6 @@ func theThreeQuestions(ctx context.Context, open *stores) (answers, error) {
if err != nil {
return answers{}, err
}
// And which machines run a module whose resources a seat nothing there holds applies (novox/hq
// ADR 0207). Each machine resolved again rather than threaded through whoResolves, whose answer
// the private network is built from and should say nothing else; a machine that does not
// resolve is already in refused, and is passed over here.
for _, n := range out.nodes {
plan, _, err := planFor(ctx, open, n.Name)
if err != nil {
if unresolvable(err) {
continue
}
return answers{}, err
}
out.unheld = append(out.unheld, plan.Unheld...)
}
out.plans, err = inv.RecentPlans(ctx, 5)
if err != nil {
return answers{}, err
@@ -526,7 +496,7 @@ func untakenModules(ctx context.Context, inv *inventory.Inventory, nodes []inven
func (a answers) well() bool {
return len(a.wrong) == 0 && len(a.quiet) == 0 && len(a.behind) == 0 &&
len(a.waiting) == 0 && len(a.refused) == 0 && a.network == "" && len(a.untaken) == 0 &&
len(a.filtered) == 0 && len(a.unheld) == 0
len(a.filtered) == 0
}
// hostSplit is which machines report which host version, for every version more than one machine
cmd/mesh-controller/supersede_test.go
-95
View File
@@ -1,95 +0,0 @@
package main
import (
"reflect"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// novox/hq issue 254, ADR 0218: a newer plan takes over what the older open plans of its repository
// and branch had not built, and closes them as superseded; another repository's plan, another
// branch's, and a plan made after it are left alone.
func TestANewerPlanSupersedesTheOlderOpenPlansOfItsRepository(t *testing.T) {
at := time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC)
sent := at.Add(time.Minute)
plan := func(id, repository, branch string, created time.Time, modules map[string]*inventory.PlanModule) inventory.Plan {
return inventory.Plan{ID: id, Repository: repository, Branch: branch, Commit: id + "-commit",
Created: created, State: inventory.PlanRolling, Modules: modules}
}
older := plan("plan-1", "novox/mesh-catalog", "main", at, map[string]*inventory.PlanModule{
"gitea": {State: "built", SentAt: &sent}, // done with: stays done
"keycloak": {State: "asked"}, // asked, not answered: folded
"plex": {}, // not yet asked: folded
"agent": {State: "built"}, // built, rolls out, not sent: folded
"notes": {State: "built"}, // built, records: nothing to send
})
stuck := plan("plan-0", "Novox/Mesh-Catalog", "", at.Add(-time.Hour), map[string]*inventory.PlanModule{
"runtime": {State: "asked"},
})
other := plan("plan-2", "novox/mesh-controller", "main", at, map[string]*inventory.PlanModule{"mesh-controller": {}})
release := plan("plan-3", "novox/mesh-catalog", "release", at, map[string]*inventory.PlanModule{"lemurs": {}})
later := plan("plan-5", "novox/mesh-catalog", "main", at.Add(2*time.Hour), map[string]*inventory.PlanModule{"later": {}})
done := plan("plan-6", "novox/mesh-catalog", "main", at, map[string]*inventory.PlanModule{"finished": {}})
done.State = inventory.PlanDone
newer := plan("plan-4", "novox/mesh-catalog", "main", at.Add(time.Hour), nil)
newer.Commit = "97b1b2b0c0ffee"
rollsOut := func(m string) bool { return m != "notes" }
folded, closed := supersededBy(newer, []inventory.Plan{stuck, older, other, release, later, done, newer}, rollsOut)
if want := []string{"agent", "keycloak", "plex", "runtime"}; !reflect.DeepEqual(folded, want) {
t.Fatalf("folded %v, wanted %v", folded, want)
}
var ids []string
for _, p := range closed {
ids = append(ids, p.ID)
if p.State != inventory.PlanSuperseded || p.Open() {
t.Errorf("%s was left %s", p.ID, p.State)
}
if !strings.Contains(p.Note, "plan-4") || !strings.Contains(p.Note, "97b1b2b0") {
t.Errorf("%s does not name the plan that superseded it: %q", p.ID, p.Note)
}
}
if want := []string{"plan-0", "plan-1"}; !reflect.DeepEqual(ids, want) {
t.Fatalf("superseded %v, wanted %v — another repository, another branch, a later plan and a "+
"finished one are left alone", ids, want)
}
if other.State != inventory.PlanRolling {
t.Fatal("the plan handed in was changed in place")
}
if line := planLine(closed[1], time.Now()); !strings.Contains(line, "superseded") {
t.Fatalf("a superseded plan reads %q", line)
}
}
// novox/hq issue 254: a person closes a plan that will not move again, by its id.
func TestAPersonClosesAStuckPlan(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
stuck := inventory.Plan{ID: "plan-97b1b2b", Repository: "novox/mesh-catalog", Commit: "97b1b2b",
Created: time.Now().UTC(), State: inventory.PlanRolling, Tier: 1, Tiers: [][]string{{"a"}, {"b"}},
Modules: map[string]*inventory.PlanModule{"a": {State: "built"}, "b": {}}}
if err := open.inventory.SavePlan(ctx, stuck); err != nil {
t.Fatal(err)
}
if err := plansCommand(ctx, []string{"close", stuck.ID}); err != nil {
t.Fatal(err)
}
closed, err := open.inventory.PlanByID(ctx, stuck.ID)
if err != nil {
t.Fatal(err)
}
if closed.State != inventory.PlanFailed || !strings.Contains(closed.Note, "closed by hand") {
t.Fatalf("the plan was left %s: %q", closed.State, closed.Note)
}
if err := plansCommand(ctx, []string{"close", stuck.ID}); err == nil {
t.Fatal("a plan already closed was closed again")
}
if argv, err := argvFor("plans", map[string]any{"close": stuck.ID}); err != nil ||
!reflect.DeepEqual(argv, []string{"plans", "close", stuck.ID}) {
t.Fatalf("the seat's verb does not close a plan: %v %v", argv, err)
}
}
cmd/mesh-controller/unheld_report_test.go
-108
View File
@@ -1,108 +0,0 @@
package main
import (
"bytes"
"os"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// An act says what it changed about the node it acted on, and nothing about the rest of the mesh
// (novox/hq ADR 0207): after the seat dependencies shipped, every `push <node>` and `assign` printed
// every node's unmet dependencies, a hundred lines around the one about the module just assigned.
func aContainer(name string) catalogue.Manifest {
return catalogue.Manifest{Module: name, Version: "1",
Resources: []map[string]any{{"id": name, "type": "container", "image": name}}}
}
func TestAnAssignmentSaysOnlyWhatItChangedOnItsOwnNode(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aContainer("web"))
register(t, open, aContainer("db"))
// anchor already lacks a runtime for db: true, and not this act's to say.
if _, err := open.inventory.Assign(ctx, "anchor", "db"); err != nil {
t.Fatal(err)
}
if _, err := open.inventory.Assign(ctx, "laptop", "db"); err != nil {
t.Fatal(err)
}
said, err := assign(ctx, open, "laptop", "web")
if err != nil {
t.Fatalf("%v\n%s", err, said)
}
if !strings.Contains(said, "web on laptop depends on "+catalogue.ContainerRuntimeSeat) {
t.Errorf("the assignment does not say what the module it assigned depends on:\n%s", said)
}
for _, not := range []string{"db on laptop", "db on anchor", "anchor:"} {
if strings.Contains(said, not) {
t.Errorf("the assignment says %q, which it did not change:\n%s", not, said)
}
}
}
func TestAnAssignmentThatMeetsADependencySaysSo(t *testing.T) {
shelf := map[string]catalogue.Manifest{
"web": aContainer("web"),
"docker": {Module: "docker", Claims: []catalogue.Claim{{Name: catalogue.ContainerRuntimeSeat}},
Resources: []map[string]any{{"id": "d", "type": "container", "image": "dind"}}},
}
lines := unheldChange(shelf, "laptop", []string{"web"}, []string{"web", "docker"})
if len(lines) != 1 || !strings.Contains(lines[0], "web on laptop now has "+catalogue.ContainerRuntimeSeat+" held") {
t.Errorf("meeting a dependency said %v", lines)
}
// Taking the dependent off says nothing: the dependency went with its module.
if lines := unheldChange(shelf, "laptop", []string{"web"}, nil); len(lines) != 0 {
t.Errorf("unassigning the dependent said %v", lines)
}
}
func TestAPushSaysANamedNodesDependenciesAndCountsTheRest(t *testing.T) {
unheld := map[string][]catalogue.Unheld{
"anchor": {{Node: "anchor", Module: "db", Seat: catalogue.ContainerRuntimeSeat}},
"laptop": {{Node: "laptop", Module: "web", Seat: catalogue.ContainerRuntimeSeat},
{Node: "laptop", Module: "sshd", Seat: catalogue.ServiceManagerSeat}},
}
var named bytes.Buffer
reportUnheldPushed(&named, true, []string{"laptop"}, unheld)
got := named.String()
if !strings.Contains(got, "laptop has 2 unmet") || !strings.Contains(got, "web on laptop") ||
!strings.Contains(got, "sshd on laptop") || strings.Contains(got, "anchor") {
t.Errorf("a named push said:\n%s", got)
}
var all bytes.Buffer
reportUnheldPushed(&all, false, []string{"anchor", "laptop", "quiet"}, unheld)
want := "anchor: 1 unmet seat dependenc(ies) — see `status`\nlaptop: 2 unmet seat dependenc(ies) — see `status`\n"
if all.String() != want {
t.Errorf("a push to every node said:\n%s\nwant\n%s", all.String(), want)
}
}
func TestOnlyTheServingControllerLogsEachChange(t *testing.T) {
read := func(f func()) string {
old := os.Stderr
r, w, _ := os.Pipe()
os.Stderr = w
f()
_ = w.Close()
os.Stderr = old
var b bytes.Buffer
_, _ = b.ReadFrom(r)
return b.String()
}
u := []catalogue.Unheld{{Node: "n1", Module: "web", Seat: catalogue.ContainerRuntimeSeat}}
if got := read(func() { logUnheld("n1", u) }); got != "" {
t.Errorf("a command logged:\n%s", got)
}
logUnheldChanges = true
defer func() { logUnheldChanges = false }()
if got := read(func() { logUnheld("n1", u) }); !strings.Contains(got, "web on n1") {
t.Errorf("the serving controller did not log a change:\n%s", got)
}
if got := read(func() { logUnheld("n1", u) }); got != "" {
t.Errorf("an unchanged report was logged again:\n%s", got)
}
}
cmd/mesh-controller/unseen.go
-335
View File
@@ -1,335 +0,0 @@
package main
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"os"
"reflect"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/inventory"
)
// No change to a machine takes effect unseen (novox/hq ADR 0217, to-be 44).
//
// Two incidents in two days had one shape: a change took effect that nobody saw before it did. A
// provisioner read an unreadable file as "nobody asks" and dropped seven databases (issue 241); one
// placement set for one module on one machine replaced its whole settings layer, and the plan then
// ran the module on an empty directory (issue 246). Here are the three guards: what a settings layer
// loses is said and refused unless meant; what a push will change can be read before it is sent; and
// a running container's data moving holds that machine's push until the operator says so. Each is
// silent when nothing is at stake, so an ordinary change goes exactly as before.
// ---- 1. what a settings layer loses ------------------------------------------------------------
// settingsChange is what replacing one layer with another adds, changes and removes, by dotted path
// to each leaf — `places.config`, not only `places` — because a layer that keeps a key and loses one
// of its entries has lost something just the same: on 2026-10-05 a node's `places` kept its name and
// lost three of its four directories.
func settingsChange(before, after map[string]any) (added, changed, removed []string) {
was, now := leaves(before, ""), leaves(after, "")
for path, v := range now {
old, had := was[path]
switch {
case !had:
added = append(added, path)
case !reflect.DeepEqual(old, v):
changed = append(changed, path)
}
}
for path := range was {
if _, kept := now[path]; !kept {
removed = append(removed, path)
}
}
sort.Strings(added)
sort.Strings(changed)
sort.Strings(removed)
return added, changed, removed
}
// leaves flattens a settings layer to its leaves by dotted path; a list is a leaf, compared whole.
func leaves(m map[string]any, prefix string) map[string]any {
out := map[string]any{}
for k, v := range m {
path := k
if prefix != "" {
path = prefix + "." + k
}
if inner, ok := v.(map[string]any); ok && len(inner) > 0 {
for p, leaf := range leaves(inner, path) {
out[p] = leaf
}
continue
}
out[path] = v
}
return out
}
// ---- 2. what a push will change -----------------------------------------------------------------
// sentResource is what is kept of one resource a machine was sent: enough to say what changed and
// whether a container's data moved, and nothing that could carry a secret — a file's content is
// kept as a digest, never as text (the record lands in the store's own backups).
type sentResource struct {
ID string `json:"id"`
Type string `json:"type"`
// Digest is of the whole resource as it was sent.
Digest string `json:"digest"`
// Fields is each field's digest, so a change can be named by field.
Fields map[string]string `json:"fields"`
// Volumes is a container's mounts as sent — paths, which are not secrets, and what a moved
// data directory is read from.
Volumes []string `json:"volumes,omitempty"`
}
// summarize is what is kept of a declaration body: its resources, in the order sent.
func summarize(body []byte) ([]sentResource, error) {
var envelope struct {
Resources []map[string]any `json:"resources"`
}
if err := json.Unmarshal(body, &envelope); err != nil {
return nil, fmt.Errorf("a declaration that is not one: %w", err)
}
out := make([]sentResource, 0, len(envelope.Resources))
for _, r := range envelope.Resources {
s := sentResource{ID: fmt.Sprint(r["id"]), Type: fmt.Sprint(r["type"]), Digest: digestValue(r),
Fields: map[string]string{}}
for k, v := range r {
s.Fields[k] = digestValue(v)
}
if s.Type == "container" {
if vols, ok := r["volumes"].([]any); ok {
for _, v := range vols {
s.Volumes = append(s.Volumes, fmt.Sprint(v))
}
}
}
out = append(out, s)
}
return out, nil
}
func digestValue(v any) string {
raw, _ := json.Marshal(v)
sum := sha256.Sum256(raw)
return hex.EncodeToString(sum[:8])
}
// changedResource is one resource sent differently, with the fields that differ.
type changedResource struct {
ID, Type string
Fields []string
}
// sentDiff is what would be sent now against what was sent last, by resource id.
type sentDiff struct {
Added, Removed []string
Changed []changedResource
}
func (d sentDiff) empty() bool {
return len(d.Added) == 0 && len(d.Removed) == 0 && len(d.Changed) == 0
}
func diffSent(before, after []sentResource) sentDiff {
was := map[string]sentResource{}
for _, r := range before {
was[r.ID] = r
}
var d sentDiff
seen := map[string]bool{}
for _, r := range after {
seen[r.ID] = true
old, had := was[r.ID]
if !had {
d.Added = append(d.Added, r.ID)
continue
}
if old.Digest == r.Digest {
continue
}
c := changedResource{ID: r.ID, Type: r.Type}
for k, v := range r.Fields {
if old.Fields[k] != v {
c.Fields = append(c.Fields, k)
}
}
for k := range old.Fields {
if _, kept := r.Fields[k]; !kept {
c.Fields = append(c.Fields, k)
}
}
sort.Strings(c.Fields)
d.Changed = append(d.Changed, c)
}
for _, r := range before {
if !seen[r.ID] {
d.Removed = append(d.Removed, r.ID)
}
}
sort.Strings(d.Added)
sort.Strings(d.Removed)
sort.Slice(d.Changed, func(a, b int) bool { return d.Changed[a].ID < d.Changed[b].ID })
return d
}
// writeDiff says a diff the way a person reads one: what is added, removed and changed, and a
// changed container's fields — a container sent differently is a container recreated.
func writeDiff(w io.Writer, node string, d sentDiff, moves []dataMove) {
if d.empty() {
fmt.Fprintf(w, "%s: nothing would change — it was last sent exactly this\n", node)
return
}
fmt.Fprintf(w, "%s would change:\n", node)
for _, id := range d.Added {
fmt.Fprintf(w, " + %s\n", id)
}
for _, id := range d.Removed {
fmt.Fprintf(w, " - %s\n", id)
}
for _, c := range d.Changed {
what := "changed"
if c.Type == "container" {
what = "recreated"
}
fmt.Fprintf(w, " ~ %s %s: %s\n", c.ID, what, strings.Join(c.Fields, ", "))
}
writeMoves(w, node, moves)
}
// ---- 3. a running module's data moving ------------------------------------------------------------
// dataMove is a container keeping a mount at the same place inside it with a different directory
// on the machine behind it: its data moving — or, as on 2026-10-05, a module about to start on an
// empty directory because the placement that pointed at its data was lost.
type dataMove struct {
Container, Inside, From, To string
}
// Module is the module the container belongs to: resource ids are `<module>.<id>`.
func (m dataMove) Module() string {
module, _, _ := strings.Cut(m.Container, ".")
return module
}
// movesIn is every data move between what a machine was sent and what it would be sent. A new
// container, a mount added or dropped and a changed image are not moves.
func movesIn(before, after []sentResource) []dataMove {
was := map[string]sentResource{}
for _, r := range before {
if r.Type == "container" {
was[r.ID] = r
}
}
var out []dataMove
for _, r := range after {
old, had := was[r.ID]
if r.Type != "container" || !had {
continue
}
from := mountSources(old.Volumes)
for inside, to := range mountSources(r.Volumes) {
if prev, mounted := from[inside]; mounted && prev != to {
out = append(out, dataMove{Container: r.ID, Inside: inside, From: prev, To: to})
}
}
}
sort.Slice(out, func(a, b int) bool {
if out[a].Container != out[b].Container {
return out[a].Container < out[b].Container
}
return out[a].Inside < out[b].Inside
})
return out
}
// mountSources is a container's mounts by the place inside it: `source:inside[:options]`.
func mountSources(volumes []string) map[string]string {
out := map[string]string{}
for _, v := range volumes {
parts := strings.SplitN(v, ":", 3)
if len(parts) < 2 {
continue
}
out[parts[1]] = parts[0]
}
return out
}
func writeMoves(w io.Writer, node string, moves []dataMove) {
for _, m := range moves {
fmt.Fprintf(w, " ! %s: %s moves from %s to %s\n", m.Container, m.Inside, m.From, m.To)
}
}
// heldMoves is the moves in a push to one machine that the operator has not said to send (`--move
// <module>`); empty when there is nothing to hold, including a machine never sent anything before.
func heldMoves(ctx context.Context, inv *inventory.Inventory, node string, body []byte, allowed map[string]bool) ([]dataMove, error) {
prev, err := inv.SentSummary(ctx, node)
if err != nil || len(prev) == 0 {
return nil, err
}
var before []sentResource
if err := json.Unmarshal(prev, &before); err != nil {
// A record this version cannot read compares with nothing: holding every push for it would
// stop the mesh on a format, which is not what is at stake.
return nil, nil
}
after, err := summarize(body)
if err != nil {
return nil, err
}
var held []dataMove
for _, m := range movesIn(before, after) {
if !allowed[m.Module()] {
held = append(held, m)
}
}
return held, nil
}
// sayHeld tells the operator why a machine was not sent, and how to send it.
func sayHeld(w io.Writer, node string, moves []dataMove) {
modules := map[string]bool{}
for _, m := range moves {
modules[m.Module()] = true
}
var names []string
for m := range modules {
names = append(names, m)
}
sort.Strings(names)
fmt.Fprintf(w, "held %s: a running module's data would move (novox/hq ADR 0217)\n", node)
writeMoves(w, node, moves)
fmt.Fprintf(w, " if that is meant: push %s --move %s\n", node, strings.Join(names, ","))
}
// writePlanDiff says what sending body to a machine would change against what it was last sent.
func writePlanDiff(ctx context.Context, inv *inventory.Inventory, node string, body []byte) error {
after, err := summarize(body)
if err != nil {
return err
}
prev, err := inv.SentSummary(ctx, node)
if err != nil {
return err
}
if len(prev) == 0 {
fmt.Printf("%s: what it was last sent is not kept yet — the first push from this version keeps "+
"it; %d resource(s) would be sent\n", node, len(after))
return nil
}
var before []sentResource
if err := json.Unmarshal(prev, &before); err != nil {
return fmt.Errorf("%s: what it was last sent is kept in a form this version does not read: %w", node, err)
}
writeDiff(os.Stdout, node, diffSent(before, after), movesIn(before, after))
return nil
}
cmd/mesh-controller/unseen_test.go
-170
View File
@@ -1,170 +0,0 @@
package main
import (
"bytes"
"context"
"reflect"
"strings"
"testing"
)
// novox/hq ADR 0217: no change to a machine takes effect unseen.
// The layer of 2026-10-05: a media server's node layer, and the one that replaced it, which kept
// `places` and lost three of its four directories and every other key.
var before = map[string]any{
"puid": 1000.0, "pgid": 1000.0,
"expose": map[string]any{"32400": "anywhere"},
"places": map[string]any{
"config": map[string]any{"path": "/srv/media/config", "owner": "1000:1000"},
"data": map[string]any{"path": "/srv/media/data", "owner": "1000:1000"},
"transcode": map[string]any{"path": "/srv/media/temp", "owner": "1000:1000"},
},
}
func TestASettingThatKeepsAKeyAndLosesItsEntriesIsSaidToRemoveThem(t *testing.T) {
after := map[string]any{"places": map[string]any{
"previews": map[string]any{"path": "/srv/pool/previews", "owner": "1000:1000"},
}}
added, changed, removed := settingsChange(before, after)
if !reflect.DeepEqual(added, []string{"places.previews.owner", "places.previews.path"}) {
t.Errorf("added %v", added)
}
if len(changed) != 0 {
t.Errorf("changed %v", changed)
}
for _, lost := range []string{"expose.32400", "pgid", "places.config.path", "places.data.owner", "puid"} {
found := false
for _, r := range removed {
found = found || r == lost
}
if !found {
t.Errorf("removing %s was not said: %v", lost, removed)
}
}
}
func TestASettingThatOnlyAddsOrChangesRemovesNothing(t *testing.T) {
after := map[string]any{
"puid": 1001.0, "pgid": 1000.0,
"expose": map[string]any{"32400": "anywhere"},
"places": map[string]any{
"config": map[string]any{"path": "/srv/media/config", "owner": "1000:1000"},
"data": map[string]any{"path": "/srv/media/data", "owner": "1000:1000"},
"transcode": map[string]any{"path": "/srv/media/temp", "owner": "1000:1000"},
"previews": map[string]any{"path": "/srv/pool/previews", "owner": "1000:1000"},
},
}
_, changed, removed := settingsChange(before, after)
if len(removed) != 0 {
t.Errorf("an additive set was said to remove %v", removed)
}
if !reflect.DeepEqual(changed, []string{"puid"}) {
t.Errorf("changed %v", changed)
}
}
// What was sent, as a push would send it: a media server's container and a file with a secret.
func declaration(configFrom string, extra ...string) []byte {
vols := `"` + configFrom + `:/config", "/srv/media/data:/data"`
for _, e := range extra {
vols += `, "` + e + `"`
}
return []byte(`{"declaration":1,"sequence":7,"resources":[
{"id":"plex.server","type":"container","image":"plex@sha256:aa","volumes":[` + vols + `]},
{"id":"plex.env","type":"file","path":"/srv/media/env","content":"TOKEN=the-secret-itself"}]}`)
}
func summarized(t *testing.T, body []byte) []sentResource {
t.Helper()
s, err := summarize(body)
if err != nil {
t.Fatal(err)
}
return s
}
func TestWhatIsKeptOfASendHoldsNoFileContent(t *testing.T) {
s := summarized(t, declaration("/srv/media/config"))
var kept bytes.Buffer
for _, r := range s {
kept.WriteString(r.ID + r.Type + r.Digest + strings.Join(r.Volumes, ","))
for k, v := range r.Fields {
kept.WriteString(k + v)
}
}
if strings.Contains(kept.String(), "the-secret-itself") {
t.Fatal("a file's content was kept in the record of what was sent")
}
if len(s) != 2 || s[0].Volumes[0] != "/srv/media/config:/config" {
t.Fatalf("summary %+v", s)
}
}
func TestADiffOfAnUnchangedMachineIsEmptyAndAChangedContainerNamesItsField(t *testing.T) {
was := summarized(t, declaration("/srv/media/config"))
if d := diffSent(was, summarized(t, declaration("/srv/media/config"))); !d.empty() {
t.Fatalf("an unchanged machine differs: %+v", d)
}
d := diffSent(was, summarized(t, declaration("/var/lib/plex/config")))
if len(d.Changed) != 1 || d.Changed[0].ID != "plex.server" || !reflect.DeepEqual(d.Changed[0].Fields, []string{"volumes"}) {
t.Fatalf("diff %+v", d)
}
var out bytes.Buffer
writeDiff(&out, "home", d, movesIn(was, summarized(t, declaration("/var/lib/plex/config"))))
for _, want := range []string{"~ plex.server recreated: volumes", "! plex.server: /config moves from /srv/media/config to /var/lib/plex/config"} {
if !strings.Contains(out.String(), want) {
t.Errorf("diff does not say %q:\n%s", want, out.String())
}
}
}
// The incident: the configuration mount would move to an empty default directory.
func TestARunningContainersDataMovingIsAMoveAndAnAddedMountIsNot(t *testing.T) {
was := summarized(t, declaration("/srv/media/config"))
moves := movesIn(was, summarized(t, declaration("/var/lib/plex/config")))
want := []dataMove{{Container: "plex.server", Inside: "/config", From: "/srv/media/config", To: "/var/lib/plex/config"}}
if !reflect.DeepEqual(moves, want) {
t.Fatalf("moves %+v", moves)
}
if moves[0].Module() != "plex" {
t.Errorf("module %q", moves[0].Module())
}
// A mount added — the previews of 2026-10-05 — is not a move.
if m := movesIn(was, summarized(t, declaration("/srv/media/config", "/srv/pool/previews:/config/Media"))); len(m) != 0 {
t.Errorf("an added mount was held as a move: %+v", m)
}
// Nor is a container the machine never ran.
if m := movesIn(nil, was); len(m) != 0 {
t.Errorf("a first send was held as a move: %+v", m)
}
}
func TestAPushNamingNoMachineIsRefusedUnlessItSaysAll(t *testing.T) {
err := pushCommand(context.Background(), nil)
if err == nil || !strings.Contains(err.Error(), "--all") {
t.Fatalf("a bare push was not refused: %v", err)
}
if err := pushCommand(context.Background(), []string{"--all", "--behind"}); err == nil {
t.Fatal("--all with --behind was accepted")
}
}
func TestTheVerbsCarryReadReplaceAndMove(t *testing.T) {
for _, c := range []struct {
verb string
args map[string]any
want []string
}{
{"settings", map[string]any{"module": "plex", "node": "home"}, []string{"settings", "show", "plex", "--node", "home"}},
{"settings", map[string]any{"module": "plex", "history": "true"}, []string{"settings", "show", "plex", "--history"}},
{"settings", map[string]any{"module": "plex", "values": "{}", "replace": "true"}, []string{"settings", "set", "plex", "{}", "--replace"}},
{"push", map[string]any{"node": "home", "move": "plex"}, []string{"push", "home", "--wait", "0", "--move", "plex"}},
{"push", map[string]any{}, []string{"push", "--behind", "--wait", "0"}},
} {
got, err := argvFor(c.verb, c.args)
if err != nil || !reflect.DeepEqual(got, c.want) {
t.Errorf("%s %v: %v %v, want %v", c.verb, c.args, got, err, c.want)
}
}
}
cmd/mesh-controller/upgrades.go
+11 -171
View File
@@ -5,8 +5,6 @@ import (
"errors"
"flag"
"fmt"
"path"
"regexp"
"strings"
"time"
@@ -56,26 +54,10 @@ func (f following) Upgraded(ctx context.Context, u link.Upgraded) error {
return nil
}
// **A plan that holds the module rolls it out, and this does not** (novox/hq issue 249, ADR
// 0218). A merge's plan builds the module and sends it one machine first, the rest once that one
// has applied it; this announcement arrives as the build registers, and sending here too — one
// machine after another without waiting for any to apply — put the new bundle on every machine in
// the same minute, whatever the plan was waiting for. A move no plan answers (a build asked by
// hand) is still this handler's.
if plans, err := inv.OpenPlans(ctx); err != nil {
return notNow(err)
} else if id := rolledOutByAPlan(plans, u.Module); id != "" {
// Said with its remedy: a plan that ends without sending it — failed, or closed by hand — leaves
// these machines behind, which `status` lists and `push --behind` sends (novox/hq issue 249).
fmt.Printf("%s moved to %s; %s rolls it out to %s — if that plan ends without sending it, "+
"`status` lists them as behind and `push --behind` sends it\n",
u.Module, shortCommit(u.Commit), id, readableList(on))
return nil
}
if decision.Together {
fmt.Printf("%s moved to %s; sending %s together\n",
u.Module, shortCommit(u.Commit), readableList(on))
return askAgainOnGrants(sendTo(ctx, f.open, on))
return sendTo(ctx, f.open, on)
}
// One at a time, and stopping at the first that fails.
//
@@ -86,34 +68,13 @@ func (f following) Upgraded(ctx context.Context, u link.Upgraded) error {
u.Module, shortCommit(u.Commit), readableList(on))
for _, node := range on {
if err := sendTo(ctx, f.open, []string{node}); err != nil {
return askAgainOnGrants(fmt.Errorf("%s did not take %s, so the machines after it were left alone: %w",
node, u.Module, err))
return fmt.Errorf("%s did not take %s, so the machines after it were left alone: %w",
node, u.Module, err)
}
}
return nil
}
// askAgainOnGrants marks a send that stopped at its grants as one to ask again (novox/hq issue 249):
// an announcement handled by a send whose memberships could not be issued is held and redelivered,
// rather than taken as handled with the machines left on the old version.
func askAgainOnGrants(err error) error {
if err != nil && errors.Is(err, errGrants) && !errors.Is(err, link.ErrTryAgain) {
return fmt.Errorf("%w: %w", link.ErrTryAgain, err)
}
return err
}
// rolledOutByAPlan is the open plan that will send a module's machines its new build — one holding
// the module that has not finished sending it — or empty when none will (novox/hq issue 249).
func rolledOutByAPlan(plans []inventory.Plan, module string) string {
for _, p := range plans {
if s, holds := p.Modules[module]; p.Open() && holds && (s == nil || (s.SentAt == nil && s.State != "failed")) {
return p.ID
}
}
return ""
}
// readableList names machines the way a sentence does, because this is read by a person deciding
// whether an upgrade went where they expected.
func readableList(names []string) string {
@@ -322,14 +283,6 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
if isHistory(m.MergedAt, lastLookAt(entries, m)) {
packaging = nil
}
// Said, never silent (novox/hq 04-ISSUES/215): a module built from this repository that follows
// another branch is not part of this merge, and whoever is waiting for its change should read why.
for _, e := range entries {
if sameRepository(e.Source.Repository, m) && !sourceIs(e.Source, m) {
fmt.Printf(" %s is built from %s/%s and follows %s, not %s; this merge leaves it out\n",
e.Manifest.Module, m.Owner, m.Repo, e.Source.Ref, m.Base)
}
}
touched := whatTheMergeTouched(from, entries, m)
for _, e := range touched {
if err := inv.SourceMoved(ctx, e.Manifest.Module, m.Commit); err != nil {
@@ -353,53 +306,7 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
for _, e := range moved {
movedNames = append(movedNames, e.Manifest.Module)
}
// Written and its first tier asked as one act on the plans (novox/hq issue 213): a timer on
// another controller reading it between the two would ask the tier again.
release, err := inv.HoldPlans(ctx, true)
if err != nil {
return notNow(err)
}
defer release()
plan := planOfMerge(m, movedNames, edges)
// **A newer plan supersedes the older open plans of this repository and branch** (novox/hq issue
// 254, ADR 0218): what they had not built is planned here again, and they are closed, so one plan
// works a repository's modules at a time and a stuck one ends at the next merge.
working, err := inv.OpenPlans(ctx)
if err != nil {
return notNow(err)
}
rollsOut := func(module string) bool {
u, err := inv.UpgradeOf(ctx, module)
return err == nil && u.RollOut
}
folded, superseded := supersededBy(plan, working, rollsOut)
if len(folded) > 0 {
held := map[string]bool{}
for _, e := range entries {
held[e.Manifest.Module] = true
}
names := map[string]bool{}
for _, name := range movedNames {
names[name] = true
}
var also []string
for _, name := range folded {
// One the catalogue no longer holds would fail the newer plan's ask; it is not this
// merge's to build.
if held[name] && !names[name] {
names[name] = true
movedNames = append(movedNames, name)
also = append(also, name)
}
}
if len(also) > 0 {
again := planOfMerge(m, movedNames, edges)
again.ID, again.Created = plan.ID, plan.Created
plan = again
fmt.Printf(" %s, left unbuilt by an older plan of %s, are planned here again\n",
strings.Join(also, ", "), plan.Repository)
}
}
if hasCycle(plan.Tiers, edges) {
fmt.Printf(" the last tier depends on itself: %s — built together, in no order\n",
strings.Join(plan.Tiers[len(plan.Tiers)-1], ", "))
@@ -407,14 +314,6 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
if err := inv.SavePlan(ctx, plan); err != nil {
return notNow(err)
}
// Closed after the newer plan is kept, never before: a controller replaced between the two leaves
// both open, which the next merge settles, rather than neither.
for _, old := range superseded {
if err := inv.SavePlan(ctx, old); err != nil {
return notNow(err)
}
fmt.Printf(" %s (%s at %s) is %s\n", old.ID, old.Repository, short(old.Commit), old.Note)
}
var tiers []string
for i, t := range plan.Tiers {
tiers = append(tiers, fmt.Sprintf("%d: %s", i, strings.Join(t, ", ")))
@@ -446,24 +345,7 @@ func sourceIs(s inventory.Source, m link.SourceMoved) bool {
if !sameRepository(s.Repository, m) {
return false
}
ref := followedBranch(s.Ref)
return ref == "" || ref == m.Base
}
// commitRef is a ref that names a commit rather than a branch: what `build --ref <commit>` asks for.
var commitRef = regexp.MustCompile(`^[0-9a-f]{7,40}$`)
// followedBranch is the branch a recorded ref means a module follows (novox/hq 04-ISSUES/215). **A
// commit is never a branch to follow.** A build asked at a commit — to try one, or to pin it during a
// fix — recorded that commit as the module's ref; every merge after it then failed to match the
// module, its plan left it out without saying so, and every plan that rebuilt it asked for that same
// old commit again. A commit recorded so is read as the repository's default branch, which is what
// the module followed before it; a branch is followed as named.
func followedBranch(ref string) string {
if commitRef.MatchString(strings.TrimSpace(ref)) {
return ""
}
return ref
return s.Ref == "" || s.Ref == m.Base
}
// sameRepository is whether a recorded repository is the one a merge names, in either spelling it
@@ -508,45 +390,25 @@ func lastLookAt(entries []inventory.Entry, m link.SourceMoved) time.Time {
//
// A change inside *another* module's directory is that module's business and not this one's, even
// when the mesh does not hold that module: `known` is every module this repository is known to hold,
// whatever branch it was registered from.
//
// **And a module the mesh has never seen is still a module** (novox/hq issue 252). A merge adding a
// new module to the catalogue repository — `modules/newmod/module.json` and its files — read as a
// change to shared code, because `modules/newmod` was nobody's known directory, and every module
// built from the repository was rebuilt and rolled out for a module none of them is. So the
// directories that hold modules are known too: the parents of the known modules' directories
// (`modules`, never the root). A changed path `<parent>/<name>/…` belongs to the module at
// `<parent>/<name>` — held or not — and rebuilds nothing else, **provided it is shown to be a
// module**: its `module.json` is among the changed files (added, changed, or removed with it). A
// directory under the same parent whose manifest the merge did not touch may as well be a shared
// library (`modules/lib`), and that is still read as shared. Rebuilding too much remains the safe
// direction: the fault this whole path exists for is a mesh that believes it is current and is not
// (novox/hq 04-ISSUES/131). A file at the root, or directly in a parent, is shared as it always was.
// whatever branch it was registered from. That is also the limit of this — a repository whose shared
// code sits inside a directory the mesh has never seen a module in reads as shared, and everything
// is rebuilt. Rebuilding too much is the safe direction: the fault this whole path exists for is a
// mesh that believes it is current and is not (novox/hq 04-ISSUES/131).
func whatTheMergeTouched(candidates, known []inventory.Entry, m link.SourceMoved) []inventory.Entry {
// Nothing said about the files, or not all of them said: everything built from it is affected.
if len(m.Paths) == 0 || m.PathsTruncated {
return candidates
}
var dirs []string
parents := map[string]bool{}
for _, e := range known {
if e.Source.Path != "" && sameRepository(e.Source.Repository, m) {
dir := strings.Trim(e.Source.Path, "/")
dirs = append(dirs, dir)
if parent := path.Dir(dir); parent != "." && parent != "/" {
parents[parent] = true
}
dirs = append(dirs, e.Source.Path)
}
}
changed := map[string]bool{}
for _, p := range m.Paths {
changed[strings.TrimPrefix(p, "/")] = true
}
for _, p := range m.Paths {
if insideAny(p, dirs) || inAModuleOfItsOwn(p, parents, changed) {
continue
if !insideAny(p, dirs) {
return candidates
}
return candidates
}
var out []inventory.Entry
for _, e := range candidates {
@@ -557,28 +419,6 @@ func whatTheMergeTouched(candidates, known []inventory.Entry, m link.SourceMoved
return out
}
// inAModuleOfItsOwn is whether a changed file is inside a module directory the mesh does not know —
// `<parent>/<name>/…` under a directory known to hold modules, whose `module.json` the same merge
// changed (novox/hq issue 252). Such a file is that module's business and nobody else's.
func inAModuleOfItsOwn(p string, parents, changed map[string]bool) bool {
p = strings.TrimPrefix(p, "/")
for parent := range parents {
rest, under := strings.CutPrefix(p, parent+"/")
if !under {
continue
}
name, _, inADirectory := strings.Cut(rest, "/")
if !inADirectory || name == "" {
// A file directly in the parent — `modules/README.md` — is about all of them.
continue
}
if changed[parent+"/"+name+"/module.json"] {
return true
}
}
return false
}
// inside is whether a changed file is in a directory: that directory itself, or under it.
func inside(path, dir string) bool {
dir = strings.Trim(dir, "/")
examples/route-proxy/forwarded_test.go
-29
View File
@@ -1,29 +0,0 @@
package main
import (
"crypto/tls"
"net/http"
"net/http/httptest"
"net/url"
"testing"
)
// A backend behind the proxy learns the client used TLS and which name it asked for, so the addresses
// it writes into its own pages are the ones a client can use (2026-10-03: a forge's Go import tag
// named an http clone URL, and Go refused the module path).
func TestABackendIsToldTheRequestWasHTTPSAndForWhichName(t *testing.T) {
var proto, host, fwdHost, fwdFor string
backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
proto, host, fwdHost, fwdFor = r.Header.Get("X-Forwarded-Proto"), r.Host, r.Header.Get("X-Forwarded-Host"), r.Header.Get("X-Forwarded-For")
}))
defer backend.Close()
where, _ := url.Parse(backend.URL)
req := httptest.NewRequest(http.MethodGet, "https://git.example.org/novox/mesh-sdk/go?go-get=1", nil)
req.TLS = &tls.ConnectionState{}
req.Host = "git.example.org"
req.RemoteAddr = "192.0.2.7:51000"
towards(where).ServeHTTP(httptest.NewRecorder(), req)
if proto != "https" || fwdHost != "git.example.org" || host != "git.example.org" || fwdFor != "192.0.2.7" {
t.Errorf("the backend was told proto=%q host=%q forwarded-host=%q for=%q", proto, host, fwdHost, fwdFor)
}
}
examples/route-proxy/main.go
+1 -15
View File
@@ -273,7 +273,7 @@ func (t *table) set(routes map[string][]rule, public map[string]bool) {
log.Printf("route %s points at %q, which is not a URL: %v", host, r.target, err)
continue
}
r.to = towards(where)
r.to = httputil.NewSingleHostReverseProxy(where)
if r.insecure {
r.to.Transport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}}
}
@@ -1060,17 +1060,3 @@ func asPort(v any) (int, bool) {
}
return 0, false
}
// towards proxies to one backend and tells it what the client asked: **X-Forwarded-Proto, -Host and
// -For**, set from the request this proxy received. A backend that builds its own addresses — a forge
// writing its clone URL into a page, a login redirect — otherwise sees the plain HTTP hop from this
// proxy and writes `http://`, though every client reached it over TLS: Go refused the forge's module
// path for exactly that on 2026-10-03, its import tag naming an http clone URL.
// The standard library's NewSingleHostReverseProxy sets only X-Forwarded-For.
func towards(where *url.URL) *httputil.ReverseProxy {
return &httputil.ReverseProxy{Rewrite: func(pr *httputil.ProxyRequest) {
pr.SetURL(where)
pr.Out.Host = pr.In.Host
pr.SetXForwarded()
}}
}
internal/artifacts/hold.go
-345
View File
@@ -1,345 +0,0 @@
package artifacts
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"net/http"
"strconv"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Every archive the store keeps is held by a manifest (novox/hq issue 253, ADR 0189).
//
// **The store's collector marks only from manifests.** The mesh's store is a stock registry, and
// its nightly `registry garbage-collect` walks every manifest in every repository, marks the blobs
// those manifests name, and deletes every blob it did not mark. An image is a manifest, so what
// the mesh keeps of an image survives. An archive was not: the builder put it in the store as a
// bare blob — upload, then `PUT ?digest=` — and nothing in the store names it. To the collector a
// bare blob is unreferenced, so the first real collection would have deleted every archive the
// mesh holds, kept or not, and every machine pinning a bundle would have found it gone. The
// collector runs `--dry-run` until this is true.
//
// **So each archive gets a holder**: the smallest OCI image manifest that names it — the empty
// config, one layer, nothing else — put in the archive's own repository, by digest, untagged. The
// collector marks it and so keeps the archive; the sweep lets go of an archive by deleting its
// holder first, which is what lets the bytes go at the next collection.
//
// **Nothing a machine reads changes.** The recorded reference stays
// `artifact-store://<module>/<artifact>/blobs/sha256:…`, and machines fetch the blob exactly as
// before. The holder is the store's bookkeeping, not a second way to reach anything.
//
// **Deterministic, so it never needs recording.** The holder is composed from the archive's digest
// and size alone, in a fixed field order with no timestamps or annotations, so the sweep can
// compute which manifest holds any archive from the reference it already has plus one HEAD for the
// size. No schema change, no second record that could disagree with the store.
const (
// mediaManifest is the type a holder is put and asked for as.
mediaManifest = "application/vnd.oci.image.manifest.v1+json"
// mediaEmpty is the OCI empty descriptor's type: a config that says nothing, for a manifest
// whose only purpose is to name its layer.
mediaEmpty = "application/vnd.oci.empty.v1+json"
// emptyDigest is the digest of `{}`, the empty config's content, fixed by the OCI spec.
emptyDigest = "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a"
// mediaArchive is the layer type an archive is held as. Every archive the builder publishes is
// `pack`'s gzipped tar, so this is the true type and not a placeholder — and it is a constant,
// not read from anywhere, because the holder must be recomputable from the reference alone.
mediaArchive = "application/vnd.oci.image.layer.v1.tar+gzip"
)
// emptyConfig is the content emptyDigest names.
var emptyConfig = []byte("{}")
// manifestAccept is what a manifest is asked for as. A registry answers a manifest HEAD only in a
// type the caller named, and answers 404 to a bare one for a manifest it holds perfectly well
// (measured 2026-09-28; internal/builder/registry.go says how that was found).
var manifestAccept = []string{
mediaManifest,
"application/vnd.docker.distribution.manifest.v2+json",
}
type descriptor struct {
MediaType string `json:"mediaType"`
Digest string `json:"digest"`
Size int64 `json:"size"`
}
type holderManifest struct {
SchemaVersion int `json:"schemaVersion"`
MediaType string `json:"mediaType"`
Config descriptor `json:"config"`
Layers []descriptor `json:"layers"`
}
// Holder is the manifest that holds an archive in the store, and its digest.
//
// A pure function of the archive's digest and size: the same two in give the same bytes out,
// always, because `encoding/json` writes a struct's fields in their declared order and there is
// nothing here that varies by when or where it was composed.
func Holder(digest string, size int64) (body []byte, holder string) {
body, err := json.Marshal(holderManifest{
SchemaVersion: 2,
MediaType: mediaManifest,
Config: descriptor{MediaType: mediaEmpty, Digest: emptyDigest, Size: int64(len(emptyConfig))},
Layers: []descriptor{{MediaType: mediaArchive, Digest: digest, Size: size}},
})
if err != nil {
// Marshalling a struct of strings and integers cannot fail.
panic(err)
}
sum := sha256.Sum256(body)
return body, "sha256:" + hex.EncodeToString(sum[:])
}
// Hold makes sure the store holds this archive by a manifest, and says whether it had to write one.
//
// Takes a reference as the mesh records it. An image is its own manifest and needs no holder, so
// it answers false and nothing is asked. Idempotent: a holder already there is left alone, which
// is what lets the sweep run it over every kept archive on every build and so backfill the bare
// blobs published before holders existed (novox/hq issue 253).
//
// Gone when the store does not hold the archive at all: there is nothing to hold, and that is a
// fact the caller reports rather than one this invents a remedy for.
func (s Store) Hold(ctx context.Context, reference string) (bool, error) {
repository, digest, archive, err := s.archive(reference)
if err != nil || !archive {
return false, err
}
size, err := s.blobSize(ctx, repository, digest)
if err != nil {
return false, err
}
return s.HoldBlob(ctx, repository, digest, size)
}
// Held is whether the store holds this archive by its manifest. Asks and changes nothing — the
// question an operator needs answered with "none unheld" before the collector is let loose.
//
// An image answers true: it is its own manifest. An archive the store does not have answers Gone.
func (s Store) Held(ctx context.Context, reference string) (bool, error) {
repository, digest, archive, err := s.archive(reference)
if err != nil {
return false, err
}
if !archive {
return true, nil
}
size, err := s.blobSize(ctx, repository, digest)
if err != nil {
return false, err
}
_, holder := Holder(digest, size)
return s.has(ctx, s.url(repository, "manifests", holder), manifestAccept...)
}
// HoldBlob puts the holder for a blob of this digest and size into its repository, unless it is
// there already. Answers whether it wrote one.
//
// The builder calls this with the size it has just uploaded; the sweep, through Hold, with the size
// the store reports. Both arrive at the same holder, which is the point of composing it.
func (s Store) HoldBlob(ctx context.Context, repository, digest string, size int64) (bool, error) {
if s.Address == "" {
return false, fmt.Errorf("this mesh has no artifact store on its network to hold %s/%s in", repository, digest)
}
body, holder := Holder(digest, size)
there, err := s.has(ctx, s.url(repository, "manifests", holder), manifestAccept...)
if err != nil {
return false, err
}
if there {
return false, nil
}
// The config must be in the repository before a manifest naming it is accepted: a registry
// refuses a manifest whose blobs it cannot find there, which is the property that makes a
// holder mean something.
if err := s.putBlob(ctx, repository, emptyDigest, emptyConfig); err != nil {
return false, err
}
// **By digest, never by tag.** A tag would be one more name to move and one more thing the
// collector's `--delete-untagged` would read as meaningful; the mesh names nothing by tag that
// it pins by digest, and an untagged manifest is kept by plain collection.
request, err := http.NewRequestWithContext(ctx, http.MethodPut,
s.url(repository, "manifests", holder), bytes.NewReader(body))
if err != nil {
return false, err
}
request.Header.Set("Content-Type", mediaManifest)
response, err := s.client().Do(request)
if err != nil {
return false, err
}
defer response.Body.Close()
if response.StatusCode != http.StatusCreated {
said, _ := io.ReadAll(io.LimitReader(response.Body, 4096))
return false, fmt.Errorf("the artifact store refused to hold %s/%s: %s %s",
repository, digest, response.Status, strings.TrimSpace(string(said)))
}
return true, nil
}
// letGoOfHolder deletes the manifest holding an archive, before the archive's own link goes.
//
// **Holder first.** Deleting the blob link alone leaves a manifest still naming the blob, and the
// collector would keep its bytes for ever on the strength of it — the sweep would record the
// archive collected while the disk said otherwise. Deleting the holder first and failing before
// the link goes leaves an unheld archive that the next sweep still offers, which is safe.
//
// A store that no longer has the blob answers Gone: without its size the holder cannot be named,
// and without the blob there is nothing left for a holder to keep. A store that never had a holder
// for it — an archive published before holders, never backfilled — answers 404 to the delete, and
// that is the outcome wanted.
func (s Store) letGoOfHolder(ctx context.Context, repository, digest string) error {
size, err := s.blobSize(ctx, repository, digest)
if err != nil {
return err
}
_, holder := Holder(digest, size)
err = s.remove(ctx, s.url(repository, "manifests", holder), repository+"/manifests/"+holder)
if err == Gone {
return nil
}
return err
}
// archive reads a recorded reference into its repository and digest, and whether it is an archive
// at all. Refuses as ErrNotOurs anything the mesh did not put in its own store.
func (s Store) archive(reference string) (repository, digest string, archive bool, err error) {
path, kept := catalogue.InArtifactStore(reference)
if !kept {
return "", "", false, fmt.Errorf("%w: %s", ErrNotOurs, reference)
}
if s.Address == "" {
return "", "", false, fmt.Errorf("this mesh has no artifact store on its network to ask about %s", reference)
}
repository, kind, digest, err := split(path)
if err != nil {
return "", "", false, err
}
return repository, digest, kind == "blobs", nil
}
// blobSize is how large the store says a blob is; Gone when it does not have it.
func (s Store) blobSize(ctx context.Context, repository, digest string) (int64, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodHead, s.url(repository, "blobs", digest), nil)
if err != nil {
return 0, err
}
response, err := s.client().Do(request)
if err != nil {
return 0, fmt.Errorf("cannot reach the artifact store at %s: %w", s.Address, err)
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusOK:
case http.StatusNotFound:
return 0, Gone
default:
return 0, fmt.Errorf("the artifact store answered %s for %s/blobs/%s", response.Status, repository, digest)
}
// Read from the header rather than ContentLength: a HEAD's ContentLength is what the response
// says it would have sent, which Go reports faithfully, but a proxy in between is free to drop
// it, and the header is what the registry itself wrote.
if length := response.Header.Get("Content-Length"); length != "" {
if n, err := strconv.ParseInt(length, 10, 64); err == nil && n >= 0 {
return n, nil
}
}
if response.ContentLength >= 0 {
return response.ContentLength, nil
}
return 0, fmt.Errorf("the artifact store holds %s/blobs/%s and will not say how large it is", repository, digest)
}
// putBlob uploads a small blob unless the repository already has it: ask where, then put it there
// naming the digest — the registry's own two steps, the same the builder takes for an archive.
func (s Store) putBlob(ctx context.Context, repository, digest string, body []byte) error {
if there, err := s.has(ctx, s.url(repository, "blobs", digest)); err != nil {
return err
} else if there {
return nil
}
start, err := http.NewRequestWithContext(ctx, http.MethodPost,
"http://"+s.Address+"/v2/"+repository+"/blobs/uploads/", nil)
if err != nil {
return err
}
begun, err := s.client().Do(start)
if err != nil {
return fmt.Errorf("cannot start an upload to %s: %w", repository, err)
}
begun.Body.Close()
if begun.StatusCode != http.StatusAccepted {
return fmt.Errorf("the artifact store answered %s when asked where to put a blob in %s", begun.Status, repository)
}
where := begun.Header.Get("Location")
if where == "" {
return fmt.Errorf("the artifact store accepted an upload to %s and said nowhere to put it", repository)
}
if strings.HasPrefix(where, "/") {
where = "http://" + s.Address + where
}
separator := "?"
if strings.Contains(where, "?") {
separator = "&"
}
put, err := http.NewRequestWithContext(ctx, http.MethodPut, where+separator+"digest="+digest, bytes.NewReader(body))
if err != nil {
return err
}
put.Header.Set("Content-Type", "application/octet-stream")
done, err := s.client().Do(put)
if err != nil {
return err
}
defer done.Body.Close()
if done.StatusCode != http.StatusCreated {
said, _ := io.ReadAll(io.LimitReader(done.Body, 4096))
return fmt.Errorf("the artifact store refused a blob in %s: %s %s", repository, done.Status, strings.TrimSpace(string(said)))
}
return nil
}
// has is whether the store answers 200 for a HEAD at that URL.
func (s Store) has(ctx context.Context, url string, accept ...string) (bool, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodHead, url, nil)
if err != nil {
return false, err
}
for _, media := range accept {
request.Header.Add("Accept", media)
}
response, err := s.client().Do(request)
if err != nil {
return false, fmt.Errorf("cannot reach the artifact store at %s: %w", s.Address, err)
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusOK:
return true, nil
case http.StatusNotFound:
return false, nil
default:
return false, fmt.Errorf("the artifact store answered %s for %s", response.Status, url)
}
}
func (s Store) url(repository, kind, digest string) string {
return "http://" + s.Address + "/v2/" + repository + "/" + kind + "/" + digest
}
func (s Store) client() *http.Client {
if s.HTTP != nil {
return s.HTTP
}
return &http.Client{Timeout: 30 * time.Second}
}
internal/artifacts/hold_test.go
-268
View File
@@ -1,268 +0,0 @@
package artifacts
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"sync"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Every kept archive is held by a manifest (novox/hq issue 253, ADR 0189).
//
// Against an in-memory registry that keeps blobs and manifests per repository and refuses what a
// registry refuses — a blob whose digest does not match, a manifest whose digest does not match
// or whose blobs the repository does not have, a manifest asked for without an Accept naming its
// type. What is asserted is this side's decisions; the live test below asserts the registry's.
type memRegistry struct {
mu sync.Mutex
blobs map[string][]byte // repository + "@" + digest
manifests map[string][]byte // repository + "@" + digest
writes []string // every PUT and DELETE, as "METHOD path"
}
func digestOf(body []byte) string {
sum := sha256.Sum256(body)
return "sha256:" + hex.EncodeToString(sum[:])
}
func (m *memRegistry) serve(t *testing.T) Store {
t.Helper()
m.blobs = map[string][]byte{}
m.manifests = map[string][]byte{}
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
m.mu.Lock()
defer m.mu.Unlock()
path := strings.TrimPrefix(r.URL.Path, "/v2/")
if r.Method == http.MethodPut || r.Method == http.MethodDelete {
m.writes = append(m.writes, r.Method+" "+r.URL.Path)
}
switch {
case r.Method == http.MethodPost && strings.HasSuffix(path, "/blobs/uploads/"):
repository := strings.TrimSuffix(path, "/blobs/uploads/")
w.Header().Set("Location", "/upload/"+repository+"?state=x")
w.WriteHeader(http.StatusAccepted)
case r.Method == http.MethodPut && strings.HasPrefix(r.URL.Path, "/upload/"):
repository := strings.TrimPrefix(r.URL.Path, "/upload/")
body, _ := io.ReadAll(r.Body)
digest := r.URL.Query().Get("digest")
if digest != digestOf(body) {
w.WriteHeader(http.StatusBadRequest)
return
}
m.blobs[repository+"@"+digest] = body
w.WriteHeader(http.StatusCreated)
case strings.Contains(path, "/blobs/"):
repository, digest, _ := strings.Cut(path, "/blobs/")
key := repository + "@" + digest
body, ok := m.blobs[key]
if !ok {
w.WriteHeader(http.StatusNotFound)
return
}
switch r.Method {
case http.MethodHead:
w.Header().Set("Content-Length", strconv.Itoa(len(body)))
w.WriteHeader(http.StatusOK)
case http.MethodDelete:
delete(m.blobs, key)
w.WriteHeader(http.StatusAccepted)
default:
w.WriteHeader(http.StatusMethodNotAllowed)
}
case strings.Contains(path, "/manifests/"):
repository, digest, _ := strings.Cut(path, "/manifests/")
key := repository + "@" + digest
switch r.Method {
case http.MethodHead:
if _, ok := m.manifests[key]; !ok || !strings.Contains(r.Header.Get("Accept"), mediaManifest) {
w.WriteHeader(http.StatusNotFound)
return
}
w.WriteHeader(http.StatusOK)
case http.MethodPut:
body, _ := io.ReadAll(r.Body)
if digest != digestOf(body) {
w.WriteHeader(http.StatusBadRequest)
return
}
var named holderManifest
if err := json.Unmarshal(body, &named); err != nil {
w.WriteHeader(http.StatusBadRequest)
return
}
for _, d := range append([]descriptor{named.Config}, named.Layers...) {
if _, ok := m.blobs[repository+"@"+d.Digest]; !ok {
w.WriteHeader(http.StatusBadRequest)
fmt.Fprintf(w, "MANIFEST_BLOB_UNKNOWN %s", d.Digest)
return
}
}
m.manifests[key] = body
w.WriteHeader(http.StatusCreated)
case http.MethodDelete:
if _, ok := m.manifests[key]; !ok {
w.WriteHeader(http.StatusNotFound)
return
}
delete(m.manifests, key)
w.WriteHeader(http.StatusAccepted)
}
default:
w.WriteHeader(http.StatusNotFound)
}
}))
t.Cleanup(server.Close)
return Store{Address: strings.TrimPrefix(server.URL, "http://")}
}
// bare puts an archive in the store the way the builder did before holders: a blob, nothing more.
func (m *memRegistry) bare(repository string, body []byte) string {
m.mu.Lock()
defer m.mu.Unlock()
digest := digestOf(body)
m.blobs[repository+"@"+digest] = body
return catalogue.ArtifactStoreScheme + repository + "/blobs/" + digest
}
func TestTheHolderIsComposedFromTheDigestAndSizeAlone(t *testing.T) {
// The sweep must arrive at the very manifest the builder wrote, with nothing recorded between
// them. Same inputs, same bytes — and a different size is a different holder, so a holder can
// never be mistaken for one of a different blob.
digest := "sha256:" + strings.Repeat("a", 64)
one, first := Holder(digest, 42)
two, second := Holder(digest, 42)
if !bytes.Equal(one, two) || first != second {
t.Fatalf("the same archive composed two holders:\n%s\n%s", one, two)
}
if _, other := Holder(digest, 43); other == first {
t.Fatal("a different size composed the same holder")
}
want := `{"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json",` +
`"config":{"mediaType":"application/vnd.oci.empty.v1+json",` +
`"digest":"sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a","size":2},` +
`"layers":[{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":"` + digest + `","size":42}]}`
if string(one) != want {
t.Fatalf("the holder is\n%s\nwant\n%s", one, want)
}
if digestOf(emptyConfig) != emptyDigest {
t.Fatalf("the empty config's digest is %s, not %s", digestOf(emptyConfig), emptyDigest)
}
}
func TestHoldBackfillsABareArchiveAndIsIdempotent(t *testing.T) {
// The archives published before this have no holder. Hold, run over every kept archive on
// every sweep, writes one the first time and nothing after.
m := &memRegistry{}
store := m.serve(t)
ctx := context.Background()
body := []byte("a theme")
reference := m.bare("shell/config", body)
if held, err := store.Held(ctx, reference); err != nil || held {
t.Fatalf("a bare blob reads as held=%v (%v)", held, err)
}
wrote, err := store.Hold(ctx, reference)
if err != nil {
t.Fatal(err)
}
if !wrote {
t.Fatal("holding a bare archive wrote nothing")
}
_, holder := Holder(digestOf(body), int64(len(body)))
if _, ok := m.manifests["shell/config@"+holder]; !ok {
t.Fatalf("the store holds manifests %v; want %s", m.manifests, holder)
}
if held, err := store.Held(ctx, reference); err != nil || !held {
t.Fatalf("after holding, held=%v (%v)", held, err)
}
writes := len(m.writes)
wrote, err = store.Hold(ctx, reference)
if err != nil {
t.Fatal(err)
}
if wrote || len(m.writes) != writes {
t.Fatalf("holding again wrote %v", m.writes[writes:])
}
}
func TestAnImageNeedsNoHolderAndAMissingArchiveIsGone(t *testing.T) {
m := &memRegistry{}
store := m.serve(t)
ctx := context.Background()
// An image is its own manifest: nothing is asked.
wrote, err := store.Hold(ctx, catalogue.ArtifactStoreScheme+"web/app@sha256:"+strings.Repeat("b", 64))
if err != nil || wrote || len(m.writes) != 0 {
t.Fatalf("holding an image wrote=%v err=%v writes=%v", wrote, err, m.writes)
}
// An archive the store does not have is a fact to report, not something to invent a holder for.
_, err = store.Hold(ctx, catalogue.ArtifactStoreScheme+"web/config/blobs/sha256:"+strings.Repeat("c", 64))
if !errors.Is(err, Gone) {
t.Fatalf("holding a missing archive answered %v, want Gone", err)
}
// And a reference that is not the mesh's is refused as such.
if _, err := store.Hold(ctx, "docker.io/library/registry@sha256:abc"); !errors.Is(err, ErrNotOurs) {
t.Fatalf("holding a vendor's image answered %v, want ErrNotOurs", err)
}
}
func TestLettingGoOfAnArchiveDeletesItsHolderFirst(t *testing.T) {
// A holder left behind would keep the bytes through every collection while the record said
// collected; the link deleted first and the holder failing after would be that exactly.
m := &memRegistry{}
store := m.serve(t)
ctx := context.Background()
body := []byte("an old theme")
reference := m.bare("shell/config", body)
if _, err := store.Hold(ctx, reference); err != nil {
t.Fatal(err)
}
m.writes = nil
if err := store.LetGo(ctx, reference); err != nil {
t.Fatal(err)
}
_, holder := Holder(digestOf(body), int64(len(body)))
want := []string{
"DELETE /v2/shell/config/manifests/" + holder,
"DELETE /v2/shell/config/blobs/" + digestOf(body),
}
if strings.Join(m.writes, "\n") != strings.Join(want, "\n") {
t.Fatalf("the store was asked\n%s\nwant\n%s", strings.Join(m.writes, "\n"), strings.Join(want, "\n"))
}
if len(m.manifests) != 0 {
t.Fatalf("a holder survived: %v", m.manifests)
}
// Asked again, the archive is already gone, which is the outcome wanted.
if err := store.LetGo(ctx, reference); !errors.Is(err, Gone) {
t.Fatalf("letting go twice answered %v, want Gone", err)
}
}
func TestLettingGoOfAnUnheldArchiveStillDeletesIt(t *testing.T) {
// An archive published before holders and let go of before any sweep held it: the holder's
// delete answers 404, which is the outcome wanted, and the blob still goes.
m := &memRegistry{}
store := m.serve(t)
reference := m.bare("shell/config", []byte("never held"))
if err := store.LetGo(context.Background(), reference); err != nil {
t.Fatal(err)
}
if len(m.blobs) != 0 {
t.Fatalf("the blob survived: %v", m.blobs)
}
}
internal/artifacts/live_registry_test.go
-130
View File
@@ -1,130 +0,0 @@
package artifacts
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"os"
"os/exec"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The registry's own collector keeps a held archive and takes a bare one (novox/hq issue 253,
// ADR 0189).
//
// Everything else here is asserted against a fake, which can only say what this side asks. This is
// the one question a fake cannot answer — what `registry garbage-collect` actually does with what
// this side wrote — and it is the whole of whether the store's nightly step may stop being a dry
// run. Against the very image the mesh's store runs:
//
// docker run -d --rm --name mesh-controller-registry -p 15000:5000 \
// -e REGISTRY_STORAGE_DELETE_ENABLED=true registry:2.8.3
// MESH_TEST_REGISTRY=127.0.0.1:15000 MESH_TEST_REGISTRY_CONTAINER=mesh-controller-registry \
// go test -run Live ./internal/artifacts/
// docker stop mesh-controller-registry
//
// Skipped without both variables: it needs a registry it may write to and collect, and a container
// to run the collector in.
func TestLiveTheRegistrysCollectorKeepsWhatIsHeldAndTakesWhatIsNot(t *testing.T) {
address := os.Getenv("MESH_TEST_REGISTRY")
container := os.Getenv("MESH_TEST_REGISTRY_CONTAINER")
if address == "" || container == "" {
t.Skip("no MESH_TEST_REGISTRY / MESH_TEST_REGISTRY_CONTAINER; see this test's comment for the registry to raise")
}
ctx := context.Background()
store := Store{Address: address}
run := time.Now().UnixNano()
// Four archives in four repositories, each a different story. Distinct bytes per run, so a
// registry reused across runs cannot answer for an earlier one.
put := func(name string) (repository, digest string, body []byte) {
repository = fmt.Sprintf("live-%d/%s", run, name)
body = []byte(fmt.Sprintf("%s archive of run %d", name, run))
digest = digestOf(body)
if err := store.putBlob(ctx, repository, digest, body); err != nil {
t.Fatal(err)
}
return repository, digest, body
}
reference := func(repository, digest string) string {
return catalogue.ArtifactStoreScheme + repository + "/blobs/" + digest
}
// Published held — what PublishArchive now does.
heldRepo, heldDigest, heldBody := put("held")
if _, err := store.HoldBlob(ctx, heldRepo, heldDigest, int64(len(heldBody))); err != nil {
t.Fatalf("the registry refused a holder: %v", err)
}
// Published bare, as before, and never held: what the collector must take.
_, bareDigest, _ := put("bare")
// Published bare and then held by the sweep: the backfill.
backRepo, backDigest, backBody := put("backfilled")
if wrote, err := store.Hold(ctx, reference(backRepo, backDigest)); err != nil || !wrote {
t.Fatalf("backfilling wrote=%v: %v", wrote, err)
}
if held, err := store.Held(ctx, reference(backRepo, backDigest)); err != nil || !held {
t.Fatalf("after backfilling, held=%v: %v", held, err)
}
// Held, and then let go of by the sweep: holder first, then the link.
goneRepo, goneDigest, _ := put("let-go")
if _, err := store.Hold(ctx, reference(goneRepo, goneDigest)); err != nil {
t.Fatal(err)
}
if err := store.LetGo(ctx, reference(goneRepo, goneDigest)); err != nil {
t.Fatalf("letting go of a held archive: %v", err)
}
collected, err := exec.CommandContext(ctx, "docker", "exec", container,
"registry", "garbage-collect", "/etc/docker/registry/config.yml").CombinedOutput()
if err != nil {
t.Fatalf("the collector failed: %v\n%s", err, collected)
}
t.Logf("the collector said:\n%s", lastLines(string(collected), 12))
// What is asserted is the bytes on the store's disk, not what the running server answers: the
// server caches blob descriptors in memory and can answer for a blob the collector removed.
onDisk := func(digest string) bool {
hex := strings.TrimPrefix(digest, "sha256:")
path := "/var/lib/registry/docker/registry/v2/blobs/sha256/" + hex[:2] + "/" + hex + "/data"
return exec.CommandContext(ctx, "docker", "exec", container, "test", "-f", path).Run() == nil
}
if !onDisk(heldDigest) {
t.Error("the collector took an archive published held")
}
if !onDisk(backDigest) {
t.Error("the collector took an archive the sweep backfilled a holder for")
}
if onDisk(bareDigest) {
t.Error("the collector kept a bare archive — then the holders prove nothing, and this test is wrong")
}
if onDisk(goneDigest) {
t.Error("the collector kept an archive the sweep let go of: its holder outlived its link")
}
// And what survived is still fetched exactly as machines fetch it: the blob, by digest.
for repository, want := range map[string][]byte{heldRepo: heldBody, backRepo: backBody} {
digest := digestOf(want)
response, err := http.Get(catalogue.Routed(reference(repository, digest), address))
if err != nil {
t.Fatal(err)
}
got, _ := io.ReadAll(response.Body)
response.Body.Close()
if response.StatusCode != http.StatusOK || !bytes.Equal(got, want) {
t.Errorf("%s answered %s with %q after collection", repository, response.Status, got)
}
}
}
func lastLines(s string, n int) string {
lines := strings.Split(strings.TrimSpace(s), "\n")
if len(lines) > n {
lines = lines[len(lines)-n:]
}
return strings.Join(lines, "\n")
}
internal/artifacts/store.go
-123
View File
@@ -1,123 +0,0 @@
// Package artifacts speaks to the mesh's artifact store over its own door.
//
// Only what the mesh needs that nothing else does: letting go of something it put there
// (novox/hq ADR 0189, issue 108), and holding every archive it keeps by a manifest so the store's
// own collector does not take it (novox/hq issue 253). Pushing is the builder's, through the container runtime; reading
// is every machine's, through its runtime. This is the one operation that belongs to the thing
// holding the records, because it is the only one that is a decision rather than a transfer.
package artifacts
import (
"context"
"errors"
"fmt"
"net/http"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Store is the artifact store at an address, as this machine reaches it.
type Store struct {
// Address is `host:port` — the store as the caller reaches it now, composed and never
// recorded (novox/hq 04-ISSUES/102).
Address string
// HTTP is the client used; nil is a client with a modest timeout.
HTTP *http.Client
}
// Gone is the answer when the store does not hold it: the outcome wanted, already true.
var Gone = errors.New("the store does not hold it")
// ErrNotOurs is a reference this sweep will not address: not the mesh's own, or naming nothing
// the store holds by digest.
//
// **A fact about the record, not about the store** (novox/hq issue 226). The two deserve opposite
// responses — skip one and go on, abandon the sweep for the other — and collapsing them into "an
// error" is how a cautious loop became one that did nothing while reporting the right number.
var ErrNotOurs = errors.New("not a reference into the mesh's artifact store")
// LetGo asks the store to drop one artifact the mesh recorded making.
//
// Takes a reference as the mesh records it — `artifact-store://<module>/<artifact>@sha256:…` for
// an image, `…/blobs/sha256:…` for an archive — because that is the identity every record uses,
// and composes the address here at the moment of use.
//
// An archive is let go of in two deletes, its holder manifest and then the blob's link (novox/hq
// issue 253); an image in one.
//
// Returns Gone when the store answers that it does not have it. That is not a failure: the sweep
// wants the artifact absent, and it is. It is distinguished from success only so a caller can say
// which of the two happened.
func (s Store) LetGo(ctx context.Context, reference string) error {
// **Strict, and deliberately** (novox/hq issue 226). Only a reference the mesh keeps in its
// own vocabulary is addressed here. `Recorded` would read `docker.io/library/registry@sha256:…`
// as the mesh's too — it cannot tell one registry host from another — so normalising belongs
// where the provenance is known, which is the sweep reading its own build records, not here
// where the only job is to refuse anything that is not plainly ours.
path, kept := catalogue.InArtifactStore(reference)
if !kept {
// Nothing the mesh put in its own store. Refused rather than attempted: composing a
// delete for a reference of unknown shape is how a sweep reaches something that is not
// the mesh's. Distinguished from a store that refuses, so a sweep skips this and goes on.
return fmt.Errorf("%w: %s", ErrNotOurs, reference)
}
if s.Address == "" {
return fmt.Errorf("this mesh has no artifact store on its network to ask about %s", reference)
}
repository, kind, digest, err := split(path)
if err != nil {
return err
}
if kind == "blobs" {
// **An archive's holder goes before the archive** (novox/hq issue 253): a manifest left
// naming the blob would keep its bytes through every collection while the record said
// collected. Gone here means the store has no such blob, so there is nothing to let go.
if err := s.letGoOfHolder(ctx, repository, digest); err != nil {
return err
}
}
return s.remove(ctx, s.url(repository, kind, digest), reference)
}
// remove asks the store to delete what is at url. Gone when it has no such thing.
func (s Store) remove(ctx context.Context, url, what string) error {
request, err := http.NewRequestWithContext(ctx, http.MethodDelete, url, nil)
if err != nil {
return err
}
response, err := s.client().Do(request)
if err != nil {
return err
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusAccepted, http.StatusOK, http.StatusNoContent:
return nil
case http.StatusNotFound:
return Gone
case http.StatusMethodNotAllowed:
// The registry was started without deletion enabled. Said plainly, because the remedy is
// a setting on the store's module and not anything about this artifact.
return fmt.Errorf(
"the artifact store refuses deletion: its server was started without it enabled "+
"(REGISTRY_STORAGE_DELETE_ENABLED), so nothing can be collected until the store "+
"module is applied again (novox/hq ADR 0189). Asking about %s", what)
default:
return fmt.Errorf("the artifact store answered %s for %s", response.Status, what)
}
}
// split reads a recorded path into the repository, which endpoint names the thing, and the digest.
//
// Two shapes, which are the two the mesh records: `<repository>@sha256:<hex>` is a manifest, and
// `<repository>/blobs/sha256:<hex>` is a blob.
func split(path string) (repository, kind, digest string, err error) {
if before, after, ok := strings.Cut(path, "@sha256:"); ok {
return before, "manifests", "sha256:" + after, nil
}
if before, after, ok := strings.Cut(path, "/blobs/sha256:"); ok {
return before, "blobs", "sha256:" + after, nil
}
return "", "", "", fmt.Errorf("%w: %q names nothing the store holds by digest", ErrNotOurs, path)
}
internal/artifacts/store_test.go
-134
View File
@@ -1,134 +0,0 @@
package artifacts
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Asking the store to let go of what the mesh no longer keeps (novox/hq ADR 0189, issue 108).
//
// A fake store records what it was asked to delete, so what is asserted is the mesh's decision
// and the shape of the request — not the registry's behaviour, which is the registry's to test.
func fakeStore(t *testing.T, answer int) (Store, *[]string) {
t.Helper()
var asked []string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodHead && strings.Contains(r.URL.Path, "/blobs/") {
// An archive's size, asked so its holder can be named (novox/hq issue 253). A store
// that does not have the thing does not have its blob either.
if answer == http.StatusNotFound {
w.WriteHeader(http.StatusNotFound)
return
}
w.Header().Set("Content-Length", "7")
w.WriteHeader(http.StatusOK)
return
}
if r.Method != http.MethodDelete {
t.Errorf("the store was asked %s %s; collecting is a delete", r.Method, r.URL.Path)
}
asked = append(asked, r.URL.Path)
w.WriteHeader(answer)
}))
t.Cleanup(server.Close)
return Store{Address: strings.TrimPrefix(server.URL, "http://")}, &asked
}
func TestAnImageAndAnArchiveAreAskedForAtTheirOwnEndpoints(t *testing.T) {
// The two shapes the mesh records: a manifest by digest, and a blob by digest. They are
// different endpoints, and asking at the wrong one answers 404 — which this would then
// record as collected, leaving the bytes on disk for ever while the record says otherwise.
store, asked := fakeStore(t, http.StatusAccepted)
ctx := context.Background()
image := catalogue.ArtifactStoreScheme + "web/app@sha256:abc123"
archive := catalogue.ArtifactStoreScheme + "web/config/blobs/sha256:def456"
if err := store.LetGo(ctx, image); err != nil {
t.Fatal(err)
}
if err := store.LetGo(ctx, archive); err != nil {
t.Fatal(err)
}
// The archive's holder goes first, then the archive (novox/hq issue 253).
_, holder := Holder("sha256:def456", 7)
want := []string{
"/v2/web/app/manifests/sha256:abc123",
"/v2/web/config/manifests/" + holder,
"/v2/web/config/blobs/sha256:def456",
}
if strings.Join(*asked, " ") != strings.Join(want, " ") {
t.Fatalf("the store was asked %v; want %v", *asked, want)
}
}
func TestAStoreThatDoesNotHaveItAnswersGone(t *testing.T) {
// The outcome wanted, already true. Told apart from success only so the sweep can say which
// happened; both are recorded, because retrying for ever is the thing to avoid.
store, _ := fakeStore(t, http.StatusNotFound)
err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/app@sha256:abc123")
if !errors.Is(err, Gone) {
t.Fatalf("a store that does not hold it answered %v, want Gone", err)
}
}
func TestAStoreWithDeletionOffSaysSoAndNamesTheRemedy(t *testing.T) {
// The registry answers 405 when it was started without deletion enabled. The remedy is a
// setting on the store's module, and saying "405" would send somebody to the wrong place.
store, _ := fakeStore(t, http.StatusMethodNotAllowed)
err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/app@sha256:abc123")
if err == nil {
t.Fatal("a store that refuses deletion was read as success")
}
if !strings.Contains(err.Error(), "REGISTRY_STORAGE_DELETE_ENABLED") {
t.Fatalf("the refusal does not name the remedy: %v", err)
}
}
func TestAReferenceThatIsNotTheMeshsOwnIsNeverAsked(t *testing.T) {
// The whole safety of the sweep is that it names only what the mesh recorded putting there.
// A reference of another shape — a vendor's image, a package version — is refused rather
// than composed into a delete somewhere that is not the mesh's store.
store, asked := fakeStore(t, http.StatusAccepted)
for _, reference := range []string{
"docker.io/library/registry@sha256:abc123",
"registry@sha256:abc123",
"1.4.2",
} {
if err := store.LetGo(context.Background(), reference); err == nil {
t.Errorf("%s was asked about; it is not a reference into the mesh's store", reference)
}
}
if len(*asked) != 0 {
t.Fatalf("the store was asked about %v", *asked)
}
}
// A reference this sweep will not address says so as ErrNotOurs, which is a fact about the
// record and not about the store (novox/hq issue 226).
//
// The sweep skips one and abandons itself for the other, so they cannot be the same error. The
// first live run met a reference recorded with the store's old address, read the refusal as "the
// store refuses everything", and collected none of the 1681 it had found.
func TestAReferenceThisSweepWillNotAddressIsToldApartFromAStoreRefusing(t *testing.T) {
store, asked := fakeStore(t, http.StatusAccepted)
for _, reference := range []string{
"docker.io/library/registry@sha256:abc123",
"127.0.0.1:5100/mesh-tools/build@sha256:abc123",
"1.4.2",
} {
err := store.LetGo(context.Background(), reference)
if !errors.Is(err, ErrNotOurs) {
t.Errorf("%s answered %v; a sweep must be able to skip it and go on", reference, err)
}
}
if len(*asked) != 0 {
t.Fatalf("the store was asked about %v", *asked)
}
}
internal/broker/consumer_from_now_test.go
-91
View File
@@ -1,91 +0,0 @@
package broker
import (
"fmt"
"os"
"testing"
"time"
"github.com/nats-io/nats.go"
)
// A consumer that reacts to announcements, made on a stream that keeps a week of them, starts from now:
// made from the start it replays every merge and build of that week (novox/hq issue 248). And a reset
// re-makes a stuck one from now, its configuration otherwise kept, and refuses a work queue.
//
// docker run -d --rm --name t -p 14231:4222 nats:2.10-alpine -js
// MESH_TEST_NATS=nats://127.0.0.1:14231 go test ./internal/broker/ -run TestAConsumerMadeFromNow
func TestAConsumerMadeFromNowNeverReplaysTheStreamsHistory(t *testing.T) {
url := os.Getenv("MESH_TEST_NATS")
if url == "" {
t.Skip("MESH_TEST_NATS unset")
}
js, err := Dial(url)
if err != nil {
t.Fatal(err)
}
defer js.Close()
const stream = "HISTORY_TEST"
_ = js.js.DeleteStream(stream)
if _, err := js.js.AddStream(&nats.StreamConfig{Name: stream, Subjects: []string{"history.>"}, Storage: nats.MemoryStorage}); err != nil {
t.Fatal(err)
}
defer func() { _ = js.js.DeleteStream(stream) }()
for i := 0; i < 5; i++ {
if _, err := js.js.Publish("history.merged", []byte(fmt.Sprint(i))); err != nil {
t.Fatal(err)
}
}
fresh := Consumer{Name: "fresh", Stream: stream, Filters: []string{"history.merged"}, Push: true,
AckWaitSeconds: 30, MaxDeliver: 5, MaxAckPending: 1, FromNow: true}
if err := js.EnsureConsumer(fresh); err != nil {
t.Fatal(err)
}
info, _ := js.js.ConsumerInfo(stream, "fresh")
if info.NumPending != 0 || info.Config.DeliverPolicy != nats.DeliverNewPolicy {
t.Fatalf("a consumer made from now holds %d of the stream's past (policy %v)", info.NumPending, info.Config.DeliverPolicy)
}
_, _ = js.js.Publish("history.merged", []byte("new"))
time.Sleep(100 * time.Millisecond)
if info, _ = js.js.ConsumerInfo(stream, "fresh"); info.NumPending != 1 {
t.Fatalf("a new announcement is not pending: %d", info.NumPending)
}
// Asserted again, it keeps where it is.
if err := js.EnsureConsumer(fresh); err != nil {
t.Fatal(err)
}
// One made from the start, as the server's default makes it, and stuck behind its history.
old := fresh
old.Name, old.FromNow = "old", false
if err := js.EnsureConsumer(old); err != nil {
t.Fatal(err)
}
if info, _ = js.js.ConsumerInfo(stream, "old"); info.NumPending != 6 {
t.Fatalf("the default should replay all six: %d", info.NumPending)
}
before, after, err := js.ResetConsumer(stream, "old")
if err != nil {
t.Fatal(err)
}
info, _ = js.js.ConsumerInfo(stream, "old")
if before.Pending != 6 || after.Pending != 0 || info.Config.MaxAckPending != 1 || info.Config.MaxDeliver != 5 ||
info.Config.AckWait != 30*time.Second || info.Config.DeliverSubject == "" {
t.Fatalf("before %+v after %+v config %+v", before, after, info.Config)
}
// Never a work queue: what is pending there is work.
const queue = "QUEUE_TEST"
_ = js.js.DeleteStream(queue)
if _, err := js.js.AddStream(&nats.StreamConfig{Name: queue, Subjects: []string{"queue.>"}, Retention: nats.WorkQueuePolicy, Storage: nats.MemoryStorage}); err != nil {
t.Fatal(err)
}
defer func() { _ = js.js.DeleteStream(queue) }()
if _, err := js.js.AddConsumer(queue, &nats.ConsumerConfig{Durable: "w", AckPolicy: nats.AckExplicitPolicy}); err != nil {
t.Fatal(err)
}
if _, _, err := js.ResetConsumer(queue, "w"); err == nil {
t.Fatal("a work queue's consumer was reset")
}
}
internal/broker/derived.go
+1 -45
View File
@@ -47,13 +47,7 @@ type Consumer struct {
// and a merge that came back rebuilt what it had just built, five times over on 2026-09-30.
// With one outstanding, the server holds the rest, and the heartbeat is keeping the message.
MaxAckPending int
// FromNow makes a consumer that does not exist yet start at the stream's end rather than its
// beginning (novox/hq issue 248). For a consumer that reacts to announcements — a merge, a build's
// outcome — on a stream that keeps a week of them: the server's default, everything the stream
// holds, replays every merge and every build of that week as if it had just happened. A consumer
// that exists keeps where it is, whatever this says; only its making is decided here.
FromNow bool
Why string
Why string
}
// seatStreamName is the stream holding a seat's inbound work. Named after the seat rather than
@@ -150,44 +144,6 @@ func ConsumerFor(p Principal) (Consumer, bool) {
}, true
}
// ModuleConsumer is one module's durable consumer, with the module and node it is for.
type ModuleConsumer struct {
Node, Module string
Consumer Consumer
}
// ConsumersOf is every module's durable consumer the composed users imply: each module user's, and
// each module a runtime carries — a carried module with no account of its own is no user (novox/hq
// issue 195), and its consumer is still the controller's to make, since the runtime reads it on the
// module's behalf (ADR 0198). One per module and node, whichever of the two named it first.
func ConsumersOf(users []Principal) []ModuleConsumer {
var out []ModuleConsumer
seen := map[string]bool{}
add := func(p Principal) {
c, needed := ConsumerFor(p)
if !needed || seen[p.Node+"/"+p.Module] {
return
}
seen[p.Node+"/"+p.Module] = true
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: c})
}
for _, p := range users {
if p.Kind == KindModule {
add(p)
}
}
for _, p := range users {
if p.Kind != KindNodeTools {
continue
}
for _, d := range p.Carries {
add(Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits,
Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches})
}
}
return out
}
// HolderConsumerFor is the worker a seat's holders share on that seat's work queue.
//
// **One worker for every holder, and each holder pulls one ask when it is idle** (novox/hq ADR
internal/broker/jetstream.go
-104
View File
@@ -1,7 +1,6 @@
package broker
import (
"context"
"crypto/sha256"
"crypto/tls"
"crypto/x509"
@@ -13,7 +12,6 @@ import (
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
)
// The JetStream side of the controller: the one place the mesh's streams and consumers are
@@ -86,13 +84,6 @@ func pinnedTo(path string) (*tls.Config, error) {
return PinnedToFingerprint(want), nil
}
// OnConn is the JetStream handle over a connection the caller already holds — the control plane's
// link — for asserting what the bus holds without dialling a second time.
func OnConn(conn *nats.Conn) *JetStream {
js, _ := conn.JetStream()
return &JetStream{conn: conn, js: js}
}
// DialPinned is Dial with the server's certificate pinned by a fingerprint the caller already holds
// — a module or a build machine that was handed one beside its credential, and has no file.
func DialPinned(url, fingerprint string, opts ...nats.Option) (*JetStream, error) {
@@ -308,9 +299,6 @@ func (j *JetStream) EnsureConsumer(c Consumer) error {
}
return nil
case errors.Is(err, nats.ErrConsumerNotFound):
if c.FromNow {
want.DeliverPolicy = nats.DeliverNewPolicy
}
if _, err := j.js.AddConsumer(c.Stream, want); err != nil {
return fmt.Errorf("creating consumer %s on %s: %w", c.Name, c.Stream, err)
}
@@ -320,51 +308,6 @@ func (j *JetStream) EnsureConsumer(c Consumer) error {
}
}
// ConsumerState is what a reset says about a consumer, before and after.
type ConsumerState struct {
DeliverPolicy string
Delivered uint64
AckFloor uint64
Pending uint64
AckPending int
}
func stateOf(info *nats.ConsumerInfo) ConsumerState {
policy, _ := info.Config.DeliverPolicy.MarshalJSON()
return ConsumerState{DeliverPolicy: strings.Trim(string(policy), `"`), Delivered: info.Delivered.Stream,
AckFloor: info.AckFloor.Stream, Pending: info.NumPending, AckPending: info.NumAckPending}
}
// ResetConsumer re-makes a consumer to start from now, its configuration otherwise unchanged (novox/hq
// issue 248): what it had not yet delivered or acknowledged is dropped, which is the point — on a stream
// that keeps history, a consumer replaying a week of announcements does nothing anyone wants. Refused on
// a work queue, where what is pending is work nobody else will do.
func (j *JetStream) ResetConsumer(stream, name string) (before, after ConsumerState, err error) {
info, err := j.js.StreamInfo(stream)
if err != nil {
return before, after, fmt.Errorf("asking about stream %s: %w", stream, err)
}
if info.Config.Retention == nats.WorkQueuePolicy {
return before, after, fmt.Errorf("%s is a work queue: what its consumer has pending is work, and a reset would drop it", stream)
}
have, err := j.js.ConsumerInfo(stream, name)
if err != nil {
return before, after, fmt.Errorf("asking about consumer %s on %s: %w", name, stream, err)
}
before = stateOf(have)
want := have.Config
want.DeliverPolicy = nats.DeliverNewPolicy
want.OptStartSeq, want.OptStartTime = 0, nil
if err := j.js.DeleteConsumer(stream, name); err != nil {
return before, after, fmt.Errorf("removing consumer %s on %s: %w", name, stream, err)
}
made, err := j.js.AddConsumer(stream, &want)
if err != nil {
return before, after, fmt.Errorf("re-making consumer %s on %s — it is gone until the controller asserts it at its next start: %w", name, stream, err)
}
return before, stateOf(made), nil
}
func retentionOf(r Retention) nats.RetentionPolicy {
switch r {
case RetentionWorkQueue:
@@ -373,50 +316,3 @@ func retentionOf(r Retention) nats.RetentionPolicy {
return nats.LimitsPolicy
}
}
// EnsureBucket creates a module's bucket if it is absent and brings its options to match if it is
// present (novox/hq ADR 0201).
//
// **An update, never a delete and recreate**, for the reason a stream is updated: recreating
// discards what the bucket holds, and what a module's state holds is data. The mesh's caps are
// asserted with the owner's options, so a bucket made by hand converges to them.
func (j *JetStream) EnsureBucket(b Bucket) error {
history := b.History
if history == 0 {
history = 1
}
js, err := jetstream.New(j.conn)
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if _, err := js.CreateOrUpdateKeyValue(ctx, jetstream.KeyValueConfig{
Bucket: b.Bucket(),
Description: b.Why(),
History: uint8(history),
TTL: time.Duration(b.TTLSeconds) * time.Second,
MaxValueSize: StateMaxValueBytes,
MaxBytes: StateMaxBytes,
Storage: jetstream.FileStorage,
}); err != nil {
return fmt.Errorf("asserting bucket %s: %w", b.Bucket(), err)
}
return nil
}
// BucketNames is every key-value bucket on the server, the mesh's and anybody else's.
func (j *JetStream) BucketNames() ([]string, error) {
js, err := jetstream.New(j.conn)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
lister := js.KeyValueStoreNames(ctx)
var out []string
for name := range lister.Name() {
out = append(out, name)
}
return out, lister.Error()
}
internal/broker/membership.go
-6
View File
@@ -45,11 +45,6 @@ type Membership struct {
// module that must tell the mesh from the world, the route proxy serving an internal name, reads
// it here rather than keeping a definition of its own.
Mesh []string `json:"mesh,omitempty"`
// State is every bucket this module's code may reach, by the name it uses for each, and whether
// it may write it (novox/hq ADR 0201): the runtime answers a bundle's state verbs from this list
// and refuses, with the reason, what is not on it — the bus enforces only the union over every
// module on the machine.
State []StateIssued `json:"state,omitempty"`
}
// Served is one address a tool is answered on.
@@ -108,7 +103,6 @@ func MembershipFor(node string, d Declared, where Placements) Membership {
m.Seats = append(m.Seats, SeatServed{Seat: s.Name, Verb: verb, Subject: seatToolSubject(s, verb, node)})
}
}
m.State = stateIssuedFor(d)
if len(d.Invokes) > 0 {
m.Reaches = map[string][]string{}
for _, t := range d.Invokes {
internal/broker/nats.go
+5 -42
View File
@@ -103,12 +103,6 @@ type Principal struct {
// permission and nothing beside it.
Invokes []string
// State is the local names of the state this principal's module keeps, and Reads the state of
// others it reads as `<module>.<name>` (novox/hq ADR 0201): a bucket each, kept by the owner's
// instances and read by whoever declares it.
State []string
Reads []string
// PasswordHash is the bcrypt hash the mesh minted. The plaintext is sealed to the principal
// and never appears here: this file is written to a node's disk and read by a server, and a
// secret that can be read from a configuration file is a secret with a wider blast radius
@@ -427,10 +421,6 @@ func PermissionsFor(p Principal) (Permissions, error) {
}
}
// 5. Its state, and the state of others it reads (novox/hq ADR 0201): every one read and
// watched, its own written too.
pub = append(pub, stateGrants(p.Module, p.State, p.Reads)...)
case KindNodeTools:
// **One process serves what every module on the machine would have served for itself**
// (novox/hq ADR 0175). Each carried module's whole tool namespace — the same grant that
@@ -475,35 +465,10 @@ func PermissionsFor(p Principal) (Permissions, error) {
pub = append(pub, invoked...)
// It says what it serves and may ask what answers (novox/hq ADR 0197): the runtime answers
// discovery for each module and seat it carries, and the console it is asks the bus.
// One service per runtime process, named for the runtime: the bus lets a principal answer each
// request once, so the runtime announces everything it carries under its own name.
sub = append(sub, announcing(append([]string{RuntimeModule}, serves...)...)...)
sub = append(sub, announcing(serves...)...)
pub = append(pub, discovering()...)
// **And it consumes for the modules it carries** (novox/hq ADR 0198, which changes ADR 0175's
// "it consumes nothing"): a module's long-running code is a bundle this runtime launches, and
// the runtime is its bus — it reads the module's own durable consumer and acknowledges what
// the module's code took. Exactly the grants the module's own principal has for that consumer,
// on its name and no other's: asking about it, pulling from it, acknowledging it. The
// consumer is still the controller's to make, from the module's own principal.
for _, d := range p.Carries {
own := Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits,
Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches}
if _, consumes := ConsumerFor(own); !consumes {
continue
}
stream, durable := consumerStream(own), consumerDurable(own)
pub = append(pub,
"$JS.API.CONSUMER.INFO."+stream+"."+durable,
"$JS.API.CONSUMER.MSG.NEXT."+stream+"."+durable,
"$JS.ACK."+stream+"."+durable+".>")
}
// **And it keeps and reads state for the modules it carries** (novox/hq ADR 0201): the union
// of what each may do with a bucket — an owner's write, a reader's read. That one module's code
// does not write another's bucket through it is the runtime's to keep, from the membership
// each assignment is issued, as it keeps each module's events under that module's own name.
for _, d := range p.Carries {
pub = append(pub, stateGrants(d.Module, stateNames(d.State), d.Reads)...)
}
// Nothing about consumers: it consumes nothing. A module's reactions to events are its
// own long-lived process, which ADR 0175 leaves where it is; what moves here is tools.
sub = unique(sub)
pub = unique(pub)
}
@@ -828,14 +793,12 @@ func invokedSubjects(invokes []string) ([]string, error) {
// protocol's discovery (novox/hq ADR 0197): the questions asked of every service, and those asked of
// each name it serves — its own and no other's, so it cannot answer for a service it is not.
func announcing(names ...string) []string {
out := []string{"$SRV.PING", "$SRV.INFO", "$SRV.STATS"}
out := []string{"$SRV.PING", "$SRV.INFO"}
for _, n := range names {
if !safeSubject.MatchString(n) {
continue
}
for _, verb := range []string{"PING", "INFO", "STATS"} {
out = append(out, "$SRV."+verb+"."+n, "$SRV."+verb+"."+n+".>")
}
out = append(out, "$SRV.PING."+n, "$SRV.PING."+n+".>", "$SRV.INFO."+n, "$SRV.INFO."+n+".>")
}
return out
}
internal/broker/nats_test.go
+8 -19
View File
@@ -378,7 +378,7 @@ func TestAModulePullsItsOwnConsumerAndNoOthers(t *testing.T) {
// their tools (novox/hq ADR 0175): every carried module's tool namespace, every held seat's verbs
// on this node, every module's membership on this node, and a call to anything. Nothing it
// consumes, because it reacts to nothing.
func TestTheRuntimeServesTheUnionAndConsumesForItsModules(t *testing.T) {
func TestTheRuntimeServesTheUnionAndConsumesNothing(t *testing.T) {
filter := Seat{Name: "node-packet-filter", Scope: "node", Serves: []string{"rules", "reload"}}
p := Principal{Kind: KindNodeTools, Node: "anchor", Module: RuntimeModule, Carries: []Declared{
{Module: "nftables", Holds: []Seat{filter}, Serves: []string{"firewall_rules"}},
@@ -408,33 +408,22 @@ func TestTheRuntimeServesTheUnionAndConsumesForItsModules(t *testing.T) {
t.Errorf("the runtime may not publish %s: %v", want, perms.Publish)
}
}
// It reads the consumer of every carried module that consumes — that module's, by its name, as
// the module's own principal could (novox/hq ADR 0198) — and of no module that consumes nothing.
for _, want := range []string{
"$JS.API.CONSUMER.INFO.EVENTS.anchor_zsh",
"$JS.API.CONSUMER.MSG.NEXT.EVENTS.anchor_zsh",
"$JS.ACK.EVENTS.anchor_zsh.>",
} {
if !contains(perms.Publish, want) {
t.Errorf("the runtime may not read zsh's consumer: %s missing from %v", want, perms.Publish)
// Nothing of what a carried module consumes, and no consumer of its own to ack.
for _, s := range perms.Subscribe {
if strings.Contains(s, ".event.") || strings.HasPrefix(s, "_DELIVER.") {
t.Errorf("the runtime was granted a delivery it has no consumer for: %s", s)
}
}
for _, s := range perms.Publish {
if (strings.HasPrefix(s, "$JS.ACK.") || strings.Contains(s, "CONSUMER")) && !strings.Contains(s, "anchor_zsh") {
t.Errorf("the runtime was granted a consumer no carried module of it consumes on: %s", s)
}
}
// It pulls; nothing is pushed to it, and it subscribes no event subject directly.
for _, s := range perms.Subscribe {
if strings.Contains(s, ".event.") || strings.HasPrefix(s, "_DELIVER.") {
t.Errorf("the runtime was granted a delivery: %s", s)
if strings.HasPrefix(s, "$JS.ACK.") || strings.Contains(s, "CONSUMER") {
t.Errorf("the runtime was granted a consumer's subject and has no consumer: %s", s)
}
}
if !perms.AllowResponses {
t.Error("the runtime answers what it is asked, and may not reply")
}
if _, needed := ConsumerFor(p); needed {
t.Error("a consumer would be made for the runtime itself; it reads its modules' consumers, never one of its own")
t.Error("a consumer would be made for the runtime, which consumes nothing")
}
// Each subject once in each list: the file is read as the mesh's authority model. One subject may
// stand in both — the runtime answers discovery on `$SRV.INFO` and, as the console, asks it
internal/broker/state.go
-169
View File
@@ -1,169 +0,0 @@
package broker
import (
"fmt"
"sort"
"strings"
)
// A module's state on the bus (novox/hq ADR 0201, design 32 §4, design 25 §3).
//
// A module names the state it keeps (`state`) and the state of others it reads (`reads`), and each
// is a key-value bucket: the server's own last-per-subject stream with direct reads, delete markers
// and watches, which is the state relationship the mesh already uses for declarations, opened to
// modules. The controller creates every bucket from the catalogue — from registration, like a
// seat's stream, so a reader may watch before the owner runs anywhere — and no module can.
//
// Pure, like everything else in this package that decides what the bus holds; jetstream.go is the
// part that asks a server.
// The mesh's caps on a bucket, the same for every module: a value is a piece of state, not a file,
// and a bucket that grew without bound would be one module filling the bus's disk for everyone.
const (
StateMaxValueBytes = 256 * 1024
StateMaxBytes = 64 * 1024 * 1024
)
// A Bucket is one module's declared state as the bus holds it.
type Bucket struct {
Module string
Name string
// History is how many values a key keeps; zero is one.
History int
// TTLSeconds is how long a value lives; zero is until replaced or deleted.
TTLSeconds int
}
// BucketName is the bucket a module's state lives in: the module and the local name joined by an
// underscore, which neither may contain, so two modules can never derive one bucket.
func BucketName(module, name string) string { return module + "_" + name }
// Bucket is this bucket's name on the bus.
func (b Bucket) Bucket() string { return BucketName(b.Module, b.Name) }
// Why is carried into the server's description of the bucket, so somebody reading the server's
// own state finds whose it is and why it is kept.
func (b Bucket) Why() string {
return fmt.Sprintf("%s's state %q (novox/hq ADR 0201): its current value per key, written by %s, "+
"read by whatever declares it reads it; kept when %s is unassigned, because it is data",
b.Module, b.Name, b.Module, b.Module)
}
// bucketOfRead is the bucket a read names, `<module>.<name>`, or false when it names none.
func bucketOfRead(read string) (string, bool) {
at := strings.LastIndex(read, ".")
if at <= 0 || at == len(read)-1 {
return "", false
}
module, name := read[:at], read[at+1:]
if !safeSubject.MatchString(module) || !safeSubject.MatchString(name) {
return "", false
}
return BucketName(module, name), true
}
// stateGrants is what a principal publishes to reach the state its modules keep and read: for every
// bucket, binding to it, reading a key directly, and an ordered consumer for listing and watching,
// created and deleted on the bucket's own stream, with its flow control answered; for a bucket an
// owner keeps, writing under the bucket's own subjects too.
//
// **Measured against a running server, 2026-10-04** (novox/hq research 024), and each one is there
// because leaving it out failed: without STREAM.INFO nothing binds; without DIRECT.GET nothing is
// read; without CONSUMER.CREATE no key is listed and nothing is watched; without CONSUMER.DELETE a
// watch cannot be stopped and lingers on the server. A write outside these is refused by the server
// — and reaches the writer as a timeout, not a refusal, which is why the runtime refuses first.
func stateGrants(module string, keeps []string, reads []string) []string {
var out []string
read := func(bucket string) {
stream := "KV_" + bucket
out = append(out,
"$JS.API.STREAM.INFO."+stream,
"$JS.API.DIRECT.GET."+stream+".>",
"$JS.API.CONSUMER.CREATE."+stream+".>",
"$JS.API.CONSUMER.DELETE."+stream+".>",
"$JS.FC."+stream+".>")
}
for _, name := range keeps {
if !safeSubject.MatchString(name) {
continue
}
bucket := BucketName(module, name)
read(bucket)
out = append(out, "$KV."+bucket+".>")
}
for _, r := range reads {
if bucket, ok := bucketOfRead(r); ok {
read(bucket)
}
}
return out
}
// StateIssued is one bucket an assignment may reach, by the name its module uses for it: its own
// state by the local name, another's as `<module>.<name>` (novox/hq ADR 0201).
type StateIssued struct {
Name string `json:"name"`
Bucket string `json:"bucket"`
Writes bool `json:"writes,omitempty"`
}
// stateIssuedFor is every bucket a module's code may reach, as its membership lists them.
func stateIssuedFor(d Declared) []StateIssued {
var out []StateIssued
for _, b := range d.State {
out = append(out, StateIssued{Name: b.Name, Bucket: BucketName(d.Module, b.Name), Writes: true})
}
for _, r := range d.Reads {
if bucket, ok := bucketOfRead(r); ok {
out = append(out, StateIssued{Name: r, Bucket: bucket})
}
}
return out
}
// stateNames is the local names of a module's own buckets.
func stateNames(buckets []Bucket) []string {
out := make([]string, 0, len(buckets))
for _, b := range buckets {
out = append(out, b.Name)
}
return out
}
// A BucketAsserter is the part of a JetStream connection bucket assertion needs.
type BucketAsserter interface {
// EnsureBucket creates the bucket if absent and brings its options to match if present, never
// discarding what it holds.
EnsureBucket(b Bucket) error
// BucketNames is every key-value bucket on the server.
BucketNames() ([]string, error)
}
// RaiseBuckets asserts every declared bucket and answers the buckets on the server that nothing
// declares any more.
//
// **Those are reported, never removed** (novox/hq ADR 0201, ADR 0030): what a module stored is
// data, and a manifest edited, a module renamed or a catalogue entry dropped is an ordinary day's
// work that must not take data with it. Removing one is a person's act.
func RaiseBuckets(a BucketAsserter, buckets []Bucket) (undeclared []string, err error) {
sorted := append([]Bucket(nil), buckets...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Bucket() < sorted[j].Bucket() })
declared := map[string]bool{}
for _, b := range sorted {
if err := a.EnsureBucket(b); err != nil {
return nil, fmt.Errorf("asserting %s's state %q: %w", b.Module, b.Name, err)
}
declared[b.Bucket()] = true
}
names, err := a.BucketNames()
if err != nil {
return nil, fmt.Errorf("listing the bus's state: %w", err)
}
for _, n := range names {
if !declared[n] {
undeclared = append(undeclared, n)
}
}
sort.Strings(undeclared)
return undeclared, nil
}
internal/broker/state_test.go
-180
View File
@@ -1,180 +0,0 @@
package broker
import (
"slices"
"strings"
"testing"
"github.com/nats-io/nats.go"
)
// The grants measured against a running server (novox/hq research 024): an owner reads and writes
// its bucket, a reader only reads, and neither reaches any other bucket.
func TestAnOwnerWritesItsStateAndAReaderOnlyReads(t *testing.T) {
owner, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "claude-code",
State: []string{"servers"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
for _, s := range []string{
"$KV.claude-code_servers.>",
"$JS.API.STREAM.INFO.KV_claude-code_servers",
"$JS.API.DIRECT.GET.KV_claude-code_servers.>",
"$JS.API.CONSUMER.CREATE.KV_claude-code_servers.>",
"$JS.API.CONSUMER.DELETE.KV_claude-code_servers.>",
"$JS.FC.KV_claude-code_servers.>",
} {
has(t, owner.Publish, s)
}
hasNot(t, owner.Publish, "$KV.>")
hasNot(t, owner.Publish, "$JS.API.>")
reader, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "console",
Reads: []string{"claude-code.servers"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
has(t, reader.Publish, "$JS.API.DIRECT.GET.KV_claude-code_servers.>")
has(t, reader.Publish, "$JS.API.CONSUMER.CREATE.KV_claude-code_servers.>")
hasNot(t, reader.Publish, "$KV.claude-code_servers.>")
for _, s := range reader.Subscribe {
if s == "$KV.claude-code_servers.>" {
t.Fatalf("a reader subscribes the bucket's subjects directly: %v", reader.Subscribe)
}
}
}
// One runtime carries every module on its machine, so its grant is the union: the owner's write
// where an owner is carried, a read where only a reader is.
func TestTheRuntimeKeepsAndReadsStateForItsModules(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindNodeTools, Node: "one", Module: RuntimeModule,
Carries: []Declared{
{Module: "claude-code", State: []Bucket{{Module: "claude-code", Name: "servers"}},
Reads: []string{"licence-manager.bindings"}},
{Module: "audit"},
}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
has(t, perms.Publish, "$KV.claude-code_servers.>")
has(t, perms.Publish, "$JS.API.DIRECT.GET.KV_licence-manager_bindings.>")
hasNot(t, perms.Publish, "$KV.licence-manager_bindings.>")
}
// A module with no state is granted nothing of any bucket — the composition of every module that
// existed before this is unchanged.
func TestAModuleWithNoStateReachesNoBucket(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "billing",
Emits: []string{"order.placed"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
for _, s := range perms.Publish {
if strings.HasPrefix(s, "$KV.") || strings.HasPrefix(s, "$JS.FC.") || strings.Contains(s, ".KV_") {
t.Fatalf("granted %q without declaring state", s)
}
}
}
// A read that names no bucket grants nothing rather than something that happens to parse.
func TestAReadThatNamesNoBucketGrantsNothing(t *testing.T) {
if got := stateGrants("a", nil, []string{"nodot", "x.", ".y", "a.b>"}); len(got) != 0 {
t.Fatalf("granted %v for reads that name no bucket", got)
}
}
// The membership lists every bucket the module's code may reach, by the name the module uses for
// it, and whether it may write it — the list the runtime refuses from.
func TestAMembershipListsTheStateItsModuleMayReach(t *testing.T) {
m := MembershipFor("one", Declared{Module: "claude-code",
State: []Bucket{{Module: "claude-code", Name: "servers"}},
Reads: []string{"licence-manager.bindings"}}, Placements{})
want := []StateIssued{
{Name: "servers", Bucket: "claude-code_servers", Writes: true},
{Name: "licence-manager.bindings", Bucket: "licence-manager_bindings"},
}
if !slices.Equal(m.State, want) {
t.Fatalf("issued %+v, want %+v", m.State, want)
}
if none := MembershipFor("one", Declared{Module: "audit"}, Placements{}); none.State != nil {
t.Fatalf("a module with no state was issued %+v", none.State)
}
}
type buckets struct {
ensured []string
on []string
}
func (b *buckets) EnsureBucket(x Bucket) error {
b.ensured = append(b.ensured, x.Bucket())
return nil
}
func (b *buckets) BucketNames() ([]string, error) { return b.on, nil }
// Every declared bucket is asserted; one on the server that nothing declares is said, not removed.
func TestRaisingStateReportsWhatNothingDeclares(t *testing.T) {
b := &buckets{on: []string{"claude-code_servers", "gone_old", "ours_by_hand"}}
undeclared, err := RaiseBuckets(b, []Bucket{{Module: "claude-code", Name: "servers"}, {Module: "a", Name: "b"}})
if err != nil {
t.Fatal(err)
}
if !slices.Equal(b.ensured, []string{"a_b", "claude-code_servers"}) {
t.Fatalf("asserted %v", b.ensured)
}
if !slices.Equal(undeclared, []string{"gone_old", "ours_by_hand"}) {
t.Fatalf("reported %v", undeclared)
}
}
// Against a real server: a bucket is created with the owner's options and the mesh's caps,
// asserting it again changes nothing and keeps what it holds, and a changed option is brought to
// match in place.
func TestABucketIsAssertedInPlace(t *testing.T) {
js := aLiveBus(t)
b := Bucket{Module: "statetest", Name: "servers"}
if _, err := RaiseBuckets(js, []Bucket{b}); err != nil {
t.Fatalf("a real server refused a module's bucket: %v", err)
}
kv, err := js.Context().KeyValue(b.Bucket())
if err != nil {
t.Fatal(err)
}
if _, err := kv.Put("all.one", []byte(`{"kept":true}`)); err != nil {
t.Fatal(err)
}
b.History = 3
if _, err := RaiseBuckets(js, []Bucket{b}); err != nil {
t.Fatalf("asserting the bucket again failed, so a restart would: %v", err)
}
got, err := kv.Get("all.one")
if err != nil || string(got.Value()) != `{"kept":true}` {
t.Fatalf("asserting again lost what the bucket held: %v %v", got, err)
}
status, err := kv.Status()
if err != nil {
t.Fatal(err)
}
if status.History() != 3 {
t.Fatalf("history is %d, the owner declared 3", status.History())
}
if s, ok := status.(*nats.KeyValueBucketStatus); ok {
if c := s.StreamInfo().Config; c.MaxMsgSize != StateMaxValueBytes || c.MaxBytes != StateMaxBytes {
t.Fatalf("the mesh's caps are not on the bucket: value %d, bucket %d", c.MaxMsgSize, c.MaxBytes)
}
}
}
// Against a real server: the handle over a connection the control plane already holds asserts a
// bucket as Dial's does — what a push uses, so a module registered since the last start has its
// bucket before its membership names it.
func TestABucketIsAssertedOverAHeldConnection(t *testing.T) {
js := aLiveBus(t)
held := OnConn(js.Conn())
if _, err := RaiseBuckets(held, []Bucket{{Module: "statetest", Name: "held"}}); err != nil {
t.Fatalf("asserting over a held connection failed: %v", err)
}
if _, err := js.Context().KeyValue("statetest_held"); err != nil {
t.Fatalf("the bucket is not there: %v", err)
}
}
internal/broker/streams.go
-1
View File
@@ -270,7 +270,6 @@ func MeshConsumers() []Consumer {
// announcement handed over behind it must wait on the server, not time out on the
// client and come back to be acted on again.
MaxAckPending: 1,
FromNow: true,
Why: "the two events the mesh's own controller reacts to, one at a time; after " +
"max-deliver it dead-letters, because an announcement it cannot act on will not " +
"become actionable",
internal/broker/testdata/composed.conf
Vendored
+4 -4
View File
@@ -25,7 +25,7 @@ accounts {
users = [
{ user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "_INBOX.enrol.>", "mesh.assignment.>", "mesh.control.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.refused", "mesh.seat.node-build-agent.accept.>"] }
subscribe: { allow: ["$JS.API.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "$SRV.STATS", "$SRV.STATS.mesh-controller", "$SRV.STATS.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.gitea.event.pull.merged", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built"] }
subscribe: { allow: ["$JS.API.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.gitea.event.pull.merged", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "enrol.one", password: "$2a$11$eeeeeeeeeeeeeeeeeeeeee", permissions: {
@@ -38,17 +38,17 @@ accounts {
} }
{ user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "$JS.API.CONSUMER.MSG.NEXT.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.one.telegram", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.telegram", "$SRV.INFO.telegram.>", "$SRV.PING", "$SRV.PING.telegram", "$SRV.PING.telegram.>", "$SRV.STATS", "$SRV.STATS.telegram", "$SRV.STATS.telegram.>", "_INBOX.one.telegram.>", "mesh.assignment.one.telegram", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.telegram", "$SRV.INFO.telegram.>", "$SRV.PING", "$SRV.PING.telegram", "$SRV.PING.telegram.>", "_INBOX.one.telegram.>", "mesh.assignment.one.telegram", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.audit", password: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_audit.>", "$JS.API.CONSUMER.INFO.EVENTS.two_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.two.audit"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.audit", "$SRV.INFO.audit.>", "$SRV.PING", "$SRV.PING.audit", "$SRV.PING.audit.>", "$SRV.STATS", "$SRV.STATS.audit", "$SRV.STATS.audit.>", "_INBOX.two.audit.>", "mesh.assignment.two.audit", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.audit", "$SRV.INFO.audit.>", "$SRV.PING", "$SRV.PING.audit", "$SRV.PING.audit.>", "_INBOX.two.audit.>", "mesh.assignment.two.audit", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.shop", password: "$2a$11$ssssssssssssssssssssss", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "$JS.API.CONSUMER.INFO.EVENTS.two_shop", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.two.shop", "mesh.mod.shop.event.order.placed", "mesh.seat.telegram-sender.accept.send"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.shop", "$SRV.INFO.shop.>", "$SRV.PING", "$SRV.PING.shop", "$SRV.PING.shop.>", "$SRV.STATS", "$SRV.STATS.shop", "$SRV.STATS.shop.>", "_INBOX.two.shop.>", "mesh.assignment.two.shop", "mesh.mod.shop.tool.>"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.shop", "$SRV.INFO.shop.>", "$SRV.PING", "$SRV.PING.shop", "$SRV.PING.shop.>", "_INBOX.two.shop.>", "mesh.assignment.two.shop", "mesh.mod.shop.tool.>"] }
allow_responses: { max: 1, ttl: "1m" }
} }
]
internal/broker/users.go
-18
View File
@@ -33,14 +33,6 @@ type Declared struct {
Watches []Seat
// Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152).
Invokes []string
// State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201).
State []Bucket
// Reads are other modules' state it reads, each `<module>.<name>` (novox/hq ADR 0201).
Reads []string
// NoAccount says the module declares no own secret named broker, so no account could ever be
// delivered to it and nothing can connect as it (novox/hq issue 195). Said in the negative so a
// record that does not say is composed as it always was.
NoAccount bool
}
// Records is what composing a user list needs to know about the mesh, and nothing more.
@@ -85,20 +77,10 @@ func Users(r Records) ([]Principal, error) {
if runtimeHere && d.Module == RuntimeModule {
continue
}
// **A module with nowhere to read an account is no user** (novox/hq issue 195). `module
// issue` refuses it one (issue 078: an account nothing reads is an orphan), so its user
// could only ever be left out of the file for want of a password — and every such module
// was named, on every status and plan, as a credential the mesh had not minted. Where the
// runtime is, it speaks for the module; where it is not, the module cannot speak at all,
// and a user would not change that.
if d.NoAccount {
continue
}
out = append(out, Principal{
Kind: KindModule, Node: node, Module: d.Module,
Emits: d.Emits, Consumes: d.Consumes, Serves: d.Serves,
Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Invokes: d.Invokes,
State: stateNames(d.State), Reads: d.Reads,
})
}
if runtimeHere {
internal/broker/users_test.go
-54
View File
@@ -296,57 +296,3 @@ func TestTheRuntimeModuleBecomesTheMachinesRuntimePrincipal(t *testing.T) {
t.Error("telegram lost its own principal when the runtime arrived on its node")
}
}
// A module that declares nowhere to read an account is no user: it could never be issued one, so it
// was only ever named as a credential the mesh had not minted (novox/hq issue 195). Where the runtime
// is, the runtime still carries it — its grants are the runtime's.
func TestAModuleThatCannotReadAnAccountIsNoUser(t *testing.T) {
r := someRecords()
r.Assigned["one"] = append(r.Assigned["one"],
Declared{Module: "packet-filter", NoAccount: true, Emits: []string{"rule.changed"}},
Declared{Module: RuntimeModule})
users, err := Users(r)
if err != nil {
t.Fatal(err)
}
for _, u := range users {
if u.Username() == "one.packet-filter" {
t.Fatalf("packet-filter declares no broker secret and was composed as a user: %v", namesOf(t, r))
}
if u.Kind == KindNodeTools {
carried := false
for _, d := range u.Carries {
carried = carried || d.Module == "packet-filter"
}
if !carried {
t.Error("the runtime stopped carrying a module that has no account of its own")
}
}
}
if names := strings.Join(namesOf(t, r), ","); !strings.Contains(names, "one.telegram") {
t.Errorf("a module that does read an account lost its user: %s", names)
}
}
// A carried module with no account still has its consumer made: the runtime reads it on the module's
// behalf (ADR 0198), and the consumer was derived from the module's own user until issue 195 took
// that user away.
func TestACarriedModuleWithNoAccountStillHasItsConsumer(t *testing.T) {
r := someRecords()
r.Assigned["one"] = append(r.Assigned["one"],
Declared{Module: "listener", NoAccount: true, Consumes: []string{"shop.order.placed"}},
Declared{Module: RuntimeModule})
r.Assigned["two"] = append(r.Assigned["two"],
Declared{Module: "auditor", Consumes: []string{"shop.order.placed"}})
users, err := Users(r)
if err != nil {
t.Fatal(err)
}
got := map[string]int{}
for _, c := range ConsumersOf(users) {
got[c.Node+"/"+c.Module]++
}
if got["one/listener"] != 1 || got["two/auditor"] != 1 || len(got) != 2 {
t.Fatalf("consumers: %v", got)
}
}
internal/builder/builder.go
+3 -106
View File
@@ -215,7 +215,7 @@ func Build(ctx context.Context, run Runner, publish Publisher,
sort.Slice(artifacts, func(i, j int) bool { return artifacts[i].Name < artifacts[j].Name })
for _, a := range artifacts {
say("artifact", "%s (%s%s) — starting", a.Name, a.Kind, langSuffix(a))
made, err := one(ctx, run, publish, manifest.Module, within, workspace, commit, credentials, a, args, held, npmrcPath, npmrc, seatBases, say)
made, err := one(ctx, run, publish, manifest.Module, within, workspace, commit, credentials, a, args, held, npmrcPath, seatBases, say)
if err != nil {
say("artifact", "%s FAILED: %v", a.Name, err)
return Result{}, err
@@ -443,7 +443,7 @@ func wantsPackages(manifest catalogue.Manifest, within string) bool {
func one(ctx context.Context, run Runner, publish Publisher,
module, tree, workspace, commit, credentials string, a catalogue.Artifact, args []string,
held map[string]string, npmrc string, registry Npmrc, seats map[string]string,
held map[string]string, npmrc string, seats map[string]string,
say func(step, format string, args ...any)) (catalogue.Built, error) {
switch a.Kind {
@@ -582,11 +582,6 @@ func one(ctx context.Context, run Runner, publish Publisher,
"holds no copy of it. Build %s first",
module, a.Name, chain.Language, chain.Base, chain.Artifact, chain.Base)
}
// The module's own packages first, where the compiler and the bundler resolve them from
// (dependencies.go); nothing at all for a module whose package.json names only the SDK.
if err := installOwn(ctx, run, tree, chain, base, registry, say); err != nil {
return catalogue.Built{}, fmt.Errorf("%s: %s: %w", module, a.Name, err)
}
say("bundle", "compiling %s in %s's toolchain", a.Language, chain.Base)
compiled, err := compile(ctx, run, tree, chain, base, a)
if err != nil {
@@ -598,12 +593,6 @@ func one(ctx context.Context, run Runner, publish Publisher,
if err != nil {
return catalogue.Built{}, fmt.Errorf("%s: writing %s's launchers failed: %w", module, a.Name, err)
}
if chain.Bundler != "" {
say("bundle", "bundling each entrypoint into one file")
if compiled, err = bundled(ctx, run, tree, chain, base, a, launchers); err != nil {
return catalogue.Built{}, fmt.Errorf("%s: bundling %s failed: %w", module, a.Name, err)
}
}
say("bundle", "compiled, packing")
body, err := pack(compiled)
if err != nil {
@@ -985,7 +974,7 @@ func compile(ctx context.Context, run Runner, tree string, chain Toolchain,
if _, err := run(ctx, tree, "docker", invocation...); err != nil {
return "", err
}
if chain.Dependencies != "" && chain.Bundler == "" {
if chain.Dependencies != "" {
// **What the bundle runs with, from the image it was compiled in** (Toolchain.Dependencies).
// A second run in the same image rather than a shell wrapped around the compiler: the
// compile line stays a plain command a reader can run by hand, and the copy is one more
@@ -1235,95 +1224,3 @@ func writeLaunchers(root string, chain Toolchain, a catalogue.Artifact) (map[str
}
return out, nil
}
// bundledSuffix is where a bundle's one-file output is written, beside what the compiler wrote.
const bundledSuffix = ".bundled"
// bundled makes every entrypoint and every launcher of a compiled bundle ONE file, in the toolchain
// image's bundler, and answers the directory to pack (novox/hq ADR 0193).
//
// **What a launched bundle runs is what it imports, and nothing else.** Every served bundle is its
// own process, so it carries its own copy of the SDK and its own dependencies inlined — the
// toolchain's whole node_modules no longer travels in every bundle. An entrypoint a process runs by
// name (`node daemon/index.js`) is bundled in place under its own name; a launcher keeps its name
// and its first line, and stays executable. A package the bundler cannot inline is named by the
// artifact (`external`), kept as an import, and only then is the toolchain's runtime directory
// copied beside the files. CommonJS inlined into an ES module still finds `require`.
func bundled(ctx context.Context, run Runner, tree string, chain Toolchain, base string,
a catalogue.Artifact, launchers map[string]string) (string, error) {
const within = "/app/modules/module"
out, final := Out(a.Name), Out(a.Name)+bundledSuffix
if err := os.RemoveAll(filepath.Join(tree, final)); err != nil {
return "", err
}
if err := os.MkdirAll(filepath.Join(tree, final), 0o755); err != nil {
return "", err
}
common := []string{"--bundle", "--platform=node", "--format=esm", "--target=node22",
"--outbase=" + out, "--outdir=" + final, "--log-level=warning",
"--banner:js=import { createRequire as __meshRequire } from 'node:module'; const require = __meshRequire(import.meta.url);"}
for _, x := range a.External {
common = append(common, "--external:"+x)
}
var plain []string
for _, e := range a.Entrypoints {
if strings.HasSuffix(e, ".js") {
plain = append(plain, out+"/"+e)
}
}
var launch []string
for _, l := range sortedValues(launchers) {
launch = append(launch, out+"/"+l)
}
// Refused by name in an image that predates the bundler, as the dependencies copy is: a bundle
// packed without it would carry nothing it imports. Run as itself: npm installs esbuild's native
// binary in place of its script, which `node` cannot run.
guard := `test -x "$0" || { echo "the toolchain image carries no bundler at $0: it predates one-file bundles, rebuild mesh-tools first" >&2; exit 1; }; exec "$0" "$@"`
step := func(entries []string, extra ...string) error {
if len(entries) == 0 {
return nil
}
invocation := []string{"run", "--rm", "--volume", tree + ":" + within, "--workdir", within, base,
"sh", "-c", guard, chain.Bundler}
invocation = append(invocation, entries...)
invocation = append(invocation, common...)
invocation = append(invocation, extra...)
_, err := run(ctx, tree, "docker", invocation...)
return err
}
if err := step(plain); err != nil {
return "", err
}
if err := step(launch, "--out-extension:.js=.mjs"); err != nil {
return "", err
}
// Plain `.js` output is an ES module; said once, as the runtime directory used to say it.
if err := os.WriteFile(filepath.Join(tree, final, "package.json"), []byte(`{"type":"module","private":true}`+"\n"), 0o644); err != nil {
return "", err
}
for _, l := range launchers {
path := filepath.Join(tree, final, filepath.FromSlash(l))
if _, err := os.Stat(path); err == nil {
if err := os.Chmod(path, 0o755); err != nil {
return "", err
}
}
}
if len(a.External) > 0 && chain.Dependencies != "" {
copying := []string{"run", "--rm", "--volume", tree + ":" + within, "--workdir", within, base,
"sh", "-c", `cp -a "$0/node_modules" "$1/"`, chain.Dependencies, final}
if _, err := run(ctx, tree, "docker", copying...); err != nil {
return "", fmt.Errorf("copying the packages %s keeps external: %w", a.Name, err)
}
}
return filepath.Join(tree, final), nil
}
func sortedValues(m map[string]string) []string {
out := make([]string, 0, len(m))
for _, v := range m {
out = append(out, v)
}
sort.Strings(out)
return out
}
internal/builder/bundle_test.go
+14 -38
View File
@@ -83,49 +83,25 @@ func TestABundleIsCompiledAndPackedWithNoDockerfile(t *testing.T) {
t.Fatalf("the bundle was not pinned: %v", got.Manifest.Resources[0])
}
// **One file per entrypoint and launcher, in the toolchain's bundler** (novox/hq ADR 0193). A
// second run in the same toolchain image bundles each into the artifact's bundled output, the SDK
// inlined, refusing by name in an image that predates the bundler; and the toolchain's
// node_modules is no longer copied into a bundle that keeps nothing external.
var bundling []string
// **And what it runs with, from the image it was compiled in** (novox/hq to-be 38 WP3). A
// second run in the same toolchain image copies the toolchain's runtime directory — the
// `"type": "module"` package.json and the pruned node_modules — into the output's root, and
// refuses by name when the image carries none rather than packing a bundle that starts nowhere.
var copied string
for _, line := range r.ran {
if strings.HasPrefix(line, "docker run") && strings.Contains(line, "esbuild") {
bundling = append(bundling, line)
if strings.HasPrefix(line, "docker run") && strings.Contains(line, "/app/runtime") {
copied = line
}
}
if len(bundling) != 2 {
t.Fatalf("want one bundling run for the entrypoints and one for the launchers:\n%s", strings.Join(r.ran, "\n"))
if copied == "" {
t.Fatalf("the bundle's dependencies were not copied in after the compile:\n%s", strings.Join(r.ran, "\n"))
}
for _, want := range []string{"mesh-tools/build@sha256:", "predates one-file bundles", "--bundle", "--format=esm",
"--platform=node", "--outdir=" + Out("code") + ".bundled", Out("code") + "/index.js"} {
if !strings.Contains(bundling[0], want) {
t.Errorf("the entrypoints' bundling lacks %q: %s", want, bundling[0])
}
if !strings.Contains(copied, "mesh-tools/build@sha256:") || !strings.Contains(copied, "predates") ||
!strings.Contains(copied, Out("code")) {
t.Fatalf("the copy does not run in the same toolchain, refuse an older image by name, or land in the artifact's output: %s", copied)
}
if !strings.Contains(bundling[1], Out("code")+"/index.serve.mjs") || !strings.Contains(bundling[1], "--out-extension:.js=.mjs") {
t.Errorf("the launcher is not bundled under its own name: %s", bundling[1])
}
if strings.Contains(strings.Join(r.ran, "\n"), "/app/runtime") {
t.Errorf("the toolchain's node_modules was copied into a bundle that keeps nothing external:\n%s", strings.Join(r.ran, "\n"))
}
if strings.Index(strings.Join(r.ran, "\n"), "--outDir") > strings.Index(strings.Join(r.ran, "\n"), "esbuild") {
t.Fatal("the bundler ran before the compile wrote its output")
}
}
// A bundle naming packages it keeps external is bundled with them as imports, and carries the
// toolchain's node_modules for them — the one case it still does.
func TestABundleKeepingAPackageExternalCarriesTheToolchainsModules(t *testing.T) {
manifest := strings.Replace(aBundle, `"entrypoints":["index.js"]`, `"entrypoints":["index.js"],"external":["sharp"]`, 1)
r, workspace := aRepository(t, manifest, map[string]string{"index.ts": "console.log(1)"})
held := map[string]string{"mesh-tools/build": "registry.invalid/mesh-tools/build@sha256:" + strings.Repeat("b", 64)}
if _, err := Build(context.Background(), compiling{r}.run, r,
"https://forge.invalid/greeter.git", "", "", workspace, held, Npmrc{}, GitCredential{}, nil); err != nil {
t.Fatal(err)
}
all := strings.Join(r.ran, "\n")
if !strings.Contains(all, "--external:sharp") || !strings.Contains(all, "/app/runtime") {
t.Errorf("an external package was not kept as an import with the toolchain's modules beside it:\n%s", all)
if strings.Index(strings.Join(r.ran, "\n"), "--outDir") > strings.Index(strings.Join(r.ran, "\n"), "/app/runtime") {
t.Fatal("the dependencies were copied before the compile wrote its output")
}
}
internal/builder/dependencies.go
-147
View File
@@ -1,147 +0,0 @@
package builder
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
)
// A module's own packages, installed before its bundle is compiled, so the bundler inlines them.
//
// **A bundle could only import what the toolchain happened to carry.** The compiler and the bundler
// resolve an import by walking up from the module's source: the module's own directory first, then
// the toolchain image's node_modules. Nothing ever put anything in the first, so a module needing a
// database driver (`pg`, `mongodb`, `mssql`) could not be a bundle at all, and kept a container whose
// recipe installed it by hand (novox/hq ADR 0198 §4: "the backend's own driver inside the bundle").
// Now the module's `package.json` says what it depends on, as any Node package does, and the build
// installs exactly that into the module's own directory before compiling.
//
// **The SDK the toolchain carries is the one a bundle is built with, whatever the module says**
// (novox/hq issue 212: the toolchain is rebuilt on every SDK release and every bundle after it). A
// module's `package.json` names `@novox/mesh-sdk` with a range — it has to, to type-check on a
// workstation — and installing that range would shadow the toolchain's copy for this module alone:
// one module compiled against an older SDK than its neighbours, chosen by a caret nobody re-reads.
// So the SDK is taken out of what is installed (and never fetched), and any copy something else
// pulls in is removed afterwards; every import of it resolves past the module's node_modules to the
// toolchain's. A module therefore cannot pin a different SDK, by design: the toolchain is the pin.
//
// **Correctness before speed.** Every build installs afresh into a fresh clone, from the lockfile
// when the module has one (`npm ci`, exact) and from its ranges otherwise; nothing installed is kept
// between builds. What is shared is npm's own download cache, a named volume, which is
// content-addressed and verified by integrity on every read — it saves the network, never the
// install. Install scripts do not run: the build node runs nobody's postinstall, and what a script
// would build natively could not be inlined into one file anyway.
// sdkPackage is the package a TypeScript bundle's launcher serves through, and the one package a
// module's own dependencies never supply (above).
const sdkPackage = "@novox/mesh-sdk"
// npmCache is the named volume npm's download cache lives in across builds on one build node.
const npmCache = "mesh-builder-npm-cache"
// ownDependencies is what a module's package.json depends on beyond the SDK, sorted; nothing when
// the module has no package.json or depends on nothing else — which builds exactly as before.
func ownDependencies(tree string) ([]string, error) {
raw, err := os.ReadFile(filepath.Join(tree, "package.json"))
if errors.Is(err, os.ErrNotExist) {
return nil, nil
}
if err != nil {
return nil, err
}
var p struct {
Dependencies map[string]string `json:"dependencies"`
}
if err := json.Unmarshal(raw, &p); err != nil {
return nil, fmt.Errorf("the module's package.json is not JSON: %w", err)
}
var names []string
for name := range p.Dependencies {
if name != sdkPackage {
names = append(names, name)
}
}
sort.Strings(names)
return names, nil
}
// installSteps is the script run inside the toolchain image, from the module's own directory ($0).
// It works in a scratch copy so the module's package.json and lockfile are never rewritten, takes
// the SDK out of what is installed, installs production dependencies only, removes any copy of the
// SDK something pulled in, and puts the result at the module's node_modules.
const installSteps = `set -e
work="$(mktemp -d)"
cp "$0/package.json" "$work/"
if [ -f "$0/package-lock.json" ]; then cp "$0/package-lock.json" "$work/"; fi
cd "$work"
node -e '
const fs = require("fs"), sdk = process.argv[1];
const p = JSON.parse(fs.readFileSync("package.json", "utf8"));
for (const k of ["dependencies", "peerDependencies", "optionalDependencies"]) if (p[k]) delete p[k][sdk];
delete p.devDependencies; delete p.scripts;
fs.writeFileSync("package.json", JSON.stringify(p));
' "$1"
shift
if [ -f package-lock.json ]; then
npm ci --omit=dev --omit=peer --ignore-scripts --no-audit --no-fund "$@"
else
npm install --omit=dev --omit=peer --ignore-scripts --no-audit --no-fund --no-package-lock "$@"
fi
find node_modules -depth -type d -path "*/node_modules/@novox/mesh-sdk" -exec rm -rf {} +
rm -rf "$0/node_modules"
cp -a node_modules "$0/node_modules"
`
// installOwn installs a TypeScript module's own production dependencies into its directory, in the
// toolchain image, before the compile — or does nothing at all for a module that has none.
func installOwn(ctx context.Context, run Runner, tree string, chain Toolchain, base string,
registry Npmrc, say func(step, format string, args ...any)) error {
if chain.Language != "typescript" {
return nil
}
deps, err := ownDependencies(tree)
if err != nil || len(deps) == 0 {
return err
}
scoped := strings.TrimSpace(registry.Scope)
if !registry.Enabled() {
// **No registry, no scoped package.** Without the mesh's registry a scoped name resolves on
// the public one, where anybody may have published it: a dependency that installs is not
// the dependency the module meant.
for _, d := range deps {
if strings.HasPrefix(d, "@novox/") {
return fmt.Errorf("the module depends on %s, and this build knows no package registry "+
"for its scope; it would resolve from the public registry, which is not where the "+
"mesh publishes it", d)
}
}
}
const within = "/app/modules/module"
invocation := []string{"run", "--rm",
"--volume", tree + ":" + within,
"--volume", npmCache + ":/root/.npm",
"--workdir", within}
var flags []string
if registry.Enabled() {
// The registry is reached where the binding says it is, which may be this machine's own
// loopback — the reason an image build that resolves packages runs on the host network too.
invocation = append(invocation, "--network", "host")
reg := strings.TrimSpace(registry.Registry)
if !strings.HasSuffix(reg, "/") {
reg += "/"
}
flags = append(flags, "--"+scoped+":registry="+reg)
}
invocation = append(invocation, base, "sh", "-c", installSteps, within, sdkPackage)
invocation = append(invocation, flags...)
say("bundle", "installing the module's own packages: %s", strings.Join(deps, ", "))
if _, err := run(ctx, tree, "docker", invocation...); err != nil {
return fmt.Errorf("installing the module's own packages (%s): %w", strings.Join(deps, ", "), err)
}
return nil
}
internal/builder/dependencies_test.go
-131
View File
@@ -1,131 +0,0 @@
package builder
import (
"context"
"strings"
"testing"
)
// A module's own packages (dependencies.go): installed into its own directory, in the toolchain,
// before the compile, so the bundler inlines them — the SDK always the toolchain's.
func buildWithPackageJSON(t *testing.T, pkg string, extra map[string]string, registry Npmrc) (*recorded, error) {
t.Helper()
files := map[string]string{"index.ts": "console.log(1)"}
if pkg != "" {
files["package.json"] = pkg
}
for k, v := range extra {
files[k] = v
}
r, workspace := aRepository(t, aBundle, files)
held := map[string]string{"mesh-tools/build": "registry.invalid/mesh-tools/build@sha256:" + strings.Repeat("b", 64)}
_, err := Build(context.Background(), compiling{r}.run, r,
"https://forge.invalid/greeter.git", "", "", workspace, held, registry, GitCredential{}, nil)
return r, err
}
func installs(r *recorded) []string {
var out []string
for _, line := range r.ran {
if strings.HasPrefix(line, "docker run") && strings.Contains(line, "npm ci") {
out = append(out, line)
}
}
return out
}
func compileIndex(r *recorded) int {
for i, line := range r.ran {
if strings.Contains(line, "--outDir") {
return i
}
}
return -1
}
func TestAModulesOwnPackagesAreInstalledInTheToolchainBeforeTheCompile(t *testing.T) {
r, err := buildWithPackageJSON(t, `{"type":"module","dependencies":{"@novox/mesh-sdk":"^0.1.0","pg":"^8"},"devDependencies":{"typescript":"^5"}}`,
nil, Npmrc{Scope: "@novox", Registry: "https://forge.invalid/api/packages/novox/npm"})
if err != nil {
t.Fatal(err)
}
got := installs(r)
if len(got) != 1 {
t.Fatalf("want one install of the module's own packages:\n%s", strings.Join(r.ran, "\n"))
}
line := got[0]
for _, want := range []string{
"mesh-tools/build@sha256:", // in the toolchain image
":/app/modules/module", // into the module's own directory
"--workdir /app/modules/module", //
npmCache + ":/root/.npm", // npm's verified download cache, and only that
"--omit=dev", "--ignore-scripts", // production packages, no build-node scripts
"npm ci", "npm install", "--no-package-lock", // the lockfile when there is one, else the ranges
"--@novox:registry=https://forge.invalid/api/packages/novox/npm/", // the scope from the mesh's registry
"--network host",
"@novox/mesh-sdk", // named, to be taken out of what is installed
} {
if !strings.Contains(line, want) {
t.Errorf("the install lacks %q:\n%s", want, line)
}
}
// The SDK is the toolchain's: never installed from the module's range, and any copy removed.
if !strings.Contains(line, `delete p[k][sdk]`) || !strings.Contains(line, `-path "*/node_modules/@novox/mesh-sdk" -exec rm -rf`) {
t.Errorf("the module's own SDK range could shadow the toolchain's SDK:\n%s", line)
}
if i, c := strings.Index(strings.Join(r.ran, "\n"), "npm ci"), compileIndex(r); c < 0 ||
i > strings.Index(strings.Join(r.ran, "\n"), "--outDir") {
t.Fatalf("the install did not run before the compile:\n%s", strings.Join(r.ran, "\n"))
}
}
// **A module with nothing beyond the SDK builds exactly as before**: the same commands, no install.
func TestAModuleDependingOnlyOnTheSDKBuildsExactlyAsBefore(t *testing.T) {
without, err := buildWithPackageJSON(t, "", nil, Npmrc{})
if err != nil {
t.Fatal(err)
}
for _, pkg := range []string{
`{"type":"module","dependencies":{"@novox/mesh-sdk":"^0.1.0"},"devDependencies":{"typescript":"^5"}}`,
`{"type":"module"}`,
} {
with, err := buildWithPackageJSON(t, pkg, map[string]string{"package-lock.json": "{}"}, Npmrc{Scope: "@novox", Registry: "https://forge.invalid/npm/"})
if err != nil {
t.Fatal(err)
}
if strings.Contains(strings.Join(with.ran, "\n"), "npm ") {
t.Fatalf("a module depending on nothing but the SDK ran npm:\n%s", strings.Join(with.ran, "\n"))
}
if len(with.ran) != len(without.ran) {
t.Fatalf("a module depending only on the SDK built differently from one with no package.json:\n%s\n---\n%s",
strings.Join(with.ran, "\n"), strings.Join(without.ran, "\n"))
}
}
}
// Without the mesh's registry a scoped package would resolve on the public one: refused by name.
func TestAScopedPackageWithNoRegistryIsRefused(t *testing.T) {
r, err := buildWithPackageJSON(t, `{"dependencies":{"@novox/mesh-sdk":"^0.1.0","@novox/other":"^1"}}`, nil, Npmrc{})
if err == nil || !strings.Contains(err.Error(), "@novox/other") {
t.Fatalf("a scoped package was installed with no registry for its scope: %v", err)
}
if strings.Contains(strings.Join(r.ran, "\n"), "--outDir") {
t.Fatal("the compile ran after the refusal")
}
// A public package installs without one, from the public registry and nothing else.
r, err = buildWithPackageJSON(t, `{"dependencies":{"mssql":"^11"}}`, nil, Npmrc{})
if err != nil {
t.Fatal(err)
}
if got := installs(r); len(got) != 1 || strings.Contains(got[0], ":registry=") || strings.Contains(got[0], "--network host") {
t.Fatalf("a public package's install: %v", got)
}
}
func TestAnUnreadablePackageJSONIsRefusedByName(t *testing.T) {
_, err := buildWithPackageJSON(t, `{"dependencies":`, nil, Npmrc{})
if err == nil || !strings.Contains(err.Error(), "package.json") {
t.Fatalf("a broken package.json was not refused by name: %v", err)
}
}
internal/builder/registry.go
+7 -29
View File
@@ -7,8 +7,6 @@ import (
"io"
"net/http"
"strings"
"github.com/novox/mesh-controller/internal/artifacts"
)
// Where built artifacts go.
@@ -68,32 +66,22 @@ func (r Registry) PublishImage(ctx context.Context, localTag, repository string)
return pinned, nil
}
// PublishArchive stores bytes as a blob, holds it by a manifest, and returns where to fetch them
// from.
// PublishArchive stores bytes as a blob and returns where to fetch them from.
//
// Two steps for the blob, which is the registry's own protocol: ask for somewhere to put it, then
// put it there naming the digest. The registry verifies the digest itself, so a blob that arrived
// corrupted is refused by the thing storing it rather than by the machine unpacking it a week
// later.
//
// **Then a manifest that names it** (novox/hq issue 253, ADR 0189). The store's own collector
// marks only from manifests, and a blob no manifest names is collected however much the mesh
// means to keep it — so an archive published bare is an archive the first nightly collection
// deletes. The holder is composed from the digest and size alone (artifacts.Holder), which is
// what lets the sweep recompute it to backfill or let go without anything being recorded here.
// What a machine is told to fetch is the blob, exactly as before.
// Two steps, which is the registry's own protocol: ask for somewhere to put it, then put it there
// naming the digest. The registry verifies the digest itself, so a blob that arrived corrupted is
// refused by the thing storing it rather than by the machine unpacking it a week later.
func (r Registry) PublishArchive(ctx context.Context, repository string, body []byte, digest string) (string, error) {
base := "http://" + r.Address + "/v2/" + repository
final := base + "/blobs/" + digest
// Already there. Blobs are immutable and named by their content, so this is not an
// optimisation — re-uploading would be asking the registry to store what it already has under
// the name it already has. It is still held: a blob published before holders existed is
// exactly the one a rebuild of the same source finds already there.
// the name it already has.
if there, err := r.has(ctx, final); err != nil {
return "", err
} else if there {
return final, r.hold(ctx, repository, digest, len(body))
return final, nil
}
start, err := http.NewRequestWithContext(ctx, http.MethodPost, base+"/blobs/uploads/", nil)
@@ -131,17 +119,7 @@ func (r Registry) PublishArchive(ctx context.Context, repository string, body []
said, _ := io.ReadAll(io.LimitReader(done.Body, 4096))
return "", fmt.Errorf("%s refused the blob: %s %s", base, done.Status, strings.TrimSpace(string(said)))
}
return final, r.hold(ctx, repository, digest, len(body))
}
// hold puts the manifest holding an archive beside it. A build whose archive could not be held is
// a failed build: recorded as published, it would be an archive the store's collector takes.
func (r Registry) hold(ctx context.Context, repository, digest string, size int) error {
store := artifacts.Store{Address: r.Address, HTTP: r.client()}
if _, err := store.HoldBlob(ctx, repository, digest, int64(size)); err != nil {
return fmt.Errorf("published %s/blobs/%s and could not hold it by a manifest: %w", repository, digest, err)
}
return nil
return final, nil
}
// has is whether this registry already holds what is at that URL.
internal/builder/registry_test.go
+6 -86
View File
@@ -9,8 +9,6 @@ import (
"net/http/httptest"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/artifacts"
)
// An OCI registry as a content-addressed blob store, which is what it is.
@@ -20,10 +18,9 @@ import (
// there is not sent again, and that a tag is never accepted as a pin.
type fakeRegistry struct {
blobs map[string][]byte
manifests map[string][]byte // "<repository>@<digest>"
puts map[string]int // blob uploads, by digest
location string
blobs map[string][]byte
uploads int
location string
}
func (f *fakeRegistry) serve(t *testing.T) *httptest.Server {
@@ -31,8 +28,6 @@ func (f *fakeRegistry) serve(t *testing.T) *httptest.Server {
if f.blobs == nil {
f.blobs = map[string][]byte{}
}
f.manifests = map[string][]byte{}
f.puts = map[string]int{}
mux := http.NewServeMux()
server := httptest.NewServer(mux)
mux.HandleFunc("/v2/", func(w http.ResponseWriter, r *http.Request) {
@@ -44,28 +39,8 @@ func (f *fakeRegistry) serve(t *testing.T) *httptest.Server {
return
}
w.WriteHeader(http.StatusNotFound)
case strings.Contains(r.URL.Path, "/manifests/"):
// The archive's holder (novox/hq issue 253): asked for with an Accept, put by digest.
repository, digest, _ := strings.Cut(strings.TrimPrefix(r.URL.Path, "/v2/"), "/manifests/")
switch r.Method {
case http.MethodHead:
if _, ok := f.manifests[repository+"@"+digest]; ok &&
strings.Contains(r.Header.Get("Accept"), "application/vnd.oci.image.manifest.v1+json") {
w.WriteHeader(http.StatusOK)
return
}
w.WriteHeader(http.StatusNotFound)
case http.MethodPut:
body, _ := io.ReadAll(r.Body)
sum := sha256.Sum256(body)
if digest != "sha256:"+hex.EncodeToString(sum[:]) {
w.WriteHeader(http.StatusBadRequest)
return
}
f.manifests[repository+"@"+digest] = body
w.WriteHeader(http.StatusCreated)
}
case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/blobs/uploads/"):
f.uploads++
where := f.location
if where == "" {
where = "/v2/upload/" + hex.EncodeToString([]byte("session"))
@@ -83,7 +58,6 @@ func (f *fakeRegistry) serve(t *testing.T) *httptest.Server {
return
}
f.blobs[digest] = body
f.puts[digest]++
w.WriteHeader(http.StatusCreated)
default:
w.WriteHeader(http.StatusNotFound)
@@ -118,57 +92,6 @@ func TestAnArchiveIsStoredAndFetchableByItsDigest(t *testing.T) {
}
}
func TestAnArchiveIsPublishedWithAManifestHoldingIt(t *testing.T) {
// The store's collector marks only from manifests, so a bare blob is one the first collection
// deletes, kept or not (novox/hq issue 253, ADR 0189). Every archive goes out held, by the
// holder the sweep can compute for itself from the digest and size.
f := &fakeRegistry{}
r := registryFor(t, f)
body := []byte("a theme")
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
where, err := r.PublishArchive(context.Background(), "shell/config", body, digest)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(where, "/v2/shell/config/blobs/"+digest) {
t.Fatalf("a machine is told to fetch %q; the blob is still what is fetched", where)
}
manifest, holder := artifacts.Holder(digest, int64(len(body)))
if got := f.manifests["shell/config@"+holder]; string(got) != string(manifest) {
t.Fatalf("the store holds %v; want the holder %s in the archive's own repository", f.manifests, holder)
}
if !strings.Contains(string(manifest), `"digest":"`+digest+`"`) {
t.Fatalf("the holder does not name the archive: %s", manifest)
}
empty := sha256.Sum256([]byte("{}"))
if _, ok := f.blobs["sha256:"+hex.EncodeToString(empty[:])]; !ok {
t.Fatal("the holder's empty config was never put, and a registry refuses a manifest without it")
}
}
func TestAnArchiveAlreadyThereIsStillHeld(t *testing.T) {
// A rebuild of the same source finds its archive already there — often one published bare,
// before holders. It is not sent again, and it is held.
f := &fakeRegistry{}
r := registryFor(t, f)
body := []byte("published bare")
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
f.blobs[digest] = body
if _, err := r.PublishArchive(context.Background(), "shell/config", body, digest); err != nil {
t.Fatal(err)
}
if f.puts[digest] != 0 {
t.Fatal("a blob already there was sent again")
}
if _, holder := artifacts.Holder(digest, int64(len(body))); f.manifests["shell/config@"+holder] == nil {
t.Fatal("a blob already there was left bare")
}
}
func TestABlobAlreadyThereIsNotSentAgain(t *testing.T) {
// Not an optimisation: blobs are named by their content, so re-uploading is asking the
// registry to store what it already has under the name it already has.
@@ -184,11 +107,8 @@ func TestABlobAlreadyThereIsNotSentAgain(t *testing.T) {
if _, err := r.PublishArchive(context.Background(), "shell/config", body, digest); err != nil {
t.Fatal(err)
}
if f.puts[digest] != 1 {
t.Fatalf("the blob was uploaded %d times", f.puts[digest])
}
if len(f.manifests) != 1 {
t.Fatalf("publishing twice left %d holders; want the one", len(f.manifests))
if f.uploads != 1 {
t.Fatalf("the blob was uploaded %d times", f.uploads)
}
}
internal/builder/standing_on_test.go
-20
View File
@@ -200,23 +200,3 @@ func TestWhatABuildReadIsTheRepositoriesItsRecipesName(t *testing.T) {
t.Fatal("a module whose recipes name no other repository read one")
}
}
// novox/hq 04-ISSUES/212: a toolchain stands on the SDK's published package, and is built with the
// exact version the mesh published — an argument that changes when the SDK does, so a rebuild after
// a release never reuses an install of the version before it.
func TestAPackageTheMeshPublishedIsPassedByItsExactVersion(t *testing.T) {
manifest := catalogue.Manifest{
Module: "mesh-tools",
Build: &catalogue.Build{
On: []catalogue.BuildsOn{{Arg: "MESH_SDK", Module: "mesh-sdk", Artifact: "lib"}},
},
}
held := map[string]string{"mesh-sdk/lib": "@novox/mesh-sdk@0.1.6"}
args, resolved, err := standingOn(context.Background(), manifest, held, noMirror)
if err != nil {
t.Fatal(err)
}
if fmt.Sprint(args) != "[--build-arg MESH_SDK=@novox/mesh-sdk@0.1.6]" || fmt.Sprint(resolved) != "[@novox/mesh-sdk@0.1.6]" {
t.Errorf("the package was passed as %v, recorded as %v", args, resolved)
}
}
internal/builder/toolchain.go
+2 -13
View File
@@ -67,23 +67,13 @@ type Toolchain struct {
// `package.json` saying `"type": "module"` — Node reads a bare `.js` as CommonJS otherwise, so a
// bundle with its dependencies and without that line still fails to start — and the pruned,
// production-only node_modules the runtime itself ships with: the SDK's and the runtime's
// dependencies, and nothing module-specific (a module's own npm dependencies are installed into
// its own directory before the compile and inlined by the bundler: dependencies.go). Empty for a
// language whose bundle carries its own —
// dependencies, and nothing module-specific yet (novox/hq ADR 0188 §5: a skeleton; a module's
// own npm dependencies are a later step). Empty for a language whose bundle carries its own —
// a Go binary is static, a Python bundle is installed with its dependencies.
//
// A toolchain image without the directory fails the build by name rather than packing a bundle
// that starts nowhere: the image predates this and must be rebuilt first.
//
// *Since the bundler (below):* copied only for a bundle that names packages it keeps external,
// which cannot be inlined; a bundle with none carries no node_modules at all.
Dependencies string
// Bundler is the bundler inside the toolchain image that makes each compiled entrypoint and each
// launcher ONE self-contained file (novox/hq ADR 0193): every served bundle is its own process
// now, so each carries its own copy of what it imports — the SDK included — and nothing else.
// A bundle shrinks from the toolchain's whole node_modules to the code it runs. Empty for a
// language whose build is already one file.
Bundler string
// SystemStamp is the variable this language's linker fills with the artifact's declared system,
// for a language whose binaries are pinned to one at link time (novox/hq ADR 0005).
//
@@ -149,7 +139,6 @@ var toolchains = []Toolchain{
Unit: UnitSources,
SourceExt: ".ts",
Dependencies: "/app/runtime",
Bundler: "/app/node_modules/esbuild/bin/esbuild",
},
{
Language: "go",
internal/catalogue/a_service_process_test.go
-166
View File
@@ -1,166 +0,0 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
// novox/hq issue 213 (ADR 0188 §1, §3): a module's own Go service is a bundle the host runs as a
// process, not an image. Each test holds one thing that had to change in the composer for the
// controller to be declared that way.
const aServiceDigest = "sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd"
// aServiceModule is the controller's shape in miniature: it answers tools of its own, its code is a
// Go bundle a process runs as an account it declares, its secrets belong to that account, it
// prepares its state, and its process replaces the container it used to run as.
func aServiceModule(t *testing.T) Manifest {
t.Helper()
raw := `{
"module": "svc", "version": "1", "tools": ["status"], "prepares": true,
"own-secrets": {"store": "${dir:state}/store"},
"secrets-owner": "svc",
"resources": [
{"id": "state", "type": "directory", "mode": "0700", "place": "mesh", "owner": "svc"},
{"id": "service", "type": "process", "name": "svc", "artifact": "code",
"run": ["./svc", "serve"], "user": "svc", "replaces": ["server"],
"env": {"SVC_STORE_FILE": "${dir:state}/store", "SVC_STORE_PORT": "${seat:mesh-store:5432}"}},
{"id": "account", "type": "user", "name": "svc", "shell": "/usr/bin/nologin", "home": "/var/lib/svc"}
],
"build": {"artifacts": [{"name": "code", "kind": "bundle", "language": "go", "system": "arch",
"from": "cmd/svc", "binary": "svc"}]}
}`
m, err := ParseManifest([]byte(raw))
if err != nil {
t.Fatalf("the service's manifest is refused: %v", err)
}
resolved, err := m.Resolve([]Built{{Name: "code", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "svc/code@" + aServiceDigest, Digest: aServiceDigest}})
if err != nil {
t.Fatal(err)
}
return resolved
}
func composeTheService(t *testing.T, with Rendering) []map[string]any {
t.Helper()
with.Needed = map[string]map[string]string{"svc": {"store": "sealed-store"}}
with.ArtifactStore = "anchor.internal:5100"
out, err := Resolution{Node: "anchor", Modules: []Manifest{aServiceModule(t)}}.Declaration(with)
if err != nil {
t.Fatalf("the service does not compose: %v", err)
}
return out
}
func indexOf(out []map[string]any, id string) int {
for i, r := range out {
if r["id"] == id {
return i
}
}
return -1
}
// A module that declares tools has every bundle served by the node's runtime unless it says
// otherwise — and the controller declares the verbs it answers as tools. Its service bundle is run
// by its own process; launched a second time by the runtime it would be a second controller
// pretending to be an MCP server.
func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
m := aServiceModule(t)
if len(m.Bundles) != 1 {
t.Fatalf("the service's bundle was not kept: %+v", m.Bundles)
}
if loads := m.Bundles[0].Loads; len(loads) != 0 {
t.Fatalf("the runtime would launch the service's own bundle as tools: %v", loads)
}
// And a bundle no resource runs still is served, as a module declaring tools always had it.
tools := Manifest{Module: "t", Version: "1", Tools: []string{"x"},
Build: &Build{Artifacts: []Artifact{{Name: "tools", Kind: ArtifactBundle, Language: "go",
System: "arch", From: "cmd/t"}}}}
resolved, err := tools.Resolve([]Built{{Name: "tools", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "t/tools@" + aServiceDigest, Digest: aServiceDigest}})
if err != nil {
t.Fatal(err)
}
if loads := resolved.Bundles[0].Loads; len(loads) != 1 || loads[0] != "t" {
t.Fatalf("a tools bundle nothing runs is no longer served: %v", loads)
}
}
// The account is created before anything is given to it. Its secrets are mesh-computed and so
// placed before the module's own resources; given to a user the machine did not have yet, they were
// refused on the first apply and the process started without them.
func TestAModulesAccountComesBeforeWhatBelongsToIt(t *testing.T) {
out := composeTheService(t, Rendering{})
account, secret := indexOf(out, "svc.account"), indexOf(out, "svc."+NeedID("store"))
if account < 0 || secret < 0 {
t.Fatalf("the account or the secret is missing: %v", out)
}
if account > secret {
t.Fatalf("the secret owned by svc is written before svc exists: account at %d, secret at %d",
account, secret)
}
if owner := out[secret]["owner"]; owner != "svc" {
t.Errorf("the secret belongs to %v, not the account its process runs as", owner)
}
}
// The process is the module's program; its preparation is the same program asked to prepare, as a
// step before it — with the same account and environment, and handing nothing over.
func TestAProcessIsPreparedByItsOwnProgram(t *testing.T) {
out := composeTheService(t, Rendering{})
step, process := indexOf(out, "svc.service-prepare"), indexOf(out, "svc.service")
if step < 0 || process < 0 || step > process {
t.Fatalf("the preparation is not a step before the process (%d, %d): %v", step, process, out)
}
s := out[step]
if s["type"] != "process" || s["run-once"] != true || s["name"] != "svc-prepare" {
t.Errorf("the preparation is not a run-once process: %v", s)
}
if run, _ := json.Marshal(s["run"]); string(run) != `["./svc","prepare"]` {
t.Errorf("the preparation runs %s", run)
}
if s["user"] != "svc" || s["source"] != out[process]["source"] {
t.Errorf("the preparation does not run the same bundle as the same account: %v", s)
}
if env, _ := s["env"].(map[string]any); env["SVC_STORE_FILE"] == nil {
t.Errorf("the preparation is not given the process's environment: %v", s["env"])
}
if _, has := s["replaces"]; has {
t.Errorf("the preparation would hand over what the process replaces: %v", s)
}
if _, has := s["args"]; has {
t.Errorf("the preparation carries a container's args: %v", s)
}
}
// What the process replaces is named as the host recorded it, `<module>.<id>`; unprefixed, the host
// matches nothing and removes the container first, as before.
func TestWhatAProcessReplacesIsNamedAsTheHostRecordedIt(t *testing.T) {
out := composeTheService(t, Rendering{})
p := out[indexOf(out, "svc.service")]
if got, _ := json.Marshal(p["replaces"]); string(got) != `["svc.server"]` {
t.Fatalf("the process replaces %s", got)
}
}
func TestWhatReplacesMayNameIsRefusedNearItsAuthor(t *testing.T) {
for what, resource := range map[string]string{
"a container": `{"id":"c","type":"container","name":"c","image":"x@` + aServiceDigest + `","replaces":["old"]}`,
"a step": `{"id":"p","type":"process","name":"p","run":["./p"],"run-once":true,"replaces":["old"]}`,
"something declared": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["p"]}`,
"another module's": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["other.old"]}`,
"not a list": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":"old"}`,
} {
raw := `{"module":"m","version":"1","resources":[` + resource + `]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "replace") {
t.Errorf("replaces on %s was accepted: %v", what, err)
}
}
ok := `{"module":"m","version":"1","resources":[{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["old"]}]}`
if _, err := ParseManifest([]byte(ok)); err != nil {
t.Errorf("a process replacing what its module no longer declares was refused: %v", err)
}
}
internal/catalogue/backup_test.go
-72
View File
@@ -1,72 +0,0 @@
package catalogue
import (
"strings"
"testing"
)
// A store provider says how its data is backed up (novox/hq ADR 0214); a provider of something that
// holds nothing does not have to.
func TestAStoreProviderWithoutABackupIsRefusedByTheCheck(t *testing.T) {
bare, err := ParseManifest([]byte(`{"module":"pg","version":"1",
"provides":[{"name":"postgres-database","scope":"mesh"}]}`))
if err != nil {
t.Fatalf("parsing refused it, and the rule is the check's: %v", err)
}
if problems := CheckBackup(bare); len(problems) != 1 || !strings.Contains(problems[0], "node-backup") {
t.Fatalf("a store with no backup passed the check: %v", problems)
}
backed, err := ParseManifest([]byte(`{"module":"pg","version":"1",
"provides":[{"name":"postgres-database","scope":"mesh"}],
"resources":[{"id":"dumps","type":"directory","mode":"0700"}],
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:dumps}"}]}`))
if err != nil {
t.Fatal(err)
}
if problems := CheckBackup(backed); len(problems) != 0 {
t.Fatalf("a store that contributes a backup was refused: %v", problems)
}
route, _ := ParseManifest([]byte(`{"module":"r","version":"1","provides":[{"name":"route","scope":"mesh"}]}`))
if problems := CheckBackup(route); len(problems) != 0 {
t.Fatalf("a route was asked for a backup: %v", problems)
}
}
// A backup contribution names its module's own directories; one it does not declare is refused
// where it is written rather than reaching the holder as the literal text.
func TestABackupNamingAnUndeclaredDirectoryIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"pg","version":"1",
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:nowhere}"}]}`))
if err == nil || !strings.Contains(err.Error(), "nowhere") {
t.Fatalf("a backup of an undeclared directory was accepted: %v", err)
}
}
// The holder receives every module's backup lines with each module's own directories filled, each
// module's under a comment naming it — and a kind written in a shell's grammar is left untouched.
func TestBackupContributionsArePlacedWithTheirModulesDirectories(t *testing.T) {
pg, err := ParseManifest([]byte(`{"module":"pg","version":"1",
"resources":[{"id":"dumps","type":"directory","path":"/srv/pg/dumps","mode":"0700"}],
"contributions":[{"seat":"node-backup","kind":"backup","content":"run pg-dump-all\npath ${dir:dumps}"}]}`))
if err != nil {
t.Fatal(err)
}
mail, err := ParseManifest([]byte(`{"module":"mail","version":"1",
"resources":[{"id":"spool","type":"directory","mode":"0700"}],
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:spool}"}]}`))
if err != nil {
t.Fatal(err)
}
placed, err := seatContributions([]Manifest{pg, mail}, BackupSeat, "backup", Rendering{})
if err != nil {
t.Fatal(err)
}
for _, want := range []string{"# pg\nrun pg-dump-all\npath /srv/pg/dumps\n", "# mail\npath /var/lib/mail/spool\n"} {
if !strings.Contains(placed, want) {
t.Errorf("placed contributions lack %q:\n%s", want, placed)
}
}
if strings.Contains(placed, "${dir:") {
t.Errorf("a directory reached the holder unfilled:\n%s", placed)
}
}
internal/catalogue/bound_into_files.go
+6 -15
View File
@@ -21,9 +21,8 @@ import (
// formats and gains no fields.
//
// **It stays name-agnostic** ([ADR 0027]). The mesh does not learn what a `postgres-database` is:
// `at`, `as`, `from` and `address` (the providing machine's private address, novox/hq ADR 0194) are
// facts about any provision at all, and everything else comes from what the provider said it
// serves — whose keys are agreed by the requirement's name, not by this file.
// `at`, `as` and `from` are facts about any provision at all, and everything else comes from what
// the provider said it serves — whose keys are agreed by the requirement's name, not by this file.
// bound is where a module says a value from one of its bindings belongs:
// ${bound:<provision>.<key>}.
@@ -47,7 +46,7 @@ func boundUsed(content string) [][2]string {
// Three facts the mesh states about any provision, plus whatever the provider said it serves. A
// module may not reach a binding it does not have — the same boundary as a secret, for the same
// reason.
func knownFor(m Manifest, needs []Needed, node string) (map[string]map[string]string, error) {
func knownFor(m Manifest, needs []Needed, node string) map[string]map[string]string {
out := map[string]map[string]string{}
for _, want := range m.Wants() {
for i := range needs {
@@ -55,20 +54,12 @@ func knownFor(m Manifest, needs []Needed, node string) (map[string]map[string]st
if n.Name != want || n.For != m.Module {
continue
}
as := ConsumerIdentity(node, IdentitySource(m.Slug, m.Module))
values := map[string]string{
"at": n.At,
"from": n.From,
"as": as,
"as": ConsumerIdentity(node, IdentitySource(m.Slug, m.Module)),
}
// What the provider derives for this consumer rather than for all of them
// (novox/hq ADR 0201). Filled here, the one place a provision and the module
// requiring it are both in hand.
served, err := ServedTo(n.Serves, as)
if err != nil {
return nil, fmt.Errorf("%s requires %s: %w", m.Module, want, err)
}
for key, value := range served {
for key, value := range n.Serves {
// The provider's own vocabulary. Rendered plainly: a port is 5432, not 5432.000000,
// which is what a float would write and what a connection string would refuse.
values[key] = plainly(value)
@@ -76,7 +67,7 @@ func knownFor(m Manifest, needs []Needed, node string) (map[string]map[string]st
out[want] = values
}
}
return out, nil
return out
}
// withOwnNames adds a module's own composed names to what it may name from one binding:
internal/catalogue/bound_into_files_test.go
-31
View File
@@ -232,34 +232,3 @@ func TestTwoModulesOnOneNodeAreTwoIdentities(t *testing.T) {
t.Fatal("one module on two machines shares an identity")
}
}
// A machine's resolver configuration must name its resolver by address — it cannot resolve the name
// of the thing it resolves names with (novox/hq ADR 0194). So a binding offers the providing
// machine's private address beside its name, and only when the machine has one.
func TestABindingOffersTheProvidersAddress(t *testing.T) {
consumer := func() Resolution {
return Resolution{
Node: "workstation",
Modules: []Manifest{{
Module: "resolv-conf",
Requires: []string{"wildcard-resolution"},
Resources: []map[string]any{{
"id": "resolv", "type": "file", "path": "/etc/resolv.conf", "mode": "0644",
"content": "nameserver ${bound:wildcard-resolution:address}\n",
}},
}},
Needs: []Needed{{Name: "wildcard-resolution", From: "anchor", At: "anchor.internal", For: "resolv-conf"}},
}
}
out, err := consumer().Declaration(Rendering{Machines: map[string]string{"anchor.internal": "10.77.0.1"}})
if err != nil {
t.Fatal(err)
}
if got := fileNamed(out, "resolv-conf.resolv")["content"]; got != "nameserver 10.77.0.1\n" {
t.Fatalf("the resolver is not named by its address: %q", got)
}
// A machine with no address yet: refused, never written with a blank where the address belongs.
if _, err := consumer().Declaration(Rendering{}); err == nil {
t.Fatal("a file naming an address the mesh does not have was composed")
}
}
internal/catalogue/build.go
+1 -60
View File
@@ -77,7 +77,6 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
// build compare equal.
out.Bundles = nil
if m.Build != nil {
run := runByAResource(m)
for _, a := range m.Build.Artifacts {
if a.Kind != ArtifactBundle {
continue
@@ -85,20 +84,9 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
made := by[a.Name]
// What the runtime loads: what the artifact said, else every entrypoint of a module
// that declares tools, else nothing (the field's own rule; see Artifact.Loads).
//
// **Never, unasked, a bundle one of the module's own resources runs** (novox/hq issue 213).
// A process the host runs is the module's service, not its tools: the controller declares
// the verbs it answers as `tools` and serves them itself, and its bundle would otherwise
// have been launched a second time by the node's runtime, as an MCP child it is not.
loads := append([]string(nil), a.Loads...)
if a.Loads == nil && len(m.Tools) > 0 && !run[a.Name] {
if a.Loads == nil && len(m.Tools) > 0 {
loads = append([]string(nil), a.Entrypoints...)
// A bundle compiled to a binary has no entrypoints: the binary is what it is, and what
// the runtime starts to serve it (novox/hq ADR 0193). So a Go tools bundle is served
// as Go — the runtime execs it — exactly as a TypeScript one is through its launcher.
if bin := BinaryOf(a); bin != "" {
loads = []string{bin}
}
}
// **Kept, never routed** (ADR 0155): the builder publishes to the store at the address
// it reached it by, and a manifest carrying that address names an installation —
@@ -220,11 +208,6 @@ func (b *Build) problems(module string) []string {
// A bundle's source is the module's own directory by definition, and what it needs to say
// is which compiler — because the mesh chooses that, and cannot choose for a module that
// has not said.
if len(a.External) > 0 && (a.Kind != ArtifactBundle || a.Language != "typescript") {
problems = append(problems, fmt.Sprintf(
"%s: %q names packages it keeps external, and only a TypeScript bundle is bundled into "+
"one file with some kept out (novox/hq ADR 0193)", module, a.Name))
}
if len(a.Env) > 0 && a.Kind != ArtifactBundle {
problems = append(problems, fmt.Sprintf(
"%s: %q is a %q and says what it is given (env). Only a bundle the node's runtime "+
@@ -259,11 +242,6 @@ func (b *Build) problems(module string) []string {
for _, e := range a.Entrypoints {
found = found || e == load
}
// A bundle compiled to a binary is one executable: the runtime loads that or nothing
// (novox/hq ADR 0193).
if bin := BinaryOf(a); bin != "" {
found = load == bin
}
if !found {
problems = append(problems, fmt.Sprintf(
"%s: %q says the runtime loads %q, which is not among its entrypoints — "+
@@ -458,40 +436,3 @@ func BinaryOf(a Artifact) string {
}
return a.Name
}
// runByAResource is the artifacts one of a module's own resources names — a process that runs it,
// a step, an archive that unpacks it — by name.
func runByAResource(m Manifest) map[string]bool {
named := map[string]bool{}
for _, r := range m.Resources {
if a, ok := r["artifact"].(string); ok && a != "" {
named[a] = true
}
}
return named
}
// undeliveredBundles says which of a module's bundles nothing would ever put on a machine (novox/hq
// 04-ISSUES/216). A bundle reaches a machine three ways: the node's runtime serves it (it says
// `loads`, or its module declares `tools`), a resource names it (a process, a step, an archive), or
// it is the runtime itself. One reached by none of them was built, recorded and pushed as success,
// and was simply absent — seven modules' tools went missing that way on 2026-10-03. Refused here,
// naming the field that would deliver it.
func undeliveredBundles(m Manifest) []string {
if m.Build == nil || m.Module == RuntimeModule {
return nil
}
named := runByAResource(m)
var problems []string
for _, a := range m.Build.Artifacts {
if a.Kind != ArtifactBundle || named[a.Name] || len(a.Loads) > 0 || len(m.Tools) > 0 {
continue
}
problems = append(problems, fmt.Sprintf(
"%s: the bundle %q would be built and never reach a machine: nothing loads it, runs it or "+
"unpacks it. A tools bundle says `loads` (the entrypoints the node's runtime serves) or its "+
"module lists its `tools`; a daemon or a step is a resource naming it (novox/hq 04-ISSUES/216)",
m.Module, a.Name))
}
return problems
}
internal/catalogue/catalogue_check_test.go
-33
View File
@@ -86,36 +86,3 @@ func TestNoCatalogueManifestNamesAnInstallation(t *testing.T) {
t.Fatalf("%d value(s) name an installation:\n %s", len(named), strings.Join(named, "\n "))
}
}
// TestEveryCatalogueStoreSaysHowItIsBackedUp is ADR 0214's check over the real catalogue: a module
// providing a store contributes a backup to node-backup, so a store added is a store backed up.
func TestEveryCatalogueStoreSaysHowItIsBackedUp(t *testing.T) {
root := catalogueRoot(t)
found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil || len(found) == 0 {
t.Fatalf("no manifests under %s: %v", root, err)
}
stores := 0
for _, p := range found {
raw, err := os.ReadFile(p)
if err != nil {
t.Fatalf("%s: %v", p, err)
}
m, err := ParseManifest(raw)
if err != nil {
continue // TestEveryCatalogueManifestParses says why
}
for _, o := range m.Provides {
if storeProvisions[o.Name] {
stores++
break
}
}
for _, problem := range CheckBackup(m) {
t.Error(problem)
}
}
if stores == 0 {
t.Fatal("no module in the catalogue provides a store, so this proved nothing")
}
}
internal/catalogue/consumer_into_serves.go
-311
View File
@@ -1,311 +0,0 @@
package catalogue
import (
"fmt"
"regexp"
"sort"
"strings"
)
// What a provider derives for one consumer, said once in the provider's definition and delivered
// to both ends (novox/hq ADR 0201, issue 124).
//
// A `serves` block is otherwise literal: the same values for every consumer. Where the provider
// *names the resource* — a bucket, a database, a vhost — the name is derived from who is asking,
// and before this the mesh had no channel for it. The provider recomputed it in its own code and
// every consumer transcribed it into its own definition by hand, which is a copy of somebody
// else's rule kept in agreement by nobody. One of three transcriptions was wrong for months.
//
// **The mesh learns no protocol here; it spells its own name in an alphabet it already knows.**
// The only fact a served value may name is the identity the mesh itself minted for the consumer,
// in one of two alphabets: as it was minted, and as a DNS label. Everything a provider wants
// around it — a prefix, a suffix, a separator — it writes around the placeholder, because a
// served value is a string.
// consumerFact is `${consumer:<fact>}` or `${consumer:<fact>:<alphabet>}`.
var consumerFact = regexp.MustCompile(`\$\{consumer:([a-z][a-z0-9-]*)(?::([a-z][a-z0-9-]*))?\}`)
// consumerFacts are what a served value may name about the consumer it is being derived for.
// One entry, deliberately: the identity is the one thing about a consumer the mesh itself chose,
// so it is the one thing the mesh can hand to a provider without either end guessing.
var consumerFacts = []string{"as"}
// consumerAlphabets are the ways the mesh will write that identity. `dns` is the mesh's own
// identifier with its separator written `-` instead of `_` — the whole of the difference between
// the alphabet the mesh mints in and the one buckets, vhosts and hostnames accept.
var consumerAlphabets = []string{"dns"}
// ServedTo fills a provider's served values for one consumer.
//
// `as` is the identity the mesh minted for that consumer — the same string it is told to present
// as a login. Values with no placeholder are returned exactly as they were, and a block with no
// placeholder at all is returned unchanged, so this costs nothing for the providers that derive
// nothing.
//
// Only strings carry placeholders. A number, a boolean or a nested object is a value the provider
// stated outright, and is left alone.
func ServedTo(serves map[string]any, as string) (map[string]any, error) {
if len(serves) == 0 {
return serves, nil
}
var out map[string]any
for _, key := range sortedAnyKeys(serves) {
text, ok := serves[key].(string)
if !ok || !strings.Contains(text, "${consumer:") {
continue
}
filled, err := consumerInto(text, as)
if err != nil {
return nil, fmt.Errorf("the value served as %q: %w", key, err)
}
if out == nil {
// Copied only once something actually changes: the caller's map is the manifest's,
// and a provider that derives nothing must not have it rewritten underneath it.
out = make(map[string]any, len(serves))
for k, v := range serves {
out[k] = v
}
}
out[key] = filled
}
if out == nil {
return serves, nil
}
return out, nil
}
// consumerInto replaces every `${consumer:…}` in one value.
//
// **A fact or an alphabet the mesh does not have is refused, not left standing.** Written through,
// the literal `${consumer:as}` would reach a configuration file and be read as a bucket name,
// failing somewhere that names neither the module nor the mesh — the same reasoning `${bound:…}`
// is refused by (boundInto).
func consumerInto(value, as string) (string, error) {
var failed error
out := consumerFact.ReplaceAllStringFunc(value, func(match string) string {
parts := consumerFact.FindStringSubmatch(match)
fact, alphabet := parts[1], parts[2]
if fact != "as" {
if failed == nil {
failed = fmt.Errorf(
"says %s, and the mesh states %s about a consumer", match, orNothing(consumerFacts))
}
return match
}
switch alphabet {
case "":
return as
case "dns":
return asDNSLabel(as)
default:
if failed == nil {
failed = fmt.Errorf(
"says %s, and the mesh writes an identity as %s", match, orNothing(consumerAlphabets))
}
return match
}
})
if failed != nil {
return "", failed
}
return out, nil
}
// asDNSLabel writes a minted identity as a DNS label.
//
// The mesh's identities are already lower-case letters, digits and `_` (ConsumerIdentity), and
// already short enough for the tightest backend they reach (CheckIdentity, twenty characters). So
// this is the separator and nothing else — no lower-casing of what is already lower case, no
// truncation to a limit the identity is already inside, no padding of a name that is already long
// enough. Each of those would be the mesh guessing at a rule it has not been given.
func asDNSLabel(as string) string {
return strings.ReplaceAll(as, "_", "-")
}
// CheckServes refuses a `serves` block that names a consumer fact or an alphabet the mesh does not
// have, when the definition is parsed rather than when a consumer is resolved.
//
// A provision nobody consumes yet still has its rule read: a definition that would be refused the
// first time somebody required it is a definition that is wrong now.
func CheckServes(m Manifest) []string {
var problems []string
for _, provision := range sortedServes(m.Serves) {
for _, key := range sortedAnyKeys(m.Serves[provision]) {
text, ok := m.Serves[provision][key].(string)
if !ok {
continue
}
// A probe identity, because what is checked is the shape of the statement and not
// what any consumer is called.
if _, err := consumerInto(text, "mesh_node_module"); err != nil {
problems = append(problems, fmt.Sprintf(
"%s serves %s, and the value it serves as %q %s", m.Module, provision, key, err))
}
}
}
return problems
}
func sortedServes(serves map[string]map[string]any) []string {
out := make([]string, 0, len(serves))
for k := range serves {
out = append(out, k)
}
sort.Strings(out)
return out
}
func sortedAnyKeys(values map[string]any) []string {
out := make([]string, 0, len(values))
for k := range values {
out = append(out, k)
}
sort.Strings(out)
return out
}
// derivedFor is what the provider on this machine derives for one consumer of one provision
// (novox/hq ADR 0201).
//
// Settled first, then derived: an operator may set a prefix on what the provider serves and the
// mesh still fills the consumer's half of it ([ADR 0174]). Only the keys that actually name the
// consumer are returned — the rest of a `serves` block is the same for every consumer and is
// already in the provider's own definition, so repeating it here would be a second copy to go
// stale.
//
// The first module in the resolved order that says it serves the provision answers, which is the
// choice servedOnThisMachine makes for the consumer's half. Nothing serving it on this machine is
// not an error: a contribution can reach a machine whose provider is a record or an adapter, and
// then there is nothing derived to tell.
func (r Resolution) derivedFor(provision, as, consumer, local string, settings SettingsBy) (map[string]any, error) {
for _, m := range r.Modules {
serves, said := m.Serves[provision]
if !said {
continue
}
var names map[string]any
for key, value := range serves {
if text, ok := value.(string); ok && strings.Contains(text, "${consumer:") {
if names == nil {
names = map[string]any{}
}
names[key] = value
}
}
if names == nil {
return nil, nil
}
// **A consumer that keeps several holders of this provision is refused** — this is issue
// 124's own failure one case to the side, and it would be just as quiet.
//
// Each holder gets its own login, `…_<local>` (ADR 0094), and a provider derives from the
// login, so it would make one resource per holder. The consumer's side has no such
// dimension: one binding file per provision, one `${bound:<provision>:<key>}`, both
// derived from the un-suffixed identity. So the provider would create the holder's
// resource and the consumer would be configured against a name nothing made — it would
// authenticate successfully and be refused on every object, which reads like a credential
// fault and is not one.
//
// Lifting this means giving the consumer's side a local dimension. That is a decision,
// not an omission, and until it is taken the mesh says so rather than guessing.
if local != "" {
return nil, fmt.Errorf(
"%s keeps several holders of %s (this one is %q), and %s derives %s for each "+
"consumer from the login the mesh minted. Each holder has its own login, and a "+
"consumer is told one value per requirement — so the two ends would name "+
"different things and nothing would compare them (novox/hq ADR 0201)",
consumer, local, provision, m.Module, orNothing(sortedAnyKeys(names)))
}
settled, err := Settle(names, settings[m.Module])
if err != nil {
return nil, fmt.Errorf("%s serving %s: %w", m.Module, provision, err)
}
derived, err := ServedTo(settled, as)
if err != nil {
return nil, fmt.Errorf("%s serving %s to %s: %w", m.Module, provision, as, err)
}
return derived, nil
}
return nil, nil
}
// notTranscribed refuses a consumer's file that writes out the value its provider derives for it,
// instead of asking for it (novox/hq ADR 0201, issue 124).
//
// **What would have caught the one wrong instance.** The object store's three consumers each wrote
// their bucket into their own configuration by hand. One of them named a predecessor's bucket, and
// nothing compared it to what the provider would actually create: the module would have
// authenticated successfully and been refused on every object, which reads like a credential fault
// and is not one. It looked authoritative for months.
//
// The test is exact and costs one string search: a definition whose file already contains the
// value the mesh is about to derive for it has written down somebody else's rule. It cannot be a
// coincidence — a derived value carries the identity the mesh minted for this very consumer on
// this very machine, which nothing else would spell out — and it cannot be checked afterwards,
// because after substitution every consumer's file contains it legitimately.
//
// Only values that actually name the consumer are judged. A provider that serves a constant under
// the same key serves the same constant to everyone, and a consumer repeating it is redundant
// rather than wrong.
func notTranscribed(resource map[string]any, known map[string]map[string]string, module string) error {
if fmt.Sprint(resource["type"]) != "file" {
return nil
}
content, ok := resource["content"].(string)
if !ok || content == "" {
return nil
}
for _, provision := range sortedKnown(known) {
values := known[provision]
identity := values["as"]
if identity == "" {
continue
}
for _, key := range sortedStringKeys(values) {
if key == "as" {
// The login is not derived from itself, and a consumer that must present it in a
// connection string legitimately has it from `${bound:…}` — which is what it will
// be after substitution, so this would judge the substitution, not the module.
continue
}
value := values[key]
if value == "" || !namesTheConsumer(value, identity) {
continue
}
if !strings.Contains(content, value) {
continue
}
return fmt.Errorf(
"%s writes %q into %v, and that is exactly what %s derives for it — a definition "+
"keeping its own copy of somebody else's naming rule is one that can disagree "+
"with it, silently. Say ${bound:%s:%s} and be told",
module, value, resource["id"], provision, provision, key)
}
}
return nil
}
// namesTheConsumer is whether a derived value was built from this consumer's identity — in the
// alphabet it was minted in, or as a DNS label. A value that does not contain it was not derived
// from it, whatever else it may be.
func namesTheConsumer(value, identity string) bool {
return strings.Contains(value, identity) || strings.Contains(value, asDNSLabel(identity))
}
func sortedKnown(known map[string]map[string]string) []string {
out := make([]string, 0, len(known))
for k := range known {
out = append(out, k)
}
sort.Strings(out)
return out
}
func sortedStringKeys(values map[string]string) []string {
out := make([]string, 0, len(values))
for k := range values {
out = append(out, k)
}
sort.Strings(out)
return out
}
internal/catalogue/contribution_dependencies_test.go
-121
View File
@@ -1,121 +0,0 @@
package catalogue
import (
"reflect"
"strings"
"testing"
)
// Defends novox/hq ADR 0210 §3: a contribution is a dependency on the seat that receives it.
func TestAContributionDependsOnTheSeatThatPlacesIt(t *testing.T) {
env := mod("theme", nil, nil, nil)
env.Environment = &Environment{Variables: map[string]string{"GTK_THEME": "Adwaita:dark"}}
path := mod("toolchain", nil, nil, nil)
path.Environment = &Environment{Path: []PathEntry{{Entry: "/opt/x/bin"}}}
shell := mod("prompt", nil, nil, nil)
shell.Shell = []ShellCode{{For: "zsh", Slot: "first", Code: "true"}}
session := mod("wallpaper", nil, nil, nil)
session.Shell = []ShellCode{{For: "xinitrc", Slot: "normal", Code: "true"}}
resources := mod("bar", nil, nil, nil)
resources.Shell = []ShellCode{{For: "xresources", Slot: "normal", Code: "x: y"}}
empty := mod("nothing", nil, nil, nil)
empty.Environment = &Environment{}
cases := map[string]struct {
m Manifest
want []string
}{
"a variable": {env, []string{EnvironmentSeat}},
"a path entry": {path, []string{EnvironmentSeat}},
"shell code": {shell, []string{LoginShellSeat}},
"the session's start": {session, []string{DisplayServerSeat}},
"the session's X resources": {resources, []string{DisplayServerSeat}},
"an empty environment": {empty, nil},
}
for name, c := range cases {
got := DependsOn(c.m)
if len(got) == 0 {
got = nil
}
if !reflect.DeepEqual(got, c.want) {
t.Errorf("%s depends on %v, want %v", name, got, c.want)
}
}
}
func TestAContributionIsMetByAHolderOnTheNodeAndRefusedWithout(t *testing.T) {
holder := mod("node-env", nil, nil, nil, Claim{Name: EnvironmentSeat})
contributor := mod("theme", nil, nil, nil)
contributor.Environment = &Environment{Variables: map[string]string{"GTK_THEME": "Adwaita:dark"}}
catalogue := map[string]Manifest{"node-env": holder, "theme": contributor}
if _, err := AssignRefusal(catalogue, "laptop", []string{"node-env"}, []string{"theme"}); err != nil {
t.Fatalf("a contributor beside the holder is refused: %v", err)
}
_, err := AssignRefusal(catalogue, "laptop", nil, []string{"theme"})
if err == nil {
t.Fatal("a contributor on a node without the holder was accepted, and its contribution would be written nowhere")
}
if !strings.Contains(err.Error(), EnvironmentSeat) || !strings.Contains(err.Error(), "node-env") {
t.Errorf("the refusal names neither the seat nor its holder: %v", err)
}
if _, err := AssignRefusal(catalogue, "laptop", nil, []string{"theme", "node-env"}); err != nil {
t.Errorf("the contributor and the holder assigned together are refused: %v", err)
}
}
func TestAHolderMeetsItsOwnContribution(t *testing.T) {
// The display server's module contributes nothing to its own seat today, but the zsh module's
// environment contributions do go to another seat: a claim meets only the seat it names.
zsh := mod("zsh", nil, nil, nil, Claim{Name: LoginShellSeat})
zsh.Shell = []ShellCode{{For: "zsh", Slot: "normal", Code: "true"}}
zsh.Environment = &Environment{Variables: map[string]string{"EDITOR": "vim"}}
catalogue := map[string]Manifest{"zsh": zsh,
"node-env": mod("node-env", nil, nil, nil, Claim{Name: EnvironmentSeat})}
unheld := UnheldDependencies(catalogue, "laptop", []Manifest{zsh}, nil)
if len(unheld) != 1 || unheld[0].Seat != EnvironmentSeat {
t.Errorf("zsh alone: unheld %v, want only %s (its shell code is its own seat's)", unheld, EnvironmentSeat)
}
}
func TestTwoModulesDeclaringOnePackageAreRefusedBeforeAnythingIsRecorded(t *testing.T) {
pacman := withResources(mod("pacman", nil, nil, nil, Claim{Name: PackageManagerSeat}),
res("package", "pacman-contrib"))
bar := withResources(mod("bar", nil, nil, nil), res("package", "bar"), res("package", "pacman-contrib"))
other := withResources(mod("other", nil, nil, nil), res("package", "other"))
catalogue := map[string]Manifest{"pacman": pacman, "bar": bar, "other": other}
err := CollisionRefusal(catalogue, "laptop", []string{"pacman"}, []string{"bar"})
if err == nil || !strings.Contains(err.Error(), "pacman-contrib") || !strings.Contains(err.Error(), "bar") {
t.Fatalf("the second owner of a package was not refused by name: %v", err)
}
if err := CollisionRefusal(catalogue, "laptop", []string{"pacman"}, []string{"other"}); err != nil {
t.Errorf("a module declaring nothing shared is refused: %v", err)
}
// A collision already on the node is status's, not a reason to refuse an unrelated assignment.
if err := CollisionRefusal(catalogue, "laptop", []string{"pacman", "bar"}, []string{"other"}); err != nil {
t.Errorf("an unrelated assignment is refused for a collision already there: %v", err)
}
}
func TestCodeForAPowerMomentDependsOnThePowerSeat(t *testing.T) {
for _, moment := range powerMoments {
m := mod("laptop", nil, nil, nil)
m.Shell = []ShellCode{{For: moment, Slot: "normal", Code: "true"}}
if got := DependsOn(m); !reflect.DeepEqual(got, []string{PowerSeat}) {
t.Errorf("code for %s depends on %v, want %s", moment, got, PowerSeat)
}
if p := m.shellProblems(); len(p) != 0 {
t.Errorf("code for %s is refused: %v", moment, p)
}
}
m := mod("laptop", nil, nil, nil)
m.Shell = []ShellCode{{For: "after-lunch", Slot: "normal", Code: "true"}}
if p := m.shellProblems(); len(p) == 0 {
t.Error("code for a moment that does not exist was accepted")
}
if s, ok := SeatNamed(PowerSeat); !ok || s.Scope != ScopeNode {
t.Errorf("%s is not a node seat of the mesh's own: %+v %v", PowerSeat, s, ok)
}
}
internal/catalogue/controller_is_a_process_test.go
-131
View File
@@ -1,131 +0,0 @@
package catalogue
import (
"encoding/json"
"fmt"
"os"
"strings"
"testing"
)
// novox/hq issue 213: the controller is a Go program and was the one piece of the mesh's own Go
// code still shipped as an image (ADR 0188 §1). Its own manifest, composed for the machine that runs
// it, is a Go bundle run by the host as a process — and no container.
func TestTheControllerIsAProcessAndNoContainer(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
m, err := ParseManifest(raw)
if err != nil {
t.Fatalf("the controller's own manifest does not parse:\n%v", err)
}
if m.Build == nil || len(m.Build.Artifacts) != 1 {
t.Fatalf("the controller builds %+v; it is one bundle", m.Build)
}
a := m.Build.Artifacts[0]
if a.Kind != ArtifactBundle || a.Language != "go" || a.System == "" || BinaryOf(a) != "mesh-controller" {
t.Fatalf("the controller's artifact is %+v, not a Go bundle naming its system and binary", a)
}
for _, c := range m.Capabilities {
if c == "container-runtime" {
t.Error("the controller still requires a container runtime on its machine")
}
}
digest := "sha256:" + strings.Repeat("c", 64)
control, err := m.Resolve([]Built{{Name: a.Name, Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "mesh-controller/" + a.Name + "@" + digest, Digest: digest}})
if err != nil {
t.Fatal(err)
}
// The node's runtime does not launch it: it serves its seat's verbs itself.
if loads := control.Bundles[0].Loads; len(loads) != 0 {
t.Errorf("the node's runtime would launch the controller as a tools bundle: %v", loads)
}
needed := map[string]map[string]string{"mesh-controller": {}}
for name := range m.OwnSecrets {
needed["mesh-controller"][name] = "sealed-" + name
}
out, err := Resolution{Node: "anchor", Modules: []Manifest{control}}.Declaration(Rendering{
Needed: needed, ArtifactStore: "anchor.internal:5100",
Seats: map[string]map[int]int{"mesh-store": {5432: 6852}},
})
if err != nil {
t.Fatalf("the controller does not compose: %v", err)
}
var process, step map[string]any
account, firstSecret := -1, -1
for i, r := range out {
switch {
case r["type"] == "container":
t.Errorf("the controller's declaration still runs a container: %v", r)
case r["id"] == "mesh-controller.controller":
process = r
case r["id"] == "mesh-controller.controller-prepare":
step = r
if process != nil {
t.Error("the controller's preparation is placed after the process it prepares for")
}
case r["type"] == "user" && r["name"] == "mesh-controller":
account = i
case strings.HasPrefix(fmt.Sprint(r["id"]), "mesh-controller.needs-") && firstSecret < 0:
firstSecret = i
}
}
if process == nil {
t.Fatalf("the controller's process is not in its declaration: %v", out)
}
if run, _ := json.Marshal(process["run"]); string(run) != `["./mesh-controller","serve"]` {
t.Errorf("the controller is run as %s, not its own bundle's binary", run)
}
if process["source"] != "anchor.internal:5100/mesh-controller/"+a.Name+"@"+digest || process["digest"] != digest {
t.Errorf("the controller's bundle is fetched from %v (%v)", process["source"], process["digest"])
}
// The user: an account the host declares, which owns what the process reads.
if process["user"] != "mesh-controller" || account < 0 {
t.Errorf("the controller runs as %v, and the account declared is at %d", process["user"], account)
}
if firstSecret >= 0 && account > firstSecret {
t.Error("the controller's secrets are written before the account they belong to exists")
}
for _, r := range out {
if strings.HasPrefix(fmt.Sprint(r["id"]), "mesh-controller.needs-") && r["owner"] != "mesh-controller" {
t.Errorf("%v belongs to %v, which the controller's process cannot read", r["id"], r["owner"])
}
}
if dir := fileNamed(out, "mesh-controller.mesh-state"); dir == nil || dir["owner"] != "mesh-controller" {
t.Errorf("the controller's state directory is not its account's to enter: %v", dir)
}
// Each mount became a path the process reads: nothing it is told is a path inside a container.
state := fmt.Sprint(fileNamed(out, "mesh-controller.mesh-state")["path"])
env, _ := process["env"].(map[string]any)
for key, value := range env {
v := fmt.Sprint(value)
if strings.HasPrefix(v, "/run/secrets") || strings.HasPrefix(v, "/broker-tls") {
t.Errorf("%s=%s is a path inside the container the controller no longer runs in", key, v)
}
if strings.HasSuffix(key, "_FILE") && !strings.HasPrefix(v, state+"/") {
t.Errorf("%s=%s is not one of the files the mesh places for it", key, v)
}
}
if env["MESH_BROKER_CERTIFICATE"] != "/var/lib/mesh-broker-tls/tls.crt" {
t.Errorf("the controller reads the broker's certificate from %v", env["MESH_BROKER_CERTIFICATE"])
}
if env["MESH_STORE_INVENTORY_PORT"] != "6852" {
t.Errorf("the controller is told the store is on %v; the node put it on 6852", env["MESH_STORE_INVENTORY_PORT"])
}
// The handover: the container it ran as goes only once this is running.
if got, _ := json.Marshal(process["replaces"]); string(got) != `["mesh-controller.server"]` {
t.Errorf("the controller's process replaces %s, not the container it ran as", got)
}
// And its state is prepared first, by the same program as the same account.
if step == nil || step["run-once"] != true || step["user"] != "mesh-controller" {
t.Fatalf("the controller's preparation is %v", step)
}
if run, _ := json.Marshal(step["run"]); string(run) != `["./mesh-controller","prepare"]` {
t.Errorf("the controller's preparation runs %s", run)
}
}
internal/catalogue/declaration.go
+12 -164
View File
@@ -155,10 +155,6 @@ type Rendering struct {
// standing beside the machines and looking as real as they do.
Machines map[string]string
// Zones is every zone a module in the mesh answers itself, where it is answered (novox/hq ADR
// 0199): the mesh's resolver forwards each one there.
Zones []ZoneAt
Settings SettingsBy
Generators map[string]Generator
// Grants are the credentials this node must create, for the provisions it offers. Passed in
@@ -419,13 +415,6 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
filtering := AsNftables(rules, with.Mesh, r.PublicDomain != "", with.Foundation,
with.OutwardLinks, with.TunnelInterface)
// **A variable two modules set is refused whether or not anything places it** (novox/hq ADR
// 0203 §5): the account has one environment, and a machine whose holder arrives later should not
// be the moment two modules are found to disagree about it.
if err := variablesSetOnce(r.Modules); err != nil {
return nil, err
}
var out []map[string]any
for _, m := range r.Modules {
if with.Adopted && m.Filtering != nil {
@@ -614,14 +603,14 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// and nothing would say so.
continue
}
first = append(first, ownedBy(r.provisionsAs(m), map[string]any{
first = append(first, map[string]any{
// One file per holder — the consumer's module with its local name after it
// where it keeps several (ADR 0094); the lab found two files with one id.
"id": GrantID(to, g.Consumer+"."+holderAs(g.From, g.Local)),
"type": "file",
"path": grantPath(m.Grants[to], g.Consumer, holderAs(g.From, g.Local)),
"sealed": g.Sealed,
}))
})
}
}
for _, to := range sortedKeys(m.Binds) {
@@ -656,16 +645,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
if err != nil {
return nil, err
}
as := ConsumerIdentity(r.Node, IdentitySource(m.Slug, m.Module))
// What the provider derives for THIS consumer, filled here where the consumer is
// known (novox/hq ADR 0201). The same fill knownFor does below, so the binding file
// and the module's `${bound:…}` substitutions cannot say different things.
told := *found
told.Serves, err = ServedTo(told.Serves, as)
if err != nil {
return nil, fmt.Errorf("%s is told about %s: %w", m.Module, to, err)
}
file, err := boundFile(told, m.Binds[to], as, own)
file, err := boundFile(*found, m.Binds[to], ConsumerIdentity(r.Node, IdentitySource(m.Slug, m.Module)), own)
if err != nil {
return nil, err
}
@@ -713,15 +693,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// Now, and not before: a module whose resources are computed replaces them wholesale, and
// merging earlier would throw away the files it still needs.
//
// **Except the module's own accounts, which go before even those** (novox/hq issue 213). What
// the mesh computes may belong to one: a module whose code runs as an account it declares has
// its secrets written owned by that account, and a file given to a user the machine does not
// have yet fails — so on the first apply the secrets were refused, the process started without
// them, and the second apply healed it, which is the fault the paragraph above describes.
// An account depends on nothing the mesh computes.
accounts, rest := accountsFirst(resources)
resources = append(append(accounts, first...), rest...)
resources = append(append([]map[string]any{}, first...), resources...)
// No container is given the mesh's names (novox/hq ADR 0148). It used to be: every
// container got the whole roster as `--add-host` entries at creation, and a name that
@@ -739,10 +711,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
return nil, err
}
// And what its bindings say, for the half of a connection that is not secret.
known, err := knownFor(m, r.Needs, r.Node)
if err != nil {
return nil, err
}
known := knownFor(m, r.Needs, r.Node)
// A requirement answered on this same machine is not in r.Needs — its binding file is
// written from `here` (above) — and so `${bound:…}` could not name it, though the file
// beside it said the same facts. Filled from the same answer, so the two cannot disagree.
@@ -759,24 +728,10 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
}
local := *answered
local.For = m.Module
here, err := knownFor(m, []Needed{local}, r.Node)
if err != nil {
return nil, err
}
for provision, values := range here {
for provision, values := range knownFor(m, []Needed{local}, r.Node) {
known[provision] = values
}
}
// And the providing machine's private address, beside its name (novox/hq ADR 0194). A name is
// what nearly every consumer wants; the one that cannot use it is a machine's resolver
// configuration, which must reach the resolver before it can resolve anything — the resolver's
// own name included. Absent when the machine has no address yet, so a file naming it is refused
// rather than written with a blank where an address belongs.
for _, values := range known {
if address := with.Machines[values["at"]]; address != "" {
values["address"] = address
}
}
// And what the module is called through each requirement it contributes to (novox/hq
// 04-ISSUES/122) — the same composition its binding file carries.
for provision, values := range known {
@@ -798,17 +753,6 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// And the machine underneath, which no binding of its own can tell it.
thisMachine := machineFacts(r, with.Names, with.MeshRange)
// **A definition that already holds the answer transcribed it** (novox/hq ADR 0201).
// Judged over what the module itself declares, and before anything is substituted: the
// mesh's own generated files — the binding, the contributions — legitimately carry the
// derived value, and after substitution so does every consumer's file, so this is the one
// moment the two can be told apart.
for _, own := range m.Resources {
if err := notTranscribed(own, known, m.Module); err != nil {
return nil, err
}
}
// Which of this module's files carry a secret, for the rule that a container may not read
// one of them as its environment without saying so (ADR 0086, issue 041).
secretFiles := secretFilesOf(resources)
@@ -910,13 +854,6 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
return nil, err
}
publishedOn(copied, m.Module, with)
// The account's environment and every module's shell code, where this module holds the
// seat that places them (novox/hq ADR 0203, ADR 0204). Gathered from every module on
// the node, as the jails are, and **last of every placeholder pass**: shell code is a
// shell's own syntax, full of `${…}` no pass above should ever be shown.
if err := contributionsInto(copied, m, r.Modules, thisMachine, with); err != nil {
return nil, err
}
copied["id"] = m.Module + "." + fmt.Sprint(resource["id"])
// A service saying what it reflects names resources within its own module, so those
// are prefixed too or they would point at nothing.
@@ -935,21 +872,6 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
if renamed := reflectsRenamed(m.Module, resource["reload-on"]); renamed != nil {
copied["reload-on"] = renamed
}
// And which of its module's containers a scheduled step holds still (novox/hq ADR 0189).
// **The loudest of the three when it is missed.** An unprefixed `restart-on` matches
// nothing and a service quietly never restarts; an unprefixed `while-stopped` names a
// container the declaration does not contain, and the host refuses the whole
// declaration — so the machine takes nothing at all, for every push, until this is
// right. That is what it did on the control node (2026-10-04).
if renamed := reflectsRenamed(m.Module, resource[WhileStopped]); renamed != nil {
copied[WhileStopped] = renamed
}
// And what a process replaces (novox/hq issue 213): a resource of this module's that it
// no longer declares, named as the host recorded it, or the host hands nothing over and
// removes it first.
if renamed := reflectsRenamed(m.Module, resource["replaces"]); renamed != nil {
copied["replaces"] = renamed
}
// **What reads one of this module's own secrets is restarted when it changes** (novox/hq
// issue 203, issue 206). A credential is re-issued by the mesh, and a container that
// mounted the old file keeps the old one open: the build machine ran for an hour on a
@@ -979,7 +901,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// plane's; making a name resolve is the module's software. Emitted as ordinary files under
// this module's name, so they are applied, reported and removed exactly as anything else
// it declares.
given, err := FactsWithZonesInto(m, r, with.Names, with.Machines, with.Accounts, with.Suffix, with.Zones)
given, err := FactsInto(m, r, with.Names, with.Machines, with.Accounts, with.Suffix)
if err != nil {
return nil, err
}
@@ -1239,41 +1161,6 @@ type Contribution struct {
// requirement's name — everything providing `reverse-proxy` understands the same shape, which
// is what makes swapping one for another cost nothing.
Values map[string]any `json:"values"`
// Derived is what this provider's own definition said it derives for this consumer, already
// derived (novox/hq ADR 0201).
//
// **The provider is told, rather than recomputing it.** A served value may name the consumer's
// identity — a bucket named for who is asking, a database prefixed with it — and before this
// the rule lived twice: once in the provisioner's code, once transcribed into every consumer's
// definition. The mesh fills the provider's own statement here and delivers the same filled
// value to the consumer, so the two cannot disagree: there is no second computation to
// disagree with.
//
// Only the keys that are per-consumer. The rest of what the provider serves is the same for
// everyone and is in its own definition, where it already is.
Derived map[string]any `json:"derived,omitempty"`
}
// provisionsAs is the account that reads what the mesh writes for this provider: the one secret
// per consumer it must open to set that consumer's password (novox/hq issue 225).
//
// **A root-owned 0600 file is one that process cannot read**, which is the same sentence already
// written above for a module's own secrets — and the grant secret is the other kind of secret
// the mesh writes for a module, so it is the same rule.
//
// Which account depends on where the module's code runs. A module whose code is a bundle is run
// by the node's tool runtime, as the node's account ([ADR 0198](0198)); one still in a container
// is whatever it declares as its secrets owner. Nothing names these paths, so the rule that
// claims a bundle's other files by the words that name them (givenTo) cannot reach them: the
// harness composes a grant secret's path from the contributions file, not from a word.
//
// Empty is root, which is what it was and what a module with no bundle and no declared owner
// still wants.
func (r Resolution) provisionsAs(m Manifest) string {
if len(m.Bundles) > 0 && r.Account != "" {
return r.Account
}
return m.SecretsOwner
}
// grantPath is where one consumer's sealed credential lands on the providing machine.
@@ -1365,17 +1252,12 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
// told about it and withdraws the login on its next pass.
continue
}
as := holderAs(ConsumerIdentity(g.Consumer, IdentitySource(g.Slug, g.From)), g.Local)
derived, err := r.derivedFor(g.Provision, as, g.From, g.Local, settings)
if err != nil {
return nil, err
}
out[g.Provision] = append(out[g.Provision], Contribution{
From: g.From, Node: g.Consumer, At: g.At, Values: g.Values, Derived: derived,
From: g.From, Node: g.Consumer, At: g.At, Values: g.Values,
// One holder per local name: the identity the consumer is known by, and the local name
// after it where the module keeps several (ADR 0094). Not a login any backend checks —
// a secret is not a login — so the identity limit does not apply to the suffix.
As: as,
As: holderAs(ConsumerIdentity(g.Consumer, IdentitySource(g.Slug, g.From)), g.Local),
Secret: grantPath(directories[g.Provision], g.Consumer, holderAs(g.From, g.Local)),
})
if granted[g.Provision] == nil {
@@ -2128,18 +2010,12 @@ func preparationTarget(m Manifest) string {
return ""
}
for _, r := range m.Resources {
// A container, or a process the host runs from a bundle the module built (novox/hq issue
// 213): the same program in the same context, hosted as a unit rather than a container.
kind := fmt.Sprint(r["type"])
if (kind != "container" && kind != "process") || !ownArtifact(r, m.Module) {
if fmt.Sprint(r["type"]) != "container" || !ownArtifact(r, m.Module) {
continue
}
if once, _ := r["run-once"].(bool); once {
continue
}
if r["schedule"] != nil {
continue
}
return fmt.Sprint(r["id"])
}
return ""
@@ -2153,10 +2029,7 @@ func ownArtifact(resource map[string]any, module string) bool {
return true
}
image, _ := resource["image"].(string)
// A process or an archive carries what was built as its source (novox/hq issue 213).
source, _ := resource["source"].(string)
return strings.HasPrefix(image, ArtifactStoreScheme+module+"/") ||
strings.HasPrefix(source, ArtifactStoreScheme+module+"/")
return strings.HasPrefix(image, ArtifactStoreScheme+module+"/")
}
// prepared is the module's own resource as the step that prepares its state: the same image, the same
@@ -2178,19 +2051,7 @@ func prepared(from map[string]any) map[string]any {
step["id"] = fmt.Sprint(from["id"]) + "-prepare"
step["name"] = fmt.Sprint(from["name"]) + "-prepare"
step["run-once"] = true
if fmt.Sprint(from["type"]) == "process" {
// A process says its whole command: the program, then its arguments. The step is the same
// program asked to prepare (novox/hq issue 213). It replaces nothing — what the process
// replaces is handed over to the process, never to the step that runs before it — and a
// step is not restarted, it runs again when what it reads changed, which `restart-on` says.
run := stringsIn(from["run"])
if len(run) > 0 {
step["run"] = []any{run[0], PreparationArgument}
}
delete(step, "replaces")
} else {
step["args"] = []any{PreparationArgument}
}
step["args"] = []any{PreparationArgument}
delete(step, "ports")
delete(step, "ip")
delete(step, "schedule")
@@ -2335,16 +2196,3 @@ func withRestartOn(have any, add []string) []any {
}
return out
}
// accountsFirst splits a module's resources into its accounts and everything else, each in the order
// written.
func accountsFirst(resources []map[string]any) (accounts, rest []map[string]any) {
for _, r := range resources {
if fmt.Sprint(r["type"]) == "user" {
accounts = append(accounts, r)
continue
}
rest = append(rest, r)
}
return accounts, rest
}
internal/catalogue/derived_for_consumer_test.go
-343
View File
@@ -1,343 +0,0 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
// What a provider derives for each consumer, said once and delivered to both ends
// (novox/hq ADR 0201, issue 124).
//
// The failure these are written against: the object store's provisioner derived each consumer's
// bucket from the login the mesh minted, in its own code, and the mesh had no channel to tell the
// consumer which bucket that was — so all three consumers wrote the answer into their own
// definitions by hand. Two were right. One named a predecessor's bucket and would have
// authenticated successfully and been refused on every object. Each of them also named the
// machine the module happens to run on, which a definition may not do.
// store is an object store in the shape minio has: it serves a region and a port to everyone, and
// a bucket named for whoever is asking.
func store() Manifest {
return Manifest{
Module: "store", Version: "1",
Provides: FromAnywhere("s3-bucket"),
Listens: []Listening{{Port: 9000, Protocol: "tcp", From: FromMesh}},
Serves: map[string]map[string]any{"s3-bucket": {
"region": "eu-west",
"bucket": "${consumer:as:dns}",
}},
Receives: map[string]string{"s3-bucket": "/var/lib/store/grants/mesh.json"},
Grants: map[string]string{"s3-bucket": "/var/lib/store/grants"},
Resources: []map[string]any{{
"id": "server", "type": "container", "name": "store", "ports": []any{"9000"},
}},
}
}
// files is a consumer that writes the bucket into its own configuration — which is the thing it
// could not do before, and had to transcribe.
func files() Manifest {
return Manifest{
Module: "files", Version: "1", Slug: "files",
Requires: []string{"s3-bucket"},
Binds: map[string]string{"s3-bucket": "/var/lib/files/store.json"},
Secrets: map[string]string{"s3-bucket": "/var/lib/files/store.secret"},
Resources: []map[string]any{{
"id": "env", "type": "file", "path": "/var/lib/files/env", "mode": "0600",
"content": "BUCKET=${bound:s3-bucket:bucket}\nREGION=${bound:s3-bucket:region}\n",
}},
}
}
// pics is a second consumer of the same provider on the same machine: two derivations, neither
// the other's.
func pics() Manifest {
return Manifest{
Module: "pics", Version: "1", Slug: "pics",
Requires: []string{"s3-bucket"},
Binds: map[string]string{"s3-bucket": "/var/lib/pics/store.json"},
Secrets: map[string]string{"s3-bucket": "/var/lib/pics/store.secret"},
Resources: []map[string]any{{
"id": "env", "type": "file", "path": "/var/lib/pics/env", "mode": "0600",
"content": "BUCKET=${bound:s3-bucket:bucket}\n",
}},
}
}
// The three places the derived value lands must agree, because agreeing is the whole point: the
// consumer's own file, the binding it reads as JSON, and the provider's contributions entry.
func TestADerivedValueReachesBothEndsAndAgrees(t *testing.T) {
r, err := Resolve(shelf(store(), files()), []string{"store", "files"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
out, err := r.Declaration(Rendering{Grants: []Grant{{
Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Values: map[string]any{}, Sealed: "c2VhbGVk",
}}})
if err != nil {
t.Fatal(err)
}
// The mesh minted this identity for the consumer; the bucket is that identity as a DNS label.
// Derived here with the mesh's own function, so the test cannot agree with a wrong rule.
as := ConsumerIdentity("workstation", IdentitySource("files", "files"))
want := strings.ReplaceAll(as, "_", "-")
if want == as || !strings.Contains(as, "_") {
t.Fatalf("the mesh's identity %q has no separator to rewrite; this test proves nothing", as)
}
env := fileNamed(out, "files.env")
if env == nil {
t.Fatalf("the consumer was given no file: %v", out)
}
if got := env["content"].(string); !strings.Contains(got, "BUCKET="+want+"\n") {
t.Errorf("the consumer's own file was not told the bucket:\n%s\nwant BUCKET=%s", got, want)
}
binding := fileNamed(out, "files.bound-s3-bucket")
if binding == nil {
t.Fatalf("the consumer was given no binding: %v", out)
}
var said struct {
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal([]byte(binding["content"].(string)), &said); err != nil {
t.Fatal(err)
}
if said.Serves["bucket"] != want {
t.Errorf("the binding says the bucket is %q, want %q", said.Serves["bucket"], want)
}
// And what is the same for everybody is still the same for everybody.
if said.Serves["region"] != "eu-west" {
t.Errorf("the binding lost what the provider serves to all: %v", said.Serves)
}
given := storeGrants(t, out)
if len(given) != 1 {
t.Fatalf("the provider was told about %d consumer(s): %v", len(given), given)
}
if given[0].Derived["bucket"] != want {
t.Errorf("the provider was told the bucket is %v, and the consumer was told %q — "+
"the two ends disagree, which is the whole failure", given[0].Derived["bucket"], want)
}
// Only the per-consumer half. The region is the same for everyone and is already in the
// provider's own definition; repeating it here would be a copy to go stale.
if _, carried := given[0].Derived["region"]; carried {
t.Errorf("the provider was handed back what it already says for everyone: %v", given[0].Derived)
}
}
// Two consumers of one provider on one machine get two buckets, and neither gets the other's.
func TestTwoConsumersOfOneProviderGetTheirOwnDerivation(t *testing.T) {
r, err := Resolve(shelf(store(), files(), pics()),
[]string{"store", "files", "pics"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
out, err := r.Declaration(Rendering{Grants: []Grant{
{Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Values: map[string]any{}, Sealed: "c2VhbGVk"},
{Provision: "s3-bucket", Consumer: "workstation", From: "pics", Slug: "pics",
Values: map[string]any{}, Sealed: "c2VhbGVk"},
}})
if err != nil {
t.Fatal(err)
}
forFiles := strings.ReplaceAll(ConsumerIdentity("workstation", IdentitySource("files", "files")), "_", "-")
forPics := strings.ReplaceAll(ConsumerIdentity("workstation", IdentitySource("pics", "pics")), "_", "-")
if forFiles == forPics {
t.Fatal("the two consumers were given the same identity; this test proves nothing")
}
if got := fileNamed(out, "files.env")["content"].(string); !strings.Contains(got, "BUCKET="+forFiles+"\n") {
t.Errorf("files was not given its own bucket:\n%s", got)
}
if got := fileNamed(out, "pics.env")["content"].(string); !strings.Contains(got, "BUCKET="+forPics+"\n") {
t.Errorf("pics was not given its own bucket:\n%s", got)
}
var buckets []any
for _, g := range storeGrants(t, out) {
buckets = append(buckets, g.Derived["bucket"])
}
if len(buckets) != 2 || buckets[0] == buckets[1] {
t.Errorf("the provider was told %v; it must be told one bucket per consumer", buckets)
}
}
// An operator may still set what the provider serves, and the mesh still derives the rest: the
// setting is laid on first, then the consumer's half is filled.
func TestASettingComposesWithADerivedValue(t *testing.T) {
r, err := Resolve(shelf(store(), files()), []string{"store", "files"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
out, err := r.Declaration(Rendering{
Settings: SettingsBy{"store": {{From: "the operator",
Values: map[string]any{"bucket": "team-${consumer:as:dns}"}}}},
Grants: []Grant{{Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Values: map[string]any{}, Sealed: "c2VhbGVk"}},
})
if err != nil {
t.Fatal(err)
}
want := "team-" + strings.ReplaceAll(ConsumerIdentity("workstation", IdentitySource("files", "files")), "_", "-")
if got := fileNamed(out, "files.env")["content"].(string); !strings.Contains(got, "BUCKET="+want+"\n") {
t.Errorf("the operator's prefix did not survive the derivation:\n%s\nwant BUCKET=%s", got, want)
}
if given := storeGrants(t, out); given[0].Derived["bucket"] != want {
t.Errorf("the provider was told %v, the consumer %q", given[0].Derived["bucket"], want)
}
}
// A fact or an alphabet the mesh does not have is refused where the definition is, not where a
// consumer happens to be resolved — and the refusal says what may be said instead.
func TestAServedValueNamingSomethingTheMeshDoesNotHaveIsRefused(t *testing.T) {
for _, c := range []struct{ value, says string }{
{"${consumer:node}", "as"},
{"${consumer:as:punycode}", "dns"},
} {
m := store()
m.Serves["s3-bucket"]["bucket"] = c.value
raw, err := json.Marshal(m)
if err != nil {
t.Fatal(err)
}
_, err = ParseManifest(raw)
if err == nil {
t.Fatalf("%s was accepted", c.value)
}
if !strings.Contains(err.Error(), c.value) {
t.Errorf("the refusal of %s does not quote it: %v", c.value, err)
}
if !strings.Contains(err.Error(), c.says) {
t.Errorf("the refusal of %s does not say what may be said (%q): %v", c.value, c.says, err)
}
}
}
// `dns` is checked against an identity the mesh actually mints, not an invented string.
func TestTheDNSAlphabetIsTheMintedIdentityWithItsSeparatorRewritten(t *testing.T) {
as := ConsumerIdentity("anchor", IdentitySource("ncloud", "nextcloud"))
if err := CheckIdentity("anchor", IdentitySource("ncloud", "nextcloud")); err != nil {
t.Fatalf("the mesh would not mint this identity at all: %v", err)
}
label := asDNSLabel(as)
if strings.Contains(label, "_") {
t.Errorf("%q is not a DNS label", label)
}
if strings.ReplaceAll(label, "-", "_") != as {
t.Errorf("%q is not %q with its separator rewritten", label, as)
}
}
// The check that would have caught the one wrong instance: a consumer that writes the derived
// value into its own definition instead of asking for it is refused, whether it transcribed the
// right answer or a predecessor's.
func TestAConsumerThatTranscribesWhatItsProviderDerivesIsRefused(t *testing.T) {
as := ConsumerIdentity("workstation", IdentitySource("files", "files"))
transcribed := strings.ReplaceAll(as, "_", "-")
m := files()
m.Resources = []map[string]any{{
"id": "env", "type": "file", "path": "/var/lib/files/env", "mode": "0600",
// Exactly what the provider will create — correct today, and a copy of a rule that is
// not this module's.
"content": "BUCKET=" + transcribed + "\n",
}}
r, err := Resolve(shelf(store(), m), []string{"store", "files"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
_, err = r.Declaration(Rendering{Grants: []Grant{{
Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Values: map[string]any{}, Sealed: "c2VhbGVk",
}}})
if err == nil {
t.Fatal("a definition holding its own copy of the provider's naming rule was accepted")
}
if !strings.Contains(err.Error(), "${bound:s3-bucket:bucket}") {
t.Errorf("the refusal does not say what to write instead: %v", err)
}
// And a constant the provider serves to everyone is not a transcription: repeating it is
// redundant, not wrong, and refusing it would be the mesh policing style.
m.Resources = []map[string]any{{
"id": "env", "type": "file", "path": "/var/lib/files/env", "mode": "0600",
"content": "REGION=eu-west\n",
}}
r, err = Resolve(shelf(store(), m), []string{"store", "files"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
if _, err := r.Declaration(Rendering{Grants: []Grant{{
Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Values: map[string]any{}, Sealed: "c2VhbGVk",
}}}); err != nil {
t.Errorf("a value the provider serves to everyone was judged a transcription: %v", err)
}
}
func storeGrants(t *testing.T, out []map[string]any) []Contribution {
t.Helper()
for _, r := range out {
if r["path"] != "/var/lib/store/grants/mesh.json" {
continue
}
var parsed struct {
Given []Contribution `json:"given"`
}
if err := json.Unmarshal([]byte(r["content"].(string)), &parsed); err != nil {
t.Fatal(err)
}
return parsed.Given
}
t.Fatalf("the provider was given no contributions file: %v", out)
return nil
}
// A consumer that keeps SEVERAL holders of one provision is refused, rather than told one thing
// while its provider is told another.
//
// **This is issue 124's own failure, one case to the side.** The mesh gives each holder its own
// login — `mesh_node_mod_<local>` (ADR 0094) — and the provider derives from the login, so it
// would make one resource per holder. The consumer's side has no such dimension: there is one
// binding file per provision and one `${bound:<provision>:<key>}`, both derived from the
// un-suffixed identity. So the provider would create `…-mod-cold` and the consumer would be
// configured against `…-mod`: it would authenticate successfully and be refused on every object,
// which is exactly the fault this whole record exists to end.
//
// Refused, loudly, at the one place that can see both halves. Lifting it means giving the
// consumer's side a local dimension, which is a decision and not an omission.
func TestAConsumerWithSeveralHoldersOfADerivingProviderIsRefused(t *testing.T) {
m := files()
// Two holders of the one provision, the shape ADR 0094 gives a module that keeps several.
m.Secrets = nil
m.SecretsMany = map[string]map[string]string{"s3-bucket": {
"hot": "/var/lib/files/hot.secret",
"cold": "/var/lib/files/cold.secret",
}}
m.Resources = []map[string]any{{
"id": "env", "type": "file", "path": "/var/lib/files/env", "mode": "0600",
"content": "BUCKET=${bound:s3-bucket:bucket}\n",
}}
r, err := Resolve(shelf(store(), m), []string{"store", "files"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
_, err = r.Declaration(Rendering{Grants: []Grant{
{Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Local: "hot", Values: map[string]any{}, Sealed: "c2VhbGVk"},
{Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Local: "cold", Values: map[string]any{}, Sealed: "c2VhbGVk"},
}})
if err == nil {
t.Fatal("a consumer with several holders of a deriving provider was accepted; " +
"its two ends would have disagreed in silence")
}
for _, want := range []string{"files", "s3-bucket", "bucket"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not name %q: %v", want, err)
}
}
}
internal/catalogue/environment_into.go
-593
View File
@@ -1,593 +0,0 @@
package catalogue
import (
"fmt"
"regexp"
"sort"
"strings"
)
// The account's environment and the login shell's code, composed from the modules a node runs
// (novox/hq ADR 0203, ADR 0204).
//
// **The same shape as the jails.** Every module may contribute — a toolchain its directory on PATH,
// a version manager a variable naming its home, a prompt the code that loads it — naming no node, no
// path and no file of the shell's (ADR 0112). The one module holding the matching seat places the
// result with a placeholder in its own file, and the controller fills it from every module on the
// node. A node not running a module has none of its contribution, and unassigning one takes its
// lines away at the next composition.
//
// **Two kinds of contribution, kept apart on purpose.** The environment is facts, which the
// controller writes in two standard formats — POSIX assignment and the service manager's
// environment.d — so a terminal, a script, the login shell's `execute` and a graphical session all
// read the same values (ADR 0203). Shell code is not a fact: it is text in one shell's syntax, which
// the controller sorts into a slot and pastes without reading, as it pastes a jail's stanza (ADR
// 0204).
// EnvironmentSeat and LoginShellSeat are the seats whose holders may place what the modules
// contributed: the account's environment, and the login shell's code.
const (
EnvironmentSeat = "node-environment"
LoginShellSeat = "node-login-shell"
// PowerSeat is the seat whose holder places code for the power moments (novox/hq ADR 0211).
PowerSeat = "node-power"
)
// Where an environment entry on PATH goes: before the account's existing PATH, or after it.
const (
PathAtStart = "start"
PathAtEnd = "end"
)
// Environment is what one module adds to the account's environment (novox/hq ADR 0203).
type Environment struct {
// Variables are names and literal values. A value may name the machine's own facts with
// ${machine:…}, resolved before anything is written, and nothing else that expands.
Variables map[string]string `json:"variables,omitempty"`
// Path is entries on the account's PATH, each at its start or its end, in the order declared.
Path []PathEntry `json:"path,omitempty"`
}
// PathEntry is one directory a module puts on the account's PATH.
type PathEntry struct {
Entry string `json:"entry"`
At string `json:"at"`
}
// ShellCode is one piece of code a module adds to a shell's startup (novox/hq ADR 0204).
type ShellCode struct {
// For is the shell the code is written in.
For string `json:"for"`
// Slot is where it runs among the other modules' code: first, normal or last. Named rather
// than numbered, because every contributor would guess a number and a collision says nothing.
Slot string `json:"slot"`
// Code is never interpreted — it is the shell's syntax, and only the shell reads it.
Code string `json:"code"`
}
// The shells and slots a contribution may name (novox/hq ADR 0204). Closed, so a typo is a refusal
// at the check rather than code that silently lands in no placeholder.
var (
knownShells = []string{"zsh", "bash", "fish"}
knownSlots = []string{"first", "normal", "last"}
// sessionFiles are the two files of the graphical session's start that read no directory, so a
// contribution to them is a slot rather than a drop-in (novox/hq ADR 0208 §4): `xinitrc` is POSIX
// code the session's start runs, `xresources` X resources merged at its start. Placed by the
// display server's holder, as a shell's slots are placed by the login shell's.
sessionFiles = []string{"xinitrc", "xresources"}
// powerMoments are the moments of a machine's power a module may run code at (novox/hq ADR
// 0211 §3): POSIX code run as root by node-power's holder, in module order, each piece bounded.
powerMoments = []string{"after-boot", "before-sleep", "after-wake", "before-shutdown", "on-mains", "on-battery"}
)
// contributionTargets is every name a contribution's `for` may take.
func contributionTargets() []string {
out := append(append([]string(nil), knownShells...), sessionFiles...)
return append(out, powerMoments...)
}
// placerOf is the seat whose holder places a contribution for this target (novox/hq ADR 0204,
// ADR 0208 §4).
func placerOf(target string) string {
if oneOf(sessionFiles, target) {
return DisplayServerSeat
}
if oneOf(powerMoments, target) {
return PowerSeat
}
return LoginShellSeat
}
// The two renderings of the environment a holder may place (novox/hq ADR 0203, decision 3).
const (
EnvironmentPOSIX = "posix"
EnvironmentSystemd = "systemd"
)
// ofEnvironment and ofShell are where a holder places what was contributed: ${environment:posix},
// ${environment:systemd} and ${shell:<shell>:<slot>}. Loose inside the braces on purpose, so a
// misspelt key is found and refused rather than left in a file as a literal nobody reads.
var (
ofEnvironment = regexp.MustCompile(`\$\{environment:([^}]*)\}`)
ofShell = regexp.MustCompile(`\$\{shell:([^}]*)\}`)
// ofContributed is either kind of contributed text, matched together so both fill in one pass.
ofContributed = regexp.MustCompile(`\$\{(shell|contribution):([^}]*)\}`)
)
// variableName is a POSIX shell variable name, which is also what environment.d accepts.
var variableName = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
// environmentProblems is what is wrong with this module's environment contribution, from the
// manifest alone.
func (m Manifest) environmentProblems() []string {
if m.Environment == nil {
return nil
}
var problems []string
for _, n := range sortedKeys(m.Environment.Variables) {
switch {
case !variableName.MatchString(n):
problems = append(problems, fmt.Sprintf(
"%s sets the variable %q, which is not a name a shell accepts: a letter or an "+
"underscore, then letters, digits and underscores", m.Module, n))
continue
case n == "PATH":
// PATH is the one variable every module shares, so no module may set it whole: a second
// setter would replace the first's entries, and the account's own PATH with them.
problems = append(problems, fmt.Sprintf(
"%s sets PATH as a variable; a module adds an entry under environment.path, at the "+
"start or the end, and PATH is composed from every module's (novox/hq ADR 0203)", m.Module))
continue
}
if why := literalProblem(m.Environment.Variables[n]); why != "" {
problems = append(problems, fmt.Sprintf(
"%s sets %s to %q, which %s — %s", m.Module, n, m.Environment.Variables[n], why, literalRule))
}
}
seen := map[string]bool{}
for i, p := range m.Environment.Path {
switch {
case p.Entry == "":
problems = append(problems, fmt.Sprintf("%s's PATH entry %d names no directory", m.Module, i+1))
case strings.Contains(ofMachine.ReplaceAllString(p.Entry, ""), ":"):
// A colon is PATH's own separator, so an entry holding one is two entries, and the
// check that it is already present would look for the wrong thing.
problems = append(problems, fmt.Sprintf(
"%s puts %q on PATH, which holds a colon, PATH's own separator", m.Module, p.Entry))
case seen[p.Entry]:
problems = append(problems, fmt.Sprintf("%s puts %q on PATH twice", m.Module, p.Entry))
default:
if why := literalProblem(p.Entry); why != "" {
problems = append(problems, fmt.Sprintf(
"%s puts %q on PATH, which %s — %s", m.Module, p.Entry, why, literalRule))
}
}
seen[p.Entry] = true
if p.At != PathAtStart && p.At != PathAtEnd {
problems = append(problems, fmt.Sprintf(
"%s puts %q on PATH at %q; an entry goes at %q or %q of the account's PATH",
m.Module, p.Entry, p.At, PathAtStart, PathAtEnd))
}
}
return problems
}
// literalRule is why a value must be literal, said with every refusal of one.
const literalRule = "a value is literal, so a POSIX shell and the service manager read it alike, and " +
"names the machine only through the mesh's own ${machine:…} facts (novox/hq ADR 0203)"
// literalProblem is why a value cannot be written, unquoted by either reader, as the same string in
// both formats — or nothing. A `$` would expand differently in each; a quote or a backslash is
// quoting in one and a character in the other; a line break ends the line in both.
func literalProblem(v string) string {
switch {
case strings.ContainsAny(v, `'"`):
return "holds a quote"
case strings.Contains(v, `\`):
return "holds a backslash"
case strings.ContainsAny(v, "\n\r"):
return "holds a line break"
case strings.ContainsRune(v, 0):
return "holds a NUL"
case strings.Contains(ofMachine.ReplaceAllString(v, ""), "$"):
return "holds a $ that is not one of the machine's ${machine:…} facts"
}
return ""
}
// shellProblems is what is wrong with this module's shell code, from the manifest alone. The code
// itself is not judged: it is the shell's syntax, which the controller does not read.
func (m Manifest) shellProblems() []string {
var problems []string
for i, c := range m.Shell {
if !oneOf(contributionTargets(), c.For) {
problems = append(problems, fmt.Sprintf(
"%s's shell code %d is for %q; the shells are %s, the session's files %s, and the power "+
"moments %s", m.Module, i+1, c.For, strings.Join(knownShells, ", "),
strings.Join(sessionFiles, ", "), strings.Join(powerMoments, ", ")))
}
if !oneOf(knownSlots, c.Slot) {
problems = append(problems, fmt.Sprintf(
"%s's shell code %d goes in the slot %q; the slots are %s", m.Module, i+1, c.Slot,
strings.Join(knownSlots, ", ")))
}
if strings.TrimSpace(c.Code) == "" {
problems = append(problems, fmt.Sprintf("%s's shell code %d has no code", m.Module, i+1))
}
}
return problems
}
// contributionPlaceholderProblems is every place this module's resources name the environment or
// the shell's code and may not — judged from the manifest, so the catalogue check refuses it before
// a mesh does, and again at composition in the same words.
func (m Manifest) contributionPlaceholderProblems() []string {
var problems []string
for _, r := range m.Resources {
problems = append(problems, placeholderProblems(m, r)...)
problems = append(problems, seatPlaceholderProblems(m, r)...)
}
return problems
}
// placeholderProblems is what is wrong with one resource's ${environment:…} and ${shell:…}.
//
// **The seat authorises it, not the placeholder** (novox/hq ADR 0203 §5, ADR 0204 §3), as the seat
// authorises the bus's user list: a module that does not hold the account's environment writing it
// would be a second writer of a file there is one of, and a module that does not hold the login
// shell writing every module's shell code would be a second shell.
func placeholderProblems(m Manifest, r map[string]any) []string {
var problems []string
for _, field := range sortedKeys(r) {
s, ok := r[field].(string)
if !ok {
continue
}
env := ofEnvironment.FindAllStringSubmatch(s, -1)
code := ofShell.FindAllStringSubmatch(s, -1)
if len(env)+len(code) == 0 {
continue
}
if field != "content" {
// Placed only where a file's bytes are, which is where every one of them is meant to go:
// a path or an owner holding several lines of shell is nothing the host could act on.
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s in its %s; the environment and the shell's code are placed "+
"only in a file's content", m.Module, r["id"], placeholderOf(env, code), field))
continue
}
for _, e := range env {
if e[1] != EnvironmentPOSIX && e[1] != EnvironmentSystemd {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s; the environment is ${environment:%s} or ${environment:%s}",
m.Module, r["id"], e[0], EnvironmentPOSIX, EnvironmentSystemd))
}
}
if len(env) > 0 && !m.ClaimsSeat(EnvironmentSeat) {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s and %s does not claim %s; the account's environment is "+
"written by that seat's holder alone (novox/hq ADR 0203)",
m.Module, r["id"], env[0][0], m.Module, EnvironmentSeat))
}
// Each placeholder judged by its own target: a shell's code is the login shell's holder's to
// place (ADR 0204), the session's files the display server's (ADR 0208 §4) — and a holder of
// one placing the other's would be a second writer of a file there is one of.
refusedFor := map[string]bool{}
for _, c := range code {
target, slot, two := strings.Cut(c[1], ":")
if !two || !oneOf(contributionTargets(), target) || !oneOf(knownSlots, slot) {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s; shell code is ${shell:<shell>:<slot>}, the shell one of "+
"%s, the session's file one of %s or the power moment one of %s, and the slot one of %s",
m.Module, r["id"], c[0], strings.Join(knownShells, ", "), strings.Join(sessionFiles, ", "),
strings.Join(powerMoments, ", "), strings.Join(knownSlots, ", ")))
continue
}
seat := placerOf(target)
if m.ClaimsSeat(seat) || refusedFor[seat] {
continue
}
refusedFor[seat] = true
if seat == PowerSeat {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s and %s does not claim %s; every module's code for a power "+
"moment is placed by the power seat's holder alone (novox/hq ADR 0211)",
m.Module, r["id"], c[0], m.Module, seat))
continue
}
if seat == DisplayServerSeat {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s and %s does not claim %s; every module's %s is placed by "+
"the display server's holder alone (novox/hq ADR 0208)",
m.Module, r["id"], c[0], m.Module, seat, target))
continue
}
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s and %s does not claim %s; every module's shell code is "+
"placed by the login shell's holder alone (novox/hq ADR 0204)",
m.Module, r["id"], c[0], m.Module, seat))
}
}
return problems
}
func placeholderOf(env, code [][]string) string {
if len(env) > 0 {
return env[0][0]
}
return code[0][0]
}
// contributedEnvironment is one node's environment, gathered and in the order it is written.
type contributedEnvironment struct {
// variables is by module in name order, each module's sorted by name.
variables []setBy
// start and end are PATH's entries in their final order, each once.
start, end []placedOn
}
type setBy struct {
module string
names []string
values map[string]string
}
type placedOn struct {
module, entry string
}
// inModuleOrder is the modules sorted by name — the order contributions are written in (novox/hq
// ADR 0203, ADR 0204), so the same set composes byte for byte whatever order they were assigned in.
func inModuleOrder(modules []Manifest) []Manifest {
out := append([]Manifest(nil), modules...)
sort.SliceStable(out, func(a, b int) bool { return out[a].Module < out[b].Module })
return out
}
// variablesSetOnce refuses a variable two modules on one node both set (novox/hq ADR 0203 §5),
// naming both. Neither is chosen: whichever was written last would win in one reader and not
// necessarily in the other, and the module that lost would not be told.
func variablesSetOnce(modules []Manifest) error {
setter := map[string]string{}
for _, m := range inModuleOrder(modules) {
if m.Environment == nil {
continue
}
for _, n := range sortedKeys(m.Environment.Variables) {
if first, taken := setter[n]; taken {
return fmt.Errorf(
"%s and %s both set %s on this machine; the account has one environment, so one "+
"of them must stop setting it (novox/hq ADR 0203)", first, m.Module, n)
}
setter[n] = m.Module
}
}
return nil
}
// environmentOn gathers every module's environment on a node, with the machine's facts in place.
//
// A PATH entry two modules both add is written once, where the first puts it: two toolchains
// sharing ~/.local/bin is ordinary, and nothing about it is in conflict.
func environmentOn(modules []Manifest, facts map[string]string) (contributedEnvironment, error) {
var env contributedEnvironment
if err := variablesSetOnce(modules); err != nil {
return env, err
}
placed := map[string]bool{}
for _, m := range inModuleOrder(modules) {
if m.Environment == nil {
continue
}
if len(m.Environment.Variables) > 0 {
set := setBy{module: m.Module, values: map[string]string{}}
for _, n := range sortedKeys(m.Environment.Variables) {
v, err := factsIn(m.Environment.Variables[n], facts, m.Module, n)
if err != nil {
return env, err
}
set.names = append(set.names, n)
set.values[n] = v
}
env.variables = append(env.variables, set)
}
for _, p := range m.Environment.Path {
entry, err := factsIn(p.Entry, facts, m.Module, "a PATH entry")
if err != nil {
return env, err
}
if strings.Contains(entry, ":") {
return env, fmt.Errorf("%s puts %q on PATH on this machine, which holds a colon, PATH's own separator",
m.Module, entry)
}
if placed[entry] {
continue
}
placed[entry] = true
if p.At == PathAtEnd {
env.end = append(env.end, placedOn{m.Module, entry})
} else {
env.start = append(env.start, placedOn{m.Module, entry})
}
}
}
return env, nil
}
// factsIn resolves a contributed value's ${machine:…} facts with this machine's — first, before
// either format is written, so both say the same thing (novox/hq ADR 0203).
func factsIn(v string, facts map[string]string, module, what string) (string, error) {
for _, key := range machineUsed(v) {
value, has := facts[key]
if !has {
return "", fmt.Errorf("%s sets %s to a value that says ${machine:%s}, and this machine says %s",
module, what, key, orNothing(namesOfFacts(facts)))
}
v = strings.ReplaceAll(v, fmt.Sprintf("${machine:%s}", key), value)
}
// Judged again once filled: a fact is the mesh's, and still has to be a literal both readers
// take alike.
if why := literalProblem(v); why != "" {
return "", fmt.Errorf("%s sets %s to %q on this machine, which %s — %s", module, what, v, why, literalRule)
}
return v, nil
}
// posix is the environment as lines a POSIX shell sources (novox/hq ADR 0203 §3): every variable
// exported, every PATH entry added only when it is missing, so sourcing the file twice — a login
// shell that starts another — changes nothing. POSIX sh only, because sh, bash and zsh all read it.
//
// The start entries are written last-first: each is put in front of PATH, so the last written ends
// up first, and the result reads in module order, then the order each module declared.
func (e contributedEnvironment) posix() string {
var b strings.Builder
for _, set := range e.variables {
fmt.Fprintf(&b, "# %s\n", set.module)
for _, n := range set.names {
fmt.Fprintf(&b, "export %s='%s'\n", n, set.values[n])
}
}
named := ""
for i := len(e.start) - 1; i >= 0; i-- {
p := e.start[i]
if p.module != named {
fmt.Fprintf(&b, "# %s\n", p.module)
named = p.module
}
fmt.Fprintf(&b, "case \":${PATH}:\" in *':%s:'*) ;; *) PATH='%s'\"${PATH:+:${PATH}}\" ;; esac\n",
p.entry, p.entry)
}
named = ""
for _, p := range e.end {
if p.module != named {
fmt.Fprintf(&b, "# %s\n", p.module)
named = p.module
}
fmt.Fprintf(&b, "case \":${PATH}:\" in *':%s:'*) ;; *) PATH=\"${PATH:+${PATH}:}\"'%s' ;; esac\n",
p.entry, p.entry)
}
if len(e.start)+len(e.end) > 0 {
b.WriteString("export PATH\n")
}
return b.String()
}
// systemd is the same environment as the service manager's environment.d reads it (novox/hq ADR
// 0203 §3), for the account's user manager and so for everything a graphical session starts. Read
// once per manager start, so it needs no guard against running twice; the account's existing PATH
// sits between the start and the end entries.
func (e contributedEnvironment) systemd() string {
var b strings.Builder
for _, set := range e.variables {
fmt.Fprintf(&b, "# %s\n", set.module)
for _, n := range set.names {
fmt.Fprintf(&b, "%s=%s\n", n, set.values[n])
}
}
if len(e.start) > 0 {
fmt.Fprintf(&b, "# %s\nPATH=%s${PATH:+:$PATH}\n", modulesOf(e.start), entriesOf(e.start))
}
if len(e.end) > 0 {
fmt.Fprintf(&b, "# %s\nPATH=${PATH:+$PATH:}%s\n", modulesOf(e.end), entriesOf(e.end))
}
return b.String()
}
// modulesOf names who contributed a line holding several modules' entries, in the order they appear.
func modulesOf(entries []placedOn) string {
var names []string
seen := map[string]bool{}
for _, p := range entries {
if !seen[p.module] {
seen[p.module] = true
names = append(names, p.module)
}
}
return strings.Join(names, ", ")
}
func entriesOf(entries []placedOn) string {
out := make([]string, len(entries))
for i, p := range entries {
out[i] = p.entry
}
return strings.Join(out, ":")
}
// shellCode is every module's code for one shell and one slot (novox/hq ADR 0204 §3): in module
// order, each module's pieces in the order it declared them, each preceded by a line naming the
// module, and empty when nothing is contributed.
func shellCode(modules []Manifest, shell, slot string) string {
var b strings.Builder
for _, m := range inModuleOrder(modules) {
named := false
for _, c := range m.Shell {
if c.For != shell || c.Slot != slot {
continue
}
if !named {
fmt.Fprintf(&b, "# %s\n", m.Module)
named = true
}
b.WriteString(c.Code)
if !strings.HasSuffix(c.Code, "\n") {
b.WriteString("\n")
}
}
}
return b.String()
}
// contributionsInto fills a holder's file with the node's environment and its shell code.
//
// **Last, after every other placeholder pass, and in one pass each.** Shell code is contributed text
// in a shell's own syntax — `${XDG_CACHE_HOME:-$HOME/.cache}`, `${(%):-%n}` — and the rendered
// environment holds `${PATH:+…}`: a scanner for the mesh's own placeholders that ran after these
// were in place would read the shell's expansions as the mesh's and refuse them, or fill a
// `${machine:…}` some module wrote for its shell to see. So nothing runs after them, the environment
// is filled before the shell's code is, and each is replaced in a single pass over what the holder
// wrote, so a contributed piece is never scanned again.
func contributionsInto(resource map[string]any, m Manifest, modules []Manifest, facts map[string]string, with Rendering) error {
if problems := append(placeholderProblems(m, resource), seatPlaceholderProblems(m, resource)...); len(problems) > 0 {
return fmt.Errorf("%s", problems[0])
}
content, ok := resource["content"].(string)
if !ok {
return nil
}
if ofEnvironment.MatchString(content) {
env, err := environmentOn(modules, facts)
if err != nil {
return err
}
content = ofEnvironment.ReplaceAllStringFunc(content, func(placeholder string) string {
if ofEnvironment.FindStringSubmatch(placeholder)[1] == EnvironmentSystemd {
return env.systemd()
}
return env.posix()
})
}
// Shell code and seat contributions in one pass (novox/hq ADR 0212): both are contributed text
// the controller does not read, so neither may be scanned after the other is in place — a
// contributed line that happened to spell the other's placeholder would be filled.
if ofContributed.MatchString(content) {
var failed error
content = ofContributed.ReplaceAllStringFunc(content, func(placeholder string) string {
found := ofContributed.FindStringSubmatch(placeholder)
first, second, _ := strings.Cut(found[2], ":")
if found[1] == "contribution" {
placed, err := seatContributions(modules, first, second, with)
if err != nil && failed == nil {
failed = err
}
return placed
}
return shellCode(modules, first, second)
})
if failed != nil {
return failed
}
}
resource["content"] = content
return nil
}
internal/catalogue/environment_into_test.go
-471
View File
@@ -1,471 +0,0 @@
package catalogue
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// Defends novox/hq ADR 0203 (the account's environment is one module's, and every module
// contributes to it) and ADR 0204 (shell code in named slots, placed by the login shell's holder).
// contributors is a fixed set of contributions, in no particular order: what the renderings are
// asserted against byte for byte. go-toolchain and zsh both put ~/.local/bin on PATH, which is the
// ordinary case of two modules sharing a directory, and is written once.
func contributors() []Manifest {
return []Manifest{
{Module: "zsh", Environment: &Environment{
Variables: map[string]string{"XDG_CONFIG_HOME": "${machine:account-home}/.config", "EDITOR": "vim"},
Path: []PathEntry{
{Entry: "${machine:account-home}/.local/bin", At: PathAtStart},
{Entry: "${machine:account-home}/bin", At: PathAtStart},
{Entry: "/opt/scripts", At: PathAtEnd},
},
}},
{Module: "go-toolchain", Environment: &Environment{
Variables: map[string]string{"GOPATH": "${machine:account-home}/go"},
Path: []PathEntry{
{Entry: "${machine:account-home}/go/bin", At: PathAtStart},
{Entry: "/usr/local/go/bin", At: PathAtStart},
{Entry: "${machine:account-home}/.local/bin", At: PathAtStart},
},
}},
{Module: "agent", Environment: &Environment{
Variables: map[string]string{"DISABLE_AUTOUPDATER": "1"},
Path: []PathEntry{{Entry: "/opt/agent/bin", At: PathAtEnd}},
}},
// A module contributing nothing is in the set and writes nothing.
{Module: "postgres"},
}
}
var operatorFacts = map[string]string{"name": "workstation", "account": "op", "account-home": "/home/op"}
// The final PATH this set composes, around whatever the account had: the start entries in module
// order and then declared order, the account's own, then the end entries.
const composedPOSIX = `# agent
export DISABLE_AUTOUPDATER='1'
# go-toolchain
export GOPATH='/home/op/go'
# zsh
export EDITOR='vim'
export XDG_CONFIG_HOME='/home/op/.config'
# zsh
case ":${PATH}:" in *':/home/op/bin:'*) ;; *) PATH='/home/op/bin'"${PATH:+:${PATH}}" ;; esac
# go-toolchain
case ":${PATH}:" in *':/home/op/.local/bin:'*) ;; *) PATH='/home/op/.local/bin'"${PATH:+:${PATH}}" ;; esac
case ":${PATH}:" in *':/usr/local/go/bin:'*) ;; *) PATH='/usr/local/go/bin'"${PATH:+:${PATH}}" ;; esac
case ":${PATH}:" in *':/home/op/go/bin:'*) ;; *) PATH='/home/op/go/bin'"${PATH:+:${PATH}}" ;; esac
# agent
case ":${PATH}:" in *':/opt/agent/bin:'*) ;; *) PATH="${PATH:+${PATH}:}"'/opt/agent/bin' ;; esac
# zsh
case ":${PATH}:" in *':/opt/scripts:'*) ;; *) PATH="${PATH:+${PATH}:}"'/opt/scripts' ;; esac
export PATH
`
const composedSystemd = `# agent
DISABLE_AUTOUPDATER=1
# go-toolchain
GOPATH=/home/op/go
# zsh
EDITOR=vim
XDG_CONFIG_HOME=/home/op/.config
# go-toolchain, zsh
PATH=/home/op/go/bin:/usr/local/go/bin:/home/op/.local/bin:/home/op/bin${PATH:+:$PATH}
# agent, zsh
PATH=${PATH:+$PATH:}/opt/agent/bin:/opt/scripts
`
func TestTheEnvironmentRendersForAPOSIXShellByteForByte(t *testing.T) {
env, err := environmentOn(contributors(), operatorFacts)
if err != nil {
t.Fatal(err)
}
if got := env.posix(); got != composedPOSIX {
t.Fatalf("the POSIX rendering is\n%s\nnot\n%s", got, composedPOSIX)
}
}
func TestTheEnvironmentRendersForTheServiceManagerByteForByte(t *testing.T) {
env, err := environmentOn(contributors(), operatorFacts)
if err != nil {
t.Fatal(err)
}
if got := env.systemd(); got != composedSystemd {
t.Fatalf("the environment.d rendering is\n%s\nnot\n%s", got, composedSystemd)
}
}
// Sourcing twice changes nothing (ADR 0203 §3): a login shell that starts another reads the file
// again, and a PATH that grew each time would be the symptom. Run by a real `sh`, because the claim
// is about what a shell does with the file, not about what the file looks like.
func TestThePOSIXEnvironmentSourcedTwiceLeavesPATHAsOnce(t *testing.T) {
sh, err := exec.LookPath("sh")
if err != nil {
t.Skip("no sh on this machine")
}
script := "PATH=/usr/bin:/bin\n" + composedPOSIX + "once=$PATH\n" + composedPOSIX +
`[ "$PATH" = "$once" ] || { echo "changed: $once -> $PATH"; exit 1; }` + "\n" +
`echo "$PATH"; echo "$GOPATH"`
out, err := exec.Command(sh, "-c", script).CombinedOutput()
if err != nil {
t.Fatalf("sourcing twice: %v\n%s", err, out)
}
lines := strings.Split(strings.TrimSpace(string(out)), "\n")
want := "/home/op/go/bin:/usr/local/go/bin:/home/op/.local/bin:/home/op/bin:/usr/bin:/bin:/opt/agent/bin:/opt/scripts"
if lines[0] != want {
t.Fatalf("PATH is %s, not %s", lines[0], want)
}
if lines[1] != "/home/op/go" {
t.Fatalf("GOPATH was not exported: %q", lines[1])
}
// And an entry the account already has stays where it is, and once.
out, err = exec.Command(sh, "-c", "PATH=/opt/scripts:/usr/bin\n"+composedPOSIX+`echo "$PATH"`).CombinedOutput()
if err != nil {
t.Fatalf("%v\n%s", err, out)
}
if got := strings.TrimSpace(string(out)); got !=
"/home/op/go/bin:/usr/local/go/bin:/home/op/.local/bin:/home/op/bin:/opt/scripts:/usr/bin:/opt/agent/bin" {
t.Fatalf("an entry already on PATH was added again or moved: %s", got)
}
}
// The environment.d rendering, read by the service manager's own generator where this machine has
// one — the same reader an account's user manager runs, so the PATH it composes is the one asserted.
func TestTheServiceManagerReadsTheSystemdRenderingAsMeant(t *testing.T) {
generator := "/usr/lib/systemd/user-environment-generators/30-systemd-environment-d-generator"
if _, err := os.Stat(generator); err != nil {
t.Skip("no environment.d generator on this machine")
}
config := t.TempDir()
if err := os.MkdirAll(filepath.Join(config, "environment.d"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(config, "environment.d", "50-mesh.conf"), []byte(composedSystemd), 0o644); err != nil {
t.Fatal(err)
}
cmd := exec.Command(generator)
cmd.Env = []string{"PATH=/usr/bin:/bin", "HOME=" + config, "XDG_CONFIG_HOME=" + config}
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("%v\n%s", err, out)
}
want := "PATH=/home/op/go/bin:/usr/local/go/bin:/home/op/.local/bin:/home/op/bin:/usr/bin:/bin:/opt/agent/bin:/opt/scripts"
if !strings.Contains(string(out), want+"\n") || !strings.Contains(string(out), "GOPATH=/home/op/go\n") {
t.Fatalf("the service manager read\n%s", out)
}
}
// Nothing contributed renders nothing, in both formats — not an empty `export PATH`.
func TestNoContributionsRenderNothing(t *testing.T) {
env, err := environmentOn([]Manifest{{Module: "postgres"}}, operatorFacts)
if err != nil {
t.Fatal(err)
}
if env.posix() != "" || env.systemd() != "" {
t.Fatalf("an empty environment rendered %q and %q", env.posix(), env.systemd())
}
}
// A ${machine:…} fact the machine does not have is refused naming the module, as a file's is.
func TestAContributedFactTheMachineLacksIsRefused(t *testing.T) {
_, err := environmentOn(contributors(), map[string]string{"name": "server"})
if err == nil || !strings.Contains(err.Error(), "go-toolchain sets GOPATH") ||
!strings.Contains(err.Error(), "${machine:account-home}") {
t.Fatalf("a missing account home was not refused by name: %v", err)
}
}
// ADR 0203 §5: two modules setting one variable are refused, both named — neither silently wins.
func TestAVariableTwoModulesSetIsRefusedNamingBoth(t *testing.T) {
modules := append(contributors(), Manifest{Module: "neovim", Environment: &Environment{
Variables: map[string]string{"EDITOR": "nvim"}}})
_, err := environmentOn(modules, operatorFacts)
if err == nil || err.Error() != "neovim and zsh both set EDITOR on this machine; the account has one "+
"environment, so one of them must stop setting it (novox/hq ADR 0203)" {
t.Fatalf("a variable set twice was not refused naming both: %v", err)
}
// And at composition, whether or not the node holds the environment.
r := Resolution{Node: "workstation", Account: "op", Modules: modules}
if _, err := r.Declaration(Rendering{}); err == nil || !strings.Contains(err.Error(), "neovim and zsh both set EDITOR") {
t.Fatalf("composition accepted a variable set twice: %v", err)
}
}
// shells contributes code for several shells and slots, in no order.
func shells() []Manifest {
return []Manifest{
{Module: "zsh-syntax-highlighting", Shell: []ShellCode{
{For: "zsh", Slot: "last", Code: "source /usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh"},
}},
{Module: "powerlevel10k", Shell: []ShellCode{
{For: "zsh", Slot: "first", Code: "if [[ -r \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\" ]]; then\n" +
" source \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\"\nfi\n"},
{For: "zsh", Slot: "normal", Code: "source ~/.local/share/powerlevel10k/powerlevel10k.zsh-theme"},
{For: "zsh", Slot: "normal", Code: "[[ -f ~/.local/share/powerlevel10k/p10k.zsh ]] && source ~/.local/share/powerlevel10k/p10k.zsh"},
}},
{Module: "zsh-autosuggestions", Shell: []ShellCode{
{For: "zsh", Slot: "normal", Code: "source /usr/share/zsh/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh"},
{For: "bash", Slot: "normal", Code: "echo not for zsh"},
}},
{Module: "direnv", Shell: []ShellCode{
{For: "fish", Slot: "last", Code: "direnv hook fish | source"},
{For: "bash", Slot: "last", Code: "eval \"$(direnv hook bash)\""},
}},
}
}
// ADR 0204 §3: a slot holds that shell's code only, in module order, each module's pieces in the
// order it declared them under a line naming it; empty when nothing is contributed.
func TestShellCodeLandsInItsSlotInModuleOrderForItsShellOnly(t *testing.T) {
if got, want := shellCode(shells(), "zsh", "normal"), "# powerlevel10k\n"+
"source ~/.local/share/powerlevel10k/powerlevel10k.zsh-theme\n"+
"[[ -f ~/.local/share/powerlevel10k/p10k.zsh ]] && source ~/.local/share/powerlevel10k/p10k.zsh\n"+
"# zsh-autosuggestions\n"+
"source /usr/share/zsh/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh\n"; got != want {
t.Fatalf("zsh's normal slot is\n%s\nnot\n%s", got, want)
}
if got, want := shellCode(shells(), "zsh", "last"), "# zsh-syntax-highlighting\n"+
"source /usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh\n"; got != want {
t.Fatalf("zsh's last slot is\n%s\nnot\n%s", got, want)
}
if got, want := shellCode(shells(), "bash", "last"), "# direnv\neval \"$(direnv hook bash)\"\n"; got != want {
t.Fatalf("bash's last slot is %q, not %q", got, want)
}
if got := shellCode(shells(), "fish", "first"); got != "" {
t.Fatalf("a slot nobody contributed to holds %q", got)
}
}
// The holder of node-login-shell, as WP3's zsh module writes its block, with its own zsh around the
// slots — which holds `${…}` of the shell's own that no mesh pass may touch either.
func zshHolder() Manifest {
return Manifest{Module: "zsh", Claims: []Claim{{Name: LoginShellSeat, Scope: ScopeNode}},
Resources: []map[string]any{
{"id": "zshrc", "type": "file", "path": "${machine:account-home}/.zshrc", "content": "" +
"${shell:zsh:first}" +
"PROMPT='%n@%m ${PWD/#$HOME/~} '\n" +
"${shell:zsh:normal}" +
"alias ll='ls -l'\n" +
"${shell:zsh:last}"},
}}
}
// The case the ordering exists for: contributed zsh code full of `${…}` reaches the file byte for
// byte, because the shell's code is placed after every other placeholder pass and in one pass — a
// scanner for the mesh's placeholders that ran after it would read `${XDG_CACHE_HOME:-…}` and
// `${(%):-%n}` as the mesh's, or fill a `${machine:…}` some module wrote for its shell to see.
func TestShellCodeReachesTheHoldersFileByteForByte(t *testing.T) {
modules := append(shells(), zshHolder(), Manifest{Module: "sly", Shell: []ShellCode{
{For: "zsh", Slot: "last", Code: "echo ${machine:account-home} ${secret:x} ${shell:zsh:first} ${environment:posix}"},
}})
r := Resolution{Node: "workstation", Account: "op", Modules: modules}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
var zshrc map[string]any
for _, res := range out {
if res["id"] == "zsh.zshrc" {
zshrc = res
}
}
if zshrc == nil {
t.Fatalf("the holder's file was not composed: %v", out)
}
if zshrc["path"] != "/home/op/.zshrc" {
t.Fatalf("the holder's own placeholders were not filled first: %v", zshrc["path"])
}
want := "# powerlevel10k\n" +
"if [[ -r \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\" ]]; then\n" +
" source \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\"\nfi\n" +
"PROMPT='%n@%m ${PWD/#$HOME/~} '\n" +
"# powerlevel10k\n" +
"source ~/.local/share/powerlevel10k/powerlevel10k.zsh-theme\n" +
"[[ -f ~/.local/share/powerlevel10k/p10k.zsh ]] && source ~/.local/share/powerlevel10k/p10k.zsh\n" +
"# zsh-autosuggestions\n" +
"source /usr/share/zsh/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh\n" +
"alias ll='ls -l'\n" +
"# sly\n" +
"echo ${machine:account-home} ${secret:x} ${shell:zsh:first} ${environment:posix}\n" +
"# zsh-syntax-highlighting\n" +
"source /usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh\n"
if got := zshrc["content"]; got != want {
t.Fatalf("the holder's .zshrc is\n%s\nnot\n%s", got, want)
}
}
// The holder of node-environment places both renderings, and they are the same as rendered alone.
func TestTheEnvironmentHolderPlacesBothRenderings(t *testing.T) {
holder := Manifest{Module: "node-env", Claims: []Claim{{Name: EnvironmentSeat, Scope: ScopeNode}},
Resources: []map[string]any{
{"id": "posix", "type": "file", "path": "${machine:account-home}/.config/mesh/environment.sh",
"content": "# The mesh's environment.\n${environment:posix}"},
{"id": "systemd", "type": "file", "path": "${machine:account-home}/.config/environment.d/50-mesh.conf",
"content": "${environment:systemd}"},
}}
r := Resolution{Node: "workstation", Account: "op", Modules: append(contributors(), holder)}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
by := map[string]any{}
for _, res := range out {
by[res["id"].(string)] = res["content"]
}
if by["node-env.posix"] != "# The mesh's environment.\n"+composedPOSIX {
t.Fatalf("the POSIX file is\n%v", by["node-env.posix"])
}
if by["node-env.systemd"] != composedSystemd {
t.Fatalf("the environment.d file is\n%v", by["node-env.systemd"])
}
}
// ADR 0203 §5 and ADR 0204 §3: a placeholder outside the seat's holder is refused — by the parser,
// which is what the catalogue check and registration run, and again at composition, in the same words.
func TestAPlaceholderOutsideTheHolderIsRefused(t *testing.T) {
for _, c := range []struct{ content, want string }{
{"${environment:posix}", "toolchain's resource rc names ${environment:posix} and toolchain does not claim node-environment"},
{"${shell:zsh:normal}", "toolchain's resource rc names ${shell:zsh:normal} and toolchain does not claim node-login-shell"},
} {
raw := `{"module":"toolchain","resources":[{"id":"rc","type":"file","path":"/etc/rc","content":"` + c.content + `"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("the catalogue check accepted %s outside its holder: %v", c.content, err)
}
m := Manifest{Module: "toolchain", Resources: []map[string]any{
{"id": "rc", "type": "file", "path": "/etc/rc", "content": c.content}}}
r := Resolution{Node: "workstation", Account: "op", Modules: []Manifest{m}}
if _, err := r.Declaration(Rendering{}); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("composition accepted %s outside its holder: %v", c.content, err)
}
}
}
// A key nobody renders is refused, not left in the file as a literal.
func TestAnUnknownPlaceholderKeyIsRefused(t *testing.T) {
for _, c := range []struct{ content, want string }{
{"${environment:foo}", "names ${environment:foo}; the environment is ${environment:posix} or ${environment:systemd}"},
{"${shell:zsh:middle}", "names ${shell:zsh:middle}; shell code is ${shell:<shell>:<slot>}"},
{"${shell:tcsh:first}", "names ${shell:tcsh:first}; shell code is ${shell:<shell>:<slot>}"},
{"${shell:zsh}", "names ${shell:zsh}; shell code is ${shell:<shell>:<slot>}"},
} {
raw := `{"module":"holder","claims":[{"name":"node-environment","scope":"node"},{"name":"node-login-shell","scope":"node"}],` +
`"resources":[{"id":"rc","type":"file","path":"/etc/rc","content":"` + c.content + `"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s was accepted: %v", c.content, err)
}
}
// And outside a file's content, where nothing could be placed.
raw := `{"module":"holder","claims":[{"name":"node-environment","scope":"node"}],` +
`"resources":[{"id":"rc","type":"file","path":"/etc/${environment:posix}","content":"x"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil ||
!strings.Contains(err.Error(), "names ${environment:posix} in its path; the environment and the shell's code are placed only in a file's content") {
t.Errorf("a placeholder in a path was accepted: %v", err)
}
}
// What ADR 0203 §2 allows a contribution to say, refused at parse when it says anything else.
func TestAMalformedEnvironmentIsRefusedAtParse(t *testing.T) {
for _, c := range []struct{ environment, want string }{
{`{"variables":{"1X":"a"}}`, `tool sets the variable "1X", which is not a name a shell accepts`},
{`{"variables":{"MY-VAR":"a"}}`, `tool sets the variable "MY-VAR", which is not a name a shell accepts`},
{`{"variables":{"PATH":"/bin"}}`, `tool sets PATH as a variable; a module adds an entry under environment.path`},
{`{"variables":{"A":"$HOME/x"}}`, `tool sets A to "$HOME/x", which holds a $ that is not one of the machine's ${machine:…} facts`},
{`{"variables":{"A":"${HOME}/x"}}`, `tool sets A to "${HOME}/x", which holds a $`},
{`{"variables":{"A":"it's"}}`, `tool sets A to "it's", which holds a quote`},
{`{"variables":{"A":"say \"hi\""}}`, `which holds a quote`},
{`{"variables":{"A":"a\\b"}}`, `which holds a backslash`},
{`{"variables":{"A":"a\nb"}}`, `which holds a line break`},
{`{"variables":{"A":"a\u0000b"}}`, `which holds a NUL`},
{`{"path":[{"entry":"","at":"start"}]}`, `tool's PATH entry 1 names no directory`},
{`{"path":[{"entry":"/a:/b","at":"start"}]}`, `tool puts "/a:/b" on PATH, which holds a colon`},
{`{"path":[{"entry":"$HOME/bin","at":"start"}]}`, `tool puts "$HOME/bin" on PATH, which holds a $`},
{`{"path":[{"entry":"/a","at":"middle"}]}`, `tool puts "/a" on PATH at "middle"; an entry goes at "start" or "end"`},
{`{"path":[{"entry":"/a"}]}`, `tool puts "/a" on PATH at ""`},
{`{"path":[{"entry":"/a","at":"start"},{"entry":"/a","at":"end"}]}`, `tool puts "/a" on PATH twice`},
{`{"variables":{"A":"x"},"paths":[]}`, `unknown field "paths"`},
} {
_, err := ParseManifest([]byte(`{"module":"tool","environment":` + c.environment + `}`))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: want %q, got %v", c.environment, c.want, err)
}
}
// What is allowed: a literal, and the machine's own facts.
if _, err := ParseManifest([]byte(`{"module":"tool","environment":{` +
`"variables":{"GOPATH":"${machine:account-home}/go","DISABLE_X":"1","ANSWER":"a b+c=d"},` +
`"path":[{"entry":"${machine:account-home}/go/bin","at":"start"},{"entry":"/opt/x","at":"end"}]}}`)); err != nil {
t.Fatalf("a well-formed environment was refused: %v", err)
}
}
func TestMalformedShellCodeIsRefusedAtParse(t *testing.T) {
for _, c := range []struct{ shell, want string }{
{`[{"for":"tcsh","slot":"normal","code":"x"}]`, `tool's shell code 1 is for "tcsh"; the shells are zsh, bash, fish`},
{`[{"for":"zsh","slot":"middle","code":"x"}]`, `tool's shell code 1 goes in the slot "middle"; the slots are first, normal, last`},
{`[{"for":"zsh","slot":"last","code":"x"},{"for":"zsh","slot":"last","code":" \n"}]`, `tool's shell code 2 has no code`},
{`[{"for":"zsh","slot":"last","code":"x","order":1}]`, `unknown field "order"`},
} {
_, err := ParseManifest([]byte(`{"module":"tool","shell":` + c.shell + `}`))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: want %q, got %v", c.shell, c.want, err)
}
}
// The code itself is never judged: a shell's own `${…}` is not the mesh's.
if _, err := ParseManifest([]byte(`{"module":"tool","shell":[{"for":"zsh","slot":"first",` +
`"code":"source \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\""}]}`)); err != nil {
t.Fatalf("shell code was judged as if it were the mesh's: %v", err)
}
}
// ADR 0203 §1 and ADR 0204 §1: both seats are the mesh's own, held once per machine; the login
// shell's contract is `execute`, described and with a schema an agent can call.
func TestTheSeatTableCarriesTheEnvironmentAndTheLoginShell(t *testing.T) {
env, ok := SeatNamed("node-environment")
if !ok || env.Scope != ScopeNode || env.Decision != "novox/hq ADR 0203" ||
len(env.Serves)+len(env.Accepts)+len(env.Emits) != 0 || env.Delivers != "" {
t.Fatalf("node-environment is not a node seat with no protocol: %+v (defined %v)", env, ok)
}
shell, ok := SeatNamed("node-login-shell")
if !ok || shell.Scope != ScopeNode || shell.Decision != "novox/hq ADR 0204" {
t.Fatalf("node-login-shell is not a node seat: %+v (defined %v)", shell, ok)
}
if len(shell.Serves) != 1 || shell.Serves[0].Name != "execute" || shell.Serves[0].Description == "" {
t.Fatalf("the login shell serves %+v, not execute alone", shell.Serves)
}
props, _ := shell.Serves[0].Input["properties"].(map[string]any)
required, _ := shell.Serves[0].Input["required"].([]string)
if _, has := props["command"]; !has || len(required) != 1 || required[0] != "command" {
t.Fatalf("execute does not require a command: %v", shell.Serves[0].Input)
}
if _, has := props["timeout_seconds"]; !has {
t.Fatalf("execute takes no timeout: %v", props)
}
}
// ADR 0204 §1: the login shell is the mesh's, so no module declares it — neither under the mesh's
// name nor under the name a module gave it before.
func TestNoModuleMayDeclareTheLoginShell(t *testing.T) {
for _, n := range []string{"login-shell", "node-login-shell", "node-environment"} {
raw := `{"module":"zsh","seats":[{"name":"` + n + `","scope":"node","serves":["execute"]}],"tools":["execute"]}`
_, err := ParseManifest([]byte(raw))
if err == nil {
t.Errorf("a module declaring %q was accepted", n)
}
}
got := strings.Join(declaredSeatProblems(Manifest{Module: "zsh",
DefinesSeats: []SeatDeclaration{{Name: "login-shell", Scope: ScopeNode}}}), "; ")
if !strings.Contains(got, `zsh declares a seat named "login-shell"; the login shell is the mesh's own seat node-login-shell`) {
t.Fatalf("declaring login-shell was not refused by name: %q", got)
}
// And a shell module claiming the mesh's seat, serving execute, is what the seat is for.
m, err := ParseManifest([]byte(`{"module":"zsh","tools":["execute"],` +
`"claims":[{"name":"node-login-shell","scope":"node"}]}`))
if err != nil {
t.Fatal(err)
}
if err := CanHold(m, Seat{Name: LoginShellSeat, Scope: ScopeNode, Serves: loginShellVerbs()}); err != nil {
t.Fatalf("a shell module claiming the seat cannot hold it: %v", err)
}
}
internal/catalogue/foundation_manifests_test.go
+17 -23
View File
@@ -1,7 +1,6 @@
package catalogue
import (
"encoding/json"
"fmt"
"os"
"reflect"
@@ -171,7 +170,7 @@ func TestTheForgeHoldsTheNpmAndGitSeats(t *testing.T) {
// **And the forge's own address follows it**, composed from the manifest in the catalogue beside
// this checkout (novox/hq 04-ISSUES/088).
//
// The forge is reached a third way that neither test above covers: by its own code, over the
// The forge is reached a third way that neither test above covers: by its own sidecar, over the
// machine's loopback, told where to go in its environment. The `2999:3000` mapping that lets the
// forge go on binding 3000 does nothing for a caller dialling the machine — so a literal there is
// wrong on every node whose assignment differs, and wrong for a second reason on a node given the
@@ -179,13 +178,13 @@ func TestTheForgeHoldsTheNpmAndGitSeats(t *testing.T) {
// in an `env` at all is a declaration, not a manifest.
func TestTheForgesOwnAddressFollowsThePortTheNodeGaveIt(t *testing.T) {
forge, err := catalogueManifest(t, "gitea").Resolve([]Built{{
Name: "code", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "gitea/code/blobs/" + bundleDigest, Digest: bundleDigest,
Name: "runtime", Kind: ArtifactImage,
Reference: "registry.example/gitea-runtime@sha256:" + strings.Repeat("a", 64),
}})
if err != nil {
t.Fatalf("the forge's manifest does not resolve against its own build: %v", err)
}
r := Resolution{Node: "anchor", Modules: []Manifest{forge, theRuntime(t)}, Needs: []Needed{
r := Resolution{Node: "anchor", Modules: []Manifest{forge}, Needs: []Needed{
{Name: "postgres-database", For: "gitea", From: "anchor", At: "127.0.0.1",
Serves: map[string]any{"port": float64(5432)}, Sealed: "sealed-db"},
{Name: "route", For: "gitea", From: "anchor"},
@@ -195,8 +194,8 @@ func TestTheForgesOwnAddressFollowsThePortTheNodeGaveIt(t *testing.T) {
// The number this node was given for the forge — the one the machine it is about to run on
// already publishes.
out, err := r.Declaration(Rendering{ArtifactStore: "anchor.internal:5101",
Needed: map[string]map[string]string{RuntimeModule: {"broker": "sealed-broker"}},
out, err := r.Declaration(Rendering{
Needed: map[string]map[string]string{"gitea": {"broker": "sealed-broker"}},
Given: map[string]map[int]int{"gitea": {3000: 2999}},
})
if err != nil {
@@ -211,19 +210,14 @@ func TestTheForgesOwnAddressFollowsThePortTheNodeGaveIt(t *testing.T) {
if published := fmt.Sprint(server["ports"]); !strings.Contains(published, "2999:3000") {
t.Fatalf("the forge is not published on the port this node gave it: %v", server["ports"])
}
// The forge's own code runs in the node's runtime (novox/hq ADR 0198), given its words there.
runtime := fileNamed(out, RuntimeModule+"."+RuntimeProcessID())
runtime := fileNamed(out, "gitea.runtime")
if runtime == nil {
t.Fatalf("the node's runtime is not in the declaration: %v", ids(out))
t.Fatalf("the forge's sidecar is not in the declaration: %v", out)
}
env, _ := runtime["env"].(map[string]string)
var given map[string]map[string]string
if err := json.Unmarshal([]byte(env[RuntimeToolEnv]), &given); err != nil {
t.Fatalf("the runtime's %s is not JSON: %q", RuntimeToolEnv, env[RuntimeToolEnv])
}
if given["gitea"]["MESH_GITEA_URL"] != "http://127.0.0.1:2999" {
t.Fatalf("the forge's code dials %v while the machine publishes the forge on 2999 — "+
"whatever reads it dials a dead port", given["gitea"]["MESH_GITEA_URL"])
env, _ := runtime["env"].(map[string]any)
if env["MESH_GITEA_URL"] != "http://127.0.0.1:2999" {
t.Fatalf("the forge's sidecar dials %v while the machine publishes the forge on 2999 — "+
"whatever reads it dials a dead port", env["MESH_GITEA_URL"])
}
}
@@ -235,13 +229,13 @@ func declaredGiteaSsh(t *testing.T, given map[int]int) map[string]any {
t.Helper()
forge := catalogueManifest(t, "gitea")
resolved, err := forge.Resolve([]Built{{
Name: "code", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "gitea/code/blobs/" + bundleDigest, Digest: bundleDigest,
Name: "runtime", Kind: ArtifactImage,
Reference: "registry.example/gitea-runtime@sha256:" + strings.Repeat("a", 64),
}})
if err != nil {
t.Fatalf("the forge's manifest does not resolve against its own build: %v", err)
}
r := Resolution{Node: "anchor", Modules: []Manifest{resolved, theRuntime(t)}, Needs: []Needed{
r := Resolution{Node: "anchor", Modules: []Manifest{resolved}, Needs: []Needed{
{Name: "postgres-database", For: "gitea", From: "anchor", At: "127.0.0.1",
Serves: map[string]any{"port": float64(5432)}, Sealed: "sealed-db"},
{Name: "route", For: "gitea", From: "anchor"},
@@ -252,8 +246,8 @@ func declaredGiteaSsh(t *testing.T, given map[int]int) map[string]any {
for k, v := range given {
givenPorts[k] = v
}
out, err := r.Declaration(Rendering{ArtifactStore: "anchor.internal:5101",
Needed: map[string]map[string]string{RuntimeModule: {"broker": "sealed-broker"}},
out, err := r.Declaration(Rendering{
Needed: map[string]map[string]string{"gitea": {"broker": "sealed-broker"}},
Ports: map[string]map[int]int{"gitea": givenPorts},
Given: map[string]map[int]int{"gitea": given},
})
internal/catalogue/genesis_image_test.go
-29
View File
@@ -1,29 +0,0 @@
package catalogue
import (
"os"
"regexp"
"testing"
)
// novox/hq issue 223: genesis raises the controller as a container built from this repository's own
// Dockerfile, with no build arguments — the manifest no longer builds an image, so nothing passes a
// base in. The Dockerfile's own default must therefore be a Go that builds this module, pinned by
// digest, and the replacement the manifest's process names must be the container genesis raises.
func TestGenesisCanBuildTheControllersImageAsItStands(t *testing.T) {
raw, err := os.ReadFile("../../Dockerfile")
if err != nil {
t.Fatal(err)
}
if !regexp.MustCompile(`(?m)^ARG GO_BASE=golang@sha256:[0-9a-f]{64}$`).Match(raw) {
t.Fatal("the Dockerfile's default Go base is not pinned by digest; genesis builds it with no arguments")
}
makefile, err := os.ReadFile("../../Makefile")
if err != nil {
t.Fatal(err)
}
pin := regexp.MustCompile(`golang@sha256:[0-9a-f]{64}`)
if string(pin.Find(raw)) != string(pin.Find(makefile)) {
t.Errorf("the Dockerfile and the Makefile build on different Go: %s, %s", pin.Find(raw), pin.Find(makefile))
}
}
internal/catalogue/grant_secret_owner_test.go
-103
View File
@@ -1,103 +0,0 @@
package catalogue
import (
"strings"
"testing"
)
// A grant secret is read by whatever provisions, and that stopped being root (novox/hq issue 225).
//
// The mesh seals one credential per consumer beside the provider's contributions file. The
// provider's harness reads both: the file to learn who asked, the secret to set their password.
// While a module's own code ran in a container as root, a root-owned 0600 file was readable by
// the thing that needed it. ADR 0198 moved that code under the node's runtime, which runs as the
// operator's account — and the secret stayed root's.
//
// **The cost was silence.** The harness says `secret not readable yet`, which is true and
// ordinary on the first pass, so four thousand refusals in three hours read as patience. No user
// was ever created, and two consumers crash-looped against a database that had never heard of
// them.
//
// The same reasoning is already written for a module's *own* secrets, three hundred lines above:
// "a root-owned 0600 file is one that process cannot read". This is that rule reaching the other
// kind of secret the mesh writes for a module.
// aProviderWithABundle is a provider whose code is a bundle the node's runtime runs — the shape
// every TypeScript provisioner has since ADR 0198.
func aProviderWithABundle() Manifest {
return Manifest{
Module: "mongodb", Version: "1",
Provides: FromAnywhere("mongodb-database"),
Receives: map[string]string{"mongodb-database": "/var/lib/mongodb/grants/mesh.json"},
Grants: map[string]string{"mongodb-database": "/var/lib/mongodb/grants"},
Bundles: []Bundle{{Name: "code", Language: "typescript"}},
Resources: []map[string]any{{
"id": "server", "type": "container", "name": "mongodb-server",
"image": "mongo@sha256:" + strings.Repeat("a", 64),
}},
}
}
func TestAGrantSecretIsOwnedByTheAccountThatProvisions(t *testing.T) {
r, err := Resolve(shelf(aProviderWithABundle()), []string{"mongodb"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
r.Account = "operator"
out, err := r.Declaration(Rendering{Grants: []Grant{{
Provision: "mongodb-database", Consumer: "workstation", From: "photos", Slug: "photos",
Values: map[string]any{}, Sealed: "c2VhbGVk",
}}})
if err != nil {
t.Fatal(err)
}
var secret map[string]any
for _, res := range out {
if res["type"] == "file" && strings.HasSuffix(fmtPath(res), ".secret") {
secret = res
}
}
if secret == nil {
t.Fatalf("no grant secret was composed at all: %v", out)
}
if got := secret["owner"]; got != "operator" {
t.Fatalf("the grant secret at %v belongs to %v; the provisioner runs as %q and a "+
"root-owned 0600 file is one it cannot read — which is silent, because the harness "+
"calls it \"not readable yet\"", fmtPath(secret), got, "operator")
}
}
// And a provider whose code still runs in a container keeps the owner it declares, so this
// changes nothing for the modules the runtime has not taken.
func TestAContainerProvidersGrantSecretKeepsItsDeclaredOwner(t *testing.T) {
m := aProviderWithABundle()
m.Bundles = nil
m.SecretsOwner = "65534:65534"
r, err := Resolve(shelf(m), []string{"mongodb"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
r.Account = "operator"
out, err := r.Declaration(Rendering{Grants: []Grant{{
Provision: "mongodb-database", Consumer: "workstation", From: "photos", Slug: "photos",
Values: map[string]any{}, Sealed: "c2VhbGVk",
}}})
if err != nil {
t.Fatal(err)
}
for _, res := range out {
if res["type"] == "file" && strings.HasSuffix(fmtPath(res), ".secret") {
if got := res["owner"]; got != "65534:65534" {
t.Fatalf("a container provider's grant secret belongs to %v, not what it declares", got)
}
return
}
}
t.Fatal("no grant secret was composed")
}
func fmtPath(r map[string]any) string {
p, _ := r["path"].(string)
return p
}
internal/catalogue/graphical_session.go
-109
View File
@@ -1,109 +0,0 @@
package catalogue
// The graphical session's seats (novox/hq ADR 0208): one module per piece of software, each piece's
// role a node seat in the mesh's own set, so i3 and sway, xterm and foot, rofi and dmenu compete for
// a role rather than each inventing one — and a machine running two of one role is refused at
// assignment instead of found by two bars on one screen.
const (
LoginManagerSeat = "node-login-manager"
DisplayServerSeat = "node-display-server"
DisplaySessionSeat = "node-display-session"
TerminalEmulatorSeat = "node-terminal-emulator"
LauncherSeat = "node-launcher"
NotifierSeat = "node-notifier"
LockScreenSeat = "node-lock-screen"
ClipboardSeat = "node-clipboard"
BarSeat = "node-bar"
CompositorSeat = "node-compositor"
SecretServiceSeat = "node-secret-service"
)
// graphicalSessionSeats are the eleven, in the order ADR 0208's table reads, each with the verbs
// research 026/05 starts it with. Three have none yet: the bar, the compositor and the secret
// service are roles a second holder competes for, and nothing has needed to ask them anything.
func graphicalSessionSeats() []Seat {
const decided = "novox/hq ADR 0208"
return []Seat{
{Name: LoginManagerSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "sessions", Description: "The sessions the login manager offers on this machine, and which " +
"one the operator account starts by default.",
Input: schema(map[string]string{}, nil)},
}},
{Name: DisplayServerSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "displays", Description: "The monitors connected now, each with its identity, its modes and " +
"where it is placed; and the layout profile in force, if one matches.",
Input: schema(map[string]string{}, nil)},
// Profiles are keyed by the monitors' identities and are the operator's data (ADR 0208 §6).
{Name: "layout", Description: "The monitor layout profiles, keyed by the connected monitors' " +
"identities: list them, save the current arrangement under a name, or apply one.",
Input: withEnum(schema(map[string]string{
"action": "list, save or apply",
"name": "the profile to save or apply (save and apply only)",
}, []string{"action"}), "action", "list", "save", "apply")},
}},
// It receives window-manager configuration lines from every other module (novox/hq ADR 0212):
// bindings, start-up commands, rules — placed by the session's holder, never written into
// its directory by the contributor.
{Name: DisplaySessionSeat, Scope: ScopeNode, Decision: decided,
Receives: []Receivable{{Kind: "config", Comment: "#"}}, Serves: []Verb{
{Name: "reload", Description: "Reload the session's configuration in place, keeping its windows.",
Input: schema(map[string]string{}, nil)},
{Name: "workspaces", Description: "The session's workspaces: each one's name, output, and whether " +
"it is visible or focused.",
Input: schema(map[string]string{}, nil)},
{Name: "windows", Description: "The session's windows: each one's title, class, workspace and " +
"whether it has focus; narrowed to one workspace when named.",
Input: schema(map[string]string{"workspace": "one workspace (optional)"}, nil)},
}},
{Name: TerminalEmulatorSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "open", Description: "Open a terminal window in the operator's session, running a command " +
"or the login shell, in a directory or the account's home.",
Input: schema(map[string]string{
"command": "what to run in it (optional; the login shell when absent)",
"directory": "where it starts (optional; the account's home when absent)",
}, nil)},
}},
{Name: LauncherSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "menu", Description: "Put a menu of choices in front of the operator and answer with the " +
"one chosen, or nothing when the menu was dismissed — the dmenu-compatible contract.",
Input: map[string]any{"type": "object", "required": []string{"choices"},
"properties": map[string]any{
"choices": map[string]any{"type": "array", "items": map[string]any{"type": "string"},
"description": "the lines to choose between, in order"},
"prompt": map[string]any{"type": "string", "description": "what the menu asks (optional)"},
}}},
}},
{Name: NotifierSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "send", Description: "Show the operator a notification.",
Input: withEnum(schema(map[string]string{
"title": "the notification's summary",
"body": "its text (optional)",
"urgency": "low, normal (the default) or critical",
}, []string{"title"}), "urgency", "low", "normal", "critical")},
{Name: "history", Description: "The notifications shown lately, newest first.",
Input: schema(map[string]string{"limit": "how many (optional, default 20)"}, nil)},
}},
{Name: LockScreenSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "lock", Description: "Lock the operator's session now.",
Input: schema(map[string]string{}, nil)},
}},
{Name: ClipboardSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "history", Description: "What the clipboard held lately, newest first.",
Input: schema(map[string]string{"limit": "how many (optional, default 20)"}, nil)},
{Name: "copy", Description: "Put text on the operator's clipboard.",
Input: schema(map[string]string{"text": "the text"}, []string{"text"})},
}},
{Name: BarSeat, Scope: ScopeNode, Decision: decided},
{Name: CompositorSeat, Scope: ScopeNode, Decision: decided},
{Name: SecretServiceSeat, Scope: ScopeNode, Decision: decided},
}
}
// withEnum narrows one string property of a schema to the values it may take, so a caller is told
// the choices by the schema rather than by a refusal.
func withEnum(s map[string]any, property string, values ...string) map[string]any {
props := s["properties"].(map[string]any)
p := props[property].(map[string]any)
p["enum"] = values
return s
}
internal/catalogue/graphical_session_test.go
-213
View File
@@ -1,213 +0,0 @@
package catalogue
import (
"encoding/json"
"reflect"
"strings"
"testing"
)
// Defends novox/hq ADR 0208: the graphical session is one module per piece, on the mesh's seats.
// §2: the eleven roles are the mesh's own node seats, each with the verbs it starts with.
func TestTheGraphicalSessionsSeatsAreTheMeshsOwnWithTheirVerbs(t *testing.T) {
want := map[string][]string{
LoginManagerSeat: {"sessions"},
DisplayServerSeat: {"displays", "layout"},
DisplaySessionSeat: {"reload", "workspaces", "windows"},
TerminalEmulatorSeat: {"open"},
LauncherSeat: {"menu"},
NotifierSeat: {"send", "history"},
LockScreenSeat: {"lock"},
ClipboardSeat: {"history", "copy"},
BarSeat: nil,
CompositorSeat: nil,
SecretServiceSeat: nil,
}
for name, verbs := range want {
s, ok := SeatNamed(name)
if !ok {
t.Errorf("%s is not in the mesh's set", name)
continue
}
if s.Scope != ScopeNode || s.Decision != "novox/hq ADR 0208" {
t.Errorf("%s is %s-scoped under %q", name, s.Scope, s.Decision)
}
var got []string
for _, v := range s.Serves {
got = append(got, v.Name)
if v.Description == "" || v.Input["type"] != "object" {
t.Errorf("%s.%s has no description or no object schema", name, v.Name)
}
}
if !reflect.DeepEqual(got, verbs) {
t.Errorf("%s serves %v, want %v", name, got, verbs)
}
}
// The launcher's menu takes a list, and the layout verb says its actions.
menu, _ := SeatNamed(LauncherSeat)
choices := menu.Serves[0].Input["properties"].(map[string]any)["choices"].(map[string]any)
if choices["type"] != "array" {
t.Errorf("menu's choices are %v, not a list", choices["type"])
}
display, _ := SeatNamed(DisplayServerSeat)
action := display.Serves[1].Input["properties"].(map[string]any)["action"].(map[string]any)
if !reflect.DeepEqual(action["enum"], []string{"list", "save", "apply"}) {
t.Errorf("layout's actions are %v", action["enum"])
}
// And they survive the store's JSON, which is where the live set comes from.
if _, err := json.Marshal(graphicalSessionSeats()); err != nil {
t.Fatal(err)
}
}
// §2: a module may claim one of them, and may not declare it as its own.
func TestNoModuleMayDeclareAGraphicalSessionSeat(t *testing.T) {
raw := `{"module":"xorg","seats":[{"name":"node-display-server","scope":"node"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "mesh's own namespace") {
t.Fatalf("a module declared node-display-server as its own: %v", err)
}
}
func displayServer(name, display string, claims ...string) Manifest {
m := Manifest{Module: name, Provides: []Offer{{Name: display, Reach: ReachMachine}}}
for _, c := range claims {
m.Claims = append(m.Claims, Claim{Name: c})
}
return m
}
func windowManager() Manifest {
return Manifest{Module: "i3", Requires: []string{"x11-display"}}
}
// §3: a display is resolved on the requiring module's own node.
func TestAMachineReachRequirementResolvesToTheProviderOnItsOwnNode(t *testing.T) {
cat := shelf(windowManager(), displayServer("xorg", "x11-display", DisplayServerSeat))
got, err := Resolve(cat, []string{"xorg", "i3"}, workstation(), World{})
if err != nil {
t.Fatalf("i3 beside xorg did not resolve: %v", err)
}
if !reflect.DeepEqual(names(got), []string{"xorg", "i3"}) && !reflect.DeepEqual(names(got), []string{"i3", "xorg"}) {
t.Errorf("resolved %v", names(got))
}
}
// §3: never answered by installing a provider, and never by another machine's.
func TestAMachineReachRequirementIsNotPulledInNorAnsweredFromAnotherNode(t *testing.T) {
cat := shelf(windowManager(),
displayServer("xorg", "x11-display", DisplayServerSeat),
displayServer("xwayland", "x11-display"))
// Another machine runs xorg and says so to the world; it does not count.
world := World{Offered: map[string][]Provider{
"x11-display": {{Node: "laptop", At: "laptop.mesh", Module: "xorg"}}}}
_, err := Resolve(cat, []string{"i3"}, workstation(), world)
if err == nil {
t.Fatal("i3 resolved on a machine with no display of its own")
}
for _, want := range []string{
`"x11-display" is wanted by i3`, "usable only on the machine that provides it",
"assign one to workstation", "xorg (holds node-display-server)", "xwayland",
} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not say %q:\n%v", want, err)
}
}
// With a single provider in the catalogue too: one candidate is still not a choice to make
// for somebody, unlike a node-scoped provision without the machine's reach.
_, err = Resolve(shelf(windowManager(), displayServer("xorg", "x11-display", DisplayServerSeat)),
[]string{"i3"}, workstation(), World{})
if err == nil {
t.Fatal("xorg was pulled in for i3")
}
// Not in the first pass, whose refusals take the machine off the network.
if _, err := Resolve(cat, []string{"i3"}, workstation(), World{Unchecked: true}); err != nil {
t.Errorf("the first pass refused: %v", err)
}
}
func TestTheMachinesReachIsAProvisionsOnlyReachAndIsNodeScoped(t *testing.T) {
for _, c := range []struct{ provides, want string }{
{`{"name":"x11-display","reach":"internal"}`, `with reach "internal"; a provision's reach is "machine" or nothing`},
{`{"name":"x11-display","scope":"mesh","reach":"machine"}`, `at scope "mesh" with the machine's reach`},
} {
_, err := ParseManifest([]byte(`{"module":"xorg","provides":[` + c.provides + `]}`))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: want %q, got %v", c.provides, c.want, err)
}
}
m, err := ParseManifest([]byte(`{"module":"xorg","provides":[{"name":"x11-display","reach":"machine"}]}`))
if err != nil {
t.Fatal(err)
}
if !m.Provides[0].MachineReach() {
t.Fatal("the reach was not read")
}
back, _ := json.Marshal(m.Provides[0])
if string(back) != `{"name":"x11-display","reach":"machine"}` {
t.Errorf("written back as %s", back)
}
}
func TestACatalogueDisagreeingAboutAProvisionsReachIsRefused(t *testing.T) {
cat := shelf(windowManager(), displayServer("xorg", "x11-display"),
Manifest{Module: "fake-x", Provides: Offers("x11-display")})
_, err := Resolve(cat, []string{"xorg", "i3"}, workstation(), World{})
if err == nil || !strings.Contains(err.Error(), `the catalogue disagrees about "x11-display"`) {
t.Fatalf("a provision with and without the machine's reach gave %v", err)
}
}
// §4: xinitrc and xresources slots, placed by the display server's holder alone.
func TestTheSessionsFilesArePlacedByTheDisplayServersHolderAlone(t *testing.T) {
xorg := Manifest{Module: "xorg", Claims: []Claim{{Name: DisplayServerSeat}},
Shell: []ShellCode{{For: "xinitrc", Slot: "first", Code: "xset s off"}},
Resources: []map[string]any{
{"id": "xinitrc", "type": "file", "path": "/home/op/.xinitrc",
"content": "${shell:xinitrc:first}${shell:xinitrc:normal}${shell:xinitrc:last}"},
{"id": "xresources", "type": "file", "path": "/home/op/.Xresources",
"content": "${shell:xresources:normal}"},
}}
i3 := Manifest{Module: "i3", Shell: []ShellCode{{For: "xinitrc", Slot: "last", Code: "exec i3"}}}
theme := Manifest{Module: "theme", Shell: []ShellCode{
{For: "xresources", Slot: "normal", Code: "Xft.dpi: 96"},
{For: "zsh", Slot: "normal", Code: "not for the session"},
}}
r := Resolution{Node: "workstation", Account: "op", Modules: []Manifest{xorg, i3, theme}}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
by := map[string]any{}
for _, res := range out {
by[res["id"].(string)] = res["content"]
}
if got := by["xorg.xinitrc"]; got != "# xorg\nxset s off\n# i3\nexec i3\n" {
t.Errorf("the .xinitrc is %q", got)
}
if got := by["xorg.xresources"]; got != "# theme\nXft.dpi: 96\n" {
t.Errorf("the .Xresources is %q", got)
}
// The contributions parse; the placeholders parse only in the holder.
if _, err := ParseManifest([]byte(`{"module":"i3","shell":[{"for":"xinitrc","slot":"last","code":"exec i3"},` +
`{"for":"xresources","slot":"normal","code":"i3.font: x"}]}`)); err != nil {
t.Fatalf("a session contribution was refused: %v", err)
}
for _, c := range []struct{ claims, content, want string }{
{``, "${shell:xinitrc:normal}", "does not claim node-display-server; every module's xinitrc is placed by the display server's holder alone"},
{`{"name":"node-login-shell"}`, "${shell:xresources:normal}", "does not claim node-display-server"},
{`{"name":"node-display-server"}`, "${shell:zsh:normal}", "does not claim node-login-shell"},
{`{"name":"node-display-server"}`, "${shell:xsession:normal}", "the session's file one of xinitrc, xresources"},
} {
raw := `{"module":"holder","claims":[` + c.claims + `],"resources":[{"id":"rc","type":"file","path":"/etc/rc","content":"` +
c.content + `"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s with claims [%s]: want %q, got %v", c.content, c.claims, c.want, err)
}
}
if _, err := ParseManifest([]byte(`{"module":"xorg","claims":[{"name":"node-display-server"}],` +
`"resources":[{"id":"rc","type":"file","path":"/home/op/.xinitrc","content":"${shell:xinitrc:last}"}]}`)); err != nil {
t.Errorf("the display server's holder could not place the session's slots: %v", err)
}
}
internal/catalogue/manifest.go
+5 -200
View File
@@ -147,17 +147,8 @@ type Offer struct {
// shared by every consumer (novox/hq ADR 0158): software that holds one password or one key
// cannot give each consumer a login of its own. The named secret must say how it is taken.
Credential *OfferCredential `json:"credential,omitempty"`
// Reach is ReachMachine for a provision usable only on the provider's own machine — a display
// (novox/hq ADR 0208 §3). Node scope already keeps a provision off other machines; what this adds
// is that a requirement for it is never answered by installing a provider: the display server is
// a seat's holder gated by the machine's graphical session, and pulling one in for whatever asked
// is the misassignment research 026 found. Unmet, the requirement is refused naming who could.
Reach string `json:"reach,omitempty"`
}
// MachineReach is whether a provision is usable only on its provider's own machine.
func (o Offer) MachineReach() bool { return o.Reach == ReachMachine }
// OfferCredential names which of the provider's own secrets a provision's consumers receive.
type OfferCredential struct {
Own string `json:"own"`
@@ -205,29 +196,27 @@ func (o *Offer) UnmarshalJSON(raw []byte) error {
Name string `json:"name"`
Scope string `json:"scope,omitempty"`
Credential *OfferCredential `json:"credential,omitempty"`
Reach string `json:"reach,omitempty"`
}
dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a provided name is either a string or {name, scope, credential, reach}: %w", err)
return fmt.Errorf("a provided name is either a string or {name, scope, credential}: %w", err)
}
o.Name, o.Scope, o.Credential, o.Reach = full.Name, full.Scope, full.Credential, full.Reach
o.Name, o.Scope, o.Credential = full.Name, full.Scope, full.Credential
return nil
}
// MarshalJSON writes back the short form when there is nothing else to say, so a manifest that
// went through the mesh comes out looking like the one that went in.
func (o Offer) MarshalJSON() ([]byte, error) {
if o.Scope == "" && o.Credential == nil && o.Reach == "" {
if o.Scope == "" && o.Credential == nil {
return json.Marshal(o.Name)
}
return json.Marshal(struct {
Name string `json:"name"`
Scope string `json:"scope,omitempty"`
Credential *OfferCredential `json:"credential,omitempty"`
Reach string `json:"reach,omitempty"`
}{o.Name, o.Scope, o.Credential, o.Reach})
}{o.Name, o.Scope, o.Credential})
}
// Manifest is everything a module says about itself.
@@ -336,15 +325,6 @@ type Manifest struct {
// person's account (design 25 §7) already had the same shape.
Invokes []string `json:"invokes,omitempty"`
// State is the current state this module keeps on the bus, by local name: each a key-value
// bucket the controller creates, which every instance of the module writes and reads
// (novox/hq ADR 0201). Not history — that is an event — and never a secret, sealed or not.
State []StateDeclaration `json:"state,omitempty"`
// Reads are other modules' state this module reads and watches, each `<module>.<name>`
// (novox/hq ADR 0201). Read-only: only the owner's instances write.
Reads []string `json:"reads,omitempty"`
// Capabilities the machine must have. A different field from Requires because the remedy
// differs: a missing module can be assigned, and a missing capability means the wrong
// machine.
@@ -528,22 +508,6 @@ type Manifest struct {
// holder. Like Filtering: one module per node gathers what every module declared and writes it.
Jailing *Jailing `json:"jailing,omitempty"`
// Environment is what this module adds to the operator account's environment: variables, and
// entries on PATH (novox/hq ADR 0203). Facts, not lines of one shell's syntax — the holder of
// node-environment places them, and the controller writes them in each reader's format. Like
// Jails: any module contributes, gathered from every module on the node, written by the holder.
Environment *Environment `json:"environment,omitempty"`
// Shell is code this module adds to the login shell's startup, for a named shell in a named
// slot (novox/hq ADR 0204). The controller never reads it: it is placed, in module order, where
// the holder of node-login-shell put the slot's placeholder.
Shell []ShellCode `json:"shell,omitempty"`
// Contributions are configuration this module gives the holder of a seat it does not hold, in
// that tool's own grammar (novox/hq ADR 0212): a seat, a kind the seat receives, and the text. The
// holder places them; the module depends on the seat (ADR 0210 §3).
Contributions []SeatContribution `json:"contributions,omitempty"`
// Guards are ports of this module's the mesh refuses on an adopted node except from the
// private network and from the machine itself (novox/hq ADR 0100) — the store's port and the
// broker's management port. The ports the software uses; the mesh guards where the machine
@@ -576,10 +540,6 @@ type Manifest struct {
// `restart-on` names to restart when the roster changes.
Facts map[string]RosterFile `json:"facts,omitempty"`
// Zone is the zone of names this module answers itself, and the listen that answers it (novox/hq
// ADR 0199). The mesh's resolver forwards the zone to it; nothing here names an address.
Zone *Zone `json:"zone,omitempty"`
// Certificate is where this module wants a certificate for its machine's name inside the
// mesh, and where the key that goes with it can be found.
//
@@ -797,11 +757,6 @@ type Artifact struct {
// list twice. A module declaring no tools has nothing the runtime loads, whatever it compiles.
Loads []string `json:"loads,omitempty"`
// External are packages a TypeScript bundle keeps as imports rather than inlining — a native
// addon, a package that reads its own files — and so carries the toolchain's node_modules for
// (novox/hq ADR 0193). Absent for nearly every bundle, which is then one file per entrypoint.
External []string `json:"external,omitempty"`
// Env is what a tools bundle is given on a machine (novox/hq ADR 0192): words and their values,
// paths and constants composed with ${dir:…} and ${port:…} exactly as a container's environment
// is, never a secret's content. The node's runtime hands it to this bundle and to no other.
@@ -1337,19 +1292,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
"%s provides %q at scope %q; a provision is %q or %q",
m.Module, p, s, ScopeNode, ScopeMesh))
}
switch {
case offer.Reach == "":
case offer.Reach != ReachMachine:
// The one reach a provision has (novox/hq ADR 0208): a provision reached over the private
// network is mesh scope, and the world reaches nothing but a name.
problems = append(problems, fmt.Sprintf(
"%s provides %q with reach %q; a provision's reach is %q or nothing",
m.Module, p, offer.Reach, ReachMachine))
case offer.At() != ScopeNode:
problems = append(problems, fmt.Sprintf(
"%s provides %q at scope %q with the machine's reach; a provision usable only on its own "+
"machine is node-scoped (novox/hq ADR 0208)", m.Module, p, offer.At()))
}
if p == m.Module {
// Harmless and worth saying: a module always provides its own name, so writing it
// suggests the author expected it not to.
@@ -1369,8 +1311,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
// module whose event names are wrong installs, starts, connects and reacts to nothing, with
// every log line saying it is fine (novox/hq 04-ISSUES/127).
problems = append(problems, EventProblems(m)...)
// And what it may call its state, and whose it may read (state.go, novox/hq ADR 0201).
problems = append(problems, StateProblems(m)...)
wellFormed := true
for _, c := range m.Claims {
if !name.MatchString(c.Name) {
@@ -1433,7 +1373,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
}
}
problems = append(problems, m.Build.problems(m.Module)...)
problems = append(problems, undeliveredBundles(m)...)
// **What provides the artifact store cannot be delivered through it** (novox/hq 04-ISSUES/029).
//
// Building publishes to the store, and the builder will not start without one. So a module
@@ -1483,10 +1422,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
"%s serves %q to whoever requires it, and does not provide it", m.Module, to))
}
}
// A served value may be derived for the consumer it is served to (novox/hq ADR 0201). Read
// here, where the definition is, rather than when somebody first requires it: a rule that
// would be refused at the first consumer is wrong from the moment it is written.
problems = append(problems, CheckServes(m)...)
for to, where := range m.Binds {
if !placedOrAbsolute(where) {
problems = append(problems, fmt.Sprintf(
@@ -1574,53 +1509,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
"program that reads what the mesh delivered and reconciles",
m.Module, r["id"]))
}
// **What a process replaces is something the module no longer declares** (novox/hq issue 213).
// The host keeps it running until the process is, then removes it: so it is named by the id the
// module used to give it, it is never a resource the module still declares — that would be
// applied and removed by one declaration — and only a process that stays up has anything to
// hand over to. Said here, near the author, as the host would refuse it far away.
ids := map[string]bool{}
for _, r := range m.Resources {
ids[fmt.Sprint(r["id"])] = true
}
for _, r := range m.Resources {
raw, present := r["replaces"]
if !present {
continue
}
if fmt.Sprint(r["type"]) != "process" {
problems = append(problems, fmt.Sprintf(
"%s: %v says what it replaces, and only a process does", m.Module, r["id"]))
continue
}
if once, _ := r["run-once"].(bool); once || r["schedule"] != nil {
problems = append(problems, fmt.Sprintf(
"%s: %v replaces something and runs once or on a schedule — only a process that stays "+
"up is there a moment later to hand over to", m.Module, r["id"]))
}
list, ok := raw.([]any)
if !ok {
problems = append(problems, fmt.Sprintf(
"%s: %v replaces %v; replaces is a list of the ids this module no longer declares",
m.Module, r["id"], raw))
continue
}
for _, item := range list {
id, ok := item.(string)
switch {
case !ok || strings.TrimSpace(id) == "":
problems = append(problems, fmt.Sprintf(
"%s: %v replaces %v, which is not an id", m.Module, r["id"], item))
case strings.Contains(id, "."):
problems = append(problems, fmt.Sprintf(
"%s: %v replaces %q; a process replaces only a resource of its own module, named "+
"by its own id", m.Module, r["id"], id))
case ids[id]:
problems = append(problems, fmt.Sprintf(
"%s: %v replaces %q, which this module still declares", m.Module, r["id"], id))
}
}
}
// **A module that prepares its state must have code the mesh can run** (novox/hq ADR 0135). The
// preparation is the module's own program in its preparation mode, so it is derived from the
// resource that runs that program — and a module declaring none has asked for something the mesh
@@ -1628,7 +1516,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
// quietly prepares nothing.
if m.Prepares && preparationTarget(m) == "" {
problems = append(problems, fmt.Sprintf(
"%s says it prepares its state, and declares no container or process running an artifact it built — "+
"%s says it prepares its state, and declares no container running an artifact it built — "+
"the preparation is this module's own program, so there has to be one for the mesh to "+
"run it in", m.Module))
}
@@ -1681,13 +1569,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
}
}
}
// **A scheduled step may hold this module's own containers still while it runs**
// (novox/hq ADR 0189). What the host judges is the declaration it receives — whether each
// id is a container placed on that machine; what belongs here is what only the definition
// shows: that the ids are this module's, that they are containers, and that the step is
// scheduled. A module naming a neighbour's container would be a module that can stop the
// mesh, and the manifest is where that is visible.
problems = append(problems, whileStoppedProblems(m, r, hasSchedule(r))...)
}
for name, own := range m.OwnSecrets {
if !placedOrAbsolute(own.Path) {
@@ -1849,13 +1730,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
problems = append(problems, m.unknownDirRefs()...)
problems = append(problems, m.unknownAccessRefs()...)
problems = append(problems, m.jailProblems()...)
// What a module adds to the account's environment and to the login shell, and the holder's
// placeholders for them (novox/hq ADR 0203, ADR 0204) — here, so the catalogue check refuses
// them in the words registration does.
problems = append(problems, m.environmentProblems()...)
problems = append(problems, m.shellProblems()...)
problems = append(problems, m.contributionPlaceholderProblems()...)
problems = append(problems, m.seatContributionProblems()...)
for i, r := range m.Resources {
id, _ := r["id"].(string)
@@ -1867,7 +1741,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
}
}
problems = append(problems, zoneProblems(m)...)
if len(problems) > 0 {
sort.Strings(problems)
return Manifest{}, fmt.Errorf("this manifest cannot be used:\n - %s",
@@ -2227,71 +2100,3 @@ func (o OwnSecrets) Paths() map[string]string {
// InstancesInterchangeable is the one value of a definition's `instances`: the module is the same
// on every machine, so any instance may answer for the module.
const InstancesInterchangeable = "interchangeable"
// WhileStopped is the resource key naming the containers a scheduled step holds still while it
// runs (novox/hq ADR 0189). Carried to the host unchanged, like `schedule`.
const WhileStopped = "while-stopped"
// hasSchedule is whether a resource declares a cadence, as a string.
func hasSchedule(r map[string]any) bool {
s, _ := r["schedule"].(string)
return s != ""
}
// whileStoppedProblems judges one container's maintenance window against its own definition
// (novox/hq ADR 0189).
//
// Three things the manifest is the only place to see: that the step is scheduled (a one-time
// offline job says *before* rather than *instead of* — at apply the host already has a window,
// because the declaration is applied in order and a run-once step gates what follows); that every
// id it names is **this module's own** container; and that it does not name itself.
//
// The host checks the fourth — that the container is actually placed on that machine — because
// that is a fact about the declaration and not about the definition.
func whileStoppedProblems(m Manifest, r map[string]any, scheduled bool) []string {
raw, present := r[WhileStopped]
if !present {
return nil
}
ids, ok := raw.([]any)
if !ok {
return []string{fmt.Sprintf(
"%s declares %s on %v as a %T; it is a list of this module's container ids",
m.Module, WhileStopped, r["id"], raw)}
}
var problems []string
if len(ids) > 0 && !scheduled {
problems = append(problems, fmt.Sprintf(
"%s declares %s on %v, which has no schedule. A maintenance window is for a recurring "+
"step: at apply the mesh already has one, because a run-once step gates what is "+
"declared after it (novox/hq ADR 0189)", m.Module, WhileStopped, r["id"]))
}
containers := map[string]bool{}
for _, own := range m.Resources {
if fmt.Sprint(own["type"]) == "container" {
containers[fmt.Sprint(own["id"])] = true
}
}
for _, each := range ids {
id, ok := each.(string)
if !ok {
problems = append(problems, fmt.Sprintf(
"%s declares %s on %v naming a %T; each entry is a container's id",
m.Module, WhileStopped, r["id"], each))
continue
}
if id == fmt.Sprint(r["id"]) {
problems = append(problems, fmt.Sprintf(
"%s declares %s on %v naming itself", m.Module, WhileStopped, r["id"]))
continue
}
if !containers[id] {
problems = append(problems, fmt.Sprintf(
"%s declares %s on %v naming %q, which is not a container this module declares. "+
"A step may hold still its own module's containers and nobody else's — one "+
"that could quiesce a neighbour could stop the mesh",
m.Module, WhileStopped, r["id"], id))
}
}
return problems
}
internal/catalogue/port_into.go
+5 -10
View File
@@ -31,7 +31,7 @@ import (
//
// So a module asks. `${port:8080}` is "the machine-side port you gave me for the 8080 I said I
// listen on", and the module writes that where it would otherwise have written a literal — in a
// file's content, or in a value of a container's or a process's `env`.
// file's content, or in a value of a container's `env`.
//
// **The environment is filled by the control plane, exactly as a bound value is.** A port is not
// secret — the mesh holds it in the clear — so there is nothing for the host to be the only
@@ -64,12 +64,7 @@ func portsUsed(content string) []int {
}
// portInto replaces a resource's ${port:…} placeholders with what this machine assigned — in a
// file's content, and in a value of a container's or a process's environment.
//
// **A process's environment is a container's** (novox/hq to-be 38 WP4c). A module's code moving out
// of its container becomes a process on the machine and still has to be told what the container
// was told; filled for one kind and not the other, the literal reached the process and was read as
// a port, and the modules that moved first wrote their run-once steps a 0600 env file instead.
// file's content, and in a value of a container's environment.
//
// A port the module did not say it listens on is refused, for the same reason a binding's unknown
// key is: the module is asking about something it never declared, and the answer would be a guess.
@@ -89,7 +84,7 @@ func portInto(resource map[string]any, module string, listens []Listening, with
}
resource["content"] = filled
case "container", "process":
case "container":
env, ok := resource["env"].(map[string]any)
if !ok {
return nil
@@ -111,8 +106,8 @@ func portInto(resource map[string]any, module string, listens []Listening, with
continue
}
value, err := portsFilledInto(written,
fmt.Sprintf("%s's %s %s sets %s to something that",
module, resource["type"], resource["name"], key), module, listens, with)
fmt.Sprintf("%s's container %s sets %s to something that",
module, resource["name"], key), module, listens, with)
if err != nil {
return err
}
internal/catalogue/process_env_test.go
-80
View File
@@ -1,80 +0,0 @@
package catalogue
import (
"strings"
"testing"
)
// **A process's environment is composed as a container's is** (novox/hq to-be 38 WP4c).
//
// A module's code moving out of its container becomes a process on the machine, and what its
// container's environment asked for — the port this machine gave the module, the place it put the
// module's directory — it still has to be told. Filled for a container and not for a process, the
// literal `${port:8080}` reached the process as its environment and was read as a port; the modules
// that moved first wrote their run-once steps an env file instead.
func processModule(env map[string]any) Manifest {
return Manifest{
Module: "showcase",
Listens: []Listening{{Port: 8080, From: FromMesh}},
Resources: []map[string]any{
{"id": "data", "type": "directory", "mode": "0700"},
{"id": "setup", "type": "process", "name": "showcase-setup", "run-once": true,
"run": []any{"/usr/bin/showcase", "setup"}, "env": env},
},
}
}
func TestAProcessIsToldItsPortAndItsPlaceInItsEnvironment(t *testing.T) {
env := map[string]any{
"SHOWCASE_URL": "http://127.0.0.1:${port:8080}",
"SHOWCASE_DATA": "${dir:data}/objects",
"SHOWCASE_DB": "127.0.0.1:${seat:mesh-store:5432}",
"GREETING": "hello",
}
out, err := Resolution{Node: "anchor", Modules: []Manifest{processModule(env)}}.Declaration(Rendering{
Ports: map[string]map[int]int{"showcase": {8080: 21000}},
Seats: map[string]map[int]int{"mesh-store": {5432: 6852}},
})
if err != nil {
t.Fatalf("a process asking for its port and its place does not compose: %v", err)
}
setup := fileNamed(out, "showcase.setup")
if setup == nil {
t.Fatalf("the process is not in the declaration: %v", out)
}
got, _ := setup["env"].(map[string]any)
for key, want := range map[string]string{
"SHOWCASE_URL": "http://127.0.0.1:21000",
"SHOWCASE_DATA": "/var/lib/showcase/data/objects",
"SHOWCASE_DB": "127.0.0.1:6852",
"GREETING": "hello",
} {
if got[key] != want {
t.Errorf("the process is told %s=%v, want %q", key, got[key], want)
}
}
if env["SHOWCASE_URL"] != "http://127.0.0.1:${port:8080}" {
t.Fatalf("composing for one machine edited the module's own manifest: %v", env)
}
}
// An unknown reference in a process's environment is refused as a container's is, naming the
// process and the variable — left alone, it would reach the machine as a literal.
func TestAProcessAskingAboutAnUndeclaredPortIsRefused(t *testing.T) {
env := map[string]any{"SHOWCASE_URL": "http://127.0.0.1:${port:9999}"}
_, err := Resolution{Node: "anchor", Modules: []Manifest{processModule(env)}}.Declaration(Rendering{})
if err == nil {
t.Fatal("a process was told a port its module never said it listens on")
}
for _, said := range []string{"showcase-setup", "SHOWCASE_URL", "${port:9999}", "8080"} {
if !strings.Contains(err.Error(), said) {
t.Errorf("the refusal does not say %q: %v", said, err)
}
}
env = map[string]any{"SHOWCASE_DATA": "${dir:date}/objects"}
if _, err := (Resolution{Node: "anchor", Modules: []Manifest{processModule(env)}}).Declaration(Rendering{}); err == nil ||
!strings.Contains(err.Error(), "${dir:date}") {
t.Fatalf("a process naming no directory of its module was not refused: %v", err)
}
}
internal/catalogue/published_ports_test.go
-75
View File
@@ -1,75 +0,0 @@
package catalogue
import (
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
)
// A container publishes only a port its module declares (novox/hq issue 227).
//
// **The short form is a question the mesh answers.** `"80"` means *publish what the software
// calls 80*, and the mesh fills in the machine's half from the port it assigned
// ([ADR 0038](0038)). It can only assign one for a port the module declared in `listens` — so a
// container publishing a number that appears nowhere in `listens` gets no assignment, and
// `publishedOn` falls back to the number as written. It escapes to the machine.
//
// That is how the photo module asked for port 80 on the control node, where the reverse proxy
// holds it: it declared its web endpoint at 4001, published a bare 80, and the container never
// started. Four other modules publish 80 quite safely — because they declare 80, so the mesh
// gives them a machine port for it. The difference is the declaration, not the number.
//
// A mapping written the long way is a module pinning both halves on purpose and is left alone.
func TestEveryPublishedPortIsOneItsModuleDeclares(t *testing.T) {
root := catalogueRoot(t)
entries, err := os.ReadDir(filepath.Join(root, "modules"))
if err != nil {
t.Fatal(err)
}
var escaped []string
for _, entry := range entries {
if !entry.IsDir() {
continue
}
raw, err := os.ReadFile(filepath.Join(root, "modules", entry.Name(), "module.json"))
if err != nil {
continue
}
m, err := ParseManifest(raw)
if err != nil {
// Whether every manifest parses is TestEveryCatalogueManifestParses's question.
continue
}
declared := map[int]bool{}
for _, l := range m.Listens {
declared[l.Port] = true
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "container" {
continue
}
listed, _ := r["ports"].([]any)
for _, p := range listed {
written := strings.Split(fmt.Sprint(p), "/")[0]
if strings.Contains(written, ":") {
continue // pinned by hand, both halves, on purpose
}
port, err := strconv.Atoi(strings.TrimSpace(written))
if err != nil || declared[port] {
continue
}
escaped = append(escaped, fmt.Sprintf(
"%s's %v publishes %d, and %s declares no such port — the mesh has nothing "+
"to assign, so %d reaches the machine as written",
m.Module, r["id"], port, m.Module, port))
}
}
}
if len(escaped) > 0 {
t.Fatalf("a container may publish only a port its module declares:\n - %s",
strings.Join(escaped, "\n - "))
}
}
internal/catalogue/resolve.go
-115
View File
@@ -147,10 +147,6 @@ type Resolution struct {
// dropped nor fatal to the rest. A module that is *required* by something running here is a
// different case — that set is incoherent and is refused (see checkCapabilities).
Unhostable []Unhostable
// Unheld is every dependency of this node's modules on a seat nothing here holds (novox/hq ADR
// 0207) — reported rather than refused while enforceSeatDependencies is off, so a node short of a
// holder still converges and `status` says what it is short of.
Unheld []Unheld
}
// Unhostable is one directly-assigned module the machine cannot run.
@@ -233,27 +229,6 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
local[o.Name] = true
}
}
// Which names are usable only on their provider's own machine (novox/hq ADR 0208 §3). Also a
// property of the name: a display one provider says is the machine's and another says is not
// would be pulled in for one consumer and refused for the next.
machineReach := map[string]bool{}
plainLocal := map[string]bool{}
for _, m := range catalogue {
for _, o := range m.Provides {
if o.MachineReach() {
machineReach[o.Name] = true
} else if o.At() == ScopeNode {
plainLocal[o.Name] = true
}
}
}
for want := range machineReach {
if plainLocal[want] {
problems = append(problems, fmt.Sprintf(
"the catalogue disagrees about %q: some modules provide it with the machine's reach and "+
"others without, so a requirement for it would be met differently by each", want))
}
}
for want := range brokered {
if local[want] {
problems = append(problems, fmt.Sprintf(
@@ -520,23 +495,6 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
continue
}
// Usable only on its provider's own machine, and not here: refused, never answered by
// installing a provider (novox/hq ADR 0208 §3). The display server is the machine's own role,
// gated by its graphical session; one pulled in for a window manager is the misassignment
// research 026 found. Another node's provider never counts — node scope is never brokered.
if machineReach[want] && !isModule(catalogue, want) {
reported[want] = true
if world.Unchecked {
// The first pass's refusals take a machine off the network; the second says it.
continue
}
problems = append(problems, fmt.Sprintf(
"%q is wanted by %s and is usable only on the machine that provides it, and nothing "+
"assigned to %s does — %s", want, because[want], node.Name,
machineReachRemedy(catalogue, want, node.Name)))
continue
}
candidates := offers[want]
switch len(candidates) {
case 0:
@@ -670,22 +628,6 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
problems = append(problems, checkResources(resolution.Modules)...)
resolution.Claims = claims
// Judged over the closure — what this node will actually run — so a holder pulled in by a
// requirement counts, and the holders' mutual dependence resolves (novox/hq ADR 0207 §3).
// Reported until the switch; refused after it, though never in the first pass, whose refusals
// make a machine vanish from the network rather than report anything.
resolution.Unheld = UnheldDependencies(catalogue, node.Name, resolution.Modules, nil)
// Only a dependency some module in the catalogue could meet is refused: one with no possible
// holder has no remedy to name, and stays a report in `status` (novox/hq ADR 0207 §4, read with
// the assign rule above it).
if enforceSeatDependencies && !world.Unchecked {
for _, u := range resolution.Unheld {
if len(u.Holders) > 0 {
problems = append(problems, u.String())
}
}
}
if len(problems) > 0 {
sort.Strings(problems)
return Resolution{}, &Refusal{Problems: problems}
@@ -862,28 +804,6 @@ func checkResources(modules []Manifest) []string {
// does not exist is the manifest's own problem, refused where it was made.
dirs := dirsFor(m, Rendering{})
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "user" {
// **An account is shared; what it is set to is not.** Several modules may need one
// login: the shell's module sets its shell, the container runtime's puts it in the
// `docker` group. The host only ever adds groups — it never takes the account out of
// one, not even when the resource that named it is undeclared — so groups from
// several modules cannot contradict each other and are not owned. A shell or a home
// is one value, and two modules setting it would each be undone by the other's
// apply: each stays one module's per node, and two are refused naming both.
name, _ := r["name"].(string)
for _, field := range []string{"shell", "home"} {
if v, ok := r[field].(string); !ok || v == "" || name == "" {
continue
}
key := "user " + field + " " + name
if other, taken := owner[key]; taken && other != m.Module {
problems = append(problems, fmt.Sprintf(
"%s and %s both set the %s of the user %q", other, m.Module, field, name))
}
owner[key] = m.Module
}
continue
}
for _, field := range []string{"path", "unit", "name", "package"} {
value, ok := r[field].(string)
if !ok || value == "" {
@@ -1099,38 +1019,3 @@ func eachLocal(needs []Needed, catalogue map[string]Manifest, n Needed) []Needed
}
return needs
}
// machineReachRemedy names what would meet a requirement with the machine's reach: every module in
// the catalogue that provides it, each with the node seats it holds — for a display, the holders of
// node-display-server (novox/hq ADR 0208 §3), named by the seat because that is the role being
// asked for, without this code knowing which seat any provision belongs to.
func machineReachRemedy(catalogue map[string]Manifest, want, node string) string {
var named []string
for _, name := range sortedKeys(catalogue) {
m := catalogue[name]
provides := false
for _, o := range m.Provides {
if o.Name == want {
provides = true
}
}
if !provides {
continue
}
var held []string
for _, c := range m.Claims {
if c.At() == ScopeNode {
held = append(held, c.Name)
}
}
if len(held) > 0 {
named = append(named, fmt.Sprintf("%s (holds %s)", name, strings.Join(held, ", ")))
} else {
named = append(named, name)
}
}
if len(named) == 0 {
return "and nothing in the catalogue provides it"
}
return fmt.Sprintf("assign one to %s: %s", node, strings.Join(named, "; "))
}
internal/catalogue/resolver_manifests_test.go
+23 -44
View File
@@ -48,12 +48,9 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
}
for _, want := range []string{
"\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n",
// The private address and loopback, never a LAN's (novox/hq ADR 0194): a device that is not a
// member cannot reach what the mesh's names point at.
"\nlisten-address=127.0.0.1\n", "\nlisten-address=${machine:address}\n", "\nbind-dynamic\n",
// No hosts file and no operator's files: the mesh's resolver answers every node (ADR 0199).
"\nno-hosts\n",
"\nconf-file=" + m.Facts["zones"].Path + "\n",
// Loopback is the mesh-wide setting's default; a machine answering its own LAN adds its
// address there (novox/hq issue 198).
"\nlisten-address=${setting:listen-addresses}\n", "\nlisten-address=${machine:address}\n", "\nbind-dynamic\n",
"\ndomain-needed\n", "\nbogus-priv\n",
"\nconf-file=" + m.Facts["node-zones"].Path + "\n",
} {
@@ -76,15 +73,7 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
t.Errorf("the resolver listens on %s", taken)
}
}
// Never a directory or a file the operator keeps: a line written for one machine's programs would
// become an answer for every node (ADR 0199).
for _, never := range []string{"conf-dir=", "addn-hosts=", "listen-address=${setting:"} {
if strings.Contains(config, never) {
t.Errorf("the mesh's resolver still reads or listens on %q", never)
}
}
// And the file that decides what the machine asks names the mesh's resolver first, by address,
// and a public one second, asked only when the first is silent (ADR 0196).
// And the file that decides what the machine asks names it there, alone.
var resolv string
for _, r := range catalogueManifest(t, "resolv-conf").Resources {
if r["path"] == "/etc/resolv.conf" {
@@ -97,16 +86,13 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
nameservers = append(nameservers, strings.TrimPrefix(line, "nameserver "))
}
}
if len(nameservers) != 2 || nameservers[0] != "${bound:wildcard-resolution:address}" || nameservers[1] != "1.1.1.1" {
t.Errorf("resolv.conf names %v; the mesh's resolver by address first, a public one second", nameservers)
if len(nameservers) != 1 || nameservers[0] != "127.0.0.1" {
t.Errorf("resolv.conf names %v; the predecessor's names the mesh's resolver alone at 127.0.0.1", nameservers)
}
if !strings.Contains(resolv, "\noptions timeout:1 attempts:1") {
t.Errorf("the fallback is not reached after one short attempt:\n%s", resolv)
}
// The split-DNS alternative points at the same resolver, or a machine that keeps
// The split-DNS alternative points at the same address, or a machine that keeps
// systemd-resolved in charge would route the mesh's suffix to nothing.
for _, r := range catalogueManifest(t, "resolved-split-dns").Resources {
if content, _ := r["content"].(string); content != "" && !strings.Contains(content, "DNS=${bound:wildcard-resolution:address}\n") {
if content, _ := r["content"].(string); content != "" && !strings.Contains(content, "DNS=127.0.0.1\n") {
t.Errorf("resolved-split-dns does not point at the resolver's address:\n%s", content)
}
}
@@ -114,8 +100,7 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
// The resolver and what points the machine at it compose on one machine, and what arrives is the
// mesh's account of every machine as a wildcard, the suffix kept local, the daemon restarting on
// that file, the machine pointed at the resolver by address, and the runtime given no resolver of
// its own but kept running across a restart (ADR 0196).
// that file, and the runtime pointed at this machine's own address.
func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
got, err := Resolve(resolverShelf(t), []string{"dnsmasq", "resolv-conf"},
Node{Name: "anchor", At: "anchor.internal"}, World{})
@@ -130,7 +115,6 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
// issue 111) — the resolver's zones read only the second, and in this scenario the two
// happen to be the same map, since nothing routed is part of it.
Names: twoMachines, Machines: twoMachines, Suffix: "internal",
Zones: []ZoneAt{{Zone: "incus", Address: "10.42.0.2", Port: 5353}},
Needed: map[string]map[string]string{"dnsmasq": {"broker": "sealed"}},
Settings: SettingsBy{"dnsmasq": {{From: "the mesh", Values: map[string]any{"listen-addresses": "127.0.0.1"}}}},
})
@@ -160,29 +144,24 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
for _, id := range service["restart-on"].([]any) {
reflects[id.(string)] = true
}
if !reflects["dnsmasq.config"] || !reflects["dnsmasq.fact-node-zones"] || !reflects["dnsmasq.fact-zones"] {
t.Errorf("the daemon does not restart on its configuration, the machines and the zones: %v", service["restart-on"])
}
if z, _ := ids["dnsmasq.fact-zones"]["content"].(string); !strings.Contains(z, "server=/incus/10.42.0.2#5353\n") {
t.Errorf("the resolver was not told to forward the zone to its answerer:\n%s", z)
if !reflects["dnsmasq.config"] || !reflects["dnsmasq.fact-node-zones"] {
t.Errorf("the daemon does not restart on its configuration and the machines file both: %v", service["restart-on"])
}
// The runtime's own file, written into (novox/hq ADR 0102) with one key, by what decides how the
// machine resolves: a restart keeps every container running. No `dns` — a container copies its
// machine's resolvers (ADR 0196), and the mesh's resolver is not written into the runtime twice.
if ids["dnsmasq.runtime-dns"] != nil {
t.Errorf("the resolver still writes the runtime's dns: %v", ids["dnsmasq.runtime-dns"])
}
runtime := ids["resolv-conf.runtime-config"]
// The runtime's own file, written into (novox/hq ADR 0102) with the keys this module states:
// where containers resolve, and that a restart keeps them running — because the runtime reads
// `dns` only when it starts, and the one restart that needs is the operator's (issue 110).
runtime := ids["dnsmasq.runtime-dns"]
if runtime == nil || runtime["path"] != "/etc/docker/daemon.json" || runtime["into"] != "json" {
t.Fatalf("live-restore is not written into the runtime's file: %v", runtime)
t.Fatalf("the runtime's dns is not written into its file: %v", runtime)
}
var keys map[string]any
if err := json.Unmarshal([]byte(runtime["content"].(string)), &keys); err != nil {
t.Fatalf("the runtime's keys are not JSON: %v", err)
}
if len(keys) != 1 || keys["live-restore"] != true {
t.Errorf("the runtime is given %v; live-restore and nothing else", keys)
dns, _ := keys["dns"].([]any)
if len(keys) != 2 || len(dns) != 1 || dns[0] != "10.42.0.1" || keys["live-restore"] != true {
t.Errorf("the runtime is given %v; containers resolve at this machine's own private-network address, a restart keeps them, and nothing else is written", keys)
}
// The runtime is reloaded when that file changes, and never restarted: a restart stops every
// container on the machine (ADR 0102), and a reload is what turns live-restore on.
@@ -192,10 +171,10 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
continue
}
if _, restarts := r["restart-on"]; restarts {
t.Errorf("the runtime is ordered restarted, which stops every container (ADR 0102): %v", r)
t.Errorf("the resolver orders the runtime restarted, which stops every container (ADR 0102): %v", r)
}
for _, on := range asStrings(r["reload-on"]) {
if on == "resolv-conf.runtime-config" {
if on == "dnsmasq.runtime-dns" {
reloaded = true
}
}
@@ -205,8 +184,8 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
}
resolv := ids["resolv-conf.resolv"]
if resolv == nil || !strings.Contains(resolv["content"].(string), "\nnameserver 10.42.0.1\nnameserver 1.1.1.1\n") {
t.Fatalf("the machine is not pointed at the resolver by address, with the public fallback: %v", resolv)
if resolv == nil || !strings.Contains(resolv["content"].(string), "\nnameserver 127.0.0.1\n") {
t.Fatalf("the machine is not pointed at the resolver: %v", resolv)
}
}
internal/catalogue/roster.go
-27
View File
@@ -61,16 +61,6 @@ type rosterView struct {
Suffix string
Names []rosterEntry
Machines []rosterEntry
// Zones is every zone a module in the mesh answers itself, with where its answerer is (novox/hq
// ADR 0199) — what the mesh's resolver forwards. Ordered by zone.
Zones []rosterZone
}
// rosterZone is one zone as a template sees it: the zone, and the address and port answering it.
type rosterZone struct {
Zone string
Address string
Port int
}
// rosterEntry is one machine as a template sees it: its bare name, its full mesh name, its address,
@@ -90,12 +80,6 @@ type rosterEntry struct {
// `machines` is only the machines — the two must not be confused (novox/hq 04-ISSUES/111), so both
// are given and the template chooses.
func FactsInto(m Manifest, r Resolution, every, machines, accounts map[string]string, suffix string) ([]map[string]any, error) {
return FactsWithZonesInto(m, r, every, machines, accounts, suffix, nil)
}
// FactsWithZonesInto is FactsInto with the mesh's zones in the view, for a template that ranges them.
func FactsWithZonesInto(m Manifest, r Resolution, every, machines, accounts map[string]string, suffix string,
zones []ZoneAt) ([]map[string]any, error) {
if len(m.Facts) == 0 {
return nil, nil
}
@@ -110,7 +94,6 @@ func FactsWithZonesInto(m Manifest, r Resolution, every, machines, accounts map[
Suffix: strings.TrimPrefix(suffixOr(suffix), "."),
Names: entriesFrom(every, accounts, suffix),
Machines: entriesFrom(machines, accounts, suffix),
Zones: zonesFrom(zones),
}
out := make([]map[string]any, 0, len(names))
@@ -240,13 +223,3 @@ func sortedNames(addresses map[string]string) []string {
sort.Strings(out)
return out
}
// zonesFrom is the zones a template ranges, ordered by zone so two renderings of one mesh are one file.
func zonesFrom(zones []ZoneAt) []rosterZone {
out := make([]rosterZone, 0, len(zones))
for _, z := range zones {
out = append(out, rosterZone{Zone: z.Zone, Address: z.Address, Port: z.Port})
}
sort.Slice(out, func(i, j int) bool { return out[i].Zone < out[j].Zone })
return out
}
internal/catalogue/runtime_test.go
-68
View File
@@ -389,71 +389,3 @@ func TestARuntimeCompiledToABinaryRunsItself(t *testing.T) {
t.Errorf("the Go runtime is not told what to serve or whose it is: %v %v", env, process["user"])
}
}
// novox/hq 04-ISSUES/216: a bundle nothing loads, runs or unpacks is refused at registration; saying
// `loads`, listing `tools`, or a resource naming it admits it.
func TestABundleNothingDeliversIsRefused(t *testing.T) {
base := func() Manifest {
return Manifest{Module: "baserow", Version: "1", Build: &Build{Artifacts: []Artifact{
{Name: "tools", Kind: ArtifactBundle, Language: "typescript", Entrypoints: []string{"tools/index.js"}}}}}
}
if p := undeliveredBundles(base()); len(p) != 1 || !strings.Contains(p[0], "never reach a machine") {
t.Fatalf("a bundle nothing delivers was admitted: %v", p)
}
loads := base()
loads.Build.Artifacts[0].Loads = []string{"tools/index.js"}
tools := base()
tools.Tools = []string{"baserow_list_rows"}
run := base()
run.Resources = []map[string]any{{"id": "daemon", "type": "process", "artifact": "tools", "run": []any{"node", "tools/index.js"}}}
runtime := base()
runtime.Module = RuntimeModule
for name, m := range map[string]Manifest{"loads": loads, "tools": tools, "a process": run, "the runtime": runtime} {
if p := undeliveredBundles(m); len(p) != 0 {
t.Errorf("a bundle delivered by %s was refused: %v", name, p)
}
}
}
// novox/hq ADR 0193: a Go tools bundle is served — its binary is what the runtime starts, delivered
// like any tools bundle, named to the runtime where a TypeScript bundle names its launcher.
func TestAGoToolsBundleIsServedByItsBinary(t *testing.T) {
with := Rendering{ArtifactStore: "anchor.internal:5101",
Needed: map[string]map[string]string{RuntimeModule: {"broker": "sealed-credential"}}}
lamp := Manifest{Module: "lamp", Version: "1", Tools: []string{"on"},
Build: &Build{Artifacts: []Artifact{{Name: "tools", Kind: ArtifactBundle, Language: "go",
System: "arch", From: "cmd/lamp-tools"}}}}
if p := lamp.Build.problems("lamp"); len(p) != 0 {
t.Fatalf("a Go tools bundle was refused: %v", p)
}
lamp, err := lamp.Resolve([]Built{{Name: "tools", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "lamp/tools/blobs/" + bundleDigest, Digest: bundleDigest}})
if err != nil {
t.Fatal(err)
}
if fmt.Sprint(lamp.Bundles[0].Loads) != "[lamp-tools]" {
t.Fatalf("the runtime loads %v from a Go bundle, want its binary", lamp.Bundles[0].Loads)
}
out, err := Resolution{Node: "anchor", Account: "ops", Modules: []Manifest{lamp, theRuntime(t)}}.Declaration(with)
if err != nil {
t.Fatal(err)
}
if fileNamed(out, "lamp."+BundleID("tools")) == nil {
t.Errorf("the Go bundle is not delivered: %v", ids(out))
}
env := fileNamed(out, RuntimeModule+"."+RuntimeProcessID())["env"].(map[string]string)
if env[RuntimeToolModules] != "lamp="+BundlePath("lamp", "tools")+"/lamp-tools" {
t.Errorf("the runtime is told %q, want the binary", env[RuntimeToolModules])
}
// An artifact may say it explicitly; naming anything but the binary is refused.
said := Manifest{Module: "lamp", Version: "1", Build: &Build{Artifacts: []Artifact{{Name: "tools",
Kind: ArtifactBundle, Language: "go", System: "arch", Binary: "lamp", Loads: []string{"lamp"}}}}}
if p := said.Build.problems("lamp"); len(p) != 0 {
t.Errorf("loads naming the binary was refused: %v", p)
}
said.Build.Artifacts[0].Loads = []string{"tools/index.js"}
if p := said.Build.problems("lamp"); len(p) == 0 {
t.Error("a Go bundle loading a file it does not contain was admitted")
}
}
internal/catalogue/seat_contributions.go
-226
View File
@@ -1,226 +0,0 @@
package catalogue
import (
"fmt"
"regexp"
"strings"
)
// A module contributes to a seat it does not hold (novox/hq ADR 0212).
//
// ADR 0210 made a tool's configuration its seat holder's, and every other module's way in a
// contribution to the seat. The environment, the shell's slots and the power moments each became a
// field of their own; this is the general form, so a new seat that takes contributions is a row in
// the seat table rather than a change to the manifest: a contribution names a seat, a kind that
// seat receives, and text in the tool's own grammar, which the controller never reads.
// HotkeysSeat is the machine's hotkey daemon (novox/hq ADR 0212 §5).
const HotkeysSeat = "node-hotkeys"
// MessageBusSeat is the machine's D-Bus (novox/hq ADR 0215).
const MessageBusSeat = "node-message-bus"
// BackupSeat is the machine's backups (novox/hq ADR 0214, to-be 43).
const BackupSeat = "node-backup"
// storeProvisions are the provisions whose provider keeps its consumers' data (novox/hq ADR 0214):
// a module providing one must say how to back it up, or the data the mesh hands out is the data it
// cannot restore — issue 241's seven databases. A provision that holds nothing worth keeping (a
// route, a cache, a name) is not here; adding one is adding a store.
var storeProvisions = map[string]bool{
"postgres-database": true,
"mssql-database": true,
"mongodb-database": true,
"s3-bucket": true,
"influxdb-api": true,
"secret": true,
}
// CheckBackup is a store provider that contributes no backup (novox/hq ADR 0214, "How it is
// checked").
//
// **The catalogue check's, not parsing's.** A manifest already registered and running was written
// before the rule; refusing it on read would make the controller refuse the very providers whose
// data the rule protects. New definitions meet it in the catalogue check, where they are written.
func CheckBackup(m Manifest) []string {
backs := false
for _, c := range m.Contributions {
if s, known := SeatNamed(c.Seat); known && s.Name == BackupSeat && c.Kind == "backup" {
backs = true
}
}
if backs {
return nil
}
var problems []string
for _, o := range m.Provides {
if storeProvisions[o.Name] {
problems = append(problems, fmt.Sprintf(
"%s provides %s, which keeps its consumers' data, and contributes no backup to %s; a "+
"store says how its data is copied (novox/hq ADR 0214)", m.Module, o.Name, BackupSeat))
}
}
return problems
}
// SeatContribution is one piece of configuration a module gives a seat's holder to place.
type SeatContribution struct {
// Seat is the seat whose holder places it.
Seat string `json:"seat"`
// Kind is which of the seat's receivable kinds it is.
Kind string `json:"kind"`
// Content is the text, in the tool's own grammar. Never interpreted.
Content string `json:"content"`
}
// ofContribution is where a holder places a kind: ${contribution:<seat>:<kind>}. Loose inside the
// braces, so a misspelt seat or kind is found and refused rather than written out as text.
var ofContribution = regexp.MustCompile(`\$\{contribution:([^}]*)\}`)
// receivable is what a seat receives of a kind, with the seat's canonical name; false when the seat
// is unknown or does not receive it.
func receivable(seat, kind string) (Seat, Receivable, bool) {
s, known := SeatNamed(seat)
if !known {
return Seat{}, Receivable{}, false
}
for _, r := range s.Receives {
if r.Kind == kind {
return s, r, true
}
}
return s, Receivable{}, false
}
// kindsOf names a seat's receivable kinds for a refusal.
func kindsOf(s Seat) string {
if len(s.Receives) == 0 {
return "it receives no contributions"
}
var kinds []string
for _, r := range s.Receives {
kinds = append(kinds, r.Kind)
}
return "it receives " + strings.Join(kinds, ", ")
}
// seatContributionProblems is what is wrong with this module's contributions, from the manifest
// alone (novox/hq ADR 0212 §2).
func (m Manifest) seatContributionProblems() []string {
var problems []string
for i, c := range m.Contributions {
s, r, ok := receivable(c.Seat, c.Kind)
// A directory a contribution names must be one of this module's own, here rather than on the
// machine — where a `${dir:x}` nobody declared would reach the holder as the literal text.
if ok && r.Dirs {
declared := map[string]string{}
for _, res := range m.Resources {
if fmt.Sprint(res["type"]) == "directory" {
declared[fmt.Sprint(res["id"])] = ""
}
}
if _, err := dirFill(c.Content, declared, m.Module); err != nil {
problems = append(problems, fmt.Sprintf("%s's contribution %d: %v", m.Module, i+1, err))
}
}
switch {
case s.Name == "":
problems = append(problems, fmt.Sprintf(
"%s's contribution %d is to the seat %q, which the mesh does not define", m.Module, i+1, c.Seat))
case !ok:
problems = append(problems, fmt.Sprintf(
"%s's contribution %d to %s is of the kind %q; %s (novox/hq ADR 0212)",
m.Module, i+1, s.Name, c.Kind, kindsOf(s)))
}
if strings.TrimSpace(c.Content) == "" {
problems = append(problems, fmt.Sprintf("%s's contribution %d has no content", m.Module, i+1))
}
}
return problems
}
// seatPlaceholderProblems is what is wrong with one resource's ${contribution:…}: placed only in a
// file's content, naming a seat and a kind it receives, and only by a module that claims that seat
// — another would be a second writer of a file there is one of (novox/hq ADR 0212 §3).
func seatPlaceholderProblems(m Manifest, r map[string]any) []string {
var problems []string
for _, field := range sortedKeys(r) {
v, ok := r[field].(string)
if !ok {
continue
}
found := ofContribution.FindAllStringSubmatch(v, -1)
if len(found) == 0 {
continue
}
if field != "content" {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s in its %s; contributions are placed only in a file's content",
m.Module, r["id"], found[0][0], field))
continue
}
for _, f := range found {
seat, kind, two := strings.Cut(f[1], ":")
s, _, ok := receivable(seat, kind)
if !two || !ok {
detail := "the mesh defines no seat " + fmt.Sprintf("%q", seat)
if s.Name != "" {
detail = s.Name + ": " + kindsOf(s)
}
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s; a contribution is ${contribution:<seat>:<kind>} (%s)",
m.Module, r["id"], f[0], detail))
continue
}
if !m.ClaimsSeat(s.Name) {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s and %s does not claim %s; every module's contributions to a "+
"seat are placed by its holder alone (novox/hq ADR 0212)",
m.Module, r["id"], f[0], m.Module, s.Name))
}
}
}
return problems
}
// seatContributions is every module's contribution of one kind to one seat (novox/hq ADR 0212 §3):
// in module order, each module's pieces in the order it declared them, each module's preceded by a
// comment line naming it in the tool's grammar, and empty when nothing is contributed. A kind that
// takes directories has each contributor's `${dir:<id>}` filled with where that module's directory
// is on this machine (novox/hq to-be 43).
func seatContributions(modules []Manifest, seat, kind string, with Rendering) (string, error) {
s, r, ok := receivable(seat, kind)
if !ok {
return "", nil
}
var failed error
var b strings.Builder
for _, m := range inModuleOrder(modules) {
named := false
for _, c := range m.Contributions {
if c.Kind != kind {
continue
}
if cs, known := SeatNamed(c.Seat); !known || cs.Name != s.Name {
continue
}
if !named {
fmt.Fprintf(&b, "%s %s\n", r.Comment, m.Module)
named = true
}
content := c.Content
if r.Dirs {
filled, err := dirFill(content, dirsFor(m, with), m.Module)
if err != nil && failed == nil {
failed = err
}
content = filled
}
b.WriteString(content)
if !strings.HasSuffix(content, "\n") {
b.WriteString("\n")
}
}
}
return b.String(), failed
}
internal/catalogue/seat_contributions_test.go
-117
View File
@@ -1,117 +0,0 @@
package catalogue
import (
"reflect"
"strings"
"testing"
)
// Defends novox/hq ADR 0212: a seat says what it receives, its holder places it, and a contribution
// depends on the seat.
func hotkeysHolder() Manifest {
return Manifest{Module: "triggerhappy", Claims: []Claim{{Name: HotkeysSeat}}, Resources: []map[string]any{
{"id": "triggers", "type": "file", "path": "/etc/triggerhappy/triggers.d/mesh.conf",
"content": "# the mesh's triggers\n${contribution:node-hotkeys:trigger}"},
}}
}
func TestAContributionReachesTheHoldersFileInModuleOrderNamedByModule(t *testing.T) {
laptop := Manifest{Module: "laptop", Contributions: []SeatContribution{
{Seat: HotkeysSeat, Kind: "trigger", Content: "KEY_PROG1 1 play ${machine:account-home}"},
{Seat: HotkeysSeat, Kind: "trigger", Content: "KEY_F21 1 touchpad\n"},
}}
another := Manifest{Module: "another", Contributions: []SeatContribution{
{Seat: HotkeysSeat, Kind: "trigger", Content: "KEY_MUTE 1 mute ${shell:zsh:first}"},
}}
unrelated := Manifest{Module: "bar", Contributions: []SeatContribution{
{Seat: DisplaySessionSeat, Kind: "config", Content: "bar { }"},
}}
r := Resolution{Node: "laptop", Account: "op", Modules: []Manifest{hotkeysHolder(), laptop, another, unrelated}}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
var file map[string]any
for _, res := range out {
if res["id"] == "triggerhappy.triggers" {
file = res
}
}
if file == nil {
t.Fatalf("the holder's file was not composed: %v", out)
}
// Module order; each module named once; text never read, so neither ${machine:…} nor ${shell:…}
// inside a contribution is filled.
want := "# the mesh's triggers\n" +
"# another\nKEY_MUTE 1 mute ${shell:zsh:first}\n" +
"# laptop\nKEY_PROG1 1 play ${machine:account-home}\nKEY_F21 1 touchpad\n"
if got := file["content"]; got != want {
t.Fatalf("the holder's file is\n%s\nnot\n%s", got, want)
}
}
func TestNoContributionPlacesNothing(t *testing.T) {
r := Resolution{Node: "laptop", Account: "op", Modules: []Manifest{hotkeysHolder()}}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
for _, res := range out {
if res["id"] == "triggerhappy.triggers" && res["content"] != "# the mesh's triggers\n" {
t.Fatalf("an empty kind left %q", res["content"])
}
}
}
func TestAContributionToASeatThatDoesNotReceiveItIsRefused(t *testing.T) {
cases := map[string]string{
`{"seat":"node-hotkeys","kind":"config","content":"x"}`: "it receives trigger",
`{"seat":"node-nothing","kind":"trigger","content":"x"}`: "the mesh does not define",
`{"seat":"node-hotkeys","kind":"trigger","content":" "}`: "has no content",
`{"seat":"node-display-session","kind":"trigger","content":"x"}`: "it receives config",
}
for c, want := range cases {
raw := `{"module":"laptop","contributions":[` + c + `]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("%s: accepted, or refused without %q: %v", c, want, err)
}
}
}
func TestAContributionPlaceholderOutsideTheHolderIsRefused(t *testing.T) {
raw := `{"module":"laptop","resources":[{"id":"t","type":"file","path":"/etc/t","content":"${contribution:node-hotkeys:trigger}"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "laptop does not claim node-hotkeys") {
t.Errorf("a placeholder outside the holder was accepted: %v", err)
}
raw = `{"module":"triggerhappy","claims":[{"name":"node-hotkeys"}],"resources":[{"id":"t","type":"file","path":"/etc/t","content":"${contribution:node-hotkeys:keys}"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "it receives trigger") {
t.Errorf("a placeholder for a kind the seat does not receive was accepted: %v", err)
}
}
func TestAContributionDependsOnItsSeat(t *testing.T) {
m := Manifest{Module: "laptop", Contributions: []SeatContribution{
{Seat: HotkeysSeat, Kind: "trigger", Content: "x"},
{Seat: DisplaySessionSeat, Kind: "config", Content: "y"},
}}
if got, want := DependsOn(m), []string{DisplaySessionSeat, HotkeysSeat}; !reflect.DeepEqual(got, want) {
t.Errorf("depends on %v, want %v", got, want)
}
catalogue := map[string]Manifest{"laptop": m, "triggerhappy": hotkeysHolder()}
if _, err := AssignRefusal(catalogue, "laptop", nil, []string{"laptop"}); err == nil ||
!strings.Contains(err.Error(), HotkeysSeat) {
t.Errorf("a contributor without the holder was accepted: %v", err)
}
}
func TestTheHotkeysSeatIsTheMeshsAndReceivesTriggers(t *testing.T) {
s, ok := SeatNamed(HotkeysSeat)
if !ok || s.Scope != ScopeNode || len(s.Receives) != 1 || s.Receives[0].Kind != "trigger" {
t.Fatalf("%+v %v", s, ok)
}
d, _ := SeatNamed(DisplaySessionSeat)
if len(d.Receives) != 1 || d.Receives[0].Kind != "config" {
t.Fatalf("the display session receives %+v", d.Receives)
}
}
internal/catalogue/seat_dependencies.go
-349
View File
@@ -1,349 +0,0 @@
package catalogue
import (
"fmt"
"sort"
"strings"
)
// A module depends on the node seats that apply its resources (novox/hq ADR 0207), and on the
// seats it contributes to (novox/hq ADR 0210).
//
// Some of what a module declares is applied through software on the machine that is itself a
// module: a service through the service manager, a package through the package manager, a container
// through the container runtime. A *capability* only says that software is installed; it does not
// say that a module of the mesh holds the role and answers for it. So the dependency is derived from
// the resources — never stated in a manifest, because a module that adds a service and forgets a
// field would pass — and is met when some module assigned to the same node holds the seat.
// The three seats that apply resources (novox/hq ADR 0207 §1). Named once, because the derivation,
// the seed and the messages all turn on these strings.
const (
ServiceManagerSeat = "node-service-manager"
PackageManagerSeat = "node-package-manager"
ContainerRuntimeSeat = "node-container-runtime"
)
// appliedThrough is which seat applies a resource of each type. **Only these three**: ADR 0207
// names them and no more. A process is supervised by the host itself, a file, a directory, an
// archive, a user or an action is the host's own act, and a module's other kinds reach the machine
// without a role in between — adding one here is a decision, not a refinement.
var appliedThrough = map[string]string{
"service": ServiceManagerSeat,
"package": PackageManagerSeat,
"container": ContainerRuntimeSeat,
}
// enforceSeatDependencies is the one-line switch ADR 0207 §4 names. Off, an unmet dependency at
// composition is *reported* — in the resolution, in `status`, once in the log — and the node still
// resolves; on, it is refused like any unresolved requirement. Off until `status` reports none,
// which is when the three holders are assigned to every node: switching it before then would stop
// every machine lacking one from being sent anything at all.
//
// Switched on 2026-10-04, when `status` first reported no unmet dependency on any node: systemd,
// pacman and docker were assigned to all four machines that afternoon (novox/hq to-be 42).
//
// A variable rather than a constant only so a test can hold both behaviours; nothing else sets it.
var enforceSeatDependencies = true
// EnforcingSeatDependencies sets the switch and returns what puts it back. For tests in other
// packages that hold the behaviour from before the switch; nothing else calls it.
func EnforcingSeatDependencies(on bool) (restore func()) {
was := enforceSeatDependencies
enforceSeatDependencies = on
return func() { enforceSeatDependencies = was }
}
// foundationModules are the pieces genesis lays before any module exists (novox/hq ADR 0207 §5):
// the host and the private network. Registered as modules so they can be assigned, but what they
// declare is the installation's, not a module's, so it is never judged. The third piece, the
// bootstrap container runtime, is not a module at all: its package and service are in the genesis
// bundle the host applies itself, and never pass through a resolution here.
//
// The private network's module is overlay.Name, written out because the overlay package composes
// on top of this one; its resources are computed, which exempts it by the rule below as well.
var foundationModules = map[string]bool{
"mesh-host": true,
"mesh-wireguard": true,
}
// isFoundation is whether a module's declarations are the foundation's rather than its own. A
// module whose resources are computed is the mesh's by construction — the private network's peer
// list and its tools are the controller's, written per node — so it counts whatever its name.
func isFoundation(m Manifest) bool {
return foundationModules[m.Module] || m.Computed != ""
}
// DependsOn is every seat a module needs held on its node, derived from the resource types it
// declares itself (novox/hq ADR 0207 §2), sorted.
//
// **The module's own `resources` only.** What the controller composes around a module — its
// filter, jails, certificates, kept files, bundles, the guard — is the mesh's, put there because the
// module is assigned, and depending on it would make the module answer for the mesh's choices.
func DependsOn(m Manifest) []string {
if isFoundation(m) {
return nil
}
seen := map[string]bool{}
for _, r := range m.Resources {
if seat, applied := appliedThrough[fmt.Sprint(r["type"])]; applied {
seen[seat] = true
}
}
for _, seat := range contributedTo(m) {
seen[seat] = true
}
out := make([]string, 0, len(seen))
for s := range seen {
out = append(out, s)
}
sort.Strings(out)
return out
}
// contributedTo is every seat a module contributes to (novox/hq ADR 0210 §3): a contribution is
// configuration only the seat's holder applies, so it is a dependency on that seat exactly as a
// resource is on the seat that applies it. The environment goes to node-environment's holder
// (ADR 0203); shell code to the holder that places it for its target — the login shell's for a
// shell, the display server's for the session's start and resources (ADR 0204, ADR 0208 §4).
func contributedTo(m Manifest) []string {
var out []string
if e := m.Environment; e != nil && (len(e.Variables) > 0 || len(e.Path) > 0) {
out = append(out, EnvironmentSeat)
}
for _, c := range m.Shell {
out = append(out, placerOf(c.For))
}
// Any other seat's contribution (novox/hq ADR 0212 §4), by the seat's canonical name; one the
// mesh does not define is refused at registration and depends on nothing here.
for _, c := range m.Contributions {
if s, known := SeatNamed(c.Seat); known {
out = append(out, s.Name)
}
}
return out
}
// claimsSeat is whether a module claims a node seat, by its current name or one it used to have
// (ADR 0122), so a rename leaves the dependency met.
func claimsSeat(m Manifest, seat string) bool {
for _, c := range m.Claims {
if c.At() != ScopeNode {
continue
}
name := c.Name
if s, known := SeatNamed(name); known {
name = s.Name
}
if name == seat {
return true
}
}
return false
}
// PossibleHolders is every module in the catalogue that claims a seat at node scope — what a
// refusal names as the remedy.
func PossibleHolders(catalogue map[string]Manifest, seat string) []string {
var out []string
for name, m := range catalogue {
if claimsSeat(m, seat) {
out = append(out, name)
}
}
sort.Strings(out)
return out
}
// Unheld is one dependency of one module on a node that nothing on that node holds.
type Unheld struct {
Node string `json:"node"`
Module string `json:"module"`
Seat string `json:"seat"`
// Holders are the modules in the catalogue that could hold the seat: assigning one meets it.
Holders []string `json:"holders"`
}
// String is the line a refusal and a report both say, so the two never drift.
func (u Unheld) String() string {
remedy := "and no module in the catalogue claims it yet"
if len(u.Holders) > 0 {
remedy = "— assign one that holds it: " + strings.Join(u.Holders, ", ")
}
return fmt.Sprintf("%s on %s depends on %s, which nothing on %s holds (novox/hq ADR 0207) %s",
u.Module, u.Node, u.Seat, u.Node, remedy)
}
// UnheldDependencies is every dependency of the modules in `judged` that the node's whole set
// leaves unmet (novox/hq ADR 0207 §3).
//
// **Judged over the whole set, never one module at a time.** The holders depend on each other:
// the service manager's own package needs the package manager, and the package manager's timer
// needs the service manager. Asked one by one, neither could ever be first; asked of the set, the
// two assigned together meet each other. A module holding a seat it depends on meets its own
// dependency. `judged` nil judges every module of the set.
func UnheldDependencies(catalogue map[string]Manifest, node string, set []Manifest, judged map[string]bool) []Unheld {
held := map[string]bool{}
for _, m := range set {
for _, seat := range nodeSeatsClaimed(m) {
held[seat] = true
}
}
var out []Unheld
for _, m := range set {
if judged != nil && !judged[m.Module] {
continue
}
for _, seat := range DependsOn(m) {
if held[seat] {
continue
}
out = append(out, Unheld{Node: node, Module: m.Module, Seat: seat,
Holders: PossibleHolders(catalogue, seat)})
}
}
sort.Slice(out, func(i, j int) bool {
if out[i].Module != out[j].Module {
return out[i].Module < out[j].Module
}
return out[i].Seat < out[j].Seat
})
return out
}
// nodeSeatsClaimed is every node seat a module claims, each by its current name (ADR 0122), so
// a dependency on any of them — a resource's or a contribution's — is met by the claim.
func nodeSeatsClaimed(m Manifest) []string {
var out []string
for _, c := range m.Claims {
if c.At() != ScopeNode {
continue
}
name := c.Name
if s, known := SeatNamed(name); known {
name = s.Name
}
out = append(out, name)
}
return out
}
// manifestsOf is the catalogue's definitions of the named modules; a name the catalogue does not
// know contributes nothing, as it does to a resolution.
func manifestsOf(catalogue map[string]Manifest, names []string) []Manifest {
var out []Manifest
seen := map[string]bool{}
for _, n := range names {
if m, known := catalogue[n]; known && !seen[n] {
seen[n] = true
out = append(out, m)
}
}
return out
}
// AssignRefusal is why assigning `adding` to a node already running `assigned` is refused, or
// nothing (novox/hq ADR 0207 §4): each new module's dependency the node's assignments, the new ones
// included, leave unmet.
//
// **Only the new modules are judged.** A node already short of a holder is reported by `status`;
// refusing an unrelated assignment for it would make the remedy — assigning the holder — refused too.
//
// **A dependency nothing in the catalogue can meet is said, not refused.** A refusal names the
// module that would meet it; with none registered there is no remedy to name, and refusing would
// stop every assignment of that kind until a module that does not exist yet is written. The answer
// still says it, and `status` reports it — before the switch and after it alike: there is never a
// remedy to name for it. The first return is those lines.
func AssignRefusal(catalogue map[string]Manifest, node string, assigned, adding []string) ([]string, error) {
set := manifestsOf(catalogue, append(append([]string(nil), assigned...), adding...))
judged := map[string]bool{}
for _, a := range adding {
judged[a] = true
}
var refused, said []string
for _, u := range UnheldDependencies(catalogue, node, set, judged) {
if len(u.Holders) == 0 {
said = append(said, u.String())
continue
}
refused = append(refused, u.String())
}
if len(refused) > 0 {
return said, &Refusal{Problems: append(refused,
fmt.Sprintf("holders that depend on each other are assigned together: `assign %s <module> <module>…`", node))}
}
return said, nil
}
// UnassignRefusal is why taking `removing` off a node running `assigned` is refused, or nothing
// (novox/hq ADR 0207): a seat the removed modules hold that nothing left on the node holds, while
// a module left there depends on it. Names the dependents, because they are what must go first —
// or the holder's replacement come.
func UnassignRefusal(catalogue map[string]Manifest, node string, assigned, removing []string) error {
gone := map[string]bool{}
for _, r := range removing {
gone[r] = true
}
var left []string
for _, a := range assigned {
if !gone[a] {
left = append(left, a)
}
}
before := map[string]bool{}
for _, u := range UnheldDependencies(catalogue, node, manifestsOf(catalogue, assigned), nil) {
before[u.Module+"\x00"+u.Seat] = true
}
dependents := map[string][]string{}
for _, u := range UnheldDependencies(catalogue, node, manifestsOf(catalogue, left), nil) {
if before[u.Module+"\x00"+u.Seat] {
continue // unmet already; not this removal's doing
}
dependents[u.Seat] = append(dependents[u.Seat], u.Module)
}
if len(dependents) == 0 {
return nil
}
seats := make([]string, 0, len(dependents))
for s := range dependents {
seats = append(seats, s)
}
sort.Strings(seats)
var problems []string
for _, s := range seats {
problems = append(problems, fmt.Sprintf(
"%s holds %s on %s, and %s depend on it (novox/hq ADR 0207) — unassign them with it, "+
"or assign another holder first", strings.Join(removing, ", "), s, node,
strings.Join(dependents[s], ", ")))
}
return &Refusal{Problems: problems}
}
// CollisionRefusal is why assigning `adding` beside `assigned` is refused for what two modules
// would both declare, or nothing (novox/hq ADR 0210 §1, 04-ISSUES/235).
//
// **Refused, not kept like an unresolved provision.** An assignment is otherwise kept when the
// node does not resolve, because assignment is not an ordering: a consumer's provider can follow.
// A collision is not an order anything can complete — no further assignment makes two owners of one
// package one owner — and kept, it leaves the node unresolvable, so the next push of anything drops
// it from the mesh. Only collisions involving a module being added are refused; one already on the
// node is `status`'s, and refusing an unrelated assignment for it would block its own remedy.
func CollisionRefusal(catalogue map[string]Manifest, node string, assigned, adding []string) error {
before := map[string]bool{}
for _, p := range checkResources(manifestsOf(catalogue, assigned)) {
before[p] = true
}
var problems []string
for _, p := range checkResources(manifestsOf(catalogue, append(append([]string(nil), assigned...), adding...))) {
if before[p] {
continue // on the node already; not this assignment's doing
}
problems = append(problems, p+" (novox/hq ADR 0210: one owner per node; the other module "+
"depends on the owner's seat instead)")
}
if len(problems) == 0 {
return nil
}
sort.Strings(problems)
return &Refusal{Problems: append(problems, fmt.Sprintf("nothing was assigned to %s", node))}
}
internal/catalogue/seat_dependencies_test.go
-247
View File
@@ -1,247 +0,0 @@
package catalogue
import (
"errors"
"reflect"
"strings"
"testing"
)
// Defends novox/hq ADR 0207: a module depends on the node seats that apply its resources.
func res(kind, id string) map[string]any {
r := map[string]any{"id": id, "type": kind}
switch kind {
case "service":
r["unit"] = id + ".service"
case "package":
r["package"] = id
case "container":
r["image"] = id
case "file":
r["path"] = "/etc/" + id
}
return r
}
func withResources(m Manifest, rs ...map[string]any) Manifest {
m.Resources = rs
return m
}
// The three holders as to-be 42 names them, each declaring what it really does: systemd's own
// package needs the package manager, pacman's timer needs the service manager, docker's package and
// service need both.
func coreThree() []Manifest {
return []Manifest{
withResources(mod("systemd", nil, nil, nil, Claim{Name: ServiceManagerSeat}), res("package", "systemd")),
withResources(mod("pacman", nil, nil, nil, Claim{Name: PackageManagerSeat}), res("service", "pacman-refresh")),
withResources(mod("docker", nil, nil, nil, Claim{Name: ContainerRuntimeSeat}),
res("package", "docker"), res("service", "docker")),
}
}
func TestADependencyIsDerivedFromEachResourceTypeThatAppliesThroughASeat(t *testing.T) {
cases := map[string][]string{
"service": {ServiceManagerSeat},
"package": {PackageManagerSeat},
"container": {ContainerRuntimeSeat},
// The host's own acts, or the mesh's: nothing in between holds a role for them.
"file": nil, "directory": nil, "process": nil, "archive": nil, "user": nil,
"action": nil, "network": nil, "access": nil,
}
for kind, want := range cases {
got := DependsOn(withResources(mod("m", nil, nil, nil), res(kind, "x")))
if len(got) == 0 {
got = nil
}
if !reflect.DeepEqual(got, want) {
t.Errorf("a %s resource depends on %v, want %v", kind, got, want)
}
}
all := DependsOn(withResources(mod("m", nil, nil, nil),
res("container", "a"), res("service", "b"), res("package", "c"), res("package", "d")))
if want := []string{ContainerRuntimeSeat, PackageManagerSeat, ServiceManagerSeat}; !reflect.DeepEqual(all, want) {
t.Errorf("a module of every kind depends on %v, want each seat once: %v", all, want)
}
}
func TestTheSeatsThatApplyResourcesAreTheMeshsOwnAtNodeScope(t *testing.T) {
for _, name := range []string{ServiceManagerSeat, PackageManagerSeat, ContainerRuntimeSeat} {
s, ok := SeatNamed(name)
if !ok {
t.Fatalf("%s is not in the mesh's set", name)
}
if s.Scope != ScopeNode {
t.Errorf("%s is held per %s, want per node", name, s.Scope)
}
}
// No verbs yet for either new seat: ADR 0207 seeds the package manager without a protocol, and
// the runtime's verbs wait for ADR 0166's acceptance.
for _, name := range []string{PackageManagerSeat, ContainerRuntimeSeat} {
if s, _ := SeatNamed(name); len(s.Serves)+len(s.Accepts)+len(s.Emits) > 0 {
t.Errorf("%s carries a protocol; ADR 0207 seeds it with none", name)
}
}
}
func TestANodeWhoseAssignmentsHoldTheSeatsResolvesWithNothingUnheld(t *testing.T) {
web := withResources(mod("web", nil, nil, nil), res("container", "web"), res("service", "web-timer"))
cat := shelf(append(coreThree(), web)...)
got, err := Resolve(cat, []string{"systemd", "pacman", "docker", "web"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
if len(got.Unheld) != 0 {
t.Errorf("a node holding all three seats reports %v", got.Unheld)
}
if _, err := AssignRefusal(cat, "workstation", []string{"systemd", "pacman", "docker"}, []string{"web"}); err != nil {
t.Errorf("assigning beside the three holders was refused: %v", err)
}
}
func TestAnAssignmentMissingAHolderIsRefusedNamingTheSeatAndItsPossibleHolders(t *testing.T) {
web := withResources(mod("web", nil, nil, nil), res("container", "web"), res("service", "web-timer"))
cat := shelf(append(coreThree(), web)...)
_, err := AssignRefusal(cat, "workstation", []string{"systemd", "pacman"}, []string{"web"})
var refusal *Refusal
if !errors.As(err, &refusal) {
t.Fatalf("web assigned to a node without a container runtime was not refused: %v", err)
}
msg := err.Error()
for _, want := range []string{"web on workstation depends on " + ContainerRuntimeSeat, "docker", "ADR 0207"} {
if !strings.Contains(msg, want) {
t.Errorf("the refusal does not say %q:\n%s", want, msg)
}
}
// What the node does hold is not named as missing.
if strings.Contains(msg, "depends on "+ServiceManagerSeat) {
t.Errorf("the refusal names a seat systemd already holds:\n%s", msg)
}
}
func TestADependencyNoCatalogueModuleCanMeetIsSaidNeverRefused(t *testing.T) {
// No runtime module in the catalogue: refusing would stop every container's assignment until one
// is written, with no remedy to name — so it is said, with the switch on as with it off.
web := withResources(mod("web", nil, nil, nil), res("container", "web"))
cat := shelf(web)
for _, on := range []bool{false, true} {
enforceSeatDependencies = on
said, err := AssignRefusal(cat, "workstation", nil, []string{"web"})
if err != nil {
t.Fatalf("switch %v: a dependency nothing could meet was refused: %v", on, err)
}
if len(said) != 1 || !strings.Contains(said[0], "no module in the catalogue claims it yet") {
t.Errorf("switch %v: the assignment does not say what it depends on: %v", on, said)
}
if _, err := Resolve(cat, []string{"web"}, workstation(), World{}); err != nil {
t.Errorf("switch %v: a dependency nothing could meet refused the node: %v", on, err)
}
}
enforceSeatDependencies = true
}
func TestTheHoldersMutualDependenceResolvesWhenAssignedTogether(t *testing.T) {
cat := shelf(coreThree()...)
// Alone, each needs the other.
if _, err := AssignRefusal(cat, "workstation", nil, []string{"systemd"}); err == nil ||
!strings.Contains(err.Error(), "pacman") {
t.Errorf("systemd alone was not refused naming pacman: %v", err)
}
if _, err := AssignRefusal(cat, "workstation", nil, []string{"pacman"}); err == nil ||
!strings.Contains(err.Error(), "systemd") {
t.Errorf("pacman alone was not refused naming systemd: %v", err)
}
// Together, in one act, they meet each other — and docker meets its own seat.
if _, err := AssignRefusal(cat, "workstation", nil, []string{"systemd", "pacman", "docker"}); err != nil {
t.Errorf("the three holders assigned together were refused: %v", err)
}
got, err := Resolve(cat, []string{"systemd", "pacman"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
if len(got.Unheld) != 0 {
t.Errorf("systemd and pacman together report %v", got.Unheld)
}
}
func TestStatusIsToldOfAnUnmetDependencyAndTheNodeStillResolves(t *testing.T) {
// Before the switch (the state the mesh ran in until every node held the three seats).
enforceSeatDependencies = false
defer func() { enforceSeatDependencies = true }()
web := withResources(mod("web", nil, nil, nil), res("container", "web"))
cat := shelf(append(coreThree(), web)...)
got, err := Resolve(cat, []string{"web"}, workstation(), World{})
if err != nil {
t.Fatalf("an unmet dependency refused the node before the switch: %v", err)
}
want := []Unheld{{Node: "workstation", Module: "web", Seat: ContainerRuntimeSeat, Holders: []string{"docker"}}}
if !reflect.DeepEqual(got.Unheld, want) {
t.Errorf("reported %+v, want %+v", got.Unheld, want)
}
}
func TestWithTheSwitchFlippedAnUnmetDependencyRefusesTheNode(t *testing.T) {
enforceSeatDependencies = true
defer func() { enforceSeatDependencies = true }()
web := withResources(mod("web", nil, nil, nil), res("container", "web"))
cat := shelf(append(coreThree(), web)...)
_, err := Resolve(cat, []string{"web"}, workstation(), World{})
if err == nil || !strings.Contains(err.Error(), ContainerRuntimeSeat) || !strings.Contains(err.Error(), "docker") {
t.Fatalf("with the switch on, an unmet dependency gave %v", err)
}
// Never in the first pass, whose refusals take a machine off the network instead.
if _, err := Resolve(cat, []string{"web"}, workstation(), World{Unchecked: true}); err != nil {
t.Errorf("the first pass refused an unmet dependency: %v", err)
}
}
func TestTheFoundationsDeclarationsAreNotJudged(t *testing.T) {
// The private network, as the controller computes it: its tools' package and its service are
// the mesh's, written per node, and so are never a module's dependency.
network := withResources(mod("mesh-wireguard", []string{"private-network"}, nil, nil),
res("package", "wireguard-tools"), res("service", "overlay-up"))
network.Computed = "mesh-wireguard"
// The host, whatever it declares.
host := withResources(mod("mesh-host", nil, nil, nil), res("file", "launcher"), res("service", "nox-mesh-host"))
cat := shelf(append(coreThree(), network, host)...)
for _, m := range []Manifest{network, host} {
if d := DependsOn(m); len(d) != 0 {
t.Errorf("%s, the foundation's, depends on %v", m.Module, d)
}
}
got, err := Resolve(cat, []string{"mesh-wireguard", "mesh-host"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
if len(got.Unheld) != 0 {
t.Errorf("the foundation on a node with no holders reports %v", got.Unheld)
}
if _, err := AssignRefusal(cat, "workstation", nil, []string{"mesh-wireguard", "mesh-host"}); err != nil {
t.Errorf("assigning the foundation was refused: %v", err)
}
}
func TestUnassigningTheLastHolderWhileDependentsRemainIsRefused(t *testing.T) {
sshd := withResources(mod("sshd", nil, nil, nil), res("service", "sshd"))
cat := shelf(append(coreThree(), sshd)...)
on := []string{"systemd", "pacman", "docker", "sshd"}
err := UnassignRefusal(cat, "workstation", on, []string{"systemd"})
if err == nil {
t.Fatal("the last service manager came off a node still running services")
}
for _, want := range []string{ServiceManagerSeat, "sshd", "pacman", "docker"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not name %q:\n%v", want, err)
}
}
// A dependent comes off freely, and the holders with everything depending on them in one act.
if err := UnassignRefusal(cat, "workstation", on, []string{"sshd"}); err != nil {
t.Errorf("a dependent's unassignment was refused: %v", err)
}
if err := UnassignRefusal(cat, "workstation", on, on); err != nil {
t.Errorf("unassigning everything together was refused: %v", err)
}
}
internal/catalogue/seat_into.go
+5 -5
View File
@@ -43,8 +43,8 @@ import (
var ofSeat = regexp.MustCompile(`\$\{seat:([a-z0-9][a-z0-9-]*):([0-9]+)\}`)
// seatInto replaces a resource's ${seat:…} placeholders with where this machine put each seat's
// holder — in a file's content, and in a value of a container's or a process's environment. The
// same places portInto fills, for the same reason: they are where a program reads a number from.
// holder — in a file's content, and in a value of a container's environment. The same two places
// portInto fills, for the same reason: they are where a process reads a number from.
func seatInto(resource map[string]any, module string, with Rendering) error {
switch fmt.Sprint(resource["type"]) {
case "file":
@@ -58,7 +58,7 @@ func seatInto(resource map[string]any, module string, with Rendering) error {
}
resource["content"] = filled
case "container", "process":
case "container":
env, ok := resource["env"].(map[string]any)
if !ok {
return nil
@@ -78,8 +78,8 @@ func seatInto(resource map[string]any, module string, with Rendering) error {
continue
}
value, err := seatsFilledInto(written,
fmt.Sprintf("%s's %s %s sets %s to something that",
module, resource["type"], resource["name"], key), with)
fmt.Sprintf("%s's container %s sets %s to something that",
module, resource["name"], key), with)
if err != nil {
return err
}
internal/catalogue/seat_into_test.go
+9 -10
View File
@@ -101,7 +101,7 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
}
// The manifest itself names them now; withSeatPorts is a no-op on it, and this holds it so.
for _, r := range m.Resources {
if r["type"] != "process" {
if r["type"] != "container" {
continue
}
env, _ := r["env"].(map[string]any)
@@ -113,9 +113,8 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
}
m = withSeatPorts(m)
control, err := m.Resolve([]Built{{
Name: "controller", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "mesh-controller/controller@sha256:" + strings.Repeat("c", 64),
Digest: "sha256:" + strings.Repeat("c", 64),
Name: "server", Kind: ArtifactImage,
Reference: ArtifactStoreScheme + "mesh-controller/server@sha256:" + strings.Repeat("c", 64),
}})
if err != nil {
t.Fatal(err)
@@ -136,9 +135,9 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
if err != nil {
t.Fatalf("the control plane does not compose: %v", err)
}
server := fileNamed(out, "mesh-controller.controller")
server := fileNamed(out, "mesh-controller.server")
if server == nil {
t.Fatalf("the control plane's process is not in the declaration: %v", out)
t.Fatalf("the control plane's container is not in the declaration: %v", out)
}
env, _ := server["env"].(map[string]any)
for key, want := range map[string]string{
@@ -152,8 +151,8 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
t.Errorf("the control plane is told %s=%v; the node put it on %s", key, env[key], want)
}
}
if got := server["source"]; got != "anchor.internal:5100/mesh-controller/controller@sha256:"+strings.Repeat("c", 64) {
t.Errorf("the control plane's own bundle is fetched from %v, not routed through the store", got)
if got := server["image"]; got != "anchor.internal:5100/mesh-controller/server@sha256:"+strings.Repeat("c", 64) {
t.Errorf("the control plane's own image is %v, not routed through the store", got)
}
// And on a mesh where the foundation is where genesis raised it, nothing is added.
@@ -161,7 +160,7 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
if err != nil {
t.Fatal(err)
}
env, _ = fileNamed(out, "mesh-controller.controller")["env"].(map[string]any)
env, _ = fileNamed(out, "mesh-controller.server")["env"].(map[string]any)
if env["MESH_STORE_INVENTORY_PORT"] != "" {
t.Errorf("with no settings, the control plane is told %v", env)
}
@@ -187,7 +186,7 @@ func withSeatPorts(m Manifest) Manifest {
out := m
out.Resources = nil
for _, r := range m.Resources {
if r["type"] != "container" && r["type"] != "process" {
if r["type"] != "container" {
out.Resources = append(out.Resources, r)
continue
}
internal/catalogue/seats.go
+3 -128
View File
@@ -40,34 +40,16 @@ type Seat struct {
// Serves carries each verb in full — name, description, schema — because a role's tools are the
// mesh's to define and an agent's to call (novox/hq ADR 0132, design 33 §2).
Serves []Verb
// Receives is what other modules may contribute to the seat's holder, by kind (novox/hq ADR
// 0212): each kind is text in the tool's own grammar, placed by the holder with
// ${contribution:<seat>:<kind>}. Compiled, never stored: like the protocol, it is the mesh's
// definition of the role, and the store's rows carry no column for it.
Receives []Receivable
// Decision is the record that made it a seat.
Decision string
}
// Receivable is one kind of contribution a seat receives (novox/hq ADR 0212 §2): its name, and the
// comment prefix of the tool's grammar, with which the controller names each contributing module.
type Receivable struct {
Kind string
Comment string
// Dirs says a contribution of this kind may name its contributor's own directories as
// `${dir:<id>}`, filled with where they are on the machine before the holder places it (novox/hq
// to-be 43): a module saying which of its data to back up names a directory the mesh placed, and
// only the mesh knows where. Off for every kind written in a tool's grammar that has its own
// `${…}` — a shell's — where the mesh filling one would change what the tool reads.
Dirs bool
}
// defaultSeats is the set the mesh ships with — the seed for the control plane's seat table and the
// fallback when it has none (novox/hq ADR 0122). It is the one place the closed set 0110 defines is
// written; the store's table is seeded from it and thereafter is the live, editable copy.
//
// In the order a person reads it: the mesh's own, then a node's.
var defaultSeats = append([]Seat{
var defaultSeats = []Seat{
// The control plane states what it did under the seat it holds (novox/hq ADR 0134): a role's
// events belong to the role, so they keep their address while the holder is replaced. No accepts,
// so no work queue is raised for it — only what its holder may say.
@@ -125,31 +107,7 @@ var defaultSeats = append([]Seat{
// that machine unresolvable in the meantime. Deleted once no registered manifest claims it.
{Name: "mesh-build-machine", Scope: ScopeMesh,
Accepts: []string{"build"}, Emits: []string{"started", "built", "log.*"}, Decision: "novox/hq ADR 0190"},
// **The mesh's one resolver** (novox/hq ADR 0194, 0196): every node's internal domain, held in one
// place, and every node and container asks it first. Delivers what a machine's resolver
// configuration requires, so that requirement resolves to the holder wherever it is placed.
{Name: "mesh-dns-resolver", Scope: ScopeMesh, Delivers: "wildcard-resolution", Decision: "novox/hq ADR 0194"},
// **Retired by ADR 0194, kept while a manifest still claims it** — the same reason as
// mesh-build-machine above: a machine still holds it until the mesh's resolver replaces it, and
// removing the row first would make that machine unresolvable. Deleted once nothing claims it.
{Name: "node-dns-resolver", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
// **A machine's /etc/hosts is one module's** (novox/hq ADR 0199): its holder writes the machine's
// own lines and keeps every other line as the operator's, changed through these three verbs on that
// machine alone. The controller holds none of it.
{Name: "node-hosts-file", Scope: ScopeNode, Decision: "novox/hq ADR 0199",
Serves: []Verb{
{Name: "entries", Description: "Every line of this machine's /etc/hosts, each marked whose it is: " +
"the operator's, or the block of the module or tool that writes it.",
Input: schema(map[string]string{}, nil)},
{Name: "add", Description: "Add one address and its names to the operator's lines of this machine's " +
"/etc/hosts — a name for this machine's own programs, not the mesh's.",
Input: schema(map[string]string{"address": "the IPv4 or IPv6 address",
"names": "the names for it, separated by spaces"}, []string{"address", "names"})},
{Name: "remove", Description: "Remove one name, or every line of one address, from the operator's " +
"lines of this machine's /etc/hosts. A line a module writes is refused, naming the module.",
Input: schema(map[string]string{"name": "a host name, or an address to remove every line of"},
[]string{"name"})},
}},
// The intrusion prevention's verbs (novox/hq ADR 0179): what a person asks a machine's ban list
// whatever keeps it — who is banned and why, ban one address, let one go. Every holder serves all
// four; the jails themselves are composed from the modules the machine runs (to-be 31).
@@ -190,50 +148,8 @@ var defaultSeats = append([]Seat{
// system or user scope; the holder answers questions and operator acts about them, each verb
// taking the unit and an optional scope. The holder runs nothing of its own: its verbs are
// served by the node tools runtime (ADR 0175).
{Name: ServiceManagerSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0177",
{Name: "node-service-manager", Scope: ScopeNode, Decision: "novox/hq ADR 0177",
Serves: serviceManagerVerbs()},
// The machine's package manager (novox/hq ADR 0207). A module declaring a `package` depends on
// it being held on its node, as one declaring a `service` depends on node-service-manager: the
// mesh's word for "something on this machine answers for installing", where a capability only
// says the software is there. No verbs yet — the seat says who answers, and what may be asked
// of it is decided when someone needs to ask.
{Name: PackageManagerSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0207"},
// The machine's container runtime (novox/hq ADR 0166, seeded now by ADR 0207): a module
// declaring a `container` depends on it being held on its node. Its verbs, and the host creating
// containers through its holder, wait for ADR 0166's acceptance — seeded without them so the
// dependency has a seat to name and the runtime's module has one to claim.
{Name: ContainerRuntimeSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0166, ADR 0207"},
// The operator account's environment (novox/hq ADR 0203): one module per machine writes it, and
// every module contributes to it. No verbs — the seat says who places the environment's files,
// and their path is its protocol: a shell sources ~/.config/mesh/environment.sh without knowing
// which module wrote it.
{Name: EnvironmentSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0203"},
// The login shell (novox/hq ADR 0204, replacing the module-declared `login-shell` of ADR 0176):
// the mesh's, so a second shell module claims the seat rather than declaring a second one, and
// the seat exists whether or not zsh's definition is registered. `execute` is the contract any
// node may call; the holder places every module's shell code in its slots.
{Name: LoginShellSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0204",
Serves: loginShellVerbs()},
// A machine's power (novox/hq ADR 0211): its holder owns logind's power handling, places the
// code modules contribute for the power moments, and publishes the machine's power states as
// its events. Every machine has one — every machine boots and shuts down. No verbs yet.
{Name: PowerSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0211"},
// The machine's hotkeys (novox/hq ADR 0212): the daemon that sees the keys the window manager
// does not — a laptop's vendor keys — run by one module per machine, which owns its
// configuration. Every other module with keys contributes trigger lines to it.
{Name: HotkeysSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0212",
Receives: []Receivable{{Kind: "trigger", Comment: "#"}}},
// The machine's message bus (novox/hq ADR 0215): its holder owns the D-Bus implementation and its
// system service, never restarts it live, and publishes curated events about it, never its traffic.
// It receives nothing yet: packages ship their own policies.
{Name: MessageBusSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0215"},
// The machine's backups (novox/hq ADR 0214, to-be 43): its holder keeps nightly restore points of
// the data every module on the machine declares, on the machine, against mistakes rather than
// disasters. A module contributes `backup` lines — what to run to take a consistent copy, and
// which of its directories to keep.
{Name: BackupSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0214",
Serves: backupVerbs(),
Receives: []Receivable{{Kind: "backup", Comment: "#", Dirs: true}}},
// Deferred (novox/hq ADR 0121): renaming to mesh-private-network is a scope + server/client
// model change, not a rename, so it stays until that is built.
{Name: "the-private-network", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
@@ -245,9 +161,7 @@ var defaultSeats = append([]Seat{
// rather than a condition in the resolver's module, so a machine running two managers is
// refused at assignment instead of found by the resolver being rewritten (novox/hq ADR 0117).
{Name: "node-uplink", Scope: ScopeNode, Decision: "novox/hq ADR 0117"},
},
// The graphical session's roles (novox/hq ADR 0208), last because they are a workstation's.
graphicalSessionSeats()...)
}
// A system seat name is the control plane's namespace: `mesh-*` for a mesh-wide role, `node-*` for
// a per-node one (novox/hq ADR 0121). A claim to a system name the mesh does not define is refused;
@@ -296,10 +210,6 @@ func UseSeats(s []Seat) {
row.Accepts, row.Emits, row.Serves = d.Accepts, d.Emits, d.Serves
}
}
// What a seat receives is never stored (novox/hq ADR 0212), so it is always the compiled one.
if d, known := byName[row.Name]; known {
row.Receives = d.Receives
}
merged = append(merged, row)
}
seats = merged
@@ -546,38 +456,3 @@ func serviceManagerVerbs() []Verb {
Input: scoped(map[string]string{"unit": unit["unit"], "lines": "how many lines from the end (default 100)"}, []string{"unit"})},
}
}
// loginShellVerbs is the contract every holder of node-login-shell serves (novox/hq ADR 0176, ADR
// 0204): one command, run the way the operator's own terminal would run it, bounded below the
// runtime's thirty-second call limit so a hung command answers rather than times the caller out.
// backupVerbs is the node-backup seat's protocol (novox/hq to-be 43): what is kept, take one now,
// and restore beside the live data — never over it.
func backupVerbs() []Verb {
return []Verb{
{Name: "backed-up", Description: "What this machine backs up: each module, what it declared, " +
"its last good night, how many restore points are kept and the repository's size.",
Input: schema(map[string]string{"module": "one module (optional)"}, nil)},
{Name: "now", Description: "Take a backup now, of one module or of every module on this " +
"machine — before a migration, a retirement or anything else that could go wrong.",
Input: schema(map[string]string{"module": "one module (optional)"}, nil)},
{Name: "restore", Description: "Restore one module's data from a restore point BESIDE the live " +
"data, never over it: each directory as <path>.restored-<date>. Swapping it in is a " +
"person's act. Lists the restore points when none is named.",
Input: schema(map[string]string{
"module": "the module",
"snapshot": "the restore point (from `backed-up`; the newest when omitted)",
"path": "one of the module's directories (all of them when omitted)",
}, []string{"module"})},
}
}
func loginShellVerbs() []Verb {
return []Verb{
{Name: "execute", Description: "Run one command on this machine as the operator account, in a " +
"non-interactive login shell in its home; answers with what it printed and how it exited.",
Input: schema(map[string]string{
"command": "the command line, as you would type it",
"timeout_seconds": "give up after this long, at most 25 (default 20)",
}, []string{"command"})},
}
}
internal/catalogue/seats_declared.go
-20
View File
@@ -25,10 +25,6 @@ import (
// and nothing to keep in step when a mesh seat is added.
const meshSeatPrefix = "mesh-"
// retiredLoginShell is the one name outside the prefix a module may not declare: the login shell's,
// from when a module declared it (novox/hq ADR 0176), before it became the mesh's (ADR 0204).
const retiredLoginShell = "login-shell"
// A SeatDeclaration is a role a module offers on the bus: what may be sent to it, what it says,
// and what it answers. A caller declares that it uses the *seat*, never the module, so the
// implementation can be replaced under it.
@@ -92,15 +88,6 @@ func declaredSeatProblems(m Manifest) []string {
"seats (novox/hq ADR 0118)", m.Module, s.Name, meshSeatPrefix+"*"))
continue
}
if s.Name == retiredLoginShell {
// The name ADR 0176 gave the login shell when the zsh module declared it. The seat is
// the mesh's now, so a module declaring the old name would be a second login shell
// beside it, with a protocol of its own (novox/hq ADR 0204).
problems = append(problems, fmt.Sprintf(
"%s declares a seat named %q; the login shell is the mesh's own seat %s, which a shell "+
"module claims and none declares (novox/hq ADR 0204)", m.Module, s.Name, LoginShellSeat))
continue
}
if seen[s.Name] {
problems = append(problems, fmt.Sprintf(
"%s declares the seat %q twice", m.Module, s.Name))
@@ -228,13 +215,6 @@ func CatalogueProblems(shelf Shelf) []string {
}
}
}
// A read of a module's state that module does not keep (novox/hq ADR 0201) — said only where the
// owner is on the shelf, as a consumer may be installed before its emitter.
var manifests []Manifest
for _, module := range shelfOrder(shelf) {
manifests = append(manifests, shelf[module])
}
problems = append(problems, StateReadsNothingDeclares(manifests)...)
sort.Strings(problems)
return problems
}
internal/catalogue/seats_test.go
+5 -11
View File
@@ -18,9 +18,7 @@ import (
// vocabulary test follows. If this fails because a seat was added, the fix is a record in novox/hq
// and a row in to-be 26, not a new number here.
func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
// A seat a later record extends names both, "novox/hq ADR 0166, ADR 0207": the one that defined
// it and the one that seeded it.
record := regexp.MustCompile(`^novox/hq ADR \d{4}(, ADR \d{4})*$`)
record := regexp.MustCompile(`^novox/hq ADR \d{4}$`)
seen := map[string]bool{}
delivered := map[string]string{}
for _, s := range Seats() {
@@ -46,14 +44,10 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
delivered[s.Delivers] = s.Name
}
}
// Thirty-eight with node-backup (novox/hq ADR 0214); thirty-seven with node-message-bus (novox/hq ADR 0215); thirty-six with mesh-dns-resolver (novox/hq ADR 0194) and node-hosts-file (ADR 0199); thirty-four
// with node-hotkeys (ADR 0212); thirty-three with node-power (ADR 0211); thirty-two since the
// graphical session's eleven (ADR 0208); twenty-one with node-package-manager and
// node-container-runtime (ADR 0207); nineteen with node-environment and node-login-shell (ADR 0203,
// ADR 0204); seventeen with node-build-agent (ADR 0190). Two fewer once the retired
// mesh-build-machine and node-dns-resolver rows go, when no registered manifest claims either.
if len(Seats()) != 38 {
t.Errorf("the mesh defines %d seats rather than 38; the set is closed, so a change here is "+
// Seventeen since node-build-agent (novox/hq ADR 0190) — sixteen once the retired
// mesh-build-machine row goes, when no registered manifest claims it any more.
if len(Seats()) != 17 {
t.Errorf("the mesh defines %d seats rather than 17; the set is closed, so a change here is "+
"a decision (novox/hq ADR 0110): %s", len(Seats()), seatNames())
}
}

Some files were not shown because too many files have changed in this diff Show More