Compare commits

...
Author SHA1 Message Date
jschoubben fd9bf5e30a mesh-sdk: the bootstrapped mesh-delivery-checker follows mesh-delivery 432df759, copied whole (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 20:51:21 +01:00
jschoubben 4cf9b72146 mesh-sdk: the bootstrapped mesh-delivery-checker is the contracts' seat at mesh-delivery 03809d8d, where try gains members (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 20:47:59 +01:00
jschoubben cffabdf21d mesh-sdk: the refusal of contracts modules and domain SDKs reads every path segment, so a folder module counts (ADR 0635 §1, ADR 0636 §8; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 19:42:00 +01:00
jschoubben a3fee9f85b Merge branch 'b9j/registry' into feat/registry-milestone 2026-11-05 19:26:06 +01:00
jschoubben 7cdcc34a6f RemovedDeliveryVerbs leaves mesh-sdk for the seat mesh-delivery's client (ADR 0634 section 2, issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 18:45:50 +01:00
jschoubben 9c1f52a902 The wanted accounts' layout leaves mesh-sdk for the grants part's package grantsstate (ADR 0634 section 1, ADR 0635 section 4, issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 18:43:21 +01:00
jschoubben fa56961ae8 mesh-sdk: refuse every contracts module and domain SDK, module names in the shared packages; the bootstrap equals its contracts' files, one row per seat (ADR 0635 §1, ADR 0636 §8; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 18:27:48 +01:00
jschoubben 0d3a10ff20 catalogue: the registry reference forms and ArtifactSweepSeat leave for the registry clients and the sweep seat's contract; seat-names keeps one rule's seat (S-8; issue 1966)
Resolve keeps its own address strip (kept, ADR 0155), which needs no seat name. The one seat name left is
ManifestNeeds' store holder (ADR 0632 §2), beside its rule in manifest_reads.go.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:38:04 +01:00
jschoubben cabd3c0a8d catalogue: PublishSeats and the npm and go registry seat names go (no user); seat-names shrinks (S-8)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:20:14 +01:00
jschoubben 7f34fc0fa4 Delete health's condition and findings: no importer is left; the module answer stays (S-7, ADR 0634 §5)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:18:52 +01:00
jschoubben 4099164aad Delete go/overlay: no importer is left; the package list, notInSDK and seat-names follow (S-7, ADR 0634 §5; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:18:36 +01:00
jschoubben 14f956e4cd Delete go/delivery: moved whole to mesh-delivery's SDK and joined; the package list, notInSDK, untilTheJoin and seat-names follow (S-7, ADR 0634 §5; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:18:12 +01:00
jschoubben 1465a8a831 Delete go/buildstep: no importer is left in the joined mesh-delivery; the package list follows (S-7, ADR 0634 §5; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:17:55 +01:00
jschoubben 39ac1917de Delete go/provisions: no importer is left in any tree; the list of packages and notInSDK follow (S-7, ADR 0634 §5; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:05:04 +01:00
jschoubben 8b6da23def Delete go/channel: no importer is left in any tree; the list of packages and notInSDK follow (S-7, ADR 0634 §5; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:05:03 +01:00
jschoubben de8c881cb6 Delete go/handacts: no importer is left in any tree; the list of packages and notInSDK follow (S-7, ADR 0634 §5; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:05:02 +01:00
jschoubben 15b65445c8 buscompose: BusRemovalGrants, ControllerLeftover* and WatchmanHears go, the grants part's busgrants holds them (ADR 0634 §5; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:04:28 +01:00
jschoubben 1815fd11b9 membership: CredentialIn and BareAddress go, no caller in any tree (ADR 0635 §4; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 17:04:01 +01:00
jschoubben 59f8659837 Package list test: composer, parts and provider may not come back (S-8 part, B9f; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 16:42:10 +01:00
jschoubben b8eb1afd14 health: readings.go (disk-load's readings, restic's last good night) leaves, held by disk-load's diskreadings and restic's backupgood (S-7 part, B9f; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 16:41:53 +01:00
jschoubben d0563ed7c1 buscompose: VaultSeat, AccountIssuedSubject and ResolvedBucket leave (no importer; the grants part's copy names them), with the untilTheJoin and seat-names entries (S-5 first part, B9f; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 16:40:52 +01:00
jschoubben 7a142c9b5b forgekey keeps the command runner: the key keeper and key parsing are ssh-client's sshkeys, KindBuilder, ExecRunnerClean and Said the delivery sdk's (S-3, B9f; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 16:39:07 +01:00
jschoubben 989a3c6670 Delete go/provider and its harness: the provisioner's client holds it, no importer left on any join head (S-7, B9f; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 16:37:59 +01:00
jschoubben 20495478b6 Delete go/parts: delivery sdk's parts holds it, no importer left on any join head (S-7, B9f; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 16:37:57 +01:00
jschoubben c84a2e58e8 Delete go/composer: delivery sdk's composer holds it, no importer left on any join head (S-7, B9f; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 16:37:56 +01:00
jschoubben a25c600810 seat list: ReloadIfUnread reads the list again once when a host's events path is live; event hub: StartUntilBound binds a late consumer without a restart (B9e; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 16:19:26 +01:00
jschoubben 9489385165 seat list: the bus of a process on its own credential (a verb called through its connection, the seat's events as handlers of the module's event hub), so the checker and the grants part read the list as every part does (B7e; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 15:35:19 +01:00
jschoubben 4111794690 event hub: a start that found no durable on the bus is tried again at the next start, so a part is not lost for the order in which the bus makes its consumer (B7e; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 15:25:56 +01:00
jschoubben b7f6cd6bf4 seat list: the mesh-catalog definition, in the bootstrap and the test list, lists manifest-accepted and manifest-refused like the contract (B7d finding 1; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 15:24:52 +01:00
jschoubben 7aea59a789 seat list: the node-engine definition follows the contract's lists in a node's facts (copies kept equal; ADR 0636 §7; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 15:19:14 +01:00
jschoubben 52ab209fcf seatlisttest equals the contracts on the join heads (renamed seats mesh-node-enroller, mesh-module-assigner; domain; B2's delivery seats); the bootstrap is held equal to it by a test; the compiled-list golden is gone (B7c finding 2; ADR 0634 §9, ADR 0636 §7; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 15:10:17 +01:00
jschoubben e648fe5374 process: one EventHub pulls the module's one durable consumer, handlers by subject pattern; the views reader registers on it instead of pulling (class fix of B7c finding 1; ADR 0635 §5; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 15:07:39 +01:00
jschoubben 7ea9572f03 keys: the version check, CheckName and IsHex64 move here from the Delivery domain's kept package, since Delivery and Identity and access share them (ADR 0634 §1, §4; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 14:40:09 +01:00
jschoubben e67f85b5fe seatlist: one test of the changed event heard before the seat answered, not two (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 14:37:53 +01:00
jschoubben 02fe0a015b views: an owner's changes are ordinary events on the shared events stream, heard on the reader's one durable consumer; Follow takes no stream, Other takes the rest (ADR 0635 §5, D-25; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 14:36:41 +01:00
jschoubben 129dc8cffb seatlist bootstrap: node-engine with its verbs seat-list and seat-list-again and its state findings, mesh-catalog as B2's definition now holds it; every bootstrap file equals its contracts file (ADR 0636 §2, §7; issue 1966) 2026-11-05 14:35:20 +01:00
jschoubben 2b9cdb5771 seatlist: Start returns at once and reads the seat on a goroutine; Ready says when the first read ended, Status says it is reading (ADR 0636 §1; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 14:35:19 +01:00
jschoubben 99e6badabf seatlist bootstrap: mesh-catalog (with seat-definitions and its two events) and mesh-delivery-checker as the delivery contracts define them, plus the domain they lack (ADR 0636 §2; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 14:32:38 +01:00
jschoubben 9fb2c42138 seatlist: a reader started while the seat was silent is heard at the seat's changed event, which reads the whole list (ADR 0636 §1; issue 1966) 2026-11-05 14:32:25 +01:00
jschoubben b81622842b seatlist: an event heard before the seat ever answered reads the whole list (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 14:32:19 +01:00
jschoubben e0d2298f58 seatlist: the again tool answers a VerbCallers-style list, so the operator's sessions can be named (ADR 0636 §7; issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-05 14:31:27 +01:00
jschoubben a0480588ce seatdef: a definition names its glossary domain, one of the eleven headings (ADR 0636 §5; issue 1966) 2026-11-05 14:30:56 +01:00
jschoubben 8dcdc9b2a4 go/motion: whether a module is being delivered on a node, over plain want and have values; read by the delivery view and the condition keeper (S-X1, ADR 0634 §1; issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-05 13:46:01 +01:00
jschoubben b7ad47ab2a Seat list reader: an integration case over a real bus, from the bootstrap alone to the seat's answer and its events (ADR 0636 §8; issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-05 13:45:30 +01:00
jschoubben 351f8abba7 views: the hand-act recorder and the finding writer come from the caller as interfaces, no hand-acts or health import (ADR 0635 §4; issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-05 13:44:38 +01:00
jschoubben e8ae3ec8d3 go/keys: the version of a kept value and the key-part escaping that more than one domain shares (ADR 0634 §4; issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-05 13:42:59 +01:00
jschoubben b611bb9ad5 Seat list reader: calls the seat mesh-catalog's seat-definitions and hears its changed and removed events, acts on nothing new until read, two runtime tools; bootstrap gains mesh-conditions, operator-channel and the contracts' current files (ADR 0636 §1-§4, §7; issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-05 13:42:20 +01:00
jschoubben 1aca4991d4 mesh-sdk compiles no seat list and imports no contracts module: catalogue takes its seats from the host, whole seats' names leave it (ADR 0635 §1, §4; issue 1966)
catalogue's working set starts empty and a host gives it the seats it read (UseSeats over SeatsFromDefinitions);
the node-engine's verbs and events take the seat's name from the caller. Tests take their seats from seatlisttest.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-05 12:54:23 +01:00
jschoubben 7ea960fe20 Seat list reader: its status for a module's health, and the foundation's bootstrapped definitions (ADR 0635 §3; issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-05 12:49:36 +01:00
jschoubben 50258215b4 Seatdef: the templates test moves to an external test package, the catalogue now reads definitions (issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-05 12:47:46 +01:00
jschoubben 651c28e70d Seat definition format: states replace records (templates, writer, key forms with value schemas; ADR 0608 §1, ADR 0634 §6; issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-05 12:47:26 +01:00
jschoubben 863b909504 Seat list reader: the seat definitions read from the bus, followed on seat-definition-changed (ADR 0635 §1, §3; issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-05 12:47:26 +01:00
jschoubben 2e4eb0be90 Catalogue: a module's needs include its templates' text (ADR 0632 §2; issue 1966)
docker's daemon file and artifact-store-tools' store address move to the image
seat's reach in their templates while their manifests stay the same, so needs
read from the manifest alone let an engine before the reading lose the file.
ModuleNeeds reads the templates TemplatePaths names; the merge message carries
the result per directory for the indexer.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 21:37:43 +01:00
jschoubben ecdbf1a668 Catalogue: the store holder's manifest needs form:seat-reach (ADR 0632 §2; issue 1966)
It is read on every node and leaves an engine before the reading without the
old way to find the store.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 21:03:59 +01:00
jschoubben 8e7001509a Catalogue: the store's seats deliver nothing, and a seat's reach is the reader capability form:seat-reach (ADR 0632 §1, §2; issue 1966)
CanHold asks the store's holder no provision once the seats stop delivering
artifact-store; a manifest naming ${seat:<seat>:reach} needs an engine that
fills it from the seat holder's listen, so the delivery holds it from one
that does not.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 20:53:50 +01:00
jschoubben bbf1166436 Put back the indirect requirements SDK-8 raised for no change of its own: antithesis-sdk-go v0.8.0 needs modules the running node-engine's composer cannot find offline (issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 19:46:58 +01:00
319 changed files with 34018 additions and 37120 deletions
-7
View File
@@ -1,7 +0,0 @@
package buildstep
// CapacitySetting is the build modules' setting of the node's build capacity (novox/hq ADR 0525 §5, ADR 0567 §12): how
// many asks the node takes at once, an integer config value set on each node's layer of mesh-config. No repository names
// a node: each build module declares it with the default 0, which pulls nothing, and the controller (or mesh-config)
// lays the node's value into the module's settings file.
const CapacitySetting = "capacity"
-5
View File
@@ -1,5 +0,0 @@
// Package buildstep is the contract of a build step every delivery part reads (novox/hq ADR 0593 §1): a step's id, the
// fetch request, a step a previous process left running and its end, a failure's words, a module tree's shape, the
// capacity setting and the launchers' names. The runner the build modules share is the Go build module's own package
// (mesh-catalog modules/mesh-build-go/buildstep), which imports this one.
package buildstep
-141
View File
@@ -1,141 +0,0 @@
package buildstep
import (
"fmt"
"strings"
"git.novox.be/novox/mesh-sdk/go/sdkversion"
)
// Failure is one test that failed, or a package or file that failed with no test to name (a build that failed, a test
// program that panicked, a test file that did not load). The JSON names are the ones mesh-build-go and the checker have
// always used; File is new and an older reader ignores it.
type Failure struct {
// Package is the Go package's import path, or the TypeScript package's directory in the workspace.
Package string `json:"package"`
// Test names the failing test, or the tests that were running when the program stopped; "" when none can be named.
Test string `json:"test,omitempty"`
// File is where the failing test is (a Go test's file and line is in its Cause; node names the test's file:line:col).
File string `json:"file,omitempty"`
// Cause is the one line that says why: the assertion's or the error's first line, a "panic:" line, a build's first
// error, or words saying how the program stopped. **Never empty, and never a passing line** (PassingLine).
Cause string `json:"cause,omitempty"`
// Output is the failing test's own output, its last lines up to OutputBytes, Cause first: an older checker reads
// only its first line.
Output string `json:"output,omitempty"`
}
// OutputBytes is the most a failure keeps of its output, and LineBytes the most of one line: four kilobytes holds a long
// assertion message and stays far under a bus message's or a verdict's bound.
const (
OutputBytes = 4096
LineBytes = 1024
)
// By names a build module and the SDK it was built with, for a result's "by": the checker names it when a failure hides
// its cause ("the build module hid the failure (<by>)").
func By(module string) string {
return module + " with mesh-sdk " + sdkversion.Version()
}
// PassingLine says a line is a test runner's word for a test or a package that passed: node's '✔', TAP's "ok N" (at any
// indent), go test's "ok \t<pkg>", "PASS" and "--- PASS". A line of a test's own that starts "ok " is none of these. Such a line is never a failure's cause, nor part of a failure's output.
func PassingLine(l string) bool {
if strings.HasPrefix(l, "ok \t") { // go test's package line, at column 0
return true
}
t := strings.TrimSpace(l)
switch {
case t == "PASS", strings.HasPrefix(t, "✔"), strings.HasPrefix(t, "--- PASS"):
return true
}
return tapOk(t)
}
// Hidden says how a failure hides why it failed, or "" when it does not: it has no cause, or its cause is a passing line.
// The checker says it loudly (issue 1901): a build module that sends such a failure does not keep the contract.
func Hidden(f Failure) string {
c := strings.TrimSpace(f.Cause)
switch {
case c == "":
return "it sent no cause"
case PassingLine(c):
return fmt.Sprintf("its cause is a passing line (%q)", CutLine(c))
}
return ""
}
// Clip joins the last lines of a failure's output that fit OutputBytes, each cut to LineBytes, and says how many earlier
// lines it left out. **Its first line is the cause**, put ahead when the kept lines do not start with it, or, with no
// cause and lines left out, the first non-empty line, then the marker: an older checker reads only the first line.
// Passing lines are dropped.
func Clip(lines []string, cause string) string {
own := make([]string, 0, len(lines))
for _, l := range lines {
if !PassingLine(l) {
own = append(own, l)
}
}
lines = own
size, from := 0, len(lines)
for from > 0 {
n := len(CutLine(lines[from-1])) + 1
if size+n > OutputBytes && from < len(lines) {
break
}
size += n
from--
}
kept := make([]string, 0, len(lines)-from+2)
head := strings.TrimSpace(cause)
for i := 0; head == "" && i < from; i++ {
head = strings.TrimSpace(lines[i])
}
if head != "" && (from > 0 || len(lines) == 0 || strings.TrimSpace(lines[from]) != head) {
kept = append(kept, CutLine(head))
}
if from > 0 {
kept = append(kept, fmt.Sprintf("... (%d earlier lines)", from))
}
for _, l := range lines[from:] {
kept = append(kept, CutLine(l))
}
return strings.Join(kept, "\n")
}
// CutLine cuts a line to LineBytes, on a character boundary.
func CutLine(l string) string {
if len(l) <= LineBytes {
return l
}
return strings.ToValidUTF8(l[:LineBytes], "") + "..."
}
// ConsoleFailure is the failure of a command that is no test runner (go vet, gofmt, tsc, a build script): its cause is the
// first line that names an error (tsc's "error TS", go vet's "file.go:12:3: ..."), else the first line that is not a
// heading ("# <package>", npm's "> script"), and its output the command's lines without passing ones.
func ConsoleFailure(pkg, test, out string) Failure {
var lines []string
for _, l := range strings.Split(strings.ReplaceAll(stripANSI(out), "\r\n", "\n"), "\n") {
if t := strings.TrimSpace(l); t != "" && !PassingLine(t) {
lines = append(lines, strings.TrimRight(l, " \t"))
}
}
cause := ""
for _, l := range lines {
if t := strings.TrimSpace(l); strings.Contains(t, "error") || strings.Contains(t, "Error") || goPosition.MatchString(t) {
cause = t
break
}
}
for _, l := range lines {
if t := strings.TrimSpace(l); cause == "" && !strings.HasPrefix(t, "# ") && !strings.HasPrefix(t, "> ") && !strings.HasPrefix(t, "npm ") {
cause = t
}
}
if cause == "" {
cause = test + " failed and wrote nothing that says why"
}
cause = CutLine(cause)
return Failure{Package: pkg, Test: test, Cause: cause, Output: Clip(lines, cause)}
}
-19
View File
@@ -1,19 +0,0 @@
package buildstep
import (
"regexp"
"strings"
)
// goPosition matches a line that starts with a Go file's position, as go vet and the compiler write an error.
var goPosition = regexp.MustCompile(`^\S+\.go:\d+(:\d+)?: `)
var ansiEscape = regexp.MustCompile("\x1b\\[[0-9;]*m")
func stripANSI(s string) string { return ansiEscape.ReplaceAllString(s, "") }
// tapOk says a line is a passing test's "ok N - name", indented or not.
func tapOk(l string) bool {
rest, ok := strings.CutPrefix(strings.TrimSpace(l), "ok ")
return ok && rest != "" && rest[0] >= '0' && rest[0] <= '9'
}
-14
View File
@@ -1,14 +0,0 @@
package buildstep
import "git.novox.be/novox/mesh-sdk/go/catalogue"
// LauncherSuffix is what a TypeScript entrypoint's launcher is called beside it: index.js is started as
// index.serve.mjs (novox/hq ADR 0193), an ES module by its own extension whatever the bundle's package.json says.
const LauncherSuffix = catalogue.LauncherSuffix
// Launchers are the launchers a bundle's build writes, by entrypoint, relative to the bundle's root
// (catalogue.Launchers): the one rule mesh-build-node, build-agent, the controller and a node taking in a build share
// (R4e, issue 1126), so the name a build writes is the name a reader expects.
func Launchers(language string, entrypoints []string) map[string]string {
return catalogue.Launchers(language, entrypoints)
}
-22
View File
@@ -1,22 +0,0 @@
package buildstep
import (
"reflect"
"testing"
)
// A TypeScript bundle's entrypoint x.js is started as x.serve.mjs beside it (novox/hq ADR 0193), the rule mesh-build-node
// and build-agent both apply; an entrypoint that is not .js gets none, and another language gets none.
func TestATypeScriptEntrypointIsStartedByTheLauncherBesideIt(t *testing.T) {
got := Launchers("typescript", []string{"index.js", "tools/x.js", "data.json"})
want := map[string]string{"index.js": "index.serve.mjs", "tools/x.js": "tools/x.serve.mjs"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("Launchers(typescript) = %v, want %v", got, want)
}
if got := Launchers("go", []string{"index.js"}); len(got) != 0 {
t.Fatalf("a Go bundle got launchers: %v", got)
}
if got := Launchers("typescript", nil); len(got) != 0 {
t.Fatalf("a bundle with no entrypoint got launchers: %v", got)
}
}
-163
View File
@@ -1,163 +0,0 @@
package buildstep
import (
"fmt"
"maps"
"path"
"regexp"
"strings"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
var (
requestCommitOK = regexp.MustCompile(`^[0-9a-f]{40}$`)
requestRepositoryOK = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]*/[A-Za-z0-9][A-Za-z0-9._-]*$`)
)
// FetchRequest is what a build module's fetch tool is asked: the repository and the full commit, the pull request's
// ref when only its head carries the commit, the trunk commit to merge onto for a merge preview, and, for a build of a
// module whose every artifact names its paths, that module's directory and its named paths (novox/hq ADR 0499 clause 4).
//
// **The paths come from the builder, not from a manifest** (ADR 0495 clause 1; the lead's decision L8 of R4e): the
// builder fills Dir and Paths from the indexer's Module value (Path, and KeptInputs2.Paths' keys), and the build module
// gives the build and test steps a workspace holding those paths, the manifest and its lock files. A request without
// them builds from the module's directory, as before.
type FetchRequest struct {
Repository string `json:"repository"`
Commit string `json:"commit"`
Ref string `json:"ref,omitempty"`
Base string `json:"base,omitempty"`
Dir string `json:"dir,omitempty"`
Paths []string `json:"paths,omitempty"`
// Packages are the version of each mesh package the build or the check builds against (delivery.StepAsk.Packages,
// novox/hq ADR 0573 §2, §4): the build module writes each into the workspace's go.mod and go.sum, whatever version
// the repository names, checksum-verified, and names them in its log.
Packages delivery.PackageVersions `json:"packages,omitempty"`
}
// FetchRequestProperties are the JSON schema properties of FetchRequest's dir and paths, for a fetch tool's Input.
func FetchRequestProperties() map[string]any {
return map[string]any{
"dir": map[string]any{"type": "string", "description": "the module's directory in the repository (\".\" for its root), with paths"},
"paths": map[string]any{"type": "array", "items": map[string]any{"type": "string"},
"description": "the paths the module names, relative to dir, from the indexer's kept inputs: the build sees only these, the manifest and its lock files"},
"packages": map[string]any{"type": "object", "additionalProperties": map[string]any{"type": "string"},
"description": "optional: the version (a full commit) of each mesh package to build against, by its module path, whatever version go.mod names (novox/hq ADR 0573 §2); the builder and the checker fill it"},
}
}
// Args is r as a tool call's arguments.
func (r FetchRequest) Args() map[string]any {
args := map[string]any{"repository": r.Repository, "commit": r.Commit}
for k, v := range map[string]string{"ref": r.Ref, "base": r.Base, "dir": r.Dir} {
if v != "" {
args[k] = v
}
}
if len(r.Paths) > 0 {
args["paths"] = append([]string(nil), r.Paths...)
}
if len(r.Packages) > 0 {
args["packages"] = map[string]any{}
for p, v := range r.Packages {
args["packages"].(map[string]any)[p] = v
}
}
return args
}
// ParseFetchRequest reads a fetch tool's arguments; arguments it does not know (a build module's own) are left to the
// caller. It refuses a repository that is not <owner>/<name>, a commit or a base that is not 40 hex digits, paths with no
// module directory or a directory with no path, a directory leaving the repository and a path leaving the directory,
// naming each.
func ParseFetchRequest(args map[string]any) (FetchRequest, error) {
var r FetchRequest
r.Repository, _ = args["repository"].(string)
r.Commit, _ = args["commit"].(string)
r.Ref, _ = args["ref"].(string)
r.Base, _ = args["base"].(string)
r.Dir, _ = args["dir"].(string)
switch v := args["paths"].(type) {
case nil:
case []string:
r.Paths = append([]string(nil), v...)
case []any:
for _, p := range v {
s, ok := p.(string)
if !ok {
return FetchRequest{}, fmt.Errorf("paths is a list of strings")
}
r.Paths = append(r.Paths, s)
}
default:
return FetchRequest{}, fmt.Errorf("paths is a list of strings")
}
packages, err := PackagesArg(args["packages"])
if err != nil {
return FetchRequest{}, err
}
r.Packages = packages
return r, r.Check()
}
// PackagesArg reads a tool call's packages argument, an object of strings (as JSON gives it) or the map an ask gives;
// nil for none.
func PackagesArg(v any) (delivery.PackageVersions, error) {
out := delivery.PackageVersions{}
switch v := v.(type) {
case nil:
return nil, nil
case delivery.PackageVersions:
maps.Copy(out, v)
case map[string]string:
maps.Copy(out, v)
case map[string]any:
for p, version := range v {
s, ok := version.(string)
if !ok {
return nil, fmt.Errorf("packages is an object of versions (strings), by package")
}
out[p] = s
}
default:
return nil, fmt.Errorf("packages is an object of versions (strings), by package")
}
if len(out) == 0 {
return nil, nil
}
return out, out.Check()
}
// Check is what ParseFetchRequest refuses.
func (r FetchRequest) Check() error {
if !requestRepositoryOK.MatchString(r.Repository) || strings.Contains(r.Repository, "..") {
return fmt.Errorf("%q is not a repository, <owner>/<name>", r.Repository)
}
if !requestCommitOK.MatchString(r.Commit) {
return fmt.Errorf("%q is not a full commit (40 hex digits)", r.Commit)
}
if r.Base != "" && !requestCommitOK.MatchString(r.Base) {
return fmt.Errorf("%q is not a trunk commit (40 hex digits)", r.Base)
}
switch {
case len(r.Paths) > 0 && r.Dir == "":
return fmt.Errorf("the request names paths and names no module directory (dir) they are relative to")
case len(r.Paths) == 0 && r.Dir != "":
return fmt.Errorf("the request names a module directory and no path")
}
if r.Dir != "" && (outside(r.Dir) || strings.HasPrefix(r.Dir, "/")) {
return fmt.Errorf("the module directory %q leaves the repository", r.Dir)
}
for _, p := range r.Paths {
if p == "" || strings.HasPrefix(p, "/") || outside(p) || path.Clean(p) == "." {
return fmt.Errorf("the path %q leaves the module's directory or is not a path in it", p)
}
}
return nil
}
func outside(p string) bool {
c := path.Clean(p)
return c == ".." || strings.HasPrefix(c, "../")
}
-64
View File
@@ -1,64 +0,0 @@
package buildstep
import (
"reflect"
"strings"
"testing"
)
// A build's fetch is handed the paths of a module that names them (novox/hq ADR 0499 clause 4): the builder fills them
// from the indexer's kept inputs and the build module reads them from the call, never from the manifest (ADR 0495
// clause 1). The request survives the trip through a tool call's arguments.
func TestAFetchRequestCarriesTheNamedPathsThroughTheToolCall(t *testing.T) {
// And the versions of the mesh packages to build against (novox/hq ADR 0573 §2; issue 1585 S4).
req := FetchRequest{Repository: "novox/mesh-sdk", Commit: strings.Repeat("a", 40), Ref: "refs/pull/7/head",
Dir: ".", Paths: []string{"package.json", "src"}, Packages: map[string]string{"git.novox.be/novox/mesh-sdk/go": strings.Repeat("d", 40)}}
args := req.Args()
// A tool call's arguments arrive as JSON: lists of any, objects of any.
args["paths"] = []any{"package.json", "src"}
if p, ok := args["packages"].(map[string]any); !ok || p["git.novox.be/novox/mesh-sdk/go"] != strings.Repeat("d", 40) {
t.Fatalf("the packages are not in the call's arguments: %v", args["packages"])
}
got, err := ParseFetchRequest(args)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(got, req) {
t.Fatalf("got %+v, want %+v", got, req)
}
plain, err := ParseFetchRequest(map[string]any{"repository": "novox/x", "commit": strings.Repeat("b", 40),
"base": strings.Repeat("c", 40), "beside": []any{}})
if err != nil || plain.Paths != nil || plain.Base != strings.Repeat("c", 40) {
t.Fatalf("a request without paths: %+v, %v", plain, err)
}
ok := map[string]any{"repository": "novox/x", "commit": strings.Repeat("b", 40)}
for _, c := range []struct {
set map[string]any
says string
}{
{map[string]any{"repository": "novox"}, "is not a repository"},
{map[string]any{"repository": "novox/../x"}, "is not a repository"},
{map[string]any{"commit": "abc"}, "is not a full commit"},
{map[string]any{"base": "abc"}, "is not a trunk commit"},
{map[string]any{"paths": []any{"src"}}, "names no module directory"},
{map[string]any{"dir": "modules/a", "paths": []any{"../b"}}, "leaves the module's directory"},
{map[string]any{"dir": "../a", "paths": []any{"src"}}, "leaves the repository"},
{map[string]any{"dir": "modules/a", "paths": []any{3}}, "paths is a list of strings"},
{map[string]any{"dir": "modules/a"}, "names a module directory and no path"},
{map[string]any{"packages": map[string]any{"git.novox.be/novox/mesh-sdk/go": "main"}}, "neither a full commit nor a Go semantic version"},
{map[string]any{"packages": map[string]any{"a b": strings.Repeat("d", 40)}}, "no module path"},
{map[string]any{"packages": []any{"x"}}, "packages is an object"},
} {
args := map[string]any{}
for k, v := range ok {
args[k] = v
}
for k, v := range c.set {
args[k] = v
}
if _, err := ParseFetchRequest(args); err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%v: %v, wanted a refusal saying %q", c.set, err, c.says)
}
}
}
-330
View File
@@ -1,330 +0,0 @@
package buildstep
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"strconv"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// RestartReason is why a step a previous process left running is ended.
const RestartReason = "the build module restarted while it ran"
// Boot is when this process began. A step with no owner (written by an older build) started at or after it is taken
// for this process's, which may still run it, so EndOne refuses it.
var Boot = time.Now()
// Owner is this process as a step's owner (delivery.Step.Owner): `<boot id>.<pid>.<start>`, the node's boot id
// (/proc/sys/kernel/random/boot_id), the pid and Boot in milliseconds, base 36. The start keeps two processes apart
// that a reboot or a pid namespace gave the same pid. A running step whose owner is another is a previous process's.
var Owner = ownerOf(bootID, os.Getpid(), Boot)
// bootID is the node's boot id as Owner holds it; noBootID when it could not be read, which EndStaleAtStart says once.
var bootID = readBootID()
const noBootID = "no-boot-id"
func ownerOf(bootID string, pid int, start time.Time) string {
return bootID + "." + strconv.Itoa(pid) + "." + strconv.FormatInt(start.UnixMilli(), 36)
}
// readBootID is the node's boot id, or noBootID where it cannot be read (the pid and the start then tell the processes
// apart).
func readBootID() string {
b, err := os.ReadFile("/proc/sys/kernel/random/boot_id")
if id := strings.TrimSpace(string(b)); err == nil && id != "" {
return id
}
return noBootID
}
// Scan reads and writes a steps state with its revisions: its keys, a key's stored value and revision (nil and 0 for
// none), a write that lands only while the key is still at the revision read (KeptState.Update), and a delete of a key
// still at the revision read; both fail with stdio.ErrChanged when another writer was first.
type Scan struct {
Keys func() ([]string, error)
Get func(key string) (json.RawMessage, uint64, error)
Update func(key string, value any, revision uint64) (uint64, error)
Delete func(key string, revision uint64) error
}
// ScanOf is the Scan of a state the process keeps.
//
// Its Delete reads the key again and deletes it only while it is still at the revision read: the runtime's
// mesh/state.delete takes no revision yet, so a write landing between that read and the delete (a few milliseconds) is
// deleted with it. For a step key that is a redelivered ask begun on this node in that moment: its running key leaves
// the steps state until its run writes its end, which its asker still reads (a watch takes puts only).
func ScanOf(s stdio.KeptState) Scan {
get := func(key string) (json.RawMessage, uint64, error) {
e, err := s.Get(key)
if err != nil || e == nil {
return nil, 0, err
}
return e.Value, e.Revision, nil
}
return Scan{Keys: s.Keys, Update: s.Update, Get: get, Delete: func(key string, revision uint64) error {
_, rev, err := get(key)
if err != nil {
return err
}
if rev != revision {
return stdio.ErrChanged
}
return s.Delete(key)
}}
}
// isAsk says a step is a pulled ask's: its key is <node>.<ask id> and its tool is "<kind> ask" (a hand-started step
// has the tool's name). An ask's key is the stream's to settle, never a restart's: the puller leaves an ask it was
// running unacknowledged so the stream redelivers it, and the asker's wait takes the first end it reads on the key.
// A key a gone process owns is deleted, never ended (EndStale); one with no owner is ended only once the asker's bound
// has passed (delivery.StepBound), when it is nobody's.
func isAsk(key string, s delivery.Step) bool {
_, _, err := delivery.ParseAskStepKey(key)
return err == nil && strings.HasSuffix(s.Tool, " ask")
}
// AskWhat says what an ask is, in one line: its kind, and its repository, head and levels, or its module and
// fingerprint.
func AskWhat(a delivery.StepAsk) string {
short := func(s string) string {
if len(s) > 12 {
return s[:12]
}
return s
}
str := func(m map[string]any, k string) string { v, _ := m[k].(string); return v }
parts := []string{a.Kind + " ask"}
switch {
case a.Check != nil:
parts = append(parts, str(a.Check.Fetch, "repository")+"@"+short(str(a.Check.Fetch, "commit")), "levels "+strings.Join(a.Check.Levels, ","))
if a.Check.Run != "" {
parts = append(parts, "run "+a.Check.Run)
}
case a.Build != nil:
parts = append(parts, "module "+a.Build.Module)
if r := str(a.Build.Fetch, "repository"); r != "" {
parts = append(parts, r+"@"+short(str(a.Build.Fetch, "commit")))
}
if a.Build.FP != "" {
parts = append(parts, "fp "+short(a.Build.FP))
}
case a.Compose != nil:
parts = append(parts, "compose", str(a.Compose.Fetch, "repository")+"@"+short(str(a.Compose.Fetch, "commit")))
}
return strings.Join(parts, ", ")
}
// endRunning writes step ended failed with why, only while its key is still at revision.
func endRunning(scan Scan, key string, step delivery.Step, revision uint64, why string) error {
step.State, step.Reason, step.Ended, step.Result = delivery.StepFailed, why, now().UTC(), nil
raw, err := delivery.EncodeStep(step)
if err != nil {
return err
}
_, err = scan.Update(key, json.RawMessage(raw), revision)
return err
}
// EndStale ends this node's running steps that no process of this module runs any more, and answers their ids. Only keys
// of node are read; a value that is no step is left alone.
// - A hand-started step with an owner (delivery.Step.Owner) that is not this process (Owner) is ended failed with
// RestartReason at once, whatever its age: its process is gone and nothing redelivers it, so nothing else would
// write its end (issue 1890, note 21).
// - A pulled ask's step with an owner that is not this process is deleted, not ended (novox/hq ADR 0525 §6): the
// stream redelivers an ask whose node stopped (the puller neither ends nor acknowledges one at its stop), and its
// asker takes the first end it reads, so an end written here would fail a check the redelivery would still run. Its
// asker's watch takes puts only and keeps waiting for the redelivery's key, its bound the backstop; the stale running
// key leaves the steps state, and with it the delivery view's and the checker's count of what runs (note 21: it
// held one of a build node's two slots).
// - A step owned by this process is left: it may run now. A redelivered ask begun on this node is owned by it.
// - A step with no owner (written by an older build) keeps the earlier rule: a hand-started step started before
// `before` (the start of this process) is ended; a pulled ask is left to the stream's redelivery (isAsk) until its
// start is older than delivery.StepBound, and then ended "past the step bound: its asker stopped waiting at <time>".
//
// Each end and delete is logged, naming the step, its owner and why. A key written by another writer since it was read
// (a redelivered ask's start, a real end) is skipped.
func EndStale(scan Scan, node string, before time.Time, logf func(string, ...any)) ([]string, error) {
keys, err := scan.Keys()
if err != nil {
return nil, fmt.Errorf("buildstep: listing the steps of %s: %w", node, err)
}
var ended []string
for _, k := range keys {
n, _, perr := delivery.ParseStepKey(k)
if perr != nil || n != node {
continue
}
raw, rev, err := scan.Get(k)
if err != nil {
return ended, fmt.Errorf("buildstep: reading the step %s: %w", k, err)
}
if len(raw) == 0 {
continue
}
step, err := delivery.DecodeStep(raw)
if err != nil || step.State != delivery.StepRunning {
continue
}
why, said := staleWhy(k, step, before)
if why == "" {
continue
}
done := "ended"
if step.Owner != "" && isAsk(k, step) {
done = "deleted, left to the stream's redelivery"
if scan.Delete == nil {
err = fmt.Errorf("the steps state takes no delete")
} else {
err = scan.Delete(k, rev)
}
} else {
err = endRunning(scan, k, step, rev, why)
}
if err != nil {
if errors.Is(err, stdio.ErrChanged) {
continue
}
return ended, fmt.Errorf("buildstep: ending the step %s: %w", k, err)
}
if logf != nil {
logf("the step %s (%s, started %s) was left running and is %s: %s; %s", k, step.Tool,
step.Started.UTC().Format(time.RFC3339), done, why, said)
}
ended = append(ended, step.ID)
}
return ended, nil
}
// staleWhy is why a running step is ended (or, for an ask with an owner, deleted) by EndStale and what the log says of
// its owner, or "" to leave it.
func staleWhy(key string, step delivery.Step, before time.Time) (why, said string) {
switch {
case step.Owner == Owner:
return "", ""
case step.Owner != "":
return RestartReason, "its owner " + step.Owner + " is not this process " + Owner
case isAsk(key, step):
if !step.Started.Add(delivery.StepBound).Before(now()) {
return "", ""
}
return "past the step bound: its asker stopped waiting at " + step.Started.Add(delivery.StepBound).UTC().Format("2006-01-02T15:04:05Z"),
"it names no owner (an older build wrote it)"
case step.Started.Before(before):
return RestartReason, "it names no owner (an older build wrote it) and started before this process"
}
return "", ""
}
// EndOne ends this node's running step id (or the key <node>.<id>) as failed with why. A step that is not running is
// refused, and so is one this process owns (Owner; for a step with no owner, one started since Boot: it may still run,
// whether a pulled ask or a step a tool started), one live reports as held, and one whose key changed since it was read.
func EndOne(scan Scan, node, id, why string, live func(id string) bool) (delivery.Step, error) {
id = strings.TrimSpace(id)
if n, rest, ok := strings.Cut(id, "."); ok {
if n != node {
return delivery.Step{}, fmt.Errorf("the step %q is of node %q, this module runs on %q", id, n, node)
}
id = rest
}
if strings.TrimSpace(why) == "" {
return delivery.Step{}, fmt.Errorf("say why the step is ended (why)")
}
key, err := delivery.StepKey(node, id)
if err != nil {
return delivery.Step{}, err
}
raw, rev, err := scan.Get(key)
if err != nil {
return delivery.Step{}, err
}
if len(raw) == 0 {
return delivery.Step{}, fmt.Errorf("no step %s in the steps state", key)
}
step, err := delivery.DecodeStep(raw)
if err != nil {
return delivery.Step{}, err
}
if step.State != delivery.StepRunning {
return delivery.Step{}, fmt.Errorf("the step %s is %s, not running", key, step.State)
}
if step.Owner == Owner || (step.Owner == "" && !step.Started.Before(Boot)) || (live != nil && live(id)) {
return delivery.Step{}, fmt.Errorf("the step %s was started by this process and may run now; only a step a previous process left is ended", key)
}
if err := endRunning(scan, key, step, rev, why); err != nil {
if errors.Is(err, stdio.ErrChanged) {
return delivery.Step{}, fmt.Errorf("the step %s changed while it was being ended; read it again", key)
}
return delivery.Step{}, err
}
step.State, step.Reason = delivery.StepFailed, why
return step, nil
}
// StaleAttempts and StaleRetry bound EndStaleAtStart's tries: the steps state is readable once the runtime serves.
var (
StaleAttempts = 6
StaleRetry = 5 * time.Second
)
// StaleServed is what EndStaleAtStart waits for before it reads the steps state: the runtime serving (stdio.Served).
// The conformance kit's left-running case (conformance.LeftRunning) hands it a closed channel, to run a module's own
// start sweep in its test.
var StaleServed = stdio.Served
// EndStaleAtStart is what a build module runs once at its start, in a goroutine: it waits for the runtime to serve,
// then ends this node's steps a previous process left running (EndStale), trying again when the state cannot be read
// yet. A step names its owner, so `before` (the start of this process) matters only for a step with no owner, written
// by an older build: such a step started before it is a previous process's. When the node's boot id cannot be read it
// says so once: Owner then tells processes apart by their pid and start only.
func EndStaleAtStart(ctx context.Context, scan Scan, node string, before time.Time, logf func(string, ...any)) {
select {
case <-ctx.Done():
return
case <-StaleServed():
}
if bootID == noBootID && logf != nil {
logf("the boot id of %s cannot be read (/proc/sys/kernel/random/boot_id): its steps' owner is %s, told apart by pid and start only", node, Owner)
}
for attempt := 1; ; attempt++ {
ended, err := EndStale(scan, node, before, logf)
if err == nil {
if logf != nil {
logf("%d steps of %s left running by a previous process ended", len(ended), node)
}
return
}
if logf != nil {
logf("ending the steps left running (try %d of %d): %v", attempt, StaleAttempts, err)
}
if attempt >= StaleAttempts {
return
}
select {
case <-ctx.Done():
return
case <-time.After(StaleRetry):
}
}
}
// EndStaleRound is EndStale as the module's periodic round runs it: at once, and only when the runtime serves (a round
// before that is skipped, the start's EndStaleAtStart and the next round cover it). It ends the steps a previous
// process left and the asks past delivery.StepBound.
func EndStaleRound(scan Scan, node string, logf func(string, ...any)) {
select {
case <-stdio.Served():
default:
return
}
if _, err := EndStale(scan, node, Boot, logf); err != nil && logf != nil {
logf("ending the steps left running: %v", err)
}
}
-162
View File
@@ -1,162 +0,0 @@
package buildstep
import (
"encoding/json"
"fmt"
"strings"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// scanOf is the steps a keys double holds, last value per key, its revision the count of its puts; a write through
// Update lands only at the revision read, as the state's does. race, when set, runs once before the next Update's check.
func scanOf(k *keys, race func(key string)) Scan {
return Scan{
Keys: func() ([]string, error) {
k.mu.Lock()
defer k.mu.Unlock()
var out []string
for key := range k.puts {
out = append(out, key)
}
return out, nil
},
Get: func(key string) (json.RawMessage, uint64, error) {
k.mu.Lock()
defer k.mu.Unlock()
s, ok := k.puts[key]
if !ok {
return nil, 0, nil
}
raw, err := delivery.EncodeStep(s[len(s)-1])
return raw, uint64(len(s)), err
},
Delete: func(key string, revision uint64) error {
k.mu.Lock()
defer k.mu.Unlock()
if uint64(len(k.puts[key])) != revision {
return stdio.ErrChanged
}
delete(k.puts, key)
return nil
},
Update: func(key string, value any, revision uint64) (uint64, error) {
if race != nil {
race(key)
}
k.mu.Lock()
n := uint64(len(k.puts[key]))
k.mu.Unlock()
if n != revision {
return 0, stdio.ErrChanged
}
return k.put(key, value)
},
}
}
// 1890 and 1890/AC6: EndStale ends this node's hand-started running steps of another owner at once, whatever their age,
// and deletes its asks of another owner (the stream redelivers them, ADR 0525 §6); of
// the steps with no owner (an older build's), the hand-started ones left running before the process began and the asks
// past the step bound (a younger ask is the stream's to redeliver); never a step this process owns or a key written
// meanwhile. EndOne ends one by hand, never one this process owns or started.
func TestEndStaleAndEndOne(t *testing.T) {
k := newKeys()
boot := time.Now()
oldBoot := Boot
Boot = boot
defer func() { Boot = oldBoot }()
gone := ownerOf("b00t", 4242, boot.Add(-time.Hour))
put := func(node, id, tool, state string, started time.Time, owner string) {
key, _ := delivery.StepKey(node, id)
_, _ = k.put(key, json.RawMessage(mustStep(t, delivery.Step{ID: id, Tool: tool, Node: node, State: state, Started: started, Owner: owner})))
}
put("ace", "old1", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour), "")
put("ace", "mjk3l9x0abcdef", "check ask", delivery.StepRunning, boot.Add(-time.Hour), "")
put("ace", "mjk3l9x0abcdee", "check ask", delivery.StepRunning, time.Now().Add(-5*time.Minute), "")
put("ace", "mjk3l9x0abcded", "check ask", delivery.StepRunning, time.Now().Add(-5*time.Minute), gone) // note 21's ghost
put("ace", "gone2", "build_go_test", delivery.StepRunning, boot.Add(time.Second), gone)
put("ace", "mine1", "check ask", delivery.StepRunning, boot.Add(-time.Hour), Owner)
put("ace", "raced1", "build_go_test", delivery.StepRunning, boot.Add(-time.Hour), "")
put("ace", "new1", "build_go_fetch", delivery.StepRunning, boot.Add(time.Second), "")
put("ace", "done1", "build_go_fetch", delivery.StepPassed, boot.Add(-time.Hour), gone)
put("g14", "other1", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour), gone)
// raced1 is ended by its own run between the read and the write.
raced := false
scan := scanOf(k, func(key string) {
if key == "ace.raced1" && !raced {
raced = true
put("ace", "raced1", "build_go_test", delivery.StepPassed, boot.Add(-time.Hour), "")
}
})
var said []string
ended, err := EndStale(scan, "ace", boot, func(f string, a ...any) { said = append(said, fmt.Sprintf(f, a...)) })
if err != nil || len(ended) != 4 {
t.Fatalf("ended %v, %v", ended, err)
}
if _, held := k.puts["ace.mjk3l9x0abcded"]; held {
t.Errorf("the ghost ask's key is still held: %+v", k.last("ace.mjk3l9x0abcded"))
}
for _, key := range []string{"ace.old1", "ace.gone2"} {
if s := k.last(key); s.State != delivery.StepFailed || s.Reason != RestartReason || s.Ended.IsZero() {
t.Errorf("%s = %+v", key, s)
}
}
if s := k.last("ace.mjk3l9x0abcdef"); s.State != delivery.StepFailed || !strings.HasPrefix(s.Reason, "past the step bound: its asker stopped waiting at ") {
t.Errorf("the old ask = %+v", s)
}
if log := strings.Join(said, "\n"); !strings.Contains(log, "ace.mjk3l9x0abcded") || !strings.Contains(log, "deleted, left to the stream's redelivery") || !strings.Contains(log, "its owner "+gone+" is not this process "+Owner) {
t.Errorf("the log does not name the ghost and its owner:\n%s", log)
}
for key, state := range map[string]string{"ace.mjk3l9x0abcdee": delivery.StepRunning, "ace.mine1": delivery.StepRunning, "ace.raced1": delivery.StepPassed,
"ace.new1": delivery.StepRunning, "ace.done1": delivery.StepPassed, "g14.other1": delivery.StepRunning} {
if s := k.last(key); s.State != state {
t.Errorf("%s = %s, want %s", key, s.State, state)
}
}
// EndOne: refuses without why, a held step, one this process owns or (with no owner) started, a step not running and
// another node's; ends a step a previous process left.
for _, c := range []struct{ id, why string }{{"old1", ""}, {"new1", "stuck"}, {"mine1", "stuck"}, {"done1", "x"}, {"g14.other1", "x"}} {
if _, err := EndOne(scan, "ace", c.id, c.why, nil); err == nil {
t.Errorf("ended %s with why %q", c.id, c.why)
}
}
put("ace", "old2", "build_go_fetch", delivery.StepRunning, boot.Add(time.Second), gone)
if _, err := EndOne(scan, "ace", "old2", "x", func(string) bool { return true }); err == nil {
t.Error("ended a held step")
}
if _, err := EndOne(scan, "ace", "ace.old2", "stuck for good", nil); err != nil {
t.Fatal(err)
}
if s := k.last("ace.old2"); s.State != delivery.StepFailed || s.Reason != "stuck for good" || s.Owner != gone {
t.Errorf("old2 = %+v", s)
}
if a, b := ownerOf("b", 1, boot), ownerOf("b", 1, boot.Add(time.Second)); a == b {
t.Errorf("two starts of one pid share the owner %q", a)
}
}
func mustStep(t *testing.T, s delivery.Step) []byte {
t.Helper()
raw, err := delivery.EncodeStep(s)
if err != nil {
t.Fatal(err)
}
return raw
}
// 1890: AskWhat names the repository, head and levels of a check ask and the module of a build ask.
func TestAskWhat(t *testing.T) {
check := delivery.StepAsk{Kind: "check", Check: &delivery.CheckJob{Fetch: map[string]any{"repository": "novox/mesh-sdk", "commit": "0123456789abcdef0123"}, Levels: []string{"unit", "integration"}}}
if got, want := AskWhat(check), "check ask, novox/mesh-sdk@0123456789ab, levels unit,integration"; got != want {
t.Errorf("check = %q", got)
}
build := delivery.StepAsk{Kind: "delivery", Build: &delivery.BuildJob{Module: "mesh-x", FP: "abcdef0123456789"}}
if got, want := AskWhat(build), "delivery ask, module mesh-x, fp abcdef012345"; got != want {
t.Errorf("build = %q", got)
}
}
-22
View File
@@ -1,22 +0,0 @@
// Package buildstep is what both build modules share: the runner of a long step that answers a tool call at once
// and ends on its key in the module's `steps` state (novox/hq ADR 0469 §2; issue 1471), the toolchain guard
// (issue 1473), the fetch of one commit over SSH (issue 1474), and the failure contract of a test level with the parsers
// of go test -json and node --test's TAP that fill it (failure.go, issue 1901; its conformance kit is package conformance).
package buildstep
import (
"crypto/rand"
"encoding/hex"
"strconv"
"time"
)
var now = time.Now
// NewID is a step id: the time in milliseconds in base 36 and six random hex digits, lower-case letters and
// digits only, so it is one key token and differs between two steps of one node even in one millisecond.
func NewID() string {
var b [3]byte
_, _ = rand.Read(b[:])
return strconv.FormatInt(now().UnixMilli(), 36) + hex.EncodeToString(b[:])
}
-72
View File
@@ -1,72 +0,0 @@
package buildstep
import (
"encoding/json"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// keys is a steps state in memory: every value put, in order, per key, and a signal for each end.
type keys struct {
mu sync.Mutex
puts map[string][]delivery.Step
ended chan string
}
func newKeys() *keys { return &keys{puts: map[string][]delivery.Step{}, ended: make(chan string, 16)} }
func (k *keys) put(key string, value any) (uint64, error) {
step, err := delivery.DecodeStep(value.(json.RawMessage))
if err != nil {
return 0, err
}
k.mu.Lock()
k.puts[key] = append(k.puts[key], step)
n := uint64(len(k.puts[key]))
k.mu.Unlock()
if step.State != delivery.StepRunning {
k.ended <- key
}
return n, nil
}
func (k *keys) last(key string) delivery.Step {
k.mu.Lock()
defer k.mu.Unlock()
s := k.puts[key]
return s[len(s)-1]
}
func (k *keys) waitEnd(t *testing.T) string {
t.Helper()
select {
case key := <-k.ended:
return key
case <-time.After(3 * time.Second):
t.Fatal("the step never ended on its key")
return ""
}
}
// 1471/AC5: ids made in one and the same millisecond differ, and each is one key token. The clock is fixed
// while no step runs, so no goroutine reads it meanwhile.
func TestTwoIDsInOneMillisecondDiffer(t *testing.T) {
was := now
defer func() { now = was }()
at := time.Date(2026, 10, 20, 12, 0, 0, 0, time.UTC)
now = func() time.Time { return at }
seen := map[string]bool{}
for i := 0; i < 50; i++ {
id := NewID()
if seen[id] {
t.Fatalf("id %q handed out twice", id)
}
seen[id] = true
if _, err := delivery.StepKey("ace", id); err != nil {
t.Errorf("id %q: %v", id, err)
}
}
}
-15
View File
@@ -1,15 +0,0 @@
package buildstep
// ModuleTree is what a merge holds of one module directory, as a build module's preview answers it (novox/hq ADR 0484
// clause 3, as ADR 0593 §2 changes it): its git tree id, the sha256 of its module.json, the sha256 of each lock file it
// has (go.sum, package-lock.json), and, of a module whose every artifact names its paths, each named path's git object
// id at the commit. The indexer alone computes from them (the named tree, the fingerprint); the build module's runner
// (mesh-build-go's buildstep.ModuleTrees) fills them.
type ModuleTree struct {
Tree string `json:"tree"`
Manifest string `json:"manifest"`
Locks map[string]string `json:"locks,omitempty"`
Paths map[string]string `json:"paths,omitempty"`
// Packages are no longer filled (ADR 0593 §2): kept so a preview written before still decodes.
Packages []string `json:"packages,omitempty"`
}
+3 -162
View File
@@ -1,129 +1,11 @@
package buscompose
import (
"encoding/json"
"fmt"
"slices"
"strings"
"time"
)
import "time"
// AccountsState is the grants part's state of wanted accounts, by bus user.
const AccountsState = "accounts"
// GrantsAccounts is that state as a reader names it (`<owner>.<name>`).
const GrantsAccounts = GrantsModule + "." + AccountsState
// AccountIssuedEvent is what the vault says when it minted an account: AccountIssued, never a value or a hash.
// AccountIssuedEvent is what the vault says when it minted an account: AccountIssued, never a value or a hash. The state of
// wanted accounts it mints from is the grants part's (grantsstate.AccountsState; novox/hq ADR 0634 §1).
const AccountIssuedEvent = "account-issued"
// AccountIssuedSubject is where it says it, as the runtime addresses mesh-vault's events.
const AccountIssuedSubject = "mesh.mod." + VaultSeat + ".event." + AccountIssuedEvent
// ClaimedSeat is one seat a module claims, as its runtime needs it (the controller's seatClaimed): the name, the scope (a
// node-scoped seat's verb carries the machine) and the verbs the runtime serves for it.
type ClaimedSeat struct {
Seat string `json:"seat"`
Scope string `json:"scope"`
Serves []string `json:"serves,omitempty"`
}
// WantedAccount is one key of the grants part's state `accounts`: an account the mesh needs, without its password.
type WantedAccount struct {
User string `json:"user"`
// Kind is KindModule (KindNodeTools for the runtime, named as a module is), KindProcess, KindProvisioner or
// KindLedger.
Kind Kind `json:"kind"`
Node string `json:"node"`
Module string `json:"module"`
// Process is a KindProcess account's process id.
Process string `json:"process,omitempty"`
// URL is the bus's address as that node reaches it, `nats://<host>:<port>`; Fingerprint the bus certificate's.
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
// Claims are a module's claims with the verbs each seat promises: none for a process or a provisioner client.
Claims []ClaimedSeat `json:"claims,omitempty"`
// At is when the grants part wrote the key: not compared (Same).
At time.Time `json:"at"`
}
// LedgersSecret is the tool runner's own secret holding its machine's ledger accounts, a map of module to credential
// (ADR 0597 §9).
const LedgersSecret = "ledgers"
// mintedKinds are the kinds of account the vault mints (ADR 0561 §1).
var mintedKinds = []Kind{KindModule, KindNodeTools, KindProcess, KindProvisioner, KindLedger}
// principal is the account as the composer names it.
func (w WantedAccount) principal() Principal {
return Principal{Kind: w.Kind, Node: w.Node, Module: w.Module, Process: w.Process}
}
// Valid says why w is not an account the vault may mint, or nil: a kind it mints, every name a subject token, the user
// the one the composer derives from them, and an address to reach the bus at.
func (w WantedAccount) Valid() error {
if !slices.Contains(mintedKinds, w.Kind) {
return fmt.Errorf("%s is a %s account, which the vault never mints (ADR 0561 §4)", w.User, w.Kind)
}
for what, v := range map[string]string{"node": w.Node, "module": w.Module} {
if !safeSubject.MatchString(v) {
return fmt.Errorf("%s: the %s %q is not a name the mesh writes", w.User, what, v)
}
}
if w.Kind == KindProcess && !safeSubject.MatchString(w.Process) {
return fmt.Errorf("%s: the process %q is not a name the mesh writes", w.User, w.Process)
}
if w.Kind != KindProcess && w.Process != "" {
return fmt.Errorf("%s: only a process's account names a process", w.User)
}
if want := w.principal().Username(); w.User != want {
return fmt.Errorf("%s is not the user of a %s account of %s on %s (that is %s)", w.User, w.Kind, w.Module, w.Node, want)
}
if !strings.HasPrefix(w.URL, "nats://") || len(w.URL) == len("nats://") {
return fmt.Errorf("%s names no bus address (url, nats://<host>:<port>)", w.User)
}
if len(w.Claims) > 0 && w.Kind != KindModule && w.Kind != KindNodeTools {
return fmt.Errorf("%s: a %s account holds no seat, so it carries no claims", w.User, w.Kind)
}
return nil
}
// Same says whether two wanted accounts want the same account: every field but when it was written.
func (w WantedAccount) Same(o WantedAccount) bool {
a, b := w, o
a.At, b.At = time.Time{}, time.Time{}
ra, _ := json.Marshal(a)
rb, _ := json.Marshal(b)
return string(ra) == string(rb)
}
// OwnSecret is the module's own secret the account is held in: `broker`, `broker-<process id>` or `provisioner-broker`
// (the controller's names, ADR 0436 §2, ADR 0349 §11); a ledger account in the tool runner's `ledgers`, which keeps every
// ledger account of its machine, by module (ADR 0597 §9).
func (w WantedAccount) OwnSecret() string {
switch w.Kind {
case KindLedger:
return LedgersSecret
case KindProcess:
return "broker-" + w.Process
case KindProvisioner:
return "provisioner-broker"
}
return "broker"
}
// ReadWantedAccount reads the key user of the state: refused when it is not a valid wanted account of that user.
func ReadWantedAccount(user string, raw []byte) (WantedAccount, error) {
var w WantedAccount
if err := json.Unmarshal(raw, &w); err != nil {
return WantedAccount{}, fmt.Errorf("%s %s is not a wanted account: %w", GrantsAccounts, user, err)
}
if w.User != user {
return WantedAccount{}, fmt.Errorf("%s %s names the user %q", GrantsAccounts, user, w.User)
}
return w, w.Valid()
}
// AccountIssued is the vault's account-issued: which account it minted, and its generation. Never a value or a hash.
type AccountIssued struct {
User string `json:"user"`
@@ -134,44 +16,3 @@ type AccountIssued struct {
Generation int `json:"generation"`
At time.Time `json:"at"`
}
// HeldCredential is the value the account's holder reads, exactly as the controller builds it today (issueWith,
// sealProcess and sealProvisionerClient): a module's `{url, fingerprint, node, module, user, password, claims}`, a
// process's without claims, a provisioner client's without module or claims; an empty fingerprint or claims left out.
func HeldCredential(w WantedAccount, password string) ([]byte, error) {
if err := w.Valid(); err != nil {
return nil, err
}
if password == "" {
return nil, fmt.Errorf("%s: an account is held with its password", w.User)
}
switch w.Kind {
case KindProcess:
return json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
Module string `json:"module"`
User string `json:"user"`
Password string `json:"password"`
}{URL: w.URL, Fingerprint: w.Fingerprint, Node: w.Node, Module: w.Module, User: w.User, Password: password})
case KindProvisioner:
return json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
User string `json:"user"`
Password string `json:"password"`
}{URL: w.URL, Fingerprint: w.Fingerprint, Node: w.Node, User: w.User, Password: password})
}
return json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
Module string `json:"module"`
User string `json:"user"`
Password string `json:"password"`
Claims []ClaimedSeat `json:"claims,omitempty"`
}{URL: w.URL, Fingerprint: w.Fingerprint, Node: w.Node, Module: w.Module, User: w.User, Password: password,
Claims: w.Claims})
}
-102
View File
@@ -1,102 +0,0 @@
package buscompose
import (
"fmt"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/health"
)
// ControllerLeftoverConsumers are the ended controller's durable consumers left on the bus, one leftover of the module
// mesh-delivery placed nowhere (issue 1126, milestone section 3), and mesh-catalog's of the controller's built-before,
// which its manifest no longer consumes and no part composes; each `<stream>.<consumer>`.
var ControllerLeftoverConsumers = []string{
"EVENTS." + ControllerName, "EVENTS." + MergesConsumerName, "EVENTS.novox_mesh-delivery", "EVENTS.novox_mesh-catalog",
"CONTROL." + ControllerName, DeadLetterNoticesStream + "." + ControllerName,
}
// ControllerLeftoverConsumersOn are ControllerLeftoverConsumers and, for each node given, the consumer the controller
// made for the node's engine (`EVENTS.<node>_mesh-host`, of restic's backed-up): the engine hears that event without a
// durable consumer, and the grants part composes none for it, so nothing reads it (issue 1126).
func ControllerLeftoverConsumersOn(nodes []string) []string {
out := slices.Clone(ControllerLeftoverConsumers)
for _, n := range nodes {
if safeSubject.MatchString(n) {
out = append(out, "EVENTS."+n+"_"+EngineModule)
}
}
return out
}
// endedController is the name the ended controller had as a module: its buckets are named for it (ADR 0590 §6).
const endedController = "mesh-controller"
// ControllerLeftoverBuckets are the ended controller's buckets, each by its local name (`KV_mesh-controller_<name>`).
var ControllerLeftoverBuckets = []string{"asked", "assignments", "builds", "calls", "checks-running", "composed",
"condition-history", "conditions", "declared-config", "declared-packages", "delivery-merges", "grants", "hand-acts",
"lease", "predefined", "rollouts", "states-announced"}
// ControllerLeftoverStreams are the ended controller's streams, by their names: CONTROL, ASSIGNMENTS, each of its buckets'
// and, for each node given, the declaration files' object bucket the controller wrote (ADR 0486 §2).
func ControllerLeftoverStreams(nodes []string) []string {
out := []string{"CONTROL", AssignmentsStream}
for _, b := range ControllerLeftoverBuckets {
out = append(out, "KV_"+BucketName(endedController, b))
}
for _, n := range nodes {
if safeSubject.MatchString(n) {
out = append(out, "OBJ_"+DeclarationFilesBucket+n)
}
}
return out
}
// BusRemovalGrants are the whole authority of an account whose bus declares removals (ADR 0564 Limits 4, ADR 0590 clause
// 6), each removal its grant: the node-engine's acts bucket read (STREAM.INFO with a subject filter lists what a removal
// would take, so the tools read before they remove) and its keys removed, which nothing else may empty when it fills; and
// the ended controller's leftovers, each consumer and each stream read and deleted by its name, for the nodes given (the
// declaration files' object buckets). A removal outside the set is refused, as the indexer's judge refuses it in a
// manifest (novox/hq ADR 0578 §1).
func BusRemovalGrants(removes, nodes []string) ([]string, error) {
var out []string
for _, r := range removes {
switch r {
case catalogue.RemovesEngineActs:
out = append(out, "$JS.API.STREAM.INFO.KV_"+EngineActsBucket, "$KV."+EngineActsBucket+".>")
case catalogue.RemovesControllerLeftovers:
for _, c := range ControllerLeftoverConsumersOn(nodes) {
out = append(out, "$JS.API.CONSUMER.INFO."+c, "$JS.API.CONSUMER.DELETE."+c)
}
for _, s := range ControllerLeftoverStreams(nodes) {
out = append(out, "$JS.API.STREAM.INFO."+s, "$JS.API.STREAM.DELETE."+s)
}
default:
return nil, fmt.Errorf("a bus account removes %q, which is not one of %s (novox/hq ADR 0564 Limits 4, ADR 0590)", r,
strings.Join([]string{catalogue.RemovesEngineActs, catalogue.RemovesControllerLeftovers}, ", "))
}
}
return out, nil
}
// WatchmanHears are the core subjects the watchman subscribes, with no durable consumer (ADR 0564 clauses 4, 5 and
// Limits 3, 11): the power seat's words (S1, ADR 0211), the providers' failing and recovered words (S8), each tool
// runner's beat (S11), the self-check's checked (S10), a consumer deleted and the bus module's slow consumers and closed
// connections (S9), the provisioner clients' reports (S21), the dead letters kept, and the lease write's events. Never the bus's API audit
// (`$JS.EVENT.ADVISORY.API`): it carries every API call's request and answer in the account, the vault's hashes among
// them (the lead, 2026-10-29).
func WatchmanHears() []string {
return []string{
"mesh.mod.power.event.*",
"mesh.mod.*.event.provisioner.failing",
"mesh.mod.*.event.provisioner.recovered",
"mesh.control.*.tools-alive",
"mesh.seat.mesh-self-check.event." + health.EventChecked,
"$JS.EVENT.ADVISORY.CONSUMER.DELETED.>",
"mesh.mod.nats.event.bus.slow-consumer",
"mesh.mod.nats.event.bus.connection-closed",
ProvAppliedSubjects,
deadLetterPrefix + ">",
}
}
+1 -5
View File
@@ -25,14 +25,10 @@ func TestTheBusNamesAgree(t *testing.T) {
sn != "mesh.ledger.m.records.snapshot" {
t.Fatalf("a ledger's subjects are %q and %q", h, sn)
}
// 1966/AC23: a wanted account of kind ledger for m on a names the user ledger.a.m and the own secret ledgers.
ledger := WantedAccount{User: "ledger.a.m", Kind: KindLedger, Node: "a", Module: "m", URL: "nats://10.0.0.1:4222"}
// 1966/AC23: a ledger account for m on a is named ledger.a.m.
if got := (Principal{Kind: KindLedger, Node: "a", Module: "m"}).Username(); got != "ledger.a.m" {
t.Fatalf("a ledger account is named %q", got)
}
if err := ledger.Valid(); err != nil || ledger.OwnSecret() != "ledgers" {
t.Fatalf("a wanted ledger account: %v, held in %q", err, ledger.OwnSecret())
}
// A seat's family is the shared events stream filtered to its subjects: a definition says the stream's age.
for _, st := range MeshStreams() {
if st.Name == EventsStream && int64(st.MaxAge) != seatdef.SharedEventsMaxAgeDays*24*60*60 {
-3
View File
@@ -46,6 +46,3 @@ type KeyedRead struct {
Read string `json:"read"`
Key string `json:"key"`
}
// ResolvedBucket is mesh-config's state of resolved values, keyed `<node>.<module>.<key>`.
const ResolvedBucket = "mesh-config_resolved"
-5
View File
@@ -1,7 +1,5 @@
package buscompose
import "git.novox.be/novox/mesh-sdk/go/catalogue"
// Declared is one module on one node, as composing its authority needs it.
type Declared struct {
Module string
@@ -102,6 +100,3 @@ type ProcessPrincipal struct {
// seat:<seat>.<verb>, or * for every tool on the mesh.
Invokes []string
}
// VaultSeat is the vault's seat: its holder alone asks the desks and the issuing providers in its own name.
const VaultSeat = catalogue.VaultSeat
-7
View File
@@ -1,7 +0,0 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// ArtifactSweepSeat is the seat artifact-store-tools holds. Not the seat mesh-artifact-store, the store itself held by
// distribution, and not the provision artifact-store it delivers.
const ArtifactSweepSeat = contractseats.MeshArtifactSweep
+2 -142
View File
@@ -1,146 +1,6 @@
package catalogue
import (
"strings"
contractseats "git.novox.be/novox/mesh-contracts/go/seats"
)
// ArtifactStoreScheme marks a reference to something in the mesh's artifact store, kept without
// the store's address.
// the store's address. How a reference is recorded, read and routed (the registry seat's form beside this one) is the
// registry clients' (mesh-delivery's contracts, `containerimageregistryclient/reference`; ADR 0634 §4).
const ArtifactStoreScheme = ArtifactStoreProvision + "://"
// Recorded is a reference as the mesh records it: the artifact store's address, if the builder wrote
// one, taken off.
//
// For a reference the builder announced — one it pushed to the store — which is the only kind
// this is called on. An image the builder named `<host>/<path>@sha256:…` is kept as its path; an
// archive it named `http://<host>/v2/<path>/blobs/<digest>` likewise. A reference with no address
// in it — an image id from a genesis build that had nowhere to publish, a package version — is
// what it was.
func Recorded(reference string) string {
if _, _, seat := InRegistrySeat(reference); seat || strings.HasPrefix(reference, ArtifactStoreScheme) {
return reference
}
if rest, isURL := strings.CutPrefix(reference, "http://"); isURL {
if _, path, ok := strings.Cut(rest, "/v2/"); ok && strings.Contains(path, "/blobs/") {
return ArtifactStoreScheme + path
}
return reference
}
host, path, ok := strings.Cut(reference, "/")
if !ok || !isRegistryHost(host) || !strings.Contains(path, "@sha256:") {
return reference
}
return ArtifactStoreScheme + path
}
// isRegistryHost is the runtime's own rule for reading the first component of a reference as a
// registry rather than as a namespace: it has a dot or a port in it, or it is localhost.
func isRegistryHost(component string) bool {
return component == "localhost" || strings.ContainsAny(component, ".:")
}
// InArtifactStore reports whether a reference is a kept one, and what it names there: `<module>/<artifact>@<digest>`
// for an image, `<module>/<artifact>/blobs/<digest>` for a bundle or an archive. **It reads both forms** (novox/hq ADR
// 0605 §9): the older `artifact-store://<path>`, and the form a registry seat answers, `<seat>/<module>/<artifact>@<digest>`
// through container-image-registry or oci-artifact-registry, whose blob it names the way the older form did.
func InArtifactStore(reference string) (path string, kept bool) {
if seat, rest, ok := InRegistrySeat(reference); ok {
if seat == OciArtifactRegistrySeat {
name, digest, _ := strings.Cut(rest, "@")
return name + "/blobs/" + digest, true
}
return rest, true
}
return strings.CutPrefix(reference, ArtifactStoreScheme)
}
// RegistrySeats are the seats a reference of the registry form names first, the two that front the artifact store
// (novox/hq ADR 0605 §1, §9).
var RegistrySeats = []string{ContainerImageRegistrySeat, OciArtifactRegistrySeat}
// The four registry seats (novox/hq ADR 0605 §1), named here so the rest of mesh-sdk takes them from the seat set
// through catalogue alone (ADR 0625 §3).
const (
NpmPackageRegistrySeat = contractseats.NpmPackageRegistry
GoPackageRegistrySeat = contractseats.GoPackageRegistry
ContainerImageRegistrySeat = contractseats.ContainerImageRegistry
OciArtifactRegistrySeat = contractseats.OciArtifactRegistry
)
// PublishSeats are the seats an artifact is published through (ADR 0605 §1).
var PublishSeats = []string{NpmPackageRegistrySeat, GoPackageRegistrySeat, ContainerImageRegistrySeat, OciArtifactRegistrySeat}
// The seats of the vault, the provisioner, the hand acts' record and the conditions' keeper, as the seat set names them,
// for the rest of mesh-sdk (ADR 0625 §3).
const (
VaultSeat = contractseats.MeshVault
ProvisionerSeat = contractseats.MeshProvisioner
)
// SeatReference is the reference a build names an image, bundle, archive or upstream copy by once it published it
// through a registry seat (novox/hq ADR 0605 §9): `<seat>/<module>/<artifact>@sha256:<digest>`, as InRegistrySeat reads
// it back. The store's address is left out, as in the older form: a reader routes it (Routed).
func SeatReference(seat, module, artifact, digest string) string {
return seat + "/" + module + "/" + artifact + "@" + digest
}
// InRegistrySeat reads a reference in the form a registry seat answers, `<seat>/<module>/<artifact>@sha256:<digest>`:
// the seat it came through and the rest. A seat's name holds no dot or port, so no image of an outside registry reads
// as one, and a kept `artifact-store://` reference is not one.
//
// It also reads the holder's own naming of an upstream copy through container-image-registry,
// `container-image-registry/upstream/<host>/<path>@sha256:<digest>`, whose rest InUpstream reads.
func InRegistrySeat(reference string) (seat, path string, ok bool) {
first, rest, found := strings.Cut(reference, "/")
if !found || !strings.Contains(rest, "@sha256:") {
return "", "", false
}
if _, _, upstream := InUpstream(rest); upstream && first == ContainerImageRegistrySeat {
return first, rest, true
}
if strings.Count(rest, "/") != 1 {
return "", "", false
}
for _, s := range RegistrySeats {
if first == s {
return s, rest, true
}
}
return "", "", false
}
// UpstreamPrefix opens the holder's own name for an upstream copy it keeps: `upstream/<host>/<path>@<digest>`, after
// `container-image-registry/` in the seat's form and after `artifact-store://` in the older one.
const UpstreamPrefix = "upstream/"
// InUpstream reads a path in the holder's naming of an upstream copy, `upstream/<host>/<path>@sha256:<digest>` (the path
// InRegistrySeat or InArtifactStore gives back): the upstream registry host and the image's path there, with its digest.
// A host is read as the runtime reads one (a dot, a port, or localhost).
func InUpstream(path string) (host, image string, ok bool) {
rest, found := strings.CutPrefix(path, UpstreamPrefix)
if !found {
return "", "", false
}
host, image, found = strings.Cut(rest, "/")
name, digest, pinned := strings.Cut(image, "@")
if !found || !isRegistryHost(host) || name == "" || strings.HasPrefix(name, "/") || strings.HasSuffix(name, "/") ||
strings.Contains(name, "//") || !pinned || !strings.HasPrefix(digest, "sha256:") || strings.Contains(digest, "/") {
return "", "", false
}
return host, image, true
}
// Routed is a kept reference as a machine fetches it, through the artifact store at `address`
// (host:port). A reference that is not a kept one is what it was.
func Routed(reference, address string) string {
path, kept := InArtifactStore(reference)
if !kept {
return reference
}
if strings.Contains(path, "/blobs/") {
return "http://" + address + "/v2/" + path
}
return address + "/" + path
}
+2 -66
View File
@@ -1,70 +1,6 @@
package catalogue
import (
"strings"
"testing"
)
// A build is recorded by what it is; where it is pushed is composed where it is used (novox/hq
// 04-ISSUES/102).
import "strings"
// digest is an artifact digest the tests build references with.
var digest = "sha256:" + strings.Repeat("d", 64)
func TestAReferenceIsRecordedWithoutTheStoresAddress(t *testing.T) {
cases := map[string]string{
"anchor.internal:5100/gitea/server@" + digest: ArtifactStoreScheme + "gitea/server@" + digest,
"localhost:5000/gitea/server@" + digest: ArtifactStoreScheme + "gitea/server@" + digest,
"http://anchor.internal:5100/v2/gitea/config/blobs/" + digest: ArtifactStoreScheme + "gitea/config/blobs/" + digest,
ArtifactStoreScheme + "gitea/server@" + digest: ArtifactStoreScheme + "gitea/server@" + digest,
digest: digest,
"@novox/sdk@1.2.3": "@novox/sdk@1.2.3",
"gitea/gitea@" + digest: "gitea/gitea@" + digest,
"https://registry.example/v2/gitea/config/blobs/" + digest: "https://registry.example/v2/gitea/config/blobs/" + digest,
// The form a registry seat answers is recorded as it is (ADR 0605 §9).
"container-image-registry/gitea/server@" + digest: "container-image-registry/gitea/server@" + digest,
"oci-artifact-registry/gitea/config@" + digest: "oci-artifact-registry/gitea/config@" + digest,
}
for announced, want := range cases {
if got := Recorded(announced); got != want {
t.Errorf("Recorded(%q) = %q, want %q", announced, got, want)
}
}
// And every reader reads both forms: a kept reference and the seat's are fetched from the same place.
for reference, want := range map[string]string{
ArtifactStoreScheme + "gitea/server@" + digest: "10.0.0.1:5000/gitea/server@" + digest,
"container-image-registry/gitea/server@" + digest: "10.0.0.1:5000/gitea/server@" + digest,
ArtifactStoreScheme + "gitea/config/blobs/" + digest: "http://10.0.0.1:5000/v2/gitea/config/blobs/" + digest,
"oci-artifact-registry/gitea/config@" + digest: "http://10.0.0.1:5000/v2/gitea/config/blobs/" + digest,
"registry.example/gitea/server@" + digest: "registry.example/gitea/server@" + digest,
"mesh-artifact-store/gitea/server@" + digest: "mesh-artifact-store/gitea/server@" + digest,
// An upstream copy under the holder's own naming, in both forms.
"container-image-registry/upstream/quay.io:443/org/app@" + digest: "10.0.0.1:5000/upstream/quay.io:443/org/app@" + digest,
ArtifactStoreScheme + "upstream/quay.io:443/org/app@" + digest: "10.0.0.1:5000/upstream/quay.io:443/org/app@" + digest,
} {
if got := Routed(reference, "10.0.0.1:5000"); got != want {
t.Errorf("Routed(%q) = %q, want %q", reference, got, want)
}
}
// A build module names what it published in the seat's form, and reads it back as the seat it came through.
for _, seat := range RegistrySeats {
ref := SeatReference(seat, "gitea", "config", digest)
if got, path, ok := InRegistrySeat(ref); !ok || got != seat || path != "gitea/config@"+digest {
t.Errorf("SeatReference(%s) = %q, read back as %q %q %v", seat, ref, got, path, ok)
}
}
// The holder's naming of an upstream copy reads through the image seat only, and InUpstream reads its host and path.
for reference, want := range map[string]bool{
"container-image-registry/upstream/quay.io:443/org/app@" + digest: true,
"oci-artifact-registry/upstream/quay.io:443/org/app@" + digest: false,
"container-image-registry/upstream/org/app@" + digest: false,
"container-image-registry/upstream/quay.io/app": false,
} {
_, path, ok := InRegistrySeat(reference)
if ok != want {
t.Errorf("InRegistrySeat(%q) read %v, want %v", reference, ok, want)
}
if host, image, up := InUpstream(path); ok && (!up || host != "quay.io:443" || image != "org/app@"+digest) {
t.Errorf("InUpstream(%q) = %q %q %v", path, host, image, up)
}
}
}
+5 -1
View File
@@ -17,7 +17,11 @@ func TestRenamingASeatDidNotRenameTheInterfaceItDelivers(t *testing.T) {
for _, pair := range []struct{ seat, delivers string }{
{"git", "git"},
{"npm-package-registry", "npm-package-registry"},
{"mesh-artifact-store", "artifact-store"},
// The store's seats deliver nothing since the provision artifact-store retired (novox/hq ADR 0632 §2): the
// store's address is its holder's listen, read through the seat.
{"mesh-artifact-store", ""},
{"container-image-registry", ""},
{"oci-artifact-registry", ""},
{"mesh-store", "postgres-database"},
{"mesh-broker", "mesh-bus"},
} {
+19 -3
View File
@@ -117,9 +117,9 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
// it reached it by, and a manifest carrying that address names an installation —
// registration refused node-tools for exactly this on 2026-10-02. The build record
// already keeps the store-relative form; the resolved manifest keeps the same, and
// composition routes it through the store a machine reaches (Routed).
// composition routes it through the store a machine reaches (the registry clients' Routed).
out.Bundles = append(out.Bundles, Bundle{
Name: a.Name, Source: Recorded(made.Reference), Digest: made.Digest,
Name: a.Name, Source: kept(made.Reference), Digest: made.Digest,
Language: a.Language, Entrypoints: append([]string(nil), a.Entrypoints...),
Loads: loads, Env: copyWords(a.Env), Launchers: copyWords(made.Launchers),
Binary: BinaryOf(a), Bus: a.Bus,
@@ -171,7 +171,7 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
// how they were made — one packed as it stood, the other compiled first — and a
// machine has no reason to care which.
// Kept, not routed, for the reason the bundles above are (ADR 0155).
filled["source"] = Recorded(artifact.Reference)
filled["source"] = kept(artifact.Reference)
filled["digest"] = artifact.Digest
// **And `${version}`, so a resource can name a place that is this build's alone**
// (novox/hq ADR 0141, 04-ISSUES/142). A component is unpacked into a directory named
@@ -363,3 +363,19 @@ func BundleLoads(m Manifest, a Artifact, run map[string]bool) []string {
}
return loads
}
// kept is a bundle's or an archive's reference without the artifact store's address, if the builder wrote one (ADR 0155:
// a resolved manifest names no installation). Only the address is read here: an `http://<host>/v2/<path>/blobs/<digest>`
// becomes `artifact-store://<path>/blobs/<digest>`; a kept reference, or one in a registry seat's form (whose first part,
// a seat's name, has no dot or port), is what it was. How the registry seats' forms are read is the registry clients'.
func kept(reference string) string {
if strings.HasPrefix(reference, ArtifactStoreScheme) {
return reference
}
if rest, isURL := strings.CutPrefix(reference, "http://"); isURL {
if _, path, ok := strings.Cut(rest, "/v2/"); ok && strings.Contains(path, "/blobs/") {
return ArtifactStoreScheme + path
}
}
return reference
}
-3
View File
@@ -1,7 +1,5 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
import "encoding/json"
// CatalogSeat is the seat of the catalog: what the mesh holds of every module (novox/hq ADR 0604 §1). Its states are
@@ -10,7 +8,6 @@ import "encoding/json"
// no verb. It names no event. What must be built again is no verb: a module whose fingerprint differs from its newest
// build, which the delivery view derives. The verbs' answers are CatalogModules, CatalogOne, CatalogProvides and
// CatalogDependents, whose field names are those of the catalog module's tools they replace.
const CatalogSeat = contractseats.MeshCatalog
// CatalogEntry is one module of the catalog: its repository, its directory there ("." for the root), the merge commit it
// was last changed at, and its new-form fingerprint or why it has none.
-3
View File
@@ -1,9 +1,6 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// CheckerSeat is the seat mesh-delivery-checker holds.
//
// Deprecated: a whole seat's name is mesh-contracts' (seats.MeshDeliveryChecker); kept for its importers until their
// next change takes it from there (testdata/seat-names.txt).
const CheckerSeat = contractseats.MeshDeliveryChecker
+1 -6
View File
@@ -1,12 +1,7 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// The two seats.
const (
WantsSeat = contractseats.MeshWants
HandActsSeat = contractseats.MeshHandActs
)
const ()
// controlRecordsDecision is the record that made the two seats.
const controlRecordsDecision = "novox/hq ADR 0565"
-17
View File
@@ -1,22 +1,5 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// DeliverySeat is the seat mesh-delivery holds.
const DeliverySeat = contractseats.MeshDelivery
// RemovedDeliveryVerbs are the delivery seat's verbs release 6 removed (novox/hq ADR 0539 clause 17): with no plan the
// gate keeps the merge's states and derives nothing after the merge. Each stays in deliveryVerbs as an optional row
// with no input, so the holder of before and the holder of release 6 both hold the seat and a caller is answered by
// name with its replacement, given here.
var RemovedDeliveryVerbs = map[string]string{
"times": "the delivery view (nox-delivery): each machine's want and have, and since when",
"what-if": "the merge check's composition, which composes every machine a change touches",
"release": "the assigner's asks (hold, restore, backup, record), answered on the phone or in the delivery view",
"stop": "a bad fingerprint is held and its canary set back by the assigner (ADR 0429 §6, ADR 0539 clause 18)",
"retire-history": "nothing to retire: the gate keeps no delivery after its merge",
}
// SeatSays is whether a consumed `<seat>.<event>` names one of the seat's own events: the seat emits it,
// exactly or by one of its patterns (`log.*`). A consumed name whose emitter is a seat and whose event the
// seat does not emit is the event of the module of that name (novox/hq ADR 0239): mesh-delivery is a seat
@@ -4,7 +4,7 @@ import "testing"
// The node-engine's compose-candidate verb (ADR 0593 §7) is optional, asks the candidate, and takes input as a boolean, as the engine and the checker read it.
func TestTheEngineServesComposeCandidate(t *testing.T) {
for _, v := range EngineVerbs() {
for _, v := range EngineVerbs(EngineSeat) {
if v.Name != "compose-candidate" {
continue
}
+8 -25
View File
@@ -1,32 +1,15 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// EngineSeat is the node seat every node's engine holds, from the seat set. The seat is defined whole in mesh-contracts
// (its contract, go/seats/nodeengine, and its client, go/clients/nodeengineclient): its verbs' and events' names and
// message types are taken from there (ADR 0627 §4).
const EngineSeat = contractseats.NodeEngine
// EngineVerbs are the node-engine seat's verbs, in the order ADR 0563 §1 names them: its definition's, read from the
// generated seat list (design 26), each with its description and its input's schema as discovery shows them.
func EngineVerbs() []Verb {
s, _ := seatIn(defaultSeats, EngineSeat)
// EngineVerbs are the node-engine seat's verbs as the seat list the host gave has them (ADR 0563 §1; design 26), each
// with its description and its input's schema as discovery shows them. The seat's name is its contract's
// (mesh-contracts' nodeengine), never this package's (ADR 0634 §4, ADR 0635 §4): the caller names it.
func EngineVerbs(seat string) []Verb {
s, _ := SeatNamed(seat)
return append([]Verb(nil), s.Serves...)
}
// EngineEvents are what the seat says: of an act, each naming the act, the path or access, the caller as the runtime
// proved it and the ask (ADR 0563 §1), and that what its engine reads of a manifest changed (ADR 0618 §4).
var EngineEvents = func() []string {
s, _ := seatIn(defaultSeats, EngineSeat)
// EngineEvents are what the node-engine seat says, as the seat list the host gave has them.
func EngineEvents(seat string) []string {
s, _ := SeatNamed(seat)
return append([]string(nil), s.Emits...)
}()
// seatIn is the seat of that name in a list.
func seatIn(list []Seat, name string) (Seat, bool) {
for _, s := range list {
if s.Name == name {
return s, true
}
}
return Seat{}, false
}
-3
View File
@@ -1,9 +1,6 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// EnrolmentSeat is the seat mesh-enrolment.
const EnrolmentSeat = contractseats.MeshEnrolment
// The seat's six verbs (ADR 0587 §1, §4-§8).
const (
+3 -5
View File
@@ -1,14 +1,12 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// EnvironmentSeat and LoginShellSeat are the seats whose holders may place what the modules
// contributed: the account's environment, and the login shell's code.
const (
EnvironmentSeat = contractseats.NodeEnvironment
LoginShellSeat = contractseats.NodeLoginShell
EnvironmentSeat = "node-environment"
LoginShellSeat = "node-login-shell"
// PowerSeat is the seat whose holder places code for the power moments (novox/hq ADR 0211).
PowerSeat = contractseats.NodePower
PowerSeat = "node-power"
)
// Where an environment entry on PATH goes: before the account's existing PATH, or after it.
+1 -3
View File
@@ -1,9 +1,7 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// FilesSeat is the role that reads and manages a node's files and accounts (novox/hq ADR 0394).
const FilesSeat = contractseats.NodeFiles
const FilesSeat = "node-files"
// FilesOwnedPaths is the word the holder of node-files is given beside the secret places: a JSON object from
// each absolute path a module owns on its node to "module (tool to use)" (novox/hq ADR 0394 §5).
+11 -13
View File
@@ -3,8 +3,6 @@ package catalogue
import (
"fmt"
"strings"
contractseats "git.novox.be/novox/mesh-contracts/go/seats"
)
// The graphical session's seats (novox/hq ADR 0208): one module per piece of software, each piece's
@@ -12,17 +10,17 @@ import (
// a role rather than each inventing one — and a machine running two of one role is refused at
// assignment instead of found by two bars on one screen.
const (
LoginManagerSeat = contractseats.NodeLoginManager
DisplayServerSeat = contractseats.NodeDisplayServer
DisplaySessionSeat = contractseats.NodeDisplaySession
TerminalEmulatorSeat = contractseats.NodeTerminalEmulator
LauncherSeat = contractseats.NodeLauncher
NotifierSeat = contractseats.NodeNotifier
LockScreenSeat = contractseats.NodeLockScreen
ClipboardSeat = contractseats.NodeClipboard
BarSeat = contractseats.NodeBar
CompositorSeat = contractseats.NodeCompositor
SecretServiceSeat = contractseats.NodeSecretService
LoginManagerSeat = "node-login-manager"
DisplayServerSeat = "node-display-server"
DisplaySessionSeat = "node-display-session"
TerminalEmulatorSeat = "node-terminal-emulator"
LauncherSeat = "node-launcher"
NotifierSeat = "node-notifier"
LockScreenSeat = "node-lock-screen"
ClipboardSeat = "node-clipboard"
BarSeat = "node-bar"
CompositorSeat = "node-compositor"
SecretServiceSeat = "node-secret-service"
)
// BarKindBlock is the bar's one received kind: a block (novox/hq ADR 0255).
+1 -1
View File
@@ -211,7 +211,7 @@ func handedBy(m Manifest, archive Built) *Handed {
return nil
}
dir := m.TemplatesDir()
h := &Handed{Source: Recorded(archive.Reference), Digest: archive.Digest, Templates: map[string]string{}}
h := &Handed{Source: kept(archive.Reference), Digest: archive.Digest, Templates: map[string]string{}}
for _, r := range m.Resources {
name, has, wrong := templateOf(m.Module, dir, r)
if !has || wrong != "" {
+1 -8
View File
@@ -1,14 +1,7 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// The four health seats.
const (
ConditionsSeat = contractseats.MeshConditions
WatchdogsSeat = contractseats.MeshWatchdogs
SelfCheckSeat = contractseats.MeshSelfCheck
HealersSeat = contractseats.MeshHealers
)
const ()
// healthDecision is the record that made the four seats.
const healthDecision = "novox/hq ADR 0564"
+6 -13
View File
@@ -15,7 +15,7 @@ func busSeatDelivering(t *testing.T, delivers string) {
t.Helper()
was := Seats()
t.Cleanup(func() { UseSeats(was) })
UseSeats([]Seat{{Name: "mesh-broker", Scope: ScopeMesh, Delivers: delivers, Decision: "test"}})
UseSeats([]Seat{{Name: "mesh-broker", Scope: ScopeMesh, Delivers: delivers, Decision: "test", Serves: verbsNamed(brokerVerbs())}})
}
// brokerVerbs are the bus seat's verbs as a literal: mesh-sdk imports no seat's contract module, a test neither
@@ -80,17 +80,10 @@ func TestCanHoldJudgesClaimScopeAndWhatTheSeatDelivers(t *testing.T) {
}
}
// The store's seat rows have no protocol columns yet; loading them must not drop the protocol the
// bus is derived from, or no role's work queue is ever raised (found live, 2026-09-28).
func TestAStoreRowWithoutAProtocolKeepsTheCompiledOne(t *testing.T) {
was := Seats()
t.Cleanup(func() { UseSeats(was) })
UseSeats([]Seat{{Name: "node-build-go", Scope: ScopeNode, Decision: "row"}})
got, ok := SeatNamed("node-build-go")
if !ok || len(got.Accepts) == 0 || !got.PulledByKind || !got.Yields {
t.Fatalf("the Go build seat lost what it accepts when loaded from the store: %+v", got)
}
if got.Decision != "row" {
t.Fatalf("the store's own columns were not kept: %+v", got)
func verbsNamed(names []string) []Verb {
var out []Verb
for _, n := range names {
out = append(out, Verb{Name: n})
}
return out
}
-7
View File
@@ -31,11 +31,4 @@ func TestTheLoginShellsExecuteIsOptionalSoAMachineMayWithholdIt(t *testing.T) {
}
}
t.Run("compiled", check)
t.Run("read back from the store", func(t *testing.T) {
before := seats
t.Cleanup(func() { seats = before })
UseSeats([]Seat{{Name: LoginShellSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0204",
Serves: []Verb{{Name: "execute"}}}})
check(t)
})
}
+4 -6
View File
@@ -1331,12 +1331,10 @@ const (
ArtifactPackage = "package"
)
// ArtifactStoreProvision is the name a module offers when it is the mesh's store for what modules
// ship — images, and archives, which are directories from a repository packed as blobs.
//
// Named here because a rule depends on it: what provides this cannot be delivered through it
// (novox/hq 04-ISSUES/029). A string compared in one place is a convention; a string a rule turns
// on is a fact, and it should be written once.
// ArtifactStoreProvision is the name the mesh's store for what modules ship (images, and archives,
// which are directories from a repository packed as blobs) was offered under until the provision
// retired (novox/hq ADR 0632 §2): no seat delivers it and no module provides it. It stays as the
// scheme of the older reference form, artifact-store://, which done keys keep (ArtifactStoreScheme).
const ArtifactStoreProvision = "artifact-store"
// Listening is one endpoint a module serves: a port it accepts connections on, and what may be said
+72 -7
View File
@@ -5,6 +5,7 @@ import (
_ "embed"
"encoding/json"
"reflect"
"regexp"
"slices"
"sort"
"strings"
@@ -38,7 +39,7 @@ var referenceForms = []string{"access", "backup", "bound", "collected", "config"
"contribution", "dir", "machine", "port", "seat", "secret", "setting"}
// ReferenceFormSeat names the reference form a registry seat answers for a build's artifact,
// `<seat>/<module>/<artifact>@sha256:<digest>` (novox/hq ADR 0605 §9; InRegistrySeat), as the reader capability
// `<seat>/<module>/<artifact>@sha256:<digest>` (novox/hq ADR 0605 §9; the registry clients' reference forms), as the reader capability
// `form:seat-reference` a node-engine built with this contract states (ADR 0618 §1, ADR 0624 §1): the builder writes it
// into a build key only for the nodes whose engines state it. The older reference forms a node-engine read before any
// stated what it reads are the baseline's; they are the builder's to name (delivery's artifact forms).
@@ -47,6 +48,37 @@ const ReferenceFormSeat = "seat-reference"
// artifactReferenceForms are the reference forms of a build's artifacts this contract reads beyond the baseline's.
var artifactReferenceForms = []string{ReferenceFormSeat}
// ReferenceFormSeatReach names how a node-engine built with this contract fills `${seat:<seat>:reach}`, as the reader
// capability `form:seat-reach` (novox/hq ADR 0632 §1): from the seat holder placement names and the listen its manifest
// opens to the mesh, the store's three seats among them, never from a module serving the retired provision
// artifact-store. An engine before it filled the store's reach only from that provision, so a manifest naming a seat's
// reach, and the store holder's own manifest, need it and are held from a node whose engine does not state it
// (ManifestNeeds).
const ReferenceFormSeatReach = "seat-reach"
// storeHolderSeat is the seat whose holder's own manifest needs form:seat-reach (novox/hq ADR 0632 §2): the one seat
// name this contract reads a rule by, and the one seat name of the package-list test's exceptions (S-8).
const storeHolderSeat = "container-image-registry"
// claimsAtMesh says whether a manifest claims seat at the mesh's scope (a claim with no scope is the seat's own). The
// store holder's own manifest, a claim of storeHolderSeat, needs form:seat-reach this way (ADR 0632 §2): read
// on every node, it leaves an engine before the reading without the old way to find the store, so it is held from every
// node whose engine does not state it.
func claimsAtMesh(top map[string]any, seat string) bool {
for _, c := range asList(top["claims"]) {
obj, _ := c.(map[string]any)
name, _ := obj["name"].(string)
scope, _ := obj["scope"].(string)
if hs, ok := SeatNamed(name); ok && hs.Name == seat && (scope == "" || scope == ScopeMesh) {
return true
}
}
return false
}
// ofSeatReach is a manifest's value naming a seat's reach, `${seat:<seat>:reach}`.
var ofSeatReach = regexp.MustCompile(`\$\{seat:[a-z0-9][a-z0-9-]*:reach\}`)
// manifestTopWords are the top-level keys Manifest.UnmarshalJSON reads apart from the fields of manifestFields: each
// read whole, its inside a map of the module's own names (requirements, local names) and values, no words of the
// contract's beneath it. `requires` and `config` are read as their fields are (manifestNeeds).
@@ -73,7 +105,8 @@ func ManifestReads() []string {
for _, f := range artifactReferenceForms {
set[ReadForm+f] = true
}
for _, s := range DefaultSeats() {
set[ReadForm+ReferenceFormSeatReach] = true
for _, s := range Seats() {
set[ReadSeat+s.Name] = true
}
return sortedSet(set)
@@ -134,10 +167,9 @@ func ManifestNeeds(raw []byte) ([]string, error) {
return nil, err
}
used := map[string]bool{}
for _, f := range referenceForms {
if bytes.Contains(raw, []byte("${"+f+":")) {
used[ReadForm+f] = true
}
textNeeds(raw, used)
if claimsAtMesh(top, storeHolderSeat) {
used[ReadForm+ReferenceFormSeatReach] = true
}
if _, has := top[ConfigField]; has || takesConfigKey.Match(raw) {
used[ReadForm+"config"] = true
@@ -188,7 +220,7 @@ func ManifestNeeds(raw []byte) ([]string, error) {
// and a bucket's share with a seat.
func seatNamesUsed(top map[string]any) []string {
known := map[string]bool{}
for _, s := range DefaultSeats() {
for _, s := range Seats() {
known[s.Name] = true
}
set := map[string]bool{}
@@ -397,6 +429,39 @@ func asList(v any) []any {
return list
}
// textNeeds adds the reference forms a manifest's or a template's text writes to used: a form the engine fills in a
// file's text is read there whether the text is inline or in a template file.
func textNeeds(text []byte, used map[string]bool) {
for _, f := range referenceForms {
if bytes.Contains(text, []byte("${"+f+":")) {
used[ReadForm+f] = true
}
}
if ofSeatReach.Match(text) {
used[ReadForm+ReferenceFormSeatReach] = true
}
}
// ModuleNeeds is ManifestNeeds of a module whose files take their text from templates (template.go): what the manifest
// needs and what each template's text writes, by template path (TemplatePaths), sorted. A module's needs are both: an
// engine renders the template's text as it renders inline content, so a template naming ${seat:x:reach} needs the
// reading as the same content inline does, though the manifest itself is unchanged (novox/hq ADR 0632 §2). A template
// named and not given is no need; the reader of the files says it could not read one.
func ModuleNeeds(raw []byte, templates map[string]string) ([]string, error) {
needs, err := ManifestNeeds(raw)
if err != nil || len(templates) == 0 {
return needs, err
}
used := map[string]bool{}
for _, n := range needs {
used[n] = true
}
for _, text := range templates {
textNeeds([]byte(text), used)
}
return sortedSet(used), nil
}
func sortedSet(set map[string]bool) []string {
out := make([]string, 0, len(set))
for k := range set {
+1 -3
View File
@@ -1,9 +1,7 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// ResolverSeat is the machine's own resolver (novox/hq ADR 0247).
const ResolverSeat = contractseats.NodeResolver
const ResolverSeat = "node-resolver"
// ResolverFile is the machine's resolver file: the uplink holder's, or the node-resolver holder's where
// one is held.
-23
View File
@@ -139,26 +139,3 @@ func TestResetFailedWantedByAndUnlinkDanglingAreRequiredVerbsOfTheServiceManager
t.Fatalf("read back from the row, a holder serving all twelve is refused: %v", err)
}
}
// The optional mark is not stored, so a seat set read back from the store's rows takes it from the
// compiled seat: otherwise `failed`, seeded into the row, would come back required.
func TestAnOptionalVerbStaysOptionalInASetReadFromTheStore(t *testing.T) {
defer UseSeats(DefaultSeats())
var rows []Seat
for _, s := range DefaultSeats() {
row := s
row.Serves = nil
for _, v := range s.Serves {
row.Serves = append(row.Serves, Verb{Name: v.Name, Description: v.Description, Input: v.Input})
}
rows = append(rows, row)
}
UseSeats(rows)
seat, _ := SeatNamed(ServiceManagerSeat)
holder := Manifest{Module: "systemd", Version: "1", Claims: []Claim{{Name: ServiceManagerSeat, Scope: ScopeNode,
Serves: []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal",
"reset-failed", "wanted-by", "unlink-dangling"}}}}
if err := CanHold(holder, seat); err != nil {
t.Fatalf("a set read from rows refuses a holder without the optional failed: %v", err)
}
}
+29 -3
View File
@@ -1,6 +1,7 @@
package catalogue
import (
"maps"
"os"
"slices"
"testing"
@@ -44,14 +45,17 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
added = append(added, r)
}
}
// The two seats the milestone renamed (ADR 0634 §9) are read by their new names; a node-engine built with the baseline
// reads the old ones, and a manifest of the new names is the milestone's to deliver after the engines.
renamed := []string{"seat:mesh-enrolment", "seat:mesh-placement"}
for _, b := range baseline {
if !slices.Contains(reads, b) {
if !slices.Contains(reads, b) && !slices.Contains(renamed, b) {
t.Errorf("the baseline's %s is no reader capability of this contract", b)
}
}
for _, want := range []string{"word:tool-runner", "word:core-credentials", "word:bus-state", "word:bus-state.storage",
// The reference form a registry seat answers (ADR 0605 §9), and the seats the registry milestone adds.
ReadForm + ReferenceFormSeat, "seat:go-package-registry", "seat:container-image-registry", "seat:oci-artifact-registry"} {
ReadForm + ReferenceFormSeat, ReadForm + ReferenceFormSeatReach, "seat:go-package-registry", "seat:container-image-registry", "seat:oci-artifact-registry"} {
if !slices.Contains(added, want) {
t.Errorf("%s is in the baseline, or not read: added since it are %v", want, added)
}
@@ -79,12 +83,34 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
"word:provides.identity.max", "word:resources"}, nil},
{"a bus-state entry of requires", `{"module":"b","version":"1","requires":["mesh-bus",{"name":"jobs","storage":"kv"}]}`,
[]string{"word:requires", "word:bus-state.storage"}, []string{"word:bus-state", "word:bus-state.name", "word:bus-state.storage"}},
// A seat's reach, filled from the seat holder's listen (ADR 0632 §1): an engine before it lacks the reading.
{"a seat's reach", `{"module":"s","version":"1","resources":[{"id":"f","type":"file","content":"${seat:container-image-registry:reach}"}]}`,
[]string{"form:seat", "form:seat-reach"}, []string{"form:seat-reach"}},
// The store holder's manifest, distribution's as ADR 0632 leaves it: no reach written, no provision served; an engine
// stating only the baseline lacks the reading that finds the store by it (ADR 0632 §2).
{"the store's holder", `{"module":"distribution","version":"1","claims":[{"name":"container-image-registry",` +
`"scope":"mesh","serves":["list"]}],"listens":[{"name":"registry","port":5000,"from":"mesh"}]}`,
[]string{"form:seat-reach", "seat:container-image-registry"}, []string{"form:seat-reach", "seat:container-image-registry"}},
// docker's daemon file as ADR 0632 leaves it: the manifest unchanged, its template naming the image seat's reach;
// TemplatePaths names the template a reader of the files reads.
{"a template naming a seat's reach", `{"module":"d","version":"1","resources":[{"id":"daemon","type":"file",` +
`"path":"/etc/d.json","template":"files/daemon.json"}]}`, []string{"form:seat", "form:seat-reach"},
[]string{"form:seat-reach"}},
{"a word this contract does not know", `{"module":"u","version":"1","newer-word":{"inside":1}}`, []string{"word:module"}, nil},
} {
needs, err := ManifestNeeds([]byte(row.raw))
templates := map[string]map[string]string{ // the templates' text a row's module names
"a template naming a seat's reach": {"files/daemon.json": `{"x":"${seat:container-image-registry:reach}"}`},
}[row.why]
needs, err := ModuleNeeds([]byte(row.raw), templates)
if err != nil {
t.Fatalf("%s: %v", row.why, err)
}
if templates != nil {
m, err := Decode([]byte(row.raw))
if paths := TemplatePaths(m); err != nil || !slices.Equal(paths, slices.Sorted(maps.Keys(templates))) {
t.Errorf("%s: TemplatePaths = %v (%v), want %v", row.why, paths, err, slices.Sorted(maps.Keys(templates)))
}
}
for _, n := range row.needs {
if !slices.Contains(needs, n) {
t.Errorf("%s: needs %v, not %s", row.why, needs, n)
-3
View File
@@ -1,9 +1,6 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// PlacementSeat is the seat the assigner holds for placement.
const PlacementSeat = contractseats.MeshPlacement
// The seat's five verbs (ADR 0567 §10).
const (
-65
View File
@@ -1,65 +0,0 @@
package catalogue_test
import (
"encoding/json"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/overlay"
)
// The manifests the control plane actually ships, resolved.
//
// Written because the earlier tests built their own manifests and passed while the real one was
// missing a claim — a whole mechanism could have been absent from what ships and every test would
// still have been green.
func provided(t *testing.T) map[string]catalogue.Manifest {
t.Helper()
out := map[string]catalogue.Manifest{}
for _, raw := range []map[string]any{
overlay.Manifest(),
} {
b, err := json.Marshal(raw)
if err != nil {
t.Fatal(err)
}
m, err := catalogue.Decode(b)
if err != nil {
t.Fatalf("a manifest this control plane ships is not valid: %v", err)
}
out[m.Module] = m
}
return out
}
func TestTheControlPlaneShipsNoNetworkingBundle(t *testing.T) {
// ADR 0226: one module for the one private network. A bundle shipped beside it again would be
// a second name every machine carries for the same thing.
shipped := provided(t)
if len(shipped) != 1 {
t.Fatalf("the control plane ships %d modules, want only %s", len(shipped), overlay.Name)
}
if _, ok := shipped["networking"]; ok {
t.Fatal("the retired networking bundle is shipped again")
}
}
// The names-need-addressing test went with the names module: names are a fact now, and a machine
// the mesh cannot place is simply left out of the file (facts_test.go) — which is the same
// protection, enforced where the file is written rather than by a provision refusing.
func withTailscale(shelf map[string]catalogue.Manifest) map[string]catalogue.Manifest {
out := map[string]catalogue.Manifest{}
for k, v := range shelf {
out[k] = v
}
// Deliberately without name-resolution of its own, which is the case that used to install
// both VPNs: the names then needed the mesh's addressing, and only WireGuard has it.
out["tailscale"] = catalogue.Manifest{
Module: "tailscale", Version: "1",
Provides: catalogue.Offers(overlay.Requirement),
Claims: []catalogue.Claim{{Name: overlay.TheNetwork, Scope: catalogue.ScopeNode}},
}
return out
}
+1 -3
View File
@@ -1,9 +1,7 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// ReadsSeat is the role that reads a node's files and accounts (novox/hq ADR 0310).
const ReadsSeat = contractseats.NodeReads
const ReadsSeat = "node-reads"
// ReadsSecretPlaces is the word the holder of node-reads is given: the places on its node where a secret is
// kept, as a JSON list of absolute paths, each refused by name before any read (novox/hq ADR 0310 §5).
-26
View File
@@ -62,29 +62,3 @@ func TestAVerbDefinitionCarriesWhatItReplaces(t *testing.T) {
t.Fatalf("replaces not read: %v", got)
}
}
// A seat row the store seeded before `replaces` existed carries none; the working set takes the compiled
// one, so the `tools` answer says it on a mesh whose rows exist.
func TestASeededRowTakesWhatItsVerbsReplaceFromTheCompiledSeat(t *testing.T) {
was := Seats()
t.Cleanup(func() { UseSeats(was) })
var row Seat
for _, s := range DefaultSeats() {
if s.Name == ServiceManagerSeat {
row = s
}
}
stored := make([]Verb, len(row.Serves))
for i, v := range row.Serves {
v.Replaces = nil
stored[i] = v
}
row.Serves = stored
UseSeats([]Seat{row})
got, _ := SeatNamed(ServiceManagerSeat)
for _, v := range got.Serves {
if v.Name == "journal" && (len(v.Replaces) == 0 || v.Replaces[0] != "journalctl") {
t.Fatalf("the seeded journal verb says it replaces %v", v.Replaces)
}
}
}
+3 -5
View File
@@ -1,15 +1,13 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// HotkeysSeat is the machine's hotkey daemon (novox/hq ADR 0212 §5).
const HotkeysSeat = contractseats.NodeHotkeys
const HotkeysSeat = "node-hotkeys"
// MessageBusSeat is the machine's D-Bus (novox/hq ADR 0215).
const MessageBusSeat = contractseats.NodeMessageBus
const MessageBusSeat = "node-message-bus"
// BackupSeat is the machine's backups (novox/hq ADR 0214, to-be 43).
const BackupSeat = contractseats.NodeBackup
const BackupSeat = "node-backup"
// SeatContribution is one piece of configuration a module gives a seat's holder to place.
type SeatContribution struct {
+3 -5
View File
@@ -4,16 +4,14 @@ import (
"fmt"
"sort"
"strings"
contractseats "git.novox.be/novox/mesh-contracts/go/seats"
)
// The three seats that apply resources (novox/hq ADR 0207 §1). Named once, because the derivation,
// the seed and the messages all turn on these strings.
const (
ServiceManagerSeat = contractseats.NodeServiceManager
PackageManagerSeat = contractseats.NodePackageManager
ContainerRuntimeSeat = contractseats.NodeContainerRuntime
ServiceManagerSeat = "node-service-manager"
PackageManagerSeat = "node-package-manager"
ContainerRuntimeSeat = "node-container-runtime"
)
// appliedThrough is which seat applies a resource of each type. **Only these three**: ADR 0207
+74
View File
@@ -0,0 +1,74 @@
package catalogue
import (
"sort"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// SeatFromDefinition is a seat in this package's types from its definition (novox/hq ADR 0608 §1, ADR 0635 §1): what
// the seat list once held, now read from the definition the indexer keeps on the bus. The definition is resolved with
// its parts (seatdef.Resolve) as the indexer keeps it. A verb's input and answer have their named types in place, and a
// contribution kind given as data gets its checks (withShapeCode), which are code and never a definition's.
func SeatFromDefinition(d seatdef.Definition) Seat {
s := Seat{Name: d.Seat, Scope: d.Scope, Replicated: d.Replicated, Delivers: d.Delivers,
Accepts: seatdef.MessageNames(d.Accepts), Emits: seatdef.MessageNames(d.Events), PulledByKind: d.PulledByKind,
Yields: d.Yields, HeldByTheEngine: d.HeldByTheEngine, Decision: d.Decision}
if len(s.Accepts) == 0 {
s.Accepts = nil
}
if len(s.Emits) == 0 {
s.Emits = nil
}
for _, v := range d.Verbs {
s.Serves = append(s.Serves, Verb{Name: v.Name, Description: v.Description,
Input: seatdef.ResolveRefs(v.Input, d.Defs), Output: seatdef.ResolveRefs(v.Answer, d.Defs),
Optional: v.Optional, Replaces: v.Replaces})
}
for _, r := range d.Receives {
rv := Receivable{Kind: r.Kind, Comment: r.Comment, Dirs: r.Dirs, Offered: r.Offered}
if r.Shape != nil {
sh := &Shape{Order: r.Shape.Order, Examples: examples(r.Shape.Examples), Text: r.Shape.Text}
for _, f := range r.Shape.Places {
sh.Places = append(sh.Places, PlaceField{Field: f.Field, Values: f.Values})
}
withShapeCode(d.Seat, r.Kind, sh)
rv.Shape = sh
}
s.Receives = append(s.Receives, rv)
}
for _, p := range d.Paths {
s.Paths = append(s.Paths, SeatPath{Path: p.Path, StepsBackFor: p.StepsBackFor})
}
return s
}
// SeatsFromDefinitions are the seats of a set of definitions, in reading order: the mesh's own, then a node's, each by
// name. A seat its holder declares itself (HolderDeclares) is not in the list, as the compiled list left it out.
func SeatsFromDefinitions(defs []seatdef.Definition) []Seat {
out := make([]Seat, 0, len(defs))
for _, d := range defs {
if d.HolderDeclares {
continue
}
out = append(out, SeatFromDefinition(d))
}
sort.SliceStable(out, func(i, j int) bool {
if (out[i].Scope == seatdef.ScopeMesh) != (out[j].Scope == seatdef.ScopeMesh) {
return out[i].Scope == seatdef.ScopeMesh
}
return out[i].Name < out[j].Name
})
return out
}
func examples(in []map[string]any) []map[string]any {
if in == nil {
return nil
}
out := make([]map[string]any, len(in))
for i, e := range in {
out[i], _ = seatdef.StringLists(e).(map[string]any)
}
return out
}
-11
View File
@@ -31,17 +31,6 @@ func TestTheSeatsThatNamePathsAreTheStartingSet(t *testing.T) {
}
}
// What the store's rows carry is never the paths: a row read back keeps the compiled ones.
func TestTheSeatPathsSurviveTheStoresRows(t *testing.T) {
before := seats
t.Cleanup(func() { seats = before })
UseSeats([]Seat{{Name: "node-hostname", Scope: ScopeNode}})
s, _ := SeatNamed("node-hostname")
if len(s.Paths) != 2 {
t.Fatalf("node-hostname read back from its row names %v", s.Paths)
}
}
func hostnameSeat(t *testing.T) Seat {
t.Helper()
s, ok := SeatNamed("node-hostname")
+10 -127
View File
@@ -3,8 +3,6 @@ package catalogue
import (
"fmt"
"strings"
contractseats "git.novox.be/novox/mesh-contracts/go/seats"
)
// Seat is one role the mesh defines.
@@ -85,130 +83,22 @@ type Receivable struct {
Offered bool
}
// defaultSeats is the set the mesh ships with — the seed for the control plane's seat table and the
// fallback when it has none (novox/hq ADR 0122). **It is generated, never written here** (novox/hq ADR 0608 §5,
// ADR 0613 §2, design 26 *The seat set has one source*): every seat's definition lives in mesh-contracts, and its
// generated seat list is what this runtime reads, in the order a person reads it: the mesh's own, then a node's. What
// a definition cannot say, being code, is attached here: a contribution kind's checks (withShapeCode).
var defaultSeats = fromContracts(contractseats.List())
// seats is the working set the lookups read, empty until a host gives it the seat list (UseSeats): mesh-sdk compiles no
// seat list and imports no contracts module (novox/hq ADR 0635 §1). A host reads the list from the bus
// (mesh-sdk go/seatlist, which holds the foundation's bootstrapped definitions until the indexer is heard) and gives
// its seats here at start and at each change.
var seats []Seat
// fromContracts is the generated seat list in this runtime's types, each data-shaped kind given its checks.
func fromContracts(list []contractseats.Seat) []Seat {
out := make([]Seat, 0, len(list))
for _, c := range list {
s := Seat{Name: c.Name, Scope: c.Scope, Replicated: c.Replicated, Delivers: c.Delivers, Accepts: c.Accepts,
Emits: c.Emits, PulledByKind: c.PulledByKind, Yields: c.Yields, HeldByTheEngine: c.HeldByTheEngine,
Decision: c.Decision}
for _, v := range c.Serves {
s.Serves = append(s.Serves, Verb{Name: v.Name, Description: v.Description, Input: v.Input, Output: v.Output,
Optional: v.Optional, Replaces: v.Replaces})
}
for _, r := range c.Receives {
rv := Receivable{Kind: r.Kind, Comment: r.Comment, Dirs: r.Dirs, Offered: r.Offered}
if r.Shape != nil {
sh := &Shape{Order: r.Shape.Order, Examples: r.Shape.Examples, Text: r.Shape.Text}
for _, f := range r.Shape.Places {
sh.Places = append(sh.Places, PlaceField{Field: f.Field, Values: f.Values})
}
withShapeCode(c.Name, r.Kind, sh)
rv.Shape = sh
}
s.Receives = append(s.Receives, rv)
}
for _, p := range c.Paths {
s.Paths = append(s.Paths, SeatPath{Path: p.Path, StepsBackFor: p.StepsBackFor})
}
out = append(out, s)
}
return out
}
// seats is the working set the lookups read. It starts as the compiled defaults and is replaced by
// what the control plane loaded from its store (novox/hq ADR 0122), so a change to the set is a
// change to data, not to this code.
var seats = defaultSeats
// DefaultSeats is the set the mesh ships with, for seeding the store's seat table.
func DefaultSeats() []Seat { return append([]Seat(nil), defaultSeats...) }
// UseSeats replaces the working set with the one the control plane read from its store.
// UseSeats replaces the working set with the seat list a host read (seatlist.Reader.Seats, via SeatsFromDefinitions).
//
// **Empty is ignored on purpose.** A store that has not been seeded yet — or one that could not be
// read — must leave the compiled defaults in force rather than emptying the set: an empty set would
// refuse every claim and could stop the control plane composing at all, which is a far worse failure
// than running on the set the binary shipped with. So the store can only ever *replace* the set with
// a non-empty one, never erase it.
// **Empty is ignored on purpose.** A list that was not read yet — or one that could not be — must leave the set in force
// rather than emptying it: an empty set would refuse every claim and could stop a composition altogether, which is a
// far worse failure than running on the set the host last had.
func UseSeats(s []Seat) {
if len(s) == 0 {
return
}
// **The store's rows carry no protocol yet, and the protocol is what the bus is derived
// from.** ADR 0129 gives a seat what it accepts, emits and serves; ADR 0122 moved the set into
// a table that has name, scope, delivers and decision and nothing else, and the columns for
// the rest are not there yet. So a row replacing a compiled entry would silently drop the
// protocol, and the roles' work queues would never be raised — found live as "no response
// from stream" the first time a build was submitted over the new bus (2026-09-28). Until the
// table gains the columns, a row without a protocol keeps the compiled one of the same name.
byName := map[string]Seat{}
for _, d := range defaultSeats {
byName[d.Name] = d
}
merged := make([]Seat, 0, len(s))
for _, row := range s {
if len(row.Accepts)+len(row.Emits)+len(row.Serves) == 0 {
if d, known := byName[row.Name]; known {
row.Accepts, row.Emits, row.Serves = d.Accepts, d.Emits, d.Serves
}
}
// What a seat receives, whether it is replicated and the paths it names are never stored
// (novox/hq ADR 0212, ADR 0223, ADR 0287), so they are always the compiled ones.
if d, known := byName[row.Name]; known {
row.Receives = d.Receives
row.Paths = d.Paths
row.Replicated = d.Replicated
row.PulledByKind = d.PulledByKind
row.Yields = d.Yields // never stored (ADR 0553)
row.HeldByTheEngine = d.HeldByTheEngine // never stored (ADR 0563)
row.Serves = optionalAsCompiled(row.Serves, d.Serves)
}
merged = append(merged, row)
}
seats = merged
}
// optionalAsCompiled is a row's verbs with each one the compiled seat marks optional marked so. The mark
// is never stored (Verb.Optional), and a row is what the working set holds once the store is read: a verb
// the seeding added to the row — `failed`, `checks` — would otherwise come back required, and every holder
// not serving it yet would be refused at registration and at handover, which the mark exists to prevent.
//
// **And what a verb replaces is the compiled one** (novox/hq ADR 0245): a row seeded before the field
// existed carries none, and re-seeding widens a verb's arguments only, so read from the row alone the
// `tools` answer — what the mesh MCP server's search and the agent's instructions are built from — would
// say no verb replaces anything on a mesh whose rows exist. The mesh's statement of what a role replaces
// is this binary's, like the optional mark.
func optionalAsCompiled(row, compiled []Verb) []Verb {
optional := map[string]bool{}
replaces := map[string][]string{}
for _, v := range compiled {
if v.Optional {
optional[v.Name] = true
}
if len(v.Replaces) > 0 {
replaces[v.Name] = v.Replaces
}
}
if len(optional) == 0 && len(replaces) == 0 {
return row
}
out := make([]Verb, len(row))
for i, v := range row {
v.Optional = optional[v.Name]
if r, ok := replaces[v.Name]; ok {
v.Replaces = r
}
out[i] = v
}
return out
seats = append([]Seat(nil), s...)
}
// aliases maps a seat's former names to its current canonical name (novox/hq ADR 0122). Loaded from
@@ -325,10 +215,3 @@ func SeatsWithAProtocol() []Seat {
}
return out
}
// BrokerSeat is the mesh's bus. Its holder is the bus, and its account — when it declares one — may
// snapshot the bus's streams and do nothing else (novox/hq ADR 0235).
//
// Deprecated: a whole seat's name is mesh-contracts' (seats.MeshBroker, meshbroker.Seat); kept for its importers
// until their next change takes it from there (testdata/seat-names.txt).
const BrokerSeat = contractseats.MeshBroker
-105
View File
@@ -1,10 +1,6 @@
package catalogue
import (
"encoding/json"
"os"
"reflect"
"slices"
"testing"
)
@@ -34,105 +30,4 @@ func TestUseSeatsReplacesTheSetButNeverEmptiesIt(t *testing.T) {
if len(Seats()) != 1 {
t.Fatalf("the working set is %d seats, not the one that was loaded", len(Seats()))
}
// **The compiled set is the generated one, and the move changed nothing a reader sees** (design 26, SD-M3, SDK-8):
// every seat the list mesh-sdk compiled before the move held (testdata, written from 5a38a12) is in the set
// generated from mesh-contracts' definitions, entry for entry, its verbs' optional marks and a kind's code
// included. A seat defined whole may add verbs, events and answers; the vault drops accept and move-asks (ADR 0620
// §3); three registry seats are new.
UseSeats(DefaultSeats())
type golden struct {
Seat
Optional []string
ShapeCode []string
}
raw, err := os.ReadFile("testdata/seats-before-move.json")
if err != nil {
t.Fatal(err)
}
var compiled []golden
if err := json.Unmarshal(raw, &compiled); err != nil {
t.Fatal(err)
}
whole := []string{"npm-package-registry", "mesh-broker", "mesh-config", "mesh-vault", "mesh-delivery-checker", "node-engine"}
dropped := map[string]bool{"mesh-vault.accept": true, "mesh-vault.move-asks": true}
// Verbs a seat at the seat list's grain gained in the milestone, each optional until its holders serve it.
addedVerbs := map[string]bool{}
// A verb whose input the whole definition corrected: compose-candidate's input is the boolean the engine and the
// checker read, where the compiled list said a word (engine_compose_candidate_test.go reads it).
retyped := map[string]bool{"node-engine.compose-candidate": true}
norm := func(v any) any {
b, _ := json.Marshal(v)
var out any
_ = json.Unmarshal(b, &out)
return out
}
var names []string
for _, want := range compiled {
names = append(names, want.Name)
got, ok := SeatNamed(want.Name)
if !ok {
t.Errorf("%s left the generated set", want.Name)
continue
}
g := golden{Seat: got}
for _, v := range got.Serves {
if v.Optional {
g.Optional = append(g.Optional, v.Name)
}
}
for _, r := range got.Receives {
if r.Shape != nil && (r.Shape.Check != nil || r.Shape.Reads != nil) {
g.ShapeCode = append(g.ShapeCode, r.Kind)
}
}
g.Serves = slices.DeleteFunc(slices.Clone(g.Serves), func(v Verb) bool { return addedVerbs[want.Name+"."+v.Name] })
g.Optional = slices.DeleteFunc(g.Optional, func(n string) bool { return addedVerbs[want.Name+"."+n] })
if slices.Contains(whole, want.Name) {
want := &want
byName := map[string]Verb{}
for _, v := range got.Serves {
v.Output = nil
byName[v.Name] = v
}
var kept []Verb
for _, v := range want.Serves {
if dropped[want.Name+"."+v.Name] {
if _, still := byName[v.Name]; still {
t.Errorf("%s still serves %s", want.Name, v.Name)
}
continue
}
kv := byName[v.Name]
if retyped[want.Name+"."+v.Name] {
kv.Input = v.Input
}
kept = append(kept, kv)
}
g.Serves, g.Emits, g.Accepts = kept, want.Emits, want.Accepts
// A verb the whole seat added (new, optional while its holders catch up) is no verb of the compiled list.
g.Optional = slices.DeleteFunc(g.Optional, func(n string) bool {
return !slices.ContainsFunc(want.Serves, func(v Verb) bool { return v.Name == n })
})
if len(g.Optional) == 0 {
g.Optional = nil
}
want.Optional = slices.DeleteFunc(want.Optional, func(n string) bool { return dropped[want.Name+"."+n] })
want.Serves = slices.DeleteFunc(slices.Clone(want.Serves), func(v Verb) bool { return dropped[want.Name+"."+v.Name] })
}
if !reflect.DeepEqual(norm(g), norm(want)) {
gb, _ := json.Marshal(g)
wb, _ := json.Marshal(want)
t.Errorf("%s is not as the compiled list held it:\n got %s\nwant %s", want.Name, gb, wb)
}
}
var added []string
for _, s := range DefaultSeats() {
if !slices.Contains(names, s.Name) {
added = append(added, s.Name)
}
}
if want := []string{"go-package-registry", "container-image-registry", "oci-artifact-registry"}; !slices.Equal(added, want) {
t.Errorf("the generated set adds %v, want %v", added, want)
}
}
+2 -4
View File
@@ -1,9 +1,7 @@
package catalogue
import contractseats "git.novox.be/novox/mesh-contracts/go/seats"
// NFSServerSeat is the role of the machine that shares directories over NFS (novox/hq ADR 0263).
const NFSServerSeat = contractseats.NodeNfsServer
const NFSServerSeat = "node-nfs-server"
// MountsSeat is the role that mounts a machine's shares and occasional sources (novox/hq ADR 0263).
const MountsSeat = contractseats.NodeMounts
const MountsSeat = "node-mounts"
+20
View File
@@ -8,6 +8,7 @@ import (
"os"
"path"
"path/filepath"
"slices"
"strings"
"unicode/utf8"
)
@@ -184,6 +185,25 @@ func fillItemTemplates(m Manifest, items []Item, text func(string) (string, bool
return out, nil
}
// TemplatePaths is every template the manifest's files and items name, by its path in the module's directory, sorted
// and once each: what a reader of the module's files reads for ModuleNeeds. A template whose name is wrong is left out;
// the judge refuses it by name.
func TemplatePaths(m Manifest) []string {
var out []string
for _, r := range m.Resources {
if name, has, wrong := templateOf(m.Module, m.TemplatesDir(), r); has && wrong == "" {
out = append(out, name)
}
}
for _, it := range m.Items {
if it.Template != "" && itemTemplateProblem(m, it.Template) == "" {
out = append(out, it.Template)
}
}
slices.Sort(out)
return slices.Compact(out)
}
// ReadTemplates is the text of each template the manifest names, read from the module's directory in a repository tree,
// by template path. Refused by name: a template the directory lacks, and anything under its templates directory
// (TemplatesDir) the build's packing refuses — that directory itself, or any directory or file under it, that is a
+38
View File
@@ -0,0 +1,38 @@
package catalogue
import "git.novox.be/novox/mesh-sdk/go/seatlist/seatlisttest"
// defaultSeats is the list the tests start from (seatlisttest: the contracts modules' definitions, read as a host reads
// them from the bus, ADR 0635 §1), and DefaultSeats gives a copy of it for a test to restore.
var defaultSeats = func() []Seat {
list := SeatsFromDefinitions(seatlisttest.Definitions())
UseSeats(list)
return list
}()
// DefaultSeats is the tests' seat list.
func DefaultSeats() []Seat { return append([]Seat(nil), defaultSeats...) }
// The whole seats' names, which the tests use and the package does not (ADR 0634 §4; ADR 0613 §5 refuses them in
// non-test code).
const (
CatalogSeat = "mesh-catalog"
CheckerSeat = "mesh-delivery-checker"
EnrolmentSeat = "mesh-node-enroller"
PlacementSeat = "mesh-module-assigner"
DeliverySeat = "mesh-delivery"
WantsSeat = "mesh-wants"
HandActsSeat = "mesh-hand-acts"
ConditionsSeat = "mesh-conditions"
WatchdogsSeat = "mesh-watchdogs"
SelfCheckSeat = "mesh-self-check"
HealersSeat = "mesh-healers"
VaultSeat = "mesh-vault"
ProvisionerSeat = "mesh-provisioner"
BrokerSeat = "mesh-broker"
EngineSeat = "node-engine"
NpmPackageRegistrySeat = "npm-package-registry"
GoPackageRegistrySeat = "go-package-registry"
ContainerImageRegistrySeat = "container-image-registry"
OciArtifactRegistrySeat = "oci-artifact-registry"
)
File diff suppressed because it is too large Load Diff
-5
View File
@@ -91,9 +91,4 @@ func TestOnlyTheResolverIsReplicated(t *testing.T) {
t.Errorf("%s replicated = %v; only mesh-dns-resolver is (ADR 0223)", s.Name, s.Replicated)
}
}
UseSeats([]Seat{{Name: "mesh-dns-resolver", Scope: ScopeMesh, Delivers: "wildcard-resolution"}})
defer UseSeats(DefaultSeats())
if s, _ := SeatNamed("mesh-dns-resolver"); !s.Replicated {
t.Error("loading the set from the store, which has no column for it, lost the resolver's replication")
}
}
-13
View File
@@ -50,16 +50,3 @@ func TestTheUplinkSeatPromisesItsVerbsAndRequiresNoneYet(t *testing.T) {
t.Fatalf("a verb the seat does not promise was accepted: %v", err)
}
}
func TestTheUplinkVerbsReadBackFromTheRowStayOptional(t *testing.T) {
before := seats
t.Cleanup(func() { seats = before })
// The row as seeding stores it: both verbs, the mark not a column.
UseSeats([]Seat{{Name: "node-uplink", Scope: ScopeNode, Serves: []Verb{{Name: "resolvers"}, {Name: "links"}}}})
seat := uplinkSeat(t)
holder := Manifest{Module: "systemd-networkd", Version: "1", Claims: []Claim{{Name: "node-uplink", Scope: ScopeNode}},
Facts: map[string]RosterFile{"resolvers": {Path: ResolverFile, Template: "x"}}}
if err := CanHold(holder, seat); err != nil {
t.Fatalf("read back from the row, the verbs are required: %v", err)
}
}
+3 -3
View File
@@ -19,9 +19,9 @@ func TestTheDeliverySeatHasNoPlanVerbs(t *testing.T) {
}
for _, name := range []string{"times", "what-if", "release", "stop", "retire-history"} {
v, ok := rows[name]
replacement, removed := RemovedDeliveryVerbs[name]
if !ok || !removed || replacement == "" {
t.Errorf("%s: in the table %v, in the removed list %v", name, ok, removed)
replacement := strings.TrimPrefix(v.Description, "Removed in R6 (ADR 0539): refused by name, naming its replacement: ")
if !ok || replacement == "" || replacement == v.Description {
t.Errorf("%s: in the table %v, replacement %q", name, ok, replacement)
continue
}
if !strings.HasPrefix(v.Description, "Removed in R6 (ADR 0539): refused by name") || !strings.Contains(v.Description, replacement) || !v.Optional {
+3 -3
View File
@@ -51,12 +51,12 @@ func TestHoldingAMeshSeatRequiresServingItsVerbs(t *testing.T) {
t.Errorf("a holder saying it serves mint: %v", err)
}
// L3: the seat mesh-catalog promises four verbs, each optional while two claimants stand (ADR 0604 §1, §6).
// L3: the seat mesh-catalog promises five verbs (the fifth is the seat list, ADR 0636 §4), each optional while two claimants stand (ADR 0604 §1, §6).
catalog, ok := SeatNamed(CatalogSeat)
if !ok || catalog.Scope != ScopeMesh || catalog.Decision != "novox/hq ADR 0604" || len(catalog.Serves) != 4 {
if !ok || catalog.Scope != ScopeMesh || catalog.Decision != "novox/hq ADR 0604" || len(catalog.Serves) != 5 {
t.Fatalf("mesh-catalog: in the set %v, %+v", ok, catalog)
}
four := []string{"modules", "module", "provides", "dependents"}
four := []string{"modules", "module", "provides", "dependents", "seat-definitions"}
for i, v := range catalog.Serves {
if v.Name != four[i] || !v.Optional || v.Input == nil {
t.Errorf("mesh-catalog's verb %d is %+v, want %s, optional, with its input", i, v, four[i])
-141
View File
@@ -1,141 +0,0 @@
// Package channel holds the shapes of the two kinded benches a channel holds, `channel` (to show and say) and `intake`
// (to say what was chosen, and to ask a proof), and the subjects they travel on (novox/hq ADR 0259 §3). The router
// sends a channel a Message, and a channel says a Choice, a LinkAsked, a Failed or its Standing, or asks a Code's proof.
// Those benches are not defined whole, so their value layouts stay in mesh-sdk (novox/hq ADR 0625 §9, ADR 0627 §4);
// the operator-channel seat's own ask and warrant are its contract's and its client's, in mesh-contracts.
package channel
import "time"
// The benches (novox/hq ADR 0259 §3).
const (
// ChannelSeat is the kinded bench a channel holds to show and say: its accepts carry the kind.
ChannelSeat = "channel"
// IntakeSeat is the kinded bench a channel holds to say what was chosen: its events and proofs carry
// the kind.
IntakeSeat = "intake"
)
// The work a channel takes, on ChannelSubject.
const (
Show = "show" // a message offering answers
Edit = "edit" // a message shown before, replaced
Send = "send" // a message offering nothing
)
// ChannelSubject is where the router sends a channel of a kind its work.
func ChannelSubject(verb, kind string) string {
return "mesh.seat." + ChannelSeat + ".accept." + verb + "." + kind
}
// What a channel says, on IntakeSubject.
const (
Chosen = "choice" // a button tapped
Link = "link" // somebody asked to be linked as the operator
)
// IntakeSubject is where a channel of a kind says what arrived.
func IntakeSubject(what, kind string) string {
return "mesh.seat." + IntakeSeat + ".event." + what + "." + kind
}
// CodeProof is the one proof verb: a code the operator typed, carried by request and reply, never kept.
const CodeProof = "code"
// ProofSubject is where a channel of a kind asks a proof.
func ProofSubject(verb, kind string) string {
return "mesh.seat." + IntakeSeat + ".proof." + verb + "." + kind
}
// Button is one answer a channel offers: its label and the router's one-time ticket for it.
type Button struct {
Label string `json:"label"`
Ticket string `json:"ticket"`
}
// Message is the work a channel takes: shown with buttons (Show), shown again in place (Edit), or said
// (Send). Handle is the router's name for it, the same across a show and its edits; the channel keeps
// which of its own messages that is. The words are the router's, shown as given.
type Message struct {
Handle string `json:"handle"`
Title string `json:"title"`
Body string `json:"body"`
Buttons []Button `json:"buttons,omitempty"`
// Note is the message's secondary words, after its body and its buttons: who asked and when, what each
// answer does, when it expires, and the ask's Details. A channel shows it smaller, collapsed or set apart
// where it can, and after the body where it cannot (novox/hq issue 639). A channel that bounds a message's
// length cuts the body first and the note only when the title and the note alone do not fit, saying each
// cut (novox/hq ADR 0347 §5).
Note string `json:"note,omitempty"`
Urgent bool `json:"urgent,omitempty"`
Silent bool `json:"silent,omitempty"`
// Reply is the Choice or LinkAsked this answers, by its ID: the channel shows it where that was made.
Reply string `json:"reply,omitempty"`
// To is the account linked as the operator on this kind, for a channel that verifies its sender: where
// the channel sends what is not a reply. The router's word, from its list; empty when none is linked.
To string `json:"to,omitempty"`
// Secret says the words carry something shown once (a link's code): the channel shows it and keeps no
// copy of it — no state, no history of its own.
Secret bool `json:"secret,omitempty"`
}
// Sender is who a channel's service says sent something: the account, the name it shows, and whether the
// service authenticated it. The router alone judges whether that is the operator.
type Sender struct {
Identity string `json:"identity"`
Display string `json:"display,omitempty"`
Authenticated bool `json:"authenticated"`
}
// Failed is a message a channel could not deliver: its handle, why, and whether trying again could help.
type Failed struct {
Handle string `json:"handle"`
Why string `json:"why"`
Permanent bool `json:"permanent,omitempty"`
At time.Time `json:"at"`
}
// Standing is what a channel says of itself, at least every five minutes and whenever it changes:
// whether it can send now, why not, and whether its edits notify nobody.
type Standing struct {
Ready bool `json:"ready"`
Why string `json:"why,omitempty"`
EditsSilently bool `json:"edits-silently,omitempty"`
At time.Time `json:"at"`
}
// What a channel says of its own delivery, on IntakeSubject.
const (
FailedWhat = "failed"
StandingWhat = "standing"
)
// Choice is a button chosen on a channel.
type Choice struct {
// ID is the channel's own for this arrival, unique, so the router acts on it once.
ID string `json:"id"`
Ticket string `json:"ticket"`
Handle string `json:"handle,omitempty"`
Sender Sender `json:"sender"`
At time.Time `json:"at"`
}
// LinkAsked is somebody on a channel asking to be linked as the operator.
type LinkAsked struct {
ID string `json:"id"`
Sender Sender `json:"sender"`
At time.Time `json:"at"`
}
// Code is a code a person typed on a channel, asked as a proof: never in an event, never kept.
type Code struct {
Sender Sender `json:"sender"`
Code string `json:"code"`
Purpose string `json:"purpose"`
}
// ProofAnswer is the router's answer to a proof: whether it was taken, and words to say to the person.
type ProofAnswer struct {
Accepted bool `json:"accepted"`
Words string `json:"words"`
}
-32
View File
@@ -1,32 +0,0 @@
package channel
import (
"encoding/json"
"strings"
"testing"
)
func TestTheSubjectsNameTheirPublisher(t *testing.T) {
for got, want := range map[string]string{
ChannelSubject(Show, "telegram"): "mesh.seat.channel.accept.show.telegram",
IntakeSubject(Chosen, "desktop"): "mesh.seat.intake.event.choice.desktop",
ProofSubject(CodeProof, "telegram"): "mesh.seat.intake.proof.code.telegram",
} {
if got != want {
t.Errorf("%s, want %s", got, want)
}
}
}
// A message's note travels beside its body, and is left out of the encoding when empty, so a channel older
// than it reads every message it read before.
func TestAMessageCarriesItsNoteApart(t *testing.T) {
raw, _ := json.Marshal(Message{Handle: "h", Title: "t", Body: "b", Note: "Asked by x."})
if !strings.Contains(string(raw), `"note":"Asked by x."`) {
t.Fatalf("the note is not encoded: %s", raw)
}
raw, _ = json.Marshal(Message{Handle: "h", Title: "t"})
if strings.Contains(string(raw), "note") {
t.Fatalf("an empty note is encoded: %s", raw)
}
}
-209
View File
@@ -1,209 +0,0 @@
// Package composer is the shape of a composition over a plain request (novox/hq ADR 0482 clause 4, issue 1621, as ADR
// 0593 §7 changes it): the request the command mesh-compose reads and the verdict it answers. The command and the
// composer that runs it are the node-engine's (mesh-host cmd/mesh-compose), built from its repository at a group's
// head, so a change to its composition code is judged by that code.
//
// **It reads its request and nothing else.** No bus, store, network, file or environment: the manifests come inside the
// request (raw, as their repository holds them), the rest of the mesh is a compose.Input, and the answer is a value.
// A manifest named only by its digest in the delivery indexer's bucket is filled before the run by Fill, by whoever
// holds the bus (the Go build module's compose job); Run refuses a digest left unfilled.
package composer
import (
"encoding/json"
"errors"
"fmt"
"slices"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Version is the version of a Request and of a Verdict; a reader refuses another.
const Version = 1
// Package is the import path of the command, which a compose job builds at a commit of the node-engine's repository
// (novox/hq ADR 0593 §7).
const Package = "github.com/novox/mesh-host/cmd/mesh-compose"
// The depths of a composition.
const (
// DepthResolve resolves each machine's set over the rest of the mesh (compose.Resolve): what the assigner judges
// while the facts snapshot is not on the bus (issue 1461).
DepthResolve = "resolve"
// DepthCompose composes each machine's whole declaration and runs the node-engine's validator over it
// (compose.Compose).
DepthCompose = "compose"
)
// Request is everything one run composes: the rest of the mesh, one or more cases of manifests composed over it, and
// the machines composed in each case.
type Request struct {
V int `json:"v"`
// Input is the composition's input without its catalogue (and without each node's kept manifests): the cases give
// those. A Catalogue or Kept it holds is ignored. It is the node-engine's composition input as JSON, which only the
// composer decodes (novox/hq ADR 0593 §7), so this shape imports no composition.
Input json.RawMessage `json:"input"`
// Cases are composed one after the other over Input, each with its own manifests; their names are unique.
Cases []Case `json:"cases"`
// Machines are the nodes composed in every case; none means the manifests are only read.
Machines []string `json:"machines,omitempty"`
// Depth is DepthResolve or DepthCompose; empty is DepthCompose.
Depth string `json:"depth,omitempty"`
// HoldingsFromClaims adds to Input's holdings every seat an assigned module claims, held on its node (the
// assigner's input, which has no holdings of record).
HoldingsFromClaims bool `json:"holdings_from_claims,omitempty"`
// Declarations asks each composed machine's declared paths (DepthCompose only).
Declarations bool `json:"declarations,omitempty"`
// Predefined is the list of predefined keys at the catalogue's root (predefined.json, novox/hq ADR 0348 §7). No
// longer read: the composer judges no manifest (novox/hq ADR 0578 §5), the indexer's judge does, and its verdicts
// come in each case's Verdicts. Kept so an asker that still sends it is read.
Predefined catalogue.PredefinedList `json:"predefined,omitempty"`
// Packages are the known packages a manifest's `packages` may name, each by its name in its own ecosystem (design 32:
// a Go package's go.mod module path, a TypeScript package's package.json name), for the indexer's judge (novox/hq ADR
// 0578 §5, ADR 0606 §4); the composer does not read them. Nil is no list at hand, and the judge judges no name; an
// empty list is a list, and every name is unknown to it.
Packages []string `json:"packages,omitzero"`
}
// Case is one catalogue composed: every module's manifest by name, and per node the manifests kept at the build it runs.
//
// Placements are, by node, the placement the case composes that node with in place of what Input says of it: a change's
// own placement files, so a check composes a catalogue pull request's placement (issue 1126, frombus.Placed); a null
// placement leaves nothing placed on the node. A case without them composes Input's.
//
// Verdicts are, by module, the indexer's judge's verdict on its manifest in Manifests (novox/hq ADR 0578 §5), put in by
// the assigner or the checker that runs the composer: a refused one is held, composed as its stub so what it offers
// still shapes the rest, and named in the case's verdict. A module without one is accepted (delivery.TakeManifestVerdict).
type Case struct {
Name string `json:"name"`
Manifests map[string]Manifest `json:"manifests"`
Verdicts map[string]delivery.ManifestVerdict `json:"verdicts,omitempty"`
Kept map[string]map[string]Manifest `json:"kept,omitempty"`
Placements map[string]*delivery.Placement `json:"placements,omitempty"`
}
// Manifest is a module's manifest as its repository holds it (Raw, the bytes), or the digest the delivery indexer keeps
// it under (Digest), which Fill replaces by the bytes. On the wire it is the manifest itself, or {"digest": "..."}.
type Manifest struct {
Raw json.RawMessage
Digest string
}
// MarshalJSON writes the raw manifest, or {"digest": ...} when only the digest is known.
func (m Manifest) MarshalJSON() ([]byte, error) {
if len(m.Raw) > 0 {
return m.Raw, nil
}
return json.Marshal(map[string]string{"digest": m.Digest})
}
// UnmarshalJSON reads {"digest": "..."} (an object with that one key) as a digest, and any other value as the raw
// manifest.
func (m *Manifest) UnmarshalJSON(raw []byte) error {
var probe map[string]json.RawMessage
if json.Unmarshal(raw, &probe) == nil && len(probe) == 1 {
if d, ok := probe["digest"]; ok {
var digest string
if err := json.Unmarshal(d, &digest); err != nil {
return fmt.Errorf("a manifest's digest is not a string: %w", err)
}
*m = Manifest{Digest: digest}
return nil
}
}
*m = Manifest{Raw: slices.Clone(raw)}
return nil
}
// Verdict is what one run answers: the mesh-sdk the command was built with, and each case's verdict in the order asked.
type Verdict struct {
V int `json:"v"`
SDK string `json:"sdk"`
Cases []CaseVerdict `json:"cases"`
}
// CaseVerdict is one case's: every manifest read or refused, every machine composed or not.
type CaseVerdict struct {
Name string `json:"name"`
Manifests map[string]ManifestRead `json:"manifests"`
Machines map[string]Machine `json:"machines,omitempty"`
}
// ManifestRead is one manifest as this mesh-sdk reads it: accepted, or refused with why: the indexer's verdict
// (Case.Verdicts) or bytes that do not decode. A field the decoder does not know is set aside, named in SetAside, and
// the manifest still accepted (novox/hq ADR 0578 §6).
type ManifestRead struct {
Accepted bool `json:"accepted"`
Refused string `json:"refused,omitempty"`
SetAside string `json:"set_aside,omitempty"`
}
// Machine is one machine composed: whether it composes (resolves, at DepthResolve), why not, the node-engine
// validator's problems and the composition's refusals (DepthCompose), and its declared paths when asked. NotJudged says,
// in words, why the composer did not judge the machine (an older node-engine's input that does not say how it composes,
// novox/hq ADR 0593 §9): the machine is then neither composed nor refused, and a reader decides by this field alone,
// never by the words of Reason.
type Machine struct {
Composes bool `json:"composes"`
NotJudged string `json:"not_judged,omitempty"`
Reason string `json:"reason,omitempty"`
Problems []string `json:"problems,omitempty"`
Refusals []string `json:"refusals,omitempty"`
Declared []Path `json:"declared,omitempty"`
}
// Path is one path a declaration puts on the machine, with its kind (the resource's type), owner, group and mode as
// declared, and the module it is of.
type Path struct {
Path string `json:"path"`
Kind string `json:"kind"`
Owner string `json:"owner,omitempty"`
Group string `json:"group,omitempty"`
Mode string `json:"mode,omitempty"`
Module string `json:"module,omitempty"`
}
// ErrUnfilled is a manifest named by a digest that nobody replaced by its bytes.
var ErrUnfilled = errors.New("composer: a manifest is named by its digest and was not filled")
// Decode reads a request; it refuses one that is not JSON or of another version.
func Decode(data []byte) (Request, error) {
var r Request
if err := json.Unmarshal(data, &r); err != nil {
return Request{}, fmt.Errorf("composer: the request is not JSON of its shape: %w", err)
}
if r.V != Version {
return Request{}, fmt.Errorf("composer: a request of version %d, this composer reads %d", r.V, Version)
}
return r, nil
}
// Fill replaces every manifest named by its digest by the bytes get answers for that digest; a digest get cannot answer
// stops it, named.
func Fill(r *Request, get func(digest string) ([]byte, error)) error {
fill := func(m map[string]Manifest) error {
for name, man := range m {
if len(man.Raw) > 0 {
continue
}
raw, err := get(man.Digest)
if err != nil {
return fmt.Errorf("composer: the manifest of %s (digest %s): %w", name, man.Digest, err)
}
m[name] = Manifest{Raw: raw}
}
return nil
}
for _, c := range r.Cases {
if err := fill(c.Manifests); err != nil {
return err
}
for _, kept := range c.Kept {
if err := fill(kept); err != nil {
return err
}
}
}
return nil
}
-58
View File
@@ -1,58 +0,0 @@
package composer
import (
"encoding/json"
"errors"
"strings"
"testing"
)
const demo = `{"module": "demo", "version": "1", "claims": [{"name": "node-files"}]}`
// A manifest named by its digest goes on the wire as {"digest": ...}, and Fill puts the bytes the indexer keeps in its
// place; a raw manifest goes as itself.
func TestAManifestIsItsBytesOrItsDigest(t *testing.T) {
r := Request{V: Version, Cases: []Case{{Name: "change", Manifests: map[string]Manifest{
"demo": {Digest: "d1"}, "raw": {Raw: json.RawMessage(`{"module":"raw","version":"1"}`)}},
Kept: map[string]map[string]Manifest{"laptop": {"demo": {Digest: "d1"}}}}}}
wire, err := json.Marshal(r)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(wire), `"demo":{"digest":"d1"}`) || !strings.Contains(string(wire), `"raw":{"module":"raw","version":"1"}`) {
t.Fatalf("on the wire: %s", wire)
}
back, err := Decode(wire)
if err != nil {
t.Fatal(err)
}
asked := 0
if err := Fill(&back, func(d string) ([]byte, error) {
asked++
if d != "d1" {
return nil, errors.New("no such digest")
}
return []byte(demo), nil
}); err != nil {
t.Fatal(err)
}
if asked != 2 || string(back.Cases[0].Manifests["demo"].Raw) != demo || string(back.Cases[0].Kept["laptop"]["demo"].Raw) != demo {
t.Fatalf("filled %d times: %+v", asked, back.Cases[0])
}
// The known packages: none is no list (not on the wire), an empty list is a list, and a list reads back whole.
for _, known := range [][]string{nil, {}, {"git.novox.be/novox/mesh-sdk", "@novox/mesh-sdk"}} {
wire, err := json.Marshal(Request{V: Version, Packages: known})
if err != nil {
t.Fatal(err)
}
back, err := Decode(wire)
if err != nil || (back.Packages == nil) != (known == nil) || strings.Join(back.Packages, ",") != strings.Join(known, ",") ||
known == nil && strings.Contains(string(wire), "packages") {
t.Errorf("the known packages %#v went as %s and read back %#v (%v)", known, wire, back.Packages, err)
}
}
bad := Request{V: Version, Cases: []Case{{Name: "c", Manifests: map[string]Manifest{"x": {Digest: "nope"}}}}}
if err := Fill(&bad, func(string) ([]byte, error) { return nil, errors.New("not there") }); err == nil || !strings.Contains(err.Error(), "nope") {
t.Fatalf("a digest the indexer does not hold: %v", err)
}
}
-136
View File
@@ -1,136 +0,0 @@
package delivery
import (
"bytes"
"encoding/json"
"sort"
"strconv"
"strings"
)
// DifferingPaths is every JSON path at which two bodies of one resource differ, sorted (issue 1126, T1: the shadow
// shows its work): object keys and list places joined by dots (`content.url`, `links.0`), a string that holds a JSON
// object or list read as that JSON and walked on (a runtime's config carried as text), a key or place one side alone
// holds named as it is. A key a secret could be under (secretWords) is named and not walked into. A body that is not
// JSON differs at "" (the whole), and a resource one side alone holds has none. At most MaxPaths: the last then says how many more there are. A path names keys
// alone, never a value, so it holds no secret.
func DifferingPaths(composed, running json.RawMessage) []string {
if composed == nil || running == nil {
return nil // one side alone holds it: its why says so
}
var a, b any
okA := unmarshalNumbers(composed, &a) == nil
okB := unmarshalNumbers(running, &b) == nil
if !okA || !okB {
if bytes.Equal(composed, running) {
return nil
}
return []string{""}
}
var out []string
walkDiffering(a, b, "", &out)
sort.Strings(out)
if len(out) > MaxPaths {
more := len(out) - (MaxPaths - 1)
out = append(out[:MaxPaths-1], "… "+strconv.Itoa(more)+" more")
}
return out
}
func unmarshalNumbers(raw []byte, into *any) error {
d := json.NewDecoder(bytes.NewReader(raw))
d.UseNumber()
return d.Decode(into)
}
// asJSON is a string holding a JSON object or list read as that value; v itself otherwise.
func asJSON(v any) any {
s, ok := v.(string)
if !ok {
return v
}
t := strings.TrimSpace(s)
if t == "" || (t[0] != '{' && t[0] != '[') {
return v
}
var parsed any
if unmarshalNumbers([]byte(t), &parsed) != nil {
return v
}
return parsed
}
func join(at, key string) string {
if at == "" {
return key
}
return at + "." + key
}
func walkDiffering(a, b any, at string, out *[]string) {
a, b = asJSON(a), asJSON(b)
ma, isMA := a.(map[string]any)
mb, isMB := b.(map[string]any)
if isMA && isMB {
keys := map[string]bool{}
for k := range ma {
keys[k] = true
}
for k := range mb {
keys[k] = true
}
for k := range keys {
x, inA := ma[k]
y, inB := mb[k]
switch {
case !inA || !inB:
*out = append(*out, join(at, k))
case secretKey(k):
// A file's secrets (SecretsField) are named entry by entry, each entry never entered; any other
// object under a secret key, a vault box included, is one path.
sx, isSX := x.(map[string]any)
sy, isSY := y.(map[string]any)
if k != SecretsField || !isSX || !isSY {
if !sameAny(x, y) {
*out = append(*out, join(at, k))
}
continue
}
for name := range sx {
if v, in := sy[name]; !in || !sameAny(sx[name], v) {
*out = append(*out, join(join(at, k), name))
}
}
for name := range sy {
if _, in := sx[name]; !in {
*out = append(*out, join(join(at, k), name))
}
}
default:
walkDiffering(x, y, join(at, k), out)
}
}
return
}
la, isLA := a.([]any)
lb, isLB := b.([]any)
if isLA && isLB {
for i := range max(len(la), len(lb)) {
if i >= len(la) || i >= len(lb) {
*out = append(*out, join(at, strconv.Itoa(i)))
continue
}
walkDiffering(la[i], lb[i], join(at, strconv.Itoa(i)), out)
}
return
}
if !sameAny(a, b) {
*out = append(*out, at)
}
}
func sameAny(a, b any) bool {
x, errA := json.Marshal(a)
y, errB := json.Marshal(b)
return errA == nil && errB == nil && bytes.Equal(x, y)
}
-564
View File
@@ -1,564 +0,0 @@
package delivery
import (
"bytes"
"encoding/json"
"fmt"
"reflect"
"strings"
)
// The assigner's keys (novox/hq ADR 0528 clauses 1 and 4, ADR 0429 §6, issue 1297). The assigner writes, by the six
// rules of ADR 0429 §6, what each node should run: `want.<node>.<module>` in its bucket wants, and beside it
// `held.<module>.<fingerprint>` for a fingerprint it holds after a failed health check. A module that leaves a node's
// modules gets its want written removed, never deleted (ADR 0408 §4). For each node and module it compares the want with
// what the controller sends (the build the controller's assignment names) and writes every difference under
// `<node>.<module>` in its bucket differences, with the rule that explains it or unexplained. In release 5 wants are
// only compared; the node agent applying them is release 6 (ADR 0429 §7). Only the assigner's credential writes these
// buckets; the watchman, the view and the controller read them.
//
// **Release 6** (ADR 0539 clauses 1, 6 and 16): a want also names the digest of the one composition it allows, written
// only once that composition exists (the seventh rule, RuleComposed), and carries the answer to a failed backup; the
// node's own want `want.<node>.node` names the digest of the node's own composition (NodeWant).
//
// **The bucket is the first word, the key the rest** (as the builder's `builds.<module>.<fp>` is the key
// `<module>.<fp>` in its builds bucket): the differences bucket's keys are `<node>.<module>`. The wants bucket holds
// two kinds of key, so each keeps its word: `want.<node>.<module>` and `held.<module>.<fingerprint hex>`.
const (
// AssignerModule is the module of the assigner part.
AssignerModule = "mesh-delivery-assigner"
// AssignerWantsBucket is the assigner's bucket of wants and holds, `<owner module>_<name>`.
AssignerWantsBucket = AssignerModule + "_wants"
// AssignerWantsHistory is the revisions the wants bucket keeps per key, so a want's history shows each move and
// the operator's manual write (ADR 0429 §6, the one escape).
AssignerWantsHistory = 20
// AssignerDifferencesBucket is the assigner's bucket of differences between a want and what the controller sends.
AssignerDifferencesBucket = AssignerModule + "_differences"
// AssignerDifferencesHistory is the revisions the differences bucket keeps per key: acceptance test 17 of design 47
// reads a day of them.
AssignerDifferencesHistory = 64
)
// The words that open the wants bucket's two kinds of key.
const (
wantWord = "want"
heldWord = "held"
)
// WantKey is the key, in AssignerWantsBucket, of a node's want for a module: `want.<node>.<module>`. A module named
// `node` is refused, since `want.<node>.node` is the node's own key (ADR 0429 §4), as HaveKey refuses it.
func WantKey(node, module string) (string, error) {
have, err := HaveKey(node, module)
if err != nil {
return "", err
}
return wantWord + "." + have, nil
}
// WantNodeKey is the key, in AssignerWantsBucket, of a node's own want: `want.<node>.node`, naming the digest of the
// node's own composition (ADR 0539 clause 6; ADR 0429 §4's node key). The assigner writes it once the node's own
// composition exists.
func WantNodeKey(node string) (string, error) {
own, err := HaveNodeKey(node)
if err != nil {
return "", err
}
return wantWord + "." + own, nil
}
// ParseWantNodeKey is the inverse of WantNodeKey; it refuses a module's want key and any key of another bucket,
// naming it.
func ParseWantNodeKey(key string) (node string, err error) {
rest, ok := strings.CutPrefix(key, wantWord+".")
if !ok {
return "", fmt.Errorf("delivery: %q is not a node's own want key, want.<node>.node", key)
}
node, _, isNode, err := ParseHaveKey(rest)
if err != nil || !isNode {
return "", fmt.Errorf("delivery: %q is not a node's own want key, want.<node>.node", key)
}
return node, nil
}
// ParseWantKey is the inverse of WantKey; it refuses a held key, a have key, the node's own want key and any key of
// another bucket, naming it.
func ParseWantKey(key string) (node, module string, err error) {
rest, ok := strings.CutPrefix(key, wantWord+".")
if !ok {
return "", "", fmt.Errorf("delivery: %q is not a want key, want.<node>.<module>", key)
}
node, module, isNode, err := ParseHaveKey(rest)
if err != nil || isNode {
return "", "", fmt.Errorf("delivery: %q is not a want key, want.<node>.<module>", key)
}
return node, module, nil
}
// HeldKey is the key, in AssignerWantsBucket, of a module's fingerprint the assigner holds: `held.<module>.<hex>`,
// the hex being the fingerprint's key part (FingerprintKeyPart).
func HeldKey(module, fingerprint string) (string, error) {
key, err := BuilderBuildKey(module, fingerprint)
if err != nil {
return "", err
}
return heldWord + "." + key, nil
}
// ParseHeldKey is the inverse of HeldKey: the module and the fingerprint (`src2:` and the hex); it refuses a want key
// and any key of another bucket, naming it.
func ParseHeldKey(key string) (module, fingerprint string, err error) {
rest, ok := strings.CutPrefix(key, heldWord+".")
if !ok {
return "", "", fmt.Errorf("delivery: %q is not a held key, held.<module>.<fingerprint hex>", key)
}
if module, fingerprint, err = ParseBuilderBuildKey(rest); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a held key: %w", key, err)
}
return module, fingerprint, nil
}
// DifferenceKey is the key, in AssignerDifferencesBucket, of the difference for a module on a node: `<node>.<module>`.
func DifferenceKey(node, module string) (string, error) {
return HaveKey(node, module)
}
// ParseDifferenceKey is the inverse of DifferenceKey; it refuses a want or held key and a node's own key, naming it.
func ParseDifferenceKey(key string) (node, module string, err error) {
node, module, isNode, err := ParseHaveKey(key)
if err != nil || isNode {
return "", "", fmt.Errorf("delivery: %q is not a difference key, <node>.<module>", key)
}
return node, module, nil
}
// The rules of ADR 0429 §6 and ADR 0539 clause 6, by number, as a want or a difference names the one that holds a node
// back.
const (
RuleBuildDone = 1 // the build of F is done
RuleNotHeld = 2 // F is not held
RuleCanary = 3 // the node is the module's canary, or the canary runs F healthy
RuleStep = 4 // at most step other nodes are ahead (step 0: no limit, ADR 0528 §3)
RuleProviders = 5 // every provider the module requires runs healthy on that node, or at its canary
RuleGrant = 6 // the module's credential for F exists (grants.<module>.<fp>)
RuleComposed = 7 // the module's composition for that node at F exists (ADR 0539 §6)
// RuleProvisions: every provision the module requires on that node is provided (novox/hq ADR 0570 §4): a want's
// words name the key and its state that hold it, as "the database of keycloak on novox is asked of postgres".
RuleProvisions = 8
)
// ruleRead is the highest rule a reader takes (novox/hq ADR 0614 §2, §4): 9 and 10 are kept for the verb wait and the
// capability hold, which a later release gives numbers of their own, so a want of that release reads on every node
// that runs this one. Nothing writes 9 or 10 yet: the verb wait is held under RuleProvisions and the capability hold
// under RuleComposed, each with words naming its edge.
const ruleRead = 10
func checkRule(rule int, of string) error {
if rule < RuleBuildDone || rule > ruleRead {
return fmt.Errorf("delivery: %s names the rule %d, which is not one of 1 to %d", of, rule, ruleRead)
}
return nil
}
// The answers a want carries to a failed backup before an apply (ADR 0539 clause 16): the node agent reads them from
// its own want and from nothing else.
const (
// BackupAgain is the answer to back the module up again before the apply.
BackupAgain = "again"
// BackupSkip is the answer to go on without a backup.
BackupSkip = "skip"
)
func checkFingerprint(fp, of string) error {
if _, err := FingerprintKeyPart(fp); err != nil {
return fmt.Errorf("delivery: %s: %w", of, err)
}
return nil
}
// Want is a node's want for a module: the fingerprint FP the node should run and, before ADR 0547's second milestone,
// the digest Composed of the one composition of it the node may apply (ADR 0539 clause 1; from that milestone the
// node composes its own, ADR 0547 §2, and a want names none), or Removed for a module that left the node's modules (then
// both are empty). When the module's newest fingerprint is not yet allowed on this node, Newest names it and Rule the
// rule that holds the node back; both are empty when FP is the newest. Backup is the answer to a failed backup before
// the apply, BackupAgain or BackupSkip (clause 16), empty when none was asked. Words say why in a sentence. The
// entry's own time says since when.
type Want struct {
V int `json:"v"`
FP string `json:"fp,omitempty"`
Composed string `json:"composed,omitempty"`
Removed bool `json:"removed,omitempty"`
Newest string `json:"newest,omitempty"`
Rule int `json:"rule,omitempty"`
Backup string `json:"backup,omitempty"`
Words string `json:"words,omitempty"`
// Put says the want was written by a put (ADR 0546), and Hold that the put holds it: the rules pass over it until it
// is released. A put's want names no newest fingerprint or rule.
Put bool `json:"put,omitempty"`
Hold bool `json:"hold,omitempty"`
// Yes is the digest of a composition a person allowed the node to move to (novox/hq ADR 0547 §3): a move of a module
// of policy record that the node agent judged to recreate a container or move its data is applied only when its
// want names the composition so allowed. Empty when none was asked.
Yes string `json:"yes,omitempty"`
// Release names the release whose Approve answered the record ask of this want's build (novox/hq ADR 0601 §1): the
// group of pull requests mesh-delivery's gate merged on that Approve. Set with the yes it gave; empty when a person's
// answer to a record ask gave it, or none was needed.
Release string `json:"release,omitempty"`
// Self says the node composes this module itself (ADR 0547 §2): the want names the build alone, and no composition.
// A want that is not Self names the one composition it allows (ADR 0539 clause 1), as before.
Self bool `json:"self,omitempty"`
}
func (w Want) check() error {
if w.Removed {
if w.FP != "" || w.Composed != "" || w.Newest != "" || w.Rule != 0 || w.Backup != "" || w.Put || w.Hold || w.Self || w.Yes != "" || w.Release != "" {
return fmt.Errorf("delivery: a removed want names no fingerprint, composition, rule, backup answer or put")
}
return nil
}
if w.Hold && !w.Put {
return fmt.Errorf("delivery: only a put holds a want")
}
if w.Put && (w.Newest != "" || w.Rule != 0) {
return fmt.Errorf("delivery: a put's want is held back by no rule")
}
if err := checkFingerprint(w.FP, "a want's fingerprint"); err != nil {
return err
}
// From ADR 0547's second milestone a node the catalogue places composes its own modules (§2): its want is Self and
// names the build alone. Any other want names the one composition it allows (ADR 0539 clause 1).
switch {
case w.Self && w.Composed != "":
return fmt.Errorf("delivery: the want of %s says the node composes it, and names a composition", w.FP)
case !w.Self && w.Composed == "":
return fmt.Errorf("delivery: the want of %s names no composition: a want names the digest of the one composition it allows (ADR 0539 clause 1), unless the node composes it itself", w.FP)
case w.Composed != "":
if err := checkDigest(w.Composed, "a want's composition digest"); err != nil {
return err
}
}
if w.Yes != "" {
if err := checkDigest(w.Yes, "the digest a want's yes allows"); err != nil {
return err
}
} else if w.Release != "" {
return fmt.Errorf("delivery: the want of %s names the release %s and no yes it gave", w.FP, w.Release)
}
switch w.Backup {
case "", BackupAgain, BackupSkip:
default:
return fmt.Errorf("delivery: the backup answer %q is not %s or %s", w.Backup, BackupAgain, BackupSkip)
}
if w.Newest == "" && w.Rule == 0 {
return nil
}
if err := checkFingerprint(w.Newest, "a want's newest fingerprint"); err != nil {
return err
}
if w.Newest == w.FP {
return fmt.Errorf("delivery: a want at its newest fingerprint names no rule holding it back")
}
return checkRule(w.Rule, "a want held back")
}
// Held is a fingerprint the assigner holds: the node whose health check failed on it, and the check's words.
type Held struct {
V int `json:"v"`
Node string `json:"node"`
Words string `json:"words,omitempty"`
}
func (h Held) check() error {
return checkName("node", h.Node)
}
// Difference is where a want and what the controller sends differ for a module on a node (ADR 0528 clause 4): the
// want's fingerprint (Want, empty when the node has no want for the module, WantRemoved when the want is removed), the
// fingerprint of the build the controller's assignment names (Sent, empty when it sends none or the build names no
// fingerprint; SentBuild names that build), and the rule that explains the difference, or Unexplained.
type Difference struct {
V int `json:"v"`
Want string `json:"want,omitempty"`
WantRemoved bool `json:"want-removed,omitempty"`
Sent string `json:"sent,omitempty"`
SentBuild string `json:"sent-build,omitempty"`
Rule int `json:"rule,omitempty"`
Unexplained bool `json:"unexplained,omitempty"`
Words string `json:"words,omitempty"`
}
func (d Difference) check() error {
if d.WantRemoved && d.Want != "" {
return fmt.Errorf("delivery: a difference with a removed want names no wanted fingerprint")
}
if d.Want != "" {
if err := checkFingerprint(d.Want, "a difference's want"); err != nil {
return err
}
}
if d.Sent != "" {
if err := checkFingerprint(d.Sent, "a difference's sent fingerprint"); err != nil {
return err
}
}
if !d.WantRemoved && d.Want == d.Sent && (d.Want != "" || d.SentBuild == "") {
return fmt.Errorf("delivery: a difference whose want and send are both %q is no difference", d.Want)
}
if d.Unexplained {
if d.Rule != 0 {
return fmt.Errorf("delivery: an unexplained difference names no rule")
}
return nil
}
return checkRule(d.Rule, "a difference")
}
// EncodeWant is w as JSON at Version; it refuses a fingerprint not of the new form, a removed want naming one, and a
// want held back without its rule.
func EncodeWant(w Want) ([]byte, error) {
if err := w.check(); err != nil {
return nil, err
}
w.V = Version
return json.Marshal(w)
}
// DecodeWant reads a want; it refuses an unknown version (ErrUnknownVersion) and what EncodeWant refuses.
func DecodeWant(data []byte) (Want, error) {
var w Want
if err := decode(data, &w); err != nil {
return Want{}, err
}
if err := w.check(); err != nil {
return Want{}, err
}
return w, nil
}
// SameWant says whether two wants are the same value, so the assigner writes only when they differ.
func SameWant(a, b Want) bool { return a == b }
// NodeWant is a node's own want (WantNodeKey): the digest Composed of the node's own composition it may apply. Words
// say why in a sentence.
type NodeWant struct {
V int `json:"v"`
Composed string `json:"composed"`
Words string `json:"words,omitempty"`
// Self says the node composes its own composition (ADR 0547 §2) and the want names none.
Self bool `json:"self,omitempty"`
}
// EncodeNodeWant is w as JSON at Version; it refuses a digest not of the sha256 form, and a want that names no
// composition unless it says the node composes its own (Self, ADR 0547 §2).
func EncodeNodeWant(w NodeWant) ([]byte, error) {
if err := w.check(); err != nil {
return nil, err
}
w.V = Version
return json.Marshal(w)
}
// DecodeNodeWant reads a node's own want; it refuses an unknown version (ErrUnknownVersion), a field a node's own want
// does not have (a fingerprint among them), and what EncodeNodeWant refuses.
func DecodeNodeWant(data []byte) (NodeWant, error) {
if err := checkVersion(data); err != nil {
return NodeWant{}, err
}
var w NodeWant
d := json.NewDecoder(bytes.NewReader(data))
d.DisallowUnknownFields()
if err := d.Decode(&w); err != nil {
return NodeWant{}, fmt.Errorf("delivery: a node's own want does not fit its shape: %w", err)
}
if err := w.check(); err != nil {
return NodeWant{}, err
}
return w, nil
}
// check refuses a node's own want that names no composition and does not say the node composes its own, or does both.
func (w NodeWant) check() error {
switch {
case w.Self && w.Composed != "":
return fmt.Errorf("delivery: the node's own want says the node composes it, and names a composition")
case w.Self:
return nil
}
return checkDigest(w.Composed, "the node's own want's composition digest")
}
// EncodeHeld is h as JSON at Version; it refuses a node name a key refuses.
func EncodeHeld(h Held) ([]byte, error) {
if err := h.check(); err != nil {
return nil, err
}
h.V = Version
return json.Marshal(h)
}
// DecodeHeld reads a hold; it refuses an unknown version (ErrUnknownVersion) and what EncodeHeld refuses.
func DecodeHeld(data []byte) (Held, error) {
var h Held
if err := decode(data, &h); err != nil {
return Held{}, err
}
if err := h.check(); err != nil {
return Held{}, err
}
return h, nil
}
// SameHeld says whether two holds are the same value.
func SameHeld(a, b Held) bool { return a == b }
// EncodeDifference is d as JSON at Version; it refuses a difference with neither a rule nor unexplained, one with both,
// a rule outside the six, and a want equal to the send.
func EncodeDifference(d Difference) ([]byte, error) {
if err := d.check(); err != nil {
return nil, err
}
d.V = Version
return json.Marshal(d)
}
// DecodeDifference reads a difference; it refuses an unknown version (ErrUnknownVersion) and what EncodeDifference
// refuses.
func DecodeDifference(data []byte) (Difference, error) {
var d Difference
if err := decode(data, &d); err != nil {
return Difference{}, err
}
if err := d.check(); err != nil {
return Difference{}, err
}
return d, nil
}
// SameDifference says whether two differences are the same value.
func SameDifference(a, b Difference) bool { return reflect.DeepEqual(a, b) }
// The assigner's composition check (novox/hq ADR 0446 clauses 1 to 5, ADR 0528 clause 5, ADR 0482 clauses 3 and 9): for
// each group the checker judges, the assigner composes the machines the change touches and writes its verdict under
// the checker's key of that check, `<escaped group>.<heads hash>` (CheckerVerdictKey's form), in its bucket
// compositions. The checker watches that key and reads it into its verdict's composition (Composition); a key not yet
// written is a composition still running.
const (
// AssignerCompositionsBucket is the assigner's bucket of composition verdicts.
AssignerCompositionsBucket = AssignerModule + "_compositions"
// AssignerCompositionsHistory is the revisions the bucket keeps per key.
AssignerCompositionsHistory = 5
// CompositionWaits is the state of a composition that waits for what it composes from (ADR 0446 clause 3: the
// indexer holds no candidates for the group yet), its Words saying what; it is no verdict.
CompositionWaits = "waits"
// CompositionRebuildWarning is the most modules a merge may rebuild before the composition warns (ADR 0446 clause 1).
CompositionRebuildWarning = 12
)
// CompositionKey is the key, in AssignerCompositionsBucket, of the composition of a group's check at its heads: the
// checker's verdict key of the same check (CheckerVerdictKey).
func CompositionKey(group, headsHash string) (string, error) {
return CheckerVerdictKey(group, headsHash)
}
// ParseCompositionKey is the inverse of CompositionKey; it refuses a want, held or difference key, naming it.
func ParseCompositionKey(key string) (group, headsHash string, err error) {
if group, headsHash, err = ParseCheckerVerdictKey(key); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a composition key, <group>.<heads hash>: %w", key, err)
}
return group, headsHash, nil
}
// CompositionUnanswered is the Words of a composition that failed only because a node placed at the head or on the
// trunk did not answer the group's placement candidate within its bound (novox/hq ADR 0559 §5): the checker takes a
// later passing composition of the same heads in without a new check, since the failure was a part of the mesh not
// heard, not the change (ADR 0548 §1).
const CompositionUnanswered = "did not answer the placement candidate within its bound; judged again when it answers"
// CompositionVerdict is the value of a composition key: the group and its heads hash, the State (pass, fail or waits),
// the Words of a refusal or of what it waits for, the mesh-sdk commit it composed with (the assigner's own, or the
// group's mesh-sdk member's head, ADR 0482 clause 3), the machines composed, the first machine that failed, the
// warnings that fail nothing (more than twelve modules rebuilt; what was already broken before the change, ADR 0446
// clause 1), and the time it took in integer milliseconds.
type CompositionVerdict struct {
V int `json:"v"`
Group string `json:"group"`
Heads string `json:"heads"`
State string `json:"state"`
Words string `json:"words,omitempty"`
SDK string `json:"sdk,omitempty"`
Machines []string `json:"machines,omitempty"`
Machine string `json:"machine,omitempty"`
Warnings []string `json:"warnings,omitempty"`
Took int64 `json:"took,omitempty"`
}
func (c CompositionVerdict) check() error {
if c.Group == "" {
return fmt.Errorf("delivery: a composition names no group")
}
if !isHex12(c.Heads) {
return fmt.Errorf("delivery: the composition's heads hash %q is not 12 lowercase hex digits", c.Heads)
}
if c.Took < 0 {
return fmt.Errorf("delivery: the composition's time is negative")
}
switch c.State {
case CompositionPass:
if c.SDK == "" {
return fmt.Errorf("delivery: a composition that passed names no mesh-sdk commit")
}
if c.Machine != "" || c.Words != "" {
return fmt.Errorf("delivery: a composition that passed names no failure")
}
case CompositionFail:
if c.Words == "" {
return fmt.Errorf("delivery: a composition that failed gives no words")
}
if c.SDK == "" {
return fmt.Errorf("delivery: a composition that failed names no mesh-sdk commit")
}
case CompositionWaits:
if c.Words == "" {
return fmt.Errorf("delivery: a composition that waits does not say for what")
}
if c.Machine != "" || len(c.Machines) > 0 {
return fmt.Errorf("delivery: a composition that waits composed no machine")
}
default:
return fmt.Errorf("delivery: the composition state %q is not %s, %s or %s", c.State, CompositionPass, CompositionFail, CompositionWaits)
}
return nil
}
// Composition is what the checker writes into its verdict from c: nil while c waits (the check is still running).
func (c CompositionVerdict) Composition() *Composition {
if c.State == CompositionWaits {
return nil
}
return &Composition{State: c.State, Words: c.Words, SDK: c.SDK, Machines: c.Machines}
}
// EncodeCompositionVerdict is c as JSON at Version; it refuses an unknown state, a pass or fail naming no mesh-sdk
// commit, a fail without words, a pass naming a failure, and a wait that composed a machine.
func EncodeCompositionVerdict(c CompositionVerdict) ([]byte, error) {
if err := c.check(); err != nil {
return nil, err
}
c.V = Version
return json.Marshal(c)
}
// DecodeCompositionVerdict reads a composition; it refuses an unknown version (ErrUnknownVersion) and what
// EncodeCompositionVerdict refuses.
func DecodeCompositionVerdict(data []byte) (CompositionVerdict, error) {
var c CompositionVerdict
if err := decode(data, &c); err != nil {
return CompositionVerdict{}, err
}
if err := c.check(); err != nil {
return CompositionVerdict{}, err
}
return c, nil
}
// SameCompositionVerdict says whether two compositions are the same value.
func SameCompositionVerdict(a, b CompositionVerdict) bool { return reflect.DeepEqual(a, b) }
-319
View File
@@ -1,319 +0,0 @@
package delivery
import (
"errors"
"strings"
"testing"
"time"
)
// 1297/B1a: a want key round-trips through its parser; a held key, a have key and a node's own key are refused,
// naming the key.
func TestWantKey(t *testing.T) {
key, err := WantKey("ace", "mesh-mail")
if err != nil || key != "want.ace.mesh-mail" {
t.Fatalf("WantKey = %q, %v", key, err)
}
node, module, err := ParseWantKey(key)
if err != nil || node != "ace" || module != "mesh-mail" {
t.Errorf("ParseWantKey = %q, %q, %v", node, module, err)
}
for _, n := range [][2]string{{"ace", "node"}, {"a.b", "m"}, {"ace", ""}, {"", "m"}, {"ace", "m*"}} {
if _, err := WantKey(n[0], n[1]); err == nil {
t.Errorf("WantKey(%q, %q) accepted", n[0], n[1])
}
}
held, _ := HeldKey("mesh-mail", fp2of('a'))
for _, bad := range []string{"ace.mesh-mail", "want.ace.node", "want.ace", "want.ace.m.x", held, "wants.ace.m", ""} {
if _, _, err := ParseWantKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseWantKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1297/B1a: a held key round-trips; a want key, a builder build key and a fingerprint not of the new form are refused.
func TestHeldKey(t *testing.T) {
fp := fp2of('b')
key, err := HeldKey("mesh-mail", fp)
if err != nil || key != "held.mesh-mail."+strings.Repeat("b", 64) {
t.Fatalf("HeldKey = %q, %v", key, err)
}
module, back, err := ParseHeldKey(key)
if err != nil || module != "mesh-mail" || back != fp {
t.Errorf("ParseHeldKey = %q, %q, %v", module, back, err)
}
src1 := SourcePrefix + strings.Repeat("1", 64)
if _, err := HeldKey("mesh-mail", src1); err == nil {
t.Errorf("a src1 fingerprint was held")
}
build, _ := BuilderBuildKey("mesh-mail", fp)
for _, bad := range []string{build, "want.ace.mesh-mail", "held.mesh-mail.abc", "held." + strings.Repeat("b", 64), ""} {
if _, _, err := ParseHeldKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseHeldKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1297/B1a: a difference key round-trips; a want or held key and a node's own key are refused.
func TestDifferenceKey(t *testing.T) {
key, err := DifferenceKey("g14", "mesh-mail")
if err != nil || key != "g14.mesh-mail" {
t.Fatalf("DifferenceKey = %q, %v", key, err)
}
node, module, err := ParseDifferenceKey(key)
if err != nil || node != "g14" || module != "mesh-mail" {
t.Errorf("ParseDifferenceKey = %q, %q, %v", node, module, err)
}
held, _ := HeldKey("mesh-mail", fp2of('a'))
for _, bad := range []string{"want.g14.mesh-mail", held, "g14.node", "g14", ""} {
if _, _, err := ParseDifferenceKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseDifferenceKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1297/B1a: wants, holds and differences round-trip through their encoders unchanged, at Version.
func TestAssignerValuesRoundTrip(t *testing.T) {
wants := []Want{
{FP: fp2of('a'), Composed: hashOf('1')},
{FP: fp2of('a'), Composed: hashOf('1'), Newest: fp2of('b'), Rule: RuleCanary, Words: "the canary ace does not run it healthy yet"},
{Removed: true, Words: "the module left g14"},
{FP: fp2of('a'), Composed: hashOf('1'), Put: true, Hold: true, Words: "put by the operator: the bus's user list"},
// A rule a later release numbers (9, 10: the verb wait, the capability hold, ADR 0614) reads here already.
{FP: fp2of('a'), Composed: hashOf('1'), Newest: fp2of('b'), Rule: 10, Words: "the verb x.y it invokes is served by no running build"},
}
for _, w := range wants {
data, err := EncodeWant(w)
if err != nil {
t.Fatalf("EncodeWant(%+v): %v", w, err)
}
got, err := DecodeWant(data)
w.V = Version
if err != nil || !SameWant(got, w) {
t.Errorf("want %+v came back %+v, %v", w, got, err)
}
}
at := time.Date(2026, 10, 25, 21, 0, 0, 0, time.UTC)
p := Put{FP: fp2of('a'), Hold: true, Why: "the bus's user list", By: "shanks.node-tools", At: at}.Said(at, PutAsked, "put asked")
p = p.Said(at, PutAsked, "put asked").Said(at.Add(time.Second), PutWanted, "want written")
p.Composed = hashOf('1')
if len(p.Steps) != 2 || p.Step != PutWanted || PutEnded(p.Step) || !PutEnded(PutReleased) {
t.Errorf("a put's steps are %+v at %s", p.Steps, p.Step)
}
pkey, _ := PutKey("novox", "nats")
if n, m, err := ParsePutKey(pkey); err != nil || n != "novox" || m != "nats" {
t.Errorf("the put key %s reads %s %s, %v", pkey, n, m, err)
}
pdata, err := EncodePut(p)
if err != nil {
t.Fatal(err)
}
if gotP, err := DecodePut(pdata); err != nil || gotP.FP != p.FP || gotP.Step != PutWanted || len(gotP.Steps) != 2 || !gotP.Hold {
t.Errorf("the put came back %+v, %v", gotP, err)
}
h := Held{Node: "ace", Words: "its health check failed: connection refused"}
data, err := EncodeHeld(h)
if err != nil {
t.Fatal(err)
}
gotH, err := DecodeHeld(data)
h.V = Version
if err != nil || !SameHeld(gotH, h) {
t.Errorf("held came back %+v, %v", gotH, err)
}
diffs := []Difference{
{Want: fp2of('a'), Sent: fp2of('b'), Rule: RuleCanary},
{Sent: fp2of('b'), Rule: RuleBuildDone},
{WantRemoved: true, Sent: fp2of('b'), Unexplained: true},
{SentBuild: "build-17", Rule: RuleGrant},
}
for _, d := range diffs {
data, err := EncodeDifference(d)
if err != nil {
t.Fatalf("EncodeDifference(%+v): %v", d, err)
}
got, err := DecodeDifference(data)
d.V = Version
if err != nil || !SameDifference(got, d) {
t.Errorf("difference %+v came back %+v, %v", d, got, err)
}
}
}
// 1297/B1a: what a want, a hold and a difference may not say is refused on encode and on decode.
func TestAssignerValuesRefused(t *testing.T) {
src1 := SourcePrefix + strings.Repeat("1", 64)
for _, w := range []Want{
{},
{FP: src1},
{Removed: true, FP: fp2of('a')},
{FP: fp2of('a'), Newest: fp2of('b')},
{FP: fp2of('a'), Newest: fp2of('a'), Rule: 3},
{FP: fp2of('a'), Composed: hashOf('1'), Newest: fp2of('b'), Rule: 11},
{FP: fp2of('a'), Composed: hashOf('1'), Hold: true},
{FP: fp2of('a'), Composed: hashOf('1'), Put: true, Newest: fp2of('b'), Rule: 3},
{Removed: true, Put: true},
} {
if _, err := EncodeWant(w); err == nil {
t.Errorf("EncodeWant(%+v) accepted", w)
}
}
if _, err := EncodeHeld(Held{Node: "a.b"}); err == nil {
t.Errorf("a held naming a dotted node was accepted")
}
for _, d := range []Difference{
{Want: fp2of('a'), Sent: fp2of('b')},
{Want: fp2of('a'), Sent: fp2of('b'), Rule: 3, Unexplained: true},
{Want: fp2of('a'), Sent: fp2of('a'), Rule: 3},
{},
{Want: fp2of('a'), Sent: fp2of('b'), Rule: 11},
{WantRemoved: true, Want: fp2of('a'), Rule: 1},
} {
if _, err := EncodeDifference(d); err == nil {
t.Errorf("EncodeDifference(%+v) accepted", d)
}
}
for _, p := range []Put{
{FP: fp2of('a'), Why: "w", Step: "walking"},
{FP: fp2of('a'), Step: PutAsked},
{FP: "src2:abc", Why: "w", Step: PutAsked},
{FP: fp2of('a'), Why: "w", Step: PutWanted, Composed: "sha256:x"},
} {
if _, err := EncodePut(p); err == nil {
t.Errorf("EncodePut(%+v) accepted", p)
}
}
if _, _, err := ParsePutKey("ace.node"); err == nil {
t.Errorf("a node's own key was read as a put")
}
if _, err := DecodeWant([]byte(`{"v":2,"fp":"` + fp2of('a') + `"}`)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("a want of version 2 was read: %v", err)
}
if _, err := DecodeDifference([]byte(`{"v":1,"want":"` + fp2of('a') + `","sent":"` + fp2of('b') + `"}`)); err == nil {
t.Errorf("a difference naming no rule was read")
}
}
// 1297/B1b: a composition key is the checker's verdict key of the same check, and round-trips; a want, held or
// difference key is refused, naming it.
func TestCompositionKey(t *testing.T) {
key, err := CompositionKey("feat/r4g", "0123456789ab")
verdict, _ := CheckerVerdictKey("feat/r4g", "0123456789ab")
if err != nil || key != verdict {
t.Fatalf("CompositionKey = %q, %v; the checker's key is %q", key, err, verdict)
}
group, heads, err := ParseCompositionKey(key)
if err != nil || group != "feat/r4g" || heads != "0123456789ab" {
t.Errorf("ParseCompositionKey = %q, %q, %v", group, heads, err)
}
held, _ := HeldKey("mesh-mail", fp2of('a'))
for _, bad := range []string{"want.ace.mesh-mail", held, "ace.mesh-mail", "g.0123456789AB", ""} {
if _, _, err := ParseCompositionKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseCompositionKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1297/B1b: a composition in each state round-trips; the checker's composition of a pass or fail carries its state,
// words, mesh-sdk commit and machines, and a wait gives none.
func TestCompositionVerdictRoundTrip(t *testing.T) {
for _, c := range []CompositionVerdict{
{Group: "g", Heads: "0123456789ab", State: CompositionPass, SDK: "7aacbc1", Machines: []string{"ace"}, Took: 120},
{Group: "g", Heads: "0123456789ab", State: CompositionFail, SDK: "7aacbc1", Machines: []string{"ace", "g14"}, Machine: "g14",
Words: "g14 runs mesh-mail, which the change removes from its source", Warnings: []string{"the merge rebuilds 13 modules"}},
{Group: "g", Heads: "0123456789ab", State: CompositionWaits, Words: "the indexer holds no candidates for the group yet"},
} {
data, err := EncodeCompositionVerdict(c)
if err != nil {
t.Fatalf("EncodeCompositionVerdict(%+v): %v", c, err)
}
got, err := DecodeCompositionVerdict(data)
c.V = Version
if err != nil || !SameCompositionVerdict(got, c) {
t.Errorf("composition %+v came back %+v, %v", c, got, err)
}
comp := got.Composition()
if c.State == CompositionWaits {
if comp != nil {
t.Errorf("a wait gave the checker a composition %+v", comp)
}
continue
}
if comp == nil || comp.State != c.State || comp.SDK != c.SDK || comp.Words != c.Words || len(comp.Machines) != len(c.Machines) {
t.Errorf("the checker's composition of %+v is %+v", c, comp)
}
if err := comp.check(); err != nil {
t.Errorf("the checker's composition of %+v is refused by the verdict: %v", c, err)
}
}
}
// 1297/B1b: what a composition may not say is refused.
func TestCompositionVerdictRefused(t *testing.T) {
for _, c := range []CompositionVerdict{
{Heads: "0123456789ab", State: CompositionPass, SDK: "x"},
{Group: "g", Heads: "0123", State: CompositionPass, SDK: "x"},
{Group: "g", Heads: "0123456789ab", State: CompositionPass},
{Group: "g", Heads: "0123456789ab", State: CompositionPass, SDK: "x", Machine: "ace"},
{Group: "g", Heads: "0123456789ab", State: CompositionFail, SDK: "x"},
{Group: "g", Heads: "0123456789ab", State: CompositionFail, Words: "w"},
{Group: "g", Heads: "0123456789ab", State: CompositionWaits},
{Group: "g", Heads: "0123456789ab", State: CompositionWaits, Words: "w", Machines: []string{"ace"}},
{Group: "g", Heads: "0123456789ab", State: CompositionNotRun},
{Group: "g", Heads: "0123456789ab", State: CompositionPass, SDK: "x", Took: -1},
} {
if _, err := EncodeCompositionVerdict(c); err == nil {
t.Errorf("EncodeCompositionVerdict(%+v) accepted", c)
}
}
}
// 1394/B1b: a grant key round-trips; a want or held key is refused, naming it; a grant round-trips and one naming an
// empty account is refused.
func TestTheGrantsIssued(t *testing.T) {
fp := fp2of('c')
key, err := PartGrantKey("mesh-delivery-assigner", fp)
if err != nil || key != "mesh-delivery-assigner."+strings.Repeat("c", 64) {
t.Fatalf("PartGrantKey = %q, %v", key, err)
}
module, back, err := ParsePartGrantKey(key)
if err != nil || module != "mesh-delivery-assigner" || back != fp {
t.Errorf("ParsePartGrantKey = %q, %q, %v", module, back, err)
}
held, _ := HeldKey("mesh-mail", fp)
for _, bad := range []string{"want.ace.mesh-mail", held, "ace.mesh-mail", ""} {
if _, _, err := ParsePartGrantKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParsePartGrantKey(%q) = %v, want a refusal naming it", bad, err)
}
}
for _, g := range []Grant{{}, {Accounts: []string{"UABC", "UDEF"}}} {
data, err := EncodeGrant(g)
if err != nil {
t.Fatal(err)
}
got, err := DecodeGrant(data)
g.V = Version
if err != nil || !SameGrant(got, g) {
t.Errorf("grant %+v came back %+v, %v", g, got, err)
}
}
if _, err := EncodeGrant(Grant{Accounts: []string{""}}); err == nil {
t.Errorf("a grant naming an empty account was accepted")
}
g := Grant{Accounts: []string{"ace.mesh-build-go", "shanks.mesh-mail.reader"}}
for _, c := range []struct {
g Grant
node, module string
names bool
}{
{g, "ace", "mesh-build-go", true},
{g, "shanks", "mesh-mail", true},
{g, "g14", "mesh-build-go", false},
{g, "ace", "mesh-build", false},
{Grant{}, "novox", "nats", true},
} {
if got := c.g.NamesNode(c.node, c.module); got != c.names {
t.Errorf("%v names %s.%s: %v, want %v", c.g.Accounts, c.node, c.module, got, c.names)
}
}
}
-348
View File
@@ -1,348 +0,0 @@
package delivery
import (
"encoding/json"
"fmt"
"reflect"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// The builder's keys (novox/hq ADR 0429 clauses 5 and 16, ADR 0437 clause 5, ADR 0405 clauses 2 and 7, ADR 0453
// clause 4): a build is `<module>.<fingerprint key part>` in the builder's bucket, one key per fingerprint, since equal
// fingerprints are equal content and the same source is never built twice. They are written only by the builder part
// from R4c and read by the checker, the assigner and the view. They are not the ended controller's build records
// (buildrecord.go: BuildKey).
//
// **The rules its writer keeps.** A builder claims a fingerprint that has no build key by creating it queued or
// building, a create that fails when the key exists, and moves it by compare-and-set; a builder reclaims only a build
// whose `by` is offline in presence. A failed key is built again as a new revision of the same key (ADR 0429 clause
// 18). Only the builder's credential writes the bucket (clause 5); every other part reads it.
const (
// BuilderModule is the module of the builder part.
BuilderModule = "mesh-delivery-builder"
// BuilderBuildsBucket is the builder's bucket of builds: `<owner module>_<name>`.
BuilderBuildsBucket = BuilderModule + "_builds"
// BuilderBuildsHistory is the revisions the bucket keeps per key.
BuilderBuildsHistory = 5
)
// The states of a build.
const (
BuildQueued = "queued"
BuildBuilding = "building"
BuildDone = "done"
BuildFailed = "failed"
)
// BuilderBuildKey is the key, in BuilderBuildsBucket, of a module's build at a fingerprint: `<module>.<hex>`, the hex
// being the fingerprint's key part (FingerprintKeyPart), since a bus key may not hold ':'. A module holding a dot or a
// character a key refuses, and a fingerprint not of the new form, are refused, naming them.
func BuilderBuildKey(module, fingerprint string) (string, error) {
if err := checkName("module", module); err != nil {
return "", err
}
part, err := FingerprintKeyPart(fingerprint)
if err != nil {
return "", err
}
return module + "." + part, nil
}
// ParseBuilderBuildKey is the inverse of BuilderBuildKey: the module and the fingerprint (`src2:` and the hex).
func ParseBuilderBuildKey(key string) (module, fingerprint string, err error) {
module, part, ok := strings.Cut(key, ".")
if !ok {
return "", "", fmt.Errorf("delivery: %q is not a builder build key, <module>.<fingerprint hex>", key)
}
if err := checkName("module", module); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a builder build key: %w", key, err)
}
if fingerprint, err = ParseFingerprintKeyPart(part); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a builder build key: %w", key, err)
}
return module, fingerprint, nil
}
// BuiltArtifact is one artifact of a done build: its name in the manifest, its kind, its content digest (`sha256:` and
// 64 hex digits) and the reference that names where it is (novox/hq ADR 0498 clauses 1, 5 and 6; the lead's decision
// L1 of R4e), by kind:
//
// bundle, archive artifact-store://<module>/<artifact>/blobs/<digest> the blob pushed to the artifact store
// image, upstream <registry>/<module>/<artifact>@<digest> the image registry's reference, which
// artifact-store://<module>/<artifact>@<digest> catalogue.Recorded keeps without its address
// package <name>@<version> the version published from the trunk
//
// From the registry milestone a done key names the reference its seat answered (novox/hq ADR 0605 §9, design 51 *The
// done key names the reference the seat answered*): `<seat>/<module>/<artifact>@<digest>`, <seat> the registry seat it
// came through (oci-artifact-registry for a bundle or an archive, container-image-registry for an image or an upstream
// copy), and `<name>@<version>` for a package as before. A key written in the older form keeps it; this reads both, and
// no build is built again for its form (catalogue.InArtifactStore reads both the same way).
//
// <artifact> is Name and <digest> is Digest. A reference is empty for a candidate's image or package, which is never
// pushed or published (ADR 0498 clause 6), and in a key done before R4e; the controller delivers only from a key whose
// every artifact names one (Referenced, clause 8). A kind left empty (a key written before kinds) is not checked.
type BuiltArtifact struct {
Name string `json:"name"`
Kind string `json:"kind,omitempty"`
Digest string `json:"digest"`
Reference string `json:"reference,omitempty"`
// References are every form this artifact is named in, Reference among them (novox/hq ADR 0618, ADR 0624): a build
// keeps each form that a counted node-engine states, away nodes included, and Reference stays the form every one
// of them reads. A reader that does not know the field reads Reference alone.
References []string `json:"references,omitempty"`
}
const artifactStore = "artifact-store://"
// referenceProblem says what is wrong with a's reference for its kind, or "".
func (a BuiltArtifact) referenceProblem() string {
r := a.Reference
if r == "" || a.Kind == "" {
return ""
}
// An upstream copy may be named by the holder's own naming, upstream/<host>/<path>@<digest>, in either form: the
// holder names it, not the module that asked for it (catalogue.InUpstream).
if path, kept := catalogue.InArtifactStore(r); kept && a.Kind == "upstream" {
if _, image, ok := catalogue.InUpstream(path); ok {
if _, digest, _ := strings.Cut(image, "@"); digest != a.Digest {
return "does not name its digest " + a.Digest
}
return ""
}
}
if seat, path, ok := catalogue.InRegistrySeat(r); ok {
want := catalogue.ContainerImageRegistrySeat
if a.Kind == "bundle" || a.Kind == "archive" {
want = catalogue.OciArtifactRegistrySeat
}
module, rest, _ := strings.Cut(path, "/")
if a.Kind == "package" || seat != want || module == "" || rest != a.Name+"@"+a.Digest {
return "is not " + want + "/<module>/" + a.Name + "@" + a.Digest
}
return ""
}
switch a.Kind {
case "bundle", "archive":
path, ok := strings.CutPrefix(r, artifactStore)
module, rest, _ := strings.Cut(path, "/")
if !ok || module == "" || rest != a.Name+"/blobs/"+a.Digest {
return "is not artifact-store://<module>/" + a.Name + "/blobs/" + a.Digest
}
case "image", "upstream":
path, digest, ok := strings.Cut(r, "@")
if !ok || digest != a.Digest {
return "does not name its digest " + a.Digest
}
var host, module, name string
if rest, kept := strings.CutPrefix(path, artifactStore); kept {
module, name, _ = strings.Cut(rest, "/")
host = "kept"
} else {
parts := strings.Split(path, "/")
if len(parts) == 3 {
host, module, name = parts[0], parts[1], parts[2]
}
}
if host == "" || !(host == "kept" || host == "localhost" || strings.ContainsAny(host, ".:")) || module == "" || name != a.Name {
return "is not <registry>/<module>/" + a.Name + "@" + a.Digest
}
case "package":
at := strings.LastIndex(r, "@")
if at <= 0 || at == len(r)-1 || r[:at] == "@" {
return "is not <name>@<version>"
}
}
return ""
}
// Referenced says whether b is done and every artifact names where it is, which the controller delivers from (novox/hq
// ADR 0498 clause 8), and names the artifacts that name no reference. A done key with no artifact (a module that builds
// none, issue 1613) is referenced.
func (b BuilderBuild) Referenced() (ok bool, missing []string) {
for _, a := range b.Artifacts {
if a.Reference == "" {
missing = append(missing, a.Name)
}
}
return b.State == BuildDone && len(missing) == 0, missing
}
// The prefixes of a failed key's error that a build module answers (novox/hq ADR 0498 clauses 2, 4 and 5; ADR 0478
// clause 2), each followed by the artifact and the words of what refused it.
const (
// NotStored: the push of a bundle, an archive or an upstream copy to the artifact store failed; built again.
NotStored = "not stored: "
// NotPinned: an upstream artifact names no digest.
NotPinned = "not pinned: "
// VersionTaken: the package registry holds the package's version with another packed file.
VersionTaken = "version taken: "
// RefusedKind: no build module builds this kind of artifact in its language.
RefusedKind = "refused kind: "
// WaitsFor: a fetch found the module's lock naming a package version the registry does not hold yet; the builder
// writes the key queued with Waits (novox/hq ADR 0500 clause 6), and it is built once the package is published.
WaitsFor = "waits for: "
)
// FailedUntilChanged says whether a failed key's error is one that is not built again until the module's fingerprint
// changes (not pinned, version taken, refused kind: the same source fails the same way); every other failed key, a
// push that was not stored included, is built again as a new revision (ADR 0429 clause 18).
func FailedUntilChanged(err string) bool {
return strings.HasPrefix(err, NotPinned) || strings.HasPrefix(err, VersionTaken) || strings.HasPrefix(err, RefusedKind)
}
// ToolchainUsed is what a build used for one language (ADR 0405 clauses 2 and 7): the installed toolchain's version and
// the build module's recipe version.
type ToolchainUsed struct {
Version string `json:"version"`
Recipe string `json:"recipe"`
}
// BuilderBuild is the value of a builder build key: its State (queued, building, done or failed), By the builder
// instance that holds it, Step what it is doing or did last, Manifest the digest of the module's manifest (the key of
// the indexer's manifests bucket), Artifacts when done, Toolchains by language when it built or is building (a done
// build names them, so two builders that differ are seen: ADR 0405 clause 2), Error when failed, and Waits when queued for a
// package version.
type BuilderBuild struct {
V int `json:"v"`
State string `json:"state"`
By string `json:"by,omitempty"`
Step string `json:"step,omitempty"`
Manifest string `json:"manifest,omitempty"`
Artifacts []BuiltArtifact `json:"artifacts,omitempty"`
Toolchains map[string]ToolchainUsed `json:"toolchains,omitempty"`
Error string `json:"error,omitempty"`
// Waits is, on a queued key only, the package version `<name>@<version>` the module's lock names and the registry
// does not yet hold: the key is not claimed until that package's trunk key is done and published (novox/hq ADR 0500
// clause 6). A reader that does not know the field reads the key as queued.
Waits string `json:"waits,omitempty"`
// Asks are, on a building key, the asks the builder put for it by build seat language (AskGo, AskNode, AskImage), each
// its id (novox/hq ADR 0525 §1; issue 1887 A4b): a builder that takes the key over at its start waits for those asks'
// steps and never puts them again. A reader that does not know the field reads the key as before.
Asks map[string]string `json:"asks,omitempty"`
// Imports are, on a done key, the mesh packages the built programs import, as the build module answered them
// (BuildAnswer.Imports; novox/hq ADR 0593 §2): the builder carries them onto the build's end (BuildLogEntry.Imports).
Imports []string `json:"imports,omitempty"`
// Needs are, on a done key only, the reader capabilities (`form:<name>`) the build's reference forms need of a
// node-engine, beside the ones its manifest needs (novox/hq ADR 0618 §1). A reader that does not know the field
// reads the build as needing none.
Needs []string `json:"needs,omitempty"`
}
func (b BuilderBuild) check() error {
if err := CheckImports(b.Imports); err != nil {
return err
}
switch b.State {
case BuildQueued, BuildBuilding, BuildDone, BuildFailed:
default:
return fmt.Errorf("delivery: the build state %q is not queued, building, done or failed", b.State)
}
if b.Manifest != "" {
if _, err := ManifestsKey(b.Manifest); err != nil {
return err
}
}
for language, t := range b.Toolchains {
if t.Version == "" {
return fmt.Errorf("delivery: the toolchain of %q has no version", language)
}
if t.Recipe == "" {
return fmt.Errorf("delivery: the toolchain of %q has no recipe version", language)
}
}
for i, a := range b.Artifacts {
if a.Name == "" {
return fmt.Errorf("delivery: artifact %d of the build has no name", i)
}
if a.Digest == "" {
return fmt.Errorf("delivery: the artifact %q of the build has no digest", a.Name)
}
if p := a.referenceProblem(); p != "" {
return fmt.Errorf("delivery: the reference %q of the %s %q %s", a.Reference, a.Kind, a.Name, p)
}
if len(a.References) > 0 && !slices.Contains(a.References, a.Reference) {
return fmt.Errorf("delivery: the references of the %s %q do not name its reference %q", a.Kind, a.Name, a.Reference)
}
for _, r := range a.References {
if p := (BuiltArtifact{Name: a.Name, Kind: a.Kind, Digest: a.Digest, Reference: r}).referenceProblem(); p != "" {
return fmt.Errorf("delivery: the reference %q of the %s %q %s", r, a.Kind, a.Name, p)
}
}
}
for _, n := range b.Needs {
if !strings.HasPrefix(n, "form:") || len(n) == len("form:") {
return fmt.Errorf("delivery: the need %q of the build is not form:<name>", n)
}
}
if len(b.Needs) > 0 && b.State != BuildDone {
return fmt.Errorf("delivery: a %s build names needs; only a done key does", b.State)
}
for language, id := range b.Asks {
if b.State != BuildBuilding {
return fmt.Errorf("delivery: a %s build names the ask %s; only a building key does", b.State, id)
}
if err := checkAskLanguage(language); err != nil {
return err
}
if err := checkAskID(id); err != nil {
return err
}
}
if b.Waits != "" {
if b.State != BuildQueued {
return fmt.Errorf("delivery: a %s build waits for %q; only a queued key waits for a package", b.State, b.Waits)
}
if p := (BuiltArtifact{Kind: "package", Reference: b.Waits}).referenceProblem(); p != "" {
return fmt.Errorf("delivery: the package a key waits for, %q, %s", b.Waits, p)
}
}
switch b.State {
case BuildQueued, BuildBuilding:
if len(b.Artifacts) > 0 || b.Error != "" {
return fmt.Errorf("delivery: a %s build carries no artifact and no error", b.State)
}
case BuildDone:
// A module that builds no artifact gets a done key too, and is delivered from it (issue 1613).
if b.Manifest == "" {
return fmt.Errorf("delivery: a done build names no manifest digest")
}
if b.Error != "" {
return fmt.Errorf("delivery: a done build carries an error")
}
case BuildFailed:
if b.Error == "" {
return fmt.Errorf("delivery: a failed build says no error")
}
}
return nil
}
// EncodeBuilderBuild is b as JSON at Version; it refuses a build its state does not allow (done without a manifest
// digest, failed without an error, an artifact without a name or digest, a toolchain without a version or a
// recipe version).
func EncodeBuilderBuild(b BuilderBuild) ([]byte, error) {
if err := b.check(); err != nil {
return nil, err
}
b.V = Version
return json.Marshal(b)
}
// DecodeBuilderBuild reads a builder build value; it refuses an unknown version (ErrUnknownVersion) and what
// EncodeBuilderBuild refuses.
func DecodeBuilderBuild(data []byte) (BuilderBuild, error) {
var b BuilderBuild
if err := decode(data, &b); err != nil {
return BuilderBuild{}, err
}
if err := b.check(); err != nil {
return BuilderBuild{}, err
}
return b, nil
}
// SameBuilderBuild says whether two builds are the same value, so the builder writes only when they differ.
func SameBuilderBuild(a, b BuilderBuild) bool { return reflect.DeepEqual(a, b) }
-157
View File
@@ -1,157 +0,0 @@
package delivery
import (
"strings"
"testing"
)
func artifactOf(name, kind, digest, reference string) BuiltArtifact {
return BuiltArtifact{Name: name, Kind: kind, Digest: digest, Reference: reference}
}
// A done key names where each artifact is (novox/hq ADR 0498 clauses 1, 5 and 6; the lead's decision L1 of R4e): a
// bundle or an archive by its artifact-store blob, an image or an upstream copy by the registry's reference to its
// digest (or the kept artifact-store form), a package by name and version. A reference of the wrong form for its kind,
// or naming another digest or artifact, is refused, naming the artifact.
func TestADoneKeyNamesEachArtifactsReferenceInTheFormOfItsKind(t *testing.T) {
d := "sha256:" + digestOf('6')
good := []BuiltArtifact{
artifactOf("code", "bundle", d, "artifact-store://mail/code/blobs/"+d),
artifactOf("files", "archive", d, "artifact-store://mail/files/blobs/"+d),
artifactOf("server", "image", d, "10.0.0.1:5000/mail/server@"+d),
artifactOf("server", "image", d, "artifact-store://mail/server@"+d),
artifactOf("nats", "upstream", d, "registry.example/nats/nats@"+d),
artifactOf("lib", "package", d, "@novox/mesh-sdk@0.1.15"),
artifactOf("lib", "package", d, "left-pad@1.3.0"),
artifactOf("old", "", d, "anything at all"),
// The form a registry seat answers (ADR 0605 §9), beside the older one.
artifactOf("code", "bundle", d, "oci-artifact-registry/mail/code@"+d),
artifactOf("files", "archive", d, "oci-artifact-registry/mail/files@"+d),
artifactOf("server", "image", d, "container-image-registry/mail/server@"+d),
artifactOf("nats", "upstream", d, "container-image-registry/nats/nats@"+d),
// An upstream copy under the holder's own naming, upstream/<host>/<path>, in either form (ADR 0624 §1).
artifactOf("app", "upstream", d, "container-image-registry/upstream/quay.io:443/org/app@"+d),
artifactOf("app", "upstream", d, "artifact-store://upstream/quay.io:443/org/app@"+d),
}
for _, a := range good {
b := doneBuild()
b.Artifacts = []BuiltArtifact{a}
if _, err := EncodeBuilderBuild(b); err != nil {
t.Errorf("%+v was refused: %v", a, err)
}
}
other := "sha256:" + digestOf('7')
bad := []BuiltArtifact{
artifactOf("code", "bundle", d, "artifact-store://mail/code/blobs/"+other),
artifactOf("code", "bundle", d, "artifact-store://mail/other/blobs/"+d),
artifactOf("code", "bundle", d, "10.0.0.1:5000/mail/code@"+d),
artifactOf("server", "image", d, "10.0.0.1:5000/mail@"+d),
artifactOf("server", "image", d, "10.0.0.1:5000/mail/server@"+other),
artifactOf("server", "image", d, "mail/server@"+d),
artifactOf("server", "image", d, "artifact-store://mail/server/blobs/"+d),
artifactOf("lib", "package", d, "@novox/mesh-sdk"),
artifactOf("lib", "package", d, "0.1.15"),
artifactOf("code", "bundle", d, "container-image-registry/mail/code@"+d),
artifactOf("server", "image", d, "oci-artifact-registry/mail/server@"+d),
artifactOf("server", "image", d, "container-image-registry/mail/server@"+other),
artifactOf("server", "image", d, "container-image-registry/mail/other@"+d),
artifactOf("lib", "package", d, "container-image-registry/mail/lib@"+d),
artifactOf("app", "upstream", d, "container-image-registry/upstream/quay.io:443/org/app@"+other),
artifactOf("app", "upstream", d, "artifact-store://upstream/quay.io:443/org/app@"+other),
artifactOf("app", "upstream", d, "oci-artifact-registry/upstream/quay.io:443/org/app@"+d),
artifactOf("server", "image", d, "container-image-registry/upstream/quay.io:443/org/server@"+d),
}
for _, a := range bad {
b := doneBuild()
b.Artifacts = []BuiltArtifact{a}
if _, err := EncodeBuilderBuild(b); err == nil || !strings.Contains(err.Error(), `"`+a.Name+`"`) {
t.Errorf("%+v: %v, wanted a refusal naming the artifact", a, err)
}
}
}
// The controller delivers only from a done key whose every artifact names a reference (ADR 0498 clause 8): Referenced
// says so and names the artifacts without one.
func TestReferencedNamesTheArtifactsWithoutAReference(t *testing.T) {
d := "sha256:" + digestOf('6')
b := doneBuild()
b.Artifacts = []BuiltArtifact{
artifactOf("code", "bundle", d, "artifact-store://mail/code/blobs/"+d),
artifactOf("server", "image", d, ""),
}
if ok, missing := b.Referenced(); ok || len(missing) != 1 || missing[0] != "server" {
t.Fatalf("Referenced = %v, %v; want false, [server]", ok, missing)
}
b.Artifacts[1].Reference = "10.0.0.1:5000/mail/server@" + d
if ok, missing := b.Referenced(); !ok || len(missing) != 0 {
t.Fatalf("Referenced = %v, %v; want true", ok, missing)
}
if ok, _ := (BuilderBuild{State: BuildQueued}).Referenced(); ok {
t.Fatal("a queued key is referenced")
}
}
// The errors a build module answers open with a fixed prefix (ADR 0498 clauses 2, 4 and 5; ADR 0478 clause 2), and three
// of them are not built again until the module's fingerprint changes; a failed push is.
func TestOnlyAPushFailureIsBuiltAgainBeforeTheFingerprintChanges(t *testing.T) {
for _, c := range []struct {
err string
again bool
}{
{NotStored + "code: the store said 500", true},
{NotPinned + "nats: upstream names no digest", false},
{VersionTaken + "@novox/mesh-sdk@0.1.15 holds another file", false},
{RefusedKind + "a package in go", false},
{"go test: 2 failed", true},
} {
if got := FailedUntilChanged(c.err); got == c.again {
t.Errorf("FailedUntilChanged(%q) = %v", c.err, got)
}
}
if NotStored != "not stored: " || NotPinned != "not pinned: " || VersionTaken != "version taken: " || RefusedKind != "refused kind: " {
t.Fatal("a prefix changed its words")
}
// A Node fetch that finds the lock naming a package version the registry does not hold fails with WaitsFor; the
// builder then writes the key queued with Waits, and the error is not one that waits for the fingerprint to change.
if WaitsFor != "waits for: " || FailedUntilChanged(WaitsFor+"@novox/mesh-sdk@0.1.15") {
t.Fatalf("WaitsFor = %q", WaitsFor)
}
}
// A key whose module's lock names a version of the group's package not yet published waits, queued, saying which
// (ADR 0500 clause 6): an older reader reads it as queued, and only a queued key waits.
func TestAKeyWaitsQueuedForAPackageVersion(t *testing.T) {
b := BuilderBuild{State: BuildQueued, Manifest: digestOf('4'), Waits: "@novox/mesh-sdk@0.1.15"}
data, err := EncodeBuilderBuild(b)
if err != nil {
t.Fatal(err)
}
back, err := DecodeBuilderBuild(data)
if err != nil || back.Waits != b.Waits || back.State != BuildQueued {
t.Fatalf("decoded %+v, %v", back, err)
}
building := BuilderBuild{State: BuildBuilding, By: "x", Waits: "@novox/mesh-sdk@0.1.15"}
if _, err := EncodeBuilderBuild(building); err == nil {
t.Fatal("a building key that waits was accepted")
}
if _, err := EncodeBuilderBuild(BuilderBuild{State: BuildQueued, Waits: "mesh-sdk"}); err == nil {
t.Fatal("a wait naming no version was accepted")
}
}
// A module that builds no artifact gets a done key too (issue 1613), and it is delivered: Referenced is ok for a done
// key with no artifact, since no artifact lacks a reference; a queued one with none still is not.
func TestADoneKeyWithNoArtifactIsReferenced(t *testing.T) {
b := BuilderBuild{State: BuildDone, Manifest: digestOf('4')}
data, err := EncodeBuilderBuild(b)
if err != nil {
t.Fatalf("a done key with no artifact was refused: %v", err)
}
back, err := DecodeBuilderBuild(data)
if err != nil {
t.Fatal(err)
}
if ok, missing := back.Referenced(); !ok || len(missing) != 0 {
t.Fatalf("Referenced = %v, %v; want true", ok, missing)
}
}
-216
View File
@@ -1,216 +0,0 @@
package delivery
import (
"encoding/json"
"errors"
"fmt"
"reflect"
"strings"
"testing"
)
func doneBuild() BuilderBuild {
return BuilderBuild{
State: BuildDone,
By: "builder.novox.1",
Step: "pushed",
Manifest: digestOf('4'),
Artifacts: []BuiltArtifact{
{Name: "code", Kind: "bundle", Digest: "sha256:" + digestOf('5')},
{Name: "server", Kind: "image", Digest: "sha256:" + digestOf('6'), Reference: "registry.example/mail/server@sha256:" + digestOf('6')},
},
Toolchains: map[string]ToolchainUsed{
"go": {Version: "1.26.0", Recipe: "3"},
"typescript": {Version: "5.9.2", Recipe: "1"},
},
}
}
func builderSamples() map[string]BuilderBuild {
failed := BuilderBuild{State: BuildFailed, By: "builder.novox.1", Step: "test", Error: "go test: 2 failed"}
both := doneBuild()
both.Needs = []string{"form:artifact-registry"}
both.Artifacts[1].References = []string{both.Artifacts[1].Reference, "artifact-store://mail/server@" + both.Artifacts[1].Digest}
return map[string]BuilderBuild{
"done with needs and both forms": both,
"queued": {State: BuildQueued, Manifest: digestOf('4')},
"building": {State: BuildBuilding, By: "builder.novox.1", Step: "fetch", Manifest: digestOf('4')},
"done": doneBuild(),
"failed": failed,
}
}
// 1358/AC1: the key is <module>.<hex>, parsing gives both back; a module with a dot, an empty one, and a fingerprint not
// of the new form are refused, naming it.
func TestBuilderBuildKey(t *testing.T) {
fp := fp2of('a')
key, err := BuilderBuildKey("mesh-mail", fp)
if err != nil || key != "mesh-mail."+strings.Repeat("a", 64) {
t.Fatalf("key = %q, %v", key, err)
}
module, back, err := ParseBuilderBuildKey(key)
if err != nil || module != "mesh-mail" || back != fp {
t.Errorf("parse = %q, %q, %v", module, back, err)
}
src1 := SourcePrefix + strings.Repeat("1", 64)
for module, fp := range map[string]string{"a.b": fp, "": fp, "mesh-mail": src1, "mail ": fp} {
if _, err := BuilderBuildKey(module, fp); err == nil {
t.Errorf("BuilderBuildKey(%q, %q) accepted", module, fp)
}
}
if _, err := BuilderBuildKey("a.b", fp); err == nil || !strings.Contains(err.Error(), "a.b") {
t.Errorf("a dotted module is not named: %v", err)
}
if _, err := BuilderBuildKey("mesh-mail", src1); err == nil || !strings.Contains(err.Error(), src1) {
t.Errorf("a src1 fingerprint is not named: %v", err)
}
for _, bad := range []string{"", "mesh-mail", "mesh-mail.", "mesh-mail.abc", "mesh-mail." + strings.Repeat("A", 64), ".x", "mesh-mail.build.7", "a.b." + strings.Repeat("a", 64)} {
if _, _, err := ParseBuilderBuildKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseBuilderBuildKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1358/AC2: a build value in each state encodes and decodes unchanged.
func TestBuilderBuildRoundTrip(t *testing.T) {
for name, b := range builderSamples() {
data, err := EncodeBuilderBuild(b)
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if !strings.HasPrefix(string(data), `{"v":1,`) {
t.Errorf("%s at %s", name, data)
}
got, err := DecodeBuilderBuild(data)
b.V = Version
if err != nil || !SameBuilderBuild(got, b) {
t.Errorf("%s: %+v %v", name, got, err)
}
}
}
// 1358/AC3: done without a manifest digest, failed without an error, an artifact without a digest
// and an unknown state are refused.
func TestBuilderBuildChecked(t *testing.T) {
cases := map[string]struct {
edit func(*BuilderBuild)
words []string
}{
"done without a manifest": {func(b *BuilderBuild) { b.Manifest = "" }, []string{"manifest"}},
"a malformed manifest": {func(b *BuilderBuild) { b.Manifest = "abc" }, []string{"abc"}},
"an artifact without a digest": {func(b *BuilderBuild) { b.Artifacts[1].Digest = "" }, []string{"digest", "server"}},
"an artifact without a name": {func(b *BuilderBuild) { b.Artifacts[0].Name = "" }, []string{"name"}},
"a need that is not a form": {func(b *BuilderBuild) { b.Needs = []string{"word:x"} }, []string{"word:x", "form:"}},
"a reference not among the references": {func(b *BuilderBuild) {
b.Artifacts[1].References = []string{"artifact-store://mail/server@" + b.Artifacts[1].Digest}
}, []string{"server", "references"}},
"a reference of another artifact": {func(b *BuilderBuild) {
b.Artifacts[1].References = []string{b.Artifacts[1].Reference, "registry.example/mail/other@" + b.Artifacts[1].Digest}
}, []string{"server", "other"}},
"an unknown state": {func(b *BuilderBuild) { b.State = "paused" }, []string{"paused"}},
"a toolchain without a version": {func(b *BuilderBuild) { b.Toolchains["go"] = ToolchainUsed{Recipe: "3"} }, []string{"go", "version"}},
"a toolchain without a recipe": {func(b *BuilderBuild) { b.Toolchains["go"] = ToolchainUsed{Version: "1.26.0"} }, []string{"go", "recipe"}},
}
for name, c := range cases {
b := doneBuild()
c.edit(&b)
_, err := EncodeBuilderBuild(b)
if err == nil {
t.Errorf("%s accepted", name)
continue
}
for _, w := range c.words {
if !strings.Contains(err.Error(), w) {
t.Errorf("%s: error %q does not name %q", name, err, w)
}
}
}
f := builderSamples()["failed"]
f.Error = ""
if _, err := EncodeBuilderBuild(f); err == nil || !strings.Contains(err.Error(), "error") {
t.Errorf("failed without an error: %v", err)
}
// a queued or building value may not already carry an outcome
q := builderSamples()["queued"]
q.Needs = []string{"form:x"}
if _, err := EncodeBuilderBuild(q); err == nil {
t.Error("a queued build with needs accepted")
}
q.Needs = nil
q.Artifacts = doneBuild().Artifacts
if _, err := EncodeBuilderBuild(q); err == nil {
t.Error("a queued build with artifacts accepted")
}
// decode refuses what encode refuses
if _, err := DecodeBuilderBuild([]byte(`{"v":1,"state":"done"}`)); err == nil {
t.Error("decoded a done build without a manifest digest")
}
}
// 1358/AC4: a done value names the toolchain version per language and the recipe version; two values that differ only
// there are not the same.
func TestBuilderBuildNamesItsToolchains(t *testing.T) {
a, b, c := doneBuild(), doneBuild(), doneBuild()
b.Toolchains["go"] = ToolchainUsed{Version: "1.26.1", Recipe: "3"}
c.Toolchains["go"] = ToolchainUsed{Version: "1.26.0", Recipe: "4"}
if SameBuilderBuild(a, b) || SameBuilderBuild(a, c) || SameBuilderBuild(b, c) {
t.Error("values differing only in a toolchain's version or recipe are the same")
}
if !SameBuilderBuild(a, doneBuild()) {
t.Error("equal values differ")
}
data, err := EncodeBuilderBuild(a)
if err != nil {
t.Fatal(err)
}
got, err := DecodeBuilderBuild(data)
if err != nil || !reflect.DeepEqual(got.Toolchains, a.Toolchains) {
t.Errorf("toolchains = %+v, %v", got.Toolchains, err)
}
}
// 1358/AC5: a value of another version, or none, is refused with ErrUnknownVersion.
func TestBuilderBuildVersion(t *testing.T) {
for _, in := range []string{`{"v":2,"state":"queued"}`, `{"state":"queued"}`} {
if _, err := DecodeBuilderBuild([]byte(in)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("DecodeBuilderBuild(%s) = %v", in, err)
}
}
// A holder's question before it stores (ADR 0605 §4): asked at the builder's address with name, version and digest;
// an answer without recorded is an error, never a yes.
for answer, want := range map[string]string{`{"recorded":true}`: "true ", `{"recorded":false,"why":"other digest"}`: "false other digest",
`{"name":"x"}`: "error", `not json`: "error"} {
b := BuilderVersions{Ask: func(addr string, body any) (json.RawMessage, error) {
if a := body.(map[string]any); addr != BuilderVersionsAddress || a["name"] != "n" || a["version"] != "0.1.2" || a["digest"] != "sha256:d" {
t.Errorf("asked %s %v", addr, body)
}
return json.RawMessage(answer), nil
}}
ok, why, err := b.Recorded("n", "0.1.2", "sha256:d")
if got := fmt.Sprintf("%v %s", ok, why); err != nil && want != "error" || err == nil && got != want {
t.Errorf("answer %s: %s, %v; want %s", answer, got, err, want)
}
}
}
// 1358/AC6: the builder's bucket and module do not collide with the controller's build layout (buildrecord.go), which
// this change leaves as it was.
func TestBuilderLayoutDoesNotCollideWithTheControllers(t *testing.T) {
if BuilderBuildsBucket != "mesh-delivery-builder_builds" || BuilderBuildsHistory != 5 || BuilderModule != "mesh-delivery-builder" {
t.Errorf("bucket %s history %d module %s", BuilderBuildsBucket, BuilderBuildsHistory, BuilderModule)
}
if BuildKey("m", "7") != "m.build.7" || NewestKey("m") != "m.newest" {
t.Error("the controller's layout changed")
}
// the two key forms are never read as each other
key, _ := BuilderBuildKey("m", fp2of('a'))
if key == BuildKey("m", "a") || key == NewestKey("m") {
t.Errorf("builder key %q equals a controller key", key)
}
if _, _, err := ParseBuilderBuildKey(BuildKey("m", digestOf('a'))); err == nil {
t.Error("a controller build key parsed as the builder's")
}
if _, _, err := ParseBuilderBuildKey(NewestKey("m")); err == nil {
t.Error("a controller newest key parsed as the builder's")
}
}
-38
View File
@@ -1,38 +0,0 @@
package delivery
import (
"encoding/json"
"fmt"
)
// The holder's question before it stores a publish (novox/hq ADR 0605 §4, design 51 *Publishing*): did the builder
// number this version with this digest? Every registry seat's holder asks it the same way, so the asking lives here
// once (ADR 0627 §2), beside the builder's layouts; no holder writes it again.
//
// The builder answers it today only as its module's tool builder_versions: no seat in the seat set names the builder's
// role yet. Once one does, BuilderVersionsAddress becomes that seat's verb and nothing else changes for a holder.
const BuilderVersionsAddress = BuilderModule + ".builder_versions"
// AskFunc is the runtime's request and reply over the bus (mesh-sdk stdio.Ask), given by the holder so this package does
// not depend on the runtime.
type AskFunc func(address string, body any) (json.RawMessage, error)
// BuilderVersions asks the builder whether it numbered a version with a digest. It answers the holder's Check:
// recorded, and why not when it is not.
type BuilderVersions struct{ Ask AskFunc }
// Recorded asks the builder about name@version with digest. An answer without `recorded` is an error, never a yes.
func (b BuilderVersions) Recorded(name, version, digest string) (bool, string, error) {
raw, err := b.Ask(BuilderVersionsAddress, map[string]any{"name": name, "version": version, "digest": digest})
if err != nil {
return false, "", err
}
var a struct {
Recorded *bool `json:"recorded"`
Why string `json:"why"`
}
if err := json.Unmarshal(raw, &a); err != nil || a.Recorded == nil {
return false, "", fmt.Errorf("builder_versions answered no recorded: %s", raw)
}
return *a.Recorded, a.Why, nil
}
-47
View File
@@ -1,47 +0,0 @@
package delivery
import "time"
// BuildRecord is the layout the ended controller's chain wrote (mesh-controller internal/chain/records.go Build, line 91),
// the fields a reader outside it read (novox/hq ADR 0437 §4 §5, issue 1223). The controller's record carries more; a decode ignores
// what is not here. Path is the module's directory in its repository, as the controller writes it (Build.Path, records.go:100,
// set from the build request trimmed of slashes at internal/inventory/chain_requests.go:19); empty for a module whose
// manifest is the repository's root. SourceInputs has no json keys of its own: it is written under its Go field names, as the controller
// writes it.
type BuildRecord struct {
ID string `json:"id"`
State string `json:"state,omitempty"`
Failed string `json:"failed,omitempty"`
Repository string `json:"repository"`
Ref string `json:"ref,omitempty"`
Module string `json:"module,omitempty"`
Path string `json:"path,omitempty"`
Commit string `json:"commit,omitempty"`
At time.Time `json:"at,omitzero"`
SourceFingerprint string `json:"source-fingerprint,omitempty"`
SourceInputs *SourceInputs `json:"source-inputs,omitempty"`
// Made is what the build made (Build.Made, records.go:110); the digest is the tail of an artifact's Reference.
Made []Artifact `json:"made,omitempty"`
}
// Artifact is one thing a build made: mesh-controller internal/chain/records.go Artifact, json keys name, kind and
// reference, nothing else (novox/hq ADR 0478 clause 8).
type Artifact struct {
Name string `json:"name"`
Kind string `json:"kind"`
Reference string `json:"reference"`
}
// BuildKey is a build's record under its module, a build that knew none under the request's key: copied from
// mesh-controller internal/chain/keys.go:35-40 (BuildKey, with RequestKey at line 42).
func BuildKey(module, id string) string {
if module == "" {
return "req." + id
}
return module + ".build." + id
}
// NewestKey is the key of a module's newest build, of any state: copied from mesh-controller internal/chain/keys.go:44.
// The key holds no record: its value is the newest build's id as raw bytes (internal/inventory/chain_builds.go:210 and
// :546), and the record lives at BuildKey(module, id). The pointer is a hint, checked against that record.
func NewestKey(module string) string { return module + ".newest" }
-148
View File
@@ -1,148 +0,0 @@
package delivery
import (
"encoding/json"
"os"
"testing"
"time"
)
// The fixture's path is what the controller writes for a build request: strings.Trim(path, "/") at mesh-controller
// internal/inventory/chain_requests.go:19, the module's directory in its repository (Build.Path, internal/chain/records.go:100).
// The fixture started as a record the controller's own encoder wrote (json.Marshal of a chain.Build, mesh-controller
// internal/chain, a throwaway test run once and deleted) with build-agent's build-1792395447088936520 values, plus a
// manifest-digest and one made artifact (shape: mesh-controller internal/chain/records.go:91-117 Build, json tags, at commit
// 1f708e2d8d72e2c27dbeaba02f14c128d700eada; an outcome has no state key and a build asked after a merge no ref, records.go:57-59), which a BuildRecord does not carry. It was then edited by hand
// to the controller's layout for an outcome: state and ref removed, the repository a clone URL (issue 1281).
func loadBuildFixture(t *testing.T) []byte {
t.Helper()
raw, err := os.ReadFile("testdata/build-record-build-agent.json")
if err != nil {
t.Fatal(err)
}
return raw
}
func TestBuildRecordDecodes(t *testing.T) { // 1223/AC1
var r BuildRecord
if err := json.Unmarshal(loadBuildFixture(t), &r); err != nil {
t.Fatal(err)
}
for _, c := range []struct{ name, got, want string }{
{"id", r.ID, "build-1792395447088936520"},
{"module", r.Module, "build-agent"},
{"repository", r.Repository, "http://novox.internal:20000/novox/mesh-controller.git"},
{"ref", r.Ref, ""},
{"commit", r.Commit, "7a74cee83f88e6585a5511c3cc53bdf0f5303d20"},
{"state", r.State, ""},
{"failed", r.Failed, ""},
{"source-fingerprint", r.SourceFingerprint, "src1:e846efe73716b78e80860baaac96153de2cb76551325753838ca245d2160a3d7"},
} {
if c.got != c.want {
t.Errorf("%s = %q, want %q", c.name, c.got, c.want)
}
}
if want := time.Date(2026, 10, 19, 9, 38, 0, 0, time.UTC); !r.At.Equal(want) {
t.Errorf("at = %v, want %v", r.At, want)
}
if !r.IsTrunkBuild() { // 1281/AC7
t.Error("the fixture, a record as the controller writes it, is not a trunk build")
}
if r.SourceInputs == nil {
t.Fatal("source-inputs missing")
}
}
func TestBuildRecordPath(t *testing.T) { // 1233/AC1
var r BuildRecord
if err := json.Unmarshal(loadBuildFixture(t), &r); err != nil {
t.Fatal(err)
}
if r.Path != "modules/build-agent" {
t.Errorf("path = %q, want modules/build-agent", r.Path)
}
// mesh-controller's manifest is its repository's root: the controller writes no path (omitempty), so the path is "".
var m map[string]json.RawMessage
if err := json.Unmarshal(loadBuildFixture(t), &m); err != nil {
t.Fatal(err)
}
delete(m, "path")
raw, _ := json.Marshal(m)
var root BuildRecord
if err := json.Unmarshal(raw, &root); err != nil {
t.Fatal(err)
}
if root.Path != "" {
t.Errorf("path of a root module = %q, want empty", root.Path)
}
// Every field 1223/AC1 checks is still equal in both.
if root.ID != r.ID || root.Module != r.Module || root.Repository != r.Repository || root.Ref != r.Ref ||
root.Commit != r.Commit || root.State != r.State || root.SourceFingerprint != r.SourceFingerprint || !root.At.Equal(r.At) {
t.Errorf("fields differ: %+v vs %+v", root, r)
}
// And the path survives a round trip through the SDK's own type.
back, _ := json.Marshal(r)
var again BuildRecord
if err := json.Unmarshal(back, &again); err != nil || again.Path != r.Path {
t.Errorf("round trip path = %q (%v)", again.Path, err)
}
}
func TestBuildRecordWithoutInputs(t *testing.T) { // 1223/AC2
var m map[string]json.RawMessage
if err := json.Unmarshal(loadBuildFixture(t), &m); err != nil {
t.Fatal(err)
}
delete(m, "source-inputs")
raw, _ := json.Marshal(m)
var r BuildRecord
if err := json.Unmarshal(raw, &r); err != nil {
t.Fatal(err)
}
if r.SourceInputs != nil {
t.Errorf("source-inputs = %+v, want nil", r.SourceInputs)
}
if r.Module != "build-agent" {
t.Errorf("module = %q, want build-agent", r.Module)
}
}
func TestBuildKeys(t *testing.T) { // 1223/AC3
// Literal values of mesh-controller internal/broker/controller_buckets.go:101 (BucketName(ControllerSeat, ChainBuilds)
// is "mesh-controller_builds") and internal/chain/keys.go:35-44.
if got := BuildKey("build-agent", "build-1"); got != "build-agent.build.build-1" {
t.Errorf("BuildKey = %q", got)
}
if got := BuildKey("", "build-1"); got != "req.build-1" {
t.Errorf("BuildKey without a module = %q", got)
}
if got := NewestKey("build-agent"); got != "build-agent.newest" {
t.Errorf("NewestKey = %q", got)
}
}
// The fixture's made artifacts are the controller's Artifact (mesh-controller internal/chain/records.go, json tags name,
// kind, reference; the controller writes no digest key, the digest is the tail of the reference). The kinds and references
// are those of the live record mesh-controller.builds printed for build-agent build-1792540091397811913 (2026-10-21 01:49).
func TestBuildRecordMade(t *testing.T) { // 1588/AC2
var r BuildRecord
if err := json.Unmarshal(loadBuildFixture(t), &r); err != nil {
t.Fatal(err)
}
want := []Artifact{
{Name: "server", Kind: "image", Reference: "artifact-store://build-agent/server@sha256:a79c13f1c727ff94959e80824cc704ea58b80d80ecdd00fd447698acebb35090"},
{Name: "files", Kind: "archive", Reference: "artifact-store://build-agent/files/blobs/sha256:bccd3f47851436d1b45a3063cc2d7e34c5c109b2d030739bfbb4b9781703f9a9"},
}
if len(r.Made) != len(want) {
t.Fatalf("made = %+v, want %+v", r.Made, want)
}
for i := range want {
if r.Made[i] != want[i] {
t.Errorf("made[%d] = %+v, want %+v", i, r.Made[i], want[i])
}
}
var none BuildRecord
if err := json.Unmarshal([]byte(`{"id":"b","repository":"a/b"}`), &none); err != nil || none.Made != nil {
t.Errorf("a record without made: %+v, %v", none, err)
}
}
-40
View File
@@ -1,40 +0,0 @@
package delivery
import (
"strings"
"testing"
)
// A candidate's module carries the git objects of its named paths at the preview (novox/hq ADR 0499 clause 2), so the
// builder's fetch request for a candidate names them; an older candidate decodes without them, and a path that is not
// clean and relative, or names no object, is refused.
func TestACandidatesModuleCarriesItsNamedPaths(t *testing.T) {
c := sampleCandidate()
paths := map[string]string{"src": strings.Repeat("a", 40), "package.json": strings.Repeat("b", 40)}
c.Pulls[0].Fps = map[string]CandidateModule{"mesh-mail": {FP2: SourcePrefix2 + strings.Repeat("1", 64),
Manifest: strings.Repeat("2", 64), Paths: paths}}
c.FPSet = FingerprintSet(c.Pulls)
data, err := EncodeCandidate(c)
if err != nil {
t.Fatal(err)
}
back, err := DecodeCandidate(data)
if err != nil || back.Pulls[0].Fps["mesh-mail"].Paths["src"] != paths["src"] {
t.Fatalf("decoded %+v, %v", back.Pulls[0].Fps, err)
}
old := sampleCandidate()
old.Pulls[0].Fps = map[string]CandidateModule{"mesh-mail": {FP2: SourcePrefix2 + strings.Repeat("1", 64)}}
old.FPSet = FingerprintSet(old.Pulls)
if data, err := EncodeCandidate(old); err != nil {
t.Fatal(err)
} else if b, err := DecodeCandidate(data); err != nil || b.Pulls[0].Fps["mesh-mail"].Paths != nil {
t.Fatalf("an older candidate: %+v, %v", b, err)
}
for _, bad := range []map[string]string{{"../x": "a"}, {"/src": "a"}, {"": "a"}, {"src": ""}} {
c.Pulls[0].Fps = map[string]CandidateModule{"mesh-mail": {FP2: SourcePrefix2 + strings.Repeat("1", 64), Paths: bad}}
c.FPSet = FingerprintSet(c.Pulls)
if _, err := EncodeCandidate(c); err == nil {
t.Errorf("%v was accepted", bad)
}
}
}
-447
View File
@@ -1,447 +0,0 @@
package delivery
import (
"encoding/json"
"fmt"
"reflect"
"strings"
"time"
)
// The checker's keys (novox/hq ADR 0447 clauses 1 to 3, ADR 0444, ADR 0446 clause 5, ADR 0453 clause 4): the verdicts of
// its checks and the passing verdicts it keeps per package and level. The checker writes them from R4c, dark, and
// writes the status `mesh/check` it decides as an owed act in its own bucket (CheckerOwedBucket, the gate's owed shape);
// the forge bridge, the only part that calls the forge, performs that act (unrequired until R4f), once per revision of
// the key (ADR 0484 clause 2). The gate reads the status from R4f, and the view and the watchman read the verdicts (a
// running check carries a pulse).
//
// **The verdict's key is the group and the heads, not the fingerprints** (the lead's limit for R4a). Research 063
// section 01 drew `verdicts.<g>.check.<fpset>`; ADR 0447 clause 1 judges a pull request's head merged onto its
// trunk's head, and a repository without modules has no fingerprint set, so the key is `<group>.<heads hash>` as the
// heads hash of HeadsHash gives it. The fpset (FingerprintSet) is not part of it.
const (
// CheckerModule is the module of the checker part.
CheckerModule = "mesh-delivery-checker"
// CheckerVerdictsBucket is the checker's bucket of verdicts, `<owner module>_<name>`.
CheckerVerdictsBucket = CheckerModule + "_verdicts"
// CheckerVerdictsHistory is the revisions the verdicts bucket keeps per key.
CheckerVerdictsHistory = 5
// CheckerKeptBucket is the checker's bucket of kept verdicts. A key is written once, by a create that fails when
// the key exists, by the checker's credential alone, from the result the checker reads from the test run and
// never from anything the code under test writes (ADR 0444 clause 5); nothing changes it afterwards.
CheckerKeptBucket = CheckerModule + "_kept"
// CheckerKeptHistory is the revisions the kept bucket keeps per key.
CheckerKeptHistory = 1
)
// The test levels (ADR 0445 clause 2) and the states of a verdict.
const (
LevelUnit = "unit"
LevelIntegration = "integration"
VerdictRunning = "running"
VerdictPass = "pass"
VerdictFail = "fail"
)
func checkLevel(level string) error {
if level != LevelUnit && level != LevelIntegration {
return fmt.Errorf("delivery: the level %q is not %s or %s", level, LevelUnit, LevelIntegration)
}
return nil
}
func isHex12(s string) bool { return len(s) == 12 && strings.Trim(s, "0123456789abcdef") == "" }
// CheckerVerdictKey is the key, in CheckerVerdictsBucket, of a group's check at its heads: `<escaped group>.<heads
// hash>`, the group escaped as CandidatesKey does and the heads hash as HeadsHash gives it (12 lowercase hex digits).
func CheckerVerdictKey(group, headsHash string) (string, error) {
g, err := groupKey(group)
if err != nil {
return "", err
}
if !isHex12(headsHash) {
return "", fmt.Errorf("delivery: %q is not a heads hash, 12 lowercase hex digits", headsHash)
}
return g + "." + headsHash, nil
}
// ParseCheckerVerdictKey is the inverse of CheckerVerdictKey.
func ParseCheckerVerdictKey(key string) (group, headsHash string, err error) {
i := strings.LastIndex(key, ".")
if i < 0 {
return "", "", fmt.Errorf("delivery: %q is not a verdict key, <group>.<heads hash>", key)
}
part, headsHash := key[:i], key[i+1:]
if !isHex12(headsHash) {
return "", "", fmt.Errorf("delivery: %q is not a verdict key: %q is not a heads hash, 12 lowercase hex digits", key, headsHash)
}
if group, err = parseGroupKey(part); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a verdict key: %w", key, err)
}
if strings.Contains(part, ".") {
return "", "", fmt.Errorf("delivery: %q is not a verdict key: the group part holds a dot", key)
}
return group, headsHash, nil
}
// LevelResult is a level's tests: how many ran in this check and how many were taken from kept verdicts (ADR 0444
// clause 2).
type LevelResult struct {
Run int `json:"run"`
Kept int `json:"kept"`
// Skipped is the tests the run skipped, so a pass never hides them (issue 1251, ADR 0484 clause 7).
Skipped int `json:"skipped,omitempty"`
}
// The states of a Composition (ADR 0484 clause 6): the composition check is the assigner's, so until it delivers a
// verdict says CompositionNotRun, and a line never says the composition passed on that.
//
// A composition not run may say why in its Words: AssignerNotRunning when no instance of the assigner is online, so a
// check never waits for a composition nobody is making (issue 1297).
const (
CompositionNotRun = "not-run"
CompositionPass = "pass"
CompositionFail = "fail"
// AssignerNotRunning is the Words of a composition not run because no instance of the assigner is online.
AssignerNotRunning = "the assigner is not running"
)
// Composition is what the check of the machines' composition made of a verdict: its State (not-run, pass or fail), the
// Words of a refusal, the mesh-sdk commit it composed with and the machines composed (ADR 0482 clause 9, ADR 0484 clause 6).
type Composition struct {
State string `json:"state"`
Words string `json:"words,omitempty"`
SDK string `json:"sdk,omitempty"`
Machines []string `json:"machines,omitempty"`
// FPSet is the candidate's fingerprint set (Candidate.FPSet) the composition judged: a carry keeps a composition that
// passed while the candidate's set is the same, and composes again when a fingerprint changed (novox/hq ADR 0548 §5).
// Absent in a value written before it, which a carry composes again.
FPSet string `json:"fpset,omitempty"`
}
// Took is the time of each part of a check in integer milliseconds (ADR 0447 clause 3): waiting in the queue, cloning,
// the unit level, the integration level's setup and its running, and the composition.
type Took struct {
Queue int64 `json:"queue"`
Clone int64 `json:"clone"`
Unit int64 `json:"unit"`
// Lint is the unit level's gofmt and go vet before its tests (ADR 0529 §1; issue 1887 D7-go); absent from a verdict
// of a build module that runs none.
Lint int64 `json:"lint,omitempty"`
IntegrationSetup int64 `json:"integration-setup"`
IntegrationRun int64 `json:"integration-run"`
Composition int64 `json:"composition"`
}
// FailedFirst names what failed first: a test (its package and name) or a machine (and the words of the composition's
// refusal); one of them, with Words optional.
type FailedFirst struct {
Test string `json:"test,omitempty"`
Machine string `json:"machine,omitempty"`
Words string `json:"words,omitempty"`
}
// MaxFailures is the most entries a verdict's Failures holds (issue 1921): fifty failures named one by one, and an entry
// after them saying how many more failed.
const MaxFailures = 50
// Verdict is the value of a verdict key: the group and its heads hash, the trunk head of each repository it was judged
// against (repository to full sha), the State (running, pass or fail), what failed first, the tests run and kept per
// level, each part's time, the machines composed and the mesh-sdk version the composition ran with (ADR 0446 clause 5).
type Verdict struct {
V int `json:"v"`
Group string `json:"group"`
Heads string `json:"heads"`
Trunk map[string]string `json:"trunk,omitempty"`
State string `json:"state"`
Failed FailedFirst `json:"failed,omitzero"`
// Failures names everything that failed, in the order judged, each with the first line of its words (issue 1921): every
// failed test, and every member's refused step, conflict, manifest refusal and the composition's failure, since a check
// runs every member, level and the composition to its end (the operator, 2026-11-02). Failed keeps the first one's
// words whole. At most MaxFailures entries, the last of a longer list saying how many more failed. Absent in a value
// written before it, and from a verdict with one failure that is no test.
Failures []FailedFirst `json:"failures,omitempty"`
Levels map[string]LevelResult `json:"levels,omitempty"`
Took Took `json:"took,omitzero"`
Machines []string `json:"machines,omitempty"`
SDK string `json:"sdk,omitempty"`
// Composition is the composition check's result; absent in a value written before it (ADR 0484 clause 6).
Composition *Composition `json:"composition,omitempty"`
// OverBound names the levels whose running went over their bound; it does not change the State (ADR 0453 clause 9).
OverBound []string `json:"over-bound,omitempty"`
// Beside is the commit of each sibling repository the check read beside the repositories it merged (ADR 0484 §4), by
// repository: a move of one puts the verdict out of date as a trunk move does, and it is carried (novox/hq ADR 0548 §6).
// A repository is in Trunk or here, never both. Absent in a value written before it.
Beside map[string]string `json:"beside,omitempty"`
// CarriedFrom names, of a carried verdict, the trunk and sibling commits it was judged on before the moves it was
// carried over, by repository (ADR 0548 §5): the verdict was rewritten for Trunk and Beside by running only the packages
// whose digest the moves changed. Absent from a verdict judged whole, and from one still running.
CarriedFrom map[string]string `json:"carried-from,omitempty"`
// Series names each change of an artifact's series the checked heads make against the trunk's manifests (design 32;
// the checker's DLV-C4), each once, in the order the check found them; the pull request's mesh/check line says the
// same, and a reader that cuts that line still has every one here. Absent when the heads change no series, and in a
// value written before it.
Series []SeriesChange `json:"series,omitempty"`
}
// SeriesChange is one change of a series: the module, its artifact (or the package's ecosystem name, as the module's
// packages name it), the series on trunk and the series the heads give it.
type SeriesChange struct {
Module string `json:"module"`
Artifact string `json:"artifact"`
From string `json:"from"`
To string `json:"to"`
}
func (c *Composition) check() error {
if c == nil {
return nil
}
switch c.State {
case CompositionNotRun:
if c.SDK != "" || len(c.Machines) > 0 {
return fmt.Errorf("delivery: the verdict's composition is not run and names no mesh-sdk commit or machines")
}
case CompositionPass:
if c.SDK == "" {
return fmt.Errorf("delivery: the verdict's composition passed and names no mesh-sdk commit")
}
case CompositionFail:
if c.Words == "" {
return fmt.Errorf("delivery: the verdict's composition failed and gives no words")
}
default:
return fmt.Errorf("delivery: the verdict's composition state %q is not %s, %s or %s", c.State, CompositionNotRun, CompositionPass, CompositionFail)
}
return nil
}
func (v Verdict) check() error {
if v.Group == "" {
return fmt.Errorf("delivery: a verdict names no group")
}
if !isHex12(v.Heads) {
return fmt.Errorf("delivery: the verdict's heads hash %q is not 12 lowercase hex digits", v.Heads)
}
for repository, head := range v.Trunk {
if head == "" {
return fmt.Errorf("delivery: the verdict's trunk head of %s is empty", repository)
}
}
for repository, commit := range v.Beside {
if commit == "" {
return fmt.Errorf("delivery: the verdict's sibling commit of %s is empty", repository)
}
if _, ok := v.Trunk[repository]; ok {
return fmt.Errorf("delivery: the verdict names %s as a trunk it merged and as a sibling it read", repository)
}
}
if len(v.CarriedFrom) > 0 {
if v.State == VerdictRunning {
return fmt.Errorf("delivery: a running verdict is carried from nothing: carried-from is written with its end")
}
moved := false
for repository, commit := range v.CarriedFrom {
now, inTrunk := v.Trunk[repository]
if !inTrunk {
now = v.Beside[repository]
}
if commit == "" || now == "" {
return fmt.Errorf("delivery: the verdict's carried-from names %s, which it judged on no commit of", repository)
}
moved = moved || commit != now
}
if !moved {
return fmt.Errorf("delivery: the verdict's carried-from names the commits it is judged on: nothing moved, so nothing was carried")
}
}
changed := map[string]bool{}
for _, c := range v.Series {
if c.Module == "" || c.Artifact == "" || c.From == "" || c.To == "" {
return fmt.Errorf("delivery: the verdict's series change %+v names no module, artifact, from or to", c)
}
if c.From == c.To {
return fmt.Errorf("delivery: the verdict's series change of %s/%s is from %s to the same series", c.Module, c.Artifact, c.From)
}
if changed[c.Module+"/"+c.Artifact] {
return fmt.Errorf("delivery: the verdict names the series change of %s/%s twice", c.Module, c.Artifact)
}
changed[c.Module+"/"+c.Artifact] = true
}
for level, r := range v.Levels {
if err := checkLevel(level); err != nil {
return err
}
if r.Run < 0 || r.Kept < 0 || r.Skipped < 0 {
return fmt.Errorf("delivery: the verdict's %s level counts a negative number of tests", level)
}
}
if err := v.Composition.check(); err != nil {
return err
}
seen := map[string]bool{}
for _, level := range v.OverBound {
if err := checkLevel(level); err != nil {
return fmt.Errorf("delivery: the verdict's over-bound names: %w", err)
}
if seen[level] {
return fmt.Errorf("delivery: the verdict's over-bound names the level %s twice", level)
}
seen[level] = true
}
for part, ms := range map[string]int64{"queue": v.Took.Queue, "clone": v.Took.Clone, "unit": v.Took.Unit,
"integration-setup": v.Took.IntegrationSetup, "integration-run": v.Took.IntegrationRun, "composition": v.Took.Composition} {
if ms < 0 {
return fmt.Errorf("delivery: the verdict's time of the part %s is negative", part)
}
}
named := v.Failed.Test != "" || v.Failed.Machine != ""
if len(v.Failures) > 0 && v.State != VerdictFail {
return fmt.Errorf("delivery: a %s verdict names no failures", v.State)
}
if len(v.Failures) > MaxFailures+1 {
return fmt.Errorf("delivery: the verdict names %d failures, more than %d and the count of the rest", len(v.Failures), MaxFailures)
}
switch v.State {
case VerdictRunning:
if v.Failed != (FailedFirst{}) {
return fmt.Errorf("delivery: a running verdict names no failure")
}
case VerdictPass:
if v.Failed != (FailedFirst{}) {
return fmt.Errorf("delivery: a pass names no failure")
}
for _, level := range []string{LevelUnit, LevelIntegration} {
if _, ok := v.Levels[level]; !ok {
return fmt.Errorf("delivery: a pass says nothing of the %s level", level)
}
}
case VerdictFail:
if !named {
return fmt.Errorf("delivery: a fail names neither a test nor a machine")
}
default:
return fmt.Errorf("delivery: the verdict state %q is not running, pass or fail", v.State)
}
return nil
}
// EncodeVerdict is v as JSON at Version; it refuses a fail that names neither a test nor a machine, a pass without both
// levels, an unknown state or level, and a negative count or time.
func EncodeVerdict(v Verdict) ([]byte, error) {
if err := v.check(); err != nil {
return nil, err
}
v.V = Version
return json.Marshal(v)
}
// DecodeVerdict reads a verdict; it refuses an unknown version (ErrUnknownVersion) and what EncodeVerdict refuses.
func DecodeVerdict(data []byte) (Verdict, error) {
var v Verdict
if err := decode(data, &v); err != nil {
return Verdict{}, err
}
if err := v.check(); err != nil {
return Verdict{}, err
}
return v, nil
}
// SameVerdict says whether two verdicts are the same value, so the checker writes only when they differ.
func SameVerdict(a, b Verdict) bool { return reflect.DeepEqual(a, b) }
// KeptKey is the key, in CheckerKeptBucket, of a kept verdict: `<level>.<digest>`, the level unit or integration (ADR
// 0445 clause 2) and the digest, lowercase hex, of everything that package's tests read at that level as the
// language's build module names it (ADR 0444 clause 1, ADR 0445 clause 5).
func KeptKey(level, digest string) (string, error) {
if err := checkLevel(level); err != nil {
return "", err
}
if digest == "" || strings.Trim(digest, "0123456789abcdef") != "" {
return "", fmt.Errorf("delivery: %q is not a digest, lowercase hex", digest)
}
return level + "." + digest, nil
}
// ParseKeptKey is the inverse of KeptKey.
func ParseKeptKey(key string) (level, digest string, err error) {
level, digest, ok := strings.Cut(key, ".")
if !ok {
return "", "", fmt.Errorf("delivery: %q is not a kept key, <level>.<digest>", key)
}
if _, err = KeptKey(level, digest); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a kept key: %w", key, err)
}
return level, digest, nil
}
// Kept is the value of a kept key: a passing verdict of one package at one level (State is pass and nothing else: a
// failure is never kept, ADR 0444 clause 1), with the repository, the package, the number of tests, the time they took
// in milliseconds, the toolchain's version and when it was kept.
type Kept struct {
V int `json:"v"`
State string `json:"state"`
Repository string `json:"repository"`
Package string `json:"package"`
Level string `json:"level"`
Tests int `json:"tests"`
Took int64 `json:"took"`
Toolchain string `json:"toolchain"`
At time.Time `json:"at"`
}
func (k Kept) check() error {
if k.State != VerdictPass {
return fmt.Errorf("delivery: a kept verdict is a pass, never %q: a failure is never kept", k.State)
}
if err := checkRepository(k.Repository); err != nil {
return err
}
if k.Package == "" {
return fmt.Errorf("delivery: a kept verdict names no package")
}
if err := checkLevel(k.Level); err != nil {
return err
}
if k.Tests < 0 || k.Took < 0 {
return fmt.Errorf("delivery: a kept verdict counts a negative number of tests or a negative time")
}
if k.Toolchain == "" {
return fmt.Errorf("delivery: a kept verdict names no toolchain version")
}
if k.At.IsZero() {
return fmt.Errorf("delivery: a kept verdict says no time")
}
return nil
}
// EncodeKept is k as JSON at Version; it refuses a kept value that is not a pass.
func EncodeKept(k Kept) ([]byte, error) {
if err := k.check(); err != nil {
return nil, err
}
k.V = Version
return json.Marshal(k)
}
// DecodeKept reads a kept value; it refuses an unknown version (ErrUnknownVersion) and what EncodeKept refuses.
func DecodeKept(data []byte) (Kept, error) {
var k Kept
if err := decode(data, &k); err != nil {
return Kept{}, err
}
if err := k.check(); err != nil {
return Kept{}, err
}
return k, nil
}
// SameKept says whether two kept values are the same value.
func SameKept(a, b Kept) bool {
at := a.At.Equal(b.At)
a.At, b.At = time.Time{}, time.Time{}
return at && a == b
}
-674
View File
@@ -1,674 +0,0 @@
package delivery
import (
"encoding/json"
"errors"
"strconv"
"strings"
"testing"
"time"
)
func passVerdict() Verdict {
return Verdict{
Group: "feat/x",
Heads: "4b5491557853",
Trunk: map[string]string{"novox/mesh-sdk": "aaaa", "novox/mesh-control": "bbbb"},
State: VerdictPass,
Levels: map[string]LevelResult{LevelUnit: {Run: 12, Kept: 301}, LevelIntegration: {Run: 3, Kept: 40}},
Took: Took{Queue: 1200, Clone: 800, Unit: 950, IntegrationSetup: 2100, IntegrationRun: 7400, Composition: 3300},
Machines: []string{"ace", "g14", "novox", "shanks"},
SDK: "v0.1.14-0.20261019091011-aa68b67389c9",
}
}
func verdictSamples() map[string]Verdict {
running := Verdict{Group: "feat/x", Heads: "4b5491557853", Trunk: map[string]string{"novox/mesh-sdk": "aaaa"}, State: VerdictRunning}
failTest := passVerdict()
failTest.State, failTest.Failed = VerdictFail, FailedFirst{Test: "novox/mesh-sdk/go/delivery TestHeadsHash"}
failTest.Levels = map[string]LevelResult{LevelUnit: {Run: 1}}
failMachine := passVerdict()
failMachine.State, failMachine.Failed = VerdictFail, FailedFirst{Machine: "ace", Words: "path /etc/x is owned by two modules"}
carried := passVerdict()
carried.Beside = map[string]string{"novox/mesh-catalog": "cccc"}
carried.CarriedFrom = map[string]string{"novox/mesh-sdk": "9999", "novox/mesh-control": "bbbb", "novox/mesh-catalog": "cccc"}
carried.Composition = &Composition{State: CompositionPass, SDK: carried.SDK, Machines: []string{"ace"}, FPSet: "fps1:00"}
series := passVerdict()
series.Series = []SeriesChange{{Module: "mesh-sdk", Artifact: "go", From: "0.1", To: "0.2"},
{Module: "mesh-contracts", Artifact: "@novox/mesh-contracts", From: "0.1", To: "1.0"}}
return map[string]Verdict{"running": running, "pass": passVerdict(), "fail by a test": failTest, "fail by a machine": failMachine,
"carried": carried, "naming its series changes": series}
}
func keptSample() Kept {
return Kept{State: VerdictPass, Repository: "novox/mesh-sdk", Package: "git.novox.be/novox/mesh-sdk/go/delivery", Level: LevelUnit,
Tests: 41, Took: 190, Toolchain: "1.26.0", At: time.Date(2026, 10, 20, 2, 0, 0, 0, time.UTC)}
}
// 1359/AC1: the key is <escaped group>.<heads hash>; parsing gives back the group and the hash.
func TestCheckerVerdictKey(t *testing.T) {
heads := []Head{{Repository: "novox/mesh-sdk", Number: 36, Head: "aaaa"}}
hh, _ := HeadsHash(heads)
for _, group := range []string{"feat/x", "fix/1.2", "plain", "odd name=1"} {
key, err := CheckerVerdictKey(group, hh)
if err != nil {
t.Fatal(err)
}
if want := EscapeName(group) + "." + hh; key != want {
t.Errorf("key = %q, want %q", key, want)
}
g, h, err := ParseCheckerVerdictKey(key)
if err != nil || g != group || h != hh {
t.Errorf("parse(%q) = %q, %q, %v", key, g, h, err)
}
}
if k, _ := CheckerVerdictKey("feat/x", hh); !strings.HasPrefix(k, "feat/x.") {
t.Errorf("a group with a slash is not kept readable: %q", k)
}
if _, err := CheckerVerdictKey("", hh); err == nil {
t.Error("an empty group accepted")
}
for _, bad := range []string{"", "feat/x", "feat/x.", "feat/x.abc", "feat/x." + strings.ToUpper(hh), ".4b5491557853", "a.b.4b5491557853", "feat/x.4b5491557853a"} {
if _, _, err := ParseCheckerVerdictKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseCheckerVerdictKey(%q) = %v, want a refusal naming it", bad, err)
}
}
if _, err := CheckerVerdictKey("feat/x", "abc"); err == nil || !strings.Contains(err.Error(), "abc") {
t.Errorf("a short heads hash: %v", err)
}
}
// 1359/AC2: a verdict in each state round-trips, with times per part in milliseconds, the tests run and kept per level,
// the machines composed and the mesh-sdk version composed with.
func TestVerdictRoundTrip(t *testing.T) {
for name, v := range verdictSamples() {
data, err := EncodeVerdict(v)
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if !strings.HasPrefix(string(data), `{"v":1,`) {
t.Errorf("%s at %s", name, data)
}
got, err := DecodeVerdict(data)
v.V = Version
if err != nil || !SameVerdict(got, v) {
t.Errorf("%s: %+v %v", name, got, err)
}
}
data, _ := EncodeVerdict(passVerdict())
data, _ = EncodeVerdict(verdictSamples()["naming its series changes"])
if !strings.Contains(string(data), `"series":[{"module":"mesh-sdk","artifact":"go","from":"0.1","to":"0.2"},`) {
t.Errorf("the series changes are not each {module, artifact, from, to}: %s", data)
}
data, _ = EncodeVerdict(passVerdict())
for _, field := range []string{`"queue":1200`, `"clone":800`, `"unit":950`, `"integration-setup":2100`, `"integration-run":7400`, `"composition":3300`, `"run":12`, `"kept":301`, `"machines":["ace","g14","novox","shanks"]`, `"sdk":"v0.1.14`} {
if !strings.Contains(string(data), field) {
t.Errorf("the value lacks %s: %s", field, data)
}
}
}
// 1359/AC3: a fail that names neither a test nor a machine, and a pass whose levels are missing, are refused.
func TestVerdictChecked(t *testing.T) {
cases := map[string]struct {
edit func(*Verdict)
words []string
}{
"fail naming nothing": {func(v *Verdict) { v.State, v.Failed = VerdictFail, FailedFirst{} }, []string{"test", "machine"}},
"fail with words only": {func(v *Verdict) { v.State, v.Failed = VerdictFail, FailedFirst{Words: "x"} }, []string{"test", "machine"}},
"pass without levels": {func(v *Verdict) { v.Levels = nil }, []string{"level"}},
"pass without unit": {func(v *Verdict) { delete(v.Levels, LevelUnit) }, []string{"unit"}},
"pass without integration": {func(v *Verdict) { delete(v.Levels, LevelIntegration) }, []string{"integration"}},
"pass naming a failure": {func(v *Verdict) { v.Failed = FailedFirst{Test: "T"} }, []string{"pass"}},
"an unknown state": {func(v *Verdict) { v.State = "maybe" }, []string{"maybe"}},
"an unknown level": {func(v *Verdict) { v.Levels["e2e"] = LevelResult{} }, []string{"e2e"}},
"a negative time": {func(v *Verdict) { v.Took.Clone = -1 }, []string{"clone"}},
"a bad heads hash": {func(v *Verdict) { v.Heads = "xyz" }, []string{"xyz"}},
"no group": {func(v *Verdict) { v.Group = "" }, []string{"group"}},
"a trunk head left empty": {func(v *Verdict) { v.Trunk["novox/mesh-sdk"] = "" }, []string{"novox/mesh-sdk"}},
"a negative count": {func(v *Verdict) { v.Levels[LevelUnit] = LevelResult{Run: -1} }, []string{"unit"}},
}
for name, c := range cases {
v := passVerdict()
c.edit(&v)
_, err := EncodeVerdict(v)
if err == nil {
t.Errorf("%s accepted", name)
continue
}
for _, w := range c.words {
if !strings.Contains(err.Error(), w) {
t.Errorf("%s: error %q does not name %q", name, err, w)
}
}
}
if _, err := DecodeVerdict([]byte(`{"v":1,"group":"g","heads":"4b5491557853","state":"fail"}`)); err == nil {
t.Error("decoded a fail naming nothing")
}
}
// 1359/AC4: the kept key is <level>.<hex digest>; a level other than unit or integration, or a digest that is not hex,
// is refused, naming it.
func TestKeptKey(t *testing.T) {
digest := strings.Repeat("ab12", 16)
for _, level := range []string{LevelUnit, LevelIntegration} {
key, err := KeptKey(level, digest)
if err != nil || key != level+"."+digest {
t.Fatalf("key = %q, %v", key, err)
}
l, d, err := ParseKeptKey(key)
if err != nil || l != level || d != digest {
t.Errorf("parse = %q, %q, %v", l, d, err)
}
}
if _, err := KeptKey("e2e", digest); err == nil || !strings.Contains(err.Error(), "e2e") {
t.Errorf("a level e2e: %v", err)
}
if _, err := KeptKey("", digest); err == nil {
t.Error("an empty level accepted")
}
for _, d := range []string{"", "xyz", "ABCD", "ab 12", "sha256:abcd", "ab12.cd"} {
if _, err := KeptKey(LevelUnit, d); err == nil || !strings.Contains(err.Error(), "\""+d+"\"") {
t.Errorf("KeptKey(unit, %q) = %v, want a refusal naming it", d, err)
}
}
for _, bad := range []string{"", "unit", "unit.", "e2e." + digest, "unit.xyz", "unit." + digest + ".x", "." + digest} {
if _, _, err := ParseKeptKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseKeptKey(%q) = %v, want a refusal naming it", bad, err)
}
}
if CheckerVerdictsBucket != "mesh-delivery-checker_verdicts" || CheckerVerdictsHistory != 5 ||
CheckerKeptBucket != "mesh-delivery-checker_kept" || CheckerKeptHistory != 1 || CheckerModule != "mesh-delivery-checker" {
t.Errorf("buckets %s %d %s %d", CheckerVerdictsBucket, CheckerVerdictsHistory, CheckerKeptBucket, CheckerKeptHistory)
}
}
// 1359/AC5: a kept value that is not a pass cannot be encoded; a failure is never kept (ADR 0444 clause 1).
func TestAFailureIsNeverKept(t *testing.T) {
data, err := EncodeKept(keptSample())
if err != nil {
t.Fatal(err)
}
got, err := DecodeKept(data)
want := keptSample()
want.V = Version
if err != nil || !SameKept(got, want) {
t.Errorf("round trip: %+v %v", got, err)
}
for _, state := range []string{VerdictFail, VerdictRunning, "", "skipped"} {
k := keptSample()
k.State = state
if _, err := EncodeKept(k); err == nil || !strings.Contains(err.Error(), "pass") {
t.Errorf("a kept value in state %q: %v", state, err)
}
}
if _, err := DecodeKept([]byte(`{"v":1,"state":"fail","repository":"novox/mesh-sdk","package":"p","level":"unit","tests":1,"took":1,"toolchain":"1.26.0","at":"2026-10-20T02:00:00Z"}`)); err == nil {
t.Error("decoded a kept failure")
}
for name, edit := range map[string]func(*Kept){
"a level e2e": func(k *Kept) { k.Level = "e2e" },
"no package": func(k *Kept) { k.Package = "" },
"a bad repository": func(k *Kept) { k.Repository = "nope" },
"no toolchain": func(k *Kept) { k.Toolchain = "" },
"no time": func(k *Kept) { k.At = time.Time{} },
"negative tests": func(k *Kept) { k.Tests = -1 },
"a negative took": func(k *Kept) { k.Took = -1 },
} {
k := keptSample()
edit(&k)
if _, err := EncodeKept(k); err == nil {
t.Errorf("%s accepted", name)
}
}
}
// 1359/AC6: a value of another version, or none, is refused with ErrUnknownVersion.
func TestCheckerVersions(t *testing.T) {
for name, dec := range map[string]func([]byte) error{
"verdict": func(b []byte) error { _, err := DecodeVerdict(b); return err },
"kept": func(b []byte) error { _, err := DecodeKept(b); return err },
} {
for _, in := range []string{`{"v":2,"state":"pass"}`, `{"state":"pass"}`} {
if err := dec([]byte(in)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("%s decode of %s = %v", name, in, err)
}
}
}
}
// oldVerdictJSON is a verdict as the SDK before issue 1672 wrote it (copied from EncodeVerdict of passVerdict's shape
// at the commit before this change): no skipped count, no composition, no over-bound names.
const oldVerdictJSON = `{"v":1,"group":"feat/x","heads":"4b5491557853","trunk":{"novox/mesh-control":"bbbb","novox/mesh-sdk":"aaaa"},"state":"pass","levels":{"integration":{"run":3,"kept":40},"unit":{"run":12,"kept":301}},"took":{"queue":1200,"clone":800,"unit":950,"integration-setup":2100,"integration-run":7400,"composition":3300},"machines":["ace","g14","novox","shanks"],"sdk":"v0.1.14-0.20261019091011-aa68b67389c9"}`
// oldVerdict is the reader of today's shape: the struct as it stood before the new fields.
type oldVerdict struct {
V int `json:"v"`
Group string `json:"group"`
Heads string `json:"heads"`
State string `json:"state"`
Levels map[string]struct{ Run, Kept int }
Machines []string
SDK string
}
// 1672/AC1: the skipped count, the composition and the over-bound names round trip and appear in the bytes.
func TestVerdictCarriesSkippedCompositionAndOverBound(t *testing.T) {
v := passVerdict()
v.Levels[LevelUnit] = LevelResult{Run: 12, Kept: 301, Skipped: 4}
v.Composition = &Composition{State: CompositionPass, SDK: v.SDK, Machines: []string{"ace", "novox"}}
v.OverBound = []string{LevelIntegration}
data, err := EncodeVerdict(v)
if err != nil {
t.Fatal(err)
}
v.Beside, v.CarriedFrom = map[string]string{"novox/mesh-catalog": "cccc"}, map[string]string{"novox/mesh-catalog": "cbcb"}
v.Composition.FPSet = "fps1:ab"
data, err = EncodeVerdict(v)
if err != nil {
t.Fatal(err)
}
for _, field := range []string{`"skipped":4`, `"composition":{"state":"pass"`, `"over-bound":["integration"]`, `"beside":{"novox/mesh-catalog":"cccc"}`,
`"carried-from":{"novox/mesh-catalog":"cbcb"}`, `"fpset":"fps1:ab"`} {
if !strings.Contains(string(data), field) {
t.Errorf("the value lacks %s: %s", field, data)
}
}
got, err := DecodeVerdict(data)
v.V = Version
if err != nil || !SameVerdict(got, v) || got.Levels[LevelUnit].Skipped != 4 || got.Composition == nil || got.Composition.State != CompositionPass {
t.Errorf("round trip = %+v, %v", got, err)
}
notRun := passVerdict()
notRun.Composition = &Composition{State: CompositionNotRun}
if _, err := EncodeVerdict(notRun); err != nil {
t.Errorf("a composition not yet run refused: %v", err)
}
// 1297: a composition not run says why when the assigner is not running, and round-trips with its words.
notRunning := passVerdict()
notRunning.Composition = &Composition{State: CompositionNotRun, Words: AssignerNotRunning}
data, err = EncodeVerdict(notRunning)
if err != nil {
t.Fatalf("a composition not run because the assigner is not running refused: %v", err)
}
if back, err := DecodeVerdict(data); err != nil || back.Composition.Words != AssignerNotRunning {
t.Errorf("its words came back as %+v, %v", back.Composition, err)
}
failed := passVerdict()
failed.State, failed.Failed = VerdictFail, FailedFirst{Machine: "ace", Words: "x"}
failed.Composition = &Composition{State: CompositionFail, Words: "path /etc/x is owned by two modules", SDK: failed.SDK, Machines: []string{"ace"}}
if _, err := EncodeVerdict(failed); err != nil {
t.Errorf("a failed composition refused: %v", err)
}
}
// 1672/AC1: an invalid new value is refused, naming the field.
func TestVerdictNewFieldsChecked(t *testing.T) {
cases := map[string]struct {
edit func(*Verdict)
words []string
}{
"a negative skipped count": {func(v *Verdict) { v.Levels[LevelUnit] = LevelResult{Run: 1, Skipped: -1} }, []string{"unit", "negative"}},
"an unknown composition": {func(v *Verdict) { v.Composition = &Composition{State: "maybe"} }, []string{"composition", "maybe"}},
"a composition with no state": {func(v *Verdict) { v.Composition = &Composition{} }, []string{"composition"}},
"a failed composition without words": {func(v *Verdict) { v.Composition = &Composition{State: CompositionFail} }, []string{"composition", "words"}},
"a passed composition without the sdk": {func(v *Verdict) { v.Composition = &Composition{State: CompositionPass, Machines: []string{"ace"}} }, []string{"composition", "sdk"}},
"a not-run composition naming machines": {func(v *Verdict) { v.Composition = &Composition{State: CompositionNotRun, Machines: []string{"ace"}} }, []string{"composition", "not run"}},
"an unknown over-bound level": {func(v *Verdict) { v.OverBound = []string{"e2e"} }, []string{"e2e"}},
"an over-bound level named twice": {func(v *Verdict) { v.OverBound = []string{LevelUnit, LevelUnit} }, []string{"over-bound", "unit"}},
// ADR 0548 §5, §6: the siblings read and the commits a verdict was carried from.
"a sibling with no commit": {func(v *Verdict) { v.Beside = map[string]string{"novox/mesh-catalog": ""} }, []string{"sibling", "mesh-catalog"}},
"a trunk named as a sibling": {func(v *Verdict) { v.Beside = map[string]string{"novox/mesh-sdk": "aaaa"} }, []string{"trunk", "sibling", "mesh-sdk"}},
"carried from an unjudged repo": {func(v *Verdict) { v.CarriedFrom = map[string]string{"novox/other": "1111"} }, []string{"carried-from", "novox/other"}},
"carried from an empty commit": {func(v *Verdict) { v.CarriedFrom = map[string]string{"novox/mesh-sdk": ""} }, []string{"carried-from", "mesh-sdk"}},
"carried over no move": {func(v *Verdict) { v.CarriedFrom = map[string]string{"novox/mesh-sdk": "aaaa"} }, []string{"nothing moved"}},
"a running verdict carried": {func(v *Verdict) { v.State, v.CarriedFrom = VerdictRunning, map[string]string{"novox/mesh-sdk": "9999"} }, []string{"running", "carried"}},
// Design 32: each series change whole, once, and a change.
"a series change with no from": {func(v *Verdict) { v.Series = []SeriesChange{{Module: "m", Artifact: "go", To: "0.2"}} }, []string{"series", "from"}},
"a series change to itself": {func(v *Verdict) { v.Series = []SeriesChange{{Module: "m", Artifact: "go", From: "0.2", To: "0.2"}} }, []string{"m/go", "same series"}},
"a series change named twice": {func(v *Verdict) {
v.Series = []SeriesChange{{Module: "m", Artifact: "go", From: "0.1", To: "0.2"}, {Module: "m", Artifact: "go", From: "0.2", To: "0.3"}}
}, []string{"m/go", "twice"}},
}
for name, c := range cases {
v := passVerdict()
c.edit(&v)
_, err := EncodeVerdict(v)
if err == nil {
t.Errorf("%s accepted", name)
continue
}
for _, w := range c.words {
if !strings.Contains(err.Error(), w) {
t.Errorf("%s: error %q does not name %q", name, err, w)
}
}
}
}
// 1672/AC2: a value written by the SDK before this change decodes with the new reader, with the new fields empty, and
// the new SDK writes the same bytes for it; a value with the new fields decodes under a reader of today's shape.
func TestVerdictOldAndNewReadersAgree(t *testing.T) {
v, err := DecodeVerdict([]byte(oldVerdictJSON))
if err != nil {
t.Fatalf("the new reader of an old value: %v", err)
}
if v.Composition != nil || v.OverBound != nil || v.Beside != nil || v.CarriedFrom != nil || v.Levels[LevelUnit].Skipped != 0 || v.Levels[LevelUnit].Run != 12 {
t.Errorf("an old value grew fields: %+v", v)
}
again, err := EncodeVerdict(v)
if err != nil || string(again) != oldVerdictJSON {
t.Errorf("the new SDK writes %s, %v for an old value, want today's bytes", again, err)
}
n := passVerdict()
n.Levels[LevelUnit] = LevelResult{Run: 12, Kept: 301, Skipped: 4}
n.Composition = &Composition{State: CompositionNotRun}
n.OverBound = []string{LevelUnit}
n.Beside = map[string]string{"novox/mesh-catalog": "cccc"}
n.CarriedFrom = map[string]string{"novox/mesh-sdk": "9999"}
n.Composition.FPSet = "fps1:00"
data, err := EncodeVerdict(n)
if err != nil {
t.Fatal(err)
}
var ov oldVerdict
if err := json.Unmarshal(data, &ov); err != nil || ov.Group != "feat/x" || ov.State != VerdictPass || ov.Levels[LevelUnit].Run != 12 || ov.SDK == "" {
t.Errorf("the old reader of a new value: %+v, %v", ov, err)
}
}
func sampleCheckerOwed() Owed {
return Owed{Act: OwedStatus, Context: StatusContextCheck, State: "success", Target: "https://mesh.example/delivery/check/36",
Text: "pass 4b5491557853 on aaaa: unit 12 run 301 kept, integration 3 run 40 kept; composition: not yet run"}
}
// 1673/AC1: the checker's owed bucket is the checker module plus _owed, and its key is the gate's owed key with the act
// status=mesh/check.
func TestCheckerOwedKeyAndBucket(t *testing.T) {
if CheckerOwedBucket != "mesh-delivery-checker_owed" || CheckerOwedHistory != 5 || PreviewsBucket != "mesh-delivery-checker_previews" || PreviewsHistory != 5 {
t.Errorf("buckets %s %d %s %d", CheckerOwedBucket, CheckerOwedHistory, PreviewsBucket, PreviewsHistory)
}
key, err := CheckerOwedKey("novox/mesh-sdk", 36, "aaaa")
if err != nil || key != "novox/mesh-sdk.36.aaaa.status=mesh/check" {
t.Fatalf("key = %q, %v", key, err)
}
repo, number, head, err := ParseCheckerOwedKey(key)
if err != nil || repo != "novox/mesh-sdk" || number != 36 || head != "aaaa" {
t.Errorf("parse = %q %d %q %v", repo, number, head, err)
}
other, _ := OwedKey("novox/mesh-sdk", 36, "aaaa", "status=mesh/merge-gate")
for _, bad := range []string{other, "novox/mesh-sdk.36.aaaa.view", "novox/mesh-sdk.36.aaaa.note", "novox/mesh-sdk.36.aaaa.comment.1", ""} {
if _, _, _, err := ParseCheckerOwedKey(bad); err == nil {
t.Errorf("checker owed key %q accepted", bad)
}
}
if _, err := CheckerOwedKey("nope", 36, "aaaa"); err == nil {
t.Error("a bad repository accepted")
}
}
// 1673/AC2: an owed value with the context mesh/check is accepted and an unknown context is refused, in the act and in the value.
func TestOwedAcceptsMeshCheckAndRefusesAnUnknownContext(t *testing.T) {
for _, ctx := range []string{StatusContextCheck, StatusContextMergeGate} {
if kind, got, err := ParseOwedAct("status=" + ctx); err != nil || kind != OwedStatus || got != ctx {
t.Errorf("ParseOwedAct(status=%s) = %q %q %v", ctx, kind, got, err)
}
}
for _, bad := range []string{"status=mesh/other", "status=check", "status=mesh/Check"} {
if _, err := OwedKey("novox/mesh-sdk", 36, "aaaa", bad); err == nil || !strings.Contains(err.Error(), bad[len("status="):]) {
t.Errorf("owed act %q: %v, want a refusal naming the context", bad, err)
}
}
data, err := EncodeCheckerOwed(sampleCheckerOwed())
if err != nil {
t.Fatal(err)
}
if got, err := DecodeCheckerOwed(data); err != nil || !SameOwed(got, sampleCheckerOwed()) && got.Context != StatusContextCheck {
t.Errorf("round trip = %+v, %v", got, err)
}
unknown := sampleCheckerOwed()
unknown.Context = "mesh/other"
if _, err := EncodeOwed(unknown); err == nil || !strings.Contains(err.Error(), "mesh/other") {
t.Errorf("an owed status of the context mesh/other: %v", err)
}
// The checker's bucket holds only its own status.
gate := sampleCheckerOwed()
gate.Context = StatusContextMergeGate
for name, o := range map[string]Owed{"the gate's context": gate, "a view": {Act: OwedView, Text: "x"}, "a note": {Act: OwedNote, Text: "x", Ref: "mesh-plan"}} {
if _, err := EncodeCheckerOwed(o); err == nil {
t.Errorf("the checker's bucket takes %s", name)
}
}
for _, state := range []string{"pending", "success", "failure"} {
o := sampleCheckerOwed()
o.State = state
if _, err := EncodeCheckerOwed(o); err != nil {
t.Errorf("state %s: %v", state, err)
}
}
}
func samplePreview() Preview {
return Preview{Repository: "novox/mesh-delivery", Head: "aaaa", Trunk: "bbbb", Merge: "cccc",
Modules: map[string]PreviewModule{
"modules/mesh-delivery-checker": {Tree: "t1", Manifest: strings.Repeat("1", 64), Locks: map[string]string{"go.sum": strings.Repeat("2", 64)}},
"modules/mesh-delivery-view": {Tree: "t2", Manifest: strings.Repeat("3", 64)},
}}
}
// 1673/AC3: the previews key is <repository>.<head>.<trunk> and parses back.
func TestPreviewKey(t *testing.T) {
key, err := PreviewKey("novox/mesh-delivery", "aaaa", "bbbb")
if err != nil || key != "novox/mesh-delivery.aaaa.bbbb" {
t.Fatalf("key = %q, %v", key, err)
}
repo, head, trunk, err := ParsePreviewKey(key)
if err != nil || repo != "novox/mesh-delivery" || head != "aaaa" || trunk != "bbbb" {
t.Errorf("parse = %q %q %q %v", repo, head, trunk, err)
}
for _, bad := range [][3]string{{"nope", "aaaa", "bbbb"}, {"novox/x", "", "bbbb"}, {"novox/x", "aaaa", ""}, {"novox/x", "a.b", "bbbb"}, {"novox/x", "aaaa", "b b"}} {
if _, err := PreviewKey(bad[0], bad[1], bad[2]); err == nil {
t.Errorf("PreviewKey%v accepted", bad)
}
}
for _, bad := range []string{"", "novox/x.aaaa", "novox/x.aaaa.bbbb.cccc", "nope.aaaa.bbbb", "novox/x..bbbb", "novox/x.aaaa."} {
if _, _, _, err := ParsePreviewKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParsePreviewKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1673/AC4: a preview round trips, and an invalid one is refused naming the field.
func TestPreviewRoundTripAndRefusals(t *testing.T) {
data, err := EncodePreview(samplePreview())
if err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(string(data), `{"v":1,`) {
t.Errorf("at %s", data)
}
got, err := DecodePreview(data)
want := samplePreview()
want.V = Version
if err != nil || !SamePreview(got, want) {
t.Errorf("round trip = %+v, %v", got, err)
}
if _, err := EncodePreview(Preview{Repository: "novox/x", Head: "aaaa", Trunk: "bbbb", Merge: "cccc"}); err != nil {
t.Errorf("a preview of a head that changes no module refused: %v", err)
}
cases := map[string]struct {
edit func(*Preview)
words []string
}{
"no repository": {func(p *Preview) { p.Repository = "" }, []string{"repository"}},
"a bad repository": {func(p *Preview) { p.Repository = "nope" }, []string{"nope"}},
"no head": {func(p *Preview) { p.Head = "" }, []string{"head"}},
"no trunk": {func(p *Preview) { p.Trunk = "" }, []string{"trunk"}},
"no merge": {func(p *Preview) { p.Merge = "" }, []string{"merge"}},
"a module with no tree": {func(p *Preview) {
p.Modules["modules/mesh-delivery-view"] = PreviewModule{Manifest: strings.Repeat("3", 64)}
}, []string{"modules/mesh-delivery-view", "tree"}},
"a bad manifest digest": {func(p *Preview) {
p.Modules["modules/mesh-delivery-view"] = PreviewModule{Tree: "t2", Manifest: "xyz"}
}, []string{"modules/mesh-delivery-view", "xyz"}},
"an empty directory": {func(p *Preview) { p.Modules[""] = PreviewModule{Tree: "t", Manifest: strings.Repeat("3", 64)} }, []string{"directory"}},
"a lock with no hash": {func(p *Preview) {
p.Modules["modules/mesh-delivery-view"] = PreviewModule{Tree: "t2", Manifest: strings.Repeat("3", 64), Locks: map[string]string{"go.sum": ""}}
}, []string{"modules/mesh-delivery-view", "go.sum"}},
}
for name, c := range cases {
p := samplePreview()
c.edit(&p)
_, err := EncodePreview(p)
if err == nil {
t.Errorf("%s accepted", name)
continue
}
for _, w := range c.words {
if !strings.Contains(err.Error(), w) {
t.Errorf("%s: error %q does not name %q", name, err, w)
}
}
}
if _, err := DecodePreview([]byte(`{"v":2,"repository":"novox/x"}`)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("a preview of version 2: %v", err)
}
if _, err := DecodePreview([]byte(`{"v":1,"repository":"novox/x","head":"a","trunk":"b"}`)); err == nil {
t.Error("decoded a preview with no merge")
}
}
// 1673/AC5: a reader of today's checker shapes ignores a field it does not know, and the checker's owed value is the
// gate's shape, so the gate's reader reads it (as indexer2_test.go does for the indexer's values).
func TestCheckerOwedIsReadableByTheGatesReaderAndAnOldOwedReaderIgnoresNewFields(t *testing.T) {
data, err := EncodeCheckerOwed(sampleCheckerOwed())
if err != nil {
t.Fatal(err)
}
var old struct {
V int `json:"v"`
Act string `json:"act"`
Context string `json:"context"`
State string `json:"state"`
}
if err := json.Unmarshal(data, &old); err != nil || old.Act != OwedStatus || old.Context != "mesh/check" {
t.Errorf("the old reader: %+v %v", old, err)
}
withNew := strings.Replace(string(data), `{"v":1,`, `{"v":1,"a-field-of-tomorrow":7,`, 1)
if _, err := DecodeOwed([]byte(withNew)); err != nil {
t.Errorf("an owed value with a field of tomorrow: %v", err)
}
p, _ := EncodePreview(samplePreview())
withNew = strings.Replace(string(p), `{"v":1,`, `{"v":1,"a-field-of-tomorrow":7,`, 1)
if _, err := DecodePreview([]byte(withNew)); err != nil {
t.Errorf("a preview with a field of tomorrow: %v", err)
}
}
// 1673/AC2 (review): the gate's recheck owes a pending status=mesh/repo-check (mesh-catalog mesh-delivery-gate owed.go,
// checks.go ctxRepo); its key and value round trip, and the checker's bucket still refuses it.
func TestTheGatesRecheckStatusRoundTripsAndTheCheckersBucketRefusesIt(t *testing.T) {
key, err := OwedKey("novox/mesh-sdk", 36, "aaaa", "status=mesh/repo-check")
if err != nil || key != "novox/mesh-sdk.36.aaaa.status=mesh/repo-check" {
t.Fatalf("key = %q, %v", key, err)
}
if _, _, _, act, err := ParseOwedKey(key); err != nil || act != "status=mesh/repo-check" {
t.Errorf("parse = %q, %v", act, err)
}
o := Owed{Act: OwedStatus, Context: StatusContextRepoCheck, State: "pending", Target: "https://mesh.example/delivery/36", Text: "checking again"}
data, err := EncodeOwed(o)
if err != nil {
t.Fatal(err)
}
if got, err := DecodeOwed(data); err != nil || !SameOwed(got, o) && got.Context != o.Context {
t.Errorf("round trip = %+v, %v", got, err)
}
if _, err := EncodeCheckerOwed(o); err == nil {
t.Error("the checker's bucket takes the gate's recheck status")
}
if _, _, _, err := ParseCheckerOwedKey(key); err == nil {
t.Error("a checker owed key of the recheck status accepted")
}
}
// 1445/AC5 (layout): a head that does not merge is a preview naming its conflicting files, with no merge commit and no
// modules; a preview with conflicts and a merge or modules is refused; a preview of today's shape (no conflicts)
// decodes with none, and a reader of today's shape reads one with conflicts.
func TestPreviewOfAHeadInConflict(t *testing.T) {
c := Preview{Repository: "novox/mesh-delivery", Head: "aaaa", Trunk: "bbbb", Conflicts: []string{"a.txt", "modules/m/module.json"}}
data, err := EncodePreview(c)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(data), `"conflicts":["a.txt","modules/m/module.json"]`) || strings.Contains(string(data), `"merge"`) {
t.Errorf("at %s", data)
}
got, err := DecodePreview(data)
c.V = Version
if err != nil || !SamePreview(got, c) {
t.Errorf("round trip = %+v, %v", got, err)
}
withMerge := c
withMerge.Merge = "cccc"
withModules := c
withModules.Modules = samplePreview().Modules
emptyPath := c
emptyPath.Conflicts = []string{""}
for name, p := range map[string]Preview{"a merge commit": withMerge, "modules": withModules, "an empty conflicting path": emptyPath} {
if _, err := EncodePreview(p); err == nil || !strings.Contains(err.Error(), "onflict") {
t.Errorf("a preview in conflict with %s: %v", name, err)
}
}
// today's bytes: a clean preview has no conflicts and decodes with none.
clean, _ := EncodePreview(samplePreview())
if strings.Contains(string(clean), "conflicts") {
t.Errorf("a clean preview carries conflicts: %s", clean)
}
if p, err := DecodePreview(clean); err != nil || p.Conflicts != nil {
t.Errorf("a clean preview decodes with conflicts %v, %v", p.Conflicts, err)
}
var old struct {
V int `json:"v"`
Repository string `json:"repository"`
Head string `json:"head"`
Trunk string `json:"trunk"`
Merge string `json:"merge"`
}
if err := json.Unmarshal(data, &old); err != nil || old.Head != "aaaa" || old.Merge != "" {
t.Errorf("the old reader: %+v %v", old, err)
}
}
// Issue 1921: a failed verdict names every failed test in Failures and reads back the same; an older reader's value
// without it reads too; Failures on a verdict that is no fail, and more than MaxFailures and the count of the rest, are
// refused.
func TestVerdictNamesEveryFailure(t *testing.T) {
v := passVerdict()
v.State = VerdictFail
v.Failed = FailedFirst{Test: "integration: x/a.TestA", Words: "a_test.go:9: want 1, got 2\nmore"}
v.Failures = []FailedFirst{{Test: "integration: x/a.TestA", Words: "a_test.go:9: want 1, got 2"}, {Test: "integration: x/b.TestB", Words: "b_test.go:3: boom"}}
data, err := EncodeVerdict(v)
if err != nil {
t.Fatal(err)
}
got, err := DecodeVerdict(data)
v.V = Version
if err != nil || !SameVerdict(got, v) || !strings.Contains(string(data), `"failures":[{"test":"integration: x/a.TestA"`) {
t.Fatalf("%s: %+v %v", data, got, err)
}
old := strings.Replace(string(data), string(data[strings.Index(string(data), `,"failures":`):strings.Index(string(data), `,"levels":`)]), "", 1)
if got, err := DecodeVerdict([]byte(old)); err != nil || len(got.Failures) != 0 || got.Failed != v.Failed {
t.Fatalf("a value without failures: %+v %v (%s)", got, err, old)
}
p := passVerdict()
p.Failures = v.Failures
if _, err := EncodeVerdict(p); err == nil {
t.Error("a pass with failures was written")
}
v.Failures = make([]FailedFirst, MaxFailures+2)
for i := range v.Failures {
v.Failures[i] = FailedFirst{Test: "unit: x.T" + strconv.Itoa(i)}
}
if _, err := EncodeVerdict(v); err == nil {
t.Error("a verdict naming more than MaxFailures and the rest was written")
}
}
-245
View File
@@ -1,245 +0,0 @@
package delivery
import (
"encoding/json"
"fmt"
"regexp"
"strings"
)
// The compositions (novox/hq ADR 0539 clauses 1 to 4). A **composition** is one module's resources composed for one node
// at one fingerprint: a node composes its own (ADR 0547 §2) and writes each it makes into its node's bucket,
// `composed-<node>` (ADR 0593 §6), under `<node>.<module>.<fingerprint hex>`, and
// the node's own resources (adoption, firewall openings, accounts, the resolver, identities and groups) under
// `<node>.node`. A composition's digest and its judgement of the move from the composition the node runs now is
// ComposedDigest.
const (
// ComposedBucketPrefix starts the name of a node's bucket of compositions, ComposedBucketFor(node).
ComposedBucketPrefix = "composed-"
// ComposedHistory is the revisions a node's composition bucket keeps per key (ADR 0539 clause 4).
ComposedHistory = 5
// ComposedKeep is how many of a module's newest fingerprints keep their compositions though no want names them
// (ADR 0539 clause 4).
ComposedKeep = 5
)
// Who writes and who reads a node's compositions (novox/hq ADR 0593 §6): the node's engine alone writes its node's
// bucket, unsigned, a record for readers that no engine applies; the grants part creates each bucket, create-only, and
// grants the readers read-only on every node's bucket. A reader declares ComposedRead in its manifest's `reads`, and the
// engine's manifest announces the state ComposedState so that name resolves.
const (
// ComposedState is the local name the node-engine's manifest announces for the compositions of every node.
ComposedState = "composed"
// ComposedRead is the `reads` entry naming it: the node-engine's module and ComposedState.
ComposedRead = "mesh-host." + ComposedState
)
// ComposedReaders are the modules that may read every node's compositions (ADR 0593 §6, §8): the provisioner makes its
// plans from them, the assigner compares them for mesh-wants.upgrade, and the view shows them.
var ComposedReaders = []string{"mesh-delivery-assigner", "mesh-delivery-view", "mesh-provisioner"}
// nodeWord is the last part of a node's own key, in the have bucket and in a node's composition bucket alike.
const nodeWord = "node"
var bucketNode = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
// ComposedBucketFor is the name of a node's bucket of compositions, "composed-<node>", as declaration.FileBucketFor
// names a node's object bucket (ADR 0488 §4). The node name must be one a bucket and a subject token accept: letters,
// digits, '_' and '-'.
func ComposedBucketFor(node string) (string, error) {
if node == "" {
return "", fmt.Errorf("delivery: the node name is empty; a node's composition bucket is named for its node")
}
if !bucketNode.MatchString(node) {
return "", fmt.Errorf("delivery: the node name %q is not one a bucket accepts: letters, digits, '_' and '-' only", node)
}
return ComposedBucketPrefix + node, nil
}
// ComposedKey is the key, in ComposedBucketFor(node) and in ControllerComposedBucket, of a module's composition for a
// node at a fingerprint: `<node>.<module>.<hex>`, the hex being the fingerprint's key part (FingerprintKeyPart). A
// module named `node` is refused, since `<node>.node` is the node's own key.
func ComposedKey(node, module, fingerprint string) (string, error) {
have, err := HaveKey(node, module)
if err != nil {
return "", err
}
part, err := FingerprintKeyPart(fingerprint)
if err != nil {
return "", err
}
return have + "." + part, nil
}
// ComposedNodeKey is the key of a node's own composition: `<node>.node`.
func ComposedNodeKey(node string) (string, error) {
return HaveNodeKey(node)
}
// ParseComposedKey is the inverse of ComposedKey and ComposedNodeKey: isNode is true for `<node>.node`, with module
// and fingerprint empty. It refuses any other shape, naming the key.
func ParseComposedKey(key string) (node, module, fingerprint string, isNode bool, err error) {
refused := func(why error) (string, string, string, bool, error) {
return "", "", "", false, fmt.Errorf("delivery: %q is not a composition key, <node>.<module>.<fingerprint hex> or <node>.node: %w", key, why)
}
parts := strings.Split(key, ".")
switch len(parts) {
case 2:
if parts[1] != nodeWord {
return refused(fmt.Errorf("a module's composition names its fingerprint"))
}
if err := checkName("node", parts[0]); err != nil {
return refused(err)
}
return parts[0], "", "", true, nil
case 3:
node, module, isNode, err := ParseHaveKey(parts[0] + "." + parts[1])
if err != nil {
return refused(err)
}
if isNode {
return refused(fmt.Errorf("the node's own key names no fingerprint"))
}
fp, err := ParseFingerprintKeyPart(parts[2])
if err != nil {
return refused(err)
}
return node, module, fp, false, nil
}
return refused(fmt.Errorf("it has %d parts", len(parts)))
}
// Judgement is the controller's judgement of a move from the composition a node runs, as its have names it, to a new
// one (ADR 0539 clauses 2 and 14, by ADR 0377 §3's tests).
type Judgement string
// The judgements of a move.
const (
// JudgedRecreates is a move that recreates a container.
JudgedRecreates Judgement = "recreates"
// JudgedMovesData is a move that gives the module's data another place.
JudgedMovesData Judgement = "moves-data"
// JudgedNeither is a move that does neither, and the judgement of a composition the node never ran.
JudgedNeither Judgement = "neither"
// JudgedCannot is a move that cannot be judged, because the composition the node runs is gone or cannot be read.
JudgedCannot Judgement = "cannot-judge"
// JudgedUnchanged is a composition equal to the module's slice of the declaration the node last applied, for a node
// whose have names no composition yet (the first want at the switch, novox/hq ADR 0540 clause 9): applying it
// changes nothing on the node, so it needs no person's yes.
JudgedUnchanged Judgement = "unchanged"
)
var digestForm = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
func checkDigest(digest, of string) error {
if !digestForm.MatchString(digest) {
return fmt.Errorf("delivery: %s %q is not sha256:<64 lowercase hex digits>", of, digest)
}
return nil
}
// ComposedDigest is the value, in ControllerComposedBucket, beside a composition: its Digest (ComposedDigestOf), and
// the Judgement of the move to it from the composition the node runs now, whose digest is Against (empty when the node
// runs no composition of the module). Words say why, and are required for a move that cannot be judged. Rewritten when
// the node's have changes.
//
// Policy is the module's upgrade policy as the controller holds it (ADR 0236, ADR 0539 clause 14), with PolicyFrom
// where it comes from and PolicyWhy why: the assigner holds a move of a module of policy record from it, the operator's
// own choice (`upgrade` with policy) included. Rewritten when the policy changes. Empty on a composition made by hand,
// whose want the operator wrote.
type ComposedDigest struct {
V int `json:"v"`
Digest string `json:"digest"`
Judgement Judgement `json:"judgement"`
Against string `json:"against,omitempty"`
Words string `json:"words,omitempty"`
// UserListOnly says the composition differs from the one the node's have names only in the bus's user list (the
// bus module's file `<module>.bus-users`), whatever the judgement says of its mount: the assigner writes such a
// revision into the bus node's want at once, outside the record hold, at the fingerprint the node runs (ADR 0539
// clause 7).
UserListOnly bool `json:"user-list-only,omitempty"`
Policy string `json:"policy,omitempty"`
PolicyFrom string `json:"policy-from,omitempty"`
PolicyWhy string `json:"policy-why,omitempty"`
}
// The upgrade policies and their sources a composition's value names: catalogue.PolicyRoll, PolicyTogether and
// PolicyRecord, and catalogue.FromPerson, FromModule, FromData and FromDefault, the same words (the compose package's
// test holds them equal; this package does not import the catalogue). "the bus" is the word an older reader derived
// for the bus module, still read from a composition it wrote; nothing derives it now.
var (
composedPolicies = []string{"roll", "together", "record"}
composedPolicyFroms = []string{"person", "module", "the bus", "irreplaceable data", "default"}
)
func oneOf(v string, of []string) bool {
for _, o := range of {
if v == o {
return true
}
}
return false
}
func (c ComposedDigest) check() error {
if err := checkDigest(c.Digest, "the composition's digest"); err != nil {
return err
}
if c.Against != "" {
if err := checkDigest(c.Against, "the digest a move is judged against"); err != nil {
return err
}
}
switch c.Judgement {
case JudgedRecreates, JudgedMovesData:
if c.Against == "" {
return fmt.Errorf("delivery: a move judged %s names the composition it is judged against", c.Judgement)
}
case JudgedNeither, JudgedUnchanged:
case JudgedCannot:
if c.Words == "" {
return fmt.Errorf("delivery: a move that cannot be judged says why")
}
default:
return fmt.Errorf("delivery: the judgement %q is not %s, %s, %s, %s or %s", c.Judgement, JudgedRecreates,
JudgedMovesData, JudgedNeither, JudgedCannot, JudgedUnchanged)
}
if c.Policy != "" || c.PolicyFrom != "" {
if !oneOf(c.Policy, composedPolicies) {
return fmt.Errorf("delivery: the upgrade policy %q is not one of %v", c.Policy, composedPolicies)
}
if !oneOf(c.PolicyFrom, composedPolicyFroms) {
return fmt.Errorf("delivery: the upgrade policy's source %q is not one of %v", c.PolicyFrom, composedPolicyFroms)
}
}
return nil
}
// PolicyRecord is the policy whose moves the assigner holds for a person's yes (ADR 0539 clause 14).
const PolicyRecord = "record"
// EncodeComposedDigest is c as JSON at Version; it refuses an unknown judgement, a digest not of the sha256 form, a
// move judged to recreate or move data against nothing, and a move that cannot be judged without its reason.
func EncodeComposedDigest(c ComposedDigest) ([]byte, error) {
if err := c.check(); err != nil {
return nil, err
}
c.V = Version
return json.Marshal(c)
}
// DecodeComposedDigest reads the value beside a composition; it refuses an unknown version (ErrUnknownVersion) and
// what EncodeComposedDigest refuses.
func DecodeComposedDigest(data []byte) (ComposedDigest, error) {
var c ComposedDigest
if err := decode(data, &c); err != nil {
return ComposedDigest{}, err
}
if err := c.check(); err != nil {
return ComposedDigest{}, err
}
return c, nil
}
// SameComposedDigest says whether two values are the same, so the controller writes only when they differ.
func SameComposedDigest(a, b ComposedDigest) bool { return a == b }
-387
View File
@@ -1,387 +0,0 @@
package delivery
import (
"bytes"
"encoding/json"
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Comparison is what CompareComposed finds, module by module: the same, differing, added by what was composed (not run
// now) and removed by it (run now, not composed). Each list is sorted by module.
type Comparison struct {
Same []string
Differs []ModuleDiffers
Added []ModuleAdded
Removed []string
}
// CompareComposed compares the compositions a node composed (by key, as compose.Compositions answers them) with the ones
// it runs, as its agent would (ADR 0559 §4: a node's answer to a placement candidate). A module is the same when its
// composition's digest is the one it runs (ComposedDigestOf, the signature left out), so the agent would apply nothing
// of it; it differs when the digests differ (both named); it is added when composed and not run, removed when run and
// not composed. The node's own composition is named HaveNodeWord. The bus's user list (catalogue.BusUsersID) is left out
// on both sides: the bus node follows it alone and never applies it from a composition it made (ADR 0547 §2, §8). A
// difference names the first resource, by its id, whose content differs or that one side alone holds; when every
// resource is equal it says what else differs (their order, the files filled, the data, the node's own parts, a file
// carried by digest).
func CompareComposed(composed map[string]Composed, running []Composed) Comparison {
return CompareComposedWith(composed, running, nil)
}
// CompareComposedWith is CompareComposed showing, for each module that differs, every differing resource with both
// bodies masked (MaskedPair); sameValue, when given, says whether two sealed values open to the same value (the node
// opens them with its own key), so a value sealed again reads as such.
func CompareComposedWith(composed map[string]Composed, running []Composed, sameValue func(a, b string) (same, known bool)) Comparison {
name := func(c Composed) string {
if c.Module == "" {
return HaveNodeWord
}
return c.Module
}
ours := map[string]Composed{}
for _, c := range composed {
ours[name(c)] = c
}
theirs := map[string]Composed{}
for _, c := range running {
theirs[name(c)] = c
}
modules := map[string]bool{}
for m := range ours {
modules[m] = true
}
for m := range theirs {
modules[m] = true
}
var out Comparison
for _, m := range sortedKeys(modules) {
a, composedHere := ours[m]
b, runs := theirs[m]
switch {
case !runs:
dig, _ := ComposedDigestOf(withoutBusUsers(a))
out.Added = append(out.Added, ModuleAdded{Module: m, Composed: dig})
case !composedHere:
out.Removed = append(out.Removed, m)
default:
if d, ok := compareOne(m, a, b, sameValue); ok {
out.Same = append(out.Same, m)
} else {
out.Differs = append(out.Differs, d)
}
}
}
return out
}
// compareOne is one module's comparison: true when the digests agree, else what differs first.
func compareOne(module string, composed, running Composed, sameValue func(a, b string) (bool, bool)) (ModuleDiffers, bool) {
composed, running = withoutBusUsers(composed), withoutBusUsers(running)
// A sealed value is judged by the value it opens to, slot by slot (issue 1126, C1): a value sealed again at the same
// field of the same resource is the box that runs, never a difference; and a box is never matched across resources.
composed = KeepSeals(composed, SealsIn(running), sameValue)
dc, errC := ComposedDigestOf(composed)
dr, errR := ComposedDigestOf(running)
if errC != nil {
return ModuleDiffers{Module: module, Running: dr, Why: "the composition has no digest: " + errC.Error()}, false
}
if errR == nil && dc == dr {
return ModuleDiffers{}, true
}
differs := func(resource, why string) (ModuleDiffers, bool) {
return ModuleDiffers{Module: module, Composed: dc, Running: dr, Resource: resource, Why: why}, false
}
// Words for the difference: resource by resource, each side with its large files by digest, so a file carried whole
// and the same file carried by digest read as one.
a, _, errA := composed.ByDigest()
b, _, errB := running.ByDigest()
if errA != nil || errB != nil {
return differs("", "the compositions differ and cannot be read resource by resource")
}
ra, rb := resourcesByID(a.Resources), resourcesByID(b.Resources)
ids := map[string]bool{}
for id := range ra {
ids[id] = true
}
for id := range rb {
ids[id] = true
}
var first ModuleDiffers
found := false
var bodies []ResourceBodies
for _, id := range sortedKeys(ids) {
x, inA := ra[id]
y, inB := rb[id]
why := ""
switch {
case !inB:
why = "composed, and not in what runs"
case !inA:
why = "in what runs, and not composed"
case !bytes.Equal(x, y):
why = "the resource differs"
default:
continue
}
if !found {
first, _ = differs(id, why)
found = true
}
if len(bodies) < MaxBodies {
// The paths are found on the whole bodies, before either is cut (issue 1126, T1: three resources over
// MaxBody could not be told apart).
c, r := MaskedPair(x, y, sameValue)
bodies = append(bodies, ResourceBodies{ID: id, Why: why, Paths: DifferingPaths(x, y),
Composed: cut(c), Running: cut(r)})
}
}
if found {
first.Bodies = bodies
return first, false
}
switch {
case !jsonEqual(a.Filled, b.Filled):
return differs("", "the files the provisioner fills differ")
case a.Data != b.Data:
return differs("", "whether it declares data differs")
case !jsonEqual(a.Own, b.Own):
return differs("", "the node's own parts differ")
case a.FP != b.FP:
return differs("", "the fingerprint differs")
}
if dA, err := ComposedDigestOf(a); err == nil {
if dB, err := ComposedDigestOf(b); err == nil && dA == dB {
return differs("", "the same content, a large file carried whole on one side and by digest on the other")
}
}
return differs("", "the same resources in another order")
}
// withoutBusUsers is c without the bus's user list resource (its id `bus-users`, or `<module>.bus-users`).
func withoutBusUsers(c Composed) Composed {
out := c
out.Resources = nil
for _, raw := range c.Resources {
var head struct {
ID string `json:"id"`
}
if json.Unmarshal(raw, &head) == nil && (head.ID == catalogue.BusUsersID() || strings.HasSuffix(head.ID, "."+catalogue.BusUsersID())) {
continue
}
out.Resources = append(out.Resources, raw)
}
return out
}
// resourcesByID is each resource canonical, by its id (a resource without one by its place).
func resourcesByID(resources []json.RawMessage) map[string][]byte {
out := make(map[string][]byte, len(resources))
for i, raw := range resources {
var head struct {
ID string `json:"id"`
}
_ = json.Unmarshal(raw, &head)
id := head.ID
if id == "" {
id = "#" + strconv.Itoa(i)
}
canon, err := canonicalJSON(raw)
if err != nil {
canon = raw
}
out[id] = canon
}
return out
}
// cut is s cut at MaxBody bytes, saying so.
func cut(s string) string {
if len(s) <= MaxBody {
return s
}
return s[:MaxBody] + "… (cut at " + strconv.Itoa(MaxBody) + " bytes)"
}
// secretWords are the words of a key whose value may hold a secret or a sealed one: masked in a shown body.
var secretWords = []string{"sealed", "secret", "password", "passwd", "token", "private", "key", "credential", "jwt", "salt", "hash"}
func secretKey(k string) bool {
k = strings.ToLower(k)
for _, w := range secretWords {
if strings.Contains(k, w) {
return true
}
}
return false
}
// MaskedPair is two bodies of one resource (either may be nil) as JSON text with every value under a key a secret could
// be in (secretWords, at any depth) masked: "<masked: the same on both sides>", "<masked: another seal of the same
// value>" or "<masked: another value>" when sameValue says so, "<masked: differs>" otherwise, "<masked>" on one side
// alone. A body that is not JSON is shown as its length only.
func MaskedPair(composed, running json.RawMessage, sameValue func(a, b string) (same, known bool)) (string, string) {
var a, b any
okA := composed != nil && json.Unmarshal(composed, &a) == nil
okB := running != nil && json.Unmarshal(running, &b) == nil
if okA && okB {
a, b = maskBoth(a, b, sameValue)
} else if okA {
a, _ = maskBoth(a, nil, nil)
} else if okB {
_, b = maskBoth(nil, b, nil)
}
show := func(v any, raw json.RawMessage, ok bool) string {
if raw == nil {
return ""
}
if !ok {
return "<not JSON: " + strconv.Itoa(len(raw)) + " bytes>"
}
var out bytes.Buffer
enc := json.NewEncoder(&out)
enc.SetEscapeHTML(false)
_ = enc.Encode(v)
return strings.TrimSuffix(out.String(), "\n")
}
return show(a, composed, okA), show(b, running, okB)
}
func maskBoth(a, b any, sameValue func(a, b string) (bool, bool)) (any, any) {
ma, isA := a.(map[string]any)
mb, isB := b.(map[string]any)
if isA || isB {
outA, outB := map[string]any{}, map[string]any{}
keys := map[string]bool{}
for k := range ma {
keys[k] = true
}
for k := range mb {
keys[k] = true
}
for k := range keys {
va, inA := ma[k]
vb, inB := mb[k]
if secretKey(k) {
x, y := maskAll(k, va, vb, inA && inB, sameValue)
if inA {
outA[k] = x
}
if inB {
outB[k] = y
}
continue
}
x, y := maskBoth(va, vb, sameValue)
if inA {
outA[k] = x
}
if inB {
outB[k] = y
}
}
var ra, rb any
if isA {
ra = outA
} else {
ra = a
}
if isB {
rb = outB
} else {
rb = b
}
return ra, rb
}
la, isLA := a.([]any)
lb, isLB := b.([]any)
if isLA || isLB {
outA := make([]any, len(la))
outB := make([]any, len(lb))
for i := range max(len(la), len(lb)) {
var x, y any
if i < len(la) {
x = la[i]
}
if i < len(lb) {
y = lb[i]
}
x, y = maskBoth(x, y, sameValue)
if i < len(la) {
outA[i] = x
}
if i < len(lb) {
outB[i] = y
}
}
var ra, rb any = a, b
if isLA {
ra = outA
}
if isLB {
rb = outB
}
return ra, rb
}
return a, b
}
// maskAll is a value under a key a secret could be under, masked as one word (maskWord); a file's secrets
// (SecretsField) alone entry by entry, each its own sealed value: "another seal of the same value" or "another value"
// per secret, its name shown (issue 1126, C1). Any other object, a vault box included, is one word, judged whole.
func maskAll(key string, a, b any, both bool, sameValue func(a, b string) (bool, bool)) (any, any) {
ma, isA := a.(map[string]any)
mb, isB := b.(map[string]any)
if key != SecretsField || (!isA && !isB) {
w := maskWord(a, b, both, sameValue)
return w, w
}
outA, outB := map[string]any{}, map[string]any{}
for k, va := range ma {
vb, inB := mb[k]
outA[k] = maskWord(va, vb, isB && inB, sameValue)
}
for k, vb := range mb {
va, inA := ma[k]
outB[k] = maskWord(va, vb, isA && inA, sameValue)
}
// A map on one side and another value on the other: the other is one word.
var ra, rb any = outA, outB
if !isA {
ra = maskWord(a, b, both, sameValue)
}
if !isB {
rb = maskWord(a, b, both, sameValue)
}
return ra, rb
}
func maskWord(a, b any, both bool, sameValue func(a, b string) (bool, bool)) string {
if !both {
return "<masked>"
}
x, _ := json.Marshal(a)
y, _ := json.Marshal(b)
if bytes.Equal(x, y) {
return "<masked: the same on both sides>"
}
if sameValue != nil {
sa, okA := a.(string)
sb, okB := b.(string)
if !okA {
sa = string(x)
}
if !okB {
sb = string(y)
}
if same, known := sameValue(sa, sb); known {
if same {
return "<masked: another seal of the same value>"
}
return "<masked: another value>"
}
}
return "<masked: differs>"
}
-125
View File
@@ -1,125 +0,0 @@
package delivery
import (
"errors"
"strings"
"testing"
)
// 1943/AC1: a node's composition bucket is named for the node, and a node with no name, or one a bucket refuses,
// is refused, naming why.
func TestComposedBucketFor(t *testing.T) {
b, err := ComposedBucketFor("ace")
if err != nil || b != "composed-ace" {
t.Fatalf("ComposedBucketFor(ace) = %q, %v", b, err)
}
for _, bad := range []string{"", "a.b", "a*", "a b", "a>"} {
if _, err := ComposedBucketFor(bad); err == nil || !strings.Contains(err.Error(), "node") {
t.Errorf("ComposedBucketFor(%q) = %v, want a refusal naming the node", bad, err)
}
}
}
// 1943/AC1: a module's composition key and the node's own key round-trip through one parser; an empty node, a
// module named node, a bad fingerprint and a key of another shape are refused, naming why.
func TestComposedKeys(t *testing.T) {
fp := fp2of('c')
key, err := ComposedKey("ace", "mesh-mail", fp)
if err != nil || key != "ace.mesh-mail."+strings.Repeat("c", 64) {
t.Fatalf("ComposedKey = %q, %v", key, err)
}
node, module, back, isNode, err := ParseComposedKey(key)
if err != nil || node != "ace" || module != "mesh-mail" || back != fp || isNode {
t.Errorf("ParseComposedKey(%q) = %q, %q, %q, %v, %v", key, node, module, back, isNode, err)
}
own, err := ComposedNodeKey("ace")
if err != nil || own != "ace.node" {
t.Fatalf("ComposedNodeKey = %q, %v", own, err)
}
node, module, back, isNode, err = ParseComposedKey(own)
if err != nil || node != "ace" || module != "" || back != "" || !isNode {
t.Errorf("ParseComposedKey(%q) = %q, %q, %q, %v, %v", own, node, module, back, isNode, err)
}
refusals := []struct {
node, module, fp, why string
}{
{"", "mesh-mail", fp, "node name is empty"},
{"ace", "node", fp, `"node" is refused`},
{"ace", "", fp, "module name is empty"},
{"ace", "mesh-mail", "src2:abc", "not a fingerprint of the new form"},
{"ace", "mesh-mail", "src1:" + strings.Repeat("c", 64), "not a fingerprint of the new form"},
{"a.b", "mesh-mail", fp, "node name"},
}
for _, r := range refusals {
if _, err := ComposedKey(r.node, r.module, r.fp); err == nil || !strings.Contains(err.Error(), r.why) {
t.Errorf("ComposedKey(%q, %q, %q) = %v, want a refusal saying %q", r.node, r.module, r.fp, err, r.why)
}
}
if _, err := ComposedNodeKey(""); err == nil {
t.Errorf("ComposedNodeKey(\"\") accepted")
}
for _, bad := range []string{"", "ace", "ace.mesh-mail", "ace.node.x", "ace.mesh-mail.abc", ".mesh-mail." + strings.Repeat("c", 64),
"ace.node." + strings.Repeat("c", 64), "ace.mesh-mail." + strings.Repeat("c", 64) + ".x", "want.ace.mesh-mail"} {
if _, _, _, _, err := ParseComposedKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseComposedKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1943/AC1: the controller's value beside a composition, its digest and the judgement of the move, round-trips; an
// unknown judgement, a digest not of the sha256 form, a move judged against nothing, a cannot-judge without its
// reason, and an upgrade policy or source outside the catalogue's words are refused, naming why; an unknown version is
// refused as one. The module's upgrade policy and its source ride beside the judgement, for the assigner's hold.
func TestComposedDigestValue(t *testing.T) {
digest := "sha256:" + strings.Repeat("d", 64)
against := "sha256:" + strings.Repeat("e", 64)
for _, v := range []ComposedDigest{
{Digest: digest, Judgement: JudgedNeither},
{Digest: digest, Judgement: JudgedNeither, Against: against},
{Digest: digest, Judgement: JudgedRecreates, Against: against, Words: "the container's image changes"},
{Digest: digest, Judgement: JudgedMovesData, Against: against},
{Digest: digest, Judgement: JudgedCannot, Words: "the composition the node runs is gone"},
{Digest: digest, Judgement: JudgedUnchanged, Words: "equal to the declaration the node last applied"},
{Digest: digest, Judgement: JudgedNeither, Policy: "record", PolicyFrom: "person", PolicyWhy: "the operator: its data"},
{Digest: digest, Judgement: JudgedNeither, Policy: "roll", PolicyFrom: "default"},
{Digest: digest, Judgement: JudgedNeither, Against: against, UserListOnly: true, Words: "only the bus's user list changes"},
} {
raw, err := EncodeComposedDigest(v)
if err != nil {
t.Fatalf("EncodeComposedDigest(%+v): %v", v, err)
}
back, err := DecodeComposedDigest(raw)
v.V = Version
if err != nil || !SameComposedDigest(back, v) {
t.Errorf("round trip of %+v = %+v, %v", v, back, err)
}
}
refusals := []struct {
v ComposedDigest
why string
}{
{ComposedDigest{Digest: digest, Judgement: "maybe"}, `judgement "maybe"`},
{ComposedDigest{Digest: digest}, `judgement ""`},
{ComposedDigest{Digest: "sha256:abc", Judgement: JudgedNeither}, "not sha256"},
{ComposedDigest{Judgement: JudgedNeither}, "not sha256"},
{ComposedDigest{Digest: digest, Judgement: JudgedRecreates}, "against"},
{ComposedDigest{Digest: digest, Judgement: JudgedMovesData}, "against"},
{ComposedDigest{Digest: digest, Judgement: JudgedCannot}, "why"},
{ComposedDigest{Digest: digest, Judgement: JudgedNeither, Against: "sha256:zz"}, "not sha256"},
{ComposedDigest{Digest: digest, Judgement: JudgedNeither, Policy: "sometimes", PolicyFrom: "person"}, `policy "sometimes"`},
{ComposedDigest{Digest: digest, Judgement: JudgedNeither, Policy: "record"}, `source ""`},
{ComposedDigest{Digest: digest, Judgement: JudgedNeither, PolicyFrom: "person"}, `policy ""`},
}
for _, r := range refusals {
if _, err := EncodeComposedDigest(r.v); err == nil || !strings.Contains(err.Error(), r.why) {
t.Errorf("EncodeComposedDigest(%+v) = %v, want a refusal saying %q", r.v, err, r.why)
}
}
if _, err := DecodeComposedDigest([]byte(`{"v":2,"digest":"` + digest + `","judgement":"neither"}`)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("version 2 = %v, want ErrUnknownVersion", err)
}
if _, err := DecodeComposedDigest([]byte(`{"v":1,"digest":"` + digest + `","judgement":"perhaps"}`)); err == nil {
t.Errorf("an unknown judgement was read")
}
}
-74
View File
@@ -1,74 +0,0 @@
package delivery
import (
"encoding/json"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// A composition that carries its plan or its refusal (ADR 0593 §6): each round-trips, and a reader refuses what it could
// not take, naming it. (compose-candidate's ask and answer are node-engine's client's, mesh-contracts
// go/clients/nodeengineclient, ADR 0627 §1.)
func TestComposeCandidateAndComposedPlanRoundTripAndRefuse(t *testing.T) {
fp := "src2:" + strings.Repeat("a", 64)
planned := Composed{Node: "ace", Module: "netcheck", FP: fp, Resources: []json.RawMessage{},
Plan: &catalogue.Plan{Files: []catalogue.PlanFile{}}}
raw, err := EncodeComposed(planned)
if err != nil {
t.Fatal(err)
}
c, err := DecodeComposed(raw)
if err != nil || c.Plan == nil {
t.Fatalf("a composition's plan did not round-trip: %v, %+v", err, c)
}
d1, _ := ComposedDigestOf(planned)
d0, _ := ComposedDigestOf(Composed{Node: "ace", Module: "netcheck", FP: fp, Resources: []json.RawMessage{}})
if d1 == "" || d1 != d0 {
t.Error("the digest moves with the plan: it is the resources' alone")
}
if SameComposed(planned, Composed{Node: "ace", Module: "netcheck", FP: fp, Resources: []json.RawMessage{}}) {
t.Error("a plan change is not a new composition value")
}
signed := strings.Replace(string(raw), `"resources"`, `"signature":{"signer":"x","sig":"y"},"resources"`, 1)
if c, err := DecodeComposed([]byte(signed)); err != nil || c.Node != "ace" {
t.Fatalf("a composition the ended controller signed does not decode: %v", err)
}
if strings.Contains(string(raw), "signature") {
t.Error("a composition is written with a signature")
}
refused := Composed{Node: "ace", Module: "netcheck", FP: fp, Refused: "the setting domain has no value"}
if raw, err = EncodeComposed(refused); err != nil {
t.Fatal(err)
}
if c, err := DecodeComposed(raw); err != nil || c.Refused != refused.Refused {
t.Fatalf("a refused composition did not round-trip: %v", err)
}
if _, err := ComposedDigestOf(refused); err == nil {
t.Error("a refused composition has a digest")
}
refused.Data = true
if _, err := EncodeComposed(refused); err == nil {
t.Error("a refused composition holding data was encoded")
}
if _, err := EncodeComposed(Composed{Node: "ace", Resources: []json.RawMessage{}, Refused: "x"}); err == nil {
t.Error("the node's own composition carried a refusal")
}
}
// A kept import form is set only with imports, and in the fingerprint's form (ADR 0593 §4).
func TestAKeptImportFormNeedsImports(t *testing.T) {
form := SourcePrefix2 + strings.Repeat("b", 64)
in := KeptInputs2{Tree: "t", Imports: []string{"git.novox.be/novox/mesh-sdk/go"}, ImportForm: form}
m := Module{FP2: form, Manifest: strings.Repeat("c", 64), Inputs2: &in, Repository: "novox/x", Commit: "c"}
if err := m.check(); err != nil && strings.Contains(err.Error(), "import form") {
t.Fatalf("a good import form refused: %v", err)
}
for _, bad := range []KeptInputs2{{Tree: "t", ImportForm: form}, {Tree: "t", Imports: in.Imports, ImportForm: "src2:short"}} {
m.Inputs2 = &bad
if err := m.check(); err == nil || !strings.Contains(err.Error(), "import form") {
t.Errorf("%+v: %v", bad, err)
}
}
}
-194
View File
@@ -1,194 +0,0 @@
package delivery
import (
"bytes"
"encoding/json"
"sort"
"strconv"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Seals is, by slot, each value one composition carries sealed to its node's key (issue 1126, C1): a file's `sealed`
// a vault box (catalogue.VaultSealedField) and each entry of a file's secrets (SecretsField), as their JSON text (a box is carried as its JSON, a sealed value as
// the string it is). A slot is one field of one resource of the composition's module (SealSlot): a box is reused only
// at the slot that holds it, never for another resource whose value opens to the same (a provider's grant and its
// consumer's secret on one node hold one value).
type Seals map[string]string
// SealedFields are the fields of a resource that carry a value sealed to the node's key.
var SealedFields = []string{"sealed", catalogue.VaultSealedField}
// SealSlot is the slot of one sealed field of one resource, by the resource's id (its place, `#<i>`, when it has none).
// The module is the composition's: a slot is never compared across compositions.
func SealSlot(resource, field string) string { return resource + "#" + field }
// SecretsField is the field of a file that carries, by name, each sealed value its content asks for
// (${secret:<name>}, catalogue's intoFile): each entry is a slot of its own (SecretSlotField), so a secret sealed again
// is judged by the value it opens to, entry by entry, like a file's `sealed` (issue 1126, C1; the seed-7 probe: about 45
// files on ace and novox differed by their secrets' bytes alone).
const SecretsField = "secrets"
// SecretSlotField is the field part of the slot of one entry of a file's secrets (SealSlot(id, SecretSlotField(name))).
func SecretSlotField(name string) string { return SecretsField + "." + name }
// sealAt is one sealed value of a resource: its slot's field, its place (a top-level field, or an entry of the
// secrets), and its raw JSON.
type sealAt struct {
field, entry string
raw json.RawMessage
}
// sealsAt is every sealed value of one resource's fields, in a fixed order: SealedFields, then the secrets by name.
func sealsAt(fields map[string]json.RawMessage) []sealAt {
var out []sealAt
for _, field := range SealedFields {
if f, has := fields[field]; has {
out = append(out, sealAt{field: field, raw: f})
}
}
var secrets map[string]json.RawMessage
if f, has := fields[SecretsField]; has && json.Unmarshal(f, &secrets) == nil {
names := make([]string, 0, len(secrets))
for name := range secrets {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
out = append(out, sealAt{field: SecretSlotField(name), entry: name, raw: secrets[name]})
}
}
return out
}
// SealsIn is every sealed value c carries, by slot; nil when it carries none.
func SealsIn(c Composed) Seals {
var out Seals
for i, raw := range c.Resources {
id, fields, ok := sealedFieldsOf(raw, i)
if !ok {
continue
}
for _, s := range sealsAt(fields) {
if text := sealedText(s.raw); text != "" {
if out == nil {
out = Seals{}
}
out[SealSlot(id, s.field)] = text
}
}
}
return out
}
// KeepSeals is c with each sealed value replaced by the one kept at the same slot, where same says the two open to the
// same value (known and same): a value the vault sealed again is then the box that runs, so the composition is the
// one that runs (ADR 0547 §3, issue 1126). A slot kept holds nothing for, or whose value differs or cannot be opened,
// keeps c's. Every other byte of a resource is kept as it is; c itself is not changed.
func KeepSeals(c Composed, kept Seals, same func(a, b string) (same, known bool)) Composed {
if len(kept) == 0 || same == nil {
return c
}
out := c
out.Resources = nil
for i, raw := range c.Resources {
id, fields, ok := sealedFieldsOf(raw, i)
if ok {
for _, s := range sealsAt(fields) {
ours := sealedText(s.raw)
theirs, held := kept[SealSlot(id, s.field)]
if !held || ours == "" || theirs == ours {
continue
}
if opens, known := same(ours, theirs); !known || !opens {
continue
}
value := json.RawMessage(theirs)
if s.raw[0] == '"' {
value, _ = json.Marshal(theirs)
}
if s.entry == "" {
if swapped, ok := withField(raw, s.field, value); ok {
raw = swapped
}
continue
}
// An entry of the secrets: swapped inside the secrets as they stand in raw, then the secrets in raw.
var now map[string]json.RawMessage
if json.Unmarshal(raw, &now) != nil {
continue
}
if secrets, ok := withField(now[SecretsField], s.entry, value); ok {
if swapped, ok := withField(raw, SecretsField, secrets); ok {
raw = swapped
}
}
}
}
out.Resources = append(out.Resources, raw)
}
return out
}
// sealedFieldsOf is a resource's id (`#<i>` when it has none) and its top-level fields, false when it is no object.
func sealedFieldsOf(raw json.RawMessage, i int) (string, map[string]json.RawMessage, bool) {
var fields map[string]json.RawMessage
if json.Unmarshal(raw, &fields) != nil {
return "", nil, false
}
var id string
if f, ok := fields["id"]; !ok || json.Unmarshal(f, &id) != nil || id == "" {
id = "#" + strconv.Itoa(i)
}
return id, fields, true
}
// sealedText is a sealed field as its text: a string as it is, a box as its JSON.
func sealedText(f json.RawMessage) string {
var text string
if json.Unmarshal(f, &text) == nil {
return text
}
var compact bytes.Buffer
if json.Compact(&compact, f) != nil {
return string(f)
}
return compact.String()
}
// withField is the object raw with its top-level field replaced by value, every other byte as it was (so a composition
// whose sealed values are those that run is the one that runs, byte for byte); false when raw is no object holding it.
func withField(raw json.RawMessage, field string, value json.RawMessage) (json.RawMessage, bool) {
d := json.NewDecoder(bytes.NewReader(raw))
if t, err := d.Token(); err != nil || t != json.Delim('{') {
return nil, false
}
for d.More() {
t, err := d.Token()
if err != nil {
return nil, false
}
key, _ := t.(string)
before := d.InputOffset()
var skip json.RawMessage
if err := d.Decode(&skip); err != nil {
return nil, false
}
if key != field {
continue
}
after := d.InputOffset()
// The span holds the colon, any space and the value: kept up to the colon, the value replaced.
span := raw[before:after]
colon := bytes.IndexByte(span, ':')
if colon < 0 {
return nil, false
}
out := make([]byte, 0, len(raw)-len(span)+colon+1+len(value))
out = append(out, raw[:int(before)+colon+1]...)
out = append(out, value...)
out = append(out, raw[after:]...)
return out, true
}
return nil, false
}
-348
View File
@@ -1,348 +0,0 @@
package delivery
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// NodeOwn is what a node's own composition (`<node>.node`) carries beside its resources: what the declaration says of
// the node as a whole (novox/hq ADR 0539 clause 2: adoption, accounts, identities and groups). A module's composition
// carries none of it.
type NodeOwn struct {
// Adoption says the node is adopted and which of its modules are taken (ADR 0100, ADR 0103); nil for a converged node.
Adoption *declaration.Adoption `json:"adoption,omitempty"`
// Identities is the identity of every module assigned to the node (ADR 0328 §4), Groups and Libraries every library
// it is reached by (ADR 0328 §3).
Identities []declaration.Identity `json:"identities,omitempty"`
Groups []declaration.Group `json:"groups,omitempty"`
Libraries []declaration.Library `json:"libraries,omitempty"`
// Provisioner is the account the provisioner's client runs as on the node (ADR 0349 §4).
Provisioner string `json:"provisioner_account,omitempty"`
}
func (o NodeOwn) empty() bool {
return o.Adoption == nil && len(o.Identities) == 0 && len(o.Groups) == 0 && len(o.Libraries) == 0 && o.Provisioner == ""
}
// Composed is a composition (ADR 0539 clause 1): one module's resources composed for one node at one fingerprint, or,
// with Module and FP empty, the node's own resources and parts (Own). Each resource is a declaration resource as the
// node-engine reads it, saying its module; it is kept as JSON and written canonically (its keys sorted, its spacing
// dropped), so its digest (ComposedDigestOf) is a function of its content alone. Data says the module declares data,
// so the node agent backs it up before it moves (ADR 0429 §14). Filled names the files of the module the provisioner
// fills (ADR 0349 §5). A file over 64 KiB travels by digest and size (ByDigest), its content in the node's object
// bucket (ADR 0539 clause 4).
type Composed struct {
V int
Node string
Module string
FP string
Data bool
Resources []json.RawMessage
Filled []string
Own NodeOwn
// Plan is the module's whole provisioning plan as the node-engine hands it to the provisioner (templates, files,
// own secrets by name, items, collections, provisions; never a value), composed node-wide (novox/hq ADR 0593 §6,
// §8): the provisioner makes every plan from it. Nil when the module hands the provisioner nothing; only on a
// module's key. It is part of the encoding (SameComposed sees a plan change) but not of the digest
// (ComposedDigestOf): a have, a want and the apply name a composition by its resources.
Plan *catalogue.Plan
// Refused is why the module cannot be composed at this fingerprint on this node, the rest of the node composed
// (ADR 0593 §6): a refused composition holds no resource, filled file, data or plan, and has no digest
// (ComposedDigestOf refuses it); only on a module's key. The node keeps running what it ran of the module.
Refused string
}
// composedWire is a composition as it is written.
type composedWire struct {
V int `json:"v"`
Node string `json:"node"`
Module string `json:"module,omitempty"`
FP string `json:"fp,omitempty"`
Data bool `json:"data,omitempty"`
Resources []json.RawMessage `json:"resources"`
Filled []string `json:"filled_files,omitempty"`
Own *NodeOwn `json:"own,omitempty"`
Plan *catalogue.Plan `json:"plan,omitempty"`
Refused string `json:"refused,omitempty"`
// Signature is read and dropped: a composition is unsigned (ADR 0593 §6), and a value the ended controller signed
// still decodes. Nothing writes it.
Signature json.RawMessage `json:"signature,omitempty"`
}
// canonicalJSON is raw with its object keys sorted and its spacing dropped, its numbers kept as written.
func canonicalJSON(raw json.RawMessage) (json.RawMessage, error) {
var v any
d := json.NewDecoder(bytes.NewReader(raw))
d.UseNumber()
if err := d.Decode(&v); err != nil {
return nil, err
}
if _, err := d.Token(); err != io.EOF {
return nil, fmt.Errorf("more after the value")
}
return json.Marshal(v)
}
func (c Composed) check() error {
if _, err := HaveNodeKey(c.Node); err != nil {
return err
}
if c.Module == "" {
if c.FP != "" {
return fmt.Errorf("delivery: the node's own composition names no fingerprint, and this one names %s", c.FP)
}
// It carries the filled paths of its own files, those of a module with no fingerprint of the new form moved
// into it (issue 1945), and never a module's data.
if c.Data {
return fmt.Errorf("delivery: the node's own composition holds no module's data")
}
} else {
if _, err := ComposedKey(c.Node, c.Module, c.FP); err != nil {
return err
}
if !c.Own.empty() {
return fmt.Errorf("delivery: the composition of %s carries the node's own parts, which only <node>.node carries", c.Module)
}
}
if c.Module == "" && (c.Plan != nil || c.Refused != "") {
return fmt.Errorf("delivery: the node's own composition carries no plan and no refusal: only a module's does")
}
if c.Refused != "" && (len(c.Resources) > 0 || len(c.Filled) > 0 || c.Data || c.Plan != nil) {
return fmt.Errorf("delivery: the composition of %s on %s is refused and holds resources, filled files, data or a plan", c.Module, c.Node)
}
seen := map[string]bool{}
files := map[string][]*declaration.File{}
for _, raw := range c.Resources {
r, err := declaration.ParseResource(raw)
if err != nil {
return fmt.Errorf("delivery: the composition of %q on %s: %w", c.Module, c.Node, err)
}
if seen[r.Identity()] {
return fmt.Errorf("delivery: the composition of %q on %s holds the resource %q twice", c.Module, c.Node, r.Identity())
}
seen[r.Identity()] = true
if r.DeclaredModule() != c.Module {
return fmt.Errorf("delivery: the resource %q is of %q, and this is the composition of %q on %s",
r.Identity(), r.DeclaredModule(), c.Module, c.Node)
}
if f, ok := r.(*declaration.File); ok {
if f.TravelsByDigest() {
return fmt.Errorf("delivery: the file %q is over %d bytes and travels by digest (ByDigest), not whole",
f.ID, declaration.FileReferenceThreshold)
}
files[f.Path] = append(files[f.Path], f)
}
}
// A file the provisioner fills says no content, so a composition that does not say it is filled would have the
// node-engine write it empty (issue 1945, live on 2026-10-25): refused, naming it.
filledHere := make(map[string]bool, len(c.Filled))
for _, path := range c.Filled {
filledHere[path] = true
}
for path, fs := range files {
for _, f := range fs {
if f.OwnerAtRuntime == declaration.RuntimeOwned && !filledHere[path] && !saysContent(f) {
return fmt.Errorf("delivery: the file %q says no content and the composition of %q on %s does not say "+
"the provisioner fills %s: the node-engine would write it empty", f.ID, c.Module, c.Node, path)
}
}
}
for _, path := range c.Filled {
if len(files[path]) == 0 {
return fmt.Errorf("delivery: the composition of %q fills %q, and declares no file there", c.Module, path)
}
for _, f := range files[path] {
if f.OwnerAtRuntime == "" {
return fmt.Errorf("delivery: the file %q is filled by the provisioner and says no owner-at-runtime", f.ID)
}
}
}
return nil
}
// saysContent says a file names what to write: content, bytes, a digest or a sealed value. A file the provisioner fills
// names none of them.
func saysContent(f *declaration.File) bool {
return f.Content != "" || f.Bytes != "" || f.Digest != "" || f.Sealed != "" || f.VaultSealed != nil
}
// wire is c checked and made canonical, ready to marshal.
func (c Composed) wire() (composedWire, error) {
if err := c.check(); err != nil {
return composedWire{}, err
}
w := composedWire{V: Version, Node: c.Node, Module: c.Module, FP: c.FP, Data: c.Data, Filled: c.Filled,
Plan: c.Plan, Refused: c.Refused, Resources: make([]json.RawMessage, len(c.Resources))}
for i, r := range c.Resources {
canon, err := canonicalJSON(r)
if err != nil {
return composedWire{}, fmt.Errorf("delivery: a resource of %q is not JSON: %w", c.Module, err)
}
w.Resources[i] = canon
}
if !c.Own.empty() {
own := c.Own
w.Own = &own
}
return w, nil
}
// EncodeComposed is c as JSON at Version, each resource canonical. It refuses a composition whose node or fingerprint
// a key refuses, a resource its kind refuses (an action among them), a resource of another module, a module's
// composition carrying the node's own parts, a file over 64 KiB carried whole, and a filled path no file declares.
func EncodeComposed(c Composed) ([]byte, error) {
w, err := c.wire()
if err != nil {
return nil, err
}
return json.Marshal(w)
}
// DecodeComposed reads a composition; it refuses an unknown version (ErrUnknownVersion), a field it does not know, and
// what EncodeComposed refuses.
func DecodeComposed(data []byte) (Composed, error) {
if err := checkVersion(data); err != nil {
return Composed{}, err
}
var w composedWire
d := json.NewDecoder(bytes.NewReader(data))
d.DisallowUnknownFields()
if err := d.Decode(&w); err != nil {
return Composed{}, fmt.Errorf("delivery: a composition does not fit its shape: %w", err)
}
c := Composed{V: w.V, Node: w.Node, Module: w.Module, FP: w.FP, Data: w.Data, Resources: w.Resources,
Filled: w.Filled, Plan: w.Plan, Refused: w.Refused}
if w.Own != nil {
c.Own = *w.Own
}
if err := c.check(); err != nil {
return Composed{}, err
}
return c, nil
}
// ComposedDigestOf is a composition's digest, "sha256:<hex>", of its encoding without its plan. A want names it (ADR 0539
// clause 1), and the node's engine writes it beside the composition (ADR 0593 §6).
//
// A refused composition has no digest: it is refused, naming the module (ADR 0593 §6).
func ComposedDigestOf(c Composed) (string, error) {
if c.Refused != "" {
return "", fmt.Errorf("delivery: the composition of %s on %s is refused (%s) and has no digest", c.Module, c.Node, c.Refused)
}
c.Plan = nil
raw, err := EncodeComposed(c)
if err != nil {
return "", err
}
sum := sha256.Sum256(raw)
return "sha256:" + hex.EncodeToString(sum[:]), nil
}
// SameComposed says whether two compositions are the same value, their plans included, so the engine writes only when
// they differ.
func SameComposed(a, b Composed) bool {
ra, errA := EncodeComposed(a)
rb, errB := EncodeComposed(b)
return errA == nil && errB == nil && bytes.Equal(ra, rb)
}
// ByDigest is c with every file over 64 KiB given by its digest and size, and the content of each by digest, for the
// writer to put in the node's object bucket (declaration.FileBucketFor) under declaration.ObjectName before it writes
// the composition (ADR 0539 clause 4, ADR 0488 §3). A file that is sealed, filled with secrets or written into another
// stays as it is (declaration.File.TravelsByDigest).
func (c Composed) ByDigest() (Composed, map[string][]byte, error) {
out := c
out.Resources = make([]json.RawMessage, len(c.Resources))
objects := map[string][]byte{}
for i, raw := range c.Resources {
out.Resources[i] = raw
r, err := declaration.ParseResource(raw)
if err != nil {
return Composed{}, nil, err
}
f, ok := r.(*declaration.File)
if !ok || !f.TravelsByDigest() {
continue
}
ref, body, err := f.AsReference()
if err != nil {
return Composed{}, nil, err
}
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
return Composed{}, nil, err
}
delete(fields, "content")
delete(fields, "bytes")
fields["digest"], _ = json.Marshal(ref.Digest)
fields["size"], _ = json.Marshal(ref.Size)
rewritten, err := json.Marshal(fields)
if err != nil {
return Composed{}, nil, err
}
out.Resources[i] = rewritten
objects[ref.Digest] = body
}
return out, objects, nil
}
// SplitComposed splits a node's composed resources into its compositions, by key (ComposedKey, ComposedNodeKey): each
// resource into the composition of the module it says, under that module's fingerprint in fps, and a resource that
// says no module into the node's own, with own. Each filled path goes with the module whose file declares it. The order
// of the resources is kept within each composition. A resource of a module fps names no fingerprint for is refused,
// naming it. Data is the caller's to set, from the module's manifest.
func SplitComposed(node string, resources []json.RawMessage, fps map[string]string, filled []string, own NodeOwn) (map[string]Composed, error) {
nodeKey, err := ComposedNodeKey(node)
if err != nil {
return nil, err
}
out := map[string]Composed{nodeKey: {Node: node, Own: own, Resources: []json.RawMessage{}}}
pathOf := map[string]string{}
for _, raw := range resources {
var head struct {
ID string `json:"id"`
Type string `json:"type"`
Module string `json:"module"`
Path string `json:"path"`
}
if err := json.Unmarshal(raw, &head); err != nil {
return nil, fmt.Errorf("delivery: a resource of %s is not JSON: %w", node, err)
}
key := nodeKey
if head.Module != "" {
fp, ok := fps[head.Module]
if !ok {
return nil, fmt.Errorf("delivery: the resource %q is of %q, whose fingerprint on %s is not given", head.ID, head.Module, node)
}
if key, err = ComposedKey(node, head.Module, fp); err != nil {
return nil, err
}
if _, ok := out[key]; !ok {
out[key] = Composed{Node: node, Module: head.Module, FP: fp}
}
}
if head.Type == string(declaration.TypeFile) && head.Module != "" {
pathOf[head.Path] = key
}
c := out[key]
c.Resources = append(c.Resources, raw)
out[key] = c
}
for _, path := range filled {
key, ok := pathOf[path]
if !ok {
return nil, fmt.Errorf("delivery: the provisioner fills %q on %s, and no module's file declares it", path, node)
}
c := out[key]
c.Filled = append(c.Filled, path)
out[key] = c
}
return out, nil
}
-364
View File
@@ -1,364 +0,0 @@
package delivery
import (
"bytes"
"compress/gzip"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"sort"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// capturedDeclaration is a node's declaration as the controller's `plan` verb answered it on 2026-10-24 at 16:19
// (testdata/declaration-<node>.json.gz; the resources carrying a sealed value or a secret taken out, and the filled
// files that named them), never typed by hand.
type capturedDeclaration struct {
Builds map[string]string `json:"builds"`
Resources []json.RawMessage `json:"resources"`
Filled []string `json:"filled_files"`
Provisioner string `json:"provisioner_account"`
}
func readCaptured(t *testing.T, node string) capturedDeclaration {
t.Helper()
f, err := os.Open("testdata/declaration-" + node + ".json.gz")
if err != nil {
t.Fatal(err)
}
defer f.Close()
z, err := gzip.NewReader(f)
if err != nil {
t.Fatal(err)
}
raw, err := io.ReadAll(z)
if err != nil {
t.Fatal(err)
}
var c capturedDeclaration
if err := json.Unmarshal(raw, &c); err != nil {
t.Fatal(err)
}
return c
}
// fpOfBuild stands for each module's fingerprint in a captured declaration, which names builds, not fingerprints.
func fpOfBuild(build string) string {
var b strings.Builder
for b.Len() < 64 {
b.WriteString(fmt.Sprintf("%x", []byte(build)))
}
return SourcePrefix2 + b.String()[:64]
}
func splitCaptured(t *testing.T, node string, c capturedDeclaration) map[string]Composed {
t.Helper()
fps := map[string]string{}
for m, b := range c.Builds {
fps[m] = fpOfBuild(b)
}
// mesh-wireguard's resources are composed with no build of it named; its fingerprint is the indexer's, as every
// module's is, and stood in for here by its name.
for _, r := range c.Resources {
var head struct {
Module string `json:"module"`
}
_ = json.Unmarshal(r, &head)
if _, ok := fps[head.Module]; head.Module != "" && !ok {
fps[head.Module] = fpOfBuild(head.Module)
}
}
split, err := SplitComposed(node, c.Resources, fps, c.Filled, NodeOwn{Provisioner: c.Provisioner})
if err != nil {
t.Fatalf("SplitComposed(%s): %v", node, err)
}
return split
}
func canonical(t *testing.T, raw json.RawMessage) string {
t.Helper()
var v any
d := json.NewDecoder(bytes.NewReader(raw))
d.UseNumber()
if err := d.Decode(&v); err != nil {
t.Fatal(err)
}
out, _ := json.Marshal(v)
return string(out)
}
// 1943/AC2: each module's slice of a captured declaration of each node, and the node's own resources, round-trip
// through the composition's encoding with the same digest; together they are the declaration's resources, each once,
// each in the composition of the module it says.
func TestCompositionsOfACapturedDeclaration(t *testing.T) {
t.Parallel()
for _, node := range []string{"ace", "g14"} {
t.Run(node, func(t *testing.T) {
t.Parallel()
oneCapturedDeclaration(t, node)
})
}
}
func oneCapturedDeclaration(t *testing.T, node string) {
{
c := readCaptured(t, node)
split := splitCaptured(t, node, c)
modules := map[string]bool{}
for m := range c.Builds {
modules[m] = true
}
var union []string
for key, comp := range split {
n, module, fp, isNode, err := ParseComposedKey(key)
if err != nil || n != node || comp.Node != node || comp.Module != module || comp.FP != fp {
t.Fatalf("%s: key %q holds %s/%s/%s (%v)", node, key, comp.Node, comp.Module, comp.FP, err)
}
if !isNode && !modules[module] && module != "mesh-wireguard" {
t.Errorf("%s: a composition of %s, which the declaration names no build of", node, module)
}
raw, err := EncodeComposed(comp)
if err != nil {
t.Fatalf("%s: EncodeComposed(%s): %v", node, key, err)
}
back, err := DecodeComposed(raw)
if err != nil {
t.Fatalf("%s: DecodeComposed(%s): %v", node, key, err)
}
if !SameComposed(back, comp) {
t.Errorf("%s: %s did not round-trip", node, key)
}
// SameComposed encodes both, so the digest of what was read back, a hash of its encoding, is the one written.
if d, err := ComposedDigestOf(back); err != nil || !digestForm.MatchString(d) {
t.Errorf("%s: %s digests %q (%v)", node, key, d, err)
}
for _, r := range back.Resources {
var head struct {
Module string `json:"module"`
}
_ = json.Unmarshal(r, &head)
if head.Module != module {
t.Errorf("%s: %s holds a resource of %q", node, key, head.Module)
}
union = append(union, canonical(t, r))
}
}
var want []string
for _, r := range c.Resources {
want = append(want, canonical(t, r))
}
sort.Strings(union)
sort.Strings(want)
if strings.Join(union, "\n") != strings.Join(want, "\n") {
t.Errorf("%s: the compositions hold %d resources, the declaration %d, or they differ", node, len(union), len(want))
}
}
}
// 1943/AC2: one changed byte of a resource is another digest; the same composition written with its keys in another
// order is the same digest.
func TestComposedDigestFollowsTheContent(t *testing.T) {
c := readCaptured(t, "g14")
split := splitCaptured(t, "g14", c)
var comp Composed
for _, v := range split {
if v.Module != "" && len(v.Resources) > 1 {
comp = v
break
}
}
d1, _ := ComposedDigestOf(comp)
var m map[string]json.RawMessage
_ = json.Unmarshal(comp.Resources[0], &m)
var keys []string
for k := range m {
keys = append(keys, k)
}
sort.Sort(sort.Reverse(sort.StringSlice(keys)))
var parts []string
for _, k := range keys {
parts = append(parts, fmt.Sprintf("%q : %s", k, m[k]))
}
reordered := json.RawMessage("{ " + strings.Join(parts, ",\n ") + " }")
shuffled := comp
shuffled.Resources = append([]json.RawMessage{reordered}, comp.Resources[1:]...)
if d2, _ := ComposedDigestOf(shuffled); d2 != d1 {
t.Errorf("the same resources in another order and spacing digest %s and %s", d1, d2)
}
changed := comp
changed.Resources = append([]json.RawMessage{json.RawMessage(strings.Replace(canonical(t, comp.Resources[0]), `"id":"`, `"id":"x`, 1))}, comp.Resources[1:]...)
if d3, _ := ComposedDigestOf(changed); d3 == d1 {
t.Errorf("a changed resource kept its digest")
}
revised := comp
revised.Data = !comp.Data
if d4, _ := ComposedDigestOf(revised); d4 == d1 {
t.Errorf("a changed data flag kept its digest")
}
}
// 1943/AC2: a file over 64 KiB is carried by digest and size, its content handed back to be put in the node's object
// bucket; a composition that carries such a file whole is refused.
func TestAComposedFileOver64KiBIsCarriedByDigest(t *testing.T) {
c := readCaptured(t, "g14")
split := splitCaptured(t, "g14", c)
var comp Composed
var at int
for _, v := range split {
for i, r := range v.Resources {
var f map[string]any
_ = json.Unmarshal(r, &f)
if f["type"] == "file" && v.Module != "" && f["bytes"] == nil && f["into"] == nil {
comp, at = v, i
break
}
}
if comp.Module != "" {
break
}
}
var f map[string]any
_ = json.Unmarshal(comp.Resources[at], &f)
// Grown past the threshold from a captured file: no captured file of this node is over 64 KiB.
body := strings.Repeat("x", declaration.FileReferenceThreshold+1)
f["content"] = body
big, _ := json.Marshal(f)
comp.Resources = append([]json.RawMessage{}, comp.Resources...)
comp.Resources[at] = big
if _, err := EncodeComposed(comp); err == nil || !strings.Contains(err.Error(), "by digest") {
t.Fatalf("a file over 64 KiB carried whole = %v, want refused", err)
}
byDigest, objects, err := comp.ByDigest()
if err != nil {
t.Fatal(err)
}
var ref map[string]any
_ = json.Unmarshal(byDigest.Resources[at], &ref)
digest := declaration.DigestOf([]byte(body))
if ref["digest"] != digest || ref["size"] != float64(len(body)) || ref["content"] != nil {
t.Errorf("the file became %v, want digest %s and size %d", ref, digest, len(body))
}
if string(objects[digest]) != body || len(objects) != 1 {
t.Errorf("the objects handed back are %d, want the file's content under its digest", len(objects))
}
if _, err := EncodeComposed(byDigest); err != nil {
t.Errorf("the composition by digest is refused: %v", err)
}
}
// 1943/AC2: an unknown version is refused as one; a module's composition carrying the node's own parts, a resource of
// another module, a resource its kind refuses, or no fingerprint, is refused, naming why.
func TestComposedRefusals(t *testing.T) {
fp := fp2of('a')
file := json.RawMessage(`{"id":"m.f","type":"file","module":"m","path":"/etc/m","content":"x"}`)
ok := Composed{Node: "ace", Module: "m", FP: fp, Resources: []json.RawMessage{file}}
raw, err := EncodeComposed(ok)
if err != nil {
t.Fatal(err)
}
if _, err := DecodeComposed(bytes.Replace(raw, []byte(`"v":1`), []byte(`"v":2`), 1)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("version 2 = %v, want ErrUnknownVersion", err)
}
refusals := []struct {
c Composed
why string
}{
{Composed{Node: "ace", Module: "m", Resources: []json.RawMessage{file}}, "fingerprint"},
{Composed{Node: "", Module: "m", FP: fp, Resources: []json.RawMessage{file}}, "node name is empty"},
{Composed{Node: "ace", Module: "other", FP: fp, Resources: []json.RawMessage{file}}, `of "m"`},
{Composed{Node: "ace", Module: "m", FP: fp, Resources: []json.RawMessage{file}, Own: NodeOwn{Provisioner: "p"}}, "node's own"},
{Composed{Node: "ace", FP: fp, Resources: []json.RawMessage{}}, "names no fingerprint"},
{Composed{Node: "ace", Resources: []json.RawMessage{file}}, `of "m"`},
{Composed{Node: "ace", Module: "m", FP: fp, Resources: []json.RawMessage{json.RawMessage(`{"id":"m.f","type":"file","module":"m","path":"/etc/m","content":"x","image":"y"}`)}}, `"image"`},
{Composed{Node: "ace", Module: "m", FP: fp, Resources: []json.RawMessage{json.RawMessage(`{"id":"m.a","type":"action","module":"m"}`)}}, "action"},
}
for _, r := range refusals {
if _, err := EncodeComposed(r.c); err == nil || !strings.Contains(err.Error(), r.why) {
t.Errorf("EncodeComposed(%s/%s) = %v, want a refusal saying %q", r.c.Node, r.c.Module, err, r.why)
}
}
}
// 1943/AC2 (issue 1150): a module whose name holds a dot keys its have, its want and its composition escaped, and the
// parsers give its name back; a name with a wildcard, whitespace or an escape of its own is still refused.
func TestADottedModuleKeysEscaped(t *testing.T) {
fp := fp2of('f')
have, err := HaveKey("ace", "novox.be")
if err != nil || have != "ace.novox=2ebe" {
t.Fatalf("HaveKey = %q, %v", have, err)
}
if n, m, isNode, err := ParseHaveKey(have); err != nil || n != "ace" || m != "novox.be" || isNode {
t.Errorf("ParseHaveKey(%q) = %q, %q, %v, %v", have, n, m, isNode, err)
}
want, err := WantKey("ace", "novox.be")
if err != nil || want != "want.ace.novox=2ebe" {
t.Fatalf("WantKey = %q, %v", want, err)
}
if n, m, err := ParseWantKey(want); err != nil || n != "ace" || m != "novox.be" {
t.Errorf("ParseWantKey(%q) = %q, %q, %v", want, n, m, err)
}
comp, err := ComposedKey("ace", "novox.be", fp)
if err != nil || comp != "ace.novox=2ebe."+strings.Repeat("f", 64) {
t.Fatalf("ComposedKey = %q, %v", comp, err)
}
if n, m, back, _, err := ParseComposedKey(comp); err != nil || n != "ace" || m != "novox.be" || back != fp {
t.Errorf("ParseComposedKey(%q) = %q, %q, %q, %v", comp, n, m, back, err)
}
for _, bad := range []string{"m*", "m x", "m=2e", "m>", ".m", "m.", "a..b"} {
if _, err := HaveKey("ace", bad); err == nil {
t.Errorf("HaveKey(ace, %q) accepted", bad)
}
}
for _, bad := range []string{"ace.m=2", "ace.m=2E", "ace.m=41", "ace.=6eode"} {
if _, _, _, err := ParseHaveKey(bad); err == nil {
t.Errorf("ParseHaveKey(%q) accepted", bad)
}
}
}
// The node's own composition carries the filled paths of the files moved into it, the files of a module with no
// fingerprint of the new form: the provisioner fills them all the same (issue 1945, live on 2026-10-25: the paths were
// dropped and the node-engine wrote those files empty). It still holds no module's data.
func TestTheNodesOwnCompositionCarriesTheFilledPathsOfItsFiles(t *testing.T) {
filled := json.RawMessage(`{"id":"power.unit","type":"file","path":"/etc/systemd/system/p.service","mode":"0644","owner-at-runtime":"owned"}`)
own := Composed{Node: "ace", Resources: []json.RawMessage{filled}, Filled: []string{"/etc/systemd/system/p.service"},
Own: NodeOwn{Provisioner: "mesh-provisioner-client"}}
raw, err := EncodeComposed(own)
if err != nil {
t.Fatalf("the node's own composition with a filled file: %v", err)
}
back, err := DecodeComposed(raw)
if err != nil || len(back.Filled) != 1 {
t.Fatalf("read back: %v %v", back.Filled, err)
}
if _, err := EncodeComposed(Composed{Node: "ace", Data: true, Resources: []json.RawMessage{filled},
Filled: own.Filled}); err == nil {
t.Error("the node's own composition was let hold a module's data")
}
}
// A file the provisioner fills says no content: in a composition that does not say it is filled, the node-engine would
// write it empty. Such a composition is refused, in the node's own as in a module's (issue 1945).
func TestAFileWithNoContentThatIsNotFilledIsRefused(t *testing.T) {
fp := fp2of('a')
for _, c := range []Composed{
{Node: "ace", Resources: []json.RawMessage{json.RawMessage(
`{"id":"power.unit","type":"file","path":"/etc/p","mode":"0644","owner-at-runtime":"owned"}`)}},
{Node: "ace", Module: "m", FP: fp, Resources: []json.RawMessage{json.RawMessage(
`{"id":"m.f","type":"file","module":"m","path":"/etc/m","mode":"0644","owner-at-runtime":"owned"}`)}},
} {
if _, err := EncodeComposed(c); err == nil || !strings.Contains(err.Error(), "empty") {
t.Errorf("%s/%s: %v, want a refusal saying it would be written empty", c.Node, c.Module, err)
}
}
// A file that says its content, even an empty one, is written as it says.
if _, err := EncodeComposed(Composed{Node: "ace", Resources: []json.RawMessage{json.RawMessage(
`{"id":"x.f","type":"file","path":"/etc/x","mode":"0644","content":""}`)}}); err != nil {
t.Errorf("a file that says it is empty: %v", err)
}
}
-132
View File
@@ -1,132 +0,0 @@
package delivery
import (
"encoding/json"
"fmt"
"strings"
"time"
)
// **The control records' layouts, out of the controller** (novox/hq ADR 0565 clauses 2, 3 and 6; issue 1953 L5-S2): what
// the indexer keeps of the catalogue's predefined list and of each open pull request's config declarations, which
// mesh-config reads in place of the controller's mirrors and `pull-declares`; and each node's declared packages, which
// its engine writes in place of the controller's mirror.
// The indexer's two states (clause 2).
const (
// IndexerPredefinedState holds one key, PredefinedListKey: the catalogue's predefined.json at the newest merge taken.
IndexerPredefinedState = "predefined"
PredefinedListKey = "list"
// IndexerConfigCandidatesState holds one key per open pull request (ConfigCandidateKey).
IndexerConfigCandidatesState = "config-candidates"
)
// PredefinedKept is the indexer's key `predefined.list`: the list whole, as predefined.json holds it, with the merge it
// was read at. A merge older than the one the key names never replaces it (ADR 0348 §7).
type PredefinedKept struct {
V int `json:"v"`
Commit string `json:"commit"`
At time.Time `json:"at"`
List json.RawMessage `json:"list"`
}
// ConfigDeclaration is one config key a module's manifest declares, in the layout of the controller's declared-config
// mirror (mesh-controller internal/inventory DeclaredMirrored) unchanged, so mesh-config reads the indexer's as it read
// the mirror's.
type ConfigDeclaration struct {
Kind string `json:"kind"`
Default json.RawMessage `json:"default"`
Why string `json:"why"`
Type string `json:"type,omitempty"`
Description string `json:"description,omitempty"`
Enum json.RawMessage `json:"enum,omitempty"`
Minimum *float64 `json:"minimum,omitempty"`
Maximum *float64 `json:"maximum,omitempty"`
Pattern string `json:"pattern,omitempty"`
Trusted bool `json:"trusted,omitempty"`
Predefined string `json:"predefined,omitempty"`
Required bool `json:"required,omitempty"`
Runtime bool `json:"runtime,omitempty"`
}
// ConfigCandidate is one open pull request's key of `config-candidates`: its head, and per module the config
// declarations the head's manifests add or change against the trunk's, an empty set when none. Written at each new
// head and each trunk merge that changes a module it names; deleted when the pull request merges or closes.
type ConfigCandidate struct {
V int `json:"v"`
Pull string `json:"pull"`
Head string `json:"head"`
Declares map[string]map[string]ConfigDeclaration `json:"declares"`
At time.Time `json:"at"`
}
// ConfigCandidateKey is a pull request's key, `owner/repository#number` with `/` and `#` as dots, the form mesh-config
// keeps a pull request's values under.
func ConfigCandidateKey(pull string) (string, error) {
owner, rest, ok := strings.Cut(strings.TrimSpace(pull), "/")
repo, number, ok2 := strings.Cut(rest, "#")
if !ok || !ok2 || owner == "" || repo == "" || number == "" || strings.ContainsAny(owner+repo+number, ".*> ") {
return "", fmt.Errorf("delivery: %q is not a pull request, owner/repository#number", pull)
}
return owner + "." + repo + "." + number, nil
}
// The node-engine's packages (clause 6): its state `packages`, one key, the node's name, written by that node's engine
// alone.
const EnginePackagesState = "packages"
// PackageDeclared is one package a node's modules declare, the controller's declared-packages mirror's entry unchanged
// (mesh-controller internal/inventory PackageDeclared): its name, the modules that declare it that way, sorted, and
// whether they declare it absent.
type PackageDeclared struct {
Package string `json:"package"`
Modules []string `json:"modules"`
Absent bool `json:"absent"`
}
// DeclaredPackages is one node's key of `packages`, the mirror's value unchanged: its packages sorted by name, absent
// after present. Fixed fields and no time, so the same declaration is the same bytes (ADR 0289 §1).
type DeclaredPackages struct {
Node string `json:"node"`
Packages []PackageDeclared `json:"packages"`
}
// mesh-config's state of the approved values held for a pull request (novox/hq ADR 0565 clause 5): read by the checker
// and the assigner's composition check, which lay a pull request's own held values over its modules' values.
const (
MeshConfigKeptState = "kept"
MeshConfigKeptRead = "mesh-config." + MeshConfigKeptState
)
// KeptValue is one approved held value, mesh-config's layout unchanged (modules/mesh-config pulls.go): kept apart from
// the values mesh-config resolves, laid only over the composition of its own pull request at its head.
type KeptValue struct {
Pull string `json:"pull"`
Head string `json:"head"`
Digest string `json:"declaration"`
Node string `json:"node"`
Module string `json:"module"`
Key string `json:"key"`
// Value is the value held, as the operator approved it.
Value json.RawMessage `json:"value"`
By string `json:"by"`
Via string `json:"via"`
Why string `json:"why"`
// Approved is who approved it, as the proposal kept it.
Approved string `json:"approved"`
Proposal string `json:"proposal"`
At time.Time `json:"at"`
}
// HeldValueKey is a held value's key in `kept`: the pull request's key (ConfigCandidateKey's form), the node, the module and
// the config key, joined by dots, as mesh-config writes it.
func HeldValueKey(pull, node, module, key string) (string, error) {
p, err := ConfigCandidateKey(pull)
if err != nil {
return "", err
}
return p + "." + node + "." + module + "." + key, nil
}
// The assigner's state of a person's upgrade policy, one key per module (clause 7; catalogue.PersonPolicy).
const AssignerPoliciesState = "policies"
-192
View File
@@ -1,192 +0,0 @@
package delivery
import (
"encoding/json"
"strings"
"testing"
"time"
)
// ADR 0565 clauses 2 and 6 (issue 1953 L5-S2): the control records' layouts read what their writers write. The
// packages value is the controller's declared-packages mirror's, as its writer's own test pins it (mesh-controller
// internal/inventory/declared_packages_test.go at 0fbb5ef3); the config declaration is its declared-config mirror's
// (DeclaredMirrored); a pull request's key is mesh-config's form.
func TestTheControlRecordsReadTheirWritersValues(t *testing.T) {
for _, c := range []struct {
name string
raw string
into any
}{
{"declared packages", `{"node":"one","packages":[{"package":"curl","modules":["backup","web"],"absent":false},` +
`{"package":"nano","modules":["web"],"absent":true}]}`, &DeclaredPackages{}},
{"a config candidate", `{"v":1,"pull":"novox/mesh-catalog#42","head":"abc","declares":{"mail":{"quota":{"kind":"config",` +
`"default":10,"why":"a quota","type":"integer","minimum":1}}},"at":"2026-10-29T12:00:00Z"}`, &ConfigCandidate{}},
{"a held value", `{"pull":"novox/mesh-catalog#42","head":"abc","declaration":"sha256:d","node":"novox","module":"mail",` +
`"key":"quota","value":20,"by":"person.jochen","via":"telegram","why":"more room","approved":"the operator",` +
`"proposal":"p-1","at":"2026-10-29T12:00:00Z"}`, &KeptValue{}},
// A node's health report as mesh-host 65fbc1e6 encodes it (its test's look, struct order kept).
{"a node's health report", `{"v":1,"node":"anchor","network":{"state":"unhealthy","since":"2026-10-29T20:00:00Z",` +
`"parts":[{"part":"resolv-conf","state":"unhealthy","writer":"NetworkManager","owner":"dnsmasq","since":"2026-10-29T20:00:00Z"}]},` +
`"peers":[{"key":"A=","addresses":["10.10.0.2/32"],"seen":false},{"key":"B=","seen":true}],` +
`"units":{"state":"degraded","failed":[{"unit":"cups.service","scope":"system","since":"2026-10-29T20:00:00Z"}]},` +
`"relogin":[{"account":"jochen","module":"brother","resource":"brother.scanner","reason":"relogin needed: the session began before scanner","since":"2026-10-29T20:00:00Z"}],` +
`"found":[{"module":"photos","resource":"photos.library","path":"/srv/photos","reason":"used as found: owned by root","since":"2026-10-29T20:00:00Z"}],` +
`"at":"2026-10-29T20:00:00Z"}`, &NodeHealth{}},
// The engine's verdict on an account declared never to become root (ADR 0266), and its contract.
{"a node's health report, the agent account judged", `{"v":1,"node":"anchor","root":[{"account":"agent",` +
`"module":"claude-code","resource":"claude-code.agent","state":"healthy","since":"2026-10-29T20:00:00Z"}],` +
`"contract":13,"at":"2026-10-29T20:00:00Z"}`, &NodeHealth{}},
// A part pointing at another machine names it (issue 774): the hub's tunnel, held at the hub's machine.
{"a node's health report, a part pointing at the hub", `{"v":1,"node":"anchor","network":{"state":"unhealthy",` +
`"since":"2026-10-29T20:00:00Z","parts":[{"part":"tunnel","state":"unhealthy","reason":"the tunnel to the hub has not handshaken for over five minutes",` +
`"toward":["hub"],"targets":["novox"],"since":"2026-10-29T20:00:00Z"}]},"at":"2026-10-29T20:00:00Z"}`, &NodeHealth{}},
// A node fallen back to its old login (ADR 0583 §6), which the watchman raises as node-account-fallback.
{"a node's health report, fallen back to its old login", `{"v":1,"node":"anchor","fallback":{"node":"anchor",` +
`"from":"node.anchor--b","why":"the new login's test-dial was refused","at":"2026-11-02T01:00:00Z"},` +
`"at":"2026-11-02T01:00:00Z"}`, &NodeHealth{}},
{"the predefined list", `{"v":1,"commit":"abc","at":"2026-10-29T12:00:00Z","list":{"domain":{"type":"string"}}}`, &PredefinedKept{}},
} {
if err := json.Unmarshal([]byte(c.raw), c.into); err != nil {
t.Fatalf("%s: %v", c.name, err)
}
again, _ := json.Marshal(c.into)
var want, got any
_ = json.Unmarshal([]byte(c.raw), &want)
_ = json.Unmarshal(again, &got)
if string(jsonOf(want)) != string(jsonOf(got)) || ((c.name == "declared packages" || strings.HasPrefix(c.name, "a node's health report")) && string(again) != c.raw) {
t.Errorf("%s reads back as %s", c.name, again)
}
}
if a, b := (NodeHealth{Node: "anchor", At: time.Now()}), (NodeHealth{Node: "anchor"}); !SameNodeHealth(a, b) ||
SameNodeHealth(a, NodeHealth{Node: "anchor", Peers: []PeerSeen{{Key: "A="}}}) {
t.Error("two reports differing only in when they were written are not the same, or differing ones are")
}
if _, err := EncodeNodeHealth(NodeHealth{}); err == nil {
t.Error("a report naming no node was encoded")
}
if _, err := EncodeNodeHealth(NodeHealth{Node: "anchor", Fallback: &LoginFallback{Node: "ace", From: "node.anchor--b",
Why: "refused", At: time.Now()}}); err == nil {
t.Error("a fall back naming another node was encoded")
}
// ADR 0571 clause 4: who holds which seat, from placement; a node seat by its node.
held := SeatHoldersOf(map[string]Placement{
"ace": {Node: "ace", Holds: []Holding{{Claim: "mesh-dns-resolver", Scope: "mesh", Module: "dnsmasq"}, {Claim: "node-files", Scope: "node", Module: "mesh-files"}}},
"novox": {Node: "novox", Holds: []Holding{{Claim: "mesh-dns-resolver", Scope: "mesh", Module: "dnsmasq"}, {Claim: "mesh-broker", Scope: "mesh", Module: "nats"},
{Claim: "photo-gallery", Scope: "mesh", Module: "photos"}}},
})
if o := held["photo-gallery"]; !o.Outside || len(o.Holders) != 1 || held["mesh-broker"].Outside {
t.Errorf("a claim outside the set is %+v", o)
}
keys := map[string][]byte{}
for k, v := range held {
keys[k], _ = json.Marshal(v)
}
if h, err := HoldersOf(keys, "mesh-dns-resolver", ""); err != nil || len(h) != 2 || h[0].Node != "ace" {
t.Errorf("a replicated seat's holders are %v, %v", h, err)
}
if h, _ := HoldersOf(keys, "node-files", "ace"); len(h) != 1 || h[0].Module != "mesh-files" || len(held) != 4 {
t.Errorf("a node seat's holder is %v, keys %v", h, held)
}
// ADR 0571 clauses 4, 5: the modules and the machines, from the indexer's and the placement seat's keys.
if m, err := ModuleNames(map[string][]byte{"letta": nil, "gitea": nil}); err != nil || strings.Join(m, ",") != "gitea,letta" {
t.Errorf("the modules are %v, %v", m, err)
}
if _, err := ModuleNames(map[string][]byte{"a.b": nil}); err == nil {
t.Error("a key that is no module's was read as one")
}
if n := MachineNames(map[string][]byte{"placement.novox": nil, "placement.ace": nil, "ace.node": nil, "ace.app": nil, "g14.app": nil, "proposal.p1": nil}); strings.Join(n, ",") != "ace,novox" {
t.Errorf("the machines are %v", n)
}
if h, _ := HoldersOf(keys, "node-files", "novox"); h != nil {
t.Errorf("a node seat nobody holds there is held by %v", h)
}
// ADR 0571 clause 1: the builder's log, a start and an end, and the duration between them.
at := time.Date(2026, 10, 29, 22, 0, 0, 0, time.UTC)
start := BuildLogEntry{Event: BuildStartedEvent, Module: "letta", FP: "src2:a", Source: "abc", Repository: "novox/mesh-catalog",
Directory: "modules/letta", Commit: "abc123", Node: "ace", At: at}
end := start
end.Event, end.At, end.Outcome, end.Manifest, end.Against = BuildEndedEvent, at.Add(3*time.Minute), "done", "sha256:m", []string{"artifact-store://base/image@sha256:b"}
end.Made = []BuiltArtifact{{Name: "image", Kind: "image", Digest: "sha256:1", Reference: "artifact-store://letta/image@sha256:1"}}
for _, e := range []BuildLogEntry{start, end} {
raw, err := EncodeBuildLogEntry(e)
back, derr := DecodeBuildLogEntry(raw)
if err != nil || derr != nil || back.Event != e.Event || back.Outcome != e.Outcome || back.Directory != "modules/letta" ||
back.Repository != "novox/mesh-catalog" || back.Commit != "abc123" {
t.Errorf("%s read back as %+v, %v %v", e.Event, back, err, derr)
}
}
if d, ok := BuildDuration(start, end); !ok || d != 3*time.Minute {
t.Errorf("the build took %s, %v", d, ok)
}
older := end
older.At, older.FP = at.Add(-time.Hour), "src2:b"
other := end
other.Module = "mesh-catalog"
if n := NewestPerModule([]BuildLogEntry{older, end, other}); n["letta"].FP != "src2:a" || len(n) != 2 {
t.Errorf("each module's newest is %v", n)
}
if e := EndedSince([]BuildLogEntry{end, start, older}, at.Add(-2*time.Hour)); len(e) != 2 || e[0].FP != "src2:b" {
t.Errorf("the ended builds since are %v", e)
}
if k, err := BuildLogKey("letta"); k != "letta" || err != nil || BuildLogRead != "mesh-delivery-builder.build-log" {
t.Errorf("the key is %q, %v", k, err)
}
if _, err := BuildLogKey("a.b"); err == nil {
t.Error("a module with a dot was taken as a key")
}
for _, edit := range []func(*BuildLogEntry){func(e *BuildLogEntry) { e.Outcome = "" }, func(e *BuildLogEntry) { e.Commit = "" },
func(e *BuildLogEntry) { e.Event, e.Outcome, e.Against = BuildStartedEvent, "", nil }} {
bad := end
edit(&bad)
if _, err := EncodeBuildLogEntry(bad); err == nil {
t.Errorf("%+v was encoded", bad)
}
}
if k, err := HeldValueKey("novox/mesh-catalog#42", "novox", "mail", "quota"); k != "novox.mesh-catalog.42.novox.mail.quota" || err != nil {
t.Errorf("a held value's key is %q, %v", k, err)
}
for pull, want := range map[string]string{"novox/mesh-catalog#42": "novox.mesh-catalog.42", "x#1": "", "a/b": "", "a/b.c#1": ""} {
if got, err := ConfigCandidateKey(pull); got != want || (want == "") != (err != nil) {
t.Errorf("%s keyed %q, %v", pull, got, err)
}
}
// ADR 0565 §6: the engine works out its node's key of `packages` from the compositions it composed, the mirror's bytes
// for the same resources (mesh-controller internal/inventory/declared_packages_test.go aNodesPackages at 0fbb5ef3):
// a module's own composition, the node's own with a folded module's resource named by its id alone, and a file.
res := func(s ...string) []json.RawMessage {
var out []json.RawMessage
for _, r := range s {
out = append(out, json.RawMessage(r))
}
return out
}
comps := []Composed{
{Node: "one", Module: "web", Resources: res(`{"id":"pkg-curl","type":"package","package":"curl"}`,
`{"id":"pkg-nano","type":"package","package":"nano","absent":true}`, `{"id":"conf","type":"file","path":"/etc/web.conf"}`)},
{Node: "one", Resources: res(`{"id":"backup.pkg-curl","type":"package","package":"curl"}`)},
}
want := `{"node":"one","packages":[{"package":"curl","modules":["backup","web"],"absent":false},` +
`{"package":"nano","modules":["web"],"absent":true}]}`
got, err := DeclaredPackagesOf("one", comps)
if raw, _ := json.Marshal(got); err != nil || string(raw) != want {
t.Errorf("the node's packages are %s (%v)", raw, err)
}
if none, _ := DeclaredPackagesOf("one", nil); string(jsonOf(none)) != `{"node":"one","packages":[]}` || EnginePackagesRead != "mesh-host.packages" {
t.Errorf("a node declaring no package is %s", jsonOf(none))
}
// Their difference with the mirror: a package only one side holds, and one whose modules differ; none when they agree.
mirror := DeclaredPackages{Node: "one", Packages: []PackageDeclared{{Package: "curl", Modules: []string{"web"}},
{Package: "nano", Modules: []string{"web"}, Absent: true}, {Package: "vim", Modules: []string{"edit"}}}}
if d := DiffDeclaredPackages(got, mirror); len(d) != 2 || d[0].Package != "curl" || strings.Join(d[0].Engine, ",") != "backup,web" ||
d[1].Package != "vim" || len(d[1].Engine) != 0 || d[1].Mirror[0] != "edit" {
t.Errorf("the difference is %+v", d)
}
if d := DiffDeclaredPackages(got, got); d == nil || len(d) != 0 {
t.Errorf("one declaration differs from itself: %+v", d)
}
}
func jsonOf(v any) []byte {
b, _ := json.Marshal(v)
return b
}
-677
View File
@@ -1,677 +0,0 @@
// Package delivery is the one home of the keys and values the parts of delivery read and write on the bus
// (novox/hq ADR 0429 §2: the key layouts another repository reads live in mesh-sdk; research 063 section 01). The
// node-engine writes the have keys, the watchman of the repository mesh-delivery writes presence and pulse, the
// view reads all of them, and the controller grants them; every one imports this package, and nothing else defines
// these names.
//
// **Buckets** are named `<owner module>_<name>` as the broker names a module's state (provider.BucketOf): the
// node-engine's `have`, and the watchman's `presence` and `pulse`.
//
// **Keys.** A have key is `<node>.<module>`, and a node's own facts are `<node>.node`, so a module may not be named
// `node`. A presence key is `<node>` for a node and `part.<part>.<instance>` for a part's instance, so a node may
// not be named `part`. A pulse key is `<bucket>.<key>`: the full name of the bucket watched, then that bucket's key,
// which may hold dots, so it splits at the first dot. A name is refused when it is empty or holds a dot, a wildcard,
// whitespace or a character a bus key refuses, and the error names it; but a module's name may hold dots between its
// parts (`novox.be`), and keys with each dot escaped as EscapeName writes it (`novox=2ebe`, issue 1150).
//
// **Values** are JSON of under about 500 bytes, each carrying its version in `v`. A reader refuses a value whose
// version is missing or one it does not know, with an error that is ErrUnknownVersion (research 063 section 05
// question 9 item 7): a writer writes a new version only once every reader's have shows a version that reads it. A
// writer writes only when the new value differs from the one before (the Same functions).
//
// **Source, candidates, modules and the forge** (issue 1200; ADR 0437 §5, release 3): the gate's bucket `source` with
// its one key `main` (every repository's merged head and the group that moved it), the indexer's `candidates.<group>`
// and `modules.<module>` (a module's source fingerprint, or none and why), and the forge bridge's messages named
// `forge.<owner>.<name>.<kind>` (source.go).
//
// **Merges, asks, owed and answers** (issue 1201; ADR 0437 §6, §8): the gate's `merges.<group>`, its asks
// `merge.<group>.<heads hash>` (the router's ask whole), its owed keys `<repository>.<number>.<head>.<act>` that the
// forge bridge performs, and the teller's answer under the ask's own key (gate.go).
//
// **The source fingerprint** (issue 1203; ADR 0437 §5): one function of named inputs that the controller's builder and
// the indexer both call, so they agree byte for byte (fingerprint.go).
//
// **The release 4 layouts** (issues 1356 to 1359; ADR 0453 clauses 4, 6 and 16): the new fingerprint form (fingerprint2.go),
// the indexer's manifests by digest and the new fields of its modules and candidates (manifests.go, source.go), the
// builder's build keys (builder.go) and the checker's verdicts and kept verdicts (checker.go). A manifests value is the
// one exception to the size above: a manifest is up to about 21 KB, whole, byte for byte. The bus module's have on the
// node holding mesh-broker is the other: its user list's handing carries a sealed box of every loaded user's hash, some
// kilobytes (handed.go).
//
// Standard library only, and this module's own asks, declaration and seal packages (a composition holds declaration
// resources; the adoption handing is a sealed box, handed.go).
package delivery
import (
"bytes"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"maps"
"reflect"
"regexp"
"slices"
"strings"
"time"
)
// The buckets of delivery's parts. Each is written by one owner: every node's node-engine writes the have keys of
// its own node only, and the watchman writes presence and pulse.
const (
HaveBucket = "node-engine_have"
PresenceBucket = WatchmanModule + "_presence"
PulseBucket = WatchmanModule + "_pulse"
)
// HaveHistory and PresenceHistory are the revisions the have and presence buckets keep per key (research 063
// section 01).
const (
HaveHistory = 20
PresenceHistory = 20
)
// PulseMoveBound is how long a move may take: a pulse key lives for this time minus the move's age, and the bucket
// expires it on its own (the server writes a delete marker), so a move that overruns it is seen to stall.
const PulseMoveBound = 120 * time.Second
// Version is the version of every value this package writes and the only one it reads.
const Version = 1
// ErrUnknownVersion is what a decode returns for a value whose version is missing or not Version. Test it with
// errors.Is; the message names the version seen and the one read.
var ErrUnknownVersion = errors.New("delivery: unknown value version")
// Phase is how far a module's change has come on a node.
type Phase string
// The phases of a module's change on a node.
const (
PhaseBackingUp Phase = "backing-up"
PhaseApplying Phase = "applying"
PhaseChecking Phase = "checking"
PhaseRunning Phase = "running"
PhaseFailed Phase = "failed"
// PhaseRemoved is written by the node-engine, once, for a module its applied declaration no longer holds and
// for which it had written a have key; the key is never deleted (issue 1369, ADR 0408 §4), so a reader can tell
// that the module stopped.
PhaseRemoved Phase = "removed"
// PhaseHeld is a module whose backup before an apply failed: it keeps running what it has, nothing of the new
// composition is applied, and the node waits for the answer in its want (novox/hq ADR 0539 clause 16). Detail
// carries the backup's own words.
PhaseHeld Phase = "held"
// PhaseWaitingYes is a module of policy record whose composed move the node agent judged to recreate a container or
// move its data (novox/hq ADR 0547 §3): nothing of it is applied, Judgement says the judgement and Waits the digest
// of the composition waiting, and the assigner asks a person; the node applies it once its want's Yes names Waits.
PhaseWaitingYes Phase = "waiting-yes"
)
// Health is what the module's own check says.
type Health string
// The healths a have carries.
const (
HealthOK Health = "ok"
HealthFailed Health = "failed"
HealthWaits Health = "waits"
HealthUnknown Health = "unknown"
)
var nameOK = regexp.MustCompile(`^[-/_=a-zA-Z0-9]+$`)
// checkName refuses a name that is empty or holds a dot, a wildcard, whitespace or a character a bus key refuses.
func checkName(kind, name string) error {
if name == "" {
return fmt.Errorf("delivery: the %s name is empty", kind)
}
if !nameOK.MatchString(name) {
return fmt.Errorf("delivery: the %s name %q holds a dot, a wildcard, whitespace or a character a key refuses", kind, name)
}
return nil
}
// EscapeName is name as a key part: a character outside letters, digits, `-`, `/` and `_` is written `=` and its
// two lowercase hex digits per byte, and `=` itself is escaped, so any non-empty name keys and a name of allowed
// characters only keys as it is. UnescapeName is its inverse.
func EscapeName(name string) string {
var b strings.Builder
for i := 0; i < len(name); i++ {
c := name[i]
switch {
case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9', c == '-', c == '/', c == '_':
b.WriteByte(c)
default:
fmt.Fprintf(&b, "=%02x", c)
}
}
return b.String()
}
// UnescapeName is the inverse of EscapeName; it refuses a part with a bare `=` or any character EscapeName leaves
// escaped.
func UnescapeName(part string) (string, error) {
var out []byte
for i := 0; i < len(part); i++ {
c := part[i]
if c != '=' {
if !nameOK.MatchString(string(c)) {
return "", fmt.Errorf("delivery: %q holds %q, which a key part escapes", part, c)
}
out = append(out, c)
continue
}
if i+3 > len(part) {
return "", fmt.Errorf("delivery: %q ends in an incomplete escape", part)
}
v, err := hex.DecodeString(part[i+1 : i+3])
if err != nil || part[i+1:i+3] != strings.ToLower(part[i+1:i+3]) {
return "", fmt.Errorf("delivery: %q holds a malformed escape %q", part, part[i:i+3])
}
out = append(out, v[0])
i += 2
}
return string(out), nil
}
// HaveKey is the key, in HaveBucket, of a module's have on a node: `<node>.<module>`. A module named `node` is
// refused, since `<node>.node` is the node's own facts.
func HaveKey(node, module string) (string, error) {
if err := checkName("node", node); err != nil {
return "", err
}
part, err := moduleKeyPart(module)
if err != nil {
return "", err
}
if module == "node" {
return "", fmt.Errorf(`delivery: the module name "node" is refused: %s.node is the node's own key`, node)
}
return node + "." + part, nil
}
var moduleNameOK = regexp.MustCompile(`^[-/_a-zA-Z0-9]+(\.[-/_a-zA-Z0-9]+)*$`)
// moduleKeyPart is a module's name as a key part (issue 1150): a name may hold a dot (`novox.be`), which a key part
// may not, so a dot is escaped as EscapeName writes it (`novox=2ebe`); a name of letters, digits, `-`, `/` and `_` keys
// as it is. A name that is empty, or holds a wildcard, whitespace, `=`, or an empty part between dots, is refused.
func moduleKeyPart(module string) (string, error) {
if module == "" {
return "", fmt.Errorf("delivery: the module name is empty")
}
if !moduleNameOK.MatchString(module) {
return "", fmt.Errorf("delivery: the module name %q holds a wildcard, whitespace, '=', an empty part between dots or a character a key refuses", module)
}
return EscapeName(module), nil
}
// parseModuleKeyPart is the inverse of moduleKeyPart; it refuses a part that is not the one moduleKeyPart writes for
// its name (an escape of a character that keys as it is, an upper-case escape, an escape of a refused character).
func parseModuleKeyPart(part string) (string, error) {
if err := checkName("module", part); err != nil {
return "", err
}
module, err := UnescapeName(part)
if err != nil {
return "", err
}
if back, err := moduleKeyPart(module); err != nil || back != part {
return "", fmt.Errorf("delivery: the module part %q is not a module name as a key writes it", part)
}
return module, nil
}
// HaveNodeWord is the last token of a node's own facts' key in HaveBucket.
const HaveNodeWord = nodeWord
// HaveNodeKey is the key, in HaveBucket, of a node's own facts: `<node>.node`.
func HaveNodeKey(node string) (string, error) {
if err := checkName("node", node); err != nil {
return "", err
}
return node + "." + HaveNodeWord, nil
}
// ParseHaveKey is the inverse of HaveKey and HaveNodeKey: isNode is true for `<node>.node`, with module empty.
func ParseHaveKey(key string) (node, module string, isNode bool, err error) {
node, module, ok := strings.Cut(key, ".")
if !ok {
return "", "", false, fmt.Errorf("delivery: %q is not a have key, <node>.<module>", key)
}
if err := checkName("node", node); err != nil {
return "", "", false, fmt.Errorf("delivery: %q is not a have key: %w", key, err)
}
if module == "node" {
return node, "", true, nil
}
module, err = parseModuleKeyPart(module)
if err != nil {
return "", "", false, fmt.Errorf("delivery: %q is not a have key: %w", key, err)
}
return node, module, false, nil
}
// PresenceNodeKey is the key, in PresenceBucket, of a node: `<node>`. A node named `part` is refused.
func PresenceNodeKey(node string) (string, error) {
if err := checkName("node", node); err != nil {
return "", err
}
if node == "part" {
return "", errors.New(`delivery: the node name "part" is refused: part.<part>.<instance> is a part's key`)
}
return node, nil
}
// PresencePartKey is the key, in PresenceBucket, of a part's instance: `part.<part>.<instance>`.
func PresencePartKey(part, instance string) (string, error) {
if err := checkName("part", part); err != nil {
return "", err
}
if err := checkName("instance", instance); err != nil {
return "", err
}
return "part." + part + "." + instance, nil
}
// ParsePresenceKey is the inverse of PresenceNodeKey and PresencePartKey: for a node's key part and instance are
// empty, for a part's key node is.
func ParsePresenceKey(key string) (node, part, instance string, err error) {
if !strings.Contains(key, ".") {
node, err = PresenceNodeKey(key)
if err != nil {
return "", "", "", fmt.Errorf("delivery: %q is not a presence key: %w", key, err)
}
return node, "", "", nil
}
fields := strings.Split(key, ".")
if len(fields) != 3 || fields[0] != "part" {
return "", "", "", fmt.Errorf("delivery: %q is not a presence key, <node> or part.<part>.<instance>", key)
}
if _, err := PresencePartKey(fields[1], fields[2]); err != nil {
return "", "", "", fmt.Errorf("delivery: %q is not a presence key: %w", key, err)
}
return "", fields[1], fields[2], nil
}
// PulseKey is the key, in PulseBucket, of the move watched at key of bucket: `<bucket>.<key>`. The bucket is the
// full bucket name and holds no dot; the key may.
func PulseKey(bucket, key string) (string, error) {
if err := checkName("bucket", bucket); err != nil {
return "", err
}
if key == "" || strings.HasPrefix(key, ".") || strings.HasSuffix(key, ".") || strings.Contains(key, "..") {
return "", fmt.Errorf("delivery: the pulse key %q is empty or has an empty part", key)
}
for _, part := range strings.Split(key, ".") {
if err := checkName("key", part); err != nil {
return "", err
}
}
return bucket + "." + key, nil
}
// ParsePulseKey is the inverse of PulseKey, splitting at the first dot.
func ParsePulseKey(pulseKey string) (bucket, key string, err error) {
bucket, key, ok := strings.Cut(pulseKey, ".")
if !ok {
return "", "", fmt.Errorf("delivery: %q is not a pulse key, <bucket>.<key>", pulseKey)
}
if _, err := PulseKey(bucket, key); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a pulse key: %w", pulseKey, err)
}
return bucket, key, nil
}
// PulseTTL is the time to live of a pulse key for a move of the given age: PulseMoveBound minus the age. At or past
// the bound the move is stalled, with no time left and nothing to write: the watchman raises it at once.
func PulseTTL(moveAge time.Duration) (ttl time.Duration, stalled bool) {
if moveAge >= PulseMoveBound {
return 0, true
}
return PulseMoveBound - moveAge, false
}
// Have is a module's state on a node, written by that node's node-engine under HaveKey. FP is the module's
// fingerprint once the indexer exists (design 47 release 3); until then Build is the build the node was sent, and
// the view compares by build.
//
// Composed is the digest of the composition the node runs for the module (novox/hq ADR 0539 clause 9): the
// controller judges a move against it (clause 14), and the assigner reads it to know a want was applied.
type Have struct {
V int `json:"v"`
FP string `json:"fp,omitempty"`
Composed string `json:"composed,omitempty"`
Build string `json:"build,omitempty"`
Phase Phase `json:"phase"`
Health Health `json:"health"`
Detail string `json:"detail,omitempty"`
// UserList is what the node-engine of the node holding mesh-broker wrote of the bus's user list, in the have of the
// bus module (novox/hq ADR 0547 §7, §8). Nil on every other have.
UserList *UserListWritten `json:"user-list,omitempty"`
// Judgement and Waits are, in phase waiting-yes, the node agent's judgement of the move and the digest of the
// composition that waits for a person's yes (ADR 0547 §3); empty otherwise.
Judgement Judgement `json:"judgement,omitempty"`
Waits string `json:"waits,omitempty"`
// BuildAlone says, in phase waiting-yes, that the move changes the module's build alone (novox/hq ADR 0601 §1): the
// node composed the build it runs again with today's settings and secrets and got the composition it runs, so the
// release that carries the new build answers the yes. False when a setting or a secret changed too, or the node could
// not tell: the move is then asked of a person as before.
BuildAlone bool `json:"build-alone,omitempty"`
// Kept says, in words, what the node keeps of the module as it runs because its manifest does not yet say what shared
// code no longer infers (that it is the tool runner, that its own secrets are the core's credentials; novox/hq ADR 0601
// §4): the watchman raises a condition from it, cleared once the have says none. Empty in every other have.
Kept []string `json:"kept,omitempty"`
// Sealed is, for a module the node composes itself, each value its running composition carries sealed to the node's
// key (a secret, a credential, a vault box), as applied (ADR 0547 §3, issue 1126): what the node's composer reuses
// after a restart while it opens to the vault's value, so a value sealed again moves no composition. Openable by the
// node alone, as the vault hands them over the bus. Empty on every other have.
Sealed []string `json:"sealed,omitempty"`
// SealedAt is Sealed by slot (Seals: a field of a resource, issue 1126, C1): the composer reuses a box only at the
// slot that holds it, so a provider and a consumer of one value on one node each keep their own. Written in Sealed's
// place by a node-engine that knows it; Sealed is read only from a have written before.
SealedAt Seals `json:"sealed-at,omitempty"`
// PullWait is, on a failed have, the pull from a store its apply failed on and waits on that store's holder to settle
// for (design 51 *What a pull meets in the swap*, issue 1966 ENG-9): nil on every other have, and on a failure that
// waits on no holder.
PullWait *PullWait `json:"pull-wait,omitempty"`
}
// PullWait is the pull a failed apply met from a store: the image, the seat it came through and that seat's holder, whose
// settled the apply is tried again on, when the pull began, and when the apply was last tried again with what triggered
// it, in words (empty while it has not been).
type PullWait struct {
Image string `json:"image"`
Seat string `json:"seat,omitempty"`
Holder string `json:"holder"`
Began time.Time `json:"began"`
Retried string `json:"retried,omitempty"`
RetriedAt time.Time `json:"retried-at,omitzero"`
}
// samePullWait says whether two pull waits are the same, their times compared as instants.
func samePullWait(a, b *PullWait) bool {
if a == nil || b == nil {
return a == b
}
return a.Image == b.Image && a.Seat == b.Seat && a.Holder == b.Holder && a.Began.Equal(b.Began) &&
a.Retried == b.Retried && a.RetriedAt.Equal(b.RetriedAt)
}
// UserListWritten is the bus's user list as the node-engine of the bus node wrote it to the bus module's file, applied
// alone and with no want (novox/hq ADR 0547 §8): where it was taken from, what that source holds, and what the file
// holds now. The bus reloads the file itself when its digest changes (the module's entrypoint), so Digest is the list
// the bus loads. Wanted differs from Digest only while a write failed, and Error says why.
type UserListWritten struct {
// Source is the key the list was taken from: the grants part's `user-list` (GrantsUserListSource).
Source string `json:"source"`
Revision uint64 `json:"revision"`
// Wanted is the digest of the list the source holds, `sha256:<hex>`.
Wanted string `json:"wanted"`
// Digest is the digest of the file on the machine after the write, `sha256:<hex>`; empty when there is none.
Digest string `json:"digest,omitempty"`
Users int `json:"users"`
At time.Time `json:"at"`
Error string `json:"error,omitempty"`
// Follows is which list the node follows now (FollowGrants, novox/hq ADR 0547 clause 8), empty from a node-engine
// before the switch.
Follows string `json:"follows,omitempty"`
// Key is the bus node's sealing key, its public half as SPKI PEM: the key the grants part seals its list to
// (clause 7). Public, as the vault holds it.
Key string `json:"key,omitempty"`
// Unwatched says why the bus node does not watch the grants part's key now (its watch refused or ended, and tried
// again), empty while it does or where it need not (ADR 0547 clause 8): it then keeps the last list it followed.
Unwatched string `json:"unwatched,omitempty"`
// Handed is the hashes of the users of the list on the machine, sealed to the vault, for its adoption of the
// accounts it holds no value for (ADR 0547 clause 9, handed.go); nil while the vault's key is not known.
Handed *AccountsHanded `json:"handed,omitempty"`
}
// HaveNode is a node's own facts, written under HaveNodeKey. Each field is the node-engine's own JSON, carried whole:
// this package names the facts and does not type them.
//
// Composed is the digest of the node's own composition it runs (ADR 0539 clause 9).
type HaveNode struct {
V int `json:"v"`
Composed string `json:"composed,omitempty"`
Windows json.RawMessage `json:"windows,omitempty"`
Busy json.RawMessage `json:"busy,omitempty"`
Strays json.RawMessage `json:"strays,omitempty"`
Filters json.RawMessage `json:"filters,omitempty"`
Profile json.RawMessage `json:"profile,omitempty"`
// Record is what the machine observes of itself, typed, for every node and composing part to read (issue 1126,
// ADR 0555 §3): the node's record lives in its own have key, so it needs no bucket, writer or grant of its own.
Record *NodeRecord `json:"record,omitempty"`
// Sealed is, on a node composing its own, each value its own composition carries sealed to the node's key, as
// applied (Have.Sealed for the node's own, issue 1126): a value the vault seals again after a restart moves no
// composition, so a restart applies nothing again.
Sealed []string `json:"sealed,omitempty"`
// SealedAt is Sealed by slot (Have.SealedAt).
SealedAt Seals `json:"sealed-at,omitempty"`
// Candidates is the node's answer to each placement candidate naming it, by proposal id (PlacementCandidate): written by
// its shadow composer whenever an answer changes, removed when the candidate key goes. The node applies nothing of a
// candidate.
Candidates map[string]CandidateAnswer `json:"candidates,omitempty"`
// Composes is whose compositions the node applies: its own (ComposesOwn, placement places it), or none, empty, with
// ComposesWhy saying why in words. Written by the node agent at each start (issue 1126). An engine from before the
// controller ended may still say `controller` (composedByTheEndedController), which is read and never written.
Composes string `json:"composes,omitempty"`
ComposesWhy string `json:"composes-why,omitempty"`
}
// ComposesOwn is a node applying its own compositions (HaveNode.Composes).
const ComposesOwn = "own"
// composedByTheEndedController is what an engine from before the controller ended (ADR 0590) wrote in Composes while
// it applied the controller's compositions: still read, so its have is not refused, and written by no engine now.
const composedByTheEndedController = "controller"
// Presence is whether a node or a part's instance is online; the entry's own time says since when. The watchman
// writes it.
type Presence struct {
V int `json:"v"`
Online bool `json:"online"`
}
// Pulse is the value of a pulse key: the watched key mirrored, its revision, and who owes the next move (a node or
// a part instance). The watchman writes it with the time to live PulseTTL gives.
type Pulse struct {
V int `json:"v"`
Bucket string `json:"bucket"`
Key string `json:"key"`
Revision uint64 `json:"revision"`
Owes string `json:"owes,omitempty"`
}
func checkVersion(data []byte) error {
var probe struct {
V *int `json:"v"`
}
if err := json.Unmarshal(data, &probe); err != nil {
return fmt.Errorf("delivery: value is not JSON: %w", err)
}
if probe.V == nil {
return fmt.Errorf("%w: no version in the value, this reader reads %d", ErrUnknownVersion, Version)
}
if *probe.V != Version {
return fmt.Errorf("%w: the value is version %d, this reader reads %d", ErrUnknownVersion, *probe.V, Version)
}
return nil
}
func decode(data []byte, into any) error {
if err := checkVersion(data); err != nil {
return err
}
if err := json.Unmarshal(data, into); err != nil {
return fmt.Errorf("delivery: value does not fit its shape: %w", err)
}
return nil
}
func (h Have) check() error {
switch h.Phase {
case PhaseBackingUp, PhaseApplying, PhaseChecking, PhaseRunning, PhaseFailed, PhaseRemoved, PhaseHeld:
if h.Judgement != "" || h.Waits != "" || h.BuildAlone {
return fmt.Errorf("delivery: only a have waiting for a yes names a judgement, the composition waiting and whether it changes the build alone")
}
if h.PullWait != nil && h.Phase != PhaseFailed {
return fmt.Errorf("delivery: only a failed have names the pull it waits on, not one %s", h.Phase)
}
case PhaseWaitingYes:
if h.Judgement != JudgedRecreates && h.Judgement != JudgedMovesData && h.Judgement != JudgedCannot {
return fmt.Errorf("delivery: a have waiting for a yes names the judgement %q, not recreates, moves-data or cannot-judge", h.Judgement)
}
if err := checkDigest(h.Waits, "the composition a have waits a yes for"); err != nil {
return err
}
default:
return fmt.Errorf("delivery: phase %q is not backing-up, applying, checking, running, failed, removed, held or waiting-yes", h.Phase)
}
if w := h.PullWait; w != nil && (w.Image == "" || w.Holder == "") {
return fmt.Errorf("delivery: a have's pull wait names no image or no holder")
}
if h.Composed != "" {
if err := checkDigest(h.Composed, "the have's composition digest"); err != nil {
return err
}
}
switch h.Health {
case HealthOK, HealthFailed, HealthWaits, HealthUnknown:
default:
return fmt.Errorf("delivery: health %q is not ok, failed, waits or unknown", h.Health)
}
return nil
}
// EncodeHave is h as JSON at Version; it refuses a phase or health outside the lists.
func EncodeHave(h Have) ([]byte, error) {
if err := h.check(); err != nil {
return nil, err
}
h.V = Version
return json.Marshal(h)
}
// DecodeHave reads a have; it refuses an unknown version (ErrUnknownVersion) and a phase or health outside the lists.
func DecodeHave(data []byte) (Have, error) {
var h Have
if err := decode(data, &h); err != nil {
return Have{}, err
}
if err := h.check(); err != nil {
return Have{}, err
}
return h, nil
}
func (n HaveNode) check() error {
switch n.Composes {
case "", ComposesOwn, composedByTheEndedController:
default:
return fmt.Errorf("delivery: a node composes %q, not %s", n.Composes, ComposesOwn)
}
for group, a := range n.Candidates {
if group == "" {
return fmt.Errorf("delivery: a node's candidate answer names no group")
}
if err := a.check(); err != nil {
return fmt.Errorf("delivery: the answer to %s's candidate: %w", group, err)
}
}
return nil
}
// EncodeHaveNode is n as JSON at Version; it refuses a candidate answer CandidateAnswer's rules refuse.
func EncodeHaveNode(n HaveNode) ([]byte, error) {
if err := n.check(); err != nil {
return nil, err
}
n.V = Version
return json.Marshal(n)
}
// DecodeHaveNode reads a node's facts; it refuses an unknown version (ErrUnknownVersion) and a candidate answer that
// is not one.
func DecodeHaveNode(data []byte) (HaveNode, error) {
var n HaveNode
if err := decode(data, &n); err != nil {
return n, err
}
if err := n.check(); err != nil {
return HaveNode{}, err
}
return n, nil
}
// EncodePresence is p as JSON at Version.
func EncodePresence(p Presence) ([]byte, error) {
p.V = Version
return json.Marshal(p)
}
// DecodePresence reads a presence; it refuses an unknown version (ErrUnknownVersion).
func DecodePresence(data []byte) (Presence, error) {
var p Presence
err := decode(data, &p)
return p, err
}
// EncodePulse is p as JSON at Version.
func EncodePulse(p Pulse) ([]byte, error) {
p.V = Version
return json.Marshal(p)
}
// DecodePulse reads a pulse; it refuses an unknown version (ErrUnknownVersion).
func DecodePulse(data []byte) (Pulse, error) {
var p Pulse
err := decode(data, &p)
return p, err
}
// SameHave says whether two haves are the same value, so a writer writes only when they differ.
func SameHave(a, b Have) bool {
ua, ub := a.UserList, b.UserList
a.UserList, b.UserList = nil, nil
if !samePullWait(a.PullWait, b.PullWait) {
return false
}
a.PullWait, b.PullWait = nil, nil
if !slices.Equal(a.Sealed, b.Sealed) || !maps.Equal(a.SealedAt, b.SealedAt) {
return false
}
a.Sealed, b.Sealed = nil, nil
a.SealedAt, b.SealedAt = nil, nil
if !reflect.DeepEqual(a, b) || (ua == nil) != (ub == nil) {
return false
}
if ua == nil {
return true
}
x, y := *ua, *ub
x.At, y.At = time.Time{}, time.Time{}
x.Handed, y.Handed = nil, nil
return x == y && ua.At.Equal(ub.At) && sameHanded(ua.Handed, ub.Handed)
}
// SameHaveNode says whether two node facts are the same value, comparing each carried JSON without its spacing.
func SameHaveNode(a, b HaveNode) bool {
return a.V == b.V && a.Composed == b.Composed && sameJSON(a.Windows, b.Windows) && sameJSON(a.Busy, b.Busy) &&
sameJSON(a.Strays, b.Strays) && sameJSON(a.Filters, b.Filters) && sameJSON(a.Profile, b.Profile) &&
(a.Record == nil) == (b.Record == nil) && (a.Record == nil || SameNodeRecord(*a.Record, *b.Record)) &&
jsonEqual(a.Sealed, b.Sealed) && maps.Equal(a.SealedAt, b.SealedAt) && jsonEqual(a.Candidates, b.Candidates) && a.Composes == b.Composes &&
a.ComposesWhy == b.ComposesWhy
}
// SamePresence says whether two presences are the same value.
func SamePresence(a, b Presence) bool { return a == b }
// SamePulse says whether two pulses are the same value.
func SamePulse(a, b Pulse) bool { return a == b }
func sameJSON(a, b json.RawMessage) bool {
var ca, cb bytes.Buffer
if json.Compact(&ca, a) != nil || json.Compact(&cb, b) != nil {
return bytes.Equal(a, b)
}
return bytes.Equal(ca.Bytes(), cb.Bytes())
}
-542
View File
@@ -1,542 +0,0 @@
package delivery
import (
"bytes"
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
)
// 1131/AC1: the have key forms and parses back; bad names are refused, naming them.
func TestHaveKey(t *testing.T) {
t.Run("presence", testPresenceKey)
t.Run("pulse", testPulseKey)
t.Run("pulse ttl", testPulseTTL)
if HaveBucket != "node-engine_have" {
t.Fatalf("HaveBucket = %q", HaveBucket)
}
key, err := HaveKey("ace", "plex")
if err != nil || key != "ace.plex" {
t.Fatalf("HaveKey = %q, %v", key, err)
}
node, module, isNode, err := ParseHaveKey(key)
if err != nil || node != "ace" || module != "plex" || isNode {
t.Fatalf("ParseHaveKey = %q %q %v %v", node, module, isNode, err)
}
if got, err := HaveNodeKey("ace"); err != nil || got != "ace.node" {
t.Fatalf("HaveNodeKey = %q, %v", got, err)
}
if _, err := HaveNodeKey("a b"); err == nil {
t.Error("HaveNodeKey with a space accepted")
}
node, module, isNode, err = ParseHaveKey("ace.node")
if err != nil || node != "ace" || module != "" || !isNode {
t.Fatalf("ParseHaveKey node = %q %q %v %v", node, module, isNode, err)
}
for _, c := range []struct{ node, module, named string }{
{"ace", "node", "node"},
{"ace", "", "module"},
{"", "plex", "node"},
{"a.b", "plex", "a.b"},
{"ace", "p*", "p*"},
{"ace", "p>", "p>"},
{"ace", "p x", "p x"},
{"ace", "p\tx", "module"},
} {
_, err := HaveKey(c.node, c.module)
if err == nil || !strings.Contains(err.Error(), c.named) {
t.Errorf("HaveKey(%q, %q) = %v, want an error naming %q", c.node, c.module, err, c.named)
}
}
for _, bad := range []string{"", "ace", "ace.", ".plex", "ace.plex.x", "ace.p x"} {
if _, _, _, err := ParseHaveKey(bad); err == nil {
t.Errorf("ParseHaveKey(%q) accepted", bad)
}
}
}
// 1131/AC2: the presence keys form and parse back; a node named part is refused.
func testPresenceKey(t *testing.T) {
if PresenceBucket != "mesh-delivery-watchman_presence" {
t.Fatalf("PresenceBucket = %q", PresenceBucket)
}
key, err := PresenceNodeKey("ace")
if err != nil || key != "ace" {
t.Fatalf("PresenceNodeKey = %q, %v", key, err)
}
node, part, instance, err := ParsePresenceKey(key)
if err != nil || node != "ace" || part != "" || instance != "" {
t.Fatalf("ParsePresenceKey node = %q %q %q %v", node, part, instance, err)
}
key, err = PresencePartKey("view", "novox")
if err != nil || key != "part.view.novox" {
t.Fatalf("PresencePartKey = %q, %v", key, err)
}
node, part, instance, err = ParsePresenceKey(key)
if err != nil || node != "" || part != "view" || instance != "novox" {
t.Fatalf("ParsePresenceKey part = %q %q %q %v", node, part, instance, err)
}
if _, err := PresenceNodeKey("part"); err == nil || !strings.Contains(err.Error(), "part") {
t.Errorf("PresenceNodeKey(part) = %v", err)
}
if _, err := PresenceNodeKey("a.b"); err == nil {
t.Error("PresenceNodeKey(a.b) accepted")
}
if _, err := PresencePartKey("", "x"); err == nil {
t.Error("PresencePartKey with empty part accepted")
}
if _, err := PresencePartKey("view", "x y"); err == nil {
t.Error("PresencePartKey with a space accepted")
}
for _, bad := range []string{"", "part", "part.view", "part.view.a.b", "a.b", "part..x"} {
if _, _, _, err := ParsePresenceKey(bad); err == nil {
t.Errorf("ParsePresenceKey(%q) accepted", bad)
}
}
}
// 1131/AC3: the pulse key splits at the first dot, the key holding dots.
func testPulseKey(t *testing.T) {
if PulseBucket != "mesh-delivery-watchman_pulse" {
t.Fatalf("PulseBucket = %q", PulseBucket)
}
key, err := PulseKey("node-engine_have", "ace.plex")
if err != nil || key != "node-engine_have.ace.plex" {
t.Fatalf("PulseKey = %q, %v", key, err)
}
bucket, rest, err := ParsePulseKey(key)
if err != nil || bucket != "node-engine_have" || rest != "ace.plex" {
t.Fatalf("ParsePulseKey = %q %q %v", bucket, rest, err)
}
for _, c := range [][2]string{{"", "k"}, {"a.b", "k"}, {"b", ""}, {"b", "k x"}, {"b", ".k"}, {"b", "k."}, {"b", "k*"}} {
if _, err := PulseKey(c[0], c[1]); err == nil {
t.Errorf("PulseKey(%q, %q) accepted", c[0], c[1])
}
}
for _, bad := range []string{"", "nodot", ".k", "b."} {
if _, _, err := ParsePulseKey(bad); err == nil {
t.Errorf("ParsePulseKey(%q) accepted", bad)
}
}
}
// 1131/AC4: each value encodes, decodes equal, and carries "v":1.
func TestRoundTrip(t *testing.T) {
t.Run("phase and health refused", testPhaseHealthRefused)
hasV1 := func(t *testing.T, b []byte) {
t.Helper()
var m map[string]any
if err := json.Unmarshal(b, &m); err != nil {
t.Fatal(err)
}
if m["v"] != float64(1) {
t.Fatalf("v = %v in %s", m["v"], b)
}
if len(b) > 500 {
t.Fatalf("value is %d bytes", len(b))
}
}
t.Run("have", func(t *testing.T) {
in := Have{Build: "b1", FP: "f1", Phase: PhaseRunning, Health: HealthOK, Detail: "up"}
b, err := EncodeHave(in)
if err != nil {
t.Fatal(err)
}
hasV1(t, b)
out, err := DecodeHave(b)
if err != nil {
t.Fatal(err)
}
in.V = Version
if !SameHave(out, in) {
t.Fatalf("got %+v want %+v", out, in)
}
})
t.Run("have with the user list", func(t *testing.T) {
at := time.Date(2026, 10, 25, 21, 40, 0, 0, time.UTC)
in := Have{FP: "f1", Phase: PhaseRunning, Health: HealthOK, UserList: &UserListWritten{Source: "novox.nats.f1",
Revision: 7, Wanted: "sha256:ab", Digest: "sha256:ab", Users: 93, At: at}}
b, err := EncodeHave(in)
if err != nil {
t.Fatal(err)
}
hasV1(t, b)
out, err := DecodeHave(b)
if err != nil {
t.Fatal(err)
}
in.V = Version
if !reflect.DeepEqual(out, in) || !out.UserList.At.Equal(at) {
t.Fatalf("got %+v want %+v", out, in)
}
// And with the hashes handed to the vault (ADR 0547 clause 9): sealed to its key, opened by it alone, every
// listed user with a bcrypt hash once, and refused to another key.
hash := "$2a$11$" + strings.Repeat("a", 53)
list := []byte("{ user: \"controller\", password: \"" + hash + "\", permissions: {\n" +
"{ user: \"node.ace\", password: \"" + hash + "\", permissions: {\n" +
"{ user: \"controller\", password: \"" + hash + "\" }\n{ user: \"nkey\", password: \"plain\" }\n")
accounts := HandedAccountsOf(list)
if len(accounts) != 2 || accounts[0].User != "controller" || accounts[1].User != "node.ace" || accounts[1].Hash != hash {
t.Fatalf("handed accounts: %+v", accounts)
}
vault, err := seal.GenerateKeyPair()
if err != nil {
t.Fatal(err)
}
other, _ := seal.GenerateKeyPair()
handed, err := HandAccounts(accounts, vault.PublicKey, "sha256:ab", at)
if err != nil {
t.Fatal(err)
}
in.UserList.Handed = &handed
b, err = EncodeHave(in)
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(b), "node.ace") || strings.Contains(string(b), hash) {
t.Fatalf("a user or a hash is in the clear: %s", b)
}
out, err = DecodeHave(b)
if err != nil || !SameHave(out, in) {
t.Fatalf("handed round trip: %v %+v", err, out.UserList)
}
opened, err := OpenHanded(*out.UserList.Handed, vault.PublicKey, vault.PrivateKey)
if err != nil || !reflect.DeepEqual(opened, accounts) {
t.Fatalf("opened %+v, %v", opened, err)
}
if _, err := OpenHanded(*out.UserList.Handed, other.PublicKey, other.PrivateKey); err == nil {
t.Fatal("a handing opened for another key")
}
if _, err := HandAccounts(accounts, "", "sha256:ab", at); err == nil {
t.Fatal("handed with no vault key")
}
})
// The grants part's list sealed to the bus node (novox/hq ADR 0547 clause 7): the key's value round-trips and refuses
// what the bus node could not follow, and the object opens with the node's key alone, to the digest the key names.
t.Run("the grants part's sealed user list", func(t *testing.T) {
in := Have{FP: "f1", Phase: PhaseRunning, Health: HealthOK, UserList: &UserListWritten{Source: GrantsUserListSource,
Revision: 3, Wanted: "sha256:ab", Digest: "sha256:ab", Users: 2, Follows: FollowGrants, Key: "PEM"}}
b, err := EncodeHave(in)
if err != nil {
t.Fatal(err)
}
if out, err := DecodeHave(b); err != nil || out.UserList.Follows != FollowGrants || out.UserList.Key != "PEM" {
t.Fatalf("got %+v, %v", out.UserList, err)
}
list := []byte("accounts { MESH { users = [ { user: \"controller\" } ] } }\n")
digest := UserListDigest(list)
node, err := seal.GenerateKeyPair()
if err != nil {
t.Fatal(err)
}
other, _ := seal.GenerateKeyPair()
g := GrantsUserList{Follow: FollowGrants, Digest: digest, Object: UserListObjectName(digest),
SealedTo: SealingKeyID(node.PublicKey), Node: "novox", Revision: 1, Users: 1}
raw, err := EncodeGrantsUserList(g)
if err != nil {
t.Fatal(err)
}
back, err := DecodeGrantsUserList(raw)
if err != nil || back.Digest != digest || back.LeftOut == nil || back.Follow != FollowGrants {
t.Fatalf("decoded %+v, %v", back, err)
}
// Issue 1126 B4a (ADR 0590): a key written with the lease's flag still reads; the controller's list is no source.
if old, err := DecodeGrantsUserList([]byte(`{"v":1,"follow":"grants","digest":"","object":"","sealed_to":"",` +
`"node":"novox","revision":3,"users":40,"left_out":[],"at":"2026-10-30T10:00:00Z","lease-handed":true}`)); err != nil || old.Revision != 3 {
t.Errorf("a key with the lease's flag decodes %+v, %v", old, err)
}
for _, bad := range []GrantsUserList{{Follow: "both", Digest: digest, Object: g.Object}, {Follow: "controller"},
{Follow: FollowGrants, Digest: digest, Object: "other"}, {Follow: FollowGrants, Digest: "md5:x", Object: "md5:x"}} {
if _, err := EncodeGrantsUserList(bad); err == nil {
t.Errorf("%+v was written", bad)
}
}
if _, err := EncodeGrantsUserList(GrantsUserList{Follow: FollowGrants}); err != nil {
t.Errorf("a switch with no list yet refused: %v", err)
}
object, err := SealUserList(list, node.PublicKey)
if err != nil {
t.Fatal(err)
}
if bytes.Contains(object, []byte("controller")) {
t.Fatal("the object carries the list in the clear")
}
opener := func(priv string) func(seal.Box) ([]byte, error) {
return func(b seal.Box) ([]byte, error) { s, err := seal.Open(b, priv); return []byte(s), err }
}
if got, err := OpenUserList(object, digest, opener(node.PrivateKey)); err != nil || !bytes.Equal(got, list) {
t.Fatalf("opened %q, %v", got, err)
}
if _, err := OpenUserList(object, digest, opener(other.PrivateKey)); err == nil {
t.Error("another key opened the list")
}
if _, err := OpenUserList(object, "sha256:00", opener(node.PrivateKey)); err == nil {
t.Error("a list of another digest was taken")
}
if _, err := SealUserList(list, "not a key"); err == nil {
t.Error("a list was sealed to no key")
}
})
t.Run("have node", func(t *testing.T) {
in := HaveNode{Windows: json.RawMessage(`{"a":1}`), Busy: json.RawMessage(`true`)}
b, err := EncodeHaveNode(in)
if err != nil {
t.Fatal(err)
}
hasV1(t, b)
out, err := DecodeHaveNode(b)
if err != nil {
t.Fatal(err)
}
if !SameHaveNode(out, HaveNode{V: Version, Windows: in.Windows, Busy: in.Busy}) {
t.Fatalf("got %+v", out)
}
})
t.Run("presence", func(t *testing.T) {
b, err := EncodePresence(Presence{Online: true})
if err != nil {
t.Fatal(err)
}
hasV1(t, b)
out, err := DecodePresence(b)
if err != nil || out != (Presence{V: Version, Online: true}) {
t.Fatalf("got %+v %v", out, err)
}
})
t.Run("pulse", func(t *testing.T) {
in := Pulse{Bucket: "node-engine_have", Key: "ace.plex", Revision: 7, Owes: "ace"}
b, err := EncodePulse(in)
if err != nil {
t.Fatal(err)
}
hasV1(t, b)
out, err := DecodePulse(b)
in.V = Version
if err != nil || out != in {
t.Fatalf("got %+v %v", out, err)
}
})
}
// 1131/AC5: a missing or unknown version is refused as ErrUnknownVersion, naming both versions.
func TestUnknownVersion(t *testing.T) {
decoders := map[string]func([]byte) error{
"have": func(b []byte) error { _, err := DecodeHave(b); return err },
"have node": func(b []byte) error { _, err := DecodeHaveNode(b); return err },
"presence": func(b []byte) error { _, err := DecodePresence(b); return err },
"pulse": func(b []byte) error { _, err := DecodePulse(b); return err },
}
for name, dec := range decoders {
t.Run(name, func(t *testing.T) {
err := dec([]byte(`{"v":2,"phase":"running","health":"ok"}`))
if !errors.Is(err, ErrUnknownVersion) || !strings.Contains(err.Error(), "2") || !strings.Contains(err.Error(), "1") {
t.Errorf("v 2: %v", err)
}
err = dec([]byte(`{"phase":"running","health":"ok"}`))
if !errors.Is(err, ErrUnknownVersion) {
t.Errorf("missing v: %v", err)
}
if err := dec([]byte(`not json`)); err == nil || errors.Is(err, ErrUnknownVersion) {
t.Errorf("garbage: %v", err)
}
})
}
}
// 1131/AC6: a phase or health outside the lists is refused on encode and decode.
func testPhaseHealthRefused(t *testing.T) {
for _, p := range []Phase{PhaseBackingUp, PhaseApplying, PhaseChecking, PhaseRunning, PhaseFailed} {
for _, h := range []Health{HealthOK, HealthFailed, HealthWaits, HealthUnknown} {
if _, err := EncodeHave(Have{Phase: p, Health: h}); err != nil {
t.Errorf("%s/%s refused: %v", p, h, err)
}
}
}
if _, err := EncodeHave(Have{Phase: "sleeping", Health: HealthOK}); err == nil || !strings.Contains(err.Error(), "sleeping") {
t.Errorf("bad phase encoded: %v", err)
}
if _, err := EncodeHave(Have{Phase: PhaseRunning, Health: "fine"}); err == nil || !strings.Contains(err.Error(), "fine") {
t.Errorf("bad health encoded: %v", err)
}
if _, err := EncodeHave(Have{Phase: PhaseRunning}); err == nil {
t.Error("empty health encoded")
}
if _, err := DecodeHave([]byte(`{"v":1,"phase":"sleeping","health":"ok"}`)); err == nil {
t.Error("bad phase decoded")
}
if _, err := DecodeHave([]byte(`{"v":1,"phase":"running","health":"fine"}`)); err == nil {
t.Error("bad health decoded")
}
// ADR 0547 §3 (issue 1126): a have waits for a yes with its judgement and the composition waiting, and no other have
// names either.
waits := Have{Phase: PhaseWaitingYes, Health: HealthOK, Judgement: JudgedRecreates, Waits: "sha256:" + strings.Repeat("a", 64)}
if raw, err := EncodeHave(waits); err != nil {
t.Errorf("a have waiting for a yes: %v", err)
} else if back, err := DecodeHave(raw); err != nil || back.Judgement != JudgedRecreates || back.Waits != waits.Waits {
t.Errorf("a have waiting for a yes came back %+v, %v", back, err)
}
// ADR 0601 §1: and whether the move changes the build alone, read back as said.
alone := waits
alone.BuildAlone = true
if raw, err := EncodeHave(alone); err != nil {
t.Errorf("a have waiting for a yes to its build alone: %v", err)
} else if back, err := DecodeHave(raw); err != nil || !back.BuildAlone {
t.Errorf("a have waiting for a yes to its build alone came back %+v, %v", back, err)
}
// ADR 0601 §4: what the node keeps as it runs, read back as said.
if raw, err := EncodeHave(Have{Phase: PhaseRunning, Health: HealthOK, Kept: []string{"runs the tool runner"}}); err != nil {
t.Errorf("a have keeping what runs: %v", err)
} else if back, err := DecodeHave(raw); err != nil || len(back.Kept) != 1 {
t.Errorf("a have keeping what runs came back %+v, %v", back, err)
}
for name, h := range map[string]Have{
"running build alone": {Phase: PhaseRunning, Health: HealthOK, BuildAlone: true},
"neither judged": {Phase: PhaseWaitingYes, Health: HealthOK, Judgement: JudgedNeither, Waits: waits.Waits},
"no waiting digest": {Phase: PhaseWaitingYes, Health: HealthOK, Judgement: JudgedMovesData},
"running judged": {Phase: PhaseRunning, Health: HealthOK, Judgement: JudgedRecreates},
} {
if _, err := EncodeHave(h); err == nil {
t.Errorf("%s: encoded", name)
}
}
}
// 1131/AC7: the pulse's time to live is 120 seconds minus the move's age; at the bound the move is stalled.
func testPulseTTL(t *testing.T) {
for _, c := range []struct {
age time.Duration
ttl time.Duration
stalled bool
}{
{0, 120 * time.Second, false},
{30 * time.Second, 90 * time.Second, false},
{119 * time.Second, time.Second, false},
{120 * time.Second, 0, true},
{200 * time.Second, 0, true},
} {
ttl, stalled := PulseTTL(c.age)
if ttl != c.ttl || stalled != c.stalled {
t.Errorf("PulseTTL(%v) = %v, %v; want %v, %v", c.age, ttl, stalled, c.ttl, c.stalled)
}
}
if PulseMoveBound != 120*time.Second || HaveHistory != 20 || PresenceHistory != 20 {
t.Error("constants")
}
}
// 1131/AC8: equal values say same, different ones do not.
func TestSame(t *testing.T) {
t.Run("have removed", testHaveRemoved)
a := Have{V: 1, Build: "b", Phase: PhaseRunning, Health: HealthOK}
b := a
if !SameHave(a, b) {
t.Error("equal haves differ")
}
b.Detail = "x"
if SameHave(a, b) {
t.Error("different haves same")
}
// The sealed values a self-composing node runs are part of its have (issue 1126).
sa, sb := a, a
sa.Sealed, sb.Sealed = []string{`{"box":1}`}, []string{`{"box":1}`}
if !SameHave(sa, sb) || SameHave(sa, a) {
t.Error("equal sealed values differ, or one said and one not are the same")
}
if raw, err := EncodeHave(sa); err != nil {
t.Error(err)
} else if back, err := DecodeHave(raw); err != nil || !SameHave(back, sa) {
t.Errorf("sealed values do not travel: %+v, %v", back, err)
}
// A failed have carries the pull it waits on, compared by what it says (issue 1966 ENG-9); only a failed one does.
began := time.Date(2026, 11, 4, 14, 0, 0, 0, time.UTC)
pa, pb := a, a
pa.Phase, pb.Phase = PhaseFailed, PhaseFailed
pa.PullWait = &PullWait{Image: "reg/x@sha256:1", Seat: "container-image-registry", Holder: "distribution", Began: began,
Retried: "distribution settled", RetriedAt: began.Add(time.Minute)}
w := *pa.PullWait
w.Began, w.RetriedAt = began.In(time.Local), w.RetriedAt.In(time.Local)
pb.PullWait = &w
if !SameHave(pa, pb) || SameHave(pa, Have{V: 1, Build: "b", Phase: PhaseFailed, Health: HealthOK}) {
t.Error("equal pull waits differ, or one said and one not are the same")
}
if raw, err := EncodeHave(pa); err != nil {
t.Error(err)
} else if back, err := DecodeHave(raw); err != nil || !SameHave(back, pa) {
t.Errorf("a pull wait does not travel: %+v, %v", back, err)
}
pb.PullWait = &PullWait{Image: "reg/x@sha256:1", Holder: "distribution", Began: began}
if SameHave(pa, pb) {
t.Error("pull waits retried and not are the same")
}
for name, h := range map[string]Have{
"on a running have": {Phase: PhaseRunning, Health: HealthOK, PullWait: pa.PullWait},
"naming no holder": {Phase: PhaseFailed, Health: HealthOK, PullWait: &PullWait{Image: "reg/x@sha256:1"}},
} {
if _, err := EncodeHave(h); err == nil {
t.Errorf("a pull wait %s was encoded", name)
}
}
// The user list is compared by what it says, never by where it is held (ADR 0547 §8).
at := time.Date(2026, 10, 25, 21, 0, 0, 0, time.UTC)
ua, ub := a, a
ua.UserList = &UserListWritten{Source: "k", Revision: 1, Wanted: "sha256:a", Digest: "sha256:a", Users: 2, At: at}
ub.UserList = &UserListWritten{Source: "k", Revision: 1, Wanted: "sha256:a", Digest: "sha256:a", Users: 2, At: at.In(time.Local)}
if !SameHave(ua, ub) || SameHave(ua, a) {
t.Error("equal user lists differ, or one said and one not are the same")
}
ub.UserList.Revision = 2
if SameHave(ua, ub) {
t.Error("different user lists same")
}
// A handing is compared by what it says: one said and one not differ, as do two to other keys.
ub.UserList.Revision = 1
ua.UserList.Handed = &AccountsHanded{From: "sha256:a", To: "sha256:k", Users: 2, At: at}
if SameHave(ua, ub) {
t.Error("a handing said and one not are the same")
}
ub.UserList.Handed = &AccountsHanded{From: "sha256:a", To: "sha256:k", Users: 2, At: at.In(time.Local)}
if !SameHave(ua, ub) {
t.Error("equal handings differ")
}
ub.UserList.Handed.To = "sha256:j"
if SameHave(ua, ub) {
t.Error("handings to other keys same")
}
n := HaveNode{V: 1, Busy: json.RawMessage(`true`)}
if !SameHaveNode(n, HaveNode{V: 1, Busy: json.RawMessage(`true`)}) || SameHaveNode(n, HaveNode{V: 1}) {
t.Error("have node")
}
if !SamePresence(Presence{V: 1, Online: true}, Presence{V: 1, Online: true}) || SamePresence(Presence{V: 1}, Presence{V: 1, Online: true}) {
t.Error("presence")
}
p := Pulse{V: 1, Bucket: "b", Key: "k", Revision: 1, Owes: "ace"}
q := p
if !SamePulse(p, q) {
t.Error("equal pulses differ")
}
q.Revision = 2
if SamePulse(p, q) {
t.Error("different pulses same")
}
}
func testHaveRemoved(t *testing.T) { // 1370/AC3
in := Have{Build: "b1", Phase: PhaseRemoved, Health: HealthUnknown, Detail: "the declaration no longer holds it"}
b, err := EncodeHave(in)
if err != nil {
t.Fatal(err)
}
out, err := DecodeHave(b)
if err != nil {
t.Fatal(err)
}
in.V = Version
if !SameHave(out, in) || out.Phase != "removed" {
t.Errorf("round trip: %+v", out)
}
}
-72
View File
@@ -1,72 +0,0 @@
package delivery
import (
"encoding/json"
"fmt"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// **The data items of each module placed on a node, as its engine composed them** (novox/hq ADR 0577 clauses 1 and 4):
// the node-engine's state `data` (EngineDataBucket), one key `<node>.<module>` (EngineDataKey) per module placed on the
// node that declares data items, written by that node's engine alone, when the module's composition changes and only
// when the value differs, and deleted when the module leaves the node. That node's disk-load watches its node's keys
// (EngineDataNodeKeys) and measures what they name; it composes nothing itself. Only the contract is here: the state's
// names, its key and its value; the engine composes the items and disk-load measures them.
// EngineDataState, EngineDataBucket and EngineDataRead are the node-engine's state data, by its local name, as the bus
// names it and as a reader declares it.
const (
EngineDataState = "data"
EngineDataBucket = "mesh-host_" + EngineDataState
EngineDataRead = "mesh-host." + EngineDataState
)
// EngineDataVersion is the version of EngineData's layout.
const EngineDataVersion = 1
// EngineData is one key's value: the module's data items on the node as the engine composed them (each item's id, its
// path resolved on that node, its measure, class, protection and what covers it), and the digest of the module's
// composition they came from (ComposedDigestOf).
type EngineData struct {
V int `json:"v"`
Node string `json:"node"`
Module string `json:"module"`
Digest string `json:"digest"`
Items []catalogue.PlacedDataItem `json:"items"`
}
// EngineDataKey is a module's key on a node, `<node>.<module>`, its module part escaped as a have's is (HaveKey).
func EngineDataKey(node, module string) (string, error) { return HaveKey(node, module) }
// EngineDataNodeKeys is the pattern of every key of a node, `<node>.*`: what that node's disk-load watches.
func EngineDataNodeKeys(node string) (string, error) {
if err := checkName("node", node); err != nil {
return "", err
}
return node + ".*", nil
}
// EncodeEngineData writes a value at this layout's version.
func EncodeEngineData(d EngineData) ([]byte, error) {
d.V = EngineDataVersion
if _, err := EngineDataKey(d.Node, d.Module); err != nil {
return nil, err
}
if d.Items == nil {
d.Items = []catalogue.PlacedDataItem{}
}
return json.Marshal(d)
}
// DecodeEngineData reads a value; it refuses another version.
func DecodeEngineData(raw []byte) (EngineData, error) {
var d EngineData
if err := json.Unmarshal(raw, &d); err != nil {
return EngineData{}, fmt.Errorf("delivery: a module's data items are not readable: %w", err)
}
if d.V != EngineDataVersion {
return EngineData{}, fmt.Errorf("delivery: a module's data items of version %d, and this reads %d", d.V, EngineDataVersion)
}
return d, nil
}
-137
View File
@@ -1,137 +0,0 @@
package delivery
import (
"encoding/json"
"fmt"
"slices"
"sort"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// **A node's declared packages, worked out by its engine** (novox/hq ADR 0565 §1, §6; issue 1953): the derivation and
// the comparison of the node-engine's state `packages` (EnginePackagesState, layout DeclaredPackages), so the engine
// that writes the key, the node seat's verb that compares it with the controller's mirror, and any reader work them out
// one way. The controller's own copy of the derivation (mesh-controller internal/inventory DeclaredPackagesValue) is
// not changed (ADR 0547 §1) and ends with its lease.
// EnginePackagesRead is the engine's state `packages` as a reader names it in `reads`: the node-engine's manifest is
// mesh-host's, so its bucket is mesh-host_packages, read as disk-load reads mesh-host.data (EngineDataRead).
const EnginePackagesRead = "mesh-host." + EnginePackagesState
// DeclaredPackagesOf is a node's key of `packages` from the compositions its engine composed for the node's wants:
// each resource of type package, by its name and whether it is declared absent, with the modules that declare it so,
// in the mirror's layout and order (names sorted, absent after present, modules sorted). A resource's module is the
// one it names, else its composition's, else the guess from its id (declaration.ModuleGuess); a package of no module
// lists none.
func DeclaredPackagesOf(node string, compositions []Composed) (DeclaredPackages, error) {
by := map[packageKey]map[string]bool{}
for _, c := range compositions {
for _, raw := range c.Resources {
var r struct {
ID string `json:"id"`
Type string `json:"type"`
Package string `json:"package"`
Absent bool `json:"absent"`
Module string `json:"module"`
}
if err := json.Unmarshal(raw, &r); err != nil {
return DeclaredPackages{}, fmt.Errorf("delivery: a resource of %s's composition %q is not readable: %w", node, c.Module, err)
}
if r.Type != "package" || r.Package == "" {
continue
}
k := packageKey{r.Package, r.Absent}
if by[k] == nil {
by[k] = map[string]bool{}
}
module := r.Module
if module == "" {
module = c.Module
}
if module == "" {
module, _ = declaration.ModuleGuess(r.ID)
}
if module != "" {
by[k][module] = true
}
}
}
v := DeclaredPackages{Node: node, Packages: []PackageDeclared{}}
for _, k := range sortedPackageKeys(by) {
modules := make([]string, 0, len(by[k]))
for m := range by[k] {
modules = append(modules, m)
}
sort.Strings(modules)
v.Packages = append(v.Packages, PackageDeclared{Package: k.name, Modules: modules, Absent: k.absent})
}
return v, nil
}
// PackageDifference is one package two declarations of a node hold differently: the modules each side declares it
// with (present, or absent when Absent), none on a side that does not hold it.
type PackageDifference struct {
Package string `json:"package"`
Absent bool `json:"absent"`
Engine []string `json:"engine"`
Mirror []string `json:"mirror"`
}
// DiffDeclaredPackages is every package the engine's key and the controller's mirror hold differently for one node, in
// the layout's order: what the node seat's verb packages answers until the lease write (ADR 0565 §6). Empty when they
// agree.
func DiffDeclaredPackages(engine, mirror DeclaredPackages) []PackageDifference {
index := func(d DeclaredPackages) map[packageKey][]string {
out := map[packageKey][]string{}
for _, p := range d.Packages {
out[packageKey{p.Package, p.Absent}] = append([]string{}, p.Modules...)
}
return out
}
e, m := index(engine), index(mirror)
union := map[packageKey]bool{}
for k := range e {
union[k] = true
}
for k := range m {
union[k] = true
}
out := []PackageDifference{}
for _, k := range sortedPackageKeys(union) {
em, inE := e[k]
mm, inM := m[k]
if inE && inM && slices.Equal(em, mm) {
continue
}
if em == nil {
em = []string{}
}
if mm == nil {
mm = []string{}
}
out = append(out, PackageDifference{Package: k.name, Absent: k.absent, Engine: em, Mirror: mm})
}
return out
}
// packageKey is a package as the layout tells it apart: its name, and whether it is declared absent.
type packageKey struct {
name string
absent bool
}
// sortedPackageKeys is the keys of m by name, absent after present.
func sortedPackageKeys[V any](m map[packageKey]V) []packageKey {
keys := make([]packageKey, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Slice(keys, func(i, j int) bool {
if keys[i].name != keys[j].name {
return keys[i].name < keys[j].name
}
return !keys[i].absent && keys[j].absent
})
return keys
}
-308
View File
@@ -1,308 +0,0 @@
package delivery
import (
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"net/netip"
"strconv"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// **The state of the seat mesh-enrolment** (novox/hq ADR 0587 §2, issue 774): the holder's bucket `enrolments`, holding
// no secret, with two kinds of key. `node.<node>`, one per open enrolment, holds an Enrolment; `address.<address>`
// reserves one private-network address for one open enrolment and holds the node's name and, from the ask, the ask and the issue's time. Every write is a
// compare-and-set at the revision read. The bucket keeps per-key expiry (`key-ttl`): a key at stage live carries the
// token's expiry, at asked and redeeming none (§5). It is read by the holder and, as a named reader, by the grants part,
// which composes the user `enrol.<node>` from a key at live and the vault's hash (§4). These constants are the seat's
// contract: every reader takes the names from here.
const (
// EnrolmentModule is the module holding the seat mesh-enrolment.
EnrolmentModule = catalogue.EnrolmentSeat
// EnrolmentsState is its state, EnrolmentsBucket the bucket, EnrolmentsRead the name a reader's manifest declares.
EnrolmentsState = "enrolments"
EnrolmentsBucket = EnrolmentModule + "_" + EnrolmentsState
EnrolmentsRead = EnrolmentModule + "." + EnrolmentsState
// The stages of an open enrolment (§2, §4, §5).
EnrolmentAsked = "asked"
EnrolmentLive = "live"
EnrolmentRedeeming = "redeeming"
// DefaultTokenLife is a token's life when issue names none; MostTokenLife the longest it may name (§4).
DefaultTokenLife = time.Hour
MostTokenLife = 24 * time.Hour
// EnrolmentUserPrefix is the bus user a joining machine dials as, `enrol.<node>` (§4).
EnrolmentUserPrefix = "enrol."
enrolmentNodePrefix = "node."
enrolmentAddressPrefix = "address."
)
// Enrolment is one key `node.<node>` of `enrolments`: the node's name, the private-network address chosen at issue, the
// why, the caller as the runtime proved it, the ask's id, the stage, when it was issued and when the token expires, and
// from redeem the digest of the public keys presented (KeysDigest). No secret and nothing derived from the token.
type Enrolment struct {
V int `json:"v"`
Node string `json:"node"`
Address string `json:"address"`
Why string `json:"why"`
Caller string `json:"caller"`
Ask string `json:"ask,omitempty"`
Stage string `json:"stage"`
Issued time.Time `json:"issued"`
Expires time.Time `json:"expires"`
KeysDigest string `json:"keys-digest,omitempty"`
// OverlayKey is the machine's tunnel public key, given at issue (the operator's decision of 2026-11-02 01:05),
// base64 of 32 bytes: at every stage, since the node enters placement with it on the operator's yes.
OverlayKey string `json:"overlay-key"`
}
// EnrolmentAddress is one key `address.<address>`: the node the address is reserved for and, from when the issue's ask
// is raised, that ask's id and the issue's time (ADR 0589 §6), so the reservation names the enrolment it holds the
// address for. The key is written when the address is reserved, before the ask: Ask and Issued are then both empty, and
// from the ask both set; one without the other is refused.
type EnrolmentAddress struct {
V int `json:"v"`
Node string `json:"node"`
Ask string `json:"ask,omitempty"`
Issued time.Time `json:"issued,omitzero"`
}
func (a EnrolmentAddress) check() error {
if err := checkName("node", a.Node); err != nil {
return err
}
if (strings.TrimSpace(a.Ask) == "") != a.Issued.IsZero() {
return fmt.Errorf("delivery: the address reserved for %s names its ask without the issue's time, or the reverse", a.Node)
}
return nil
}
// EnrolmentIssuedID is the id of the enrolment-issued event of node's enrolment issued at issued:
// `enrolment-issued.<node>.<issue time in Unix nanoseconds>`, what enrolment-expired names (EnrolmentExpiry).
func EnrolmentIssuedID(node string, issued time.Time) string {
return catalogue.EnrolmentIssuedEvent + "." + node + "." + strconv.FormatInt(issued.UnixNano(), 10)
}
// EnrolmentNodeKey is a node's key in `enrolments`.
func EnrolmentNodeKey(node string) string { return enrolmentNodePrefix + node }
// EnrolmentAddressKey is an address's key in `enrolments`: the address as it is, its dots kept (a key's tokens).
func EnrolmentAddressKey(address string) string { return enrolmentAddressPrefix + address }
// EnrolmentKeyNode is the node a key `node.<node>` names, and whether it is one.
func EnrolmentKeyNode(key string) (string, bool) {
node, ok := strings.CutPrefix(key, enrolmentNodePrefix)
return node, ok && checkName("node", node) == nil
}
// EnrolmentKeyAddress is the address a key `address.<address>` names, and whether it is one.
func EnrolmentKeyAddress(key string) (string, bool) {
address, ok := strings.CutPrefix(key, enrolmentAddressPrefix)
if !ok {
return "", false
}
_, err := netip.ParseAddr(address)
return address, err == nil
}
// EnrolmentUser is the bus user a node enrols as.
func EnrolmentUser(node string) string { return EnrolmentUserPrefix + node }
func (e Enrolment) check() error {
if err := checkName("node", e.Node); err != nil {
return err
}
if _, err := netip.ParseAddr(e.Address); err != nil {
return fmt.Errorf("delivery: the enrolment of %s names the address %q, which is not an address", e.Node, e.Address)
}
if strings.TrimSpace(e.Why) == "" || strings.TrimSpace(e.Caller) == "" {
return fmt.Errorf("delivery: the enrolment of %s names no why or no caller", e.Node)
}
if k, err := base64.StdEncoding.DecodeString(e.OverlayKey); err != nil || len(k) != 32 {
return fmt.Errorf("delivery: the enrolment of %s names no tunnel key of 32 bytes in base64", e.Node)
}
switch e.Stage {
case EnrolmentAsked, EnrolmentLive:
if e.KeysDigest != "" {
return fmt.Errorf("delivery: the enrolment of %s at %s names presented keys, which only a redeem gives", e.Node, e.Stage)
}
case EnrolmentRedeeming:
if err := checkDigest(e.KeysDigest, "the enrolment's keys digest"); err != nil {
return err
}
default:
return fmt.Errorf("delivery: the enrolment of %s is at stage %q, not asked, live or redeeming", e.Node, e.Stage)
}
if e.Issued.IsZero() || !e.Expires.After(e.Issued) {
return fmt.Errorf("delivery: the enrolment of %s names no issue or an expiry not after it", e.Node)
}
return nil
}
// EncodeEnrolment is e as JSON at Version; it refuses what DecodeEnrolment refuses.
func EncodeEnrolment(e Enrolment) ([]byte, error) {
if err := e.check(); err != nil {
return nil, err
}
e.V = Version
return json.Marshal(e)
}
// DecodeEnrolment reads a key `node.<node>`; it refuses an unknown version (ErrUnknownVersion), a node that is not a
// name, an address that is not one, an unknown stage, presented keys before redeem or none at it, and an expiry not
// after the issue.
func DecodeEnrolment(data []byte) (Enrolment, error) {
var e Enrolment
if err := decode(data, &e); err != nil {
return Enrolment{}, err
}
if err := e.check(); err != nil {
return Enrolment{}, err
}
return e, nil
}
// EncodeEnrolmentAddress is a key `address.<address>` reserving it for node: with no ask and a zero issued when it is
// reserved, with the issue's ask and time once the ask is raised. It refuses what DecodeEnrolmentAddress refuses.
func EncodeEnrolmentAddress(node, ask string, issued time.Time) ([]byte, error) {
a := EnrolmentAddress{V: Version, Node: node, Ask: ask, Issued: issued}
if err := a.check(); err != nil {
return nil, err
}
return json.Marshal(a)
}
// DecodeEnrolmentAddress reads a key `address.<address>`; it refuses no node, and an ask without the issue's time or
// the reverse.
func DecodeEnrolmentAddress(data []byte) (EnrolmentAddress, error) {
var a EnrolmentAddress
if err := decode(data, &a); err != nil {
return EnrolmentAddress{}, err
}
if err := a.check(); err != nil {
return EnrolmentAddress{}, err
}
return a, nil
}
// TokenLife is the life issue's `expires` names: DefaultTokenLife when empty, refused when it is not a duration, not
// positive or longer than MostTokenLife.
func TokenLife(expires string) (time.Duration, error) {
if strings.TrimSpace(expires) == "" {
return DefaultTokenLife, nil
}
d, err := time.ParseDuration(strings.TrimSpace(expires))
if err != nil || d <= 0 || d > MostTokenLife {
return 0, fmt.Errorf("delivery: a token lasts a duration up to %s, such as 1h, not %q", MostTokenLife, expires)
}
return d, nil
}
// EnrolmentExpiry is what enrolment-expired carries (§2): the node, and the enrolment-issued event that carried the
// key's content, named by its message id (the bundle's publish answers no stream sequence, and §2 asks only that the
// event be named), since the bus deleted the key before the holder heard of it.
type EnrolmentExpiry struct {
V int `json:"v"`
Node string `json:"node"`
IssuedID string `json:"issued-event"`
}
func (x EnrolmentExpiry) check() error {
if err := checkName("node", x.Node); err != nil {
return err
}
if strings.TrimSpace(x.IssuedID) == "" {
return fmt.Errorf("delivery: the expiry of %s's enrolment names no enrolment-issued event", x.Node)
}
return nil
}
// EncodeEnrolmentExpiry is x as JSON at Version; it refuses what DecodeEnrolmentExpiry refuses.
func EncodeEnrolmentExpiry(x EnrolmentExpiry) ([]byte, error) {
if err := x.check(); err != nil {
return nil, err
}
x.V = Version
return json.Marshal(x)
}
// DecodeEnrolmentExpiry reads enrolment-expired's body; it refuses an unknown version, a node that is not a name and
// no issued event named.
func DecodeEnrolmentExpiry(data []byte) (EnrolmentExpiry, error) {
var x EnrolmentExpiry
if err := decode(data, &x); err != nil {
return EnrolmentExpiry{}, err
}
if err := x.check(); err != nil {
return EnrolmentExpiry{}, err
}
return x, nil
}
// **Redeem** (§6): what a joining node-engine sends, and what the holder answers. Shared by the holder and the engine,
// which live in two repositories.
// Redeem is the redeem verb's call: the node, the public halves of its keys and its identity key's signature over
// RedeemProof. No token: the bus's login check verified it.
type Redeem struct {
Node string `json:"node"`
PublicKey string `json:"public_key"`
OverlayKey string `json:"overlay_key,omitempty"`
SealingKey string `json:"sealing_key"`
ServingKey string `json:"serving_key,omitempty"`
Profile map[string]any `json:"profile,omitempty"`
Tunnel json.RawMessage `json:"tunnel,omitempty"`
Proof string `json:"proof"`
}
// RedeemProof is the bytes a node signs with its identity key when it redeems: its name and every public key it
// presents, so a proof cannot be moved to another node or other keys. The holder builds the same bytes.
func RedeemProof(node, public, overlay, sealing, serving string) []byte {
return []byte("novox-mesh-redeem v1\x00" + node + "\x00" + public + "\x00" + overlay + "\x00" + sealing + "\x00" + serving)
}
// KeysDigest is the digest of the public keys a redeem presents, `sha256:<hex>`: what the key at redeeming and the
// events carry, the one hash an event carries (§1).
func KeysDigest(public, overlay, sealing, serving string) string {
sum := sha256.Sum256([]byte(strings.Join([]string{public, overlay, sealing, serving}, "\x00")))
return "sha256:" + hex.EncodeToString(sum[:])
}
// KeysDigest is r's keys' digest.
func (r Redeem) KeysDigest() string {
return KeysDigest(r.PublicKey, r.OverlayKey, r.SealingKey, r.ServingKey)
}
// ProofBytes is r's proof decoded from base64.
func (r Redeem) ProofBytes() ([]byte, error) { return base64.StdEncoding.DecodeString(r.Proof) }
// RedeemAnswer is what redeem answers (§6): the node, the login it dials as next, its account sealed to the sealing key it
// presented (base64), its certificate as PEM, the vault's public signing half (base64), and the vault's signature over
// SignedBytes under the context of KindEnrolment. The engine refuses an answer not signed by the key its token carries.
type RedeemAnswer struct {
Node string `json:"node"`
Login string `json:"login"`
Account string `json:"account"`
Certificate string `json:"certificate"`
Signer string `json:"signer"`
Signature string `json:"signature,omitempty"`
}
// SignedBytes is what the vault signs of a: a without its signature, as JSON.
func (a RedeemAnswer) SignedBytes() []byte {
a.Signature = ""
raw, _ := json.Marshal(a)
return raw
}
// The vault's enrolment verbs (ADR 0561 §5, ADR 0587 §3, §5), asked by mesh-enrolment alone, and their answers are the
// vault's contract's (mesh-contracts meshvault: VerbEnrolmentToken, EnrolmentTokenAnswer, VerbNodeAccount,
// NodeAccountAnswer; novox/hq ADR 0627 §1).
-17
View File
@@ -1,17 +0,0 @@
package delivery
// SourceInputs are what the ended controller's source fingerprint (src1) was of, as its build records hold them
// (BuildRecord), read and never computed again (novox/hq ADR 0437 §5): the module, the git tree of
// its directory, its bases (references, of which only the digest counts), the root tree of each repository its
// artifacts read (by artifact), each artifact's toolchain line, and the reason the source does not pin the build
// when it does not (a build that resolves packages when built). A context is the value the builder records for it:
// a repository's root tree, or for an image that compiles a Go program the `narrow:` hash of the build source it was
// handed (mesh-controller internal/builder/builder.go:1146-1156).
type SourceInputs struct {
Module string
Tree string
Bases []string
Contexts map[string]string
Toolchains map[string]string
Unpinned string
}
-43
View File
@@ -1,43 +0,0 @@
package delivery
import (
"fmt"
"strings"
)
// SourcePrefix2 opens every source fingerprint of the new form. The fingerprint is computed by the indexer alone, in
// its own package (novox/hq ADR 0593 §2); this is its form, read by every part.
const SourcePrefix2 = "src2:"
// Toolchain2 is what an artifact's toolchain enters the new fingerprint as (novox/hq ADR 0405 clause 7): the
// artifact's language, the installed toolchain's version and the build module's recipe version, in place of the
// toolchain image's digest. A toolchain update so changes the fingerprints of that language's modules only.
type Toolchain2 struct {
Language string
Version string
Recipe string
}
// FingerprintKeyPart is a new-form fingerprint as a key part: its 64 hex digits without the prefix, since a bus key
// may not hold ':'. It refuses a value that is not `src2:` and 64 lowercase hex digits, a src1 fingerprint included,
// naming it.
func FingerprintKeyPart(fp string) (string, error) {
part, ok := strings.CutPrefix(fp, SourcePrefix2)
if !ok || !isHex64(part) {
return "", fmt.Errorf("delivery: %q is not a fingerprint of the new form, %s and 64 lowercase hex digits", fp, SourcePrefix2)
}
return part, nil
}
// ParseFingerprintKeyPart is the inverse of FingerprintKeyPart. It refuses a part that is not 64 lowercase hex
// digits, a src1 fingerprint or a prefixed value included, naming it.
func ParseFingerprintKeyPart(part string) (string, error) {
if !isHex64(part) {
return "", fmt.Errorf("delivery: %q is not the key part of a fingerprint, 64 lowercase hex digits", part)
}
return SourcePrefix2 + part, nil
}
func isHex64(s string) bool {
return len(s) == 64 && strings.Trim(s, "0123456789abcdef") == ""
}
-34
View File
@@ -1,34 +0,0 @@
package delivery
import (
"strings"
"testing"
)
// 1356/AC6: the key part holds only bus key characters and parsing gives the fingerprint back; a src1 or malformed
// value is refused, naming it.
func TestFingerprintKeyPart(t *testing.T) {
fp := SourcePrefix2 + strings.Repeat("ab", 32)
part, err := FingerprintKeyPart(fp)
if err != nil {
t.Fatal(err)
}
if len(part) != 64 || strings.Trim(part, "0123456789abcdef") != "" || !nameOK.MatchString(part) {
t.Errorf("key part %q is not 64 lowercase hex characters a key allows", part)
}
back, err := ParseFingerprintKeyPart(part)
if err != nil || back != fp {
t.Errorf("parse(%q) = %q, %v; want %q", part, back, err, fp)
}
src1 := SourcePrefix + strings.Repeat("c", 64)
for _, bad := range []string{src1, "src2:", "src2:xyz", "", "src2:" + strings.Repeat("A", 64), "src2:" + strings.Repeat("a", 63)} {
if _, err := FingerprintKeyPart(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("FingerprintKeyPart(%q) = %v, want a refusal naming it", bad, err)
}
}
for _, bad := range []string{src1, "src2:" + strings.Repeat("a", 64), strings.Repeat("a", 63), strings.Repeat("A", 64), "", "zz"} {
if _, err := ParseFingerprintKeyPart(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseFingerprintKeyPart(%q) = %v, want a refusal naming it", bad, err)
}
}
}
-136
View File
@@ -1,136 +0,0 @@
package delivery
import (
"strings"
"testing"
)
const oddManifest = "{\n\t\"module\": \"a\",\n \"z\": 1, \"a\": [ \"x\",\"y\" ],\n \"why\": \"é — <b> & \\n\"\n}\n\n"
func mergedWithInputs() Forge {
return Forge{Kind: ForgeMerged, Repository: "novox/mesh-sdk", Number: 36, Head: "aaaa", Merge: "cccc", Root: "t0",
Trees: map[string]string{"a": "t1", "b": "t2"},
Manifests: map[string]string{"a": oddManifest},
Locks: map[string]map[string]string{"a": {"go.sum": strings.Repeat("ab", 32)}}}
}
// 1442/AC1
func TestForgeMergedCarriesManifestsAndLocks(t *testing.T) {
f := mergedWithInputs()
data, err := EncodeForge(f)
if err != nil {
t.Fatal(err)
}
for _, frag := range []string{`"manifests":{"a":`, `"locks":{"a":{"go.sum":`} {
if !strings.Contains(string(data), frag) {
t.Errorf("the encoding lacks %s: %s", frag, data)
}
}
got, err := DecodeForge(data)
if err != nil {
t.Fatal(err)
}
if got.Manifests["a"] != oddManifest || !SameForge(got, withV(f)) {
t.Errorf("the message changed on the way: %q", got.Manifests["a"])
}
// ADR 0565 §2 (issue 1953 D1): a merged message carries the predefined list exact, or says the merge holds none; a
// pull message carries neither.
f.Predefined = []byte(`{"domain":{"type":"string"}}`)
if data, err := EncodeForge(f); err != nil {
t.Fatal(err)
} else if got, err := DecodeForge(data); err != nil || string(got.Predefined) != `{"domain":{"type":"string"}}` {
t.Errorf("the predefined list read back as %s, %v", got.Predefined, err)
}
f.Predefined, f.PredefinedRead = nil, true
if _, err := EncodeForge(f); err != nil {
t.Errorf("a merge holding no predefined list: %v", err)
}
pull := Forge{Kind: ForgePull, Repository: "novox/mesh-sdk", Number: 1, Head: "h", Action: PullOpened, PredefinedRead: true}
if _, err := EncodeForge(pull); err == nil || !strings.Contains(err.Error(), "only a merged message") {
t.Errorf("a pull message carrying the predefined list: %v", err)
}
}
func withV(f Forge) Forge { f.V = Version; return f }
// 1442/AC2
func TestForgeMergedRefusesBadInputs(t *testing.T) {
for name, tc := range map[string]struct {
mut func(*Forge)
want string
}{
"manifest outside trees": {func(f *Forge) { f.Manifests["zz"] = "{}" }, `"zz"`},
"locks outside trees": {func(f *Forge) { f.Locks["yy"] = map[string]string{"go.sum": "h"} }, `"yy"`},
"empty manifest": {func(f *Forge) { f.Manifests["b"] = "" }, `"b"`},
"empty lock hash": {func(f *Forge) { f.Locks["a"]["go.sum"] = "" }, `"a"`},
// No repository holds placement (ADR 0567 §11): a message carrying placement files is refused, on a merge or a pull.
"placement files": {func(f *Forge) { f.RetiredPlacements = []byte(`{"anchor":"{}"}`) }, "mesh-placement"},
"placements read": {func(f *Forge) { f.RetiredRead = true }, "mesh-placement"},
// ADR 0565 §2 (issue 1953 D1): the predefined list is one JSON object, said present or absent, never both.
"predefined not an object": {func(f *Forge) { f.Predefined = []byte(`["domain"]`) }, "one JSON object"},
"predefined both": {func(f *Forge) { f.Predefined = []byte(`{}`); f.PredefinedRead = true }, "says the merge holds none"},
// A deleted branch names its branch and carries nothing else (design 51 *Tagged versions*).
"branch-deleted with no branch": {func(f *Forge) { *f = Forge{Kind: ForgeBranchDeleted, Repository: "novox/mesh-sdk"} }, "needs the branch"},
"branch-deleted with a head": {func(f *Forge) {
*f = Forge{Kind: ForgeBranchDeleted, Repository: "novox/mesh-sdk", Branch: "feat/x", Head: "aaaa"}
}, "carries nothing else"},
} {
f := mergedWithInputs()
tc.mut(&f)
if _, err := EncodeForge(f); err == nil || !strings.Contains(err.Error(), tc.want) {
t.Errorf("%s: %v, want a refusal naming %s", name, err, tc.want)
}
}
}
// 1442/AC3
func TestForgePullCarriesManifestsAndLocks(t *testing.T) {
pull := func(action string) Forge {
return Forge{Kind: ForgePull, Repository: "novox/mesh-sdk", Number: 1, Head: "h", Action: action,
Dirs: []string{"a"}, DirsKnown: true, Manifests: map[string]string{"a": oddManifest},
Locks: map[string]map[string]string{"a": {"go.sum": strings.Repeat("0", 64)}}}
}
for _, action := range []string{PullOpened, PullUpdated} {
data, err := EncodeForge(pull(action))
if err != nil {
t.Fatalf("%s: %v", action, err)
}
if got, err := DecodeForge(data); err != nil || got.Manifests["a"] != oddManifest {
t.Errorf("%s: %v", action, err)
}
}
f := pull(PullOpened)
f.Manifests["b"] = "{}"
if _, err := EncodeForge(f); err == nil || !strings.Contains(err.Error(), `"b"`) {
t.Errorf("a manifest outside Dirs: %v", err)
}
f = pull(PullOpened)
f.Locks["c"] = map[string]string{"go.sum": "h"}
if _, err := EncodeForge(f); err == nil || !strings.Contains(err.Error(), `"c"`) {
t.Errorf("locks outside Dirs: %v", err)
}
for name, f := range map[string]Forge{
"refused manifests": {Kind: ForgeRefused, Repository: "a/b", Act: "x", Words: "no", Manifests: map[string]string{"a": "{}"}},
"refused locks": {Kind: ForgeRefused, Repository: "a/b", Act: "x", Words: "no", Locks: map[string]map[string]string{"a": {"go.sum": "h"}}},
"open manifests": {Kind: ForgeOpen, Manifests: map[string]string{"a": "{}"}},
"open locks": {Kind: ForgeOpen, Locks: map[string]map[string]string{"a": {"go.sum": "h"}}},
} {
if _, err := EncodeForge(f); err == nil {
t.Errorf("%s was encoded", name)
}
}
}
// 1442/AC4
func TestForgeFromAnOlderBridge(t *testing.T) {
old := `{"v":1,"kind":"merged","repository":"novox/mesh-sdk","number":36,"head":"aaaa","merge":"cccc","trees":{"a":"t1"},"root":"t0"}`
f, err := DecodeForge([]byte(old))
if err != nil || f.Manifests != nil || f.Locks != nil {
t.Fatalf("%+v, %v", f, err)
}
data, err := EncodeForge(f)
// the older message is written with the struct's key order, so the bytes compare as they are
if err != nil || string(data) != old {
t.Errorf("%s, %v, want exactly %s", data, err, old)
}
}
-527
View File
@@ -1,527 +0,0 @@
package delivery
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"reflect"
"regexp"
"sort"
"strconv"
"strings"
)
// The gate's buckets and the teller's (ADR 0437 §6, §8). The gate writes merges (each pull request's expected
// head, its state and each merge commit), its asks (`merge.<group>.<heads hash>`, the router's ask whole) and owed
// (one key per act it owes the forge, which the forge bridge performs once per revision). The teller writes its
// answer under the ask's own key in its bucket answers.
const (
MergesBucket = GateModule + "_merges"
AsksBucket = GateModule + "_asks"
OwedBucket = GateModule + "_owed"
AnswersBucket = TellerModule + "_answers"
MergesHistory = 10
AsksHistory = 5
OwedHistory = 5
AnswersHistory = 5
)
// The contexts of the statuses the mesh owes the forge: the gate's, the delivery's view of a head and of a group
// (design 47), and the checker's. From R4e the checker's mesh/check is the one status a repository's main requires,
// in place of mesh/merge-gate and mesh/repo-check (ADR 0503 clauses 3 and 5); a main that requires only
// mesh/repo-check (hq) keeps it, and the controller writes it for a repository the checker writes no status for.
const (
StatusContextMergeGate = "mesh/merge-gate"
StatusContextDelivery = "mesh/delivery"
StatusContextDeliveryGroup = "mesh/delivery-group"
StatusContextRepoCheck = "mesh/repo-check" // the gate's pending status of a recheck (mesh-delivery.recheck)
StatusContextCheck = "mesh/check"
)
var statusContexts = []string{StatusContextMergeGate, StatusContextDelivery, StatusContextDeliveryGroup, StatusContextRepoCheck, StatusContextCheck}
// AskKindMerge is the kind of the gate's ask for a group's merge, the first part of its key.
const AskKindMerge = "merge"
// The acts the gate owes the forge, the last part of an owed key.
const (
OwedStatus = "status"
OwedView = "view"
OwedNote = "note"
OwedComment = "comment"
)
// The states of a pull request in a merge.
type MergeState string
// The states a merge's pull request is in.
const (
MergeMerging MergeState = "merging"
MergeMerged MergeState = "merged"
MergeRefused MergeState = "refused"
)
// The states a status the gate owes the forge may carry.
var statusStates = []string{"pending", "success", "failure", "error", "warning"}
var hashOK = regexp.MustCompile(`^[0-9a-f]{12}$`)
// MergesKey is the key, in MergesBucket, of a group: `<group>`.
func MergesKey(group string) (string, error) {
return groupKey(group)
}
// MergeAskKey is the key, in AsksBucket, of the ask for merging a group at these heads: `merge.<group>.<heads
// hash>`. The teller's answer has the same key in AnswersBucket (AnswerKey).
func MergeAskKey(group string, heads []Head) (string, error) {
part, err := groupKey(group)
if err != nil {
return "", err
}
hash, err := HeadsHash(heads)
if err != nil {
return "", err
}
return AskKindMerge + "." + part + "." + hash, nil
}
// SettleGroupPrefix begins the group of a settle ask (novox/hq ADR 0556 clause 5): a person's ask to settle deliveries held
// because their merge reached the trunk without a passing check. It is kept in the gate's asks under MergeAskKey like a merge
// ask, so the teller carries it unchanged; its options bind the act AskVerb names for it. Git forbids a colon in a branch
// name, so no pull request's group (its branch) ever begins with it.
const SettleGroupPrefix = "settle:"
// SettleGroup is the group of the settle ask of these delivery ids: the prefix and twelve hex digits of the sha256 over the
// ids, sorted and joined by a newline, so the same deliveries asked again keep one key.
func SettleGroup(ids []string) string {
sorted := append([]string(nil), ids...)
sort.Strings(sorted)
sum := sha256.Sum256([]byte(strings.Join(sorted, "\n")))
return SettleGroupPrefix + hex.EncodeToString(sum[:])[:12]
}
// AskVerb is the act the options of an ask of the gate bind, by its group: a settle ask's `mesh-delivery.settle`, any other
// `mesh-delivery.merge`. The gate composes the act with it and the teller checks a warrant against it.
func AskVerb(group string) string {
if strings.HasPrefix(group, SettleGroupPrefix) {
return "mesh-delivery.settle"
}
return "mesh-delivery.merge"
}
// ParseAskKey is the inverse of MergeAskKey: the kind, the group and the heads hash.
func ParseAskKey(key string) (kind, group, hash string, err error) {
parts := strings.Split(key, ".")
if len(parts) != 3 || parts[0] != AskKindMerge {
return "", "", "", fmt.Errorf("delivery: %q is not an ask key, merge.<group>.<heads hash>", key)
}
group, err = parseGroupKey(parts[1])
if err != nil {
return "", "", "", fmt.Errorf("delivery: %q is not an ask key: %w", key, err)
}
if !hashOK.MatchString(parts[2]) {
return "", "", "", fmt.Errorf("delivery: %q is not an ask key: %q is not 12 hex digits", key, parts[2])
}
return parts[0], group, parts[2], nil
}
// AnswerKey is the key, in AnswersBucket, of the teller's answer to the ask at askKey: the same string.
func AnswerKey(askKey string) string { return askKey }
// AskID is the id the teller gives the router for the ask at askKey. The router's ids refuse dots (its usable id),
// so in the readable part a dot becomes `-` and any other byte outside letters, digits, `_` and `-` becomes `_`; that part is cut to fit, and every id ends in `-` and
// the first 12 hex digits of the sha256 of the whole key, so two keys never give one id. It cannot be read back:
// the teller keeps the key beside it.
func AskID(askKey string) string {
const most = 64 // the router's bound on an id
const tail = 13 // "-" and 12 hex digits
readable := []byte(askKey)
for i, c := range readable {
switch {
case c >= 'A' && c <= 'Z', c >= 'a' && c <= 'z', c >= '0' && c <= '9':
case c == '.':
readable[i] = '-'
default: // a slash, an equals sign or any other byte the router's id refuses
readable[i] = '_'
}
}
if len(readable) > 0 && (readable[0] == '-' || readable[0] == '_') {
readable[0] = 'k' // an id starts with a letter or digit
}
if len(readable) > most-tail {
readable = readable[:most-tail]
}
sum := sha256.Sum256([]byte(askKey))
return string(readable) + "-" + hex.EncodeToString(sum[:])[:12]
}
// AskingID is the id of the asking-th writing of the ask at askKey. Asking 0 and 1 give AskID(askKey), so the first
// asking's id is unchanged; asking n of 2 or more gives the AskID form of askKey + "." + n, so each asking has an id
// of its own (at most 64 bytes, ending in the 12 hex digits of the hash of that whole string) and the router, which
// drops an ask under an id it keeps as a redelivery, takes each as new and an answer to an earlier asking is never
// taken for a later one.
func AskingID(askKey string, asking int) string {
if asking < 2 {
return AskID(askKey)
}
return AskID(askKey + "." + strconv.Itoa(asking))
}
// checkStatusContext refuses a status context the mesh does not owe the forge.
func checkStatusContext(context string) error {
for _, c := range statusContexts {
if context == c {
return nil
}
}
return fmt.Errorf("delivery: the status context %q is not one of %s", context, strings.Join(statusContexts, ", "))
}
func checkOwedAct(act string) (kind, context string, err error) {
kind, context, hasContext := strings.Cut(act, "=")
switch kind {
case OwedStatus:
if !hasContext {
return "", "", fmt.Errorf("delivery: the owed act %q is a status without its context, status=<context>", act)
}
if err := checkStatusContext(context); err != nil {
return "", "", err
}
return kind, context, nil
case OwedView, OwedNote:
if hasContext {
return "", "", fmt.Errorf("delivery: the owed act %q takes no context", act)
}
return kind, "", nil
}
if kind, seq, ok := strings.Cut(act, "."); ok && kind == OwedComment {
if n, err := strconv.Atoi(seq); err != nil || n < 1 || strconv.Itoa(n) != seq {
return "", "", fmt.Errorf("delivery: the owed act %q is a comment without its sequence, comment.<seq> with seq of 1 or more", act)
}
return OwedComment, seq, nil
}
return "", "", fmt.Errorf("delivery: the owed act %q is not status=<context>, view, note or comment.<seq>", act)
}
// OwedCommentAct is the act of the seq-th comment on a head: `comment.<seq>`, seq of 1 or more, so a second comment
// has a key of its own and does not overwrite the first.
func OwedCommentAct(seq int) (string, error) {
if seq < 1 {
return "", fmt.Errorf("delivery: a comment's sequence is 1 or more, not %d", seq)
}
return OwedComment + "." + strconv.Itoa(seq), nil
}
// ParseOwedAct splits an owed act into its kind and, for a status, its context, for a comment its sequence as
// digits.
func ParseOwedAct(act string) (kind, context string, err error) { return checkOwedAct(act) }
// OwedKey is the key, in OwedBucket, of an act the gate owes the forge on a pull request at a head:
// `<repository>.<number>.<head>.<act>`, the act being `status=<context>` (mesh/merge-gate), `view`, `note` or
// `comment.<seq>`. The number is 0 for a commit no pull request names, and never negative.
func OwedKey(repository string, number int, head, act string) (string, error) {
repo, err := escapeRepository(repository)
if err != nil {
return "", err
}
if number < 0 {
return "", fmt.Errorf("delivery: an owed key's pull request number is 0 (none) or more, not %d", number)
}
if err := checkName("head", head); err != nil {
return "", err
}
if _, _, err := checkOwedAct(act); err != nil {
return "", err
}
return repo + "." + strconv.Itoa(number) + "." + head + "." + act, nil
}
// ParseOwedKey is the inverse of OwedKey; act is whole, as `status=<context>` for a status.
func ParseOwedKey(key string) (repository string, number int, head, act string, err error) {
parts := strings.SplitN(key, ".", 4) // a comment's act holds a dot
if len(parts) != 4 {
return "", 0, "", "", fmt.Errorf("delivery: %q is not an owed key, <repository>.<number>.<head>.<act>", key)
}
number, convErr := strconv.Atoi(parts[1])
if convErr != nil {
return "", 0, "", "", fmt.Errorf("delivery: %q is not an owed key: %q is not a pull request number", key, parts[1])
}
repository, err = unescapeRepository(parts[0])
if err != nil {
return "", 0, "", "", fmt.Errorf("delivery: %q is not an owed key: %w", key, err)
}
if _, err := OwedKey(repository, number, parts[2], parts[3]); err != nil {
return "", 0, "", "", fmt.Errorf("delivery: %q is not an owed key: %w", key, err)
}
return repository, number, parts[2], parts[3], nil
}
// Merge is the value of a merges key: the ask approved and, for each pull request, the head the approval expects
// and how far its merge has come. Approved is the Approve's heads of every member of the group, ApprovedKey of
// the member to its full head sha: the forge's merge tool wants them for every sibling whichever member it is
// handed (ADR 0450 §3). A merge written before Approved existed has none and is not checked against it.
type Merge struct {
V int `json:"v"`
Group string `json:"group"`
Ask string `json:"ask"`
Approved map[string]string `json:"approved,omitempty"`
Pulls []MergePull `json:"pulls"`
}
// ApprovedKey is the key of a member in Merge.Approved: `<repository>@<number>`.
func ApprovedKey(repository string, number int) string {
return repository + "@" + strconv.Itoa(number)
}
// MergePull is one pull request of a merge: its expected head, its state, the merge commit once merged, the
// forge's words when refused.
type MergePull struct {
Head
State MergeState `json:"state"`
Commit string `json:"commit,omitempty"`
Words string `json:"words,omitempty"`
}
func (m Merge) check() error {
if m.Group == "" {
return fmt.Errorf("delivery: a merge names no group")
}
if m.Ask == "" {
return fmt.Errorf("delivery: a merge names no ask")
}
if len(m.Pulls) == 0 {
return fmt.Errorf("delivery: a merge names no pull request")
}
for key, sha := range m.Approved {
if sha == "" {
return fmt.Errorf("delivery: the approved head of %s is empty", key)
}
}
for _, p := range m.Pulls {
who := fmt.Sprintf("%s#%d", p.Repository, p.Number)
if len(m.Approved) > 0 {
sha, ok := m.Approved[ApprovedKey(p.Repository, p.Number)]
if !ok {
return fmt.Errorf("delivery: the pull request %s is handed to the merge but is not among the approved heads", who)
}
if sha != p.Head.Head {
return fmt.Errorf("delivery: the pull request %s is handed at head %q but the Approve was for %q", who, p.Head.Head, sha)
}
}
switch p.State {
case MergeMerging:
case MergeMerged:
if p.Commit == "" {
return fmt.Errorf("delivery: the merged pull request %s has no merge commit", who)
}
case MergeRefused:
if p.Words == "" {
return fmt.Errorf("delivery: the refused pull request %s has no words of the forge", who)
}
default:
return fmt.Errorf("delivery: the state %q of %s is not merging, merged or refused", p.State, who)
}
}
return nil
}
// Ask is the value of an asks key: the router's ask whole as the gate composes it, so the teller publishes it
// unchanged, and the group and heads it was composed for. The ask is the operator-channel seat's (mesh-contracts
// operatorchannel.Ask), carried here as its JSON: mesh-sdk names no seat defined whole (novox/hq ADR 0627 §4), so its
// id (AskingID of its key and asking), its options and its digest are checked by the gate and the teller through
// operator-channel's client.
type Ask struct {
V int `json:"v"`
Group string `json:"group"`
Heads []Head `json:"heads"`
Ask json.RawMessage `json:"ask"`
// Asking counts the times the gate wrote this group's ask at these heads: 0 or 1 for the first, n for the n-th.
// The router drops an ask whose id it has already kept as a redelivery, so each asking has its own id
// (AskingID); the first asking's id is unchanged and the field is omitted for it.
Asking int `json:"asking,omitempty"`
}
func (a Ask) check() error {
if a.Asking < 0 {
return fmt.Errorf("delivery: the ask's asking %d is negative", a.Asking)
}
if _, err := MergeAskKey(a.Group, a.Heads); err != nil {
return fmt.Errorf("delivery: an ask needs its group and at least one pull request's head: %w", err)
}
return routerAsk(a.Ask)
}
// routerAsk refuses a value that carries no router's ask: none, null, or JSON that is no object. What the ask says
// is operator-channel's to check, through its client (novox/hq ADR 0627 §4).
func routerAsk(raw json.RawMessage) error {
var fields map[string]json.RawMessage
if len(bytes.TrimSpace(raw)) == 0 || json.Unmarshal(raw, &fields) != nil || fields == nil {
return fmt.Errorf("delivery: the value carries no router's ask")
}
return nil
}
// Answer is the teller's answer to an ask: the option chosen, the warrant's id and its words.
type Answer struct {
V int `json:"v"`
Option string `json:"option"`
Warrant string `json:"warrant"`
Words string `json:"words"`
}
func (a Answer) check() error {
for _, f := range []struct{ name, value string }{{"option", a.Option}, {"warrant", a.Warrant}, {"words", a.Words}} {
if f.value == "" {
return fmt.Errorf("delivery: an answer has no %s", f.name)
}
}
return nil
}
// Owed is the value of an owed key: what the forge bridge performs. A status needs Context, State and Target (the
// address the status points at), a view and a comment need Text, a note needs Text and Ref (the notes ref,
// mesh-plan).
type Owed struct {
V int `json:"v"`
Act string `json:"act"`
Context string `json:"context,omitempty"`
Target string `json:"target,omitempty"`
State string `json:"state,omitempty"`
Text string `json:"text,omitempty"`
Ref string `json:"ref,omitempty"`
}
func (o Owed) check() error {
switch o.Act {
case OwedStatus:
if o.Context == "" {
return fmt.Errorf("delivery: an owed status has no context")
}
if err := checkStatusContext(o.Context); err != nil {
return err
}
if o.Target == "" {
return fmt.Errorf("delivery: an owed status has no target")
}
for _, s := range statusStates {
if o.State == s {
return nil
}
}
return fmt.Errorf("delivery: the owed status state %q is not %s", o.State, strings.Join(statusStates, ", "))
case OwedView, OwedComment:
if o.Text == "" {
return fmt.Errorf("delivery: an owed %s has no text", o.Act)
}
case OwedNote:
if o.Text == "" {
return fmt.Errorf("delivery: an owed note has no text")
}
if o.Ref == "" {
return fmt.Errorf("delivery: an owed note has no ref")
}
default:
return fmt.Errorf("delivery: the owed act %q is not status, view, note or comment", o.Act)
}
return nil
}
// EncodeMerge is m as JSON at Version; it refuses a merged pull request without its merge commit, a refused one
// without the forge's words and a state outside the list.
func EncodeMerge(m Merge) ([]byte, error) {
if err := m.check(); err != nil {
return nil, err
}
m.V = Version
return json.Marshal(m)
}
// DecodeMerge reads a merge; it refuses an unknown version (ErrUnknownVersion) and what EncodeMerge refuses.
func DecodeMerge(data []byte) (Merge, error) {
var m Merge
if err := decode(data, &m); err != nil {
return Merge{}, err
}
if err := m.check(); err != nil {
return Merge{}, err
}
return m, nil
}
// EncodeAsk is a as JSON at Version; it refuses an ask whose id is not the one its group and heads give, or with
// no option.
func EncodeAsk(a Ask) ([]byte, error) {
if err := a.check(); err != nil {
return nil, err
}
a.V = Version
return json.Marshal(a)
}
// DecodeAsk reads an ask; it refuses an unknown version (ErrUnknownVersion) and what EncodeAsk refuses.
func DecodeAsk(data []byte) (Ask, error) {
var a Ask
if err := decode(data, &a); err != nil {
return Ask{}, err
}
if err := a.check(); err != nil {
return Ask{}, err
}
return a, nil
}
// EncodeAnswer is a as JSON at Version; it refuses an answer missing its option, warrant or words.
func EncodeAnswer(a Answer) ([]byte, error) {
if err := a.check(); err != nil {
return nil, err
}
a.V = Version
return json.Marshal(a)
}
// DecodeAnswer reads an answer; it refuses an unknown version (ErrUnknownVersion) and what EncodeAnswer refuses.
func DecodeAnswer(data []byte) (Answer, error) {
var a Answer
if err := decode(data, &a); err != nil {
return Answer{}, err
}
if err := a.check(); err != nil {
return Answer{}, err
}
return a, nil
}
// EncodeOwed is o as JSON at Version; it refuses an act missing what it needs.
func EncodeOwed(o Owed) ([]byte, error) {
if err := o.check(); err != nil {
return nil, err
}
o.V = Version
return json.Marshal(o)
}
// DecodeOwed reads an owed act; it refuses an unknown version (ErrUnknownVersion) and what EncodeOwed refuses.
func DecodeOwed(data []byte) (Owed, error) {
var o Owed
if err := decode(data, &o); err != nil {
return Owed{}, err
}
if err := o.check(); err != nil {
return Owed{}, err
}
return o, nil
}
// SameMerge says whether two merges are the same value, so a writer writes only when they differ.
func SameMerge(a, b Merge) bool { return reflect.DeepEqual(a, b) }
// SameAsk says whether two asks are the same value.
func SameAsk(a, b Ask) bool { return reflect.DeepEqual(a, b) }
// SameAnswer says whether two answers are the same value.
func SameAnswer(a, b Answer) bool { return a == b }
// SameOwed says whether two owed acts are the same value.
func SameOwed(a, b Owed) bool { return a == b }
-648
View File
@@ -1,648 +0,0 @@
package delivery
import (
"encoding/json"
"errors"
"regexp"
"strconv"
"strings"
"testing"
"time"
)
func gateHeads() []Head {
return []Head{
{Repository: "novox/mesh-sdk", Number: 36, Head: "aaaa"},
{Repository: "novox/mesh-control", Number: 7, Head: "bbbb"},
}
}
func TestMergeKeys(t *testing.T) { // 1201/AC1
if k, err := MergesKey("feat/x"); err != nil || k != "feat/x" {
t.Fatalf("merges key = %q, %v", k, err)
}
if _, err := MergesKey(""); err == nil {
t.Error("an empty group accepted")
}
hh, _ := HeadsHash(gateHeads())
key, err := MergeAskKey("feat/x", gateHeads())
if err != nil || key != "merge.feat/x."+hh {
t.Fatalf("ask key = %q, %v", key, err)
}
kind, group, hash, err := ParseAskKey(key)
if err != nil || kind != AskKindMerge || group != "feat/x" || hash != hh {
t.Fatalf("parse ask key = %q %q %q %v", kind, group, hash, err)
}
if AnswerKey(key) != key {
t.Error("the answer's key is not the ask's key")
}
if _, err := MergeAskKey("feat/x", nil); err == nil {
t.Error("an ask key without heads accepted")
}
if _, err := MergeAskKey("", gateHeads()); err == nil {
t.Error("an ask key without a group accepted")
}
for _, bad := range []string{"", "merge", "merge.g", "merge.g.h.x", "other.g.0123456789ab", "merge..0123456789ab", "merge.g.nothex", "merge.a=zz.0123456789ab", "merge.a=4.0123456789ab", "merge.a=4A.0123456789ab", "merge.a+b.0123456789ab"} {
if _, _, _, err := ParseAskKey(bad); err == nil {
t.Errorf("parse ask key %q accepted", bad)
}
}
}
func TestAskID(t *testing.T) { // 1201/AC2
k1, _ := MergeAskKey("feat/x", gateHeads())
k2, _ := MergeAskKey("feat/y", gateHeads())
k3, _ := MergeAskKey("feat/x", gateHeads()[:1])
long, _ := MergeAskKey("feat/"+strings.Repeat("long-", 20), gateHeads())
long2, _ := MergeAskKey("feat/"+strings.Repeat("long-", 20)+"2", gateHeads())
slash, _ := MergeAskKey("a/b", gateHeads())
under, _ := MergeAskKey("a_b", gateHeads())
eq, _ := MergeAskKey("feat=x", gateHeads())
eqs, _ := MergeAskKey("feat_x", gateHeads())
seen := map[string]string{}
for _, k := range []string{k1, k2, k3, long, long2, slash, under, eq, eqs, "=odd.key"} {
id := AskID(k)
if strings.Contains(id, ".") {
t.Errorf("id %q holds a dot", id)
}
if !routerUsableID(id) {
t.Errorf("id %q is not usable by the router", id)
}
if other, dup := seen[id]; dup {
t.Errorf("keys %q and %q give the id %q", k, other, id)
}
seen[id] = k
}
if AskID(k1) != AskID(k1) {
t.Error("the id is not stable")
}
// 1280 (ADR 0556 clause 5): a settle ask's group is the same for the same ids in any order, keys like any group, and its
// options bind mesh-delivery.settle; every other ask binds mesh-delivery.merge.
g := SettleGroup([]string{"novox/b@222222222222", "novox/a@111111111111"})
if g != SettleGroup([]string{"novox/a@111111111111", "novox/b@222222222222"}) || !strings.HasPrefix(g, SettleGroupPrefix) || len(g) != len(SettleGroupPrefix)+12 {
t.Errorf("SettleGroup = %q", g)
}
if k, err := MergeAskKey(g, gateHeads()); err != nil || !routerUsableID(AskID(k)) {
t.Errorf("the settle ask's key %q: %v", k, err)
}
for group, want := range map[string]string{g: "mesh-delivery.settle", "feat/x": "mesh-delivery.merge",
"settle/foo": "mesh-delivery.merge", "settle": "mesh-delivery.merge"} { // a branch named settle/... is a merge's group
if got := AskVerb(group); got != want {
t.Errorf("AskVerb(%q) = %q, want %q", group, got, want)
}
}
}
func TestOwedKey(t *testing.T) { // 1201/AC3
for _, act := range []string{"status=mesh/merge-gate", "view", "note", "comment.1", "comment.12"} {
key, err := OwedKey("novox/mesh-sdk", 36, "aaaa", act)
if err != nil || key != "novox/mesh-sdk.36.aaaa."+act {
t.Fatalf("owed key for %s = %q, %v", act, key, err)
}
repo, number, head, gotAct, err := ParseOwedKey(key)
if err != nil || repo != "novox/mesh-sdk" || number != 36 || head != "aaaa" || gotAct != act {
t.Fatalf("parse %q = %q %d %q %q %v", key, repo, number, head, gotAct, err)
}
}
kind, context, err := ParseOwedAct("status=mesh/merge-gate")
if err != nil || kind != OwedStatus || context != "mesh/merge-gate" {
t.Errorf("owed act = %q %q %v", kind, context, err)
}
for _, bad := range []string{"", "merge", "status", "status=", "view=x", "status=a.b", "statuses=x", "comment", "comment.", "comment.0", "comment.-1", "comment.01", "comment.x", "comment=1"} {
if _, err := OwedKey("novox/mesh-sdk", 36, "aaaa", bad); err == nil {
t.Errorf("owed act %q accepted", bad)
}
}
if _, err := OwedKey("nope", 36, "aaaa", "view"); err == nil {
t.Error("a bad repository accepted")
}
if _, err := OwedKey("novox/mesh-sdk", -1, "aaaa", "view"); err == nil {
t.Error("a negative pull request number accepted")
}
if _, err := OwedKey("novox/mesh-sdk", 36, "", "view"); err == nil {
t.Error("an empty head accepted")
}
for _, bad := range []string{"", "novox/mesh-sdk.36.aaaa", "novox/mesh-sdk.x.aaaa.view", "novox/mesh-sdk.-1.aaaa.view", "novox/mesh-sdk.36.aaaa.comment", "novox/mesh-sdk.36.aaaa.merge", "a.b.36.aaaa.view"} {
if _, _, _, _, err := ParseOwedKey(bad); err == nil {
t.Errorf("parse owed key %q accepted", bad)
}
}
}
func sampleMerge() Merge {
return Merge{Group: "feat/x", Ask: "merge.feat/x.0123456789ab", Pulls: []MergePull{
{Head: gateHeads()[0], State: MergeMerged, Commit: "cccc"},
{Head: gateHeads()[1], State: MergeRefused, Words: "the forge says no"},
{Head: Head{Repository: "novox/other", Number: 1, Head: "dddd"}, State: MergeMerging},
}}
}
func sampleTurn() Turn {
at := time.Date(2026, 10, 26, 9, 0, 0, 0, time.UTC)
return Turn{Repository: "novox/mesh-sdk", Waiting: []TurnWait{
{Group: "feat/x", Heads: mustHash(gateHeads()), Ask: "merge.feat/x." + mustHash(gateHeads()), Approved: at, Since: at,
WaitsFor: "feat/y.0123456789ab", WaitsSince: at.Add(-time.Minute), Step: "the check ask of novox/mesh-sdk#40"},
{Group: "fix/1.2", Heads: "0123456789ac", Approved: at.Add(time.Minute), Since: at.Add(time.Minute)},
}}
}
func sampleAsk() Ask {
key, _ := MergeAskKey("feat/x", gateHeads())
return Ask{Group: "feat/x", Heads: gateHeads(), Ask: routerAskJSON(AskID(key), "Merge into main: group feat/x")}
}
// routerAskJSON is a router's ask as the gate carries it: the operator-channel seat's JSON, which mesh-sdk does not
// type (novox/hq ADR 0627 §4).
func routerAskJSON(id, headline string) json.RawMessage {
raw, _ := json.Marshal(map[string]any{"id": id, "headline": headline, "explanation": "The group passed its checks.",
"who": "operator", "expires": "2026-10-20T08:00:00Z", "on-expiry": "nothing is merged",
"options": []map[string]string{{"id": "approve", "label": "Approve", "does": "merges", "level": "approve", "binds": "sha256:00"}}})
return raw
}
// routerUsableID is the router's rule for an ask's id (operator-channel's client, UsableID), held here to check that
// AskID and AskingID give ids the router takes.
var routerUsableID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$`).MatchString
func sampleOwed() []Owed {
return []Owed{
{Act: OwedStatus, Context: "mesh/merge-gate", State: "success", Target: "https://mesh.example/delivery/36"},
{Act: OwedView, Text: "the delivery reaches ace"},
{Act: OwedNote, Text: "plan: mesh-mail to ace", Ref: "mesh-plan"},
{Act: OwedComment, Text: "merged"},
}
}
func TestGateValuesRoundTrip(t *testing.T) { // 1201/AC4
hasV := func(name string, data []byte, err error) {
t.Helper()
if err != nil {
t.Fatalf("%s: %v", name, err)
}
var probe map[string]any
if json.Unmarshal(data, &probe) != nil || probe["v"] != float64(1) {
t.Fatalf("%s: %s does not carry v:1", name, data)
}
}
m := sampleMerge()
data, err := EncodeMerge(m)
hasV("merge", data, err)
m.V = Version
if got, err := DecodeMerge(data); err != nil || !SameMerge(got, m) {
t.Errorf("merge round trip: %+v %v", got, err)
}
a := sampleAsk()
data, err = EncodeAsk(a)
hasV("ask", data, err)
a.V = Version
if got, err := DecodeAsk(data); err != nil || !SameAsk(got, a) {
t.Errorf("ask round trip: %+v %v", got, err)
}
an := Answer{Option: "approve", Warrant: "w-1", Words: "approved on telegram"}
data, err = EncodeAnswer(an)
hasV("answer", data, err)
an.V = Version
if got, err := DecodeAnswer(data); err != nil || !SameAnswer(got, an) {
t.Errorf("answer round trip: %+v %v", got, err)
}
for _, o := range sampleOwed() {
data, err = EncodeOwed(o)
hasV("owed "+o.Act, data, err)
o.V = Version
if got, err := DecodeOwed(data); err != nil || !SameOwed(got, o) {
t.Errorf("owed %s round trip: %+v %v", o.Act, got, err)
}
}
tu := sampleTurn()
data, err = EncodeTurn(tu)
hasV("turn", data, err)
tu.V = Version
if got, err := DecodeTurn(data); err != nil || !SameTurn(got, tu) {
t.Errorf("turn round trip: %+v %v", got, err)
}
for _, field := range []string{`"waits-for":"feat/y.0123456789ab"`, `"waits-since":`, `"approved":`, `"step":`} {
if !strings.Contains(string(data), field) {
t.Errorf("the turn lacks %s: %s", field, data)
}
}
// ADR 0548 §3: a turn holds off every check but those of the heads that wait in it, whoever else waits there.
if !TurnHolds(tu, "fedcba987654") || TurnHolds(tu, tu.Waiting[0].Heads) || TurnHolds(tu, tu.Waiting[1].Heads) || TurnHolds(Turn{}, "fedcba987654") {
t.Error("TurnHolds: a turn holds other heads' checks and never the check of heads waiting in it")
}
for name, dec := range map[string]func([]byte) error{
"turn": func(b []byte) error { _, err := DecodeTurn(b); return err },
"merge": func(b []byte) error { _, err := DecodeMerge(b); return err },
"ask": func(b []byte) error { _, err := DecodeAsk(b); return err },
"answer": func(b []byte) error { _, err := DecodeAnswer(b); return err },
"owed": func(b []byte) error { _, err := DecodeOwed(b); return err },
} {
for _, in := range []string{`{"group":"g"}`, `{"v":2}`} {
if err := dec([]byte(in)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("%s decode of %s = %v, want ErrUnknownVersion", name, in, err)
}
}
}
}
func TestGateValuesRefused(t *testing.T) { // 1201/AC5
wantErr := func(name string, err error, words ...string) {
t.Helper()
if err == nil {
t.Errorf("%s accepted", name)
return
}
for _, w := range words {
if !strings.Contains(err.Error(), w) {
t.Errorf("%s: error %q does not name %q", name, err, w)
}
}
}
m := sampleMerge()
m.Pulls[0].Commit = ""
_, err := EncodeMerge(m)
wantErr("merged without a merge commit", err, "commit")
m = sampleMerge()
m.Pulls[1].Words = ""
_, err = EncodeMerge(m)
wantErr("refused without words", err, "words")
m = sampleMerge()
m.Pulls[2].State = "done"
_, err = EncodeMerge(m)
wantErr("unknown state", err, "done")
m = sampleMerge()
m.Pulls = nil
_, err = EncodeMerge(m)
wantErr("merge without pulls", err, "pull")
m = sampleMerge()
m.Ask = ""
_, err = EncodeMerge(m)
wantErr("merge without its ask", err, "ask")
a := sampleAsk()
for _, raw := range []string{"", "null", "[]", `"ask"`} {
a.Ask = json.RawMessage(raw)
_, err = EncodeAsk(a)
wantErr("a value with no router's ask "+raw, err, "no router's ask")
}
a = sampleAsk()
a.Heads = nil
_, err = EncodeAsk(a)
wantErr("ask without heads", err, "pull request")
for name, an := range map[string]Answer{
"option": {Warrant: "w", Words: "x"},
"warrant": {Option: "approve", Words: "x"},
"words": {Option: "approve", Warrant: "w"},
} {
_, err = EncodeAnswer(an)
wantErr("answer without "+name, err, name)
}
for name, o := range map[string]Owed{
"context": {Act: OwedStatus, State: "success", Target: "t"},
"state": {Act: OwedStatus, Context: "mesh/merge-gate", State: "green", Target: "t"},
"target": {Act: OwedStatus, Context: "mesh/merge-gate", State: "success"},
"text": {Act: OwedView},
"ref": {Act: OwedNote, Text: "line"},
"comment": {Act: OwedComment},
"act": {Act: "merge"},
} {
_, err = EncodeOwed(o)
wantErr("owed missing "+name, err)
}
_, err = EncodeOwed(Owed{Act: OwedNote, Text: "line"})
wantErr("note without ref", err, "ref")
for name, c := range map[string]struct {
edit func(*Turn)
words []string
}{
"no delivery": {func(t *Turn) { t.Waiting = nil }, []string{"no delivery", "deleted"}},
"a repository in capitals": {func(t *Turn) { t.Repository = "novox/Mesh-SDK" }, []string{"lowercase"}},
"no repository": {func(t *Turn) { t.Repository = "" }, []string{"repository"}},
"no group": {func(t *Turn) { t.Waiting[0].Group = "" }, []string{"group"}},
"a bad heads hash": {func(t *Turn) { t.Waiting[0].Heads = "xyz" }, []string{"xyz"}},
"the same heads twice": {func(t *Turn) { t.Waiting[1].Heads = t.Waiting[0].Heads }, []string{"twice"}},
"no Approve time": {func(t *Turn) { t.Waiting[1].Approved = time.Time{} }, []string{"Approve"}},
"out of the Approve's order": {func(t *Turn) { t.Waiting[1].Approved = t.Waiting[0].Approved.Add(-time.Hour) }, []string{"order"}},
"waits for no verdict key": {func(t *Turn) { t.Waiting[0].WaitsFor = "feat/y" }, []string{"verdict key"}},
"a step and no check": {func(t *Turn) { t.Waiting[1].Step = "x" }, []string{"names no check"}},
} {
tu := sampleTurn()
c.edit(&tu)
_, err = EncodeTurn(tu)
wantErr("turn with "+name, err, c.words...)
}
_, err = EncodeOwed(Owed{Act: OwedStatus, State: "success"})
wantErr("status without context", err, "context")
}
func mustHash(h []Head) string { s, _ := HeadsHash(h); return s }
func TestGateSame(t *testing.T) { // 1201/AC6
if !SameMerge(sampleMerge(), sampleMerge()) || !SameAsk(sampleAsk(), sampleAsk()) {
t.Error("equal values differ")
}
m := sampleMerge()
m.Pulls[0].Commit = "other"
if SameMerge(m, sampleMerge()) {
t.Error("different merges are the same")
}
a := sampleAsk()
a.Ask = routerAskJSON(AskID("merge.feat/x."+mustHash(gateHeads())), "other")
if SameAsk(a, sampleAsk()) {
t.Error("different asks are the same")
}
if !SameAnswer(Answer{Option: "a", Warrant: "w", Words: "x"}, Answer{Option: "a", Warrant: "w", Words: "x"}) ||
SameAnswer(Answer{Option: "a", Warrant: "w", Words: "x"}, Answer{Option: "b", Warrant: "w", Words: "x"}) {
t.Error("SameAnswer wrong")
}
if !SameOwed(sampleOwed()[0], sampleOwed()[0]) || SameOwed(sampleOwed()[0], sampleOwed()[1]) {
t.Error("SameOwed wrong")
}
}
// today's keys: names of letters, digits, - / and _ key exactly as before the escape.
func TestKeysOfPlainNamesAreUnchanged(t *testing.T) { // 1323/AC2
// And the buckets and their history are the decided layout (once TestGateBuckets; merged to pay for issue 774's
// token test).
if MergesBucket != "mesh-delivery-gate_merges" || AsksBucket != "mesh-delivery-gate_asks" ||
OwedBucket != "mesh-delivery-gate_owed" || AnswersBucket != "mesh-delivery-teller_answers" ||
MergesHistory != 10 || AsksHistory != 5 || OwedHistory != 5 || AnswersHistory != 5 ||
TurnsBucket != "mesh-delivery-gate_turns" || TurnsHistory != 5 {
t.Error("a bucket or history differs from the decided layout")
}
hh, _ := HeadsHash(gateHeads())
for _, name := range []string{"feat/x", "fix/1323-escape", "feat/delivery_source-keys", "main", "A/B_c-9"} {
if k, err := MergesKey(name); err != nil || k != name {
t.Errorf("merges key %q = %q, %v", name, k, err)
}
if k, err := CandidatesKey(name); err != nil || k != name {
t.Errorf("candidates key %q = %q, %v", name, k, err)
}
if k, err := MergeAskKey(name, gateHeads()); err != nil || k != "merge."+name+"."+hh {
t.Errorf("ask key %q = %q, %v", name, k, err)
}
}
if k, err := OwedKey("novox/mesh-sdk", 36, "aaaa", "status=mesh/merge-gate"); err != nil || k != "novox/mesh-sdk.36.aaaa.status=mesh/merge-gate" {
t.Errorf("owed key = %q, %v", k, err)
}
}
func TestKeysEscapeAndRoundTrip(t *testing.T) { // 1323/AC1
hh, _ := HeadsHash(gateHeads())
for _, name := range []string{"fix/1.2", "feat/a+b", "feat/a=b", "a.b+c=d", "fix/ü", "x y"} {
esc := EscapeName(name)
if strings.ContainsAny(esc, ".+ ") {
t.Errorf("escape of %q = %q holds a dot, plus or space", name, esc)
}
if back, err := UnescapeName(esc); err != nil || back != name {
t.Errorf("unescape(%q) = %q, %v; want %q", esc, back, err, name)
}
if k, err := MergesKey(name); err != nil || k != esc {
t.Errorf("merges key %q = %q, %v", name, k, err)
}
ck, err := CandidatesKey(name)
if err != nil {
t.Fatal(err)
}
if g, err := ParseCandidatesKey(ck); err != nil || g != name {
t.Errorf("candidates round trip %q = %q, %v", name, g, err)
}
ak, err := MergeAskKey(name, gateHeads())
if err != nil {
t.Fatal(err)
}
if _, g, h, err := ParseAskKey(ak); err != nil || g != name || h != hh {
t.Errorf("ask round trip %q = %q %q %v", name, g, h, err)
}
if AnswerKey(ak) != ak {
t.Error("answer key differs from the ask key")
}
// the ask and the answer of such a group encode and decode
ask := Ask{Group: name, Heads: gateHeads(), Ask: routerAskJSON(AskID(ak), "ok")}
if data, err := EncodeAsk(ask); err != nil {
t.Errorf("encode ask for %q: %v", name, err)
} else if _, err := DecodeAsk(data); err != nil {
t.Errorf("decode ask for %q: %v", name, err)
}
repo := "novox/" + name[strings.Index(name, "/")+1:]
if !strings.Contains(name, "/") {
repo = "novox/" + name
}
ok, err := OwedKey(repo, 3, "aaaa", "view")
if err != nil {
t.Errorf("owed key for repository %q: %v", repo, err)
continue
}
if r, n, h, a, err := ParseOwedKey(ok); err != nil || r != repo || n != 3 || h != "aaaa" || a != "view" {
t.Errorf("owed round trip %q = %q %d %q %q %v", repo, r, n, h, a, err)
}
// ADR 0548 §3: a repository's turn key is <owner>.<name>, lowercase, each part escaped.
tk, err := TurnsKey(repo)
if err != nil || strings.Count(tk, ".") != 1 {
t.Errorf("turns key for repository %q = %q, %v", repo, tk, err)
} else if r, err := ParseTurnsKey(tk); err != nil || r != strings.ToLower(repo) {
t.Errorf("turns round trip %q = %q, %v", repo, r, err)
}
}
if k, err := TurnsKey("novox/Mesh-SDK"); err != nil || k != "novox.mesh-sdk" {
t.Errorf("turns key = %q, %v; want novox.mesh-sdk", k, err)
}
for _, bad := range []string{"", "novox", "novox.", ".x", "a.b.c", "Novox.mesh-sdk", "novox.a=zz"} {
if _, err := ParseTurnsKey(bad); err == nil {
t.Errorf("parse turns key %q accepted", bad)
}
}
if EscapeName("a.b") != "a=2eb" || EscapeName("a=b") != "a=3db" || EscapeName("a+b") != "a=2bb" {
t.Errorf("escapes: %q %q %q", EscapeName("a.b"), EscapeName("a=b"), EscapeName("a+b"))
}
m := Merge{Group: "fix/1.2+x=y", Ask: "merge.x.0123456789ab", Pulls: []MergePull{{Head: gateHeads()[0], State: MergeMerging}}}
if data, err := EncodeMerge(m); err != nil {
t.Errorf("encode merge: %v", err)
} else if back, err := DecodeMerge(data); err != nil || back.Group != m.Group {
t.Errorf("decode merge: %v %v", back.Group, err)
}
}
func approvedMerge() Merge {
m := sampleMerge()
m.Approved = map[string]string{
ApprovedKey("novox/mesh-sdk", 36): "aaaa",
ApprovedKey("novox/mesh-control", 7): "bbbb",
ApprovedKey("novox/other", 1): "dddd",
}
return m
}
func TestMergeCarriesApprovedHeads(t *testing.T) { // 1321/AC1
m := approvedMerge()
m.Pulls = m.Pulls[:1] // two members approved besides the one handed so far
m.Pulls[0].State = MergeMerging
m.Pulls[0].Commit = ""
data, err := EncodeMerge(m)
if err != nil {
t.Fatal(err)
}
back, err := DecodeMerge(data)
if err != nil {
t.Fatal(err)
}
m.V = Version
if !SameMerge(back, m) || len(back.Approved) != 3 || back.Approved["novox/mesh-control@7"] != "bbbb" {
t.Errorf("decoded %+v, want %+v", back, m)
}
}
func TestMergeRefusesAPullNotApproved(t *testing.T) { // 1321/AC2
m := approvedMerge()
delete(m.Approved, ApprovedKey("novox/other", 1))
if _, err := EncodeMerge(m); err == nil || !strings.Contains(err.Error(), "novox/other#1") {
t.Errorf("a handed pull missing from the approved heads: %v", err)
}
m = approvedMerge()
m.Approved[ApprovedKey("novox/other", 1)] = "eeee"
if _, err := EncodeMerge(m); err == nil {
t.Error("a handed pull at another head than the Approve's accepted")
}
m = approvedMerge()
m.Approved[ApprovedKey("novox/other", 1)] = ""
if _, err := EncodeMerge(m); err == nil {
t.Error("an empty approved head accepted")
}
data, _ := json.Marshal(approvedMerge())
var raw map[string]any
_ = json.Unmarshal(data, &raw)
raw["v"] = Version
raw["approved"] = map[string]string{"novox/mesh-sdk@36": "aaaa"}
bad, _ := json.Marshal(raw)
if _, err := DecodeMerge(bad); err == nil {
t.Error("decoding a merge whose pulls are not all approved accepted")
}
}
func TestMergeWithoutApprovedHeadsDecodes(t *testing.T) { // 1321/AC3
old := `{"v":` + strconv.Itoa(Version) + `,"group":"feat/x","ask":"merge.feat/x.0123456789ab","pulls":[{"repository":"novox/mesh-sdk","number":36,"head":"aaaa","state":"merging"}]}`
m, err := DecodeMerge([]byte(old))
if err != nil || len(m.Approved) != 0 || len(m.Pulls) != 1 {
t.Fatalf("old merge = %+v, %v", m, err)
}
data, err := EncodeMerge(sampleMerge())
if err != nil || strings.Contains(string(data), "approved") {
t.Errorf("a merge without approved heads wrote them: %s %v", data, err)
}
}
func TestOwedStatusCarriesItsTarget(t *testing.T) { // 1322/AC1
o := Owed{Act: OwedStatus, Context: "mesh/delivery", State: "success", Target: "https://mesh.example/delivery/36"}
data, err := EncodeOwed(o)
if err != nil {
t.Fatal(err)
}
back, err := DecodeOwed(data)
o.V = Version
if err != nil || !SameOwed(back, o) || back.Target != o.Target {
t.Errorf("decoded %+v, %v; want %+v", back, err, o)
}
if _, err := EncodeOwed(Owed{Act: OwedStatus, Context: "mesh/delivery", State: "success"}); err == nil || !strings.Contains(err.Error(), "target") {
t.Errorf("a status without a target: %v", err)
}
}
func TestOwedKeyNumberZero(t *testing.T) { // 1322/AC2
key, err := OwedKey("novox/mesh-sdk", 0, "aaaa", "status=mesh/delivery")
if err != nil || key != "novox/mesh-sdk.0.aaaa.status=mesh/delivery" {
t.Fatalf("owed key for number 0 = %q, %v", key, err)
}
repo, number, head, act, err := ParseOwedKey(key)
if err != nil || repo != "novox/mesh-sdk" || number != 0 || head != "aaaa" || act != "status=mesh/delivery" {
t.Errorf("parse = %q %d %q %q %v", repo, number, head, act, err)
}
if _, err := OwedKey("novox/mesh-sdk", -1, "aaaa", "view"); err == nil {
t.Error("a negative number accepted by OwedKey")
}
if _, _, _, _, err := ParseOwedKey("novox/mesh-sdk.-1.aaaa.view"); err == nil {
t.Error("a negative number accepted by ParseOwedKey")
}
}
func TestOwedCommentsHaveASequence(t *testing.T) { // 1322/AC3
a1, err := OwedCommentAct(1)
a2, err2 := OwedCommentAct(2)
if err != nil || err2 != nil || a1 != "comment.1" || a2 != "comment.2" {
t.Fatalf("acts = %q %q %v %v", a1, a2, err, err2)
}
if _, err := OwedCommentAct(0); err == nil {
t.Error("comment sequence 0 accepted")
}
k1, err := OwedKey("novox/mesh-sdk", 36, "aaaa", a1)
k2, err2 := OwedKey("novox/mesh-sdk", 36, "aaaa", a2)
if err != nil || err2 != nil || k1 == k2 {
t.Fatalf("keys = %q %q %v %v", k1, k2, err, err2)
}
for key, seq := range map[string]string{k1: "1", k2: "2"} {
_, _, _, act, err := ParseOwedKey(key)
if err != nil {
t.Fatal(err)
}
kind, got, err := ParseOwedAct(act)
if err != nil || kind != OwedComment || got != seq {
t.Errorf("parse %q = %q %q %v; want comment %s", key, kind, got, err, seq)
}
}
}
func TestAskingID(t *testing.T) { // 1370/AC1
k, _ := MergeAskKey("feat/x", gateHeads())
long, _ := MergeAskKey("feat/"+strings.Repeat("long-", 20), gateHeads())
for _, key := range []string{k, long} {
if AskingID(key, 1) != AskID(key) || AskingID(key, 0) != AskID(key) {
t.Errorf("the first asking's id is not AskID: %q %q %q", AskingID(key, 0), AskingID(key, 1), AskID(key))
}
seen := map[string]int{AskID(key): 1}
for n := 2; n <= 12; n++ {
id := AskingID(key, n)
if other, dup := seen[id]; dup {
t.Errorf("askings %d and %d give the id %q", n, other, id)
}
seen[id] = n
if len(id) > 64 || !routerUsableID(id) {
t.Errorf("id %q of asking %d is not usable by the router", id, n)
}
if id != AskID(key+"."+strconv.Itoa(n)) {
t.Errorf("id of asking %d is not the AskID form of key.%d: %q", n, n, id)
}
}
}
}
func TestAskChecksItsAsking(t *testing.T) { // 1370/AC2
key, _ := MergeAskKey("feat/x", gateHeads())
a := sampleAsk()
if _, err := EncodeAsk(a); err != nil {
t.Fatalf("asking 0: %v", err)
}
a.Asking = 1
if _, err := EncodeAsk(a); err != nil {
t.Fatalf("asking 1 under the first id: %v", err)
}
a.Asking = 3
a.Ask = routerAskJSON(AskingID(key, 3), "Merge into main: group feat/x")
data, err := EncodeAsk(a)
if err != nil {
t.Fatalf("asking 3 under its id: %v", err)
}
a.V = Version
if got, err := DecodeAsk(data); err != nil || !SameAsk(got, a) || got.Asking != 3 {
t.Errorf("asking round trip: %+v %v", got, err)
}
a.Asking = -1
if _, err := EncodeAsk(a); err == nil {
t.Error("a negative asking was accepted")
}
// the first asking is written without the field
data, _ = EncodeAsk(sampleAsk())
if strings.Contains(string(data), `"asking"`) {
t.Errorf("asking 0 is written: %s", data)
}
}
-95
View File
@@ -1,95 +0,0 @@
package delivery
import (
"encoding/json"
"fmt"
"reflect"
"strings"
)
// **The grants issued** (novox/hq ADR 0528 clause 2, ADR 0429 §6 rule 6, ADR 0547 §4, §5, ADR 0590): when the grants
// part has issued a module's bus credentials for a fingerprint, it writes `<module>.<fingerprint hex>` in its state
// `grants` (PartGrantsBucket). The assigner reads it: a node's want moves to a fingerprint only once its grant exists.
// Only the grants part writes the bucket.
const (
// PartGrantsState is the grants part's state of issued credentials by fingerprint, as its manifest names it.
PartGrantsState = "grants"
// PartGrantsBucket is that state's bucket, `<owner module>_<name>`.
PartGrantsBucket = "mesh-grants_" + PartGrantsState
// PartGrantsRead is that state as a reader names it in `reads`.
PartGrantsRead = "mesh-grants." + PartGrantsState
// PartGrantsHistory is the revisions the bucket keeps per key.
PartGrantsHistory = 5
)
// PartGrantKey is the key, in PartGrantsBucket, of a module's credential issued for a fingerprint: `<module>.<hex>`,
// the hex being the fingerprint's key part (FingerprintKeyPart).
func PartGrantKey(module, fingerprint string) (string, error) {
return BuilderBuildKey(module, fingerprint)
}
// ParsePartGrantKey is the inverse of PartGrantKey; it refuses a want or held key, naming it.
func ParsePartGrantKey(key string) (module, fingerprint string, err error) {
if module, fingerprint, err = ParseBuilderBuildKey(key); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a grant key, <module>.<fingerprint hex>: %w", key, err)
}
return module, fingerprint, nil
}
// Grant is the value of a grant key: the bus accounts (their public keys) the module's processes were issued
// for that fingerprint, none for a module whose grants the tool runner's account carries, so a reissue for another
// account is a new revision. The entry's own time says when it was issued.
type Grant struct {
V int `json:"v"`
Accounts []string `json:"accounts,omitempty"`
}
func (g Grant) check() error {
for _, a := range g.Accounts {
if strings.TrimSpace(a) == "" || strings.ContainsAny(a, " \t\n") {
return fmt.Errorf("delivery: a grant names an empty or spaced account %q", a)
}
}
return nil
}
// EncodeGrant is g as JSON at Version; it refuses an empty or spaced account.
func EncodeGrant(g Grant) ([]byte, error) {
if err := g.check(); err != nil {
return nil, err
}
g.V = Version
return json.Marshal(g)
}
// DecodeGrant reads a grant; it refuses an unknown version (ErrUnknownVersion) and an empty or spaced account.
func DecodeGrant(data []byte) (Grant, error) {
var g Grant
if err := decode(data, &g); err != nil {
return Grant{}, err
}
if err := g.check(); err != nil {
return Grant{}, err
}
return g, nil
}
// SameGrant says whether two grants are the same value.
func SameGrant(a, b Grant) bool { return reflect.DeepEqual(a, b) }
// NamesNode says whether the grant names module's accounts on node: its own `<node>.<module>` or a process's
// `<node>.<module>.<process>`. A grant that names no account (a module that does not speak on the bus) names every
// node. The assigner wants a node only once the grant names its account (issue 1945), and a put waits for it (ADR
// 0546 clause 4).
func (g Grant) NamesNode(node, module string) bool {
if len(g.Accounts) == 0 {
return true
}
own := node + "." + module
for _, a := range g.Accounts {
if a == own || strings.HasPrefix(a, own+".") {
return true
}
}
return false
}
-125
View File
@@ -1,125 +0,0 @@
package delivery
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"regexp"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
)
// The adoption handing of novox/hq ADR 0547 clause 9 (issue 1945): the vault adopts, once and before `take-over`, the
// bcrypt hashes of the bus accounts it holds no value for (the controller's own, the node accounts, the enrolled
// devices) from the list the bus has loaded. The node-engine of the node holding mesh-broker is the one part that has
// that list, so it hands the hashes in the bus module's have, beside what it says of the list it wrote: sealed to the
// vault's own key, which it learns from the vault's answer to `node-key`, so that of every reader of the have bucket
// the vault alone can open them. The have says in the clear only how many users it hands, from which list and to which
// key; the users' names stay sealed, as the list itself is (ADR 0547 clause 7: it names every account).
//
// It hands every user of the loaded list; the vault takes from it only the users it holds no value for, and never
// replaces a hash it made from a value it holds, so the node-engine never needs to read the vault's state (which the
// grants part alone is to read, clause 9).
// AccountsHanded is the hashes of the loaded list's users, sealed to the vault, in UserListWritten.
type AccountsHanded struct {
// From is the digest of the list on the machine the hashes are taken from, `sha256:<hex>` (UserListWritten.Digest).
From string `json:"from"`
// To is the fingerprint of the vault's key the box is sealed to (KeyFingerprint).
To string `json:"to"`
// Users is how many users the box holds.
Users int `json:"users"`
// Box is the JSON of []HandedAccount, sealed to the vault's key (ADR 0206).
Box seal.Box `json:"box"`
At time.Time `json:"at"`
}
// HandedAccount is one user of an AccountsHanded box, opened: the bus user and the bcrypt hash the loaded list
// carries for it.
type HandedAccount struct {
User string `json:"user"`
Hash string `json:"hash"`
}
// KeyFingerprint is a public key's fingerprint as a handing names it: `sha256:<hex>` of its PEM, without the space
// around it.
func KeyFingerprint(publicPEM string) string {
sum := sha256.Sum256([]byte(strings.TrimSpace(publicPEM)))
return "sha256:" + hex.EncodeToString(sum[:])
}
// listedAccount is one user of a bus user list as the composer writes it (buscompose), with its password's hash.
var listedAccount = regexp.MustCompile(`\{\s*user:\s*"([^"]+)",\s*password:\s*"([^"]+)"`)
// HandedAccountsOf is every user of a bus user list that carries a bcrypt hash, in the list's order. A user listed
// twice is handed once, as the list first names it.
func HandedAccountsOf(list []byte) []HandedAccount {
var out []HandedAccount
seen := map[string]bool{}
for _, m := range listedAccount.FindAllSubmatch(list, -1) {
user, hash := string(m[1]), string(m[2])
if seen[user] || !IsBcrypt(hash) {
continue
}
seen[user] = true
out = append(out, HandedAccount{User: user, Hash: hash})
}
return out
}
// bcryptHash is the form of a bcrypt hash: version, cost, and 53 characters of salt and digest.
var bcryptHash = regexp.MustCompile(`^\$2[abxy]?\$[0-9]{2}\$[./A-Za-z0-9]{53}$`)
// IsBcrypt says whether s has the form of a bcrypt hash.
func IsBcrypt(s string) bool { return bcryptHash.MatchString(s) }
// HandAccounts seals the accounts to the vault's public key (SPKI PEM) as the handing of the list whose digest is from.
func HandAccounts(accounts []HandedAccount, vaultPEM, from string, at time.Time) (AccountsHanded, error) {
if strings.TrimSpace(vaultPEM) == "" {
return AccountsHanded{}, errors.New("delivery: no vault key to seal the handed accounts to")
}
body, err := json.Marshal(accounts)
if err != nil {
return AccountsHanded{}, err
}
box, err := seal.Seal(string(body), vaultPEM)
if err != nil {
return AccountsHanded{}, fmt.Errorf("delivery: the handed accounts could not be sealed to the vault's key: %w", err)
}
return AccountsHanded{From: from, To: KeyFingerprint(vaultPEM), Users: len(accounts), Box: box, At: at.UTC()}, nil
}
// OpenHanded opens a handing with the vault's private key (PEM), refusing one sealed to another key than publicPEM's
// and a box that does not hold as many users as it says.
func OpenHanded(h AccountsHanded, publicPEM, privatePEM string) ([]HandedAccount, error) {
if h.To != KeyFingerprint(publicPEM) {
return nil, fmt.Errorf("delivery: the handing is sealed to the key %s, not to this one (%s)", h.To,
KeyFingerprint(publicPEM))
}
plain, err := seal.Open(h.Box, privatePEM)
if err != nil {
return nil, fmt.Errorf("delivery: the handing does not open: %w", err)
}
var out []HandedAccount
if err := json.Unmarshal([]byte(plain), &out); err != nil {
return nil, fmt.Errorf("delivery: the handing holds no accounts: %w", err)
}
if len(out) != h.Users {
return nil, fmt.Errorf("delivery: the handing says %d users and holds %d", h.Users, len(out))
}
return out, nil
}
// sameHanded says whether two handings say the same: the same list, key, count and box, whatever zone their time is in.
func sameHanded(a, b *AccountsHanded) bool {
if a == nil || b == nil {
return a == b
}
x, y := *a, *b
x.At, y.At = time.Time{}, time.Time{}
return x == y && a.At.Equal(b.At)
}
-104
View File
@@ -1,104 +0,0 @@
package delivery
import (
"encoding/json"
"fmt"
"sort"
"time"
)
// Whether the artifact store still holds what a done key names (novox/hq issue 1937). A build module pushes a bundle or
// an archive as a bare blob (ADR 0498 §1), and the store's nightly collector deletes every blob no manifest names. A done
// key can therefore name a blob that is gone, and reusing it sends a definition every machine fails on. So the builder
// asks before it reuses a done key, and the artifact sweep says when it found a gone blob. Both are the seat
// mesh-artifact-sweep's (novox/hq ADR 0571 §2, ADR 0590): the controller that served them first has ended.
const (
// HeldVerb is the artifact sweep seat's verb the builder asks, as a module's invokes names it: references in, each
// answered held or gone.
HeldVerb = "seat:mesh-artifact-sweep.held"
// ArtifactsGoneEvent is the artifact sweep seat's event said when the store does not hold an artifact a key names:
// as a consumer names it, `mesh-artifact-sweep.artifacts-gone`.
ArtifactsGoneEvent = "artifacts-gone"
// ArtifactsGoneSubject is where it is published (the seat's event subject).
ArtifactsGoneSubject = "mesh.seat.mesh-artifact-sweep.event." + ArtifactsGoneEvent
)
// HeldAnswer is the answer of HeldVerb: the references the store holds, and those it answered 404 for. A reference the
// store could not be asked about is in neither, and named in Unknown with why.
type HeldAnswer struct {
Store string `json:"store,omitempty"`
Held []string `json:"held"`
Gone []string `json:"gone"`
Unknown map[string]string `json:"unknown,omitempty"`
}
// DecodeHeldAnswer reads HeldVerb's answer: the verb answers {output, ok, answer}, the answer being HeldAnswer. A
// reference asked that the answer names in none of held, gone and unknown is an error: an answer that leaves one out
// would let a key with a gone blob be reused.
func DecodeHeldAnswer(raw []byte, asked []string) (HeldAnswer, error) {
var wrapped struct {
Output string `json:"output"`
OK bool `json:"ok"`
Answer json.RawMessage `json:"answer"`
}
if err := json.Unmarshal(raw, &wrapped); err != nil {
return HeldAnswer{}, fmt.Errorf("the answer of %s is not {output, ok, answer}: %w", HeldVerb, err)
}
if !wrapped.OK || len(wrapped.Answer) == 0 {
return HeldAnswer{}, fmt.Errorf("%s did not answer: %s", HeldVerb, wrapped.Output)
}
var a HeldAnswer
if err := json.Unmarshal(wrapped.Answer, &a); err != nil {
return HeldAnswer{}, fmt.Errorf("the answer of %s is not {held, gone}: %w", HeldVerb, err)
}
named := map[string]bool{}
for _, r := range a.Held {
named[r] = true
}
for _, r := range a.Gone {
named[r] = true
}
for r := range a.Unknown {
named[r] = true
}
for _, r := range asked {
if !named[r] {
return HeldAnswer{}, fmt.Errorf("the answer of %s names nothing of %s", HeldVerb, r)
}
}
return a, nil
}
// ArtifactsGone is the body of ArtifactsGoneEvent: the module whose builder's key names a gone artifact,
// the key and its fingerprint, and every reference of it the store does not hold.
type ArtifactsGone struct {
Event string `json:"event"`
Module string `json:"module"`
Key string `json:"key"`
Fingerprint string `json:"fingerprint"`
Gone []string `json:"gone"`
At time.Time `json:"at"`
}
// EncodeArtifactsGone is the event's body, the references sorted.
func EncodeArtifactsGone(e ArtifactsGone) ([]byte, error) {
e.Event = ArtifactsGoneEvent
e.Gone = append([]string(nil), e.Gone...)
sort.Strings(e.Gone)
if e.Module == "" || e.Fingerprint == "" || len(e.Gone) == 0 {
return nil, fmt.Errorf("delivery: %s names a module, a fingerprint and at least one gone reference", ArtifactsGoneEvent)
}
return json.Marshal(e)
}
// DecodeArtifactsGone reads the event's body, refusing one that names no module, fingerprint or reference.
func DecodeArtifactsGone(raw []byte) (ArtifactsGone, error) {
var e ArtifactsGone
if err := json.Unmarshal(raw, &e); err != nil {
return ArtifactsGone{}, fmt.Errorf("delivery: %s is not one: %w", ArtifactsGoneEvent, err)
}
if e.Event != ArtifactsGoneEvent || e.Module == "" || e.Fingerprint == "" || len(e.Gone) == 0 {
return ArtifactsGone{}, fmt.Errorf("delivery: %s names no module, fingerprint or gone reference: %s", ArtifactsGoneEvent, raw)
}
return e, nil
}
-68
View File
@@ -1,68 +0,0 @@
package delivery
import (
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
const (
refA = "artifact-store://node-tools/runtime/blobs/sha256:a46b5d51a30e8fe71107d75ceb46e3187022fe51f551d0498b8be817f22435cb"
refB = "artifact-store://mesh-host/host-arch/blobs/sha256:8a752f914f3523904a5442cae37ce3cb1ad297d6789b0f871585bce4d0c2ea00"
)
// 1937/AC1 (the contract): the controller's answer reads into held and gone, and an answer leaving a reference out is refused.
func TestDecodeHeldAnswer(t *testing.T) {
raw := []byte(`{"output":"…","ok":true,"answer":{"store":"s:5100","held":["` + refA + `"],"gone":["` + refB + `"]}}`)
a, err := DecodeHeldAnswer(raw, []string{refA, refB})
if err != nil {
t.Fatal(err)
}
if len(a.Held) != 1 || a.Held[0] != refA || len(a.Gone) != 1 || a.Gone[0] != refB {
t.Fatalf("read %+v", a)
}
if _, err := DecodeHeldAnswer(raw, []string{refA, refB, "artifact-store://x/y/blobs/sha256:00"}); err == nil ||
!strings.Contains(err.Error(), "names nothing of") {
t.Fatalf("an answer leaving a reference out was taken: %v", err)
}
if _, err := DecodeHeldAnswer([]byte(`{"output":"the store is down","ok":false}`), nil); err == nil ||
!strings.Contains(err.Error(), "the store is down") {
t.Fatalf("a failed verb was taken: %v", err)
}
}
// 1937/AC2 (the contract): the event's body names the module, the fingerprint and the gone references, sorted.
func TestArtifactsGoneRoundTrip(t *testing.T) {
raw, err := EncodeArtifactsGone(ArtifactsGone{Module: "node-tools", Key: "node-tools.75", Fingerprint: "src2:75",
Gone: []string{refB, refA}, At: time.Unix(0, 0).UTC()})
if err != nil {
t.Fatal(err)
}
e, err := DecodeArtifactsGone(raw)
if err != nil {
t.Fatal(err)
}
if e.Event != ArtifactsGoneEvent || e.Module != "node-tools" || len(e.Gone) != 2 || e.Gone[0] != refB {
t.Fatalf("read %+v", e)
}
if _, err := EncodeArtifactsGone(ArtifactsGone{Module: "m", Fingerprint: "src2:1"}); err == nil {
t.Fatal("an event naming no gone reference was made")
}
if _, err := DecodeArtifactsGone([]byte(`{"event":"artifacts-gone","module":"m"}`)); err == nil {
t.Fatal("an event naming no fingerprint was read")
}
}
// The verb and the event are on the artifact sweep seat's contract under the names the builder uses (ADR 0571 §2).
func TestHeldOnTheArtifactSweepSeat(t *testing.T) {
seat, ok := catalogue.SeatNamed(catalogue.ArtifactSweepSeat)
if !ok || HeldVerb != "seat:"+seat.Name+".held" || ArtifactsGoneSubject != "mesh.seat."+seat.Name+".event."+ArtifactsGoneEvent {
t.Fatalf("HeldVerb %q, ArtifactsGoneSubject %q", HeldVerb, ArtifactsGoneSubject)
}
if !slices.ContainsFunc(seat.Serves, func(v catalogue.Verb) bool { return v.Name == "held" }) || !slices.Contains(seat.Emits, ArtifactsGoneEvent) {
t.Fatalf("the sweep seat serves %v and emits %v", seat.Serves, seat.Emits)
}
}

Some files were not shown because too many files have changed in this diff Show More