Compare commits

..
Author SHA1 Message Date
jschoubben b7abe203a9 Keep, deprecated, the exported names the group removed while running parts use them: the checker builds the running node-engine's composer beside this mesh-sdk (ADR 0548 §6); the second group removes them once their users are delivered
mesh/delivery ready: it is merged on the Approve
mesh/check pass b7abe203a929 on 2341e8e98377 | unit 1186 run, 5555 kept, 27 skipped | integration 1205 run, 5717 kept, 35 skipped | over bound: unit (…
mesh/delivery-group group feat/l3b-bus ready: every member ready, and composed together they pass
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 18:42:08 +01:00
jschoubben cf5812101b The seat mesh-catalog's verbs' answers are its contract: CatalogModules, CatalogOne, CatalogProvides and CatalogDependents, with the field names of the catalog module's tools they replace; the module graph is no part of mesh-sdk (the operator, 2026-11-03: only contracts)
mesh/delivery-group group feat/l3b-bus checking: 4 of 4 member(s) ready
mesh/check checking cf5812101b0b on 2341e8e98377 | novox/mesh-sdk#165 in group feat/l3b-bus
mesh/delivery closed: a newer head of the same pull request
2026-11-03 16:51:54 +01:00
jschoubben fdedf290d5 provisioner_retire_approve and provisioner_retire_reject name no removed controller verb either: they ask nobody, and the operator's answer on the operator channel is named in why and by (ADR 0570 §5, ADR 0604 §8) 2026-11-03 16:51:14 +01:00
jschoubben 6e2f08f96a provisioner_delete's description names no removed controller verb: it asks nobody, so the operator is asked on the operator channel first and named in why and by (ADR 0604 §8) 2026-11-03 16:51:14 +01:00
jschoubben 9ab75bab49 The seat mesh-catalog's contract: four verbs read from its holder's states, modules, module, provides and dependents, each optional while the module that claimed it before still stands (ADR 0604 §1, §6) 2026-11-03 16:51:14 +01:00
jschoubben 413dd13fe3 Accept a merged message with no trees, so a merge that leaves a repository with no module.json still reaches the indexer (issue 1126) 2026-11-03 16:41:06 +01:00
jschoubben 1b9475e15f FPSrc is 0 when a recompute with no merge moved the fingerprint: no release carries that build (ADR 0601 §1; review round 2) 2026-11-03 16:41:06 +01:00
jschoubben 493e69e883 A module value carries the source revision at which its fingerprint last changed, a have says what the node keeps as it runs for a manifest that does not yet say it, and a want names a release only beside the yes it gave (ADR 0601 §1, §4; review round 1) 2026-11-03 16:41:06 +01:00
jschoubben 636c2c1b0c A want names the release whose Approve answered its build's record ask, and a have waiting for a yes says whether the move changes the build alone (ADR 0601 §1) 2026-11-03 16:41:06 +01:00
jschoubben 439b77e07d A node the sandbox did not judge is a field of the contract: composer.Machine.NotJudged and the answer's verdict not-judged, so the checker decides by them and never by words (ADR 0593 §9; the prepare's seams finding on "not judged") 2026-11-03 16:41:06 +01:00
jschoubben 6655bdff50 The check that the catalogue package names no module reads the names from the catalogue's module directories alone, no list beside them (ADR 0601 §4) 2026-11-03 16:41:06 +01:00
jschoubben 189f28e4e1 mesh-sdk's catalogue package names no module: the tool runner says it is one (tool-runner), the foundation is what is computed, the kept streams are the judge's, and the ended controller's dependence check goes with it (issue 774, the operator 2026-11-03) 2026-11-03 16:41:06 +01:00
jschoubben f6e214533b mesh-sdk names no module: the core's credentials are what a manifest says (core-credentials), and the bus's upgrade policy is the one its manifest declares (issue 774, the operator 2026-11-03) 2026-11-03 16:41:06 +01:00
jschoubben b6cc9572f2 A manifest may say its own secrets are the core's credentials (core-credentials), so a reader asks the module rather than a list here (issue 774, the operator 2026-11-03) 2026-11-03 16:41:06 +01:00
jschoubben 4087280895 A manifest whose judge does not know a field has the verdict waits, held like a refusal by every reader (ADR 0578); in a modules value it is written as a refusal naming the fields, so a reader pinned before holds it too 2026-11-03 16:41:06 +01:00
mesh-tools 2341e8e983 Merge pull request 'Bus state by template: the typed requires entry, the binding and the calls package (issue 1966, B1)' (#164) from feat/bus-state into main 2026-11-03 15:24:27 +00:00
jschoubben f8bd12e4b7 Encode each part of an index key, so no two keys meet and every part is a bus token (issue 1966 S4)
mesh/delivery merged: source.main moved to novox/mesh-sdk@2341e8e98377
mesh/delivery-group group feat/bus-state merged: every member is merged
mesh/check pass f8bd12e4b7b1 on 7c015b652d36 | unit 6460 run, 262 kept, 27 skipped | integration 6639 run, 264 kept, 35 skipped | over bound: unit, in…
A raw value holding "." collided with a longer path, and a space is refused
by the bus: every byte outside [A-Za-z0-9_-] of a field value or an instance
path part is written =XX (review round 1 of issue 1966 B1).
2026-11-03 15:28:43 +01:00
jschoubben 15fc2c4ce4 An index's key and entry are one contract of the calls package (issue 1966 S4)
IndexKey and IndexEntry {instance, sequence}: the owner's watcher and the
tool runner's reindex write the same, so a reindex leaves the index as the
watcher keeps it (the lead's ruling on issue 1966 B1; ADR 0597 §4).
2026-11-03 15:12:38 +01:00
jschoubben 092260870b A ledger's binding carries its types (issue 1966 S3)
The grants part copies them from the manifest-accepted entry, so the tool
runner checks an operation against its type without reading another
part's state (ADR 0596; the lead's ruling on issue 1966 B1).
2026-11-03 15:12:19 +01:00
jschoubben cc64794d4c Name a ledger's history and snapshot subjects in one place (issue 1966 S3)
The grants part grants on them, the tool runner appends and snapshots on
them and the bus module creates the history over them, so the three read
one contract (the lead's ruling on issue 1966 B1; ADR 0597 §8, §9).
2026-11-03 15:11:59 +01:00
jschoubben 25c08d09ec Name the _meta key the tool runner hands a bundle the id of the tool call in flight under (issue 1966 S4)
MetaCall, mesh/call: the id ApplyRequest.Call names, so the runner stamps
the caller it proved as the operation's author (ADR 0597 §7; the lead's
ruling on issue 1966 B1).
2026-11-03 13:26:46 +01:00
jschoubben d3e10f68c5 A bus-state entry stays off Requires, which names provisions (issue 1966 S1)
A local name such as records is no provision: on Requires it would send
the placement looking for a provider nothing offers (the lead's ruling on
issue 1966 B1). BusState alone holds the typed entries.
2026-11-03 13:26:27 +01:00
jschoubben 6ec6cad968 A bus account kind ledger, named ledger.<node>.<module> and held in the tool runner's ledgers (issue 1966 S6)
ADR 0597 §9 and the lead's Q1 path: one ledger credential per module
assignment, minted by the vault like every kind; the grants part's copy
of buscompose gains the same kind alike (D5b).
2026-11-03 13:24:42 +01:00
jschoubben 2122ba52c0 The indexer's manifest-accepted event carries the manifest's bus-state entries whole (issue 1966 S5)
ADR 0595 §4, design 25 § 12.2: the bus module hears the accepted verdict
as an event with the entries, never reading the indexer's state; AcceptedOf
builds the body and EventID names it once per digest and judge.
2026-11-03 13:24:04 +01:00
jschoubben 510d983cc6 A calls package of its own for objects, streams, work queues and ledgers (issue 1966 S4)
ADR 0595 §9, ADR 0597 §10, design 25 § 12.5: the request and answer types
and method names of the bundle-pipe calls the tool runner serves, apply
naming the tool call it serves (ADR 0597 §7) and answering taken, its
sequence and state behind. It imports nothing of mesh-sdk, so a change
rebuilds only its importers; the package list names it.
2026-11-03 13:23:23 +01:00
jschoubben e9c319c5a0 A membership carries its module's bus-state bindings, by local name (issue 1966 S3)
ADR 0595 §1, §7: the binding travels on the bus module's created and the
grants part writes it into the membership; omitted when empty, so a
membership without bindings encodes byte for byte as before. The two
address tests are one table.
2026-11-03 13:21:24 +01:00
jschoubben 0bec849c40 Name the storage types, templates, options and ledger effects, and give a ledger its types (issue 1966 S2)
The words of design 25 § 12.1 and ADR 0597 §1-2 as constants, and a
ledger's types (schema, operations each with its effect, indexes) on the
bus-state entry, so the judge, the bus module and the tool runner read one
contract. No bound and no rule: those are the judge's (ADR 0578 §7).
2026-11-03 13:20:35 +01:00
jschoubben 779ad9ac10 A requires entry naming a storage type asks for bus state, read leniently beside the plain names (issue 1966 S1)
ADR 0595 §1: the typed long form {name, storage, template, options,
configuration} decodes into Manifest.BusState, its name also on Requires;
a key it does not know is set aside and named, never refused (ADR 0578 §3).
The requirements tests are one table.
2026-11-03 13:19:40 +01:00
40 changed files with 1414 additions and 205 deletions
+382
View File
@@ -0,0 +1,382 @@
// Package buscalls is the contract of the calls a bundle makes on its own bus state through the tool runner, over the
// bundle's pipe (novox/hq ADR 0595 §9, ADR 0597 §10, design 25 § 12.5): the object store's, a stream's read-back and
// fill, a work queue's size and dead letters, and the ledger's. Each call names an entry by the local name its module
// asks for it by in `requires`; the tool runner serves it from the module's membership's bindings, its own entries
// only. Only the shapes and the method names are here: what the tool runner refuses, and why, is its own.
//
// It is a package of its own, imported only by the modules that use these calls and by the tool runner, so a change
// here rebuilds those and not the module library's importers (ADR 0595 §9). It imports nothing of mesh-sdk.
package buscalls
import (
"encoding/json"
"fmt"
"strings"
"time"
)
// MetaCall is where in a `tools/call`'s `_meta` the tool runner hands a bundle the id of the tool call in flight: the id
// ApplyRequest.Call names, so the runner stamps the caller it proved for that call as the operation's author (novox/hq
// ADR 0597 §7). The runner writes `_meta`; a caller writes only `arguments`, so no caller can name another call.
const MetaCall = "mesh/call"
// The methods a bundle asks the tool runner, one per call.
const (
ObjectsPut = "mesh/objects.put"
ObjectsGet = "mesh/objects.get"
ObjectsList = "mesh/objects.list"
ObjectsDelete = "mesh/objects.delete"
ObjectsInfo = "mesh/objects.info"
ObjectsWatch = "mesh/objects.watch"
StreamRead = "mesh/stream.read"
StreamFill = "mesh/stream.fill"
WorkQueueSize = "mesh/workqueue.size"
WorkQueueDead = "mesh/workqueue.dead"
LedgerApply = "mesh/ledger.apply"
LedgerGet = "mesh/ledger.get"
LedgerHistory = "mesh/ledger.history"
LedgerWatch = "mesh/ledger.watch"
LedgerQuery = "mesh/ledger.query"
// The ledger's repairs, which the tool runner also serves as tools of every module owning a ledger, its own
// ledgers only (design 25 § 12.5).
LedgerRebuild = "mesh/ledger.rebuild"
LedgerReindex = "mesh/ledger.reindex"
LedgerSnapshot = "mesh/ledger.snapshot"
LedgerLag = "mesh/ledger.lag"
)
// The methods the tool runner sends the bundle for a watch it asked, each naming the watch by its id, as `mesh/log`
// does for a log.
const (
ObjectsWatched = "mesh/objects"
LedgerWatched = "mesh/ledger"
)
// Fill is how much an entry holds against its cap: its messages (or objects), its bytes and its cap in bytes.
type Fill struct {
Count uint64 `json:"count"`
Bytes uint64 `json:"bytes"`
MaxBytes uint64 `json:"max_bytes"`
}
// Done is the answer of a call that answers nothing but that it was done.
type Done struct{}
// --- the object store
// ObjectsPutRequest stores one object whole under its name, replacing one of that name.
type ObjectsPutRequest struct {
Objects string `json:"objects"`
Name string `json:"name"`
Data []byte `json:"data"`
Description string `json:"description,omitempty"`
}
// ObjectsNameRequest names one object of an object store: get, delete.
type ObjectsNameRequest struct {
Objects string `json:"objects"`
Name string `json:"name"`
}
// ObjectsRequest names an object store: list, info.
type ObjectsRequest struct {
Objects string `json:"objects"`
}
// ObjectsWatchRequest asks the tool runner to send every object stored or deleted, as ObjectsChange under the watch's id.
type ObjectsWatchRequest struct {
Objects string `json:"objects"`
Watch string `json:"watch"`
}
// ObjectInfo is one object as its store holds it: put's answer, and each of list's.
type ObjectInfo struct {
Name string `json:"name"`
Size uint64 `json:"size"`
Chunks uint32 `json:"chunks"`
Digest string `json:"digest"`
Description string `json:"description,omitempty"`
Modified time.Time `json:"modified"`
Deleted bool `json:"deleted,omitempty"`
}
// ObjectData is get's answer: the object's info and its bytes.
type ObjectData struct {
Info ObjectInfo `json:"info"`
Data []byte `json:"data"`
}
// ObjectsListed is list's answer: every object the store holds.
type ObjectsListed struct {
Objects []ObjectInfo `json:"objects"`
}
// ObjectsChange is one object stored or deleted, sent under ObjectsWatched; Current says it was there when the watch
// began.
type ObjectsChange struct {
Watch string `json:"watch"`
Object ObjectInfo `json:"object"`
Current bool `json:"current"`
}
// --- a stream
// StreamReadRequest reads a module's own stream from a sequence on, at most Limit messages, Subjects narrowing it.
type StreamReadRequest struct {
Stream string `json:"stream"`
From uint64 `json:"from"`
Subjects string `json:"subjects,omitempty"`
Limit int `json:"limit,omitempty"`
}
// StreamRequest names a stream: fill.
type StreamRequest struct {
Stream string `json:"stream"`
}
// StreamMessage is one message of a stream.
type StreamMessage struct {
Sequence uint64 `json:"sequence"`
Subject string `json:"subject"`
Body json.RawMessage `json:"body"`
At time.Time `json:"at"`
}
// StreamPage is read's answer: the messages in order, the sequence to read on from (0 when nothing follows) and the
// stream's last sequence when it was read.
type StreamPage struct {
Messages []StreamMessage `json:"messages"`
Next uint64 `json:"next"`
Last uint64 `json:"last"`
}
// StreamFilled is fill's answer: the stream's fill and its first and last sequence.
type StreamFilled struct {
Fill
First uint64 `json:"first"`
Last uint64 `json:"last"`
}
// --- a work queue
// WorkQueueRequest names a work queue: size.
type WorkQueueRequest struct {
Queue string `json:"queue"`
}
// WorkQueueDeadRequest asks a work queue's dead letters, at most Limit of them.
type WorkQueueDeadRequest struct {
Queue string `json:"queue"`
Limit int `json:"limit,omitempty"`
}
// WorkQueueSized is size's answer: the messages waiting to be taken, against the queue's cap.
type WorkQueueSized struct {
Fill
}
// DeadLetter is one message delivered as often as its queue allows and never settled.
type DeadLetter struct {
Sequence uint64 `json:"sequence"`
Subject string `json:"subject"`
Deliveries int `json:"deliveries"`
Body json.RawMessage `json:"body,omitempty"`
At time.Time `json:"at"`
}
// WorkQueueDeadLetters is dead's answer.
type WorkQueueDeadLetters struct {
Dead []DeadLetter `json:"dead"`
}
// --- the ledger (ADR 0597)
// ApplyRequest applies one operation to one instance of a ledger. Call is the id of the tool call the operation
// serves, as the tool runner handed it under MetaCall, whose caller the tool runner proved and stamps as the author
// (ADR 0597 §7); empty, the operation carries the owning module's account alone. An author the bundle writes into Body
// is never taken as the author.
type ApplyRequest struct {
Ledger string `json:"ledger"`
Instance string `json:"instance"`
Operation string `json:"operation"`
Body json.RawMessage `json:"body,omitempty"`
Call string `json:"call,omitempty"`
}
// Applied is apply's answer: the operation was taken, with its sequence in the instance's history; StateBehind says
// the state could not follow because its bucket is full, and catches up at the next apply after room is made (ADR
// 0597 §6).
type Applied struct {
Taken bool `json:"taken"`
Sequence uint64 `json:"sequence"`
StateBehind bool `json:"state_behind,omitempty"`
}
// Author is who an operation is by, as `apply` stamped it (ADR 0597 §7): the owning module's account, and the caller
// of the tool call it served, as the tool runner proved it.
type Author struct {
Account string `json:"account"`
Caller string `json:"caller,omitempty"`
}
// InstanceRequest names one instance of a ledger: get, snapshot.
type InstanceRequest struct {
Ledger string `json:"ledger"`
Instance string `json:"instance"`
}
// InstanceState is get's answer: the instance's state and the sequence of the last operation it holds.
type InstanceState struct {
Instance string `json:"instance"`
State json.RawMessage `json:"state"`
Sequence uint64 `json:"sequence"`
}
// HistoryRequest reads an instance's operations from a sequence on, at most Limit of them.
type HistoryRequest struct {
Ledger string `json:"ledger"`
Instance string `json:"instance"`
From uint64 `json:"from,omitempty"`
Limit int `json:"limit,omitempty"`
}
// Operation is one operation as the history keeps it.
type Operation struct {
Sequence uint64 `json:"sequence"`
Instance string `json:"instance"`
Operation string `json:"operation"`
Body json.RawMessage `json:"body,omitempty"`
Author Author `json:"author"`
At time.Time `json:"at"`
}
// HistoryPage is history's answer: the operations oldest first, the sequence to read on from (0 when nothing
// follows) and the instance's last sequence when it was read.
type HistoryPage struct {
Operations []Operation `json:"operations"`
Next uint64 `json:"next"`
Last uint64 `json:"last"`
}
// LedgerWatchRequest asks the tool runner to send every change of the instances Instances names (an instance, with
// `*` for one name and `>` for the rest) as InstanceChange under the watch's id: the current states first.
type LedgerWatchRequest struct {
Ledger string `json:"ledger"`
Instances string `json:"instances"`
Watch string `json:"watch"`
}
// InstanceChange is one instance's new state, sent under LedgerWatched; Current says it was the state when the watch
// began.
type InstanceChange struct {
Watch string `json:"watch"`
Instance string `json:"instance"`
State json.RawMessage `json:"state"`
Sequence uint64 `json:"sequence"`
Current bool `json:"current"`
}
// QueryRequest reads a declared index of a ledger: the entries whose key Key names (a key, with `*` for one name and
// `>` for the rest; empty is every key), at most Limit of them.
type QueryRequest struct {
Ledger string `json:"ledger"`
Index string `json:"index"`
Key string `json:"key,omitempty"`
Limit int `json:"limit,omitempty"`
}
// IndexEntry is the value of one key of a declared index (IndexKey): the instance it points at, and the sequence of
// the operation it was written from. The owner's watcher and the tool runner's reindex both write it so.
type IndexEntry struct {
Instance string `json:"instance"`
Sequence uint64 `json:"sequence"`
}
// IndexKey is the key of an instance in a declared index: the values of the index's fields, in its order, then the
// parts of the instance's path, all joined by ".". Each value and each path part is encoded (keyToken): every byte
// outside [A-Za-z0-9_-] is written "=" and two upper-case hex digits ("." is "=2E", " " is "=20", "=" is "=3D"), so two
// different keys never meet and every part is a token the bus takes. `17950.listing.a17` is the key of price 17950
// and instance `listing/a17`. The owner's watcher and the tool runner's reindex both write it, so a reindex leaves the
// index as the watcher keeps it.
func IndexKey(fields []string, instance string) string {
parts := make([]string, 0, len(fields)+2)
for _, f := range fields {
parts = append(parts, keyToken(f))
}
for _, p := range strings.Split(instance, "/") {
parts = append(parts, keyToken(p))
}
return strings.Join(parts, ".")
}
// keyToken is v with every byte outside [A-Za-z0-9_-] written "=" and two upper-case hex digits (IndexKey).
func keyToken(v string) string {
var b strings.Builder
for i := 0; i < len(v); i++ {
c := v[i]
if c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || c == '_' || c == '-' {
b.WriteByte(c)
continue
}
fmt.Fprintf(&b, "=%02X", c)
}
return b.String()
}
// QueryAnswer is query's answer.
type QueryAnswer struct {
Entries []IndexEntry `json:"entries"`
}
// RebuildFrom is what a rebuild derives the state from: the history, or the snapshot and the operations after it.
const (
RebuildFromHistory = "history"
RebuildFromSnapshot = "snapshot"
)
// RebuildRequest derives an instance's state again (every instance's when Instance is empty), From the history or the
// snapshot.
type RebuildRequest struct {
Ledger string `json:"ledger"`
Instance string `json:"instance,omitempty"`
From string `json:"from,omitempty"`
}
// ReindexRequest builds a declared index again from the history.
type ReindexRequest struct {
Ledger string `json:"ledger"`
Index string `json:"index"`
}
// LedgerRequest names a ledger: lag.
type LedgerRequest struct {
Ledger string `json:"ledger"`
}
// Repaired is the answer of rebuild, reindex and snapshot: how many instances (or index entries) it wrote.
type Repaired struct {
Written int `json:"written"`
}
// InstanceLag is one instance whose state or snapshot is behind its history: the sequences each holds.
type InstanceLag struct {
Instance string `json:"instance"`
History uint64 `json:"history"`
State uint64 `json:"state"`
Snapshot uint64 `json:"snapshot"`
}
// IndexLag is how far one declared index is behind the history.
type IndexLag struct {
Index string `json:"index"`
Behind uint64 `json:"behind"`
}
// Lagged is lag's answer: every instance and index behind its history; none when nothing is behind.
type Lagged struct {
Instances []InstanceLag `json:"instances,omitempty"`
Indexes []IndexLag `json:"indexes,omitempty"`
}
+11 -3
View File
@@ -31,7 +31,8 @@ type ClaimedSeat struct {
// WantedAccount is one key of the grants part's state `accounts`: an account the mesh needs, without its password.
type WantedAccount struct {
User string `json:"user"`
// Kind is KindModule (KindNodeTools for the runtime, named as a module is), KindProcess or KindProvisioner.
// Kind is KindModule (KindNodeTools for the runtime, named as a module is), KindProcess, KindProvisioner or
// KindLedger.
Kind Kind `json:"kind"`
Node string `json:"node"`
Module string `json:"module"`
@@ -46,8 +47,12 @@ type WantedAccount struct {
At time.Time `json:"at"`
}
// LedgersSecret is the tool runner's own secret holding its machine's ledger accounts, a map of module to credential
// (ADR 0597 §9).
const LedgersSecret = "ledgers"
// mintedKinds are the kinds of account the vault mints (ADR 0561 §1).
var mintedKinds = []Kind{KindModule, KindNodeTools, KindProcess, KindProvisioner}
var mintedKinds = []Kind{KindModule, KindNodeTools, KindProcess, KindProvisioner, KindLedger}
// principal is the account as the composer names it.
func (w WantedAccount) principal() Principal {
@@ -93,9 +98,12 @@ func (w WantedAccount) Same(o WantedAccount) bool {
}
// OwnSecret is the module's own secret the account is held in: `broker`, `broker-<process id>` or `provisioner-broker`
// (the controller's names, ADR 0436 §2, ADR 0349 §11).
// (the controller's names, ADR 0436 §2, ADR 0349 §11); a ledger account in the tool runner's `ledgers`, which keeps every
// ledger account of its machine, by module (ADR 0597 §9).
func (w WantedAccount) OwnSecret() string {
switch w.Kind {
case KindLedger:
return LedgersSecret
case KindProcess:
return "broker-" + w.Process
case KindProvisioner:
+4 -1
View File
@@ -30,6 +30,9 @@ func ControllerLeftoverConsumersOn(nodes []string) []string {
return out
}
// endedController is the name the ended controller had as a module: its buckets are named for it (ADR 0590 §6).
const endedController = "mesh-controller"
// ControllerLeftoverBuckets are the ended controller's buckets, each by its local name (`KV_mesh-controller_<name>`).
var ControllerLeftoverBuckets = []string{"asked", "assignments", "builds", "calls", "checks-running", "composed",
"condition-history", "conditions", "declared-config", "declared-packages", "delivery-merges", "grants", "hand-acts",
@@ -40,7 +43,7 @@ var ControllerLeftoverBuckets = []string{"asked", "assignments", "builds", "call
func ControllerLeftoverStreams(nodes []string) []string {
out := []string{"CONTROL", AssignmentsStream}
for _, b := range ControllerLeftoverBuckets {
out = append(out, "KV_"+BucketName(catalogue.EndedControllerName, b))
out = append(out, "KV_"+BucketName(endedController, b))
}
for _, n := range nodes {
if safeSubject.MatchString(n) {
+14
View File
@@ -18,4 +18,18 @@ func TestTheBusNamesAgree(t *testing.T) {
if LogStreamName("mesh-delivery-builder", "build-log") == "" || LogSubject("mesh-delivery-builder", "build-log") == "" {
t.Fatal("a log has no stream or subject")
}
// 1966/AC20: a ledger's history and snapshot subjects are one contract, which the grants part grants a ledger
// account its publish on.
if h, sn := LedgerHistorySubject("m", "records"), LedgerSnapshotSubject("m", "records"); h != "mesh.ledger.m.records.history" ||
sn != "mesh.ledger.m.records.snapshot" {
t.Fatalf("a ledger's subjects are %q and %q", h, sn)
}
// 1966/AC23: a wanted account of kind ledger for m on a names the user ledger.a.m and the own secret ledgers.
ledger := WantedAccount{User: "ledger.a.m", Kind: KindLedger, Node: "a", Module: "m", URL: "nats://10.0.0.1:4222"}
if got := (Principal{Kind: KindLedger, Node: "a", Module: "m"}).Username(); got != "ledger.a.m" {
t.Fatalf("a ledger account is named %q", got)
}
if err := ledger.Valid(); err != nil || ledger.OwnSecret() != "ledgers" {
t.Fatalf("a wanted ledger account: %v, held in %q", err, ledger.OwnSecret())
}
}
+22
View File
@@ -44,8 +44,26 @@ const (
// exactly what its bus says — never root, never the operator account, and never able to answer a tool
// call or a seat's verb (§3). One per process and machine.
KindProcess Kind = "process"
// KindLedger is one module assignment's ledger credential (novox/hq ADR 0597 §9, design 25 § 12.4): held by that
// machine's tool runner, never by the module's own code, and the only account that publishes on that module's ledger
// histories and snapshots. One per module assignment, whose module asks for a ledger in `requires`.
KindLedger Kind = "ledger"
)
// LedgerHistorySubject and LedgerSnapshotSubject are the subject prefixes of one ledger's history and of its instances'
// snapshots (novox/hq ADR 0597 §1, §8, §9): an instance's subject is the prefix, ".", and the instance's path with each
// "/" written "." (`decision/0290` is `….decision.0290`). One contract: the grants part grants a ledger account its
// publish on them, the tool runner appends and snapshots on them, and the bus module creates the history's stream over
// them.
func LedgerHistorySubject(module, local string) string {
return "mesh.ledger." + module + "." + local + ".history"
}
// LedgerSnapshotSubject is the subject prefix of one ledger's instances' snapshots (LedgerHistorySubject).
func LedgerSnapshotSubject(module, local string) string {
return "mesh.ledger." + module + "." + local + ".snapshot"
}
// Seat is a role on the bus as a principal relates to it: the subjects it accepts, and those it
// emits (novox/hq ADR 0118, design 29 §5).
type Seat struct {
@@ -272,6 +290,10 @@ func (p Principal) Username() string {
return p.Login
}
return "node." + p.Node
case KindLedger:
// `ledger.<machine>.<module>`: never `<node>.<module>`, which is the module's own account or the tool runner's,
// so the module's code never holds it.
return "ledger." + p.Node + "." + p.Module
case KindProvisioner:
// Named for the machine, as a node is: one client per machine, whichever module runs it, and never
// `<node>.<module>`, which is the module's own account or the tool runner's.
@@ -1,8 +1,12 @@
package catalogue
import (
"go/ast"
"go/parser"
"go/token"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
@@ -18,6 +22,7 @@ import (
// design 28 task 5.4, not converted.
func TestNoCatalogueManifestNamesAmqp(t *testing.T) {
modules, err := filepath.Glob(filepath.Join(beside.Catalogue(t), "*", "module.json"))
modules0 := modules
if err != nil || len(modules) == 0 {
t.Fatalf("no manifests in the catalogue, so this proved nothing: %v", err)
}
@@ -31,4 +36,44 @@ func TestNoCatalogueManifestNamesAmqp(t *testing.T) {
filepath.Base(filepath.Dir(path)))
}
}
// Nor does this package's code name any module the catalogue holds (ADR 0601 §4): what a module is, its manifest
// says. The names are read from the catalogue's module directories, never from a list here; every string literal of
// the package's non-test files is read, and a module's name is refused unless it is also a seat's name or one of the
// words below, which name something else.
t.Run("the package names no module", func(t *testing.T) {
modules := map[string]bool{}
for _, path := range modules0 {
modules[filepath.Base(filepath.Dir(path))] = true
}
// Words that are a module's name and name something else here: a language, an environment's rendering, a package.
elsewhere := map[string]bool{"go": true, "systemd": true, "nftables": true}
fset := token.NewFileSet()
files, _ := filepath.Glob("*.go")
for _, f := range files {
// kept_until_delivered.go keeps, deprecated, the names removed while running parts still use them; the second
// group removes it and this exemption together.
if strings.HasSuffix(f, "_test.go") || f == "kept_until_delivered.go" {
continue
}
parsed, err := parser.ParseFile(fset, f, nil, 0)
if err != nil {
t.Fatal(err)
}
ast.Inspect(parsed, func(n ast.Node) bool {
lit, ok := n.(*ast.BasicLit)
if !ok || lit.Kind != token.STRING {
return true
}
word, err := strconv.Unquote(lit.Value)
if err != nil || !modules[word] || elsewhere[word] {
return true
}
if _, seat := SeatNamed(word); seat {
return true
}
t.Errorf("%s names the module %s", fset.Position(lit.Pos()), word)
return true
})
}
})
}
+207
View File
@@ -0,0 +1,207 @@
package catalogue
import (
"bytes"
"encoding/json"
"fmt"
"strings"
)
// Bus state is what a module keeps on the bus, asked for as a provision of the bus module in `requires` (novox/hq ADR
// 0595 §1, design 32 § 4): an entry of `requires` in its typed long form, the local name, the storage type, the template
// and the template's options. This is the contract only (ADR 0578 §7, ADR 0593 §1): the entry's shape, decoded
// leniently. Every rule that judges an entry (a template known, an option within its bound, a fixed setting not given)
// is the indexer's judge's.
// The storage types (design 25 § 12.1): the stream is the base; the key-value bucket, the object store and the ledger
// are built on it, each with calls of its own (ADR 0595 §2).
const (
StorageStream = "stream"
StorageKeyValue = "key-value"
StorageObjectStore = "object-store"
StorageLedger = "ledger"
)
// The six templates, each a storage type with a predefined configuration (ADR 0595 §2, design 25 § 12.1): state on
// key-value, events, work queue and log on stream, objects on object store, ledger on ledger. A bare stream names
// none and gives its own Configuration.
const (
TemplateState = "state"
TemplateEvents = "events"
TemplateWorkQueue = "work-queue"
TemplateLog = "log"
TemplateObjects = "objects"
TemplateLedger = "ledger"
)
// The option names, a template's settings a module may change (design 25 § 12.1's table), written as the old sections
// wrote them, so a module that moves carries its old settings as options unchanged (ADR 0595 §11). Which option a
// template takes, and its bound, is the judge's.
const (
// OptionPastValues is how many past values a state's key keeps (the old `state`'s `history`).
OptionPastValues = "history"
// OptionTimeToLive is how long a state's value lives, in seconds.
OptionTimeToLive = "ttl-seconds"
// OptionLifetimePerKey is whether a state's key may be given a lifetime of its own.
OptionLifetimePerKey = "key-ttl"
// OptionPerMachine is whether a state is kept one bucket per machine.
OptionPerMachine = "per-machine"
// OptionAgeDays is how long a stream keeps a message, in days.
OptionAgeDays = "max-age-days"
// OptionCapMiB is the cap of a stream, a bucket or an object store, in MiB.
OptionCapMiB = "max-mib"
// OptionGrant names the trusted config value granting another module a part of it (ADR 0363 §4, ADR 0592 §9).
OptionGrant = "grant"
// OptionDeliveries is how many times a work queue's message is delivered before it is a dead letter.
OptionDeliveries = "max-deliveries"
// OptionDuplicates is how long, in seconds, a work queue detects a message sent twice.
OptionDuplicates = "duplicates"
)
// ConfigWhenFull is what a configuration of its own says a bare stream does when full: WhenFullRefuse or
// WhenFullDropOldest. In a template it is fixed, never an option (ADR 0595 §3).
const (
ConfigWhenFull = "when-full"
WhenFullRefuse = "refuse"
WhenFullDropOldest = "drop-oldest"
)
// The ledger effects, what an operation does to an instance's state (ADR 0597 §2): one of this fixed set.
const (
// EffectSetFields sets the state's fields to the operation's Fields.
EffectSetFields = "set-fields"
// EffectSetEntry sets one entry of the state's map Into, under the key the operation's field Key holds, to its Fields.
EffectSetEntry = "set-entry"
// EffectAppend appends the operation's Fields to the state's list Into.
EffectAppend = "append"
// EffectRemove removes the entry of the map Into under the operation's field Key, or, naming no Into, the instance's
// state.
EffectRemove = "remove"
)
// LedgerType is one type of a ledger (ADR 0597 §1): a schema and the operations allowed on it, with its indexes. An
// instance of it is named by a path whose first part is the type's name (`decision/0290`).
type LedgerType struct {
Name string `json:"name"`
// Schema is the type's fields, by name.
Schema map[string]LedgerField `json:"schema"`
// Operations are the operations allowed on an instance, each with its effect.
Operations []LedgerOperation `json:"operations"`
// Indexes are its declared indexes, each a second key-value bucket kept by a watcher (ADR 0597 §4).
Indexes []LedgerIndex `json:"indexes,omitempty"`
}
// LedgerField is one field of a ledger type's schema.
type LedgerField struct {
// Type is the field's value type, as the schema names it.
Type string `json:"type"`
// DigestOf names the text field this field is the digest of (ADR 0597 §2): `apply` refuses an operation whose
// digest does not match its text.
DigestOf string `json:"digest-of,omitempty"`
}
// LedgerOperation is one operation a ledger type allows, and its effect on the state.
type LedgerOperation struct {
Name string `json:"name"`
// Effect is one of the ledger effects (EffectSetFields, EffectSetEntry, EffectAppend, EffectRemove).
Effect string `json:"effect"`
// Fields are the operation's fields its effect copies into the state; it may carry more (ADR 0597 §2).
Fields []string `json:"fields,omitempty"`
// Into is the state's map (EffectSetEntry, EffectRemove) or list (EffectAppend) the effect writes.
Into string `json:"into,omitempty"`
// Key is the operation's field holding the entry's key (EffectSetEntry, EffectRemove).
Key string `json:"key,omitempty"`
// Sequence and Author are whether the effect also writes the operation's sequence and its proven author.
Sequence bool `json:"sequence,omitempty"`
Author bool `json:"author,omitempty"`
}
// LedgerIndex is one declared index of a ledger type: its name, and the fields it is kept on.
type LedgerIndex struct {
Name string `json:"name"`
On []string `json:"on"`
}
// BusStateField is where a stored manifest keeps its bus-state entries: beside `requires`, which stays a plain list of
// names, as OptionalRequiresKey keeps the optional ones.
const BusStateField = "bus-state"
// BusStateEntry is one piece of a module's bus state, as its manifest asks for it.
type BusStateEntry struct {
// Name is its local name; the mesh, never the module, names it on the bus (ADR 0201 §4).
Name string `json:"name"`
// Storage is its storage type (design 25 § 12.1): how the bus keeps it.
Storage string `json:"storage"`
// Template is the predefined configuration of that storage type it uses; empty for a bare stream, which gives
// Configuration instead.
Template string `json:"template,omitempty"`
// Options are the template's settings the module changes, by option name; each value as the manifest wrote it.
Options map[string]json.RawMessage `json:"options,omitempty"`
// Configuration is a configuration of the module's own, where no template fits: a bare stream's, or a ledger's
// whose history expires (ADR 0595 §2).
Configuration map[string]json.RawMessage `json:"configuration,omitempty"`
// Types are a ledger's types (ADR 0597 §1), for the storage type ledger only.
Types []LedgerType `json:"types,omitempty"`
}
// isBusStateEntry is whether a `requires` item in its long form asks for bus state: it names a storage type.
func isBusStateEntry(raw json.RawMessage) bool {
var keys map[string]json.RawMessage
if json.Unmarshal(raw, &keys) != nil {
return false
}
_, has := keys["storage"]
return has
}
// readBusStateEntry reads one bus-state entry leniently (ADR 0578 §3): a key it does not know, at any depth, is set
// aside and its path named, never refused. An error is only an entry of the wrong shape.
func readBusStateEntry(raw json.RawMessage) (BusStateEntry, []string, error) {
var tree any
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
if err := dec.Decode(&tree); err != nil {
return BusStateEntry{}, nil, err
}
var aside []string
strict := func(e *BusStateEntry) error {
b, err := json.Marshal(tree)
if err != nil {
return err
}
d := json.NewDecoder(bytes.NewReader(b))
d.DisallowUnknownFields()
return typedUnknown(d.Decode(e))
}
for tries := 0; ; tries++ {
var e BusStateEntry
err := strict(&e)
if err == nil {
return e, aside, nil
}
unknown := asUnknownField(err)
if unknown == nil || tries > 64 {
return BusStateEntry{}, nil, err
}
path, ok := removalThatHelps(tree, unknown.Field, err.Error(), func() error { var e BusStateEntry; return strict(&e) })
if !ok {
if removeEvery(tree, unknown.Field) == 0 {
return BusStateEntry{}, nil, err
}
path = "…." + unknown.Field
}
aside = append(aside, path)
}
}
// setAsideAlso adds paths read without to what a manifest set aside (SetAside), in its words.
func setAsideAlso(unknown string, paths []string) string {
if len(paths) == 0 {
return unknown
}
if unknown == "" {
field := paths[0][strings.LastIndex(paths[0], ".")+1:]
unknown = fmt.Sprintf("json: unknown field %q", field)
}
return unknown + " (read without " + strings.Join(paths, ", ") + ")"
}
+81
View File
@@ -0,0 +1,81 @@
package catalogue
import "encoding/json"
// CatalogSeat is the seat of the catalog: what the mesh holds of every module (novox/hq ADR 0604 §1). Its states are
// its holder's: the modules the trunk holds, their manifests by digest and the module graph (the indexer's states
// `modules`, `manifests` and `graph`), announced with who may read them; a reader that only watches reads them and calls
// no verb. It names no event. What must be built again is no verb: a module whose fingerprint differs from its newest
// build, which the delivery view derives. The verbs' answers are CatalogModules, CatalogOne, CatalogProvides and
// CatalogDependents, whose field names are those of the catalog module's tools they replace.
const CatalogSeat = "mesh-catalog"
// catalogVerbs are the seat's four verbs, each read from its holder's states. Each is optional until the holder that
// runs serves it (design 33 §7): the module that claimed the seat before serves none of them, and a judge holding them
// required would read it as outgrown for as long as both claims stand (novox/hq ADR 0604 §6).
func catalogVerbs() []Verb {
return []Verb{
{Name: "modules", Optional: true, Description: "Every module the trunk holds, each with its repository, its " +
"directory there, the merge commit it was last changed at and its fingerprint (or why it has none): the " +
"catalog of the mesh, read from its holder's state. Reads only.",
Input: schema(map[string]string{}, nil)},
{Name: "module", Optional: true, Description: "One module whole: its repository, directory, commit and " +
"fingerprint, and its manifest as the trunk holds it. Older versions are in its repository's history. Reads only.",
Input: schema(map[string]string{"module": "the module's name"}, []string{"module"})},
{Name: "provides", Optional: true, Description: "Which modules provide a provision (a module provides its own " +
"name too) and which require it, from the module graph's provision edges. Reads only.",
Input: schema(map[string]string{"provision": "the provision, such as postgres-database"}, []string{"provision"})},
{Name: "dependents", Optional: true, Description: "The modules whose build edges reach a module, directly or " +
"through others (a base, a mesh package it publishes, the build module that builds them): what a change of it " +
"builds again, from the module graph. Reads only.",
Input: schema(map[string]string{"module": "the module's name"}, []string{"module"})},
}
}
// CatalogEntry is one module of the catalog: its repository, its directory there ("." for the root), the merge commit it
// was last changed at, and its new-form fingerprint or why it has none.
type CatalogEntry struct {
Module string `json:"module"`
Repository string `json:"repository"`
Path string `json:"path"`
Commit string `json:"commit"`
FP string `json:"fp,omitempty"`
None string `json:"none,omitempty"`
}
// CatalogModule is one module whole: its entry and its manifest as the trunk holds it.
type CatalogModule struct {
CatalogEntry
Manifest json.RawMessage `json:"manifest"`
}
// CatalogEnd is a module that provides, requires or depends on something, with its commit; Scope is a provider's scope
// (a module provides its own name at node scope).
type CatalogEnd struct {
Module string `json:"module"`
Commit string `json:"commit"`
Scope string `json:"scope,omitempty"`
}
// CatalogModules is the answer of the verb modules: every module the trunk holds, sorted by name.
type CatalogModules struct {
Modules []CatalogEntry `json:"modules"`
}
// CatalogOne is the answer of the verb module.
type CatalogOne struct {
Module CatalogModule `json:"module"`
}
// CatalogProvides is the answer of the verb provides: the modules that provide the provision and those that require it.
type CatalogProvides struct {
Provision string `json:"provision"`
Provides []CatalogEnd `json:"provides"`
Requires []CatalogEnd `json:"requires"`
}
// CatalogDependents is the answer of the verb dependents: the modules whose build edges reach the module.
type CatalogDependents struct {
Module string `json:"module"`
Dependents []CatalogEnd `json:"dependents"`
}
@@ -1,58 +0,0 @@
package catalogue
import (
"slices"
"strings"
"testing"
)
// Issue 1126 B1/AC1, B2/AC1 (ADR 0590 clause 5): the seat set holds no mesh-controller, and the dependence check refuses
// every manifest that reads (the lease too), invokes, consumes, waits on, uses, claims, requires or is mesh-controller.
func TestNothingMayNameTheController(t *testing.T) {
for _, s := range DefaultSeats() {
if s.Name == EndedControllerName {
t.Errorf("the seat set still holds %s", s.Name)
}
}
if _, ok := SeatNamed(EndedControllerName); ok {
t.Errorf("SeatNamed still finds %s", EndedControllerName)
}
for _, row := range []struct {
name, manifest string
want []string // "<kind> <what>"
}{
{"a read of the lease", `{"module":"mod-l","version":"1","reads":["mesh-controller.lease"]}`, []string{"read mesh-controller.lease"}},
{"the controller's own manifest", `{"module":"mesh-controller","version":"1"}`, []string{"is mesh-controller"}},
{"a claim of its seat", `{"module":"a-controller","version":"1","claims":[{"name":"mesh-controller","scope":"mesh"}]}`,
[]string{"claims mesh-controller"}},
{"each declaration kind; every tool (*) is none",
`{"module":"mod-i","version":"1","invokes":["seat:mesh-controller.root-free","mesh-controller.status","*","nats.nats_users"],` +
`"consumes":["mesh-controller.artifacts-gone","mod-x.thing"],"uses":["mesh-controller"],"waits":["mesh-controller.checked"],` +
`"requires":["mesh-controller"]}`,
[]string{"consume mesh-controller.artifacts-gone", "invoke mesh-controller.status", "invoke seat:mesh-controller.root-free",
"requires mesh-controller", "uses mesh-controller", "waits mesh-controller.checked"}},
{"a process's own invoke", `{"module":"mod-p","version":"1","resources":[{"id":"account","type":"user","name":"mod-p"},` +
`{"id":"worker","type":"process","run":["./w"],"user":"mod-p","bus":{"invokes":["seat:mesh-controller.held"]}}]}`,
[]string{"invoke seat:mesh-controller.held"}},
{"a module on its own seats", `{"module":"mod-ok","version":"1","reads":["mesh-grants.accounts"],"invokes":["*"]}`, nil},
} {
m, err := Decode([]byte(row.manifest))
if err != nil {
t.Fatalf("%s: %v", row.name, err)
}
var got []string
for _, d := range Dependencies(map[string]Manifest{m.Module: m}) {
got = append(got, d.Kind+" "+d.What)
}
if !slices.Equal(got, row.want) {
t.Errorf("%s: dependencies %q, want %q", row.name, got, row.want)
}
refused := ManifestRefusals(map[string]Manifest{m.Module: m})
if len(refused) != len(row.want) || len(refused) > 0 && !strings.Contains(refused[0], "ADR 0590") {
t.Errorf("%s: refused %q", row.name, refused)
}
}
if got := ManifestRefusals(map[string]Manifest{}); len(got) != 1 || !strings.Contains(got[0], "no manifest") {
t.Errorf("no manifest given: %q", got)
}
}
+4 -5
View File
@@ -4,11 +4,10 @@ import (
"encoding/json"
)
// CoreCredentialModules are the modules whose own secrets are the core's credentials: the bus, mesh-provisioner and
// its client. The node-engine and the controller hold no own secret of a module's. **Fixed here, in the controller's
// code, as the clause says** — and the vault keeps the same list (mesh-vault's CoreModules), answering the controller
// for these alone.
var CoreCredentialModules = []string{"mesh-provisioner", "mesh-provisioner-client", "nats"}
// CarriesCoreCredentials says whether a module's own secrets are the core's credentials, as its manifest says
// (Manifest.CoreCredentials): the vault seals them to the machine, and the node-engine and the vault hold no list of
// such modules.
func CarriesCoreCredentials(m Manifest) bool { return m.CoreCredentials }
// VaultSealedField is the field of a file the vault sealed to its machine (mesh-host's File.VaultSealed).
const VaultSealedField = "vault-sealed"
+1 -1
View File
@@ -84,7 +84,7 @@ type Rendering struct {
// makes. Nil on every composition that is sent, and on every other question, which then refuse
// a secret not made with a *NotMadeError.
Foreseen map[string]map[string]bool
// VaultSealed is, for a module on the core's list (CoreCredentialModules), each own secret as the vault sealed it
// VaultSealed is, for a module whose manifest says its own secrets are the core's credentials (core-credentials), each own secret as the vault sealed it
// to this machine (the mesh's sealed box, as the vault answered it), keyed like Needed (novox/hq ADR 0349 §16,
// issue 755). Given only for a machine whose engine opens the box (VaultBoxContract): its file then carries the
// box and never the controller's own sealed value. Nil everywhere else, and the composition is as it was.
+54
View File
@@ -1,6 +1,8 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
@@ -15,4 +17,56 @@ func TestAManifestMaySayWhatItInvokes(t *testing.T) {
if len(m.Invokes) != 2 || m.Invokes[1] != "*" {
t.Fatalf("invokes not read: %v", m.Invokes)
}
// And that its own secrets are the core's credentials (issue 774): read, and written back as said.
for raw, core := range map[string]bool{`{"module":"bus","version":"1","core-credentials":true}`: true, `{"module":"app","version":"1"}`: false} {
m, err := decodeForTest([]byte(raw))
if err != nil || m.CoreCredentials != core || m.UnknownField() != "" {
t.Errorf("%s: core credentials %v, %v, %q", raw, m.CoreCredentials, err, m.UnknownField())
}
back, _ := json.Marshal(m)
if strings.Contains(string(back), "core-credentials") != core {
t.Errorf("%s written back as %s", raw, back)
}
if CarriesCoreCredentials(m) != core {
t.Errorf("%s: carries the core's credentials %v", raw, !core)
}
}
// And that it is the tool runner (issue 774): read, written back, and found among others by that alone.
runner, err := decodeForTest([]byte(`{"module":"any-name","version":"1","tool-runner":true}`))
if err != nil || !IsToolRunner(runner) || runner.UnknownField() != "" {
t.Fatalf("a tool runner: %v %q", err, runner.UnknownField())
}
if back, _ := json.Marshal(runner); !strings.Contains(string(back), `"tool-runner":true`) {
t.Errorf("a tool runner written back as %s", back)
}
other, _ := decodeForTest([]byte(`{"module":"node-tools","version":"1"}`))
if got, ok := ToolRunnerOf([]Manifest{other, runner}); !ok || got.Module != "any-name" {
t.Errorf("the tool runner found is %q, %v: by what it says, never by its name", got.Module, ok)
}
// A module's seats are derived from what it declares, whatever its name: a service on a machine is the service
// manager's, the node-engine's name exempts nothing; a module whose resources are computed depends on none.
host, _ := decodeForTest([]byte(`{"module":"mesh-host","version":"1","resources":[{"id":"s","type":"service","name":"s.service"}]}`))
if got := DependsOn(nil, host); len(got) != 1 || got[0] != ServiceManagerSeat {
t.Errorf("a service by the node-engine's name depends on %v", got)
}
if got := DependsOn(nil, Manifest{Module: "net", Computed: "net", Resources: host.Resources}); len(got) != 0 {
t.Errorf("a computed module depends on %v", got)
}
// And its upgrade policy, which holds for the bus as for any module: providing the bus derives nothing (issue 774).
for raw, want := range map[string]string{
`{"module":"bus","version":"1","provides":[{"name":"mesh-bus"}],"upgrade":{"policy":"record","why":"its streams"}}`: PolicyRecord + " " + FromModule,
`{"module":"bus","version":"1","provides":[{"name":"mesh-bus"}]}`: PolicyRoll + " " + FromDefault,
} {
m, err := decodeForTest([]byte(raw))
if err != nil {
t.Fatal(err)
}
if policy, from, _ := DerivedUpgrade(m); policy+" "+from != want {
t.Errorf("%s: derived %s %s, want %s", raw, policy, from, want)
}
chosen := &PersonPolicy{Policy: PolicyTogether, Why: "w", By: "a person"}
if policy, from, _, _ := Upgrade(m, chosen); policy != PolicyTogether || from != FromPerson {
t.Errorf("%s: a person's choice gave %s %s", raw, policy, from)
}
}
}
@@ -1,5 +1,12 @@
package catalogue
// This file keeps, deprecated, the exported names feat/l3b-bus removed from this package while a running part still
// uses them: an exported name is removed only after every user of it is delivered. The checker builds the running
// node-engine's composer (mesh-host main) beside this mesh-sdk (ADR 0548 §6), and mesh-host main, mesh-delivery main
// (the indexer's judge, the assigner's and mesh-grants' code and tests) and mesh-tools main (node-tools' tests) name
// them. Nothing in L3's own heads uses any of them. The second group removes this file, and its exemption in
// TestNoCatalogueManifestNamesAmqp, once mesh-host a24a1d72 and L3's mesh-delivery and mesh-tools heads run.
import (
"fmt"
"slices"
@@ -7,10 +14,14 @@ import (
"strings"
)
// Deprecated: kept until its users are delivered (the head of this file).
//
// EndedControllerName is the name the ended controller had, as its module and as its seat: no manifest may name it.
const EndedControllerName = "mesh-controller"
// The kinds of a declaration that names the controller.
//
// Deprecated: kept until their users are delivered (the head of this file).
const (
DependsRead = "read"
DependsInvoke = "invoke"
@@ -22,6 +33,8 @@ const (
DependsIs = "is"
)
// Deprecated: kept until its users are delivered (the head of this file).
//
// Dependency is one declaration of a manifest that names the controller.
type Dependency struct {
Module string `json:"module"`
@@ -43,6 +56,8 @@ func invokesController(invoke string) bool {
return onController(invoke)
}
// Deprecated: kept until its users are delivered (the head of this file).
//
// Dependencies are every declaration naming the controller in the manifests given, by module.
func Dependencies(manifests map[string]Manifest) []Dependency {
var out []Dependency
@@ -118,6 +133,8 @@ func Dependencies(manifests map[string]Manifest) []Dependency {
return out
}
// Deprecated: kept until its users are delivered (the head of this file).
//
// ManifestRefusals is the check over a repository's manifests, by module name: one reason per declaration naming the
// controller, none when it passes. None given fails it, since nothing read can hide a declaration.
func ManifestRefusals(manifests map[string]Manifest) []string {
@@ -135,3 +152,37 @@ func ManifestRefusals(manifests map[string]Manifest) []string {
}
return out
}
// Deprecated: a manifest says it (Manifest.CoreCredentials, CarriesCoreCredentials); kept until its users are
// delivered (the head of this file).
//
// CoreCredentialModules are the modules whose own secrets are the core's credentials: the bus, mesh-provisioner and
// its client.
var CoreCredentialModules = []string{"mesh-provisioner", "mesh-provisioner-client", "nats"}
// Deprecated: a manifest says it (Manifest.ToolRunner, IsToolRunner); kept until its users are delivered (the head
// of this file).
//
// RuntimeModule is the module that is the node's tool runtime.
const RuntimeModule = "node-tools"
// Deprecated: the kept streams are the indexer's judge's; kept until its users are delivered (the head of this file).
//
// StreamsKeptUntilMoved are the streams declared before the check that a module keeps its data in the bus's
// predefined forms, each allowed by name until its module moves it.
var StreamsKeptUntilMoved = map[string][]string{
"brother": {"device-asks"},
"mesh-build-go": {"toolchains-go"},
"mesh-build-node": {"toolchains-node"},
"mesh-mail": {"mail", "mail-sends"},
"only-office": {"convert-asks"},
"mesh-delivery-assigner": {"placement"},
"mesh-delivery-checker": {"checks"},
"mesh-delivery-forge-bridge": {"forge"},
}
// Deprecated: nothing derives it now (DerivedUpgrade reads the bus's own manifest); kept until its users are
// delivered (the head of this file).
//
// FromBus is the source an older DerivedUpgrade gave the bus's policy.
const FromBus = "the bus"
+26 -6
View File
@@ -478,6 +478,10 @@ type Manifest struct {
// RequireShares is who else may use the bucket a module requires, by requirement (BucketShare, novox/hq ADR 0468
// §8): filled from the long form's `shares`, or from the stored form's `requires-shares`; never written by hand.
RequireShares map[string][]BucketShare `json:"-"`
// BusState is what the module keeps on the bus, each entry asked for in `requires` in its typed long form
// (BusStateEntry, novox/hq ADR 0595 §1); its local name is never in Requires, which names provisions. Stored
// beside `requires` (BusStateField), which stays a plain list of names.
BusState []BusStateEntry `json:"bus-state,omitempty"`
// Emits are the events this module publishes, named **locally**: `order.placed`, not a subject
// and not a routing key. The mesh derives where it lands (design 29 §1), so reorganising the
@@ -636,7 +640,7 @@ type Manifest struct {
// Upgrade is how this module's new builds reach its machines (novox/hq ADR 0236): rolled out one
// machine first and gated when unsaid; `together`, or `record` — wait for a person's push — with
// why. A person's choice through the `upgrade` verb stands over it; the bus records whatever it says.
// why. A person's choice through the `upgrade` verb stands over it. The bus says `record` itself.
Upgrade *UpgradePolicy `json:"upgrade,omitempty"`
// Unhealthy is how loud this module's unhealthy condition is (novox/hq ADR 0355): a warning when unsaid;
@@ -797,6 +801,17 @@ type Manifest struct {
// start, and is then replaced.
OwnSecrets OwnSecrets `json:"own-secrets,omitempty"`
// ToolRunner says this module is the node's tool runner (issue 774): the one program on each node that loads every
// assigned module's tools bundle, served by its runtime process and the only module that may declare `waits`. Said by
// the module, never by a name in mesh-sdk.
ToolRunner bool `json:"tool-runner,omitempty"`
// CoreCredentials says this module's own secrets are the core's credentials (novox/hq ADR 0349 §16, issue 755): the
// vault seals each to the machine, and the node-engine writes the vault's box into its file, never a value of its
// own (Rendering.VaultSealed). Said by the module, never by a list in mesh-sdk (the operator, 2026-11-03: mesh-sdk names
// no module): the bus and the provisioner and its client say it.
CoreCredentials bool `json:"core-credentials,omitempty"`
// SecretsOwner is who the files holding this module's secrets belong to on the machine —
// `uid:gid`, or a name — when its process is not root.
//
@@ -1556,18 +1571,21 @@ func (m *Manifest) UnmarshalJSON(raw []byte) error {
}
var optionalRequires map[string]bool
var requireShares map[string][]BucketShare
var busState []BusStateEntry
var busStateAside []string
if requires, ok := keys["requires"]; ok && string(requires) != "null" {
names, optional, shares, err := readRequires(requires)
read, err := readRequires(requires)
if err != nil {
return err
}
plainNames, err := json.Marshal(names)
plainNames, err := json.Marshal(read.names)
if err != nil {
return err
}
keys["requires"] = plainNames
optionalRequires = optional
requireShares = shares
optionalRequires = read.optional
requireShares = read.shares
busState, busStateAside = read.busState, read.aside
}
// The stored form's shares (RequireSharesKey).
if stored, ok := keys[RequireSharesKey]; ok {
@@ -1631,7 +1649,9 @@ func (m *Manifest) UnmarshalJSON(raw []byte) error {
}
}
*m = Manifest(fields)
m.unknown = unknown
m.unknown = setAsideAlso(unknown, busStateAside)
// A bus-state entry of `requires`, beside those of the stored form (BusStateField).
m.BusState = append(m.BusState, busState...)
m.forms = forms
m.OptionalRequires = optionalRequires
m.RequireShares = requireShares
+33 -8
View File
@@ -21,34 +21,59 @@ type requirementForm struct {
Shares []BucketShare `json:"shares,omitempty"`
}
// readRequires reads `requires` in both of its forms — a bare name, or {name, optional} — into the
// names and which of them are optional.
func readRequires(raw json.RawMessage) ([]string, map[string]bool, map[string][]BucketShare, error) {
// requiresRead is `requires` as readRequires read it.
type requiresRead struct {
names []string
optional map[string]bool
shares map[string][]BucketShare
// busState are the entries asking for bus state (BusStateEntry), never named in names, which name provisions; aside
// the paths of the keys they had that this package does not know, set aside (ADR 0578 §3).
busState []BusStateEntry
aside []string
}
// readRequires reads `requires` in its forms — a bare name, {name, optional}, or a bus-state entry (BusStateEntry,
// novox/hq ADR 0595 §1) — into the names of the provisions, which of them are optional, and the bus-state entries,
// whose local names stay off the names: a local name is no provision, and nothing provides it.
func readRequires(raw json.RawMessage) (requiresRead, error) {
var entries []json.RawMessage
if err := json.Unmarshal(raw, &entries); err != nil {
return nil, nil, nil, fmt.Errorf("requires: a list of names, or of {name, optional}: %w", err)
return requiresRead{}, fmt.Errorf("requires: a list of names, or of {name, optional}: %w", err)
}
var names []string
optional := map[string]bool{}
var shares map[string][]BucketShare
var busState []BusStateEntry
var aside []string
for i, e := range entries {
var plain string
if err := json.Unmarshal(e, &plain); err == nil {
names = append(names, plain)
continue
}
if isBusStateEntry(e) {
entry, unknown, err := readBusStateEntry(e)
if err != nil {
return requiresRead{}, fmt.Errorf("requires[%d]: a bus-state entry is {name, storage, template, options, configuration, types} (novox/hq ADR 0595 §1): %w", i, err)
}
busState = append(busState, entry)
for _, path := range unknown {
aside = append(aside, fmt.Sprintf("requires[%d]%s", i, path))
}
continue
}
var full requirementForm
dec := json.NewDecoder(bytes.NewReader(e))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return nil, nil, nil, fmt.Errorf("requires[%d]: a name, or {name, optional} (novox/hq ADR 0330): %w", i, err)
return requiresRead{}, fmt.Errorf("requires[%d]: a name, or {name, optional} (novox/hq ADR 0330): %w", i, err)
}
if strings.TrimSpace(full.Name) == "" {
return nil, nil, nil, fmt.Errorf("requires[%d] names no requirement: {name, optional}", i)
return requiresRead{}, fmt.Errorf("requires[%d] names no requirement: {name, optional}", i)
}
names = append(names, full.Name)
if err := checkBucketShares(full.Name, full.Shares); err != nil {
return nil, nil, nil, fmt.Errorf("requires[%d]: %w", i, err)
return requiresRead{}, fmt.Errorf("requires[%d]: %w", i, err)
}
if len(full.Shares) > 0 {
if shares == nil {
@@ -63,7 +88,7 @@ func readRequires(raw json.RawMessage) ([]string, map[string]bool, map[string][]
if len(optional) == 0 {
optional = nil
}
return names, optional, shares, nil
return requiresRead{names: names, optional: optional, shares: shares, busState: busState, aside: aside}, nil
}
// OptionalRequiresKey is where a stored manifest keeps which of its requirements are optional: beside
+96 -17
View File
@@ -2,6 +2,7 @@ package catalogue
import (
"encoding/json"
"reflect"
"strings"
"testing"
)
@@ -9,9 +10,8 @@ import (
// A requirement a module can run without (novox/hq ADR 0330, issue 166). The shape of the issue:
// grafana runs without a data source and requires `influxdb-api` softly; a mesh without influxdb
// could not assign grafana at all, and one with influxdb in its catalogue would have had it installed
// for grafana.
func TestARequirementIsHardUnlessMarkedOptional(t *testing.T) {
// for grafana. And `requires` in its typed long form, a module's bus state (novox/hq ADR 0595 §1).
func TestARequirementsLongForm(t *testing.T) {
m := mustParse(t, `{"module":"grafana","version":"1",
"requires":["route",{"name":"influxdb-api","optional":true},{"name":"oidc-client"}]}`)
if strings.Join(m.Requires, " ") != "route influxdb-api oidc-client" {
@@ -24,23 +24,12 @@ func TestARequirementIsHardUnlessMarkedOptional(t *testing.T) {
t.Error("a requirement marked optional read as hard")
}
// And written back as it was read, so a manifest through the store keeps its form.
raw, err := json.Marshal(m)
if err != nil {
t.Fatal(err)
}
var again Manifest
if err := json.Unmarshal(raw, &again); err != nil {
t.Fatal(err)
}
again := roundTrip(t, m)
if !again.IsOptional("influxdb-api") || again.IsOptional("route") || len(again.Requires) != 3 {
t.Fatalf("written and read back, it says %v optional %v (%s)", again.Requires, again.OptionalRequires, raw)
t.Fatalf("written and read back, it says %v optional %v", again.Requires, again.OptionalRequires)
}
if strings.Contains(string(raw), `"name":`) {
t.Errorf("a requirement was stored in the long form: %s", raw)
}
}
func TestARequirementsLongFormRefusesWhatItDoesNotKnow(t *testing.T) {
// The long form {name, optional} refuses what it does not know.
for _, raw := range []string{
`{"module":"x","version":"1","requires":[{"name":"influxdb-api","optinal":true}]}`,
`{"module":"x","version":"1","requires":[{"optional":true}]}`,
@@ -49,4 +38,94 @@ func TestARequirementsLongFormRefusesWhatItDoesNotKnow(t *testing.T) {
t.Errorf("accepted %s", raw)
}
}
records := BusStateEntry{Name: "records", Storage: "ledger", Template: "ledger",
Options: map[string]json.RawMessage{"max-mib": json.RawMessage(`1024`)}}
// A ledger type with each of the four effects.
listing := LedgerType{Name: "listing",
Schema: map[string]LedgerField{"price": {Type: "number"}, "seen": {Type: "list"}, "notes": {Type: "map"},
"text": {Type: "text"}, "digest": {Type: "text", DigestOf: "text"}},
Operations: []LedgerOperation{
{Name: "priced", Effect: EffectSetFields, Fields: []string{"price"}, Sequence: true, Author: true},
{Name: "noted", Effect: EffectSetEntry, Fields: []string{"text", "digest"}, Into: "notes", Key: "title"},
{Name: "seen", Effect: EffectAppend, Fields: []string{"at"}, Into: "seen"},
{Name: "unnoted", Effect: EffectRemove, Into: "notes", Key: "title"},
},
Indexes: []LedgerIndex{{Name: "cheapest", On: []string{"price"}}}}
typeJSON, err := json.Marshal(listing)
if err != nil {
t.Fatal(err)
}
ledger := records
ledger.Types = []LedgerType{listing}
for _, c := range []struct {
name string
requires string
names string
busState []BusStateEntry
aside []string
}{
// 1966/AC1: a plain name and a bus-state entry: Requires names the plain one only, since it names provisions,
// and BusState holds the entry whole.
{"a plain name and a bus-state entry", `["postgres-database",
{"name":"records","storage":"ledger","template":"ledger","options":{"max-mib":1024}}]`,
"postgres-database", []BusStateEntry{records}, nil},
// 1966/AC2: an unknown key is set aside and named, never refused; the entry decodes.
{"a bus-state entry with a key it does not know", `[
{"name":"records","storage":"ledger","template":"ledger","colour":"red","options":{"max-mib":1024}}]`,
"", []BusStateEntry{records}, []string{"requires[0].colour"}},
// 1966/AC3: a ledger type with each of the four effects round-trips through JSON unchanged.
{"a ledger with its type", `[{"name":"records","storage":"ledger","template":"ledger","options":{"max-mib":1024},
"types":[` + string(typeJSON) + `]}]`, "", []BusStateEntry{ledger}, nil},
} {
got, err := decodeForTest([]byte(`{"module":"x","version":"1","requires":` + c.requires + `}`))
if err != nil {
t.Errorf("%s: refused: %v", c.name, err)
continue
}
if strings.Join(got.Requires, " ") != c.names {
t.Errorf("%s: requires read as %v, want %s", c.name, got.Requires, c.names)
}
if !reflect.DeepEqual(got.BusState, c.busState) {
t.Errorf("%s: bus state read as %+v, want %+v", c.name, got.BusState, c.busState)
}
for _, path := range c.aside {
if !strings.Contains(got.SetAside(), path) {
t.Errorf("%s: %s is not named among what was set aside: %q", c.name, path, got.SetAside())
}
}
if len(c.aside) == 0 && got.SetAside() != "" {
t.Errorf("%s: set aside %q", c.name, got.SetAside())
}
// Stored and read back, the entries stay whole and `requires` stays plain names.
back := roundTrip(t, got)
if !reflect.DeepEqual(back.BusState, c.busState) || strings.Join(back.Requires, " ") != c.names {
t.Errorf("%s: read back as %v %+v", c.name, back.Requires, back.BusState)
}
}
}
// roundTrip is m written as the store keeps it and read again; the stored form keeps `requires` plain names.
func roundTrip(t *testing.T, m Manifest) Manifest {
t.Helper()
raw, err := json.Marshal(m)
if err != nil {
t.Fatal(err)
}
var stored struct {
Requires []any `json:"requires"`
}
if err := json.Unmarshal(raw, &stored); err != nil {
t.Fatal(err)
}
for _, r := range stored.Requires {
if _, plain := r.(string); !plain {
t.Errorf("a requirement was stored in the long form: %s", raw)
}
}
var again Manifest
if err := json.Unmarshal(raw, &again); err != nil {
t.Fatalf("%v: %s", err, raw)
}
return again
}
+13 -3
View File
@@ -1,8 +1,18 @@
package catalogue
// RuntimeModule is the module that is the node's tool runtime. Mirrored in the broker package,
// which composes a principal of its own for it; the agreement test there holds the two to one string.
const RuntimeModule = "node-tools"
// IsToolRunner says whether a manifest is the node's tool runner's, as it says itself (`tool-runner`, issue 774): the
// one program on each node that loads every assigned module's tools bundle. Never known here by its name.
func IsToolRunner(m Manifest) bool { return m.ToolRunner }
// ToolRunnerOf is the module among manifests that says it is the tool runner, and whether one does.
func ToolRunnerOf(manifests []Manifest) (Manifest, bool) {
for _, m := range manifests {
if IsToolRunner(m) {
return m, true
}
}
return Manifest{}, false
}
// BundleRoot is where a machine keeps the tools bundles the mesh delivers to it: under the mesh's
// own directory, beside the daemons the host unpacks there, and never where a package manager also
+5 -17
View File
@@ -24,24 +24,12 @@ var appliedThrough = map[string]string{
"container": ContainerRuntimeSeat,
}
// foundationModules are the pieces genesis lays before any module exists (novox/hq ADR 0207 §5):
// the host and the private network. Registered as modules so they can be assigned, but what they
// declare is the installation's, not a module's, so it is never judged. The third piece, the
// bootstrap container runtime, is not a module at all: its package and service are in the genesis
// bundle the host applies itself, and never pass through a resolution here.
//
// The private network's module is overlay.Name, written out because the overlay package composes
// on top of this one; its resources are computed, which exempts it by the rule below as well.
var foundationModules = map[string]bool{
"mesh-host": true,
"mesh-wireguard": true,
}
// isFoundation is whether a module's declarations are the foundation's rather than its own. A
// module whose resources are computed is the mesh's by construction — the private network's peer
// list and its tools are the controller's, written per node — so it counts whatever its name.
// isFoundation is whether a module's declarations are the foundation's rather than its own: a module whose resources are
// computed (`computed`, the private network's peer list written per node) is the mesh's by construction. Read from the
// manifest alone, never from its name (issue 774, the operator 2026-11-03): the node-engine's own manifest declares no
// resource a seat applies, so it depends on none by the rules below.
func isFoundation(m Manifest) bool {
return foundationModules[m.Module] || m.Computed != ""
return m.Computed != ""
}
// DependsOn is every seat a module needs held on its node: derived from the resource types it
+2 -1
View File
@@ -272,7 +272,8 @@ var defaultSeats = append([]Seat{
// images and archives, by digest — which is why the provision is the artifact store and not an
// image registry.
{Name: "mesh-artifact-store", Scope: ScopeMesh, Delivers: "artifact-store", Decision: "novox/hq ADR 0075"},
{Name: "mesh-catalog", Scope: ScopeMesh, Decision: "novox/hq ADR 0121"},
// The catalog (novox/hq ADR 0604): what the mesh holds of every module, its verbs read from its holder's states (CatalogSeat).
{Name: CatalogSeat, Scope: ScopeMesh, Serves: catalogVerbs(), Decision: "novox/hq ADR 0604"},
// The delivery's owner (novox/hq ADR 0239): one commit's journey from its pull request's head to every
// machine, and a group of them, recorded and ordered by its holder, which asks the controller for each act.
// What it says it says as its module (`mesh-delivery.transition`), through the runtime that launches it.
-16
View File
@@ -370,19 +370,3 @@ func uncovered(patterns []string) []string {
slices.Sort(out)
return out
}
// **A module keeps its data in the bus's predefined forms** (the lead and the operator, 2026-10-29; the operator's rule of
// 2026-10-22, "data storage is done using our predefined nats bus options"): a state, a log or an object store, which the
// runtime reads and writes for its tools and other modules read by a declared `reads`. A stream of its own is outside
// them and refused, naming it. StreamsKeptUntilMoved are the streams declared before this check, each allowed by name
// until its module moves it; the list only shrinks.
var StreamsKeptUntilMoved = map[string][]string{
"brother": {"device-asks"},
"mesh-build-go": {"toolchains-go"},
"mesh-build-node": {"toolchains-node"},
"mesh-mail": {"mail", "mail-sends"},
"only-office": {"convert-asks"},
"mesh-delivery-assigner": {"placement"},
"mesh-delivery-checker": {"checks"},
"mesh-delivery-forge-bridge": {"forge"},
}
+4 -4
View File
@@ -75,7 +75,7 @@ func prunedFields(keys map[string]json.RawMessage) (manifestFields, []string, er
removed = append(removed, k)
break
}
path, ok := removalThatHelps(k, tree[k], unknown.Field, err.Error())
path, ok := removalThatHelps(tree[k], unknown.Field, err.Error(), func() error { return decodesAlone(k, tree[k]) })
if !ok {
// The same key unknown in two entries alike: no one removal changes the words.
// Every occurrence goes, and the field is judged again.
@@ -113,8 +113,8 @@ func decodesAlone(k string, v any) error {
}
// removalThatHelps removes, from v, the one occurrence of key whose removal changes what the strict
// decoder says of field k, and says where it was. Every other occurrence is left as it was.
func removalThatHelps(k string, v any, key, said string) (string, bool) {
// decoder says (decodes, which decodes v strictly), and says where it was. Every other occurrence is left as it was.
func removalThatHelps(v any, key, said string, decodes func() error) (string, bool) {
var found bool
var where string
var walk func(node any, at string) bool
@@ -126,7 +126,7 @@ func removalThatHelps(k string, v any, key, said string) (string, bool) {
// Accepted only when the decoder is past it: nothing left, or an unknown key said
// elsewhere. A different kind of error means the removal broke the entry; the same
// words mean this was not the occurrence it refused.
err := decodesAlone(k, v)
err := decodes()
if err == nil || (asUnknownField(err) != nil && err.Error() != said) {
found, where = true, at+"."+key
return true
+4 -11
View File
@@ -30,25 +30,19 @@ type UpgradePolicy struct {
const (
FromPerson = "person"
FromModule = "module"
FromBus = "the bus"
FromData = "irreplaceable data"
FromDefault = "default"
)
// DerivedUpgrade is the policy a module's manifest gives it when no person has chosen one, with where
// it came from and why (ADR 0236):
// it came from and why (ADR 0236), from what the manifest says alone (the operator, 2026-11-03: no module is
// treated apart here; the bus says `record` in its own manifest):
//
// - **the bus is never rolled**: a module that provides the mesh's bus records, whatever it says — its
// upgrade is a planned step a person starts (to-be 45 §8);
// - a module that says its policy has it;
// - a module that keeps irreplaceable data records — sending the previous build cannot undo what a new
// one did to data that cannot be had again, so a person takes it, after a backup;
// - everything else rolls out, one machine first.
func DerivedUpgrade(m Manifest) (policy, from, why string) {
if ProvidesBus(m) {
return PolicyRecord, FromBus, "the bus is replaced only as a planned step a person starts (`bus upgrade`): " +
"its streams are snapshotted first and checked after"
}
if m.Upgrade != nil && m.Upgrade.Policy != "" {
return m.Upgrade.Policy, FromModule, m.Upgrade.Why
}
@@ -126,11 +120,10 @@ func (p PersonPolicy) Check() error {
// Upgrade is a module's policy and where it comes from (ADR 0565 clause 7): the one derivation every reader takes,
// `mesh-wants.upgrade` and each node agent's judgement of a move alike. A person's choice over the module's word, its
// irreplaceable data and the default; the bus is never rolled, whoever says so (ADR 0236). By names the person when one
// chose.
// irreplaceable data and the default. By names the person when one chose.
func Upgrade(m Manifest, chosen *PersonPolicy) (policy, from, why, by string) {
policy, from, why = DerivedUpgrade(m)
if from == FromBus || chosen == nil {
if chosen == nil {
return policy, from, why, ""
}
return chosen.Policy, FromPerson, chosen.Why, chosen.By
+48
View File
@@ -1,6 +1,7 @@
package catalogue
import (
"encoding/json"
"slices"
"strings"
"testing"
@@ -50,6 +51,53 @@ func TestHoldingAMeshSeatRequiresServingItsVerbs(t *testing.T) {
t.Errorf("a holder saying it serves mint: %v", err)
}
// L3: the seat mesh-catalog promises four verbs, each optional while two claimants stand (ADR 0604 §1, §6).
catalog, ok := SeatNamed(CatalogSeat)
if !ok || catalog.Scope != ScopeMesh || catalog.Decision != "novox/hq ADR 0604" || len(catalog.Serves) != 4 {
t.Fatalf("mesh-catalog: in the set %v, %+v", ok, catalog)
}
four := []string{"modules", "module", "provides", "dependents"}
for i, v := range catalog.Serves {
if v.Name != four[i] || !v.Optional || v.Input == nil {
t.Errorf("mesh-catalog's verb %d is %+v, want %s, optional, with its input", i, v, four[i])
}
}
// The answers keep the field names of the catalog module's tools they replace (catalog_modules, catalog_module,
// catalog_provides, catalog_dependents, as mesh_call answered them on 2026-11-03).
for _, c := range []struct {
answer any
want []string
}{
{CatalogModules{Modules: []CatalogEntry{{}}}, []string{`"modules":[{"module":`, `"repository":`, `"path":`, `"commit":`}},
{CatalogOne{Module: CatalogModule{Manifest: json.RawMessage(`{}`)}}, []string{`"module":{"module":`, `"manifest":{}`}},
{CatalogProvides{Provides: []CatalogEnd{{Scope: "mesh"}}, Requires: []CatalogEnd{{}}}, []string{`"provides":[{"module":"","commit":"","scope":"mesh"}]`, `"requires":[{"module":"","commit":""}]`}},
{CatalogDependents{Module: "m", Dependents: []CatalogEnd{}}, []string{`"module":"m"`, `"dependents":[]`}},
} {
raw, _ := json.Marshal(c.answer)
for _, w := range c.want {
if !strings.Contains(string(raw), w) {
t.Errorf("the answer %s does not hold %s", raw, w)
}
}
}
cataloguer := func(serves ...string) Manifest {
return Manifest{Module: "a-catalog", Tools: serves, Claims: []Claim{{Name: CatalogSeat, Scope: ScopeMesh, Serves: serves}}}
}
for name, c := range map[string]struct {
serves []string
word string
}{
"serving the four": {four, ""},
"serving three, optional": {four[:3], ""},
"serving none, the old one": {nil, ""},
"serving a fifth": {append(slices.Clone(four), "rebuild"), "rebuild"},
} {
err := CanHold(cataloguer(c.serves...), catalog)
if (c.word == "") != (err == nil) || (err != nil && !strings.Contains(err.Error(), c.word)) {
t.Errorf("a mesh-catalog holder %s: %v", name, err)
}
}
was := Seats()
t.Cleanup(func() { UseSeats(was) })
UseSeats([]Seat{{Name: "mesh-controller", Scope: ScopeMesh, Decision: "test",
+10 -6
View File
@@ -135,13 +135,17 @@ type ManifestRead struct {
}
// Machine is one machine composed: whether it composes (resolves, at DepthResolve), why not, the node-engine
// validator's problems and the composition's refusals (DepthCompose), and its declared paths when asked.
// validator's problems and the composition's refusals (DepthCompose), and its declared paths when asked. NotJudged says,
// in words, why the composer did not judge the machine (an older node-engine's input that does not say how it composes,
// novox/hq ADR 0593 §9): the machine is then neither composed nor refused, and a reader decides by this field alone,
// never by the words of Reason.
type Machine struct {
Composes bool `json:"composes"`
Reason string `json:"reason,omitempty"`
Problems []string `json:"problems,omitempty"`
Refusals []string `json:"refusals,omitempty"`
Declared []Path `json:"declared,omitempty"`
Composes bool `json:"composes"`
NotJudged string `json:"not_judged,omitempty"`
Reason string `json:"reason,omitempty"`
Problems []string `json:"problems,omitempty"`
Refusals []string `json:"refusals,omitempty"`
Declared []Path `json:"declared,omitempty"`
}
// Path is one path a declaration puts on the machine, with its kind (the resource's type), owner, group and mode as
+7 -1
View File
@@ -193,6 +193,10 @@ type Want struct {
// of policy record that the node agent judged to recreate a container or move its data is applied only when its
// want names the composition so allowed. Empty when none was asked.
Yes string `json:"yes,omitempty"`
// Release names the release whose Approve answered the record ask of this want's build (novox/hq ADR 0601 §1): the
// group of pull requests mesh-delivery's gate merged on that Approve. Set with the yes it gave; empty when a person's
// answer to a record ask gave it, or none was needed.
Release string `json:"release,omitempty"`
// Self says the node composes this module itself (ADR 0547 §2): the want names the build alone, and no composition.
// A want that is not Self names the one composition it allows (ADR 0539 clause 1), as before.
Self bool `json:"self,omitempty"`
@@ -200,7 +204,7 @@ type Want struct {
func (w Want) check() error {
if w.Removed {
if w.FP != "" || w.Composed != "" || w.Newest != "" || w.Rule != 0 || w.Backup != "" || w.Put || w.Hold || w.Self || w.Yes != "" {
if w.FP != "" || w.Composed != "" || w.Newest != "" || w.Rule != 0 || w.Backup != "" || w.Put || w.Hold || w.Self || w.Yes != "" || w.Release != "" {
return fmt.Errorf("delivery: a removed want names no fingerprint, composition, rule, backup answer or put")
}
return nil
@@ -230,6 +234,8 @@ func (w Want) check() error {
if err := checkDigest(w.Yes, "the digest a want's yes allows"); err != nil {
return err
}
} else if w.Release != "" {
return fmt.Errorf("delivery: the want of %s names the release %s and no yes it gave", w.FP, w.Release)
}
switch w.Backup {
case "", BackupAgain, BackupSkip:
+11 -5
View File
@@ -31,6 +31,12 @@ const (
// starting): no module is named, Refused says why in words, and the checker takes it as a node that did not answer
// (ComposeNoAnswer, ADR 0593 §9), deciding by this verdict and never by the words.
ComposeCandidateUnready = "unready"
// ComposeCandidateNotJudged is the verdict for a node the checker's sandbox composed and could not judge (an older
// node-engine's input that does not say how it composes, novox/hq ADR 0593 §9): no module is named, Refused says why in
// words, and the checker names the node with them and passes the check with that warning, never silent and never a
// failure of the change. The checker decides by this verdict, never by the words (ADR 0601 §4's kind: a contract is a
// field, not a phrase).
ComposeCandidateNotJudged = "not-judged"
// The states of one module's composition with the candidate: composed (Digest set), refused by the candidate
// (Words set), or held, refused without the candidate too (Words set), which counts toward no verdict.
@@ -115,13 +121,13 @@ func (a ComposeCandidateAnswer) check() error {
if a.Took < 0 {
return fmt.Errorf("delivery: the answer of %s took %d ms", a.Node, a.Took)
}
if a.Verdict == ComposeCandidateUnready && a.Refused == "" {
return fmt.Errorf("delivery: %s is not ready to compose and its answer says no why", a.Node)
if (a.Verdict == ComposeCandidateUnready || a.Verdict == ComposeCandidateNotJudged) && a.Refused == "" {
return fmt.Errorf("delivery: %s's answer is %s and says no why", a.Node, a.Verdict)
}
if a.Refused != "" {
if len(a.Modules) > 0 || a.Verdict != ComposeCandidateRefused && a.Verdict != ComposeCandidateUnready {
return fmt.Errorf("delivery: %s could not compose the candidate (%s), and its answer names modules or is neither %s nor %s",
a.Node, a.Refused, ComposeCandidateRefused, ComposeCandidateUnready)
if len(a.Modules) > 0 || a.Verdict != ComposeCandidateRefused && a.Verdict != ComposeCandidateUnready && a.Verdict != ComposeCandidateNotJudged {
return fmt.Errorf("delivery: %s could not compose the candidate (%s), and its answer names modules or is neither %s, %s nor %s",
a.Node, a.Refused, ComposeCandidateRefused, ComposeCandidateUnready, ComposeCandidateNotJudged)
}
return nil
}
+17
View File
@@ -78,6 +78,23 @@ func TestComposeCandidateAndComposedPlanRoundTripAndRefuse(t *testing.T) {
if _, err := EncodeComposeCandidateAnswer(unready); err == nil {
t.Error("an unready answer naming modules was encoded")
}
// A node the sandbox did not judge: its own verdict, with words and no module (ADR 0593 §9, ADR 0601 §4).
notJudged := ComposeCandidateAnswer{Node: "ace", Candidate: key, Verdict: ComposeCandidateNotJudged, Refused: "its node-engine sends no running state"}
if raw, err := EncodeComposeCandidateAnswer(notJudged); err != nil {
t.Errorf("a not-judged answer was not encoded: %v", err)
} else if back, err := DecodeComposeCandidateAnswer(raw); err != nil || back.Verdict != ComposeCandidateNotJudged {
t.Errorf("a not-judged answer read back as %+v, %v", back, err)
}
for name, mutate := range map[string]func(*ComposeCandidateAnswer){
"no why": func(a *ComposeCandidateAnswer) { a.Refused = "" },
"modules too": func(a *ComposeCandidateAnswer) { a.Modules = good.Modules },
} {
a := notJudged
mutate(&a)
if _, err := EncodeComposeCandidateAnswer(a); err == nil {
t.Errorf("a not-judged answer with %s was encoded", name)
}
}
if _, err := DecodeComposeCandidateAnswer([]byte(strings.Replace(string(raw), `"took"`, `"other":1,"took"`, 1))); err == nil {
t.Error("an answer with an unknown key was read")
}
+3 -2
View File
@@ -164,8 +164,9 @@ type ComposedDigest struct {
}
// The upgrade policies and their sources a composition's value names: catalogue.PolicyRoll, PolicyTogether and
// PolicyRecord, and catalogue.FromPerson, FromModule, FromBus, FromData and FromDefault, the same words (the compose
// package's test holds them equal; this package does not import the catalogue).
// PolicyRecord, and catalogue.FromPerson, FromModule, FromData and FromDefault, the same words (the compose package's
// test holds them equal; this package does not import the catalogue). "the bus" is the word an older reader derived
// for the bus module, still read from a composition it wrote; nothing derives it now.
var (
composedPolicies = []string{"roll", "together", "record"}
composedPolicyFroms = []string{"person", "module", "the bus", "irreplaceable data", "default"}
+11 -2
View File
@@ -351,6 +351,15 @@ type Have struct {
// composition that waits for a person's yes (ADR 0547 §3); empty otherwise.
Judgement Judgement `json:"judgement,omitempty"`
Waits string `json:"waits,omitempty"`
// BuildAlone says, in phase waiting-yes, that the move changes the module's build alone (novox/hq ADR 0601 §1): the
// node composed the build it runs again with today's settings and secrets and got the composition it runs, so the
// release that carries the new build answers the yes. False when a setting or a secret changed too, or the node could
// not tell: the move is then asked of a person as before.
BuildAlone bool `json:"build-alone,omitempty"`
// Kept says, in words, what the node keeps of the module as it runs because its manifest does not yet say what shared
// code no longer infers (that it is the tool runner, that its own secrets are the core's credentials; novox/hq ADR 0601
// §4): the watchman raises a condition from it, cleared once the have says none. Empty in every other have.
Kept []string `json:"kept,omitempty"`
// Sealed is, for a module the node composes itself, each value its running composition carries sealed to the node's
// key (a secret, a credential, a vault box), as applied (ADR 0547 §3, issue 1126): what the node's composer reuses
// after a restart while it opens to the vault's value, so a value sealed again moves no composition. Openable by the
@@ -476,8 +485,8 @@ func decode(data []byte, into any) error {
func (h Have) check() error {
switch h.Phase {
case PhaseBackingUp, PhaseApplying, PhaseChecking, PhaseRunning, PhaseFailed, PhaseRemoved, PhaseHeld:
if h.Judgement != "" || h.Waits != "" {
return fmt.Errorf("delivery: only a have waiting for a yes names a judgement and the composition waiting")
if h.Judgement != "" || h.Waits != "" || h.BuildAlone {
return fmt.Errorf("delivery: only a have waiting for a yes names a judgement, the composition waiting and whether it changes the build alone")
}
case PhaseWaitingYes:
if h.Judgement != JudgedRecreates && h.Judgement != JudgedMovesData && h.Judgement != JudgedCannot {
+18 -3
View File
@@ -380,10 +380,25 @@ func testPhaseHealthRefused(t *testing.T) {
} else if back, err := DecodeHave(raw); err != nil || back.Judgement != JudgedRecreates || back.Waits != waits.Waits {
t.Errorf("a have waiting for a yes came back %+v, %v", back, err)
}
// ADR 0601 §1: and whether the move changes the build alone, read back as said.
alone := waits
alone.BuildAlone = true
if raw, err := EncodeHave(alone); err != nil {
t.Errorf("a have waiting for a yes to its build alone: %v", err)
} else if back, err := DecodeHave(raw); err != nil || !back.BuildAlone {
t.Errorf("a have waiting for a yes to its build alone came back %+v, %v", back, err)
}
// ADR 0601 §4: what the node keeps as it runs, read back as said.
if raw, err := EncodeHave(Have{Phase: PhaseRunning, Health: HealthOK, Kept: []string{"runs the tool runner"}}); err != nil {
t.Errorf("a have keeping what runs: %v", err)
} else if back, err := DecodeHave(raw); err != nil || len(back.Kept) != 1 {
t.Errorf("a have keeping what runs came back %+v, %v", back, err)
}
for name, h := range map[string]Have{
"neither judged": {Phase: PhaseWaitingYes, Health: HealthOK, Judgement: JudgedNeither, Waits: waits.Waits},
"no waiting digest": {Phase: PhaseWaitingYes, Health: HealthOK, Judgement: JudgedMovesData},
"running judged": {Phase: PhaseRunning, Health: HealthOK, Judgement: JudgedRecreates},
"running build alone": {Phase: PhaseRunning, Health: HealthOK, BuildAlone: true},
"neither judged": {Phase: PhaseWaitingYes, Health: HealthOK, Judgement: JudgedNeither, Waits: waits.Waits},
"no waiting digest": {Phase: PhaseWaitingYes, Health: HealthOK, Judgement: JudgedMovesData},
"running judged": {Phase: PhaseRunning, Health: HealthOK, Judgement: JudgedRecreates},
} {
if _, err := EncodeHave(h); err == nil {
t.Errorf("%s: encoded", name)
+13 -7
View File
@@ -404,10 +404,15 @@ type Module struct {
Manifest string `json:"manifest,omitempty"`
// Inputs2 are the new-form inputs the merge gave the module, kept so the indexer computes FP2 again when a build
// module's toolchain versions appear, with no merge (novox/hq ADR 0465 clause 2). It needs Manifest.
Inputs2 *KeptInputs2 `json:"inputs2,omitempty"`
Src uint64 `json:"src,omitempty"`
Repository string `json:"repository"`
Commit string `json:"commit"`
Inputs2 *KeptInputs2 `json:"inputs2,omitempty"`
Src uint64 `json:"src,omitempty"`
// FPSrc is the source revision at which FP2 last changed (novox/hq ADR 0601 §1): Src when a merge of the module's own
// repository moved it, and 0 when a recompute with no merge moved it (a toolchain, a package, a base), since no release
// carries that build. The release that moved the source to it is the one that carries the build; 0 is not known, and no
// release is taken to carry it. Src and Commit stay where the module was last merged and say nothing of that.
FPSrc uint64 `json:"fp-src,omitempty"`
Repository string `json:"repository"`
Commit string `json:"commit"`
// Path is the module's directory in its repository, trimmed of slashes, and RootPath (".") when its manifest is the
// repository's root (novox/hq ADR 0478 clause 9, issue 1615): the build tools need it. Empty means not known: a value
// written before the indexer learned paths has none.
@@ -683,9 +688,10 @@ func (f Forge) check() error {
if f.Root == "" {
return fmt.Errorf("delivery: a merged message needs the repository's root tree")
}
if len(f.Trees) == 0 {
return fmt.Errorf("delivery: a merged message needs the tree of at least one directory holding a module.json")
}
// No trees is a merge that leaves the repository with no module.json anywhere (a documents repository, or one
// whose last module was removed): it is written all the same, so the indexer takes the merge in and drops the
// repository's modules (novox/hq issue 1126). An older writer always sent at least one tree, so none is unambiguous;
// such a message carries no manifests, locks or paths (checkInputs below: each names a directory of its trees).
for dir, tree := range f.Trees {
if tree == "" {
return fmt.Errorf("delivery: a merged message's tree of the directory %q is empty", dir)
+9 -3
View File
@@ -179,7 +179,7 @@ func TestSourceValuesRoundTrip(t *testing.T) { // 1200/AC3
t.Errorf("candidate round trip: %+v %v", got, err)
}
for _, m := range []Module{
{FP: "src1:abc", Src: 5, Repository: "novox/mesh-sdk", Commit: "cccc"},
{FP: "src1:abc", Src: 5, FPSrc: 5, Repository: "novox/mesh-sdk", Commit: "cccc"},
{None: "a build that resolves packages when built", Repository: "novox/mesh-sdk", Commit: "cccc"},
} {
data, err = EncodeModule(m)
@@ -272,9 +272,15 @@ func TestSourceValuesRefused(t *testing.T) { // 1200/AC4
_, err = EncodeForge(f)
wantErr("merged without root", err, "root")
f = sampleForges()[1]
f.Trees = nil
f.Trees, f.Manifests, f.Locks, f.Paths = nil, nil, nil, nil
if data, err := EncodeForge(f); err != nil {
t.Errorf("a merged message of a repository with no module.json is refused: %v", err)
} else if got, err := DecodeForge(data); err != nil || len(got.Trees) != 0 {
t.Errorf("a merged message with no trees comes back as %+v, %v", got, err)
}
f.Manifests = map[string]string{"modules/a": "{}"}
_, err = EncodeForge(f)
wantErr("merged without trees", err, "tree")
wantErr("merged with a manifest and no trees", err, "modules/a")
f = sampleForges()[1]
f.Trees = map[string]string{"modules/a": ""}
_, err = EncodeForge(f)
+28
View File
@@ -6,6 +6,8 @@ import (
"slices"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// The verdicts on manifests (novox/hq ADR 0578 §1, §3): the indexer judges every manifest it takes in once, with its own
@@ -37,6 +39,11 @@ const (
// raises it as a condition beside the indexer's finding.
ManifestRefusedEvent = "manifest-refused"
// ManifestAcceptedEvent is the indexer's event saying a manifest that asks for bus state was accepted
// (ManifestAccepted), carrying its bus-state entries whole, so the bus module creates them without reading the
// indexer's state (novox/hq ADR 0595 §4, design 25 § 12.2).
ManifestAcceptedEvent = "manifest-accepted"
// ManifestOutgrown is the finding of a manifest the indexer accepted before and a stricter judge finds problems in
// (novox/hq ADR 0579 §1): it stays accepted, its problems are the verdict's Warnings, and the watchman raises it as a
// warning until the manifest is mended.
@@ -204,3 +211,24 @@ type ManifestRefused struct {
func RefusedOf(digest string, v ManifestVerdict) ManifestRefused {
return ManifestRefused{V: Version, Module: v.Module, Digest: digest, Judge: v.Judge, Problems: v.Problems}
}
// ManifestAccepted is the body of the indexer's event ManifestAcceptedEvent: the module, the digest of its manifest, the
// judge, and every bus-state entry the manifest asks for, whole (novox/hq ADR 0595 §4).
type ManifestAccepted struct {
V int `json:"v"`
Module string `json:"module"`
Digest string `json:"digest"`
Judge string `json:"judge"`
BusState []catalogue.BusStateEntry `json:"bus-state"`
}
// AcceptedOf is the event body saying v, the verdict accepting the manifest m of digest.
func AcceptedOf(digest string, v ManifestVerdict, m catalogue.Manifest) ManifestAccepted {
return ManifestAccepted{V: Version, Module: v.Module, Digest: digest, Judge: v.Judge, BusState: m.BusState}
}
// EventID is the event's id, one per manifest digest and judge, so the bus takes it once:
// `manifest-accepted-<digest>-<judge>`.
func (a ManifestAccepted) EventID() string {
return ManifestAcceptedEvent + "-" + a.Digest + "-" + a.Judge
}
+28
View File
@@ -1,9 +1,13 @@
package delivery
import (
"encoding/json"
"reflect"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// A verdict on a manifest round-trips, refuses what it cannot be, and a reader takes a missing one as accepted (novox/hq
@@ -75,6 +79,30 @@ func TestAManifestVerdictIsTheContractReadersTake(t *testing.T) {
if r, _ := TakeManifestVerdict(m.VerdictOf("x")); r || len(m.VerdictOf("x").Warnings) != 1 {
t.Fatalf("an outgrown module: %+v", m.VerdictOf("x"))
}
// 1966/AC6: the accepted event of a manifest with two bus-state entries carries both whole, and its id is
// manifest-accepted-<digest>-<judge>.
entries := []catalogue.BusStateEntry{
{Name: "drafts", Storage: catalogue.StorageKeyValue, Template: catalogue.TemplateState,
Options: map[string]json.RawMessage{catalogue.OptionPastValues: json.RawMessage(`5`)}},
{Name: "records", Storage: catalogue.StorageLedger, Template: catalogue.TemplateLedger,
Types: []catalogue.LedgerType{{Name: "decision", Operations: []catalogue.LedgerOperation{{Name: "submit", Effect: catalogue.EffectSetFields}}}}},
}
digest := strings.Repeat("d", 64)
accepted := AcceptedOf(digest, ManifestVerdict{Module: "mesh-knowledge", State: VerdictAccepted, Judge: "src2:j"},
catalogue.Manifest{Module: "mesh-knowledge", BusState: entries})
raw, err := json.Marshal(accepted)
if err != nil {
t.Fatal(err)
}
var heard ManifestAccepted
if err := json.Unmarshal(raw, &heard); err != nil || heard.Module != "mesh-knowledge" || heard.Digest != digest ||
heard.V != Version || !reflect.DeepEqual(heard.BusState, entries) {
t.Fatalf("the accepted event reads as %+v (%v)", heard, err)
}
if id := heard.EventID(); id != "manifest-accepted-"+digest+"-src2:j" {
t.Fatalf("the accepted event's id is %q", id)
}
// A manifest with a field the judge does not know waits (clause 3 of the record on a release's Approve): readers hold it
// as a refused one, it round-trips through a modules value, an old value decodes unchanged, and it is not a refusal.
waits := ManifestVerdict{Module: "nats", State: VerdictManifestWaits, Unknown: []string{"core-credentials", "tool-runner"}, Judge: "j"}
+4
View File
@@ -24,6 +24,8 @@ func TestAWantNamesItsComposition(t *testing.T) {
// ADR 0547 §2, §3 (issue 1126): from the second milestone a want names the build alone, and the yes a person gave.
{FP: fp2of('a'), Self: true},
{FP: fp2of('a'), Self: true, Yes: hashOf('3')},
// ADR 0601 §1: the yes the release's Approve gave names the release.
{FP: fp2of('a'), Self: true, Yes: hashOf('3'), Release: "feat/l3b-bus"},
} {
raw, err := EncodeWant(w)
if err != nil {
@@ -43,6 +45,8 @@ func TestAWantNamesItsComposition(t *testing.T) {
{Want{FP: fp2of('a'), Self: true, Composed: hashOf('1')}, "names a composition"},
{Want{FP: fp2of('a'), Self: true, Yes: "sha256:12"}, "a want's yes allows"},
{Want{Removed: true, Self: true}, "removed want names no"},
{Want{Removed: true, Release: "feat/l3b-bus"}, "removed want names no"},
{Want{FP: fp2of('a'), Self: true, Release: "feat/l3b-bus"}, "and no yes it gave"},
{Want{FP: fp2of('a'), Composed: "sha256:12"}, "not sha256"},
{Want{Removed: true, Composed: hashOf('1')}, "removed want names no"},
{Want{FP: fp2of('a'), Composed: hashOf('1'), Backup: "later"}, `backup answer "later"`},
+32
View File
@@ -7,6 +7,8 @@ package membership
import (
"encoding/json"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// AssignmentsStream holds every assignment's membership, the newest per subject.
@@ -57,6 +59,36 @@ type Membership struct {
Hears []HeardIssued `json:"hears,omitempty"`
// Waits are the event families a session may wait on through the mesh MCP server (novox/hq ADR 0367 §2).
Waits []WaitIssued `json:"waits,omitempty"`
// Bindings are the module's bus state as the bus module created it, one per entry its manifest asks for in
// `requires` (novox/hq ADR 0595 §1, §7): written by the grants part when it hears the bus module's `created`.
Bindings []BusBinding `json:"bindings,omitempty"`
}
// BusBinding is one bus-state entry's names on the bus, by the local name its module asks for it by (novox/hq ADR 0595
// §1, design 25 § 12.2): the module names only locally, and reaches its own entries, and only those, through these.
// Which names are set follows the storage type: a stream's Stream and Subject, a key-value bucket's Bucket, an object
// store's ObjectStore, and a ledger's History, State, Indexes and Snapshot.
type BusBinding struct {
Local string `json:"local"`
// Storage and Template are the entry's storage type and template (catalogue.Storage…, catalogue.Template…).
Storage string `json:"storage"`
Template string `json:"template,omitempty"`
// Stream and Subject are a stream's name and the subject its messages are on; Subject is also a ledger's history's.
Stream string `json:"stream,omitempty"`
Subject string `json:"subject,omitempty"`
// Bucket is a key-value bucket's name.
Bucket string `json:"bucket,omitempty"`
// ObjectStore is an object store's name.
ObjectStore string `json:"object-store,omitempty"`
// Ledger, for the storage type ledger: its history stream (one subject per instance), its state bucket, each
// declared index's bucket by index name, and the subject of the instances' snapshots (ADR 0597 §1, §4, §8).
History string `json:"history,omitempty"`
State string `json:"state,omitempty"`
Indexes map[string]string `json:"indexes,omitempty"`
Snapshot string `json:"snapshot,omitempty"`
// Types are a ledger's types, copied by the grants part from the entry the manifest-accepted event carries, so the
// tool runner checks an operation against its type without reading another part's state (ADR 0596, ADR 0597 §1).
Types []catalogue.LedgerType `json:"types,omitempty"`
}
// Served is one address a tool is answered on.
+31 -18
View File
@@ -4,33 +4,26 @@ import (
"encoding/json"
"reflect"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// A credential is read out of a bus address, and the bare address carries no credential and a scheme.
func TestACredentialIsReadOutOfABusAddress(t *testing.T) {
for _, c := range []struct{ in, user, password, bare string }{
{"nats://controller:secret@127.0.0.1:4222", "controller", "secret", "nats://127.0.0.1:4222"},
{"controller:secret@127.0.0.1:4222", "controller", "secret", "127.0.0.1:4222"},
{"nats://127.0.0.1:4222", "", "", "nats://127.0.0.1:4222"},
{"127.0.0.1:4222", "", "", "127.0.0.1:4222"},
for _, c := range []struct{ in, user, password, bare, schemed string }{
{"nats://controller:secret@127.0.0.1:4222", "controller", "secret", "nats://127.0.0.1:4222", "nats://127.0.0.1:4222"},
{"controller:secret@127.0.0.1:4222", "controller", "secret", "127.0.0.1:4222", "nats://127.0.0.1:4222"},
{"nats://127.0.0.1:4222", "", "", "nats://127.0.0.1:4222", "nats://127.0.0.1:4222"},
{"127.0.0.1:4222", "", "", "127.0.0.1:4222", "nats://127.0.0.1:4222"},
// A password containing an at-sign: split on the last one.
{"nats://controller:a@b@127.0.0.1:4222", "controller", "a@b", "nats://127.0.0.1:4222"},
{"nats://controller:a@b@127.0.0.1:4222", "controller", "a@b", "nats://127.0.0.1:4222", "nats://127.0.0.1:4222"},
} {
user, password, bare := CredentialIn(c.in)
if user != c.user || password != c.password || bare != c.bare {
t.Errorf("%q read as %q/%q at %q; wanted %q/%q at %q", c.in, user, password, bare, c.user, c.password, c.bare)
}
}
}
func TestABareAddressCarriesNoCredentialAndAScheme(t *testing.T) {
for in, want := range map[string]string{
"nats://controller:secret@10.0.0.1:4222": "nats://10.0.0.1:4222",
"controller:secret@10.0.0.1:4222": "nats://10.0.0.1:4222",
"nats://10.0.0.1:4222": "nats://10.0.0.1:4222",
"10.0.0.1:4222": "nats://10.0.0.1:4222",
} {
if got := BareAddress(in); got != want {
t.Errorf("BareAddress(%q) = %q, want %q", in, got, want)
if got := BareAddress(c.in); got != c.schemed {
t.Errorf("BareAddress(%q) = %q, want %q", c.in, got, c.schemed)
}
}
}
@@ -66,6 +59,26 @@ func TestAMembershipRoundTripsAsIssued(t *testing.T) {
if !reflect.DeepEqual(in, out) {
t.Fatalf("round trip changed it:\n%+v\n%+v", in, out)
}
// 1966/AC4: a membership with a ledger binding, its types among it, round-trips; one without bindings encodes as before, byte for byte.
bound := in
bound.Bindings = []BusBinding{{Local: "records", Storage: "ledger", Template: "ledger", History: "forge_records",
Subject: "forge_records.>", State: "forge_records_state", Indexes: map[string]string{"search": "forge_records_search"},
Snapshot: "forge_records_snapshot.>",
Types: []catalogue.LedgerType{{Name: "decision", Schema: map[string]catalogue.LedgerField{"title": {Type: "text"}},
Operations: []catalogue.LedgerOperation{{Name: "propose", Effect: catalogue.EffectSetFields, Fields: []string{"title"}}}}}}}
boundRaw, err := json.Marshal(bound)
if err != nil {
t.Fatal(err)
}
var boundBack Membership
if err := json.Unmarshal(boundRaw, &boundBack); err != nil || !reflect.DeepEqual(bound, boundBack) {
t.Fatalf("a membership with a ledger binding read back as %+v (%v)", boundBack, err)
}
before, _ := json.Marshal(Membership{Node: "ace", Module: "forge", Tools: "t", Emits: "e", Serves: []Served{{Subject: "s"}},
State: []StateIssued{{Name: "repos", Bucket: "forge_repos", Writes: true}}})
if string(before) != `{"node":"ace","module":"forge","serves":[{"subject":"s"}],"emits":"e","tools":"t","state":[{"name":"repos","bucket":"forge_repos","writes":true}]}` {
t.Fatalf("a membership without bindings encodes otherwise than before: %s", before)
}
var wire map[string]json.RawMessage
_ = json.Unmarshal(raw, &wire)
for _, k := range []string{"receives", "mesh", "seat-traffic"} {
+72 -1
View File
@@ -1,14 +1,19 @@
package stdio
import (
"encoding/json"
"go/ast"
"go/parser"
"go/token"
"io/fs"
"path/filepath"
"reflect"
"slices"
"strconv"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// meshSDKPackages is every package mesh-sdk holds (novox/hq ADR 0593 §1): what every module shares, the value shapes
@@ -16,7 +21,7 @@ import (
// Logic only some modules need is its owner module's package. A package added here is a decision to say in the pull
// request: does every module share it, or is it a shape more than one part reads?
var meshSDKPackages = []string{
".", "announce", "announcetest", "asker", "asks", "buildstep", "buscompose", "busretry", "calladdress", "catalogue",
".", "announce", "announcetest", "asker", "asks", "buildstep", "buscalls", "buscompose", "busretry", "calladdress", "catalogue",
"composer", "declaration", "delivery", "forgekey", "handacts", "health", "internal/beside", "kvlimit", "membership",
"objectlink", "outward", "overlay", "pernode", "process", "provider", "provider/harness", "provisions",
"sdkversion", "seal", "testbus",
@@ -56,6 +61,16 @@ func TestMeshSDKHoldsOnlyWhatEveryModuleShares(t *testing.T) {
if err != nil {
return err
}
// 1966/AC5: the calls package imports nothing of mesh-sdk but the catalogue's types, so a change here rebuilds
// only its importers (ADR 0595 §9).
if dir == "buscalls" {
for _, imp := range file.Imports {
p, _ := strconv.Unquote(imp.Path.Value)
if strings.HasPrefix(p, sdkModule) && p != sdkModule+"/catalogue" {
t.Errorf("%s imports %s: the calls package imports nothing of mesh-sdk but catalogue (ADR 0595 §9)", path, p)
}
}
}
for _, decl := range file.Decls {
fn, ok := decl.(*ast.FuncDecl)
if !ok {
@@ -90,4 +105,60 @@ func TestMeshSDKHoldsOnlyWhatEveryModuleShares(t *testing.T) {
t.Errorf("mesh-sdk's list keeps %s, which is gone: take it off the list", want)
}
}
// 1966/AC5: the call an apply names is the id the tool runner hands under `_meta`'s mesh/call.
if buscalls.MetaCall != "mesh/call" {
t.Errorf("the tool call's id is handed under %q, not mesh/call", buscalls.MetaCall)
}
// 1966/AC5: an index's key is its fields' values, then the instance's path, joined by "."; its entry names the
// instance and the sequence.
for _, c := range []struct {
fields []string
instance string
want string
}{
{[]string{"17950", "volvo"}, "listing/a17", "17950.volvo.listing.a17"},
// A value holding "." and one that does not meet in no key.
{[]string{"a.b"}, "c", "a=2Eb.c"},
{[]string{"a"}, "b/c", "a.b.c"},
// A title with spaces is a token the bus takes; "=" itself is encoded.
{[]string{"How it is checked", "x=y"}, "decision/0290", "How=20it=20is=20checked.x=3Dy.decision.0290"},
} {
k := buscalls.IndexKey(c.fields, c.instance)
if k != c.want {
t.Errorf("the index key of %q and %q is %q, want %q", c.fields, c.instance, k, c.want)
}
for _, token := range strings.Split(k, ".") {
if token == "" || strings.ContainsAny(token, " *>\t") {
t.Errorf("the index key %q holds a token the bus refuses: %q", k, token)
}
}
}
if buscalls.IndexKey([]string{"a.b"}, "c") == buscalls.IndexKey([]string{"a"}, "b/c") {
t.Error("two different index keys meet")
}
entry, _ := json.Marshal(buscalls.IndexEntry{Instance: "listing/a17", Sequence: 3})
if string(entry) != `{"instance":"listing/a17","sequence":3}` {
t.Errorf("an index entry is %s", entry)
}
// 1966/AC5: an apply naming the tool call it serves, and its answer with the state behind, round-trip.
for _, c := range []struct {
name string
in, back any
}{
{"apply", &buscalls.ApplyRequest{Ledger: "records", Instance: "decision/0290", Operation: "submit",
Body: json.RawMessage(`{"why":"x"}`), Call: "call-17"}, &buscalls.ApplyRequest{}},
{"applied", &buscalls.Applied{Taken: true, Sequence: 5, StateBehind: true}, &buscalls.Applied{}},
} {
raw, err := json.Marshal(c.in)
if err != nil {
t.Fatal(err)
}
if err := json.Unmarshal(raw, c.back); err != nil || !reflect.DeepEqual(c.in, c.back) {
t.Errorf("%s read back as %+v from %s (%v)", c.name, c.back, raw, err)
}
}
}
// sdkModule is mesh-sdk's Go module path.
const sdkModule = "git.novox.be/novox/mesh-sdk/go"
+9 -6
View File
@@ -482,7 +482,7 @@ func (h *Harness) retirementBody(change string, consumers []map[string]any, extr
}
// RetirementTools are the four tools every provider serves for retirement, the same names in every
// module: the controller's `retire` and `cleanup` verbs ask them on the provider's machine.
// module: a caller asks them on the provider's machine, with the operator's answer named in why and by.
func RetirementTools(h *Harness) []stdio.Tool {
if h == nil {
return nil
@@ -493,7 +493,7 @@ func RetirementTools(h *Harness) []stdio.Tool {
who := map[string]any{
"why": map[string]any{"type": "string", "description": "why — required, kept in the hand-act log"},
"by": map[string]any{"type": "string", "description": "who decided"},
"via": map[string]any{"type": "string", "description": "mesh-controller when asked through its verbs"},
"via": map[string]any{"type": "string", "description": "the part or session that asks, when not the caller itself"},
}
withSet := map[string]any{"consumers": map[string]any{"type": "array", "items": map[string]any{"type": "string"},
"description": "the set, exactly as provisioner_retirement names it"}}
@@ -518,7 +518,8 @@ func RetirementTools(h *Harness) []stdio.Tool {
}},
{Name: "provisioner_retire_approve",
Description: "Retire exactly the set waiting for a person (or the set rejected earlier): access disabled, data " +
"kept and marked to delete. Use the controller's `retire approve`, which records the hand act.",
"kept and marked to delete. It asks nobody: the operator's answer on the operator channel to the provisioner's " +
"ask (novox/hq ADR 0570 §5) is named in why and by, which the hand-act log keeps.",
Input: withSet,
Run: func(args map[string]any) (any, error) {
ctx, cancel := ask()
@@ -530,8 +531,9 @@ func RetirementTools(h *Harness) []stdio.Tool {
return map[string]any{"retired": orEmptyList(done)}, nil
}},
{Name: "provisioner_retire_reject",
Description: "Keep the set waiting for a person active. Use the controller's `retire reject`.",
Input: withSet,
Description: "Keep the set waiting for a person active. It asks nobody: the operator's answer on the operator channel " +
"(novox/hq ADR 0570 §5) is named in why and by, which the hand-act log keeps.",
Input: withSet,
Run: func(args map[string]any) (any, error) {
kept, err := h.Reject(strs(args["consumers"]), argString(args, "why"), argString(args, "by"), argString(args, "via"))
if err != nil {
@@ -541,7 +543,8 @@ func RetirementTools(h *Harness) []stdio.Tool {
}},
{Name: "provisioner_delete",
Description: "Delete one RETIRED consumer — its login and its data — for good. Refused for anything active or " +
"asked for. Use the controller's `cleanup delete`, which records the hand act.",
"asked for. It asks nobody: ask the operator on the operator channel first, and name that answer in why and " +
"by, which the hand-act log keeps (novox/hq ADR 0604 §8).",
Input: withOne,
Run: func(args map[string]any) (any, error) {
consumer := argString(args, "consumer")
+4
View File
@@ -482,6 +482,10 @@ func TestTheToolsAnswer(t *testing.T) {
tools := map[string]func(map[string]any) (any, error){}
for _, tool := range RetirementTools(w.h) {
tools[tool.Name] = tool.Run
// L3: the delete, approve and reject name no removed controller verb, and say where the operator answers (ADR 0604 §8).
if tool.Name != "provisioner_retirement" && (strings.Contains(tool.Description, "controller") || !strings.Contains(tool.Description, "operator channel")) {
t.Errorf("%s says %q", tool.Name, tool.Description)
}
}
if len(tools) != 4 {
t.Fatalf("%d tools", len(tools))