novox/mesh-sdk
9
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Revert G's mesh-sdk merge (#189): G merged half, and today's validator refuses G's delivery parts
mesh/delivery merged: source.main moved to novox/mesh-sdk@27da949140c6
Details
mesh/check being carried to 8afc3ebdf371 on 93e0a7b50d90 | from 27da949140c6

Browse Source 
The group feat/47-G-join11 merged 3 of 17 members on 2026-11-13; G's mesh-sdk alone on main breaks 25 trunk
modules. The operator chose to put the order right: a reader release first, then G whole. This restores main
to 8fbba810 exactly.

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
This commit is contained in:
jschoubben
2026-11-13 11:12:48 +01:00
parent b9915abf51
commit 8afc3ebdf3
212 changed files with 2436 additions and 32088 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
-11
View File
@@ -115,17 +115,6 @@ runtime's (`node-tools/internal/launch`, mesh-tools).
values and then every change. Through the runtime that launched the bundle, which refuses what the values and then every change. Through the runtime that launched the bundle, which refuses what the
module did not declare. Current values, never history and never a secret (novox/hq ADR 0201). module did not declare. Current values, never history and never a secret (novox/hq ADR 0201).
### `runtime`, `callers` and `views` — holding a seat from TypeScript
The TypeScript side of what the Go side's `stdio` and `views` give a seat's holder, with the same rules (design 47
*The forge module's tools come from mesh-sdk's TypeScript side*): `runtime.seatPublish` says an event or submits an
accept on a seat (`mesh/seat.publish`) and `runtime.config` reads a config value; `callers.callerOf(args)` is the
caller the runtime proved in the call's `_meta`, never an argument, and `callers.refusal` judges it against a
holder's list per verb (`parseVerbCallers`, `configuredVerbCallers`: `*` any proved caller, `operator` any person);
`views.Follower` keeps a first view of an owner's state from its pages and its events, asking again only on a gap,
and `views.statusTools` serves its read and ask-again tools (`<module>_first_views`, `<module>_first_view_again`)
over the reader's state `first-views`. `waitFor(view)` and `holds(view)` say the revision view a part waits for and
holds; past five minutes `raiseWaits` is told and `viewWaitsFinding` says the part's `view-waits` finding. Checked by `test/stdio.test.ts` and `test/sdk.test.ts`.
### `seats` and `channel` — publishing on a seat, the channel's benches (Go) ### `seats` and `channel` — publishing on a seat, the channel's benches (Go)
A seat may name its caller or its kind (novox/hq ADR 0259): `stdio.SeatPublish` submits an accept or says A seat may name its caller or its kind (novox/hq ADR 0259): `stdio.SeatPublish` submits an accept or says
an event under the module's own name or kind, `stdio.SeatTake` takes a held seat's work, `stdio.SeatProve` an event under the module's own name or kind, `stdio.SeatTake` takes a held seat's work, `stdio.SeatProve`
go/approval/act.go
-296
View File
@@ -1,296 +0,0 @@
package approval
import (
"bytes"
"encoding/json"
"fmt"
"math"
"sort"
"strconv"
"strings"
"unicode/utf8"
"git.novox.be/novox/mesh-sdk/go/operatorask"
"git.novox.be/novox/mesh-sdk/go/outward"
)
// Call is one call the runner serves, as the gate judges it: a module's tool, or a seat verb its module serves.
type Call struct {
// Address is the address as called: `<module>.<tool>` or `<seat>.<verb>`.
Address string
// Module is the module whose bundle serves it; Seat the seat of a verb, empty for a tool; Name the tool or the verb.
Module, Seat, Name string
// Fingerprint is the build the bundle serves, from its composed environment (§5).
Fingerprint string
// Caller is the caller the bus proved.
Caller string
// Args are the call's arguments, a JSON object.
Args json.RawMessage
// Description is the tool's or verb's own description, whose first line the ask explains it by.
Description string
// Own says it is one of the runner's own tools served under the module's name (its ledger repairs), not its bundle's.
Own bool
}
// MostArguments is the most a marked call's arguments may hold: more is refused, asking nothing (§9).
const MostArguments = 16 * 1024
// arguments are a call's arguments by name; a body that is not an object is refused.
func arguments(raw json.RawMessage) (map[string]json.RawMessage, error) {
args := map[string]json.RawMessage{}
if len(bytes.TrimSpace(raw)) == 0 || string(bytes.TrimSpace(raw)) == "null" {
return args, nil
}
if err := json.Unmarshal(raw, &args); err != nil {
return nil, fmt.Errorf("its arguments are not a JSON object: %v", err)
}
return args, nil
}
// refusedUnasked says why a marked call is refused at once, asking nothing (§9): its arguments hold a secret's shape, or are
// more than MostArguments.
func refusedUnasked(raw json.RawMessage) string {
if len(raw) > MostArguments {
return fmt.Sprintf("its arguments are %d bytes, more than the %d a call that needs the operator's approval may carry", len(raw), MostArguments)
}
if r, ok := outward.CheckSecret(string(raw)); !ok {
return fmt.Sprintf("its arguments hold %s, which no ask may carry", r.What)
}
return ""
}
// ActOf is the act a marked call binds (§8): `address`, `node` (none given for a process holder, lead answer A-255), `module` and `tool` or `seat` and `verb`, `build`, and
// `arg.<name>` for each argument given, a string as itself and any other value as its JSON in RFC 8785's canonical form.
func ActOf(c Call, node string) (operatorask.Act, error) {
act := operatorask.Act{"address": c.Address, "build": c.Fingerprint}
if node != "" { // a process holder's act names none (lead answer A-255)
act["node"] = node
}
if c.Seat == "" {
act["module"], act["tool"] = c.Module, c.Name
} else {
act["seat"], act["verb"] = c.Seat, c.Name
}
args, err := arguments(c.Args)
if err != nil {
return nil, err
}
for name, raw := range args {
var s string
if json.Unmarshal(raw, &s) == nil {
act["arg."+name] = s
continue
}
canon, err := Canonical(raw)
if err != nil {
return nil, fmt.Errorf("the argument %s cannot be put in canonical form: %v", name, err)
}
act["arg."+name] = canon
}
return act, nil
}
// Canonical is a JSON value in RFC 8785's canonical form: no white space, an object's members sorted by their names' UTF-16
// code units, a number as ECMAScript writes a double, a string with only the escapes the RFC names.
func Canonical(raw json.RawMessage) (string, error) {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var v any
if err := dec.Decode(&v); err != nil {
return "", err
}
var b strings.Builder
if err := canonicalValue(&b, v); err != nil {
return "", err
}
return b.String(), nil
}
func canonicalValue(b *strings.Builder, v any) error {
switch x := v.(type) {
case nil:
b.WriteString("null")
case bool:
b.WriteString(strconv.FormatBool(x))
case json.Number:
f, err := strconv.ParseFloat(string(x), 64)
if err != nil || math.IsInf(f, 0) || math.IsNaN(f) {
return fmt.Errorf("the number %s is not a double", x)
}
b.WriteString(es6Number(f))
case string:
canonicalString(b, x)
case []any:
b.WriteByte('[')
for i, e := range x {
if i > 0 {
b.WriteByte(',')
}
if err := canonicalValue(b, e); err != nil {
return err
}
}
b.WriteByte(']')
case map[string]any:
names := make([]string, 0, len(x))
for k := range x {
names = append(names, k)
}
sort.Slice(names, func(i, j int) bool { return utf16Less(names[i], names[j]) })
b.WriteByte('{')
for i, k := range names {
if i > 0 {
b.WriteByte(',')
}
canonicalString(b, k)
b.WriteByte(':')
if err := canonicalValue(b, x[k]); err != nil {
return err
}
}
b.WriteByte('}')
default:
return fmt.Errorf("a value of type %T", v)
}
return nil
}
// es6Number is a double as ECMAScript's Number.prototype.toString writes it (RFC 8785 §3.2.2.3).
func es6Number(f float64) string {
if f == 0 {
return "0"
}
abs := math.Abs(f)
if abs >= 1e-6 && abs < 1e21 {
return strconv.FormatFloat(f, 'f', -1, 64)
}
s := strconv.FormatFloat(f, 'e', -1, 64) // 1e+21, 1.5e-07
mant, exp, _ := strings.Cut(s, "e")
sign, digits := exp[:1], strings.TrimLeft(exp[1:], "0")
return mant + "e" + sign + digits
}
func canonicalString(b *strings.Builder, s string) {
b.WriteByte('"')
for _, r := range s {
switch {
case r == '"':
b.WriteString(`\"`)
case r == '\\':
b.WriteString(`\\`)
case r == '\b':
b.WriteString(`\b`)
case r == '\f':
b.WriteString(`\f`)
case r == '\n':
b.WriteString(`\n`)
case r == '\r':
b.WriteString(`\r`)
case r == '\t':
b.WriteString(`\t`)
case r < 0x20:
fmt.Fprintf(b, `\u%04x`, r)
default:
b.WriteRune(r)
}
}
b.WriteByte('"')
}
func utf16Less(a, b string) bool {
ua, ub := utf16Of(a), utf16Of(b)
for i := 0; i < len(ua) && i < len(ub); i++ {
if ua[i] != ub[i] {
return ua[i] < ub[i]
}
}
return len(ua) < len(ub)
}
func utf16Of(s string) []uint16 {
out := make([]uint16, 0, utf8.RuneCountInString(s))
for _, r := range s {
if r >= 0x10000 {
r -= 0x10000
out = append(out, uint16(0xD800+(r>>10)), uint16(0xDC00+(r&0x3FF)))
} else {
out = append(out, uint16(r))
}
}
return out
}
// Fingerprint is how an act is shown: its digest's first 24 hexadecimal digits in groups of four (§10).
func Fingerprint(digest string) string {
hex := strings.TrimPrefix(digest, "sha256:")
if len(hex) > 24 {
hex = hex[:24]
}
var groups []string
for len(hex) > 4 {
groups, hex = append(groups, hex[:4]), hex[4:]
}
return strings.Join(append(groups, hex), " ")
}
// The bounds of the arguments shown in an ask (§11).
const (
ShownBytes, ShownLines = 2000, 40
PartBytes, PartLines = 1200, 20
)
// shown is a call's arguments as an ask shows them (§11), one per line as "name: value", and whether all were shown whole:
// an argument whole where outward.Check passes it, else its shape, ‹path›, ‹address› or ‹withheld›; the whole when it is at
// most ShownBytes and ShownLines, else its first PartLines lines and PartBytes bytes, marked NOT SHOWN IN FULL.
func shown(raw json.RawMessage) (string, bool) {
args, _ := arguments(raw)
names := make([]string, 0, len(args))
for n := range args {
names = append(names, n)
}
sort.Strings(names)
whole := true
var lines []string
for _, n := range names {
var value string
if json.Unmarshal(args[n], &value) != nil {
value = string(args[n])
}
if r, ok := outward.Check(value); !ok {
value, whole = shapeOf(r), false
}
lines = append(lines, n+": "+value)
}
text := strings.Join(lines, "\n")
if all := strings.Split(text, "\n"); len(text) > ShownBytes || len(all) > ShownLines {
if len(all) > PartLines {
all = all[:PartLines]
}
text = strings.Join(all, "\n")
if len(text) > PartBytes {
text = cutBytes(text, PartBytes)
}
text += "\nNOT SHOWN IN FULL"
whole = false
}
return text, whole
}
// shapeOf is what an argument outward.Check refuses is shown as.
func shapeOf(r outward.Refusal) string {
switch r.Class {
case "path":
return "‹path›"
case "address":
return "‹address›"
}
return "‹withheld›"
}
// cutBytes is s at most n bytes, cut at a character's end.
func cutBytes(s string, n int) string {
for n > 0 && !utf8.RuneStart(s[n]) {
n--
}
return s[:n]
}
go/approval/approval_test.go
-80
View File
@@ -1,80 +0,0 @@
package approval
import (
"encoding/json"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/operatorask"
)
// call is a fixture call of the marked tool wipe.
func call(name, args string) Call {
return Call{Address: "shell." + name, Module: "shell", Name: name, Fingerprint: "src2:1", Caller: "agent", Args: json.RawMessage(args),
Description: "Wipes a scratch directory.\nMore words."}
}
// Design 46 7-approval-mark §8, §10, §11: the act's fields give each call its digest, and the ask shows a call's arguments
// in their shape and its headline cut (the gate's other cases run on the tool runner's fixture mesh, node-tools
// internal/approval).
func TestTheActAndItsAsk(t *testing.T) {
base := call("wipe", `{"dir":"scratch","n":1}`)
digest := func(c Call) string {
act, err := ActOf(c, "n1")
if err != nil {
t.Fatal(err)
}
d, _ := operatorask.ActDigest(act)
return d
}
act, _ := ActOf(base, "n1")
want := operatorask.Act{"address": "shell.wipe", "node": "n1", "module": "shell", "tool": "wipe", "build": "src2:1",
"arg.dir": "scratch", "arg.n": "1"}
if len(act) != len(want) {
t.Fatalf("act %v", act)
}
for k, v := range want {
if act[k] != v {
t.Errorf("act[%s] = %q, want %q", k, act[k], v)
}
}
seen := map[string]string{digest(base): "the call"}
for name, c := range map[string]Call{
"an omitted argument": call("wipe", `{"dir":"scratch"}`),
"another number": call("wipe", `{"dir":"scratch","n":2}`),
"an object": call("wipe", `{"dir":{"name":"scratch"},"n":1}`),
"a changed build": func() Call { c := base; c.Fingerprint = "src2:2"; return c }(),
"another node's seat": {Address: "node-login-shell.execute", Module: "shell", Seat: "node-login-shell", Name: "execute", Fingerprint: "src2:1"},
} {
d := digest(c)
if other, dup := seen[d]; dup {
t.Errorf("%s gives the digest of %s", name, other)
}
seen[d] = name
}
if got, _ := Canonical(json.RawMessage(`{"b":[1.0,2.5e-7,1e21],"a":"é<"}`)); got != `{"a":"é<","b":[1,2.5e-7,1e+21]}` {
t.Errorf("canonical form %s", got)
}
// §11: a path, an address and a long argument are shown in their shape and cut.
text, whole := shown(json.RawMessage(`{"dir":"/home/someone/scratch","host":"10.0.0.7","ok":"yes"}`))
if whole || text != "dir: ‹path›\nhost: ‹address›\nok: yes" {
t.Errorf("shown %q whole %v", text, whole)
}
long, _ := json.Marshal(map[string]string{"text": strings.Repeat("word ", 500)})
if text, whole := shown(long); whole || !strings.HasSuffix(text, "NOT SHOWN IN FULL") || len(text) > PartBytes+len("\nNOT SHOWN IN FULL") {
t.Errorf("a long argument shown whole %v, %d bytes", whole, len(text))
}
// §10: the headline, cut with … to 60 characters, ending Part shown.
for _, tc := range []struct{ name, node, want string }{
{"wipe", "n1", "Run wipe on n1?"},
{strings.Repeat("x", 50), "n1", "Run " + strings.Repeat("x", 50) + "?"},
{strings.Repeat("y", 80), "n1", "Run " + strings.Repeat("y", 54) + "…?"},
} {
if got := headlineOf("Run", tc.name, tc.node, true); got != tc.want || len([]rune(got)) > 60 {
t.Errorf("headline %q, want %q", got, tc.want)
}
}
if got := headlineOf("Run", "wipe", "n1", false); got != "Run wipe on n1? Part shown" {
t.Errorf("headline %q", got)
}
}
go/approval/ask.go
-239
View File
@@ -1,239 +0,0 @@
package approval
import (
"crypto/rand"
"encoding/hex"
"encoding/json"
"fmt"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/operatorask"
)
// StateName is the runner's own state its asks and claims are kept in, per machine (§13).
const StateName = "approvals"
// Store is the runner's state `approvals` (§13): a key's value, a create that refuses a key that exists and an update by
// compare-and-set, each failing with stdio.ErrChanged when another write came first; a put, and its keys.
type Store interface {
Get(key string) (*stdio.StateEntry, error)
Create(key string, value any) (uint64, error)
Update(key string, value any, revision uint64) (uint64, error)
Put(key string, value any) (uint64, error)
Keys() ([]string, error)
}
// The bounds of an ask (§10, §12, §13, §14, §17).
const (
AskLasts = 60 * time.Minute
RecordWithin = 5 * time.Second
RunWithin = 30 * time.Minute
DeclinedFor = 10 * time.Minute
MostKeptValue = 256 * 1024
OnExpiry = "nothing runs; call again to ask anew"
)
// The options every ask offers (§10), both at level approve, both binding the act's digest.
const (
OptionApprove = "approve"
OptionDecline = "decline"
)
// actKey is the key naming the newest ask of an act (§13), its digest escaped as a key's name is, since the bus holds no `:` in a key (lead answer A-209).
func (g *Gate) actKey(digest string) string { return g.prefix() + "act." + keys.EscapeName(digest) }
// prefix is what the gate's keys begin with: the tool runner's machine, `<node>.`, since its state is shared by every
// machine's runner (§13); none for a module's process, whose state approvals is its module's own (design 47 G §677).
func (g *Gate) prefix() string {
if g.d.Holding {
return ""
}
return g.d.Node + "."
}
// askKey is the key the client keeps an ask under (§13).
func (g *Gate) askKey(id string) string { return g.prefix() + "ask." + id }
// newID is an ask's id: 128 random bits.
func newID() string {
var b [16]byte
_, _ = rand.Read(b[:])
return hex.EncodeToString(b[:])
}
// askFor is the ask a marked call raises (§10, §11).
func (g *Gate) askFor(c Call, digest, id string) Ask {
args, whole := shown(c.Args)
name := c.Name
if c.Seat != "" {
name = c.Seat + "." + c.Name
}
headline := headlineOf("Run", name, g.d.Node, whole)
first, _, _ := strings.Cut(strings.TrimSpace(c.Description), "\n")
if first == "" {
first = name
}
how := "called through the mesh's tools, which cannot tell who called."
if g.d.Holding {
how = "called by " + c.Caller + "."
}
lines := []string{first, "On " + g.d.Node + ", " + how}
if args != "" {
lines = append(lines, args)
}
if !whole {
read := "mesh_approvals on " + g.d.Node
if g.d.Holding {
read = c.Seat + ".approvals"
}
lines = append(lines, "The whole is read with "+read+", whose fingerprint must be the one on the phone.")
}
lines = append(lines, "Fingerprint "+Fingerprint(digest))
return Ask{ID: id, About: digest, Headline: headline,
Explanation: strings.Join(lines, "\n"), Expires: formatTime(g.d.Now().Add(AskLasts)), OnExpiry: OnExpiry,
Options: []Option{
{ID: OptionApprove, Label: "Approve", Does: "the same call, made again within 30 minutes, runs once", Binds: digest},
{ID: OptionDecline, Label: "Decline", Does: "nothing runs; the same call is not asked again for 10 minutes", Binds: digest}}}
}
// headlineOf is "<verb> <names> on <node>?", "<verb> <names>?" when that is longer than the router allows, and the names cut
// with "…" when that too is longer; ending "Part shown" whenever an argument is not shown whole (§10, Q-HL). verb is "Run"
// for a call and "Wait on" for a wait (design 34 every-event §35).
func headlineOf(verb, name, node string, whole bool) string {
suffix := ""
if !whole {
suffix = " Part shown"
}
max := HeadlineLength
fits := func(s string) bool { return len([]rune(s)) <= max }
if h := verb + " " + name + " on " + node + "?" + suffix; fits(h) {
return h
}
if h := verb + " " + name + "?" + suffix; fits(h) {
return h
}
room := max - len([]rune(verb+" …?"+suffix))
r := []rune(name)
if room < 1 {
room = 1
}
if len(r) > room {
r = r[:room]
}
return verb + " " + string(r) + "…?" + suffix
}
// firstCall is a marked call whose act has no ask open: refused at once when its arguments may not be asked (§9), else
// asked (§10, §12).
func (g *Gate) firstCall(c Call) (any, error) {
if why := refusedUnasked(c.Args); why != "" {
return nil, g.refused(c, "", why)
}
act, err := ActOf(c, g.d.Node)
if err != nil {
return nil, g.refused(c, "", err.Error())
}
digest, err := operatorask.ActDigest(act)
if err != nil {
return nil, g.refused(c, "", err.Error())
}
return nil, g.raise(c, act, digest, "")
}
// raise raises the ask of a marked call whose act has none open (§10): kept in the runner's state with the act and the whole
// arguments, published under the module's name, its act's key naming it; then the 5 seconds for the router's record and
// the first answer (§12), always an error, never a result.
func (g *Gate) raise(c Call, act operatorask.Act, digest, markedSince string) error {
id := newID()
if err := g.keepAsk(c, act, digest, g.askFor(c, digest, id)); err != nil {
return err
}
return g.firstAnswer(c, id, digest, markedSince)
}
// keepAsk raises an ask and keeps it in the runner's state with the act and the whole arguments, its act's key naming it
// (§10, §13); refused, as an error, when it cannot be.
func (g *Gate) keepAsk(c Call, act operatorask.Act, digest string, a Ask) error {
if size := keptSize(a, act, c.Args); size > MostKeptValue {
return g.refused(c, digest, fmt.Sprintf("its ask would keep %d bytes, more than the %d a value of the state %s holds", size, MostKeptValue, StateName))
}
if err := g.d.Channel.Ask(c.Module, a); err != nil {
return g.refused(c, digest, "the ask could not be raised: "+err.Error())
}
if err := g.keepCall(a.ID, act, c.Args); err != nil {
g.stateFailed(err)
return g.refused(c, digest, "the ask was raised and its call could not be kept: "+err.Error())
}
if _, err := g.d.State.Put(g.actKey(digest), a.ID); err != nil {
g.stateFailed(err)
return g.refused(c, digest, "the ask was raised and its act could not be kept: "+err.Error())
}
return nil
}
// keptSize is the bytes the kept ask would take.
func keptSize(a Ask, act operatorask.Act, args json.RawMessage) int {
raw, _ := json.Marshal(Kept[Ask, Warrant]{Ask: a, Act: act, Arguments: args})
return len(raw)
}
// keepCall writes the act and the whole arguments beside the ask the client kept (§13).
func (g *Gate) keepCall(id string, act operatorask.Act, args json.RawMessage) error {
e, err := g.d.State.Get(g.askKey(id))
if err != nil || e == nil {
return fmt.Errorf("the kept ask %s cannot be read: %v", id, err)
}
var k kept
if err := json.Unmarshal(e.Value, &k); err != nil {
return err
}
k.Act, k.Arguments = act, args
_, err = g.d.State.Update(g.askKey(id), k, e.Revision)
return err
}
// firstAnswer waits up to RecordWithin for the router's record of the ask and says the call was not run (§12).
func (g *Gate) firstAnswer(c Call, id, digest, markedSince string) error {
rec, _ := g.awaitRecord(c.Module, id)
since := ""
if markedSince != "" {
since = "; marked since revision " + markedSince
}
if rec == nil {
return fmt.Errorf("Not run, asked; not yet taken by the router (ask %s, fingerprint %s); make the same call again later%s",
id, Fingerprint(digest), since)
}
return fmt.Errorf("Not run: this needs the operator's approval. Asked on %s at %s (ask %s, fingerprint %s); make the same call "+
"again within 30 minutes of the approval; the ask expires at %s%s", orNone(strings.Join(rec.OfferedOn, ", ")), rec.Opened,
id, Fingerprint(digest), rec.Expires, since)
}
// record is the router's record of an ask under the module's name, nil when it holds none.
func (g *Gate) record(module, id string) (*Record, error) {
return g.d.Channel.Record(module, id)
}
// awaitRecord reads the router's record of a new ask until it is there, within RecordWithin: the seat answers a key's
// value and says nothing when the router takes an ask, so it is read again after a growing pause (§12).
func (g *Gate) awaitRecord(module, id string) (*Record, error) {
until := g.d.Now().Add(RecordWithin)
pause := 50 * time.Millisecond
for {
rec, err := g.record(module, id)
if rec != nil || g.d.Now().Add(pause).After(until) {
return rec, err
}
<-g.d.After(pause)
pause *= 2
}
}
// refused is a marked call refused, run nothing, written in the runner's log with the ask's fingerprint and never its
// arguments (§19).
func (g *Gate) refused(c Call, digest, why string) error {
g.d.Logf("[approval] %s on %s is refused (fingerprint %s): %s", c.Address, g.d.Node, orNone(Fingerprint(digest)), why)
return fmt.Errorf("Not run: %s needs the operator's approval and %s", c.Address, why)
}
go/approval/channel.go
-115
View File
@@ -1,115 +0,0 @@
package approval
import (
"encoding/json"
"time"
"git.novox.be/novox/mesh-sdk/go/operatorask"
)
// What the gate reads beyond itself crosses in the gate's own terms, its caller translating from the generated contracts
// (design 47 G lead answers A-219, A-224, A-228): the operator channel (Channel), the catalog's entries (Entry, through
// Catalog), the finding writer (FindingWriter) and the seat holders (Deps.Holder). No form of a generated contract is
// copied here: what the router keeps whole (an ask, a warrant) the gate holds as the bytes it was given.
// Channel is the operator channel as the gate reaches it (A-219, A-228): operatorchannelclient implements it over the
// generated types, keeping each ask in the gate's own state under the marked module's name.
type Channel interface {
// Ask checks the ask, keeps it in the gate's state as a Kept and publishes it under module's name, at the level
// approve, to the operator (design 46 7-approval-mark §10).
Ask(module string, a Ask) error
// Record is the router's record of module's ask id, read under module's own name; nil while the router holds none
// (design 47 G §700).
Record(module, id string) (*Record, error)
// Claim takes the warrant once (design 46 7-approval-mark §14, §15): it holds it to the ask kept, checks that the
// chosen option binds act and marks the kept ask acted, then calls run; act's run happens only when it answers
// Claimed. A warrant already taken answers ClaimDone, one that does not hold ClaimRefused with why.
Claim(module string, w Warrant, act operatorask.Act, run func()) (Claim, error)
// Cancel takes module's ask back.
Cancel(module, id string) error
// Decided is the subject the router's `decided` for module is heard on.
Decided(module string) string
// Seat is the seat an ask is raised on, whose holder must be there for a marked call to be asked (§19).
Seat() string
}
// Claim is what came of a warrant handed to Channel.Claim.
type Claim string
const (
Claimed Claim = "acted"
ClaimRefused Claim = "refused"
ClaimDone Claim = "already acted on"
// ClaimEnded is a warrant of an ask that ended without a choice.
ClaimEnded Claim = "ended without a choice"
)
// Ask is an ask the gate raises: two options, each binding the act's digest (design 46 7-approval-mark §10, §11).
type Ask struct {
ID, About, Headline, Explanation, Expires, OnExpiry string
Options []Option
}
// Option is one option of an ask.
type Option struct{ ID, Label, Does, Binds string }
// Record is the router's record of an ask as the gate reads it.
type Record struct {
State, Answer, Opened, Expires string
OfferedOn []string
Warrant *Warrant
}
// Warrant is a person's choice as the gate reads it; Whole is the warrant as the router recorded it, which Claim holds to
// the kept ask.
type Warrant struct {
Ask, Asker, Option, Channel, At string
Whole json.RawMessage
}
// Kept is one ask as its asker keeps it (design 46 7-approval-mark §13): the ask, beside it the act the ask binds, the
// call's whole arguments, and whether the act's run was recorded in the hand-act log. A is the ask's form and W the
// warrant's: operatorchannelclient's Kept is Kept of the generated contract's, the gate's of the bytes kept (A-228).
type Kept[A, W any] struct {
Ask A `json:"ask"`
Asked time.Time `json:"asked"`
Acted bool `json:"acted,omitempty"`
Ended string `json:"ended,omitempty"`
Answer *W `json:"answer,omitempty"`
// Act is the act the ask's options bind, as its asker digested it.
Act operatorask.Act `json:"act,omitempty"`
// Arguments are the asked call's arguments whole.
Arguments json.RawMessage `json:"arguments,omitempty"`
// Recorded says the act's run is in the hand-act log (`mesh-hand-acts.warranted` answered).
Recorded bool `json:"recorded,omitempty"`
}
// kept is the gate's own reading of a Kept: the ask and the warrant as the bytes kept, so a write back keeps them whole.
type kept = Kept[json.RawMessage, json.RawMessage]
// HeadlineLength is the longest headline the router takes, in characters; operatorchannelclient re-exports it.
const HeadlineLength = 60
// Entry is a module's entry in the catalog as the marks are read from it (design 46 7-approval-mark §5): its module, the
// build's fingerprint, the seat verbs it serves as `<seat>.<verb>`, and its manifest.
type Entry struct {
Module, Fingerprint string
Serves []string
Manifest json.RawMessage
}
// formatTime is a time as the router's records carry it: RFC 3339 to the nanosecond; the zero time is empty.
func formatTime(t time.Time) string {
if t.IsZero() {
return ""
}
return t.Format(time.RFC3339Nano)
}
// parseTime reads such a time; empty is the zero time.
func parseTime(s string) (time.Time, error) {
if s == "" {
return time.Time{}, nil
}
return time.Parse(time.RFC3339Nano, s)
}
go/approval/closed.go
-100
View File
@@ -1,100 +0,0 @@
package approval
import (
"fmt"
"time"
)
// UnreadableAfter is how long the state approvals fails before the runner's finding says so (§19).
const UnreadableAfter = 5 * time.Minute
// unreadableKey is the bound key whose expiry raises `approvals-unreadable` (lead answer A-157).
func (g *Gate) unreadableKey() string { return g.d.Node + ".unreadable." + StateName }
// held says why no marked call can be asked now, or "" (§19): no module holds operator-channel, as the assigner's view
// of the seat's holders says, or that view is not read.
func (g *Gate) held() string {
if g.d.Channel == nil {
return "the tool runner reaches no operator channel to ask on"
}
if g.d.Holder == nil && g.d.Holding {
return "" // a process holder reads no view of the seats' holders: an ask nobody takes is said by its gate-unanswered (design 47 G §677)
}
if g.d.Holder == nil {
return "the tool runner reads no view of the seats' holders, so it cannot tell that a module holds " + g.d.Channel.Seat()
}
on := g.d.Channel.Seat()
ok, err := g.d.Holder(on)
switch {
case err != nil:
return fmt.Sprintf("whether a module holds %s cannot be read: %v", on, err)
case !ok:
return "no module holds " + on + ", so nobody can be asked"
}
return ""
}
// stateFailed notes that the state approvals could not be read or written: no marked call runs on this machine until it
// can, the failed read or write is made again at the next marked call and at the next membership change (A-157), and
// five minutes on the runner's finding says it (§19).
func (g *Gate) stateFailed(err error) {
g.mu.Lock()
first := g.failingSince.IsZero()
if first {
g.failingSince = g.d.Now()
}
g.mu.Unlock()
if first {
g.d.Logf("[approval] the state %s on %s cannot be read or written (%v): no marked call runs here until it can", StateName, g.d.Node, err)
if g.d.Bounds != nil && !g.d.Holding {
if e := g.d.Bounds.CreateTTL(g.unreadableKey(), map[string]string{"since": g.d.Now().UTC().Format(time.RFC3339)}, UnreadableAfter); e != nil {
g.d.Logf("[approval] the bound of the state %s's failure is not kept: %v", StateName, e)
}
}
}
}
// stateRead notes that the state approvals was read or written: a failure noted before ends, its finding cleared.
func (g *Gate) stateRead() {
g.mu.Lock()
was := !g.failingSince.IsZero()
g.failingSince = time.Time{}
g.mu.Unlock()
if !was {
return
}
if g.d.Bounds != nil && !g.d.Holding {
_ = g.d.Bounds.Delete(g.unreadableKey())
}
g.finding(StateName, KindApprovalsUnreadable, false, "the state "+StateName+" is read again", "")
}
// failing says the state approvals is failing now.
func (g *Gate) failing() bool {
g.mu.Lock()
defer g.mu.Unlock()
return !g.failingSince.IsZero()
}
// StateAgain makes the failed read of the state approvals again: at the next membership change the runner hears (A-157).
func (g *Gate) StateAgain() {
if g.d.State == nil || !g.failing() {
return
}
if _, err := g.d.State.Keys(); err != nil {
return
}
g.stateRead()
}
// unreadableExpired is the bound key of a failing state expiring: the finding raised while the state still fails.
func (g *Gate) unreadableExpired() {
g.mu.Lock()
since := g.failingSince
g.mu.Unlock()
if since.IsZero() {
return
}
g.finding(StateName, KindApprovalsUnreadable, true, fmt.Sprintf("the state %s on %s cannot be read or written since %s: no marked call "+
"runs here until it can", StateName, g.d.Node, since.UTC().Format(time.RFC3339)), "approvals unreadable on "+g.d.Node)
}
go/approval/gate.go
-178
View File
@@ -1,178 +0,0 @@
package approval
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"sync"
"time"
)
// The runner's findings of the gate (design 46 7-approval-mark §4, §16, §19), written through its held writer.
const (
KindMarkUnserved = "mark-unserved"
KindCallUnanswered = "call-unanswered"
KindApprovalsUnreadable = "approvals-unreadable"
)
// Kinds are the finding kinds the gate writes through the runner's writer.
var Kinds = []string{KindMarkUnserved, KindCallUnanswered, KindApprovalsUnreadable}
// FindingWriter raises and clears the gate's findings by kind and id (A-228): its caller writes each as the contract's
// finding, about its own module on its machine, through its held writer.
type FindingWriter interface {
Raise(ctx context.Context, kind, id, summary, headline string) error
Clear(ctx context.Context, kind, id, why string) error
}
// Deps is what the gate reaches the mesh through: the runner's connection in its one place, fakes in a test.
type Deps struct {
// Node is the machine the runner serves; Runner its own module's name, the subject of its findings.
Node, Runner string
// Catalog is the catalog half the marks are read from.
Catalog Catalog
// Wait waits, within the bound, for the catalog half's first view, and is nil once it is read or why not.
Wait func(within time.Duration) error
// Findings is the runner's held writer; nil says a finding in the log only.
Findings FindingWriter
Logf func(string, ...any)
Now func() time.Time
// After is the clock's pause; nil is time.After.
After func(time.Duration) <-chan time.Time
// State is the runner's state approvals; nil when its membership binds none, so every marked call is refused (§13).
State Store
// Channel is the operator channel, its asks kept in State under this machine's keys (A-219, A-228).
Channel Channel
// Warranted asks `mesh-hand-acts.warranted` {asker, ask}, each try a kept call open while open says so (§16, A-157).
Warranted func(asker, ask string, open func() (bool, error)) error
// Bounds is the runner's key-ttl state bound (A-157, A-158), or a process holder's state approvals itself (A-255); nil
// keeps no bound.
Bounds Bounds
// Holder says whether a module holds a seat, from the assigner's view of the seats' holders the runner follows (§19).
Holder func(seat string) (bool, error)
// Build is the tool runner's own fingerprint, read from its MESH_FINGERPRINT at its start: the build a marked wait's
// act binds; empty while unknown, when every marked wait is refused (design 34 every-event §35).
Build string
// Hear waits within the bound for the first event on the subjects whose body match takes, or the caller's going: a
// marked wait's ask's `decided` (every-event §36); false when none came.
Hear func(ctx context.Context, subjects []string, within time.Duration, match func(json.RawMessage) bool) (bool, error)
// Go runs what goes on beside the call (a run's recording); nil is a goroutine.
Go func(func())
// Holding says the gate is a module's process's seat server's (design 47 G §677): its keys are its module's own, its
// calls held as waiting calls and their ends said through Say; Runner is then the holder's module.
Holding bool
// Say publishes one gated-ended on its subject with the message id given (§677).
Say func(subject string, body any, id string) error
}
// Gate is the approval gate: every call of a module the tool runner serves goes through it before the bundle (§20), and a
// marked verb a module's process serves before its handler (design 47 G §677; lead answer A-224).
type Gate struct {
d Deps
mu sync.Mutex
// registered is each module's tools as its bundle registered them at its last load.
registered map[string]map[string]bool
// unserved is each module whose marks name a tool its bundle does not register, with those tools.
unserved map[string][]string
// unmarkedAt is each caller's address it last called unmarked, so a mark added since is named (§12).
unmarkedAt map[string]bool
// inFlight is each ask whose recording runs now.
inFlight map[string]bool
// failingSince is when the state approvals began to fail; zero while it is read.
failingSince time.Time
// handlers are a process holder's marked verbs' handlers, by `<seat>.<verb>` (design 47 G §677).
handlers map[string]func(string, json.RawMessage) (any, error)
// served is each module this runner serves, with the build its bundle serves, as its last load said (§18).
served map[string]string
}
// New is a gate over d.
func New(d Deps) *Gate {
if d.Logf == nil {
d.Logf = func(string, ...any) {}
}
if d.Now == nil {
d.Now = time.Now
}
if d.After == nil {
d.After = time.After
}
if d.Go == nil {
d.Go = func(f func()) { go f() }
}
return &Gate{d: d, registered: map[string]map[string]bool{}, unserved: map[string][]string{}}
}
// Unserved is, of the module's marked tools, those its bundle does not register (§4): such a bundle is not loaded, its
// finding `mark-unserved` raised; a bundle that serves them, or marks that no longer name them, clear it. Before the marks
// are read nothing is known, and every marked call is refused until they are (§19).
func (g *Gate) Unserved(module, fingerprint string, registered []string) []string {
set := map[string]bool{}
for _, t := range registered {
set[t] = true
}
g.mu.Lock()
g.registered[module] = set
if g.served == nil {
g.served = map[string]string{}
}
g.served[module] = fingerprint
g.mu.Unlock()
if g.d.Catalog == nil || g.d.Catalog.Ready() != nil {
return nil
}
return g.checkServed(Derive(g.d.Catalog, module, fingerprint))
}
// checkServed raises or clears the module's `mark-unserved` as its marks and its bundle's registrations stand now, and is
// the tools it marks and does not register.
func (g *Gate) checkServed(m Marks) []string {
g.mu.Lock()
set := g.registered[m.Module]
var missing []string
for t := range m.Tools {
if !set[t] {
missing = append(missing, t)
}
}
sort.Strings(missing)
was := g.unserved[m.Module]
if len(missing) > 0 {
g.unserved[m.Module] = missing
} else {
delete(g.unserved, m.Module)
}
g.mu.Unlock()
switch {
case len(missing) > 0 && strings.Join(was, ",") != strings.Join(missing, ","):
g.finding(m.Module, KindMarkUnserved, true, fmt.Sprintf("%s's marks (%s, catalog revision %s) name %s, which its bundle on %s does not "+
"register: no tool of %s is served here", m.Module, sourceOf(m, missing[0]), m.Revision, strings.Join(missing, ", "), g.d.Node, m.Module),
fmt.Sprintf("%s is not served on %s", m.Module, g.d.Node))
case len(missing) == 0 && len(was) > 0:
g.finding(m.Module, KindMarkUnserved, false, "a bundle that serves every marked tool, or marks without the tool, is loaded", "")
}
return missing
}
func sourceOf(m Marks, tool string) string { return m.Tools[tool] }
// finding writes or clears one of the gate's findings, its id `<node>.<what>` escaped as a key's name is,
// about the runner's module on this machine (lead answer A-154).
func (g *Gate) finding(what, kind string, raised bool, summary, headline string) {
id := g.findingID(what)
if g.d.Findings == nil {
g.d.Logf("[approval] %s %s: %s", kind, id, summary)
return
}
var err error
if raised {
err = g.d.Findings.Raise(context.Background(), kind, id, summary, headline)
} else {
err = g.d.Findings.Clear(context.Background(), kind, id, summary)
}
if err != nil {
g.d.Logf("[approval] the finding %s of %s is not kept: %v", kind, id, err)
}
}
go/approval/held.go
-699
View File
@@ -1,699 +0,0 @@
package approval
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/operatorask"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// A marked verb a module's process serves is one call with a gate before its running (design 47 G §677): the holder keeps
// the call, asks under its own module's name, and on the Approve runs that same call, saying its end on the seat's event
// `gated-ended`. These are the holder's half of the gate; the caller's is process.Conn's call.
// WaitHeader is the bus header a call carries to say it waits for its end on `gated-ended` (lead answer A-244): a header,
// never an argument, so no verb's input and no act's digest changes. WaitHeld is its one value.
const (
WaitHeader = "x-wait"
WaitHeld = "held"
)
// Held is the holder's answer to a call that waits (§677): the ask it waits on and its own call's id, in the reply's field
// `held` beside result, error and node (A-244).
type Held struct {
Asked string `json:"asked"`
Call string `json:"call"`
}
// GatedEndedEvent is the seat's event a call's end is said on, kept last per subject for GatedEndedKeptFor (§677).
const (
GatedEndedEvent = seatdef.GatedEndedEvent
GatedEndedKeptFor = seatdef.GatedEndedKeptDays * 24 * time.Hour
)
// The outcomes of a held call (§677).
const (
OutcomeRan = "ran"
OutcomeDeclined = "declined"
OutcomeExpired = "expired"
OutcomeRefused = "refused"
)
// KindGateUnanswered is the holder's finding of a waiting call past its ask's 60 minutes with no decided heard (§677),
// its id the call's.
const KindGateUnanswered = "gate-unanswered"
// GatedEnded is a held call's end as the holder says it (§677): no argument is in it.
type GatedEnded struct {
Call string `json:"call"`
Ask string `json:"ask"`
Act string `json:"act"`
Verb string `json:"verb"`
Outcome string `json:"outcome"`
Answer any `json:"answer,omitempty"`
Approval string `json:"approval,omitempty"`
Error string `json:"error,omitempty"`
}
// GatedEndedSubject is the subject one call's end is said on: `mesh.seat.<seat>.event.gated-ended.<caller>.<call>`, the
// caller the account the bus proved, escaped as a key's name is (§677, §73).
func GatedEndedSubject(seat, caller, call string) string {
return "mesh.seat." + seat + ".event." + GatedEndedEvent + "." + keys.EscapeName(caller) + "." + call
}
// WaitingCall is one call the holder keeps under `call.<id>` (§677): the call whole, the caller the bus proved, its ask,
// and once it ended its outcome, and whether that end was said.
type WaitingCall struct {
ID string `json:"id"`
Caller string `json:"caller"`
Ask string `json:"ask"`
Digest string `json:"digest"`
Act operatorask.Act `json:"act"`
Address string `json:"address"`
Seat string `json:"seat"`
Verb string `json:"verb"`
Arguments json.RawMessage `json:"arguments,omitempty"`
// Made is when the call was made; Asked when its ask (the newest, after an ask anew) was raised or taken.
Made time.Time `json:"made"`
Asked time.Time `json:"asked"`
// End is its end once it ended; Said says End was said on gated-ended.
End *GatedEnded `json:"end,omitempty"`
Said bool `json:"said,omitempty"`
}
func callKey(id string) string { return "call." + id }
func claimedKey(id string) string { return "claimed." + id }
// Serves names the handler a waiting call of seat's verb runs on its Approve (§677): process.Conn's AnswerSeat gives each
// marked verb's.
func (g *Gate) Serves(seat, verb string, h func(caller string, args json.RawMessage) (any, error)) {
g.mu.Lock()
defer g.mu.Unlock()
if g.handlers == nil {
g.handlers = map[string]func(string, json.RawMessage) (any, error){}
}
g.handlers[seat+"."+verb] = h
}
func (g *Gate) handler(seat, verb string) func(string, json.RawMessage) (any, error) {
g.mu.Lock()
defer g.mu.Unlock()
return g.handlers[seat+"."+verb]
}
// Hold is a call of a marked verb the holder serves (§677). A call whose arguments may not be asked, or that comes while
// the state approvals cannot be read or written, is refused at once and nothing is kept. One that waits (the header
// WaitHeader) is kept as a waiting call and answered at once with Held; one that does not takes ADR 0286 §37's path: it is
// refused naming its act's ask, and the same call made again after the Approve runs. A call of an act declined within 10
// minutes ends declined at once, unasked.
func (g *Gate) Hold(c Call, wait bool, run func() (any, error)) (any, *Held, error) {
if c.Fingerprint == "" {
c.Fingerprint = g.d.Build
}
if c.Fingerprint == "" {
return nil, nil, g.refused(c, "", "the holder's own fingerprint is unknown, so no warrant can bind its build")
}
if g.d.State == nil {
return nil, nil, g.refused(c, "", "its holder keeps no state "+StateName+" to keep its calls in")
}
if why := refusedUnasked(c.Args); why != "" {
return nil, nil, g.refused(c, "", why)
}
if why := g.held(); why != "" {
return nil, nil, g.refused(c, "", why)
}
act, err := ActOf(c, "") // a process holder's act names no node, so two instances raise one ask (A-255)
if err != nil {
return nil, nil, g.refused(c, "", err.Error())
}
digest, err := operatorask.ActDigest(act)
if err != nil {
return nil, nil, g.refused(c, "", err.Error())
}
if !wait {
answer, err := g.unheld(c, act, digest, run)
return answer, nil, err
}
id, err := g.askOf(c, act, digest)
if err != nil {
return nil, nil, err
}
now := g.d.Now().UTC()
w := WaitingCall{ID: newID(), Caller: c.Caller, Ask: id, Digest: digest, Act: act, Address: c.Address, Seat: c.Seat,
Verb: c.Name, Arguments: c.Args, Made: now, Asked: now}
if _, err := g.d.State.Create(callKey(w.ID), w); err != nil {
g.stateFailed(err)
return nil, nil, g.refused(c, digest, "its call cannot be kept in "+StateName+": "+err.Error())
}
g.stateRead()
g.armUnanswered(w)
g.d.Go(func() { g.Decided(id) }) // a choice made before the call was kept is acted on now
return nil, &Held{Asked: id, Call: w.ID}, nil
}
// askOf is the ask a call of the act takes (§677): the one `act.<digest>` names while it is open, else a new one, written
// with the bus's create (so two instances raise one ask) or, over an ended one, by compare-and-set on its revision. A call
// of an act declined within 10 minutes is refused with the time it may be asked again.
func (g *Gate) askOf(c Call, act operatorask.Act, digest string) (string, error) {
key := g.actKey(digest)
for {
e, err := g.d.State.Get(key)
if err != nil {
g.stateFailed(err)
return "", g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
}
g.stateRead()
id := newID()
if e == nil {
if _, err := g.d.State.Create(key, id); err != nil {
if isChanged(err) {
continue
}
g.stateFailed(err)
return "", g.refused(c, digest, "its act cannot be kept in "+StateName+": "+err.Error())
}
return id, g.raiseHeld(c, act, digest, id)
}
var named string
if err := json.Unmarshal(e.Value, &named); err != nil || named == "" {
return "", g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName))
}
open, until, err := g.openAsk(c.Module, named)
if err != nil {
return "", g.refused(c, digest, err.Error())
}
if !until.IsZero() {
return "", fmt.Errorf("Not run: the operator declined ask %s; the same call is not asked again until %s", named, formatTime(until))
}
if open {
return named, nil
}
if _, err := g.d.State.Update(key, id, e.Revision); err != nil {
if isChanged(err) {
continue
}
g.stateFailed(err)
return "", g.refused(c, digest, "its act cannot be kept in "+StateName+": "+err.Error())
}
return id, g.raiseHeld(c, act, digest, id)
}
}
// openAsk says whether an ask is open for another call of its act: raised and not yet ended by the router; and, when it was
// declined within 10 minutes, until when no call of its act is asked.
func (g *Gate) openAsk(module, id string) (bool, time.Time, error) {
rec, err := g.record(module, id)
if err != nil {
return false, time.Time{}, fmt.Errorf("the router's record of its ask %s cannot be read: %v", id, err)
}
if rec == nil {
k, _, err := g.kept(id)
return err == nil && k != nil && k.Ended == "", time.Time{}, nil // raised and not yet taken by the router
}
switch rec.State {
case "open", "held-back":
return true, time.Time{}, nil
case "chosen":
if w := rec.Warrant; w != nil && w.Option == OptionDecline {
at, _ := parseTime(w.At)
if until := at.Add(DeclinedFor); g.d.Now().Before(until) {
return false, until, nil
}
}
}
return false, time.Time{}, nil
}
// raiseHeld raises the ask the act's key names, under the holder's module's name, keeping the act and the arguments beside
// it (§677; design 46 7-approval-mark §10).
func (g *Gate) raiseHeld(c Call, act operatorask.Act, digest, id string) error {
return g.keepAskHeld(c, act, digest, g.askFor(c, digest, id))
}
// keepAskHeld is keepAsk without its write of the act's key, which askOf made first.
func (g *Gate) keepAskHeld(c Call, act operatorask.Act, digest string, a Ask) error {
if size := keptSize(a, act, c.Args); size > MostKeptValue {
return g.refused(c, digest, fmt.Sprintf("its ask would keep %d bytes, more than the %d a value of the state %s holds", size, MostKeptValue, StateName))
}
if err := g.d.Channel.Ask(c.Module, a); err != nil {
return g.refused(c, digest, "the ask could not be raised: "+err.Error())
}
if err := g.keepCall(a.ID, act, c.Args); err != nil {
g.stateFailed(err)
return g.refused(c, digest, "the ask was raised and its call could not be kept: "+err.Error())
}
return nil
}
// unheld is a call that does not wait (ADR 0286 §37's path, §677): run when its act's ask is chosen Approve within 30
// minutes and unclaimed, claimed once; otherwise refused naming the ask, raising one when none is open.
func (g *Gate) unheld(c Call, act operatorask.Act, digest string, run func() (any, error)) (any, error) {
named, err := g.d.State.Get(g.actKey(digest))
if err != nil {
g.stateFailed(err)
return nil, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
}
g.stateRead()
if named != nil {
var id string
if json.Unmarshal(named.Value, &id) == nil && id != "" {
rec, err := g.record(c.Module, id)
if err != nil {
return nil, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error())
}
if rec != nil && rec.State == "chosen" && rec.Warrant != nil && rec.Warrant.Option == OptionApprove {
if at, _ := parseTime(rec.Warrant.At); !g.d.Now().After(at.Add(RunWithin)) {
out := g.claimRun(c.Module, id, *rec.Warrant, act, run)
switch out.outcome {
case OutcomeRan:
return g.ran(c, id, rec, out.answer, out.err)
case OutcomeRefused:
return nil, g.refused(c, digest, out.why)
}
}
}
}
}
id, err := g.askOf(c, act, digest)
if err != nil {
return nil, err
}
return nil, g.firstAnswer(c, id, digest, "")
}
// claimed is what came of a claim of a warrant for one call.
type claimed struct {
outcome string // OutcomeRan, OutcomeRefused, or "" when another took it
answer any
err error
why string
}
// claimRun holds the warrant to the ask and the call's act, claims it once at `claimed.<ask id>` with the bus's create, and
// runs the call (§677; ADR 0286 §27, §28, §30, §31): of two instances, or two deliveries, one runs.
func (g *Gate) claimRun(module, id string, w Warrant, act operatorask.Act, run func() (any, error)) claimed {
var out claimed
var claimErr error
h, err := g.d.Channel.Claim(module, w, act, func() {
if _, err := g.d.State.Create(claimedKey(id), map[string]string{"at": formatTime(g.d.Now().UTC())}); err != nil {
claimErr = err
return
}
out.outcome = OutcomeRan
out.answer, out.err = run()
})
switch {
case h == Claimed && claimErr != nil && !isChanged(claimErr):
g.stateFailed(claimErr)
return claimed{outcome: OutcomeRefused, why: "its claim cannot be written in " + StateName + ": " + claimErr.Error()}
case h == Claimed:
return out
case h == ClaimDone:
return claimed{}
case h == "" && err != nil:
g.stateFailed(err)
return claimed{outcome: OutcomeRefused, why: "its state " + StateName + " cannot be written: " + err.Error()}
default: // refused, or the act about to run is not the one the option bound: nothing runs
return claimed{outcome: OutcomeRefused, why: "its warrant does not hold: " + errText(err)}
}
}
// Decided acts on the router's own record of one of the holder's asks (§677): heard on the holder's `decided`, never from
// its body, and at the start for each ask a waiting call waits on. On the Approve the oldest waiting call of the ask runs,
// within 30 minutes of the choice, else it ends expired; every other is asked anew. A Decline ends every one declined, an
// ask that ended without a choice every one expired, and one the router refused every one refused with its words.
func (g *Gate) Decided(id string) {
module := g.d.Runner
rec, err := g.record(module, id)
if err != nil {
g.d.Logf("[approval] the router's record of ask %s cannot be read (%v): read again at the next start", id, err)
return
}
if rec == nil || rec.State == "open" || rec.State == "held-back" {
return
}
calls, err := g.waiting(func(w WaitingCall) bool { return w.Ask == id })
if err != nil {
g.stateFailed(err)
g.d.Logf("[approval] the waiting calls of ask %s cannot be read (%v): read again at the next start", id, err)
return
}
if len(calls) == 0 {
return
}
end := func(w WaitingCall, outcome, why string) {
g.end(w, GatedEnded{Outcome: outcome, Error: why})
}
switch {
case rec.State == "refused":
for _, w := range calls {
end(w, OutcomeRefused, "the router refused its ask "+id+": "+orNone(rec.Answer))
}
return
case rec.State != "chosen":
for _, w := range calls {
end(w, OutcomeExpired, "its ask "+id+" ended "+rec.State+" with no choice; nothing ran")
}
return
case rec.Warrant == nil:
for _, w := range calls {
end(w, OutcomeRefused, "the router's record of its ask "+id+" is chosen and carries no warrant")
}
return
case rec.Warrant.Option == OptionDecline:
for _, w := range calls {
end(w, OutcomeDeclined, "the operator declined ask "+id+"; nothing ran")
}
return
}
first, rest := calls[0], calls[1:]
at, _ := parseTime(rec.Warrant.At)
if g.d.Now().After(at.Add(RunWithin)) {
end(first, OutcomeExpired, "the Approve of ask "+id+" was heard more than 30 minutes after it was given; nothing ran")
} else {
g.runWaiting(first, rec)
}
for _, w := range rest {
g.anew(w)
}
}
// runWaiting claims and runs one waiting call on its ask's warrant, saying its end.
func (g *Gate) runWaiting(w WaitingCall, rec *Record) {
h := g.handler(w.Seat, w.Verb)
c := Call{Address: w.Address, Module: g.d.Runner, Seat: w.Seat, Name: w.Verb, Caller: w.Caller, Args: w.Arguments}
if h == nil {
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: "the holder serves no handler of " + w.Address + " now; nothing ran"})
return
}
out := g.claimRun(g.d.Runner, w.Ask, *rec.Warrant, w.Act, func() (any, error) { return h(w.Caller, w.Arguments) })
switch out.outcome {
case "": // another instance, or another call of the same ask, took the warrant
return
case OutcomeRefused:
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: out.why})
return
}
answered, err := g.ran(c, w.Ask, rec, out.answer, out.err)
if err != nil {
g.end(w, GatedEnded{Outcome: OutcomeRan, Error: err.Error()})
return
}
a := answered.(Answered)
g.end(w, GatedEnded{Outcome: OutcomeRan, Answer: a.Answer, Approval: a.Approval})
}
// anew asks a waiting call's act anew, as a new ask (ADR 0286 §33: a second run takes a second approval); its end comes
// when its own ask ends.
func (g *Gate) anew(w WaitingCall) {
c := Call{Address: w.Address, Module: g.d.Runner, Seat: w.Seat, Name: w.Verb, Caller: w.Caller, Args: w.Arguments,
Fingerprint: w.Act["build"]}
id, err := g.askOf(c, w.Act, w.Digest)
if err != nil {
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: err.Error()})
return
}
if err := g.updateCall(w.ID, func(k *WaitingCall) { k.Ask, k.Asked = id, g.d.Now().UTC() }); err != nil {
g.d.Logf("[approval] the call %s, asked anew as ask %s, cannot be kept (%v)", w.ID, id, err)
return
}
w.Ask, w.Asked = id, g.d.Now().UTC()
g.rearmUnanswered(w)
g.d.Go(func() { g.Decided(id) })
}
// end keeps a waiting call's end, then says it on gated-ended (§677); an end not said is said again at the next start.
func (g *Gate) end(w WaitingCall, e GatedEnded) {
e.Call, e.Ask, e.Act, e.Verb = w.ID, w.Ask, w.Digest, w.Verb
ended := false
if err := g.updateCall(w.ID, func(k *WaitingCall) {
if k.End == nil {
k.End, ended = &e, true
}
}); err != nil {
g.d.Logf("[approval] the end %s of call %s cannot be kept (%v)", e.Outcome, w.ID, err)
return
}
if ended {
g.say(w.ID)
}
}
// say says a kept end on gated-ended and marks it said; its finding gate-unanswered is cleared.
func (g *Gate) say(id string) {
e, err := g.d.State.Get(callKey(id))
if err != nil || e == nil {
return
}
var w WaitingCall
if json.Unmarshal(e.Value, &w) != nil || w.End == nil || w.Said {
return
}
// Its bound key still kept says gate-unanswered was not raised, and the key goes; one the bus ended is cleared once the
// end is said (A-255).
b, err := g.d.State.Get(unansweredHeldKey(id))
raised := err == nil && b == nil
if b != nil && g.d.Bounds != nil {
_ = g.d.Bounds.Delete(unansweredHeldKey(id))
}
if g.d.Say == nil {
g.d.Logf("[approval] the end %s of call %s is not said: the holder says no gated-ended", w.End.Outcome, id)
return
}
if err := g.d.Say(GatedEndedSubject(w.Seat, w.Caller, w.ID), *w.End, GatedEndedEvent+"."+w.ID); err != nil {
g.d.Logf("[approval] the end %s of call %s is not said (%v): said again at the next start", w.End.Outcome, id, err)
return
}
g.d.Logf("[approval] %s by %s ended %s (call %s, ask %s)", w.Address, w.Caller, w.End.Outcome, id, w.Ask)
_ = g.updateCall(id, func(k *WaitingCall) { k.Said = true })
if raised && g.d.Findings != nil {
_ = g.d.Findings.Clear(nilCtx(), KindGateUnanswered, id, "the call's end, "+w.End.Outcome+", is said")
}
}
// updateCall changes a kept call by compare-and-set, again when another write came first.
func (g *Gate) updateCall(id string, change func(*WaitingCall)) error {
for {
e, err := g.d.State.Get(callKey(id))
if err != nil || e == nil {
return fmt.Errorf("the call %s cannot be read: %v", id, err)
}
var w WaitingCall
if err := json.Unmarshal(e.Value, &w); err != nil {
return err
}
change(&w)
if _, err := g.d.State.Update(callKey(id), w, e.Revision); err == nil {
return nil
} else if !isChanged(err) {
return err
}
}
}
// waiting is every kept call that has not ended and that keep takes, oldest first.
func (g *Gate) waiting(keep func(WaitingCall) bool) ([]WaitingCall, error) {
all, err := g.calls()
if err != nil {
return nil, err
}
var out []WaitingCall
for _, w := range all {
if w.End == nil && keep(w) {
out = append(out, w)
}
}
return out, nil
}
// calls is every kept call, oldest first.
func (g *Gate) calls() ([]WaitingCall, error) {
names, err := g.d.State.Keys()
if err != nil {
return nil, err
}
var out []WaitingCall
for _, key := range names {
if !strings.HasPrefix(key, "call.") {
continue
}
e, err := g.d.State.Get(key)
if err != nil {
return nil, err
}
var w WaitingCall
if e == nil || json.Unmarshal(e.Value, &w) != nil {
continue
}
out = append(out, w)
}
sort.Slice(out, func(i, j int) bool { return out[i].Made.Before(out[j].Made) })
return out, nil
}
// unansweredHeldKey is a waiting call's bound key in the holder's state approvals (lead answer A-255): created with the
// ask's 60 minutes as its own time to live, its expiry heard as the bus's event raises gate-unanswered.
func unansweredHeldKey(call string) string { return "unanswered." + call }
// armUnanswered keeps the waiting call's bound key (§677; A-255): when its ask's 60 minutes pass with no end of it, the bus
// ends the key and the holder, hearing that, raises gate-unanswered; no clock waits in the process. A call already past its
// 60 minutes (its key ended while the holder was away) is judged at once.
func (g *Gate) armUnanswered(w WaitingCall) {
left := w.Asked.Add(AskLasts).Sub(g.d.Now())
if left <= 0 {
g.heldUnanswered(w.ID)
return
}
if g.d.Bounds == nil {
g.d.Logf("[approval] the call %s waits on ask %s with no bound kept: the holder's %s keeps no key with its own time to live",
w.ID, w.Ask, StateName)
return
}
if err := g.d.Bounds.CreateTTL(unansweredHeldKey(w.ID), map[string]string{"call": w.ID, "ask": w.Ask}, left); err != nil && !isChanged(err) {
g.d.Logf("[approval] the bound of call %s on ask %s is not kept: %v", w.ID, w.Ask, err)
}
}
// rearmUnanswered keeps a call asked anew under a new bound of its new ask's 60 minutes.
func (g *Gate) rearmUnanswered(w WaitingCall) {
if g.d.Bounds != nil {
_ = g.d.Bounds.Delete(unansweredHeldKey(w.ID))
}
g.armUnanswered(w)
}
// heldUnanswered is a waiting call's bound ended (A-255): gate-unanswered raised while the call has no end.
func (g *Gate) heldUnanswered(id string) {
e, err := g.d.State.Get(callKey(id))
if err != nil || e == nil {
return
}
var w WaitingCall
if json.Unmarshal(e.Value, &w) != nil || w.End != nil || g.d.Findings == nil {
return
}
if err := g.d.Findings.Raise(nilCtx(), KindGateUnanswered, w.ID, fmt.Sprintf("the call %s of %s by %s waits on ask %s, "+
"past its %s with no decided heard: nothing ran; the holder's %s.approvals reads it", w.ID, w.Address, w.Caller, w.Ask,
AskLasts, w.Seat), "a call waits on an unanswered ask"); err != nil {
g.d.Logf("[approval] the finding %s of call %s is not kept: %v", KindGateUnanswered, w.ID, err)
}
}
// watchHeld hears the waiting calls' bound keys end (A-255) for the life of the holder's connection.
func (g *Gate) watchHeld() error {
if g.d.Bounds == nil {
return nil
}
_, err := g.d.Bounds.Watch("unanswered.*", func(key string) { g.heldUnanswered(strings.TrimPrefix(key, "unanswered.")) })
return err
}
// StartHolding is the holder's read at its start (§677): every waiting call is read, each ask already decided acted on as
// when its decided is heard, each end not yet said said, each waiting call's 60 minutes armed, and every acted, unrecorded
// run recorded. A state that cannot be read leaves every marked call refused until it can.
func (g *Gate) StartHolding() error {
all, err := g.calls()
if err != nil {
g.stateFailed(err)
return fmt.Errorf("the waiting calls in %s cannot be read: %w", StateName, err)
}
g.stateRead()
if err := g.watchHeld(); err != nil {
return fmt.Errorf("the bound keys in %s cannot be watched: %w", StateName, err)
}
asks := map[string]bool{}
for _, w := range all {
switch {
case w.End != nil && !w.Said:
g.say(w.ID)
case w.End == nil:
g.armUnanswered(w)
asks[w.Ask] = true
}
}
ids := make([]string, 0, len(asks))
for id := range asks {
ids = append(ids, id)
}
sort.Strings(ids)
for _, id := range ids {
g.Decided(id)
}
g.Recover()
return nil
}
// Approvals is the seat's verb `approvals` (§677): the waiting and ended calls with their caller, ask and state, the
// `act.` keys and the claims; a call's arguments only to that call's caller.
type Approvals struct {
Calls []ListedCall `json:"calls"`
Acts []ListedAct `json:"acts"`
Claims []string `json:"claims"`
}
// ListedCall is one kept call as `approvals` lists it.
type ListedCall struct {
Call string `json:"call"`
Caller string `json:"caller"`
Ask string `json:"ask"`
Verb string `json:"verb"`
State string `json:"state"`
Made time.Time `json:"made"`
End *GatedEnded `json:"end,omitempty"`
Arguments json.RawMessage `json:"arguments,omitempty"`
}
// ListedAct is one `act.` key: its digest, escaped as kept, and the ask it names.
type ListedAct struct {
Act string `json:"act"`
Ask string `json:"ask"`
}
// Approvals answers the verb for caller, the account the bus proved.
func (g *Gate) Approvals(caller string) (Approvals, error) {
out := Approvals{Calls: []ListedCall{}, Acts: []ListedAct{}, Claims: []string{}}
if g.d.State == nil {
return out, fmt.Errorf("the holder keeps no state %s", StateName)
}
all, err := g.calls()
if err != nil {
return out, fmt.Errorf("the state %s cannot be read: %w", StateName, err)
}
for _, w := range all {
l := ListedCall{Call: w.ID, Caller: w.Caller, Ask: w.Ask, Verb: w.Verb, State: "waiting", Made: w.Made, End: w.End}
if w.End != nil {
l.State = w.End.Outcome
}
if caller != "" && caller == w.Caller {
l.Arguments = w.Arguments
}
out.Calls = append(out.Calls, l)
}
names, err := g.d.State.Keys()
if err != nil {
return out, fmt.Errorf("the state %s cannot be read: %w", StateName, err)
}
sort.Strings(names)
for _, key := range names {
switch {
case strings.HasPrefix(key, "act."):
e, err := g.d.State.Get(key)
var id string
if err == nil && e != nil {
_ = json.Unmarshal(e.Value, &id)
}
out.Acts = append(out.Acts, ListedAct{Act: strings.TrimPrefix(key, "act."), Ask: id})
case strings.HasPrefix(key, "claimed."):
out.Claims = append(out.Claims, strings.TrimPrefix(key, "claimed."))
}
}
return out, nil
}
func nilCtx() context.Context { return context.Background() }
go/approval/held_test.go
-439
View File
@@ -1,439 +0,0 @@
package approval
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"sync"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/operatorask"
)
// memStore is the holder's state approvals in memory: create refuses a key that exists, update is compare-and-set.
type memStore struct {
mu sync.Mutex
vals map[string][]byte
revs map[string]uint64
rev uint64
fail error
}
func newMem() *memStore { return &memStore{vals: map[string][]byte{}, revs: map[string]uint64{}} }
func (m *memStore) Get(key string) (*stdio.StateEntry, error) {
m.mu.Lock()
defer m.mu.Unlock()
if m.fail != nil {
return nil, m.fail
}
v, ok := m.vals[key]
if !ok {
return nil, nil
}
return &stdio.StateEntry{Key: key, Value: v, Revision: m.revs[key]}, nil
}
func (m *memStore) write(key string, value any, check func(exists bool, rev uint64) bool) (uint64, error) {
m.mu.Lock()
defer m.mu.Unlock()
if m.fail != nil {
return 0, m.fail
}
_, exists := m.vals[key]
if !check(exists, m.revs[key]) {
return 0, stdio.ErrChanged
}
raw, err := json.Marshal(value)
if err != nil {
return 0, err
}
m.rev++
m.vals[key], m.revs[key] = raw, m.rev
return m.rev, nil
}
func (m *memStore) Create(key string, value any) (uint64, error) {
return m.write(key, value, func(exists bool, _ uint64) bool { return !exists })
}
func (m *memStore) Update(key string, value any, revision uint64) (uint64, error) {
return m.write(key, value, func(exists bool, rev uint64) bool { return exists && rev == revision })
}
func (m *memStore) Put(key string, value any) (uint64, error) {
return m.write(key, value, func(bool, uint64) bool { return true })
}
func (m *memStore) Keys() ([]string, error) {
m.mu.Lock()
defer m.mu.Unlock()
if m.fail != nil {
return nil, m.fail
}
var out []string
for k := range m.vals {
out = append(out, k)
}
sort.Strings(out)
return out, nil
}
// bounds is the holder's bound keys in its state approvals as a fake (A-255): each key kept in the state with its time to
// live, expire ending it as the bus does and handing it to the watch.
type bounds struct {
state *memStore
mu sync.Mutex
ttl map[string]time.Duration
watched func(string)
}
func (b *bounds) CreateTTL(key string, value any, ttl time.Duration) error {
if _, err := b.state.Create(key, value); err != nil {
return err
}
b.mu.Lock()
defer b.mu.Unlock()
b.ttl[key] = ttl
return nil
}
func (b *bounds) Delete(key string) error {
b.state.delete(key)
b.mu.Lock()
defer b.mu.Unlock()
delete(b.ttl, key)
return nil
}
func (b *bounds) Watch(pattern string, expired func(string)) (func(), error) {
b.mu.Lock()
defer b.mu.Unlock()
b.watched = expired
return func() {}, nil
}
func (b *bounds) expire(key string) {
b.mu.Lock()
_, ok := b.ttl[key]
delete(b.ttl, key)
w := b.watched
b.mu.Unlock()
b.state.delete(key)
if ok && w != nil {
w(key)
}
}
func (m *memStore) delete(key string) {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.vals, key)
delete(m.revs, key)
}
// router is the operator channel as a fake: it keeps each ask the gate raises, and the test decides its record.
type router struct {
mu sync.Mutex
state Store
asks []Ask
records map[string]*Record
}
func (r *router) Ask(module string, a Ask) error {
raw, _ := json.Marshal(a)
if _, err := r.state.Create("ask."+a.ID, kept{Ask: raw, Asked: time.Now()}); err != nil {
return err
}
r.mu.Lock()
defer r.mu.Unlock()
r.asks = append(r.asks, a)
return nil
}
func (r *router) Record(module, id string) (*Record, error) {
r.mu.Lock()
defer r.mu.Unlock()
return r.records[id], nil
}
func (r *router) Claim(module string, w Warrant, act operatorask.Act, run func()) (Claim, error) {
e, _ := r.state.Get("ask." + w.Ask)
if e == nil {
return ClaimRefused, fmt.Errorf("no ask %s", w.Ask)
}
var k kept
_ = json.Unmarshal(e.Value, &k)
if k.Acted {
return ClaimDone, nil
}
var a Ask
_ = json.Unmarshal(k.Ask, &a)
digest, _ := operatorask.ActDigest(act)
if a.About != digest {
return ClaimRefused, fmt.Errorf("the option binds another act")
}
k.Acted = true
if _, err := r.state.Update("ask."+w.Ask, k, e.Revision); err != nil {
return ClaimDone, nil
}
run()
return Claimed, nil
}
func (r *router) Cancel(module, id string) error { return nil }
func (r *router) Decided(module string) string { return "decided." + module }
func (r *router) Seat() string { return "operator-channel" }
func (r *router) choose(id, option string, at time.Time) {
r.mu.Lock()
defer r.mu.Unlock()
r.records[id] = &Record{State: "chosen", OfferedOn: []string{"telegram"},
Warrant: &Warrant{Ask: id, Asker: "holder", Option: option, Channel: "telegram", At: formatTime(at)}}
}
type findings struct {
mu sync.Mutex
raised map[string]bool
heard chan string
}
func (f *findings) Raise(_ context.Context, kind, id, _, _ string) error {
f.mu.Lock()
defer f.mu.Unlock()
f.raised[kind+" "+id] = true
if f.heard != nil {
f.heard <- kind + " " + id
}
return nil
}
func (f *findings) Clear(_ context.Context, kind, id, _ string) error {
f.mu.Lock()
defer f.mu.Unlock()
delete(f.raised, kind+" "+id)
return nil
}
// Design 47 G §677 (lead answers A-158, A-209, A-244): a process holder's marked verb is one call with a gate before its
// running, its waiting calls, act keys and claims kept in its own state.
func TestAHolderHoldsAMarkedCall(t *testing.T) {
now := time.Date(2026, 11, 12, 10, 0, 0, 0, time.UTC)
type world struct {
g *Gate
state *memStore
router *router
said map[string]GatedEnded
ran *int
recs *int
finds *findings
after chan time.Time
bounds *bounds
}
newGate := func(state *memStore, r *router, ran, recs *int, said map[string]GatedEnded, after chan time.Time, f *findings, b *bounds) *Gate {
g := New(Deps{Node: "n1", Runner: "holder", Holding: true, Build: "fp1", State: state, Channel: r, Findings: f, Bounds: b,
Now: func() time.Time { return now }, Go: func(f func()) { f() },
After: func(d time.Duration) <-chan time.Time {
if d < time.Minute { // the record's short pauses pass at once
fired := make(chan time.Time)
close(fired)
return fired
}
return after
},
Warranted: func(asker, ask string, open func() (bool, error)) error {
*recs++
return nil
},
Say: func(subject string, body any, id string) error {
said[subject] = body.(GatedEnded)
return nil
}})
g.Serves("wiper", "wipe", func(caller string, args json.RawMessage) (any, error) { *ran++; return "wiped", nil })
return g
}
setup := func() world {
state := newMem()
r := &router{state: state, records: map[string]*Record{}}
ran, recs := 0, 0
w := world{state: state, router: r, said: map[string]GatedEnded{}, ran: &ran, recs: &recs,
finds: &findings{raised: map[string]bool{}}, after: make(chan time.Time), bounds: &bounds{state: state, ttl: map[string]time.Duration{}}}
w.g = newGate(state, r, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
return w
}
call := func(caller, args string) Call {
return Call{Address: "wiper.wipe", Module: "holder", Seat: "wiper", Name: "wipe", Caller: caller, Args: json.RawMessage(args)}
}
hold := func(t *testing.T, w world, c Call, wait bool) (any, *Held, error) {
return w.g.Hold(c, wait, func() (any, error) { *w.ran++; return "wiped", nil })
}
ended := func(w world, caller, id string) GatedEnded { return w.said[GatedEndedSubject("wiper", caller, id)] }
t.Run("asked, run once on the Approve, its end said to its caller", func(t *testing.T) {
w := setup()
_, h, err := hold(t, w, call("n1.caller", `{"dir":"scratch"}`), true)
if err != nil || h == nil || h.Asked == "" || h.Call == "" {
t.Fatalf("held %+v, %v", h, err)
}
if e, _ := w.state.Get("call." + h.Call); e == nil {
t.Fatal("no call kept")
}
w.router.choose(h.Asked, OptionApprove, now)
w.g.Decided(h.Asked)
w.g.Decided(h.Asked) // heard twice: run once
end := ended(w, "n1.caller", h.Call)
if *w.ran != 1 || end.Outcome != OutcomeRan || end.Answer != "wiped" || !strings.Contains(end.Approval, h.Asked) || *w.recs != 1 {
t.Fatalf("ran %d, recorded %d, end %+v", *w.ran, *w.recs, end)
}
if e, _ := w.state.Get("claimed." + h.Asked); e == nil {
t.Error("no claim kept")
}
a, _ := w.g.Approvals("someone.else")
if len(a.Calls) != 1 || a.Calls[0].Arguments != nil || a.Calls[0].State != OutcomeRan || len(a.Acts) != 1 || len(a.Claims) != 1 {
t.Errorf("approvals %+v", a)
}
if a, _ := w.g.Approvals("n1.caller"); string(a.Calls[0].Arguments) != `{"dir":"scratch"}` {
t.Errorf("its caller reads %s", a.Calls[0].Arguments)
}
})
t.Run("a second call of the act takes the same ask, asked anew after the first runs", func(t *testing.T) {
w := setup()
_, h1, _ := hold(t, w, call("n1.a", `{}`), true)
now = now.Add(time.Second)
_, h2, _ := hold(t, w, call("n1.b", `{}`), true)
if h1.Asked != h2.Asked || h1.Call == h2.Call {
t.Fatalf("asks %s and %s", h1.Asked, h2.Asked)
}
w.router.choose(h1.Asked, OptionApprove, now)
w.g.Decided(h1.Asked)
if *w.ran != 1 || ended(w, "n1.a", h1.Call).Outcome != OutcomeRan || ended(w, "n1.b", h2.Call).Outcome != "" {
t.Fatalf("ran %d; ends %v", *w.ran, w.said)
}
e, _ := w.state.Get("call." + h2.Call)
var second WaitingCall
_ = json.Unmarshal(e.Value, &second)
if second.Ask == h1.Asked || len(w.router.asks) != 2 {
t.Errorf("the second call waits on %s (asks %d)", second.Ask, len(w.router.asks))
}
})
t.Run("a decline runs nothing; a call within 10 minutes ends declined unasked", func(t *testing.T) {
w := setup()
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
w.router.choose(h.Asked, OptionDecline, now)
w.g.Decided(h.Asked)
if *w.ran != 0 || ended(w, "n1.a", h.Call).Outcome != OutcomeDeclined {
t.Fatalf("ran %d, end %+v", *w.ran, ended(w, "n1.a", h.Call))
}
if _, h2, err := hold(t, w, call("n1.a", `{}`), true); err == nil || h2 != nil || !strings.Contains(err.Error(), "declined") || len(w.router.asks) != 1 {
t.Errorf("a call within 10 minutes: %v, %d asks", err, len(w.router.asks))
}
})
t.Run("heard 31 minutes after the choice, it ends expired", func(t *testing.T) {
w := setup()
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
w.router.choose(h.Asked, OptionApprove, now.Add(-31*time.Minute))
w.g.Decided(h.Asked)
if *w.ran != 0 || ended(w, "n1.a", h.Call).Outcome != OutcomeExpired {
t.Fatalf("ran %d, end %+v", *w.ran, ended(w, "n1.a", h.Call))
}
})
t.Run("a secret-shaped argument refused, nothing kept", func(t *testing.T) {
w := setup()
if _, h, err := hold(t, w, call("n1.a", `{"token":"ghp_0123456789abcdefghijklmnopqrstuvwxyzAB"}`), true); err == nil || h != nil {
t.Fatalf("held %+v, %v", h, err)
}
if keys, _ := w.state.Keys(); len(keys) != 0 || len(w.router.asks) != 0 || len(w.bounds.ttl) != 0 {
t.Errorf("kept %v, asked %d", keys, len(w.router.asks))
}
})
t.Run("a call with no wait refused with the ask, and run when made again after the Approve", func(t *testing.T) {
w := setup()
_, h, err := hold(t, w, call("n1.a", `{}`), false)
if err == nil || h != nil || len(w.router.asks) != 1 || !strings.Contains(err.Error(), w.router.asks[0].ID) {
t.Fatalf("first: %v", err)
}
w.router.choose(w.router.asks[0].ID, OptionApprove, now)
answer, _, err := hold(t, w, call("n1.a", `{}`), false)
if a, ok := answer.(Answered); err != nil || !ok || a.Answer != "wiped" || *w.ran != 1 {
t.Fatalf("again: %+v, %v", answer, err)
}
})
t.Run("a restart between the ask and the Approve runs the waiting call once", func(t *testing.T) {
w := setup()
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
w.router.choose(h.Asked, OptionApprove, now)
again := newGate(w.state, w.router, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
if err := again.StartHolding(); err != nil || *w.ran != 1 || ended(w, "n1.a", h.Call).Outcome != OutcomeRan {
t.Fatalf("start: %v, ran %d", err, *w.ran)
}
_ = again.StartHolding()
if *w.ran != 1 {
t.Errorf("ran %d after a second start", *w.ran)
}
})
t.Run("past its ask's 60 minutes with no decided, gate-unanswered on its bound key's end", func(t *testing.T) {
// A-255: a bound key with the ask's 60 minutes to live in the state approvals, never a clock wait.
w := setup()
if err := w.g.StartHolding(); err != nil {
t.Fatal(err)
}
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
key := "unanswered." + h.Call
if ttl := w.bounds.ttl[key]; ttl != AskLasts {
t.Fatalf("bound %s kept for %s (%v)", key, ttl, w.bounds.ttl)
}
if w.finds.raised[KindGateUnanswered+" "+h.Call] {
t.Fatal("raised before its bound ended")
}
w.bounds.expire(key)
if !w.finds.raised[KindGateUnanswered+" "+h.Call] {
t.Errorf("not raised on its bound's end: %v", w.finds.raised)
}
w.router.choose(h.Asked, OptionDecline, now)
w.g.Decided(h.Asked)
if w.finds.raised[KindGateUnanswered+" "+h.Call] {
t.Error("not cleared once its end is said")
}
})
t.Run("a call whose bound ended while the holder was away is judged at its start", func(t *testing.T) {
w := setup()
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
w.bounds.watched = nil
w.bounds.expire("unanswered." + h.Call) // the bus ended it with nobody watching
now = now.Add(AskLasts + time.Minute)
defer func() { now = now.Add(-AskLasts - time.Minute) }()
again := newGate(w.state, w.router, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
if err := again.StartHolding(); err != nil || !w.finds.raised[KindGateUnanswered+" "+h.Call] {
t.Errorf("start: %v, raised %v", err, w.finds.raised)
}
})
t.Run("two instances of the holder raise one ask and run once", func(t *testing.T) {
// A-255: a process holder's act names no node.
w := setup()
other := New(Deps{Node: "n2", Runner: "holder", Holding: true, Build: "fp1", State: w.state, Channel: w.router,
Findings: w.finds, Bounds: w.bounds, Now: func() time.Time { return now }, Go: func(f func()) { f() },
Say: func(subject string, body any, id string) error { w.said[subject] = body.(GatedEnded); return nil }})
other.Serves("wiper", "wipe", func(string, json.RawMessage) (any, error) { *w.ran++; return "wiped", nil })
_, h1, _ := hold(t, w, call("n1.a", `{}`), true)
_, h2, err := other.Hold(call("n2.b", `{}`), true, nil)
if err != nil || h2 == nil || h2.Asked != h1.Asked || len(w.router.asks) != 1 {
t.Fatalf("second instance: %+v, %v; asks %d", h2, err, len(w.router.asks))
}
if strings.Contains(w.router.asks[0].About, "n1") {
t.Errorf("the ask binds %s", w.router.asks[0].About)
}
var first WaitingCall
e, _ := w.state.Get("call." + h1.Call)
_ = json.Unmarshal(e.Value, &first)
if _, ok := first.Act["node"]; ok {
t.Errorf("the act names a node: %v", first.Act)
}
w.router.choose(h1.Asked, OptionApprove, now)
w.g.Decided(h1.Asked)
other.Decided(h1.Asked)
if *w.ran != 1 {
t.Errorf("ran %d", *w.ran)
}
})
t.Run("a state that cannot be read refuses every marked call", func(t *testing.T) {
w := setup()
w.state.fail = fmt.Errorf("the bus is away")
if _, h, err := hold(t, w, call("n1.a", `{}`), true); err == nil || h != nil || *w.ran != 0 {
t.Fatalf("held %+v, %v", h, err)
}
})
}
go/approval/marks.go
-172
View File
@@ -1,172 +0,0 @@
// Package approval is the one gate on a call that needs the operator's approval (design 46 7-approval-mark; design 47 G
// §677, lead answer A-224): the tool runner's and a module's process's seat server's. It holds the marks it reads from the
// catalog half, the act it digests, the ask it raises on the seat operator-channel under the marked module's name, the judging of the same call again on the router's record, the claim, the recording in the
// hand-act log, and the refusals when a part of the check is missing (§5, §8 to §20). No module's name is written here.
package approval
import (
"encoding/json"
"fmt"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// Catalog is what the marks are read from: the catalog half the runner follows (design 46 7-approval-mark §5, design 47 G
// §196, §443) and the seat definitions it carries (§2).
type Catalog interface {
// Ready is nil once the catalog half's first view is read, else says what it waits for.
Ready() error
// Revision is the catalog revision the marks are read at, as the runner names it.
Revision() string
// Served is the entry of the build a bundle serves, by the fingerprint its composed environment carries; false when
// none is read (not yet, or not found), with why.
Served(fingerprint string) (Entry, bool, string)
// Newest is the module's entry at the newest revision; false when the catalog half lists none.
Newest(module string) (Entry, bool)
// Definitions are the seat definitions the catalog half carries, by seat.
Definitions() map[string]seatdef.Definition
// Entries are the newest entry of every module in every followed branch's half, a marked wait's marks (lead answer
// A-168).
Entries() []Entry
}
// Marks is what one module's calls are gated by: its marked tools and seat verbs, and the subjects of its marked events and
// restricted streams, each with its source (§18: its module's entry in the catalog half, or its seat's definition).
type Marks struct {
Module string
// Tools are its marked tools by name, Verbs its marked seat verbs as `<seat>.<verb>`, Subjects the bus subjects of its
// marked events and restricted streams (design 34 every-event §33); each to its source.
Tools, Verbs, Subjects map[string]string
// Revision is the catalog revision read; Served the served entry's fingerprint, empty when it could not be read.
Revision, Served string
// Unread says why the served entry could not be read: every marked call is then refused (§5, §19).
Unread string
}
// Marked says whether a tool or a seat verb of the module needs the operator's approval, and its source.
func (m Marks) Marked(seat, name string) (bool, string) {
if seat == "" {
src, ok := m.Tools[name]
return ok, src
}
src, ok := m.Verbs[seat+"."+name]
return ok, src
}
// Any says the module marks anything at all.
func (m Marks) Any() bool { return len(m.Tools)+len(m.Verbs)+len(m.Subjects) > 0 }
// marked is what an entry's manifest says of its marks: `needs-approval`, its events and its streams.
type marked struct {
NeedsApproval []string `json:"needs-approval"`
Emits []string `json:"emits"`
// Streams are read for their events and whether each carries a `grant`, and no other word, so a word added to a
// stream's declaration later never hides its marks.
Streams []struct {
Events []string `json:"events"`
Grant json.RawMessage `json:"grant"`
} `json:"streams"`
}
// Derive is a module's marks from its served entry and its newest one, unioned while they differ so a mark is never lost
// before or after the module moves (§5, ADR 0286 §3), and the seat definitions' verb marks for every verb an entry serves,
// which no holder unmarks (§2, §3). fingerprint is the build its bundle serves; empty, or an entry not read, leaves the
// marks of the newest entry and the seats, with Unread saying why.
func Derive(c Catalog, module, fingerprint string) Marks {
m := Marks{Module: module, Tools: map[string]string{}, Verbs: map[string]string{}, Subjects: map[string]string{},
Revision: c.Revision()}
defs := c.Definitions()
var entries []Entry
if fingerprint == "" {
m.Unread = "its bundle's environment carries no fingerprint, so the build it serves cannot be read"
} else if e, ok, why := c.Served(fingerprint); ok {
entries, m.Served = append(entries, e), fingerprint
} else {
m.Unread = fmt.Sprintf("the catalog half's entry of %s cannot be read: %s", fingerprint, why)
}
if e, ok := c.Newest(module); ok && (len(entries) == 0 || e.Fingerprint != entries[0].Fingerprint) {
entries = append(entries, e)
}
for _, e := range entries {
m.add(e, defs)
}
return m
}
// add takes one entry's marks into m: its `needs-approval` (a tool, a verb it serves or an event it declares), the seat
// definitions' marks of every verb it serves, and every stream it declares with a `grant`, each with its source.
func (m *Marks) add(e Entry, defs map[string]seatdef.Definition) {
source := "entry " + e.Module + " " + orNone(e.Fingerprint)
var said marked
_ = json.Unmarshal(e.Manifest, &said)
serves := map[string]bool{}
for _, sv := range e.Serves {
serves[sv] = true
seat, verb, _ := strings.Cut(sv, ".")
for _, v := range defs[seat].Verbs {
if v.Name == verb && v.NeedsApproval {
m.Verbs[sv] = "seat " + seat
}
}
}
events := map[string]bool{}
for _, ev := range said.Emits {
events[ev] = true
}
for _, s := range said.Streams {
for _, ev := range s.Events {
events[ev] = true
}
}
for _, name := range said.NeedsApproval {
switch {
case serves[name]:
if _, bySeat := m.Verbs[name]; !bySeat {
m.Verbs[name] = source
}
case events[name]:
for _, s := range eventSubjects(e.Module, name) {
m.Subjects[s] = source
}
default:
m.Tools[name] = source
}
}
for _, s := range said.Streams {
if len(s.Grant) == 0 || string(s.Grant) == "null" {
continue
}
wire := make([]string, 0, len(s.Events))
for _, ev := range s.Events {
wire = append(wire, buscompose.WirePattern(ev))
}
for _, sub := range (buscompose.ModuleStream{Module: e.Module, Events: wire}).Subjects() {
m.Subjects[sub] = source
if !strings.HasSuffix(sub, ">") {
m.Subjects[sub+".>"] = source
}
}
}
}
// eventSubjects are a marked event's subjects on the bus (design 34 every-event §33): said on the module's events and on its
// own stream, each also with every subject below it.
func eventSubjects(module, event string) []string {
ev := buscompose.WirePattern(event)
out := []string{"mesh.mod." + module + ".event." + ev, buscompose.ModuleStreamSubject(module, ev)}
if !strings.HasSuffix(ev, ">") {
out = append(out, out[0]+".>", out[1]+".>")
}
sort.Strings(out)
return out
}
func orNone(s string) string {
if s == "" {
return "none"
}
return s
}
go/approval/record.go
-255
View File
@@ -1,255 +0,0 @@
package approval
import (
"encoding/json"
"errors"
"fmt"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/keys"
)
// The bounds of a run's recording (§13, §16; lead answers A-157, A-158).
const (
UnansweredAfter = 5 * time.Minute
KeptFor = 48 * time.Hour
SaidBefore = time.Hour
)
// Bounds is the runner's key-ttl state `bound` (lead answers A-157, A-158): a key created with its own time to live, which
// the bus removes when it passes, every watch hearing the removal as an expiry, so the gate's waits need no timer.
type Bounds interface {
CreateTTL(key string, value any, ttl time.Duration) error
Delete(key string) error
// Watch hands expired every key under the pattern whose time to live passed, until the stop is called.
Watch(pattern string, expired func(key string)) (stop func(), err error)
}
// Answered is a call run on the operator's approval: the tool's own answer and the line naming the approval (§16), in the
// form a process holder's `gated-ended` says it (design 47 G §677; lead answer A-158).
type Answered struct {
Answer any `json:"answer"`
Approval string `json:"approval"`
}
// The bound keys of a run (lead answer A-158; §13).
func (g *Gate) unansweredKey(id string) string { return g.d.Node + ".unanswered." + id }
func (g *Gate) unrecordedKey(id string) string { return g.d.Node + ".unrecorded." + id }
// ran is a call run on the operator's approval: answered with the tool's answer and the approval's line, and its warrant
// recorded in the hand-act log (§16).
func (g *Gate) ran(c Call, id string, rec *Record, answer any, err error) (any, error) {
channel := ""
at := ""
if w := rec.Warrant; w != nil {
channel, at = w.Channel, w.At
}
if channel == "" {
channel = strings.Join(rec.OfferedOn, ", ")
}
line := fmt.Sprintf("run on the operator's approval via %s at %s, ask %s", orNone(channel), at, id)
g.d.Logf("[approval] %s on %s %s", c.Address, g.d.Node, line)
g.bindRecording(c.Module, id)
g.d.Go(func() { g.recordRun(c.Module, id) })
if err != nil {
return nil, fmt.Errorf("%w (%s)", err, line)
}
return Answered{Answer: answer, Approval: line}, nil
}
// bindRecording keeps the run's two bound keys: one that raises `call-unanswered` when `warranted` has not answered within
// five minutes, one that says an unrecorded run an hour before its key's 48 hours end (§13, §16).
func (g *Gate) bindRecording(module, id string) {
if g.d.Bounds == nil || g.d.Holding { // a process holder's bound keys are its waiting calls' alone (A-255)
return
}
for key, ttl := range map[string]time.Duration{g.unansweredKey(id): UnansweredAfter, g.unrecordedKey(id): KeptFor - SaidBefore} {
if err := g.d.Bounds.CreateTTL(key, map[string]string{"asker": module, "ask": id}, ttl); err != nil {
g.d.Logf("[approval] the bound %s of the run of ask %s is not kept: %v", key, id, err)
}
}
}
// recordRun calls `mesh-hand-acts.warranted` for one acted ask, each try a kept call open while the ask lacks `recorded`
// (§16, A-157), and on its answer writes `recorded` and clears what waited on it. One recording per ask at a time.
func (g *Gate) recordRun(module, id string) {
if !g.recording(id, true) {
return
}
defer g.recording(id, false)
if g.d.Warranted == nil {
g.d.Logf("[approval] the run of ask %s is not recorded: the tool runner reaches no hand-act log", id)
return
}
err := g.d.Warranted(module, id, func() (bool, error) { return g.unrecorded(id) })
g.judged(id) // made again: its answer, or the bound of its key, judges it from now
if err != nil {
g.d.Logf("[approval] mesh-hand-acts.warranted has not answered for ask %s (%v): asked again when the hand-act seat's "+
"holder is heard again and at the next start", id, err)
return
}
if err := g.markRecorded(id); err != nil {
g.d.Logf("[approval] ask %s is recorded in the hand-act log and its mark recorded is not kept: %v", id, err)
return
}
if g.d.Bounds != nil {
_ = g.d.Bounds.Delete(g.unansweredKey(id))
_ = g.d.Bounds.Delete(g.unrecordedKey(id))
}
g.finding(id, KindCallUnanswered, false, "mesh-hand-acts.warranted answered for ask "+id, "")
}
// recording claims or releases an ask's recording: false when one runs already.
func (g *Gate) recording(id string, start bool) bool {
g.mu.Lock()
defer g.mu.Unlock()
if g.inFlight == nil {
g.inFlight = map[string]bool{}
}
if start {
if g.inFlight[id] {
return false
}
g.inFlight[id] = true
return true
}
delete(g.inFlight, id)
return true
}
// kept is the ask the client keeps under its id, with its revision; nil when there is none.
func (g *Gate) kept(id string) (*kept, uint64, error) {
e, err := g.d.State.Get(g.askKey(id))
if err != nil || e == nil {
return nil, 0, err
}
var k kept
if err := json.Unmarshal(e.Value, &k); err != nil {
return nil, 0, err
}
return &k, e.Revision, nil
}
// unrecorded says the ask is acted on and its run not yet recorded: the key that keeps `warranted` open (§16).
func (g *Gate) unrecorded(id string) (bool, error) {
k, _, err := g.kept(id)
if err != nil || k == nil {
return false, err
}
return k.Acted && !k.Recorded, nil
}
// markRecorded writes `recorded` on the kept ask by compare-and-set, again when another write came first.
func (g *Gate) markRecorded(id string) error {
for {
k, rev, err := g.kept(id)
if err != nil || k == nil {
return fmt.Errorf("the kept ask cannot be read: %v", err)
}
if k.Recorded {
return nil
}
k.Recorded = true
if _, err := g.d.State.Update(g.askKey(id), k, rev); err == nil {
return nil
} else if !isChanged(err) {
return err
}
}
}
// Recover calls `warranted` again for every acted, unrecorded ask this machine's keys hold within their 48 hours (§16): at
// each start, and when the hand-act seat's holder is heard again (A-157).
func (g *Gate) Recover() {
if g.d.State == nil {
return
}
keys, err := g.d.State.Keys()
if err != nil {
g.d.Logf("[approval] the acted, unrecorded asks cannot be read (%v): read again at the next start or holder heard", err)
return
}
prefix := g.prefix() + "ask."
for _, key := range keys {
if !strings.HasPrefix(key, prefix) {
continue
}
id := strings.TrimPrefix(key, prefix)
k, _, err := g.kept(id)
if err != nil || k == nil || !k.Acted || k.Recorded || g.d.Now().Sub(k.Asked) > KeptFor {
continue
}
module := moduleOf(k)
g.pend(id)
g.d.Go(func() { g.recordRun(module, id) })
}
}
// moduleOf is the module an ask was asked under: the warrant's asker.
func moduleOf(k *kept) string {
var w struct {
Asker string `json:"asker"`
}
if k.Answer != nil {
_ = json.Unmarshal(*k.Answer, &w)
}
return w.Asker
}
// WatchBounds hears the run's bound keys expire (A-157): `call-unanswered` raised when `warranted` has not answered within
// five minutes; an unrecorded run said in the log an hour before its key's 48 hours end (§13).
func (g *Gate) WatchBounds() (func(), error) {
if g.d.Bounds == nil {
return func() {}, nil
}
return g.d.Bounds.Watch(g.d.Node+".**", func(key string) {
if key == g.unreadableKey() {
g.unreadableExpired()
return
}
rest := strings.TrimPrefix(key, g.d.Node+".")
kind, id, _ := strings.Cut(rest, ".")
if open, err := g.unrecorded(id); err != nil || !open {
return
}
switch kind {
case "unanswered":
g.finding(id, KindCallUnanswered, true, fmt.Sprintf("mesh-hand-acts.warranted has not answered for ask %s, run on the "+
"operator's approval on %s, within %s: the run is not yet in the hand-act log", id, g.d.Node, UnansweredAfter),
"a run's hand-act is unrecorded on "+g.d.Node)
case "unrecorded":
g.d.Logf("[approval] ERROR the run of ask %s on %s is still unrecorded an hour before its key's 48 hours end", id, g.d.Node)
}
})
}
// Pender is the held writer's pending (views.HeldWriter's Pend and Judged), by a finding's kind and id: the findings it
// cannot judge yet since its start, named in its `findings-stated` (design 47 G §384).
type Pender interface {
Pend(kind, id string, since time.Time, bound time.Duration)
Judged(kind, id string)
}
// pend names an acted, unrecorded ask's `call-unanswered` in the runner's `findings-stated` until its `warranted` is made
// again (§16; design 47 G §384), its bound the five minutes the finding waits.
func (g *Gate) pend(id string) {
if p, ok := g.d.Findings.(Pender); ok {
p.Pend(KindCallUnanswered, g.findingID(id), g.d.Now().UTC(), UnansweredAfter)
}
}
// judged ends that pending.
func (g *Gate) judged(id string) {
if p, ok := g.d.Findings.(Pender); ok {
p.Judged(KindCallUnanswered, g.findingID(id))
}
}
// findingID is the id of a finding about what on this machine, `<node>.<what>` escaped as a key's name is (lead answer
// A-154).
func (g *Gate) findingID(what string) string { return keys.EscapeName(g.d.Node + "." + what) }
// isChanged says a compare-and-set lost to another write.
func isChanged(err error) bool { return errors.Is(err, stdio.ErrChanged) }
go/approval/run.go
-176
View File
@@ -1,176 +0,0 @@
package approval
import (
"encoding/json"
"fmt"
"strings"
"git.novox.be/novox/mesh-sdk/go/operatorask"
)
// Run is a call through the gate (§20): an unmarked call runs; a marked one runs only on the operator's approval of that
// one call, judged on the router's record and claimed once (§14, §15), and is otherwise answered as an error saying why
// (§12, §17, §19). run is called at most once.
func (g *Gate) Run(c Call, run func() (any, error)) (any, error) {
m, err := g.marksOf(c)
if err != nil {
return nil, err
}
if missing := g.unservedOf(c.Module); len(missing) > 0 {
return nil, fmt.Errorf("Not run: %s's marks name %s, which its bundle on %s does not register, so none of its tools is "+
"served here (design 46 7-approval-mark §4)", c.Module, strings.Join(missing, ", "), g.d.Node)
}
marked, _ := m.Marked(c.Seat, c.Name)
since := g.markedSince(c, marked, m.Revision)
if !marked {
return run()
}
if c.Own {
return nil, g.refused(c, "", "it is one of the tool runner's own tools, which no ask covers")
}
if m.Unread != "" {
return nil, g.refused(c, "", m.Unread)
}
if b, ok := g.d.State.(Binding); g.d.State == nil || ok && !b.Bound() {
return nil, g.refused(c, "", "the tool runner's membership binds no state "+StateName+" to keep its asks in")
}
if why := refusedUnasked(c.Args); why != "" {
return nil, g.refused(c, "", why)
}
if why := g.held(); why != "" {
return nil, g.refused(c, "", why)
}
act, err := ActOf(c, g.d.Node)
if err != nil {
return nil, g.refused(c, "", err.Error())
}
digest, err := operatorask.ActDigest(act)
if err != nil {
return nil, g.refused(c, "", err.Error())
}
named, err := g.d.State.Get(g.actKey(digest))
if err != nil {
g.stateFailed(err)
return nil, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
}
g.stateRead()
if named == nil {
return nil, g.raise(c, act, digest, since)
}
var id string
if err := json.Unmarshal(named.Value, &id); err != nil || id == "" {
return nil, g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName))
}
return g.judge(c, act, digest, id, since, run)
}
// Binding is a Store that says whether the runner's membership binds the state now (§13).
type Binding interface{ Bound() bool }
// marksOf is the module's marks as the catalog half has them now; before its first view, the call waits for it within
// RecordWithin and is then judged, refused naming mesh_first_views when it is still unread (§19).
func (g *Gate) marksOf(c Call) (Marks, error) {
if g.d.Catalog == nil {
return Marks{}, g.refused(c, "", "the tool runner reads no catalog half, so its marks are not read")
}
if err := g.d.Catalog.Ready(); err != nil {
if g.d.Wait != nil {
err = g.d.Wait(RecordWithin)
}
if err != nil {
return Marks{}, fmt.Errorf("Not run: the marks of %s are not read yet (%v); mesh_first_views says what the tool runner "+
"waits for", c.Module, err)
}
}
m := Derive(g.d.Catalog, c.Module, c.Fingerprint)
g.checkServed(m)
return m, nil
}
func (g *Gate) unservedOf(module string) []string {
g.mu.Lock()
defer g.mu.Unlock()
return g.unserved[module]
}
// markedSince is the revision a call was marked at when the same caller last called it unmarked (§12), empty otherwise.
func (g *Gate) markedSince(c Call, marked bool, revision string) string {
key := c.Caller + " " + c.Address
g.mu.Lock()
defer g.mu.Unlock()
if g.unmarkedAt == nil {
g.unmarkedAt = map[string]bool{}
}
was := g.unmarkedAt[key]
if marked {
delete(g.unmarkedAt, key)
if was {
return revision
}
return ""
}
g.unmarkedAt[key] = true
return ""
}
// judge is the same call made again, judged on the router's own record of its act's newest ask (§14 to §17).
func (g *Gate) judge(c Call, act operatorask.Act, digest, id, since string, run func() (any, error)) (any, error) {
rec, err := g.record(c.Module, id)
if err != nil {
return nil, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error())
}
if rec == nil {
return nil, g.firstAnswer(c, id, digest, since) // §17: not yet taken by the router
}
again := func(why string) (any, error) {
err := g.raise(c, act, digest, since)
return nil, fmt.Errorf("The ask %s %s; asked again. %w", id, why, err)
}
switch rec.State {
case "open", "held-back":
return nil, fmt.Errorf("Not run: still waiting for the operator's answer to ask %s (fingerprint %s); it expires at %s",
id, Fingerprint(digest), rec.Expires)
case "refused":
return nil, g.refused(c, digest, "the router refused its ask "+id+": "+orNone(rec.Answer))
case "chosen":
default: // expired, cancelled, replaced: each asks again (§15, §17)
return again("ended " + rec.State)
}
w := rec.Warrant
if w == nil {
return nil, g.refused(c, digest, "the router's record of its ask "+id+" is chosen and carries no warrant")
}
at, _ := parseTime(w.At)
if w.Option == OptionDecline {
if until := at.Add(DeclinedFor); g.d.Now().Before(until) {
return nil, fmt.Errorf("Not run: the operator declined ask %s; the same call is not asked again until %s",
id, formatTime(until))
}
return again("was declined more than 10 minutes ago")
}
if g.d.Now().After(at.Add(RunWithin)) {
return again("was approved too long ago (more than 30 minutes)")
}
var answer any
var ran error
h, err := g.d.Channel.Claim(c.Module, *w, act, func() { answer, ran = run() })
switch {
case h == ClaimDone:
return again("was already run")
case h == ClaimRefused:
return nil, g.refused(c, digest, "its warrant does not hold: "+errText(err))
case h == "" && err != nil:
g.stateFailed(err)
return nil, g.refused(c, digest, "its state "+StateName+" cannot be written: "+err.Error())
case err != nil: // the act about to run is not the one the option bound: nothing runs, the warrant stays used
return nil, g.refused(c, digest, err.Error())
}
return g.ran(c, id, rec, answer, ran)
}
func errText(err error) string {
if err == nil {
return "no reason given"
}
return err.Error()
}
go/approval/tools.go
-172
View File
@@ -1,172 +0,0 @@
package approval
import (
"encoding/json"
"fmt"
"sort"
"strings"
"time"
)
// The mesh MCP server's mesh_approvals reads and steers what the gate asked on this machine (§18).
// Listed is one ask of the last day as mesh_approvals lists it: its state and its act's fingerprint.
type Listed struct {
Ask string `json:"ask"`
Address string `json:"address"`
Module string `json:"module"`
Asked time.Time `json:"asked"`
State string `json:"state"`
Fingerprint string `json:"fingerprint"`
}
// Whole is one ask read whole: its arguments and the act's digest.
type Whole struct {
Listed
Digest string `json:"digest"`
Act map[string]string `json:"act"`
Arguments json.RawMessage `json:"arguments"`
Recorded bool `json:"recorded"`
}
// stateOf is a kept ask's state as the runner knows it.
func stateOf(acted, recorded bool, ended string) string {
switch {
case acted && recorded:
return "run, recorded"
case acted:
return "run, not yet recorded"
case ended != "":
return ended
}
return "asked"
}
// List is the calls asked on this machine in the last day, newest first (§18).
func (g *Gate) List() ([]Listed, error) {
if g.d.State == nil {
return nil, fmt.Errorf("the tool runner's membership binds no state %s", StateName)
}
keys, err := g.d.State.Keys()
if err != nil {
return nil, err
}
prefix := g.d.Node + ".ask."
out := []Listed{}
for _, key := range keys {
if !strings.HasPrefix(key, prefix) {
continue
}
w, err := g.Read(strings.TrimPrefix(key, prefix))
if err != nil || g.d.Now().Sub(w.Asked) > 24*time.Hour {
continue
}
out = append(out, w.Listed)
}
sort.Slice(out, func(i, j int) bool { return out[i].Asked.After(out[j].Asked) })
return out, nil
}
// Read is one ask whole: its arguments and the act's digest (§18).
func (g *Gate) Read(id string) (Whole, error) {
if g.d.State == nil {
return Whole{}, fmt.Errorf("the tool runner's membership binds no state %s", StateName)
}
k, _, err := g.kept(id)
if err != nil {
return Whole{}, err
}
if k == nil {
return Whole{}, fmt.Errorf("no ask %s is kept on %s", id, g.d.Node)
}
module := k.Act["module"]
if module == "" {
module = moduleOf(k)
}
var ask struct {
About string `json:"about"`
}
_ = json.Unmarshal(k.Ask, &ask)
return Whole{Listed: Listed{Ask: id, Address: k.Act["address"], Module: module, Asked: k.Asked,
State: stateOf(k.Acted, k.Recorded, k.Ended), Fingerprint: Fingerprint(ask.About)},
Digest: ask.About, Act: k.Act, Arguments: k.Arguments, Recorded: k.Recorded}, nil
}
// Applied are the marks the gate applies on this machine, each module's marked tools and verbs with their source and the
// catalog revision read (§18).
func (g *Gate) Applied() []Marks {
g.mu.Lock()
modules := make([]string, 0, len(g.served))
for m := range g.served {
modules = append(modules, m)
}
g.mu.Unlock()
sort.Strings(modules)
var out []Marks
if g.d.Catalog == nil {
return out
}
for _, m := range modules {
if marks := Derive(g.d.Catalog, m, g.servedFingerprint(m)); marks.Any() {
out = append(out, marks)
}
}
return out
}
func (g *Gate) servedFingerprint(module string) string {
g.mu.Lock()
defer g.mu.Unlock()
return g.served[module]
}
// Describe says whether a tool or a seat verb of a module is marked, and its source, for mesh_describe (§18): the served
// build's marks where this machine serves the module, else the newest entry's and its seat's.
func (g *Gate) Describe(module, seat, name string) (bool, string) {
if g.d.Catalog == nil || g.d.Catalog.Ready() != nil {
return false, ""
}
if seat != "" && module == "" {
for s, d := range g.d.Catalog.Definitions() {
if s != seat {
continue
}
for _, v := range d.Verbs {
if v.Name == name && v.NeedsApproval {
return true, "seat " + seat
}
}
}
return false, ""
}
return Derive(g.d.Catalog, module, g.servedFingerprint(module)).Marked(seat, name)
}
// Again asks `warranted` again for one acted, unrecorded ask, by hand, in the kept-call form of §16 (§18); the caller
// records the hand act first.
func (g *Gate) Again(id string) error {
w, err := g.Read(id)
if err != nil {
return err
}
if !strings.HasPrefix(w.State, "run, not yet") {
return fmt.Errorf("the ask %s is %s: only a run not yet recorded is asked again", id, w.State)
}
g.recordRun(w.Module, id)
if again, _ := g.Read(id); !again.Recorded {
return fmt.Errorf("mesh-hand-acts.warranted has not answered for ask %s yet", id)
}
return nil
}
// Cancel takes an open ask back through operator-channel's client (§18); the caller records the hand act first.
func (g *Gate) Cancel(id string) error {
w, err := g.Read(id)
if err != nil {
return err
}
if w.State != "asked" {
return fmt.Errorf("the ask %s is %s, not open", id, w.State)
}
return g.d.Channel.Cancel(w.Module, id)
}
go/approval/wait.go
-290
View File
@@ -1,290 +0,0 @@
package approval
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/operatorask"
"git.novox.be/novox/mesh-sdk/go/outward"
)
// Wait is one mesh_wait call as the gate judges it (design 34 every-event §34 to §36): the event names given, the subjects
// they became (after `from`), the filter and the timeout.
type Wait struct {
Names []string
From string
Subjects []string
Filter map[string]any
Timeout time.Duration
}
// Waited is a wait through the gate (every-event §34 to §36). An unmarked wait is heard at once by hear, given how long it
// may wait. A marked one is heard only on the operator's warrant for that one wait, its timeout running from then; with
// none it subscribes to nothing, its ask raised once under the runner's own module's name, and is answered as an error at
// its timeout, when its caller goes, or as soon as the router's record says the ask was chosen, never in a hot loop. c is
// the call as the console makes it: its address, its tool's name, its caller and its arguments as given.
func (g *Gate) Waited(ctx context.Context, c Call, w Wait, hear func(within time.Duration) (any, error)) (any, error) {
deadline := g.d.Now().Add(w.Timeout)
// A wait from a seat is never marked, and a seat's event never is (§32, §34).
if w.From == "seat" || !outsideSeats(w.Subjects) {
return hear(w.Timeout)
}
if g.d.Catalog == nil {
return nil, g.holdRefused(ctx, deadline, "this wait is refused: the tool runner reads no catalog half, so the marks are not read")
}
if err := g.d.Catalog.Ready(); err != nil {
// Before the marks are read nothing is asked: the wait waits for the first view within its own timeout and is
// then judged, and is refused at its timeout when it does not come (§34).
if g.d.Wait != nil {
err = g.waitCatalog(ctx, w.Timeout)
}
if err != nil {
return nil, g.holdRefused(ctx, deadline, fmt.Sprintf("this wait is refused at its timeout: the marks are not yet read (%v); "+
"mesh_first_views says what the tool runner waits for", err))
}
}
if !overlapsAny(w.Subjects, g.waitMarks()) {
return hear(deadline.Sub(g.d.Now()))
}
c.Module, c.Seat, c.Fingerprint = g.d.Runner, "", g.d.Build
if c.Fingerprint == "" {
return nil, g.holdRefused(ctx, deadline, "this wait needs the operator's approval, and the tool runner's own fingerprint is "+
"unknown (its MESH_FINGERPRINT was not set at its start), so no warrant can bind its build")
}
if b, ok := g.d.State.(Binding); g.d.State == nil || ok && !b.Bound() {
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", "the tool runner's membership binds no state "+StateName+" to keep its asks in").Error())
}
if why := refusedUnasked(c.Args); why != "" {
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", why).Error())
}
if why := g.held(); why != "" {
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", why).Error())
}
act, err := ActOf(c, g.d.Node)
if err != nil {
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", err.Error()).Error())
}
digest, err := operatorask.ActDigest(act)
if err != nil {
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", err.Error()).Error())
}
named, err := g.d.State.Get(g.actKey(digest))
if err != nil {
g.stateFailed(err)
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error()).Error())
}
g.stateRead()
anew := func() (any, error) {
id := newID()
if err := g.keepAsk(c, act, digest, g.askForWait(c, w, digest, id)); err != nil {
return nil, g.holdRefused(ctx, deadline, err.Error())
}
return nil, g.holdAsk(ctx, deadline, c.Module, id, digest, true)
}
if named == nil {
return anew()
}
var id string
if err := json.Unmarshal(named.Value, &id); err != nil || id == "" {
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName)).Error())
}
rec, err := g.record(c.Module, id)
switch {
case err != nil:
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error()).Error())
case rec == nil, rec.State == "open", rec.State == "held-back":
return nil, g.holdAsk(ctx, deadline, c.Module, id, digest, false)
case rec.State == "refused":
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router refused its ask "+id+": "+orNone(rec.Answer)).Error())
case rec.State != "chosen":
return anew() // expired, cancelled, replaced: asked again (§15, §17)
}
wt := rec.Warrant
if wt == nil {
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router's record of its ask "+id+" is chosen and carries no warrant").Error())
}
at, _ := parseTime(wt.At)
if wt.Option == OptionDecline {
if g.d.Now().Before(at.Add(DeclinedFor)) {
return nil, g.holdRefused(ctx, deadline, declinedWords(id, at))
}
return anew()
}
if g.d.Now().After(at.Add(RunWithin)) {
return anew()
}
var answer any
var ran error
// On the warrant it subscribes, and the wait's timeout runs from then (§35).
h, err := g.d.Channel.Claim(c.Module, *wt, act, func() { answer, ran = hear(w.Timeout) })
switch {
case h == ClaimDone:
return anew() // already run: one warrant is one wait (§35)
case h == ClaimRefused:
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its warrant does not hold: "+errText(err)).Error())
case h == "" && err != nil:
g.stateFailed(err)
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its state "+StateName+" cannot be written: "+err.Error()).Error())
case err != nil:
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, err.Error()).Error())
}
return g.ran(c, id, rec, answer, ran)
}
// outsideSeats says a subject is not a seat's: only such a subject can be marked (§32, §34).
func outsideSeats(subjects []string) bool {
for _, s := range subjects {
if !strings.HasPrefix(s, "mesh.seat.") {
return true
}
}
return false
}
// waitMarks are the marked subjects of every module the catalog half lists (§33): the newest entry of every followed
// branch, and the served entry of each module this runner serves (lead answer A-168), so a mark is never lost.
func (g *Gate) waitMarks() map[string]string {
m := Marks{Tools: map[string]string{}, Verbs: map[string]string{}, Subjects: map[string]string{}}
defs := g.d.Catalog.Definitions()
for _, e := range g.d.Catalog.Entries() {
m.add(e, defs)
}
g.mu.Lock()
served := make(map[string]string, len(g.served))
for module, fp := range g.served {
served[module] = fp
}
g.mu.Unlock()
for _, fp := range served {
if e, ok, _ := g.d.Catalog.Served(fp); ok {
m.add(e, defs)
}
}
return m.Subjects
}
// overlapsAny says a subject a wait subscribes overlaps a marked subject (§34).
func overlapsAny(subjects []string, marked map[string]string) bool {
for _, s := range subjects {
for mk := range marked {
if buscompose.Overlaps(s, mk) {
return true
}
}
}
return false
}
// waitCatalog waits for the catalog half's first view within the bound, or until the caller goes.
func (g *Gate) waitCatalog(ctx context.Context, within time.Duration) error {
done := make(chan error, 1)
go func() { done <- g.d.Wait(within) }()
select {
case err := <-done:
return err
case <-ctx.Done():
return ctx.Err()
}
}
// pause waits until the deadline or the caller's going.
func (g *Gate) pause(ctx context.Context, deadline time.Time) {
if left := deadline.Sub(g.d.Now()); left > 0 {
select {
case <-g.d.After(left):
case <-ctx.Done():
}
}
}
// holdRefused is a refused marked wait, said at its timeout or when its caller goes, never at once, so a watcher's loop
// is never a hot one (§36).
func (g *Gate) holdRefused(ctx context.Context, deadline time.Time, words string) error {
g.pause(ctx, deadline)
if !strings.HasPrefix(words, "Not run") {
words = "Not run: " + words
}
return fmt.Errorf("%s", words)
}
// holdAsk holds a marked wait whose ask is open, subscribing to nothing but its ask's `decided` (§36): it answers, as an
// error, at its timeout, when its caller goes, or once the router's record says the ask was chosen, the event only waking it.
func (g *Gate) holdAsk(ctx context.Context, deadline time.Time, module, id, digest string, first bool) error {
for {
left := deadline.Sub(g.d.Now())
heard := false
if left > 0 && ctx.Err() == nil {
var err error
if g.d.Hear != nil {
heard, err = g.d.Hear(ctx, []string{g.d.Channel.Decided(module)}, left, func(body json.RawMessage) bool {
var w struct {
Ask string `json:"ask"`
}
return json.Unmarshal(body, &w) == nil && w.Ask == id
})
}
if g.d.Hear == nil || err != nil {
g.pause(ctx, deadline)
}
}
rec, _ := g.record(module, id)
if heard && (rec == nil || rec.State != "chosen") {
continue // the event only wakes it: the record says nothing chosen yet
}
return g.waitWords(rec, id, digest, first)
}
}
// waitWords are §12's and §17's words for a marked wait not run, "this wait" for "the call" (§36).
func (g *Gate) waitWords(rec *Record, id, digest string, first bool) error {
switch {
case rec == nil:
return fmt.Errorf("Not run, asked; not yet taken by the router (ask %s, fingerprint %s); make this wait again later", id, Fingerprint(digest))
case rec.State == "chosen" && rec.Warrant != nil && rec.Warrant.Option == OptionDecline:
at, _ := parseTime(rec.Warrant.At)
return fmt.Errorf("%s", declinedWords(id, at))
case rec.State == "chosen", first:
return fmt.Errorf("Not run: this wait needs the operator's approval. Asked on %s at %s (ask %s, fingerprint %s); make this wait "+
"again within 30 minutes of the approval; the ask expires at %s", orNone(strings.Join(rec.OfferedOn, ", ")), rec.Opened,
id, Fingerprint(digest), rec.Expires)
case rec.State == "open", rec.State == "held-back":
return fmt.Errorf("Not run: still waiting for the operator's answer to ask %s (fingerprint %s); it expires at %s",
id, Fingerprint(digest), rec.Expires)
}
return fmt.Errorf("Not run: the ask %s ended %s; make this wait again to ask anew", id, rec.State)
}
func declinedWords(id string, at time.Time) string {
return fmt.Sprintf("Not run: the operator declined ask %s; this wait is not asked again until %s", id,
formatTime(at.Add(DeclinedFor)))
}
// askForWait is a marked wait's ask (§35): "Wait on <names> on <node>?", shortened as §10 says, and an explanation naming
// every event name one per line, the filter, and the bound as the timeout in minutes, then the act's fingerprint.
func (g *Gate) askForWait(c Call, w Wait, digest, id string) Ask {
a := g.askFor(c, digest, id)
filter, whole := "none", true
if len(w.Filter) > 0 {
raw, _ := json.Marshal(w.Filter)
filter = string(raw)
if r, ok := outward.Check(filter); !ok {
filter, whole = shapeOf(r), false
}
}
a.Headline = headlineOf("Wait on", strings.Join(w.Names, ", "), g.d.Node, whole)
lines := append([]string{"Wait on " + g.d.Node + ", called through the mesh's tools, which cannot tell who called, for:"}, w.Names...)
lines = append(lines, "Filter: "+filter, fmt.Sprintf("Timeout: %s minutes", minutes(w.Timeout)), "Fingerprint "+Fingerprint(digest))
a.Explanation = strings.Join(lines, "\n")
a.Options[0].Does = "the same wait, made again within 30 minutes, subscribes once, its timeout running from then"
a.Options[1].Does = "nothing is heard; the same wait is not asked again for 10 minutes"
return a
}
// minutes is a timeout in minutes, as few digits as say it.
func minutes(d time.Duration) string {
return strings.TrimSuffix(strings.TrimRight(fmt.Sprintf("%.2f", d.Minutes()), "0"), ".")
}
go/bus_waits_for_serve_test.go
+1 -8
View File
@@ -9,7 +9,7 @@ import (
// A bus function called before Serve waits for Serve and proceeds (issue 1966): a module that starts a goroutine // A bus function called before Serve waits for Serve and proceeds (issue 1966): a module that starts a goroutine
// before stdio.Serve and calls the bus there is no longer refused. It is refused only when the process never serves. // before stdio.Serve and calls the bus there is no longer refused. It is refused only when the process never serves.
// The test drives the wait itself: serveWait is the test's own timer channel, no row sleeps on a clock. // The test drives the wait itself: serveWait is the test's own timer channel, no row sleeps on a clock.
func subBusFunctionsWaitForServe(t *testing.T) { func TestBusFunctionsWaitForServe(t *testing.T) {
const refusal = "before Serve: a bundle reaches the bus through the runtime that launched it" const refusal = "before Serve: a bundle reaches the bus through the runtime that launched it"
for _, tc := range []struct { for _, tc := range []struct {
name string name string
@@ -72,10 +72,3 @@ func subBusFunctionsWaitForServe(t *testing.T) {
}) })
} }
} }
// TestBusFunctionsWaitForServe runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestBusFunctionsWaitForServe(t *testing.T) {
t.Run("BusFunctionsWaitForServe", subBusFunctionsWaitForServe)
t.Run("ACapabilityWordIsReadOnceAndNamedWhenNotStated", subACapabilityWordIsReadOnceAndNamedWhenNotStated)
t.Run("ABundleAsksAnAcknowledgedPublishAndASubscriptionForNewMessages", subABundleAsksAnAcknowledgedPublishAndASubscriptionForNewMessages)
}
go/busarchive/archive.go
-162
View File
@@ -1,162 +0,0 @@
package busarchive
import (
"archive/tar"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
)
// Archive is a snapshot read back: its manifest, and each file beside it unpacked into a scratch
// directory of its own, removed by Close.
type Archive struct {
Manifest Manifest
dir string
files map[string]string
}
// ReadArchive unpacks a snapshot. Only the files the manifest names are kept, and only under names
// that are one stream's directory: an archive is data, and a path in it is never trusted to say where
// on the machine a file goes.
func ReadArchive(r io.Reader, scratch string) (*Archive, error) {
dir, err := os.MkdirTemp(scratch, "mesh-nats-restore-*")
if err != nil {
return nil, err
}
a := &Archive{dir: dir, files: map[string]string{}}
tr := tar.NewReader(r)
first := true
for {
hdr, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
a.Close()
return nil, fmt.Errorf("the archive does not read: %w", err)
}
if first {
if hdr.Name != ManifestName {
a.Close()
return nil, fmt.Errorf("the archive starts with %q, not its manifest: it is not a snapshot of the bus", hdr.Name)
}
raw, err := io.ReadAll(io.LimitReader(tr, 64<<20))
if err != nil {
a.Close()
return nil, err
}
if err := json.Unmarshal(raw, &a.Manifest); err != nil {
a.Close()
return nil, fmt.Errorf("its manifest: %w", err)
}
if a.Manifest.Format != 1 {
a.Close()
return nil, fmt.Errorf("its manifest is format %d; this program reads format 1", a.Manifest.Format)
}
first = false
continue
}
if !a.named(hdr.Name) {
a.Close()
return nil, fmt.Errorf("the archive holds %q, which its manifest does not name", hdr.Name)
}
local := filepath.Join(dir, fmt.Sprintf("%04d", len(a.files)))
f, err := os.OpenFile(local, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
if err != nil {
a.Close()
return nil, err
}
_, err = io.Copy(f, tr)
f.Close()
if err != nil {
a.Close()
return nil, err
}
a.files[hdr.Name] = local
}
if first {
a.Close()
return nil, errors.New("the archive is empty")
}
for _, s := range a.Manifest.Streams {
if !ValidName(s.Name) || s.Meta != "streams/"+s.Name+"/backup.json" || s.Archive != "streams/"+s.Name+"/stream.tar.s2" {
a.Close()
return nil, fmt.Errorf("the manifest names a stream %q at paths that are not its own", s.Name)
}
}
return a, nil
}
func (a *Archive) named(name string) bool {
for _, s := range a.Manifest.Streams {
if name == s.Meta || name == s.Archive {
return true
}
}
return false
}
// Close removes the unpacked files.
func (a *Archive) Close() { os.RemoveAll(a.dir) }
// Verify holds every file to the manifest's size and checksum, and reads every stream's archive to
// its end.
func (a *Archive) Verify() error {
for _, s := range a.Manifest.Streams {
for _, f := range []struct{ name, sum string }{{s.Meta, s.MetaSHA256}, {s.Archive, s.ArchiveSHA256}} {
local, ok := a.files[f.name]
if !ok {
return fmt.Errorf("%s is named in the manifest and missing from the archive", f.name)
}
got, size, err := sha256Of(local)
if err != nil {
return err
}
if got != f.sum {
return fmt.Errorf("%s does not match the checksum its manifest gives", f.name)
}
if f.name == s.Archive && size != s.ArchiveBytes {
return fmt.Errorf("%s is %d bytes; its manifest says %d", f.name, size, s.ArchiveBytes)
}
}
af, err := os.Open(a.files[s.Archive])
if err != nil {
return err
}
err = ReadsThrough(af)
af.Close()
if err != nil {
return fmt.Errorf("%s does not read: %w", s.Archive, err)
}
}
return nil
}
func sha256Of(path string) (string, int64, error) {
f, err := os.Open(path)
if err != nil {
return "", 0, err
}
defer f.Close()
h := sha256.New()
n, err := io.Copy(h, f)
if err != nil {
return "", 0, err
}
return hex.EncodeToString(h.Sum(nil)), n, nil
}
// Open is one stream's meta and archive.
func (a *Archive) Open(s StreamEntry) (meta []byte, archive *os.File, err error) {
meta, err = os.ReadFile(a.files[s.Meta])
if err != nil {
return nil, nil, err
}
archive, err = os.Open(a.files[s.Archive])
return meta, archive, err
}
go/busarchive/types.go
-86
View File
@@ -1,86 +0,0 @@
// Package busarchive is the bus's snapshot archive: its manifest, reading one back and holding it to
// its checksums. The snapshot program (cmd/mesh-nats-snapshot) writes and restores with it and the bus's
// tool nats_snapshot_verify (cmd/nats-tools) checks with it, so the two never differ on what a good
// snapshot is.
package busarchive
import (
"archive/tar"
"io"
"strings"
"time"
"github.com/klauspost/compress/s2"
)
// Manifest is the first entry of an archive: what was taken, when, how long it took, and the checksum
// of every file beside it. It is what `verify` and `restore` hold the archive to.
type Manifest struct {
Format int `json:"format"`
Server string `json:"server_version"`
Started time.Time `json:"started"`
Finished time.Time `json:"finished"`
Seconds float64 `json:"seconds"`
// Messages and StreamBytes are the streams' own totals at their snapshots; ArchiveBytes what the
// archives take, compressed.
Messages uint64 `json:"messages"`
StreamBytes int64 `json:"stream_bytes"`
ArchiveBytes int64 `json:"archive_bytes"`
Streams []StreamEntry `json:"streams"`
// Skipped are streams that hold nothing across a restart (memory storage), said rather than
// silently left out.
Skipped []Skipped `json:"skipped,omitempty"`
}
// StreamEntry is one stream as it was snapshotted.
type StreamEntry struct {
Name string `json:"name"`
Subjects []string `json:"subjects,omitempty"`
Retention string `json:"retention,omitempty"`
Messages uint64 `json:"messages"`
Bytes uint64 `json:"bytes"`
FirstSeq uint64 `json:"first_seq"`
LastSeq uint64 `json:"last_seq"`
Deleted int `json:"deleted,omitempty"`
Consumers int `json:"consumers"`
// Meta is the server's own description of the stream (its configuration and state), in the
// shape the restore API and the nats CLI take back: backup.json.
Meta string `json:"meta"`
MetaSHA256 string `json:"meta_sha256"`
// Archive is the server's snapshot of the stream, as the server sent it: stream.tar.s2.
Archive string `json:"archive"`
ArchiveBytes int64 `json:"archive_bytes"`
ArchiveSHA256 string `json:"archive_sha256"`
Seconds float64 `json:"seconds"`
}
// Skipped is a stream not snapshotted, and why.
type Skipped struct {
Name string `json:"name"`
Why string `json:"why"`
}
// ManifestName is the archive's first entry.
const ManifestName = "manifest.json"
// ReadsThrough is whether a stream's archive decompresses and lists to its end.
func ReadsThrough(r io.Reader) error {
tr := tar.NewReader(s2.NewReader(r))
for {
_, err := tr.Next()
if err == io.EOF {
return nil
}
if err != nil {
return err
}
if _, err := io.Copy(io.Discard, tr); err != nil {
return err
}
}
}
// ValidName is a stream name that is one safe path component.
func ValidName(name string) bool {
return name != "" && name != "." && name != ".." && !strings.ContainsAny(name, "/\\\x00 *>")
}
go/busarchive/verifyfile.go
-109
View File
@@ -1,109 +0,0 @@
package busarchive
import (
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"strings"
"time"
)
// StreamReport is one stream of a verified snapshot.
type StreamReport struct {
Name string `json:"name"`
LastSeq uint64 `json:"last_seq"`
}
// Report is the answer of VerifyFile: which file was held to its manifest, and whether it passed.
type Report struct {
File string `json:"file"`
Modified time.Time `json:"modified"`
Size int64 `json:"size"`
Taken *time.Time `json:"taken,omitempty"`
Streams []StreamReport `json:"streams"`
Skipped []Skipped `json:"skipped,omitempty"`
Passed bool `json:"passed"`
// Fault is the first thing wrong, empty when the snapshot passed.
Fault string `json:"fault,omitempty"`
}
// VerifyFile verifies one snapshot inside dir: the file called name, or the newest *.tar when name is
// empty. Nothing outside dir is read and nothing is written there; a snapshot that fails is an answer
// (Passed false, the first Fault), and only a directory without a snapshot is an error.
func VerifyFile(dir, name string) (*Report, error) {
path, info, err := pick(dir, name)
if err != nil {
return nil, err
}
r := &Report{File: filepath.Base(path), Modified: info.ModTime().UTC(), Size: info.Size(), Streams: []StreamReport{}}
f, err := os.Open(path)
if err != nil {
r.Fault = err.Error()
return r, nil
}
defer f.Close()
a, err := ReadArchive(f, "")
if err != nil {
r.Fault = err.Error()
return r, nil
}
defer a.Close()
taken := a.Manifest.Finished
r.Taken = &taken
r.Skipped = a.Manifest.Skipped
for _, s := range a.Manifest.Streams {
r.Streams = append(r.Streams, StreamReport{Name: s.Name, LastSeq: s.LastSeq})
}
if err := a.Verify(); err != nil {
r.Fault = err.Error()
return r, nil
}
r.Passed = true
return r, nil
}
// pick names the regular file to verify, and nothing outside dir: a name is one path component, and a
// symbolic link is not followed out of the directory.
func pick(dir, name string) (string, fs.FileInfo, error) {
if dir == "" {
return "", nil, errors.New("the snapshots directory is not set")
}
if name != "" {
if name != filepath.Base(name) || name == "." || name == ".." || strings.ContainsRune(name, 0) {
return "", nil, fmt.Errorf("%q is not a file name inside the snapshots directory", name)
}
p := filepath.Join(dir, name)
info, err := os.Lstat(p)
if err != nil {
return "", nil, fmt.Errorf("no snapshot %s in the snapshots directory: %w", name, err)
}
if !info.Mode().IsRegular() {
return "", nil, fmt.Errorf("%s is not a regular file", name)
}
return p, info, nil
}
entries, err := os.ReadDir(dir)
if err != nil {
return "", nil, fmt.Errorf("the snapshots directory does not read: %w", err)
}
var best string
var bestInfo fs.FileInfo
for _, e := range entries {
if !strings.HasSuffix(e.Name(), ".tar") {
continue
}
info, err := e.Info()
if err != nil || !info.Mode().IsRegular() {
continue
}
if bestInfo == nil || info.ModTime().After(bestInfo.ModTime()) {
best, bestInfo = e.Name(), info
}
}
if bestInfo == nil {
return "", nil, errors.New("the snapshots directory holds no .tar snapshot")
}
return filepath.Join(dir, best), bestInfo, nil
}
go/busarchive/verifyfile_test.go
-182
View File
@@ -1,182 +0,0 @@
package busarchive
import (
"archive/tar"
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/klauspost/compress/s2"
)
func sum(b []byte) string { h := sha256.Sum256(b); return hex.EncodeToString(h[:]) }
// streamArchive is a stream's archive as the server sends it: a tar of one file, s2-compressed.
func streamArchive(t *testing.T) []byte {
t.Helper()
var raw bytes.Buffer
tw := tar.NewWriter(&raw)
body := []byte(strings.Repeat("a message of the stream\n", 50))
if err := tw.WriteHeader(&tar.Header{Name: "msgs.blk", Mode: 0o600, Size: int64(len(body))}); err != nil {
t.Fatal(err)
}
tw.Write(body)
tw.Close()
var out bytes.Buffer
w := s2.NewWriter(&out)
w.Write(raw.Bytes())
w.Close()
return out.Bytes()
}
// snapshotTar builds a snapshot of two streams; the mutations change what the manifest says or what is
// stored, so each fault is made in the one place it is found.
func snapshotTar(t *testing.T, mutate func(m *Manifest, files map[string][]byte)) []byte {
t.Helper()
files := map[string][]byte{}
m := Manifest{Format: 1, Server: "2.11.17", Finished: time.Date(2026, 10, 22, 3, 0, 0, 0, time.UTC)}
for i, name := range []string{"EVENTS", "KV_conditions"} {
meta := []byte(`{"config":{"name":"` + name + `"}}`)
arc := streamArchive(t)
e := StreamEntry{Name: name, LastSeq: uint64(10 * (i + 1)),
Meta: "streams/" + name + "/backup.json", MetaSHA256: sum(meta),
Archive: "streams/" + name + "/stream.tar.s2", ArchiveBytes: int64(len(arc)), ArchiveSHA256: sum(arc)}
m.Streams = append(m.Streams, e)
files[e.Meta], files[e.Archive] = meta, arc
}
if mutate != nil {
mutate(&m, files)
}
var out bytes.Buffer
tw := tar.NewWriter(&out)
put := func(name string, b []byte) {
if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0o600, Size: int64(len(b))}); err != nil {
t.Fatal(err)
}
tw.Write(b)
}
raw, _ := json.Marshal(m)
put(ManifestName, raw)
for _, s := range m.Streams {
put(s.Meta, files[s.Meta])
put(s.Archive, files[s.Archive])
}
tw.Close()
return out.Bytes()
}
func put(t *testing.T, dir, name string, b []byte, at time.Time) {
t.Helper()
p := filepath.Join(dir, name)
if err := os.WriteFile(p, b, 0o600); err != nil {
t.Fatal(err)
}
os.Chtimes(p, at, at)
}
func subAGoodSnapshotPassesAndSaysItsStreams(t *testing.T) {
dir := t.TempDir()
good := snapshotTar(t, nil)
put(t, dir, "bus.tar", good, time.Now())
r, err := VerifyFile(dir, "")
if err != nil {
t.Fatal(err)
}
if !r.Passed || r.Fault != "" || r.File != "bus.tar" || r.Size != int64(len(good)) {
t.Fatalf("%+v", r)
}
if r.Taken == nil || !r.Taken.Equal(time.Date(2026, 10, 22, 3, 0, 0, 0, time.UTC)) {
t.Errorf("taken %v", r.Taken)
}
if len(r.Streams) != 2 || r.Streams[0] != (StreamReport{"EVENTS", 10}) || r.Streams[1] != (StreamReport{"KV_conditions", 20}) {
t.Errorf("streams %+v", r.Streams)
}
}
func subACorruptedChecksumFailsNamingTheFile(t *testing.T) {
dir := t.TempDir()
put(t, dir, "bus.tar", snapshotTar(t, func(m *Manifest, files map[string][]byte) {
m.Streams[1].ArchiveSHA256 = sum([]byte("something else"))
}), time.Now())
r, err := VerifyFile(dir, "bus.tar")
if err != nil {
t.Fatal(err)
}
if r.Passed || !strings.Contains(r.Fault, "streams/KV_conditions/stream.tar.s2") || !strings.Contains(r.Fault, "checksum") {
t.Fatalf("%+v", r)
}
if len(r.Streams) != 2 {
t.Errorf("a failed snapshot still says its streams: %+v", r.Streams)
}
}
func subATruncatedSnapshotFails(t *testing.T) {
dir := t.TempDir()
good := snapshotTar(t, nil)
put(t, dir, "bus.tar", good[:len(good)*2/3], time.Now())
r, err := VerifyFile(dir, "")
if err != nil {
t.Fatal(err)
}
if r.Passed || r.Fault == "" {
t.Fatalf("a truncated snapshot passed: %+v", r)
}
}
func subAnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums(t *testing.T) {
dir := t.TempDir()
put(t, dir, "bus.tar", snapshotTar(t, func(m *Manifest, files map[string][]byte) {
bad := []byte("not an s2 stream at all")
s := &m.Streams[0]
files[s.Archive] = bad
s.ArchiveSHA256, s.ArchiveBytes = sum(bad), int64(len(bad))
}), time.Now())
r, _ := VerifyFile(dir, "")
if r.Passed || !strings.Contains(r.Fault, "does not read") {
t.Fatalf("%+v", r)
}
}
func subTheNewestTarIsVerifiedAndOnlyInsideTheDirectory(t *testing.T) {
dir := t.TempDir()
now := time.Now()
put(t, dir, "old.tar", []byte("junk"), now.Add(-48*time.Hour))
put(t, dir, "bus.tar", snapshotTar(t, nil), now)
put(t, dir, "bus.tar.partial", []byte("half"), now.Add(time.Hour))
r, err := VerifyFile(dir, "")
if err != nil || r.File != "bus.tar" || !r.Passed {
t.Fatalf("%+v %v", r, err)
}
if r, _ := VerifyFile(dir, "old.tar"); r.Passed || r.File != "old.tar" {
t.Errorf("a named file is the one checked: %+v", r)
}
outside := t.TempDir()
put(t, outside, "x.tar", snapshotTar(t, nil), now)
for _, name := range []string{"../" + filepath.Base(outside) + "/x.tar", outside + "/x.tar", ".."} {
if _, err := VerifyFile(dir, name); err == nil {
t.Errorf("%q was read", name)
}
}
os.Symlink(filepath.Join(outside, "x.tar"), filepath.Join(dir, "link.tar"))
if _, err := VerifyFile(dir, "link.tar"); err == nil {
t.Error("a symbolic link out of the directory was followed")
}
if _, err := VerifyFile(t.TempDir(), ""); err == nil {
t.Error("an empty directory is not a snapshot")
}
}
// TestAGoodSnapshotPassesAndSaysItsStreams runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestAGoodSnapshotPassesAndSaysItsStreams(t *testing.T) {
t.Run("AGoodSnapshotPassesAndSaysItsStreams", subAGoodSnapshotPassesAndSaysItsStreams)
t.Run("ACorruptedChecksumFailsNamingTheFile", subACorruptedChecksumFailsNamingTheFile)
t.Run("ATruncatedSnapshotFails", subATruncatedSnapshotFails)
t.Run("AnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums", subAnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums)
t.Run("TheNewestTarIsVerifiedAndOnlyInsideTheDirectory", subTheNewestTarIsVerifiedAndOnlyInsideTheDirectory)
}
go/buscalls/buscalls.go
+1 -100
View File
@@ -28,15 +28,6 @@ const (
ObjectsDelete = "mesh/objects.delete" ObjectsDelete = "mesh/objects.delete"
ObjectsInfo = "mesh/objects.info" ObjectsInfo = "mesh/objects.info"
ObjectsWatch = "mesh/objects.watch" ObjectsWatch = "mesh/objects.watch"
// ObjectsSentGet reads an object of a sender's store of sent objects by the reference a reply or an event carried
// (design 47 G §347, §407), served under the standing grant the grants part derives from the manifests; the answer is
// ObjectData.
ObjectsSentGet = "mesh/objects.sent-get"
// ObjectsPutChunk and ObjectsGetChunk are the put session's and the get session's calls (design 47 G §518, §345, §490),
// under the capability word put-sessions (stdio.CapPutSessions): a value larger than ChunkBytes is put as chunks in order,
// each answered before the next, and a get that answers ObjectsChunked is read chunk by chunk.
ObjectsPutChunk = "mesh/objects.put-chunk"
ObjectsGetChunk = "mesh/objects.get-chunk"
StreamRead = "mesh/stream.read" StreamRead = "mesh/stream.read"
StreamFill = "mesh/stream.fill" StreamFill = "mesh/stream.fill"
@@ -51,11 +42,6 @@ const (
LedgerHistory = "mesh/ledger.history" LedgerHistory = "mesh/ledger.history"
LedgerWatch = "mesh/ledger.watch" LedgerWatch = "mesh/ledger.watch"
LedgerQuery = "mesh/ledger.query" LedgerQuery = "mesh/ledger.query"
// LedgerFill is one ledger's history's fill against its cap, asked with a LedgerRequest and answered as a Fill
// (design 47 G §462, §587); LedgerRemove removes one instance, asked with a LedgerRemoveRequest and answered as
// LedgerRemoved. Both are under the capability word LedgerUpkeepCalls.
LedgerFill = "mesh/ledger.fill"
LedgerRemove = "mesh/ledger.remove"
// The ledger's repairs, which the tool runner also serves as tools of every module owning a ledger, its own // The ledger's repairs, which the tool runner also serves as tools of every module owning a ledger, its own
// ledgers only (design 25 § 12.5). // ledgers only (design 25 § 12.5).
LedgerRebuild = "mesh/ledger.rebuild" LedgerRebuild = "mesh/ledger.rebuild"
@@ -64,19 +50,6 @@ const (
LedgerLag = "mesh/ledger.lag" LedgerLag = "mesh/ledger.lag"
) )
// The capability words of the families this package's calls belong to (design 47 G §488, §622): a tool runner states a
// word once it answers the family, and a bundle fails at once, naming the word and its node, on a call whose word is not
// stated. Which calls each word covers is the tool runner's (§462).
const (
BusStateCalls = "bus-state-calls"
LedgerCalls = "ledger-calls"
LedgerUpkeepCalls = "ledger-upkeep-calls"
SentObjectsCalls = "sent-objects-calls"
// StreamPublishWord is the family of StreamPublish alone, the word a module asks for in `capabilities` before it
// says an event on its own bound stream (design 42 zfs-owns-zfs, A-44).
StreamPublishWord = "stream-publish"
)
// The methods the tool runner sends the bundle for a watch it asked, each naming the watch by its id, as `mesh/log` // The methods the tool runner sends the bundle for a watch it asked, each naming the watch by its id, as `mesh/log`
// does for a log. // does for a log.
const ( const (
@@ -104,54 +77,6 @@ type ObjectsPutRequest struct {
Description string `json:"description,omitempty"` Description string `json:"description,omitempty"`
} }
// ChunkBytes is the size of one chunk of a put or get session, and so the largest value one call puts or gets (design 47 G
// §518, A-21): it fits under the bus's default maximum payload, whatever the bus sets it to.
const ChunkBytes = 512 << 10
// ObjectsPutChunkRequest is one chunk of a put session (design 47 G §518, A-21): the object `name` in the store `objects`,
// `call` the bundle's id for this put, `seq` from 1 and one more each chunk, `last` true on the last chunk alone, and the
// chunk's bytes; the first chunk also carries the put's `description` (A-25).
type ObjectsPutChunkRequest struct {
Objects string `json:"objects"`
Name string `json:"name"`
Call string `json:"call"`
Seq uint32 `json:"seq"`
Last bool `json:"last,omitempty"`
Data []byte `json:"data"`
// Description is the put's, carried on the first chunk alone (design 47 G A-25).
Description string `json:"description,omitempty"`
}
// ObjectsChunkTaken is the answer to a chunk that is not the last: the sequence taken, or the reason it was not.
type ObjectsChunkTaken struct {
Seq uint32 `json:"seq"`
Error string `json:"error,omitempty"`
}
// ObjectsChunked is the answer of a get of a value over one chunk (design 47 G §490, §518, §620): the count of chunks (or
// true, for a sent object), the whole size, its `sha256:` digest and the session's `call`. Its Chunked is absent from the
// answer of a value read in one call.
type ObjectsChunked struct {
Chunked json.RawMessage `json:"chunked"`
Size uint64 `json:"size"`
Digest string `json:"digest"`
Call string `json:"call"`
}
// ObjectsGetChunkRequest asks one chunk of a get session by its call and sequence.
type ObjectsGetChunkRequest struct {
Call string `json:"call"`
Seq uint32 `json:"seq"`
}
// ObjectsChunk is a get session's chunk: the sequence, whether it is the last, its bytes, or the reason it is none.
type ObjectsChunk struct {
Seq uint32 `json:"seq"`
Last bool `json:"last"`
Data []byte `json:"data"`
Error string `json:"error,omitempty"`
}
// ObjectsNameRequest names one object of an object store: get, delete. // ObjectsNameRequest names one object of an object store: get, delete.
type ObjectsNameRequest struct { type ObjectsNameRequest struct {
Objects string `json:"objects"` Objects string `json:"objects"`
@@ -180,15 +105,6 @@ type ObjectInfo struct {
Deleted bool `json:"deleted,omitempty"` Deleted bool `json:"deleted,omitempty"`
} }
// ObjectsSentGetRequest names one sent object by the reference that carried it (stateobjects.Ref's fields, which this
// package does not import): the sender's store, the object's name, and its size and digest, which the reader checks.
type ObjectsSentGetRequest struct {
Store string `json:"store"`
Object string `json:"object"`
Size int64 `json:"size"`
Digest string `json:"sha256"`
}
// ObjectData is get's answer: the object's info and its bytes. // ObjectData is get's answer: the object's info and its bytes.
type ObjectData struct { type ObjectData struct {
Info ObjectInfo `json:"info"` Info ObjectInfo `json:"info"`
@@ -451,26 +367,11 @@ type ReindexRequest struct {
Index string `json:"index"` Index string `json:"index"`
} }
// LedgerRequest names a ledger: lag, fill. // LedgerRequest names a ledger: lag.
type LedgerRequest struct { type LedgerRequest struct {
Ledger string `json:"ledger"` Ledger string `json:"ledger"`
} }
// LedgerRemoveRequest names one instance of a ledger to remove from its history, its state and its indexes (design 47 G
// §587).
type LedgerRemoveRequest struct {
Ledger string `json:"ledger"`
Instance string `json:"instance"`
}
// LedgerRemoved is remove's answer: the instance, the operations removed and the bytes the history held less after it;
// an instance the ledger does not hold answers 0 and 0, so a removal said twice changes nothing.
type LedgerRemoved struct {
Instance string `json:"instance"`
Operations uint64 `json:"operations"`
Bytes uint64 `json:"bytes"`
}
// Repaired is the answer of rebuild, reindex and snapshot: how many instances (or index entries) it wrote. // Repaired is the answer of rebuild, reindex and snapshot: how many instances (or index entries) it wrote.
type Repaired struct { type Repaired struct {
Written int `json:"written"` Written int `json:"written"`
go/buscalls/seatchunks.go
-155
View File
@@ -1,155 +0,0 @@
package buscalls
import (
"encoding/json"
"errors"
"fmt"
"io"
"strconv"
"time"
"github.com/nats-io/nats.go"
)
// SeatChunks is the send of one seat verb's answer file in chunks (design 47 G §681, A-64, A-139), a verb whose seat
// definition says `chunks: true`: the holder's process sends the nth chunk on mesh.seat.<seat>.chunk.<verb>.<call>.<n>,
// and the caller acknowledges it on mesh.seat.<seat>.ack.<verb>.<call>.<n> within SeatAckWithin before the next is sent.
// `call` is the 128 random bits the caller chose. The holder sends with it and the caller receives with it: one form.
type SeatChunks struct {
Seat, Verb, Call string
}
// SeatChunkBytes is at most what one chunk carries: 512 KiB.
const SeatChunkBytes = 512 << 10
// SeatAckWithin is how long the holder waits for a chunk's acknowledgement, and so how long a caller waits for the next chunk.
const SeatAckWithin = 30 * time.Second
// SeatChunk is the body of one chunk: its number from 1, how many there are, and its bytes.
type SeatChunk struct {
N int64 `json:"n"`
Of int64 `json:"of"`
Data []byte `json:"data"`
}
// Chunk is the subject of the nth chunk.
func (c SeatChunks) Chunk(n int64) string {
return "mesh.seat." + c.Seat + ".chunk." + c.Verb + "." + c.Call + "." + strconv.FormatInt(n, 10)
}
// Ack is the subject the nth chunk is acknowledged on.
func (c SeatChunks) Ack(n int64) string {
return "mesh.seat." + c.Seat + ".ack." + c.Verb + "." + c.Call + "." + strconv.FormatInt(n, 10)
}
// Chunks is every chunk subject of this call: what a caller subscribes to before it asks the verb.
func (c SeatChunks) Chunks() string {
return "mesh.seat." + c.Seat + ".chunk." + c.Verb + "." + c.Call + ".*"
}
// SeatAcks is every acknowledgement of a verb's sends: what the holder subscribes to.
func SeatAcks(seat, verb string) string {
return "mesh.seat." + seat + ".ack." + verb + ".*.*"
}
// ErrSeatChunkMissed is a chunk that did not come within its bound: the holder ended the send.
var ErrSeatChunkMissed = errors.New("a chunk did not come within its bound, so the send ended")
// Receive reads `of` chunks in order, writes each one's bytes to w and acknowledges it before the next is sent. next
// yields the subject and body of the next chunk heard on Chunks(), or ErrSeatChunkMissed at its bound (SeatAckWithin);
// ack publishes on a subject. A chunk out of order, of another count or on another subject ends the receive, and so the
// send, since it is never acknowledged. It answers the bytes written.
func (c SeatChunks) Receive(of int64, next func() (subject string, body []byte, err error), ack func(subject string) error, w io.Writer) (int64, error) {
var written int64
for n := int64(1); n <= of; n++ {
subject, body, err := next()
if err != nil {
return written, fmt.Errorf("chunk %d of %d: %w", n, of, err)
}
if subject != c.Chunk(n) {
return written, fmt.Errorf("chunk %d of %d came on %s, not %s", n, of, subject, c.Chunk(n))
}
var ch SeatChunk
if err := json.Unmarshal(body, &ch); err != nil {
return written, fmt.Errorf("chunk %d of %d does not read: %w", n, of, err)
}
if ch.N != n || ch.Of != of {
return written, fmt.Errorf("chunk %d of %d says it is %d of %d", n, of, ch.N, ch.Of)
}
k, err := w.Write(ch.Data)
written += int64(k)
if err != nil {
return written, fmt.Errorf("chunk %d of %d cannot be written: %w", n, of, err)
}
if err := ack(c.Ack(n)); err != nil {
return written, fmt.Errorf("chunk %d of %d cannot be acknowledged: %w", n, of, err)
}
}
return written, nil
}
// ChunkLine is the core connection as a send of chunks reaches it, which a *nats.Conn is: a chunk is published on its subject and
// kept on no stream, and the acknowledgements are subscribed to, since the caller asks each as a request.
type ChunkLine interface {
Publish(subject string, data []byte) error
Subscribe(subject string, cb nats.MsgHandler) (*nats.Subscription, error)
}
// Send sends body in chunks of at most SeatChunkBytes, the holder's side of Receive (design 47 G §681, A-139, A-237): it subscribes
// to the verb's acknowledgements (SeatAcks), answers each as the request it is, publishes the nth chunk on Chunk(n) only once the
// n-1th is acknowledged on Ack(n-1), and ends the subscription when the send ends. An acknowledgement that does not come within
// `within` (SeatAckWithin where the caller has no other bound) ends the send with ErrSeatChunkMissed, and logf says where. Another
// call's acknowledgements are skipped. It blocks until the send ends, so a verb answers its chunk count first and runs it after.
func (c SeatChunks) Send(line ChunkLine, body []byte, within time.Duration, logf func(string, ...any)) error {
if logf == nil {
logf = func(string, ...any) {}
}
acks := make(chan string, 1)
sub, err := line.Subscribe(SeatAcks(c.Seat, c.Verb), func(m *nats.Msg) {
if m.Reply != "" {
_ = m.Respond([]byte("{}"))
}
select {
case acks <- m.Subject:
default:
}
})
if err != nil {
return fmt.Errorf("the acknowledgements cannot be heard, so no chunk is sent: %w", err)
}
defer func() { _ = sub.Unsubscribe() }()
total := int64((len(body) + SeatChunkBytes - 1) / SeatChunkBytes)
for n := int64(1); n <= total; n++ {
from := (n - 1) * SeatChunkBytes
to := min(int64(len(body)), from+SeatChunkBytes)
data, err := json.Marshal(SeatChunk{N: n, Of: total, Data: body[from:to]})
if err != nil {
return err
}
if err := line.Publish(c.Chunk(n), data); err != nil {
logf("the send %s of %s ends at chunk %d of %d: %v", c.Call, c.Verb, n, total, err)
return fmt.Errorf("chunk %d of %d: %w", n, total, err)
}
if !c.acknowledged(acks, c.Ack(n), within) {
logf("the send %s of %s ends at chunk %d of %d: no acknowledgement within %s", c.Call, c.Verb, n, total, within)
return fmt.Errorf("chunk %d of %d: %w", n, total, ErrSeatChunkMissed)
}
}
return nil
}
// acknowledged waits within for the acknowledgement on subject, skipping another call's.
func (c SeatChunks) acknowledged(acks <-chan string, subject string, within time.Duration) bool {
timer := time.NewTimer(within)
defer timer.Stop()
for {
select {
case got := <-acks:
if got == subject {
return true
}
case <-timer.C:
return false
}
}
}
go/buscalls/seatchunks_test.go
-72
View File
@@ -1,72 +0,0 @@
package buscalls
import (
"bytes"
"encoding/json"
"errors"
"strings"
"testing"
)
// TestSeatChunksReceive: the chunks of a send are written in order and each acknowledged before the next; a missed,
// misplaced or miscounted chunk ends the receive unacknowledged (design 47 G §681, A-139).
func TestSeatChunksReceive(t *testing.T) {
c := SeatChunks{Seat: "mesh-broker", Verb: "snapshot-get", Call: "0123456789abcdef0123456789abcdef"}
if c.Chunk(2) != "mesh.seat.mesh-broker.chunk.snapshot-get.0123456789abcdef0123456789abcdef.2" ||
c.Ack(2) != "mesh.seat.mesh-broker.ack.snapshot-get.0123456789abcdef0123456789abcdef.2" ||
c.Chunks() != "mesh.seat.mesh-broker.chunk.snapshot-get.0123456789abcdef0123456789abcdef.*" ||
SeatAcks("mesh-broker", "snapshot-get") != "mesh.seat.mesh-broker.ack.snapshot-get.*.*" {
t.Fatalf("subjects: %s %s %s", c.Chunk(2), c.Ack(2), c.Chunks())
}
type sent struct {
subject string
body SeatChunk
missed bool
}
good := []sent{{subject: c.Chunk(1), body: SeatChunk{N: 1, Of: 3, Data: []byte("ab")}},
{subject: c.Chunk(2), body: SeatChunk{N: 2, Of: 3, Data: []byte("cd")}},
{subject: c.Chunk(3), body: SeatChunk{N: 3, Of: 3, Data: []byte("e")}}}
for _, tc := range []struct {
name string
sends []sent
want string
acks int
err string
}{
{name: "three chunks", sends: good, want: "abcde", acks: 3},
{name: "a missed chunk", sends: append(good[:1:1], sent{missed: true}), want: "ab", acks: 1, err: "chunk 2 of 3"},
{name: "out of order", sends: []sent{good[1]}, acks: 0, err: "came on"},
{name: "another count", sends: []sent{{subject: c.Chunk(1), body: SeatChunk{N: 1, Of: 2, Data: []byte("x")}}}, err: "says it is 1 of 2"},
} {
t.Run(tc.name, func(t *testing.T) {
i, acks := 0, 0
next := func() (string, []byte, error) {
if i >= len(tc.sends) || tc.sends[i].missed {
return "", nil, ErrSeatChunkMissed
}
s := tc.sends[i]
i++
body, _ := json.Marshal(s.body)
return s.subject, body, nil
}
ack := func(subject string) error {
acks++
if subject != c.Ack(int64(acks)) {
t.Fatalf("ack %d on %s", acks, subject)
}
return nil
}
var out bytes.Buffer
n, err := c.Receive(3, next, ack, &out)
if tc.err == "" && err != nil || tc.err != "" && (err == nil || !strings.Contains(err.Error(), tc.err)) {
t.Fatalf("err %v, want %q", err, tc.err)
}
if tc.name == "a missed chunk" && !errors.Is(err, ErrSeatChunkMissed) {
t.Fatalf("a missed chunk is not ErrSeatChunkMissed: %v", err)
}
if out.String() != tc.want || n != int64(len(tc.want)) || acks != tc.acks {
t.Fatalf("wrote %q (%d), %d acks; want %q, %d acks", out.String(), n, acks, tc.want, tc.acks)
}
})
}
}
go/buscalls/seatsend_test.go
-94
View File
@@ -1,94 +0,0 @@
package buscalls
import (
"bytes"
"encoding/json"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
)
// lineMem is a core connection in memory: what Send publishes is kept, and when ackOf says so the chunk is acknowledged as the
// caller does, with a request on the ack subject, whose reply Send must give.
type lineMem struct {
mu sync.Mutex
handler nats.MsgHandler
subject string
chunks map[string][]byte
order []string
ackOf func(n int64) (string, bool)
}
func (l *lineMem) Subscribe(subject string, cb nats.MsgHandler) (*nats.Subscription, error) {
l.mu.Lock()
defer l.mu.Unlock()
l.handler, l.subject = cb, subject
return nil, nil
}
func (l *lineMem) Publish(subject string, data []byte) error {
l.mu.Lock()
l.chunks[subject] = data
l.order = append(l.order, subject)
h := l.handler
l.mu.Unlock()
var c SeatChunk
_ = json.Unmarshal(data, &c)
if sub, ok := l.ackOf(c.N); ok {
go h(&nats.Msg{Subject: sub, Reply: "_INBOX.x", Sub: nil})
}
return nil
}
// TestSeatChunksSend: the bytes go in chunks of at most SeatChunkBytes, each published only after the one before is
// acknowledged, and the receive of the same form reads them back; an acknowledgement missed or of another call ends the
// send (design 47 G §681, A-237, A-139).
func TestSeatChunksSend(t *testing.T) {
c := SeatChunks{Seat: "mesh-catalog", Verb: "input", Call: "0123456789abcdef"}
body := bytes.Repeat([]byte("x"), 2*SeatChunkBytes+5)
for _, tc := range []struct {
name string
ackOf func(int64) (string, bool)
err string
sent int
}{
{name: "every chunk acknowledged", ackOf: func(n int64) (string, bool) { return c.Ack(n), true }, sent: 3},
{name: "no acknowledgement", ackOf: func(int64) (string, bool) { return "", false }, err: "chunk 1 of 3", sent: 1},
{name: "another call's acknowledgement", ackOf: func(n int64) (string, bool) {
return SeatChunks{Seat: "mesh-catalog", Verb: "input", Call: "fedcba9876543210"}.Ack(n), true
}, err: "chunk 1 of 3", sent: 1},
} {
t.Run(tc.name, func(t *testing.T) {
l := &lineMem{chunks: map[string][]byte{}, ackOf: tc.ackOf}
err := c.Send(l, body, time.Millisecond*50, func(string, ...any) {})
if tc.err == "" && err != nil || tc.err != "" && (err == nil || !strings.Contains(err.Error(), tc.err)) {
t.Fatalf("err %v, want %q", err, tc.err)
}
if len(l.order) != tc.sent {
t.Fatalf("%d chunks sent, want %d", len(l.order), tc.sent)
}
if l.subject != SeatAcks("mesh-catalog", "input") {
t.Fatalf("acknowledgements heard on %q", l.subject)
}
if tc.err != "" {
return
}
var got bytes.Buffer
i := 0
_, err = c.Receive(3, func() (string, []byte, error) {
s := l.order[i]
i++
return s, l.chunks[s], nil
}, func(string) error { return nil }, &got)
if err != nil || !bytes.Equal(got.Bytes(), body) {
t.Fatalf("received %d bytes, %v", got.Len(), err)
}
})
}
if err := c.Send(&lineMem{chunks: map[string][]byte{}}, nil, time.Second, nil); err != nil {
t.Errorf("no bytes: %v", err)
}
}
go/buscompose/engine.go
+12
View File
@@ -0,0 +1,12 @@
package buscompose
// EngineModule is the node-engine's manifest's module name, and EngineActsState its state the acts asked through the
// node seat and its own asks are kept in (ADR 0563 §2, §3: "the engine's own state"): the bucket mesh-host_acts, keys
// `<node>.act.<id>`, `<node>.open.<act>.<digest>` and `<node>.ask.<id>`, each written by that node's engine alone.
const (
EngineModule = "mesh-host"
EngineActsState = "acts"
)
// EngineActsBucket is that state's bucket.
var EngineActsBucket = BucketName(EngineModule, EngineActsState)
go/buscompose/events.go
-106
View File
@@ -1,106 +0,0 @@
package buscompose
import (
"fmt"
"regexp"
"strings"
)
// The one place the mesh's event subjects are named (design 34 every-event §1, §25, §26), so neither the grants part
// nor the tool runner spells the event namespace itself.
// The namespaces an event lands in, as `from` names them (design 34 every-event §13, §26).
const (
fromSeat = "seat"
fromModule = "module"
fromStream = "stream"
)
// eventNamespaces are the three, each with the prefix its subjects start with, in EveryEvent's order.
var eventNamespaces = []struct{ from, prefix string }{
{fromSeat, "mesh.seat."},
{fromModule, "mesh.mod."},
{fromStream, moduleStreamPrefix},
}
// EveryEvent is the subjects the tool runner's account hears: every seat's events, every module's, and every module
// stream's, a restricted one included, in that order (novox/hq ADR 0670 §1, ADR 0671 §1).
func EveryEvent() []string {
out := make([]string, 0, len(eventNamespaces))
for _, ns := range eventNamespaces {
out = append(out, ns.prefix+"*.event.>")
}
return out
}
// eventToken is the form of each token of an event name.
var eventToken = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
// EventSubjects is the subjects one event name lands on. A name is `<emitter>.<event>`, optionally followed by further
// tokens (`operator-channel.decided.claude-code`), each token of the form `[A-Za-z0-9_-]+`. With from empty it answers
// six subjects, per namespace (seat, module, stream) `mesh.<ns>.<emitter>.event.<rest>` and the same followed by `.>`;
// with from `seat`, `module` or `stream` only that namespace's two. It refuses, naming the rule, a name of fewer than two
// tokens, an empty token or a token outside that form (so `*` and `>` are refused), and a from of any other word.
func EventSubjects(name, from string) ([]string, error) {
tokens := strings.Split(name, ".")
if len(tokens) < 2 {
return nil, fmt.Errorf("event name %q has fewer than two tokens: a name is <seat or module>.<event>", name)
}
for _, t := range tokens {
if !eventToken.MatchString(t) {
return nil, fmt.Errorf("event name %q has the token %q: each token of a name matches [A-Za-z0-9_-]+, "+
"so no token is empty and none is * or >", name, t)
}
}
emitter, rest := tokens[0], strings.Join(tokens[1:], ".")
var out []string
for _, ns := range eventNamespaces {
if from != "" && from != ns.from {
continue
}
subject := ns.prefix + emitter + ".event." + rest
out = append(out, subject, subject+".>")
}
if out == nil {
return nil, fmt.Errorf("from %q is none of %s, %s or %s", from, fromSeat, fromModule, fromStream)
}
return out, nil
}
// EventName is the inverse of EventSubjects for a heard subject: for `mesh.seat.<emitter>.event.<rest>`,
// `mesh.mod.<emitter>.event.<rest>` or `mesh.stream.<emitter>.event.<rest>` it answers the name `<emitter>.<rest>` and
// its namespace (`seat`, `module` or `stream`); any other subject is answered as not an event (ok false).
func EventName(subject string) (name, from string, ok bool) {
for _, ns := range eventNamespaces {
after, found := strings.CutPrefix(subject, ns.prefix)
if !found {
continue
}
emitter, rest, found := strings.Cut(after, ".event.")
if !found || emitter == "" || rest == "" || strings.Contains(emitter, ".") {
return "", "", false
}
return emitter + "." + rest, ns.from, true
}
return "", "", false
}
// Overlaps says whether some subject matches both a and b, a wildcard on either side: `*` stands for one token and
// `>`, last, for one token or more (design 34 every-event §34).
func Overlaps(a, b string) bool {
at, bt := strings.Split(a, "."), strings.Split(b, ".")
for i := 0; ; i++ {
switch {
case i == len(at) && i == len(bt):
return true
case i == len(at) || i == len(bt):
return false
case at[i] == ">" || bt[i] == ">":
return true
case at[i] == "*" || bt[i] == "*" || at[i] == bt[i]:
continue
default:
return false
}
}
}
go/buscompose/events_test.go
-98
View File
@@ -1,98 +0,0 @@
package buscompose
import (
"reflect"
"strings"
"testing"
)
// The mesh's event subjects are named in one place: EveryEvent, EventSubjects, EventName and Overlaps (design 34
// every-event §1, §25, §26, §34, §37).
func TestTheEventSubjectsAreNamedInOnePlace(t *testing.T) {
// EE-S1/AC1 (E §1, §25): the three subjects, in order.
if got, want := EveryEvent(), []string{"mesh.seat.*.event.>", "mesh.mod.*.event.>", "mesh.stream.*.event.>"}; !reflect.DeepEqual(got, want) {
t.Fatalf("EveryEvent is %v, want %v", got, want)
}
// EE-S1/AC2, AC3 (E §1, §26): a name's subjects, all six or one namespace's two, and the refusals naming the rule.
subjects := []struct {
name, from string
want []string
refused string
}{
{name: "operator-channel.decided.claude-code", want: []string{
"mesh.seat.operator-channel.event.decided.claude-code", "mesh.seat.operator-channel.event.decided.claude-code.>",
"mesh.mod.operator-channel.event.decided.claude-code", "mesh.mod.operator-channel.event.decided.claude-code.>",
"mesh.stream.operator-channel.event.decided.claude-code", "mesh.stream.operator-channel.event.decided.claude-code.>",
}},
{name: "mesh-delivery-view.delivered", from: "seat", want: []string{
"mesh.seat.mesh-delivery-view.event.delivered", "mesh.seat.mesh-delivery-view.event.delivered.>"}},
{name: "mesh-delivery-view.delivered", from: "module", want: []string{
"mesh.mod.mesh-delivery-view.event.delivered", "mesh.mod.mesh-delivery-view.event.delivered.>"}},
{name: "mesh-mail.mail", from: "stream", want: []string{
"mesh.stream.mesh-mail.event.mail", "mesh.stream.mesh-mail.event.mail.>"}},
{name: "mesh-delivery-view", refused: "fewer than two tokens"},
{name: "mesh-delivery-view..delivered", refused: "[A-Za-z0-9_-]+"},
{name: "mesh-delivery-view.*", refused: "[A-Za-z0-9_-]+"},
{name: "mesh-delivery-view.>", refused: "[A-Za-z0-9_-]+"},
{name: "mesh-delivery-view.delivered", from: "bench", refused: "none of seat, module or stream"},
}
for _, c := range subjects {
got, err := EventSubjects(c.name, c.from)
if c.refused != "" {
if err == nil || !strings.Contains(err.Error(), c.refused) {
t.Errorf("EventSubjects(%q, %q) answered %v, %v; want refused naming %q", c.name, c.from, got, err, c.refused)
}
continue
}
if err != nil || !reflect.DeepEqual(got, c.want) {
t.Errorf("EventSubjects(%q, %q) answered %v, %v; want %v", c.name, c.from, got, err, c.want)
}
// EE-S1/AC5 (E §1): every subject it answers is covered by one of EveryEvent's.
for _, s := range got {
covered := false
for _, e := range EveryEvent() {
covered = covered || Overlaps(e, s)
}
if !covered {
t.Errorf("%s is covered by none of EveryEvent", s)
}
}
}
// EE-S1/AC4 (E §1, §26): a heard subject's name and namespace, and any other subject not an event.
names := []struct{ subject, name, from string }{
{"mesh.seat.operator-channel.event.decided.claude-code", "operator-channel.decided.claude-code", "seat"},
{"mesh.mod.mesh-delivery-view.event.delivered", "mesh-delivery-view.delivered", "module"},
{"mesh.stream.mesh-delivery-checker.event.verdict", "mesh-delivery-checker.verdict", "stream"},
{"mesh.seat.intake.call.accept.ace", "", ""},
{"mesh.mod.netcheck.call.netcheck_dns", "", ""},
{"$KV.views.x", "", ""},
{"mesh.seat..event.x", "", ""},
}
for _, c := range names {
name, from, ok := EventName(c.subject)
if name != c.name || from != c.from || ok != (c.name != "") {
t.Errorf("EventName(%q) answered %q, %q, %v; want %q, %q", c.subject, name, from, ok, c.name, c.from)
}
}
// EE-S1/AC6 (E §34, §37): overlapping subjects, a wildcard on either side.
overlaps := []struct {
a, b string
want bool
}{
{"mesh.stream.mail.event.mail.>", "mesh.stream.mail.event.mail.*.message", true},
{"mesh.mod.mail.event.x", "mesh.stream.mail.event.x", false},
{"mesh.mod.mail.event.x.>", "mesh.mod.mail.event.x", false},
{"mesh.mod.mail.event.*", "mesh.mod.*.event.x", true},
{"mesh.mod.mail.event.x", "mesh.mod.mail.event.x", true},
{"mesh.mod.*.event.>", "mesh.mod.mail.event.x.ace", true},
{"mesh.mod.mail.event.x.ace", "mesh.mod.mail.event.x", false},
}
for _, c := range overlaps {
if Overlaps(c.a, c.b) != c.want || Overlaps(c.b, c.a) != c.want {
t.Errorf("Overlaps(%q, %q) is not %v both ways", c.a, c.b, c.want)
}
}
}
go/buscompose/modstreams.go
-14
View File
@@ -3,7 +3,6 @@ package buscompose
import ( import (
"fmt" "fmt"
"sort" "sort"
"strings"
) )
// moduleStreamPrefix starts every subject a module's own stream takes. // moduleStreamPrefix starts every subject a module's own stream takes.
@@ -15,19 +14,6 @@ func ModuleStreamSubject(module, event string) string {
return moduleStreamPrefix + module + ".event." + event return moduleStreamPrefix + module + ".event." + event
} }
// WirePattern is a stream's local event pattern as the bus spells it: `**` is `>`, `*` stays `*`. A reader turns a
// stream's events into the bus's subjects with it (design 46 7-approval-mark §5): the grants part composing a stream, and
// the tool runner deriving a restricted stream's marked subjects.
func WirePattern(local string) string {
parts := strings.Split(local, ".")
for i, p := range parts {
if p == "**" {
parts[i] = ">"
}
}
return strings.Join(parts, ".")
}
// A ModuleStream is one module's declared stream as the bus holds it. // A ModuleStream is one module's declared stream as the bus holds it.
type ModuleStream struct { type ModuleStream struct {
Module string Module string
go/buscompose/names_test.go
-14
View File
@@ -29,20 +29,6 @@ func TestTheBusNamesAgree(t *testing.T) {
if got := (Principal{Kind: KindLedger, Node: "a", Module: "m"}).Username(); got != "ledger.a.m" { if got := (Principal{Kind: KindLedger, Node: "a", Module: "m"}).Username(); got != "ledger.a.m" {
t.Fatalf("a ledger account is named %q", got) t.Fatalf("a ledger account is named %q", got)
} }
// G §469: a state's bucket is capped at its max-mib; naming none is unbounded.
if (Bucket{}).MaxBytes() != -1 || (Bucket{MaxMiB: 2}).MaxBytes() != 2<<20 {
t.Fatal("a bucket's cap is not its max-mib in bytes")
}
// Design 46 7-approval-mark §5: a stream's local event pattern as the bus spells it.
for local, wire := range map[string]string{"mail.**": "mail.>", "mail.*.message": "mail.*.message", "sent": "sent"} {
if got := WirePattern(local); got != wire {
t.Errorf("WirePattern(%q) = %q, want %q", local, got, wire)
}
}
// Design 46 7-approval-mark §13: a per-machine state reaches its machine's key and the keys below it, no other machine's.
if got := PerMachineKeys("ace"); len(got) != 2 || got[0] != "ace" || got[1] != "ace.>" {
t.Errorf("a per-machine state reaches %v on ace", got)
}
// A seat's family is the shared events stream filtered to its subjects: a definition says the stream's age. // A seat's family is the shared events stream filtered to its subjects: a definition says the stream's age.
for _, st := range MeshStreams() { for _, st := range MeshStreams() {
if st.Name == EventsStream && int64(st.MaxAge) != seatdef.SharedEventsMaxAgeDays*24*60*60 { if st.Name == EventsStream && int64(st.MaxAge) != seatdef.SharedEventsMaxAgeDays*24*60*60 {
go/buscompose/nats.go
+2 -2
View File
@@ -163,8 +163,8 @@ type Principal struct {
// instances and read by whoever declares it. // instances and read by whoever declares it.
State []string State []string
Reads []string Reads []string
// PerMachine is the local names of its state keyed by machine, reaching the keys of Node (PerMachineKeys) and // PerMachine is the local names of its state keyed by machine, reaching the key named for Node and
// no other machine's; KeyedReads the keys of others' state it reads one key at a time (novox/hq ADR 0260). // no other; KeyedReads the keys of others' state it reads one key at a time (novox/hq ADR 0260).
PerMachine []string PerMachine []string
KeyedReads []KeyedRead KeyedReads []KeyedRead
// Logs is the local names of the logs this principal's module keeps (novox/hq ADR 0297). // Logs is the local names of the logs this principal's module keeps (novox/hq ADR 0297).
go/buscompose/state.go
+2 -17
View File
@@ -18,29 +18,14 @@ type Bucket struct {
History int History int
// TTLSeconds is how long a value lives; zero is until replaced or deleted. // TTLSeconds is how long a value lives; zero is until replaced or deleted.
TTLSeconds int TTLSeconds int
// PerMachine says each machine's instance writes only the keys of its machine (novox/hq ADR 0260; design 46 // PerMachine says each machine's instance writes only the key named for its machine (novox/hq ADR
// 7-approval-mark §13): the key named for its machine and the keys below it, PerMachineKeys, and no other machine's. // 0260), and is granted that key and no other.
PerMachine bool PerMachine bool
// KeyTTL says the bucket allows a time to live per key (novox/hq ADR 0433 §1): its owner writes each key with its // KeyTTL says the bucket allows a time to live per key (novox/hq ADR 0433 §1): its owner writes each key with its
// own, and the server writes a delete marker, kept KeyTTLMarkerTTL, when it ends. // own, and the server writes a delete marker, kept KeyTTLMarkerTTL, when it ends.
KeyTTL bool KeyTTL bool
// MaxMiB is the bucket's cap in MiB (design 47 G §469); zero is no cap.
MaxMiB int
} }
// MaxBytes is the bucket's cap in bytes as the bus's maximum size: -1, unbounded, where it names none.
func (b Bucket) MaxBytes() int64 {
if b.MaxMiB <= 0 {
return -1
}
return int64(b.MaxMiB) * 1024 * 1024
}
// PerMachineKeys are the keys a machine's instance of a per-machine state reaches (design 46 7-approval-mark §13): the key
// named for its machine, as every module's per-machine key keeps its form, and the keys below it, `<node>.>`; no other
// machine's. A membership's per-machine state (membership.StateIssued's Key) reaches the same.
func PerMachineKeys(node string) []string { return []string{node, node + ".>"} }
// BucketName is the bucket a module's state lives in: the module and the local name joined by an // BucketName is the bucket a module's state lives in: the module and the local name joined by an
// underscore, which neither may contain, so two modules can never derive one bucket. // underscore, which neither may contain, so two modules can never derive one bucket.
func BucketName(module, name string) string { return module + "_" + name } func BucketName(module, name string) string { return module + "_" + name }
go/buscompose/users.go
+3
View File
@@ -22,6 +22,9 @@ type Declared struct {
Watches []Seat Watches []Seat
// Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152). // Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152).
Invokes []string Invokes []string
// Waits are the event families a session may wait on through the tool runner, the tool runner's module's
// alone (novox/hq ADR 0367 §2): its account subscribes them for a call, through no consumer.
Waits []Wait
// State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201). // State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201).
State []Bucket State []Bucket
// Logs are the logs it keeps, each a stream its instances append to (novox/hq ADR 0297). // Logs are the logs it keeps, each a stream its instances append to (novox/hq ADR 0297).
go/buscompose/waits.go
+18
View File
@@ -0,0 +1,18 @@
package buscompose
// Wait is one event family the tool runner's module declares: its name as the manifest gives it, and the
// emitter — a seat or a module — and the event, as the inventory resolved them.
type Wait struct {
// Name is the family as the manifest names it, `<emitter>.<event>`: what a session asks for.
Name string
Emitter string
// Seat says the emitter is a seat, whose events live under the seat's namespace (novox/hq ADR 0121).
Seat bool
// Event is the event, with any further token the subject carries: `decided.claude-code`, the asker of a
// seat that names it last (ADR 0259 §3).
Event string
// Last names what the seat's event subject ends in — "kind" for a kinded bench, "asker" for a seat naming
// its caller (ADR 0259 §3) — and is empty for an event that ends at its name. A family needing it is waited
// on with it given, and refused without.
Last string
}
go/busretry/busretry.go
-39
View File
@@ -84,45 +84,6 @@ func DoRepeatable[T any](what string, try func() (T, error), blackout func() boo
}) })
} }
// Open is what an asker passes a kept call beside the ask's key (design 47 G §351, §425, §460): Open reads the asker's own
// state and says whether the key is still open; Replaced, when the asker has one, says an event that replaces the ask was
// heard. The ask is made again only while Open answers yes and Replaced says no. An error of Open is written to Logf
// (standard error when nil), naming the key, and the ask is made again: a kept call is one the callee takes once by a
// key, so asking again is safe and not asking would be silent. Open is always given.
type Open struct {
Key string
Open func(key string) (bool, error)
Replaced func() bool
Logf func(format string, args ...any)
}
func (o Open) stillOpen() bool {
if o.Replaced != nil && o.Replaced() {
return false
}
open, err := o.Open(o.Key)
if err != nil {
logf := o.Logf
if logf == nil {
logf = func(f string, a ...any) { fmt.Fprintf(os.Stderr, "[mesh-sdk] "+f+"\n", a...) }
}
logf("busretry: whether the ask %q is still open could not be read, so it is asked again: %v", o.Key, err)
return true
}
return open
}
// DoKept is DoRepeatable for a kept call that is asked again only while its key is open and not replaced (Open).
func DoKept[T any](what string, o Open, try func() (T, error), blackout func() bool) (T, error) {
if o.Open == nil {
var zero T
return zero, fmt.Errorf("%s: a kept call names its key's open function", what)
}
return do(what, try, func(err error) bool {
return (TimedOut(err) || (Undelivered(err) && (blackout == nil || blackout()))) && o.stillOpen()
})
}
// TimedOut says err is a request's deadline passing without an answer: the request may have been delivered, so only a // TimedOut says err is a request's deadline passing without an answer: the request may have been delivered, so only a
// call marked safe to make again (DoRepeatable) asks it again. A callee's own answer (Answered) never counts. // call marked safe to make again (DoRepeatable) asks it again. A callee's own answer (Answered) never counts.
func TimedOut(err error) bool { func TimedOut(err error) bool {
go/busretry/busretry_test.go
+1 -48
View File
@@ -14,7 +14,7 @@ import (
// Issue 774 (bus blackouts at every user-list reload): only a request that reached no one is asked again, within the // Issue 774 (bus blackouts at every user-list reload): only a request that reached no one is asked again, within the
// bound; a blackout probe that says the bus answers ends the wait at once. // bound; a blackout probe that says the bus answers ends the wait at once.
func subABusThatAnswersNothingIsWaitedOut(t *testing.T) { func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
for _, row := range []struct { for _, row := range []struct {
name string name string
errs []error // what each try answers; the last repeats errs []error // what each try answers; the last repeats
@@ -68,50 +68,3 @@ func subABusThatAnswersNothingIsWaitedOut(t *testing.T) {
} }
} }
} }
// Design 47 G §351, §460: a kept call is asked again only while its key is open and not replaced; an error of `open` is
// logged naming the key and the ask is made again.
func subAKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen(t *testing.T) {
for _, row := range []struct {
name string
open func(n int) (bool, error) // by the number of asks made so far
replaced bool
tries int
logged bool
}{
{"open throughout, then answered", func(int) (bool, error) { return true, nil }, false, 3, false},
{"closed after the first ask: not asked again", func(int) (bool, error) { return false, nil }, false, 1, false},
{"replaced: not asked again", func(int) (bool, error) { return true, nil }, true, 1, false},
{"open errs: logged and asked again", func(int) (bool, error) { return false, errors.New("state unreadable") }, false, 3, true},
} {
t.Cleanup(Waits(10*time.Second, time.Millisecond, 4*time.Millisecond))
tries, logs := 0, []string{}
o := Open{Key: "ask-7", Open: func(k string) (bool, error) {
if k != "ask-7" {
t.Errorf("%s: open asked about %q", row.name, k)
}
return row.open(tries)
}, Replaced: func() bool { return row.replaced },
Logf: func(f string, a ...any) { logs = append(logs, fmt.Sprintf(f, a...)) }}
_, err := DoKept("the test", o, func() (int, error) {
tries++
if tries < 3 {
return 0, nats.ErrNoResponders
}
return tries, nil
}, nil)
if tries != row.tries || (row.tries == 3) != (err == nil) || (len(logs) > 0) != row.logged ||
(row.logged && !strings.Contains(logs[0], "ask-7")) {
t.Errorf("%s: %d asks, err %v, logs %v", row.name, tries, err, logs)
}
}
if _, err := DoKept("x", Open{Key: "k"}, func() (int, error) { return 0, nil }, nil); err == nil {
t.Errorf("a kept call with no open function was made")
}
}
// TestABusThatAnswersNothingIsWaitedOut runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
t.Run("ABusThatAnswersNothingIsWaitedOut", subABusThatAnswersNothingIsWaitedOut)
t.Run("AKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen", subAKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen)
}
go/bussnap/restore.go
-158
View File
@@ -1,158 +0,0 @@
package bussnap
import (
"archive/tar"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"time"
"git.novox.be/novox/mesh-sdk/go/busarchive"
"github.com/nats-io/nats.go"
)
// The keeper's restore in place of a snapshot it took (design 47 G §681): each stream of the file put back under its own
// name, only where no stream of that name is live.
// APIStreamDelete is the subject the keeper removes its own partial stream with.
const APIStreamDelete = "$JS.API.STREAM.DELETE."
const restoreChunk = 64 * 1024
// ManifestOf reads the manifest of a snapshot file: the streams it holds, without unpacking them.
func ManifestOf(path string) (*busarchive.Manifest, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
tr := tar.NewReader(f)
hdr, err := tr.Next()
if err != nil {
return nil, fmt.Errorf("the snapshot does not read: %w", err)
}
if hdr.Name != busarchive.ManifestName {
return nil, fmt.Errorf("the snapshot starts with %q, not its manifest", hdr.Name)
}
raw, err := io.ReadAll(io.LimitReader(tr, 64<<20))
if err != nil {
return nil, err
}
var m busarchive.Manifest
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("its manifest: %w", err)
}
return &m, nil
}
// Live says whether a stream of that name exists.
func Live(ctx context.Context, nc *nats.Conn, name string) (bool, error) {
err := Request(ctx, nc, APIStreamInfo+name, nil, nil)
var api *APIError
switch {
case err == nil:
return true, nil
case errors.As(err, &api) && api.Code == 404:
return false, nil
}
return false, err
}
// DeleteStream removes a stream; one that is not there is no error.
func DeleteStream(ctx context.Context, nc *nats.Conn, name string) error {
err := Request(ctx, nc, APIStreamDelete+name, nil, nil)
var api *APIError
if errors.As(err, &api) && api.Code == 404 {
return nil
}
return err
}
// RestoreFile puts every stream of the snapshot file back under its own name, each sent in chunks the server answers; it
// ends with the context's bound. A stream already there is the caller's to have refused beforehand.
func RestoreFile(ctx context.Context, nc *nats.Conn, path, scratch string, log func(string, ...any)) error {
f, err := os.Open(path)
if err != nil {
return err
}
defer f.Close()
a, err := busarchive.ReadArchive(f, scratch)
if err != nil {
return err
}
defer a.Close()
if err := a.Verify(); err != nil {
return err
}
for _, s := range a.Manifest.Streams {
started := time.Now()
if err := restoreStream(ctx, nc, a, s); err != nil {
return fmt.Errorf("stream %s: %w", s.Name, err)
}
log("%s: restored in %.2fs", s.Name, time.Since(started).Seconds())
}
return nil
}
func restoreStream(ctx context.Context, nc *nats.Conn, a *busarchive.Archive, s busarchive.StreamEntry) error {
meta, archive, err := a.Open(s)
if err != nil {
return err
}
defer archive.Close()
var req map[string]json.RawMessage
if err := json.Unmarshal(meta, &req); err != nil || req["config"] == nil {
return errors.New("its meta is not a stream's configuration and state")
}
var resp struct {
DeliverSubject string `json:"deliver_subject"`
}
if err := Request(ctx, nc, APIStreamRestore+s.Name, req, &resp); err != nil {
return err
}
if resp.DeliverSubject == "" {
return errors.New("the server gave nowhere to send the archive")
}
buf := make([]byte, restoreChunk)
for {
n, err := archive.Read(buf)
if n > 0 {
reply, rerr := nc.RequestWithContext(ctx, resp.DeliverSubject, buf[:n])
if rerr != nil {
return fmt.Errorf("sending the archive: %w", rerr)
}
if len(reply.Data) > 0 {
return fmt.Errorf("the server refused the archive: %s", Truncate(reply.Data))
}
}
if err == io.EOF {
break
}
if err != nil {
return err
}
}
final, err := nc.RequestWithContext(ctx, resp.DeliverSubject, nil)
if err != nil {
return fmt.Errorf("finishing the restore: %w", err)
}
var created struct {
Error *APIError `json:"error"`
State StreamState `json:"state"`
}
if err := json.Unmarshal(final.Data, &created); err != nil {
return fmt.Errorf("the server's last answer is not JSON: %q", Truncate(final.Data))
}
if created.Error != nil {
return created.Error
}
if created.State.LastSeq < s.LastSeq || (s.Messages > 0 && created.State.Messages == 0) {
return fmt.Errorf("restored with %d message(s) up to sequence %d; the snapshot held %d up to %d",
created.State.Messages, created.State.LastSeq, s.Messages, s.LastSeq)
}
return nil
}
go/bussnap/restore_integration_test.go
-63
View File
@@ -1,63 +0,0 @@
//go:build integration
package bussnap_test
import (
"context"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/bussnap"
"git.novox.be/novox/mesh-sdk/go/testbus"
"github.com/nats-io/nats.go"
)
// TestOneStreamIsTakenRemovedAndRestored: the take and restore of one stream is mesh-sdk's (design 47 G §655, A-125):
// a stream's snapshot taken with Into, the stream deleted, and RestoreFile puts it back under its own name with its
// messages; a second restore over the live name is refused by the server.
func TestOneStreamIsTakenRemovedAndRestored(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
nc, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := nc.JetStream()
if err != nil {
t.Fatal(err)
}
if _, err := js.AddStream(&nats.StreamConfig{Name: "KEPT_one", Subjects: []string{"kept.one.>"}}); err != nil {
t.Fatal(err)
}
for _, m := range []string{"a", "b", "c"} {
if _, err := js.Publish("kept.one."+m, []byte(m)); err != nil {
t.Fatal(err)
}
}
file, err := bussnap.Into(nc, t.TempDir())(ctx, []string{"KEPT_one"})
if err != nil {
t.Fatal(err)
}
m, err := bussnap.ManifestOf(file)
if err != nil || len(m.Streams) != 1 || m.Streams[0].Name != "KEPT_one" {
t.Fatalf("the manifest says %+v, %v; want the one stream KEPT_one", m, err)
}
if err := bussnap.DeleteStream(ctx, nc, "KEPT_one"); err != nil {
t.Fatal(err)
}
if live, err := bussnap.Live(ctx, nc, "KEPT_one"); err != nil || live {
t.Fatalf("live after its deletion: %v, %v", live, err)
}
if err := bussnap.RestoreFile(ctx, nc, file, t.TempDir(), t.Logf); err != nil {
t.Fatal(err)
}
info, err := js.StreamInfo("KEPT_one")
if err != nil || info.State.Msgs != 3 || info.State.LastSeq != 3 {
t.Fatalf("restored as %+v, %v; want 3 messages up to sequence 3", info, err)
}
if err := bussnap.RestoreFile(ctx, nc, file, t.TempDir(), t.Logf); err == nil {
t.Fatal("a restore over the live name was taken")
}
}
go/bussnap/snapshot.go
-433
View File
@@ -1,433 +0,0 @@
// Package bussnap takes a snapshot of the bus's streams with the server's own snapshot API (novox/hq ADR 0235): the
// nightly dump of every stream, and the bus module's keeper's snapshot of one entry before it is rebuilt or removed
// (design 25 § 12.3).
package bussnap
import (
"archive/tar"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"slices"
"sort"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/busarchive"
"github.com/nats-io/nats.go"
)
// The JetStream API's subjects this program speaks. Each is granted to the bus's own module and
// nothing else is (mesh-controller internal/broker, BusSnapshotGrants); a test holds the two lists
// equal against a real server.
const (
APIStreamNames = "$JS.API.STREAM.NAMES"
APIStreamInfo = "$JS.API.STREAM.INFO."
APIStreamSnapshot = "$JS.API.STREAM.SNAPSHOT."
APIStreamRestore = "$JS.API.STREAM.RESTORE."
)
// ChunkSize is what the server is asked to send at a time: its own default. The server sends up to
// an 8 MiB window ahead of the acknowledgements and waits for them past it, so one stream costs the
// server at most that much in flight, whatever its size.
const ChunkSize = 128 * 1024
// SnapshotOptions bounds a snapshot.
type SnapshotOptions struct {
// StreamTimeout is the longest one stream may take; past it the snapshot fails, naming it.
StreamTimeout time.Duration
// Pause is a rest between two streams, so a night's snapshot is never one long burst.
Pause time.Duration
// TempDir is where one stream's archive waits until it is written out; the system's when empty.
TempDir string
Log func(format string, args ...any)
// Streams, when set, are the only streams taken (the keeper's snapshot of one entry); every stream when empty.
Streams []string
}
// APIError is the error every JetStream API answer may carry.
type APIError struct {
Code int `json:"code"`
ErrCode int `json:"err_code"`
Description string `json:"description"`
}
func (e *APIError) Error() string { return fmt.Sprintf("%s (%d)", e.Description, e.Code) }
// StreamState is what this program reads of a stream's state; the whole of it is kept as the server
// gave it, in the stream's meta.
type StreamState struct {
Messages uint64 `json:"messages"`
Bytes uint64 `json:"bytes"`
FirstSeq uint64 `json:"first_seq"`
LastSeq uint64 `json:"last_seq"`
NumDelete int `json:"num_deleted"`
Consumers int `json:"consumer_count"`
}
type streamConfig struct {
Name string `json:"name"`
Subjects []string `json:"subjects"`
Retention string `json:"retention"`
Storage string `json:"storage"`
}
// Request asks the JetStream API and decodes its answer, refusing an answer that carries an error.
func Request(ctx context.Context, nc *nats.Conn, subject string, body any, into any) error {
var payload []byte
if body != nil {
var err error
if payload, err = json.Marshal(body); err != nil {
return err
}
}
rctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
msg, err := nc.RequestWithContext(rctx, subject, payload)
if err != nil {
if errors.Is(err, context.DeadlineExceeded) || errors.Is(err, nats.ErrTimeout) || errors.Is(err, nats.ErrNoResponders) {
return fmt.Errorf("%s was not answered (%v): the bus refuses a subject this user is not granted by not answering it", subject, err)
}
return fmt.Errorf("%s: %w", subject, err)
}
var wrapped struct {
Error *APIError `json:"error"`
}
if err := json.Unmarshal(msg.Data, &wrapped); err != nil {
return fmt.Errorf("%s answered something that is not JSON: %q", subject, Truncate(msg.Data))
}
if wrapped.Error != nil {
return fmt.Errorf("%s: %w", subject, wrapped.Error)
}
if into != nil {
return json.Unmarshal(msg.Data, into)
}
return nil
}
func Truncate(b []byte) string {
if len(b) > 200 {
return string(b[:200]) + "…"
}
return string(b)
}
// StreamNames is every stream in the account, in name order, all pages of them.
func StreamNames(ctx context.Context, nc *nats.Conn) ([]string, error) {
var names []string
for {
var page struct {
Total int `json:"total"`
Offset int `json:"offset"`
Limit int `json:"limit"`
Streams []string `json:"streams"`
}
if err := Request(ctx, nc, APIStreamNames, map[string]int{"offset": len(names)}, &page); err != nil {
return nil, err
}
names = append(names, page.Streams...)
if len(page.Streams) == 0 || len(names) >= page.Total {
break
}
}
sort.Strings(names)
return names, nil
}
// Snapshot writes every stream, one at a time, as a tar: the manifest first, then each stream's
// meta and archive under streams/<name>/. Any stream failing fails the whole: a partial copy of the
// bus written where a whole one is expected is the silent failure this exists to prevent.
func Snapshot(ctx context.Context, nc *nats.Conn, out io.Writer, o SnapshotOptions) (*busarchive.Manifest, error) {
if o.StreamTimeout <= 0 {
o.StreamTimeout = 10 * time.Minute
}
if o.Log == nil {
o.Log = func(string, ...any) {}
}
m := &busarchive.Manifest{Format: 1, Server: nc.ConnectedServerVersion(), Started: time.Now().UTC()}
names, err := StreamNames(ctx, nc)
if err != nil {
return nil, err
}
if len(o.Streams) > 0 {
var only []string
for _, n := range names {
if slices.Contains(o.Streams, n) {
only = append(only, n)
}
}
if len(only) != len(o.Streams) {
return nil, fmt.Errorf("the bus holds %v of the streams %v asked for: a snapshot missing one would pass for a whole one", only, o.Streams)
}
names = only
}
if len(names) == 0 {
// The mesh's own streams always exist once the controller has run; a bus with none is not
// one to call backed up.
return nil, errors.New("the bus holds no streams; a snapshot of nothing would pass for a backup")
}
type taken struct {
entry busarchive.StreamEntry
meta []byte
archive *os.File
}
var files []taken
defer func() {
for _, t := range files {
t.archive.Close()
os.Remove(t.archive.Name())
}
}()
for i, name := range names {
if i > 0 && o.Pause > 0 {
select {
case <-time.After(o.Pause):
case <-ctx.Done():
return nil, ctx.Err()
}
}
var info struct {
Config streamConfig `json:"config"`
}
if err := Request(ctx, nc, APIStreamInfo+name, nil, &info); err != nil {
return nil, err
}
if info.Config.Storage == "memory" {
m.Skipped = append(m.Skipped, busarchive.Skipped{Name: name, Why: "memory storage: it holds nothing across a restart, and the server cannot snapshot it"})
o.Log("%s: skipped, memory storage", name)
continue
}
f, err := os.CreateTemp(o.TempDir, "mesh-nats-snapshot-*")
if err != nil {
return nil, err
}
files = append(files, taken{archive: f})
sctx, cancel := context.WithTimeout(ctx, o.StreamTimeout)
entry, meta, err := snapshotStream(sctx, nc, name, f)
cancel()
if err != nil {
return nil, fmt.Errorf("stream %s: %w", name, err)
}
files[len(files)-1].entry, files[len(files)-1].meta = entry, meta
m.Messages += entry.Messages
m.StreamBytes += int64(entry.Bytes)
m.ArchiveBytes += entry.ArchiveBytes
o.Log("%s: %d message(s), %s, last sequence %d, %d consumer(s) → %s in %.2fs",
name, entry.Messages, Human(int64(entry.Bytes)), entry.LastSeq, entry.Consumers, Human(entry.ArchiveBytes), entry.Seconds)
}
m.Finished = time.Now().UTC()
m.Seconds = m.Finished.Sub(m.Started).Seconds()
for _, t := range files {
m.Streams = append(m.Streams, t.entry)
}
tw := tar.NewWriter(out)
manifest, err := json.MarshalIndent(m, "", " ")
if err != nil {
return nil, err
}
if err := writeEntry(tw, busarchive.ManifestName, append(manifest, '\n'), m.Finished); err != nil {
return nil, err
}
for _, t := range files {
if err := writeEntry(tw, t.entry.Meta, t.meta, m.Finished); err != nil {
return nil, err
}
if _, err := t.archive.Seek(0, io.SeekStart); err != nil {
return nil, err
}
hdr := &tar.Header{Name: t.entry.Archive, Mode: 0o600, Size: t.entry.ArchiveBytes, ModTime: m.Finished, Format: tar.FormatPAX}
if err := tw.WriteHeader(hdr); err != nil {
return nil, err
}
if _, err := io.Copy(tw, t.archive); err != nil {
return nil, err
}
}
if err := tw.Close(); err != nil {
return nil, err
}
return m, nil
}
func writeEntry(tw *tar.Writer, name string, body []byte, at time.Time) error {
if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0o600, Size: int64(len(body)), ModTime: at, Format: tar.FormatPAX}); err != nil {
return err
}
_, err := tw.Write(body)
return err
}
// snapshotStream takes one stream's snapshot into f: the server's chunks, each acknowledged as it
// arrives so the server's window keeps moving, until the server's last, empty message. It answers
// the stream's entry (the archive's size and checksum included) and its meta.
//
// The protocol is the server's (nats-server jetstream_api.go, streamSnapshot): the request names a
// subject to deliver to; the server answers with the stream's configuration and its state at the
// snapshot, then sends the archive in chunks whose reply subject is the acknowledgement it waits for
// past its window — two seconds without one and it gives up with "408 No Flow Response". The end is
// an empty message whose status header says 204, or the error that ended it.
func snapshotStream(ctx context.Context, nc *nats.Conn, name string, f *os.File) (busarchive.StreamEntry, []byte, error) {
started := time.Now()
deliver := nc.NewRespInbox()
hash := sha256.New()
var written int64
var mu sync.Mutex
var failed error
done := make(chan struct{})
var once sync.Once
finish := func(err error) {
mu.Lock()
if failed == nil {
failed = err
}
mu.Unlock()
once.Do(func() { close(done) })
}
sub, err := nc.Subscribe(deliver, func(msg *nats.Msg) {
if len(msg.Data) == 0 {
if status := msg.Header.Get("Status"); status != "" && status != "204" {
finish(fmt.Errorf("the server ended the snapshot: %s %s", status, msg.Header.Get("Description")))
return
}
finish(nil)
return
}
// Acknowledged before it is written: the chunk is already in memory, and a slow disk must not
// read to the server as a reader that went away.
if msg.Reply != "" {
if err := nc.Publish(msg.Reply, nil); err != nil {
finish(fmt.Errorf("acknowledging a chunk: %w", err))
return
}
}
if _, err := f.Write(msg.Data); err != nil {
finish(fmt.Errorf("writing the archive: %w", err))
return
}
hash.Write(msg.Data)
written += int64(len(msg.Data))
})
if err != nil {
return busarchive.StreamEntry{}, nil, err
}
defer sub.Unsubscribe()
// Every chunk is held until the callback has it; the window bounds how many there can be.
if err := sub.SetPendingLimits(-1, -1); err != nil {
return busarchive.StreamEntry{}, nil, err
}
if err := nc.FlushWithContext(ctx); err != nil {
return busarchive.StreamEntry{}, nil, err
}
var resp struct {
Config json.RawMessage `json:"config"`
State json.RawMessage `json:"state"`
}
req := map[string]any{"deliver_subject": deliver, "chunk_size": ChunkSize}
if err := Request(ctx, nc, APIStreamSnapshot+name, req, &resp); err != nil {
return busarchive.StreamEntry{}, nil, err
}
select {
case <-done:
case <-ctx.Done():
return busarchive.StreamEntry{}, nil, fmt.Errorf("not finished within its bound: %w", ctx.Err())
}
// The subscription is drained before the archive is read back, so no callback is still writing.
_ = sub.Unsubscribe()
mu.Lock()
err = failed
mu.Unlock()
if err != nil {
return busarchive.StreamEntry{}, nil, err
}
if written == 0 {
return busarchive.StreamEntry{}, nil, errors.New("the server sent an empty archive")
}
if err := f.Sync(); err != nil {
return busarchive.StreamEntry{}, nil, err
}
var cfg streamConfig
var state StreamState
if err := json.Unmarshal(resp.Config, &cfg); err != nil {
return busarchive.StreamEntry{}, nil, fmt.Errorf("its configuration: %w", err)
}
if err := json.Unmarshal(resp.State, &state); err != nil {
return busarchive.StreamEntry{}, nil, fmt.Errorf("its state: %w", err)
}
// The archive reads to its end — s2 frames, then a tar — before it is called a snapshot.
if _, err := f.Seek(0, io.SeekStart); err != nil {
return busarchive.StreamEntry{}, nil, err
}
if err := busarchive.ReadsThrough(f); err != nil {
return busarchive.StreamEntry{}, nil, fmt.Errorf("the archive the server sent does not read: %w", err)
}
// The meta, in the shape the restore API takes and the nats CLI writes (backup.json): the
// server's own configuration and state, kept as raw JSON so a field this program does not know
// is never dropped on the way back.
meta, err := json.MarshalIndent(map[string]json.RawMessage{"config": resp.Config, "state": resp.State}, "", " ")
if err != nil {
return busarchive.StreamEntry{}, nil, err
}
metaSum := sha256.Sum256(meta)
return busarchive.StreamEntry{
Name: name, Subjects: cfg.Subjects, Retention: cfg.Retention,
Messages: state.Messages, Bytes: state.Bytes, FirstSeq: state.FirstSeq, LastSeq: state.LastSeq,
Deleted: state.NumDelete, Consumers: state.Consumers,
Meta: "streams/" + name + "/backup.json", MetaSHA256: hex.EncodeToString(metaSum[:]),
Archive: "streams/" + name + "/stream.tar.s2", ArchiveBytes: written,
ArchiveSHA256: hex.EncodeToString(hash.Sum(nil)), Seconds: time.Since(started).Seconds(),
}, meta, nil
}
// Human is a size in bytes as a person reads it: 1.5 MiB.
func Human(n int64) string {
const unit = 1024
if n < unit {
return fmt.Sprintf("%d B", n)
}
div, exp := int64(unit), 0
for v := n / unit; v >= unit; v /= unit {
div *= unit
exp++
}
return fmt.Sprintf("%.1f %ciB", float64(n)/float64(div), "KMGTPE"[exp])
}
// Into is the keeper's snapshot of some streams before it rebuilds or removes them (design 25 § 12.3): one tar under
// dir, named for the streams and the time, written whole before its name is given, so a partial one is never taken
// for it. It answers the file's path.
func Into(nc *nats.Conn, dir string) func(ctx context.Context, streams []string) (string, error) {
return func(ctx context.Context, streams []string) (string, error) {
if dir == "" {
return "", errors.New("no snapshots directory is set (MESH_NATS_SNAPSHOTS_DIR), so no snapshot is taken and nothing is removed")
}
name := filepath.Join(dir, "entry-"+strings.Join(streams, "+")+"-"+time.Now().UTC().Format("20060102T150405.000000Z")+".tar")
f, err := os.CreateTemp(dir, ".entry-*.partial")
if err != nil {
return "", err
}
defer os.Remove(f.Name())
if _, err := Snapshot(ctx, nc, f, SnapshotOptions{Streams: streams, TempDir: dir}); err != nil {
f.Close()
return "", err
}
if err := f.Close(); err != nil {
return "", err
}
return name, os.Rename(f.Name(), name)
}
}
go/busstate/buscalls_test.go
-30
View File
@@ -1,30 +0,0 @@
package busstate
import (
"encoding/json"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// Design 47 G §587, §622: the ledger's upkeep calls have their requests and answers, and every family its word.
func subTheLedgerUpkeepCallsAndTheirWords(t *testing.T) {
var removal buscalls.LedgerRemoveRequest
if err := json.Unmarshal([]byte(`{"ledger":"records","instance":"r1"}`), &removal); err != nil || removal.Ledger != "records" || removal.Instance != "r1" {
t.Errorf("the removal read as %+v (%v)", removal, err)
}
raw, _ := json.Marshal(buscalls.LedgerRemoved{Instance: "r1", Operations: 3, Bytes: 900})
if string(raw) != `{"instance":"r1","operations":3,"bytes":900}` {
t.Errorf("the removal's answer reads %s", raw)
}
raw, _ = json.Marshal(buscalls.Fill{Count: 2, Bytes: 10, MaxBytes: 100})
if string(raw) != `{"count":2,"bytes":10,"max_bytes":100}` || buscalls.LedgerFill != "mesh/ledger.fill" || buscalls.LedgerRemove != "mesh/ledger.remove" {
t.Errorf("the fill reads %s, calls %s %s", raw, buscalls.LedgerFill, buscalls.LedgerRemove)
}
for word, want := range map[string]string{buscalls.BusStateCalls: "bus-state-calls", buscalls.LedgerCalls: "ledger-calls",
buscalls.LedgerUpkeepCalls: "ledger-upkeep-calls", buscalls.SentObjectsCalls: "sent-objects-calls"} {
if word != want {
t.Errorf("the word %q is %q", want, word)
}
}
}
go/busstate/busstate.go
+1 -26
View File
@@ -31,39 +31,14 @@ func call[T any](method string, params any) (T, error) {
} }
// Put stores one object whole under its name, replacing one of that name, and answers it as the store holds it. // Put stores one object whole under its name, replacing one of that name, and answers it as the store holds it.
//
// A value larger than buscalls.ChunkBytes is put as a session of chunks (design 47 G §518, §345): every object is one
// object, never split over several.
func (o Objects) Put(name string, data []byte, description string) (buscalls.ObjectInfo, error) { func (o Objects) Put(name string, data []byte, description string) (buscalls.ObjectInfo, error) {
if len(data) > buscalls.ChunkBytes {
return o.putSession(name, data, description)
}
return call[buscalls.ObjectInfo](buscalls.ObjectsPut, buscalls.ObjectsPutRequest{Objects: o.local, Name: name, return call[buscalls.ObjectInfo](buscalls.ObjectsPut, buscalls.ObjectsPutRequest{Objects: o.local, Name: name,
Data: data, Description: description}) Data: data, Description: description})
} }
// Get is one object, its info and its bytes. // Get is one object, its info and its bytes.
//
// A value the runtime answers as a session (`chunked`) is read chunk by chunk and checked against its digest (design 47 G §490).
func (o Objects) Get(name string) (buscalls.ObjectData, error) { func (o Objects) Get(name string) (buscalls.ObjectData, error) {
raw, err := stdio.BusState(buscalls.ObjectsGet, buscalls.ObjectsNameRequest{Objects: o.local, Name: name}) return call[buscalls.ObjectData](buscalls.ObjectsGet, buscalls.ObjectsNameRequest{Objects: o.local, Name: name})
if err != nil {
return buscalls.ObjectData{}, err
}
return ReadObject(name, raw)
}
// ReadObject is the object a runtime's answer to a get (or to a sent-get, design 47 G §620) holds: the value itself, or the
// session of chunks it opened, read whole.
func ReadObject(name string, raw json.RawMessage) (buscalls.ObjectData, error) {
if c, ok := chunkedAnswer(raw); ok {
return getSession(name, c)
}
var out buscalls.ObjectData
if err := json.Unmarshal(raw, &out); err != nil {
return out, fmt.Errorf("the runtime's answer for %s: %w", name, err)
}
return out, nil
} }
// List is every object the store holds. // List is every object the store holds.
go/busstate/busstate_test.go
+1 -8
View File
@@ -14,7 +14,7 @@ import (
// A bundle keeps objects in its own object store through the runtime (ADR 0595 §2, §9; A3's request for distribution's // A bundle keeps objects in its own object store through the runtime (ADR 0595 §2, §9; A3's request for distribution's
// incoming): each call is the runtime's method of buscalls, naming the store by its local name, and answers as the // incoming): each call is the runtime's method of buscalls, naming the store by its local name, and answers as the
// runtime answered; a method that is no bus-state call is refused before the runtime is asked. // runtime answered; a method that is no bus-state call is refused before the runtime is asked.
func subABundleKeepsObjectsThroughTheRuntime(t *testing.T) { func TestABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
t.Setenv(stdio.ServedAs, "distribution") t.Setenv(stdio.ServedAs, "distribution")
inR, inW := io.Pipe() inR, inW := io.Pipe()
outR, outW := io.Pipe() outR, outW := io.Pipe()
@@ -100,10 +100,3 @@ func subABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
t.Errorf("put asked %v", p) t.Errorf("put asked %v", p)
} }
} }
// TestABundleKeepsObjectsThroughTheRuntime runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
t.Run("ABundleKeepsObjectsThroughTheRuntime", subABundleKeepsObjectsThroughTheRuntime)
t.Run("TheLedgerUpkeepCallsAndTheirWords", subTheLedgerUpkeepCallsAndTheirWords)
t.Run("AValueLargerThanOneChunkIsPutAndGotBySession", subAValueLargerThanOneChunkIsPutAndGotBySession)
}
go/busstate/session.go
-113
View File
@@ -1,113 +0,0 @@
package busstate
import (
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"strings"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// The put and get sessions of a value larger than one chunk (design 47 G §518, §345, §490, A-21): a bundle sends and asks the
// chunks as calls on its channel to the tool runner, which holds the session; nothing of it is on the bus. The capability
// word put-sessions names the two calls, so a tool runner that does not state it refuses the session at once.
// callID is the bundle's id for one session: random, so two sessions never share one.
func callID() string {
var b [12]byte
_, _ = rand.Read(b[:])
return hex.EncodeToString(b[:])
}
// put sends data as a put session: chunks of buscalls.ChunkBytes in order, each answered before the next is sent, the
// answer to the last being the put's.
func (o Objects) putSession(name string, data []byte, description string) (buscalls.ObjectInfo, error) {
if err := stdio.RequireCapability(stdio.CapPutSessions); err != nil {
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, larger than one call carries: %w", name, err)
}
id := callID()
for seq, at := uint32(1), 0; ; seq++ {
end := min(at+buscalls.ChunkBytes, len(data))
last := end == len(data)
req := buscalls.ObjectsPutChunkRequest{Objects: o.local, Name: name, Call: id, Seq: seq, Last: last, Data: data[at:end]}
if seq == 1 {
req.Description = description // the first chunk carries it (design 47 G A-25)
}
raw, err := stdio.BusState(buscalls.ObjectsPutChunk, req)
if err != nil {
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, chunk %d: %w", name, seq, err)
}
var taken buscalls.ObjectsChunkTaken
if err := json.Unmarshal(raw, &taken); err != nil {
return buscalls.ObjectInfo{}, fmt.Errorf("the runtime's answer to a chunk of %s: %w", name, err)
}
if taken.Error != "" {
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, chunk %d: %s", name, seq, taken.Error)
}
if last {
var info buscalls.ObjectInfo
if err := json.Unmarshal(raw, &info); err != nil {
return info, fmt.Errorf("the runtime's answer to the put of %s: %w", name, err)
}
return info, nil
}
if taken.Seq != seq {
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s: chunk %d was answered as chunk %d", name, seq, taken.Seq)
}
at = end
}
}
// getSession reads the chunks of the session the runtime opened for name, and checks the whole against the digest and the
// size it answered.
func getSession(name string, c buscalls.ObjectsChunked) (buscalls.ObjectData, error) {
if err := stdio.RequireCapability(stdio.CapPutSessions); err != nil {
return buscalls.ObjectData{}, fmt.Errorf("getting %s, larger than one call carries: %w", name, err)
}
var whole []byte
for seq := uint32(1); ; seq++ {
raw, err := stdio.BusState(buscalls.ObjectsGetChunk, buscalls.ObjectsGetChunkRequest{Call: c.Call, Seq: seq})
if err != nil {
return buscalls.ObjectData{}, fmt.Errorf("getting %s, chunk %d: %w", name, seq, err)
}
var chunk buscalls.ObjectsChunk
if err := json.Unmarshal(raw, &chunk); err != nil {
return buscalls.ObjectData{}, fmt.Errorf("the runtime's chunk %d of %s: %w", seq, name, err)
}
if chunk.Error != "" {
return buscalls.ObjectData{}, fmt.Errorf("getting %s, chunk %d: %s", name, seq, chunk.Error)
}
if chunk.Seq != seq {
return buscalls.ObjectData{}, fmt.Errorf("getting %s: chunk %d was answered as chunk %d", name, seq, chunk.Seq)
}
whole = append(whole, chunk.Data...)
if chunk.Last {
break
}
}
sum := sha256.Sum256(whole)
if c.Digest != "" && !strings.EqualFold(c.Digest, "sha256:"+hex.EncodeToString(sum[:])) {
return buscalls.ObjectData{}, fmt.Errorf("getting %s: what was read is not the value the runtime named by %s", name, c.Digest)
}
if c.Size != 0 && c.Size != uint64(len(whole)) {
return buscalls.ObjectData{}, fmt.Errorf("getting %s: %d bytes were read, the runtime said %d", name, len(whole), c.Size)
}
return buscalls.ObjectData{Info: buscalls.ObjectInfo{Name: name, Size: uint64(len(whole)), Digest: c.Digest}, Data: whole}, nil
}
// chunkedAnswer is the session an answer to a get opens, if it is not the value itself.
func chunkedAnswer(raw json.RawMessage) (buscalls.ObjectsChunked, bool) {
var c buscalls.ObjectsChunked
if json.Unmarshal(raw, &c) != nil {
return c, false
}
switch strings.TrimSpace(string(c.Chunked)) {
case "", "null", "0", "false":
return c, false
}
return c, c.Call != ""
}
go/busstate/session_test.go
-126
View File
@@ -1,126 +0,0 @@
package busstate
import (
"bufio"
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"io"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// serveRuntime runs a bundle whose tool `go` runs fn in a goroutine and answers every call the bundle makes through handle,
// until done is closed by fn returning.
func serveRuntime(t *testing.T, fn func(), handle func(method string, params map[string]any) any) {
t.Helper()
t.Setenv(stdio.ServedAs, "distribution")
inR, inW := io.Pipe()
outR, outW := io.Pipe()
done := make(chan struct{})
go func() {
_ = stdio.ServeOn("", []stdio.Tool{{Name: "go", Run: func(map[string]any) (any, error) {
go func() { fn(); close(done) }()
return "started", nil
}}}, inR, outW)
}()
t.Cleanup(func() { _ = inW.Close() })
lines := bufio.NewScanner(outR)
lines.Buffer(make([]byte, 0, 1<<20), 64<<20)
send := func(m map[string]any) { b, _ := json.Marshal(m); _, _ = inW.Write(append(b, '\n')) }
send(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": map[string]any{"name": "go"}})
deadline := time.After(10 * time.Second)
for {
got := make(chan map[string]any, 1)
go func() {
if lines.Scan() {
var m map[string]any
_ = json.Unmarshal(lines.Bytes(), &m)
got <- m
}
}()
select {
case <-done:
return
case <-deadline:
t.Fatal("the bundle did not finish")
case m := <-got:
method, _ := m["method"].(string)
if method == "" {
continue
}
p, _ := m["params"].(map[string]any)
send(map[string]any{"jsonrpc": "2.0", "id": m["id"], "result": handle(method, p)})
}
}
}
// Design 47 G §518, §345, §490 (A-21): a value larger than one chunk is put as chunks in order under one call id, each
// answered before the next, with the store named in each; a get the runtime answers as a session is read chunk by chunk and
// checked against its digest.
func subAValueLargerThanOneChunkIsPutAndGotBySession(t *testing.T) {
value := bytes.Repeat([]byte("0123456789abcdef"), (2*buscalls.ChunkBytes+1000)/16)
sum := sha256.Sum256(value)
digest := "sha256:" + hex.EncodeToString(sum[:])
var stored []byte
var seqs []uint32
calls := map[string]bool{}
var gotValue []byte
var putErr, getErr, tamperErr error
var tampered bool
serveRuntime(t, func() {
o := ObjectsOf("incoming")
_, putErr = o.Put("big", value, "a big one")
d, err := o.Get("big")
gotValue, getErr = d.Data, err
tampered = true
_, tamperErr = o.Get("big")
}, func(method string, p map[string]any) any {
switch method {
case stdio.CapabilitiesCall:
return map[string]any{"words": []string{stdio.CapPutSessions}}
case buscalls.ObjectsPutChunk:
calls[p["call"].(string)] = true
if p["objects"] != "incoming" || p["name"] != "big" {
t.Errorf("a chunk named %v in %v", p["name"], p["objects"])
}
raw, _ := json.Marshal(p["data"])
var b []byte
_ = json.Unmarshal(raw, &b)
if (p["description"] == "a big one") != (p["seq"].(float64) == 1) {
t.Errorf("chunk %v carries the description %v: the first chunk alone does", p["seq"], p["description"])
}
stored = append(stored, b...)
seqs = append(seqs, uint32(p["seq"].(float64)))
if p["last"] == true {
return buscalls.ObjectInfo{Name: "big", Size: uint64(len(stored))}
}
return buscalls.ObjectsChunkTaken{Seq: uint32(p["seq"].(float64))}
case buscalls.ObjectsGet:
d := digest
if tampered {
d = "sha256:" + hex.EncodeToString(make([]byte, 32))
}
return map[string]any{"chunked": 3, "size": len(stored), "digest": d, "call": "g1"}
case buscalls.ObjectsGetChunk:
seq := int(p["seq"].(float64))
at := (seq - 1) * buscalls.ChunkBytes
return buscalls.ObjectsChunk{Seq: uint32(seq), Last: seq == 3, Data: stored[at:min(at+buscalls.ChunkBytes, len(stored))]}
}
t.Errorf("unexpected call %s", method)
return nil
})
if putErr != nil || getErr != nil {
t.Fatalf("put %v, get %v", putErr, getErr)
}
if len(seqs) != 3 || seqs[0] != 1 || seqs[1] != 2 || seqs[2] != 3 || len(calls) != 1 || !bytes.Equal(stored, value) || !bytes.Equal(gotValue, value) {
t.Errorf("chunks %v under %d call ids, stored %d bytes, got %d", seqs, len(calls), len(stored), len(gotValue))
}
if tamperErr == nil {
t.Error("a value that is not the one the runtime named by its digest was accepted")
}
}
go/caller_test.go
-16
View File
@@ -140,22 +140,6 @@ func TestACallerInTheArgumentsIsNeverTheCaller(t *testing.T) {
t.Errorf("%s by %q allowed: %v, want %v", c.verb, c.caller, got, c.want) t.Errorf("%s by %q allowed: %v, want %v", c.verb, c.caller, got, c.want)
} }
} }
// A-295: an entry `seat:<seat>` allows the modules the part's holder function answers for the seat, and
// nobody without holders.
seats := VerbCallers{"again": {AnyPerson, "seat:node-tool-runner", "seat:mesh-healers"}}
holders := func(seat string) []string {
return map[string][]string{"node-tool-runner": {"node-tools"}, "mesh-healers": {"mesh-delivery-watchman"}}[seat]
}
for _, c := range []struct {
caller string
holders func(string) []string
want bool
}{{"ace.node-tools", holders, true}, {"g14.mesh-delivery-watchman", holders, true}, {"ace.mesh-build-go", holders, false},
{"person.jo", holders, true}, {"ace.node-tools", nil, false}} {
if got := seats.AllowedWith("again", c.caller, c.holders) == nil; got != c.want {
t.Errorf("again by %q (holders %v) allowed: %v, want %v", c.caller, c.holders != nil, got, c.want)
}
}
if _, err := ParseVerbCallers(`{"mirror":["x"]}`, []string{"publish"}, nil); err == nil { if _, err := ParseVerbCallers(`{"mirror":["x"]}`, []string{"publish"}, nil); err == nil {
t.Error("a list naming a verb the holder does not serve was read") t.Error("a list naming a verb the holder does not serve was read")
} }
go/callers.go
+2 -17
View File
@@ -80,29 +80,14 @@ func ParseVerbCallers(raw string, verbs []string, defaults VerbCallers) (VerbCal
return c, nil return c, nil
} }
// CallerSeatPrefix opens an entry naming a seat, `seat:<seat>`: every module holding it, so a list names no module (design 47
// G A-210, A-295).
const CallerSeatPrefix = "seat:"
// Allowed answers nil when caller, as the runtime proved it, may call verb; else the refusal naming both. An unproven // Allowed answers nil when caller, as the runtime proved it, may call verb; else the refusal naming both. An unproven
// caller is never allowed, not even by AnyCaller. An entry `seat:<seat>` allows no one here (AllowedWith). // caller is never allowed, not even by AnyCaller.
func (c VerbCallers) Allowed(verb, caller string) error { return c.AllowedWith(verb, caller, nil) } func (c VerbCallers) Allowed(verb, caller string) error {
// AllowedWith is Allowed whose entries `seat:<seat>` allow a module caller whose module holders answers for that seat:
// the part's holder function, which answers from what it last heard of the assigner's seat-holders view (design 47 G A-295),
// so no call is made per check. A nil holders, or a seat it answers none for, allows no one by that entry.
func (c VerbCallers) AllowedWith(verb, caller string, holders func(seat string) []string) error {
if caller == "" || caller == Unproven { if caller == "" || caller == Unproven {
return fmt.Errorf("%s: refused: the request carries no caller the tool runner proved", verb) return fmt.Errorf("%s: refused: the request carries no caller the tool runner proved", verb)
} }
person := strings.HasPrefix(caller, "person.") person := strings.HasPrefix(caller, "person.")
for _, entry := range c[verb] { for _, entry := range c[verb] {
if seat, ok := strings.CutPrefix(entry, CallerSeatPrefix); ok {
if !person && holders != nil && slices.Contains(holders(seat), ModuleOf(caller)) {
return nil
}
continue
}
switch entry { switch entry {
case AnyCaller: case AnyCaller:
return nil return nil
go/capabilities.go
-87
View File
@@ -1,87 +0,0 @@
package stdio
import (
"encoding/json"
"errors"
"fmt"
"os"
"slices"
"sync"
"time"
)
// The call a bundle reads its tool runner's capability words with, and the words of the families that sit beside
// `mesh/publish` and `mesh/subscribe` (design 47 G §576, §622). The words of the bus-state families are in buscalls.
const (
// CapabilitiesCall is answered with {words}: the capability words the tool runner states.
CapabilitiesCall = "mesh/capabilities"
// CapPutSessions is the put and get sessions' word (§457, §518), CapFindingCalls the acknowledged publish and the
// subscription for new messages the finding writer uses (§559). CapSeatSettleDelay (seat.go) is the third.
CapPutSessions = "put-sessions"
CapFindingCalls = "finding-calls"
)
// capabilityWords are the words the tool runner states, read once at the first call that needs one and kept for as long
// as the bundle runs: a bundle lives no longer than the tool runner that started it (design 47 G §576).
type capabilityWords struct {
mu sync.Mutex
read bool
words []string
}
var capabilities capabilityWords
// get answers the words, asking once with ask. A tool runner of before G answers `mesh/capabilities` as an unknown call,
// which reads as no word stated; any other failure is returned and asked again at the next call.
func (c *capabilityWords) get(ask func() (json.RawMessage, error)) ([]string, error) {
c.mu.Lock()
defer c.mu.Unlock()
if c.read {
return c.words, nil
}
raw, err := ask()
switch {
case errors.Is(err, errors.ErrUnsupported):
c.words = nil
case err != nil:
return nil, fmt.Errorf("reading the tool runner's capability words: %w", err)
default:
var a struct {
Words []string `json:"words"`
}
if err := json.Unmarshal(raw, &a); err != nil {
return nil, fmt.Errorf("the tool runner's capability words: %w", err)
}
c.words = a.Words
}
c.read = true
return c.words, nil
}
// require is nil when word is among the stated ones, else an error naming the word and the node (design 47 G §488, §576).
func (c *capabilityWords) require(word string, ask func() (json.RawMessage, error)) error {
words, err := c.get(ask)
if err != nil {
return err
}
if slices.Contains(words, word) {
return nil
}
node := os.Getenv("MESH_NODE")
if node == "" {
node = "unknown"
}
return fmt.Errorf("the tool runner of node %s does not state the capability word %q: a call of its family is refused until it does", node, word)
}
// RequireCapability fails at once, naming the word and the node, when the tool runner this bundle runs under does not
// state word; it reads the words once (design 47 G §488, §576).
func RequireCapability(word string) error {
return capabilities.require(word, func() (json.RawMessage, error) {
c := awaitChannel()
if c == nil {
return nil, errors.New("a capability is read through the runtime that launched the bundle, and Serve has not run")
}
return c.ask(CapabilitiesCall, map[string]any{}, 30*time.Second)
})
}
go/capabilities_test.go
-37
View File
@@ -1,37 +0,0 @@
package stdio
import (
"encoding/json"
"errors"
"strings"
"testing"
)
// Design 47 G §576: the words are read once, an unknown-call answer reads as none, and a word not stated fails naming it
// and the node (from MESH_NODE).
func subACapabilityWordIsReadOnceAndNamedWhenNotStated(t *testing.T) {
t.Setenv("MESH_NODE", "ace")
asked := 0
var c capabilityWords
ask := func() (json.RawMessage, error) { asked++; return json.RawMessage(`{"words":["put-sessions"]}`), nil }
if err := c.require(CapPutSessions, ask); err != nil {
t.Fatal(err)
}
err := c.require(CapFindingCalls, ask)
if err == nil || !strings.Contains(err.Error(), CapFindingCalls) || !strings.Contains(err.Error(), "ace") || asked != 1 {
t.Errorf("a word not stated: %v after %d asks", err, asked)
}
var old capabilityWords
unknown := func() (json.RawMessage, error) {
asked++
return nil, &refused{message: "no mesh/capabilities", code: CodeNoMethod}
}
if err := old.require(CapPutSessions, unknown); err == nil || !strings.Contains(err.Error(), CapPutSessions) {
t.Errorf("a tool runner of before G stated %v", err)
}
var failing capabilityWords
if err := failing.require(CapPutSessions, func() (json.RawMessage, error) { return nil, errors.New("pipe closed") }); err == nil ||
!strings.Contains(err.Error(), "pipe closed") || failing.read {
t.Errorf("a failed read: %v, kept %v", err, failing.read)
}
}
go/catalogue/a_service_process_test.go
+1 -8
View File
@@ -54,7 +54,7 @@ func indexOf(out []map[string]any, id string) int {
// otherwise — and the controller declares the verbs it answers as tools. Its service bundle is run // otherwise — and the controller declares the verbs it answers as tools. Its service bundle is run
// by its own process; launched a second time by the runtime it would be a second controller // by its own process; launched a second time by the runtime it would be a second controller
// pretending to be an MCP server. // pretending to be an MCP server.
func subABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) { func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
m := aServiceModule(t) m := aServiceModule(t)
if len(m.Bundles) != 1 { if len(m.Bundles) != 1 {
t.Fatalf("the service's bundle was not kept: %+v", m.Bundles) t.Fatalf("the service's bundle was not kept: %+v", m.Bundles)
@@ -75,10 +75,3 @@ func subABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
t.Fatalf("a tools bundle nothing runs is no longer served: %v", loads) t.Fatalf("a tools bundle nothing runs is no longer served: %v", loads)
} }
} }
// TestABundleItsOwnProcessRunsIsNotServedByTheRuntime runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
t.Run("ABundleItsOwnProcessRunsIsNotServedByTheRuntime", subABundleItsOwnProcessRunsIsNotServedByTheRuntime)
t.Run("TheBusNamesOfAnEntryAreOneRule", subTheBusNamesOfAnEntryAreOneRule)
t.Run("AStarEmitterHearsEachDeclaringModulesUnrestrictedStream", subAStarEmitterHearsEachDeclaringModulesUnrestrictedStream)
}
go/catalogue/busnames.go
-88
View File
@@ -1,88 +0,0 @@
package catalogue
import (
"regexp"
"slices"
"sort"
"strings"
)
// The names of what a module keeps on the bus, as one rule (design 25 § 12.1; novox/hq ADR 0666 §16): the bus module
// creates them from a module's entries and the grants part grants them, and both take the names from here.
// BusNameSeparator joins a module's name and an entry's local name into the name the mesh gives the entry on the bus.
const BusNameSeparator = "_"
// SentObjectsLocal is the local name of a module's store of sent objects (the bodies of a reply or an event over the
// per-message limit): the bus module plans `<module>_sent-objects` for every module, and no entry may take the name.
const SentObjectsLocal = "sent-objects"
// ObjectsSuffix ends the local name of the object store beside a state: `<state>-objects`.
const ObjectsSuffix = "-objects"
// safeToken is a name that is one plain subject token.
var safeToken = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
// BusName is `<module>_<local>`, the name the mesh gives an entry on the bus.
func BusName(module, local string) string { return module + BusNameSeparator + local }
// StateObjectsName is the local name of the object store that holds the large values of the state `state`.
func StateObjectsName(state string) string { return state + ObjectsSuffix }
// SentObjectsName is the bus name of a module's store of sent objects.
func SentObjectsName(module string) string { return BusName(module, SentObjectsLocal) }
// BusStateStreams are the streams one bus-state entry of module is kept in on the bus, by the mesh's names
// `<module>_<local>`: a stream's own; a key-value bucket's KV_ stream with the OBJ_ stream of its object store
// `<local>-objects`; an object store's OBJ_ stream; a ledger's history stream, its snapshot stream, and its state and
// index buckets' KV_ streams. A name that is not one plain subject token names nothing.
func BusStateStreams(module string, e BusStateEntry) []string {
base := BusName(module, e.Name)
if !safeToken.MatchString(base) || strings.Contains(e.Name, BusNameSeparator) || strings.Contains(module, BusNameSeparator) {
return nil
}
var out []string
switch e.Storage {
case StorageKeyValue:
out = []string{"KV_" + base}
if e.Template == TemplateState {
out = append(out, "OBJ_"+BusName(module, StateObjectsName(e.Name)))
}
case StorageObjectStore:
out = []string{"OBJ_" + base}
case StorageLedger:
out = []string{base, base + "_snapshot", "KV_" + base + "_state"}
for _, t := range e.Types {
for _, ix := range t.Indexes {
if safeToken.MatchString(ix.Name) {
out = append(out, "KV_"+base+"_index-"+ix.Name)
}
}
}
default:
out = []string{base}
}
sort.Strings(out)
return slices.Compact(out)
}
// ObjectNameClashes are the local names of a module's bus-state entries that meet a name the mesh makes itself: the
// name of a state's object store (`<state>-objects` beside the state `<state>`) and the store of sent objects. The
// indexer's judge refuses each.
func ObjectNameClashes(entries []BusStateEntry) []string {
states := map[string]bool{}
for _, e := range entries {
if e.Storage == StorageKeyValue && e.Template == TemplateState {
states[e.Name] = true
}
}
var out []string
for _, e := range entries {
stem, ok := strings.CutSuffix(e.Name, ObjectsSuffix)
if e.Name == SentObjectsLocal || (ok && states[stem]) {
out = append(out, e.Name)
}
}
sort.Strings(out)
return slices.Compact(out)
}
go/catalogue/busnames_test.go
-38
View File
@@ -1,38 +0,0 @@
package catalogue
import (
"slices"
"testing"
)
// ADR 0666 §16: the bus names of an entry are one rule. A state entry answers its bucket's stream and the stream of
// its object store; a local name that meets a name the mesh makes is a clash for the judge.
func subTheBusNamesOfAnEntryAreOneRule(t *testing.T) {
state := BusStateEntry{Name: "views", Storage: StorageKeyValue, Template: TemplateState}
rows := []struct {
name string
entry BusStateEntry
streams []string
}{
{"a state: its bucket and its object store", state, []string{"KV_m_views", "OBJ_m_views-objects"}},
{"a key-value that is not a state template: its bucket only", BusStateEntry{Name: "x", Storage: StorageKeyValue}, []string{"KV_m_x"}},
{"an object store", BusStateEntry{Name: "blobs", Storage: StorageObjectStore, Template: TemplateObjects}, []string{"OBJ_m_blobs"}},
{"a stream", BusStateEntry{Name: "log", Storage: StorageStream, Template: TemplateLog}, []string{"m_log"}},
{"a ledger", BusStateEntry{Name: "l", Storage: StorageLedger, Template: TemplateLedger}, []string{"KV_m_l_state", "m_l", "m_l_snapshot"}},
{"a name that is not one token", BusStateEntry{Name: "a.b", Storage: StorageStream}, nil},
{"a local name holding the separator", BusStateEntry{Name: "a_b", Storage: StorageStream}, nil},
}
for _, r := range rows {
if got := BusStateStreams("m", r.entry); !slices.Equal(got, r.streams) {
t.Errorf("%s: streams %v, want %v", r.name, got, r.streams)
}
}
if got := SentObjectsName("m"); got != "m_sent-objects" {
t.Errorf("sent store %q", got)
}
clash := []BusStateEntry{state, {Name: "views-objects", Storage: StorageObjectStore}, {Name: "other-objects", Storage: StorageObjectStore},
{Name: SentObjectsLocal, Storage: StorageObjectStore}}
if got := ObjectNameClashes(clash); !slices.Equal(got, []string{"sent-objects", "views-objects"}) {
t.Errorf("clashes %v", got)
}
}
go/catalogue/busstate.go
-6
View File
@@ -50,9 +50,6 @@ const (
OptionAgeDays = "max-age-days" OptionAgeDays = "max-age-days"
// OptionCapMiB is the cap of a stream, a bucket or an object store, in MiB. // OptionCapMiB is the cap of a stream, a bucket or an object store, in MiB.
OptionCapMiB = "max-mib" OptionCapMiB = "max-mib"
// OptionObjectsCapMiB is the cap of a state's objects store `<state>-objects`, in MiB, where the state needs
// another than the one the bus module plans (design 47 G §513, §547).
OptionObjectsCapMiB = "objects-max-mib"
// OptionGrant names the trusted config value granting another module a part of it (ADR 0363 §4, ADR 0592 §9). // OptionGrant names the trusted config value granting another module a part of it (ADR 0363 §4, ADR 0592 §9).
OptionGrant = "grant" OptionGrant = "grant"
// OptionDeliveries is how many times a work queue's message is delivered before it is a dead letter. // OptionDeliveries is how many times a work queue's message is delivered before it is a dead letter.
@@ -145,9 +142,6 @@ type BusStateEntry struct {
Configuration map[string]json.RawMessage `json:"configuration,omitempty"` Configuration map[string]json.RawMessage `json:"configuration,omitempty"`
// Types are a ledger's types (ADR 0597 §1), for the storage type ledger only. // Types are a ledger's types (ADR 0597 §1), for the storage type ledger only.
Types []LedgerType `json:"types,omitempty"` Types []LedgerType `json:"types,omitempty"`
// Events are the events an entry of the template events carries, named locally, as a part's finding stream
// lists its three finding events; the catalog graph entry carries them (design 47 G §529).
Events []string `json:"events,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47). // KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout KeptLayout
} }
go/catalogue/delivery_seat_test.go
+55
View File
@@ -1,11 +1,66 @@
package catalogue package catalogue
import ( import (
"reflect"
"strings"
"testing" "testing"
"git.novox.be/novox/mesh-sdk/go/seatdef" "git.novox.be/novox/mesh-sdk/go/seatdef"
) )
// The delivery seat answers "what did the mesh's checks say of this pull request?" as a verb of its own,
// `checks` (novox/hq ADR 0239): read by repository and a pull request's number or a commit.
func TestTheDeliverySeatPromisesChecks(t *testing.T) {
seat, ok := SeatNamed(DeliverySeat)
if !ok {
t.Fatal("the delivery seat is not in the set")
}
var checks *Verb
for i := range seat.Serves {
if seat.Serves[i].Name == "checks" {
checks = &seat.Serves[i]
}
}
if checks == nil {
t.Fatalf("the delivery seat promises %v and not checks", VerbNames(seat.Serves))
}
props, _ := checks.Input["properties"].(map[string]any)
for _, arg := range []string{"repository", "number", "commit"} {
if _, has := props[arg]; !has {
t.Errorf("checks takes no %q", arg)
}
}
if req, _ := checks.Input["required"].([]string); !reflect.DeepEqual(req, []string{"repository"}) {
t.Errorf("checks requires %v, wanted only the repository", req)
}
// Added after the seat's holder shipped, it is optional: the holder serving the ten verbs before it still
// holds the seat, and one serving all eleven does too — so the controller and the catalogue can move in
// either order, and no check of the catalogue fails on a module nobody touched between the two.
if !checks.Optional {
t.Fatal("checks is a condition of holding before its holder serves it")
}
var before []string
for _, v := range VerbNames(seat.Serves) {
if v != "checks" {
before = append(before, v)
}
}
m := Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh, Serves: before}}}
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder without checks yet: %v", err)
}
m.Claims[0].Serves = VerbNames(seat.Serves)
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder serving every verb: %v", err)
}
// Every other verb is still a condition of holding.
m.Claims[0].Serves = append([]string{"checks"}, before[1:]...)
if err := CanHold(m, seat); err == nil || !strings.Contains(err.Error(), before[0]) {
t.Fatalf("a holder without %s: %v", before[0], err)
}
}
// An event whose subject carries parts is said by its subject with `*` for each part, so a consumer naming one // An event whose subject carries parts is said by its subject with `*` for each part, so a consumer naming one
// instance or every instance of it is heard as the seat's (design 47 *What each step says*). // instance or every instance of it is heard as the seat's (design 47 *What each step says*).
func TestAnEventWithPartsIsTheSeatsByItsSubject(t *testing.T) { func TestAnEventWithPartsIsTheSeatsByItsSubject(t *testing.T) {
go/catalogue/environment_into_test.go
+2 -3
View File
@@ -35,9 +35,8 @@ func TestTheSeatTableCarriesTheEnvironmentAndTheLoginShell(t *testing.T) {
if !ok || shell.Scope != ScopeNode || shell.Decision != "novox/hq ADR 0204" { if !ok || shell.Scope != ScopeNode || shell.Decision != "novox/hq ADR 0204" {
t.Fatalf("node-login-shell is not a node seat: %+v (defined %v)", shell, ok) t.Fatalf("node-login-shell is not a node seat: %+v (defined %v)", shell, ok)
} }
// A-264, A-265: its marked execute gains the verb approvals beside it. if len(shell.Serves) != 1 || shell.Serves[0].Name != "execute" || shell.Serves[0].Description == "" {
if len(shell.Serves) != 2 || shell.Serves[0].Name != "execute" || shell.Serves[0].Description == "" || shell.Serves[1].Name != "approvals" { t.Fatalf("the login shell serves %+v, not execute alone", shell.Serves)
t.Fatalf("the login shell serves %+v, not execute and approvals", shell.Serves)
} }
props, _ := shell.Serves[0].Input["properties"].(map[string]any) props, _ := shell.Serves[0].Input["properties"].(map[string]any)
required, _ := shell.Serves[0].Input["required"].([]string) required, _ := shell.Serves[0].Input["required"].([]string)
go/catalogue/invokes_test.go
-14
View File
@@ -17,15 +17,6 @@ func TestAManifestMaySayWhatItInvokes(t *testing.T) {
if len(m.Invokes) != 2 || m.Invokes[1] != "*" { if len(m.Invokes) != 2 || m.Invokes[1] != "*" {
t.Fatalf("invokes not read: %v", m.Invokes) t.Fatalf("invokes not read: %v", m.Invokes)
} }
// An asker names a build seat's queue as seat:<seat>.accept.<queue> (design 47 G A-231), and nothing else reads as one.
for invoke, want := range map[string]string{"seat:node-build-go.accept.check": "node-build-go/check",
"seat:node-build-go.run": "", "node-build-go.accept.check": "", "seat:node-build-go.accept.": "",
"seat:node-build-go.accept.check.mesh-catalog": "", "*": ""} {
seat, queue, ok := QueueInvoke(invoke)
if got := seat + "/" + queue; ok != (want != "") || ok && got != want {
t.Errorf("QueueInvoke(%q) = %s, %v; want %q", invoke, got, ok, want)
}
}
// And that its own secrets are the core's credentials (issue 774): read, and written back as said. // And that its own secrets are the core's credentials (issue 774): read, and written back as said.
for raw, core := range map[string]bool{`{"module":"bus","version":"1","core-credentials":true}`: true, `{"module":"app","version":"1"}`: false} { for raw, core := range map[string]bool{`{"module":"bus","version":"1","core-credentials":true}`: true, `{"module":"app","version":"1"}`: false} {
m, err := decodeForTest([]byte(raw)) m, err := decodeForTest([]byte(raw))
@@ -48,11 +39,6 @@ func TestAManifestMaySayWhatItInvokes(t *testing.T) {
if back, _ := json.Marshal(runner); !strings.Contains(string(back), `"tool-runner":true`) { if back, _ := json.Marshal(runner); !strings.Contains(string(back), `"tool-runner":true`) {
t.Errorf("a tool runner written back as %s", back) t.Errorf("a tool runner written back as %s", back)
} }
// EE-S2/AC1: the tool runner names no events to wait on any more (novox/hq ADR 0670 §2): a stored manifest still
// carrying `waits` is read without it, the word named unknown (design 34 every-event §2).
if old, err := decodeForTest([]byte(`{"module":"node-tools","version":"1","tool-runner":true,"waits":["a.b"]}`)); err != nil || !strings.Contains(old.UnknownField(), `"waits"`) {
t.Errorf("a manifest carrying waits: %v, unknown %q", err, old.UnknownField())
}
other, _ := decodeForTest([]byte(`{"module":"node-tools","version":"1"}`)) other, _ := decodeForTest([]byte(`{"module":"node-tools","version":"1"}`))
if got, ok := ToolRunnerOf([]Manifest{other, runner}); !ok || got.Module != "any-name" { if got, ok := ToolRunnerOf([]Manifest{other, runner}); !ok || got.Module != "any-name" {
t.Errorf("the tool runner found is %q, %v: by what it says, never by its name", got.Module, ok) t.Errorf("the tool runner found is %q, %v: by what it says, never by its name", got.Module, ok)
go/catalogue/kept_form.go
+3 -35
View File
@@ -1,8 +1,6 @@
package catalogue package catalogue
import ( import (
"bytes"
"encoding/json"
"fmt" "fmt"
"sort" "sort"
) )
@@ -19,39 +17,9 @@ import (
type KeptLayout struct { type KeptLayout struct {
// Form is the form of the kept data's layout, 0 where none is said. // Form is the form of the kept data's layout, 0 where none is said.
Form int `json:"form,omitempty"` Form int `json:"form,omitempty"`
// CatchUp is the module's own tool a restore of this kept data is followed by, with its arguments, named by the // CatchUp is the module's own tool a restore of this kept data is followed by, named by the manifest that raised its
// manifest that raised its form; nil where none is (design 47 G §631). // form; empty where none is.
CatchUp *CatchUp `json:"catch-up,omitempty"` CatchUp string `json:"catch-up,omitempty"`
}
// CatchUp is a manifest's `catch-up`: the tool and the arguments it is called with (design 47 G §631).
type CatchUp struct {
Tool string `json:"tool"`
Arguments map[string]any `json:"arguments,omitempty"`
}
// UnmarshalJSON reads the object {tool, arguments} whole, and a string of before G as {tool, arguments: {}}.
func (c *CatchUp) UnmarshalJSON(raw []byte) error {
trimmed := bytes.TrimSpace(raw)
if len(trimmed) > 0 && trimmed[0] == '"' {
var tool string
if err := json.Unmarshal(trimmed, &tool); err != nil {
return err
}
*c = CatchUp{Tool: tool}
return nil
}
var full struct {
Tool string `json:"tool"`
Arguments map[string]any `json:"arguments"`
}
dec := json.NewDecoder(bytes.NewReader(trimmed))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a catch-up is {tool, arguments}: %w", typedUnknown(err))
}
*c = CatchUp{Tool: full.Tool, Arguments: full.Arguments}
return nil
} }
// refuse is the one reading of a layout's form every kept kind's decoder calls: a form is a whole number from 0, and one // refuse is the one reading of a layout's form every kept kind's decoder calls: a form is a whole number from 0, and one
go/catalogue/kept_form_test.go
+7 -7
View File
@@ -13,8 +13,8 @@ func TestAKeptDataSaysItsFormAndItsCatchUp(t *testing.T) {
m, err := decodeForTest([]byte(`{"module":"x","version":"1","tools":["x_take_in"], m, err := decodeForTest([]byte(`{"module":"x","version":"1","tools":["x_take_in"],
"state": ["seen", {"name":"modules","history":20,"form":1,"catch-up":"x_take_in"}], "state": ["seen", {"name":"modules","history":20,"form":1,"catch-up":"x_take_in"}],
"logs": [{"name":"acts","form":2}], "logs": [{"name":"acts","form":2}],
"streams": [{"name":"forge","events":["merged"],"form":1,"catch-up":{"tool":"x_take_in","arguments":{"all":true}}}], "streams": [{"name":"forge","events":["merged"],"form":1,"catch-up":"x_take_in"}],
"objects": [{"name":"revisions","max-mib":64,"form":1,"catch-up":{"tool":"x_take_in","arguments":{"all":true}}}], "objects": [{"name":"revisions","max-mib":64,"form":1,"catch-up":"x_take_in"}],
"requires": [ "requires": [
{"name":"records","storage":"ledger","template":"ledger","form":3,"catch-up":"x_take_in"}, {"name":"records","storage":"ledger","template":"ledger","form":3,"catch-up":"x_take_in"},
{"name":"holds","storage":"key-value","template":"state"}, {"name":"holds","storage":"key-value","template":"state"},
@@ -27,13 +27,13 @@ func TestAKeptDataSaysItsFormAndItsCatchUp(t *testing.T) {
want := []KeptDeclared{ want := []KeptDeclared{
{"acts", KeptKindLog, KeptLayout{Form: 2}}, {"acts", KeptKindLog, KeptLayout{Form: 2}},
{"blobs", KeptKindObjects, KeptLayout{Form: 4}}, {"blobs", KeptKindObjects, KeptLayout{Form: 4}},
{"forge", KeptKindStream, KeptLayout{1, &CatchUp{Tool: "x_take_in", Arguments: map[string]any{"all": true}}}}, {"forge", KeptKindStream, KeptLayout{1, "x_take_in"}},
{"history", KeptKindLog, KeptLayout{Form: 1}}, {"history", KeptKindLog, KeptLayout{Form: 1}},
{"holds", KeptKindState, KeptLayout{}}, {"holds", KeptKindState, KeptLayout{}},
{"marks", KeptKindStream, KeptLayout{}}, {"marks", KeptKindStream, KeptLayout{}},
{"modules", KeptKindState, KeptLayout{1, &CatchUp{Tool: "x_take_in"}}}, {"modules", KeptKindState, KeptLayout{1, "x_take_in"}},
{"records", KeptKindLedger, KeptLayout{3, &CatchUp{Tool: "x_take_in"}}}, {"records", KeptKindLedger, KeptLayout{3, "x_take_in"}},
{"revisions", KeptKindObjects, KeptLayout{1, &CatchUp{Tool: "x_take_in", Arguments: map[string]any{"all": true}}}}, {"revisions", KeptKindObjects, KeptLayout{1, "x_take_in"}},
{"seen", KeptKindState, KeptLayout{}}, {"seen", KeptKindState, KeptLayout{}},
} }
if got := KeptDeclarations(m); !reflect.DeepEqual(got, want) { if got := KeptDeclarations(m); !reflect.DeepEqual(got, want) {
@@ -60,7 +60,7 @@ func TestAKeptDataSaysItsFormAndItsCatchUp(t *testing.T) {
} }
changes := FormChanges(running, applied) changes := FormChanges(running, applied)
wantChanges := []FormChange{ wantChanges := []FormChange{
{Name: "acts", Kind: KeptKindLog, Running: &KeptLayout{}, Applied: &KeptLayout{1, &CatchUp{Tool: "x_take_in"}}}, {Name: "acts", Kind: KeptKindLog, Running: &KeptLayout{}, Applied: &KeptLayout{1, "x_take_in"}},
{Name: "fresh", Kind: KeptKindState, Applied: &KeptLayout{}}, {Name: "fresh", Kind: KeptKindState, Applied: &KeptLayout{}},
{Name: "modules", Kind: KeptKindState, Running: &KeptLayout{}}, {Name: "modules", Kind: KeptKindState, Running: &KeptLayout{}},
} }
go/catalogue/login_shell_execute_test.go
+3 -60
View File
@@ -1,12 +1,6 @@
package catalogue package catalogue
import ( import "testing"
"encoding/json"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// The login shell's `execute` runs any command as the operator account, which can become root without a // The login shell's `execute` runs any command as the operator account, which can become root without a
// person, so the operator withheld it on the control-node until a call to it needs a person's approval // person, so the operator withheld it on the control-node until a call to it needs a person's approval
@@ -20,14 +14,12 @@ func TestTheLoginShellsExecuteIsOptionalSoAMachineMayWithholdIt(t *testing.T) {
if !ok { if !ok {
t.Fatal("node-login-shell is not defined") t.Fatal("node-login-shell is not defined")
} }
// A-264, A-265: its marked execute gains the verb approvals beside it. if len(seat.Serves) != 1 || seat.Serves[0].Name != "execute" {
if len(seat.Serves) != 2 || seat.Serves[0].Name != "execute" || seat.Serves[1].Name != "approvals" { t.Fatalf("the login shell promises %+v, not execute alone", seat.Serves)
t.Fatalf("the login shell promises %+v, not execute and approvals", seat.Serves)
} }
if !seat.Serves[0].Optional { if !seat.Serves[0].Optional {
t.Fatal("execute is required, so a holder that withholds it on one machine could not hold the seat there") t.Fatal("execute is required, so a holder that withholds it on one machine could not hold the seat there")
} }
// A-266: no holder names the gate's verb approvals, so zsh's claim holds as it is written.
withholding := Manifest{Module: "zsh", Version: "1", Claims: []Claim{{Name: LoginShellSeat, Scope: ScopeNode}}} withholding := Manifest{Module: "zsh", Version: "1", Claims: []Claim{{Name: LoginShellSeat, Scope: ScopeNode}}}
if err := CanHold(withholding, seat); err != nil { if err := CanHold(withholding, seat); err != nil {
t.Fatalf("a holder serving no execute is refused: %v", err) t.Fatalf("a holder serving no execute is refused: %v", err)
@@ -40,52 +32,3 @@ func TestTheLoginShellsExecuteIsOptionalSoAMachineMayWithholdIt(t *testing.T) {
} }
t.Run("compiled", check) t.Run("compiled", check)
} }
// Design 47 G §677, A-266, A-270: a process that serves a marked verb holds the gate only in Go and with a state `approvals`
// declared with key-ttl; no holder names the verb approvals where a verb is marked, and a seat with none leaves it to its holder.
func TestAHolderOfAMarkedVerbDeclaresItsApprovalsStateWithKeyTTL(t *testing.T) {
seat, ok := SeatNamed(LoginShellSeat)
if !ok {
t.Fatal("node-login-shell is not defined")
}
holder := func(language string, state ...BusStateEntry) Manifest {
return Manifest{Module: "zsh", Version: "1", Claims: []Claim{{Name: LoginShellSeat, Scope: ScopeNode, Serves: []string{"execute"}}},
Resources: []map[string]any{{"id": "p", "type": "process", "artifact": "program", "serves": []any{"seat:" + LoginShellSeat + ".execute"}}},
Build: &Build{Artifacts: []Artifact{{Name: "program", Language: language}}}, BusState: state}
}
keyed := BusStateEntry{Name: "approvals", Storage: StorageKeyValue, Template: TemplateState,
Options: map[string]json.RawMessage{OptionLifetimePerKey: json.RawMessage("true")}}
plain := BusStateEntry{Name: "approvals", Storage: StorageKeyValue, Template: TemplateState}
for name, c := range map[string]struct {
m Manifest
refused string
}{
"go with key-ttl": {holder("go", keyed), ""},
"go without key-ttl": {holder("go", plain), "key-ttl"},
"go with no state": {holder("go"), "key-ttl"},
"typescript with key-ttl": {holder("typescript", keyed), "only Go"},
} {
err := CanHold(c.m, seat)
if (c.refused == "") != (err == nil) || err != nil && !strings.Contains(err.Error(), c.refused) {
t.Errorf("%s: %v, want refusal containing %q", name, err, c.refused)
}
}
// A seat with no marked verb: a holder serving no verb named approvals is still refused for it.
open := Seat{Name: "s", Scope: ScopeNode, Serves: []Verb{{Name: "a"}, {Name: seatdef.ApprovalsVerb}}}
if got := unservedVerbs([]string{"a"}, open.Serves); len(got) != 1 || got[0] != "approvals" {
t.Errorf("a seat with no marked verb: approvals unserved is %v, want it named", got)
}
}
// Design 47 G A-275: InvokedVerbs reads both forms of an `invokes` entry and refuses one of neither.
func TestInvokedVerbsReadsBothFormsAndRefusesAnother(t *testing.T) {
got, err := InvokedVerbs(json.RawMessage(`["seat:a.b",{"verb":"seat:c.d","fields":["x"]}]`))
if err != nil || len(got) != 2 || got[0] != "seat:a.b" || got[1] != "seat:c.d" {
t.Fatalf("%v, %v", got, err)
}
for _, bad := range []string{`[3]`, `{"verb":"x"}`, `[{"verb":""}]`} {
if _, err := InvokedVerbs(json.RawMessage(bad)); err == nil {
t.Errorf("%s was read", bad)
}
}
}
go/catalogue/manifest.go
+10 -18
View File
@@ -592,6 +592,13 @@ type Manifest struct {
// changes. A long-form entry naming no field is a verb of Invokes alone. // changes. A long-form entry naming no field is a verb of Invokes alone.
InvokeFields map[string][]string `json:"-"` InvokeFields map[string][]string `json:"-"`
// Waits are the event families a session may wait on through the mesh MCP server's `mesh_wait`, each
// `<emitter>.<event>` as `consumes` names one, a seat's or a module's (novox/hq ADR 0367 §2). Only the tool
// runner's module declares them, one by one and never by a wildcard. **A grant apart from `consumes`**: the
// controller grants the tool runner's account a plain subscription on those subjects alone, with no
// durable consumer, and issues the same list in its membership; no call or setting adds one (§6).
Waits []string `json:"waits,omitempty"`
// State is the current state this module keeps on the bus, by local name: each a key-value // State is the current state this module keeps on the bus, by local name: each a key-value
// bucket the controller creates, which every instance of the module writes and reads // bucket the controller creates, which every instance of the module writes and reads
// (novox/hq ADR 0201). Not history — that is an event — and never a secret, sealed or not. // (novox/hq ADR 0201). Not history — that is an event — and never a secret, sealed or not.
@@ -810,7 +817,7 @@ type Manifest struct {
OwnSecrets OwnSecrets `json:"own-secrets,omitempty"` OwnSecrets OwnSecrets `json:"own-secrets,omitempty"`
// ToolRunner says this module is the node's tool runner (issue 774): the one program on each node that loads every // ToolRunner says this module is the node's tool runner (issue 774): the one program on each node that loads every
// assigned module's tools bundle, served by its runtime process. Said by // assigned module's tools bundle, served by its runtime process and the only module that may declare `waits`. Said by
// the module, never by a name in mesh-sdk. // the module, never by a name in mesh-sdk.
ToolRunner bool `json:"tool-runner,omitempty"` ToolRunner bool `json:"tool-runner,omitempty"`
@@ -1245,9 +1252,8 @@ type Artifact struct {
Needs []string `json:"needs,omitempty"` Needs []string `json:"needs,omitempty"`
// Calls are the capability words (design 47 G §488, §605) a package's code calls of the tool runner: the delivery plan // Calls are the capability words (design 47 G §488, §605) a package's code calls of the tool runner: the delivery plan
// derives, for each module whose build resolved a version of such a package, an ordering edge to the tool runner stating // derives, for each module whose build resolved a version of such a package, an ordering edge to the module whose
// those words on each node, so no manifest names the tool runner's module. Said on a package only; the manifest // artifact States the word. Read here so the second reader release keeps a manifest that names them (§621).
// validator checks their form.
Calls []string `json:"calls,omitempty"` Calls []string `json:"calls,omitempty"`
// States are the capability words (design 47 G §488) the tool runner's artifact states, the edge of a package's Calls // States are the capability words (design 47 G §488) the tool runner's artifact states, the edge of a package's Calls
@@ -2265,17 +2271,3 @@ const InstancesInterchangeable = "interchangeable"
// WhileStopped is the resource key naming the containers a scheduled step holds still while it // WhileStopped is the resource key naming the containers a scheduled step holds still while it
// runs (novox/hq ADR 0189). Carried to the host unchanged, like `schedule`. // runs (novox/hq ADR 0189). Carried to the host unchanged, like `schedule`.
const WhileStopped = "while-stopped" const WhileStopped = "while-stopped"
// InvokedVerbs is the verbs of a manifest's `invokes` in either form, a verb or {verb, fields} (design 47 G §558, A-275): the
// one reader of the field for a part that holds no Manifest. An entry of neither form is an error, never none.
func InvokedVerbs(raw json.RawMessage) ([]string, error) {
keys := map[string]json.RawMessage{"invokes": raw}
if _, err := readInvokes(keys); err != nil {
return nil, err
}
var verbs []string
if err := json.Unmarshal(keys["invokes"], &verbs); err != nil {
return nil, fmt.Errorf("invokes: a list of verbs, or of {verb, fields}: %w", err)
}
return verbs, nil
}
go/catalogue/optional_requirements_test.go
+1 -14
View File
@@ -81,16 +81,6 @@ func TestARequirementsLongForm(t *testing.T) {
// Issue 1966: a key-value entry of `requires` is kept state beside the old section's, by its local name. // Issue 1966: a key-value entry of `requires` is kept state beside the old section's, by its local name.
{"a key-value entry beside an old state", `[{"name":"holds","storage":"key-value","template":"state"}], "state": ["seen"]`, {"a key-value entry beside an old state", `[{"name":"holds","storage":"key-value","template":"state"}], "state": ["seen"]`,
"", []BusStateEntry{{Name: "holds", Storage: StorageKeyValue, Template: TemplateState}}, nil, ""}, "", []BusStateEntry{{Name: "holds", Storage: StorageKeyValue, Template: TemplateState}}, nil, ""},
// G §529: a template events entry's field `events` is carried, not set aside; G §513: `objects-max-mib` is a
// state's named option.
{"a finding stream and a state's objects cap", `[{"name":"finding-events","storage":"stream","template":"events",
"options":{"max-age-days":7,"max-mib":16},"events":["finding-written","finding-cleared","findings-stated"]},
{"name":"findings","storage":"key-value","template":"state","options":{"` + OptionObjectsCapMiB + `":512}}]`,
"", []BusStateEntry{{Name: "finding-events", Storage: StorageStream, Template: TemplateEvents,
Options: map[string]json.RawMessage{"max-age-days": json.RawMessage(`7`), "max-mib": json.RawMessage(`16`)},
Events: []string{"finding-written", "finding-cleared", "findings-stated"}},
{Name: "findings", Storage: StorageKeyValue, Template: TemplateState,
Options: map[string]json.RawMessage{"objects-max-mib": json.RawMessage(`512`)}}}, nil, ""},
// A field given twice at any depth is refused, named by its path: a decoder would keep the last without a word. // A field given twice at any depth is refused, named by its path: a decoder would keep the last without a word.
{"a field given twice", `[], "requires": ["postgres-database"]`, "", nil, nil, "the field requires is given twice"}, {"a field given twice", `[], "requires": ["postgres-database"]`, "", nil, nil, "the field requires is given twice"},
{"a field given twice deeper", `[{"name":"records","storage":"ledger","name":"other","template":"ledger"}]`, {"a field given twice deeper", `[{"name":"records","storage":"ledger","name":"other","template":"ledger"}]`,
@@ -115,11 +105,8 @@ func TestARequirementsLongForm(t *testing.T) {
} }
// Issue 1966: a ledger is no kept state; KeptStates names the old section's and the key-value entries' alone. // Issue 1966: a ledger is no kept state; KeptStates names the old section's and the key-value entries' alone.
var keeps []string var keeps []string
switch c.name { if c.name == "a key-value entry beside an old state" {
case "a key-value entry beside an old state":
keeps = []string{"holds", "seen"} keeps = []string{"holds", "seen"}
case "a finding stream and a state's objects cap":
keeps = []string{"findings"}
} }
if kept := KeptStates(got); !reflect.DeepEqual(kept, keeps) { if kept := KeptStates(got); !reflect.DeepEqual(kept, keeps) {
t.Errorf("%s: kept states %v, want %v", c.name, kept, keeps) t.Errorf("%s: kept states %v, want %v", c.name, kept, keeps)
go/catalogue/own_manifest_test.go
+7 -12
View File
@@ -8,7 +8,7 @@ import (
) )
// mesh-sdk's own manifest declares its two packages (design 32, SDK-10): the TypeScript `lib` and the Go `go`, built // mesh-sdk's own manifest declares its two packages (design 32, SDK-10): the TypeScript `lib` and the Go `go`, built
// from the directory go, each without a series (design 47 G §411), so the Go build module publishes mesh-sdk's Go package through the Go seat // from the directory go, each in series 0.1, so the Go build module publishes mesh-sdk's Go package through the Go seat
// (design 51). Its words are judged by the group's own indexer, delivered with them (ADR 0621 §4). // (design 51). Its words are judged by the group's own indexer, delivered with them (ADR 0621 §4).
func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) { func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
raw, err := os.ReadFile("../../module.json") raw, err := os.ReadFile("../../module.json")
@@ -19,25 +19,21 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("mesh-sdk's module.json does not parse: %v", err) t.Fatalf("mesh-sdk's module.json does not parse: %v", err)
} }
if m.Version != "0.1" || m.UnknownField() != "" {
t.Errorf("mesh-sdk's version is %q (unknown word %q), want 0.1 (design 47 G §368, §372)", m.Version, m.UnknownField())
}
if m.Build == nil || len(m.Build.Artifacts) == 0 { if m.Build == nil || len(m.Build.Artifacts) == 0 {
t.Fatal("mesh-sdk's module.json declares no artifact") t.Fatal("mesh-sdk's module.json declares no artifact")
} }
packages := map[string]Artifact{} packages := map[string]Artifact{}
for _, a := range m.Build.Artifacts { for _, a := range m.Build.Artifacts {
packages[a.Language] = a packages[a.Language] = a
if a.Kind != ArtifactPackage || a.Series != "" { if a.Kind != ArtifactPackage || a.Series != DefaultSeries {
t.Errorf("artifact %q is a %s in series %q, want a package in none (design 47 G §411)", a.Name, a.Kind, a.Series) t.Errorf("artifact %q is a %s in series %q, want a package in %s", a.Name, a.Kind, a.Series, DefaultSeries)
} }
} }
if g := packages["go"]; g.Name != "go" || !slices.Equal(g.Paths, []string{"go"}) { if g := packages["go"]; g.Name != "go" || !slices.Equal(g.Paths, []string{"go"}) {
t.Errorf("mesh-sdk's Go package is %+v, want go built from go", g) t.Errorf("mesh-sdk's Go package is %+v, want go built from go", g)
} }
// Its named paths, so the indexer holds them and a change to them is a changed package (design 47 G A-285). if l := packages["typescript"]; l.Name != "lib" || l.Paths != nil {
if l := packages["typescript"]; l.Name != "lib" || !slices.Equal(l.Paths, []string{"package.json", "package-lock.json", "tsconfig.json", "src"}) { t.Errorf("mesh-sdk's TypeScript package is %+v", l)
t.Errorf("mesh-sdk's TypeScript package is %+v, want lib built from package.json, package-lock.json, tsconfig.json and src", l)
} }
// What a reader reads (novox/hq ADR 0614 §4): the baseline is the contract of mesh-sdk f02f142, which every running // What a reader reads (novox/hq ADR 0614 §4): the baseline is the contract of mesh-sdk f02f142, which every running
@@ -50,9 +46,8 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
} }
} }
// The two seats the milestone renamed (ADR 0634 §9) are read by their new names; a node-engine built with the baseline // The two seats the milestone renamed (ADR 0634 §9) are read by their new names; a node-engine built with the baseline
// reads the old ones, and a manifest of the new names is the milestone's to deliver after the engines. The word // reads the old ones, and a manifest of the new names is the milestone's to deliver after the engines.
// `waits` is no longer read: novox/hq ADR 0670 §2 retired it. renamed := []string{"seat:mesh-enrolment", "seat:mesh-placement"}
renamed := []string{"seat:mesh-enrolment", "seat:mesh-placement", "word:waits"}
for _, b := range baseline { for _, b := range baseline {
if !slices.Contains(reads, b) && !slices.Contains(renamed, b) { if !slices.Contains(reads, b) && !slices.Contains(renamed, b) {
t.Errorf("the baseline's %s is no reader capability of this contract", b) t.Errorf("the baseline's %s is no reader capability of this contract", b)
go/catalogue/process_bus.go
+7 -54
View File
@@ -13,15 +13,12 @@ const ProcessBusField = "bus"
// ProcessBus is what one process of a module does on the bus (novox/hq ADR 0436 §1, §2, §3, §5): its own bus // ProcessBus is what one process of a module does on the bus (novox/hq ADR 0436 §1, §2, §3, §5): its own bus
// account, granted exactly this and nothing of the module's other grants or the tool runner's — never a seat // account, granted exactly this and nothing of the module's other grants or the tool runner's — never a seat
// to hold, never a restricted stream heard, never a warrant published (§3, §5). It answers a seat's verb only // to hold or serve, never a tool call or seat verb answered, never a restricted stream heard, never a
// where its resource declares `serves` beside this bus, never inside it (design 47 G A-115, A-127; ProcessServes). // warrant published (§3, §5).
type ProcessBus struct { type ProcessBus struct {
// Emits are the events this process emits, named locally, as the module's own Emits names them // Emits are the events this process emits, named locally, as the module's own Emits names them
// (EmitsAll): a process emits only what its module may emit as a whole. // (EmitsAll): a process emits only what its module may emit as a whole.
Emits []string `json:"emits,omitempty"` Emits []string `json:"emits,omitempty"`
// Consumes are the events this process hears, each as its module's `consumes` names it: only an event its module
// consumes (design 47 G A-340, after design 42 zfs-owns-zfs V5 and G1).
Consumes []string `json:"consumes,omitempty"`
// Queues are the module's own work-queue streams (ADR 0436 §6) this process consumes. // Queues are the module's own work-queue streams (ADR 0436 §6) this process consumes.
Queues []string `json:"queues,omitempty"` Queues []string `json:"queues,omitempty"`
// State is the module's own state, by local name, this process reads and writes. // State is the module's own state, by local name, this process reads and writes.
@@ -57,8 +54,6 @@ const (
type BusAccount struct { type BusAccount struct {
ID string ID string
Bus ProcessBus Bus ProcessBus
// Serves are the seat verbs the process answers (ProcessServes): empty for a tools bundle's account.
Serves []string
// Tools says it is a tools bundle's, run by the node's tool runtime, not a process the node runs. // Tools says it is a tools bundle's, run by the node's tool runtime, not a process the node runs.
Tools bool Tools bool
} }
@@ -68,11 +63,11 @@ type BusAccount struct {
// bundle's is read from the repository's build, or from the resolved manifest's bundles. // bundle's is read from the repository's build, or from the resolved manifest's bundles.
func (m Manifest) BusAccounts() []BusAccount { func (m Manifest) BusAccounts() []BusAccount {
var out []BusAccount var out []BusAccount
buses, serves := m.ProcessBuses(), m.ProcessServes() buses := m.ProcessBuses()
for _, r := range m.Resources { for _, r := range m.Resources {
id := fmt.Sprint(r["id"]) id := fmt.Sprint(r["id"])
if b, ok := buses[id]; ok && fmt.Sprint(r["type"]) == "process" { if b, ok := buses[id]; ok && fmt.Sprint(r["type"]) == "process" {
out = append(out, BusAccount{ID: id, Bus: b, Serves: serves[id]}) out = append(out, BusAccount{ID: id, Bus: b})
} }
} }
seen := map[string]bool{} seen := map[string]bool{}
@@ -93,8 +88,8 @@ func (m Manifest) BusAccounts() []BusAccount {
return out return out
} }
// UnmarshalJSON reads a process's bus strictly: an unknown key — a seat to hold or claim, or // UnmarshalJSON reads a process's bus strictly: an unknown key — serves, a seat to hold or claim, or
// anything else that would answer a tool call — is refused here, by name (novox/hq ADR // anything else that would answer a tool call or a seat's verb — is refused here, by name (novox/hq ADR
// 0436 §3). // 0436 §3).
func (b *ProcessBus) UnmarshalJSON(raw []byte) error { func (b *ProcessBus) UnmarshalJSON(raw []byte) error {
type plain ProcessBus type plain ProcessBus
@@ -102,7 +97,7 @@ func (b *ProcessBus) UnmarshalJSON(raw []byte) error {
dec := json.NewDecoder(bytes.NewReader(raw)) dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields() dec.DisallowUnknownFields()
if err := dec.Decode(&p); err != nil { if err := dec.Decode(&p); err != nil {
return fmt.Errorf("a process's bus is {emits, consumes, queues, state, asks, invokes, removes, keeps}; `serves` is a field of the process beside its bus (design 47 G A-127): %w", typedUnknown(err)) return fmt.Errorf("a process's bus is {emits, queues, state, asks, invokes, keeps}: %w", typedUnknown(err))
} }
*b = ProcessBus(p) *b = ProcessBus(p)
return nil return nil
@@ -144,45 +139,3 @@ func (m Manifest) ProcessBuses() map[string]ProcessBus {
} }
return out return out
} }
// ProcessServesField is the key a process resource declares the seat verbs it answers on, beside its `bus` and never inside
// it (design 47 G A-115, A-127).
const ProcessServesField = "serves"
// ProcessServes is the seat verbs each process of this module answers, by its id: each seat:<seat>.<verb>, read from the
// process resource's own `serves` (design 47 G A-115), whether or not the process declares a bus. A `serves` that is not a
// list of strings is read as none; the validator names it (ProcessServesProblem).
func (m Manifest) ProcessServes() map[string][]string {
var out map[string][]string
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "process" {
continue
}
list, err := ProcessServesOf(r)
if err != nil || len(list) == 0 {
continue
}
if out == nil {
out = map[string][]string{}
}
out[fmt.Sprint(r["id"])] = list
}
return out
}
// ProcessServesOf is the `serves` of one process resource, and an error where it is not a list of strings.
func ProcessServesOf(r map[string]any) ([]string, error) {
raw, present := r[ProcessServesField]
if !present {
return nil, nil
}
body, err := json.Marshal(raw)
if err != nil {
return nil, err
}
var list []string
if err := json.Unmarshal(body, &list); err != nil {
return nil, fmt.Errorf("a process's serves is a list of seat:<seat>.<verb>: %w", err)
}
return list, nil
}
go/catalogue/process_bus_test.go
-40
View File
@@ -1,10 +1,5 @@
package catalogue package catalogue
import (
"encoding/json"
"testing"
)
func strSliceEqual(a, b []string) bool { func strSliceEqual(a, b []string) bool {
if len(a) != len(b) { if len(a) != len(b) {
return false return false
@@ -16,38 +11,3 @@ func strSliceEqual(a, b []string) bool {
} }
return true return true
} }
// Design 47 G A-115, A-127: `serves` is a field of the process beside its bus; inside the bus it is refused.
func TestProcessServesIsReadBesideTheBusAndRefusedInsideIt(t *testing.T) {
if err := json.Unmarshal([]byte(`{"serves":["seat:mesh-broker.list"]}`), &ProcessBus{}); err == nil {
t.Error("a serves inside bus was accepted")
}
if err := json.Unmarshal([]byte(`{"holds":["x"]}`), &ProcessBus{}); err == nil {
t.Error("an unknown key was accepted")
}
// Design 47 G A-340: a process's bus names the events it hears under `consumes`.
var heard ProcessBus
if err := json.Unmarshal([]byte(`{"consumes":["mesh-conditions.findings-asked"]}`), &heard); err != nil ||
!strSliceEqual(heard.Consumes, []string{"mesh-conditions.findings-asked"}) {
t.Errorf("a bus's consumes read as %v (%v)", heard.Consumes, err)
}
m := Manifest{Module: "nats", Resources: []map[string]any{
{"id": "keeper", "type": "process", "serves": []any{"seat:mesh-broker.snapshot-get", "seat:mesh-broker.list"},
"bus": map[string]any{"keeps": true}},
{"id": "plain", "type": "process", "serves": []any{"seat:mesh-broker.snapshot-remove"}},
{"id": "mute", "type": "process"},
}}
got := m.ProcessServes()
if !strSliceEqual(got["keeper"], []string{"seat:mesh-broker.snapshot-get", "seat:mesh-broker.list"}) ||
!strSliceEqual(got["plain"], []string{"seat:mesh-broker.snapshot-remove"}) || len(got) != 2 {
t.Errorf("process serves = %v", got)
}
accounts := m.BusAccounts()
if len(accounts) != 1 || !strSliceEqual(accounts[0].Serves, got["keeper"]) {
t.Errorf("the bus account's serves = %+v", accounts)
}
bad := Manifest{Resources: []map[string]any{{"id": "x", "type": "process", "serves": "seat:a.b"}}}
if _, err := ProcessServesOf(bad.Resources[0]); err == nil {
t.Error("a serves that is not a list was accepted")
}
}
go/catalogue/reader_release_test.go
+2 -4
View File
@@ -16,8 +16,7 @@ func TestTheReaderReleaseReadsTheManifestWordsGWrites(t *testing.T) {
m, err := decodeForTest([]byte(`{"module":"mesh-watcher","version":"1.4","needs-approval":["restart","node-engine.composition","finding-written"],` + m, err := decodeForTest([]byte(`{"module":"mesh-watcher","version":"1.4","needs-approval":["restart","node-engine.composition","finding-written"],` +
`"consumes":["*.finding-written","mesh-conditions.findings-asked"],` + `"consumes":["*.finding-written","mesh-conditions.findings-asked"],` +
`"streams":[{"name":"mesh-watcher-findings","events":["finding-written","finding-cleared","findings-stated"],"max-age-days":7,"max-mib":16}],` + `"streams":[{"name":"mesh-watcher-findings","events":["finding-written","finding-cleared","findings-stated"],"max-age-days":7,"max-mib":16}],` +
`"build":{"artifacts":[{"name":"program","kind":"bundle","language":"go","needs":["mesh-grants:program","node-tools:runtime"]},` + `"build":{"artifacts":[{"name":"program","kind":"bundle","language":"go","needs":["mesh-grants:program","node-tools:runtime"]}]}}`))
`{"name":"go","kind":"package","language":"go","calls":["put-sessions","finding-calls"]}]}}`))
if err != nil { if err != nil {
t.Fatalf("a manifest with G's words was refused: %v", err) t.Fatalf("a manifest with G's words was refused: %v", err)
} }
@@ -25,8 +24,7 @@ func TestTheReaderReleaseReadsTheManifestWordsGWrites(t *testing.T) {
t.Fatalf("a word was set aside: %v (needs-approval %v)", m.UnknownField(), m.NeedsApproval) t.Fatalf("a word was set aside: %v (needs-approval %v)", m.UnknownField(), m.NeedsApproval)
} }
if m.Version != "1.4" || m.Consumes[0] != "*.finding-written" || m.Streams[0].Name != "mesh-watcher-findings" || if m.Version != "1.4" || m.Consumes[0] != "*.finding-written" || m.Streams[0].Name != "mesh-watcher-findings" ||
len(m.Build.Artifacts[0].Needs) != 2 || m.Build.Artifacts[0].Needs[0] != "mesh-grants:program" || len(m.Build.Artifacts[0].Needs) != 2 || m.Build.Artifacts[0].Needs[0] != "mesh-grants:program" {
len(m.Build.Artifacts[1].Calls) != 2 || m.Build.Artifacts[1].Calls[0] != "put-sessions" {
t.Fatalf("read as %+v", m) t.Fatalf("read as %+v", m)
} }
} }
go/catalogue/replays_test.go
+36
View File
@@ -0,0 +1,36 @@
package catalogue
import (
"slices"
"testing"
)
// **R298 — a seat's new verb lands in either repository first** (novox/hq issue 298). On 2026-10-07 the
// delivery seat gained `checks`: the seat is defined here and held by a module in the catalogue, and
// neither could merge first — a controller promising the verb refused the holder that did not serve it
// yet, and a holder serving it was refused by the controller that did not promise it. The outcome
// asserted: the holder as it was, and the holder serving `checks` too, both hold the delivery seat.
//
// Written only with what the catalogue package had before the fix (SeatNamed, VerbNames, CanHold), so the
// prover can lay it over that commit.
func TestReplay298ADeliverySeatHolderHoldsBeforeAndAfterItServesChecks(t *testing.T) {
seat, ok := SeatNamed(DeliverySeat)
if !ok {
t.Fatal("the delivery seat is not in the set")
}
var before []string
for _, v := range VerbNames(seat.Serves) {
if v != "checks" {
before = append(before, v)
}
}
holder := func(serves []string) Manifest {
return Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh, Serves: serves}}}
}
if err := CanHold(holder(before), seat); err != nil {
t.Fatalf("the holder as the catalogue had it, before it serves checks, is refused: %v", err)
}
if err := CanHold(holder(append(slices.Clone(before), "checks")), seat); err != nil {
t.Fatalf("the holder serving checks is refused — the catalogue cannot merge first: %v", err)
}
}
go/catalogue/seat_from_definition.go
+1 -7
View File
@@ -23,19 +23,13 @@ func SeatFromDefinition(d seatdef.Definition) Seat {
for _, v := range d.Verbs { for _, v := range d.Verbs {
s.Serves = append(s.Serves, Verb{Name: v.Name, Description: v.Description, s.Serves = append(s.Serves, Verb{Name: v.Name, Description: v.Description,
Input: seatdef.ResolveRefs(v.Input, d.Defs), Output: seatdef.ResolveRefs(v.Answer, d.Defs), Input: seatdef.ResolveRefs(v.Input, d.Defs), Output: seatdef.ResolveRefs(v.Answer, d.Defs),
Optional: v.Optional, Replaces: v.Replaces, NeedsApproval: v.NeedsApproval}) Optional: v.Optional, Replaces: v.Replaces})
} }
for _, st := range d.States { for _, st := range d.States {
s.States = append(s.States, st.Name)
if st.View { if st.View {
s.Views = append(s.Views, st.Name) s.Views = append(s.Views, st.Name)
} }
} }
for _, a := range d.Accepts {
if a.Stream != "" {
s.AcceptedOn = append(s.AcceptedOn, a.Stream+"."+a.Name)
}
}
for _, r := range d.Receives { for _, r := range d.Receives {
rv := Receivable{Kind: r.Kind, Comment: r.Comment, Dirs: r.Dirs, Offered: r.Offered} rv := Receivable{Kind: r.Kind, Comment: r.Comment, Dirs: r.Dirs, Offered: r.Offered}
if r.Shape != nil { if r.Shape != nil {
go/catalogue/seat_invokes.go
-12
View File
@@ -18,15 +18,3 @@ func SeatInvoke(invoke string) (seat, verb string, ok bool) {
seat, verb, ok = strings.Cut(rest, ".") seat, verb, ok = strings.Cut(rest, ".")
return seat, verb, ok && seat != "" && verb != "" return seat, verb, ok && seat != "" && verb != ""
} }
// QueueInvoke splits an invoke `seat:<seat>.accept.<queue>`, by which an asker names a build seat's queue it asks (design 47 G
// §544, §601, A-231), into its seat and queue; ok is false for anything else. The one reader of that form: the manifest
// validator, the indexer's served-verb edge and the grants part read it here.
func QueueInvoke(invoke string) (seat, queue string, ok bool) {
seat, verb, ok := SeatInvoke(invoke)
if !ok {
return "", "", false
}
queue, ok = strings.CutPrefix(verb, "accept.")
return seat, queue, ok && queue != "" && !strings.Contains(queue, ".")
}
go/catalogue/seat_pass.go
+6 -67
View File
@@ -5,36 +5,14 @@ import (
"sort" "sort"
) )
// An assignment as the passing of a seat reads it: a module on a node, with its placement's watched branch (design 47 G // An assignment as the passing of a seat reads it: a module on a machine, with the manifest of the build that machine
// §459). The build it runs is resolved by SeatsToPassOn from what the caller heard and its reader of entries. // runs, else the trunk's (novox/hq ADR 0660 §2).
type Assignment struct { type Assignment struct {
Node string
Module string
// Branch is the watched branch of the assignment's placement, whose newest entry of the module is taken where the node
// said no `running` for it (§199).
Branch string
}
// Heard is what the caller heard of a module on a node from the node-engine seat's `running`: a fingerprint, or Stopped
// (`stopped: true` and no fingerprint, §404). Nothing heard since its consumer began is no Heard at all.
type Heard struct {
Fingerprint string
Stopped bool
}
// EntryReader is the caller's view of the catalog half (design 47 G §459, §443): the entry's manifest by its
// fingerprint, and a module's entry on a watched branch's newest revision. ok is false where it answers no entry.
type EntryReader interface {
ByFingerprint(fingerprint string) (m Manifest, ok bool)
OnBranch(module, branch string) (m Manifest, ok bool)
}
// resolved is an assignment with the manifest of the build it runs. Unread is one whose entry the reader did not answer:
// it is still assigned, so its holdings stand, and it claims nothing.
type resolved struct {
Node string Node string
Manifest Manifest Manifest Manifest
Unread bool // Unread is an assignment whose manifest could not be read at all: it is still assigned, so its holdings stand,
// and it claims nothing.
Unread bool
} }
// SeatPass is one seat whose holder on record has left it, and the only assignment that can take it. // SeatPass is one seat whose holder on record has left it, and the only assignment that can take it.
@@ -55,46 +33,7 @@ type SeatPass struct {
// be read. Only mesh seats held once are read: a replicated seat's holders are added by an act (ADR 0223), a kinded // be read. Only mesh seats held once are read: a replicated seat's holders are added by an act (ADR 0223), a kinded
// bench is held by its claims, and a seat with several holders on record is a store that disagrees with the mesh. // bench is held by its claims, and a seat with several holders on record is a store that disagrees with the mesh.
// Nothing here records a pass (ADR 0660 §4). // Nothing here records a pass (ADR 0660 §4).
// func SeatsToPassOn(holdings []Held, assignments []Assignment) ([]SeatPass, []string) {
// The assignments' builds are resolved here, by one rule for both callers (design 47 G §199, §459, §404): the heard
// fingerprint's entry; with nothing heard, the module's entry on its placement's branch; a `stopped` module claims nothing
// on that node; and where the reader answers no entry, no seat that assignment could claim passes, the holder on record
// stays, and the answer says it, naming the node, the module and the fingerprint. The function reads no bus.
func SeatsToPassOn(holdings []Held, assignments []Assignment, heard func(node, module string) (Heard, bool),
entries EntryReader) ([]SeatPass, []string) {
var rs []resolved
var said []string
for _, a := range assignments {
h, ok := Heard{}, false
if heard != nil {
h, ok = heard(a.Node, a.Module)
}
if ok && h.Stopped {
rs = append(rs, resolved{Node: a.Node, Manifest: Manifest{Module: a.Module}})
continue
}
var m Manifest
var found bool
fp := "none heard, the newest on " + a.Branch
if ok {
fp = h.Fingerprint
m, found = entries.ByFingerprint(h.Fingerprint)
} else {
m, found = entries.OnBranch(a.Module, a.Branch)
}
if !found {
said = append(said, fmt.Sprintf("no entry is answered for %s on %s (%s): no seat it could claim passes and the holder on record stays",
a.Module, a.Node, fp))
rs = append(rs, resolved{Node: a.Node, Manifest: Manifest{Module: a.Module}, Unread: true})
continue
}
rs = append(rs, resolved{Node: a.Node, Manifest: m})
}
passes, more := seatsToPass(holdings, rs)
return passes, append(said, more...)
}
func seatsToPass(holdings []Held, assignments []resolved) ([]SeatPass, []string) {
bySeat := map[string][]Held{} bySeat := map[string][]Held{}
var order []string var order []string
for _, h := range holdings { for _, h := range holdings {
go/catalogue/seat_pass_test.go
+15 -72
View File
@@ -1,7 +1,6 @@
package catalogue package catalogue
import ( import (
"fmt"
"strings" "strings"
"testing" "testing"
) )
@@ -28,51 +27,6 @@ func recorded(node, module string) []Held {
return []Held{{Claim: "mesh-broker", Scope: ScopeMesh, Node: node, Module: module}} return []Held{{Claim: "mesh-broker", Scope: ScopeMesh, Node: node, Module: module}}
} }
// asg is a case's assignment in the earlier form: a node with the manifest of the build it runs, which the harness
// hears as a fingerprint and serves through a reader of entries; Unread is an entry the reader does not answer.
type asg struct {
Node string
Manifest Manifest
Unread bool
// Silent is a node that said no `running`: the module's entry on its branch is taken. Stopped says it stopped.
Silent, Stopped bool
}
type entriesFixture struct{ byFP, byBranch map[string]Manifest }
func (e entriesFixture) ByFingerprint(fp string) (Manifest, bool) { m, ok := e.byFP[fp]; return m, ok }
func (e entriesFixture) OnBranch(module, branch string) (Manifest, bool) {
m, ok := e.byBranch[module+"@"+branch]
return m, ok
}
func passOn(rec []Held, as []asg) ([]SeatPass, []string) {
e := entriesFixture{map[string]Manifest{}, map[string]Manifest{}}
var assignments []Assignment
heard := map[string]Heard{}
for i, a := range as {
fp := fmt.Sprintf("fp%d", i)
assignments = append(assignments, Assignment{Node: a.Node, Module: a.Manifest.Module, Branch: "trunk"})
switch {
case a.Stopped:
heard[a.Node+"/"+a.Manifest.Module] = Heard{Stopped: true}
case a.Silent:
if !a.Unread {
e.byBranch[a.Manifest.Module+"@trunk"] = a.Manifest
}
default:
heard[a.Node+"/"+a.Manifest.Module] = Heard{Fingerprint: fp}
if !a.Unread {
e.byFP[fp] = a.Manifest
}
}
}
return SeatsToPassOn(rec, assignments, func(node, module string) (Heard, bool) {
h, ok := heard[node+"/"+module]
return h, ok
}, e)
}
// novox/hq ADR 0660 "How it is checked", and ADR 0449 §4's original case. // novox/hq ADR 0660 "How it is checked", and ADR 0449 §4's original case.
func TestSeatsToPassOn(t *testing.T) { func TestSeatsToPassOn(t *testing.T) {
passSeat(t) passSeat(t)
@@ -80,57 +34,46 @@ func TestSeatsToPassOn(t *testing.T) {
cases := []struct { cases := []struct {
name string name string
rec []Held rec []Held
as []asg as []Assignment
to string // "node/module" of the one pass, "" for none to string // "node/module" of the one pass, "" for none
saidW []string saidW []string
}{ }{
{"holder assigned nowhere, one claimant (ADR 0449 §4)", rec, {"holder assigned nowhere, one claimant (ADR 0449 §4)", rec,
[]asg{{Node: "novox", Manifest: broker("new")}}, "novox/new", nil}, []Assignment{{Node: "novox", Manifest: broker("new")}}, "novox/new", nil},
{"holder assigned but its manifest dropped the claim", rec, {"holder assigned but its manifest dropped the claim", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "novox", Manifest: broker("new")}}, "novox/new", nil}, []Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "novox", Manifest: broker("new")}}, "novox/new", nil},
{"holder claims but cannot hold the seat", rec, {"holder claims but cannot hold the seat", rec,
[]asg{{Node: "novox", Manifest: Manifest{Module: "old", Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}}}}, []Assignment{{Node: "novox", Manifest: Manifest{Module: "old", Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}}}},
{Node: "ace", Manifest: broker("new")}}, "ace/new", nil}, {Node: "ace", Manifest: broker("new")}}, "ace/new", nil},
{"the running build still claims: the holder keeps the seat", rec, {"the running build still claims: the holder keeps the seat", rec,
[]asg{{Node: "novox", Manifest: broker("old")}, {Node: "novox", Manifest: broker("new")}}, "", nil}, []Assignment{{Node: "novox", Manifest: broker("old")}, {Node: "novox", Manifest: broker("new")}}, "", nil},
{"one of the holder's two nodes still claims it", rec, {"one of the holder's two nodes still claims it", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: broker("old")}, []Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: broker("old")},
{Node: "g14", Manifest: broker("new")}}, "", nil}, {Node: "g14", Manifest: broker("new")}}, "", nil},
{"holder's manifest unread: still assigned", rec, {"holder's manifest unread: still assigned", rec,
[]asg{{Node: "novox", Manifest: Manifest{Module: "old"}, Unread: true}, {Node: "novox", Manifest: broker("new")}}, "", []string{"no entry is answered for old on novox"}}, []Assignment{{Node: "novox", Manifest: Manifest{Module: "old"}, Unread: true}, {Node: "novox", Manifest: broker("new")}}, "", nil},
{"an unread assignment is no claimant", rec, {"an unread assignment is no claimant", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: Manifest{Module: "new"}, Unread: true}}, "", []string{"no entry is answered for new on ace"}}, []Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: Manifest{Module: "new"}, Unread: true}}, "", nil},
{"a claimant whose running manifest does not claim is not counted", rec, {"a claimant whose running manifest does not claim is not counted", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: dropped("new")}}, "", nil}, []Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: dropped("new")}}, "", nil},
{"no claimant keeps the record", rec, {"no claimant keeps the record", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "novox", Manifest: dropped("other")}}, "", nil}, []Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "novox", Manifest: dropped("other")}}, "", nil},
{"holder gone, no claimant keeps the record", rec, nil, "", nil}, {"holder gone, no claimant keeps the record", rec, nil, "", nil},
{"one module on two nodes is two claimants: none passed, said", rec, {"one module on two nodes is two claimants: none passed, said", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: broker("new")}, {Node: "novox", Manifest: broker("new")}}, []Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: broker("new")}, {Node: "novox", Manifest: broker("new")}},
"", []string{"old", "new on ace", "new on novox", "seat mesh-broker --to <node>/<module>"}}, "", []string{"old", "new on ace", "new on novox", "seat mesh-broker --to <node>/<module>"}},
{"two claimants, holder gone: said", rec, {"two claimants, holder gone: said", rec,
[]asg{{Node: "novox", Manifest: broker("a")}, {Node: "ace", Manifest: broker("b")}}, []Assignment{{Node: "novox", Manifest: broker("a")}, {Node: "ace", Manifest: broker("b")}},
"", []string{"a on novox", "b on ace", "seat mesh-broker --to <node>/<module>"}}, "", []string{"a on novox", "b on ace", "seat mesh-broker --to <node>/<module>"}},
{"a seat with two holders on record is not read", append(rec[:1:1], Held{Claim: "mesh-broker", Scope: ScopeMesh, Node: "ace", Module: "x"}), {"a seat with two holders on record is not read", append(rec[:1:1], Held{Claim: "mesh-broker", Scope: ScopeMesh, Node: "ace", Module: "x"}),
[]asg{{Node: "novox", Manifest: broker("new")}}, "", nil}, []Assignment{{Node: "novox", Manifest: broker("new")}}, "", nil},
// Design 47 G §459, §199, §404.
{"nothing heard: the module's entry on its branch is taken", rec,
[]asg{{Node: "novox", Manifest: dropped("old"), Silent: true}, {Node: "ace", Manifest: broker("new"), Silent: true}}, "ace/new", nil},
{"nothing heard of the holder: its entry on the branch still claims, so it keeps the seat", rec,
[]asg{{Node: "novox", Manifest: broker("old"), Silent: true}, {Node: "ace", Manifest: broker("new")}}, "", nil},
{"a stopped holder passes no seat to itself: the one claimant takes it", rec,
[]asg{{Node: "novox", Manifest: broker("old"), Stopped: true}, {Node: "ace", Manifest: broker("new")}}, "ace/new", nil},
{"a stopped module claims nothing", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: broker("new"), Stopped: true}}, "", nil},
{"an entry not answered keeps the holder on record and says it", rec,
[]asg{{Node: "novox", Manifest: dropped("old"), Unread: true}, {Node: "ace", Manifest: broker("new")}}, "", []string{"no entry is answered for old on novox (fp0)"}},
{"a replicated seat is never passed", {"a replicated seat is never passed",
[]Held{{Claim: "mesh-dns-resolver", Scope: ScopeMesh, Node: "novox", Module: "gone"}}, []Held{{Claim: "mesh-dns-resolver", Scope: ScopeMesh, Node: "novox", Module: "gone"}},
[]asg{{Node: "novox", Manifest: Manifest{Module: "r", Claims: []Claim{{Name: "mesh-dns-resolver", Scope: ScopeMesh}}}}}, "", nil}, []Assignment{{Node: "novox", Manifest: Manifest{Module: "r", Claims: []Claim{{Name: "mesh-dns-resolver", Scope: ScopeMesh}}}}}, "", nil},
} }
for _, c := range cases { for _, c := range cases {
t.Run(c.name, func(t *testing.T) { t.Run(c.name, func(t *testing.T) {
passes, said := passOn(c.rec, c.as) passes, said := SeatsToPassOn(c.rec, c.as)
if c.to == "" && len(passes) != 0 || c.to != "" && len(passes) != 1 { if c.to == "" && len(passes) != 0 || c.to != "" && len(passes) != 1 {
t.Fatalf("passes=%+v said=%v", passes, said) t.Fatalf("passes=%+v said=%v", passes, said)
} }
go/catalogue/seats.go
+9 -88
View File
@@ -2,9 +2,7 @@ package catalogue
import ( import (
"fmt" "fmt"
"slices"
"strings" "strings"
"sync/atomic"
) )
// Seat is one role the mesh defines. // Seat is one role the mesh defines.
@@ -47,15 +45,9 @@ type Seat struct {
// Serves carries each verb in full — name, description, schema — because a role's tools are the // Serves carries each verb in full — name, description, schema — because a role's tools are the
// mesh's to define and an agent's to call (novox/hq ADR 0132, design 33 §2). // mesh's to define and an agent's to call (novox/hq ADR 0132, design 33 §2).
Serves []Verb Serves []Verb
// States are the names of the state the seat's holder keeps, which a module reads by `seat:<seat>.<state>` (design 47 G // Views are the names of the states the seat offers as views, which a module follows by `consumes`
// A-116, A-128). Compiled from the definition's states. // `seat:<seat>.views.<state>` (design 47 G §616, §699). Compiled from the definition's states marked `view`.
States []string
// Views are the names of those states the seat offers as views, which a module follows by `consumes`
// `seat:<seat>.views.<state>` (design 47 G §616, §699, A-131). Compiled from the definition's states marked `view`.
Views []string Views []string
// AcceptedOn are the accepted kinds that ride a stream, each as `<stream>.<kind>`, which a part names in `invokes` as
// `seat:<seat>.<stream>.<kind>` (design 47 G A-102, §415). Compiled from the definition's accepts.
AcceptedOn []string
// Receives is what other modules may contribute to the seat's holder, by kind (novox/hq ADR // Receives is what other modules may contribute to the seat's holder, by kind (novox/hq ADR
// 0212): each kind is text in the tool's own grammar, placed by the holder with // 0212): each kind is text in the tool's own grammar, placed by the holder with
// ${contribution:<seat>:<kind>}. Compiled, never stored: like the protocol, it is the mesh's // ${contribution:<seat>:<kind>}. Compiled, never stored: like the protocol, it is the mesh's
@@ -98,15 +90,7 @@ type Receivable struct {
// seat list and imports no contracts module (novox/hq ADR 0635 §1). A host reads the list from the bus // seat list and imports no contracts module (novox/hq ADR 0635 §1). A host reads the list from the bus
// (mesh-sdk go/seatlist, which holds the foundation's bootstrapped definitions until the indexer is heard) and gives // (mesh-sdk go/seatlist, which holds the foundation's bootstrapped definitions until the indexer is heard) and gives
// its seats here at start and at each change. // its seats here at start and at each change.
var seats atomic.Pointer[[]Seat] var seats []Seat
// working is the set in force, none before a host gives one.
func working() []Seat {
if p := seats.Load(); p != nil {
return *p
}
return nil
}
// UseSeats replaces the working set with the seat list a host read (seatlist.Reader.Seats, via SeatsFromDefinitions). // UseSeats replaces the working set with the seat list a host read (seatlist.Reader.Seats, via SeatsFromDefinitions).
// //
@@ -117,8 +101,7 @@ func UseSeats(s []Seat) {
if len(s) == 0 { if len(s) == 0 {
return return
} }
set := append([]Seat(nil), s...) seats = append([]Seat(nil), s...)
seats.Store(&set)
} }
// aliases maps a seat's former names to its current canonical name (novox/hq ADR 0122). Loaded from // aliases maps a seat's former names to its current canonical name (novox/hq ADR 0122). Loaded from
@@ -128,20 +111,20 @@ var aliases = map[string]string{}
// Seats is every seat the mesh defines, in reading order. // Seats is every seat the mesh defines, in reading order.
func Seats() []Seat { func Seats() []Seat {
return append([]Seat(nil), working()...) return append([]Seat(nil), seats...)
} }
// SeatNamed is the seat a name refers to, whether that is its current name or one it used to have // SeatNamed is the seat a name refers to, whether that is its current name or one it used to have
// (novox/hq ADR 0122). A former name resolves to the seat's canonical row, so a rename breaks no // (novox/hq ADR 0122). A former name resolves to the seat's canonical row, so a rename breaks no
// reference to the old name. // reference to the old name.
func SeatNamed(name string) (Seat, bool) { func SeatNamed(name string) (Seat, bool) {
for _, s := range working() { for _, s := range seats {
if s.Name == name { if s.Name == name {
return s, true return s, true
} }
} }
if canonical, aliased := aliases[name]; aliased { if canonical, aliased := aliases[name]; aliased {
for _, s := range working() { for _, s := range seats {
if s.Name == canonical { if s.Name == canonical {
return s, true return s, true
} }
@@ -155,7 +138,7 @@ func SeatDelivering(provision string) (Seat, bool) {
if provision == "" { if provision == "" {
return Seat{}, false return Seat{}, false
} }
for _, s := range working() { for _, s := range seats {
if s.Delivers == provision { if s.Delivers == provision {
return s, true return s, true
} }
@@ -202,9 +185,6 @@ func CanHold(m Manifest, seat Seat) error {
"serves or among its own tools", "serves or among its own tools",
m.Module, seat.Name, strings.Join(missing, ", ")) m.Module, seat.Name, strings.Join(missing, ", "))
} }
if err := approvalsStateRefusal(m, seat); err != nil {
return err
}
// And nothing the seat does not promise: a verb named here that the protocol lacks is served // And nothing the seat does not promise: a verb named here that the protocol lacks is served
// to nobody, which is a typo the holder would otherwise discover as a caller's timeout. // to nobody, which is a typo the holder would otherwise discover as a caller's timeout.
if extra := unpromised(claimed.Serves, seat.Serves); len(extra) > 0 { if extra := unpromised(claimed.Serves, seat.Serves); len(extra) > 0 {
@@ -215,65 +195,6 @@ func CanHold(m Manifest, seat Seat) error {
return nil return nil
} }
// approvalsStateRefusal is why a process of m that serves a marked verb of seat cannot hold the gate, or nothing (design 47 G
// §677, A-270): only Go holds the gate, and its template state `approvals` (a `requires` entry) is declared with `key-ttl`,
// since its bound keys `unanswered.<call>` need it. A marked verb served by the tool runner's bundle needs neither.
func approvalsStateRefusal(m Manifest, seat Seat) error {
var marked []string
for _, v := range seat.Serves {
if v.NeedsApproval {
marked = append(marked, "seat:"+seat.Name+"."+v.Name)
}
}
if len(marked) == 0 {
return nil
}
for id, serves := range m.ProcessServes() {
var verbs []string
for _, s := range serves {
if slices.Contains(marked, s) {
verbs = append(verbs, s)
}
}
if len(verbs) == 0 {
continue
}
if language := processLanguage(m, id); language != "" && !compilesToABinary(language) {
return fmt.Errorf("%s serves %s from its process %s written in %s: only Go holds the gate of a marked verb "+
"(design 47 G §677)", m.Module, strings.Join(verbs, ", "), id, language)
}
kept := false
for _, e := range m.BusState {
if e.Name == "approvals" && e.Template == TemplateState && string(e.Options[OptionLifetimePerKey]) == "true" {
kept = true
}
}
if !kept {
return fmt.Errorf("%s serves %s from its process %s but declares no state `approvals` with the option key-ttl "+
"(design 47 G §677, A-270): a holder of a marked verb keeps its calls' bound keys there", m.Module, strings.Join(verbs, ", "), id)
}
}
return nil
}
// processLanguage is the language of the artifact the process resource id runs, or empty where the manifest names none.
func processLanguage(m Manifest, id string) string {
if m.Build == nil {
return ""
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "process" || fmt.Sprint(r["id"]) != id {
continue
}
for _, a := range m.Build.Artifacts {
if a.Name == fmt.Sprint(r["artifact"]) {
return a.Language
}
}
}
return ""
}
func providesAt(m Manifest, provision, scope string) bool { func providesAt(m Manifest, provision, scope string) bool {
for _, o := range m.Provides { for _, o := range m.Provides {
if o.Name == provision && o.At() == scope { if o.Name == provision && o.At() == scope {
@@ -290,7 +211,7 @@ func providesAt(m Manifest, provision, scope string) bool {
// nothing on the bus and needs no queue. // nothing on the bus and needs no queue.
func SeatsWithAProtocol() []Seat { func SeatsWithAProtocol() []Seat {
var out []Seat var out []Seat
for _, s := range working() { for _, s := range seats {
if len(s.Accepts) > 0 || len(s.Emits) > 0 || len(s.Serves) > 0 { if len(s.Accepts) > 0 || len(s.Emits) > 0 || len(s.Serves) > 0 {
out = append(out, s) out = append(out, s)
} }
go/catalogue/state.go
+6 -21
View File
@@ -36,7 +36,7 @@ type StateDeclaration struct {
// `spent`). The judge's rule and the grants follow from it. // `spent`). The judge's rule and the grants follow from it.
ReadOnly bool `json:"read-only,omitempty"` ReadOnly bool `json:"read-only,omitempty"`
// MaxMiB is the bucket's cap in MiB, between ObjectsLeastMiB and ObjectsMostMiB, as an old objects section's // MaxMiB is the bucket's cap in MiB, between ObjectsLeastMiB and ObjectsMostMiB, as an old objects section's
// is (design 47 G §469, §695); zero is no cap, as on main. // is (design 47 G §469, §494); zero is no cap, as before.
MaxMiB int `json:"max-mib,omitempty"` MaxMiB int `json:"max-mib,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47). // KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout KeptLayout
@@ -78,30 +78,15 @@ func (s StateDeclaration) MarshalJSON() ([]byte, error) {
return json.Marshal(plain(s)) return json.Marshal(plain(s))
} }
// SeatReadPrefix opens a read of the state a seat's holder keeps: `seat:<seat>.<state>` (design 47 G A-116, A-128). // SeatViewPrefix opens a consumed view of a seat, `seat:<seat>.views.<state>` (design 47 G §616).
const SeatReadPrefix = "seat:" const SeatViewPrefix = "seat:"
// SeatRead splits a read of a seat holder's state, `seat:<seat>.<state>`, into the seat and the local name; ok is false for a
// read that is not of this kind (a module's `<module>.<state>`, which ReadState splits), and err says why a `seat:` read is
// malformed. The holder is not in it: it is resolved where the read is made (seatholder), so a new holder is read with no
// change of the manifest.
func SeatRead(read string) (seat, local string, ok bool, err error) {
rest, isSeat := strings.CutPrefix(read, SeatReadPrefix)
if !isSeat {
return "", "", false, nil
}
at := strings.LastIndex(rest, ".")
if at <= 0 || at == len(rest)-1 || !stateName.MatchString(rest[:at]) || !stateName.MatchString(rest[at+1:]) {
return "", "", true, fmt.Errorf("%q does not name a seat and its state: a read of a seat holder's state is seat:<seat>.<name>", read)
}
return rest[:at], rest[at+1:], true, nil
}
// SeatViewConsume splits a consumed view of a seat, `seat:<seat>.views.<state>`, into the seat and the state; ok is false for a // SeatViewConsume splits a consumed view of a seat, `seat:<seat>.views.<state>`, into the seat and the state; ok is false for a
// consumed entry that is not of this kind (an event, `<emitter>.<event>`), and err says why a `seat:` entry is malformed. The // consumed entry that is not of this kind (an event, `<emitter>.<event>`), and err says why a `seat:` entry is malformed. The
// holder is not in it: the grants part resolves it from the catalog half (design 47 G §616, A-131). // holder is not in it: it is resolved where the view is followed, so a new holder is followed with no change of the
// manifest (design 47 G §616, §621).
func SeatViewConsume(consume string) (seat, state string, ok bool, err error) { func SeatViewConsume(consume string) (seat, state string, ok bool, err error) {
rest, isSeat := strings.CutPrefix(consume, SeatReadPrefix) rest, isSeat := strings.CutPrefix(consume, SeatViewPrefix)
if !isSeat { if !isSeat {
return "", "", false, nil return "", "", false, nil
} }
go/catalogue/state_test.go
-65
View File
@@ -4,8 +4,6 @@ import (
"encoding/json" "encoding/json"
"strings" "strings"
"testing" "testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
) )
// A module declares the state it keeps and the state it reads (novox/hq ADR 0201), a bucket by its // A module declares the state it keeps and the state it reads (novox/hq ADR 0201), a bucket by its
@@ -48,71 +46,8 @@ func TestAStateMayHaveKeysThatExpireOneByOne(t *testing.T) {
if string(out) != `[{"name":"pulse","key-ttl":true},"stuck",{"name":"seen","history":5},{"name":"spent","read-only":true}]` { if string(out) != `[{"name":"pulse","key-ttl":true},"stuck",{"name":"seen","history":5},{"name":"spent","read-only":true}]` {
t.Fatalf("written back as %s", out) t.Fatalf("written back as %s", out)
} }
// G §469, §695: an old state entry takes max-mib, written back, bounded as an objects section's.
capped, err := decodeForTest([]byte(`{"module":"x","version":"1","state":[{"name":"a","max-mib":256},"b"]}`))
if err != nil || capped.State[0].MaxMiB != 256 || capped.State[1].MaxMiB != 0 {
t.Fatalf("max-mib read as %+v, %v", capped.State, err)
}
if out, _ := json.Marshal(capped.State); string(out) != `[{"name":"a","max-mib":256},"b"]` {
t.Fatalf("max-mib written back as %s", out)
}
for _, bad := range []string{"0", "-1", "8193"} {
if _, err := decodeForTest([]byte(`{"module":"x","version":"1","state":[{"name":"a","max-mib":` + bad + `}]}`)); err == nil {
t.Errorf("max-mib %s was read", bad)
}
}
if _, err := decodeForTest([]byte(`{"module":"x","version":"1","state":[{"name":"a","max-mib":8192}]}`)); err != nil {
t.Errorf("max-mib 8192 refused: %v", err)
}
if _, err := decodeForTest([]byte(`{"module":"nats","version":"1","state":[{"name":"spent","read-only":true,"read-only":false}]}`)); err == nil || if _, err := decodeForTest([]byte(`{"module":"nats","version":"1","state":[{"name":"spent","read-only":true,"read-only":false}]}`)); err == nil ||
!strings.Contains(err.Error(), "the field state[0].read-only is given twice") { !strings.Contains(err.Error(), "the field state[0].read-only is given twice") {
t.Fatalf("read-only given twice: %v", err) t.Fatalf("read-only given twice: %v", err)
} }
} }
// A read of a seat holder's state is its own kind, `seat:<seat>.<state>` (design 47 G A-128), and a seat's compiled form
// carries the names of the states its definition declares.
func TestSeatRead(t *testing.T) {
for _, row := range []struct {
read, seat, local string
ok, bad bool
}{
{"seat:mesh-provisioner.plans", "mesh-provisioner", "plans", true, false},
{"mesh-provisioner.plans", "", "", false, false},
{"seat:mesh-provisioner", "", "", true, true},
{"seat:.plans", "", "", true, true},
{"seat:Mesh.plans", "", "", true, true},
} {
seat, local, ok, err := SeatRead(row.read)
if seat != row.seat || local != row.local || ok != row.ok || (err != nil) != row.bad {
t.Errorf("SeatRead(%q) = %q, %q, %v, %v", row.read, seat, local, ok, err)
}
}
for _, row := range []struct {
consume, seat, state string
ok, bad bool
}{
{"seat:mesh-provisioner.views.plans", "mesh-provisioner", "plans", true, false},
{"mesh-provisioner.views.plans", "", "", false, false},
{"seat:mesh-provisioner.plans", "", "", true, true},
{"seat:.views.plans", "", "", true, true},
} {
seat, state, ok, err := SeatViewConsume(row.consume)
if seat != row.seat || state != row.state || ok != row.ok || (err != nil) != row.bad {
t.Errorf("SeatViewConsume(%q) = %q, %q, %v, %v", row.consume, seat, state, ok, err)
}
}
s := SeatFromDefinition(seatdef.Definition{Seat: "mesh-provisioner", Scope: "mesh",
States: []seatdef.State{{Name: "plans", View: true}, {Name: "provisions"}},
Accepts: []seatdef.Message{{Name: "step", Stream: "check"}, {Name: "step"}}})
if len(s.States) != 2 || s.States[0] != "plans" || s.States[1] != "provisions" || len(s.Views) != 1 || s.Views[0] != "plans" ||
len(s.AcceptedOn) != 1 || s.AcceptedOn[0] != "check.step" {
t.Errorf("the seat's states = %v, views = %v, accepted on = %v", s.States, s.Views, s.AcceptedOn)
}
// A-177 (design 46 approval mark §2, §4): a compiled verb carries its definition's approval mark.
marked := SeatFromDefinition(seatdef.Definition{Seat: "node-login-shell",
Verbs: []seatdef.Verb{{Name: "execute", NeedsApproval: true}, {Name: "read"}}})
if len(marked.Serves) != 2 || !marked.Serves[0].NeedsApproval || marked.Serves[1].NeedsApproval {
t.Errorf("the compiled verbs' marks = %+v", marked.Serves)
}
}
go/catalogue/streams.go
+4 -40
View File
@@ -64,9 +64,6 @@ type StreamDeclaration struct {
// Duplicates is a work-queue stream's duplicate window in hours, at most its maximum age; refused // Duplicates is a work-queue stream's duplicate window in hours, at most its maximum age; refused
// without retention work-queue (novox/hq ADR 0436 §6). // without retention work-queue (novox/hq ADR 0436 §6).
Duplicates int `json:"duplicates,omitempty"` Duplicates int `json:"duplicates,omitempty"`
// LastPerSubject says the stream keeps only the last message of each subject, as a seat event's `keep` does (design 47 G §703,
// §396, A-181): a part that follows it reads it whole at each start (§556).
LastPerSubject bool `json:"last-per-subject,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47). // KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout KeptLayout
} }
@@ -89,7 +86,7 @@ func (s *StreamDeclaration) UnmarshalJSON(raw []byte) error {
dec.DisallowUnknownFields() dec.DisallowUnknownFields()
if err := dec.Decode(&p); err != nil { if err := dec.Decode(&p); err != nil {
return fmt.Errorf("a stream is {name, events, max-age-days, max-mib, grant, retention, max-deliveries, "+ return fmt.Errorf("a stream is {name, events, max-age-days, max-mib, grant, retention, max-deliveries, "+
"duplicates, last-per-subject, form, catch-up}: %w", typedUnknown(err)) "duplicates, form, catch-up}: %w", typedUnknown(err))
} }
if err := p.KeptLayout.refuse(fmt.Sprintf("stream %q", p.Name)); err != nil { if err := p.KeptLayout.refuse(fmt.Sprintf("stream %q", p.Name)); err != nil {
return err return err
@@ -290,47 +287,14 @@ type HeardPart struct {
// stream, granted or not, and the `consumes` left for EVENTS — those naming something the provider emits // stream, granted or not, and the `consumes` left for EVENTS — those naming something the provider emits
// there. // there.
// //
// Only an entry naming its emitter literally reaches a restricted stream: a wildcard emitter, `**` among them, // Only an entry naming its emitter literally reaches a module's stream: a wildcard emitter, `**` among them,
// never does, so nothing that hears every event hears a restricted stream by saying so. A `*` emitter before one // is EVENTS' alone, so nothing that hears every event hears a restricted stream by saying so.
// event reaches each declaring module's unrestricted stream (design 47 G §386), only where defined, the function the
// grants part passes, says the event is one a seat definition of the seat list defines or one of mesh-sdk's finding
// events (§474, §644); any other `*` entry stays EVENTS' as before, and nil defines none.
func Hearing(consumer Manifest, shelf map[string]Manifest, machine string, func Hearing(consumer Manifest, shelf map[string]Manifest, machine string,
values func(provider, setting string) []any, defined func(event string) bool) (parts []HeardPart, touches bool, onEvents []string) { values func(provider, setting string) []any) (parts []HeardPart, touches bool, onEvents []string) {
byStream := map[string]*HeardPart{} byStream := map[string]*HeardPart{}
var order []string var order []string
for _, c := range consumer.Consumes { for _, c := range consumer.Consumes {
emitter, rest, named := strings.Cut(c, ".") emitter, rest, named := strings.Cut(c, ".")
if named && emitter == OneName && rest != "" && !strings.ContainsAny(rest, "*>") && rest != TheRest && defined != nil && defined(rest) {
// A `*` emitter (design 47 G §386): the event from every module that declares it on a stream, never a
// restricted one; the entry stays EVENTS' too, for the modules that say it there.
onEvents = append(onEvents, c)
names := make([]string, 0, len(shelf))
for name := range shelf {
names = append(names, name)
}
slices.Sort(names)
for _, name := range names {
hearing, _ := shelf[name].StreamsHearing(rest)
for _, s := range hearing {
if s.Grant != nil {
continue
}
touches = true
id := name + "/" + s.BusName()
if byStream[id] == nil {
byStream[id] = &HeardPart{Emitter: name, Stream: s}
order = append(order, id)
}
for _, p := range s.Events {
if x, ok := narrow(rest, p); ok {
byStream[id].Patterns = append(byStream[id].Patterns, x)
}
}
}
}
continue
}
provider, known := shelf[emitter] provider, known := shelf[emitter]
if !named || emitter == OneName || emitter == TheRest || !known || len(provider.Streams) == 0 { if !named || emitter == OneName || emitter == TheRest || !known || len(provider.Streams) == 0 {
onEvents = append(onEvents, c) onEvents = append(onEvents, c)
go/catalogue/streams_test.go
+7 -48
View File
@@ -85,7 +85,7 @@ func TestAConsumerHearsOnlyWhatItAsksAndIsGranted(t *testing.T) {
} }
reader := Manifest{Module: "reader", Consumes: []string{"mesh-mail.mail.**", "mesh-mail.account.added", "billing.paid"}} reader := Manifest{Module: "reader", Consumes: []string{"mesh-mail.mail.**", "mesh-mail.account.added", "billing.paid"}}
parts, touches, onEvents := Hearing(reader, shelf, "ace", values, nil) parts, touches, onEvents := Hearing(reader, shelf, "ace", values)
if !touches || len(parts) != 1 || parts[0].Stream.BusName() != "MAIL" { if !touches || len(parts) != 1 || parts[0].Stream.BusName() != "MAIL" {
t.Fatalf("heard %+v", parts) t.Fatalf("heard %+v", parts)
} }
@@ -96,28 +96,28 @@ func TestAConsumerHearsOnlyWhatItAsksAndIsGranted(t *testing.T) {
if !slices.Equal(onEvents, []string{"mesh-mail.account.added", "billing.paid"}) { if !slices.Equal(onEvents, []string{"mesh-mail.account.added", "billing.paid"}) {
t.Fatalf("left on EVENTS: %v", onEvents) t.Fatalf("left on EVENTS: %v", onEvents)
} }
parts, _, _ = Hearing(reader, shelf, "shanks", values, nil) parts, _, _ = Hearing(reader, shelf, "shanks", values)
if len(parts) != 1 || len(parts[0].Patterns) != 6 { if len(parts) != 1 || len(parts[0].Patterns) != 6 {
t.Fatalf("on shanks reader hears %+v", parts) t.Fatalf("on shanks reader hears %+v", parts)
} }
narrow := Manifest{Module: "reader", Consumes: []string{"mesh-mail.mail.*.message.added"}} narrow := Manifest{Module: "reader", Consumes: []string{"mesh-mail.mail.*.message.added"}}
parts, _, _ = Hearing(narrow, shelf, "ace", values, nil) parts, _, _ = Hearing(narrow, shelf, "ace", values)
if len(parts) != 1 || !slices.Equal(parts[0].Patterns, []string{"mail.mesh.message.added"}) { if len(parts) != 1 || !slices.Equal(parts[0].Patterns, []string{"mail.mesh.message.added"}) {
t.Fatalf("a narrower consumes heard %+v", parts) t.Fatalf("a narrower consumes heard %+v", parts)
} }
stranger := Manifest{Module: "stranger", Consumes: []string{"mesh-mail.mail.**"}} stranger := Manifest{Module: "stranger", Consumes: []string{"mesh-mail.mail.**"}}
parts, touches, onEvents = Hearing(stranger, shelf, "ace", values, nil) parts, touches, onEvents = Hearing(stranger, shelf, "ace", values)
if len(parts) != 0 || !touches || len(onEvents) != 0 { if len(parts) != 0 || !touches || len(onEvents) != 0 {
t.Fatalf("a module not granted heard %+v (touches %v, on EVENTS %v)", parts, touches, onEvents) t.Fatalf("a module not granted heard %+v (touches %v, on EVENTS %v)", parts, touches, onEvents)
} }
everything := Manifest{Module: "audit", Consumes: []string{"**", "*.mail.**"}} everything := Manifest{Module: "audit", Consumes: []string{"**", "*.mail.**"}}
parts, touches, _ = Hearing(everything, shelf, "ace", values, nil) parts, touches, _ = Hearing(everything, shelf, "ace", values)
if len(parts) != 0 || touches { if len(parts) != 0 || touches {
t.Fatalf("a wildcard emitter reached a module's stream: %+v", parts) t.Fatalf("a wildcard emitter reached a module's stream: %+v", parts)
} }
parts, _, _ = Hearing(reader, shelf, "ace", func(string, string) []any { return nil }, nil) parts, _, _ = Hearing(reader, shelf, "ace", func(string, string) []any { return nil })
if len(parts) != 0 { if len(parts) != 0 {
t.Fatalf("with no setting reader heard %+v", parts) t.Fatalf("with no setting reader heard %+v", parts)
} }
@@ -127,7 +127,7 @@ func TestAConsumerHearsOnlyWhatItAsksAndIsGranted(t *testing.T) {
func TestAnUnrestrictedStreamIsHeardByWhoeverAsks(t *testing.T) { func TestAnUnrestrictedStreamIsHeardByWhoeverAsks(t *testing.T) {
feed := Manifest{Module: "feed", Streams: []StreamDeclaration{{Name: "items", Events: []string{"item.*.added", "item.*.gone"}}}} feed := Manifest{Module: "feed", Streams: []StreamDeclaration{{Name: "items", Events: []string{"item.*.added", "item.*.gone"}}}}
reader := Manifest{Module: "reader", Consumes: []string{"feed.item.**", "feed.item.x.added"}} reader := Manifest{Module: "reader", Consumes: []string{"feed.item.**", "feed.item.x.added"}}
parts, touches, onEvents := Hearing(reader, map[string]Manifest{"feed": feed}, "ace", nil, nil) parts, touches, onEvents := Hearing(reader, map[string]Manifest{"feed": feed}, "ace", nil)
if !touches || len(onEvents) != 0 || len(parts) != 1 || if !touches || len(onEvents) != 0 || len(parts) != 1 ||
!slices.Equal(parts[0].Patterns, []string{"item.*.added", "item.*.gone"}) { !slices.Equal(parts[0].Patterns, []string{"item.*.added", "item.*.gone"}) {
t.Fatalf("heard %+v, on EVENTS %v", parts, onEvents) t.Fatalf("heard %+v, on EVENTS %v", parts, onEvents)
@@ -152,44 +152,3 @@ func TestNarrowingTwoPatterns(t *testing.T) {
t.Fatalf("uncovered: %v", got) t.Fatalf("uncovered: %v", got)
} }
} }
// Design 47 G §386: `*.<event>` reaches the unrestricted stream of each module that declares the event, and never a
// restricted one; a wildcard before anything wider still reaches none.
func subAStarEmitterHearsEachDeclaringModulesUnrestrictedStream(t *testing.T) {
mail, err := decodeForTest([]byte(mailManifest(t, mailStream, `"emits":["account.added"],`)))
if err != nil {
t.Fatal(err)
}
a := Manifest{Module: "a", Streams: []StreamDeclaration{{Name: "a-findings", Events: []string{"finding-written"}}}}
b := Manifest{Module: "b", Streams: []StreamDeclaration{{Name: "b-findings", Events: []string{"finding-written", "finding-cleared"}}}}
other := Manifest{Module: "c", Streams: []StreamDeclaration{{Name: "c-stuff", Events: []string{"other"}}}}
shelf := map[string]Manifest{"a": a, "b": b, "c": other, "mesh-mail": mail}
keeper := Manifest{Module: "keeper", Consumes: []string{"*.finding-written"}}
isDefined := func(e string) bool { return e == "finding-written" }
parts, touches, onEvents := Hearing(keeper, shelf, "ace", func(string, string) []any { return nil }, isDefined)
if !touches || len(parts) != 2 || parts[0].Emitter != "a" || parts[1].Emitter != "b" ||
!slices.Equal(parts[1].Patterns, []string{"finding-written"}) || !slices.Equal(onEvents, []string{"*.finding-written"}) {
t.Fatalf("heard %+v (touches %v, on EVENTS %v)", parts, touches, onEvents)
}
// §474, §644: a `*` entry before an event no definition names keeps its meaning on main, the shared stream alone
undefined := Manifest{Module: "late", Consumes: []string{"*.finding-written"}}
if parts, touches, onEvents := Hearing(undefined, shelf, "ace", func(string, string) []any { return nil },
func(string) bool { return false }); len(parts) != 0 || touches || !slices.Equal(onEvents, []string{"*.finding-written"}) {
t.Fatalf("an undefined event heard %+v (touches %v, on EVENTS %v)", parts, touches, onEvents)
}
restricted := Manifest{Module: "audit", Consumes: []string{"*.mail.mesh.message.added", "*.>"}}
if parts, _, _ := Hearing(restricted, shelf, "ace", func(string, string) []any { return nil }, nil); len(parts) != 0 {
t.Fatalf("a wildcard reached %+v", parts)
}
}
// A stream declares that it keeps the last message of each subject (design 47 G §703, A-181), and says no unless it does.
func TestAStreamDeclaresItIsKeptLastPerSubject(t *testing.T) {
m, err := decodeForTest([]byte(`{"module":"feed","version":"1","streams":[{"name":"ends","events":["end.**"],"last-per-subject":true},{"name":"all","events":["a.**"]}]}`))
if err != nil {
t.Fatal(err)
}
if !m.Streams[0].LastPerSubject || m.Streams[1].LastPerSubject {
t.Fatalf("last-per-subject read as %+v", m.Streams)
}
}
go/catalogue/verbs.go
+1 -14
View File
@@ -4,8 +4,6 @@ import (
"bytes" "bytes"
"encoding/json" "encoding/json"
"fmt" "fmt"
"git.novox.be/novox/mesh-sdk/go/seatdef"
) )
// A Verb is one tool a role serves: its name, what it does, and the schema of its arguments and of // A Verb is one tool a role serves: its name, what it does, and the schema of its arguments and of
@@ -39,11 +37,6 @@ type Verb struct {
// A command line is matched by its words in order, the first being the command's name; `edit // A command line is matched by its words in order, the first being the command's name; `edit
// /etc/hosts` and `HOSTALIASES` name the guard's two local work-arounds for a mesh name. // /etc/hosts` and `HOSTALIASES` name the guard's two local work-arounds for a mesh name.
Replaces []string `json:"replaces,omitempty"` Replaces []string `json:"replaces,omitempty"`
// NeedsApproval is the verb's approval mark as its seat's definition gives it (seatdef.Verb.NeedsApproval): the
// operator approves a call of it first, for every holder (design 46 *The approval mark* §2, §4; lead answer A-177),
// read from here by the manifest validator and every reader of the seat list. Never read from a manifest's own
// verb (a module marks in its needs-approval) nor said: compiled from the definition alone, as Optional.
NeedsApproval bool `json:"-"`
} }
func (v *Verb) UnmarshalJSON(raw []byte) error { func (v *Verb) UnmarshalJSON(raw []byte) error {
@@ -140,15 +133,9 @@ func unservedVerbs(tools []string, promised []Verb) []string {
for _, t := range tools { for _, t := range tools {
has[t] = true has[t] = true
} }
marked := false
for _, v := range promised {
marked = marked || v.NeedsApproval
}
var missing []string var missing []string
for _, v := range promised { for _, v := range promised {
// The verb approvals is the gate's, served wherever a marked verb is (the seat server or the tool runner), so no if !has[v.Name] && !v.Optional {
// holder names it there (design 47 G lead answer A-266); a seat with no marked verb leaves it to its holder.
if !has[v.Name] && !v.Optional && !(marked && v.Name == seatdef.ApprovalsVerb) {
missing = append(missing, v.Name) missing = append(missing, v.Name)
} }
} }
go/catalogue/verbs_r6_test.go
+51
View File
@@ -0,0 +1,51 @@
package catalogue
import (
"strings"
"testing"
)
// 1943/AC4: the delivery seat keeps the merge's verbs and no longer offers times, what-if, release, stop or
// retire-history: each is a row with no input, optional so the holder of before and the holder of release 6 both hold
// the seat, refusing it by name; mark-ready, served by nobody since issue 1022, is gone.
func TestTheDeliverySeatHasNoPlanVerbs(t *testing.T) {
seat, ok := SeatNamed(DeliverySeat)
if !ok {
t.Fatal("the delivery seat is not in the set")
}
rows := map[string]Verb{}
for _, v := range seat.Serves {
rows[v.Name] = v
}
for _, name := range []string{"times", "what-if", "release", "stop", "retire-history"} {
v, ok := rows[name]
replacement := strings.TrimPrefix(v.Description, "Removed in R6 (ADR 0539): refused by name, naming its replacement: ")
if !ok || replacement == "" || replacement == v.Description {
t.Errorf("%s: in the table %v, replacement %q", name, ok, replacement)
continue
}
if !strings.HasPrefix(v.Description, "Removed in R6 (ADR 0539): refused by name") || !strings.Contains(v.Description, replacement) || !v.Optional {
t.Errorf("%s = %q (optional %v), want an optional row refusing it by name", name, v.Description, v.Optional)
}
if props, _ := v.Input["properties"].(map[string]any); len(props) > 0 {
t.Errorf("%s still takes %v", name, props)
}
}
if _, ok := rows["mark-ready"]; ok {
t.Errorf("mark-ready is still promised")
}
for _, kept := range []string{"deliveries", "show", "groups", "checks", "table", "recheck", "close", "stalled", "settle"} {
if v, ok := rows[kept]; !ok || strings.HasPrefix(v.Description, "Removed") {
t.Errorf("%s is not kept (ADR 0539 clause 17)", kept)
}
}
gate := Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh,
Serves: []string{"deliveries", "show", "groups", "checks", "table", "stalled", "recheck", "close"}}}}
if err := CanHold(gate, seat); err != nil {
t.Errorf("the gate of release 6 cannot hold the seat: %v", err)
}
gate.Claims[0].Serves = append(gate.Claims[0].Serves, "times", "what-if", "release", "stop", "retire-history")
if err := CanHold(gate, seat); err != nil {
t.Errorf("the gate of release 5 cannot hold the seat: %v", err)
}
}
go/catalogue/verbs_test.go
+9 -8
View File
@@ -51,15 +51,15 @@ func TestHoldingAMeshSeatRequiresServingItsVerbs(t *testing.T) {
t.Errorf("a holder saying it serves mint: %v", err) t.Errorf("a holder saying it serves mint: %v", err)
} }
// The seat mesh-catalog promises the verbs its definition names (milestone 47/G, design 47 G section 307). // L3: the seat mesh-catalog promises five verbs (the fifth is the seat list, ADR 0636 §4), each optional while two claimants stand (ADR 0604 §1, §6).
catalog, ok := SeatNamed(CatalogSeat) catalog, ok := SeatNamed(CatalogSeat)
if !ok || catalog.Scope != ScopeMesh || len(catalog.Serves) < 5 { if !ok || catalog.Scope != ScopeMesh || catalog.Decision != "novox/hq ADR 0604" || len(catalog.Serves) != 5 {
t.Fatalf("mesh-catalog: in the set %v, %+v", ok, catalog) t.Fatalf("mesh-catalog: in the set %v, %+v", ok, catalog)
} }
four := VerbNames(catalog.Serves) four := []string{"modules", "module", "provides", "dependents", "seat-definitions"}
for i, v := range catalog.Serves { for i, v := range catalog.Serves {
if v.Input == nil { if v.Name != four[i] || !v.Optional || v.Input == nil {
t.Errorf("mesh-catalog's verb %d is %+v, with no input", i, v) t.Errorf("mesh-catalog's verb %d is %+v, want %s, optional, with its input", i, v, four[i])
} }
} }
// The answers keep the field names of the catalog module's tools they replace (catalog_modules, catalog_module, // The answers keep the field names of the catalog module's tools they replace (catalog_modules, catalog_module,
@@ -87,9 +87,10 @@ func TestHoldingAMeshSeatRequiresServingItsVerbs(t *testing.T) {
serves []string serves []string
word string word string
}{ }{
"serving the four": {four, ""}, "serving the four": {four, ""},
"serving one short": {four[:len(four)-1], "does not serve " + four[len(four)-1]}, "serving three, optional": {four[:3], ""},
"serving a fifth": {append(slices.Clone(four), "rebuild"), "rebuild"}, "serving none, the old one": {nil, ""},
"serving a fifth": {append(slices.Clone(four), "rebuild"), "rebuild"},
} { } {
err := CanHold(cataloguer(c.serves...), catalog) err := CanHold(cataloguer(c.serves...), catalog)
if (c.word == "") != (err == nil) || (err != nil && !strings.Contains(err.Error(), c.word)) { if (c.word == "") != (err == nil) || (err != nil && !strings.Contains(err.Error(), c.word)) {
go/catalogue/waits_test.go
+17
View File
@@ -0,0 +1,17 @@
package catalogue
import (
"testing"
)
// novox/hq ADR 0367 §2: the tool runner's module names, one by one, the event families a session may wait on.
func TestTheToolRunnerMayNameTheEventsASessionWaitsOn(t *testing.T) {
m, err := decodeForTest([]byte(`{"module":"node-tools","version":"1",` +
`"waits":["mesh-knowledge.reviewed","operator-channel.decided.claude-code"]}`))
if err != nil {
t.Fatal(err)
}
if len(m.Waits) != 2 || m.Waits[1] != "operator-channel.decided.claude-code" {
t.Fatalf("waits not read: %v", m.Waits)
}
}
go/go.mod
+1 -1
View File
@@ -3,7 +3,6 @@ module git.novox.be/novox/mesh-sdk/go
go 1.26.0 go 1.26.0
require ( require (
github.com/klauspost/compress v1.20.0
github.com/nats-io/nats-server/v2 v2.11.17 github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0 github.com/nats-io/nats.go v1.54.0
) )
@@ -11,6 +10,7 @@ require (
require ( require (
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/minio/highwayhash v1.0.4 // indirect github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect github.com/nats-io/nkeys v0.4.16 // indirect
go/membership/membership.go
+9 -8
View File
@@ -50,9 +50,6 @@ type Membership struct {
Mesh []string `json:"mesh,omitempty"` Mesh []string `json:"mesh,omitempty"`
// State is every bucket this module's code may reach, and whether it may write it (novox/hq ADR 0201). // State is every bucket this module's code may reach, and whether it may write it (novox/hq ADR 0201).
State []StateIssued `json:"state,omitempty"` State []StateIssued `json:"state,omitempty"`
// Objects maps the name of each old `objects` section of a foundation part to its object store, as State maps its
// buckets (design 47 G §619).
Objects map[string]string `json:"objects,omitempty"`
// Logs is every log this module's code may reach (novox/hq ADR 0297). // Logs is every log this module's code may reach (novox/hq ADR 0297).
Logs []LogIssued `json:"logs,omitempty"` Logs []LogIssued `json:"logs,omitempty"`
// SeatTraffic is what this module's code may submit, say, hear, take, ask, answer and read on seats // SeatTraffic is what this module's code may submit, say, hear, take, ask, answer and read on seats
@@ -61,6 +58,8 @@ type Membership struct {
// Streams are this module's own streams, Hears the module streams it hears (novox/hq ADR 0363). // Streams are this module's own streams, Hears the module streams it hears (novox/hq ADR 0363).
Streams []StreamIssued `json:"streams,omitempty"` Streams []StreamIssued `json:"streams,omitempty"`
Hears []HeardIssued `json:"hears,omitempty"` Hears []HeardIssued `json:"hears,omitempty"`
// Waits are the event families a session may wait on through the mesh MCP server (novox/hq ADR 0367 §2).
Waits []WaitIssued `json:"waits,omitempty"`
// Bindings are the module's bus state as the bus module created it, one per entry its manifest asks for in // Bindings are the module's bus state as the bus module created it, one per entry its manifest asks for in
// `requires` (novox/hq ADR 0595 §1, §7): written by the grants part when it hears the bus module's `created`. // `requires` (novox/hq ADR 0595 §1, §7): written by the grants part when it hears the bus module's `created`.
Bindings []BusBinding `json:"bindings,omitempty"` Bindings []BusBinding `json:"bindings,omitempty"`
@@ -80,9 +79,6 @@ type BusBinding struct {
Subject string `json:"subject,omitempty"` Subject string `json:"subject,omitempty"`
// Bucket is a key-value bucket's name. // Bucket is a key-value bucket's name.
Bucket string `json:"bucket,omitempty"` Bucket string `json:"bucket,omitempty"`
// Objects is the bus name of a state's `<state>-objects` store (design 47 G §399, §642); the module's sent store is a
// binding entry of its own, the local name `sent-objects`, its ObjectStore.
Objects string `json:"objects,omitempty"`
// ObjectStore is an object store's name. // ObjectStore is an object store's name.
ObjectStore string `json:"object-store,omitempty"` ObjectStore string `json:"object-store,omitempty"`
// Ledger, for the storage type ledger: its history stream (one subject per instance), its state bucket, each // Ledger, for the storage type ledger: its history stream (one subject per instance), its state bucket, each
@@ -110,8 +106,7 @@ type SeatServed struct {
} }
// StateIssued is one bucket an assignment may reach, by the name its module uses for it (novox/hq ADR 0201). // StateIssued is one bucket an assignment may reach, by the name its module uses for it (novox/hq ADR 0201).
// Key, when set, names the machine whose keys it may reach (novox/hq ADR 0260; design 46 7-approval-mark §13): that key and the // Key, when set, is the one key it may reach (novox/hq ADR 0260): this machine's.
// keys below it (buscompose.PerMachineKeys), no other machine's.
// //
// PerNode says the state is kept one bucket per node (novox/hq ADR 0593 §6: `composed-<node>`): Bucket is then the // PerNode says the state is kept one bucket per node (novox/hq ADR 0593 §6: `composed-<node>`): Bucket is then the
// buckets' prefix, each bucket the prefix and a node's name, and a key's first part names its node and so its bucket. // buckets' prefix, each bucket the prefix and a node's name, and a key's first part names its node and so its bucket.
@@ -173,6 +168,12 @@ type HeardIssued struct {
Consumer string `json:"consumer"` Consumer string `json:"consumer"`
} }
// WaitIssued is one family as a membership issues it: the name a session asks for, and its subject.
type WaitIssued struct {
Event string `json:"event"`
Subject string `json:"subject"`
}
// StateIssued is the state this membership issues its module's code by a local name, and whether it does: what every // StateIssued is the state this membership issues its module's code by a local name, and whether it does: what every
// runtime that serves a module's state calls reads, so no reader keeps a second copy of the rule (novox/hq issue 1966). // runtime that serves a module's state calls reads, so no reader keeps a second copy of the rule (novox/hq issue 1966).
// An entry of the old `state` list wins (ADR 0201); otherwise a binding of a key-value entry the module asks for in // An entry of the old `state` list wins (ADR 0201); otherwise a binding of a key-value entry the module asks for in
go/membership/membership_test.go
-12
View File
@@ -3,7 +3,6 @@ package membership
import ( import (
"encoding/json" "encoding/json"
"reflect" "reflect"
"strings"
"testing" "testing"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
@@ -55,17 +54,6 @@ func TestAMembershipRoundTripsAsIssued(t *testing.T) {
if err := json.Unmarshal(boundRaw, &boundBack); err != nil || !reflect.DeepEqual(bound, boundBack) { if err := json.Unmarshal(boundRaw, &boundBack); err != nil || !reflect.DeepEqual(bound, boundBack) {
t.Fatalf("a membership with a ledger binding read back as %+v (%v)", boundBack, err) t.Fatalf("a membership with a ledger binding read back as %+v (%v)", boundBack, err)
} }
// Design 47 G §642, §619: a state's binding carries its objects store, the sent store is an entry of its own, and a
// foundation part's old `objects` sections map to their stores.
withObjects := in
withObjects.Objects = map[string]string{"blobs": "forge_blobs"}
withObjects.Bindings = []BusBinding{{Local: "repos", Storage: catalogue.StorageKeyValue, Template: catalogue.TemplateState, Bucket: "forge_repos", Objects: "forge_repos-objects"},
{Local: "sent-objects", Storage: catalogue.StorageObjectStore, ObjectStore: "forge_sent"}}
objRaw, _ := json.Marshal(withObjects)
var objBack Membership
if err := json.Unmarshal(objRaw, &objBack); err != nil || !reflect.DeepEqual(withObjects, objBack) || !strings.Contains(string(objRaw), `"objects":"forge_repos-objects"`) {
t.Fatalf("a membership with objects read back as %+v (%v)", objBack, err)
}
before, _ := json.Marshal(Membership{Node: "ace", Module: "forge", Tools: "t", Emits: "e", Serves: []Served{{Subject: "s"}}, before, _ := json.Marshal(Membership{Node: "ace", Module: "forge", Tools: "t", Emits: "e", Serves: []Served{{Subject: "s"}},
State: []StateIssued{{Name: "repos", Bucket: "forge_repos", Writes: true}}}) State: []StateIssued{{Name: "repos", Bucket: "forge_repos", Writes: true}}})
if string(before) != `{"node":"ace","module":"forge","serves":[{"subject":"s"}],"emits":"e","tools":"t","state":[{"name":"repos","bucket":"forge_repos","writes":true}]}` { if string(before) != `{"node":"ace","module":"forge","serves":[{"subject":"s"}],"emits":"e","tools":"t","state":[{"name":"repos","bucket":"forge_repos","writes":true}]}` {
go/operatorask/act.go
-51
View File
@@ -1,51 +0,0 @@
package operatorask
import (
"crypto/sha256"
"encoding/hex"
"fmt"
"sort"
"strconv"
"strings"
)
// An Act is what an option of an ask binds (novox/hq ADR 0286 §10, §11): named fields, each a string — the verb, the
// machine, the level, and each argument under a name of its own ("arg.delivery"). Flat on purpose: its digest is over
// these names and values alone, never over how a language happens to encode a struct. It is mesh-sdk's (design 47 G
// lead answers A-198, A-219), so the seat server's gate in go/process digests an act as every asker does;
// operatorchannelclient re-exports it.
type Act map[string]string
// ActDigest is the digest an asker puts in an option's binds: SHA-256 over the act's canonical encoding, written
// "sha256:<hex>". The same names and values give the same digest in any language, whatever order they were set in; a
// field renamed, added or emptied gives another.
func ActDigest(act Act) (string, error) {
if len(act) == 0 {
return "", fmt.Errorf("the act cannot be digested: it names nothing")
}
names := make([]string, 0, len(act))
for k := range act {
if k == "" {
return "", fmt.Errorf("the act cannot be digested: a field has no name")
}
names = append(names, k)
}
sort.Strings(names)
var b strings.Builder
b.WriteString("novox.act.v1\n")
for _, k := range names {
canonical(&b, k)
canonical(&b, act[k])
}
sum := sha256.Sum256([]byte(b.String()))
return "sha256:" + hex.EncodeToString(sum[:]), nil
}
// canonical writes one value as its length in bytes, a colon, the bytes and a newline: no value can be read as
// another's end or start, so two different sequences of values never encode the same.
func canonical(b *strings.Builder, v string) {
b.WriteString(strconv.Itoa(len(v)))
b.WriteByte(':')
b.WriteString(v)
b.WriteByte('\n')
}
go/operatorask/act_test.go
-34
View File
@@ -1,34 +0,0 @@
package operatorask
import (
"crypto/sha256"
"encoding/hex"
"testing"
)
// Design 47 G A-198: the act's form and its digest are mesh-sdk's, so a seat server in go/process digests an act as
// every asker does, with no import of the contracts; the digest is canonical over the act's named fields.
func TestTheActsDigestIsCanonicalOverItsNamedFields(t *testing.T) {
a1, _ := ActDigest(Act{"verb": "release", "arg.delivery": "hq"})
b := Act{}
b["arg.delivery"] = "hq"
b["verb"] = "release"
if a2, _ := ActDigest(b); a1 != a2 {
t.Error("the order an act was built in changed its digest")
}
x, _ := ActDigest(Act{"a": "b\n1:c"})
y, _ := ActDigest(Act{"a": "b", "c": ""})
if x == y {
t.Error("two different acts encode the same")
}
if _, err := ActDigest(Act{}); err == nil {
t.Error("an empty act was digested")
}
if _, err := ActDigest(Act{"": "x"}); err == nil {
t.Error("an act with a nameless field was digested")
}
sum := sha256.Sum256([]byte("novox.act.v1\n4:verb\n7:release\n"))
if got, _ := ActDigest(Act{"verb": "release"}); got != "sha256:"+hex.EncodeToString(sum[:]) {
t.Errorf("the act's encoding is not the one written down: %s", got)
}
}
go/operatorask/operatorask.go
-94
View File
@@ -1,94 +0,0 @@
// Package operatorask is a module's asking of a person through the seat that routes asks to the operator (design 47 G
// A-101, §700, A-74): the seat's holder is read by seatlist.HolderOf, an ask is published under the module's own name, the
// router's record of it is read for the warrant, and the router's `decided` is heard. It is shared code, so no module
// keeps its own copy; it names no seat, verb or module: the generated contracts' names and types are its parameters.
package operatorask
import (
"encoding/json"
"errors"
"fmt"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/seatlist"
)
// Caller calls a seat verb or module tool as `seat:<seat>.<verb>` or `<module>.<tool>` and returns its answer
// (stdio.Ask is one; so is a process's calls).
type Caller = seatlist.HolderCaller
// Contract is what the asker takes from the generated contracts: the asking seat and its state of the router's records.
type Contract struct {
Seat string
// StateAsks is the seat's state holding the router's record of each ask.
StateAsks string
}
// Bucket is the bucket of the router's records: the seat's state in the bucket of the seat's holder, as seatlist reads it
// (design 47 G A-126).
func Bucket(call Caller, c Contract) (string, error) {
module, err := seatlist.HolderOf(call, c.Seat)
if err != nil {
return "", fmt.Errorf("the router's record of an ask is not read: %w", err)
}
return module + "_" + c.StateAsks, nil
}
// Asker asks as one module. W is the generated contract's warrant type.
type Asker[W any] struct {
Module string
Contract Contract
// Call reaches the seat list's verb.
Call Caller
// Record reads one record of a bucket under the module's own name (process.Conn.Ask).
Record func(bucket, id string) (*stdio.StateEntry, error)
// Publish publishes a body de-duplicated by id: the same id twice is one message (stdio.SeatPublish).
Publish func(subject string, body any, id string) error
// Subscribe hears an event (stdio.Subscribe).
Subscribe func(pattern string, handle func(stdio.Envelope) error) error
// AskSubject and DecidedSubject are the generated contract's subjects for Module.
AskSubject, DecidedSubject func(module string) string
}
// Ask publishes the ask (the generated contract's Ask) under the module's own name, de-duplicated by its id.
func (a *Asker[W]) Ask(id string, ask any) error {
if a.Publish == nil || a.AskSubject == nil {
return errors.New("the asker has no way to publish")
}
return a.Publish(a.AskSubject(a.Module), ask, id)
}
// Where is an ask's state as the router keeps it, and its warrant once there is one; a record not yet there is an empty
// state. It is an error while the seat's holder or the record cannot be read.
func (a *Asker[W]) Where(id string) (state string, warrant *W, err error) {
bucket, err := Bucket(a.Call, a.Contract)
if err != nil {
return "", nil, err
}
e, err := a.Record(bucket, id)
if err != nil || e == nil {
return "", nil, err
}
var rec struct {
State string `json:"state"`
Warrant *W `json:"warrant"`
}
if err := json.Unmarshal(e.Value, &rec); err != nil {
return "", nil, fmt.Errorf("the router's record of %s cannot be read: %w", id, err)
}
return rec.State, rec.Warrant, nil
}
// Hear hands handle each warrant the router says to this module; a handler's error leaves it for the bus to give again.
func (a *Asker[W]) Hear(handle func(W) error) error {
if a.Subscribe == nil || a.DecidedSubject == nil {
return errors.New("the asker has no way to hear")
}
return a.Subscribe(a.DecidedSubject(a.Module), func(env stdio.Envelope) error {
var w W
if err := json.Unmarshal(env.Body, &w); err != nil {
return fmt.Errorf("%s holds what is no warrant: %w", env.Key, err)
}
return handle(w)
})
}
go/operatorask/operatorask_test.go
-60
View File
@@ -1,60 +0,0 @@
package operatorask
import (
"encoding/json"
"strings"
"testing"
stdio "git.novox.be/novox/mesh-sdk/go"
)
type warrant struct {
Ask string `json:"ask"`
Choice string `json:"choice"`
}
// Design 47 G A-101: one asker finds the seat's holder, publishes, reads the warrant and hears decided.
func TestTheAskerFindsTheHolderPublishesReadsTheWarrantAndHearsDecided(t *testing.T) {
var asked []string
call := func(key string, body any) (json.RawMessage, error) {
asked = append(asked, key)
return json.RawMessage(`{"seats":[{"seat":"other","holders":[{"module":"x"}]},{"seat":"router-seat","holders":[{"module":"holder"}]}]}`), nil
}
var bucket, id, subject string
var published any
var heard func(stdio.Envelope) error
a := &Asker[warrant]{
Module: "me", Contract: Contract{Seat: "router-seat", StateAsks: "asks"},
Call: call,
Record: func(b, i string) (*stdio.StateEntry, error) {
bucket, id = b, i
return &stdio.StateEntry{Value: json.RawMessage(`{"state":"chosen","warrant":{"ask":"a1","choice":"go"}}`)}, nil
},
Publish: func(s string, body any, _ string) error { subject, published = s, body; return nil },
Subscribe: func(_ string, h func(stdio.Envelope) error) error { heard = h; return nil },
AskSubject: func(m string) string { return "ask." + m },
DecidedSubject: func(m string) string { return "decided." + m },
}
if err := a.Ask("a1", "q"); err != nil || subject != "ask.me" || published != "q" {
t.Errorf("Ask published %v on %q: %v", published, subject, err)
}
state, w, err := a.Where("a1")
if err != nil || state != "chosen" || w == nil || w.Choice != "go" || bucket != "holder_asks" || id != "a1" || len(asked) != 1 || asked[0] != "seat:mesh-module-assigner.assignments" {
t.Errorf("Where = %q %+v %v, read %q %q after %v", state, w, err, bucket, id, asked)
}
var got []warrant
if err := a.Hear(func(w warrant) error { got = append(got, w); return nil }); err != nil {
t.Fatal(err)
}
if err := heard(stdio.Envelope{Key: "decided.me", Body: []byte(`{"ask":"a1","choice":"go"}`)}); err != nil || len(got) != 1 || got[0].Ask != "a1" {
t.Errorf("heard %v: %v", got, err)
}
if err := heard(stdio.Envelope{Key: "decided.me", Body: []byte(`nope`)}); err == nil {
t.Errorf("a body that is no warrant was taken")
}
// No holder: nothing is read and the reason is said.
a.Call = func(string, any) (json.RawMessage, error) { return json.RawMessage(`{"seats":[]}`), nil }
if _, _, err := a.Where("a1"); err == nil || !strings.Contains(err.Error(), "no module holds the seat router-seat") {
t.Errorf("with no holder Where answered %v", err)
}
}
go/packages_list_test.go
+11 -33
View File
@@ -23,10 +23,10 @@ import (
// Logic only some modules need is its owner module's package. A package added here is a decision to say in the pull // Logic only some modules need is its owner module's package. A package added here is a decision to say in the pull
// request: does every module share it, or is it a shape more than one part reads? // request: does every module share it, or is it a shape more than one part reads?
var meshSDKPackages = []string{ var meshSDKPackages = []string{
".", "announce", "announcetest", "busarchive", "buscalls", "buscompose", "busstate", "bussnap", "busretry", "calladdress", "catalogue", ".", "announce", "announcetest", "buscalls", "buscompose", "busstate", "busretry", "calladdress", "catalogue",
"declaration", "forgekey", "health", "internal/beside", "keys", "kvlimit", "membership", "motion", "declaration", "forgekey", "health", "internal/beside", "keys", "kvlimit", "membership", "motion",
"objectlink", "outward", "pernode", "process", "objectlink", "outward", "pernode", "process",
"sdkversion", "seal", "seatdef", "seatlist", "seatlist/seatlisttest", "operatorask", "approval", "stateobjects", "testbus", "views", "views/viewstest", "sdkversion", "seal", "seatdef", "seatlist", "seatlist/seatlisttest", "testbus", "views", "views/viewstest",
} }
// clientSeats are the seats whose logic is their client's in mesh-contracts (novox/hq ADR 0627 §1, §4): mesh-sdk names // clientSeats are the seats whose logic is their client's in mesh-contracts (novox/hq ADR 0627 §1, §4): mesh-sdk names
@@ -186,15 +186,6 @@ func TestMeshSDKHoldsOnlyWhatEveryModuleShares(t *testing.T) {
} }
} }
} }
// 1966/C-S10: the any-size package is imported by the modules that use it only, never by the module library (the
// root package, about 112 importers) or the process package, so a change to it rebuilds none of theirs (ADR 0595 §9).
if dir == "." || dir == "process" {
for _, imp := range file.Imports {
if p, _ := strconv.Unquote(imp.Path.Value); p == sdkModule+"/stateobjects" {
t.Errorf("%s imports %s: the module library and the process package never import the any-size package (ADR 0595 §9)", path, p)
}
}
}
for _, decl := range file.Decls { for _, decl := range file.Decls {
fn, ok := decl.(*ast.FuncDecl) fn, ok := decl.(*ast.FuncDecl)
if !ok { if !ok {
@@ -301,13 +292,6 @@ func TestMeshSDKHoldsOnlyWhatEveryModuleShares(t *testing.T) {
if string(entry) != `{"instance":"listing/a17","sequence":3}` { if string(entry) != `{"instance":"listing/a17","sequence":3}` {
t.Errorf("an index entry is %s", entry) t.Errorf("an index entry is %s", entry)
} }
// G §462: a ledger's fill is one call, answered as one buscalls.Fill.
if buscalls.LedgerFill != "mesh/ledger.fill" {
t.Errorf("a ledger's fill call is %q", buscalls.LedgerFill)
}
if raw, _ := json.Marshal(buscalls.Fill{Count: 3, Bytes: 10, MaxBytes: 100}); string(raw) != `{"count":3,"bytes":10,"max_bytes":100}` {
t.Errorf("a fill is %s", raw)
}
// A tool's answer reads the same whatever wraps it (buscalls.Unwrap, the lead's ruling 2026-11-04 13:12). // A tool's answer reads the same whatever wraps it (buscalls.Unwrap, the lead's ruling 2026-11-04 13:12).
for _, c := range []struct { for _, c := range []struct {
name, raw, answer, said string name, raw, answer, said string
@@ -327,12 +311,6 @@ func TestMeshSDKHoldsOnlyWhatEveryModuleShares(t *testing.T) {
t.Errorf("%s: unwrapped %s, %q, %v; want %s, %q, %v", c.name, answer, said, ok, c.answer, c.said, c.ok) t.Errorf("%s: unwrapped %s, %q, %v; want %s, %q, %v", c.name, answer, said, ok, c.answer, c.said, c.ok)
} }
} }
// design 47 G §347: a sent object is asked by its reference's fields.
sentAsk := buscalls.ObjectsSentGetRequest{Store: "m_sent-objects", Object: "o", Size: 3 << 20, Digest: strings.Repeat("b", 64)}
if raw, err := json.Marshal(sentAsk); err != nil || buscalls.ObjectsSentGet != "mesh/objects.sent-get" ||
string(raw) != `{"store":"m_sent-objects","object":"o","size":3145728,"sha256":"`+strings.Repeat("b", 64)+`"}` {
t.Errorf("the sent-get request is %s, %v", raw, err)
}
// 1966/AC5: an apply naming the tool call it serves, and its answer with the state behind, round-trip. // 1966/AC5: an apply naming the tool call it serves, and its answer with the state behind, round-trip.
for _, c := range []struct { for _, c := range []struct {
name string name string
@@ -359,15 +337,15 @@ const sdkModule = "git.novox.be/novox/mesh-sdk/go"
// that is a folder module of a repository (mesh-networking's modules/mesh-connectivity-contracts) is refused too. // that is a folder module of a repository (mesh-networking's modules/mesh-connectivity-contracts) is refused too.
func TestContractsAndDomainSDKsAreRecognisedInEverySegment(t *testing.T) { func TestContractsAndDomainSDKsAreRecognisedInEverySegment(t *testing.T) {
for path, want := range map[string]bool{ for path, want := range map[string]bool{
"git.novox.be/novox/mesh-contracts/go/seats/meshconfig": true, "git.novox.be/novox/mesh-contracts/go/seats/meshconfig": true,
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog": true, "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog": true,
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk": true, "git.novox.be/novox/mesh-delivery/sdk": true,
"git.novox.be/novox/mesh-networking/modules/mesh-connectivity-contracts/go/seats/x": true, "git.novox.be/novox/mesh-networking/modules/mesh-connectivity-contracts/go/seats/x": true,
"git.novox.be/novox/mesh-networking/modules/mesh-connectivity-sdk": true, "git.novox.be/novox/mesh-networking/modules/mesh-connectivity-sdk": true,
"git.novox.be/novox/mesh-sdk/go/catalogue": false, "git.novox.be/novox/mesh-sdk/go/catalogue": false,
"git.novox.be/novox/mesh-sdk/go/seatlist": false, "git.novox.be/novox/mesh-sdk/go/seatlist": false,
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-indexer": false, "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-indexer": false,
"github.com/nats-io/nats.go": false, "github.com/nats-io/nats.go": false,
} { } {
if got := contractsOrDomainSDK(path); got != want { if got := contractsOrDomainSDK(path); got != want {
t.Errorf("contractsOrDomainSDK(%q) = %v, want %v", path, got, want) t.Errorf("contractsOrDomainSDK(%q) = %v, want %v", path, got, want)
go/process/account.go
+1 -1
View File
@@ -2,7 +2,7 @@
// the account the controller delivers, connects as it with its pinned fingerprint and its inbox prefix, and // the account the controller delivers, connects as it with its pinned fingerprint and its inbox prefix, and
// offers emitting an event, the module's states with compare-and-set, consuming a work queue, reading the // offers emitting an event, the module's states with compare-and-set, consuming a work queue, reading the
// router's records of the module's own asks, and calling a tool or verb — each only as the account's grants // router's records of the module's own asks, and calling a tool or verb — each only as the account's grants
// allow. It answers its seat's verbs through AnswerSeat alone (design 47 G, lead answer A-107). A sibling of stdio, // allow. It offers no Serve and no way to serve a tool: a process never answers a call. A sibling of stdio,
// which it imports only for stdio's pure types (ErrChanged, CodeChanged, StateEntry, EmitOptions); it never // which it imports only for stdio's pure types (ErrChanged, CodeChanged, StateEntry, EmitOptions); it never
// touches stdio's current, channel or live(), which stay the bundle's own way to the bus. // touches stdio's current, channel or live(), which stay the bundle's own way to the bus.
package process package process
go/process/account_test.go
+1 -54
View File
@@ -19,7 +19,7 @@ import (
// 1171/AC2: given an account file broker-<process id> in the controller's form {url, fingerprint, node, // 1171/AC2: given an account file broker-<process id> in the controller's form {url, fingerprint, node,
// module, user, password, claims}, when the process package reads it, then it yields those fields and // module, user, password, claims}, when the process package reads it, then it yields those fields and
// tolerates claims. // tolerates claims.
func subReadAccountYieldsEveryFieldAndToleratesClaims(t *testing.T) { func TestReadAccountYieldsEveryFieldAndToleratesClaims(t *testing.T) {
path := writeAccount(t, map[string]any{ path := writeAccount(t, map[string]any{
"url": "nats://novox:4222", "fingerprint": "sha256:ab", "node": "novox", "module": "mesh-mail", "url": "nats://novox:4222", "fingerprint": "sha256:ab", "node": "novox", "module": "mesh-mail",
"user": "novox.mesh-mail.send", "password": "secret-pw", "user": "novox.mesh-mail.send", "password": "secret-pw",
@@ -295,56 +295,3 @@ func TestModuleAccountNamesAndDialOptions(t *testing.T) {
t.Errorf("a fingerprint did not pin the connection") t.Errorf("a fingerprint did not pin the connection")
} }
} }
// W1.3 (design 47 *The fingerprint handover*): a process's own build is the fingerprint the node-engine handed it, read
// once at its start; with none, or with one that is no single word, it says its own build is unknown and why.
func subTheOwnBuildIsTheFingerprintTheProcessWasStartedWith(t *testing.T) {
for _, c := range []struct{ name, env, fp, unknown string }{
{"handed", "src2:1480985e", "src2:1480985e", ""},
{"handed with spaces around", " src2:1480985e\n", "src2:1480985e", ""},
{"not handed", "", "", "own build unknown: started without MESH_FINGERPRINT"},
{"not one word", "src2:1 src2:2", "", `own build unknown: MESH_FINGERPRINT "src2:1 src2:2" is not one fingerprint`},
} {
b := ReadOwnBuild(func(k string) string {
if k != FingerprintEnv {
t.Fatalf("%s: read %s", c.name, k)
}
return c.env
})
if b.Fingerprint != c.fp || b.Unknown != c.unknown {
t.Errorf("%s: got %+v, want fingerprint %q, unknown %q", c.name, b, c.fp, c.unknown)
}
}
var said []Build
keptSay, keptOnce := sayOwnBuild, ownBuild
t.Cleanup(func() { sayOwnBuild, ownBuild = keptSay, keptOnce })
sayOwnBuild, ownBuild = func(b Build) { said = append(said, b) }, sync.OnceValue(readAndSay)
t.Setenv(FingerprintEnv, "src2:aaaa")
first := OwnBuild()
t.Setenv(FingerprintEnv, "src2:bbbb")
if again := OwnBuild(); again != first || first.Fingerprint != "src2:aaaa" {
t.Errorf("read once: first %+v, then %+v", first, again)
}
if len(said) != 1 || said[0].String() != "src2:aaaa" {
t.Errorf("the journal is told once, at the first read: %+v", said)
}
// Both ways a process reaches the bus read it at the start, before they dial: a refused dial still said it.
for name, dial := range map[string]func(Account) error{
"Connect": func(a Account) error { _, err := Connect(a); return err },
"Dial": func(a Account) error { _, _, err := a.Dial(nats.RetryOnFailedConnect(false)); return err },
} {
said, ownBuild = nil, sync.OnceValue(readAndSay)
if err := dial(Account{URL: "nats://127.0.0.1:1", User: "n1.m", Password: "pw"}); err == nil {
t.Fatalf("%s: dialled a port nothing listens on", name)
}
if len(said) != 1 || said[0].Fingerprint != "src2:bbbb" {
t.Errorf("%s: the process's own build is not said at its start: %+v", name, said)
}
}
}
// TestReadAccountYieldsEveryFieldAndToleratesClaims runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestReadAccountYieldsEveryFieldAndToleratesClaims(t *testing.T) {
t.Run("ReadAccountYieldsEveryFieldAndToleratesClaims", subReadAccountYieldsEveryFieldAndToleratesClaims)
t.Run("TheOwnBuildIsTheFingerprintTheProcessWasStartedWith", subTheOwnBuildIsTheFingerprintTheProcessWasStartedWith)
}
go/process/binding.go
-191
View File
@@ -1,191 +0,0 @@
package process
import (
"encoding/json"
"fmt"
"slices"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/membership"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// Binding is how a process with an account of its own reaches the kept data its `requires` declares: Names takes a local
// name and gives the entry's bus names (bucket, stream and subjects, object stores). It asks `show` of the seat
// the bus's seat for the module and each local name at its start, as a kept call, and asks again when it hears the seat's
// `recreated` or `dropped` for that entry (Register). Until `show` answers, Names answers an error naming the entry and
// the process acts on nothing that needs it; Waiting says so for the part's health (design 47 G §577; ADR 0283).
type Binding struct {
seat string
module string
call func(key string, body any) (json.RawMessage, error)
logf func(string, ...any)
mu sync.Mutex
entries map[string]*boundEntry
locals []string
// spawn runs an ask of its own; a test makes it synchronous.
spawn func(func())
}
type boundEntry struct {
names membership.BusBinding
have bool
why string // why there is no answer yet
ask int // the latest ask; an older answer is dropped
}
// NewBinding is a Binding of module's entries, asked of the seat that holds the bus's state (the caller names it, as it
// names the seat in its `invokes` as `seat:<seat>.show`: this package names no seat), by their local names, asking through call (a Conn's CallKept); logf says
// what it waits for (nil says nothing).
func NewBinding(seat, module string, call func(key string, body any) (json.RawMessage, error), logf func(string, ...any), locals ...string) *Binding {
if logf == nil {
logf = func(string, ...any) {}
}
b := &Binding{seat: seat, module: module, call: call, logf: logf, entries: map[string]*boundEntry{}, spawn: func(f func()) { go f() }}
for _, l := range locals {
if _, ok := b.entries[l]; !ok {
b.entries[l] = &boundEntry{why: "not asked yet"}
b.locals = append(b.locals, l)
}
}
return b
}
// Names are the bus names of the entry local, or an error naming the entry when `show` has not answered for it (or it is
// no entry this Binding was made for).
func (b *Binding) Names(local string) (membership.BusBinding, error) {
b.mu.Lock()
defer b.mu.Unlock()
e := b.entries[local]
switch {
case e == nil:
return membership.BusBinding{}, fmt.Errorf("%s asks no bus entry %q: its Binding holds %s", b.module, local, strings.Join(b.locals, ", "))
case !e.have:
return membership.BusBinding{}, fmt.Errorf("the bus entry %q of %s is not bound yet: %s", local, b.module, e.why)
}
return e.names, nil
}
// Asks says whether local is an entry this Binding was made for.
func (b *Binding) Asks(local string) bool {
b.mu.Lock()
defer b.mu.Unlock()
return b.entries[local] != nil
}
// Waiting is why the process waits, naming each entry `show` has not answered for, or empty when every one is bound.
func (b *Binding) Waiting() string {
b.mu.Lock()
defer b.mu.Unlock()
var out []string
for _, l := range b.locals {
if e := b.entries[l]; !e.have {
out = append(out, fmt.Sprintf("the bus entry %q (%s)", l, e.why))
}
}
if len(out) == 0 {
return ""
}
return "waits for " + strings.Join(out, ", ")
}
// Start asks `show` for every entry once, each as a kept call of its own that returns when answered, so the process's
// start waits for none.
func (b *Binding) Start() {
for _, l := range b.locals {
l := l
b.spawn(func() { b.Ask(l) })
}
}
// Ask asks `show` for the entry local once more, as a kept call, and keeps its answer; the answer of an ask replaced by a
// later one is dropped.
func (b *Binding) Ask(local string) {
b.mu.Lock()
e := b.entries[local]
if e == nil {
b.mu.Unlock()
return
}
e.ask++
mine := e.ask
b.mu.Unlock()
names, err := b.show(local)
b.mu.Lock()
defer b.mu.Unlock()
if e.ask != mine {
return
}
if err != nil {
e.have, e.why = false, err.Error()
b.logf("binding: the bus entry %q of %s is not bound: %v", local, b.module, err)
return
}
e.names, e.have, e.why = names, true, ""
}
func (b *Binding) show(local string) (membership.BusBinding, error) {
raw, err := b.call("seat:"+b.seat+".show", map[string]string{"module": b.module, "local-name": local})
if err != nil {
return membership.BusBinding{}, err
}
var held struct {
Binding *membership.BusBinding `json:"binding"`
State string `json:"state"`
Why string `json:"why"`
}
if err := json.Unmarshal(raw, &held); err != nil {
return membership.BusBinding{}, fmt.Errorf("show answered something that is not an entry: %w", err)
}
if held.Binding == nil {
return membership.BusBinding{}, fmt.Errorf("the bus holds no objects for it yet (%s %s)", held.State, held.Why)
}
return *held.Binding, nil
}
// Register makes the Binding hear `recreated` and `dropped` of the seat on hub, and asks every entry again at each bind
// of the hub's consumer, an event said before it was made being undelivered (ADR 0668). Register before the hub starts.
func (b *Binding) Register(hub *EventHub) error {
for _, event := range []string{"recreated", "dropped"} {
if err := hub.Handle(seatdef.EventSubject(b.seat, event), b.Hear); err != nil {
return err
}
}
hub.OnBound(func(time.Time) {
for _, l := range b.locals {
l := l
b.spawn(func() { b.Ask(l) })
}
})
return nil
}
// Hear takes a `recreated` or `dropped` of the seat: one for an entry of this module's Binding is asked again, and a
// dropped one is unbound until `show` answers. Any other is acknowledged and left.
func (b *Binding) Hear(m jetstream.Msg) {
defer func() { _ = m.Ack() }()
var said struct {
Module string `json:"module"`
Local string `json:"local-name"`
}
if json.Unmarshal(m.Data(), &said) != nil || said.Module != b.module {
return
}
b.mu.Lock()
e := b.entries[said.Local]
if e != nil && strings.HasSuffix(m.Subject(), ".dropped") {
e.have, e.why = false, "the bus dropped it"
}
b.mu.Unlock()
if e != nil {
b.spawn(func() { b.Ask(said.Local) })
}
}
// Locals are the local names this Binding holds, in the order made.
func (b *Binding) Locals() []string { return slices.Clone(b.locals) }
go/process/binding_test.go
-93
View File
@@ -1,93 +0,0 @@
package process
import (
"encoding/json"
"errors"
"strings"
"sync"
"testing"
"github.com/nats-io/nats.go/jetstream"
)
type bindMsg struct {
jetstream.Msg
subject string
data string
acked bool
}
func (m *bindMsg) Subject() string { return m.subject }
func (m *bindMsg) Data() []byte { return []byte(m.data) }
func (m *bindMsg) Ack() error { m.acked = true; return nil }
// design 47 G §577: a process reaches the kept data its `requires` declares through its Binding: `show` of the seat
// mesh-broker at the start, asked again on `recreated` and `dropped` of the entry, and a wait naming the entry until it answers.
func TestABindingAsksShowAndAgainOnRecreatedAndDropped(t *testing.T) {
var mu sync.Mutex
bucket, answers := "m_performed", true
var asked []string
call := func(key string, body any) (json.RawMessage, error) {
mu.Lock()
defer mu.Unlock()
asked = append(asked, key+" "+toJSON(body))
if !answers {
return nil, errors.New("no responders")
}
return json.RawMessage(`{"binding":{"local":"performed","storage":"key-value","bucket":"` + bucket + `"}}`), nil
}
b := NewBinding("mesh-broker", "m", call, nil, "performed", "events")
b.spawn = func(f func()) { f() }
if _, err := b.Names("performed"); err == nil || !strings.Contains(err.Error(), `"performed"`) {
t.Fatalf("before show answered: %v", err)
}
if w := b.Waiting(); !strings.Contains(w, `"performed"`) || !strings.Contains(w, `"events"`) {
t.Fatalf("waiting words %q name neither entry", w)
}
if _, err := b.Names("nope"); err == nil {
t.Fatal("an entry this Binding was not made for was answered")
}
b.Ask("performed")
b.Ask("events")
got, err := b.Names("performed")
if err != nil || got.Bucket != "m_performed" || b.Waiting() != "" {
t.Fatalf("after show: %+v, %v, waiting %q", got, err, b.Waiting())
}
if !strings.Contains(asked[0], `seat:mesh-broker.show {"local-name":"performed","module":"m"}`) {
t.Fatalf("asked %v", asked)
}
// recreated for this module's entry: asked again, the new names kept
mu.Lock()
bucket = "m_performed2"
mu.Unlock()
for _, m := range []*bindMsg{
{subject: "mesh.seat.mesh-broker.event.recreated", data: `{"module":"other","local-name":"performed"}`},
{subject: "mesh.seat.mesh-broker.event.recreated", data: `{"module":"m","local-name":"performed"}`},
} {
b.Hear(m)
if !m.acked {
t.Errorf("%s not acknowledged", m.data)
}
}
if n, _ := b.Names("performed"); n.Bucket != "m_performed2" {
t.Fatalf("recreated was not asked again: %+v", n)
}
// dropped: unbound until show answers, which it does not
mu.Lock()
answers = false
mu.Unlock()
b.Hear(&bindMsg{subject: "mesh.seat.mesh-broker.event.dropped", data: `{"module":"m","local-name":"performed"}`})
if _, err := b.Names("performed"); err == nil || !strings.Contains(err.Error(), "no responders") {
t.Fatalf("dropped left it bound: %v", err)
}
// an entry the bus holds no objects for is waited for, not bound
b2 := NewBinding("mesh-broker", "m", func(string, any) (json.RawMessage, error) {
return json.RawMessage(`{"state":"waiting","why":"not made"}`), nil
}, nil, "x")
b2.Ask("x")
if _, err := b2.Names("x"); err == nil || !strings.Contains(err.Error(), "not made") {
t.Fatalf("an entry with no binding: %v", err)
}
}
func toJSON(v any) string { d, _ := json.Marshal(v); return string(d) }
go/process/call.go
+3 -27
View File
@@ -10,7 +10,6 @@ import (
"github.com/nats-io/nats.go" "github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/busretry" "git.novox.be/novox/mesh-sdk/go/busretry"
"git.novox.be/novox/mesh-sdk/go/calladdress" "git.novox.be/novox/mesh-sdk/go/calladdress"
) )
@@ -25,7 +24,8 @@ var CallTimeout = 30 * time.Second
// mesh-provisioner-client/secrets.go:59-61, 93-97, 119, 125-133). key is "seat:<seat>.<verb>" for a seat's // mesh-provisioner-client/secrets.go:59-61, 93-97, 119, 125-133). key is "seat:<seat>.<verb>" for a seat's
// verb, or "<module>.<tool>" for a module's own tool, as a process's bus.invokes names it. The reply is // verb, or "<module>.<tool>" for a module's own tool, as a process's bus.invokes names it. The reply is
// {result, error, node} (mesh-tools node-tools/internal/bus/bus.go answerOf); a non-empty error is the // {result, error, node} (mesh-tools node-tools/internal/bus/bus.go answerOf); a non-empty error is the
// call's failure. A process answers calls through AnswerSeat (serve.go). A key naming its machine after '@' (`seat:node-service-manager.stop@novox`) is CallOn of that // call's failure. It offers no way to answer a call — a process reaches the bus only as itself, and
// never serves anything. A key naming its machine after '@' (`seat:node-service-manager.stop@novox`) is CallOn of that
// machine; any other malformed key is refused before anything is published (calladdress; novox/hq issue 774). // machine; any other malformed key is refused before anything is published (calladdress; novox/hq issue 774).
func (c *Conn) Call(key string, body any) (json.RawMessage, error) { func (c *Conn) Call(key string, body any) (json.RawMessage, error) {
called, fallback, err := callSubjectsOn(key, "", c.callerToken()) called, fallback, err := callSubjectsOn(key, "", c.callerToken())
@@ -76,20 +76,6 @@ func (c *Conn) CallKeptOn(node, key string, body any) (json.RawMessage, error) {
return c.callKeptOn(called, fallback, body) return c.callKeptOn(called, fallback, body)
} }
// CallKeptWhileOpen is CallKept for an ask the asker can close or replace: it is asked again only while open says its
// key is still open (design 47 G §351, §460).
func (c *Conn) CallKeptWhileOpen(open busretry.Open, key string, body any) (json.RawMessage, error) {
called, fallback, err := callSubjectsOn(key, "", c.callerToken())
if err != nil {
return nil, err
}
data, err := json.Marshal(body)
if err != nil {
return nil, err
}
return busretry.DoKept("the call on "+fallback, open, askBoth(c.request, called, fallback, data), nil)
}
func (c *Conn) callKeptOn(called, fallback string, body any) (json.RawMessage, error) { func (c *Conn) callKeptOn(called, fallback string, body any) (json.RawMessage, error) {
data, err := json.Marshal(body) data, err := json.Marshal(body)
if err != nil { if err != nil {
@@ -189,13 +175,7 @@ func callSubjectsOn(key, node, token string) (called, fallback string, err error
func (c *Conn) request(subject string, data []byte) (json.RawMessage, error) { func (c *Conn) request(subject string, data []byte) (json.RawMessage, error) {
ctx, cancel := context.WithTimeout(context.Background(), CallTimeout) ctx, cancel := context.WithTimeout(context.Background(), CallTimeout)
defer cancel() defer cancel()
req := nats.NewMsg(subject) msg, err := c.nc.RequestWithContext(ctx, subject, data)
req.Data = data
if strings.HasPrefix(subject, "mesh.seat.") {
// Every seat call says it waits for a held call's end; a verb that is not marked ignores it (lead answer A-244).
req.Header.Set(approval.WaitHeader, approval.WaitHeld)
}
msg, err := c.nc.RequestMsgWithContext(ctx, req)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -203,7 +183,6 @@ func (c *Conn) request(subject string, data []byte) (json.RawMessage, error) {
Result json.RawMessage `json:"result"` Result json.RawMessage `json:"result"`
Error string `json:"error"` Error string `json:"error"`
Node string `json:"node"` Node string `json:"node"`
Held *approval.Held `json:"held"`
} }
if err := json.Unmarshal(msg.Data, &r); err != nil { if err := json.Unmarshal(msg.Data, &r); err != nil {
return nil, fmt.Errorf("%s answered something that is not a seat's answer: %w", subject, err) return nil, fmt.Errorf("%s answered something that is not a seat's answer: %w", subject, err)
@@ -211,8 +190,5 @@ func (c *Conn) request(subject string, data []byte) (json.RawMessage, error) {
if r.Error != "" { if r.Error != "" {
return nil, answered(r.Error) return nil, answered(r.Error)
} }
if r.Held != nil {
return c.awaitGated(subject, *r.Held)
}
return r.Result, nil return r.Result, nil
} }
go/process/call_integration_test.go
-95
View File
@@ -6,10 +6,6 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"strings"
"sync"
"sync/atomic"
"testing" "testing"
"time" "time"
@@ -285,94 +281,3 @@ func TestCallOnReachesTheNamedNodeAlone(t *testing.T) {
default: default:
} }
} }
// brokerSeat is the fixture seat's definition as a seat list holds it: its three verbs, each its class's default.
func brokerSeat() (seatdef.Definition, bool) {
return seatdef.Definition{Seat: "broker-seat", Verbs: []seatdef.Verb{{Name: "ping"}, {Name: "fail"}, {Name: "boom"}}}, true
}
// A-107: a process answers its seat's verbs over the bus. Given two instances of one holder answering seat
// `broker-seat`'s verbs, a call on the caller-named subject reaches exactly one, which is handed the caller the
// subject names (dots written `~`) and replies {result, node}; a call on the subject naming none is refused; a
// handler's error is the reply's error, a panic too; and after stop nothing answers.
func TestAnswerSeatServesVerbsWithTheProvedCallerInOneQueueGroup(t *testing.T) {
url, account := grantedSendProcess(t)
adminNC, _ := adminJS(t, url)
account.User, account.Password, account.Node = adminUser, adminPass, "novox"
var heard sync.Map
count := func(name string) SeatHandler {
return func(caller string, body json.RawMessage) (any, error) {
n, _ := heard.LoadOrStore(name, new(int32))
atomic.AddInt32(n.(*int32), 1)
return map[string]string{"caller": caller, "body": string(body)}, nil
}
}
verbs := func(name string) map[string]SeatHandler {
return map[string]SeatHandler{
"ping": count(name),
"fail": func(string, json.RawMessage) (any, error) { return nil, errors.New("refused: not yours") },
"boom": func(string, json.RawMessage) (any, error) { panic("oops") },
}
}
var stops []func()
for _, name := range []string{"a", "b"} {
conn, err := Connect(account)
if err != nil {
t.Fatal(err)
}
defer conn.Close()
stop, err := conn.AnswerSeat("broker-seat", brokerSeat, verbs(name))
if err != nil {
t.Fatalf("AnswerSeat: %v", err)
}
stops = append(stops, stop)
}
ask := func(subject string) (struct {
Result map[string]string `json:"result"`
Error string `json:"error"`
Node string `json:"node"`
}, error) {
var r struct {
Result map[string]string `json:"result"`
Error string `json:"error"`
Node string `json:"node"`
}
msg, err := adminNC.Request(subject, []byte(`{"x":1}`), 5*time.Second)
if err != nil {
return r, err
}
return r, json.Unmarshal(msg.Data, &r)
}
r, err := ask("mesh.seat.broker-seat.call.ping.novox~mesh-delivery-builder")
if err != nil || r.Result["caller"] != "novox.mesh-delivery-builder" || r.Result["body"] != `{"x":1}` || r.Node != "novox" {
t.Fatalf("a caller-named call: %+v, %v", r, err)
}
// An unproven caller is refused before the handler runs (design 47 G §610; lead answers A-165, A-179, A-186).
r, err = ask("mesh.seat.broker-seat.tool.ping")
if err != nil || r.Result != nil || !strings.Contains(r.Error, "broker-seat.ping takes no unproven caller") {
t.Fatalf("an unproven call: %+v, %v", r, err)
}
if _, err := ask("mesh.seat.broker-seat.call.ping.novox~mesh-delivery-builder"); err != nil {
t.Fatal(err)
}
total := int32(0)
heard.Range(func(_, v any) bool { total += atomic.LoadInt32(v.(*int32)); return true })
if total != 2 {
t.Errorf("two proven calls were answered %d times in all, want once each (one queue group) and the unproven none", total)
}
if r, err = ask("mesh.seat.broker-seat.call.fail.x"); err != nil || r.Error != "refused: not yours" {
t.Errorf("a handler's error: %+v, %v", r, err)
}
if r, err = ask("mesh.seat.broker-seat.call.boom.x"); err != nil || !strings.Contains(r.Error, "oops") {
t.Errorf("a handler's panic: %+v, %v", r, err)
}
for _, stop := range stops {
stop()
}
_ = adminNC.Flush()
if _, err := adminNC.Request("mesh.seat.broker-seat.call.ping.x", nil, time.Second); !errors.Is(err, nats.ErrNoResponders) {
t.Errorf("after stop a call reached someone: %v", err)
}
}
go/process/call_test.go
-51
View File
@@ -3,7 +3,6 @@ package process
import ( import (
"context" "context"
"encoding/json" "encoding/json"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -137,53 +136,3 @@ func TestACallIsAskedAgainOnlyWhenItReachedNoOne(t *testing.T) {
} }
} }
} }
// A-107: AnswerSeat reads the caller from a call's last token (dots written `~`), "" for anything that is not a
// bus user's name; and it refuses what it cannot serve before subscribing. A-179, A-186: before a handler runs, an
// unproven caller is refused, and every caller while the seat's definition is unread, names no such verb or opens it to
// none; a proven caller of a verb the definition names runs it.
func TestUserOfTokenAndAnswerSeatRefusals(t *testing.T) {
for token, want := range map[string]string{"novox~mesh-delivery-builder": "novox.mesh-delivery-builder", "person~jo": "person.jo",
"plain": "plain", "": "", "a b": "", "a*": "", "~": ""} {
if got := userOfToken(token); got != want {
t.Errorf("userOfToken(%q) = %q, want %q", token, got, want)
}
}
c := &Conn{}
ok := func(string, json.RawMessage) (any, error) { return nil, nil }
defined := func() (seatdef.Definition, bool) {
return seatdef.Definition{Seat: "seat", Verbs: []seatdef.Verb{{Name: "v"}}}, true
}
for name, verbs := range map[string]map[string]SeatHandler{"none": nil, "nil handler": {"v": nil}, "bad verb": {"a b": ok}, "wildcard": {"*": ok}} {
if _, err := c.AnswerSeat("seat", defined, verbs); err == nil {
t.Errorf("AnswerSeat accepted %s", name)
}
}
if _, err := c.AnswerSeat("seat", nil, map[string]SeatHandler{"v": ok}); err == nil {
t.Error("AnswerSeat accepted no definition")
}
ran := 0
run := func(string, json.RawMessage) (any, error) { ran++; return "ran", nil }
for _, row := range []struct {
name, caller string
def func() (seatdef.Definition, bool)
says string
}{
{"an unproven caller", "", defined, "takes no unproven caller"},
{"the definition unread", "novox.mesh-recovery", func() (seatdef.Definition, bool) { return seatdef.Definition{}, false }, "not yet read"},
{"a verb the definition does not name", "novox.mesh-recovery", func() (seatdef.Definition, bool) { return seatdef.Definition{Seat: "seat"}, true }, "names no verb v"},
{"a verb open to none", "novox.mesh-recovery", func() (seatdef.Definition, bool) {
return seatdef.Definition{Seat: "seat", Verbs: []seatdef.Verb{{Name: "v", Callers: &seatdef.Callers{List: []string{}}}}}, true
}, "open to no caller"},
{"a marked verb with no gate (§677)", "novox.mesh-recovery", func() (seatdef.Definition, bool) {
return seatdef.Definition{Seat: "seat", Verbs: []seatdef.Verb{{Name: "v", NeedsApproval: true}}}, true
}, "through no gate"},
{"a proven caller", "novox.mesh-recovery", defined, ""},
} {
before := ran
_, _, err := c.admitted("seat", "v", row.def, run, nil)(row.caller, nil, false)
if row.says == "" && (err != nil || ran != before+1) || row.says != "" && (err == nil || !strings.Contains(err.Error(), row.says) || ran != before) {
t.Errorf("%s: %v, ran %d times", row.name, err, ran-before)
}
}
}
go/process/chunks.go
-110
View File
@@ -1,110 +0,0 @@
package process
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"time"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/calladdress"
)
// CallChunks calls a seat verb whose answer file is sent in chunks (a verb its seat definition marks `chunks`, design 47 G
// §681, A-139), the one way a process receives such an answer (A-291): it subscribes to the chunks of call (the 128 random
// bits the verb's input carries as its call) before it asks, so none is missed, asks key ("seat:<seat>.<verb>[@<node>]")
// with body as Call does, hands the verb's answer to accept (nil accepts every answer; an error ends the call before a chunk
// is taken), then receives the number of chunks the answer's member `chunks` names, in order, acknowledging each, into w.
// It answers the verb's answer. A chunk that does not come within buscalls.SeatAckWithin, or ctx ending, ends the receive.
func (c *Conn) CallChunks(ctx context.Context, key, call string, body any, accept func(answer json.RawMessage) error, w io.Writer) (json.RawMessage, error) {
a, err := calladdress.Parse(key)
if err != nil {
return nil, err
}
if a.Seat == "" {
return nil, fmt.Errorf("%q names no seat verb, so it sends no chunks", key)
}
if call == "" {
return nil, fmt.Errorf("the call of %s names no call token for its chunks", key)
}
send := buscalls.SeatChunks{Seat: a.Seat, Verb: a.Verb, Call: call}
chunks := make(chan *nats.Msg, 4)
sub, err := c.nc.ChanSubscribe(send.Chunks(), chunks)
if err != nil {
return nil, fmt.Errorf("the chunks of %s cannot be heard: %w", key, err)
}
defer func() { _ = sub.Unsubscribe() }()
answer, err := c.Call(key, body)
if err != nil {
return nil, err
}
if accept != nil {
if err := accept(answer); err != nil {
return answer, err
}
}
var count struct {
Chunks int64 `json:"chunks"`
}
if err := json.Unmarshal(answer, &count); err != nil {
return answer, fmt.Errorf("%s answered no count of its chunks: %w", key, err)
}
next := func() (string, []byte, error) {
bound := time.NewTimer(buscalls.SeatAckWithin)
defer bound.Stop()
select {
case m := <-chunks:
return m.Subject, m.Data, nil
case <-bound.C:
return "", nil, buscalls.ErrSeatChunkMissed
case <-ctx.Done():
return "", nil, ctx.Err()
}
}
ack := func(subject string) error {
_, err := c.nc.Request(subject, []byte("{}"), buscalls.SeatAckWithin)
return err
}
if _, err := send.Receive(count.Chunks, next, ack, w); err != nil {
return answer, fmt.Errorf("the answer of %s: %w", key, err)
}
return answer, nil
}
// CallFile is CallChunks for an answer read whole, in the form a bundle's ordinary seat call answers a chunked verb (design 47
// G A-249, A-253): the verb's answer with the whole file added as its member `file` (base64), none when no chunk was sent. The
// call token is body's member `call`.
func (c *Conn) CallFile(key string, body any) (json.RawMessage, error) {
call := callOf(body)
var file bytes.Buffer
answer, err := c.CallChunks(context.Background(), key, call, body, nil, &file)
if err != nil {
return nil, err
}
var whole map[string]any
if err := json.Unmarshal(answer, &whole); err != nil {
return nil, fmt.Errorf("%s answered what is no object: %w", key, err)
}
if file.Len() > 0 {
whole["file"] = base64.StdEncoding.EncodeToString(file.Bytes())
}
return json.Marshal(whole)
}
// callOf is the member call of a verb's input.
func callOf(body any) string {
raw, err := json.Marshal(body)
if err != nil {
return ""
}
var in struct {
Call string `json:"call"`
}
_ = json.Unmarshal(raw, &in)
return in.Call
}
go/process/chunks_integration_test.go
-71
View File
@@ -1,71 +0,0 @@
//go:build integration
package process
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"errors"
"strings"
"testing"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/testbus"
)
// Design 47 G A-291: a process receives a seat verb's chunked answer through one helper: it subscribes to the call's chunks
// before it asks, acknowledges each, and answers the whole body (CallChunks into a writer, CallFile as the member file);
// an answer accept refuses takes no chunk.
func TestAProcessReceivesAChunkedSeatAnswerWhole(t *testing.T) {
url := testbus.URL(t)
holder, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer holder.Close()
body := []byte(strings.Repeat("0123456789", 2*buscalls.SeatChunkBytes/10+7)) // three chunks
of := int64((len(body) + buscalls.SeatChunkBytes - 1) / buscalls.SeatChunkBytes)
sub, err := holder.Subscribe("mesh.seat.node-build-go.call.output.ace.>", func(m *nats.Msg) {
var in struct{ Ask, Call string }
_ = json.Unmarshal(m.Data, &in)
answer, _ := json.Marshal(map[string]any{"result": map[string]any{"ask": in.Ask, "chunks": of}})
_ = m.Respond(answer)
go func() {
_ = buscalls.SeatChunks{Seat: "node-build-go", Verb: "output", Call: in.Call}.Send(holder, body, buscalls.SeatAckWithin, t.Logf)
}()
})
if err != nil {
t.Fatal(err)
}
defer func() { _ = sub.Unsubscribe() }()
_ = holder.Flush()
nc, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
c := &Conn{nc: nc, account: Account{User: "novox.mesh-delivery-checker"}}
defer c.Close()
var got bytes.Buffer
answer, err := c.CallChunks(context.Background(), "seat:node-build-go.output@ace", "c1", map[string]any{"ask": "a1", "call": "c1"}, nil, &got)
if err != nil || !bytes.Equal(got.Bytes(), body) || !strings.Contains(string(answer), `"ask":"a1"`) {
t.Fatalf("answered %s and %d bytes (%v), want the verb's answer and the whole %d bytes", answer, got.Len(), err, len(body))
}
raw, err := c.CallFile("seat:node-build-go.output@ace", map[string]any{"ask": "a2", "call": "c2"})
var file struct{ Ask, File string }
if err != nil || json.Unmarshal(raw, &file) != nil || file.Ask != "a2" {
t.Fatalf("CallFile answered %.80s, %v", raw, err)
}
if whole, _ := base64.StdEncoding.DecodeString(file.File); !bytes.Equal(whole, body) {
t.Fatalf("CallFile's file holds %d bytes, want %d", len(whole), len(body))
}
refused := errors.New("too large for this disk")
got.Reset()
if _, err := c.CallChunks(context.Background(), "seat:node-build-go.output@ace", "c3", map[string]any{"ask": "a3", "call": "c3"},
func(json.RawMessage) error { return refused }, &got); !errors.Is(err, refused) || got.Len() != 0 {
t.Fatalf("an answer accept refused: %v, %d bytes taken", err, got.Len())
}
}
go/process/conn.go
+2 -3
View File
@@ -15,8 +15,8 @@ import (
) )
// Conn is one process's own connection to the bus (novox/hq ADR 0436 clause 4): dialled with the account // Conn is one process's own connection to the bus (novox/hq ADR 0436 clause 4): dialled with the account
// the controller delivered, pinned to its fingerprint, on its own inbox prefix. It answers its seat's verbs // the controller delivered, pinned to its fingerprint, on its own inbox prefix. It offers no Serve and no
// through AnswerSeat alone (design 47 G, lead answer A-107) and offers no other way to serve a tool. // way to serve a tool — a process reaches the bus only as itself, and never answers a call.
type Conn struct { type Conn struct {
nc *nats.Conn nc *nats.Conn
account Account account Account
@@ -53,7 +53,6 @@ var refusedPublish = regexp.MustCompile(`(?i)Publish to "(\S+)"`)
// permission violation refusing a publish this connection is waiting an answer to, and cancels that wait // permission violation refusing a publish this connection is waiting an answer to, and cancels that wait
// at once. // at once.
func Connect(account Account, opts ...nats.Option) (*Conn, error) { func Connect(account Account, opts ...nats.Option) (*Conn, error) {
OwnBuild()
c := &Conn{account: account, pending: map[string]map[uint64]context.CancelCauseFunc{}} c := &Conn{account: account, pending: map[string]map[uint64]context.CancelCauseFunc{}}
nc, err := nats.Connect(account.URL, c.options(opts)...) nc, err := nats.Connect(account.URL, c.options(opts)...)
if err != nil { if err != nil {
go/process/consumers.go
-131
View File
@@ -1,131 +0,0 @@
package process
import (
"context"
"encoding/json"
"sync"
"time"
"github.com/nats-io/nats.go/jetstream"
)
// ConsumersState is the name of the state a module keeps its durable consumers' facts in, one key per consumer by its name,
// history 1 (design 47 G A-166, A-176): what a module's tool reads of a consumer, since a tool runs in a runtime of its own and
// cannot read the consumer's information.
const ConsumersState = "consumers"
// ConsumerFacts is one durable consumer as ConsumerLedger keeps it.
type ConsumerFacts struct {
// Stream and Name are the consumer's stream and its name.
Stream string `json:"stream"`
Name string `json:"name"`
// Acknowledged is the stream sequence of the last message applied.
Acknowledged uint64 `json:"acknowledged"`
// Pending is the messages the consumer has not been given yet, as the last message said.
Pending uint64 `json:"pending"`
// LastEvent is the subject of the last event applied and LastAt its own `at`, or when the bus stored it where it has none.
LastEvent string `json:"last-event,omitempty"`
LastAt time.Time `json:"last-at,omitempty"`
// OldestUnacknowledged is when the oldest message delivered and not yet applied was delivered; zero when none is.
OldestUnacknowledged time.Time `json:"oldest-unacknowledged,omitempty"`
}
// ConsumerSink stores one key of the module's state consumers.
type ConsumerSink func(ctx context.Context, key string, value []byte) error
// ConsumerLedger writes a durable consumer's facts into the module's state consumers after each message delivered and each message
// applied, so a tool reads them and nothing is read on a timer (design 47 G A-166, A-176). One ledger serves one consumer.
type ConsumerLedger struct {
stream, name string
sink ConsumerSink
now func() time.Time
logf func(string, ...any)
mu sync.Mutex
facts ConsumerFacts
unacks map[uint64]time.Time // delivered and not applied, by stream sequence
}
// NewConsumerLedger is the ledger of the consumer name on stream, writing through sink; a nil now is the clock and a nil logf says
// nothing.
func NewConsumerLedger(stream, name string, sink ConsumerSink, now func() time.Time, logf func(string, ...any)) *ConsumerLedger {
if now == nil {
now = time.Now
}
if logf == nil {
logf = func(string, ...any) {}
}
return &ConsumerLedger{stream: stream, name: name, sink: sink, now: now, logf: logf, unacks: map[uint64]time.Time{},
facts: ConsumerFacts{Stream: stream, Name: name}}
}
// Delivered notes that m was given to the consumer, and writes the facts; a message given again keeps its first delivery's time.
func (l *ConsumerLedger) Delivered(m jetstream.Msg) {
md, err := m.Metadata()
if err != nil {
l.logf("consumers: %s's message has no metadata, its delivery is not kept: %v", l.name, err)
return
}
l.mu.Lock()
// A message delivered again keeps the time it was first delivered, so one that never settles grows old (A-187): asked again
// in this run, the time kept; after a restart, which kept none, the time the bus stored it, never later than its first
// delivery.
if _, known := l.unacks[md.Sequence.Stream]; !known {
first := l.now().UTC()
if md.NumDelivered > 1 && !md.Timestamp.IsZero() && md.Timestamp.Before(first) {
first = md.Timestamp.UTC()
}
l.unacks[md.Sequence.Stream] = first
}
l.facts.Pending = md.NumPending
l.mu.Unlock()
l.write()
}
// Applied notes that m was applied and acknowledged, at the event's own `at` (zero: when the bus stored it), and writes the facts.
func (l *ConsumerLedger) Applied(m jetstream.Msg, at time.Time) {
md, err := m.Metadata()
if err != nil {
l.logf("consumers: %s's message has no metadata, its application is not kept: %v", l.name, err)
return
}
if at.IsZero() {
at = md.Timestamp
}
l.mu.Lock()
delete(l.unacks, md.Sequence.Stream)
if md.Sequence.Stream > l.facts.Acknowledged {
l.facts.Acknowledged = md.Sequence.Stream
}
l.facts.Pending = md.NumPending
l.facts.LastEvent, l.facts.LastAt = m.Subject(), at.UTC()
l.mu.Unlock()
l.write()
}
// Facts are the facts now.
func (l *ConsumerLedger) Facts() ConsumerFacts {
l.mu.Lock()
defer l.mu.Unlock()
f := l.facts
f.OldestUnacknowledged = time.Time{}
for _, t := range l.unacks {
if f.OldestUnacknowledged.IsZero() || t.Before(f.OldestUnacknowledged) {
f.OldestUnacknowledged = t
}
}
return f
}
func (l *ConsumerLedger) write() {
if l.sink == nil {
return
}
data, err := json.Marshal(l.Facts())
if err == nil {
err = l.sink(context.Background(), l.name, data)
}
if err != nil {
l.logf("consumers: the facts of %s are not written to the state %s: %v", l.name, ConsumersState, err)
}
}
go/process/consumers_test.go
-67
View File
@@ -1,67 +0,0 @@
package process
import (
"context"
"encoding/json"
"testing"
"time"
"github.com/nats-io/nats.go/jetstream"
)
type ledgerMsg struct {
jetstream.Msg
subject string
seq uint64
pending uint64
stored time.Time
times uint64
}
func (m ledgerMsg) Subject() string { return m.subject }
func (m ledgerMsg) Metadata() (*jetstream.MsgMetadata, error) {
return &jetstream.MsgMetadata{Sequence: jetstream.SequencePair{Stream: m.seq, Consumer: m.seq}, NumPending: m.pending, Timestamp: m.stored, NumDelivered: max(m.times, 1)}, nil
}
func TestConsumerLedgerKeepsTheFactsAfterEachMessage(t *testing.T) {
kept := map[string][]byte{}
now := time.Date(2026, 11, 12, 9, 0, 0, 0, time.UTC)
l := NewConsumerLedger("EVENTS", "n1_m", func(_ context.Context, key string, value []byte) error { kept[key] = value; return nil }, func() time.Time { return now }, nil)
read := func() ConsumerFacts {
var f ConsumerFacts
if err := json.Unmarshal(kept["n1_m"], &f); err != nil {
t.Fatal(err)
}
return f
}
a, b := ledgerMsg{subject: "mesh.seat.s.event.a", seq: 7, pending: 1}, ledgerMsg{subject: "mesh.seat.s.event.b", seq: 8, pending: 0}
l.Delivered(a)
now = now.Add(time.Minute)
l.Delivered(b)
if f := read(); f.Pending != 0 || !f.OldestUnacknowledged.Equal(now.Add(-time.Minute)) {
t.Errorf("after two deliveries: %+v, want pending 0 and the first one's delivery as the oldest unacknowledged", f)
}
// A message delivered again keeps its first delivery's time (A-187): asked again in this run, and after a restart, where the
// new ledger knows none, the time the bus stored it.
now = now.Add(time.Minute)
l.Delivered(ledgerMsg{subject: a.subject, seq: 7, pending: 0, times: 2})
if f := read(); !f.OldestUnacknowledged.Equal(now.Add(-2 * time.Minute)) {
t.Errorf("after a delivery again: %+v, want the first delivery kept as the oldest unacknowledged", f)
}
stored := now.Add(-time.Hour)
again := NewConsumerLedger("EVENTS", "n1_m", func(_ context.Context, key string, value []byte) error { kept[key] = value; return nil }, func() time.Time { return now }, nil)
again.Delivered(ledgerMsg{subject: a.subject, seq: 7, stored: stored, times: 3})
if f := read(); !f.OldestUnacknowledged.Equal(stored) {
t.Errorf("after a restart's delivery again: %+v, want the time the bus stored it", f)
}
now = now.Add(-time.Minute)
at := now.Add(-time.Hour)
l.Applied(a, at)
if f := read(); f.Acknowledged != 7 || f.LastEvent != "mesh.seat.s.event.a" || !f.LastAt.Equal(at) || !f.OldestUnacknowledged.Equal(now) {
t.Errorf("after the first applied: %+v, want sequence 7, its event and at, and the second one's delivery as the oldest", f)
}
l.Applied(b, time.Time{})
if f := read(); f.Acknowledged != 8 || !f.OldestUnacknowledged.IsZero() {
t.Errorf("after both applied: %+v, want sequence 8 and none unacknowledged", f)
}
}
go/process/emit.go
+1 -20
View File
@@ -19,25 +19,6 @@ import (
// (EmitOptions.MessageID) is sent as x-event-id, and as the bus's own dedup id (Nats-Msg-Id), so a second // (EmitOptions.MessageID) is sent as x-event-id, and as the bus's own dedup id (Nats-Msg-Id), so a second
// publish inside the stream's duplicate window is dropped; empty is a random one, as stdio.Emit on main. // publish inside the stream's duplicate window is dropped; empty is a random one, as stdio.Emit on main.
func (c *Conn) Emit(eventType string, body any, opts ...stdio.EmitOptions) error { func (c *Conn) Emit(eventType string, body any, opts ...stdio.EmitOptions) error {
return c.emitOn("mesh.mod."+c.account.Module+".event."+eventType, body, opts)
}
// EmitSeat says a seat's event as its holder (design 47 G A-10, §276, §498): one call, publishing on
// mesh.seat.<seat>.event.<event>, with the same envelope, headers and message id as Emit. The grants part grants every
// holder the publish of its held seats' event subjects.
func (c *Conn) EmitSeat(seat, event string, body any, opts ...stdio.EmitOptions) error {
return c.emitOn("mesh.seat."+seat+".event."+event, body, opts)
}
// Accept sends the seat's holder an accepted kind (design 47 G §677, lead answer A-273): one call, publishing on
// mesh.seat.<seat>.accept.<kind>.<caller>, the caller being this process's module, with the same envelope, headers and
// message id as Emit. The grants part grants every user of a seat the publish of its accept subjects that name it.
func (c *Conn) Accept(seat, kind string, body any, opts ...stdio.EmitOptions) error {
return c.emitOn("mesh.seat."+seat+".accept."+kind+"."+c.account.Module, body, opts)
}
// emitOn publishes body on subject with the SDK's envelope.
func (c *Conn) emitOn(subject string, body any, opts []stdio.EmitOptions) error {
data, err := json.Marshal(body) data, err := json.Marshal(body)
if err != nil { if err != nil {
return err return err
@@ -51,7 +32,7 @@ func (c *Conn) emitOn(subject string, body any, opts []stdio.EmitOptions) error
if id == "" { if id == "" {
id = eventID() id = eventID()
} }
msg := nats.NewMsg(subject) msg := nats.NewMsg("mesh.mod." + c.account.Module + ".event." + eventType)
msg.Header.Set("x-event-id", id) msg.Header.Set("x-event-id", id)
msg.Header.Set("x-source", c.account.Module) msg.Header.Set("x-source", c.account.Module)
msg.Header.Set("x-node", c.account.Node) msg.Header.Set("x-node", c.account.Node)
go/process/emit_integration_test.go
+3 -19
View File
@@ -19,7 +19,7 @@ func TestEmitTwiceWithTheSameIDIsKeptToOneWithTheSDKsHeaders(t *testing.T) {
account.Node = "novox" account.Node = "novox"
_, adminJetStream := adminJS(t, url) _, adminJetStream := adminJS(t, url)
stream, err := adminJetStream.CreateStream(ctxFor(t), jetstream.StreamConfig{ stream, err := adminJetStream.CreateStream(ctxFor(t), jetstream.StreamConfig{
Name: "MAIL_EVENTS", Subjects: []string{"mesh.mod.mesh-mail.event.>", "mesh.seat.mail-sender.event.>", "mesh.seat.mail-sender.accept.>"}, Duplicates: time.Minute, Name: "MAIL_EVENTS", Subjects: []string{"mesh.mod.mesh-mail.event.>"}, Duplicates: time.Minute,
}) })
if err != nil { if err != nil {
t.Fatalf("the admin could not make the events stream: %v", err) t.Fatalf("the admin could not make the events stream: %v", err)
@@ -37,28 +37,12 @@ func TestEmitTwiceWithTheSameIDIsKeptToOneWithTheSDKsHeaders(t *testing.T) {
} }
} }
// A seat's event goes on mesh.seat.<seat>.event.<event> with the same headers (design 47 G A-10).
if err := conn.EmitSeat("mail-sender", "sent", map[string]any{"send": "s1"}, stdio.EmitOptions{MessageID: "seat-s1"}); err != nil {
t.Fatalf("EmitSeat: %v", err)
}
if m, err := stream.GetLastMsgForSubject(ctxFor(t), "mesh.seat.mail-sender.event.sent"); err != nil || m.Header.Get("x-event-id") != "seat-s1" || m.Header.Get("x-source") != "mesh-mail" {
t.Fatalf("the seat's event: %v, %v", m, err)
}
// An accepted kind goes on mesh.seat.<seat>.accept.<kind>.<caller>, the caller this process's module (A-273).
if err := conn.Accept("mail-sender", "send", map[string]any{"send": "s2"}, stdio.EmitOptions{MessageID: "accept-s2"}); err != nil {
t.Fatalf("Accept: %v", err)
}
if m, err := stream.GetLastMsgForSubject(ctxFor(t), "mesh.seat.mail-sender.accept.send.mesh-mail"); err != nil || m.Header.Get("x-event-id") != "accept-s2" || m.Header.Get("x-source") != "mesh-mail" {
t.Fatalf("the accepted kind: %v, %v", m, err)
}
info, err := stream.Info(ctxFor(t)) info, err := stream.Info(ctxFor(t))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if info.State.Msgs != 3 { if info.State.Msgs != 1 {
t.Fatalf("the stream holds %d messages, want 3: one of the doubled publish, one of the seat's event, one accepted", info.State.Msgs) t.Fatalf("the stream holds %d messages, want exactly 1 (the second publish is a duplicate)", info.State.Msgs)
} }
msg, err := stream.GetMsg(ctxFor(t), 1) msg, err := stream.GetMsg(ctxFor(t), 1)
go/process/event_hub_integration_test.go
+1 -215
View File
@@ -4,7 +4,6 @@ package process_test
import ( import (
"context" "context"
"strings"
"testing" "testing"
"time" "time"
@@ -12,13 +11,12 @@ import (
"github.com/nats-io/nats.go/jetstream" "github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/process" "git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"git.novox.be/novox/mesh-sdk/go/testbus" "git.novox.be/novox/mesh-sdk/go/testbus"
) )
// A module whose consumer the bus makes after the module started is not lost for that order: the hub tries again after // A module whose consumer the bus makes after the module started is not lost for that order: the hub tries again after
// each wait and calls live once it is bound (the checker's "until it starts again" hole). // each wait and calls live once it is bound (the checker's "until it starts again" hole).
func subTheHubBindsALateConsumerAndSaysItIsLive(t *testing.T) { func TestTheHubBindsALateConsumerAndSaysItIsLive(t *testing.T) {
nc, err := nats.Connect(testbus.URL(t)) nc, err := nats.Connect(testbus.URL(t))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -67,215 +65,3 @@ func subTheHubBindsALateConsumerAndSaysItIsLive(t *testing.T) {
} }
hub.Stop() hub.Stop()
} }
// W1.12: a module hears a seat's event kept last per subject on its own stream through its durable, made by the maker
// after the events were said, and is handed the last of each subject; without the consumer the call says so.
func subAModuleHearsASeatEventKeptOnItsOwnStream(t *testing.T) {
nc, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
kept, subjects := seatdef.KeptStream("node-engine", seatdef.Message{Name: "running"}), []string{"mesh.seat.node-engine.event.running.>"}
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: kept, Subjects: subjects, MaxMsgsPerSubject: 1}); err != nil {
t.Fatal(err)
}
for _, m := range []struct{ subject, fp string }{
{"mesh.seat.node-engine.event.running.ace.mesh-tools", "fp1"},
{"mesh.seat.node-engine.event.running.ace.mesh-tools", "fp2"},
{"mesh.seat.node-engine.event.running.g14.mesh-tools", "fp3"},
} {
if _, err := js.Publish(ctx, m.subject, []byte(m.fp)); err != nil {
t.Fatal(err)
}
}
cred := process.Account{Node: "ace", Module: "mesh-grants"}
got := make(chan string, 4)
hear := func(m jetstream.Msg) { got <- m.Subject() + "=" + string(m.Data()); _ = m.Ack() }
if _, err := cred.HearSeatEvents(ctx, js, kept, hear, t.Logf); err == nil {
t.Fatal("no consumer was made, and hearing said nothing")
}
// The durable has gone past every message (it starts from now), so only the start read, an ordered consumer on the stream
// kept last per subject, gives the last of each subject (design 47 G §641, A-19).
durable := cred.SeatEventConsumerConfig(subjects, true)
durable.DeliverPolicy = jetstream.DeliverNewPolicy
if _, err := js.CreateOrUpdateConsumer(ctx, kept, durable); err != nil {
t.Fatal(err)
}
var reads []string
cc, err := cred.HearSeatEvents(ctx, js, kept, hear, t.Logf, process.OnRead(func(s string) { reads = append(reads, s) }))
if err != nil {
t.Fatal(err)
}
defer cc.Stop()
heard := map[string]bool{}
for len(heard) < 2 {
select {
case h := <-got:
heard[h] = true
case <-ctx.Done():
t.Fatalf("heard %v", heard)
}
}
if !heard["mesh.seat.node-engine.event.running.ace.mesh-tools=fp2"] || !heard["mesh.seat.node-engine.event.running.g14.mesh-tools=fp3"] {
t.Errorf("heard %v, want the last of each subject", heard)
}
if len(reads) != 2 || reads[0] != process.ReadWaiting || reads[1] != process.ReadDone {
t.Errorf("the read said %v, want waiting then read", reads)
}
// then the durable is followed
if _, err := js.Publish(ctx, "mesh.seat.node-engine.event.running.ace.mesh-tools", []byte("fp4")); err != nil {
t.Fatal(err)
}
select {
case h := <-got:
if h != "mesh.seat.node-engine.event.running.ace.mesh-tools=fp4" {
t.Errorf("the durable heard %s", h)
}
case <-ctx.Done():
t.Fatal("the durable was not followed after the read")
}
}
// A-181: a module hears another module's own stream kept last per subject (a manifest's `streams` entry with `last-per-subject`) as
// it hears a seat's: the last message of each subject read whole at its start, though its durable began after them, then the durable.
func subAModuleHearsAnotherModulesStreamKeptLastPerSubject(t *testing.T) {
nc, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
stream, subject := "BUILDER_BUILDS", process.ModuleStreamSubject("builder", "build.ended.>")
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: stream, Subjects: []string{subject}, MaxMsgsPerSubject: 1}); err != nil {
t.Fatal(err)
}
for _, m := range []struct{ event, v string }{{"build.ended.web.fp1", "old"}, {"build.ended.web.fp1", "new"}, {"build.ended.web.fp2", "two"}} {
if _, err := js.Publish(ctx, process.ModuleStreamSubject("builder", m.event), []byte(m.v)); err != nil {
t.Fatal(err)
}
}
cred := process.Account{Node: "ace", Module: "assigner"}
durable := cred.SeatEventConsumerConfig([]string{subject}, true)
durable.DeliverPolicy = jetstream.DeliverNewPolicy
if _, err := js.CreateOrUpdateConsumer(ctx, stream, durable); err != nil {
t.Fatal(err)
}
got := make(chan string, 4)
var reads []string
cc, err := cred.HearStream(ctx, js, stream, func(m jetstream.Msg) { got <- string(m.Data()); _ = m.Ack() }, t.Logf, process.OnRead(func(s string) { reads = append(reads, s) }))
if err != nil {
t.Fatal(err)
}
defer cc.Stop()
heard := map[string]bool{}
for len(heard) < 2 {
select {
case h := <-got:
heard[h] = true
case <-ctx.Done():
t.Fatalf("heard %v", heard)
}
}
if !heard["new"] || !heard["two"] || heard["old"] || len(reads) != 2 {
t.Errorf("heard %v with reads %v, want the last of each subject, read whole", heard, reads)
}
}
// A-233: a module hears a stream kept by age with FromFirst: at every start the stream is read whole from its first message, though
// its durable began after them, then the durable is followed; without FromFirst only the durable is.
func subAModuleHearsAStreamKeptByAgeFromItsFirstMessage(t *testing.T) {
nc, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
kept, subjects := seatdef.KeptStream("mesh-catalog", seatdef.Message{Name: "version-released"}), []string{"mesh.seat.mesh-catalog.event.version-released"}
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: kept, Subjects: subjects, MaxAge: 90 * 24 * time.Hour}); err != nil {
t.Fatal(err)
}
for _, v := range []string{"web 1.0.0", "web 1.1.0", "api 2.0.0"} {
if _, err := js.Publish(ctx, subjects[0], []byte(v)); err != nil {
t.Fatal(err)
}
}
cred := process.Account{Node: "ace", Module: "assigner"}
durable := cred.SeatEventConsumerConfig(subjects, false)
durable.DeliverPolicy = jetstream.DeliverNewPolicy
if _, err := js.CreateOrUpdateConsumer(ctx, kept, durable); err != nil {
t.Fatal(err)
}
got := make(chan string, 8)
hear := func(m jetstream.Msg) { got <- string(m.Data()); _ = m.Ack() }
var reads []string
cc, err := cred.HearSeatEvents(ctx, js, kept, hear, t.Logf, process.FromFirst(), process.OnRead(func(s string) { reads = append(reads, s) }))
if err != nil {
t.Fatal(err)
}
var heard []string
for len(heard) < 3 {
select {
case h := <-got:
heard = append(heard, h)
case <-ctx.Done():
t.Fatalf("heard %v, want every message from the first", heard)
}
}
if strings.Join(heard, ",") != "web 1.0.0,web 1.1.0,api 2.0.0" || len(reads) != 2 {
t.Errorf("heard %v with reads %v, want every message in order, read whole", heard, reads)
}
if _, err := js.Publish(ctx, subjects[0], []byte("web 1.2.0")); err != nil {
t.Fatal(err)
}
select {
case h := <-got:
if h != "web 1.2.0" {
t.Errorf("the durable heard %s", h)
}
case <-ctx.Done():
t.Fatal("the durable was not followed after the read")
}
cc.Stop()
// Without FromFirst, a stream kept by age is not read: the durable alone, past every message, hands the next one first.
cc, err = cred.HearSeatEvents(ctx, js, kept, hear, t.Logf)
if err != nil {
t.Fatal(err)
}
defer cc.Stop()
if _, err := js.Publish(ctx, subjects[0], []byte("next")); err != nil {
t.Fatal(err)
}
select {
case h := <-got:
if h != "next" {
t.Errorf("without FromFirst %s was handed before the next message", h)
}
case <-ctx.Done():
t.Fatal("the durable was not followed")
}
}
// TestTheHubBindsALateConsumerAndSaysItIsLive runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestTheHubBindsALateConsumerAndSaysItIsLive(t *testing.T) {
t.Run("TheHubBindsALateConsumerAndSaysItIsLive", subTheHubBindsALateConsumerAndSaysItIsLive)
t.Run("AModuleHearsASeatEventKeptOnItsOwnStream", subAModuleHearsASeatEventKeptOnItsOwnStream)
t.Run("AModuleHearsAnotherModulesStreamKeptLastPerSubject", subAModuleHearsAnotherModulesStreamKeptLastPerSubject)
t.Run("AModuleHearsAStreamKeptByAgeFromItsFirstMessage", subAModuleHearsAStreamKeptByAgeFromItsFirstMessage)
}
go/process/event_hub_test.go
-21
View File
@@ -1,8 +1,6 @@
package process package process
import ( import (
"context"
"strings"
"testing" "testing"
"github.com/nats-io/nats.go/jetstream" "github.com/nats-io/nats.go/jetstream"
@@ -37,23 +35,4 @@ func TestTheHubMatchesSubjectsAndRefusesOverlappingPatterns(t *testing.T) {
if err := h.Handle("mesh.mod.q.event.>", nop); err != nil { if err := h.Handle("mesh.mod.q.event.>", nop); err != nil {
t.Fatal(err) t.Fatal(err)
} }
// W1.12: a seat's event kept in a stream of its own is heard on one bus name and one durable, the maker's and the
// hearer's alike; an event kept last per subject is delivered from the last of each subject. The name is the
// contract's Stream<E> as it stands (seatdef.KeptStream), never made again here: a dotted or unprefixed name is
// refused by name before the bus is asked.
a := Account{Node: "ace", Module: "mesh-grants"}
for _, stream := range []string{"node-engine.running", "SEAT_node-engine.running", "SEAT_", "EVENTS"} {
if _, err := a.HearSeatEvents(context.Background(), nil, stream, func(jetstream.Msg) {}, nil); err == nil ||
!strings.Contains(err.Error(), "is not the bus's name of a seat event's stream") {
t.Errorf("hearing %q: %v", stream, err)
}
}
cfg := a.SeatEventConsumerConfig([]string{"mesh.seat.node-engine.event.running.>"}, true)
if cfg.Durable != "ace_mesh-grants" || cfg.DeliverPolicy != jetstream.DeliverLastPerSubjectPolicy ||
cfg.AckPolicy != jetstream.AckExplicitPolicy || len(cfg.FilterSubjects) != 1 {
t.Errorf("the durable on a stream kept last per subject: %+v", cfg)
}
if cfg := a.SeatEventConsumerConfig(nil, false); cfg.DeliverPolicy != jetstream.DeliverAllPolicy {
t.Errorf("the durable on a stream kept with an age starts from its first message, got %v", cfg.DeliverPolicy)
}
} }

Some files were not shown because too many files have changed in this diff Show More