Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 jschoubben
|
959b1575a8 | ||
|
jschoubben
|
209ebeee03 | ||
|
jschoubben
|
5bc8d829ff |
@@ -9,15 +9,6 @@ does not belong.* Per-module code (a service's API client, its tool implementati
|
||||
create-a-resource adapter) lives **in the module**, never here — that coupling is exactly what
|
||||
turned the old SDK into constant maintenance and made every edit rebuild every module.
|
||||
|
||||
**A seat's code is not here** (novox/hq ADR 0625, ADR 0627). mesh-sdk is the runtime every module uses (the bus
|
||||
connection, the tools bundle's runtime, state and its helpers) and a seat-neutral part two or more seats' clients or
|
||||
holders share (sending a file in parts, `go/parts`). A seat defined whole has its generated contract, and its client
|
||||
where callers share logic over it, in mesh-contracts (`go/seats/<seat>`, `go/clients/<seat>client`); the seat list is
|
||||
mesh-contracts' `go/seats`, which `catalogue` alone imports. The value layouts of seats not yet defined whole stay
|
||||
here until their domain moves, and no new seat-specific type is added. `packages_list_test.go` holds all of it:
|
||||
mesh-sdk imports no contracts module but the seat set in `catalogue`, names none of the seats with a client outside
|
||||
`catalogue`, and holds none of the packages that moved (`asks`, `asker`, `publish`).
|
||||
|
||||
## What is in it
|
||||
|
||||
Five small areas, each a thing a module's `tools/` or `provisioner/` imports:
|
||||
@@ -115,31 +106,21 @@ runtime's (`node-tools/internal/launch`, mesh-tools).
|
||||
values and then every change. Through the runtime that launched the bundle, which refuses what the
|
||||
module did not declare. Current values, never history and never a secret (novox/hq ADR 0201).
|
||||
|
||||
### `runtime`, `callers` and `views` — holding a seat from TypeScript
|
||||
The TypeScript side of what the Go side's `stdio` and `views` give a seat's holder, with the same rules (design 47
|
||||
*The forge module's tools come from mesh-sdk's TypeScript side*): `runtime.seatPublish` says an event or submits an
|
||||
accept on a seat (`mesh/seat.publish`) and `runtime.config` reads a config value; `callers.callerOf(args)` is the
|
||||
caller the runtime proved in the call's `_meta`, never an argument, and `callers.refusal` judges it against a
|
||||
holder's list per verb (`parseVerbCallers`, `configuredVerbCallers`: `*` any proved caller, `operator` any person);
|
||||
`views.Follower` keeps a first view of an owner's state from its pages and its events, asking again only on a gap,
|
||||
and `views.statusTools` serves its read and ask-again tools (`<module>_first_views`, `<module>_first_view_again`)
|
||||
over the reader's state `first-views`. `waitFor(view)` and `holds(view)` say the revision view a part waits for and
|
||||
holds; past five minutes `raiseWaits` is told and `viewWaitsFinding` says the part's `view-waits` finding. Checked by `test/stdio.test.ts` and `test/sdk.test.ts`.
|
||||
|
||||
### `seats` and `channel` — publishing on a seat, the channel's benches (Go)
|
||||
### `seats` and `asks` — asking a person, answering on a channel (Go)
|
||||
A seat may name its caller or its kind (novox/hq ADR 0259): `stdio.SeatPublish` submits an accept or says
|
||||
an event under the module's own name or kind, `stdio.SeatTake` takes a held seat's work, `stdio.SeatProve`
|
||||
and `stdio.SeatAnswer` carry a proof (a code typed by a person, by request and reply, never kept),
|
||||
`stdio.SeatRecord` reads a record kept for the module, and `State(...).Create` and `.Update` write by
|
||||
compare-and-set. The runtime does each only as far as the module's membership lists. `go/channel` holds the
|
||||
shapes of the two benches a channel holds, `channel` and `intake` (the `Message` a channel takes, the `Choice`,
|
||||
`LinkAsked`, `Failed`, `Standing` and `Code` it says, with the subjects each travels on): those benches are not
|
||||
defined whole, so their layouts stay here (ADR 0625 §9).
|
||||
Asking a person is the `operator-channel` seat's, defined whole in mesh-contracts: its `Ask` and `Warrant` are its
|
||||
generated contract (`go/seats/operatorchannel`), and the checks an ask is held to, the warrant's (`For`, `Says`,
|
||||
`Performs`) and an asking module's client (`Ask`, `Cancel`, `Where`, `Listen`, `Take`, each warrant handed once,
|
||||
marked acted on by compare-and-set) are its client, `go/clients/operatorchannelclient` (ADR 0627 §1). A lost
|
||||
compare-and-set is `stdio.ErrChanged`, told by the runtime's error code.
|
||||
compare-and-set. The runtime does each only as far as the module's membership lists. `go/asks` is the
|
||||
contract on top: the `Ask` an asker publishes and the checks it is held to (its `Whole` is what a channel that proves
|
||||
who answers shows in place of its `Explanation`, and its `Details` what the Details answer shows, hq issue 383), the `Warrant` it acts on
|
||||
(`For`, `Says`), and the `Message`, `Choice`, `LinkAsked` and `Code` a channel takes and says, with the
|
||||
subjects each travels on.
|
||||
`go/asker` is an asking module's client (ADR 0259 §10): `Ask` keeps an ask in the module's own state and then
|
||||
publishes it under the module's name; `Cancel`; `Where` reads its state from the router's record; `Listen` and
|
||||
`Take` hand the module each warrant once — marked acted on by compare-and-set before the module acts, and only
|
||||
for an ask it made, an option it offered, at that option's level, before it expired. A lost compare-and-set is
|
||||
`stdio.ErrChanged`, told by the runtime's error code.
|
||||
`State(...).Increment(key, attempts)` is a counter on top of the two (for numbers handed out once, such as an
|
||||
issue's): it reads the key, writes one more at the revision read, and on a lost write reads and tries again, at
|
||||
most `attempts` times; past that it is `stdio.ErrChanged` and nothing was counted.
|
||||
|
||||
@@ -1,296 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
"git.novox.be/novox/mesh-sdk/go/outward"
|
||||
)
|
||||
|
||||
// Call is one call the runner serves, as the gate judges it: a module's tool, or a seat verb its module serves.
|
||||
type Call struct {
|
||||
// Address is the address as called: `<module>.<tool>` or `<seat>.<verb>`.
|
||||
Address string
|
||||
// Module is the module whose bundle serves it; Seat the seat of a verb, empty for a tool; Name the tool or the verb.
|
||||
Module, Seat, Name string
|
||||
// Fingerprint is the build the bundle serves, from its composed environment (§5).
|
||||
Fingerprint string
|
||||
// Caller is the caller the bus proved.
|
||||
Caller string
|
||||
// Args are the call's arguments, a JSON object.
|
||||
Args json.RawMessage
|
||||
// Description is the tool's or verb's own description, whose first line the ask explains it by.
|
||||
Description string
|
||||
// Own says it is one of the runner's own tools served under the module's name (its ledger repairs), not its bundle's.
|
||||
Own bool
|
||||
}
|
||||
|
||||
// MostArguments is the most a marked call's arguments may hold: more is refused, asking nothing (§9).
|
||||
const MostArguments = 16 * 1024
|
||||
|
||||
// arguments are a call's arguments by name; a body that is not an object is refused.
|
||||
func arguments(raw json.RawMessage) (map[string]json.RawMessage, error) {
|
||||
args := map[string]json.RawMessage{}
|
||||
if len(bytes.TrimSpace(raw)) == 0 || string(bytes.TrimSpace(raw)) == "null" {
|
||||
return args, nil
|
||||
}
|
||||
if err := json.Unmarshal(raw, &args); err != nil {
|
||||
return nil, fmt.Errorf("its arguments are not a JSON object: %v", err)
|
||||
}
|
||||
return args, nil
|
||||
}
|
||||
|
||||
// refusedUnasked says why a marked call is refused at once, asking nothing (§9): its arguments hold a secret's shape, or are
|
||||
// more than MostArguments.
|
||||
func refusedUnasked(raw json.RawMessage) string {
|
||||
if len(raw) > MostArguments {
|
||||
return fmt.Sprintf("its arguments are %d bytes, more than the %d a call that needs the operator's approval may carry", len(raw), MostArguments)
|
||||
}
|
||||
if r, ok := outward.CheckSecret(string(raw)); !ok {
|
||||
return fmt.Sprintf("its arguments hold %s, which no ask may carry", r.What)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// ActOf is the act a marked call binds (§8): `address`, `node` (none given for a process holder, lead answer A-255), `module` and `tool` or `seat` and `verb`, `build`, and
|
||||
// `arg.<name>` for each argument given, a string as itself and any other value as its JSON in RFC 8785's canonical form.
|
||||
func ActOf(c Call, node string) (operatorask.Act, error) {
|
||||
act := operatorask.Act{"address": c.Address, "build": c.Fingerprint}
|
||||
if node != "" { // a process holder's act names none (lead answer A-255)
|
||||
act["node"] = node
|
||||
}
|
||||
if c.Seat == "" {
|
||||
act["module"], act["tool"] = c.Module, c.Name
|
||||
} else {
|
||||
act["seat"], act["verb"] = c.Seat, c.Name
|
||||
}
|
||||
args, err := arguments(c.Args)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for name, raw := range args {
|
||||
var s string
|
||||
if json.Unmarshal(raw, &s) == nil {
|
||||
act["arg."+name] = s
|
||||
continue
|
||||
}
|
||||
canon, err := Canonical(raw)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the argument %s cannot be put in canonical form: %v", name, err)
|
||||
}
|
||||
act["arg."+name] = canon
|
||||
}
|
||||
return act, nil
|
||||
}
|
||||
|
||||
// Canonical is a JSON value in RFC 8785's canonical form: no white space, an object's members sorted by their names' UTF-16
|
||||
// code units, a number as ECMAScript writes a double, a string with only the escapes the RFC names.
|
||||
func Canonical(raw json.RawMessage) (string, error) {
|
||||
dec := json.NewDecoder(bytes.NewReader(raw))
|
||||
dec.UseNumber()
|
||||
var v any
|
||||
if err := dec.Decode(&v); err != nil {
|
||||
return "", err
|
||||
}
|
||||
var b strings.Builder
|
||||
if err := canonicalValue(&b, v); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return b.String(), nil
|
||||
}
|
||||
|
||||
func canonicalValue(b *strings.Builder, v any) error {
|
||||
switch x := v.(type) {
|
||||
case nil:
|
||||
b.WriteString("null")
|
||||
case bool:
|
||||
b.WriteString(strconv.FormatBool(x))
|
||||
case json.Number:
|
||||
f, err := strconv.ParseFloat(string(x), 64)
|
||||
if err != nil || math.IsInf(f, 0) || math.IsNaN(f) {
|
||||
return fmt.Errorf("the number %s is not a double", x)
|
||||
}
|
||||
b.WriteString(es6Number(f))
|
||||
case string:
|
||||
canonicalString(b, x)
|
||||
case []any:
|
||||
b.WriteByte('[')
|
||||
for i, e := range x {
|
||||
if i > 0 {
|
||||
b.WriteByte(',')
|
||||
}
|
||||
if err := canonicalValue(b, e); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
b.WriteByte(']')
|
||||
case map[string]any:
|
||||
names := make([]string, 0, len(x))
|
||||
for k := range x {
|
||||
names = append(names, k)
|
||||
}
|
||||
sort.Slice(names, func(i, j int) bool { return utf16Less(names[i], names[j]) })
|
||||
b.WriteByte('{')
|
||||
for i, k := range names {
|
||||
if i > 0 {
|
||||
b.WriteByte(',')
|
||||
}
|
||||
canonicalString(b, k)
|
||||
b.WriteByte(':')
|
||||
if err := canonicalValue(b, x[k]); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
b.WriteByte('}')
|
||||
default:
|
||||
return fmt.Errorf("a value of type %T", v)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// es6Number is a double as ECMAScript's Number.prototype.toString writes it (RFC 8785 §3.2.2.3).
|
||||
func es6Number(f float64) string {
|
||||
if f == 0 {
|
||||
return "0"
|
||||
}
|
||||
abs := math.Abs(f)
|
||||
if abs >= 1e-6 && abs < 1e21 {
|
||||
return strconv.FormatFloat(f, 'f', -1, 64)
|
||||
}
|
||||
s := strconv.FormatFloat(f, 'e', -1, 64) // 1e+21, 1.5e-07
|
||||
mant, exp, _ := strings.Cut(s, "e")
|
||||
sign, digits := exp[:1], strings.TrimLeft(exp[1:], "0")
|
||||
return mant + "e" + sign + digits
|
||||
}
|
||||
|
||||
func canonicalString(b *strings.Builder, s string) {
|
||||
b.WriteByte('"')
|
||||
for _, r := range s {
|
||||
switch {
|
||||
case r == '"':
|
||||
b.WriteString(`\"`)
|
||||
case r == '\\':
|
||||
b.WriteString(`\\`)
|
||||
case r == '\b':
|
||||
b.WriteString(`\b`)
|
||||
case r == '\f':
|
||||
b.WriteString(`\f`)
|
||||
case r == '\n':
|
||||
b.WriteString(`\n`)
|
||||
case r == '\r':
|
||||
b.WriteString(`\r`)
|
||||
case r == '\t':
|
||||
b.WriteString(`\t`)
|
||||
case r < 0x20:
|
||||
fmt.Fprintf(b, `\u%04x`, r)
|
||||
default:
|
||||
b.WriteRune(r)
|
||||
}
|
||||
}
|
||||
b.WriteByte('"')
|
||||
}
|
||||
|
||||
func utf16Less(a, b string) bool {
|
||||
ua, ub := utf16Of(a), utf16Of(b)
|
||||
for i := 0; i < len(ua) && i < len(ub); i++ {
|
||||
if ua[i] != ub[i] {
|
||||
return ua[i] < ub[i]
|
||||
}
|
||||
}
|
||||
return len(ua) < len(ub)
|
||||
}
|
||||
|
||||
func utf16Of(s string) []uint16 {
|
||||
out := make([]uint16, 0, utf8.RuneCountInString(s))
|
||||
for _, r := range s {
|
||||
if r >= 0x10000 {
|
||||
r -= 0x10000
|
||||
out = append(out, uint16(0xD800+(r>>10)), uint16(0xDC00+(r&0x3FF)))
|
||||
} else {
|
||||
out = append(out, uint16(r))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Fingerprint is how an act is shown: its digest's first 24 hexadecimal digits in groups of four (§10).
|
||||
func Fingerprint(digest string) string {
|
||||
hex := strings.TrimPrefix(digest, "sha256:")
|
||||
if len(hex) > 24 {
|
||||
hex = hex[:24]
|
||||
}
|
||||
var groups []string
|
||||
for len(hex) > 4 {
|
||||
groups, hex = append(groups, hex[:4]), hex[4:]
|
||||
}
|
||||
return strings.Join(append(groups, hex), " ")
|
||||
}
|
||||
|
||||
// The bounds of the arguments shown in an ask (§11).
|
||||
const (
|
||||
ShownBytes, ShownLines = 2000, 40
|
||||
PartBytes, PartLines = 1200, 20
|
||||
)
|
||||
|
||||
// shown is a call's arguments as an ask shows them (§11), one per line as "name: value", and whether all were shown whole:
|
||||
// an argument whole where outward.Check passes it, else its shape, ‹path›, ‹address› or ‹withheld›; the whole when it is at
|
||||
// most ShownBytes and ShownLines, else its first PartLines lines and PartBytes bytes, marked NOT SHOWN IN FULL.
|
||||
func shown(raw json.RawMessage) (string, bool) {
|
||||
args, _ := arguments(raw)
|
||||
names := make([]string, 0, len(args))
|
||||
for n := range args {
|
||||
names = append(names, n)
|
||||
}
|
||||
sort.Strings(names)
|
||||
whole := true
|
||||
var lines []string
|
||||
for _, n := range names {
|
||||
var value string
|
||||
if json.Unmarshal(args[n], &value) != nil {
|
||||
value = string(args[n])
|
||||
}
|
||||
if r, ok := outward.Check(value); !ok {
|
||||
value, whole = shapeOf(r), false
|
||||
}
|
||||
lines = append(lines, n+": "+value)
|
||||
}
|
||||
text := strings.Join(lines, "\n")
|
||||
if all := strings.Split(text, "\n"); len(text) > ShownBytes || len(all) > ShownLines {
|
||||
if len(all) > PartLines {
|
||||
all = all[:PartLines]
|
||||
}
|
||||
text = strings.Join(all, "\n")
|
||||
if len(text) > PartBytes {
|
||||
text = cutBytes(text, PartBytes)
|
||||
}
|
||||
text += "\nNOT SHOWN IN FULL"
|
||||
whole = false
|
||||
}
|
||||
return text, whole
|
||||
}
|
||||
|
||||
// shapeOf is what an argument outward.Check refuses is shown as.
|
||||
func shapeOf(r outward.Refusal) string {
|
||||
switch r.Class {
|
||||
case "path":
|
||||
return "‹path›"
|
||||
case "address":
|
||||
return "‹address›"
|
||||
}
|
||||
return "‹withheld›"
|
||||
}
|
||||
|
||||
// cutBytes is s at most n bytes, cut at a character's end.
|
||||
func cutBytes(s string, n int) string {
|
||||
for n > 0 && !utf8.RuneStart(s[n]) {
|
||||
n--
|
||||
}
|
||||
return s[:n]
|
||||
}
|
||||
@@ -1,80 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
)
|
||||
|
||||
// call is a fixture call of the marked tool wipe.
|
||||
func call(name, args string) Call {
|
||||
return Call{Address: "shell." + name, Module: "shell", Name: name, Fingerprint: "src2:1", Caller: "agent", Args: json.RawMessage(args),
|
||||
Description: "Wipes a scratch directory.\nMore words."}
|
||||
}
|
||||
|
||||
// Design 46 7-approval-mark §8, §10, §11: the act's fields give each call its digest, and the ask shows a call's arguments
|
||||
// in their shape and its headline cut (the gate's other cases run on the tool runner's fixture mesh, node-tools
|
||||
// internal/approval).
|
||||
func TestTheActAndItsAsk(t *testing.T) {
|
||||
base := call("wipe", `{"dir":"scratch","n":1}`)
|
||||
digest := func(c Call) string {
|
||||
act, err := ActOf(c, "n1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
d, _ := operatorask.ActDigest(act)
|
||||
return d
|
||||
}
|
||||
act, _ := ActOf(base, "n1")
|
||||
want := operatorask.Act{"address": "shell.wipe", "node": "n1", "module": "shell", "tool": "wipe", "build": "src2:1",
|
||||
"arg.dir": "scratch", "arg.n": "1"}
|
||||
if len(act) != len(want) {
|
||||
t.Fatalf("act %v", act)
|
||||
}
|
||||
for k, v := range want {
|
||||
if act[k] != v {
|
||||
t.Errorf("act[%s] = %q, want %q", k, act[k], v)
|
||||
}
|
||||
}
|
||||
seen := map[string]string{digest(base): "the call"}
|
||||
for name, c := range map[string]Call{
|
||||
"an omitted argument": call("wipe", `{"dir":"scratch"}`),
|
||||
"another number": call("wipe", `{"dir":"scratch","n":2}`),
|
||||
"an object": call("wipe", `{"dir":{"name":"scratch"},"n":1}`),
|
||||
"a changed build": func() Call { c := base; c.Fingerprint = "src2:2"; return c }(),
|
||||
"another node's seat": {Address: "node-login-shell.execute", Module: "shell", Seat: "node-login-shell", Name: "execute", Fingerprint: "src2:1"},
|
||||
} {
|
||||
d := digest(c)
|
||||
if other, dup := seen[d]; dup {
|
||||
t.Errorf("%s gives the digest of %s", name, other)
|
||||
}
|
||||
seen[d] = name
|
||||
}
|
||||
if got, _ := Canonical(json.RawMessage(`{"b":[1.0,2.5e-7,1e21],"a":"é<"}`)); got != `{"a":"é<","b":[1,2.5e-7,1e+21]}` {
|
||||
t.Errorf("canonical form %s", got)
|
||||
}
|
||||
// §11: a path, an address and a long argument are shown in their shape and cut.
|
||||
text, whole := shown(json.RawMessage(`{"dir":"/home/someone/scratch","host":"10.0.0.7","ok":"yes"}`))
|
||||
if whole || text != "dir: ‹path›\nhost: ‹address›\nok: yes" {
|
||||
t.Errorf("shown %q whole %v", text, whole)
|
||||
}
|
||||
long, _ := json.Marshal(map[string]string{"text": strings.Repeat("word ", 500)})
|
||||
if text, whole := shown(long); whole || !strings.HasSuffix(text, "NOT SHOWN IN FULL") || len(text) > PartBytes+len("\nNOT SHOWN IN FULL") {
|
||||
t.Errorf("a long argument shown whole %v, %d bytes", whole, len(text))
|
||||
}
|
||||
// §10: the headline, cut with … to 60 characters, ending Part shown.
|
||||
for _, tc := range []struct{ name, node, want string }{
|
||||
{"wipe", "n1", "Run wipe on n1?"},
|
||||
{strings.Repeat("x", 50), "n1", "Run " + strings.Repeat("x", 50) + "?"},
|
||||
{strings.Repeat("y", 80), "n1", "Run " + strings.Repeat("y", 54) + "…?"},
|
||||
} {
|
||||
if got := headlineOf("Run", tc.name, tc.node, true); got != tc.want || len([]rune(got)) > 60 {
|
||||
t.Errorf("headline %q, want %q", got, tc.want)
|
||||
}
|
||||
}
|
||||
if got := headlineOf("Run", "wipe", "n1", false); got != "Run wipe on n1? Part shown" {
|
||||
t.Errorf("headline %q", got)
|
||||
}
|
||||
}
|
||||
@@ -1,239 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/keys"
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
)
|
||||
|
||||
// StateName is the runner's own state its asks and claims are kept in, per machine (§13).
|
||||
const StateName = "approvals"
|
||||
|
||||
// Store is the runner's state `approvals` (§13): a key's value, a create that refuses a key that exists and an update by
|
||||
// compare-and-set, each failing with stdio.ErrChanged when another write came first; a put, and its keys.
|
||||
type Store interface {
|
||||
Get(key string) (*stdio.StateEntry, error)
|
||||
Create(key string, value any) (uint64, error)
|
||||
Update(key string, value any, revision uint64) (uint64, error)
|
||||
Put(key string, value any) (uint64, error)
|
||||
Keys() ([]string, error)
|
||||
}
|
||||
|
||||
// The bounds of an ask (§10, §12, §13, §14, §17).
|
||||
const (
|
||||
AskLasts = 60 * time.Minute
|
||||
RecordWithin = 5 * time.Second
|
||||
RunWithin = 30 * time.Minute
|
||||
DeclinedFor = 10 * time.Minute
|
||||
MostKeptValue = 256 * 1024
|
||||
OnExpiry = "nothing runs; call again to ask anew"
|
||||
)
|
||||
|
||||
// The options every ask offers (§10), both at level approve, both binding the act's digest.
|
||||
const (
|
||||
OptionApprove = "approve"
|
||||
OptionDecline = "decline"
|
||||
)
|
||||
|
||||
// actKey is the key naming the newest ask of an act (§13), its digest escaped as a key's name is, since the bus holds no `:` in a key (lead answer A-209).
|
||||
func (g *Gate) actKey(digest string) string { return g.prefix() + "act." + keys.EscapeName(digest) }
|
||||
|
||||
// prefix is what the gate's keys begin with: the tool runner's machine, `<node>.`, since its state is shared by every
|
||||
// machine's runner (§13); none for a module's process, whose state approvals is its module's own (design 47 G §677).
|
||||
func (g *Gate) prefix() string {
|
||||
if g.d.Holding {
|
||||
return ""
|
||||
}
|
||||
return g.d.Node + "."
|
||||
}
|
||||
|
||||
// askKey is the key the client keeps an ask under (§13).
|
||||
func (g *Gate) askKey(id string) string { return g.prefix() + "ask." + id }
|
||||
|
||||
// newID is an ask's id: 128 random bits.
|
||||
func newID() string {
|
||||
var b [16]byte
|
||||
_, _ = rand.Read(b[:])
|
||||
return hex.EncodeToString(b[:])
|
||||
}
|
||||
|
||||
// askFor is the ask a marked call raises (§10, §11).
|
||||
func (g *Gate) askFor(c Call, digest, id string) Ask {
|
||||
args, whole := shown(c.Args)
|
||||
name := c.Name
|
||||
if c.Seat != "" {
|
||||
name = c.Seat + "." + c.Name
|
||||
}
|
||||
headline := headlineOf("Run", name, g.d.Node, whole)
|
||||
first, _, _ := strings.Cut(strings.TrimSpace(c.Description), "\n")
|
||||
if first == "" {
|
||||
first = name
|
||||
}
|
||||
how := "called through the mesh's tools, which cannot tell who called."
|
||||
if g.d.Holding {
|
||||
how = "called by " + c.Caller + "."
|
||||
}
|
||||
lines := []string{first, "On " + g.d.Node + ", " + how}
|
||||
if args != "" {
|
||||
lines = append(lines, args)
|
||||
}
|
||||
if !whole {
|
||||
read := "mesh_approvals on " + g.d.Node
|
||||
if g.d.Holding {
|
||||
read = c.Seat + ".approvals"
|
||||
}
|
||||
lines = append(lines, "The whole is read with "+read+", whose fingerprint must be the one on the phone.")
|
||||
}
|
||||
lines = append(lines, "Fingerprint "+Fingerprint(digest))
|
||||
return Ask{ID: id, About: digest, Headline: headline,
|
||||
Explanation: strings.Join(lines, "\n"), Expires: formatTime(g.d.Now().Add(AskLasts)), OnExpiry: OnExpiry,
|
||||
Options: []Option{
|
||||
{ID: OptionApprove, Label: "Approve", Does: "the same call, made again within 30 minutes, runs once", Binds: digest},
|
||||
{ID: OptionDecline, Label: "Decline", Does: "nothing runs; the same call is not asked again for 10 minutes", Binds: digest}}}
|
||||
}
|
||||
|
||||
// headlineOf is "<verb> <names> on <node>?", "<verb> <names>?" when that is longer than the router allows, and the names cut
|
||||
// with "…" when that too is longer; ending "Part shown" whenever an argument is not shown whole (§10, Q-HL). verb is "Run"
|
||||
// for a call and "Wait on" for a wait (design 34 every-event §35).
|
||||
func headlineOf(verb, name, node string, whole bool) string {
|
||||
suffix := ""
|
||||
if !whole {
|
||||
suffix = " Part shown"
|
||||
}
|
||||
max := HeadlineLength
|
||||
fits := func(s string) bool { return len([]rune(s)) <= max }
|
||||
if h := verb + " " + name + " on " + node + "?" + suffix; fits(h) {
|
||||
return h
|
||||
}
|
||||
if h := verb + " " + name + "?" + suffix; fits(h) {
|
||||
return h
|
||||
}
|
||||
room := max - len([]rune(verb+" …?"+suffix))
|
||||
r := []rune(name)
|
||||
if room < 1 {
|
||||
room = 1
|
||||
}
|
||||
if len(r) > room {
|
||||
r = r[:room]
|
||||
}
|
||||
return verb + " " + string(r) + "…?" + suffix
|
||||
}
|
||||
|
||||
// firstCall is a marked call whose act has no ask open: refused at once when its arguments may not be asked (§9), else
|
||||
// asked (§10, §12).
|
||||
func (g *Gate) firstCall(c Call) (any, error) {
|
||||
if why := refusedUnasked(c.Args); why != "" {
|
||||
return nil, g.refused(c, "", why)
|
||||
}
|
||||
act, err := ActOf(c, g.d.Node)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
digest, err := operatorask.ActDigest(act)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
return nil, g.raise(c, act, digest, "")
|
||||
}
|
||||
|
||||
// raise raises the ask of a marked call whose act has none open (§10): kept in the runner's state with the act and the whole
|
||||
// arguments, published under the module's name, its act's key naming it; then the 5 seconds for the router's record and
|
||||
// the first answer (§12), always an error, never a result.
|
||||
func (g *Gate) raise(c Call, act operatorask.Act, digest, markedSince string) error {
|
||||
id := newID()
|
||||
if err := g.keepAsk(c, act, digest, g.askFor(c, digest, id)); err != nil {
|
||||
return err
|
||||
}
|
||||
return g.firstAnswer(c, id, digest, markedSince)
|
||||
}
|
||||
|
||||
// keepAsk raises an ask and keeps it in the runner's state with the act and the whole arguments, its act's key naming it
|
||||
// (§10, §13); refused, as an error, when it cannot be.
|
||||
func (g *Gate) keepAsk(c Call, act operatorask.Act, digest string, a Ask) error {
|
||||
if size := keptSize(a, act, c.Args); size > MostKeptValue {
|
||||
return g.refused(c, digest, fmt.Sprintf("its ask would keep %d bytes, more than the %d a value of the state %s holds", size, MostKeptValue, StateName))
|
||||
}
|
||||
if err := g.d.Channel.Ask(c.Module, a); err != nil {
|
||||
return g.refused(c, digest, "the ask could not be raised: "+err.Error())
|
||||
}
|
||||
if err := g.keepCall(a.ID, act, c.Args); err != nil {
|
||||
g.stateFailed(err)
|
||||
return g.refused(c, digest, "the ask was raised and its call could not be kept: "+err.Error())
|
||||
}
|
||||
if _, err := g.d.State.Put(g.actKey(digest), a.ID); err != nil {
|
||||
g.stateFailed(err)
|
||||
return g.refused(c, digest, "the ask was raised and its act could not be kept: "+err.Error())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// keptSize is the bytes the kept ask would take.
|
||||
func keptSize(a Ask, act operatorask.Act, args json.RawMessage) int {
|
||||
raw, _ := json.Marshal(Kept[Ask, Warrant]{Ask: a, Act: act, Arguments: args})
|
||||
return len(raw)
|
||||
}
|
||||
|
||||
// keepCall writes the act and the whole arguments beside the ask the client kept (§13).
|
||||
func (g *Gate) keepCall(id string, act operatorask.Act, args json.RawMessage) error {
|
||||
e, err := g.d.State.Get(g.askKey(id))
|
||||
if err != nil || e == nil {
|
||||
return fmt.Errorf("the kept ask %s cannot be read: %v", id, err)
|
||||
}
|
||||
var k kept
|
||||
if err := json.Unmarshal(e.Value, &k); err != nil {
|
||||
return err
|
||||
}
|
||||
k.Act, k.Arguments = act, args
|
||||
_, err = g.d.State.Update(g.askKey(id), k, e.Revision)
|
||||
return err
|
||||
}
|
||||
|
||||
// firstAnswer waits up to RecordWithin for the router's record of the ask and says the call was not run (§12).
|
||||
func (g *Gate) firstAnswer(c Call, id, digest, markedSince string) error {
|
||||
rec, _ := g.awaitRecord(c.Module, id)
|
||||
since := ""
|
||||
if markedSince != "" {
|
||||
since = "; marked since revision " + markedSince
|
||||
}
|
||||
if rec == nil {
|
||||
return fmt.Errorf("Not run, asked; not yet taken by the router (ask %s, fingerprint %s); make the same call again later%s",
|
||||
id, Fingerprint(digest), since)
|
||||
}
|
||||
return fmt.Errorf("Not run: this needs the operator's approval. Asked on %s at %s (ask %s, fingerprint %s); make the same call "+
|
||||
"again within 30 minutes of the approval; the ask expires at %s%s", orNone(strings.Join(rec.OfferedOn, ", ")), rec.Opened,
|
||||
id, Fingerprint(digest), rec.Expires, since)
|
||||
}
|
||||
|
||||
// record is the router's record of an ask under the module's name, nil when it holds none.
|
||||
func (g *Gate) record(module, id string) (*Record, error) {
|
||||
return g.d.Channel.Record(module, id)
|
||||
}
|
||||
|
||||
// awaitRecord reads the router's record of a new ask until it is there, within RecordWithin: the seat answers a key's
|
||||
// value and says nothing when the router takes an ask, so it is read again after a growing pause (§12).
|
||||
func (g *Gate) awaitRecord(module, id string) (*Record, error) {
|
||||
until := g.d.Now().Add(RecordWithin)
|
||||
pause := 50 * time.Millisecond
|
||||
for {
|
||||
rec, err := g.record(module, id)
|
||||
if rec != nil || g.d.Now().Add(pause).After(until) {
|
||||
return rec, err
|
||||
}
|
||||
<-g.d.After(pause)
|
||||
pause *= 2
|
||||
}
|
||||
}
|
||||
|
||||
// refused is a marked call refused, run nothing, written in the runner's log with the ask's fingerprint and never its
|
||||
// arguments (§19).
|
||||
func (g *Gate) refused(c Call, digest, why string) error {
|
||||
g.d.Logf("[approval] %s on %s is refused (fingerprint %s): %s", c.Address, g.d.Node, orNone(Fingerprint(digest)), why)
|
||||
return fmt.Errorf("Not run: %s needs the operator's approval and %s", c.Address, why)
|
||||
}
|
||||
@@ -1,115 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
)
|
||||
|
||||
// What the gate reads beyond itself crosses in the gate's own terms, its caller translating from the generated contracts
|
||||
// (design 47 G lead answers A-219, A-224, A-228): the operator channel (Channel), the catalog's entries (Entry, through
|
||||
// Catalog), the finding writer (FindingWriter) and the seat holders (Deps.Holder). No form of a generated contract is
|
||||
// copied here: what the router keeps whole (an ask, a warrant) the gate holds as the bytes it was given.
|
||||
|
||||
// Channel is the operator channel as the gate reaches it (A-219, A-228): operatorchannelclient implements it over the
|
||||
// generated types, keeping each ask in the gate's own state under the marked module's name.
|
||||
type Channel interface {
|
||||
// Ask checks the ask, keeps it in the gate's state as a Kept and publishes it under module's name, at the level
|
||||
// approve, to the operator (design 46 7-approval-mark §10).
|
||||
Ask(module string, a Ask) error
|
||||
// Record is the router's record of module's ask id, read under module's own name; nil while the router holds none
|
||||
// (design 47 G §700).
|
||||
Record(module, id string) (*Record, error)
|
||||
// Claim takes the warrant once (design 46 7-approval-mark §14, §15): it holds it to the ask kept, checks that the
|
||||
// chosen option binds act and marks the kept ask acted, then calls run; act's run happens only when it answers
|
||||
// Claimed. A warrant already taken answers ClaimDone, one that does not hold ClaimRefused with why.
|
||||
Claim(module string, w Warrant, act operatorask.Act, run func()) (Claim, error)
|
||||
// Cancel takes module's ask back.
|
||||
Cancel(module, id string) error
|
||||
// Decided is the subject the router's `decided` for module is heard on.
|
||||
Decided(module string) string
|
||||
// Seat is the seat an ask is raised on, whose holder must be there for a marked call to be asked (§19).
|
||||
Seat() string
|
||||
}
|
||||
|
||||
// Claim is what came of a warrant handed to Channel.Claim.
|
||||
type Claim string
|
||||
|
||||
const (
|
||||
Claimed Claim = "acted"
|
||||
ClaimRefused Claim = "refused"
|
||||
ClaimDone Claim = "already acted on"
|
||||
// ClaimEnded is a warrant of an ask that ended without a choice.
|
||||
ClaimEnded Claim = "ended without a choice"
|
||||
)
|
||||
|
||||
// Ask is an ask the gate raises: two options, each binding the act's digest (design 46 7-approval-mark §10, §11).
|
||||
type Ask struct {
|
||||
ID, About, Headline, Explanation, Expires, OnExpiry string
|
||||
Options []Option
|
||||
}
|
||||
|
||||
// Option is one option of an ask.
|
||||
type Option struct{ ID, Label, Does, Binds string }
|
||||
|
||||
// Record is the router's record of an ask as the gate reads it.
|
||||
type Record struct {
|
||||
State, Answer, Opened, Expires string
|
||||
OfferedOn []string
|
||||
Warrant *Warrant
|
||||
}
|
||||
|
||||
// Warrant is a person's choice as the gate reads it; Whole is the warrant as the router recorded it, which Claim holds to
|
||||
// the kept ask.
|
||||
type Warrant struct {
|
||||
Ask, Asker, Option, Channel, At string
|
||||
Whole json.RawMessage
|
||||
}
|
||||
|
||||
// Kept is one ask as its asker keeps it (design 46 7-approval-mark §13): the ask, beside it the act the ask binds, the
|
||||
// call's whole arguments, and whether the act's run was recorded in the hand-act log. A is the ask's form and W the
|
||||
// warrant's: operatorchannelclient's Kept is Kept of the generated contract's, the gate's of the bytes kept (A-228).
|
||||
type Kept[A, W any] struct {
|
||||
Ask A `json:"ask"`
|
||||
Asked time.Time `json:"asked"`
|
||||
Acted bool `json:"acted,omitempty"`
|
||||
Ended string `json:"ended,omitempty"`
|
||||
Answer *W `json:"answer,omitempty"`
|
||||
// Act is the act the ask's options bind, as its asker digested it.
|
||||
Act operatorask.Act `json:"act,omitempty"`
|
||||
// Arguments are the asked call's arguments whole.
|
||||
Arguments json.RawMessage `json:"arguments,omitempty"`
|
||||
// Recorded says the act's run is in the hand-act log (`mesh-hand-acts.warranted` answered).
|
||||
Recorded bool `json:"recorded,omitempty"`
|
||||
}
|
||||
|
||||
// kept is the gate's own reading of a Kept: the ask and the warrant as the bytes kept, so a write back keeps them whole.
|
||||
type kept = Kept[json.RawMessage, json.RawMessage]
|
||||
|
||||
// HeadlineLength is the longest headline the router takes, in characters; operatorchannelclient re-exports it.
|
||||
const HeadlineLength = 60
|
||||
|
||||
// Entry is a module's entry in the catalog as the marks are read from it (design 46 7-approval-mark §5): its module, the
|
||||
// build's fingerprint, the seat verbs it serves as `<seat>.<verb>`, and its manifest.
|
||||
type Entry struct {
|
||||
Module, Fingerprint string
|
||||
Serves []string
|
||||
Manifest json.RawMessage
|
||||
}
|
||||
|
||||
// formatTime is a time as the router's records carry it: RFC 3339 to the nanosecond; the zero time is empty.
|
||||
func formatTime(t time.Time) string {
|
||||
if t.IsZero() {
|
||||
return ""
|
||||
}
|
||||
return t.Format(time.RFC3339Nano)
|
||||
}
|
||||
|
||||
// parseTime reads such a time; empty is the zero time.
|
||||
func parseTime(s string) (time.Time, error) {
|
||||
if s == "" {
|
||||
return time.Time{}, nil
|
||||
}
|
||||
return time.Parse(time.RFC3339Nano, s)
|
||||
}
|
||||
@@ -1,100 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// UnreadableAfter is how long the state approvals fails before the runner's finding says so (§19).
|
||||
const UnreadableAfter = 5 * time.Minute
|
||||
|
||||
// unreadableKey is the bound key whose expiry raises `approvals-unreadable` (lead answer A-157).
|
||||
func (g *Gate) unreadableKey() string { return g.d.Node + ".unreadable." + StateName }
|
||||
|
||||
// held says why no marked call can be asked now, or "" (§19): no module holds operator-channel, as the assigner's view
|
||||
// of the seat's holders says, or that view is not read.
|
||||
func (g *Gate) held() string {
|
||||
if g.d.Channel == nil {
|
||||
return "the tool runner reaches no operator channel to ask on"
|
||||
}
|
||||
if g.d.Holder == nil && g.d.Holding {
|
||||
return "" // a process holder reads no view of the seats' holders: an ask nobody takes is said by its gate-unanswered (design 47 G §677)
|
||||
}
|
||||
if g.d.Holder == nil {
|
||||
return "the tool runner reads no view of the seats' holders, so it cannot tell that a module holds " + g.d.Channel.Seat()
|
||||
}
|
||||
on := g.d.Channel.Seat()
|
||||
ok, err := g.d.Holder(on)
|
||||
switch {
|
||||
case err != nil:
|
||||
return fmt.Sprintf("whether a module holds %s cannot be read: %v", on, err)
|
||||
case !ok:
|
||||
return "no module holds " + on + ", so nobody can be asked"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// stateFailed notes that the state approvals could not be read or written: no marked call runs on this machine until it
|
||||
// can, the failed read or write is made again at the next marked call and at the next membership change (A-157), and
|
||||
// five minutes on the runner's finding says it (§19).
|
||||
func (g *Gate) stateFailed(err error) {
|
||||
g.mu.Lock()
|
||||
first := g.failingSince.IsZero()
|
||||
if first {
|
||||
g.failingSince = g.d.Now()
|
||||
}
|
||||
g.mu.Unlock()
|
||||
if first {
|
||||
g.d.Logf("[approval] the state %s on %s cannot be read or written (%v): no marked call runs here until it can", StateName, g.d.Node, err)
|
||||
if g.d.Bounds != nil && !g.d.Holding {
|
||||
if e := g.d.Bounds.CreateTTL(g.unreadableKey(), map[string]string{"since": g.d.Now().UTC().Format(time.RFC3339)}, UnreadableAfter); e != nil {
|
||||
g.d.Logf("[approval] the bound of the state %s's failure is not kept: %v", StateName, e)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// stateRead notes that the state approvals was read or written: a failure noted before ends, its finding cleared.
|
||||
func (g *Gate) stateRead() {
|
||||
g.mu.Lock()
|
||||
was := !g.failingSince.IsZero()
|
||||
g.failingSince = time.Time{}
|
||||
g.mu.Unlock()
|
||||
if !was {
|
||||
return
|
||||
}
|
||||
if g.d.Bounds != nil && !g.d.Holding {
|
||||
_ = g.d.Bounds.Delete(g.unreadableKey())
|
||||
}
|
||||
g.finding(StateName, KindApprovalsUnreadable, false, "the state "+StateName+" is read again", "")
|
||||
}
|
||||
|
||||
// failing says the state approvals is failing now.
|
||||
func (g *Gate) failing() bool {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
return !g.failingSince.IsZero()
|
||||
}
|
||||
|
||||
// StateAgain makes the failed read of the state approvals again: at the next membership change the runner hears (A-157).
|
||||
func (g *Gate) StateAgain() {
|
||||
if g.d.State == nil || !g.failing() {
|
||||
return
|
||||
}
|
||||
if _, err := g.d.State.Keys(); err != nil {
|
||||
return
|
||||
}
|
||||
g.stateRead()
|
||||
}
|
||||
|
||||
// unreadableExpired is the bound key of a failing state expiring: the finding raised while the state still fails.
|
||||
func (g *Gate) unreadableExpired() {
|
||||
g.mu.Lock()
|
||||
since := g.failingSince
|
||||
g.mu.Unlock()
|
||||
if since.IsZero() {
|
||||
return
|
||||
}
|
||||
g.finding(StateName, KindApprovalsUnreadable, true, fmt.Sprintf("the state %s on %s cannot be read or written since %s: no marked call "+
|
||||
"runs here until it can", StateName, g.d.Node, since.UTC().Format(time.RFC3339)), "approvals unreadable on "+g.d.Node)
|
||||
}
|
||||
@@ -1,178 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The runner's findings of the gate (design 46 7-approval-mark §4, §16, §19), written through its held writer.
|
||||
const (
|
||||
KindMarkUnserved = "mark-unserved"
|
||||
KindCallUnanswered = "call-unanswered"
|
||||
KindApprovalsUnreadable = "approvals-unreadable"
|
||||
)
|
||||
|
||||
// Kinds are the finding kinds the gate writes through the runner's writer.
|
||||
var Kinds = []string{KindMarkUnserved, KindCallUnanswered, KindApprovalsUnreadable}
|
||||
|
||||
// FindingWriter raises and clears the gate's findings by kind and id (A-228): its caller writes each as the contract's
|
||||
// finding, about its own module on its machine, through its held writer.
|
||||
type FindingWriter interface {
|
||||
Raise(ctx context.Context, kind, id, summary, headline string) error
|
||||
Clear(ctx context.Context, kind, id, why string) error
|
||||
}
|
||||
|
||||
// Deps is what the gate reaches the mesh through: the runner's connection in its one place, fakes in a test.
|
||||
type Deps struct {
|
||||
// Node is the machine the runner serves; Runner its own module's name, the subject of its findings.
|
||||
Node, Runner string
|
||||
// Catalog is the catalog half the marks are read from.
|
||||
Catalog Catalog
|
||||
// Wait waits, within the bound, for the catalog half's first view, and is nil once it is read or why not.
|
||||
Wait func(within time.Duration) error
|
||||
// Findings is the runner's held writer; nil says a finding in the log only.
|
||||
Findings FindingWriter
|
||||
Logf func(string, ...any)
|
||||
Now func() time.Time
|
||||
// After is the clock's pause; nil is time.After.
|
||||
After func(time.Duration) <-chan time.Time
|
||||
// State is the runner's state approvals; nil when its membership binds none, so every marked call is refused (§13).
|
||||
State Store
|
||||
// Channel is the operator channel, its asks kept in State under this machine's keys (A-219, A-228).
|
||||
Channel Channel
|
||||
// Warranted asks `mesh-hand-acts.warranted` {asker, ask}, each try a kept call open while open says so (§16, A-157).
|
||||
Warranted func(asker, ask string, open func() (bool, error)) error
|
||||
// Bounds is the runner's key-ttl state bound (A-157, A-158), or a process holder's state approvals itself (A-255); nil
|
||||
// keeps no bound.
|
||||
Bounds Bounds
|
||||
// Holder says whether a module holds a seat, from the assigner's view of the seats' holders the runner follows (§19).
|
||||
Holder func(seat string) (bool, error)
|
||||
// Build is the tool runner's own fingerprint, read from its MESH_FINGERPRINT at its start: the build a marked wait's
|
||||
// act binds; empty while unknown, when every marked wait is refused (design 34 every-event §35).
|
||||
Build string
|
||||
// Hear waits within the bound for the first event on the subjects whose body match takes, or the caller's going: a
|
||||
// marked wait's ask's `decided` (every-event §36); false when none came.
|
||||
Hear func(ctx context.Context, subjects []string, within time.Duration, match func(json.RawMessage) bool) (bool, error)
|
||||
// Go runs what goes on beside the call (a run's recording); nil is a goroutine.
|
||||
Go func(func())
|
||||
// Holding says the gate is a module's process's seat server's (design 47 G §677): its keys are its module's own, its
|
||||
// calls held as waiting calls and their ends said through Say; Runner is then the holder's module.
|
||||
Holding bool
|
||||
// Say publishes one gated-ended on its subject with the message id given (§677).
|
||||
Say func(subject string, body any, id string) error
|
||||
}
|
||||
|
||||
// Gate is the approval gate: every call of a module the tool runner serves goes through it before the bundle (§20), and a
|
||||
// marked verb a module's process serves before its handler (design 47 G §677; lead answer A-224).
|
||||
type Gate struct {
|
||||
d Deps
|
||||
mu sync.Mutex
|
||||
// registered is each module's tools as its bundle registered them at its last load.
|
||||
registered map[string]map[string]bool
|
||||
// unserved is each module whose marks name a tool its bundle does not register, with those tools.
|
||||
unserved map[string][]string
|
||||
// unmarkedAt is each caller's address it last called unmarked, so a mark added since is named (§12).
|
||||
unmarkedAt map[string]bool
|
||||
// inFlight is each ask whose recording runs now.
|
||||
inFlight map[string]bool
|
||||
// failingSince is when the state approvals began to fail; zero while it is read.
|
||||
failingSince time.Time
|
||||
// handlers are a process holder's marked verbs' handlers, by `<seat>.<verb>` (design 47 G §677).
|
||||
handlers map[string]func(string, json.RawMessage) (any, error)
|
||||
// served is each module this runner serves, with the build its bundle serves, as its last load said (§18).
|
||||
served map[string]string
|
||||
}
|
||||
|
||||
// New is a gate over d.
|
||||
func New(d Deps) *Gate {
|
||||
if d.Logf == nil {
|
||||
d.Logf = func(string, ...any) {}
|
||||
}
|
||||
if d.Now == nil {
|
||||
d.Now = time.Now
|
||||
}
|
||||
if d.After == nil {
|
||||
d.After = time.After
|
||||
}
|
||||
if d.Go == nil {
|
||||
d.Go = func(f func()) { go f() }
|
||||
}
|
||||
return &Gate{d: d, registered: map[string]map[string]bool{}, unserved: map[string][]string{}}
|
||||
}
|
||||
|
||||
// Unserved is, of the module's marked tools, those its bundle does not register (§4): such a bundle is not loaded, its
|
||||
// finding `mark-unserved` raised; a bundle that serves them, or marks that no longer name them, clear it. Before the marks
|
||||
// are read nothing is known, and every marked call is refused until they are (§19).
|
||||
func (g *Gate) Unserved(module, fingerprint string, registered []string) []string {
|
||||
set := map[string]bool{}
|
||||
for _, t := range registered {
|
||||
set[t] = true
|
||||
}
|
||||
g.mu.Lock()
|
||||
g.registered[module] = set
|
||||
if g.served == nil {
|
||||
g.served = map[string]string{}
|
||||
}
|
||||
g.served[module] = fingerprint
|
||||
g.mu.Unlock()
|
||||
if g.d.Catalog == nil || g.d.Catalog.Ready() != nil {
|
||||
return nil
|
||||
}
|
||||
return g.checkServed(Derive(g.d.Catalog, module, fingerprint))
|
||||
}
|
||||
|
||||
// checkServed raises or clears the module's `mark-unserved` as its marks and its bundle's registrations stand now, and is
|
||||
// the tools it marks and does not register.
|
||||
func (g *Gate) checkServed(m Marks) []string {
|
||||
g.mu.Lock()
|
||||
set := g.registered[m.Module]
|
||||
var missing []string
|
||||
for t := range m.Tools {
|
||||
if !set[t] {
|
||||
missing = append(missing, t)
|
||||
}
|
||||
}
|
||||
sort.Strings(missing)
|
||||
was := g.unserved[m.Module]
|
||||
if len(missing) > 0 {
|
||||
g.unserved[m.Module] = missing
|
||||
} else {
|
||||
delete(g.unserved, m.Module)
|
||||
}
|
||||
g.mu.Unlock()
|
||||
switch {
|
||||
case len(missing) > 0 && strings.Join(was, ",") != strings.Join(missing, ","):
|
||||
g.finding(m.Module, KindMarkUnserved, true, fmt.Sprintf("%s's marks (%s, catalog revision %s) name %s, which its bundle on %s does not "+
|
||||
"register: no tool of %s is served here", m.Module, sourceOf(m, missing[0]), m.Revision, strings.Join(missing, ", "), g.d.Node, m.Module),
|
||||
fmt.Sprintf("%s is not served on %s", m.Module, g.d.Node))
|
||||
case len(missing) == 0 && len(was) > 0:
|
||||
g.finding(m.Module, KindMarkUnserved, false, "a bundle that serves every marked tool, or marks without the tool, is loaded", "")
|
||||
}
|
||||
return missing
|
||||
}
|
||||
|
||||
func sourceOf(m Marks, tool string) string { return m.Tools[tool] }
|
||||
|
||||
// finding writes or clears one of the gate's findings, its id `<node>.<what>` escaped as a key's name is,
|
||||
// about the runner's module on this machine (lead answer A-154).
|
||||
func (g *Gate) finding(what, kind string, raised bool, summary, headline string) {
|
||||
id := g.findingID(what)
|
||||
if g.d.Findings == nil {
|
||||
g.d.Logf("[approval] %s %s: %s", kind, id, summary)
|
||||
return
|
||||
}
|
||||
var err error
|
||||
if raised {
|
||||
err = g.d.Findings.Raise(context.Background(), kind, id, summary, headline)
|
||||
} else {
|
||||
err = g.d.Findings.Clear(context.Background(), kind, id, summary)
|
||||
}
|
||||
if err != nil {
|
||||
g.d.Logf("[approval] the finding %s of %s is not kept: %v", kind, id, err)
|
||||
}
|
||||
}
|
||||
@@ -1,699 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/keys"
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
"git.novox.be/novox/mesh-sdk/go/seatdef"
|
||||
)
|
||||
|
||||
// A marked verb a module's process serves is one call with a gate before its running (design 47 G §677): the holder keeps
|
||||
// the call, asks under its own module's name, and on the Approve runs that same call, saying its end on the seat's event
|
||||
// `gated-ended`. These are the holder's half of the gate; the caller's is process.Conn's call.
|
||||
|
||||
// WaitHeader is the bus header a call carries to say it waits for its end on `gated-ended` (lead answer A-244): a header,
|
||||
// never an argument, so no verb's input and no act's digest changes. WaitHeld is its one value.
|
||||
const (
|
||||
WaitHeader = "x-wait"
|
||||
WaitHeld = "held"
|
||||
)
|
||||
|
||||
// Held is the holder's answer to a call that waits (§677): the ask it waits on and its own call's id, in the reply's field
|
||||
// `held` beside result, error and node (A-244).
|
||||
type Held struct {
|
||||
Asked string `json:"asked"`
|
||||
Call string `json:"call"`
|
||||
}
|
||||
|
||||
// GatedEndedEvent is the seat's event a call's end is said on, kept last per subject for GatedEndedKeptFor (§677).
|
||||
const (
|
||||
GatedEndedEvent = seatdef.GatedEndedEvent
|
||||
GatedEndedKeptFor = seatdef.GatedEndedKeptDays * 24 * time.Hour
|
||||
)
|
||||
|
||||
// The outcomes of a held call (§677).
|
||||
const (
|
||||
OutcomeRan = "ran"
|
||||
OutcomeDeclined = "declined"
|
||||
OutcomeExpired = "expired"
|
||||
OutcomeRefused = "refused"
|
||||
)
|
||||
|
||||
// KindGateUnanswered is the holder's finding of a waiting call past its ask's 60 minutes with no decided heard (§677),
|
||||
// its id the call's.
|
||||
const KindGateUnanswered = "gate-unanswered"
|
||||
|
||||
// GatedEnded is a held call's end as the holder says it (§677): no argument is in it.
|
||||
type GatedEnded struct {
|
||||
Call string `json:"call"`
|
||||
Ask string `json:"ask"`
|
||||
Act string `json:"act"`
|
||||
Verb string `json:"verb"`
|
||||
Outcome string `json:"outcome"`
|
||||
Answer any `json:"answer,omitempty"`
|
||||
Approval string `json:"approval,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// GatedEndedSubject is the subject one call's end is said on: `mesh.seat.<seat>.event.gated-ended.<caller>.<call>`, the
|
||||
// caller the account the bus proved, escaped as a key's name is (§677, §73).
|
||||
func GatedEndedSubject(seat, caller, call string) string {
|
||||
return "mesh.seat." + seat + ".event." + GatedEndedEvent + "." + keys.EscapeName(caller) + "." + call
|
||||
}
|
||||
|
||||
// WaitingCall is one call the holder keeps under `call.<id>` (§677): the call whole, the caller the bus proved, its ask,
|
||||
// and once it ended its outcome, and whether that end was said.
|
||||
type WaitingCall struct {
|
||||
ID string `json:"id"`
|
||||
Caller string `json:"caller"`
|
||||
Ask string `json:"ask"`
|
||||
Digest string `json:"digest"`
|
||||
Act operatorask.Act `json:"act"`
|
||||
Address string `json:"address"`
|
||||
Seat string `json:"seat"`
|
||||
Verb string `json:"verb"`
|
||||
Arguments json.RawMessage `json:"arguments,omitempty"`
|
||||
// Made is when the call was made; Asked when its ask (the newest, after an ask anew) was raised or taken.
|
||||
Made time.Time `json:"made"`
|
||||
Asked time.Time `json:"asked"`
|
||||
// End is its end once it ended; Said says End was said on gated-ended.
|
||||
End *GatedEnded `json:"end,omitempty"`
|
||||
Said bool `json:"said,omitempty"`
|
||||
}
|
||||
|
||||
func callKey(id string) string { return "call." + id }
|
||||
func claimedKey(id string) string { return "claimed." + id }
|
||||
|
||||
// Serves names the handler a waiting call of seat's verb runs on its Approve (§677): process.Conn's AnswerSeat gives each
|
||||
// marked verb's.
|
||||
func (g *Gate) Serves(seat, verb string, h func(caller string, args json.RawMessage) (any, error)) {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
if g.handlers == nil {
|
||||
g.handlers = map[string]func(string, json.RawMessage) (any, error){}
|
||||
}
|
||||
g.handlers[seat+"."+verb] = h
|
||||
}
|
||||
|
||||
func (g *Gate) handler(seat, verb string) func(string, json.RawMessage) (any, error) {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
return g.handlers[seat+"."+verb]
|
||||
}
|
||||
|
||||
// Hold is a call of a marked verb the holder serves (§677). A call whose arguments may not be asked, or that comes while
|
||||
// the state approvals cannot be read or written, is refused at once and nothing is kept. One that waits (the header
|
||||
// WaitHeader) is kept as a waiting call and answered at once with Held; one that does not takes ADR 0286 §37's path: it is
|
||||
// refused naming its act's ask, and the same call made again after the Approve runs. A call of an act declined within 10
|
||||
// minutes ends declined at once, unasked.
|
||||
func (g *Gate) Hold(c Call, wait bool, run func() (any, error)) (any, *Held, error) {
|
||||
if c.Fingerprint == "" {
|
||||
c.Fingerprint = g.d.Build
|
||||
}
|
||||
if c.Fingerprint == "" {
|
||||
return nil, nil, g.refused(c, "", "the holder's own fingerprint is unknown, so no warrant can bind its build")
|
||||
}
|
||||
if g.d.State == nil {
|
||||
return nil, nil, g.refused(c, "", "its holder keeps no state "+StateName+" to keep its calls in")
|
||||
}
|
||||
if why := refusedUnasked(c.Args); why != "" {
|
||||
return nil, nil, g.refused(c, "", why)
|
||||
}
|
||||
if why := g.held(); why != "" {
|
||||
return nil, nil, g.refused(c, "", why)
|
||||
}
|
||||
act, err := ActOf(c, "") // a process holder's act names no node, so two instances raise one ask (A-255)
|
||||
if err != nil {
|
||||
return nil, nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
digest, err := operatorask.ActDigest(act)
|
||||
if err != nil {
|
||||
return nil, nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
if !wait {
|
||||
answer, err := g.unheld(c, act, digest, run)
|
||||
return answer, nil, err
|
||||
}
|
||||
id, err := g.askOf(c, act, digest)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
now := g.d.Now().UTC()
|
||||
w := WaitingCall{ID: newID(), Caller: c.Caller, Ask: id, Digest: digest, Act: act, Address: c.Address, Seat: c.Seat,
|
||||
Verb: c.Name, Arguments: c.Args, Made: now, Asked: now}
|
||||
if _, err := g.d.State.Create(callKey(w.ID), w); err != nil {
|
||||
g.stateFailed(err)
|
||||
return nil, nil, g.refused(c, digest, "its call cannot be kept in "+StateName+": "+err.Error())
|
||||
}
|
||||
g.stateRead()
|
||||
g.armUnanswered(w)
|
||||
g.d.Go(func() { g.Decided(id) }) // a choice made before the call was kept is acted on now
|
||||
return nil, &Held{Asked: id, Call: w.ID}, nil
|
||||
}
|
||||
|
||||
// askOf is the ask a call of the act takes (§677): the one `act.<digest>` names while it is open, else a new one, written
|
||||
// with the bus's create (so two instances raise one ask) or, over an ended one, by compare-and-set on its revision. A call
|
||||
// of an act declined within 10 minutes is refused with the time it may be asked again.
|
||||
func (g *Gate) askOf(c Call, act operatorask.Act, digest string) (string, error) {
|
||||
key := g.actKey(digest)
|
||||
for {
|
||||
e, err := g.d.State.Get(key)
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
return "", g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
|
||||
}
|
||||
g.stateRead()
|
||||
id := newID()
|
||||
if e == nil {
|
||||
if _, err := g.d.State.Create(key, id); err != nil {
|
||||
if isChanged(err) {
|
||||
continue
|
||||
}
|
||||
g.stateFailed(err)
|
||||
return "", g.refused(c, digest, "its act cannot be kept in "+StateName+": "+err.Error())
|
||||
}
|
||||
return id, g.raiseHeld(c, act, digest, id)
|
||||
}
|
||||
var named string
|
||||
if err := json.Unmarshal(e.Value, &named); err != nil || named == "" {
|
||||
return "", g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName))
|
||||
}
|
||||
open, until, err := g.openAsk(c.Module, named)
|
||||
if err != nil {
|
||||
return "", g.refused(c, digest, err.Error())
|
||||
}
|
||||
if !until.IsZero() {
|
||||
return "", fmt.Errorf("Not run: the operator declined ask %s; the same call is not asked again until %s", named, formatTime(until))
|
||||
}
|
||||
if open {
|
||||
return named, nil
|
||||
}
|
||||
if _, err := g.d.State.Update(key, id, e.Revision); err != nil {
|
||||
if isChanged(err) {
|
||||
continue
|
||||
}
|
||||
g.stateFailed(err)
|
||||
return "", g.refused(c, digest, "its act cannot be kept in "+StateName+": "+err.Error())
|
||||
}
|
||||
return id, g.raiseHeld(c, act, digest, id)
|
||||
}
|
||||
}
|
||||
|
||||
// openAsk says whether an ask is open for another call of its act: raised and not yet ended by the router; and, when it was
|
||||
// declined within 10 minutes, until when no call of its act is asked.
|
||||
func (g *Gate) openAsk(module, id string) (bool, time.Time, error) {
|
||||
rec, err := g.record(module, id)
|
||||
if err != nil {
|
||||
return false, time.Time{}, fmt.Errorf("the router's record of its ask %s cannot be read: %v", id, err)
|
||||
}
|
||||
if rec == nil {
|
||||
k, _, err := g.kept(id)
|
||||
return err == nil && k != nil && k.Ended == "", time.Time{}, nil // raised and not yet taken by the router
|
||||
}
|
||||
switch rec.State {
|
||||
case "open", "held-back":
|
||||
return true, time.Time{}, nil
|
||||
case "chosen":
|
||||
if w := rec.Warrant; w != nil && w.Option == OptionDecline {
|
||||
at, _ := parseTime(w.At)
|
||||
if until := at.Add(DeclinedFor); g.d.Now().Before(until) {
|
||||
return false, until, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
return false, time.Time{}, nil
|
||||
}
|
||||
|
||||
// raiseHeld raises the ask the act's key names, under the holder's module's name, keeping the act and the arguments beside
|
||||
// it (§677; design 46 7-approval-mark §10).
|
||||
func (g *Gate) raiseHeld(c Call, act operatorask.Act, digest, id string) error {
|
||||
return g.keepAskHeld(c, act, digest, g.askFor(c, digest, id))
|
||||
}
|
||||
|
||||
// keepAskHeld is keepAsk without its write of the act's key, which askOf made first.
|
||||
func (g *Gate) keepAskHeld(c Call, act operatorask.Act, digest string, a Ask) error {
|
||||
if size := keptSize(a, act, c.Args); size > MostKeptValue {
|
||||
return g.refused(c, digest, fmt.Sprintf("its ask would keep %d bytes, more than the %d a value of the state %s holds", size, MostKeptValue, StateName))
|
||||
}
|
||||
if err := g.d.Channel.Ask(c.Module, a); err != nil {
|
||||
return g.refused(c, digest, "the ask could not be raised: "+err.Error())
|
||||
}
|
||||
if err := g.keepCall(a.ID, act, c.Args); err != nil {
|
||||
g.stateFailed(err)
|
||||
return g.refused(c, digest, "the ask was raised and its call could not be kept: "+err.Error())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// unheld is a call that does not wait (ADR 0286 §37's path, §677): run when its act's ask is chosen Approve within 30
|
||||
// minutes and unclaimed, claimed once; otherwise refused naming the ask, raising one when none is open.
|
||||
func (g *Gate) unheld(c Call, act operatorask.Act, digest string, run func() (any, error)) (any, error) {
|
||||
named, err := g.d.State.Get(g.actKey(digest))
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
return nil, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
|
||||
}
|
||||
g.stateRead()
|
||||
if named != nil {
|
||||
var id string
|
||||
if json.Unmarshal(named.Value, &id) == nil && id != "" {
|
||||
rec, err := g.record(c.Module, id)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error())
|
||||
}
|
||||
if rec != nil && rec.State == "chosen" && rec.Warrant != nil && rec.Warrant.Option == OptionApprove {
|
||||
if at, _ := parseTime(rec.Warrant.At); !g.d.Now().After(at.Add(RunWithin)) {
|
||||
out := g.claimRun(c.Module, id, *rec.Warrant, act, run)
|
||||
switch out.outcome {
|
||||
case OutcomeRan:
|
||||
return g.ran(c, id, rec, out.answer, out.err)
|
||||
case OutcomeRefused:
|
||||
return nil, g.refused(c, digest, out.why)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
id, err := g.askOf(c, act, digest)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return nil, g.firstAnswer(c, id, digest, "")
|
||||
}
|
||||
|
||||
// claimed is what came of a claim of a warrant for one call.
|
||||
type claimed struct {
|
||||
outcome string // OutcomeRan, OutcomeRefused, or "" when another took it
|
||||
answer any
|
||||
err error
|
||||
why string
|
||||
}
|
||||
|
||||
// claimRun holds the warrant to the ask and the call's act, claims it once at `claimed.<ask id>` with the bus's create, and
|
||||
// runs the call (§677; ADR 0286 §27, §28, §30, §31): of two instances, or two deliveries, one runs.
|
||||
func (g *Gate) claimRun(module, id string, w Warrant, act operatorask.Act, run func() (any, error)) claimed {
|
||||
var out claimed
|
||||
var claimErr error
|
||||
h, err := g.d.Channel.Claim(module, w, act, func() {
|
||||
if _, err := g.d.State.Create(claimedKey(id), map[string]string{"at": formatTime(g.d.Now().UTC())}); err != nil {
|
||||
claimErr = err
|
||||
return
|
||||
}
|
||||
out.outcome = OutcomeRan
|
||||
out.answer, out.err = run()
|
||||
})
|
||||
switch {
|
||||
case h == Claimed && claimErr != nil && !isChanged(claimErr):
|
||||
g.stateFailed(claimErr)
|
||||
return claimed{outcome: OutcomeRefused, why: "its claim cannot be written in " + StateName + ": " + claimErr.Error()}
|
||||
case h == Claimed:
|
||||
return out
|
||||
case h == ClaimDone:
|
||||
return claimed{}
|
||||
case h == "" && err != nil:
|
||||
g.stateFailed(err)
|
||||
return claimed{outcome: OutcomeRefused, why: "its state " + StateName + " cannot be written: " + err.Error()}
|
||||
default: // refused, or the act about to run is not the one the option bound: nothing runs
|
||||
return claimed{outcome: OutcomeRefused, why: "its warrant does not hold: " + errText(err)}
|
||||
}
|
||||
}
|
||||
|
||||
// Decided acts on the router's own record of one of the holder's asks (§677): heard on the holder's `decided`, never from
|
||||
// its body, and at the start for each ask a waiting call waits on. On the Approve the oldest waiting call of the ask runs,
|
||||
// within 30 minutes of the choice, else it ends expired; every other is asked anew. A Decline ends every one declined, an
|
||||
// ask that ended without a choice every one expired, and one the router refused every one refused with its words.
|
||||
func (g *Gate) Decided(id string) {
|
||||
module := g.d.Runner
|
||||
rec, err := g.record(module, id)
|
||||
if err != nil {
|
||||
g.d.Logf("[approval] the router's record of ask %s cannot be read (%v): read again at the next start", id, err)
|
||||
return
|
||||
}
|
||||
if rec == nil || rec.State == "open" || rec.State == "held-back" {
|
||||
return
|
||||
}
|
||||
calls, err := g.waiting(func(w WaitingCall) bool { return w.Ask == id })
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
g.d.Logf("[approval] the waiting calls of ask %s cannot be read (%v): read again at the next start", id, err)
|
||||
return
|
||||
}
|
||||
if len(calls) == 0 {
|
||||
return
|
||||
}
|
||||
end := func(w WaitingCall, outcome, why string) {
|
||||
g.end(w, GatedEnded{Outcome: outcome, Error: why})
|
||||
}
|
||||
switch {
|
||||
case rec.State == "refused":
|
||||
for _, w := range calls {
|
||||
end(w, OutcomeRefused, "the router refused its ask "+id+": "+orNone(rec.Answer))
|
||||
}
|
||||
return
|
||||
case rec.State != "chosen":
|
||||
for _, w := range calls {
|
||||
end(w, OutcomeExpired, "its ask "+id+" ended "+rec.State+" with no choice; nothing ran")
|
||||
}
|
||||
return
|
||||
case rec.Warrant == nil:
|
||||
for _, w := range calls {
|
||||
end(w, OutcomeRefused, "the router's record of its ask "+id+" is chosen and carries no warrant")
|
||||
}
|
||||
return
|
||||
case rec.Warrant.Option == OptionDecline:
|
||||
for _, w := range calls {
|
||||
end(w, OutcomeDeclined, "the operator declined ask "+id+"; nothing ran")
|
||||
}
|
||||
return
|
||||
}
|
||||
first, rest := calls[0], calls[1:]
|
||||
at, _ := parseTime(rec.Warrant.At)
|
||||
if g.d.Now().After(at.Add(RunWithin)) {
|
||||
end(first, OutcomeExpired, "the Approve of ask "+id+" was heard more than 30 minutes after it was given; nothing ran")
|
||||
} else {
|
||||
g.runWaiting(first, rec)
|
||||
}
|
||||
for _, w := range rest {
|
||||
g.anew(w)
|
||||
}
|
||||
}
|
||||
|
||||
// runWaiting claims and runs one waiting call on its ask's warrant, saying its end.
|
||||
func (g *Gate) runWaiting(w WaitingCall, rec *Record) {
|
||||
h := g.handler(w.Seat, w.Verb)
|
||||
c := Call{Address: w.Address, Module: g.d.Runner, Seat: w.Seat, Name: w.Verb, Caller: w.Caller, Args: w.Arguments}
|
||||
if h == nil {
|
||||
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: "the holder serves no handler of " + w.Address + " now; nothing ran"})
|
||||
return
|
||||
}
|
||||
out := g.claimRun(g.d.Runner, w.Ask, *rec.Warrant, w.Act, func() (any, error) { return h(w.Caller, w.Arguments) })
|
||||
switch out.outcome {
|
||||
case "": // another instance, or another call of the same ask, took the warrant
|
||||
return
|
||||
case OutcomeRefused:
|
||||
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: out.why})
|
||||
return
|
||||
}
|
||||
answered, err := g.ran(c, w.Ask, rec, out.answer, out.err)
|
||||
if err != nil {
|
||||
g.end(w, GatedEnded{Outcome: OutcomeRan, Error: err.Error()})
|
||||
return
|
||||
}
|
||||
a := answered.(Answered)
|
||||
g.end(w, GatedEnded{Outcome: OutcomeRan, Answer: a.Answer, Approval: a.Approval})
|
||||
}
|
||||
|
||||
// anew asks a waiting call's act anew, as a new ask (ADR 0286 §33: a second run takes a second approval); its end comes
|
||||
// when its own ask ends.
|
||||
func (g *Gate) anew(w WaitingCall) {
|
||||
c := Call{Address: w.Address, Module: g.d.Runner, Seat: w.Seat, Name: w.Verb, Caller: w.Caller, Args: w.Arguments,
|
||||
Fingerprint: w.Act["build"]}
|
||||
id, err := g.askOf(c, w.Act, w.Digest)
|
||||
if err != nil {
|
||||
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: err.Error()})
|
||||
return
|
||||
}
|
||||
if err := g.updateCall(w.ID, func(k *WaitingCall) { k.Ask, k.Asked = id, g.d.Now().UTC() }); err != nil {
|
||||
g.d.Logf("[approval] the call %s, asked anew as ask %s, cannot be kept (%v)", w.ID, id, err)
|
||||
return
|
||||
}
|
||||
w.Ask, w.Asked = id, g.d.Now().UTC()
|
||||
g.rearmUnanswered(w)
|
||||
g.d.Go(func() { g.Decided(id) })
|
||||
}
|
||||
|
||||
// end keeps a waiting call's end, then says it on gated-ended (§677); an end not said is said again at the next start.
|
||||
func (g *Gate) end(w WaitingCall, e GatedEnded) {
|
||||
e.Call, e.Ask, e.Act, e.Verb = w.ID, w.Ask, w.Digest, w.Verb
|
||||
ended := false
|
||||
if err := g.updateCall(w.ID, func(k *WaitingCall) {
|
||||
if k.End == nil {
|
||||
k.End, ended = &e, true
|
||||
}
|
||||
}); err != nil {
|
||||
g.d.Logf("[approval] the end %s of call %s cannot be kept (%v)", e.Outcome, w.ID, err)
|
||||
return
|
||||
}
|
||||
if ended {
|
||||
g.say(w.ID)
|
||||
}
|
||||
}
|
||||
|
||||
// say says a kept end on gated-ended and marks it said; its finding gate-unanswered is cleared.
|
||||
func (g *Gate) say(id string) {
|
||||
e, err := g.d.State.Get(callKey(id))
|
||||
if err != nil || e == nil {
|
||||
return
|
||||
}
|
||||
var w WaitingCall
|
||||
if json.Unmarshal(e.Value, &w) != nil || w.End == nil || w.Said {
|
||||
return
|
||||
}
|
||||
// Its bound key still kept says gate-unanswered was not raised, and the key goes; one the bus ended is cleared once the
|
||||
// end is said (A-255).
|
||||
b, err := g.d.State.Get(unansweredHeldKey(id))
|
||||
raised := err == nil && b == nil
|
||||
if b != nil && g.d.Bounds != nil {
|
||||
_ = g.d.Bounds.Delete(unansweredHeldKey(id))
|
||||
}
|
||||
if g.d.Say == nil {
|
||||
g.d.Logf("[approval] the end %s of call %s is not said: the holder says no gated-ended", w.End.Outcome, id)
|
||||
return
|
||||
}
|
||||
if err := g.d.Say(GatedEndedSubject(w.Seat, w.Caller, w.ID), *w.End, GatedEndedEvent+"."+w.ID); err != nil {
|
||||
g.d.Logf("[approval] the end %s of call %s is not said (%v): said again at the next start", w.End.Outcome, id, err)
|
||||
return
|
||||
}
|
||||
g.d.Logf("[approval] %s by %s ended %s (call %s, ask %s)", w.Address, w.Caller, w.End.Outcome, id, w.Ask)
|
||||
_ = g.updateCall(id, func(k *WaitingCall) { k.Said = true })
|
||||
if raised && g.d.Findings != nil {
|
||||
_ = g.d.Findings.Clear(nilCtx(), KindGateUnanswered, id, "the call's end, "+w.End.Outcome+", is said")
|
||||
}
|
||||
}
|
||||
|
||||
// updateCall changes a kept call by compare-and-set, again when another write came first.
|
||||
func (g *Gate) updateCall(id string, change func(*WaitingCall)) error {
|
||||
for {
|
||||
e, err := g.d.State.Get(callKey(id))
|
||||
if err != nil || e == nil {
|
||||
return fmt.Errorf("the call %s cannot be read: %v", id, err)
|
||||
}
|
||||
var w WaitingCall
|
||||
if err := json.Unmarshal(e.Value, &w); err != nil {
|
||||
return err
|
||||
}
|
||||
change(&w)
|
||||
if _, err := g.d.State.Update(callKey(id), w, e.Revision); err == nil {
|
||||
return nil
|
||||
} else if !isChanged(err) {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// waiting is every kept call that has not ended and that keep takes, oldest first.
|
||||
func (g *Gate) waiting(keep func(WaitingCall) bool) ([]WaitingCall, error) {
|
||||
all, err := g.calls()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out []WaitingCall
|
||||
for _, w := range all {
|
||||
if w.End == nil && keep(w) {
|
||||
out = append(out, w)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// calls is every kept call, oldest first.
|
||||
func (g *Gate) calls() ([]WaitingCall, error) {
|
||||
names, err := g.d.State.Keys()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out []WaitingCall
|
||||
for _, key := range names {
|
||||
if !strings.HasPrefix(key, "call.") {
|
||||
continue
|
||||
}
|
||||
e, err := g.d.State.Get(key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var w WaitingCall
|
||||
if e == nil || json.Unmarshal(e.Value, &w) != nil {
|
||||
continue
|
||||
}
|
||||
out = append(out, w)
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].Made.Before(out[j].Made) })
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// unansweredHeldKey is a waiting call's bound key in the holder's state approvals (lead answer A-255): created with the
|
||||
// ask's 60 minutes as its own time to live, its expiry heard as the bus's event raises gate-unanswered.
|
||||
func unansweredHeldKey(call string) string { return "unanswered." + call }
|
||||
|
||||
// armUnanswered keeps the waiting call's bound key (§677; A-255): when its ask's 60 minutes pass with no end of it, the bus
|
||||
// ends the key and the holder, hearing that, raises gate-unanswered; no clock waits in the process. A call already past its
|
||||
// 60 minutes (its key ended while the holder was away) is judged at once.
|
||||
func (g *Gate) armUnanswered(w WaitingCall) {
|
||||
left := w.Asked.Add(AskLasts).Sub(g.d.Now())
|
||||
if left <= 0 {
|
||||
g.heldUnanswered(w.ID)
|
||||
return
|
||||
}
|
||||
if g.d.Bounds == nil {
|
||||
g.d.Logf("[approval] the call %s waits on ask %s with no bound kept: the holder's %s keeps no key with its own time to live",
|
||||
w.ID, w.Ask, StateName)
|
||||
return
|
||||
}
|
||||
if err := g.d.Bounds.CreateTTL(unansweredHeldKey(w.ID), map[string]string{"call": w.ID, "ask": w.Ask}, left); err != nil && !isChanged(err) {
|
||||
g.d.Logf("[approval] the bound of call %s on ask %s is not kept: %v", w.ID, w.Ask, err)
|
||||
}
|
||||
}
|
||||
|
||||
// rearmUnanswered keeps a call asked anew under a new bound of its new ask's 60 minutes.
|
||||
func (g *Gate) rearmUnanswered(w WaitingCall) {
|
||||
if g.d.Bounds != nil {
|
||||
_ = g.d.Bounds.Delete(unansweredHeldKey(w.ID))
|
||||
}
|
||||
g.armUnanswered(w)
|
||||
}
|
||||
|
||||
// heldUnanswered is a waiting call's bound ended (A-255): gate-unanswered raised while the call has no end.
|
||||
func (g *Gate) heldUnanswered(id string) {
|
||||
e, err := g.d.State.Get(callKey(id))
|
||||
if err != nil || e == nil {
|
||||
return
|
||||
}
|
||||
var w WaitingCall
|
||||
if json.Unmarshal(e.Value, &w) != nil || w.End != nil || g.d.Findings == nil {
|
||||
return
|
||||
}
|
||||
if err := g.d.Findings.Raise(nilCtx(), KindGateUnanswered, w.ID, fmt.Sprintf("the call %s of %s by %s waits on ask %s, "+
|
||||
"past its %s with no decided heard: nothing ran; the holder's %s.approvals reads it", w.ID, w.Address, w.Caller, w.Ask,
|
||||
AskLasts, w.Seat), "a call waits on an unanswered ask"); err != nil {
|
||||
g.d.Logf("[approval] the finding %s of call %s is not kept: %v", KindGateUnanswered, w.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
// watchHeld hears the waiting calls' bound keys end (A-255) for the life of the holder's connection.
|
||||
func (g *Gate) watchHeld() error {
|
||||
if g.d.Bounds == nil {
|
||||
return nil
|
||||
}
|
||||
_, err := g.d.Bounds.Watch("unanswered.*", func(key string) { g.heldUnanswered(strings.TrimPrefix(key, "unanswered.")) })
|
||||
return err
|
||||
}
|
||||
|
||||
// StartHolding is the holder's read at its start (§677): every waiting call is read, each ask already decided acted on as
|
||||
// when its decided is heard, each end not yet said said, each waiting call's 60 minutes armed, and every acted, unrecorded
|
||||
// run recorded. A state that cannot be read leaves every marked call refused until it can.
|
||||
func (g *Gate) StartHolding() error {
|
||||
all, err := g.calls()
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
return fmt.Errorf("the waiting calls in %s cannot be read: %w", StateName, err)
|
||||
}
|
||||
g.stateRead()
|
||||
if err := g.watchHeld(); err != nil {
|
||||
return fmt.Errorf("the bound keys in %s cannot be watched: %w", StateName, err)
|
||||
}
|
||||
asks := map[string]bool{}
|
||||
for _, w := range all {
|
||||
switch {
|
||||
case w.End != nil && !w.Said:
|
||||
g.say(w.ID)
|
||||
case w.End == nil:
|
||||
g.armUnanswered(w)
|
||||
asks[w.Ask] = true
|
||||
}
|
||||
}
|
||||
ids := make([]string, 0, len(asks))
|
||||
for id := range asks {
|
||||
ids = append(ids, id)
|
||||
}
|
||||
sort.Strings(ids)
|
||||
for _, id := range ids {
|
||||
g.Decided(id)
|
||||
}
|
||||
g.Recover()
|
||||
return nil
|
||||
}
|
||||
|
||||
// Approvals is the seat's verb `approvals` (§677): the waiting and ended calls with their caller, ask and state, the
|
||||
// `act.` keys and the claims; a call's arguments only to that call's caller.
|
||||
type Approvals struct {
|
||||
Calls []ListedCall `json:"calls"`
|
||||
Acts []ListedAct `json:"acts"`
|
||||
Claims []string `json:"claims"`
|
||||
}
|
||||
|
||||
// ListedCall is one kept call as `approvals` lists it.
|
||||
type ListedCall struct {
|
||||
Call string `json:"call"`
|
||||
Caller string `json:"caller"`
|
||||
Ask string `json:"ask"`
|
||||
Verb string `json:"verb"`
|
||||
State string `json:"state"`
|
||||
Made time.Time `json:"made"`
|
||||
End *GatedEnded `json:"end,omitempty"`
|
||||
Arguments json.RawMessage `json:"arguments,omitempty"`
|
||||
}
|
||||
|
||||
// ListedAct is one `act.` key: its digest, escaped as kept, and the ask it names.
|
||||
type ListedAct struct {
|
||||
Act string `json:"act"`
|
||||
Ask string `json:"ask"`
|
||||
}
|
||||
|
||||
// Approvals answers the verb for caller, the account the bus proved.
|
||||
func (g *Gate) Approvals(caller string) (Approvals, error) {
|
||||
out := Approvals{Calls: []ListedCall{}, Acts: []ListedAct{}, Claims: []string{}}
|
||||
if g.d.State == nil {
|
||||
return out, fmt.Errorf("the holder keeps no state %s", StateName)
|
||||
}
|
||||
all, err := g.calls()
|
||||
if err != nil {
|
||||
return out, fmt.Errorf("the state %s cannot be read: %w", StateName, err)
|
||||
}
|
||||
for _, w := range all {
|
||||
l := ListedCall{Call: w.ID, Caller: w.Caller, Ask: w.Ask, Verb: w.Verb, State: "waiting", Made: w.Made, End: w.End}
|
||||
if w.End != nil {
|
||||
l.State = w.End.Outcome
|
||||
}
|
||||
if caller != "" && caller == w.Caller {
|
||||
l.Arguments = w.Arguments
|
||||
}
|
||||
out.Calls = append(out.Calls, l)
|
||||
}
|
||||
names, err := g.d.State.Keys()
|
||||
if err != nil {
|
||||
return out, fmt.Errorf("the state %s cannot be read: %w", StateName, err)
|
||||
}
|
||||
sort.Strings(names)
|
||||
for _, key := range names {
|
||||
switch {
|
||||
case strings.HasPrefix(key, "act."):
|
||||
e, err := g.d.State.Get(key)
|
||||
var id string
|
||||
if err == nil && e != nil {
|
||||
_ = json.Unmarshal(e.Value, &id)
|
||||
}
|
||||
out.Acts = append(out.Acts, ListedAct{Act: strings.TrimPrefix(key, "act."), Ask: id})
|
||||
case strings.HasPrefix(key, "claimed."):
|
||||
out.Claims = append(out.Claims, strings.TrimPrefix(key, "claimed."))
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func nilCtx() context.Context { return context.Background() }
|
||||
@@ -1,439 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
)
|
||||
|
||||
// memStore is the holder's state approvals in memory: create refuses a key that exists, update is compare-and-set.
|
||||
type memStore struct {
|
||||
mu sync.Mutex
|
||||
vals map[string][]byte
|
||||
revs map[string]uint64
|
||||
rev uint64
|
||||
fail error
|
||||
}
|
||||
|
||||
func newMem() *memStore { return &memStore{vals: map[string][]byte{}, revs: map[string]uint64{}} }
|
||||
|
||||
func (m *memStore) Get(key string) (*stdio.StateEntry, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.fail != nil {
|
||||
return nil, m.fail
|
||||
}
|
||||
v, ok := m.vals[key]
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
return &stdio.StateEntry{Key: key, Value: v, Revision: m.revs[key]}, nil
|
||||
}
|
||||
|
||||
func (m *memStore) write(key string, value any, check func(exists bool, rev uint64) bool) (uint64, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.fail != nil {
|
||||
return 0, m.fail
|
||||
}
|
||||
_, exists := m.vals[key]
|
||||
if !check(exists, m.revs[key]) {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
raw, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
m.rev++
|
||||
m.vals[key], m.revs[key] = raw, m.rev
|
||||
return m.rev, nil
|
||||
}
|
||||
|
||||
func (m *memStore) Create(key string, value any) (uint64, error) {
|
||||
return m.write(key, value, func(exists bool, _ uint64) bool { return !exists })
|
||||
}
|
||||
func (m *memStore) Update(key string, value any, revision uint64) (uint64, error) {
|
||||
return m.write(key, value, func(exists bool, rev uint64) bool { return exists && rev == revision })
|
||||
}
|
||||
func (m *memStore) Put(key string, value any) (uint64, error) {
|
||||
return m.write(key, value, func(bool, uint64) bool { return true })
|
||||
}
|
||||
func (m *memStore) Keys() ([]string, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.fail != nil {
|
||||
return nil, m.fail
|
||||
}
|
||||
var out []string
|
||||
for k := range m.vals {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// bounds is the holder's bound keys in its state approvals as a fake (A-255): each key kept in the state with its time to
|
||||
// live, expire ending it as the bus does and handing it to the watch.
|
||||
type bounds struct {
|
||||
state *memStore
|
||||
mu sync.Mutex
|
||||
ttl map[string]time.Duration
|
||||
watched func(string)
|
||||
}
|
||||
|
||||
func (b *bounds) CreateTTL(key string, value any, ttl time.Duration) error {
|
||||
if _, err := b.state.Create(key, value); err != nil {
|
||||
return err
|
||||
}
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
b.ttl[key] = ttl
|
||||
return nil
|
||||
}
|
||||
func (b *bounds) Delete(key string) error {
|
||||
b.state.delete(key)
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
delete(b.ttl, key)
|
||||
return nil
|
||||
}
|
||||
func (b *bounds) Watch(pattern string, expired func(string)) (func(), error) {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
b.watched = expired
|
||||
return func() {}, nil
|
||||
}
|
||||
func (b *bounds) expire(key string) {
|
||||
b.mu.Lock()
|
||||
_, ok := b.ttl[key]
|
||||
delete(b.ttl, key)
|
||||
w := b.watched
|
||||
b.mu.Unlock()
|
||||
b.state.delete(key)
|
||||
if ok && w != nil {
|
||||
w(key)
|
||||
}
|
||||
}
|
||||
|
||||
func (m *memStore) delete(key string) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
delete(m.vals, key)
|
||||
delete(m.revs, key)
|
||||
}
|
||||
|
||||
// router is the operator channel as a fake: it keeps each ask the gate raises, and the test decides its record.
|
||||
type router struct {
|
||||
mu sync.Mutex
|
||||
state Store
|
||||
asks []Ask
|
||||
records map[string]*Record
|
||||
}
|
||||
|
||||
func (r *router) Ask(module string, a Ask) error {
|
||||
raw, _ := json.Marshal(a)
|
||||
if _, err := r.state.Create("ask."+a.ID, kept{Ask: raw, Asked: time.Now()}); err != nil {
|
||||
return err
|
||||
}
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.asks = append(r.asks, a)
|
||||
return nil
|
||||
}
|
||||
func (r *router) Record(module, id string) (*Record, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
return r.records[id], nil
|
||||
}
|
||||
func (r *router) Claim(module string, w Warrant, act operatorask.Act, run func()) (Claim, error) {
|
||||
e, _ := r.state.Get("ask." + w.Ask)
|
||||
if e == nil {
|
||||
return ClaimRefused, fmt.Errorf("no ask %s", w.Ask)
|
||||
}
|
||||
var k kept
|
||||
_ = json.Unmarshal(e.Value, &k)
|
||||
if k.Acted {
|
||||
return ClaimDone, nil
|
||||
}
|
||||
var a Ask
|
||||
_ = json.Unmarshal(k.Ask, &a)
|
||||
digest, _ := operatorask.ActDigest(act)
|
||||
if a.About != digest {
|
||||
return ClaimRefused, fmt.Errorf("the option binds another act")
|
||||
}
|
||||
k.Acted = true
|
||||
if _, err := r.state.Update("ask."+w.Ask, k, e.Revision); err != nil {
|
||||
return ClaimDone, nil
|
||||
}
|
||||
run()
|
||||
return Claimed, nil
|
||||
}
|
||||
func (r *router) Cancel(module, id string) error { return nil }
|
||||
func (r *router) Decided(module string) string { return "decided." + module }
|
||||
func (r *router) Seat() string { return "operator-channel" }
|
||||
|
||||
func (r *router) choose(id, option string, at time.Time) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.records[id] = &Record{State: "chosen", OfferedOn: []string{"telegram"},
|
||||
Warrant: &Warrant{Ask: id, Asker: "holder", Option: option, Channel: "telegram", At: formatTime(at)}}
|
||||
}
|
||||
|
||||
type findings struct {
|
||||
mu sync.Mutex
|
||||
raised map[string]bool
|
||||
heard chan string
|
||||
}
|
||||
|
||||
func (f *findings) Raise(_ context.Context, kind, id, _, _ string) error {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.raised[kind+" "+id] = true
|
||||
if f.heard != nil {
|
||||
f.heard <- kind + " " + id
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (f *findings) Clear(_ context.Context, kind, id, _ string) error {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
delete(f.raised, kind+" "+id)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Design 47 G §677 (lead answers A-158, A-209, A-244): a process holder's marked verb is one call with a gate before its
|
||||
// running, its waiting calls, act keys and claims kept in its own state.
|
||||
func TestAHolderHoldsAMarkedCall(t *testing.T) {
|
||||
now := time.Date(2026, 11, 12, 10, 0, 0, 0, time.UTC)
|
||||
type world struct {
|
||||
g *Gate
|
||||
state *memStore
|
||||
router *router
|
||||
said map[string]GatedEnded
|
||||
ran *int
|
||||
recs *int
|
||||
finds *findings
|
||||
after chan time.Time
|
||||
bounds *bounds
|
||||
}
|
||||
newGate := func(state *memStore, r *router, ran, recs *int, said map[string]GatedEnded, after chan time.Time, f *findings, b *bounds) *Gate {
|
||||
g := New(Deps{Node: "n1", Runner: "holder", Holding: true, Build: "fp1", State: state, Channel: r, Findings: f, Bounds: b,
|
||||
Now: func() time.Time { return now }, Go: func(f func()) { f() },
|
||||
After: func(d time.Duration) <-chan time.Time {
|
||||
if d < time.Minute { // the record's short pauses pass at once
|
||||
fired := make(chan time.Time)
|
||||
close(fired)
|
||||
return fired
|
||||
}
|
||||
return after
|
||||
},
|
||||
Warranted: func(asker, ask string, open func() (bool, error)) error {
|
||||
*recs++
|
||||
return nil
|
||||
},
|
||||
Say: func(subject string, body any, id string) error {
|
||||
said[subject] = body.(GatedEnded)
|
||||
return nil
|
||||
}})
|
||||
g.Serves("wiper", "wipe", func(caller string, args json.RawMessage) (any, error) { *ran++; return "wiped", nil })
|
||||
return g
|
||||
}
|
||||
setup := func() world {
|
||||
state := newMem()
|
||||
r := &router{state: state, records: map[string]*Record{}}
|
||||
ran, recs := 0, 0
|
||||
w := world{state: state, router: r, said: map[string]GatedEnded{}, ran: &ran, recs: &recs,
|
||||
finds: &findings{raised: map[string]bool{}}, after: make(chan time.Time), bounds: &bounds{state: state, ttl: map[string]time.Duration{}}}
|
||||
w.g = newGate(state, r, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
|
||||
return w
|
||||
}
|
||||
call := func(caller, args string) Call {
|
||||
return Call{Address: "wiper.wipe", Module: "holder", Seat: "wiper", Name: "wipe", Caller: caller, Args: json.RawMessage(args)}
|
||||
}
|
||||
hold := func(t *testing.T, w world, c Call, wait bool) (any, *Held, error) {
|
||||
return w.g.Hold(c, wait, func() (any, error) { *w.ran++; return "wiped", nil })
|
||||
}
|
||||
ended := func(w world, caller, id string) GatedEnded { return w.said[GatedEndedSubject("wiper", caller, id)] }
|
||||
|
||||
t.Run("asked, run once on the Approve, its end said to its caller", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, err := hold(t, w, call("n1.caller", `{"dir":"scratch"}`), true)
|
||||
if err != nil || h == nil || h.Asked == "" || h.Call == "" {
|
||||
t.Fatalf("held %+v, %v", h, err)
|
||||
}
|
||||
if e, _ := w.state.Get("call." + h.Call); e == nil {
|
||||
t.Fatal("no call kept")
|
||||
}
|
||||
w.router.choose(h.Asked, OptionApprove, now)
|
||||
w.g.Decided(h.Asked)
|
||||
w.g.Decided(h.Asked) // heard twice: run once
|
||||
end := ended(w, "n1.caller", h.Call)
|
||||
if *w.ran != 1 || end.Outcome != OutcomeRan || end.Answer != "wiped" || !strings.Contains(end.Approval, h.Asked) || *w.recs != 1 {
|
||||
t.Fatalf("ran %d, recorded %d, end %+v", *w.ran, *w.recs, end)
|
||||
}
|
||||
if e, _ := w.state.Get("claimed." + h.Asked); e == nil {
|
||||
t.Error("no claim kept")
|
||||
}
|
||||
a, _ := w.g.Approvals("someone.else")
|
||||
if len(a.Calls) != 1 || a.Calls[0].Arguments != nil || a.Calls[0].State != OutcomeRan || len(a.Acts) != 1 || len(a.Claims) != 1 {
|
||||
t.Errorf("approvals %+v", a)
|
||||
}
|
||||
if a, _ := w.g.Approvals("n1.caller"); string(a.Calls[0].Arguments) != `{"dir":"scratch"}` {
|
||||
t.Errorf("its caller reads %s", a.Calls[0].Arguments)
|
||||
}
|
||||
})
|
||||
t.Run("a second call of the act takes the same ask, asked anew after the first runs", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h1, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
now = now.Add(time.Second)
|
||||
_, h2, _ := hold(t, w, call("n1.b", `{}`), true)
|
||||
if h1.Asked != h2.Asked || h1.Call == h2.Call {
|
||||
t.Fatalf("asks %s and %s", h1.Asked, h2.Asked)
|
||||
}
|
||||
w.router.choose(h1.Asked, OptionApprove, now)
|
||||
w.g.Decided(h1.Asked)
|
||||
if *w.ran != 1 || ended(w, "n1.a", h1.Call).Outcome != OutcomeRan || ended(w, "n1.b", h2.Call).Outcome != "" {
|
||||
t.Fatalf("ran %d; ends %v", *w.ran, w.said)
|
||||
}
|
||||
e, _ := w.state.Get("call." + h2.Call)
|
||||
var second WaitingCall
|
||||
_ = json.Unmarshal(e.Value, &second)
|
||||
if second.Ask == h1.Asked || len(w.router.asks) != 2 {
|
||||
t.Errorf("the second call waits on %s (asks %d)", second.Ask, len(w.router.asks))
|
||||
}
|
||||
})
|
||||
t.Run("a decline runs nothing; a call within 10 minutes ends declined unasked", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
w.router.choose(h.Asked, OptionDecline, now)
|
||||
w.g.Decided(h.Asked)
|
||||
if *w.ran != 0 || ended(w, "n1.a", h.Call).Outcome != OutcomeDeclined {
|
||||
t.Fatalf("ran %d, end %+v", *w.ran, ended(w, "n1.a", h.Call))
|
||||
}
|
||||
if _, h2, err := hold(t, w, call("n1.a", `{}`), true); err == nil || h2 != nil || !strings.Contains(err.Error(), "declined") || len(w.router.asks) != 1 {
|
||||
t.Errorf("a call within 10 minutes: %v, %d asks", err, len(w.router.asks))
|
||||
}
|
||||
})
|
||||
t.Run("heard 31 minutes after the choice, it ends expired", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
w.router.choose(h.Asked, OptionApprove, now.Add(-31*time.Minute))
|
||||
w.g.Decided(h.Asked)
|
||||
if *w.ran != 0 || ended(w, "n1.a", h.Call).Outcome != OutcomeExpired {
|
||||
t.Fatalf("ran %d, end %+v", *w.ran, ended(w, "n1.a", h.Call))
|
||||
}
|
||||
})
|
||||
t.Run("a secret-shaped argument refused, nothing kept", func(t *testing.T) {
|
||||
w := setup()
|
||||
if _, h, err := hold(t, w, call("n1.a", `{"token":"ghp_0123456789abcdefghijklmnopqrstuvwxyzAB"}`), true); err == nil || h != nil {
|
||||
t.Fatalf("held %+v, %v", h, err)
|
||||
}
|
||||
if keys, _ := w.state.Keys(); len(keys) != 0 || len(w.router.asks) != 0 || len(w.bounds.ttl) != 0 {
|
||||
t.Errorf("kept %v, asked %d", keys, len(w.router.asks))
|
||||
}
|
||||
})
|
||||
t.Run("a call with no wait refused with the ask, and run when made again after the Approve", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, err := hold(t, w, call("n1.a", `{}`), false)
|
||||
if err == nil || h != nil || len(w.router.asks) != 1 || !strings.Contains(err.Error(), w.router.asks[0].ID) {
|
||||
t.Fatalf("first: %v", err)
|
||||
}
|
||||
w.router.choose(w.router.asks[0].ID, OptionApprove, now)
|
||||
answer, _, err := hold(t, w, call("n1.a", `{}`), false)
|
||||
if a, ok := answer.(Answered); err != nil || !ok || a.Answer != "wiped" || *w.ran != 1 {
|
||||
t.Fatalf("again: %+v, %v", answer, err)
|
||||
}
|
||||
})
|
||||
t.Run("a restart between the ask and the Approve runs the waiting call once", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
w.router.choose(h.Asked, OptionApprove, now)
|
||||
again := newGate(w.state, w.router, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
|
||||
if err := again.StartHolding(); err != nil || *w.ran != 1 || ended(w, "n1.a", h.Call).Outcome != OutcomeRan {
|
||||
t.Fatalf("start: %v, ran %d", err, *w.ran)
|
||||
}
|
||||
_ = again.StartHolding()
|
||||
if *w.ran != 1 {
|
||||
t.Errorf("ran %d after a second start", *w.ran)
|
||||
}
|
||||
})
|
||||
t.Run("past its ask's 60 minutes with no decided, gate-unanswered on its bound key's end", func(t *testing.T) {
|
||||
// A-255: a bound key with the ask's 60 minutes to live in the state approvals, never a clock wait.
|
||||
w := setup()
|
||||
if err := w.g.StartHolding(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
key := "unanswered." + h.Call
|
||||
if ttl := w.bounds.ttl[key]; ttl != AskLasts {
|
||||
t.Fatalf("bound %s kept for %s (%v)", key, ttl, w.bounds.ttl)
|
||||
}
|
||||
if w.finds.raised[KindGateUnanswered+" "+h.Call] {
|
||||
t.Fatal("raised before its bound ended")
|
||||
}
|
||||
w.bounds.expire(key)
|
||||
if !w.finds.raised[KindGateUnanswered+" "+h.Call] {
|
||||
t.Errorf("not raised on its bound's end: %v", w.finds.raised)
|
||||
}
|
||||
w.router.choose(h.Asked, OptionDecline, now)
|
||||
w.g.Decided(h.Asked)
|
||||
if w.finds.raised[KindGateUnanswered+" "+h.Call] {
|
||||
t.Error("not cleared once its end is said")
|
||||
}
|
||||
})
|
||||
t.Run("a call whose bound ended while the holder was away is judged at its start", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
w.bounds.watched = nil
|
||||
w.bounds.expire("unanswered." + h.Call) // the bus ended it with nobody watching
|
||||
now = now.Add(AskLasts + time.Minute)
|
||||
defer func() { now = now.Add(-AskLasts - time.Minute) }()
|
||||
again := newGate(w.state, w.router, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
|
||||
if err := again.StartHolding(); err != nil || !w.finds.raised[KindGateUnanswered+" "+h.Call] {
|
||||
t.Errorf("start: %v, raised %v", err, w.finds.raised)
|
||||
}
|
||||
})
|
||||
t.Run("two instances of the holder raise one ask and run once", func(t *testing.T) {
|
||||
// A-255: a process holder's act names no node.
|
||||
w := setup()
|
||||
other := New(Deps{Node: "n2", Runner: "holder", Holding: true, Build: "fp1", State: w.state, Channel: w.router,
|
||||
Findings: w.finds, Bounds: w.bounds, Now: func() time.Time { return now }, Go: func(f func()) { f() },
|
||||
Say: func(subject string, body any, id string) error { w.said[subject] = body.(GatedEnded); return nil }})
|
||||
other.Serves("wiper", "wipe", func(string, json.RawMessage) (any, error) { *w.ran++; return "wiped", nil })
|
||||
_, h1, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
_, h2, err := other.Hold(call("n2.b", `{}`), true, nil)
|
||||
if err != nil || h2 == nil || h2.Asked != h1.Asked || len(w.router.asks) != 1 {
|
||||
t.Fatalf("second instance: %+v, %v; asks %d", h2, err, len(w.router.asks))
|
||||
}
|
||||
if strings.Contains(w.router.asks[0].About, "n1") {
|
||||
t.Errorf("the ask binds %s", w.router.asks[0].About)
|
||||
}
|
||||
var first WaitingCall
|
||||
e, _ := w.state.Get("call." + h1.Call)
|
||||
_ = json.Unmarshal(e.Value, &first)
|
||||
if _, ok := first.Act["node"]; ok {
|
||||
t.Errorf("the act names a node: %v", first.Act)
|
||||
}
|
||||
w.router.choose(h1.Asked, OptionApprove, now)
|
||||
w.g.Decided(h1.Asked)
|
||||
other.Decided(h1.Asked)
|
||||
if *w.ran != 1 {
|
||||
t.Errorf("ran %d", *w.ran)
|
||||
}
|
||||
})
|
||||
t.Run("a state that cannot be read refuses every marked call", func(t *testing.T) {
|
||||
w := setup()
|
||||
w.state.fail = fmt.Errorf("the bus is away")
|
||||
if _, h, err := hold(t, w, call("n1.a", `{}`), true); err == nil || h != nil || *w.ran != 0 {
|
||||
t.Fatalf("held %+v, %v", h, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,172 +0,0 @@
|
||||
// Package approval is the one gate on a call that needs the operator's approval (design 46 7-approval-mark; design 47 G
|
||||
// §677, lead answer A-224): the tool runner's and a module's process's seat server's. It holds the marks it reads from the
|
||||
// catalog half, the act it digests, the ask it raises on the seat operator-channel under the marked module's name, the judging of the same call again on the router's record, the claim, the recording in the
|
||||
// hand-act log, and the refusals when a part of the check is missing (§5, §8 to §20). No module's name is written here.
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buscompose"
|
||||
"git.novox.be/novox/mesh-sdk/go/seatdef"
|
||||
)
|
||||
|
||||
// Catalog is what the marks are read from: the catalog half the runner follows (design 46 7-approval-mark §5, design 47 G
|
||||
// §196, §443) and the seat definitions it carries (§2).
|
||||
type Catalog interface {
|
||||
// Ready is nil once the catalog half's first view is read, else says what it waits for.
|
||||
Ready() error
|
||||
// Revision is the catalog revision the marks are read at, as the runner names it.
|
||||
Revision() string
|
||||
// Served is the entry of the build a bundle serves, by the fingerprint its composed environment carries; false when
|
||||
// none is read (not yet, or not found), with why.
|
||||
Served(fingerprint string) (Entry, bool, string)
|
||||
// Newest is the module's entry at the newest revision; false when the catalog half lists none.
|
||||
Newest(module string) (Entry, bool)
|
||||
// Definitions are the seat definitions the catalog half carries, by seat.
|
||||
Definitions() map[string]seatdef.Definition
|
||||
// Entries are the newest entry of every module in every followed branch's half, a marked wait's marks (lead answer
|
||||
// A-168).
|
||||
Entries() []Entry
|
||||
}
|
||||
|
||||
// Marks is what one module's calls are gated by: its marked tools and seat verbs, and the subjects of its marked events and
|
||||
// restricted streams, each with its source (§18: its module's entry in the catalog half, or its seat's definition).
|
||||
type Marks struct {
|
||||
Module string
|
||||
// Tools are its marked tools by name, Verbs its marked seat verbs as `<seat>.<verb>`, Subjects the bus subjects of its
|
||||
// marked events and restricted streams (design 34 every-event §33); each to its source.
|
||||
Tools, Verbs, Subjects map[string]string
|
||||
// Revision is the catalog revision read; Served the served entry's fingerprint, empty when it could not be read.
|
||||
Revision, Served string
|
||||
// Unread says why the served entry could not be read: every marked call is then refused (§5, §19).
|
||||
Unread string
|
||||
}
|
||||
|
||||
// Marked says whether a tool or a seat verb of the module needs the operator's approval, and its source.
|
||||
func (m Marks) Marked(seat, name string) (bool, string) {
|
||||
if seat == "" {
|
||||
src, ok := m.Tools[name]
|
||||
return ok, src
|
||||
}
|
||||
src, ok := m.Verbs[seat+"."+name]
|
||||
return ok, src
|
||||
}
|
||||
|
||||
// Any says the module marks anything at all.
|
||||
func (m Marks) Any() bool { return len(m.Tools)+len(m.Verbs)+len(m.Subjects) > 0 }
|
||||
|
||||
// marked is what an entry's manifest says of its marks: `needs-approval`, its events and its streams.
|
||||
type marked struct {
|
||||
NeedsApproval []string `json:"needs-approval"`
|
||||
Emits []string `json:"emits"`
|
||||
// Streams are read for their events and whether each carries a `grant`, and no other word, so a word added to a
|
||||
// stream's declaration later never hides its marks.
|
||||
Streams []struct {
|
||||
Events []string `json:"events"`
|
||||
Grant json.RawMessage `json:"grant"`
|
||||
} `json:"streams"`
|
||||
}
|
||||
|
||||
// Derive is a module's marks from its served entry and its newest one, unioned while they differ so a mark is never lost
|
||||
// before or after the module moves (§5, ADR 0286 §3), and the seat definitions' verb marks for every verb an entry serves,
|
||||
// which no holder unmarks (§2, §3). fingerprint is the build its bundle serves; empty, or an entry not read, leaves the
|
||||
// marks of the newest entry and the seats, with Unread saying why.
|
||||
func Derive(c Catalog, module, fingerprint string) Marks {
|
||||
m := Marks{Module: module, Tools: map[string]string{}, Verbs: map[string]string{}, Subjects: map[string]string{},
|
||||
Revision: c.Revision()}
|
||||
defs := c.Definitions()
|
||||
var entries []Entry
|
||||
if fingerprint == "" {
|
||||
m.Unread = "its bundle's environment carries no fingerprint, so the build it serves cannot be read"
|
||||
} else if e, ok, why := c.Served(fingerprint); ok {
|
||||
entries, m.Served = append(entries, e), fingerprint
|
||||
} else {
|
||||
m.Unread = fmt.Sprintf("the catalog half's entry of %s cannot be read: %s", fingerprint, why)
|
||||
}
|
||||
if e, ok := c.Newest(module); ok && (len(entries) == 0 || e.Fingerprint != entries[0].Fingerprint) {
|
||||
entries = append(entries, e)
|
||||
}
|
||||
for _, e := range entries {
|
||||
m.add(e, defs)
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// add takes one entry's marks into m: its `needs-approval` (a tool, a verb it serves or an event it declares), the seat
|
||||
// definitions' marks of every verb it serves, and every stream it declares with a `grant`, each with its source.
|
||||
func (m *Marks) add(e Entry, defs map[string]seatdef.Definition) {
|
||||
source := "entry " + e.Module + " " + orNone(e.Fingerprint)
|
||||
var said marked
|
||||
_ = json.Unmarshal(e.Manifest, &said)
|
||||
serves := map[string]bool{}
|
||||
for _, sv := range e.Serves {
|
||||
serves[sv] = true
|
||||
seat, verb, _ := strings.Cut(sv, ".")
|
||||
for _, v := range defs[seat].Verbs {
|
||||
if v.Name == verb && v.NeedsApproval {
|
||||
m.Verbs[sv] = "seat " + seat
|
||||
}
|
||||
}
|
||||
}
|
||||
events := map[string]bool{}
|
||||
for _, ev := range said.Emits {
|
||||
events[ev] = true
|
||||
}
|
||||
for _, s := range said.Streams {
|
||||
for _, ev := range s.Events {
|
||||
events[ev] = true
|
||||
}
|
||||
}
|
||||
for _, name := range said.NeedsApproval {
|
||||
switch {
|
||||
case serves[name]:
|
||||
if _, bySeat := m.Verbs[name]; !bySeat {
|
||||
m.Verbs[name] = source
|
||||
}
|
||||
case events[name]:
|
||||
for _, s := range eventSubjects(e.Module, name) {
|
||||
m.Subjects[s] = source
|
||||
}
|
||||
default:
|
||||
m.Tools[name] = source
|
||||
}
|
||||
}
|
||||
for _, s := range said.Streams {
|
||||
if len(s.Grant) == 0 || string(s.Grant) == "null" {
|
||||
continue
|
||||
}
|
||||
wire := make([]string, 0, len(s.Events))
|
||||
for _, ev := range s.Events {
|
||||
wire = append(wire, buscompose.WirePattern(ev))
|
||||
}
|
||||
for _, sub := range (buscompose.ModuleStream{Module: e.Module, Events: wire}).Subjects() {
|
||||
m.Subjects[sub] = source
|
||||
if !strings.HasSuffix(sub, ">") {
|
||||
m.Subjects[sub+".>"] = source
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// eventSubjects are a marked event's subjects on the bus (design 34 every-event §33): said on the module's events and on its
|
||||
// own stream, each also with every subject below it.
|
||||
func eventSubjects(module, event string) []string {
|
||||
ev := buscompose.WirePattern(event)
|
||||
out := []string{"mesh.mod." + module + ".event." + ev, buscompose.ModuleStreamSubject(module, ev)}
|
||||
if !strings.HasSuffix(ev, ">") {
|
||||
out = append(out, out[0]+".>", out[1]+".>")
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
func orNone(s string) string {
|
||||
if s == "" {
|
||||
return "none"
|
||||
}
|
||||
return s
|
||||
}
|
||||
@@ -1,255 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/keys"
|
||||
)
|
||||
|
||||
// The bounds of a run's recording (§13, §16; lead answers A-157, A-158).
|
||||
const (
|
||||
UnansweredAfter = 5 * time.Minute
|
||||
KeptFor = 48 * time.Hour
|
||||
SaidBefore = time.Hour
|
||||
)
|
||||
|
||||
// Bounds is the runner's key-ttl state `bound` (lead answers A-157, A-158): a key created with its own time to live, which
|
||||
// the bus removes when it passes, every watch hearing the removal as an expiry, so the gate's waits need no timer.
|
||||
type Bounds interface {
|
||||
CreateTTL(key string, value any, ttl time.Duration) error
|
||||
Delete(key string) error
|
||||
// Watch hands expired every key under the pattern whose time to live passed, until the stop is called.
|
||||
Watch(pattern string, expired func(key string)) (stop func(), err error)
|
||||
}
|
||||
|
||||
// Answered is a call run on the operator's approval: the tool's own answer and the line naming the approval (§16), in the
|
||||
// form a process holder's `gated-ended` says it (design 47 G §677; lead answer A-158).
|
||||
type Answered struct {
|
||||
Answer any `json:"answer"`
|
||||
Approval string `json:"approval"`
|
||||
}
|
||||
|
||||
// The bound keys of a run (lead answer A-158; §13).
|
||||
func (g *Gate) unansweredKey(id string) string { return g.d.Node + ".unanswered." + id }
|
||||
func (g *Gate) unrecordedKey(id string) string { return g.d.Node + ".unrecorded." + id }
|
||||
|
||||
// ran is a call run on the operator's approval: answered with the tool's answer and the approval's line, and its warrant
|
||||
// recorded in the hand-act log (§16).
|
||||
func (g *Gate) ran(c Call, id string, rec *Record, answer any, err error) (any, error) {
|
||||
channel := ""
|
||||
at := ""
|
||||
if w := rec.Warrant; w != nil {
|
||||
channel, at = w.Channel, w.At
|
||||
}
|
||||
if channel == "" {
|
||||
channel = strings.Join(rec.OfferedOn, ", ")
|
||||
}
|
||||
line := fmt.Sprintf("run on the operator's approval via %s at %s, ask %s", orNone(channel), at, id)
|
||||
g.d.Logf("[approval] %s on %s %s", c.Address, g.d.Node, line)
|
||||
g.bindRecording(c.Module, id)
|
||||
g.d.Go(func() { g.recordRun(c.Module, id) })
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w (%s)", err, line)
|
||||
}
|
||||
return Answered{Answer: answer, Approval: line}, nil
|
||||
}
|
||||
|
||||
// bindRecording keeps the run's two bound keys: one that raises `call-unanswered` when `warranted` has not answered within
|
||||
// five minutes, one that says an unrecorded run an hour before its key's 48 hours end (§13, §16).
|
||||
func (g *Gate) bindRecording(module, id string) {
|
||||
if g.d.Bounds == nil || g.d.Holding { // a process holder's bound keys are its waiting calls' alone (A-255)
|
||||
return
|
||||
}
|
||||
for key, ttl := range map[string]time.Duration{g.unansweredKey(id): UnansweredAfter, g.unrecordedKey(id): KeptFor - SaidBefore} {
|
||||
if err := g.d.Bounds.CreateTTL(key, map[string]string{"asker": module, "ask": id}, ttl); err != nil {
|
||||
g.d.Logf("[approval] the bound %s of the run of ask %s is not kept: %v", key, id, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// recordRun calls `mesh-hand-acts.warranted` for one acted ask, each try a kept call open while the ask lacks `recorded`
|
||||
// (§16, A-157), and on its answer writes `recorded` and clears what waited on it. One recording per ask at a time.
|
||||
func (g *Gate) recordRun(module, id string) {
|
||||
if !g.recording(id, true) {
|
||||
return
|
||||
}
|
||||
defer g.recording(id, false)
|
||||
if g.d.Warranted == nil {
|
||||
g.d.Logf("[approval] the run of ask %s is not recorded: the tool runner reaches no hand-act log", id)
|
||||
return
|
||||
}
|
||||
err := g.d.Warranted(module, id, func() (bool, error) { return g.unrecorded(id) })
|
||||
g.judged(id) // made again: its answer, or the bound of its key, judges it from now
|
||||
if err != nil {
|
||||
g.d.Logf("[approval] mesh-hand-acts.warranted has not answered for ask %s (%v): asked again when the hand-act seat's "+
|
||||
"holder is heard again and at the next start", id, err)
|
||||
return
|
||||
}
|
||||
if err := g.markRecorded(id); err != nil {
|
||||
g.d.Logf("[approval] ask %s is recorded in the hand-act log and its mark recorded is not kept: %v", id, err)
|
||||
return
|
||||
}
|
||||
if g.d.Bounds != nil {
|
||||
_ = g.d.Bounds.Delete(g.unansweredKey(id))
|
||||
_ = g.d.Bounds.Delete(g.unrecordedKey(id))
|
||||
}
|
||||
g.finding(id, KindCallUnanswered, false, "mesh-hand-acts.warranted answered for ask "+id, "")
|
||||
}
|
||||
|
||||
// recording claims or releases an ask's recording: false when one runs already.
|
||||
func (g *Gate) recording(id string, start bool) bool {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
if g.inFlight == nil {
|
||||
g.inFlight = map[string]bool{}
|
||||
}
|
||||
if start {
|
||||
if g.inFlight[id] {
|
||||
return false
|
||||
}
|
||||
g.inFlight[id] = true
|
||||
return true
|
||||
}
|
||||
delete(g.inFlight, id)
|
||||
return true
|
||||
}
|
||||
|
||||
// kept is the ask the client keeps under its id, with its revision; nil when there is none.
|
||||
func (g *Gate) kept(id string) (*kept, uint64, error) {
|
||||
e, err := g.d.State.Get(g.askKey(id))
|
||||
if err != nil || e == nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
var k kept
|
||||
if err := json.Unmarshal(e.Value, &k); err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
return &k, e.Revision, nil
|
||||
}
|
||||
|
||||
// unrecorded says the ask is acted on and its run not yet recorded: the key that keeps `warranted` open (§16).
|
||||
func (g *Gate) unrecorded(id string) (bool, error) {
|
||||
k, _, err := g.kept(id)
|
||||
if err != nil || k == nil {
|
||||
return false, err
|
||||
}
|
||||
return k.Acted && !k.Recorded, nil
|
||||
}
|
||||
|
||||
// markRecorded writes `recorded` on the kept ask by compare-and-set, again when another write came first.
|
||||
func (g *Gate) markRecorded(id string) error {
|
||||
for {
|
||||
k, rev, err := g.kept(id)
|
||||
if err != nil || k == nil {
|
||||
return fmt.Errorf("the kept ask cannot be read: %v", err)
|
||||
}
|
||||
if k.Recorded {
|
||||
return nil
|
||||
}
|
||||
k.Recorded = true
|
||||
if _, err := g.d.State.Update(g.askKey(id), k, rev); err == nil {
|
||||
return nil
|
||||
} else if !isChanged(err) {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Recover calls `warranted` again for every acted, unrecorded ask this machine's keys hold within their 48 hours (§16): at
|
||||
// each start, and when the hand-act seat's holder is heard again (A-157).
|
||||
func (g *Gate) Recover() {
|
||||
if g.d.State == nil {
|
||||
return
|
||||
}
|
||||
keys, err := g.d.State.Keys()
|
||||
if err != nil {
|
||||
g.d.Logf("[approval] the acted, unrecorded asks cannot be read (%v): read again at the next start or holder heard", err)
|
||||
return
|
||||
}
|
||||
prefix := g.prefix() + "ask."
|
||||
for _, key := range keys {
|
||||
if !strings.HasPrefix(key, prefix) {
|
||||
continue
|
||||
}
|
||||
id := strings.TrimPrefix(key, prefix)
|
||||
k, _, err := g.kept(id)
|
||||
if err != nil || k == nil || !k.Acted || k.Recorded || g.d.Now().Sub(k.Asked) > KeptFor {
|
||||
continue
|
||||
}
|
||||
module := moduleOf(k)
|
||||
g.pend(id)
|
||||
g.d.Go(func() { g.recordRun(module, id) })
|
||||
}
|
||||
}
|
||||
|
||||
// moduleOf is the module an ask was asked under: the warrant's asker.
|
||||
func moduleOf(k *kept) string {
|
||||
var w struct {
|
||||
Asker string `json:"asker"`
|
||||
}
|
||||
if k.Answer != nil {
|
||||
_ = json.Unmarshal(*k.Answer, &w)
|
||||
}
|
||||
return w.Asker
|
||||
}
|
||||
|
||||
// WatchBounds hears the run's bound keys expire (A-157): `call-unanswered` raised when `warranted` has not answered within
|
||||
// five minutes; an unrecorded run said in the log an hour before its key's 48 hours end (§13).
|
||||
func (g *Gate) WatchBounds() (func(), error) {
|
||||
if g.d.Bounds == nil {
|
||||
return func() {}, nil
|
||||
}
|
||||
return g.d.Bounds.Watch(g.d.Node+".**", func(key string) {
|
||||
if key == g.unreadableKey() {
|
||||
g.unreadableExpired()
|
||||
return
|
||||
}
|
||||
rest := strings.TrimPrefix(key, g.d.Node+".")
|
||||
kind, id, _ := strings.Cut(rest, ".")
|
||||
if open, err := g.unrecorded(id); err != nil || !open {
|
||||
return
|
||||
}
|
||||
switch kind {
|
||||
case "unanswered":
|
||||
g.finding(id, KindCallUnanswered, true, fmt.Sprintf("mesh-hand-acts.warranted has not answered for ask %s, run on the "+
|
||||
"operator's approval on %s, within %s: the run is not yet in the hand-act log", id, g.d.Node, UnansweredAfter),
|
||||
"a run's hand-act is unrecorded on "+g.d.Node)
|
||||
case "unrecorded":
|
||||
g.d.Logf("[approval] ERROR the run of ask %s on %s is still unrecorded an hour before its key's 48 hours end", id, g.d.Node)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Pender is the held writer's pending (views.HeldWriter's Pend and Judged), by a finding's kind and id: the findings it
|
||||
// cannot judge yet since its start, named in its `findings-stated` (design 47 G §384).
|
||||
type Pender interface {
|
||||
Pend(kind, id string, since time.Time, bound time.Duration)
|
||||
Judged(kind, id string)
|
||||
}
|
||||
|
||||
// pend names an acted, unrecorded ask's `call-unanswered` in the runner's `findings-stated` until its `warranted` is made
|
||||
// again (§16; design 47 G §384), its bound the five minutes the finding waits.
|
||||
func (g *Gate) pend(id string) {
|
||||
if p, ok := g.d.Findings.(Pender); ok {
|
||||
p.Pend(KindCallUnanswered, g.findingID(id), g.d.Now().UTC(), UnansweredAfter)
|
||||
}
|
||||
}
|
||||
|
||||
// judged ends that pending.
|
||||
func (g *Gate) judged(id string) {
|
||||
if p, ok := g.d.Findings.(Pender); ok {
|
||||
p.Judged(KindCallUnanswered, g.findingID(id))
|
||||
}
|
||||
}
|
||||
|
||||
// findingID is the id of a finding about what on this machine, `<node>.<what>` escaped as a key's name is (lead answer
|
||||
// A-154).
|
||||
func (g *Gate) findingID(what string) string { return keys.EscapeName(g.d.Node + "." + what) }
|
||||
|
||||
// isChanged says a compare-and-set lost to another write.
|
||||
func isChanged(err error) bool { return errors.Is(err, stdio.ErrChanged) }
|
||||
@@ -1,176 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
)
|
||||
|
||||
// Run is a call through the gate (§20): an unmarked call runs; a marked one runs only on the operator's approval of that
|
||||
// one call, judged on the router's record and claimed once (§14, §15), and is otherwise answered as an error saying why
|
||||
// (§12, §17, §19). run is called at most once.
|
||||
func (g *Gate) Run(c Call, run func() (any, error)) (any, error) {
|
||||
m, err := g.marksOf(c)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if missing := g.unservedOf(c.Module); len(missing) > 0 {
|
||||
return nil, fmt.Errorf("Not run: %s's marks name %s, which its bundle on %s does not register, so none of its tools is "+
|
||||
"served here (design 46 7-approval-mark §4)", c.Module, strings.Join(missing, ", "), g.d.Node)
|
||||
}
|
||||
marked, _ := m.Marked(c.Seat, c.Name)
|
||||
since := g.markedSince(c, marked, m.Revision)
|
||||
if !marked {
|
||||
return run()
|
||||
}
|
||||
if c.Own {
|
||||
return nil, g.refused(c, "", "it is one of the tool runner's own tools, which no ask covers")
|
||||
}
|
||||
if m.Unread != "" {
|
||||
return nil, g.refused(c, "", m.Unread)
|
||||
}
|
||||
if b, ok := g.d.State.(Binding); g.d.State == nil || ok && !b.Bound() {
|
||||
return nil, g.refused(c, "", "the tool runner's membership binds no state "+StateName+" to keep its asks in")
|
||||
}
|
||||
if why := refusedUnasked(c.Args); why != "" {
|
||||
return nil, g.refused(c, "", why)
|
||||
}
|
||||
if why := g.held(); why != "" {
|
||||
return nil, g.refused(c, "", why)
|
||||
}
|
||||
act, err := ActOf(c, g.d.Node)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
digest, err := operatorask.ActDigest(act)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
named, err := g.d.State.Get(g.actKey(digest))
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
return nil, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
|
||||
}
|
||||
g.stateRead()
|
||||
if named == nil {
|
||||
return nil, g.raise(c, act, digest, since)
|
||||
}
|
||||
var id string
|
||||
if err := json.Unmarshal(named.Value, &id); err != nil || id == "" {
|
||||
return nil, g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName))
|
||||
}
|
||||
return g.judge(c, act, digest, id, since, run)
|
||||
}
|
||||
|
||||
// Binding is a Store that says whether the runner's membership binds the state now (§13).
|
||||
type Binding interface{ Bound() bool }
|
||||
|
||||
// marksOf is the module's marks as the catalog half has them now; before its first view, the call waits for it within
|
||||
// RecordWithin and is then judged, refused naming mesh_first_views when it is still unread (§19).
|
||||
func (g *Gate) marksOf(c Call) (Marks, error) {
|
||||
if g.d.Catalog == nil {
|
||||
return Marks{}, g.refused(c, "", "the tool runner reads no catalog half, so its marks are not read")
|
||||
}
|
||||
if err := g.d.Catalog.Ready(); err != nil {
|
||||
if g.d.Wait != nil {
|
||||
err = g.d.Wait(RecordWithin)
|
||||
}
|
||||
if err != nil {
|
||||
return Marks{}, fmt.Errorf("Not run: the marks of %s are not read yet (%v); mesh_first_views says what the tool runner "+
|
||||
"waits for", c.Module, err)
|
||||
}
|
||||
}
|
||||
m := Derive(g.d.Catalog, c.Module, c.Fingerprint)
|
||||
g.checkServed(m)
|
||||
return m, nil
|
||||
}
|
||||
|
||||
func (g *Gate) unservedOf(module string) []string {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
return g.unserved[module]
|
||||
}
|
||||
|
||||
// markedSince is the revision a call was marked at when the same caller last called it unmarked (§12), empty otherwise.
|
||||
func (g *Gate) markedSince(c Call, marked bool, revision string) string {
|
||||
key := c.Caller + " " + c.Address
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
if g.unmarkedAt == nil {
|
||||
g.unmarkedAt = map[string]bool{}
|
||||
}
|
||||
was := g.unmarkedAt[key]
|
||||
if marked {
|
||||
delete(g.unmarkedAt, key)
|
||||
if was {
|
||||
return revision
|
||||
}
|
||||
return ""
|
||||
}
|
||||
g.unmarkedAt[key] = true
|
||||
return ""
|
||||
}
|
||||
|
||||
// judge is the same call made again, judged on the router's own record of its act's newest ask (§14 to §17).
|
||||
func (g *Gate) judge(c Call, act operatorask.Act, digest, id, since string, run func() (any, error)) (any, error) {
|
||||
rec, err := g.record(c.Module, id)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error())
|
||||
}
|
||||
if rec == nil {
|
||||
return nil, g.firstAnswer(c, id, digest, since) // §17: not yet taken by the router
|
||||
}
|
||||
again := func(why string) (any, error) {
|
||||
err := g.raise(c, act, digest, since)
|
||||
return nil, fmt.Errorf("The ask %s %s; asked again. %w", id, why, err)
|
||||
}
|
||||
switch rec.State {
|
||||
case "open", "held-back":
|
||||
return nil, fmt.Errorf("Not run: still waiting for the operator's answer to ask %s (fingerprint %s); it expires at %s",
|
||||
id, Fingerprint(digest), rec.Expires)
|
||||
case "refused":
|
||||
return nil, g.refused(c, digest, "the router refused its ask "+id+": "+orNone(rec.Answer))
|
||||
case "chosen":
|
||||
default: // expired, cancelled, replaced: each asks again (§15, §17)
|
||||
return again("ended " + rec.State)
|
||||
}
|
||||
w := rec.Warrant
|
||||
if w == nil {
|
||||
return nil, g.refused(c, digest, "the router's record of its ask "+id+" is chosen and carries no warrant")
|
||||
}
|
||||
at, _ := parseTime(w.At)
|
||||
if w.Option == OptionDecline {
|
||||
if until := at.Add(DeclinedFor); g.d.Now().Before(until) {
|
||||
return nil, fmt.Errorf("Not run: the operator declined ask %s; the same call is not asked again until %s",
|
||||
id, formatTime(until))
|
||||
}
|
||||
return again("was declined more than 10 minutes ago")
|
||||
}
|
||||
if g.d.Now().After(at.Add(RunWithin)) {
|
||||
return again("was approved too long ago (more than 30 minutes)")
|
||||
}
|
||||
var answer any
|
||||
var ran error
|
||||
h, err := g.d.Channel.Claim(c.Module, *w, act, func() { answer, ran = run() })
|
||||
switch {
|
||||
case h == ClaimDone:
|
||||
return again("was already run")
|
||||
case h == ClaimRefused:
|
||||
return nil, g.refused(c, digest, "its warrant does not hold: "+errText(err))
|
||||
case h == "" && err != nil:
|
||||
g.stateFailed(err)
|
||||
return nil, g.refused(c, digest, "its state "+StateName+" cannot be written: "+err.Error())
|
||||
case err != nil: // the act about to run is not the one the option bound: nothing runs, the warrant stays used
|
||||
return nil, g.refused(c, digest, err.Error())
|
||||
}
|
||||
return g.ran(c, id, rec, answer, ran)
|
||||
}
|
||||
|
||||
func errText(err error) string {
|
||||
if err == nil {
|
||||
return "no reason given"
|
||||
}
|
||||
return err.Error()
|
||||
}
|
||||
@@ -1,172 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The mesh MCP server's mesh_approvals reads and steers what the gate asked on this machine (§18).
|
||||
|
||||
// Listed is one ask of the last day as mesh_approvals lists it: its state and its act's fingerprint.
|
||||
type Listed struct {
|
||||
Ask string `json:"ask"`
|
||||
Address string `json:"address"`
|
||||
Module string `json:"module"`
|
||||
Asked time.Time `json:"asked"`
|
||||
State string `json:"state"`
|
||||
Fingerprint string `json:"fingerprint"`
|
||||
}
|
||||
|
||||
// Whole is one ask read whole: its arguments and the act's digest.
|
||||
type Whole struct {
|
||||
Listed
|
||||
Digest string `json:"digest"`
|
||||
Act map[string]string `json:"act"`
|
||||
Arguments json.RawMessage `json:"arguments"`
|
||||
Recorded bool `json:"recorded"`
|
||||
}
|
||||
|
||||
// stateOf is a kept ask's state as the runner knows it.
|
||||
func stateOf(acted, recorded bool, ended string) string {
|
||||
switch {
|
||||
case acted && recorded:
|
||||
return "run, recorded"
|
||||
case acted:
|
||||
return "run, not yet recorded"
|
||||
case ended != "":
|
||||
return ended
|
||||
}
|
||||
return "asked"
|
||||
}
|
||||
|
||||
// List is the calls asked on this machine in the last day, newest first (§18).
|
||||
func (g *Gate) List() ([]Listed, error) {
|
||||
if g.d.State == nil {
|
||||
return nil, fmt.Errorf("the tool runner's membership binds no state %s", StateName)
|
||||
}
|
||||
keys, err := g.d.State.Keys()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
prefix := g.d.Node + ".ask."
|
||||
out := []Listed{}
|
||||
for _, key := range keys {
|
||||
if !strings.HasPrefix(key, prefix) {
|
||||
continue
|
||||
}
|
||||
w, err := g.Read(strings.TrimPrefix(key, prefix))
|
||||
if err != nil || g.d.Now().Sub(w.Asked) > 24*time.Hour {
|
||||
continue
|
||||
}
|
||||
out = append(out, w.Listed)
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].Asked.After(out[j].Asked) })
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Read is one ask whole: its arguments and the act's digest (§18).
|
||||
func (g *Gate) Read(id string) (Whole, error) {
|
||||
if g.d.State == nil {
|
||||
return Whole{}, fmt.Errorf("the tool runner's membership binds no state %s", StateName)
|
||||
}
|
||||
k, _, err := g.kept(id)
|
||||
if err != nil {
|
||||
return Whole{}, err
|
||||
}
|
||||
if k == nil {
|
||||
return Whole{}, fmt.Errorf("no ask %s is kept on %s", id, g.d.Node)
|
||||
}
|
||||
module := k.Act["module"]
|
||||
if module == "" {
|
||||
module = moduleOf(k)
|
||||
}
|
||||
var ask struct {
|
||||
About string `json:"about"`
|
||||
}
|
||||
_ = json.Unmarshal(k.Ask, &ask)
|
||||
return Whole{Listed: Listed{Ask: id, Address: k.Act["address"], Module: module, Asked: k.Asked,
|
||||
State: stateOf(k.Acted, k.Recorded, k.Ended), Fingerprint: Fingerprint(ask.About)},
|
||||
Digest: ask.About, Act: k.Act, Arguments: k.Arguments, Recorded: k.Recorded}, nil
|
||||
}
|
||||
|
||||
// Applied are the marks the gate applies on this machine, each module's marked tools and verbs with their source and the
|
||||
// catalog revision read (§18).
|
||||
func (g *Gate) Applied() []Marks {
|
||||
g.mu.Lock()
|
||||
modules := make([]string, 0, len(g.served))
|
||||
for m := range g.served {
|
||||
modules = append(modules, m)
|
||||
}
|
||||
g.mu.Unlock()
|
||||
sort.Strings(modules)
|
||||
var out []Marks
|
||||
if g.d.Catalog == nil {
|
||||
return out
|
||||
}
|
||||
for _, m := range modules {
|
||||
if marks := Derive(g.d.Catalog, m, g.servedFingerprint(m)); marks.Any() {
|
||||
out = append(out, marks)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (g *Gate) servedFingerprint(module string) string {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
return g.served[module]
|
||||
}
|
||||
|
||||
// Describe says whether a tool or a seat verb of a module is marked, and its source, for mesh_describe (§18): the served
|
||||
// build's marks where this machine serves the module, else the newest entry's and its seat's.
|
||||
func (g *Gate) Describe(module, seat, name string) (bool, string) {
|
||||
if g.d.Catalog == nil || g.d.Catalog.Ready() != nil {
|
||||
return false, ""
|
||||
}
|
||||
if seat != "" && module == "" {
|
||||
for s, d := range g.d.Catalog.Definitions() {
|
||||
if s != seat {
|
||||
continue
|
||||
}
|
||||
for _, v := range d.Verbs {
|
||||
if v.Name == name && v.NeedsApproval {
|
||||
return true, "seat " + seat
|
||||
}
|
||||
}
|
||||
}
|
||||
return false, ""
|
||||
}
|
||||
return Derive(g.d.Catalog, module, g.servedFingerprint(module)).Marked(seat, name)
|
||||
}
|
||||
|
||||
// Again asks `warranted` again for one acted, unrecorded ask, by hand, in the kept-call form of §16 (§18); the caller
|
||||
// records the hand act first.
|
||||
func (g *Gate) Again(id string) error {
|
||||
w, err := g.Read(id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !strings.HasPrefix(w.State, "run, not yet") {
|
||||
return fmt.Errorf("the ask %s is %s: only a run not yet recorded is asked again", id, w.State)
|
||||
}
|
||||
g.recordRun(w.Module, id)
|
||||
if again, _ := g.Read(id); !again.Recorded {
|
||||
return fmt.Errorf("mesh-hand-acts.warranted has not answered for ask %s yet", id)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Cancel takes an open ask back through operator-channel's client (§18); the caller records the hand act first.
|
||||
func (g *Gate) Cancel(id string) error {
|
||||
w, err := g.Read(id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if w.State != "asked" {
|
||||
return fmt.Errorf("the ask %s is %s, not open", id, w.State)
|
||||
}
|
||||
return g.d.Channel.Cancel(w.Module, id)
|
||||
}
|
||||
@@ -1,290 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buscompose"
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
"git.novox.be/novox/mesh-sdk/go/outward"
|
||||
)
|
||||
|
||||
// Wait is one mesh_wait call as the gate judges it (design 34 every-event §34 to §36): the event names given, the subjects
|
||||
// they became (after `from`), the filter and the timeout.
|
||||
type Wait struct {
|
||||
Names []string
|
||||
From string
|
||||
Subjects []string
|
||||
Filter map[string]any
|
||||
Timeout time.Duration
|
||||
}
|
||||
|
||||
// Waited is a wait through the gate (every-event §34 to §36). An unmarked wait is heard at once by hear, given how long it
|
||||
// may wait. A marked one is heard only on the operator's warrant for that one wait, its timeout running from then; with
|
||||
// none it subscribes to nothing, its ask raised once under the runner's own module's name, and is answered as an error at
|
||||
// its timeout, when its caller goes, or as soon as the router's record says the ask was chosen, never in a hot loop. c is
|
||||
// the call as the console makes it: its address, its tool's name, its caller and its arguments as given.
|
||||
func (g *Gate) Waited(ctx context.Context, c Call, w Wait, hear func(within time.Duration) (any, error)) (any, error) {
|
||||
deadline := g.d.Now().Add(w.Timeout)
|
||||
// A wait from a seat is never marked, and a seat's event never is (§32, §34).
|
||||
if w.From == "seat" || !outsideSeats(w.Subjects) {
|
||||
return hear(w.Timeout)
|
||||
}
|
||||
if g.d.Catalog == nil {
|
||||
return nil, g.holdRefused(ctx, deadline, "this wait is refused: the tool runner reads no catalog half, so the marks are not read")
|
||||
}
|
||||
if err := g.d.Catalog.Ready(); err != nil {
|
||||
// Before the marks are read nothing is asked: the wait waits for the first view within its own timeout and is
|
||||
// then judged, and is refused at its timeout when it does not come (§34).
|
||||
if g.d.Wait != nil {
|
||||
err = g.waitCatalog(ctx, w.Timeout)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, g.holdRefused(ctx, deadline, fmt.Sprintf("this wait is refused at its timeout: the marks are not yet read (%v); "+
|
||||
"mesh_first_views says what the tool runner waits for", err))
|
||||
}
|
||||
}
|
||||
if !overlapsAny(w.Subjects, g.waitMarks()) {
|
||||
return hear(deadline.Sub(g.d.Now()))
|
||||
}
|
||||
c.Module, c.Seat, c.Fingerprint = g.d.Runner, "", g.d.Build
|
||||
if c.Fingerprint == "" {
|
||||
return nil, g.holdRefused(ctx, deadline, "this wait needs the operator's approval, and the tool runner's own fingerprint is "+
|
||||
"unknown (its MESH_FINGERPRINT was not set at its start), so no warrant can bind its build")
|
||||
}
|
||||
if b, ok := g.d.State.(Binding); g.d.State == nil || ok && !b.Bound() {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", "the tool runner's membership binds no state "+StateName+" to keep its asks in").Error())
|
||||
}
|
||||
if why := refusedUnasked(c.Args); why != "" {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", why).Error())
|
||||
}
|
||||
if why := g.held(); why != "" {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", why).Error())
|
||||
}
|
||||
act, err := ActOf(c, g.d.Node)
|
||||
if err != nil {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", err.Error()).Error())
|
||||
}
|
||||
digest, err := operatorask.ActDigest(act)
|
||||
if err != nil {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", err.Error()).Error())
|
||||
}
|
||||
named, err := g.d.State.Get(g.actKey(digest))
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error()).Error())
|
||||
}
|
||||
g.stateRead()
|
||||
anew := func() (any, error) {
|
||||
id := newID()
|
||||
if err := g.keepAsk(c, act, digest, g.askForWait(c, w, digest, id)); err != nil {
|
||||
return nil, g.holdRefused(ctx, deadline, err.Error())
|
||||
}
|
||||
return nil, g.holdAsk(ctx, deadline, c.Module, id, digest, true)
|
||||
}
|
||||
if named == nil {
|
||||
return anew()
|
||||
}
|
||||
var id string
|
||||
if err := json.Unmarshal(named.Value, &id); err != nil || id == "" {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName)).Error())
|
||||
}
|
||||
rec, err := g.record(c.Module, id)
|
||||
switch {
|
||||
case err != nil:
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error()).Error())
|
||||
case rec == nil, rec.State == "open", rec.State == "held-back":
|
||||
return nil, g.holdAsk(ctx, deadline, c.Module, id, digest, false)
|
||||
case rec.State == "refused":
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router refused its ask "+id+": "+orNone(rec.Answer)).Error())
|
||||
case rec.State != "chosen":
|
||||
return anew() // expired, cancelled, replaced: asked again (§15, §17)
|
||||
}
|
||||
wt := rec.Warrant
|
||||
if wt == nil {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router's record of its ask "+id+" is chosen and carries no warrant").Error())
|
||||
}
|
||||
at, _ := parseTime(wt.At)
|
||||
if wt.Option == OptionDecline {
|
||||
if g.d.Now().Before(at.Add(DeclinedFor)) {
|
||||
return nil, g.holdRefused(ctx, deadline, declinedWords(id, at))
|
||||
}
|
||||
return anew()
|
||||
}
|
||||
if g.d.Now().After(at.Add(RunWithin)) {
|
||||
return anew()
|
||||
}
|
||||
var answer any
|
||||
var ran error
|
||||
// On the warrant it subscribes, and the wait's timeout runs from then (§35).
|
||||
h, err := g.d.Channel.Claim(c.Module, *wt, act, func() { answer, ran = hear(w.Timeout) })
|
||||
switch {
|
||||
case h == ClaimDone:
|
||||
return anew() // already run: one warrant is one wait (§35)
|
||||
case h == ClaimRefused:
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its warrant does not hold: "+errText(err)).Error())
|
||||
case h == "" && err != nil:
|
||||
g.stateFailed(err)
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its state "+StateName+" cannot be written: "+err.Error()).Error())
|
||||
case err != nil:
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, err.Error()).Error())
|
||||
}
|
||||
return g.ran(c, id, rec, answer, ran)
|
||||
}
|
||||
|
||||
// outsideSeats says a subject is not a seat's: only such a subject can be marked (§32, §34).
|
||||
func outsideSeats(subjects []string) bool {
|
||||
for _, s := range subjects {
|
||||
if !strings.HasPrefix(s, "mesh.seat.") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// waitMarks are the marked subjects of every module the catalog half lists (§33): the newest entry of every followed
|
||||
// branch, and the served entry of each module this runner serves (lead answer A-168), so a mark is never lost.
|
||||
func (g *Gate) waitMarks() map[string]string {
|
||||
m := Marks{Tools: map[string]string{}, Verbs: map[string]string{}, Subjects: map[string]string{}}
|
||||
defs := g.d.Catalog.Definitions()
|
||||
for _, e := range g.d.Catalog.Entries() {
|
||||
m.add(e, defs)
|
||||
}
|
||||
g.mu.Lock()
|
||||
served := make(map[string]string, len(g.served))
|
||||
for module, fp := range g.served {
|
||||
served[module] = fp
|
||||
}
|
||||
g.mu.Unlock()
|
||||
for _, fp := range served {
|
||||
if e, ok, _ := g.d.Catalog.Served(fp); ok {
|
||||
m.add(e, defs)
|
||||
}
|
||||
}
|
||||
return m.Subjects
|
||||
}
|
||||
|
||||
// overlapsAny says a subject a wait subscribes overlaps a marked subject (§34).
|
||||
func overlapsAny(subjects []string, marked map[string]string) bool {
|
||||
for _, s := range subjects {
|
||||
for mk := range marked {
|
||||
if buscompose.Overlaps(s, mk) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// waitCatalog waits for the catalog half's first view within the bound, or until the caller goes.
|
||||
func (g *Gate) waitCatalog(ctx context.Context, within time.Duration) error {
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- g.d.Wait(within) }()
|
||||
select {
|
||||
case err := <-done:
|
||||
return err
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
}
|
||||
}
|
||||
|
||||
// pause waits until the deadline or the caller's going.
|
||||
func (g *Gate) pause(ctx context.Context, deadline time.Time) {
|
||||
if left := deadline.Sub(g.d.Now()); left > 0 {
|
||||
select {
|
||||
case <-g.d.After(left):
|
||||
case <-ctx.Done():
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// holdRefused is a refused marked wait, said at its timeout or when its caller goes, never at once, so a watcher's loop
|
||||
// is never a hot one (§36).
|
||||
func (g *Gate) holdRefused(ctx context.Context, deadline time.Time, words string) error {
|
||||
g.pause(ctx, deadline)
|
||||
if !strings.HasPrefix(words, "Not run") {
|
||||
words = "Not run: " + words
|
||||
}
|
||||
return fmt.Errorf("%s", words)
|
||||
}
|
||||
|
||||
// holdAsk holds a marked wait whose ask is open, subscribing to nothing but its ask's `decided` (§36): it answers, as an
|
||||
// error, at its timeout, when its caller goes, or once the router's record says the ask was chosen, the event only waking it.
|
||||
func (g *Gate) holdAsk(ctx context.Context, deadline time.Time, module, id, digest string, first bool) error {
|
||||
for {
|
||||
left := deadline.Sub(g.d.Now())
|
||||
heard := false
|
||||
if left > 0 && ctx.Err() == nil {
|
||||
var err error
|
||||
if g.d.Hear != nil {
|
||||
heard, err = g.d.Hear(ctx, []string{g.d.Channel.Decided(module)}, left, func(body json.RawMessage) bool {
|
||||
var w struct {
|
||||
Ask string `json:"ask"`
|
||||
}
|
||||
return json.Unmarshal(body, &w) == nil && w.Ask == id
|
||||
})
|
||||
}
|
||||
if g.d.Hear == nil || err != nil {
|
||||
g.pause(ctx, deadline)
|
||||
}
|
||||
}
|
||||
rec, _ := g.record(module, id)
|
||||
if heard && (rec == nil || rec.State != "chosen") {
|
||||
continue // the event only wakes it: the record says nothing chosen yet
|
||||
}
|
||||
return g.waitWords(rec, id, digest, first)
|
||||
}
|
||||
}
|
||||
|
||||
// waitWords are §12's and §17's words for a marked wait not run, "this wait" for "the call" (§36).
|
||||
func (g *Gate) waitWords(rec *Record, id, digest string, first bool) error {
|
||||
switch {
|
||||
case rec == nil:
|
||||
return fmt.Errorf("Not run, asked; not yet taken by the router (ask %s, fingerprint %s); make this wait again later", id, Fingerprint(digest))
|
||||
case rec.State == "chosen" && rec.Warrant != nil && rec.Warrant.Option == OptionDecline:
|
||||
at, _ := parseTime(rec.Warrant.At)
|
||||
return fmt.Errorf("%s", declinedWords(id, at))
|
||||
case rec.State == "chosen", first:
|
||||
return fmt.Errorf("Not run: this wait needs the operator's approval. Asked on %s at %s (ask %s, fingerprint %s); make this wait "+
|
||||
"again within 30 minutes of the approval; the ask expires at %s", orNone(strings.Join(rec.OfferedOn, ", ")), rec.Opened,
|
||||
id, Fingerprint(digest), rec.Expires)
|
||||
case rec.State == "open", rec.State == "held-back":
|
||||
return fmt.Errorf("Not run: still waiting for the operator's answer to ask %s (fingerprint %s); it expires at %s",
|
||||
id, Fingerprint(digest), rec.Expires)
|
||||
}
|
||||
return fmt.Errorf("Not run: the ask %s ended %s; make this wait again to ask anew", id, rec.State)
|
||||
}
|
||||
|
||||
func declinedWords(id string, at time.Time) string {
|
||||
return fmt.Sprintf("Not run: the operator declined ask %s; this wait is not asked again until %s", id,
|
||||
formatTime(at.Add(DeclinedFor)))
|
||||
}
|
||||
|
||||
// askForWait is a marked wait's ask (§35): "Wait on <names> on <node>?", shortened as §10 says, and an explanation naming
|
||||
// every event name one per line, the filter, and the bound as the timeout in minutes, then the act's fingerprint.
|
||||
func (g *Gate) askForWait(c Call, w Wait, digest, id string) Ask {
|
||||
a := g.askFor(c, digest, id)
|
||||
filter, whole := "none", true
|
||||
if len(w.Filter) > 0 {
|
||||
raw, _ := json.Marshal(w.Filter)
|
||||
filter = string(raw)
|
||||
if r, ok := outward.Check(filter); !ok {
|
||||
filter, whole = shapeOf(r), false
|
||||
}
|
||||
}
|
||||
a.Headline = headlineOf("Wait on", strings.Join(w.Names, ", "), g.d.Node, whole)
|
||||
lines := append([]string{"Wait on " + g.d.Node + ", called through the mesh's tools, which cannot tell who called, for:"}, w.Names...)
|
||||
lines = append(lines, "Filter: "+filter, fmt.Sprintf("Timeout: %s minutes", minutes(w.Timeout)), "Fingerprint "+Fingerprint(digest))
|
||||
a.Explanation = strings.Join(lines, "\n")
|
||||
a.Options[0].Does = "the same wait, made again within 30 minutes, subscribes once, its timeout running from then"
|
||||
a.Options[1].Does = "nothing is heard; the same wait is not asked again for 10 minutes"
|
||||
return a
|
||||
}
|
||||
|
||||
// minutes is a timeout in minutes, as few digits as say it.
|
||||
func minutes(d time.Duration) string {
|
||||
return strings.TrimSuffix(strings.TrimRight(fmt.Sprintf("%.2f", d.Minutes()), "0"), ".")
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
// Package asker is a module's client for asking a person (novox/hq ADR 0259 §10): it publishes an ask under
|
||||
// the module's own name, cancels it, reads its state from the router's record, and hands the module each
|
||||
// warrant once — only one for an ask it made, for an option it offered, at that option's level, before it
|
||||
// expired. The module then acts with its own grants.
|
||||
//
|
||||
// It keeps what it asked, and which warrants it acted on, in a state of the module's own (declared in its
|
||||
// manifest, by default `asked`): the ask is written before it is published, and a warrant is marked acted on
|
||||
// by compare-and-set before the module's handler runs, so a redelivery, or a second instance, acts once.
|
||||
package asker
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||
)
|
||||
|
||||
// Client asks as one module.
|
||||
type Client struct {
|
||||
// Module is the asking module's name: the one the bus grants its asks under.
|
||||
Module string
|
||||
// State is the module's own state the asks are kept in.
|
||||
State Store
|
||||
// Now is the clock; nil is time.Now.
|
||||
Now func() time.Time
|
||||
// Publish, Record and Subscribe reach the bus; nil is the stdio SDK's, through the module's runtime.
|
||||
Publish func(subject string, body any, id string) error
|
||||
Record func(key string) (*stdio.StateEntry, error)
|
||||
Subscribe func(pattern string, handle func(stdio.Envelope) error) error
|
||||
}
|
||||
|
||||
// Store is where the asks are kept: stdio.State(name) is one.
|
||||
type Store interface {
|
||||
Get(key string) (*stdio.StateEntry, error)
|
||||
Create(key string, value any) (uint64, error)
|
||||
Update(key string, value any, revision uint64) (uint64, error)
|
||||
}
|
||||
|
||||
// New is a client for the module the runtime serves, keeping its asks in the state named.
|
||||
func New(state string) *Client {
|
||||
return &Client{Module: os.Getenv("MESH_MODULE"), State: stdio.State(state)}
|
||||
}
|
||||
|
||||
func (c *Client) now() time.Time {
|
||||
if c.Now != nil {
|
||||
return c.Now()
|
||||
}
|
||||
return time.Now()
|
||||
}
|
||||
|
||||
// Kept is one ask as the module keeps it.
|
||||
type Kept struct {
|
||||
Ask asks.Ask `json:"ask"`
|
||||
Asked time.Time `json:"asked"`
|
||||
Acted bool `json:"acted,omitempty"`
|
||||
Ended asks.Outcome `json:"ended,omitempty"`
|
||||
Answer *asks.Warrant `json:"answer,omitempty"`
|
||||
}
|
||||
|
||||
func key(id string) string { return "ask." + id }
|
||||
|
||||
// Ask checks an ask, keeps it, and publishes it under the module's own name. An ask with an id already kept
|
||||
// is refused: a new question is a new id.
|
||||
func (c *Client) Ask(a asks.Ask) error {
|
||||
if err := a.Check(c.now()); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := c.State.Create(key(a.ID), Kept{Ask: a, Asked: c.now()}); err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
return fmt.Errorf("an ask %s was already made: a new question is a new id", a.ID)
|
||||
}
|
||||
return err
|
||||
}
|
||||
return c.publish(asks.AskSubject(c.Module), a, a.ID)
|
||||
}
|
||||
|
||||
// Cancel takes an ask back: the router ends it and edits every copy.
|
||||
func (c *Client) Cancel(id string) error {
|
||||
return c.publish(asks.CancelSubject(c.Module), map[string]string{"id": id}, "cancel."+id)
|
||||
}
|
||||
|
||||
func (c *Client) publish(subject string, body any, id string) error {
|
||||
if c.Publish != nil {
|
||||
return c.Publish(subject, body, id)
|
||||
}
|
||||
return stdio.SeatPublish(subject, body, id)
|
||||
}
|
||||
|
||||
// Where is an ask's state as the router keeps it: its state and its warrant, if any — read under the
|
||||
// module's own name and no other's.
|
||||
func (c *Client) Where(id string) (state string, w *asks.Warrant, err error) {
|
||||
read := c.Record
|
||||
if read == nil {
|
||||
read = func(k string) (*stdio.StateEntry, error) { return stdio.SeatRecord("", k) }
|
||||
}
|
||||
e, err := read(id)
|
||||
if err != nil || e == nil {
|
||||
return "", nil, err
|
||||
}
|
||||
var rec struct {
|
||||
State string `json:"state"`
|
||||
Warrant *asks.Warrant `json:"warrant"`
|
||||
}
|
||||
if err := json.Unmarshal(e.Value, &rec); err != nil {
|
||||
return "", nil, fmt.Errorf("the router's record of %s cannot be read: %w", id, err)
|
||||
}
|
||||
return rec.State, rec.Warrant, nil
|
||||
}
|
||||
|
||||
// Handled is what came of a warrant: acted on, refused (and why), or already done.
|
||||
type Handled string
|
||||
|
||||
const (
|
||||
Acted Handled = "acted"
|
||||
Refused Handled = "refused"
|
||||
Done Handled = "already acted on"
|
||||
Recorded Handled = "ended without a choice"
|
||||
)
|
||||
|
||||
// Take hands act a warrant once, after holding it to the ask it answers. act performs the option with the
|
||||
// module's own grants; its error is said and the warrant stays acted on (a person's decision is not
|
||||
// performed twice; the module says the failure in its own record).
|
||||
func (c *Client) Take(w asks.Warrant, act func(asks.Option, asks.Warrant) error) (Handled, error) {
|
||||
if w.Asker != c.Module {
|
||||
return Refused, fmt.Errorf("a warrant for %s reached %s", w.Asker, c.Module)
|
||||
}
|
||||
e, err := c.State.Get(key(w.Ask))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if e == nil {
|
||||
return Refused, fmt.Errorf("%s holds no ask %s", c.Module, w.Ask)
|
||||
}
|
||||
var k Kept
|
||||
if err := json.Unmarshal(e.Value, &k); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if k.Acted || k.Ended != "" {
|
||||
return Done, nil
|
||||
}
|
||||
if w.Outcome != asks.OutcomeChosen {
|
||||
k.Ended = w.Outcome
|
||||
k.Answer = &w
|
||||
if _, err := c.State.Update(key(w.Ask), k, e.Revision); err != nil && !errors.Is(err, stdio.ErrChanged) {
|
||||
return "", err
|
||||
}
|
||||
return Recorded, nil
|
||||
}
|
||||
o, err := w.For(c.Module, k.Ask)
|
||||
if err != nil {
|
||||
return Refused, err
|
||||
}
|
||||
// Marked before acting: of two deliveries, or two instances, only the one whose write stands acts.
|
||||
k.Acted, k.Ended, k.Answer = true, asks.OutcomeChosen, &w
|
||||
if _, err := c.State.Update(key(w.Ask), k, e.Revision); err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
return Done, nil
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
return Acted, act(o, w)
|
||||
}
|
||||
|
||||
// Listen hands every warrant for this module to Take, with act. It asks the runtime for the module's
|
||||
// events matching the router's `decided` subject for this module.
|
||||
func (c *Client) Listen(act func(asks.Option, asks.Warrant) error, said func(Handled, asks.Warrant, error)) error {
|
||||
subscribe := c.Subscribe
|
||||
if subscribe == nil {
|
||||
subscribe = stdio.Subscribe
|
||||
}
|
||||
return subscribe(asks.Seat+".decided."+c.Module, func(e stdio.Envelope) error {
|
||||
if !strings.HasPrefix(e.Key, asks.Seat+".decided.") {
|
||||
return nil
|
||||
}
|
||||
var w asks.Warrant
|
||||
if err := json.Unmarshal(e.Body, &w); err != nil {
|
||||
if said != nil {
|
||||
said(Refused, w, fmt.Errorf("a warrant could not be read: %w", err))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
h, err := c.Take(w, act)
|
||||
if h == "" && err != nil {
|
||||
return err // the state could not be reached: offered again
|
||||
}
|
||||
if said != nil {
|
||||
said(h, w, err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
package asker
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||
)
|
||||
|
||||
type mem struct {
|
||||
vals map[string]json.RawMessage
|
||||
revs map[string]uint64
|
||||
n uint64
|
||||
}
|
||||
|
||||
func (m *mem) Get(k string) (*stdio.StateEntry, error) {
|
||||
v, ok := m.vals[k]
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
return &stdio.StateEntry{Key: k, Value: v, Revision: m.revs[k]}, nil
|
||||
}
|
||||
func (m *mem) Create(k string, v any) (uint64, error) {
|
||||
if _, ok := m.vals[k]; ok {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
return m.put(k, v)
|
||||
}
|
||||
func (m *mem) Update(k string, v any, rev uint64) (uint64, error) {
|
||||
if rev == 0 || m.revs[k] != rev {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
return m.put(k, v)
|
||||
}
|
||||
func (m *mem) put(k string, v any) (uint64, error) {
|
||||
raw, _ := json.Marshal(v)
|
||||
m.n++
|
||||
m.vals[k], m.revs[k] = raw, m.n
|
||||
return m.n, nil
|
||||
}
|
||||
|
||||
var now = time.Date(2026, 10, 8, 14, 0, 0, 0, time.UTC)
|
||||
|
||||
func client() (*Client, map[string][]string) {
|
||||
sent := map[string][]string{}
|
||||
c := &Client{Module: "mesh-delivery", State: &mem{vals: map[string]json.RawMessage{}, revs: map[string]uint64{}},
|
||||
Now: func() time.Time { return now },
|
||||
Publish: func(subject string, body any, id string) error {
|
||||
sent[subject] = append(sent[subject], id)
|
||||
return nil
|
||||
}}
|
||||
return c, sent
|
||||
}
|
||||
|
||||
// releaseHQ is the act the ask's one option binds.
|
||||
var releaseHQ = map[string]string{"verb": "release", "delivery": "hq"}
|
||||
|
||||
func held() asks.Ask {
|
||||
binds, _ := asks.ActDigest(releaseHQ)
|
||||
return asks.Ask{ID: "d-1", Headline: "Delivery of hq held", Explanation: "Needs you: release it, or stop it.",
|
||||
Who: asks.Operator, Expires: now.Add(time.Hour), OnExpiry: "the delivery stays held",
|
||||
Options: []asks.Option{{ID: "release", Label: "Release", Does: "it goes on", Level: asks.Approve, Binds: binds}}}
|
||||
}
|
||||
|
||||
func warrant() asks.Warrant {
|
||||
return asks.Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: asks.OutcomeChosen, Option: "release",
|
||||
Label: "Release", Level: asks.Approve, At: now.Add(time.Minute), AskDigest: held().Digest(),
|
||||
By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"}}
|
||||
}
|
||||
|
||||
func TestAnAskIsKeptThenPublishedUnderTheModulesOwnName(t *testing.T) {
|
||||
c, sent := client()
|
||||
if err := c.Ask(held()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := sent[asks.AskSubject("mesh-delivery")]; len(got) != 1 || got[0] != "d-1" {
|
||||
t.Fatalf("%v", sent)
|
||||
}
|
||||
if err := c.Ask(held()); err == nil || !strings.Contains(err.Error(), "already made") {
|
||||
t.Errorf("the same id was asked twice: %v", err)
|
||||
}
|
||||
bad := held()
|
||||
bad.Who = "agent"
|
||||
if err := c.Ask(bad); err == nil {
|
||||
t.Error("an ask the contract refuses was published")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAWarrantIsActedOnOnceAndOnlyForItsOwnAsk(t *testing.T) {
|
||||
c, _ := client()
|
||||
_ = c.Ask(held())
|
||||
acted := 0
|
||||
act := func(o asks.Option, _ asks.Warrant) error {
|
||||
if err := o.Performs(releaseHQ); err != nil {
|
||||
t.Errorf("the bound act: %v", err)
|
||||
}
|
||||
acted++
|
||||
return nil
|
||||
}
|
||||
if h, err := c.Take(warrant(), act); h != Acted || err != nil || acted != 1 {
|
||||
t.Fatalf("%s %v %d", h, err, acted)
|
||||
}
|
||||
if h, _ := c.Take(warrant(), act); h != Done || acted != 1 {
|
||||
t.Errorf("a redelivery was acted on: %s", h)
|
||||
}
|
||||
for why, change := range map[string]func(*asks.Warrant){
|
||||
"another asker": func(w *asks.Warrant) { w.Asker = "mesh-controller" },
|
||||
"an ask not made": func(w *asks.Warrant) { w.Ask = "d-9" },
|
||||
"another level": func(w *asks.Warrant) { w.Level = asks.Acknowledge },
|
||||
"an option not offered": func(w *asks.Warrant) { w.Option = "delete" },
|
||||
"after its expiry": func(w *asks.Warrant) { w.At = now.Add(2 * time.Hour) },
|
||||
"another ask's digest": func(w *asks.Warrant) { w.AskDigest = "sha256:00" },
|
||||
} {
|
||||
c2, _ := client()
|
||||
_ = c2.Ask(held())
|
||||
w := warrant()
|
||||
change(&w)
|
||||
if h, _ := c2.Take(w, act); h == Acted {
|
||||
t.Errorf("a warrant from %s was acted on", why)
|
||||
}
|
||||
}
|
||||
if acted != 1 {
|
||||
t.Errorf("acted %d times", acted)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskEndedWithoutAChoiceIsRecordedAndNothingDone(t *testing.T) {
|
||||
c, _ := client()
|
||||
_ = c.Ask(held())
|
||||
w := warrant()
|
||||
w.Outcome, w.Option, w.By = asks.OutcomeExpired, "", nil
|
||||
if h, err := c.Take(w, func(asks.Option, asks.Warrant) error { t.Fatal("acted"); return nil }); h != Recorded || err != nil {
|
||||
t.Fatalf("%s %v", h, err)
|
||||
}
|
||||
if h, _ := c.Take(warrant(), func(asks.Option, asks.Warrant) error { t.Fatal("acted after it ended"); return nil }); h != Done {
|
||||
t.Errorf("%s", h)
|
||||
}
|
||||
}
|
||||
+578
@@ -0,0 +1,578 @@
|
||||
// Package asks is the contract of asking a person and answering on a channel (novox/hq ADR 0259): the
|
||||
// shapes an asker, the router and a channel exchange on the bus, and the subjects they travel on. No
|
||||
// transport and no channel's service: an asker publishes an Ask under its own name and acts on the Warrant
|
||||
// it hears; the router holds the ask, sends channels a Message, and judges the Choice a channel says; a
|
||||
// channel shows a Message and says what was chosen and by whom, as its service authenticated it.
|
||||
package asks
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The seats (novox/hq ADR 0259 §3).
|
||||
const (
|
||||
// Seat is held by the router: an ask and its cancel are its accepts, a warrant its event, each named by
|
||||
// the asker.
|
||||
Seat = "operator-channel"
|
||||
// ChannelSeat is the kinded bench a channel holds to show and say: its accepts carry the kind.
|
||||
ChannelSeat = "channel"
|
||||
// IntakeSeat is the kinded bench a channel holds to say what was chosen: its events and proofs carry
|
||||
// the kind.
|
||||
IntakeSeat = "intake"
|
||||
)
|
||||
|
||||
// AskSubject is where an asker publishes an ask, CancelSubject its cancel, and DecidedSubject where it
|
||||
// hears the warrant, or the ask's end without one.
|
||||
func AskSubject(asker string) string { return "mesh.seat." + Seat + ".accept.ask." + asker }
|
||||
func CancelSubject(asker string) string { return "mesh.seat." + Seat + ".accept.cancel." + asker }
|
||||
func DecidedSubject(asker string) string { return "mesh.seat." + Seat + ".event.decided." + asker }
|
||||
|
||||
// The work a channel takes, on ChannelSubject.
|
||||
const (
|
||||
Show = "show" // a message offering answers
|
||||
Edit = "edit" // a message shown before, replaced
|
||||
Send = "send" // a message offering nothing
|
||||
)
|
||||
|
||||
// ChannelSubject is where the router sends a channel of a kind its work.
|
||||
func ChannelSubject(verb, kind string) string {
|
||||
return "mesh.seat." + ChannelSeat + ".accept." + verb + "." + kind
|
||||
}
|
||||
|
||||
// What a channel says, on IntakeSubject.
|
||||
const (
|
||||
Chosen = "choice" // a button tapped
|
||||
Link = "link" // somebody asked to be linked as the operator
|
||||
)
|
||||
|
||||
// IntakeSubject is where a channel of a kind says what arrived.
|
||||
func IntakeSubject(what, kind string) string {
|
||||
return "mesh.seat." + IntakeSeat + ".event." + what + "." + kind
|
||||
}
|
||||
|
||||
// CodeProof is the one proof verb: a code the operator typed, carried by request and reply, never kept.
|
||||
const CodeProof = "code"
|
||||
|
||||
// ProofSubject is where a channel of a kind asks a proof.
|
||||
func ProofSubject(verb, kind string) string {
|
||||
return "mesh.seat." + IntakeSeat + ".proof." + verb + "." + kind
|
||||
}
|
||||
|
||||
// A Level is how much proof an option's answer needs (novox/hq ADR 0234 §8, the glossary's assurance level).
|
||||
type Level string
|
||||
|
||||
const (
|
||||
// Acknowledge performs only what any granted principal may already do: silencing, details. No proof.
|
||||
Acknowledge Level = "acknowledge"
|
||||
// Approve needs one proof: a verified sender (a linked account, linked an hour or more) or a code.
|
||||
Approve Level = "approve"
|
||||
// Destroy needs two proofs, one of them a code.
|
||||
Destroy Level = "destroy"
|
||||
)
|
||||
|
||||
// Rank orders the levels; an unknown level ranks above every known one, so it is never taken as less.
|
||||
func (l Level) Rank() int {
|
||||
switch l {
|
||||
case Acknowledge:
|
||||
return 0
|
||||
case Approve:
|
||||
return 1
|
||||
case Destroy:
|
||||
return 2
|
||||
}
|
||||
return 3
|
||||
}
|
||||
|
||||
// Operator is the one role an ask may be answered by today.
|
||||
const Operator = "operator"
|
||||
|
||||
// Option is one answer an ask offers: a label for the button, what it does in plain words, its level.
|
||||
type Option struct {
|
||||
ID string `json:"id"`
|
||||
Label string `json:"label"`
|
||||
Does string `json:"does"`
|
||||
Level Level `json:"level"`
|
||||
// Binds is the digest of exactly what the asker performs when this option is chosen — the verb, the
|
||||
// machine and every argument — as ActDigest gives it. It travels in the ask, so the router's warrant,
|
||||
// which names the ask's digest, names it too: a warrant then authorises that act and no other, and an
|
||||
// asker whose record of the act changed after it asked finds the digests differ and does nothing.
|
||||
// Required on an option above acknowledge.
|
||||
Binds string `json:"binds,omitempty"`
|
||||
}
|
||||
|
||||
// An Act is what an option binds: named fields, each a string — the verb, the machine, the level, and each
|
||||
// argument under a name of its own ("arg.delivery"). Flat on purpose: its digest is over these names and
|
||||
// values alone, never over how a language happens to encode a struct.
|
||||
type Act map[string]string
|
||||
|
||||
// ActDigest is the digest an asker puts in Option.Binds: SHA-256 over the act's canonical encoding (canonical),
|
||||
// written "sha256:<hex>". The same names and values give the same digest in any language, whatever order
|
||||
// they were set in; a field renamed, added or emptied gives another.
|
||||
func ActDigest(act Act) (string, error) {
|
||||
if len(act) == 0 {
|
||||
return "", fmt.Errorf("the act cannot be digested: it names nothing")
|
||||
}
|
||||
keys := make([]string, 0, len(act))
|
||||
for k := range act {
|
||||
if k == "" {
|
||||
return "", fmt.Errorf("the act cannot be digested: a field has no name")
|
||||
}
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
var b strings.Builder
|
||||
b.WriteString("novox.act.v1\n")
|
||||
for _, k := range keys {
|
||||
canonical(&b, k)
|
||||
canonical(&b, act[k])
|
||||
}
|
||||
sum := sha256.Sum256([]byte(b.String()))
|
||||
return "sha256:" + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
|
||||
// canonical writes one value as its length in bytes, a colon, the bytes and a newline: no value can be read
|
||||
// as another's end or start, so two different sequences of values never encode the same.
|
||||
func canonical(b *strings.Builder, v string) {
|
||||
b.WriteString(strconv.Itoa(len(v)))
|
||||
b.WriteByte(':')
|
||||
b.WriteString(v)
|
||||
b.WriteByte('\n')
|
||||
}
|
||||
|
||||
// Digest is the digest of the ask exactly as its asker published it: its id, words, options with what each
|
||||
// binds, who answers, and its expiry. The router puts it in the warrant (Warrant.AskDigest), and an asker
|
||||
// acts only on a warrant whose digest is that of the ask it keeps — so a warrant answers one ask, as the
|
||||
// person was shown it, and nothing published under the same id before or after.
|
||||
//
|
||||
// It is over the ask's named fields in a fixed order, each written canonically, and the expiry as UTC
|
||||
// RFC 3339 to the nanosecond — never over a language's encoding of the struct, so a field added to Ask
|
||||
// later changes no digest until it is added here, on purpose. Whole and Details (novox/hq issue 383) follow
|
||||
// the options only when the ask gives either: an ask without them digests as it did before they existed, so
|
||||
// a router and an asker of different builds still agree on every such ask.
|
||||
func (a Ask) Digest() string {
|
||||
var b strings.Builder
|
||||
b.WriteString("novox.ask.v1\n")
|
||||
for _, v := range []string{a.ID, a.Headline, a.Explanation, a.Who,
|
||||
a.Expires.UTC().Format(time.RFC3339Nano), a.OnExpiry, a.About, strconv.FormatBool(a.Urgent),
|
||||
strconv.Itoa(len(a.Options))} {
|
||||
canonical(&b, v)
|
||||
}
|
||||
for _, o := range a.Options {
|
||||
for _, v := range []string{o.ID, o.Label, o.Does, string(o.Level), o.Binds} {
|
||||
canonical(&b, v)
|
||||
}
|
||||
}
|
||||
// Whole and Details are written whenever either or WholeDetails is given, and WholeDetails after them whenever
|
||||
// it is given (novox/hq ADR 0347): the fields after the options are then always one of three fixed shapes —
|
||||
// none, the two, or the three — and the count of values the sequence holds tells them apart, so no two asks
|
||||
// encode the same (the review of mesh-sdk #25). An ask with none of them digests as before they existed, and
|
||||
// one without WholeDetails as before it existed.
|
||||
if a.Whole != "" || a.Details != "" || a.WholeDetails != "" {
|
||||
canonical(&b, a.Whole)
|
||||
canonical(&b, a.Details)
|
||||
}
|
||||
if a.WholeDetails != "" {
|
||||
canonical(&b, a.WholeDetails)
|
||||
}
|
||||
sum := sha256.Sum256([]byte(b.String()))
|
||||
return "sha256:" + hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
// Ask is a request for a person's word (novox/hq ADR 0259 §4).
|
||||
type Ask struct {
|
||||
// ID is the asker's own, unique to it.
|
||||
ID string `json:"id"`
|
||||
// Headline names the thing and what is wrong, in a few plain words; Explanation is what happened and
|
||||
// what it means. Both are held to the plain rule and the content rule by the router.
|
||||
Headline string `json:"headline"`
|
||||
Explanation string `json:"explanation"`
|
||||
Options []Option `json:"options"`
|
||||
// Who may answer: Operator.
|
||||
Who string `json:"who"`
|
||||
// Expires is when the ask ends unanswered; OnExpiry is what the asker then does, in words the person
|
||||
// is shown ("the delivery stays held"). An ask that authorises never defaults.
|
||||
Expires time.Time `json:"expires"`
|
||||
OnExpiry string `json:"on-expiry"`
|
||||
// About is what the ask is about (a condition's key): a newer ask about it replaces the older.
|
||||
About string `json:"about,omitempty"`
|
||||
Urgent bool `json:"urgent,omitempty"`
|
||||
// Whole is the explanation with its exact values whole, shown in place of Explanation on a channel kind
|
||||
// that proves who answers (verified-sender) and carries the ask's answers (novox/hq issue 383): what the
|
||||
// person approves — a mount point, a share, a private address — must be readable where they approve it.
|
||||
// The asker withholds a value shaped like a secret in it, and the router refuses the ask when one is left;
|
||||
// Explanation stays under the whole content rule everywhere else. Empty, Explanation is shown everywhere.
|
||||
Whole string `json:"whole,omitempty"`
|
||||
// Details are the ask's secondary words, line by line under the content rule: who proposed it, a
|
||||
// fingerprint, how to read it whole. The router shows them in the note of the ask's own message
|
||||
// (Message.Note), after what the ask's options do and when it expires — never in its body (novox/hq
|
||||
// issue 639).
|
||||
Details string `json:"details,omitempty"`
|
||||
// WholeDetails are the Details with their exact values whole, shown in place of Details where Whole is shown
|
||||
// in place of the explanation (a kind that proves who answers, given the ask's answers): every value an
|
||||
// approval sets, whole, so the person reads exactly what they approve (novox/hq issue 383). Held, like
|
||||
// Whole, to the secret rule alone; the router refuses the ask when a secret's shape is left in it. Empty,
|
||||
// Details are shown everywhere.
|
||||
WholeDetails string `json:"whole-details,omitempty"`
|
||||
}
|
||||
|
||||
// What an ask's explanation is held to, so it reads at a glance on a phone (novox/hq issue 639): a summary
|
||||
// of two or three short sentences on its first line — what changes, why, what it costs or risks — then at
|
||||
// most three points, a line each, only where they add something. Whole is held to the same. Who asked, when,
|
||||
// the fingerprint and how to read it whole are Details, never the explanation.
|
||||
const (
|
||||
SummaryLength = 300
|
||||
MostPoints = 3
|
||||
PointLength = 160
|
||||
)
|
||||
|
||||
// Glance says why an explanation does not read at a glance, or nil: its first line, the summary, longer than
|
||||
// SummaryLength; more than MostPoints lines after it; a point longer than PointLength. Blank lines are not
|
||||
// counted. An asker holds its asks to it in its own tests; the router says, without refusing, an ask that
|
||||
// does not hold to it.
|
||||
func Glance(explanation string) error {
|
||||
var lines []string
|
||||
for _, l := range strings.Split(explanation, "\n") {
|
||||
if strings.TrimSpace(l) != "" {
|
||||
lines = append(lines, strings.TrimSpace(l))
|
||||
}
|
||||
}
|
||||
var problems []string
|
||||
switch {
|
||||
case len(lines) == 0:
|
||||
problems = append(problems, "it has no summary")
|
||||
case len([]rune(lines[0])) > SummaryLength:
|
||||
problems = append(problems, fmt.Sprintf("its summary is %d characters, more than %d", len([]rune(lines[0])), SummaryLength))
|
||||
}
|
||||
if len(lines) > 1+MostPoints {
|
||||
problems = append(problems, fmt.Sprintf("it has %d points after its summary, more than %d", len(lines)-1, MostPoints))
|
||||
}
|
||||
for i, l := range lines {
|
||||
if i > 0 && len([]rune(l)) > PointLength {
|
||||
problems = append(problems, fmt.Sprintf("its point %d is %d characters, more than %d", i, len([]rune(l)), PointLength))
|
||||
}
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
return errors.New("the explanation does not read at a glance: " + strings.Join(problems, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// The bounds of an ask (novox/hq ADR 0234 §8, ADR 0259 §4).
|
||||
const (
|
||||
MostOptions = 4
|
||||
MostOpen = 3
|
||||
ApproveLasts = 24 * time.Hour
|
||||
DestroyLasts = 10 * time.Minute
|
||||
HeadlineLength = 60
|
||||
LabelLength = 24
|
||||
)
|
||||
|
||||
var usableID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$`)
|
||||
|
||||
// UsableID says whether a name can be an ask's or an option's id: a key and a subject token both.
|
||||
func UsableID(id string) bool { return usableID.MatchString(id) }
|
||||
|
||||
// Highest is the highest level among the ask's options.
|
||||
func (a Ask) Highest() Level {
|
||||
high := Acknowledge
|
||||
for _, o := range a.Options {
|
||||
if o.Level.Rank() > high.Rank() {
|
||||
high = o.Level
|
||||
}
|
||||
}
|
||||
return high
|
||||
}
|
||||
|
||||
// Option is the option of this id, or false.
|
||||
func (a Ask) Option(id string) (Option, bool) {
|
||||
for _, o := range a.Options {
|
||||
if o.ID == id {
|
||||
return o, true
|
||||
}
|
||||
}
|
||||
return Option{}, false
|
||||
}
|
||||
|
||||
// Check is what an ask is held to before anything is shown: every refusal, in words its asker can act on.
|
||||
func (a Ask) Check(now time.Time) error {
|
||||
var problems []string
|
||||
say := func(format string, args ...any) { problems = append(problems, fmt.Sprintf(format, args...)) }
|
||||
if !UsableID(a.ID) {
|
||||
say("its id %q is not letters, digits, - and _, at most 64", a.ID)
|
||||
}
|
||||
if strings.TrimSpace(a.Headline) == "" || len([]rune(a.Headline)) > HeadlineLength {
|
||||
say("its headline is empty or longer than %d characters", HeadlineLength)
|
||||
}
|
||||
if strings.TrimSpace(a.Explanation) == "" {
|
||||
say("it explains nothing")
|
||||
}
|
||||
if a.Who != Operator {
|
||||
say("it is answered by %q, and only the operator answers today", a.Who)
|
||||
}
|
||||
if len(a.Options) == 0 || len(a.Options) > MostOptions {
|
||||
say("it offers %d options, and an ask offers one to %d", len(a.Options), MostOptions)
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, o := range a.Options {
|
||||
switch {
|
||||
case !UsableID(o.ID):
|
||||
say("an option's id %q is not letters, digits, - and _", o.ID)
|
||||
case seen[o.ID]:
|
||||
say("the option %s is offered twice", o.ID)
|
||||
}
|
||||
seen[o.ID] = true
|
||||
if strings.TrimSpace(o.Label) == "" || len([]rune(o.Label)) > LabelLength {
|
||||
say("the option %s's label is empty or longer than %d characters", o.ID, LabelLength)
|
||||
}
|
||||
if strings.TrimSpace(o.Does) == "" {
|
||||
say("the option %s does not say what it does", o.ID)
|
||||
}
|
||||
if o.Level.Rank() > Destroy.Rank() {
|
||||
say("the option %s has the level %q, which is none of acknowledge, approve, destroy", o.ID, o.Level)
|
||||
}
|
||||
if o.Level != Acknowledge && !strings.HasPrefix(o.Binds, "sha256:") {
|
||||
say("the option %s authorises and does not bind what it performs (its binds is not an ActDigest)", o.ID)
|
||||
}
|
||||
}
|
||||
if !a.Expires.After(now) {
|
||||
say("it expires before it is asked")
|
||||
}
|
||||
switch a.Highest() {
|
||||
case Approve:
|
||||
if a.Expires.After(now.Add(ApproveLasts)) {
|
||||
say("an ask that approves lasts at most %s", ApproveLasts)
|
||||
}
|
||||
case Destroy:
|
||||
if a.Expires.After(now.Add(DestroyLasts)) {
|
||||
say("an ask that destroys lasts at most %s", DestroyLasts)
|
||||
}
|
||||
}
|
||||
if a.Highest() != Acknowledge && strings.TrimSpace(a.OnExpiry) == "" {
|
||||
say("it does not say what happens when nobody answers, and an ask that authorises never defaults")
|
||||
}
|
||||
if a.About != "" && strings.ContainsAny(a.About, " \n") {
|
||||
say("what it is about is a key, without spaces")
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
return errors.New("the ask is refused: " + strings.Join(problems, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Outcome is how an ask ended.
|
||||
type Outcome string
|
||||
|
||||
const (
|
||||
OutcomeChosen Outcome = "chosen" // a person chose an option: a warrant
|
||||
OutcomeExpired Outcome = "expired" // nobody answered in time
|
||||
OutcomeCancelled Outcome = "cancelled" // its asker took it back
|
||||
OutcomeReplaced Outcome = "replaced" // a newer ask about the same thing replaced it
|
||||
OutcomeRefused Outcome = "refused" // it was never shown: Words says why
|
||||
)
|
||||
|
||||
// Person is who chose, as the router verified them.
|
||||
type Person struct {
|
||||
// Who is the role: Operator.
|
||||
Who string `json:"who"`
|
||||
// Kind is the channel kind they answered on, Identity their account on that service, Display the name
|
||||
// the service shows, and Verified how the router knew it was them.
|
||||
Kind string `json:"kind"`
|
||||
Identity string `json:"identity"`
|
||||
Display string `json:"display,omitempty"`
|
||||
Verified string `json:"verified"`
|
||||
}
|
||||
|
||||
// Warrant is the router's record that a person chose one option of one ask, or the ask's end without one
|
||||
// (novox/hq ADR 0259 §6). It carries no secret.
|
||||
type Warrant struct {
|
||||
Ask string `json:"ask"`
|
||||
Asker string `json:"asker"`
|
||||
About string `json:"about,omitempty"`
|
||||
Outcome Outcome `json:"outcome"`
|
||||
// Option, Label and Level are the option chosen; By who chose it, Channel the module it came through,
|
||||
// Proofs which proofs were present (P1, P2, P3).
|
||||
Option string `json:"option,omitempty"`
|
||||
Label string `json:"label,omitempty"`
|
||||
Level Level `json:"level,omitempty"`
|
||||
By *Person `json:"by,omitempty"`
|
||||
Channel string `json:"channel,omitempty"`
|
||||
Proofs []string `json:"proofs,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
// AskDigest is the digest of the ask as the router took it (Ask.Digest): the warrant answers that ask
|
||||
// alone, with the options it bound.
|
||||
AskDigest string `json:"ask-digest,omitempty"`
|
||||
// Words are why an ask ended without a choice, or what refused it.
|
||||
Words string `json:"words,omitempty"`
|
||||
}
|
||||
|
||||
// Says is the warrant in the words an asker records with its act: "the operator, via telegram (user id
|
||||
// verified), chose Release".
|
||||
func (w Warrant) Says() string {
|
||||
if w.Outcome != OutcomeChosen || w.By == nil {
|
||||
return fmt.Sprintf("no person chose: the ask %s %s", w.Ask, w.Outcome)
|
||||
}
|
||||
via := w.By.Kind
|
||||
if w.By.Verified != "" {
|
||||
via += " (" + w.By.Verified + ")"
|
||||
}
|
||||
return fmt.Sprintf("the %s, via %s, chose %s", w.By.Who, via, w.Label)
|
||||
}
|
||||
|
||||
// For checks a warrant against the ask its asker made: the same asker and ask, the same ask's digest (so the
|
||||
// same words, options and binds), a choice, an option the ask offered, at that option's level, before the
|
||||
// ask expired. An asker acts on nothing else, and then performs only what the option's Binds names.
|
||||
func (w Warrant) For(asker string, a Ask) (Option, error) {
|
||||
if w.Asker != asker || w.Ask != a.ID {
|
||||
return Option{}, fmt.Errorf("the warrant is for %s's ask %s, not %s's %s", w.Asker, w.Ask, asker, a.ID)
|
||||
}
|
||||
if w.Outcome != OutcomeChosen || w.By == nil || w.By.Who != a.Who {
|
||||
return Option{}, fmt.Errorf("the ask %s ended %s; no person chose", a.ID, w.Outcome)
|
||||
}
|
||||
if d := a.Digest(); w.AskDigest != d {
|
||||
return Option{}, fmt.Errorf("the warrant answers an ask whose digest is %q, and the ask %s kept here is %s: "+
|
||||
"it was not the ask the person was shown", w.AskDigest, a.ID, d)
|
||||
}
|
||||
o, offered := a.Option(w.Option)
|
||||
if !offered {
|
||||
return Option{}, fmt.Errorf("the ask %s offered no option %s", a.ID, w.Option)
|
||||
}
|
||||
if w.Level != o.Level {
|
||||
return Option{}, fmt.Errorf("the option %s is %s, and the warrant says %s", o.ID, o.Level, w.Level)
|
||||
}
|
||||
// A choice made after the ask expired is no answer to it, whatever the router said. An ask that says no
|
||||
// expiry, or a warrant that says no time, is no answer either: neither can be shown to be in time (the
|
||||
// confirmation review of 2026-10-09).
|
||||
if a.Expires.IsZero() {
|
||||
return Option{}, fmt.Errorf("the ask %s says no expiry, so no answer to it can be in time", a.ID)
|
||||
}
|
||||
if w.At.IsZero() {
|
||||
return Option{}, fmt.Errorf("the warrant for the ask %s says no time it was given, so it cannot be shown to be in time", a.ID)
|
||||
}
|
||||
if w.At.After(a.Expires) {
|
||||
return Option{}, fmt.Errorf("the warrant was given at %s, after the ask %s expired at %s",
|
||||
w.At.UTC().Format(time.RFC3339), a.ID, a.Expires.UTC().Format(time.RFC3339))
|
||||
}
|
||||
return o, nil
|
||||
}
|
||||
|
||||
// Performs checks that the act an asker is about to perform is the one the chosen option bound when it
|
||||
// asked: the act's digest equals the option's Binds. An acknowledge option that bound nothing passes.
|
||||
func (o Option) Performs(act Act) error {
|
||||
if o.Binds == "" && o.Level == Acknowledge {
|
||||
return nil
|
||||
}
|
||||
d, err := ActDigest(act)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d != o.Binds {
|
||||
return fmt.Errorf("the option %s bound %s, and the act about to be performed is %s: nothing is done", o.ID, o.Binds, d)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Button is one answer a channel offers: its label and the router's one-time ticket for it.
|
||||
type Button struct {
|
||||
Label string `json:"label"`
|
||||
Ticket string `json:"ticket"`
|
||||
}
|
||||
|
||||
// Message is the work a channel takes: shown with buttons (Show), shown again in place (Edit), or said
|
||||
// (Send). Handle is the router's name for it, the same across a show and its edits; the channel keeps
|
||||
// which of its own messages that is. The words are the router's, shown as given.
|
||||
type Message struct {
|
||||
Handle string `json:"handle"`
|
||||
Title string `json:"title"`
|
||||
Body string `json:"body"`
|
||||
Buttons []Button `json:"buttons,omitempty"`
|
||||
// Note is the message's secondary words, after its body and its buttons: who asked and when, what each
|
||||
// answer does, when it expires, and the ask's Details. A channel shows it smaller, collapsed or set apart
|
||||
// where it can, and after the body where it cannot (novox/hq issue 639). A channel that bounds a message's
|
||||
// length cuts the body first and the note only when the title and the note alone do not fit, saying each
|
||||
// cut (novox/hq ADR 0347 §5).
|
||||
Note string `json:"note,omitempty"`
|
||||
Urgent bool `json:"urgent,omitempty"`
|
||||
Silent bool `json:"silent,omitempty"`
|
||||
// Reply is the Choice or LinkAsked this answers, by its ID: the channel shows it where that was made.
|
||||
Reply string `json:"reply,omitempty"`
|
||||
// To is the account linked as the operator on this kind, for a channel that verifies its sender: where
|
||||
// the channel sends what is not a reply. The router's word, from its list; empty when none is linked.
|
||||
To string `json:"to,omitempty"`
|
||||
// Secret says the words carry something shown once (a link's code): the channel shows it and keeps no
|
||||
// copy of it — no state, no history of its own.
|
||||
Secret bool `json:"secret,omitempty"`
|
||||
}
|
||||
|
||||
// Sender is who a channel's service says sent something: the account, the name it shows, and whether the
|
||||
// service authenticated it. The router alone judges whether that is the operator.
|
||||
type Sender struct {
|
||||
Identity string `json:"identity"`
|
||||
Display string `json:"display,omitempty"`
|
||||
Authenticated bool `json:"authenticated"`
|
||||
}
|
||||
|
||||
// Failed is a message a channel could not deliver: its handle, why, and whether trying again could help.
|
||||
type Failed struct {
|
||||
Handle string `json:"handle"`
|
||||
Why string `json:"why"`
|
||||
Permanent bool `json:"permanent,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// Standing is what a channel says of itself, at least every five minutes and whenever it changes:
|
||||
// whether it can send now, why not, and whether its edits notify nobody.
|
||||
type Standing struct {
|
||||
Ready bool `json:"ready"`
|
||||
Why string `json:"why,omitempty"`
|
||||
EditsSilently bool `json:"edits-silently,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// What a channel says of its own delivery, on IntakeSubject.
|
||||
const (
|
||||
FailedWhat = "failed"
|
||||
StandingWhat = "standing"
|
||||
)
|
||||
|
||||
// Choice is a button chosen on a channel.
|
||||
type Choice struct {
|
||||
// ID is the channel's own for this arrival, unique, so the router acts on it once.
|
||||
ID string `json:"id"`
|
||||
Ticket string `json:"ticket"`
|
||||
Handle string `json:"handle,omitempty"`
|
||||
Sender Sender `json:"sender"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// LinkAsked is somebody on a channel asking to be linked as the operator.
|
||||
type LinkAsked struct {
|
||||
ID string `json:"id"`
|
||||
Sender Sender `json:"sender"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// Code is a code a person typed on a channel, asked as a proof: never in an event, never kept.
|
||||
type Code struct {
|
||||
Sender Sender `json:"sender"`
|
||||
Code string `json:"code"`
|
||||
Purpose string `json:"purpose"`
|
||||
}
|
||||
|
||||
// ProofAnswer is the router's answer to a proof: whether it was taken, and words to say to the person.
|
||||
type ProofAnswer struct {
|
||||
Accepted bool `json:"accepted"`
|
||||
Words string `json:"words"`
|
||||
}
|
||||
|
||||
// EngineAsker is the asker name a node's engine asks the operator under (novox/hq ADR 0563 §3, which changes ADR 0259
|
||||
// §2 to admit a node's engine as an asker): `node-engine-<node>`, the last token of the by-caller subjects granted to that
|
||||
// node's engine account alone, so no engine acts on another node's warrant.
|
||||
func EngineAsker(node string) string { return "node-engine-" + node }
|
||||
@@ -0,0 +1,367 @@
|
||||
package asks
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
var now = time.Date(2026, 10, 8, 14, 0, 0, 0, time.UTC)
|
||||
|
||||
func held() Ask {
|
||||
return Ask{ID: "d-1", Headline: "openrazer delivery held", Explanation: "Needs you: release it, or stop it.",
|
||||
Who: Operator, Expires: now.Add(time.Hour), OnExpiry: "the delivery stays held", About: "delivery.openrazer",
|
||||
Options: []Option{
|
||||
{ID: "release", Label: "Release", Does: "the delivery goes on", Level: Approve, Binds: bind("release", "openrazer")},
|
||||
{ID: "stop", Label: "Stop", Does: "the delivery ends", Level: Approve, Binds: bind("stop", "openrazer")},
|
||||
{ID: "silence", Label: "Silence for a week", Does: "nothing more is said for a week", Level: Acknowledge},
|
||||
}}
|
||||
}
|
||||
|
||||
func bind(verb, what string) string {
|
||||
d, err := ActDigest(Act{"verb": verb, "delivery": what})
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
func TestASoundAskPasses(t *testing.T) {
|
||||
if err := held().Check(now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskIsRefusedInWordsForEachFault(t *testing.T) {
|
||||
for want, change := range map[string]func(*Ask){
|
||||
"is not letters": func(a *Ask) { a.ID = "a.b" },
|
||||
"headline is empty": func(a *Ask) { a.Headline = "" },
|
||||
"explains nothing": func(a *Ask) { a.Explanation = " " },
|
||||
"only the operator answers": func(a *Ask) { a.Who = "agent" },
|
||||
"offers 0 options": func(a *Ask) { a.Options = nil },
|
||||
"offers 5 options": func(a *Ask) { a.Options = append(a.Options, a.Options[0], a.Options[1]) },
|
||||
"is offered twice": func(a *Ask) { a.Options[1].ID = "release" },
|
||||
"does not say what it does": func(a *Ask) { a.Options[0].Does = "" },
|
||||
"none of acknowledge": func(a *Ask) { a.Options[0].Level = "admin" },
|
||||
"expires before it is asked": func(a *Ask) { a.Expires = now },
|
||||
"approves lasts at most": func(a *Ask) { a.Expires = now.Add(25 * time.Hour) },
|
||||
"destroys lasts at most": func(a *Ask) { a.Options[0].Level = Destroy },
|
||||
"never defaults": func(a *Ask) { a.OnExpiry = "" },
|
||||
"is a key, without spaces": func(a *Ask) { a.About = "a b" },
|
||||
"does not bind what it": func(a *Ask) { a.Options[0].Binds = "" },
|
||||
} {
|
||||
a := held()
|
||||
change(&a)
|
||||
if err := a.Check(now); err == nil || !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("want a refusal saying %q, got %v", want, err)
|
||||
}
|
||||
}
|
||||
ack := held()
|
||||
ack.Options = ack.Options[2:]
|
||||
ack.OnExpiry = ""
|
||||
ack.Expires = now.Add(7 * 24 * time.Hour)
|
||||
if err := ack.Check(now); err != nil {
|
||||
t.Errorf("an ask that only acknowledges needs no expiry words and may last a week: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskerActsOnlyOnAWarrantForItsOwnAskAndOption(t *testing.T) {
|
||||
w := Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: OutcomeChosen, Option: "release", Label: "Release",
|
||||
Level: Approve, By: &Person{Who: Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"},
|
||||
AskDigest: held().Digest(), At: now.Add(time.Minute)}
|
||||
if o, err := w.For("mesh-delivery", held()); err != nil || o.ID != "release" {
|
||||
t.Fatalf("a sound warrant was refused: %v", err)
|
||||
}
|
||||
if got := w.Says(); got != "the operator, via telegram (user id verified), chose Release" {
|
||||
t.Errorf("the warrant says %q", got)
|
||||
}
|
||||
for why, change := range map[string]func(*Warrant){
|
||||
"another asker": func(w *Warrant) { w.Asker = "mesh-controller" },
|
||||
"another ask": func(w *Warrant) { w.Ask = "d-2" },
|
||||
"no choice": func(w *Warrant) { w.Outcome = OutcomeExpired },
|
||||
"nobody": func(w *Warrant) { w.By = nil },
|
||||
"not offered": func(w *Warrant) { w.Option = "delete" },
|
||||
"a lower level": func(w *Warrant) { w.Level = Acknowledge },
|
||||
"another role": func(w *Warrant) { w.By = &Person{Who: "agent"} },
|
||||
"no digest": func(w *Warrant) { w.AskDigest = "" },
|
||||
"no time given": func(w *Warrant) { w.At = time.Time{} },
|
||||
"after expiry": func(w *Warrant) { w.At = now.Add(2 * time.Hour) },
|
||||
"another ask's digest": func(w *Warrant) {
|
||||
other := held()
|
||||
other.Options[0].Binds = bind("release", "another")
|
||||
w.AskDigest = other.Digest()
|
||||
},
|
||||
} {
|
||||
c := w
|
||||
by := *w.By
|
||||
c.By = &by
|
||||
change(&c)
|
||||
if _, err := c.For("mesh-delivery", held()); err == nil {
|
||||
t.Errorf("a warrant with %s was acted on", why)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheSubjectsNameTheirPublisher(t *testing.T) {
|
||||
for got, want := range map[string]string{
|
||||
AskSubject("mesh-delivery"): "mesh.seat.operator-channel.accept.ask.mesh-delivery",
|
||||
DecidedSubject("mesh-delivery"): "mesh.seat.operator-channel.event.decided.mesh-delivery",
|
||||
ChannelSubject(Show, "telegram"): "mesh.seat.channel.accept.show.telegram",
|
||||
IntakeSubject(Chosen, "desktop"): "mesh.seat.intake.event.choice.desktop",
|
||||
ProofSubject(CodeProof, "telegram"): "mesh.seat.intake.proof.code.telegram",
|
||||
} {
|
||||
if got != want {
|
||||
t.Errorf("%s, want %s", got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A warrant answers one ask as it was shown, and authorises the act its option bound and no other
|
||||
// (novox/hq ADR 0259 §6: the warrant is bound to the exact ask and its arguments).
|
||||
func TestAWarrantAuthorisesOnlyTheActItsOptionBound(t *testing.T) {
|
||||
a := held()
|
||||
// The digest survives the trip through the bus: what the router decodes and digests is what was sent.
|
||||
raw, err := json.Marshal(a)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var decoded Ask
|
||||
if err := json.Unmarshal(raw, &decoded); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if decoded.Digest() != a.Digest() {
|
||||
t.Fatalf("the ask's digest changed on the way: %s, %s", decoded.Digest(), a.Digest())
|
||||
}
|
||||
local := a
|
||||
local.Expires = a.Expires.In(time.FixedZone("CEST", 2*3600))
|
||||
if local.Digest() != a.Digest() {
|
||||
t.Error("the same instant in another zone digests differently")
|
||||
}
|
||||
// Changing anything the person was shown, or what an option binds, changes the digest.
|
||||
for what, change := range map[string]func(*Ask){
|
||||
"the headline": func(a *Ask) { a.Headline += "!" },
|
||||
"the explanation": func(a *Ask) { a.Explanation += " " },
|
||||
"a label": func(a *Ask) { a.Options[0].Label = "Go" },
|
||||
"what it binds": func(a *Ask) { a.Options[0].Binds = bind("release", "another") },
|
||||
"the expiry": func(a *Ask) { a.Expires = a.Expires.Add(time.Second) },
|
||||
"what it is about": func(a *Ask) { a.About = "delivery.other" },
|
||||
} {
|
||||
c := held()
|
||||
change(&c)
|
||||
if c.Digest() == a.Digest() {
|
||||
t.Errorf("changing %s leaves the digest", what)
|
||||
}
|
||||
}
|
||||
o, _ := a.Option("release")
|
||||
if err := o.Performs(Act{"verb": "release", "delivery": "openrazer"}); err != nil {
|
||||
t.Errorf("the bound act was refused: %v", err)
|
||||
}
|
||||
if err := o.Performs(Act{"verb": "release", "delivery": "another"}); err == nil {
|
||||
t.Error("an act the option did not bind was let through")
|
||||
}
|
||||
ack, _ := a.Option("silence")
|
||||
if err := ack.Performs(Act{"anything": "x"}); err != nil {
|
||||
t.Errorf("an acknowledgement that bound nothing: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The digests are over named fields in a canonical encoding (the review of 2026-10-09): not over Go's
|
||||
// encoding of a struct, so a field added to Ask or Option, a JSON tag renamed, or a map built in another
|
||||
// order changes nothing it should not — and every field that is shown or bound is in it.
|
||||
func TestDigestsAreCanonicalOverNamedFields(t *testing.T) {
|
||||
a1, _ := ActDigest(Act{"verb": "release", "arg.delivery": "hq"})
|
||||
b := Act{}
|
||||
b["arg.delivery"] = "hq"
|
||||
b["verb"] = "release"
|
||||
a2, _ := ActDigest(b)
|
||||
if a1 != a2 {
|
||||
t.Error("the order an act was built in changed its digest")
|
||||
}
|
||||
// A value cannot be read as two: "a"+"b=c" is not "a=b"+"c".
|
||||
x, _ := ActDigest(Act{"a": "b\n1:c"})
|
||||
y, _ := ActDigest(Act{"a": "b", "c": ""})
|
||||
if x == y {
|
||||
t.Error("two different acts encode the same")
|
||||
}
|
||||
if _, err := ActDigest(Act{}); err == nil {
|
||||
t.Error("an empty act was digested")
|
||||
}
|
||||
// The encoding, by hand: any language reproduces it.
|
||||
sum := sha256.Sum256([]byte("novox.act.v1\n4:verb\n7:release\n"))
|
||||
if got, _ := ActDigest(Act{"verb": "release"}); got != "sha256:"+hex.EncodeToString(sum[:]) {
|
||||
t.Errorf("the act's encoding is not the one written down: %s", got)
|
||||
}
|
||||
|
||||
// Every exported field of Ask and Option is in the digest: changing any one changes it.
|
||||
base := held()
|
||||
mutations := map[string]func(*Ask){
|
||||
"ID": func(a *Ask) { a.ID = "other" },
|
||||
"Headline": func(a *Ask) { a.Headline += "!" },
|
||||
"Explanation": func(a *Ask) { a.Explanation += "!" },
|
||||
"Who": func(a *Ask) { a.Who = "someone" },
|
||||
"Expires": func(a *Ask) { a.Expires = a.Expires.Add(time.Nanosecond) },
|
||||
"OnExpiry": func(a *Ask) { a.OnExpiry += "!" },
|
||||
"About": func(a *Ask) { a.About += "!" },
|
||||
"Urgent": func(a *Ask) { a.Urgent = !a.Urgent },
|
||||
"Options": func(a *Ask) { a.Options = a.Options[:1] },
|
||||
"Option.ID": func(a *Ask) { a.Options = append([]Option(nil), a.Options...); a.Options[0].ID = "x" },
|
||||
"Option.Label": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Label += "!"
|
||||
},
|
||||
"Option.Does": func(a *Ask) { a.Options = append([]Option(nil), a.Options...); a.Options[0].Does += "!" },
|
||||
"Option.Level": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Level = Destroy
|
||||
},
|
||||
"Option.Binds": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Binds = "sha256:00"
|
||||
},
|
||||
}
|
||||
for name, mutate := range mutations {
|
||||
c := base
|
||||
mutate(&c)
|
||||
if c.Digest() == base.Digest() {
|
||||
t.Errorf("changing %s left the digest the same", name)
|
||||
}
|
||||
}
|
||||
// Whole and Details (issue 383) are in the digest once given, and change it; an ask without either digests
|
||||
// exactly as before they existed — the encoding by hand, so no build can drift from it unseen.
|
||||
plain := Ask{ID: "p", Headline: "h", Explanation: "e", Who: Operator, Expires: now, OnExpiry: "o", About: "a",
|
||||
Options: []Option{{ID: "ok", Label: "OK", Does: "d", Level: Acknowledge}}}
|
||||
byHand := sha256.Sum256([]byte("novox.ask.v1\n1:p\n1:h\n1:e\n8:operator\n20:2026-10-08T14:00:00Z\n1:o\n1:a\n5:false\n1:1\n" +
|
||||
"2:ok\n2:OK\n1:d\n11:acknowledge\n0:\n"))
|
||||
if plain.Digest() != "sha256:"+hex.EncodeToString(byHand[:]) {
|
||||
t.Errorf("an ask without Whole and Details does not digest as it did before them: %s", plain.Digest())
|
||||
}
|
||||
withWhole, withDetails := plain, plain
|
||||
withWhole.Whole = "e, whole"
|
||||
withDetails.Details = "fingerprint 1234"
|
||||
if withWhole.Digest() == plain.Digest() || withDetails.Digest() == plain.Digest() || withWhole.Digest() == withDetails.Digest() {
|
||||
t.Error("Whole or Details left the digest the same")
|
||||
}
|
||||
both := withWhole
|
||||
both.Details = "fingerprint 1234"
|
||||
changed := both
|
||||
changed.Whole += "!"
|
||||
if changed.Digest() == both.Digest() {
|
||||
t.Error("changing Whole beside Details left the digest the same")
|
||||
}
|
||||
// A field added to Ask or Option without being added to Digest fails here.
|
||||
if n := reflect.TypeOf(Ask{}).NumField(); n != 12 {
|
||||
t.Errorf("Ask has %d fields, and Digest covers 12: add the new one to Digest and to this test", n)
|
||||
}
|
||||
if n := reflect.TypeOf(Option{}).NumField(); n != 5 {
|
||||
t.Errorf("Option has %d fields, and Digest covers 5: add the new one to Digest and to this test", n)
|
||||
}
|
||||
// The same instant in another zone is the same ask.
|
||||
z := base
|
||||
z.Expires = z.Expires.In(time.FixedZone("x", 3600))
|
||||
if z.Digest() != base.Digest() {
|
||||
t.Error("the expiry's zone changed the digest")
|
||||
}
|
||||
}
|
||||
|
||||
// An ask that says no expiry is answered by no warrant (the confirmation review of 2026-10-09): nothing could
|
||||
// show the answer was in time.
|
||||
func TestAWarrantForAnAskWithNoExpiryIsRefused(t *testing.T) {
|
||||
a := held()
|
||||
a.Expires = time.Time{}
|
||||
w := Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: OutcomeChosen, Option: "release", Label: "Release",
|
||||
Level: Approve, By: &Person{Who: Operator, Kind: "telegram", Identity: "42"}, AskDigest: a.Digest(),
|
||||
At: now.Add(time.Minute)}
|
||||
if _, err := w.For("mesh-delivery", a); err == nil {
|
||||
t.Error("a warrant for an ask with no expiry was taken")
|
||||
}
|
||||
}
|
||||
|
||||
// An explanation that reads at a glance passes; a wall of text says each way it is one (novox/hq issue 639).
|
||||
func TestAnExplanationReadsAtAGlance(t *testing.T) {
|
||||
if err := Glance("Your agents' memory follows you to every machine. A file changed on two keeps both.\n\n• Only the memory folders are carried\n• A conflict reaches you as a warning"); err != nil {
|
||||
t.Fatalf("a summary and two points: %v", err)
|
||||
}
|
||||
var wall strings.Builder
|
||||
wall.WriteString("ADR 0343 was proposed by fix-367 and approved by the knowledge base's own review.")
|
||||
for i := 1; i <= 13; i++ {
|
||||
wall.WriteString("\n" + strings.Repeat("A clause lead. ", 3))
|
||||
}
|
||||
err := Glance(wall.String())
|
||||
if err == nil || !strings.Contains(err.Error(), "13 points") {
|
||||
t.Fatalf("thirteen clause lines pass as a glance: %v", err)
|
||||
}
|
||||
if err := Glance(strings.Repeat("word ", 70)); err == nil || !strings.Contains(err.Error(), "summary is 349") {
|
||||
t.Fatalf("a long summary passes: %v", err)
|
||||
}
|
||||
if err := Glance("short.\n" + strings.Repeat("p", PointLength+1)); err == nil || !strings.Contains(err.Error(), "point 1") {
|
||||
t.Fatalf("a long point passes: %v", err)
|
||||
}
|
||||
if err := Glance(" \n"); err == nil {
|
||||
t.Fatal("an empty explanation passes")
|
||||
}
|
||||
}
|
||||
|
||||
// A message's note travels beside its body, and is left out of the encoding when empty, so a channel older
|
||||
// than it reads every message it read before.
|
||||
func TestAMessageCarriesItsNoteApart(t *testing.T) {
|
||||
raw, _ := json.Marshal(Message{Handle: "h", Title: "t", Body: "b", Note: "Asked by x."})
|
||||
if !strings.Contains(string(raw), `"note":"Asked by x."`) {
|
||||
t.Fatalf("the note is not encoded: %s", raw)
|
||||
}
|
||||
raw, _ = json.Marshal(Message{Handle: "h", Title: "t"})
|
||||
if strings.Contains(string(raw), "note") {
|
||||
t.Fatalf("an empty note is encoded: %s", raw)
|
||||
}
|
||||
}
|
||||
|
||||
// WholeDetails is digested when given, and an ask without it digests as before it existed: a router and an asker
|
||||
// of different builds still agree on every such ask.
|
||||
func TestWholeDetailsAreDigestedOnlyWhenGiven(t *testing.T) {
|
||||
a := Ask{ID: "a", Headline: "h", Explanation: "e", Details: "d"}
|
||||
before := a.Digest()
|
||||
a.WholeDetails = "+ shares: media=/storage/media"
|
||||
if a.Digest() == before {
|
||||
t.Fatal("the whole details are not in the digest")
|
||||
}
|
||||
b := a
|
||||
b.WholeDetails = "+ shares: media=/storage/other"
|
||||
if a.Digest() == b.Digest() {
|
||||
t.Fatal("two whole details digest the same")
|
||||
}
|
||||
a.WholeDetails = ""
|
||||
if a.Digest() != before {
|
||||
t.Fatal("an ask without whole details digests otherwise than before")
|
||||
}
|
||||
}
|
||||
|
||||
// No two asks encode the same (the review of mesh-sdk #25): whole details given beside nothing else are not read as
|
||||
// a Whole and Details that happen to spell the same values, nor as any other split of the three fields.
|
||||
func TestWholeDetailsCollideWithNoOtherAsk(t *testing.T) {
|
||||
base := Ask{ID: "a", Headline: "h", Explanation: "e"}
|
||||
shapes := []Ask{}
|
||||
for _, f := range []struct{ whole, details, wholeDetails string }{
|
||||
{"", "", "+ x: 1"},
|
||||
{"whole-details", "+ x: 1", ""},
|
||||
{"", "+ x: 1", ""},
|
||||
{"+ x: 1", "", ""},
|
||||
{"", "", "whole-details"},
|
||||
{"", "+ x: 1", "+ x: 1"},
|
||||
{"+ x: 1", "+ x: 1", ""},
|
||||
{"", "", ""},
|
||||
} {
|
||||
a := base
|
||||
a.Whole, a.Details, a.WholeDetails = f.whole, f.details, f.wholeDetails
|
||||
shapes = append(shapes, a)
|
||||
}
|
||||
seen := map[string]int{}
|
||||
for i, a := range shapes {
|
||||
if j, ok := seen[a.Digest()]; ok {
|
||||
t.Errorf("asks %d and %d digest the same: %+v / %+v", j, i, shapes[j], a)
|
||||
}
|
||||
seen[a.Digest()] = i
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
package asks
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ConditionSubject is where a module says a condition to the operator: a by-caller accept of the operator-channel
|
||||
// seat, `mesh.seat.operator-channel.accept.condition.<publisher>` (novox/hq ADR 0437 §9). The router reads the
|
||||
// verb at the fifth part and the caller at the sixth, so the publisher is one token, the module's own name; it is
|
||||
// refused when empty or when it holds a dot, a wildcard or whitespace.
|
||||
func ConditionSubject(publisher string) (string, error) {
|
||||
if publisher == "" {
|
||||
return "", errors.New("asks: the condition's publisher is empty")
|
||||
}
|
||||
if strings.ContainsAny(publisher, ".*> \t\r\n") || strings.ContainsFunc(publisher, func(r rune) bool { return r <= ' ' }) {
|
||||
return "", fmt.Errorf("asks: the condition's publisher %q is not one token: it holds a dot, a wildcard or whitespace", publisher)
|
||||
}
|
||||
return "mesh.seat." + Seat + ".accept.condition." + publisher, nil
|
||||
}
|
||||
|
||||
// ConditionVersion is the version of the condition value this package writes and the only one it reads.
|
||||
const ConditionVersion = 1
|
||||
|
||||
// ErrUnknownConditionVersion is what DecodeCondition returns for a value whose version is missing or not
|
||||
// ConditionVersion. Test it with errors.Is.
|
||||
var ErrUnknownConditionVersion = errors.New("asks: unknown condition version")
|
||||
|
||||
// The words a condition's explanation opens with (novox/hq ADR 0253 §1).
|
||||
const (
|
||||
NothingToDo = "Nothing for you to do."
|
||||
NeedsYou = "Needs you: "
|
||||
)
|
||||
|
||||
// The severities and states of a condition.
|
||||
const (
|
||||
SeverityUrgent = "urgent"
|
||||
SeverityWarning = "warning"
|
||||
StateRaised = "raised"
|
||||
StateCleared = "cleared"
|
||||
)
|
||||
|
||||
// Scope is what a condition is about, under the controller's names.
|
||||
type Scope struct {
|
||||
Scope string `json:"scope"`
|
||||
ID string `json:"id"`
|
||||
Machine string `json:"machine,omitempty"`
|
||||
}
|
||||
|
||||
// Action is one answer a notification offers, under the controller's names.
|
||||
type Action struct {
|
||||
Label string `json:"label"`
|
||||
Verb string `json:"verb"`
|
||||
Machine string `json:"machine,omitempty"`
|
||||
Arguments map[string]string `json:"arguments,omitempty"`
|
||||
Level Level `json:"level,omitempty"`
|
||||
}
|
||||
|
||||
// Condition is a condition a module says to the operator, under the names of the controller's conditions: its key,
|
||||
// severity, subject, state (raised or cleared), summary and the plain words of ADR 0253. A cleared condition needs
|
||||
// only its key, subject and state. The router alone chooses the channels.
|
||||
type Condition struct {
|
||||
V int `json:"v"`
|
||||
Key string `json:"key"`
|
||||
Severity string `json:"severity,omitempty"`
|
||||
Subject Scope `json:"subject"`
|
||||
State string `json:"state"`
|
||||
Summary string `json:"summary,omitempty"`
|
||||
Headline string `json:"headline,omitempty"`
|
||||
Needs string `json:"needs,omitempty"`
|
||||
Explanation string `json:"explanation,omitempty"`
|
||||
Resolved string `json:"resolved,omitempty"`
|
||||
Actions []Action `json:"actions,omitempty"`
|
||||
}
|
||||
|
||||
func (c Condition) check() error {
|
||||
if c.Key == "" {
|
||||
return errors.New("asks: a condition has no key")
|
||||
}
|
||||
if c.Subject.Scope == "" || c.Subject.ID == "" {
|
||||
return errors.New("asks: a condition's subject needs its scope and id")
|
||||
}
|
||||
switch c.State {
|
||||
case StateCleared:
|
||||
return nil
|
||||
case StateRaised:
|
||||
default:
|
||||
return fmt.Errorf("asks: the condition's state %q is not raised or cleared", c.State)
|
||||
}
|
||||
switch c.Severity {
|
||||
case SeverityUrgent, SeverityWarning:
|
||||
default:
|
||||
return fmt.Errorf("asks: the condition's severity %q is not urgent or warning", c.Severity)
|
||||
}
|
||||
if c.Headline == "" {
|
||||
return errors.New("asks: a raised condition has no headline")
|
||||
}
|
||||
if n := len([]rune(c.Headline)); n > HeadlineLength {
|
||||
return fmt.Errorf("asks: the condition's headline is %d characters, more than %d", n, HeadlineLength)
|
||||
}
|
||||
verdict := NothingToDo
|
||||
if c.Needs != "" {
|
||||
verdict = NeedsYou + c.Needs
|
||||
}
|
||||
if !strings.HasPrefix(c.Explanation, verdict) {
|
||||
return fmt.Errorf("asks: the condition's explanation must open with %q", verdict)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// EncodeCondition is c as JSON at ConditionVersion; it refuses what ADR 0253 and ADR 0437 §9 refuse.
|
||||
func EncodeCondition(c Condition) ([]byte, error) {
|
||||
if err := c.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.V = ConditionVersion
|
||||
return json.Marshal(c)
|
||||
}
|
||||
|
||||
// DecodeCondition reads a condition; it refuses an unknown version (ErrUnknownConditionVersion) and what
|
||||
// EncodeCondition refuses.
|
||||
func DecodeCondition(data []byte) (Condition, error) {
|
||||
var probe struct {
|
||||
V *int `json:"v"`
|
||||
}
|
||||
if err := json.Unmarshal(data, &probe); err != nil {
|
||||
return Condition{}, fmt.Errorf("asks: the condition is not JSON: %w", err)
|
||||
}
|
||||
if probe.V == nil {
|
||||
return Condition{}, fmt.Errorf("%w: no version in the value, this reader reads %d", ErrUnknownConditionVersion, ConditionVersion)
|
||||
}
|
||||
if *probe.V != ConditionVersion {
|
||||
return Condition{}, fmt.Errorf("%w: the value is version %d, this reader reads %d", ErrUnknownConditionVersion, *probe.V, ConditionVersion)
|
||||
}
|
||||
var c Condition
|
||||
if err := json.Unmarshal(data, &c); err != nil {
|
||||
return Condition{}, fmt.Errorf("asks: the condition does not fit its shape: %w", err)
|
||||
}
|
||||
if err := c.check(); err != nil {
|
||||
return Condition{}, err
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// SameCondition says whether two conditions are the same value.
|
||||
func SameCondition(a, b Condition) bool { return reflect.DeepEqual(a, b) }
|
||||
@@ -0,0 +1,143 @@
|
||||
package asks
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestConditionSubject(t *testing.T) { // 1202/AC1
|
||||
s, err := ConditionSubject("mesh-delivery-teller")
|
||||
if err != nil || s != "mesh.seat.operator-channel.accept.condition.mesh-delivery-teller" {
|
||||
t.Fatalf("subject = %q, %v", s, err)
|
||||
}
|
||||
// The router splits an accept subject in six parts and reads the verb at 4 and the caller at 5.
|
||||
parts := strings.Split(s, ".")
|
||||
if len(parts) != 6 || parts[4] != "condition" || parts[5] != "mesh-delivery-teller" {
|
||||
t.Errorf("subject %q does not keep the router's shape: %v", s, parts)
|
||||
}
|
||||
for _, bad := range []string{"", "a.b", "a*", "a>", "a b", "a\tb", "a\nb"} {
|
||||
if _, err := ConditionSubject(bad); err == nil {
|
||||
t.Errorf("publisher %q accepted", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func raisedCondition() Condition {
|
||||
return Condition{
|
||||
Key: "delivery.stuck.mesh-mail", Severity: "warning",
|
||||
Subject: Scope{Scope: "module", ID: "mesh-mail", Machine: "ace"}, State: "raised",
|
||||
Summary: "plan 12 stopped", Headline: "Delivery of mesh-mail is stuck",
|
||||
Needs: "approve the merge.", Explanation: "Needs you: approve the merge. The plan waits.",
|
||||
Resolved: "The delivery of mesh-mail moved on.",
|
||||
Actions: []Action{{Label: "Approve", Verb: "mesh-controller.assign", Machine: "ace",
|
||||
Arguments: map[string]string{"why": ""}, Level: Approve}},
|
||||
}
|
||||
}
|
||||
|
||||
func clearedCondition() Condition {
|
||||
return Condition{Key: "delivery.stuck.mesh-mail", Subject: Scope{Scope: "module", ID: "mesh-mail"}, State: "cleared"}
|
||||
}
|
||||
|
||||
func TestConditionRoundTrip(t *testing.T) { // 1202/AC2
|
||||
for _, c := range []Condition{raisedCondition(), clearedCondition()} {
|
||||
data, err := EncodeCondition(c)
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", c.State, err)
|
||||
}
|
||||
var probe map[string]any
|
||||
if err := json.Unmarshal(data, &probe); err != nil || probe["v"] != float64(1) {
|
||||
t.Fatalf("%s: %s does not carry v:1", c.State, data)
|
||||
}
|
||||
c.V = ConditionVersion
|
||||
got, err := DecodeCondition(data)
|
||||
if err != nil || !SameCondition(got, c) {
|
||||
t.Errorf("%s round trip: %+v %v", c.State, got, err)
|
||||
}
|
||||
}
|
||||
data, _ := EncodeCondition(raisedCondition())
|
||||
var probe map[string]any
|
||||
_ = json.Unmarshal(data, &probe)
|
||||
for _, name := range []string{"key", "severity", "subject", "summary", "headline", "needs", "explanation", "resolved", "actions", "state"} {
|
||||
if _, ok := probe[name]; !ok {
|
||||
t.Errorf("the JSON has no %q: %s", name, data)
|
||||
}
|
||||
}
|
||||
subject := probe["subject"].(map[string]any)
|
||||
for _, name := range []string{"scope", "id", "machine"} {
|
||||
if _, ok := subject[name]; !ok {
|
||||
t.Errorf("the subject has no %q: %s", name, data)
|
||||
}
|
||||
}
|
||||
action := probe["actions"].([]any)[0].(map[string]any)
|
||||
for _, name := range []string{"label", "verb", "machine", "arguments", "level"} {
|
||||
if _, ok := action[name]; !ok {
|
||||
t.Errorf("the action has no %q: %s", name, data)
|
||||
}
|
||||
}
|
||||
for _, in := range []string{`{"key":"k"}`, `{"v":2,"key":"k"}`} {
|
||||
if _, err := DecodeCondition([]byte(in)); !errors.Is(err, ErrUnknownConditionVersion) {
|
||||
t.Errorf("decode of %s = %v, want ErrUnknownConditionVersion", in, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConditionVerdict(t *testing.T) { // 1202/AC3
|
||||
c := raisedCondition()
|
||||
c.Needs, c.Explanation = "", "The plan waits."
|
||||
_, err := EncodeCondition(c)
|
||||
if err == nil || !strings.Contains(err.Error(), "Nothing for you to do.") {
|
||||
t.Errorf("empty needs without its verdict: %v", err)
|
||||
}
|
||||
c.Explanation = "Nothing for you to do. The plan waits."
|
||||
if _, err := EncodeCondition(c); err != nil {
|
||||
t.Errorf("empty needs with its verdict refused: %v", err)
|
||||
}
|
||||
c = raisedCondition()
|
||||
c.Explanation = "Nothing for you to do. The plan waits."
|
||||
_, err = EncodeCondition(c)
|
||||
if err == nil || !strings.Contains(err.Error(), "Needs you: approve the merge.") {
|
||||
t.Errorf("needs set with the wrong verdict: %v", err)
|
||||
}
|
||||
c.Explanation = "Needs you: something else. The plan waits."
|
||||
if _, err = EncodeCondition(c); err == nil {
|
||||
t.Error("needs set with other needs in the verdict accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestConditionRefused(t *testing.T) { // 1202/AC4
|
||||
mutate := func(f func(*Condition)) error {
|
||||
c := raisedCondition()
|
||||
f(&c)
|
||||
_, err := EncodeCondition(c)
|
||||
return err
|
||||
}
|
||||
for name, f := range map[string]func(*Condition){
|
||||
"severity": func(c *Condition) { c.Severity = "critical" },
|
||||
"state": func(c *Condition) { c.State = "open" },
|
||||
"key": func(c *Condition) { c.Key = "" },
|
||||
"headline": func(c *Condition) { c.Headline = "" },
|
||||
"long headline": func(c *Condition) { c.Headline = strings.Repeat("h", HeadlineLength+1) },
|
||||
} {
|
||||
if err := mutate(f); err == nil {
|
||||
t.Errorf("a condition with a bad %s accepted", name)
|
||||
}
|
||||
}
|
||||
if err := mutate(func(c *Condition) { c.Headline = strings.Repeat("h", HeadlineLength) }); err != nil {
|
||||
t.Errorf("a headline of the greatest length refused: %v", err)
|
||||
}
|
||||
if _, err := EncodeCondition(clearedCondition()); err != nil {
|
||||
t.Errorf("a cleared condition with only key, subject and state refused: %v", err)
|
||||
}
|
||||
c := clearedCondition()
|
||||
c.Key = ""
|
||||
if _, err := EncodeCondition(c); err == nil {
|
||||
t.Error("a cleared condition without its key accepted")
|
||||
}
|
||||
c = clearedCondition()
|
||||
c.State = "gone"
|
||||
if _, err := EncodeCondition(c); err == nil {
|
||||
t.Error("a condition with an unknown state accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
package buildstep
|
||||
|
||||
// CapacitySetting is the build modules' setting of the node's build capacity (novox/hq ADR 0525 §5, ADR 0567 §12): how
|
||||
// many asks the node takes at once, an integer config value set on each node's layer of mesh-config. No repository names
|
||||
// a node: each build module declares it with the default 0, which pulls nothing, and the controller (or mesh-config)
|
||||
// lays the node's value into the module's settings file.
|
||||
const CapacitySetting = "capacity"
|
||||
@@ -0,0 +1,5 @@
|
||||
// Package buildstep is the contract of a build step every delivery part reads (novox/hq ADR 0593 §1): a step's id, the
|
||||
// fetch request, a step a previous process left running and its end, a failure's words, a module tree's shape, the
|
||||
// capacity setting and the launchers' names. The runner the build modules share is the Go build module's own package
|
||||
// (mesh-catalog modules/mesh-build-go/buildstep), which imports this one.
|
||||
package buildstep
|
||||
@@ -0,0 +1,141 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/sdkversion"
|
||||
)
|
||||
|
||||
// Failure is one test that failed, or a package or file that failed with no test to name (a build that failed, a test
|
||||
// program that panicked, a test file that did not load). The JSON names are the ones mesh-build-go and the checker have
|
||||
// always used; File is new and an older reader ignores it.
|
||||
type Failure struct {
|
||||
// Package is the Go package's import path, or the TypeScript package's directory in the workspace.
|
||||
Package string `json:"package"`
|
||||
// Test names the failing test, or the tests that were running when the program stopped; "" when none can be named.
|
||||
Test string `json:"test,omitempty"`
|
||||
// File is where the failing test is (a Go test's file and line is in its Cause; node names the test's file:line:col).
|
||||
File string `json:"file,omitempty"`
|
||||
// Cause is the one line that says why: the assertion's or the error's first line, a "panic:" line, a build's first
|
||||
// error, or words saying how the program stopped. **Never empty, and never a passing line** (PassingLine).
|
||||
Cause string `json:"cause,omitempty"`
|
||||
// Output is the failing test's own output, its last lines up to OutputBytes, Cause first: an older checker reads
|
||||
// only its first line.
|
||||
Output string `json:"output,omitempty"`
|
||||
}
|
||||
|
||||
// OutputBytes is the most a failure keeps of its output, and LineBytes the most of one line: four kilobytes holds a long
|
||||
// assertion message and stays far under a bus message's or a verdict's bound.
|
||||
const (
|
||||
OutputBytes = 4096
|
||||
LineBytes = 1024
|
||||
)
|
||||
|
||||
// By names a build module and the SDK it was built with, for a result's "by": the checker names it when a failure hides
|
||||
// its cause ("the build module hid the failure (<by>)").
|
||||
func By(module string) string {
|
||||
return module + " with mesh-sdk " + sdkversion.Version()
|
||||
}
|
||||
|
||||
// PassingLine says a line is a test runner's word for a test or a package that passed: node's '✔', TAP's "ok N" (at any
|
||||
// indent), go test's "ok \t<pkg>", "PASS" and "--- PASS". A line of a test's own that starts "ok " is none of these. Such a line is never a failure's cause, nor part of a failure's output.
|
||||
func PassingLine(l string) bool {
|
||||
if strings.HasPrefix(l, "ok \t") { // go test's package line, at column 0
|
||||
return true
|
||||
}
|
||||
t := strings.TrimSpace(l)
|
||||
switch {
|
||||
case t == "PASS", strings.HasPrefix(t, "✔"), strings.HasPrefix(t, "--- PASS"):
|
||||
return true
|
||||
}
|
||||
return tapOk(t)
|
||||
}
|
||||
|
||||
// Hidden says how a failure hides why it failed, or "" when it does not: it has no cause, or its cause is a passing line.
|
||||
// The checker says it loudly (issue 1901): a build module that sends such a failure does not keep the contract.
|
||||
func Hidden(f Failure) string {
|
||||
c := strings.TrimSpace(f.Cause)
|
||||
switch {
|
||||
case c == "":
|
||||
return "it sent no cause"
|
||||
case PassingLine(c):
|
||||
return fmt.Sprintf("its cause is a passing line (%q)", CutLine(c))
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// Clip joins the last lines of a failure's output that fit OutputBytes, each cut to LineBytes, and says how many earlier
|
||||
// lines it left out. **Its first line is the cause**, put ahead when the kept lines do not start with it, or, with no
|
||||
// cause and lines left out, the first non-empty line, then the marker: an older checker reads only the first line.
|
||||
// Passing lines are dropped.
|
||||
func Clip(lines []string, cause string) string {
|
||||
own := make([]string, 0, len(lines))
|
||||
for _, l := range lines {
|
||||
if !PassingLine(l) {
|
||||
own = append(own, l)
|
||||
}
|
||||
}
|
||||
lines = own
|
||||
size, from := 0, len(lines)
|
||||
for from > 0 {
|
||||
n := len(CutLine(lines[from-1])) + 1
|
||||
if size+n > OutputBytes && from < len(lines) {
|
||||
break
|
||||
}
|
||||
size += n
|
||||
from--
|
||||
}
|
||||
kept := make([]string, 0, len(lines)-from+2)
|
||||
head := strings.TrimSpace(cause)
|
||||
for i := 0; head == "" && i < from; i++ {
|
||||
head = strings.TrimSpace(lines[i])
|
||||
}
|
||||
if head != "" && (from > 0 || len(lines) == 0 || strings.TrimSpace(lines[from]) != head) {
|
||||
kept = append(kept, CutLine(head))
|
||||
}
|
||||
if from > 0 {
|
||||
kept = append(kept, fmt.Sprintf("... (%d earlier lines)", from))
|
||||
}
|
||||
for _, l := range lines[from:] {
|
||||
kept = append(kept, CutLine(l))
|
||||
}
|
||||
return strings.Join(kept, "\n")
|
||||
}
|
||||
|
||||
// CutLine cuts a line to LineBytes, on a character boundary.
|
||||
func CutLine(l string) string {
|
||||
if len(l) <= LineBytes {
|
||||
return l
|
||||
}
|
||||
return strings.ToValidUTF8(l[:LineBytes], "") + "..."
|
||||
}
|
||||
|
||||
// ConsoleFailure is the failure of a command that is no test runner (go vet, gofmt, tsc, a build script): its cause is the
|
||||
// first line that names an error (tsc's "error TS", go vet's "file.go:12:3: ..."), else the first line that is not a
|
||||
// heading ("# <package>", npm's "> script"), and its output the command's lines without passing ones.
|
||||
func ConsoleFailure(pkg, test, out string) Failure {
|
||||
var lines []string
|
||||
for _, l := range strings.Split(strings.ReplaceAll(stripANSI(out), "\r\n", "\n"), "\n") {
|
||||
if t := strings.TrimSpace(l); t != "" && !PassingLine(t) {
|
||||
lines = append(lines, strings.TrimRight(l, " \t"))
|
||||
}
|
||||
}
|
||||
cause := ""
|
||||
for _, l := range lines {
|
||||
if t := strings.TrimSpace(l); strings.Contains(t, "error") || strings.Contains(t, "Error") || goPosition.MatchString(t) {
|
||||
cause = t
|
||||
break
|
||||
}
|
||||
}
|
||||
for _, l := range lines {
|
||||
if t := strings.TrimSpace(l); cause == "" && !strings.HasPrefix(t, "# ") && !strings.HasPrefix(t, "> ") && !strings.HasPrefix(t, "npm ") {
|
||||
cause = t
|
||||
}
|
||||
}
|
||||
if cause == "" {
|
||||
cause = test + " failed and wrote nothing that says why"
|
||||
}
|
||||
cause = CutLine(cause)
|
||||
return Failure{Package: pkg, Test: test, Cause: cause, Output: Clip(lines, cause)}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// goPosition matches a line that starts with a Go file's position, as go vet and the compiler write an error.
|
||||
var goPosition = regexp.MustCompile(`^\S+\.go:\d+(:\d+)?: `)
|
||||
|
||||
var ansiEscape = regexp.MustCompile("\x1b\\[[0-9;]*m")
|
||||
|
||||
func stripANSI(s string) string { return ansiEscape.ReplaceAllString(s, "") }
|
||||
|
||||
// tapOk says a line is a passing test's "ok N - name", indented or not.
|
||||
func tapOk(l string) bool {
|
||||
rest, ok := strings.CutPrefix(strings.TrimSpace(l), "ok ")
|
||||
return ok && rest != "" && rest[0] >= '0' && rest[0] <= '9'
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package buildstep
|
||||
|
||||
import "git.novox.be/novox/mesh-sdk/go/catalogue"
|
||||
|
||||
// LauncherSuffix is what a TypeScript entrypoint's launcher is called beside it: index.js is started as
|
||||
// index.serve.mjs (novox/hq ADR 0193), an ES module by its own extension whatever the bundle's package.json says.
|
||||
const LauncherSuffix = catalogue.LauncherSuffix
|
||||
|
||||
// Launchers are the launchers a bundle's build writes, by entrypoint, relative to the bundle's root
|
||||
// (catalogue.Launchers): the one rule mesh-build-node, build-agent, the controller and a node taking in a build share
|
||||
// (R4e, issue 1126), so the name a build writes is the name a reader expects.
|
||||
func Launchers(language string, entrypoints []string) map[string]string {
|
||||
return catalogue.Launchers(language, entrypoints)
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A TypeScript bundle's entrypoint x.js is started as x.serve.mjs beside it (novox/hq ADR 0193), the rule mesh-build-node
|
||||
// and build-agent both apply; an entrypoint that is not .js gets none, and another language gets none.
|
||||
func TestATypeScriptEntrypointIsStartedByTheLauncherBesideIt(t *testing.T) {
|
||||
got := Launchers("typescript", []string{"index.js", "tools/x.js", "data.json"})
|
||||
want := map[string]string{"index.js": "index.serve.mjs", "tools/x.js": "tools/x.serve.mjs"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("Launchers(typescript) = %v, want %v", got, want)
|
||||
}
|
||||
if got := Launchers("go", []string{"index.js"}); len(got) != 0 {
|
||||
t.Fatalf("a Go bundle got launchers: %v", got)
|
||||
}
|
||||
if got := Launchers("typescript", nil); len(got) != 0 {
|
||||
t.Fatalf("a bundle with no entrypoint got launchers: %v", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"maps"
|
||||
"path"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
)
|
||||
|
||||
var (
|
||||
requestCommitOK = regexp.MustCompile(`^[0-9a-f]{40}$`)
|
||||
requestRepositoryOK = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]*/[A-Za-z0-9][A-Za-z0-9._-]*$`)
|
||||
)
|
||||
|
||||
// FetchRequest is what a build module's fetch tool is asked: the repository and the full commit, the pull request's
|
||||
// ref when only its head carries the commit, the trunk commit to merge onto for a merge preview, and, for a build of a
|
||||
// module whose every artifact names its paths, that module's directory and its named paths (novox/hq ADR 0499 clause 4).
|
||||
//
|
||||
// **The paths come from the builder, not from a manifest** (ADR 0495 clause 1; the lead's decision L8 of R4e): the
|
||||
// builder fills Dir and Paths from the indexer's Module value (Path, and KeptInputs2.Paths' keys), and the build module
|
||||
// gives the build and test steps a workspace holding those paths, the manifest and its lock files. A request without
|
||||
// them builds from the module's directory, as before.
|
||||
type FetchRequest struct {
|
||||
Repository string `json:"repository"`
|
||||
Commit string `json:"commit"`
|
||||
Ref string `json:"ref,omitempty"`
|
||||
Base string `json:"base,omitempty"`
|
||||
Dir string `json:"dir,omitempty"`
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
// Packages are the version of each mesh package the build or the check builds against (delivery.StepAsk.Packages,
|
||||
// novox/hq ADR 0573 §2, §4): the build module writes each into the workspace's go.mod and go.sum, whatever version
|
||||
// the repository names, checksum-verified, and names them in its log.
|
||||
Packages delivery.PackageVersions `json:"packages,omitempty"`
|
||||
}
|
||||
|
||||
// FetchRequestProperties are the JSON schema properties of FetchRequest's dir and paths, for a fetch tool's Input.
|
||||
func FetchRequestProperties() map[string]any {
|
||||
return map[string]any{
|
||||
"dir": map[string]any{"type": "string", "description": "the module's directory in the repository (\".\" for its root), with paths"},
|
||||
"paths": map[string]any{"type": "array", "items": map[string]any{"type": "string"},
|
||||
"description": "the paths the module names, relative to dir, from the indexer's kept inputs: the build sees only these, the manifest and its lock files"},
|
||||
"packages": map[string]any{"type": "object", "additionalProperties": map[string]any{"type": "string"},
|
||||
"description": "optional: the version (a full commit) of each mesh package to build against, by its module path, whatever version go.mod names (novox/hq ADR 0573 §2); the builder and the checker fill it"},
|
||||
}
|
||||
}
|
||||
|
||||
// Args is r as a tool call's arguments.
|
||||
func (r FetchRequest) Args() map[string]any {
|
||||
args := map[string]any{"repository": r.Repository, "commit": r.Commit}
|
||||
for k, v := range map[string]string{"ref": r.Ref, "base": r.Base, "dir": r.Dir} {
|
||||
if v != "" {
|
||||
args[k] = v
|
||||
}
|
||||
}
|
||||
if len(r.Paths) > 0 {
|
||||
args["paths"] = append([]string(nil), r.Paths...)
|
||||
}
|
||||
if len(r.Packages) > 0 {
|
||||
args["packages"] = map[string]any{}
|
||||
for p, v := range r.Packages {
|
||||
args["packages"].(map[string]any)[p] = v
|
||||
}
|
||||
}
|
||||
return args
|
||||
}
|
||||
|
||||
// ParseFetchRequest reads a fetch tool's arguments; arguments it does not know (a build module's own) are left to the
|
||||
// caller. It refuses a repository that is not <owner>/<name>, a commit or a base that is not 40 hex digits, paths with no
|
||||
// module directory or a directory with no path, a directory leaving the repository and a path leaving the directory,
|
||||
// naming each.
|
||||
func ParseFetchRequest(args map[string]any) (FetchRequest, error) {
|
||||
var r FetchRequest
|
||||
r.Repository, _ = args["repository"].(string)
|
||||
r.Commit, _ = args["commit"].(string)
|
||||
r.Ref, _ = args["ref"].(string)
|
||||
r.Base, _ = args["base"].(string)
|
||||
r.Dir, _ = args["dir"].(string)
|
||||
switch v := args["paths"].(type) {
|
||||
case nil:
|
||||
case []string:
|
||||
r.Paths = append([]string(nil), v...)
|
||||
case []any:
|
||||
for _, p := range v {
|
||||
s, ok := p.(string)
|
||||
if !ok {
|
||||
return FetchRequest{}, fmt.Errorf("paths is a list of strings")
|
||||
}
|
||||
r.Paths = append(r.Paths, s)
|
||||
}
|
||||
default:
|
||||
return FetchRequest{}, fmt.Errorf("paths is a list of strings")
|
||||
}
|
||||
packages, err := PackagesArg(args["packages"])
|
||||
if err != nil {
|
||||
return FetchRequest{}, err
|
||||
}
|
||||
r.Packages = packages
|
||||
return r, r.Check()
|
||||
}
|
||||
|
||||
// PackagesArg reads a tool call's packages argument, an object of strings (as JSON gives it) or the map an ask gives;
|
||||
// nil for none.
|
||||
func PackagesArg(v any) (delivery.PackageVersions, error) {
|
||||
out := delivery.PackageVersions{}
|
||||
switch v := v.(type) {
|
||||
case nil:
|
||||
return nil, nil
|
||||
case delivery.PackageVersions:
|
||||
maps.Copy(out, v)
|
||||
case map[string]string:
|
||||
maps.Copy(out, v)
|
||||
case map[string]any:
|
||||
for p, version := range v {
|
||||
s, ok := version.(string)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("packages is an object of versions (strings), by package")
|
||||
}
|
||||
out[p] = s
|
||||
}
|
||||
default:
|
||||
return nil, fmt.Errorf("packages is an object of versions (strings), by package")
|
||||
}
|
||||
if len(out) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
return out, out.Check()
|
||||
}
|
||||
|
||||
// Check is what ParseFetchRequest refuses.
|
||||
func (r FetchRequest) Check() error {
|
||||
if !requestRepositoryOK.MatchString(r.Repository) || strings.Contains(r.Repository, "..") {
|
||||
return fmt.Errorf("%q is not a repository, <owner>/<name>", r.Repository)
|
||||
}
|
||||
if !requestCommitOK.MatchString(r.Commit) {
|
||||
return fmt.Errorf("%q is not a full commit (40 hex digits)", r.Commit)
|
||||
}
|
||||
if r.Base != "" && !requestCommitOK.MatchString(r.Base) {
|
||||
return fmt.Errorf("%q is not a trunk commit (40 hex digits)", r.Base)
|
||||
}
|
||||
switch {
|
||||
case len(r.Paths) > 0 && r.Dir == "":
|
||||
return fmt.Errorf("the request names paths and names no module directory (dir) they are relative to")
|
||||
case len(r.Paths) == 0 && r.Dir != "":
|
||||
return fmt.Errorf("the request names a module directory and no path")
|
||||
}
|
||||
if r.Dir != "" && (outside(r.Dir) || strings.HasPrefix(r.Dir, "/")) {
|
||||
return fmt.Errorf("the module directory %q leaves the repository", r.Dir)
|
||||
}
|
||||
for _, p := range r.Paths {
|
||||
if p == "" || strings.HasPrefix(p, "/") || outside(p) || path.Clean(p) == "." {
|
||||
return fmt.Errorf("the path %q leaves the module's directory or is not a path in it", p)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func outside(p string) bool {
|
||||
c := path.Clean(p)
|
||||
return c == ".." || strings.HasPrefix(c, "../")
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A build's fetch is handed the paths of a module that names them (novox/hq ADR 0499 clause 4): the builder fills them
|
||||
// from the indexer's kept inputs and the build module reads them from the call, never from the manifest (ADR 0495
|
||||
// clause 1). The request survives the trip through a tool call's arguments.
|
||||
func TestAFetchRequestCarriesTheNamedPathsThroughTheToolCall(t *testing.T) {
|
||||
// And the versions of the mesh packages to build against (novox/hq ADR 0573 §2; issue 1585 S4).
|
||||
req := FetchRequest{Repository: "novox/mesh-sdk", Commit: strings.Repeat("a", 40), Ref: "refs/pull/7/head",
|
||||
Dir: ".", Paths: []string{"package.json", "src"}, Packages: map[string]string{"git.novox.be/novox/mesh-sdk/go": strings.Repeat("d", 40)}}
|
||||
args := req.Args()
|
||||
// A tool call's arguments arrive as JSON: lists of any, objects of any.
|
||||
args["paths"] = []any{"package.json", "src"}
|
||||
if p, ok := args["packages"].(map[string]any); !ok || p["git.novox.be/novox/mesh-sdk/go"] != strings.Repeat("d", 40) {
|
||||
t.Fatalf("the packages are not in the call's arguments: %v", args["packages"])
|
||||
}
|
||||
got, err := ParseFetchRequest(args)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(got, req) {
|
||||
t.Fatalf("got %+v, want %+v", got, req)
|
||||
}
|
||||
plain, err := ParseFetchRequest(map[string]any{"repository": "novox/x", "commit": strings.Repeat("b", 40),
|
||||
"base": strings.Repeat("c", 40), "beside": []any{}})
|
||||
if err != nil || plain.Paths != nil || plain.Base != strings.Repeat("c", 40) {
|
||||
t.Fatalf("a request without paths: %+v, %v", plain, err)
|
||||
}
|
||||
|
||||
ok := map[string]any{"repository": "novox/x", "commit": strings.Repeat("b", 40)}
|
||||
for _, c := range []struct {
|
||||
set map[string]any
|
||||
says string
|
||||
}{
|
||||
{map[string]any{"repository": "novox"}, "is not a repository"},
|
||||
{map[string]any{"repository": "novox/../x"}, "is not a repository"},
|
||||
{map[string]any{"commit": "abc"}, "is not a full commit"},
|
||||
{map[string]any{"base": "abc"}, "is not a trunk commit"},
|
||||
{map[string]any{"paths": []any{"src"}}, "names no module directory"},
|
||||
{map[string]any{"dir": "modules/a", "paths": []any{"../b"}}, "leaves the module's directory"},
|
||||
{map[string]any{"dir": "../a", "paths": []any{"src"}}, "leaves the repository"},
|
||||
{map[string]any{"dir": "modules/a", "paths": []any{3}}, "paths is a list of strings"},
|
||||
{map[string]any{"dir": "modules/a"}, "names a module directory and no path"},
|
||||
{map[string]any{"packages": map[string]any{"git.novox.be/novox/mesh-sdk/go": "v0.1.7"}}, "no full commit"},
|
||||
{map[string]any{"packages": map[string]any{"a b": strings.Repeat("d", 40)}}, "no module path"},
|
||||
{map[string]any{"packages": []any{"x"}}, "packages is an object"},
|
||||
} {
|
||||
args := map[string]any{}
|
||||
for k, v := range ok {
|
||||
args[k] = v
|
||||
}
|
||||
for k, v := range c.set {
|
||||
args[k] = v
|
||||
}
|
||||
if _, err := ParseFetchRequest(args); err == nil || !strings.Contains(err.Error(), c.says) {
|
||||
t.Errorf("%v: %v, wanted a refusal saying %q", c.set, err, c.says)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,330 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
)
|
||||
|
||||
// RestartReason is why a step a previous process left running is ended.
|
||||
const RestartReason = "the build module restarted while it ran"
|
||||
|
||||
// Boot is when this process began. A step with no owner (written by an older build) started at or after it is taken
|
||||
// for this process's, which may still run it, so EndOne refuses it.
|
||||
var Boot = time.Now()
|
||||
|
||||
// Owner is this process as a step's owner (delivery.Step.Owner): `<boot id>.<pid>.<start>`, the node's boot id
|
||||
// (/proc/sys/kernel/random/boot_id), the pid and Boot in milliseconds, base 36. The start keeps two processes apart
|
||||
// that a reboot or a pid namespace gave the same pid. A running step whose owner is another is a previous process's.
|
||||
var Owner = ownerOf(bootID, os.Getpid(), Boot)
|
||||
|
||||
// bootID is the node's boot id as Owner holds it; noBootID when it could not be read, which EndStaleAtStart says once.
|
||||
var bootID = readBootID()
|
||||
|
||||
const noBootID = "no-boot-id"
|
||||
|
||||
func ownerOf(bootID string, pid int, start time.Time) string {
|
||||
return bootID + "." + strconv.Itoa(pid) + "." + strconv.FormatInt(start.UnixMilli(), 36)
|
||||
}
|
||||
|
||||
// readBootID is the node's boot id, or noBootID where it cannot be read (the pid and the start then tell the processes
|
||||
// apart).
|
||||
func readBootID() string {
|
||||
b, err := os.ReadFile("/proc/sys/kernel/random/boot_id")
|
||||
if id := strings.TrimSpace(string(b)); err == nil && id != "" {
|
||||
return id
|
||||
}
|
||||
return noBootID
|
||||
}
|
||||
|
||||
// Scan reads and writes a steps state with its revisions: its keys, a key's stored value and revision (nil and 0 for
|
||||
// none), a write that lands only while the key is still at the revision read (KeptState.Update), and a delete of a key
|
||||
// still at the revision read; both fail with stdio.ErrChanged when another writer was first.
|
||||
type Scan struct {
|
||||
Keys func() ([]string, error)
|
||||
Get func(key string) (json.RawMessage, uint64, error)
|
||||
Update func(key string, value any, revision uint64) (uint64, error)
|
||||
Delete func(key string, revision uint64) error
|
||||
}
|
||||
|
||||
// ScanOf is the Scan of a state the process keeps.
|
||||
//
|
||||
// Its Delete reads the key again and deletes it only while it is still at the revision read: the runtime's
|
||||
// mesh/state.delete takes no revision yet, so a write landing between that read and the delete (a few milliseconds) is
|
||||
// deleted with it. For a step key that is a redelivered ask begun on this node in that moment: its running key leaves
|
||||
// the steps state until its run writes its end, which its asker still reads (a watch takes puts only).
|
||||
func ScanOf(s stdio.KeptState) Scan {
|
||||
get := func(key string) (json.RawMessage, uint64, error) {
|
||||
e, err := s.Get(key)
|
||||
if err != nil || e == nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
return e.Value, e.Revision, nil
|
||||
}
|
||||
return Scan{Keys: s.Keys, Update: s.Update, Get: get, Delete: func(key string, revision uint64) error {
|
||||
_, rev, err := get(key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if rev != revision {
|
||||
return stdio.ErrChanged
|
||||
}
|
||||
return s.Delete(key)
|
||||
}}
|
||||
}
|
||||
|
||||
// isAsk says a step is a pulled ask's: its key is <node>.<ask id> and its tool is "<kind> ask" (a hand-started step
|
||||
// has the tool's name). An ask's key is the stream's to settle, never a restart's: the puller leaves an ask it was
|
||||
// running unacknowledged so the stream redelivers it, and the asker's wait takes the first end it reads on the key.
|
||||
// A key a gone process owns is deleted, never ended (EndStale); one with no owner is ended only once the asker's bound
|
||||
// has passed (delivery.StepBound), when it is nobody's.
|
||||
func isAsk(key string, s delivery.Step) bool {
|
||||
_, _, err := delivery.ParseAskStepKey(key)
|
||||
return err == nil && strings.HasSuffix(s.Tool, " ask")
|
||||
}
|
||||
|
||||
// AskWhat says what an ask is, in one line: its kind, and its repository, head and levels, or its module and
|
||||
// fingerprint.
|
||||
func AskWhat(a delivery.StepAsk) string {
|
||||
short := func(s string) string {
|
||||
if len(s) > 12 {
|
||||
return s[:12]
|
||||
}
|
||||
return s
|
||||
}
|
||||
str := func(m map[string]any, k string) string { v, _ := m[k].(string); return v }
|
||||
parts := []string{a.Kind + " ask"}
|
||||
switch {
|
||||
case a.Check != nil:
|
||||
parts = append(parts, str(a.Check.Fetch, "repository")+"@"+short(str(a.Check.Fetch, "commit")), "levels "+strings.Join(a.Check.Levels, ","))
|
||||
if a.Check.Run != "" {
|
||||
parts = append(parts, "run "+a.Check.Run)
|
||||
}
|
||||
case a.Build != nil:
|
||||
parts = append(parts, "module "+a.Build.Module)
|
||||
if r := str(a.Build.Fetch, "repository"); r != "" {
|
||||
parts = append(parts, r+"@"+short(str(a.Build.Fetch, "commit")))
|
||||
}
|
||||
if a.Build.FP != "" {
|
||||
parts = append(parts, "fp "+short(a.Build.FP))
|
||||
}
|
||||
case a.Compose != nil:
|
||||
parts = append(parts, "compose", str(a.Compose.Fetch, "repository")+"@"+short(str(a.Compose.Fetch, "commit")))
|
||||
}
|
||||
return strings.Join(parts, ", ")
|
||||
}
|
||||
|
||||
// endRunning writes step ended failed with why, only while its key is still at revision.
|
||||
func endRunning(scan Scan, key string, step delivery.Step, revision uint64, why string) error {
|
||||
step.State, step.Reason, step.Ended, step.Result = delivery.StepFailed, why, now().UTC(), nil
|
||||
raw, err := delivery.EncodeStep(step)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = scan.Update(key, json.RawMessage(raw), revision)
|
||||
return err
|
||||
}
|
||||
|
||||
// EndStale ends this node's running steps that no process of this module runs any more, and answers their ids. Only keys
|
||||
// of node are read; a value that is no step is left alone.
|
||||
// - A hand-started step with an owner (delivery.Step.Owner) that is not this process (Owner) is ended failed with
|
||||
// RestartReason at once, whatever its age: its process is gone and nothing redelivers it, so nothing else would
|
||||
// write its end (issue 1890, note 21).
|
||||
// - A pulled ask's step with an owner that is not this process is deleted, not ended (novox/hq ADR 0525 §6): the
|
||||
// stream redelivers an ask whose node stopped (the puller neither ends nor acknowledges one at its stop), and its
|
||||
// asker takes the first end it reads, so an end written here would fail a check the redelivery would still run. Its
|
||||
// asker's watch takes puts only and keeps waiting for the redelivery's key, its bound the backstop; the stale running
|
||||
// key leaves the steps state, and with it the delivery view's and the checker's count of what runs (note 21: it
|
||||
// held one of a build node's two slots).
|
||||
// - A step owned by this process is left: it may run now. A redelivered ask begun on this node is owned by it.
|
||||
// - A step with no owner (written by an older build) keeps the earlier rule: a hand-started step started before
|
||||
// `before` (the start of this process) is ended; a pulled ask is left to the stream's redelivery (isAsk) until its
|
||||
// start is older than delivery.StepBound, and then ended "past the step bound: its asker stopped waiting at <time>".
|
||||
//
|
||||
// Each end and delete is logged, naming the step, its owner and why. A key written by another writer since it was read
|
||||
// (a redelivered ask's start, a real end) is skipped.
|
||||
func EndStale(scan Scan, node string, before time.Time, logf func(string, ...any)) ([]string, error) {
|
||||
keys, err := scan.Keys()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("buildstep: listing the steps of %s: %w", node, err)
|
||||
}
|
||||
var ended []string
|
||||
for _, k := range keys {
|
||||
n, _, perr := delivery.ParseStepKey(k)
|
||||
if perr != nil || n != node {
|
||||
continue
|
||||
}
|
||||
raw, rev, err := scan.Get(k)
|
||||
if err != nil {
|
||||
return ended, fmt.Errorf("buildstep: reading the step %s: %w", k, err)
|
||||
}
|
||||
if len(raw) == 0 {
|
||||
continue
|
||||
}
|
||||
step, err := delivery.DecodeStep(raw)
|
||||
if err != nil || step.State != delivery.StepRunning {
|
||||
continue
|
||||
}
|
||||
why, said := staleWhy(k, step, before)
|
||||
if why == "" {
|
||||
continue
|
||||
}
|
||||
done := "ended"
|
||||
if step.Owner != "" && isAsk(k, step) {
|
||||
done = "deleted, left to the stream's redelivery"
|
||||
if scan.Delete == nil {
|
||||
err = fmt.Errorf("the steps state takes no delete")
|
||||
} else {
|
||||
err = scan.Delete(k, rev)
|
||||
}
|
||||
} else {
|
||||
err = endRunning(scan, k, step, rev, why)
|
||||
}
|
||||
if err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
continue
|
||||
}
|
||||
return ended, fmt.Errorf("buildstep: ending the step %s: %w", k, err)
|
||||
}
|
||||
if logf != nil {
|
||||
logf("the step %s (%s, started %s) was left running and is %s: %s; %s", k, step.Tool,
|
||||
step.Started.UTC().Format(time.RFC3339), done, why, said)
|
||||
}
|
||||
ended = append(ended, step.ID)
|
||||
}
|
||||
return ended, nil
|
||||
}
|
||||
|
||||
// staleWhy is why a running step is ended (or, for an ask with an owner, deleted) by EndStale and what the log says of
|
||||
// its owner, or "" to leave it.
|
||||
func staleWhy(key string, step delivery.Step, before time.Time) (why, said string) {
|
||||
switch {
|
||||
case step.Owner == Owner:
|
||||
return "", ""
|
||||
case step.Owner != "":
|
||||
return RestartReason, "its owner " + step.Owner + " is not this process " + Owner
|
||||
case isAsk(key, step):
|
||||
if !step.Started.Add(delivery.StepBound).Before(now()) {
|
||||
return "", ""
|
||||
}
|
||||
return "past the step bound: its asker stopped waiting at " + step.Started.Add(delivery.StepBound).UTC().Format("2006-01-02T15:04:05Z"),
|
||||
"it names no owner (an older build wrote it)"
|
||||
case step.Started.Before(before):
|
||||
return RestartReason, "it names no owner (an older build wrote it) and started before this process"
|
||||
}
|
||||
return "", ""
|
||||
}
|
||||
|
||||
// EndOne ends this node's running step id (or the key <node>.<id>) as failed with why. A step that is not running is
|
||||
// refused, and so is one this process owns (Owner; for a step with no owner, one started since Boot: it may still run,
|
||||
// whether a pulled ask or a step a tool started), one live reports as held, and one whose key changed since it was read.
|
||||
func EndOne(scan Scan, node, id, why string, live func(id string) bool) (delivery.Step, error) {
|
||||
id = strings.TrimSpace(id)
|
||||
if n, rest, ok := strings.Cut(id, "."); ok {
|
||||
if n != node {
|
||||
return delivery.Step{}, fmt.Errorf("the step %q is of node %q, this module runs on %q", id, n, node)
|
||||
}
|
||||
id = rest
|
||||
}
|
||||
if strings.TrimSpace(why) == "" {
|
||||
return delivery.Step{}, fmt.Errorf("say why the step is ended (why)")
|
||||
}
|
||||
key, err := delivery.StepKey(node, id)
|
||||
if err != nil {
|
||||
return delivery.Step{}, err
|
||||
}
|
||||
raw, rev, err := scan.Get(key)
|
||||
if err != nil {
|
||||
return delivery.Step{}, err
|
||||
}
|
||||
if len(raw) == 0 {
|
||||
return delivery.Step{}, fmt.Errorf("no step %s in the steps state", key)
|
||||
}
|
||||
step, err := delivery.DecodeStep(raw)
|
||||
if err != nil {
|
||||
return delivery.Step{}, err
|
||||
}
|
||||
if step.State != delivery.StepRunning {
|
||||
return delivery.Step{}, fmt.Errorf("the step %s is %s, not running", key, step.State)
|
||||
}
|
||||
if step.Owner == Owner || (step.Owner == "" && !step.Started.Before(Boot)) || (live != nil && live(id)) {
|
||||
return delivery.Step{}, fmt.Errorf("the step %s was started by this process and may run now; only a step a previous process left is ended", key)
|
||||
}
|
||||
if err := endRunning(scan, key, step, rev, why); err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
return delivery.Step{}, fmt.Errorf("the step %s changed while it was being ended; read it again", key)
|
||||
}
|
||||
return delivery.Step{}, err
|
||||
}
|
||||
step.State, step.Reason = delivery.StepFailed, why
|
||||
return step, nil
|
||||
}
|
||||
|
||||
// StaleAttempts and StaleRetry bound EndStaleAtStart's tries: the steps state is readable once the runtime serves.
|
||||
var (
|
||||
StaleAttempts = 6
|
||||
StaleRetry = 5 * time.Second
|
||||
)
|
||||
|
||||
// StaleServed is what EndStaleAtStart waits for before it reads the steps state: the runtime serving (stdio.Served).
|
||||
// The conformance kit's left-running case (conformance.LeftRunning) hands it a closed channel, to run a module's own
|
||||
// start sweep in its test.
|
||||
var StaleServed = stdio.Served
|
||||
|
||||
// EndStaleAtStart is what a build module runs once at its start, in a goroutine: it waits for the runtime to serve,
|
||||
// then ends this node's steps a previous process left running (EndStale), trying again when the state cannot be read
|
||||
// yet. A step names its owner, so `before` (the start of this process) matters only for a step with no owner, written
|
||||
// by an older build: such a step started before it is a previous process's. When the node's boot id cannot be read it
|
||||
// says so once: Owner then tells processes apart by their pid and start only.
|
||||
func EndStaleAtStart(ctx context.Context, scan Scan, node string, before time.Time, logf func(string, ...any)) {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-StaleServed():
|
||||
}
|
||||
if bootID == noBootID && logf != nil {
|
||||
logf("the boot id of %s cannot be read (/proc/sys/kernel/random/boot_id): its steps' owner is %s, told apart by pid and start only", node, Owner)
|
||||
}
|
||||
for attempt := 1; ; attempt++ {
|
||||
ended, err := EndStale(scan, node, before, logf)
|
||||
if err == nil {
|
||||
if logf != nil {
|
||||
logf("%d steps of %s left running by a previous process ended", len(ended), node)
|
||||
}
|
||||
return
|
||||
}
|
||||
if logf != nil {
|
||||
logf("ending the steps left running (try %d of %d): %v", attempt, StaleAttempts, err)
|
||||
}
|
||||
if attempt >= StaleAttempts {
|
||||
return
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(StaleRetry):
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// EndStaleRound is EndStale as the module's periodic round runs it: at once, and only when the runtime serves (a round
|
||||
// before that is skipped, the start's EndStaleAtStart and the next round cover it). It ends the steps a previous
|
||||
// process left and the asks past delivery.StepBound.
|
||||
func EndStaleRound(scan Scan, node string, logf func(string, ...any)) {
|
||||
select {
|
||||
case <-stdio.Served():
|
||||
default:
|
||||
return
|
||||
}
|
||||
if _, err := EndStale(scan, node, Boot, logf); err != nil && logf != nil {
|
||||
logf("ending the steps left running: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,162 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
)
|
||||
|
||||
// scanOf is the steps a keys double holds, last value per key, its revision the count of its puts; a write through
|
||||
// Update lands only at the revision read, as the state's does. race, when set, runs once before the next Update's check.
|
||||
func scanOf(k *keys, race func(key string)) Scan {
|
||||
return Scan{
|
||||
Keys: func() ([]string, error) {
|
||||
k.mu.Lock()
|
||||
defer k.mu.Unlock()
|
||||
var out []string
|
||||
for key := range k.puts {
|
||||
out = append(out, key)
|
||||
}
|
||||
return out, nil
|
||||
},
|
||||
Get: func(key string) (json.RawMessage, uint64, error) {
|
||||
k.mu.Lock()
|
||||
defer k.mu.Unlock()
|
||||
s, ok := k.puts[key]
|
||||
if !ok {
|
||||
return nil, 0, nil
|
||||
}
|
||||
raw, err := delivery.EncodeStep(s[len(s)-1])
|
||||
return raw, uint64(len(s)), err
|
||||
},
|
||||
Delete: func(key string, revision uint64) error {
|
||||
k.mu.Lock()
|
||||
defer k.mu.Unlock()
|
||||
if uint64(len(k.puts[key])) != revision {
|
||||
return stdio.ErrChanged
|
||||
}
|
||||
delete(k.puts, key)
|
||||
return nil
|
||||
},
|
||||
Update: func(key string, value any, revision uint64) (uint64, error) {
|
||||
if race != nil {
|
||||
race(key)
|
||||
}
|
||||
k.mu.Lock()
|
||||
n := uint64(len(k.puts[key]))
|
||||
k.mu.Unlock()
|
||||
if n != revision {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
return k.put(key, value)
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// 1890 and 1890/AC6: EndStale ends this node's hand-started running steps of another owner at once, whatever their age,
|
||||
// and deletes its asks of another owner (the stream redelivers them, ADR 0525 §6); of
|
||||
// the steps with no owner (an older build's), the hand-started ones left running before the process began and the asks
|
||||
// past the step bound (a younger ask is the stream's to redeliver); never a step this process owns or a key written
|
||||
// meanwhile. EndOne ends one by hand, never one this process owns or started.
|
||||
func TestEndStaleAndEndOne(t *testing.T) {
|
||||
k := newKeys()
|
||||
boot := time.Now()
|
||||
oldBoot := Boot
|
||||
Boot = boot
|
||||
defer func() { Boot = oldBoot }()
|
||||
gone := ownerOf("b00t", 4242, boot.Add(-time.Hour))
|
||||
put := func(node, id, tool, state string, started time.Time, owner string) {
|
||||
key, _ := delivery.StepKey(node, id)
|
||||
_, _ = k.put(key, json.RawMessage(mustStep(t, delivery.Step{ID: id, Tool: tool, Node: node, State: state, Started: started, Owner: owner})))
|
||||
}
|
||||
put("ace", "old1", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour), "")
|
||||
put("ace", "mjk3l9x0abcdef", "check ask", delivery.StepRunning, boot.Add(-time.Hour), "")
|
||||
put("ace", "mjk3l9x0abcdee", "check ask", delivery.StepRunning, time.Now().Add(-5*time.Minute), "")
|
||||
put("ace", "mjk3l9x0abcded", "check ask", delivery.StepRunning, time.Now().Add(-5*time.Minute), gone) // note 21's ghost
|
||||
put("ace", "gone2", "build_go_test", delivery.StepRunning, boot.Add(time.Second), gone)
|
||||
put("ace", "mine1", "check ask", delivery.StepRunning, boot.Add(-time.Hour), Owner)
|
||||
put("ace", "raced1", "build_go_test", delivery.StepRunning, boot.Add(-time.Hour), "")
|
||||
put("ace", "new1", "build_go_fetch", delivery.StepRunning, boot.Add(time.Second), "")
|
||||
put("ace", "done1", "build_go_fetch", delivery.StepPassed, boot.Add(-time.Hour), gone)
|
||||
put("g14", "other1", "build_go_fetch", delivery.StepRunning, boot.Add(-time.Hour), gone)
|
||||
// raced1 is ended by its own run between the read and the write.
|
||||
raced := false
|
||||
scan := scanOf(k, func(key string) {
|
||||
if key == "ace.raced1" && !raced {
|
||||
raced = true
|
||||
put("ace", "raced1", "build_go_test", delivery.StepPassed, boot.Add(-time.Hour), "")
|
||||
}
|
||||
})
|
||||
var said []string
|
||||
ended, err := EndStale(scan, "ace", boot, func(f string, a ...any) { said = append(said, fmt.Sprintf(f, a...)) })
|
||||
if err != nil || len(ended) != 4 {
|
||||
t.Fatalf("ended %v, %v", ended, err)
|
||||
}
|
||||
if _, held := k.puts["ace.mjk3l9x0abcded"]; held {
|
||||
t.Errorf("the ghost ask's key is still held: %+v", k.last("ace.mjk3l9x0abcded"))
|
||||
}
|
||||
for _, key := range []string{"ace.old1", "ace.gone2"} {
|
||||
if s := k.last(key); s.State != delivery.StepFailed || s.Reason != RestartReason || s.Ended.IsZero() {
|
||||
t.Errorf("%s = %+v", key, s)
|
||||
}
|
||||
}
|
||||
if s := k.last("ace.mjk3l9x0abcdef"); s.State != delivery.StepFailed || !strings.HasPrefix(s.Reason, "past the step bound: its asker stopped waiting at ") {
|
||||
t.Errorf("the old ask = %+v", s)
|
||||
}
|
||||
if log := strings.Join(said, "\n"); !strings.Contains(log, "ace.mjk3l9x0abcded") || !strings.Contains(log, "deleted, left to the stream's redelivery") || !strings.Contains(log, "its owner "+gone+" is not this process "+Owner) {
|
||||
t.Errorf("the log does not name the ghost and its owner:\n%s", log)
|
||||
}
|
||||
for key, state := range map[string]string{"ace.mjk3l9x0abcdee": delivery.StepRunning, "ace.mine1": delivery.StepRunning, "ace.raced1": delivery.StepPassed,
|
||||
"ace.new1": delivery.StepRunning, "ace.done1": delivery.StepPassed, "g14.other1": delivery.StepRunning} {
|
||||
if s := k.last(key); s.State != state {
|
||||
t.Errorf("%s = %s, want %s", key, s.State, state)
|
||||
}
|
||||
}
|
||||
// EndOne: refuses without why, a held step, one this process owns or (with no owner) started, a step not running and
|
||||
// another node's; ends a step a previous process left.
|
||||
for _, c := range []struct{ id, why string }{{"old1", ""}, {"new1", "stuck"}, {"mine1", "stuck"}, {"done1", "x"}, {"g14.other1", "x"}} {
|
||||
if _, err := EndOne(scan, "ace", c.id, c.why, nil); err == nil {
|
||||
t.Errorf("ended %s with why %q", c.id, c.why)
|
||||
}
|
||||
}
|
||||
put("ace", "old2", "build_go_fetch", delivery.StepRunning, boot.Add(time.Second), gone)
|
||||
if _, err := EndOne(scan, "ace", "old2", "x", func(string) bool { return true }); err == nil {
|
||||
t.Error("ended a held step")
|
||||
}
|
||||
if _, err := EndOne(scan, "ace", "ace.old2", "stuck for good", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if s := k.last("ace.old2"); s.State != delivery.StepFailed || s.Reason != "stuck for good" || s.Owner != gone {
|
||||
t.Errorf("old2 = %+v", s)
|
||||
}
|
||||
if a, b := ownerOf("b", 1, boot), ownerOf("b", 1, boot.Add(time.Second)); a == b {
|
||||
t.Errorf("two starts of one pid share the owner %q", a)
|
||||
}
|
||||
}
|
||||
|
||||
func mustStep(t *testing.T, s delivery.Step) []byte {
|
||||
t.Helper()
|
||||
raw, err := delivery.EncodeStep(s)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
|
||||
// 1890: AskWhat names the repository, head and levels of a check ask and the module of a build ask.
|
||||
func TestAskWhat(t *testing.T) {
|
||||
check := delivery.StepAsk{Kind: "check", Check: &delivery.CheckJob{Fetch: map[string]any{"repository": "novox/mesh-sdk", "commit": "0123456789abcdef0123"}, Levels: []string{"unit", "integration"}}}
|
||||
if got, want := AskWhat(check), "check ask, novox/mesh-sdk@0123456789ab, levels unit,integration"; got != want {
|
||||
t.Errorf("check = %q", got)
|
||||
}
|
||||
build := delivery.StepAsk{Kind: "delivery", Build: &delivery.BuildJob{Module: "mesh-x", FP: "abcdef0123456789"}}
|
||||
if got, want := AskWhat(build), "delivery ask, module mesh-x, fp abcdef012345"; got != want {
|
||||
t.Errorf("build = %q", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
// Package buildstep is what both build modules share: the runner of a long step that answers a tool call at once
|
||||
// and ends on its key in the module's `steps` state (novox/hq ADR 0469 §2; issue 1471), the toolchain guard
|
||||
// (issue 1473), the fetch of one commit over SSH (issue 1474), and the failure contract of a test level with the parsers
|
||||
// of go test -json and node --test's TAP that fill it (failure.go, issue 1901; its conformance kit is package conformance).
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
var now = time.Now
|
||||
|
||||
// NewID is a step id: the time in milliseconds in base 36 and six random hex digits, lower-case letters and
|
||||
// digits only, so it is one key token and differs between two steps of one node even in one millisecond.
|
||||
func NewID() string {
|
||||
var b [3]byte
|
||||
_, _ = rand.Read(b[:])
|
||||
return strconv.FormatInt(now().UnixMilli(), 36) + hex.EncodeToString(b[:])
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
)
|
||||
|
||||
// keys is a steps state in memory: every value put, in order, per key, and a signal for each end.
|
||||
type keys struct {
|
||||
mu sync.Mutex
|
||||
puts map[string][]delivery.Step
|
||||
ended chan string
|
||||
}
|
||||
|
||||
func newKeys() *keys { return &keys{puts: map[string][]delivery.Step{}, ended: make(chan string, 16)} }
|
||||
|
||||
func (k *keys) put(key string, value any) (uint64, error) {
|
||||
step, err := delivery.DecodeStep(value.(json.RawMessage))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
k.mu.Lock()
|
||||
k.puts[key] = append(k.puts[key], step)
|
||||
n := uint64(len(k.puts[key]))
|
||||
k.mu.Unlock()
|
||||
if step.State != delivery.StepRunning {
|
||||
k.ended <- key
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
func (k *keys) last(key string) delivery.Step {
|
||||
k.mu.Lock()
|
||||
defer k.mu.Unlock()
|
||||
s := k.puts[key]
|
||||
return s[len(s)-1]
|
||||
}
|
||||
|
||||
func (k *keys) waitEnd(t *testing.T) string {
|
||||
t.Helper()
|
||||
select {
|
||||
case key := <-k.ended:
|
||||
return key
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the step never ended on its key")
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
// 1471/AC5: ids made in one and the same millisecond differ, and each is one key token. The clock is fixed
|
||||
// while no step runs, so no goroutine reads it meanwhile.
|
||||
func TestTwoIDsInOneMillisecondDiffer(t *testing.T) {
|
||||
was := now
|
||||
defer func() { now = was }()
|
||||
at := time.Date(2026, 10, 20, 12, 0, 0, 0, time.UTC)
|
||||
now = func() time.Time { return at }
|
||||
seen := map[string]bool{}
|
||||
for i := 0; i < 50; i++ {
|
||||
id := NewID()
|
||||
if seen[id] {
|
||||
t.Fatalf("id %q handed out twice", id)
|
||||
}
|
||||
seen[id] = true
|
||||
if _, err := delivery.StepKey("ace", id); err != nil {
|
||||
t.Errorf("id %q: %v", id, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
package buildstep
|
||||
|
||||
// ModuleTree is what a merge holds of one module directory, as a build module's preview answers it (novox/hq ADR 0484
|
||||
// clause 3, as ADR 0593 §2 changes it): its git tree id, the sha256 of its module.json, the sha256 of each lock file it
|
||||
// has (go.sum, package-lock.json), and, of a module whose every artifact names its paths, each named path's git object
|
||||
// id at the commit. The indexer alone computes from them (the named tree, the fingerprint); the build module's runner
|
||||
// (mesh-build-go's buildstep.ModuleTrees) fills them.
|
||||
type ModuleTree struct {
|
||||
Tree string `json:"tree"`
|
||||
Manifest string `json:"manifest"`
|
||||
Locks map[string]string `json:"locks,omitempty"`
|
||||
Paths map[string]string `json:"paths,omitempty"`
|
||||
// Packages are no longer filled (ADR 0593 §2): kept so a preview written before still decodes.
|
||||
Packages []string `json:"packages,omitempty"`
|
||||
}
|
||||
@@ -1,81 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A bus function called before Serve waits for Serve and proceeds (issue 1966): a module that starts a goroutine
|
||||
// before stdio.Serve and calls the bus there is no longer refused. It is refused only when the process never serves.
|
||||
// The test drives the wait itself: serveWait is the test's own timer channel, no row sleeps on a clock.
|
||||
func subBusFunctionsWaitForServe(t *testing.T) {
|
||||
const refusal = "before Serve: a bundle reaches the bus through the runtime that launched it"
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
serve string // "first": before the call; "during": while the call waits; "never"
|
||||
wantOK bool
|
||||
}{
|
||||
{"a call before Serve completes after Serve", "during", true},
|
||||
{"a call whose wait ends before Serve is refused", "never", false},
|
||||
{"a call after Serve is as before", "first", true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
servesAlone(t)
|
||||
expire := make(chan time.Time)
|
||||
oldAfter := serveTimer
|
||||
serveTimer = func() (<-chan time.Time, func()) { return expire, func() {} }
|
||||
t.Cleanup(func() { serveTimer = oldAfter })
|
||||
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
stopped := make(chan struct{})
|
||||
serve := func() {
|
||||
go func() { _ = ServeOn("m", nil, inR, outW); close(stopped) }()
|
||||
}
|
||||
t.Cleanup(func() { _ = inW.Close(); go func() { _, _ = io.Copy(io.Discard, outR) }() })
|
||||
|
||||
type result struct {
|
||||
c *channel
|
||||
err error
|
||||
}
|
||||
done := make(chan result, 1)
|
||||
if tc.serve == "first" {
|
||||
serve()
|
||||
<-Served()
|
||||
c, err := live()
|
||||
done <- result{c, err}
|
||||
} else {
|
||||
go func() { c, err := live(); done <- result{c, err} }()
|
||||
select {
|
||||
case r := <-done:
|
||||
t.Fatalf("live() returned before Serve and before its wait ended: %v", r.err)
|
||||
default:
|
||||
}
|
||||
if tc.serve == "during" {
|
||||
serve()
|
||||
} else {
|
||||
expire <- time.Time{}
|
||||
}
|
||||
}
|
||||
r := <-done
|
||||
if tc.wantOK != (r.err == nil && r.c != nil) {
|
||||
t.Fatalf("live() = %v, %v; want ok=%v", r.c, r.err, tc.wantOK)
|
||||
}
|
||||
if !tc.wantOK && r.err.Error() != refusal {
|
||||
t.Fatalf("refusal words changed: %v", r.err)
|
||||
}
|
||||
if tc.serve != "never" {
|
||||
_ = inW.Close()
|
||||
<-stopped
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestBusFunctionsWaitForServe runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
|
||||
func TestBusFunctionsWaitForServe(t *testing.T) {
|
||||
t.Run("BusFunctionsWaitForServe", subBusFunctionsWaitForServe)
|
||||
t.Run("ACapabilityWordIsReadOnceAndNamedWhenNotStated", subACapabilityWordIsReadOnceAndNamedWhenNotStated)
|
||||
t.Run("ABundleAsksAnAcknowledgedPublishAndASubscriptionForNewMessages", subABundleAsksAnAcknowledgedPublishAndASubscriptionForNewMessages)
|
||||
}
|
||||
@@ -1,162 +0,0 @@
|
||||
package busarchive
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
// Archive is a snapshot read back: its manifest, and each file beside it unpacked into a scratch
|
||||
// directory of its own, removed by Close.
|
||||
type Archive struct {
|
||||
Manifest Manifest
|
||||
dir string
|
||||
files map[string]string
|
||||
}
|
||||
|
||||
// ReadArchive unpacks a snapshot. Only the files the manifest names are kept, and only under names
|
||||
// that are one stream's directory: an archive is data, and a path in it is never trusted to say where
|
||||
// on the machine a file goes.
|
||||
func ReadArchive(r io.Reader, scratch string) (*Archive, error) {
|
||||
dir, err := os.MkdirTemp(scratch, "mesh-nats-restore-*")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
a := &Archive{dir: dir, files: map[string]string{}}
|
||||
tr := tar.NewReader(r)
|
||||
first := true
|
||||
for {
|
||||
hdr, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("the archive does not read: %w", err)
|
||||
}
|
||||
if first {
|
||||
if hdr.Name != ManifestName {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("the archive starts with %q, not its manifest: it is not a snapshot of the bus", hdr.Name)
|
||||
}
|
||||
raw, err := io.ReadAll(io.LimitReader(tr, 64<<20))
|
||||
if err != nil {
|
||||
a.Close()
|
||||
return nil, err
|
||||
}
|
||||
if err := json.Unmarshal(raw, &a.Manifest); err != nil {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("its manifest: %w", err)
|
||||
}
|
||||
if a.Manifest.Format != 1 {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("its manifest is format %d; this program reads format 1", a.Manifest.Format)
|
||||
}
|
||||
first = false
|
||||
continue
|
||||
}
|
||||
if !a.named(hdr.Name) {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("the archive holds %q, which its manifest does not name", hdr.Name)
|
||||
}
|
||||
local := filepath.Join(dir, fmt.Sprintf("%04d", len(a.files)))
|
||||
f, err := os.OpenFile(local, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
|
||||
if err != nil {
|
||||
a.Close()
|
||||
return nil, err
|
||||
}
|
||||
_, err = io.Copy(f, tr)
|
||||
f.Close()
|
||||
if err != nil {
|
||||
a.Close()
|
||||
return nil, err
|
||||
}
|
||||
a.files[hdr.Name] = local
|
||||
}
|
||||
if first {
|
||||
a.Close()
|
||||
return nil, errors.New("the archive is empty")
|
||||
}
|
||||
for _, s := range a.Manifest.Streams {
|
||||
if !ValidName(s.Name) || s.Meta != "streams/"+s.Name+"/backup.json" || s.Archive != "streams/"+s.Name+"/stream.tar.s2" {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("the manifest names a stream %q at paths that are not its own", s.Name)
|
||||
}
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
func (a *Archive) named(name string) bool {
|
||||
for _, s := range a.Manifest.Streams {
|
||||
if name == s.Meta || name == s.Archive {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Close removes the unpacked files.
|
||||
func (a *Archive) Close() { os.RemoveAll(a.dir) }
|
||||
|
||||
// Verify holds every file to the manifest's size and checksum, and reads every stream's archive to
|
||||
// its end.
|
||||
func (a *Archive) Verify() error {
|
||||
for _, s := range a.Manifest.Streams {
|
||||
for _, f := range []struct{ name, sum string }{{s.Meta, s.MetaSHA256}, {s.Archive, s.ArchiveSHA256}} {
|
||||
local, ok := a.files[f.name]
|
||||
if !ok {
|
||||
return fmt.Errorf("%s is named in the manifest and missing from the archive", f.name)
|
||||
}
|
||||
got, size, err := sha256Of(local)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if got != f.sum {
|
||||
return fmt.Errorf("%s does not match the checksum its manifest gives", f.name)
|
||||
}
|
||||
if f.name == s.Archive && size != s.ArchiveBytes {
|
||||
return fmt.Errorf("%s is %d bytes; its manifest says %d", f.name, size, s.ArchiveBytes)
|
||||
}
|
||||
}
|
||||
af, err := os.Open(a.files[s.Archive])
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = ReadsThrough(af)
|
||||
af.Close()
|
||||
if err != nil {
|
||||
return fmt.Errorf("%s does not read: %w", s.Archive, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func sha256Of(path string) (string, int64, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
defer f.Close()
|
||||
h := sha256.New()
|
||||
n, err := io.Copy(h, f)
|
||||
if err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
return hex.EncodeToString(h.Sum(nil)), n, nil
|
||||
}
|
||||
|
||||
// Open is one stream's meta and archive.
|
||||
func (a *Archive) Open(s StreamEntry) (meta []byte, archive *os.File, err error) {
|
||||
meta, err = os.ReadFile(a.files[s.Meta])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
archive, err = os.Open(a.files[s.Archive])
|
||||
return meta, archive, err
|
||||
}
|
||||
@@ -1,86 +0,0 @@
|
||||
// Package busarchive is the bus's snapshot archive: its manifest, reading one back and holding it to
|
||||
// its checksums. The snapshot program (cmd/mesh-nats-snapshot) writes and restores with it and the bus's
|
||||
// tool nats_snapshot_verify (cmd/nats-tools) checks with it, so the two never differ on what a good
|
||||
// snapshot is.
|
||||
package busarchive
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"io"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/klauspost/compress/s2"
|
||||
)
|
||||
|
||||
// Manifest is the first entry of an archive: what was taken, when, how long it took, and the checksum
|
||||
// of every file beside it. It is what `verify` and `restore` hold the archive to.
|
||||
type Manifest struct {
|
||||
Format int `json:"format"`
|
||||
Server string `json:"server_version"`
|
||||
Started time.Time `json:"started"`
|
||||
Finished time.Time `json:"finished"`
|
||||
Seconds float64 `json:"seconds"`
|
||||
// Messages and StreamBytes are the streams' own totals at their snapshots; ArchiveBytes what the
|
||||
// archives take, compressed.
|
||||
Messages uint64 `json:"messages"`
|
||||
StreamBytes int64 `json:"stream_bytes"`
|
||||
ArchiveBytes int64 `json:"archive_bytes"`
|
||||
Streams []StreamEntry `json:"streams"`
|
||||
// Skipped are streams that hold nothing across a restart (memory storage), said rather than
|
||||
// silently left out.
|
||||
Skipped []Skipped `json:"skipped,omitempty"`
|
||||
}
|
||||
|
||||
// StreamEntry is one stream as it was snapshotted.
|
||||
type StreamEntry struct {
|
||||
Name string `json:"name"`
|
||||
Subjects []string `json:"subjects,omitempty"`
|
||||
Retention string `json:"retention,omitempty"`
|
||||
Messages uint64 `json:"messages"`
|
||||
Bytes uint64 `json:"bytes"`
|
||||
FirstSeq uint64 `json:"first_seq"`
|
||||
LastSeq uint64 `json:"last_seq"`
|
||||
Deleted int `json:"deleted,omitempty"`
|
||||
Consumers int `json:"consumers"`
|
||||
// Meta is the server's own description of the stream (its configuration and state), in the
|
||||
// shape the restore API and the nats CLI take back: backup.json.
|
||||
Meta string `json:"meta"`
|
||||
MetaSHA256 string `json:"meta_sha256"`
|
||||
// Archive is the server's snapshot of the stream, as the server sent it: stream.tar.s2.
|
||||
Archive string `json:"archive"`
|
||||
ArchiveBytes int64 `json:"archive_bytes"`
|
||||
ArchiveSHA256 string `json:"archive_sha256"`
|
||||
Seconds float64 `json:"seconds"`
|
||||
}
|
||||
|
||||
// Skipped is a stream not snapshotted, and why.
|
||||
type Skipped struct {
|
||||
Name string `json:"name"`
|
||||
Why string `json:"why"`
|
||||
}
|
||||
|
||||
// ManifestName is the archive's first entry.
|
||||
const ManifestName = "manifest.json"
|
||||
|
||||
// ReadsThrough is whether a stream's archive decompresses and lists to its end.
|
||||
func ReadsThrough(r io.Reader) error {
|
||||
tr := tar.NewReader(s2.NewReader(r))
|
||||
for {
|
||||
_, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := io.Copy(io.Discard, tr); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ValidName is a stream name that is one safe path component.
|
||||
func ValidName(name string) bool {
|
||||
return name != "" && name != "." && name != ".." && !strings.ContainsAny(name, "/\\\x00 *>")
|
||||
}
|
||||
@@ -1,109 +0,0 @@
|
||||
package busarchive
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// StreamReport is one stream of a verified snapshot.
|
||||
type StreamReport struct {
|
||||
Name string `json:"name"`
|
||||
LastSeq uint64 `json:"last_seq"`
|
||||
}
|
||||
|
||||
// Report is the answer of VerifyFile: which file was held to its manifest, and whether it passed.
|
||||
type Report struct {
|
||||
File string `json:"file"`
|
||||
Modified time.Time `json:"modified"`
|
||||
Size int64 `json:"size"`
|
||||
Taken *time.Time `json:"taken,omitempty"`
|
||||
Streams []StreamReport `json:"streams"`
|
||||
Skipped []Skipped `json:"skipped,omitempty"`
|
||||
Passed bool `json:"passed"`
|
||||
// Fault is the first thing wrong, empty when the snapshot passed.
|
||||
Fault string `json:"fault,omitempty"`
|
||||
}
|
||||
|
||||
// VerifyFile verifies one snapshot inside dir: the file called name, or the newest *.tar when name is
|
||||
// empty. Nothing outside dir is read and nothing is written there; a snapshot that fails is an answer
|
||||
// (Passed false, the first Fault), and only a directory without a snapshot is an error.
|
||||
func VerifyFile(dir, name string) (*Report, error) {
|
||||
path, info, err := pick(dir, name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
r := &Report{File: filepath.Base(path), Modified: info.ModTime().UTC(), Size: info.Size(), Streams: []StreamReport{}}
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
r.Fault = err.Error()
|
||||
return r, nil
|
||||
}
|
||||
defer f.Close()
|
||||
a, err := ReadArchive(f, "")
|
||||
if err != nil {
|
||||
r.Fault = err.Error()
|
||||
return r, nil
|
||||
}
|
||||
defer a.Close()
|
||||
taken := a.Manifest.Finished
|
||||
r.Taken = &taken
|
||||
r.Skipped = a.Manifest.Skipped
|
||||
for _, s := range a.Manifest.Streams {
|
||||
r.Streams = append(r.Streams, StreamReport{Name: s.Name, LastSeq: s.LastSeq})
|
||||
}
|
||||
if err := a.Verify(); err != nil {
|
||||
r.Fault = err.Error()
|
||||
return r, nil
|
||||
}
|
||||
r.Passed = true
|
||||
return r, nil
|
||||
}
|
||||
|
||||
// pick names the regular file to verify, and nothing outside dir: a name is one path component, and a
|
||||
// symbolic link is not followed out of the directory.
|
||||
func pick(dir, name string) (string, fs.FileInfo, error) {
|
||||
if dir == "" {
|
||||
return "", nil, errors.New("the snapshots directory is not set")
|
||||
}
|
||||
if name != "" {
|
||||
if name != filepath.Base(name) || name == "." || name == ".." || strings.ContainsRune(name, 0) {
|
||||
return "", nil, fmt.Errorf("%q is not a file name inside the snapshots directory", name)
|
||||
}
|
||||
p := filepath.Join(dir, name)
|
||||
info, err := os.Lstat(p)
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("no snapshot %s in the snapshots directory: %w", name, err)
|
||||
}
|
||||
if !info.Mode().IsRegular() {
|
||||
return "", nil, fmt.Errorf("%s is not a regular file", name)
|
||||
}
|
||||
return p, info, nil
|
||||
}
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("the snapshots directory does not read: %w", err)
|
||||
}
|
||||
var best string
|
||||
var bestInfo fs.FileInfo
|
||||
for _, e := range entries {
|
||||
if !strings.HasSuffix(e.Name(), ".tar") {
|
||||
continue
|
||||
}
|
||||
info, err := e.Info()
|
||||
if err != nil || !info.Mode().IsRegular() {
|
||||
continue
|
||||
}
|
||||
if bestInfo == nil || info.ModTime().After(bestInfo.ModTime()) {
|
||||
best, bestInfo = e.Name(), info
|
||||
}
|
||||
}
|
||||
if bestInfo == nil {
|
||||
return "", nil, errors.New("the snapshots directory holds no .tar snapshot")
|
||||
}
|
||||
return filepath.Join(dir, best), bestInfo, nil
|
||||
}
|
||||
@@ -1,182 +0,0 @@
|
||||
package busarchive
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/klauspost/compress/s2"
|
||||
)
|
||||
|
||||
func sum(b []byte) string { h := sha256.Sum256(b); return hex.EncodeToString(h[:]) }
|
||||
|
||||
// streamArchive is a stream's archive as the server sends it: a tar of one file, s2-compressed.
|
||||
func streamArchive(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
var raw bytes.Buffer
|
||||
tw := tar.NewWriter(&raw)
|
||||
body := []byte(strings.Repeat("a message of the stream\n", 50))
|
||||
if err := tw.WriteHeader(&tar.Header{Name: "msgs.blk", Mode: 0o600, Size: int64(len(body))}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tw.Write(body)
|
||||
tw.Close()
|
||||
var out bytes.Buffer
|
||||
w := s2.NewWriter(&out)
|
||||
w.Write(raw.Bytes())
|
||||
w.Close()
|
||||
return out.Bytes()
|
||||
}
|
||||
|
||||
// snapshotTar builds a snapshot of two streams; the mutations change what the manifest says or what is
|
||||
// stored, so each fault is made in the one place it is found.
|
||||
func snapshotTar(t *testing.T, mutate func(m *Manifest, files map[string][]byte)) []byte {
|
||||
t.Helper()
|
||||
files := map[string][]byte{}
|
||||
m := Manifest{Format: 1, Server: "2.11.17", Finished: time.Date(2026, 10, 22, 3, 0, 0, 0, time.UTC)}
|
||||
for i, name := range []string{"EVENTS", "KV_conditions"} {
|
||||
meta := []byte(`{"config":{"name":"` + name + `"}}`)
|
||||
arc := streamArchive(t)
|
||||
e := StreamEntry{Name: name, LastSeq: uint64(10 * (i + 1)),
|
||||
Meta: "streams/" + name + "/backup.json", MetaSHA256: sum(meta),
|
||||
Archive: "streams/" + name + "/stream.tar.s2", ArchiveBytes: int64(len(arc)), ArchiveSHA256: sum(arc)}
|
||||
m.Streams = append(m.Streams, e)
|
||||
files[e.Meta], files[e.Archive] = meta, arc
|
||||
}
|
||||
if mutate != nil {
|
||||
mutate(&m, files)
|
||||
}
|
||||
var out bytes.Buffer
|
||||
tw := tar.NewWriter(&out)
|
||||
put := func(name string, b []byte) {
|
||||
if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0o600, Size: int64(len(b))}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tw.Write(b)
|
||||
}
|
||||
raw, _ := json.Marshal(m)
|
||||
put(ManifestName, raw)
|
||||
for _, s := range m.Streams {
|
||||
put(s.Meta, files[s.Meta])
|
||||
put(s.Archive, files[s.Archive])
|
||||
}
|
||||
tw.Close()
|
||||
return out.Bytes()
|
||||
}
|
||||
|
||||
func put(t *testing.T, dir, name string, b []byte, at time.Time) {
|
||||
t.Helper()
|
||||
p := filepath.Join(dir, name)
|
||||
if err := os.WriteFile(p, b, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
os.Chtimes(p, at, at)
|
||||
}
|
||||
|
||||
func subAGoodSnapshotPassesAndSaysItsStreams(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
good := snapshotTar(t, nil)
|
||||
put(t, dir, "bus.tar", good, time.Now())
|
||||
r, err := VerifyFile(dir, "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !r.Passed || r.Fault != "" || r.File != "bus.tar" || r.Size != int64(len(good)) {
|
||||
t.Fatalf("%+v", r)
|
||||
}
|
||||
if r.Taken == nil || !r.Taken.Equal(time.Date(2026, 10, 22, 3, 0, 0, 0, time.UTC)) {
|
||||
t.Errorf("taken %v", r.Taken)
|
||||
}
|
||||
if len(r.Streams) != 2 || r.Streams[0] != (StreamReport{"EVENTS", 10}) || r.Streams[1] != (StreamReport{"KV_conditions", 20}) {
|
||||
t.Errorf("streams %+v", r.Streams)
|
||||
}
|
||||
}
|
||||
|
||||
func subACorruptedChecksumFailsNamingTheFile(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
put(t, dir, "bus.tar", snapshotTar(t, func(m *Manifest, files map[string][]byte) {
|
||||
m.Streams[1].ArchiveSHA256 = sum([]byte("something else"))
|
||||
}), time.Now())
|
||||
r, err := VerifyFile(dir, "bus.tar")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if r.Passed || !strings.Contains(r.Fault, "streams/KV_conditions/stream.tar.s2") || !strings.Contains(r.Fault, "checksum") {
|
||||
t.Fatalf("%+v", r)
|
||||
}
|
||||
if len(r.Streams) != 2 {
|
||||
t.Errorf("a failed snapshot still says its streams: %+v", r.Streams)
|
||||
}
|
||||
}
|
||||
|
||||
func subATruncatedSnapshotFails(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
good := snapshotTar(t, nil)
|
||||
put(t, dir, "bus.tar", good[:len(good)*2/3], time.Now())
|
||||
r, err := VerifyFile(dir, "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if r.Passed || r.Fault == "" {
|
||||
t.Fatalf("a truncated snapshot passed: %+v", r)
|
||||
}
|
||||
}
|
||||
|
||||
func subAnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
put(t, dir, "bus.tar", snapshotTar(t, func(m *Manifest, files map[string][]byte) {
|
||||
bad := []byte("not an s2 stream at all")
|
||||
s := &m.Streams[0]
|
||||
files[s.Archive] = bad
|
||||
s.ArchiveSHA256, s.ArchiveBytes = sum(bad), int64(len(bad))
|
||||
}), time.Now())
|
||||
r, _ := VerifyFile(dir, "")
|
||||
if r.Passed || !strings.Contains(r.Fault, "does not read") {
|
||||
t.Fatalf("%+v", r)
|
||||
}
|
||||
}
|
||||
|
||||
func subTheNewestTarIsVerifiedAndOnlyInsideTheDirectory(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
now := time.Now()
|
||||
put(t, dir, "old.tar", []byte("junk"), now.Add(-48*time.Hour))
|
||||
put(t, dir, "bus.tar", snapshotTar(t, nil), now)
|
||||
put(t, dir, "bus.tar.partial", []byte("half"), now.Add(time.Hour))
|
||||
r, err := VerifyFile(dir, "")
|
||||
if err != nil || r.File != "bus.tar" || !r.Passed {
|
||||
t.Fatalf("%+v %v", r, err)
|
||||
}
|
||||
if r, _ := VerifyFile(dir, "old.tar"); r.Passed || r.File != "old.tar" {
|
||||
t.Errorf("a named file is the one checked: %+v", r)
|
||||
}
|
||||
outside := t.TempDir()
|
||||
put(t, outside, "x.tar", snapshotTar(t, nil), now)
|
||||
for _, name := range []string{"../" + filepath.Base(outside) + "/x.tar", outside + "/x.tar", ".."} {
|
||||
if _, err := VerifyFile(dir, name); err == nil {
|
||||
t.Errorf("%q was read", name)
|
||||
}
|
||||
}
|
||||
os.Symlink(filepath.Join(outside, "x.tar"), filepath.Join(dir, "link.tar"))
|
||||
if _, err := VerifyFile(dir, "link.tar"); err == nil {
|
||||
t.Error("a symbolic link out of the directory was followed")
|
||||
}
|
||||
if _, err := VerifyFile(t.TempDir(), ""); err == nil {
|
||||
t.Error("an empty directory is not a snapshot")
|
||||
}
|
||||
}
|
||||
|
||||
// TestAGoodSnapshotPassesAndSaysItsStreams runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
|
||||
func TestAGoodSnapshotPassesAndSaysItsStreams(t *testing.T) {
|
||||
t.Run("AGoodSnapshotPassesAndSaysItsStreams", subAGoodSnapshotPassesAndSaysItsStreams)
|
||||
t.Run("ACorruptedChecksumFailsNamingTheFile", subACorruptedChecksumFailsNamingTheFile)
|
||||
t.Run("ATruncatedSnapshotFails", subATruncatedSnapshotFails)
|
||||
t.Run("AnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums", subAnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums)
|
||||
t.Run("TheNewestTarIsVerifiedAndOnlyInsideTheDirectory", subTheNewestTarIsVerifiedAndOnlyInsideTheDirectory)
|
||||
}
|
||||
@@ -1,50 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import "encoding/json"
|
||||
|
||||
// Unwrap reads a tool's answer whatever wraps it on its way from the tool runner (the lead's ruling 2026-11-04 13:12:
|
||||
// one reading, not a copy per module): a JSON string holding the answer; the protocol's content list, whose first
|
||||
// text is the answer and whose isError says it failed; the {output, ok, answer} form; the tool runner's {result,
|
||||
// error} envelope, whose error is the call's. It answers the answer itself, the words said with it (the output, the
|
||||
// error, or the answer's own text), and whether the tool says it succeeded. A bare answer is itself, succeeded.
|
||||
func Unwrap(raw json.RawMessage) (answer json.RawMessage, said string, ok bool) {
|
||||
var s string
|
||||
if json.Unmarshal(raw, &s) == nil {
|
||||
return Unwrap(json.RawMessage(s))
|
||||
}
|
||||
var m map[string]json.RawMessage
|
||||
if json.Unmarshal(raw, &m) != nil {
|
||||
return raw, string(raw), true
|
||||
}
|
||||
if content, has := m["content"]; has {
|
||||
var items []struct {
|
||||
Text string `json:"text"`
|
||||
}
|
||||
var e struct {
|
||||
IsError bool `json:"isError"`
|
||||
}
|
||||
_ = json.Unmarshal(raw, &e)
|
||||
if json.Unmarshal(content, &items) == nil && len(items) > 0 {
|
||||
inner, out, ok := Unwrap(json.RawMessage(items[0].Text))
|
||||
return inner, out, ok && !e.IsError
|
||||
}
|
||||
}
|
||||
var v struct {
|
||||
Output string `json:"output"`
|
||||
OK *bool `json:"ok"`
|
||||
Answer json.RawMessage `json:"answer"`
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
if json.Unmarshal(raw, &v) == nil {
|
||||
switch {
|
||||
case v.OK != nil:
|
||||
return v.Answer, v.Output, *v.OK
|
||||
case v.Error != "":
|
||||
return nil, v.Error, false
|
||||
case len(v.Result) > 0:
|
||||
return Unwrap(v.Result)
|
||||
}
|
||||
}
|
||||
return raw, string(raw), true
|
||||
}
|
||||
@@ -1,497 +0,0 @@
|
||||
// Package buscalls is the contract of the calls a bundle makes on its own bus state through the tool runner, over the
|
||||
// bundle's pipe (novox/hq ADR 0595 §9, ADR 0597 §10, design 25 § 12.5): the object store's, a stream's read-back and
|
||||
// fill, a work queue's size and dead letters, and the ledger's. Each call names an entry by the local name its module
|
||||
// asks for it by in `requires`; the tool runner serves it from the module's membership's bindings, its own entries
|
||||
// only. Only the shapes and the method names are here: what the tool runner refuses, and why, is its own.
|
||||
//
|
||||
// It is a package of its own, imported only by the modules that use these calls and by the tool runner, so a change
|
||||
// here rebuilds those and not the module library's importers (ADR 0595 §9). It imports nothing of mesh-sdk.
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// MetaCall is where in a `tools/call`'s `_meta` the tool runner hands a bundle the id of the tool call in flight: the id
|
||||
// ApplyRequest.Call names, so the runner stamps the caller it proved for that call as the operation's author (novox/hq
|
||||
// ADR 0597 §7). The runner writes `_meta`; a caller writes only `arguments`, so no caller can name another call.
|
||||
const MetaCall = "mesh/call"
|
||||
|
||||
// The methods a bundle asks the tool runner, one per call.
|
||||
const (
|
||||
ObjectsPut = "mesh/objects.put"
|
||||
ObjectsGet = "mesh/objects.get"
|
||||
ObjectsList = "mesh/objects.list"
|
||||
ObjectsDelete = "mesh/objects.delete"
|
||||
ObjectsInfo = "mesh/objects.info"
|
||||
ObjectsWatch = "mesh/objects.watch"
|
||||
// ObjectsSentGet reads an object of a sender's store of sent objects by the reference a reply or an event carried
|
||||
// (design 47 G §347, §407), served under the standing grant the grants part derives from the manifests; the answer is
|
||||
// ObjectData.
|
||||
ObjectsSentGet = "mesh/objects.sent-get"
|
||||
// ObjectsPutChunk and ObjectsGetChunk are the put session's and the get session's calls (design 47 G §518, §345, §490),
|
||||
// under the capability word put-sessions (stdio.CapPutSessions): a value larger than ChunkBytes is put as chunks in order,
|
||||
// each answered before the next, and a get that answers ObjectsChunked is read chunk by chunk.
|
||||
ObjectsPutChunk = "mesh/objects.put-chunk"
|
||||
ObjectsGetChunk = "mesh/objects.get-chunk"
|
||||
|
||||
StreamRead = "mesh/stream.read"
|
||||
StreamFill = "mesh/stream.fill"
|
||||
// StreamPublish says one event on the module's own bound stream (A-42).
|
||||
StreamPublish = "mesh/stream.publish"
|
||||
|
||||
WorkQueueSize = "mesh/workqueue.size"
|
||||
WorkQueueDead = "mesh/workqueue.dead"
|
||||
|
||||
LedgerApply = "mesh/ledger.apply"
|
||||
LedgerGet = "mesh/ledger.get"
|
||||
LedgerHistory = "mesh/ledger.history"
|
||||
LedgerWatch = "mesh/ledger.watch"
|
||||
LedgerQuery = "mesh/ledger.query"
|
||||
// LedgerFill is one ledger's history's fill against its cap, asked with a LedgerRequest and answered as a Fill
|
||||
// (design 47 G §462, §587); LedgerRemove removes one instance, asked with a LedgerRemoveRequest and answered as
|
||||
// LedgerRemoved. Both are under the capability word LedgerUpkeepCalls.
|
||||
LedgerFill = "mesh/ledger.fill"
|
||||
LedgerRemove = "mesh/ledger.remove"
|
||||
// The ledger's repairs, which the tool runner also serves as tools of every module owning a ledger, its own
|
||||
// ledgers only (design 25 § 12.5).
|
||||
LedgerRebuild = "mesh/ledger.rebuild"
|
||||
LedgerReindex = "mesh/ledger.reindex"
|
||||
LedgerSnapshot = "mesh/ledger.snapshot"
|
||||
LedgerLag = "mesh/ledger.lag"
|
||||
)
|
||||
|
||||
// The capability words of the families this package's calls belong to (design 47 G §488, §622): a tool runner states a
|
||||
// word once it answers the family, and a bundle fails at once, naming the word and its node, on a call whose word is not
|
||||
// stated. Which calls each word covers is the tool runner's (§462).
|
||||
const (
|
||||
BusStateCalls = "bus-state-calls"
|
||||
LedgerCalls = "ledger-calls"
|
||||
LedgerUpkeepCalls = "ledger-upkeep-calls"
|
||||
SentObjectsCalls = "sent-objects-calls"
|
||||
// StreamPublishWord is the family of StreamPublish alone, the word a module asks for in `capabilities` before it
|
||||
// says an event on its own bound stream (design 42 zfs-owns-zfs, A-44).
|
||||
StreamPublishWord = "stream-publish"
|
||||
)
|
||||
|
||||
// The methods the tool runner sends the bundle for a watch it asked, each naming the watch by its id, as `mesh/log`
|
||||
// does for a log.
|
||||
const (
|
||||
ObjectsWatched = "mesh/objects"
|
||||
LedgerWatched = "mesh/ledger"
|
||||
)
|
||||
|
||||
// Fill is how much an entry holds against its cap: its messages (or objects), its bytes and its cap in bytes.
|
||||
type Fill struct {
|
||||
Count uint64 `json:"count"`
|
||||
Bytes uint64 `json:"bytes"`
|
||||
MaxBytes uint64 `json:"max_bytes"`
|
||||
}
|
||||
|
||||
// Done is the answer of a call that answers nothing but that it was done.
|
||||
type Done struct{}
|
||||
|
||||
// --- the object store
|
||||
|
||||
// ObjectsPutRequest stores one object whole under its name, replacing one of that name.
|
||||
type ObjectsPutRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Name string `json:"name"`
|
||||
Data []byte `json:"data"`
|
||||
Description string `json:"description,omitempty"`
|
||||
}
|
||||
|
||||
// ChunkBytes is the size of one chunk of a put or get session, and so the largest value one call puts or gets (design 47 G
|
||||
// §518, A-21): it fits under the bus's default maximum payload, whatever the bus sets it to.
|
||||
const ChunkBytes = 512 << 10
|
||||
|
||||
// ObjectsPutChunkRequest is one chunk of a put session (design 47 G §518, A-21): the object `name` in the store `objects`,
|
||||
// `call` the bundle's id for this put, `seq` from 1 and one more each chunk, `last` true on the last chunk alone, and the
|
||||
// chunk's bytes; the first chunk also carries the put's `description` (A-25).
|
||||
type ObjectsPutChunkRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Name string `json:"name"`
|
||||
Call string `json:"call"`
|
||||
Seq uint32 `json:"seq"`
|
||||
Last bool `json:"last,omitempty"`
|
||||
Data []byte `json:"data"`
|
||||
// Description is the put's, carried on the first chunk alone (design 47 G A-25).
|
||||
Description string `json:"description,omitempty"`
|
||||
}
|
||||
|
||||
// ObjectsChunkTaken is the answer to a chunk that is not the last: the sequence taken, or the reason it was not.
|
||||
type ObjectsChunkTaken struct {
|
||||
Seq uint32 `json:"seq"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// ObjectsChunked is the answer of a get of a value over one chunk (design 47 G §490, §518, §620): the count of chunks (or
|
||||
// true, for a sent object), the whole size, its `sha256:` digest and the session's `call`. Its Chunked is absent from the
|
||||
// answer of a value read in one call.
|
||||
type ObjectsChunked struct {
|
||||
Chunked json.RawMessage `json:"chunked"`
|
||||
Size uint64 `json:"size"`
|
||||
Digest string `json:"digest"`
|
||||
Call string `json:"call"`
|
||||
}
|
||||
|
||||
// ObjectsGetChunkRequest asks one chunk of a get session by its call and sequence.
|
||||
type ObjectsGetChunkRequest struct {
|
||||
Call string `json:"call"`
|
||||
Seq uint32 `json:"seq"`
|
||||
}
|
||||
|
||||
// ObjectsChunk is a get session's chunk: the sequence, whether it is the last, its bytes, or the reason it is none.
|
||||
type ObjectsChunk struct {
|
||||
Seq uint32 `json:"seq"`
|
||||
Last bool `json:"last"`
|
||||
Data []byte `json:"data"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// ObjectsNameRequest names one object of an object store: get, delete.
|
||||
type ObjectsNameRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
// ObjectsRequest names an object store: list, info.
|
||||
type ObjectsRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
}
|
||||
|
||||
// ObjectsWatchRequest asks the tool runner to send every object stored or deleted, as ObjectsChange under the watch's id.
|
||||
type ObjectsWatchRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Watch string `json:"watch"`
|
||||
}
|
||||
|
||||
// ObjectInfo is one object as its store holds it: put's answer, and each of list's.
|
||||
type ObjectInfo struct {
|
||||
Name string `json:"name"`
|
||||
Size uint64 `json:"size"`
|
||||
Chunks uint32 `json:"chunks"`
|
||||
Digest string `json:"digest"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Modified time.Time `json:"modified"`
|
||||
Deleted bool `json:"deleted,omitempty"`
|
||||
}
|
||||
|
||||
// ObjectsSentGetRequest names one sent object by the reference that carried it (stateobjects.Ref's fields, which this
|
||||
// package does not import): the sender's store, the object's name, and its size and digest, which the reader checks.
|
||||
type ObjectsSentGetRequest struct {
|
||||
Store string `json:"store"`
|
||||
Object string `json:"object"`
|
||||
Size int64 `json:"size"`
|
||||
Digest string `json:"sha256"`
|
||||
}
|
||||
|
||||
// ObjectData is get's answer: the object's info and its bytes.
|
||||
type ObjectData struct {
|
||||
Info ObjectInfo `json:"info"`
|
||||
Data []byte `json:"data"`
|
||||
}
|
||||
|
||||
// ObjectsListed is list's answer: every object the store holds.
|
||||
type ObjectsListed struct {
|
||||
Objects []ObjectInfo `json:"objects"`
|
||||
}
|
||||
|
||||
// ObjectsChange is one object stored or deleted, sent under ObjectsWatched; Current says it was there when the watch
|
||||
// began.
|
||||
type ObjectsChange struct {
|
||||
Watch string `json:"watch"`
|
||||
Object ObjectInfo `json:"object"`
|
||||
Current bool `json:"current"`
|
||||
}
|
||||
|
||||
// --- a stream
|
||||
|
||||
// StreamReadRequest reads a module's own stream from a sequence on, at most Limit messages, Subjects narrowing it.
|
||||
type StreamReadRequest struct {
|
||||
Stream string `json:"stream"`
|
||||
From uint64 `json:"from"`
|
||||
Subjects string `json:"subjects,omitempty"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// StreamRequest names a stream: fill.
|
||||
type StreamRequest struct {
|
||||
Stream string `json:"stream"`
|
||||
}
|
||||
|
||||
// StreamMessage is one message of a stream.
|
||||
type StreamMessage struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Subject string `json:"subject"`
|
||||
Body json.RawMessage `json:"body"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// StreamPage is read's answer: the messages in order, the sequence to read on from (0 when nothing follows) and the
|
||||
// stream's last sequence when it was read.
|
||||
type StreamPage struct {
|
||||
Messages []StreamMessage `json:"messages"`
|
||||
Next uint64 `json:"next"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// StreamPublishRequest says one event on a module's own stream, by the stream's local name: the tool runner publishes
|
||||
// Body on the binding's subject plus "." plus Event, acknowledged, with ID as the message id (design 42 zfs-owns-zfs, A-42).
|
||||
type StreamPublishRequest struct {
|
||||
Stream string `json:"stream"`
|
||||
Event string `json:"event"`
|
||||
Body json.RawMessage `json:"body"`
|
||||
ID string `json:"id,omitempty"`
|
||||
}
|
||||
|
||||
// StreamPublished is publish's answer: the sequence the stream acknowledged the event at.
|
||||
type StreamPublished struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
}
|
||||
|
||||
// StreamFilled is fill's answer: the stream's fill and its first and last sequence.
|
||||
type StreamFilled struct {
|
||||
Fill
|
||||
First uint64 `json:"first"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// --- a work queue
|
||||
|
||||
// WorkQueueRequest names a work queue: size.
|
||||
type WorkQueueRequest struct {
|
||||
Queue string `json:"queue"`
|
||||
}
|
||||
|
||||
// WorkQueueDeadRequest asks a work queue's dead letters, at most Limit of them.
|
||||
type WorkQueueDeadRequest struct {
|
||||
Queue string `json:"queue"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// WorkQueueSized is size's answer: the messages waiting to be taken, against the queue's cap.
|
||||
type WorkQueueSized struct {
|
||||
Fill
|
||||
}
|
||||
|
||||
// DeadLetter is one message delivered as often as its queue allows and never settled.
|
||||
type DeadLetter struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Subject string `json:"subject"`
|
||||
Deliveries int `json:"deliveries"`
|
||||
Body json.RawMessage `json:"body,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// WorkQueueDeadLetters is dead's answer.
|
||||
type WorkQueueDeadLetters struct {
|
||||
Dead []DeadLetter `json:"dead"`
|
||||
}
|
||||
|
||||
// --- the ledger (ADR 0597)
|
||||
|
||||
// ApplyRequest applies one operation to one instance of a ledger. Call is the id of the tool call the operation
|
||||
// serves, as the tool runner handed it under MetaCall, whose caller the tool runner proved and stamps as the author
|
||||
// (ADR 0597 §7); empty, the operation carries the owning module's account alone. An author the bundle writes into Body
|
||||
// is never taken as the author.
|
||||
type ApplyRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
Operation string `json:"operation"`
|
||||
Body json.RawMessage `json:"body,omitempty"`
|
||||
Call string `json:"call,omitempty"`
|
||||
}
|
||||
|
||||
// Applied is apply's answer: the operation was taken, with its sequence in the instance's history; StateBehind says
|
||||
// the state could not follow because its bucket is full, and catches up at the next apply after room is made (ADR
|
||||
// 0597 §6).
|
||||
type Applied struct {
|
||||
Taken bool `json:"taken"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
StateBehind bool `json:"state_behind,omitempty"`
|
||||
}
|
||||
|
||||
// Author is who an operation is by, as `apply` stamped it (ADR 0597 §7): the owning module's account, and the caller
|
||||
// of the tool call it served, as the tool runner proved it.
|
||||
type Author struct {
|
||||
Account string `json:"account"`
|
||||
Caller string `json:"caller,omitempty"`
|
||||
}
|
||||
|
||||
// InstanceRequest names one instance of a ledger: get, snapshot.
|
||||
type InstanceRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
}
|
||||
|
||||
// InstanceState is get's answer: the instance's state and the sequence of the last operation it holds.
|
||||
type InstanceState struct {
|
||||
Instance string `json:"instance"`
|
||||
State json.RawMessage `json:"state"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
}
|
||||
|
||||
// HistoryRequest reads an instance's operations from a sequence on, at most Limit of them.
|
||||
type HistoryRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
From uint64 `json:"from,omitempty"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// Operation is one operation as the history keeps it.
|
||||
type Operation struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Instance string `json:"instance"`
|
||||
Operation string `json:"operation"`
|
||||
Body json.RawMessage `json:"body,omitempty"`
|
||||
Author Author `json:"author"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// HistoryPage is history's answer: the operations oldest first, the sequence to read on from (0 when nothing
|
||||
// follows) and the instance's last sequence when it was read.
|
||||
type HistoryPage struct {
|
||||
Operations []Operation `json:"operations"`
|
||||
Next uint64 `json:"next"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// LedgerWatchRequest asks the tool runner to send every change of the instances Instances names (an instance, with
|
||||
// `*` for one name and `>` for the rest) as InstanceChange under the watch's id: the current states first.
|
||||
type LedgerWatchRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instances string `json:"instances"`
|
||||
Watch string `json:"watch"`
|
||||
}
|
||||
|
||||
// InstanceChange is one instance's new state, sent under LedgerWatched; Current says it was the state when the watch
|
||||
// began.
|
||||
type InstanceChange struct {
|
||||
Watch string `json:"watch"`
|
||||
Instance string `json:"instance"`
|
||||
State json.RawMessage `json:"state"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Current bool `json:"current"`
|
||||
}
|
||||
|
||||
// QueryRequest reads a declared index of a ledger: the entries whose key Key names (a key, with `*` for one name and
|
||||
// `>` for the rest; empty is every key), at most Limit of them.
|
||||
type QueryRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Index string `json:"index"`
|
||||
Key string `json:"key,omitempty"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// IndexEntry is the value of one key of a declared index (IndexKey): the instance it points at, and the sequence of
|
||||
// the operation it was written from. The owner's watcher and the tool runner's reindex both write it so.
|
||||
type IndexEntry struct {
|
||||
Instance string `json:"instance"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
}
|
||||
|
||||
// IndexKey is the key of an instance in a declared index: the values of the index's fields, in its order, then the
|
||||
// parts of the instance's path, all joined by ".". Each value and each path part is encoded (keyToken): every byte
|
||||
// outside [A-Za-z0-9_-] is written "=" and two upper-case hex digits ("." is "=2E", " " is "=20", "=" is "=3D"), so two
|
||||
// different keys never meet and every part is a token the bus takes. `17950.listing.a17` is the key of price 17950
|
||||
// and instance `listing/a17`. The owner's watcher and the tool runner's reindex both write it, so a reindex leaves the
|
||||
// index as the watcher keeps it.
|
||||
func IndexKey(fields []string, instance string) string {
|
||||
parts := make([]string, 0, len(fields)+2)
|
||||
for _, f := range fields {
|
||||
parts = append(parts, keyToken(f))
|
||||
}
|
||||
for _, p := range strings.Split(instance, "/") {
|
||||
parts = append(parts, keyToken(p))
|
||||
}
|
||||
return strings.Join(parts, ".")
|
||||
}
|
||||
|
||||
// keyToken is v with every byte outside [A-Za-z0-9_-] written "=" and two upper-case hex digits (IndexKey).
|
||||
func keyToken(v string) string {
|
||||
var b strings.Builder
|
||||
for i := 0; i < len(v); i++ {
|
||||
c := v[i]
|
||||
if c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || c == '_' || c == '-' {
|
||||
b.WriteByte(c)
|
||||
continue
|
||||
}
|
||||
fmt.Fprintf(&b, "=%02X", c)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// QueryAnswer is query's answer.
|
||||
type QueryAnswer struct {
|
||||
Entries []IndexEntry `json:"entries"`
|
||||
}
|
||||
|
||||
// RebuildFrom is what a rebuild derives the state from: the history, or the snapshot and the operations after it.
|
||||
const (
|
||||
RebuildFromHistory = "history"
|
||||
RebuildFromSnapshot = "snapshot"
|
||||
)
|
||||
|
||||
// RebuildRequest derives an instance's state again (every instance's when Instance is empty), From the history or the
|
||||
// snapshot.
|
||||
type RebuildRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance,omitempty"`
|
||||
From string `json:"from,omitempty"`
|
||||
}
|
||||
|
||||
// ReindexRequest builds a declared index again from the history.
|
||||
type ReindexRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Index string `json:"index"`
|
||||
}
|
||||
|
||||
// LedgerRequest names a ledger: lag, fill.
|
||||
type LedgerRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
}
|
||||
|
||||
// LedgerRemoveRequest names one instance of a ledger to remove from its history, its state and its indexes (design 47 G
|
||||
// §587).
|
||||
type LedgerRemoveRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
}
|
||||
|
||||
// LedgerRemoved is remove's answer: the instance, the operations removed and the bytes the history held less after it;
|
||||
// an instance the ledger does not hold answers 0 and 0, so a removal said twice changes nothing.
|
||||
type LedgerRemoved struct {
|
||||
Instance string `json:"instance"`
|
||||
Operations uint64 `json:"operations"`
|
||||
Bytes uint64 `json:"bytes"`
|
||||
}
|
||||
|
||||
// Repaired is the answer of rebuild, reindex and snapshot: how many instances (or index entries) it wrote.
|
||||
type Repaired struct {
|
||||
Written int `json:"written"`
|
||||
}
|
||||
|
||||
// InstanceLag is one instance whose state or snapshot is behind its history: the sequences each holds.
|
||||
type InstanceLag struct {
|
||||
Instance string `json:"instance"`
|
||||
History uint64 `json:"history"`
|
||||
State uint64 `json:"state"`
|
||||
Snapshot uint64 `json:"snapshot"`
|
||||
}
|
||||
|
||||
// IndexLag is how far one declared index is behind the history.
|
||||
type IndexLag struct {
|
||||
Index string `json:"index"`
|
||||
Behind uint64 `json:"behind"`
|
||||
}
|
||||
|
||||
// Lagged is lag's answer: every instance and index behind its history; none when nothing is behind.
|
||||
type Lagged struct {
|
||||
Instances []InstanceLag `json:"instances,omitempty"`
|
||||
Indexes []IndexLag `json:"indexes,omitempty"`
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A-42: stream.publish is a method of its own, beside read and fill, and its request and answer keep their field names.
|
||||
func TestStreamPublishIsAMethodWithItsShapes(t *testing.T) {
|
||||
if StreamPublish != "mesh/stream.publish" || StreamPublish == StreamRead || StreamPublish == StreamFill {
|
||||
t.Fatalf("method %q", StreamPublish)
|
||||
}
|
||||
b, _ := json.Marshal(StreamPublishRequest{Stream: "events", Event: "scrub.requested", Body: json.RawMessage(`{"a":1}`), ID: "x"})
|
||||
if string(b) != `{"stream":"events","event":"scrub.requested","body":{"a":1},"id":"x"}` {
|
||||
t.Errorf("request %s", b)
|
||||
}
|
||||
a, _ := json.Marshal(StreamPublished{Sequence: 7})
|
||||
if string(a) != `{"sequence":7}` {
|
||||
t.Errorf("answer %s", a)
|
||||
}
|
||||
}
|
||||
@@ -1,155 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// SeatChunks is the send of one seat verb's answer file in chunks (design 47 G §681, A-64, A-139), a verb whose seat
|
||||
// definition says `chunks: true`: the holder's process sends the nth chunk on mesh.seat.<seat>.chunk.<verb>.<call>.<n>,
|
||||
// and the caller acknowledges it on mesh.seat.<seat>.ack.<verb>.<call>.<n> within SeatAckWithin before the next is sent.
|
||||
// `call` is the 128 random bits the caller chose. The holder sends with it and the caller receives with it: one form.
|
||||
type SeatChunks struct {
|
||||
Seat, Verb, Call string
|
||||
}
|
||||
|
||||
// SeatChunkBytes is at most what one chunk carries: 512 KiB.
|
||||
const SeatChunkBytes = 512 << 10
|
||||
|
||||
// SeatAckWithin is how long the holder waits for a chunk's acknowledgement, and so how long a caller waits for the next chunk.
|
||||
const SeatAckWithin = 30 * time.Second
|
||||
|
||||
// SeatChunk is the body of one chunk: its number from 1, how many there are, and its bytes.
|
||||
type SeatChunk struct {
|
||||
N int64 `json:"n"`
|
||||
Of int64 `json:"of"`
|
||||
Data []byte `json:"data"`
|
||||
}
|
||||
|
||||
// Chunk is the subject of the nth chunk.
|
||||
func (c SeatChunks) Chunk(n int64) string {
|
||||
return "mesh.seat." + c.Seat + ".chunk." + c.Verb + "." + c.Call + "." + strconv.FormatInt(n, 10)
|
||||
}
|
||||
|
||||
// Ack is the subject the nth chunk is acknowledged on.
|
||||
func (c SeatChunks) Ack(n int64) string {
|
||||
return "mesh.seat." + c.Seat + ".ack." + c.Verb + "." + c.Call + "." + strconv.FormatInt(n, 10)
|
||||
}
|
||||
|
||||
// Chunks is every chunk subject of this call: what a caller subscribes to before it asks the verb.
|
||||
func (c SeatChunks) Chunks() string {
|
||||
return "mesh.seat." + c.Seat + ".chunk." + c.Verb + "." + c.Call + ".*"
|
||||
}
|
||||
|
||||
// SeatAcks is every acknowledgement of a verb's sends: what the holder subscribes to.
|
||||
func SeatAcks(seat, verb string) string {
|
||||
return "mesh.seat." + seat + ".ack." + verb + ".*.*"
|
||||
}
|
||||
|
||||
// ErrSeatChunkMissed is a chunk that did not come within its bound: the holder ended the send.
|
||||
var ErrSeatChunkMissed = errors.New("a chunk did not come within its bound, so the send ended")
|
||||
|
||||
// Receive reads `of` chunks in order, writes each one's bytes to w and acknowledges it before the next is sent. next
|
||||
// yields the subject and body of the next chunk heard on Chunks(), or ErrSeatChunkMissed at its bound (SeatAckWithin);
|
||||
// ack publishes on a subject. A chunk out of order, of another count or on another subject ends the receive, and so the
|
||||
// send, since it is never acknowledged. It answers the bytes written.
|
||||
func (c SeatChunks) Receive(of int64, next func() (subject string, body []byte, err error), ack func(subject string) error, w io.Writer) (int64, error) {
|
||||
var written int64
|
||||
for n := int64(1); n <= of; n++ {
|
||||
subject, body, err := next()
|
||||
if err != nil {
|
||||
return written, fmt.Errorf("chunk %d of %d: %w", n, of, err)
|
||||
}
|
||||
if subject != c.Chunk(n) {
|
||||
return written, fmt.Errorf("chunk %d of %d came on %s, not %s", n, of, subject, c.Chunk(n))
|
||||
}
|
||||
var ch SeatChunk
|
||||
if err := json.Unmarshal(body, &ch); err != nil {
|
||||
return written, fmt.Errorf("chunk %d of %d does not read: %w", n, of, err)
|
||||
}
|
||||
if ch.N != n || ch.Of != of {
|
||||
return written, fmt.Errorf("chunk %d of %d says it is %d of %d", n, of, ch.N, ch.Of)
|
||||
}
|
||||
k, err := w.Write(ch.Data)
|
||||
written += int64(k)
|
||||
if err != nil {
|
||||
return written, fmt.Errorf("chunk %d of %d cannot be written: %w", n, of, err)
|
||||
}
|
||||
if err := ack(c.Ack(n)); err != nil {
|
||||
return written, fmt.Errorf("chunk %d of %d cannot be acknowledged: %w", n, of, err)
|
||||
}
|
||||
}
|
||||
return written, nil
|
||||
}
|
||||
|
||||
// ChunkLine is the core connection as a send of chunks reaches it, which a *nats.Conn is: a chunk is published on its subject and
|
||||
// kept on no stream, and the acknowledgements are subscribed to, since the caller asks each as a request.
|
||||
type ChunkLine interface {
|
||||
Publish(subject string, data []byte) error
|
||||
Subscribe(subject string, cb nats.MsgHandler) (*nats.Subscription, error)
|
||||
}
|
||||
|
||||
// Send sends body in chunks of at most SeatChunkBytes, the holder's side of Receive (design 47 G §681, A-139, A-237): it subscribes
|
||||
// to the verb's acknowledgements (SeatAcks), answers each as the request it is, publishes the nth chunk on Chunk(n) only once the
|
||||
// n-1th is acknowledged on Ack(n-1), and ends the subscription when the send ends. An acknowledgement that does not come within
|
||||
// `within` (SeatAckWithin where the caller has no other bound) ends the send with ErrSeatChunkMissed, and logf says where. Another
|
||||
// call's acknowledgements are skipped. It blocks until the send ends, so a verb answers its chunk count first and runs it after.
|
||||
func (c SeatChunks) Send(line ChunkLine, body []byte, within time.Duration, logf func(string, ...any)) error {
|
||||
if logf == nil {
|
||||
logf = func(string, ...any) {}
|
||||
}
|
||||
acks := make(chan string, 1)
|
||||
sub, err := line.Subscribe(SeatAcks(c.Seat, c.Verb), func(m *nats.Msg) {
|
||||
if m.Reply != "" {
|
||||
_ = m.Respond([]byte("{}"))
|
||||
}
|
||||
select {
|
||||
case acks <- m.Subject:
|
||||
default:
|
||||
}
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("the acknowledgements cannot be heard, so no chunk is sent: %w", err)
|
||||
}
|
||||
defer func() { _ = sub.Unsubscribe() }()
|
||||
total := int64((len(body) + SeatChunkBytes - 1) / SeatChunkBytes)
|
||||
for n := int64(1); n <= total; n++ {
|
||||
from := (n - 1) * SeatChunkBytes
|
||||
to := min(int64(len(body)), from+SeatChunkBytes)
|
||||
data, err := json.Marshal(SeatChunk{N: n, Of: total, Data: body[from:to]})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := line.Publish(c.Chunk(n), data); err != nil {
|
||||
logf("the send %s of %s ends at chunk %d of %d: %v", c.Call, c.Verb, n, total, err)
|
||||
return fmt.Errorf("chunk %d of %d: %w", n, total, err)
|
||||
}
|
||||
if !c.acknowledged(acks, c.Ack(n), within) {
|
||||
logf("the send %s of %s ends at chunk %d of %d: no acknowledgement within %s", c.Call, c.Verb, n, total, within)
|
||||
return fmt.Errorf("chunk %d of %d: %w", n, total, ErrSeatChunkMissed)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// acknowledged waits within for the acknowledgement on subject, skipping another call's.
|
||||
func (c SeatChunks) acknowledged(acks <-chan string, subject string, within time.Duration) bool {
|
||||
timer := time.NewTimer(within)
|
||||
defer timer.Stop()
|
||||
for {
|
||||
select {
|
||||
case got := <-acks:
|
||||
if got == subject {
|
||||
return true
|
||||
}
|
||||
case <-timer.C:
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,72 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestSeatChunksReceive: the chunks of a send are written in order and each acknowledged before the next; a missed,
|
||||
// misplaced or miscounted chunk ends the receive unacknowledged (design 47 G §681, A-139).
|
||||
func TestSeatChunksReceive(t *testing.T) {
|
||||
c := SeatChunks{Seat: "mesh-broker", Verb: "snapshot-get", Call: "0123456789abcdef0123456789abcdef"}
|
||||
if c.Chunk(2) != "mesh.seat.mesh-broker.chunk.snapshot-get.0123456789abcdef0123456789abcdef.2" ||
|
||||
c.Ack(2) != "mesh.seat.mesh-broker.ack.snapshot-get.0123456789abcdef0123456789abcdef.2" ||
|
||||
c.Chunks() != "mesh.seat.mesh-broker.chunk.snapshot-get.0123456789abcdef0123456789abcdef.*" ||
|
||||
SeatAcks("mesh-broker", "snapshot-get") != "mesh.seat.mesh-broker.ack.snapshot-get.*.*" {
|
||||
t.Fatalf("subjects: %s %s %s", c.Chunk(2), c.Ack(2), c.Chunks())
|
||||
}
|
||||
type sent struct {
|
||||
subject string
|
||||
body SeatChunk
|
||||
missed bool
|
||||
}
|
||||
good := []sent{{subject: c.Chunk(1), body: SeatChunk{N: 1, Of: 3, Data: []byte("ab")}},
|
||||
{subject: c.Chunk(2), body: SeatChunk{N: 2, Of: 3, Data: []byte("cd")}},
|
||||
{subject: c.Chunk(3), body: SeatChunk{N: 3, Of: 3, Data: []byte("e")}}}
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
sends []sent
|
||||
want string
|
||||
acks int
|
||||
err string
|
||||
}{
|
||||
{name: "three chunks", sends: good, want: "abcde", acks: 3},
|
||||
{name: "a missed chunk", sends: append(good[:1:1], sent{missed: true}), want: "ab", acks: 1, err: "chunk 2 of 3"},
|
||||
{name: "out of order", sends: []sent{good[1]}, acks: 0, err: "came on"},
|
||||
{name: "another count", sends: []sent{{subject: c.Chunk(1), body: SeatChunk{N: 1, Of: 2, Data: []byte("x")}}}, err: "says it is 1 of 2"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
i, acks := 0, 0
|
||||
next := func() (string, []byte, error) {
|
||||
if i >= len(tc.sends) || tc.sends[i].missed {
|
||||
return "", nil, ErrSeatChunkMissed
|
||||
}
|
||||
s := tc.sends[i]
|
||||
i++
|
||||
body, _ := json.Marshal(s.body)
|
||||
return s.subject, body, nil
|
||||
}
|
||||
ack := func(subject string) error {
|
||||
acks++
|
||||
if subject != c.Ack(int64(acks)) {
|
||||
t.Fatalf("ack %d on %s", acks, subject)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
var out bytes.Buffer
|
||||
n, err := c.Receive(3, next, ack, &out)
|
||||
if tc.err == "" && err != nil || tc.err != "" && (err == nil || !strings.Contains(err.Error(), tc.err)) {
|
||||
t.Fatalf("err %v, want %q", err, tc.err)
|
||||
}
|
||||
if tc.name == "a missed chunk" && !errors.Is(err, ErrSeatChunkMissed) {
|
||||
t.Fatalf("a missed chunk is not ErrSeatChunkMissed: %v", err)
|
||||
}
|
||||
if out.String() != tc.want || n != int64(len(tc.want)) || acks != tc.acks {
|
||||
t.Fatalf("wrote %q (%d), %d acks; want %q, %d acks", out.String(), n, acks, tc.want, tc.acks)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,94 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// lineMem is a core connection in memory: what Send publishes is kept, and when ackOf says so the chunk is acknowledged as the
|
||||
// caller does, with a request on the ack subject, whose reply Send must give.
|
||||
type lineMem struct {
|
||||
mu sync.Mutex
|
||||
handler nats.MsgHandler
|
||||
subject string
|
||||
chunks map[string][]byte
|
||||
order []string
|
||||
ackOf func(n int64) (string, bool)
|
||||
}
|
||||
|
||||
func (l *lineMem) Subscribe(subject string, cb nats.MsgHandler) (*nats.Subscription, error) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
l.handler, l.subject = cb, subject
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (l *lineMem) Publish(subject string, data []byte) error {
|
||||
l.mu.Lock()
|
||||
l.chunks[subject] = data
|
||||
l.order = append(l.order, subject)
|
||||
h := l.handler
|
||||
l.mu.Unlock()
|
||||
var c SeatChunk
|
||||
_ = json.Unmarshal(data, &c)
|
||||
if sub, ok := l.ackOf(c.N); ok {
|
||||
go h(&nats.Msg{Subject: sub, Reply: "_INBOX.x", Sub: nil})
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// TestSeatChunksSend: the bytes go in chunks of at most SeatChunkBytes, each published only after the one before is
|
||||
// acknowledged, and the receive of the same form reads them back; an acknowledgement missed or of another call ends the
|
||||
// send (design 47 G §681, A-237, A-139).
|
||||
func TestSeatChunksSend(t *testing.T) {
|
||||
c := SeatChunks{Seat: "mesh-catalog", Verb: "input", Call: "0123456789abcdef"}
|
||||
body := bytes.Repeat([]byte("x"), 2*SeatChunkBytes+5)
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
ackOf func(int64) (string, bool)
|
||||
err string
|
||||
sent int
|
||||
}{
|
||||
{name: "every chunk acknowledged", ackOf: func(n int64) (string, bool) { return c.Ack(n), true }, sent: 3},
|
||||
{name: "no acknowledgement", ackOf: func(int64) (string, bool) { return "", false }, err: "chunk 1 of 3", sent: 1},
|
||||
{name: "another call's acknowledgement", ackOf: func(n int64) (string, bool) {
|
||||
return SeatChunks{Seat: "mesh-catalog", Verb: "input", Call: "fedcba9876543210"}.Ack(n), true
|
||||
}, err: "chunk 1 of 3", sent: 1},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
l := &lineMem{chunks: map[string][]byte{}, ackOf: tc.ackOf}
|
||||
err := c.Send(l, body, time.Millisecond*50, func(string, ...any) {})
|
||||
if tc.err == "" && err != nil || tc.err != "" && (err == nil || !strings.Contains(err.Error(), tc.err)) {
|
||||
t.Fatalf("err %v, want %q", err, tc.err)
|
||||
}
|
||||
if len(l.order) != tc.sent {
|
||||
t.Fatalf("%d chunks sent, want %d", len(l.order), tc.sent)
|
||||
}
|
||||
if l.subject != SeatAcks("mesh-catalog", "input") {
|
||||
t.Fatalf("acknowledgements heard on %q", l.subject)
|
||||
}
|
||||
if tc.err != "" {
|
||||
return
|
||||
}
|
||||
var got bytes.Buffer
|
||||
i := 0
|
||||
_, err = c.Receive(3, func() (string, []byte, error) {
|
||||
s := l.order[i]
|
||||
i++
|
||||
return s, l.chunks[s], nil
|
||||
}, func(string) error { return nil }, &got)
|
||||
if err != nil || !bytes.Equal(got.Bytes(), body) {
|
||||
t.Fatalf("received %d bytes, %v", got.Len(), err)
|
||||
}
|
||||
})
|
||||
}
|
||||
if err := c.Send(&lineMem{chunks: map[string][]byte{}}, nil, time.Second, nil); err != nil {
|
||||
t.Errorf("no bytes: %v", err)
|
||||
}
|
||||
}
|
||||
+154
-3
@@ -1,11 +1,121 @@
|
||||
package buscompose
|
||||
|
||||
import "time"
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// AccountIssuedEvent is what the vault says when it minted an account: AccountIssued, never a value or a hash. The state of
|
||||
// wanted accounts it mints from is the grants part's (grantsstate.AccountsState; novox/hq ADR 0634 §1).
|
||||
// AccountsState is the grants part's state of wanted accounts, by bus user.
|
||||
const AccountsState = "accounts"
|
||||
|
||||
// GrantsAccounts is that state as a reader names it (`<owner>.<name>`).
|
||||
const GrantsAccounts = GrantsModule + "." + AccountsState
|
||||
|
||||
// AccountIssuedEvent is what the vault says when it minted an account: AccountIssued, never a value or a hash.
|
||||
const AccountIssuedEvent = "account-issued"
|
||||
|
||||
// AccountIssuedSubject is where it says it, as the runtime addresses mesh-vault's events.
|
||||
const AccountIssuedSubject = "mesh.mod." + VaultSeat + ".event." + AccountIssuedEvent
|
||||
|
||||
// ClaimedSeat is one seat a module claims, as its runtime needs it (the controller's seatClaimed): the name, the scope (a
|
||||
// node-scoped seat's verb carries the machine) and the verbs the runtime serves for it.
|
||||
type ClaimedSeat struct {
|
||||
Seat string `json:"seat"`
|
||||
Scope string `json:"scope"`
|
||||
Serves []string `json:"serves,omitempty"`
|
||||
}
|
||||
|
||||
// WantedAccount is one key of the grants part's state `accounts`: an account the mesh needs, without its password.
|
||||
type WantedAccount struct {
|
||||
User string `json:"user"`
|
||||
// Kind is KindModule (KindNodeTools for the runtime, named as a module is), KindProcess or KindProvisioner.
|
||||
Kind Kind `json:"kind"`
|
||||
Node string `json:"node"`
|
||||
Module string `json:"module"`
|
||||
// Process is a KindProcess account's process id.
|
||||
Process string `json:"process,omitempty"`
|
||||
// URL is the bus's address as that node reaches it, `nats://<host>:<port>`; Fingerprint the bus certificate's.
|
||||
URL string `json:"url"`
|
||||
Fingerprint string `json:"fingerprint,omitempty"`
|
||||
// Claims are a module's claims with the verbs each seat promises: none for a process or a provisioner client.
|
||||
Claims []ClaimedSeat `json:"claims,omitempty"`
|
||||
// At is when the grants part wrote the key: not compared (Same).
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// mintedKinds are the kinds of account the vault mints (ADR 0561 §1).
|
||||
var mintedKinds = []Kind{KindModule, KindNodeTools, KindProcess, KindProvisioner}
|
||||
|
||||
// principal is the account as the composer names it.
|
||||
func (w WantedAccount) principal() Principal {
|
||||
return Principal{Kind: w.Kind, Node: w.Node, Module: w.Module, Process: w.Process}
|
||||
}
|
||||
|
||||
// Valid says why w is not an account the vault may mint, or nil: a kind it mints, every name a subject token, the user
|
||||
// the one the composer derives from them, and an address to reach the bus at.
|
||||
func (w WantedAccount) Valid() error {
|
||||
if !slices.Contains(mintedKinds, w.Kind) {
|
||||
return fmt.Errorf("%s is a %s account, which the vault never mints (ADR 0561 §4)", w.User, w.Kind)
|
||||
}
|
||||
for what, v := range map[string]string{"node": w.Node, "module": w.Module} {
|
||||
if !safeSubject.MatchString(v) {
|
||||
return fmt.Errorf("%s: the %s %q is not a name the mesh writes", w.User, what, v)
|
||||
}
|
||||
}
|
||||
if w.Kind == KindProcess && !safeSubject.MatchString(w.Process) {
|
||||
return fmt.Errorf("%s: the process %q is not a name the mesh writes", w.User, w.Process)
|
||||
}
|
||||
if w.Kind != KindProcess && w.Process != "" {
|
||||
return fmt.Errorf("%s: only a process's account names a process", w.User)
|
||||
}
|
||||
if want := w.principal().Username(); w.User != want {
|
||||
return fmt.Errorf("%s is not the user of a %s account of %s on %s (that is %s)", w.User, w.Kind, w.Module, w.Node, want)
|
||||
}
|
||||
if !strings.HasPrefix(w.URL, "nats://") || len(w.URL) == len("nats://") {
|
||||
return fmt.Errorf("%s names no bus address (url, nats://<host>:<port>)", w.User)
|
||||
}
|
||||
if len(w.Claims) > 0 && w.Kind != KindModule && w.Kind != KindNodeTools {
|
||||
return fmt.Errorf("%s: a %s account holds no seat, so it carries no claims", w.User, w.Kind)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Same says whether two wanted accounts want the same account: every field but when it was written.
|
||||
func (w WantedAccount) Same(o WantedAccount) bool {
|
||||
a, b := w, o
|
||||
a.At, b.At = time.Time{}, time.Time{}
|
||||
ra, _ := json.Marshal(a)
|
||||
rb, _ := json.Marshal(b)
|
||||
return string(ra) == string(rb)
|
||||
}
|
||||
|
||||
// OwnSecret is the module's own secret the account is held in: `broker`, `broker-<process id>` or `provisioner-broker`
|
||||
// (the controller's names, ADR 0436 §2, ADR 0349 §11).
|
||||
func (w WantedAccount) OwnSecret() string {
|
||||
switch w.Kind {
|
||||
case KindProcess:
|
||||
return "broker-" + w.Process
|
||||
case KindProvisioner:
|
||||
return "provisioner-broker"
|
||||
}
|
||||
return "broker"
|
||||
}
|
||||
|
||||
// ReadWantedAccount reads the key user of the state: refused when it is not a valid wanted account of that user.
|
||||
func ReadWantedAccount(user string, raw []byte) (WantedAccount, error) {
|
||||
var w WantedAccount
|
||||
if err := json.Unmarshal(raw, &w); err != nil {
|
||||
return WantedAccount{}, fmt.Errorf("%s %s is not a wanted account: %w", GrantsAccounts, user, err)
|
||||
}
|
||||
if w.User != user {
|
||||
return WantedAccount{}, fmt.Errorf("%s %s names the user %q", GrantsAccounts, user, w.User)
|
||||
}
|
||||
return w, w.Valid()
|
||||
}
|
||||
|
||||
// AccountIssued is the vault's account-issued: which account it minted, and its generation. Never a value or a hash.
|
||||
type AccountIssued struct {
|
||||
User string `json:"user"`
|
||||
@@ -16,3 +126,44 @@ type AccountIssued struct {
|
||||
Generation int `json:"generation"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// HeldCredential is the value the account's holder reads, exactly as the controller builds it today (issueWith,
|
||||
// sealProcess and sealProvisionerClient): a module's `{url, fingerprint, node, module, user, password, claims}`, a
|
||||
// process's without claims, a provisioner client's without module or claims; an empty fingerprint or claims left out.
|
||||
func HeldCredential(w WantedAccount, password string) ([]byte, error) {
|
||||
if err := w.Valid(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if password == "" {
|
||||
return nil, fmt.Errorf("%s: an account is held with its password", w.User)
|
||||
}
|
||||
switch w.Kind {
|
||||
case KindProcess:
|
||||
return json.Marshal(struct {
|
||||
URL string `json:"url"`
|
||||
Fingerprint string `json:"fingerprint,omitempty"`
|
||||
Node string `json:"node"`
|
||||
Module string `json:"module"`
|
||||
User string `json:"user"`
|
||||
Password string `json:"password"`
|
||||
}{URL: w.URL, Fingerprint: w.Fingerprint, Node: w.Node, Module: w.Module, User: w.User, Password: password})
|
||||
case KindProvisioner:
|
||||
return json.Marshal(struct {
|
||||
URL string `json:"url"`
|
||||
Fingerprint string `json:"fingerprint,omitempty"`
|
||||
Node string `json:"node"`
|
||||
User string `json:"user"`
|
||||
Password string `json:"password"`
|
||||
}{URL: w.URL, Fingerprint: w.Fingerprint, Node: w.Node, User: w.User, Password: password})
|
||||
}
|
||||
return json.Marshal(struct {
|
||||
URL string `json:"url"`
|
||||
Fingerprint string `json:"fingerprint,omitempty"`
|
||||
Node string `json:"node"`
|
||||
Module string `json:"module"`
|
||||
User string `json:"user"`
|
||||
Password string `json:"password"`
|
||||
Claims []ClaimedSeat `json:"claims,omitempty"`
|
||||
}{URL: w.URL, Fingerprint: w.Fingerprint, Node: w.Node, Module: w.Module, User: w.User, Password: password,
|
||||
Claims: w.Claims})
|
||||
}
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
package buscompose
|
||||
|
||||
// EngineModule is the node-engine's manifest's module name, and EngineActsState its state the acts asked through the
|
||||
// node seat and its own asks are kept in (ADR 0563 §2, §3: "the engine's own state"): the bucket mesh-host_acts, keys
|
||||
// `<node>.act.<id>`, `<node>.open.<act>.<digest>` and `<node>.ask.<id>`, each written by that node's engine alone.
|
||||
const (
|
||||
EngineModule = "mesh-host"
|
||||
EngineActsState = "acts"
|
||||
)
|
||||
|
||||
// EngineActsBucket is that state's bucket.
|
||||
var EngineActsBucket = BucketName(EngineModule, EngineActsState)
|
||||
@@ -1,106 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The one place the mesh's event subjects are named (design 34 every-event §1, §25, §26), so neither the grants part
|
||||
// nor the tool runner spells the event namespace itself.
|
||||
|
||||
// The namespaces an event lands in, as `from` names them (design 34 every-event §13, §26).
|
||||
const (
|
||||
fromSeat = "seat"
|
||||
fromModule = "module"
|
||||
fromStream = "stream"
|
||||
)
|
||||
|
||||
// eventNamespaces are the three, each with the prefix its subjects start with, in EveryEvent's order.
|
||||
var eventNamespaces = []struct{ from, prefix string }{
|
||||
{fromSeat, "mesh.seat."},
|
||||
{fromModule, "mesh.mod."},
|
||||
{fromStream, moduleStreamPrefix},
|
||||
}
|
||||
|
||||
// EveryEvent is the subjects the tool runner's account hears: every seat's events, every module's, and every module
|
||||
// stream's, a restricted one included, in that order (novox/hq ADR 0670 §1, ADR 0671 §1).
|
||||
func EveryEvent() []string {
|
||||
out := make([]string, 0, len(eventNamespaces))
|
||||
for _, ns := range eventNamespaces {
|
||||
out = append(out, ns.prefix+"*.event.>")
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// eventToken is the form of each token of an event name.
|
||||
var eventToken = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
|
||||
|
||||
// EventSubjects is the subjects one event name lands on. A name is `<emitter>.<event>`, optionally followed by further
|
||||
// tokens (`operator-channel.decided.claude-code`), each token of the form `[A-Za-z0-9_-]+`. With from empty it answers
|
||||
// six subjects, per namespace (seat, module, stream) `mesh.<ns>.<emitter>.event.<rest>` and the same followed by `.>`;
|
||||
// with from `seat`, `module` or `stream` only that namespace's two. It refuses, naming the rule, a name of fewer than two
|
||||
// tokens, an empty token or a token outside that form (so `*` and `>` are refused), and a from of any other word.
|
||||
func EventSubjects(name, from string) ([]string, error) {
|
||||
tokens := strings.Split(name, ".")
|
||||
if len(tokens) < 2 {
|
||||
return nil, fmt.Errorf("event name %q has fewer than two tokens: a name is <seat or module>.<event>", name)
|
||||
}
|
||||
for _, t := range tokens {
|
||||
if !eventToken.MatchString(t) {
|
||||
return nil, fmt.Errorf("event name %q has the token %q: each token of a name matches [A-Za-z0-9_-]+, "+
|
||||
"so no token is empty and none is * or >", name, t)
|
||||
}
|
||||
}
|
||||
emitter, rest := tokens[0], strings.Join(tokens[1:], ".")
|
||||
var out []string
|
||||
for _, ns := range eventNamespaces {
|
||||
if from != "" && from != ns.from {
|
||||
continue
|
||||
}
|
||||
subject := ns.prefix + emitter + ".event." + rest
|
||||
out = append(out, subject, subject+".>")
|
||||
}
|
||||
if out == nil {
|
||||
return nil, fmt.Errorf("from %q is none of %s, %s or %s", from, fromSeat, fromModule, fromStream)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// EventName is the inverse of EventSubjects for a heard subject: for `mesh.seat.<emitter>.event.<rest>`,
|
||||
// `mesh.mod.<emitter>.event.<rest>` or `mesh.stream.<emitter>.event.<rest>` it answers the name `<emitter>.<rest>` and
|
||||
// its namespace (`seat`, `module` or `stream`); any other subject is answered as not an event (ok false).
|
||||
func EventName(subject string) (name, from string, ok bool) {
|
||||
for _, ns := range eventNamespaces {
|
||||
after, found := strings.CutPrefix(subject, ns.prefix)
|
||||
if !found {
|
||||
continue
|
||||
}
|
||||
emitter, rest, found := strings.Cut(after, ".event.")
|
||||
if !found || emitter == "" || rest == "" || strings.Contains(emitter, ".") {
|
||||
return "", "", false
|
||||
}
|
||||
return emitter + "." + rest, ns.from, true
|
||||
}
|
||||
return "", "", false
|
||||
}
|
||||
|
||||
// Overlaps says whether some subject matches both a and b, a wildcard on either side: `*` stands for one token and
|
||||
// `>`, last, for one token or more (design 34 every-event §34).
|
||||
func Overlaps(a, b string) bool {
|
||||
at, bt := strings.Split(a, "."), strings.Split(b, ".")
|
||||
for i := 0; ; i++ {
|
||||
switch {
|
||||
case i == len(at) && i == len(bt):
|
||||
return true
|
||||
case i == len(at) || i == len(bt):
|
||||
return false
|
||||
case at[i] == ">" || bt[i] == ">":
|
||||
return true
|
||||
case at[i] == "*" || bt[i] == "*" || at[i] == bt[i]:
|
||||
continue
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The mesh's event subjects are named in one place: EveryEvent, EventSubjects, EventName and Overlaps (design 34
|
||||
// every-event §1, §25, §26, §34, §37).
|
||||
func TestTheEventSubjectsAreNamedInOnePlace(t *testing.T) {
|
||||
// EE-S1/AC1 (E §1, §25): the three subjects, in order.
|
||||
if got, want := EveryEvent(), []string{"mesh.seat.*.event.>", "mesh.mod.*.event.>", "mesh.stream.*.event.>"}; !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("EveryEvent is %v, want %v", got, want)
|
||||
}
|
||||
|
||||
// EE-S1/AC2, AC3 (E §1, §26): a name's subjects, all six or one namespace's two, and the refusals naming the rule.
|
||||
subjects := []struct {
|
||||
name, from string
|
||||
want []string
|
||||
refused string
|
||||
}{
|
||||
{name: "operator-channel.decided.claude-code", want: []string{
|
||||
"mesh.seat.operator-channel.event.decided.claude-code", "mesh.seat.operator-channel.event.decided.claude-code.>",
|
||||
"mesh.mod.operator-channel.event.decided.claude-code", "mesh.mod.operator-channel.event.decided.claude-code.>",
|
||||
"mesh.stream.operator-channel.event.decided.claude-code", "mesh.stream.operator-channel.event.decided.claude-code.>",
|
||||
}},
|
||||
{name: "mesh-delivery-view.delivered", from: "seat", want: []string{
|
||||
"mesh.seat.mesh-delivery-view.event.delivered", "mesh.seat.mesh-delivery-view.event.delivered.>"}},
|
||||
{name: "mesh-delivery-view.delivered", from: "module", want: []string{
|
||||
"mesh.mod.mesh-delivery-view.event.delivered", "mesh.mod.mesh-delivery-view.event.delivered.>"}},
|
||||
{name: "mesh-mail.mail", from: "stream", want: []string{
|
||||
"mesh.stream.mesh-mail.event.mail", "mesh.stream.mesh-mail.event.mail.>"}},
|
||||
{name: "mesh-delivery-view", refused: "fewer than two tokens"},
|
||||
{name: "mesh-delivery-view..delivered", refused: "[A-Za-z0-9_-]+"},
|
||||
{name: "mesh-delivery-view.*", refused: "[A-Za-z0-9_-]+"},
|
||||
{name: "mesh-delivery-view.>", refused: "[A-Za-z0-9_-]+"},
|
||||
{name: "mesh-delivery-view.delivered", from: "bench", refused: "none of seat, module or stream"},
|
||||
}
|
||||
for _, c := range subjects {
|
||||
got, err := EventSubjects(c.name, c.from)
|
||||
if c.refused != "" {
|
||||
if err == nil || !strings.Contains(err.Error(), c.refused) {
|
||||
t.Errorf("EventSubjects(%q, %q) answered %v, %v; want refused naming %q", c.name, c.from, got, err, c.refused)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err != nil || !reflect.DeepEqual(got, c.want) {
|
||||
t.Errorf("EventSubjects(%q, %q) answered %v, %v; want %v", c.name, c.from, got, err, c.want)
|
||||
}
|
||||
// EE-S1/AC5 (E §1): every subject it answers is covered by one of EveryEvent's.
|
||||
for _, s := range got {
|
||||
covered := false
|
||||
for _, e := range EveryEvent() {
|
||||
covered = covered || Overlaps(e, s)
|
||||
}
|
||||
if !covered {
|
||||
t.Errorf("%s is covered by none of EveryEvent", s)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// EE-S1/AC4 (E §1, §26): a heard subject's name and namespace, and any other subject not an event.
|
||||
names := []struct{ subject, name, from string }{
|
||||
{"mesh.seat.operator-channel.event.decided.claude-code", "operator-channel.decided.claude-code", "seat"},
|
||||
{"mesh.mod.mesh-delivery-view.event.delivered", "mesh-delivery-view.delivered", "module"},
|
||||
{"mesh.stream.mesh-delivery-checker.event.verdict", "mesh-delivery-checker.verdict", "stream"},
|
||||
{"mesh.seat.intake.call.accept.ace", "", ""},
|
||||
{"mesh.mod.netcheck.call.netcheck_dns", "", ""},
|
||||
{"$KV.views.x", "", ""},
|
||||
{"mesh.seat..event.x", "", ""},
|
||||
}
|
||||
for _, c := range names {
|
||||
name, from, ok := EventName(c.subject)
|
||||
if name != c.name || from != c.from || ok != (c.name != "") {
|
||||
t.Errorf("EventName(%q) answered %q, %q, %v; want %q, %q", c.subject, name, from, ok, c.name, c.from)
|
||||
}
|
||||
}
|
||||
|
||||
// EE-S1/AC6 (E §34, §37): overlapping subjects, a wildcard on either side.
|
||||
overlaps := []struct {
|
||||
a, b string
|
||||
want bool
|
||||
}{
|
||||
{"mesh.stream.mail.event.mail.>", "mesh.stream.mail.event.mail.*.message", true},
|
||||
{"mesh.mod.mail.event.x", "mesh.stream.mail.event.x", false},
|
||||
{"mesh.mod.mail.event.x.>", "mesh.mod.mail.event.x", false},
|
||||
{"mesh.mod.mail.event.*", "mesh.mod.*.event.x", true},
|
||||
{"mesh.mod.mail.event.x", "mesh.mod.mail.event.x", true},
|
||||
{"mesh.mod.*.event.>", "mesh.mod.mail.event.x.ace", true},
|
||||
{"mesh.mod.mail.event.x.ace", "mesh.mod.mail.event.x", false},
|
||||
}
|
||||
for _, c := range overlaps {
|
||||
if Overlaps(c.a, c.b) != c.want || Overlaps(c.b, c.a) != c.want {
|
||||
t.Errorf("Overlaps(%q, %q) is not %v both ways", c.a, c.b, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/catalogue"
|
||||
"git.novox.be/novox/mesh-sdk/go/health"
|
||||
)
|
||||
|
||||
// ControllerLeftoverConsumers are the ended controller's durable consumers left on the bus, one leftover of the module
|
||||
// mesh-delivery placed nowhere (issue 1126, milestone section 3), and mesh-catalog's of the controller's built-before,
|
||||
// which its manifest no longer consumes and no part composes; each `<stream>.<consumer>`.
|
||||
var ControllerLeftoverConsumers = []string{
|
||||
"EVENTS." + ControllerName, "EVENTS." + MergesConsumerName, "EVENTS.novox_mesh-delivery", "EVENTS.novox_mesh-catalog",
|
||||
"CONTROL." + ControllerName, DeadLetterNoticesStream + "." + ControllerName,
|
||||
}
|
||||
|
||||
// ControllerLeftoverConsumersOn are ControllerLeftoverConsumers and, for each node given, the consumer the controller
|
||||
// made for the node's engine (`EVENTS.<node>_mesh-host`, of restic's backed-up): the engine hears that event without a
|
||||
// durable consumer, and the grants part composes none for it, so nothing reads it (issue 1126).
|
||||
func ControllerLeftoverConsumersOn(nodes []string) []string {
|
||||
out := slices.Clone(ControllerLeftoverConsumers)
|
||||
for _, n := range nodes {
|
||||
if safeSubject.MatchString(n) {
|
||||
out = append(out, "EVENTS."+n+"_"+EngineModule)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ControllerLeftoverBuckets are the ended controller's buckets, each by its local name (`KV_mesh-controller_<name>`).
|
||||
var ControllerLeftoverBuckets = []string{"asked", "assignments", "builds", "calls", "checks-running", "composed",
|
||||
"condition-history", "conditions", "declared-config", "declared-packages", "delivery-merges", "grants", "hand-acts",
|
||||
"lease", "predefined", "rollouts", "states-announced"}
|
||||
|
||||
// ControllerLeftoverStreams are the ended controller's streams, by their names: CONTROL, ASSIGNMENTS, each of its buckets'
|
||||
// and, for each node given, the declaration files' object bucket the controller wrote (ADR 0486 §2).
|
||||
func ControllerLeftoverStreams(nodes []string) []string {
|
||||
out := []string{"CONTROL", AssignmentsStream}
|
||||
for _, b := range ControllerLeftoverBuckets {
|
||||
out = append(out, "KV_"+BucketName(catalogue.EndedControllerName, b))
|
||||
}
|
||||
for _, n := range nodes {
|
||||
if safeSubject.MatchString(n) {
|
||||
out = append(out, "OBJ_"+DeclarationFilesBucket+n)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// BusRemovalGrants are the whole authority of an account whose bus declares removals (ADR 0564 Limits 4, ADR 0590 clause
|
||||
// 6), each removal its grant: the node-engine's acts bucket read (STREAM.INFO with a subject filter lists what a removal
|
||||
// would take, so the tools read before they remove) and its keys removed, which nothing else may empty when it fills; and
|
||||
// the ended controller's leftovers, each consumer and each stream read and deleted by its name, for the nodes given (the
|
||||
// declaration files' object buckets). A removal outside the set is refused, as the indexer's judge refuses it in a
|
||||
// manifest (novox/hq ADR 0578 §1).
|
||||
func BusRemovalGrants(removes, nodes []string) ([]string, error) {
|
||||
var out []string
|
||||
for _, r := range removes {
|
||||
switch r {
|
||||
case catalogue.RemovesEngineActs:
|
||||
out = append(out, "$JS.API.STREAM.INFO.KV_"+EngineActsBucket, "$KV."+EngineActsBucket+".>")
|
||||
case catalogue.RemovesControllerLeftovers:
|
||||
for _, c := range ControllerLeftoverConsumersOn(nodes) {
|
||||
out = append(out, "$JS.API.CONSUMER.INFO."+c, "$JS.API.CONSUMER.DELETE."+c)
|
||||
}
|
||||
for _, s := range ControllerLeftoverStreams(nodes) {
|
||||
out = append(out, "$JS.API.STREAM.INFO."+s, "$JS.API.STREAM.DELETE."+s)
|
||||
}
|
||||
default:
|
||||
return nil, fmt.Errorf("a bus account removes %q, which is not one of %s (novox/hq ADR 0564 Limits 4, ADR 0590)", r,
|
||||
strings.Join([]string{catalogue.RemovesEngineActs, catalogue.RemovesControllerLeftovers}, ", "))
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// WatchmanHears are the core subjects the watchman subscribes, with no durable consumer (ADR 0564 clauses 4, 5 and
|
||||
// Limits 3, 11): the power seat's words (S1, ADR 0211), the providers' failing and recovered words (S8), each tool
|
||||
// runner's beat (S11), the self-check's checked (S10), a consumer deleted and the bus module's slow consumers and closed
|
||||
// connections (S9), the provisioner clients' reports (S21), the dead letters kept, and the lease write's events. Never the bus's API audit
|
||||
// (`$JS.EVENT.ADVISORY.API`): it carries every API call's request and answer in the account, the vault's hashes among
|
||||
// them (the lead, 2026-10-29).
|
||||
func WatchmanHears() []string {
|
||||
return []string{
|
||||
"mesh.mod.power.event.*",
|
||||
"mesh.mod.*.event.provisioner.failing",
|
||||
"mesh.mod.*.event.provisioner.recovered",
|
||||
"mesh.control.*.tools-alive",
|
||||
"mesh.seat.mesh-self-check.event." + health.EventChecked,
|
||||
"$JS.EVENT.ADVISORY.CONSUMER.DELETED.>",
|
||||
"mesh.mod.nats.event.bus.slow-consumer",
|
||||
"mesh.mod.nats.event.bus.connection-closed",
|
||||
ProvAppliedSubjects,
|
||||
deadLetterPrefix + ">",
|
||||
}
|
||||
}
|
||||
@@ -3,7 +3,6 @@ package buscompose
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// moduleStreamPrefix starts every subject a module's own stream takes.
|
||||
@@ -15,19 +14,6 @@ func ModuleStreamSubject(module, event string) string {
|
||||
return moduleStreamPrefix + module + ".event." + event
|
||||
}
|
||||
|
||||
// WirePattern is a stream's local event pattern as the bus spells it: `**` is `>`, `*` stays `*`. A reader turns a
|
||||
// stream's events into the bus's subjects with it (design 46 7-approval-mark §5): the grants part composing a stream, and
|
||||
// the tool runner deriving a restricted stream's marked subjects.
|
||||
func WirePattern(local string) string {
|
||||
parts := strings.Split(local, ".")
|
||||
for i, p := range parts {
|
||||
if p == "**" {
|
||||
parts[i] = ">"
|
||||
}
|
||||
}
|
||||
return strings.Join(parts, ".")
|
||||
}
|
||||
|
||||
// A ModuleStream is one module's declared stream as the bus holds it.
|
||||
type ModuleStream struct {
|
||||
Module string
|
||||
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/membership"
|
||||
"git.novox.be/novox/mesh-sdk/go/seatdef"
|
||||
)
|
||||
|
||||
// The names this package keeps for the grants part's composer (mesh-grants internal/buscompose, ADR 0593 §1) agree with
|
||||
@@ -19,34 +18,4 @@ func TestTheBusNamesAgree(t *testing.T) {
|
||||
if LogStreamName("mesh-delivery-builder", "build-log") == "" || LogSubject("mesh-delivery-builder", "build-log") == "" {
|
||||
t.Fatal("a log has no stream or subject")
|
||||
}
|
||||
// 1966/AC20: a ledger's history and snapshot subjects are one contract, which the grants part grants a ledger
|
||||
// account its publish on.
|
||||
if h, sn := LedgerHistorySubject("m", "records"), LedgerSnapshotSubject("m", "records"); h != "mesh.ledger.m.records.history" ||
|
||||
sn != "mesh.ledger.m.records.snapshot" {
|
||||
t.Fatalf("a ledger's subjects are %q and %q", h, sn)
|
||||
}
|
||||
// 1966/AC23: a ledger account for m on a is named ledger.a.m.
|
||||
if got := (Principal{Kind: KindLedger, Node: "a", Module: "m"}).Username(); got != "ledger.a.m" {
|
||||
t.Fatalf("a ledger account is named %q", got)
|
||||
}
|
||||
// G §469: a state's bucket is capped at its max-mib; naming none is unbounded.
|
||||
if (Bucket{}).MaxBytes() != -1 || (Bucket{MaxMiB: 2}).MaxBytes() != 2<<20 {
|
||||
t.Fatal("a bucket's cap is not its max-mib in bytes")
|
||||
}
|
||||
// Design 46 7-approval-mark §5: a stream's local event pattern as the bus spells it.
|
||||
for local, wire := range map[string]string{"mail.**": "mail.>", "mail.*.message": "mail.*.message", "sent": "sent"} {
|
||||
if got := WirePattern(local); got != wire {
|
||||
t.Errorf("WirePattern(%q) = %q, want %q", local, got, wire)
|
||||
}
|
||||
}
|
||||
// Design 46 7-approval-mark §13: a per-machine state reaches its machine's key and the keys below it, no other machine's.
|
||||
if got := PerMachineKeys("ace"); len(got) != 2 || got[0] != "ace" || got[1] != "ace.>" {
|
||||
t.Errorf("a per-machine state reaches %v on ace", got)
|
||||
}
|
||||
// A seat's family is the shared events stream filtered to its subjects: a definition says the stream's age.
|
||||
for _, st := range MeshStreams() {
|
||||
if st.Name == EventsStream && int64(st.MaxAge) != seatdef.SharedEventsMaxAgeDays*24*60*60 {
|
||||
t.Errorf("the shared events stream keeps %d s, a seat's definition says %d days", st.MaxAge, seatdef.SharedEventsMaxAgeDays)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+2
-24
@@ -44,26 +44,8 @@ const (
|
||||
// exactly what its bus says — never root, never the operator account, and never able to answer a tool
|
||||
// call or a seat's verb (§3). One per process and machine.
|
||||
KindProcess Kind = "process"
|
||||
// KindLedger is one module assignment's ledger credential (novox/hq ADR 0597 §9, design 25 § 12.4): held by that
|
||||
// machine's tool runner, never by the module's own code, and the only account that publishes on that module's ledger
|
||||
// histories and snapshots. One per module assignment, whose module asks for a ledger in `requires`.
|
||||
KindLedger Kind = "ledger"
|
||||
)
|
||||
|
||||
// LedgerHistorySubject and LedgerSnapshotSubject are the subject prefixes of one ledger's history and of its instances'
|
||||
// snapshots (novox/hq ADR 0597 §1, §8, §9): an instance's subject is the prefix, ".", and the instance's path with each
|
||||
// "/" written "." (`decision/0290` is `….decision.0290`). One contract: the grants part grants a ledger account its
|
||||
// publish on them, the tool runner appends and snapshots on them, and the bus module creates the history's stream over
|
||||
// them.
|
||||
func LedgerHistorySubject(module, local string) string {
|
||||
return "mesh.ledger." + module + "." + local + ".history"
|
||||
}
|
||||
|
||||
// LedgerSnapshotSubject is the subject prefix of one ledger's instances' snapshots (LedgerHistorySubject).
|
||||
func LedgerSnapshotSubject(module, local string) string {
|
||||
return "mesh.ledger." + module + "." + local + ".snapshot"
|
||||
}
|
||||
|
||||
// Seat is a role on the bus as a principal relates to it: the subjects it accepts, and those it
|
||||
// emits (novox/hq ADR 0118, design 29 §5).
|
||||
type Seat struct {
|
||||
@@ -163,8 +145,8 @@ type Principal struct {
|
||||
// instances and read by whoever declares it.
|
||||
State []string
|
||||
Reads []string
|
||||
// PerMachine is the local names of its state keyed by machine, reaching the keys of Node (PerMachineKeys) and
|
||||
// no other machine's; KeyedReads the keys of others' state it reads one key at a time (novox/hq ADR 0260).
|
||||
// PerMachine is the local names of its state keyed by machine, reaching the key named for Node and
|
||||
// no other; KeyedReads the keys of others' state it reads one key at a time (novox/hq ADR 0260).
|
||||
PerMachine []string
|
||||
KeyedReads []KeyedRead
|
||||
// Logs is the local names of the logs this principal's module keeps (novox/hq ADR 0297).
|
||||
@@ -290,10 +272,6 @@ func (p Principal) Username() string {
|
||||
return p.Login
|
||||
}
|
||||
return "node." + p.Node
|
||||
case KindLedger:
|
||||
// `ledger.<machine>.<module>`: never `<node>.<module>`, which is the module's own account or the tool runner's,
|
||||
// so the module's code never holds it.
|
||||
return "ledger." + p.Node + "." + p.Module
|
||||
case KindProvisioner:
|
||||
// Named for the machine, as a node is: one client per machine, whichever module runs it, and never
|
||||
// `<node>.<module>`, which is the module's own account or the tool runner's.
|
||||
|
||||
+5
-17
@@ -18,29 +18,14 @@ type Bucket struct {
|
||||
History int
|
||||
// TTLSeconds is how long a value lives; zero is until replaced or deleted.
|
||||
TTLSeconds int
|
||||
// PerMachine says each machine's instance writes only the keys of its machine (novox/hq ADR 0260; design 46
|
||||
// 7-approval-mark §13): the key named for its machine and the keys below it, PerMachineKeys, and no other machine's.
|
||||
// PerMachine says each machine's instance writes only the key named for its machine (novox/hq ADR
|
||||
// 0260), and is granted that key and no other.
|
||||
PerMachine bool
|
||||
// KeyTTL says the bucket allows a time to live per key (novox/hq ADR 0433 §1): its owner writes each key with its
|
||||
// own, and the server writes a delete marker, kept KeyTTLMarkerTTL, when it ends.
|
||||
KeyTTL bool
|
||||
// MaxMiB is the bucket's cap in MiB (design 47 G §469); zero is no cap.
|
||||
MaxMiB int
|
||||
}
|
||||
|
||||
// MaxBytes is the bucket's cap in bytes as the bus's maximum size: -1, unbounded, where it names none.
|
||||
func (b Bucket) MaxBytes() int64 {
|
||||
if b.MaxMiB <= 0 {
|
||||
return -1
|
||||
}
|
||||
return int64(b.MaxMiB) * 1024 * 1024
|
||||
}
|
||||
|
||||
// PerMachineKeys are the keys a machine's instance of a per-machine state reaches (design 46 7-approval-mark §13): the key
|
||||
// named for its machine, as every module's per-machine key keeps its form, and the keys below it, `<node>.>`; no other
|
||||
// machine's. A membership's per-machine state (membership.StateIssued's Key) reaches the same.
|
||||
func PerMachineKeys(node string) []string { return []string{node, node + ".>"} }
|
||||
|
||||
// BucketName is the bucket a module's state lives in: the module and the local name joined by an
|
||||
// underscore, which neither may contain, so two modules can never derive one bucket.
|
||||
func BucketName(module, name string) string { return module + "_" + name }
|
||||
@@ -61,3 +46,6 @@ type KeyedRead struct {
|
||||
Read string `json:"read"`
|
||||
Key string `json:"key"`
|
||||
}
|
||||
|
||||
// ResolvedBucket is mesh-config's state of resolved values, keyed `<node>.<module>.<key>`.
|
||||
const ResolvedBucket = "mesh-config_resolved"
|
||||
|
||||
@@ -22,6 +22,9 @@ type Declared struct {
|
||||
Watches []Seat
|
||||
// Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152).
|
||||
Invokes []string
|
||||
// Waits are the event families a session may wait on through the tool runner, the tool runner's module's
|
||||
// alone (novox/hq ADR 0367 §2): its account subscribes them for a call, through no consumer.
|
||||
Waits []Wait
|
||||
// State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201).
|
||||
State []Bucket
|
||||
// Logs are the logs it keeps, each a stream its instances append to (novox/hq ADR 0297).
|
||||
@@ -97,3 +100,6 @@ type ProcessPrincipal struct {
|
||||
// seat:<seat>.<verb>, or * for every tool on the mesh.
|
||||
Invokes []string
|
||||
}
|
||||
|
||||
// VaultSeat is the vault's seat: its holder alone asks the desks and the issuing providers in its own name.
|
||||
const VaultSeat = "mesh-vault"
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
package buscompose
|
||||
|
||||
// Wait is one event family the tool runner's module declares: its name as the manifest gives it, and the
|
||||
// emitter — a seat or a module — and the event, as the inventory resolved them.
|
||||
type Wait struct {
|
||||
// Name is the family as the manifest names it, `<emitter>.<event>`: what a session asks for.
|
||||
Name string
|
||||
Emitter string
|
||||
// Seat says the emitter is a seat, whose events live under the seat's namespace (novox/hq ADR 0121).
|
||||
Seat bool
|
||||
// Event is the event, with any further token the subject carries: `decided.claude-code`, the asker of a
|
||||
// seat that names it last (ADR 0259 §3).
|
||||
Event string
|
||||
// Last names what the seat's event subject ends in — "kind" for a kinded bench, "asker" for a seat naming
|
||||
// its caller (ADR 0259 §3) — and is empty for an event that ends at its name. A family needing it is waited
|
||||
// on with it given, and refused without.
|
||||
Last string
|
||||
}
|
||||
+6
-79
@@ -8,13 +8,10 @@
|
||||
// have been delivered. The wait is bounded (Bound, about 30 s, backing off), and said on standard error when it ends
|
||||
// without an answer.
|
||||
//
|
||||
// DoRepeatable is the form for a call its caller keeps the answer of and marks safe to make again (a read, or a write
|
||||
// carrying its own message id): it also asks again on a timeout, and its blackout probe is optional (novox/hq ADR 0668,
|
||||
// extending ADR 0615 §6 to every kept call; issue 1966).
|
||||
// This is the quick fix. The thorough one is a reload that keeps the account's imports, in the bus's holder.
|
||||
package busretry
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
@@ -43,8 +40,7 @@ func Waits(bound, first, most time.Duration) (restore func()) {
|
||||
}
|
||||
|
||||
// Undelivered says err means the request reached no one: no responders, JetStream not enabled (for the account), or
|
||||
// no response from the stream after the client's own retries on no responders (either JetStream client's: the old
|
||||
// one's publish says it for no responders, review round 1 of issue 1966); also said in words by a runtime that
|
||||
// no response from the stream after the client's own retries on no responders; also said in words by a runtime that
|
||||
// passed the error on.
|
||||
func Undelivered(err error) bool {
|
||||
var answered *Answered
|
||||
@@ -53,7 +49,6 @@ func Undelivered(err error) bool {
|
||||
}
|
||||
if errors.Is(err, nats.ErrNoResponders) || errors.Is(err, jetstream.ErrJetStreamNotEnabled) ||
|
||||
errors.Is(err, jetstream.ErrJetStreamNotEnabledForAccount) || errors.Is(err, jetstream.ErrNoStreamResponse) ||
|
||||
errors.Is(err, nats.ErrNoStreamResponse) ||
|
||||
errors.Is(err, nats.ErrJetStreamNotEnabled) || errors.Is(err, nats.ErrJetStreamNotEnabledForAccount) {
|
||||
return true
|
||||
}
|
||||
@@ -71,71 +66,8 @@ func (a *Answered) Error() string { return a.Words }
|
||||
// given, is asked after each such error and Do stops waiting when it says the bus is not blacked out: a call nobody
|
||||
// answers because its callee is not running fails at once, as before. what names the request in the error.
|
||||
func Do[T any](what string, try func() (T, error), blackout func() bool) (T, error) {
|
||||
return do(what, try, func(err error) bool { return Undelivered(err) && (blackout == nil || blackout()) })
|
||||
}
|
||||
|
||||
// DoRepeatable is Do for a call the caller marks safe to make again (a read, or a write carrying its message id, which
|
||||
// the callee deduplicates): a timeout is asked again too, and with no blackout probe a request that reached no one is
|
||||
// asked again whether or not the bus is blacked out, since a callee that is restarting answers nothing for seconds
|
||||
// (novox/hq ADR 0668; issue 1966). Past Bound the error says which fault it last met.
|
||||
func DoRepeatable[T any](what string, try func() (T, error), blackout func() bool) (T, error) {
|
||||
return do(what, try, func(err error) bool {
|
||||
return TimedOut(err) || (Undelivered(err) && (blackout == nil || blackout()))
|
||||
})
|
||||
}
|
||||
|
||||
// Open is what an asker passes a kept call beside the ask's key (design 47 G §351, §425, §460): Open reads the asker's own
|
||||
// state and says whether the key is still open; Replaced, when the asker has one, says an event that replaces the ask was
|
||||
// heard. The ask is made again only while Open answers yes and Replaced says no. An error of Open is written to Logf
|
||||
// (standard error when nil), naming the key, and the ask is made again: a kept call is one the callee takes once by a
|
||||
// key, so asking again is safe and not asking would be silent. Open is always given.
|
||||
type Open struct {
|
||||
Key string
|
||||
Open func(key string) (bool, error)
|
||||
Replaced func() bool
|
||||
Logf func(format string, args ...any)
|
||||
}
|
||||
|
||||
func (o Open) stillOpen() bool {
|
||||
if o.Replaced != nil && o.Replaced() {
|
||||
return false
|
||||
}
|
||||
open, err := o.Open(o.Key)
|
||||
if err != nil {
|
||||
logf := o.Logf
|
||||
if logf == nil {
|
||||
logf = func(f string, a ...any) { fmt.Fprintf(os.Stderr, "[mesh-sdk] "+f+"\n", a...) }
|
||||
}
|
||||
logf("busretry: whether the ask %q is still open could not be read, so it is asked again: %v", o.Key, err)
|
||||
return true
|
||||
}
|
||||
return open
|
||||
}
|
||||
|
||||
// DoKept is DoRepeatable for a kept call that is asked again only while its key is open and not replaced (Open).
|
||||
func DoKept[T any](what string, o Open, try func() (T, error), blackout func() bool) (T, error) {
|
||||
if o.Open == nil {
|
||||
var zero T
|
||||
return zero, fmt.Errorf("%s: a kept call names its key's open function", what)
|
||||
}
|
||||
return do(what, try, func(err error) bool {
|
||||
return (TimedOut(err) || (Undelivered(err) && (blackout == nil || blackout()))) && o.stillOpen()
|
||||
})
|
||||
}
|
||||
|
||||
// TimedOut says err is a request's deadline passing without an answer: the request may have been delivered, so only a
|
||||
// call marked safe to make again (DoRepeatable) asks it again. A callee's own answer (Answered) never counts.
|
||||
func TimedOut(err error) bool {
|
||||
var answered *Answered
|
||||
if err == nil || errors.As(err, &answered) {
|
||||
return false
|
||||
}
|
||||
return errors.Is(err, context.DeadlineExceeded) || errors.Is(err, nats.ErrTimeout)
|
||||
}
|
||||
|
||||
func do[T any](what string, try func() (T, error), again func(error) bool) (T, error) {
|
||||
v, err := try()
|
||||
if !again(err) {
|
||||
if !Undelivered(err) || (blackout != nil && !blackout()) {
|
||||
return v, err
|
||||
}
|
||||
start, wait := time.Now(), FirstWait
|
||||
@@ -143,17 +75,12 @@ func do[T any](what string, try func() (T, error), again func(error) bool) (T, e
|
||||
time.Sleep(wait)
|
||||
wait = min(wait*2, MostWait)
|
||||
v, err = try()
|
||||
if !again(err) {
|
||||
if !Undelivered(err) || (blackout != nil && !blackout()) {
|
||||
return v, err
|
||||
}
|
||||
}
|
||||
if TimedOut(err) {
|
||||
err = fmt.Errorf("%s: no answer within its deadline, asked again for %s: %w", what, time.Since(start).Round(time.Second), err)
|
||||
} else {
|
||||
err = fmt.Errorf("%s: the bus answered nothing for %s, asked again until then (a user-list reload drops the "+
|
||||
"account's imports for a few seconds, a callee that restarts answers nothing; this was longer): %w", what,
|
||||
time.Since(start).Round(time.Second), err)
|
||||
}
|
||||
err = fmt.Errorf("%s: the bus answered nothing for %s, asked again until then (a user-list reload drops the "+
|
||||
"account's imports for a few seconds; this was longer): %w", what, time.Since(start).Round(time.Second), err)
|
||||
fmt.Fprintf(os.Stderr, "[mesh-sdk] %v\n", err)
|
||||
return v, err
|
||||
}
|
||||
|
||||
@@ -14,45 +14,32 @@ import (
|
||||
|
||||
// Issue 774 (bus blackouts at every user-list reload): only a request that reached no one is asked again, within the
|
||||
// bound; a blackout probe that says the bus answers ends the wait at once.
|
||||
func subABusThatAnswersNothingIsWaitedOut(t *testing.T) {
|
||||
func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
|
||||
for _, row := range []struct {
|
||||
name string
|
||||
errs []error // what each try answers; the last repeats
|
||||
blackout func() bool
|
||||
tries int // at least
|
||||
atMost int
|
||||
delivered bool
|
||||
repeatable bool // DoRepeatable, a call marked safe to make again (ADR 0668)
|
||||
name string
|
||||
errs []error // what each try answers; the last repeats
|
||||
blackout func() bool
|
||||
tries int // at least
|
||||
atMost int
|
||||
delivered bool
|
||||
}{
|
||||
{"no responders, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, nil, 3, 3, true, false},
|
||||
{"jetstream not enabled, then an answer", []error{jetstream.ErrJetStreamNotEnabled, nil}, nil, 2, 2, true, false},
|
||||
{"said in words by a runtime", []error{errors.New("nats: no responders available for request"), nil}, nil, 2, 2, true, false},
|
||||
{"a timeout may have been delivered: never asked again", []error{context.DeadlineExceeded}, nil, 1, 1, false, false},
|
||||
{"an answer that refuses is an answer", []error{errors.New("refused")}, nil, 1, 1, false, false},
|
||||
{"nobody answers on a bus that answers: the callee is not running", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false, false},
|
||||
{"blacked out past the bound: said", []error{jetstream.ErrJetStreamNotEnabledForAccount}, func() bool { return true }, 2, 1000, false, false},
|
||||
// Issue 1966, ADR 0668: a call kept and safe to make again is asked again on a timeout, and on no responders
|
||||
// from a callee that restarts on a bus that answers; Do still asks neither.
|
||||
{"repeatable: timed out twice, then an answer", []error{context.DeadlineExceeded, nats.ErrTimeout, nil}, nil, 3, 3, true, true},
|
||||
{"repeatable: no responders with no probe, then an answer", []error{nats.ErrNoResponders, nil}, nil, 2, 2, true, true},
|
||||
{"repeatable: a callee's answer is an answer", []error{&Answered{Words: "context deadline exceeded"}}, nil, 1, 1, false, true},
|
||||
{"repeatable: nobody answers and the probe says the bus answers", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false, true},
|
||||
{"repeatable: no response from the stream, either client's, then an answer", []error{nats.ErrNoStreamResponse, jetstream.ErrNoStreamResponse, nil}, nil, 3, 3, true, true},
|
||||
{"timed out past the bound: said", []error{context.DeadlineExceeded}, nil, 2, 1000, false, true},
|
||||
{"no responders, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, nil, 3, 3, true},
|
||||
{"jetstream not enabled, then an answer", []error{jetstream.ErrJetStreamNotEnabled, nil}, nil, 2, 2, true},
|
||||
{"said in words by a runtime", []error{errors.New("nats: no responders available for request"), nil}, nil, 2, 2, true},
|
||||
{"a timeout may have been delivered: never asked again", []error{context.DeadlineExceeded}, nil, 1, 1, false},
|
||||
{"an answer that refuses is an answer", []error{errors.New("refused")}, nil, 1, 1, false},
|
||||
{"nobody answers on a bus that answers: the callee is not running", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false},
|
||||
{"blacked out past the bound: said", []error{jetstream.ErrJetStreamNotEnabledForAccount}, func() bool { return true }, 2, 1000, false},
|
||||
} {
|
||||
// A row that ends in an answer waits on a bound of seconds, which no slow machine reaches; only the row that
|
||||
// runs past it has a short one, and asks only that it was asked again at least once.
|
||||
bound := 10 * time.Second
|
||||
if strings.Contains(row.name, "past the bound") {
|
||||
if strings.HasPrefix(row.name, "blacked out") {
|
||||
bound = 100 * time.Millisecond
|
||||
}
|
||||
t.Cleanup(Waits(bound, time.Millisecond, 4*time.Millisecond))
|
||||
tries := 0
|
||||
do := Do[int]
|
||||
if row.repeatable {
|
||||
do = DoRepeatable[int]
|
||||
}
|
||||
_, err := do("the test", func() (int, error) {
|
||||
_, err := Do("the test", func() (int, error) {
|
||||
e := row.errs[min(tries, len(row.errs)-1)]
|
||||
tries++
|
||||
return tries, e
|
||||
@@ -63,55 +50,5 @@ func subABusThatAnswersNothingIsWaitedOut(t *testing.T) {
|
||||
if strings.HasPrefix(row.name, "blacked out") && (!Undelivered(err) || !strings.Contains(fmt.Sprint(err), "the bus answered nothing")) {
|
||||
t.Errorf("%s: %v", row.name, err)
|
||||
}
|
||||
if strings.HasPrefix(row.name, "timed out") && (!TimedOut(err) || !strings.Contains(fmt.Sprint(err), "the test: no answer within its deadline")) {
|
||||
t.Errorf("%s: %v", row.name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Design 47 G §351, §460: a kept call is asked again only while its key is open and not replaced; an error of `open` is
|
||||
// logged naming the key and the ask is made again.
|
||||
func subAKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen(t *testing.T) {
|
||||
for _, row := range []struct {
|
||||
name string
|
||||
open func(n int) (bool, error) // by the number of asks made so far
|
||||
replaced bool
|
||||
tries int
|
||||
logged bool
|
||||
}{
|
||||
{"open throughout, then answered", func(int) (bool, error) { return true, nil }, false, 3, false},
|
||||
{"closed after the first ask: not asked again", func(int) (bool, error) { return false, nil }, false, 1, false},
|
||||
{"replaced: not asked again", func(int) (bool, error) { return true, nil }, true, 1, false},
|
||||
{"open errs: logged and asked again", func(int) (bool, error) { return false, errors.New("state unreadable") }, false, 3, true},
|
||||
} {
|
||||
t.Cleanup(Waits(10*time.Second, time.Millisecond, 4*time.Millisecond))
|
||||
tries, logs := 0, []string{}
|
||||
o := Open{Key: "ask-7", Open: func(k string) (bool, error) {
|
||||
if k != "ask-7" {
|
||||
t.Errorf("%s: open asked about %q", row.name, k)
|
||||
}
|
||||
return row.open(tries)
|
||||
}, Replaced: func() bool { return row.replaced },
|
||||
Logf: func(f string, a ...any) { logs = append(logs, fmt.Sprintf(f, a...)) }}
|
||||
_, err := DoKept("the test", o, func() (int, error) {
|
||||
tries++
|
||||
if tries < 3 {
|
||||
return 0, nats.ErrNoResponders
|
||||
}
|
||||
return tries, nil
|
||||
}, nil)
|
||||
if tries != row.tries || (row.tries == 3) != (err == nil) || (len(logs) > 0) != row.logged ||
|
||||
(row.logged && !strings.Contains(logs[0], "ask-7")) {
|
||||
t.Errorf("%s: %d asks, err %v, logs %v", row.name, tries, err, logs)
|
||||
}
|
||||
}
|
||||
if _, err := DoKept("x", Open{Key: "k"}, func() (int, error) { return 0, nil }, nil); err == nil {
|
||||
t.Errorf("a kept call with no open function was made")
|
||||
}
|
||||
}
|
||||
|
||||
// TestABusThatAnswersNothingIsWaitedOut runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
|
||||
func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
|
||||
t.Run("ABusThatAnswersNothingIsWaitedOut", subABusThatAnswersNothingIsWaitedOut)
|
||||
t.Run("AKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen", subAKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen)
|
||||
}
|
||||
|
||||
@@ -1,158 +0,0 @@
|
||||
package bussnap
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/busarchive"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// The keeper's restore in place of a snapshot it took (design 47 G §681): each stream of the file put back under its own
|
||||
// name, only where no stream of that name is live.
|
||||
|
||||
// APIStreamDelete is the subject the keeper removes its own partial stream with.
|
||||
const APIStreamDelete = "$JS.API.STREAM.DELETE."
|
||||
|
||||
const restoreChunk = 64 * 1024
|
||||
|
||||
// ManifestOf reads the manifest of a snapshot file: the streams it holds, without unpacking them.
|
||||
func ManifestOf(path string) (*busarchive.Manifest, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
tr := tar.NewReader(f)
|
||||
hdr, err := tr.Next()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the snapshot does not read: %w", err)
|
||||
}
|
||||
if hdr.Name != busarchive.ManifestName {
|
||||
return nil, fmt.Errorf("the snapshot starts with %q, not its manifest", hdr.Name)
|
||||
}
|
||||
raw, err := io.ReadAll(io.LimitReader(tr, 64<<20))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var m busarchive.Manifest
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
return nil, fmt.Errorf("its manifest: %w", err)
|
||||
}
|
||||
return &m, nil
|
||||
}
|
||||
|
||||
// Live says whether a stream of that name exists.
|
||||
func Live(ctx context.Context, nc *nats.Conn, name string) (bool, error) {
|
||||
err := Request(ctx, nc, APIStreamInfo+name, nil, nil)
|
||||
var api *APIError
|
||||
switch {
|
||||
case err == nil:
|
||||
return true, nil
|
||||
case errors.As(err, &api) && api.Code == 404:
|
||||
return false, nil
|
||||
}
|
||||
return false, err
|
||||
}
|
||||
|
||||
// DeleteStream removes a stream; one that is not there is no error.
|
||||
func DeleteStream(ctx context.Context, nc *nats.Conn, name string) error {
|
||||
err := Request(ctx, nc, APIStreamDelete+name, nil, nil)
|
||||
var api *APIError
|
||||
if errors.As(err, &api) && api.Code == 404 {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// RestoreFile puts every stream of the snapshot file back under its own name, each sent in chunks the server answers; it
|
||||
// ends with the context's bound. A stream already there is the caller's to have refused beforehand.
|
||||
func RestoreFile(ctx context.Context, nc *nats.Conn, path, scratch string, log func(string, ...any)) error {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
a, err := busarchive.ReadArchive(f, scratch)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer a.Close()
|
||||
if err := a.Verify(); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, s := range a.Manifest.Streams {
|
||||
started := time.Now()
|
||||
if err := restoreStream(ctx, nc, a, s); err != nil {
|
||||
return fmt.Errorf("stream %s: %w", s.Name, err)
|
||||
}
|
||||
log("%s: restored in %.2fs", s.Name, time.Since(started).Seconds())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func restoreStream(ctx context.Context, nc *nats.Conn, a *busarchive.Archive, s busarchive.StreamEntry) error {
|
||||
meta, archive, err := a.Open(s)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer archive.Close()
|
||||
var req map[string]json.RawMessage
|
||||
if err := json.Unmarshal(meta, &req); err != nil || req["config"] == nil {
|
||||
return errors.New("its meta is not a stream's configuration and state")
|
||||
}
|
||||
var resp struct {
|
||||
DeliverSubject string `json:"deliver_subject"`
|
||||
}
|
||||
if err := Request(ctx, nc, APIStreamRestore+s.Name, req, &resp); err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.DeliverSubject == "" {
|
||||
return errors.New("the server gave nowhere to send the archive")
|
||||
}
|
||||
buf := make([]byte, restoreChunk)
|
||||
for {
|
||||
n, err := archive.Read(buf)
|
||||
if n > 0 {
|
||||
reply, rerr := nc.RequestWithContext(ctx, resp.DeliverSubject, buf[:n])
|
||||
if rerr != nil {
|
||||
return fmt.Errorf("sending the archive: %w", rerr)
|
||||
}
|
||||
if len(reply.Data) > 0 {
|
||||
return fmt.Errorf("the server refused the archive: %s", Truncate(reply.Data))
|
||||
}
|
||||
}
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
final, err := nc.RequestWithContext(ctx, resp.DeliverSubject, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("finishing the restore: %w", err)
|
||||
}
|
||||
var created struct {
|
||||
Error *APIError `json:"error"`
|
||||
State StreamState `json:"state"`
|
||||
}
|
||||
if err := json.Unmarshal(final.Data, &created); err != nil {
|
||||
return fmt.Errorf("the server's last answer is not JSON: %q", Truncate(final.Data))
|
||||
}
|
||||
if created.Error != nil {
|
||||
return created.Error
|
||||
}
|
||||
if created.State.LastSeq < s.LastSeq || (s.Messages > 0 && created.State.Messages == 0) {
|
||||
return fmt.Errorf("restored with %d message(s) up to sequence %d; the snapshot held %d up to %d",
|
||||
created.State.Messages, created.State.LastSeq, s.Messages, s.LastSeq)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,63 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package bussnap_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/bussnap"
|
||||
"git.novox.be/novox/mesh-sdk/go/testbus"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// TestOneStreamIsTakenRemovedAndRestored: the take and restore of one stream is mesh-sdk's (design 47 G §655, A-125):
|
||||
// a stream's snapshot taken with Into, the stream deleted, and RestoreFile puts it back under its own name with its
|
||||
// messages; a second restore over the live name is refused by the server.
|
||||
func TestOneStreamIsTakenRemovedAndRestored(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
nc, err := nats.Connect(testbus.URL(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer nc.Close()
|
||||
js, err := nc.JetStream()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := js.AddStream(&nats.StreamConfig{Name: "KEPT_one", Subjects: []string{"kept.one.>"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, m := range []string{"a", "b", "c"} {
|
||||
if _, err := js.Publish("kept.one."+m, []byte(m)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
file, err := bussnap.Into(nc, t.TempDir())(ctx, []string{"KEPT_one"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m, err := bussnap.ManifestOf(file)
|
||||
if err != nil || len(m.Streams) != 1 || m.Streams[0].Name != "KEPT_one" {
|
||||
t.Fatalf("the manifest says %+v, %v; want the one stream KEPT_one", m, err)
|
||||
}
|
||||
if err := bussnap.DeleteStream(ctx, nc, "KEPT_one"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if live, err := bussnap.Live(ctx, nc, "KEPT_one"); err != nil || live {
|
||||
t.Fatalf("live after its deletion: %v, %v", live, err)
|
||||
}
|
||||
if err := bussnap.RestoreFile(ctx, nc, file, t.TempDir(), t.Logf); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
info, err := js.StreamInfo("KEPT_one")
|
||||
if err != nil || info.State.Msgs != 3 || info.State.LastSeq != 3 {
|
||||
t.Fatalf("restored as %+v, %v; want 3 messages up to sequence 3", info, err)
|
||||
}
|
||||
if err := bussnap.RestoreFile(ctx, nc, file, t.TempDir(), t.Logf); err == nil {
|
||||
t.Fatal("a restore over the live name was taken")
|
||||
}
|
||||
}
|
||||
@@ -1,433 +0,0 @@
|
||||
// Package bussnap takes a snapshot of the bus's streams with the server's own snapshot API (novox/hq ADR 0235): the
|
||||
// nightly dump of every stream, and the bus module's keeper's snapshot of one entry before it is rebuilt or removed
|
||||
// (design 25 § 12.3).
|
||||
package bussnap
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/busarchive"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// The JetStream API's subjects this program speaks. Each is granted to the bus's own module and
|
||||
// nothing else is (mesh-controller internal/broker, BusSnapshotGrants); a test holds the two lists
|
||||
// equal against a real server.
|
||||
const (
|
||||
APIStreamNames = "$JS.API.STREAM.NAMES"
|
||||
APIStreamInfo = "$JS.API.STREAM.INFO."
|
||||
APIStreamSnapshot = "$JS.API.STREAM.SNAPSHOT."
|
||||
APIStreamRestore = "$JS.API.STREAM.RESTORE."
|
||||
)
|
||||
|
||||
// ChunkSize is what the server is asked to send at a time: its own default. The server sends up to
|
||||
// an 8 MiB window ahead of the acknowledgements and waits for them past it, so one stream costs the
|
||||
// server at most that much in flight, whatever its size.
|
||||
const ChunkSize = 128 * 1024
|
||||
|
||||
// SnapshotOptions bounds a snapshot.
|
||||
type SnapshotOptions struct {
|
||||
// StreamTimeout is the longest one stream may take; past it the snapshot fails, naming it.
|
||||
StreamTimeout time.Duration
|
||||
// Pause is a rest between two streams, so a night's snapshot is never one long burst.
|
||||
Pause time.Duration
|
||||
// TempDir is where one stream's archive waits until it is written out; the system's when empty.
|
||||
TempDir string
|
||||
Log func(format string, args ...any)
|
||||
// Streams, when set, are the only streams taken (the keeper's snapshot of one entry); every stream when empty.
|
||||
Streams []string
|
||||
}
|
||||
|
||||
// APIError is the error every JetStream API answer may carry.
|
||||
type APIError struct {
|
||||
Code int `json:"code"`
|
||||
ErrCode int `json:"err_code"`
|
||||
Description string `json:"description"`
|
||||
}
|
||||
|
||||
func (e *APIError) Error() string { return fmt.Sprintf("%s (%d)", e.Description, e.Code) }
|
||||
|
||||
// StreamState is what this program reads of a stream's state; the whole of it is kept as the server
|
||||
// gave it, in the stream's meta.
|
||||
type StreamState struct {
|
||||
Messages uint64 `json:"messages"`
|
||||
Bytes uint64 `json:"bytes"`
|
||||
FirstSeq uint64 `json:"first_seq"`
|
||||
LastSeq uint64 `json:"last_seq"`
|
||||
NumDelete int `json:"num_deleted"`
|
||||
Consumers int `json:"consumer_count"`
|
||||
}
|
||||
|
||||
type streamConfig struct {
|
||||
Name string `json:"name"`
|
||||
Subjects []string `json:"subjects"`
|
||||
Retention string `json:"retention"`
|
||||
Storage string `json:"storage"`
|
||||
}
|
||||
|
||||
// Request asks the JetStream API and decodes its answer, refusing an answer that carries an error.
|
||||
func Request(ctx context.Context, nc *nats.Conn, subject string, body any, into any) error {
|
||||
var payload []byte
|
||||
if body != nil {
|
||||
var err error
|
||||
if payload, err = json.Marshal(body); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
rctx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||
defer cancel()
|
||||
msg, err := nc.RequestWithContext(rctx, subject, payload)
|
||||
if err != nil {
|
||||
if errors.Is(err, context.DeadlineExceeded) || errors.Is(err, nats.ErrTimeout) || errors.Is(err, nats.ErrNoResponders) {
|
||||
return fmt.Errorf("%s was not answered (%v): the bus refuses a subject this user is not granted by not answering it", subject, err)
|
||||
}
|
||||
return fmt.Errorf("%s: %w", subject, err)
|
||||
}
|
||||
var wrapped struct {
|
||||
Error *APIError `json:"error"`
|
||||
}
|
||||
if err := json.Unmarshal(msg.Data, &wrapped); err != nil {
|
||||
return fmt.Errorf("%s answered something that is not JSON: %q", subject, Truncate(msg.Data))
|
||||
}
|
||||
if wrapped.Error != nil {
|
||||
return fmt.Errorf("%s: %w", subject, wrapped.Error)
|
||||
}
|
||||
if into != nil {
|
||||
return json.Unmarshal(msg.Data, into)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func Truncate(b []byte) string {
|
||||
if len(b) > 200 {
|
||||
return string(b[:200]) + "…"
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
// StreamNames is every stream in the account, in name order, all pages of them.
|
||||
func StreamNames(ctx context.Context, nc *nats.Conn) ([]string, error) {
|
||||
var names []string
|
||||
for {
|
||||
var page struct {
|
||||
Total int `json:"total"`
|
||||
Offset int `json:"offset"`
|
||||
Limit int `json:"limit"`
|
||||
Streams []string `json:"streams"`
|
||||
}
|
||||
if err := Request(ctx, nc, APIStreamNames, map[string]int{"offset": len(names)}, &page); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
names = append(names, page.Streams...)
|
||||
if len(page.Streams) == 0 || len(names) >= page.Total {
|
||||
break
|
||||
}
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names, nil
|
||||
}
|
||||
|
||||
// Snapshot writes every stream, one at a time, as a tar: the manifest first, then each stream's
|
||||
// meta and archive under streams/<name>/. Any stream failing fails the whole: a partial copy of the
|
||||
// bus written where a whole one is expected is the silent failure this exists to prevent.
|
||||
func Snapshot(ctx context.Context, nc *nats.Conn, out io.Writer, o SnapshotOptions) (*busarchive.Manifest, error) {
|
||||
if o.StreamTimeout <= 0 {
|
||||
o.StreamTimeout = 10 * time.Minute
|
||||
}
|
||||
if o.Log == nil {
|
||||
o.Log = func(string, ...any) {}
|
||||
}
|
||||
m := &busarchive.Manifest{Format: 1, Server: nc.ConnectedServerVersion(), Started: time.Now().UTC()}
|
||||
names, err := StreamNames(ctx, nc)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(o.Streams) > 0 {
|
||||
var only []string
|
||||
for _, n := range names {
|
||||
if slices.Contains(o.Streams, n) {
|
||||
only = append(only, n)
|
||||
}
|
||||
}
|
||||
if len(only) != len(o.Streams) {
|
||||
return nil, fmt.Errorf("the bus holds %v of the streams %v asked for: a snapshot missing one would pass for a whole one", only, o.Streams)
|
||||
}
|
||||
names = only
|
||||
}
|
||||
if len(names) == 0 {
|
||||
// The mesh's own streams always exist once the controller has run; a bus with none is not
|
||||
// one to call backed up.
|
||||
return nil, errors.New("the bus holds no streams; a snapshot of nothing would pass for a backup")
|
||||
}
|
||||
|
||||
type taken struct {
|
||||
entry busarchive.StreamEntry
|
||||
meta []byte
|
||||
archive *os.File
|
||||
}
|
||||
var files []taken
|
||||
defer func() {
|
||||
for _, t := range files {
|
||||
t.archive.Close()
|
||||
os.Remove(t.archive.Name())
|
||||
}
|
||||
}()
|
||||
|
||||
for i, name := range names {
|
||||
if i > 0 && o.Pause > 0 {
|
||||
select {
|
||||
case <-time.After(o.Pause):
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
}
|
||||
}
|
||||
var info struct {
|
||||
Config streamConfig `json:"config"`
|
||||
}
|
||||
if err := Request(ctx, nc, APIStreamInfo+name, nil, &info); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if info.Config.Storage == "memory" {
|
||||
m.Skipped = append(m.Skipped, busarchive.Skipped{Name: name, Why: "memory storage: it holds nothing across a restart, and the server cannot snapshot it"})
|
||||
o.Log("%s: skipped, memory storage", name)
|
||||
continue
|
||||
}
|
||||
f, err := os.CreateTemp(o.TempDir, "mesh-nats-snapshot-*")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
files = append(files, taken{archive: f})
|
||||
sctx, cancel := context.WithTimeout(ctx, o.StreamTimeout)
|
||||
entry, meta, err := snapshotStream(sctx, nc, name, f)
|
||||
cancel()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("stream %s: %w", name, err)
|
||||
}
|
||||
files[len(files)-1].entry, files[len(files)-1].meta = entry, meta
|
||||
m.Messages += entry.Messages
|
||||
m.StreamBytes += int64(entry.Bytes)
|
||||
m.ArchiveBytes += entry.ArchiveBytes
|
||||
o.Log("%s: %d message(s), %s, last sequence %d, %d consumer(s) → %s in %.2fs",
|
||||
name, entry.Messages, Human(int64(entry.Bytes)), entry.LastSeq, entry.Consumers, Human(entry.ArchiveBytes), entry.Seconds)
|
||||
}
|
||||
m.Finished = time.Now().UTC()
|
||||
m.Seconds = m.Finished.Sub(m.Started).Seconds()
|
||||
for _, t := range files {
|
||||
m.Streams = append(m.Streams, t.entry)
|
||||
}
|
||||
|
||||
tw := tar.NewWriter(out)
|
||||
manifest, err := json.MarshalIndent(m, "", " ")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := writeEntry(tw, busarchive.ManifestName, append(manifest, '\n'), m.Finished); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, t := range files {
|
||||
if err := writeEntry(tw, t.entry.Meta, t.meta, m.Finished); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := t.archive.Seek(0, io.SeekStart); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
hdr := &tar.Header{Name: t.entry.Archive, Mode: 0o600, Size: t.entry.ArchiveBytes, ModTime: m.Finished, Format: tar.FormatPAX}
|
||||
if err := tw.WriteHeader(hdr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := io.Copy(tw, t.archive); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if err := tw.Close(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
func writeEntry(tw *tar.Writer, name string, body []byte, at time.Time) error {
|
||||
if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0o600, Size: int64(len(body)), ModTime: at, Format: tar.FormatPAX}); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err := tw.Write(body)
|
||||
return err
|
||||
}
|
||||
|
||||
// snapshotStream takes one stream's snapshot into f: the server's chunks, each acknowledged as it
|
||||
// arrives so the server's window keeps moving, until the server's last, empty message. It answers
|
||||
// the stream's entry (the archive's size and checksum included) and its meta.
|
||||
//
|
||||
// The protocol is the server's (nats-server jetstream_api.go, streamSnapshot): the request names a
|
||||
// subject to deliver to; the server answers with the stream's configuration and its state at the
|
||||
// snapshot, then sends the archive in chunks whose reply subject is the acknowledgement it waits for
|
||||
// past its window — two seconds without one and it gives up with "408 No Flow Response". The end is
|
||||
// an empty message whose status header says 204, or the error that ended it.
|
||||
func snapshotStream(ctx context.Context, nc *nats.Conn, name string, f *os.File) (busarchive.StreamEntry, []byte, error) {
|
||||
started := time.Now()
|
||||
deliver := nc.NewRespInbox()
|
||||
|
||||
hash := sha256.New()
|
||||
var written int64
|
||||
var mu sync.Mutex
|
||||
var failed error
|
||||
done := make(chan struct{})
|
||||
var once sync.Once
|
||||
finish := func(err error) {
|
||||
mu.Lock()
|
||||
if failed == nil {
|
||||
failed = err
|
||||
}
|
||||
mu.Unlock()
|
||||
once.Do(func() { close(done) })
|
||||
}
|
||||
sub, err := nc.Subscribe(deliver, func(msg *nats.Msg) {
|
||||
if len(msg.Data) == 0 {
|
||||
if status := msg.Header.Get("Status"); status != "" && status != "204" {
|
||||
finish(fmt.Errorf("the server ended the snapshot: %s %s", status, msg.Header.Get("Description")))
|
||||
return
|
||||
}
|
||||
finish(nil)
|
||||
return
|
||||
}
|
||||
// Acknowledged before it is written: the chunk is already in memory, and a slow disk must not
|
||||
// read to the server as a reader that went away.
|
||||
if msg.Reply != "" {
|
||||
if err := nc.Publish(msg.Reply, nil); err != nil {
|
||||
finish(fmt.Errorf("acknowledging a chunk: %w", err))
|
||||
return
|
||||
}
|
||||
}
|
||||
if _, err := f.Write(msg.Data); err != nil {
|
||||
finish(fmt.Errorf("writing the archive: %w", err))
|
||||
return
|
||||
}
|
||||
hash.Write(msg.Data)
|
||||
written += int64(len(msg.Data))
|
||||
})
|
||||
if err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
defer sub.Unsubscribe()
|
||||
// Every chunk is held until the callback has it; the window bounds how many there can be.
|
||||
if err := sub.SetPendingLimits(-1, -1); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
if err := nc.FlushWithContext(ctx); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
Config json.RawMessage `json:"config"`
|
||||
State json.RawMessage `json:"state"`
|
||||
}
|
||||
req := map[string]any{"deliver_subject": deliver, "chunk_size": ChunkSize}
|
||||
if err := Request(ctx, nc, APIStreamSnapshot+name, req, &resp); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
select {
|
||||
case <-done:
|
||||
case <-ctx.Done():
|
||||
return busarchive.StreamEntry{}, nil, fmt.Errorf("not finished within its bound: %w", ctx.Err())
|
||||
}
|
||||
// The subscription is drained before the archive is read back, so no callback is still writing.
|
||||
_ = sub.Unsubscribe()
|
||||
mu.Lock()
|
||||
err = failed
|
||||
mu.Unlock()
|
||||
if err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
if written == 0 {
|
||||
return busarchive.StreamEntry{}, nil, errors.New("the server sent an empty archive")
|
||||
}
|
||||
if err := f.Sync(); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
|
||||
var cfg streamConfig
|
||||
var state StreamState
|
||||
if err := json.Unmarshal(resp.Config, &cfg); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, fmt.Errorf("its configuration: %w", err)
|
||||
}
|
||||
if err := json.Unmarshal(resp.State, &state); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, fmt.Errorf("its state: %w", err)
|
||||
}
|
||||
// The archive reads to its end — s2 frames, then a tar — before it is called a snapshot.
|
||||
if _, err := f.Seek(0, io.SeekStart); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
if err := busarchive.ReadsThrough(f); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, fmt.Errorf("the archive the server sent does not read: %w", err)
|
||||
}
|
||||
|
||||
// The meta, in the shape the restore API takes and the nats CLI writes (backup.json): the
|
||||
// server's own configuration and state, kept as raw JSON so a field this program does not know
|
||||
// is never dropped on the way back.
|
||||
meta, err := json.MarshalIndent(map[string]json.RawMessage{"config": resp.Config, "state": resp.State}, "", " ")
|
||||
if err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
metaSum := sha256.Sum256(meta)
|
||||
return busarchive.StreamEntry{
|
||||
Name: name, Subjects: cfg.Subjects, Retention: cfg.Retention,
|
||||
Messages: state.Messages, Bytes: state.Bytes, FirstSeq: state.FirstSeq, LastSeq: state.LastSeq,
|
||||
Deleted: state.NumDelete, Consumers: state.Consumers,
|
||||
Meta: "streams/" + name + "/backup.json", MetaSHA256: hex.EncodeToString(metaSum[:]),
|
||||
Archive: "streams/" + name + "/stream.tar.s2", ArchiveBytes: written,
|
||||
ArchiveSHA256: hex.EncodeToString(hash.Sum(nil)), Seconds: time.Since(started).Seconds(),
|
||||
}, meta, nil
|
||||
}
|
||||
|
||||
// Human is a size in bytes as a person reads it: 1.5 MiB.
|
||||
func Human(n int64) string {
|
||||
const unit = 1024
|
||||
if n < unit {
|
||||
return fmt.Sprintf("%d B", n)
|
||||
}
|
||||
div, exp := int64(unit), 0
|
||||
for v := n / unit; v >= unit; v /= unit {
|
||||
div *= unit
|
||||
exp++
|
||||
}
|
||||
return fmt.Sprintf("%.1f %ciB", float64(n)/float64(div), "KMGTPE"[exp])
|
||||
}
|
||||
|
||||
// Into is the keeper's snapshot of some streams before it rebuilds or removes them (design 25 § 12.3): one tar under
|
||||
// dir, named for the streams and the time, written whole before its name is given, so a partial one is never taken
|
||||
// for it. It answers the file's path.
|
||||
func Into(nc *nats.Conn, dir string) func(ctx context.Context, streams []string) (string, error) {
|
||||
return func(ctx context.Context, streams []string) (string, error) {
|
||||
if dir == "" {
|
||||
return "", errors.New("no snapshots directory is set (MESH_NATS_SNAPSHOTS_DIR), so no snapshot is taken and nothing is removed")
|
||||
}
|
||||
name := filepath.Join(dir, "entry-"+strings.Join(streams, "+")+"-"+time.Now().UTC().Format("20060102T150405.000000Z")+".tar")
|
||||
f, err := os.CreateTemp(dir, ".entry-*.partial")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer os.Remove(f.Name())
|
||||
if _, err := Snapshot(ctx, nc, f, SnapshotOptions{Streams: streams, TempDir: dir}); err != nil {
|
||||
f.Close()
|
||||
return "", err
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return name, os.Rename(f.Name(), name)
|
||||
}
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// busStateCalls are the method prefixes of the calls a bundle makes on its own bus state through the runtime (novox/hq
|
||||
// ADR 0595 §9, design 25 § 12.5): their shapes and names are package buscalls's, and package busstate the typed way
|
||||
// to make them.
|
||||
var busStateCalls = []string{"mesh/objects.", "mesh/stream.", "mesh/workqueue.", "mesh/ledger."}
|
||||
|
||||
// BusState asks the runtime one call on this bundle's own bus state, an entry its module asks for in `requires`, named
|
||||
// by its local name in params; the runtime serves it from the module's own bindings only. A method that is none of
|
||||
// those calls is refused here, before the runtime is asked.
|
||||
func BusState(method string, params any) (json.RawMessage, error) {
|
||||
known := false
|
||||
for _, p := range busStateCalls {
|
||||
known = known || strings.HasPrefix(method, p)
|
||||
}
|
||||
if !known {
|
||||
return nil, fmt.Errorf("%s is no call on a bundle's own bus state (%s)", method, strings.Join(busStateCalls, ", "))
|
||||
}
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return c.ask(method, params, 60*time.Second)
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// Design 47 G §587, §622: the ledger's upkeep calls have their requests and answers, and every family its word.
|
||||
func subTheLedgerUpkeepCallsAndTheirWords(t *testing.T) {
|
||||
var removal buscalls.LedgerRemoveRequest
|
||||
if err := json.Unmarshal([]byte(`{"ledger":"records","instance":"r1"}`), &removal); err != nil || removal.Ledger != "records" || removal.Instance != "r1" {
|
||||
t.Errorf("the removal read as %+v (%v)", removal, err)
|
||||
}
|
||||
raw, _ := json.Marshal(buscalls.LedgerRemoved{Instance: "r1", Operations: 3, Bytes: 900})
|
||||
if string(raw) != `{"instance":"r1","operations":3,"bytes":900}` {
|
||||
t.Errorf("the removal's answer reads %s", raw)
|
||||
}
|
||||
raw, _ = json.Marshal(buscalls.Fill{Count: 2, Bytes: 10, MaxBytes: 100})
|
||||
if string(raw) != `{"count":2,"bytes":10,"max_bytes":100}` || buscalls.LedgerFill != "mesh/ledger.fill" || buscalls.LedgerRemove != "mesh/ledger.remove" {
|
||||
t.Errorf("the fill reads %s, calls %s %s", raw, buscalls.LedgerFill, buscalls.LedgerRemove)
|
||||
}
|
||||
for word, want := range map[string]string{buscalls.BusStateCalls: "bus-state-calls", buscalls.LedgerCalls: "ledger-calls",
|
||||
buscalls.LedgerUpkeepCalls: "ledger-upkeep-calls", buscalls.SentObjectsCalls: "sent-objects-calls"} {
|
||||
if word != want {
|
||||
t.Errorf("the word %q is %q", want, word)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,84 +0,0 @@
|
||||
// Package busstate is how a bundle uses its own bus state through the runtime (novox/hq ADR 0595 §9, design 25 §
|
||||
// 12.5): the object store's put, get, list, delete and fill, each naming an entry by the local name its module asks for
|
||||
// it by in `requires`. The runtime serves the calls from the module's own bindings, its own entries only. The shapes
|
||||
// are package buscalls's; this package makes the calls.
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// Objects is one object store the module asks for in `requires` (the template objects, ADR 0595 §2), by its local name.
|
||||
type Objects struct{ local string }
|
||||
|
||||
// ObjectsOf is the object store of that local name.
|
||||
func ObjectsOf(local string) Objects { return Objects{local: local} }
|
||||
|
||||
func call[T any](method string, params any) (T, error) {
|
||||
var out T
|
||||
raw, err := stdio.BusState(method, params)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(raw, &out); err != nil {
|
||||
return out, fmt.Errorf("the runtime's answer to %s: %w", method, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Put stores one object whole under its name, replacing one of that name, and answers it as the store holds it.
|
||||
//
|
||||
// A value larger than buscalls.ChunkBytes is put as a session of chunks (design 47 G §518, §345): every object is one
|
||||
// object, never split over several.
|
||||
func (o Objects) Put(name string, data []byte, description string) (buscalls.ObjectInfo, error) {
|
||||
if len(data) > buscalls.ChunkBytes {
|
||||
return o.putSession(name, data, description)
|
||||
}
|
||||
return call[buscalls.ObjectInfo](buscalls.ObjectsPut, buscalls.ObjectsPutRequest{Objects: o.local, Name: name,
|
||||
Data: data, Description: description})
|
||||
}
|
||||
|
||||
// Get is one object, its info and its bytes.
|
||||
//
|
||||
// A value the runtime answers as a session (`chunked`) is read chunk by chunk and checked against its digest (design 47 G §490).
|
||||
func (o Objects) Get(name string) (buscalls.ObjectData, error) {
|
||||
raw, err := stdio.BusState(buscalls.ObjectsGet, buscalls.ObjectsNameRequest{Objects: o.local, Name: name})
|
||||
if err != nil {
|
||||
return buscalls.ObjectData{}, err
|
||||
}
|
||||
return ReadObject(name, raw)
|
||||
}
|
||||
|
||||
// ReadObject is the object a runtime's answer to a get (or to a sent-get, design 47 G §620) holds: the value itself, or the
|
||||
// session of chunks it opened, read whole.
|
||||
func ReadObject(name string, raw json.RawMessage) (buscalls.ObjectData, error) {
|
||||
if c, ok := chunkedAnswer(raw); ok {
|
||||
return getSession(name, c)
|
||||
}
|
||||
var out buscalls.ObjectData
|
||||
if err := json.Unmarshal(raw, &out); err != nil {
|
||||
return out, fmt.Errorf("the runtime's answer for %s: %w", name, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// List is every object the store holds.
|
||||
func (o Objects) List() ([]buscalls.ObjectInfo, error) {
|
||||
l, err := call[buscalls.ObjectsListed](buscalls.ObjectsList, buscalls.ObjectsRequest{Objects: o.local})
|
||||
return l.Objects, err
|
||||
}
|
||||
|
||||
// Delete removes one object.
|
||||
func (o Objects) Delete(name string) error {
|
||||
_, err := call[buscalls.Done](buscalls.ObjectsDelete, buscalls.ObjectsNameRequest{Objects: o.local, Name: name})
|
||||
return err
|
||||
}
|
||||
|
||||
// Fill is how much the store holds against its cap.
|
||||
func (o Objects) Fill() (buscalls.Fill, error) {
|
||||
return call[buscalls.Fill](buscalls.ObjectsInfo, buscalls.ObjectsRequest{Objects: o.local})
|
||||
}
|
||||
@@ -1,109 +0,0 @@
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// A bundle keeps objects in its own object store through the runtime (ADR 0595 §2, §9; A3's request for distribution's
|
||||
// incoming): each call is the runtime's method of buscalls, naming the store by its local name, and answers as the
|
||||
// runtime answered; a method that is no bus-state call is refused before the runtime is asked.
|
||||
func subABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
|
||||
t.Setenv(stdio.ServedAs, "distribution")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
type result struct {
|
||||
what string
|
||||
err error
|
||||
got any
|
||||
}
|
||||
results := make(chan result, 8)
|
||||
go func() {
|
||||
_ = stdio.ServeOn("", []stdio.Tool{{Name: "keep", Run: func(map[string]any) (any, error) {
|
||||
go func() {
|
||||
in := ObjectsOf("incoming")
|
||||
info, err := in.Put("t1.part0", []byte("hello"), "a part")
|
||||
results <- result{"put", err, info.Digest}
|
||||
data, err := in.Get("t1.part0")
|
||||
results <- result{"get", err, string(data.Data)}
|
||||
list, err := in.List()
|
||||
results <- result{"list", err, len(list)}
|
||||
results <- result{"delete", in.Delete("t1.part0"), nil}
|
||||
fill, err := in.Fill()
|
||||
results <- result{"fill", err, fill.MaxBytes}
|
||||
_, err = stdio.BusState("mesh/state.get", nil)
|
||||
results <- result{"refused", err, nil}
|
||||
}()
|
||||
return "started", nil
|
||||
}}}, inR, outW)
|
||||
}()
|
||||
lines := bufio.NewScanner(outR)
|
||||
next := func() map[string]any {
|
||||
done := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m map[string]any
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
done <- m
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case m := <-done:
|
||||
return m
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the bundle said nothing")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
send := func(m map[string]any) { b, _ := json.Marshal(m); _, _ = inW.Write(append(b, '\n')) }
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": map[string]any{"name": "keep"}})
|
||||
answers := map[string]any{
|
||||
buscalls.ObjectsPut: buscalls.ObjectInfo{Name: "t1.part0", Size: 5, Digest: "SHA-256=x"},
|
||||
buscalls.ObjectsGet: buscalls.ObjectData{Info: buscalls.ObjectInfo{Name: "t1.part0"}, Data: []byte("hello")},
|
||||
buscalls.ObjectsList: buscalls.ObjectsListed{Objects: []buscalls.ObjectInfo{{Name: "t1.part0"}}},
|
||||
buscalls.ObjectsDelete: buscalls.Done{},
|
||||
buscalls.ObjectsInfo: buscalls.Fill{Count: 0, MaxBytes: 20 << 30},
|
||||
}
|
||||
asked := map[string]map[string]any{}
|
||||
for len(asked) < len(answers) {
|
||||
m := next()
|
||||
method, _ := m["method"].(string)
|
||||
if method == "" {
|
||||
continue // the tool's own answer
|
||||
}
|
||||
p, _ := m["params"].(map[string]any)
|
||||
asked[method] = p
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": m["id"], "result": answers[method]})
|
||||
}
|
||||
want := map[string]any{"put": "SHA-256=x", "get": "hello", "list": 1, "delete": nil, "fill": uint64(20 << 30)}
|
||||
for range 5 {
|
||||
r := <-results
|
||||
if r.err != nil || r.got != want[r.what] {
|
||||
t.Errorf("%s answered %v, %v; want %v", r.what, r.got, r.err, want[r.what])
|
||||
}
|
||||
}
|
||||
if r := <-results; r.err == nil {
|
||||
t.Error("a call that is not on the bundle's bus state was asked of the runtime")
|
||||
}
|
||||
for method, p := range asked {
|
||||
if p["objects"] != "incoming" {
|
||||
t.Errorf("%s named the store %v, not its local name", method, p["objects"])
|
||||
}
|
||||
}
|
||||
if p := asked[buscalls.ObjectsPut]; p["name"] != "t1.part0" || p["data"] != "aGVsbG8=" || p["description"] != "a part" {
|
||||
t.Errorf("put asked %v", p)
|
||||
}
|
||||
}
|
||||
|
||||
// TestABundleKeepsObjectsThroughTheRuntime runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
|
||||
func TestABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
|
||||
t.Run("ABundleKeepsObjectsThroughTheRuntime", subABundleKeepsObjectsThroughTheRuntime)
|
||||
t.Run("TheLedgerUpkeepCallsAndTheirWords", subTheLedgerUpkeepCallsAndTheirWords)
|
||||
t.Run("AValueLargerThanOneChunkIsPutAndGotBySession", subAValueLargerThanOneChunkIsPutAndGotBySession)
|
||||
}
|
||||
@@ -1,113 +0,0 @@
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// The put and get sessions of a value larger than one chunk (design 47 G §518, §345, §490, A-21): a bundle sends and asks the
|
||||
// chunks as calls on its channel to the tool runner, which holds the session; nothing of it is on the bus. The capability
|
||||
// word put-sessions names the two calls, so a tool runner that does not state it refuses the session at once.
|
||||
|
||||
// callID is the bundle's id for one session: random, so two sessions never share one.
|
||||
func callID() string {
|
||||
var b [12]byte
|
||||
_, _ = rand.Read(b[:])
|
||||
return hex.EncodeToString(b[:])
|
||||
}
|
||||
|
||||
// put sends data as a put session: chunks of buscalls.ChunkBytes in order, each answered before the next is sent, the
|
||||
// answer to the last being the put's.
|
||||
func (o Objects) putSession(name string, data []byte, description string) (buscalls.ObjectInfo, error) {
|
||||
if err := stdio.RequireCapability(stdio.CapPutSessions); err != nil {
|
||||
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, larger than one call carries: %w", name, err)
|
||||
}
|
||||
id := callID()
|
||||
for seq, at := uint32(1), 0; ; seq++ {
|
||||
end := min(at+buscalls.ChunkBytes, len(data))
|
||||
last := end == len(data)
|
||||
req := buscalls.ObjectsPutChunkRequest{Objects: o.local, Name: name, Call: id, Seq: seq, Last: last, Data: data[at:end]}
|
||||
if seq == 1 {
|
||||
req.Description = description // the first chunk carries it (design 47 G A-25)
|
||||
}
|
||||
raw, err := stdio.BusState(buscalls.ObjectsPutChunk, req)
|
||||
if err != nil {
|
||||
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, chunk %d: %w", name, seq, err)
|
||||
}
|
||||
var taken buscalls.ObjectsChunkTaken
|
||||
if err := json.Unmarshal(raw, &taken); err != nil {
|
||||
return buscalls.ObjectInfo{}, fmt.Errorf("the runtime's answer to a chunk of %s: %w", name, err)
|
||||
}
|
||||
if taken.Error != "" {
|
||||
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, chunk %d: %s", name, seq, taken.Error)
|
||||
}
|
||||
if last {
|
||||
var info buscalls.ObjectInfo
|
||||
if err := json.Unmarshal(raw, &info); err != nil {
|
||||
return info, fmt.Errorf("the runtime's answer to the put of %s: %w", name, err)
|
||||
}
|
||||
return info, nil
|
||||
}
|
||||
if taken.Seq != seq {
|
||||
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s: chunk %d was answered as chunk %d", name, seq, taken.Seq)
|
||||
}
|
||||
at = end
|
||||
}
|
||||
}
|
||||
|
||||
// getSession reads the chunks of the session the runtime opened for name, and checks the whole against the digest and the
|
||||
// size it answered.
|
||||
func getSession(name string, c buscalls.ObjectsChunked) (buscalls.ObjectData, error) {
|
||||
if err := stdio.RequireCapability(stdio.CapPutSessions); err != nil {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s, larger than one call carries: %w", name, err)
|
||||
}
|
||||
var whole []byte
|
||||
for seq := uint32(1); ; seq++ {
|
||||
raw, err := stdio.BusState(buscalls.ObjectsGetChunk, buscalls.ObjectsGetChunkRequest{Call: c.Call, Seq: seq})
|
||||
if err != nil {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s, chunk %d: %w", name, seq, err)
|
||||
}
|
||||
var chunk buscalls.ObjectsChunk
|
||||
if err := json.Unmarshal(raw, &chunk); err != nil {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("the runtime's chunk %d of %s: %w", seq, name, err)
|
||||
}
|
||||
if chunk.Error != "" {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s, chunk %d: %s", name, seq, chunk.Error)
|
||||
}
|
||||
if chunk.Seq != seq {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s: chunk %d was answered as chunk %d", name, seq, chunk.Seq)
|
||||
}
|
||||
whole = append(whole, chunk.Data...)
|
||||
if chunk.Last {
|
||||
break
|
||||
}
|
||||
}
|
||||
sum := sha256.Sum256(whole)
|
||||
if c.Digest != "" && !strings.EqualFold(c.Digest, "sha256:"+hex.EncodeToString(sum[:])) {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s: what was read is not the value the runtime named by %s", name, c.Digest)
|
||||
}
|
||||
if c.Size != 0 && c.Size != uint64(len(whole)) {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s: %d bytes were read, the runtime said %d", name, len(whole), c.Size)
|
||||
}
|
||||
return buscalls.ObjectData{Info: buscalls.ObjectInfo{Name: name, Size: uint64(len(whole)), Digest: c.Digest}, Data: whole}, nil
|
||||
}
|
||||
|
||||
// chunkedAnswer is the session an answer to a get opens, if it is not the value itself.
|
||||
func chunkedAnswer(raw json.RawMessage) (buscalls.ObjectsChunked, bool) {
|
||||
var c buscalls.ObjectsChunked
|
||||
if json.Unmarshal(raw, &c) != nil {
|
||||
return c, false
|
||||
}
|
||||
switch strings.TrimSpace(string(c.Chunked)) {
|
||||
case "", "null", "0", "false":
|
||||
return c, false
|
||||
}
|
||||
return c, c.Call != ""
|
||||
}
|
||||
@@ -1,126 +0,0 @@
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// serveRuntime runs a bundle whose tool `go` runs fn in a goroutine and answers every call the bundle makes through handle,
|
||||
// until done is closed by fn returning.
|
||||
func serveRuntime(t *testing.T, fn func(), handle func(method string, params map[string]any) any) {
|
||||
t.Helper()
|
||||
t.Setenv(stdio.ServedAs, "distribution")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
_ = stdio.ServeOn("", []stdio.Tool{{Name: "go", Run: func(map[string]any) (any, error) {
|
||||
go func() { fn(); close(done) }()
|
||||
return "started", nil
|
||||
}}}, inR, outW)
|
||||
}()
|
||||
t.Cleanup(func() { _ = inW.Close() })
|
||||
lines := bufio.NewScanner(outR)
|
||||
lines.Buffer(make([]byte, 0, 1<<20), 64<<20)
|
||||
send := func(m map[string]any) { b, _ := json.Marshal(m); _, _ = inW.Write(append(b, '\n')) }
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": map[string]any{"name": "go"}})
|
||||
deadline := time.After(10 * time.Second)
|
||||
for {
|
||||
got := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m map[string]any
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
got <- m
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case <-done:
|
||||
return
|
||||
case <-deadline:
|
||||
t.Fatal("the bundle did not finish")
|
||||
case m := <-got:
|
||||
method, _ := m["method"].(string)
|
||||
if method == "" {
|
||||
continue
|
||||
}
|
||||
p, _ := m["params"].(map[string]any)
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": m["id"], "result": handle(method, p)})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Design 47 G §518, §345, §490 (A-21): a value larger than one chunk is put as chunks in order under one call id, each
|
||||
// answered before the next, with the store named in each; a get the runtime answers as a session is read chunk by chunk and
|
||||
// checked against its digest.
|
||||
func subAValueLargerThanOneChunkIsPutAndGotBySession(t *testing.T) {
|
||||
value := bytes.Repeat([]byte("0123456789abcdef"), (2*buscalls.ChunkBytes+1000)/16)
|
||||
sum := sha256.Sum256(value)
|
||||
digest := "sha256:" + hex.EncodeToString(sum[:])
|
||||
var stored []byte
|
||||
var seqs []uint32
|
||||
calls := map[string]bool{}
|
||||
var gotValue []byte
|
||||
var putErr, getErr, tamperErr error
|
||||
var tampered bool
|
||||
serveRuntime(t, func() {
|
||||
o := ObjectsOf("incoming")
|
||||
_, putErr = o.Put("big", value, "a big one")
|
||||
d, err := o.Get("big")
|
||||
gotValue, getErr = d.Data, err
|
||||
tampered = true
|
||||
_, tamperErr = o.Get("big")
|
||||
}, func(method string, p map[string]any) any {
|
||||
switch method {
|
||||
case stdio.CapabilitiesCall:
|
||||
return map[string]any{"words": []string{stdio.CapPutSessions}}
|
||||
case buscalls.ObjectsPutChunk:
|
||||
calls[p["call"].(string)] = true
|
||||
if p["objects"] != "incoming" || p["name"] != "big" {
|
||||
t.Errorf("a chunk named %v in %v", p["name"], p["objects"])
|
||||
}
|
||||
raw, _ := json.Marshal(p["data"])
|
||||
var b []byte
|
||||
_ = json.Unmarshal(raw, &b)
|
||||
if (p["description"] == "a big one") != (p["seq"].(float64) == 1) {
|
||||
t.Errorf("chunk %v carries the description %v: the first chunk alone does", p["seq"], p["description"])
|
||||
}
|
||||
stored = append(stored, b...)
|
||||
seqs = append(seqs, uint32(p["seq"].(float64)))
|
||||
if p["last"] == true {
|
||||
return buscalls.ObjectInfo{Name: "big", Size: uint64(len(stored))}
|
||||
}
|
||||
return buscalls.ObjectsChunkTaken{Seq: uint32(p["seq"].(float64))}
|
||||
case buscalls.ObjectsGet:
|
||||
d := digest
|
||||
if tampered {
|
||||
d = "sha256:" + hex.EncodeToString(make([]byte, 32))
|
||||
}
|
||||
return map[string]any{"chunked": 3, "size": len(stored), "digest": d, "call": "g1"}
|
||||
case buscalls.ObjectsGetChunk:
|
||||
seq := int(p["seq"].(float64))
|
||||
at := (seq - 1) * buscalls.ChunkBytes
|
||||
return buscalls.ObjectsChunk{Seq: uint32(seq), Last: seq == 3, Data: stored[at:min(at+buscalls.ChunkBytes, len(stored))]}
|
||||
}
|
||||
t.Errorf("unexpected call %s", method)
|
||||
return nil
|
||||
})
|
||||
if putErr != nil || getErr != nil {
|
||||
t.Fatalf("put %v, get %v", putErr, getErr)
|
||||
}
|
||||
if len(seqs) != 3 || seqs[0] != 1 || seqs[1] != 2 || seqs[2] != 3 || len(calls) != 1 || !bytes.Equal(stored, value) || !bytes.Equal(gotValue, value) {
|
||||
t.Errorf("chunks %v under %d call ids, stored %d bytes, got %d", seqs, len(calls), len(stored), len(gotValue))
|
||||
}
|
||||
if tamperErr == nil {
|
||||
t.Error("a value that is not the one the runtime named by its digest was accepted")
|
||||
}
|
||||
}
|
||||
+6
-38
@@ -22,19 +22,8 @@ const MetaCaller = "mesh/caller"
|
||||
// parts are safe in a subject. Anything else the runtime named is not a caller and reads as Unproven.
|
||||
var busUser = regexp.MustCompile(`^[A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*$`)
|
||||
|
||||
// MetaCall is where in a `tools/call`'s `_meta` the runtime names the id of the call in flight, the id a ledger's
|
||||
// apply names so the runtime stamps the caller it proved as the operation's author (novox/hq ADR 0597 §7):
|
||||
// buscalls.MetaCall, written here too so this package imports nothing of buscalls.
|
||||
const MetaCall = "mesh/call"
|
||||
|
||||
// callID is the shape of a call's id: one subject-safe token, as the runtime makes it. Anything else reads as none.
|
||||
var callID = regexp.MustCompile(`^[A-Za-z0-9_-]{1,128}$`)
|
||||
|
||||
// inFlight is what the runtime said of one call while its tool runs.
|
||||
type inFlight struct{ caller, call string }
|
||||
|
||||
// calls holds the caller and the id of every call while its tool runs, by the arguments map that call was handed.
|
||||
var calls sync.Map // unsafe.Pointer → inFlight
|
||||
// calls holds the caller of every call while its tool runs, by the arguments map that call was handed.
|
||||
var calls sync.Map // unsafe.Pointer → string
|
||||
|
||||
func argsKey(args map[string]any) unsafe.Pointer {
|
||||
return reflect.ValueOf(args).UnsafePointer()
|
||||
@@ -48,32 +37,11 @@ func CallerOf(args map[string]any) string {
|
||||
return Unproven
|
||||
}
|
||||
if c, ok := calls.Load(argsKey(args)); ok {
|
||||
return c.(inFlight).caller
|
||||
return c.(string)
|
||||
}
|
||||
return Unproven
|
||||
}
|
||||
|
||||
// CallIDOf is the id of the call that was handed args, as the runtime named it under MetaCall: what a ledger's apply
|
||||
// names (buscalls.ApplyRequest.Call) so the operation's author is the caller the runtime proved. Empty for arguments
|
||||
// no call was handed, or a call the runtime named no id for; read from `_meta`, never from the arguments.
|
||||
func CallIDOf(args map[string]any) string {
|
||||
if args == nil {
|
||||
return ""
|
||||
}
|
||||
if c, ok := calls.Load(argsKey(args)); ok {
|
||||
return c.(inFlight).call
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// callIn is the call id a `tools/call`'s `_meta` names, or empty.
|
||||
func callIn(meta map[string]any) string {
|
||||
if s, ok := meta[MetaCall].(string); ok && callID.MatchString(s) {
|
||||
return s
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// callerIn is the caller a `tools/call`'s `_meta` names, or Unproven.
|
||||
func callerIn(meta map[string]any) string {
|
||||
if s, ok := meta[MetaCaller].(string); ok && busUser.MatchString(s) {
|
||||
@@ -82,10 +50,10 @@ func callerIn(meta map[string]any) string {
|
||||
return Unproven
|
||||
}
|
||||
|
||||
// runAs runs a tool with the caller and the call's id held for its arguments for as long as it runs.
|
||||
func runAs(t Tool, caller, call string, args map[string]any) (any, error) {
|
||||
// runAs runs a tool with the caller held for its arguments for as long as it runs.
|
||||
func runAs(t Tool, caller string, args map[string]any) (any, error) {
|
||||
key := argsKey(args)
|
||||
calls.Store(key, inFlight{caller, call})
|
||||
calls.Store(key, caller)
|
||||
defer calls.Delete(key)
|
||||
return t.Run(args)
|
||||
}
|
||||
|
||||
+5
-127
@@ -3,37 +3,28 @@ package stdio
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// callWith serves one tool that answers the caller CallerOf reads from its arguments, sends it one tools/call
|
||||
// with the params given, and returns the caller it answered.
|
||||
func callWith(t *testing.T, params map[string]any) string {
|
||||
caller, _ := callAnswering(t, params)
|
||||
return caller
|
||||
}
|
||||
|
||||
// callAnswering is callWith answering the call's id CallIDOf read too.
|
||||
func callAnswering(t *testing.T, params map[string]any) (string, string) {
|
||||
servesAlone(t)
|
||||
t.Helper()
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
go func() {
|
||||
_ = ServeOn("tracker", []Tool{{Name: "who", Run: func(args map[string]any) (any, error) {
|
||||
return map[string]any{"caller": CallerOf(args), "call": CallIDOf(args)}, nil
|
||||
return map[string]any{"caller": CallerOf(args)}, nil
|
||||
}}}, inR, outW)
|
||||
}()
|
||||
defer inW.Close()
|
||||
b, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": params})
|
||||
_, _ = inW.Write(append(b, '\n'))
|
||||
lines := bufio.NewScanner(outR)
|
||||
done := make(chan [2]string, 1)
|
||||
done := make(chan string, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m struct {
|
||||
@@ -46,20 +37,19 @@ func callAnswering(t *testing.T, params map[string]any) (string, string) {
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
var answer struct {
|
||||
Caller string `json:"caller"`
|
||||
Call string `json:"call"`
|
||||
}
|
||||
if len(m.Result.Content) > 0 {
|
||||
_ = json.Unmarshal([]byte(m.Result.Content[0].Text), &answer)
|
||||
}
|
||||
done <- [2]string{answer.Caller, answer.Call}
|
||||
done <- answer.Caller
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case got := <-done:
|
||||
return got[0], got[1]
|
||||
return got
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the bundle did not answer the call")
|
||||
return "", ""
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
@@ -72,28 +62,6 @@ func TestAToolReadsTheCallerTheRuntimeNamed(t *testing.T) {
|
||||
if got != "shanks.node-tools" {
|
||||
t.Fatalf("the tool read caller %q, want the one the runtime named", got)
|
||||
}
|
||||
// The call's id is the runtime's too (ADR 0597 §7): read from `_meta`, never from the arguments, and none when it
|
||||
// is not one token.
|
||||
for _, c := range []struct {
|
||||
args, meta map[string]any
|
||||
want string
|
||||
}{
|
||||
{map[string]any{}, map[string]any{MetaCaller: "shanks.node-tools", MetaCall: "c-7Hq2"}, "c-7Hq2"},
|
||||
{map[string]any{}, map[string]any{MetaCaller: "shanks.node-tools"}, ""},
|
||||
{map[string]any{MetaCall: "c-forged"}, map[string]any{}, ""},
|
||||
{map[string]any{}, map[string]any{MetaCall: "a b"}, ""},
|
||||
{map[string]any{}, map[string]any{MetaCall: 7}, ""},
|
||||
} {
|
||||
if _, call := callAnswering(t, map[string]any{"name": "who", "arguments": c.args, "_meta": c.meta}); call != c.want {
|
||||
t.Errorf("args %v, meta %v: the tool read call %q, want %q", c.args, c.meta, call, c.want)
|
||||
}
|
||||
}
|
||||
if CallIDOf(map[string]any{"x": 1}) != "" || CallIDOf(nil) != "" {
|
||||
t.Error("arguments no call was handed have a call id")
|
||||
}
|
||||
if MetaCall != buscalls.MetaCall {
|
||||
t.Errorf("the call id's place is %q here and %q in buscalls", MetaCall, buscalls.MetaCall)
|
||||
}
|
||||
}
|
||||
|
||||
// A caller named in the arguments is not the caller: an argument is the caller's to write.
|
||||
@@ -123,55 +91,6 @@ func TestACallerInTheArgumentsIsNeverTheCaller(t *testing.T) {
|
||||
if _, err := ParseCallers([]byte(`{"callers":["x"],"more":1}`)); err == nil {
|
||||
t.Error("a list with a field it does not know was read")
|
||||
}
|
||||
// A holder's list per verb (ADR 0627 §2): each verb's entries as above, plus every proved caller (AnyCaller) or
|
||||
// every operator's session (AnyPerson); an unproven caller never, a verb the holder does not serve refused.
|
||||
verbs, err := ParseVerbCallers(`{"publish":["mesh-build-go"],"versions":["*"],"release":["operator","sweep"]}`,
|
||||
[]string{"publish", "versions", "release", "remove"}, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, c := range []struct {
|
||||
verb, caller string
|
||||
want bool
|
||||
}{{"publish", "ace.mesh-build-go", true}, {"publish", "ace.mesh-build-node", false}, {"versions", "g14.anything", true},
|
||||
{"versions", Unproven, false}, {"release", "person.jo", true}, {"release", "ace.operator", false},
|
||||
{"release", "novox.sweep", true}, {"remove", "novox.sweep", false}} {
|
||||
if got := verbs.Allowed(c.verb, c.caller) == nil; got != c.want {
|
||||
t.Errorf("%s by %q allowed: %v, want %v", c.verb, c.caller, got, c.want)
|
||||
}
|
||||
}
|
||||
// A-295: an entry `seat:<seat>` allows the modules the part's holder function answers for the seat, and
|
||||
// nobody without holders.
|
||||
seats := VerbCallers{"again": {AnyPerson, "seat:node-tool-runner", "seat:mesh-healers"}}
|
||||
holders := func(seat string) []string {
|
||||
return map[string][]string{"node-tool-runner": {"node-tools"}, "mesh-healers": {"mesh-delivery-watchman"}}[seat]
|
||||
}
|
||||
for _, c := range []struct {
|
||||
caller string
|
||||
holders func(string) []string
|
||||
want bool
|
||||
}{{"ace.node-tools", holders, true}, {"g14.mesh-delivery-watchman", holders, true}, {"ace.mesh-build-go", holders, false},
|
||||
{"person.jo", holders, true}, {"ace.node-tools", nil, false}} {
|
||||
if got := seats.AllowedWith("again", c.caller, c.holders) == nil; got != c.want {
|
||||
t.Errorf("again by %q (holders %v) allowed: %v, want %v", c.caller, c.holders != nil, got, c.want)
|
||||
}
|
||||
}
|
||||
if _, err := ParseVerbCallers(`{"mirror":["x"]}`, []string{"publish"}, nil); err == nil {
|
||||
t.Error("a list naming a verb the holder does not serve was read")
|
||||
}
|
||||
if d, _ := ParseVerbCallers(" ", nil, VerbCallers{"publish": {"x"}}); len(d["publish"]) != 1 {
|
||||
t.Error("an empty value is not the holder's defaults")
|
||||
}
|
||||
// As mesh-config resolves the value: the object itself, or written as a string; null is the defaults.
|
||||
for _, raw := range []string{`{"publish":["a"]}`, `"{\"publish\":[\"a\"]}"`, `null`} {
|
||||
c, err := parseConfiguredVerbCallers(json.RawMessage(raw), []string{"publish"}, VerbCallers{"publish": {"a"}})
|
||||
if err != nil || c.Allowed("publish", "ace.a") != nil {
|
||||
t.Errorf("the config value %s read as %v, %v", raw, c, err)
|
||||
}
|
||||
}
|
||||
if ModuleOf("ace.mesh-build-go") != "mesh-build-go" {
|
||||
t.Error("ModuleOf does not read the module of a proved caller")
|
||||
}
|
||||
}
|
||||
|
||||
// A runtime that names no caller — one from before issue 365, or a call on the subject that names none — is
|
||||
@@ -200,44 +119,3 @@ func TestArgumentsOfNoCallHaveNoCaller(t *testing.T) {
|
||||
t.Fatalf("CallerOf(nil) answered %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A holder's caller list is held (issue 1966): the runtime is asked once, by a watch; a later call never asks again, so
|
||||
// an ask that timed out refuses no verb, and a change that cannot be read keeps the list last read.
|
||||
func TestAHeldCallerListIsAskedOnceAndSurvivesAnUnreadableChange(t *testing.T) {
|
||||
verbs, defaults := []string{"publish"}, VerbCallers{"publish": {"a"}}
|
||||
var asks int
|
||||
var push func(ProvisionChange) error
|
||||
fail := true
|
||||
read := holdVerbCallers(func(key string, h func(ProvisionChange) error) error {
|
||||
asks++
|
||||
if fail {
|
||||
return errors.New("nats: timeout")
|
||||
}
|
||||
push = h
|
||||
return h(ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`{"publish":["b"]}`)})
|
||||
}, "callers", verbs, defaults)
|
||||
if _, err := read(); err == nil {
|
||||
t.Fatal("a list never read was answered")
|
||||
}
|
||||
fail = false
|
||||
c, err := read()
|
||||
if err != nil || c.Allowed("publish", "ace.b") != nil || asks != 2 {
|
||||
t.Fatalf("after the ask answered: %v, %v, asked %d times", c, err, asks)
|
||||
}
|
||||
for i := 0; i < 3; i++ {
|
||||
if _, err := read(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if asks != 2 {
|
||||
t.Errorf("a held list was asked again: %d asks", asks)
|
||||
}
|
||||
_ = push(ProvisionChange{Kind: "config", Key: "callers", Value: json.RawMessage(`{"nothing":["x"]}`)})
|
||||
if c, err := read(); err != nil || c.Allowed("publish", "ace.b") != nil {
|
||||
t.Errorf("an unreadable change dropped the list last read: %v, %v", c, err)
|
||||
}
|
||||
_ = push(ProvisionChange{Kind: "config", Key: "callers", Value: json.RawMessage(`{"publish":["c"]}`)})
|
||||
if c, _ := read(); c.Allowed("publish", "ace.c") != nil || c.Allowed("publish", "ace.b") == nil {
|
||||
t.Errorf("a change was not held: %v", c)
|
||||
}
|
||||
}
|
||||
|
||||
-159
@@ -4,9 +4,7 @@ import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// Callers is a module's trusted list of who may call a verb of its (novox/hq ADR 0459 §2, ADR 0563 §8), as the mesh
|
||||
@@ -48,160 +46,3 @@ func (c Callers) Allows(caller string) bool {
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// VerbCallers is a holder's trusted list per verb (novox/hq ADR 0627 §2, design 51 *Who may call which verb*): for each
|
||||
// verb of the seats it holds, the callers that may call it, each entry read as Callers reads it. Two entries more stand
|
||||
// for a whole class: AnyCaller lets every proved caller call the verb (a read the mesh keeps open), AnyPerson every
|
||||
// operator's session the runtime proved as person.<name>. It names no seat and no verb: the holder gives its verbs and
|
||||
// its defaults, and keeps any rule of its own seat (who may never be on a list) beside it.
|
||||
type VerbCallers map[string][]string
|
||||
|
||||
// The entries of a VerbCallers list that stand for a class of callers.
|
||||
const (
|
||||
AnyCaller = "*"
|
||||
AnyPerson = "operator"
|
||||
)
|
||||
|
||||
// ParseVerbCallers reads a holder's config value of verb to callers; an empty value is defaults. A verb not among verbs
|
||||
// is refused, so a list that names a verb the holder does not serve is said at once.
|
||||
func ParseVerbCallers(raw string, verbs []string, defaults VerbCallers) (VerbCallers, error) {
|
||||
if strings.TrimSpace(raw) == "" {
|
||||
return defaults, nil
|
||||
}
|
||||
var c VerbCallers
|
||||
if err := json.Unmarshal([]byte(raw), &c); err != nil {
|
||||
return nil, fmt.Errorf("callers is not a JSON object of verb to callers: %w", err)
|
||||
}
|
||||
for verb := range c {
|
||||
if !slices.Contains(verbs, verb) {
|
||||
return nil, fmt.Errorf("callers names the verb %q, which the holder does not serve", verb)
|
||||
}
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// CallerSeatPrefix opens an entry naming a seat, `seat:<seat>`: every module holding it, so a list names no module (design 47
|
||||
// G A-210, A-295).
|
||||
const CallerSeatPrefix = "seat:"
|
||||
|
||||
// Allowed answers nil when caller, as the runtime proved it, may call verb; else the refusal naming both. An unproven
|
||||
// caller is never allowed, not even by AnyCaller. An entry `seat:<seat>` allows no one here (AllowedWith).
|
||||
func (c VerbCallers) Allowed(verb, caller string) error { return c.AllowedWith(verb, caller, nil) }
|
||||
|
||||
// AllowedWith is Allowed whose entries `seat:<seat>` allow a module caller whose module holders answers for that seat:
|
||||
// the part's holder function, which answers from what it last heard of the assigner's seat-holders view (design 47 G A-295),
|
||||
// so no call is made per check. A nil holders, or a seat it answers none for, allows no one by that entry.
|
||||
func (c VerbCallers) AllowedWith(verb, caller string, holders func(seat string) []string) error {
|
||||
if caller == "" || caller == Unproven {
|
||||
return fmt.Errorf("%s: refused: the request carries no caller the tool runner proved", verb)
|
||||
}
|
||||
person := strings.HasPrefix(caller, "person.")
|
||||
for _, entry := range c[verb] {
|
||||
if seat, ok := strings.CutPrefix(entry, CallerSeatPrefix); ok {
|
||||
if !person && holders != nil && slices.Contains(holders(seat), ModuleOf(caller)) {
|
||||
return nil
|
||||
}
|
||||
continue
|
||||
}
|
||||
switch entry {
|
||||
case AnyCaller:
|
||||
return nil
|
||||
case AnyPerson:
|
||||
if person {
|
||||
return nil
|
||||
}
|
||||
default:
|
||||
if (Callers{entry}).Allows(caller) {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("%s: refused for %s: not on the holder's caller list for %s", verb, caller, verb)
|
||||
}
|
||||
|
||||
// ModuleOf is the module of a proved module caller, which the runtime names <machine>.<module>.
|
||||
func ModuleOf(proved string) string {
|
||||
if _, module, ok := strings.Cut(proved, "."); ok {
|
||||
return module
|
||||
}
|
||||
return proved
|
||||
}
|
||||
|
||||
// ConfiguredVerbCallers is the holder's list as mesh-config resolved its config value key for this machine (Config):
|
||||
// a JSON object of verb to callers, or that object written as a string; an unset or empty value is defaults. A value
|
||||
// that cannot be read is an error, never the defaults, so a broken list is said, not widened.
|
||||
func ConfiguredVerbCallers(key string, verbs []string, defaults VerbCallers) (VerbCallers, error) {
|
||||
raw, err := Config(key)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the config value %s cannot be read: %w", key, err)
|
||||
}
|
||||
return parseConfiguredVerbCallers(raw, verbs, defaults)
|
||||
}
|
||||
|
||||
// HoldVerbCallers is ConfiguredVerbCallers held: the returned reader asks the runtime once, by a watch of the config
|
||||
// value key (WatchConfig), and from then on answers the list it holds, which each change of the value replaces. A call
|
||||
// never asks the runtime again, so a runtime that is slow or away for a moment (a timed-out ask) refuses no verb; a
|
||||
// change that cannot be read keeps the list last read and is said on the bundle's standard error, so a broken list is
|
||||
// said and never widened. Only a reader that has not yet read any value answers an error, and its next call asks again.
|
||||
func HoldVerbCallers(key string, verbs []string, defaults VerbCallers) func() (VerbCallers, error) {
|
||||
return holdVerbCallers(WatchConfig, key, verbs, defaults)
|
||||
}
|
||||
|
||||
func holdVerbCallers(watch func(string, func(ProvisionChange) error) error, key string, verbs []string, defaults VerbCallers) func() (VerbCallers, error) {
|
||||
var start, mu sync.Mutex
|
||||
var held VerbCallers
|
||||
var bad error
|
||||
handle := func(c ProvisionChange) error {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if c.Refused != "" {
|
||||
bad = fmt.Errorf("the config value %s is not answered: %s", key, c.Refused)
|
||||
} else if l, err := parseConfiguredVerbCallers(c.Value, verbs, defaults); err != nil {
|
||||
bad = fmt.Errorf("the config value %s cannot be read: %w", key, err)
|
||||
} else {
|
||||
held, bad = l, nil
|
||||
return nil
|
||||
}
|
||||
if held != nil {
|
||||
sayf("%v: the list last read stays", bad)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
read := func() (VerbCallers, error) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if held != nil {
|
||||
return held, nil
|
||||
}
|
||||
if bad != nil {
|
||||
return nil, bad
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
return func() (VerbCallers, error) {
|
||||
if l, err := read(); l != nil || err != nil {
|
||||
return l, err
|
||||
}
|
||||
start.Lock()
|
||||
defer start.Unlock()
|
||||
if l, err := read(); l != nil || err != nil {
|
||||
return l, err
|
||||
}
|
||||
if err := watch(key, handle); err != nil {
|
||||
return nil, fmt.Errorf("the config value %s cannot be read: %w", key, err)
|
||||
}
|
||||
l, err := read()
|
||||
if l == nil && err == nil {
|
||||
err = fmt.Errorf("the config value %s cannot be read: the runtime handed over no value", key)
|
||||
}
|
||||
return l, err
|
||||
}
|
||||
}
|
||||
|
||||
func parseConfiguredVerbCallers(raw json.RawMessage, verbs []string, defaults VerbCallers) (VerbCallers, error) {
|
||||
var text string
|
||||
if err := json.Unmarshal(raw, &text); err == nil || len(bytes.TrimSpace(raw)) == 0 || string(bytes.TrimSpace(raw)) == "null" {
|
||||
return ParseVerbCallers(text, verbs, defaults)
|
||||
}
|
||||
return ParseVerbCallers(string(raw), verbs, defaults)
|
||||
}
|
||||
|
||||
@@ -1,87 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"slices"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The call a bundle reads its tool runner's capability words with, and the words of the families that sit beside
|
||||
// `mesh/publish` and `mesh/subscribe` (design 47 G §576, §622). The words of the bus-state families are in buscalls.
|
||||
const (
|
||||
// CapabilitiesCall is answered with {words}: the capability words the tool runner states.
|
||||
CapabilitiesCall = "mesh/capabilities"
|
||||
// CapPutSessions is the put and get sessions' word (§457, §518), CapFindingCalls the acknowledged publish and the
|
||||
// subscription for new messages the finding writer uses (§559). CapSeatSettleDelay (seat.go) is the third.
|
||||
CapPutSessions = "put-sessions"
|
||||
CapFindingCalls = "finding-calls"
|
||||
)
|
||||
|
||||
// capabilityWords are the words the tool runner states, read once at the first call that needs one and kept for as long
|
||||
// as the bundle runs: a bundle lives no longer than the tool runner that started it (design 47 G §576).
|
||||
type capabilityWords struct {
|
||||
mu sync.Mutex
|
||||
read bool
|
||||
words []string
|
||||
}
|
||||
|
||||
var capabilities capabilityWords
|
||||
|
||||
// get answers the words, asking once with ask. A tool runner of before G answers `mesh/capabilities` as an unknown call,
|
||||
// which reads as no word stated; any other failure is returned and asked again at the next call.
|
||||
func (c *capabilityWords) get(ask func() (json.RawMessage, error)) ([]string, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if c.read {
|
||||
return c.words, nil
|
||||
}
|
||||
raw, err := ask()
|
||||
switch {
|
||||
case errors.Is(err, errors.ErrUnsupported):
|
||||
c.words = nil
|
||||
case err != nil:
|
||||
return nil, fmt.Errorf("reading the tool runner's capability words: %w", err)
|
||||
default:
|
||||
var a struct {
|
||||
Words []string `json:"words"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &a); err != nil {
|
||||
return nil, fmt.Errorf("the tool runner's capability words: %w", err)
|
||||
}
|
||||
c.words = a.Words
|
||||
}
|
||||
c.read = true
|
||||
return c.words, nil
|
||||
}
|
||||
|
||||
// require is nil when word is among the stated ones, else an error naming the word and the node (design 47 G §488, §576).
|
||||
func (c *capabilityWords) require(word string, ask func() (json.RawMessage, error)) error {
|
||||
words, err := c.get(ask)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if slices.Contains(words, word) {
|
||||
return nil
|
||||
}
|
||||
node := os.Getenv("MESH_NODE")
|
||||
if node == "" {
|
||||
node = "unknown"
|
||||
}
|
||||
return fmt.Errorf("the tool runner of node %s does not state the capability word %q: a call of its family is refused until it does", node, word)
|
||||
}
|
||||
|
||||
// RequireCapability fails at once, naming the word and the node, when the tool runner this bundle runs under does not
|
||||
// state word; it reads the words once (design 47 G §488, §576).
|
||||
func RequireCapability(word string) error {
|
||||
return capabilities.require(word, func() (json.RawMessage, error) {
|
||||
c := awaitChannel()
|
||||
if c == nil {
|
||||
return nil, errors.New("a capability is read through the runtime that launched the bundle, and Serve has not run")
|
||||
}
|
||||
return c.ask(CapabilitiesCall, map[string]any{}, 30*time.Second)
|
||||
})
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Design 47 G §576: the words are read once, an unknown-call answer reads as none, and a word not stated fails naming it
|
||||
// and the node (from MESH_NODE).
|
||||
func subACapabilityWordIsReadOnceAndNamedWhenNotStated(t *testing.T) {
|
||||
t.Setenv("MESH_NODE", "ace")
|
||||
asked := 0
|
||||
var c capabilityWords
|
||||
ask := func() (json.RawMessage, error) { asked++; return json.RawMessage(`{"words":["put-sessions"]}`), nil }
|
||||
if err := c.require(CapPutSessions, ask); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err := c.require(CapFindingCalls, ask)
|
||||
if err == nil || !strings.Contains(err.Error(), CapFindingCalls) || !strings.Contains(err.Error(), "ace") || asked != 1 {
|
||||
t.Errorf("a word not stated: %v after %d asks", err, asked)
|
||||
}
|
||||
var old capabilityWords
|
||||
unknown := func() (json.RawMessage, error) {
|
||||
asked++
|
||||
return nil, &refused{message: "no mesh/capabilities", code: CodeNoMethod}
|
||||
}
|
||||
if err := old.require(CapPutSessions, unknown); err == nil || !strings.Contains(err.Error(), CapPutSessions) {
|
||||
t.Errorf("a tool runner of before G stated %v", err)
|
||||
}
|
||||
var failing capabilityWords
|
||||
if err := failing.require(CapPutSessions, func() (json.RawMessage, error) { return nil, errors.New("pipe closed") }); err == nil ||
|
||||
!strings.Contains(err.Error(), "pipe closed") || failing.read {
|
||||
t.Errorf("a failed read: %v, kept %v", err, failing.read)
|
||||
}
|
||||
}
|
||||
@@ -54,7 +54,7 @@ func indexOf(out []map[string]any, id string) int {
|
||||
// otherwise — and the controller declares the verbs it answers as tools. Its service bundle is run
|
||||
// by its own process; launched a second time by the runtime it would be a second controller
|
||||
// pretending to be an MCP server.
|
||||
func subABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
|
||||
func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
|
||||
m := aServiceModule(t)
|
||||
if len(m.Bundles) != 1 {
|
||||
t.Fatalf("the service's bundle was not kept: %+v", m.Bundles)
|
||||
@@ -75,10 +75,3 @@ func subABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
|
||||
t.Fatalf("a tools bundle nothing runs is no longer served: %v", loads)
|
||||
}
|
||||
}
|
||||
|
||||
// TestABundleItsOwnProcessRunsIsNotServedByTheRuntime runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
|
||||
func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
|
||||
t.Run("ABundleItsOwnProcessRunsIsNotServedByTheRuntime", subABundleItsOwnProcessRunsIsNotServedByTheRuntime)
|
||||
t.Run("TheBusNamesOfAnEntryAreOneRule", subTheBusNamesOfAnEntryAreOneRule)
|
||||
t.Run("AStarEmitterHearsEachDeclaringModulesUnrestrictedStream", subAStarEmitterHearsEachDeclaringModulesUnrestrictedStream)
|
||||
}
|
||||
|
||||
@@ -1,12 +1,8 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -22,7 +18,6 @@ import (
|
||||
// design 28 task 5.4, not converted.
|
||||
func TestNoCatalogueManifestNamesAmqp(t *testing.T) {
|
||||
modules, err := filepath.Glob(filepath.Join(beside.Catalogue(t), "*", "module.json"))
|
||||
modules0 := modules
|
||||
if err != nil || len(modules) == 0 {
|
||||
t.Fatalf("no manifests in the catalogue, so this proved nothing: %v", err)
|
||||
}
|
||||
@@ -36,42 +31,4 @@ func TestNoCatalogueManifestNamesAmqp(t *testing.T) {
|
||||
filepath.Base(filepath.Dir(path)))
|
||||
}
|
||||
}
|
||||
// Nor does this package's code name any module the catalogue holds (ADR 0601 §4): what a module is, its manifest
|
||||
// says. The names are read from the catalogue's module directories, never from a list here; every string literal of
|
||||
// the package's non-test files is read, and a module's name is refused unless it is also a seat's name or one of the
|
||||
// words below, which name something else.
|
||||
t.Run("the package names no module", func(t *testing.T) {
|
||||
modules := map[string]bool{}
|
||||
for _, path := range modules0 {
|
||||
modules[filepath.Base(filepath.Dir(path))] = true
|
||||
}
|
||||
// Words that are a module's name and name something else here: a language, an environment's rendering, a package.
|
||||
elsewhere := map[string]bool{"go": true, "systemd": true, "nftables": true}
|
||||
fset := token.NewFileSet()
|
||||
files, _ := filepath.Glob("*.go")
|
||||
for _, f := range files {
|
||||
if strings.HasSuffix(f, "_test.go") {
|
||||
continue
|
||||
}
|
||||
parsed, err := parser.ParseFile(fset, f, nil, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ast.Inspect(parsed, func(n ast.Node) bool {
|
||||
lit, ok := n.(*ast.BasicLit)
|
||||
if !ok || lit.Kind != token.STRING {
|
||||
return true
|
||||
}
|
||||
word, err := strconv.Unquote(lit.Value)
|
||||
if err != nil || !modules[word] || elsewhere[word] {
|
||||
return true
|
||||
}
|
||||
if _, seat := SeatNamed(word); seat {
|
||||
return true
|
||||
}
|
||||
t.Errorf("%s names the module %s", fset.Position(lit.Pos()), word)
|
||||
return true
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
package catalogue
|
||||
|
||||
// ArtifactSweepSeat is the seat artifact-store-tools holds. Not the seat mesh-artifact-store, the store itself held by
|
||||
// distribution, and not the provision artifact-store it delivers.
|
||||
const ArtifactSweepSeat = "mesh-artifact-sweep"
|
||||
|
||||
// artifactSweepVerbs are the seat's verbs, with the controller's arguments so a caller changes only the seat it asks.
|
||||
func artifactSweepVerbs() []Verb {
|
||||
return []Verb{
|
||||
{Name: "artifacts", Description: "Every artifact the mesh made that lives in its artifact store, each with its " +
|
||||
"repository, digest, kind (image or archive) and state: kept, and why (a want names it, or one of the five " +
|
||||
"newest builds of its module in the builder's log), eligible (made, kept for no reason, not yet let go), or " +
|
||||
"unrecorded (in the store and named by nothing the mesh made: never removed). Reads only.",
|
||||
Input: schema(map[string]string{
|
||||
"repository": "only this repository's references, as <module>/<artifact> (the counts stay whole)",
|
||||
"collected": "\"true\": list the collected references too (counted always)",
|
||||
}, nil, "collected")},
|
||||
{Name: "held", Description: "Whether the artifact store holds each reference given, asked with one HEAD each: " +
|
||||
"held, gone (the store answered 404) or unknown with why. The builder asks it before it reuses a done key. " +
|
||||
"Reads only.",
|
||||
Input: listed(schema(map[string]string{
|
||||
"references": "the artifacts' references, artifact-store://<module>/<artifact>/blobs/sha256:<hex>: a list, " +
|
||||
"or one text separated by commas",
|
||||
}, []string{"references"}), "references")},
|
||||
{Name: "collect", Description: "The artifact store's sweep, now: what is eligible is let go of. Without confirm a " +
|
||||
"dry run: what would be let go, nothing held or deleted. With confirm and why: every kept archive held first, " +
|
||||
"then each eligible artifact let go of, oldest first, an index with the platform manifests no kept index " +
|
||||
"names, each deletion recorded in the hand-act log. Bounded by most (500 by default, at most 5000) and 45 " +
|
||||
"seconds; what is left is said. Never touches a digest the mesh did not make.",
|
||||
Input: schema(map[string]string{
|
||||
"confirm": "\"true\": let go of what is eligible (needs why); without it nothing is done",
|
||||
"why": "with confirm: why, required and recorded",
|
||||
"most": "let go of at most this many (500 by default, at most 5000)",
|
||||
}, nil, "confirm")},
|
||||
{Name: "mirrors", Description: "The bases the mesh copied into its artifact store, each kept (a kept build stood on " +
|
||||
"it), eligible (only builds no longer kept stood on it) or unrecorded (no build the builder's log holds stood on " +
|
||||
"it: never removed), by what the builder's log says each build stood on. Records nothing by hand.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
{Name: "images", Description: "Every container image one machine's want and have name, each with the resources " +
|
||||
"naming it: what the machine keeps when it prunes its images. Reads only.",
|
||||
Input: schema(map[string]string{"node": "the machine's name"}, []string{"node"})},
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,58 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ArtifactStoreScheme marks a reference to something in the mesh's artifact store, kept without
|
||||
// the store's address. How a reference is recorded, read and routed (the registry seat's form beside this one) is the
|
||||
// registry clients' (mesh-delivery's contracts, `containerimageregistryclient/reference`; ADR 0634 §4).
|
||||
// the store's address.
|
||||
const ArtifactStoreScheme = ArtifactStoreProvision + "://"
|
||||
|
||||
// Recorded is a reference as the mesh records it: the artifact store's address, if the builder wrote
|
||||
// one, taken off.
|
||||
//
|
||||
// For a reference the builder announced — one it pushed to the store — which is the only kind
|
||||
// this is called on. An image the builder named `<host>/<path>@sha256:…` is kept as its path; an
|
||||
// archive it named `http://<host>/v2/<path>/blobs/<digest>` likewise. A reference with no address
|
||||
// in it — an image id from a genesis build that had nowhere to publish, a package version — is
|
||||
// what it was.
|
||||
func Recorded(reference string) string {
|
||||
if strings.HasPrefix(reference, ArtifactStoreScheme) {
|
||||
return reference
|
||||
}
|
||||
if rest, isURL := strings.CutPrefix(reference, "http://"); isURL {
|
||||
if _, path, ok := strings.Cut(rest, "/v2/"); ok && strings.Contains(path, "/blobs/") {
|
||||
return ArtifactStoreScheme + path
|
||||
}
|
||||
return reference
|
||||
}
|
||||
host, path, ok := strings.Cut(reference, "/")
|
||||
if !ok || !isRegistryHost(host) || !strings.Contains(path, "@sha256:") {
|
||||
return reference
|
||||
}
|
||||
return ArtifactStoreScheme + path
|
||||
}
|
||||
|
||||
// isRegistryHost is the runtime's own rule for reading the first component of a reference as a
|
||||
// registry rather than as a namespace: it has a dot or a port in it, or it is localhost.
|
||||
func isRegistryHost(component string) bool {
|
||||
return component == "localhost" || strings.ContainsAny(component, ".:")
|
||||
}
|
||||
|
||||
// InArtifactStore reports whether a reference is a kept one, and what it names there.
|
||||
func InArtifactStore(reference string) (path string, kept bool) {
|
||||
return strings.CutPrefix(reference, ArtifactStoreScheme)
|
||||
}
|
||||
|
||||
// Routed is a kept reference as a machine fetches it, through the artifact store at `address`
|
||||
// (host:port). A reference that is not a kept one is what it was.
|
||||
func Routed(reference, address string) string {
|
||||
path, kept := InArtifactStore(reference)
|
||||
if !kept {
|
||||
return reference
|
||||
}
|
||||
if strings.Contains(path, "/blobs/") {
|
||||
return "http://" + address + "/v2/" + path
|
||||
}
|
||||
return address + "/" + path
|
||||
}
|
||||
|
||||
@@ -1,6 +1,29 @@
|
||||
package catalogue
|
||||
|
||||
import "strings"
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A build is recorded by what it is; where it is pushed is composed where it is used (novox/hq
|
||||
// 04-ISSUES/102).
|
||||
|
||||
// digest is an artifact digest the tests build references with.
|
||||
var digest = "sha256:" + strings.Repeat("d", 64)
|
||||
|
||||
func TestAReferenceIsRecordedWithoutTheStoresAddress(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"anchor.internal:5100/gitea/server@" + digest: ArtifactStoreScheme + "gitea/server@" + digest,
|
||||
"localhost:5000/gitea/server@" + digest: ArtifactStoreScheme + "gitea/server@" + digest,
|
||||
"http://anchor.internal:5100/v2/gitea/config/blobs/" + digest: ArtifactStoreScheme + "gitea/config/blobs/" + digest,
|
||||
ArtifactStoreScheme + "gitea/server@" + digest: ArtifactStoreScheme + "gitea/server@" + digest,
|
||||
digest: digest,
|
||||
"@novox/sdk@1.2.3": "@novox/sdk@1.2.3",
|
||||
"gitea/gitea@" + digest: "gitea/gitea@" + digest,
|
||||
"https://registry.example/v2/gitea/config/blobs/" + digest: "https://registry.example/v2/gitea/config/blobs/" + digest,
|
||||
}
|
||||
for announced, want := range cases {
|
||||
if got := Recorded(announced); got != want {
|
||||
t.Errorf("Recorded(%q) = %q, want %q", announced, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17,11 +17,7 @@ func TestRenamingASeatDidNotRenameTheInterfaceItDelivers(t *testing.T) {
|
||||
for _, pair := range []struct{ seat, delivers string }{
|
||||
{"git", "git"},
|
||||
{"npm-package-registry", "npm-package-registry"},
|
||||
// The store's seats deliver nothing since the provision artifact-store retired (novox/hq ADR 0632 §2): the
|
||||
// store's address is its holder's listen, read through the seat.
|
||||
{"mesh-artifact-store", ""},
|
||||
{"container-image-registry", ""},
|
||||
{"oci-artifact-registry", ""},
|
||||
{"mesh-artifact-store", "artifact-store"},
|
||||
{"mesh-store", "postgres-database"},
|
||||
{"mesh-broker", "mesh-bus"},
|
||||
} {
|
||||
|
||||
+3
-19
@@ -117,9 +117,9 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
|
||||
// it reached it by, and a manifest carrying that address names an installation —
|
||||
// registration refused node-tools for exactly this on 2026-10-02. The build record
|
||||
// already keeps the store-relative form; the resolved manifest keeps the same, and
|
||||
// composition routes it through the store a machine reaches (the registry clients' Routed).
|
||||
// composition routes it through the store a machine reaches (Routed).
|
||||
out.Bundles = append(out.Bundles, Bundle{
|
||||
Name: a.Name, Source: kept(made.Reference), Digest: made.Digest,
|
||||
Name: a.Name, Source: Recorded(made.Reference), Digest: made.Digest,
|
||||
Language: a.Language, Entrypoints: append([]string(nil), a.Entrypoints...),
|
||||
Loads: loads, Env: copyWords(a.Env), Launchers: copyWords(made.Launchers),
|
||||
Binary: BinaryOf(a), Bus: a.Bus,
|
||||
@@ -171,7 +171,7 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
|
||||
// how they were made — one packed as it stood, the other compiled first — and a
|
||||
// machine has no reason to care which.
|
||||
// Kept, not routed, for the reason the bundles above are (ADR 0155).
|
||||
filled["source"] = kept(artifact.Reference)
|
||||
filled["source"] = Recorded(artifact.Reference)
|
||||
filled["digest"] = artifact.Digest
|
||||
// **And `${version}`, so a resource can name a place that is this build's alone**
|
||||
// (novox/hq ADR 0141, 04-ISSUES/142). A component is unpacked into a directory named
|
||||
@@ -363,19 +363,3 @@ func BundleLoads(m Manifest, a Artifact, run map[string]bool) []string {
|
||||
}
|
||||
return loads
|
||||
}
|
||||
|
||||
// kept is a bundle's or an archive's reference without the artifact store's address, if the builder wrote one (ADR 0155:
|
||||
// a resolved manifest names no installation). Only the address is read here: an `http://<host>/v2/<path>/blobs/<digest>`
|
||||
// becomes `artifact-store://<path>/blobs/<digest>`; a kept reference, or one in a registry seat's form (whose first part,
|
||||
// a seat's name, has no dot or port), is what it was. How the registry seats' forms are read is the registry clients'.
|
||||
func kept(reference string) string {
|
||||
if strings.HasPrefix(reference, ArtifactStoreScheme) {
|
||||
return reference
|
||||
}
|
||||
if rest, isURL := strings.CutPrefix(reference, "http://"); isURL {
|
||||
if _, path, ok := strings.Cut(rest, "/v2/"); ok && strings.Contains(path, "/blobs/") {
|
||||
return ArtifactStoreScheme + path
|
||||
}
|
||||
}
|
||||
return reference
|
||||
}
|
||||
|
||||
@@ -43,4 +43,49 @@ func TestTheCataloguesBusIsBackedUpBySnapshot(t *testing.T) {
|
||||
if !found {
|
||||
t.Fatal("the bus declares no item for its streams")
|
||||
}
|
||||
// 774/S3 (ADR 0594 §7): the one list of what is the bus server names the bus's container and every resource that
|
||||
// restarts it, read from the catalogue's nats manifest at mesh-catalog 0b2d393b; a module not providing the bus
|
||||
// has none.
|
||||
t.Run("the bus server's list", func(t *testing.T) {
|
||||
raw, err := os.ReadFile("testdata/catalogue-0b2d393b-nats.module.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
nats, err := decodeForTest(raw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
server := BusServerOf(nats)
|
||||
var containers int
|
||||
for _, r := range nats.Resources {
|
||||
if r["type"] != "container" {
|
||||
continue
|
||||
}
|
||||
containers++
|
||||
restarts, _ := r["restart-on"].([]any)
|
||||
if !server[r["id"].(string)] || len(restarts) == 0 {
|
||||
t.Errorf("the bus's container %v (restarted on %v) is not all in the server's list %v", r["id"], restarts, server)
|
||||
}
|
||||
for _, id := range restarts {
|
||||
if !server[id.(string)] {
|
||||
t.Errorf("%v restarts the bus server and is not in its list %v", id, server)
|
||||
}
|
||||
}
|
||||
}
|
||||
if containers != 1 || len(server) != len(BusServer) {
|
||||
t.Errorf("the bus has %d containers and its server is %v of %v", containers, server, BusServer)
|
||||
}
|
||||
xterm, err := os.ReadFile("testdata/catalogue-835bf22b-xterm.module.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
other, err := decodeForTest(xterm)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
other.Resources = append(other.Resources, map[string]any{"id": "server", "type": "container"})
|
||||
if got := BusServerOf(other); len(got) != 0 {
|
||||
t.Errorf("a module not providing the bus has a bus server %v", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -1,88 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The names of what a module keeps on the bus, as one rule (design 25 § 12.1; novox/hq ADR 0666 §16): the bus module
|
||||
// creates them from a module's entries and the grants part grants them, and both take the names from here.
|
||||
|
||||
// BusNameSeparator joins a module's name and an entry's local name into the name the mesh gives the entry on the bus.
|
||||
const BusNameSeparator = "_"
|
||||
|
||||
// SentObjectsLocal is the local name of a module's store of sent objects (the bodies of a reply or an event over the
|
||||
// per-message limit): the bus module plans `<module>_sent-objects` for every module, and no entry may take the name.
|
||||
const SentObjectsLocal = "sent-objects"
|
||||
|
||||
// ObjectsSuffix ends the local name of the object store beside a state: `<state>-objects`.
|
||||
const ObjectsSuffix = "-objects"
|
||||
|
||||
// safeToken is a name that is one plain subject token.
|
||||
var safeToken = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
|
||||
|
||||
// BusName is `<module>_<local>`, the name the mesh gives an entry on the bus.
|
||||
func BusName(module, local string) string { return module + BusNameSeparator + local }
|
||||
|
||||
// StateObjectsName is the local name of the object store that holds the large values of the state `state`.
|
||||
func StateObjectsName(state string) string { return state + ObjectsSuffix }
|
||||
|
||||
// SentObjectsName is the bus name of a module's store of sent objects.
|
||||
func SentObjectsName(module string) string { return BusName(module, SentObjectsLocal) }
|
||||
|
||||
// BusStateStreams are the streams one bus-state entry of module is kept in on the bus, by the mesh's names
|
||||
// `<module>_<local>`: a stream's own; a key-value bucket's KV_ stream with the OBJ_ stream of its object store
|
||||
// `<local>-objects`; an object store's OBJ_ stream; a ledger's history stream, its snapshot stream, and its state and
|
||||
// index buckets' KV_ streams. A name that is not one plain subject token names nothing.
|
||||
func BusStateStreams(module string, e BusStateEntry) []string {
|
||||
base := BusName(module, e.Name)
|
||||
if !safeToken.MatchString(base) || strings.Contains(e.Name, BusNameSeparator) || strings.Contains(module, BusNameSeparator) {
|
||||
return nil
|
||||
}
|
||||
var out []string
|
||||
switch e.Storage {
|
||||
case StorageKeyValue:
|
||||
out = []string{"KV_" + base}
|
||||
if e.Template == TemplateState {
|
||||
out = append(out, "OBJ_"+BusName(module, StateObjectsName(e.Name)))
|
||||
}
|
||||
case StorageObjectStore:
|
||||
out = []string{"OBJ_" + base}
|
||||
case StorageLedger:
|
||||
out = []string{base, base + "_snapshot", "KV_" + base + "_state"}
|
||||
for _, t := range e.Types {
|
||||
for _, ix := range t.Indexes {
|
||||
if safeToken.MatchString(ix.Name) {
|
||||
out = append(out, "KV_"+base+"_index-"+ix.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
default:
|
||||
out = []string{base}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return slices.Compact(out)
|
||||
}
|
||||
|
||||
// ObjectNameClashes are the local names of a module's bus-state entries that meet a name the mesh makes itself: the
|
||||
// name of a state's object store (`<state>-objects` beside the state `<state>`) and the store of sent objects. The
|
||||
// indexer's judge refuses each.
|
||||
func ObjectNameClashes(entries []BusStateEntry) []string {
|
||||
states := map[string]bool{}
|
||||
for _, e := range entries {
|
||||
if e.Storage == StorageKeyValue && e.Template == TemplateState {
|
||||
states[e.Name] = true
|
||||
}
|
||||
}
|
||||
var out []string
|
||||
for _, e := range entries {
|
||||
stem, ok := strings.CutSuffix(e.Name, ObjectsSuffix)
|
||||
if e.Name == SentObjectsLocal || (ok && states[stem]) {
|
||||
out = append(out, e.Name)
|
||||
}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return slices.Compact(out)
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// ADR 0666 §16: the bus names of an entry are one rule. A state entry answers its bucket's stream and the stream of
|
||||
// its object store; a local name that meets a name the mesh makes is a clash for the judge.
|
||||
func subTheBusNamesOfAnEntryAreOneRule(t *testing.T) {
|
||||
state := BusStateEntry{Name: "views", Storage: StorageKeyValue, Template: TemplateState}
|
||||
rows := []struct {
|
||||
name string
|
||||
entry BusStateEntry
|
||||
streams []string
|
||||
}{
|
||||
{"a state: its bucket and its object store", state, []string{"KV_m_views", "OBJ_m_views-objects"}},
|
||||
{"a key-value that is not a state template: its bucket only", BusStateEntry{Name: "x", Storage: StorageKeyValue}, []string{"KV_m_x"}},
|
||||
{"an object store", BusStateEntry{Name: "blobs", Storage: StorageObjectStore, Template: TemplateObjects}, []string{"OBJ_m_blobs"}},
|
||||
{"a stream", BusStateEntry{Name: "log", Storage: StorageStream, Template: TemplateLog}, []string{"m_log"}},
|
||||
{"a ledger", BusStateEntry{Name: "l", Storage: StorageLedger, Template: TemplateLedger}, []string{"KV_m_l_state", "m_l", "m_l_snapshot"}},
|
||||
{"a name that is not one token", BusStateEntry{Name: "a.b", Storage: StorageStream}, nil},
|
||||
{"a local name holding the separator", BusStateEntry{Name: "a_b", Storage: StorageStream}, nil},
|
||||
}
|
||||
for _, r := range rows {
|
||||
if got := BusStateStreams("m", r.entry); !slices.Equal(got, r.streams) {
|
||||
t.Errorf("%s: streams %v, want %v", r.name, got, r.streams)
|
||||
}
|
||||
}
|
||||
if got := SentObjectsName("m"); got != "m_sent-objects" {
|
||||
t.Errorf("sent store %q", got)
|
||||
}
|
||||
clash := []BusStateEntry{state, {Name: "views-objects", Storage: StorageObjectStore}, {Name: "other-objects", Storage: StorageObjectStore},
|
||||
{Name: SentObjectsLocal, Storage: StorageObjectStore}}
|
||||
if got := ObjectNameClashes(clash); !slices.Equal(got, []string{"sent-objects", "views-objects"}) {
|
||||
t.Errorf("clashes %v", got)
|
||||
}
|
||||
}
|
||||
@@ -1,232 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Bus state is what a module keeps on the bus, asked for as a provision of the bus module in `requires` (novox/hq ADR
|
||||
// 0595 §1, design 32 § 4): an entry of `requires` in its typed long form, the local name, the storage type, the template
|
||||
// and the template's options. This is the contract only (ADR 0578 §7, ADR 0593 §1): the entry's shape, decoded
|
||||
// leniently. Every rule that judges an entry (a template known, an option within its bound, a fixed setting not given)
|
||||
// is the indexer's judge's.
|
||||
|
||||
// The storage types (design 25 § 12.1): the stream is the base; the key-value bucket, the object store and the ledger
|
||||
// are built on it, each with calls of its own (ADR 0595 §2).
|
||||
const (
|
||||
StorageStream = "stream"
|
||||
StorageKeyValue = "key-value"
|
||||
StorageObjectStore = "object-store"
|
||||
StorageLedger = "ledger"
|
||||
)
|
||||
|
||||
// The six templates, each a storage type with a predefined configuration (ADR 0595 §2, design 25 § 12.1): state on
|
||||
// key-value, events, work queue and log on stream, objects on object store, ledger on ledger. A bare stream names
|
||||
// none and gives its own Configuration.
|
||||
const (
|
||||
TemplateState = "state"
|
||||
TemplateEvents = "events"
|
||||
TemplateWorkQueue = "work-queue"
|
||||
TemplateLog = "log"
|
||||
TemplateObjects = "objects"
|
||||
TemplateLedger = "ledger"
|
||||
)
|
||||
|
||||
// The option names, a template's settings a module may change (design 25 § 12.1's table), written as the old sections
|
||||
// wrote them, so a module that moves carries its old settings as options unchanged (ADR 0595 §11). Which option a
|
||||
// template takes, and its bound, is the judge's.
|
||||
const (
|
||||
// OptionPastValues is how many past values a state's key keeps (the old `state`'s `history`).
|
||||
OptionPastValues = "history"
|
||||
// OptionTimeToLive is how long a state's value lives, in seconds.
|
||||
OptionTimeToLive = "ttl-seconds"
|
||||
// OptionLifetimePerKey is whether a state's key may be given a lifetime of its own.
|
||||
OptionLifetimePerKey = "key-ttl"
|
||||
// OptionPerMachine is whether a state is kept one bucket per machine.
|
||||
OptionPerMachine = "per-machine"
|
||||
// OptionAgeDays is how long a stream keeps a message, in days.
|
||||
OptionAgeDays = "max-age-days"
|
||||
// OptionCapMiB is the cap of a stream, a bucket or an object store, in MiB.
|
||||
OptionCapMiB = "max-mib"
|
||||
// OptionObjectsCapMiB is the cap of a state's objects store `<state>-objects`, in MiB, where the state needs
|
||||
// another than the one the bus module plans (design 47 G §513, §547).
|
||||
OptionObjectsCapMiB = "objects-max-mib"
|
||||
// OptionGrant names the trusted config value granting another module a part of it (ADR 0363 §4, ADR 0592 §9).
|
||||
OptionGrant = "grant"
|
||||
// OptionDeliveries is how many times a work queue's message is delivered before it is a dead letter.
|
||||
OptionDeliveries = "max-deliveries"
|
||||
// OptionDuplicates is how long, in seconds, a work queue detects a message sent twice.
|
||||
OptionDuplicates = "duplicates"
|
||||
)
|
||||
|
||||
// ConfigWhenFull is what a configuration of its own says a bare stream does when full: WhenFullRefuse or
|
||||
// WhenFullDropOldest. In a template it is fixed, never an option (ADR 0595 §3).
|
||||
const (
|
||||
ConfigWhenFull = "when-full"
|
||||
WhenFullRefuse = "refuse"
|
||||
WhenFullDropOldest = "drop-oldest"
|
||||
)
|
||||
|
||||
// The ledger effects, what an operation does to an instance's state (ADR 0597 §2): one of this fixed set.
|
||||
const (
|
||||
// EffectSetFields sets the state's fields to the operation's Fields.
|
||||
EffectSetFields = "set-fields"
|
||||
// EffectSetEntry sets one entry of the state's map Into, under the key the operation's field Key holds, to its Fields.
|
||||
EffectSetEntry = "set-entry"
|
||||
// EffectAppend appends the operation's Fields to the state's list Into.
|
||||
EffectAppend = "append"
|
||||
// EffectRemove removes the entry of the map Into under the operation's field Key, or, naming no Into, the instance's
|
||||
// state.
|
||||
EffectRemove = "remove"
|
||||
)
|
||||
|
||||
// LedgerType is one type of a ledger (ADR 0597 §1): a schema and the operations allowed on it, with its indexes. An
|
||||
// instance of it is named by a path whose first part is the type's name (`decision/0290`).
|
||||
type LedgerType struct {
|
||||
Name string `json:"name"`
|
||||
// Schema is the type's fields, by name.
|
||||
Schema map[string]LedgerField `json:"schema"`
|
||||
// Operations are the operations allowed on an instance, each with its effect.
|
||||
Operations []LedgerOperation `json:"operations"`
|
||||
// Indexes are its declared indexes, each a second key-value bucket kept by a watcher (ADR 0597 §4).
|
||||
Indexes []LedgerIndex `json:"indexes,omitempty"`
|
||||
}
|
||||
|
||||
// LedgerField is one field of a ledger type's schema.
|
||||
type LedgerField struct {
|
||||
// Type is the field's value type, as the schema names it.
|
||||
Type string `json:"type"`
|
||||
// DigestOf names the text field this field is the digest of (ADR 0597 §2): `apply` refuses an operation whose
|
||||
// digest does not match its text.
|
||||
DigestOf string `json:"digest-of,omitempty"`
|
||||
}
|
||||
|
||||
// LedgerOperation is one operation a ledger type allows, and its effect on the state.
|
||||
type LedgerOperation struct {
|
||||
Name string `json:"name"`
|
||||
// Effect is one of the ledger effects (EffectSetFields, EffectSetEntry, EffectAppend, EffectRemove).
|
||||
Effect string `json:"effect"`
|
||||
// Fields are the operation's fields its effect copies into the state; it may carry more (ADR 0597 §2).
|
||||
Fields []string `json:"fields,omitempty"`
|
||||
// Into is the state's map (EffectSetEntry, EffectRemove) or list (EffectAppend) the effect writes.
|
||||
Into string `json:"into,omitempty"`
|
||||
// Key is the operation's field holding the entry's key (EffectSetEntry, EffectRemove).
|
||||
Key string `json:"key,omitempty"`
|
||||
// Sequence and Author are whether the effect also writes the operation's sequence and its proven author.
|
||||
Sequence bool `json:"sequence,omitempty"`
|
||||
Author bool `json:"author,omitempty"`
|
||||
}
|
||||
|
||||
// LedgerIndex is one declared index of a ledger type: its name, and the fields it is kept on.
|
||||
type LedgerIndex struct {
|
||||
Name string `json:"name"`
|
||||
On []string `json:"on"`
|
||||
}
|
||||
|
||||
// BusStateField is where a stored manifest keeps its bus-state entries: beside `requires`, which stays a plain list of
|
||||
// names, as OptionalRequiresKey keeps the optional ones.
|
||||
const BusStateField = "bus-state"
|
||||
|
||||
// BusStateEntry is one piece of a module's bus state, as its manifest asks for it.
|
||||
type BusStateEntry struct {
|
||||
// Name is its local name; the mesh, never the module, names it on the bus (ADR 0201 §4).
|
||||
Name string `json:"name"`
|
||||
// Storage is its storage type (design 25 § 12.1): how the bus keeps it.
|
||||
Storage string `json:"storage"`
|
||||
// Template is the predefined configuration of that storage type it uses; empty for a bare stream, which gives
|
||||
// Configuration instead.
|
||||
Template string `json:"template,omitempty"`
|
||||
// Options are the template's settings the module changes, by option name; each value as the manifest wrote it.
|
||||
Options map[string]json.RawMessage `json:"options,omitempty"`
|
||||
// Configuration is a configuration of the module's own, where no template fits: a bare stream's, or a ledger's
|
||||
// whose history expires (ADR 0595 §2).
|
||||
Configuration map[string]json.RawMessage `json:"configuration,omitempty"`
|
||||
// Types are a ledger's types (ADR 0597 §1), for the storage type ledger only.
|
||||
Types []LedgerType `json:"types,omitempty"`
|
||||
// Events are the events an entry of the template events carries, named locally, as a part's finding stream
|
||||
// lists its three finding events; the catalog graph entry carries them (design 47 G §529).
|
||||
Events []string `json:"events,omitempty"`
|
||||
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
|
||||
KeptLayout
|
||||
}
|
||||
|
||||
// isBusStateEntry is whether a `requires` item in its long form asks for bus state: it names a storage type.
|
||||
func isBusStateEntry(raw json.RawMessage) bool {
|
||||
var keys map[string]json.RawMessage
|
||||
if json.Unmarshal(raw, &keys) != nil {
|
||||
return false
|
||||
}
|
||||
_, has := keys["storage"]
|
||||
return has
|
||||
}
|
||||
|
||||
// readBusStateEntry reads one bus-state entry leniently (ADR 0578 §3): a key it does not know, at any depth, is set
|
||||
// aside and its path named, never refused. An error is only an entry of the wrong shape.
|
||||
func readBusStateEntry(raw json.RawMessage) (BusStateEntry, []string, error) {
|
||||
var tree any
|
||||
dec := json.NewDecoder(bytes.NewReader(raw))
|
||||
dec.UseNumber()
|
||||
if err := dec.Decode(&tree); err != nil {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
var aside []string
|
||||
strict := func(e *BusStateEntry) error {
|
||||
b, err := json.Marshal(tree)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d := json.NewDecoder(bytes.NewReader(b))
|
||||
d.DisallowUnknownFields()
|
||||
return typedUnknown(d.Decode(e))
|
||||
}
|
||||
for tries := 0; ; tries++ {
|
||||
var e BusStateEntry
|
||||
err := strict(&e)
|
||||
if err == nil {
|
||||
if err := e.KeptLayout.refuse(fmt.Sprintf("requires entry %q", e.Name)); err != nil {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
return e, aside, nil
|
||||
}
|
||||
unknown := asUnknownField(err)
|
||||
if unknown == nil || tries > 64 {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
path, ok := removalThatHelps(tree, unknown.Field, err.Error(), func() error { var e BusStateEntry; return strict(&e) })
|
||||
if !ok {
|
||||
if removeEvery(tree, unknown.Field) == 0 {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
path = "…." + unknown.Field
|
||||
}
|
||||
aside = append(aside, path)
|
||||
}
|
||||
}
|
||||
|
||||
// setAsideAlso adds paths read without to what a manifest set aside (SetAside), in its words.
|
||||
func setAsideAlso(unknown string, paths []string) string {
|
||||
if len(paths) == 0 {
|
||||
return unknown
|
||||
}
|
||||
if unknown == "" {
|
||||
field := paths[0][strings.LastIndex(paths[0], ".")+1:]
|
||||
unknown = fmt.Sprintf("json: unknown field %q", field)
|
||||
}
|
||||
return unknown + " (read without " + strings.Join(paths, ", ") + ")"
|
||||
}
|
||||
|
||||
// KeptStates are the local names of the state a manifest keeps, each a key-value bucket of its own: the old `state`
|
||||
// section's and every key-value entry it asks for in `requires` (ADR 0595 §1, §11), sorted, once each. It is the one
|
||||
// rule every reader of what a module keeps calls (novox/hq issue 1966): a process's `bus.state` and a seat
|
||||
// contribution's state name one of these, in either form.
|
||||
func KeptStates(m Manifest) []string {
|
||||
var out []string
|
||||
for _, k := range KeptDeclarations(m) {
|
||||
if k.Kind == KeptKindState {
|
||||
out = append(out, k.Name)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -1,58 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import "encoding/json"
|
||||
|
||||
// CatalogSeat is the seat of the catalog: what the mesh holds of every module (novox/hq ADR 0604 §1). Its states are
|
||||
// its holder's: the modules the trunk holds, their manifests by digest and the module graph (the indexer's states
|
||||
// `modules`, `manifests` and `graph`), announced with who may read them; a reader that only watches reads them and calls
|
||||
// no verb. It names no event. What must be built again is no verb: a module whose fingerprint differs from its newest
|
||||
// build, which the delivery view derives. The verbs' answers are CatalogModules, CatalogOne, CatalogProvides and
|
||||
// CatalogDependents, whose field names are those of the catalog module's tools they replace.
|
||||
|
||||
// CatalogEntry is one module of the catalog: its repository, its directory there ("." for the root), the merge commit it
|
||||
// was last changed at, and its new-form fingerprint or why it has none.
|
||||
type CatalogEntry struct {
|
||||
Module string `json:"module"`
|
||||
Repository string `json:"repository"`
|
||||
Path string `json:"path"`
|
||||
Commit string `json:"commit"`
|
||||
FP string `json:"fp,omitempty"`
|
||||
None string `json:"none,omitempty"`
|
||||
}
|
||||
|
||||
// CatalogModule is one module whole: its entry and its manifest as the trunk holds it.
|
||||
type CatalogModule struct {
|
||||
CatalogEntry
|
||||
Manifest json.RawMessage `json:"manifest"`
|
||||
}
|
||||
|
||||
// CatalogEnd is a module that provides, requires or depends on something, with its commit; Scope is a provider's scope
|
||||
// (a module provides its own name at node scope).
|
||||
type CatalogEnd struct {
|
||||
Module string `json:"module"`
|
||||
Commit string `json:"commit"`
|
||||
Scope string `json:"scope,omitempty"`
|
||||
}
|
||||
|
||||
// CatalogModules is the answer of the verb modules: every module the trunk holds, sorted by name.
|
||||
type CatalogModules struct {
|
||||
Modules []CatalogEntry `json:"modules"`
|
||||
}
|
||||
|
||||
// CatalogOne is the answer of the verb module.
|
||||
type CatalogOne struct {
|
||||
Module CatalogModule `json:"module"`
|
||||
}
|
||||
|
||||
// CatalogProvides is the answer of the verb provides: the modules that provide the provision and those that require it.
|
||||
type CatalogProvides struct {
|
||||
Provision string `json:"provision"`
|
||||
Provides []CatalogEnd `json:"provides"`
|
||||
Requires []CatalogEnd `json:"requires"`
|
||||
}
|
||||
|
||||
// CatalogDependents is the answer of the verb dependents: the modules whose build edges reach the module.
|
||||
type CatalogDependents struct {
|
||||
Module string `json:"module"`
|
||||
Dependents []CatalogEnd `json:"dependents"`
|
||||
}
|
||||
@@ -1,6 +1,32 @@
|
||||
package catalogue
|
||||
|
||||
// CheckerSeat is the seat mesh-delivery-checker holds.
|
||||
//
|
||||
// Deprecated: a whole seat's name is mesh-contracts' (seats.MeshDeliveryChecker); kept for its importers until their
|
||||
// next change takes it from there (testdata/seat-names.txt).
|
||||
const CheckerSeat = "mesh-delivery-checker"
|
||||
|
||||
// checkerVerbs are the checker seat's verbs: ask a try, read one, end one by hand. Optional until the checker serves them
|
||||
// everywhere.
|
||||
func checkerVerbs() []Verb {
|
||||
return []Verb{
|
||||
{Name: "try", Optional: true, Description: "Try a pushed head before its pull request is opened: the checker " +
|
||||
"judges it as it judges a pull request's head, merged onto its trunk's head, its unit and integration " +
|
||||
"levels through the build modules, and owes the forge nothing. Answers the try's id at once; try-read " +
|
||||
"reads where it stands. A try of the same head that waits or runs is the answer to a second ask; at most " +
|
||||
"4 wait or run at once, mesh-wide.",
|
||||
Input: schema(map[string]string{"repository": "<owner>/<name>",
|
||||
"head": "the full commit to try, 40 or 64 hex digits",
|
||||
"trunk": "the trunk commit to merge it onto, or the trunk's branch for its head now; the trunk's head when absent",
|
||||
"tests": "a go test -run pattern: only those tests run, and nothing is kept",
|
||||
"level": "unit or integration: only this level; both when absent"},
|
||||
[]string{"repository", "head"}),
|
||||
Replaces: []string{"sh merge-check.sh", "docker run postgres"}},
|
||||
{Name: "try-read", Optional: true, Description: "Where a try stands, as stored: queued, running, or its end " +
|
||||
"(pass, fail, or error when it could not run), with what was said of it, the head and the trunk commit " +
|
||||
"it was merged onto, the time it took and what failed first. Read-only.",
|
||||
Input: schema(map[string]string{"id": "the try's id, try-<digits>"}, []string{"id"})},
|
||||
{Name: "try-end", Optional: true, Description: "End a try that waits or runs, as error, by hand: say why (kept in " +
|
||||
"what it says). Its place among the tries is free at once and the next try of its head asks anew with a new " +
|
||||
"id. A try that ended already is refused.",
|
||||
Input: schema(map[string]string{"id": "the try's id, try-<digits>", "why": "why it is ended, in words"},
|
||||
[]string{"id", "why"})},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -114,7 +114,6 @@ type configDeclaration struct {
|
||||
Default any `json:"default,omitempty"`
|
||||
Required bool `json:"required,omitempty"`
|
||||
Runtime bool `json:"runtime,omitempty"`
|
||||
RenamedFrom string `json:"renamed-from,omitempty"`
|
||||
}
|
||||
|
||||
// MarshalJSON writes a settings declaration exactly as before, and a config declaration without the settings
|
||||
@@ -126,7 +125,7 @@ func (d SettingDeclaration) MarshalJSON() ([]byte, error) {
|
||||
}
|
||||
return json.Marshal(configDeclaration{Type: d.Type, Description: d.Description, Enum: d.Enum,
|
||||
Minimum: d.Minimum, Maximum: d.Maximum, Pattern: d.Pattern, Trusted: d.Trusted, Predefined: d.Predefined,
|
||||
Default: d.Default, Required: d.Required, Runtime: d.Runtime, RenamedFrom: d.RenamedFrom})
|
||||
Default: d.Default, Required: d.Required, Runtime: d.Runtime})
|
||||
}
|
||||
|
||||
// validationProblems is what is wrong with a declaration's validation, and its default against it.
|
||||
|
||||
@@ -132,33 +132,6 @@ func TestAConfigManifestIsWrittenBackInItsOwnForm(t *testing.T) {
|
||||
if !bytes.Equal(raw, twice) {
|
||||
t.Fatalf("read again it is written\n%s\nwhere it was\n%s", twice, raw)
|
||||
}
|
||||
|
||||
// The registry milestone's words round-trip the same (design 32): a config value's old name, an artifact's series
|
||||
// and contracts marker, the series a module accepts of a package; absent, a series is 0.1.
|
||||
declared := []byte(`{"module":"c","version":"1","config":{"font":{"type":"string","renamed-from":"font-name"}},
|
||||
"packages":{"mesh-sdk":"0.2"},
|
||||
"build":{"artifacts":[{"name":"go","kind":"package","language":"go","paths":["go"],"series":"0.2",
|
||||
"generated-from":"seats"},{"name":"lib","kind":"package","language":"typescript"}]}}`)
|
||||
m, err = decodeForTest(declared)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if m.Settings["font"].RenamedFrom != "font-name" || m.AcceptedSeries("mesh-sdk") != "0.2" ||
|
||||
m.AcceptedSeries("mesh-contracts") != DefaultSeries || m.Build.Artifacts[0].SeriesOf() != "0.2" ||
|
||||
m.AcceptedSeries("mesh-sdk", "0.1") != "0.2" || m.AcceptedSeries("mesh-contracts", "0.10", "x", "0.3", "1.0") != "0.3" ||
|
||||
m.Build.Artifacts[0].GeneratedFrom != "seats" || m.Build.Artifacts[1].SeriesOf() != "0.1" {
|
||||
t.Fatalf("read %+v", m)
|
||||
}
|
||||
raw, _ = json.Marshal(m)
|
||||
for _, said := range []string{`"renamed-from":"font-name"`, `"packages":{"mesh-sdk":"0.2"}`, `"series":"0.2"`,
|
||||
`"generated-from":"seats"`} {
|
||||
if !strings.Contains(string(raw), said) {
|
||||
t.Errorf("written back without %s: %s", said, raw)
|
||||
}
|
||||
}
|
||||
if strings.Count(string(raw), `"series"`) != 1 {
|
||||
t.Errorf("a series nobody said is written: %s", raw)
|
||||
}
|
||||
}
|
||||
|
||||
// A module in the settings form is written back byte for byte as before.
|
||||
|
||||
@@ -1,7 +1,54 @@
|
||||
package catalogue
|
||||
|
||||
// The two seats.
|
||||
const ()
|
||||
const (
|
||||
WantsSeat = "mesh-wants"
|
||||
HandActsSeat = "mesh-hand-acts"
|
||||
)
|
||||
|
||||
// controlRecordsDecision is the record that made the two seats.
|
||||
const controlRecordsDecision = "novox/hq ADR 0565"
|
||||
|
||||
// wantsVerbs are the assigner's verbs on mesh-wants (ADR 0565 clauses 7, 8, 9, 15).
|
||||
func wantsVerbs() []Verb {
|
||||
return []Verb{
|
||||
{Name: "upgrade", Description: "Whether each module's new builds wait for a person: its policy (roll, together or record) " +
|
||||
"and where it comes from (a person, the module, the bus, its irreplaceable data, the default); the bus is never " +
|
||||
"rolled. With module, that one; with module, policy and why, a person's choice for it, recorded as a hand act; " +
|
||||
"with policy default, the choice taken back. With compare, each node's own composition and the assigner's findings.",
|
||||
Input: schema(map[string]string{
|
||||
"module": "one module",
|
||||
"policy": "with module: roll, together, record, or default to take a person's choice back",
|
||||
"why": "with policy: why, required and kept",
|
||||
"compare": "\"true\": each node's own composition, by its digest, and the assigner's findings",
|
||||
}, nil, "compare")},
|
||||
{Name: "bus", Description: "What a bus upgrade would do: the fingerprint of the bus module each node runs against the " +
|
||||
"indexer's newest, and the last bus steps from the hand-act log and the assigner's puts of the bus. Read-only: " +
|
||||
"the step itself is the put on the operator's warrant.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
}
|
||||
}
|
||||
|
||||
// handActsVerbs are mesh-hand-acts' verbs (ADR 0565 clause 10), as the controller's took them.
|
||||
func handActsVerbs() []Verb {
|
||||
return []Verb{
|
||||
{Name: "hand-act", Description: "Record one act done by hand: what, why, its cause in a word or a condition's kind " +
|
||||
"(the word a second act for the same reason uses) and the condition it addresses. Who did it is the caller the " +
|
||||
"runtime proves, never said by the caller. Said as act-recorded.",
|
||||
Input: schema(map[string]string{
|
||||
"what": "what was done, in a line",
|
||||
"why": "why it had to be done by hand",
|
||||
"cause": "the cause in a word, or a condition's kind",
|
||||
"condition": "the key of the condition it addressed, if any (optional)",
|
||||
"verb": "the act's verb in the log: a repair when absent, `hand-act drill` for a drill, or a recorder's own",
|
||||
}, []string{"what", "why"})},
|
||||
{Name: "hand-acts", Description: "What was done by hand lately, over this log and the controller's: who, why and the " +
|
||||
"cause of each, and the causes recorded twice within a fortnight (a healer is wanted).",
|
||||
Input: schema(map[string]string{"days": "how many days back (default 14)", "cause": "only this cause",
|
||||
"who": "only acts whose who contains this", "warrants": "\"true\": every warrant recorded"}, nil, "warrants")},
|
||||
{Name: "warranted", Description: "Record what a module did on the operator's warrant: who chose, through which channel, " +
|
||||
"with which proofs and which answer read from the router's own record of that module's ask, never from the " +
|
||||
"caller; once per ask. Refused for an ask still open, ended without a choice, or not that module's.",
|
||||
Input: schema(map[string]string{"asker": "the module that asked", "ask": "its ask's id"}, []string{"asker", "ask"})},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// EndedControllerName is the name the ended controller had, as its module and as its seat: no manifest may name it.
|
||||
const EndedControllerName = "mesh-controller"
|
||||
|
||||
// The kinds of a declaration that names the controller.
|
||||
const (
|
||||
DependsRead = "read"
|
||||
DependsInvoke = "invoke"
|
||||
DependsConsume = "consume"
|
||||
DependsUses = "uses"
|
||||
DependsWaits = "waits"
|
||||
DependsClaims = "claims"
|
||||
DependsRequires = "requires"
|
||||
DependsIs = "is"
|
||||
)
|
||||
|
||||
// Dependency is one declaration of a manifest that names the controller.
|
||||
type Dependency struct {
|
||||
Module string `json:"module"`
|
||||
// By is the account declaring it when it is not the module's own: `process <id>` or `tools <binary>`.
|
||||
By string `json:"by,omitempty"`
|
||||
Kind string `json:"kind"`
|
||||
What string `json:"what"`
|
||||
}
|
||||
|
||||
// onController says whether a name `<owner>.<rest>` is the controller's.
|
||||
func onController(name string) bool { return strings.HasPrefix(name, EndedControllerName+".") }
|
||||
|
||||
// invokesController says whether an invoke names a verb of the controller's seat or a tool of its module. `*` names none:
|
||||
// it is the console's grant to every tool.
|
||||
func invokesController(invoke string) bool {
|
||||
if seat, _, ok := SeatInvoke(invoke); ok {
|
||||
return seat == EndedControllerName
|
||||
}
|
||||
return onController(invoke)
|
||||
}
|
||||
|
||||
// Dependencies are every declaration naming the controller in the manifests given, by module.
|
||||
func Dependencies(manifests map[string]Manifest) []Dependency {
|
||||
var out []Dependency
|
||||
add := func(module, by, kind, what string) {
|
||||
if d := (Dependency{Module: module, By: by, Kind: kind, What: what}); !slices.Contains(out, d) {
|
||||
out = append(out, d)
|
||||
}
|
||||
}
|
||||
for name, m := range manifests {
|
||||
if name == EndedControllerName || m.Module == EndedControllerName {
|
||||
add(name, "", DependsIs, EndedControllerName)
|
||||
}
|
||||
for _, r := range m.Reads {
|
||||
if onController(r) {
|
||||
add(name, "", DependsRead, r)
|
||||
}
|
||||
}
|
||||
for _, i := range m.Invokes {
|
||||
if invokesController(i) {
|
||||
add(name, "", DependsInvoke, i)
|
||||
}
|
||||
}
|
||||
for _, c := range m.Consumes {
|
||||
if onController(c) {
|
||||
add(name, "", DependsConsume, c)
|
||||
}
|
||||
}
|
||||
for _, w := range m.Waits {
|
||||
if onController(w) {
|
||||
add(name, "", DependsWaits, w)
|
||||
}
|
||||
}
|
||||
for _, u := range m.Uses {
|
||||
if u == EndedControllerName {
|
||||
add(name, "", DependsUses, u)
|
||||
}
|
||||
}
|
||||
for _, c := range m.Claims {
|
||||
if c.Name == EndedControllerName {
|
||||
add(name, "", DependsClaims, c.Name)
|
||||
}
|
||||
}
|
||||
for _, r := range m.Requires {
|
||||
if r == EndedControllerName || onController(r) {
|
||||
add(name, "", DependsRequires, r)
|
||||
}
|
||||
}
|
||||
for _, a := range m.BusAccounts() {
|
||||
by := "process " + a.ID
|
||||
if a.Tools {
|
||||
by = "tools " + a.ID
|
||||
}
|
||||
for _, i := range a.Bus.Invokes {
|
||||
if invokesController(i) {
|
||||
add(name, by, DependsInvoke, i)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool {
|
||||
a, b := out[i], out[j]
|
||||
if a.Module != b.Module {
|
||||
return a.Module < b.Module
|
||||
}
|
||||
if a.Kind != b.Kind {
|
||||
return a.Kind < b.Kind
|
||||
}
|
||||
if a.What != b.What {
|
||||
return a.What < b.What
|
||||
}
|
||||
return a.By < b.By
|
||||
})
|
||||
return out
|
||||
}
|
||||
|
||||
// ManifestRefusals is the check over a repository's manifests, by module name: one reason per declaration naming the
|
||||
// controller, none when it passes. None given fails it, since nothing read can hide a declaration.
|
||||
func ManifestRefusals(manifests map[string]Manifest) []string {
|
||||
if len(manifests) == 0 {
|
||||
return []string{"no manifest was read: with nothing read, nothing naming the controller can be judged absent"}
|
||||
}
|
||||
var out []string
|
||||
for _, d := range Dependencies(manifests) {
|
||||
who := d.Module
|
||||
if d.By != "" {
|
||||
who += " (" + d.By + ")"
|
||||
}
|
||||
out = append(out, fmt.Sprintf("%s declares %s %s: the controller ended by its unassignment and nothing may name it "+
|
||||
"(ADR 0590 clause 5)", who, d.Kind, d.What))
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Issue 1126 B1/AC1, B2/AC1 (ADR 0590 clause 5): the seat set holds no mesh-controller, and the dependence check refuses
|
||||
// every manifest that reads (the lease too), invokes, consumes, waits on, uses, claims, requires or is mesh-controller.
|
||||
func TestNothingMayNameTheController(t *testing.T) {
|
||||
for _, s := range DefaultSeats() {
|
||||
if s.Name == EndedControllerName {
|
||||
t.Errorf("the seat set still holds %s", s.Name)
|
||||
}
|
||||
}
|
||||
if _, ok := SeatNamed(EndedControllerName); ok {
|
||||
t.Errorf("SeatNamed still finds %s", EndedControllerName)
|
||||
}
|
||||
for _, row := range []struct {
|
||||
name, manifest string
|
||||
want []string // "<kind> <what>"
|
||||
}{
|
||||
{"a read of the lease", `{"module":"mod-l","version":"1","reads":["mesh-controller.lease"]}`, []string{"read mesh-controller.lease"}},
|
||||
{"the controller's own manifest", `{"module":"mesh-controller","version":"1"}`, []string{"is mesh-controller"}},
|
||||
{"a claim of its seat", `{"module":"a-controller","version":"1","claims":[{"name":"mesh-controller","scope":"mesh"}]}`,
|
||||
[]string{"claims mesh-controller"}},
|
||||
{"each declaration kind; every tool (*) is none",
|
||||
`{"module":"mod-i","version":"1","invokes":["seat:mesh-controller.root-free","mesh-controller.status","*","nats.nats_users"],` +
|
||||
`"consumes":["mesh-controller.artifacts-gone","mod-x.thing"],"uses":["mesh-controller"],"waits":["mesh-controller.checked"],` +
|
||||
`"requires":["mesh-controller"]}`,
|
||||
[]string{"consume mesh-controller.artifacts-gone", "invoke mesh-controller.status", "invoke seat:mesh-controller.root-free",
|
||||
"requires mesh-controller", "uses mesh-controller", "waits mesh-controller.checked"}},
|
||||
{"a process's own invoke", `{"module":"mod-p","version":"1","resources":[{"id":"account","type":"user","name":"mod-p"},` +
|
||||
`{"id":"worker","type":"process","run":["./w"],"user":"mod-p","bus":{"invokes":["seat:mesh-controller.held"]}}]}`,
|
||||
[]string{"invoke seat:mesh-controller.held"}},
|
||||
{"a module on its own seats", `{"module":"mod-ok","version":"1","reads":["mesh-grants.accounts"],"invokes":["*"]}`, nil},
|
||||
} {
|
||||
m, err := Decode([]byte(row.manifest))
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", row.name, err)
|
||||
}
|
||||
var got []string
|
||||
for _, d := range Dependencies(map[string]Manifest{m.Module: m}) {
|
||||
got = append(got, d.Kind+" "+d.What)
|
||||
}
|
||||
if !slices.Equal(got, row.want) {
|
||||
t.Errorf("%s: dependencies %q, want %q", row.name, got, row.want)
|
||||
}
|
||||
refused := ManifestRefusals(map[string]Manifest{m.Module: m})
|
||||
if len(refused) != len(row.want) || len(refused) > 0 && !strings.Contains(refused[0], "ADR 0590") {
|
||||
t.Errorf("%s: refused %q", row.name, refused)
|
||||
}
|
||||
}
|
||||
if got := ManifestRefusals(map[string]Manifest{}); len(got) != 1 || !strings.Contains(got[0], "no manifest") {
|
||||
t.Errorf("no manifest given: %q", got)
|
||||
}
|
||||
}
|
||||
@@ -4,10 +4,11 @@ import (
|
||||
"encoding/json"
|
||||
)
|
||||
|
||||
// CarriesCoreCredentials says whether a module's own secrets are the core's credentials, as its manifest says
|
||||
// (Manifest.CoreCredentials): the vault seals them to the machine, and the node-engine and the vault hold no list of
|
||||
// such modules.
|
||||
func CarriesCoreCredentials(m Manifest) bool { return m.CoreCredentials }
|
||||
// CoreCredentialModules are the modules whose own secrets are the core's credentials: the bus, mesh-provisioner and
|
||||
// its client. The node-engine and the controller hold no own secret of a module's. **Fixed here, in the controller's
|
||||
// code, as the clause says** — and the vault keeps the same list (mesh-vault's CoreModules), answering the controller
|
||||
// for these alone.
|
||||
var CoreCredentialModules = []string{"mesh-provisioner", "mesh-provisioner-client", "nats"}
|
||||
|
||||
// VaultSealedField is the field of a file the vault sealed to its machine (mesh-host's File.VaultSealed).
|
||||
const VaultSealedField = "vault-sealed"
|
||||
|
||||
@@ -84,7 +84,7 @@ type Rendering struct {
|
||||
// makes. Nil on every composition that is sent, and on every other question, which then refuse
|
||||
// a secret not made with a *NotMadeError.
|
||||
Foreseen map[string]map[string]bool
|
||||
// VaultSealed is, for a module whose manifest says its own secrets are the core's credentials (core-credentials), each own secret as the vault sealed it
|
||||
// VaultSealed is, for a module on the core's list (CoreCredentialModules), each own secret as the vault sealed it
|
||||
// to this machine (the mesh's sealed box, as the vault answered it), keyed like Needed (novox/hq ADR 0349 §16,
|
||||
// issue 755). Given only for a machine whose engine opens the box (VaultBoxContract): its file then carries the
|
||||
// box and never the controller's own sealed value. Nil everywhere else, and the composition is as it was.
|
||||
@@ -238,11 +238,6 @@ type Rendering struct {
|
||||
// when no holder is reachable yet; see seat_into.go for which seats are answered.
|
||||
SeatReach map[string]string
|
||||
|
||||
// SeatHolder is the module holding each seat SeatReach answers, by seat, from the holdings the composition has (novox/hq
|
||||
// design 51, issue 1966 ENG-10): written beside an image routed through the seat, for the node-engine to wait on when its
|
||||
// pull fails. Absent for a seat with no holder on record.
|
||||
SeatHolder map[string]string
|
||||
|
||||
// Built is every `<module>/<artifact>` the mesh has built. What tells a reference recorded
|
||||
// with an address — before references were kept without one — from an image a module runs
|
||||
// straight from a public registry.
|
||||
@@ -278,8 +273,8 @@ type Rendering struct {
|
||||
Hands bool
|
||||
Provisioner string
|
||||
// Assigned is, by module, whether the vault judged it handed (AssignmentHands of the plan in its assignment), as the
|
||||
// vault answered the node composing itself. Never the rule: the node hands by the manifest, its build and the machine,
|
||||
// and reads this only to say where the vault disagrees (issue 1966). Nil where the vault said nothing.
|
||||
// vault answered the node composing itself; where it names a module, Handing follows it alone (issue 774). Nil, or a
|
||||
// module it does not name, is judged by the manifest and the machine, as the composer of the plans does.
|
||||
Assigned map[string]bool
|
||||
}
|
||||
|
||||
|
||||
+1
-53
@@ -1,9 +1,7 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
@@ -11,65 +9,15 @@ import (
|
||||
// contract knows as it declares it, and a field it does not know set aside and named (SetAside), never refused. It
|
||||
// judges nothing: no name, seat, bus or catalogue rule is applied, since the indexer judged the manifest once and its
|
||||
// verdict is in the indexer's state (delivery.VerdictsBucket); a reader takes the verdict beside it
|
||||
// (delivery.TakeVerdict). An error is only bytes that are no manifest: not JSON, a field of the wrong shape, or one field
|
||||
// given twice in one object at any depth, named by its path: a decoder keeps the last of two equal keys without a word,
|
||||
// so a merge of two branches that each added the field would lose one side silently (as mesh-catalog's checks/manifests
|
||||
// refuses it).
|
||||
// (delivery.TakeVerdict). An error is only bytes that are no manifest: not JSON, or a field of the wrong shape.
|
||||
func Decode(raw []byte) (Manifest, error) {
|
||||
var m Manifest
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
return Manifest{}, err
|
||||
}
|
||||
if twice := givenTwice(raw); twice != "" {
|
||||
return Manifest{}, fmt.Errorf("the field %s is given twice; a manifest gives each field once", twice)
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// givenTwice is the first field an object of raw gives twice, as its path (`invokes`, `build.artifacts[0].env`), or empty.
|
||||
func givenTwice(raw []byte) string {
|
||||
d := json.NewDecoder(bytes.NewReader(raw))
|
||||
var walk func(path string) string
|
||||
walk = func(path string) string {
|
||||
t, err := d.Token()
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
switch t {
|
||||
case json.Delim('{'):
|
||||
seen := map[string]bool{}
|
||||
for d.More() {
|
||||
k, err := d.Token()
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
key, _ := k.(string)
|
||||
at := key
|
||||
if path != "" {
|
||||
at = path + "." + key
|
||||
}
|
||||
if seen[key] {
|
||||
return at
|
||||
}
|
||||
seen[key] = true
|
||||
if twice := walk(at); twice != "" {
|
||||
return twice
|
||||
}
|
||||
}
|
||||
_, _ = d.Token()
|
||||
case json.Delim('['):
|
||||
for i := 0; d.More(); i++ {
|
||||
if twice := walk(fmt.Sprintf("%s[%d]", path, i)); twice != "" {
|
||||
return twice
|
||||
}
|
||||
}
|
||||
_, _ = d.Token()
|
||||
}
|
||||
return ""
|
||||
}
|
||||
return walk("")
|
||||
}
|
||||
|
||||
// SetAside names what Decode set aside of this manifest: each field it does not know, as the decoder words the first
|
||||
// and with the path of every one it read without, or "" when it knew every field. A reader says it in its own words
|
||||
// (its journal, its finding) and reads the rest; the part that needs the field takes the newer package (ADR 0578 §6).
|
||||
|
||||
@@ -1,5 +1,90 @@
|
||||
package catalogue
|
||||
|
||||
// DeliverySeat is the seat mesh-delivery holds.
|
||||
const DeliverySeat = "mesh-delivery"
|
||||
|
||||
// RemovedDeliveryVerbs are the delivery seat's verbs release 6 removed (novox/hq ADR 0539 clause 17): with no plan the
|
||||
// gate keeps the merge's states and derives nothing after the merge. Each stays in deliveryVerbs as an optional row
|
||||
// with no input, so the holder of before and the holder of release 6 both hold the seat and a caller is answered by
|
||||
// name with its replacement, given here.
|
||||
var RemovedDeliveryVerbs = map[string]string{
|
||||
"times": "the delivery view (nox-delivery): each machine's want and have, and since when",
|
||||
"what-if": "the merge check's composition, which composes every machine a change touches",
|
||||
"release": "the assigner's asks (hold, restore, backup, record), answered on the phone or in the delivery view",
|
||||
"stop": "a bad fingerprint is held and its canary set back by the assigner (ADR 0429 §6, ADR 0539 clause 18)",
|
||||
"retire-history": "nothing to retire: the gate keeps no delivery after its merge",
|
||||
}
|
||||
|
||||
// deliveryRemovedInR6 is the row of a verb release 6 removed from the delivery seat.
|
||||
func deliveryRemovedInR6(name string) Verb {
|
||||
return Verb{Name: name, Description: "Removed in R6 (ADR 0539): refused by name, naming its replacement: " +
|
||||
RemovedDeliveryVerbs[name], Input: schema(nil, nil), Optional: true}
|
||||
}
|
||||
|
||||
// deliveryVerbs are the delivery seat's tools: what reads the merges and checks, and the acts of a person and healer H2
|
||||
// on a merge (ADR 0539 clause 17).
|
||||
func deliveryVerbs() []Verb {
|
||||
return []Verb{
|
||||
{Name: "deliveries", Description: "Every delivery not final, and those that ended in the last day, one line " +
|
||||
"each: its id (owner/repository@commit), its state, its group, what it waits for and since when. Narrowed " +
|
||||
"by state, repository or group.",
|
||||
Input: schema(map[string]string{"state": "one state, e.g. delivering or held",
|
||||
"repository": "owner/repository", "group": "a group's id (its branch name)",
|
||||
"all": "\"true\": the final ones of the last thirty days too"}, nil, "all")},
|
||||
deliveryRemovedInR6("times"),
|
||||
{Name: "show", Description: "One delivery or group whole: its delivery plan (what it builds, what each " +
|
||||
"machine receives, what is not an ordinary send), every transition with when and why, the machine " +
|
||||
"steps of its walk, its group and its order.",
|
||||
Input: schema(map[string]string{"id": "a delivery's id, or a group's"}, []string{"id"})},
|
||||
{Name: "groups", Description: "Every delivery group: its members in order, why each pair is ordered " +
|
||||
"(declared, built by, version skew, engine before controller, by name), its composed check and its " +
|
||||
"state, derived from its members.",
|
||||
Input: schema(map[string]string{"all": "\"true\": the groups that ended too"}, nil, "all")},
|
||||
deliveryRemovedInR6("what-if"),
|
||||
{Name: "checks", Description: "What the mesh's checks said of a pull request's head or of one commit: each " +
|
||||
"of the commit's mesh statuses (mesh/check, the one a main requires from R4e, hq's mesh/repo-check, " +
|
||||
"mesh/delivery, …) with its state, " +
|
||||
"description and when it was set; the merge check's full verdict as the controller said it — each " +
|
||||
"layer's summary, the machine that ran it, when, its build id and its report; and whether the branch's " +
|
||||
"protection would let it merge, every required status being success.",
|
||||
Input: schema(map[string]string{"repository": "owner/repository",
|
||||
"number": "a pull request's number: its head is read",
|
||||
"commit": "a commit's sha, or the start of one, instead of a pull request"}, []string{"repository"}),
|
||||
// Added after the seat's first holder shipped: optional until mesh-delivery serves it everywhere.
|
||||
Optional: true},
|
||||
{Name: "table", Description: "The state table every delivery runs by: each transition with its guard, " +
|
||||
"each state's bound and what healer H2 may do once it has passed; and the machine steps' table.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
{Name: "stalled", Description: "Every delivery held past its state's bound, with the transition the table " +
|
||||
"lets healer H2 take for it, or none.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
{Name: "recheck", Description: "Check a rejected or ready delivery again: it goes back to proposed and the " +
|
||||
"controller is asked for its check. With why.",
|
||||
Input: schema(map[string]string{"id": "the delivery's id", "why": "why, kept with the transition"},
|
||||
[]string{"id", "why"})},
|
||||
deliveryRemovedInR6("release"),
|
||||
deliveryRemovedInR6("stop"),
|
||||
{Name: "close", Description: "Healer H2's verb: take the one transition the table names for a delivery held " +
|
||||
"past its bound. Refused for any other.",
|
||||
Input: schema(map[string]string{"id": "the delivery's id", "why": "the condition it answers"},
|
||||
[]string{"id", "why"})},
|
||||
deliveryRemovedInR6("retire-history"),
|
||||
{Name: "settle", Description: "Ask the operator to settle deliveries held because their merge reached the " +
|
||||
"trunk without a passing check: one ask on the operator's channel for all of them, as an Approve is asked. " +
|
||||
"The verb moves nothing; on the operator's answer each goes to merged, and whether it is on the machines " +
|
||||
"is derived as for any merge (novox/hq ADR 0556). Refused for a delivery not held, or held for another " +
|
||||
"reason, naming why.",
|
||||
Input: map[string]any{"type": "object", "additionalProperties": false, "required": []string{"ids", "why"},
|
||||
"properties": map[string]any{
|
||||
"ids": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "minItems": 1,
|
||||
"description": "the held deliveries' ids (owner/repository@commit)"},
|
||||
"why": map[string]any{"type": "string", "description": "why they may be settled, shown to the operator"},
|
||||
}},
|
||||
// Added after the seat's release 6 holder shipped: optional until the gate serves it everywhere.
|
||||
Optional: true},
|
||||
}
|
||||
}
|
||||
|
||||
// SeatSays is whether a consumed `<seat>.<event>` names one of the seat's own events: the seat emits it,
|
||||
// exactly or by one of its patterns (`log.*`). A consumed name whose emitter is a seat and whose event the
|
||||
// seat does not emit is the event of the module of that name (novox/hq ADR 0239): mesh-delivery is a seat
|
||||
|
||||
@@ -1,22 +1,60 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/seatdef"
|
||||
)
|
||||
|
||||
// An event whose subject carries parts is said by its subject with `*` for each part, so a consumer naming one
|
||||
// instance or every instance of it is heard as the seat's (design 47 *What each step says*).
|
||||
func TestAnEventWithPartsIsTheSeatsByItsSubject(t *testing.T) {
|
||||
s := SeatFromDefinition(seatdef.Definition{Seat: "mesh-delivery", Scope: "mesh", Events: []seatdef.Message{
|
||||
{Name: "owed", Subject: "owed.<repository>.<number>"}, {Name: "pull.updated"}}})
|
||||
for _, c := range []struct {
|
||||
event string
|
||||
says bool
|
||||
}{{"owed.*.*", true}, {"owed.novox-mesh-sdk.12", true}, {"owed", false}, {"pull.updated", true}, {"pull.merged", false}} {
|
||||
if got := SeatSays(s.Emits, c.event); got != c.says {
|
||||
t.Errorf("%s: says %v, want %v (emits %v)", c.event, got, c.says, s.Emits)
|
||||
// The delivery seat answers "what did the mesh's checks say of this pull request?" as a verb of its own,
|
||||
// `checks` (novox/hq ADR 0239): read by repository and a pull request's number or a commit.
|
||||
func TestTheDeliverySeatPromisesChecks(t *testing.T) {
|
||||
seat, ok := SeatNamed(DeliverySeat)
|
||||
if !ok {
|
||||
t.Fatal("the delivery seat is not in the set")
|
||||
}
|
||||
var checks *Verb
|
||||
for i := range seat.Serves {
|
||||
if seat.Serves[i].Name == "checks" {
|
||||
checks = &seat.Serves[i]
|
||||
}
|
||||
}
|
||||
if checks == nil {
|
||||
t.Fatalf("the delivery seat promises %v and not checks", VerbNames(seat.Serves))
|
||||
}
|
||||
props, _ := checks.Input["properties"].(map[string]any)
|
||||
for _, arg := range []string{"repository", "number", "commit"} {
|
||||
if _, has := props[arg]; !has {
|
||||
t.Errorf("checks takes no %q", arg)
|
||||
}
|
||||
}
|
||||
if req, _ := checks.Input["required"].([]string); !reflect.DeepEqual(req, []string{"repository"}) {
|
||||
t.Errorf("checks requires %v, wanted only the repository", req)
|
||||
}
|
||||
|
||||
// Added after the seat's holder shipped, it is optional: the holder serving the ten verbs before it still
|
||||
// holds the seat, and one serving all eleven does too — so the controller and the catalogue can move in
|
||||
// either order, and no check of the catalogue fails on a module nobody touched between the two.
|
||||
if !checks.Optional {
|
||||
t.Fatal("checks is a condition of holding before its holder serves it")
|
||||
}
|
||||
var before []string
|
||||
for _, v := range VerbNames(seat.Serves) {
|
||||
if v != "checks" {
|
||||
before = append(before, v)
|
||||
}
|
||||
}
|
||||
m := Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh, Serves: before}}}
|
||||
if err := CanHold(m, seat); err != nil {
|
||||
t.Fatalf("a holder without checks yet: %v", err)
|
||||
}
|
||||
m.Claims[0].Serves = VerbNames(seat.Serves)
|
||||
if err := CanHold(m, seat); err != nil {
|
||||
t.Fatalf("a holder serving every verb: %v", err)
|
||||
}
|
||||
// Every other verb is still a condition of holding.
|
||||
m.Claims[0].Serves = append([]string{"checks"}, before[1:]...)
|
||||
if err := CanHold(m, seat); err == nil || !strings.Contains(err.Error(), before[0]) {
|
||||
t.Fatalf("a holder without %s: %v", before[0], err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,15 +2,15 @@ package catalogue
|
||||
|
||||
import "testing"
|
||||
|
||||
// The node-engine's compose-candidate verb (ADR 0593 §7) is optional, asks the candidate, and takes input as a boolean, as the engine and the checker read it.
|
||||
// The node-engine's compose-candidate verb (ADR 0593 §7) is optional, asks the candidate, and takes input as a switch.
|
||||
func TestTheEngineServesComposeCandidate(t *testing.T) {
|
||||
for _, v := range EngineVerbs(EngineSeat) {
|
||||
for _, v := range EngineVerbs() {
|
||||
if v.Name != "compose-candidate" {
|
||||
continue
|
||||
}
|
||||
props, _ := v.Input["properties"].(map[string]any)
|
||||
input, _ := props["input"].(map[string]any)
|
||||
if !v.Optional || props["candidate"] == nil || input == nil || input["type"] != "boolean" {
|
||||
if !v.Optional || props["candidate"] == nil || input == nil || input["enum"] == nil {
|
||||
t.Fatalf("compose-candidate reads as %+v", v)
|
||||
}
|
||||
return
|
||||
|
||||
+80
-10
@@ -1,15 +1,85 @@
|
||||
package catalogue
|
||||
|
||||
// EngineVerbs are the node-engine seat's verbs as the seat list the host gave has them (ADR 0563 §1; design 26), each
|
||||
// with its description and its input's schema as discovery shows them. The seat's name is its contract's
|
||||
// (mesh-contracts' nodeengine), never this package's (ADR 0634 §4, ADR 0635 §4): the caller names it.
|
||||
func EngineVerbs(seat string) []Verb {
|
||||
s, _ := SeatNamed(seat)
|
||||
return append([]Verb(nil), s.Serves...)
|
||||
// EngineSeat is the node seat every node's engine holds.
|
||||
const EngineSeat = "node-engine"
|
||||
|
||||
// EngineVerbs are the node-engine seat's verbs, in the order ADR 0563 §1 names them.
|
||||
func EngineVerbs() []Verb {
|
||||
return []Verb{
|
||||
{Name: "hand-over", Description: "Hand a directory this machine uses as found to the mesh (novox/hq ADR 0563 §3, " +
|
||||
"§5): the engine judges the path as it does every hand-over (absolute, as it states it, used as found, " +
|
||||
"reached through no link) and refuses in words, or asks the operator on their channel at the level approve, " +
|
||||
"binding the owner, group and mode it will set. Nothing changes on the call: only the operator's yes hands " +
|
||||
"it over. One ask is open per path; acts shows where it stands.",
|
||||
Input: schema(map[string]string{"path": "the directory's absolute path, as the engine states it",
|
||||
"why": "why it is asked, in words the operator reads"}, []string{"path"})},
|
||||
{Name: "access-act", Description: "Ask the operator for the take act on a library whose root holds data outside its " +
|
||||
"group (novox/hq ADR 0328 §7, ADR 0563 §3, §4), on the library's home engine: the access as declared and its " +
|
||||
"root as last declared, judged and refused in words or asked at the level approve. Only the operator's yes " +
|
||||
"sets the library's group, group write and setgid under its root. One ask is open per access.",
|
||||
Input: schema(map[string]string{"act": "the act: take", "access": "the access, library:<name>",
|
||||
"why": "why it is asked, in words the operator reads"}, []string{"act", "access"})},
|
||||
{Name: "judge-directories", Description: "Which of this machine's declared directories the engine would refuse to " +
|
||||
"make or keep, one line each (novox/hq ADR 0563 §2). Read-only: changes no owner or mode.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
{Name: "setuid-search", Description: "Throw away the engine's last search for setuid programs and start a full one " +
|
||||
"(novox/hq ADR 0563 §2); until it completes root-free answers that the machine is not judged yet. Refused while " +
|
||||
"one runs, and when one was started within the hour, naming when the next may start.",
|
||||
Input: schema(map[string]string{"why": "why it is asked, in words"}, nil)},
|
||||
{Name: "root-free", Description: "Whether this machine is root-free (novox/hq ADR 0259 §8, ADR 0563 §6): free only " +
|
||||
"on the engine's own verdict, at most fifteen minutes old, that the account agents run as cannot become root " +
|
||||
"without a person, and the login shell's execute not served here. Anything else is no pass, and says why. " +
|
||||
"Answered by this machine's engine alone.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
{Name: "acts", Description: "Every act this machine's engine was asked through this seat (novox/hq ADR 0563 §9): " +
|
||||
"each hand-over, access act and setuid search, open, done or refused, with who asked, the ask and the " +
|
||||
"operator's warrant. Read-only.",
|
||||
Input: schema(map[string]string{"limit": "at most this many, newest", "id": "one act, by its id"}, nil)},
|
||||
// The packages this node's compositions declare (novox/hq ADR 0565 §6, §15): optional until every engine serves it.
|
||||
{Name: "packages", Optional: true, Description: "The packages this machine's compositions declare, present or absent, " +
|
||||
"each with the modules that declare it, as the engine keeps them in its key of `packages`; until the controller " +
|
||||
"hands its lease away, beside the controller's mirror for this machine and their difference. With write, the " +
|
||||
"key written again from what the engine composed.",
|
||||
Input: schema(map[string]string{"write": "\"true\": write this machine's key again from what the engine composed"}, nil, "write")},
|
||||
// Read-only (issue 774): why a module waits for a yes, or what its next apply would change. Optional until every
|
||||
// engine serves it.
|
||||
{Name: "composition", Optional: true, Description: "One module's composition on this machine beside what runs of " +
|
||||
"it, read-only: the composition its want's build has in the bucket composed-<node> (the latest revision, or the " +
|
||||
"one whose digest is given, from the revisions the bucket keeps), the digest of each kept revision, what runs " +
|
||||
"(applied, its have, or the kept declaration), its want and have (phase running, failed, held or waiting-yes, " +
|
||||
"and the digest that waits for a yes), and the move's judgement with its words: why a record module waits for " +
|
||||
"a yes. With diff, every resource added, removed or changed, a provider's received file naming the consumers " +
|
||||
"that join or leave.",
|
||||
Input: schema(map[string]string{"module": "the module, as placed on this machine; node for the node's own",
|
||||
"digest": "a composition's digest (sha256:…), to read that revision rather than the latest",
|
||||
"diff": "\"true\": list the resources that differ from what runs"}, []string{"module"})},
|
||||
// The recovery of a composition that failed (issue 774, 2026-10-31: ace kept route-proxy failed for a vault that
|
||||
// answered again a minute later). Optional until every engine serves it.
|
||||
{Name: "compose", Optional: true, Description: "Compose this machine's modules now and say how it went, so a " +
|
||||
"composition that failed, or a module held, recovers without a restart: what the pass refused or held, each " +
|
||||
"with why, and when the engine composes again on its own after a seat that did not answer (a deadline, no " +
|
||||
"responders: retried with a growing wait; a refusal never). With module, that module's apply is also tried " +
|
||||
"again. Changes only what a pass the engine makes on its own would change.",
|
||||
Input: schema(map[string]string{"module": "a module placed on this machine whose apply is also tried again (optional)"}, nil)},
|
||||
// A composition that does not exist yet (novox/hq ADR 0593 §7). Optional until every engine serves it: a node
|
||||
// whose engine does not is a node that does not answer (§9).
|
||||
{Name: "compose-candidate", Optional: true, Description: "Compose a check's candidate on this machine in shadow, " +
|
||||
"with its own sealed values and the indexer's manifest verdicts, and say per module placed here whether it " +
|
||||
"composes (with its digest and whether the candidate changes it), is refused by the candidate or is held " +
|
||||
"without it, each with why, and the verdict of the machine's own composition (novox/hq ADR 0593 §7). Read-only: " +
|
||||
"applies nothing and writes nothing. Answered by this machine's engine alone, within the asker's bound.",
|
||||
Input: schema(map[string]string{"candidate": "the key of the indexer's candidate, <group>.<heads hash>",
|
||||
"input": "\"true\": also answer the composition input the candidate was composed from, for the sandbox path"},
|
||||
[]string{"candidate"}, "input")},
|
||||
}
|
||||
}
|
||||
|
||||
// EngineEvents are what the node-engine seat says, as the seat list the host gave has them.
|
||||
func EngineEvents(seat string) []string {
|
||||
s, _ := SeatNamed(seat)
|
||||
return append([]string(nil), s.Emits...)
|
||||
// EngineEvents are what the seat says of an act, each naming the act, the path or access, the caller as the runtime
|
||||
// proved it and the ask (ADR 0563 §1).
|
||||
var EngineEvents = []string{"asked", "done", "refused"}
|
||||
|
||||
// engineSeat is the seat's row in the closed set.
|
||||
func engineSeat() Seat {
|
||||
return Seat{Name: EngineSeat, Scope: ScopeNode, Serves: EngineVerbs(), Emits: append([]string(nil), EngineEvents...),
|
||||
HeldByTheEngine: true, Decision: "novox/hq ADR 0563"}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package catalogue
|
||||
|
||||
// EnrolmentSeat is the seat mesh-enrolment.
|
||||
const EnrolmentSeat = "mesh-enrolment"
|
||||
|
||||
// The seat's six verbs (ADR 0587 §1, §4-§8).
|
||||
const (
|
||||
@@ -28,3 +29,62 @@ var EnrolmentEvents = []string{EnrolmentIssuedEvent, NodeEnrolledEvent, Enrolmen
|
||||
|
||||
// enrolmentDecision is the record that made the seat.
|
||||
const enrolmentDecision = "novox/hq ADR 0587"
|
||||
|
||||
// enrolmentVerbs are the seat's verbs: what reads all it holds and does by hand all it does (ADR 0587 §8). Not optional:
|
||||
// the seat is new with its holder, so no holder of before serves fewer.
|
||||
func enrolmentVerbs() []Verb {
|
||||
str := func(description string) map[string]any {
|
||||
return map[string]any{"type": "string", "description": description}
|
||||
}
|
||||
return []Verb{
|
||||
{Name: EnrolmentVerbIssue, Description: "Issue a one-time token for a new machine, on the operator's Approve: " +
|
||||
"refused at once, naming every finding, for a name that is not a node name or that a node record, a placement " +
|
||||
"key or an open enrolment holds, and for every call before the controller's lease is handed away. Otherwise " +
|
||||
"reserves the lowest free private-network address, asks the operator under the seat's own name and, on the " +
|
||||
"warrant, adds the node to placement with its tunnel key, asks the vault for the token and says enrolment-issued. " +
|
||||
"The token itself is read only through tokens.",
|
||||
Input: schema(map[string]string{
|
||||
"node": "the new machine's name in the mesh",
|
||||
"why": "why it joins, shown to the operator with the ask",
|
||||
"expires": "how long the token lasts, a duration such as 1h or 24h: one hour when not given, at most 24 hours",
|
||||
// The operator's decision of 2026-11-02 01:05: the tunnel key is given at issue, so on the Approve the node
|
||||
// enters placement with it and the hub peers it before redeem.
|
||||
"overlay_key": "the new machine's private network (tunnel) public key, as `nox-mesh-host key` prints it",
|
||||
}, []string{"node", "why", "overlay_key"})},
|
||||
{Name: EnrolmentVerbRedeem, Description: "Spend a token: asked by a joining node-engine dialled as enrol.<node> " +
|
||||
"alone, with the public halves of its keys and its identity key's proof over them. Spends the token, asks the " +
|
||||
"vault for the node's account and certificate, writes the node into placement, and answers the account sealed " +
|
||||
"to the node, its certificate and the vault's public half, signed by the vault. Asked again with the same keys " +
|
||||
"it finishes what is not done and answers the same; with other keys it is refused.",
|
||||
Input: map[string]any{"type": "object", "additionalProperties": false,
|
||||
"required": []string{"node", "public_key", "sealing_key", "proof"},
|
||||
"properties": map[string]any{
|
||||
"node": str("the node the token was issued for"),
|
||||
"public_key": str("the identity key's public half, base64"),
|
||||
"overlay_key": str("the private network key's public half"),
|
||||
"sealing_key": str("the sealing key's public half, which the account is sealed to"),
|
||||
"serving_key": str("the serving key's public half, which the certificate binds"),
|
||||
"profile": map[string]any{"type": "object", "description": "what the machine is, as its record says it"},
|
||||
"tunnel": map[string]any{"type": "object", "description": "a tunnel found on the machine: everything but its private key"},
|
||||
"proof": str("the identity key's signature over the redeem's proof bytes (delivery.RedeemProof), base64"),
|
||||
}}},
|
||||
{Name: EnrolmentVerbConfirm, Description: "Confirm an enrolment: asked by the node-engine dialled as its own " +
|
||||
"node login alone, the runtime's proof of that caller being the confirmation. Says node-enrolled and ends the " +
|
||||
"enrolment.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
{Name: EnrolmentVerbRevoke, Description: "End an enrolment at any stage, with why, without an Approve: withdraws " +
|
||||
"an open ask, deletes the node's key and its address, has the vault retire an account it already minted for " +
|
||||
"it, records the act as a hand act and says enrolment-revoked.",
|
||||
Input: schema(map[string]string{"node": "the node", "why": "why, recorded with the hand act"},
|
||||
[]string{"node", "why"})},
|
||||
{Name: EnrolmentVerbTokens, Description: "Read every open enrolment, or one: its stage, expiry, ask and revision, " +
|
||||
"the bucket's fill against its cap. For a node at live, its " +
|
||||
"token sealed to the operator key, asked of the vault at that moment and relayed as the vault sealed it. With " +
|
||||
"history, the ended enrolments from the seat's events, as far back as the events are kept.",
|
||||
Input: schema(map[string]string{"node": "one node", "history": "\"true\": the ended enrolments too"},
|
||||
nil, "history")},
|
||||
{Name: EnrolmentVerbAgain, Description: "Raise an issue's ask again when the operator's channel does not hold it, " +
|
||||
"and nothing more.",
|
||||
Input: schema(map[string]string{"node": "the node whose ask is raised again"}, []string{"node"})},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -35,9 +35,8 @@ func TestTheSeatTableCarriesTheEnvironmentAndTheLoginShell(t *testing.T) {
|
||||
if !ok || shell.Scope != ScopeNode || shell.Decision != "novox/hq ADR 0204" {
|
||||
t.Fatalf("node-login-shell is not a node seat: %+v (defined %v)", shell, ok)
|
||||
}
|
||||
// A-264, A-265: its marked execute gains the verb approvals beside it.
|
||||
if len(shell.Serves) != 2 || shell.Serves[0].Name != "execute" || shell.Serves[0].Description == "" || shell.Serves[1].Name != "approvals" {
|
||||
t.Fatalf("the login shell serves %+v, not execute and approvals", shell.Serves)
|
||||
if len(shell.Serves) != 1 || shell.Serves[0].Name != "execute" || shell.Serves[0].Description == "" {
|
||||
t.Fatalf("the login shell serves %+v, not execute alone", shell.Serves)
|
||||
}
|
||||
props, _ := shell.Serves[0].Input["properties"].(map[string]any)
|
||||
required, _ := shell.Serves[0].Input["required"].([]string)
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"strings"
|
||||
)
|
||||
|
||||
// FilesSeat is the role that reads and manages a node's files and accounts (novox/hq ADR 0394).
|
||||
const FilesSeat = "node-files"
|
||||
|
||||
@@ -12,3 +16,73 @@ const (
|
||||
"secret is kept, or a known credential file, is refused by name"
|
||||
overrideWord = "a reason, to change a path a module owns anyway (optional)"
|
||||
)
|
||||
|
||||
// filesVerbs is the contract every holder of node-files serves (novox/hq ADR 0394 §1): the reads of node-reads,
|
||||
// with their descriptions restated for the operator's account, and the writes.
|
||||
func filesVerbs() []Verb {
|
||||
var verbs []Verb
|
||||
for _, v := range readsVerbs() {
|
||||
v.Description = strings.Replace(v.Description,
|
||||
"Read as the holder's own account, which holds no secret: a component it may not reach says so.",
|
||||
"", 1)
|
||||
v.Description = strings.TrimSpace(v.Description) + " Read " + filesAsAccount + "."
|
||||
verbs = append(verbs, v)
|
||||
}
|
||||
write := func(name, description string, in map[string]string, required []string, optional []string, replaces ...string) Verb {
|
||||
in["reason"] = "why the file is changed, for the audit trail"
|
||||
in["override"] = overrideWord
|
||||
return Verb{Name: name, Description: description + " Refuses a path a module owns " +
|
||||
"unless override names a reason; done as the node's operator account, through sudo -n where its rights do not reach.",
|
||||
Input: schema(in, append(required, "reason"), append(optional, "override")...), Replaces: replaces}
|
||||
}
|
||||
return append(verbs,
|
||||
write("write", "Write one text file whole, replacing what it holds; the old content goes to the node's trash.",
|
||||
map[string]string{"path": "an absolute path", "content": "the text to write",
|
||||
"mode": "the file's mode, such as 0644 (optional)"},
|
||||
[]string{"path", "content"}, []string{"mode"}, "tee", ">"),
|
||||
write("copy", "Copy a file or directory to a path on this node. A target on another node (<node>:/path) is "+
|
||||
"refused: copying between nodes comes in step 2 (issue 903). An existing target is overwritten, the old one "+
|
||||
"kept in the node's trash for 7 days.",
|
||||
map[string]string{"from": "an absolute path", "to": "an absolute path on this node"},
|
||||
[]string{"from", "to"}, nil, "cp", "cp -a", "ssh <node> cp"),
|
||||
write("move", "Move or rename a file or directory. A target on another node is refused: copying between nodes "+
|
||||
"comes in step 2 (issue 903). An existing target is overwritten, the old one kept in the node's trash for 7 days.",
|
||||
map[string]string{"from": "an absolute path", "to": "an absolute path"},
|
||||
[]string{"from", "to"}, nil, "mv"),
|
||||
write("delete", "Delete a file or directory into the node's trash, where restore can bring it back.",
|
||||
map[string]string{"path": "an absolute path", "recursive": "\"true\": delete a directory with what it holds"},
|
||||
[]string{"path"}, []string{"recursive"}, "rm", "rm -r"),
|
||||
write("mkdir", "Make a directory.",
|
||||
map[string]string{"path": "an absolute path", "parents": "\"true\": make the missing parents too"},
|
||||
[]string{"path"}, []string{"parents"}, "mkdir", "mkdir -p"),
|
||||
write("chmod", "Change the mode of a file or directory.",
|
||||
map[string]string{"path": "an absolute path", "mode": "the mode, such as 0644 or u+x",
|
||||
"recursive": "\"true\": also everything under a directory"},
|
||||
[]string{"path", "mode"}, []string{"recursive"}, "chmod"),
|
||||
write("chown", "Change the owner and group of a file or directory.",
|
||||
map[string]string{"path": "an absolute path", "owner": "an account's name or number",
|
||||
"group": "a group's name or number (optional)", "recursive": "\"true\": also everything under a directory"},
|
||||
[]string{"path", "owner"}, []string{"group", "recursive"}, "chown", "chgrp"),
|
||||
Verb{Name: "upload", Description: "Send one file of this node into a bucket of the object store, over a short signed link " +
|
||||
"and never over the bus; answers the key (the file's SHA-256 when none is given), the size and the SHA-256. " +
|
||||
"At most 100 MiB. A place a secret is kept, a known credential file, or a link to one is refused by name, " +
|
||||
"before anything is sent. Read " + filesAsAccount + ".",
|
||||
Input: schema(map[string]string{"path": "an absolute path to a file", "bucket": "the bucket the file goes into",
|
||||
"key": "the object's key (optional; the file's SHA-256 when not given)"}, []string{"path", "bucket"})},
|
||||
// download is a plain Verb, not through write(): write() types its optional arguments as true/false
|
||||
// switches, which a hash and a reason are not (issue 1719).
|
||||
Verb{Name: "download", Description: "Write one object of a bucket to a file of this node, over a short signed link and " +
|
||||
"never over the bus; the old file goes to the node's trash. At most 100 MiB. The bytes must hash to sha256, or to " +
|
||||
"the key when the key is a SHA-256; otherwise no file is written. Refuses a path a module owns " +
|
||||
"unless override names a reason; done as the node's operator account, through sudo -n where its rights do not reach.",
|
||||
Input: schema(map[string]string{"bucket": "the bucket", "key": "the object's key", "path": "an absolute path",
|
||||
"sha256": "the SHA-256 the bytes must have (optional; the key when that is a SHA-256)",
|
||||
"reason": "why the file is changed, for the audit trail",
|
||||
"override": overrideWord}, []string{"bucket", "key", "path", "reason"})},
|
||||
Verb{Name: "trash", Description: "What the node's trash holds: each deleted or replaced item with its id, " +
|
||||
"original path and when it went.",
|
||||
Input: schema(map[string]string{}, nil), Replaces: []string{"ls ~/.local/share/Trash"}},
|
||||
write("restore", "Put an item of the node's trash back where it was; refuses when something is there now.",
|
||||
map[string]string{"id": "an id from trash"}, []string{"id"}, nil, "trash-restore"),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -23,6 +23,117 @@ const (
|
||||
SecretServiceSeat = "node-secret-service"
|
||||
)
|
||||
|
||||
// graphicalSessionSeats are the eleven, in the order ADR 0208's table reads, each with the verbs
|
||||
// research 026/05 starts it with. Three have none yet: the bar (which receives blocks instead,
|
||||
// ADR 0255), the compositor and the secret
|
||||
// service are roles a second holder competes for, and nothing has needed to ask them anything.
|
||||
func graphicalSessionSeats() []Seat {
|
||||
const decided = "novox/hq ADR 0208"
|
||||
return []Seat{
|
||||
{Name: LoginManagerSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
|
||||
{Name: "sessions", Description: "The sessions the login manager offers on this machine, and which " +
|
||||
"one the operator account starts by default.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
}},
|
||||
{Name: DisplayServerSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
|
||||
{Name: "displays", Description: "The monitors connected now, each with its identity, its modes and " +
|
||||
"where it is placed; and the layout profile in force, if one matches.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
// Profiles are keyed by the monitors' identities and are the operator's data (ADR 0208 §6).
|
||||
{Name: "layout", Description: "The monitor layout profiles, keyed by the connected monitors' " +
|
||||
"identities: list them, save the current arrangement under a name, or apply one.",
|
||||
Input: withEnum(schema(map[string]string{
|
||||
"action": "list, save or apply",
|
||||
"name": "the profile to save or apply (save and apply only)",
|
||||
}, []string{"action"}), "action", "list", "save", "apply")},
|
||||
}},
|
||||
// It receives window-manager configuration lines from every other module (novox/hq ADR 0212):
|
||||
// bindings, start-up commands, rules — placed by the session's holder, never written into
|
||||
// its directory by the contributor.
|
||||
{Name: DisplaySessionSeat, Scope: ScopeNode, Decision: decided,
|
||||
Receives: []Receivable{{Kind: "config", Comment: "#"}}, Serves: []Verb{
|
||||
{Name: "reload", Description: "Reload the session's configuration in place, keeping its windows.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
{Name: "workspaces", Description: "The session's workspaces: each one's name, output, and whether " +
|
||||
"it is visible or focused.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
{Name: "windows", Description: "The session's windows: each one's title, class, workspace and " +
|
||||
"whether it has focus; narrowed to one workspace when named.",
|
||||
Input: schema(map[string]string{"workspace": "one workspace (optional)"}, nil)},
|
||||
}},
|
||||
{Name: TerminalEmulatorSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
|
||||
{Name: "open", Description: "Open a terminal window in the operator's session, running a command " +
|
||||
"or the login shell, in a directory or the account's home.",
|
||||
Input: schema(map[string]string{
|
||||
"command": "what to run in it (optional; the login shell when absent)",
|
||||
"directory": "where it starts (optional; the account's home when absent)",
|
||||
}, nil)},
|
||||
}},
|
||||
{Name: LauncherSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
|
||||
{Name: "menu", Description: "Put a menu of choices in front of the operator and answer with the " +
|
||||
"one chosen, or nothing when the menu was dismissed — the dmenu-compatible contract.",
|
||||
Input: map[string]any{"type": "object", "required": []string{"choices"},
|
||||
"properties": map[string]any{
|
||||
"choices": map[string]any{"type": "array", "items": map[string]any{"type": "string"},
|
||||
"description": "the lines to choose between, in order"},
|
||||
"prompt": map[string]any{"type": "string", "description": "what the menu asks (optional)"},
|
||||
}}},
|
||||
// A value the operator types and nobody sees (novox/hq ADR 0259 §10): a hidden prompt whose answer
|
||||
// is sealed to the asker's key, so it is never plaintext on the bus or in any call's record.
|
||||
// **Optional while its holders catch up** (ADR 0246): rofi serves it once this is live.
|
||||
{Name: "secret", Optional: true, Description: "Ask the operator for a value in a prompt that " +
|
||||
"does not show what is typed, and answer it sealed to the key the asker gives — never in " +
|
||||
"the clear — or cancelled when the prompt was dismissed or not answered in time.",
|
||||
// By name, never by words (the review of 2026-10-09, M4): the holder writes the prompt from the
|
||||
// module, the secret and the machine, and says the controller asks — the bus lets nobody else
|
||||
// ask it (broker.ControllerOnly) — so no caller puts words of its own before the operator.
|
||||
Input: schema(map[string]string{
|
||||
"module": "the module whose own secret is asked for",
|
||||
"secret": "the own secret's name",
|
||||
"node": "the machine the module runs on",
|
||||
"seal_to": "the asker's public sealing key: the answer is sealed to it",
|
||||
"timeout_seconds": "give up after this long (optional)",
|
||||
}, []string{"module", "secret", "node", "seal_to"})},
|
||||
}},
|
||||
{Name: NotifierSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
|
||||
{Name: "send", Description: "Show the operator a notification.",
|
||||
Input: withEnum(schema(map[string]string{
|
||||
"title": "the notification's summary",
|
||||
"body": "its text (optional)",
|
||||
"urgency": "low, normal (the default) or critical",
|
||||
}, []string{"title"}), "urgency", "low", "normal", "critical")},
|
||||
{Name: "history", Description: "The notifications shown lately, newest first.",
|
||||
Input: schema(map[string]string{"limit": "how many (optional, default 20)"}, nil)},
|
||||
}},
|
||||
{Name: LockScreenSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
|
||||
{Name: "lock", Description: "Lock the operator's session now.",
|
||||
Input: schema(map[string]string{}, nil)},
|
||||
}},
|
||||
{Name: ClipboardSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
|
||||
{Name: "history", Description: "What the clipboard held lately, newest first.",
|
||||
Input: schema(map[string]string{"limit": "how many (optional, default 20)"}, nil)},
|
||||
{Name: "copy", Description: "Put text on the operator's clipboard.",
|
||||
Input: schema(map[string]string{"text": "the text"}, []string{"text"})},
|
||||
}},
|
||||
// It receives blocks as data (novox/hq ADR 0255): what a block shows, said for any bar, which
|
||||
// the holder renders in its own grammar. A module that knows something about the machine — the
|
||||
// power module its battery — gives the block; the bar knows no machine's hardware.
|
||||
{Name: BarSeat, Scope: ScopeNode, Decision: decided,
|
||||
Receives: []Receivable{{Kind: BarKindBlock, Comment: "#", Offered: true, Shape: barBlockShape()}}},
|
||||
{Name: CompositorSeat, Scope: ScopeNode, Decision: decided},
|
||||
{Name: SecretServiceSeat, Scope: ScopeNode, Decision: decided},
|
||||
}
|
||||
}
|
||||
|
||||
// withEnum narrows one string property of a schema to the values it may take, so a caller is told
|
||||
// the choices by the schema rather than by a refusal.
|
||||
func withEnum(s map[string]any, property string, values ...string) map[string]any {
|
||||
props := s["properties"].(map[string]any)
|
||||
p := props[property].(map[string]any)
|
||||
p["enum"] = values
|
||||
return s
|
||||
}
|
||||
|
||||
// BarKindBlock is the bar's one received kind: a block (novox/hq ADR 0255).
|
||||
const BarKindBlock = "block"
|
||||
|
||||
@@ -60,6 +171,34 @@ type barOption struct {
|
||||
required bool
|
||||
}
|
||||
|
||||
// barBlockShape is the shape of a block (novox/hq ADR 0255): which bar, where on it, in what order,
|
||||
// what it shows, and that kind's options.
|
||||
//
|
||||
// - bar: top or bottom;
|
||||
// - place: resources (beside the processor, memory and disks) or status (beside the sound, before the
|
||||
// clock) — the two places every holder keeps for contributed blocks;
|
||||
// - order: 0–99 within the place, 50 when absent;
|
||||
// - shows: battery or command; options: what that kind takes.
|
||||
func barBlockShape() *Shape {
|
||||
return &Shape{
|
||||
Text: true,
|
||||
Places: []PlaceField{{Field: "bar", Values: []string{"bottom", "top"}},
|
||||
{Field: "place", Values: []string{"resources", "status"}}},
|
||||
Order: "order",
|
||||
Check: barBlockProblems,
|
||||
Reads: barBlockReads,
|
||||
Examples: []map[string]any{
|
||||
{"bar": "bottom", "place": "status", "shows": "battery"},
|
||||
{"bar": "bottom", "place": "status", "shows": "battery", "options": map[string]any{"device": "BAT0"}},
|
||||
{"bar": "bottom", "place": "status", "shows": "command",
|
||||
"options": map[string]any{"command": "echo \"a line\"", "interval": float64(5)}},
|
||||
{"bar": "bottom", "place": "status", "shows": "state", "options": map[string]any{"state": "power.draw"}},
|
||||
{"bar": "bottom", "place": "status", "shows": "state",
|
||||
"options": map[string]any{"state": "power.draw", "key": "a-machine"}},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func barBlockProblems(data map[string]any) []string {
|
||||
var problems []string
|
||||
for field := range data {
|
||||
@@ -106,11 +245,3 @@ func barBlockProblems(data map[string]any) []string {
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// withShapeCode gives a data-shaped kind of the generated seat list the checks its definition cannot carry, being code:
|
||||
// the bar's block (novox/hq ADR 0255, ADR 0260).
|
||||
func withShapeCode(seat, kind string, sh *Shape) {
|
||||
if seat == BarSeat && kind == BarKindBlock {
|
||||
sh.Check, sh.Reads = barBlockProblems, barBlockReads
|
||||
}
|
||||
}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user