5 Commits
Author SHA1 Message Date
jochen 98778376ca Ask at most three at a time, wait out a refusal, need a router, and act only on a claimed open ask, as the review asked (hq ADR 0259)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.66s)
mesh/delivery rejected: the gate failed or could not run, or the repository's own check failed
mesh/delivery-group group feat/the-controller-asks-the-operator rejected: a member's own check failed
2026-10-08 18:37:30 +02:00
jochen 7ea2ba4ea7 Ask the operator for a condition's answers and act on the warrant, so release, stop, start and restart can be answered from any channel that proves who answered (hq ADR 0259) 2026-10-08 18:32:13 +02:00
jochen a1b7be8896 Serve a trusted holder from a runtime of its own account, refuse it in the machine's runtime, and say while an agent can become root where it runs (hq ADR 0259 §8)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery ready: it delivers once merged
mesh/delivery-group group feat/asks-answered-on-any-channel rejected: a member's own check failed
2026-10-08 18:31:31 +02:00
jochen eb62bb130a Carry a channel's capabilities on its claim and tell the router every kind, so an answer is judged by the controller's record and not the channel's word 2026-10-08 18:23:52 +02:00
jochen 01b796f90d Grant a seat's traffic by caller and by kind, so an ask's asker and a channel's kind are facts the bus enforces (hq ADR 0259) 2026-10-08 18:23:52 +02:00
46 changed files with 3764 additions and 53 deletions
+569
View File
@@ -0,0 +1,569 @@
package main
// The controller asks, and acts on the operator's warrant (novox/hq ADR 0259 §6). It holds no channel, no
// identity and no factor: it asks the router like any other module, and performs the answer chosen with its
// own grant.
//
// - **For every open, unsilenced condition that needs the operator and names its answers**, one ask is
// published on the `operator-channel` seat under the controller's own name: the condition's words, its
// actions as options at their levels (Silence acknowledges; Release, Stop, Start and Restart approve),
// answered by the operator, expiring after a day (a week when every option only acknowledges). A
// condition that clears, is silenced, or changes its answers has its ask cancelled; an ask that expired
// unanswered is asked again while the condition lasts. Each ask is kept in the controller's bucket
// `asked`, so a restart neither asks twice nor forgets.
// - **On a warrant**, heard on the seat's event under the controller's own name (which only the router may
// say), the controller acts once per ask: only for an ask it holds, only for the option it offered at
// that option's level, and only while the condition is still open. It performs the action as itself —
// a silence through its own conditions, any other through the verb the action names — with the warrant's
// words as its why, and records it in the hand-act log as the operator's decision, naming the channel,
// the ask and the proofs. An ask that ended without a choice is recorded and nothing is done.
// - **A warrant it missed** while away is read from the router's record of its asks, under its own name.
import (
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// The asker's name on the seat: the controller's module.
const askerName = broker.ControllerSeat
// How long an ask lasts: a day when an answer approves, a week when every answer only acknowledges.
const (
// askApproveFor is a day less a margin, so an ask is never refused at the router for lasting a day and
// a moment (the SDK's bound is a day).
askApproveFor = 24*time.Hour - 10*time.Minute
askAcknowledgeFor = 7 * 24 * time.Hour
// askEvery is how often what is open is asked about again, beside every change.
askEvery = time.Minute
// askCatchUpAfter is how old an open ask is before the router's record of it is read: a warrant heard
// on the event needs no reading.
askCatchUpAfter = 2 * time.Minute
// askAgainAfterAnswer is how long a condition the operator answered is not asked about again with the
// same answers: what was chosen takes a while to clear it, and asking again at once would ask twice.
askAgainAfterAnswer = time.Hour
// askMostOpen is how many asks the controller holds open at once (the router refuses a fourth): the
// most urgent conditions first, then the oldest.
askMostOpen = asks.MostOpen
)
// What became of an ask, as the controller keeps it.
const (
askOpen = "open"
askCancelled = "cancelled"
)
// asked is one ask the controller made, as it keeps it.
type asked struct {
ID string `json:"id"`
Condition string `json:"condition"`
// Channels is what the channels were when it was asked (asker.channels): an ask the router refused is not
// asked again until the condition's answers or the channels change.
Channels string `json:"channels,omitempty"`
Ask asks.Ask `json:"ask"`
Actions []conditions.Action `json:"actions"`
// Options are the actions by option id.
Options map[string]int `json:"options"`
State string `json:"state"`
Opened time.Time `json:"opened"`
Ended time.Time `json:"ended,omitempty"`
Warrant *asks.Warrant `json:"warrant,omitempty"`
// Acted is what the controller did on the warrant: empty before it did anything, "acting" while it acts,
// then "done", "failed: …" or "nothing: …". Anything but empty is never acted on again.
Acted string `json:"acted,omitempty"`
}
// askedStore keeps the asks (broker.AskedBucket).
type askedStore interface {
Get(ctx context.Context, id string) (*asked, error)
Put(ctx context.Context, a asked) error
All(ctx context.Context) ([]asked, error)
// Claim marks an open ask acting, by compare-and-set, and says whether this write stood: of two
// deliveries of one warrant, or two controllers, only the one whose write stands acts.
Claim(ctx context.Context, id string, w asks.Warrant) (bool, error)
}
// asker is the controller asking the operator and acting on the answer.
type asker struct {
open func(ctx context.Context) ([]conditions.Condition, error)
silence func(ctx context.Context, key string, d time.Duration, by, why string) error
store askedStore
// publish puts a message on a subject's stream, de-duplicated by id.
publish func(ctx context.Context, subject string, body []byte, id string) error
// call performs an action's verb with its arguments, as the controller.
call func(ctx context.Context, a conditions.Action, args map[string]string) error
// record writes the hand-act log.
record func(ctx context.Context, act link.HandAct) error
// routerRecord reads the router's record of an ask for a warrant missed; nil reads nothing.
routerRecord func(ctx context.Context, id string) (*asks.Warrant, error)
// routerHere says whether a router holds the seat and takes asks under the asker's name; nil is yes.
routerHere func(ctx context.Context) (bool, error)
// channels is what the channels are now, as a fingerprint: who holds which kind, promising what.
channels func(ctx context.Context) string
now func() time.Time
logf func(string, ...any)
saidNoRouter bool
mu sync.Mutex
nudged chan struct{}
}
func (a *asker) nudge() {
if a == nil {
return
}
a.mu.Lock()
if a.nudged == nil {
a.nudged = make(chan struct{}, 1)
}
ch := a.nudged
a.mu.Unlock()
select {
case ch <- struct{}{}:
default:
}
}
// keep asks until ctx ends: now, on every change of a condition, and every askEvery.
func (a *asker) keep(ctx context.Context) {
a.nudge()
tick := time.NewTicker(askEvery)
defer tick.Stop()
a.mu.Lock()
nudged := a.nudged
a.mu.Unlock()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
case <-nudged:
}
if err := a.reconcile(ctx); err != nil {
a.logf("what the operator is asked could not be brought up to date: %v", err)
}
}
}
// wants says whether a condition is one to ask about now.
func wants(c conditions.Condition, now time.Time) bool {
return len(c.Actions) > 0 && c.Needs != "" && !c.SilencedAt(now)
}
func sameAsked(a []conditions.Action, b []conditions.Action) bool {
x, _ := json.Marshal(a)
y, _ := json.Marshal(b)
return string(x) == string(y)
}
// reconcile brings what is asked in line with what is open.
func (a *asker) reconcile(ctx context.Context) error {
now := a.now()
if a.routerHere != nil {
here, err := a.routerHere(ctx)
if err != nil {
return err
}
if !here {
if !a.saidNoRouter {
a.logf("no router takes asks under the controller's name (a module declaring %s with its ask "+
"named by its caller, assigned): the operator is asked nothing until one is", broker.AsksSeat)
a.saidNoRouter = true
}
return nil
}
a.saidNoRouter = false
}
channels := ""
if a.channels != nil {
channels = a.channels(ctx)
}
open, err := a.open(ctx)
if err != nil {
return err
}
all, err := a.store.All(ctx)
if err != nil {
return err
}
byCondition := map[string]asked{}
for _, r := range all {
if r.State == askOpen {
if prior, held := byCondition[r.Condition]; !held || r.Opened.After(prior.Opened) {
byCondition[r.Condition] = r
}
}
}
// A warrant missed while away, read from the router's record.
if a.routerRecord != nil {
for _, r := range byCondition {
if now.Sub(r.Opened) < askCatchUpAfter {
continue
}
if w, err := a.routerRecord(ctx, r.ID); err == nil && w != nil {
body, _ := json.Marshal(w)
if err := a.Decided(ctx, body); err != nil {
return err
}
}
}
if all, err = a.store.All(ctx); err != nil {
return err
}
byCondition = map[string]asked{}
for _, r := range all {
if r.State == askOpen {
byCondition[r.Condition] = r
}
}
}
// What the operator answered lately, by condition: not asked again at once; and what the router refused,
// newest first: not asked again until the answers or the channels change.
answered, refused := map[string]asked{}, map[string]asked{}
for _, r := range all {
if r.State == string(asks.OutcomeChosen) && now.Sub(r.Ended) < askAgainAfterAnswer {
answered[r.Condition] = r
}
if r.State == string(asks.OutcomeRefused) {
if prior, has := refused[r.Condition]; !has || r.Opened.After(prior.Opened) {
refused[r.Condition] = r
}
}
}
wanted := map[string]bool{}
// The most urgent first, then the oldest: those are asked when no more than askMostOpen may be.
sort.SliceStable(open, func(i, j int) bool {
ui, uj := open[i].Severity == conditions.Urgent, open[j].Severity == conditions.Urgent
if ui != uj {
return ui
}
if !open[i].Raised.Equal(open[j].Raised) {
return open[i].Raised.Before(open[j].Raised)
}
return open[i].Key < open[j].Key
})
openNow := 0
for _, c := range open {
if r, held := byCondition[c.Key]; held && wants(c, now) && sameAsked(r.Actions, c.Actions) && now.Before(r.Ask.Expires) {
openNow++
}
}
for _, c := range open {
if !wants(c, now) {
continue
}
wanted[c.Key] = true
if r, was := refused[c.Key]; was && sameAsked(r.Actions, c.Actions) && r.Channels == channels {
if _, held := byCondition[c.Key]; !held {
continue // refused, and nothing it was refused for has changed
}
}
if r, done := answered[c.Key]; done && sameAsked(r.Actions, c.Actions) {
if _, held := byCondition[c.Key]; !held {
continue
}
}
if r, held := byCondition[c.Key]; held {
switch {
case !sameAsked(r.Actions, c.Actions):
if err := a.cancel(ctx, r, "its answers changed"); err != nil {
return err
}
case !now.Before(r.Ask.Expires):
// Expired unanswered: the router says so too; asked again below while it lasts.
r.State, r.Ended = string(asks.OutcomeExpired), now
if err := a.store.Put(ctx, r); err != nil {
return err
}
openNow--
default:
continue
}
}
if openNow >= askMostOpen {
continue // asked when one of the open ones ends, most urgent first
}
if err := a.ask(ctx, c, channels); err != nil {
a.logf("the operator could not be asked about %s: %v", c.Key, err)
continue
}
openNow++
}
for key, r := range byCondition {
if !wanted[key] {
if err := a.cancel(ctx, r, "the condition ended, was silenced or needs nothing now"); err != nil {
return err
}
}
}
return nil
}
// optionID is an action's label as an option's id: "Silence for a week" is silence-for-a-week.
func optionID(label string) string {
var b strings.Builder
dash := false
for _, r := range strings.ToLower(label) {
switch {
case r >= 'a' && r <= 'z', r >= '0' && r <= '9':
b.WriteRune(r)
dash = false
case !dash && b.Len() > 0:
b.WriteByte('-')
dash = true
}
}
return strings.TrimSuffix(b.String(), "-")
}
// doesWords is what an action does, in the words an option says it with.
func doesWords(act conditions.Action) string {
switch {
case act.Arguments["silence"] != "":
return "nothing more is said of it for a week"
case act.Verb == "mesh-delivery.release":
return "the delivery goes on"
case act.Verb == "mesh-delivery.stop":
return "the delivery ends"
case act.Verb == broker.ControllerSeat+".plans" && act.Arguments["go"] != "":
return "the delivery starts"
case act.Verb == broker.ControllerSeat+".plans" && act.Arguments["stop"] != "":
return "the delivery is stopped"
case strings.HasSuffix(act.Verb, ".restart"):
return "its service is restarted on " + act.Machine
}
return strings.ToLower(act.Label)
}
// askText is a condition's words as an ask says them: without where an answer is given when no channel can
// give it (FromMeshMCPServer), since the ask is answered on a channel and the router says where else.
func askText(s string) string {
for _, with := range []string{", " + FromMeshMCPServer, " " + FromMeshMCPServer} {
s = strings.ReplaceAll(s, with, ".")
}
return strings.ReplaceAll(s, "..", ".")
}
// askOf is the ask a condition is asked with.
func askOf(id string, c conditions.Condition, now time.Time) (asks.Ask, map[string]int) {
q := asks.Ask{ID: id, Headline: c.Headline, Explanation: askText(c.Explanation), Who: asks.Operator,
OnExpiry: "nothing is done, and you are asked again while it lasts", About: c.Key,
Urgent: c.Severity == conditions.Urgent}
options := map[string]int{}
approves := false
for i, act := range c.Actions {
level := asks.Level(act.Level)
if level == "" {
level = asks.Approve // an action that says nothing of its level is never taken for less
}
approves = approves || level != asks.Acknowledge
oid := optionID(act.Label)
options[oid] = i
q.Options = append(q.Options, asks.Option{ID: oid, Label: act.Label, Does: doesWords(act), Level: level})
}
q.Expires = now.Add(askAcknowledgeFor)
if approves {
q.Expires = now.Add(askApproveFor)
}
return q, options
}
func newAskID() string {
var b [8]byte
_, _ = rand.Read(b[:])
return "c" + hex.EncodeToString(b[:])
}
// ask publishes one ask about a condition, and keeps it.
func (a *asker) ask(ctx context.Context, c conditions.Condition, channels string) error {
now := a.now()
id := newAskID()
q, options := askOf(id, c, now)
if err := q.Check(now); err != nil {
return err
}
body, err := json.Marshal(q)
if err != nil {
return err
}
if err := a.publish(ctx, asks.AskSubject(askerName), body, "ask."+id); err != nil {
return err
}
a.logf("asked the operator about %s (%s): %d answer(s)", c.Key, id, len(q.Options))
return a.store.Put(ctx, asked{ID: id, Condition: c.Key, Ask: q, Actions: c.Actions, Options: options,
State: askOpen, Opened: now, Channels: channels})
}
// cancel takes an ask back.
func (a *asker) cancel(ctx context.Context, r asked, why string) error {
body, _ := json.Marshal(map[string]string{"id": r.ID})
if err := a.publish(ctx, asks.CancelSubject(askerName), body, "cancel."+r.ID); err != nil {
a.logf("the ask %s about %s could not be taken back (%v); taken back at the next look", r.ID, r.Condition, err)
return nil
}
a.logf("took back the ask %s about %s: %s", r.ID, r.Condition, why)
r.State, r.Ended = askCancelled, a.now()
return a.store.Put(ctx, r)
}
// Decided takes the router's word on one of the controller's asks (link.Decider). An error is returned only
// when what was decided could not be kept, so the word is held and heard again.
func (a *asker) Decided(ctx context.Context, body []byte) error {
var w asks.Warrant
if err := json.Unmarshal(body, &w); err != nil {
a.logf("the router's word on an ask could not be read; ignored: %v", err)
return nil
}
if w.Asker != askerName {
a.logf("REFUSED a warrant for %s's ask %s: the controller acts only on its own", w.Asker, w.Ask)
return nil
}
r, err := a.store.Get(ctx, w.Ask)
if err != nil {
return err
}
if r == nil {
a.logf("REFUSED a warrant for the ask %s, which the controller does not hold", w.Ask)
return nil
}
if r.Acted != "" {
return nil // heard again: acted on once
}
now := a.now()
if w.Outcome != asks.OutcomeChosen {
r.State, r.Ended, r.Warrant = string(w.Outcome), now, &w
r.Acted = "nothing: the ask " + string(w.Outcome)
if w.Words != "" {
r.Acted += ": " + w.Words
}
a.logf("the ask %s about %s ended %s; nothing is done", r.ID, r.Condition, w.Outcome)
return a.store.Put(ctx, *r)
}
if r.State != askOpen {
// Cancelled, replaced or expired in the controller's own record: no answer to it is acted on.
a.logf("REFUSED a warrant for the ask %s, which is %s in the controller's own record", r.ID, r.State)
return nil
}
option, err := w.For(askerName, r.Ask)
if err != nil {
a.logf("REFUSED a warrant for the ask %s: %v", r.ID, err)
return nil
}
index, offered := r.Options[option.ID]
if !offered || index >= len(r.Actions) {
a.logf("REFUSED a warrant for the ask %s: it chose %s, which no action stands for", r.ID, option.ID)
return nil
}
act := r.Actions[index]
r.State, r.Warrant = string(asks.OutcomeChosen), &w
open, err := a.open(ctx)
if err != nil {
return err
}
stillOpen := false
for _, c := range open {
stillOpen = stillOpen || c.Key == r.Condition
}
if !stillOpen {
// The asker checks the state is still what it asked about before it acts (to-be 46 §10, step 7).
r.Ended, r.Acted = now, "nothing: the condition ended before the answer"
a.logf("%s, for %s, which ended meanwhile: nothing is done", w.Says(), r.Condition)
return a.store.Put(ctx, *r)
}
// Claimed before acting, by compare-and-set: only the delivery whose write stands acts (security review
// of 2026-10-08, finding 9).
claimed, err := a.store.Claim(ctx, r.ID, w)
if err != nil {
return err
}
if !claimed {
a.logf("the warrant for the ask %s was already taken by another delivery; nothing more is done", r.ID)
return nil
}
r.Acted = "acting"
why := fmt.Sprintf("%s (ask %s)", w.Says(), r.ID)
args := map[string]string{}
for k, v := range act.Arguments {
args[k] = v
}
if v, takes := args["why"]; takes && v == "" {
args["why"] = why
}
var acted error
if act.Arguments["silence"] != "" {
acted = a.silence(ctx, act.Arguments["silence"], conditions.MaxSilence, byWords(w), why)
} else {
acted = a.call(ctx, act, args)
}
r.Ended = a.now()
r.Acted = "done"
if acted != nil {
r.Acted = "failed: " + acted.Error()
}
if err := a.store.Put(ctx, *r); err != nil {
return err
}
verbArgs := []string{act.Verb}
if act.Machine != "" {
verbArgs = append(verbArgs, "on "+act.Machine)
}
keys := make([]string, 0, len(args))
for k := range args {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
if k != "why" {
verbArgs = append(verbArgs, k+"="+args[k])
}
}
if err := a.record(ctx, link.HandAct{Verb: handActWarrant, Args: verbArgs, Why: why, By: byWords(w),
Cause: conditions.CauseOperatorAnswer, Condition: r.Condition, Via: viaWords(w), Ask: r.ID,
Proofs: w.Proofs, RequestedBy: r.Condition, Outcome: r.Acted}); err != nil {
a.logf("%s was done, and could NOT be recorded in the hand-act log: %v", why, err)
}
a.logf("%s: %s", why, r.Acted)
return nil
}
// handActWarrant is the verb an act the operator chose on a warrant is recorded under: a person's decision,
// never a repair (handActVerbs).
const handActWarrant = "warrant"
// byWords is who chose, as the hand-act log says it: "the operator, as telegram identity 42".
func byWords(w asks.Warrant) string {
if w.By == nil {
return "the operator"
}
return fmt.Sprintf("the %s, as %s identity %s", w.By.Who, w.By.Kind, w.By.Identity)
}
// viaWords is the channel an answer came through: its module and kind, and how the sender was known.
func viaWords(w asks.Warrant) string {
if w.By == nil {
return w.Channel
}
via := w.Channel + " (" + w.By.Kind + ")"
if w.By.Verified != "" {
via += ", " + w.By.Verified
}
return via
}
// errNotGranted is an action whose verb the controller's grant does not name.
var errNotGranted = errors.New("the controller's grant does not name this verb")
+476
View File
@@ -0,0 +1,476 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// novox/hq ADR 0259 §6: the controller asks the operator for the answers its conditions name, and performs
// the one chosen on the router's warrant — once, for its own ask, the option offered, at its level.
type memAskedStore map[string]asked
func (m memAskedStore) Get(_ context.Context, id string) (*asked, error) {
r, ok := m[id]
if !ok {
return nil, nil
}
return &r, nil
}
func (m memAskedStore) Put(_ context.Context, r asked) error { m[r.ID] = r; return nil }
func (m memAskedStore) Claim(_ context.Context, id string, w asks.Warrant) (bool, error) {
r, ok := m[id]
if !ok || r.State != askOpen || r.Acted != "" {
return false, nil
}
r.State, r.Warrant, r.Acted = string(asks.OutcomeChosen), &w, "acting"
m[id] = r
return true, nil
}
func (m memAskedStore) All(context.Context) ([]asked, error) {
var out []asked
for _, r := range m {
out = append(out, r)
}
return out, nil
}
type published struct {
subject, id string
body []byte
}
type askerRig struct {
a *asker
open []conditions.Condition
store memAskedStore
sent []published
called []string
silenced []string
acts []link.HandAct
now time.Time
}
func newAskerRig(t *testing.T) *askerRig {
r := &askerRig{store: memAskedStore{}, now: time.Date(2026, 10, 8, 14, 0, 0, 0, time.UTC)}
r.a = &asker{
open: func(context.Context) ([]conditions.Condition, error) { return r.open, nil },
silence: func(_ context.Context, key string, d time.Duration, by, why string) error {
r.silenced = append(r.silenced, key+" for "+d.String()+" by "+by+" because "+why)
return nil
},
store: r.store,
publish: func(_ context.Context, subject string, body []byte, id string) error {
r.sent = append(r.sent, published{subject, id, body})
return nil
},
call: func(_ context.Context, a conditions.Action, args map[string]string) error {
raw, _ := json.Marshal(args)
r.called = append(r.called, a.Verb+"@"+a.Machine+" "+string(raw))
return nil
},
record: func(_ context.Context, act link.HandAct) error { r.acts = append(r.acts, act); return nil },
now: func() time.Time { return r.now },
logf: t.Logf,
}
return r
}
func heldCondition() conditions.Condition {
o := stalledObservations([]stalledLine{{ID: "novox/hq@055550802096", State: "held", For: "36h2m6s",
Bound: "24h0m0s", H2: "none: the state is the operator's"}})[0]
return conditions.Condition{Key: o.Key(), Kind: o.Kind, Severity: conditions.Warning, Headline: o.Headline,
Explanation: conditions.Verdict(o.Needs, o.Explanation), Needs: o.Needs, Actions: o.Actions}
}
func unitsCondition() conditions.Condition {
key := "machine.shanks.units"
return conditions.Condition{Key: key, Kind: "machine-units", Severity: conditions.Warning,
Headline: "3 failed services on shanks", Explanation: "Needs you: mend or remove them on shanks, or silence this.",
Needs: "mend or remove them on shanks, or silence this.", Actions: []conditions.Action{conditions.SilenceAction(key)}}
}
func (r *askerRig) asksSent(t *testing.T) []asks.Ask {
t.Helper()
var out []asks.Ask
for _, p := range r.sent {
if p.subject != asks.AskSubject("mesh-controller") {
continue
}
var q asks.Ask
if err := json.Unmarshal(p.body, &q); err != nil {
t.Fatal(err)
}
out = append(out, q)
}
return out
}
func TestAnAskIsMadeForEachConditionThatNamesItsAnswers(t *testing.T) {
r := newAskerRig(t)
quiet := conditions.Condition{Key: "machine.ace.silent", Headline: "ace silent", Explanation: "Nothing for you to do. x"}
r.open = []conditions.Condition{heldCondition(), unitsCondition(), quiet}
if err := r.a.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
sent := r.asksSent(t)
if len(sent) != 2 {
t.Fatalf("asked %d times: %+v", len(sent), sent)
}
byAbout := map[string]asks.Ask{}
for _, q := range sent {
byAbout[q.About] = q
if err := q.Check(r.now); err != nil {
t.Errorf("%s: %v", q.About, err)
}
}
held := byAbout[heldCondition().Key]
if len(held.Options) != 2 || held.Options[0].Label != "Release" || held.Options[0].Level != asks.Approve ||
held.Options[1].ID != "stop" || held.Expires != r.now.Add(askApproveFor) || held.Who != asks.Operator ||
held.OnExpiry == "" {
t.Errorf("the held delivery is asked %+v", held)
}
units := byAbout["machine.shanks.units"]
if len(units.Options) != 1 || units.Options[0].Level != asks.Acknowledge || units.Expires != r.now.Add(askAcknowledgeFor) {
t.Errorf("the failed units are asked %+v", units)
}
// No second ask while one is open.
r.now = r.now.Add(time.Minute)
_ = r.a.reconcile(context.Background())
if n := len(r.asksSent(t)); n != 2 {
t.Errorf("asked again while open: %d", n)
}
}
func TestAnAskIsTakenBackWhenItsConditionEndsAndAskedAgainAfterItExpires(t *testing.T) {
r := newAskerRig(t)
r.open = []conditions.Condition{heldCondition(), unitsCondition()}
_ = r.a.reconcile(context.Background())
// The units are silenced, the held delivery lasts past its ask's day.
units := unitsCondition()
units.Silenced = &conditions.Silence{Until: r.now.Add(48 * time.Hour)}
r.open = []conditions.Condition{heldCondition(), units}
r.now = r.now.Add(askApproveFor)
if err := r.a.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
var cancels int
for _, p := range r.sent {
if p.subject == asks.CancelSubject("mesh-controller") {
cancels++
}
}
if cancels != 1 {
t.Errorf("cancels %d, want the silenced one's", cancels)
}
if sent := r.asksSent(t); len(sent) != 3 || sent[2].About != heldCondition().Key {
t.Errorf("the expired ask was not asked again: %+v", sent)
}
}
// warrantFor is the router's warrant for the open ask about a condition, choosing an option by label.
func (r *askerRig) warrantFor(t *testing.T, condition, label string) asks.Warrant {
t.Helper()
for _, a := range r.store {
if a.Condition != condition || a.State != askOpen {
continue
}
for _, o := range a.Ask.Options {
if o.Label == label {
return asks.Warrant{Ask: a.ID, Asker: "mesh-controller", About: condition, Outcome: asks.OutcomeChosen,
Option: o.ID, Label: o.Label, Level: o.Level, Channel: "telegram", Proofs: []string{"P1"}, At: r.now,
By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"}}
}
}
}
t.Fatalf("no open ask about %s offers %s", condition, label)
return asks.Warrant{}
}
func answerWith(t *testing.T, r *askerRig, w asks.Warrant) {
t.Helper()
body, _ := json.Marshal(w)
if err := r.a.Decided(context.Background(), body); err != nil {
t.Fatal(err)
}
}
func TestAWarrantIsActedOnOnce(t *testing.T) {
r := newAskerRig(t)
r.open = []conditions.Condition{heldCondition()}
_ = r.a.reconcile(context.Background())
w := r.warrantFor(t, heldCondition().Key, "Release")
answerWith(t, r, w)
answerWith(t, r, w) // heard again
if len(r.called) != 1 {
t.Fatalf("called %v", r.called)
}
want := `mesh-delivery.release@ {"id":"novox/hq@055550802096","why":"the operator, via telegram (user id verified), chose Release (ask ` + w.Ask + `)"}`
if r.called[0] != want {
t.Errorf("called\n %s\nwant\n %s", r.called[0], want)
}
if len(r.acts) != 1 {
t.Fatalf("hand-acts %+v", r.acts)
}
act := r.acts[0]
if act.Verb != handActWarrant || act.By != "the operator, as telegram identity 42" ||
act.Via != "telegram (telegram), user id verified" || act.Ask != w.Ask || strings.Join(act.Proofs, ",") != "P1" ||
act.Cause != conditions.CauseOperatorAnswer || act.Condition != heldCondition().Key || act.Outcome != "done" {
t.Errorf("the hand-act %+v", act)
}
if !personsDecision(act) {
t.Error("an act on a warrant counts as a repair")
}
if got := r.store[w.Ask]; got.State != string(asks.OutcomeChosen) || got.Acted != "done" {
t.Errorf("kept %+v", got)
}
}
func TestAWarrantThatIsNotForItsOwnAskIsRefused(t *testing.T) {
for name, change := range map[string]func(*asks.Warrant){
"another asker": func(w *asks.Warrant) { w.Asker = "mesh-delivery" },
"an ask not held": func(w *asks.Warrant) { w.Ask = "c0000000000000000" },
"an option not offered": func(w *asks.Warrant) { w.Option = "delete" },
"another level": func(w *asks.Warrant) { w.Level = asks.Acknowledge },
"no person": func(w *asks.Warrant) { w.By = nil },
} {
t.Run(name, func(t *testing.T) {
r := newAskerRig(t)
r.open = []conditions.Condition{heldCondition()}
_ = r.a.reconcile(context.Background())
w := r.warrantFor(t, heldCondition().Key, "Stop")
change(&w)
answerWith(t, r, w)
if len(r.called)+len(r.acts)+len(r.silenced) != 0 {
t.Errorf("acted on it: %v %v %v", r.called, r.acts, r.silenced)
}
})
}
}
func TestEachAnswerCallsExactlyItsVerb(t *testing.T) {
plan := "plan-1791454185265004861"
waiting := conditions.Condition{Key: "plan." + plan + ".waiting", Severity: conditions.Urgent,
Headline: "openrazer delivery waiting to start", Needs: "start it, or stop it.",
Explanation: "Needs you: start it, or stop it.", Actions: waitingActions(plan, conditions.Urgent)}
module := conditions.Condition{Key: "module.openrazer.g14.unhealthy", Severity: conditions.Warning,
Headline: "openrazer not working on g14", Needs: "restart its service openrazer-daemon on g14.",
Explanation: "Needs you: restart it.", Actions: []conditions.Action{{Label: "Restart",
Verb: "node-service-manager.restart", Machine: "g14", Level: conditions.LevelApprove,
Arguments: map[string]string{"unit": "openrazer-daemon.service", "scope": "user"}}}}
for _, tc := range []struct {
c conditions.Condition
label string
want string
}{
{waiting, "Start", `mesh-controller.plans@ {"cause":"operator-answer","go":"` + plan + `","why":"`},
{waiting, "Stop", `mesh-controller.plans@ {"cause":"operator-answer","stop":"` + plan + `","why":"`},
{module, "Restart", `node-service-manager.restart@g14 {"scope":"user","unit":"openrazer-daemon.service"}`},
} {
r := newAskerRig(t)
r.open = []conditions.Condition{tc.c}
_ = r.a.reconcile(context.Background())
answerWith(t, r, r.warrantFor(t, tc.c.Key, tc.label))
if len(r.called) != 1 || !strings.HasPrefix(r.called[0], tc.want) {
t.Errorf("%s: called %v, want %s…", tc.label, r.called, tc.want)
}
}
}
func TestASilenceChosenIsTheControllersOwnAndAnAnswerToAnAsk(t *testing.T) {
r := newAskerRig(t)
r.open = []conditions.Condition{unitsCondition()}
_ = r.a.reconcile(context.Background())
w := r.warrantFor(t, "machine.shanks.units", "Silence for a week")
w.Level, w.Proofs = asks.Acknowledge, nil
w.By = &asks.Person{Who: asks.Operator, Kind: "desktop", Identity: "g14",
Verified: "a desk click: whoever was at the operator's session on g14"}
w.Channel = "desk-channel"
answerWith(t, r, w)
if len(r.called) != 0 || len(r.silenced) != 1 || !strings.HasPrefix(r.silenced[0], "machine.shanks.units for 168h0m0s by the operator, as desktop identity g14") {
t.Fatalf("silenced %v, called %v", r.silenced, r.called)
}
if len(r.acts) != 1 || r.acts[0].Cause != conditions.CauseOperatorAnswer || len(r.acts[0].Proofs) != 0 {
t.Errorf("%+v", r.acts)
}
}
func TestAnAskThatEndedWithoutAChoiceDoesNothing(t *testing.T) {
r := newAskerRig(t)
r.open = []conditions.Condition{heldCondition()}
_ = r.a.reconcile(context.Background())
w := r.warrantFor(t, heldCondition().Key, "Release")
w.Outcome, w.Option, w.Label, w.Level, w.By, w.Words = asks.OutcomeExpired, "", "", "", nil, "nobody answered in time"
answerWith(t, r, w)
if len(r.called)+len(r.acts) != 0 || r.store[w.Ask].State != string(asks.OutcomeExpired) ||
!strings.HasPrefix(r.store[w.Ask].Acted, "nothing") {
t.Errorf("called %v acts %v kept %+v", r.called, r.acts, r.store[w.Ask])
}
// And a choice for a condition that ended meanwhile does nothing either.
r2 := newAskerRig(t)
r2.open = []conditions.Condition{heldCondition()}
_ = r2.a.reconcile(context.Background())
w2 := r2.warrantFor(t, heldCondition().Key, "Release")
r2.open = nil
answerWith(t, r2, w2)
if len(r2.called) != 0 || r2.store[w2.Ask].Acted != "nothing: the condition ended before the answer" {
t.Errorf("%v %+v", r2.called, r2.store[w2.Ask])
}
}
func TestAWarrantMissedWhileAwayIsReadFromTheRoutersRecord(t *testing.T) {
r := newAskerRig(t)
r.open = []conditions.Condition{heldCondition()}
_ = r.a.reconcile(context.Background())
w := r.warrantFor(t, heldCondition().Key, "Stop")
r.a.routerRecord = func(_ context.Context, id string) (*asks.Warrant, error) {
if id != w.Ask {
return nil, errors.New("another ask")
}
return &w, nil
}
r.now = r.now.Add(askCatchUpAfter)
if err := r.a.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
if len(r.called) != 1 || !strings.HasPrefix(r.called[0], "mesh-delivery.stop@") {
t.Errorf("called %v", r.called)
}
if n := len(r.asksSent(t)); n != 1 {
t.Errorf("asked again after the answer: %d", n)
}
}
// After review (2026-10-08): a refused ask is not asked again until its answers or the channels change.
func TestAnAskTheRouterRefusedWaitsUntilSomethingChanges(t *testing.T) {
r := newAskerRig(t)
channels := "channel/telegram=telegram@anchor[choice]own:true"
r.a.channels = func(context.Context) string { return channels }
r.open = []conditions.Condition{heldCondition()}
_ = r.a.reconcile(context.Background())
first := r.asksSent(t)[0]
refusal, _ := json.Marshal(asks.Warrant{Ask: first.ID, Asker: "mesh-controller", Outcome: asks.OutcomeRefused,
Words: "no channel can carry any of its answers now", At: r.now})
if err := r.a.Decided(context.Background(), refusal); err != nil {
t.Fatal(err)
}
if got := r.store[first.ID]; got.State != string(asks.OutcomeRefused) || !strings.Contains(got.Acted, "nothing") {
t.Fatalf("the refusal was kept as %+v", got)
}
for i := 0; i < 3; i++ {
r.now = r.now.Add(askEvery)
_ = r.a.reconcile(context.Background())
}
if n := len(r.asksSent(t)); n != 1 {
t.Fatalf("asked again %d time(s) though nothing changed", n-1)
}
channels = "channel/telegram=telegram@anchor[choice,verified-sender]own:true"
_ = r.a.reconcile(context.Background())
if n := len(r.asksSent(t)); n != 2 {
t.Errorf("not asked again once the channels changed: %d", n)
}
}
// After review: at most three asks open at once, the most urgent first, then the oldest.
func TestAtMostThreeAsksAreOpenTheMostUrgentFirst(t *testing.T) {
r := newAskerRig(t)
var open []conditions.Condition
for i := 0; i < 4; i++ {
c := unitsCondition()
c.Key = "machine.m" + string(rune('a'+i)) + ".units"
c.Actions = []conditions.Action{conditions.SilenceAction(c.Key)}
c.Raised = r.now.Add(-time.Duration(10-i) * time.Hour)
open = append(open, c)
}
urgent := heldCondition()
urgent.Severity, urgent.Raised = conditions.Urgent, r.now.Add(-time.Minute)
r.open = append(open, urgent)
_ = r.a.reconcile(context.Background())
sent := r.asksSent(t)
if len(sent) != askMostOpen || sent[0].About != urgent.Key || sent[1].About != "machine.ma.units" || sent[2].About != "machine.mb.units" {
var about []string
for _, q := range sent {
about = append(about, q.About)
}
t.Fatalf("asked %v", about)
}
}
// After review: nothing is asked while no router takes asks under the controller's name, and that is said once.
func TestNothingIsAskedWithoutARouter(t *testing.T) {
r := newAskerRig(t)
var said []string
r.a.logf = func(f string, a ...any) { said = append(said, f) }
r.a.routerHere = func(context.Context) (bool, error) { return false, nil }
r.open = []conditions.Condition{heldCondition()}
_ = r.a.reconcile(context.Background())
_ = r.a.reconcile(context.Background())
if len(r.asksSent(t)) != 0 {
t.Error("asked with no router")
}
n := 0
for _, s := range said {
if strings.Contains(s, "no router takes asks") {
n++
}
}
if n != 1 {
t.Errorf("said %d times", n)
}
}
// After review: the condition's words keep where an answer is given without a channel; the ask's text does not.
func TestTheAskDropsWhereItIsAnsweredAndTheConditionKeepsIt(t *testing.T) {
c := heldCondition()
if !strings.Contains(c.Explanation, FromMeshMCPServer) {
t.Fatalf("the condition lost where it is answered: %q", c.Explanation)
}
q, _ := askOf("x", c, time.Now())
if strings.Contains(q.Explanation, "mesh MCP server") || !strings.HasPrefix(q.Explanation, "Needs you: release it, or stop it.") {
t.Errorf("the ask says %q", q.Explanation)
}
if askApproveFor >= 24*time.Hour {
t.Errorf("an approving ask lasts %s, which the SDK may refuse at its bound", askApproveFor)
}
}
// After review (security finding 9): a warrant is acted on only for an ask open in the controller's own record,
// once the claim stands, and never when given after the ask expired.
func TestAWarrantIsActedOnlyForAnOpenAskItClaimsBeforeItExpired(t *testing.T) {
r := newAskerRig(t)
r.open = []conditions.Condition{heldCondition()}
_ = r.a.reconcile(context.Background())
w := r.warrantFor(t, heldCondition().Key, "Release")
late := w
late.At = r.store[w.Ask].Ask.Expires.Add(time.Minute)
body, _ := json.Marshal(late)
_ = r.a.Decided(context.Background(), body)
if len(r.called) != 0 {
t.Fatalf("acted on a warrant given after the ask expired: %v", r.called)
}
// Claimed already by another delivery: nothing done here.
kept := r.store[w.Ask]
kept.Acted = "acting"
r.store[w.Ask] = kept
body, _ = json.Marshal(w)
_ = r.a.Decided(context.Background(), body)
if len(r.called) != 0 {
t.Fatalf("acted though the claim was another's: %v", r.called)
}
// Cancelled in its own record: refused.
kept.Acted, kept.State = "", askCancelled
r.store[w.Ask] = kept
_ = r.a.Decided(context.Background(), body)
if len(r.called) != 0 {
t.Errorf("acted on a cancelled ask: %v", r.called)
}
}
+295
View File
@@ -0,0 +1,295 @@
package main
// The asker on the bus: its asks in the controller's bucket `asked`, its asks and cancels published on the
// seat under the controller's name, the verbs a warrant chooses called with the controller's grant, and the
// router's record of its asks read under its name (novox/hq ADR 0259).
import (
"context"
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// askerFrom is the serving controller's asker; nil in any other process.
var askerFrom *asker
// askWithin is how long a verb a warrant chose is given to answer.
const askWithin = time.Minute
type busAsked struct{ conn *nats.Conn }
func (b busAsked) kv(ctx context.Context) (jetstream.KeyValue, error) {
js, err := jetstream.New(b.conn)
if err != nil {
return nil, err
}
return js.KeyValue(ctx, broker.AskedBucket)
}
func (b busAsked) Get(ctx context.Context, id string) (*asked, error) {
kv, err := b.kv(ctx)
if err != nil {
return nil, err
}
e, err := kv.Get(ctx, id)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, nil
}
if err != nil {
return nil, err
}
var r asked
return &r, json.Unmarshal(e.Value(), &r)
}
func (b busAsked) Put(ctx context.Context, r asked) error {
kv, err := b.kv(ctx)
if err != nil {
return err
}
body, err := json.Marshal(r)
if err != nil {
return err
}
_, err = kv.Put(ctx, r.ID, body)
return err
}
func (b busAsked) All(ctx context.Context) ([]asked, error) {
kv, err := b.kv(ctx)
if err != nil {
return nil, err
}
lister, err := kv.ListKeys(ctx)
if err != nil {
return nil, err
}
defer func() { _ = lister.Stop() }()
var out []asked
for k := range lister.Keys() {
e, err := kv.Get(ctx, k)
if err != nil {
continue
}
var r asked
if json.Unmarshal(e.Value(), &r) == nil {
out = append(out, r)
}
}
return out, nil
}
// Claim marks an open ask acting, by compare-and-set on its key's revision: only the write that stands acts.
func (b busAsked) Claim(ctx context.Context, id string, w asks.Warrant) (bool, error) {
kv, err := b.kv(ctx)
if err != nil {
return false, err
}
e, err := kv.Get(ctx, id)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return false, nil
}
if err != nil {
return false, err
}
var r asked
if err := json.Unmarshal(e.Value(), &r); err != nil {
return false, err
}
if r.State != askOpen || r.Acted != "" {
return false, nil
}
r.State, r.Warrant, r.Acted = string(asks.OutcomeChosen), &w, "acting"
body, err := json.Marshal(r)
if err != nil {
return false, err
}
if _, err := kv.Update(ctx, id, body, e.Revision()); err != nil {
var api *jetstream.APIError
if errors.Is(err, jetstream.ErrKeyExists) || (errors.As(err, &api) && api.ErrorCode == jetstream.JSErrCodeStreamWrongLastSequence) {
return false, nil
}
return false, err
}
return true, nil
}
// callAction performs an action's verb as the controller, through the grant that names it.
func callAction(conn *nats.Conn) func(ctx context.Context, a conditions.Action, args map[string]string) error {
return func(ctx context.Context, a conditions.Action, args map[string]string) error {
seat, verb, ok := strings.Cut(a.Verb, ".")
if !ok {
return fmt.Errorf("%q names no seat and verb", a.Verb)
}
body := map[string]any{}
for k, v := range args {
body[k] = v
}
if seat == catalogue.DeliverySeat {
_, err := askDeliveryOwner(ctx, conn, verb, body)
return err
}
granted := false
for _, v := range broker.VerbsTheControllerActsOnAWarrant {
granted = granted || (v.Seat == seat && v.Verb == verb)
}
if !granted {
return fmt.Errorf("%s: %w", a.Verb, errNotGranted)
}
var answer link.Answer
var err error
if a.Machine != "" {
answer, err = link.AskSeatTool(ctx, conn, seat, verb, a.Machine, body, askWithin)
} else {
answer, err = link.AskMeshSeatTool(ctx, conn, seat, verb, body, askWithin)
}
if err != nil {
return err
}
if answer.Error != "" {
return fmt.Errorf("%s refused: %s", a.Verb, answer.Error)
}
return nil
}
}
// routerRecordOf reads the router's record of one of the controller's asks, under its name, and answers
// how it ended when it did: the bucket is the one the asks seat's declarer names as its records.
func routerRecordOf(conn *nats.Conn, inv *inventory.Inventory) func(ctx context.Context, id string) (*asks.Warrant, error) {
return func(ctx context.Context, id string) (*asks.Warrant, error) {
bucket, err := asksRecords(ctx, inv)
if err != nil || bucket == "" {
return nil, err
}
reply, err := conn.RequestWithContext(ctx, "$JS.API.DIRECT.GET.KV_"+bucket+".$KV."+bucket+"."+askerName+"."+id, nil)
if err != nil {
return nil, err
}
if reply.Header.Get("Status") != "" {
return nil, nil // none, or not readable: the event says it
}
var rec struct {
State string `json:"state"`
Warrant *asks.Warrant `json:"warrant"`
}
if json.Unmarshal(reply.Data, &rec) != nil || rec.State == "open" || rec.Warrant == nil {
return nil, nil
}
return rec.Warrant, nil
}
}
// asksRecords is the bucket the asks seat's declarer keeps its record of asks in.
func asksRecords(ctx context.Context, inv *inventory.Inventory) (string, error) {
declared, err := inv.Catalogue(ctx)
if err != nil {
return "", err
}
for _, m := range declared {
for _, s := range m.DefinesSeats {
if s.Name == broker.AsksSeat && len(s.Records) > 0 {
return broker.BucketName(m.Module, s.Records[0]), nil
}
}
}
return "", nil
}
// routerHereIn says whether a module declaring the asks seat, with its ask named by its caller, is assigned:
// without it nothing takes an ask, and asking would only fill a queue nobody reads.
func routerHereIn(inv *inventory.Inventory) func(ctx context.Context) (bool, error) {
return func(ctx context.Context) (bool, error) {
entries, err := inv.Catalogued(ctx)
if err != nil {
return false, err
}
for _, e := range entries {
for _, s := range e.Manifest.DefinesSeats {
if s.Name == broker.AsksSeat && s.NamedByCaller("ask") && len(e.On) > 0 {
return true, nil
}
}
}
return false, nil
}
}
// channelsIn is what the channels are now, as a fingerprint: each module claiming a kind of the channel
// bench, where, promising what, and whether of its own account. An ask the router refused is asked again
// once this changes.
func channelsIn(inv *inventory.Inventory) func(ctx context.Context) string {
return func(ctx context.Context) string {
entries, err := inv.Catalogued(ctx)
if err != nil {
return ""
}
var parts []string
for _, e := range entries {
for _, c := range e.Manifest.Claims {
if c.Kind == "" || !catalogue.KindedBenches[c.Name] {
continue
}
on := append([]string(nil), e.On...)
sort.Strings(on)
caps := append([]string(nil), c.Capabilities...)
sort.Strings(caps)
parts = append(parts, fmt.Sprintf("%s/%s=%s@%s[%s]own:%t", c.Name, c.Kind, e.Manifest.Module,
strings.Join(on, ","), strings.Join(caps, ","), e.Manifest.RunsAs != ""))
}
}
sort.Strings(parts)
return strings.Join(parts, ";")
}
}
// startAsking makes the serving controller's asker and hands it the router's words.
func startAsking(ctx context.Context, open *stores, server *link.Server, conn *nats.Conn, keeper *conditions.Keeper) {
js, err := jetstream.New(conn)
if err != nil {
fmt.Printf("the operator cannot be asked: %v\n", err)
return
}
a := &asker{
open: keeper.Open,
silence: func(ctx context.Context, key string, d time.Duration, by, why string) error {
_, err := keeper.Silence(ctx, key, d, by, why)
return err
},
store: busAsked{conn: conn},
publish: func(ctx context.Context, subject string, body []byte, id string) error {
_, err := js.Publish(ctx, subject, body, jetstream.WithMsgID(id))
return err
},
call: callAction(conn),
record: func(ctx context.Context, act link.HandAct) error {
_, err := link.RecordHandAct(ctx, conn, act)
return err
},
routerRecord: routerRecordOf(conn, open.inventory),
routerHere: routerHereIn(open.inventory),
channels: channelsIn(open.inventory),
now: time.Now,
logf: func(format string, args ...any) { fmt.Printf(format+"\n", args...) },
}
if err := server.Decides(a); err != nil {
fmt.Printf("the operator's answers cannot be heard, so nothing is asked: %v\n", err)
return
}
askerFrom = a
go a.keep(ctx)
}
+48
View File
@@ -53,9 +53,26 @@ func assertBusObjects(ctx context.Context, inv *inventory.Inventory, r broker.Ra
if err != nil { if err != nil {
return nil, err return nil, err
} }
// And the work queues of seats that name their caller or their kind, with each holder's worker
// (novox/hq ADR 0259 §3): an ask queues until the router takes it, a channel's work until that kind
// takes it.
trafficStreams, trafficWorkers, err := seatTrafficObjects(ctx, inv)
if err != nil {
return nil, err
}
// Every one tried, and every failure named: one module's consumer the bus refuses is no reason // Every one tried, and every failure named: one module's consumer the bus refuses is no reason
// the modules after it in the list hear nothing (novox/hq issue 208, where this runs on each send). // the modules after it in the list hear nothing (novox/hq issue 208, where this runs on each send).
var failed []error var failed []error
for _, s := range trafficStreams {
if err := r.EnsureStream(s); err != nil {
failed = append(failed, fmt.Errorf("the work queue %s: %w", s.Name, err))
}
}
for _, c := range trafficWorkers {
if err := r.EnsureConsumer(c); err != nil {
failed = append(failed, fmt.Errorf("the worker %s on %s: %w", c.Name, c.Stream, err))
}
}
for _, c := range consumers { for _, c := range consumers {
if err := r.EnsureConsumer(c.Consumer); err != nil { if err := r.EnsureConsumer(c.Consumer); err != nil {
failed = append(failed, fmt.Errorf("how %s on %s hears what it consumes: %w", c.Module, c.Node, err)) failed = append(failed, fmt.Errorf("how %s on %s hears what it consumes: %w", c.Module, c.Node, err))
@@ -130,6 +147,37 @@ func moduleConsumers(ctx context.Context, inv *inventory.Inventory) ([]broker.Mo
return broker.ConsumersOf(users), nil return broker.ConsumersOf(users), nil
} }
// seatTrafficObjects is the work queues and workers of seats that name their caller or their kind, from
// the records the user list is composed from.
func seatTrafficObjects(ctx context.Context, inv *inventory.Inventory) ([]broker.Stream, []broker.Consumer, error) {
records, err := inv.BusRecords(ctx)
if err != nil {
return nil, nil, err
}
users, err := broker.Users(records)
if err != nil {
return nil, nil, err
}
streams, workers := broker.SeatTrafficObjects(users)
// And the queue of every such seat the catalogue declares, held or not: work queues from registration,
// so what is submitted before a holder is assigned waits for it (the correctness review of 2026-10-08).
declared, err := inv.DeclaredTrafficSeats(ctx)
if err != nil {
return nil, nil, err
}
have := map[string]bool{}
for _, s := range streams {
have[s.Name] = true
}
for _, s := range broker.TrafficQueues(declared) {
if !have[s.Name] {
streams = append(streams, s)
have[s.Name] = true
}
}
return streams, workers, nil
}
// moduleConsumerCount is how many modules hear what they consume, for the raise's one line. // moduleConsumerCount is how many modules hear what they consume, for the raise's one line.
func moduleConsumerCount(ctx context.Context, inv *inventory.Inventory) (int, error) { func moduleConsumerCount(ctx context.Context, inv *inventory.Inventory) (int, error) {
consumers, err := moduleConsumers(ctx, inv) consumers, err := moduleConsumers(ctx, inv)
+1 -1
View File
@@ -46,7 +46,7 @@ func keeperOn(ctx context.Context, conn *nats.Conn) (*conditions.Keeper, error)
Say: func(format string, args ...any) { fmt.Fprintf(os.Stderr, format+"\n", args...) }, Say: func(format string, args ...any) { fmt.Fprintf(os.Stderr, format+"\n", args...) },
// What status leads with changed: composed again soon (a nudge outside the serving controller // What status leads with changed: composed again soon (a nudge outside the serving controller
// does nothing). // does nothing).
Changed: statusFrom.nudge, Changed: func() { statusFrom.nudge(); askerFrom.nudge() },
// Written under the lease, carrying its epoch (novox/hq to-be 45 §6). // Written under the lease, carrying its epoch (novox/hq to-be 45 §6).
Epoch: func() (uint64, error) { return theLease.epoch(context.WithoutCancel(ctx)) }}), nil Epoch: func() (uint64, error) { return theLease.epoch(context.WithoutCancel(ctx)) }}), nil
} }
@@ -136,12 +136,13 @@ func TestTheDeliveryOwnerIsAskedOverTheBus(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
for _, verb := range []string{"stalled", "close"} { // And release and stop, which the operator's warrant chooses (novox/hq ADR 0259).
for _, verb := range []string{"stalled", "close", "release", "stop"} {
if !slices.Contains(granted.Publish, link.SeatToolSubject(catalogue.DeliverySeat, verb)) { if !slices.Contains(granted.Publish, link.SeatToolSubject(catalogue.DeliverySeat, verb)) {
t.Errorf("the controller may not ask %s.%s", catalogue.DeliverySeat, verb) t.Errorf("the controller may not ask %s.%s", catalogue.DeliverySeat, verb)
} }
} }
if _, err := askDeliveryOwner(t.Context(), nil, "stop", nil); err == nil || !strings.Contains(err.Error(), "grant") { if _, err := askDeliveryOwner(t.Context(), nil, "retire-history", nil); err == nil || !strings.Contains(err.Error(), "grant") {
t.Fatalf("a verb the grant does not name was asked: %v", err) t.Fatalf("a verb the grant does not name was asked: %v", err)
} }
conn, err := nats.Connect(testbus.URL(t)) conn, err := nats.Connect(testbus.URL(t))
+5
View File
@@ -116,6 +116,11 @@ var probeRegistry = []probe{
{ID: probeDeliveriesID, Asserts: "no delivery is held past its state's bound unsaid: mesh-delivery's " + {ID: probeDeliveriesID, Asserts: "no delivery is held past its state's bound unsaid: mesh-delivery's " +
"`stalled`, each with the transition its table lets healer H2 take", From: "ADR 0239", "`stalled`, each with the transition its table lets healer H2 take", From: "ADR 0239",
Kind: kindDeliveryStalled, Phase: 3, run: probeDeliveries}, Kind: kindDeliveryStalled, Phase: 3, run: probeDeliveries},
// Root where the trusted parties run (novox/hq ADR 0259 §8): while an agent can become root there without a
// person, an answer proven there proves nothing.
{ID: "D-root", Asserts: "no agent can become root without a person on a machine where the router or a channel " +
"proving its sender runs: not by its own account, and not through a tool that runs its command as an account " +
"that can", From: "ADR 0259 §8", Kind: kindAgentRoot, Phase: 2, run: probeAgentRoot},
{ID: "DW", Asserts: "the watchdogs of the signals table ran within three of their intervals", {ID: "DW", Asserts: "the watchdogs of the signals table ran within three of their intervals",
From: "ADR 0227 rule 6: the watchers are watched", Kind: "watchdogs-silent", Phase: 1, run: probeWatchdogs}, From: "ADR 0227 rule 6: the watchers are watched", Kind: "watchdogs-silent", Phase: 1, run: probeWatchdogs},
// The core's health definitions (novox/hq to-be 45 §8, ADR 0236): what a core component's new build is // The core's health definitions (novox/hq to-be 45 §8, ADR 0236): what a core component's new build is
+10 -3
View File
@@ -60,11 +60,18 @@ var handActVerbs = []handActVerb{
// a person's word (ADR 0242), which the push itself reads from what it carried (recorded_push.go). // a person's word (ADR 0242), which the push itself reads from what it carried (recorded_push.go).
{Verb: "push", Decision: "a recorded build moves only by a person's push: that push is the word its " + {Verb: "push", Decision: "a recorded build moves only by a person's push: that push is the word its " +
"upgrade policy asks for (ADR 0242)", DecidedWhen: pushedRecorded}, "upgrade policy asks for (ADR 0242)", DecidedWhen: pushedRecorded},
{Verb: "plans stop"}, // Stopping or starting a walk the operator chose on a warrant (novox/hq ADR 0259) is their decision.
{Verb: "plans stop", Decision: "the operator's answer to an ask is their decision, not a repair (ADR 0259)",
DecidedFor: []string{conditions.CauseOperatorAnswer}},
{Verb: "plans close"}, {Verb: "plans close"},
// A walk started by a person instead of its delivery's owner (novox/hq ADR 0239): the owner down, or // A walk started by a person instead of its delivery's owner (novox/hq ADR 0239): the owner down, or
// not trusted with it — either is a repair the owner should have made. // not trusted with it — either is a repair the owner should have made. Unless the operator chose it on
{Verb: "plans go"}, // a warrant (ADR 0259).
{Verb: "plans go", Decision: "the operator's answer to an ask is their decision, not a repair (ADR 0259)",
DecidedFor: []string{conditions.CauseOperatorAnswer}},
// An act the operator chose on a warrant (novox/hq ADR 0259): asked by the controller, answered on a
// channel that proved who answered, performed by the controller as itself.
{Verb: handActWarrant, Decision: "the operator chose it, answering what the controller asked (ADR 0259)"},
{Verb: "broker consumer-reset"}, {Verb: "broker consumer-reset"},
// Silencing the same condition twice says the condition, or what it watches, wants mending — unless // Silencing the same condition twice says the condition, or what it watches, wants mending — unless
// it is the operator's answer on a notification: a decision to live with it (novox/hq ADR 0258). // it is the operator's answer on a notification: a decision to live with it (novox/hq ADR 0258).
+1
View File
@@ -383,6 +383,7 @@ func moduleUnhealthyObservation(module, node string, rs []inventory.ResourceHeal
Explanation: fmt.Sprintf("%s on %s is not healthy: %s. It clears as soon as it runs again.", module, node, Explanation: fmt.Sprintf("%s on %s is not healthy: %s. It clears as soon as it runs again.", module, node,
namesWords(plain, 3)), namesWords(plain, 3)),
Needs: needs, Needs: needs,
Actions: moduleActions(node, rs),
Resolved: fmt.Sprintf("%s works again on %s", module, node)} Resolved: fmt.Sprintf("%s works again on %s", module, node)}
} }
+48 -4
View File
@@ -648,6 +648,8 @@ func walkWaitingWords(w waitFacts, in time.Duration, severity conditions.Severit
} }
// waitingNeeds is what the operator does about a walk waiting past its urgent bound: nothing before it. // waitingNeeds is what the operator does about a walk waiting past its urgent bound: nothing before it.
// Start and Stop are also asked of the operator (novox/hq ADR 0259); the condition's own words keep saying
// where they are given without a channel, and the ask's text drops that (askText).
func waitingNeeds(severity conditions.Severity) string { func waitingNeeds(severity conditions.Severity) string {
if severity == conditions.Urgent { if severity == conditions.Urgent {
return "start it, or stop it, " + FromMeshMCPServer return "start it, or stop it, " + FromMeshMCPServer
@@ -655,6 +657,20 @@ func waitingNeeds(severity conditions.Severity) string {
return "" return ""
} }
// waitingActions are the answers to a walk waiting past its urgent bound: start it, or stop it — the plan's
// own verbs, approved by the operator (novox/hq ADR 0259). None before the bound.
func waitingActions(plan string, severity conditions.Severity) []conditions.Action {
if severity != conditions.Urgent || plan == "" {
return nil
}
return []conditions.Action{
{Label: "Start", Verb: "mesh-controller.plans", Level: conditions.LevelApprove,
Arguments: map[string]string{"go": plan, "why": "", "cause": conditions.CauseOperatorAnswer}},
{Label: "Stop", Verb: "mesh-controller.plans", Level: conditions.LevelApprove,
Arguments: map[string]string{"stop": plan, "why": "", "cause": conditions.CauseOperatorAnswer}},
}
}
// moduleNeeds is what the operator can do about a module unhealthy on a machine: log in again where its // moduleNeeds is what the operator can do about a module unhealthy on a machine: log in again where its
// account's groups wait for it (ADR 0252), restart a failed service, or nothing where the mesh restarts it. // account's groups wait for it (ADR 0252), restart a failed service, or nothing where the mesh restarts it.
// No answer is offered for a restart: a desk click performs only an acknowledgement (ADR 0258). // No answer is offered for a restart: a desk click performs only an acknowledgement (ADR 0258).
@@ -669,11 +685,35 @@ func moduleNeeds(node string, rs []inventory.ResourceHealth) string {
} }
} }
if unit != "" { if unit != "" {
// Also asked of the operator (moduleActions); the ask's text drops where (askText).
return fmt.Sprintf("restart its service %s on %s %s", unit, node, FromMeshMCPServer) return fmt.Sprintf("restart its service %s on %s %s", unit, node, FromMeshMCPServer)
} }
return "" return ""
} }
// moduleActions are the answers to a module unhealthy on a machine: restart its failed service there,
// approved by the operator (novox/hq ADR 0259) — none when the mesh restarts it, or a new login is what it
// waits for.
func moduleActions(node string, rs []inventory.ResourceHealth) []conditions.Action {
for _, r := range rs {
if strings.Contains(r.Reason, "relogin needed") {
return nil
}
}
for _, r := range rs {
if r.Kind != link.KindUnit || r.Target == "" {
continue
}
scope := "system"
if r.Account != "" {
scope = "user"
}
return []conditions.Action{{Label: "Restart", Verb: "node-service-manager.restart", Machine: node,
Level: conditions.LevelApprove, Arguments: map[string]string{"unit": r.Target, "scope": scope}}}
}
return nil
}
// FromMeshMCPServer ends what the operator needs when no notification can do it (ADR 0258), naming the mesh MCP // FromMeshMCPServer ends what the operator needs when no notification can do it (ADR 0258), naming the mesh MCP
// server (the glossary's word; "console" is retired): the answer is not an // server (the glossary's word; "console" is retired): the answer is not an
// acknowledgement, so it is given where the operator is known to be the one asking, until answers are // acknowledgement, so it is given where the operator is known to be the one asking, until answers are
@@ -725,15 +765,19 @@ func stalledWords(l stalledLine, o conditions.Observation) (headline, explanatio
long = "for " + humanDuration(d) long = "for " + humanDuration(d)
} }
if o.Resolver == conditions.ResolverOperator { if o.Resolver == conditions.ResolverOperator {
// Words only: releasing or stopping a delivery is not an acknowledgement, so no desk click // Asked of the operator, approved on a channel that proves who answered (novox/hq ADR 0259); the
// performs it (ADR 0258). // router says where each can be answered, so the words do not.
release := conditions.Action{Label: "Release", Verb: "mesh-delivery.release", Level: conditions.LevelApprove,
Arguments: map[string]string{"id": l.ID, "why": ""}}
stop := conditions.Action{Label: "Stop", Verb: "mesh-delivery.stop", Level: conditions.LevelApprove,
Arguments: map[string]string{"id": l.ID, "why": ""}}
switch held { switch held {
case "held": case "held":
needs = "release it, or stop it, " + FromMeshMCPServer needs, actions = "release it, or stop it, "+FromMeshMCPServer, []conditions.Action{release, stop}
case "ready", "checked": case "ready", "checked":
needs = "merge its pull request, or close it." needs = "merge its pull request, or close it."
default: default:
needs = "stop it " + FromMeshMCPServer needs, actions = "stop it "+FromMeshMCPServer, []conditions.Action{stop}
} }
} }
return fmt.Sprintf("Delivery of %s %s %s", name, held, long), return fmt.Sprintf("Delivery of %s %s %s", name, held, long),
+24 -7
View File
@@ -65,13 +65,21 @@ func TestADeliveryWaitingNeedsNothingUntilItsBoundThenOffersStartAndStop(t *test
t.Errorf("the summary lost the way on for whoever looks closer: %q", got[0].Summary) t.Errorf("the summary lost the way on for whoever looks closer: %q", got[0].Summary)
} }
// Past four hours it is urgent, and offers the controller's own answers. // Past four hours it is urgent, and asks the operator to start or stop it (novox/hq ADR 0259): the plan's
// own verbs, approved, which the controller performs on the warrant. The router says where to answer.
f.waits[0].since = now.Add(-5 * time.Hour) f.waits[0].since = now.Add(-5 * time.Hour)
got = watchWaits(f) got = watchWaits(f)
plainExample(t, got[0], "openrazer delivery waiting to start", plainExample(t, got[0], "openrazer delivery waiting to start",
"Needs you: start it, or stop it, from the mesh MCP server; this notification cannot do it. The change to openrazer is merged and built, and mesh-delivery (the "+ "Needs you: start it, or stop it, from the mesh MCP server; this notification cannot do it. The change to openrazer is merged and built, and mesh-delivery (the "+
"module that decides when a delivery goes out) has not let it start for 5 hours, so mesh-delivery may "+ "module that decides when a delivery goes out) has not let it start for 5 hours, so mesh-delivery may "+
"be stuck.") "be stuck.", "Start", "Stop")
for i, want := range []string{"go", "stop"} {
a := got[0].Actions[i]
if a.Verb != "mesh-controller.plans" || a.Arguments[want] != "plan-1791454185265004861" ||
a.Level != conditions.LevelApprove || a.Arguments["cause"] != conditions.CauseOperatorAnswer {
t.Errorf("%s: %+v", a.Label, a)
}
}
// Many modules are counted, not listed in the headline. // Many modules are counted, not listed in the headline.
f.waits[0].modules = []string{"a", "b", "c", "d"} f.waits[0].modules = []string{"a", "b", "c", "d"}
@@ -82,16 +90,20 @@ func TestADeliveryWaitingNeedsNothingUntilItsBoundThenOffersStartAndStop(t *test
} }
// **A module unhealthy**: "openrazer on g14 is not healthy: its unit openrazer-daemon.service failed in the // **A module unhealthy**: "openrazer on g14 is not healthy: its unit openrazer-daemon.service failed in the
// account's own service manager (exit-code)". Restarting is not an acknowledgement, so it is said in words // account's own service manager (exit-code)". Restarting is not an acknowledgement: it is asked of the
// and offered as no answer (ADR 0258). // operator at the approve level (novox/hq ADR 0259), so a desk click never performs it (ADR 0258).
func TestAModuleUnhealthyAsksForARestartInWords(t *testing.T) { func TestAModuleUnhealthyAsksForARestartInWords(t *testing.T) {
o := moduleUnhealthyObservation("openrazer", "g14", []inventory.ResourceHealth{{Kind: link.KindUnit, o := moduleUnhealthyObservation("openrazer", "g14", []inventory.ResourceHealth{{Kind: link.KindUnit,
Resource: "openrazer-daemon", Target: "openrazer-daemon.service", Resource: "openrazer-daemon", Target: "openrazer-daemon.service", Account: "jochen",
Reason: "failed in the account's own service manager (exit-code)", Since: time.Now()}}) Reason: "failed in the account's own service manager (exit-code)", Since: time.Now()}})
plainExample(t, o, "openrazer not working on g14", plainExample(t, o, "openrazer not working on g14",
"Needs you: restart its service openrazer-daemon on g14 from the mesh MCP server; this notification cannot do it. "+ "Needs you: restart its service openrazer-daemon on g14 from the mesh MCP server; this notification cannot do it. "+
"openrazer on g14 is not healthy: its service openrazer-daemon stopped with an error. It clears as soon "+ "openrazer on g14 is not healthy: its service openrazer-daemon stopped with an error. It clears as soon "+
"as it runs again.") "as it runs again.", "Restart")
if a := o.Actions[0]; a.Verb != "node-service-manager.restart" || a.Machine != "g14" || a.Level != conditions.LevelApprove ||
a.Arguments["unit"] != "openrazer-daemon.service" || a.Arguments["scope"] != "user" {
t.Errorf("restart: %+v", a)
}
// An account waiting for a new login (ADR 0252) asks for the login, held to the plain rule. // An account waiting for a new login (ADR 0252) asks for the login, held to the plain rule.
o = moduleUnhealthyObservation("openrazer", "g14", []inventory.ResourceHealth{{Kind: "account", o = moduleUnhealthyObservation("openrazer", "g14", []inventory.ResourceHealth{{Kind: "account",
Resource: "operator-in-group", Target: "jochen", Reason: "relogin needed: the account is in the group"}}) Resource: "operator-in-group", Target: "jochen", Reason: "relogin needed: the account is in the group"}})
@@ -154,7 +166,12 @@ func TestADeliveryHeldAsksForReleaseOrStopInWords(t *testing.T) {
Bound: "24h0m0s", H2: "none: the state is the operator's", Says: "it waits for the operator"}}) Bound: "24h0m0s", H2: "none: the state is the operator's", Says: "it waits for the operator"}})
plainExample(t, got[0], "Delivery of hq held for 36 hours", plainExample(t, got[0], "Delivery of hq held for 36 hours",
"Needs you: release it, or stop it, from the mesh MCP server; this notification cannot do it. A delivery of hq has been held for 36 hours, past its limit.", "Needs you: release it, or stop it, from the mesh MCP server; this notification cannot do it. A delivery of hq has been held for 36 hours, past its limit.",
) "Release", "Stop")
for i, verb := range []string{"mesh-delivery.release", "mesh-delivery.stop"} {
if a := got[0].Actions[i]; a.Verb != verb || a.Arguments["id"] != "novox/hq@055550802096" || a.Level != conditions.LevelApprove {
t.Errorf("%+v", a)
}
}
} }
// **Every kind the controller raises has plain words**, and its words are plain for a subject of every // **Every kind the controller raises has plain words**, and its words are plain for a subject of every
+194
View File
@@ -0,0 +1,194 @@
package main
import (
"context"
"encoding/json"
"fmt"
"slices"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// The self-check's probe of who can become root where the trusted parties run (novox/hq ADR 0259 §8, rule 3).
//
// The router and every channel proving its sender run as accounts of their own, so that no agent reads what
// they hold or speaks as them. **Root on their machine undoes all of it.** So on every machine where a module
// of its own account runs, this probe asks two questions, and raises an urgent condition while either is yes:
//
// 1. can an account an agent runs as become root without a person there — passwordless sudo, or a group
// that is root by another name (docker, disk)? The agent's account is the one the coding-agent module on
// that machine names (`agent_account`), and the operator's account while it names none;
// 2. can an agent run a command it chooses, through the mesh's own tools, as an account that can — the login
// shell's `execute` runs as the machine's runtime account, which the mesh's acting tools give passwordless
// sudo?
//
// The measurement is the sudo module's on that machine (`sudo_escalation`); a machine it does not run on, or
// that does not answer, is a probe that could not run — said, never taken for "no".
// kindAgentRoot is the condition an agent able to become root where a trusted party runs raises.
const kindAgentRoot = "agent-root"
// The tools the probe asks, of the modules on each machine.
const (
agentModule = "claude-code"
agentStatusTool = "claude_code_status"
sudoModule = "sudo"
sudoEscalation = "sudo_escalation"
loginShellSeat = "node-login-shell"
)
// escalation is the sudo module's answer for one account.
type escalation struct {
Account string `json:"account"`
Root bool `json:"root_without_a_person"`
Why string `json:"why"`
}
// agentRootFacts is what one machine says, as the probe reads it.
type agentRootFacts struct {
Machine string
Trusted []string // the modules of their own account on it
Agent string // the account agents run as there; "" when none run there
AgentNamed bool // the coding-agent module named it, rather than it being taken for the operator's
Runtime string // the account the machine's runtime runs as
LoginShell bool // the login shell seat is held there, running commands as the runtime's account
Answers map[string]escalation
}
// agentRootObservations judges one machine's facts: an urgent condition while an agent can become root there
// without a person, one way or the other, naming which.
func agentRootObservations(f agentRootFacts) []conditions.Observation {
var ways []string
if f.Agent != "" {
if e := f.Answers[f.Agent]; e.Root {
named := "the operator's account, which agents run as while the coding-agent module names no other"
if f.AgentNamed {
named = "the account agents run as"
}
ways = append(ways, fmt.Sprintf("%s (%s) can become root without a person: %s", f.Agent, named, e.Why))
}
}
if f.LoginShell && f.Runtime != "" {
if e := f.Answers[f.Runtime]; e.Root {
ways = append(ways, fmt.Sprintf("the login shell runs any command an agent gives it as %s, which can "+
"become root without a person: %s", f.Runtime, e.Why))
}
}
if len(ways) == 0 {
return nil
}
sort.Strings(f.Trusted)
return []conditions.Observation{{Scope: conditions.ScopeMachine, ID: f.Machine, Token: kindAgentRoot,
Machine: f.Machine, Kind: kindAgentRoot, Severity: conditions.Urgent,
Summary: fmt.Sprintf("an agent can become root on %s without a person, where %s run as accounts of their "+
"own; until it cannot, the router approves nothing proven there (novox/hq ADR 0259 §8): %s",
f.Machine, strings.Join(f.Trusted, ", "), strings.Join(ways, "; ")),
Headline: "Root without you on " + f.Machine,
Needs: "choose how programs working for you on " + f.Machine + " stop becoming root without asking you.",
Explanation: "The modules that prove your answers from your phone run on " + f.Machine + ", and a program " +
"working for you there can become root without asking you, so it could answer in your name. Until " +
"that changes, answers from your phone can only acknowledge.",
Resolved: "Nothing on " + f.Machine + " becomes root without you any more"}}
}
// probeAgentRoot is the probe: every machine a module of its own account runs on, judged.
func probeAgentRoot(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
inv := d.open.inventory
entries, err := inv.Catalogued(ctx)
if err != nil {
return nil, err
}
facts := map[string]*agentRootFacts{}
agentOn, sudoOn := map[string]bool{}, map[string]bool{}
for _, e := range entries {
for _, node := range e.On {
switch {
case e.Manifest.RunsAs != "":
f := facts[node]
if f == nil {
f = &agentRootFacts{Machine: node, Answers: map[string]escalation{}}
facts[node] = f
}
f.Trusted = append(f.Trusted, e.Manifest.Module)
case e.Manifest.Module == agentModule:
agentOn[node] = true
case e.Manifest.Module == sudoModule:
sudoOn[node] = true
}
}
}
for _, e := range entries {
if e.Manifest.ClaimsSeat(loginShellSeat) {
for _, node := range e.On {
if f := facts[node]; f != nil {
f.LoginShell = true
}
}
}
}
machines := make([]string, 0, len(facts))
for m := range facts {
machines = append(machines, m)
}
sort.Strings(machines)
var out []conditions.Observation
var unread []string
for _, m := range machines {
f := facts[m]
record, err := inv.NodeByName(ctx, m)
if err != nil {
unread = append(unread, m+": "+err.Error())
continue
}
f.Runtime = record.Account
if agentOn[m] {
f.Agent = record.Account
if a, err := link.AskModuleToolOn(ctx, d.js.Conn(), agentModule, agentStatusTool, m, map[string]any{}, 10*time.Second); err == nil && a.Error == "" {
var status struct {
AgentAccount string `json:"agent_account"`
}
if json.Unmarshal(a.Result, &status) == nil && status.AgentAccount != "" {
f.Agent, f.AgentNamed = status.AgentAccount, true
}
}
}
if !sudoOn[m] {
unread = append(unread, m+": the sudo module is not assigned there, so who can become root is not measured")
continue
}
var accounts []string
for _, a := range []string{f.Agent, f.Runtime} {
if a != "" && !slices.Contains(accounts, a) {
accounts = append(accounts, a)
}
}
if len(accounts) == 0 {
continue
}
a, err := link.AskModuleToolOn(ctx, d.js.Conn(), sudoModule, sudoEscalation, m, map[string]any{"accounts": accounts}, 15*time.Second)
if err == nil && a.Error != "" {
err = fmt.Errorf("%s", a.Error)
}
if err != nil {
unread = append(unread, m+": "+err.Error())
continue
}
var answers []escalation
if err := json.Unmarshal(a.Result, &answers); err != nil {
unread = append(unread, m+": the sudo module's answer could not be read: "+err.Error())
continue
}
for _, e := range answers {
f.Answers[e.Account] = e
}
out = append(out, agentRootObservations(*f)...)
}
if len(unread) > 0 {
return out, fmt.Errorf("who can become root could not be measured: %s", strings.Join(unread, "; "))
}
return out, nil
}
@@ -0,0 +1,60 @@
package main
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/conditions"
)
// novox/hq ADR 0259 §8, rule 3: the probe fails today — agents run as the operator's account, which has
// passwordless sudo, and the login shell runs their commands as it — and passes only once neither holds.
func TestAnAgentAbleToBecomeRootWhereTheRouterRunsIsSaid(t *testing.T) {
root := escalation{Root: true, Why: "(ALL : ALL) NOPASSWD: ALL"}
none := escalation{Why: "no rule lets it without a password"}
base := func() agentRootFacts {
return agentRootFacts{Machine: "anchor", Trusted: []string{"telegram", "messenger"}, Runtime: "ops",
Answers: map[string]escalation{}}
}
today := base()
today.Agent, today.LoginShell = "ops", true
today.Answers["ops"] = root
got := agentRootObservations(today)
if len(got) != 1 || got[0].Severity != conditions.Urgent || got[0].Kind != kindAgentRoot ||
!strings.Contains(got[0].Summary, "the operator's account") || !strings.Contains(got[0].Summary, "the login shell") {
t.Fatalf("today: %+v", got)
}
agentsMoved := base()
agentsMoved.Agent, agentsMoved.AgentNamed, agentsMoved.LoginShell = "agents", true, true
agentsMoved.Answers["agents"], agentsMoved.Answers["ops"] = none, root
if got := agentRootObservations(agentsMoved); len(got) != 1 || strings.Contains(got[0].Summary, "agents (") ||
!strings.Contains(got[0].Summary, "the login shell") {
t.Errorf("agents of their own account, the login shell still the runtime's: %+v", got)
}
closed := base()
closed.Agent, closed.AgentNamed = "agents", true
closed.Answers["agents"], closed.Answers["ops"] = none, root
if got := agentRootObservations(closed); len(got) != 0 {
t.Errorf("agents of their own account and no login shell there: %+v", got)
}
noAgents := base()
noAgents.Answers["ops"] = root
if got := agentRootObservations(noAgents); len(got) != 0 {
t.Errorf("no agent runs there and no login shell is held: %+v", got)
}
}
// Its words are plain, as every condition's are (novox/hq ADR 0253).
func TestTheRootConditionIsSaidInPlainWords(t *testing.T) {
f := agentRootFacts{Machine: "anchor", Trusted: []string{"telegram"}, Runtime: "ops", Agent: "ops",
Answers: map[string]escalation{"ops": {Root: true, Why: "x"}}}
o := agentRootObservations(f)[0]
if why, ok := conditions.PlainWords(conditions.Words{Headline: o.Headline, Explanation: o.Explanation,
Needs: o.Needs, Resolved: o.Resolved}, "anchor"); !ok {
t.Errorf("not plain: %s", why)
}
}
+6
View File
@@ -135,6 +135,12 @@ func handOver(ctx context.Context, seatName, to string, adding bool) error {
if !ok || nodeName == "" || module == "" { if !ok || nodeName == "" || module == "" {
return fmt.Errorf("the new holder is named <node>/<module>, not %q", to) return fmt.Errorf("the new holder is named <node>/<module>, not %q", to)
} }
// A kinded bench is held once per kind, by the claims themselves (novox/hq ADR 0234 §2, ADR 0259): the
// record of who holds a seat has no kind, so a handover would name one holder for every kind.
if catalogue.KindedBenches[seatName] {
return fmt.Errorf("%s is a kinded bench: each kind is held by the module claiming it, and is not handed "+
"over by `seat` — assign the module that claims the kind, or unassign the one that does", seatName)
}
open, err := openStores(ctx) open, err := openStores(ctx)
if err != nil { if err != nil {
return err return err
+12
View File
@@ -1,6 +1,8 @@
package main package main
import ( import (
"context"
"strings"
"testing" "testing"
"github.com/novox/mesh-controller/internal/catalogue" "github.com/novox/mesh-controller/internal/catalogue"
@@ -62,3 +64,13 @@ func TestAClaimOutsideTheSetIsShownNotHidden(t *testing.T) {
t.Fatalf("a claim outside the set was not shown: %+v", outside) t.Fatalf("a claim outside the set was not shown: %+v", outside)
} }
} }
// A kinded bench is not handed over by `seat`: each kind is held by its claim (novox/hq ADR 0259).
func TestAKindedBenchIsNotHandedOver(t *testing.T) {
for _, bench := range []string{"channel", "intake"} {
err := handOver(context.Background(), bench, "anchor/telegram", false)
if err == nil || !strings.Contains(err.Error(), "is a kinded bench") {
t.Errorf("%s: %v", bench, err)
}
}
}
+1
View File
@@ -361,6 +361,7 @@ func watchWaits(f *signalFacts) []conditions.Observation {
Headline: deliveryName(w.modules, w.repository) + " waiting to start", Headline: deliveryName(w.modules, w.repository) + " waiting to start",
Explanation: walkWaitingWords(w, in, severity), Explanation: walkWaitingWords(w, in, severity),
Needs: waitingNeeds(severity), Needs: waitingNeeds(severity),
Actions: waitingActions(w.id, severity),
Resolved: deliveryName(w.modules, w.repository) + " no longer waiting"}) Resolved: deliveryName(w.modules, w.repository) + " no longer waiting"})
} }
return out return out
+3
View File
@@ -673,6 +673,9 @@ func watchTheMesh(ctx context.Context, open *stores, server *link.Server, bus li
// under the lease and the brake, every act said. // under the lease and the brake, every act said.
healers := newHealing(open, keeper, bus, server.JetStream()) healers := newHealing(open, keeper, bus, server.JetStream())
go healers.keep(watching) go healers.keep(watching)
// And the asker (novox/hq ADR 0259): what needs the operator and names its answers is asked of them,
// and the answer chosen is performed on its warrant.
startAsking(watching, open, server, bus.Conn, keeper)
go forgettingOldHeals(watching, open.inventory) go forgettingOldHeals(watching, open.inventory)
fmt.Printf("watching the mesh: %d signal(s) every %s, %d probe(s) every %s; what is wrong is kept in %s "+ fmt.Printf("watching the mesh: %d signal(s) every %s, %d probe(s) every %s; what is wrong is kept in %s "+
"and said as %s events\n", len(watchedRows()), watchEvery, len(runnableProbes()), doctorEvery, "and said as %s events\n", len(watchedRows()), watchEvery, len(runnableProbes()), doctorEvery,
+2 -2
View File
@@ -3,7 +3,9 @@ module github.com/novox/mesh-controller
go 1.26.0 go 1.26.0
require ( require (
git.novox.be/novox/mesh-sdk/go v0.1.8-0.20261008162031-55090da7e08f
github.com/jackc/pgx/v5 v5.10.0 github.com/jackc/pgx/v5 v5.10.0
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0 github.com/nats-io/nats.go v1.54.0
github.com/novox/mesh-host v0.0.0 github.com/novox/mesh-host v0.0.0
golang.org/x/crypto v0.57.0 golang.org/x/crypto v0.57.0
@@ -19,10 +21,8 @@ require (
github.com/klauspost/compress v1.20.0 // indirect github.com/klauspost/compress v1.20.0 // indirect
github.com/minio/highwayhash v1.0.4 // indirect github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/nats-server/v2 v2.11.17 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect github.com/nats-io/nuid v1.0.1 // indirect
go.uber.org/automaxprocs v1.6.0 // indirect
golang.org/x/sync v0.23.0 // indirect golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect golang.org/x/sys v0.48.0 // indirect
golang.org/x/text v0.42.0 // indirect golang.org/x/text v0.42.0 // indirect
+2 -6
View File
@@ -1,9 +1,7 @@
git.novox.be/novox/mesh-host v0.0.0-20261006095519-3e80b7ae325e h1:g9h4QRaAMg5yaJLwqtb0FoOs23DVGUYpW6qvnQ3oY5A=
git.novox.be/novox/mesh-host v0.0.0-20261006095519-3e80b7ae325e/go.mod h1:VlilMCRZ5yyNXg7SNigNBLr0Gt32jrGw5KSNq5JAVYs=
git.novox.be/novox/mesh-host v0.0.0-20261007120832-bdd44154ccac h1:KvnKtJ2rWeIE/t4GweK+JL0OjKSNxsrVP3/nMdpii8o=
git.novox.be/novox/mesh-host v0.0.0-20261007120832-bdd44154ccac/go.mod h1:VlilMCRZ5yyNXg7SNigNBLr0Gt32jrGw5KSNq5JAVYs=
git.novox.be/novox/mesh-host v0.0.0-20261007162834-56e2ebec4bac h1:yLtFS0pDCCqIE9Zx8hgXEFG9fUWzf8L9WQoKV+Amk1E= git.novox.be/novox/mesh-host v0.0.0-20261007162834-56e2ebec4bac h1:yLtFS0pDCCqIE9Zx8hgXEFG9fUWzf8L9WQoKV+Amk1E=
git.novox.be/novox/mesh-host v0.0.0-20261007162834-56e2ebec4bac/go.mod h1:VlilMCRZ5yyNXg7SNigNBLr0Gt32jrGw5KSNq5JAVYs= git.novox.be/novox/mesh-host v0.0.0-20261007162834-56e2ebec4bac/go.mod h1:VlilMCRZ5yyNXg7SNigNBLr0Gt32jrGw5KSNq5JAVYs=
git.novox.be/novox/mesh-sdk/go v0.1.8-0.20261008162031-55090da7e08f h1:BNvyWq899GwP7F3sY4ACieB5a5fnFAq+sJ9lP6HQ5qI=
git.novox.be/novox/mesh-sdk/go v0.1.8-0.20261008162031-55090da7e08f/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -40,8 +38,6 @@ github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UV
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=
go.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M= golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA= golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To= golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
+53
View File
@@ -0,0 +1,53 @@
package broker
import (
"slices"
"testing"
)
// novox/hq ADR 0259 §6: the controller asks the operator through the router's seat as any user of it, under
// its own name, hears its own warrants, reads its own record, and calls the verbs a warrant chooses.
func TestTheControllerAsksUnderItsOwnNameAndCallsTheVerbsAWarrantChooses(t *testing.T) {
records := Records{Nodes: []string{"anchor"}, Assigned: map[string][]Declared{"anchor": {
{Module: "messenger", Holds: []Seat{operatorChannel()}},
}}}
users, err := Users(records)
if err != nil {
t.Fatal(err)
}
got := perms(t, users[0])
for _, s := range []string{
"mesh.seat.operator-channel.accept.ask.mesh-controller",
"mesh.seat.operator-channel.accept.cancel.mesh-controller",
"$JS.API.DIRECT.GET.KV_messenger_asks.$KV.messenger_asks.mesh-controller.c1",
"mesh.seat.mesh-delivery.tool.release", "mesh.seat.mesh-delivery.tool.stop",
"mesh.seat.node-service-manager.tool.restart.g14", "mesh.seat.mesh-controller.tool.plans",
} {
if !allowed(got.Publish, s) {
t.Errorf("the controller may not publish %s", s)
}
}
for _, s := range []string{
"mesh.seat.operator-channel.accept.ask.mesh-delivery",
"mesh.seat.operator-channel.event.decided.mesh-controller",
// (A direct get of another asker's record is not refused here: the controller holds the whole
// JetStream API, as the only writer of stream definitions.)
"mesh.seat.node-service-manager.tool.stop.g14",
} {
if allowed(got.Publish, s) {
t.Errorf("the controller may publish %s", s)
}
}
if !allowed(got.Subscribe, DecidedSubject) || allowed(got.Subscribe, "mesh.seat.operator-channel.event.decided.mesh-delivery") {
t.Error("the controller does not hear exactly its own warrants")
}
// Its events consumer carries them, so a controller that was away hears what was decided meanwhile.
if !slices.Contains(ControllerFollows, DecidedSubject) {
t.Error("the controller does not follow its warrants")
}
// Without a holder of the seat it is granted no ask at all.
alone, _ := Users(Records{Nodes: []string{"anchor"}, Assigned: map[string][]Declared{}})
if allowed(perms(t, alone[0]).Publish, "mesh.seat.operator-channel.accept.ask.mesh-controller") {
t.Error("asked a seat nobody holds")
}
}
+21 -2
View File
@@ -34,8 +34,15 @@ var (
// LeaseBucket holds the controller's lease (to-be 45 §6): one key, `holder`, which the instance // LeaseBucket holds the controller's lease (to-be 45 §6): one key, `holder`, which the instance
// allowed to act writes by compare-and-set and renews; its revision when taken is the epoch. // allowed to act writes by compare-and-set and renews; its revision when taken is the epoch.
LeaseBucket = BucketName(ControllerSeat, "lease") LeaseBucket = BucketName(ControllerSeat, "lease")
// AskedBucket keeps what the controller asked the operator about its conditions (novox/hq ADR 0259):
// each ask by its id, its options and the actions they stand for, how it ended and whether the
// controller acted on its warrant — so a restart neither asks twice nor acts twice.
AskedBucket = BucketName(ControllerSeat, "asked")
) )
// AskedKeptFor is how long an ask is kept after it was made: a month, as the router keeps its own.
const AskedKeptFor = 30 * 24 * time.Hour
// LeaseTTL is how long the lease's key lives unrenewed (to-be 45 §6): fifteen seconds, renewed // LeaseTTL is how long the lease's key lives unrenewed (to-be 45 §6): fifteen seconds, renewed
// every five. The bucket's age, so the bus forgets a holder that stopped renewing. // every five. The bucket's age, so the bus forgets a holder that stopped renewing.
const LeaseTTL = 15 * time.Second const LeaseTTL = 15 * time.Second
@@ -59,12 +66,12 @@ const (
// IsControllerBucket says a bucket is the controller's own, not a module's state nothing declares. // IsControllerBucket says a bucket is the controller's own, not a module's state nothing declares.
func IsControllerBucket(bucket string) bool { func IsControllerBucket(bucket string) bool {
return bucket == CallsBucket || bucket == HandActsBucket || bucket == ConditionsBucket || return bucket == CallsBucket || bucket == HandActsBucket || bucket == ConditionsBucket ||
bucket == ConditionHistoryBucket || bucket == LeaseBucket bucket == ConditionHistoryBucket || bucket == LeaseBucket || bucket == AskedBucket
} }
// ControllerBuckets are the controller's own buckets, in the order they are asserted. // ControllerBuckets are the controller's own buckets, in the order they are asserted.
func ControllerBuckets() []string { func ControllerBuckets() []string {
return []string{LeaseBucket, CallsBucket, HandActsBucket, ConditionsBucket, ConditionHistoryBucket} return []string{LeaseBucket, CallsBucket, HandActsBucket, ConditionsBucket, ConditionHistoryBucket, AskedBucket}
} }
// ControllerBucketsAsserter is what raising the controller's buckets needs of a connection. // ControllerBucketsAsserter is what raising the controller's buckets needs of a connection.
@@ -149,6 +156,18 @@ func (j *JetStream) EnsureControllerBuckets() error {
}); err != nil { }); err != nil {
return fmt.Errorf("asserting bucket %s: %w", ConditionHistoryBucket, err) return fmt.Errorf("asserting bucket %s: %w", ConditionHistoryBucket, err)
} }
if _, err := js.CreateOrUpdateKeyValue(ctx, jetstream.KeyValueConfig{
Bucket: AskedBucket,
Description: "what the controller asked the operator about its conditions, and what came of each (novox/hq " +
"ADR 0259): written by the controller alone; an ask acted on is acted on once",
History: 1,
TTL: AskedKeptFor,
MaxValueSize: 32 << 10,
MaxBytes: 32 << 20,
Storage: jetstream.FileStorage,
}); err != nil {
return fmt.Errorf("asserting bucket %s: %w", AskedBucket, err)
}
return nil return nil
} }
@@ -1,9 +1,15 @@
package broker package broker
import ( import (
"context"
"slices" "slices"
"strings" "strings"
"testing" "testing"
"time"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/testbus"
) )
// **The controller may write every bucket it writes** (novox/hq to-be 45 §1, issue 269). Writing a // **The controller may write every bucket it writes** (novox/hq to-be 45 §1, issue 269). Writing a
@@ -59,3 +65,34 @@ func TestTheWatchedSignalsMayBeSaidAndHeard(t *testing.T) {
t.Error("the controller may not ask who answers, or hears every API call") t.Error("the controller may not ask who answers, or hears every API call")
} }
} }
// The controller's record of what it asked the operator is bounded (correctness review of 2026-10-08): one
// value a key, a month's age, and a size it cannot outgrow.
func TestWhatTheControllerAskedIsBounded(t *testing.T) {
js, err := Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer js.Close()
if err := js.EnsureControllerBuckets(); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
kv, err := jetstream.New(js.Conn())
if err != nil {
t.Fatal(err)
}
bucket, err := kv.KeyValue(ctx, AskedBucket)
if err != nil {
t.Fatal(err)
}
status, err := bucket.Status(ctx)
if err != nil {
t.Fatal(err)
}
info := status.(*jetstream.KeyValueBucketStatus).StreamInfo()
if status.History() != 1 || status.TTL() != AskedKeptFor || info.Config.MaxBytes <= 0 || info.Config.MaxBytes > 64<<20 {
t.Errorf("history %d, age %s, bytes %d", status.History(), status.TTL(), info.Config.MaxBytes)
}
}
+3 -1
View File
@@ -125,7 +125,9 @@ func ConsumerFor(p Principal) (Consumer, bool) {
// A module that reacts to anything — a module's events or a role's (novox/hq ADR 0121). Watching // A module that reacts to anything — a module's events or a role's (novox/hq ADR 0121). Watching
// a role was missing here, so the one module that does it got no consumer at all: it started, // a role was missing here, so the one module that does it got no consumer at all: it started,
// connected, and its graph stayed empty with nothing anywhere reporting why. // connected, and its graph stayed empty with nothing anywhere reporting why.
if p.Kind != KindModule || (len(p.Consumes) == 0 && len(p.Watches) == 0) { // And one that hears its own answers on a seat it uses (novox/hq ADR 0259 §3): an asker's warrants.
hearsItsOwn := len(SeatTrafficOf(p.Module, nil, p.Uses, nil).Subscribe) > 0
if p.Kind != KindModule || (len(p.Consumes) == 0 && len(p.Watches) == 0 && !hearsItsOwn) {
return Consumer{}, false return Consumer{}, false
} }
perms, err := PermissionsFor(p) perms, err := PermissionsFor(p)
+65
View File
@@ -50,6 +50,12 @@ type Membership struct {
// and refuses, with the reason, what is not on it — the bus enforces only the union over every // and refuses, with the reason, what is not on it — the bus enforces only the union over every
// module on the machine. // module on the machine.
State []StateIssued `json:"state,omitempty"` State []StateIssued `json:"state,omitempty"`
// SeatTraffic is what this module's code may submit, say, hear, take, ask, answer and read on seats
// that name their caller or their kind (novox/hq ADR 0259 §3). The runtime carrying the module
// publishes, takes and answers for it only what is listed here: the bus enforces only the union
// over every module on the machine, so one module's code reaching another's name or kind through
// the runtime is the runtime's to refuse.
SeatTraffic *SeatTraffic `json:"seat-traffic,omitempty"`
} }
// Served is one address a tool is answered on. // Served is one address a tool is answered on.
@@ -78,6 +84,8 @@ type Placements struct {
// Interchangeable is each module whose definition says its instances are the same anywhere, // Interchangeable is each module whose definition says its instances are the same anywhere,
// so the module's plain subject is issued to all of them in one queue. // so the module's plain subject is issued to all of them in one queue.
Interchangeable map[string]bool Interchangeable map[string]bool
// Kinds is every kind held of a kinded bench, by whom and with what capabilities (ADR 0259 §5).
Kinds []KindHeld
} }
// AnswersForTheModule says whether an instance of a module on one machine is issued the module's // AnswersForTheModule says whether an instance of a module on one machine is issued the module's
@@ -109,6 +117,20 @@ func MembershipFor(node string, d Declared, where Placements) Membership {
} }
} }
m.State = stateIssuedFor(d, node) m.State = stateIssuedFor(d, node)
t := SeatTrafficOf(d.Module, d.Holds, d.Uses, d.Watches)
for _, s := range append(append([]Seat{}, d.Uses...), d.Watches...) {
if !s.Kinded {
continue
}
for _, k := range where.Kinds {
if k.Seat == s.Name && !kindListed(t.Kinds, k) {
t.Kinds = append(t.Kinds, k)
}
}
}
if len(t.Publish)+len(t.Subscribe)+len(t.Answers)+len(t.Workers)+len(t.Records)+len(t.Kinds) > 0 {
m.SeatTraffic = &t
}
if len(d.Invokes) > 0 { if len(d.Invokes) > 0 {
m.Reaches = map[string][]string{} m.Reaches = map[string][]string{}
for _, t := range d.Invokes { for _, t := range d.Invokes {
@@ -145,5 +167,48 @@ func PlacementsOf(r Records, interchangeable map[string]bool) Placements {
for _, nodes := range p.Nodes { for _, nodes := range p.Nodes {
sort.Strings(nodes) sort.Strings(nodes)
} }
for node, declared := range r.Assigned {
for _, d := range declared {
for _, s := range d.Holds {
if s.Kinded && s.Kind != "" {
p.Kinds = append(p.Kinds, KindHeld{Seat: s.Name, Kind: s.Kind, Module: d.Module, Node: node,
Capabilities: placedCapabilities(s.Capabilities, d.RunsAs)})
}
}
}
}
sort.Slice(p.Kinds, func(i, j int) bool {
a, b := p.Kinds[i], p.Kinds[j]
if a.Seat != b.Seat {
return a.Seat < b.Seat
}
if a.Kind != b.Kind {
return a.Kind < b.Kind
}
return a.Node < b.Node
})
return p return p
} }
// placedCapabilities is what a kind's claim promises, as far as its placement lets the router believe it
// (novox/hq ADR 0259 §8): `verified-sender` only from a holder that runs as an account of its own, on a bus
// account of its own — never one the machine's runtime carries as the operator's account.
func placedCapabilities(declared []string, runsAs string) []string {
var out []string
for _, c := range declared {
if c == "verified-sender" && runsAs == "" {
continue
}
out = append(out, c)
}
return out
}
func kindListed(list []KindHeld, k KindHeld) bool {
for _, x := range list {
if x.Seat == k.Seat && x.Kind == k.Kind && x.Module == k.Module && x.Node == k.Node {
return true
}
}
return false
}
+77 -2
View File
@@ -63,6 +63,23 @@ type Seat struct {
Emits []string Emits []string
Serves []string Serves []string
Versions []string // protocol versions served beside the current one; empty for v1 only Versions []string // protocol versions served beside the current one; empty for v1 only
// Kinded says the seat is a kinded bench (novox/hq ADR 0234 §2, ADR 0259 §3): each holder claims one
// kind, and its verbs' subjects carry it. Kind is the kind this principal's claim names, for a seat it
// holds.
Kinded bool
Kind string
// ByCaller are the accepts and emits whose last token names the calling module (ADR 0259 §3).
ByCaller []string
// Proofs are the seat's proof verbs: core request and reply, never on a stream (ADR 0259 §3).
Proofs []string
// Records are the holder's buckets, by their full name, each user reads under its own name.
Records []string
// Capabilities are what this principal's claim of a kinded bench promises (ADR 0234 §2).
Capabilities []string
// DeclaredBy is the module that declares the seat. On a kinded bench it alone submits work to a kind
// and answers its proofs (novox/hq ADR 0259 §8): the router, not any user or watcher of the bench.
DeclaredBy string
} }
// A Principal is one user of the bus. Its permissions are derived from what it declares and // A Principal is one user of the bus. Its permissions are derived from what it declares and
@@ -169,8 +186,17 @@ var VerbsTheBusStepAsks = []SeatVerb{{Seat: "node-backup", Verb: "now"}}
// VerbsTheControllerAsksTheDeliveryOwner are the mesh-delivery seat's verbs the controller calls (novox/hq // VerbsTheControllerAsksTheDeliveryOwner are the mesh-delivery seat's verbs the controller calls (novox/hq
// ADR 0239): its self-check reads `stalled`, and healer H2 takes the one transition the table allows // ADR 0239): its self-check reads `stalled`, and healer H2 takes the one transition the table allows
// through `close`. A mesh seat's verb is flat: no machine in the subject. // through `close`. A mesh seat's verb is flat: no machine in the subject.
//
// And, since novox/hq ADR 0259, `release` and `stop`: the controller asks the operator for them about a
// delivery held past its bound, and calls them on the operator's warrant, with its why.
var VerbsTheControllerAsksTheDeliveryOwner = []SeatVerb{{Seat: "mesh-delivery", Verb: "stalled"}, var VerbsTheControllerAsksTheDeliveryOwner = []SeatVerb{{Seat: "mesh-delivery", Verb: "stalled"},
{Seat: "mesh-delivery", Verb: "close"}} {Seat: "mesh-delivery", Verb: "close"}, {Seat: "mesh-delivery", Verb: "release"}, {Seat: "mesh-delivery", Verb: "stop"}}
// VerbsTheControllerActsOnAWarrant are the other seat verbs the controller calls when the operator's warrant
// chooses them (novox/hq ADR 0259): a machine's service restarted, and a walk started or stopped through the
// controller's own `plans`. Named one by one; a node seat's on any machine, a mesh seat's flat.
var VerbsTheControllerActsOnAWarrant = []SeatVerb{{Seat: "node-service-manager", Verb: "restart"},
{Seat: ControllerSeat, Verb: "plans"}}
// perMachineEvents are a node-scoped seat's events about the holder itself, whose last token is the // perMachineEvents are a node-scoped seat's events about the holder itself, whose last token is the
// holder's machine (novox/hq ADR 0219): `paused.<node>`, the build agent saying whether it takes work. // holder's machine (novox/hq ADR 0219): `paused.<node>`, the build agent saying whether it takes work.
@@ -370,6 +396,20 @@ func PermissionsFor(p Principal) (Permissions, error) {
for _, v := range VerbsTheControllerAsksTheDeliveryOwner { for _, v := range VerbsTheControllerAsksTheDeliveryOwner {
pub = append(pub, "mesh.seat."+v.Seat+".tool."+v.Verb) pub = append(pub, "mesh.seat."+v.Seat+".tool."+v.Verb)
} }
// And the verbs a warrant chooses (novox/hq ADR 0259): a node seat's on any machine, its own flat.
for _, v := range VerbsTheControllerActsOnAWarrant {
if v.Seat == ControllerSeat {
pub = append(pub, "mesh.seat."+v.Seat+".tool."+v.Verb)
continue
}
pub = append(pub, "mesh.seat."+v.Seat+".tool."+v.Verb+".*")
}
// And asking the operator (novox/hq ADR 0259): an ask and its cancel under its own name, its warrants
// heard under its own name, the record of its asks read under its own name — as any user of the seat,
// derived the same way, from the seat its holder declares.
tp, ts := SeatTrafficOf(ControllerSeat, nil, p.Uses, nil).grants()
pub = append(pub, tp...)
sub = append(sub, ts...)
// And asks who answers (novox/hq to-be 45 §4, D3): the self-check finds every seat's holder by // And asks who answers (novox/hq to-be 45 §4, D3): the self-check finds every seat's holder by
// the same discovery the console reads. The question only; the answers come to its own inbox. // the same discovery the console reads. The question only; the answers come to its own inbox.
pub = append(pub, "$SRV.INFO") pub = append(pub, "$SRV.INFO")
@@ -525,6 +565,9 @@ func PermissionsFor(p Principal) (Permissions, error) {
// 2b. Events of a role it watches, under the seat's own namespace. Subscribe only: watching a // 2b. Events of a role it watches, under the seat's own namespace. Subscribe only: watching a
// role is hearing what it announced, not taking part in it. // role is hearing what it announced, not taking part in it.
for _, w := range p.Watches { for _, w := range p.Watches {
if w.Kinded {
continue // composed by SeatTrafficOf below
}
for _, e := range w.Emits { for _, e := range w.Emits {
sub = append(sub, seatSubject(w, "event", e)) sub = append(sub, seatSubject(w, "event", e))
} }
@@ -543,6 +586,13 @@ func PermissionsFor(p Principal) (Permissions, error) {
// 3. Seats it holds: full participation. // 3. Seats it holds: full participation.
for _, s := range p.Holds { for _, s := range p.Holds {
if s.isNewTraffic() {
// Composed by SeatTrafficOf below, worker and all; only its tools are served here.
for _, t := range s.Serves {
sub = append(sub, seatToolSubject(s, t, p.Node))
}
continue
}
// Taking work from the role's queue: the worker consumer every holder shares (asked // Taking work from the role's queue: the worker consumer every holder shares (asked
// about, pulled from, acknowledged), on the seat's own stream (novox/hq ADR 0190). A // about, pulled from, acknowledged), on the seat's own stream (novox/hq ADR 0190). A
// holder pulls — asks the consumer for its next message, answered on its own inbox — // holder pulls — asks the consumer for its next message, answered on its own inbox —
@@ -585,7 +635,7 @@ func PermissionsFor(p Principal) (Permissions, error) {
// seat's inbound subject and watch other modules' traffic, nor publish its outbound // seat's inbound subject and watch other modules' traffic, nor publish its outbound
// events and lie about outcomes (design 29 §2). // events and lie about outcomes (design 29 §2).
for _, s := range p.Uses { for _, s := range p.Uses {
for _, a := range s.Accepts { for _, a := range plainVerbs(s, s.Accepts) {
pub = append(pub, seatSubject(s, "accept", a)) pub = append(pub, seatSubject(s, "accept", a))
} }
for _, t := range s.Serves { for _, t := range s.Serves {
@@ -598,6 +648,12 @@ func PermissionsFor(p Principal) (Permissions, error) {
pub = append(pub, stateGrants(stateAccess{Module: p.Module, Node: p.Node, Keeps: p.State, pub = append(pub, stateGrants(stateAccess{Module: p.Module, Node: p.Node, Keeps: p.State,
PerMachine: p.PerMachine, Reads: p.Reads, KeyedReads: p.KeyedReads})...) PerMachine: p.PerMachine, Reads: p.Reads, KeyedReads: p.KeyedReads})...)
// 6. Its traffic on seats that name their caller or their kind, ask proofs or keep records
// (novox/hq ADR 0259 §3).
tp, ts := SeatTrafficOf(p.Module, p.Holds, p.Uses, p.Watches).grants()
pub = append(pub, tp...)
sub = append(sub, ts...)
case KindNodeTools: case KindNodeTools:
// **One process serves what every module on the machine would have served for itself** // **One process serves what every module on the machine would have served for itself**
// (novox/hq ADR 0175). Each carried module's whole tool namespace — the same grant that // (novox/hq ADR 0175). Each carried module's whole tool namespace — the same grant that
@@ -675,6 +731,25 @@ func PermissionsFor(p Principal) (Permissions, error) {
pub = append(pub, stateGrants(stateAccess{Module: d.Module, Node: p.Node, Keeps: stateNames(d.State), pub = append(pub, stateGrants(stateAccess{Module: d.Module, Node: p.Node, Keeps: stateNames(d.State),
PerMachine: perMachineNames(d.State), Reads: d.Reads, KeyedReads: d.KeyedReads})...) PerMachine: perMachineNames(d.State), Reads: d.Reads, KeyedReads: d.KeyedReads})...)
} }
// **Never the traffic of a trusted holder** (novox/hq ADR 0259 §8): the machine's runtime runs as the
// operator's account, which every agent runs as, so a module saying warrants or speaking for a kind
// that proves its sender is never composed into it — refused here, naming it, whatever registration
// let through.
for _, d := range p.Carries {
if why := trustedTraffic(d); why != "" {
return Permissions{}, fmt.Errorf("%s on %s is carried by the machine's runtime, and %s: it runs "+
"as an account of its own, never the runtime's (novox/hq ADR 0259)", d.Module, p.Node, why)
}
}
// **And the seat traffic of the modules it carries** (novox/hq ADR 0259 §3): a bundle reaches the
// bus only through its runtime, so the runtime is granted the union. That one module's code does
// not publish under another's name or kind through it is the runtime's to keep, from the seat
// traffic each module's membership lists.
for _, d := range p.Carries {
tp, ts := SeatTrafficOf(d.Module, d.Holds, d.Uses, d.Watches).grants()
pub = append(pub, tp...)
sub = append(sub, ts...)
}
sub = unique(sub) sub = unique(sub)
pub = unique(pub) pub = unique(pub)
} }
+305
View File
@@ -0,0 +1,305 @@
package broker
import "sort"
// What a module may say and take on the seats it holds, uses and watches, beyond tools (novox/hq ADR
// 0259 §3, to-be 46 §10).
//
// Three rules are added to the ones a seat always had, each a subject whose last token names who may
// publish it, granted to that publisher alone — the way a node seat's event about a machine carries the
// machine (ADR 0219):
//
// - **A verb named by its caller** (`by-caller`). A user of the seat submits that accept, and hears that
// event, under its own module's name and no other: `accept.ask.<module>`, `event.decided.<module>`. The
// holder takes every caller's accept and says the event to any caller. So an ask's asker is a fact the
// server enforces, and a warrant reaches only the asker it is for.
// - **A kinded bench** (ADR 0234 §2). A holder claims one kind and takes its own kind's accepts, says its
// own kind's events and asks its own kind's proofs, and nothing of another kind; a user submits to any
// kind. Each kind has its own worker on the seat's queue, so a holder that is away keeps its work and
// holds up no other kind.
// - **A proof** (`proofs`): core request and reply on `mesh.seat.<seat>.proof.<verb>.<kind>`. No stream's
// subjects cover it, so what travels there — a code typed by the operator — is never persisted. A kinded
// holder asks with its own kind; the modules that watch the seat answer.
//
// And one read: **a holder's records**, a bucket the seat names, read by each user under its own name only
// (`$KV.<bucket>.<module>.>`), so an asker reads the state of its own asks and no other asker's.
// Worker is one durable consumer a holder pulls a seat's work from.
type Worker struct {
Stream string
Consumer string
Filter string
}
// SeatTraffic is one module's seat traffic beyond tools. Publish and Subscribe are subject patterns, in the
// server's wildcards; the runtime that carries the module checks a bundle's request against them, since
// the runtime's own principal holds the union of every module it carries.
type SeatTraffic struct {
// Publish is what it submits (accepts of seats it uses), says (events of seats it holds) and asks
// (proofs of seats it holds a kind of).
Publish []string `json:"publish,omitempty"`
// Subscribe is what it hears: events of seats it uses that are named by caller, events of seats it
// watches, and accepts of seats it holds.
Subscribe []string `json:"subscribe,omitempty"`
// Answers is the proof subjects it answers, as a watcher of a kinded seat.
Answers []string `json:"answers,omitempty"`
// Workers are the work queues it takes from, as a holder.
Workers []Worker `json:"workers,omitempty"`
// Records is the direct-get subjects of the records it reads under its own name.
Records []string `json:"records,omitempty"`
// Kinds are the holders of every kinded bench it uses or watches, with what each promises: the one
// account of which channel is which, and what it can carry, that the router judges an answer by. The
// controller's, from the claims, never a channel's word (novox/hq ADR 0259 §5).
Kinds []KindHeld `json:"kinds,omitempty"`
}
// KindHeld is one kind of a kinded bench and who holds it.
type KindHeld struct {
Seat string `json:"seat"`
Kind string `json:"kind"`
Module string `json:"module"`
Node string `json:"node"`
Capabilities []string `json:"capabilities,omitempty"`
}
// KindedBenches are the seats that may be kinded (ADR 0234 §2): making another is a decision, recorded.
var KindedBenches = map[string]bool{"channel": true, "intake": true}
// WorkerName is the worker a seat's holders pull from: one for the seat, or one per kind on a kinded bench.
func WorkerName(seat, kind string) string {
if kind == "" {
return "SEAT_" + upperSnake(seat) + "_worker"
}
return "SEAT_" + upperSnake(seat) + "_" + upperSnake(kind) + "_worker"
}
func namesVerb(list []string, s string) bool {
for _, x := range list {
if x == s {
return true
}
}
return false
}
// SeatTrafficOf derives one module's seat traffic from the seats it holds, uses and watches. Only seats
// carrying one of the rules above are read: every other seat is composed as it always was.
func SeatTrafficOf(module string, holds, uses, watches []Seat) SeatTraffic {
var t SeatTraffic
for _, s := range holds {
if !s.isNewTraffic() {
continue
}
kind := ""
if s.Kinded {
kind = s.Kind
if kind == "" || !safeSubject.MatchString(kind) {
// A kinded claim without a usable kind is refused at registration; here it is granted
// nothing, which is the same answer at the last place it could be asked.
continue
}
}
if len(s.Accepts) > 0 {
stream := seatStreamName(s.Name)
filter := "mesh.seat." + s.Name + ".accept.>"
if kind != "" {
filter = "mesh.seat." + s.Name + ".accept.*." + kind
}
t.Workers = append(t.Workers, Worker{Stream: stream, Consumer: WorkerName(s.Name, kind), Filter: filter})
}
for _, a := range s.Accepts {
switch {
case kind != "":
t.Subscribe = append(t.Subscribe, seatSubject(s, "accept", a+"."+kind))
case namesVerb(s.ByCaller, a):
t.Subscribe = append(t.Subscribe, seatSubject(s, "accept", a+".*"))
}
}
for _, e := range s.Emits {
switch {
case kind != "":
t.Publish = append(t.Publish, seatSubject(s, "event", e+"."+kind))
case namesVerb(s.ByCaller, e):
t.Publish = append(t.Publish, seatSubject(s, "event", e+".*"))
}
}
if kind != "" {
for _, v := range s.Proofs {
t.Publish = append(t.Publish, seatSubject(s, "proof", v+"."+kind))
}
}
}
for _, s := range uses {
if !s.isNewTraffic() {
continue
}
for _, a := range s.Accepts {
switch {
case namesVerb(s.ByCaller, a):
t.Publish = append(t.Publish, seatSubject(s, "accept", a+"."+module))
case s.Kinded && module == s.DeclaredBy:
// Work for a kind is put on its queue by the bench's own router, and by no other user.
t.Publish = append(t.Publish, seatSubject(s, "accept", a+".*"))
}
}
for _, e := range s.Emits {
if namesVerb(s.ByCaller, e) {
t.Subscribe = append(t.Subscribe, seatSubject(s, "event", e+"."+module))
}
}
for _, b := range s.Records {
if !safeSubject.MatchString(b) {
continue
}
t.Records = append(t.Records, "$JS.API.DIRECT.GET.KV_"+b+".$KV."+b+"."+module+".>")
}
}
for _, w := range watches {
if w.Kinded {
for _, e := range w.Emits {
t.Subscribe = append(t.Subscribe, seatSubject(w, "event", e+".*"))
}
if module == w.DeclaredBy {
// A code is answered by the bench's own router, and by no other watcher.
for _, v := range w.Proofs {
t.Answers = append(t.Answers, seatSubject(w, "proof", v+".*"))
}
}
}
}
t.Publish = unique(t.Publish)
t.Subscribe = unique(t.Subscribe)
t.Answers = unique(t.Answers)
t.Records = unique(t.Records)
sort.Slice(t.Workers, func(i, j int) bool { return t.Workers[i].Consumer < t.Workers[j].Consumer })
return t
}
// grants is the bus permissions seat traffic needs: the subjects themselves, and the JetStream API a
// worker is pulled and acknowledged through and a record is read through.
func (t SeatTraffic) grants() (pub, sub []string) {
pub = append(pub, t.Publish...)
sub = append(sub, t.Subscribe...)
sub = append(sub, t.Answers...)
for _, w := range t.Workers {
pub = append(pub,
"$JS.API.CONSUMER.INFO."+w.Stream+"."+w.Consumer,
"$JS.API.CONSUMER.MSG.NEXT."+w.Stream+"."+w.Consumer,
"$JS.ACK."+w.Stream+"."+w.Consumer+".>")
}
pub = append(pub, t.Records...)
return pub, sub
}
// isNewTraffic says whether a seat carries any of the rules above, so a seat that carries none is
// composed exactly as before them.
func (s Seat) isNewTraffic() bool {
return s.Kinded || len(s.ByCaller) > 0 || len(s.Proofs) > 0 || len(s.Records) > 0
}
// plainVerbs is a seat's accepts or emits with those the rules above compose taken out: a verb named by
// its caller and every verb of a kinded bench are composed by SeatTrafficOf and nowhere else.
func plainVerbs(s Seat, verbs []string) []string {
if s.Kinded {
return nil
}
var out []string
for _, v := range verbs {
if !namesVerb(s.ByCaller, v) {
out = append(out, v)
}
}
return out
}
// SeatTrafficObjects is the work queues and workers the seat traffic of every composed user implies
// (novox/hq ADR 0259 §3): a queue for each seat a holder takes work from, and each holder's worker on it —
// one per kind on a kinded bench, filtered to that kind, so the kinds never take each other's work. Only
// seats carrying the rules above; the mesh's own seats' queues are RaiseSeats'.
func SeatTrafficObjects(users []Principal) ([]Stream, []Consumer) {
streams := map[string]Stream{}
consumers := map[string]Consumer{}
add := func(module string, holds []Seat) {
for _, w := range SeatTrafficOf(module, holds, nil, nil).Workers {
seat := ""
for _, s := range holds {
if seatStreamName(s.Name) == w.Stream {
seat = s.Name
}
}
streams[w.Stream] = Stream{
Name: w.Stream,
Subjects: []string{"mesh.seat." + seat + ".accept.>"},
Retention: RetentionWorkQueue,
MaxAge: 7 * 24 * 60 * 60,
Why: "work submitted to the " + seat + " seat; its holders take it, each kind its own, and it queues while nobody does",
}
consumers[w.Consumer] = Consumer{
Name: w.Consumer,
Stream: w.Stream,
Filters: []string{w.Filter},
AckWaitSeconds: 60,
// No bound on redelivery: a channel away for a day keeps its work, offered again later and
// later by its holder's runtime (novox/hq ADR 0259; the correctness review of 2026-10-08).
MaxDeliver: 0,
Why: module + " holds " + seat + "; it pulls one ask at a time and acknowledges once it has " +
"recorded it, so a crash redelivers rather than loses",
}
}
}
for _, p := range users {
switch p.Kind {
case KindModule:
add(p.Module, p.Holds)
case KindNodeTools:
for _, d := range p.Carries {
add(d.Module, d.Holds)
}
}
}
var ss []Stream
for _, s := range streams {
ss = append(ss, s)
}
sort.Slice(ss, func(i, j int) bool { return ss[i].Name < ss[j].Name })
var cs []Consumer
for _, c := range consumers {
cs = append(cs, c)
}
sort.Slice(cs, func(i, j int) bool { return cs[i].Name < cs[j].Name })
return ss, cs
}
// trustedTraffic is why a module's seat traffic is the trusted holder's (novox/hq ADR 0259 §8), or "": it
// says a seat's event to one caller each (a warrant), or holds a kind of a kinded bench that proves its sender.
func trustedTraffic(d Declared) string {
for _, s := range d.Holds {
for _, e := range s.Emits {
if namesVerb(s.ByCaller, e) {
return "it says " + s.Name + "'s " + e + " to one caller each"
}
}
if s.Kinded && namesVerb(s.Capabilities, "verified-sender") {
return "it holds " + s.Name + " of kind " + s.Kind + ", which proves its sender"
}
}
return ""
}
// TrafficQueues is the work queue of every seat naming its caller or its kind that accepts work, held or not
// (novox/hq ADR 0259 §3): what is submitted before a holder is assigned waits for it.
func TrafficQueues(seats []Seat) []Stream {
var out []Stream
seen := map[string]bool{}
for _, s := range seats {
if len(s.Accepts) == 0 || !s.isNewTraffic() || seen[s.Name] {
continue
}
seen[s.Name] = true
out = append(out, Stream{Name: seatStreamName(s.Name), Subjects: []string{"mesh.seat." + s.Name + ".accept.>"},
Retention: RetentionWorkQueue, MaxAge: 7 * 24 * 60 * 60,
Why: "work submitted to the " + s.Name + " seat; its holders take it, each kind its own, and it queues while nobody does"})
}
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out
}
+340
View File
@@ -0,0 +1,340 @@
package broker
import (
"strings"
"testing"
)
// The seats of novox/hq ADR 0259 §3, as the messenger declares them.
func operatorChannel() Seat {
return Seat{Name: "operator-channel", Scope: "mesh", Accepts: []string{"ask", "cancel"},
Emits: []string{"decided"}, Serves: []string{"open", "history", "notify"},
ByCaller: []string{"ask", "cancel", "decided"}, Records: []string{"messenger_asks"}}
}
func channelSeat(kind string) Seat {
return Seat{Name: "channel", Scope: "mesh", Accepts: []string{"show", "edit", "send"}, Kinded: true, Kind: kind,
DeclaredBy: "messenger"}
}
func intakeSeat(kind string) Seat {
return Seat{Name: "intake", Scope: "mesh", Emits: []string{"choice", "link"}, Proofs: []string{"code"},
Kinded: true, Kind: kind, DeclaredBy: "messenger"}
}
func allowed(patterns []string, subject string) bool {
for _, p := range patterns {
if subjectMatches(p, subject) {
return true
}
}
return false
}
func perms(t *testing.T, p Principal) Permissions {
t.Helper()
got, err := PermissionsFor(p)
if err != nil {
t.Fatal(err)
}
return got
}
func TestAnAskerAsksAndHearsUnderItsOwnNameOnly(t *testing.T) {
asker := Principal{Kind: KindModule, Node: "anchor", Module: "mesh-delivery", Uses: []Seat{operatorChannel()}}
got := perms(t, asker)
for _, s := range []string{
"mesh.seat.operator-channel.accept.ask.mesh-delivery",
"mesh.seat.operator-channel.accept.cancel.mesh-delivery",
"$JS.API.DIRECT.GET.KV_messenger_asks.$KV.messenger_asks.mesh-delivery.a1",
} {
if !allowed(got.Publish, s) {
t.Errorf("an asker may not publish %s", s)
}
}
for _, s := range []string{
"mesh.seat.operator-channel.accept.ask.mesh-controller",
"mesh.seat.operator-channel.accept.ask.*",
"mesh.seat.operator-channel.event.decided.mesh-delivery",
"$JS.API.DIRECT.GET.KV_messenger_asks.$KV.messenger_asks.mesh-controller.a1",
"$KV.messenger_asks.mesh-delivery.a1",
} {
if allowed(got.Publish, s) {
t.Errorf("an asker may publish %s, which is not its own to submit", s)
}
}
if !allowed(got.Subscribe, "mesh.seat.operator-channel.event.decided.mesh-delivery") {
t.Error("an asker does not hear its own warrants")
}
if allowed(got.Subscribe, "mesh.seat.operator-channel.event.decided.mesh-controller") {
t.Error("an asker hears another asker's warrants")
}
// And its own consumer carries its warrants, so a restart catches up.
c, ok := ConsumerFor(asker)
if !ok || !allowed(c.Filters, "mesh.seat.operator-channel.event.decided.mesh-delivery") {
t.Errorf("the asker's consumer does not carry its warrants: %v", c.Filters)
}
}
func TestOnlyTheHolderPublishesAWarrant(t *testing.T) {
users, err := Users(Records{
Nodes: []string{"anchor"},
Assigned: map[string][]Declared{"anchor": {
{Module: "messenger", Holds: []Seat{operatorChannel()}, Uses: []Seat{channelSeat("")},
Watches: []Seat{{Name: "intake", Emits: []string{"choice", "link"}, Kinded: true, Proofs: []string{"code"}, DeclaredBy: "messenger"}}},
{Module: "mesh-delivery", Uses: []Seat{operatorChannel()}},
{Module: "telegram", Holds: []Seat{channelSeat("telegram"), intakeSeat("telegram")}},
}},
})
if err != nil {
t.Fatal(err)
}
for _, u := range users {
got := perms(t, u)
says := allowed(got.Publish, "mesh.seat.operator-channel.event.decided.mesh-delivery")
if says != (u.Module == "messenger") {
t.Errorf("%s %s publish a warrant", u.Username(), map[bool]string{true: "may", false: "may not"}[says])
}
}
}
func TestTheHolderTakesEveryCallersAskThroughItsWorker(t *testing.T) {
got := perms(t, Principal{Kind: KindModule, Node: "anchor", Module: "messenger", Holds: []Seat{operatorChannel()}})
if !allowed(got.Subscribe, "mesh.seat.operator-channel.accept.ask.mesh-delivery") {
t.Error("the router does not take an ask")
}
for _, s := range []string{
"$JS.API.CONSUMER.MSG.NEXT.SEAT_OPERATOR_CHANNEL.SEAT_OPERATOR_CHANNEL_worker",
"$JS.ACK.SEAT_OPERATOR_CHANNEL.SEAT_OPERATOR_CHANNEL_worker.x",
"mesh.seat.operator-channel.event.decided.mesh-controller",
} {
if !allowed(got.Publish, s) {
t.Errorf("the router may not publish %s", s)
}
}
if allowed(got.Publish, "mesh.seat.operator-channel.accept.ask.messenger") {
t.Error("the holder may ask its own seat without using it")
}
}
func TestAKindedHolderReachesItsOwnKindAndNoOther(t *testing.T) {
got := perms(t, Principal{Kind: KindModule, Node: "anchor", Module: "telegram",
Holds: []Seat{channelSeat("telegram"), intakeSeat("telegram")}})
for _, s := range []string{
"mesh.seat.intake.event.choice.telegram", "mesh.seat.intake.event.link.telegram",
"mesh.seat.intake.proof.code.telegram",
"$JS.API.CONSUMER.MSG.NEXT.SEAT_CHANNEL.SEAT_CHANNEL_TELEGRAM_worker",
} {
if !allowed(got.Publish, s) {
t.Errorf("telegram may not publish %s", s)
}
}
for _, s := range []string{
"mesh.seat.intake.event.choice.desktop", "mesh.seat.intake.proof.code.desktop",
"mesh.seat.channel.accept.show.telegram",
"$JS.API.CONSUMER.MSG.NEXT.SEAT_CHANNEL.SEAT_CHANNEL_DESKTOP_worker",
"mesh.seat.operator-channel.event.decided.mesh-delivery",
} {
if allowed(got.Publish, s) {
t.Errorf("telegram may publish %s", s)
}
}
if !allowed(got.Subscribe, "mesh.seat.channel.accept.show.telegram") ||
allowed(got.Subscribe, "mesh.seat.channel.accept.show.desktop") {
t.Error("telegram does not take exactly its own kind's work")
}
if allowed(got.Subscribe, "mesh.seat.intake.proof.code.telegram") {
t.Error("a channel answers its own proofs")
}
}
func TestTheWatcherAnswersProofsAndHearsEveryKind(t *testing.T) {
got := perms(t, Principal{Kind: KindModule, Node: "anchor", Module: "messenger",
Uses: []Seat{channelSeat("")},
Watches: []Seat{{Name: "intake", Emits: []string{"choice"}, Kinded: true, Proofs: []string{"code"}, DeclaredBy: "messenger"}}})
for _, s := range []string{"mesh.seat.intake.event.choice.telegram", "mesh.seat.intake.proof.code.desktop"} {
if !allowed(got.Subscribe, s) {
t.Errorf("the router does not hear %s", s)
}
}
if !allowed(got.Publish, "mesh.seat.channel.accept.show.telegram") {
t.Error("the router cannot send a channel its work")
}
if allowed(got.Publish, "mesh.seat.intake.event.choice.telegram") || allowed(got.Publish, "mesh.seat.intake.proof.code.telegram") {
t.Error("the router may say a channel's answer or proof")
}
}
func TestNoStreamKeepsAProof(t *testing.T) {
users, _ := Users(Records{Nodes: []string{"anchor"}, Assigned: map[string][]Declared{"anchor": {
{Module: "telegram", Holds: []Seat{channelSeat("telegram"), intakeSeat("telegram")}},
{Module: "messenger", Holds: []Seat{operatorChannel()}},
}}})
streams, _ := SeatTrafficObjects(users)
streams = append(streams, MeshStreams()...)
for _, s := range streams {
for _, subject := range s.Subjects {
if subjectMatches(subject, "mesh.seat.intake.proof.code.telegram") {
t.Errorf("%s keeps a proof (%s)", s.Name, subject)
}
}
}
}
func TestEachKindHasAWorkerOfItsOwn(t *testing.T) {
users, _ := Users(Records{Nodes: []string{"anchor"}, Assigned: map[string][]Declared{"anchor": {
{Module: "telegram", Holds: []Seat{channelSeat("telegram")}},
{Module: "desk-channel", Holds: []Seat{channelSeat("desktop")}},
{Module: "messenger", Holds: []Seat{operatorChannel()}},
}}})
streams, workers := SeatTrafficObjects(users)
names := map[string]string{}
for _, w := range workers {
names[w.Name] = strings.Join(w.Filters, ",")
}
want := map[string]string{
"SEAT_CHANNEL_TELEGRAM_worker": "mesh.seat.channel.accept.*.telegram",
"SEAT_CHANNEL_DESKTOP_worker": "mesh.seat.channel.accept.*.desktop",
"SEAT_OPERATOR_CHANNEL_worker": "mesh.seat.operator-channel.accept.>",
}
for n, f := range want {
if names[n] != f {
t.Errorf("worker %s filters %q, want %q", n, names[n], f)
}
}
if len(streams) != 2 {
t.Errorf("want the queues of channel and operator-channel, got %v", streams)
}
}
func TestTheRuntimeIsGrantedTheUnionAndTheMembershipEachModulesShare(t *testing.T) {
telegram := Declared{Module: "telegram", Holds: []Seat{channelSeat("telegram"), intakeSeat("telegram")}}
desk := Declared{Module: "desk-channel", Holds: []Seat{channelSeat("desktop"), intakeSeat("desktop")}}
got := perms(t, Principal{Kind: KindNodeTools, Node: "anchor", Module: RuntimeModule, Carries: []Declared{telegram, desk}})
for _, s := range []string{"mesh.seat.intake.event.choice.telegram", "mesh.seat.intake.event.choice.desktop"} {
if !allowed(got.Publish, s) {
t.Errorf("the runtime may not publish %s for a module it carries", s)
}
}
m := MembershipFor("anchor", telegram, Placements{})
if m.SeatTraffic == nil || !allowed(m.SeatTraffic.Publish, "mesh.seat.intake.event.choice.telegram") ||
allowed(m.SeatTraffic.Publish, "mesh.seat.intake.event.choice.desktop") {
t.Errorf("telegram's membership does not list exactly its own kind: %+v", m.SeatTraffic)
}
if plain := MembershipFor("anchor", Declared{Module: "plain"}, Placements{}); plain.SeatTraffic != nil {
t.Error("a module with no such seat is given seat traffic")
}
}
func TestASeatWithoutTheNewRulesIsComposedAsBefore(t *testing.T) {
old := Seat{Name: "node-build-agent", Scope: "node", Accepts: []string{"build"}, Emits: []string{"built"}}
got := perms(t, Principal{Kind: KindModule, Node: "anchor", Module: "builder", Holds: []Seat{old}})
for _, s := range []string{"mesh.seat.node-build-agent.event.built",
"$JS.API.CONSUMER.MSG.NEXT.SEAT_NODE_BUILD_AGENT.SEAT_NODE_BUILD_AGENT_worker"} {
if !allowed(got.Publish, s) {
t.Errorf("an old seat's holder lost %s", s)
}
}
if !allowed(got.Subscribe, "mesh.seat.node-build-agent.accept.build") {
t.Error("an old seat's holder lost its accept")
}
}
// The router learns which channel is which, and what each promises, from the controller's membership:
// the claims, never a channel's word (ADR 0259 §5).
func TestTheRoutersMembershipNamesEveryKindAndItsCapabilities(t *testing.T) {
tg := channelSeat("telegram")
tg.Capabilities = []string{"choice", "verified-sender"}
desk := channelSeat("desktop")
desk.Capabilities = []string{"choice"}
router := Declared{Module: "messenger", Holds: []Seat{operatorChannel()}, Uses: []Seat{channelSeat("")}}
records := Records{Nodes: []string{"anchor", "laptop"}, Assigned: map[string][]Declared{
"anchor": {router, {Module: "telegram", Holds: []Seat{tg}, RunsAs: "telegram"}},
"laptop": {{Module: "desk-channel", Holds: []Seat{desk}}},
}}
where := PlacementsOf(records, nil)
m := MembershipFor("anchor", router, where)
if m.SeatTraffic == nil || len(m.SeatTraffic.Kinds) != 2 {
t.Fatalf("the router is not told the kinds: %+v", m.SeatTraffic)
}
byKind := map[string]KindHeld{}
for _, k := range m.SeatTraffic.Kinds {
byKind[k.Kind] = k
}
if k := byKind["telegram"]; k.Module != "telegram" || k.Node != "anchor" || !namesVerb(k.Capabilities, "verified-sender") {
t.Errorf("telegram is %+v", k)
}
if k := byKind["desktop"]; k.Module != "desk-channel" || namesVerb(k.Capabilities, "verified-sender") {
t.Errorf("the desk is %+v", k)
}
if other := MembershipFor("anchor", Declared{Module: "mesh-delivery", Uses: []Seat{operatorChannel()}}, where); other.SeatTraffic != nil && len(other.SeatTraffic.Kinds) > 0 {
t.Error("an asker is told the channels")
}
}
// novox/hq ADR 0259 §8: only the bench's own router answers its proofs and puts work on a kind's queue.
func TestOnlyTheBenchsRouterAnswersProofsAndSubmitsWork(t *testing.T) {
other := perms(t, Principal{Kind: KindModule, Node: "anchor", Module: "eavesdropper",
Uses: []Seat{channelSeat("")},
Watches: []Seat{{Name: "intake", Emits: []string{"choice"}, Kinded: true, Proofs: []string{"code"}, DeclaredBy: "messenger"}}})
if allowed(other.Subscribe, "mesh.seat.intake.proof.code.telegram") {
t.Error("a watcher that is not the router answers codes")
}
if allowed(other.Publish, "mesh.seat.channel.accept.show.telegram") {
t.Error("a user that is not the router puts work on a kind's queue")
}
if !allowed(other.Subscribe, "mesh.seat.intake.event.choice.telegram") {
t.Error("a watcher no longer hears the bench's events")
}
}
// novox/hq ADR 0259 §8: the machine's runtime runs as the operator's account; it never carries a module
// that says warrants or speaks for a kind proving its sender, and such a module has its own account.
func TestTheMachinesRuntimeNeverCarriesATrustedHolder(t *testing.T) {
tg := channelSeat("telegram")
tg.Capabilities = []string{"choice", "verified-sender"}
for name, d := range map[string]Declared{
"the router": {Module: "messenger", Holds: []Seat{operatorChannel()}},
"a verified channel": {Module: "telegram", Holds: []Seat{tg}},
} {
if _, err := PermissionsFor(Principal{Kind: KindNodeTools, Node: "anchor", Module: RuntimeModule,
Carries: []Declared{d}}); err == nil || !strings.Contains(err.Error(), "an account of its own") {
t.Errorf("%s was composed into the machine's runtime: %v", name, err)
}
}
desk := channelSeat("desktop")
desk.Capabilities = []string{"choice"}
if _, err := PermissionsFor(Principal{Kind: KindNodeTools, Node: "anchor", Module: RuntimeModule,
Carries: []Declared{{Module: "desk-channel", Holds: []Seat{desk}}}}); err != nil {
t.Errorf("a channel proving nothing was refused: %v", err)
}
users, err := Users(Records{Nodes: []string{"anchor"}, Assigned: map[string][]Declared{"anchor": {
{Module: RuntimeModule}, {Module: "telegram", Holds: []Seat{tg}, RunsAs: "telegram"},
{Module: "messenger", Holds: []Seat{operatorChannel()}, RunsAs: "messenger"},
}}})
if err != nil {
t.Fatal(err)
}
for _, u := range users {
if u.Kind == KindNodeTools {
for _, d := range u.Carries {
if d.RunsAs != "" {
t.Errorf("the machine's runtime carries %s", d.Module)
}
}
if _, err := PermissionsFor(u); err != nil {
t.Errorf("the runtime could not be composed: %v", err)
}
}
}
}
// novox/hq ADR 0259 §8: verified-sender reaches the router only from a holder of its own account.
func TestVerifiedSenderIsBelievedOnlyFromAHolderOfItsOwnAccount(t *testing.T) {
if got := placedCapabilities([]string{"choice", "verified-sender"}, ""); namesVerb(got, "verified-sender") {
t.Errorf("a carried holder keeps verified-sender: %v", got)
}
if got := placedCapabilities([]string{"choice", "verified-sender"}, "telegram"); !namesVerb(got, "verified-sender") {
t.Errorf("a holder of its own account lost verified-sender: %v", got)
}
}
+11
View File
@@ -270,8 +270,19 @@ var ControllerFollows = []string{
// seat to check before it merges — every machine of the facts snapshot composed with the change. // seat to check before it merges — every machine of the facts snapshot composed with the change.
// Appended, because the index is a name. // Appended, because the index is a name.
moduleEventSubject("gitea", "pull.updated"), moduleEventSubject("gitea", "pull.updated"),
// **The operator's answers to what the controller asked** (novox/hq ADR 0259): the router's warrant, or
// the end of an ask without one, said to the controller alone under its own name. On the stream, so a
// controller that was away hears what was decided meanwhile. Appended, because the index is a name.
DecidedSubject,
} }
// AsksSeat is the seat an ask is made on and its warrant heard from (novox/hq ADR 0259): the router's.
const AsksSeat = "operator-channel"
// DecidedSubject is where the router says the controller's warrants: the seat's event named by the
// controller as its caller.
var DecidedSubject = seatEventSubject(AsksSeat, "decided."+ControllerSeat)
// The provider standing events, by their local names. Written here as well as in the catalogue // The provider standing events, by their local names. Written here as well as in the catalogue
// (catalogue.ProvisionerEvents), which this package cannot import; a test keeps them agreeing. // (catalogue.ProvisionerEvents), which this package cannot import; a test keeps them agreeing.
const ( const (
+2 -2
View File
@@ -24,8 +24,8 @@ accounts {
jetstream: enabled jetstream: enabled
users = [ users = [
{ user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: { { user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>", "$KV.SEAT_NODE_BUILD_AGENT_cancelled.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.assignment.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-build-machine.tool.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.checked", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.plan-moved", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.rolled-back", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.mesh-delivery.tool.close", "mesh.seat.mesh-delivery.tool.stalled", "mesh.seat.node-backup.tool.backed-up.*", "mesh.seat.node-backup.tool.now.*", "mesh.seat.node-build-agent.accept.>", "mesh.seat.node-build-agent.tool.>", "mesh.seat.node-intrusion-prevention.tool.banned.*"] } publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>", "$KV.SEAT_NODE_BUILD_AGENT_cancelled.>", "$KV.mesh-controller_asked.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.assignment.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-build-machine.tool.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.checked", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.plan-moved", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.rolled-back", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.mesh-controller.tool.plans", "mesh.seat.mesh-delivery.tool.close", "mesh.seat.mesh-delivery.tool.release", "mesh.seat.mesh-delivery.tool.stalled", "mesh.seat.mesh-delivery.tool.stop", "mesh.seat.node-backup.tool.backed-up.*", "mesh.seat.node-backup.tool.now.*", "mesh.seat.node-build-agent.accept.>", "mesh.seat.node-build-agent.tool.>", "mesh.seat.node-intrusion-prevention.tool.banned.*", "mesh.seat.node-service-manager.tool.restart.*"] }
subscribe: { allow: ["$JS.API.>", "$JS.EVENT.ADVISORY.CONSUMER.DELETED.>", "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "$SRV.STATS", "$SRV.STATS.mesh-controller", "$SRV.STATS.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered", "mesh.mod.*.event.provisioner.retirement", "mesh.mod.gitea.event.pull.merged", "mesh.mod.gitea.event.pull.updated", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built"] } subscribe: { allow: ["$JS.API.>", "$JS.EVENT.ADVISORY.CONSUMER.DELETED.>", "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "$SRV.STATS", "$SRV.STATS.mesh-controller", "$SRV.STATS.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered", "mesh.mod.*.event.provisioner.retirement", "mesh.mod.gitea.event.pull.merged", "mesh.mod.gitea.event.pull.updated", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built", "mesh.seat.operator-channel.event.decided.mesh-controller"] }
allow_responses: { max: 1, ttl: "1m" } allow_responses: { max: 1, ttl: "1m" }
} } } }
{ user: "enrol.one", password: "$2a$11$eeeeeeeeeeeeeeeeeeeeee", permissions: { { user: "enrol.one", password: "$2a$11$eeeeeeeeeeeeeeeeeeeeee", permissions: {
+29 -5
View File
@@ -50,6 +50,9 @@ type Declared struct {
// Checks are the module's own tools its health asks, each `<module>.<tool>` (novox/hq ADR 0240, to-be // Checks are the module's own tools its health asks, each `<module>.<tool>` (novox/hq ADR 0240, to-be
// 48 §3): the machine's node-engine asks them of its own node tools, and is granted that and no more. // 48 §3): the machine's node-engine asks them of its own node tools, and is granted that and no more.
Checks []string Checks []string
// RunsAs is the account the module runs as in a runtime of its own (novox/hq ADR 0259 §8): it is never
// carried by the machine's runtime, and reaches the bus on its own account.
RunsAs string
} }
// Records is what composing a user list needs to know about the mesh, and nothing more. // Records is what composing a user list needs to know about the mesh, and nothing more.
@@ -75,7 +78,7 @@ type Records struct {
// is a mesh that cannot be told anything, and there is no state of the records in which that is // is a mesh that cannot be told anything, and there is no state of the records in which that is
// correct. // correct.
func Users(r Records) ([]Principal, error) { func Users(r Records) ([]Principal, error) {
out := []Principal{{Kind: KindController}} out := []Principal{{Kind: KindController, Uses: asksSeatOf(r)}}
for _, node := range sortedCopy(r.Nodes) { for _, node := range sortedCopy(r.Nodes) {
witness := false witness := false
@@ -120,10 +123,13 @@ func Users(r Records) ([]Principal, error) {
}) })
} }
if runtimeHere { if runtimeHere {
out = append(out, Principal{ var carried []Declared
Kind: KindNodeTools, Node: node, Module: RuntimeModule, for _, d := range r.Assigned[node] {
Carries: append([]Declared(nil), r.Assigned[node]...), if d.RunsAs == "" {
}) carried = append(carried, d)
}
}
out = append(out, Principal{Kind: KindNodeTools, Node: node, Module: RuntimeModule, Carries: carried})
} }
} }
for _, node := range sortedCopy(r.Enrolling) { for _, node := range sortedCopy(r.Enrolling) {
@@ -189,3 +195,21 @@ func sortedNames(in map[string][]string) []string {
// controllerModule is the controller's module: the machine assigned it witnesses its upgrades. // controllerModule is the controller's module: the machine assigned it witnesses its upgrades.
const controllerModule = "mesh-controller" const controllerModule = "mesh-controller"
// asksSeatOf is the seat an ask is made on, as its holder declares it (novox/hq ADR 0259): the controller
// asks the operator through it like any other user, and is granted what its declaration names for a caller.
// None while nothing holds it.
func asksSeatOf(r Records) []Seat {
for _, node := range sortedCopy(r.Nodes) {
for _, d := range r.Assigned[node] {
for _, s := range d.Holds {
if s.Name == AsksSeat && namesVerb(s.ByCaller, "ask") {
seat := s
seat.Kind, seat.Capabilities = "", nil
return []Seat{seat}
}
}
}
}
return nil
}
+15 -1
View File
@@ -1066,9 +1066,23 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
} }
owner[fmt.Sprint(process["id"])] = RuntimeModule owner[fmt.Sprint(process["id"])] = RuntimeModule
out = append(out, process) out = append(out, process)
// What each module's bundles are given is read as the account the runtime runs as. // And a runtime of its own for each module of its own account, after it (novox/hq ADR 0259 §8).
owns, err := r.ownRuntimes(with)
if err != nil {
return nil, err
}
for _, p := range owns {
id := fmt.Sprint(p["id"])
owner[id] = strings.TrimSuffix(id, "."+OwnRuntimeID())
out = append(out, p)
}
// What each module's bundles are given is read as the account the runtime runs as — not what a
// module of its own account is given, which its own account reads.
words := map[string]map[string]string{} words := map[string]map[string]string{}
for _, m := range r.Modules { for _, m := range r.Modules {
if m.RunsAs != "" {
continue
}
w, err := bundleWords(m, with) w, err := bundleWords(m, with)
if err != nil { if err != nil {
return nil, err return nil, err
+161
View File
@@ -0,0 +1,161 @@
package catalogue
import (
"strings"
"testing"
)
// The router of novox/hq ADR 0259: it declares the operator's seat and the two kinded benches.
func router() Manifest {
return Manifest{Module: "messenger", Tools: []string{"open", "history", "notify"},
State: []StateDeclaration{{Name: "asks"}}, RunsAs: "messenger", SecretsOwner: "messenger",
DefinesSeats: []SeatDeclaration{
{Name: "operator-channel", Scope: ScopeMesh, Accepts: []string{"ask", "cancel"}, Emits: []string{"decided"},
ByCaller: []string{"ask", "cancel", "decided"}, Records: []string{"asks"},
Serves: []Verb{{Name: "open"}, {Name: "history"}, {Name: "notify"}}},
{Name: "channel", Scope: ScopeMesh, Kinded: true, Accepts: []string{"show", "edit", "send"}},
{Name: "intake", Scope: ScopeMesh, Kinded: true, Emits: []string{"choice", "link"}, Proofs: []string{"code"}},
},
Claims: []Claim{{Name: "operator-channel", Scope: ScopeMesh}},
Uses: []string{"channel"}}
}
func aChannel(module, kind string) Manifest {
return Manifest{Module: module, Claims: []Claim{
{Name: "channel", Scope: ScopeMesh, Kind: kind}, {Name: "intake", Scope: ScopeMesh, Kind: kind}}}
}
func TestTwoChannelsOfDifferentKindsHoldTheBenches(t *testing.T) {
shelf := Shelf{"messenger": router(), "telegram": aChannel("telegram", "telegram"),
"desk-channel": aChannel("desk-channel", "desktop")}
if got := problemsFor(t, shelf); got != "" {
t.Fatalf("two kinds were refused: %s", got)
}
for _, m := range shelf {
if got := declaredSeatProblems(m); len(got) > 0 {
t.Fatalf("%s: %v", m.Module, got)
}
}
}
func TestASecondClaimOfOneKindIsRefused(t *testing.T) {
got := problemsFor(t, Shelf{"messenger": router(), "telegram": aChannel("telegram", "telegram"),
"telegram-two": aChannel("telegram-two", "telegram")})
if !strings.Contains(got, `of kind "telegram", which telegram already claims`) {
t.Fatalf("a second holder of one kind stood: %s", got)
}
}
func TestAKindedBenchNeedsAKindAndNoOtherSeatTakesOne(t *testing.T) {
got := problemsFor(t, Shelf{"messenger": router(), "nameless": aChannel("nameless", "")})
if !strings.Contains(got, "claims the kinded bench channel and names no kind") {
t.Fatalf("a claim without a kind stood: %s", got)
}
odd := Manifest{Module: "odd", Claims: []Claim{{Name: "operator-channel", Scope: ScopeMesh, Kind: "telegram"}}}
got = problemsFor(t, Shelf{"messenger": router(), "odd": odd})
if !strings.Contains(got, "only a kinded bench takes a kind") {
t.Fatalf("a kind on a seat that is not kinded stood: %s", got)
}
dotted := problemsFor(t, Shelf{"messenger": router(), "dotted": aChannel("dotted", "a.b")})
if !strings.Contains(dotted, "not a usable name") {
t.Fatalf("a kind that would widen a subject stood: %s", dotted)
}
}
func TestOnlyChannelAndIntakeAreKinded(t *testing.T) {
m := Manifest{Module: "x", DefinesSeats: []SeatDeclaration{{Name: "pager", Kinded: true, Accepts: []string{"page"}}}}
if got := strings.Join(declaredSeatProblems(m), "; "); !strings.Contains(got, "only channel and intake are kinded") {
t.Fatalf("another kinded bench was declared: %s", got)
}
}
func TestTheNewRulesAreHeldToWhatTheSeatSays(t *testing.T) {
m := Manifest{Module: "x", DefinesSeats: []SeatDeclaration{{Name: "thing", Accepts: []string{"do"},
ByCaller: []string{"undo"}, Proofs: []string{"code"}, Records: []string{"nothing"}}}}
got := strings.Join(declaredSeatProblems(m), "; ")
for _, want := range []string{"names thing.undo by its caller, which the seat neither accepts nor emits",
"declares proofs on thing, which is not kinded", `read its records "nothing", which it keeps no state of`} {
if !strings.Contains(got, want) {
t.Errorf("not refused: %q in %s", want, got)
}
}
}
// Two kinds on one machine are two holders, and one kind on two machines is a second claimant.
func TestEachKindIsItsOwnHolderWhenResolved(t *testing.T) {
modules := []Manifest{aChannel("telegram", "telegram"), aChannel("desk-channel", "desktop")}
held, problems := checkClaims(modules, Node{Name: "anchor"}, nil, nil)
if len(problems) > 0 || len(held) != 4 {
t.Fatalf("two kinds on one machine: held %v, problems %v", held, problems)
}
_, problems = checkClaims([]Manifest{aChannel("telegram", "telegram")}, Node{Name: "home"}, held, nil)
if len(problems) == 0 {
t.Fatal("one kind held on two machines was not refused")
}
}
// A channel's capabilities come from the fixed vocabulary, and only a kinded claim carries any.
func TestCapabilitiesAreTheVocabularysAndOnlyOnAKindedClaim(t *testing.T) {
good := aChannel("telegram", "telegram")
good.Claims[0].Capabilities = []string{"deliver", "choice", "verified-sender", "max-length:4096"}
good.RunsAs = "telegram"
if got := problemsFor(t, Shelf{"messenger": router(), "telegram": good}); got != "" {
t.Fatalf("the vocabulary was refused: %s", got)
}
bad := aChannel("telegram", "telegram")
bad.Claims[0].Capabilities = []string{"trusted", "max-length:lots"}
got := problemsFor(t, Shelf{"messenger": router(), "telegram": bad})
for _, w := range []string{`"trusted"`, `"max-length:lots"`} {
if !strings.Contains(got, w+", which channel-capabilities/1 does not have") {
t.Errorf("%s was not refused: %s", w, got)
}
}
odd := Manifest{Module: "odd", Claims: []Claim{{Name: "operator-channel", Scope: ScopeMesh, Capabilities: []string{"deliver"}}}}
if got := problemsFor(t, Shelf{"messenger": router(), "odd": odd}); !strings.Contains(got, "only a kinded bench's claim carries them") {
t.Errorf("capabilities on a seat that is not kinded stood: %s", got)
}
}
// novox/hq ADR 0259 §8: a module saying warrants, or speaking for a kind that proves its sender, runs as an
// account of its own — never carried by the machine's runtime, which runs as the operator's account.
func TestATrustedHolderMustRunAsAnAccountOfItsOwn(t *testing.T) {
r := router()
r.RunsAs = ""
if got := problemsFor(t, Shelf{"messenger": r}); !strings.Contains(got, "messenger must run as an account of its own") {
t.Errorf("a router on the machine's runtime stood: %s", got)
}
tg := aChannel("telegram", "telegram")
tg.Claims[0].Capabilities = []string{"choice", "verified-sender"}
if got := problemsFor(t, Shelf{"messenger": router(), "telegram": tg}); !strings.Contains(got, "telegram must run as an account of its own") {
t.Errorf("a verified channel on the machine's runtime stood: %s", got)
}
desk := aChannel("desk-channel", "desktop")
desk.Claims[0].Capabilities = []string{"choice"}
if got := problemsFor(t, Shelf{"messenger": router(), "desk-channel": desk}); got != "" {
t.Errorf("a channel proving nothing was held to it: %s", got)
}
}
func TestRunsAsIsAnAccountOfTheModulesOwn(t *testing.T) {
ok := Manifest{Module: "telegram", RunsAs: "telegram", SecretsOwner: "telegram",
OwnSecrets: OwnSecrets{"broker": {Path: "/var/lib/telegram/broker"}},
Resources: []map[string]any{{"id": "account", "type": "user", "name": "telegram"}}}
if got := RunsAsProblems(ok); len(got) != 0 {
t.Fatalf("a sound runs-as was refused: %v", got)
}
for want, change := range map[string]func(*Manifest){
"never root": func(m *Manifest) { m.RunsAs, m.SecretsOwner = "root", "root" },
"not an account name": func(m *Manifest) { m.RunsAs = "${machine:account}" },
"which it does not make": func(m *Manifest) { m.Resources = nil },
"declares no own secret": func(m *Manifest) { m.OwnSecrets = nil },
"they are the account's own": func(m *Manifest) { m.SecretsOwner = "" },
} {
m := ok
m.Resources = append([]map[string]any(nil), ok.Resources...)
m.OwnSecrets = OwnSecrets{"broker": {Path: "/x"}}
change(&m)
if got := strings.Join(RunsAsProblems(m), "; "); !strings.Contains(got, want) {
t.Errorf("want %q, got %q", want, got)
}
}
}
+15
View File
@@ -64,6 +64,13 @@ type Claim struct {
// text/template over one piece — its fields and `module` — in the tool's own grammar (novox/hq ADR // text/template over one piece — its fields and `module` — in the tool's own grammar (novox/hq ADR
// 0255). The data is the mesh's, the format the holder's, as a module's facts template is. // 0255). The data is the mesh's, the format the holder's, as a module's facts template is.
Renders map[string]string `json:"renders,omitempty"` Renders map[string]string `json:"renders,omitempty"`
// Kind is the kind this module holds a kinded bench as (novox/hq ADR 0234 §2, ADR 0259): `telegram`,
// `desktop`. Refused on any other seat, and a second claim of one kind is refused.
Kind string `json:"kind,omitempty"`
// Capabilities are what a channel of this kind promises, from the fixed vocabulary
// channel-capabilities/1 (novox/hq ADR 0234 §2): the router judges an answer by these, read from the
// controller's record of this claim and never from the channel.
Capabilities []string `json:"capabilities,omitempty"`
} }
// ServesFor is what this claim offers a seat's protocol: the verbs it names, else the module's // ServesFor is what this claim offers a seat's protocol: the verbs it names, else the module's
@@ -635,6 +642,13 @@ type Manifest struct {
// cannot use. // cannot use.
SecretsOwner string `json:"secrets-owner,omitempty"` SecretsOwner string `json:"secrets-owner,omitempty"`
// RunsAs is the account this module's tools bundle runs as, in a runtime of its own on a bus account of
// its own (novox/hq ADR 0259 §8): never the machine's runtime, which runs as the operator's account and
// carries every module on the machine. The account is one the module makes (a `user` resource of that
// name), owns its secrets (`secrets-owner`), and is neither root nor the operator's. Required of a module
// that says a warrant, or speaks for a channel kind that proves its sender.
RunsAs string `json:"runs-as,omitempty"`
// Keeps is where this module wants every operator-sealed secret in the mesh written — the // Keeps is where this module wants every operator-sealed secret in the mesh written — the
// vault's field, and so far nobody else's (novox/hq ADR 0085, amended). // vault's field, and so far nobody else's (novox/hq ADR 0085, amended).
// //
@@ -1610,6 +1624,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
// prefix. Whether a seat anybody names exists, and whether a holder answers for it, are // prefix. Whether a seat anybody names exists, and whether a holder answers for it, are
// facts about the catalogue and are checked at registration (CatalogueProblems). // facts about the catalogue and are checked at registration (CatalogueProblems).
problems = append(problems, declaredSeatProblems(m)...) problems = append(problems, declaredSeatProblems(m)...)
problems = append(problems, RunsAsProblems(m)...)
if m.Computed != "" && len(m.Resources) > 0 { if m.Computed != "" && len(m.Resources) > 0 {
// One or the other. A module that both ships files and has them computed would leave // One or the other. A module that both ships files and has them computed would leave
// nobody able to say where a given file came from. // nobody able to say where a given file came from.
+13 -3
View File
@@ -115,6 +115,16 @@ type Held struct {
Node string Node string
Module string Module string
Site string Site string
// Kind is the kind a kinded bench is held as (novox/hq ADR 0234 §2): each kind is its own holder.
Kind string
}
// heldKey is what one holder holds: the seat, and its kind on a kinded bench.
func heldKey(claim, kind string) string {
if kind == "" {
return canonicalSeat(claim)
}
return canonicalSeat(claim) + "/" + kind
} }
// Resolution is what a node should run, and why. // Resolution is what a node should run, and why.
@@ -864,7 +874,7 @@ func checkClaims(modules []Manifest, node Node, elsewhere []Held, holdings []Hel
// **One seat under either of its names** (novox/hq ADR 0122): a manifest registered before // **One seat under either of its names** (novox/hq ADR 0122): a manifest registered before
// a rename claims the former name, and one written after it the current — two claimants of // a rename claims the former name, and one written after it the current — two claimants of
// one seat, compared by the seat they resolve to and not by how each spelled it. // one seat, compared by the seat they resolve to and not by how each spelled it.
seat := canonicalSeat(c.Name) seat := heldKey(c.Name, c.Kind)
if other, taken := byScope[scope][seat]; taken { if other, taken := byScope[scope][seat]; taken {
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%s and %s both claim %q, and only one thing may hold it per %s", "%s and %s both claim %q, and only one thing may hold it per %s",
@@ -873,14 +883,14 @@ func checkClaims(modules []Manifest, node Node, elsewhere []Held, holdings []Hel
} }
byScope[scope][seat] = m.Module byScope[scope][seat] = m.Module
held = append(held, Held{Claim: c.Name, Scope: scope, Node: node.Name, held = append(held, Held{Claim: c.Name, Scope: scope, Node: node.Name,
Module: m.Module, Site: node.Site}) Module: m.Module, Site: node.Site, Kind: c.Kind})
} }
} }
// And against the rest of the mesh, for the scopes that reach past this machine. // And against the rest of the mesh, for the scopes that reach past this machine.
for _, h := range held { for _, h := range held {
for _, e := range elsewhere { for _, e := range elsewhere {
if e.Node == node.Name || canonicalSeat(e.Claim) != canonicalSeat(h.Claim) || e.Scope != h.Scope { if e.Node == node.Name || heldKey(e.Claim, e.Kind) != heldKey(h.Claim, h.Kind) || e.Scope != h.Scope {
continue continue
} }
switch h.Scope { switch h.Scope {
+82
View File
@@ -165,6 +165,10 @@ func (r Resolution) runtimeProcess(with Rendering) (map[string]any, error) {
if with.Adopted && m.Filtering != nil { if with.Adopted && m.Filtering != nil {
continue continue
} }
if m.RunsAs != "" {
// Served by a runtime of its own, on its own account (ownRuntimes): never the machine's.
continue
}
words, err := bundleWords(m, with) words, err := bundleWords(m, with)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -223,6 +227,84 @@ func (r Resolution) runtimeProcess(with Rendering) (map[string]any, error) {
return process, nil return process, nil
} }
// OwnRuntimeID names the process a module of its own account is served by (novox/hq ADR 0259 §8).
func OwnRuntimeID() string { return "own-runtime" }
// ownRuntimes are the processes the modules of their own account are served by (novox/hq ADR 0259 §8): each
// the machine's runtime program — the same build, run from the same source — serving that one module alone,
// as the module's own account, on the module's own bus credential. Never the machine's runtime, which runs
// as the operator's account and carries every module on the machine.
func (r Resolution) ownRuntimes(with Rendering) ([]map[string]any, error) {
var own []Manifest
for _, m := range r.Modules {
if m.RunsAs != "" && !(with.Adopted && m.Filtering != nil) {
own = append(own, m)
}
}
if len(own) == 0 {
return nil, nil
}
var runtime *Manifest
for i := range r.Modules {
if r.Modules[i].Module == RuntimeModule {
runtime = &r.Modules[i]
}
}
if runtime == nil || len(runtime.Bundles) != 1 || runtime.Bundles[0].Binary == "" {
return nil, fmt.Errorf("%s runs as its own account in a runtime of its own, and %s is not here to run it "+
"from: assign %s to %s first (novox/hq ADR 0259)", own[0].Module, RuntimeModule, RuntimeModule, r.Node)
}
program := runtime.Bundles[0]
var out []map[string]any
for _, m := range own {
credential, declared := m.OwnSecrets["broker"]
if !declared {
return nil, fmt.Errorf("%s runs as its own account and declares no own secret broker", m.Module)
}
var served, restartOn []string
for _, b := range m.Bundles {
for _, load := range b.Loads {
if launcher, has := b.Launchers[load]; has {
load = launcher
}
served = append(served, m.Module+"="+BundlePath(m.Module, b.Name)+"/"+load)
}
if len(b.Loads) > 0 {
restartOn = append(restartOn, m.Module+"."+BundleID(b.Name))
}
}
if len(served) == 0 {
return nil, fmt.Errorf("%s runs as its own account and its build produced no bundle to serve", m.Module)
}
sort.Strings(served)
restartOn = append(restartOn, m.Module+"."+NeedID("broker"))
sort.Strings(restartOn)
env := map[string]string{RuntimeToolModules: strings.Join(served, ","), RuntimeBrokerFile: credential.Path}
words, err := bundleWords(m, with)
if err != nil {
return nil, err
}
if len(words) > 0 {
body, err := json.Marshal(map[string]map[string]string{m.Module: words})
if err != nil {
return nil, err
}
env[RuntimeToolEnv] = string(body)
}
process := map[string]any{
"id": m.Module + "." + OwnRuntimeID(), "type": "process", "name": m.Module + "-runtime",
"source": program.Source, "digest": program.Digest,
"run": []any{"./" + program.Binary}, "env": env, "restart-on": toAny(restartOn),
"user": m.RunsAs,
}
if err := artifactsInto(process, RuntimeModule, with); err != nil {
return nil, err
}
out = append(out, process)
}
return out, nil
}
func toAny(in []string) []any { func toAny(in []string) []any {
out := make([]any, 0, len(in)) out := make([]any, 0, len(in))
for _, s := range in { for _, s := range in {
+46
View File
@@ -457,3 +457,49 @@ func TestAGoToolsBundleIsServedByItsBinary(t *testing.T) {
t.Error("a Go bundle loading a file it does not contain was admitted") t.Error("a Go bundle loading a file it does not contain was admitted")
} }
} }
// novox/hq ADR 0259 §8: a module of its own account is served by a runtime of its own — the machine's
// runtime program, as that account, on that module's own credential — and never by the machine's runtime,
// which runs as the operator's account and is given none of its words.
func TestAModuleOfItsOwnAccountIsServedByARuntimeOfItsOwn(t *testing.T) {
with := Rendering{ArtifactStore: "anchor.internal:5101",
Needed: map[string]map[string]string{RuntimeModule: {"broker": "sealed-credential"}, "telegram": {"broker": "own"}}}
goRuntime := Manifest{Module: RuntimeModule, Version: "1",
OwnSecrets: OwnSecrets{"broker": {Path: "/var/lib/mesh/" + RuntimeModule + "/broker"}},
Build: &Build{Artifacts: []Artifact{{Name: "runtime", Kind: ArtifactBundle, Language: "go",
System: "arch", From: "cmd/node-tools"}}}}
goRuntime, err := goRuntime.Resolve([]Built{{Name: "runtime", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + RuntimeModule + "/runtime/blobs/" + bundleDigest, Digest: bundleDigest}})
if err != nil {
t.Fatal(err)
}
telegram := aToolsModule(t, "telegram", "tools/index.js")
telegram.RunsAs, telegram.SecretsOwner = "telegram", "telegram"
telegram.OwnSecrets = OwnSecrets{"broker": {Path: "/var/lib/telegram/broker"}}
out, err := Resolution{Node: "anchor", Account: "ops",
Modules: []Manifest{aToolsModule(t, "nftables", "tools/index.js"), telegram, goRuntime}}.Declaration(with)
if err != nil {
t.Fatal(err)
}
machine := fileNamed(out, RuntimeModule+"."+RuntimeProcessID())
if served := machine["env"].(map[string]string)[RuntimeToolModules]; strings.Contains(served, "telegram") || !strings.Contains(served, "nftables") {
t.Errorf("the machine's runtime serves %q", served)
}
own := fileNamed(out, "telegram."+OwnRuntimeID())
if own == nil {
t.Fatalf("telegram has no runtime of its own: %v", ids(out))
}
env := own["env"].(map[string]string)
if own["user"] != "telegram" || fmt.Sprint(own["run"]) != "[./node-tools]" ||
env[RuntimeBrokerFile] != "/var/lib/telegram/broker" ||
env[RuntimeToolModules] != "telegram="+BundleRoot+"/telegram/tools/tools/index.js" {
t.Errorf("its own runtime: user %v run %v env %v", own["user"], own["run"], env)
}
if _, told := env[RuntimeOperatorAccount]; told {
t.Error("a runtime of a module's own account is told the operator's account")
}
// Without the machine's runtime to run it from, it is refused in words.
if _, err := (Resolution{Node: "anchor", Modules: []Manifest{telegram}}).ownRuntimes(with); err == nil {
t.Error("a module of its own account composed without a runtime program")
}
}
+222
View File
@@ -2,6 +2,7 @@ package catalogue
import ( import (
"fmt" "fmt"
"regexp"
"sort" "sort"
"strings" "strings"
) )
@@ -51,6 +52,35 @@ type SeatDeclaration struct {
// owns its own, which is why a seat is also the answer for a module that needs retention // owns its own, which is why a seat is also the answer for a module that needs retention
// its events cannot have. // its events cannot have.
RetainSeconds int `json:"retain-seconds,omitempty"` RetainSeconds int `json:"retain-seconds,omitempty"`
// Kinded makes the seat a kinded bench (novox/hq ADR 0234 §2, ADR 0259 §3): its holders are different
// modules, each claiming one kind, and each verb's subject carries the kind. Only the benches in
// KindedBenches may be kinded; making another is a decision, recorded.
Kinded bool `json:"kinded,omitempty"`
// ByCaller are accepts and emits whose subject's last token names the calling module (ADR 0259 §3): a
// user submits such an accept, and hears such an event, under its own name and no other.
ByCaller []string `json:"by-caller,omitempty"`
// Proofs are verbs carried as core request and reply, never on a stream: what travels on them (a code
// the operator typed) is never kept (ADR 0259 §3). On a kinded bench a holder asks with its own kind and
// the modules watching the seat answer.
Proofs []string `json:"proofs,omitempty"`
// Records are state buckets of the declaring module that each user reads under its own name — the
// keys `<user>.…` and no other (ADR 0259 §3).
Records []string `json:"records,omitempty"`
}
// KindedBenches are the seats that may be kinded (novox/hq ADR 0234 §2): `channel` sends to the operator,
// `intake` takes what the operator answers. Another is a decision, recorded, as ADR 0223 asks of a bench.
var KindedBenches = map[string]bool{"channel": true, "intake": true}
// NamedByCaller says whether one of the seat's verbs is named by its caller.
func (s SeatDeclaration) NamedByCaller(verb string) bool {
for _, v := range s.ByCaller {
if v == verb {
return true
}
}
return false
} }
// At is this declaration's scope, with the default applied. Mesh by default, because a seat // At is this declaration's scope, with the default applied. Mesh by default, because a seat
@@ -132,6 +162,7 @@ func declaredSeatProblems(m Manifest) []string {
"%s declares %s.%s, which is not a usable verb", m.Module, s.Name, v)) "%s declares %s.%s, which is not a usable verb", m.Module, s.Name, v))
} }
} }
problems = append(problems, trafficProblems(m, s)...)
} }
for _, u := range m.Uses { for _, u := range m.Uses {
@@ -142,6 +173,56 @@ func declaredSeatProblems(m Manifest) []string {
return problems return problems
} }
// trafficProblems is what one declaration of the rules of ADR 0259 §3 can be judged on alone.
func trafficProblems(m Manifest, s SeatDeclaration) []string {
var problems []string
if s.Kinded && !KindedBenches[s.Name] {
problems = append(problems, fmt.Sprintf(
"%s declares %s as a kinded bench; only channel and intake are kinded, and another is a "+
"decision, recorded (novox/hq ADR 0234)", m.Module, s.Name))
}
if s.Kinded && len(s.ByCaller) > 0 {
problems = append(problems, fmt.Sprintf(
"%s declares %s kinded and names verbs by their caller; a kinded bench's subjects carry the kind",
m.Module, s.Name))
}
for _, v := range s.ByCaller {
inAccepts, inEmits := false, false
for _, a := range s.Accepts {
inAccepts = inAccepts || a == v
}
for _, e := range s.Emits {
inEmits = inEmits || e == v
}
if !inAccepts && !inEmits {
problems = append(problems, fmt.Sprintf(
"%s names %s.%s by its caller, which the seat neither accepts nor emits", m.Module, s.Name, v))
}
}
for _, v := range s.Proofs {
if !name.MatchString(v) || strings.Contains(v, ".") {
problems = append(problems, fmt.Sprintf("%s declares the proof %s.%s, which is not a usable verb",
m.Module, s.Name, v))
}
}
if len(s.Proofs) > 0 && !s.Kinded {
problems = append(problems, fmt.Sprintf(
"%s declares proofs on %s, which is not kinded; a proof is asked by a holder of a kind",
m.Module, s.Name))
}
for _, r := range s.Records {
kept := false
for _, st := range m.State {
kept = kept || st.Name == r
}
if !kept {
problems = append(problems, fmt.Sprintf(
"%s says %s's users read its records %q, which it keeps no state of", m.Module, s.Name, r))
}
}
return problems
}
// A Shelf is every manifest the mesh has registered, by module name. // A Shelf is every manifest the mesh has registered, by module name.
type Shelf map[string]Manifest type Shelf map[string]Manifest
@@ -182,8 +263,19 @@ func CatalogueProblems(shelf Shelf) []string {
return ok return ok
} }
// Who claims each kind of a kinded bench, so a second claim of one kind is refused (ADR 0234 §2).
kindsTaken := map[string]string{}
for _, module := range shelfOrder(shelf) { for _, module := range shelfOrder(shelf) {
m := shelf[module] m := shelf[module]
for _, c := range m.Claims {
if c.Kind != "" {
if _, isModuleSeat := declared[c.Name]; !isModuleSeat {
problems = append(problems, fmt.Sprintf(
"%s claims %s of kind %q, and only a kinded bench takes a kind", module, c.Name, c.Kind))
}
}
}
// A `uses` naming nothing is where ADR 0110's guarantee lands under a derived set: the // A `uses` naming nothing is where ADR 0110's guarantee lands under a derived set: the
// same refusal, at the same moment, from a set nobody maintains by hand. // same refusal, at the same moment, from a set nobody maintains by hand.
@@ -214,6 +306,7 @@ func CatalogueProblems(shelf Shelf) []string {
} }
continue continue
} }
problems = append(problems, kindProblems(module, c, s, kindsTaken)...)
if c.At() != s.At() { if c.At() != s.At() {
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%s claims %s at scope %q, and %s declares it at %s", "%s claims %s at scope %q, and %s declares it at %s",
@@ -228,6 +321,16 @@ func CatalogueProblems(shelf Shelf) []string {
} }
} }
} }
// **A trusted holder runs as its own account** (novox/hq ADR 0259 §8): a module saying warrants, or
// speaking for a kind that proves its sender, is never carried by a machine's runtime.
for _, module := range shelfOrder(shelf) {
m := shelf[module]
if why := TrustedHolding(m, declared); why != "" && m.RunsAs == "" {
problems = append(problems, fmt.Sprintf(
"%s must run as an account of its own (runs-as): %s, and the machine's runtime runs as the "+
"operator's account, which every agent runs as (novox/hq ADR 0259)", module, why))
}
}
// A read of a module's state that module does not keep (novox/hq ADR 0201) — said only where the // A read of a module's state that module does not keep (novox/hq ADR 0201) — said only where the
// owner is on the shelf, as a consumer may be installed before its emitter. // owner is on the shelf, as a consumer may be installed before its emitter.
var manifests []Manifest var manifests []Manifest
@@ -239,6 +342,63 @@ func CatalogueProblems(shelf Shelf) []string {
return problems return problems
} }
// ChannelCapabilities is the fixed vocabulary `channel-capabilities/1` (novox/hq ADR 0234 §2): a word
// outside it is refused. `max-length:<N>` takes a number.
var ChannelCapabilities = map[string]bool{
"deliver": true, "reaches-away": true, "loud": true, "silent": true, "edit": true,
"reaches-when-mesh-down": true, "private": true,
"choice": true, "reply": true, "threads": true, "operator-first": true,
"verified-sender": true, "exact-render": true, "code-factor": true, "key-factor": true,
}
var maxLength = regexp.MustCompile(`^max-length:[1-9][0-9]{0,6}$`)
// capabilityProblems are the words of a claim outside the vocabulary, and capabilities on a claim of a
// seat that is not kinded.
func capabilityProblems(module string, c Claim, kinded bool) []string {
if len(c.Capabilities) == 0 {
return nil
}
if !kinded {
return []string{fmt.Sprintf("%s claims %s with capabilities, and only a kinded bench's claim carries them",
module, c.Name)}
}
var problems []string
for _, w := range c.Capabilities {
if !ChannelCapabilities[w] && !maxLength.MatchString(w) {
problems = append(problems, fmt.Sprintf(
"%s claims %s with the capability %q, which channel-capabilities/1 does not have", module, c.Name, w))
}
}
return problems
}
// kindProblems is a claim judged against a declared seat's kind: a kinded bench takes one claim per kind,
// a usable name; any other seat takes none.
func kindProblems(module string, c Claim, s SeatDeclaration, taken map[string]string) []string {
if problems := capabilityProblems(module, c, s.Kinded); len(problems) > 0 {
return problems
}
switch {
case !s.Kinded && c.Kind != "":
return []string{fmt.Sprintf("%s claims %s of kind %q, and only a kinded bench takes a kind",
module, c.Name, c.Kind)}
case !s.Kinded:
return nil
case c.Kind == "":
return []string{fmt.Sprintf("%s claims the kinded bench %s and names no kind", module, c.Name)}
case !name.MatchString(c.Kind) || strings.Contains(c.Kind, "."):
return []string{fmt.Sprintf("%s claims %s of kind %q, which is not a usable name", module, c.Name, c.Kind)}
}
key := c.Name + "/" + c.Kind
if first, ok := taken[key]; ok && first != module {
return []string{fmt.Sprintf("%s claims %s of kind %q, which %s already claims; a kind has one holder",
module, c.Name, c.Kind, first)}
}
taken[key] = module
return nil
}
// unserved is what a seat's protocol promises and the claimant does not answer. Only the tools // unserved is what a seat's protocol promises and the claimant does not answer. Only the tools
// are checked: `accepts` and `emits` are wired by the runtime from the declaration, while a tool // are checked: `accepts` and `emits` are wired by the runtime from the declaration, while a tool
// is code the module either has or has not written — under the claim's serves, or among its own. // is code the module either has or has not written — under the claim's serves, or among its own.
@@ -266,3 +426,65 @@ func shelfOrder(shelf Shelf) []string {
sort.Strings(out) sort.Strings(out)
return out return out
} }
var accountName = regexp.MustCompile(`^[a-z_][a-z0-9_-]{0,30}$`)
// RunsAsProblems is what one manifest's `runs-as` is held to (novox/hq ADR 0259 §8): an account of the
// module's own making — a `user` resource of that name — that owns its secrets, with a bus account of its own,
// and that is neither root nor the operator's.
func RunsAsProblems(m Manifest) []string {
if m.RunsAs == "" {
return nil
}
var problems []string
say := func(format string, a ...any) { problems = append(problems, fmt.Sprintf(format, a...)) }
switch {
case !accountName.MatchString(m.RunsAs):
say("%s runs as %q, which is not an account name of the module's own", m.Module, m.RunsAs)
return problems
case m.RunsAs == "root":
say("%s runs as root; a module of its own account runs as an account it makes, never root", m.Module)
}
made := false
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "user" && fmt.Sprint(r["name"]) == m.RunsAs {
made = true
}
}
if !made {
say("%s runs as %s, which it does not make: a user resource named %s", m.Module, m.RunsAs, m.RunsAs)
}
if _, has := m.OwnSecrets["broker"]; !has {
say("%s runs as its own account and declares no own secret broker: its runtime reaches the bus on an "+
"account of its own", m.Module)
}
if m.SecretsOwner != m.RunsAs {
say("%s runs as %s, and its secrets belong to %q: they are the account's own", m.Module, m.RunsAs, m.SecretsOwner)
}
return problems
}
// TrustedHolding is why a module must run as its own account (novox/hq ADR 0259 §8), or "": it holds a seat
// whose events it says to one caller each (a warrant), or speaks for a kind of a kinded bench that proves
// its sender. Neither may be carried by the machine's runtime, which runs as the operator's account.
func TrustedHolding(m Manifest, declared map[string]SeatDeclaration) string {
for _, c := range m.Claims {
s, ok := declared[c.Name]
if !ok {
continue
}
for _, e := range s.Emits {
if s.NamedByCaller(e) {
return fmt.Sprintf("it holds %s, whose %s it says to one caller each", c.Name, e)
}
}
if s.Kinded {
for _, capability := range c.Capabilities {
if capability == "verified-sender" {
return fmt.Sprintf("it holds %s of kind %s, which proves its sender", c.Name, c.Kind)
}
}
}
}
return ""
}
+12 -1
View File
@@ -53,8 +53,19 @@ type Action struct {
Verb string `json:"verb"` Verb string `json:"verb"`
Machine string `json:"machine,omitempty"` Machine string `json:"machine,omitempty"`
Arguments map[string]string `json:"arguments,omitempty"` Arguments map[string]string `json:"arguments,omitempty"`
// Level is how much proof its answer needs (novox/hq ADR 0234 §8, ADR 0259): LevelAcknowledge for what
// any granted principal may already do, LevelApprove for what only the operator's proven word does.
// The controller asks for every action, and performs the one chosen on the warrant the router issues.
Level string `json:"level,omitempty"`
} }
// The assurance levels an action's answer needs (novox/hq ADR 0234 §8): acknowledge, approve. Destroy is
// not asked for by any condition: nothing carries its second proof yet.
const (
LevelAcknowledge = "acknowledge"
LevelApprove = "approve"
)
// The two verdicts an explanation opens with. // The two verdicts an explanation opens with.
const ( const (
NothingToDo = "Nothing for you to do." NothingToDo = "Nothing for you to do."
@@ -66,7 +77,7 @@ const (
// only kind of answer a desk click performs until answers are authorised (novox/hq ADR 0258). Its cause // only kind of answer a desk click performs until answers are authorised (novox/hq ADR 0258). Its cause
// marks it as an answer, which the hand-act log does not count as a repair. // marks it as an answer, which the hand-act log does not count as a repair.
func SilenceAction(key string) Action { func SilenceAction(key string) Action {
return Action{Label: "Silence for a week", Verb: "mesh-controller.conditions", return Action{Label: "Silence for a week", Verb: "mesh-controller.conditions", Level: LevelAcknowledge,
Arguments: map[string]string{"silence": key, "for": "7d", "why": "", "cause": CauseOperatorAnswer}} Arguments: map[string]string{"silence": key, "for": "7d", "why": "", "cause": CauseOperatorAnswer}}
} }
+52 -7
View File
@@ -34,9 +34,12 @@ func (i *Inventory) BusRecords(ctx context.Context) (broker.Records, error) {
// that seat promises. **Across the whole catalogue, not one manifest**: a seat is declared by // that seat promises. **Across the whole catalogue, not one manifest**: a seat is declared by
// one module and held by another, which is the whole reason a seat exists (ADR 0118). // one module and held by another, which is the whole reason a seat exists (ADR 0118).
seats := map[string]catalogue.SeatDeclaration{} seats := map[string]catalogue.SeatDeclaration{}
// And who declared each, so a seat's records are named as the declaring module's buckets (ADR 0259).
declarers := map[string]string{}
for _, m := range declared { for _, m := range declared {
for _, s := range m.DefinesSeats { for _, s := range m.DefinesSeats {
seats[s.Name] = s seats[s.Name] = s
declarers[s.Name] = m.Module
} }
} }
// And the mesh's own, which carry protocol too (novox/hq ADR 0121). Added after the modules' // And the mesh's own, which carry protocol too (novox/hq ADR 0121). Added after the modules'
@@ -78,7 +81,7 @@ func (i *Inventory) BusRecords(ctx context.Context) (broker.Records, error) {
"%s is assigned to %s and is not in the catalogue, so what it may say cannot "+ "%s is assigned to %s and is not in the catalogue, so what it may say cannot "+
"be derived", module, n.Name) "be derived", module, n.Name)
} }
d := declaredFor(m, seats) d := declaredFor(m, seats, declarers)
// And the state offered to it as a seat's holder by the modules beside it (novox/hq ADR 0255). // And the state offered to it as a seat's holder by the modules beside it (novox/hq ADR 0255).
// For this machine's key alone (novox/hq ADR 0260): a bar shows its own machine's draw. // For this machine's key alone (novox/hq ADR 0260): a bar shows its own machine's draw.
for _, sr := range catalogue.ReadsGranted(m, onMachine(declared, modules), n.Name) { for _, sr := range catalogue.ReadsGranted(m, onMachine(declared, modules), n.Name) {
@@ -121,7 +124,7 @@ func onMachine(declared map[string]catalogue.Manifest, modules []string) []catal
// declaredFor is one module's manifest as the composer needs it: what it says about itself, and the // declaredFor is one module's manifest as the composer needs it: what it says about itself, and the
// protocol of every seat it holds or uses. // protocol of every seat it holds or uses.
func declaredFor(m catalogue.Manifest, seats map[string]catalogue.SeatDeclaration) broker.Declared { func declaredFor(m catalogue.Manifest, seats map[string]catalogue.SeatDeclaration, declarers map[string]string) broker.Declared {
// A consumed name is a module's event unless it names a seat, and only somebody holding the seat // A consumed name is a module's event unless it names a seat, and only somebody holding the seat
// set can tell (novox/hq ADR 0121). Split here, because the composer cannot look at a name and // set can tell (novox/hq ADR 0121). Split here, because the composer cannot look at a name and
// know — and a role's event read as a module's is a subscription to a namespace nobody owns. // know — and a role's event read as a module's is a subscription to a namespace nobody owns.
@@ -132,6 +135,16 @@ func declaredFor(m catalogue.Manifest, seats map[string]catalogue.SeatDeclaratio
if named { if named {
// A seat's event when the seat says it; else the event of the module of that name — a seat and // A seat's event when the seat says it; else the event of the module of that name — a seat and
// the module holding it may share a name (mesh-delivery, novox/hq ADR 0239). // the module holding it may share a name (mesh-delivery, novox/hq ADR 0239).
if s, isASeat := seats[emitter]; isASeat && s.Kinded {
// A kinded bench's event is named `<event>` or `<event>.*` and heard from every kind; its
// proofs are answered by whoever watches it (ADR 0259 §3).
ev := strings.TrimSuffix(event, ".*")
if catalogue.SeatSays(s.Emits, ev) {
watches = append(watches, broker.Seat{Name: s.Name, Scope: s.Scope, Emits: []string{ev},
Kinded: true, Proofs: s.Proofs, DeclaredBy: declarers[s.Name]})
continue
}
}
if s, isASeat := seats[emitter]; isASeat && catalogue.SeatSays(s.Emits, event) { if s, isASeat := seats[emitter]; isASeat && catalogue.SeatSays(s.Emits, event) {
watches = append(watches, broker.Seat{Name: s.Name, Emits: []string{event}}) watches = append(watches, broker.Seat{Name: s.Name, Emits: []string{event}})
continue continue
@@ -155,6 +168,8 @@ func declaredFor(m catalogue.Manifest, seats map[string]catalogue.SeatDeclaratio
Reads: m.Reads, Reads: m.Reads,
// And the tools its health asks (novox/hq ADR 0240): the machine's node-engine is granted them. // And the tools its health asks (novox/hq ADR 0240): the machine's node-engine is granted them.
Checks: catalogue.HealthChecks(m), Checks: catalogue.HealthChecks(m),
// And whether it runs as an account of its own (novox/hq ADR 0259 §8).
RunsAs: m.RunsAs,
} }
// Whether it can be given an account at all: delivered as its own secret named broker, so one // Whether it can be given an account at all: delivered as its own secret named broker, so one
// that declares none has nowhere to read it (novox/hq issue 195). // that declares none has nowhere to read it (novox/hq issue 195).
@@ -168,12 +183,15 @@ func declaredFor(m catalogue.Manifest, seats map[string]catalogue.SeatDeclaratio
// Every seat with a protocol, the mesh's own included. One that says only who does a job is // Every seat with a protocol, the mesh's own included. One that says only who does a job is
// not here and grants nothing, which is most of them. // not here and grants nothing, which is most of them.
if s, hasAProtocol := seats[c.Name]; hasAProtocol { if s, hasAProtocol := seats[c.Name]; hasAProtocol {
d.Holds = append(d.Holds, asSeat(s)) held := asSeat(s, declarers[s.Name])
held.Kind = c.Kind
held.Capabilities = c.Capabilities
d.Holds = append(d.Holds, held)
} }
} }
for _, name := range m.Uses { for _, name := range m.Uses {
if s, declaredSomewhere := seats[name]; declaredSomewhere { if s, declaredSomewhere := seats[name]; declaredSomewhere {
d.Uses = append(d.Uses, asSeat(s)) d.Uses = append(d.Uses, asSeat(s, declarers[s.Name]))
} }
} }
return d return d
@@ -204,9 +222,17 @@ func (i *Inventory) DeclaredBuckets(ctx context.Context) ([]broker.Bucket, error
return out, nil return out, nil
} }
func asSeat(s catalogue.SeatDeclaration) broker.Seat { func asSeat(s catalogue.SeatDeclaration, declarer string) broker.Seat {
return broker.Seat{Name: s.Name, Scope: s.Scope, Accepts: s.Accepts, Emits: s.Emits, seat := broker.Seat{Name: s.Name, Scope: s.Scope, Accepts: s.Accepts, Emits: s.Emits,
Serves: catalogue.VerbNames(s.Serves)} Serves: catalogue.VerbNames(s.Serves), Kinded: s.Kinded, ByCaller: s.ByCaller, Proofs: s.Proofs,
DeclaredBy: declarer}
// A seat's records are its declaring module's buckets, named as the bus holds them (ADR 0259 §3).
for _, r := range s.Records {
if declarer != "" {
seat.Records = append(seat.Records, broker.BucketName(declarer, r))
}
}
return seat
} }
// MeshSeats are the mesh's own seats that carry a protocol, as the bus needs them: what to make a work // MeshSeats are the mesh's own seats that carry a protocol, as the bus needs them: what to make a work
@@ -277,3 +303,22 @@ func heldHere(claimed []broker.Seat, holdings []catalogue.Held, node, module str
} }
return out return out
} }
// DeclaredTrafficSeats is every seat any registered module declares that names its caller or its kind
// (novox/hq ADR 0259 §3), as the bus needs it: assigned or not, so its work queue exists from registration.
func (i *Inventory) DeclaredTrafficSeats(ctx context.Context) ([]broker.Seat, error) {
declared, err := i.Catalogue(ctx)
if err != nil {
return nil, fmt.Errorf("cannot read the catalogue: %w", err)
}
var out []broker.Seat
for _, m := range declared {
for _, s := range m.DefinesSeats {
seat := asSeat(s, m.Module)
if seat.Kinded || len(seat.ByCaller) > 0 {
out = append(out, seat)
}
}
}
return out, nil
}
+4
View File
@@ -47,6 +47,10 @@ var Contracts = map[string]Contract{
KindPullUpdated: {Unordered: "a pull request's head, asked to be checked: each head is its own commit, and its " + KindPullUpdated: {Unordered: "a pull request's head, asked to be checked: each head is its own commit, and its " +
"verdict is set on that commit alone, so a head heard late is checked and judged as itself and never " + "verdict is set on that commit alone, so a head heard late is checked and judged as itself and never " +
"stands for a newer one (novox/hq to-be 45 §9)"}, "stands for a newer one (novox/hq to-be 45 §9)"},
KindDecided: {Unordered: "the router's word on one ask, by the ask's id: an ask is ended once, by compare-and-set " +
"at the router, and the controller acts on it once, recording that it did under the ask's id — so a word " +
"heard again, or late, does nothing more (novox/hq ADR 0259)",
Tests: []string{"TestAWarrantIsActedOnOnce"}},
KindCatchUp: {Unordered: "a catalogue asking what it missed: answered from the record, whenever asked"}, KindCatchUp: {Unordered: "a catalogue asking what it missed: answered from the record, whenever asked"},
KindProvisioner: {Unordered: "a provider's newest word about a consumer, said again every fifteen minutes " + KindProvisioner: {Unordered: "a provider's newest word about a consumer, said again every fifteen minutes " +
"while it holds (ADR 0224): the condition keeps the last observed, and S8 says when the words stop. " + "while it holds (ADR 0224): the condition keeps the last observed, and S8 says when the words stop. " +
+10
View File
@@ -49,6 +49,16 @@ type HandAct struct {
Kind string `json:"kind,omitempty"` Kind string `json:"kind,omitempty"`
// Carried is what such a push moved, one "module from → to" per module, so the log says it. // Carried is what such a push moved, one "module from → to" per module, so the log says it.
Carried []string `json:"carried,omitempty"` Carried []string `json:"carried,omitempty"`
// Via, Ask, Proofs and RequestedBy are an act the operator chose on a warrant (novox/hq ADR 0234 §8, ADR
// 0259): the channel it came through (module and kind, and how the sender was known), the ask's id, the
// proofs present (P1, P2, P3), and what asked (a condition's key). By then names the operator as that
// kind's identity. Absent from every other act.
Via string `json:"via,omitempty"`
Ask string `json:"ask,omitempty"`
Proofs []string `json:"proofs,omitempty"`
RequestedBy string `json:"requested-by,omitempty"`
// Outcome is what came of an act recorded after it was done: done, or the verb's refusal.
Outcome string `json:"outcome,omitempty"`
} }
// KindRecordedBuilds is a push that only moved recorded builds: the person's word their `record` policy // KindRecordedBuilds is a push that only moved recorded builds: the person's word their `record` policy
+3
View File
@@ -44,6 +44,9 @@ const (
// KindPullUpdated is the forge announcing a pull request's new head: checked before it merges // KindPullUpdated is the forge announcing a pull request's new head: checked before it merges
// (novox/hq to-be 45 §9). // (novox/hq to-be 45 §9).
KindPullUpdated = "pull-updated" KindPullUpdated = "pull-updated"
// KindDecided is the router's warrant for an ask the controller made, or that ask's end without one
// (novox/hq ADR 0259): said to the controller alone, under its own name.
KindDecided = "decided"
) )
// Control is one thing a node or a module said, as the controller must act on it. // Control is one thing a node or a module said, as the controller must act on it.
+3 -1
View File
@@ -53,7 +53,7 @@ func Nats(js *broker.JetStream) Inbound {
// whatever was asked for — and not at all when nothing was. // whatever was asked for — and not at all when nothing was.
func (n *natsInbound) Also(kind string) error { func (n *natsInbound) Also(kind string) error {
switch kind { switch kind {
case KindModuleMoved, KindCatchUp, KindSourceMoved, KindProvisioner, KindPullUpdated: case KindModuleMoved, KindCatchUp, KindSourceMoved, KindProvisioner, KindPullUpdated, KindDecided:
n.follows[kind] = true n.follows[kind] = true
return nil return nil
default: default:
@@ -259,6 +259,8 @@ func kindOfSubject(subject string) (string, bool) {
return KindSourceMoved, true return KindSourceMoved, true
case PullUpdatedSubject: case PullUpdatedSubject:
return KindPullUpdated, true return KindPullUpdated, true
case broker.DecidedSubject:
return KindDecided, true
case BuildOutcome(), BuildOutcomeOf(TheBuildMachineBefore): case BuildOutcome(), BuildOutcomeOf(TheBuildMachineBefore):
// A build's outcome is the role's event now, so it arrives on the events stream rather than // A build's outcome is the role's event now, so it arrives on the events stream rather than
// the control branch — and is acted on by the same handler, because what the controller does // the control branch — and is acted on by the same handler, because what the controller does
+38 -1
View File
@@ -70,7 +70,7 @@ type Checker interface {
} }
// PullUpdatedSubject is where the forge's pull requests land: the controller's own follow of them. // PullUpdatedSubject is where the forge's pull requests land: the controller's own follow of them.
var PullUpdatedSubject = broker.ControllerFollows[len(broker.ControllerFollows)-1] var PullUpdatedSubject = "mesh.mod.gitea.event.pull.updated"
// Server acts on what nodes and modules say. // Server acts on what nodes and modules say.
// //
@@ -96,6 +96,8 @@ type Server struct {
checker Checker checker Checker
// healths keeps what machines say of their long-running resources (novox/hq ADR 0240). // healths keeps what machines say of their long-running resources (novox/hq ADR 0240).
healths Healths healths Healths
// decider acts on the operator's warrants for what the controller asked (novox/hq ADR 0259).
decider Decider
log *log.Logger log *log.Logger
// giveUp is how long one message is held for the store; zero means GiveUpAfter. // giveUp is how long one message is held for the store; zero means GiveUpAfter.
@@ -138,6 +140,21 @@ func (s *Server) Checks(c Checker) error {
return nil return nil
} }
// Decider is what the controller does with the router's word on an ask it made (novox/hq ADR 0259): act
// on a warrant once, or record how the ask ended without one.
type Decider interface {
Decided(ctx context.Context, body []byte) error
}
// Decides says what to do about the router's word on the controller's asks, and asks for it delivered.
func (s *Server) Decides(d Decider) error {
if err := s.inbound.Also(KindDecided); err != nil {
return err
}
s.decider = d
return nil
}
// Answers says what to do about a catalogue's catch-up request, and asks for them to be delivered. // Answers says what to do about a catalogue's catch-up request, and asks for them to be delivered.
func (s *Server) Answers(r Replayer) error { func (s *Server) Answers(r Replayer) error {
if err := s.inbound.Also(KindCatchUp); err != nil { if err := s.inbound.Also(KindCatchUp); err != nil {
@@ -210,6 +227,8 @@ func (s *Server) act(ctx context.Context, m Control) {
s.provisioner(ctx, m) s.provisioner(ctx, m)
case KindPullUpdated: case KindPullUpdated:
s.pullUpdated(ctx, m) s.pullUpdated(ctx, m)
case KindDecided:
s.decided(ctx, m)
default: default:
// Dropped: a message nothing understands will not be understood on the next attempt // Dropped: a message nothing understands will not be understood on the next attempt
// either, and asking for it again would spin. // either, and asking for it again would spin.
@@ -656,6 +675,24 @@ func (s *Server) pullUpdated(ctx context.Context, m Control) {
_ = m.Took() _ = m.Took()
} }
// decided hands the router's word on an ask to the decider; a failure to keep what it did is held for the
// store, like any word that must not be lost.
func (s *Server) decided(ctx context.Context, m Control) {
if s.decider == nil {
_ = m.Took()
return
}
err := s.decider.Decided(ctx, m.Body())
switch s.decide(ctx, m, "the operator's word on an ask", "", "", err) {
case Hold:
return
}
if err != nil {
s.log.Printf("the operator's word on an ask could not be kept: %v", err)
}
_ = m.Took()
}
// saysWhatItDid states what a machine now runs, or what it would not take, as a fact on the bus // saysWhatItDid states what a machine now runs, or what it would not take, as a fact on the bus
// (novox/hq ADR 0134). // (novox/hq ADR 0134).
// //
+382
View File
@@ -0,0 +1,382 @@
// Package asks is the contract of asking a person and answering on a channel (novox/hq ADR 0259): the
// shapes an asker, the router and a channel exchange on the bus, and the subjects they travel on. No
// transport and no channel's service: an asker publishes an Ask under its own name and acts on the Warrant
// it hears; the router holds the ask, sends channels a Message, and judges the Choice a channel says; a
// channel shows a Message and says what was chosen and by whom, as its service authenticated it.
package asks
import (
"errors"
"fmt"
"regexp"
"strings"
"time"
)
// The seats (novox/hq ADR 0259 §3).
const (
// Seat is held by the router: an ask and its cancel are its accepts, a warrant its event, each named by
// the asker.
Seat = "operator-channel"
// ChannelSeat is the kinded bench a channel holds to show and say: its accepts carry the kind.
ChannelSeat = "channel"
// IntakeSeat is the kinded bench a channel holds to say what was chosen: its events and proofs carry
// the kind.
IntakeSeat = "intake"
)
// AskSubject is where an asker publishes an ask, CancelSubject its cancel, and DecidedSubject where it
// hears the warrant, or the ask's end without one.
func AskSubject(asker string) string { return "mesh.seat." + Seat + ".accept.ask." + asker }
func CancelSubject(asker string) string { return "mesh.seat." + Seat + ".accept.cancel." + asker }
func DecidedSubject(asker string) string { return "mesh.seat." + Seat + ".event.decided." + asker }
// The work a channel takes, on ChannelSubject.
const (
Show = "show" // a message offering answers
Edit = "edit" // a message shown before, replaced
Send = "send" // a message offering nothing
)
// ChannelSubject is where the router sends a channel of a kind its work.
func ChannelSubject(verb, kind string) string {
return "mesh.seat." + ChannelSeat + ".accept." + verb + "." + kind
}
// What a channel says, on IntakeSubject.
const (
Chosen = "choice" // a button tapped
Link = "link" // somebody asked to be linked as the operator
)
// IntakeSubject is where a channel of a kind says what arrived.
func IntakeSubject(what, kind string) string {
return "mesh.seat." + IntakeSeat + ".event." + what + "." + kind
}
// CodeProof is the one proof verb: a code the operator typed, carried by request and reply, never kept.
const CodeProof = "code"
// ProofSubject is where a channel of a kind asks a proof.
func ProofSubject(verb, kind string) string {
return "mesh.seat." + IntakeSeat + ".proof." + verb + "." + kind
}
// A Level is how much proof an option's answer needs (novox/hq ADR 0234 §8, the glossary's assurance level).
type Level string
const (
// Acknowledge performs only what any granted principal may already do: silencing, details. No proof.
Acknowledge Level = "acknowledge"
// Approve needs one proof: a verified sender (a linked account, linked an hour or more) or a code.
Approve Level = "approve"
// Destroy needs two proofs, one of them a code.
Destroy Level = "destroy"
)
// Rank orders the levels; an unknown level ranks above every known one, so it is never taken as less.
func (l Level) Rank() int {
switch l {
case Acknowledge:
return 0
case Approve:
return 1
case Destroy:
return 2
}
return 3
}
// Operator is the one role an ask may be answered by today.
const Operator = "operator"
// Option is one answer an ask offers: a label for the button, what it does in plain words, its level.
type Option struct {
ID string `json:"id"`
Label string `json:"label"`
Does string `json:"does"`
Level Level `json:"level"`
}
// Ask is a request for a person's word (novox/hq ADR 0259 §4).
type Ask struct {
// ID is the asker's own, unique to it.
ID string `json:"id"`
// Headline names the thing and what is wrong, in a few plain words; Explanation is what happened and
// what it means. Both are held to the plain rule and the content rule by the router.
Headline string `json:"headline"`
Explanation string `json:"explanation"`
Options []Option `json:"options"`
// Who may answer: Operator.
Who string `json:"who"`
// Expires is when the ask ends unanswered; OnExpiry is what the asker then does, in words the person
// is shown ("the delivery stays held"). An ask that authorises never defaults.
Expires time.Time `json:"expires"`
OnExpiry string `json:"on-expiry"`
// About is what the ask is about (a condition's key): a newer ask about it replaces the older.
About string `json:"about,omitempty"`
Urgent bool `json:"urgent,omitempty"`
}
// The bounds of an ask (novox/hq ADR 0234 §8, ADR 0259 §4).
const (
MostOptions = 4
MostOpen = 3
ApproveLasts = 24 * time.Hour
DestroyLasts = 10 * time.Minute
HeadlineLength = 60
LabelLength = 24
)
var usableID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$`)
// UsableID says whether a name can be an ask's or an option's id: a key and a subject token both.
func UsableID(id string) bool { return usableID.MatchString(id) }
// Highest is the highest level among the ask's options.
func (a Ask) Highest() Level {
high := Acknowledge
for _, o := range a.Options {
if o.Level.Rank() > high.Rank() {
high = o.Level
}
}
return high
}
// Option is the option of this id, or false.
func (a Ask) Option(id string) (Option, bool) {
for _, o := range a.Options {
if o.ID == id {
return o, true
}
}
return Option{}, false
}
// Check is what an ask is held to before anything is shown: every refusal, in words its asker can act on.
func (a Ask) Check(now time.Time) error {
var problems []string
say := func(format string, args ...any) { problems = append(problems, fmt.Sprintf(format, args...)) }
if !UsableID(a.ID) {
say("its id %q is not letters, digits, - and _, at most 64", a.ID)
}
if strings.TrimSpace(a.Headline) == "" || len([]rune(a.Headline)) > HeadlineLength {
say("its headline is empty or longer than %d characters", HeadlineLength)
}
if strings.TrimSpace(a.Explanation) == "" {
say("it explains nothing")
}
if a.Who != Operator {
say("it is answered by %q, and only the operator answers today", a.Who)
}
if len(a.Options) == 0 || len(a.Options) > MostOptions {
say("it offers %d options, and an ask offers one to %d", len(a.Options), MostOptions)
}
seen := map[string]bool{}
for _, o := range a.Options {
switch {
case !UsableID(o.ID):
say("an option's id %q is not letters, digits, - and _", o.ID)
case seen[o.ID]:
say("the option %s is offered twice", o.ID)
}
seen[o.ID] = true
if strings.TrimSpace(o.Label) == "" || len([]rune(o.Label)) > LabelLength {
say("the option %s's label is empty or longer than %d characters", o.ID, LabelLength)
}
if strings.TrimSpace(o.Does) == "" {
say("the option %s does not say what it does", o.ID)
}
if o.Level.Rank() > Destroy.Rank() {
say("the option %s has the level %q, which is none of acknowledge, approve, destroy", o.ID, o.Level)
}
}
if !a.Expires.After(now) {
say("it expires before it is asked")
}
switch a.Highest() {
case Approve:
if a.Expires.After(now.Add(ApproveLasts)) {
say("an ask that approves lasts at most %s", ApproveLasts)
}
case Destroy:
if a.Expires.After(now.Add(DestroyLasts)) {
say("an ask that destroys lasts at most %s", DestroyLasts)
}
}
if a.Highest() != Acknowledge && strings.TrimSpace(a.OnExpiry) == "" {
say("it does not say what happens when nobody answers, and an ask that authorises never defaults")
}
if a.About != "" && strings.ContainsAny(a.About, " \n") {
say("what it is about is a key, without spaces")
}
if len(problems) > 0 {
return errors.New("the ask is refused: " + strings.Join(problems, "; "))
}
return nil
}
// Outcome is how an ask ended.
type Outcome string
const (
OutcomeChosen Outcome = "chosen" // a person chose an option: a warrant
OutcomeExpired Outcome = "expired" // nobody answered in time
OutcomeCancelled Outcome = "cancelled" // its asker took it back
OutcomeReplaced Outcome = "replaced" // a newer ask about the same thing replaced it
OutcomeRefused Outcome = "refused" // it was never shown: Words says why
)
// Person is who chose, as the router verified them.
type Person struct {
// Who is the role: Operator.
Who string `json:"who"`
// Kind is the channel kind they answered on, Identity their account on that service, Display the name
// the service shows, and Verified how the router knew it was them.
Kind string `json:"kind"`
Identity string `json:"identity"`
Display string `json:"display,omitempty"`
Verified string `json:"verified"`
}
// Warrant is the router's record that a person chose one option of one ask, or the ask's end without one
// (novox/hq ADR 0259 §6). It carries no secret.
type Warrant struct {
Ask string `json:"ask"`
Asker string `json:"asker"`
About string `json:"about,omitempty"`
Outcome Outcome `json:"outcome"`
// Option, Label and Level are the option chosen; By who chose it, Channel the module it came through,
// Proofs which proofs were present (P1, P2, P3).
Option string `json:"option,omitempty"`
Label string `json:"label,omitempty"`
Level Level `json:"level,omitempty"`
By *Person `json:"by,omitempty"`
Channel string `json:"channel,omitempty"`
Proofs []string `json:"proofs,omitempty"`
At time.Time `json:"at"`
// Words are why an ask ended without a choice, or what refused it.
Words string `json:"words,omitempty"`
}
// Says is the warrant in the words an asker records with its act: "the operator, via telegram (user id
// verified), chose Release".
func (w Warrant) Says() string {
if w.Outcome != OutcomeChosen || w.By == nil {
return fmt.Sprintf("no person chose: the ask %s %s", w.Ask, w.Outcome)
}
via := w.By.Kind
if w.By.Verified != "" {
via += " (" + w.By.Verified + ")"
}
return fmt.Sprintf("the %s, via %s, chose %s", w.By.Who, via, w.Label)
}
// For checks a warrant against the ask its asker made: the same asker and ask, a choice, an option the ask
// offered, at that option's level. An asker acts on nothing else.
func (w Warrant) For(asker string, a Ask) (Option, error) {
if w.Asker != asker || w.Ask != a.ID {
return Option{}, fmt.Errorf("the warrant is for %s's ask %s, not %s's %s", w.Asker, w.Ask, asker, a.ID)
}
if w.Outcome != OutcomeChosen || w.By == nil || w.By.Who != a.Who {
return Option{}, fmt.Errorf("the ask %s ended %s; no person chose", a.ID, w.Outcome)
}
o, offered := a.Option(w.Option)
if !offered {
return Option{}, fmt.Errorf("the ask %s offered no option %s", a.ID, w.Option)
}
if w.Level != o.Level {
return Option{}, fmt.Errorf("the option %s is %s, and the warrant says %s", o.ID, o.Level, w.Level)
}
// A choice made after the ask expired is no answer to it, whatever the router said.
if !w.At.IsZero() && w.At.After(a.Expires) {
return Option{}, fmt.Errorf("the warrant was given at %s, after the ask %s expired at %s",
w.At.UTC().Format(time.RFC3339), a.ID, a.Expires.UTC().Format(time.RFC3339))
}
return o, nil
}
// Button is one answer a channel offers: its label and the router's one-time ticket for it.
type Button struct {
Label string `json:"label"`
Ticket string `json:"ticket"`
}
// Message is the work a channel takes: shown with buttons (Show), shown again in place (Edit), or said
// (Send). Handle is the router's name for it, the same across a show and its edits; the channel keeps
// which of its own messages that is. The words are the router's, shown as given.
type Message struct {
Handle string `json:"handle"`
Title string `json:"title"`
Body string `json:"body"`
Buttons []Button `json:"buttons,omitempty"`
Urgent bool `json:"urgent,omitempty"`
Silent bool `json:"silent,omitempty"`
// Reply is the Choice or LinkAsked this answers, by its ID: the channel shows it where that was made.
Reply string `json:"reply,omitempty"`
// To is the account linked as the operator on this kind, for a channel that verifies its sender: where
// the channel sends what is not a reply. The router's word, from its list; empty when none is linked.
To string `json:"to,omitempty"`
}
// Sender is who a channel's service says sent something: the account, the name it shows, and whether the
// service authenticated it. The router alone judges whether that is the operator.
type Sender struct {
Identity string `json:"identity"`
Display string `json:"display,omitempty"`
Authenticated bool `json:"authenticated"`
}
// Failed is a message a channel could not deliver: its handle, why, and whether trying again could help.
type Failed struct {
Handle string `json:"handle"`
Why string `json:"why"`
Permanent bool `json:"permanent,omitempty"`
At time.Time `json:"at"`
}
// Standing is what a channel says of itself, at least every five minutes and whenever it changes:
// whether it can send now, why not, and whether its edits notify nobody.
type Standing struct {
Ready bool `json:"ready"`
Why string `json:"why,omitempty"`
EditsSilently bool `json:"edits-silently,omitempty"`
At time.Time `json:"at"`
}
// What a channel says of its own delivery, on IntakeSubject.
const (
FailedWhat = "failed"
StandingWhat = "standing"
)
// Choice is a button chosen on a channel.
type Choice struct {
// ID is the channel's own for this arrival, unique, so the router acts on it once.
ID string `json:"id"`
Ticket string `json:"ticket"`
Handle string `json:"handle,omitempty"`
Sender Sender `json:"sender"`
At time.Time `json:"at"`
}
// LinkAsked is somebody on a channel asking to be linked as the operator.
type LinkAsked struct {
ID string `json:"id"`
Sender Sender `json:"sender"`
At time.Time `json:"at"`
}
// Code is a code a person typed on a channel, asked as a proof: never in an event, never kept.
type Code struct {
Sender Sender `json:"sender"`
Code string `json:"code"`
Purpose string `json:"purpose"`
}
// ProofAnswer is the router's answer to a proof: whether it was taken, and words to say to the person.
type ProofAnswer struct {
Accepted bool `json:"accepted"`
Words string `json:"words"`
}
+3 -2
View File
@@ -1,3 +1,6 @@
# git.novox.be/novox/mesh-sdk/go v0.1.8-0.20261008162031-55090da7e08f
## explicit; go 1.22
git.novox.be/novox/mesh-sdk/go/asks
# github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op # github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op
## explicit; go 1.24.0 ## explicit; go 1.24.0
github.com/antithesishq/antithesis-sdk-go/assert github.com/antithesishq/antithesis-sdk-go/assert
@@ -79,8 +82,6 @@ github.com/nats-io/nuid
## explicit; go 1.26.0 ## explicit; go 1.26.0
github.com/novox/mesh-host/internal/declaration github.com/novox/mesh-host/internal/declaration
github.com/novox/mesh-host/validate github.com/novox/mesh-host/validate
# go.uber.org/automaxprocs v1.6.0
## explicit; go 1.20
# golang.org/x/crypto v0.57.0 # golang.org/x/crypto v0.57.0
## explicit; go 1.26.0 ## explicit; go 1.26.0
golang.org/x/crypto/acme golang.org/x/crypto/acme