novox/mesh-controller
6
0
Fork 0
Code Issues Pull Requests 5 Releases Wiki Activity

Compare commits

merge into: novox/mesh-controller:adr-0297-module-logs
Branches Tags
novox/mesh-controller:main novox/mesh-controller:adr-0297-module-logs novox/mesh-controller:feat/bus-view-principal novox/mesh-controller:fix/383-a-proposal-reads-whole-on-the-phone novox/mesh-controller:fix/353-the-asker-judges-the-controllers-own-grant novox/mesh-controller:feat/the-controller-asks-the-operator novox/mesh-controller:test/plan-diff-left-out-line novox/mesh-controller:fix/the-forges-first-rule-and-a-kept-identity novox/mesh-controller:fix/the-registry-trust-test-reads-the-runtimes-module novox/mesh-controller:fix/the-runtime-file-is-the-runtimes novox/mesh-controller:fix/issue-211-a-bundle-stands-on-its-toolchain novox/mesh-controller:feat/build-edges-are-recorded novox/mesh-controller:fix/several-contributions-collide-in-grants-v2
..
pull from: novox/mesh-controller:feat/bus-view-principal
Branches Tags
novox/mesh-controller:main novox/mesh-controller:adr-0297-module-logs novox/mesh-controller:feat/bus-view-principal novox/mesh-controller:fix/383-a-proposal-reads-whole-on-the-phone novox/mesh-controller:fix/353-the-asker-judges-the-controllers-own-grant novox/mesh-controller:feat/the-controller-asks-the-operator novox/mesh-controller:test/plan-diff-left-out-line novox/mesh-controller:fix/the-forges-first-rule-and-a-kept-identity novox/mesh-controller:fix/the-registry-trust-test-reads-the-runtimes-module novox/mesh-controller:fix/the-runtime-file-is-the-runtimes novox/mesh-controller:fix/issue-211-a-bundle-stands-on-its-toolchain novox/mesh-controller:feat/build-edges-are-recorded novox/mesh-controller:fix/several-contributions-collide-in-grants-v2
2 Commits
adr-0297-module-logs .. feat/bus-view-principal
Author SHA1 Message Date
jochen dc5a8208a2 The view reads directly and makes no consumer: a consumer's deliver subject publishes anywhere
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
Details
mesh/repo-check pass: its merge-check.sh passed
Details
mesh/delivery ready: it delivers once merged
Details 
Review of the view (research 036): granted CONSUMER.CREATE on the bucket's stream, the view made a
push consumer delivering to mesh.mod.mesh-issues.event.opened, and a module subscribed there
received the bucket's entry as the tracker's event — measured on 2.11.17. The server does not hold a
deliver subject to its creator's permissions, so a consumer grant is a publish to any subject.

The view now binds and reads directly, nothing else: STREAM.INFO, DIRECT.GET by key, and the batch
DIRECT.GET (multi_last) that lists every key's newest value into its inbox. No watch: the page
re-reads the key a tracker event names (every event carries the number). The live test lists with
the batch, follows a moved event to the re-read, and is refused the consumer and the watch with
nothing reaching the event subject; the mutation (CONSUMER.CREATE back) fails three tests.

The credential says how to read, and that the step making it work is the next `bus upgrade` while
a new bus build waits, not a push.
 2026-10-10 16:15:16 +02:00
jochen a5a355aeec The view: one read-only bus user for a page in a browser, composed like every other (hq research 036)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
Details
mesh/repo-check pass: its merge-check.sh passed
Details
mesh/delivery superseded: a newer head of the same pull request
Details 
Research 036 names the gap (G1): no way for a browser to reach the bus. The bus module now listens
over WebSocket (mesh-catalog, nats); this is who connects there. The view is a fixed principal
(broker.KindView, user `view`) composed into the user list like every user once its credential is
minted, and left out once it is forgotten: it subscribes the issue tracker's events (opened, moved,
noted, linked), the controller's plan-moved and condition-raised/changed/cleared, and the delivery
owner's transition and group; it publishes only the JetStream API requests a read-only watcher of
the tracker's bucket (mesh-issues_issues) makes — STREAM.INFO, DIRECT.GET, CONSUMER.CREATE/INFO/
DELETE, flow control — answered in its own inbox; no reply, no tool, no event, no `$KV` write.

`bus view-credential` mints and prints it once (hash kept, like a person's); `bus view-revoke`
forgets it, real at the next composition. Tests: the composed grants are exactly these and a write
grant of any shape fails; the view is composed only once minted; and against a real server read from
the composed file over WebSocket, the view binds the bucket, reads a key, watches a put land, and is
refused a put, a delete and an event, the bucket unchanged.
 2026-10-10 16:01:32 +02:00
274 changed files with 959 additions and 24570 deletions
Expand all files Collapse all files
Makefile
+10 -18
View File
@@ -99,22 +99,16 @@ proxy-image:
@echo
@docker image inspect $(PROXY_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The whole check. Raises a database, runs the repository's own check against it -- merge-check.sh,
# the very script `mesh/repo-check` runs -- and takes the database down again, including when the
# check fails, which is why the teardown is not conditional and the script's exit status is the
# target's.
# The whole gate. Raises a database, runs everything against it, and takes it down again --
# including when the tests fail, which is why the teardown is not conditional.
#
# **It calls the script rather than restating it** (novox/hq issue 431): this target used to list its
# own steps, its formatting step walked vendor/ where the script's does not, and it failed on
# third-party code no change could fix -- so a developer's check and the gate disagreed, and the steps
# after formatting never ran through it. The script is the one list of steps; this target only gives it
# a database (MESH_TEST_POSTGRES, exported above).
#
# **Packages in parallel, under the race detector, each test on a bus of its own** (internal/testbus),
# as the script runs them: a bus per test, of the release the mesh runs, makes the suite the same in any
# order, and the timeout bounds a hang to a failure with a stack, never a stalled gate.
check: postgres
@sh merge-check.sh ; status=$$? ; docker rm -f $(PG_CONTAINER) >/dev/null 2>&1 || true ; exit $$status
# **Packages in parallel, under the race detector, each test on a bus of its own** (internal/testbus).
# It was one package at a time against one shared bus, because the live tests assert, read and remove
# the mesh's own objects by their fixed names, and two packages at once deleted what the other read; the
# suite was red run as Go runs it and read as noise. A bus per test, of the release the mesh runs, made
# it the same in any order. The timeout bounds a hang to a failure with a stack, never a stalled gate.
check: fmt vet postgres
@go test -race -timeout 15m ./... ; status=$$? ; $(MAKE) postgres-stop ; exit $$status
# Without a database the store's tests skip rather than fail, so this is the honest subset and not
# the gate.
@@ -124,10 +118,8 @@ test:
vet:
go vet ./...
# Quick steps for a developer, not part of `check`. The directories gofmt reads must be the ones
# merge-check.sh lists, never `.`, which walks vendor/ (novox/hq issue 431).
fmt:
@unformatted=$$(gofmt -l cmd internal examples) ; \
@unformatted=$$(gofmt -l . 2>/dev/null) ; \
if [ -n "$$unformatted" ] ; then echo "not gofmt'd:" ; echo "$$unformatted" ; exit 1 ; fi
postgres:
cmd/mesh-controller/advisories_counted_test.go
-180
View File
@@ -1,180 +0,0 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
// **A refusal the bus said once is counted once** (novox/hq issue 402). The bus refused the controller
// one publish at 18:52:59 UTC on 2026-10-10 and never again, yet S9 looked at the one remembered
// advisory every half minute for the hour it is kept, and the condition said "observed 15 time(s), last
// 13s ago" with a refusal in its evidence every 30 seconds: two of us read it as a refusal going on and
// went looking for a missing grant. The condition counts what the bus said, with when it last said it,
// never how often the controller looked.
func TestARefusalSaidOnceIsCountedOnceHoweverOftenItIsLookedAt(t *testing.T) {
refused := time.Date(2026, 10, 10, 18, 52, 59, 0, time.UTC)
now := refused.Add(10 * time.Second)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
// The words the live condition bus.controller.refused carried on 2026-10-10.
said := "the bus refused the controller: nats: permissions violation: Permissions Violation for " +
`Publish to "mesh.seat.mesh-delivery.tool.times"`
advisory := link.Advisory{Kind: link.AdvisoryRefused, ID: "controller", Said: said,
First: refused, Last: refused, Count: 1}
look := func() conditions.Condition {
t.Helper()
f := &signalFacts{now: now, host: "novox", advisories: []link.Advisory{advisory}}
if err := k.Reconcile(t.Context(), "S9", watchAdvisories(f)); err != nil {
t.Fatal(err)
}
c, found, err := conditions.ReadOne(t.Context(), store, "bus.controller.refused")
if err != nil || !found {
t.Fatalf("bus.controller.refused: found %v, %v", found, err)
}
return c
}
var c conditions.Condition
for range 15 {
c = look()
now = now.Add(30 * time.Second)
}
if c.Observations != 1 || len(c.Evidence) != 1 {
t.Fatalf("one refusal, looked at 15 times, reads as observed %d time(s) with %d evidence lines: %+v",
c.Observations, len(c.Evidence), c.Evidence)
}
if !c.LastObserved.Equal(refused) || !c.Evidence[0].At.Equal(refused) {
t.Fatalf("last observed %s, evidence at %s: the refusal was at %s", c.LastObserved, c.Evidence[0].At, refused)
}
// The bus refuses it three times more between two looks: the condition counts four refusals, says
// the newest, and adds one line of evidence for the look that saw them.
again := now.Add(-5 * time.Second)
advisory.Last, advisory.Count = again, 4
c = look()
if c.Observations != 4 || len(c.Evidence) != 2 || !c.LastObserved.Equal(again) || !c.Evidence[0].At.Equal(again) {
t.Fatalf("four refusals read as observed %d time(s), last %s, evidence %+v", c.Observations, c.LastObserved, c.Evidence)
}
if !strings.Contains(c.Summary, "(4 times since 18:52 UTC)") {
t.Fatalf("summary %q", c.Summary)
}
now = now.Add(30 * time.Second)
if c = look(); c.Observations != 4 || len(c.Evidence) != 2 {
t.Fatalf("a look with nothing new counted: observed %d time(s), evidence %+v", c.Observations, c.Evidence)
}
}
// **A condition that looks at the present still counts its looks** (novox/hq issue 402 changes only
// what reads a record): a machine silent for three looks was observed three times, and says so.
func TestAConditionOfThePresentCountsItsLooks(t *testing.T) {
now := time.Date(2026, 10, 10, 19, 0, 0, 0, time.UTC)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
var c conditions.Condition
for range 3 {
var err error
if c, err = k.Observe(t.Context(), conditions.Observation{Scope: conditions.ScopeMachine, ID: "ace",
Kind: "silent", Machine: "ace", Severity: conditions.Warning, Summary: "ace has not been heard from",
Source: "S1"}); err != nil {
t.Fatal(err)
}
now = now.Add(30 * time.Second)
}
if c.Observations != 3 || len(c.Evidence) != 3 || !c.LastObserved.Equal(now.Add(-30*time.Second)) {
t.Fatalf("observed %d time(s), %d evidence lines, last %s", c.Observations, len(c.Evidence), c.LastObserved)
}
}
// **One key, one watcher** (novox/hq issue 440). A consumer's max-deliveries condition is said from
// DEAD_LETTERS while it holds a message the consumer gave up on (issue 330). A max-deliveries advisory
// without a token names the same key; read beside it, each look added an observation and a line of
// evidence through the dead-letter half, and the two summaries overwrote each other on every look. The
// dead-letter row alone says that key, and counts the messages given up on, never the looks.
func TestAHeldDeadLetterIsCountedByWhatWasGivenUpNotByTheLooks(t *testing.T) {
gaveUp := time.Date(2026, 10, 10, 21, 4, 0, 0, time.UTC)
now := gaveUp.Add(20 * time.Second)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
t.Cleanup(func() { k.Close(t.Context()) })
const key = "bus.EVENTS.media_sonarr.max-deliveries"
held := map[string]int{"EVENTS.media_sonarr": 1}
newest := map[string]time.Time{"EVENTS.media_sonarr": gaveUp}
advisory := link.Advisory{Kind: link.AdvisoryMaxDeliveries, ID: "EVENTS.media_sonarr", Stream: "EVENTS",
Consumer: "media_sonarr", Said: "sonarr on media handed message 7 over 5 times and gave up on it",
First: gaveUp, Last: gaveUp, Count: 1}
look := func() conditions.Condition {
t.Helper()
f := &signalFacts{now: now, host: "novox", advisories: []link.Advisory{advisory},
deadLetters: held, deadLettersNewest: newest}
if err := k.Reconcile(t.Context(), "S9", watchAdvisories(f)); err != nil {
t.Fatal(err)
}
c, found, err := conditions.ReadOne(t.Context(), store, key)
if err != nil || !found {
t.Fatalf("%s: found %v, %v", key, found, err)
}
if !strings.Contains(c.Summary, "dead-letters verb") {
t.Fatalf("the summary is not the dead-letter row's: %q", c.Summary)
}
return c
}
var c conditions.Condition
for range 5 {
c = look()
now = now.Add(30 * time.Second)
}
if c.Observations != 1 || len(c.Evidence) != 1 || !c.LastObserved.Equal(gaveUp) {
t.Fatalf("one message given up on, looked at five times, reads as observed %d time(s), last %s, "+
"with %d evidence lines: %+v", c.Observations, c.LastObserved, len(c.Evidence), c.Evidence)
}
// The consumer gives up on a second message between two looks: two given up on, one more line.
again := now.Add(-10 * time.Second)
held["EVENTS.media_sonarr"], newest["EVENTS.media_sonarr"] = 2, again
c = look()
if c.Observations != 2 || len(c.Evidence) != 2 || !c.LastObserved.Equal(again) {
t.Fatalf("two given up on read as observed %d time(s), last %s, evidence %+v", c.Observations,
c.LastObserved, c.Evidence)
}
now = now.Add(30 * time.Second)
if c = look(); c.Observations != 2 || len(c.Evidence) != 2 {
t.Fatalf("a look with nothing new counted: observed %d time(s), evidence %+v", c.Observations, c.Evidence)
}
}
// The advisory still says the max-deliveries it alone knows: a message given up on that could not be
// kept, under a key of its own (issue 330), which issue 440's change leaves as it was.
func TestAMessageThatCouldNotBeKeptIsStillSaidFromTheAdvisory(t *testing.T) {
at := time.Date(2026, 10, 10, 21, 4, 0, 0, time.UTC)
f := &signalFacts{now: at, host: "novox", advisories: []link.Advisory{{Kind: link.AdvisoryMaxDeliveries,
ID: "EVENTS.media_sonarr", Stream: "EVENTS", Consumer: "media_sonarr", Token: link.AdvisoryNotKept,
Said: "sonarr on media gave up on message 7, and it could not be kept", First: at, Last: at, Count: 1}}}
said := watchAdvisories(f)
if len(said) != 1 || said[0].Key() != "bus.EVENTS.media_sonarr.not-kept" {
t.Fatalf("%+v", said)
}
}
// A max-deliveries advisory without a token names a consumer's dead-letter key, which only DEAD_LETTERS
// says (novox/hq issues 330 and 440): with nothing held, the advisory alone raises nothing.
func TestAMaxDeliveriesAdvisoryAloneRaisesNothing(t *testing.T) {
at := time.Date(2026, 10, 10, 21, 4, 0, 0, time.UTC)
f := &signalFacts{now: at, host: "novox", advisories: []link.Advisory{{Kind: link.AdvisoryMaxDeliveries,
ID: "EVENTS.media_sonarr", Stream: "EVENTS", Consumer: "media_sonarr",
Said: "sonarr on media handed message 7 over 5 times and gave up on it", First: at, Last: at, Count: 1}}}
if said := watchAdvisories(f); len(said) != 0 {
t.Fatalf("said %+v", said)
}
}
cmd/mesh-controller/batches.go
+1 -48
View File
@@ -478,8 +478,7 @@ func spelledOf(m inventory.BatchedMerge) string {
// namedOf is one merge as a walk or batch names it: its commit, and its pull request and moves as announced.
func namedOf(m inventory.BatchedMerge, repository string) inventory.PlanMerge {
k := announcedOf(m)
return inventory.PlanMerge{Repository: repository, Commit: m.Commit, Number: k.Number, Title: k.Title, Moves: k.Moves,
Merged: m.Merged, Heard: m.Heard}
return inventory.PlanMerge{Repository: repository, Commit: m.Commit, Number: k.Number, Title: k.Title, Moves: k.Moves}
}
// laterMerge says a was merged after b: by the forge's merge time, then by when each was heard.
@@ -773,8 +772,6 @@ func planBatch(ctx context.Context, open *stores, batch *inventory.Plan, carry [
}
plan.Delivery.Merges = named
plan.Delivery.Alone = batch.Delivery != nil && batch.Delivery.Alone
// **Its moments and its class** (novox/hq ADR 0282 decision 6): measured, never acted on.
plan.Times = walkTimesAtCut(*batch, plan, entries, now)
if len(moved) == 0 {
plan.State = inventory.PlanDone
plan.Tiers = [][]string{}
@@ -823,50 +820,6 @@ func planBatch(ctx context.Context, open *stores, batch *inventory.Plan, carry [
return nil
}
// walkTimesAtCut is a walk's own moments as it is cut (novox/hq ADR 0282 decision 6): when its batch's window
// closed — no merge for the window's length, or its maximum, whichever came first — when it was cut, and its
// class. A merge walked alone had no window.
func walkTimesAtCut(batch, walk inventory.Plan, entries []inventory.Entry, now time.Time) *inventory.PlanTimes {
cut := now
t := &inventory.PlanTimes{Cut: &cut, Class: classOf(walk, entries)}
if batch.Delivery != nil && batch.Delivery.Batch != nil {
w := batch.Delivery.Batch
closed := w.ClosesAt
if !w.AtMost.IsZero() && w.AtMost.Before(closed) {
closed = w.AtMost
}
if !closed.IsZero() {
if closed.After(now) {
closed = now
}
t.WindowClosed = &closed
}
}
return t
}
// resolverSeat is the seat of the mesh's resolver: a module claiming it is a core module (ADR 0282 decision 1).
const resolverSeat = "mesh-dns-resolver"
// classOf is a walk's class (novox/hq ADR 0282 decision 1): core when it walks a module on the controller's own
// path or one holding the mesh's resolver, leaf otherwise.
func classOf(walk inventory.Plan, entries []inventory.Entry) string {
resolvers := map[string]bool{}
for _, e := range entries {
for _, c := range e.Manifest.Claims {
if c.Name == resolverSeat {
resolvers[e.Manifest.Module] = true
}
}
}
for m := range walk.Modules {
if _, own := onTheControllersPath[m]; own || resolvers[m] {
return inventory.ClassCore
}
}
return inventory.ClassLeaf
}
// combinedMerges is a batch's merges as one merge per repository's branch: the latest, with every file the
// merges of it changed and said to be a module's — on a linear trunk the latest contains the others. A file is
// removed when the last merge of the batch that changed it removed it. merges are in the order they were made.
cmd/mesh-controller/bucket_log_fill.go
-207
View File
@@ -1,207 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"sort"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/conditions"
)
// **A filling bucket or log is said before it is full** (novox/hq ADR 0297 §6, issue 501).
//
// A module's bucket and its log each have a cap on the bus, and when either is full the bus refuses
// the next write: the module stops doing what it writes for, and nothing said so before. On
// 2026-10-11 the issue tracker's bucket held a sixth of its cap after two days, growing toward a stop
// nobody would have heard coming. So the self-check reads every module's bucket and log, and one at
// fillRaiseAt of its cap or more raises a condition naming the module, the bucket or log, and how full
// it is.
//
// **Cleared by observation below fillClearBelow, not below fillRaiseAt**: a bucket or log hovering at
// its threshold would otherwise be raised and cleared on every run. Between the two, an open condition
// is kept and none is raised.
//
// **One that cannot be read is said, and the rest are still judged.** An unanswered question about one
// stream is no reason to know nothing of the others: it is a finding of its own, naming the bucket or
// log and why, and a fill condition already open for it is kept, because not knowing is not a pass.
// The fill thresholds, in percent of a bucket's or log's cap.
const (
fillRaiseAt = 75
fillClearBelow = 70
)
// The conditions the fill probe raises: one filling, and one that could not be read.
const (
kindBucketOrLogFilling = "bucket-or-log-filling"
kindBucketOrLogUnread = "bucket-or-log-unread"
)
// probeFillID is the probe's id in the registry.
const probeFillID = "D-fill"
// bucketOrLog is one module's bucket or log as the fill probe reads it.
type bucketOrLog struct {
Module string
// Kind is `bucket` or `log`; Name its local name; Stream the stream it is on the bus.
Kind, Name, Stream string
}
// filled is how full one bucket or log is, read from the bus.
type filled struct {
Bytes, Max uint64
}
// percent is how full, in whole percent, rounded down.
func (f filled) percent() uint64 {
if f.Max == 0 {
return 0
}
return f.Bytes * 100 / f.Max
}
// fillKey is where a bucket's or log's fill condition is kept.
func fillKey(s bucketOrLog) string { return conditions.Key(conditions.ScopeBus, s.Stream, "filling") }
// fillObservation is what one bucket's or log's fill says: a finding at fillRaiseAt or above, and,
// while its condition is open, at fillClearBelow or above too; nothing otherwise, which is what clears
// it. The operator's words are the kind's (plain_words.go); the summary and the evidence name the
// module, the bucket or log, and the fill.
func fillObservation(s bucketOrLog, f filled, open bool) (conditions.Observation, bool) {
if f.Max == 0 {
return conditions.Observation{}, false // one with no cap cannot fill
}
// Compared in bytes, not rounded percent: 74.9% is not 75%.
atRaise := f.Bytes*100 >= f.Max*fillRaiseAt
aboveClear := f.Bytes*100 >= f.Max*fillClearBelow
if !atRaise && !(open && aboveClear) {
return conditions.Observation{}, false
}
pct := f.percent()
return conditions.Observation{
Scope: conditions.ScopeBus, ID: s.Stream, Token: "filling", Kind: kindBucketOrLogFilling,
Severity: conditions.Warning,
Summary: fmt.Sprintf("%s's %s %s holds %s of %s, %d%%: at its cap the bus refuses the module's next write",
s.Module, s.Kind, s.Name, mibWords(f.Bytes), mibWords(f.Max), pct),
Said: fmt.Sprintf("module %s, %s %s (stream %s): %d of %d bytes, %d%%", s.Module, s.Kind, s.Name,
s.Stream, f.Bytes, f.Max, pct),
}, true
}
// unreadObservation says one bucket or log could not be read, and why. Asked over the network, so one
// look can be wrong: raised on the second look in a row (confirm.go).
func unreadObservation(s bucketOrLog, why error) conditions.Observation {
return conditions.Observation{
Scope: conditions.ScopeBus, ID: s.Stream, Token: "unread", Kind: kindBucketOrLogUnread,
Severity: conditions.Warning, Confirm: true,
Summary: fmt.Sprintf("%s's %s %s could not be read, so how full it is is not known", s.Module, s.Kind, s.Name),
Said: fmt.Sprintf("module %s, %s %s (stream %s): %v", s.Module, s.Kind, s.Name, s.Stream, why),
}
}
// keptFilling is an open fill condition said again for a bucket or log that could not be read this
// time: not knowing how full it is now is no reason to say it has room. Only ever made from a condition
// read back as open — its own summary and severity, never words made up for it.
func keptFilling(s bucketOrLog, open conditions.Condition, why error) conditions.Observation {
return conditions.Observation{
Scope: conditions.ScopeBus, ID: s.Stream, Token: "filling", Kind: kindBucketOrLogFilling,
Severity: open.Severity, Summary: open.Summary,
Said: fmt.Sprintf("module %s, %s %s (stream %s): not read this time (%v); kept open as it was",
s.Module, s.Kind, s.Name, s.Stream, why),
}
}
// mibWords is a size as a person reads it, in MiB with one decimal.
func mibWords(b uint64) string {
return fmt.Sprintf("%.1f MiB", float64(b)/(1024*1024))
}
// readFill is how full one bucket or log is on the bus; found false when it is not on the bus.
type readFill func(ctx context.Context, s bucketOrLog) (f filled, found bool, err error)
// openFill is the fill condition open under a key, if one is, or why it could not be read.
type openFill func(ctx context.Context, key string) (conditions.Condition, bool, error)
// judgeFills judges every bucket and log on its own: one that cannot be read is said as unread, with
// its fill condition kept only when that condition was read back and is open — when it cannot be read
// either, nothing is said of its fill, which the unread finding already covers. Every other is judged
// by its fill; for one of those, a condition that cannot be read is taken as open, so an unknown never
// clears a fill measured between fillClearBelow and fillRaiseAt.
func judgeFills(ctx context.Context, all []bucketOrLog, read readFill, open openFill) []conditions.Observation {
var out []conditions.Observation
for _, s := range all {
f, found, err := read(ctx, s)
if err != nil {
out = append(out, unreadObservation(s, err))
if c, isOpen, cerr := open(ctx, fillKey(s)); cerr == nil && isOpen {
out = append(out, keptFilling(s, c, err))
}
continue
}
if !found {
continue // not on the bus: created on the controller's next raise, and nothing there can fill
}
_, isOpen, cerr := open(ctx, fillKey(s))
isOpen = isOpen || cerr != nil
if o, said := fillObservation(s, f, isOpen); said {
out = append(out, o)
}
}
sort.SliceStable(out, func(i, j int) bool { return out[i].Key() < out[j].Key() })
return out
}
// declaredBucketsAndLogs is every module's bucket and log the catalogue declares, by its stream.
func declaredBucketsAndLogs(ctx context.Context, d *doctor) ([]bucketOrLog, error) {
buckets, err := d.open.inventory.DeclaredBuckets(ctx)
if err != nil {
return nil, err
}
logs, err := d.open.inventory.DeclaredLogs(ctx)
if err != nil {
return nil, err
}
var out []bucketOrLog
for _, b := range buckets {
out = append(out, bucketOrLog{Module: b.Module, Kind: "bucket", Name: b.Name, Stream: "KV_" + b.Bucket()})
}
for _, l := range logs {
out = append(out, bucketOrLog{Module: l.Module, Kind: "log", Name: l.Name, Stream: l.Stream()})
}
return out, nil
}
// probeFill reads every module's bucket and log on the bus and says each one filling toward its cap,
// and each one that could not be read.
func probeFill(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
all, err := declaredBucketsAndLogs(ctx, d)
if err != nil {
return nil, err
}
js := d.js.Context()
read := func(ctx context.Context, s bucketOrLog) (filled, bool, error) {
if err := ctx.Err(); err != nil {
return filled{}, false, err
}
info, err := js.StreamInfo(s.Stream, nats.Context(ctx))
switch {
case errors.Is(err, nats.ErrStreamNotFound):
return filled{}, false, nil
case err != nil:
return filled{}, false, err
case info.Config.MaxBytes <= 0:
return filled{}, true, nil
}
return filled{Bytes: info.State.Bytes, Max: uint64(info.Config.MaxBytes)}, true, nil
}
open := func(ctx context.Context, key string) (conditions.Condition, bool, error) {
if d.keeper == nil {
return conditions.Condition{}, false, nil
}
return d.keeper.Get(ctx, key)
}
return judgeFills(ctx, all, read, open), nil
}
cmd/mesh-controller/bucket_log_fill_test.go
-165
View File
@@ -1,165 +0,0 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/conditions"
)
// **A filling log or bucket is said at three quarters of its cap and cleared below seven tenths**
// (novox/hq ADR 0297 §6): raised at 75% naming the module, the bucket or log and its fill; not at 74.9%;
// kept between 70% and 75% while it is open, and not raised there when it is not; and cleared — said no
// more — once it reads below 70%, open or not.
func TestAFillingBucketOrLogIsRaisedAtThreeQuartersAndClearedBelowSevenTenths(t *testing.T) {
const mib = 1024 * 1024
log := bucketOrLog{Module: "mesh-issues", Kind: "log", Name: "changes", Stream: "LOG_mesh-issues_changes"}
bucket := bucketOrLog{Module: "mesh-issues", Kind: "bucket", Name: "issues", Stream: "KV_mesh-issues_issues"}
for _, c := range []struct {
name string
of bucketOrLog
bytes uint64
max uint64
open bool
said bool
}{
{"a log at exactly 75%", log, 768 * mib, 1024 * mib, false, true},
{"a bucket at exactly 75%", bucket, 48 * mib, 64 * mib, false, true},
{"a bucket full", bucket, 64 * mib, 64 * mib, false, true},
{"a log just under 75%", log, 768*mib - 1, 1024 * mib, false, false},
{"a bucket at 72%, not open", bucket, 64 * mib * 72 / 100, 64 * mib, false, false},
{"a bucket at 72%, open: kept", bucket, 64 * mib * 72 / 100, 64 * mib, true, true},
{"a log at exactly 70%, open: kept", log, 700 * mib, 1000 * mib, true, true},
{"a log just under 70%, open: cleared", log, 700*mib - 1, 1000 * mib, true, false},
{"a bucket at 10%, open: cleared", bucket, 64 * mib / 10, 64 * mib, true, false},
{"a bucket with no cap", bucket, 100, 0, true, false},
} {
o, said := fillObservation(c.of, filled{Bytes: c.bytes, Max: c.max}, c.open)
if said != c.said {
t.Errorf("%s: said %v, want %v", c.name, said, c.said)
continue
}
if !said {
continue
}
if o.Key() != fillKey(c.of) || o.Kind != kindBucketOrLogFilling || o.Severity != conditions.Warning {
t.Errorf("%s: raised as %s (%s, %s)", c.name, o.Key(), o.Kind, o.Severity)
}
for _, part := range []string{c.of.Module, c.of.Kind + " " + c.of.Name, "%", " of "} {
if !strings.Contains(o.Summary, part) {
t.Errorf("%s: does not name %q: %q", c.name, part, o.Summary)
}
}
if !strings.Contains(o.Said, "bytes") {
t.Errorf("%s: the evidence does not say the fill in bytes: %q", c.name, o.Said)
}
}
o, _ := fillObservation(log, filled{Bytes: 800 * mib, Max: 1024 * mib}, false)
if !strings.Contains(o.Summary, "800.0 MiB of 1024.0 MiB, 78%") {
t.Errorf("the fill is said as %q", o.Summary)
}
}
// **One bucket or log that cannot be read does not stop the others being judged** (review of #231): it is
// said as unread with its reason, a fill condition open for it is kept, and every other bucket and log
// is judged by its own fill.
func TestAnUnreadableBucketOrLogIsSaidAndTheRestAreStillJudged(t *testing.T) {
const mib = 1024 * 1024
broken := bucketOrLog{Module: "a", Kind: "bucket", Name: "broken", Stream: "KV_a_broken"}
brokenOpen := bucketOrLog{Module: "a", Kind: "log", Name: "was-filling", Stream: "LOG_a_was-filling"}
full := bucketOrLog{Module: "b", Kind: "log", Name: "changes", Stream: "LOG_b_changes"}
empty := bucketOrLog{Module: "c", Kind: "bucket", Name: "quiet", Stream: "KV_c_quiet"}
absent := bucketOrLog{Module: "d", Kind: "bucket", Name: "not-yet", Stream: "KV_d_not-yet"}
refusal := errors.New("the bus did not answer")
read := func(_ context.Context, s bucketOrLog) (filled, bool, error) {
switch s {
case broken, brokenOpen:
return filled{}, false, refusal
case full:
return filled{Bytes: 900 * mib, Max: 1000 * mib}, true, nil
case empty:
return filled{Bytes: 1, Max: 64 * mib}, true, nil
}
return filled{}, false, nil
}
open := func(_ context.Context, key string) (conditions.Condition, bool, error) {
if key == fillKey(brokenOpen) {
return conditions.Condition{Key: key, Severity: conditions.Warning,
Summary: "a's log was-filling holds 800.0 MiB of 1024.0 MiB, 78%"}, true, nil
}
return conditions.Condition{}, false, nil
}
got := map[string]conditions.Observation{}
for _, o := range judgeFills(context.Background(), []bucketOrLog{broken, brokenOpen, full, empty, absent}, read, open) {
got[o.Key()] = o
}
for _, s := range []bucketOrLog{broken, brokenOpen} {
o, said := got[conditions.Key(conditions.ScopeBus, s.Stream, "unread")]
if !said || o.Kind != kindBucketOrLogUnread || !o.Confirm || !strings.Contains(o.Said, refusal.Error()) {
t.Errorf("%s could not be read and was said as %+v", s.Stream, o)
}
}
if o, kept := got[fillKey(brokenOpen)]; !kept || !strings.Contains(o.Said, "kept open") {
t.Errorf("an open fill condition of a log not read this time was not kept: %+v", o)
}
if _, said := got[fillKey(broken)]; said {
t.Error("a bucket not read and not filling was said filling")
}
if o, said := got[fillKey(full)]; !said || o.Kind != kindBucketOrLogFilling {
t.Errorf("a log at 90%% beside an unreadable one was not judged: %+v", got)
}
if len(got) != 4 {
t.Errorf("said %d findings, want 4 (two unread, one kept, one filling): %v", len(got), got)
}
}
// Both kinds have plain words of their own, which hold to the plain rule (novox/hq ADR 0253).
func TestAFillingOrUnreadBucketOrLogIsSaidInPlainWords(t *testing.T) {
for _, kind := range []string{kindBucketOrLogFilling, kindBucketOrLogUnread} {
wording, has := plainWordings[kind]
if !has {
t.Errorf("%s has no plain words", kind)
continue
}
if why, ok := conditions.PlainWords(wording(conditions.Observation{Kind: kind})); !ok {
t.Errorf("%s: %s", kind, why)
}
}
}
// **When neither the bucket or log nor its condition can be read, nothing is said of its fill** (second
// review of #231): the unread finding covers it, and no fill is invented for it. For one that is read and
// measured between seven tenths and three quarters, a condition that cannot be read is taken as open, so
// an unknown never clears it.
func TestNothingIsSaidOfAFillWhenNeitherItNorItsConditionCanBeRead(t *testing.T) {
const mib = 1024 * 1024
broken := bucketOrLog{Module: "a", Kind: "log", Name: "changes", Stream: "LOG_a_changes"}
between := bucketOrLog{Module: "b", Kind: "bucket", Name: "issues", Stream: "KV_b_issues"}
read := func(_ context.Context, s bucketOrLog) (filled, bool, error) {
if s == broken {
return filled{}, false, errors.New("the bus did not answer")
}
return filled{Bytes: 72 * mib, Max: 100 * mib}, true, nil
}
open := func(context.Context, string) (conditions.Condition, bool, error) {
return conditions.Condition{}, false, errors.New("the conditions bucket did not answer")
}
got := map[string]conditions.Observation{}
for _, o := range judgeFills(context.Background(), []bucketOrLog{broken, between}, read, open) {
got[o.Key()] = o
}
if o, said := got[fillKey(broken)]; said {
t.Fatalf("a fill was said for a log whose fill and condition could not be read: %+v", o)
}
if _, said := got[conditions.Key(conditions.ScopeBus, broken.Stream, "unread")]; !said {
t.Error("the log that could not be read was not said as unread")
}
if _, kept := got[fillKey(between)]; !kept {
t.Error("a bucket at 72% whose condition could not be read was cleared")
}
if len(got) != 2 {
t.Errorf("said %d findings, want 2: %v", len(got), got)
}
}
cmd/mesh-controller/bus_step.go
+9 -2
View File
@@ -151,6 +151,13 @@ func busCommand(ctx context.Context, args []string) error {
if len(args) > 0 && !strings.HasPrefix(args[0], "-") {
sub, args = args[0], args[1:]
}
// The view's credential, a terminal line like a person's (bus_view.go).
switch sub {
case "view-credential":
return busViewCredential(ctx, args)
case "view-revoke":
return busViewRevoke(ctx, args)
}
set := flag.NewFlagSet("bus", flag.ContinueOnError)
snapshot := set.String("snapshot-taken", "", "where the streams' snapshot a person took is, while the mesh takes none itself")
reversible := set.Bool("reversible", false, "the new version can be undone by putting the old one back")
@@ -160,14 +167,14 @@ func busCommand(ctx context.Context, args []string) error {
if rest, err := parseAround(set, args); err != nil {
return err
} else if len(rest) > 0 {
return errors.New("bus [upgrade --why … --reversible|--irreversible [--snapshot-taken <where>]]")
return errors.New(busUsage)
}
switch sub {
case "":
return busStatus(ctx)
case "upgrade":
default:
return fmt.Errorf("bus says what a bus upgrade would do, or `bus upgrade` — not %q", sub)
return fmt.Errorf("bus says what a bus upgrade would do, or `bus upgrade`, `bus view-credential`, `bus view-revoke` — not %q", sub)
}
// Everything refused before anything is done.
if err := why.require("bus upgrade"); err != nil {
cmd/mesh-controller/bus_view.go
+117
View File
@@ -0,0 +1,117 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"net"
"strconv"
"strings"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
)
// The view's credential: the one read-only user a page in a browser connects to the bus as, over the
// bus module's WebSocket listener (novox/hq research 036, gap G1; broker.KindView).
//
// **A terminal line, like a person's credential** (operator.go): printed once, never stored — the mesh
// keeps a hash — and revoked by forgetting the row, which the next composition of the user list makes
// real. There is one view; issuing it again rotates its password.
const busUsage = "bus [upgrade --why … --reversible|--irreversible [--snapshot-taken <where>] | view-credential | view-revoke]"
// busWebSocketPort is the port the bus module's WebSocket listener is published on, mirrored from the
// nats module's manifest (its `bus-websocket` opening), because the credential names where to connect
// and the controller does not read the module's configuration. Reached across the overlay only: the
// opening is from the mesh, and the mesh's filter admits nothing else.
const busWebSocketPort = 4223
func busViewCredential(ctx context.Context, args []string) error {
if len(args) != 0 {
return errors.New("bus view-credential takes nothing: there is one view, and this prints its credential once")
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
// Refused here rather than at the next composition, where it would stop the whole file.
if _, err := broker.PermissionsFor(broker.Principal{Kind: broker.KindView, PasswordHash: "x"}); err != nil {
return err
}
password, err := inv.MintBusPassword(ctx, inventory.BusUser{Username: broker.ViewUser, Kind: inventory.BusView})
if err != nil {
return err
}
where, err := broker.FromEnvironment()
if err != nil && !errors.Is(err, broker.ErrNotConfigured) {
return err
}
host := where.Address
if h, _, err := net.SplitHostPort(where.Address); err == nil {
host = h
}
websocket := ""
if host != "" {
websocket = "ws://" + net.JoinHostPort(host, strconv.Itoa(busWebSocketPort))
}
held, err := json.Marshal(struct {
WebSocket string `json:"websocket,omitempty"`
URL string `json:"url,omitempty"`
Fingerprint string `json:"fingerprint,omitempty"`
User string `json:"user"`
Password string `json:"password"`
InboxPrefix string `json:"inbox_prefix"`
Hears []string `json:"hears"`
Reads string `json:"reads"`
HowToRead string `json:"how_to_read"`
}{
WebSocket: websocket, URL: "nats://" + where.Address, Fingerprint: where.Fingerprint,
User: broker.ViewUser, Password: password,
// The client must make its inboxes under the view's own prefix: its subscribe grant is
// `_INBOX.view.>` and no wider (design 25 §4), and a client's default inbox is not under it.
InboxPrefix: "_INBOX." + broker.ViewUser,
Hears: broker.ViewHears, Reads: broker.ViewBucket,
HowToRead: "direct reads only, no watch (a consumer is refused): list with a request to $JS.API.DIRECT.GET.KV_" +
broker.ViewBucket + ` carrying {"multi_last":["$KV.` + broker.ViewBucket + `.>"]}, answered until a 204 status; ` +
"read one key with $JS.API.DIRECT.GET.KV_" + broker.ViewBucket + ".$KV." + broker.ViewBucket + ".<number> " +
"(nats.js: kvm.open(bucket, {allow_direct: true}), never create); re-read the key an event's number names",
})
if err != nil {
return err
}
fmt.Printf("issued the view, which hears %s and reads the bucket %s, and nothing else\n",
strings.Join(broker.ViewHears, ", "), broker.ViewBucket)
fmt.Println(" this is the only time the credential is printed; the mesh keeps a hash")
fmt.Println(" it works once the bus has been told, which is the next push to the machine holding mesh-broker —")
fmt.Println(" and while a new build of the bus module waits for that machine, the next `bus upgrade` a person starts,")
fmt.Println(" which is also what brings the WebSocket listener it connects through")
fmt.Println()
fmt.Println(string(held))
return nil
}
func busViewRevoke(ctx context.Context, args []string) error {
if len(args) != 0 {
return errors.New("bus view-revoke takes nothing: there is one view")
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
if err := open.inventory.ForgetBusUser(ctx, broker.ViewUser); err != nil {
return err
}
// **Revoked at the next composition, not now** — as a person is (operator revoke): the bus's users
// are a file, and the credential stops working when the file no longer names it.
fmt.Println("the view is forgotten, and its credential stops working at the next composition — " +
"push the machine holding mesh-broker to make it so")
return nil
}
cmd/mesh-controller/check.go
-8
View File
@@ -179,14 +179,6 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
}
fmt.Fprintf(out, ", keeps state %s", strings.Join(kept, ", "))
}
// And the logs it keeps, with their caps (novox/hq ADR 0297).
if len(m.Logs) > 0 {
kept := make([]string, 0, len(m.Logs))
for _, l := range m.Logs {
kept = append(kept, fmt.Sprintf("%s (%d MiB)", l.Name, l.Cap()))
}
fmt.Fprintf(out, ", keeps log %s", strings.Join(kept, ", "))
}
if len(m.Reads) > 0 {
fmt.Fprintf(out, ", reads %s", strings.Join(m.Reads, ", "))
}
cmd/mesh-controller/check_test.go
+5 -4
View File
@@ -6,10 +6,8 @@ import (
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
"github.com/novox/mesh-controller/internal/catalogue"
"testing"
)
// The check anybody can run is the check registration runs (novox/hq issue 148, ADR 0037): a manifest
@@ -58,7 +56,10 @@ func TestModuleCheckJudgesBetweenTheManifestsGiven(t *testing.T) {
// The real catalogue passes the command, the way it passes the test that used to be the only check.
func TestModuleCheckPassesTheCatalogue(t *testing.T) {
root := beside.Catalogue(t)
root := filepath.Join("..", "..", "..", "mesh-catalog", "modules")
if _, err := os.Stat(root); err != nil {
t.Skipf("catalogue sibling not present: %v", err)
}
paths, err := manifestsUnder(root)
if err != nil || len(paths) == 0 {
t.Fatalf("no manifests under %s: %v", root, err)
cmd/mesh-controller/checks.go
+5 -46
View File
@@ -5,8 +5,6 @@ import (
"encoding/json"
"fmt"
"path"
"regexp"
"runtime/debug"
"slices"
"sort"
"strings"
@@ -248,11 +246,9 @@ func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scop
if dir == "mesh-controller" {
beside["mesh-controller-main"] = link.CheckedOut{Repository: url, Ref: refs["mesh-controller-main"]}
if e.Source.Seat != "" {
for _, sibling := range []string{"mesh-lab", "mesh-sdk"} {
if url, err := clone(inventory.Source{Seat: e.Source.Seat, Repository: siblingOf(e.Source.Repository,
sibling)}); err == nil {
beside[sibling] = link.CheckedOut{Repository: url, Ref: refs[sibling]}
}
if lab, err := clone(inventory.Source{Seat: e.Source.Seat, Repository: siblingOf(e.Source.Repository,
"mesh-lab")}); err == nil {
beside["mesh-lab"] = link.CheckedOut{Repository: lab, Ref: refs["mesh-lab"]}
}
}
}
@@ -275,54 +271,17 @@ func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scop
// catalogue, whose checkout beside is what tests read its files from, at its main, what the next merge
// builds from; and beside the controller its main, for a judge the running controller predates, and the
// lab's main, whose replays every check runs. **One rule, read by the check the controller asks for and by
// the facts snapshot** (Facts.Beside), so a check run by hand clones what the build seat clones. Beside the
// controller also the SDK, checked out at the commit the running controller's go.mod pins (sdkPinned), not
// one a desktop holds (novox/hq issue 449). The checkout only places the clone: the conformance test reads the
// fixtures at the pin of the tree under check, from the clone's history, so a pull request moving the SDK is
// judged against the SDK it moves to (internal/link/conformance_test.go).
// the facts snapshot** (Facts.Beside), so a check run by hand clones what the build seat clones.
func besideRefs(dir, running string) map[string]string {
switch dir {
case "mesh-catalog":
return map[string]string{dir: "main"}
case "mesh-controller":
return map[string]string{dir: running, "mesh-controller-main": "main", "mesh-lab": "main",
"mesh-sdk": sdkPinned()}
return map[string]string{dir: running, "mesh-controller-main": "main", "mesh-lab": "main"}
}
return map[string]string{dir: running}
}
// sdkModule is the Go module of the SDK the controller is built against.
const sdkModule = "git.novox.be/novox/mesh-sdk/go"
// pseudoCommit is the commit a Go pseudo-version names: v0.1.11-0.20261009143344-f047d0a4a970 → f047d0a4a970.
var pseudoCommit = regexp.MustCompile(`-([0-9a-f]{12})$`)
// sdkPinned is the ref of the SDK repository this controller was built from, as its go.mod pins it and its
// build records it: a pseudo-version's commit, or a release's tag (the SDK tags its Go module under go/).
// "main" only for a binary that records no SDK version — a local replace — which a running controller is
// not (novox/hq issue 449).
func sdkPinned() string {
info, ok := debug.ReadBuildInfo()
if !ok {
return "main"
}
for _, dep := range info.Deps {
if dep.Path != sdkModule {
continue
}
if dep.Replace != nil {
dep = dep.Replace
}
if m := pseudoCommit.FindStringSubmatch(dep.Version); m != nil {
return m[1]
}
if strings.HasPrefix(dep.Version, "v") {
return "go/" + dep.Version
}
}
return "main"
}
// siblingOf is another repository of the same owner: novox/mesh-controller → novox/mesh-lab.
func siblingOf(repository, name string) string {
if cut := strings.LastIndex(repository, "/"); cut >= 0 {
cmd/mesh-controller/data_test.go
-38
View File
@@ -169,44 +169,6 @@ func TestAShrinkOfMoreThanHalfIsUrgent(t *testing.T) {
}
}
// THE FALSE ALARM (issue 368), replayed through the store D13 reads: an agent's home moved from the
// operator's own home (94.7 MB) to the agent account's fresh one (490 B), and `data-shrank` was raised
// for data that was never lost. A moved item is read against its new path only, so nothing is raised —
// and a genuine shrink at the new path, a week of history later, still is.
func TestAMovedPathIsNoShrinkAndAShrinkThereStillIs(t *testing.T) {
inv := inventory.ForTest(t)
ctx := t.Context()
shelf := shelfFor(t, houseManifest)
declared := []inventory.DeclaredData{{Module: "house", Item: "config", Class: "irreplaceable", Owned: true}}
start := time.Now().Add(-6 * time.Hour)
measure := func(at time.Time, path string, size int64) []conditions.Observation {
t.Helper()
if _, err := inv.RecordData(ctx, "home", declared, map[string]map[string]inventory.Measurement{"house": {
"config": {Path: path, Size: bytesOf(size), MeasuredAt: when(at), LastWrite: when(at),
LastBackup: when(at)}}}, "", at); err != nil {
t.Fatal(err)
}
records, err := inv.Data(ctx)
if err != nil {
t.Fatal(err)
}
peaks, err := inv.DataPeaks(ctx, at.Add(-shrinkWindow))
if err != nil {
t.Fatal(err)
}
return dataFindings(records, peaks, shelf, nil, nil, at)
}
measure(start, "/home/operator/.claude", 94_700_000)
if got := findingsByKind(measure(start.Add(10*time.Minute), "/home/agent/.claude", 490)); got[kindDataShrank].Kind != "" {
t.Fatalf("a moved path raised a shrink: %+v", got[kindDataShrank])
}
measure(start.Add(2*time.Hour), "/home/agent/.claude", 300<<20)
got := findingsByKind(measure(start.Add(4*time.Hour), "/home/agent/.claude", 1<<20))[kindDataShrank]
if got.Severity != conditions.Urgent || !strings.Contains(got.Summary, "shrank") {
t.Fatalf("a genuine shrink at the new path was not raised: %+v", got)
}
}
// Data said to be written all the time and not written; data with no backup or an old one — urgent when
// irreplaceable, a warning when valuable; and a new item given its bound before it is said.
func TestQuietDataAndMissingBackupsAreSaidByClass(t *testing.T) {
cmd/mesh-controller/deadletters.go
+1 -5
View File
@@ -144,15 +144,11 @@ func actOnDeadLetter(ctx context.Context, on *busHandles, act string, id uint64,
}
switch act {
case "deliver":
delivered, to, err := link.DeliverAgain(on.js, id)
_, to, err := link.DeliverAgain(on.js, id)
if err != nil {
return nil, err
}
answer["delivered_on"] = to
if delivered.Original != "" {
// What became of the ask in its seat's queue (novox/hq issue 334): removed, or left, and why.
answer["original"] = delivered.Original
}
answer["done"] = fmt.Sprintf("dead letter %d was delivered again to %s, and nobody else; it is no longer kept",
id, consumerWho(d.Stream, d.Consumer))
case "drop":
cmd/mesh-controller/delivery.go
-72
View File
@@ -596,81 +596,12 @@ func deliveryCommand(ctx context.Context, args []string) error {
if err != nil {
return err
}
// Each walk's phases, with the rest's reports (novox/hq ADR 0282 decision 6).
now := time.Now()
for i := range walks {
walks[i].Phases = walks[i].WalkPhases(now, appliedFrom(ctx, inv))
}
return answer(map[string]any{"held": deliverySeatHeld(entries), "walks": walks,
"own-path": sortedKeysOf(ownPathWords())})
}
return fmt.Errorf("delivery %s: plan, order, check, go, stop or walks", sub)
}
// appliedFrom answers a machine's first report after a send from the controller's `apply` durations; a lookup
// that fails is a report not read, which leaves the walk's end unknown rather than wrong.
func appliedFrom(ctx context.Context, inv *inventory.Inventory) inventory.AppliedLookup {
return func(node string, sent time.Time) (inventory.AppliedReport, bool) {
r, ok, err := inv.FirstAppliedAfter(ctx, node, sent)
if err != nil {
fmt.Fprintf(os.Stderr, "the report of %s after %s could not be read: %v\n", node, sent.Format(time.RFC3339), err)
return inventory.AppliedReport{}, false
}
return r, ok
}
}
// recordWalkPhases keeps, once, each phase of every walk ended lately whose end is known, as a duration of kind
// walk-phase per class (novox/hq ADR 0282 decision 6): what `durations` summarises. A walk whose end is unknown
// past ApplySilentAfter keeps its measured phases without its total.
func recordWalkPhases(ctx context.Context, inv *inventory.Inventory, now time.Time) error {
recent, err := inv.RecentPlans(ctx, 30)
if err != nil {
return err
}
for _, p := range recent {
if p.State != inventory.PlanDone || p.Release != nil || now.Sub(p.Updated) > 2*time.Hour {
continue
}
anyKept, totalKept, err := inv.WalkPhasesKept(ctx, p.ID)
if err != nil {
return err
}
if totalKept {
continue
}
ph := p.WalkPhases(now, appliedFrom(ctx, inv))
if ph != nil && ph.End == nil && anyKept {
continue // kept without its end; kept again only once its end is known
}
if ph == nil || (ph.End == nil && now.Sub(p.Updated) < inventory.ApplySilentAfter+time.Minute) {
continue
}
class := ph.Class
if class == "" {
class = "unclassed"
}
for _, x := range ph.Phases {
if x.State != inventory.PhaseMeasured || x.Start == nil {
continue
}
if err := inv.RecordDuration(ctx, inventory.Duration{Kind: inventory.DurationWalkPhase,
Subject: class + "/" + x.Name, Ref: fmt.Sprintf("%s/%s/%d", p.ID, x.Name, x.Tier), Started: *x.Start,
Took: time.Duration(x.TookMS) * time.Millisecond, Detail: p.Named()}); err != nil {
return err
}
}
if ph.End != nil && ph.From != nil {
if err := inv.RecordDuration(ctx, inventory.Duration{Kind: inventory.DurationWalkPhase,
Subject: class + "/total", Ref: p.ID + "/total", Started: *ph.From,
Took: time.Duration(ph.TotalMS) * time.Millisecond, Detail: ph.Said}); err != nil {
return err
}
}
}
return nil
}
// ownPathWords is the controller's own path as words, for an answer.
func ownPathWords() map[string]string { return onTheControllersPath }
@@ -797,9 +728,6 @@ func sayPlanMoved(ctx context.Context, bus link.Bus, p inventory.Plan) {
}
func publishPlanMoved(ctx context.Context, bus link.Bus, p inventory.Plan) {
// Its phases so far (novox/hq ADR 0282 decision 6): the rest's reports come after the walk ends, and are
// read by whoever asks for the walk (`delivery walks`).
p.Phases = p.WalkPhases(time.Now(), nil)
body, err := json.Marshal(p)
if err != nil {
return
cmd/mesh-controller/delivery_conditions.go
+4 -26
View File
@@ -37,34 +37,12 @@ type stalledLine struct {
// Number is the pull request's, for a line of a head the forge never announced (novox/hq issue 347).
Number int `json:"number,omitempty"`
State string `json:"state"`
// Waiting are the merge checks a line of the state `unanswered` waits on, as mesh-delivery says each:
// "mesh/merge-gate pending since <UTC time>", "mesh/repo-check never set" (novox/hq issue 438).
Waiting []string `json:"waiting,omitempty"`
// Checks are the same merge checks as data, which the controller words in the operator's own time (novox/hq
// issue 443): a time inside a finished sentence cannot be said again in another zone. A mesh-delivery from before
// says none, and Waiting is said as it reads.
Checks []waitingCheck `json:"checks,omitempty"`
For string `json:"for"`
Bound string `json:"bound"`
H2 string `json:"h2"`
Says string `json:"says"`
For string `json:"for"`
Bound string `json:"bound"`
H2 string `json:"h2"`
Says string `json:"says"`
}
// waitingCheck is one merge check a stalled line waits on, as mesh-delivery gives it: its context, its state —
// "pending", or "never-set" — and, for a pending one, since when in RFC 3339, empty when the forge did not say.
type waitingCheck struct {
Context string `json:"context"`
State string `json:"state"`
Since string `json:"since,omitempty"`
}
// wordsNow is the time the words are said at, which says whether a time needs its day; a seam a test replaces.
var wordsNow = time.Now
// wordsZone is the zone a stalled line's times are said in: the controller's local zone, as every other time in its
// messages. A seam a test replaces, so that no test writes time.Local, which every goroutine of the package reads.
var wordsZone = func() *time.Location { return time.Local }
// operatorsOnly is whether the table leaves H2 nothing to do for the line: the state is the operator's.
func (l stalledLine) operatorsOnly() bool { return l.H2 == "" || strings.HasPrefix(l.H2, "none") }
cmd/mesh-controller/doctor.go
-11
View File
@@ -116,11 +116,6 @@ var probeRegistry = []probe{
{ID: probeDeliveriesID, Asserts: "no delivery is held past its state's bound unsaid: mesh-delivery's " +
"`stalled`, each with the transition its table lets healer H2 take", From: "ADR 0239",
Kind: kindDeliveryStalled, Phase: 3, run: probeDeliveries},
// The delivery budgets (novox/hq ADR 0282 decision 7): the newest delivery of each class within its budget,
// read from mesh-delivery's `times`; a measurement said, never a delivery held.
{ID: probeBudgetsID, Asserts: "the newest delivery of a leaf module ran on every machine within five minutes of " +
"its merge, and of a core module within ten: mesh-delivery's `times`", From: "ADR 0282",
Kind: kindOverBudget, Phase: 3, run: probeBudgets},
// A client of the bus reconnecting in a loop (novox/hq issue 327), from the server's record of closed
// connections, which the bus's own module reads.
{ID: probeReconnectsID, Asserts: "no user of the bus had its connection dropped more than twelve times in the " +
@@ -136,12 +131,6 @@ var probeRegistry = []probe{
{ID: "D-root", Asserts: "no agent can become root without a person on a machine where the router or a channel " +
"proving its sender runs: not by its own account, and not through a tool that runs its command as an account " +
"that can", From: "ADR 0259 §8", Kind: kindRootNotFree, Phase: 2, run: probeAgentRoot},
// A module's bucket or log filling toward its cap (novox/hq ADR 0297 §6): said at three quarters, cleared
// below seven tenths, so one at its threshold is not raised and cleared on every run. One that cannot be
// read is said on its own, and every other is still judged.
{ID: probeFillID, Asserts: "every module's bucket and log holds less than three quarters of its cap; one " +
"said filling is cleared once it holds less than seven tenths", From: "ADR 0297, issue 501",
Kind: kindBucketOrLogFilling, Raises: []string{kindBucketOrLogUnread}, Phase: 1, run: probeFill},
{ID: "DW", Asserts: "the watchdogs of the signals table ran within three of their intervals",
From: "ADR 0227 rule 6: the watchers are watched", Kind: "watchdogs-silent", Phase: 1, run: probeWatchdogs},
// The core's health definitions (novox/hq to-be 45 §8, ADR 0236): what a core component's new build is
cmd/mesh-controller/facts.go
-4
View File
@@ -514,10 +514,6 @@ func composedAndValidated(ctx context.Context, open *stores, node string, gens m
if declared.Epoch, err = open.inventory.SentEpoch(ctx, record.ID); err != nil {
return sendable{}, nil, err
}
// And the generation it was last sent, as the would-send is (novox/hq issue 234).
if declared.Generation, err = open.inventory.SentGeneration(ctx, record.ID); err != nil {
return sendable{}, nil, err
}
body, err := declared.Body()
if err != nil {
return sendable{}, nil, err
cmd/mesh-controller/failed_wait_provider_test.go
-66
View File
@@ -1,66 +0,0 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A provider whose wait fails the controller's check lists who waits on it (novox/hq issue 450).
//
// A wait that does not check out is judged unhealthy (ADR 0283 decision 3), so the provider raises its unhealthy
// condition and its consumers are held under it (ADR 0240 rule 5). What is stored is what the machine said, the
// wait unchecked: read as said, the provider seems to wait for the operator, and the held consumers were never
// listed on the condition that is open.
// refusedWait is a wait for a secret the database's manifest does not declare: it fails the check.
var refusedWait = inventory.Wait{Part: "postgres-database", Secret: "certificate", What: "the database's certificate"}
// judgedRefused is the provider's resource as the controller judges it: unhealthy, saying why.
func judgedRefused() inventory.ResourceHealth {
r := waitingDatabaseFor(refusedWait)
r.State, r.Waits = link.StateUnhealthy, nil
r.Reason = "says it waits for the secret certificate, which db does not declare"
return r
}
// The consumer's statement arrives after the provider's, so it is the consumer's judging (sayWaiters) that must list
// it at the provider's unhealthy condition, made urgent by who waits on it.
func TestAProviderWhoseWaitFailedItsCheckListsWhoWaitsOnIt(t *testing.T) {
k, _ := withConditionsInMemory(t)
stored := waitingDatabaseFor(refusedWait)
open := judgeBoth(t, k, []inventory.ResourceHealth{judgedRefused()},
map[string]inventory.NodeHealth{"anchor": {Node: "anchor", Resources: []inventory.ResourceHealth{stored}}},
shopFailingBeside(stored))
provider := conditionOf(open, moduleUnhealthyKey("db", "anchor"))
if len(open) != 1 || provider == nil {
t.Fatalf("a provider whose wait failed its check and a consumer of it raised %v; want the provider's "+
"unhealthy alone", openKeysOf(open))
}
if said := provider.Evidence[0].Said; !strings.Contains(said, "shop on laptop") {
t.Fatalf("the provider's unhealthy condition does not list shop on laptop as waiting on it: %s", said)
}
if provider.Severity != conditions.Urgent {
t.Fatalf("the provider's unhealthy condition is %s with a consumer waiting on it; want urgent", provider.Severity)
}
}
// A provider that waits for a secret already given is unhealthy to its consumers too, before its own condition opens:
// they are held under it as under any unhealthy provider, never as waiting for the operator, and its condition, when
// open, is said as unhealthy with who waits on it.
func TestAProviderWaitingForASecretAlreadyGivenIsUnhealthyToItsConsumers(t *testing.T) {
given := holdingOf(nil, shopFailingBeside(waitingDatabase()), shopOnTheDatabase)
given.waits.given["db@anchor"] = map[string]time.Time{"licence": time.Now().Add(-time.Hour)}
by, held := given.heldWith("laptop", "shop", []inventory.ResourceHealth{failingConsumer("shop")})
if !held || by.provider != theDatabase || len(by.waits) > 0 {
t.Fatalf("shop under a database waiting for a licence already given: held %v under %v with waits %v; want "+
"held under db on anchor as unhealthy, no waits", held, by.provider, by.waits)
}
if _, uncovered := given.waitingUncovered("laptop", "shop", []inventory.ResourceHealth{failingConsumer("shop")}); uncovered {
t.Fatalf("a provider whose wait failed its check is said as waiting for another part")
}
}
cmd/mesh-controller/gate.go
+11 -79
View File
@@ -121,13 +121,10 @@ type gateFacts struct {
health map[string]inventory.NodeHealth
healthErr error
// heldOn is, per "<module>@<machine>", the provider its findings are held under (ADR 0240 rule 5).
heldOn map[string]heldReading
heldOn map[string]string
// groupsAdded is, per module, whether the move judged puts an account in a group its previous build did
// not (issue 318 review): the only move whose wait for a new login is excused.
groupsAdded map[string]bool
// waits is what the controller holds to check a module's wait for the operator (novox/hq ADR 0283): the
// manifest of each module's build judged, and the secrets given on each machine.
waits operatorWaitFacts
// sent is, per machine, the declaration the gate's own send carried there (novox/hq issue 352): a
// report is held against it, never against the send made last. sentBuilds is what each machine was
// last sent of every module, and judged the commit of each module this gate judges: a machine last
@@ -137,29 +134,6 @@ type gateFacts struct {
commits map[string]string
}
// heldReading is the provider a module's findings are held under, as "<module> on <machine>", and, when that
// provider only waits for the operator for the part the module needs, its wait in one sentence (novox/hq issue
// 405): the module's gate then reads as a wait for a person (ADR 0254), a pass carrying the wait, as ADR 0283
// decision 4 reads the waiting provider itself. A walk never waits on the operator's secret, there or here.
type heldReading struct {
on, waits string
}
// heldReadings is, per "<module>@<machine>" in every machine's newest statement, what its findings are held under.
func heldReadings(hold *holding) map[string]heldReading {
out := map[string]heldReading{}
for machine := range hold.healths {
for module, by := range hold.heldModules(machine) {
reading := heldReading{on: by.provider.Module + " on " + by.provider.Node}
if len(by.waits) > 0 {
reading.waits = operatorWaitSaid(by.provider.Module, by.provider.Node, by.waits)
}
out[module+"@"+machine] = reading
}
}
return out
}
// reportedOn says a machine's last report is on what the gate sent it (novox/hq issue 352): on that
// declaration, or one it was sent after it — or, for a gate kept before sends were kept on it, on the
// declaration last sent. On 2026-10-09 a release's gate read the control node's report against a newer
@@ -225,7 +199,12 @@ var gatherGateFacts = func(ctx context.Context, open *stores, component string)
// Whose findings wait on an unhealthy provider (ADR 0240 rule 5): their gates wait, not fail.
if f.healthErr == nil && f.openErr == nil {
if hold, err := readHolding(ctx, inv, f.open); err == nil {
f.heldOn = heldReadings(hold)
f.heldOn = map[string]string{}
for machine := range f.health {
for module, p := range hold.heldModules(machine) {
f.heldOn[module+"@"+machine] = p.Module + " on " + p.Node
}
}
}
}
if theLease != nil {
@@ -278,11 +257,10 @@ func judgeHealth(module, component string, m catalogue.Manifest, machine string,
firstLine(f.openErr.Error())
}
for _, c := range f.open {
// A wait for a person's new login, for a directory used as found to be handed over, or for the
// operator's secret or setting, is the module's reading, not a fault raised since the send: the gate
// reads it from the statement below (ADR 0254, novox/hq issue 339, ADR 0283).
if c.Source == gateProbe || c.OpenAt(since) || c.Kind == kindReloginNeeded || c.Kind == kindUsedAsFound ||
c.Kind == kindNeedsOperator {
// A wait for a person's new login, or for a directory used as found to be handed over, is the module's
// reading, not a fault raised since the send: the gate reads it from the statement below (ADR 0254,
// novox/hq issue 339).
if c.Source == gateProbe || c.OpenAt(since) || c.Kind == kindReloginNeeded || c.Kind == kindUsedAsFound {
continue
}
onIt := c.Subject.Machine == machine || slices.Contains(c.Subject.Also, machine) ||
@@ -499,7 +477,6 @@ func judgeMoves(ctx context.Context, open *stores, g *inventory.PlanGate, pairs
return "", err
}
facts.groupsAdded = movesAddingGroups(ctx, open.inventory, g, pairs, shelf)
facts.waits = gateWaitFacts(ctx, open.inventory, g, pairs, shelf, facts.health)
facts.sent = g.Sent
facts.commits, facts.sentBuilds = judgedCommits(g, pairs), map[string]map[string]string{}
// A module this gate put back at once (putBackBroken) was sent its earlier build by the gate itself:
@@ -626,7 +603,6 @@ func judgeMoves(ctx context.Context, open *stores, g *inventory.PlanGate, pairs
pastBound := now.Sub(*g.Since) > gateBound
switch {
case worst == healthBroken:
g.Read(now, false, g.BrokenWhy)
var judging []string
for _, m := range modules {
if reading[m] != healthBroken && !passedAlone(m) {
@@ -645,10 +621,8 @@ func judgeMoves(ctx context.Context, open *stores, g *inventory.PlanGate, pairs
// Waiting on a provider that is unhealthy: not a pass, and not a failure at the bound either —
// the provider's own condition says what is wrong (ADR 0240 rule 5).
g.Passes, g.LastPass, g.Last, g.Failing = 0, nil, why, failing
g.Read(now, false, why)
case worst == healthNotYet:
g.Passes, g.LastPass, g.Last, g.Failing = 0, nil, why, failing
g.Read(now, false, why)
if pastBound {
fail(fmt.Sprintf("not healthy within %s of its apply: %s", gateBound, why))
}
@@ -656,7 +630,6 @@ func judgeMoves(ctx context.Context, open *stores, g *inventory.PlanGate, pairs
// Healthy, or waiting for a person (ADR 0254): a pass, the wait carried along in the verdict.
g.Passes++
g.LastPass, g.Last, g.Failing = &now, "", nil
g.Read(now, true, "")
if g.Passes >= gatePasses && settled {
decide(g, inventory.GatePassed, fmt.Sprintf("healthy %d times over %s", g.Passes,
now.Sub(*g.Since).Round(time.Second))+waitsSaid(g.Waits), now)
@@ -774,47 +747,6 @@ func movesAddingGroups(ctx context.Context, inv *inventory.Inventory, g *invento
return out
}
// gateWaitFacts reads what checks the waits for the operator of the modules a gate judges (novox/hq ADR 0283): the
// manifest of the build judged — the one it moves to, else the catalogue's — and the secrets given on each machine
// that says a module of them waits.
func gateWaitFacts(ctx context.Context, inv *inventory.Inventory, g *inventory.PlanGate, pairs []judged,
shelf map[string]catalogue.Manifest, health map[string]inventory.NodeHealth) operatorWaitFacts {
var f operatorWaitFacts
judgedManifests := map[string]catalogue.Manifest{}
byMachine := map[string][]string{}
for _, j := range pairs {
waiting := false
for _, r := range health[j.node].Resources {
if r.Module == j.module && r.State == link.StateWaiting {
waiting = true
}
}
if !waiting {
continue
}
byMachine[j.node] = append(byMachine[j.node], j.module)
if _, done := judgedManifests[j.module]; done {
continue
}
to := g.To
for _, c := range g.Carried {
if c.Module == j.module {
to = c.To
break
}
}
if m, found, err := inv.ManifestAt(ctx, j.module, to); err == nil && found {
judgedManifests[j.module] = m
} else if m, known := shelf[j.module]; known {
judgedManifests[j.module] = m
}
}
for machine, modules := range byMachine {
readWaitFacts(ctx, inv, machine, modules, judgedManifests, &f)
}
return f
}
// decide sets a gate's verdict.
func decide(g *inventory.PlanGate, verdict, why string, now time.Time) {
g.Verdict, g.Why, g.JudgedAt = verdict, why, &now
cmd/mesh-controller/generation.go
-222
View File
@@ -1,222 +0,0 @@
package main
import (
"context"
"fmt"
"io"
"os"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The assignment generation a declaration was composed from, and the record of every send (novox/hq
// issue 234).
//
// On 2026-10-04 a declaration newer in sequence than every other named four fewer modules than the
// assignments held, and a machine applied it and undeclared all four. The sequence orders arrival and
// cannot tell a later send that carries an older view of the assignments. So a declaration now carries
// the generation of the assignments it was composed from — a counter the store raises in the same
// transaction as every assignment change — and a machine refuses one older than it applied, unless it is
// a gate's put-back. And every send is written down with who sent it, from which generation and naming
// which modules: the controller logged no send then, and which process sent the stale declaration could
// not be read back from anything the mesh kept.
// kindOlderGeneration is S20's kind: a machine refused a send for the generation it was composed from.
const kindOlderGeneration = "older-generation"
// generationRefusalsSaid is how long a refused send is said after its refusal: an hour, as an advisory is.
const generationRefusalsSaid = advisoryQuiet
// stamp gives a composed declaration the order allotted to it before it was composed: its sequence, and
// the epoch and generation when the machine reads them — and keeps the generation and acting epoch for the
// record of the send whether or not the machine is sent them.
//
// **No put-back mark is sent.** A gate puts a machine back by composing it again (sendRollout), so its
// declaration is allotted here like any other and carries the generation as it stands — never older than
// what the machine applied. The node-engine reads a `put_back` key (mesh-host#82); this controller never
// needs to send one.
func (o order) stamp(d *sendable) {
d.Sequence, d.Epoch = o.sequence, o.epoch
d.composedFrom, d.actingEpoch = o.generation, o.acting
d.Generation = 0
if o.readsGeneration && o.generation > 0 {
d.Generation = o.generation
}
}
// sentRecord is what the record of a send keeps beside its digest.
type sentRecord struct {
sequence int64
epoch uint64
generation int64
// toldGeneration is the generation on the wire, which the would-send is stamped with: zero for a machine
// whose node-engine has not said it reads one.
toldGeneration int64
sender string
modules []string
}
// sentRecordOf is a composed send's record: its sender is the caller this process acts for.
func sentRecordOf(ctx context.Context, d sendable) sentRecord {
return sentRecord{sequence: d.Sequence, epoch: d.actingEpoch, generation: d.composedFrom,
toldGeneration: d.Generation, sender: senderOf(callerOf(ctx)), modules: d.modules}
}
// callerOf is who asked for what this process does: the seat call's caller when ctx belongs to one, the
// caller the controller ran this command for, or the account at the shell.
func callerOf(ctx context.Context) string {
if c := link.CallerIn(ctx); c != "" {
return c
}
return link.Caller()
}
// senderOf is a send's sender in words: who asked, and the process and build that composed it — the
// answer issue 234 could not find, because two controllers and a one-shot push were all sending then.
func senderOf(caller string) string {
host, _ := os.Hostname()
return fmt.Sprintf("%s (pid %d on %s, build %s)", caller, os.Getpid(), host, version)
}
// namedModules are the modules a declaration names: its machine's set, less what was left out of it.
func namedModules(plan catalogue.Resolution, leftOut map[string]string) []string {
out := make([]string, 0, len(plan.Modules))
for _, m := range plan.Modules {
if _, left := leftOut[m.Module]; !left {
out = append(out, m.Module)
}
}
sort.Strings(out)
return out
}
// heardGenerationRefusal keeps a machine's refusal of a send for its generation, so S20 names its sender
// and says it (novox/hq issue 234): on the send it refused or — when the mesh has no record of sending that
// sequence — as a refusal of its own, naming the sender as unknown. Either way S20 is raised: a refusal the
// mesh cannot attribute is the louder fact, not a quieter one.
//
// **And raises the mesh's counter past what the machine applied, when the refused send was composed from
// the counter as it stands** (counterBehind): then the sender's view was not stale — the counter is behind
// the machine, which is a store put back from a backup — and every send after would be refused for ever. A
// stale sender's generation is below the counter, and the counter is left alone for it.
//
// Nothing is sent from here. This runs in the controller's receive loop, and a push from it would hold that
// loop, and the machine's hold, for as long as the push takes — the deaf controller of issues 184 and 185.
// S20 names `push <node>` for that case instead.
func heardGenerationRefusal(ctx context.Context, inv *inventory.Inventory, report link.Report) {
r := report.OlderGeneration
if r == nil || inv == nil || report.Node == "" {
return
}
node, err := inv.NodeByName(ctx, report.Node)
if err != nil {
fmt.Fprintf(os.Stderr, "mesh-controller: %s refused a send for its generation, and the machine cannot be "+
"read, so it is not raised: %v\n", report.Node, err)
return
}
var raised int64
if now, err := inv.AssignmentGeneration(ctx); err != nil {
fmt.Fprintf(os.Stderr, "mesh-controller: %s refused a send for its generation, and the mesh's own cannot be "+
"read: %v\n", report.Node, err)
} else if counterBehind(*r, now) {
if raised, err = inv.RaiseAssignmentGeneration(ctx, r.Applied); err != nil {
fmt.Fprintf(os.Stderr, "mesh-controller: the assignment generation (%d) is behind what %s applied (%d), "+
"and could not be raised: %v\n", now, report.Node, r.Applied, err)
raised = 0
} else {
fmt.Fprintf(os.Stderr, "mesh-controller: the assignment generation was %d, behind what %s applied (%d) — "+
"a store put back from a backup — and is raised to %d; `push %s` sends it what the mesh holds now\n",
now, report.Node, r.Applied, raised, report.Node)
}
}
send, found, err := inv.RefusedSend(ctx, node.ID, report.Sequence, r.Applied, raised)
if err == nil && !found {
send, err = inv.RecordUnrecordedRefusal(ctx, inventory.Send{Node: node.ID, Sequence: report.Sequence,
Epoch: report.Epoch, Generation: r.Generation, Digest: report.Declared, RefusedApplied: r.Applied,
CounterRaisedTo: raised})
}
if err != nil {
fmt.Fprintf(os.Stderr, "mesh-controller: %s refused send %d for its generation, and the refusal could not be "+
"kept, so it is not raised: %v\n", report.Node, report.Sequence, err)
return
}
fmt.Fprintf(os.Stderr, "mesh-controller: %s refused send %d from %s: composed from assignment generation %d, "+
"and it applied %d\n", report.Node, report.Sequence, send.Sender, r.Generation, r.Applied)
}
// counterBehind says a refusal shows the mesh's counter behind the machine rather than a stale sender: the
// refused send carried the counter as it stands now, and the machine applied more than that.
func counterBehind(r link.GenerationRefusal, now int64) bool {
return r.Generation >= now && r.Applied > r.Generation
}
// watchGenerationRefusals is S20: every send a machine refused within the hour for the generation it was
// composed from, naming who sent it (novox/hq issue 234). One per machine, the newest refusal.
func watchGenerationRefusals(f *signalFacts) []conditions.Observation {
seen := map[string]bool{}
var out []conditions.Observation
for _, s := range f.refusedSends {
if s.RefusedAt == nil || f.now.Sub(*s.RefusedAt) > generationRefusalsSaid || seen[s.NodeName] {
continue
}
seen[s.NodeName] = true
o := conditions.Observation{Scope: conditions.ScopeMachine, ID: s.NodeName, Kind: kindOlderGeneration,
Machine: s.NodeName, Severity: conditions.Warning,
Summary: fmt.Sprintf("%s refused sequence %d from %s: it was composed from assignment generation %d, and "+
"%s applied generation %d — a sender composing from a view of the assignments the mesh has moved "+
"past; it named %s", s.NodeName, s.Sequence, s.Sender, s.Generation, s.NodeName, s.RefusedApplied,
modulesWords(s.Modules)),
Said: fmt.Sprintf("refused at %s", s.RefusedAt.UTC().Format(time.RFC3339)),
Headline: fmt.Sprintf("%s refused an out-of-date update", s.NodeName),
Explanation: fmt.Sprintf("Something sent %s an update made from an older list of what runs there. %s "+
"refused it, so nothing was removed. Nothing for you to do unless it repeats.", s.NodeName, s.NodeName),
Resolved: fmt.Sprintf("%s has had no out-of-date update for an hour", s.NodeName)}
if s.CounterRaisedTo > 0 {
// Not a stale sender: the mesh's counter was behind the machine (a store put back from a backup) and
// was raised; nothing has sent the machine its declaration since, so a person is asked to.
o.Summary = fmt.Sprintf("%s refused sequence %d from %s: it was composed from assignment generation %d, "+
"the mesh's own, and %s applied generation %d — the mesh's counter was behind the machine (a store "+
"put back from a backup?) and is raised to %d; `push %s` sends it what the mesh holds now",
s.NodeName, s.Sequence, s.Sender, s.Generation, s.NodeName, s.RefusedApplied, s.CounterRaisedTo,
s.NodeName)
o.Explanation = fmt.Sprintf("Needs you: push %s. The mesh's record was older than %s, so %s refused its "+
"update and kept what it had. The record is repaired; a push sends the update again.",
s.NodeName, s.NodeName, s.NodeName)
}
out = append(out, o)
}
return out
}
// modulesWords is a send's modules as a sentence says them.
func modulesWords(modules []string) string {
if len(modules) == 0 {
return "no module the mesh recorded"
}
return strings.Join(modules, ", ")
}
// writeLastSend says what a machine was last told, by whom and from which generation (novox/hq issue 234):
// nothing when the mesh has not recorded a send to it.
func writeLastSend(ctx context.Context, w io.Writer, inv *inventory.Inventory, node string) error {
s, found, err := inv.LastSend(ctx, node)
if err != nil || !found {
return err
}
generation := "no generation recorded"
if s.Generation > 0 {
generation = fmt.Sprintf("assignment generation %d", s.Generation)
}
fmt.Fprintf(w, "%s was last sent sequence %d at %s by %s, composed from %s, naming %s\n", node, s.Sequence, s.SentAt.Local().Format("2006-01-02 15:04:05"), s.Sender, generation, modulesWords(s.Modules))
if s.RefusedAt != nil {
fmt.Fprintf(w, " and refused it at %s: it had applied generation %d\n",
s.RefusedAt.Local().Format("2006-01-02 15:04:05"), s.RefusedApplied)
}
return nil
}
cmd/mesh-controller/generation_test.go
-148
View File
@@ -1,148 +0,0 @@
package main
import (
"encoding/json"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The assignment generation a declaration was composed from (novox/hq issue 234).
func bodyKeys(t *testing.T, s sendable) map[string]any {
t.Helper()
raw, err := s.Body()
if err != nil {
t.Fatal(err)
}
var keys map[string]any
if err := json.Unmarshal(raw, &keys); err != nil {
t.Fatal(err)
}
return keys
}
// **The generation goes on the wire only to a machine whose node-engine said it reads one**: an older
// node-engine decodes strictly and refuses an unknown key, whole. No put-back mark is ever sent: a gate
// composes its put-back afresh, with the generation as it stands.
func TestTheGenerationIsSentOnlyToAMachineThatReadsOne(t *testing.T) {
resources := []map[string]any{{"id": "a", "type": "file", "path": "/etc/a", "content": "x\n"}}
reads := order{sequence: 12, epoch: 57, generation: 40, readsGeneration: true, acting: 57}
var told sendable
told.Resources = resources
reads.stamp(&told)
keys := bodyKeys(t, told)
if keys["generation"] != float64(40) || keys["sequence"] != float64(12) {
t.Fatalf("a machine that reads a generation was sent %v", keys)
}
if _, there := keys["put_back"]; there {
t.Fatalf("a put-back mark was sent: %v", keys)
}
if told.composedFrom != 40 || told.actingEpoch != 57 {
t.Errorf("the send does not keep what it was composed from for its record: %+v", told)
}
older := order{sequence: 12, epoch: 57, generation: 40, readsGeneration: false, acting: 57}
var untold sendable
untold.Resources = resources
older.stamp(&untold)
if _, there := bodyKeys(t, untold)["generation"]; there {
t.Fatal("a machine whose node-engine never said it reads a generation was sent one")
}
if untold.composedFrom != 40 {
t.Errorf("the generation it was composed from is recorded whether or not it was sent: %+v", untold)
}
}
// **The sender is named**: the caller of the seat call when there is one, else the shell's account, and the
// process and build that composed it.
func TestASendNamesItsSender(t *testing.T) {
said := senderOf("g14.node-tools, through the mesh-controller seat")
if !strings.Contains(said, "g14.node-tools") || !strings.Contains(said, "pid ") || !strings.Contains(said, "build ") {
t.Fatalf("the sender reads %q", said)
}
}
// refusedSend is a send a machine refused for its generation at a moment.
func refusedSend(at time.Time) inventory.Send {
return inventory.Send{NodeName: "anchor", Sequence: 12, Epoch: 57, Generation: 38, RefusedApplied: 40,
Sender: "a one-shot push by jochen at a shell on anchor (pid 4242 on anchor, build 2026.10.11)",
Modules: []string{"docker"}, SentAt: at.Add(-time.Second), RefusedAt: &at, Recorded: true}
}
// **A refused send is raised naming its sender** (novox/hq issue 234): who sent it, from which generation,
// against which the machine applied, and its sequence — the facts that took a morning to look for.
func TestARefusedSendIsRaisedNamingItsSender(t *testing.T) {
now := time.Date(2026, 10, 11, 12, 0, 0, 0, time.UTC)
f := calm(now)
f.refusedSends = []inventory.Send{refusedSend(now.Add(-time.Minute))}
got := watchGenerationRefusals(f)
if len(got) != 1 {
t.Fatalf("%+v", got)
}
o := got[0]
if o.Key() != "machine.anchor.older-generation" || o.Machine != "anchor" {
t.Errorf("raised as %s about %q", o.Key(), o.Machine)
}
for _, want := range []string{"a one-shot push by jochen", "generation 38", "generation 40", "sequence 12"} {
if !strings.Contains(o.Summary, want) {
t.Errorf("the summary does not say %q: %s", want, o.Summary)
}
}
if o.Headline == "" || o.Explanation == "" || o.Resolved == "" {
t.Errorf("the condition is not worded for the operator: %+v", o)
}
}
// **A refusal of a send the mesh has no record of is raised too, its sender said to be unknown** — never
// only a line on stderr.
func TestARefusalOfAnUnrecordedSendIsRaisedSayingTheSenderIsUnknown(t *testing.T) {
now := time.Date(2026, 10, 11, 12, 0, 0, 0, time.UTC)
f := calm(now)
s := refusedSend(now.Add(-time.Minute))
s.Recorded, s.Sender, s.Modules = false, "a sender the mesh has no record of (no send of this sequence was recorded)", nil
f.refusedSends = []inventory.Send{s}
got := watchGenerationRefusals(f)
if len(got) != 1 || !strings.Contains(got[0].Summary, "no record of") {
t.Fatalf("an unattributed refusal was not raised as one: %+v", got)
}
}
// **When the refusal showed the mesh's counter behind the machine, the condition asks for a push** — it was
// raised, and nothing has sent the machine its declaration since — and does not say there is nothing to do.
func TestACounterBehindTheMachineAsksForAPush(t *testing.T) {
now := time.Date(2026, 10, 11, 12, 0, 0, 0, time.UTC)
f := calm(now)
s := refusedSend(now.Add(-time.Minute))
s.Generation, s.RefusedApplied, s.CounterRaisedTo = 12, 40, 41
f.refusedSends = []inventory.Send{s}
got := watchGenerationRefusals(f)
if len(got) != 1 {
t.Fatalf("%+v", got)
}
if !strings.Contains(got[0].Summary, "`push anchor`") || !strings.Contains(got[0].Explanation, "push anchor") ||
strings.Contains(got[0].Explanation, "Nothing for you to do") {
t.Errorf("a counter behind the machine does not ask for a push: %s / %s", got[0].Summary, got[0].Explanation)
}
}
// **Only a refusal of the counter as it stands is the counter behind**: a stale sender's generation is below
// it, and the counter is left alone for that.
func TestOnlyARefusalOfTheCounterAsItStandsRaisesIt(t *testing.T) {
for _, c := range []struct {
refused, applied, now int64
behind bool
}{
{refused: 12, applied: 40, now: 12, behind: true}, // the store was put back: the mesh says 12, the machine had 40
{refused: 38, applied: 40, now: 41, behind: false}, // a stale sender: the counter is already past
{refused: 38, applied: 40, now: 40, behind: false}, // a stale sender: the counter is where the machine is
} {
r := link.GenerationRefusal{Generation: c.refused, Applied: c.applied}
if got := counterBehind(r, c.now); got != c.behind {
t.Errorf("refused %d, applied %d, counter %d: behind %v, want %v", c.refused, c.applied, c.now, got, c.behind)
}
}
}
cmd/mesh-controller/held_back_test.go
+1 -1
View File
@@ -128,7 +128,7 @@ func (r *recordedDelivery) grant(context.Context, []readyNode) error { return ni
func (r *recordedDelivery) declare(ctx context.Context, s readyNode, body []byte) (string, error) {
r.declared = append(r.declared, s.node)
return recordSent(ctx, r.inv, s.node, body, s.declared.Builds, s.declared.Epoch, sentRecordOf(ctx, s.declared))
return recordSent(ctx, r.inv, s.node, body, s.declared.Builds, s.declared.Epoch)
}
// aResolver is a module built from a repository, at a commit, with something on the machine that
cmd/mesh-controller/issue_204_test.go
+1 -1
View File
@@ -77,7 +77,7 @@ func TestASendIsRecordedEvenWhenTheSenderIsBeingCancelled(t *testing.T) {
}
cancel() // the sender is going away: its context is cancelled between the send and the record
body := []byte(`{"declaration":1,"resources":[]}`)
digest, err := recordSent(ctx, inv, "anchor", body, nil, 0, sentRecord{sender: "a test"})
digest, err := recordSent(ctx, inv, "anchor", body, nil, 0)
if err != nil {
// NodeByName on the cancelled context may itself refuse; the record must still be possible
// through the detached context, so look the node up again on a live one.
cmd/mesh-controller/main.go
+1 -1
View File
@@ -311,7 +311,7 @@ func usage() {
the self-check: the last verdict, a run now, the probes, the signals' ages
healers [--days N] [--json] the healers, what they did lately, and their brake (to-be 45 §7)
durations [--kind K] [--days N] [--json]
apply, heartbeat, plan-tier, build and walk-phase durations
apply, heartbeat, plan-tier and build durations, per machine or module
collection [--json] kept archives held/unheld by a manifest, and what the sweep may let go
builder issue <name> a broker account for a build machine, scoped to build work,
delivered as the builder module's broker secret (module add it first)
cmd/mesh-controller/merge_gate_test.go
+3 -25
View File
@@ -390,10 +390,9 @@ func TestAWithheldPathIsStoodInForByAPath(t *testing.T) {
// controller's ask and by the facts — and finds the repository it checks from its origin.
func TestACheckByHandClonesWhatTheSeatClones(t *testing.T) {
for dir, refs := range map[string]map[string]string{
"mesh-catalog": {"mesh-catalog": "main"},
"mesh-host": {"mesh-host": "c0ffee"},
"mesh-controller": {"mesh-controller": "c0ffee", "mesh-controller-main": "main", "mesh-lab": "main",
"mesh-sdk": sdkPinnedByGoMod(t)},
"mesh-catalog": {"mesh-catalog": "main"},
"mesh-host": {"mesh-host": "c0ffee"},
"mesh-controller": {"mesh-controller": "c0ffee", "mesh-controller-main": "main", "mesh-lab": "main"},
} {
got := besideRefs(dir, "c0ffee")
for d, ref := range refs {
@@ -413,24 +412,3 @@ func TestACheckByHandClonesWhatTheSeatClones(t *testing.T) {
}
}
}
// sdkPinnedByGoMod is the SDK commit this tree's go.mod pins, read from the file rather than the build, so
// sdkPinned is held to what the repository says (novox/hq issue 449).
func sdkPinnedByGoMod(t *testing.T) string {
t.Helper()
raw, err := os.ReadFile(filepath.Join("..", "..", "go.mod"))
if err != nil {
t.Fatal(err)
}
for _, line := range strings.Split(string(raw), "\n") {
fields := strings.Fields(line)
if len(fields) >= 2 && fields[0] == sdkModule {
if m := pseudoCommit.FindStringSubmatch(fields[1]); m != nil {
return m[1]
}
return "go/" + fields[1]
}
}
t.Fatalf("go.mod requires no %s", sdkModule)
return ""
}
cmd/mesh-controller/module_health.go
+36 -166
View File
@@ -74,21 +74,9 @@ func stateHealth(ctx context.Context, inv *inventory.Inventory, k *conditions.Ke
kept := inventory.ResourceHealth{Module: r.Module, Resource: r.Resource, Kind: r.Kind, Target: r.Target,
State: r.State, Reason: r.Reason, Since: r.Since, Streak: r.Streak, Restarts: r.Restarts,
Check: r.Check, Needs: r.Needs, Account: r.Account, Root: r.Root}
for _, w := range r.Waits {
kept.Waits = append(kept.Waits, inventory.Wait{Part: w.Part, Secret: w.Secret, Setting: w.Setting, What: w.What})
}
resources = append(resources, kept)
}
// **A wait for the operator is checked before it is excused** (novox/hq ADR 0283): a waiting resource whose
// wait does not check out is judged unhealthy, saying why; one that does is kept beside the unhealthy ones, so
// judgeModuleHealth can say it as needs-operator. What is stored is what the machine said.
var wf operatorWaitFacts
if mods := waitingModules(resources); len(mods) > 0 {
readWaitFacts(ctx, inv, node, mods, nil, &wf)
}
for _, r := range checkWaiting(node, resources, wf) {
if (r.State == link.StateUnhealthy || r.State == link.StateWaiting) && r.Module != "" {
unhealthy[r.Module] = append(unhealthy[r.Module], r)
if r.State == link.StateUnhealthy && r.Module != "" {
unhealthy[r.Module] = append(unhealthy[r.Module], kept)
}
}
streaks := map[string]int{}
@@ -147,15 +135,16 @@ func judgeModuleHealth(ctx context.Context, inv *inventory.Inventory, k *conditi
}
standing := map[string]conditions.Condition{}
for _, c := range open {
if (c.Kind == kindModuleUnhealthy || c.Kind == kindReloginNeeded || c.Kind == kindUsedAsFound ||
c.Kind == kindNeedsOperator) &&
if (c.Kind == kindModuleUnhealthy || c.Kind == kindReloginNeeded || c.Kind == kindUsedAsFound) &&
c.Subject.Machine == node {
standing[c.Key] = c
}
}
hold, err := readHoldingFor(ctx, inv, open)
if err != nil {
return err
var hold *holding
if inv != nil {
if hold, err = readHolding(ctx, inv, open); err != nil {
return err
}
}
var problems []string
modules := make([]string, 0, len(unhealthy))
@@ -170,39 +159,6 @@ func judgeModuleHealth(ctx context.Context, inv *inventory.Inventory, k *conditi
heldOn := map[string]string{}
providers := map[catalogue.Chosen]bool{}
for _, m := range modules {
// **A part that waits for the operator is said as that** (novox/hq ADR 0283): its waits already checked,
// the operator's, never urgent, its words naming the act.
if waits, waiting := operatorWait(m, unhealthy[m]); waiting {
o := needsOperatorObservation(m, node, waits, unhealthy[m])
// **A provider waiting for the operator says who waits on it** (novox/hq issue 405), as one waiting for a
// login does: its consumers are held under it.
if hold != nil {
sayWaitingOn(&o, hold.waitersOn(catalogue.Chosen{Node: node, Module: m}))
}
seen[o.Key()] = true
became[m] = kindNeedsOperator
if _, isOpen := standing[o.Key()]; streaks[m] < moduleUnhealthyAfter && !isOpen {
continue
}
if _, err := k.Observe(ctx, o); err != nil {
problems = append(problems, err.Error())
}
continue
}
// **A wait for the operator beside anything else is said too** (novox/hq issue 405): a module with a checked
// wait beside a new login owed, a directory used as found or a fault of its own is said as that, and the
// operator's secret or setting it waits for was not mentioned until the other cleared. Its needs-operator
// condition stands beside the other, on the same looks; what follows judges the rest without the wait.
if waits, rest := besideAWait(m, unhealthy[m]); len(waits) > 0 {
o := needsOperatorObservation(m, node, waits, waitingOf(m, unhealthy[m]))
seen[o.Key()] = true
if _, isOpen := standing[o.Key()]; streaks[m] >= moduleUnhealthyAfter || isOpen {
if _, err := k.Observe(ctx, o); err != nil {
problems = append(problems, err.Error())
}
}
unhealthy[m] = rest
}
// **A directory used as found is said as that** (novox/hq issue 339): the operator's to hand over at the
// machine, never urgent — nothing is broken by the wait that a person was not told of — and its own kind,
// so the gate never reads it as a fault of the build that happened to be sent beside it.
@@ -239,24 +195,13 @@ func judgeModuleHealth(ctx context.Context, inv *inventory.Inventory, k *conditi
}
o := moduleUnhealthyObservation(m, node, unhealthy[m])
if hold != nil {
if by, held := hold.heldWith(node, m, unhealthy[m]); held {
// Held under the provider's condition: nothing of its own, and the provider's says it waits. The
// clearing line says which the provider is: unhealthy, or waiting for the operator (issue 405).
p := by.provider
heldOn[o.Key()] = p.Module + " on " + p.Node + ", which is unhealthy"
if len(by.waits) > 0 {
heldOn[o.Key()] = p.Module + " on " + p.Node + ", which waits for you"
}
if p, held := hold.heldUnder(node, m, unhealthy[m]); held {
// Held under the provider's condition: nothing of its own, and the provider's says it waits.
heldOn[o.Key()] = p.Module + " on " + p.Node
providers[p] = true
continue
}
sayWaitingOn(&o, hold.waitersOn(catalogue.Chosen{Node: node, Module: m}))
// A provider that waits for the operator for a part this finding cannot be matched to does not hold it
// (novox/hq issue 405): raised as its own, and saying the provider waits, so neither is hidden.
if p, waiting := hold.waitingUncovered(node, m, unhealthy[m]); waiting {
o.Said += fmt.Sprintf("; %s on %s waits for you, but not for anything %s is known to need, so %s's "+
"fault is said on its own", p.Module, p.Node, m, m)
}
}
seen[o.Key()] = true
became[m] = kindModuleUnhealthy
@@ -283,11 +228,8 @@ func judgeModuleHealth(ctx context.Context, inv *inventory.Inventory, k *conditi
if c.Kind == kindUsedAsFound {
why = fmt.Sprintf("%s says no directory of %s is used as found any more", node, module)
}
if c.Kind == kindNeedsOperator {
why = fmt.Sprintf("%s says %s no longer waits for the operator", node, module)
}
if on, held := heldOn[key]; held {
why = fmt.Sprintf("what %s finds on %s waits on %s: held under its condition", module, node, on)
why = fmt.Sprintf("what %s finds on %s waits on %s, which is unhealthy: held under its condition", module, node, on)
}
// **A condition that became the other kind** is not "working again" (issue 318 review): its clearing
// line says what it became.
@@ -296,12 +238,6 @@ func judgeModuleHealth(ctx context.Context, inv *inventory.Inventory, k *conditi
case c.Kind == kindModuleUnhealthy && became[module] == kindReloginNeeded:
why = fmt.Sprintf("%s on %s now waits only for a new login", module, node)
resolved = fmt.Sprintf("%s on %s now waits only for a new login", module, node)
case c.Kind == kindModuleUnhealthy && became[module] == kindNeedsOperator:
why = fmt.Sprintf("%s on %s now only waits for the operator", module, node)
resolved = fmt.Sprintf("%s on %s now only waits for you", module, node)
case c.Kind == kindNeedsOperator && became[module] == kindModuleUnhealthy:
why = fmt.Sprintf("%s on %s no longer only waits for the operator, and is not healthy", module, node)
resolved = fmt.Sprintf("What %s on %s waited for is given, and it still does not work", module, node)
case c.Kind == kindReloginNeeded && became[module] == kindModuleUnhealthy:
why = fmt.Sprintf("%s on %s no longer waits for a new login, and is not healthy", module, node)
resolved = fmt.Sprintf("The new login on %s is done, and %s still does not work", node, module)
@@ -326,74 +262,36 @@ func judgeModuleHealth(ctx context.Context, inv *inventory.Inventory, k *conditi
// sayWaiters observes a provider's open condition again, with who waits on it, from its machine's newest
// statement. Nothing when its condition is not open: it is raised by its own statements, on its own looks.
func sayWaiters(ctx context.Context, k *conditions.Keeper, hold *holding, p catalogue.Chosen, now time.Time) error {
// Its statement as it was judged, its waits checked (novox/hq issue 450): a wait that failed the check is
// unhealthy, so the condition built here is the one its own statement raised, and who waits on it is listed.
var raisedAt *conditions.Condition
for i, c := range hold.open {
if c.Key == moduleUnhealthyKey(p.Module, p.Node) || c.Key == reloginKey(p.Module, p.Node) {
raisedAt = &hold.open[i]
}
}
if raisedAt == nil {
return nil
}
var rs []inventory.ResourceHealth
for _, r := range hold.checked(p.Node) {
if r.Module == p.Module && (r.State == link.StateUnhealthy || r.State == link.StateWaiting) {
for _, r := range hold.healths[p.Node].Resources {
if r.Module == p.Module && r.State == link.StateUnhealthy {
rs = append(rs, r)
}
}
if len(rs) == 0 {
return nil
}
// The condition its own statement says it under: needs-operator while it only waits for the operator (novox/hq
// issue 405), else that for the rest of it, a wait beside it said on its own.
var o conditions.Observation
if waits, waiting := operatorWait(p.Module, rs); waiting {
o = needsOperatorObservation(p.Module, p.Node, waits, rs)
} else {
_, rest := besideAWait(p.Module, rs)
o = moduleUnhealthyObservation(p.Module, p.Node, rest)
if said, waits := personWait(p.Module, p.Node, rest); waits {
o = reloginObservation(p.Module, p.Node, said, operatorOn(ctx, hold.inv, p.Node), rest)
}
o := moduleUnhealthyObservation(p.Module, p.Node, rs)
if said, waits := personWait(p.Module, p.Node, rs); waits {
o = reloginObservation(p.Module, p.Node, said, operatorOn(ctx, hold.inv, p.Node), rs)
}
raised := false
for _, c := range hold.open {
raised = raised || c.Key == o.Key()
}
if !raised {
return nil // not open yet, or open as another kind: its own statement says it next
if o.Key() != raisedAt.Key {
return nil // its own statement says it next
}
sayWaitingOn(&o, hold.waitersOn(p))
_, err := k.Observe(ctx, o)
return err
}
// besideAWait is, for a module with something not healthy beside a part waiting for the operator, the waits (checked
// already) and the rest without the waiting parts (novox/hq issue 405); no waits when nothing waits, or when the
// module only waits (operatorWait says that whole). Pure.
func besideAWait(module string, rs []inventory.ResourceHealth) ([]inventory.Wait, []inventory.ResourceHealth) {
if _, only := operatorWait(module, rs); only {
return nil, rs
}
var waits []inventory.Wait
var rest []inventory.ResourceHealth
for _, r := range rs {
if r.Module == module && r.State == link.StateWaiting {
waits = append(waits, r.Waits...)
continue
}
rest = append(rest, r)
}
if len(waits) == 0 {
return nil, rs
}
return waits, rest
}
// waitingOf is a module's waiting resources: the evidence of its wait.
func waitingOf(module string, rs []inventory.ResourceHealth) []inventory.ResourceHealth {
var out []inventory.ResourceHealth
for _, r := range rs {
if r.Module == module && r.State == link.StateWaiting {
out = append(out, r)
}
}
return out
}
// reloginKey is a module's relogin-needed condition on a machine.
func reloginKey(module, node string) string {
return conditions.Key(conditions.ScopeModule, module+"."+node, kindReloginNeeded)
@@ -465,15 +363,12 @@ func waitingAccounts(module string, rs []inventory.ResourceHealth) []string {
}
// sayWaitingOn adds to a module's condition the consumers held under it (to-be 48 §6): urgent while anyone
// waits on it, whether it is not working or waits for a new login. **A wait for the operator's secret or setting
// stays a warning** (ADR 0283 decision 5, novox/hq issue 405): who waits on it is listed, and nothing escalates it.
// waits on it, whether it is not working or waits for a new login.
func sayWaitingOn(o *conditions.Observation, waiters []string) {
if len(waiters) == 0 {
return
}
if o.Kind != kindNeedsOperator {
o.Severity = conditions.Urgent
}
o.Severity = conditions.Urgent
o.Said += "; " + waitingWords(waiters)
o.Summary += fmt.Sprintf("; %d consumer(s) wait on it", len(waiters))
o.Explanation += fmt.Sprintf(" %d module(s) that depend on it wait for it.", len(waiters))
@@ -525,11 +420,6 @@ func reasonWords(r inventory.ResourceHealth) string {
case "":
return "is unhealthy"
}
// A wait for the operator that did not check out is said as the controller found it (ADR 0283): names of
// secrets and settings only, never what the check itself said.
if strings.HasPrefix(r.Reason, waitRefusedPrefix) {
return r.Reason
}
// What a declared check found says an endpoint, a path or an address: evidence, never the summary the
// operator's channel carries (ADR 0234 §6). The summary names the check.
if r.Check != "" {
@@ -621,9 +511,7 @@ func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (hea
if h.HeardAt.Before(since) {
return healthNotYet, fmt.Sprintf("%s has not said how what %s runs is since it was sent", machine, module)
}
// **A wait for the operator is checked first** (novox/hq ADR 0283): one that does not check out is unhealthy.
resources := checkWaiting(machine, h.Resources, f.waits)
wait, waits := personWait(module, machine, resources)
wait, waits := personWait(module, machine, h.Resources)
// **Only a build whose own send put the account in a new group is excused** (issue 318 review): read from
// what the controller sent, never from when the machine says the wait began — that time is the engine's
// memory, reset by its restart and moved by a change of words. A build that adds no account group cannot
@@ -631,18 +519,11 @@ func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (hea
if waits && !f.groupsAdded[module] {
waits = false
}
var found, onWaiting []string
var forOperator []inventory.Wait
for _, r := range resources {
var found []string
for _, r := range h.Resources {
if r.Module != module {
continue
}
// **Any build is excused while its wait for the operator checks out** (ADR 0283 decision 4): a secret not
// given is owed by every build alike, so it is no fault of this one, and the verdict carries it.
if r.State == link.StateWaiting {
forOperator = append(forOperator, r.Waits...)
continue
}
if waits && r.State == link.StateUnhealthy {
continue
}
@@ -660,16 +541,8 @@ func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (hea
return healthNotYet, fmt.Sprintf("its %s %s on %s is still starting", r.Kind, r.Resource, machine)
case link.StateUnhealthy:
if on, held := f.heldOn[module+"@"+machine]; held {
// **Held under a provider that only waits for the operator** (novox/hq issue 405): a wait for a
// person, a pass carrying the wait (ADR 0254, ADR 0283 decision 4) — the walk never waits for the
// operator's secret, whichever module owes it.
if on.waits != "" {
onWaiting = append(onWaiting, fmt.Sprintf("its %s %s on %s waits on %s, which waits for you: %s",
r.Kind, r.Resource, machine, on.on, on.waits))
continue
}
return healthWaiting, fmt.Sprintf("its %s %s on %s waits on %s, which is unhealthy", r.Kind,
r.Resource, machine, on.on)
r.Resource, machine, on)
}
return healthNotYet, fmt.Sprintf("its %s %s on %s %s", r.Kind, r.Resource, machine, reasonWords(r))
default:
@@ -677,14 +550,11 @@ func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (hea
reasonAfter(r.Reason))
}
}
if waits || len(found) > 0 || len(forOperator) > 0 || len(onWaiting) > 0 {
said := onWaiting
if waits || len(found) > 0 {
var said []string
if waits {
said = append(said, wait)
}
if len(forOperator) > 0 {
said = append(said, operatorWaitSaid(module, machine, forOperator))
}
if len(found) > 0 {
said = append(said, fmt.Sprintf("on %s, %s uses %s as found and waits for the operator to hand it over "+
"(`nox node hand-over %s <directory>` on the control-node)", machine, module, strings.Join(found, ", "), machine))
cmd/mesh-controller/network_test.go
+4 -5
View File
@@ -3,12 +3,10 @@ package main
import (
"context"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/overlay"
@@ -241,12 +239,13 @@ func TestATakeoverIsNotComposedForAHubPlacedOffItsTunnel(t *testing.T) {
}
}
// theResolver is the catalogue's dnsmasq module as it is (internal/beside).
// theResolver is the catalogue's dnsmasq module as it is, or the test is skipped where the
// catalogue is not beside this checkout.
func theResolver(t *testing.T) catalogue.Manifest {
t.Helper()
raw, err := os.ReadFile(filepath.Join(beside.Catalogue(t), "dnsmasq", "module.json"))
raw, err := os.ReadFile("../../../mesh-catalog/modules/dnsmasq/module.json")
if err != nil {
t.Fatal(err)
t.Skipf("the catalogue is not beside this checkout: %v", err)
}
m, err := catalogue.ParseManifest(raw)
if err != nil {
cmd/mesh-controller/operator_wait.go
-261
View File
@@ -1,261 +0,0 @@
package main
import (
"context"
"fmt"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A part that waits for the operator's secret or setting (novox/hq ADR 0283, issue 386).
//
// **A module's tool check may say it waits**: nothing of it is wrong but a part that cannot work until the operator
// gives one of its own secrets or one of its settings. The node-engine states such a resource `waiting`, with what
// it waits for. A module saying so is an assertion, so **the controller checks each wait before it excuses it**:
//
// - a wait for a secret names an own secret the module's manifest declares — by its name, or as a member of a
// secret family — said `"issued-by": "outside"`, and the store holds no value a person gave for it on that
// machine;
// - a wait for a setting names a setting the manifest declares (checked by name only: the controller cannot tell
// whether a free-form value covers a part, and the needs-operator condition is where a false one shows).
//
// An excused wait is read by the first-node gate as *waits for a person* (ADR 0254), a pass carried in the verdict,
// for any build of the module — a secret not given is owed by every build alike. It is said to the operator as
// `module.<module>.<machine>.needs-operator`, naming the act. A wait that fails the check is judged unhealthy, saying
// why, and raises the module's `unhealthy` condition.
// kindNeedsOperator is a module's condition while a part of it waits for the operator's secret or setting.
const kindNeedsOperator = "needs-operator"
// needsOperatorKey is a module's needs-operator condition on a machine.
func needsOperatorKey(module, node string) string {
return conditions.Key(conditions.ScopeModule, module+"."+node, kindNeedsOperator)
}
// operatorWaitFacts is what the controller holds to check a module's waits: the manifest judged per module, and per
// "<module>@<machine>" the own secrets a person gave there, with when. A module or a machine absent is not known,
// and no wait of it is excused.
type operatorWaitFacts struct {
manifests map[string]catalogue.Manifest
given map[string]map[string]time.Time
}
// checkWait is nil when a wait is excused, and otherwise why not, in words. Pure.
func checkWait(module, machine string, w inventory.Wait, f operatorWaitFacts) error {
m, known := f.manifests[module]
if !known {
return fmt.Errorf("says it waits for %s, and the mesh holds no manifest of %s to check it against", waitNames(w), module)
}
switch {
case w.Secret != "" && w.Setting != "", w.Secret == "" && w.Setting == "":
return fmt.Errorf("says it waits, naming %s, where a wait names one secret or one setting", waitNames(w))
case w.Setting != "":
if _, declared := m.Settings[w.Setting]; !declared {
return fmt.Errorf("says it waits for the setting %s, which %s does not declare", w.Setting, module)
}
return nil
}
own, _, declared := m.OwnSecrets.Lookup(w.Secret)
if !declared {
return fmt.Errorf("says it waits for the secret %s, which %s does not declare", w.Secret, module)
}
if own.IssuedBy != catalogue.IssuedOutside {
return fmt.Errorf("says it waits for the secret %s, which the mesh makes itself: only a secret issued outside "+
"the mesh waits for the operator", w.Secret)
}
given, readable := f.given[module+"@"+machine]
if !readable {
return fmt.Errorf("says it waits for the secret %s, and what was given on %s could not be read", w.Secret, machine)
}
if at, was := given[w.Secret]; was {
return fmt.Errorf("says it waits for the secret %s, which was given at %s", w.Secret,
at.UTC().Format("2006-01-02 15:04 MST"))
}
return nil
}
// waitRefusedPrefix opens every reason checkWait gives, so the words of a refused wait are told from a check's own.
const waitRefusedPrefix = "says it waits"
// waitNames is what a wait names, as "the secret x" or "the setting y".
func waitNames(w inventory.Wait) string {
switch {
case w.Secret != "" && w.Setting != "":
return "the secret " + w.Secret + " and the setting " + w.Setting
case w.Secret != "":
return "the secret " + w.Secret
case w.Setting != "":
return "the setting " + w.Setting
}
return "nothing"
}
// checkWaiting reads one machine's resources against the facts: every waiting resource whose waits all check out is
// kept as said; one with a wait that does not, or with no wait at all, is answered as unhealthy with why. Pure; the
// statement as kept is not changed.
func checkWaiting(machine string, rs []inventory.ResourceHealth, f operatorWaitFacts) []inventory.ResourceHealth {
out := make([]inventory.ResourceHealth, 0, len(rs))
for _, r := range rs {
if r.State == link.StateWaiting {
var why error
if len(r.Waits) == 0 {
why = fmt.Errorf("says it waits, and names nothing it waits for")
}
for _, w := range r.Waits {
if why == nil {
why = checkWait(r.Module, machine, w, f)
}
}
if why != nil {
r.State, r.Reason = link.StateUnhealthy, why.Error()
}
}
out = append(out, r)
}
return out
}
// operatorWait is whether everything not healthy of a module on a machine is waiting with its waits checked
// (checkWaiting already applied), and those waits. A module with anything unhealthy, starting or unknown beside it
// does not wait: it is judged as before.
func operatorWait(module string, rs []inventory.ResourceHealth) ([]inventory.Wait, bool) {
var waits []inventory.Wait
for _, r := range rs {
if r.Module != module {
continue
}
switch r.State {
case link.StateHealthy:
case link.StateWaiting:
waits = append(waits, r.Waits...)
default:
return nil, false
}
}
return waits, len(waits) > 0
}
// operatorWaitSaid is a module's wait for the operator in one sentence, for the gate's verdict and the condition's
// summary: what the operator gives and what it names, and for a secret the line that opens the desk prompt.
func operatorWaitSaid(module, machine string, waits []inventory.Wait) string {
var parts []string
for _, w := range waits {
part := fmt.Sprintf("%s (%s", w.What, waitNames(w))
if w.Secret != "" {
part += fmt.Sprintf(", given with `nox secret ask %s %s %s`", machine, module, w.Secret)
}
parts = append(parts, part+")")
}
return fmt.Sprintf("%s on %s waits for the operator: %s", module, machine, strings.Join(parts, "; "))
}
// needsOperatorObservation is a module whose only parts not healthy wait for the operator (ADR 0283): the operator's,
// a warning however long it stands, its plain words naming the act and never saying there is nothing to do.
func needsOperatorObservation(module, node string, waits []inventory.Wait, rs []inventory.ResourceHealth) conditions.Observation {
o := moduleUnhealthyObservation(module, node, rs)
o.Token, o.Kind, o.Resolver, o.Severity = kindNeedsOperator, kindNeedsOperator, conditions.ResolverOperator, conditions.Warning
o.Summary = operatorWaitSaid(module, node, waits)
w := needsOperatorWords(module, node, waits)
o.Headline, o.Explanation, o.Needs, o.Resolved, o.Actions = w.Headline, w.Explanation, w.Needs, w.Resolved, nil
return o
}
// needsOperatorWords is what the operator reads of a module waiting for them (ADR 0253, ADR 0283): the act, for a
// secret typed at the machine's desk prompt and for a setting approved when an agent proposes it. The secret's and
// the setting's names, and the line, are in the summary for whoever looks closer.
func needsOperatorWords(module, node string, waits []inventory.Wait) words {
var acts []string
seen := map[string]bool{}
secret := false
for _, w := range waits {
var act string
switch {
case w.Secret != "":
act, secret = fmt.Sprintf("type %s at %s's desk prompt", w.What, node), true
case w.Setting != "":
act = fmt.Sprintf("approve %s of %s on %s when it is proposed to you", w.Setting, module, node)
}
if act != "" && !seen[act] {
seen[act] = true
acts = append(acts, act)
}
}
needs := strings.Join(acts, "; and ") + "."
// Plain words hold one sentence of at most conditions.NeedsMax characters: several acts are named in the
// summary instead.
if len(acts) == 0 || len(needs) > conditions.NeedsMax {
needs = fmt.Sprintf("give what %s waits for on %s; the details name each secret and setting.", module, node)
}
explanation := fmt.Sprintf("Part of %s on %s cannot work until you give what it waits for.", module, node)
if secret {
explanation += " A hidden prompt opens at the desk when the secret is asked for, and what you type there " +
"is sealed to the machine."
}
explanation += " Its update is in place and nothing was undone; it carries on by itself once it is given."
return words{
Headline: fmt.Sprintf("%s waits for you on %s", module, node),
Needs: needs,
Explanation: explanation,
Resolved: fmt.Sprintf("%s on %s no longer waits for you", module, node),
}
}
// readWaitFacts reads what the controller holds to check the waits of the modules named on one machine: the
// manifests (the catalogue's, or those given) and the secrets given there. A read that fails leaves that module
// unknown, so none of its waits is excused.
func readWaitFacts(ctx context.Context, inv *inventory.Inventory, machine string, modules []string,
manifests map[string]catalogue.Manifest, f *operatorWaitFacts) {
if f.manifests == nil {
f.manifests = map[string]catalogue.Manifest{}
}
if f.given == nil {
f.given = map[string]map[string]time.Time{}
}
if inv == nil {
return
}
var shelf map[string]catalogue.Manifest
sort.Strings(modules)
for _, module := range modules {
if _, has := f.manifests[module]; !has {
if m, given := manifests[module]; given {
f.manifests[module] = m
} else {
if shelf == nil {
var err error
if shelf, err = inv.Catalogue(ctx); err != nil {
shelf = map[string]catalogue.Manifest{}
}
}
if m, known := shelf[module]; known {
f.manifests[module] = m
}
}
}
if _, read := f.given[module+"@"+machine]; read {
continue
}
if given, err := inv.GivenOwnSecrets(ctx, machine, module); err == nil {
f.given[module+"@"+machine] = given
}
}
}
// waitingModules is every module with a waiting resource in a statement.
func waitingModules(rs []inventory.ResourceHealth) []string {
seen := map[string]bool{}
var out []string
for _, r := range rs {
if r.State == link.StateWaiting && r.Module != "" && !seen[r.Module] {
seen[r.Module] = true
out = append(out, r.Module)
}
}
return out
}
cmd/mesh-controller/operator_wait_test.go
-264
View File
@@ -1,264 +0,0 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A part that waits for the operator's secret or setting (novox/hq ADR 0283, issue 386).
var mountsManifest = catalogue.Manifest{Module: "mounts", Version: "1",
Settings: map[string]catalogue.SettingDeclaration{"smb-users": {}, "sources": {}},
OwnSecrets: catalogue.OwnSecrets{
"smb-password-*": {Path: "/s/smb-password-*.secret", IssuedBy: catalogue.IssuedOutside},
"broker": {Path: "/s/broker"},
"made": {Path: "/s/made", Taken: catalogue.TakenAtStart},
"licence": {Path: "/s/licence", IssuedBy: catalogue.IssuedOutside},
}}
var passwordWait = inventory.Wait{Part: "the source games", Secret: "smb-password-games",
What: "the password of the source games"}
var usernameWait = inventory.Wait{Part: "the source games", Setting: "smb-users", What: "the username of the source games"}
func waitingResource(waits ...inventory.Wait) inventory.ResourceHealth {
return inventory.ResourceHealth{Module: "mounts", Resource: "mounts.watch", Kind: "process",
Target: "mesh-mounts-watch.service", State: link.StateWaiting, Check: "tool",
Reason: "the source games waits for its password", Waits: waits}
}
func factsGiven(given map[string]time.Time) operatorWaitFacts {
return operatorWaitFacts{manifests: map[string]catalogue.Manifest{"mounts": mountsManifest},
given: map[string]map[string]time.Time{"mounts@workstation": given}}
}
// Rule 3: a wait is excused only when what it names is the module's, issued outside the mesh, and not given there.
func TestAWaitIsExcusedOnlyWhenItChecksOut(t *testing.T) {
at := time.Date(2026, 10, 10, 15, 8, 0, 0, time.UTC)
for _, c := range []struct {
name string
w inventory.Wait
f operatorWaitFacts
says string // "" when excused
}{
{"a member of an outside family, not given", passwordWait, factsGiven(nil), ""},
{"an outside secret by name, not given", inventory.Wait{Part: "p", Secret: "licence", What: "w"}, factsGiven(nil), ""},
{"a declared setting", usernameWait, factsGiven(nil), ""},
{"a member given", passwordWait, factsGiven(map[string]time.Time{"smb-password-games": at}), "was given at 2026-10-10 15:08"},
{"a secret not declared", inventory.Wait{Part: "p", Secret: "smb-credentials", What: "w"}, factsGiven(nil), "does not declare"},
{"a secret the mesh makes", inventory.Wait{Part: "p", Secret: "made", What: "w"}, factsGiven(nil), "mesh makes itself"},
{"the bus account", inventory.Wait{Part: "p", Secret: "broker", What: "w"}, factsGiven(nil), "mesh makes itself"},
{"a setting not declared", inventory.Wait{Part: "p", Setting: "logins", What: "w"}, factsGiven(nil), "does not declare"},
{"both", inventory.Wait{Part: "p", Secret: "licence", Setting: "smb-users", What: "w"}, factsGiven(nil), "one secret or one setting"},
{"neither", inventory.Wait{Part: "p", What: "w"}, factsGiven(nil), "one secret or one setting"},
{"no manifest known", passwordWait, operatorWaitFacts{}, "no manifest"},
{"what was given cannot be read", passwordWait,
operatorWaitFacts{manifests: map[string]catalogue.Manifest{"mounts": mountsManifest}}, "could not be read"},
} {
err := checkWait("mounts", "workstation", c.w, c.f)
switch {
case c.says == "" && err != nil:
t.Errorf("%s: refused: %v", c.name, err)
case c.says != "" && (err == nil || !strings.Contains(err.Error(), c.says)):
t.Errorf("%s: %v; want a refusal saying %q", c.name, err, c.says)
}
}
}
// A waiting resource whose wait does not check out, or that names nothing, is judged unhealthy, saying why.
func TestAWaitThatFailsItsCheckIsUnhealthy(t *testing.T) {
given := factsGiven(map[string]time.Time{"smb-password-games": time.Now()})
got := checkWaiting("workstation", []inventory.ResourceHealth{waitingResource(passwordWait)}, given)
if got[0].State != link.StateUnhealthy || !strings.Contains(got[0].Reason, "was given") {
t.Fatalf("a wait for a secret given: %+v", got[0])
}
got = checkWaiting("workstation", []inventory.ResourceHealth{waitingResource()}, factsGiven(nil))
if got[0].State != link.StateUnhealthy || !strings.Contains(got[0].Reason, "names nothing") {
t.Fatalf("a wait naming nothing: %+v", got[0])
}
got = checkWaiting("workstation", []inventory.ResourceHealth{waitingResource(passwordWait, usernameWait)}, factsGiven(nil))
if got[0].State != link.StateWaiting {
t.Fatalf("two waits that check out: %+v", got[0])
}
}
// Rule 4: the gate passes a module whose only parts not healthy wait for the operator, carrying the wait; anything
// else beside it is judged as before; and any build is excused, not only one that added something.
func TestTheGatePassesAWaitForTheOperatorCarriedAlong(t *testing.T) {
now := time.Now()
since := now.Add(-time.Minute)
healthy := inventory.ResourceHealth{Module: "mounts", Resource: "mounts.apply", Kind: "process",
Target: "mesh-mounts-apply.service", State: link.StateHealthy}
f := gateFacts{now: now, waits: factsGiven(nil), groupsAdded: map[string]bool{"mounts": false},
health: map[string]inventory.NodeHealth{"workstation": {Node: "workstation", HeardAt: now,
Resources: []inventory.ResourceHealth{healthy, waitingResource(passwordWait)}}}}
h, why := moduleHealthWord("mounts", "workstation", since, f)
if h != healthPerson || !strings.Contains(why, "waits for the operator: the password of the source games") ||
!strings.Contains(why, "nox secret ask workstation mounts smb-password-games") {
t.Fatalf("an excused wait reads %v %q; want a wait for a person naming the act", h, why)
}
// A second resource unhealthy beside it: not yet, as before.
down := healthy
down.State, down.Reason = link.StateUnhealthy, "down"
f.health["workstation"] = inventory.NodeHealth{Node: "workstation", HeardAt: now,
Resources: []inventory.ResourceHealth{down, waitingResource(passwordWait)}}
if h, why := moduleHealthWord("mounts", "workstation", since, f); h != healthNotYet {
t.Fatalf("a resource down beside the wait reads %v %q", h, why)
}
// The password given and the module still saying it waits: not excused.
f.waits = factsGiven(map[string]time.Time{"smb-password-games": now})
f.health["workstation"] = inventory.NodeHealth{Node: "workstation", HeardAt: now,
Resources: []inventory.ResourceHealth{healthy, waitingResource(passwordWait)}}
if h, why := moduleHealthWord("mounts", "workstation", since, f); h != healthNotYet || !strings.Contains(why, "was given") {
t.Fatalf("a wait for a secret given reads %v %q", h, why)
}
// Facts never read (no manifest): never excused.
f.waits = operatorWaitFacts{}
if h, _ := moduleHealthWord("mounts", "workstation", since, f); h != healthNotYet {
t.Fatalf("a wait nothing could check reads %v", h)
}
}
func needsOperatorOpen(t *testing.T, k *conditions.Keeper) (*conditions.Condition, []conditions.Condition) {
t.Helper()
open, err := k.Open(t.Context())
if err != nil {
t.Fatal(err)
}
for i, c := range open {
if c.Key == needsOperatorKey("mounts", "workstation") {
return &open[i], open
}
}
return nil, open
}
// Rule 5: two statements of an excused wait raise needs-operator, the operator's, a warning however long, naming the
// act; a statement without it clears it.
func TestTheNeedsOperatorConditionNamesTheAct(t *testing.T) {
k, _ := withConditionsInMemory(t)
ctx := t.Context()
rs := map[string][]inventory.ResourceHealth{"mounts": {waitingResource(passwordWait)}}
if err := judgeModuleHealth(ctx, nil, k, "workstation", rs, map[string]int{"mounts": 1}, time.Now()); err != nil {
t.Fatal(err)
}
if got, _ := needsOperatorOpen(t, k); got != nil {
t.Fatal("raised on one statement")
}
if err := judgeModuleHealth(ctx, nil, k, "workstation", rs, map[string]int{"mounts": 2}, time.Now()); err != nil {
t.Fatal(err)
}
got, open := needsOperatorOpen(t, k)
if got == nil {
t.Fatalf("not raised on two statements: %+v", open)
}
for _, c := range open {
if c.Kind == kindModuleUnhealthy {
t.Fatalf("raised as a fault too: %+v", c)
}
}
if got.Kind != kindNeedsOperator || got.Resolver != conditions.ResolverOperator || got.Severity != conditions.Warning {
t.Fatalf("the condition: %+v", got)
}
if !strings.Contains(got.Needs, "type the password of the source games at workstation's desk prompt") ||
!strings.Contains(got.Explanation, "hidden prompt opens at the desk") {
t.Fatalf("its needs do not name the act: %q", got.Needs)
}
if strings.Contains(strings.ToLower(got.Explanation), "nothing for you") || !strings.Contains(got.Explanation, "nothing was undone") {
t.Fatalf("its explanation: %q", got.Explanation)
}
if !strings.Contains(got.Summary, "smb-password-games") || !strings.Contains(got.Summary, "nox secret ask workstation mounts smb-password-games") {
t.Fatalf("its summary does not name the secret and the line: %q", got.Summary)
}
// Long open is still a warning: only the operator can end it.
if err := judgeModuleHealth(ctx, nil, k, "workstation", rs, map[string]int{"mounts": 3}, time.Now().Add(48*time.Hour)); err != nil {
t.Fatal(err)
}
if got, _ := needsOperatorOpen(t, k); got == nil || got.Severity == conditions.Urgent {
t.Fatalf("after two days: %+v", got)
}
// Given: the next statement does not say it, and it clears.
if err := judgeModuleHealth(ctx, nil, k, "workstation", map[string][]inventory.ResourceHealth{}, nil, time.Now()); err != nil {
t.Fatal(err)
}
if got, _ := needsOperatorOpen(t, k); got != nil {
t.Fatal("not cleared once given")
}
}
func TestASettingsWaitAsksForTheApproval(t *testing.T) {
k, _ := withConditionsInMemory(t)
ctx := t.Context()
rs := map[string][]inventory.ResourceHealth{"mounts": {waitingResource(usernameWait)}}
for i := 1; i <= 2; i++ {
if err := judgeModuleHealth(ctx, nil, k, "workstation", rs, map[string]int{"mounts": i}, time.Now()); err != nil {
t.Fatal(err)
}
}
got, _ := needsOperatorOpen(t, k)
if got == nil || !strings.Contains(got.Needs, "approve smb-users of mounts on workstation when it is proposed to you") {
t.Fatalf("the condition: %+v", got)
}
}
// A wait that fails its check is the module's own fault: unhealthy, with why, and no needs-operator.
func TestAWaitThatFailsItsCheckRaisesUnhealthy(t *testing.T) {
k, _ := withConditionsInMemory(t)
ctx := t.Context()
checked := checkWaiting("workstation", []inventory.ResourceHealth{waitingResource(passwordWait)},
factsGiven(map[string]time.Time{"smb-password-games": time.Now()}))
rs := map[string][]inventory.ResourceHealth{"mounts": checked}
for i := 1; i <= 2; i++ {
if err := judgeModuleHealth(ctx, nil, k, "workstation", rs, map[string]int{"mounts": i}, time.Now()); err != nil {
t.Fatal(err)
}
}
got, open := needsOperatorOpen(t, k)
if got != nil {
t.Fatalf("a wait for a secret given raised needs-operator: %+v", got)
}
unhealthy := false
for _, c := range open {
unhealthy = unhealthy || c.Key == moduleUnhealthyKey("mounts", "workstation")
}
if !unhealthy {
t.Fatalf("not raised as unhealthy: %+v", open)
}
}
// A module that waited and is then broken says so when the wait clears, and the other way round.
func TestANeedsOperatorThatBecameUnhealthySaysSo(t *testing.T) {
k, _ := withConditionsInMemory(t)
ctx := t.Context()
waiting := map[string][]inventory.ResourceHealth{"mounts": {waitingResource(passwordWait)}}
for i := 1; i <= 2; i++ {
if err := judgeModuleHealth(ctx, nil, k, "workstation", waiting, map[string]int{"mounts": i}, time.Now()); err != nil {
t.Fatal(err)
}
}
broken := waitingResource()
broken.State, broken.Reason, broken.Waits = link.StateUnhealthy, "the source games refused its login", nil
for i := 3; i <= 4; i++ {
if err := judgeModuleHealth(ctx, nil, k, "workstation", map[string][]inventory.ResourceHealth{"mounts": {broken}},
map[string]int{"mounts": i}, time.Now()); err != nil {
t.Fatal(err)
}
}
got, open := needsOperatorOpen(t, k)
if got != nil {
t.Fatal("needs-operator still open after it became a fault")
}
found := false
for _, c := range open {
found = found || c.Key == moduleUnhealthyKey("mounts", "workstation")
}
if !found {
t.Fatalf("the fault is not raised: %+v", open)
}
}
cmd/mesh-controller/over_budget.go
-124
View File
@@ -1,124 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A delivery over its budget is loud (novox/hq ADR 0282 decision 7, issue 382): the self-check reads
// mesh-delivery's `times` and raises the warning `delivery.<class>.over-budget` when the newest delivery of a
// class whose delivery time is known took longer than its class's budget — five minutes for a leaf module, ten
// for a core module — naming the delivery and its longest phase. It clears when the next delivery of that class
// lands within its budget. Measurement only: the condition says, it never holds or acts on a delivery.
// probeBudgetsID is the probe that reads the delivery times.
const probeBudgetsID = "D16"
// kindOverBudget is what a delivery over its class's budget raises.
const kindOverBudget = "over-budget"
// deliveryBudgets are the budgets by class (ADR 0282 decision 1); the probe raises nothing for another class.
var deliveryBudgets = map[string]time.Duration{inventory.ClassLeaf: 5 * time.Minute, inventory.ClassCore: 10 * time.Minute}
// timesAnswer is what mesh-delivery's `times` answers, as far as the probe reads it.
type timesAnswer struct {
Classes []timesClass `json:"classes"`
}
// timesClass is one class's line of `times`.
type timesClass struct {
Class string `json:"class"`
Latest *timesLatest `json:"latest,omitempty"`
}
// timesLatest is the newest delivery of a class whose delivery time is known.
type timesLatest struct {
ID string `json:"id"`
TookMS int64 `json:"took_ms"`
Longest string `json:"longest,omitempty"`
Landed string `json:"landed,omitempty"`
}
// deliveryTimes is what the delivery's owner says of its delivery times; nothing when no holder is on record or
// none answers (D3 says that one).
func deliveryTimes(ctx context.Context, conn *nats.Conn, held bool) (*timesAnswer, error) {
if !held {
return nil, nil
}
raw, err := askDeliveryOwner(ctx, conn, "times", map[string]any{})
if errors.Is(err, link.ErrNothingServes) {
return nil, nil
}
if err != nil {
return nil, err
}
var a timesAnswer
if err := json.Unmarshal(raw, &a); err != nil {
return nil, fmt.Errorf("%s.times answered something unreadable: %w", catalogue.DeliverySeat, err)
}
return &a, nil
}
// overBudgetObservations are the conditions of the classes whose newest delivery took longer than its budget:
// strictly longer, so a delivery of exactly its budget is within it.
func overBudgetObservations(a *timesAnswer) []conditions.Observation {
if a == nil {
return nil
}
var out []conditions.Observation
for _, c := range a.Classes {
budget, ok := deliveryBudgets[c.Class]
if !ok || c.Latest == nil {
continue
}
took := time.Duration(c.Latest.TookMS) * time.Millisecond
if took <= budget {
continue
}
longest := c.Latest.Longest
if longest == "" {
longest = "not known"
}
out = append(out, conditions.Observation{Scope: conditions.ScopeDelivery, ID: c.Class, Kind: kindOverBudget,
Severity: conditions.Warning,
Summary: fmt.Sprintf("the %s delivery %s took %s from its merge to running everywhere, over its budget of %s; "+
"its longest phase: %s — `mesh-delivery.times`", c.Class, c.Latest.ID, humanDuration(took),
humanDuration(budget), longest),
Said: fmt.Sprintf("%s took %s (budget %s), longest phase %s", c.Latest.ID, took.Round(time.Second), budget, longest),
Headline: fmt.Sprintf("A %s delivery took %s, over its %s budget", c.Class, humanDuration(took),
humanDuration(budget)),
Explanation: fmt.Sprintf("The delivery %s took %s from its merge until every machine ran it; a %s module is "+
"held to %s (ADR 0282). Most of the time went to %s. Nothing was held or changed because of this: it is "+
"a measurement.", c.Latest.ID, humanDuration(took), c.Class, humanDuration(budget), longest),
Resolved: fmt.Sprintf("the next %s delivery lands within %s", c.Class, humanDuration(budget)),
})
}
return out
}
// probeBudgets is D16: the newest delivery of each class lands within its class's budget.
func probeBudgets(ctx context.Context, d *doctor) ([]conditions.Observation, error) {
entries, err := d.open.inventory.Catalogued(ctx)
if err != nil {
return nil, err
}
var conn *nats.Conn
if d.js != nil {
conn = d.js.Conn()
}
a, err := deliveryTimes(ctx, conn, deliverySeatHeld(entries))
if err != nil {
return nil, err
}
return overBudgetObservations(a), nil
}
cmd/mesh-controller/over_budget_test.go
-111
View File
@@ -1,111 +0,0 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// A leaf delivery of 5 minutes 10 seconds raises delivery.leaf.over-budget naming its longest phase; one of
// exactly five minutes, a core one of nine and a class without a budget raise nothing (ADR 0282 decision 7).
func TestADeliveryOverItsBudgetIsLoud(t *testing.T) {
a := &timesAnswer{Classes: []timesClass{
{Class: inventory.ClassLeaf, Latest: &timesLatest{ID: "novox/mesh-catalog@abc", TookMS: (5*time.Minute + 10*time.Second).Milliseconds(),
Longest: "judgement 2m40s"}},
{Class: inventory.ClassCore, Latest: &timesLatest{ID: "novox/mesh-controller@def", TookMS: (9 * time.Minute).Milliseconds(),
Longest: "build 1m"}},
{Class: "unclassed", Latest: &timesLatest{ID: "x@y", TookMS: time.Hour.Milliseconds()}},
}}
obs := overBudgetObservations(a)
if len(obs) != 1 {
t.Fatalf("one class over its budget, got %d: %+v", len(obs), obs)
}
o := obs[0]
if o.Key() != "delivery.leaf.over-budget" || !strings.Contains(o.Summary, "judgement 2m40s") ||
!strings.Contains(o.Summary, "novox/mesh-catalog@abc") {
t.Fatalf("the condition names its delivery and longest phase: %s — %s", o.Key(), o.Summary)
}
// The next leaf delivery within its budget clears it: the probe raises nothing for the class.
a.Classes[0].Latest = &timesLatest{ID: "novox/mesh-catalog@ghi", TookMS: (5 * time.Minute).Milliseconds()}
if obs := overBudgetObservations(a); len(obs) != 0 {
t.Fatalf("a delivery of exactly its budget is within it: %+v", obs)
}
a.Classes[1].Latest.TookMS = (10*time.Minute + time.Second).Milliseconds()
if obs := overBudgetObservations(a); len(obs) != 1 || obs[0].Key() != "delivery.core.over-budget" {
t.Fatalf("a core delivery over ten minutes: %+v", obs)
}
if obs := overBudgetObservations(nil); obs != nil {
t.Fatal("no answer raises nothing")
}
// No delivery of a class with a known time yet: nothing said of it.
if obs := overBudgetObservations(&timesAnswer{Classes: []timesClass{{Class: inventory.ClassLeaf}}}); len(obs) != 0 {
t.Fatalf("a class with no delivery: %+v", obs)
}
}
// The probe's question is one the controller's grant names: a question the bus refuses checks nothing.
func TestTheControllerMayAskForTheDeliveryTimes(t *testing.T) {
found := false
for _, v := range broker.VerbsTheControllerAsksTheDeliveryOwner {
found = found || (v.Seat == catalogue.DeliverySeat && v.Verb == "times")
}
if !found {
t.Fatal("the grant does not name mesh-delivery.times")
}
}
// A walk's class is core when it walks a module of the controller's own path or one holding the mesh's resolver,
// leaf otherwise; its window closed at its batch's window, or its maximum, never after its cut.
func TestAWalksClassAndWindowAreReadAtItsCut(t *testing.T) {
entries := []inventory.Entry{
{Manifest: catalogue.Manifest{Module: "dnsmasq", Claims: []catalogue.Claim{{Name: resolverSeat}}}},
{Manifest: catalogue.Manifest{Module: "gitea"}},
}
walk := func(modules ...string) inventory.Plan {
p := inventory.Plan{Modules: map[string]*inventory.PlanModule{}}
for _, m := range modules {
p.Modules[m] = &inventory.PlanModule{}
}
return p
}
for want, w := range map[string]inventory.Plan{
inventory.ClassLeaf: walk("gitea"), inventory.ClassCore: walk("gitea", "mesh-controller"),
} {
if got := classOf(w, entries); got != want {
t.Errorf("%v: %s, want %s", w.Modules, got, want)
}
}
if got := classOf(walk("dnsmasq"), entries); got != inventory.ClassCore {
t.Errorf("the resolver's holder is core: %s", got)
}
now := time.Date(2026, 10, 10, 18, 0, 0, 0, time.UTC)
batch := inventory.Plan{Delivery: &inventory.PlanDelivery{Batch: &inventory.PlanBatch{
ClosesAt: now.Add(-20 * time.Second), AtMost: now.Add(5 * time.Minute)}}}
times := walkTimesAtCut(batch, walk("gitea"), entries, now)
if times.Cut == nil || !times.Cut.Equal(now) || times.WindowClosed == nil || !times.WindowClosed.Equal(now.Add(-20*time.Second)) ||
times.Class != inventory.ClassLeaf {
t.Fatalf("times at the cut: %+v", times)
}
batch.Delivery.Batch.AtMost = now.Add(-time.Minute)
if times := walkTimesAtCut(batch, walk("gitea"), entries, now); !times.WindowClosed.Equal(now.Add(-time.Minute)) {
t.Fatalf("a window closed at its maximum: %v", times.WindowClosed)
}
if times := walkTimesAtCut(inventory.Plan{Delivery: &inventory.PlanDelivery{Alone: true}}, walk("gitea"), entries, now); times.WindowClosed != nil {
t.Fatalf("a merge walked alone had no window: %v", times.WindowClosed)
}
}
// What each machine of the rest was sent is kept only for the machines the send reached.
func TestTheRestIsKeptForTheMachinesTheSendReached(t *testing.T) {
got := restOf([]string{"ace", "g14"}, []string{"ace", "shanks"}, map[string]inventory.SentDeclaration{"ace": {Digest: "d1"}})
if len(got) != 1 || got["ace"].Digest != "d1" {
t.Fatalf("rest: %+v", got)
}
if restOf([]string{"g14"}, []string{"ace"}, nil) != nil {
t.Fatal("no machine reached: nothing kept")
}
}
cmd/mesh-controller/plain_words.go
-94
View File
@@ -221,16 +221,6 @@ var plainWordings = map[string]func(conditions.Observation) words{
w := usedAsFoundObservation(orModule(module), machineOr(o, "a machine"), o.Summary, nil)
return words{Headline: w.Headline, Explanation: w.Explanation, Needs: w.Needs, Resolved: w.Resolved}
}),
kindNeedsOperator: worded(func(o conditions.Observation) words {
// The observation carries the act itself (ADR 0283); these are its words when only the kind is known.
module := ""
if o.Scope == conditions.ScopeModule && o.Machine != "" {
module = strings.TrimSuffix(o.ID, "."+o.Machine)
}
node := machineOr(o, "a machine")
w := needsOperatorWords(orModule(module), node, nil)
return w
}),
kindProviderFailing: worded(func(o conditions.Observation) words {
thing, consumer := conditions.ThingWords(o), idPart(o, 2)
if consumer == "" {
@@ -606,19 +596,6 @@ var plainWordings = map[string]func(conditions.Observation) words{
Explanation: "The bus refused messages from a part of the mesh, so what they carried did not happen.",
Resolved: "Resolved: the bus takes the messages again"}
}),
kindBucketOrLogFilling: worded(func(o conditions.Observation) words {
return words{Headline: "A module's bucket or log on the bus is filling up",
Needs: "decide whether to raise its cap or have the module keep less.",
Explanation: "A bucket or log a module keeps on the bus holds three quarters of its cap or more. When it " +
"is full, the bus refuses what the module writes there.",
Resolved: "It has room again"}
}),
kindBucketOrLogUnread: worded(func(o conditions.Observation) words {
return words{Headline: "A module's bucket or log could not be read",
Explanation: "The controller could not read how full a bucket or log a module keeps on the bus is, so it " +
"cannot say whether it is filling up. It asks again at its next check.",
Resolved: "It can be read again"}
}),
"stream-wrong": worded(func(o conditions.Observation) words {
return words{Headline: "Part of the bus's storage is wrong",
Needs: "check the machine the bus runs on; the details say what is missing.",
@@ -841,9 +818,6 @@ func stalledWords(l stalledLine, o conditions.Observation) (headline, explanatio
"to check it: the forge never announced it. It cannot merge until it is checked.", pull, long),
fmt.Sprintf("%s has a merge check now, or is closed", pull), needs, nil
}
if l.State == "unanswered" {
return unansweredWords(l, o)
}
held := l.State
if held == "" {
held = "held"
@@ -873,74 +847,6 @@ func stalledWords(l stalledLine, o conditions.Observation) (headline, explanatio
fmt.Sprintf("Delivery of %s is no longer %s", name, held), needs, actions
}
// unansweredWords are the plain words of a pull request's head whose merge check was asked and has not answered
// within its bound (novox/hq issue 438): mesh-delivery holds no delivery of it, so there is nothing to stop, release
// or close, and no action is offered. The operator's acts are a new commit, which asks the check again, or a look
// at the build queue, where a check that never ran may still wait.
func unansweredWords(l stalledLine, o conditions.Observation) (headline, explanation, resolved, needs string,
actions []conditions.Action) {
repository, _, _ := strings.Cut(l.ID, "@")
pull := "A pull request of " + repoName(repository)
if l.Number > 0 {
pull = fmt.Sprintf("Pull request %s #%d", repoName(repository), l.Number)
}
long, limit := "for too long", ""
if d, err := time.ParseDuration(l.For); err == nil {
long = "for " + humanDuration(d)
}
if d, err := time.ParseDuration(l.Bound); err == nil {
limit = ", past its limit of " + humanDuration(d)
}
which := "Its merge check"
if said := uncheckedWaitWords(l); len(said) > 0 {
which = "Its merge check (" + strings.Join(said, ", ") + ")"
}
if o.Resolver == conditions.ResolverOperator {
needs = "push a new commit to its branch, which asks the check again, or see whether its check still waits " +
"in the build queue: mesh-controller.queue."
}
return fmt.Sprintf("%s's merge check has not answered %s", pull, long),
fmt.Sprintf("%s is open on a branch that requires the merge check. %s was asked and has not answered %s%s. "+
"It cannot merge until its check answers.", pull, which, long, limit),
fmt.Sprintf("%s has an answer from its merge check, or is closed", pull), needs, nil
}
// uncheckedWaitWords are the merge checks an unanswered line waits on, as the operator reads them: from the checks
// given as data, each time in the controller's local zone, as every other time in a message — "mesh/merge-gate
// pending since 02:11" — never the UTC time inside mesh-delivery's finished words, which cannot be said again in
// another zone (novox/hq issue 443). A line from a mesh-delivery that gives no such data is said by its words.
func uncheckedWaitWords(l stalledLine) []string {
if len(l.Checks) == 0 {
return l.Waiting
}
out := make([]string, 0, len(l.Checks))
for _, c := range l.Checks {
switch c.State {
case "never-set":
out = append(out, c.Context+" never set")
case "pending":
out = append(out, c.Context+" pending"+sinceWords(c.Since))
default:
out = append(out, c.Context+" not answered")
}
}
return out
}
// sinceWords is " since 02:11" in the controller's local zone, with its day when that is not today ("since 23:05 on
// 9 Oct"), so the time stays absolute; a time not given, or not readable, is said so and never made up.
func sinceWords(since string) string {
at, err := time.Parse(time.RFC3339, since)
if err != nil {
return ", since a time the forge did not say"
}
local, today := at.In(wordsZone()), wordsNow().In(wordsZone())
if local.YearDay() == today.YearDay() && local.Year() == today.Year() {
return " since " + local.Format("15:04")
}
return " since " + local.Format("15:04 on 2 Jan")
}
// causeWords is a hand-act's cause as a person says it.
func causeWords(cause string) string {
return strings.NewReplacer(".", " ", "_", " ").Replace(cause)
cmd/mesh-controller/plain_words_test.go
-105
View File
@@ -1,7 +1,6 @@
package main
import (
"encoding/json"
"regexp"
"strings"
"testing"
@@ -348,107 +347,3 @@ func TestAnUnannouncedPullRequestSaysToPushANewCommit(t *testing.T) {
t.Fatalf("it reads %q / %q", o.Headline, o.Explanation)
}
}
// **A pull request whose merge check never answered says which check, since when, and what to do** (novox/hq issue
// 438): mesh-delivery says one as a stalled line in state `unanswered`, the line below exactly as its test makes it.
// No delivery of the head is held, so there is nothing to stop, release or close: no action is offered, and the
// operator's acts are a new commit, which asks the check again, or a look at the build queue.
func TestAnUnansweredMergeCheckSaysWhichCheckAndWhatToDo(t *testing.T) {
var l stalledLine
if err := json.Unmarshal([]byte(`{"id":"novox/mesh-controller@c11222026a3b","number":212,"state":"unanswered",`+
`"waiting":["mesh/merge-gate pending since 2026-10-11T00:11:39Z","mesh/repo-check never set"],"for":"1h1m0s",`+
`"bound":"1h0m0s","h2":"none: no delivery of it is held here, so there is none to close — the operator's",`+
`"says":"novox/mesh-controller#212 is open on main, which requires the merge check, and its head's check `+
`started and never answered: mesh/merge-gate pending since 2026-10-11T00:11:39Z, mesh/repo-check never set. `+
`No delivery of it is held here, so nothing asks it again. A new commit on its branch announces it and asks `+
`its check"}`), &l); err != nil {
t.Fatal(err)
}
obs := stalledObservations([]stalledLine{l})
if len(obs) != 1 || obs[0].Resolver != conditions.ResolverOperator {
t.Fatalf("an unanswered merge check is not the operator's: %+v", obs)
}
o := obs[0]
t.Logf("headline: %s\nexplanation: %s\nneeds: %s\nresolved: %s", o.Headline, o.Explanation, o.Needs, o.Resolved)
if len(o.Actions) != 0 {
t.Fatalf("it offers an action a head with no delivery cannot take: %+v", o.Actions)
}
if strings.Contains(o.Needs, "stop") || strings.Contains(o.Needs, "release") || !strings.Contains(o.Needs, "commit") ||
!strings.Contains(o.Needs, "in the build queue: mesh-controller.queue") || strings.Contains(o.Needs, "build seat") {
t.Fatalf("it says to %q", o.Needs)
}
if o.Headline != "Pull request mesh-controller #212's merge check has not answered for 61 minutes" {
t.Fatalf("its headline reads %q", o.Headline)
}
for _, want := range []string{"mesh/merge-gate pending since 2026-10-11T00:11:39Z", "mesh/repo-check never set",
"past its limit of 60 minutes"} {
if !strings.Contains(o.Explanation, want) {
t.Fatalf("its explanation does not say %q: %q", want, o.Explanation)
}
}
if strings.Contains(o.Explanation, "never asked") || strings.Contains(o.Explanation, "never announced") {
t.Fatalf("it says the mesh was never asked, of a head whose check started: %q", o.Explanation)
}
// A line from a mesh-delivery that names no checks still says the check did not answer, and offers nothing.
var bare stalledLine
if err := json.Unmarshal([]byte(`{"id":"novox/mesh-controller@c11222026a3b","number":212,"state":"unanswered",`+
`"for":"1h1m0s","bound":"1h0m0s","h2":"none: no delivery of it is held here, so there is none to close — the operator's"}`),
&bare); err != nil {
t.Fatal(err)
}
o = stalledObservations([]stalledLine{bare})[0]
if len(o.Actions) != 0 || !strings.Contains(o.Explanation, "has not answered") || strings.Contains(o.Headline, "Delivery of") {
t.Fatalf("a line naming no checks reads %q / %q, actions %+v", o.Headline, o.Explanation, o.Actions)
}
}
// **An unanswered merge check's time reaches the operator in their own time, never as raw UTC** (novox/hq issue
// 443): mesh-delivery says each check waited on as data — its context, its state and since when, in RFC 3339 — beside
// the finished words it gave before, and the controller words it in its local zone, as every other time in a message.
// The line is the one mesh-delivery says, as in the report of issue 443, with the checks it now carries.
func TestAnUnansweredMergeChecksTimeIsSaidInLocalTime(t *testing.T) {
// The operator's zone given through the seam, never by writing time.Local: other tests' goroutines read it, and
// the build seat runs the suite under the race detector.
wasZone, wasNow := wordsZone, wordsNow
wordsZone = func() *time.Location { return time.FixedZone("CEST", 2*60*60) }
wordsNow = func() time.Time { return time.Date(2026, 10, 11, 1, 12, 39, 0, time.UTC) }
t.Cleanup(func() { wordsZone, wordsNow = wasZone, wasNow })
var l stalledLine
if err := json.Unmarshal([]byte(`{"id":"novox/mesh-controller@c11222026a3b","number":212,"state":"unanswered",`+
`"waiting":["mesh/merge-gate pending since 2026-10-11T00:11:39Z","mesh/repo-check never set"],`+
`"checks":[{"context":"mesh/merge-gate","state":"pending","since":"2026-10-11T00:11:39Z"},`+
`{"context":"mesh/repo-check","state":"never-set"}],"for":"1h1m0s",`+
`"bound":"1h0m0s","h2":"none: no delivery of it is held here, so there is none to close — the operator's",`+
`"says":"novox/mesh-controller#212 is open on main, which requires the merge check, and its head's check `+
`started and never answered: mesh/merge-gate pending since 2026-10-11T00:11:39Z, mesh/repo-check never set. `+
`No delivery of it is held here, so nothing asks it again. A new commit on its branch announces it and asks `+
`its check"}`), &l); err != nil {
t.Fatal(err)
}
o := stalledObservations([]stalledLine{l})[0]
t.Logf("explanation: %s", o.Explanation)
for _, raw := range []string{"2026-10-11T00:11:39Z", "00:11", "UTC"} {
if strings.Contains(o.Explanation, raw) || strings.Contains(o.Headline, raw) {
t.Fatalf("the operator reads %q: %q / %q", raw, o.Headline, o.Explanation)
}
}
for _, want := range []string{"mesh/merge-gate pending since 02:11", "mesh/repo-check never set"} {
if !strings.Contains(o.Explanation, want) {
t.Fatalf("its explanation does not say %q: %q", want, o.Explanation)
}
}
// A check pending since another day than today says which day, so the time stays absolute.
l.Checks[0].Since = "2026-10-09T21:05:00Z"
if o = stalledObservations([]stalledLine{l})[0]; !strings.Contains(o.Explanation, "mesh/merge-gate pending since 23:05 on 9 Oct") {
t.Fatalf("a check pending since an earlier day reads %q", o.Explanation)
}
// A pending check whose time the forge did not say is said so, with no time made up.
l.Checks[0].Since = ""
if o = stalledObservations([]stalledLine{l})[0]; !strings.Contains(o.Explanation, "mesh/merge-gate pending, since a time the forge did not say") {
t.Fatalf("a pending check without its time reads %q", o.Explanation)
}
}
cmd/mesh-controller/plan.go
+2 -34
View File
@@ -420,8 +420,7 @@ func declarationWith(ctx context.Context, open *stores, node string,
out := sendable{Resources: composed.Resources, Adoption: adoption,
Received: composed.Received, Mesh: with.Mesh, BusUsers: with.BusUsers,
LeftOut: sortedKeysOf(composed.LeftOut), leftOutWhy: composed.LeftOut, withheld: with.Withheld,
unbound: with.Unbound, foreseen: composed.Foreseen, unplaced: composed.Unplaced,
modules: namedModules(plan, composed.LeftOut)}
unbound: with.Unbound, foreseen: composed.Foreseen, unplaced: composed.Unplaced}
// And which build of each module it carries, for the send to record (novox/hq issue 259, ADR
// 0221). Read only on the send path: a question about what would be sent records nothing.
if choosing == Allocating {
@@ -665,33 +664,7 @@ func renderingFor(ctx context.Context, open *stores, node string,
needed := map[string]map[string]string{}
foreseen := map[string]map[string]bool{}
for _, m := range plan.Modules {
// **A secret family is never made** (novox/hq ADR 0283): each member a person gave on this machine is
// placed, and one not given is nothing — the module says it waits for it.
for _, family := range m.OwnSecrets.Families() {
members, err := inv.GivenMembers(ctx, node, m.Module, family)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
for _, g := range members {
if _, fam, ok := m.OwnSecrets.Lookup(g.Name); !ok || fam != family {
continue // a longer family's member, or a name no longer of this family
}
if !g.Current {
if choosing == Allocating {
return catalogue.Rendering{}, inventory.Node{}, fmt.Errorf(
"%s on %s holds %q, which was given to the mesh rather than made by it, and %s has "+
"since generated a new sealing key. The mesh cannot make another; give it again",
m.Module, node, g.Name, node)
}
continue
}
if needed[m.Module] == nil {
needed[m.Module] = map[string]string{}
}
needed[m.Module][g.Name] = g.Sealed
}
}
for name := range m.OwnSecrets.Plain() {
for name := range m.OwnSecrets {
// Minted on the send path and only read on every other. Making one is an insert, and
// a question that writes is a question that can block against the machine it is about.
var sealed string
@@ -1291,11 +1264,6 @@ func planCommand(ctx context.Context, args []string) error {
return nil
}
// What the machine was last told, by whom and from which assignment generation (novox/hq issue 234),
// beside what it would be told now. A record that cannot be read is said, not taken for none.
if err := writeLastSend(ctx, os.Stdout, open.inventory, args[0]); err != nil {
fmt.Printf("what %s was last sent could not be read: %v\n", args[0], err)
}
// Which modules a push would leave out, and why — said before the plan, since the plan is of
// what the machine would be told (novox/hq ADR 0163, rule 6). Judged, never composed: `plan`
// without --json allocates nothing.
cmd/mesh-controller/plan_retry.go
+3 -26
View File
@@ -288,24 +288,13 @@ func retryPlan(ctx context.Context, open *stores, id string) (string, error) {
"those walks answer them; a newer merge, or `rebuild <module>`, builds again", p.ID)
}
}
// Settled from the build records before anything is judged (novox/hq issue 457): a build of the tier
// that failed after the plan did is as failed as the one that failed it. What toRetry says is the
// failed set every step below works from.
var failed []string
if p.Tier < len(p.Tiers) {
recorded, byID, err := recordsOfAsked(ctx, inv, &p, p.Tiers[p.Tier])
if err != nil {
return "", err
}
failed = toRetry(&p, recorded, byID)
}
if err := retryRefusal(p, plans); err != nil {
return "", err
}
if again := unjudgedAtGate(p); len(failed) == 0 && len(again) > 0 {
if again := unjudgedAtGate(p); len(failedIn(p)) == 0 && len(again) > 0 {
return retryTierWhole(ctx, open, &p, again)
}
if len(failed) == 0 {
if len(failedIn(p)) == 0 {
return retryRollouts(ctx, open, &p)
}
entries, err := inv.Catalogued(ctx)
@@ -316,6 +305,7 @@ func retryPlan(ctx context.Context, open *stores, id string) (string, error) {
for _, e := range entries {
byName[e.Manifest.Module] = e
}
failed := failedIn(p)
var asked []string
for _, m := range failed {
askModule(ctx, &p, m, byName)
@@ -535,16 +525,3 @@ func retryTierWhole(ctx context.Context, open *stores, p *inventory.Plan, again
func sendAgain(s *inventory.PlanModule) {
s.First, s.FirstAt, s.Gate, s.GatedBy, s.Previous, s.Why = nil, nil, nil, "", "", ""
}
// toRetry is the modules a retry asks again: every one of the tier that failed, the plan's state
// settled from the build records first (novox/hq issue 457). A plan fails on the first failure in its
// tier, and an outcome arriving after that finds no open plan to answer — it is kept only in the build
// records. Read from the plan alone, a retry asked only the build that failed first, and the records
// then failed the plan again on the next: each failed build of a tier took a retry of its own. What
// still runs is left asked, and its outcome is the plan's once the retry sets it building.
func toRetry(p *inventory.Plan, recorded map[string][]inventory.Build, byID map[string]inventory.Build) []string {
if p.Tier < len(p.Tiers) {
settleFromRecords(p, p.Tiers[p.Tier], recorded, byID)
}
return failedIn(*p)
}
cmd/mesh-controller/plan_retry_tier_test.go
-57
View File
@@ -1,57 +0,0 @@
package main
import (
"reflect"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// A retry asks every failed build of the tier, not one (novox/hq issue 457). Seen 2026-10-11: gitea
// and plex both failed in tier 0 while the registry was held still; gitea's failure failed the plan,
// and plex's, arriving after, found no open plan and was kept only in the build records. The first
// retry asked gitea alone, the records then failed the plan again on plex, and a second retry asked
// plex.
func TestARetryAsksEveryFailedBuildOfTheTier(t *testing.T) {
asked := time.Date(2026, 10, 11, 1, 29, 0, 0, time.UTC)
failedAt := asked.Add(2 * time.Minute)
p := inventory.Plan{ID: "plan-457", State: inventory.PlanFailed, Tier: 0,
Tiers: [][]string{{"gitea", "plex"}}, Note: "gitea failed to build in tier 0",
Modules: map[string]*inventory.PlanModule{
"gitea": {State: "failed", AskedAt: &asked, Build: "build-gitea", Why: "cannot reach the registry"},
"plex": {State: "asked", AskedAt: &asked, Build: "build-plex"},
}}
byID := map[string]inventory.Build{
"build-plex": {ID: "build-plex", Module: "plex", At: failedAt, Failed: "cannot reach the registry"},
}
if got := toRetry(&p, nil, byID); !reflect.DeepEqual(got, []string{"gitea", "plex"}) {
t.Fatalf("a retry of a tier where gitea and plex failed asks %v", got)
}
}
// A build of the tier still running when the plan failed is not asked again: its outcome is the plan's
// once the retry sets it building, and one that is recorded built is taken as built.
func TestARetryLeavesABuildThatRunsOrWorked(t *testing.T) {
asked := time.Date(2026, 10, 11, 1, 29, 0, 0, time.UTC)
builtAt := asked.Add(3 * time.Minute)
p := inventory.Plan{ID: "plan-457", State: inventory.PlanFailed, Tier: 0,
Tiers: [][]string{{"a", "b", "c"}},
Modules: map[string]*inventory.PlanModule{
"a": {State: "failed", AskedAt: &asked, Build: "build-a", Why: "broken"},
"b": {State: "asked", AskedAt: &asked, Build: "build-b"},
"c": {State: "asked", AskedAt: &asked, Build: "build-c"},
}}
byID := map[string]inventory.Build{"build-c": {ID: "build-c", Module: "c", At: builtAt, Commit: "c0ffee"}}
if got := toRetry(&p, nil, byID); !reflect.DeepEqual(got, []string{"a"}) {
t.Fatalf("asks %v, want a alone", got)
}
if s := p.Modules["c"]; s.State != "built" || s.Commit != "c0ffee" {
t.Errorf("c, recorded built, is %+v", s)
}
if s := p.Modules["b"]; s.State != "asked" {
t.Errorf("b, still building, is %+v", s)
}
}
cmd/mesh-controller/provider_hold.go
+21 -151
View File
@@ -37,32 +37,6 @@ type holding struct {
shelf map[string]catalogue.Manifest
// providers memoises providerFor by machine, consumer and provision.
providers map[string]providerLookup
// waits is what the waits of a statement are checked against, read as they are asked for (novox/hq issue 450).
waits operatorWaitFacts
}
// checked is a machine's newest statement as the controller judges it: each wait checked (ADR 0283 decision 3), so
// a wait that does not check out reads as unhealthy, as it did when the statement was judged (novox/hq issue 450).
// What is stored is what the machine said, the waits unchecked; read as stored, a provider whose wait failed its
// check seems to wait for the operator while its unhealthy condition is open.
func (h *holding) checked(machine string) []inventory.ResourceHealth {
rs := h.healths[machine].Resources
mods := waitingModules(rs)
if len(mods) == 0 {
return rs
}
if h.waits.manifests == nil {
h.waits.manifests = map[string]catalogue.Manifest{}
}
for _, module := range mods {
if _, has := h.waits.manifests[module]; !has {
if m, ok := h.manifestOf(module); ok {
h.waits.manifests[module] = m
}
}
}
readWaitFacts(h.ctx, h.inv, machine, mods, nil, &h.waits)
return checkWaiting(machine, rs, h.waits)
}
type providerLookup struct {
@@ -79,15 +53,6 @@ func readHolding(ctx context.Context, inv *inventory.Inventory, open []condition
return &holding{ctx: ctx, inv: inv, healths: healths, open: open, providers: map[string]providerLookup{}}, nil
}
// readHoldingFor is how a judging reads its holding: nothing without a store. A variable so a test can hand a
// judging the record it holds under (novox/hq issue 405).
var readHoldingFor = func(ctx context.Context, inv *inventory.Inventory, open []conditions.Condition) (*holding, error) {
if inv == nil {
return nil, nil
}
return readHolding(ctx, inv, open)
}
// heldFinding says a resource's state is a finding of its declared check that names a provision: what
// may be held. Down and restarting are liveness, the resource's own.
func heldFinding(r inventory.ResourceHealth) bool {
@@ -141,137 +106,42 @@ func (h *holding) lookUpProvider(machine, consumer, provision string) (catalogue
return catalogue.Chosen{}, false
}
// providerState is how a provider stands on the record: unhealthy when its unhealthy condition is open or its
// machine's newest statement says a resource of it is unhealthy; else waiting for the operator when that statement
// says a resource of it waits, with the waits it names, or its needs-operator condition is open (waits then
// unknown); else healthy.
func (h *holding) providerState(p catalogue.Chosen) (unhealthy, waiting bool, waits []inventory.Wait) {
// unhealthy says a provider is unhealthy on the record: its condition is open, or its machine's newest
// statement says a resource of it is unhealthy.
func (h *holding) unhealthy(p catalogue.Chosen) bool {
key := moduleUnhealthyKey(p.Module, p.Node)
for _, c := range h.open {
if c.Key == moduleUnhealthyKey(p.Module, p.Node) {
return true, false, nil
if c.Key == key {
return true
}
}
for _, r := range h.checked(p.Node) {
if r.Module != p.Module {
continue
}
switch r.State {
case link.StateUnhealthy:
return true, false, nil
case link.StateWaiting:
waiting = true
waits = append(waits, r.Waits...)
for _, r := range h.healths[p.Node].Resources {
if r.Module == p.Module && r.State == link.StateUnhealthy {
return true
}
}
if !waiting {
for _, c := range h.open {
waiting = waiting || c.Key == needsOperatorKey(p.Module, p.Node)
}
}
return false, waiting, waits
}
// manifestOf is a module's manifest from the catalogue, read once; false when it cannot be read.
func (h *holding) manifestOf(module string) (catalogue.Manifest, bool) {
if h.shelf == nil && h.inv != nil {
shelf, err := h.inv.Catalogue(h.ctx)
if err != nil {
return catalogue.Manifest{}, false
}
h.shelf = shelf
}
m, ok := h.shelf[module]
return m, ok
}
// covering is the waits of a provider that cover a provision it gives (novox/hq issue 405): a module that waits
// says nothing else of it is wrong and names each part that waits (ADR 0283 decision 1), so only a consumer of
// the waiting part waits on it. A wait covers a provision when its part is that provision, or the secret it waits
// for is the provision's shared credential (ADR 0158). Nothing when no wait can be matched: a consumer failing
// then is not held, since holding it would hide a fault that may be its own.
func (h *holding) covering(p catalogue.Chosen, provision string, waits []inventory.Wait) []inventory.Wait {
credential := ""
if m, ok := h.manifestOf(p.Module); ok {
credential, _ = m.SharedCredentialOf(provision)
}
var out []inventory.Wait
for _, w := range waits {
if w.Part == provision || (w.Secret != "" && w.Secret == credential) {
out = append(out, w)
}
}
return out
}
// heldUnderProvider is the provider a consumer's findings are held under, and — when that provider only waits for the
// operator, for the part the consumer needs — the waits that hold it.
type heldUnderProvider struct {
provider catalogue.Chosen
// waits is set when every finding held waits on a provider that only waits for the operator: the consumer's
// gate then reads as ADR 0254's waits for a person, a pass with the wait carried (ADR 0283 decision 4).
waits []inventory.Wait
return false
}
// heldUnder is the provider a consumer's unhealthy resources wait on: when every one of them is a finding
// of a check naming a provision whose provider for this consumer is unhealthy on the record, or waits for the
// operator for the part that gives it (novox/hq issue 405). False when any is the consumer's own.
// of a check naming a provision whose provider for this consumer is unhealthy on the record. False when any
// is the consumer's own.
func (h *holding) heldUnder(machine, module string, rs []inventory.ResourceHealth) (catalogue.Chosen, bool) {
by, held := h.heldWith(machine, module, rs)
return by.provider, held
}
func (h *holding) heldWith(machine, module string, rs []inventory.ResourceHealth) (heldUnderProvider, bool) {
var on heldUnderProvider
onlyWaits := true
var on catalogue.Chosen
for _, r := range rs {
if r.State != link.StateUnhealthy {
continue
}
if !heldFinding(r) {
return heldUnderProvider{}, false
return catalogue.Chosen{}, false
}
p, ok := h.providerFor(machine, module, r.Needs)
if !ok || (p.Node == machine && p.Module == module) {
return heldUnderProvider{}, false
if !ok || (p.Node == machine && p.Module == module) || !h.unhealthy(p) {
return catalogue.Chosen{}, false
}
unhealthy, waiting, waits := h.providerState(p)
switch {
case unhealthy:
onlyWaits = false
case waiting:
covered := h.covering(p, r.Needs, waits)
if len(covered) == 0 {
return heldUnderProvider{}, false
}
on.waits = append(on.waits, covered...)
default:
return heldUnderProvider{}, false
}
on.provider = p
on = p
}
if !onlyWaits {
on.waits = nil
}
return on, on.provider.Module != ""
}
// waitingUncovered is a provider of a consumer's failing finding that waits for the operator for a part the
// finding cannot be matched to (novox/hq issue 405): the consumer is raised on its own, and its condition says
// the provider waits, so neither is hidden.
func (h *holding) waitingUncovered(machine, module string, rs []inventory.ResourceHealth) (catalogue.Chosen, bool) {
for _, r := range rs {
if r.State != link.StateUnhealthy || !heldFinding(r) {
continue
}
p, ok := h.providerFor(machine, module, r.Needs)
if !ok || (p.Node == machine && p.Module == module) {
continue
}
if unhealthy, waiting, waits := h.providerState(p); !unhealthy && waiting && len(h.covering(p, r.Needs, waits)) == 0 {
return p, true
}
}
return catalogue.Chosen{}, false
return on, on.Module != ""
}
// waitersOn is every consumer held under a provider, as "<module> on <machine>", sorted.
@@ -295,8 +165,8 @@ func (h *holding) waitersOn(p catalogue.Chosen) []string {
}
// heldModules is, for one machine's statement, each module whose finding is held, with the provider.
func (h *holding) heldModules(machine string) map[string]heldUnderProvider {
out := map[string]heldUnderProvider{}
func (h *holding) heldModules(machine string) map[string]catalogue.Chosen {
out := map[string]catalogue.Chosen{}
byModule := map[string][]inventory.ResourceHealth{}
for _, r := range h.healths[machine].Resources {
if r.Module != "" && r.State == link.StateUnhealthy {
@@ -304,7 +174,7 @@ func (h *holding) heldModules(machine string) map[string]heldUnderProvider {
}
}
for module, rs := range byModule {
if on, held := h.heldWith(machine, module, rs); held {
if on, held := h.heldUnder(machine, module, rs); held {
out[module] = on
}
}
cmd/mesh-controller/push.go
+9 -79
View File
@@ -356,16 +356,10 @@ func declare(ctx context.Context, args []string) error {
// the mesh did not compose it, so a push that does not name this machine treats it as held.
// The epoch it carried, if a person wrote one in, is what the machine heard.
var carried struct {
Epoch uint64 `json:"epoch"`
Sequence int64 `json:"sequence"`
Generation int64 `json:"generation"`
Epoch uint64 `json:"epoch"`
}
_ = json.Unmarshal(raw, &carried)
// And recorded as every send is (novox/hq issue 234): by hand, from what it carried; the modules it
// named are not known, since the mesh did not compose it.
if _, err := recordSent(ctx, inv, node, raw, nil, carried.Epoch, sentRecord{sequence: carried.Sequence,
epoch: carried.Epoch, generation: carried.Generation, toldGeneration: carried.Generation,
sender: senderOf(callerOf(ctx)) + ", a declaration sent by hand"}); err != nil {
if _, err := recordSent(ctx, inv, node, raw, nil, carried.Epoch); err != nil {
return err
}
fmt.Printf("sent %s a signed declaration (%d bytes)\n", node, len(raw))
@@ -804,7 +798,7 @@ func composeEach(names []string, allot func(node string) (order, error),
refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err))
continue
}
numbered.stamp(&declared)
declared.Sequence, declared.Epoch = numbered.sequence, numbered.epoch
if len(declared.Resources) == 0 {
// Sent, not skipped (novox/hq issue 127). A node whose declaration composes to
// nothing may have HELD something before — the broker opening a placement gave it,
@@ -1022,8 +1016,7 @@ func (b overTheBus) declare(ctx context.Context, s readyNode, body []byte) (stri
}
// After it is away, not before. A digest recorded for something that failed to send would make
// the machine look current for a declaration it never received.
digest, err := recordSent(ctx, b.open.inventory, s.node, body, s.declared.Builds, s.declared.Epoch,
sentRecordOf(ctx, s.declared))
digest, err := recordSent(ctx, b.open.inventory, s.node, body, s.declared.Builds, s.declared.Epoch)
if err != nil {
return "", err
}
@@ -1235,7 +1228,7 @@ func sendToEach(ctx context.Context, open *stores, names []string) ([]string, er
refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err))
continue
}
numbered.stamp(&declared)
declared.Sequence, declared.Epoch = numbered.sequence, numbered.epoch
reportLeftOut(name, declared)
sending = append(sending, readyNode{name, declared})
}
@@ -1300,15 +1293,6 @@ func issueMemberships(ctx context.Context, open *stores, server *link.Server, se
if _, err := broker.RaiseBuckets(broker.OnConn(bus.Conn), buckets); err != nil {
return fmt.Errorf("the modules' state could not be asserted on the bus: %w", err)
}
// **And every declared log, for the same reason** (novox/hq ADR 0297 §2): a membership names its
// logs, and a module whose log does not exist fails its first append.
logs, err := open.inventory.DeclaredLogs(ctx)
if err != nil {
return fmt.Errorf("the modules' logs could not be read, so no log was asserted: %w", err)
}
if _, err := broker.RaiseLogs(broker.OnConn(bus.Conn), logs); err != nil {
return fmt.Errorf("the modules' logs could not be asserted on the bus: %w", err)
}
// Every membership is tried, and the first failure named once.
issued := 0
refused := map[string]error{}
@@ -1409,11 +1393,6 @@ func wouldSendFrom(ctx context.Context, open *stores,
if declared.Epoch, err = open.inventory.SentEpoch(ctx, n.ID); err != nil {
return nil, err
}
// And the generation it was last sent, for the same reason (novox/hq issue 234): an assignment
// elsewhere in the mesh is not a change of this machine.
if declared.Generation, err = open.inventory.SentGeneration(ctx, n.ID); err != nil {
return nil, err
}
body, err := declared.Body()
if err != nil {
return nil, err
@@ -1492,28 +1471,13 @@ func raiseTheBus(ctx context.Context, inv *inventory.Inventory, address string)
fmt.Printf("the bus holds state nothing declares any more, kept because it is data: %s — "+
"removing it is a person's act\n", strings.Join(undeclared, ", "))
}
// Every module's log (novox/hq ADR 0297), from the catalogue, as its buckets: one that nothing
// declares any more is said and kept — a log is a module's record, and no path of the mesh removes it.
logs, err := inv.DeclaredLogs(ctx)
if err != nil {
return err
}
unlogged, err := broker.RaiseLogs(js, logs)
if err != nil {
return err
}
if len(unlogged) > 0 {
fmt.Printf("the bus holds logs nothing declares any more, kept because they are data: %s — "+
"removing one is a person's act\n", strings.Join(unlogged, ", "))
}
// And how every module hears what it consumes: asserted with the rest above, counted here.
hearing, err := moduleConsumerCount(ctx, inv)
if err != nil {
return err
}
fmt.Printf("the bus at %s has its streams, %d machine(s) can hear a declaration, %d module(s) "+
"can hear what they consume, %d bucket(s) of state and %d log(s)\n", broker.BareAddress(address), len(names), hearing,
len(buckets), len(logs))
"can hear what they consume, and %d bucket(s) of state\n", broker.BareAddress(address), len(names), hearing, len(buckets))
return nil
}
@@ -1567,12 +1531,6 @@ var epochForActs = func(ctx context.Context) (uint64, error) { return theLease.e
type order struct {
sequence int64
epoch uint64
// generation is the assignment generation read before the composition, and readsGeneration whether the
// machine's node-engine said it reads one (novox/hq issue 234); acting is the lease epoch the sender acts
// under, whether or not the machine is sent it — both kept for the record of the send.
generation int64
readsGeneration bool
acting uint64
}
// allot takes the next sequence for a machine — the number its next declaration carries — under the
@@ -1586,7 +1544,6 @@ func allot(ctx context.Context, inv *inventory.Inventory, node string) (order, e
if err != nil {
return order{}, err
}
acting := epoch
if epoch > 0 {
reads, err := inv.ReadsEpoch(ctx, record.ID)
if err != nil {
@@ -1596,24 +1553,11 @@ func allot(ctx context.Context, inv *inventory.Inventory, node string) (order, e
epoch = 0
}
}
// **The generation is read here, before the composition reads a single assignment** (novox/hq issue
// 234). A generation only grows, so one read before is never newer than the view composed after it:
// the declaration may claim a generation older than its content, never newer — and a claim newer than
// the content is exactly the stale send the machine must be able to refuse.
generation, err := inv.AssignmentGeneration(ctx)
if err != nil {
return order{}, err
}
readsGeneration, err := inv.ReadsGeneration(ctx, record.ID)
if err != nil {
return order{}, err
}
seq, err := inv.NextSequence(ctx, record.ID)
if err != nil {
return order{}, err
}
return order{sequence: seq, epoch: epoch, generation: generation, readsGeneration: readsGeneration,
acting: acting}, nil
return order{sequence: seq, epoch: epoch}, nil
}
// recordSent writes down what a machine was just sent, and returns the digest.
@@ -1628,7 +1572,7 @@ func allot(ctx context.Context, inv *inventory.Inventory, node string) (order, e
// And the build of each module it carried (novox/hq issue 259, ADR 0221), nil when that is not known:
// what tells a machine held back by a policy or a plan from one a push left behind.
func recordSent(ctx context.Context, inv *inventory.Inventory, node string, body []byte,
builds map[string]string, epoch uint64, sent sentRecord) (string, error) {
builds map[string]string, epoch uint64) (string, error) {
kept, cancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
defer cancel()
record, err := inv.NodeByName(kept, node)
@@ -1636,21 +1580,7 @@ func recordSent(ctx context.Context, inv *inventory.Inventory, node string, body
return "", err
}
digest := digestOf(body)
// The digest and the generation it carried are written in one transaction (novox/hq issue 234), so the
// would-send is never stamped with a generation the machine was not sent; and the send itself, who sent
// it and from which generation, beside them. A record of the send that cannot be written does not make a
// send that is away look failed: it is said, loudly, and the machine's own record stands.
err = inv.RecordSentWith(kept, record.ID, digest, builds, epoch, sent.toldGeneration, inventory.Send{
Sequence: sent.sequence, Epoch: int64(sent.epoch), Sender: sent.sender, Generation: sent.generation,
Digest: digest, Modules: sent.modules})
var unrecorded *inventory.SendNotRecordedError
switch {
case errors.As(err, &unrecorded):
fmt.Fprintf(os.Stderr, "mesh-controller: SEND NOT RECORDED: %s was sent declaration %s (sequence %d), and who "+
"sent it and from which generation could not be written down, so a refusal of it will name no sender "+
"(novox/hq issue 234): %v\n", node, short(digest), sent.sequence, unrecorded.Err)
fmt.Printf("%s: sent, and the record of who sent it could NOT be written: %v\n", node, unrecorded.Err)
case err != nil:
if err := inv.RecordSentUnder(kept, record.ID, digest, builds, epoch); err != nil {
return "", err
}
// And what it was, summarised, so the next push can be compared with it (novox/hq ADR 0217).
cmd/mesh-controller/release_plan.go
+20 -59
View File
@@ -510,14 +510,6 @@ func advancePlans(ctx context.Context, open *stores) {
advanceHeld(ctx, open)
}
// keepWalkPhases keeps where the walks that ended lately spent their time (novox/hq ADR 0282), outside the hold
// on the plans so it never lengthens it: measured, never acted on, and an error only said.
func keepWalkPhases(ctx context.Context, open *stores) {
if err := recordWalkPhases(ctx, open.inventory, time.Now()); err != nil {
fmt.Printf("plans: the phases of the walks ended lately could not be kept: %v\n", err)
}
}
// advanceHeld is advancePlans for a caller already holding the plans.
func advanceHeld(ctx context.Context, open *stores) {
inv := open.inventory
@@ -649,9 +641,26 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
// the controller in its first tier: the build that produced the new one is recorded, and the
// plan never hears it. The record is the fact; a build recorded after the ask is that tier's
// outcome, whoever was listening.
recorded, byID, err := recordsOfAsked(ctx, inv, p, tier)
if err != nil {
return false, err
recorded := map[string][]inventory.Build{}
byID := map[string]inventory.Build{}
for _, m := range tier {
if s := p.Modules[m]; s != nil && s.State == "asked" {
builds, err := inv.Builds(ctx, m, 5)
if err != nil {
return false, err
}
recorded[m] = builds
// Its own ask's record, by id — found even when the outcome named no module (ADR 0219).
if s.Build != "" {
b, found, err := inv.BuildByID(ctx, s.Build)
if err != nil {
return false, err
}
if found {
byID[s.Build] = b
}
}
}
}
if settleFromRecords(p, tier, recorded, byID) {
return true, nil
@@ -855,12 +864,8 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
strings.Join(machines, ", "), p.Tier, err)
}
now := time.Now().UTC()
// What each machine of the rest was sent, kept for when it reports it applied (novox/hq ADR 0282 decision
// 6): measured, never acted on.
sentWhat := sentNow(ctx, open.inventory, sent)
for _, m := range rest {
p.Modules[m].SentAt = &now
p.Modules[m].Rest = restOf(restTo[m], sent, sentWhat)
}
fmt.Printf("%s: tier %d built; sent %s to %s, one send each\n", p.ID, p.Tier, strings.Join(rest, ", "),
strings.Join(sent, ", "))
@@ -941,20 +946,6 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
return true, nil
}
// restOf is, for one module, every machine of its rest that the send reached and what it carried there.
func restOf(to, sent []string, what map[string]inventory.SentDeclaration) map[string]inventory.SentDeclaration {
out := map[string]inventory.SentDeclaration{}
for _, n := range to {
if slices.Contains(sent, n) {
out[n] = what[n]
}
}
if len(out) == 0 {
return nil
}
return out
}
// firstSend sends one machine, in one send, every module of the plan's tier whose first machine it is
// (novox/hq issue 281), and records the send on each: what that machine ran of it before — read once,
// before the send, so a module the same send carries is never read as already moved — the machines it
@@ -1202,7 +1193,6 @@ func sayUnsent(p *inventory.Plan, rollsOut func(string) bool) {
// planTicker advances open plans on a timer, for the steps outcomes alone cannot take.
func planTicker(ctx context.Context, open *stores) {
advancePlans(ctx, open)
keepWalkPhases(ctx, open)
tick := time.NewTicker(30 * time.Second)
defer tick.Stop()
for {
@@ -1211,7 +1201,6 @@ func planTicker(ctx context.Context, open *stores) {
return
case <-tick.C:
advancePlans(ctx, open)
keepWalkPhases(ctx, open)
}
}
}
@@ -1759,34 +1748,6 @@ func splitList(s string) []string {
return out
}
// recordsOfAsked is what settleFromRecords reads: for every module of the tier still `asked`, its last
// builds and the record of its own ask, by id.
func recordsOfAsked(ctx context.Context, inv *inventory.Inventory, p *inventory.Plan, tier []string) (
map[string][]inventory.Build, map[string]inventory.Build, error) {
recorded := map[string][]inventory.Build{}
byID := map[string]inventory.Build{}
for _, m := range tier {
if s := p.Modules[m]; s != nil && s.State == "asked" {
builds, err := inv.Builds(ctx, m, 5)
if err != nil {
return nil, nil, err
}
recorded[m] = builds
// Its own ask's record, by id — found even when the outcome named no module (ADR 0219).
if s.Build != "" {
b, found, err := inv.BuildByID(ctx, s.Build)
if err != nil {
return nil, nil, err
}
if found {
byID[s.Build] = b
}
}
}
}
return recorded, byID, nil
}
// settleFromRecords marks every module of the tier still `asked` built — or failed — from a build
// recorded after it was asked, and says whether it changed anything (novox/hq 04-ISSUES/214).
// Newest first, as Builds answers: the first record after the ask is the outcome of that ask.
cmd/mesh-controller/seatverbs.go
+3 -14
View File
@@ -1366,7 +1366,7 @@ func splitCommandLine(line string) ([]string, error) {
var words []string
var cur strings.Builder
inWord := false
quote, opened := rune(0), 0
quote := rune(0)
runes := []rune(line)
for i := 0; i < len(runes); i++ {
r := runes[i]
@@ -1387,7 +1387,7 @@ func splitCommandLine(line string) ([]string, error) {
cur.WriteRune(r)
}
case r == '\'' || r == '"':
quote, opened = r, i
quote = r
inWord = true
case r == '\\' && i+1 < len(runes):
i++
@@ -1405,7 +1405,7 @@ func splitCommandLine(line string) ([]string, error) {
}
}
if quote != 0 {
return nil, unclosedQuote(quote, opened)
return nil, fmt.Errorf("command has an unclosed %c quote", quote)
}
if inWord {
words = append(words, cur.String())
@@ -1413,17 +1413,6 @@ func splitCommandLine(line string) ([]string, error) {
return words, nil
}
// unclosedQuote is the refusal of a line whose quote is never closed. Most often an apostrophe inside
// a single-quoted value ended that quote early and a later quote was left open, so the refusal says
// where the open quote is and how a quote is written inside a quoted value — the shell's own two
// ways, which this splitter already reads (novox/hq issue 294). It quotes none of the line: a refusal
// is kept on the bus as the call's answer, and the line may carry a setting's value.
func unclosedQuote(quote rune, opened int) error {
return fmt.Errorf("command has an unclosed %c quote, opened at character %d — an apostrophe "+
"inside a single-quoted value ends it; write a ' inside single quotes as '\\'' (it'\\''s), or use "+
"double quotes and write \\\" for a \" and \\\\ for a \\ inside them", quote, opened+1)
}
// seatAnnouncement is what the controller says it serves on the bus (novox/hq ADR 0197): the
// mesh-controller seat, one endpoint per verb it answers, each with the seat's own description and
// argument schema — the same facts `tools` answers from the records, as NATS's services format.
cmd/mesh-controller/seatverbs_test.go
-45
View File
@@ -415,48 +415,3 @@ func TestNoVerbSetsTheOperatorsKeyOrRevealsASecret(t *testing.T) {
t.Fatalf("behind %v: %v", behind, err)
}
}
// A value with a quote in it can be said on a `command` line, as in a shell, and a line whose quote
// is left open is refused naming where it opened and how a quote is written, quoting none of it (novox/hq issue 294: an
// apostrophe inside a single-quoted JSON value cut the line, and the refusal named no cause).
func TestAQuotedValueCanHoldAQuote(t *testing.T) {
for _, c := range []struct{ line, want string }{
{`settings set claude-code '{"role":"the operator'\''s laptop"}'`, `{"role":"the operator's laptop"}`},
{`settings set claude-code "{\"role\":\"the operator's laptop\"}"`, `{"role":"the operator's laptop"}`},
{"x \"a \\\\ b\nc\"", "a \\ b\nc"},
{`x 'a\b'`, `a\b`},
} {
argv, err := splitCommandLine(c.line)
if err != nil || argv[len(argv)-1] != c.want {
t.Errorf("%s: read back %q %v, want %q", c.line, argv, err, c.want)
}
}
_, err := splitCommandLine(`settings set claude-code '{"role":"the operator's laptop"}' --node g14`)
if err == nil {
t.Fatal("an apostrophe that leaves a quote open was accepted")
}
for _, want := range []string{"character 57", `'\''`, `\"`} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not say %q: %v", want, err)
}
}
if strings.Contains(err.Error(), "laptop") || strings.Contains(err.Error(), "g14") {
t.Errorf("the refusal quotes the line, which may carry a setting's value: %v", err)
}
}
// Every value reads back as it was when written the way the refusal says: single-quoted with '\”
// for each quote, or double-quoted with \ before each " and \ — newlines and backslashes included.
func TestAQuotedValueRoundTrips(t *testing.T) {
for _, v := range []string{`plain`, `it's`, `say "hi"`, `back\slash\`, "two\nlines", `'"\'\"`, `''`, ``} {
single := "x '" + strings.ReplaceAll(v, "'", `'\''`) + "'"
double := `x "` + strings.NewReplacer(`\`, `\\`, `"`, `\"`).Replace(v) + `"`
for _, line := range []string{single, double} {
argv, err := splitCommandLine(line)
if err != nil || len(argv) != 2 || argv[1] != v {
t.Errorf("%s: read back %q %v, want %q", line, argv, err, v)
}
}
}
}
cmd/mesh-controller/sendable.go
-16
View File
@@ -27,19 +27,6 @@ type sendable struct {
// said it reads one is sent none, because an older node-engine refuses a key it does not know, whole
// (link/order.go, the contract).
Epoch uint64
// Generation is the assignment generation it was composed from (novox/hq issue 234): a machine that
// applied a later one refuses it, so a send composed from a view of the assignments the mesh has moved
// past cannot undeclare what is still assigned. Zero is not sent at all — every machine whose
// node-engine has not said it reads one is sent none, for the reason the epoch is not (an older
// node-engine refuses a key it does not know, whole).
Generation int64
// composedFrom is the generation read before this declaration was composed, and actingEpoch the
// lease epoch its sender acted under — whether or not the machine is sent either — for the record of
// the send; never on the wire.
composedFrom int64
actingEpoch uint64
// modules are the modules this declaration names, for the record of the send; never on the wire.
modules []string
// Adoption is nil for a converged node, and then the body is byte for byte what it was before
// adoption existed: an older host parses the envelope strictly and would refuse the key.
Adoption *adoptionEnvelope
@@ -107,9 +94,6 @@ func (s sendable) Body() ([]byte, error) {
if len(s.LeftOut) > 0 {
envelope["left_out"] = s.LeftOut
}
if s.Generation > 0 {
envelope["generation"] = s.Generation
}
// An empty declaration is deliberate here — the node owns nothing the mesh put there
// (novox/hq issue 127) — and the host refuses an empty body unless it is told the emptiness
// is meant, so a truncated or mis-composed body is never mistaken for "own nothing".
cmd/mesh-controller/signals.go
+1 -36
View File
@@ -226,20 +226,6 @@ var signalsTable = []signalRow{
newest: func(f *signalFacts) time.Time {
return newestOf(f.batches, func(b batchFacts) time.Time { return b.closed })
}},
{Row: "S20", Signal: "a send refused for the assignment generation it was composed from",
Emitter: "node-engine", Trigger: "each refusal (novox/hq issue 234)",
Bound: "none: raised at the first refusal, naming its sender, the generation it came from and the one the " +
"machine applied; cleared an hour after the last",
Kind: kindOlderGeneration, Severity: conditions.Warning, Phase: 2,
needs: func(f *signalFacts) error { return f.refusedSendsErr }, watch: watchGenerationRefusals,
newest: func(f *signalFacts) time.Time {
return newestOf(f.refusedSends, func(s inventory.Send) time.Time {
if s.RefusedAt == nil {
return time.Time{}
}
return *s.RefusedAt
})
}},
{Row: "S17", Signal: "a send held for the bus's planned step is told to a person", Emitter: "controller's plan",
Trigger: "each send refused because it would replace the bus outside its step (novox/hq issue 336)",
Bound: "none: raised at the first refusal, for the operator, naming what waits, the bus build from and to, " +
@@ -537,14 +523,6 @@ func watchAdvisories(f *signalFacts) []conditions.Observation {
if a.Kind == link.AdvisoryConsumerLost && !f.lostConsumers[a.Stream+"."+a.Consumer] {
continue // it exists again, or the mesh no longer expects it: a removal, not a loss
}
if a.Kind == link.AdvisoryMaxDeliveries && a.Token == "" {
// A consumer's max-deliveries key is the dead-letter row's (novox/hq issue 330): said while
// DEAD_LETTERS holds what it gave up on, cleared when that is delivered again or dropped. The
// listener records no such advisory today; one read here as well added a look to the count and
// overwrote the row's words on every look (novox/hq issue 440). Only one that could not be kept
// (AdvisoryNotKept), which DEAD_LETTERS cannot say, is said from here.
continue
}
severity := conditions.Warning
times := ""
if a.Count > 1 {
@@ -555,10 +533,7 @@ func watchAdvisories(f *signalFacts) []conditions.Observation {
machine = f.host
}
o := conditions.Observation{Scope: conditions.ScopeBus, ID: a.ID, Kind: a.Kind, Token: a.Token,
Machine: machine, Severity: severity, Summary: a.Said + times, Said: a.Said,
// The advisory is remembered and read again on every look for an hour: the condition counts
// what the bus said and when, not the looks (novox/hq issue 402).
Happened: a.Last, Times: a.Count}
Machine: machine, Severity: severity, Summary: a.Said + times, Said: a.Said}
if a.Kind == link.AdvisoryMaxDeliveries && a.Token == link.AdvisoryNotKept {
o.Machine = consumerMachine(a.Stream, a.Consumer)
o.Headline = clip(conditions.Capital(fmt.Sprintf("%s gave up on a message, not kept",
@@ -590,17 +565,7 @@ func watchDeadLetters(f *signalFacts) []conditions.Observation {
messages, them = fmt.Sprintf("%d messages", n), "them"
}
who := consumerWho(stream, consumer)
// DEAD_LETTERS is a record of what was given up on: the condition says when the newest held message
// was kept, and its count is a running total of the messages given up on while it is open, never
// how often the controller looked (novox/hq issues 402 and 440). It is at least the number held now,
// and does not go down when one is delivered again or dropped. Without that time it counts its
// looks, as a source of the present does.
happened, times := f.deadLettersNewest[key], 0
if !happened.IsZero() {
times = n
}
out = append(out, conditions.Observation{Scope: conditions.ScopeBus, ID: key, Kind: link.AdvisoryMaxDeliveries,
Happened: happened, Times: times,
Machine: consumerMachine(stream, consumer), Severity: conditions.Warning,
Summary: fmt.Sprintf("%s gave up on %s; %s kept in %s until delivered again or dropped, with why, "+
"through the controller's dead-letters verb", link.ConsumerInWords(stream, consumer), messages,
cmd/mesh-controller/signals_test.go
+4 -13
View File
@@ -100,15 +100,14 @@ var suppressions = map[string]suppression{
inside: func(f *signalFacts) { f.standings = []conditions.Condition{standingSaid(f.now.Add(-29 * time.Minute))} },
past: func(f *signalFacts) { f.standings = []conditions.Condition{standingSaid(f.now.Add(-31 * time.Minute))} },
},
// A message given up on and not kept: the one max-deliveries advisory S9 says itself (issue 440).
"S9": {
inside: func(f *signalFacts) {
f.advisories = []link.Advisory{{Kind: link.AdvisoryMaxDeliveries, ID: "EVENTS.anchor_shop",
Token: link.AdvisoryNotKept, Said: "gave up, not kept", First: f.now.Add(-2 * time.Hour), Last: f.now.Add(-61 * time.Minute), Count: 1}}
f.advisories = []link.Advisory{{Kind: link.AdvisoryMaxDeliveries, ID: "EVENTS.anchor_shop", Said: "gave up",
First: f.now.Add(-2 * time.Hour), Last: f.now.Add(-61 * time.Minute), Count: 1}}
},
past: func(f *signalFacts) {
f.advisories = []link.Advisory{{Kind: link.AdvisoryMaxDeliveries, ID: "EVENTS.anchor_shop",
Token: link.AdvisoryNotKept, Said: "gave up, not kept", First: f.now.Add(-2 * time.Hour), Last: f.now.Add(-59 * time.Minute), Count: 1}}
f.advisories = []link.Advisory{{Kind: link.AdvisoryMaxDeliveries, ID: "EVENTS.anchor_shop", Said: "gave up",
First: f.now.Add(-2 * time.Hour), Last: f.now.Add(-59 * time.Minute), Count: 1}}
},
},
"S10": {
@@ -178,14 +177,6 @@ var suppressions = map[string]suppression{
commit: "c0ffee001122", modules: []string{"app"}, since: f.now.Add(-time.Second)}}}
},
},
// A send refused by its machine for the generation it was composed from (novox/hq issue 234): said at
// once, naming its sender, for an hour after. Inside: the last such refusal was more than an hour ago.
"S20": {
inside: func(f *signalFacts) {
f.refusedSends = []inventory.Send{refusedSend(f.now.Add(-61 * time.Minute))}
},
past: func(f *signalFacts) { f.refusedSends = []inventory.Send{refusedSend(f.now.Add(-time.Second))} },
},
// Twice by hand within a fortnight is a healer wanted; once, or the first of two a day too old, is not.
"S15": {
inside: func(f *signalFacts) {
cmd/mesh-controller/status_summary.go
-5
View File
@@ -196,11 +196,6 @@ func (l nudgingListener) Heard(ctx context.Context, report link.Report) (bool, e
if report.Ordered() {
link.StaleRefusals.Lifetime(report.Node, report.RefusedOlder, now)
}
// A send refused for the generation it was composed from is kept on the send it refused, so S20 names
// its sender (novox/hq issue 234).
if report.OlderGeneration != nil {
heardGenerationRefusal(ctx, l.Enrolment.Inventory, report)
}
// What the machine's witnesses put back and stand by (novox/hq ADR 0236): read by the gate and its
// probe. Only from an account of the machine — not a word that a declaration was set aside, nor a rekey.
if report.Superseded == "" && report.Rekey == nil && report.Node != "" {
cmd/mesh-controller/waiting_provider_test.go
-384
View File
@@ -1,384 +0,0 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A provider waiting for the operator holds its consumers, and a wait beside another wait is said (novox/hq
// issue 405, found in the review of ADR 0283's controller change).
//
// The shapes are those of issue 386 (mounts waiting for smb-password-games: waitingResource, passwordWait) and of
// the openrazer wait on the workstation of 2026-10-11 (relogin, userUnit): an account in its group whose session
// began before it was, and its unit failed in that account's own service manager.
// waitingDatabase is a provider whose only part not healthy waits for the operator's secret: a database's
// process stated waiting, as the node-engine states a tool check answering waits (ADR 0283 decision 2). Its wait
// names the part that waits by the provision it gives.
func waitingDatabase() inventory.ResourceHealth {
return waitingDatabaseFor(inventory.Wait{Part: "postgres-database", Secret: "licence",
What: "the licence key of the database"})
}
func waitingDatabaseFor(waits ...inventory.Wait) inventory.ResourceHealth {
return inventory.ResourceHealth{Module: "db", Resource: "db.server", Kind: "process", Target: "db.service",
State: link.StateWaiting, Check: "tool", Reason: "the database waits for its licence", Waits: waits}
}
// backupsWait is a wait of the same provider for another part than the one its consumers need.
var backupsWait = inventory.Wait{Part: "the nightly backups", Secret: "backup-key", What: "the key of the backups"}
// failingConsumer is a consumer whose check that needs the database fails.
func failingConsumer(module string) inventory.ResourceHealth {
return inventory.ResourceHealth{Module: module, Resource: module + ".web", Kind: "container", Target: module,
State: link.StateUnhealthy, Reason: "http /health on web: answered 500", Check: "http", Needs: "postgres-database"}
}
// dbSecrets is what the database's waits name: own secrets issued outside the mesh, so a wait for one checks out
// while nobody gave it (ADR 0283 decision 3, novox/hq issue 450).
var dbSecrets = catalogue.OwnSecrets{
"licence": {Path: "/s/licence", IssuedBy: catalogue.IssuedOutside},
"backup-key": {Path: "/s/backup-key", IssuedBy: catalogue.IssuedOutside},
}
// holdingOf is one reading of the record without a store: the newest statements, the open conditions, the
// catalogue, what was given on the provider's machine (nothing), and each consumer's provider already looked up,
// as providerFor memoises it.
func holdingOf(open []conditions.Condition, healths map[string]inventory.NodeHealth, bound map[[3]string]catalogue.Chosen) *holding {
h := &holding{ctx: context.Background(), healths: healths, open: open, providers: map[string]providerLookup{},
shelf: map[string]catalogue.Manifest{"db": {Module: "db", Version: "1", OwnSecrets: dbSecrets,
Provides: []catalogue.Offer{{Name: "postgres-database", Scope: catalogue.ScopeMesh}}}},
waits: operatorWaitFacts{given: map[string]map[string]time.Time{"db@anchor": {}}}}
for k, p := range bound {
h.providers[k[0]+"\x00"+k[1]+"\x00"+k[2]] = providerLookup{p, true}
}
return h
}
var theDatabase = catalogue.Chosen{Node: "anchor", Module: "db"}
var shopOnTheDatabase = map[[3]string]catalogue.Chosen{{"laptop", "shop", "postgres-database"}: theDatabase}
func shopFailingBeside(provider ...inventory.ResourceHealth) map[string]inventory.NodeHealth {
return map[string]inventory.NodeHealth{
"anchor": {Node: "anchor", Resources: provider},
"laptop": {Node: "laptop", Resources: []inventory.ResourceHealth{failingConsumer("shop")}},
}
}
// (1) A provider whose only part not healthy waits for the operator holds the findings of the consumers of the
// waiting part under it (ADR 0240 rule 5); a consumer of another part, or one whose part cannot be matched, is
// its own (ADR 0283 decision 1: a module that waits says nothing else of it is wrong).
func TestAProviderWaitingForTheOperatorHoldsOnlyTheConsumersOfTheWaitingPart(t *testing.T) {
shop := []inventory.ResourceHealth{failingConsumer("shop")}
unhealthyDB := waitingDatabase()
unhealthyDB.State, unhealthyDB.Waits, unhealthyDB.Reason = link.StateUnhealthy, nil, "its tool check: refused"
healthyDB := waitingDatabase()
healthyDB.State, healthyDB.Waits = link.StateHealthy, nil
byCredential := holdingOf(nil, shopFailingBeside(waitingDatabaseFor(inventory.Wait{Part: "the server",
Secret: "licence", What: "the licence key"})), shopOnTheDatabase)
byCredential.shelf["db"] = catalogue.Manifest{Module: "db", Version: "1", OwnSecrets: dbSecrets, Provides: []catalogue.Offer{{
Name: "postgres-database", Credential: &catalogue.OfferCredential{Own: "licence"}}}}
for _, c := range []struct {
name string
hold *holding
held bool
onlyWaits bool
uncovered bool
}{
{"the waiting part is the provision", holdingOf(nil, shopFailingBeside(waitingDatabase()), shopOnTheDatabase), true, true, false},
{"the wait is for the provision's shared credential", byCredential, true, true, false},
{"the wait is for another part", holdingOf(nil, shopFailingBeside(waitingDatabaseFor(backupsWait)), shopOnTheDatabase), false, false, true},
{"only the needs-operator condition, its parts not said", holdingOf(
[]conditions.Condition{{Key: needsOperatorKey("db", "anchor"), Kind: kindNeedsOperator}},
shopFailingBeside(), shopOnTheDatabase), false, false, true},
{"an unhealthy provider holds as before", holdingOf(nil, shopFailingBeside(unhealthyDB), shopOnTheDatabase), true, false, false},
{"a healthy provider holds nothing", holdingOf(nil, shopFailingBeside(healthyDB), shopOnTheDatabase), false, false, false},
} {
by, held := c.hold.heldWith("laptop", "shop", shop)
if held != c.held || (held && by.provider != theDatabase) || (len(by.waits) > 0) != c.onlyWaits {
t.Errorf("%s: held %v under %v with waits %v; want held %v, only waits %v", c.name, held, by.provider,
by.waits, c.held, c.onlyWaits)
}
if _, uncovered := c.hold.waitingUncovered("laptop", "shop", shop); uncovered != c.uncovered {
t.Errorf("%s: said as a provider waiting for another part %v; want %v", c.name, uncovered, c.uncovered)
}
}
}
// (1) The consumer's first-node gate under a provider that only waits for the operator passes as a wait for a
// person (ADR 0254), carrying the wait (ADR 0283 decision 4); under an unhealthy provider it waits, as before.
func TestAConsumersGateUnderAWaitingProviderPassesCarryingTheWait(t *testing.T) {
now := time.Now()
since := now.Add(-time.Minute)
for _, c := range []struct {
name string
provider inventory.ResourceHealth
want health
says []string
}{
{"a provider that only waits", waitingDatabase(), healthPerson,
[]string{"waits on db on anchor, which waits for you", "the licence key of the database", "nox secret ask anchor db licence"}},
{"an unhealthy provider", func() inventory.ResourceHealth {
r := waitingDatabase()
r.State, r.Waits, r.Reason = link.StateUnhealthy, nil, "its tool check: refused"
return r
}(), healthWaiting, []string{"waits on db on anchor, which is unhealthy"}},
} {
healths := shopFailingBeside(c.provider)
for m, h := range healths {
h.HeardAt = now
healths[m] = h
}
f := gateFacts{now: now, health: healths, heldOn: heldReadings(holdingOf(nil, healths, shopOnTheDatabase))}
h, why := moduleHealthWord("shop", "laptop", since, f)
if h != c.want {
t.Errorf("%s: the consumer's gate reads %v %q; want %v", c.name, h, why, c.want)
continue
}
for _, s := range c.says {
if !strings.Contains(why, s) {
t.Errorf("%s: the gate's reading %q does not say %q", c.name, why, s)
}
}
}
}
// judgeBoth judges the provider's statement and then the consumer's, two looks each, over a record that changes
// as the statements do.
func judgeBoth(t *testing.T, k *conditions.Keeper, provider []inventory.ResourceHealth,
byProvider, byConsumer map[string]inventory.NodeHealth) []conditions.Condition {
t.Helper()
ctx := t.Context()
was := readHoldingFor
t.Cleanup(func() { readHoldingFor = was })
healths := byProvider
readHoldingFor = func(_ context.Context, _ *inventory.Inventory, open []conditions.Condition) (*holding, error) {
return holdingOf(open, healths, shopOnTheDatabase), nil
}
for look := 1; look <= 2; look++ {
healths = byProvider
if err := judgeModuleHealth(ctx, nil, k, "anchor", map[string][]inventory.ResourceHealth{"db": provider},
map[string]int{"db": look}, time.Now()); err != nil {
t.Fatal(err)
}
}
for look := 1; look <= 2; look++ {
healths = byConsumer
if err := judgeModuleHealth(ctx, nil, k, "laptop", map[string][]inventory.ResourceHealth{"shop": {failingConsumer("shop")}},
map[string]int{"shop": look}, time.Now()); err != nil {
t.Fatal(err)
}
}
open, err := k.Open(ctx)
if err != nil {
t.Fatal(err)
}
return open
}
func conditionOf(open []conditions.Condition, key string) *conditions.Condition {
for i, c := range open {
if c.Key == key {
return &open[i]
}
}
return nil
}
// (1) The consumer raises nothing of its own; the provider's needs-operator condition lists who waits on it and
// stays a warning (ADR 0283 decision 5: never escalated). The consumer's statement arrives after the provider's,
// so it is the consumer's judging (sayWaiters) that lists it at the provider — no store involved.
func TestAWaitingProvidersConditionListsWhoWaitsOnItAndStaysAWarning(t *testing.T) {
k, _ := withConditionsInMemory(t)
open := judgeBoth(t, k, []inventory.ResourceHealth{waitingDatabase()},
map[string]inventory.NodeHealth{"anchor": {Node: "anchor", Resources: []inventory.ResourceHealth{waitingDatabase()}}}, shopFailingBeside(waitingDatabase()))
provider := conditionOf(open, needsOperatorKey("db", "anchor"))
if len(open) != 1 || provider == nil {
t.Fatalf("a provider waiting for the operator and a consumer of its waiting part raised %v; want the "+
"provider's needs-operator alone", openKeysOf(open))
}
if said := provider.Evidence[0].Said; !strings.Contains(said, "shop on laptop") {
t.Fatalf("the provider's needs-operator does not list shop on laptop as waiting on it: %s", said)
}
if provider.Severity != conditions.Warning {
t.Fatalf("the provider's needs-operator became %s with a consumer waiting; it stays a warning", provider.Severity)
}
if provider.Resolver != conditions.ResolverOperator || !strings.Contains(provider.Needs, "desk prompt") {
t.Fatalf("the provider's condition: %+v", provider)
}
}
// (1) A consumer of another part than the one waiting is raised on its own, and says its provider waits.
func TestAConsumerOfAnotherPartIsRaisedOnItsOwn(t *testing.T) {
k, _ := withConditionsInMemory(t)
backups := waitingDatabaseFor(backupsWait)
open := judgeBoth(t, k, []inventory.ResourceHealth{backups}, shopFailingBeside(backups), shopFailingBeside(backups))
consumer := conditionOf(open, moduleUnhealthyKey("shop", "laptop"))
if consumer == nil || conditionOf(open, needsOperatorKey("db", "anchor")) == nil {
t.Fatalf("raised %v; want shop's own unhealthy beside db's needs-operator", openKeysOf(open))
}
if said := consumer.Evidence[0].Said; !strings.Contains(said, "db on anchor waits for you, but not for anything shop is known to need, so shop's fault is said on its own") {
t.Fatalf("the consumer's condition does not say its provider waits: %s", said)
}
}
// relogin and userUnit are person_wait_test.go's; mounts is said here with the same account and unit beside its
// wait for the password.
func reloginBesideAWait() []inventory.ResourceHealth {
return []inventory.ResourceHealth{relogin("mounts", "operator"), userUnit("mounts", "operator"),
waitingResource(passwordWait)}
}
// (2) A module with a checked wait beside a relogin-needed account says both: the new login, and the act the
// operator owes it.
func TestAWaitBesideAReloginIsSaid(t *testing.T) {
k, _ := withConditionsInMemory(t)
ctx := t.Context()
for look := 1; look <= 2; look++ {
if err := judgeModuleHealth(ctx, nil, k, "workstation", map[string][]inventory.ResourceHealth{"mounts": reloginBesideAWait()},
map[string]int{"mounts": look}, time.Now()); err != nil {
t.Fatal(err)
}
}
got, open := needsOperatorOpen(t, k)
if got == nil {
t.Fatalf("a wait for the operator beside a relogin is not said: %v", openKeysOf(open))
}
if !strings.Contains(got.Summary, "smb-password-games") || got.Severity != conditions.Warning {
t.Fatalf("the needs-operator beside the relogin: %+v", got)
}
relogged := false
for _, c := range open {
relogged = relogged || c.Key == reloginKey("mounts", "workstation")
}
if !relogged {
t.Fatalf("the relogin is no longer said beside the wait: %v", openKeysOf(open))
}
// The login done, the wait stays said and the relogin clears.
if err := judgeModuleHealth(ctx, nil, k, "workstation", map[string][]inventory.ResourceHealth{"mounts": {waitingResource(passwordWait)}},
map[string]int{"mounts": 3}, time.Now()); err != nil {
t.Fatal(err)
}
got, open = needsOperatorOpen(t, k)
if got == nil || len(open) != 1 {
t.Fatalf("after the new login: %v", openKeysOf(open))
}
}
// (2) The same beside a directory used as found.
func TestAWaitBesideADirectoryUsedAsFoundIsSaid(t *testing.T) {
k, _ := withConditionsInMemory(t)
ctx := t.Context()
found := foundDirectory("mounts", time.Now().Add(-time.Hour))
for look := 1; look <= 2; look++ {
if err := judgeModuleHealth(ctx, nil, k, "workstation", map[string][]inventory.ResourceHealth{
"mounts": {found, waitingResource(passwordWait)}}, map[string]int{"mounts": look}, time.Now()); err != nil {
t.Fatal(err)
}
}
got, open := needsOperatorOpen(t, k)
if got == nil {
t.Fatalf("a wait for the operator beside a directory used as found is not said: %v", openKeysOf(open))
}
asFound := false
for _, c := range open {
asFound = asFound || c.Key == usedAsFoundKey("mounts", "workstation")
}
if !asFound {
t.Fatalf("the directory used as found is no longer said beside the wait: %v", openKeysOf(open))
}
}
// (2) Beside a fault of its own, the wait is said too, and the fault's words do not count the waiting part among
// what fails.
func TestAWaitBesideAFaultIsSaidAndTheFaultIsTheFaultAlone(t *testing.T) {
k, _ := withConditionsInMemory(t)
ctx := t.Context()
down := inventory.ResourceHealth{Module: "mounts", Resource: "mounts.apply", Kind: "process",
Target: "mesh-mounts-apply.service", State: link.StateUnhealthy, Reason: "down"}
for look := 1; look <= 2; look++ {
if err := judgeModuleHealth(ctx, nil, k, "workstation", map[string][]inventory.ResourceHealth{
"mounts": {down, waitingResource(passwordWait)}}, map[string]int{"mounts": look}, time.Now()); err != nil {
t.Fatal(err)
}
}
got, open := needsOperatorOpen(t, k)
if got == nil {
t.Fatalf("a wait for the operator beside a fault is not said: %v", openKeysOf(open))
}
var fault *conditions.Condition
for i, c := range open {
if c.Key == moduleUnhealthyKey("mounts", "workstation") {
fault = &open[i]
}
}
if fault == nil {
t.Fatalf("the fault is not said: %v", openKeysOf(open))
}
if strings.Contains(fault.Summary, "mounts.watch") {
t.Fatalf("the fault's summary counts the waiting part as failing: %q", fault.Summary)
}
}
func openKeysOf(cs []conditions.Condition) []string {
var out []string
for _, c := range cs {
out = append(out, c.Key)
}
return out
}
// A consumer raised on its own, whose provider then waits for the operator for the part it needs, is cleared saying
// which the provider is: it waits for you, not that it is unhealthy (novox/hq issue 405 review).
func TestAConsumerHeldOnceItsProviderWaitsSaysWhichItIs(t *testing.T) {
k, _ := withConditionsInMemory(t)
ctx := t.Context()
start := time.Now().Add(-time.Second)
healthy := waitingDatabase()
healthy.State, healthy.Waits = link.StateHealthy, nil
healths := shopFailingBeside(healthy)
was := readHoldingFor
t.Cleanup(func() { readHoldingFor = was })
readHoldingFor = func(_ context.Context, _ *inventory.Inventory, open []conditions.Condition) (*holding, error) {
return holdingOf(open, healths, shopOnTheDatabase), nil
}
shop := map[string][]inventory.ResourceHealth{"shop": {failingConsumer("shop")}}
for look := 1; look <= 2; look++ {
if err := judgeModuleHealth(ctx, nil, k, "laptop", shop, map[string]int{"shop": look}, time.Now()); err != nil {
t.Fatal(err)
}
}
if open, _ := k.Open(ctx); conditionOf(open, moduleUnhealthyKey("shop", "laptop")) == nil {
t.Fatalf("shop beside a healthy provider is not raised: %v", openKeysOf(open))
}
healths = shopFailingBeside(waitingDatabase())
if err := judgeModuleHealth(ctx, nil, k, "laptop", shop, map[string]int{"shop": 3}, time.Now()); err != nil {
t.Fatal(err)
}
var why string
for deadline := time.Now().Add(2 * time.Second); why == "" && time.Now().Before(deadline); {
events, err := k.HistorySince(ctx, start)
if err != nil {
t.Fatal(err)
}
for _, e := range events {
if e.Key == moduleUnhealthyKey("shop", "laptop") && e.Change == conditions.ChangeCleared {
why = e.Why
}
}
if why == "" {
time.Sleep(10 * time.Millisecond)
}
}
if !strings.Contains(why, "waits on db on anchor, which waits for you") {
t.Fatalf("shop's clearing says %q; want it to say db on anchor waits for you", why)
}
}
cmd/mesh-controller/watchdogs.go
+2 -20
View File
@@ -55,10 +55,6 @@ type signalFacts struct {
waits []waitFacts
// batches are the batches not yet cut (S18, S19, novox/hq ADR 0276).
batches []batchFacts
// refusedSends are the sends a machine refused within the hour for the generation they were composed
// from, each naming its sender (S20, novox/hq issue 234).
refusedSends []inventory.Send
refusedSendsErr error
loop loopFacts
loopErr error
@@ -83,11 +79,8 @@ type signalFacts struct {
lostConsumers map[string]bool
// deadLetters are how many messages DEAD_LETTERS holds per consumer, by `<stream>.<consumer>`
// (novox/hq issue 330): each consumer's max-deliveries condition is open while it holds any.
deadLetters map[string]int
// deadLettersNewest is when DEAD_LETTERS kept the newest message it holds for each of those
// consumers: the condition counts the messages given up on, not the looks (novox/hq issue 440).
deadLettersNewest map[string]time.Time
advisoriesErr error
deadLetters map[string]int
advisoriesErr error
selfCheck selfCheckFacts
@@ -357,18 +350,7 @@ func (w *watchdogs) gather(ctx context.Context) *signalFacts {
if f.advisoriesErr == nil && w.js != nil {
f.deadLetters, f.advisoriesErr = link.HeldDeadLetters(w.js.Context())
}
if f.advisoriesErr == nil && len(f.deadLetters) > 0 {
f.deadLettersNewest, f.advisoriesErr = link.NewestDeadLetters(w.js.Context(), f.deadLetters)
for key := range f.deadLetters {
if _, ok := f.deadLettersNewest[key]; !ok && f.advisoriesErr == nil {
// Delivered again or dropped between the two reads: left out of this look, rather than
// observed without a time and counted as a look (novox/hq issue 440).
delete(f.deadLetters, key)
}
}
}
f.handActs, f.handActsErr = w.gatherHandActs(ctx, now)
f.refusedSends, f.refusedSendsErr = inv.RefusedSendsSince(ctx, now.Add(-generationRefusalsSaid))
f.facts.taken, _, f.facts.began, f.facts.err = exportedFacts.last()
return f
}
go.mod
+2 -2
View File
@@ -3,7 +3,7 @@ module github.com/novox/mesh-controller
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c
git.novox.be/novox/mesh-sdk/go v0.1.11-0.20261009143344-f047d0a4a970
github.com/jackc/pgx/v5 v5.10.0
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
@@ -35,4 +35,4 @@ require (
// committed. Every build (the build agent's `go build`, the Dockerfile) compiles from vendor/ and
// fetches nothing; go refuses to build when vendor/ and this file disagree, so a pin moved without
// `go mod vendor` fails loudly, at once, everywhere.
replace github.com/novox/mesh-host => git.novox.be/novox/mesh-host v0.0.0-20261011092230-20d5af9b2d5f
replace github.com/novox/mesh-host => git.novox.be/novox/mesh-host v0.0.0-20261009231844-b8c854611812
go.sum
+4 -4
View File
@@ -1,7 +1,7 @@
git.novox.be/novox/mesh-host v0.0.0-20261011092230-20d5af9b2d5f h1:8N5OW2mdTNIck2pe4EciTYX5NsrPsHrTLENGNIWYNTU=
git.novox.be/novox/mesh-host v0.0.0-20261011092230-20d5af9b2d5f/go.mod h1:tcTK4LMs1d6JpZUwy3hSHMFG/MIuDr/XFs7/nbeaW/c=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c h1:l5onJwoIeH8yE/PjVlEeoPyXBsHp2NQiWLllz2fwX7s=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-host v0.0.0-20261009231844-b8c854611812 h1:pzVzwF5VMWaTECxu8+Pd1dNoOHNEm7upC5wPadQTkBw=
git.novox.be/novox/mesh-host v0.0.0-20261009231844-b8c854611812/go.mod h1:K3/xEzVgmrNKLMV2vv4M80MwmPnQNXqvQ4C5Jj0fJT4=
git.novox.be/novox/mesh-sdk/go v0.1.11-0.20261009143344-f047d0a4a970 h1:9tFDQsgmI+4X7/BpZGXIr+HemPKE7YddYGqWV0lINAI=
git.novox.be/novox/mesh-sdk/go v0.1.11-0.20261009143344-f047d0a4a970/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
internal/beside/beside.go
-95
View File
@@ -1,95 +0,0 @@
// Package beside is where a test finds another repository of the mesh it reads: the catalogue's manifests,
// the node-engine's genesis template (novox/hq issue 432), the SDK's conformance fixtures (issue 449).
//
// A test used to read the checkout beside this one, `../../../mesh-catalog`, so its verdict depended on
// whatever sat on the machine running it: a stale or dirty checkout failed it on a desktop, and where none
// was beside it skipped and said nothing. Now there are two inputs, both named, and no third:
//
// - In a merge check, the build seat clones each core repository beside the one checked (the catalogue
// at its main, the node-engine at the commit the mesh runs) and says where in MESH_CHECK_BESIDE. A
// test judges against those clones, so agreement with the other repository is checked where
// `mesh/repo-check` runs; a repository missing there fails the test, never skips it. Which clones a
// check gets is chosen from the inventory: mesh-catalog by the source of the `nats` module, mesh-host
// by the source of `mesh-host`, and mesh-sdk as the controller's sibling at the commit the controller's
// go.mod pins. In a mesh where either module has no source repository nothing is
// cloned, and these tests fail loudly with "not beside this check": a cause in the setup, not in the
// change. And in a delivery group that holds a mesh-catalog pull request, these tests read the
// catalogue's main, not the group's head.
// - Anywhere else, the test judges against the copy captured in this repository's testdata/beside, at the
// commit testdata/beside/CAPTURED names. Never against a developer's own checkout: to judge one, set
// MESH_CHECK_BESIDE to the directory holding it, as the check does.
//
// The captured manifests are named module.json.captured, and put back as module.json in a directory of
// the test's own: a module.json anywhere in this repository is a module of it to the forge's announcer and
// the planner, and a merge would build and register a hundred copies of the catalogue's.
package beside
import (
"io/fs"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
)
// Env is where a merge check says the repositories cloned beside the one checked are (internal/builder's
// EnvBeside, repeated here so a test does not import the builder).
const Env = "MESH_CHECK_BESIDE"
// CapturedSuffix is what a captured file's name carries that the repository's own does not.
const CapturedSuffix = ".captured"
// Dir is the named repository a test reads — the clone beside a merge check, or the captured copy — and
// says which in the test's log.
func Dir(t testing.TB, repository string) string {
t.Helper()
if root := os.Getenv(Env); root != "" {
dir := filepath.Join(root, repository)
if _, err := os.Stat(dir); err != nil {
t.Fatalf("%s is not beside this check in %s=%s, so its agreement with this repository cannot be "+
"judged here: %v", repository, Env, root, err)
}
t.Logf("judged against %s as cloned beside this check", dir)
return dir
}
from := filepath.Join(Captured(), repository)
if _, err := os.Stat(from); err != nil {
t.Fatalf("no captured copy of %s at %s: %v", repository, from, err)
}
dir := filepath.Join(t.TempDir(), repository)
err := filepath.WalkDir(from, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
rel, _ := filepath.Rel(from, path)
into := filepath.Join(dir, strings.TrimSuffix(rel, CapturedSuffix))
if d.IsDir() {
return os.MkdirAll(into, 0o755)
}
raw, err := os.ReadFile(path)
if err != nil {
return err
}
return os.WriteFile(into, raw, 0o644)
})
if err != nil {
t.Fatalf("the captured copy of %s could not be laid out: %v", repository, err)
}
t.Logf("judged against the copy of %s captured in testdata/beside (see CAPTURED); a merge check "+
"judges it against the repository's own clone", repository)
return dir
}
// Catalogue is the catalogue's modules directory, as Dir finds the catalogue.
func Catalogue(t testing.TB) string {
t.Helper()
return filepath.Join(Dir(t, "mesh-catalog"), "modules")
}
// Captured is this repository's testdata/beside, found from this file rather than from the working
// directory, so a test in any package reaches it.
func Captured() string {
_, file, _, _ := runtime.Caller(0)
return filepath.Join(filepath.Dir(file), "..", "..", "testdata", "beside")
}
internal/broker/agreement_catalogue_test.go
+3 -4
View File
@@ -8,7 +8,6 @@ import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
"github.com/novox/mesh-controller/internal/catalogue"
)
@@ -75,10 +74,10 @@ func TestTheAgreementCheckCatchesASubscriptionThatMatchesNothing(t *testing.T) {
func theCataloguesEvents(t *testing.T) ([]AnEmitter, []AConsumer, []DeclaredSeat) {
t.Helper()
root := beside.Catalogue(t)
root := filepath.Join("..", "..", "..", "mesh-catalog", "modules")
entries, err := os.ReadDir(root)
if err != nil {
t.Fatal(err)
t.Skipf("catalogue sibling not present: %v", err)
}
var emitters []AnEmitter
var consumers []AConsumer
@@ -120,7 +119,7 @@ func theCataloguesEvents(t *testing.T) ([]AnEmitter, []AConsumer, []DeclaredSeat
}
}
if len(emitters) == 0 {
t.Fatalf("no module under %s emits anything, so this proved nothing", root)
t.Skip("no manifests found beside this checkout")
}
return emitters, consumers, seats
}
internal/broker/callers.go
-117
View File
@@ -1,117 +0,0 @@
package broker
import (
"regexp"
"strings"
)
// A tool call names its caller (novox/hq issue 365, by ADR 0259 §3's precedent for asks).
//
// mesh.mod.<module>.call.<tool>[.<node>].<caller>
// mesh.seat.<seat>.call.<verb>[.<node>].<caller>
//
// The last token is the bus user that published the call, and each user is granted these subjects with its own
// name there and no other — the caller is a fact the server enforces, and the tool runtime hands it to the
// module from the subject a call arrived on (mesh-tools node-tools internal/bus caller.go holds the same shape).
//
// **A kind of its own, `call`, not the `tool` subject with a token appended.** Every grant to call a tool is a
// wildcard over the `tool` kind — `.tool.<t>.*` for the instance on any machine, `mesh.mod.*.tool.>` for every
// tool — and either would match a `tool` subject with any caller appended, so a caller could name another.
// Under `call` nothing is granted but the caller-named subjects. No stream's filter covers it: a tool call is
// never persisted (design 25 §3).
//
// **Derived from the `tool` grants, in one place** (callerNamed): wherever a principal may call or answer a
// tool, it may call it in its own name, or answer it naming any caller — so no kind of principal is left
// unable to call in its own name, and a new grant to call is one in both shapes by construction.
//
// The `tool` grants stay beside these for one release, so every caller and every runtime moves without a gap:
// their retirement is hq issue 464.
const CallKind = "call"
var userName = regexp.MustCompile(`^[A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*$`)
// CallerToken is a bus user's name as the last token of a call it publishes: each dot written `~`, which no part
// of a user's name may hold (safeSubject), so the token names that user and no other. "" for a name that is not
// a user's.
func CallerToken(user string) string {
if !userName.MatchString(user) {
return ""
}
return strings.ReplaceAll(user, ".", "~")
}
// toolGrant splits a grant on the `tool` kind — `mesh.mod.<m>.tool.<rest>` or `mesh.seat.<s>.tool.<rest>`, any
// part possibly a wildcard — at the kind; ok is false for any other subject.
func toolGrant(subject string) (head, rest string, ok bool) {
parts := strings.SplitN(subject, ".", 5)
if len(parts) != 5 || parts[0] != "mesh" || (parts[1] != "mod" && parts[1] != "seat") || parts[3] != "tool" ||
parts[2] == "" || parts[4] == "" {
return "", "", false
}
return parts[0] + "." + parts[1] + "." + parts[2], parts[4], true
}
// CalledSubject is where a call to a tool subject goes naming its caller; "" when the subject is not a tool's
// or the user is not a bus user.
func CalledSubject(subject, user string) string {
head, rest, ok := toolGrant(subject)
token := CallerToken(user)
if !ok || token == "" || strings.ContainsAny(rest, "*>") {
return ""
}
return head + "." + CallKind + "." + rest + "." + token
}
// CalledPattern is what a holder answering a tool subject also subscribes, to hear the calls that name their
// caller: the same address under the `call` kind, any caller last. "" for a subject that is not a tool's.
func CalledPattern(subject string) string {
head, rest, ok := toolGrant(subject)
if !ok || strings.ContainsAny(rest, "*>") {
return ""
}
return head + "." + CallKind + "." + rest + ".*"
}
// calledPublish is a grant to call on the `tool` kind, as the same grant in the caller's own name: its last
// token the caller's, and nothing that reaches past it. A `>` is every tail a call carries — the tool alone or
// the tool and the machine — so it becomes both, each ending in the caller.
func calledPublish(grant, token string) []string {
head, rest, ok := toolGrant(grant)
if !ok || token == "" {
return nil
}
base := head + "." + CallKind + "."
switch {
case rest == ">":
return []string{base + "*." + token, base + "*.*." + token}
case strings.HasSuffix(rest, ".>"):
return []string{base + strings.TrimSuffix(rest, ">") + "*." + token}
}
return []string{base + rest + "." + token}
}
// calledSubscribe is a grant to answer on the `tool` kind, as the same grant for the calls naming any caller.
func calledSubscribe(grant string) []string {
head, rest, ok := toolGrant(grant)
if !ok {
return nil
}
base := head + "." + CallKind + "."
if strings.HasSuffix(rest, ">") {
return []string{base + rest}
}
return []string{base + rest + ".*"}
}
// callerNamed adds, beside a principal's grants on the `tool` kind, the same grants under `call`: to publish in
// its own name, to subscribe naming anybody.
func callerNamed(user string, pub, sub []string) ([]string, []string) {
token := CallerToken(user)
for _, g := range pub {
pub = append(pub, calledPublish(g, token)...)
}
for _, g := range sub {
sub = append(sub, calledSubscribe(g)...)
}
return unique(pub), unique(sub)
}
internal/broker/callers_live_test.go
-100
View File
@@ -1,100 +0,0 @@
package broker
import (
"errors"
"os"
"path/filepath"
"testing"
"time"
"github.com/nats-io/nats-server/v2/server"
"github.com/nats-io/nats.go"
"golang.org/x/crypto/bcrypt"
)
// **On a real server, as composed** (novox/hq issue 365): a machine's runtime calls in its own name, and the
// server refuses it a call naming another — the grant, not the runtime, is what makes the caller a fact.
func TestAServerComposedFromTheGrantsRefusesACallNamingAnother(t *testing.T) {
hash, err := bcrypt.GenerateFromPassword([]byte("pw"), bcrypt.MinCost)
if err != nil {
t.Fatal(err)
}
ledger := Seat{Name: "issue-tracker", Scope: "mesh", Serves: []string{"open"}}
accounts, err := ComposeAccounts([]Principal{
{Kind: KindNodeTools, Node: "novox", Module: RuntimeModule, PasswordHash: string(hash),
Carries: []Declared{{Module: "mesh-issues", Holds: []Seat{ledger}}}},
{Kind: KindNodeTools, Node: "shanks", Module: RuntimeModule, PasswordHash: string(hash)},
})
if err != nil {
t.Fatal(err)
}
conf := filepath.Join(t.TempDir(), "bus.conf")
if err := os.WriteFile(conf, []byte("listen: 127.0.0.1:-1\njetstream { store_dir: "+
`"`+t.TempDir()+`"`+" }\n"+accounts), 0o600); err != nil {
t.Fatal(err)
}
opts, err := server.ProcessConfigFile(conf)
if err != nil {
t.Fatalf("the composed accounts do not parse: %v", err)
}
opts.NoLog, opts.NoSigs = true, true
s, err := server.NewServer(opts)
if err != nil {
t.Fatal(err)
}
go s.Start()
if !s.ReadyForConnections(10 * time.Second) {
t.Fatal("the bus did not come up")
}
defer s.Shutdown()
holder, err := nats.Connect(s.ClientURL(), nats.UserInfo("novox.node-tools", "pw"))
if err != nil {
t.Fatal(err)
}
defer holder.Close()
heard := make(chan string, 4)
sub, err := holder.Subscribe("mesh.seat.issue-tracker.call.open.*", func(m *nats.Msg) {
heard <- m.Subject
_ = m.Respond([]byte(`{"result":{}}`))
})
if err != nil {
t.Fatal(err)
}
_ = holder.Flush()
if !sub.IsValid() {
t.Fatal("the holder may not hear the caller-named calls to its seat")
}
refusals := make(chan error, 4)
caller, err := nats.Connect(s.ClientURL(), nats.UserInfo("shanks.node-tools", "pw"),
nats.CustomInboxPrefix("_INBOX.shanks.node-tools"),
nats.ErrorHandler(func(_ *nats.Conn, _ *nats.Subscription, err error) { refusals <- err }))
if err != nil {
t.Fatal(err)
}
defer caller.Close()
if _, err := caller.Request("mesh.seat.issue-tracker.call.open.shanks~node-tools", []byte(`{}`), 3*time.Second); err != nil {
t.Fatalf("a call in the caller's own name was not answered: %v", err)
}
if got := <-heard; got != "mesh.seat.issue-tracker.call.open.shanks~node-tools" {
t.Fatalf("heard %s", got)
}
if err := caller.Publish("mesh.seat.issue-tracker.call.open.novox~node-tools", []byte(`{}`)); err != nil {
t.Fatal(err)
}
_ = caller.Flush()
select {
case err := <-refusals:
if !errors.Is(err, nats.ErrPermissionViolation) {
t.Errorf("the server said %v, want a permissions violation", err)
}
case <-time.After(3 * time.Second):
t.Error("the server did not refuse a call naming another caller")
}
select {
case got := <-heard:
t.Errorf("a call naming another reached the holder: %s", got)
case <-time.After(200 * time.Millisecond):
}
}
internal/broker/callers_test.go
-119
View File
@@ -1,119 +0,0 @@
package broker
import (
"strings"
"testing"
)
// **A tool call names its caller, and the bus lets each user name itself alone** (novox/hq issue 365, by ADR
// 0259 §3's precedent for asks): wherever a principal may call a tool, it may call it on the caller-named
// subject — kind `call`, its own bus user last, dots written `~` — and on no subject naming anybody else.
func TestEachCredentialMayCallOnlyInItsOwnName(t *testing.T) {
ledger := Seat{Name: "node-desk", Scope: "node", Serves: []string{"who"}}
tracker := Seat{Name: "issue-tracker", Scope: "mesh", Serves: []string{"open"}}
for _, c := range []struct {
p Principal
may []string
mayNot []string
subject []string // what it subscribes, to answer calls naming any caller
}{
{p: Principal{Kind: KindPerson, Module: "jochen", Invokes: []string{"*"}},
may: []string{"mesh.mod.ledger.call.who.person~jochen", "mesh.mod.ledger.call.who.anchor.person~jochen",
"mesh.seat.issue-tracker.call.open.person~jochen", "mesh.seat.node-desk.call.who.anchor.person~jochen"},
mayNot: []string{"mesh.mod.ledger.call.who.shanks~node-tools", "mesh.mod.ledger.call.who.anchor.controller",
"mesh.seat.issue-tracker.call.open.person~somebody", "mesh.mod.ledger.call.who.person"}},
{p: Principal{Kind: KindNodeTools, Node: "shanks", Module: RuntimeModule,
Carries: []Declared{{Module: "ledger", Holds: []Seat{ledger, tracker}}}},
may: []string{"mesh.mod.ledger.call.who.shanks~node-tools", "mesh.seat.issue-tracker.call.open.shanks~node-tools"},
mayNot: []string{"mesh.mod.ledger.call.who.novox~node-tools", "mesh.seat.issue-tracker.call.open.controller"},
subject: []string{"mesh.mod.ledger.call.who.novox~node-tools", "mesh.seat.node-desk.call.who.shanks.person~jochen", "mesh.seat.issue-tracker.call.open.controller"}},
{p: Principal{Kind: KindModule, Node: "two", Module: "shop", Invokes: []string{"ledger.who", "seat:issue-tracker.open"}},
may: []string{"mesh.mod.ledger.call.who.two~shop", "mesh.mod.ledger.call.who.anchor.two~shop",
"mesh.seat.issue-tracker.call.open.two~shop"},
mayNot: []string{"mesh.mod.ledger.call.other.two~shop", "mesh.mod.ledger.call.who.one~shop",
"mesh.seat.issue-tracker.call.open.one~telegram"}},
{p: Principal{Kind: KindNode, Node: "one", Checks: []string{"ledger.health"}},
may: []string{"mesh.mod.ledger.call.health.one.node~one"},
mayNot: []string{"mesh.mod.ledger.call.health.two.node~one", "mesh.mod.ledger.call.health.one.node~two"}},
{p: Principal{Kind: KindModule, Node: "one", Module: "ledger", Holds: []Seat{ledger, tracker}},
subject: []string{"mesh.mod.ledger.call.who.person~jochen", "mesh.mod.ledger.call.who.one.controller",
"mesh.seat.node-desk.call.who.one.two~shop", "mesh.seat.issue-tracker.call.open.two~shop"}},
} {
perms, err := PermissionsFor(c.p)
if err != nil {
t.Fatalf("%s: %v", c.p.Username(), err)
}
for _, s := range c.may {
if !MayPublish(perms, s) {
t.Errorf("%s may not call %s, in its own name", c.p.Username(), s)
}
}
for _, s := range c.mayNot {
if MayPublish(perms, s) {
t.Errorf("%s may call %s, naming somebody else", c.p.Username(), s)
}
}
for _, s := range c.subject {
if !MaySubscribe(perms, s) {
t.Errorf("%s does not hear %s, a call to what it serves", c.p.Username(), s)
}
}
}
}
// The subjects that name no caller stay granted beside the caller-named ones for one release, so a caller
// moves over without a gap (their retirement: hq issue 464).
func TestTheSubjectsThatNameNoCallerStayGrantedForOneRelease(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindModule, Node: "two", Module: "shop", Invokes: []string{"ledger.who"}})
if err != nil {
t.Fatal(err)
}
for _, s := range []string{"mesh.mod.ledger.tool.who", "mesh.mod.ledger.tool.who.anchor"} {
if !MayPublish(perms, s) {
t.Errorf("the old subject %s is no longer granted", s)
}
}
}
// The desk's hidden prompt is the controller's alone on the caller-named subjects too (the review of
// 2026-10-09, M4): a grant of every tool does not reach it there either.
func TestTheDesksPromptIsTheControllersAloneUnderCallToo(t *testing.T) {
for _, p := range []Principal{{Kind: KindPerson, Module: "jochen", Invokes: []string{"*"}},
{Kind: KindNodeTools, Node: "shanks", Module: RuntimeModule}} {
perms, err := PermissionsFor(p)
if err != nil {
t.Fatal(err)
}
token := CallerToken(p.Username())
for _, s := range []string{"mesh.seat.node-launcher.call.secret.shanks." + token,
"mesh.mod.shell.call.node-launcher.secret.shanks." + token, "mesh.mod.shell.call.node-launcher.secret." + token} {
if MayPublish(perms, s) {
t.Errorf("%s may publish %s, the desk's hidden prompt", p.Username(), s)
}
}
}
}
// The controller's grants are the installer's first user list too, so they move with hq issue 464: this release
// it calls and serves on the subjects that name no caller alone, and nobody may call in its name.
func TestTheControllerKeepsTheSubjectsThatNameNoCallerThisRelease(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindController})
if err != nil {
t.Fatal(err)
}
for _, s := range append(perms.Publish, perms.Subscribe...) {
if strings.Contains(s, "."+CallKind+".") {
t.Errorf("the controller is granted %s, which the installer's first user list does not carry", s)
}
}
for _, p := range []Principal{{Kind: KindPerson, Module: "jochen", Invokes: []string{"*"}},
{Kind: KindNodeTools, Node: "shanks", Module: RuntimeModule}} {
perms, err := PermissionsFor(p)
if err != nil {
t.Fatal(err)
}
if MayPublish(perms, "mesh.mod.ledger.call.who.controller") || MayPublish(perms, "mesh.seat.mesh-controller.call.status.controller") {
t.Errorf("%s may call in the controller's name", p.Username())
}
}
}
internal/broker/genesis_template_test.go
+3 -6
View File
@@ -10,8 +10,6 @@ import (
"testing"
"golang.org/x/crypto/bcrypt"
"github.com/novox/mesh-controller/internal/beside"
)
// **The first user list the installer carries must be the one the controller would compose.**
@@ -78,14 +76,13 @@ func theCarriedAccounts(t *testing.T) string {
return ""
}
// theTemplate is the installer's bundle, as resources: the node-engine's, as a merge check clones it at
// the commit the mesh runs, or as captured in testdata/beside (internal/beside, novox/hq issue 432).
// theTemplate is the installer's bundle, as resources.
func theTemplate(t *testing.T) []map[string]any {
t.Helper()
path := filepath.Join(beside.Dir(t, "mesh-host"), "examples", "foundation-first-node-nats.lock")
path := filepath.Join("..", "..", "..", "mesh-host", "examples", "foundation-first-node-nats.lock")
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
t.Skipf("the host's checkout is not beside this one: %v", err)
}
// The template is JSON with line comments, which is how every one of them is written.
var lines []string
internal/broker/invokes_test.go
+2 -3
View File
@@ -42,9 +42,8 @@ func TestInvokingGrantsNothingButTheCall(t *testing.T) {
if strings.Contains(p, ".event.") {
t.Errorf("a module that only invokes may publish %q, an event it never declared", p)
}
// A role's tools are tools (ADR 0132), named by their caller or not (novox/hq issue 365); a role's work
// queue and events are not.
if strings.HasPrefix(p, "mesh.seat.") && !strings.Contains(p, ".tool.") && !strings.Contains(p, ".call.") {
// A role's tools are tools (ADR 0132); a role's work queue and events are not.
if strings.HasPrefix(p, "mesh.seat.") && !strings.Contains(p, ".tool.") {
t.Errorf("a module that only invokes may publish %q, a seat it neither holds nor uses", p)
}
}
internal/broker/jetstream.go
-38
View File
@@ -448,41 +448,3 @@ func (j *JetStream) BucketNames() ([]string, error) {
}
return out, lister.Error()
}
// EnsureLog creates a module's log if it is absent and brings its configuration to match if it is
// present (novox/hq ADR 0297).
//
// **An update, never a delete and recreate**, for a bucket's reason: recreating discards what the log
// holds, and a log is a module's record. A configuration the server will not change in place (its
// storage, say, on a stream made by hand) is said as this assertion's error and the stream is left as
// it is — never removed to be made again.
func (j *JetStream) EnsureLog(l Log) error {
js, err := jetstream.New(j.conn)
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if _, err := js.CreateOrUpdateStream(ctx, l.Config()); err != nil {
return fmt.Errorf("asserting log %s: %w", l.Stream(), err)
}
return nil
}
// LogStreams is every log stream on the server: every stream whose name starts with LogStreamPrefix.
func (j *JetStream) LogStreams() ([]string, error) {
js, err := jetstream.New(j.conn)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
lister := js.StreamNames(ctx)
var out []string
for name := range lister.Name() {
if strings.HasPrefix(name, LogStreamPrefix) {
out = append(out, name)
}
}
return out, lister.Err()
}
internal/broker/log.go
-195
View File
@@ -1,195 +0,0 @@
package broker
import (
"fmt"
"sort"
"strings"
"github.com/nats-io/nats.go/jetstream"
)
// A module's log on the bus (novox/hq ADR 0297): its record of operations, each entry appended under
// a key and kept as long as the log is.
//
// A log follows a bucket in every respect (ADR 0201): the module names it locally, the mesh derives
// its stream and subjects, the controller creates it from the catalogue on every raise and from
// registration, never a module, and **no path of the mesh removes one** — a log whose declaration
// is gone is reported, as a bucket is. Unlike a bucket, a log is its owner's alone: no other module
// reads it, so there is no read of a log to grant or issue.
//
// Pure, but for the stream's configuration, which is the server's own type so that what is tested
// is exactly what is sent; jetstream.go is the part that asks a server.
// The mesh's caps on a log: what an entry's value may weigh, what a message on the log's stream may
// weigh — the value and its headers, which the server counts in a message's size, so a value of the
// full size still fits with the expected-last-sequence header an append carries — and what a log holds
// when its module says nothing and at most.
const (
LogMaxEntryBytes = 256 * 1024
LogMaxMessageBytes = LogMaxEntryBytes + 4*1024
LogDefaultMiB = 1024
LogMostMiB = 8192
)
// A Log is one module's declared log as the bus holds it.
type Log struct {
Module string
Name string
// MaxMiB is its cap in MiB; zero is LogDefaultMiB.
MaxMiB int
}
// LogStreamPrefix starts every log's stream name, which no other stream of the mesh's starts with.
const LogStreamPrefix = "LOG_"
// LogStreamName is the stream a module's log lives in: `LOG_<module>_<name>`. The module and the
// local name are each one token with no underscore, so two modules can never derive one stream.
func LogStreamName(module, name string) string { return LogStreamPrefix + module + "_" + name }
// LogSubject is the subject a log's entries are published under, without the key: an entry for key K
// is on `<LogSubject>.<K>`.
func LogSubject(module, name string) string { return "mesh.log." + module + "." + name }
// Stream is this log's stream name.
func (l Log) Stream() string { return LogStreamName(l.Module, l.Name) }
// Subject is this log's subject, without the key.
func (l Log) Subject() string { return LogSubject(l.Module, l.Name) }
// MaxBytes is this log's cap in bytes.
func (l Log) MaxBytes() int64 {
mib := l.MaxMiB
if mib <= 0 {
mib = LogDefaultMiB
}
return int64(mib) * 1024 * 1024
}
// Why is carried into the server's description of the stream, so somebody reading the server's own
// state finds whose it is and why it is kept.
func (l Log) Why() string {
return fmt.Sprintf("%s's log %q (novox/hq ADR 0297): its record of operations, one entry per operation "+
"under its key, appended by %s alone and kept as long as the log; never removed by the mesh, because "+
"it is data", l.Module, l.Name, l.Module)
}
// Config is the stream a log is, field by field as novox/hq ADR 0297 fixes it: a file stream kept by
// limits, with no maximum age and no cap per key, whose message is an entry's value and 4 KiB of
// headers at most, that refuses a new entry when full rather than
// drop an old one, and that refuses deleting an entry or purging it. Direct gets are allowed, which
// is how the runtime reads it.
func (l Log) Config() jetstream.StreamConfig {
return jetstream.StreamConfig{
Name: l.Stream(),
Description: l.Why(),
Subjects: []string{l.Subject() + ".>"},
Storage: jetstream.FileStorage,
Retention: jetstream.LimitsPolicy,
MaxAge: 0,
MaxMsgs: -1,
MaxMsgsPerSubject: -1,
MaxBytes: l.MaxBytes(),
MaxMsgSize: LogMaxMessageBytes,
Discard: jetstream.DiscardNew,
AllowDirect: true,
DenyDelete: true,
DenyPurge: true,
Replicas: 1,
}
}
// LogIssued is one log an assignment may reach, by the name its module uses for it (novox/hq ADR
// 0297): its stream, the subject its entries go under, and whether it may append. Only the owner's
// instances are issued a log, so Writes is always true today; it is said so the runtime need not
// assume it.
type LogIssued struct {
Name string `json:"name"`
Stream string `json:"stream"`
Subject string `json:"subject"`
Writes bool `json:"writes"`
}
// logsIssuedFor is every log a module's code may reach, as its membership lists them: its own.
func logsIssuedFor(d Declared) []LogIssued {
var out []LogIssued
for _, l := range d.Logs {
if !safeSubject.MatchString(d.Module) || !safeSubject.MatchString(l.Name) {
continue
}
out = append(out, LogIssued{Name: l.Name, Stream: LogStreamName(d.Module, l.Name),
Subject: LogSubject(d.Module, l.Name), Writes: true})
}
return out
}
// logGrants is what a principal publishes to reach the logs its module keeps: for each, appending
// under the log's subjects, binding to its stream, and reading it directly. Nothing more — the
// runtime reads a log by direct gets alone and makes no consumer on it — and nothing of any other
// module's log, because a log is its owner's alone. Replies come on the principal's inbox, as for a
// bucket.
func logGrants(module string, names []string) []string {
if !safeSubject.MatchString(module) {
return nil
}
var out []string
for _, name := range names {
if !safeSubject.MatchString(name) {
continue
}
stream := LogStreamName(module, name)
out = append(out,
LogSubject(module, name)+".>",
"$JS.API.STREAM.INFO."+stream,
"$JS.API.DIRECT.GET."+stream,
"$JS.API.DIRECT.GET."+stream+".>")
}
return out
}
// logNames is the local names of a module's logs.
func logNames(logs []Log) []string {
out := make([]string, 0, len(logs))
for _, l := range logs {
out = append(out, l.Name)
}
return out
}
// A LogAsserter is the part of a JetStream connection log assertion needs. It has no way to remove a
// log, by design: nothing the mesh runs asks for one.
type LogAsserter interface {
// EnsureLog creates the log's stream if absent and brings its configuration to match if present,
// never deleting or recreating it.
EnsureLog(l Log) error
// LogStreams is every log stream on the server: every stream named with LogStreamPrefix.
LogStreams() ([]string, error)
}
// RaiseLogs asserts every declared log and answers the log streams on the server that nothing
// declares any more.
//
// **Those are reported, never removed** (novox/hq ADR 0297 §2, ADR 0201 §15–16): a log is a module's
// record, and a manifest edited, a module renamed or unassigned is an ordinary day's work that must
// not take a record with it. Removing one is a person's act, outside the mesh.
func RaiseLogs(a LogAsserter, logs []Log) (undeclared []string, err error) {
sorted := append([]Log(nil), logs...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Stream() < sorted[j].Stream() })
declared := map[string]bool{}
for _, l := range sorted {
if err := a.EnsureLog(l); err != nil {
return nil, fmt.Errorf("asserting %s's log %q: %w", l.Module, l.Name, err)
}
declared[l.Stream()] = true
}
names, err := a.LogStreams()
if err != nil {
return nil, fmt.Errorf("listing the bus's logs: %w", err)
}
for _, n := range names {
if strings.HasPrefix(n, LogStreamPrefix) && !declared[n] {
undeclared = append(undeclared, n)
}
}
sort.Strings(undeclared)
return undeclared, nil
}
internal/broker/log_test.go
-294
View File
@@ -1,294 +0,0 @@
package broker
import (
"context"
"encoding/json"
"go/ast"
"go/parser"
"go/token"
"io/fs"
"path/filepath"
"slices"
"strings"
"testing"
"time"
"github.com/nats-io/nats.go/jetstream"
)
// **The stream a log is, field by field** (novox/hq ADR 0297 §2, the shared contract): a file stream
// kept by limits, no maximum age, no cap per key, the declared cap, a message of at most 260 KiB (a
// value of 256 KiB and 4 KiB of headers),
// refusing what comes next when full, read directly, refusing a delete or a purge, one replica, and
// a description that says whose it is and why.
func TestALogsStreamIsAsTheDecisionFixesIt(t *testing.T) {
got := Log{Module: "mesh-issues", Name: "changes"}.Config()
want := jetstream.StreamConfig{
Name: "LOG_mesh-issues_changes",
Description: got.Description,
Subjects: []string{"mesh.log.mesh-issues.changes.>"},
Storage: jetstream.FileStorage,
Retention: jetstream.LimitsPolicy,
MaxAge: 0,
MaxMsgs: -1,
MaxMsgsPerSubject: -1,
MaxBytes: 1024 * 1024 * 1024,
MaxMsgSize: 260 * 1024,
Discard: jetstream.DiscardNew,
AllowDirect: true,
DenyDelete: true,
DenyPurge: true,
Replicas: 1,
}
a, _ := json.Marshal(got)
b, _ := json.Marshal(want)
if string(a) != string(b) {
t.Fatalf("the log's stream is\n %s\nwant\n %s", a, b)
}
if !strings.Contains(got.Description, "mesh-issues") || !strings.Contains(got.Description, "ADR 0297") {
t.Fatalf("the description does not say whose the log is and why: %q", got.Description)
}
if c := (Log{Module: "m", Name: "n", MaxMiB: 8192}).Config(); c.MaxBytes != 8192*1024*1024 {
t.Fatalf("a cap of 8192 MiB is %d bytes", c.MaxBytes)
}
}
// **The membership names each of the owner's logs** with exactly the field names the runtime reads:
// `logs`, and in each `name`, `stream`, `subject`, `writes`. A module with no log is issued none, and
// the field is absent.
func TestAMembershipListsItsModulesLogs(t *testing.T) {
m := MembershipFor("one", Declared{Module: "mesh-issues",
Logs: []Log{{Module: "mesh-issues", Name: "changes"}}}, Placements{})
raw, err := json.Marshal(m)
if err != nil {
t.Fatal(err)
}
var back map[string]json.RawMessage
if err := json.Unmarshal(raw, &back); err != nil {
t.Fatal(err)
}
if got := string(back["logs"]); got !=
`[{"name":"changes","stream":"LOG_mesh-issues_changes","subject":"mesh.log.mesh-issues.changes","writes":true}]` {
t.Fatalf("the membership's logs are %s", got)
}
none, _ := json.Marshal(MembershipFor("one", Declared{Module: "audit"}, Placements{}))
if strings.Contains(string(none), `"logs"`) {
t.Fatalf("a module with no log was issued logs: %s", none)
}
}
// logGrantsOf is the grants of a principal that are about logs.
func logGrantsOf(publish []string) []string {
var out []string
for _, s := range publish {
if strings.HasPrefix(s, "mesh.log.") || strings.Contains(s, ".LOG_") {
out = append(out, s)
}
}
slices.Sort(out)
return out
}
// **The runtime is granted exactly the contract's subjects for each log it carries, and nothing else
// of any log**: appending under the log's subjects, binding to its stream, reading it directly. No
// consumer, no delete, no purge, and nothing of a log its modules do not keep.
func TestTheRuntimeIsGrantedItsModulesLogsAndNoMore(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindNodeTools, Node: "one", Module: RuntimeModule,
Carries: []Declared{
{Module: "mesh-issues", Logs: []Log{{Module: "mesh-issues", Name: "changes"}}},
{Module: "audit"},
}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
want := []string{
"$JS.API.DIRECT.GET.LOG_mesh-issues_changes",
"$JS.API.DIRECT.GET.LOG_mesh-issues_changes.>",
"$JS.API.STREAM.INFO.LOG_mesh-issues_changes",
"mesh.log.mesh-issues.changes.>",
}
if got := logGrantsOf(perms.Publish); !slices.Equal(got, want) {
t.Fatalf("the runtime is granted\n %q\nwant\n %q", got, want)
}
for _, s := range perms.Subscribe {
if strings.HasPrefix(s, "mesh.log.") || strings.Contains(s, "LOG_") {
t.Fatalf("the runtime subscribes a log's subjects: %q", s)
}
}
// A module running on its own account is granted the same for its own logs.
own, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "mesh-issues",
Logs: []string{"changes"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
if got := logGrantsOf(own.Publish); !slices.Equal(got, want) {
t.Fatalf("the module's own account is granted\n %q\nwant\n %q", got, want)
}
// And a module with no log, nothing of any.
none, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "audit", PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
if got := logGrantsOf(none.Publish); len(got) != 0 {
t.Fatalf("a module with no log is granted %q", got)
}
}
// A log name or module that is not one plain token is issued and granted nothing rather than a
// pattern that happens to parse.
func TestALogThatNamesNoStreamGrantsNothing(t *testing.T) {
if got := logGrants("a", []string{"x.y", "x>", "*", ""}); len(got) != 0 {
t.Fatalf("granted %v for logs that name no stream", got)
}
if got := logGrants("a.b", []string{"c"}); len(got) != 0 {
t.Fatalf("granted %v for a module that is not one token", got)
}
}
type logs struct {
ensured []string
on []string
}
func (l *logs) EnsureLog(x Log) error { l.ensured = append(l.ensured, x.Stream()); return nil }
func (l *logs) LogStreams() ([]string, error) { return l.on, nil }
// Every declared log is asserted; one on the server that nothing declares is said, not removed — and
// the asserter has no way to remove one.
func TestRaisingLogsReportsWhatNothingDeclares(t *testing.T) {
l := &logs{on: []string{"LOG_mesh-issues_changes", "LOG_gone_old", "KV_not_a_log"}}
undeclared, err := RaiseLogs(l, []Log{{Module: "mesh-issues", Name: "changes"}, {Module: "a", Name: "b"}})
if err != nil {
t.Fatal(err)
}
if !slices.Equal(l.ensured, []string{"LOG_a_b", "LOG_mesh-issues_changes"}) {
t.Fatalf("asserted %v", l.ensured)
}
if !slices.Equal(undeclared, []string{"LOG_gone_old"}) {
t.Fatalf("reported %v", undeclared)
}
}
// **No path of the mesh deletes or purges a log or a bucket, or recreates one** (novox/hq ADR 0297 §2,
// ADR 0201 §15–16): no code of this repository outside its tests calls the client's stream or bucket
// delete, or purges a stream, but for the controller lease's own stream, which purges its own history
// below a sequence (internal/lease). A new call is a decision, not a refactor.
func TestNoPathDeletesALogOrABucket(t *testing.T) {
removers := map[string]bool{"DeleteStream": true, "DeleteKeyValue": true, "PurgeStream": true,
"DeleteObjectStore": true}
root := filepath.Join("..", "..")
var found []string
for _, dir := range []string{"cmd", "internal"} {
err := filepath.WalkDir(filepath.Join(root, dir), func(path string, e fs.DirEntry, err error) error {
if err != nil || e.IsDir() || !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") {
return err
}
f, err := parser.ParseFile(token.NewFileSet(), path, nil, 0)
if err != nil {
return err
}
ast.Inspect(f, func(n ast.Node) bool {
call, ok := n.(*ast.CallExpr)
if !ok {
return true
}
sel, ok := call.Fun.(*ast.SelectorExpr)
if !ok {
return true
}
name := sel.Sel.Name
if removers[name] || (name == "Purge" && !strings.Contains(filepath.ToSlash(path), "internal/lease/")) {
found = append(found, path+": "+name)
}
return true
})
for _, lit := range stringsIn(f) {
if strings.Contains(lit, "$JS.API.STREAM.DELETE") || strings.Contains(lit, "$JS.API.STREAM.PURGE") {
// Only the writers table names it, as a subject no one but the controller may publish.
if !strings.HasSuffix(filepath.ToSlash(path), "internal/broker/writers.go") {
found = append(found, path+": "+lit)
}
}
}
return nil
})
if err != nil {
t.Fatal(err)
}
}
if len(found) > 0 {
t.Fatalf("a path of the mesh removes a stream, a bucket or what one holds:\n %s", strings.Join(found, "\n "))
}
}
// stringsIn is every string literal of a file.
func stringsIn(f *ast.File) []string {
var out []string
ast.Inspect(f, func(n ast.Node) bool {
if lit, ok := n.(*ast.BasicLit); ok && lit.Kind == token.STRING {
out = append(out, lit.Value)
}
return true
})
return out
}
// Against a real server: a log is created as its configuration says, asserting it again keeps what
// it holds, a changed cap is brought to match in place, an entry cannot be deleted from it, and one
// nothing declares any more is reported and stays.
func TestALogIsAssertedInPlaceAndNeverRemoved(t *testing.T) {
bus := aLiveBus(t)
l := Log{Module: "logtest", Name: "changes", MaxMiB: 1}
if _, err := RaiseLogs(bus, []Log{l}); err != nil {
t.Fatalf("a real server refused a module's log: %v", err)
}
js, err := jetstream.New(bus.Conn())
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
ack, err := js.Publish(ctx, l.Subject()+".7", []byte(`{"op":"opened"}`))
if err != nil {
t.Fatalf("an append to the log was refused: %v", err)
}
stream, err := js.Stream(ctx, l.Stream())
if err != nil {
t.Fatal(err)
}
have := stream.CachedInfo().Config
want := l.Config()
if have.Storage != want.Storage || have.Retention != want.Retention || have.MaxAge != 0 ||
have.MaxMsgsPerSubject != -1 || have.MaxBytes != want.MaxBytes || have.MaxMsgSize != want.MaxMsgSize ||
have.Discard != jetstream.DiscardNew || !have.AllowDirect || !have.DenyDelete || !have.DenyPurge ||
have.Replicas != 1 || !slices.Equal(have.Subjects, want.Subjects) {
t.Fatalf("the server holds the log as %+v", have)
}
if err := stream.DeleteMsg(ctx, ack.Sequence); err == nil {
t.Fatal("an entry was deleted from a log")
}
l.MaxMiB = 2
if _, err := RaiseLogs(bus, []Log{l}); err != nil {
t.Fatalf("asserting the log again failed, so a restart would: %v", err)
}
info, err := stream.Info(ctx)
if err != nil {
t.Fatal(err)
}
if info.Config.MaxBytes != 2*1024*1024 {
t.Fatalf("the changed cap was not brought to match: %d", info.Config.MaxBytes)
}
if info.State.Msgs < 1 {
t.Fatal("asserting the log again lost what it held")
}
undeclared, err := RaiseLogs(bus, nil)
if err != nil {
t.Fatal(err)
}
if !slices.Contains(undeclared, l.Stream()) {
t.Fatalf("a log nothing declares was not reported: %v", undeclared)
}
if _, err := js.Stream(ctx, l.Stream()); err != nil {
t.Fatalf("a log nothing declares is gone: %v", err)
}
}
internal/broker/membership.go
-5
View File
@@ -51,10 +51,6 @@ type Membership struct {
// and refuses, with the reason, what is not on it — the bus enforces only the union over every
// module on the machine.
State []StateIssued `json:"state,omitempty"`
// Logs is every log this module's code may reach, by the name it uses for each (novox/hq ADR 0297):
// its own and no other's, since a log is its owner's alone. The runtime answers a bundle's log verbs
// from this list and refuses, with the reason, a log not on it.
Logs []LogIssued `json:"logs,omitempty"`
// SeatTraffic is what this module's code may submit, say, hear, take, ask, answer and read on seats
// that name their caller or their kind (novox/hq ADR 0259 §3). The runtime carrying the module
// publishes, takes and answers for it only what is listed here: the bus enforces only the union
@@ -125,7 +121,6 @@ func MembershipFor(node string, d Declared, where Placements) Membership {
}
}
m.State = stateIssuedFor(d, node)
m.Logs = logsIssuedFor(d)
t := SeatTrafficOf(d.Module, d.Holds, d.Uses, d.Watches)
for _, s := range append(append([]Seat{}, d.Uses...), d.Watches...) {
if !s.Kinded {
internal/broker/nats.go
+68 -47
View File
@@ -42,8 +42,64 @@ const (
// Its authority is the union of what the modules it carries would each have had for their
// tools — and nothing of what they consume, because tools are what it runs, not reactions.
KindNodeTools Kind = "node-tools"
// KindView is the one read-only principal a view onto the bus connects as (novox/hq research 036,
// gap G1): a page in a browser, over the bus module's WebSocket listener, watching the issue tracker.
// Fixed, and derived from no declaration: what it hears is ViewHears, what it reads is ViewBucket,
// and it publishes nothing but direct reads of that one bucket (ViewReads), each answered in its own
// inbox. Composed like every other user, into the same
// file, once its credential is minted (`bus view-credential`); forgotten like every other user
// (`bus view-revoke`), at the next composition.
KindView Kind = "view"
)
// ViewUser is the view's one username: there is one view, and it is nobody's machine or module.
const ViewUser = "view"
// ViewBucket is the state the view reads: the issue tracker's issues, as the bus names the bucket
// (mesh-issues's state `issues`, novox/hq ADR 0201).
var ViewBucket = BucketName("mesh-issues", "issues")
// ViewHears are the events the view subscribes, each named: the issue tracker's own, the controller's
// walks and conditions, and the delivery owner's — what a page about issues shows beside them. Subscribe
// only, and no stream or consumer of its own: a page hears what happens while it is open, and reads the
// bucket for everything before.
var ViewHears = []string{
moduleEventSubject("mesh-issues", "opened"),
moduleEventSubject("mesh-issues", "moved"),
moduleEventSubject("mesh-issues", "noted"),
moduleEventSubject("mesh-issues", "linked"),
seatEventSubject(ControllerSeat, "plan-moved"),
seatEventSubject(ControllerSeat, "condition-raised"),
seatEventSubject(ControllerSeat, "condition-changed"),
seatEventSubject(ControllerSeat, "condition-cleared"),
moduleEventSubject("mesh-delivery", "transition"),
moduleEventSubject("mesh-delivery", "group"),
}
// ViewReads are the JetStream API requests the view makes, on ViewBucket's stream and no other: binding
// (STREAM.INFO), and direct reads — one key by its subject (`DIRECT.GET.<stream>.$KV.<bucket>.<key>`), and
// the batch form on the bare subject, which answers the newest value of every key (`multi_last`) into the
// asker's inbox. The page lists the bucket with the batch, and re-reads one key when the tracker's event
// names it (every event carries the issue's `number`).
//
// **No consumer, deliberately, and so no watch.** A KV watch is a push consumer, and a push consumer's
// deliver subject is the creator's choice, delivered by the server's own client — which the server does
// not hold to the creator's permissions. Measured on 2.11.17 (2026-10-10): the view, granted
// CONSUMER.CREATE on this stream, made a consumer delivering to `mesh.mod.mesh-issues.event.opened`, and
// a module subscribed there received the bucket's entry as the tracker's event. A grant of CONSUMER.CREATE
// is a publish to any subject in the account; the view publishes nothing, so it has none (nor
// CONSUMER.DELETE, which would let it delete a module's consumer). Nothing here is a write either: no
// `$KV.<bucket>.>`, which is what a put or a delete publishes to, and no STREAM.* that defines, purges or
// deletes.
func ViewReads() []string {
stream := "KV_" + ViewBucket
return []string{
"$JS.API.STREAM.INFO." + stream,
"$JS.API.DIRECT.GET." + stream,
"$JS.API.DIRECT.GET." + stream + ".>",
}
}
// RuntimeModule is the module that IS the node's tool runtime (novox/hq ADR 0175). Where it is
// assigned, the mesh composes one runtime principal for the machine in place of that module's own,
// and the per-module containers that served tools until then stop being the way tools reach a node.
@@ -132,8 +188,6 @@ type Principal struct {
// no other; KeyedReads the keys of others' state it reads one key at a time (novox/hq ADR 0260).
PerMachine []string
KeyedReads []KeyedRead
// Logs is the local names of the logs this principal's module keeps (novox/hq ADR 0297).
Logs []string
// SnapshotsTheBus is the bus's own module, the one holding mesh-broker (novox/hq ADR 0235). Its
// whole authority is BusSnapshotGrants: it copies the streams for the night's backup and nothing
@@ -165,21 +219,6 @@ type Principal struct {
// goes with the retired seat row.
var seatsTheControllerAsks = []string{"node-build-agent", "mesh-build-machine"}
// TheControllersAsk says whether a message of stream, published on subject, is an ask the controller
// itself makes: one on the accept subject of a seat in seatsTheControllerAsks, kept in that seat's own
// work queue. Such an ask, given up on by the seat's worker, can be delivered again with the authority
// the controller already holds — the publish on that seat's accepts and the stream API to remove the
// original — and no other can (novox/hq issue 334, ADR 0264's consequences: no grant over the seats'
// queues).
func TheControllersAsk(stream, subject string) bool {
for _, seat := range seatsTheControllerAsks {
if stream == seatStreamName(seat) && strings.HasPrefix(subject, "mesh.seat."+seat+".accept.") {
return true
}
}
return false
}
// SeatVerb is one verb of one seat, on every machine holding it.
type SeatVerb struct{ Seat, Verb string }
@@ -214,10 +253,6 @@ func ControllerOnly() []string {
for _, base := range []string{"mesh.seat." + v.Seat + ".tool." + v.Verb, "mesh.mod.*.tool." + v.Seat + "." + v.Verb} {
out = append(out, base, base+".*")
}
// And where a call names its caller (novox/hq issue 365): any machine, any caller.
for _, base := range []string{"mesh.seat." + v.Seat + "." + CallKind + "." + v.Verb, "mesh.mod.*." + CallKind + "." + v.Seat + "." + v.Verb} {
out = append(out, base+".>")
}
}
return out
}
@@ -253,11 +288,8 @@ func MaySubscribe(perms Permissions, subject string) bool {
//
// And, since novox/hq ADR 0259, `release` and `stop`: the controller asks the operator for them about a
// delivery held past its bound, and calls them on the operator's warrant, with its why.
//
// And, since novox/hq ADR 0282, `times`: the self-check reads the delivery times to say a delivery over its budget.
var VerbsTheControllerAsksTheDeliveryOwner = []SeatVerb{{Seat: "mesh-delivery", Verb: "stalled"},
{Seat: "mesh-delivery", Verb: "close"}, {Seat: "mesh-delivery", Verb: "release"}, {Seat: "mesh-delivery", Verb: "stop"},
{Seat: "mesh-delivery", Verb: "times"}}
{Seat: "mesh-delivery", Verb: "close"}, {Seat: "mesh-delivery", Verb: "release"}, {Seat: "mesh-delivery", Verb: "stop"}}
// VerbsTheControllerActsOnAWarrant are the other seat verbs the controller calls when the operator's warrant
// chooses them (novox/hq ADR 0259): a machine's service restarted, and a walk started or stopped through the
@@ -285,6 +317,8 @@ func (p Principal) Username() string {
switch p.Kind {
case KindPerson:
return "person." + p.Module
case KindView:
return ViewUser
case KindModule, KindNodeTools:
// The runtime is named exactly as the module it stands for would have been: the mesh
// issues its credential through the same path a module's takes (`module issue`), and
@@ -557,6 +591,14 @@ func PermissionsFor(p Principal) (Permissions, error) {
// itself, its replies to the asker's own inbox.
pub = append(pub, discovering()...)
case KindView:
// Hears what it is for and reads one bucket, and nothing else (ViewHears, ViewReads): no tool,
// no event of its own, no stream, no bucket written. Its requests are answered in its own
// inbox, granted below with the person's; a reply to anything is never permitted, because
// nothing is ever asked of it.
sub = append(sub, ViewHears...)
pub = append(pub, ViewReads()...)
case KindEnrolment:
// A leaked token is useless for anything but enrolling: it cannot read a declaration, hear
// an event, or subscribe any inbox but the one its own token derives (design 25 §6).
@@ -748,8 +790,6 @@ func PermissionsFor(p Principal) (Permissions, error) {
// watched, its own written too.
pub = append(pub, stateGrants(stateAccess{Module: p.Module, Node: p.Node, Keeps: p.State,
PerMachine: p.PerMachine, Reads: p.Reads, KeyedReads: p.KeyedReads})...)
// And its logs (novox/hq ADR 0297): appended to and read directly, its own alone.
pub = append(pub, logGrants(p.Module, p.Logs)...)
// 6. Its traffic on seats that name their caller or their kind, ask proofs or keep records
// (novox/hq ADR 0259 §3).
@@ -837,12 +877,6 @@ func PermissionsFor(p Principal) (Permissions, error) {
pub = append(pub, stateGrants(stateAccess{Module: d.Module, Node: p.Node, Keeps: stateNames(d.State),
PerMachine: perMachineNames(d.State), Reads: d.Reads, KeyedReads: d.KeyedReads})...)
}
// **And it keeps the logs of the modules it carries** (novox/hq ADR 0297): each module's own, the
// union over them. That one module's code does not append to another's log through it is the
// runtime's to keep, from the logs each membership lists.
for _, d := range p.Carries {
pub = append(pub, logGrants(d.Module, logNames(d.Logs))...)
}
// **Never the traffic of a trusted holder** (novox/hq ADR 0259 §8): the machine's runtime runs as the
// operator's account, which every agent runs as, so a module saying warrants or speaking for a kind
// that proves its sender is never composed into it — refused here, naming it, whatever registration
@@ -866,7 +900,7 @@ func PermissionsFor(p Principal) (Permissions, error) {
pub = unique(pub)
}
if p.Kind == KindPerson {
if p.Kind == KindPerson || p.Kind == KindView {
// An inbox to hear answers in, and nothing else. No ack subject: a person has no durable
// consumer, because nothing is delivered to a person — they ask and are answered.
sub = append(sub, p.inbox())
@@ -889,19 +923,6 @@ func PermissionsFor(p Principal) (Permissions, error) {
pub = append(pub, "$JS.ACK."+consumerStream(p)+"."+consumerDurable(p)+".>")
}
// **And every tool grant again, naming its caller** (novox/hq issue 365): a call in this principal's own
// name, and an answer to a call naming anybody. The `tool` grants above stay for one release beside these,
// so callers and runtimes move without a gap; their retirement is hq issue 464.
//
// **Not the controller's, this release.** Its grants are also the installer's first user list
// (TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose), judged against the node-engine the mesh
// runs, so a change to them waits on a node-engine delivery. It calls and serves on the subjects that name
// no caller meanwhile — a caller-named call to its seat reaches nobody and is asked again on those at once —
// and moves with issue 464.
if p.Kind != KindController {
pub, sub = callerNamed(p.Username(), pub, sub)
}
sort.Strings(pub)
sort.Strings(sub)
// One writer per piece of state (novox/hq to-be 45 §1): a grant that would make a second is
internal/broker/nats_golden_test.go
+2
View File
@@ -31,6 +31,8 @@ func TestTheComposedConfigMatchesTheGolden(t *testing.T) {
// The bus's own module: the snapshot API and its inbox, nothing else (novox/hq ADR 0235).
{Kind: KindModule, Node: "one", Module: "nats", SnapshotsTheBus: true,
Serves: []string{"nats_streams"}, PasswordHash: "$2a$11$bbbbbbbbbbbbbbbbbbbbbb"},
// The view: hears the issue tracker and reads its bucket, writes nothing (research 036).
{Kind: KindView, PasswordHash: "$2a$11$vvvvvvvvvvvvvvvvvvvvvv"},
})
if err != nil {
t.Fatal(err)
internal/broker/nats_test.go
+3 -3
View File
@@ -240,9 +240,9 @@ func TestAPersonReachesNothingButTools(t *testing.T) {
perms, _ := PermissionsFor(Principal{Kind: KindPerson, Module: "jo",
Invokes: []string{"*"}, PasswordHash: "x"})
for _, p := range perms.Publish {
// A tool call — named by its caller or not (novox/hq issue 365) — or asking what answers (novox/hq
// ADR 0197), a question every service answers about itself, which claims nothing and controls nothing.
if !strings.Contains(p, ".tool.") && !strings.Contains(p, ".call.") && !strings.HasPrefix(p, "$SRV.") {
// A tool call, or asking what answers (novox/hq ADR 0197) — a question every service
// answers about itself, which claims nothing and controls nothing.
if !strings.Contains(p, ".tool.") && !strings.HasPrefix(p, "$SRV.") {
t.Errorf("a person may publish %q, which is not a tool call", p)
}
}
internal/broker/testdata/composed.conf
Vendored
+8 -4
View File
@@ -24,7 +24,7 @@ accounts {
jetstream: enabled
users = [
{ user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.DEAD_LETTER_NOTICES.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>", "$KV.SEAT_NODE_BUILD_AGENT_cancelled.>", "$KV.mesh-controller_asked.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.again.>", "mesh.assignment.>", "mesh.events.dead.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-build-machine.tool.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.checked", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.plan-moved", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.rolled-back", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.mesh-controller.tool.plans", "mesh.seat.mesh-delivery.tool.close", "mesh.seat.mesh-delivery.tool.release", "mesh.seat.mesh-delivery.tool.stalled", "mesh.seat.mesh-delivery.tool.stop", "mesh.seat.mesh-delivery.tool.times", "mesh.seat.node-backup.tool.backed-up.*", "mesh.seat.node-backup.tool.now.*", "mesh.seat.node-build-agent.accept.>", "mesh.seat.node-build-agent.tool.>", "mesh.seat.node-intrusion-prevention.tool.banned.*", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-service-manager.tool.restart.*"] }
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.DEAD_LETTER_NOTICES.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>", "$KV.SEAT_NODE_BUILD_AGENT_cancelled.>", "$KV.mesh-controller_asked.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.again.>", "mesh.assignment.>", "mesh.events.dead.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-build-machine.tool.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.checked", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.plan-moved", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.rolled-back", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.mesh-controller.tool.plans", "mesh.seat.mesh-delivery.tool.close", "mesh.seat.mesh-delivery.tool.release", "mesh.seat.mesh-delivery.tool.stalled", "mesh.seat.mesh-delivery.tool.stop", "mesh.seat.node-backup.tool.backed-up.*", "mesh.seat.node-backup.tool.now.*", "mesh.seat.node-build-agent.accept.>", "mesh.seat.node-build-agent.tool.>", "mesh.seat.node-intrusion-prevention.tool.banned.*", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-service-manager.tool.restart.*"] }
subscribe: { allow: ["$JS.API.>", "$JS.EVENT.ADVISORY.CONSUMER.DELETED.>", "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "$SRV.STATS", "$SRV.STATS.mesh-controller", "$SRV.STATS.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered", "mesh.mod.*.event.provisioner.retirement", "mesh.mod.gitea.event.pull.merged", "mesh.mod.gitea.event.pull.updated", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built", "mesh.seat.operator-channel.event.decided.mesh-controller"] }
allow_responses: { max: 1, ttl: "1m" }
} }
@@ -43,19 +43,23 @@ accounts {
} }
{ user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "$JS.API.CONSUMER.MSG.NEXT.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.one.telegram", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.telegram", "$SRV.INFO.telegram.>", "$SRV.PING", "$SRV.PING.telegram", "$SRV.PING.telegram.>", "$SRV.STATS", "$SRV.STATS.telegram", "$SRV.STATS.telegram.>", "_INBOX.one.telegram.>", "mesh.assignment.one.telegram", "mesh.mod.telegram.call.>", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.telegram", "$SRV.INFO.telegram.>", "$SRV.PING", "$SRV.PING.telegram", "$SRV.PING.telegram.>", "$SRV.STATS", "$SRV.STATS.telegram", "$SRV.STATS.telegram.>", "_INBOX.one.telegram.>", "mesh.assignment.one.telegram", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.audit", password: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_audit.>", "$JS.API.CONSUMER.INFO.EVENTS.two_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.two.audit"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.audit", "$SRV.INFO.audit.>", "$SRV.PING", "$SRV.PING.audit", "$SRV.PING.audit.>", "$SRV.STATS", "$SRV.STATS.audit", "$SRV.STATS.audit.>", "_INBOX.two.audit.>", "mesh.assignment.two.audit", "mesh.mod.audit.call.>", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.audit", "$SRV.INFO.audit.>", "$SRV.PING", "$SRV.PING.audit", "$SRV.PING.audit.>", "$SRV.STATS", "$SRV.STATS.audit", "$SRV.STATS.audit.>", "_INBOX.two.audit.>", "mesh.assignment.two.audit", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.shop", password: "$2a$11$ssssssssssssssssssssss", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "$JS.API.CONSUMER.INFO.EVENTS.two_shop", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.two.shop", "mesh.mod.shop.event.order.placed", "mesh.seat.telegram-sender.accept.send"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.shop", "$SRV.INFO.shop.>", "$SRV.PING", "$SRV.PING.shop", "$SRV.PING.shop.>", "$SRV.STATS", "$SRV.STATS.shop", "$SRV.STATS.shop.>", "_INBOX.two.shop.>", "mesh.assignment.two.shop", "mesh.mod.shop.call.>", "mesh.mod.shop.tool.>"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.shop", "$SRV.INFO.shop.>", "$SRV.PING", "$SRV.PING.shop", "$SRV.PING.shop.>", "$SRV.STATS", "$SRV.STATS.shop", "$SRV.STATS.shop.>", "_INBOX.two.shop.>", "mesh.assignment.two.shop", "mesh.mod.shop.tool.>"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "view", password: "$2a$11$vvvvvvvvvvvvvvvvvvvvvv", permissions: {
publish: { allow: ["$JS.API.DIRECT.GET.KV_mesh-issues_issues", "$JS.API.DIRECT.GET.KV_mesh-issues_issues.>", "$JS.API.STREAM.INFO.KV_mesh-issues_issues"] }
subscribe: { allow: ["_INBOX.view.>", "mesh.mod.mesh-delivery.event.group", "mesh.mod.mesh-delivery.event.transition", "mesh.mod.mesh-issues.event.linked", "mesh.mod.mesh-issues.event.moved", "mesh.mod.mesh-issues.event.noted", "mesh.mod.mesh-issues.event.opened", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.plan-moved"] }
} }
]
}
}
internal/broker/users.go
+6 -3
View File
@@ -35,8 +35,6 @@ type Declared struct {
Invokes []string
// State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201).
State []Bucket
// Logs are the logs it keeps, each a stream its instances append to (novox/hq ADR 0297).
Logs []Log
// Reads are other modules' state it reads, each `<module>.<name>` (novox/hq ADR 0201).
Reads []string
// KeyedReads are keys of other modules' state it reads, each one key alone: what a seat's holder is
@@ -69,6 +67,9 @@ type Records struct {
Enrolling []string
// People is each person's name against the tools they may invoke, `*` for an administrator.
People map[string][]string
// View says the mesh minted the view's credential (`bus view-credential`), so the one read-only
// view principal is composed (KindView); forgotten, it is left out, like a person.
View bool
// Interchangeable is each module whose definition says its instances are the same anywhere
// (ADR 0160), which decides whether the module's plain subject is issued to every instance.
Interchangeable map[string]bool
@@ -126,7 +127,6 @@ func Users(r Records) ([]Principal, error) {
Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Invokes: d.Invokes,
State: stateNames(d.State), Reads: d.Reads, SnapshotsTheBus: d.SnapshotsTheBus,
PerMachine: perMachineNames(d.State), KeyedReads: d.KeyedReads,
Logs: logNames(d.Logs),
})
}
if runtimeHere {
@@ -145,6 +145,9 @@ func Users(r Records) ([]Principal, error) {
for _, person := range sortedNames(r.People) {
out = append(out, Principal{Kind: KindPerson, Module: person, Invokes: r.People[person]})
}
if r.View {
out = append(out, Principal{Kind: KindView})
}
// Refused here rather than discovered by the server. Two users with one name is a file the
// server reads as one of them, and which one depends on the order — so a module assigned to a
internal/broker/view_live_test.go
+230
View File
@@ -0,0 +1,230 @@
package broker
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/nats-io/nats-server/v2/server"
"github.com/nats-io/nats.go"
"golang.org/x/crypto/bcrypt"
)
// The view against a real server, over WebSocket (novox/hq research 036): the composed user list is
// what the server reads, the listener is the shape the bus module declares (no TLS, compression on,
// reached across the overlay only), and the view with its credential binds the issue tracker's bucket,
// lists it with one batch read, follows a tracker event to re-read the key it names — and is refused
// every write: a put, a delete, an event, and a consumer delivering onto the tracker's event subject.
//
// go test ./internal/broker/ -run TestTheView
func TestTheViewReadsTheIssuesOverWebSocketAndWritesNothing(t *testing.T) {
hash := func(password string) string {
h, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.MinCost)
if err != nil {
t.Fatal(err)
}
return string(h)
}
const node = "anchor"
tracker := Principal{Kind: KindModule, Node: node, Module: "mesh-issues", State: []string{"issues"},
Emits: []string{"opened", "moved"}, PasswordHash: hash("tracker")}
accounts, err := ComposeAccounts([]Principal{
{Kind: KindController, PasswordHash: hash("controller")},
tracker,
{Kind: KindView, PasswordHash: hash("view")},
})
if err != nil {
t.Fatal(err)
}
conf := filepath.Join(t.TempDir(), "accounts.conf")
if err := os.WriteFile(conf, []byte(accounts), 0o600); err != nil {
t.Fatal(err)
}
// The server reads the composed file as the bus does — through its own parser — and listens as the
// bus module's configuration says: a WebSocket listener without TLS and with compression, beside
// the client port. Ports chosen by the system, so this runs beside a live bus.
opts, err := server.ProcessConfigFile(conf)
if err != nil {
t.Fatalf("the server refused the composed user list: %v", err)
}
opts.Host, opts.Port = "127.0.0.1", server.RANDOM_PORT
opts.JetStream, opts.StoreDir = true, t.TempDir()
opts.NoLog, opts.NoSigs = true, true
opts.Websocket = server.WebsocketOpts{Host: "127.0.0.1", Port: server.RANDOM_PORT, NoTLS: true, Compression: true}
s, err := server.NewServer(opts)
if err != nil {
t.Fatal(err)
}
go s.Start()
if !s.ReadyForConnections(30 * time.Second) {
s.Shutdown()
t.Fatal("the server did not come up")
}
t.Cleanup(func() { s.Shutdown(); s.WaitForShutdown() })
dial := func(url, user, password string, refused chan<- string) *nats.Conn {
t.Helper()
nc, err := nats.Connect(url, nats.UserInfo(user, password), nats.CustomInboxPrefix("_INBOX."+user),
nats.Compression(true), nats.ErrorHandler(func(_ *nats.Conn, _ *nats.Subscription, err error) {
if refused != nil && errors.Is(err, nats.ErrPermissionViolation) {
refused <- err.Error()
}
}))
if err != nil {
t.Fatalf("%s could not connect to %s: %v", user, url, err)
}
t.Cleanup(nc.Close)
return nc
}
// The controller defines the bucket, as it does for every module's state; the tracker writes it.
controller := dial(s.ClientURL(), "controller", "controller", nil)
cjs, _ := controller.JetStream()
if _, err := cjs.CreateKeyValue(&nats.KeyValueConfig{Bucket: ViewBucket, History: 8}); err != nil {
t.Fatalf("the controller could not define %s: %v", ViewBucket, err)
}
trackerConn := dial(s.ClientURL(), tracker.Username(), "tracker", nil)
tjs, _ := trackerConn.JetStream()
tkv, err := tjs.KeyValue(ViewBucket)
if err != nil {
t.Fatal(err)
}
if _, err := tkv.Put("365", []byte(`{"number":365,"status":"open"}`)); err != nil {
t.Fatalf("the tracker could not write its own bucket: %v", err)
}
// The view, over WebSocket with its credential.
refused := make(chan string, 8)
view := dial(s.WebsocketURL(), ViewUser, "view", refused)
if !strings.HasPrefix(view.ConnectedUrl(), "ws://") {
t.Fatalf("the view is connected to %s, not over WebSocket", view.ConnectedUrl())
}
vjs, _ := view.JetStream(nats.MaxWait(3 * time.Second))
vkv, err := vjs.KeyValue(ViewBucket)
if err != nil {
t.Fatalf("the view could not bind %s: %v", ViewBucket, err)
}
if got, err := vkv.Get("365"); err != nil {
t.Fatalf("the view could not read a key: %v", err)
} else if !strings.Contains(string(got.Value()), `"number":365`) {
t.Fatalf("the view read %q", got.Value())
}
if _, err := tkv.Put("366", []byte(`{"number":366,"status":"open"}`)); err != nil {
t.Fatal(err)
}
// The list: one batch read, the newest value of every key, into the view's own inbox, ended by the
// server's end-of-batch status (204).
listed := map[string]string{}
inbox := view.NewRespInbox()
batch, err := view.SubscribeSync(inbox)
if err != nil {
t.Fatal(err)
}
if err := view.PublishRequest("$JS.API.DIRECT.GET.KV_"+ViewBucket, inbox,
[]byte(`{"multi_last":["$KV.`+ViewBucket+`.>"]}`)); err != nil {
t.Fatal(err)
}
for {
m, err := batch.NextMsg(5 * time.Second)
if err != nil {
t.Fatalf("the view's batch read ended without its end-of-batch (%d keys so far): %v", len(listed), err)
}
if m.Header.Get("Status") == "204" {
break
}
if status := m.Header.Get("Status"); status != "" {
t.Fatalf("the batch read answered %s %s", status, m.Header.Get("Description"))
}
listed[m.Header.Get("Nats-Subject")] = string(m.Data)
}
_ = batch.Unsubscribe()
for _, key := range []string{"365", "366"} {
if !strings.Contains(listed["$KV."+ViewBucket+"."+key], `"number":`+key) {
t.Errorf("the batch read did not list %s: %v", key, listed)
}
}
// A change followed: the tracker moves 365 and says so; the view hears the event and reads the key
// it names.
moved, err := view.SubscribeSync("mesh.mod.mesh-issues.event.moved")
if err != nil {
t.Fatal(err)
}
_ = view.Flush()
if _, err := tkv.Put("365", []byte(`{"number":365,"status":"located"}`)); err != nil {
t.Fatal(err)
}
if err := trackerConn.Publish("mesh.mod.mesh-issues.event.moved", []byte(`{"number":365,"to":"located"}`)); err != nil {
t.Fatal(err)
}
if _, err := moved.NextMsg(5 * time.Second); err != nil {
t.Fatalf("the view did not hear the tracker's event: %v", err)
}
if got, err := vkv.Get("365"); err != nil || !strings.Contains(string(got.Value()), "located") {
t.Fatalf("after the event the view read %v (%v)", got, err)
}
// **The hole a watch would open, shut** (ViewReads): a consumer delivering onto the tracker's event
// subject would have the server republish the bucket there, as the tracker. Refused, and nothing
// reaches a module listening on that subject.
listener, err := trackerConn.SubscribeSync("mesh.mod.mesh-issues.event.opened")
if err != nil {
t.Fatal(err)
}
_ = trackerConn.Flush()
if _, err := vjs.AddConsumer("KV_"+ViewBucket, &nats.ConsumerConfig{Name: "w", DeliverSubject: "mesh.mod.mesh-issues.event.opened",
AckPolicy: nats.AckNonePolicy, FilterSubject: "$KV." + ViewBucket + ".>"}); err == nil {
t.Error("the view made a consumer")
}
if _, err := vkv.WatchAll(); err == nil {
t.Error("the view made a watch, which is a consumer")
}
if m, err := listener.NextMsg(2 * time.Second); err == nil {
t.Errorf("a message reached the tracker's event subject from the view: %q", m.Data)
}
// The refusals of the consumer create land as permission violations too; drained before the writes.
drain := time.After(500 * time.Millisecond)
for draining := true; draining; {
select {
case <-refused:
case <-drain:
draining = false
}
}
// And every write is refused: the server says so, and the bucket is unchanged.
if _, err := vkv.Put("367", []byte(`{"number":367}`)); err == nil {
t.Error("the view put a key")
}
if err := vkv.Delete("365"); err == nil {
t.Error("the view deleted a key")
}
if err := view.Publish("mesh.mod.mesh-issues.event.opened", []byte(`{"number":367}`)); err != nil {
t.Fatal(err)
}
_ = view.Flush()
violations := map[string]bool{}
deadline := time.After(10 * time.Second)
for len(violations) < 3 {
select {
case v := <-refused:
for _, subject := range []string{"$KV." + ViewBucket + ".367", "$KV." + ViewBucket + ".365", "mesh.mod.mesh-issues.event.opened"} {
if strings.Contains(v, subject) {
violations[subject] = true
}
}
case <-deadline:
t.Fatalf("the server refused %d of the view's 3 writes as permission violations", len(violations))
}
}
if _, err := tkv.Get("367"); !errors.Is(err, nats.ErrKeyNotFound) {
t.Errorf("after the view's put, 367 is %v", err)
}
if _, err := tkv.Get("365"); err != nil {
t.Errorf("after the view's delete, 365 is gone: %v", err)
}
}
internal/broker/view_test.go
+144
View File
@@ -0,0 +1,144 @@
package broker
import (
"reflect"
"sort"
"testing"
)
// The view (novox/hq research 036): one read-only user, composed like every other, whose whole
// authority is a list here — so a grant that is not on the list fails a test, not a review.
// Exactly what it hears, exactly what it asks, and nothing it could write or answer. A mutation that
// adds a publish grant — `$KV.<bucket>.>`, an event, a tool — fails here.
func TestTheViewHearsAndReadsAndCanPublishNothingElse(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindView})
if err != nil {
t.Fatal(err)
}
wantSub := append(append([]string(nil), ViewHears...), "_INBOX.view.>")
sort.Strings(wantSub)
if !reflect.DeepEqual(perms.Subscribe, wantSub) {
t.Errorf("the view subscribes\n %v\nand should subscribe exactly\n %v", perms.Subscribe, wantSub)
}
wantPub := ViewReads()
sort.Strings(wantPub)
if !reflect.DeepEqual(perms.Publish, wantPub) {
t.Errorf("the view publishes\n %v\nand should publish exactly\n %v", perms.Publish, wantPub)
}
if len(perms.PublishDeny) != 0 {
t.Errorf("the view needs no deny, because nothing it may publish reaches the controller's own: %v", perms.PublishDeny)
}
if perms.AllowResponses {
t.Error("the view may answer, and nothing is ever asked of it")
}
// Every publish grant is binding the one bucket's stream or reading it directly. **The mutation this
// holds against**: a write grant of any shape, and a consumer of any shape — a consumer's deliver
// subject is the creator's choice, so creating one is publishing anywhere (ViewReads).
stream := "KV_" + ViewBucket
for _, p := range perms.Publish {
readOnly := p == "$JS.API.STREAM.INFO."+stream ||
p == "$JS.API.DIRECT.GET."+stream ||
p == "$JS.API.DIRECT.GET."+stream+".>"
if !readOnly {
t.Errorf("the view is granted a publish on %q, which is not a read of %s", p, ViewBucket)
}
}
for _, refused := range []string{
"$KV." + ViewBucket + ".365", // a put or a delete
"$KV.mesh-controller_conditions.x", // another bucket
"$JS.API.STREAM.CREATE." + stream, // defining the stream
"$JS.API.STREAM.PURGE." + stream, // emptying it
"$JS.API.STREAM.DELETE." + stream, // deleting it
"$JS.API.STREAM.MSG.DELETE." + stream, // deleting a message
"$JS.API.CONSUMER.CREATE.KV_mesh-controller_conditions.x", // reading another bucket
"$JS.API.CONSUMER.CREATE." + stream + ".w.$KV." + ViewBucket + ".>", // a watch: delivers anywhere
"$JS.API.CONSUMER.CREATE." + stream, // an unnamed consumer
"$JS.API.CONSUMER.DELETE." + stream + ".a_mesh-issues", // a module's consumer
"$JS.FC." + stream + ".x",
"$JS.API.DIRECT.GET.KV_mesh-controller_conditions", // another bucket, directly
"$JS.API.STREAM.INFO.EVENTS", // the events stream
"$JS.API.INFO", // the account
"mesh.mod.mesh-issues.event.opened", // claiming the tracker said something
"mesh.mod.mesh-issues.tool.open", // opening an issue
"mesh.seat.issue-tracker.tool.open", // through the seat
"mesh.seat.issue-tracker.tool.open.novox", // on one machine
"mesh.seat.mesh-controller.tool.status", // the controller's verbs
"mesh.seat.mesh-controller.event.plan-moved",
"$SRV.PING",
"_INBOX.controller.x",
} {
if MayPublish(perms, refused) {
t.Errorf("the view may publish %q", refused)
}
}
for _, refused := range []string{
"mesh.mod.mesh-issues.tool.open", // a tool asked of the tracker
"mesh.mod.telegram.event.received", // another module's events
"mesh.seat.mesh-controller.event.applied",
"mesh.control.novox.report",
"_INBOX.controller.x",
"_INBOX.person.jochen.x",
"_DELIVER.controller.EVENTS",
} {
if MaySubscribe(perms, refused) {
t.Errorf("the view may subscribe %q", refused)
}
}
for _, heard := range []string{
"mesh.mod.mesh-issues.event.opened",
"mesh.mod.mesh-issues.event.moved",
"mesh.mod.mesh-issues.event.noted",
"mesh.mod.mesh-issues.event.linked",
"mesh.seat.mesh-controller.event.plan-moved",
"mesh.seat.mesh-controller.event.condition-raised",
"mesh.seat.mesh-controller.event.condition-changed",
"mesh.seat.mesh-controller.event.condition-cleared",
"mesh.mod.mesh-delivery.event.transition",
"mesh.mod.mesh-delivery.event.group",
"_INBOX.view.abc",
} {
if !MaySubscribe(perms, heard) {
t.Errorf("the view cannot subscribe %q", heard)
}
}
}
// Composed once the mesh minted its credential, and not before: its row is the whole record of it.
func TestTheViewIsComposedOnlyOnceItsCredentialIsMinted(t *testing.T) {
without, err := Users(Records{Nodes: []string{"anchor"}})
if err != nil {
t.Fatal(err)
}
for _, p := range without {
if p.Kind == KindView {
t.Fatal("the view is composed before its credential was minted")
}
}
with, err := Users(Records{Nodes: []string{"anchor"}, View: true})
if err != nil {
t.Fatal(err)
}
views := 0
for _, p := range with {
if p.Kind == KindView {
views++
if p.Username() != ViewUser {
t.Errorf("the view is called %q, and its row is %q", p.Username(), ViewUser)
}
}
}
if views != 1 {
t.Fatalf("%d view users composed; there is one view", views)
}
// And without its hash it is named as missing, like any user — never written as a user anybody is.
_, missing := WithPasswords(with, map[string]string{})
found := false
for _, m := range missing {
found = found || m == ViewUser
}
if !found {
t.Error("a view with no password was not named as missing one")
}
}
internal/builder/builder.go
+6 -107
View File
@@ -7,10 +7,8 @@ import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
"os"
"os/exec"
"path"
@@ -161,18 +159,17 @@ func build(ctx context.Context, run Runner, publish Publisher,
}
// The credential is a file git reads, never an argument: a URL carrying a password in argv
// would be readable by anything that can list processes for as long as a clone runs.
credentials, forget, err := storeCredential(workspace, forge)
if err != nil {
return Result{}, err
credentials := ""
if forge.URL != "" {
credentials = filepath.Join(workspace, "git-credentials")
if err := os.WriteFile(credentials, []byte(forge.URL+"\n"), 0o600); err != nil {
return Result{}, err
}
}
defer forget()
tree := filepath.Join(workspace, "source")
if err := os.RemoveAll(tree); err != nil {
return Result{}, err
}
// Removed when the build ends, whatever happens: the tree holds the .npmrc a build may be handed, in the
// workspace a later check's container mounts as its HOME (novox/hq issue 462).
defer os.RemoveAll(tree)
// A fresh clone every time rather than a fetch into a tree that is already there. A build
// that reuses a working tree can succeed because of something a previous build left behind,
// and that is a build nobody can reproduce.
@@ -180,9 +177,6 @@ func build(ctx context.Context, run Runner, publish Publisher,
say("clone", "FAILED: %v", err)
return Result{}, fmt.Errorf("cannot clone %s: %w", repository, err)
}
if err := recordedWithoutUserinfo(ctx, run, tree, repository); err != nil {
return Result{}, err
}
say("clone", "done")
if ref != "" {
if _, err := run(ctx, tree, "git", "checkout", "--quiet", ref); err != nil {
@@ -254,8 +248,6 @@ func build(ctx context.Context, run Runner, publish Publisher,
if err := os.WriteFile(npmrcPath, []byte(content), 0o600); err != nil {
return Result{}, fmt.Errorf("cannot write the package-registry credential for the build: %w", err)
}
// Only for as long as the build: a later check's container mounts this workspace (novox/hq issue 462).
defer os.Remove(npmrcPath)
say("packages", "resolving %s from the mesh's package registry", npmrc.Scope)
src.notPinned("it resolves packages from the mesh's registry at build time")
}
@@ -449,9 +441,6 @@ func contextFrom(ctx context.Context, run Runner, workspace, artifact, credentia
if _, err := run(ctx, workspace, "git", cloneWith(credentials, "clone", "--quiet", url, dir)...); err != nil {
return "", fmt.Errorf("cannot clone %s: %w", url, err)
}
if err := recordedWithoutUserinfo(ctx, run, dir, url); err != nil {
return "", err
}
if from.Ref != "" {
if _, err := run(ctx, dir, "git", "checkout", "--quiet", from.Ref); err != nil {
return "", fmt.Errorf("%s has no %s: %w", from.Repository, from.Ref, err)
@@ -478,96 +467,6 @@ func contextURL(from catalogue.ArtifactContext, seats map[string]string) (string
return strings.TrimRight(base, "/") + "/" + strings.TrimSuffix(strings.Trim(from.Repository, "/"), ".git") + ".git", nil
}
// storeCredential writes the forge credential where git's credential store reads it, and the function that
// removes it again; "" and nothing to remove when the builder holds none.
//
// **Never inside the workspace** (novox/hq issue 462): a merge check's toolchain container mounts the
// workspace as its HOME, so a credential kept there — even one a build left behind — is readable by any pull
// request's merge-check.sh, and what it prints is kept on the bus. So it lives in a directory of its own
// outside the workspace, made private, and a credential an older builder left in the workspace is removed.
func storeCredential(workspace string, forge GitCredential) (string, func(), error) {
if err := os.Remove(filepath.Join(workspace, "git-credentials")); err != nil && !errors.Is(err, os.ErrNotExist) {
return "", nil, fmt.Errorf("a credential left in the workspace cannot be removed: %w", err)
}
if forge.URL == "" {
return "", func() {}, nil
}
dir, err := os.MkdirTemp("", "mesh-forge-credential-")
if err != nil {
return "", nil, err
}
forget := func() { os.RemoveAll(dir) }
if withinDir(workspace, dir) {
forget()
return "", nil, fmt.Errorf("the temporary directory %s is inside the workspace %s, which a check's "+
"container mounts: the forge credential is not written where it could read it", dir, workspace)
}
path := filepath.Join(dir, "git-credentials")
if err := os.WriteFile(path, []byte(forge.URL+"\n"), 0o600); err != nil {
forget()
return "", nil, err
}
return path, forget, nil
}
// recordedWithoutUserinfo makes a clone record the URL it came from without userinfo: git keeps it as given in
// the clone's .git/config, inside the workspace a check's container mounts (novox/hq issue 462).
func recordedWithoutUserinfo(ctx context.Context, run Runner, clone, repository string) error {
bare, carried := withoutUserinfo(repository)
if !carried {
return nil
}
if _, err := run(ctx, clone, "git", "remote", "set-url", "origin", bare); err != nil {
return fmt.Errorf("the clone of %s keeps its credential: %w", bare, err)
}
return nil
}
// forgetLeftCredentials removes what an earlier build or an older builder left in the workspace that holds
// a credential: the forge's git-credentials, and every .npmrc a build wrote into a tree it cloned. Run
// before a check, whose container mounts the workspace as its HOME (novox/hq issue 462). The Go caches are
// not walked: no build writes a credential there, and they hold more files than everything else.
func forgetLeftCredentials(workspace string) error {
if err := os.Remove(filepath.Join(workspace, "git-credentials")); err != nil && !errors.Is(err, os.ErrNotExist) {
return err
}
return filepath.WalkDir(workspace, func(p string, d fs.DirEntry, err error) error {
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil
}
return err
}
if d.IsDir() && p != workspace && (d.Name() == "go-cache" || d.Name() == "go-modules") &&
filepath.Dir(p) == workspace {
return filepath.SkipDir
}
if !d.IsDir() && d.Name() == ".npmrc" {
if err := os.Remove(p); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("a package-registry credential left at %s cannot be removed: %w", p, err)
}
}
return nil
})
}
// withinDir is whether path is dir or under it, both made absolute and resolved.
func withinDir(dir, path string) bool {
d, err1 := filepath.Abs(dir)
p, err2 := filepath.Abs(path)
if err1 != nil || err2 != nil {
return true
}
if r, err := filepath.EvalSymlinks(d); err == nil {
d = r
}
if r, err := filepath.EvalSymlinks(p); err == nil {
p = r
}
rel, err := filepath.Rel(d, p)
return err != nil || rel == "." || filepath.IsLocal(rel)
}
// cloneWith is a git invocation that may offer a stored credential.
//
// The first `-c credential.helper=` clears every helper the environment might carry, so exactly
internal/builder/builder_test.go
+15 -19
View File
@@ -438,34 +438,30 @@ func TestABuildOffersTheForgesCredentialThroughGitsOwnStore(t *testing.T) {
if err != nil {
t.Fatal(err)
}
stored := filepath.Join(workspace, "git-credentials")
clone := r.ran[0]
stored := storeNamedIn(t, clone)
if !strings.Contains(clone, "credential.helper=store --file="+stored) {
t.Fatalf("the clone does not name the credential store: %s", clone)
}
for _, line := range r.ran {
if strings.Contains(line, "sw0rdfi5h") {
t.Fatalf("the secret is in a command line, readable by anything that can list processes: %s", line)
}
}
// Outside the workspace a check's container mounts, and gone once the build is (novox/hq issue 462); what
// it held while git read it is checked with the check's clones (TestACheckContainerSeesNoForgeCredential).
if withinDir(workspace, stored) {
t.Fatalf("the credential store %s is inside the workspace %s", stored, workspace)
raw, err := os.ReadFile(stored)
if err != nil {
t.Fatal(err)
}
if _, err := os.Stat(stored); !os.IsNotExist(err) {
t.Fatalf("the credential store outlives the build: %v", err)
if strings.TrimSpace(string(raw)) != "http://mesh_novox_builder:sw0rdfi5h@forge.invalid:20000" {
t.Fatalf("the store does not hold the credential as given: %q", raw)
}
if _, err := os.Stat(filepath.Join(workspace, "git-credentials")); !os.IsNotExist(err) {
t.Fatal("a credential file is left in the workspace")
info, err := os.Stat(stored)
if err != nil {
t.Fatal(err)
}
}
// storeNamedIn is the credential store a git command line offers.
func storeNamedIn(t *testing.T, line string) string {
t.Helper()
_, after, ok := strings.Cut(line, "credential.helper=store --file=")
if !ok {
t.Fatalf("the clone does not name the credential store: %s", line)
if info.Mode().Perm() != 0o600 {
t.Fatalf("the credential file is readable beyond its owner: %v", info.Mode())
}
return strings.Fields(after)[0]
}
// Without a credential, a clone is exactly the invocation it always was, and no credential file
@@ -510,7 +506,7 @@ func TestAContextCloneCarriesTheSameCredentialStore(t *testing.T) {
if err != nil {
t.Fatal(err)
}
stored := storeNamedIn(t, r.ran[0])
stored := filepath.Join(workspace, "git-credentials")
var contextClone string
for _, line := range r.ran {
if strings.Contains(line, "clone") && strings.Contains(line, "source.git") {
internal/builder/check.go
+16 -137
View File
@@ -146,9 +146,6 @@ type Layer struct {
Verdict string
Summary string
Modules []string
// Failed is what the repository's own check printed that names what failed, picked from the whole of
// its output and said at the end of the verdict's report (novox/hq issue 460). Empty when it passed.
Failed string
}
// CheckScript is what a repository declares its own merge check as: run from its root, with the
@@ -229,13 +226,7 @@ func ScriptToolchain(script []byte) string {
// Check runs one merge check. An error is that it could not run; the verdict is then "error".
func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry string, forge GitCredential,
log Log) (CheckVerdict, error) {
// Everything a check says goes to the build's log, which the bus keeps: said redacted (novox/hq issue 462).
redact := redactorFor(forge)
say := logging(func(step, message string) {
if log != nil {
log(step, redact.redact(message))
}
})
say := logging(log)
began := time.Now()
ctx, stop := context.WithTimeout(ctx, CheckTimeout)
defer stop()
@@ -248,26 +239,20 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
return CheckVerdict{}, err
}
defer os.RemoveAll(root)
// The forge credential lives outside the workspace the check's containers mount, and only while the
// check clones (novox/hq issue 462).
credentials, forget, err := storeCredential(workspace, forge)
if err != nil {
return CheckVerdict{}, err
}
defer forget()
if err := forgetLeftCredentials(workspace); err != nil {
return CheckVerdict{}, err
credentials := ""
if forge.URL != "" {
credentials = filepath.Join(workspace, "git-credentials")
if err := os.WriteFile(credentials, []byte(forge.URL+"\n"), 0o600); err != nil {
return CheckVerdict{}, err
}
}
clone := func(repository, ref, dir string) error {
if _, err := run(ctx, root, "git", cloneWith(credentials, "clone", "--quiet", repository, dir)...); err != nil {
return fmt.Errorf("cannot clone %s: %s", redact.redact(repository), redact.redact(err.Error()))
}
if err := recordedWithoutUserinfo(ctx, run, filepath.Join(root, dir), repository); err != nil {
return err
return fmt.Errorf("cannot clone %s: %w", repository, err)
}
if ref != "" {
if _, err := run(ctx, filepath.Join(root, dir), "git", "checkout", "--quiet", ref); err != nil {
return fmt.Errorf("%s has no %s: %w", redact.redact(repository), ref, err)
return fmt.Errorf("%s has no %s: %w", repository, ref, err)
}
}
return nil
@@ -335,9 +320,6 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
}
}
// Every clone is made: the credential is gone before anything of the check runs (novox/hq issue 462).
forget()
// The facts, and the versions they say the mesh runs.
if registry == "" {
return CheckVerdict{}, errors.New("no artifact store to read the facts snapshot from")
@@ -494,11 +476,8 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
}
}
// What the check printed travels in the verdict to the forge and the controller: redacted as the log is.
v.Gate.Summary, v.Repo.Summary, v.Repo.Failed = redact.redact(v.Gate.Summary), redact.redact(v.Repo.Summary),
redact.redact(v.Repo.Failed)
v.Verdict, v.Summary = v.Gate.Verdict, v.Gate.Summary
v.Report, v.Took = redact.redact(withWhatFailed(out.String(), v.Repo)), time.Since(began)
v.Report, v.Took = out.String(), time.Since(began)
say("check", "gate %s — %s; repository %s — %s (%s)", strings.ToUpper(v.Gate.Verdict), v.Gate.Summary,
strings.ToUpper(v.Repo.Verdict), v.Repo.Summary, v.Took.Round(time.Second))
return v, nil
@@ -536,29 +515,17 @@ func ownCheck(ctx context.Context, spec CheckSpec, parts []ScriptPart, tree stri
say("check", "running its %s in the mesh's %s toolchain", part.Script, part.Toolchain)
fmt.Fprintf(out, "--- its %s (%s toolchain)\n", part.Script, part.Toolchain)
var own tail
var picked failures
logged := &toTheLog{say: say}
cmd := command(image, part.Script)
inItsOwnGroup(cmd)
w := io.MultiWriter(out, &own, &picked, logged)
w := io.MultiWriter(out, &own)
cmd.Stdout, cmd.Stderr = w, w
err := cmd.Run()
logged.close(picked.String())
switch {
switch err := cmd.Run(); {
case timedOut():
return &Layer{Verdict: "error", Summary: fmt.Sprintf("its %s ran past %s and was ended", part.Script, CheckTimeout),
Failed: picked.String()}
return &Layer{Verdict: "error", Summary: fmt.Sprintf("its %s ran past %s and was ended", part.Script, CheckTimeout)}
case ctx.Err() != nil:
return nil
case err != nil:
// Named from the whole run, not the tail: what failed may be thousands of lines from the end
// (novox/hq issue 460).
said := picked.said()
if said == "" {
said = whatFailed(own.String())
}
return &Layer{Verdict: "fail", Summary: passedSoFar(ran) + "its " + part.Script + " failed: " + said,
Failed: picked.String()}
return &Layer{Verdict: "fail", Summary: passedSoFar(ran) + "its " + part.Script + " failed: " + whatFailed(own.String())}
}
ran = append(ran, fmt.Sprintf("%s (%s)", part.Script, part.Toolchain))
}
@@ -568,86 +535,6 @@ func ownCheck(ctx context.Context, spec CheckSpec, parts []ScriptPart, tree stri
return &Layer{Verdict: "pass", Summary: "its " + CheckScript + " passed, each part in its toolchain: " + strings.Join(ran, ", ")}
}
// withWhatFailed is a check's report with what its repository's own check names as failed said last, whole:
// the report's tail is the last reportLines lines, and the controller keeps the end of a report it cuts, so
// what failed travels whatever came before or after it (novox/hq issue 460).
func withWhatFailed(report string, repo *Layer) string {
if repo == nil || repo.Failed == "" {
return report
}
return report + "\n--- what failed, picked from the whole of its output (the whole is in the build's log)\n" +
repo.Failed
}
// logLines bounds the lines of a repository's own check that go into the build's log. The bus keeps 10 000
// messages per subject (EVENTS' MaxMsgsPerSubject), and a build's log is one subject: past that, its first
// lines would be lost. The build's own lines are a few hundred at most, so the check's output takes 8 000.
const logLines = 8000
// logLineBytes bounds one line of it, so a line that pastes a document does not fill a message.
const logLineBytes = 4 << 10
// toTheLog says each line a repository's own check prints in the build's log, as it prints it — the whole
// output kept for the build and read with `builds` and its id (novox/hq issue 460), where before only the
// verdict's tail of it was kept. Past logLines it says how many lines it left out, and the lines that name
// what failed after them.
type toTheLog struct {
say func(step, format string, args ...any)
partial []byte
said int
dropped int
}
func (l *toTheLog) Write(p []byte) (int, error) {
l.partial = append(l.partial, p...)
for {
i := bytes.IndexByte(l.partial, '\n')
if i < 0 {
break
}
l.line(string(bytes.TrimRight(l.partial[:i], "\r")))
l.partial = l.partial[i+1:]
}
if len(l.partial) > logLineBytes {
l.line(string(l.partial))
l.partial = nil
}
return len(p), nil
}
func (l *toTheLog) line(line string) {
if l.said >= logLines {
l.dropped++
return
}
if len(line) > logLineBytes {
line = line[:logLineBytes] + fmt.Sprintf(" … (%d bytes more)", len(line)-logLineBytes)
}
l.said++
// Redacted by its shape before the bus keeps it (novox/hq issue 462); Check's own say adds the secrets
// the builder knows.
l.say("output", "%s", redactor{}.redact(line))
}
// close says the last line, and, when lines were left out, how many and what failed.
func (l *toTheLog) close(failed string) {
if len(l.partial) > 0 {
l.line(string(l.partial))
l.partial = nil
}
if l.dropped == 0 {
return
}
l.say("output", "… %d more line(s) of its output are not in this log, which keeps its first %d", l.dropped, logLines)
if failed == "" {
return
}
l.say("output", "--- what failed, picked from the whole of its output")
for _, line := range strings.Split(failed, "\n") {
l.say("output", "%s", redactor{}.redact(line))
}
}
// passedSoFar is what of a check's parts passed before the one that did not, said first.
func passedSoFar(ran []string) string {
if len(ran) == 0 {
@@ -1131,17 +1018,9 @@ func whatFailed(s string) string {
return "not gofmt'd by the toolchain's gofmt: " + strings.Join(files, ", ")
}
}
// A panic, a data race or a build error outside any test says more than the package's bare FAIL line
// (novox/hq issue 460).
for _, said := range []func(string) bool{
func(l string) bool { return strings.HasPrefix(l, "--- FAIL:") },
func(l string) bool { return strings.HasPrefix(l, "panic:") || strings.HasPrefix(l, "fatal error:") },
func(l string) bool { return l == "WARNING: DATA RACE" },
goError.MatchString,
func(l string) bool { return strings.HasPrefix(l, "FAIL\t") },
} {
for _, prefix := range []string{"--- FAIL:", "FAIL\t", "panic:"} {
for _, line := range lines {
if line = strings.TrimSpace(line); said(line) {
if line = strings.TrimSpace(line); strings.HasPrefix(line, prefix) {
return line
}
}
internal/builder/check_credential_test.go
-251
View File
@@ -1,251 +0,0 @@
package builder
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"io/fs"
"net/http"
"net/http/httptest"
"net/url"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/facts"
)
// **A check's container never sees the forge credential** (novox/hq issue 462): the toolchain container
// mounts the workspace as HOME, so a credential kept there — or a clone's .git/config carrying one — is
// readable by any pull request's merge-check.sh, and printed, kept on the bus for days.
const (
forgeSecret = "sw0rdfi5h-forge"
forgeURL = "http://mesh_novox_builder:" + forgeSecret + "@forge.invalid:20000"
besideSecret = "b3side-t0ken"
npmSecret = "npm-s3cret-t0ken"
)
// aFactsRegistry is an artifact store holding the facts snapshot, and nothing else.
func aFactsRegistry(t *testing.T) string {
t.Helper()
body, err := json.Marshal(facts.Facts{Format: facts.Format, Taken: time.Now().UTC(),
Versions: facts.Versions{Bus: "2.11.17", Store: "17.11"}, Machines: []facts.Machine{{Name: "abcdef", Length: 6}}})
if err != nil {
t.Fatal(err)
}
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
manifest, _ := json.Marshal(map[string]any{"schemaVersion": 2, "layers": []map[string]any{
{"mediaType": facts.MediaType, "digest": digest, "size": len(body)}}})
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.Contains(r.URL.Path, "/manifests/"):
w.Write(manifest)
case strings.HasSuffix(r.URL.Path, "/blobs/"+digest):
w.Write(body)
default:
http.NotFound(w, r)
}
}))
t.Cleanup(srv.Close)
return strings.TrimPrefix(srv.URL, "http://")
}
// bareURL is a URL with its userinfo left out.
func bareURL(t *testing.T, raw string) string {
u, err := url.Parse(raw)
if err != nil {
t.Fatal(err)
}
u.User = nil
return u.String()
}
func TestACheckContainerSeesNoForgeCredential(t *testing.T) {
repo, head := aCheckedRepository(t, map[string]string{CheckScript: "echo checked\n"})
besideRepo, besideHead := aCheckedRepository(t, map[string]string{"README": "beside"})
// A clone source that carries userinfo, as a forge's clone URL may: git records it as given in the
// clone's .git/config, which the container reads.
besideURL := "file://beside-user:" + besideSecret + "@" + besideRepo
workspace := t.TempDir()
var stores []string
reached := false
var leaks []string
run := func(ctx context.Context, dir, name string, args ...string) (string, error) {
switch name {
case "git":
for _, a := range args {
if f, ok := strings.CutPrefix(a, "credential.helper=store --file="); ok {
stores = append(stores, f)
raw, err := os.ReadFile(f)
if err != nil || strings.TrimSpace(string(raw)) != forgeURL {
t.Errorf("git is offered a store that does not hold the credential as given: %q, %v", raw, err)
}
if info, err := os.Stat(f); err == nil && info.Mode().Perm() != 0o600 {
t.Errorf("the credential store is readable beyond its owner: %v", info.Mode())
}
}
}
if len(args) >= 2 && args[len(args)-3] == "--quiet" && hasString(args, "clone") {
source := args[len(args)-2]
if u, err := url.Parse(source); err == nil && u.User != nil {
// Git cannot reach a file:// URL with userinfo; clone it without, then record it as git
// would have: as given.
clone := append(append([]string{}, args[:len(args)-2]...), bareURL(t, source), args[len(args)-1])
if out, err := Command(ctx, dir, "git", clone...); err != nil {
return out, err
}
return Command(ctx, filepath.Join(dir, args[len(args)-1]), "git", "remote", "set-url", "origin", source)
}
}
return Command(ctx, dir, name, args...)
case "docker":
if !reached {
reached = true
// The first container: everything the workspace holds is what the toolchain container sees.
filepath.WalkDir(workspace, func(path string, d fs.DirEntry, err error) error {
if err != nil || d.IsDir() {
return nil
}
raw, _ := os.ReadFile(path)
s := string(raw)
if strings.Contains(s, forgeSecret) || strings.Contains(s, besideSecret) ||
strings.Contains(s, npmSecret) || d.Name() == "git-credentials" || d.Name() == ".npmrc" ||
strings.Contains(s, "credential.helper") {
leaks = append(leaks, path)
}
return nil
})
for _, f := range stores {
if _, err := os.Stat(f); !errors.Is(err, os.ErrNotExist) {
leaks = append(leaks, f+" (still there when the first container runs)")
}
if rel, err := filepath.Rel(workspace, f); err == nil && !strings.HasPrefix(rel, "..") {
leaks = append(leaks, f+" (inside the workspace the container mounts)")
}
}
}
if len(args) > 0 && args[0] == "ps" {
return "", nil
}
return "", errors.New("no container runtime in this test")
}
return "", errors.New("unexpected command " + name)
}
// A credential an older builder left in the workspace is removed too: the forge's, and the .npmrc a
// build wrote into the tree it cloned.
if err := os.WriteFile(filepath.Join(workspace, "git-credentials"), []byte(forgeURL+"\n"), 0o600); err != nil {
t.Fatal(err)
}
for _, dir := range []string{"source/x", "context-server"} {
if err := os.MkdirAll(filepath.Join(workspace, dir), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(workspace, dir, ".npmrc"),
[]byte("//forge.invalid/api/packages/novox/npm/:_authToken="+npmSecret+"\n"), 0o600); err != nil {
t.Fatal(err)
}
}
_, err := Check(t.Context(), run, CheckSpec{ID: "check-462", Repository: repo, Ref: head, Owner: "novox",
Repo: "mesh-controller", Number: 1, Toolchain: "golang", Beside: map[string]Beside{
"mesh-catalog": {Repository: besideURL, Ref: besideHead}}}, workspace, aFactsRegistry(t),
GitCredential{URL: forgeURL}, nil)
if err == nil {
t.Fatal("the check ran past its first container in a test with none")
}
if !reached {
t.Fatalf("the check never reached its first container: %v", err)
}
if len(stores) == 0 {
t.Fatal("no clone was offered the forge credential")
}
if len(leaks) > 0 {
t.Fatalf("the check's container sees the credential:\n%s", strings.Join(leaks, "\n"))
}
}
// Every line a repository's own check prints is published to the build's log redacted.
func TestACheckLinePublishedToTheLogIsRedacted(t *testing.T) {
var said []string
say := func(step, format string, args ...any) {
if step == "output" && len(args) > 0 {
said = append(said, args[0].(string))
}
}
var out tail
layer := ownCheck(t.Context(), CheckSpec{Toolchain: "golang"}, []ScriptPart{{Toolchain: "go", Script: CheckScript}},
t.TempDir(), &out, func() bool { return false }, func(string, string) *exec.Cmd {
return exec.CommandContext(t.Context(), "sh", "-c", "echo cloning http://mesh_builder:t0ps3cret-forge@forge.invalid/novox/x.git; "+
"echo token ghp_abcdefghijklmnopqrstuvwxyz0123456789")
}, say)
if layer == nil || layer.Verdict != "pass" {
t.Fatalf("the check answered %+v\n%s", layer, out.String())
}
joined := strings.Join(said, "\n")
if strings.Contains(joined, "t0ps3cret-forge") || strings.Contains(joined, "ghp_abcdef") {
t.Fatalf("a secret the check printed is published to the build's log:\n%s", joined)
}
if !strings.Contains(joined, "http://mesh_builder:[redacted: a password in a URI]@forge.invalid/novox/x.git") {
t.Fatalf("the line is not said with what was there named:\n%s", joined)
}
}
func TestTheRedactorHidesTheForgeCredentialAndShapes(t *testing.T) {
r := redactorFor(GitCredential{URL: forgeURL})
for in, want := range map[string]string{
"the secret alone: " + forgeSecret: "the secret alone: [redacted: the forge credential]",
"go test ./... ok": "go test ./... ok",
"--password hunter22 and done": "--password [redacted: the word after --password] and done",
"commit 3b6b54a0c1d2e3f4a5b6c7d8e9f0": "commit 3b6b54a0c1d2e3f4a5b6c7d8e9f0",
} {
if got := r.redact(in); got != want {
t.Errorf("%q redacted as %q, want %q", in, got, want)
}
}
}
// A build's clone of a URL carrying userinfo records it without, so no build leaves a credential in
// workspace/source/.git/config while it runs either (novox/hq issue 462); the tree itself is gone when the
// build ends.
func TestABuildsCloneRecordsNoUserinfo(t *testing.T) {
repo, _ := aCheckedRepository(t, map[string]string{ManifestName: `{"module":"plain","version":"1"}`})
source := "file://build-user:" + besideSecret + "@" + repo
workspace := t.TempDir()
tree := filepath.Join(workspace, "source")
var configs []string
run := func(ctx context.Context, dir, name string, args ...string) (string, error) {
if name == "git" && hasString(args, "clone") && args[len(args)-2] == source {
clone := append(append([]string{}, args[:len(args)-2]...), bareURL(t, source), args[len(args)-1])
if out, err := Command(ctx, dir, "git", clone...); err != nil {
return out, err
}
return Command(ctx, args[len(args)-1], "git", "remote", "set-url", "origin", source)
}
if name == "git" && len(args) > 0 && args[0] == "rev-parse" {
raw, _ := os.ReadFile(filepath.Join(tree, ".git", "config"))
configs = append(configs, string(raw))
}
return Command(ctx, dir, name, args...)
}
if _, err := Build(t.Context(), run, &recorded{}, source, "", "", workspace, nil, Npmrc{}, GitCredential{}, nil); err != nil {
t.Fatal(err)
}
if len(configs) == 0 {
t.Fatal("the build never read its clone")
}
for _, c := range configs {
if strings.Contains(c, besideSecret) || strings.Contains(c, "build-user") {
t.Fatalf("the build's clone records the credential it was cloned with:\n%s", c)
}
}
if _, err := os.Stat(tree); !os.IsNotExist(err) {
t.Fatalf("the build's tree outlives the build: %v", err)
}
}
internal/builder/failures.go
-285
View File
@@ -1,285 +0,0 @@
package builder
import (
"bytes"
"fmt"
"regexp"
"strings"
)
// **What failed is picked from the whole of a check's output, as it streams** (novox/hq issue 460).
//
// A repository's own check kept only the last reportLines lines of what it printed, and named what failed
// from those. A long store-backed run prints its logs after the failure: the `--- FAIL:` line of mesh-controller
// #218 was pushed out of the tail by the package's own log lines, and the verdict said only
// "FAIL <package> 268.072s" — the test that failed could not be named, and the fix was a guess. #220's failure
// came a few dozen lines before the end, inside the tail, and was named. The length of what came after the
// failure decided it, nothing else.
//
// So every line is looked at as it is written, and the lines that name what failed are kept whole, wherever
// in the output they are: each `--- FAIL:` line with its indented message lines, a `panic:` with its first
// frames, a `WARNING: DATA RACE` report, each `FAIL\t<package>` line, a build error with its package, and
// the files gofmt lists. The rest is cut as before.
// Bounds on what is picked, so a run that fails everywhere still travels in a verdict: the controller cuts a
// report to its last 60 KiB (maxCheckReport), and what failed is said at the report's end, so it stays inside.
const (
// failureLines and failureBytes bound everything picked from one run.
failureLines = 400
failureBytes = 32 << 10
// failureLineBytes bounds one picked line: a message that pastes a whole document is cut, said.
failureLineBytes = 1 << 10
// The lines kept after the line that opens each kind of block.
failMessageLines = 30 // a --- FAIL's messages
panicLines = 40 // a panic's first frames
raceLines = 80 // a data race report, to its closing rule
buildLines = 40 // a package's build errors
gofmtLines = 50 // the files gofmt lists
// failedNames bounds how many further failing tests the summary names after the first.
failedNames = 5
)
// goError is a compiler's or vet's line: a Go file, a line, and what is wrong there.
var goError = regexp.MustCompile(`^\S+\.go:\d+(:\d+)?: `)
// raceRule is the line the race detector opens and closes its report with.
const raceRule = "=================="
type failureBlock int
const (
noBlock failureBlock = iota
inFail
inPanic
inRace
inBuild
inGofmt
)
// failures keeps the lines of a check's output that name what failed. It is an io.Writer, fed the same
// bytes as the report's tail.
type failures struct {
partial []byte
kept []string
size int
dropped int
block failureBlock
indent int // a --- FAIL line's indentation: its messages are indented deeper
left int // lines the open block may still keep
headers []string
tests []string // every failing test's --- FAIL line, in order
lastRule bool // the line before was the race detector's rule, which opens its report
}
func (f *failures) Write(p []byte) (int, error) {
f.partial = append(f.partial, p...)
for {
i := bytes.IndexByte(f.partial, '\n')
if i < 0 {
break
}
f.line(strings.TrimRight(string(f.partial[:i]), "\r"))
f.partial = f.partial[i+1:]
}
// A line with no end, longer than any line is kept, is judged by its start.
if len(f.partial) > failureLineBytes*4 {
f.line(string(f.partial))
f.partial = nil
}
return len(p), nil
}
// flush judges what is left of a last line with no newline.
func (f *failures) flush() {
if len(f.partial) > 0 {
f.line(strings.TrimRight(string(f.partial), "\r"))
f.partial = nil
}
}
func indentOf(line string) int {
return len(line) - len(strings.TrimLeft(line, " \t"))
}
func (f *failures) keep(line string) {
if f.size >= failureBytes || len(f.kept) >= failureLines {
f.dropped++
return
}
if len(line) > failureLineBytes {
line = line[:failureLineBytes] + fmt.Sprintf(" … (%d bytes more)", len(line)-failureLineBytes)
}
f.kept = append(f.kept, line)
f.size += len(line) + 1
}
func (f *failures) open(block failureBlock, lines int) {
f.block, f.left = block, lines
}
func (f *failures) line(line string) {
trimmed := strings.TrimSpace(line)
rule := trimmed == raceRule
// A line that opens a block ends whatever block was open.
if f.opens(line, trimmed) {
f.lastRule = rule
return
}
switch f.block {
case inFail:
if trimmed != "" && indentOf(line) > f.indent && f.left > 0 {
f.left--
f.keep(line)
f.lastRule = rule
return
}
case inPanic:
if f.left > 0 && !strings.HasPrefix(line, "FAIL") && !strings.HasPrefix(line, "ok \t") &&
!strings.HasPrefix(line, "exit status") {
f.left--
f.keep(line)
f.lastRule = rule
return
}
case inRace:
if f.left > 0 {
f.left--
f.keep(line)
if rule {
f.block = noBlock
}
f.lastRule = rule
return
}
case inBuild:
if f.left > 0 && trimmed != "" && !strings.HasPrefix(line, "FAIL") && !strings.HasPrefix(line, "ok \t") &&
!strings.HasPrefix(line, "? \t") {
f.left--
f.keep(line)
f.lastRule = rule
return
}
case inGofmt:
if f.left > 0 && trimmed != "" {
f.left--
f.keep(line)
f.lastRule = rule
return
}
}
f.block = noBlock
switch {
case strings.HasPrefix(line, "# "):
// A package's name before its build errors — kept only when an error follows it.
f.headers = append(f.headers, line)
if len(f.headers) > 4 {
f.headers = f.headers[1:]
}
case len(f.headers) > 0 && goError.MatchString(line):
for _, h := range f.headers {
f.keep(h)
}
f.headers = nil
f.keep(line)
f.open(inBuild, buildLines)
case strings.HasPrefix(line, "FAIL\t"):
f.headers = nil
f.keep(line)
default:
f.headers = nil
}
f.lastRule = rule
}
// opens keeps a line that opens a block of what failed, and opens the block. False for any other line.
func (f *failures) opens(line, trimmed string) bool {
switch {
case strings.HasPrefix(trimmed, "--- FAIL:"):
f.headers = nil
f.keep(line)
f.tests = append(f.tests, trimmed)
f.indent = indentOf(line)
f.open(inFail, failMessageLines)
case strings.HasPrefix(line, "panic:") || strings.HasPrefix(line, "fatal error:"):
f.headers = nil
f.keep(line)
f.open(inPanic, panicLines)
case trimmed == "WARNING: DATA RACE":
f.headers = nil
if f.lastRule {
f.keep(raceRule)
}
f.keep(line)
f.open(inRace, raceLines)
case strings.HasPrefix(trimmed, "not gofmt'd:"):
f.headers = nil
f.keep(line)
f.open(inGofmt, gofmtLines)
default:
return false
}
return true
}
// String is every line picked, in the order the run printed them, and how many more were left out.
func (f *failures) String() string {
f.flush()
s := strings.Join(f.kept, "\n")
if f.dropped > 0 {
s += fmt.Sprintf("\n… %d more line(s) naming what failed, past the %d lines or %d KiB kept", f.dropped,
failureLines, failureBytes>>10)
}
return s
}
// said is what the repository layer's summary says failed: the first thing that failed, whole, then the
// further failing tests by name. Empty when nothing was picked.
func (f *failures) said() string {
picked := f.String()
if picked == "" {
return ""
}
first := whatFailed(picked)
var more []string
for _, t := range f.tests {
if t == first {
continue
}
// A subtest's parent fails with it; its name is in the subtest's.
name := strings.Fields(strings.TrimPrefix(t, "--- FAIL:"))
if len(name) > 0 {
more = append(more, name[0])
}
}
more = withoutParents(more)
switch {
case len(more) == 0:
return first
case len(more) > failedNames:
return fmt.Sprintf("%s; and %s and %d more", first, strings.Join(more[:failedNames], ", "),
len(more)-failedNames)
default:
return first + "; and " + strings.Join(more, ", ")
}
}
// withoutParents drops a test whose subtest is also named.
func withoutParents(names []string) []string {
var out []string
for _, n := range names {
parent := false
for _, m := range names {
if strings.HasPrefix(m, n+"/") {
parent = true
break
}
}
if !parent {
out = append(out, n)
}
}
return out
}
internal/builder/failures_test.go
-182
View File
@@ -1,182 +0,0 @@
package builder
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// ownCheckOf runs a repository's own check whose script prints a captured output and fails, as the build
// seat runs it, and answers the layer and every line it said in the build's log.
func ownCheckOf(t *testing.T, printed string) (*Layer, []string) {
t.Helper()
tree := t.TempDir()
if err := os.WriteFile(filepath.Join(tree, "printed.txt"), []byte(printed), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(tree, CheckScript), []byte("cat printed.txt\nexit 1\n"), 0o755); err != nil {
t.Fatal(err)
}
var out tail
var logged []string
layer := ownCheck(t.Context(), CheckSpec{Toolchain: "go-image"}, []ScriptPart{{"go", CheckScript}}, tree, &out,
func() bool { return false },
func(_, script string) *exec.Cmd {
cmd := exec.CommandContext(t.Context(), "sh", script)
cmd.Dir = tree
return cmd
}, func(step, format string, args ...any) {
if step == "output" {
logged = append(logged, fmt.Sprintf(format, args...))
}
})
if layer == nil || layer.Verdict != "fail" {
t.Fatalf("a failing check answered %+v", layer)
}
return layer, logged
}
func captured(t *testing.T, name string) string {
t.Helper()
body, err := os.ReadFile(filepath.Join("testdata", "issue460", name))
if err != nil {
t.Fatal(err)
}
return string(body)
}
// **novox/hq issue 460**: mesh-controller #218's check failed with "FAIL <package> 268.072s" and no test
// name: the package's own logs, printed after its `--- FAIL:` lines, pushed them out of the last 200 lines,
// which was all the summary was named from. The first failing test is named, and the others after it,
// however much came after them.
func TestAFailureEarlyInALongRunIsStillNamed(t *testing.T) {
layer, _ := ownCheckOf(t, captured(t, "a-long-run-failing-early.txt"))
if !strings.Contains(layer.Summary, "failed: --- FAIL: TestTheEnvelopeIsPinned (0.00s)") {
t.Errorf("a failure 600 lines from the end is said as %q", layer.Summary)
}
if !strings.Contains(layer.Summary, "TestSubtests/the_hub") {
t.Errorf("the second failing test is not named: %q", layer.Summary)
}
}
// One -race run of every package: the first failure is the earliest, not the last within the tail.
func TestARaceRunOfEveryPackageNamesItsFirstFailure(t *testing.T) {
layer, _ := ownCheckOf(t, captured(t, "a-race-run-of-every-package.txt"))
if !strings.Contains(layer.Summary, "failed: --- FAIL: TestTheEnvelopeIsPinned (0.00s)") ||
!strings.Contains(layer.Summary, "TestAMapIsNil") || !strings.Contains(layer.Summary, "TestACounterIsShared") {
t.Errorf("a run failing in four packages is said as %q", layer.Summary)
}
}
// A package that does not build is named by its error, not by its bare "[build failed]".
func TestABuildErrorIsNamedByTheError(t *testing.T) {
layer, _ := ownCheckOf(t, captured(t, "a-build-error.txt"))
if !strings.HasSuffix(layer.Summary, "failed: broken/broken.go:3:28: undefined: undefinedThing") {
t.Errorf("a build error is said as %q", layer.Summary)
}
}
// The whole of what the check printed is in the build's log, line by line — where before none of it was.
func TestTheWholeOutputIsInTheBuildsLog(t *testing.T) {
printed := captured(t, "a-long-run-failing-early.txt")
_, logged := ownCheckOf(t, printed)
want := strings.Split(strings.TrimRight(printed, "\n"), "\n")
if len(logged) != len(want) {
t.Fatalf("the build's log holds %d line(s) of the %d printed", len(logged), len(want))
}
for i := range want {
if logged[i] != want[i] {
t.Fatalf("line %d is logged as %q, printed as %q", i+1, logged[i], want[i])
}
}
}
// What failed is kept whole in the verdict: each --- FAIL with its messages, a panic with its first frames,
// a data race report from rule to rule — and none of the log lines around them.
func TestTheVerdictKeepsWhatFailedWhole(t *testing.T) {
layer, _ := ownCheckOf(t, captured(t, "a-long-run-failing-early.txt"))
want := "--- FAIL: TestTheEnvelopeIsPinned (0.00s)\n" +
" early_test.go:9: the envelope's subject is \"mesh.a\", not \"mesh.b\"\n" +
" early_test.go:10: a second line of the same failure\n" +
"--- FAIL: TestSubtests (0.00s)\n" +
" --- FAIL: TestSubtests/the_hub (0.00s)\n" +
" early_test.go:14: the hub does not compose\n" +
"FAIL\texample.com/cap/early\t"
if !strings.HasPrefix(layer.Failed, want) || strings.Contains(layer.Failed, "kept what home-server says") {
t.Errorf("the early failure is kept as:\n%s", layer.Failed)
}
layer, _ = ownCheckOf(t, captured(t, "a-panic.txt"))
for _, line := range []string{"--- FAIL: TestAMapIsNil (0.00s)", "panic: assignment to entry in nil map",
"[running]:", "example.com/cap/panics.TestAMapIsNil(", "/src/cap/panics/panics_test.go:7",
"FAIL\texample.com/cap/panics\t"} {
if !strings.Contains(layer.Failed, line) {
t.Errorf("a panic is kept without %q:\n%s", line, layer.Failed)
}
}
layer, _ = ownCheckOf(t, captured(t, "a-data-race.txt"))
race := captured(t, "a-data-race.txt")
report := race[:strings.Index(race, "--- FAIL:")]
if !strings.HasPrefix(layer.Failed, report) ||
!strings.Contains(layer.Failed, "--- FAIL: TestACounterIsShared (0.00s)\n testing.go:1865: race detected") {
t.Errorf("a data race is kept as:\n%s", layer.Failed)
}
layer, _ = ownCheckOf(t, captured(t, "a-build-error.txt"))
if layer.Failed != "# example.com/cap/broken\nbroken/broken.go:3:28: undefined: undefinedThing\n"+
"FAIL\texample.com/cap/broken [build failed]" {
t.Errorf("a build error is kept as:\n%s", layer.Failed)
}
}
// A run that fails everywhere is bounded in what it keeps, and what it keeps survives the controller's cut
// of a report to its last 60 KiB, said at the report's end.
func TestWhatFailedIsBoundedAndSurvivesTheReportsCut(t *testing.T) {
var b strings.Builder
for i := range 3000 {
fmt.Fprintf(&b, "--- FAIL: TestNumber%d (0.00s)\n x_test.go:1: %s\n", i, strings.Repeat("why ", 600))
fmt.Fprintf(&b, "2026/10/11 01:30:03 a log line %d\n", i)
}
b.WriteString("FAIL\tx/everything\t1s\nFAIL\n")
layer, logged := ownCheckOf(t, b.String())
if len(layer.Failed) > failureBytes+2*failureLineBytes || !strings.Contains(layer.Failed, "more line(s) naming what failed") {
t.Errorf("what failed in a run that fails everywhere is %d bytes, ending %q", len(layer.Failed),
layer.Failed[max(0, len(layer.Failed)-200):])
}
if !strings.Contains(layer.Summary, "--- FAIL: TestNumber0 (0.00s); and TestNumber1, ") ||
!strings.Contains(layer.Summary, "and 2994 more") {
t.Errorf("a run failing 3000 tests is said as %q", layer.Summary)
}
var out tail
out.Write([]byte(b.String()))
report := withWhatFailed(out.String(), layer)
const maxCheckReport = 60 << 10 // as the controller cuts it, from the front
if len(report) > maxCheckReport {
report = report[len(report)-maxCheckReport:]
}
if !strings.Contains(report, "--- what failed") || !strings.Contains(report, "--- FAIL: TestNumber0 (0.00s)") {
t.Error("what failed did not survive the controller's cut of the report")
}
// The build's log keeps its first logLines lines, says how many it left out, then what failed.
if len(logged) < logLines+2 || logged[logLines] != "… 1002 more line(s) of its output are not in this log, which keeps its first 8000" ||
logged[logLines+1] != "--- what failed, picked from the whole of its output" ||
logged[logLines+2] != "--- FAIL: TestNumber0 (0.00s)" {
t.Errorf("the log past its bound says %q", logged[logLines:min(len(logged), logLines+3)])
}
}
// A passing check says nothing of what failed.
func TestAPassingCheckReportsAsBefore(t *testing.T) {
if got := withWhatFailed("ok\tx\t1s", &Layer{Verdict: "pass"}); got != "ok\tx\t1s" {
t.Errorf("a passing report became %q", got)
}
if got := withWhatFailed("r", nil); got != "r" {
t.Errorf("no repository layer became %q", got)
}
}
internal/builder/mirror.go
-5
View File
@@ -429,11 +429,6 @@ func (r Registry) copyBlob(ctx context.Context, src *source, where upstream, dig
if response.ContentLength > 0 {
put.ContentLength = response.ContentLength
}
// **Not waited for if refused** (novox/hq issue 457): the body streams from upstream and cannot be
// read twice, so a registry held still between the POST above and this PUT fails the copy with
// "its body cannot be read twice" rather than waiting. Accepted: the POST a moment before already
// waited the registry out, so the window is the length of one upstream fetch, and the build fails
// loudly, to be asked again, rather than buffering every base blob in memory.
done, err := r.client().Do(put)
if err != nil {
return fmt.Errorf("cannot upload blob %s: %w", digest, err)
internal/builder/packages_test.go
+6 -18
View File
@@ -70,16 +70,7 @@ func TestNpmrcDisabledUntilThereIsARegistry(t *testing.T) {
func TestAnImageBuildGetsTheCredentialInTheContextAndHostNetwork(t *testing.T) {
r, workspace := aRepository(t, withBoth, map[string]string{"Dockerfile": "FROM scratch\nCOPY .npmrc ./", "files/x": "y"})
n := Npmrc{Scope: "@novox", Registry: "https://forge.invalid/api/packages/novox/npm/", Token: "t"}
npmrc := filepath.Join(workspace, "source", ".npmrc")
inContext := false
run := func(ctx context.Context, dir, name string, args ...string) (string, error) {
if name == "docker" && len(args) > 0 && args[0] == "build" {
_, err := os.Stat(npmrc)
inContext = err == nil
}
return r.run(ctx, dir, name, args...)
}
if _, err := Build(context.Background(), run, r,
if _, err := Build(context.Background(), r.run, r,
"https://forge.invalid/meshboard.git", "", "", workspace, nil, n, GitCredential{}, nil); err != nil {
t.Fatalf("the build failed: %v", err)
}
@@ -99,14 +90,11 @@ func TestAnImageBuildGetsTheCredentialInTheContextAndHostNetwork(t *testing.T) {
if !strings.Contains(build, "--network host") {
t.Fatalf("the build was not given the host network to reach the registry: %s", build)
}
// The .npmrc is written into the build context (the source tree), where a Dockerfile COPYs it, and
// removed with the tree when the build ends: a later check's container mounts the workspace (novox/hq
// issue 462).
if !inContext {
t.Fatal("the credential was not in the build context when the image was built")
}
if _, err := os.Stat(npmrc); !os.IsNotExist(err) {
t.Fatalf("the credential outlives the build in the workspace: %v", err)
// The .npmrc is written into the build context (the source tree), where a Dockerfile COPYs it.
tree := filepath.Join(workspace, "source")
npmrc := filepath.Join(tree, ".npmrc")
if _, err := os.Stat(npmrc); err != nil {
t.Fatalf("the credential was not written into the build context: %v", err)
}
}
internal/builder/redact.go
-191
View File
@@ -1,191 +0,0 @@
package builder
// **Every line of a check's output is redacted before it is kept** (novox/hq issue 462).
//
// A repository's own check prints into the build's log, which the bus keeps for days and anyone who may read
// its events reads, and into the verdict, which the forge shows on the pull request. Software prints what it
// was given — a URL carrying a password, a token in a flag — and a pull request may print on purpose.
// So a line is said only after every secret the builder knows (the forge credential's password) and every
// value whose shape says it is one is replaced by a mark naming what was there, as the journal verb does.
//
// Copied from the journal tool's redactor (mesh-catalog, modules/systemd/cmd/systemd-tools/secrets.go,
// itself a copy of the docker module's), narrowed to a line's shapes, with the token shapes a check's output
// may carry added. A third copy: sharing them through mesh-sdk is novox/hq issue 471.
import (
"net/url"
"regexp"
"strings"
)
// secretName is a variable name that says its value is a secret.
var secretName = regexp.MustCompile(`(?i)(pass(word|wd|phrase)?|secret|token|api_?key|private_?key|access_?key|credential|auth)`)
// notAValue is a name that says its value is where a secret is, not the secret: a file or a path.
var notAValue = regexp.MustCompile(`(?i)(_FILE|FILE|_PATH|_DIR)$`)
// uriPassword is a URI carrying a password in its userinfo: scheme://user:password@.
var uriPassword = regexp.MustCompile(`[A-Za-z][A-Za-z0-9+.-]*://[^\s/:@'"]*:([^\s/@'"]+)@`)
// tokenShaped are tokens recognised by their own prefix, whatever surrounds them: a forge's or a host's
// access token, a JSON web token, a NATS seed.
var tokenShaped = []struct {
name string
re *regexp.Regexp
}{
{"an access token", regexp.MustCompile(`\b(gh[pousr]_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,}|glpat-[A-Za-z0-9_-]{20,}|xox[abpr]-[A-Za-z0-9-]{10,}|sk-ant-[A-Za-z0-9_-]{20,})`)},
{"a JSON web token", regexp.MustCompile(`\beyJ[A-Za-z0-9_-]{8,}\.eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]+`)},
{"a NATS seed", regexp.MustCompile(`\bS[ACNOU][A-Z2-7]{56}\b`)},
}
// masked is a password a program already hid: ***, xxx, <redacted>, [REDACTED].
var masked = regexp.MustCompile(`^(\*+|x+|X+|<[^>]*>|\[[^\]]*\]|%2A+)$`)
// ordinary is a value under a secret's name that is not one: a path, an address, a number, a switch.
var ordinary = regexp.MustCompile(`^(/.*|[A-Za-z][A-Za-z0-9+.-]*://.*|[0-9.]+[a-z]?|(?i:true|false|yes|no|on|off|none|null))$`)
// leastSecret is the shortest value compared as a secret: a shorter one matches ordinary words.
const leastSecret = 6
// passwordFlags take a secret as their next word, or after `=`, whatever the program.
var passwordFlags = map[string]bool{
"-P": true, "--password": true, "--pass": true, "--passwd": true, "--secret": true, "--secret-key": true,
"--token": true, "--api-key": true, "--apikey": true, "--auth": true,
}
// knownSecret is one value the builder holds, by the name it is said under.
type knownSecret struct {
Name string
Value string
}
// redactor hides the secrets it knows and those a line's shapes say are secrets.
type redactor struct{ known []knownSecret }
// redactorFor knows the forge credential's password, and its user's name with it, in every form git or a
// program may print them.
func redactorFor(forge GitCredential) redactor {
var r redactor
if forge.URL == "" {
return r
}
for _, m := range uriPassword.FindAllStringSubmatch(forge.URL, -1) {
r.add("the forge credential", m[1])
if dec, err := url.PathUnescape(m[1]); err == nil && dec != m[1] {
r.add("the forge credential", dec)
}
}
return r
}
func (r *redactor) add(name, value string) {
if len(value) < leastSecret || masked.MatchString(value) {
return
}
for _, k := range r.known {
if k.Value == value {
return
}
}
r.known = append(r.known, knownSecret{name, value})
}
// redact is a text with every known secret, every value its shape says is one, and every password inside a
// URI replaced by a mark naming what was there. Line by line: a shape is judged within its line.
func (r redactor) redact(text string) string {
if !strings.ContainsAny(text, "\n") {
return r.line(text)
}
lines := strings.Split(text, "\n")
for i, l := range lines {
lines[i] = r.line(l)
}
return strings.Join(lines, "\n")
}
func (r redactor) line(line string) string {
replace := func(s knownSecret) {
for _, f := range forms(s.Value) {
line = strings.ReplaceAll(line, f, "[redacted: "+s.Name+"]")
}
}
for _, s := range r.known {
replace(s)
}
for _, s := range shaped(line) {
replace(s)
}
line = uriPassword.ReplaceAllStringFunc(line, func(m string) string {
sub := uriPassword.FindStringSubmatch(m)
if masked.MatchString(sub[1]) || strings.HasPrefix(sub[1], "[redacted") {
return m
}
return strings.TrimSuffix(m, sub[1]+"@") + "[redacted: a password in a URI]@"
})
for _, t := range tokenShaped {
line = t.re.ReplaceAllString(line, "[redacted: "+t.name+"]")
}
return line
}
// forms are the ways a value may appear printed: as given, and URL-encoded.
func forms(value string) []string {
out := []string{value}
for _, f := range []string{url.QueryEscape(value), url.PathEscape(value)} {
if f != value && !hasString(out, f) {
out = append(out, f)
}
}
return out
}
func hasString(list []string, s string) bool {
for _, x := range list {
if x == s {
return true
}
}
return false
}
// shaped are the values a line carries by their shape: the word after a password flag, or the value of one
// given with `=`, and a NAME=value whose name says secret.
func shaped(line string) []knownSecret {
var out []knownSecret
add := func(name, value string) {
value = strings.Trim(value, `"',;`)
if len(value) < leastSecret || masked.MatchString(value) || ordinary.MatchString(value) ||
strings.HasPrefix(value, "[redacted") {
return
}
out = append(out, knownSecret{name, value})
}
words := strings.Fields(line)
for i, w := range words {
if flag, value, ok := strings.Cut(w, "="); ok && strings.HasPrefix(flag, "-") {
if passwordFlags[flag] {
add("the value of "+flag, value)
}
continue
}
if name, value, ok := strings.Cut(w, "="); ok && name != "" && secretName.MatchString(name) &&
!notAValue.MatchString(name) && !strings.ContainsAny(name, "/:") {
add("the value of "+name, value)
continue
}
if i+1 < len(words) && passwordFlags[w] {
add("the word after "+w, words[i+1])
}
}
return out
}
// withoutUserinfo is a URL with its userinfo left out, and whether it carried any.
func withoutUserinfo(raw string) (string, bool) {
u, err := url.Parse(raw)
if err != nil || u.User == nil {
return raw, false
}
u.User = nil
return u.String(), true
}
internal/builder/registry.go
+3 -9
View File
@@ -52,11 +52,7 @@ func (r Registry) PublishImage(ctx context.Context, localTag, repository string)
if _, err := r.Run(ctx, "", "docker", "tag", localTag, remote); err != nil {
return "", err
}
// The push waits for a registry held still, as every request to it does (novox/hq issue 457).
if err := waitForRegistry(ctx, r.Address, "docker push "+remote, func() error {
_, err := r.Run(ctx, "", "docker", "push", remote)
return err
}); err != nil {
if _, err := r.Run(ctx, "", "docker", "push", remote); err != nil {
return "", err
}
out, err := r.Run(ctx, "", "docker", "inspect", "--format", "{{index .RepoDigests 0}}", remote)
@@ -173,13 +169,11 @@ func (r Registry) has(ctx context.Context, url string, accept ...string) (bool,
return response.StatusCode == http.StatusOK, nil
}
// client is the client for the registry and for upstream, whose requests to the registry wait out a
// registry held still (novox/hq issue 457).
func (r Registry) client() *http.Client {
if r.HTTP != nil {
return waiting(r.HTTP, r.Address)
return r.HTTP
}
return waiting(http.DefaultClient, r.Address)
return http.DefaultClient
}
// separator is whether the upload location already carries a query.
internal/builder/registry_wait.go
-137
View File
@@ -1,137 +0,0 @@
package builder
import (
"context"
"errors"
"fmt"
"net/http"
"strings"
"syscall"
"time"
)
// A build waits out a registry that refuses connections, for a bounded time (novox/hq issue 457).
//
// **Why waiting, and why here.** The store's nightly collection holds the registry still for its run —
// about a minute and a half, measured on 2026-10-11 — and a build that reached the registry in that
// window failed on "connection refused", and its whole delivery plan with it: a plan failed for a
// pause the mesh itself scheduled. The other design weighed was the collection telling the controller
// it holds the registry, and the controller holding build asks while it runs. Waiting here is smaller
// and covers more: it is local to the one place that talks to the registry, needs no new message
// between modules, and also carries a build over any other short outage — a registry restarted by its
// own update, say. A refusal is the one error waited for: nothing was sent, so trying again cannot
// do anything twice, and it is what a registry that is stopped answers.
//
// **Bounded, and loud past the bound.** registryWait is longer than the collection holds the registry
// (five minutes against about one and a half), so the pause the mesh schedules is always waited out,
// and a registry that is really down still fails the build — saying how long it was refused — rather
// than holding a build machine for ever. Every wait is said in the build's log, with why, and so is
// the registry answering again.
var (
// registryWait is how long a build waits for a registry that refuses, per call that found it so.
registryWait = 5 * time.Minute
// registryFirstPause is the first pause between tries; each pause doubles, up to registryMostPause.
registryFirstPause = time.Second
)
// registryMostPause is the longest pause between two tries: short enough that a build goes on within
// seconds of the registry answering again.
const registryMostPause = 10 * time.Second
// refused is whether an error is a connection refused: from a dial here, or as a command such as docker
// said it in its output.
func refused(err error) bool {
return err != nil && (errors.Is(err, syscall.ECONNREFUSED) || strings.Contains(err.Error(), "connection refused"))
}
// waitForRegistry runs try, and while it fails because the registry at address refuses connections,
// tries again with a growing pause until registryWait has passed. what names the call, for the log.
func waitForRegistry(ctx context.Context, address, what string, try func() error) error {
err := try()
if !refused(err) {
return err
}
started := time.Now()
pause := registryFirstPause
tell("registry", "%s: refused; the build waits for the registry at %s, for up to %s — it is held still while "+
"the store's nightly collection runs, about a minute and a half (novox/hq issue 457)",
what, address, registryWait)
for {
left := registryWait - time.Since(started)
if left <= 0 {
tell("registry", "%s: the registry at %s still refuses after %s; the build fails", what, address,
time.Since(started).Round(time.Second))
return fmt.Errorf("the registry at %s refused every connection for %s, longer than its nightly "+
"collection holds it still, so it is down, not paused: %w",
address, time.Since(started).Round(time.Millisecond), err)
}
wait := min(pause, left)
select {
case <-ctx.Done():
return fmt.Errorf("stopped while waiting for the registry at %s: %w (last: %v)", address, ctx.Err(), err)
case <-time.After(wait):
}
pause = min(pause*2, registryMostPause)
if err = try(); !refused(err) {
// Said as it is: the registry answering is only the build going on when the call worked.
if err == nil {
tell("registry", "%s: the registry at %s answers again after %s; the build goes on", what, address,
time.Since(started).Round(time.Millisecond))
} else {
tell("registry", "%s: the registry at %s no longer refuses after %s, and answered with: %v", what,
address, time.Since(started).Round(time.Millisecond), err)
}
return err
}
}
}
// waitingTransport waits for the registry on every request to it, and on none to anywhere else: the
// same client copies from upstream registries, whose refusals are theirs to answer.
type waitingTransport struct {
base http.RoundTripper
address string
}
func (t waitingTransport) RoundTrip(request *http.Request) (*http.Response, error) {
if request.URL.Host != t.address {
return t.base.RoundTrip(request)
}
var response *http.Response
tries := 0
err := waitForRegistry(request.Context(), t.address, request.Method+" "+request.URL.Path, func() error {
attempt := request
if tries > 0 && request.Body != nil && request.Body != http.NoBody {
// A body is sent again only when it can be read again; one that cannot is not retried.
if request.GetBody == nil {
return fmt.Errorf("%s %s cannot be sent again: its body cannot be read twice", request.Method, request.URL)
}
body, err := request.GetBody()
if err != nil {
return err
}
attempt = request.Clone(request.Context())
attempt.Body = body
}
tries++
var err error
response, err = t.base.RoundTrip(attempt)
return err
})
return response, err
}
// waiting is a client like c whose requests to the registry wait for it.
func waiting(c *http.Client, address string) *http.Client {
if _, already := c.Transport.(waitingTransport); already {
return c
}
copied := *c
base := c.Transport
if base == nil {
base = http.DefaultTransport
}
copied.Transport = waitingTransport{base: base, address: address}
return &copied
}
internal/builder/registry_wait_test.go
-154
View File
@@ -1,154 +0,0 @@
package builder
import (
"context"
"crypto/sha256"
"encoding/hex"
"net"
"net/http"
"strings"
"sync"
"testing"
"time"
)
// A registry held still for a while (novox/hq issue 457): the store's nightly collection stops it for
// about a minute and a half, and a build in that window was failed, and its whole plan with it, for a
// pause the mesh itself scheduled. A build waits it out — for a bound longer than the collection
// holds it — says so in its log, and still fails, loudly, past the bound.
// heldStill is a registry address that refuses every connection until it starts answering after
// pause, or never when pause is negative.
func heldStill(t *testing.T, f *fakeRegistry, pause time.Duration) string {
t.Helper()
handler := f.serve(t).Config.Handler
reserved, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
address := reserved.Addr().String()
reserved.Close() // refused from here on: nothing listens
if pause < 0 {
return address
}
server := &http.Server{Handler: handler}
go func() {
time.Sleep(pause)
l, err := net.Listen("tcp", address)
if err != nil {
t.Errorf("cannot answer at %s again: %v", address, err)
return
}
_ = server.Serve(l)
}()
t.Cleanup(func() { _ = server.Close() })
return address
}
// saying collects what a build says, as the build machine's per-build Said does.
func saying(t *testing.T) func() []string {
t.Helper()
var mu sync.Mutex
var lines []string
was := Said
Said = func(step, message string) {
mu.Lock()
defer mu.Unlock()
lines = append(lines, step+": "+message)
}
t.Cleanup(func() { Said = was })
return func() []string {
mu.Lock()
defer mu.Unlock()
return append([]string(nil), lines...)
}
}
// waitingFor shortens the bound and the first pause, so a test waits for milliseconds.
func waitingFor(t *testing.T, bound time.Duration) {
t.Helper()
wasBound, wasFirst := registryWait, registryFirstPause
registryWait, registryFirstPause = bound, 20*time.Millisecond
t.Cleanup(func() { registryWait, registryFirstPause = wasBound, wasFirst })
}
func TestABuildWaitsForARegistryHeldStillAndGoesOn(t *testing.T) {
waitingFor(t, 5*time.Second)
said := saying(t)
f := &fakeRegistry{}
r := Registry{Address: heldStill(t, f, 400*time.Millisecond)}
body := []byte("a theme")
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
if _, err := r.PublishArchive(context.Background(), "shell/config", body, digest); err != nil {
t.Fatalf("a registry refusing for 400ms failed the build: %v", err)
}
if string(f.blobs[digest]) != "a theme" {
t.Fatalf("the registry holds %q", f.blobs[digest])
}
log := strings.Join(said(), "\n")
if !strings.Contains(log, "waits for the registry at "+r.Address) || !strings.Contains(log, "nightly collection") {
t.Errorf("the build's log does not say it waited for the registry, and why:\n%s", log)
}
if !strings.Contains(log, "answers again") {
t.Errorf("the build's log does not say the registry came back:\n%s", log)
}
}
func TestABuildFailsLoudlyOnARegistryRefusingPastTheBound(t *testing.T) {
waitingFor(t, 300*time.Millisecond)
said := saying(t)
f := &fakeRegistry{}
r := Registry{Address: heldStill(t, f, -1)}
body := []byte("a theme")
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
started := time.Now()
_, err := r.PublishArchive(context.Background(), "shell/config", body, digest)
if err == nil {
t.Fatal("a registry that never answered published the archive")
}
if !strings.Contains(err.Error(), "refused every connection for") || !strings.Contains(err.Error(), "connection refused") {
t.Errorf("the failure does not say it waited and was refused throughout: %v", err)
}
if waited := time.Since(started); waited < 300*time.Millisecond {
t.Errorf("failed after %s, before the bound", waited)
}
if log := strings.Join(said(), "\n"); !strings.Contains(log, "waits for the registry") {
t.Errorf("the build's log does not say it waited:\n%s", log)
}
}
func TestAnImagePushWaitsForARegistryHeldStill(t *testing.T) {
waitingFor(t, 5*time.Second)
said := saying(t)
pushes := 0
run := func(_ context.Context, _ string, name string, args ...string) (string, error) {
switch args[0] {
case "push":
pushes++
if pushes < 3 {
return "", errorString("docker push: dial tcp 127.0.0.1:5000: connect: connection refused")
}
case "inspect":
return "127.0.0.1:5000/m/server@sha256:abc\n", nil
}
return "", nil
}
r := Registry{Address: "127.0.0.1:5000", Run: run}
if _, err := r.PublishImage(context.Background(), "local", "m/server"); err != nil {
t.Fatalf("a push refused twice failed the build: %v", err)
}
if pushes != 3 {
t.Errorf("pushed %d times, want 3", pushes)
}
if log := strings.Join(said(), "\n"); !strings.Contains(log, "waits for the registry") {
t.Errorf("the build's log does not say it waited:\n%s", log)
}
}
type errorString string
func (e errorString) Error() string { return string(e) }
internal/builder/testdata/issue460/CAPTURED
Vendored
-7
View File
@@ -1,7 +0,0 @@
Captured 2026-10-11 from real `go test` runs (go1.27.1, linux/amd64) of a throwaway module whose
packages fail each way a repository's own check fails: a test failing early in a run its own logs
then fill (early: 600 log lines after the failures), a panic, a data race under -race, a build error,
and one -race run of every package. Only the module's directory was rewritten to /src/cap.
They are the output shapes of novox/hq issue 460: mesh-controller #218's check printed its
`--- FAIL:` line before the store-backed package's logs, which pushed it out of the report's tail.
internal/builder/testdata/issue460/a-build-error.txt
Vendored
-5
View File
@@ -1,5 +0,0 @@
# example.com/cap/broken
broken/broken.go:3:28: undefined: undefinedThing
FAIL example.com/cap/broken [build failed]
ok example.com/cap/fine (cached)
FAIL
internal/builder/testdata/issue460/a-data-race.txt
Vendored
-31
View File
@@ -1,31 +0,0 @@
==================
WARNING: DATA RACE
Read at 0x00c000018398 by goroutine 8:
example.com/cap/race.TestACounterIsShared.func1()
/src/cap/race/race_test.go:13 +0x7b
Previous write at 0x00c000018398 by goroutine 9:
example.com/cap/race.TestACounterIsShared.func1()
/src/cap/race/race_test.go:13 +0x8d
Goroutine 8 (running) created at:
example.com/cap/race.TestACounterIsShared()
/src/cap/race/race_test.go:13 +0x78
testing.tRunner()
/usr/lib/go/src/testing/testing.go:2193 +0x21c
testing.(*T).Run.gowrap1()
/usr/lib/go/src/testing/testing.go:2258 +0x38
Goroutine 9 (finished) created at:
example.com/cap/race.TestACounterIsShared()
/src/cap/race/race_test.go:13 +0x78
testing.tRunner()
/usr/lib/go/src/testing/testing.go:2193 +0x21c
testing.(*T).Run.gowrap1()
/usr/lib/go/src/testing/testing.go:2258 +0x38
==================
--- FAIL: TestACounterIsShared (0.00s)
testing.go:1865: race detected during execution of test
FAIL
FAIL example.com/cap/race 0.008s
FAIL
internal/builder/testdata/issue460/a-long-run-failing-early.txt
Vendored
-609
View File
@@ -1,609 +0,0 @@
--- FAIL: TestTheEnvelopeIsPinned (0.00s)
early_test.go:9: the envelope's subject is "mesh.a", not "mesh.b"
early_test.go:10: a second line of the same failure
--- FAIL: TestSubtests (0.00s)
--- FAIL: TestSubtests/the_hub (0.00s)
early_test.go:14: the hub does not compose
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d0
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d1
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d2
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d3
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d4
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d5
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d6
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d7
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d8
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d9
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d10
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d11
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d12
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d13
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d14
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d15
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d16
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d17
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d18
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d19
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d20
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d21
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d22
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d23
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d24
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d25
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d26
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d27
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d28
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d29
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d30
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d31
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d32
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d33
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d34
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d35
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d36
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d37
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d38
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d39
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d40
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d41
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d42
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d43
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d44
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d45
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d46
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d47
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d48
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d49
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d50
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d51
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d52
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d53
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d54
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d55
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d56
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d57
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d58
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d59
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d60
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d61
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d62
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d63
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d64
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d65
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d66
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d67
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d68
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d69
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d70
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d71
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d72
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d73
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d74
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d75
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d76
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d77
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d78
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d79
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d80
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d81
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d82
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d83
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d84
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d85
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d86
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d87
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d88
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d89
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d90
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d91
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d92
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d93
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d94
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d95
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d96
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d97
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d98
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d99
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d100
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d101
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d102
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d103
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d104
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d105
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d106
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d107
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d108
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d109
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d110
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d111
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d112
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d113
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d114
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d115
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d116
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d117
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d118
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d119
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d120
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d121
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d122
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d123
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d124
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d125
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d126
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d127
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d128
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d129
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d130
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d131
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d132
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d133
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d134
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d135
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d136
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d137
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d138
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d139
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d140
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d141
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d142
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d143
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d144
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d145
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d146
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d147
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d148
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d149
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d150
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d151
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d152
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d153
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d154
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d155
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d156
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d157
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d158
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d159
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d160
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d161
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d162
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d163
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d164
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d165
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d166
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d167
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d168
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d169
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d170
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d171
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d172
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d173
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d174
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d175
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d176
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d177
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d178
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d179
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d180
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d181
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d182
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d183
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d184
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d185
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d186
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d187
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d188
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d189
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d190
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d191
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d192
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d193
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d194
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d195
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d196
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d197
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d198
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d199
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d200
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d201
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d202
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d203
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d204
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d205
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d206
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d207
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d208
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d209
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d210
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d211
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d212
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d213
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d214
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d215
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d216
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d217
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d218
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d219
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d220
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d221
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d222
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d223
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d224
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d225
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d226
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d227
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d228
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d229
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d230
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d231
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d232
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d233
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d234
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d235
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d236
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d237
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d238
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d239
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d240
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d241
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d242
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d243
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d244
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d245
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d246
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d247
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d248
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d249
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d250
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d251
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d252
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d253
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d254
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d255
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d256
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d257
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d258
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d259
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d260
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d261
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d262
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d263
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d264
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d265
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d266
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d267
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d268
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d269
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d270
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d271
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d272
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d273
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d274
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d275
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d276
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d277
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d278
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d279
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d280
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d281
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d282
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d283
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d284
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d285
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d286
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d287
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d288
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d289
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d290
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d291
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d292
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d293
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d294
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d295
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d296
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d297
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d298
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d299
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d300
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d301
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d302
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d303
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d304
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d305
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d306
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d307
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d308
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d309
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d310
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d311
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d312
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d313
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d314
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d315
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d316
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d317
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d318
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d319
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d320
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d321
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d322
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d323
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d324
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d325
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d326
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d327
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d328
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d329
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d330
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d331
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d332
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d333
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d334
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d335
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d336
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d337
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d338
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d339
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d340
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d341
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d342
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d343
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d344
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d345
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d346
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d347
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d348
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d349
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d350
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d351
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d352
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d353
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d354
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d355
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d356
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d357
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d358
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d359
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d360
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d361
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d362
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d363
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d364
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d365
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d366
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d367
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d368
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d369
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d370
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d371
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d372
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d373
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d374
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d375
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d376
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d377
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d378
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d379
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d380
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d381
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d382
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d383
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d384
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d385
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d386
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d387
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d388
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d389
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d390
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d391
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d392
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d393
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d394
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d395
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d396
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d397
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d398
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d399
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d400
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d401
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d402
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d403
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d404
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d405
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d406
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d407
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d408
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d409
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d410
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d411
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d412
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d413
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d414
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d415
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d416
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d417
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d418
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d419
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d420
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d421
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d422
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d423
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d424
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d425
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d426
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d427
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d428
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d429
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d430
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d431
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d432
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d433
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d434
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d435
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d436
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d437
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d438
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d439
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d440
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d441
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d442
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d443
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d444
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d445
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d446
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d447
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d448
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d449
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d450
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d451
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d452
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d453
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d454
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d455
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d456
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d457
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d458
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d459
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d460
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d461
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d462
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d463
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d464
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d465
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d466
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d467
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d468
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d469
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d470
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d471
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d472
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d473
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d474
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d475
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d476
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d477
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d478
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d479
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d480
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d481
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d482
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d483
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d484
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d485
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d486
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d487
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d488
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d489
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d490
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d491
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d492
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d493
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d494
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d495
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d496
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d497
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d498
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d499
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d500
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d501
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d502
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d503
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d504
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d505
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d506
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d507
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d508
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d509
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d510
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d511
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d512
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d513
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d514
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d515
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d516
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d517
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d518
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d519
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d520
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d521
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d522
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d523
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d524
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d525
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d526
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d527
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d528
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d529
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d530
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d531
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d532
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d533
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d534
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d535
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d536
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d537
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d538
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d539
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d540
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d541
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d542
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d543
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d544
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d545
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d546
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d547
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d548
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d549
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d550
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d551
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d552
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d553
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d554
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d555
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d556
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d557
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d558
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d559
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d560
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d561
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d562
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d563
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d564
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d565
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d566
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d567
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d568
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d569
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d570
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d571
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d572
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d573
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d574
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d575
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d576
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d577
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d578
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d579
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d580
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d581
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d582
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d583
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d584
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d585
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d586
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d587
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d588
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d589
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d590
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d591
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d592
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d593
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d594
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d595
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d596
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d597
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d598
2026/10/11 04:22:00 kept what home-server says about the machine, and refused its account of declaration d599
FAIL
FAIL example.com/cap/early 0.003s
FAIL
internal/builder/testdata/issue460/a-panic.txt
Vendored
-18
View File
@@ -1,18 +0,0 @@
--- FAIL: TestAMapIsNil (0.00s)
panic: assignment to entry in nil map [recovered, repanicked]
goroutine 18 [running]:
testing.tRunner.func1.2({0x6b6810, 0x6ee000})
/usr/lib/go/src/testing/testing.go:2123 +0x232
testing.tRunner.func1()
/usr/lib/go/src/testing/testing.go:2126 +0x329
panic({0x6b6810?, 0x6ee000?})
/usr/lib/go/src/runtime/panic.go:859 +0x125
example.com/cap/panics.TestAMapIsNil(0x1e97d59dc248?)
/src/cap/panics/panics_test.go:7 +0x28
testing.tRunner(0x1e97d59dc248, 0x6d41f8)
/usr/lib/go/src/testing/testing.go:2193 +0xea
created by testing.(*T).Run in goroutine 1
/usr/lib/go/src/testing/testing.go:2258 +0x4d4
FAIL example.com/cap/panics 0.004s
FAIL
internal/builder/testdata/issue460/a-race-run-of-every-package.txt
Vendored
-660
View File
@@ -1,660 +0,0 @@
# example.com/cap/broken
broken/broken.go:3:28: undefined: undefinedThing
FAIL example.com/cap/broken [build failed]
--- FAIL: TestTheEnvelopeIsPinned (0.00s)
early_test.go:9: the envelope's subject is "mesh.a", not "mesh.b"
early_test.go:10: a second line of the same failure
--- FAIL: TestSubtests (0.00s)
--- FAIL: TestSubtests/the_hub (0.00s)
early_test.go:14: the hub does not compose
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d0
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d1
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d2
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d3
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d4
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d5
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d6
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d7
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d8
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d9
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d10
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d11
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d12
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d13
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d14
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d15
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d16
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d17
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d18
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d19
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d20
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d21
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d22
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d23
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d24
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d25
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d26
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d27
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d28
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d29
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d30
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d31
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d32
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d33
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d34
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d35
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d36
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d37
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d38
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d39
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d40
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d41
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d42
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d43
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d44
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d45
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d46
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d47
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d48
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d49
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d50
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d51
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d52
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d53
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d54
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d55
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d56
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d57
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d58
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d59
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d60
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d61
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d62
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d63
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d64
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d65
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d66
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d67
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d68
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d69
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d70
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d71
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d72
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d73
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d74
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d75
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d76
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d77
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d78
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d79
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d80
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d81
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d82
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d83
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d84
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d85
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d86
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d87
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d88
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d89
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d90
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d91
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d92
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d93
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d94
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d95
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d96
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d97
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d98
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d99
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d100
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d101
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d102
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d103
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d104
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d105
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d106
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d107
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d108
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d109
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d110
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d111
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d112
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d113
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d114
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d115
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d116
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d117
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d118
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d119
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d120
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d121
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d122
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d123
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d124
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d125
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d126
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d127
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d128
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d129
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d130
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d131
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d132
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d133
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d134
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d135
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d136
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d137
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d138
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d139
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d140
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d141
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d142
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d143
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d144
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d145
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d146
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d147
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d148
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d149
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d150
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d151
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d152
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d153
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d154
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d155
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d156
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d157
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d158
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d159
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d160
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d161
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d162
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d163
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d164
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d165
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d166
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d167
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d168
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d169
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d170
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d171
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d172
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d173
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d174
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d175
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d176
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d177
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d178
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d179
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d180
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d181
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d182
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d183
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d184
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d185
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d186
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d187
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d188
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d189
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d190
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d191
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d192
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d193
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d194
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d195
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d196
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d197
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d198
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d199
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d200
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d201
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d202
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d203
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d204
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d205
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d206
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d207
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d208
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d209
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d210
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d211
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d212
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d213
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d214
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d215
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d216
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d217
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d218
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d219
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d220
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d221
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d222
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d223
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d224
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d225
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d226
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d227
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d228
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d229
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d230
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d231
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d232
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d233
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d234
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d235
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d236
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d237
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d238
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d239
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d240
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d241
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d242
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d243
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d244
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d245
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d246
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d247
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d248
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d249
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d250
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d251
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d252
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d253
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d254
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d255
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d256
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d257
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d258
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d259
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d260
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d261
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d262
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d263
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d264
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d265
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d266
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d267
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d268
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d269
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d270
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d271
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d272
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d273
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d274
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d275
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d276
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d277
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d278
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d279
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d280
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d281
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d282
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d283
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d284
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d285
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d286
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d287
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d288
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d289
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d290
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d291
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d292
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d293
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d294
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d295
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d296
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d297
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d298
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d299
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d300
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d301
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d302
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d303
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d304
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d305
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d306
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d307
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d308
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d309
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d310
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d311
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d312
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d313
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d314
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d315
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d316
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d317
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d318
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d319
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d320
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d321
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d322
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d323
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d324
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d325
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d326
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d327
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d328
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d329
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d330
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d331
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d332
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d333
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d334
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d335
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d336
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d337
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d338
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d339
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d340
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d341
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d342
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d343
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d344
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d345
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d346
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d347
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d348
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d349
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d350
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d351
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d352
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d353
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d354
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d355
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d356
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d357
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d358
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d359
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d360
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d361
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d362
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d363
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d364
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d365
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d366
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d367
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d368
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d369
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d370
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d371
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d372
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d373
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d374
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d375
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d376
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d377
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d378
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d379
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d380
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d381
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d382
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d383
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d384
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d385
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d386
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d387
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d388
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d389
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d390
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d391
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d392
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d393
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d394
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d395
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d396
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d397
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d398
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d399
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d400
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d401
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d402
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d403
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d404
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d405
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d406
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d407
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d408
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d409
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d410
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d411
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d412
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d413
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d414
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d415
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d416
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d417
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d418
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d419
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d420
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d421
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d422
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d423
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d424
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d425
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d426
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d427
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d428
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d429
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d430
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d431
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d432
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d433
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d434
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d435
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d436
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d437
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d438
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d439
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d440
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d441
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d442
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d443
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d444
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d445
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d446
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d447
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d448
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d449
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d450
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d451
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d452
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d453
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d454
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d455
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d456
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d457
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d458
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d459
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d460
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d461
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d462
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d463
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d464
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d465
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d466
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d467
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d468
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d469
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d470
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d471
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d472
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d473
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d474
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d475
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d476
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d477
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d478
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d479
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d480
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d481
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d482
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d483
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d484
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d485
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d486
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d487
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d488
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d489
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d490
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d491
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d492
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d493
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d494
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d495
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d496
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d497
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d498
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d499
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d500
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d501
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d502
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d503
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d504
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d505
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d506
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d507
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d508
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d509
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d510
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d511
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d512
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d513
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d514
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d515
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d516
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d517
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d518
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d519
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d520
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d521
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d522
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d523
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d524
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d525
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d526
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d527
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d528
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d529
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d530
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d531
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d532
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d533
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d534
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d535
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d536
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d537
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d538
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d539
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d540
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d541
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d542
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d543
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d544
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d545
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d546
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d547
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d548
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d549
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d550
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d551
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d552
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d553
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d554
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d555
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d556
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d557
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d558
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d559
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d560
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d561
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d562
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d563
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d564
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d565
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d566
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d567
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d568
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d569
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d570
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d571
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d572
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d573
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d574
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d575
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d576
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d577
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d578
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d579
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d580
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d581
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d582
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d583
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d584
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d585
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d586
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d587
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d588
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d589
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d590
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d591
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d592
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d593
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d594
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d595
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d596
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d597
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d598
2026/10/11 04:21:53 kept what home-server says about the machine, and refused its account of declaration d599
FAIL
FAIL example.com/cap/early 0.013s
ok example.com/cap/fine (cached)
--- FAIL: TestAMapIsNil (0.00s)
panic: assignment to entry in nil map [recovered, repanicked]
goroutine 35 [running]:
testing.tRunner.func1.2({0x7c6e60, 0x806640})
/usr/lib/go/src/testing/testing.go:2123 +0x419
testing.tRunner.func1()
/usr/lib/go/src/testing/testing.go:2126 +0x65f
panic({0x7c6e60?, 0x806640?})
/usr/lib/go/src/runtime/panic.go:859 +0x125
example.com/cap/panics.TestAMapIsNil(0xc0001d6248?)
/src/cap/panics/panics_test.go:7 +0x32
testing.tRunner(0xc0001d6248, 0x7e5f78)
/usr/lib/go/src/testing/testing.go:2193 +0x21d
created by testing.(*T).Run in goroutine 1
/usr/lib/go/src/testing/testing.go:2258 +0xb13
FAIL example.com/cap/panics 0.011s
==================
WARNING: DATA RACE
Read at 0x00c000018398 by goroutine 9:
example.com/cap/race.TestACounterIsShared.func1()
/src/cap/race/race_test.go:13 +0x7b
Previous write at 0x00c000018398 by goroutine 8:
example.com/cap/race.TestACounterIsShared.func1()
/src/cap/race/race_test.go:13 +0x8d
Goroutine 9 (running) created at:
example.com/cap/race.TestACounterIsShared()
/src/cap/race/race_test.go:13 +0x78
testing.tRunner()
/usr/lib/go/src/testing/testing.go:2193 +0x21c
testing.(*T).Run.gowrap1()
/usr/lib/go/src/testing/testing.go:2258 +0x38
Goroutine 8 (finished) created at:
example.com/cap/race.TestACounterIsShared()
/src/cap/race/race_test.go:13 +0x78
testing.tRunner()
/usr/lib/go/src/testing/testing.go:2193 +0x21c
testing.(*T).Run.gowrap1()
/usr/lib/go/src/testing/testing.go:2258 +0x38
==================
--- FAIL: TestACounterIsShared (0.00s)
testing.go:1865: race detected during execution of test
FAIL
FAIL example.com/cap/race 0.009s
FAIL
internal/catalogue/amqp_is_not_a_provision_test.go
+4 -6
View File
@@ -5,8 +5,6 @@ import (
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
)
// **The word does not come back through a manifest** (novox/hq ADR 0131). A module that wants
@@ -30,12 +28,12 @@ func TestAManifestRequiringAmqpIsRefused(t *testing.T) {
}
}
// And the catalogue (internal/beside) names it nowhere — the three modules that did are removed under
// design 28 task 5.4, not converted.
// And the catalogue as checked out beside this repository names it nowhere — the three modules that
// did are removed under design 28 task 5.4, not converted.
func TestNoCatalogueManifestNamesAmqp(t *testing.T) {
modules, err := filepath.Glob(filepath.Join(beside.Catalogue(t), "*", "module.json"))
modules, err := filepath.Glob("../../../mesh-catalog/modules/*/module.json")
if err != nil || len(modules) == 0 {
t.Fatalf("no manifests in the catalogue, so this proved nothing: %v", err)
t.Skip("the catalogue is not checked out beside this repository")
}
for _, path := range modules {
raw, err := os.ReadFile(path)
internal/catalogue/bus_snapshot_test.go
+2 -5
View File
@@ -2,11 +2,8 @@ package catalogue
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
)
// The module holding mesh-broker is the bus, and its account is granted the bus's snapshot API and
@@ -37,9 +34,9 @@ func TestTheBussAccountSaysNothingOnTheBus(t *testing.T) {
// dump into its snapshots, it has the account the dump runs as, and the dump runs the snapshot
// program in the bus's own container.
func TestTheCataloguesBusIsBackedUpBySnapshot(t *testing.T) {
raw, err := os.ReadFile(filepath.Join(beside.Catalogue(t), "nats", "module.json"))
raw, err := os.ReadFile("../../../mesh-catalog/modules/nats/module.json")
if err != nil {
t.Fatal(err)
t.Skip("the catalogue is not checked out beside this repository")
}
m, err := ParseManifest(raw)
if err != nil {
internal/catalogue/catalogue_check_test.go
+12 -7
View File
@@ -5,21 +5,26 @@ import (
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
)
// TestEveryCatalogueManifestParses runs the real catalogue through the real gate.
//
// Not a fixture: the point is whether the manifests as written are accepted by the control plane that
// will read them, and a copy of one manifest proves nothing about the other seventy-one.
// catalogueRoot is the catalogue these checks run over: in a merge check the clone the build seat put
// beside this one, elsewhere the copy captured in testdata/beside (internal/beside). A check that only
// ran when somebody remembered a variable was a check nobody ran (novox/hq issue 134), and one that read
// whatever checkout sat beside judged the machine, not the change (novox/hq issue 432): it never skips.
// catalogueRoot is the catalogue these checks run over: MESH_CATALOGUE when set, else the checkout
// beside this one, the way the main layout has it. A check that only ran when somebody remembered a
// variable was a check nobody ran (novox/hq issue 134, 2026-09-30); it skips only when there is no
// catalogue to be found at all.
func catalogueRoot(t *testing.T) string {
t.Helper()
return beside.Dir(t, "mesh-catalog")
if root := os.Getenv("MESH_CATALOGUE"); root != "" {
return root
}
sibling := filepath.Join("..", "..", "..", "mesh-catalog")
if _, err := os.Stat(filepath.Join(sibling, "modules")); err != nil {
t.Skip("no catalogue beside this checkout and MESH_CATALOGUE unset")
}
return sibling
}
func TestEveryCatalogueManifestParses(t *testing.T) {
internal/catalogue/catrust_manifest_test.go
+2 -5
View File
@@ -2,11 +2,8 @@ package catalogue
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
)
// **A machine trusts the mesh's authority because a module put its root there** (novox/hq ADR
@@ -19,9 +16,9 @@ import (
// itself — a plain client trusting an internal name on a machine holding this, and failing on one
// that does not — is the lab's, and cannot be had here.
func TestCaTrustRendersTheAuthorityItWasBoundTo(t *testing.T) {
raw, err := os.ReadFile(filepath.Join(beside.Catalogue(t), "ca-trust", "module.json"))
raw, err := os.ReadFile("../../../mesh-catalog/modules/ca-trust/module.json")
if err != nil {
t.Fatal(err)
t.Skipf("the catalogue is not beside this checkout: %v", err)
}
m, err := ParseManifest(raw)
if err != nil {
internal/catalogue/declaration.go
+5 -39
View File
@@ -606,18 +606,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
"mode": "0600",
})
}
// **A secret family's members, as given** (novox/hq ADR 0283): one file each at the family's path, and
// nothing for a member not given — the mesh never makes one, and the module says it waits for it.
for _, name := range sortedKeys(with.Needed[m.Module]) {
own, family, ok := m.OwnSecrets.Lookup(name)
if !ok || family == "" || with.Needed[m.Module][name] == "" {
continue
}
first = append(first, ownedBy(m.SecretsOwner, map[string]any{
"id": NeedID(name), "type": "file", "path": own.Path, "sealed": with.Needed[m.Module][name],
}))
}
for _, name := range sortedKeys(m.OwnSecrets.Plain()) {
for _, name := range sortedKeys(m.OwnSecrets) {
sealed := with.Needed[m.Module][name]
if sealed == "" && with.Foreseen[m.Module][name] {
// Not made, and the next send makes it: composed with a stand-in so that whatever
@@ -911,15 +900,6 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
return nil, err
}
}
// **And every placeholder no pass fills where it stands is refused** (novox/hq issue 231):
// judged here, over the definition as written and before any pass, by the function the
// catalogue check runs — so a manifest registered by an older binary is judged too, and
// what a setting or a binding later puts into a file is its software's text, never swept.
for _, own := range m.Resources {
if problems := unconsumedPlaceholders(m.Module, own); len(problems) > 0 {
return nil, fmt.Errorf("%s", problems[0])
}
}
// Which of this module's files carry a secret, for the rule that a container may not read
// one of them as its environment without saying so (ADR 0086, issue 041).
@@ -1076,7 +1056,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// credential the mesh had replaced, because its manifest restarted it on its
// environment file and nobody had thought to name the credential too. Composed here so
// no manifest has to say it, for a container or a daemon that names the secret's path.
if reads := secretsReadBy(copied, m, with.Needed[m.Module]); len(reads) > 0 {
if reads := secretsReadBy(copied, m); len(reads) > 0 {
copied["restart-on"] = withRestartOn(copied["restart-on"], reads)
}
// **A version prepares its state before it runs** (novox/hq ADR 0135). Derived from the
@@ -2392,12 +2372,7 @@ func portOfEndpoint(values map[string]any, ports map[string]int) {
// — named in its volumes, its environment or its env-files by the secret's placed path — as
// restart-on ids. Nothing for other shapes, and nothing for a scheduled or run-once process, which
// the host refuses a restart-on for (it runs again anyway, and reads the file afresh).
//
// **A member of a secret family given here is read the same way** (novox/hq issue 405, ADR 0283 decision 6):
// it is an own secret placed at its own path, and a container that mounted it would keep the value it
// started with when the operator gives it again. given is the module's own secrets sealed to this
// machine; a member not given is no file, so nothing restarts on it.
func secretsReadBy(resource map[string]any, m Manifest, given map[string]string) []string {
func secretsReadBy(resource map[string]any, m Manifest) []string {
kind := fmt.Sprint(resource["type"])
if kind != "container" && kind != "process" {
return nil
@@ -2409,18 +2384,9 @@ func secretsReadBy(resource map[string]any, m Manifest, given map[string]string)
for _, key := range []string{"volumes", "env", "env-file"} {
mentioned = append(mentioned, stringsIn(resource[key])...)
}
paths := map[string]string{}
for name, own := range m.OwnSecrets.Plain() {
paths[name] = own.Path
}
for name, sealed := range given {
if own, family, ok := m.OwnSecrets.Lookup(name); ok && family != "" && sealed != "" {
paths[name] = own.Path
}
}
var out []string
for _, name := range sortedKeys(paths) {
path := paths[name]
for _, name := range sortedKeys(m.OwnSecrets) {
path := m.OwnSecrets[name].Path
if path == "" {
continue
}
internal/catalogue/delivery_seat.go
-9
View File
@@ -18,15 +18,6 @@ func deliveryVerbs() []Verb {
Input: schema(map[string]string{"state": "one state, e.g. delivering or held",
"repository": "owner/repository", "group": "a group's id (its branch name)",
"all": "\"true\": the final ones of the last thirty days too"}, nil, "all")},
// Delivery time against the delivery budgets (novox/hq ADR 0282): what probe D16 reads, optional until its
// holder serves it.
{Name: "times", Description: "Delivery time: from a merge to every machine of its walk running the build. " +
"Each class's delivery budget (a leaf module five minutes, a core module ten), the last days' median and " +
"worst, how many within and over, the newest delivery of each class, every delivery over its budget with its " +
"longest phase, and the delivered ones whose time is not known. Given a delivery's id, its time phase by phase.",
Input: schema(map[string]string{"days": "how many days back (default 7)", "id": "one delivery's id: its phases"}, nil),
// Optional until mesh-delivery serves it: its holder lives in the catalogue (design 33 §7).
Optional: true},
{Name: "show", Description: "One delivery or group whole: its delivery plan (what it builds, what each " +
"machine receives, what is not an ordinary send), every transition with when and why, the machine " +
"steps of its walk, its group and its order.",
internal/catalogue/delivery_seat_test.go
-36
View File
@@ -97,39 +97,3 @@ func TestTheDeliverySeatPromisesRetireHistoryOptionally(t *testing.T) {
t.Fatalf("a holder serving retire-history: %v", err)
}
}
// The seat says delivery times (novox/hq ADR 0282, issue 382): `times`, over some days or for one delivery. Added
// after the holder shipped, it is optional, so the holder that does not serve it yet still holds the seat, and one
// that does is not refused for a verb the seat lacks.
func TestTheDeliverySeatPromisesTimesOptionally(t *testing.T) {
seat, _ := SeatNamed(DeliverySeat)
var times *Verb
for i := range seat.Serves {
if seat.Serves[i].Name == "times" {
times = &seat.Serves[i]
}
}
if times == nil || !times.Optional {
t.Fatalf("the delivery seat promises %v, times optionally", VerbNames(seat.Serves))
}
props, _ := times.Input["properties"].(map[string]any)
for _, arg := range []string{"days", "id"} {
if _, has := props[arg]; !has {
t.Errorf("times takes no %q", arg)
}
}
var without []string
for _, v := range VerbNames(seat.Serves) {
if v != "times" {
without = append(without, v)
}
}
m := Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh, Serves: without}}}
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder without times yet: %v", err)
}
m.Claims[0].Serves = VerbNames(seat.Serves)
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder serving times: %v", err)
}
}
internal/catalogue/environment_into.go
-3
View File
@@ -226,9 +226,6 @@ func (m Manifest) contributionPlaceholderProblems() []string {
for _, r := range m.Resources {
problems = append(problems, placeholderProblems(m, r)...)
problems = append(problems, seatPlaceholderProblems(m, r)...)
// And every placeholder no pass fills where it stands: a misspelt namespace, a key its
// namespace cannot take, or a field its pass does not read (novox/hq issue 231).
problems = append(problems, unconsumedPlaceholders(m.Module, r)...)
}
return problems
}
internal/catalogue/environment_into_test.go
+15 -49
View File
@@ -134,61 +134,27 @@ func TestThePOSIXEnvironmentSourcedTwiceLeavesPATHAsOnce(t *testing.T) {
// The environment.d rendering, read by the service manager's own generator where this machine has
// one — the same reader an account's user manager runs, so the PATH it composes is the one asserted.
//
// The generator also reads the machine's own environment.d (/etc, /run, /usr/lib, /usr/local/lib), and
// no option points it elsewhere: a desktop whose snapd appends its bin directory failed this, on main,
// for a file the mesh never wrote (novox/hq issue 432). So the generator is run twice, once without the
// mesh's file, and what the machine's own files make of PATH and set is held out of the verdict.
//
// A machine without the generator — the build seat's toolchain image holds no systemd — cannot judge
// this and says so: there the rendering is held byte for byte by
// TestTheEnvironmentRendersForTheServiceManagerByteForByte, and this reading only where systemd runs.
func TestTheServiceManagerReadsTheSystemdRenderingAsMeant(t *testing.T) {
generator := "/usr/lib/systemd/user-environment-generators/30-systemd-environment-d-generator"
if _, err := os.Stat(generator); err != nil {
t.Skip("NOT JUDGED: no environment.d generator on this machine, so the service manager's reading " +
"of the rendering is judged only where systemd runs; the rendering itself is held byte for byte " +
"by TestTheEnvironmentRendersForTheServiceManagerByteForByte")
t.Skip("no environment.d generator on this machine")
}
const base = "/usr/bin:/bin"
read := func(conf string) map[string]string {
t.Helper()
config := t.TempDir()
if err := os.MkdirAll(filepath.Join(config, "environment.d"), 0o755); err != nil {
t.Fatal(err)
}
if conf != "" {
if err := os.WriteFile(filepath.Join(config, "environment.d", "50-mesh.conf"), []byte(conf), 0o644); err != nil {
t.Fatal(err)
}
}
cmd := exec.Command(generator)
cmd.Env = []string{"PATH=" + base, "HOME=" + config, "XDG_CONFIG_HOME=" + config}
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("%v\n%s", err, out)
}
vars := map[string]string{}
for _, line := range strings.Split(strings.TrimSpace(string(out)), "\n") {
if k, v, ok := strings.Cut(line, "="); ok {
vars[k] = v
}
}
return vars
config := t.TempDir()
if err := os.MkdirAll(filepath.Join(config, "environment.d"), 0o755); err != nil {
t.Fatal(err)
}
machine, read50 := read(""), read(composedSystemd)
// What the machine's own files do to PATH: nothing, or append to it. One that replaces or prepends
// leaves nothing this test can say about the mesh's file, and says so rather than guess.
added, ok := strings.CutPrefix(machine["PATH"], base)
if machine["PATH"] != "" && !ok {
t.Skipf("NOT JUDGED: this machine's own environment.d sets PATH to %s, not %s with something after it, so "+
"what the mesh's file adds cannot be told apart from it", machine["PATH"], base)
if err := os.WriteFile(filepath.Join(config, "environment.d", "50-mesh.conf"), []byte(composedSystemd), 0o644); err != nil {
t.Fatal(err)
}
want := "/home/op/go/bin:/usr/local/go/bin:/home/op/.local/bin:/home/op/bin:/usr/bin:/bin:/opt/agent/bin:/opt/scripts" + added
if read50["PATH"] != want || read50["GOPATH"] != "/home/op/go" {
t.Fatalf("the service manager read PATH=%s GOPATH=%s, not PATH=%s GOPATH=/home/op/go (the machine's own "+
"files add %q to PATH)", read50["PATH"], read50["GOPATH"], want, added)
cmd := exec.Command(generator)
cmd.Env = []string{"PATH=/usr/bin:/bin", "HOME=" + config, "XDG_CONFIG_HOME=" + config}
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("%v\n%s", err, out)
}
want := "PATH=/home/op/go/bin:/usr/local/go/bin:/home/op/.local/bin:/home/op/bin:/usr/bin:/bin:/opt/agent/bin:/opt/scripts"
if !strings.Contains(string(out), want+"\n") || !strings.Contains(string(out), "GOPATH=/home/op/go\n") {
t.Fatalf("the service manager read\n%s", out)
}
}
internal/catalogue/foundation_manifests_test.go
+4 -12
View File
@@ -4,24 +4,19 @@ import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
)
// The catalogue's foundation modules as they are — in a merge check the catalogue cloned beside it,
// elsewhere the copy captured in testdata/beside (internal/beside, novox/hq issue 432) — parsed by the
// real parser (novox/hq ADR 0100):
// The catalogue's foundation modules as they are, parsed by the real parser (novox/hq ADR 0100):
// the store and the broker say which of their ports the mesh guards on an adopted node, and the
// filter module loads its table through a unit of its own whose stop deletes only that table.
func catalogueManifest(t *testing.T, module string) Manifest {
t.Helper()
raw, err := os.ReadFile(filepath.Join(beside.Catalogue(t), module, "module.json"))
raw, err := os.ReadFile("../../../mesh-catalog/modules/" + module + "/module.json")
if err != nil {
t.Fatal(err)
t.Skipf("the catalogue is not beside this checkout: %v", err)
}
m, err := ParseManifest(raw)
if err != nil {
@@ -130,11 +125,8 @@ func TestTheForgesPortIsGivenLikeAnyOtherProvidersPort(t *testing.T) {
// seat's holder the answer when more than one module provides it — so a carried copy would be a
// second answer to the same question, free to drift from the first. Asserted gone, not merely
// unused.
//
// The builder is the build-agent on every machine since novox/hq ADR 0190; this read the retired
// `builder` and, finding no manifest, skipped unseen from then until novox/hq issue 432.
func TestTheBuilderRequiresTheRegistryTheNpmSeatDelivers(t *testing.T) {
builder := catalogueManifest(t, "build-agent")
builder := catalogueManifest(t, "builder")
seat, _ := SeatNamed("npm-package-registry")
var requires bool
for _, r := range builder.Requires {
internal/catalogue/logs.go
-105
View File
@@ -1,105 +0,0 @@
package catalogue
import (
"bytes"
"encoding/json"
"fmt"
)
// What a module may call its logs (novox/hq ADR 0297).
//
// A log is a module's record of operations: entries appended under a key, each kept as long as the
// log is, in the order they came. A module names each log it owns **locally** — `changes`, never a
// stream or a subject (ADR 0201 §4) — and the mesh derives the stream from the module and the local
// name, as it derives a bucket. So the rule for a log's name is a state name's: one plain token, and
// a module that keeps a log has a name that is one plain token too.
// The mesh's caps on a log, in MiB: what a module may ask, and what it gets when it asks nothing.
const (
LogLeastMiB = 1
LogMostMiB = 8192
LogDefaultMiB = 1024
)
// LogDeclaration is one log a module owns: its local name, and how large it may grow.
type LogDeclaration struct {
Name string `json:"name"`
// MaxMiB is the log's cap in MiB; zero is LogDefaultMiB. When full, the log refuses new entries
// and never drops old ones.
MaxMiB int `json:"max-mib,omitempty"`
}
// Cap is the log's cap in MiB, the default where none is said.
func (l LogDeclaration) Cap() int {
if l.MaxMiB == 0 {
return LogDefaultMiB
}
return l.MaxMiB
}
// UnmarshalJSON reads a log as its bare name, or as {name, max-mib}.
func (l *LogDeclaration) UnmarshalJSON(raw []byte) error {
trimmed := bytes.TrimSpace(raw)
if len(trimmed) > 0 && trimmed[0] == '"' {
return json.Unmarshal(trimmed, &l.Name)
}
var full struct {
Name string `json:"name"`
MaxMiB *int `json:"max-mib"`
}
dec := json.NewDecoder(bytes.NewReader(trimmed))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a log is either a name or {name, max-mib}: %w", typedUnknown(err))
}
l.Name = full.Name
l.MaxMiB = 0
if full.MaxMiB != nil {
// Said, and said as nothing: refused rather than read as the default, which it did not say.
if *full.MaxMiB == 0 {
return fmt.Errorf("log %q: max-mib is between %d and %d, not 0", full.Name, LogLeastMiB, LogMostMiB)
}
l.MaxMiB = *full.MaxMiB
}
return nil
}
// MarshalJSON writes back the short form when there is nothing else to say.
func (l LogDeclaration) MarshalJSON() ([]byte, error) {
if l.MaxMiB == 0 {
return json.Marshal(l.Name)
}
type plain LogDeclaration
return json.Marshal(plain(l))
}
// LogProblems is what is wrong with a manifest's logs.
//
// Refused at registration, for a bucket's reason: a stream name the bus cannot hold, or a cap the
// mesh would not grant, is a module that installs, starts, and is refused on its first append.
func LogProblems(m Manifest) []string {
var problems []string
if len(m.Logs) > 0 && !stateName.MatchString(m.Module) {
problems = append(problems, fmt.Sprintf(
"%s keeps a log, and a module's name is part of its logs' names, which take one plain "+
"name — no dot (novox/hq ADR 0297)", m.Module))
}
seen := map[string]bool{}
for _, l := range m.Logs {
switch {
case !stateName.MatchString(l.Name):
problems = append(problems, fmt.Sprintf(
"%s keeps log %q: a log is named locally — lower-case letters, digits and hyphens, "+
"no dot and no underscore; the mesh derives the stream (novox/hq ADR 0297)", m.Module, l.Name))
case seen[l.Name]:
problems = append(problems, fmt.Sprintf("%s keeps log %q twice", m.Module, l.Name))
}
seen[l.Name] = true
if l.MaxMiB != 0 && (l.MaxMiB < LogLeastMiB || l.MaxMiB > LogMostMiB) {
problems = append(problems, fmt.Sprintf(
"%s caps log %q at %d MiB; a log holds between %d and %d MiB (novox/hq ADR 0297)",
m.Module, l.Name, l.MaxMiB, LogLeastMiB, LogMostMiB))
}
}
return problems
}
internal/catalogue/logs_test.go
-68
View File
@@ -1,68 +0,0 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
// A module declares the logs it keeps (novox/hq ADR 0297 §1): a log by its bare name, or with its cap
// in MiB; the default cap where none is said.
func TestAManifestMaySayWhatLogsItKeeps(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"mesh-issues","version":"1",` +
`"logs":["changes",{"name":"moves","max-mib":2048}]}`))
if err != nil {
t.Fatal(err)
}
if len(m.Logs) != 2 || m.Logs[0].Name != "changes" || m.Logs[1].Name != "moves" {
t.Fatalf("logs not read: %+v", m.Logs)
}
if m.Logs[0].Cap() != LogDefaultMiB || m.Logs[0].Cap() != 1024 || m.Logs[1].Cap() != 2048 {
t.Fatalf("caps read as %d and %d", m.Logs[0].Cap(), m.Logs[1].Cap())
}
out, _ := json.Marshal(m.Logs)
if string(out) != `["changes",{"name":"moves","max-mib":2048}]` {
t.Fatalf("written back as %s", out)
}
for _, edge := range []string{`{"name":"a","max-mib":1}`, `{"name":"a","max-mib":8192}`} {
if _, err := ParseManifest([]byte(`{"module":"a","version":"1","logs":[` + edge + `]}`)); err != nil {
t.Errorf("%s is inside the caps and was refused: %v", edge, err)
}
}
}
// The catalogue check refuses a log outside the caps, with a name that is not one plain token, with a
// field the mesh does not know, or kept by a module whose own name is not one plain token.
func TestALogIsNamedLocallyAndCapped(t *testing.T) {
for _, c := range []struct{ manifest, says string }{
{`{"module":"a","version":"1","logs":["mesh.changes"]}`, `keeps log "mesh.changes": a log is named locally`},
{`{"module":"a","version":"1","logs":["my_changes"]}`, `keeps log "my_changes"`},
{`{"module":"a","version":"1","logs":["Changes"]}`, `keeps log "Changes"`},
{`{"module":"a","version":"1","logs":["c","c"]}`, `keeps log "c" twice`},
{`{"module":"a","version":"1","logs":[{"name":"c","max-mib":8193}]}`, `between 1 and 8192 MiB`},
{`{"module":"a","version":"1","logs":[{"name":"c","max-mib":-1}]}`, `between 1 and 8192 MiB`},
{`{"module":"a","version":"1","logs":[{"name":"c","max-mib":0}]}`, `max-mib is between 1 and 8192, not 0`},
{`{"module":"a","version":"1","logs":[{"name":"c","stream":"LOG_x"}]}`, `{name, max-mib}`},
{`{"module":"a.b","version":"1","logs":["c"]}`, `no dot`},
} {
_, err := ParseManifest([]byte(c.manifest))
if err == nil {
t.Errorf("%s was accepted", c.manifest)
continue
}
if !strings.Contains(err.Error(), c.says) {
t.Errorf("%s refused for the wrong reason: %v", c.manifest, err)
}
}
}
// **Across the whole catalogue**: every log is named locally and capped within the mesh's caps.
func TestEveryManifestsLogsAreLocalAndCapped(t *testing.T) {
var problems []string
for _, m := range theCatalogue(t) {
problems = append(problems, LogProblems(m)...)
}
if len(problems) > 0 {
t.Fatalf("the catalogue's logs are not what ADR 0297 says:\n %s", strings.Join(problems, "\n "))
}
}
internal/catalogue/manifest.go
+1 -102
View File
@@ -465,11 +465,6 @@ type Manifest struct {
// (novox/hq ADR 0201). Not history — that is an event — and never a secret, sealed or not.
State []StateDeclaration `json:"state,omitempty"`
// Logs are the logs of operations this module keeps on the bus, by local name: each a stream the
// controller creates and never removes, which every instance of the module appends to and reads
// (novox/hq ADR 0297). A log is its owner's alone: no other module reads it.
Logs []LogDeclaration `json:"logs,omitempty"`
// Settings are the defaults this module gives its settings (novox/hq ADR 0262): each key a file,
// a contribution or a served fact asks for as `${setting:<key>}`, its default, and why that
// default. Only a preference has one — a font size, a width, a number of workers — and a value
@@ -1512,7 +1507,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
name string
n int
}{{"consumes", len(m.Consumes)}, {"uses", len(m.Uses)}, {"invokes", len(m.Invokes)},
{"state", len(m.State)}, {"logs", len(m.Logs)}, {"reads", len(m.Reads)}} {
{"state", len(m.State)}, {"reads", len(m.Reads)}} {
if f.n > 0 {
said = append(said, f.name)
}
@@ -1623,8 +1618,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
problems = append(problems, EventProblems(m)...)
// And what it may call its state, and whose it may read (state.go, novox/hq ADR 0201).
problems = append(problems, StateProblems(m)...)
// And what it may call its logs, and how large it may ask them to grow (logs.go, novox/hq ADR 0297).
problems = append(problems, LogProblems(m)...)
// And the defaults it gives its settings (setting_defaults.go, novox/hq ADR 0262).
problems = append(problems, SettingProblems(m)...)
wellFormed := true
@@ -1949,37 +1942,6 @@ func ParseManifest(raw []byte) (Manifest, error) {
problems = append(problems, whileStoppedProblems(m, r, hasSchedule(r))...)
}
for name, own := range m.OwnSecrets {
// **A family is issued outside the mesh, and lands one file per member** (novox/hq ADR 0283): the mesh
// never makes a member, so a family the mesh may make would be one nothing ever fills.
if IsFamily(name) {
if !memberRest.MatchString(strings.TrimSuffix(FamilyPrefix(name), "-")) {
problems = append(problems, fmt.Sprintf(
"%s declares the secret family %q, whose prefix is not a name", m.Module, name))
}
if own.IssuedBy != IssuedOutside {
problems = append(problems, fmt.Sprintf(
"%s declares the secret family %q without \"issued-by\": %q; the mesh never makes a "+
"member of a family, so only a party outside the mesh can fill one (novox/hq ADR 0283)",
m.Module, name, IssuedOutside))
}
if strings.Count(own.Path, "*") != 1 {
problems = append(problems, fmt.Sprintf(
"%s keeps the secret family %q at %q, which does not hold exactly one *: each member lands "+
"where the * is replaced by its name (novox/hq ADR 0283)", m.Module, name, own.Path))
}
for other := range m.OwnSecrets {
if other != name && !IsFamily(other) {
if rest := strings.TrimPrefix(other, FamilyPrefix(name)); rest != other && memberRest.MatchString(rest) {
problems = append(problems, fmt.Sprintf(
"%s declares the secret %q, which is also a member of its family %q — a name names one",
m.Module, other, name))
}
}
}
} else if strings.Contains(name, "*") {
problems = append(problems, fmt.Sprintf(
"%s declares the secret %q: a * only ends a family's name, as \"<prefix>-*\"", m.Module, name))
}
if !placedOrAbsolute(own.Path) {
problems = append(problems, fmt.Sprintf(
"%s needs %q at %q, which is neither an absolute path nor a placed one", m.Module, name, own.Path))
@@ -2587,69 +2549,6 @@ func (o OwnSecrets) MarshalJSON() ([]byte, error) {
return json.Marshal(entries)
}
// A secret family (novox/hq ADR 0283): an own secret declared under a name ending in FamilySuffix is one
// member per part the module's settings name — `smb-password-*` holds `smb-password-games`,
// `smb-password-library` — each given by its full name, placed at the family's path with its one `*` replaced
// by what follows the prefix. The mesh never makes a member: a family is issued outside the mesh, and a member
// not given is no file.
const FamilySuffix = "-*"
// memberRest is what may follow a family's prefix: a name's characters.
var memberRest = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
// IsFamily says an own secret's declared name is a family's.
func IsFamily(name string) bool { return strings.HasSuffix(name, FamilySuffix) }
// FamilyPrefix is what every member of a family begins with: the declared name without its `*`.
func FamilyPrefix(family string) string { return strings.TrimSuffix(family, "*") }
// Lookup resolves an own secret by the name it is given under: declared by that name, or a member of a family
// (the longest prefix that fits), with the family's path filled for the member. family is the family's
// declared name, or "" for a secret declared by name.
func (o OwnSecrets) Lookup(name string) (s OwnSecret, family string, ok bool) {
if s, ok := o[name]; ok && !IsFamily(name) {
return s, "", true
}
for declared, f := range o {
if !IsFamily(declared) {
continue
}
prefix := FamilyPrefix(declared)
rest := strings.TrimPrefix(name, prefix)
if !strings.HasPrefix(name, prefix) || !memberRest.MatchString(rest) {
continue
}
if family == "" || len(declared) > len(family) {
s, family, ok = OwnSecret{Path: strings.Replace(f.Path, "*", rest, 1), Taken: f.Taken, IssuedBy: f.IssuedBy}, declared, true
}
}
return s, family, ok
}
// Plain is every own secret declared by its own name: what the mesh makes when not given, and places by
// name. A family is not one, and is never made.
func (o OwnSecrets) Plain() OwnSecrets {
out := make(OwnSecrets, len(o))
for name, s := range o {
if !IsFamily(name) {
out[name] = s
}
}
return out
}
// Families is every family's declared name, sorted.
func (o OwnSecrets) Families() []string {
var out []string
for name := range o {
if IsFamily(name) {
out = append(out, name)
}
}
sort.Strings(out)
return out
}
// Paths is each own secret's path by name — the shape every placement and file walk reads.
func (o OwnSecrets) Paths() map[string]string {
out := make(map[string]string, len(o))
internal/catalogue/mounts_test.go
+8 -6
View File
@@ -5,8 +5,6 @@ import (
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
)
// A bind mount the module never declared is refused where it is written (novox/hq 04-ISSUES/026,
@@ -69,12 +67,16 @@ func TestTheRuntimeSocketIsGrantedByTheCapabilityAndNotOtherwise(t *testing.T) {
}
}
// **Every manifest in the catalogue (internal/beside) passes**, so the rule is not one the catalogue
// is already breaking.
// **Every manifest in the catalogue beside this checkout passes**, so the rule is not one the
// catalogue is already breaking. Skipped, aloud, where the catalogue is not there.
func TestEveryCatalogueManifestDeclaresWhatItMounts(t *testing.T) {
files, _ := filepath.Glob(filepath.Join(beside.Catalogue(t), "*", "module.json"))
root := os.Getenv("MESH_CATALOG")
if root == "" {
root = "../../../mesh-catalog"
}
files, _ := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if len(files) == 0 {
t.Fatal("no manifests in the catalogue, so this proved nothing")
t.Skipf("no catalogue at %s (set MESH_CATALOG to a checkout)", root)
}
for _, file := range files {
raw, err := os.ReadFile(file)
internal/catalogue/no_subjects_test.go
+7 -10
View File
@@ -7,8 +7,6 @@ import (
"regexp"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/beside"
)
// **A manifest holds no subject** (novox/hq design 29 §1).
@@ -19,10 +17,10 @@ import (
// held by construction is one a later field breaks quietly, with the symptom appearing as a
// permission that does not match a subject rather than as a manifest that was wrong.
func TestNoManifestContainsASubject(t *testing.T) {
root := beside.Catalogue(t)
root := filepath.Join("..", "..", "..", "mesh-catalog", "modules")
entries, err := os.ReadDir(root)
if err != nil {
t.Fatal(err)
t.Skipf("catalogue sibling not present: %v", err)
}
// Anything in the mesh's own subject space, and anything shaped like a wire address.
@@ -65,7 +63,7 @@ func TestNoManifestContainsASubject(t *testing.T) {
walk(e.Name(), "", m)
}
if checked == 0 {
t.Fatal("no manifests read, so this proved nothing")
t.Skip("no manifests read")
}
if len(found) > 0 {
t.Errorf("a manifest names a subject, so reorganising the subject space would mean "+
@@ -93,14 +91,13 @@ func TestEveryManifestsEventNamesAreLocal(t *testing.T) {
}
}
// theCatalogue is every manifest of the catalogue internal/beside finds, parsed the way registration
// parses one.
// theCatalogue is every manifest beside this checkout, parsed the way registration parses one.
func theCatalogue(t *testing.T) []Manifest {
t.Helper()
root := beside.Catalogue(t)
root := filepath.Join("..", "..", "..", "mesh-catalog", "modules")
entries, err := os.ReadDir(root)
if err != nil {
t.Fatal(err)
t.Skipf("catalogue sibling not present: %v", err)
}
var out []Manifest
for _, e := range entries {
@@ -118,7 +115,7 @@ func theCatalogue(t *testing.T) []Manifest {
out = append(out, m)
}
if len(out) == 0 {
t.Fatalf("no manifests under %s, so this proved nothing", root)
t.Skip("no manifests found beside this checkout")
}
return out
}
internal/catalogue/public_issuer_test.go
+7 -10
View File
@@ -2,10 +2,7 @@ package catalogue
import (
"os"
"path/filepath"
"testing"
"github.com/novox/mesh-controller/internal/beside"
)
// The public issuer is the proxy's own fact (novox/hq ADR 0226), and the folding of it must not
@@ -89,22 +86,22 @@ func TestRouteProxyKeepsItsPublicAccountDirectory(t *testing.T) {
}
}
// The modules ADR 0226 retired stay retired, and nothing asks for what they provided. A module is
// in the catalogue when its manifest is: a directory left behind with no manifest in it (a build's
// leftovers, untracked by git) is not a module, and failed this on a desktop (novox/hq issue 432).
// The modules ADR 0226 retired stay retired, and nothing asks for what they provided.
func TestTheRetiredNetworkingModulesAreNotInTheCatalogue(t *testing.T) {
modules := beside.Catalogue(t)
if _, err := os.Stat("../../../mesh-catalog/modules"); err != nil {
t.Skipf("the catalogue is not beside this checkout: %v", err)
}
for _, gone := range []string{"public-acme", "dhcpcd", "cloudflare-dns"} {
if _, err := os.Stat(filepath.Join(modules, gone, "module.json")); err == nil {
if _, err := os.Stat("../../../mesh-catalog/modules/" + gone); err == nil {
t.Errorf("%s is in the catalogue again; ADR 0226 retired it", gone)
}
}
entries, err := os.ReadDir(modules)
entries, err := os.ReadDir("../../../mesh-catalog/modules")
if err != nil {
t.Fatal(err)
}
for _, e := range entries {
raw, err := os.ReadFile(filepath.Join(modules, e.Name(), "module.json"))
raw, err := os.ReadFile("../../../mesh-catalog/modules/" + e.Name() + "/module.json")
if err != nil {
continue
}

Some files were not shown because too many files have changed in this diff Show More