Compare commits

..
Author SHA1 Message Date
jochen 5955c4a767 Merge remote-tracking branch 'origin/main' into feat/mail-list 2026-10-23 11:53:29 +02:00
jochen 96b201920f Merge remote-tracking branch 'origin/main' into feat/mail-list 2026-10-23 01:07:15 +02:00
jochen d14ef647d6 issue 1859: mark the page after an act by a one-shot cookie, not its address
Review round 2 F3 (ADR 0517 §3, 1858/AC5): acted=1 stayed in the address, so a reload, a restored tab or a
bookmark of it recorded no page opened or opened. An act's 303 (and a mode chosen) now sets mail_acted for
10 seconds, HttpOnly and SameSite=Strict, Secure but on 127.0.0.1; the next list or pane it reaches records
nothing and clears it; the address carries no marker. The local run keeps the browser's other cookies.
2026-10-22 20:29:17 +02:00
jochen af89a69670 issue 1859: the page shown after an act records nothing, so each act leaves one line
Review round 1 F1 (ADR 0517 §3, §5): a form act went back to the list or the pane, which recorded page
opened or opened again, and the script's refresh did too, so last looked became the last act and opened
counts grew. The redirect after an act carries acted=1 and the refresh an X-Mail-Refresh header; that page
records nothing. The 1860 test now follows each redirect and counts every line, and reminds the task (F2,
1858/AC9), which the store's own integration test also reads back as a date.
2026-10-22 20:26:54 +02:00
jochen 4e68ec63c9 issue 1860: prove the operator's list end to end (integration, compiled only)
Both bundles start whole: mesh-mail-reader against a fresh database on the real store and an added event
on a real bus, mesh-mail-web's mail-reading asks carried to the reader's own tools/call as mesh-mail-web,
with stand-ins for mail-store and a fixed judgement. A task due today shows under today, ticked through
the page it is under handled today with one handled line holding no mail text, undo returns it, and a
correction from the reading pane keeps a lesson and one corrected line, within two minutes. pgx joins
mesh-mail-web's go.mod for the test's fresh database; the lead's try runs it.
2026-10-22 20:18:34 +02:00
jochen d8144b704d issue 1859: run the mail page locally with invented mail (1859.5)
So the operator can click through the page on their own machine before it is delivered: cmd/mail-page-dev
serves internal/web's real handlers on 127.0.0.1 alone, over invented mail in two accounts and an in-memory
mail reading that fills every section, with an always-signed-in stand-in; nothing reaches the bus, a
database, Keycloak or a mail server. -now fixes today. The README says how to run it and what is a stand-in.
2026-10-22 20:14:24 +02:00
jochen f554928ce1 issue 1859: modes by account, and the reading pane with its corrections; name a nameless sender (1859.4, 1850)
ADR 0515 §4, §5: the trusted setting modes names the accounts Work and Personal show, Everything shows every
granted account; /read shows one mail as escaped text with its tasks, promises and attachments as downloads,
records it opened, and offers its corrections (mark and group, not a task, not a promise, read later), each
calling correct. Issue 1850: a sender with no display name is named by its address, on the list, the pane and
All mail.
2026-10-22 20:09:29 +02:00
jochen ab152687e5 issue 1859: tick, undo and remind an item, and choose a mode, from the page (1859.3)
ADR 0517 §1, §2, §4, §5: /item and /mode are posts behind the session's token, each calling its one
mail-reading verb (handle, unhandle, remind, record) for the operator; the script gets done or failed with
the cause as JSON, a plain form goes back to the list or shows it again saying what failed and why.
2026-10-22 20:05:31 +02:00
jochen 7e83b99615 issue 1859: serve one script and one stylesheet from the page's own origin (1859.2)
ADR 0515 §6: /static/app.js and /static/app.css are built in and served with their types and nosniff; the
policy allows script and style from the origin alone and connect-src 'self'; no page carries an inline script,
a style or a handler. The script posts the page's forms without a reload and says a failure on the page.
2026-10-22 20:04:26 +02:00
jochen ad4f312263 issue 1859: open the mail page on the operator's list (1859.1)
ADR 0515 §1, §5: / greets the operator with what the mail signal told today and when they last looked, then
shows today, this week, later, the promises by kind, good to know, read later, handled today and the scam
warnings, from mail-reading.list; it records the page opened. The classical list moves to /all, as All mail.
2026-10-22 20:03:21 +02:00
jochen 4f3ebe370b issue 1859: answer the mail page's greeting and reading pane from mail reading
ADR 0515 §1, §5: list also answers the mails the mail signal told of today, and each line its mail's folder
and UID; entries takes an id and answers that one entry whole, with its sender's address, tasks, promises
and read-later, for the reading pane's corrections.
2026-10-22 20:00:44 +02:00
jochen f24a4d858a issue 1859: move the mail page into internal/web, so the local run can serve it
A program cannot import a main package; the page's code moves to modules/mesh-mail-web/internal/web, and
cmd/mesh-mail-web is a two-line program calling it, so cmd/mail-page-dev (1859.5) serves the same handlers.
Nothing the module does changes.
2026-10-22 19:58:43 +02:00
jochen 4071349078 issue 1858: prove each act and its line in the real store (integration, compiled only)
The lead's try runs it: an act and its line kept together, a failed act keeping neither, correct's three
writes together, the removal of old lines, and handling gone with its entry.
2026-10-22 19:56:31 +02:00
jochen 0e9004664c issue 1858: write correct's line in its own transaction, and guard who writes the log (1858.6)
ADR 0517 §5: correct keeps the entry's change, its reading lesson and one corrected line together, or none
of them; a test reads the source to prove only the acts write the action log, and each new verb refuses an
ungranted caller. The README and the callers setting name the twelve verbs.
2026-10-22 19:56:16 +02:00
jochen 0ed94e5466 issue 1858: remove action lines older than 365 days each night (1858.5)
ADR 0517 §9: the action log keeps a line one year; the removal runs at start and each night at 03:00
(Europe/Brussels) and says how many it removed. Handling goes with its entry by the schema's cascade.
2026-10-22 19:54:57 +02:00
jochen da8a6a6df4 issue 1858: record the acts that change no entry, and answer the action log (1858.4)
ADR 0517 §3 to §5: record writes one line for a mail opened, a mode chosen or the page opened (the last
looked list answers); every other line is its own verb's. actions answers the log newest first.
2026-10-22 19:54:28 +02:00
jochen fac091756e issue 1858: answer the operator's list for a mode (1858.3)
ADR 0517 §4, ADR 0515 §1: list gives tasks under today, this week and later by due date in the operator's
zone, promises by kind, good to know, read later, handled today with its undo and the scam warnings, for the
mode's accounts, and when the operator last looked.
2026-10-22 19:53:47 +02:00
jochen a5ffe4eea7 issue 1858: handle, unhandle and remind an item, each with its own line (1858.2)
ADR 0517 §1, §2, §4, §5: an item is a task, a promise or the mail itself; each act and its action line are
kept in one transaction, a refused or failed act writes none, an undo lasts until the day ends and a
reminder shows the item again the next day (Europe/Brussels). The seat's serves list in mesh-mail's
manifest names the new verbs.
2026-10-22 19:51:54 +02:00
jochen fdd56d2259 issue 1858: keep the operator's handling and the action log (1858.1)
ADR 0517 §1 to §3, §9 to §11: migration 0005 and schema.sql add item_handling (per item, gone with its
entry) and action_line (what, mail id, item, mode, when; never mail text); the store writes each act and its
line in one transaction.
2026-10-22 19:50:15 +02:00
jochen 75ce0e8a8c issue 1857: read the last 14 days of inbox mail once more (1857.5)
ADR 0516 §5: so the operator's list does not start empty, the inbox mail of the last 14 days of each granted
account is read again in ordinary batches, after any mail just heard; it is kept as done in its own table
(migration 0004), and the journal says its start, end and count. The reader now invokes mail-store.search.
2026-10-22 19:48:21 +02:00
jochen af54720907 issue 1857: let correct say not a task, remove a promise, or set read later (1857.4)
ADR 0516 §4: each is kept as a reading lesson, as a change of mark or mail group is, so later batches with a
mail of that sender follow it; an item number the entry does not hold is refused.
2026-10-22 19:42:04 +02:00
jochen 6b759f708e issue 1857: judge at most 6 mails a batch (1857.3)
ADR 0516 §3: the answer grows by each mail's tasks and promises, so a batch holds 6 mails, not 10, to stay
within its 40 s; a test proves the largest question stays under 64 KiB with every mail whole.
2026-10-22 19:41:17 +02:00
jochen c3b504e26e issue 1857: ask and check each mail's tasks, promises and read-later (1857.2)
ADR 0516 §1, §2: the answer schema asks at most 3 tasks and 3 promises in the model's own words and the
read-later flag; ParseAnswer refuses a 4th, an unknown kind, a line over 120 characters or a bad date, and a
line sharing a run of 8 words with its mail's text fails the batch, so nothing copied is ever kept.
2026-10-22 19:40:33 +02:00
jochen daff20d351 issue 1857: keep a mail entry's tasks, promises and read-later (1857.1)
ADR 0516 §1: an entry holds at most 3 tasks and 3 promises in the model's own words, and whether it is read
later. Migration 0004 and schema.sql add mail_task, mail_promise and read_later; the store writes and reads
them with their entry, in one transaction.
2026-10-22 19:39:15 +02:00
jochen a3626e7920 issue 1580: say no batch has run yet until one has, then when it succeeded
The health answered that the last batch judgement succeeded on a fresh database, before any batch ran.
reader_health keeps when a batch last succeeded (migration 0003, backfilled from the last judged entry),
and the health and status say it, or that no batch has run yet.
2026-10-22 19:37:20 +02:00
jochen 92dc8e313c issue 1849: keep one mail entry per message, by its Message-ID, else its folder and UID
Gmail files one message in INBOX and All Mail, each with its own UID; the reader kept an entry per folder.
waiting_mail and mail_entry are now unique per account and message key (migration 0002 and schema.sql), a
mail that already has its entry no longer waits again, and the test double keeps PgStore's own keys
(issue 1548). A schema model test proves schema.sql equals the migrations in order.
2026-10-22 19:36:19 +02:00
989 changed files with 36974 additions and 59347 deletions
-13
View File
@@ -14,7 +14,6 @@ modules/claude-code/claude-code
modules/mesh-watcher/mesh-watcher
modules/dnsmasq/dnsmasq
modules/mesh-issues/cmd/mesh-issues/mesh-issues
modules/mesh-issues/mesh-issues
modules/mounts/cmd/mounts/mounts
modules/mesh-knowledge/cmd/mesh-knowledge/mesh-knowledge
modules/mesh-knowledge/mesh-knowledge
@@ -29,15 +28,3 @@ modules/brother/cmd/brother/brother
modules/mesh-build-go/cmd/mesh-build-go/mesh-build-go
modules/mesh-config/mesh-config
modules/mesh-build-image/cmd/mesh-build-image/mesh-build-image
*.test
# binaries a local go build leaves in a module root
/checks/paths/paths
/modules/mesh-build-go/mesh-build-go
/modules/mesh-delivery-gate/mesh-delivery-gate
modules/mesh-config/cmd/mesh-config/mesh-config
# Built by their modules' build step (issue 1577), never committed.
modules/forticlient/cmd/forticlient-tools/forticlient-tools
modules/networkmanager/uplink-tools
checks/words/words
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest.
modules/*/module.json
-4
View File
@@ -1,7 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/config
go 1.22
require git.novox.be/novox/mesh-catalog/checks/trunk v0.0.0
replace git.novox.be/novox/mesh-catalog/checks/trunk => ../trunk
+1 -1
View File
@@ -110,7 +110,7 @@ func Check(root string) (map[string][]string, error) {
// The modules taking a whole layer were found after the list was first written and before the trunk held a copy
// (issue 735), so they stand in it too.
var seed = map[string]bool{
"fail2ban": true, "influxdb": true, "keycloak": true, "mailu": true, "searxng": true,
"betterbird": true, "fail2ban": true, "influxdb": true, "keycloak": true, "mailu": true, "searxng": true,
"baserow": true, "confluence": true, "gitea": true, "gitlab": true, "grafana": true, "home-assistant": true,
"icecast": true, "jira": true, "letta": true, "matrix": true, "nodered": true, "supabase": true, "unifi": true,
}
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest and the files it names.
modules
-33
View File
@@ -1,33 +0,0 @@
package main
import (
"path/filepath"
"testing"
"git.novox.be/novox/mesh-catalog/checks/trunk"
)
// The script's comparison of this list with the trunk's (novox/hq ADR 0529 clause 2): the list of modules not yet moved gains none
// the trunk's copy at the merge base does not hold. The catalogue itself is judged against the list by
// TestTheCatalogueSaysConfigInEveryMovedModule. Skipped, and said, where no trunk is beside.
func TestTheUnmovedListHoldsNoModuleTheTrunksDoesNot(t *testing.T) {
path, ok := trunk.Copy(t, "checks/config/unmoved")
found, err := Check(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
unmoved, err := Unmoved("unmoved")
if err != nil {
t.Fatal(err)
}
base := seed
if ok {
if base, err = Unmoved(path); err != nil {
t.Fatal(err)
}
}
_, _, added := Judge(found, unmoved, base)
for _, m := range added {
t.Errorf("checks/config/unmoved gains %s, which the trunk's copy does not hold: the list only shrinks", m)
}
}
-5
View File
@@ -1,5 +0,0 @@
module checks/credentialform
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0
-2
View File
@@ -1,2 +0,0 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0 h1:lKU3HjNXGiVbLW2hUgFjU/TMTUYMlNnnmZxG5T7biQA=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
-96
View File
@@ -1,96 +0,0 @@
// Package credentialform refuses an offer that does not say the form of its credential: an own secret, an issuer, or
// `form` pair or none (the lead, 2026-10-29). It reads every modules/*/module.json through mesh-sdk's
// CredentialFormProblems.
//
// It refuses a route received as a file too (issue 774, research 072 row 34, the operator on 2026-10-31: "there is only
// 1 path, the other is simply deprecated/invalid"): a route is a provision, served from the seat mesh-provisioner's
// plans alone, and a manifest's `receives.route` is the deprecated path. A module still moving off it is listed in
// stillReceivingRoutes with why; every refusal names them all, and a listed module that no longer receives one is
// refused too, so the list only shrinks.
package credentialform
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Problems is every offer of the manifests under root/modules that says no credential form, one line each.
func Problems(root string) ([]string, error) {
files, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
sort.Strings(files)
var out []string
var receiving []string
reads := map[string]bool{}
var binders []string
for _, f := range files {
raw, err := os.ReadFile(f)
if err != nil {
return nil, err
}
var m catalogue.Manifest
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("%s: %w", f, err)
}
out = append(out, catalogue.CredentialFormProblems(m)...)
if _, received := m.Receives["route"]; received {
receiving = append(receiving, m.Module)
} else if _, listed := stillReceivingRoutes[m.Module]; listed {
out = append(out, fmt.Sprintf("%s no longer receives route; take it off stillReceivingRoutes", m.Module))
}
var kept struct {
Reads []string `json:"reads"`
State []json.RawMessage `json:"state"`
}
_ = json.Unmarshal(raw, &kept) // reads may hold objects in another module; only the provisioner's are read
if m.Module == "mesh-provisioner" {
for _, r := range kept.Reads {
reads[r] = true
}
}
if len(m.Provides) > 0 && keepsBound(kept.State) {
binders = append(binders, m.Module)
}
}
for _, module := range binders {
if !reads[module+".bound"] {
out = append(out, fmt.Sprintf("%s provides and keeps a `bound` state, and mesh-provisioner's manifest does not read %s.bound", module, module))
}
}
for _, module := range receiving {
if _, listed := stillReceivingRoutes[module]; !listed {
out = append(out, fmt.Sprintf("%s receives route as a file: a route is a provision, served from the seat "+
"mesh-provisioner's plans alone (issue 774); still on the deprecated path: %s", module,
strings.Join(receiving, ", ")))
}
}
return out, nil
}
// stillReceivingRoutes is every module still receiving its routes as a file, and why it has not moved yet.
var stillReceivingRoutes = map[string]string{
"route-adapter": "migration scaffolding writing into a predecessor's proxy (novox/hq ADR 0104), deleted when that " +
"migration ends; it reads the received file its TypeScript adapter turns into the predecessor's route files",
}
// keepsBound says whether a manifest's `state` names `bound`, as a plain name or an object with a name.
func keepsBound(state []json.RawMessage) bool {
for _, s := range state {
var name string
var obj struct {
Name string `json:"name"`
}
if json.Unmarshal(s, &name) == nil && name == "bound" || json.Unmarshal(s, &obj) == nil && obj.Name == "bound" {
return true
}
}
return false
}
-54
View File
@@ -1,54 +0,0 @@
package credentialform
import (
"os"
"path/filepath"
"strings"
"testing"
)
func aCatalogue(t *testing.T, manifests map[string]string) string {
t.Helper()
root := t.TempDir()
for module, body := range manifests {
dir := filepath.Join(root, "modules", module)
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "module.json"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
return root
}
// An offer that says no form is refused and named; one that says pair, none or an own secret passes; a route received
// as a file is refused but for the module still moving off it; and the real catalogue is clean.
func TestEveryOfferSaysTheFormOfItsCredential(t *testing.T) {
root := aCatalogue(t, map[string]string{
"silent": `{"module":"silent","provides":[{"name":"thing","scope":"mesh"}]}`,
"binder": `{"module":"binder","provides":[{"name":"d","credential":{"form":"pair"}}],"state":["bound"]}`,
"mesh-provisioner": `{"module":"mesh-provisioner","reads":["other.bound"]}`,
"says": `{"module":"says","provides":[{"name":"a","credential":{"form":"pair"}},{"name":"b","credential":{"form":"none"}},{"name":"c","credential":{"own":"jwt"}}]}`,
// A route received as a file (issue 774, research 072 row 34): refused, naming every module still on it; the one
// listed as still moving passes.
"old-proxy": `{"module":"old-proxy","receives":{"route":"/var/lib/old-proxy/mesh.json"}}`,
"route-adapter": `{"module":"route-adapter","receives":{"route":"/var/lib/route-adapter/mesh.json"}}`,
})
got, err := Problems(root)
if err != nil {
t.Fatal(err)
}
if len(got) != 3 || !strings.Contains(got[0], `silent provides "thing"`) ||
!strings.Contains(got[1], "binder") || !strings.Contains(got[1], "mesh-provisioner") ||
!strings.HasPrefix(got[2], "old-proxy receives route") || !strings.Contains(got[2], "old-proxy, route-adapter") {
t.Fatalf("refused %v, wanted the silent offer, the binder the provisioner does not read and the received route", got)
}
real, err := Problems(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
if len(real) != 0 {
t.Fatalf("the catalogue has offers with no credential form:\n%s", strings.Join(real, "\n"))
}
}
-3
View File
@@ -1,3 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (issue 1936).
# every manifest, for what each offer says of its credential's form.
modules/*/module.json
-128
View File
@@ -1,128 +0,0 @@
// Package failurekit refuses a build module whose tests do not run mesh-sdk's conformance kit (issue 1901, its thorough
// fix): a module that claims a node-build-* seat and serves a test step must hold a test that calls conformance.Run, so
// its parse of the test runner's output is held to the failure contract (a failing test named with its cause, never a
// passing line in its place). A build module with no test step (mesh-build-image builds images and runs no tests) is
// exempt from that, and said so. Every build module, test step or not, must also hold a test that calls
// conformance.LeftRunning with its own start sweep, so a step a previous process left running is ended at its start
// (issue 1890).
package failurekit
import (
"encoding/json"
"fmt"
"io/fs"
"os"
"path/filepath"
"sort"
"strings"
)
// Judged is what the check says of one build module: Refused says why it fails the check, "" when it passes; Exempt
// says why the kit does not apply.
type Judged struct {
Module, Refused, Exempt string
}
type manifest struct {
Claims []struct {
Name string `json:"name"`
} `json:"claims"`
Tools []json.RawMessage `json:"tools"`
}
// Judge reads every modules/<m>/module.json of the catalogue at root and judges each module that claims a node-build-*
// seat.
func Judge(root string) ([]Judged, error) {
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
sort.Strings(manifests)
var out []Judged
for _, p := range manifests {
raw, err := os.ReadFile(p)
if err != nil {
return nil, err
}
var m manifest
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("%s: %w", p, err)
}
seat := ""
for _, c := range m.Claims {
if strings.HasPrefix(c.Name, "node-build-") {
seat = c.Name
}
}
if seat == "" {
continue
}
dir := filepath.Dir(p)
j := Judged{Module: filepath.Base(dir)}
var refused []string
test := testTool(m.Tools)
switch {
case test == "":
j.Exempt = fmt.Sprintf("it claims %s and serves no test step, so the failure contract's cases do not apply", seat)
default:
runs, err := runsTheKit(dir, "conformance.Run(")
if err != nil {
return nil, err
}
if !runs {
refused = append(refused, fmt.Sprintf("it claims %s and serves the test step %s, and no test of it calls conformance.Run "+
"(mesh-sdk go/buildstep/conformance): run the kit with the module's own parse path (issue 1901)", seat, test))
}
}
// Every build module keeps steps, so every one runs the left-running case, test step or not (issue 1890).
sweeps, err := runsTheKit(dir, "conformance.LeftRunning(")
if err != nil {
return nil, err
}
if !sweeps {
refused = append(refused, fmt.Sprintf("it claims %s, and no test of it calls conformance.LeftRunning "+
"(mesh-sdk go/buildstep/conformance): run the kit's left-running case with the module's own start sweep (issue 1890)", seat))
}
j.Refused = strings.Join(refused, "; ")
out = append(out, j)
}
return out, nil
}
// testTool is the name of the module's test step tool (<prefix>_test), or "".
func testTool(tools []json.RawMessage) string {
for _, raw := range tools {
var name string
if json.Unmarshal(raw, &name) != nil {
var t struct {
Name string `json:"name"`
}
if json.Unmarshal(raw, &t) != nil {
continue
}
name = t.Name
}
if strings.HasSuffix(name, "_test") {
return name
}
}
return ""
}
// runsTheKit says a _test.go file under dir calls the kit's function call (such as "conformance.Run(").
func runsTheKit(dir, call string) (bool, error) {
found := false
err := filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
if err != nil || found || d.IsDir() || !strings.HasSuffix(p, "_test.go") {
return err
}
raw, err := os.ReadFile(p)
if err != nil {
return err
}
found = strings.Contains(string(raw), call) &&
strings.Contains(string(raw), `"git.novox.be/novox/mesh-sdk/go/buildstep/conformance"`)
return nil
})
return found, err
}
-77
View File
@@ -1,77 +0,0 @@
package failurekit
import (
"os"
"path/filepath"
"testing"
)
// 1901/AC7 and 1890/AC8: a build module with a test step and no kit is refused, one that runs the kit passes, one with no
// test step is exempt from the failure cases but not from the left-running case, a module that runs only the failure
// cases is refused, and a module that claims no node-build-* seat is not judged.
func TestABuildModuleWithATestStepMustRunTheKit(t *testing.T) {
root := t.TempDir()
const imp = "package main\n\nimport \"git.novox.be/novox/mesh-sdk/go/buildstep/conformance\"\n\n"
const sweep = "func TestSweep(t *testing.T) { conformance.LeftRunning(t, nil) }\n"
kit := imp + "func TestKit(t *testing.T) { conformance.Run(t, conformance.GoTest, nil) }\n" + sweep
for path, body := range map[string]string{
"modules/a/module.json": `{"claims":[{"name":"node-build-a"}],"tools":["build_a_fetch","build_a_test"]}`,
"modules/a/cmd/a/check_test.go": "package main\n",
"modules/b/module.json": `{"claims":[{"name":"node-build-b"}],"tools":[{"name":"build_b_test"}]}`,
"modules/b/cmd/b/kit_test.go": kit,
"modules/c/module.json": `{"claims":[{"name":"node-build-c"}],"tools":["build_c_build"]}`,
"modules/c/cmd/c/kit_test.go": imp + sweep,
"modules/f/module.json": `{"claims":[{"name":"node-build-f"}],"tools":["build_f_build"]}`,
"modules/g/module.json": `{"claims":[{"name":"node-build-g"}],"tools":["build_g_test"]}`,
"modules/g/cmd/g/kit_test.go": imp + "func TestKit(t *testing.T) { conformance.Run(t, conformance.GoTest, nil) }\n",
"modules/d/module.json": `{"claims":[{"name":"other"}],"tools":["d_test"]}`,
"modules/e/module.json": `{"claims":[{"name":"node-build-e"}],"tools":["build_e_test"]}`,
"modules/e/cmd/e/kit.go": kit, // not a test file
} {
p := filepath.Join(root, path)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
got, err := Judge(root)
if err != nil {
t.Fatal(err)
}
want := map[string]string{"a": "refused", "b": "passes", "c": "exempt", "e": "refused", "f": "refused", "g": "refused"}
if len(got) != len(want) {
t.Fatalf("judged %+v, want a, b, c, e, f and g", got)
}
for _, j := range got {
is := "passes"
if j.Refused != "" {
is = "refused"
} else if j.Exempt != "" {
is = "exempt"
}
if is != want[j.Module] {
t.Errorf("%s %s (%+v), want it %s", j.Module, is, j, want[j.Module])
}
}
}
// Every build module of this catalogue with a test step runs the kit.
func TestTheCataloguesBuildModulesRunTheKit(t *testing.T) {
got, err := Judge(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
if len(got) == 0 {
t.Fatal("no module claims a node-build-* seat: the check reads the wrong tree")
}
for _, j := range got {
switch {
case j.Refused != "":
t.Errorf("%s: %s", j.Module, j.Refused)
case j.Exempt != "":
t.Logf("%s is exempt: %s", j.Module, j.Exempt)
}
}
}
-3
View File
@@ -1,3 +0,0 @@
module git.novox.be/novox/mesh-catalog/checks/failurekit
go 1.22
-7
View File
@@ -1,7 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Judge("../.."): every manifest, and the directory of each module claiming a node-build-* seat, walked for a test that
# calls the kit. The build modules are the mesh-build-* directories; a build module named otherwise needs its line here.
modules/*/module.json
modules/mesh-build-*
-8
View File
@@ -1,8 +0,0 @@
# The calls of the controller's hand-act log a manifest of this catalogue may still invoke, each `<module> <verb> <why>`:
# the fallback of a recorder (mesh-sdk handacts.Recorder) while the seat mesh-hand-acts has no holder that answers, until
# the controller's lease is written to a holder that is no controller. The check of this directory (handacts) refuses any
# other, and an entry whose module no longer invokes it: the list only shrinks, and it is empty before the lease write
# (novox/hq ADR 0565 §11; issue 1953, issue 774).
mesh-config hand-act fallback until the lease write: its sets, clears and held values record through mesh-hand-acts first
mesh-config warranted fallback until the lease write: its warrants record through mesh-hand-acts first
mesh-healers drill fallback until the lease write: a drill records through mesh-hand-acts first
-3
View File
@@ -1,3 +0,0 @@
module git.novox.be/novox/mesh-catalog/checks/handacts
go 1.22
-109
View File
@@ -1,109 +0,0 @@
// Command handacts refuses a manifest that records into the controller's hand-act log (novox/hq ADR 0565 §11): every
// recorder records through the seat mesh-hand-acts from its delivery, so no manifest invokes
// `seat:mesh-controller.hand-act`, `.warranted` or `.drill`. The recorders' fallbacks to the controller's log while the
// seat has no holder are named in checks/handacts/fallbacks, each module and verb with why; that list only shrinks, and
// an entry whose module no longer invokes the verb is refused too.
//
// go run . <catalog root>
//
// Exits 1 on a finding.
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
)
// Verbs are the controller's hand-act verbs, as a manifest invokes them, and Allowed the list of named fallbacks.
var Verbs = []string{"hand-act", "warranted", "drill"}
const (
invokePrefix = "seat:mesh-controller."
Allowed = "checks/handacts/fallbacks"
)
// Findings is every refusal in the catalog at root, sorted.
func Findings(root string) ([]string, error) {
allowed := map[string]bool{}
raw, err := os.ReadFile(filepath.Join(root, Allowed))
if err != nil && !os.IsNotExist(err) {
return nil, err
}
for _, line := range strings.Split(string(raw), "\n") {
if f := strings.Fields(line); len(f) >= 3 && !strings.HasPrefix(f[0], "#") {
allowed[f[0]+" "+f[1]] = true
} else if len(f) > 0 && !strings.HasPrefix(f[0], "#") {
return nil, fmt.Errorf("%s: %q is not `<module> <verb> <why>`", Allowed, line)
}
}
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
var out []string
invoked := map[string]bool{}
for _, path := range manifests {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var m struct {
Module string `json:"module"`
Invokes []string `json:"invokes"`
}
if json.Unmarshal(raw, &m) != nil || m.Module == "mesh-controller" {
continue
}
for _, inv := range m.Invokes {
verb, ok := strings.CutPrefix(strings.TrimSpace(inv), invokePrefix)
if !ok || !contains(Verbs, verb) {
continue
}
invoked[m.Module+" "+verb] = true
if !allowed[m.Module+" "+verb] {
out = append(out, fmt.Sprintf("%s invokes %s%s: record through seat:mesh-hand-acts.%s (novox/hq ADR 0565 §11), "+
"or name the fallback in %s", m.Module, invokePrefix, verb, verb, Allowed))
}
}
}
for entry := range allowed {
if !invoked[entry] {
out = append(out, fmt.Sprintf("%s names %q, which is no longer invoked: take it off the list, which only shrinks",
Allowed, entry))
}
}
sort.Strings(out)
return out, nil
}
func contains(list []string, s string) bool {
for _, x := range list {
if x == s {
return true
}
}
return false
}
func main() {
if len(os.Args) != 2 {
fmt.Fprintln(os.Stderr, "usage: handacts <catalog root>")
os.Exit(2)
}
found, err := Findings(os.Args[1])
if err != nil {
fmt.Println(err)
os.Exit(1)
}
for _, f := range found {
fmt.Println(f)
}
if len(found) > 0 {
os.Exit(1)
}
fmt.Println("no manifest records into the controller's hand-act log but the fallbacks named")
}
-59
View File
@@ -1,59 +0,0 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
)
// No manifest invokes the controller's hand-act verbs but a named fallback; a named fallback no longer invoked is
// refused (novox/hq ADR 0565 §11's catalog test).
func TestFindings(t *testing.T) {
for _, row := range []struct {
name string
files map[string]string
want string // empty: clean
}{
{"clean", map[string]string{"modules/a/module.json": `{"module":"a","invokes":["seat:mesh-hand-acts.hand-act"]}`}, ""},
{"an unnamed call", map[string]string{"modules/a/module.json": `{"module":"a","invokes":["seat:mesh-controller.warranted"]}`},
"a invokes seat:mesh-controller.warranted"},
{"a named fallback, and the controller's own", map[string]string{
"modules/a/module.json": `{"module":"a","invokes":["seat:mesh-controller.drill"]}`,
"modules/mesh-controller/module.json": `{"module":"mesh-controller","invokes":["seat:mesh-controller.hand-act"]}`,
Allowed: "# why\na drill fallback until the lease write\n"}, ""},
{"a fallback no longer invoked", map[string]string{Allowed: "gone hand-act fallback\n"}, `names "gone hand-act"`},
{"another verb of the controller", map[string]string{"modules/a/module.json": `{"module":"a","invokes":["seat:mesh-controller.conditions"]}`}, ""},
} {
t.Run(row.name, func(t *testing.T) {
root := t.TempDir()
for path, text := range row.files {
p := filepath.Join(root, path)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte(text), 0o644); err != nil {
t.Fatal(err)
}
}
got, err := Findings(root)
if err != nil {
t.Fatal(err)
}
if all := strings.Join(got, "\n"); (row.want == "") != (all == "") || !strings.Contains(all, row.want) {
t.Fatalf("findings = %q, want one containing %q", all, row.want)
}
})
}
}
// The run of this check over the real tree.
func TestTheCatalogRecordsOnlyThroughTheSeat(t *testing.T) {
got, err := Findings(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
for _, f := range got {
t.Error(f)
}
}
-3
View File
@@ -1,3 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (issue 1936).
# every manifest, for what it invokes.
modules/*/module.json
-14
View File
@@ -1,14 +0,0 @@
module checks/health
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031081231-d352fe51b455
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.48.0 // indirect
)
-16
View File
@@ -1,16 +0,0 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0 h1:lKU3HjNXGiVbLW2hUgFjU/TMTUYMlNnnmZxG5T7biQA=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031081231-d352fe51b455 h1:1rAqVPyu07xkacqJrDaqUtoqLQJmklfyXC/5erRJnEk=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031081231-d352fe51b455/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
github.com/nats-io/nkeys v0.4.16/go.mod h1:llLgWoI0o4z/Q57q2R1kHfmocyhGV6VG/U18Glg1Afs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
-70
View File
@@ -1,70 +0,0 @@
// Package health is the catalogue's count of long-running resources that do not say how they are ready (novox/hq ADR 0240 rule
// 8), kept in health-undeclared and judged by a test over the real tree (ADR 0529 clauses 2 and 4): it replaces the step of
// merge-check.sh that read it out of the controller's `module check`. Every manifest is read through mesh-sdk's catalogue
// package, the one the controller counts with.
package health
import (
"fmt"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Count is the long-running resources without `health` of every module under root/modules.
func Count(root string) (int, error) {
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return 0, err
}
sort.Strings(manifests)
if len(manifests) == 0 {
return 0, fmt.Errorf("no manifest under %s: the count would be zero for any catalogue", filepath.Join(root, "modules"))
}
n := 0
for _, path := range manifests {
m, err := catalogue.ReadTreeManifest(root, filepath.Dir(path))
if err != nil {
return 0, fmt.Errorf("%s: %w", path, err)
}
n += len(catalogue.Undeclared(m))
}
return n, nil
}
// Kept is the number the file keeps: the first number of its first line.
func Kept(path string) (int, error) {
raw, err := os.ReadFile(path)
if err != nil {
return 0, err
}
first, _, _ := strings.Cut(string(raw), "\n")
digits := ""
for _, r := range strings.TrimSpace(first) {
if r < '0' || r > '9' {
break
}
digits += string(r)
}
n, err := strconv.Atoi(digits)
if err != nil {
return 0, fmt.Errorf("%s does not begin with the number it keeps: %q", path, first)
}
return n, nil
}
// Judge says what is wrong with a count against the kept number; empty when they agree. It only goes down: a rise is refused,
// and a fall asks the new number written in health-undeclared, so the count cannot rise again.
func Judge(kept, counted int) string {
switch {
case counted > kept:
return fmt.Sprintf("the long-running resources without health rose from %d to %d: declare how each new one is ready (ADR 0240 rule 8)", kept, counted)
case counted < kept:
return fmt.Sprintf("the long-running resources without health fell from %d to %d: write %d in health-undeclared, so the count cannot rise again", kept, counted, counted)
}
return ""
}
-88
View File
@@ -1,88 +0,0 @@
package health
import (
"os"
"path/filepath"
"strings"
"testing"
)
func aCatalogue(t *testing.T, manifests map[string]string) string {
t.Helper()
root := t.TempDir()
for module, body := range manifests {
dir := filepath.Join(root, "modules", module)
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "module.json"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
return root
}
// A service that stays up and does not say how it is ready is counted; one that says, and a file, are not.
func TestOnlyALongRunningResourceWithoutHealthIsCounted(t *testing.T) {
root := aCatalogue(t, map[string]string{
"quiet": `{"module":"quiet","resources":[{"id":"svc","type":"service","unit":"quiet.service","state":"running"}]}`,
"says": `{"module":"says","resources":[{"id":"svc","type":"service","unit":"says.service","state":"running",` +
`"health":{"kind":"unit"}}]}`,
"file": `{"module":"file","resources":[{"id":"cfg","type":"file","path":"/etc/file.conf","content":"x"}]}`,
})
counted, err := Count(root)
if err != nil {
t.Fatal(err)
}
if counted != 1 {
t.Fatalf("counted %d, wanted the one quiet service", counted)
}
}
func TestTheCountMayOnlyGoDown(t *testing.T) {
for _, c := range []struct {
kept, counted int
says string
}{
{58, 58, ""},
{58, 59, "rose from 58 to 59"},
{58, 57, "fell from 58 to 57"},
} {
got := Judge(c.kept, c.counted)
if (c.says == "") != (got == "") || !strings.Contains(got, c.says) {
t.Errorf("kept %d counted %d: %q, wanted %q", c.kept, c.counted, got, c.says)
}
}
}
func TestTheKeptCountIsTheFirstNumberOfTheFile(t *testing.T) {
path := filepath.Join(t.TempDir(), "health-undeclared")
if err := os.WriteFile(path, []byte("58\nThe long-running resources ...\n"), 0o644); err != nil {
t.Fatal(err)
}
if n, err := Kept(path); err != nil || n != 58 {
t.Fatalf("%d %v", n, err)
}
if err := os.WriteFile(path, []byte("many\n"), 0o644); err != nil {
t.Fatal(err)
}
if _, err := Kept(path); err == nil {
t.Fatal("a file with no number gave a count")
}
}
// The script's `module check` run over the real tree (novox/hq ADR 0240 rule 8, ADR 0529 clause 2 and 4): the long-running
// resources that do not say how they are ready, counted as the controller counts them, are the number health-undeclared keeps.
func TestTheCatalogueSaysHowItsLongRunningResourcesAreReady(t *testing.T) {
kept, err := Kept(filepath.Join("..", "..", "health-undeclared"))
if err != nil {
t.Fatal(err)
}
counted, err := Count(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
if say := Judge(kept, counted); say != "" {
t.Fatal(say)
}
}
-10
View File
@@ -1,10 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# every manifest through mesh-sdk's ReadTreeManifest: the root's predefined keys and each templated module's
# templates (files/ by default, or the directory its manifest names); and the root's kept count of resources that do not say how they are ready.
modules/*/module.json
predefined.json
modules/*/files
modules/*/templates
health-undeclared
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest.
modules/*/module.json
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest, and whether the path each image artifact builds from is there.
modules
-12
View File
@@ -61,15 +61,3 @@ func TestAModuleThatIsNotTypeScriptIsNotJudged(t *testing.T) {
t.Fatalf("%v %v", got, err)
}
}
// The script's run of this check over the real tree (novox/hq ADR 0529 clause 2): every TypeScript module that names packages
// beyond the SDK commits its lock.
func TestTheCatalogueCommitsTheLocksItNeeds(t *testing.T) {
found, err := Check("../..")
if err != nil {
t.Fatal(err)
}
for _, f := range found {
t.Errorf("%s names packages beyond the SDK (%v) and has no package-lock.json: commit one made by the build module's toolchain (novox/hq ADR 0469 §5)", f.Module, f.Packages)
}
}
-7
View File
@@ -1,7 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest, a TypeScript module's package.json and whether its lock is there.
modules/*/module.json
modules/*/package.json
modules/*/package-lock.json
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest.
modules/*/module.json
-14
View File
@@ -1,14 +0,0 @@
module checks/manifests
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031081231-d352fe51b455
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.48.0 // indirect
)
-16
View File
@@ -1,16 +0,0 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030181247-4c1384faefa0 h1:t04baH2Xz/NUsOEV3ctKY59XjiY5WmEuststvH+XeOs=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030181247-4c1384faefa0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031081231-d352fe51b455 h1:1rAqVPyu07xkacqJrDaqUtoqLQJmklfyXC/5erRJnEk=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261031081231-d352fe51b455/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
github.com/nats-io/nkeys v0.4.16/go.mod h1:llLgWoI0o4z/Q57q2R1kHfmocyhGV6VG/U18Glg1Afs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
-61
View File
@@ -1,61 +0,0 @@
// Package manifests judges every module's manifest as the node-engine reads it, and the catalogue as one shelf: each
// modules/*/module.json through mesh-sdk's ParseManifestWith against the predefined.json at the root, with its templates
// read from the tree beside it (WithTemplatesFrom), then the rules between manifests (CatalogueProblems,
// IdentityProblems, DataProblems).
//
// Why it is here (issue 774): on 2026-10-30 mesh-catalog ca8d6f33 gave power's bar item two attributes its bar places
// by none of. ParseManifest refuses that, and every node-engine refused to compose at 19:13; no check of the pull
// request had parsed the manifest. That judgement was mesh-controller's `module check`, which the merge gate ran over
// every manifest before the checker moved to mesh-delivery; the checker runs this repository's Go tests and the
// composition, and no test parsed a manifest whole. This one does, in the same functions the nodes use.
package manifests
import (
"fmt"
"os"
"path/filepath"
"sort"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Problems is what is wrong with the manifests under root/modules, one line each: a manifest the mesh refuses, named by
// its path, then the rules between the manifests that parse.
func Problems(root string) ([]string, error) {
files, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
sort.Strings(files)
list, err := catalogue.ReadPredefined(root)
if err != nil {
return nil, err
}
var out []string
shelf := catalogue.Shelf{}
for _, f := range files {
raw, err := os.ReadFile(f)
if err != nil {
return nil, err
}
rel, _ := filepath.Rel(root, f)
m, err := catalogue.ParseManifestWith(raw, list)
if err == nil {
m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(f))
}
if err != nil {
out = append(out, fmt.Sprintf("%s: %v", rel, err))
continue
}
if _, twice := shelf[m.Module]; twice {
out = append(out, fmt.Sprintf("%s: %s was already given; two manifests name one module", rel, m.Module))
continue
}
shelf[m.Module] = m
}
between := catalogue.CatalogueProblems(shelf)
between = append(between, catalogue.IdentityProblems(shelf, catalogue.DefaultLongestMachine)...)
between = append(between, catalogue.DataProblems(shelf)...)
sort.Strings(between)
return append(out, between...), nil
}
-51
View File
@@ -1,51 +0,0 @@
package manifests
import (
"os"
"path/filepath"
"strings"
"testing"
)
// Every manifest of the catalogue is one the nodes can use, and the catalogue holds together as one shelf. A row is a
// manifest written into a catalogue of its own: power's bar item as mesh-catalog ca8d6f33 gave it (issue 774) is
// refused naming the attribute, and the same item without it passes.
func TestEveryManifestIsOneTheNodesCanUse(t *testing.T) {
for _, row := range []struct {
name, attributes, refused string
}{
{name: "the bar places by bar and place", attributes: `"bar":"bottom","place":"status","order":"45"`},
{name: "an attribute the bar places by none of (ca8d6f33)",
attributes: `"bar":"bottom","place":"status","order":"45","shows":"state","state":"power.draw"`,
refused: `"shows" is none of them`},
} {
root := t.TempDir()
dir := filepath.Join(root, "modules", "bar-user")
if err := os.MkdirAll(filepath.Join(dir, "files"), 0o755); err != nil {
t.Fatal(err)
}
manifest := `{"module":"bar-user","version":"1","moved":true,` +
`"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},` +
`"items":[{"collection":"contribution:node-bar:block","attributes":{` + row.attributes + `},"template":"files/bar"}]}`
for path, body := range map[string]string{"module.json": manifest, "files/bar": "{\"shows\":\"command\"}\n"} {
if err := os.WriteFile(filepath.Join(dir, path), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
got, err := Problems(root)
if err != nil {
t.Fatal(err)
}
joined := strings.Join(got, "\n")
if (row.refused == "") != (len(got) == 0) || !strings.Contains(joined, row.refused) {
t.Errorf("%s: refused %q, want %q", row.name, joined, row.refused)
}
}
real, err := Problems(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
if len(real) != 0 {
t.Fatalf("the catalogue holds manifests the nodes cannot use:\n%s", strings.Join(real, "\n"))
}
}
-4
View File
@@ -1,4 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (issue 1936).
# every module's directory (its manifest and the templates beside it), and the list of predefined keys.
modules
predefined.json
-3
View File
@@ -1,3 +0,0 @@
module git.novox.be/novox/mesh-catalog/checks/nobinaries
go 1.22
-84
View File
@@ -1,84 +0,0 @@
// Package nobinaries refuses a compiled program anywhere in the repository (issue 1577; first a checks/ binary slipped past a modules/-only rule): it is build output, made by the module's build step,
// and `go build` in a module's folder writes one beside go.mod where `git add -A` takes it.
package nobinaries
import (
"bytes"
"fmt"
"io"
"os"
"os/exec"
"path/filepath"
"strings"
)
// Kind says what executable format the first bytes of a file are, or "" when they are none.
func Kind(head []byte) string {
switch {
case bytes.HasPrefix(head, []byte("\x7fELF")):
return "ELF"
case bytes.HasPrefix(head, []byte("MZ")):
return "PE"
case bytes.HasPrefix(head, []byte{0xcf, 0xfa, 0xed, 0xfe}), bytes.HasPrefix(head, []byte{0xce, 0xfa, 0xed, 0xfe}),
bytes.HasPrefix(head, []byte{0xfe, 0xed, 0xfa, 0xcf}), bytes.HasPrefix(head, []byte{0xfe, 0xed, 0xfa, 0xce}),
bytes.HasPrefix(head, []byte{0xca, 0xfe, 0xba, 0xbe}) && len(head) >= 8 && head[7] < 0x20 && head[6] == 0 && head[5] == 0 && head[4] == 0:
return "Mach-O"
}
return ""
}
// Files are the files of the catalogue's root that the change holds, anywhere: the tracked ones when the root is a git
// checkout (an ignored binary a local build left is not the change's), else every file there but .git's.
func Files(root string) ([]string, error) {
cmd := exec.Command("git", "-C", root, "ls-files", "-z")
if out, err := cmd.Output(); err == nil && len(out) > 0 {
var files []string
for _, f := range strings.Split(strings.TrimRight(string(out), "\x00"), "\x00") {
if _, err := os.Lstat(filepath.Join(root, f)); err == nil { // a file the change deletes is not there
files = append(files, f)
}
}
return files, nil
}
var files []string
err := filepath.WalkDir(root, func(p string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() && d.Name() == ".git" {
return filepath.SkipDir
}
if d.IsDir() || !d.Type().IsRegular() {
return nil
}
rel, _ := filepath.Rel(root, p)
files = append(files, rel)
return nil
})
return files, err
}
// Find names each file of files (paths from root) that is a compiled program, with its format.
func Find(root string, files []string) ([]string, error) {
var found []string
for _, f := range files {
p := filepath.Join(root, f)
if st, err := os.Lstat(p); err != nil || !st.Mode().IsRegular() {
continue
}
fh, err := os.Open(p)
if err != nil {
return nil, err
}
head := make([]byte, 8)
n, err := io.ReadFull(fh, head)
fh.Close()
if err != nil && err != io.ErrUnexpectedEOF && err != io.EOF {
return nil, err
}
if k := Kind(head[:n]); k != "" {
found = append(found, fmt.Sprintf("%s (%s)", f, k))
}
}
return found, nil
}
-89
View File
@@ -1,89 +0,0 @@
package nobinaries
import (
"os"
"os/exec"
"path/filepath"
"testing"
)
func TestFindNamesEachCompiledProgramAndNothingElse(t *testing.T) {
root := t.TempDir()
files := map[string][]byte{
"modules/a/a": []byte("\x7fELF\x02\x01\x01\x00rest"),
"modules/b/cmd/b/b.exe": []byte("MZ\x90\x00"),
"modules/c/c": {0xcf, 0xfa, 0xed, 0xfe, 7, 0, 0, 1},
"modules/d/main.go": []byte("package main\n"),
"modules/e/short": []byte("\x7f"),
"modules/f/empty": {},
"modules/g/Foo.class": {0xca, 0xfe, 0xba, 0xbe, 0, 0, 0, 52}, // Java, not Mach-O
"checks/words/words": []byte("\x7fELF\x02\x01\x01\x00rest"), // outside modules/ too
}
var names []string
for f, b := range files {
p := filepath.Join(root, f)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, b, 0o644); err != nil {
t.Fatal(err)
}
names = append(names, f)
}
got, err := Find(root, names)
if err != nil {
t.Fatal(err)
}
if len(got) != 4 {
t.Fatalf("found %v; wanted modules/a/a, modules/b/cmd/b/b.exe and modules/c/c and checks/words/words only", got)
}
}
// The catalogue's own tree holds no compiled program: a compiled program is build output, so remove it and add it to .gitignore.
func TestTheCatalogueHoldsNoCompiledProgram(t *testing.T) {
root := filepath.Join("..", "..")
files, err := Files(root)
if err != nil {
t.Fatal(err)
}
found, err := Find(root, files)
if err != nil {
t.Fatal(err)
}
for _, f := range found {
t.Errorf("%s: a compiled program is build output: remove it and add it to .gitignore (issue 1577)", f)
}
}
// A compiled program outside modules/ is refused too (a binary under checks/ slipped past the modules/-only rule): Files holds
// every tracked file of a git checkout and every file of a plain folder, and Find names the binary in both.
func TestAProgramUnderChecksIsRefused(t *testing.T) {
for _, git := range []bool{true, false} {
root := t.TempDir()
p := filepath.Join(root, "checks", "words", "words")
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte("\x7fELF\x02\x01\x01\x00rest"), 0o644); err != nil {
t.Fatal(err)
}
if git {
for _, args := range [][]string{{"init", "-q"}, {"add", "-A"}} {
if out, err := exec.Command("git", append([]string{"-C", root}, args...)...).CombinedOutput(); err != nil {
t.Fatalf("git %v: %v: %s", args, err, out)
}
}
}
files, err := Files(root)
if err != nil {
t.Fatal(err)
}
found, err := Find(root, files)
if err != nil {
t.Fatal(err)
}
if len(found) != 1 {
t.Errorf("git=%v: found %v; wanted checks/words/words (ELF)", git, found)
}
}
}
-1
View File
@@ -1 +0,0 @@
# every tracked file of the repository (git ls-files, no path limit): the whole tree, so this package's verdict is never kept.
-4
View File
@@ -1,4 +0,0 @@
# The modules of this catalogue that still read the controller's assignments (mesh-controller.assignments), each with
# why. The check of this directory (noplacement) refuses any other module that reads them, and an entry here whose module no longer
# does: the list only shrinks, and it is empty before the controller's lease is written to a holder that is no
# controller (novox/hq ADR 0549 §3, §6; issue 774).
-7
View File
@@ -1,7 +0,0 @@
module git.novox.be/novox/mesh-catalog/checks/noplacement
go 1.22
require git.novox.be/novox/mesh-catalog/checks/trunk v0.0.0
replace git.novox.be/novox/mesh-catalog/checks/trunk => ../trunk
-170
View File
@@ -1,170 +0,0 @@
// Command noplacement refuses what the repository path of placement left behind (novox/hq ADR 0567 §10, §12).
//
// Placement, which module runs on which node, is bus state held by the seat mesh-placement; no file of the
// catalogue says it, and no manifest reads it from the delivery indexer, which no longer keeps it. It refuses:
//
// 1. a `placement` directory (or file) at the root of the catalogue;
//
// 2. a `particulars` file at the root (the mesh's node names come from the checker's run, §16);
//
// 3. a manifest naming a state, stream, event or bucket of the indexer's placement (`mesh-delivery-indexer.placement…`
// or `mesh-delivery-indexer_placement…`) in any string value, such as `reads`;
//
// 4. a manifest, other than mesh-controller's, that reads the controller's assignments (`mesh-controller.assignments`):
// every reader takes placement from the seat mesh-placement (novox/hq ADR 0549 §3, ADR 0567), and the lease write is
// refused while one still reads them (ADR 0549 §6; issue 774). The modules that still read them for what the
// placement does not carry are named in checks/noplacement/assignments-readers, each with why; that list only
// shrinks: an entry whose module no longer reads them is refused too.
//
// go run . <catalogue root>
//
// Exits 1 on a finding.
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
)
// Prefixes are the names of the indexer's placement parts; a manifest string starting with one names them.
var Prefixes = []string{"mesh-delivery-indexer.placement", "mesh-delivery-indexer_placement"}
// Findings is every refusal in the catalogue at root, sorted.
func Findings(root string) ([]string, error) {
var out []string
if _, err := os.Lstat(filepath.Join(root, "placement")); err == nil {
out = append(out, "placement: the repository holds no placement; it is bus state of the seat mesh-placement (novox/hq ADR 0567 §10)")
}
if _, err := os.Lstat(filepath.Join(root, "particulars")); err == nil {
out = append(out, "particulars: the repository lists no node names; the checker hands them to the check (novox/hq ADR 0567 §16)")
}
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
sort.Strings(manifests)
for _, file := range manifests {
raw, err := os.ReadFile(file)
if err != nil {
return nil, err
}
var doc any
if err := json.Unmarshal(raw, &doc); err != nil {
return nil, fmt.Errorf("%s: %w", file, err)
}
module := filepath.Base(filepath.Dir(file))
for _, s := range named(doc, nil) {
out = append(out, fmt.Sprintf("%s: names %q, which the indexer no longer keeps; placement is the seat mesh-placement's (novox/hq ADR 0567 §12)", module, s))
}
}
readers, err := assignmentsReaders(root)
if err != nil {
return nil, err
}
out = append(out, readers...)
sort.Strings(out)
return out, nil
}
// AssignmentsRead is the controller's assignments as a manifest reads them, and Allowed the file, relative to the
// catalogue's root, that names the modules still reading them: `<module> <why>` per line, `#` a comment.
const (
AssignmentsRead = "mesh-controller.assignments"
Allowed = "checks/noplacement/assignments-readers"
)
// assignmentsReaders is every finding on the reads of the controller's assignments: a module other than the controller
// reading them that Allowed does not name, and a module Allowed names that no longer reads them (issue 774).
func assignmentsReaders(root string) ([]string, error) {
allowed := map[string]bool{}
raw, err := os.ReadFile(filepath.Join(root, Allowed))
if err != nil && !os.IsNotExist(err) {
return nil, err
}
for _, line := range strings.Split(string(raw), "\n") {
if f := strings.Fields(line); len(f) > 0 && !strings.HasPrefix(f[0], "#") {
allowed[f[0]] = true
}
}
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
var found []string
reads := map[string]bool{}
for _, path := range manifests {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var m struct {
Module string `json:"module"`
Reads []string `json:"reads"`
}
if json.Unmarshal(raw, &m) != nil || m.Module == "mesh-controller" {
continue // a manifest that does not read is the manifest checks' to refuse
}
for _, r := range m.Reads {
if strings.TrimSpace(r) != AssignmentsRead {
continue
}
reads[m.Module] = true
if !allowed[m.Module] {
found = append(found, fmt.Sprintf("%s reads %s: read where modules run from mesh-delivery-assigner.placement, "+
"the seat mesh-placement's (novox/hq ADR 0549 §3, ADR 0567); the lease write is refused while a module reads "+
"the assignments (ADR 0549 §6)", m.Module, AssignmentsRead))
}
}
}
for module := range allowed {
if !reads[module] {
found = append(found, fmt.Sprintf("%s names %s, which no longer reads %s: take it off the list, which only shrinks",
Allowed, module, AssignmentsRead))
}
}
return found, nil
}
// named collects the strings of a decoded manifest that start with one of the Prefixes.
func named(v any, into []string) []string {
switch v := v.(type) {
case string:
for _, p := range Prefixes {
if strings.HasPrefix(v, p) {
return append(into, v)
}
}
case []any:
for _, e := range v {
into = named(e, into)
}
case map[string]any:
for _, e := range v {
into = named(e, into)
}
}
return into
}
func main() {
if len(os.Args) != 2 {
fmt.Fprintln(os.Stderr, "usage: noplacement <catalogue root>")
os.Exit(2)
}
found, err := Findings(os.Args[1])
if err != nil {
fmt.Println(err)
os.Exit(1)
}
for _, f := range found {
fmt.Println(f)
}
if len(found) > 0 {
os.Exit(1)
}
fmt.Println("no placement directory, no manifest reads the indexer's placement, and only the modules listed read the controller's assignments")
}
-88
View File
@@ -1,88 +0,0 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-catalog/checks/trunk"
)
func TestFindingsRefuseWhatThePlacementPathLeft(t *testing.T) {
for _, row := range []struct {
name string
files map[string]string
want string // empty: clean
}{
{"clean", map[string]string{"modules/a/module.json": `{"module":"a","reads":["mesh-delivery-assigner.placement"]}`}, ""},
{"a placement directory", map[string]string{"placement/ace.json": `{}`}, "placement: the repository holds no placement"},
{"a particulars file", map[string]string{"particulars": "ace\n"}, "particulars: the repository lists no node names"},
{"a read of the indexer's placement", map[string]string{"modules/a/module.json": `{"module":"a","reads":["mesh-delivery-indexer.placement"]}`}, `a: names "mesh-delivery-indexer.placement"`},
{"a read of its candidates", map[string]string{"modules/b/module.json": `{"module":"b","reads":["mesh-delivery-indexer.placement-candidates"]}`}, "placement-candidates"},
{"its event", map[string]string{"modules/c/module.json": `{"module":"c","on":[{"event":"mesh-delivery-indexer.placement-changed"}]}`}, "placement-changed"},
// Issue 774 (ADR 0549 §3, §6): a reader of the controller's assignments is refused unless the list names it, the
// controller's own manifest is not judged, and a name on the list whose module no longer reads them is refused.
{"an unlisted reader of the assignments", map[string]string{"modules/r/module.json": `{"module":"r","reads":["mesh-controller.assignments"]}`}, "r reads mesh-controller.assignments"},
{"a listed reader and the controller", map[string]string{"modules/l/module.json": `{"module":"l","reads":["mesh-controller.assignments"]}`,
"modules/mesh-controller/module.json": `{"module":"mesh-controller","reads":["mesh-controller.assignments"]}`, Allowed: "# why\nl its plan\n"}, ""},
{"a listed module that reads them no more", map[string]string{Allowed: "gone no longer reads\n"}, "names gone, which no longer reads"},
{"the seat's own state is allowed", map[string]string{"modules/d/module.json": `{"module":"d","reads":["mesh-placement.placement"]}`}, ""},
} {
t.Run(row.name, func(t *testing.T) {
root := t.TempDir()
for path, text := range row.files {
p := filepath.Join(root, path)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte(text), 0o644); err != nil {
t.Fatal(err)
}
}
got, err := Findings(root)
if err != nil {
t.Fatal(err)
}
if all := strings.Join(got, "\n"); (row.want == "") != (all == "") || !strings.Contains(all, row.want) {
t.Fatalf("findings = %q, want one containing %q", all, row.want)
}
})
}
}
// The script's run of this check over the real tree (novox/hq ADR 0529 clause 2).
func TestTheCatalogueHoldsNoPlacementOfTheRepositoryPath(t *testing.T) {
got, err := Findings(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
for _, f := range got {
t.Error(f)
}
// Issue 774 (ADR 0549 §3, §6; the comparison of novox/hq ADR 0529 clause 2): the list of modules still reading the
// controller's assignments gains none the trunk's copy at the merge base does not hold. Its first copy gains nothing.
path, ok := trunk.Copy(t, Allowed)
if !ok {
return
}
names := func(path string) map[string]bool {
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
out := map[string]bool{}
for _, line := range strings.Split(string(raw), "\n") {
if f := strings.Fields(line); len(f) > 0 && !strings.HasPrefix(f[0], "#") {
out[f[0]] = true
}
}
return out
}
base := names(path)
for module := range names(filepath.Join("..", "..", Allowed)) {
if !base[module] {
t.Errorf("%s gains %s, which the trunk's copy does not hold: the list only shrinks", Allowed, module)
}
}
}
-6
View File
@@ -1,6 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# the root, for a placement directory, and every manifest, for a read of the indexer's placement states.
?placement
modules/*/module.json
-3
View File
@@ -1,3 +0,0 @@
module checks/noscripts
go 1.22
-19
View File
@@ -1,19 +0,0 @@
package noscripts
import (
"errors"
"os"
"path/filepath"
"testing"
)
// The catalogue's checks are Go tests (novox/hq ADR 0529 clause 5, ADR 0445): the checker runs every nested Go module's tests by
// the language's convention and no script at the root says what else to run. A script back at the root would be a check nobody
// runs (ADR 0503 §3: since R4e the checker has not run them), so it is refused.
func TestNoScriptOfTheOldCheckStandsAtTheRoot(t *testing.T) {
for _, name := range []string{"merge-check.sh", "merge-check-typescript.sh", "check-nested-modules.sh"} {
if _, err := os.Stat(filepath.Join("..", "..", name)); !errors.Is(err, os.ErrNotExist) {
t.Errorf("%s stands at the catalogue's root (%v): what it ran is a Go test in checks/ or a step of the build module", name, err)
}
}
}
-7
View File
@@ -1,7 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# that none of the old check scripts stands at the root.
?merge-check.sh
?merge-check-typescript.sh
?check-nested-modules.sh
-4
View File
@@ -1,7 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/owned
go 1.22
require git.novox.be/novox/mesh-catalog/checks/trunk v0.0.0
replace git.novox.be/novox/mesh-catalog/checks/trunk => ../trunk
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest; its lists are this package's own files.
modules/*/module.json
-41
View File
@@ -1,41 +0,0 @@
package main
import (
"testing"
"git.novox.be/novox/mesh-catalog/checks/trunk"
)
// The script's run of this check over the real tree (novox/hq ADR 0529 clause 2): every rule holds.
func TestTheCatalogueKeepsEveryRuleOfOwnedPaths(t *testing.T) {
found, err := Check("../..", nil)
if err != nil {
t.Fatal(err)
}
for _, f := range found {
t.Errorf("%s: %s: %s (rule %d, novox/hq ADR 0422 §8)", f.Module, f.ID, f.Why, f.Rule)
}
}
// And the list of system directories' known exceptions holds no line the trunk's copy at the merge base does not (rule 5). A trunk
// with none had no list: its first copy. Skipped, and said, where no trunk is beside.
func TestTheSystemListHoldsNoLineTheTrunksDoesNot(t *testing.T) {
path, ok := trunk.Copy(t, "checks/owned/system-known")
var base []string
if ok {
lines, err := readLines(path)
if err != nil {
t.Fatal(err)
}
base = append([]string{}, lines...)
}
found, err := Check("../..", base)
if err != nil {
t.Fatal(err)
}
for _, f := range found {
if f.Rule == 5 {
t.Errorf("%s: %s: %s (rule %d, issue 1841)", f.Module, f.ID, f.Why, f.Rule)
}
}
}
+7 -22
View File
@@ -2,9 +2,8 @@
// issue 1838): a module is for any mesh, so no manifest, file, code string, identifier, comment, test name or
// fixture of a covered module names a node of this one.
//
// The names are the mesh's own, not the repository's: the checker hands them in the environment value
// MESH_NODE_NAMES, comma-separated, from the node records it reads (novox/hq ADR 0567 §16). A run with none fails
// loudly, since it would pass on anything. The modules covered are in `covered` at checks/particulars/covered, one a line. A name is matched
// The names are in the file `particulars` at the catalogue's root, one a line, `#` comments — the one place
// they may be written. The modules covered are in `covered` at checks/particulars/covered, one a line. A name is matched
// whole and in any case; a hyphen, an underscore or a CamelCase boundary parts one, so a longer word holding a name is another word.
// The forge is not a node name: its host (git.novox.be) and the organisation path (novox/<repo>) are blanked
// before a line is read.
@@ -35,28 +34,14 @@ func main() {
if len(os.Args) > 1 {
root = os.Args[1]
}
os.Exit(check(root, namesFromEnv()))
}
// NamesEnv is the environment value that carries the mesh's node names (mesh-sdk delivery.NodeNamesEnv).
const NamesEnv = "MESH_NODE_NAMES"
// namesFromEnv is the names NamesEnv holds, comma-separated, trimmed, empty entries left out.
func namesFromEnv() []string {
var out []string
for _, n := range strings.Split(os.Getenv(NamesEnv), ",") {
if n = strings.TrimSpace(n); n != "" {
out = append(out, n)
}
}
return out
os.Exit(check(root))
}
// check runs the check over a catalogue and returns the exit code: 0 clean, 1 a finding, 2 a list unreadable.
func check(root string, names []string) int {
var err error
if len(names) == 0 {
err = fmt.Errorf("no node names (%s is empty) — the check would pass on anything", NamesEnv)
func check(root string) int {
names, err := readList(filepath.Join(root, "particulars"))
if err == nil && len(names) == 0 {
err = fmt.Errorf("particulars lists no name — the check would pass on anything")
}
var covered []string
if err == nil {
+7 -15
View File
@@ -74,24 +74,26 @@ func TestANodeNamedPathIsRefusedThroughRun(t *testing.T) {
}
}
func TestCheckReturnsOneOnAFindingAndTwoOnNoNames(t *testing.T) {
func TestCheckReturnsOneOnAFindingAndTwoOnAnUnreadableList(t *testing.T) {
root := t.TempDir()
write := func(p, s string) {
p = filepath.Join(root, p)
os.MkdirAll(filepath.Dir(p), 0o755)
os.WriteFile(p, []byte(s), 0o644)
}
write("particulars", "ace\n")
write("checks/particulars/covered", "kernel\n")
write("modules/kernel/a", "clean\n")
if got := check(root, []string{"ace"}); got != 0 {
if got := check(root); got != 0 {
t.Errorf("clean = %d, want 0", got)
}
write("modules/kernel/a", "on ace\n")
if got := check(root, []string{"ace"}); got != 1 {
if got := check(root); got != 1 {
t.Errorf("finding = %d, want 1", got)
}
if got := check(root, nil); got != 2 {
t.Errorf("no names = %d, want 2", got)
os.Remove(filepath.Join(root, "particulars"))
if got := check(root); got != 2 {
t.Errorf("no list = %d, want 2", got)
}
}
@@ -112,13 +114,3 @@ func TestOnlyCoveredModulesAreRead(t *testing.T) {
t.Errorf("got %v", got)
}
}
// The script's run of this check over the real tree (novox/hq ADR 0529 clause 2): no node name of this mesh in a covered module.
func TestTheCoveredModulesNameNoNode(t *testing.T) {
if _, set := os.LookupEnv(NamesEnv); !set {
t.Skip("ADR 0567 §16: this checker hands no node names (before D14 is delivered); the skip goes once it is")
}
if code := check("../..", namesFromEnv()); code != 0 {
t.Fatalf("the check of the real catalogue exits %d (see what it printed above)", code)
}
}
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# check("../.."): every file of the modules the covered list names. The node names come from the run (MESH_NODE_NAMES).
modules
-4
View File
@@ -1,7 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/paths
go 1.22
require git.novox.be/novox/mesh-catalog/checks/trunk v0.0.0
replace git.novox.be/novox/mesh-catalog/checks/trunk => ../trunk
+1 -2
View File
@@ -251,8 +251,7 @@ func main() {
}
var grown []string
if baseDir == "" {
fmt.Fprintln(os.Stderr, "NOT COMPARED: no base branch's lists were given (-base <dir>), so additions to them were not judged; that is a failure, not a pass (issue 1295)")
os.Exit(2)
fmt.Println("NOT COMPARED: no base branch's lists were given, so additions to them were not judged")
} else {
baseShared, _, err1 := list(filepath.Join(baseDir, "shared"))
baseKnown, _, err2 := list(filepath.Join(baseDir, "known"))
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest and every file of a module.
modules
-63
View File
@@ -1,63 +0,0 @@
package main
import (
"path/filepath"
"testing"
"git.novox.be/novox/mesh-catalog/checks/trunk"
)
// The script's run of this check over the real tree (novox/hq ADR 0529 clause 2): no module names another's state on its machine,
// and the known list holds only what still occurs.
func TestNoModuleNamesAnothersState(t *testing.T) {
root := filepath.Join("..", "..")
found, err := Check(root)
if err != nil {
t.Fatal(err)
}
shared, _, err := list("shared")
if err != nil {
t.Fatal(err)
}
known, _, err := list("known")
if err != nil {
t.Fatal(err)
}
refused, gone := Judge(found, shared, known)
for _, f := range refused {
t.Error(f)
}
for _, k := range gone {
t.Errorf("%q is in checks/paths/known and no longer occurs: take it out, so it cannot come back", k)
}
}
// And neither list gains a line the trunk's copies at the merge base do not hold, unless the line says why after `# allow:`. Fails, saying
// NOT COMPARED, where no trunk is beside (issue 1295).
func TestTheListsOfPathsHoldNoLineTheTrunksDoNot(t *testing.T) {
sharedPath, okShared := trunk.Copy(t, "checks/paths/shared")
knownPath, okKnown := trunk.Copy(t, "checks/paths/known")
if !okShared || !okKnown {
t.Log("the trunk holds not both lists: their first copies, to which nothing was added")
return
}
shared, sharedWhy, err := list("shared")
if err != nil {
t.Fatal(err)
}
known, knownWhy, err := list("known")
if err != nil {
t.Fatal(err)
}
baseShared, _, err := list(sharedPath)
if err != nil {
t.Fatal(err)
}
baseKnown, _, err := list(knownPath)
if err != nil {
t.Fatal(err)
}
for _, g := range append(Grown("shared", shared, sharedWhy, baseShared), Grown("known", known, knownWhy, baseKnown)...) {
t.Error(g)
}
}
-1
View File
@@ -1,4 +1,3 @@
# Paths every program on a machine reads as the machine's own, not as a message from the module that
# writes them (checks/paths, novox/hq ADR 0260). One per line, with why.
/etc/hosts # every resolver on the machine reads it; the hostname module writes it for all of them
/var/lib/mesh-build-node-cache # allow: the Go build module gives its integration level the node's installed Node toolchains read-only, so the Node build module's tests build offline (issue 1891, ADR 0533)
-3
View File
@@ -1,3 +0,0 @@
module checks/planhands
go 1.26.0
-48
View File
@@ -1,48 +0,0 @@
// Package planhands refuses a module's line that takes a plan existing for its module handed to the provisioner (issue
// 774). Whether a plan hands its module is mesh-sdk's one rule, catalogue.Plan.Hands: since the seat mesh-provisioner keys a
// provision its provider's helper answers for a module not handed, a plan naming provisions alone exists for modules
// whose files and own secrets stay their node's (on 2026-10-30 the vault read every such plan as handed and answered
// twenty modules' own secrets to nobody, and the provisioner client refused each as a plan it could not act on). A line
// that tests whether a plan exists either asks Hands on the same line, or says "not handing": it reads only who binds
// whom, or that there is no plan at all.
package planhands
import (
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
)
// existence is a test of whether an assignment's or a plan's `provisioning` is there: nil, or an empty raw value.
var existence = regexp.MustCompile(`Provisioning\s*[!=]=\s*nil|len\([^)]*Provisioning\)\s*[!=]=\s*0`)
// Problems is each such line under root's modules/ that neither asks Hands nor says "not handing", as
// `path:line: text`, sorted. Test files are left out: a test may build a plan by hand.
func Problems(root string) ([]string, error) {
var out []string
err := filepath.WalkDir(filepath.Join(root, "modules"), func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() || !strings.HasSuffix(p, ".go") || strings.HasSuffix(p, "_test.go") {
return nil
}
raw, err := os.ReadFile(p)
if err != nil {
return err
}
rel, _ := filepath.Rel(root, p)
for i, line := range strings.Split(string(raw), "\n") {
if existence.MatchString(line) && !strings.Contains(line, ".Hands()") && !strings.Contains(line, "not handing") {
out = append(out, fmt.Sprintf("%s:%d: %s", rel, i+1, strings.TrimSpace(line)))
}
}
return nil
})
sort.Strings(out)
return out, err
}
-45
View File
@@ -1,45 +0,0 @@
package planhands
import (
"os"
"path/filepath"
"strings"
"testing"
)
// A line that takes a plan existing for its module handed is refused and named; one that asks Hands, one marked "not
// handing" and any test file pass; and the real catalogue is clean.
func TestNoModuleTakesAPlanExistingForItsModuleHanded(t *testing.T) {
root := t.TempDir()
for f, body := range map[string]string{
"modules/vault/own.go": "if a.Provisioning != nil {\n",
"modules/client/plan.go": "if len(a.Provisioning) == 0 {\n",
"modules/vault/rule.go": "return a.Provisioning != nil && a.Provisioning.Hands(), nil\n",
"modules/client/binds.go": "if a.Provisioning != nil { // binds, not handing\n",
"modules/vault/own_test.go": "if a.Provisioning == nil {\n",
"modules/client/notgo/plan.md": "if a.Provisioning == nil {\n",
} {
p := filepath.Join(root, f)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
got, err := Problems(root)
if err != nil {
t.Fatal(err)
}
if len(got) != 2 || !strings.HasPrefix(got[0], "modules/client/plan.go:1:") || !strings.HasPrefix(got[1], "modules/vault/own.go:1:") {
t.Fatalf("refused %v, wanted the client's and the vault's existence tests alone", got)
}
real, err := Problems(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
if len(real) > 0 {
t.Errorf("a module takes a plan existing for its module handed; ask catalogue.Plan.Hands, or mark a line that reads "+
"only who binds whom \"not handing\":\n%s", strings.Join(real, "\n"))
}
}
-3
View File
@@ -1,3 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (issue 1936).
# every module whole (a directory is read whole, so a module's Go source at any depth enters the digest), for a line that takes a plan existing for its module handed.
modules
-4
View File
@@ -1,7 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/predefined
go 1.22
require git.novox.be/novox/mesh-catalog/checks/trunk v0.0.0
replace git.novox.be/novox/mesh-catalog/checks/trunk => ../trunk
-6
View File
@@ -1,6 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# the root's list of predefined keys and every manifest that names one.
predefined.json
modules/*/module.json
-41
View File
@@ -1,41 +0,0 @@
package main
import (
"path/filepath"
"sort"
"testing"
"git.novox.be/novox/mesh-catalog/checks/trunk"
)
// The script's run of this check over the real tree (novox/hq ADR 0529 clause 2): predefined.json removes no key a module still
// names, judged against the trunk's copy at the merge base. A trunk with none had no keys; with no trunk to read, nothing can be
// seen removed, and the test says so.
func TestThePredefinedListRemovesNoKeyAModuleStillNames(t *testing.T) {
tree, err := Keys(filepath.Join("..", "..", File))
if err != nil {
t.Fatal(err)
}
path, ok := trunk.Copy(t, File)
if !ok {
t.Log("no trunk copy of " + File + ": no removal can be seen")
return
}
base, err := Keys(path)
if err != nil {
t.Fatal(err)
}
named, err := Named(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
removed := Removed(base, tree, named)
keys := make([]string, 0, len(removed))
for k := range removed {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
t.Errorf("%q is removed from %s and still named by %v", k, File, removed[k])
}
}
-5
View File
@@ -1,5 +0,0 @@
module checks/readable
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0
-2
View File
@@ -1,2 +0,0 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0 h1:lKU3HjNXGiVbLW2hUgFjU/TMTUYMlNnnmZxG5T7biQA=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
-333
View File
@@ -1,333 +0,0 @@
// Command readable refuses a catalogue where a module reads another module's state that the owner does not announce
// readable (novox/hq ADR 0495 clause 3, issue 1790): the controller grants a read only from the owner's announcement, so
// such a read is refused on the live mesh, said only in the controller's journal. The sweep of 2026-10-30 (issue 774)
// found three: gitea's `bound` (read by mesh-provisioner, so no forge account was bound), disk-load's `readings` (read by
// restic, nfs-server and mounts) and restic's `good` (read by the self-check). The merge gate's check of reads against
// the owner's manifest went with clause 5, and announcetest holds a module to its own list, not to its readers.
//
// An owner announces from its code. A Go module announces when its code imports mesh-sdk's go/announce: every
// announce.State literal there is read, its Name a string, a constant of the package, or a constant of mesh-sdk this
// check knows (Known). A TypeScript module announces when its code says the event `states` or calls the SDK's keeps:
// every `{ name: "...", readable: true }` there is read. An owner that announces nothing from code is read as before
// on the mesh, and is not judged; neither is an owner outside the catalogue (another repository's).
//
// go run . <catalog root>
//
// Exits 1 on a finding.
package main
import (
"encoding/json"
"fmt"
"go/ast"
"go/parser"
"go/token"
"os"
"path/filepath"
"regexp"
"sort"
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/health"
)
// Known are the mesh-sdk constants an announcement may name a state by, by the selector the code writes.
var Known = map[string]string{
"health.GoodState": health.GoodState,
"health.ReadingsState": health.ReadingsState,
}
const announcePkg = `"git.novox.be/novox/mesh-sdk/go/announce"`
// Announced is what one owner's code announces: each state, readable or not.
type Announced map[string]bool
// Findings is every refusal in the catalogue at root, sorted.
func Findings(root string) ([]string, error) {
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
dirs := map[string]string{}
reads := map[string][]string{}
for _, path := range manifests {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var m struct {
Module string `json:"module"`
Reads []json.RawMessage `json:"reads"`
}
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
dirs[m.Module] = filepath.Dir(path)
for _, r := range m.Reads {
var s string
if json.Unmarshal(r, &s) == nil {
reads[m.Module] = append(reads[m.Module], s)
}
}
}
var out []string
owners := map[string]Announced{}
for reader, rs := range reads {
for _, r := range rs {
owner, state, ok := strings.Cut(r, ".")
dir, inCatalogue := dirs[owner]
if !ok || !inCatalogue || owner == reader {
continue
}
a, seen := owners[owner]
if !seen {
var problems []string
a, problems, err = AnnouncedBy(dir)
if err != nil {
return nil, err
}
owners[owner] = a
for _, p := range problems {
out = append(out, owner+": "+p)
}
}
if a == nil {
continue // announces nothing from code: read as before
}
readable, keeps := a[state]
switch {
case !keeps:
out = append(out, fmt.Sprintf("%s reads %s.%s, which %s's code does not announce: the read is refused on the mesh", reader, owner, state, owner))
case !readable:
out = append(out, fmt.Sprintf("%s reads %s.%s, which %s's code announces not readable: the read is refused on the mesh; announce it readable", reader, owner, state, owner))
}
}
}
sort.Strings(out)
return dedupe(out), nil
}
func dedupe(in []string) []string {
var out []string
for i, s := range in {
if i == 0 || s != in[i-1] {
out = append(out, s)
}
}
return out
}
var (
tsAnnounces = regexp.MustCompile(`emit\(\s*"states"|\bkeeps\(`)
tsState = regexp.MustCompile(`\{\s*name:\s*"([a-z0-9][a-z0-9-]*)"\s*(,\s*readable:\s*(true|false)\s*)?\}`)
)
// AnnouncedBy is what the module in dir announces from its code, nil when it announces nothing; problems are
// announcements whose state's name cannot be read.
func AnnouncedBy(dir string) (Announced, []string, error) {
var out Announced
var problems []string
pkgs := map[string][]*ast.File{}
fset := token.NewFileSet()
err := filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
if n := d.Name(); path != dir && (n == "node_modules" || n == "dist" || n == "test" || n == "testdata" || strings.HasPrefix(n, ".")) {
return filepath.SkipDir
}
return nil
}
name := d.Name()
switch {
case strings.HasSuffix(name, ".go") && !strings.HasSuffix(name, "_test.go"):
f, err := parser.ParseFile(fset, path, nil, 0)
if err != nil {
return err
}
pkgs[filepath.Dir(path)] = append(pkgs[filepath.Dir(path)], f)
case strings.HasSuffix(name, ".ts") && !strings.HasSuffix(name, ".d.ts") && !strings.HasSuffix(name, ".test.ts"):
raw, err := os.ReadFile(path)
if err != nil {
return err
}
if !tsAnnounces.Match(raw) && !strings.Contains(string(raw), "KEPT") {
return nil
}
for _, m := range tsState.FindAllStringSubmatch(string(raw), -1) {
if out == nil {
out = Announced{}
}
out[m[1]] = out[m[1]] || m[3] == "true"
}
}
return nil
})
if err != nil {
return nil, nil, err
}
for _, files := range pkgs {
imports := false
consts := map[string]ast.Expr{}
for _, f := range files {
for _, im := range f.Imports {
if im.Path.Value == announcePkg && (im.Name == nil || im.Name.Name == "announce") {
imports = true
}
}
for _, decl := range f.Decls {
g, ok := decl.(*ast.GenDecl)
if !ok || g.Tok != token.CONST {
continue
}
for _, spec := range g.Specs {
v := spec.(*ast.ValueSpec)
for i, n := range v.Names {
if i < len(v.Values) {
consts[n.Name] = v.Values[i]
}
}
}
}
}
if !imports {
continue
}
if out == nil {
out = Announced{}
}
for _, f := range files {
ast.Inspect(f, func(n ast.Node) bool {
lit, ok := n.(*ast.CompositeLit)
if !ok {
return true
}
states := []*ast.CompositeLit{}
switch {
case isAnnounceState(lit.Type):
states = append(states, lit)
case isStateSlice(lit.Type):
for _, e := range lit.Elts {
if c, ok := e.(*ast.CompositeLit); ok && (c.Type == nil || isAnnounceState(c.Type)) {
states = append(states, c)
}
}
default:
return true
}
for _, s := range states {
name, readable, problem := stateOf(s, consts)
if problem != "" {
problems = append(problems, fmt.Sprintf("%s: %s", fset.Position(s.Pos()), problem))
continue
}
out[name] = out[name] || readable
}
return false
})
}
}
return out, problems, nil
}
func isAnnounceState(e ast.Expr) bool {
s, ok := e.(*ast.SelectorExpr)
if !ok {
return false
}
x, ok := s.X.(*ast.Ident)
return ok && x.Name == "announce" && s.Sel.Name == "State"
}
func isStateSlice(e ast.Expr) bool {
a, ok := e.(*ast.ArrayType)
return ok && isAnnounceState(a.Elt)
}
// stateOf is one announce.State literal's name and whether it is readable (Readable true, or Readers named).
func stateOf(lit *ast.CompositeLit, consts map[string]ast.Expr) (string, bool, string) {
var name string
var readable bool
named := false
for _, e := range lit.Elts {
kv, ok := e.(*ast.KeyValueExpr)
if !ok {
return "", false, "an announce.State without field names cannot be read; name its fields"
}
key, _ := kv.Key.(*ast.Ident)
if key == nil {
continue
}
switch key.Name {
case "Name":
v, ok := resolve(kv.Value, consts, 0)
if !ok {
return "", false, fmt.Sprintf("the state's name %s cannot be read: name it by a string, a constant of the package, or a mesh-sdk constant in checks/readable's Known", exprString(kv.Value))
}
name, named = v, true
case "Readable":
if id, ok := kv.Value.(*ast.Ident); ok {
readable = readable || id.Name == "true"
}
case "Readers":
readable = true
}
}
if !named {
return "", false, "an announce.State without a Name"
}
return name, readable, ""
}
func resolve(e ast.Expr, consts map[string]ast.Expr, depth int) (string, bool) {
if depth > 8 {
return "", false
}
switch v := e.(type) {
case *ast.BasicLit:
if v.Kind == token.STRING {
s, err := strconv.Unquote(v.Value)
return s, err == nil
}
case *ast.Ident:
if c, ok := consts[v.Name]; ok {
return resolve(c, consts, depth+1)
}
case *ast.SelectorExpr:
s, ok := Known[exprString(v)]
return s, ok
}
return "", false
}
func exprString(e ast.Expr) string {
switch v := e.(type) {
case *ast.Ident:
return v.Name
case *ast.SelectorExpr:
return exprString(v.X) + "." + v.Sel.Name
case *ast.BasicLit:
return v.Value
}
return fmt.Sprintf("%T", e)
}
func main() {
root := "."
if len(os.Args) > 1 {
root = os.Args[1]
}
found, err := Findings(root)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
for _, f := range found {
fmt.Println(f)
}
if len(found) > 0 {
os.Exit(1)
}
}
-90
View File
@@ -1,90 +0,0 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
)
// A read of another module's state is refused here when the owner's code announces that state not readable, or not at
// all (the sweep of 2026-10-30, issue 774); an owner that announces nothing from code, or lives in another repository,
// is not judged.
func TestFindings(t *testing.T) {
const goOwner = "package main\n\nimport \"git.novox.be/novox/mesh-sdk/go/announce\"\n\n"
for _, row := range []struct {
name string
files map[string]string
want string // empty: clean
}{
{"go, readable by a constant", map[string]string{
"modules/r/module.json": `{"module":"r","reads":["o.s"]}`,
"modules/o/module.json": `{"module":"o"}`,
"modules/o/cmd/o/main.go": goOwner + "const S = \"s\"\n\nvar _ = announce.State{Name: S, Readable: true}\n",
"modules/o/cmd/o/main_test.go": goOwner + "var _ = announce.State{Name: \"s\"}\n"}, ""},
{"go, not readable", map[string]string{
"modules/r/module.json": `{"module":"r","reads":["o.s"]}`,
"modules/o/module.json": `{"module":"o"}`,
"modules/o/cmd/o/main.go": goOwner + "var Kept = []announce.State{{Name: \"t\", Readable: true}, {Name: \"s\"}}\n"},
"r reads o.s, which o's code announces not readable"},
{"go, a known mesh-sdk name and readers", map[string]string{
"modules/r/module.json": `{"module":"r","reads":["o.good","o.s"]}`,
"modules/o/module.json": `{"module":"o"}`,
"modules/o/cmd/o/main.go": goOwner + "var _ = []announce.State{{Name: health.GoodState, Readable: true}, {Name: \"s\", Readers: []string{\"r\"}}}\n"}, ""},
{"go, a name that cannot be read", map[string]string{
"modules/r/module.json": `{"module":"r","reads":["o.s"]}`,
"modules/o/module.json": `{"module":"o"}`,
"modules/o/cmd/o/main.go": goOwner + "var _ = announce.State{Name: other.S, Readable: true}\n"},
"the state's name other.S cannot be read"},
{"go, a state not announced", map[string]string{
"modules/r/module.json": `{"module":"r","reads":["o.s"]}`,
"modules/o/module.json": `{"module":"o"}`,
"modules/o/cmd/o/main.go": goOwner + "var _ = announce.State{Name: \"t\"}\n"},
"r reads o.s, which o's code does not announce"},
{"typescript, not readable", map[string]string{
"modules/r/module.json": `{"module":"r","reads":["o.bound"]}`,
"modules/o/module.json": `{"module":"o"}`,
"modules/o/kept.ts": `export const KEPT = [{ name: "announced" }, { name: "bound" }];`},
"r reads o.bound, which o's code announces not readable"},
{"typescript, readable", map[string]string{
"modules/r/module.json": `{"module":"r","reads":["o.bound"]}`,
"modules/o/module.json": `{"module":"o"}`,
"modules/o/index.ts": `await keeps([{ name: "bound", readable: true }]);`}, ""},
{"an owner that announces nothing, and one outside the catalogue", map[string]string{
"modules/r/module.json": `{"module":"r","reads":["o.s","mesh-controller.lease"]}`,
"modules/o/module.json": `{"module":"o"}`,
"modules/o/cmd/o/main.go": "package main\n\nvar _ = 1\n"}, ""},
} {
t.Run(row.name, func(t *testing.T) {
root := t.TempDir()
for path, text := range row.files {
p := filepath.Join(root, path)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte(text), 0o644); err != nil {
t.Fatal(err)
}
}
got, err := Findings(root)
if err != nil {
t.Fatal(err)
}
joined := strings.Join(got, "\n")
if row.want == "" && len(got) != 0 || row.want != "" && !strings.Contains(joined, row.want) {
t.Errorf("findings %q, want %q", joined, row.want)
}
})
}
}
// The catalogue itself: every state a module reads of another module that announces from code is announced readable.
func TestTheCatalogueReadsOnlyWhatItsOwnersAnnounceReadable(t *testing.T) {
got, err := Findings(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
for _, f := range got {
t.Error(f)
}
}
-6
View File
@@ -1,6 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (issue 1936).
# every manifest, for what it reads; and every module's Go and TypeScript code, for the states it announces.
modules/*/module.json
modules/*/cmd
modules/*/*.go
modules/*/*.ts
-4
View File
@@ -1,7 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/secrets
go 1.22
require git.novox.be/novox/mesh-catalog/checks/trunk v0.0.0
replace git.novox.be/novox/mesh-catalog/checks/trunk => ../trunk
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest and the files it names.
modules
-25
View File
@@ -1,25 +0,0 @@
package main
import (
"testing"
"git.novox.be/novox/mesh-catalog/checks/trunk"
)
// The script's run of this check over the real tree (novox/hq ADR 0529 clause 2): the known list gains no line the trunk's copy
// at the merge base does not hold. Without a trunk copy the lines in seed are the trunk's.
func TestTheKnownListHoldsNoLineTheTrunksDoesNot(t *testing.T) {
known, err := Known("known")
if err != nil {
t.Fatal(err)
}
base := seed
if path, ok := trunk.Copy(t, "checks/secrets/known"); ok {
if base, err = Known(path); err != nil {
t.Fatal(err)
}
}
for _, line := range Added(known, base) {
t.Errorf("checks/secrets/known gains %q, which the trunk's copy does not hold: the list only shrinks — protect the key instead (novox/hq issue 342)", line)
}
}
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest.
modules/*/module.json
-4
View File
@@ -1,7 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/templates
go 1.22
require git.novox.be/novox/mesh-catalog/checks/trunk v0.0.0
replace git.novox.be/novox/mesh-catalog/checks/trunk => ../trunk
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest and its templates.
modules
-32
View File
@@ -1,32 +0,0 @@
package main
import (
"testing"
"git.novox.be/novox/mesh-catalog/checks/trunk"
)
// The script's comparison of this list with the trunk's (novox/hq ADR 0529 clause 2): the list of modules keeping file text inline
// gains none the trunk's copy at the merge base does not hold. The catalogue itself is judged against the list by
// TestTheCatalogueKeepsFileTextInlineOnlyWhereListed. Skipped, and said, where no trunk is beside.
func TestTheInlineListHoldsNoModuleTheTrunksDoesNot(t *testing.T) {
path, ok := trunk.Copy(t, "checks/templates/inline")
found, err := Check("../..")
if err != nil {
t.Fatal(err)
}
listed, err := Listed(listFile)
if err != nil {
t.Fatal(err)
}
base := seed
if ok {
if base, err = Listed(path); err != nil {
t.Fatal(err)
}
}
_, _, added := Judge(found, listed, base)
for _, m := range added {
t.Errorf("checks/templates/inline gains %s, which the trunk's copy does not hold: the list only shrinks", m)
}
}
-5
View File
@@ -1,5 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest.
modules/*/module.json
-3
View File
@@ -1,3 +0,0 @@
module git.novox.be/novox/mesh-catalog/checks/trunk
go 1.22
-65
View File
@@ -1,65 +0,0 @@
// Package trunk is where a catalogue check finds the trunk's copy of a list that may only shrink (novox/hq ADR 0529 clause 2):
// the file as the catalogue's base commit holds it, to judge a change's list against.
//
// The checker fetches the change's own repository at its base commit beside the change, as it fetches a sibling, and says where
// in MESH_CHECK_BESIDE; the copy is under the repository's name (the same convention the other repositories' tests use). With
// MESH_CHECK_BESIDE unset, or set without the catalogue's copy in it, the test fails saying NOT COMPARED WITH THE TRUNK and
// why: the checker places the change's own repository since R4g (issue 1295), so a missing copy is a fault, never a pass. A trunk that holds no such file had
// none: the list's first copy, which nothing was added to.
package trunk
import (
"errors"
"fmt"
"os"
"path/filepath"
"testing"
)
// Repository is the catalogue's name beside a check.
const Repository = "mesh-catalog"
// BesideEnv is where a merge check says the clones beside the one checked are.
const BesideEnv = "MESH_CHECK_BESIDE"
// Copy is the trunk's copy of the file at rel (a path from the catalogue's root), written to a file of the test's own: its path,
// and whether the trunk holds one (false: its first copy). It fails the test when no copy of the catalogue is beside.
func Copy(t testing.TB, rel string) (string, bool) {
t.Helper()
raw, found, err := Read(rel)
switch {
case err != nil:
t.Fatal(err)
case !found:
t.Logf("the trunk holds no %s: its first copy", rel)
return "", false
}
path := filepath.Join(t.TempDir(), filepath.Base(rel))
if err := os.WriteFile(path, raw, 0o644); err != nil {
t.Fatal(err)
}
return path, true
}
// Read is the trunk's copy of rel and whether the trunk holds one. err is a trunk not beside the check (MESH_CHECK_BESIDE unset,
// or without the catalogue's directory), or a copy unreadable: a comparison that cannot be made is never a pass.
func Read(rel string) (content []byte, found bool, err error) {
beside := os.Getenv(BesideEnv)
if beside == "" {
return nil, false, fmt.Errorf("NOT COMPARED WITH THE TRUNK: %s is not set, so the trunk's copy of %s is not beside this run; the checker places the change's own repository there (issue 1295)", BesideEnv, rel)
}
dir := filepath.Join(beside, Repository)
if _, err := os.Stat(dir); errors.Is(err, os.ErrNotExist) {
return nil, false, fmt.Errorf("NOT COMPARED WITH THE TRUNK: %s is not beside this check in %s=%s, so %s is not judged against the trunk's copy; the checker places the change's own repository there at its trunk (issue 1295)", Repository, BesideEnv, beside, rel)
} else if err != nil {
return nil, false, err
}
raw, err := os.ReadFile(filepath.Join(dir, rel))
if errors.Is(err, os.ErrNotExist) {
return nil, false, nil
}
if err != nil {
return nil, false, err
}
return raw, true, nil
}
-43
View File
@@ -1,43 +0,0 @@
package trunk
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestTheCloneBesideTheCheckIsTheTrunk(t *testing.T) {
beside := t.TempDir()
dir := filepath.Join(beside, Repository, "checks", "x")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "known"), []byte("a\nb\n"), 0o644); err != nil {
t.Fatal(err)
}
t.Setenv(BesideEnv, beside)
raw, found, err := Read("checks/x/known")
if err != nil || !found || string(raw) != "a\nb\n" {
t.Fatalf("read %q found %v err %v", raw, found, err)
}
if _, found, err := Read("checks/x/absent"); err != nil || found {
t.Fatalf("a file the trunk does not hold: found %v err %v; wanted its first copy", found, err)
}
}
func TestWithNothingBesideTheComparisonFailsAndSaysNotMade(t *testing.T) {
t.Setenv(BesideEnv, "")
_, found, err := Read("checks/x/known")
if found || err == nil || !strings.Contains(err.Error(), "NOT COMPARED") {
t.Fatalf("found %v, err %v; wanted a failure saying NOT COMPARED", found, err)
}
}
func TestACloneBesideWithoutTheCatalogueFailsAndSaysNotMade(t *testing.T) {
t.Setenv(BesideEnv, t.TempDir()) // the siblings are there, the catalogue's own base is not
_, found, err := Read("checks/x/known")
if found || err == nil || !strings.Contains(err.Error(), "NOT COMPARED") || !strings.Contains(err.Error(), "not beside") {
t.Fatalf("found %v, err %v; wanted a failure naming the missing copy", found, err)
}
}
-3
View File
@@ -1,3 +0,0 @@
# The modules of the catalogue whose texts still name the build agent or the toolchain images, and why (checks/words,
# buildagent_test.go). One module directory a line, with its reason after a #. It only shrinks.
modules/mesh-build-go # build.go:45 says "the build agent's recipe" in a step's description: the asks lane's module (issue 1887), not E6's to edit
-57
View File
@@ -1,57 +0,0 @@
package main
import (
"os"
"strings"
"testing"
)
// The build agent and the toolchain images are gone (novox/hq ADR 0524 §5, issue 1295 and E6 of milestone R4g): no text a module
// can show names them. Where the word stands for a reason that is another lane's or still waits for its change, buildagent-allowed
// names the module's directory with that reason and the date it is dated from; the list only shrinks: a directory on it that no
// longer names the word is refused, so the line is taken out.
var gone = []string{"build-agent", "build agent", "toolchain image"}
func allowedModules(t *testing.T) map[string]string {
t.Helper()
raw, err := os.ReadFile("buildagent-allowed")
if err != nil {
t.Fatal(err)
}
out := map[string]string{}
for _, line := range strings.Split(string(raw), "\n") {
entry, why, _ := strings.Cut(line, "#")
if entry = strings.TrimSpace(entry); entry != "" && !strings.HasPrefix(line, "#") {
if strings.TrimSpace(why) == "" {
t.Fatalf("%q is allowed without a reason", entry)
}
out[entry] = strings.TrimSpace(why)
}
}
return out
}
func TestNoTextOfTheCatalogueNamesTheBuildAgentOutsideTheModulesAllowed(t *testing.T) {
texts, _, err := collect("../..")
if err != nil {
t.Fatal(err)
}
allowed := allowedModules(t)
seen := map[string]bool{}
for _, f := range check(gone, texts) {
module := f
if rest, ok := strings.CutPrefix(f, "modules/"); ok {
module = "modules/" + strings.SplitN(rest, "/", 2)[0]
}
if _, ok := allowed[module]; ok {
seen[module] = true
continue
}
t.Errorf("%s (the build agent and the toolchain images are gone: say what stands in their place)", f)
}
for module, why := range allowed {
if !seen[module] {
t.Errorf("buildagent-allowed names %s (%s), which names the word no more: take the line out, so it cannot come back", module, why)
}
}
}
-3
View File
@@ -58,9 +58,6 @@ func checkLines(lines []retiredLine, texts []text) []string {
for _, l := range lines {
rx := linePattern(l)
for _, t := range texts {
if !strings.Contains(t.s, l.daemon) { // the pattern holds the daemon's name whole: the cheap test first
continue
}
if m := rx.FindStringSubmatchIndex(t.s); m != nil {
line := t.line
if line > 0 {
+36 -71
View File
@@ -43,63 +43,18 @@ func main() {
if len(os.Args) > 1 {
root = os.Args[1]
}
os.Exit(run(root))
}
// run reads every text a module can show under root and answers the command's exit code: 0 clean, 1 a finding, 2 a list or a
// file unreadable. The command prints what it found; a test of the real catalogue reads the code.
func run(root string) int {
words, err := readList(filepath.Join(root, "retired-words"))
if err != nil {
fmt.Fprintln(os.Stderr, "words:", err)
return 2
os.Exit(2)
}
if len(words) == 0 {
fmt.Fprintln(os.Stderr, "words: retired-words lists no word — the check would pass on anything")
return 2
os.Exit(2)
}
texts, files, err := collect(root)
if err != nil {
fmt.Fprintln(os.Stderr, "words:", err)
return 2
}
findings := check(words, texts)
for _, f := range findings {
fmt.Println(f)
}
if len(findings) > 0 {
fmt.Printf("words: %d use(s) of a word the glossary retired (novox/hq ADR 0244) — say it in the glossary's word\n", len(findings))
return 1
}
// And no daemon's line (novox/hq ADR 0272): what a module tells the operator to type is a `nox` line.
lines, err := readLines(filepath.Join(root, "retired-lines"))
if err != nil {
fmt.Fprintln(os.Stderr, "words:", err)
return 2
}
if len(lines) == 0 {
fmt.Fprintln(os.Stderr, "words: retired-lines lists no line — the check would pass on anything")
return 2
}
if findings := checkLines(lines, texts); len(findings) > 0 {
for _, f := range findings {
fmt.Println(f)
}
fmt.Printf("words: %d daemon's line(s) in what a module shows — the operator types one command, nox (novox/hq ADR 0272)\n",
len(findings))
return 1
}
fmt.Printf("words: %d retired words and %d daemons' lines, none in the %d strings of %d files\n", len(words), len(lines),
len(texts), files)
return 0
}
// collect reads every text a module can show under root: the strings of its own code that is not a test, and every string of
// its manifests.
func collect(root string) ([]text, int, error) {
var texts []text
files := 0
err := filepath.WalkDir(filepath.Join(root, "modules"), func(path string, d fs.DirEntry, err error) error {
err = filepath.WalkDir(filepath.Join(root, "modules"), func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
@@ -132,7 +87,38 @@ func collect(root string) ([]text, int, error) {
texts = append(texts, found...)
return nil
})
return texts, files, err
if err != nil {
fmt.Fprintln(os.Stderr, "words:", err)
os.Exit(2)
}
findings := check(words, texts)
for _, f := range findings {
fmt.Println(f)
}
if len(findings) > 0 {
fmt.Printf("words: %d use(s) of a word the glossary retired (novox/hq ADR 0244) — say it in the glossary's word\n", len(findings))
os.Exit(1)
}
// And no daemon's line (novox/hq ADR 0272): what a module tells the operator to type is a `nox` line.
lines, err := readLines(filepath.Join(root, "retired-lines"))
if err != nil {
fmt.Fprintln(os.Stderr, "words:", err)
os.Exit(2)
}
if len(lines) == 0 {
fmt.Fprintln(os.Stderr, "words: retired-lines lists no line — the check would pass on anything")
os.Exit(2)
}
if findings := checkLines(lines, texts); len(findings) > 0 {
for _, f := range findings {
fmt.Println(f)
}
fmt.Printf("words: %d daemon's line(s) in what a module shows — the operator types one command, nox (novox/hq ADR 0272)\n",
len(findings))
os.Exit(1)
}
fmt.Printf("words: %d retired words and %d daemons' lines, none in the %d strings of %d files\n", len(words), len(lines),
len(texts), files)
}
// readList reads retired-words: one word or phrase a line; blank lines and lines starting with # are not words.
@@ -166,22 +152,9 @@ func pattern(word string) *regexp.Regexp {
func check(words []string, texts []text) []string {
var out []string
// A text can hold a word only if it holds the word's first part, in any case: the regular expression is run on those alone,
// which keeps the check of the whole catalogue within the unit level's bound.
lower := make([]string, len(texts))
for i, t := range texts {
lower[i] = strings.ToLower(t.s)
}
for _, w := range words {
rx := pattern(w)
first := ""
if parts := strings.Fields(w); len(parts) > 0 {
first = strings.ToLower(parts[0])
}
for i, t := range texts {
if !strings.Contains(lower[i], first) {
continue
}
for _, t := range texts {
if m := rx.FindStringSubmatchIndex(t.s); m != nil {
line := t.line
if line > 0 {
@@ -206,9 +179,6 @@ func excerpt(s string, from, to int) string {
return strings.Join(strings.Fields(s[a:b]), " ")
}
// modulePath matches a string that is only a module or import path ("git.example.org/org/repo/go/pkg").
var modulePath = regexp.MustCompile(`^[a-z0-9.-]+\.[a-z]+/[A-Za-z0-9_./-]+$`)
// goStrings returns a Go file's string literals, adjacent ones joined across `+` as the program joins them.
func goStrings(path, rel string) ([]text, error) {
src, err := os.ReadFile(path)
@@ -234,11 +204,6 @@ func goStrings(path, rel string) ([]text, error) {
if err != nil {
v = lit
}
if modulePath.MatchString(v) {
// An import path or another module path names code, not words an agent reads.
cur, joining = nil, false
continue
}
if cur != nil && joining {
cur.s += v
} else {
-8
View File
@@ -87,11 +87,3 @@ func TestRetiredLinesAreReadAsDaemonAndCommand(t *testing.T) {
t.Fatal("a line with no command was read")
}
}
// The script's run of this check over the real tree (novox/hq ADR 0529 clause 2): no word the glossary retired, and no
// daemon's line, in any text a module's tools can show an agent.
func TestTheCatalogueHoldsNoRetiredWordAndNoDaemonsLine(t *testing.T) {
if code := run("../.."); code != 0 {
t.Fatalf("the check of the real catalogue exits %d (see what it printed above)", code)
}
}
-7
View File
@@ -1,7 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# run("../.."): the root's lists of retired words and lines, and every module's texts.
retired-words
retired-lines
modules
+1 -1
View File
@@ -1,5 +1,5 @@
58
The long-running resources of this catalogue that do not say how they are ready (`health`, novox/hq ADR 0240
rule 8), as `module check` counts them. It may only go down: checks/health fails a change that raises it, and
rule 8), as `module check` counts them. It may only go down: merge-check.sh fails a change that raises it, and
one that lowers it writes the new number on the first line. From 2026-11-18, or once it is 0, a long-running
resource without `health` is refused.
+140
View File
@@ -0,0 +1,140 @@
#!/bin/sh
# The catalogue's TypeScript part of its own check (novox/hq issue 302's `mesh-check-also`), declared in
# merge-check.sh's header and run by the build seat in the TypeScript toolchain, after the Go part. The media
# catalogue's twin is issue 311. Before this, a TypeScript module's change ran nothing of its own before it
# merged: the Go part said "NOT TESTED HERE", and a type error in the forge module would first have shown in
# its walk.
#
# For every touched module that has a package.json, in a copy of it outside the clone:
#
# 1. its dependencies are the toolchain's own (/app/node_modules: the mesh's SDK, Node's types and what the
# runtime carries), copied beside it, never fetched: the check holds no credential for the package
# registry. A module that depends on a package the toolchain does not hold **fails the check**, by name:
# it could not be type-checked or tested, and what could not run is a failure, never a note (novox/hq
# issue 385);
# 2. every source that is not a test is type-checked with the toolchain's compiler and the flags a build
# compiles with — the same check the build makes, before the merge rather than in the walk;
# 3. its tests (test/*.test.ts) run under Node's own runner with types stripped, as its own `npm test` does
# after its build: the sources are compiled to dist/ and beside themselves first, so a test that imports
# the compiled module, or a source that imports its sibling by its .js name, finds it.
#
# Said, never passed silently: a module with no tests is said NOT TESTED on every run that touches it —
# nothing of it could fail. A module whose tests exist and could not run fails (novox/hq issue 385).
#
# By hand: MESH_CHECK_MODULES=gitea sh merge-check-typescript.sh, in the toolchain image (it reads
# /app/node_modules), or with TOOLCHAIN_MODULES pointing at a node_modules that holds the same.
#
# **In an agent's session, outside the toolchain image** (novox/hq issue 742), the same node_modules is
# assembled first, in a throwaway directory under the check's own workspace, the way the image's `deps` stage
# assembles it: the toolchain's own package.json and lock (mesh-tools' node-tools/, read from the forge this
# clone came from, so the list is never copied here) installed with `npm ci`, then the SDK the mesh last
# published (`@novox/mesh-sdk@latest`), the @novox scope from the forge's package registry, read without a
# credential. Node and npm are the dev-tools module's; nothing is installed on the machine, no npm
# configuration or cache of the account's is read or written. Only when neither TOOLCHAIN_MODULES nor
# /app/node_modules is there: the forge's check, in the image, fetches nothing, as before. The forge is
# MESH_FORGE (default: the origin remote's), the owner MESH_FORGE_OWNER (default: the origin's owner).
set -eu
work=$(mktemp -d)
trap 'rm -rf "$work"' EXIT
deps="${TOOLCHAIN_MODULES:-/app/node_modules}"
if [ -z "${TOOLCHAIN_MODULES:-}" ] && [ ! -f "$deps/typescript/bin/tsc" ]; then
origin=$(git remote get-url origin 2>/dev/null || true)
forge="${MESH_FORGE:-$(printf '%s\n' "$origin" | sed -n 's#^\(https\{0,1\}://[^/]*\)/.*#\1#p')}"
owner="${MESH_FORGE_OWNER:-$(printf '%s\n' "$origin" | sed -n 's#^https\{0,1\}://[^/]*/\([^/]*\)/.*#\1#p')}"
if [ -z "$forge" ] || [ -z "$owner" ]; then
echo "NOT CHECKED: no TypeScript toolchain at $deps, and no forge to assemble one from (origin is '$origin'; set MESH_FORGE and MESH_FORGE_OWNER, or TOOLCHAIN_MODULES)"
exit 1
fi
echo "no TypeScript toolchain at $deps: assembling one from $forge/$owner/mesh-tools node-tools/ and $owner's package registry (novox/hq issue 742)"
tc="$work/toolchain"
mkdir -p "$tc"
printf '@%s:registry=%s/api/packages/%s/npm/\n' "$owner" "$forge" "$owner" >"$work/npmrc"
# Read with Node's own fetch (dev-tools declares Node, not curl); a status that is not 2xx fails.
if ! { node -e '
const [base, dir] = process.argv.slice(1);
Promise.all(["package.json", "package-lock.json"].map(async (f) => {
const r = await fetch(base + f);
if (!r.ok) throw new Error(base + f + ": " + r.status);
require("fs").writeFileSync(dir + "/" + f, await r.text());
})).catch((e) => { console.error(e.message); process.exit(1); });' \
"$forge/$owner/mesh-tools/raw/branch/main/node-tools/" "$tc" &&
(cd "$tc" && export npm_config_userconfig="$work/npmrc" npm_config_cache="$work/npm-cache" &&
npm ci --ignore-scripts --no-audit --no-fund --loglevel=error &&
npm install --ignore-scripts --no-audit --no-fund --no-save --loglevel=error "@$owner/mesh-sdk@latest"); }; then
echo "NOT CHECKED: the TypeScript toolchain could not be assembled from $forge (novox/hq issue 742)"
exit 1
fi
deps="$tc/node_modules"
fi
tsc="$deps/typescript/bin/tsc"
if [ ! -f "$tsc" ]; then
echo "NOT CHECKED: no TypeScript compiler at $tsc — this part runs in the TypeScript toolchain"
exit 1
fi
# The modules the change reaches: the planner's answer (hq ADR 0238), and every module a changed file is in —
# a change to a module's tests alone moves no build (hq ADR 0267), and its tests must still run.
touched=$({
printf '%s\n' "${MESH_CHECK_MODULES:-}" | tr ',' '\n' | sed -e 's#^modules/##' -e '/\//d'
printf '%s\n' "${MESH_CHECK_CHANGED:-}" | tr ',' '\n' | sed -n 's#^modules/\([^/]*\)/.*#\1#p'
} | sed '/^$/d' | sort -u)
failed=""
for m in $touched; do
d="modules/$m"
[ -f "$d/package.json" ] || continue
copy="$work/$m"
mkdir -p "$copy"
(cd "$d" && tar --exclude=./node_modules --exclude=./dist -cf - .) | (cd "$copy" && tar -xf -)
# What it depends on, against what the toolchain holds.
missing=$(node -e '
const p = require(process.argv[1]); const have = require("fs").readdirSync(process.argv[2]);
const scoped = (n) => n.startsWith("@") ? n.split("/")[0] : n;
const names = Object.keys({ ...(p.dependencies || {}), ...(p.devDependencies || {}) });
const held = (n) => have.includes(n) || (n.startsWith("@") && have.includes(scoped(n)) &&
require("fs").existsSync(process.argv[2] + "/" + n));
console.log(names.filter((n) => n !== "typescript" && !held(n)).join(" "));' "$copy/package.json" "$deps")
if [ -n "$missing" ]; then
echo "FAILED: $d could not be checked — it depends on $missing, which the toolchain does not hold and the check cannot fetch (novox/hq issue 385)"
failed="$failed $m"
continue
fi
mkdir -p "$copy/node_modules"
for p in "$deps"/*; do
case "$(basename "$p")" in typescript | esbuild | @esbuild) continue ;; esac
cp -r "$p" "$copy/node_modules/"
done
sources=$(cd "$copy" && find . -name '*.ts' ! -name '*.d.ts' ! -path './test/*' ! -path './node_modules/*' ! -path './dist/*' | sort)
if [ -n "$sources" ]; then
echo "type-checking $d (TypeScript)"
# shellcheck disable=SC2086 # the module's sources, none with a space in its name
if ! (cd "$copy" && node "$tsc" --noEmit --module NodeNext --moduleResolution NodeNext --target ES2022 \
--types node $sources); then
failed="$failed $m"
continue
fi
else
echo "$d has no TypeScript sources of its own to type-check: its tests read its files"
fi
if ! ls "$copy"/test/*.test.ts >/dev/null 2>&1; then
echo "NOT TESTED: $d is TypeScript and has no tests of its own"
continue
fi
if [ -n "$sources" ]; then
# shellcheck disable=SC2086
(cd "$copy" && node "$tsc" --module NodeNext --moduleResolution NodeNext --target ES2022 --types node \
--rootDir . --outDir dist $sources && node "$tsc" --module NodeNext --moduleResolution NodeNext \
--target ES2022 --types node --rootDir . --outDir . $sources)
fi
echo "testing $d (TypeScript)"
if ! (cd "$copy" && node --test --test-reporter=spec --experimental-strip-types \
--no-warnings=ExperimentalWarning test/*.test.ts); then
failed="$failed $m"
fi
done
if [ -n "$failed" ]; then
echo "TypeScript modules that failed:$failed"
exit 1
fi
Executable
+233
View File
@@ -0,0 +1,233 @@
#!/bin/sh
# mesh-check-toolchain: go
# mesh-check-also: typescript merge-check-typescript.sh
#
# The catalogue's own check (novox/hq ADR 0237 as amended): the second layer of a pull request's merge
# check, `mesh/repo-check`, run by the build seat in the mesh's Go toolchain.
#
# The gate — the manifests the change touches through the running controller's module check, every machine
# of the facts snapshot composed with them and validated by the node-engine's own validator, the replays
# against this tree — is the build seat's first layer (`mesh/merge-gate`), run because the mesh's module
# graph builds these modules from here. It is not repeated here. This is the repository's own:
#
# 1. every manifest through the controller's module check, so one module's change cannot leave another
# it shares a rule with refused (MESH_GATE is the controller the mesh runs) — and the count of
# long-running resources without `health` held to the number in health-undeclared, which only goes down;
# 2. the Go tests of every module the change touches that has them, under the race detector when the
# toolchain has a C compiler. A requirement of the mesh's own forge the module cache does not hold (a
# pinned SDK commit) is taken from its repository's clone beside the check (checks/deps); a module whose
# dependencies still cannot be had **fails the check**, by name and why — its tests could not run, and a
# test that could not run is a failure, never a note (novox/hq issue 385). The integration tests (`//go:build
# integration`) run here, with -tags integration, and only here. A module written in
# TypeScript is type-checked and tested by
# merge-check-typescript.sh, declared above, in the TypeScript toolchain (novox/hq issues 302, 311);
# 3. the words (novox/hq ADR 0244): no word the glossary retired for the tools, as copied in
# retired-words, in any text a module's tools can show an agent — every string in its own code that is
# not a test, and every description in its manifest. Run over every module, not only the touched ones,
# so a word newly retired is found everywhere it stands. The check's own test runs first.
# 4. the paths (novox/hq ADR 0260): no module names a path under /run, /var/lib, /usr/local/lib or /etc
# that another module's manifest places — modules talk over the bus, not through each other's files.
# Over every module; checks/paths/shared and checks/paths/known narrow it, and known only shrinks.
# 5. the agent's home (novox/hq ADR 0266): no module unpacks an archive, or writes into a file, under
# ${machine:agent-home} — the node-engine does both after checking the path, and the agent account owns
# that home and could swap a link in between.
# 6. the secrets (novox/hq issue 342): no mergeable file fills a key with a vault secret that a setting could
# replace — the key is `protected`, or left out of the file's `takes-settings`. checks/secrets/known names
# the nested ones the controller cannot protect alone yet, and only shrinks.
# 7. the list of predefined keys (novox/hq ADR 0348 clause 7): a change to predefined.json removes no key a
# module still names, judged against the trunk's copy at the merge base. Whether a removed key still holds a
# value at the mesh or a node is mesh-config's to refuse; this check reads no store. The list itself and
# every manifest against it are judged by the controller's module check in step 1.
# 8. the config form (novox/hq ADR 0348 clauses 1 and 11): no module says `settings`, `takes-settings` or
# `${setting:<key>}`, in its manifest or a template it names, or has a mergeable file naming no keys (which
# takes its whole layer), unless checks/config/unmoved lists it as not yet moved; that list only shrinks,
# judged against the trunk's copy at the merge base.
# 9. the file text (novox/hq ADR 0349 §2): no module keeps a file's text inline in its manifest (`content`)
# unless checks/templates/inline lists it as not yet moved; each names its template in files/. That list
# only shrinks, judged against the trunk's copy at the merge base.
# 10. the mail-store seat (novox/hq ADR 0378 §3, issue 800): no manifest but modules/mesh-mail claims it.
# 11. owned paths (novox/hq ADR 0422 §8, issue 1024; it holds ~/.ssh for ssh-client as ADR 0415 §6 asked):
# every directory and archive says `unknown` (move-out, module or program) and a valid `knows`; `move-out`
# only under a home; no module declares a path inside a directory of another module; and none is, lies in or
# holds a directory a system package owns under /var/lib (checks/owned/system, issue 1841).
# checks/sshdir (ADR 0415 §6) stays while main runs it; it goes under issue 1035 once main runs checks/owned.
# 12. the image builder (novox/hq ADR 0487 §3, issue 1704): only modules/buildah provides rootless-image-build.
# 13. the image's place (novox/hq ADR 0487 §1, issue 1707): every image builds from a Dockerfile in its own module's directory.
set -eu
(cd checks/words && go test -count=1 . && go run . ../..)
# Node names (novox/hq ADR 0507 §9, issue 1838): no node name of this mesh in a covered module; the names are in
# `particulars`, the covered modules in checks/particulars/covered.
(cd checks/particulars && go test -count=1 . && go run . ../..)
# The toolchains (novox/hq ADR 0477 clause 4): the package go only in modules/go, nodejs and npm only in modules/nodejs.
(cd checks/toolchains && go test -count=1 . && go run . ../..)
# The image builder (novox/hq ADR 0487 §3): only modules/buildah provides rootless-image-build, docker never.
(cd checks/imagebuild && go test -count=1 . && go run . ../..)
# The image's place (novox/hq ADR 0487 §1): every image builds from a Dockerfile in its own module's directory.
(cd checks/imageplace && go test -count=1 . && go run . ../..)
# The Node lock files (novox/hq ADR 0469 §5, issue 1868): a TypeScript module that names packages beyond the SDK commits its lock.
(cd checks/locks && go test -count=1 . && go run . ../..)
# The agent's home (novox/hq ADR 0266): no archive, and no file written into, under it.
(cd checks/agenthome && go test -count=1 . && go run . ../..)
# The mail-store seat (novox/hq ADR 0378 §3): only mesh-mail may claim it.
(cd checks/mailstore && go test -count=1 . && go run . ../..)
# ~/.ssh (novox/hq ADR 0415 §6): only ssh-client may name a path under it. Main's merge check runs this line, so
# checks/sshdir stays until main runs checks/owned; it goes under issue 1035.
(cd checks/sshdir && go test -count=1 . && go run . ../..)
# Owned paths (novox/hq ADR 0422 §8): unknown files named per directory; no path inside another module's directory;
# none in a system package's directory under /var/lib (issue 1841), the known exceptions only shrinking: judged
# against the trunk's copy of checks/owned/system-known at the merge base; a trunk without one is the list's first copy.
owned_base=""
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
if git cat-file -e "$merge_base:checks/owned/system-known" 2>/dev/null; then
owned_base=$(mktemp)
git show "$merge_base:checks/owned/system-known" >"$owned_base"
fi
else
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; checks/owned/system-known is held to the list it was first written with"
fi
(cd checks/owned && go test -count=1 . && if [ -n "$owned_base" ]; then go run . -base "$owned_base" ../..; else go run . ../..; fi)
# The secrets (novox/hq issue 342): a password is not a setting, so no setting can replace the vault's. The
# known list only shrinks: judged against the trunk's copy at the merge base, a line it does not hold is refused.
secrets_base=""
secrets_ref="origin/${MESH_CHECK_BASE:-main}"
if merge_base=$(git merge-base HEAD "$secrets_ref" 2>/dev/null); then
if git cat-file -e "$merge_base:checks/secrets/known" 2>/dev/null; then
secrets_base=$(mktemp)
git show "$merge_base:checks/secrets/known" >"$secrets_base"
fi
else
echo "NOT COMPARED WITH THE TRUNK: no merge base with $secrets_ref; checks/secrets/known is held to the list it was first written with"
fi
(cd checks/secrets && go test -count=1 . && if [ -n "$secrets_base" ]; then go run . -base "$secrets_base" ../..; else go run . ../..; fi)
# The list of predefined keys (novox/hq ADR 0348 clause 7): no key removed that a module still names. The trunk's
# list at the merge base is what it is judged against; a trunk without the file had none.
predefined_base=""
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
predefined_base=$(mktemp)
if ! git show "$merge_base:predefined.json" >"$predefined_base" 2>/dev/null; then
echo '{}' >"$predefined_base"
fi
else
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; no removal from predefined.json can be seen"
fi
(cd checks/predefined && go test -count=1 . && if [ -n "$predefined_base" ]; then go run . -base "$predefined_base" ../..; else go run . ../..; fi)
# The config form (novox/hq ADR 0348 §1, §11): a moved module says no old name, and the list of modules not yet moved
# only shrinks, judged against the trunk's copy at the merge base; a trunk without one is the list's first copy.
unmoved_base=""
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
if git cat-file -e "$merge_base:checks/config/unmoved" 2>/dev/null; then
unmoved_base=$(mktemp)
git show "$merge_base:checks/config/unmoved" >"$unmoved_base"
fi
else
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; checks/config/unmoved is held to the list it was first written with"
fi
(cd checks/config && go test -count=1 . && if [ -n "$unmoved_base" ]; then go run . -base "$unmoved_base" ../..; else go run . ../..; fi)
# The file text (novox/hq ADR 0349 §2): a file's text is a template in files/, outside the modules not yet moved; the
# list of those only shrinks, judged against the trunk's copy at the merge base, a trunk without one being its first.
inline_base=""
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
if git cat-file -e "$merge_base:checks/templates/inline" 2>/dev/null; then
inline_base=$(mktemp)
git show "$merge_base:checks/templates/inline" >"$inline_base"
fi
else
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; checks/templates/inline is held to the list it was first written with"
fi
(cd checks/templates && go test -count=1 . && if [ -n "$inline_base" ]; then go run . -base "$inline_base" ../..; else go run . ../..; fi)
# The lists that narrow the paths check may not grow unseen: the base branch's copies are what they are
# judged against (MESH_CHECK_BASE, main when unset), read from the clone the check runs in.
base_lists=""
base_ref="origin/${MESH_CHECK_BASE:-main}"
if git rev-parse --verify --quiet "$base_ref^{commit}" >/dev/null 2>&1 &&
git cat-file -e "$base_ref:checks/paths/known" 2>/dev/null && git cat-file -e "$base_ref:checks/paths/shared" 2>/dev/null; then
base_lists=$(mktemp -d)
git show "$base_ref:checks/paths/known" >"$base_lists/known"
git show "$base_ref:checks/paths/shared" >"$base_lists/shared"
fi
(cd checks/paths && go test -count=1 . && if [ -n "$base_lists" ]; then go run . -base "$base_lists" ../..; else go run . ../..; fi)
if [ -n "${MESH_GATE:-}" ]; then
checked=$("$MESH_GATE" module check modules/*/module.json) || { printf '%s\n' "$checked"; exit 1; }
# **The count only goes down** (novox/hq ADR 0240 rule 8): the long-running resources that do not say how
# they are ready, as the controller counts them, against the number kept in health-undeclared. A change
# that raises it fails; one that lowers it writes the new number there, so it can never rise again.
kept=$(sed -n 's/^\([0-9][0-9]*\).*/\1/p' health-undeclared | head -n 1)
counted=$(printf '%s\n' "$checked" | sed -n 's/^long-running resources without health: \([0-9][0-9]*\)$/\1/p')
if [ -z "$counted" ]; then
echo "NOT COUNTED: the controller the mesh runs is older than the count of resources without health (ADR 0240 Phase B)"
elif [ "$counted" -gt "$kept" ]; then
echo "the long-running resources without health rose from $kept to $counted: declare how each new one is ready (ADR 0240 rule 8)"
exit 1
elif [ "$counted" -lt "$kept" ]; then
echo "the long-running resources without health fell from $kept to $counted: write $counted in health-undeclared, so the count cannot rise again"
exit 1
else
echo "long-running resources without health: $counted, as kept"
fi
else
echo "NOT CHECKED: no controller was built beside this check, so the manifests were not read by one"
fi
race=""
if command -v gcc >/dev/null 2>&1; then race="-race"; else echo "NOT RACE-CHECKED: the toolchain holds no C compiler"; fi
# The modules the change reaches are the controller's planner's answer (MESH_CHECK_MODULES, hq ADR 0238):
# a module by its name, a new one by its directory. By hand, without it, the directories the changed files
# are in.
if [ -n "${MESH_CHECK_MODULES:-}" ]; then
touched=$(printf '%s\n' "$MESH_CHECK_MODULES" | tr ',' '\n' | sed -e 's#^modules/##' -e '/\//d' | sort -u)
else
touched=$(printf '%s\n' "${MESH_CHECK_CHANGED:-}" | tr ',' '\n' | sed -n 's#^modules/\([^/]*\)/.*#\1#p' | sort -u)
fi
# A module's dependencies: fetched, or a forge requirement taken from its clone beside the check
# (MESH_CHECK_BESIDE); its tests run against them (checks/deps, novox/hq issue 385). The helper's own test
# runs first.
(cd checks/deps && go test -count=1 .)
deps=$(mktemp -d)
trap 'rm -rf "$deps"' EXIT
(cd checks/deps && go build -o "$deps/deps" .)
untestable=""
for m in $touched; do
[ -d "modules/$m" ] || continue
if [ ! -f "modules/$m/go.mod" ]; then
[ -f "modules/$m/package.json" ] && echo "modules/$m is TypeScript: merge-check-typescript.sh type-checks and tests it"
continue
fi
if ! modfile=$(GOPRIVATE=git.novox.be "$deps/deps" -beside "${MESH_CHECK_BESIDE:-}" -work "$deps" "modules/$m"); then
echo "FAILED: modules/$m could not be tested — $modfile"
untestable="$untestable modules/$m"
continue
fi
[ -n "$modfile" ] && echo "modules/$m takes its forge dependencies from the clones beside the check"
echo "testing modules/$m"
unformatted=$(cd "modules/$m" && gofmt -l .)
if [ -n "$unformatted" ]; then
echo "modules/$m is not gofmt'd: $unformatted"
exit 1
fi
cgo=0
[ -n "$race" ] && cgo=1
# The integration tests (a file whose first line is `//go:build integration`: a test that starts a bus, a store or
# another process) run here and nowhere else: the unit suite stays under a second (novox/hq playbook 07).
(cd "modules/$m" && CGO_ENABLED=$cgo GOPRIVATE=git.novox.be go vet -tags integration ${modfile:+"-modfile=$modfile"} ./... &&
CGO_ENABLED=$cgo GOPRIVATE=git.novox.be go test -tags integration ${modfile:+"-modfile=$modfile"} $race -count=1 ./...)
# The docker tests of the bus's snapshot program and entrypoint (novox/hq ADR 0480 clause 6) start throwaway nats
# containers, and the Go toolchain container holds no docker, so above they skip with a "NOT RUN" line. Where the
# build seat says docker exists (MESH_CHECK_DOCKER=1) they run here by name, and a missing docker fails the check
# (MESH_REQUIRE_DOCKER=1 makes the tests fail instead of skip). Without it the step says it did not run.
if [ "$m" = nats ]; then
if [ "${MESH_CHECK_DOCKER:-}" = 1 ]; then
(cd modules/nats && MESH_REQUIRE_DOCKER=1 CGO_ENABLED=$cgo GOPRIVATE=git.novox.be go test -tags integration ${modfile:+"-modfile=$modfile"} -count=1 \
-run 'TestTheDeclaredDumpRunsInTheBussImage|TestTheEntrypointReloadsTheServerWithoutDroppingAConnection|TestTheStoreSurvivesTheNewDeclarationTheOldAndTheNewAgain|TestASnapshotOfALiveBusRestoresToTheSameContent|TestTheSnapshotUserCannotChangeTheBus' \
./cmd/mesh-nats-snapshot) || { echo "the bus's docker tests failed"; exit 1; }
else
echo "NOT RUN: the bus's five docker tests (modules/nats/cmd/mesh-nats-snapshot) skipped: this check has no docker (MESH_CHECK_DOCKER is not 1)"
fi
fi
done
if [ -n "$untestable" ]; then
echo "the check fails: the tests of$untestable could not run (novox/hq issue 385)"
exit 1
fi
@@ -1,9 +0,0 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# the copies of this package in every module that carries one.
modules/xorg/internal/desktop/*.go
modules/lemurs/internal/desktop/*.go
modules/i3/internal/desktop/*.go
modules/xterm/internal/desktop/*.go
modules/adwaita/internal/desktop/*.go
@@ -23,7 +23,7 @@ import (
// manifest's.
const NamesFileEnv = "MESH_ARTIFACT_STORE_NAMES_FILE"
// CatalogueModules and CatalogueModule are the catalog's tools these definitions are read through.
// CatalogueModules and CatalogueModule are the catalogue's tools these definitions are read through.
const (
CatalogueModules = "mesh-catalog.catalog_modules"
CatalogueModule = "mesh-catalog.catalog_module"
@@ -143,21 +143,21 @@ func PinsIn(manifest json.RawMessage, names StoreNames) []pinned {
return out
}
// Definitions is what the catalog's definitions pin in the store, and what of them could not be read.
// Definitions is what the catalogue's definitions pin in the store, and what of them could not be read.
type Definitions struct {
// Pins are by Key(repository, digest).
Pins map[string][]Pin
// Read is how many definitions were read; Unread says, per module, why its definition was not.
Read int
Unread map[string]string
// NotAsked says why the catalog could not be asked at all, when it could not.
// NotAsked says why the catalogue could not be asked at all, when it could not.
NotAsked string
// NamesNotRead says why the store's older names could not be read, when they could not.
NamesNotRead string
Names StoreNames
}
// Catalogue reads the definitions through the catalog's tools, and keeps each definition it read by its
// Catalogue reads the definitions through the catalogue's tools, and keeps each definition it read by its
// module and commit, which never change together.
type Catalogue struct {
Ask Asking
@@ -171,21 +171,21 @@ type catalogueEntry struct {
Commit string `json:"commit"`
}
// Read asks the catalog for every module and reads each definition, bounded by DefinitionsTimeout.
// Read asks the catalogue for every module and reads each definition, bounded by DefinitionsTimeout.
func (c *Catalogue) Read(ctx context.Context, names StoreNames) *Definitions {
defs := &Definitions{Pins: map[string][]Pin{}, Unread: map[string]string{}, Names: names}
if c == nil || c.Ask == nil {
defs.NotAsked = "this bundle has no way to ask the catalog"
defs.NotAsked = "this bundle has no way to ask the catalogue"
return defs
}
raw, err := c.Ask(CatalogueModules, map[string]any{})
if err != nil {
defs.NotAsked = "the catalog's modules could not be asked: " + err.Error()
defs.NotAsked = "the catalogue's modules could not be asked: " + err.Error()
return defs
}
answer, output, ok := answerOf(raw)
if !ok {
defs.NotAsked = "the catalog refused its modules: " + lastLine(output)
defs.NotAsked = "the catalogue refused its modules: " + lastLine(output)
return defs
}
var list struct {
@@ -193,7 +193,7 @@ func (c *Catalogue) Read(ctx context.Context, names StoreNames) *Definitions {
Error string `json:"error"`
}
if err := json.Unmarshal(answer, &list); err != nil || list.Error != "" || list.Modules == nil {
defs.NotAsked = "the catalog's modules answer is not readable: " + firstOf(list.Error, errText(err), lastLine(output))
defs.NotAsked = "the catalogue's modules answer is not readable: " + firstOf(list.Error, errText(err), lastLine(output))
return defs
}
c.mu.Lock()
@@ -1,129 +0,0 @@
package main
// The artifact store's door and the bus, as the holder reaches them through this module's runtime.
import (
"context"
"encoding/json"
"fmt"
"os"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"artifactstoretools/internal/registry"
)
// door is registry.Store at the store's address as this machine reaches it.
type door struct{ s registry.Store }
func (d *door) Address() string { return d.s.Address }
func (d *door) Hold(ctx context.Context, r string) (bool, error) { return d.s.Hold(ctx, r) }
func (d *door) Holds(ctx context.Context, r string) error { return d.s.Holds(ctx, r) }
func (d *door) IsIndex(ctx context.Context, r string) (bool, error) {
return d.s.IsIndex(ctx, r)
}
func (d *door) LetGo(ctx context.Context, r string) error { return d.s.LetGo(ctx, r) }
func (d *door) SpareKept(ctx context.Context, keptImages, touching []string) error {
spare, err := d.s.SpareKeptIndexes(ctx, keptImages, touching)
if err == nil {
d.s.Spare = spare
}
return err
}
// DoorAt reads the store's address (host:port) from the file the manifest fills from the seat mesh-artifact-store's
// reach, each time, so a store that moved is reached where it is now.
func DoorAt(file string) func() (Door, error) {
return func() (Door, error) {
if file == "" {
return nil, fmt.Errorf("this bundle was given no file naming the artifact store's address")
}
raw, err := os.ReadFile(file)
if err != nil {
return nil, fmt.Errorf("the artifact store's address: %w", err)
}
address := strings.TrimSpace(string(raw))
if address == "" {
return nil, fmt.Errorf("the artifact store's address is empty in %s", file)
}
return &door{s: registry.Store{Address: address}}, nil
}
}
// runtimeBus is the holder's bus through the runtime: its reads, its findings and its seat's event.
type runtimeBus struct{}
func (runtimeBus) Keys(state string) ([]string, error) { return stdio.State(state).Keys() }
func (runtimeBus) Get(state, key string) ([]byte, bool, error) {
e, err := stdio.State(state).Get(key)
if err != nil || e == nil {
return nil, false, err
}
return e.Value, true, nil
}
func (runtimeBus) History(state, key string) ([]stdio.StateRevision, error) {
return stdio.State(state).History(key)
}
func (runtimeBus) LogSince(read string, at time.Time) ([]stdio.LogEntry, error) {
var out []stdio.LogEntry
page, err := stdio.Log(read).Since(at, "", 0)
for err == nil {
out = append(out, page.Entries...)
if page.Next == 0 {
return out, nil
}
page, err = stdio.Log(read).Read(page.Next, "", 0)
}
return nil, err
}
func (runtimeBus) Put(state, key string, value any) error {
_, err := stdio.State(state).Put(key, value)
return err
}
func (runtimeBus) Delete(state, key string) error { return stdio.State(state).Delete(key) }
func (runtimeBus) Publish(subject string, body any) error {
return stdio.SeatPublish(subject, body, "")
}
// follow watches the builder's done keys and the wants: a build's end sweeps (once the lease names no controller) and
// observes again; a want changed observes again. Event-driven: nothing reads them on a timer.
func follow(h *Holder) {
<-stdio.Served()
watch := func(state string, handle func(stdio.StateChange) error) {
for {
if err := stdio.State(state).Watch("", handle); err == nil {
return
} else {
h.logf("watching %s: %v; asked again in 10s", state, err)
}
time.Sleep(10 * time.Second)
}
}
observe := func() {
if err := h.Observe(context.Background()); err != nil {
h.logf("image-missing could not be observed: %v", err)
}
}
watch(BuildsRead, func(c stdio.StateChange) error {
if c.Current || c.Op != stdio.OpPut {
return nil
}
var b struct {
State string `json:"state"`
}
if json.Unmarshal(c.Value, &b) == nil && b.State == "done" {
go func() { h.AfterBuild(c.Key); observe() }()
}
return nil
})
watch(WantsRead, func(c stdio.StateChange) error {
if !c.Current {
go observe()
}
return nil
})
observe()
}
File diff suppressed because it is too large Load Diff
@@ -1,260 +0,0 @@
//go:build integration
package main
// The seat mesh-artifact-sweep's holder whole against a real bus (novox/hq ADR 0571 clause 2, issue 1955): the builder's done
// keys, the wants and the controller's lease in a real nats-server's key-value buckets, a build's end heard by a watch
// of the builds as the runtime hands it over, each deletion recorded by a request to the seat mesh-hand-acts on the bus,
// artifacts-gone said on the seat's subject, and S25's finding written to the holder's own bucket. Only the store's
// door is in memory: its HTTP is the copied registry package's, tested there.
import (
"context"
"encoding/json"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/handacts"
"git.novox.be/novox/mesh-sdk/go/testbus"
)
// realBus is the holder's Bus over a real nats-server: each read state a bucket named as the runtime names it.
type realBus struct {
ctx context.Context
nc *nats.Conn
js jetstream.JetStream
// log is the builder's log as the runtime's log verbs would read it (mesh-tools adds them; the sdk's own read of a
// log is tested in mesh-sdk's process package).
log []stdio.LogEntry
}
func (b realBus) LogSince(_ string, at time.Time) ([]stdio.LogEntry, error) {
var out []stdio.LogEntry
for _, e := range b.log {
if !e.At.Before(at) {
out = append(out, e)
}
}
return out, nil
}
func bucketOf(state string) string { return strings.Replace(state, ".", "_", 1) }
func (b realBus) kv(state string) (jetstream.KeyValue, error) {
return b.js.KeyValue(b.ctx, bucketOf(state))
}
func (b realBus) Keys(state string) ([]string, error) {
kv, err := b.kv(state)
if err != nil {
return nil, err
}
keys, err := kv.Keys(b.ctx)
if err == jetstream.ErrNoKeysFound {
return nil, nil
}
return keys, err
}
func (b realBus) Get(state, key string) ([]byte, bool, error) {
kv, err := b.kv(state)
if err != nil {
return nil, false, err
}
e, err := kv.Get(b.ctx, key)
if err == jetstream.ErrKeyNotFound {
return nil, false, nil
}
if err != nil {
return nil, false, err
}
return e.Value(), true, nil
}
func (b realBus) History(state, key string) ([]stdio.StateRevision, error) {
kv, err := b.kv(state)
if err != nil {
return nil, err
}
h, err := kv.History(b.ctx, key)
if err != nil {
return nil, err
}
var out []stdio.StateRevision
for _, e := range h {
out = append(out, stdio.StateRevision{Key: key, Value: e.Value(), Revision: e.Revision(), At: e.Created()})
}
return out, nil
}
func (b realBus) Put(state, key string, value any) error {
kv, err := b.kv(state)
if err != nil {
return err
}
raw, _ := json.Marshal(value)
_, err = kv.Put(b.ctx, key, raw)
return err
}
func (b realBus) Delete(state, key string) error {
kv, err := b.kv(state)
if err != nil {
return err
}
return kv.Delete(b.ctx, key)
}
func (b realBus) Publish(subject string, body any) error {
raw, _ := json.Marshal(body)
return b.nc.Publish(subject, raw)
}
func TestTheSeatSweepsAtABuildsEndOnARealBusOnceTheLeaseIsAway(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
nc, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
for _, s := range []string{BuildsRead, WantsRead, HavesRead, LeaseRead, ManifestsRead, "artifact-store-tools." + FindingsState} {
if _, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: bucketOf(s), History: 5}); err != nil {
t.Fatal(err)
}
}
bus := realBus{ctx: ctx, nc: nc, js: js}
// The mesh as aMesh lays it, copied onto the real buckets: letta's seven builds, a want, the store's listing.
mem, memb, door, _ := aMesh(t, &buscompose.LeaseHolder{Instance: "c1", Host: "novox", Epoch: 3})
bus.log = memb.log
for _, s := range []string{BuildsRead, WantsRead, LeaseRead, ManifestsRead} {
keys, _ := memb.Keys(s)
for _, k := range keys[:min(len(keys), 6)] { // the seventh build ends later, on the bus
raw, _, _ := memb.Get(s, k)
if err := bus.Put(s, k, json.RawMessage(raw)); err != nil {
t.Fatal(err)
}
}
}
time.Sleep(20 * time.Millisecond) // each build's end is its key's time on the bus: the seventh the newest
// The seat mesh-hand-acts, answering every act it records; artifacts-gone heard on the seat's subject.
var mu sync.Mutex
var acts, gone []string
if _, err := nc.Subscribe("mesh.seat."+handacts.Seat+".tool.hand-act", func(m *nats.Msg) {
mu.Lock()
acts = append(acts, string(m.Data))
mu.Unlock()
_ = m.Respond([]byte(`{"ok":true}`))
}); err != nil {
t.Fatal(err)
}
if _, err := nc.Subscribe(GoneSubject, func(m *nats.Msg) {
mu.Lock()
gone = append(gone, string(m.Data))
mu.Unlock()
}); err != nil {
t.Fatal(err)
}
h := &Holder{Bus: findingsAt{realBus: bus}, Door: mem.Door, Listed: mem.Listed, Now: time.Now, Logf: t.Logf}
h.Acts = handacts.Recorder{LeaseAway: h.LeaseAway, Ask: func(key string, body any) (json.RawMessage, error) {
raw, _ := json.Marshal(body)
m, err := nc.Request("mesh.seat."+strings.TrimPrefix(strings.Replace(key, ".", ".tool.", 1), "seat:"), raw, 5*time.Second)
if err != nil {
return nil, err
}
return m.Data, nil
}}
// A build's end heard from the builds bucket, as follow hands it over.
kv, _ := bus.kv(BuildsRead)
w, err := kv.WatchAll(ctx, jetstream.UpdatesOnly())
if err != nil {
t.Fatal(err)
}
ended := func() {
keys, _ := memb.Keys(BuildsRead)
raw, _, _ := memb.Get(BuildsRead, keys[6])
if err := bus.Put(BuildsRead, keys[6], json.RawMessage(raw)); err != nil {
t.Fatal(err)
}
select {
case e := <-w.Updates():
h.AfterBuild(e.Key())
case <-ctx.Done():
t.Fatal("the build's end was not heard")
}
}
// While the lease names the controller, a build's end lets nothing go.
ended()
if len(door.asked) != 0 {
t.Fatalf("before the lease write the store was asked %v", door.asked)
}
// The lease handed away: the next build's end holds the kept archives and lets go of letta's 2nd, which the 7th moved
// past the five newest and no want names (the want keeps the 1st), recording each deletion.
if err := bus.Put(LeaseRead, buscompose.ControllerLeaseKey, buscompose.LeaseHolder{HandedTo: "mesh-grants", Why: "the lease write"}); err != nil {
t.Fatal(err)
}
ended()
deadline := time.Now().Add(5 * time.Second)
for {
mu.Lock()
n := len(gone)
mu.Unlock()
if n > 0 || time.Now().After(deadline) {
break
}
time.Sleep(20 * time.Millisecond)
}
mu.Lock()
defer mu.Unlock()
if len(acts) != 3 || !strings.Contains(acts[0], archiveRef("letta", 2)) || !strings.Contains(acts[1], imageRef("letta", 2)) ||
!strings.Contains(acts[2], mirror(1)) {
t.Fatalf("the hand acts recorded are %v; the store was asked %v", acts, door.asked)
}
if len(gone) != 1 || !strings.Contains(gone[0], `"fingerprint":"`+fp(2)+`"`) {
t.Fatalf("artifacts-gone said %v", gone)
}
// S25 through the real findings bucket: letta's wanted image missing from the store is a finding there.
view, _ := mem.Listed(ctx)
repo, digest, _ := splitReference(imageRef("letta", 1))
delete(view.L.Repos[repo].Revisions, digest)
if err := h.Observe(ctx); err != nil {
t.Fatal(err)
}
if keys, _ := bus.Keys("artifact-store-tools." + FindingsState); len(keys) != 1 || keys[0] != "module.letta.ace.image-missing" {
t.Fatalf("the findings are %v", keys)
}
}
// findingsAt reads and writes the holder's own findings in its bucket, artifact-store-tools_findings.
type findingsAt struct{ realBus }
func (f findingsAt) own(state string) string {
if state == FindingsState {
return Module + "." + state
}
return state
}
func (f findingsAt) Keys(state string) ([]string, error) { return f.realBus.Keys(f.own(state)) }
func (f findingsAt) Get(state, key string) ([]byte, bool, error) {
return f.realBus.Get(f.own(state), key)
}
func (f findingsAt) Put(state, key string, value any) error {
return f.realBus.Put(f.own(state), key, value)
}
func (f findingsAt) Delete(state, key string) error { return f.realBus.Delete(f.own(state), key) }
@@ -1,351 +0,0 @@
package main
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"sync"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/handacts"
"git.novox.be/novox/mesh-sdk/go/health"
"artifactstoretools/internal/registry"
)
// memBus is the holder's bus in memory: states by name, each key's history, and what was published.
type memBus struct {
mu sync.Mutex
states map[string]map[string][]byte
history map[string][]stdio.StateRevision
log []stdio.LogEntry
published []string
}
func (b *memBus) LogSince(read string, at time.Time) ([]stdio.LogEntry, error) {
var out []stdio.LogEntry
for _, e := range b.log {
if read == delivery.BuildLogRead && !e.At.Before(at) {
out = append(out, e)
}
}
return out, nil
}
func (b *memBus) Keys(state string) ([]string, error) {
b.mu.Lock()
defer b.mu.Unlock()
var out []string
for k := range b.states[state] {
out = append(out, k)
}
sort.Strings(out)
return out, nil
}
func (b *memBus) Get(state, key string) ([]byte, bool, error) {
b.mu.Lock()
defer b.mu.Unlock()
v, ok := b.states[state][key]
return v, ok, nil
}
func (b *memBus) History(state, key string) ([]stdio.StateRevision, error) {
return b.history[state+" "+key], nil
}
func (b *memBus) Put(state, key string, value any) error {
raw, _ := json.Marshal(value)
b.mu.Lock()
defer b.mu.Unlock()
if b.states[state] == nil {
b.states[state] = map[string][]byte{}
}
b.states[state][key] = raw
return nil
}
func (b *memBus) Delete(state, key string) error {
b.mu.Lock()
defer b.mu.Unlock()
delete(b.states[state], key)
return nil
}
func (b *memBus) Publish(subject string, body any) error {
raw, _ := json.Marshal(body)
b.mu.Lock()
defer b.mu.Unlock()
b.published = append(b.published, subject+" "+string(raw))
return nil
}
// memDoor is the store's door in memory: what it holds, which references are indexes, and what was asked.
type memDoor struct {
holds map[string]bool
indexes map[string]bool
asked []string
}
func (d *memDoor) Address() string { return "10.0.0.5:5000" }
func (d *memDoor) Hold(_ context.Context, r string) (bool, error) {
d.asked = append(d.asked, "hold "+r)
if !d.holds[r] {
return false, registry.Gone
}
return false, nil
}
func (d *memDoor) Holds(_ context.Context, r string) error {
if !d.holds[r] {
return registry.Gone
}
return nil
}
func (d *memDoor) IsIndex(_ context.Context, r string) (bool, error) { return d.indexes[r], nil }
func (d *memDoor) LetGo(_ context.Context, r string) error {
d.asked = append(d.asked, "let go "+r)
delete(d.holds, r)
return nil
}
func (d *memDoor) SpareKept(context.Context, []string, []string) error {
d.asked = append(d.asked, "spare")
return nil
}
// mirror is a copy of an upstream base in the store, the n-th.
func mirror(n int) string {
return "artifact-store://upstream/docker.io/library/alpine@sha256:" + strings.Repeat(fmt.Sprintf("%x", n%16), 64)
}
// pinnedImage is the image letta's manifest names by itself, at the store's address.
const pinnedImage = "10.0.0.5:5000/letta/pinned@sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"
func digestOf(ref string) string { _, d, _ := splitReference(ref); return d }
func fp(n int) string { return "src2:" + strings.Repeat(fmt.Sprintf("%x", n%16), 64) }
func imageRef(module string, n int) string {
return "artifact-store://" + module + "/image@sha256:" + strings.Repeat(fmt.Sprintf("%x", n%16), 64)
}
func archiveRef(module string, n int) string {
return "artifact-store://" + module + "/files/blobs/sha256:" + strings.Repeat(fmt.Sprintf("%x", (n+8)%16), 64)
}
// A mesh with seven done builds of letta, the n-th ending n hours after a day began, each with an image and an archive;
// a want names the first on ace; the store lists them all and one image no build names; the lease as given.
func aMesh(t *testing.T, lease *buscompose.LeaseHolder) (*Holder, *memBus, *memDoor, *[]string) {
t.Helper()
day := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
b := &memBus{states: map[string]map[string][]byte{}, history: map[string][]stdio.StateRevision{}}
door := &memDoor{holds: map[string]bool{}, indexes: map[string]bool{}}
listing := &Listing{Blobs: map[string]int64{}, Repos: map[string]*Repo{}}
for n := 1; n <= 7; n++ {
key, _ := delivery.BuilderBuildKey("letta", fp(n))
raw, err := delivery.EncodeBuilderBuild(delivery.BuilderBuild{State: delivery.BuildDone, Manifest: strings.Repeat("d", 64), Artifacts: []delivery.BuiltArtifact{
{Name: "image", Kind: "image", Digest: digestOf(imageRef("letta", n)), Reference: imageRef("letta", n)},
{Name: "files", Kind: "archive", Digest: digestOf(archiveRef("letta", n)), Reference: archiveRef("letta", n)},
{Name: "pkg", Kind: "package", Digest: "sha256:" + strings.Repeat("f", 64), Reference: "letta@1.0." + fmt.Sprint(n)}}})
if err != nil {
t.Fatal(err)
}
_ = b.Put(BuildsRead, key, json.RawMessage(raw))
b.history[BuildsRead+" "+key] = []stdio.StateRevision{{Key: key, Value: raw, At: day.Add(time.Duration(n) * time.Hour)}}
for _, r := range []string{imageRef("letta", n), archiveRef("letta", n)} {
door.holds[r] = true
repo, digest, kind := splitReference(r)
if kind == "archive" {
listing.Blobs[digest] = 10
} else {
listing.repo(repo).Revisions[digest] = true
}
}
}
listing.repo("letta/image").Revisions["sha256:"+strings.Repeat("e", 64)] = true // no build names it
listing.repo("letta/pinned").Revisions["sha256:"+strings.Repeat("c", 64)] = true
// Three copies of a base: letta's 2nd stood on the first, its 7th on the second, and no build on the third.
for n := 1; n <= 3; n++ {
repo, digest, _ := splitReference(mirror(n))
listing.repo(repo).Revisions[digest] = true
door.holds[mirror(n)] = true
}
for n := 1; n <= 7; n++ {
e := delivery.BuildLogEntry{Event: delivery.BuildEndedEvent, Module: "letta", FP: fp(n), Repository: "novox/x", Commit: "c",
Node: "ace", At: day.Add(time.Duration(n) * time.Hour), Outcome: "done"}
switch n {
case 2:
e.Against = []string{mirror(1)}
case 7:
e.Against = []string{mirror(2)}
}
raw, _ := delivery.EncodeBuildLogEntry(e)
b.log = append(b.log, stdio.LogEntry{Sequence: uint64(n), Key: "letta", Value: raw, At: e.At})
}
manifest, _ := delivery.EncodeManifest(delivery.Manifest{Module: "letta",
JSON: `{"module":"letta","resources":[{"id":"app","type":"container","image":"` + pinnedImage + `"},` +
`{"id":"built","type":"container","image":"${artifact:image}"}]}`})
mk, _ := delivery.ManifestsKey(strings.Repeat("d", 64))
_ = b.Put(ManifestsRead, mk, json.RawMessage(manifest))
want, _ := json.Marshal(delivery.Want{V: delivery.Version, FP: fp(1), Self: true})
_ = b.Put(WantsRead, "want.ace.letta", json.RawMessage(want))
if lease != nil {
_ = b.Put(LeaseRead, buscompose.ControllerLeaseKey, lease)
}
var acts []string
h := &Holder{Bus: b, Door: func() (Door, error) { return door, nil },
Listed: func(context.Context) (*View, error) { return &View{L: listing}, nil },
Now: func() time.Time { return day.Add(48 * time.Hour) },
Acts: handacts.Recorder{Ask: func(key string, body any) (json.RawMessage, error) {
raw, _ := json.Marshal(body)
acts = append(acts, key+" "+string(raw))
return json.RawMessage(`{}`), nil
}}}
h.Acts.LeaseAway = h.LeaseAway
return h, b, door, &acts
}
// ADR 0571 clause 2: the keep set from the store's listing, the wants and the five newest builds; nothing let go of while
// the lease names a controller; once it names another holder, a build's end lets go of the eligible within the bound,
// leaves an index for a confirmed collect, records each deletion and says artifacts-gone.
func TestTheSeatKeepsTheFiveNewestAndTheWantedAndSweepsOnlyOnceTheLeaseIsAway(t *testing.T) {
controller := &buscompose.LeaseHolder{Instance: "c1", Host: "novox", Epoch: 3}
handed := &buscompose.LeaseHolder{HandedTo: "mesh-grants", HandedBy: "person.jochen", Why: "the lease write"}
h, _, _, _ := aMesh(t, controller)
r, err := h.Artifacts("")
if err != nil {
t.Fatal(err)
}
states := map[string]string{}
for _, x := range r.References {
states[x.Reference] = x.State
}
// letta's 3rd to 7th are the five newest; its 1st a want names on ace, however old; its 2nd is eligible; the package
// and the manifest no build names are no reference of the seat's.
for n, want := range map[int]string{1: "kept", 2: "eligible", 3: "kept", 7: "kept"} {
if states[imageRef("letta", n)] != want || states[archiveRef("letta", n)] != want {
t.Errorf("letta's build %d is %s and %s, not %s", n, states[imageRef("letta", n)], states[archiveRef("letta", n)], want)
}
}
// And the copies: the one letta's 2nd stood on is eligible with it, the 7th's kept; the third no build names.
if states[mirror(1)] != "eligible" || states[mirror(2)] != "kept" || states[mirror(3)] != "" {
t.Errorf("the copies are %s, %s and %q", states[mirror(1)], states[mirror(2)], states[mirror(3)])
}
if m, err := h.MirrorsNow(); err != nil || len(m.Copies) != 3 || m.Copies[2].State != "unrecorded" || m.Held != "" {
t.Errorf("mirrors answered %+v: %v", m, err)
}
if len(r.References) != 16 || r.Counts["kept"] != 13 || r.Counts["eligible"] != 3 || r.Store != "10.0.0.5:5000" {
t.Errorf("the references are %d, counted %v, at %q", len(r.References), r.Counts, r.Store)
}
for _, c := range []struct {
name string
lease *buscompose.LeaseHolder
index bool
most int
letGo []string
left int
skipped int
unlog int
}{
{name: "the lease names a controller", lease: controller},
{name: "no lease key", lease: nil},
{name: "handed away", lease: handed, most: MostPerSweep, letGo: []string{archiveRef("letta", 2), imageRef("letta", 2), mirror(1)}},
{name: "an index waits", lease: handed, index: true, most: MostPerSweep, letGo: []string{archiveRef("letta", 2), mirror(1)}, skipped: 1},
{name: "bounded", lease: handed, most: 1, letGo: []string{archiveRef("letta", 2)}, left: 2},
// A kept build the log no longer holds: what it stood on is unknown, so no copy goes.
{name: "a kept build's bases unknown", lease: handed, most: MostPerSweep, letGo: []string{archiveRef("letta", 2), imageRef("letta", 2)}, unlog: 3},
} {
t.Run(c.name, func(t *testing.T) {
h, b, door, acts := aMesh(t, c.lease)
door.indexes[imageRef("letta", 2)] = c.index
if c.unlog > 0 {
b.log = append(b.log[:c.unlog-1:c.unlog-1], b.log[c.unlog:]...)
}
if c.most == 0 {
h.AfterBuild("letta.x")
if _, err := h.Collect("tidy", true, 0); err == nil {
t.Error("a confirmed collect was let through while the controller sweeps")
}
if len(door.asked) != 0 || len(*acts) != 0 || len(b.published) != 0 {
t.Fatalf("before the lease write the store was asked %v, acts %v, said %v", door.asked, *acts, b.published)
}
return
}
a, err := h.Sweep(context.Background(), Sweeping{Most: c.most, By: "after the build letta.x"})
if err != nil {
t.Fatal(err)
}
if fmt.Sprint(a.LetGo) != fmt.Sprint(c.letGo) || a.Left != c.left || a.Skipped != c.skipped || a.Held != 6 || a.Missing != 0 {
t.Fatalf("let go of %v (left %d, skipped %d, held %d, missing %d), want %v", a.LetGo, a.Left, a.Skipped, a.Held, a.Missing, c.letGo)
}
// Every kept archive held before the first deletion.
first := -1
for i, x := range door.asked {
if strings.HasPrefix(x, "let go ") && first < 0 {
first = i
}
}
if first != 6 {
t.Errorf("the store was asked %v: the six kept archives are held first", door.asked)
}
if len(*acts) != len(c.letGo) || !strings.HasPrefix((*acts)[0], "seat:mesh-hand-acts.hand-act ") ||
!strings.Contains((*acts)[0], c.letGo[0]) {
t.Errorf("the hand acts recorded are %v", *acts)
}
if len(b.published) != 1 || !strings.HasPrefix(b.published[0], GoneSubject+" ") ||
!strings.Contains(b.published[0], `"fingerprint":"`+fp(2)+`"`) {
t.Errorf("said %v", b.published)
}
})
}
}
// S25 (image-missing): a want whose build names an image the store does not hold is a finding in the seat's own state,
// written once and deleted once the store holds it again; images names what a machine's wants and haves name.
func TestAWantedImageTheStoreLacksIsAFindingAndImagesNamesWantsAndHaves(t *testing.T) {
h, b, _, _ := aMesh(t, nil)
view, _ := h.Listed(context.Background())
repo, digest, _ := splitReference(imageRef("letta", 1))
delete(view.L.Repos[repo].Revisions, digest)
h.Listed = func(context.Context) (*View, error) { return view, nil }
if err := h.Observe(context.Background()); err != nil {
t.Fatal(err)
}
key := health.FindingKey(health.ScopeModule, "letta.ace", ImageMissing)
raw, found, _ := b.Get(FindingsState, key)
f, err := health.DecodeFinding(raw)
if !found || err != nil || f.Subject.Machine != "ace" || f.Source != "S25" || !strings.Contains(f.Evidence[0].Said, imageRef("letta", 1)) {
t.Fatalf("the finding is %+v (%v, %v)", f, found, err)
}
view.L.Repos[repo].Revisions[digest] = true
if err := h.Observe(context.Background()); err != nil {
t.Fatal(err)
}
if _, found, _ := b.Get(FindingsState, key); found {
t.Fatal("the finding stayed once the store held the image again")
}
// An image the manifest pins by itself, under the store's address, is judged as the controller judged it.
delete(view.L.Repos["letta/pinned"].Revisions, "sha256:"+strings.Repeat("c", 64))
if err := h.Observe(context.Background()); err != nil {
t.Fatal(err)
}
if raw, found, _ := b.Get(FindingsState, key); !found || !strings.Contains(string(raw), "letta/pinned@sha256:") {
t.Fatalf("the pinnedImage image missing is %s", raw)
}
have, _ := json.Marshal(delivery.Have{V: delivery.Version, FP: fp(7), Phase: "running"})
_ = b.Put(HavesRead, "ace.letta", json.RawMessage(have))
images, err := h.ImagesOf("ace", "10.0.0.5:5000")
if err != nil || len(images.Images) != 3 || !images.SentKnown || images.Images[0].Image != "10.0.0.5:5000/letta/image@sha256:"+strings.Repeat("1", 64) ||
fmt.Sprint(images.Images[0].In) != "[declared]" || fmt.Sprint(images.Images[1].In) != "[sent]" ||
images.Images[2].Image != pinnedImage || fmt.Sprint(images.Images[2].In) != "[declared sent]" {
t.Fatalf("the images of ace are %+v: %v", images, err)
}
}
@@ -1,38 +1,30 @@
// The artifact-store-tools module's Go bundle (novox/hq ADR 0251 §1–3, ADR 0571 clause 2, to-be 51): a process the
// node's runtime launches and speaks MCP over stdio to. It says what the artifact store holds — its repositories, its
// size, and what the mesh keeps of each manifest — read from the store's own files through the store's own container,
// and it holds the seat mesh-artifact-sweep: the keep set, the sweep at each build's end once the controller's lease names
// no controller, and S25's image-missing.
// The artifact-store-tools module's Go bundle (novox/hq ADR 0251 §1–3, to-be 51): a process the node's
// runtime launches and speaks MCP over stdio to. It says what the artifact store holds — its
// repositories, its size, and what the controller's records say of each manifest — read from the
// store's own files through the store's own container, and writing nothing. A collection is asked of
// the controller, which decides and records what it lets go of (ADR 0189).
//
// **A module beside the store, not the store's own.** A module that provides the artifact store cannot also build an
// artifact: building publishes to the store, so the store would be needed to create itself. So these tools are a
// second module, assigned to the machine that holds the store. stdout is the protocol; what this bundle says, it says
// on stderr.
// **A module beside the store, not the store's own.** A module that provides the artifact store cannot
// also build an artifact: building publishes to the store, so the store would be needed to create
// itself, and the controller refuses that manifest. So these tools are a second module, assigned to the
// machine that holds the store. stdout is the protocol; what this bundle says, it says on stderr.
package main
import (
"fmt"
"os"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/handacts"
)
func logf(format string, a ...any) { fmt.Fprintf(os.Stderr, format+"\n", a...) }
func main() {
s := &Store{
Container: os.Getenv("MESH_ARTIFACT_STORE_CONTAINER"),
Run: ExecRunner,
UID: os.Getuid(),
}
h := &Holder{Bus: runtimeBus{}, Door: DoorAt(os.Getenv("MESH_ARTIFACT_STORE_ADDRESS_FILE")), Listed: s.View,
Now: time.Now, Logf: logf}
h.Acts = handacts.Recorder{Ask: stdio.Ask, LeaseAway: h.LeaseAway}
go follow(h)
// An empty name serves as the module the runtime names (MESH_SERVED_MODULE): artifact-store-tools.
if err := stdio.Serve("", append(Tools(s, h, &Catalogue{Ask: stdio.Ask}), SeatTools(h, s)...)); err != nil {
if err := stdio.Serve("", Tools(s, Controller{Ask: stdio.Ask}, &Catalogue{Ask: stdio.Ask})); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
@@ -1,122 +0,0 @@
package main
// The seat mesh-artifact-sweep's verbs (novox/hq ADR 0571 clause 2; catalogue.ArtifactSweepSeat), each a tool named
// `artifact-store.<verb>` as the runtime serves a seat's verb, answering as the controller's verb answered: {output,
// ok, answer}.
import (
"fmt"
"strings"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// verbAnswer is the controller's verbs' answer shape, which the builder and docker read (answerOf).
type verbAnswer struct {
Output string `json:"output"`
OK bool `json:"ok"`
Answer any `json:"answer"`
}
// SeatTools are the seat's verbs over h; s reads the store's listing for mirrors.
func SeatTools(h *Holder, s *Store) []stdio.Tool {
seat := catalogue.ArtifactSweepSeat + "."
return []stdio.Tool{
{Name: seat + "artifacts", Description: "Every artifact the mesh made that the store holds: kept and why, eligible, " +
"or with collected those it no longer holds.",
Run: func(args map[string]any) (any, error) {
r, err := h.Artifacts(text(args, "repository"))
if err != nil {
return nil, err
}
if text(args, "collected") != "true" {
refs := r.References[:0:0]
for _, x := range r.References {
if x.State != "collected" {
refs = append(refs, x)
}
}
r.References = refs
}
return verbAnswer{Output: fmt.Sprintf("%d kept, %d eligible, %d collected\n", r.Counts["kept"],
r.Counts["eligible"], r.Counts["collected"]), OK: true, Answer: r}, nil
}},
{Name: seat + "held", Description: "Whether the store holds each reference given: held, gone or unknown with why.",
Run: func(args map[string]any) (any, error) {
a, err := h.Held(listOf(args["references"]))
if err != nil {
return nil, err
}
return verbAnswer{Output: fmt.Sprintf("the artifact store at %s holds %d of the %d reference(s) asked\n",
a.Store, len(a.Held), len(a.Held)+len(a.Gone)+len(a.Unknown)), OK: true, Answer: a}, nil
}},
{Name: seat + "collect", Description: "The sweep now: a dry run without confirm; with confirm and why, every kept " +
"archive held, then each eligible artifact let go of, each recorded in the hand-act log.",
Run: func(args map[string]any) (any, error) {
most, err := count(args, "most", MostPerSweep, 1, MostAtAll)
if err != nil {
return nil, err
}
a, err := h.Collect(text(args, "why"), text(args, "confirm") == "true", most)
if err != nil {
return nil, err
}
return verbAnswer{Output: fmt.Sprintf("let go of %d, would let go of %d, %d left\n", len(a.LetGo),
len(a.WouldLetGo), a.Left), OK: true, Answer: a}, nil
}},
{Name: seat + "mirrors", Description: "The bases the mesh copied into its store, each kept (a kept build stood on " +
"it), eligible (only builds no longer kept stood on it) or unrecorded (no build the builder's log holds stood " +
"on it: never removed).",
Run: func(args map[string]any) (any, error) {
if text(args, "record") != "" {
return nil, fmt.Errorf("mirrors records nothing by hand on this seat: a copy is recorded by the builds the " +
"builder's log says stood on it (novox/hq ADR 0571 clause 1)")
}
m, err := h.MirrorsNow()
if err != nil {
return nil, err
}
return verbAnswer{Output: fmt.Sprintf("%d copies\n", len(m.Copies)), OK: true, Answer: m}, nil
}},
{Name: seat + "images", Description: "Every container image one machine's wants and haves name.",
Run: func(args map[string]any) (any, error) {
node := text(args, "node")
if node == "" {
return nil, fmt.Errorf("images names the machine")
}
address := ""
if d, err := h.Door(); err == nil {
address = d.Address()
}
a, err := h.ImagesOf(node, address)
if err != nil {
return nil, err
}
return verbAnswer{Output: fmt.Sprintf("%s: %d images\n", node, len(a.Images)), OK: true, Answer: a}, nil
}},
}
}
// listOf is a list argument: a list, or one text separated by commas.
func listOf(v any) []string {
var out []string
add := func(s string) {
for _, p := range strings.Split(s, ",") {
if p = strings.TrimSpace(p); p != "" && !contains(out, p) {
out = append(out, p)
}
}
}
switch x := v.(type) {
case string:
add(x)
case []any:
for _, e := range x {
if s, ok := e.(string); ok {
add(s)
}
}
}
return out
}
@@ -1,8 +1,6 @@
package main
import (
"git.novox.be/novox/mesh-sdk/go/catalogue"
"context"
"encoding/json"
"fmt"
@@ -413,11 +411,6 @@ func TestTheToolsServedAreTheToolsTheManifestNames(t *testing.T) {
var m struct {
Tools []string `json:"tools"`
Invokes []string `json:"invokes"`
Reads []string `json:"reads"`
Claims []struct {
Name string `json:"name"`
Serves []string `json:"serves"`
} `json:"claims"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
@@ -435,30 +428,10 @@ func TestTheToolsServedAreTheToolsTheManifestNames(t *testing.T) {
if !reflect.DeepEqual(served, listed) {
t.Fatalf("served %v, manifest %v", served, listed)
}
// ADR 0571 clause 2: it holds the seat mesh-artifact-sweep and serves its verbs, and calls no verb of the controller's;
// the lease is the one controller state it reads (clause 10's exemption).
want := []string{"seat:mesh-hand-acts.hand-act", CatalogueModules, CatalogueModule}
want := []string{"seat:mesh-controller.artifacts", "seat:mesh-controller.collect", CatalogueModules, CatalogueModule}
if !reflect.DeepEqual(m.Invokes, want) {
t.Errorf("invokes %v", m.Invokes)
}
var verbs []string
for _, tl := range SeatTools(&Holder{}, &Store{}) {
verbs = append(verbs, strings.TrimPrefix(tl.Name, catalogue.ArtifactSweepSeat+"."))
}
seat, _ := catalogue.SeatNamed(catalogue.ArtifactSweepSeat)
var contract []string
for _, v := range seat.Serves {
contract = append(contract, v.Name)
}
if len(m.Claims) != 1 || m.Claims[0].Name != catalogue.ArtifactSweepSeat || !reflect.DeepEqual(m.Claims[0].Serves, verbs) ||
!reflect.DeepEqual(verbs, contract) {
t.Errorf("claims %+v, serves %v, the seat's verbs %v", m.Claims, verbs, contract)
}
for _, r := range m.Reads {
if strings.HasPrefix(r, "mesh-controller.") && r != LeaseRead {
t.Errorf("reads the controller's %s", r)
}
}
}
func TestASocketRefusalIsAskedAgainThroughSudo(t *testing.T) {
@@ -11,16 +11,9 @@ import (
stdio "git.novox.be/novox/mesh-sdk/go"
)
// Recorded is what says which references the mesh keeps and lets go of them: the seat's holder (Holder, novox/hq ADR
// 0571 clause 2), or before it the controller (Controller), each in the controller's answer's shape.
type Recorded interface {
Artifacts(repository string) (*Records, error)
Collect(why string, confirm bool, most int) (*CollectAnswer, error)
}
// Tools are the artifact store's tools (novox/hq ADR 0251 §1). The first three only read; artifact_store_collect
// only reads too, unless it is a real run, and then it asks the seat's sweep, which decides and deletes.
func Tools(s *Store, c Recorded, cat *Catalogue) []stdio.Tool {
// only reads too, unless it is a real run, and then it asks the controller, which decides and deletes.
func Tools(s *Store, c Controller, cat *Catalogue) []stdio.Tool {
ctx := context.Background()
repoArg := map[string]any{"type": "string", "description": "one repository, as the store names it (<module>/<artifact>)"}
return []stdio.Tool{
@@ -61,7 +54,7 @@ func Tools(s *Store, c Recorded, cat *Catalogue) []stdio.Tool {
Name: "artifact_store_references",
Description: "What the controller's records say of each manifest the artifact store holds: kept (a definition " +
"names it, or one of the five most recent builds of a module the mesh holds), named by a definition (no " +
"record names it, and a definition in the catalog pins it by digest, under the store's current name or " +
"record names it, and a definition in the catalogue pins it by digest, under the store's current name or " +
"an older one: the module is said), the holder of a kept archive, " +
"eligible (the mesh made it and keeps it for no reason), the holder of an eligible archive, let go yet present, " +
"a named document the controller keeps under a tag, or unrecorded — no record names it. A platform manifest " +

Some files were not shown because too many files have changed in this diff Show More