Compare commits

..
Author SHA1 Message Date
jschoubben d1692c6ade Probe placement seed-10 for issue 1126: the four placement files, not to be merged
mesh/delivery closed: the pull request closed unmerged
mesh/check checking d1692c6ade73 on 58309a972945
2026-10-28 16:38:26 +01:00
mesh-tools 58309a9729 Merge pull request 'power: every reboot, poweroff and halt is bounded at 10 minutes (group fix/shutdown-bound, issue 1126)' (#547) from fix/shutdown-bound into main 2026-10-28 15:05:06 +00:00
jschoubben 6724dab0cf power: bound every shutdown at ten minutes, not systemd's thirty (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-catalog@58309a972945
mesh/delivery-group group fix/shutdown-bound merged: every member is merged
mesh/check pass 6724dab0cf96 on 387ee59140c4 | unit 2192 run, 3794 kept, 57 skipped | integration 2208 run, 3829 kept, 57 skipped | over bound: unit, …
On ace's reboot of 2026-10-28 every unit had stopped within 17 s, yet the bus and
polkit were started again during the shutdown, nothing stopped them a second
time, and the reboot waited on reboot.target's 30-minute job timeout, silent,
until the operator forced it. node-power owns the shutdown on every machine, so
its drop-ins on reboot, poweroff and halt shorten the bound; each target keeps
its own forcing action, so the last phase still ends every process, unmounts
and syncs. The cause, node-tools starting bundles again while it stops, is
fixed in mesh-tools.
2026-10-28 15:30:54 +01:00
mesh-tools 387ee59140 Merge pull request 'mesh-build-image: a node that cannot build here now stays healthy and says not eligible, so it never holds the delivery (issue 1126)' (#546) from fix/1126-ineligible-healthy into main 2026-10-28 14:20:40 +00:00
mesh-tools 85bbeb37e3 Merge pull request 'mesh-vault: own answers a node within its bound — assignments from a watched mirror, one read per module, one synced record per answer (issue 1126)' (#545) from fix/1126-vault-own into main 2026-10-28 14:20:35 +00:00
jschoubben 363039f337 mesh-build-image README: a node without tun is healthy and not eligible, as the module now says (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-catalog@387ee59140c4
mesh/check checking 363039f337ad on 387ee59140c4
2026-10-28 15:01:42 +01:00
jschoubben a9d96b30be mesh-vault: a watch of the assignments begun again keeps nothing an earlier attempt took, and a module the mirror lacks is read directly before a credential is refused (issue 1126, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-catalog@85bbeb37e3e0
mesh/delivery-group group fix/1126-vault-own merged: every member is merged
mesh/check pass a9d96b30be8d on 69056948bfa8 | unit 1639 run, 4201 kept, 46 skipped | integration 1674 run, 4215 kept, 47 skipped | over bound: unit, …
A watch that failed partway kept what it had taken; begun again, the runtime does not hand a key deleted meanwhile, so
a removed assignment read as standing and credentialRefusal could hand a consumer credential to a node no longer
running its module. Each attempt now begins from nothing held, and a late delivery to an older attempt changes nothing.
A 'not on the node' refusal is confirmed by a direct read first, since a watched mirror can be behind.
2026-10-28 14:55:36 +01:00
jschoubben 644a34cf49 mesh-build-image stays healthy on a node that cannot build here now (no tun) and says it is not eligible: the assigner read unhealthy as a failed delivery and held the build (issue 1126) 2026-10-28 14:50:28 +01:00
jschoubben 277a4d0060 mesh-vault answers own in one pass: assignments from its watch, each module's once, one synced record per answer (issue 1126)
Seed-9's probe (13:31-13:36Z): novox's own of 57 secrets and 146 credentials took 26 s, past the node-engine's 15 s,
and ace and novox refused their whole shadow answer. own read the controller's assignment over the bus once per secret
and per credential (203 reads), read every record of every node, and synced the hand-out record once per value, under
the one lock every node's answers share.

Now the vault watches the assignments mirror (a watch, never a poll) and reads it from memory once caught up, over the
bus until then; one answer reads each module's assignment once; only own/<node>/ records are read; and every hand-out
of one answer is recorded in one synced write, none handed when it cannot be.

Quick fix weighed: a longer bound in the node-engine (VaultWithin). Thorough fix chosen: the cost grows with every
module and every node, and the lock is shared by every shadow answer and every apply.
2026-10-28 14:47:20 +01:00
mesh-tools 69056948bf Merge pull request 'The kernel module loads tun at boot and keeps the running kernel's modules across an upgrade; mesh-build-image takes no ask where it cannot build (group fix/ace-tun)' (#543) from fix/ace-tun into main 2026-10-28 13:07:42 +00:00
mesh-tools 9cdac09ad9 Merge pull request 'knowledge-base.progress takes since: what changed in the last hour; issue-tracker.changes says each issue's moves from what to what' (#541) from feat/progress-since into main 2026-10-28 11:56:32 +00:00
jschoubben 86d37a14e7 mesh-build-image: pin mesh-sdk 1618837, the group's head (fix/ace-tun)
mesh/delivery merged: source.main moved to novox/mesh-catalog@69056948bfa8
mesh/delivery-group group fix/ace-tun merged: every member is merged
mesh/check pass 86d37a14e7ab on 9cdac09ad962 | carried from 25f615456423 | unit 2293 run, 3984 kept, 16 skipped | integration 2331 run, 4031 kept, 24 …
2026-10-28 12:50:47 +01:00
jschoubben d13f74df40 A node that cannot open /dev/net/tun takes no image build, and every node loads tun at boot and keeps its running kernel's modules
An image build node up 24 days had lost its running kernel's modules to a linux-lts upgrade, so tun
could not load and every RUN line failed in pasta. mesh-build-image now proves the tun device and,
whatever its look refuses, judges its node not eligible (capacity 0 with the reason), so the asks go
to a node that can build them. The kernel module loads tun at boot and declares kernel-modules-hook,
which keeps the running kernel's modules across an upgrade. Needs mesh-sdk fix/ace-tun (Puller.Eligible).
2026-10-28 12:44:58 +01:00
jschoubben dfe490d9e2 issue-tracker.changes reads each issue's moves over a window from the tracker's own log, so progress since says from what each issue moved, not only where it is
mesh/delivery merged: source.main moved to novox/mesh-catalog@9cdac09ad962
mesh/check being carried to dfe490d9e245 on 9cdac09ad962 | from 25f615456423
2026-10-28 12:33:01 +01:00
jschoubben 4f317201c9 knowledge-base.progress takes since: only what changed in a window, read from the tracker's and mesh-delivery's history, so a five-minute report says what moved (the operator, 2026-10-28) 2026-10-28 12:33:01 +01:00
54 changed files with 1899 additions and 64 deletions
+17 -1
View File
@@ -53,6 +53,22 @@ built for the fallback kernel too) after its boot-entries service, so the entry
| file | `/etc/pacman.d/hooks/90-mesh-kernel-boot-entries.hook` | runs the script after a kernel image is installed, upgraded or removed |
| packages | `linux-lts`, `linux-lts-headers` | declared after the service and the hook |
## Kernel modules at boot, and the running kernel's modules kept
A machine that runs for weeks meets a kernel upgrade that removes `/usr/lib/modules/<the running kernel>`: from then on
no module that was not loaded yet can be loaded until the next boot. On 2026-10-28 a build node, up 24 days on
`linux-lts` 6.18.54 with only 6.18.58's modules on disk, had `/dev/net/tun` and no tun driver, and every image build
there failed in buildah's network setup ("Failed to open() /dev/net/tun: No such device").
| | what | notes |
|---|---|---|
| file | `/etc/modules-load.d/mesh-kernel.conf` | the kernel modules loaded at every boot by systemd-modules-load: `tun` (pasta, which buildah uses for a RUN line's network, and every user-space tunnel open `/dev/net/tun`). Read at boot only: nothing is loaded by an apply |
| package | `kernel-modules-hook` | keeps the running kernel's modules across an upgrade (saved before the transaction, put back after), so a module first needed later still loads |
| service | `linux-modules-cleanup.service` | the package's oneshot, enabled at boot and stopped otherwise: it moves the modules of a kernel that is neither running nor owned by a package to `/usr/lib/modules/.old`, which the package's tmpfiles rule empties after four weeks. An apply never starts it |
A machine whose running kernel's modules are already gone loads nothing new until it is started again; the image build
module says so in its health and takes no image build there meanwhile.
## Tools
| tool | | answers |
@@ -63,7 +79,7 @@ built for the fallback kernel too) after its boot-entries service, so the entry
## Tests
`go test ./...`: the tool over a stand-in tree holding the control node's own files of 2026-10-14; the
manifest's shape (the default, the order, the unit and its sandbox, reload, restart and health); the boot-entries script run with `sh` over stand-in roots (a systemd-boot entries directory, a GRUB menu, a hook that copies kernels to one fixed name): the entry from the booted command line, a menu generated again only when the kernels changed, no loader, no room, the taken-over hook; `kernel_boot` over the same roots; and the huge-pages script as the mesh
manifest's shape (the default, the order, the unit and its sandbox, reload, restart and health); the kernel modules loaded at boot (`tun`) and the package and boot service that keep the running kernel's modules; the boot-entries script run with `sh` over stand-in roots (a systemd-boot entries directory, a GRUB menu, a hook that copies kernels to one fixed name): the entry from the booted command line, a menu generated again only when the kernels changed, no loader, no room, the taken-over hook; `kernel_boot` over the same roots; and the huge-pages script as the mesh
writes it, run with `sh` against a stand-in directory: it sets and reads back, keeps what it found once,
rotates the journal once and not for `always` nor while the machine starts, goes straight to a new value on a
reload without rewriting what it found, gives back what it found and then keeps nothing, refuses a value the kernel
@@ -24,7 +24,7 @@ func TestTheDefaultIsMadviseAndTheUnitFollowsTheSetting(t *testing.T) {
for _, r := range m.Resources {
ids = append(ids, r["id"].(string))
}
if strings.Join(ids, " ") != "state huge-pages unit apply boot-entries boot-unit boot-hook boot-apply linux-lts linux-lts-headers" {
if strings.Join(ids, " ") != "state huge-pages unit apply boot-entries boot-unit boot-hook boot-apply linux-lts linux-lts-headers modules-load modules-keep modules-cleanup" {
t.Fatalf("order: %v", ids)
}
script := m.resource(t, "huge-pages")
@@ -57,6 +57,31 @@ func TestTheDefaultIsMadviseAndTheUnitFollowsTheSetting(t *testing.T) {
if h := apply["health"].(map[string]any); h["kind"] != "unit" {
t.Fatalf("health: %v", h)
}
// A build node, 2026-10-28: up 24 days on linux-lts 6.18.54 after an upgrade to 6.18.58 removed the running kernel's modules,
// so tun could not be loaded when a build first asked for it. tun is loaded at every boot, before an upgrade can take
// its file away, and the running kernel's modules are kept across an upgrade (kernel-modules-hook) and cleaned up at
// the next boot.
load := m.resource(t, "modules-load")
if load["path"] != "/etc/modules-load.d/mesh-kernel.conf" || load["mode"] != "0644" {
t.Fatalf("modules-load: %v", load)
}
var names []string
for _, l := range strings.Split(load.text(t), "\n") {
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "#") {
names = append(names, l)
}
}
if strings.Join(names, " ") != "tun" {
t.Fatalf("the modules loaded at boot: %v", names)
}
if keep := m.resource(t, "modules-keep"); keep["type"] != "package" || keep["package"] != "kernel-modules-hook" {
t.Fatalf("modules-keep: %v", keep)
}
// A oneshot at boot: enabled, and stopped once it ran, never started by an apply (it moves the modules of kernels no
// package owns away).
if c := m.resource(t, "modules-cleanup"); c["unit"] != "linux-modules-cleanup.service" || c["state"] != "stopped" || c["boot"] != "enabled" {
t.Fatalf("modules-cleanup: %v", c)
}
}
// bed is the script as the mesh writes it for one value of the setting, a stand-in transparent_hugepage
+4
View File
@@ -0,0 +1,4 @@
# Written by the mesh's kernel module (systemd-modules-load reads it at boot).
# tun: buildah's network for a RUN line (pasta) and every other user-space tunnel open /dev/net/tun.
# Loaded at boot, so an upgrade that removes the running kernel's modules cannot keep it out later.
tun
+19
View File
@@ -100,6 +100,25 @@
"id": "linux-lts-headers",
"type": "package",
"package": "linux-lts-headers"
},
{
"id": "modules-load",
"type": "file",
"path": "/etc/modules-load.d/mesh-kernel.conf",
"mode": "0644",
"template": "files/modules-load.conf"
},
{
"id": "modules-keep",
"type": "package",
"package": "kernel-modules-hook"
},
{
"id": "modules-cleanup",
"type": "service",
"unit": "linux-modules-cleanup.service",
"state": "stopped",
"boot": "enabled"
}
],
"build": {
+8 -1
View File
@@ -16,7 +16,14 @@ credential and an environment built from scratch.
The account's subordinate user and group ranges are not declared anywhere: the node-engine makes every `user` account
with `useradd`, which allocates them. The module's health reads `/etc/subuid` and `/etc/subgid` and is unhealthy
without a range for the account, without buildah, or when a user namespace cannot be made.
when the module cannot run: without a range for the account, without buildah, when a user namespace cannot be made,
or when the disk is under its floor. A node where `/dev/net/tun` cannot be opened stays **healthy** and is **not
eligible** (buildah makes a RUN line's network with pasta, which needs the tun driver; on 2026-10-28 a build node had
the device node and no driver, so every image build there failed with "Failed to open() /dev/net/tun: No such
device"): the seat's puller judges by a capacity of 0 with that reason (`capacity.<node>`, which `build_usage` shows),
so the asks go to a node that can build them, and its health says "not eligible" with the cause, so a node that
cannot build here now never holds the module's delivery for every node. The kernel module loads `tun` at boot; a look
that passes again wakes the puller.
The Dockerfile's steps may use the network (ADR 0487 §6); nothing a step can read holds a key.
@@ -59,7 +59,7 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
load = func(key string) ([]byte, error) { return m.ReadPauses.Get(key) }
}
return &buildstep.Puller{Source: source, Language: delivery.AskImage, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load,
Capacity: capacity, WatchConfig: watch, CoresPerStep: cores, BuildHours: hours, Logf: m.Say,
Capacity: capacity, WatchConfig: watch, CoresPerStep: cores, BuildHours: hours, Eligible: m.Eligible, Logf: m.Say,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
if err := m.Check(ctx); err != nil {
return nil, err
@@ -71,4 +71,17 @@ func TestAPulledImageAskCopiesThenBuilds(t *testing.T) {
t.Errorf("%s.%s is not served", Seat, verb)
}
}
// A node without /dev/net/tun (a build node, 2026-10-28) takes no ask: its capacity is 0 and says why, so the ask goes to a node
// that can build it instead of failing in buildah's network setup; once it is there again, the capacity is back.
tun := m.Tun
m.Tun = t.TempDir() + "/none"
_ = m.Check(context.Background())
if c := p.CapacityDetail(); c.Effective != 0 || !strings.Contains(c.Why, "not eligible: "+m.Tun+" cannot be opened") {
t.Errorf("without /dev/net/tun the capacity = %+v", c)
}
m.Tun = tun
_ = m.Check(context.Background())
if c := p.CapacityDetail(); c.Effective != 1 || c.Ineligible != "" {
t.Errorf("with /dev/net/tun again the capacity = %+v", c)
}
}
@@ -190,6 +190,7 @@ func moduleWith(t *testing.T, f *fake, states *keptStates) *Module {
SubUIDFile: write("subuid", "someone:100000:65536\nmesh-build-image:624288:65536\n"),
SubGIDFile: write("subgid", "mesh-build-image:624288:65536\n"),
RegistryFile: write("registry", f.address()+"\n"),
Tun: write("tun", ""),
RegistryWait: 2 * time.Second,
RegistryPause: 5 * time.Millisecond,
}
@@ -2,6 +2,7 @@ package main
import (
"context"
"errors"
"fmt"
"net/http"
"sync"
@@ -38,6 +39,8 @@ type Module struct {
Work, Store string
// SubUIDFile and SubGIDFile are the subordinate id files; /etc/subuid and /etc/subgid when empty.
SubUIDFile, SubGIDFile string
// Tun is the tun device buildah's network for a RUN line (pasta) opens; TunDevice when empty.
Tun string
// RegistryFile holds the registry's address, host:port, which the manifest binds from the artifact-store provision.
RegistryFile string
// RemoteBase is where repositories are fetched from, "ssh://mesh-forge/" when empty; FetchOptions are the options of
@@ -62,6 +65,10 @@ type Module struct {
spacesOnce sync.Once
workspaces *buildstep.Workspaces
// lastMu guards last: the refusal of the last look ("" when it could build), which Eligible answers.
lastMu sync.Mutex
last string
}
// Look is what the module found out about its machine.
@@ -90,15 +97,65 @@ func (m *Module) look(ctx context.Context) (Look, error) {
if err := m.userNamespace(ctx); err != nil {
return l, &buildstep.Refusal{Text: err.Error(), Have: l.Buildah}
}
if err := m.tunDevice(); err != nil {
return l, &notEligible{&buildstep.Refusal{Text: err.Error(), Have: l.Buildah}}
}
return l, nil
}
// Check is the guard: nil when the machine can build, else the refusal.
// notEligible is a refusal of a machine that runs the module fine but cannot build here now (no tun device: its asks go
// to another node). It is a *buildstep.Refusal for a step, and the health stays healthy for it (issue 1126): the
// assigner reads unhealthy as a failed delivery and holds the build for every node. What cannot run (buildah missing,
// no ranges, no namespace, no runtime directory, no room) stays unhealthy.
type notEligible struct{ *buildstep.Refusal }
func (e *notEligible) Unwrap() error { return e.Refusal }
// isNotEligible says whether err is a refusal that leaves the module healthy.
func isNotEligible(err error) bool {
var n *notEligible
return errors.As(err, &n)
}
// Check is the guard: nil when the machine can build, else the refusal. Every look is kept for Eligible, and a change
// wakes the puller, so a node that can build again takes asks at once.
func (m *Module) Check(ctx context.Context) error {
_, err := m.look(ctx)
m.keep(err)
return err
}
// keep records a look's outcome for Eligible and wakes the puller when it changed.
func (m *Module) keep(err error) {
why := ""
if err != nil {
why = err.Error()
}
m.lastMu.Lock()
changed := m.last != why
m.last = why
m.lastMu.Unlock()
if changed && m.Say != nil {
if why == "" {
m.Say("this machine is eligible to build images again")
} else {
m.Say("this machine takes no image build now: %s", why)
}
}
if changed && m.Held != nil {
m.Held.Wake()
}
}
// Eligible is the puller's eligibility (buildstep.Puller.Eligible): the refusal of the module's last look, "" when it
// could build. While the machine cannot build, the node's capacity is 0 and says why, so its asks go to a node that can
// build them instead of failing here (a build node without /dev/net/tun, 2026-10-28).
func (m *Module) Eligible() string {
m.lastMu.Lock()
defer m.lastMu.Unlock()
return m.last
}
// Start looks once, as the module starts, and then every CheckRound until ctx ends; it also removes old workspaces.
func (m *Module) Start(ctx context.Context) {
for {
@@ -119,20 +176,25 @@ func (m *Module) Start(ctx context.Context) {
// healthy it names the buildah installed and the account's ranges (ADR 0487 §2: the ranges are proven here, not declared).
func (m *Module) Health(ctx context.Context) health.Answer {
l, err := m.look(ctx)
if err != nil {
m.keep(err)
if err != nil && !isNotEligible(err) {
return health.Unhealthy(err.Error())
}
if err := m.room(); err != nil {
return health.Unhealthy(err.Error())
}
return health.Healthy(fmt.Sprintf("buildah %s installed; %s holds %d subordinate uids and %d gids; a user namespace can be made", l.Buildah, l.Account, l.UIDs, l.GIDs))
if err != nil {
// Not eligible: the module runs, it takes no image ask here now, and says why; the capacity record says it too.
return health.Healthy("not eligible to build images, so it takes no image ask: " + err.Error())
}
return health.Healthy(fmt.Sprintf("buildah %s installed; %s holds %d subordinate uids and %d gids; a user namespace can be made; %s opens", l.Buildah, l.Account, l.UIDs, l.GIDs, m.tunPath()))
}
// Status is what the module reports of itself.
func (m *Module) Status(ctx context.Context) map[string]any {
l, err := m.look(ctx)
out := map[string]any{"node": m.Node, "account": m.Account, "buildah": l.Buildah, "subuids": l.UIDs, "subgids": l.GIDs, "refusing": err != nil,
"cpu": m.cpuStatus()}
"eligible": err == nil, "cpu": m.cpuStatus()}
if err != nil {
out["why"] = err.Error()
}
@@ -2,7 +2,9 @@ package main
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
@@ -43,21 +45,88 @@ func TestHealthIsUnhealthyWithoutBuildah(t *testing.T) {
}
// The ranges are read from the two files, one each: without one in /etc/subuid or in /etc/subgid the account has no ids to
// map, and the health says which file.
func TestHealthIsUnhealthyWithoutARangeInEitherFile(t *testing.T) {
for file, set := range map[string]func(*Module, string){
"subuid": func(m *Module, p string) { m.SubUIDFile = p },
"subgid": func(m *Module, p string) { m.SubGIDFile = p },
// map, and the health says which file. And without /dev/net/tun (a build node on 2026-10-28: its running kernel's modules were
// gone, so tun could not be loaded) buildah's network for a RUN line cannot be made: the health says the node is not
// eligible, and the puller judges by 0 so its asks go to a node that can build them.
func TestHealthIsUnhealthyWithoutWhatABuildNeeds(t *testing.T) {
for _, c := range []struct {
name string
set func(*Module, string)
says []string
}{
{"subuid", func(m *Module, p string) { m.SubUIDFile = p }, []string{"/subuid", "mesh-build-image"}},
{"subgid", func(m *Module, p string) { m.SubGIDFile = p }, []string{"/subgid", "mesh-build-image"}},
} {
f := newFake(t)
m := moduleWith(t, f, newStates())
p := t.TempDir() + "/" + file
p := t.TempDir() + "/" + c.name
os.WriteFile(p, []byte("someone-else:100000:65536\n"), 0o644)
set(m, p)
c.set(m, p)
a := m.Health(context.Background())
if a.Healthy || !strings.Contains(a.Why, p) || !strings.Contains(a.Why, "mesh-build-image") {
t.Errorf("%s: health = %+v, want unhealthy naming %s and the account", file, a, p)
for _, want := range c.says {
if a.Healthy || !strings.Contains(a.Why, want) {
t.Errorf("%s: health = %+v, want unhealthy saying %q", c.name, a, want)
}
}
// Whatever the look refuses, the node is not eligible, with the same words.
if got := m.Eligible(); got != a.Why {
t.Errorf("%s: eligible says %q, the health %q", c.name, got, a.Why)
}
}
}
// 1126: a node that cannot build here now (no tun device) stays healthy, saying "not eligible" and why; the assigner reads
// unhealthy as a failed delivery and held the new build for every node. The eligibility, the status and the log say it too.
func TestANodeWithoutTunIsHealthyAndSaysItIsNotEligible(t *testing.T) {
f := newFake(t)
m := moduleWith(t, f, newStates())
var said []string
m.Say = func(format string, a ...any) { said = append(said, fmt.Sprintf(format, a...)) }
m.Tun = t.TempDir() + "/dev/net/tun"
a := m.Health(context.Background())
if a.State != health.StateHealthy || !a.Healthy {
t.Fatalf("health = %+v, want healthy", a)
}
for _, want := range []string{"not eligible", "/dev/net/tun cannot be opened", "tun"} {
if !strings.Contains(a.Why, want) {
t.Errorf("health says %q, lacks %q", a.Why, want)
}
}
if got := m.Eligible(); !strings.Contains(got, "not eligible: ") {
t.Errorf("eligible says %q", got)
}
if s := m.Status(context.Background()); s["eligible"] != false || s["refusing"] != true {
t.Errorf("status = %v, want not eligible", s)
}
// One log line for the change, none for the same look again; one when it can build again.
m.Health(context.Background())
if len(said) != 1 || !strings.Contains(said[0], "takes no image build now") {
t.Errorf("said = %q, want one line for the change", said)
}
m.Tun = filepath.Join(t.TempDir(), "tun")
os.WriteFile(m.Tun, nil, 0o644)
if a := m.Health(context.Background()); !a.Healthy || strings.Contains(a.Why, "not eligible") {
t.Errorf("health = %+v, want plain healthy", a)
}
if len(said) != 2 || !strings.Contains(said[1], "eligible to build images again") {
t.Errorf("said = %q, want the return said", said)
}
// A step still ends refused with the cause named.
m.Tun = t.TempDir() + "/none"
if err := m.Check(context.Background()); err == nil || !strings.Contains(err.Error(), "not eligible") {
t.Errorf("check = %v", err)
}
}
// 1126: what cannot run stays unhealthy even beside a missing tun: buildah missing is not "not eligible".
func TestWithoutBuildahTheNodeIsUnhealthyNotIneligible(t *testing.T) {
m := moduleWith(t, newFake(t), newStates())
m.Tun = t.TempDir() + "/none"
m.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
return forgekey.Ran{Status: 127, Err: "exec: \"buildah\": executable file not found in $PATH"}
}
if a := m.Health(context.Background()); a.Healthy {
t.Errorf("health = %+v, want unhealthy", a)
}
}
@@ -2,11 +2,13 @@ package main
// What the module proves about its own machine before it builds (novox/hq ADR 0487 §2, issue 1703a): buildah runs,
// the account holds subordinate user and group ranges (the node-engine's useradd allocates them for every `user`
// account, so nothing declares them; this reads them), and a user namespace can be made.
// account, so nothing declares them; this reads them), a user namespace can be made, and the tun device opens (buildah
// makes a RUN line's network with pasta, which needs it; a build node on 2026-10-28 had the device node and no tun driver).
import (
"bufio"
"context"
"errors"
"fmt"
"os"
"path/filepath"
@@ -18,6 +20,9 @@ import (
// ProbeTimeout bounds each probe: they read the machine and fetch nothing.
const ProbeTimeout = 20 * time.Second
// TunDevice is the tun device pasta opens for a RUN line's network.
const TunDevice = "/dev/net/tun"
// Default subordinate id files.
const (
SubUIDFile = "/etc/subuid"
@@ -133,3 +138,23 @@ func (m *Module) ensureRuntimeDir() error {
}
return nil
}
func (m *Module) tunPath() string {
if m.Tun != "" {
return m.Tun
}
return TunDevice
}
// tunDevice opens the tun device as the account does for a build, and says the node is not eligible when it cannot: a
// device node without its driver answers "no such device" (the tun kernel module is not loaded and cannot be, as when the
// running kernel's modules were removed by an upgrade), and every RUN line would fail in buildah's network setup.
func (m *Module) tunDevice() error {
f, err := os.OpenFile(m.tunPath(), os.O_RDWR, 0)
if err != nil {
return fmt.Errorf("not eligible: %s cannot be opened (%v), so buildah cannot make a RUN line's network (pasta): "+
"the tun kernel module is not loaded here (the kernel module loads it at boot; a machine whose running kernel's "+
"modules are gone needs a reboot)", m.tunPath(), errors.Unwrap(err))
}
return f.Close()
}
@@ -54,7 +54,7 @@ func (m *Module) Tools(steps []StepTool) []stdio.Tool {
Run: func(map[string]any) (any, error) { return m.Status(m.Ctx), nil }},
stdio.Tool{Name: HealthTool,
Description: "How the image build module is: unhealthy while buildah cannot be run, while its account holds no " +
"subordinate user or group range, or while a user namespace cannot be made, saying which; healthy otherwise. Read-only.",
"subordinate user or group range, or while a user namespace cannot be made, saying which; healthy otherwise. A machine that cannot build here now (no tun device) stays healthy and says in its words \"not eligible\", with the cause; it takes no image ask meanwhile (issue 1126). Read-only.",
Run: func(map[string]any) (any, error) { return m.Health(m.Ctx), nil }},
)
}
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildimage
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261027002948-db1007ffdda1
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261028115016-1618837ea9fe
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261027002948-db1007ffdda1 h1:9qyUcrvlklxDRBUCBqGH1L3oximOk554IfaPrTMgaTA=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261027002948-db1007ffdda1/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261028115016-1618837ea9fe h1:QDWi8k7zxaRPnJ+V/izoy6A19hZt1LTLl3uksZSlqOg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261028115016-1618837ea9fe/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
+1
View File
@@ -33,6 +33,7 @@ what it refuses.
| `reserve` | `kind` (`decision` or `research`), optionally `for` (one line: what the record is about) | the next number of that kind, spent forever; answers `kind` and `number` (and `prefix`, the number as the record's name starts with it); see below |
| `fix-times` | optionally `days` (1 to 90, default 7) | each class's fix budget, the median and worst fix time of the issues whose last fix was delivered in the window, within and over, and every issue over its budget with its longest phase; see *A fix's time* |
| `learn-fixes` | `number` | asks again for the fixes of one issue the tracker keeps no record of (a pull request merged before it heard deliveries): the delivery, then the forge, say whether each is merged (and the commit it landed as) or closed without merging; answers what each is kept as and the delivery line |
| `changes` | `since` (a window such as `60m`, at most `24h`), `number` (optional) | each issue's moves over the window, read from the tracker's own log `changes`: opened, moved, resolved or wontfix, the status it moved from and to, who moved it (and the worker, for diagnosing or fixing) and when, oldest first, at most 500; the window's first entry is found by its time, so a day's window reads only that day of the log. knowledge-base.progress with `since` asks it for each move's "from" |
**The cycle's rules, at the write** (hq playbook 03, which `00-META/checks/cycle.py` enforced on the
folders until the import, and hq ADR 0284): a status is one of `open`,
+2 -1
View File
@@ -23,7 +23,7 @@ const Seat = "issue-tracker"
// Verbs are the seat's verbs, as the manifest declares and claims them.
var Verbs = []string{"open", "get", "list", "move", "note", "link", "milestones", "milestone-set", "milestone-move", "import",
"groom", "set", "tracking", "reserve", "fix-times", "learn-fixes"}
"groom", "set", "tracking", "reserve", "fix-times", "learn-fixes", "changes"}
func logf(format string, a ...any) { fmt.Fprintf(os.Stderr, format+"\n", a...) }
@@ -943,5 +943,6 @@ func Tools(t *Tracker) []stdio.Tool {
}
return t.LearnFixes(n)
}},
changesTool(t),
}
}
@@ -72,6 +72,13 @@ func TestAnIssueRaceIntoTwoMilestonesOverTheBusEndsInOne(t *testing.T) {
if len(hist.History) != 1 {
t.Fatalf("history %d values", len(hist.History))
}
// The moves history over the real log: the open, from nothing to open, read by the other instance.
var ch ChangesRead
_ = json.Unmarshal(must(two.call("changes", map[string]any{"since": "60m"})), &ch)
if len(ch.Changes) != 1 || ch.Changes[0].Number != 365 || ch.Changes[0].Kind != OpOpened || ch.Changes[0].To != StatusOpen ||
ch.Changes[0].Title != "a milestone's issue" {
t.Fatalf("changes %+v", ch)
}
evMu.Lock()
defer evMu.Unlock()
var kinds []string
@@ -0,0 +1,222 @@
package main
// changes: each issue's moves over a window, read from the tracker's own record, its log `changes` (novox/hq ADR 0300),
// which holds every operation with who made it and when. The bucket `issues` keeps only what they add up to, so this
// is the one place an issue's earlier status is known: knowledge-base.progress with since asks it for the "from" of
// each move (the operator, 2026-10-28: the progress made in the last 60 minutes).
import (
"encoding/json"
"fmt"
"sort"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// MostChangesWindow is the longest window changes reads.
const MostChangesWindow = 24 * time.Hour
// MostChanges is the most changes one answer holds, the newest kept.
const MostChanges = 500
// changesSlack is how far before the window the log is read from: an entry's time is the bus's, an operation's the
// tracker's, and the two may differ by a little.
const changesSlack = time.Minute
// Change is one issue's status changing: opened, moved, resolved or wontfix, from what to what, by whom and when.
type Change struct {
Number uint64 `json:"number"`
Title string `json:"title,omitempty"`
Kind string `json:"kind"`
From string `json:"from"`
To string `json:"to"`
By string `json:"by"`
// Worker is who works on it, for a move to diagnosing or fixing: whom by names (the caller is By).
Worker string `json:"worker,omitempty"`
At time.Time `json:"at"`
}
// ChangesRead is the answer of changes.
type ChangesRead struct {
Since time.Time `json:"since"`
Changes []Change `json:"changes"`
// Entries is how many entries of the log were read for it.
Entries int `json:"entries-read"`
Said string `json:"said"`
}
// Changes reads the moves of the last window, of every issue or of one.
func (t *Tracker) Changes(window time.Duration, number uint64) (*ChangesRead, error) {
if window <= 0 || window > MostChangesWindow {
return nil, fmt.Errorf("since is a window more than nothing and at most 24h, not %s", window)
}
if t.Log == nil {
return nil, errNoLog
}
from := t.now().Add(-window)
out := &ChangesRead{Since: from, Changes: []Change{}}
var numbers []uint64
if number != 0 {
numbers = []uint64{number}
} else {
var err error
if numbers, err = t.movedSince(from, out); err != nil {
return nil, err
}
}
for _, n := range numbers {
entries, err := t.readKey(n, 1)
if err != nil {
return nil, err
}
if len(entries) == 0 {
if number != 0 {
return nil, fmt.Errorf("%d is not an issue the tracker holds", n)
}
continue
}
out.Entries += len(entries)
var cur *Issue
for _, e := range entries {
var op Operation
if err := json.Unmarshal(e.Value, &op); err != nil {
return nil, fmt.Errorf("the entry at %d is no operation this module reads: %v", e.Sequence, err)
}
next, err := fold(cur, &op, e.Sequence)
if err != nil {
return nil, fmt.Errorf("issue %d cannot be folded from its log: %v", n, err)
}
was := ""
if cur != nil {
was = cur.Status
}
cur = next
if op.Op == OpCarried || op.At.Before(from) || (was == next.Status && was != "") {
continue
}
kind := EventMoved
switch {
case was == "" && (op.Op == OpOpened || op.Op == OpImported):
kind = op.Op
case next.Status == StatusResolved || next.Status == StatusWontfix:
kind = next.Status
}
c := Change{Number: n, Kind: kind, From: was, To: next.Status, By: op.By, At: op.At}
if next.Worker != nil && (next.Status == StatusDiagnosing || next.Status == StatusFixing) {
c.Worker = next.Worker.By
}
out.Changes = append(out.Changes, c)
}
for i := range out.Changes {
if out.Changes[i].Number == n && cur != nil {
out.Changes[i].Title = cur.Title
}
}
}
sort.SliceStable(out.Changes, func(i, j int) bool {
if !out.Changes[i].At.Equal(out.Changes[j].At) {
return out.Changes[i].At.Before(out.Changes[j].At)
}
return out.Changes[i].Number < out.Changes[j].Number
})
cut := ""
if len(out.Changes) > MostChanges {
cut = fmt.Sprintf("; the oldest %d are left out", len(out.Changes)-MostChanges)
out.Changes = out.Changes[len(out.Changes)-MostChanges:]
}
out.Said = fmt.Sprintf("%d changes of status since %s, from the tracker's log (%d entries read)%s.",
len(out.Changes), from.Format(time.RFC3339), out.Entries, cut)
return out, nil
}
// movedSince is every issue an operation of which was written from the window on, found in the whole log by its
// entries' times: the first entry of the window is searched for, then the log is read on from it.
func (t *Tracker) movedSince(from time.Time, out *ChangesRead) ([]uint64, error) {
first, err := t.Log.Read(1, "", 1)
if err != nil {
return nil, fmt.Errorf("the log could not be read: %v", err)
}
if len(first.Entries) == 0 {
return nil, nil
}
lo, hi := first.Entries[0].Sequence, first.Last
start := from.Add(-changesSlack)
// The first sequence whose entry is at or after start: the entries are in the order they were written.
for lo < hi {
mid := lo + (hi-lo)/2
p, err := t.Log.Read(mid, "", 1)
if err != nil {
return nil, fmt.Errorf("the log could not be read at %d: %v", mid, err)
}
out.Entries++
if len(p.Entries) == 0 || !p.Entries[0].At.Before(start) {
hi = mid
} else {
lo = p.Entries[0].Sequence + 1
}
}
seen := map[uint64]bool{}
var numbers []uint64
for seq := lo; seq != 0; {
p, err := t.Log.Read(seq, "", logPage)
if err != nil {
return nil, fmt.Errorf("the log could not be read from %d: %v", seq, err)
}
out.Entries += len(p.Entries)
for _, e := range p.Entries {
n, ok := numberOfKey(e.Key)
if !ok || seen[n] || e.At.Before(start) || !movesStatus(e) {
continue
}
seen[n] = true
numbers = append(numbers, n)
}
seq = p.Next
}
return numbers, nil
}
// movesStatus says whether a log entry may change an issue's status: one that holds the issue whole, or sets it.
func movesStatus(e stdio.LogEntry) bool {
var op Operation
if json.Unmarshal(e.Value, &op) != nil {
return true
}
_, sets := op.Set["status"]
return sets || (op.Issue != nil && op.Op != OpCarried)
}
// changesTool is the verb, its arguments read as the other verbs read them.
func changesTool(t *Tracker) stdio.Tool {
return stdio.Tool{Name: Seat + ".changes",
Description: "Each issue's moves over a window, from the tracker's own log of every change: the issue moves history " +
"of the last minutes or hours (since, such as 60m, at most 24h), every issue's or one's (number), each change " +
"opened, moved, resolved or wontfix with the status it moved from and to, who moved it and when, oldest first. " +
"What the issues are now is list and get; this is what they were before. Refused: a window of nothing or over " +
"24h, a number that is not an issue.",
Input: schema([]string{"since"}, map[string]any{
"since": text("the window, back from now: 60m, 5m, 2h, at most 24h"),
"number": number("only this issue"),
}),
Run: func(m map[string]any) (any, error) {
a, err := readArgs("changes", m, "since", "number")
if err != nil {
return nil, err
}
s, err := a.str("since")
if err != nil {
return nil, err
}
d, err := time.ParseDuration(strings.TrimSpace(s))
if err != nil {
return nil, fmt.Errorf("since is a window such as 60m, 5m or 2h, not %q", s)
}
n, err := a.number("number", false)
if err != nil {
return nil, err
}
return t.Changes(d, n)
}}
}
@@ -0,0 +1,76 @@
package main
import (
"encoding/json"
"strings"
"testing"
"time"
)
// changes reads the tracker's own log over a window (the lead, 2026-10-28, for knowledge-base.progress since): each
// move from what to what, by whom and when, and each open, as the tracker's verbs wrote them into its log here.
func TestChangesSaysEachMoveFromWhatToWhatOverAWindow(t *testing.T) {
h := newHarness(t)
clock := time.Date(2026, 10, 28, 9, 0, 0, 0, time.UTC)
h.Tracker.Now = func() time.Time { return clock }
at := func(hh, mm int) { clock = time.Date(2026, 10, 28, hh, mm, 0, 0, time.UTC); h.log.clock = clock }
at(9, 0)
a := h.open(t, "an issue fixed in the window", "hq")
at(9, 10)
b := h.open(t, "an issue given up in the window")
at(10, 0)
if _, err := h.Move(MoveAsk{Number: a, Status: StatusDiagnosing, By: "a@g14"}); err != nil {
t.Fatal(err)
}
at(10, 30)
if _, err := h.Move(MoveAsk{Number: a, Status: StatusFixing, By: "a@g14"}); err != nil {
t.Fatal(err)
}
at(10, 40)
if _, err := h.Note(a, "a note moves nothing"); err != nil {
t.Fatal(err)
}
at(11, 0)
if _, err := h.Move(MoveAsk{Number: a, Status: StatusResolved, FixedBy: []string{"hq#1"}}); err != nil {
t.Fatal(err)
}
at(11, 5)
if _, err := h.Move(MoveAsk{Number: b, Status: StatusWontfix, Reason: "the module is retired"}); err != nil {
t.Fatal(err)
}
at(11, 30)
say := func(r *ChangesRead) string {
var lines []string
for _, c := range r.Changes {
lines = append(lines, strings.Join([]string{c.At.Format("15:04"), c.Kind, itoa(int(c.Number)), c.From + ">" + c.To, c.By + "/" + c.Worker}, " "))
}
return strings.Join(lines, "\n")
}
got, err := h.call(t, "changes", map[string]any{"since": "60m"})
if err != nil {
t.Fatal(err)
}
r := got.(*ChangesRead)
if want := "10:30 moved 365 diagnosing>fixing " + Unproven + "/a@g14\n11:00 resolved 365 fixing>resolved " + Unproven + "/\n11:05 wontfix 366 open>wontfix " + Unproven + "/"; say(r) != want {
t.Errorf("the last hour:\n%s\nwant\n%s", say(r), want)
}
if !r.Since.Equal(time.Date(2026, 10, 28, 10, 30, 0, 0, time.UTC)) || r.Changes[0].Title != "an issue fixed in the window" || !strings.Contains(r.Said, "3 changes") {
t.Errorf("since %v, title %q, said %q", r.Since, r.Changes[0].Title, r.Said)
}
got, err = h.call(t, "changes", map[string]any{"since": "3h", "number": b})
if err != nil {
t.Fatal(err)
}
if want := "09:10 opened 366 >open " + Unproven + "/\n11:05 wontfix 366 open>wontfix " + Unproven + "/"; say(got.(*ChangesRead)) != want {
t.Errorf("one issue:\n%s", say(got.(*ChangesRead)))
}
raw, _ := json.Marshal(got)
if !strings.Contains(string(raw), `"from":"open","to":"wontfix"`) {
t.Errorf("as JSON: %s", raw)
}
for _, bad := range []map[string]any{{"since": "25h"}, {"since": "0m"}, {}, {"since": "60m", "number": 9999}} {
if _, err := h.call(t, "changes", bad); err == nil {
t.Errorf("%v was answered", bad)
}
}
}
+4 -2
View File
@@ -22,7 +22,8 @@
"tracking",
"reserve",
"fix-times",
"learn-fixes"
"learn-fixes",
"changes"
]
}
],
@@ -46,7 +47,8 @@
"tracking",
"reserve",
"fix-times",
"learn-fixes"
"learn-fixes",
"changes"
]
}
],
+1 -1
View File
@@ -27,7 +27,7 @@ operator, it is accepted only on the operator's Approve, given on a channel that
| `import` | `dry-run`, `commit` | hq's decision records and research efforts at the commit the records module stands at, each whole at its own number |
| `return` | `number`, `reason` | drafted clauses sent back to be drafted again; refused for a record whose clauses another record changed or links by number |
| `withdraw` | `number`, `kind`, `why` | a reserved or proposed record withdrawn, as a new version naming why; its number stays spent. A pending review is cleared and a waiting operator's ask is ended. Refused for a record already accepted, superseded or withdrawn |
| `progress` | `of` (`mesh`, `design <n>`, `milestone <design>/<id>`, `issue <n>`, `decision <n>` or `ADR <n>`, `research <n>`), `detail` (`minimal` or `detailed`), `bound` | how far it has come (hq issue 1947): each child done ✓, moving ▶, waiting ⏸ or stuck ✗ with when it last moved, worked out from the issue tracker's rows and mesh-delivery's deliveries and groups, never written (ADR 0424 §1). `minimal` is one line per milestone, for a report every five minutes (two or three bus calls); `detailed` the report's tree with each issue's pull requests, the groups holding them and what it depends on. A design's title and state are read through the records module, cached by hq's commit; its milestones and their issues are the issue tracker's milestone records (`issue-tracker.milestones`, issue 1781), asked once per answer; a design with none is refused, naming the seed (`issue-tracker.import` with `milestones`) |
| `progress` | `of` (`mesh`, `design <n>`, `milestone <design>/<id>`, `issue <n>`, `decision <n>` or `ADR <n>`, `research <n>`), `detail` (`minimal` or `detailed`), `bound`, `since` (a window such as `60m`, at most `24h`) | how far it has come (hq issue 1947): each child done ✓, moving ▶, waiting ⏸ or stuck ✗ with when it last moved, worked out from the issue tracker's rows and mesh-delivery's deliveries and groups, never written (ADR 0424 §1). `minimal` is one line per milestone, for a report every five minutes (two or three bus calls); `detailed` the report's tree with each issue's pull requests, the groups holding them and what it depends on. A design's title and state are read through the records module, cached by hq's commit; its milestones and their issues are the issue tracker's milestone records (`issue-tracker.milestones`, issue 1781), asked once per answer; a design with none is refused, naming the seed (`issue-tracker.import` with `milestones`). With `since`, only what changed in the window under `of`, headed "N changes in the last 60 min": milestones moved (from what to what, from `issue-tracker.milestones` `one`), issues opened or moved from what to what (from `issue-tracker.changes`, the tracker's log), noted or linked (from `issue-tracker.get`), pull requests opened, checked, merged, held or closed (from `mesh-delivery.show`'s transitions and check). It keeps no state; its warnings say when the tracker cannot answer `changes` (a move is then said by the status it reached, not from what) and that mesh-delivery keeps no time a merge reached its machines |
`ask`, `get`, `holds` and `history` stay within `bound` bytes (4096 unless asked for, at most 65536) and say what they cut
and how to read the rest. The module's own tool `knowledge_health` answers its health (below).
@@ -34,6 +34,8 @@ type Milestone struct {
// Status is open or closed, with its Reason; a closed milestone holding no issue is done.
Status string
Reason string
// Moved is who last moved it and when, as the tracker says it; nil before its first move.
Moved *stamp
}
var (
@@ -1130,14 +1130,19 @@ func Tools(k *KB, caller func(map[string]any) string) []stdio.Tool {
"milestone, cheap enough for a report every five minutes; detailed gives the report's tree: each " +
"milestone not done with its issues, their pull requests, the groups holding them and what each waits " +
"for, for about every thirty. A design's milestones and their issues are the issue tracker's milestone " +
"records (its milestones); a design with none is refused, naming the seed. Within a bound it states, 4 KiB unless asked for more, saying what it cut.",
"records (its milestones); a design with none is refused, naming the seed. Within a bound it states, 4 KiB unless asked for more, saying what it cut. " +
"With since, a window such as 60m or 5m (at most 24h), it says only the progress made since then, such as in the last hour: " +
"the milestones and issues that moved (from what to what where the tracker keeps it), were opened, noted or resolved, " +
"the pull requests opened, checked, merged or held and the deliveries that reached their machines, and what became stuck, " +
"headed by how many changes in the last N minutes.",
Input: schema([]string{"of"}, map[string]any{
"of": text("what: mesh, design 47, milestone 47/R6, issue 1945, decision 493 (or ADR 0493) or research 72"),
"detail": map[string]any{"type": "string", "enum": []string{DetailMinimal, DetailDetailed},
"description": "minimal (the default): one line per child; detailed: the tree with issues, pull requests and groups"},
"bound": boundArg()}),
"bound": boundArg(),
"since": text("a window such as 60m, 5m or 2h, at most 24h: only what changed in it, from the tracker's and mesh-delivery's history")}),
Run: func(m map[string]any) (any, error) {
a, err := readArgs("progress", m, "of", "detail", "bound")
a, err := readArgs("progress", m, "of", "detail", "bound", "since")
if err != nil {
return nil, err
}
@@ -1151,6 +1156,9 @@ func Tools(k *KB, caller func(map[string]any) string) []stdio.Tool {
if p.Bound, err = a.num("bound", false); err != nil {
return nil, err
}
if p.Since, err = a.str("since"); err != nil {
return nil, err
}
return k.Progress(p)
}},
}
@@ -80,11 +80,11 @@ func TestItDeclaresAndHoldsTheKnowledgeBaseSeatOnceForTheMesh(t *testing.T) {
}
if !reflect.DeepEqual(m.Invokes, []string{"seat:issue-tracker.reserve", "claude-code.claude_code_judge", "claude-code.claude_code_judged", "records.records_status",
"records.records_list", "records.records_read", "seat:operator-channel.notify", "seat:issue-tracker.list", "seat:issue-tracker.get",
"seat:issue-tracker.milestones", "seat:mesh-delivery.deliveries", "seat:mesh-delivery.show", "seat:mesh-delivery.groups"}) {
"seat:issue-tracker.milestones", "seat:issue-tracker.changes", "seat:mesh-delivery.deliveries", "seat:mesh-delivery.show", "seat:mesh-delivery.groups"}) {
t.Fatalf("invokes %v", m.Invokes)
}
src := ""
for _, f := range []string{"main.go", "hq.go", "progress.go", "milestones.go"} {
for _, f := range []string{"main.go", "hq.go", "progress.go", "progress_since.go", "milestones.go"} {
src += readFile(t, f)
}
for _, inv := range m.Invokes {
@@ -7,6 +7,7 @@ package main
import (
"encoding/json"
"fmt"
"time"
)
// MilestonesVerb is the tracker's verb that answers the milestones.
@@ -20,6 +21,13 @@ type trackerMilestone struct {
Status string `json:"status"`
Reason string `json:"reason"`
Issues []int `json:"issues"`
Moved *stamp `json:"moved"`
}
// stamp is who did something and when, as the tracker says it.
type stamp struct {
By string `json:"by"`
At time.Time `json:"at"`
}
// withMilestones is each design with its milestones as the tracker holds them, in their table's order; design 0 asks
@@ -41,7 +49,7 @@ func (k *KB) withMilestones(ds []*Design, design int) ([]*Design, error) {
}
by := map[int][]Milestone{}
for _, m := range l.Milestones {
by[m.Design] = append(by[m.Design], Milestone{ID: m.ID, Outcome: m.Outcome, Issues: m.Issues, Status: m.Status, Reason: m.Reason})
by[m.Design] = append(by[m.Design], Milestone{ID: m.ID, Outcome: m.Outcome, Issues: m.Issues, Status: m.Status, Reason: m.Reason, Moved: m.Moved})
}
out := make([]*Design, 0, len(ds))
for _, d := range ds {
@@ -56,6 +56,8 @@ type ProgressAsk struct {
Of string
Detail string
Bound int
// Since is a window, such as 60m: the answer is then only what changed in it (progress_since.go).
Since string
}
// Item is one thing whose progress is said: a design, a milestone, an issue, a pull request.
@@ -81,6 +83,11 @@ type Progress struct {
Source string `json:"source"`
ReadAt time.Time `json:"read-at"`
Cut string `json:"cut,omitempty"`
// Since and Changes are said for an answer about a window: how long it is and how many changes it holds.
Since string `json:"since,omitempty"`
Changes int `json:"changes,omitempty"`
// scope is what the answer is about, kept for a window's answer; nil without since.
scope *sinceScope
}
// trackerRow is a row of `issue-tracker.list`.
@@ -137,9 +144,15 @@ func (k *KB) Progress(a ProgressAsk) (*Progress, error) {
if k.Seats == nil {
return nil, errors.New("the knowledge base reaches no other seat here")
}
window, err := parseSince(a.Since)
if err != nil {
return nil, err
}
of := strings.Join(strings.Fields(a.Of), " ")
p := &Progress{Of: of, Detail: detail, ReadAt: k.now()}
var err error
if window > 0 {
p.scope = &sinceScope{rows: map[int]trackerRow{}}
}
switch {
case strings.EqualFold(of, "mesh"):
err = k.progressMesh(p)
@@ -168,6 +181,9 @@ func (k *KB) Progress(a ProgressAsk) (*Progress, error) {
if err != nil {
return nil, err
}
if window > 0 {
k.progressSince(p, window)
}
// A cut takes the warnings from the last: the calls that failed are said before the answers that did not parse.
callsFirst(p.Warnings)
fitProgress(p, boundOf(a.Bound))
@@ -340,8 +356,9 @@ func (k *KB) progressDesign(p *Progress, number int, milestone string) error {
}
items, warn := milestoneItems(ms, rows, missing)
p.Warnings = append(p.Warnings, warn...)
p.scope.hold(rows, ms)
var extra *extras
if p.Detail == DetailDetailed {
if p.Detail == DetailDetailed && p.scope == nil {
if extra, err = k.readExtras(items); err != nil {
return err
}
@@ -394,6 +411,10 @@ func (k *KB) progressMesh(p *Progress) error {
for _, n := range missingList {
missing[n] = true
}
p.scope.hold(rows, with...)
if p.scope != nil {
p.scope.every = true
}
top := Item{ID: "mesh", Title: "the designs being built"}
var kids []Item
for _, d := range with {
@@ -441,6 +462,11 @@ func (k *KB) progressIssue(p *Progress, n int) error {
}
it.ID = fmt.Sprintf("issue %d", n)
p.Source = "the issue tracker and mesh-delivery"
if p.scope != nil {
p.scope.hold(rows)
p.State, p.Items = it.State, []Item{it}
return nil
}
extra, err := k.readExtras([]Item{{Children: []Item{{ID: strconv.Itoa(n), State: StateMoving}}}})
if err != nil {
return err
@@ -513,6 +539,9 @@ func (k *KB) progressRecord(p *Progress, kind string, n int) error {
p.Warnings = append(p.Warnings, fmt.Sprintf("%d issues are linked to %s; the first %d are counted", l.Matched, name, len(l.Rows)))
}
for _, r := range l.Rows {
if p.scope != nil {
p.scope.rows[r.Number] = r
}
if it, counts := issueItem(r); counts {
kids = append(kids, it)
}
@@ -58,12 +58,24 @@ func TestProgressOverTheBus(t *testing.T) {
t.Errorf("detailed: no %q in\n%s", want, p.Tree)
}
}
// A window over the bus: since is taken and the answer says how many changes it holds (none moved in the captured rows).
raw, err = b.call(Seat+".progress", map[string]any{"of": "milestone 47/R6", "since": "60m"})
if err != nil {
t.Fatal(err)
}
_ = json.Unmarshal(raw, &p)
if !strings.HasPrefix(p.Tree, "0 changes in the last 60 min under milestone 47/R6 (since ") || p.Since != "1h0m0s" {
t.Errorf("since over the bus:\n%s", p.Tree)
}
if _, err := b.call(Seat+".progress", map[string]any{"of": "design 47", "since": "2d"}); err == nil {
t.Error("a window of two days was taken")
}
if _, err := b.call(Seat+".progress", map[string]any{"of": "design 47", "by": "someone@else"}); err == nil {
t.Error("an argument naming the caller was taken")
}
seats.mu.Lock()
defer seats.mu.Unlock()
if seats.asked["seat:issue-tracker.list"] != 2 || seats.asked["seat:mesh-delivery.groups"] != 1 || seats.asked[MilestonesVerb] != 2 {
if seats.asked["seat:issue-tracker.list"] != 3 || seats.asked["seat:mesh-delivery.groups"] != 1 || seats.asked[MilestonesVerb] != 3 {
t.Fatalf("asked %v", seats.asked)
}
}
@@ -0,0 +1,498 @@
package main
// progress with since: only what changed in a window under of (the operator, 2026-10-28: "can the progress tool also
// report the progress made since the last call, or maybe better, the last 60 minutes"). A window, not a caller's last
// call: the knowledge base keeps no state of its own, so it reads what the other seats keep, each a request and reply
// over the bus:
//
// - the issue tracker's moves over the window, from its own log (`issue-tracker.changes`): each issue opened or
// moved, from what to what, by whom and when;
// - the issue tracker's milestones with their last move, and a moved one's history (`issue-tracker.milestones`
// with one): what it was before, what it is now, who moved it and when;
// - each issue that moved in the window whole (`issue-tracker.get`): when it was opened, its notes and links with who
// and when, its worker since when, and when it was closed;
// - mesh-delivery's deliveries and each one's transitions and its check (`mesh-delivery.deliveries`, `.show`).
//
// When the tracker cannot say its moves (a tracker without `changes`), a move is said by the status it reached, with its
// worker's since or its closed time, not from what it moved, and the answer says so. mesh-delivery keeps no time a
// merge reached its machines, so "on the machines" is as it reads now, said too.
import (
"encoding/json"
"fmt"
"sort"
"strconv"
"strings"
"time"
)
// MostSince is the longest window since asks: the deliveries mesh-delivery lists end a day back.
const MostSince = 24 * time.Hour
// parseSince reads a window such as 60m, 5m or 2h; empty is none.
func parseSince(s string) (time.Duration, error) {
s = strings.TrimSpace(s)
if s == "" {
return 0, nil
}
d, err := time.ParseDuration(s)
if err != nil || d <= 0 || d > MostSince {
return 0, fmt.Errorf("since is a window such as 60m, 5m or 2h, more than nothing and at most %s, not %q", "24h", s)
}
return d, nil
}
// sinceScope is what an answer about a window is about: the issues under of, as the tracker answered their rows, and
// the milestones; every says that of is the mesh, so every delivery counts.
type sinceScope struct {
rows map[int]trackerRow
milestones []sinceMilestone
every bool
}
type sinceMilestone struct {
design int
m Milestone
}
// hold keeps the rows and the designs' milestones for a window's answer; on a nil scope it does nothing.
func (s *sinceScope) hold(rows map[int]trackerRow, ds ...*Design) {
if s == nil {
return
}
for n, r := range rows {
s.rows[n] = r
}
for _, d := range ds {
for _, m := range d.Milestones {
s.milestones = append(s.milestones, sinceMilestone{design: d.Number, m: m})
}
}
}
// change is one thing that changed in the window, with its changes, newest last.
type change struct {
line string
last time.Time
events []event
}
type event struct {
at time.Time
line string
// brief is the event in the change's one line, when it differs from line.
brief string
}
// window is the time between from and to.
type window struct{ from, to time.Time }
func (w window) holds(t time.Time) bool { return !t.IsZero() && !t.Before(w.from) && !t.After(w.to) }
// progressSince replaces the answer's tree with what changed in the window under p.scope.
func (k *KB) progressSince(p *Progress, d time.Duration) {
w := window{from: p.ReadAt.Add(-d), to: p.ReadAt}
p.Since, p.Items = d.String(), nil
var milestones, issues, pulls []change
milestones = k.milestoneChanges(p, w)
issues = k.issueChanges(p, w)
pulls = k.pullChanges(p, w)
for _, cs := range [][]change{milestones, issues, pulls} {
sort.SliceStable(cs, func(i, j int) bool { return cs[i].last.After(cs[j].last) })
}
all := append(append(milestones, issues...), pulls...)
for _, c := range all {
p.Changes += len(c.events)
}
var b strings.Builder
noun := "changes"
if p.Changes == 1 {
noun = "change"
}
fmt.Fprintf(&b, "%d %s in the last %d min under %s (since %s)\n", p.Changes, noun, int(d.Minutes()), p.Of, when(&w.from))
for i, c := range all {
branch, under := "├── ", "│ "
if i == len(all)-1 {
branch, under = "└── ", " "
}
b.WriteString(branch + c.line + "\n")
if p.Detail != DetailDetailed {
continue
}
for j, e := range c.events {
br := "├── "
if j == len(c.events)-1 {
br = "└── "
}
b.WriteString(under + br + e.line + "\n")
}
}
p.Tree = strings.TrimRight(b.String(), "\n")
p.Source += "; the window from the tracker's milestones and their history, its issues' notes, links, workers and " +
"closes, and mesh-delivery's transitions and checks"
}
// milestoneChanges is each milestone moved in the window, from its history: what it was and what it is.
func (k *KB) milestoneChanges(p *Progress, w window) []change {
var out []change
asked := 0
for _, sm := range p.scope.milestones {
m := sm.m
if m.Moved == nil || !w.holds(m.Moved.At) {
continue
}
ref := fmt.Sprintf("%d/%s", sm.design, m.ID)
now := m.Status
if m.Reason != "" {
now += ": " + m.Reason
}
was := "a state its history does not say"
if asked < MostGets {
asked++
if prev, err := k.milestoneBefore(ref); err != nil {
p.Warnings = append(p.Warnings, fmt.Sprintf("issue-tracker.milestones one %s: %v; said without what it was", ref, err))
} else if prev != "" {
was = prev
}
}
mark := marks[StateMoving]
if m.Status == "closed" {
mark = marks[StateDone]
}
line := fmt.Sprintf("%s milestone %s %s → %s · by %s %s", mark, ref, was, now, m.Moved.By, when(&m.Moved.At))
out = append(out, change{line: line, last: m.Moved.At, events: []event{{at: m.Moved.At, line: fmt.Sprintf("moved %s by %s", when(&m.Moved.At), m.Moved.By)}}})
}
return out
}
// milestoneBefore is a milestone's status before its last move, from the tracker's history of it.
func (k *KB) milestoneBefore(ref string) (string, error) {
raw, err := k.Seats(MilestonesVerb, map[string]any{"one": ref})
if err != nil {
return "", err
}
var r struct {
History []struct {
Status string `json:"status"`
} `json:"history"`
}
if err := json.Unmarshal(raw, &r); err != nil {
return "", fmt.Errorf("answered something else: %v", err)
}
if len(r.History) < 2 {
return "", nil
}
return r.History[len(r.History)-2].Status, nil
}
// issueGotWhole is the part of `issue-tracker.get` a window reads.
type issueGotWhole struct {
Status string `json:"status"`
Opened time.Time `json:"opened"`
OpenedBy string `json:"opened-by"`
Closed *time.Time `json:"closed"`
FixedBy []string `json:"fixed-by"`
Worker *struct {
By string `json:"by"`
Since time.Time `json:"since"`
} `json:"worker"`
Notes []struct {
At time.Time `json:"at"`
By string `json:"by"`
Text string `json:"text"`
} `json:"notes"`
Links []struct {
At time.Time `json:"at"`
By string `json:"by"`
Kind string `json:"kind"`
Target string `json:"target"`
} `json:"links"`
}
// issueChanges is each issue under of that moved in the window, read whole: opened, moved, noted, linked, closed.
func (k *KB) issueChanges(p *Progress, w window) []change {
var moved []trackerRow
for _, r := range p.scope.rows {
if r.Moved != nil && w.holds(*r.Moved) {
moved = append(moved, r)
}
}
sort.Slice(moved, func(i, j int) bool { return moved[i].Moved.After(*moved[j].Moved) })
var moves map[int][]trackerMove
var merr error
if len(moved) > 0 {
if moves, merr = k.trackerMoves(w); merr != nil {
p.Warnings = append(p.Warnings, fmt.Sprintf("issue-tracker.changes: %v; a move is said by the status it reached, "+
"its worker's since and its closed time, not from what it moved", merr))
}
}
var out []change
for i, r := range moved {
it, _ := issueItem(r)
if it.State == "" {
it.State = StateDone
}
head := fmt.Sprintf("%s %d %s — ", marks[it.State], r.Number, clip(r.Title, 80))
fallback := func(why string) {
p.Warnings = append(p.Warnings, why)
line := fmt.Sprintf("moved %s, now %s", when(r.Moved), r.Status)
if r.Worker != "" {
line += " (" + r.Worker + ")"
}
out = append(out, change{line: head + line, last: *r.Moved, events: []event{{at: *r.Moved, line: line}}})
}
if i >= MostGets {
fallback(fmt.Sprintf("more than %d issues moved; the rest are said by their rows' moved time", MostGets))
continue
}
raw, err := k.Seats("seat:issue-tracker.get", map[string]any{"number": r.Number})
if err != nil {
fallback(fmt.Sprintf("issue-tracker.get %d: %v; said by its row's moved time", r.Number, err))
continue
}
var g issueGotWhole
if err := json.Unmarshal(raw, &g); err != nil {
fallback(fmt.Sprintf("issue-tracker.get %d answered something else: %v; said by its row's moved time", r.Number, err))
continue
}
var evs []event
var says []string
if merr == nil {
for _, m := range moves[r.Number] {
if m.Kind == "opened" || m.Kind == "imported" {
evs = append(evs, event{at: m.At, line: fmt.Sprintf("%s %s by %s", m.Kind, when(&m.At), m.By)})
says = append(says, m.Kind+" by "+m.By+" "+when(&m.At))
continue
}
who, line := m.By, fmt.Sprintf("%s → %s %s by %s", m.From, m.To, when(&m.At), m.By)
if m.Worker != "" {
who, line = m.Worker, line+", worker "+m.Worker
}
short := fmt.Sprintf("%s → %s by %s %s", m.From, m.To, who, when(&m.At))
if (m.To == "resolved" || m.To == "wontfix") && len(g.FixedBy) > 0 {
short += ", fixed by " + strings.Join(g.FixedBy, ", ")
}
evs = append(evs, event{at: m.At, line: line})
says = append(says, short)
}
}
if merr != nil && w.holds(g.Opened) {
evs = append(evs, event{at: g.Opened, line: fmt.Sprintf("opened %s by %s", when(&g.Opened), g.OpenedBy)})
says = append(says, "opened by "+g.OpenedBy+" "+when(&g.Opened))
}
if merr != nil && g.Worker != nil && w.holds(g.Worker.Since) && (g.Status == "diagnosing" || g.Status == "fixing") {
s := fmt.Sprintf("→ %s by %s %s", g.Status, g.Worker.By, when(&g.Worker.Since))
evs = append(evs, event{at: g.Worker.Since, line: s})
says = append(says, s)
}
if merr != nil && g.Closed != nil && w.holds(*g.Closed) {
s := fmt.Sprintf("→ %s %s", g.Status, when(g.Closed))
if len(g.FixedBy) > 0 {
s += ", fixed by " + strings.Join(g.FixedBy, ", ")
}
evs = append(evs, event{at: *g.Closed, line: s})
says = append(says, s)
}
notes := 0
var last string
for _, n := range g.Notes {
if w.holds(n.At) {
notes++
last = fmt.Sprintf("the last %s by %s", when(&n.At), n.By)
evs = append(evs, event{at: n.At, line: fmt.Sprintf("noted %s by %s: %s", when(&n.At), n.By, clip(n.Text, 80))})
}
}
if notes == 1 {
says = append(says, "1 note, "+last)
} else if notes > 1 {
says = append(says, fmt.Sprintf("%d notes, %s", notes, last))
}
links := 0
for _, l := range g.Links {
if w.holds(l.At) {
links++
evs = append(evs, event{at: l.At, line: fmt.Sprintf("linked %s %s by %s %s", l.Kind, l.Target, l.By, when(&l.At))})
}
}
if links > 0 {
says = append(says, fmt.Sprintf("%d links", links))
}
if len(evs) == 0 {
// It moved by what the tracker records of nobody: its fix's delivery said a new state.
s := fmt.Sprintf("moved %s, now %s", when(r.Moved), it.Why)
evs, says = []event{{at: *r.Moved, line: s}}, []string{s}
}
if it.State == StateStuck {
says = append(says, "stuck: "+it.Why)
}
sort.SliceStable(evs, func(i, j int) bool { return evs[i].at.Before(evs[j].at) })
out = append(out, change{line: head + strings.Join(says, "; "), last: *r.Moved, events: evs})
}
return out
}
// deliveryWhole is the part of `mesh-delivery.show` a window reads.
type deliveryWhole struct {
Delivery struct {
Repository string `json:"repository"`
Number int `json:"number"`
Issues []int `json:"issues"`
State string `json:"state"`
Check *struct {
At time.Time `json:"at"`
Gate string `json:"gate"`
Summary string `json:"summary"`
} `json:"check"`
Transitions []struct {
At time.Time `json:"at"`
Event string `json:"event"`
To string `json:"to"`
Why string `json:"why"`
} `json:"transitions"`
} `json:"delivery"`
Machines string `json:"machines"`
Waits string `json:"waits"`
}
// pullChanges is each pull request naming an issue under of (every one, for the mesh) that was opened, checked,
// merged, held or closed in the window.
func (k *KB) pullChanges(p *Progress, w window) []change {
raw, err := k.Seats("seat:mesh-delivery.deliveries", map[string]any{})
if err != nil {
p.Warnings = append(p.Warnings, fmt.Sprintf("mesh-delivery does not answer, so no pull request is said: %v", err))
return nil
}
var dl struct {
Deliveries []deliveryRow `json:"deliveries"`
}
if err := json.Unmarshal(raw, &dl); err != nil {
p.Warnings = append(p.Warnings, fmt.Sprintf("mesh-delivery.deliveries answered something else, so no pull request is said: %v", err))
return nil
}
var out []change
shown, reached := 0, false
for _, d := range dl.Deliveries {
// A delivery's since is its last transition; one not final may still have been checked since.
if final(d.State) && !w.holds(d.Since) {
continue
}
if shown == MostShows {
p.Warnings = append(p.Warnings, fmt.Sprintf("more than %d deliveries moved; the rest are not read", MostShows))
break
}
shown++
raw, err := k.Seats("seat:mesh-delivery.show", map[string]any{"id": d.ID})
if err != nil {
p.Warnings = append(p.Warnings, fmt.Sprintf("mesh-delivery.show %s: %v", d.ID, err))
continue
}
var s deliveryWhole
if err := json.Unmarshal(raw, &s); err != nil {
p.Warnings = append(p.Warnings, fmt.Sprintf("mesh-delivery.show %s answered something else: %v", d.ID, err))
continue
}
under := p.scope.every
var names []string
for _, n := range s.Delivery.Issues {
names = append(names, strconv.Itoa(n))
if _, ok := p.scope.rows[n]; ok {
under = true
}
}
if !under {
continue
}
var evs []event
for _, t := range s.Delivery.Transitions {
if !w.holds(t.At) {
continue
}
switch {
case t.Event == "announced":
evs = append(evs, event{at: t.At, line: "opened " + when(&t.At)})
case t.To == "merged" || t.To == "closed" || strings.HasPrefix(t.To, "held") || t.To == "delivered":
evs = append(evs, event{t.At, fmt.Sprintf("%s %s: %s", t.To, when(&t.At), clip(t.Why, 80)), t.To + " " + when(&t.At)})
}
}
if c := s.Delivery.Check; c != nil && w.holds(c.At) {
evs = append(evs, event{c.At, fmt.Sprintf("check %s %s: %s", c.Gate, when(&c.At), clip(c.Summary, 80)), fmt.Sprintf("check %s %s", c.Gate, when(&c.At))})
}
if len(evs) == 0 {
continue
}
sort.SliceStable(evs, func(i, j int) bool { return evs[i].at.Before(evs[j].at) })
var says []string
for _, e := range evs {
if e.brief != "" {
says = append(says, e.brief)
} else {
says = append(says, e.line)
}
}
state := s.Delivery.State
mark := marks[StateMoving]
switch {
case state == "merged" || state == "delivered":
mark = marks[StateDone]
case strings.HasPrefix(state, "held"):
mark = marks[StateStuck]
case state == "closed":
mark = marks[StateWaiting]
}
repo := s.Delivery.Repository
if i := strings.LastIndex(repo, "/"); i >= 0 {
repo = repo[i+1:]
}
line := fmt.Sprintf("%s %s#%d (%s) %s", mark, repo, s.Delivery.Number, strings.Join(names, ", "), strings.Join(says, " · "))
switch {
case state == "merged" && s.Machines != "":
line += ", " + s.Machines
reached = true
case !final(state):
line += ", now " + state
if s.Waits != "" {
line += ", waits for " + s.Waits
}
}
out = append(out, change{line: line, last: evs[len(evs)-1].at, events: evs})
}
if reached {
p.Warnings = append(p.Warnings, "mesh-delivery keeps no time a merge reached its machines: \"on the machines\" is as it reads now")
}
return out
}
// trackerMove is one change of `issue-tracker.changes`: an issue opened or moved, from what to what, by whom and when.
type trackerMove struct {
Number int `json:"number"`
Kind string `json:"kind"`
From string `json:"from"`
To string `json:"to"`
By string `json:"by"`
Worker string `json:"worker"`
At time.Time `json:"at"`
}
// trackerMoves asks the tracker for every issue's moves in the window, from its own log, by issue, oldest first.
func (k *KB) trackerMoves(w window) (map[int][]trackerMove, error) {
raw, err := k.Seats(ChangesVerb, map[string]any{"since": w.to.Sub(w.from).String()})
if err != nil {
return nil, err
}
var r struct {
Changes []trackerMove `json:"changes"`
}
if err := json.Unmarshal(raw, &r); err != nil {
return nil, fmt.Errorf("answered something else: %v", err)
}
out := map[int][]trackerMove{}
for _, m := range r.Changes {
if w.holds(m.At) {
out[m.Number] = append(out[m.Number], m)
}
}
return out, nil
}
// ChangesVerb is the tracker's verb that answers its issues' moves over a window.
const ChangesVerb = "seat:issue-tracker.changes"
@@ -2,6 +2,8 @@ package main
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"reflect"
@@ -393,3 +395,150 @@ func TestProgressOfAnIssueAndARecord(t *testing.T) {
t.Error("an issue the tracker does not hold was answered")
}
}
// sinceWorld plays the tracker and the delivery seat from what they answered on 2026-10-28 (testdata/progress/since):
// a get or a show it holds no answer for is refused, as a seat that cannot answer. The tracker's changes answers what
// mesh-issues' verb made of 1422's open and move to fixing (testdata/progress/README.md); without changes it is
// refused, as by a tracker that has no such verb yet.
func sinceWorld(t *testing.T, changes bool) func(string, any) (json.RawMessage, error) {
t.Helper()
read := func(name string) json.RawMessage { return readFixture(t, filepath.Join("since", name)) }
var rows []json.RawMessage
if err := json.Unmarshal(read("rows.json"), &rows); err != nil {
t.Fatal(err)
}
byNumber := map[int]json.RawMessage{}
for _, r := range rows {
var n struct{ Number int }
_ = json.Unmarshal(r, &n)
byNumber[n.Number] = r
}
shows := map[string]string{"novox/mesh-catalog@470687ea4cce": "show-mesh-catalog-539.json", "novox/mesh-catalog@f5d4b2f86ed4": "show-mesh-catalog-537.json",
"novox/mesh-delivery@2b801f5bd49a": "show-mesh-delivery-94.json", "novox/de-spiegel@fba6dbe3259e": "show-de-spiegel-6.json"}
return func(key string, body any) (json.RawMessage, error) {
raw, _ := json.Marshal(body)
var a struct {
Numbers []int `json:"numbers"`
Number int `json:"number"`
ID string `json:"id"`
One string `json:"one"`
}
_ = json.Unmarshal(raw, &a)
switch key {
case MilestonesVerb:
return read("milestone-47-6.json"), nil
case "seat:issue-tracker.list":
var out []json.RawMessage
for _, n := range a.Numbers {
if r, ok := byNumber[n]; ok {
out = append(out, r)
}
}
return json.Marshal(map[string]any{"rows": out, "matched": len(out)})
case "seat:issue-tracker.changes":
if changes {
return read("changes.json"), nil
}
return nil, errors.New("mesh-issues has no tool issue-tracker.changes")
case "seat:issue-tracker.get":
if a.Number == 1422 || a.Number == 1956 {
return read(fmt.Sprintf("get-%d.json", a.Number)), nil
}
case "seat:mesh-delivery.deliveries":
return read("deliveries.json"), nil
case "seat:mesh-delivery.groups":
return json.RawMessage(`[]`), nil
case "seat:mesh-delivery.show":
if f, ok := shows[a.ID]; ok {
return read(f), nil
}
}
return nil, os.ErrNotExist
}
}
// The operator, 2026-10-28: "can the progress tool also report the progress made since the last call, or maybe better,
// the last 60 minutes". With since, an answer lists only what changed in that window under of, read from the
// tracker's notes, links, workers and closes, its milestones' history and mesh-delivery's transitions and checks.
func TestProgressSinceAWindowSaysOnlyWhatChanged(t *testing.T) {
at := func(s string) *time.Time { v, _ := time.Parse(time.RFC3339, s); return &v }
k := &KB{Records: progressRecords(), Seats: sinceWorld(t, true), Now: func() time.Time { return *at("2026-10-28T11:13:30Z") }}
p, err := k.Progress(ProgressAsk{Of: "issue 1422", Since: "2h"})
if err != nil {
t.Fatal(err)
}
lines := strings.Split(p.Tree, "\n")
if lines[0] != "17 changes in the last 120 min under issue 1422 (since "+when(at("2026-10-28T09:13:30Z"))+")" || p.Changes != 17 || p.Since != "2h0m0s" {
t.Fatalf("%d changes, since %q:\n%s", p.Changes, p.Since, p.Tree)
}
for _, want := range []string{
"▶ 1422 " + clip("The catch-up reads the trunk 'main' of repositories the catalogue follows by another branch or by a commit, and the forge holds none", 80) + " — open → fixing by fix-trunk-unknown@shanks " + when(at("2026-10-28T09:38:37Z")) +
"; 9 notes, the last " + when(at("2026-10-28T11:05:21Z")) + " by shanks.node-tools",
"✓ mesh-catalog#537 (1422) opened " + when(at("2026-10-28T10:20:56Z")) + " · check pass " + when(at("2026-10-28T10:30:45Z")) + " · merged " + when(at("2026-10-28T10:34:49Z")) + ", on the machines",
"✓ mesh-delivery#94 (1126, 1422) opened ",
"▶ mesh-catalog#539 (1422, 1956) opened " + when(at("2026-10-28T11:07:42Z")) + ", now ready, waits for its merge",
} {
if !strings.Contains(p.Tree, want) {
t.Errorf("no %q in\n%s", want, p.Tree)
}
}
if strings.Contains(p.Tree, "de-spiegel") || strings.Contains(p.Tree, "triage") {
t.Errorf("a change outside issue 1422 or the window is said:\n%s", p.Tree)
}
w := strings.Join(p.Warnings, "\n")
if !strings.Contains(w, "keeps no time a merge reached its machines") || strings.Contains(w, "issue-tracker.changes") {
t.Errorf("the warnings: %v", p.Warnings)
}
// A tracker that cannot say its moves: each is said by the status it reached, and that is said.
old := &KB{Records: progressRecords(), Seats: sinceWorld(t, false), Now: k.Now}
o, err := old.Progress(ProgressAsk{Of: "issue 1422", Since: "2h"})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(o.Tree, " — → fixing by fix-trunk-unknown@shanks "+when(at("2026-10-28T09:38:37Z"))) || o.Changes != 17 ||
!strings.Contains(strings.Join(o.Warnings, "\n"), "issue-tracker.changes: mesh-issues has no tool issue-tracker.changes; a move is said by the status it reached") {
t.Errorf("without changes:\n%s\n%v", o.Tree, o.Warnings)
}
d, err := k.Progress(ProgressAsk{Of: "issue 1422", Since: "2h", Detail: DetailDetailed})
if err != nil {
t.Fatal(err)
}
for _, want := range []string{"│ ├── open → fixing " + when(at("2026-10-28T09:38:37Z")) + " by shanks.node-tools, worker fix-trunk-unknown@shanks",
"│ ├── noted " + when(at("2026-10-28T09:38:09Z")) + " by shanks.node-tools: lead-47@shanks 2026-10-28 09:40Z.",
" └── merged " + when(at("2026-10-28T10:34:49Z")) + ": on the trunk its modules follow: merged there"} {
if !strings.Contains(d.Tree, want) {
t.Errorf("detailed: no %q in\n%s", want, d.Tree)
}
}
// A milestone's issue the tracker's get cannot answer falls back to its row's moved time, said as such.
m, err := k.Progress(ProgressAsk{Of: "milestone 47/L2", Since: "60m"})
if err != nil {
t.Fatal(err)
}
for _, want := range []string{"6 changes in the last 60 min under milestone 47/L2", "▶ 1126 " + clip("About 49,800 controller code lines stay after the delivery model, against a first limit of 25,000", 80) + " — moved " +
when(at("2026-10-28T11:13:28Z")) + ", now fixing (lead-47@shanks)", "✓ de-spiegel#6 (1126) check pass ", "✓ mesh-delivery#94 (1126, 1422) opened "} {
if !strings.Contains(m.Tree, want) {
t.Errorf("milestone: no %q in\n%s", want, m.Tree)
}
}
if w := strings.Join(m.Warnings, "\n"); !strings.Contains(w, "issue-tracker.get 1126") {
t.Errorf("the fallback is not said: %v", m.Warnings)
}
if small, _ := k.Progress(ProgressAsk{Of: "issue 1422", Since: "2h", Detail: DetailDetailed, Bound: LeastBound}); small == nil || size(small) > LeastBound || !strings.Contains(small.Cut, "raise bound") {
t.Errorf("a window's answer is not kept within its bound: %+v", small)
}
// A milestone's move, from its history: what it was before, what it is now, who moved it and when.
k.Now = func() time.Time { return *at("2026-10-26T18:40:00Z") }
ds, err := k.Progress(ProgressAsk{Of: "design 47", Since: "60m"})
if err != nil {
t.Fatal(err)
}
if ds.Tree != "1 change in the last 60 min under design 47 (since "+when(at("2026-10-26T17:40:00Z"))+")\n└── ✓ milestone 47/6 open → closed: folded into R4g · by shanks.node-tools "+when(at("2026-10-26T18:10:10Z")) {
t.Errorf("design:\n%s", ds.Tree)
}
for _, bad := range []string{"0m", "-5m", "25h", "a day", "5"} {
if _, err := k.Progress(ProgressAsk{Of: "issue 1422", Since: bad}); err == nil || !strings.Contains(err.Error(), "since") {
t.Errorf("since %q: %v", bad, err)
}
}
}
@@ -8,3 +8,12 @@ Captured for the progress verb's tests (novox/hq issue 1947), never written by h
- `tracker-milestones-47.json`: the answer of `issue-tracker.milestones` for design 47 (issue 1781), made by running
the tracker's own table parse (mesh-issues `parseTable`, its seed) over `hq/`'s design 47, each issue two rows name
left in neither, as the seed refuses it; the tracker held no milestone records yet to capture from.
- `since/`: answers of the live tracker and delivery seat on 2026-10-28 between 11:10Z and 11:20Z, for `since`
(issue 1422 and its pull requests): `issue-tracker.list` rows, `issue-tracker.get` 1422 and 1956 (the report,
the triage and each note's text after its first sentence or two cut), `mesh-delivery.deliveries` (its first ten
rows) and `.show` of four deliveries (the ready record, the ask and the paths cut), and `issue-tracker.milestones`
with `one` 47/6, its milestones joined by 47/L2's row from `one` 47/L2.
- `since/changes.json`: the answer of mesh-issues' `issue-tracker.changes` (window 2h at 2026-10-28 11:13:30Z) made by
the tracker's own code over its log, replaying issue 1422's open (10-20 14:36:07Z) and its move to fixing by
fix-trunk-unknown@shanks (10-28 09:38:37Z) through its verbs: the live verb was not delivered yet to capture from, and
the status 1422 moved from (open) is the replay's, as no verb read the live log then. Recapture once it is delivered.
@@ -0,0 +1,17 @@
{
"since": "2026-10-28T09:13:30Z",
"changes": [
{
"number": 1422,
"title": "The catch-up reads the trunk 'main' of repositories the catalogue follows by another branch or by a commit, and the forge holds none",
"kind": "moved",
"from": "open",
"to": "fixing",
"by": "shanks.node-tools",
"worker": "fix-trunk-unknown@shanks",
"at": "2026-10-28T09:38:37Z"
}
],
"entries-read": 4,
"said": "1 changes of status since 2026-10-28T09:13:30Z, from the tracker's log (4 entries read)."
}
@@ -0,0 +1,12 @@
{"deliveries": [
{"for": "4m8s", "id": "novox/mesh-catalog@470687ea4cce", "number": 539, "since": "2026-10-28T11:07:43Z", "state": "ready", "waits": "its merge"},
{"for": "14m17s", "id": "novox/mesh-catalog@fc5335360967", "number": 538, "since": "2026-10-28T10:57:33Z", "state": "closed"},
{"for": "26m23s", "id": "novox/de-spiegel@fba6dbe3259e", "machines": "on the machines", "number": 6, "since": "2026-10-28T10:45:28Z", "state": "merged"},
{"for": "36m56s", "group": "fix/1422-trunk-unknown", "id": "novox/mesh-delivery@2b801f5bd49a", "machines": "on the machines", "number": 94, "since": "2026-10-28T10:34:54Z", "state": "merged"},
{"for": "37m2s", "group": "fix/1422-trunk-unknown", "id": "novox/mesh-catalog@f5d4b2f86ed4", "machines": "on the machines", "number": 537, "since": "2026-10-28T10:34:49Z", "state": "merged"},
{"for": "56m30s", "id": "novox/hq@dd36b7557ae6", "machines": "on the machines", "number": 489, "since": "2026-10-28T10:15:20Z", "state": "merged"},
{"for": "1h3m4s", "id": "novox/mesh-host@6502ba6ad6e6", "machines": "on the machines", "number": 159, "since": "2026-10-28T10:08:47Z", "state": "merged"},
{"for": "1h24m56s", "group": "fix/1126-seed-same", "id": "novox/mesh-host@c912bcab6e73", "machines": "on the machines", "number": 158, "since": "2026-10-28T09:46:55Z", "state": "merged"},
{"for": "2h51m31s", "id": "novox/mesh-catalog@c2606f99178a", "number": 534, "since": "2026-10-28T08:20:20Z", "state": "closed"},
{"for": "3h25m11s", "group": "fix/1126-shadow-bodies", "id": "novox/mesh-host@82d0e125b2c0", "machines": "on the machines", "number": 157, "since": "2026-10-28T07:46:40Z", "state": "merged"}
]}
@@ -0,0 +1,31 @@
{
"fixing-from": "2026-10-28T09:38:37Z",
"links": [
{"at": "2026-10-20T14:37:00Z", "by": "triage", "kind": "relates-to", "target": "1421"},
{"at": "2026-10-20T14:37:00Z", "by": "triage", "kind": "relates-to", "target": "1420"}
],
"located-in": ["mesh-controller cmd/mesh-controller/catch_up.go"],
"needs": "none",
"notes": [
{"at": "2026-10-20T14:37:00Z", "by": "triage", "text": "Triage of this issue: none of 429 older issues not resolved (the 429 most recent of 493; the titles of the rest did not fit the question) asks the same question."},
{"at": "2026-10-20T14:44:57Z", "by": "triage", "text": "Triage: issue 1424 relates to this one, opened later."},
{"at": "2026-10-20T14:45:10Z", "by": "triage", "text": "Triage: issue 1425 relates to this one, opened later."},
{"at": "2026-10-28T09:38:09Z", "by": "shanks.node-tools", "text": "lead-47@shanks 2026-10-28 09:40Z. New kind beside this issue (diag-despiegel-check@shanks): the checker judges a pull request only when the gate's source.main names its repository"},
{"at": "2026-10-28T09:43:50Z", "by": "shanks.node-tools", "text": "fix-trunk-unknown@shanks, 2026-10-28. Quick fix: the checker says loudly that a pull request cannot be judged when source.main names no trunk head of its repository."},
{"at": "2026-10-28T09:51:00Z", "by": "shanks.node-tools", "text": "fix-trunk-unknown@shanks, 2026-10-28, part (b) built locally, not pushed."},
{"at": "2026-10-28T10:00:02Z", "by": "shanks.node-tools", "text": "lead-47@shanks 2026-10-28 10:05Z. de-spiegel to main (the operator, 09:58Z: 'yes use the main branch please')."},
{"at": "2026-10-28T10:07:56Z", "by": "shanks.node-tools", "text": "lead-47@shanks 2026-10-28 10:10Z. Side finding (tools-gitea-base@shanks): modules/gitea guard.ts and gitea_api's refusal text cite 'ADR 0266'"},
{"at": "2026-10-28T10:40:43Z", "by": "shanks.node-tools", "text": "lead-47@shanks 2026-10-28 10:41Z. Fix group delivered in part (gate 4c091b08, gitea a6cc3509 on novox at 10:39Z; indexer applying; bridge and checker pending)."},
{"at": "2026-10-28T10:44:38Z", "by": "shanks.node-tools", "text": "lead-47@shanks 2026-10-28 10:46Z. Subtask (the operator asked why a retarget waits for a poll)"},
{"at": "2026-10-28T10:45:16Z", "by": "shanks.node-tools", "text": "lead-47@shanks 2026-10-28 10:49Z. Correction to note 10"},
{"at": "2026-10-28T11:05:21Z", "by": "shanks.node-tools", "text": "Gitea polling, quick versus thorough (fix-gitea-events@shanks, branch fix/1422-gitea-events, commit 470687ea)."}
],
"number": 1422,
"opened": "2026-10-20T14:36:07Z",
"opened-by": "shanks.node-tools",
"ready": true,
"severity": "minor",
"status": "fixing",
"title": "The catch-up reads the trunk 'main' of repositories the catalogue follows by another branch or by a commit, and the forge holds none",
"worker": {"active": "2026-10-28T11:05:21Z", "by": "fix-trunk-unknown@shanks", "since": "2026-10-28T09:38:37Z"}
}
@@ -0,0 +1,19 @@
{
"links": [
{"at": "2026-10-28T11:07:31Z", "by": "triage", "kind": "relates-to", "target": "1903"},
{"at": "2026-10-28T11:07:31Z", "by": "triage", "kind": "relates-to", "target": "1327"},
{"at": "2026-10-28T11:07:31Z", "by": "triage", "kind": "relates-to", "target": "1420"},
{"at": "2026-10-28T11:07:31Z", "by": "triage", "kind": "relates-to", "target": "1184"}
],
"needs": "decision",
"notes": [
{"at": "2026-10-28T11:07:31Z", "by": "triage", "text": "Triage of this issue: not a duplicate; closest is 1903."}
],
"number": 1956,
"opened": "2026-10-28T11:07:10Z",
"opened-by": "shanks.node-tools",
"ready": true,
"severity": "major",
"status": "open",
"title": "Parts of the mesh still poll or wait on timers instead of reacting to events on the bus"
}
@@ -0,0 +1,9 @@
{"history": [
{"design": 47, "id": "6", "made": {"at": "2026-10-26T18:09:47Z", "by": "shanks.node-tools"}, "order": 9, "outcome": "Folded into release 4 by ADR 0429 §10 (the fetch over SSH in R4c, the cut-over in R4e): installed toolchains, builders fetching over SSH with their own key, and an SDK change shipping with its users are built in the builder's new home, so the builder is changed once", "status": "open"},
{"design": 47, "id": "6", "made": {"at": "2026-10-26T18:09:47Z", "by": "shanks.node-tools"}, "moved": {"at": "2026-10-26T18:10:10Z", "by": "shanks.node-tools"}, "order": 9, "outcome": "Folded into release 4 by ADR 0429 §10 (the fetch over SSH in R4c, the cut-over in R4e): installed toolchains, builders fetching over SSH with their own key, and an SDK change shipping with its users are built in the builder's new home, so the builder is changed once", "reason": "folded into R4g", "status": "closed"}
],
"milestones": [
{"design": 47, "done": 0, "id": "6", "issues": [], "made": {"at": "2026-10-26T18:09:47Z", "by": "shanks.node-tools"}, "moved": {"at": "2026-10-26T18:10:10Z", "by": "shanks.node-tools"}, "order": 9, "outcome": "Folded into release 4 by ADR 0429 §10 (the fetch over SSH in R4c, the cut-over in R4e): installed toolchains, builders fetching over SSH with their own key, and an SDK change shipping with its users are built in the builder's new home, so the builder is changed once", "reason": "folded into R4g", "ref": "47/6", "status": "closed"},
{"design": 47, "done": 0, "id": "L2", "issues": [1126], "made": {"at": "2026-10-26T18:09:47Z", "by": "shanks.node-tools"}, "order": 31, "outcome": "Nodes compose: each node agent composes its own modules, and no node applies a composition made elsewhere but the bus's user list (ADR 0547 §11)", "ref": "47/L2", "status": "open"}
],
"said": "2 milestones."}
@@ -0,0 +1,5 @@
[
{"moved": "2026-10-28T11:13:28Z", "needs": "decision", "number": 1126, "owner": "mesh-controller", "ready": false, "severity": "minor", "since": "2026-10-27T01:01:41Z", "status": "fixing", "title": "About 49,800 controller code lines stay after the delivery model, against a first limit of 25,000", "worker": "lead-47@shanks"},
{"moved": "2026-10-28T11:05:21Z", "needs": "none", "number": 1422, "owner": "mesh-controller cmd/mesh-controller/catch_up.go", "ready": true, "severity": "minor", "since": "2026-10-28T09:38:37Z", "status": "fixing", "title": "The catch-up reads the trunk 'main' of repositories the catalogue follows by another branch or by a commit, and the forge holds none", "worker": "fix-trunk-unknown@shanks"},
{"moved": "2026-10-28T11:07:31Z", "needs": "decision", "number": 1956, "owner": "mesh-host; mesh-catalog modules/gitea; mesh-delivery; mesh-controller", "ready": true, "severity": "major", "status": "open", "title": "Parts of the mesh still poll or wait on timers instead of reacting to events on the bus"}
]
@@ -0,0 +1,10 @@
{"delivery": {"id": "novox/de-spiegel@fba6dbe3259e", "number": 6, "repository": "novox/de-spiegel", "issues": [1126], "state": "merged", "since": "2026-10-28T10:45:28.285947172Z",
"title": "Take the practice's public names from config, not the definition (issue 1126)",
"check": {"at": "2026-10-28T10:44:13.035037566Z", "gate": "pass", "head": "fba6dbe3259e", "status": "mesh/check", "summary": "pass fba6dbe3259e on 927d302714af | unit 0 run 0 kept | int 0 run 0 kept | clone 31s lint 177ms | composition passed", "trunk": "927d302714af"},
"merged_at": "2026-10-28T10:45:00Z",
"transitions": [
{"at": "2026-10-28T08:49:52.467925371Z", "event": "announced", "from": "", "to": "proposed", "why": "novox/de-spiegel#6's head announced"},
{"at": "2026-10-28T08:49:54.046089933Z", "event": "accepted", "from": "proposed", "to": "ready", "why": "a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false)"},
{"at": "2026-10-28T10:45:28.285947172Z", "event": "merged", "from": "ready", "to": "merged", "why": "on the trunk its modules follow: merged there"}
]},
"machines": "on the machines", "waits": ""}
@@ -0,0 +1,10 @@
{"delivery": {"id": "novox/mesh-catalog@f5d4b2f86ed4", "number": 537, "repository": "novox/mesh-catalog", "issues": [1422], "group": "fix/1422-trunk-unknown", "state": "merged", "since": "2026-10-28T10:34:49.628764012Z",
"title": "Gate seeds a never-merged repository's trunk head and follows a retargeted base; gitea announces a retarget and sets a default branch and a pull request's base (issue 1422; group fix/1422-trunk-unknown)",
"check": {"at": "2026-10-28T10:30:45.793798647Z", "gate": "pass", "head": "f5d4b2f86ed4", "status": "mesh/check", "summary": "pass f5d4b2f86ed4 on 9fd95693db54 | unit 1329 run, 3852 kept, 13 skipped | integration 1330 run, 3968 kept, 21 skipped | over bound: unit, …", "trunk": "9fd95693db54"},
"merged_at": "2026-10-28T10:34:47Z",
"transitions": [
{"at": "2026-10-28T10:20:56.142821047Z", "event": "announced", "from": "", "to": "proposed", "why": "novox/mesh-catalog#537's head announced"},
{"at": "2026-10-28T10:20:58.259805356Z", "event": "accepted", "from": "proposed", "to": "ready", "why": "a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false)"},
{"at": "2026-10-28T10:34:49.628764012Z", "event": "merged", "from": "ready", "to": "merged", "why": "on the trunk its modules follow: merged there"}
]},
"machines": "on the machines", "waits": ""}
@@ -0,0 +1,8 @@
{"delivery": {"id": "novox/mesh-catalog@470687ea4cce", "number": 539, "repository": "novox/mesh-catalog", "issues": [1422, 1956], "state": "ready", "since": "2026-10-28T11:07:43.555244108Z",
"title": "gitea announces from the forge's events, not a poll: an edited base and a close are announced at once, one pass at start (issue 1422)",
"check_state": "pending", "merged_at": "0001-01-01T00:00:00Z",
"transitions": [
{"at": "2026-10-28T11:07:42.740718441Z", "event": "announced", "from": "", "to": "proposed", "why": "novox/mesh-catalog#539's head announced"},
{"at": "2026-10-28T11:07:43.555244108Z", "event": "accepted", "from": "proposed", "to": "ready", "why": "a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false)"}
]},
"waits": "its merge"}
@@ -0,0 +1,10 @@
{"delivery": {"id": "novox/mesh-delivery@2b801f5bd49a", "number": 94, "repository": "novox/mesh-delivery", "issues": [1126, 1422], "group": "fix/1422-trunk-unknown", "state": "merged", "since": "2026-10-28T10:34:54.759088217Z",
"title": "Say loudly what cannot be judged, read only a merge's changed directories, and move the taken key only by an applied merge (issues 1422, 1126; group fix/1422-trunk-unknown)",
"check": {"at": "2026-10-28T10:30:45.875693927Z", "gate": "pass", "head": "2b801f5bd49a", "status": "mesh/check", "summary": "pass 2b801f5bd49a on ae7ec702a5e9 | unit 1329 run, 3852 kept, 13 skipped | integration 1330 run, 3968 kept, 21 skipped | over bound: unit, …", "trunk": "ae7ec702a5e9"},
"merged_at": "2026-10-28T10:34:51Z",
"transitions": [
{"at": "2026-10-28T10:20:50.017669046Z", "event": "announced", "from": "", "to": "proposed", "why": "novox/mesh-delivery#94's head announced"},
{"at": "2026-10-28T10:20:51.900542745Z", "event": "accepted", "from": "proposed", "to": "ready", "why": "a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false)"},
{"at": "2026-10-28T10:34:54.759088217Z", "event": "merged", "from": "ready", "to": "merged", "why": "on the trunk its modules follow: merged there"}
]},
"machines": "on the machines", "waits": ""}
+1
View File
@@ -88,6 +88,7 @@
"seat:issue-tracker.list",
"seat:issue-tracker.get",
"seat:issue-tracker.milestones",
"seat:issue-tracker.changes",
"seat:mesh-delivery.deliveries",
"seat:mesh-delivery.show",
"seat:mesh-delivery.groups"
@@ -4,6 +4,9 @@ import (
"encoding/json"
"fmt"
"strings"
"sync"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// Who may ask `get` for which secret (novox/hq ADR 0349 §11, design 57 V4, issue 647).
@@ -108,6 +111,9 @@ func fieldSet(f json.RawMessage) bool {
// that the module's plan there names.
type Declared struct {
Read AssignmentReader
// Confirm, when set, reads an assignment directly (over the bus): a module Read does not hold is confirmed through it
// before a credential is refused, since a watched mirror can be behind (issue 1126, review round 1).
Confirm AssignmentReader
// Store is the vault's store, whose record of who holds a provision's one credential (pair.go) the rule reads.
Store *Store
}
@@ -207,3 +213,77 @@ func (d Declared) May(caller, name string) (bool, string) {
// bus error rather than an answer. The vault says "holds no secret named ... yet", which the client waits on and asks
// again, and records no refusal.
const notYetPrefix = "not yet: "
// WatchedAssignments is the assignments mirror as the vault's watch of it hands it over (issue 1126, 2026-10-28: novox's
// `own` read one assignment over the bus per own secret and per credential, 203 reads, and answered after 26 s, past
// the node-engine's 15 s). The state is the bus's; the vault watches it, never polls it, and reads it from memory once the
// watch has handed every current value. Until then, and when the watch could not be begun, a read goes to the bus
// (Fallback), as before.
type WatchedAssignments struct {
mu sync.Mutex
held map[string][]byte
caught bool
attempt uint64
Fallback AssignmentReader
}
// WatchAttempt is one attempt at the watch: what it hands over counts only while it is the latest attempt, so a watch
// that failed partway and is begun again keeps nothing it took (a key deleted meanwhile is never handed again), and a
// late delivery to an older attempt changes nothing (issue 1126, review round 1).
type WatchAttempt struct {
w *WatchedAssignments
n uint64
}
// Begin starts a new attempt from nothing held: reads go to the bus until it has caught up.
func (w *WatchedAssignments) Begin() WatchAttempt {
w.mu.Lock()
defer w.mu.Unlock()
w.attempt++
w.held, w.caught = map[string][]byte{}, false
return WatchAttempt{w: w, n: w.attempt}
}
// Take keeps one change the attempt handed over, a value put or a key deleted, while it is the latest attempt.
func (a WatchAttempt) Take(c stdio.StateChange) error {
w := a.w
w.mu.Lock()
defer w.mu.Unlock()
if a.n != w.attempt {
return nil
}
if c.Op == "delete" || c.Op == "purge" || len(c.Value) == 0 {
delete(w.held, c.Key)
return nil
}
w.held[c.Key] = append([]byte(nil), c.Value...)
return nil
}
// Caught says the attempt handed every current value: reads are answered from memory from now on, if it is the latest.
func (a WatchAttempt) Caught() {
w := a.w
w.mu.Lock()
defer w.mu.Unlock()
if a.n == w.attempt {
w.caught = true
}
}
// Read is one value of the mirror, `<node>.<module>`: nil when there is none.
func (w *WatchedAssignments) Read(key string) ([]byte, error) {
w.mu.Lock()
if w.caught {
raw, ok := w.held[key]
w.mu.Unlock()
if !ok {
return nil, nil
}
return append([]byte(nil), raw...), nil
}
w.mu.Unlock()
if w.Fallback == nil {
return nil, fmt.Errorf("the assignments mirror is not watched yet, and nothing reads it from the bus")
}
return w.Fallback(key)
}
+29 -2
View File
@@ -49,13 +49,37 @@ func build() *Env {
}
// Who may ask (askers.go): a node's provisioner client, for what its module's plan there names.
// Where it sees a consumer bind a provider (pair.go): the same mirror, every key.
env.Vault = &Vault{Store: store, Own: own, Askers: Declared{Read: readAssignment, Store: store},
// The assignments as the vault's watch keeps them (watchAssignments), read over the bus until it has caught up.
assigned = &WatchedAssignments{Fallback: readAssignment}
env.Vault = &Vault{Store: store, Own: own, Askers: Declared{Read: assigned.Read, Confirm: readAssignment, Store: store},
Assignments: Mirror{Keys: assignmentKeys, Read: readAssignment}, Emit: emit, Ask: stdio.Ask,
Now: time.Now, Log: say, Accounts: busAccounts{}, Haves: readHave,
HaveKeys: func() ([]string, error) { return stdio.State(HaveRead).Keys() }}
return env
}
// assigned is the assignments mirror as the vault's watch keeps it.
var assigned *WatchedAssignments
// watchAssignments begins the watch of the assignments mirror the vault's rules read (issue 1126, 2026-10-28): a
// watch, never a poll, so `own` and `get` read no assignment over the bus once it has caught up. It answers whether the
// watch began; one that did not is begun again at the next minute pass, and reads go to the bus meanwhile.
func watchAssignments() bool {
if assigned == nil {
return true
}
// Each attempt begins from nothing held: one that failed partway leaves nothing a later one would not hand again.
attempt := assigned.Begin()
if err := stdio.State(AssignmentsRead).Watch("", attempt.Take); err != nil {
assigned.Begin() // what the failed attempt took counts for nothing, and reads go to the bus
say("the assignments are not watched, so each is read over the bus; begun again at the next pass: %v", err)
return false
}
attempt.Caught()
say("the assignments are watched: the vault's rules read them from memory")
return true
}
// readHave is one key of the node-engines' have bucket, read through the runtime under the vault's own account (its
// manifest `reads` it), or nil when there is none.
func readHave(key string) ([]byte, error) {
@@ -149,11 +173,14 @@ func main() {
written, failed := env.Vault.HashHeld()
say("hashed %d bus accounts at start; %d could not be, tried again every minute: %v", len(written), len(failed), failed)
}
watched := watchHandings(env)
watched, watching := watchHandings(env), watchAssignments()
for {
if !watched {
watched = watchHandings(env)
}
if !watching {
watching = watchAssignments()
}
env.pass()
if env.Vault != nil {
env.Vault.HashOwed()
+74 -15
View File
@@ -70,7 +70,7 @@ func credentialNames(c CredentialAsked) []string {
// credentialRefusal says why node may not have c, or "": a consumer's credential to the machine its consuming module
// stands on, a provider's to the providing machine, and nothing to a machine that is neither.
func (v *Vault) credentialRefusal(node string, c CredentialAsked) string {
func (v *Vault) credentialRefusal(node string, c CredentialAsked, plan *planReader) string {
for _, part := range []string{c.Provision, c.Consumer, c.ConsumerModule, c.Provider} {
if !namePart.MatchString(part) {
return fmt.Sprintf("%q is not a name the mesh writes", part)
@@ -85,14 +85,19 @@ func (v *Vault) credentialRefusal(node string, c CredentialAsked) string {
return fmt.Sprintf("%s asks as a consumer on %s, and is not that machine", node, c.Consumer)
}
// The consuming module stands on the machine: read, never assumed (an assignment that cannot be read refuses).
read, why := v.assignmentReader(node)
raw, why, err := plan.read(c.ConsumerModule)
if why != "" {
return why
}
raw, err := read(node + "." + c.ConsumerModule)
if err != nil {
return fmt.Sprintf(notYetPrefix+"the assignment %s.%s could not be read: %v", node, c.ConsumerModule, err)
}
if raw == nil {
// Confirmed directly before refusing: the mirror may not hold a module assigned a moment ago.
if raw, err = plan.confirmed(c.ConsumerModule); err != nil {
return fmt.Sprintf(notYetPrefix+"the assignment %s.%s could not be read: %v", node, c.ConsumerModule, err)
}
}
if raw == nil {
return fmt.Sprintf("%s is not on %s: neither assigned there nor required by a module there", c.ConsumerModule, node)
}
@@ -106,6 +111,56 @@ func (v *Vault) credentialRefusal(node string, c CredentialAsked) string {
return fmt.Sprintf("the role %q is not consumer or provider", c.Role)
}
// planReader reads the assignments of one machine's modules for one answer of `own`, each module's once (issue 1126,
// 2026-10-28: one read per own secret and per credential, 203 on novox, held the answer past the node-engine's bound).
type planReader struct {
node string
reader AssignmentReader
confirm AssignmentReader
why string
got map[string]planRead
}
type planRead struct {
raw []byte
err error
}
func (v *Vault) plansOf(node string) *planReader {
read, why := v.assignmentReader(node)
var confirm AssignmentReader
if d, ok := v.Askers.(Declared); ok && d.Confirm != nil {
confirm = standing(d.Confirm)
}
return &planReader{node: node, confirm: confirm, reader: read, why: why, got: map[string]planRead{}}
}
// read is module's assignment on the machine, nil when there is none, or why the vault reads none.
func (p *planReader) read(module string) ([]byte, string, error) {
if p.why != "" {
return nil, p.why, nil
}
r, ok := p.got[module]
if !ok {
r.raw, r.err = p.reader(p.node + "." + module)
p.got[module] = r
}
return r.raw, "", r.err
}
// confirmed is module's assignment read directly, for one the reader does not hold (a watched mirror can be behind),
// nil when there is none or nothing reads directly.
func (p *planReader) confirmed(module string) ([]byte, error) {
if p.confirm == nil {
return nil, nil
}
raw, err := p.confirm(p.node + "." + module)
if err == nil && raw != nil {
p.got[module] = planRead{raw: raw}
}
return raw, err
}
// assignmentReader is the vault's reader of the assignments, or why it has none: the rule `own` keeps reads them,
// and without them it refuses rather than hands (a rule that cannot be read is no rule).
func (v *Vault) assignmentReader(node string) (AssignmentReader, string) {
@@ -177,17 +232,21 @@ func (v *Vault) OwnOfNode(caller string, asked ...CredentialAsked) (OwnAnswer, e
if err != nil {
return OwnAnswer{}, fmt.Errorf("%s's sealing key as recorded is not usable: %w", node, err)
}
all, err := v.Store.List()
// Only this machine's own records are read, each module's assignment once, and every hand-out recorded in one synced
// write after all are sealed: none is handed when that record cannot be written (issue 1126, 2026-10-28).
all, err := v.Store.ListPrefix("own/" + node + "/")
if err != nil {
return OwnAnswer{}, err
}
plan := v.plansOf(node)
var handings []Handing
out := OwnAnswer{Node: node, Key: key}
for _, r := range all {
parts := strings.Split(r.Name, "/")
if len(parts) != 4 || parts[0] != "own" || parts[1] != node {
continue
}
handed, err := v.handed(node, parts[2])
handed, err := plan.handed(parts[2])
if err != nil {
return OwnAnswer{}, err
}
@@ -202,9 +261,7 @@ func (v *Vault) OwnOfNode(caller string, asked ...CredentialAsked) (OwnAnswer, e
if err != nil {
return OwnAnswer{}, err
}
if err := v.recordHanding(Handing{To: caller, Name: r.Name, Generation: r.Generation, Outcome: "handed"}); err != nil {
return OwnAnswer{}, fmt.Errorf("%s is not handed out: who was handed it could not be recorded: %w", r.Name, err)
}
handings = append(handings, Handing{To: caller, Name: r.Name, Generation: r.Generation, Outcome: "handed"})
out.Own = append(out.Own, NodeOwnSecret{Module: parts[2], Name: parts[3], Origin: composeOrigin(r.Origin),
Generation: r.Generation, Sealed: sealed})
}
@@ -215,7 +272,7 @@ func (v *Vault) OwnOfNode(caller string, asked ...CredentialAsked) (OwnAnswer, e
return out.Own[i].Name < out.Own[j].Name
})
for _, c := range asked {
if why := v.credentialRefusal(node, c); why != "" {
if why := v.credentialRefusal(node, c, plan); why != "" {
return OwnAnswer{}, fmt.Errorf("%s may not ask for the credential of %s from %s to %s on %s: %s", caller,
c.Provision, c.Provider, c.ConsumerModule, c.Consumer, why)
}
@@ -248,9 +305,7 @@ func (v *Vault) OwnOfNode(caller string, asked ...CredentialAsked) (OwnAnswer, e
if err != nil {
return OwnAnswer{}, err
}
if err := v.recordHanding(Handing{To: caller, Name: name, Generation: r.Generation, Outcome: "handed"}); err != nil {
return OwnAnswer{}, fmt.Errorf("%s is not handed out: who was handed it could not be recorded: %w", name, err)
}
handings = append(handings, Handing{To: caller, Name: name, Generation: r.Generation, Outcome: "handed"})
out.Credentials = append(out.Credentials, NodeCredential{CredentialAsked: c, Name: name, Generation: r.Generation, Sealed: sealed})
found = true
break
@@ -259,6 +314,10 @@ func (v *Vault) OwnOfNode(caller string, asked ...CredentialAsked) (OwnAnswer, e
out.Missing = append(out.Missing, c)
}
}
if err := v.recordHandings(handings); err != nil {
return OwnAnswer{}, fmt.Errorf("nothing is handed out to %s: who was handed its %d values could not be recorded: %w",
caller, len(handings), err)
}
v.log("handed %s's node-engine the %d own secrets of its modules and %d credentials (%d not held), sealed to its key",
node, len(out.Own), len(out.Credentials), len(out.Missing))
return out, nil
@@ -267,12 +326,12 @@ func (v *Vault) OwnOfNode(caller string, asked ...CredentialAsked) (OwnAnswer, e
// handed says the plan of module on node hands it to the provisioner, so its secrets are filled from `get`, not composed.
// A plan that cannot be read is an error, never "not handed": `own` refuses rather than hands a secret the provisioner
// holds.
func (v *Vault) handed(node, module string) (bool, error) {
read, why := v.assignmentReader(node)
func (p *planReader) handed(module string) (bool, error) {
node := p.node
raw, why, err := p.read(module)
if why != "" {
return false, errors.New(why)
}
raw, err := read(node + "." + module)
if err != nil {
return false, fmt.Errorf(notYetPrefix+"the assignment %s.%s could not be read, so whether it is handed to the "+
"provisioner is not known: %w", node, module, err)
@@ -2,9 +2,12 @@ package main
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/seal"
)
@@ -119,3 +122,117 @@ func TestOwnAnswersAMachineItsModulesSecretsSealedToItsKey(t *testing.T) {
t.Errorf("own with no reader of the assignments: %v", err)
}
}
// Issue 1126, 2026-10-28 (seed-9's probe: novox's 57 own secrets and 146 credentials took 26 s, past the node-engine's
// 15 s): one answer of `own` reads each module's assignment once, and records every hand-out in one synced write.
func TestOwnReadsEachAssignmentOnceAndSyncsItsRecordOnce(t *testing.T) {
plans := assignments{"anchor.web": `{"node":"anchor","module":"web"}`, "anchor.db": `{"node":"anchor","module":"db"}`}
reads := map[string]int{}
v := newVault(t, nil)
v.Askers = Declared{Read: func(key string) ([]byte, error) { reads[key]++; return plans.read(key) }, Store: v.Store}
pub, _ := aSealingKey(t)
for _, m := range []MoveOf{
{Node: "anchor", Module: "web", Name: "password", Origin: "made", Generation: "g-1", SealingKey: pub},
{Node: "anchor", Module: "web", Name: "api-key", Origin: "made", Generation: "g-1", SealingKey: pub},
{Node: "anchor", Module: "web", Name: "cookie", Origin: "made", Generation: "g-1", SealingKey: pub},
{Node: "anchor", Module: "db", Name: "root", Origin: "made", Generation: "g-1", SealingKey: pub},
{Provider: "anchor", Provision: "postgres", Consumer: "anchor", ConsumerModule: "web", Origin: "made", Generation: "g-1", SealingKey: pub},
{Provider: "anchor", Provision: "redis", Consumer: "anchor", ConsumerModule: "web", Origin: "made", Generation: "g-1", SealingKey: pub},
} {
if _, err := moveIn(t, v, "node.anchor", m, "v-"+m.Name+m.Provision); err != nil {
t.Fatal(err)
}
}
for k := range reads {
delete(reads, k)
}
syncs, handed := v.Store.syncs, len(mustHanded(t, v))
got, err := v.OwnOfNode("node.anchor",
CredentialAsked{Role: "consumer", Provision: "postgres", Consumer: "anchor", ConsumerModule: "web", Provider: "anchor"},
CredentialAsked{Role: "consumer", Provision: "redis", Consumer: "anchor", ConsumerModule: "web", Provider: "anchor"})
if err != nil || len(got.Own) != 4 || len(got.Credentials) != 2 {
t.Fatalf("anchor was answered %+v, %v", got, err)
}
for key, n := range reads {
if n != 1 {
t.Errorf("the assignment %s was read %d times in one answer", key, n)
}
}
if len(reads) != 2 {
t.Errorf("the answer read %v, want each of anchor's two modules once", reads)
}
if n := v.Store.syncs - syncs; n != 1 {
t.Errorf("one answer synced the record %d times", n)
}
if n := len(mustHanded(t, v)) - handed; n != 6 {
t.Errorf("one answer recorded %d hand-outs, want 6", n)
}
// Review round 1, note (b): a consumer module the mirror does not hold yet (its watch behind) is read directly before
// the credential is refused.
v.Askers = Declared{Read: func(string) ([]byte, error) { return nil, nil }, Confirm: plans.read, Store: v.Store}
if got, err := v.OwnOfNode("node.anchor", CredentialAsked{Role: "consumer", Provision: "postgres", Consumer: "anchor",
ConsumerModule: "web", Provider: "anchor"}); err != nil || len(got.Credentials) != 1 {
t.Errorf("a module the mirror does not hold yet, read directly: %+v, %v", got, err)
}
v.Askers = Declared{Read: plans.read, Store: v.Store}
// A record that cannot be written hands nothing, the batch as one.
if err := os.Remove(filepath.Join(v.dir, "store", "handed.jsonl")); err != nil {
t.Fatal(err)
}
if err := os.Mkdir(filepath.Join(v.dir, "store", "handed.jsonl"), 0o700); err != nil {
t.Fatal(err)
}
if a, err := v.OwnOfNode("node.anchor"); err == nil {
t.Errorf("handed with no record of it: %+v", a)
}
}
func mustHanded(t *testing.T, v testVault) []Handing {
t.Helper()
h, err := v.Store.Handed(1000)
if err != nil {
t.Fatal(err)
}
return h
}
// Issue 1126, 2026-10-28: the vault keeps the assignments mirror as its watch hands it over, so `own` and `get` read
// no assignment over the bus once the watch has caught up; before that a read goes to the bus.
func TestTheWatchedAssignmentsAnswerWithoutTheBusOnceCaughtUp(t *testing.T) {
asked := 0
w := &WatchedAssignments{Fallback: func(key string) ([]byte, error) { asked++; return []byte(`{"from":"bus"}`), nil }}
if raw, err := w.Read("anchor.web"); err != nil || string(raw) != `{"from":"bus"}` || asked != 1 {
t.Errorf("before the watch caught up: %s, %v, %d asks", raw, err, asked)
}
// A first watch takes two values and fails before it caught up; a late value still arrives on it.
first := w.Begin()
for _, c := range []stdio.StateChange{
{Key: "anchor.web", Op: "put", Value: []byte(`{"node":"anchor","module":"web"}`), Current: true},
{Key: "anchor.db", Op: "put", Value: []byte(`{"node":"anchor","module":"db"}`), Current: true},
} {
if err := first.Take(c); err != nil {
t.Fatal(err)
}
}
// Review round 1: the watch begun again holds anchor.db no more (removed meanwhile); what the first took is gone.
second := w.Begin()
_ = second.Take(stdio.StateChange{Key: "anchor.web", Op: "put", Value: []byte(`{"node":"anchor","module":"web"}`), Current: true})
_ = first.Take(stdio.StateChange{Key: "anchor.db", Op: "put", Value: []byte(`{"node":"anchor","module":"db"}`)})
first.Caught()
if raw, _ := w.Read("anchor.db"); string(raw) != `{"from":"bus"}` {
t.Errorf("the failed watch's caught up counted: %s", raw)
}
second.Caught()
_ = first.Take(stdio.StateChange{Key: "anchor.db", Op: "put", Value: []byte(`{"node":"anchor","module":"db"}`)})
_ = second.Take(stdio.StateChange{Key: "anchor.web", Op: "put", Value: []byte(`{"node":"anchor","module":"web","v":2}`)})
asked = 0
for key, want := range map[string]string{"anchor.web": `{"node":"anchor","module":"web","v":2}`, "anchor.db": "", "laptop.x": ""} {
raw, err := w.Read(key)
if err != nil || string(raw) != want || (want == "") != (raw == nil) {
t.Errorf("%s read %q, %v; want %q", key, raw, err, want)
}
}
if asked != 0 {
t.Errorf("caught up, the mirror asked the bus %d times", asked)
}
}
+27 -7
View File
@@ -81,6 +81,8 @@ type Store struct {
handedMu sync.Mutex
rolled int
// syncs counts the record's synced appends since the vault started: how a test proves one sync per answer.
syncs int
// asksMu serialises the record of asks, so two asks for one secret at once do not both pass its bounds.
asksMu sync.Mutex
}
@@ -173,22 +175,37 @@ type Handing struct {
// RecordHanding appends one handing to the store's record, synced before it returns. It says whether the
// record was rolled to make room.
func (s *Store) RecordHanding(h Handing) (rolled bool, err error) {
raw, err := json.Marshal(h)
if err != nil {
return false, err
return s.RecordHandings([]Handing{h})
}
// RecordHandings appends every handing of one answer to the store's record in one write, synced once before it returns
// (issue 1126, 2026-10-28: one sync per secret, under the one lock every node's answers share, held `own` past the
// node-engine's bound). It says whether the record was rolled to make room; nothing is written when one cannot be.
func (s *Store) RecordHandings(hs []Handing) (rolled bool, err error) {
if len(hs) == 0 {
return false, nil
}
var raw []byte
for _, h := range hs {
line, err := json.Marshal(h)
if err != nil {
return false, err
}
raw = append(append(raw, line...), '\n')
}
s.handedMu.Lock()
defer s.handedMu.Unlock()
path := filepath.Join(s.dir, "handed.jsonl")
max := s.HandedBound()
if info, err := os.Stat(path); err == nil && info.Mode().IsRegular() && info.Size()+int64(len(raw))+1 > max {
if info, err := os.Stat(path); err == nil && info.Mode().IsRegular() && info.Size()+int64(len(raw)) > max {
if err := s.roll(path); err != nil {
return false, err
}
s.rolled++
rolled = true
}
return rolled, appendSynced(path, append(raw, '\n'))
s.syncs++
return rolled, appendSynced(path, raw)
}
// roll moves handed.jsonl to .1, .1 to .2 and so on, dropping what falls past HandedKept.
@@ -266,7 +283,10 @@ func (s *Store) HandedBound() int64 {
}
// List is every record, by name.
func (s *Store) List() ([]Record, error) {
func (s *Store) List() ([]Record, error) { return s.ListPrefix("") }
// ListPrefix is every record whose name begins with prefix, by name: only those files are read.
func (s *Store) ListPrefix(prefix string) ([]Record, error) {
entries, err := os.ReadDir(filepath.Join(s.dir, "secrets"))
if err != nil {
return nil, err
@@ -277,7 +297,7 @@ func (s *Store) List() ([]Record, error) {
continue
}
name, err := url.PathUnescape(strings.TrimSuffix(e.Name(), ".json"))
if err != nil {
if err != nil || !strings.HasPrefix(name, prefix) {
continue
}
r, ok, err := s.Read(name)
+17 -4
View File
@@ -235,11 +235,24 @@ func (v *Vault) announce(c Changed) {
// recordHanding keeps who asked for what, and what they got, in the vault's own store, and says loudly
// when the record had to be rolled to make room.
func (v *Vault) recordHanding(h Handing) error {
h.At = v.Now().UTC()
rolled, err := v.Store.RecordHanding(h)
func (v *Vault) recordHanding(h Handing) error { return v.recordHandings([]Handing{h}) }
// recordHandings keeps every handing of one answer in one synced write, as recordHanding keeps one.
func (v *Vault) recordHandings(hs []Handing) error {
if len(hs) == 0 {
return nil
}
at := v.Now().UTC()
for i := range hs {
hs[i].At = at
}
rolled, err := v.Store.RecordHandings(hs)
if err != nil {
v.log("the record of %s asking for %s could not be written: %v", h.To, h.Name, err)
names := make([]string, len(hs))
for i, h := range hs {
names[i] = h.Name
}
v.log("the record of %s asking for %s could not be written: %v", hs[0].To, strings.Join(names, ", "), err)
return err
}
if rolled {
@@ -167,8 +167,10 @@ func TestNoBucketHoldsASecret(t *testing.T) {
code = []byte(strings.Replace(string(code), "stdio.State(BusAccountsState).Put(", "", 1))
code = []byte(strings.Replace(string(code), "stdio.State(BusAccountsState).Keys(", "", 1))
}
// The haves, in main.go alone: one read, one watch and their keys (own_drift), never a write.
// The haves, in main.go alone: one read, one watch and their keys (own_drift), never a write; and one watch of
// the assignments, which the vault's rules read from memory once it caught up (issue 1126).
if f == "main.go" {
code = []byte(strings.Replace(string(code), "stdio.State(AssignmentsRead).Watch(", "", 1))
code = []byte(strings.Replace(string(code), "stdio.State(HaveRead).Get(", "", 1))
code = []byte(strings.Replace(string(code), "stdio.State(HaveRead).Watch(", "", 1))
code = []byte(strings.Replace(string(code), "stdio.State(HaveRead).Keys(", "", 1))
+4
View File
@@ -14,6 +14,7 @@ states on the bus.
| `/usr/local/lib/mesh-power/bin/power-moment` | the runner: each module's piece on its own, as root, with `sh`, bounded (`MESH_POWER_BOUND`, default 30 s); every outcome in the journal under `mesh-power` |
| five units | `mesh-power-after-boot`, `-before-shutdown` (its stop is the shutdown, while the network is still up), `-before-sleep`, `-after-wake`, `-supply` |
| `mesh-power-draw` (process) | samples the power drawn for the module's bundle, which puts it on the bus as `power.draw` (below) |
| `reboot.target.d`, `poweroff.target.d`, `halt.target.d` (`mesh-power.conf`) | a shutdown is bounded at ten minutes, not systemd's thirty; the target keeps its own forcing action (issue 1126: on ace's reboot of 2026-10-28 the bus was started again during the shutdown, nothing stopped it a second time, and the reboot waited 29 minutes) |
| `/etc/udev/rules.d/90-mesh-power.rules` | a power supply's change starts `mesh-power-supply`, which runs `on-mains` or `on-battery` once per change of source |
## The moments, and how a module adds code
@@ -146,4 +147,7 @@ Each directory or archive this module declares says how it treats a file it did
- `/etc/systemd/system/hibernate.target.d`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `/etc/systemd/system/hybrid-sleep.target.d`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `/etc/systemd/system/suspend-then-hibernate.target.d`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `/etc/systemd/system/reboot.target.d`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `/etc/systemd/system/poweroff.target.d`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `/etc/systemd/system/halt.target.d`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `/run/mesh-power`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
+50
View File
@@ -127,3 +127,53 @@ func TestTheBarBlocksAreTwoItemsOfTheBarsCollection(t *testing.T) {
}
}
}
// A shutdown is bounded at ten minutes, not systemd's thirty (issue 1126, ace's reboot of 2026-10-28): every unit had
// stopped within 17 s, yet the bus and the authorization manager were started again during the shutdown, nothing
// stopped them a second time, and the reboot waited on its job's own timeout of 30 min with nothing on the console.
// The bound keeps each target's own forcing action (reboot-force and the like): systemd's last phase still ends
// every process, unmounts and syncs. Ten minutes stays above the longest stop the catalogue declares (120 s) chained.
func TestEveryShutdownIsBoundedAtTenMinutes(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m struct {
Resources []map[string]any `json:"resources"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
byPath := map[string]map[string]any{}
for _, r := range m.Resources {
if p, ok := r["path"].(string); ok {
byPath[p] = r
}
}
for _, target := range []string{"reboot", "poweroff", "halt"} {
dir := "/etc/systemd/system/" + target + ".target.d"
if d := byPath[dir]; d == nil || d["type"] != "directory" || d["unknown"] != "program" {
t.Errorf("%s: no directory resource %s whose unknown files are the program's: %v", target, dir, d)
}
f := byPath[dir+"/mesh-power.conf"]
if f == nil || f["type"] != "file" {
t.Errorf("%s: no file resource %s/mesh-power.conf", target, dir)
continue
}
name, _ := f["template"].(string)
text, err := os.ReadFile(filepath.Join("../..", name))
if err != nil {
t.Errorf("%s: %v", target, err)
continue
}
var lines []string
for _, l := range strings.Split(string(text), "\n") {
if l != "" && !strings.HasPrefix(l, "#") {
lines = append(lines, l)
}
}
if strings.Join(lines, "\n") != "[Unit]\nJobTimeoutSec=10min" {
t.Errorf("%s: %s sets %q, want only [Unit] JobTimeoutSec=10min (the target keeps its own action)", target, name, lines)
}
}
}
+42
View File
@@ -314,6 +314,48 @@
"mode": "0644",
"template": "templates/systemd/system/suspend-then-hibernate.target.d/mesh-power.conf"
},
{
"id": "reboot-drop-ins",
"type": "directory",
"path": "/etc/systemd/system/reboot.target.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "reboot-bound",
"type": "file",
"path": "/etc/systemd/system/reboot.target.d/mesh-power.conf",
"mode": "0644",
"template": "templates/systemd/system/reboot.target.d/mesh-power.conf"
},
{
"id": "poweroff-drop-ins",
"type": "directory",
"path": "/etc/systemd/system/poweroff.target.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "poweroff-bound",
"type": "file",
"path": "/etc/systemd/system/poweroff.target.d/mesh-power.conf",
"mode": "0644",
"template": "templates/systemd/system/poweroff.target.d/mesh-power.conf"
},
{
"id": "halt-drop-ins",
"type": "directory",
"path": "/etc/systemd/system/halt.target.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "halt-bound",
"type": "file",
"path": "/etc/systemd/system/halt.target.d/mesh-power.conf",
"mode": "0644",
"template": "templates/systemd/system/halt.target.d/mesh-power.conf"
},
{
"id": "udev-rule",
"type": "file",
@@ -0,0 +1,8 @@
# Written by the mesh (module power, novox/hq ADR 0211; issue 1126). Replaced on every push.
#
# A shutdown is bounded at ten minutes, not systemd's thirty. A unit started again during the shutdown
# (on ace, 2026-10-28, the bus and the authorization manager) is never stopped a second time, and the
# shutdown then waits on this job's timeout with nothing on the console. The target keeps its own
# forcing action: the last phase still ends every process, unmounts and syncs.
[Unit]
JobTimeoutSec=10min
@@ -0,0 +1,8 @@
# Written by the mesh (module power, novox/hq ADR 0211; issue 1126). Replaced on every push.
#
# A shutdown is bounded at ten minutes, not systemd's thirty. A unit started again during the shutdown
# (on ace, 2026-10-28, the bus and the authorization manager) is never stopped a second time, and the
# shutdown then waits on this job's timeout with nothing on the console. The target keeps its own
# forcing action: the last phase still ends every process, unmounts and syncs.
[Unit]
JobTimeoutSec=10min
@@ -0,0 +1,8 @@
# Written by the mesh (module power, novox/hq ADR 0211; issue 1126). Replaced on every push.
#
# A shutdown is bounded at ten minutes, not systemd's thirty. A unit started again during the shutdown
# (on ace, 2026-10-28, the bus and the authorization manager) is never stopped a second time, and the
# shutdown then waits on this job's timeout with nothing on the console. The target keeps its own
# forcing action: the last phase still ends every process, unmounts and syncs.
[Unit]
JobTimeoutSec=10min