Compare commits

..
Author SHA1 Message Date
mesh-tools ceb3aac9b3 Merge pull request 'Build modules size their steps from the node's build share once it is given (issue 1806, change A)' (#499) from fix/build-cpu-yields into main 2026-10-26 14:22:50 +00:00
jschoubben 2aecbe8055 Merge main into fix/build-cpu-yields; pin mesh-sdk at its head (ADR 0553, ADR 0554)
mesh/delivery merged: source.main moved to novox/mesh-catalog@ceb3aac9b39a
mesh/check checking 2aecbe80553c on ceb3aac9b39a | novox/mesh-catalog#499 in group fix/build-cpu-yields
mesh/delivery-group group fix/build-cpu-yields merged: every member is merged
2026-10-26 14:47:27 +01:00
jschoubben 3def293d94 Build modules: no thread limit without the composer's share; with it, the share over their own capacity (ADR 0553 round 2)
Change A rebuilds the three build modules, so their runtimes restart once; it
changes nothing else about their builds until change B gives them the share.
2026-10-26 14:41:12 +01:00
mesh-tools 9a8e54d988 Merge pull request 'mesh-build-go says which packages it cannot test here, and mesh-build-node's tests need the Node toolchain (issue 890, ADR 0552)' (#498) from feat/890-placement into main 2026-10-26 13:40:41 +00:00
mesh-tools 2da9329df4 Merge pull request 'pacman: the round and the history tools say in plain words what they answer' (#497) from fix/pacman-findable into main 2026-10-26 13:39:16 +00:00
jschoubben ecbee53cfd Merge main into fix/build-cpu-yields; pin mesh-sdk at its head (ADR 0553) 2026-10-26 14:23:33 +01:00
jschoubben dcdf4238c7 Build modules: no cap of their own; a step's threads are its part of the node's one share (ADR 0553, issue 1806)
Review round 1: a cap per module let three build modules take three shares
of the node. The share is now one node value held by the node-engine's
yielding slice; each step divides it by every build slot of the node, from the
composer's words, or assumes every build seat at its own capacity without them.
2026-10-26 14:22:05 +01:00
jschoubben d3423f54ed mesh-build-go: resolve the go.mod and go.sum the main merge left in conflict; pin mesh-sdk 33395f68 (issue 890)
mesh/delivery merged: source.main moved to novox/mesh-catalog@9a8e54d988fe
mesh/delivery-group group feat/890-placement merged: every member is merged
mesh/check checking d3423f54ed58 on 9a8e54d988fe | novox/mesh-catalog#498 in group feat/890-placement
2026-10-26 14:06:37 +01:00
jschoubben 0e322d087f mesh-build-go: pin mesh-sdk 33395f68, feat/890-placement merged with main (issue 890) 2026-10-26 14:05:38 +01:00
jschoubben 0300a6fd99 pacman: drop the plain-word phrases from replaces (it holds shell commands only, ADR 0245); the descriptions' first lines carry them
mesh/delivery merged: source.main moved to novox/mesh-catalog@2da9329df448
mesh/check being carried to 0300a6fd99b5 on 2da9329df448 | from b1e610b2135a
2026-10-26 14:04:09 +01:00
jschoubben d21b81b00a mesh-build-go: node-build-node also needs bubblewrap and a toolchain installed whole (its mark and lock), as mesh-build-node's integration tests do; pins mesh-sdk 45564e69 (issue 890, review round 1) 2026-10-26 14:02:27 +01:00
mesh-tools b1e610b213 Merge pull request 'Build modules end the steps a previous process left running, and serve build_<lang>_step_end (issue 1890)' (#496) from fix/1890-left-running-steps into main 2026-10-26 12:57:34 +00:00
jschoubben 0134697f97 Build modules: the round ends asks past the step bound; pin the sdk head (issue 1890)
mesh/delivery merged: source.main moved to novox/mesh-catalog@b1e610b2135a
mesh/delivery-group group fix/1890-left-running-steps merged: every member is merged
mesh/check pass 0134697f9739 on b1e610b2135a | carried from a4f5df797b68, mesh-sdk b4cd95e7c4a4 | unit 3132 run, 3022 kept, 7 skipped | integration 31…
2026-10-26 13:52:53 +01:00
jschoubben 21583d14ad Pin mesh-sdk at fix/build-cpu-yields' head (issue 1806) 2026-10-26 13:49:54 +01:00
mesh-tools a4f5df797b Merge pull request 'gitea: the tools' forge account reads every repository (the operator, 2026-10-26)' (#495) from fix/forge-account-reads-all into main 2026-10-26 12:46:37 +00:00
jschoubben ac2c56f2a9 Build modules: steps end by compare-and-set; pin the sdk head with the review's fixes (issue 1890) 2026-10-26 13:44:00 +01:00
jschoubben b4a7bb9dea gitea: say that a repository made private later is granted only at the next start or by hand, and let one unreadable permission refuse its repository, not the whole pass
mesh/delivery merged: source.main moved to novox/mesh-catalog@a4f5df797b68
mesh/check being carried to b4a7bb9dea50 on a4f5df797b68
2026-10-26 13:43:46 +01:00
jschoubben 64b9ded2db Build modules: a per-node processor cap, and each step's threads within it (issue 1806)
The operator's desktop slows while it builds (shanks at 92%, the Go build
runtime at 1650% of 24 processors). The build modules declare cpu-cap
(default shanks=50,g14=50,ace=75), which the composer turns into the runtime's
quota at idle priority; each step's go, node and buildah commands get
GOMAXPROCS and UV_THREADPOOL_SIZE for their part of it, and the status tools
show the limits and what the kernel holds the runtime to.
2026-10-26 13:41:05 +01:00
jschoubben 67d6a5a7c1 mesh-build-go: report the needs a package's check-needs marker names, test it only where the node has them, decline an ask needing what the node lacks (issue 890)
mesh-build-node's integration tests read the Node toolchain installed only where the node holds node-build-node, and
skipped on ace: its package now declares node-build-node beside its tests; the integration digest names it, a node
lacking it leaves the package unmet for the checker to place, and declines a placed ask. Re-pins mesh-sdk feat/890-placement.
2026-10-26 13:37:52 +01:00
jschoubben 5fe5f26e7e Build modules: a restart ends the steps it left running, a step shows what it is, build_*_step_end ends one (issue 1890) 2026-10-26 13:27:46 +01:00
jschoubben b3221b6ab8 pacman: a plain-words search finds pacman_round and pacman_upgrades first (answer on the first line, phrases they replace) 2026-10-26 13:27:09 +01:00
jschoubben 0e7e220d49 gitea: give the tools' account read on every repository, at the start and on the forge's events, so a user's private repository no longer answers its tools 404 2026-10-26 13:22:08 +01:00
mesh-tools 879600202a Merge pull request 'Vault: adopt the hashes of bus accounts it holds no value for, handed sealed by the bus node (issue 1945, ADR 0550 clause 4)' (#494) from feat/1945-user-list-switch into main 2026-10-26 12:21:47 +00:00
mesh-tools 565d0a848a Merge pull request 'Gate: merge turns, a merge waits for a running check of its repository, merges on a carried verdict with no second ask (ADR 0548, issue 568)' (#493) from feat/548-one-check into main 2026-10-26 12:01:28 +00:00
mesh-tools 71e094b2de Merge pull request 'mesh-build-go: count skipped tests per package, so the checker keeps no pass that skipped (issue 890)' (#492) from fix/checker-skips-not-kept into main 2026-10-26 12:01:13 +00:00
mesh-tools f96704fa0f Merge pull request 'mesh-build-node tests: never install a toolchain from the registry in the integration level; skip as NOT RUN when none is installed (issue 890)' (#491) from fix/build-node-tests-offline into main 2026-10-26 11:34:17 +00:00
mesh-tools 86fdf74a0a Merge pull request 'Desk channel: keep what was shown in memory while the store does not answer, so a clearing replaces the urgent notification (issue 384)' (#490) from fix/desk-clear-lost into main 2026-10-26 11:33:47 +00:00
mesh-tools 843281c0e5 Merge pull request 'Knowledge base: withdraw a clause's 'changed by' mark once no accepted record changes it (issue 466)' (#489) from fix/kb-stale-marks into main 2026-10-26 11:33:34 +00:00
jschoubben e9279ce3c2 gate: find an approved merge by its ask, not by which member's run was seen first, so a group's turns are written before its first member is handed at every look (ADR 0548 §3; issue 568, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-catalog@565d0a848a85
mesh/delivery-group group feat/548-one-check merged: every member is merged
mesh/check checking e9279ce3c209 on 565d0a848a85 | novox/mesh-catalog#493 in group feat/548-one-check
2026-10-26 12:32:46 +01:00
jschoubben c9414b1c98 Vault: pin mesh-sdk at the switch group's head c2a410bb, one sdk for the adoption and the switch (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-catalog@879600202a09
mesh/delivery-group group feat/1945-user-list-switch merged: every member is merged
mesh/check checking c9414b1c9856 on 879600202a09 | novox/mesh-catalog#494 in group feat/1945-user-list-switch
2026-10-26 12:31:56 +01:00
jschoubben ea53b2f920 mesh-build-go: each package counts its skipped tests (issue 890)
mesh/delivery merged: source.main moved to novox/mesh-catalog@71e094b2de59
mesh/delivery-group group fix/checker-skips-not-kept merged: every member is merged
mesh/check failed: ace: the check of novox/mesh-catalog#492 refused: the installed go version could not be read: `go version` with GOTOOLCHAIN=local f…
2026-10-26 12:29:19 +01:00
jschoubben 99cc073e2a gate: pin mesh-sdk at the group's head 9c61918ac36f, where a turn never holds the check of heads waiting in it (ADR 0548 §3; issue 568, review round 1) 2026-10-26 12:15:20 +01:00
jschoubben b94150fcdb gitea: the merge tool reads a carried verdict's line as judged on the new trunk, so it merges a carried verdict (ADR 0548 §8; issue 568, review round 1) 2026-10-26 12:15:20 +01:00
jschoubben a01715ddf5 Skip the Node build module's toolchain tests by name where no Node toolchain is installed, instead of installing one from the registry in a level that has no network; merge three tests that share one fixture
mesh/delivery merged: source.main moved to novox/mesh-catalog@f96704fa0ff4
mesh/check checking a01715ddf508 on 565d0a848a85
On ace, which holds no Node build module, five integration tests each spent 70 s in npm's network retries and failed every mesh-catalog check run there (five verdicts since 00:07). The tests now copy the node's installed toolchain or are skipped as NOT RUN; the registry is never reached. Three tests sharing the fixture are one, so the toolchain is copied three times instead of five (10 integration tests in the module, now 8).
2026-10-26 12:05:09 +01:00
jschoubben a30c168512 desk-channel: take the work when the store does not answer, and keep what it holds at the next work and the minute's standing
mesh/delivery merged: source.main moved to novox/mesh-catalog@86fdf74a0a83
mesh/check checking a30c16851246 on 565d0a848a85
Review round 1: offering the work again let a redelivered raise follow its own clearing and
replace it, and a redelivery is bounded where an outage is not. The work is now taken; what
was shown stays in memory, recalled first, and is kept at the start of the next work and at
the minute's standing, stopping at the first write the store does not answer. A desk that
restarts before then loses where those are shown, and its log said how many it held.
2026-10-26 11:46:21 +01:00
jschoubben f885250913 desk-channel: keep where a message is shown when the store does not answer, so its clearing still replaces it
On 2026-10-26 09:18 the bus reloading its accounts made the desk's state store answer
'no responders' just as the urgent 'A new build of mesh-controller cannot be delivered'
was shown. The desk logged that it could not keep the notification ids and acked the work;
the router's clearing at 09:34 then found no record, was shown as a new notification, and
the urgent one stayed on the operator's screen for hours.

Now what was shown and not kept is held in memory, recalled before the store by edits,
reminders, clicks and a notifier restart, kept at the next piece of work, and its work is
offered again until the store keeps it, without being shown twice. An edit of a message
nothing records is said in the journal.
2026-10-26 11:46:09 +01:00
jschoubben 7627503d9d gate: pin mesh-sdk at the group's head f5c233a9288c, which holds the merge turns and the carried verdict (ADR 0548 §10; issue 568) 2026-10-26 11:45:30 +01:00
jschoubben 351c7c4bf4 gate: deliveries and show say an approved merge waits for the check of the named delivery on the named repository, and the turns bucket is read by the checker alone, written by the gate alone (ADR 0548 §3, §9; issue 568 G6) 2026-10-26 11:44:29 +01:00
jschoubben 55a26b22cf gate: delivery_turns reads every merge turn with the check it waits for, and delivery_hand_now hands a held merge at once on the operator's word (ADR 0548 §4; issue 568 G5) 2026-10-26 11:43:40 +01:00
jschoubben 100e74f283 gate: an approved merge whose trunk moved waits for its carried verdict and merges on it with no second ask, never on the old one (ADR 0548 §8; issue 568 G4) 2026-10-26 11:43:18 +01:00
jschoubben c8e1e2b2fa gate: a merge held past 30 minutes by another delivery's check is a stalled line naming that check and since when, said as delivery.<id>.stalled (ADR 0548 §4; issue 568 G3) 2026-10-26 11:41:42 +01:00
jschoubben d0d00f922a gate: an approved merge writes its turn on every repository it goes into before it is handed, kept for a group until its last member merged, read by the checker (ADR 0548 §3; issue 568 G2) 2026-10-26 11:41:26 +01:00
jschoubben c2bd7760b2 gate: hand no approved merge into a repository a running check of other heads judges, and hand it on that check's end (ADR 0548 §2; issue 568 G1) 2026-10-26 11:40:35 +01:00
jschoubben 447e6a13cd Knowledge base: withdraw a clause's changed mark when its changing record's accepted links drop it, on every judgement and by a tool (stale marks)
mesh/delivery merged: source.main moved to novox/mesh-catalog@843281c0e5f0
mesh/check checking 447e6a13cd68 on 565d0a848a85
2026-10-26 11:22:12 +01:00
80 changed files with 2050 additions and 124 deletions
+107 -9
View File
@@ -15,6 +15,13 @@ package main
// the desk says in its standing when that happened (`answers-lost-at`), and the router, which alone knows
// what is still open and unanswered, says that again under its handle — replacing the notification in place,
// so its buttons work again. Nothing resolved, silenced or cancelled comes back.
// - **A store that does not answer does not lose where a message is shown, while the desk runs.** A message
// is shown first and its notification ids kept after; when the state store does not answer then (the bus
// reloading its accounts, 2026-10-26 09:18: the very outage that raised the condition), the work is taken
// all the same and the ids are held in memory, recalled before the store, and kept at the start of the next
// work and at the minute's standing. Otherwise the clearing of that message finds no notification to
// replace and the urgent one stays on screen for good. A desk that restarts while it still holds some loses
// where those are shown; the log says so when they are held.
// - **Standing**: it can send when machines are named in its settings; which of them answered last is in its
// status. A message no machine took is said to the router as failed, and tried again by it.
@@ -93,6 +100,9 @@ type Desk struct {
pause map[string]pauseSeen
// lost is when each machine's notifier last started again: the answers its notifications waited for are lost.
lost map[string]time.Time
// unkept is what was shown and not yet kept in the store, because it did not answer: recalled from here
// before the store, and kept at the next piece of work.
unkept map[string]Shown
}
func (d *Desk) init() {
@@ -107,6 +117,9 @@ func (d *Desk) init() {
if d.lost == nil {
d.lost = map[string]time.Time{}
}
if d.unkept == nil {
d.unkept = map[string]Shown{}
}
if d.Now == nil {
d.Now = time.Now
}
@@ -138,6 +151,7 @@ func (d *Desk) Take(subject string, body json.RawMessage) error {
d.Logf("[desk-channel] work on %s could not be read; taken and dropped", subject)
return nil
}
d.KeepUnkept()
machines := d.Machines()
if m.Secret {
// Shown only where the notifier said lately it is not paused (the review of 2026-10-09): one held back
@@ -167,12 +181,16 @@ func (d *Desk) Take(subject string, body json.RawMessage) error {
switch {
case edit:
var err error
if before, err = d.Store.Get(m.Handle); err != nil {
if before, err = d.recall(m.Handle); err != nil {
return err
}
if before == nil {
d.Logf("[desk-channel] an edit of %s, which nothing here records as shown: shown as a new message; "+
"a notification it was to replace, if any, stays", m.Handle)
}
case !m.Secret:
var err error
if before, err = d.Store.Get(m.Handle); err != nil {
if before, err = d.recall(m.Handle); err != nil {
d.Logf("[desk-channel] what is shown under this handle could not be read (%v): shown as a new message", err)
before = nil
}
@@ -218,12 +236,91 @@ func (d *Desk) Take(subject string, body json.RawMessage) error {
// Shown once and kept nowhere of this module's (a link's code): no edit or re-showing reaches it.
return nil
}
if err := d.Store.Put(m.Handle, s); err != nil {
d.Logf("[desk-channel] shown, and what was shown could not be kept: an edit of it will not reach it: %v", err)
}
d.keep(m.Handle, s)
return nil
}
// keep keeps what a handle shows in the store. When the store does not answer it is held in memory, where an
// edit, a reminder and a click still find it, and kept later (KeepUnkept). The work is taken either way: offered
// again it would race the message's own later edits, and a redelivery is bounded where an outage is not.
func (d *Desk) keep(handle string, s Shown) {
err := d.Store.Put(handle, s)
d.mu.Lock()
defer d.mu.Unlock()
if err == nil {
delete(d.unkept, handle)
return
}
d.unkept[handle] = s
d.Logf("[desk-channel] shown, and where it is shown could not be kept yet (%v): held in memory, so its edit "+
"and clearing still reach it, and kept when the store answers; %d held, lost if the desk restarts first",
err, len(d.unkept))
}
// KeepUnkept keeps in the store what it did not answer for before: at the start of each piece of work and at
// the minute's standing. It stops at the first write the store does not answer, so one call waits on it once.
func (d *Desk) KeepUnkept() {
d.init()
d.mu.Lock()
held := make(map[string]Shown, len(d.unkept))
for h, s := range d.unkept {
held[h] = s
}
d.mu.Unlock()
for h, s := range held {
if err := d.Store.Put(h, s); err != nil {
return
}
d.mu.Lock()
if cur, ok := d.unkept[h]; ok && sameShown(cur, s) {
delete(d.unkept, h)
}
d.mu.Unlock()
d.Logf("[desk-channel] where %s is shown is kept now", h)
}
}
func (d *Desk) unkeptOf(handle string) (Shown, bool) {
d.mu.Lock()
defer d.mu.Unlock()
s, ok := d.unkept[handle]
return s, ok
}
// recall is what a handle shows: from memory when the store did not keep it yet, else from the store.
func (d *Desk) recall(handle string) (*Shown, error) {
if s, ok := d.unkeptOf(handle); ok {
return &s, nil
}
return d.Store.Get(handle)
}
// handles is every handle shown: the store's and those held in memory.
func (d *Desk) handles() ([]string, error) {
hs, err := d.Store.Handles()
d.mu.Lock()
defer d.mu.Unlock()
seen := map[string]bool{}
for _, h := range hs {
seen[h] = true
}
for h := range d.unkept {
if !seen[h] {
hs = append(hs, h)
}
}
if err != nil && len(d.unkept) == 0 {
return nil, err
}
return hs, nil
}
func sameShown(a, b Shown) bool {
x, err1 := json.Marshal(a)
y, err2 := json.Marshal(b)
return err1 == nil && err2 == nil && string(x) == string(y)
}
// show puts one message on one machine's notifier, in place of `replace` when it names one.
func (d *Desk) show(machine string, m asks.Message, replace int) (int, error) {
urgency := "normal"
@@ -308,12 +405,12 @@ func (d *Desk) Chosen(event, machine, ticket string) {
}
func (d *Desk) find(ticket string) (string, *Shown) {
handles, err := d.Store.Handles()
handles, err := d.handles()
if err != nil {
return "", nil
}
for _, h := range handles {
s, err := d.Store.Get(h)
s, err := d.recall(h)
if err != nil || s == nil {
continue
}
@@ -356,9 +453,9 @@ func (d *Desk) Started(event, machine string) int {
d.lost[machine] = d.Now()
d.mu.Unlock()
n := 0
if handles, err := d.Store.Handles(); err == nil {
if handles, err := d.handles(); err == nil {
for _, h := range handles {
if s, err := d.Store.Get(h); err == nil && s != nil && len(s.Message.Buttons) > 0 {
if s, err := d.recall(h); err == nil && s != nil && len(s.Message.Buttons) > 0 {
if _, here := s.On[machine]; here {
n++
}
@@ -472,6 +569,7 @@ func (d *Desk) Status() map[string]any {
"it proves nothing about who clicked: any process of the operator's account can choose an action, so it answers acknowledgements only",
"an answer is taken only on a machine its message was shown on; a message with buttons stays until dismissed",
"what it remembers is never shown again: when a notifier started again, the answers its notifications waited for are lost; the desk says when in its standing (answers-lost-at), and the router says again what is still open and unanswered, replacing those notifications in place — nothing resolved, silenced or cancelled comes back (issue 384)",
"a store that does not answer does not lose where a message is shown while the desk runs: it is held in memory, its edit and clearing still replace it in place, and it is kept when the store answers again; a desk restarted before then loses where those are shown, and its log said how many it held",
"it can send while a named machine's notifier said lately that it is not paused; a link's code is shown only on such a machine, kept nowhere, and sent secret so the notifier keeps it out of its history",
}}
if s.Why != "" {
@@ -3,6 +3,7 @@ package main
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
@@ -363,9 +364,64 @@ func TestAMessageSaidAgainUnderItsHandleReplacesItsNotification(t *testing.T) {
}
}
// downStore is a store that does not answer while down: the bus reloading its accounts (2026-10-26 09:18).
type downStore struct {
memShown
down *bool
}
func (s downStore) Get(h string) (*Shown, error) {
if *s.down {
return nil, errors.New("nats: no responders available for request")
}
return s.memShown.Get(h)
}
func (s downStore) Put(h string, v Shown) error {
if *s.down {
return errors.New("nats: no responders available for request")
}
return s.memShown.Put(h, v)
}
// The review of PR 176: a replace that fails on a machine keeps that machine's notification on record, so a
// later edit or clearing still reaches it; failing everywhere, it is said to the router as failed.
// later edit or clearing still reaches it; failing everywhere, it is said to the router as failed. And
// (2026-10-26 09:18, an urgent "A new build of mesh-controller cannot be delivered" left on screen for hours
// after its clearing): a message shown while the store does not answer keeps where it is shown in memory, its
// work is taken, its clearing replaces it in place, and it is kept once the store answers.
func TestAFailedReplaceKeepsTheNotificationItWasToReplace(t *testing.T) {
// The raise, then its clearing, taken while the store does not answer; the store then stays down or comes
// back. Each is taken (never offered again, where a redelivered raise would follow its clearing), the
// clearing replaces the urgent notification in place, and the screen and the store end on the clearing.
for _, back := range []bool{false, true} {
t.Run(fmt.Sprintf("the store does not answer, back %v", back), func(t *testing.T) {
d, shown, _, store := newDesk(t, "laptop")
down := true
d.Store = downStore{store, &down}
raised, _ := json.Marshal(asks.Message{Handle: "m9", Title: "A new build cannot be delivered", Urgent: true})
if err := d.Take(asks.ChannelSubject(asks.Send, Kind), raised); err != nil {
t.Fatalf("the raise was not taken: %v", err)
}
cleared, _ := json.Marshal(asks.Message{Handle: "m9", Title: "Cleared: a new build can be delivered"})
if err := d.Take(asks.ChannelSubject(asks.Edit, Kind), cleared); err != nil {
t.Fatalf("the clearing was not taken: %v", err)
}
d.KeepUnkept() // the minute's standing, store still down
down = !back
d.KeepUnkept()
take(t, d, asks.Send, asks.Message{Handle: "m10", Title: "another"})
if len(*shown) != 3 || (*shown)[1].args["replace_id"] != 1 || (*shown)[1].args["summary"] != "Cleared: a new build can be delivered" {
t.Fatalf("the clearing did not replace the urgent notification: %+v", *shown)
}
s, kept := store["m9"]
if back != kept || (kept && s.Message.Title != "Cleared: a new build can be delivered") {
t.Fatalf("store back %v, kept %v: %+v", back, kept, store)
}
if got, _ := d.recall("m9"); got == nil || got.Message.Title != "Cleared: a new build can be delivered" {
t.Fatalf("recalled %+v", got)
}
})
}
d, shown, said, store := newDesk(t, "laptop", "desktop")
take(t, d, asks.Send, asks.Message{Handle: "m1", Title: "2 failed services on shanks"})
first := store["m1"].On
@@ -100,6 +100,7 @@ func run(d *Desk) {
}()
go func() {
for {
d.KeepUnkept()
d.SayStanding()
time.Sleep(time.Minute)
}
+23
View File
@@ -275,6 +275,29 @@ reported no key gets no entry, and its client says it waits. Keys titled `mesh:`
a key added by hand never is. Every consumer is made again every ten minutes and at once when a key changes.
`go test ./cmd/forge-account-provider ./internal/forgeaccount` runs it against a fake forge.
## The tools' account reads every repository (the operator, 2026-10-26)
The tools act as the plain account `mesh-tools` (ADR 0365 §2, issue 469), which writes to the organisation `novox`
through its team. Any other repository it saw only when public, so a private repository of a user answered every tool
404 and was missing from every list. `readall.ts` makes the account a collaborator that **reads only** on each
repository it cannot read: one repository at a time, no organisation membership, the least the forge offers.
- **When:** one pass over every repository of every user and organisation at the start (a forge not answering yet is
asked again a minute apart until one pass ran), and on the forge's system webhook: `forge.repository.created` and
`forge.fork` at once, and the first event of any repository this process has not seen readable (the forge may not
send a creation for every owner). Nothing polls for it.
- **Not caught:** a repository made private after it was seen readable. The forge sends no event for a change of
visibility, and the start's pass already saw it readable, so its events ask nothing. It is granted at the module's
next start, or now by `gitea_grant_read`; `gitea_unreadable_repos` shows it meanwhile. A repository whose permission
the forge will not answer is named as refused (or, in the listing, with permission `unknown`) and the pass goes on.
- **With what:** the site administrator's login (`MESH_GITEA_ADMIN_PASSWORD_FILE`), because adding a collaborator asks a
repository's administration, which on another user's repository only the site administrator holds. It is used for
four calls alone: list every repository, read one, read the account's permission on one, add the account as a read
collaborator. The tools' own calls keep their token.
- **Tools:** `gitea_unreadable_repos` (every repository the account cannot read, the permission the forge gives it
there and why) and `gitea_grant_read` (the same grant by hand, now, on every repository or on those named; each grant
read back from the forge, a refusal answered in the forge's own words).
## Files it does not know
Each directory or archive this module declares says how it treats a file it did not write (ADR 0422, ADR 0416).
+60
View File
@@ -853,6 +853,25 @@ interface AdminResponse {
readonly body: any;
}
/** A repository as the site administrator sees it: what saying why an account cannot read it needs (readall.ts). */
export interface ForgeRepository {
full_name: string;
owner: string;
name: string;
private: boolean;
archived: boolean;
/** The owner's own visibility: public, limited or private. A public repository of an owner not public is hidden
* from whoever is no member. */
owner_visibility: string;
}
function forgeRepository(r: any): ForgeRepository {
const owner = String(r?.owner?.login ?? r?.owner?.username ?? "");
const name = String(r?.name ?? "");
return { full_name: String(r?.full_name ?? `${owner}/${name}`), owner, name, private: r?.private === true,
archived: r?.archived === true, owner_visibility: String(r?.owner?.visibility ?? "public") };
}
/**
* The forge's admin client, over **basic auth** — gitea's own code, living in the module, used only
* by the provisioner (novox/hq ADR 0048/0076).
@@ -970,6 +989,47 @@ export class GiteaAdmin {
return repos;
}
// ---- The tools' account reads every repository (readall.ts) ----
/** Every repository on the forge, of every user and organisation: the search as the site administrator, whom the
* forge answers every repository, private ones and those of private owners included. Only a read. */
async everyRepository(): Promise<ForgeRepository[]> {
const all: ForgeRepository[] = [];
for (let page = 1; page <= 200; page++) {
const res = await this.request(`/repos/search?limit=50&page=${page}`);
if (res.status !== 200) GiteaAdmin.fail("/repos/search", res);
const list: any[] = Array.isArray(res.body?.data) ? res.body.data : [];
all.push(...list.map(forgeRepository));
if (list.length < 50) break;
}
return all;
}
/** One repository as the site administrator sees it, or null when the forge holds none by that name. */
async repository(owner: string, repo: string): Promise<ForgeRepository | null> {
const res = await this.request(repoPath(owner, repo));
if (res.status === 404) return null;
if (res.status !== 200) GiteaAdmin.fail(repoPath(owner, repo), res);
return forgeRepository(res.body);
}
/** What the forge itself says an account may do on a repository: none, read, write, admin or owner. */
async repositoryPermission(owner: string, repo: string, user: string): Promise<string> {
const path = `${repoPath(owner, repo)}/collaborators/${encodeURIComponent(user)}/permission`;
const res = await this.request(path);
if (res.status !== 200) GiteaAdmin.fail(path, res);
return String(res.body?.permission ?? "none");
}
/** Make an account a collaborator of a repository that reads only. The forge asks a repository's administrator for
* it, which on a repository of another user only the site administrator is. Idempotent: the forge answers 204 again. */
async addReadCollaborator(owner: string, repo: string, user: string): Promise<void> {
const path = `${repoPath(owner, repo)}/collaborators/${encodeURIComponent(user)}`;
const res = await this.request(path, { method: "PUT", body: JSON.stringify({ permission: "read" }) });
if (res.status === 204 || res.status === 200) return;
GiteaAdmin.fail(path, res);
}
/** Ensure the npm-owner org exists. 201 created, 2xx/404-then-created, and 422/409 (a concurrent
* create won the race) are all success. */
async ensureOrg(name: string): Promise<void> {
+27 -2
View File
@@ -60,6 +60,8 @@ import { HOOK_PATH, HOOK_PORT, Pace, claimHead, ensureSystemHook, eventsTool, ke
import { keepSiblings, noteComposed, noteVerdict, readSiblings, siblingsFile, siblingsOf, weighSiblings } from "./siblings.js";
import { bindInBackground } from "./bind.js";
import { announcementOf, keepTook, lateVerdictComment, outOfDateWords, sameCommit, tookFile, weighTrunks } from "./trunk.js";
import { ReadAll, readAllTools } from "./readall.js";
import { TOOLS_ACCOUNT } from "./tools-account.js";
// Without a way to a token — configured, or mintable with the admin account (token.ts) — there is
// nothing to watch; log and stay quiet rather than crash the runtime. With one, the first poll mints
@@ -500,8 +502,14 @@ if (!gitea) pace.unheard("the forge is not watched: no way to a token", false);
// Said with `emit`, not the SDK's `keeps`, which the toolchain's SDK does not have yet: switch to `keeps` once that SDK is 0.1.15 or later (issue 1852).
await emit("states", statesAnnouncement(process.env.MESH_MODULE || "gitea", process.env.MESH_NODE));
// **The tools' account reads every repository** (readall.ts; the operator, 2026-10-26): granted with the site
// administrator's login, which the module holds for setting the forge up and uses here for that grant alone. Made on
// first use, so a module whose admin login is missing still serves the rest and the tools say why.
let readAllOnce: ReadAll | null = null;
const readAll = (): ReadAll => (readAllOnce ??= new ReadAll(GiteaAdmin.fromEnv(), process.env.MESH_GITEA_TOOLS_USER || TOOLS_ACCOUNT));
// gitea_events is served whether or not the forge is watched: it says so (novox/hq issue 729).
registerModuleTools("gitea", () => [eventsTool(pace), ...(gitea ? [commitStatusTool({
registerModuleTools("gitea", () => [eventsTool(pace), ...readAllTools(readAll), ...(gitea ? [commitStatusTool({
setCommitStatus: (owner, repo, sha, status) => (gitea as GiteaClient).setCommitStatus(owner, repo, sha, status),
gates,
}), mergeTool(gitea, { gates })] : [])]);
@@ -543,7 +551,12 @@ async function listenForForge(client: GiteaClient): Promise<boolean> {
const server = receiver({ secret, port, heard: (kind, delivery) => pace.heard(kind, delivery),
refused: (reason, from) => pace.refused(reason, from),
publish: (event) => emit(event.name, event.body),
act: (event) => actOn(client, event) });
act: async (event) => {
// A repository made, or one not yet seen readable, is granted read before anything acts on it (readall.ts).
await Promise.resolve().then(() => readAll().onEvent(event)).catch((err) =>
console.error(`[gitea] whether ${TOOLS_ACCOUNT} reads the repository of ${event.name} could not be settled — ${err instanceof Error ? err.message : err}`));
await actOn(client, event);
} });
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(port, host, () => resolve());
@@ -594,6 +607,18 @@ if (gitea) {
// of the node starts left gitea_commit_status, gitea_merge_pull_request and gitea_events served nowhere. Until it is
// listening, Pace keeps the polls at 30 s, so nothing is missed meanwhile; listenForForge says its own failure.
void listenForForge(client);
// The start's one pass over every repository (readall.ts), not awaited for the same reason. A forge not answering yet
// is asked again, a minute apart, until one pass has run; after that only the forge's events grant.
const readAllAtStart = (): void => void Promise.resolve().then(() => readAll().atStart()).catch((err) => {
console.error(`[gitea] whether ${TOOLS_ACCOUNT} reads every repository could not be settled; asked again in a minute — ${err instanceof Error ? err.message : err}`);
setTimeout(readAllAtStart, 60_000);
});
try {
readAll();
readAllAtStart();
} catch (err) {
console.error(`[gitea] ${TOOLS_ACCOUNT} is not given read on every repository: no site administrator's login — ${err instanceof Error ? err.message : err}`);
}
// Main takes no push and only the tools' account merges, in every repository, kept and not only when a token is
// minted: at the start and every ten minutes after, so a repository made later gets it too (issue 948, ADR 0398 §2).
// Only while the setting keep-main-rules is true (mainrules.ts); false, the default, leaves every repository's rule as set.
+210
View File
@@ -0,0 +1,210 @@
// The tools' account reads every repository on the forge (the operator, 2026-10-26: "the mesh's forge account should
// get read access to all repositories").
//
// Since issue 469 the tools act as the plain account `mesh-tools`, which writes to the organisation novox's
// repositories through its team (tools-account.ts). Every other repository it saw only when public: a private
// repository of a user (jschoubben/car-hunter, jschoubben/snake) answered its tools 404 and was missing from every
// list, while that user's public ones were there. So the module makes its account a collaborator that reads only on
// each repository it cannot read, the least the forge offers: one repository, read, no organisation membership.
//
// - **at the start**, one pass over every repository of every user and organisation;
// - **on the forge's own event** (the system webhook, webhook.ts): `forge.repository.created` and `forge.fork` at
// once, and the first event of any repository this process has not yet seen readable, since the forge may not
// send a repository's creation for every owner. Nothing polls for it.
//
// **Not caught: a repository made private later.** The forge sends no event for a change of visibility, and a public
// repository is already seen readable by the start's pass, so its events ask nothing. Such a repository is granted at
// the module's next start, or now by `gitea_grant_read`; `gitea_unreadable_repos` shows it meanwhile.
//
// **The grant needs the site administrator's login.** Adding a collaborator asks the repository's administration,
// which on another user's repository only the site administrator holds; the module holds that login for setting the
// forge up (MESH_GITEA_ADMIN_PASSWORD_FILE). Here it is used through `ReadAllForge` alone: list the repositories, read
// the account's permission on one, add the account as a collaborator that reads. Nothing else, and never for the tools'
// own calls, which keep their token.
//
// What the forge refuses (a repository that went meanwhile, an archived one if the forge will not take a collaborator
// on it) is said per repository and not retried; the tool `gitea_unreadable_repos` shows what is left, and
// `gitea_grant_read` grants again now.
import type { ForgeRepository } from "./client.js";
/** What the module asks of the forge for this, with the site administrator's login (GiteaAdmin, client.ts). */
export interface ReadAllForge {
everyRepository(): Promise<ForgeRepository[]>;
repository(owner: string, repo: string): Promise<ForgeRepository | null>;
repositoryPermission(owner: string, repo: string, user: string): Promise<string>;
addReadCollaborator(owner: string, repo: string, user: string): Promise<void>;
}
/** A repository the account cannot read, and why. */
export interface Unreadable {
repository: string;
permission: string;
private: boolean;
archived: boolean;
why: string;
}
export interface Granted {
/** Repositories looked at. */
checked: number;
/** Made readable now, by a read collaborator. */
granted: string[];
/** The forge refused the grant: each repository and the forge's own words. */
refused: { repository: string; why: string }[];
}
const READS = new Set(["read", "write", "admin", "owner"]);
/** Why the account cannot read a repository, from what the forge says of it. */
export function whyUnreadable(repo: ForgeRepository, account: string): string {
const parts: string[] = [];
if (repo.private) parts.push(`a private repository of ${repo.owner}, on which ${account} is no collaborator and in no team that covers it`);
else if (repo.owner_visibility !== "public") {
parts.push(`a public repository of ${repo.owner}, whose own visibility is ${repo.owner_visibility}: the forge shows it to members only`);
} else parts.push(`the forge answers ${account} no read on it`);
if (repo.archived) parts.push("archived");
return parts.join("; ");
}
export class ReadAll {
/** Repositories this process has seen the account read, by full name: an event of one of them asks nothing more. */
readonly readable = new Set<string>();
constructor(private readonly forge: ReadAllForge, readonly account: string,
private readonly log: (line: string) => void = (l) => console.log(`[gitea] ${l}`)) {}
/** Every repository the account cannot read, and why, as the forge says it now. */
async unreadable(): Promise<{ checked: number; unreadable: Unreadable[] }> {
const repos = await this.forge.everyRepository();
const out: Unreadable[] = [];
for (const repo of repos) {
try {
const u = await this.judge(repo);
if (u) out.push(u);
} catch (err) {
// A permission the forge would not answer is said for that repository; the rest are still looked at.
out.push({ repository: repo.full_name, permission: "unknown", private: repo.private, archived: repo.archived,
why: `the forge did not answer ${this.account}'s permission on it — ${err instanceof Error ? err.message : String(err)}` });
}
}
return { checked: repos.length, unreadable: out };
}
/** Grant read on every repository the account cannot read (or on those named, `owner/name`). */
async grant(only?: string[]): Promise<Granted> {
let repos: ForgeRepository[];
if (only && only.length) {
repos = [];
for (const full of only) {
const [owner, name, ...rest] = full.split("/");
if (!owner || !name || rest.length) throw new Error(`"${full}" is not a repository: say owner/name`);
const repo = await this.forge.repository(owner, name);
if (!repo) throw new Error(`the forge holds no repository ${full}`);
repos.push(repo);
}
} else repos = await this.forge.everyRepository();
const result: Granted = { checked: repos.length, granted: [], refused: [] };
for (const repo of repos) {
let u: Unreadable | null;
try {
u = await this.judge(repo);
} catch (err) {
// One repository whose permission cannot be read is a refusal of its own, never the end of the pass.
result.refused.push({ repository: repo.full_name, why: `its permission could not be read — ${err instanceof Error ? err.message : String(err)}` });
continue;
}
if (!u) continue;
try {
await this.forge.addReadCollaborator(repo.owner, repo.name, this.account);
} catch (err) {
result.refused.push({ repository: repo.full_name, why: err instanceof Error ? err.message : String(err) });
continue;
}
// Read back: the forge's own word that the account reads it now, never the grant's answer alone.
let after: string;
try {
after = await this.forge.repositoryPermission(repo.owner, repo.name, this.account);
} catch (err) {
result.refused.push({ repository: repo.full_name, why: `granted, but its permission could not be read back — ${err instanceof Error ? err.message : String(err)}` });
continue;
}
if (READS.has(after)) {
this.readable.add(repo.full_name);
result.granted.push(repo.full_name);
} else {
result.refused.push({ repository: repo.full_name, why: `the grant was answered, but the forge still gives ${this.account} "${after}"` });
}
}
return result;
}
/** The start's one pass: grant what is missing and say it. Answers the result; a failure is the caller's to say. */
async atStart(): Promise<Granted> {
const r = await this.grant();
this.log(`${this.account} reads every repository: ${r.checked} looked at, ${r.granted.length} granted read now` +
`${r.granted.length ? ` (${r.granted.join(", ")})` : ""}${r.refused.length ? `, ${r.refused.length} refused: ` +
r.refused.map((x) => `${x.repository} — ${x.why}`).join("; ") : ""}`);
return r;
}
/** A forge event: a repository made (or forked), or the first event of one not seen readable, is granted read now. */
async onEvent(event: { name: string; body: unknown }): Promise<void> {
const b = (event.body ?? {}) as { owner?: string; repo?: string };
if (!b.owner || !b.repo) return;
const full = `${b.owner}/${b.repo}`;
if (event.name === "forge.repository.deleted") {
this.readable.delete(full);
return;
}
const made = event.name === "forge.repository.created" || event.name === "forge.fork";
// Seen readable once, its events ask nothing: so a repository made private after that is not caught here (the
// forge says no change of visibility), only at the next start or by gitea_grant_read.
if (!made && this.readable.has(full)) return;
const r = await this.grant([full]);
if (r.granted.length) this.log(`${this.account} was given read on ${full}, on the forge's ${event.name}`);
for (const x of r.refused) this.log(`${this.account} could not be given read on ${x.repository} — ${x.why}`);
}
private async judge(repo: ForgeRepository): Promise<Unreadable | null> {
const permission = await this.forge.repositoryPermission(repo.owner, repo.name, this.account);
if (READS.has(permission)) {
this.readable.add(repo.full_name);
return null;
}
this.readable.delete(repo.full_name);
return { repository: repo.full_name, permission, private: repo.private, archived: repo.archived, why: whyUnreadable(repo, this.account) };
}
}
type Tool = { name: string; description: string; input: Record<string, unknown>; run: (args: Readonly<Record<string, unknown>>) => Promise<unknown> };
/** The two tools: what the account cannot read and why, and the grant now. */
export function readAllTools(readAll: () => ReadAll): Tool[] {
return [
{
name: "gitea_unreadable_repos",
description: "Every repository on the forge the tools' own account (mesh-tools) cannot read, and why: the " +
"permission the forge gives it there, whether the repository is private or archived, and in words the reason " +
"(a private repository of a user with the account no collaborator, an owner not public). Empty when it reads " +
"every repository. Read with the site administrator's login, which alone sees every repository.",
input: {},
run: async () => readAll().unreadable(),
},
{
name: "gitea_grant_read",
description: "Give the tools' own account (mesh-tools) read on every repository it cannot read now, or on the " +
"repositories named: each becomes a collaborator that reads only, the least the forge offers. The module does " +
"this itself at its start and on the forge's event of a repository made; this is the same act by hand. Answers " +
"how many were looked at, which were granted (each read back from the forge) and which the forge refused, in its " +
"own words. Needs the site administrator's login, used for this grant alone.",
input: {
repositories: { type: "array", items: { type: "string" }, description: "owner/name of each repository; empty or absent, every repository" },
},
run: async (args) => {
const named = Array.isArray(args.repositories) ? args.repositories.map(String) : undefined;
return readAll().grant(named);
},
},
];
}
+144 -1
View File
@@ -26,6 +26,18 @@ interface Account { admin: boolean; password: string; active: boolean }
interface Token { id: number; name: string; sha1: string; user: string; scopes: string[] }
interface Team { id: number; name: string; units: any; members: Set<string> }
interface ForgeRepo { owner: string; name: string; full: string; private: boolean; archived?: boolean; ownerVisibility: string; collaborators: Map<string, string> }
const forgeRepos: ForgeRepo[] = [];
const repoFaults = { refuse: new Set<string>(), permission: new Set<string>() };
function addRepo(full: string, isPrivate: boolean, ownerVisibility = "public"): ForgeRepo {
const [owner, name] = full.split("/");
const r = { owner, name, full, private: isPrivate, ownerVisibility, collaborators: new Map<string, string>() };
forgeRepos.push(r);
return r;
}
const repoJSON = (r: ForgeRepo) => ({ full_name: r.full, name: r.name, owner: { login: r.owner, visibility: r.ownerVisibility },
private: r.private, archived: r.archived === true });
function fakeForge() {
const users = new Map<string, Account>([[ADMIN, { admin: true, password: PASSWORD, active: true }]]);
const tokens: Token[] = [];
@@ -61,6 +73,13 @@ function fakeForge() {
return null;
};
const isAdmin = (u: string | null) => u !== null && users.get(u)?.admin === true;
const permissionOf = (u: string | null, r: ForgeRepo): string => {
if (isAdmin(u)) return "admin";
if (u === r.owner) return "owner";
if (r.owner === "novox" && teams.some((t) => t.members.has(u ?? "") && t.units.units_map?.["repo.code"] === "write")) return "write";
if (u && r.collaborators.has(u)) return r.collaborators.get(u)!;
return !r.private && r.ownerVisibility === "public" ? "read" : "none";
};
const server = createServer(async (req, res) => {
const url = new URL(req.url ?? "/", "http://fake");
@@ -176,7 +195,32 @@ function fakeForge() {
if (!writes) return json(res, 403, { message: "write on code required" });
return json(res, 201, { context: (await read(req)).context, creator: { login: me } });
}
if (req.method === "GET" && p === "/repos/search") return json(res, 200, { data: [] });
// Every repository of every owner, and who reads it (readall.ts): the site administrator reads all; the tools' account
// a repository of novox through its team, a public one of a public owner, and one it is a collaborator of.
if (req.method === "GET" && p === "/repos/search") {
const page = Number(url.searchParams.get("page") ?? "1");
const seen = forgeRepos.filter((r) => permissionOf(me, r) !== "none");
return json(res, 200, { data: page === 1 ? seen.map(repoJSON) : [] });
}
m = p.match(/^\/repos\/([^/]+)\/([^/]+)(?:\/collaborators\/([^/]+)(\/permission)?)?$/);
if (m) {
const repo = forgeRepos.find((r) => r.owner === decodeURIComponent(m![1]) && r.name === decodeURIComponent(m![2]));
if (!repo || permissionOf(me, repo) === "none") return json(res, 404, { message: "repository does not exist" });
if (req.method === "GET" && !m[3]) return json(res, 200, repoJSON(repo));
const user = decodeURIComponent(m[3] ?? "");
if (req.method === "GET" && m[4]) {
if (repoFaults.permission.has(repo.full)) return json(res, 500, { message: "the forge is not well" });
if (!isAdmin(me) && me !== user) return json(res, 403, { message: "only the user or an administrator" });
return json(res, 200, { permission: permissionOf(user, repo) });
}
if (req.method === "PUT" && !m[4]) {
// gitea's reqAdmin: the repository's administration, which on a repository of another user the site administrator alone holds.
if (!isAdmin(me) && me !== repo.owner) return json(res, 403, { message: "user should be an owner or a collaborator with admin write of a repository" });
if (repoFaults.refuse.has(repo.full)) return json(res, 422, { message: "the forge would not have it" });
repo.collaborators.set(user, (await read(req))?.permission ?? "write");
return json(res, 204, null);
}
}
return json(res, 404, { message: "no such route in the fake" });
});
return new Promise<{ url: string; users: typeof users; tokens: typeof tokens; teams: typeof teams; calls: string[]; mainRules: typeof mainRules;
@@ -402,3 +446,102 @@ test("a mint with keep-main-rules false (the default) writes no branch protectio
const writes = forge.calls.slice(from).filter((c: string) => /branch_protections/.test(c) && !/^GET /.test(c) && !/^(GET|HEAD) /.test(c));
assert.deepEqual(writes, []);
});
// The tools' account reads every repository (readall.ts; the operator, 2026-10-26): jschoubben/car-hunter and
// jschoubben/snake, private repositories of a user, answered the tools 404 while that user's public ones were seen.
test("the start's pass gives the tools' account read, as a collaborator that reads only, on each repository it could not read, and on no other", async () => {
const env = await environment();
await fresh(env).listRepos(); // the account and its team exist, as after any mint
forgeRepos.splice(0);
addRepo("novox/hq", true);
addRepo("jschoubben/txt-game", false);
addRepo("jschoubben/car-hunter", true);
addRepo("jschoubben/snake", true);
addRepo("shy/open", false, "limited");
addRepo("elsewhere/secret", true);
const { GiteaAdmin } = await import("../dist/client.js");
const { ReadAll, readAllTools } = await import("../dist/readall.js");
const lines: string[] = [];
const readAll = new ReadAll(GiteaAdmin.fromEnv(env), TOOLS_ACCOUNT, (l: string) => lines.push(l));
const [unreadableTool, grantTool] = readAllTools(() => readAll);
// What the tool says before: the four the account cannot read, each with why.
const before = await unreadableTool.run({}) as any;
assert.equal(before.checked, 6);
assert.deepEqual(before.unreadable.map((u: any) => u.repository).sort(),
["elsewhere/secret", "jschoubben/car-hunter", "jschoubben/snake", "shy/open"]);
assert.match(before.unreadable.find((u: any) => u.repository === "jschoubben/car-hunter").why,
/a private repository of jschoubben, on which mesh-tools is no collaborator/);
assert.match(before.unreadable.find((u: any) => u.repository === "shy/open").why, /visibility is limited/);
assert.deepEqual((await fresh(env).listAllRepos()).map((r) => r.full_name).sort(), ["jschoubben/txt-game", "novox/hq"]);
// The account's own token cannot grant itself: the forge asks the repository's administration (the reason for the admin login).
const own = (await readFile(join(env.MESH_GITEA_STATE_DIR, "token"), "utf8")).trim();
const self = await fetch(`${forge.url}/api/v1/repos/jschoubben/car-hunter/collaborators/${TOOLS_ACCOUNT}`,
{ method: "PUT", headers: { Authorization: `token ${own}`, "Content-Type": "application/json" }, body: JSON.stringify({ permission: "read" }) });
assert.equal(self.status, 404, "a repository the account cannot read is not even there for it");
const from = forge.calls.length;
const r = await readAll.atStart();
assert.deepEqual(r.granted.sort(), ["elsewhere/secret", "jschoubben/car-hunter", "jschoubben/snake", "shy/open"]);
assert.deepEqual(r.refused, []);
const puts = forge.calls.slice(from).filter((c) => c.startsWith("PUT "));
assert.equal(puts.length, 4, "the repositories it read already were left alone");
assert.ok(puts.every((c) => c.endsWith(` as ${ADMIN}`)), "only the administrator's login grants");
for (const repo of forgeRepos.filter((x) => x.collaborators.size)) assert.equal(repo.collaborators.get(TOOLS_ACCOUNT), "read", `${repo.full} reads only`);
assert.match(lines.join("\n"), /6 looked at, 4 granted read now/);
// After: the account's own token sees every repository, and the tool names none.
assert.equal((await fresh(env).listAllRepos()).length, 6);
assert.deepEqual(((await unreadableTool.run({})) as any).unreadable, []);
// Granting again asks nothing of the forge but reads.
const again = forge.calls.length;
assert.deepEqual(((await grantTool.run({})) as any).granted, []);
assert.ok(forge.calls.slice(again).every((c) => c.startsWith("GET ")));
});
test("on the forge's events: a repository made is granted read at once, a known readable one asks nothing, a refusal is said, not thrown", async () => {
const env = await environment();
await fresh(env).listRepos();
forgeRepos.splice(0);
addRepo("jschoubben/txt-game", false);
const { GiteaAdmin } = await import("../dist/client.js");
const { ReadAll, readAllTools } = await import("../dist/readall.js");
const lines: string[] = [];
const readAll = new ReadAll(GiteaAdmin.fromEnv(env), TOOLS_ACCOUNT, (l: string) => lines.push(l));
await readAll.atStart();
const cases: { name: string; event: string; repo: string; make?: boolean; refuse?: boolean; unanswered?: boolean; asks: boolean; reads: string }[] = [
{ name: "a repository made", event: "forge.repository.created", repo: "jschoubben/new-private", make: true, asks: true, reads: "read" },
{ name: "a fork made", event: "forge.fork", repo: "someone/fork", make: true, asks: true, reads: "read" },
{ name: "a push to a repository seen readable", event: "forge.push", repo: "jschoubben/txt-game", asks: false, reads: "read" },
{ name: "a push to a private repository never seen", event: "forge.push", repo: "jschoubben/quiet", make: true, asks: true, reads: "read" },
{ name: "a repository the forge refuses the grant on", event: "forge.repository.created", repo: "jschoubben/stuck", make: true, refuse: true, asks: true, reads: "none" },
{ name: "a repository whose permission the forge will not answer", event: "forge.repository.created", repo: "jschoubben/mute", make: true, unanswered: true, asks: true, reads: "none" },
];
for (const c of cases) {
if (c.make) addRepo(c.repo, true);
if (c.refuse) repoFaults.refuse.add(c.repo);
if (c.unanswered) repoFaults.permission.add(c.repo);
const from = forge.calls.length;
const [owner, repo] = c.repo.split("/");
await readAll.onEvent({ name: c.event, body: { owner, repo } });
assert.equal(forge.calls.length > from, c.asks, `${c.name}: asks the forge`);
assert.equal(forgeRepos.find((x) => x.full === c.repo)!.collaborators.get(TOOLS_ACCOUNT) ?? (c.make ? "none" : "read"), c.reads, c.name);
}
assert.match(lines.join("\n"), /could not be given read on jschoubben\/stuck — .*422/);
const named = await (readAllTools(() => readAll)[1].run({ repositories: ["jschoubben/stuck"] })) as any;
assert.deepEqual(named.refused.map((x: any) => x.repository), ["jschoubben/stuck"]);
await assert.rejects(readAllTools(() => readAll)[1].run({ repositories: ["no-slash"] }), /not a repository/);
assert.match(lines.join("\n"), /could not be given read on jschoubben\/mute — its permission could not be read — .*500/);
// A whole pass goes past the repository it cannot judge: every other one is still granted, and the listing names it.
addRepo("jschoubben/after-mute", true);
const pass = await readAll.grant();
assert.ok(pass.granted.includes("jschoubben/after-mute"), "the pass did not stop at the repository it could not judge");
assert.ok(pass.refused.some((x: any) => x.repository === "jschoubben/mute" && /permission could not be read/.test(x.why)));
const listed = await readAllTools(() => readAll)[0].run({}) as any;
assert.equal(listed.unreadable.find((u: any) => u.repository === "jschoubben/mute").permission, "unknown");
repoFaults.refuse.clear();
repoFaults.permission.clear();
});
+3
View File
@@ -34,6 +34,9 @@ test("both statuses say the head and the trunk commit the verdict judged, first,
const [old] = statusesFor({ ...c, trunk: undefined, gate: { verdict: "pass", summary: "every machine composes" } });
assert.equal(old.description, "pass: every machine composes");
assert.equal(trunkNamed(old.description), undefined);
// ADR 0548 §8: a carried verdict's line names the head on the new trunk first, so the merge tool merges it on that trunk.
assert.equal(trunkNamed(`pass ${HEAD.slice(0, 12)} on ${MOVED.slice(0, 12)} | carried from ${TRUNK.slice(0, 12)}, mesh-catalog 0123456789ab | unit 0 run, 4 kept`),
MOVED.slice(0, 12));
});
test("a trunk's move puts an open head's merge check out of date, naming both commits, unless a check is on its way", () => {
@@ -43,6 +43,7 @@ func (m *Module) askSteps(steps []StepTool) buildstep.AskSteps {
return delivery.ToolchainUsed{Version: installed, Recipe: m.Recipe.Version()}, nil
},
Kept: m.KeptLookup,
Has: m.Has,
Release: func(ws string) error { return m.spaces().Release(ws) },
Logf: m.Say,
}
@@ -96,7 +97,7 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
}
}
return &buildstep.Puller{Source: source, Language: delivery.AskGo, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load,
Capacity: capacity, WatchConfig: watch, Logf: m.Say,
Capacity: capacity, WatchConfig: watch, Logf: m.Say, Has: m.Has,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
if err := m.Check(ctx); err != nil {
return nil, err
@@ -115,3 +116,11 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
func (m *Module) SeatTools(p *buildstep.Puller) []stdio.Tool {
return buildstep.SeatTools(Seat, "Go", p)
}
// StepEndTool ends one step left running (issue 1890); the module's steps state is the one it reads and writes.
const StepEndTool = "build_go_step_end"
// stepScan is the module's steps state as buildstep reads it.
func (m *Module) stepScan() buildstep.Scan {
return buildstep.ScanOf(stdio.State(delivery.StepsState))
}
@@ -183,12 +183,12 @@ func (m *Module) buildEnv(dir string) []string {
} else if modfile, ok := m.pinModfile(dir); ok {
mod += " -modfile=" + modfile // a pin on a pull request's head the fetch placed beside the workspace (pins.go)
}
return []string{
return append([]string{
"CGO_ENABLED=0",
"GOFLAGS=" + recipeFlags + " " + mod,
"GOTOOLCHAIN=local",
"GOMODCACHE=" + filepath.Join(m.Cache, "gomod"),
"GOCACHE=" + filepath.Join(m.Cache, "gobuild"),
"GOPROXY=off",
}
}, m.threadEnv()...)
}
@@ -0,0 +1,36 @@
package main
import (
"git.novox.be/novox/mesh-sdk/go/buildcpu"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
// The processors a step may take (novox/hq ADR 0553, issue 1806). The node-engine holds this module's runtime, with
// every other build module's of the node, in one slice at idle priority under the node's one share of its processors:
// that quota bounds the total. Threads only size each step: the share the composer gives (buildstep.NodeLimits) over
// this module's own capacity, so a step does not wait in the kernel's throttle. Without the composer's share there is
// no slice to size a step to, and no thread limit is set.
// cpuLimits is what one step is held to now.
func (m *Module) cpuLimits() buildcpu.Limits {
slots := 1
switch {
case m.Capacity != nil:
slots = m.Capacity()
case m.Settings != "":
slots = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, nil)()
}
return buildstep.NodeLimits(slots)
}
// threadEnv is the environment (nothing without a share) every go command of a step is given for its threads: GOMAXPROCS sets its -p, the compiler's
// concurrency, a test binary's processors and -parallel.
func (m *Module) threadEnv() []string {
return buildcpu.Env(m.cpuLimits().Threads)
}
// cpuStatus is what the status tool says of it: the limits a step is given, and what the kernel holds this runtime
// to (its unit and its slice), read from its cgroup rather than assumed.
func (m *Module) cpuStatus() map[string]any {
return map[string]any{"limits": m.cpuLimits(), "in-force": buildcpu.ReadInForce()}
}
@@ -11,6 +11,7 @@ import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
@@ -19,6 +20,8 @@ import (
"sort"
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// DigestVersion is the form of the digest's text: changed whenever what the digest hashes changes in kind.
@@ -39,6 +42,33 @@ const besideVariable = "MESH_CHECK_BESIDE"
// test log) with each verdict.
const ReadsOutsideMarker = "reads-outside"
// NeedsMarker is the name of a file in a package's directory that names, one per line, what its integration tests need
// of the node they run on (issue 890; mesh-sdk delivery.AskNeeds): `node-build-node` for tests that read the Node build
// module's installed toolchain. The digest reports it (DigestResult.Needs); a check ask leaves such a package untested on
// a node that lacks it, and the checker asks it again of a node that has it. A marker beside the tests, as
// ReadsOutsideMarker is, and not a manifest field: it is a property of the package's tests, read only where they are
// run, and the manifest describes the module on the machines. The unit level is never given anything of the node (ADR
// 0533 §1), so a marker binds the integration level alone. Blank lines and lines starting with # are not read.
const NeedsMarker = "check-needs"
// readNeeds is what the marker in dir names; nil when there is none.
func readNeeds(dir string) ([]string, error) {
raw, err := os.ReadFile(filepath.Join(dir, NeedsMarker))
if errors.Is(err, fs.ErrNotExist) {
return nil, nil
}
if err != nil {
return nil, err
}
var needs []string
for _, l := range strings.Split(string(raw), "\n") {
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "#") {
needs = append(needs, l)
}
}
return needs, nil
}
// integrationTags are the build tags of the integration level, its run and its digest alike: integration, and isolation
// for the bubblewrap proofs (mesh-sdk's and the build modules'), which a build node can run.
var integrationTags = []string{"integration", "isolation"}
@@ -101,6 +131,8 @@ type DigestResult struct {
// ReadsOutside names the packages that carry the marker: each has a digest that no kept verdict can have (a fresh
// random one), so the check runs it every time.
ReadsOutside []string `json:"reads_outside,omitempty"`
// Needs are the packages whose tests at this level need something of the node, as their NeedsMarker names it.
Needs []delivery.PackageNeeds `json:"needs,omitempty"`
// JudgedBy is the program that ran the digest (ADR 0535 §5): "installed", or "candidate <merge commit>" under the
// runner workspace.
JudgedBy string `json:"judged_by,omitempty"`
@@ -206,12 +238,13 @@ func (m *Module) digestHere(ctx context.Context, workspace, level string) (Diges
if err != nil {
return DigestResult{}, err
}
if err := digestModule(&res, pkgs, tree, header, besideHeader); err != nil {
if err := digestModule(&res, pkgs, tree, header, besideHeader, level == LevelIntegration); err != nil {
return DigestResult{}, err
}
}
sort.Strings(res.WithoutTests)
sort.Strings(res.ReadsOutside)
sort.Slice(res.Needs, func(i, j int) bool { return res.Needs[i].Package < res.Needs[j].Package })
for _, p := range res.ReadsOutside {
if m.Say != nil {
m.Say("the %s level of %s always runs: its directory carries %s, so its tests read files outside it that no digest names", level, p, ReadsOutsideMarker)
@@ -256,7 +289,7 @@ func parsePackages(out string) (map[string]*goPackage, error) {
// digestModule adds to res the digest of each root package of one go.mod directory. A root is a package of the main
// module that is listed, not only a dependency, and not a variant of another.
func digestModule(res *DigestResult, pkgs map[string]*goPackage, tree string, header, beside []string) error {
func digestModule(res *DigestResult, pkgs map[string]*goPackage, tree string, header, beside []string, needs bool) error {
roots := []string{}
for path, p := range pkgs {
if p.ForTest == "" && !p.DepOnly && !p.Standard && !strings.HasSuffix(path, ".test") && p.Module != nil && p.Module.Main {
@@ -274,6 +307,15 @@ func digestModule(res *DigestResult, pkgs map[string]*goPackage, tree string, he
if testMain == nil {
return fmt.Errorf("go list gave no test binary for %s", path)
}
if needs {
n, err := readNeeds(root.Dir)
if err != nil {
return fmt.Errorf("%s: its %s: %w", path, NeedsMarker, err)
}
if len(n) > 0 {
res.Needs = append(res.Needs, delivery.PackageNeeds{Package: path, Needs: n})
}
}
if _, err := os.Stat(filepath.Join(root.Dir, ReadsOutsideMarker)); err == nil {
// The digest of a package that reads outside itself is one nothing can have kept: the key is a fresh random
// hex, so the ask finds no kept verdict and runs it, and what the checker writes under it is never read again.
@@ -213,6 +213,16 @@ func TestAPackageWithTheReadsOutsideMarkerIsNeverKept(t *testing.T) {
if first.Packages[a] != plain.Packages[a] || first.Packages[a] != second.Packages[a] {
t.Errorf("a has no marker and its digest moved: %s, %s, before %s", first.Packages[a], second.Packages[a], plain.Packages[a])
}
// Issue 890/AC3: a package whose directory carries the needs marker is named with its needs at the integration level,
// comments and blank lines not read, and at the unit level not at all.
edit(t, tree, "a/"+NeedsMarker, "# its integration tests copy the installed Node toolchain\n\nnode-build-node\n")
if n := digests(t, m, LevelIntegration).Needs; len(n) != 1 || n[0].Package != a || strings.Join(n[0].Needs, ",") != "node-build-node" {
t.Errorf("the integration level's needs = %+v, want %s needing node-build-node", n, a)
}
if n := digests(t, m, LevelUnit).Needs; len(n) != 0 {
t.Errorf("the unit level names needs: %+v", n)
}
}
// Issue 1935: a catalogue module's packages read ../../module.json and the module's files: a change to them changes the
@@ -292,7 +292,7 @@ func (m *Module) goEnv() []string {
if m.GoModCache != "" {
modcache = m.GoModCache
}
return []string{
return append([]string{
"GOTOOLCHAIN=local",
"GOFLAGS=-mod=mod",
"GOMODCACHE=" + modcache,
@@ -300,7 +300,7 @@ func (m *Module) goEnv() []string {
"GOPRIVATE=git.novox.be",
"GIT_TERMINAL_PROMPT=0",
"GIT_SSH_COMMAND=ssh -o BatchMode=yes",
}
}, m.threadEnv()...)
}
// goEnvFor is goEnv for a command run in dir: a directory that vendors (vendor/modules.txt) is read from its vendor
@@ -5,10 +5,13 @@ import (
"os"
"path/filepath"
"reflect"
"runtime"
"strconv"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buildcpu"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/forgekey"
"git.novox.be/novox/mesh-sdk/go/health"
@@ -28,6 +31,7 @@ func toolchainValues(env []string) []string {
// 1572/AC1 (ADR 0477 §1): every environment a step gives its go commands says GOTOOLCHAIN=local, once, and names no
// release.
func TestEveryStepEnvironmentSaysGotoolchainLocal(t *testing.T) {
t.Setenv("MESH_BUILD_CPU_SHARE", "50")
m := &Module{Cache: t.TempDir()}
vend := t.TempDir()
os.MkdirAll(filepath.Join(vend, "vendor"), 0o755)
@@ -41,6 +45,24 @@ func TestEveryStepEnvironmentSaysGotoolchainLocal(t *testing.T) {
if got := toolchainValues(env); len(got) != 1 || got[0] != "GOTOOLCHAIN=local" {
t.Errorf("%s: GOTOOLCHAIN settings %v, want exactly GOTOOLCHAIN=local", name, got)
}
// Issue 1806: and its threads, its part of the node's processor cap (probeEnv runs no build).
if procs := strings.Count(strings.Join(env, "\n"), "GOMAXPROCS="); name != "probeEnv" && procs != 1 {
t.Errorf("%s: GOMAXPROCS said %d times, want once: %v", name, procs, env)
}
}
// Without the composer's share no thread limit; with it, the share over its own three slots (ADR 0553 §4).
t.Setenv("MESH_BUILD_CPU_SHARE", "")
m = &Module{Cache: t.TempDir(), Node: "shanks", Capacity: func() int { return 3 }}
if l := m.cpuLimits(); l.Threads != 0 || strings.Contains(strings.Join(m.goEnv(), " "), "GOMAXPROCS=") {
t.Errorf("without the share: %+v, goEnv %v", l, m.goEnv())
}
t.Setenv("MESH_BUILD_CPU_SHARE", "50")
want := buildcpu.Threads(50, runtime.NumCPU(), 3)
if l := m.cpuLimits(); l.Slots != 3 || !strings.Contains(strings.Join(m.goEnv(), " "), "GOMAXPROCS="+strconv.Itoa(want)) {
t.Errorf("limits %+v, goEnv %v; want %d threads", l, m.goEnv(), want)
}
if st := m.cpuStatus(); st["limits"].(buildcpu.Limits) != m.cpuLimits() {
t.Errorf("the status says %v", st)
}
}
@@ -40,7 +40,7 @@ func (m *Module) isolation(workspace string, write bool) buildstep.Isolation {
Workspace: filepath.Join(m.Work, workspace),
Caches: m.caches(),
WriteCaches: write,
Env: []string{"PATH=" + path, "LC_ALL=C"},
Env: append([]string{"PATH=" + path, "LC_ALL=C"}, m.threadEnv()...),
}
}
@@ -11,11 +11,13 @@ import (
"encoding/json"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// levelArg is the first argument that makes this binary a step's shut-off process: the integration level's, and, when
@@ -140,6 +142,49 @@ func (m *Module) nodeToolchains() []string {
return []string{dir}
}
// nodeToolchainMark is the file the Node build module writes in a toolchain directory once it is installed whole
// (mesh-build-node toolchain.go installedMark).
const nodeToolchainMark = ".installed"
// Has says whether this node has a need a check ask or a package's tests name (issue 890, ADR 0552; mesh-sdk
// delivery.AskNeeds), read from the node itself at every ask: node-build-node when the Node build module's toolchains
// directory, given to the integration level as nodeToolchains gives it, holds a toolchain installed whole (its mark and
// its package-lock.json readable by this account), and node, npm and bwrap are installed: what mesh-build-node's
// integration tests (realMachine) look for before they run. Any other need it lacks.
func (m *Module) Has(need string) bool {
if m.HasNeed != nil {
return m.HasNeed(need)
}
switch need {
case delivery.NeedNodeBuildNode:
dirs := m.nodeToolchains()
if len(dirs) == 0 {
return false
}
for _, p := range []string{"node", "npm", "bwrap"} {
if _, err := exec.LookPath(p); err != nil {
return false
}
}
entries, err := os.ReadDir(dirs[0])
if err != nil {
return false
}
for _, e := range entries {
dir := filepath.Join(dirs[0], e.Name())
if _, err := os.Stat(filepath.Join(dir, nodeToolchainMark)); err != nil {
continue
}
if f, err := os.Open(filepath.Join(dir, "package-lock.json")); err == nil {
f.Close()
return true
}
}
return false
}
return false
}
func lastLine(s string) string {
lines := strings.Split(strings.TrimSpace(s), "\n")
return strings.TrimSpace(lines[len(lines)-1])
@@ -41,6 +41,8 @@ type PackageResult struct {
Tests int `json:"tests"`
// NoTests is a package with no test files (at this level: a file with the other level's build tag is none).
NoTests bool `json:"no_tests,omitempty"`
// Skipped is how many of the package's tests skipped: a pass that skipped proved less than it says, so the checker keeps no verdict for it.
Skipped int `json:"skipped,omitempty"`
}
// TestTime is one test and the time it took.
@@ -227,6 +229,7 @@ func (r *reader) event(ev event) {
r.res.Tests++
if ev.Action == "skip" {
r.res.Skipped++
p.Skipped++
}
r.times = append(r.times, TestTime{Package: ev.Package, Test: ev.Test, Elapsed: ev.Elapsed})
if ev.Action == "fail" {
@@ -8,9 +8,11 @@ package main
import (
"context"
"fmt"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"os"
"os/user"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
@@ -70,7 +72,13 @@ func main() {
os.Exit(1)
}
steps := m.StepTools()
booted := time.Now()
puller := m.Puller(steps, nil)
m.Held = puller
if m.Node != "" {
// A step a previous process left running is ended (issue 1890); this process's own steps start after booted.
go buildstep.EndStaleAtStart(ctx, m.stepScan(), m.Node, booted, logf)
}
if m.Node != "" {
// The seat's asks, pulled once Serve has the runtime's channel; a pull before it is said and tried again.
go func() { _ = puller.Pull(ctx) }()
@@ -28,6 +28,8 @@ type Module struct {
// Toolchains and Steps are the Put of the states of those names.
Toolchains, Steps buildstep.Put
// ReadSteps and ReadToolchainsState read those two states back for the read tools (issue 1562).
// Held is the puller whose running asks a step-end tool will not end (issue 1890); nil in a test.
Held *buildstep.Puller
ReadSteps, ReadToolchainsState StateRead
// Pauses is the Put of the pauses state, which keeps the node's pause key without an age; ReadPauses reads it back.
Pauses buildstep.Put
@@ -73,6 +75,8 @@ type Module struct {
// NodeToolchains is where the Node build module keeps its installed toolchains, bound read-only into the
// integration level (ADR 0533 §1); NodeToolchains when empty. A test points it elsewhere.
NodeToolchains string
// HasNeed answers Has in place of the node's own reading; a test's.
HasNeed func(need string) bool
// StartBus starts the throwaway bus of a run in dir and answers its address and what stops it (the module's own
// program, StartBus in bus.go, when nil).
StartBus func(ctx context.Context, dir string) (url string, stop func(), err error)
@@ -121,6 +125,7 @@ func (m *Module) Start(ctx context.Context) {
m.Say("the toolchain: %v", err)
}
m.CleanWorkspaces(time.Now())
buildstep.EndStaleRound(m.stepScan(), m.Node, m.Say) // asks past the step bound, on the module's own round (issue 1890)
select {
case <-ctx.Done():
return
@@ -184,13 +189,14 @@ func (m *Module) Short(run func(ctx context.Context, args map[string]any) (any,
}
// Status is what the module reports of itself: the node, the recipe, the toolchain found and why it refuses, if it
// does.
// does, and the processors its steps may take (issue 1806).
func (m *Module) Status(ctx context.Context) map[string]any {
err := m.Check(ctx)
installed, _ := m.Probe(ctx)
out := map[string]any{
"node": m.Node, "recipe": m.Recipe.Version(), "installed": installed,
"postgres-major": m.Recipe.PostgresMajor, "nats-server": m.Recipe.NatsServer, "refusing": err != nil,
"cpu": m.cpuStatus(),
}
if err != nil {
out["why"] = err.Error()
@@ -115,7 +115,7 @@ func TestAGoThatCannotBeRunRefusesEveryStepAndHealthIsUnhealthy(t *testing.T) {
{Name: "short_probe", Run: run},
}, m.StepTools()...)
for _, tool := range m.Tools(steps) {
if tool.Name == StatusTool || tool.Name == HealthTool || tool.Name == StepReadTool || tool.Name == StepsReadTool || tool.Name == ToolchainsReadTool || tool.Name == ReleaseTool {
if tool.Name == StatusTool || tool.Name == HealthTool || tool.Name == StepReadTool || tool.Name == StepsReadTool || tool.Name == ToolchainsReadTool || tool.Name == ReleaseTool || tool.Name == StepEndTool {
continue
}
var reason string
@@ -71,6 +71,47 @@ func TestANodeWithoutNodeToolchainsRunsTheLevelWithNothingMoreBound(t *testing.T
if slices.Contains(s.roBinds, missing) {
t.Errorf("a missing %s is bound: %v", missing, s.roBinds)
}
// Issue 890/AC4: such a node lacks node-build-node; one with an installed toolchain and node and npm has it; nothing
// else is had.
bin := t.TempDir()
for _, p := range []string{"node", "npm", "bwrap"} {
if err := os.WriteFile(filepath.Join(bin, p), []byte("#!/bin/sh\n"), 0o755); err != nil {
t.Fatal(err)
}
}
t.Setenv("PATH", bin)
m := &Module{NodeToolchains: missing}
if m.Has(delivery.NeedNodeBuildNode) {
t.Error("a node with no Node toolchain has node-build-node")
}
installed := t.TempDir()
m.NodeToolchains = installed
if m.Has(delivery.NeedNodeBuildNode) {
t.Error("a node with an empty toolchains directory has node-build-node")
}
half := filepath.Join(installed, "0.1.0")
if err := os.Mkdir(half, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(half, "package-lock.json"), []byte("{}"), 0o644); err != nil {
t.Fatal(err)
}
if m.Has(delivery.NeedNodeBuildNode) {
t.Error("a toolchain without its installed mark counts")
}
if err := os.WriteFile(filepath.Join(half, nodeToolchainMark), nil, 0o644); err != nil {
t.Fatal(err)
}
if !m.Has(delivery.NeedNodeBuildNode) || m.Has("gpu") {
t.Errorf("with a toolchain installed: node-build-node %v, gpu %v; want true, false", m.Has(delivery.NeedNodeBuildNode), m.Has("gpu"))
}
if err := os.Remove(filepath.Join(bin, "bwrap")); err != nil {
t.Fatal(err)
}
if m.Has(delivery.NeedNodeBuildNode) {
t.Error("a node with no bubblewrap installed has node-build-node")
}
}
// 1891 (ADR 0533 §1): the unit level is not given them.
@@ -101,3 +101,25 @@ func TestALoggedPanicIsNotTheCause(t *testing.T) { // 1901/AC4
t.Errorf("1901/AC4: failures = %+v, want the timeout's panic as the cause", res.Failed)
}
}
// A package counts its own skipped tests, so the checker withholds a kept verdict from that package alone (issue 890).
func TestEachPackageCountsItsOwnSkippedTests(t *testing.T) {
log := `{"Action":"run","Package":"m/a","Test":"TestA"}
{"Action":"skip","Package":"m/a","Test":"TestA","Elapsed":0}
{"Action":"run","Package":"m/b","Test":"TestB"}
{"Action":"pass","Package":"m/b","Test":"TestB","Elapsed":0}
{"Action":"pass","Package":"m/a","Elapsed":0}
{"Action":"pass","Package":"m/b","Elapsed":0}
`
res := &TestResult{}
rd := newReader(res, ".")
rd.read(log)
rd.finish()
got := map[string]int{}
for _, p := range res.Packages {
got[p.Package] = p.Skipped
}
if res.Skipped != 1 || got["m/a"] != 1 || got["m/b"] != 0 {
t.Errorf("skipped = %d, per package %v, want 1 with m/a 1 and m/b 0", res.Skipped, got)
}
}
@@ -128,6 +128,7 @@ type StepLine struct {
Started string `json:"started"`
Ended string `json:"ended,omitempty"`
Reason string `json:"reason,omitempty"`
What string `json:"what,omitempty"`
}
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero). Only
@@ -180,7 +181,7 @@ func (m *Module) RecentSteps(limit int) (lines []StepLine, cut bool, err error)
}
out := make([]StepLine, 0, len(steps))
for _, s := range steps {
l := StepLine{ID: s.ID, Tool: s.Tool, State: s.State, Started: s.Started.UTC().Format("2006-01-02T15:04:05Z"), Reason: s.Reason}
l := StepLine{ID: s.ID, Tool: s.Tool, State: s.State, Started: s.Started.UTC().Format("2006-01-02T15:04:05Z"), Reason: s.Reason, What: s.What}
if !s.Ended.IsZero() {
l.Ended = s.Ended.UTC().Format("2006-01-02T15:04:05Z")
}
@@ -96,7 +96,7 @@ func TestStepsToolListsRecentStepsOfThisNode(t *testing.T) {
t0 := time.Date(2026, 10, 21, 9, 0, 0, 0, time.UTC)
putStep(t, steps, delivery.Step{ID: "a", Tool: "build_go_fetch", Node: "ace", State: delivery.StepPassed, Started: t0, Ended: t0, Result: json.RawMessage(`1`)})
putStep(t, steps, delivery.Step{ID: "b", Tool: "build_go_test", Node: "ace", State: delivery.StepRunning, Started: t0.Add(time.Minute)})
putStep(t, steps, delivery.Step{ID: "c", Tool: "build_go_build", Node: "ace", State: delivery.StepFailed, Started: t0.Add(2 * time.Minute), Reason: "boom"})
putStep(t, steps, delivery.Step{ID: "c", Tool: "build_go_build", Node: "ace", State: delivery.StepFailed, Started: t0.Add(2 * time.Minute), Reason: "boom", What: "check ask, novox/x@abc"})
putStep(t, steps, delivery.Step{ID: "d", Tool: "build_go_build", Node: "g14", State: delivery.StepPassed, Started: t0.Add(time.Hour)})
m := readModule(steps, fakeState{})
out, err := tool(t, m, StepsReadTool)(map[string]any{})
@@ -111,7 +111,7 @@ func TestStepsToolListsRecentStepsOfThisNode(t *testing.T) {
if !slices.Equal(ids, []string{"c", "b", "a"}) {
t.Errorf("ids = %v, want c b a", ids)
}
if lines[0].Reason != "boom" || lines[0].State != delivery.StepFailed {
if lines[0].Reason != "boom" || lines[0].What != "check ask, novox/x@abc" || lines[0].State != delivery.StepFailed {
t.Errorf("first = %+v", lines[0])
}
out, _ = tool(t, m, StepsReadTool)(map[string]any{"limit": float64(1)})
@@ -2,6 +2,7 @@ package main
import (
"context"
"git.novox.be/novox/mesh-sdk/go/buildstep"
stdio "git.novox.be/novox/mesh-sdk/go"
)
@@ -40,6 +41,7 @@ func (m *Module) Tools(steps []StepTool) []stdio.Tool {
tools = append(tools, t)
}
tools = append(tools, m.ReadTools()...)
tools = append(tools, buildstep.StepEndTool(StepEndTool, m.Node, m.stepScan(), m.Held))
tools = append(tools, m.releaseTool())
return append(tools,
stdio.Tool{Name: StatusTool,
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildgo
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026134601-17b7cc480d6d
github.com/distribution/distribution/v3 v3.1.2
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4 h1:05sVel/UOvz4/g0dxrDOeVUL7BGakVrxPBlqRq02ZhM=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026134601-17b7cc480d6d h1:7F/dFerm01wvuGLPgJ67/VfdcvDexc+BQEE3qzOrlu4=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026134601-17b7cc480d6d/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/alicebob/miniredis/v2 v2.35.0 h1:QwLphYqCEAo1eu1TqPRN2jgVMPBweeQcR21jeqDCONI=
github.com/alicebob/miniredis/v2 v2.35.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
+1
View File
@@ -39,6 +39,7 @@
"build_go_release",
"build_go_step",
"build_go_steps",
"build_go_step_end",
"build_go_toolchains",
"build_go_status",
"build_go_health"
@@ -66,3 +66,11 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
func (m *Module) SeatTools(p *buildstep.Puller) []stdio.Tool {
return buildstep.SeatTools(Seat, "image", p)
}
// StepEndTool ends one step left running (issue 1890); the module's steps state is the one it reads and writes.
const StepEndTool = "build_image_step_end"
// stepScan is the module's steps state as buildstep reads it.
func (m *Module) stepScan() buildstep.Scan {
return buildstep.ScanOf(stdio.State(delivery.StepsState))
}
@@ -69,6 +69,7 @@ func setup(t *testing.T, edit func(string) string) (*Module, *fake, *keptStates)
// Dockerfile of the module's own directory and that directory as the context, the target, the pinned base as a build
// argument; no mount, no credential file, a runtime directory of its own, and it answers the image's digest.
func TestBuildRunsBuildahRootlessAndAnswersTheDigest(t *testing.T) {
t.Setenv("MESH_BUILD_CPU_SHARE", "50") // the composer's share (ADR 0553): buildah is given its threads
m, f, states := setup(t, same)
res, st := runBuild(t, m, states, nil)
if st.State != delivery.StepPassed {
@@ -110,6 +111,9 @@ func TestBuildRunsBuildahRootlessAndAnswersTheDigest(t *testing.T) {
if !strings.Contains(env, "XDG_RUNTIME_DIR="+m.Store+"/xdg") {
t.Errorf("the environment of %q lacks the module's own runtime directory: %s", c.line, env)
}
if !strings.Contains(env, "GOMAXPROCS=") {
t.Errorf("the environment of %q gives buildah no thread limit under a share (ADR 0553): %s", c.line, env)
}
}
// The image is removed from the store once the digest is read, so the store does not grow with every build.
if len(f.called("rmi")) != 1 {
@@ -0,0 +1,36 @@
package main
import (
"git.novox.be/novox/mesh-sdk/go/buildcpu"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
// The processors a step may take (novox/hq ADR 0553, issue 1806). The node-engine holds this module's runtime, with
// every other build module's of the node, in one slice at idle priority under the node's one share of its processors:
// that quota bounds the total. Threads only size each step: the share the composer gives (buildstep.NodeLimits) over
// this module's own capacity, so a step does not wait in the kernel's throttle. Without the composer's share there is
// no slice to size a step to, and no thread limit is set.
// cpuLimits is what one step is held to now.
func (m *Module) cpuLimits() buildcpu.Limits {
slots := 1
switch {
case m.Capacity != nil:
slots = m.Capacity()
case m.Settings != "":
slots = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, nil)()
}
return buildstep.NodeLimits(slots)
}
// threadEnv is the environment (nothing without a share) buildah is given for its threads (it is a Go program). What a build's RUN lines start
// runs in the container's own environment, inside the runtime's cgroup: the slice's quota and the nice level bind it.
func (m *Module) threadEnv() []string {
return buildcpu.Env(m.cpuLimits().Threads)
}
// cpuStatus is what the status tool says of it: the limits a step is given, and what the kernel holds this runtime
// to (its unit and its slice), read from its cgroup rather than assumed.
func (m *Module) cpuStatus() map[string]any {
return map[string]any{"limits": m.cpuLimits(), "in-force": buildcpu.ReadInForce()}
}
@@ -9,9 +9,11 @@ package main
import (
"context"
"fmt"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"os"
"os/user"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
@@ -57,7 +59,13 @@ func main() {
os.Exit(1)
}
steps := m.StepTools()
booted := time.Now()
puller := m.Puller(steps, nil)
m.Held = puller
if m.Node != "" {
// A step a previous process left running is ended (issue 1890); this process's own steps start after booted.
go buildstep.EndStaleAtStart(ctx, m.stepScan(), m.Node, booted, logf)
}
if m.Node != "" {
// The seat's asks, pulled once Serve has the runtime's channel; a pull before it is said and tried again.
go func() { _ = puller.Pull(ctx) }()
@@ -22,7 +22,9 @@ type Module struct {
// Account and UID are the account the module runs as; Home is its home.
Account, UID, Home string
// Steps is the Put of the steps state; ReadSteps reads it back for the read tools.
Steps buildstep.Put
Steps buildstep.Put
// Held is the puller whose running asks a step-end tool will not end (issue 1890); nil in a test.
Held *buildstep.Puller
ReadSteps StateRead
// Pauses is the Put of the pauses state, which keeps the node's pause key without an age; ReadPauses reads it back.
Pauses buildstep.Put
@@ -104,6 +106,7 @@ func (m *Module) Start(ctx context.Context) {
m.Say("this machine cannot build images: %v", err)
}
m.CleanWorkspaces(m.clock())
buildstep.EndStaleRound(m.stepScan(), m.Node, m.Say) // asks past the step bound, on the module's own round (issue 1890)
select {
case <-ctx.Done():
return
@@ -128,7 +131,8 @@ func (m *Module) Health(ctx context.Context) health.Answer {
// Status is what the module reports of itself.
func (m *Module) Status(ctx context.Context) map[string]any {
l, err := m.look(ctx)
out := map[string]any{"node": m.Node, "account": m.Account, "buildah": l.Buildah, "subuids": l.UIDs, "subgids": l.GIDs, "refusing": err != nil}
out := map[string]any{"node": m.Node, "account": m.Account, "buildah": l.Buildah, "subuids": l.UIDs, "subgids": l.GIDs, "refusing": err != nil,
"cpu": m.cpuStatus()}
if err != nil {
out["why"] = err.Error()
}
@@ -117,7 +117,7 @@ func (m *Module) storeFlags() []string {
// error, and the next look refuses with it.
func (m *Module) env() []string {
_ = m.ensureRuntimeDir()
return []string{"XDG_RUNTIME_DIR=" + m.runtimeDir()}
return append([]string{"XDG_RUNTIME_DIR=" + m.runtimeDir()}, m.threadEnv()...)
}
// runtimeDir is buildah's runtime directory, inside the store.
@@ -127,6 +127,7 @@ type StepLine struct {
Started string `json:"started"`
Ended string `json:"ended,omitempty"`
Reason string `json:"reason,omitempty"`
What string `json:"what,omitempty"`
}
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero). Only
@@ -179,7 +180,7 @@ func (m *Module) RecentSteps(limit int) (lines []StepLine, cut bool, err error)
}
out := make([]StepLine, 0, len(steps))
for _, s := range steps {
l := StepLine{ID: s.ID, Tool: s.Tool, State: s.State, Started: s.Started.UTC().Format("2006-01-02T15:04:05Z"), Reason: s.Reason}
l := StepLine{ID: s.ID, Tool: s.Tool, State: s.State, Started: s.Started.UTC().Format("2006-01-02T15:04:05Z"), Reason: s.Reason, What: s.What}
if !s.Ended.IsZero() {
l.Ended = s.Ended.UTC().Format("2006-01-02T15:04:05Z")
}
@@ -2,6 +2,7 @@ package main
import (
"context"
"git.novox.be/novox/mesh-sdk/go/buildstep"
stdio "git.novox.be/novox/mesh-sdk/go"
)
@@ -44,6 +45,7 @@ func (m *Module) Tools(steps []StepTool) []stdio.Tool {
tools = append(tools, t)
}
tools = append(tools, m.ReadTools()...)
tools = append(tools, buildstep.StepEndTool(StepEndTool, m.Node, m.stepScan(), m.Held))
tools = append(tools, m.releaseTool())
return append(tools,
stdio.Tool{Name: StatusTool,
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildimage
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026134601-17b7cc480d6d
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8 h1:x4mvY7yiGTP5lnzxbiwyj1o24p05hAmzZP7xHg3XX2M=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261025175617-518dae5ec9c8/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026134601-17b7cc480d6d h1:7F/dFerm01wvuGLPgJ67/VfdcvDexc+BQEE3qzOrlu4=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026134601-17b7cc480d6d/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
+1
View File
@@ -33,6 +33,7 @@
"build_image_release",
"build_image_step",
"build_image_steps",
"build_image_step_end",
"build_image_status",
"build_image_health"
],
@@ -74,3 +74,11 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
func (m *Module) SeatTools(p *buildstep.Puller) []stdio.Tool {
return buildstep.SeatTools(Seat, "Node", p)
}
// StepEndTool ends one step left running (issue 1890); the module's steps state is the one it reads and writes.
const StepEndTool = "build_node_step_end"
// stepScan is the module's steps state as buildstep reads it.
func (m *Module) stepScan() buildstep.Scan {
return buildstep.ScanOf(stdio.State(delivery.StepsState))
}
@@ -17,11 +17,11 @@ import (
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// realMachine gives the tests the real toolchain of the module's own lock and makes two workspaces holding one fixture
// bundle. The toolchain is the one this node's installed Node build module keeps (ADR 0533 §1): the integration level
// is given its toolchains read-only and no network, so it is copied into the test's own cache from there. Only where
// no toolchain is installed (a desk) is it installed with `npm ci --ignore-scripts` from the mesh's registry, as the
// module does, which needs the network.
// realMachine gives the tests the real toolchain of the module's own lock and makes three workspaces holding one fixture
// bundle. The toolchain is the one this node's installed Node build module keeps (ADR 0533 §1), copied into the test's
// own cache: the integration level is given its toolchains read-only and no network, so it is never installed from the
// registry here. Where no Node build module has installed one (a node without it, a desk), the test is skipped by
// name: an install from the public registry inside a shut-off level only fails, after 70 s of npm's retries.
func realMachine(t *testing.T) (*Module, string) {
t.Helper()
for _, p := range []string{"node", "npm", "bwrap"} { // bwrap: tsc, esbuild and the scripts run shut off (1309)
@@ -44,16 +44,19 @@ func realMachine(t *testing.T) (*Module, string) {
run := forgekey.ExecRunnerFor(0)
m := &Module{Node: "ace", Recipe: lock, Work: work, Run: run, Ctx: context.Background(),
Toolchain: &Toolchain{Paths: Paths{Home: home, Cache: cache}, Run: run, Registry: func() (Registry, error) {
return Registry{URL: "https://git.novox.be/api/packages/novox/npm/"}, nil
return Registry{}, fmt.Errorf("the integration test installs no toolchain from a registry: it has no network")
}}}
withStore(t, m)
if from := installedToolchain(t, filepath.Join(CacheDir, "toolchain"), filepath.Join(home, "toolchain", "package-lock.json")); from != "" {
copyToolchain(t, from, m.Toolchain.Paths.Dir(m.Recipe.Version()))
from := installedToolchain(t, filepath.Join(CacheDir, "toolchain"), filepath.Join(home, "toolchain", "package-lock.json"))
if from == "" {
t.Skipf("NOT RUN: no Node toolchain is installed under %s for this account to read (no Node build module on this node); "+
"the integration level has no network to install one, so this runs on a node that holds the Node build module", filepath.Join(CacheDir, "toolchain"))
}
if err := m.Toolchain.Ensure(context.Background(), m.Recipe.Version()); err != nil {
t.Fatalf("no toolchain of this lock is installed under %s, and installing it from the registry failed: %v", filepath.Join(CacheDir, "toolchain"), err)
copyToolchain(t, from, m.Toolchain.Paths.Dir(m.Recipe.Version()))
if err := m.Toolchain.Ensure(context.Background(), m.Recipe.Version()); err != nil { // present: opens it, installs nothing
t.Fatalf("the toolchain copied from %s cannot be used: %v", from, err)
}
for _, id := range []string{"w1", "w2"} {
for _, id := range []string{"w1", "w2", "w3"} {
for p, body := range map[string]string{
"modules/m/module.json": `{"module":"m","build":{"artifacts":[{"name":"tools","kind":"bundle","language":"typescript","entrypoints":["tools/index.js"]}]}}`,
"modules/m/package.json": `{"name":"m","type":"module","scripts":{"test":"node -e \"process.exit(0)\""}}`,
@@ -65,10 +68,36 @@ func realMachine(t *testing.T) (*Module, string) {
return m, work
}
// 1490/AC2: a fixture TypeScript bundle importing @novox/mesh-sdk, built twice by the real tsc and esbuild, has equal
// digests, and its archive holds the bundled .js, the .serve.mjs launcher and package.json.
func TestAFixtureBundleBuiltTwiceHasOneDigest(t *testing.T) {
// 1490/AC2, 1491 and issue 1869, in one test so the toolchain is copied once: the real type-check and test script of the
// fixture pass the unit level; a package whose test imports its own build output (as mesh-sdk's do) passes the unit level,
// the test step emitting it with the real tsc, shut off, before the test script runs; and the fixture TypeScript bundle
// importing @novox/mesh-sdk, built twice by the real tsc and esbuild, has equal digests, its archive holding the bundled
// .js, the .serve.mjs launcher and package.json. The unit levels run in w3, so the two builds' workspaces stay alike.
func TestTheFixturePassesTheUnitLevelAndBuildsTwiceToOneDigest(t *testing.T) {
m, work := realMachine(t)
out, err := m.test(context.Background(), map[string]any{"workspace": "w3", "module": "modules/m", "level": "unit"})
if err != nil {
t.Fatal(err)
}
if r := out.(*TestResult); !r.Passed || r.NoTests || len(r.Ran) != 2 {
t.Errorf("the fixture's unit level: result = %+v", r)
}
for p, body := range map[string]string{
"modules/p/package.json": `{"name":"p","type":"module","scripts":{"build":"tsc","test":"node --test test/a.test.mjs","prepare":"tsc"}}`,
"modules/p/tsconfig.json": `{"compilerOptions":{"module":"NodeNext","moduleResolution":"NodeNext","target":"ES2022","outDir":"dist","rootDir":"src"},"include":["src"]}`,
"modules/p/src/index.ts": "export const answer: number = 42;\n",
"modules/p/test/a.test.mjs": "import { test } from 'node:test';\nimport assert from 'node:assert';\nimport { answer } from '../dist/index.js';\n" +
"test('the build is there', () => assert.equal(answer, 42));\n",
} {
write(t, filepath.Join(work, "w3", "tree", filepath.FromSlash(p)), body, 0o644)
}
out, err = m.test(context.Background(), map[string]any{"workspace": "w3", "module": "modules/p", "level": "unit"})
if err != nil {
t.Fatal(err)
}
if r := out.(*TestResult); !r.Passed || !slices.Equal(r.Ran, []string{"build", "test"}) {
t.Errorf("the package importing its build: result = %+v", r)
}
var digests []string
for _, id := range []string{"w1", "w2"} {
out, err := m.build(context.Background(), map[string]any{"workspace": id, "module": "modules/m", "artifact": "tools"})
@@ -89,43 +118,6 @@ func TestAFixtureBundleBuiltTwiceHasOneDigest(t *testing.T) {
}
}
// 1491: the real type-check and test script of the same fixture pass the unit level.
func TestTheFixturePassesTheUnitLevel(t *testing.T) {
m, _ := realMachine(t)
out, err := m.test(context.Background(), map[string]any{"workspace": "w1", "module": "modules/m", "level": "unit"})
if err != nil {
t.Fatal(err)
}
if r := out.(*TestResult); !r.Passed || r.NoTests || len(r.Ran) != 2 {
t.Errorf("result = %+v", r)
}
}
// Issue 1869: a package whose test imports its own build output (as mesh-sdk's do) passes the unit level: the test step
// emits it with the real tsc, shut off, before the test script runs.
func TestAPackageWhoseTestsImportItsBuildPassesTheUnitLevel(t *testing.T) {
if _, err := exec.LookPath("bwrap"); err != nil {
t.Skip("NOT RUN: no bubblewrap installed where this check runs; every build node has it (the module declares it)")
}
m, work := realMachine(t)
for p, body := range map[string]string{
"modules/p/package.json": `{"name":"p","type":"module","scripts":{"build":"tsc","test":"node --test test/a.test.mjs","prepare":"tsc"}}`,
"modules/p/tsconfig.json": `{"compilerOptions":{"module":"NodeNext","moduleResolution":"NodeNext","target":"ES2022","outDir":"dist","rootDir":"src"},"include":["src"]}`,
"modules/p/src/index.ts": "export const answer: number = 42;\n",
"modules/p/test/a.test.mjs": "import { test } from 'node:test';\nimport assert from 'node:assert';\nimport { answer } from '../dist/index.js';\n" +
"test('the build is there', () => assert.equal(answer, 42));\n",
} {
write(t, filepath.Join(work, "w1", "tree", filepath.FromSlash(p)), body, 0o644)
}
out, err := m.test(context.Background(), map[string]any{"workspace": "w1", "module": "modules/p", "level": "unit"})
if err != nil {
t.Fatal(err)
}
if r := out.(*TestResult); !r.Passed || !slices.Equal(r.Ran, []string{"build", "test"}) {
t.Errorf("result = %+v", r)
}
}
// installedToolchain is the toolchain directory under toolchains that this node's Node build module installed: the
// one installed from the lock at lockPath, byte for byte, or, when the change carries a lock no delivered release has
// installed yet, the one installed last, said in the test's log (ADR 0533 §3: a new lock is proven by the module that
@@ -0,0 +1,3 @@
# Its integration tests copy the Node toolchain the Node build module installed on this node and run node and npm
# (realMachine, build_integration_test.go): they run only where the node holds node-build-node (issue 890).
node-build-node
@@ -0,0 +1,36 @@
package main
import (
"git.novox.be/novox/mesh-sdk/go/buildcpu"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
// The processors a step may take (novox/hq ADR 0553, issue 1806). The node-engine holds this module's runtime, with
// every other build module's of the node, in one slice at idle priority under the node's one share of its processors:
// that quota bounds the total. Threads only size each step: the share the composer gives (buildstep.NodeLimits) over
// this module's own capacity, so a step does not wait in the kernel's throttle. Without the composer's share there is
// no slice to size a step to, and no thread limit is set.
// cpuLimits is what one step is held to now.
func (m *Module) cpuLimits() buildcpu.Limits {
slots := 1
switch {
case m.Capacity != nil:
slots = m.Capacity()
case m.Settings != "":
slots = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, nil)()
}
return buildstep.NodeLimits(slots)
}
// threadEnv is the environment (nothing without a share) every command of a step is given for its threads: GOMAXPROCS for esbuild and every Go
// program, UV_THREADPOOL_SIZE for Node's worker pool.
func (m *Module) threadEnv() []string {
return buildcpu.Env(m.cpuLimits().Threads)
}
// cpuStatus is what the status tool says of it: the limits a step is given, and what the kernel holds this runtime
// to (its unit and its slice), read from its cgroup rather than assumed.
func (m *Module) cpuStatus() map[string]any {
return map[string]any{"limits": m.cpuLimits(), "in-force": buildcpu.ReadInForce()}
}
@@ -4,6 +4,7 @@ import (
"context"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"time"
@@ -225,6 +226,7 @@ func mustReadDir(t *testing.T, dir string) []string {
// are given PATH, HOME and the locale and their own words; and the real clean runner, run on `env` here, prints only
// those.
func TestNoCommandOfTheStepSeesATokenOfTheProcess(t *testing.T) {
t.Setenv("MESH_BUILD_CPU_SHARE", "50")
t.Setenv("MESH_BUS_CREDENTIAL", "hunter2")
t.Setenv("NPM_TOKEN", "hunter3")
files := typeScriptModule(`{"dependencies":{"lodash":"^4"}}`)
@@ -245,6 +247,11 @@ func TestNoCommandOfTheStepSeesATokenOfTheProcess(t *testing.T) {
if len(c.env) < 3 || !strings.HasPrefix(c.env[0], "PATH=") || !strings.HasPrefix(c.env[1], "HOME=") || c.env[2] != "LC_ALL=C" {
t.Errorf("%s was given %v, want PATH, HOME and the locale first", c.name, c.env)
}
// And its threads, its part of the node's share the composer gave (ADR 0553).
if !slices.ContainsFunc(c.env, func(e string) bool { return strings.HasPrefix(e, "GOMAXPROCS=") }) ||
!slices.ContainsFunc(c.env, func(e string) bool { return strings.HasPrefix(e, "UV_THREADPOOL_SIZE=") }) {
t.Errorf("%s was given %v, without its threads", c.name, c.env)
}
}
// The runner main wires: what it hands a command is exactly what the step gave.
ran := forgekey.ExecRunnerClean(10*time.Second)(context.Background(), append(m.baseEnv(), "ONLY=this"), "env")
@@ -35,7 +35,7 @@ func (m *Module) isolation(root string, write bool) buildstep.Isolation {
ReadOnly: []string{m.toolchainDir()},
Caches: []string{m.npmCache()},
WriteCaches: write,
Env: []string{"PATH=" + pathOrDefault(), "LC_ALL=C"},
Env: append([]string{"PATH=" + pathOrDefault(), "LC_ALL=C"}, m.threadEnv()...),
}
}
@@ -9,10 +9,12 @@ package main
import (
"context"
"fmt"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"os"
"os/user"
"path/filepath"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
@@ -69,7 +71,13 @@ func main() {
os.Exit(1)
}
steps := m.StepTools()
booted := time.Now()
puller := m.Puller(steps, nil)
m.Held = puller
if m.Node != "" {
// A step a previous process left running is ended (issue 1890); this process's own steps start after booted.
go buildstep.EndStaleAtStart(ctx, m.stepScan(), m.Node, booted, logf)
}
if m.Node != "" {
// The seat's asks, pulled once Serve has the runtime's channel; a pull before it is said and tried again.
go func() { _ = puller.Pull(ctx) }()
@@ -42,6 +42,8 @@ type Module struct {
// Toolchains and Steps are the Put of the states of those names.
Toolchains, Steps buildstep.Put
// ReadSteps and ReadToolchainsState read those two states back for the read tools (issue 1562).
// Held is the puller whose running asks a step-end tool will not end (issue 1890); nil in a test.
Held *buildstep.Puller
ReadSteps, ReadToolchainsState StateRead
// Pauses is the Put of the pauses state, which keeps the node's pause key without an age; ReadPauses reads it back.
Pauses buildstep.Put
@@ -137,6 +139,7 @@ func (m *Module) Start(ctx context.Context) {
m.Say("the toolchain: %v", err)
}
m.CleanWorkspaces(m.now())
buildstep.EndStaleRound(m.stepScan(), m.Node, m.Say) // asks past the step bound, on the module's own round (issue 1890)
select {
case <-ctx.Done():
return
@@ -202,7 +205,8 @@ func (m *Module) Status(ctx context.Context) map[string]any {
err := m.Check(ctx)
installed, _ := m.Probe(ctx)
r := m.recipe()
out := map[string]any{"node": m.Node, "recipe": r.Version(), "installed": installed, "refusing": err != nil}
out := map[string]any{"node": m.Node, "recipe": r.Version(), "installed": installed, "refusing": err != nil,
"cpu": m.cpuStatus()}
if m.Toolchain != nil {
out["toolchain-dir"] = m.Toolchain.Paths.Dir(r.Version())
out["toolchain-installed"] = m.Toolchain.Present(r.Version())
@@ -227,7 +231,7 @@ const defaultPath = "/usr/local/bin:/usr/bin:/bin"
// files and git configuration) and a fixed locale. A command is given these and its own words, never this process's
// environment, which holds the module's bus credentials (ADR 0466 §4).
func (m *Module) baseEnv() []string {
return []string{"PATH=" + pathOrDefault(), "HOME=" + m.Toolchain.homeOrEmpty(), "LC_ALL=C"}
return append([]string{"PATH=" + pathOrDefault(), "HOME=" + m.Toolchain.homeOrEmpty(), "LC_ALL=C"}, m.threadEnv()...)
}
func pathOrDefault() string {
@@ -117,7 +117,7 @@ func TestANodeThatCannotBeRunRefusesEveryStepAndHealthIsUnhealthy(t *testing.T)
{Name: "short_probe", Run: run},
}, (&Module{}).StepTools()...)
for _, tool := range m.Tools(steps) {
if tool.Name == StatusTool || tool.Name == HealthTool || tool.Name == StepReadTool || tool.Name == StepsReadTool || tool.Name == ToolchainsReadTool || tool.Name == ReleaseTool {
if tool.Name == StatusTool || tool.Name == HealthTool || tool.Name == StepReadTool || tool.Name == StepsReadTool || tool.Name == ToolchainsReadTool || tool.Name == ReleaseTool || tool.Name == StepEndTool {
continue
}
var reason string
@@ -128,6 +128,7 @@ type StepLine struct {
Started string `json:"started"`
Ended string `json:"ended,omitempty"`
Reason string `json:"reason,omitempty"`
What string `json:"what,omitempty"`
}
// RecentSteps are this node's steps, the latest started first, at most limit (ten when limit is not above zero). Only
@@ -180,7 +181,7 @@ func (m *Module) RecentSteps(limit int) (lines []StepLine, cut bool, err error)
}
out := make([]StepLine, 0, len(steps))
for _, s := range steps {
l := StepLine{ID: s.ID, Tool: s.Tool, State: s.State, Started: s.Started.UTC().Format("2006-01-02T15:04:05Z"), Reason: s.Reason}
l := StepLine{ID: s.ID, Tool: s.Tool, State: s.State, Started: s.Started.UTC().Format("2006-01-02T15:04:05Z"), Reason: s.Reason, What: s.What}
if !s.Ended.IsZero() {
l.Ended = s.Ended.UTC().Format("2006-01-02T15:04:05Z")
}
@@ -96,7 +96,7 @@ func TestStepsToolListsRecentStepsOfThisNode(t *testing.T) {
t0 := time.Date(2026, 10, 21, 9, 0, 0, 0, time.UTC)
putStep(t, steps, delivery.Step{ID: "a", Tool: "build_node_fetch", Node: "ace", State: delivery.StepPassed, Started: t0, Ended: t0, Result: json.RawMessage(`1`)})
putStep(t, steps, delivery.Step{ID: "b", Tool: "build_node_test", Node: "ace", State: delivery.StepRunning, Started: t0.Add(time.Minute)})
putStep(t, steps, delivery.Step{ID: "c", Tool: "build_node_build", Node: "ace", State: delivery.StepFailed, Started: t0.Add(2 * time.Minute), Reason: "boom"})
putStep(t, steps, delivery.Step{ID: "c", Tool: "build_node_build", Node: "ace", State: delivery.StepFailed, Started: t0.Add(2 * time.Minute), Reason: "boom", What: "check ask, novox/x@abc"})
putStep(t, steps, delivery.Step{ID: "d", Tool: "build_node_build", Node: "g14", State: delivery.StepPassed, Started: t0.Add(time.Hour)})
m := readModule(steps, fakeState{})
out, err := tool(t, m, StepsReadTool)(map[string]any{})
@@ -111,7 +111,7 @@ func TestStepsToolListsRecentStepsOfThisNode(t *testing.T) {
if !slices.Equal(ids, []string{"c", "b", "a"}) {
t.Errorf("ids = %v, want c b a", ids)
}
if lines[0].Reason != "boom" || lines[0].State != delivery.StepFailed {
if lines[0].Reason != "boom" || lines[0].What != "check ask, novox/x@abc" || lines[0].State != delivery.StepFailed {
t.Errorf("first = %+v", lines[0])
}
out, _ = tool(t, m, StepsReadTool)(map[string]any{"limit": float64(1)})
@@ -2,6 +2,7 @@ package main
import (
"context"
"git.novox.be/novox/mesh-sdk/go/buildstep"
stdio "git.novox.be/novox/mesh-sdk/go"
)
@@ -40,6 +41,7 @@ func (m *Module) Tools(steps []StepTool) []stdio.Tool {
tools = append(tools, t)
}
tools = append(tools, m.ReadTools()...)
tools = append(tools, buildstep.StepEndTool(StepEndTool, m.Node, m.stepScan(), m.Held))
tools = append(tools, m.releaseTool())
return append(tools,
stdio.Tool{Name: StatusTool,
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildnode
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026134601-17b7cc480d6d
github.com/distribution/distribution/v3 v3.1.2
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4 h1:05sVel/UOvz4/g0dxrDOeVUL7BGakVrxPBlqRq02ZhM=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026095702-79649c3fa9f4/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026134601-17b7cc480d6d h1:7F/dFerm01wvuGLPgJ67/VfdcvDexc+BQEE3qzOrlu4=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026134601-17b7cc480d6d/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/alicebob/miniredis/v2 v2.35.0 h1:QwLphYqCEAo1eu1TqPRN2jgVMPBweeQcR21jeqDCONI=
github.com/alicebob/miniredis/v2 v2.35.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
+1
View File
@@ -34,6 +34,7 @@
"build_node_release",
"build_node_step",
"build_node_steps",
"build_node_step_end",
"build_node_toolchains",
"build_node_status",
"build_node_health"
@@ -0,0 +1,246 @@
package main
import (
"fmt"
"strings"
"sync"
"testing"
"time"
sdk "git.novox.be/novox/mesh-sdk/go/delivery"
)
// One check per delivery (novox/hq ADR 0548), the gate's part: nothing merges into a repository while a check of other heads
// judges it (§2); an approved merge holds a turn on its repositories (§3); a held merge past its bound is a stalled line, read
// and handed by two tools (§4); an approved delivery whose trunk moved merges on its carried verdict with no second ask (§8).
const otherHead = "c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3"
// approvedLone is a lone delivery into oneRepo whose check passed, approved by the operator (the world approves at once).
func approvedLone(w *world) string {
w.t.Helper()
p := pr(oneRepo, 1, oneHead, "feat/one")
requiresCheck(w, p)
w.h.PullUpdated(p)
putCheckerVerdict(w, "pass", headOf(oneRepo, 1, oneHead))
w.h.CommitStatus(checkStatus(oneRepo, oneHead, "success", oneTrunk))
return IDOf(oneRepo, oneHead)
}
// memTurns is the bucket turns in memory; it logs each write with the merges the fake bridge had made by then.
type memTurns struct {
mu sync.Mutex
m map[string]sdk.Turn
log []string
merges func() int
}
func newMemTurns(w *world) *memTurns {
return &memTurns{m: map[string]sdk.Turn{}, merges: func() int { return len(w.forge.merges) }}
}
func (f *memTurns) All() (map[string]sdk.Turn, error) {
f.mu.Lock()
defer f.mu.Unlock()
out := map[string]sdk.Turn{}
for k, v := range f.m {
out[k] = v
}
return out, nil
}
func (f *memTurns) Put(t sdk.Turn) error {
if _, err := sdk.EncodeTurn(t); err != nil {
return err
}
f.mu.Lock()
defer f.mu.Unlock()
f.m[t.Repository] = t
f.log = append(f.log, fmt.Sprintf("put %s after %d merges", t.Repository, f.merges()))
return nil
}
func (f *memTurns) Delete(repository string) error {
f.mu.Lock()
defer f.mu.Unlock()
delete(f.m, repository)
f.log = append(f.log, fmt.Sprintf("delete %s after %d merges", repository, f.merges()))
return nil
}
func (f *memTurns) said() string { f.mu.Lock(); defer f.mu.Unlock(); return strings.Join(f.log, "; ") }
func TestOneCheckPerDeliveryAtTheGate(t *testing.T) {
for name, run := range map[string]func(t *testing.T){
// §2: a merge into a repository a running verdict names is not written to merges, and is written on that verdict's pass.
"an approved merge waits for a running check on its repository and merges on its end": func(t *testing.T) {
w := newWorld(t)
putCheckerVerdict(w, "running", headOf(oneRepo, 9, otherHead))
id := approvedLone(w)
w.settleAll()
if w.op.count() != 1 || len(w.forge.merges) != 0 {
t.Fatalf("asked %d times, merges %v: want the Approve asked and the merge held", w.op.count(), w.forge.merges)
}
d := w.delivery(id)
if d.Ready == nil || !strings.Contains(d.Ready.Waits, "waits for the check of g") || !strings.Contains(d.Ready.Waits, oneRepo) ||
d.Ready.HeldSince == nil {
t.Fatalf("the merge does not say what it waits for: %+v", d.Ready)
}
// §9: deliveries says what it waits for.
for _, l := range w.h.Deliveries("", "", "", false) {
if l.ID == id && !strings.Contains(l.Waits, "approved; waits for the check of g") {
t.Errorf("deliveries says it waits for %q", l.Waits)
}
}
// §4: past 30 minutes the held merge is a stalled line naming the check it waits for and since when.
for _, l := range w.h.Stalled() {
if l.ID == id {
t.Fatalf("a stalled line within the bound: %+v", l)
}
}
w.later(turnBound + time.Minute)
w.settleAll()
var line *StalledLine
for _, l := range w.h.Stalled() {
if l.ID == id {
line = &l
}
}
if line == nil || line.Bound != "30m0s" || !strings.Contains(line.Says, "waits for the check of g") || !strings.Contains(line.Says, "held since") ||
!strings.Contains(line.Says, "checker_status") || !strings.Contains(line.H2, "delivery_hand_now") {
t.Fatalf("the stalled line of the held merge = %+v", line)
}
putCheckerVerdict(w, "pass", headOf(oneRepo, 9, otherHead))
w.settleAll()
if len(w.forge.merges) != 1 || w.op.count() != 1 {
t.Fatalf("after the check's end: merges %v, asked %d times; want merged with no second ask", w.forge.merges, w.op.count())
}
},
"a running check on another repository holds nothing": func(t *testing.T) {
w := newWorld(t)
putCheckerVerdict(w, "running", headOf(oneRepo2, 9, otherHead))
approvedLone(w)
w.settleAll()
if len(w.forge.merges) != 1 {
t.Fatalf("merges %v: a check of another repository held the merge (%s)", w.forge.merges, w.logs())
}
},
"the delivery's own running verdict does not hold it": func(t *testing.T) {
w := newWorld(t)
w.op.auto = false
id := approvedLone(w)
w.settleAll()
putCheckerVerdict(w, "running", headOf(oneRepo, 1, oneHead)) // its own heads judged again (a carry)
for _, k := range w.op.open() {
w.op.answer(k, OptionApprove)
}
w.settleAll()
if d := w.delivery(id); len(w.forge.merges) != 1 || d.Ready == nil || d.Ready.HeldSince != nil {
t.Fatalf("merges %v, run %+v: its own check held it", w.forge.merges, d.Ready)
}
},
// §3: the turn is written before the verdicts are read for the merge, and deleted after the last merge.
"a lone merge's turn is written before it is handed and deleted once it merged": func(t *testing.T) {
w := newWorld(t)
turns := newMemTurns(w)
w.h.Turns = turns
approvedLone(w)
w.settleAll()
if len(w.forge.merges) != 1 || turns.said() != "put "+oneRepo+" after 0 merges; delete "+oneRepo+" after 1 merges" {
t.Fatalf("merges %v; the turns were %q", w.forge.merges, turns.said())
}
},
"a group's turns stand on every member's repository until its last member merged": func(t *testing.T) {
w := newWorld(t)
turns := newMemTurns(w)
w.h.Turns = turns
a, b := pr(oneRepo, 1, oneHead, "feat/two"), pr(oneRepo2, 2, oneHead2, "feat/two")
requiresCheck(w, a, b)
w.h.PullUpdated(a)
w.h.PullUpdated(b)
w.h.wantForge()
putCheckerVerdict(w, "pass", headOf(oneRepo, 1, oneHead), headOf(oneRepo2, 2, oneHead2))
w.h.CommitStatus(checkStatus(oneRepo, oneHead, "success", oneTrunk))
w.h.CommitStatus(checkStatus(oneRepo2, oneHead2, "success", oneTrunk))
for range 3 {
w.settleAll()
}
want := "put " + oneRepo2 + " after 0 merges; put " + oneRepo + " after 0 merges; delete " + oneRepo2 + " after 2 merges; delete " + oneRepo + " after 2 merges"
if len(w.forge.merges) != 2 || turns.said() != want {
t.Fatalf("merges %v; the turns were %q, want %q", w.forge.merges, turns.said(), want)
}
},
// §8: an approved delivery whose trunk moved goes back to proposed, waits for its carried verdict rather than being
// handed on the old one, and merges on it with no second ask.
"an approved merge whose trunk moved merges on its carried verdict with no second ask": func(t *testing.T) {
w := newWorld(t)
putCheckerVerdict(w, "running", headOf(oneRepo, 9, otherHead)) // holds the approved merge (§2)
id := approvedLone(w)
w.settleAll()
if w.op.count() != 1 || len(w.forge.merges) != 0 {
t.Fatalf("asked %d times, merges %v; want approved and held", w.op.count(), w.forge.merges)
}
w.h.PullMerged(merged(oneRepo, 5, otherHead, oneTrunk2)) // a merge at the forge moves the trunk
putCheckerVerdict(w, "pass", headOf(oneRepo, 9, otherHead))
w.settleAll()
if len(w.forge.merges) != 0 {
t.Fatalf("merges %v: handed on a verdict judged on the old trunk", w.forge.merges)
}
d := w.delivery(id)
if d.Ready == nil || !strings.Contains(d.Ready.Waits, "being carried") {
t.Fatalf("after the trunk moved it waits for %q, want its verdict carried", d.Ready.Waits)
}
// The checker's carried line: the head on the new trunk, carried from the old one.
carried := checkStatus(oneRepo, oneHead, "success", oneTrunk2)
carried.Description = "pass " + oneHead[:12] + " on " + oneTrunk2[:12] + " | carried from " + oneTrunk[:12] + " | unit 0 run, 4 kept"
if head, trunk := checkLine(carried.Description); head != oneHead[:12] || trunk != oneTrunk2[:12] {
t.Fatalf("the carried line reads as %s on %s", head, trunk)
}
if head, trunk := checkLine("failed: a test"); head != "" || trunk != "" {
t.Fatalf("a line that names no commits gave %q %q", head, trunk)
}
putCheckerVerdict(w, "pass", headOf(oneRepo, 1, oneHead))
w.h.CommitStatus(carried)
w.settleAll()
if w.op.count() != 1 {
t.Errorf("the Approve was asked %d times, want once: it stands through the carry", w.op.count())
}
if d := w.delivery(id); d.Check == nil || d.Check.Trunk != oneTrunk2[:12] || len(w.forge.merges) != 1 {
t.Errorf("after the carry: verdict %+v, merges %v; want merged on the carried verdict", d.Check, w.forge.merges)
}
},
// §4: delivery_turns reads every turn with the check it waits for; delivery_hand_now hands a held merge at once on the
// operator's word, and refuses one no check holds.
"the turns are read, and a held merge is handed now on the operator's word": func(t *testing.T) {
w := newWorld(t)
w.h.Turns = newMemTurns(w)
putCheckerVerdict(w, "running", headOf(oneRepo, 9, otherHead))
id := approvedLone(w)
w.settleAll()
read := w.h.TurnsRead()
lines := read["turns"].([]TurnLine)
if len(lines) != 1 || lines[0].Repository != oneRepo || len(lines[0].Turn.Waiting) != 1 || lines[0].Turn.Waiting[0].WaitsFor == "" ||
!strings.Contains(lines[0].Waits["feat/one"].(string), "the check g.") || read["written"] != true {
t.Fatalf("delivery_turns = %+v", read)
}
if _, err := w.h.HandNow(id, ""); err == nil || !strings.Contains(err.Error(), "why") {
t.Errorf("a hand without the operator's why: %v", err)
}
if _, err := w.h.HandNow(IDOf(oneRepo2, oneHead2), "x"); err == nil {
t.Error("an unknown delivery was handed")
}
if _, err := w.h.HandNow(id, "the operator, on telegram: merge it now"); err != nil {
t.Fatal(err)
}
w.settleAll()
if len(w.forge.merges) != 1 || w.delivery(id).Ready.HandNow == "" {
t.Fatalf("merges %v after the hand", w.forge.merges)
}
if _, err := w.h.HandNow(id, "again"); err == nil {
t.Error("a merged delivery was handed again")
}
},
} {
t.Run(name, run)
}
}
@@ -224,6 +224,9 @@ func (h *Holder) nextStall(now time.Time) time.Time {
consider(r.RefusedSince.Add(h.answeredRefusalBound(d.Repository)))
consider(r.RefusedSince.Add(outOfDateFor))
}
if r.HeldSince != nil {
consider(r.HeldSince.Add(turnBound))
}
}
return next
}
@@ -12,6 +12,7 @@ import (
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
sdk "git.novox.be/novox/mesh-sdk/go/delivery"
)
// The holder of the mesh-delivery seat (novox/hq ADR 0239, to-be 47): the one writer of deliveries and
@@ -75,8 +76,14 @@ type Holder struct {
// unchecked are the open pull requests' heads seen without a merge check (unchecked.go).
unchecked map[string]unchecked
// checker are the checker's verdicts by the hash of the heads they judged, as its verdicts bucket holds them
// (onestatus.go, issue 1302).
checker map[string]checkerVerdict
// (onestatus.go, issue 1302); runningSince when this owner first saw each running (ADR 0548 §2, turns.go).
checker map[string]checkerVerdict
runningSince map[string]time.Time
// Turns is the gate's bucket turns (turns.go, ADR 0548 §3); nil keeps none. turnsHeld are the turns as written there,
// by repository, read once at the first look (turnsLoaded).
Turns TurnStore
turnsHeld map[string]sdk.Turn
turnsLoaded bool
// tried counts the owed effects Flush tried since this owner started, by how each ended (novox/hq issue 407).
tried owedTries
// forgeWanted is whether the forge's open pull requests are to be read at the next React: a check may have
@@ -126,6 +133,7 @@ func (h *Holder) init() {
h.lostRead = map[string]time.Time{}
h.unchecked = map[string]unchecked{}
h.checker = map[string]checkerVerdict{}
h.runningSince, h.turnsHeld = map[string]time.Time{}, map[string]sdk.Turn{}
h.started = h.Now()
}
@@ -6,7 +6,7 @@ import "git.novox.be/novox/mesh-sdk/go/announce"
// issue 1790): the module says it itself, from its code, before it serves, and a reader is granted what it said
// (clause 3). A state is readable when another module's manifest reads it (keeps_test.go names the readers). keeps_test.go holds the list to the
// manifest's `state`, and keeps_integration_test.go that the module announces it on a bus.
var Kept = []announce.State{{Name: "asks", Readable: true}, {Name: "conditions"}, {Name: "deliveries"}, {Name: "groups"}, {Name: "merges", Readable: true}, {Name: "owed", Readable: true}, {Name: "pulls", Readable: true}, {Name: "source", Readable: true}}
var Kept = []announce.State{{Name: "asks", Readable: true}, {Name: "conditions"}, {Name: "deliveries"}, {Name: "groups"}, {Name: "merges", Readable: true}, {Name: "owed", Readable: true}, {Name: "pulls", Readable: true}, {Name: "source", Readable: true}, {Name: "turns", Readable: true}}
// announceKept says Kept through keeps, which is stdio.Keeps.
func announceKept(keeps func(...announce.State) error) error { return keeps(Kept...) }
@@ -17,6 +17,9 @@ var readers = map[string][]string{
"pulls": {"build-agent"}, // in mesh-controller
"owed": {"mesh-delivery-forge-bridge"},
"source": {"mesh-delivery-indexer", "mesh-delivery-checker"},
// ADR 0548 §3, §4: the checker reads the merge turns and claims no check of a repository one names; the gate alone writes
// them (a state is written by its owner's account alone, ADR 0495 §3: announcing it readable grants a read, never a write).
"turns": {"mesh-delivery-checker"},
}
// 1790/S: the states the module announces are the states its manifest declares, no more, no fewer, and the readable
@@ -65,6 +65,7 @@ func main() {
Operator: kvOperator{},
Merges: kvMerges{},
Source: kvSource{},
Turns: kvTurns{},
Owed: kvOwed{},
Pulls: kvPulls{},
// The gate says what stalled to the operator itself (issue 1275, ADR 0437 §9).
@@ -357,6 +358,24 @@ func tools(h *Holder, l *listening) []stdio.Tool {
}
return h.Close(strArg(a, "id"), strArg(a, "why"))
}},
{Name: "delivery_turns",
Description: "Every merge turn (novox/hq ADR 0548 §3): per repository, the approved merges that wait to merge into it in the order of " +
"their Approve, the check of another delivery each waits for and since when, whether the gate wrote the turn to its bucket " +
"turns yet (the checker claims no check of a repository a turn names), and the bound past which a held merge is a stalled " +
"line. Read-only.",
Run: func(map[string]any) (any, error) { return h.TurnsRead(), nil }},
{Name: "delivery_hand_now",
Description: "On the operator's word, hand an approved merge held for another delivery's running check to the merge at once " +
"(novox/hq ADR 0548 §4): the check it waited for runs on and is carried to the new trunk after the merge. Refused for a " +
"delivery that is not an approved merge held by a check, naming why. The why is kept on the delivery.",
Input: map[string]any{"type": "object", "properties": map[string]any{"id": str("the delivery's id (owner/repository@commit)"),
"why": str("the operator's word, kept on the delivery")}, "required": []string{"id", "why"}},
Run: func(a map[string]any) (any, error) {
if err := need(a, "id", "why"); err != nil {
return nil, err
}
return h.HandNow(strArg(a, "id"), strArg(a, "why"))
}},
{Name: "delivery_status",
Description: "Whether this owner works: how many deliveries in each state, the groups, what is owed to " +
"the forge and the bus and not yet done, how many owed effects were tried since it started (done, failed " +
@@ -91,9 +91,10 @@ func TestItClaimsTheDeliverySeatAndFollowsWhatItHandles(t *testing.T) {
if !reflect.DeepEqual(m.Emits, []string{"transition", "group", "merged"}) {
t.Fatalf("emits %v", m.Emits)
}
// Eight buckets, and no `asked` (the teller asks, ADR 0437 §6): ADR 0450 §3, and the conditions it raised (issue 1275).
// Nine buckets, and no `asked` (the teller asks, ADR 0437 §6): ADR 0450 §3, the conditions it raised (issue 1275) and the
// merge turns (ADR 0548 §3).
buckets := m.buckets()
if !reflect.DeepEqual(buckets, []string{"deliveries", "groups", "pulls", "asks", "merges", "owed", "source", "conditions"}) {
if !reflect.DeepEqual(buckets, []string{"deliveries", "groups", "pulls", "asks", "merges", "owed", "source", "conditions", "turns"}) {
t.Fatalf("state %v", buckets)
}
// What it reads: the teller's answers and the checker's verdicts (issue 1302). The carry from today's module's buckets, the
@@ -72,8 +72,12 @@ func (h *Holder) CheckerVerdictChanged(key string, value []byte) {
if i := strings.LastIndex(key, "."); i >= 0 {
hash = key[i+1:]
}
if h.runningSince == nil {
h.runningSince = map[string]time.Time{}
}
if len(value) == 0 || string(value) == "null" {
delete(h.checker, hash)
delete(h.runningSince, hash)
h.Poke()
return
}
@@ -85,6 +89,11 @@ func (h *Holder) CheckerVerdictChanged(key string, value []byte) {
if v.Heads == "" {
v.Heads = hash
}
if prev, ok := h.checker[v.Heads]; v.State == checkerRunning && (!ok || prev.State != checkerRunning) {
h.runningSince[v.Heads] = h.Now() // a check starts: a merge it holds says since when (ADR 0548 §2)
} else if v.State != checkerRunning {
delete(h.runningSince, v.Heads)
}
h.checker[v.Heads] = v
h.Poke()
}
@@ -199,14 +199,3 @@ func TestAClosedPullRequestAsksNoForgetChecks(t *testing.T) {
w.h.PullUpdated(p)
w.h.PullClosed(p)
}
// 1302: the mesh/check line gives the head and the trunk it judged.
func TestTheCheckLineGivesHeadAndTrunk(t *testing.T) {
head, trunk := checkLine("pass " + oneHead[:12] + " on " + oneTrunk[:12] + " | unit 4 run, 0 kept | group g of 2 pull requests")
if head != oneHead[:12] || trunk != oneTrunk[:12] {
t.Fatalf("head %q trunk %q", head, trunk)
}
if head, trunk = checkLine("failed: a test"); head != "" || trunk != "" {
t.Fatalf("a line that names no commits gave %q %q", head, trunk)
}
}
@@ -121,6 +121,13 @@ type ReadyRun struct {
Declined *time.Time `json:"declined,omitempty"`
Warrant string `json:"warrant,omitempty"`
Refused string `json:"refused,omitempty"`
// One check per delivery (novox/hq ADR 0548 §2, §4, turns.go): HeldSince is when the approved merge began to wait for the
// check of other heads HeldFor names (its verdict key), running on a repository it goes into; HandNow the operator's word
// that handed it at once past that check.
HeldSince *time.Time `json:"held_since,omitempty"`
HeldFor string `json:"held_for,omitempty"`
HandNow string `json:"hand_now,omitempty"`
}
// MergeBranch is the one branch this owner merges into (novox/hq ADR 0398 §4): a merge into any other is its author's.
@@ -244,6 +251,9 @@ func (h *Holder) readyAsks(now time.Time) []func() {
if waits == "" {
waits, approvalAsk = h.approval(d, now)
}
if waits == "" && gates {
waits = h.turnWaits(d, now) // nothing merges into a repository while a check of other heads judges it (ADR 0548 §2)
}
if waits != r.Waits {
r.Waits = waits
h.keep(d)
@@ -267,6 +277,7 @@ func (h *Holder) readyAsks(now time.Time) []func() {
asks = append(asks, a)
}
}
asks = append(asks, h.turnWrites()...) // the merge turns, after the holds of this look (ADR 0548 §3)
asks = append(asks, h.settleMerges()...)
asks = append(asks, h.settleForgeMerges()...)
asks = append(asks, h.withdrawnKeys()...)
@@ -376,6 +387,9 @@ func (h *Holder) readyWaits(d *Delivery) string {
case d.State == Proposed && d.outOfDate() != "" && d.AskedAgain != nil:
return "its check against the trunk's head is asked and has not answered"
case d.State == Proposed && d.outOfDate() != "" && d.Check.Passes() && h.covering(d) != nil:
// Approved, and its trunk moved: the checker carries its verdict to the trunk's head by its package digests (ADR 0548
// §5, §8). The Approve stands and is not asked again; the merge waits for the carried verdict, not a fresh check.
return "approved; its verdict is being carried to the trunk's head " + shortOf(d.TrunkNow, 12) + " (" + d.outOfDate() + ")"
case d.State == Proposed && d.outOfDate() != "":
return "its verdict is out of date: its check against the trunk's head is asked before the operator is"
case d.State == Proposed && d.AskedAgain != nil:
@@ -0,0 +1,454 @@
package main
import (
"encoding/json"
"fmt"
"sort"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
sdk "git.novox.be/novox/mesh-sdk/go/delivery"
)
// **One check per delivery** (novox/hq ADR 0548 §2 to §4, issue 568). While a check runs, nothing merges into a repository it
// judges, and an approved merge holds a turn on its repositories so no new check starts there:
//
// - the hold (§2): an approved merge is handed to the forge bridge only when no verdict of the checker's verdicts bucket in
// the state running names, among its trunk heads, a repository the merge goes into; the verdict of the merge's own heads
// excepted. The gate watches that bucket already (onestatus.go), so the verdict leaving running gives the waiting merge its
// turn at once, with no timer. Siblings a check reads beside are not held (§2). Only while mesh/check gates the merge;
// - the turn (§3): before it reads the verdicts for a merge, the gate writes one key per repository the merge goes into in its
// bucket turns (`<owner>.<name>`, sdk.TurnsKey): the deliveries that wait there in the order of their Approve, the check
// each waits for and since when; for a group on every member's repository until its last member merged. The checker reads
// the bucket and claims no check of a repository a turn names. The key goes when the last delivery in it merged or left;
// - the bound and the tools (§4): a merge held past turnBound is a stalled line; delivery_turns reads every turn and
// delivery_hand_now hands a waiting merge at once on the operator's word, the check it waited for then carried.
// turnBound is how long an approved merge may wait for another delivery's check before it is a stalled line (ADR 0548 §4).
const turnBound = 30 * time.Minute
// TurnStore is the gate's bucket turns: one key per repository, written by the gate alone.
type TurnStore interface {
// All reads every turn, by repository (lowercase).
All() (map[string]sdk.Turn, error)
Put(t sdk.Turn) error
Delete(repository string) error
}
// kvTurns is the bucket turns over the runtime.
type kvTurns struct{}
func (kvTurns) All() (map[string]sdk.Turn, error) {
keys, err := stdio.State("turns").Keys()
if err != nil {
return nil, err
}
out := map[string]sdk.Turn{}
for _, k := range keys {
e, err := stdio.State("turns").Get(k)
if err != nil {
return nil, err
}
if e == nil {
continue
}
repository, err := sdk.ParseTurnsKey(k)
if err != nil {
continue // not a turn's key: never the gate's, left
}
t, err := sdk.DecodeTurn(e.Value)
if err != nil {
t = sdk.Turn{Repository: repository} // unreadable: rewritten or deleted at the next look
}
out[repository] = t
}
return out, nil
}
func (kvTurns) Put(t sdk.Turn) error {
key, err := sdk.TurnsKey(t.Repository)
if err != nil {
return err
}
raw, err := sdk.EncodeTurn(t)
if err != nil {
return err
}
_, err = stdio.State("turns").Put(key, json.RawMessage(raw))
return err
}
func (kvTurns) Delete(repository string) error {
key, err := sdk.TurnsKey(repository)
if err != nil {
return err
}
return stdio.State("turns").Delete(key)
}
// approvedMerge is an approved merge not yet done: the run holding its Approve, the deliveries it covers, the hash of their
// heads as the checker keys their verdict (lowercase, onestatus.go headsOf), and the repositories it goes into (lowercase).
type approvedMerge struct {
owner *Delivery
run *ReadyRun
group string
heads string
repos []string
}
// approvedMerges are the merges the operator approved that are not done, in the order of their Approve: each ask once, its
// run on the delivery that asked it. A declined, refused, merged, rejected or final one has left. Under the lock.
func (h *Holder) approvedMerges() []approvedMerge {
var out []approvedMerge
seen := map[string]bool{}
ids := make([]string, 0, len(h.deliveries))
for id := range h.deliveries {
ids = append(ids, id)
}
sort.Strings(ids) // the same owner of an ask at every look, whichever member's run holds it
for _, id := range ids {
d := h.deliveries[id]
r := d.Ready
if r == nil || r.Approved == nil || r.Declined != nil || r.AskID == "" || len(r.Heads) == 0 || seen[r.AskID] ||
d.State.Final() || d.State == Rejected || d.Base != MergeBranch || r.RefusedSince != nil || h.allMerged(r.Heads) {
continue
}
seen[r.AskID] = true
var heads []sdk.Head
repos := map[string]bool{}
for id, sha := range r.Heads {
m := h.deliveries[id]
if m == nil {
continue
}
heads = append(heads, sdk.Head{Repository: strings.ToLower(m.Repository), Number: m.Number, Head: strings.ToLower(sha)})
repos[strings.ToLower(m.Repository)] = true
}
hash, err := sdk.HeadsHash(heads)
if err != nil {
continue
}
am := approvedMerge{owner: d, run: r, group: askGroup(d), heads: hash}
for repo := range repos {
am.repos = append(am.repos, repo)
}
sort.Strings(am.repos)
out = append(out, am)
}
sort.SliceStable(out, func(i, j int) bool {
if !out[i].run.Approved.Equal(*out[j].run.Approved) {
return out[i].run.Approved.Before(*out[j].run.Approved)
}
return out[i].heads < out[j].heads
})
return out
}
// mergeOfDelivery is the approved merge covering d, if one is approved.
func (h *Holder) mergeOfDelivery(d *Delivery) (approvedMerge, bool) {
c := h.covering(d)
if c == nil || c.Approved == nil {
return approvedMerge{}, false
}
for _, am := range h.approvedMerges() {
// By the ask, not the run: a group's members may each hold a run of the same ask, and the merge is the ask's.
if am.run.AskID == c.AskID {
return am, true
}
}
return approvedMerge{}, false
}
// runningCheck is a check of the checker in the state running that names repo among its trunk heads, its heads not own: its
// verdict key and since when the gate has seen it run. False when none runs there. Under the lock.
func (h *Holder) runningCheck(repo, own string) (key string, v checkerVerdict, since time.Time, ok bool) {
hashes := make([]string, 0, len(h.checker))
for hash := range h.checker {
hashes = append(hashes, hash)
}
sort.Strings(hashes)
for _, hash := range hashes {
v := h.checker[hash]
if v.State != checkerRunning || v.Heads == own {
continue
}
for r := range v.Trunk {
if strings.EqualFold(r, repo) {
key, err := sdk.CheckerVerdictKey(v.Group, v.Heads)
if err != nil {
key = v.Group + "." + v.Heads
}
return key, v, h.runningSince[v.Heads], true
}
}
}
return "", checkerVerdict{}, time.Time{}, false
}
// turnWaits is what an approved merge waits for before it is handed (ADR 0548 §2, §3): its turns written, then the end of
// every check of other heads running on a repository it goes into. Empty when it may be handed. It keeps on the run since
// when it is held and for which check, for the stalled line and the tools. Under the lock.
func (h *Holder) turnWaits(d *Delivery, now time.Time) string {
am, ok := h.mergeOfDelivery(d)
if !ok {
return ""
}
r := am.run
if h.Turns != nil {
if !h.turnsLoaded {
return "approved; its merge turn waits for the turns the gate wrote before to be read"
}
for _, repo := range am.repos {
if !turnNames(h.turnsHeld[repo], am.heads) {
return "approved; its merge turn on " + repo + " is being written, so no check starts there before it merges"
}
}
}
if r.HandNow != "" {
if r.HeldSince != nil {
r.HeldSince, r.HeldFor = nil, ""
h.keep(am.owner)
}
return "" // handed now on the operator's word: the check it waited for is carried after the merge
}
for _, repo := range am.repos {
key, v, since, running := h.runningCheck(repo, am.heads)
if !running {
continue
}
if r.HeldSince == nil || r.HeldFor != key {
at := now
r.HeldSince, r.HeldFor = &at, key
h.keep(am.owner)
h.Logf("[mesh-delivery] %s is approved and held: the check %s runs on %s, and nothing merges there while it does (ADR 0548 §2)", am.owner.ID, key, repo)
}
words := fmt.Sprintf("approved; waits for the check of %s (heads %s) on %s", v.Group, v.Heads, repo)
if !since.IsZero() {
words += ", running since " + since.UTC().Format(time.RFC3339)
}
return words + " (novox/hq ADR 0548 §2)"
}
if r.HeldSince != nil {
r.HeldSince, r.HeldFor = nil, ""
h.keep(am.owner)
}
return ""
}
// turnNames says whether a turn names the heads.
func turnNames(t sdk.Turn, heads string) bool {
for _, w := range t.Waiting {
if w.Heads == heads {
return true
}
}
return false
}
// wantedTurns are the turns the approved merges hold now, by repository: each merge on every repository it goes into, in
// the order of their Approve, with the check of other heads it waits for there. Under the lock.
func (h *Holder) wantedTurns() map[string]sdk.Turn {
out := map[string]sdk.Turn{}
for _, am := range h.approvedMerges() {
for _, repo := range am.repos {
t := out[repo]
t.Repository = repo
w := sdk.TurnWait{Group: am.group, Heads: am.heads, Ask: am.run.AskID, Approved: am.run.Approved.UTC(), Since: am.run.Approved.UTC()}
if key, _, since, ok := h.runningCheck(repo, am.heads); ok {
w.WaitsFor = key
if !since.IsZero() {
w.WaitsSince = since.UTC()
}
}
t.Waiting = append(t.Waiting, w)
out[repo] = t
}
}
return out
}
// turnWrites are the writes that bring the bucket turns to what the approved merges hold: a turn written where it changed,
// a key deleted where its last delivery merged or left. The first look reads what a gate before this one wrote. Under the
// lock; the writes run outside it, and each that lands is kept and asks for a look.
func (h *Holder) turnWrites() []func() {
if h.Turns == nil {
return nil
}
if !h.turnsLoaded {
return []func(){func() {
held, err := h.Turns.All()
h.mu.Lock()
defer h.mu.Unlock()
if err != nil {
h.Logf("[mesh-delivery] the merge turns written before cannot be read, so no merge is handed yet: %v; asked again in %s", err, askAgainAfter)
h.again("the merge turns written before, read")
return
}
h.turnsHeld, h.turnsLoaded = held, true
h.Poke()
}}
}
want := map[string]sdk.Turn{}
if h.checkGates() {
want = h.wantedTurns() // while the check gates no merge, no merge waits for one, and no turn holds a check
}
var out []func()
repos := make([]string, 0, len(want)+len(h.turnsHeld))
for repo := range want {
repos = append(repos, repo)
}
for repo := range h.turnsHeld {
if _, ok := want[repo]; !ok {
repos = append(repos, repo)
}
}
sort.Strings(repos)
for _, repo := range repos {
t, wanted := want[repo]
held, had := h.turnsHeld[repo]
switch {
case wanted && had && sameTurnValue(held, t):
case wanted:
out = append(out, func() {
err := h.Turns.Put(t)
h.mu.Lock()
defer h.mu.Unlock()
if err != nil {
h.Logf("[mesh-delivery] the merge turn on %s cannot be written: %v; asked again in %s", repo, err, askAgainAfter)
h.again("a merge turn, written")
return
}
h.turnsHeld[repo] = t
h.Poke()
})
default:
out = append(out, func() {
err := h.Turns.Delete(repo)
h.mu.Lock()
defer h.mu.Unlock()
if err != nil {
h.Logf("[mesh-delivery] the merge turn on %s cannot be deleted: %v; asked again in %s", repo, err, askAgainAfter)
h.again("a merge turn, deleted")
return
}
delete(h.turnsHeld, repo)
h.Logf("[mesh-delivery] the merge turn on %s is deleted: no approved merge waits on it, so the checker may start a check there", repo)
h.Poke()
})
}
}
return out
}
// sameTurnValue says whether two turns hold the same value, the version aside.
func sameTurnValue(a, b sdk.Turn) bool {
a.V, b.V = 0, 0
return sdk.SameTurn(a, b)
}
// turnLines are the stalled lines of approved merges held past turnBound by another delivery's check (ADR 0548 §4). Under the
// lock.
func (h *Holder) turnLines(now time.Time) []StalledLine {
var out []StalledLine
for _, d := range h.deliveries {
r := d.Ready
if r == nil || r.HeldSince == nil || r.Merged != nil || d.MergedAs != "" || d.State.Final() || now.Sub(*r.HeldSince) < turnBound {
continue
}
words := r.Waits
if words == "" {
words = "approved; waits for the check " + r.HeldFor
}
out = append(out, StalledLine{ID: d.ID, Number: d.Number, State: d.State, For: now.Sub(*r.HeldSince).Round(time.Second).String(),
Bound: turnBound.String(),
H2: "none: the check it waits for ends, or the operator hands the merge now with delivery_hand_now — the operator's",
Says: fmt.Sprintf("%s#%d at %s is %s, held since %s; the step that check is at is read with the checker's checker_status",
d.Repository, d.Number, shortOf(r.Head, 12), words, r.HeldSince.UTC().Format(time.RFC3339))})
}
return out
}
// TurnLine is one repository's turn as delivery_turns reads it: the turn as the gate holds it and, for each delivery waiting,
// the check it waits for in words.
type TurnLine struct {
Repository string `json:"repository"`
Turn sdk.Turn `json:"turn"`
Waits map[string]any `json:"waits,omitempty"`
}
// TurnsRead is the tool delivery_turns: every turn the gate holds, with the check each waiting delivery waits for, since when,
// and whether the merge was handed now on the operator's word.
func (h *Holder) TurnsRead() map[string]any {
h.mu.Lock()
defer h.mu.Unlock()
want := h.wantedTurns()
repos := make([]string, 0, len(want))
for repo := range want {
repos = append(repos, repo)
}
sort.Strings(repos)
lines := []TurnLine{}
for _, repo := range repos {
l := TurnLine{Repository: repo, Turn: want[repo], Waits: map[string]any{}}
for _, w := range want[repo].Waiting {
says := "nothing: no check of other heads runs on " + repo
if w.WaitsFor != "" {
says = "the check " + w.WaitsFor
if !w.WaitsSince.IsZero() {
says += ", running since " + w.WaitsSince.Format(time.RFC3339)
}
}
l.Waits[w.Group] = says
}
lines = append(lines, l)
}
out := map[string]any{"turns": lines, "check-gates-merge": h.checkGates(), "bound": turnBound.String()}
if h.Turns != nil {
out["written"] = h.turnsLoaded
var unwritten []string
for _, repo := range repos {
if held, ok := h.turnsHeld[repo]; !ok || !sameTurnValue(held, want[repo]) {
unwritten = append(unwritten, repo)
}
}
for repo := range h.turnsHeld {
if _, ok := want[repo]; !ok {
unwritten = append(unwritten, repo+" (to delete)")
}
}
sort.Strings(unwritten)
if len(unwritten) > 0 {
out["not-yet-written"] = unwritten
}
}
return out
}
// HandNow is the tool delivery_hand_now (ADR 0548 §4): on the operator's word an approved merge held for another delivery's
// check is handed at once; that check runs on and is carried to the new trunk after the merge. Refused for a delivery that is
// not an approved merge waiting, naming why.
func (h *Holder) HandNow(id, why string) (string, error) {
h.mu.Lock()
defer h.mu.Unlock()
d := h.deliveries[id]
if d == nil {
return "", fmt.Errorf("no delivery %q", id)
}
if strings.TrimSpace(why) == "" {
return "", fmt.Errorf("say why: the hand is the operator's word, kept on the delivery")
}
am, ok := h.mergeOfDelivery(d)
if !ok {
return "", fmt.Errorf("%s is not an approved merge waiting: nothing to hand (its state is %s)", id, d.State)
}
if am.run.HeldSince == nil {
return "", fmt.Errorf("%s waits for no check of another delivery: %s", id, firstOf(am.run.Waits, "it is handed at its turn"))
}
am.run.HandNow = why
h.keep(am.owner)
h.mergeAgainNow(d)
h.Logf("[mesh-delivery] %s is handed now on the operator's word, past the check %s it waited for, which is carried after the merge: %s",
id, am.run.HeldFor, why)
return fmt.Sprintf("%s is handed to the merge at the next look; the check %s it waited for runs on and is carried to the new trunk", id, am.run.HeldFor), nil
}
@@ -26,6 +26,10 @@ type Line struct {
// waitsFor is what a delivery waits for, in words.
func (h *Holder) waitsFor(d *Delivery) string {
if r := d.Ready; r != nil && r.Merged == nil && strings.HasPrefix(r.Waits, "approved;") {
// Approved and waiting for another delivery's check, or for its own verdict's carry (ADR 0548 §2, §8, §9).
return r.Waits
}
switch d.State {
case Proposed:
return "its check"
@@ -218,6 +222,7 @@ func (h *Holder) Stalled() []StalledLine {
}
out = append(out, h.withChecks(h.uncheckedLines(now))...)
out = append(out, h.readyLines(now)...)
out = append(out, h.turnLines(now)...) // an approved merge held past its bound by another delivery's check (ADR 0548 §4)
out = append(out, h.groupLines(now)...)
sort.Slice(out, func(i, j int) bool { return out[i].ID < out[j].ID })
return out
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-gate
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024210118-748c564cdf7c
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026111253-9c61918ac36f
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024210118-748c564cdf7c h1:pY2YpHSSsfDPBrOEiWTNSIPn0zdnAayRmtmmyQ3hCHY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024210118-748c564cdf7c/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026111253-9c61918ac36f h1:9+TZpNq8Ifj1+qmbidE6l/7qkbua4yLdzRMrezNvf1A=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026111253-9c61918ac36f/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
+8 -2
View File
@@ -61,14 +61,20 @@
"name": "source",
"history": 64
},
"conditions"
"conditions",
{
"name": "turns",
"history": 5
}
],
"reads": [
"mesh-delivery-teller.answers",
"mesh-delivery-checker.verdicts"
],
"tools": [
"delivery_status"
"delivery_hand_now",
"delivery_status",
"delivery_turns"
],
"config": {
"refusal-stall-floor-minutes": {
@@ -515,6 +515,9 @@ func (k *KB) accept(ctx context.Context, held *Entry, w *Record, by, why, warran
// judgeSet holds the whole record set to its 4 rules and keeps what fails, which the module's health answers.
func (k *KB) judgeSet(ctx context.Context) []Finding {
if _, err := k.RecomputeMarks(ctx); err != nil {
k.logf("[mesh-knowledge] %v", err)
}
all, err := k.Store.All(ctx)
if err != nil {
k.logf("[mesh-knowledge] the whole record set cannot be read to judge it: %v", err)
@@ -99,7 +99,7 @@ func serve(k *KB, status string) {
k.kick = make(chan struct{}, 1)
go hear(k)
go background(k)
if err := stdio.Serve("", append(Tools(k, callerOf), healthTool(status, k.now), highestTool(k))); err != nil {
if err := stdio.Serve("", append(Tools(k, callerOf), healthTool(status, k.now), highestTool(k), marksTool(k))); err != nil {
logf("%v", err)
os.Exit(1)
}
@@ -119,7 +119,7 @@ func TestItsProcessesAreThePreparationTheActAndTheJudge(t *testing.T) {
}
h, _ := watch["health"].(map[string]any)
if watch == nil || watch["run"].([]any)[1] != "watch" || h["kind"] != "tool" || h["tool"] != HealthTool ||
!reflect.DeepEqual(m.Tools, []string{HealthTool, HighestTool}) {
!reflect.DeepEqual(m.Tools, []string{HealthTool, HighestTool, MarksTool}) {
t.Fatalf("watch %v, tools %v", watch, m.Tools)
}
// The watch's verdict is whether the database serves, so its check names the provision it exercises: while the
@@ -0,0 +1,137 @@
package main
// The marks on a clause (changed, and by what) follow the accepted links, both ways: an acceptance that adds a
// changes link sets the mark (accept), and a later accepted version that drops the link withdraws it here. Without
// this a mark outlived its link, so holds and the whole-set rule `partial` said "changed" while links and get said
// nothing (ADR 0536 v6 changed ADR 0525 §10; v12 only cited it).
import (
"context"
"fmt"
"strconv"
"strings"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// MarksTool is the module's tool that recomputes every mark now.
const MarksTool = "knowledge_marks"
// markedBy reads "ADR 0536 §4" or "research 045" as the record it names.
func markedBy(s string) (Key, bool) {
f := strings.Fields(s)
if len(f) < 2 {
return Key{}, false
}
n, err := strconv.Atoi(f[1])
if err != nil {
return Key{}, false
}
switch strings.ToLower(f[0]) {
case "adr":
return Key{KindDecision, n}, true
case "research":
return Key{KindResearch, n}, true
}
return Key{}, false
}
// StaleMarks drops, on every record of the set, each "changed by" mark whose changing record is accepted and no
// longer links the clause with a changes link; a clause left with no mark is in force again. A mark of a record
// not accepted (a change confirmed ahead of its acceptance) or of a record not held stays. It changes the
// records in place and answers the keys it changed with what it did.
func StaleMarks(set []*Record) map[Key][]string {
byKey := map[Key]*Record{}
for _, r := range set {
byKey[r.Key()] = r
}
did := map[Key][]string{}
for _, t := range set {
if t.Status == StatusSuperseded {
continue
}
for i := range t.Clauses {
c := &t.Clauses[i]
if c.Status != ClauseChanged {
continue
}
var kept []string
for _, by := range c.ChangedBy {
from, ok := markedBy(by)
src := byKey[from]
if !ok || src == nil || !src.Accepted() || changesClause(src, t.Key(), c.N) {
kept = append(kept, by)
continue
}
did[t.Key()] = append(did[t.Key()], fmt.Sprintf("§%d no longer changed by %s", c.N, by))
}
if len(kept) != len(c.ChangedBy) {
c.ChangedBy = kept
if len(kept) == 0 {
c.Status = ClauseInForce
}
}
}
}
return did
}
func changesClause(src *Record, to Key, clause int) bool {
for _, l := range src.Links {
if k, ok := l.To.Record(); ok && l.Kind == LinkChanges && k == to && l.To.Clause == clause {
return true
}
}
return false
}
// RecomputeMarks withdraws every stale mark of the record set now, each a new version of the record it frees, and
// says what it withdrew. judgeSet runs it first, so an acceptance, the background work every few minutes and a
// start all agree with the links.
func (k *KB) RecomputeMarks(ctx context.Context) ([]string, error) {
all, err := k.Store.All(ctx)
if err != nil {
return nil, fmt.Errorf("the record set cannot be read to recompute its marks: %v", err)
}
set := make([]*Record, len(all))
for i, e := range all {
set[i] = e.Current.Clone()
}
did := StaleMarks(set)
var said []string
for i, e := range all {
what := did[e.Current.Key()]
if len(what) == 0 {
continue
}
why := strings.Join(what, "; ") + " (its changing record's accepted links no longer change it)"
if err := k.Store.Save(ctx, Version{By: "the knowledge base", At: k.now(), Why: why, Record: set[i]}, true, false, e.Revision); err != nil {
k.logf("[mesh-knowledge] the marks of %s could not be recomputed (%v): the next judgement tries again", e.Current.Key(), err)
continue
}
k.emit(EventChanged, set[i], "the knowledge base", why, e.Versions+1)
said = append(said, fmt.Sprintf("%s: %s", e.Current.Key(), why))
}
return said, nil
}
func marksTool(k *KB) stdio.Tool {
return stdio.Tool{Name: MarksTool,
Description: "Recompute every clause's changed and changed-by marks from the accepted links now, withdrawing each mark whose " +
"changing record no longer links the clause (the clause is in force again), and answer what it withdrew. The " +
"knowledge base does this on every acceptance and every few minutes; this runs it at once.",
Input: schema(nil, map[string]any{}),
Run: func(m map[string]any) (any, error) {
if _, err := readArgs(MarksTool, m); err != nil {
return nil, err
}
k.mu.Lock()
said, err := k.RecomputeMarks(context.Background())
k.mu.Unlock()
if err != nil {
return nil, err
}
findings := k.judgeSet(context.Background())
return map[string]any{"withdrawn": said, "findings": len(findings)}, nil
}}
}
@@ -117,6 +117,37 @@ func TestARecordAcceptedCitingANewDecisionLetsItBeAcceptedAndDroppingTheLinkIsLo
})
}
// A mark follows the accepted links: a later accepted version that drops a changes link withdraws the clause's
// mark, so holds, get and the rule `partial` agree with links (ADR 0536 v6 changed ADR 0525 §10; v12 only cited it).
func TestACorrectionThatDropsAChangesLinkWithdrawsTheClausesMark(t *testing.T) {
eachStore(t, func(t *testing.T, s Store) {
w := imported(t, s)
n := w.reserved(t, "author@g14")
a := valid(n) // changes 0276 §1
w.accepted(t, "author@g14", a)
if c := mustGet(t, s, Key{KindDecision, 276}).Current.Clause(1); c.Status != ClauseChanged || len(c.ChangedBy) != 1 {
t.Fatalf("0276 §1 is not marked changed by the acceptance: %+v", c)
}
a.Links = a.Links[:1] // only extends 0276 now
a.Insight = "it no longer changes 0276 §1"
w.accepted(t, "author@g14", a)
c := mustGet(t, s, Key{KindDecision, 276}).Current.Clause(1)
if c.Status != ClauseInForce || len(c.ChangedBy) != 0 {
t.Fatalf("0276 §1 keeps a mark no accepted link makes: %+v", c)
}
f, _ := s.Findings(context.Background())
for _, x := range f {
if x.Rule == "partial" {
t.Fatalf("a stale mark is still a finding: %+v", x)
}
}
// Recomputing again changes nothing.
if said, err := w.k.RecomputeMarks(context.Background()); err != nil || len(said) != 0 {
t.Fatalf("a second recompute withdrew %v (%v)", said, err)
}
})
}
// Each of the 4 whole-set rules, failing on a record set made to break it.
func TestEachWholeSetRuleFails(t *testing.T) {
acc := func(n int, links ...Link) *Record {
+2 -1
View File
@@ -58,7 +58,8 @@
],
"tools": [
"knowledge_health",
"knowledge_highest"
"knowledge_highest",
"knowledge_marks"
],
"emits": [
"reserved",
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-vault
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026102838-174789941936
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026112527-c2a410bb6376
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
golang.org/x/crypto v0.57.0
+2
View File
@@ -2,6 +2,8 @@ git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021100209-cb54756c3865 h1:VLir9NZb
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021100209-cb54756c3865/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026102838-174789941936 h1:HGBoWimZXYZp+Qn9VCstYDitOP84t9EUQYd1yM9j2eg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026102838-174789941936/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026112527-c2a410bb6376 h1:by2zBASLb8h5jA1IiYJWykXwLnHYpuZh/zFNa4NsbOU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261026112527-c2a410bb6376/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
+4 -2
View File
@@ -246,7 +246,8 @@ func tools(m *Machine) []stdio.Tool {
},
{
Name: "pacman_upgrades",
Description: "Every full upgrade and rollback of every node, from the bus (ADR 0508 §1): each run's node, kind, what started it, " +
Description: "Upgrade history: when each node's last system upgrade or rollback ran and how it ended; one run whole, with its steps.\n" +
"Every full upgrade and rollback of every node, from the bus (ADR 0508 §1): each run's node, kind, what started it, " +
"its round, when it began and ended, its outcome (refused, finished, failed or running) and whether its after-upgrade check passed; " +
"narrowed to a node and to days, newest first. With a run, that run whole, every step with its time: the node's named, else this one's.",
Input: schema(map[string]any{
@@ -315,7 +316,8 @@ func tools(m *Machine) []stdio.Tool {
},
{
Name: "pacman_round",
Description: "Where the week's upgrade rounds stand (ADR 0508 §5, §6): each round's order and opening, and every node's state in it: " +
Description: "Did this week's system upgrade run on every node, and how did each end (the weekly upgrade round, last upgrade of all nodes).\n" +
"Where the week's upgrade rounds stand (ADR 0508 §5, §6): each round's order and opening, and every node's state in it: " +
"waiting (for the node named), turn, stalled (its turn came longer than round-start-minutes ago), running, passed, failed " +
"(the round stops there) or refused, with since when. With pass, a node and a reason, asks the operator on their channel to let " +
"the round go on past that node; it goes on only on the operator's Approve.",