Compare commits

..
Author SHA1 Message Date
jschoubben 8bc54a3e27 Merge remote-tracking branch 'origin/main' into fix/888-forge-merge-moves-source 2026-10-23 14:17:34 +02:00
jschoubben 15378cfba7 Merge pull request 'The build modules refuse every step and report unhealthy under a 20 GiB free-space floor, release workspaces, and sweep after 2 h (issues 1839 and 1894, ADR 0532)' (#458) from fix/1839-build-floor into main
Reviewed-on: #458
2026-10-23 11:29:04 +00:00
jschoubben 539a31103b mesh-build-image: let the module's own guard decide whether the real-buildah tests run, so a check sandbox where rootless buildah cannot unshare skips them loudly instead of failing every catalogue check (issue 1894)
mesh/delivery delivered
mesh/delivery-group group fix/1839-build-floor delivered: every member is delivered
mesh/check failed: g14: the check ran past its bound of 30m0s, stuck at: test the integration level of novox/mesh-catalog#458 on g14 | 539a31103b53 on…
2026-10-23 12:38:09 +02:00
jschoubben dfcdf234c5 Never move source.main back for a forge-heard merge, by dating the head it holds (issue 888, review round 1)
A merge the gate handed is known by the bridge's word before the forge's, and finish wrote it already; an older forge-only
merge heard in between overwrote it, since only the forge's word was dated and the write never compared with the head held.
A merge is now dated by either word, and the write leaves a head that is a later merge, or one the gate cannot date while
the merge is older than the records it keeps (a pruned newer merge).
2026-10-23 12:36:58 +02:00
jschoubben 5bc9578634 Move source.main for a merge heard only from the forge, so the checker judges on the trunk that is (issue 888)
The checker reads the trunk only from the gate's source.main, and the gate moved it only in finish, for a merge it handed to
the forge bridge on an Approve. A pull request merged at the forge directly moved every other delivery's trunk (trunkMoved)
and left source.main behind, so the checker kept judging candidates on the trunk before it (catalogue 89e15c7b after
6c0024e4, 2026-10-23).

settleForgeMerges now writes such a merge into source.main by compare-and-set, never a merge replayed as history or older
than one of its repository already heard; an approved delivery merged at the forge before the gate handed it is moved by
finish with its group, once.
2026-10-23 12:23:40 +02:00
jschoubben 3ca48ff230 Refuse build steps under a free-space floor and release workspaces, so the build modules stop filling a build node's disk (issue 1839)
mesh/delivery-group group fix/1839-build-floor checking: 1 of 2 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: novox: the check ran past its bound of 30m0s, stuck at: test the unit level of novox/mesh-catalog#458 on novox | 3ca48ff23005 on 89…
mesh-build-go, mesh-build-node and mesh-build-image refuse every fetch, test
and build, and say unhealthy, while their work or cache filesystem has less
than 20 GiB free (build-agent's floor), so the builder chooses another node
instead of filling this one. Each serves build_<x>_release, removing a
workspace once its caller's last step answered; a failed fetch removes its
own; a workspace never released goes two hours after its last use (ADR 0509)
in place of a day, never while a step holds it. Pins mesh-sdk's
fix/1839-build-floor (buildstep.Floor, buildstep.Workspaces).
2026-10-23 12:17:58 +02:00
jschoubben 6c0024e43b Merge pull request 'Make the catalogue's tests pass in the build sandbox, and give the integration level the installed Node toolchain read-only (issue 1891, ADR 0533)' (#459) from fix/logrotate-base-parses into main
Reviewed-on: #459
2026-10-23 09:48:22 +00:00
jschoubben a2e5a20f52 mesh-build-node: write the install's registry credential in the account's home, never beneath the opened cache, and refuse to open a toolchain holding one (issue 1891 review, ADR 0533 §2)
mesh/delivery failed: its walk failed, carrying its own commit: a gate on a first machine (what it carried put back), a build, a machine
mesh/check failed: integration: meshbuildnode/cmd/mesh-build-node.TestAFixtureBundleBuiltTwiceHasOneDigest: build_integration_test.go:71: no toolchain…
2026-10-23 11:45:20 +02:00
jschoubben 845479d7cd mesh-build-node: prove the npm download cache beside the toolchains stays its owner's (ADR 0533 §2) 2026-10-23 11:30:26 +02:00
jschoubben 2ebcb18097 realMachine: a change carrying a lock no release has installed runs with the toolchain installed last, and says so, so a lock change is not refused by its own check (issue 1891, ADR 0533 §3) 2026-10-23 11:23:11 +02:00
jschoubben 00e54d46d7 Give the integration level the node's installed Node toolchains read-only, so the Node build module's tests build offline (issue 1891, ADR 0533)
mesh-build-go binds /var/lib/mesh-build-node-cache/toolchain read-only into the
integration level's one shut-off process when it is there; the unit level never.
mesh-build-node opens its installed toolchains to other accounts (folders 0755,
files 0644, writable by its owner alone; its cache 0711, so the npm cache beside
them stays its own), and realMachine copies the toolchain of its own lock from
there instead of npm ci from the registry, which the level cannot reach.
2026-10-23 11:22:05 +02:00
jschoubben 933370df7c gitea, minio: compile the sources beside themselves in the test script, as the old check did (issue 1891)
The resolve hook needed the typescript package, which mesh-build-node keeps
out of the workspace (notCopied): tsc is reached only as the bin/tsc
command. So the test scripts now do what merge-check-typescript.sh did
before the tests: tsc over the module's sources with --outDir ., so the
sources' ./x.js imports resolve. The emitted .js is ignored by git.
2026-10-23 10:22:41 +02:00
jschoubben 437c524b81 gitea, minio, mosquitto: tests run as the Node build module runs them, tsc -p . then npm test (issue 1891)
The old TypeScript check compiled every source beside itself, and into
dist/, before it ran the tests; mesh-build-node runs `tsc -p .` (these
tsconfigs say noEmit) and then the package's test script, so:

- gitea and minio: the tests import the .ts sources, which import each
  other as ./x.js. A resolve hook in test/ resolves such an import, when no
  .js is there, to the .ts beside it and compiles it with the toolchain's
  TypeScript (gitea's sources use parameter properties, which type
  stripping refuses). 57 of gitea's tests and minio's links test failed.
- mosquitto: its ctrl test imports ../dist/client.js; its test script now
  builds first, as gitea's does.

Test scripts and a test helper only; what is built and delivered is the same.
2026-10-23 10:18:57 +02:00
jschoubben dc96d7c2dd gitea: take a rule's null contexts as the forge's type says, so the strict build the checker runs compiles (issue 1891)
The checker builds a package with a build script by its tsconfig (strict),
where requiredOf and TrunkForge's appliedProtection said
status_check_contexts is string[] and BranchProtection says string[] | null:
four TS2345 errors, hidden on main behind earlier failures. Types only; the
emitted code is the same, and a test holds that null contexts require nothing.
2026-10-23 10:00:09 +02:00
jschoubben f9247342d1 pacman, ssh-client: tests read nothing of the machine, so the unit level passes shut off and on a desk (issue 1891)
pacman-conf proved the shipped pacman.conf through its Include of
/etc/pacman.d/mirrorlist, which the shut-off unit level does not have: the
proven file now reads a mirror list of its own, and a guard test holds that
every repository of the shipped file includes the list reflector writes.

ssh-client's Test case for a run without an agent read the account's
SSH_AUTH_SOCK, so it failed on any desk with an agent: it now sets its own,
and also proves the environment's agent is the one reported.
2026-10-23 09:38:40 +02:00
jschoubben b1d875410a logrotate: dry-run the base against logs of its own, so the shut-off unit level does not stat the machine's /var/log (issue 1891)
The unit level runs in bubblewrap, where /var/log is not there; the base's
/var/log/wtmp block has no missingok, so logrotate -d failed its stat and
every mesh-catalog check that reached this test failed.
2026-10-23 09:10:09 +02:00
mesh-tools 89e15c7bcd Merge pull request 'R4e (mesh-catalog): build modules push and narrow, the gate stops judging, gitea writes mesh/check from the checker's owed state, and the census fixes' (#457) from feat/r4e into main 2026-10-22 22:12:51 +00:00
mesh-tools a21da33d48 Merge pull request 'Make buildah's runtime directory before every buildah run, so mesh-build-image's health does not fail for want of it (issue 1884)' (#456) from fix/1884-runtime-dir into main 2026-10-22 21:48:47 +00:00
jschoubben b53b0e7e2c Make buildah's runtime directory before every buildah run, so a machine without it is not unhealthy forever (issue 1884)
mesh/merge-gate head b53b0e7e2cc4 merged into trunk 48dd41db91a6 — pass: builds mesh-build-image → ace, shanks; no bus step; every machine composes with th…
mesh/repo-check head b53b0e7e2cc4 merged into trunk 48dd41db91a6 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: modules/gitea.test: ✔ the same merge, head or close announced twice has the same id, and another has another (7.92618ms) | b5…
The look made <store>/xdg only after `buildah --version`, which refuses first
when its XDG_RUNTIME_DIR is missing: the directory was never made on ace.
2026-10-22 23:39:51 +02:00
mesh-tools 48dd41db91 Merge pull request 'Fill a module again when a node-engine returns after any failed fill, not only a missing socket (issue 1873)' (#455) from fix/1873-fill-path into main 2026-10-22 21:19:09 +00:00
jschoubben 16331cdfdc Fill a module again when the node-engine's socket returns after any failed fill, not only one for want of the socket (issue 1873)
mesh/merge-gate head 16331cdfdce6 merged into trunk 66c404fd39a5 — pass: builds mesh-provisioner-client → ace, g14, novox, shanks; no bus step; every machi…
mesh/repo-check head 16331cdfdce6 merged into trunk 66c404fd39a5 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: mesh-build-go: no go build node qualifies: builders disagree: go1.27.1-X:nodwarf5 recipe 74744709d277ded9 on g14; go1.27.3-X:nodwar…
On ace postgres's fill of superuser.secret failed at 22:12 with the engine's own
"is a directory", what a container runtime had left at the path. The engine
delivered at 22:42 clears such a directory, but the client tried again only
modules whose fill had failed for want of the socket (issue 1165), so postgres
was never filled again and its old failure was said every 30 s. A newer engine
may write what the older could not: any failed fill is now tried again when a
node-engine makes its socket again.
2026-10-22 22:56:16 +02:00
jschoubben ab3722d3b3 Test that a fill the node-engine failed is tried again when its socket returns (issue 1873) 2026-10-22 22:53:39 +02:00
mesh-tools 66c404fd39 Merge pull request 'Let a run read from a node's pacman log never hold back a new upgrade, so the mesh can upgrade nodes at all (issue 1841)' (#454) from fix/1841-stale-run into main 2026-10-22 20:14:07 +00:00
jschoubben 3ad9d42be9 Name the captured log's machine by its role, not its name (issue 1841)
mesh/merge-gate head 3ad9d42be923 merged into trunk 083515da1be7 — pass: builds pacman → ace, g14, novox, shanks; no bus step; every machine composes with …
mesh/repo-check head 3ad9d42be923 merged into trunk 083515da1be7 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: i3/cmd/i3-tools.TestTheMainFileAndEveryModulesContributionLoadTogether: manifest_test.go:247: expected the launcher, clipboar…
2026-10-22 22:05:27 +02:00
mesh-tools 083515da1b Merge pull request 'Give /var/lib/systemd and /var/lib/postgres back to their packages and keep the two modules' own files under /var/lib/mesh (issue 1873, delivery A)' (#453) from fix/1873-system-state-dirs into main 2026-10-22 20:04:28 +00:00
jschoubben ac5d257a1c Let no run read from pacman's log, nor a run whose transaction ended, hold an asked upgrade back; read one run of the addressed machine whole (issue 1841)
Every run imported from the log ends at finished or failed and is never checked after, so the
one-run-at-a-time rule counted each as still going: novox's ask of 2026-10-22 22:00 was refused on
its install-time run 20220123T194900Z, and every node holds such runs. The rule now holds back a new
run only for a run the mesh ran whose transaction may still go on.

pacman_upgrades refused a run without its node, and the console takes node out of the arguments of
an address that names the machine, so no run could be read whole: a run without a node is now that
machine's.
2026-10-22 22:04:23 +02:00
jschoubben 896bd88380 Write mesh/check from the checker's owed state, read by a watch (R2-1)
mesh/repo-check head 896bd88380b3 merged into trunk a21da33d4857 beside mesh-controller#441@d51324df — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.s…
mesh/merge-gate head 896bd88380b3 merged into trunk a21da33d4857 — pass: group feat/r4e composed: every machine composes with the change as it did without …
mesh/delivery covered after its stop: a later walk that contains it was delivered: plan-1792707174627079192 (novox/mesh-catalog at 89e15c7b into main, do…
mesh/delivery-group group feat/r4e delivered: every member is delivered
mesh/check failed: ace: novox/mesh-catalog#457: modules/gitea/go.mod names mesh-sdk 104533e8c0c9, which is neither on mesh-sdk's trunk nor the group's…
R4e round 2 finding R2-1, ADR 0521: the delivery plan draws no edge that
orders the forge module before the forge bridge, and the forge module's
durable consumer keeps its deliver policy when its filters change, so a
check.status the bridge emitted before the forge module consumed it was lost.
An in-memory revision guard also forgot itself on a restart.

The forge module now reads mesh-delivery-checker.owed, which the checker
announces readable, by a watch: the watch hands over the latest value of every
key whenever the module starts, so a status written while it was away is
written then, and only the latest. Each mesh/check is written once per
revision, an older one dropped; a forge refusal is asked again on a bound
until written or overtaken; a refused read is said once and asked again until
granted. No tool writes mesh/check, and no check.status is consumed.
2026-10-22 21:45:53 +02:00
jschoubben 24f50c7c86 mesh-build-go: name go.mod among the narrowing test's paths, since mesh-sdk 104533e8 keeps only the manifest and the lock files beside them (review finding S1) 2026-10-22 21:39:55 +02:00
jschoubben e0bf87c2e3 Give /var/lib/systemd and /var/lib/postgres back to their packages, and keep the modules' files in their own directories (issue 1873)
mesh/repo-check head e0bf87c2e3da merged into trunk ef1e98f8678c — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/merge-gate head e0bf87c2e3da merged into trunk ef1e98f8678c — pass: group fix/1873-system-state-dirs composed: every machine composes with the change …
mesh/delivery delivered
mesh/delivery-group group fix/1873-system-state-dirs delivered: every member is delivered
mesh/check failed: unit: i3/cmd/i3-tools.TestTheMainFileAndEveryModulesContributionLoadTogether: manifest_test.go:247: expected the launcher, clipboar…
The systemd and postgres modules' state directories said place ".", which is
/var/lib/systemd and /var/lib/postgres: directories the systemd and postgresql
packages own (root 0755; postgres 0700). The node-engine took both before issue
339's rule and gave them to the operator's account, 0700, on every machine; postgres's
grants directory, placed by default, lay inside /var/lib/postgres too.

What each module keeps moves to a directory of its own, "data", placed "mesh"
(/var/lib/mesh/systemd, /var/lib/mesh/postgres): the unlink record, and postgres's
own secrets (the same values: the vault keys them by name), consumers.json and
grants. The store's data stays /var/lib/mesh-store, untouched.

The node-engine keeps no found owner or mode for a directory, and forgets one no
longer declared as it stands; it sets the declared owner and mode on a directory it
recorded at the same path under the same id. So for this delivery "state" names
the package's directory with the package's owner and mode, and the engine gives it
back; the next change drops "state". The postgres container is recreated, its
secret's volume moving.

checks/owned/system-known keeps systemd state and postgres state, declared for
this delivery only; postgres's consumers and grants leave the list.
2026-10-22 21:31:58 +02:00
mesh-tools ef1e98f867 Merge pull request 'Keep the pacman keeper's state in a directory of its own, not in the package database's /var/lib/pacman (issue 1841)' (#452) from fix/1841-keeper-state-dir into main 2026-10-22 19:31:24 +00:00
jschoubben e3db481491 Pin every R4e member module to mesh-sdk's final R4e head 104533e8, which holds review fixes S1 and S2, so the group pins one SDK 2026-10-22 21:29:19 +02:00
jschoubben 21b4925892 Merge remote-tracking branch 'origin/fix/r4e-k1' into r4e-int2 2026-10-22 21:28:24 +02:00
jschoubben 098c6d3ec7 Place pacman's state outside the package manager's own directory, and refuse a module directory in a system package's (issue 1841)
mesh/merge-gate head 098c6d3ec70f merged into trunk 2f05e70734cd — pass: builds pacman → ace, g14, novox, shanks; no bus step; every machine composes with …
mesh/repo-check head 098c6d3ec70f merged into trunk 2f05e70734cd — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery delivered
mesh/check failed: unit: clipmenu/cmd/clipmenu-tools.TestTheLauncherWaitsQuietlyWhileAnotherClipmenudHoldsTheLock: manifest_test.go:199: the launcher …
pacman's state directory said place ".", which resolves to /var/lib/pacman: the
package manager's database directory. The node-engine used it as found and
reported it unhealthy on every machine, and wrote keeper.json into it. It is now
the mesh's directory for the module (place "mesh", /var/lib/mesh/pacman), as
docker's is; keeper.json, the keeper's bus secret and the root helper's socket
follow through ${dir:state}.

checks/owned rule 5 refuses a directory that is, lies in or holds a /var/lib
directory a system package owns (checks/owned/system, read from shanks through
pacman_owns and tmpfiles.d), and any resource inside one. systemd and postgres
collide today and are listed in checks/owned/system-known under issue 1873; that
list only shrinks.
2026-10-22 21:19:15 +02:00
jschoubben 8d3f81c2c0 Write mesh/check from the forge bridge's own event, never from a tool (K1)
Review finding K1 of R4e, issue 1301 S4 (ADR 0503 §3, ADR 0513 §4: the forge
requires mesh/check alone). gitea_check_status assumed only the bridge could
call it, but a tool is told no caller and node-tools invokes every tool, so any
session could set mesh/check to success and merge past the checker: the fault
issue 293 closed for the two statuses before it.

gitea now consumes mesh-delivery-forge-bridge.check.status, a subject the bus
grants to the bridge's own account alone, and check.ts writes mesh/check from
it: the source is checked, the head's whole sha and the state, and a revision
older than one already written on the head is dropped so a redelivered stale
pending never hides a verdict; a forge refusal throws, so the event is
delivered again. gitea_check_status is removed, and gitea_commit_status keeps
refusing mesh/check.
2026-10-22 21:14:01 +02:00
88 changed files with 2823 additions and 248 deletions
+181 -13
View File
@@ -11,13 +11,23 @@
// another module, after ${dir:<id>} is resolved within each module and both homes are read as one root.
// Archives are not containers for this rule. This holds for ~/.ssh what checks/sshdir held (ADR 0415 §6).
//
// go run . <catalogue root>
// 5. a directory that is, lies inside or holds a directory a system package owns under /var/lib, or any other
// resource inside one (issue 1841: pacman's state directory, placed with "place": ".", was /var/lib/pacman, the
// package manager's own database directory). A placed directory is resolved as the mesh resolves it with the
// default data root: "." is /var/lib/<module>, "mesh" /var/lib/mesh/<module>, neither /var/lib/<module>/<id>.
// The system directories are checks/owned/system, captured from a machine (pacman -Qo, /usr/lib/tmpfiles.d);
// checks/owned/system-known names the exceptions still to fix, by module, resource and issue. That list only
// shrinks: an entry that no longer collides is refused, and with -base one the trunk's copy does not hold.
//
// go run . [-base <trunk's system-known>] <catalogue root>
//
// Exits 1 on a finding.
package main
import (
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"path"
@@ -27,6 +37,12 @@ import (
"strings"
)
// The lists of rule 5, relative to the catalogue's root.
const (
SystemList = "checks/owned/system"
SystemKnown = "checks/owned/system-known"
)
// Home is the one root both machine homes are read as.
const Home = "${home}"
@@ -43,9 +59,9 @@ type Finding struct {
}
type resource struct {
ID, Type, Path, Unknown string
Knows []string
HasKnows bool
ID, Type, Path, Unknown, Place string
Knows []string
HasKnows bool
}
type parsed struct {
@@ -65,6 +81,7 @@ func load(file string) (*parsed, error) {
ID string `json:"id"`
Type string `json:"type"`
Path string `json:"path"`
Place string `json:"place"`
Unknown *string `json:"unknown"`
Knows []string `json:"knows"`
} `json:"resources"`
@@ -81,7 +98,7 @@ func load(file string) (*parsed, error) {
out.Module = filepath.Base(filepath.Dir(file))
}
for _, r := range m.Resources {
x := resource{ID: r.ID, Type: r.Type, Path: r.Path, Knows: r.Knows, HasKnows: r.Knows != nil}
x := resource{ID: r.ID, Type: r.Type, Path: r.Path, Place: r.Place, Knows: r.Knows, HasKnows: r.Knows != nil}
if r.Unknown != nil {
x.Unknown = *r.Unknown
if x.Unknown == "" {
@@ -143,14 +160,23 @@ type declared struct {
}
// Check reads every manifest under root/modules and answers what it refuses.
func Check(root string) ([]Finding, error) {
func Check(root string, base []string) ([]Finding, error) {
files, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
sort.Strings(files)
system, err := readSystem(filepath.Join(root, SystemList))
if err != nil {
return nil, err
}
known, err := readLines(filepath.Join(root, SystemKnown))
if err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, err
}
var out []Finding
var paths []declared
collided := map[string]bool{}
type container struct{ module, path string }
var containers []container
for _, file := range files {
@@ -170,6 +196,7 @@ func Check(root string) ([]Finding, error) {
dirs[r.ID] = "/placed/" + m.Module + "/" + r.ID
}
}
out = append(out, inSystem(m, system, known, collided)...)
for _, r := range m.Resources {
p := norm(r.Path, dirs)
if p != "" {
@@ -212,6 +239,28 @@ func Check(root string) ([]Finding, error) {
}
}
}
for _, k := range known {
if !collided[k] {
module, id, _ := strings.Cut(k, " ")
id, _, _ = strings.Cut(id, " ")
out = append(out, Finding{5, module, id, fmt.Sprintf("is listed in %s, and no longer lies in a system "+
"package's directory: take the line out, so the list only shrinks", SystemKnown)})
}
}
if base != nil {
held := map[string]bool{}
for _, b := range base {
held[b] = true
}
for _, k := range known {
if !held[k] {
module, id, _ := strings.Cut(k, " ")
id, _, _ = strings.Cut(id, " ")
out = append(out, Finding{5, module, id, fmt.Sprintf("is a new line in %s, which the trunk's copy "+
"does not hold: a module's directory is placed outside a system package's, never listed", SystemKnown)})
}
}
}
for _, d := range paths {
for _, c := range containers {
if c.module != d.module && (d.path == c.path || strings.HasPrefix(d.path, c.path+"/")) {
@@ -222,21 +271,140 @@ func Check(root string) ([]Finding, error) {
return out, nil
}
func main() {
root := "."
if len(os.Args) > 1 {
root = os.Args[1]
// systemDir is one directory a system package owns, and what says so.
type systemDir struct{ Path, Source string }
// readLines is a list's lines, without comments and blank lines.
func readLines(file string) ([]string, error) {
raw, err := os.ReadFile(file)
if err != nil {
return nil, err
}
found, err := Check(root)
var lines []string
for _, line := range strings.Split(string(raw), "\n") {
if line = strings.TrimSpace(line); line != "" && !strings.HasPrefix(line, "#") {
lines = append(lines, line)
}
}
return lines, nil
}
// readSystem is the list of system directories: "<absolute path> <package>: <how it was read>" per line. Absent or
// empty, the check fails: a list that says nothing would let every module through.
func readSystem(file string) ([]systemDir, error) {
lines, err := readLines(file)
if err != nil {
return nil, fmt.Errorf("rule 5 reads the system packages' directories from %s: %w", file, err)
}
if len(lines) == 0 {
return nil, fmt.Errorf("%s names no system package's directory, so rule 5 would pass everything", file)
}
var out []systemDir
for _, line := range lines {
p, source, _ := strings.Cut(line, " ")
if !strings.HasPrefix(p, "/var/lib/") || strings.TrimSpace(source) == "" {
return nil, fmt.Errorf("%s: %q is not \"/var/lib/<directory> <package>: <how it was read>\"", file, line)
}
out = append(out, systemDir{path.Clean(p), strings.TrimSpace(source)})
}
return out, nil
}
// placedAt is where the mesh places a pathless directory with the default data root (mesh-sdk catalogue dirsFor).
func placedAt(module string, r resource) string {
switch r.Place {
case ".":
return "/var/lib/" + module
case "mesh":
return "/var/lib/mesh/" + module
}
return "/var/lib/" + module + "/" + r.ID
}
// inSystem is rule 5 for one module: each resource whose resolved path collides with a system package's directory,
// unless system-known lists it (then it is marked collided, so a stale line is found).
func inSystem(m *parsed, system []systemDir, known []string, collided map[string]bool) []Finding {
real := map[string]string{}
for _, r := range m.Resources {
if r.Type == "directory" && r.Path == "" && r.ID != "" {
real[r.ID] = placedAt(m.Module, r)
}
}
for _, r := range m.Resources {
if r.ID != "" && r.Path != "" {
real[r.ID] = r.Path
}
}
var out []Finding
for _, r := range m.Resources {
p := ""
if r.Path != "" {
p = norm(r.Path, real)
} else if r.Type == "directory" {
p = real[r.ID]
}
if !strings.HasPrefix(p, "/") {
continue
}
for _, s := range system {
inside := p == s.Path || strings.HasPrefix(p, s.Path+"/")
holds := r.Type == "directory" && strings.HasPrefix(s.Path, p+"/")
if !inside && !holds {
continue
}
listed := ""
for _, k := range known {
if strings.HasPrefix(k, m.Module+" "+r.ID+" ") || k == m.Module+" "+r.ID {
listed = k
}
}
if listed != "" {
collided[listed] = true
break
}
how := "is or lies inside"
if holds {
how = "holds"
}
out = append(out, Finding{5, m.Module, r.ID, fmt.Sprintf("resolves to %q, which %s %s, the directory of the "+
"system package %s; place it under /var/lib/mesh/%s (\"place\": \"mesh\") or a path of its own", p, how,
s.Path, s.Source, m.Module)})
break
}
}
return out
}
func main() {
baseFile := flag.String("base", "", "the trunk's copy of "+SystemKnown+": a line it does not hold is refused")
flag.Parse()
root := "."
if flag.NArg() > 0 {
root = flag.Arg(0)
}
var base []string
if *baseFile != "" {
lines, err := readLines(*baseFile)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
base = append([]string{}, lines...)
}
found, err := Check(root, base)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
for _, f := range found {
fmt.Printf("%s: %s: %s (rule %d, novox/hq ADR 0422 §8, issue 1024)\n", f.Module, f.ID, f.Why, f.Rule)
cite := "novox/hq ADR 0422 §8, issue 1024"
if f.Rule == 5 {
cite = "issue 1841"
}
fmt.Printf("%s: %s: %s (rule %d, %s)\n", f.Module, f.ID, f.Why, f.Rule, cite)
}
if len(found) > 0 {
os.Exit(1)
}
fmt.Println("owned paths: every directory and archive says how it treats unknown files; no module declares inside another's directory")
fmt.Println("owned paths: every directory and archive says how it treats unknown files; no module declares inside another's directory, or in a system package's")
}
+102 -1
View File
@@ -20,7 +20,10 @@ func manifest(t *testing.T, root, module, body string) {
func check(t *testing.T, root string) []Finding {
t.Helper()
found, err := Check(root)
if _, err := os.Stat(filepath.Join(root, SystemList)); os.IsNotExist(err) {
lists(t, root, "/var/lib/pacman pacman: pacman -Qo\n", "")
}
found, err := Check(root, nil)
if err != nil {
t.Fatal(err)
}
@@ -35,6 +38,20 @@ func only(t *testing.T, found []Finding, rule int, module, id string) {
}
}
// lists writes the catalogue's list of system directories and its known exceptions.
func lists(t *testing.T, root, system, known string) {
t.Helper()
if err := os.MkdirAll(filepath.Join(root, "checks", "owned"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, SystemList), []byte(system), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, SystemKnown), []byte(known), 0o644); err != nil {
t.Fatal(err)
}
}
func dir(id, path, rest string) string {
return `{"id":"` + id + `","type":"directory","path":"` + path + `"` + rest + `}`
}
@@ -145,3 +162,87 @@ func TestPlacedDirectoriesOfTwoModulesAreTheirOwn(t *testing.T) {
t.Fatalf("refused %+v", found)
}
}
// 1841/AC18: a module's directory that resolves to a /var/lib directory a system package owns, lies inside one or
// holds one, is refused, whether placed (place ".", or by default beneath the module's own) or stated; a placed
// directory saying place "mesh" lies under /var/lib/mesh and passes (issue 1841: pacman's state was /var/lib/pacman).
func TestADirectoryOfASystemPackageIsRefused(t *testing.T) {
system := "/var/lib/pacman pacman: pacman -Qo\n/var/lib/systemd/coredump systemd: tmpfiles.d/systemd.conf\n"
for name, resources := range map[string][]string{
"placed-own": {`{"id":"state","type":"directory","place":".","unknown":"program"}`},
"placed-beneath": {`{"id":"coredump","type":"directory","unknown":"program"}`},
"stated-inside": {dir("d", "/var/lib/pacman/keeper", `,"unknown":"program"`)},
"stated-holding": {dir("d", "/var/lib/systemd", `,"unknown":"program"`)},
"file-inside": {`{"id":"state","type":"directory","place":"mesh","unknown":"program"}`,
`{"id":"f","type":"file","path":"/var/lib/pacman/keeper.json"}`},
} {
module := map[string]string{"placed-own": "pacman", "placed-beneath": "systemd"}[name]
if module == "" {
module = "keeper"
}
root := t.TempDir()
lists(t, root, system, "")
manifest(t, root, module, mod(module, resources...))
found := check(t, root)
if len(found) != 1 || found[0].Rule != 5 || found[0].Module != module ||
!strings.Contains(found[0].Why, "/var/lib/") || !strings.Contains(found[0].Why, "package") {
t.Errorf("%s: found %+v; want one finding of rule 5 for %s naming the path and the package", name, found, module)
}
}
root := t.TempDir()
lists(t, root, system, "")
manifest(t, root, "pacman", mod("pacman",
`{"id":"state","type":"directory","place":"mesh","unknown":"program"}`,
`{"id":"f","type":"file","path":"${dir:state}/keeper.json"}`,
dir("other", "/var/lib/pacman-other", `,"unknown":"program"`)))
if found := check(t, root); len(found) != 0 {
t.Fatalf("a directory under /var/lib/mesh/pacman was refused: %+v", found)
}
}
// 1841/AC18: a known exception passes; one that no longer collides is refused, so the list only shrinks; one the trunk's
// copy does not hold is refused; and an absent list of system directories fails the check rather than passing it.
func TestKnownSystemExceptionsOnlyShrink(t *testing.T) {
system := "/var/lib/systemd systemd: tmpfiles.d/systemd.conf\n"
known := "# comment\nsystemd state issue 1873\n"
root := t.TempDir()
lists(t, root, system, known)
manifest(t, root, "systemd", mod("systemd", `{"id":"state","type":"directory","place":".","unknown":"program"}`))
if found := check(t, root); len(found) != 0 {
t.Fatalf("a known exception was refused: %+v", found)
}
if found, err := Check(root, []string{"systemd state issue 1873"}); err != nil || len(found) != 0 {
t.Fatalf("a known exception held by the trunk was refused: %+v %v", found, err)
}
if found, err := Check(root, []string{}); err != nil || len(found) != 1 || found[0].Rule != 5 {
t.Fatalf("an exception the trunk does not hold passed: %+v %v", found, err)
}
manifest(t, root, "systemd", mod("systemd", `{"id":"state","type":"directory","place":"mesh","unknown":"program"}`))
only(t, check(t, root), 5, "systemd", "state")
root = t.TempDir()
manifest(t, root, "a", mod("a"))
if err := os.MkdirAll(filepath.Join(root, "checks", "owned"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, SystemList), []byte("# nothing\n"), 0o644); err != nil {
t.Fatal(err)
}
if _, err := Check(root, nil); err == nil {
t.Fatal("an empty list of system directories passed")
}
}
// 1841/AC18: the catalogue as it stands passes rule 5 with its own lists.
func TestTheCatalogueKeepsOutOfSystemDirectories(t *testing.T) {
found, err := Check("../..", nil)
if err != nil {
t.Fatal(err)
}
for _, f := range found {
if f.Rule == 5 {
t.Errorf("%s: %s: %s", f.Module, f.ID, f.Why)
}
}
}
+23
View File
@@ -0,0 +1,23 @@
# The directories under /var/lib that a system package owns (rule 5 of checks/owned, issue 1841). A module's
# directory, placed or stated, may not be one of these, lie inside one or hold one: the package's program and the
# package manager keep their own files there, and the node-engine would judge, chmod or chown the directory.
#
# One line per directory: "<path> <package>: <how it was read>". Captured on the workstation on 2026-10-22, through
# the mesh: pacman's own database by `pacman -Qo` (pacman_owns), the rest from the `d`, `D` and `Q` lines of
# /usr/lib/tmpfiles.d (node-reads.search for "/var/lib/"). A directory found to belong to a package is added here.
/var/lib/pacman pacman: pacman -Qo /var/lib/pacman/ (pacman 7.1.0)
/var/lib/alsa alsa-utils: tmpfiles.d/alsa-utils.conf
/var/lib/colord colord: tmpfiles.d/colord.conf
/var/lib/containers podman: tmpfiles.d/podman.conf
/var/lib/dbus dbus: tmpfiles.d/dbus.conf
/var/lib/geoclue geoclue: tmpfiles.d/geoclue.conf
/var/lib/lastlog util-linux: tmpfiles.d/lastlog2.conf
/var/lib/libuuid util-linux: tmpfiles.d/uuidd.conf
/var/lib/machines systemd: tmpfiles.d/systemd-nspawn.conf
/var/lib/openvpn/chroot networkmanager-openvpn: tmpfiles.d/nm-openvpn-tmpfiles.conf
/var/lib/portables systemd: tmpfiles.d/portables.conf
/var/lib/postgres postgresql: tmpfiles.d/postgresql.conf
/var/lib/private systemd: tmpfiles.d/systemd.conf
/var/lib/rpcbind rpcbind: tmpfiles.d/rpcbind.conf
/var/lib/systemd systemd: tmpfiles.d/systemd.conf
/var/lib/tpm2-tss/system/keystore tpm2-tss: tmpfiles.d/tpm2-tss-fapi.conf
+5
View File
@@ -0,0 +1,5 @@
# Modules whose directory lies in a system package's directory today (rule 5 of checks/owned, issue 1841), each with
# the issue that moves it out: "<module> <resource id> issue <number>". This list only shrinks: a line whose module
# no longer collides is refused, and so is a line the trunk's copy does not hold.
postgres state issue 1873
systemd state issue 1873
+15 -3
View File
@@ -48,7 +48,8 @@
# 10. the mail-store seat (novox/hq ADR 0378 §3, issue 800): no manifest but modules/mesh-mail claims it.
# 11. owned paths (novox/hq ADR 0422 §8, issue 1024; it holds ~/.ssh for ssh-client as ADR 0415 §6 asked):
# every directory and archive says `unknown` (move-out, module or program) and a valid `knows`; `move-out`
# only under a home; and no module declares a path inside a directory of another module.
# only under a home; no module declares a path inside a directory of another module; and none is, lies in or
# holds a directory a system package owns under /var/lib (checks/owned/system, issue 1841).
# checks/sshdir (ADR 0415 §6) stays while main runs it; it goes under issue 1035 once main runs checks/owned.
# 12. the image builder (novox/hq ADR 0487 §3, issue 1704): only modules/buildah provides rootless-image-build.
# 13. the image's place (novox/hq ADR 0487 §1, issue 1707): every image builds from a Dockerfile in its own module's directory.
@@ -73,8 +74,19 @@ set -eu
# ~/.ssh (novox/hq ADR 0415 §6): only ssh-client may name a path under it. Main's merge check runs this line, so
# checks/sshdir stays until main runs checks/owned; it goes under issue 1035.
(cd checks/sshdir && go test -count=1 . && go run . ../..)
# Owned paths (novox/hq ADR 0422 §8): unknown files named per directory; no path inside another module's directory.
(cd checks/owned && go test -count=1 . && go run . ../..)
# Owned paths (novox/hq ADR 0422 §8): unknown files named per directory; no path inside another module's directory;
# none in a system package's directory under /var/lib (issue 1841), the known exceptions only shrinking: judged
# against the trunk's copy of checks/owned/system-known at the merge base; a trunk without one is the list's first copy.
owned_base=""
if merge_base=$(git merge-base HEAD "origin/${MESH_CHECK_BASE:-main}" 2>/dev/null); then
if git cat-file -e "$merge_base:checks/owned/system-known" 2>/dev/null; then
owned_base=$(mktemp)
git show "$merge_base:checks/owned/system-known" >"$owned_base"
fi
else
echo "NOT COMPARED WITH THE TRUNK: no merge base with origin/${MESH_CHECK_BASE:-main}; checks/owned/system-known is held to the list it was first written with"
fi
(cd checks/owned && go test -count=1 . && if [ -n "$owned_base" ]; then go run . -base "$owned_base" ../..; else go run . ../..; fi)
# The secrets (novox/hq issue 342): a password is not a setting, so no setting can replace the vault's. The
# known list only shrinks: judged against the trunk's copy at the merge base, a line it does not hold is refused.
secrets_base=""
+2
View File
@@ -0,0 +1,2 @@
# The test script compiles the sources beside themselves before the tests import them (package.json, issue 1891).
*.js
+188
View File
@@ -0,0 +1,188 @@
// **mesh/check is written from the checker's owed state, read by a watch** (ADR 0521; R4e round 2 finding R2-1; finding K1
// before it; novox/hq issue 1301; ADR 0447 §1, ADR 0503 §3: main requires mesh/check alone).
//
// No tool writes mesh/check: a tool is told no caller, and the console every agent session speaks through (node-tools)
// invokes every tool, so a tool that wrote it could be called by any session to merge past the checker (the fault issue
// 293 closed for the two statuses before it). Nor does an event: nothing in the delivery plan orders this module before the
// forge bridge, and a durable consumer whose filters change keeps its deliver policy, so an event said before this module
// consumed it was lost. The checker keeps what it owes the forge in its state `owed` (ADR 0484 §2), which it announces
// readable (ADR 0495 §2-§3), and only the checker writes it. This module watches it: the watch hands over the latest value of
// every key when it starts, whenever that is, and then every change. Each key whose act is the status mesh/check is
// written on its head, once per revision; a revision older than one already written on the key is dropped.
//
// A read refused (the grant not yet composed) is said once and asked again on a bound until it holds. A forge that refuses
// a status is asked again on a bound, until it writes or a newer revision of the key comes.
import type { CommitStatus } from "./client.js";
import { bindUntilHeld, bindWait } from "./bind.js";
import { checkName } from "./guard.js";
import { CHECK_STATUS } from "./pulls.js";
/** The checker's state this module reads, and the act of its keys this module writes. */
export const CHECKER_OWED = "mesh-delivery-checker.owed";
export const CHECK_ACT = `status=${CHECK_STATUS}`;
const states = new Set(["pending", "success", "error", "failure", "warning"]);
/** What writing the check needs of the forge. */
export interface CheckForge {
setCommitStatus(owner: string, repo: string, sha: string, status: CommitStatus): Promise<void>;
}
/** One change of the checker's owed, as the SDK's state watch hands it. */
export interface OwedChange {
key: string;
op: "put" | "delete";
value?: unknown;
revision: number;
current?: boolean;
}
/** A key part as mesh-sdk go/delivery EscapeName writes it, read back: `=` and two lowercase hex digits per byte. */
function unescapeName(part: string): string {
if (!/^([-/_a-zA-Z0-9]|=[0-9a-f]{2})+$/.test(part)) throw new Error(`${JSON.stringify(part)} is not an escaped name`);
return Buffer.from(part.replace(/=([0-9a-f]{2})/g, (_, h) => String.fromCharCode(parseInt(h, 16))), "latin1").toString("utf8");
}
/** The pull request and head of the checker's owed key (mesh-sdk go/delivery CheckerOwedKey):
* `<owner>/<name>.<number>.<head>.status=mesh/check`, owner and name escaped. */
export function owedKeyOf(key: string): { owner: string; repo: string; number: number; sha: string } {
const parts = String(key ?? "").split(".");
if (parts.length !== 4) throw new Error(`${JSON.stringify(key)} is not an owed key, <repository>.<number>.<head>.<act>`);
const [repository, number, sha, act] = parts;
if (act !== CHECK_ACT) throw new Error(`${JSON.stringify(key)} owes ${JSON.stringify(act)}, not ${CHECK_ACT}`);
const m = /^([^/]+)\/([^/]+)$/.exec(repository);
if (!m) throw new Error(`${JSON.stringify(key)} names no <owner>/<name>`);
if (!/^(0|[1-9][0-9]*)$/.test(number)) throw new Error(`${JSON.stringify(key)} names no pull request number`);
if (!/^([0-9a-f]{40}|[0-9a-f]{64})$/i.test(sha)) throw new Error(`${JSON.stringify(key)} names no head's whole sha`);
return { owner: checkName("owner", unescapeName(m[1])), repo: checkName("name", unescapeName(m[2])), number: Number(number), sha };
}
/** The status an owed change asks for, checked as mesh-sdk go/delivery DecodeCheckerOwed checks it; or why it is not one. */
export function checkStatusOf(change: OwedChange): { owner: string; repo: string; sha: string; revision: number; status: CommitStatus } {
if (change.op !== "put") throw new Error(`a ${change.op} owes nothing`);
const { owner, repo, sha } = owedKeyOf(change.key);
const revision = change.revision;
if (typeof revision !== "number" || !Number.isSafeInteger(revision) || revision < 0) {
throw new Error(`${JSON.stringify(revision)} is not a revision`);
}
const v = change.value;
if (!v || typeof v !== "object") throw new Error("the value is no owed act");
const o = v as Record<string, unknown>;
if (o.v !== 1) throw new Error(`the owed act's version ${JSON.stringify(o.v)} is not one this module reads`);
if (o.act !== "status" || o.context !== CHECK_STATUS) {
throw new Error(`the owed act is ${JSON.stringify(o.act)} ${JSON.stringify(o.context)}, not the status ${CHECK_STATUS}`);
}
const state = String(o.state ?? "");
if (!states.has(state)) throw new Error(`${JSON.stringify(o.state)} is not a status the forge keeps`);
let description = String(o.text ?? "").replace(/\s+/g, " ").trim();
if (description.length > 140) description = description.slice(0, 139) + "…";
const target = typeof o.target === "string" && /^https?:\/\//.test(o.target) ? o.target : undefined;
const status: CommitStatus = { state: state as CommitStatus["state"], context: CHECK_STATUS, description,
...(target ? { target_url: target } : {}) };
return { owner, repo, sha, revision, status };
}
/** The one writer of mesh/check: each owed revision of the checker's, in revision order per key. */
export class CheckStatuses {
/** The newest revision written of each key, and the newest seen; the oldest keys go past what any forge holds open. */
private written = new Map<string, number>();
private seen = new Map<string, number>();
/** The turn each key's writes wait on, and the asks again still running. */
private turns = new Map<string, Promise<unknown>>();
private again = new Set<Promise<void>>();
private forge: CheckForge;
private say: (line: string) => void;
private sleep: (ms: number) => Promise<void>;
constructor(forge: CheckForge, say: (line: string) => void = (l) => console.error(l),
sleep: (ms: number) => Promise<void> = (ms) => new Promise((r) => setTimeout(r, ms))) {
this.forge = forge;
this.say = say;
this.sleep = sleep;
}
/** Write the change's status: "written", or "dropped" for one that owes no mesh/check or is older than one written. A
* forge that refuses throws. */
async apply(change: OwedChange): Promise<"written" | "dropped"> {
let asked;
try {
asked = checkStatusOf(change);
} catch (err) {
this.say(`[gitea] ${CHECKER_OWED} ${change?.key} at revision ${change?.revision} writes nothing: ${err instanceof Error ? err.message : err}`);
return "dropped";
}
const key = change.key;
const seen = this.seen.get(key);
if (seen === undefined || asked.revision > seen) this.remember(this.seen, key, asked.revision);
// One key's writes take turns, so two at once are weighed one after the other.
const before = this.turns.get(key) ?? Promise.resolve();
const turn = before.then(async (): Promise<"written" | "dropped"> => {
const newest = this.written.get(key);
if (newest !== undefined && asked.revision < newest) {
this.say(`[gitea] ${asked.owner}/${asked.repo}@${asked.sha.slice(0, 12)}: mesh/check at revision ${asked.revision} is older than ${newest}, already written, and is dropped`);
return "dropped";
}
await this.forge.setCommitStatus(asked.owner, asked.repo, asked.sha, asked.status);
this.remember(this.written, key, asked.revision);
return "written";
});
const settled = turn.catch(() => {});
this.turns.set(key, settled);
void settled.then(() => { if (this.turns.get(key) === settled) this.turns.delete(key); });
return turn;
}
/** What the watch hands over: written, or when the forge refuses, asked again on a bound in the background until it is
* written or a newer revision of the key was seen. Never throws: a watch's handler that throws loses the change. */
async take(change: OwedChange): Promise<void> {
try {
await this.apply(change);
} catch (err) {
const run = this.askAgain(change, err);
this.again.add(run);
void run.finally(() => this.again.delete(run));
}
}
/** Resolves when no ask again is running. */
async settled(): Promise<void> {
while (this.again.size) await Promise.all([...this.again]);
}
private async askAgain(change: OwedChange, first: unknown): Promise<void> {
let err = first;
for (let attempt = 1; ; attempt++) {
const wait = bindWait(attempt);
this.say(`[gitea] ${change.key}: mesh/check at revision ${change.revision} could not be written (${err instanceof Error ? err.message : err}); asked again in ${wait / 1000}s`);
await this.sleep(wait);
const newest = this.seen.get(change.key);
if (newest !== undefined && newest > change.revision) return;
try {
await this.apply(change);
return;
} catch (e) {
err = e;
}
}
}
private remember(map: Map<string, number>, key: string, revision: number): void {
map.delete(key);
map.set(key, revision);
if (map.size > 4000) map.delete(map.keys().next().value as string);
}
}
/**
* Watch the checker's owed with `watch` (the SDK's `state(CHECKER_OWED).watch`), handing each change to `checks`. A watch
* refused — the read not granted yet — is said once and asked again on a bound until it holds (bind.ts). Never rejects.
*/
export async function watchOwed(watch: (handler: (change: OwedChange) => Promise<void>) => Promise<void>, checks: CheckStatuses,
sleep?: (ms: number) => Promise<void>, say: (line: string) => void = (l) => console.error(l)): Promise<void> {
let refusals = 0;
await bindUntilHeld(CHECKER_OWED, () => watch((change) => checks.take(change)), sleep, (line) => {
if (/could not be bound/.test(line) && refusals++ > 0) return;
say(line);
});
}
+4 -49
View File
@@ -101,10 +101,10 @@ export function deliveryStatus(context: string, state: string, description: stri
if (!/^mesh\/[a-z-]+$/.test(context)) throw new Error(`${context} is not a status of the mesh's own`);
if (!states.has(state)) throw new Error(`${state} is not a status the forge keeps`);
// mesh/check is the checker's verdict and the one status main requires (ADR 0447 §1): no hand sets it in any
// state, as issue 293 closed for the two before it. The forge bridge writes it through gitea_check_status, a tool
// only its account may call (issue 1301).
// state, as issue 293 closed for the two before it. No tool writes it, since a tool is told no caller (R4e finding K1):
// check.ts writes it from the checker's owed state (ADR 0521).
if (context === CHECK_STATUS) {
throw new Error(`${CHECK_STATUS} is the checker's status: the forge bridge writes it through gitea_check_status, never this tool`);
throw new Error(`${CHECK_STATUS} is the checker's status: it is written from mesh-delivery-checker.owed alone, never by a tool`);
}
// The merge check's statuses are its verdict's, set when the controller says it, and a branch may require
// them: never set to a verdict by hand, so nothing passes — or fails — a merge past the check (novox/hq
@@ -136,7 +136,7 @@ export interface StatusWriters {
export function commitStatusTool(writers: StatusWriters): ToolDefinition {
return {
name: "gitea_commit_status",
description: "Set one of the mesh's statuses on a commit (mesh/delivery, mesh/delivery-group): pending, success, error, failure or warning, a short description, and the page it links to. mesh/check, the status main requires, is refused: the checker's forge bridge writes it through gitea_check_status. hq's mesh/repo-check (and mesh/merge-gate, kept for a revert) take pending only — a recheck putting them back until the fresh verdict sets them — on the head's whole sha, in their turn after the check's earlier writes.",
description: "Set one of the mesh's statuses on a commit (mesh/delivery, mesh/delivery-group): pending, success, error, failure or warning, a short description, and the page it links to. mesh/check, the status main requires, is refused: no tool writes it: it is written from the checker's own state. hq's mesh/repo-check (and mesh/merge-gate, kept for a revert) take pending only — a recheck putting them back until the fresh verdict sets them — on the head's whole sha, in their turn after the check's earlier writes.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
@@ -163,48 +163,3 @@ export function commitStatusTool(writers: StatusWriters): ToolDefinition {
},
};
}
/** What the check's status tool writes with: the forge. */
export interface CheckStatusWriter {
setCommitStatus(owner: string, repo: string, sha: string, status: CommitStatus): Promise<void>;
}
/**
* The `gitea_check_status` tool (novox/hq issue 1301, ADR 0447 §1): the one way `mesh/check` is written. Its arguments
* are gitea_commit_status's, so the forge bridge sends an owed status's arguments unchanged to it: owner, repo, sha
* (the head's whole sha), state, description, target_url, and context, which may be left out and is mesh/check when
* given. **Who may write it is the bus's grant**: the runtime tells a tool no caller, so only a module that lists
* `gitea.gitea_check_status` in its manifest's `invokes` may call it, and the forge bridge alone does. Served beside
* gitea_commit_status, from the watcher's bundle.
*/
export function checkStatusTool(writer: CheckStatusWriter): ToolDefinition {
return {
name: "gitea_check_status",
description: "Set mesh/check, the checker's verdict and the one status main requires, on a pull request's head: pending, success, error, failure or warning, a short description, and the page it links to. The checker's forge bridge alone may call it (its manifest's invokes); no other status is written here, and no other tool writes this one.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
sha: { type: "string", description: "the head's whole sha" },
context: { type: "string", description: "mesh/check, or left out" },
state: { type: "string", description: "pending | success | error | failure | warning" },
description: { type: "string", description: "one line" },
target_url: { type: "string", description: "the page it links to (the check's report)" },
},
run: async (args) => {
const owner = checkName("owner", args.owner), repo = checkName("name", args.repo), sha = String(args.sha ?? "");
if (args.context !== undefined && args.context !== null && String(args.context) !== CHECK_STATUS) {
throw new Error(`${JSON.stringify(args.context)}: this tool writes ${CHECK_STATUS} alone. Nothing was sent`);
}
if (!/^([0-9a-f]{40}|[0-9a-f]{64})$/i.test(sha)) {
throw new Error(`${JSON.stringify(args.sha)} is not a head's whole sha: ${CHECK_STATUS} names the head whole. Nothing was sent`);
}
// deliveryStatus checks the state and cuts the description; it refuses mesh/check, so it is asked for a
// context of the mesh's own and the status is named mesh/check after.
const checked = deliveryStatus("mesh/delivery", String(args.state), String(args.description ?? ""),
args.target_url ? String(args.target_url) : undefined);
const status = { ...checked, context: CHECK_STATUS };
await writer.setCommitStatus(owner, repo, sha, status);
return { set: status };
},
};
}
+17 -5
View File
@@ -13,8 +13,12 @@
// commit's statuses: `mesh/merge-gate`, the modules of the mesh's graph the change touches, and
// `mesh/repo-check`, the repository's own merge-check.sh (novox/hq ADR 0237 as amended); with a comment
// saying why when the gate is not a pass or the repository's own check failed. Of the two, only those the base
// still requires are written (issue 1301): main requires `mesh/check` — the checker's, written by its forge bridge
// through gitea_check_status — everywhere but hq, whose main requires `mesh/repo-check` alone (ADR 0503 §5).
// still requires are written (issue 1301): main requires `mesh/check` — the checker's — everywhere but hq, whose main
// requires `mesh/repo-check` alone (ADR 0503 §5).
//
// Reads mesh-delivery-checker.owed — what the checker owes the forge, mesh/check on each head it judged — by a watch, and
// writes it (check.ts; ADR 0521, R4e finding R2-1). No tool writes mesh/check (finding K1), and no event: the watch hands
// over the latest value of every key whenever this module starts.
//
// Consumes node-build-agent.started — a build machine took an ask — and says, on the head a pending check is of,
// which machine builds it since when (novox/hq issue 446). This module is the one writer of mesh/merge-gate.
@@ -41,7 +45,9 @@
import { emit, on } from "@novox/mesh-sdk/events";
import { registerModuleTools } from "@novox/mesh-sdk/tools";
import { statesAnnouncement } from "./kept.js";
import { checkStatusTool, commitStatusTool } from "./delivery.js";
import { commitStatusTool } from "./delivery.js";
import { CHECKER_OWED, CheckStatuses, watchOwed, type OwedChange } from "./check.js";
import { state } from "@novox/mesh-sdk/state";
import { announce, keepWatermark, readWatermark } from "./announce.js";
import { mergeTool } from "./merge.js";
import { GiteaAdmin, GiteaClient, movedSince } from "./client.js";
@@ -475,6 +481,11 @@ const gates = new GateWriter({
headChecks: (owner, repo, sha) => (gitea as GiteaClient).headChecks(owner, repo, sha),
});
// The one writer of mesh/check, from the checker's owed (check.ts; ADR 0521).
const checks = new CheckStatuses({
setCommitStatus: (owner, repo, sha, status) => (gitea as GiteaClient).setCommitStatus(owner, repo, sha, status),
});
// **The status tool is served here, beside the writer** (novox/hq issue 456): the runtime starts each bundle as its own
// process, so a recheck's pending written from the tools' bundle bypassed this writer's chain. Without a forge to
// reach, nothing is served, as the tools' bundle serves nothing then.
@@ -491,8 +502,6 @@ await emit("states", statesAnnouncement(process.env.MESH_MODULE || "gitea", proc
registerModuleTools("gitea", () => [eventsTool(pace), ...(gitea ? [commitStatusTool({
setCommitStatus: (owner, repo, sha, status) => (gitea as GiteaClient).setCommitStatus(owner, repo, sha, status),
gates,
}), checkStatusTool({
setCommitStatus: (owner, repo, sha, status) => (gitea as GiteaClient).setCommitStatus(owner, repo, sha, status),
}), mergeTool(gitea, { gates })] : [])]);
// **A build machine took an ask** (the build seat's `started`, novox/hq ADR 0157): said on the head it checks, when
@@ -621,4 +630,7 @@ if (gitea) {
})],
["node-build-agent.started", () => on("node-build-agent.started", buildStarted)],
], () => console.log("[gitea] watching for new repositories and merged pull requests"));
// The checker's mesh/check, from its owed state (check.ts; ADR 0521): a read not granted yet is said once and asked again,
// apart from the binds above, so neither waits for the other.
void watchOwed((handler) => state<OwedChange["value"]>(CHECKER_OWED).watch((change) => handler(change)), checks);
}
+2 -1
View File
@@ -543,6 +543,7 @@
"ssh-client.keys",
"mesh-build-go.keys",
"mesh-build-node.keys",
"mesh-build-image.keys"
"mesh-build-image.keys",
"mesh-delivery-checker.owed"
]
}
+1 -1
View File
@@ -6,7 +6,7 @@
"private": true,
"scripts": {
"build": "tsc client.ts token.ts index.ts provisioner/index.ts provisioner/teams.ts tools/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"test": "npm run build && node --test --experimental-strip-types 'test/*.test.ts'"
"test": "npm run build && tsc --module NodeNext --moduleResolution NodeNext --target ES2022 --types node --rootDir . --outDir . $(find . -name '*.ts' ! -name '*.d.ts' ! -path './test/*' ! -path './node_modules/*' ! -path './dist/*') && node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.1"
+3 -3
View File
@@ -19,13 +19,13 @@ export const CHECK_CONTEXT = "mesh/merge-gate";
/** The status context of the repository's own merge-check.sh, the check's second layer (novox/hq ADR 0237). */
export const REPO_CHECK_CONTEXT = "mesh/repo-check";
/** The one status main requires (novox/hq ADR 0447 §1, issue 1301): the checker's verdict, written by its forge bridge
* through gitea_check_status. Its line names the head and the trunk commit it judged: "pass <head12> on <trunk12> …". */
/** The one status main requires (novox/hq ADR 0447 §1, issue 1301): the checker's verdict, written by this module from the
* checker's owed state (check.ts, ADR 0521). Its line names the head and the trunk commit it judged: "pass <head12> on <trunk12> …". */
export const CHECK_STATUS = "mesh/check";
/** The statuses a branch's rule requires, from the rule the forge applies to it: none when it is unprotected or checks
* no status; undefined when the rule could not be read. */
export function requiredOf(rule: { enable_status_check?: boolean; status_check_contexts?: string[] } | null | undefined): string[] | undefined {
export function requiredOf(rule: { enable_status_check?: boolean; status_check_contexts?: string[] | null } | null | undefined): string[] | undefined {
if (rule === undefined) return undefined;
if (!rule || !rule.enable_status_check) return [];
return (rule.status_check_contexts ?? []).filter(Boolean);
+73
View File
@@ -36,3 +36,76 @@ test("importing the bundle resolves while the broker keeps refusing the binds",
clearTimeout(timer);
assert.match(out, /imported/, `the import did not resolve within 3 s while a bind was refused; stderr: ${err.slice(0, 600)}`);
});
// **The bundle writes mesh/check from the checker's owed state, and serves no tool that writes it** (ADR 0521; R4e round 2
// finding R2-1; finding K1). The bundle is imported whole, as the runtime imports it, while its read of the checker's owed
// is refused (the grant not yet composed): the watch is asked again, and once granted it is handed the bucket as the bus
// keeps it, the latest revision of each key, then an older revision delivered late. The forge stand-in sees the latest
// written, and nothing after it.
test("a bundle whose read of the checker's owed is granted late writes the latest mesh/check, and serves no gitea_check_status", async () => {
const index = pathToFileURL(join(here, "..", "dist", "index.js")).href;
const sha = "0123456789abcdef0123456789abcdef01234567";
const script = `
const http = await import("node:http");
const statuses = [];
const forge = http.createServer((req, res) => {
let b = ""; req.on("data", (d) => (b += d)); req.on("end", () => {
if (req.method === "POST" && /\\/statuses\\//.test(req.url)) statuses.push(req.url + " " + b);
res.setHeader("content-type", "application/json"); res.end(req.method === "GET" ? "[]" : "{}");
});
});
await new Promise((r) => forge.listen(0, "127.0.0.1", r));
process.env.MESH_GITEA_URL = "http://127.0.0.1:" + forge.address().port;
const bound = [];
const { useBroker } = await import("@novox/mesh-sdk/messaging");
useBroker(() => ({
subscribe: async (pattern) => { bound.push(pattern); return () => {}; },
publish: async () => {}, request: async () => { throw new Error("no responders"); },
handle: async () => () => {}, close: async () => {},
}));
const key = "novox/mesh-sdk.12.${sha}.status=mesh/check";
const owed = (state, text) => ({ v: 1, act: "status", context: "mesh/check", state, text, target: "https://view.example/c" });
const watched = [];
let granted = 0;
const { useStateChannel } = await import("@novox/mesh-sdk/state");
useStateChannel({
ask: async () => null,
watch: async (params, deliver) => {
watched.push(params.state);
if (watched.length === 1) throw new Error('gitea keeps and reads no state called "mesh-delivery-checker.owed"');
granted++;
// Two revisions were written while the read was refused; the bucket hands the latest, then a late older one.
await deliver({ state: params.state, key, op: "put", value: owed("success", "pass at 4"), revision: 4, current: true });
await deliver({ state: params.state, key, op: "put", value: owed("pending", "pending at 3"), revision: 3, current: false });
},
});
await import(${JSON.stringify(index)});
const { collectTools } = await import("@novox/mesh-sdk/tools");
const tools = collectTools().flatMap((r) => r.tools.map((t) => t.name));
for (let i = 0; i < 300 && !(granted && statuses.length); i++) await new Promise((r) => setTimeout(r, 20));
await new Promise((r) => setTimeout(r, 200));
console.log(JSON.stringify({ tools, bound, watched, statuses }));
process.exit(0);`;
const env = { ...process.env, MESH_GITEA_TOKEN: "x" };
for (const k of ["MESH_GITEA_STATE_DIR", "MESH_GITEA_ADMIN_USER", "MESH_GITEA_ADMIN_PASSWORD", "MESH_SERVED_MODULE", "MESH_GITEA_URL"]) delete env[k];
const child = spawn(process.execPath, ["--input-type=module", "-e", script], { cwd: join(here, ".."), env, stdio: ["ignore", "pipe", "pipe"] });
let out = "";
let err = "";
child.stdout.on("data", (d) => (out += d));
child.stderr.on("data", (d) => (err += d));
const ended = new Promise<void>((r) => child.on("exit", () => r()));
const timer = setTimeout(() => child.kill("SIGKILL"), 15000);
await ended;
clearTimeout(timer);
const line = out.split("\n").find((l) => l.startsWith("{"));
assert.ok(line, `the bundle said nothing; stdout: ${out.slice(0, 400)} stderr: ${err.slice(0, 600)}`);
const got = JSON.parse(line!) as { tools: string[]; bound: string[]; watched: string[]; statuses: string[] };
assert.ok(got.tools.includes("gitea_commit_status"), `the bundle's tools: ${got.tools}`);
assert.ok(!got.tools.includes("gitea_check_status"), "no tool writes mesh/check");
assert.ok(!got.bound.some((p) => p.includes("check.status")), `bound: ${got.bound}`);
assert.deepEqual(got.watched, ["mesh-delivery-checker.owed", "mesh-delivery-checker.owed"], "refused, then asked again");
assert.equal(got.statuses.length, 1, `statuses written: ${got.statuses}`);
assert.match(got.statuses[0], new RegExp(`/repos/novox/mesh-sdk/statuses/${sha} .*"context":"mesh/check"`));
assert.match(got.statuses[0], /"state":"success".*pass at 4/);
assert.match(err, /mesh-delivery-checker\.owed could not be bound/, "the refusal is said");
});
+135
View File
@@ -0,0 +1,135 @@
// **mesh/check is written from the checker's owed state, read by a watch** (ADR 0521; R4e round 2 finding R2-1; finding K1
// before it). No tool writes mesh/check, since a tool is told no caller and node-tools invokes every tool. Nor does an event:
// nothing in the delivery plan orders this module before the forge bridge, and a durable consumer whose filters change keeps
// its deliver policy, so an event said before this module consumed it was lost. A watch of the checker's state hands over
// the latest value of every key when it starts, whenever that is.
import { test } from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
const sha = "0123456789abcdef0123456789abcdef01234567";
const key = (s = sha, repo = "novox/mesh-sdk", n = 12) => `${repo}.${n}.${s}.status=mesh/check`;
const owed = (over: Record<string, unknown> = {}) => ({ v: 1, act: "status", context: "mesh/check", state: "success",
text: "pass 0123456789ab on aaaaaaaaaaaa", target: "https://view.example/checks/g.1", ...over });
const put = (k: string, value: unknown, revision: number, current = false) =>
({ state: "mesh-delivery-checker.owed", key: k, op: "put" as const, value, revision, current });
function forge() {
const sent: string[] = [];
let failing = 0;
return {
sent,
fail: (n: number) => { failing = n; },
setCommitStatus: async (o: string, r: string, s: string, st: any) => {
if (failing > 0) { failing--; throw new Error("Gitea API /repos/x/statuses: 502 bad gateway"); }
sent.push(`${o}/${r}@${s.slice(0, 8)} ${st.context} ${st.state} ${st.description}${st.target_url ? " " + st.target_url : ""}`);
},
};
}
const noSleep = async () => {};
test("the checker's owed is what this module reads, and it consumes no check.status", async () => {
const { CHECKER_OWED } = await import("../check.ts");
assert.equal(CHECKER_OWED, "mesh-delivery-checker.owed");
const manifest = JSON.parse(readFileSync(new URL("../module.json", import.meta.url), "utf8"));
assert.ok(manifest.reads.includes(CHECKER_OWED), "gitea's manifest reads the checker's owed");
assert.ok(!manifest.consumes.some((c: string) => c.includes("check.status")), "no check.status is consumed");
});
test("an owed mesh/check writes the status on its head: its state, its line cut and its link", async () => {
const { CheckStatuses } = await import("../check.ts");
const f = forge();
const checks = new CheckStatuses(f, () => {}, noSleep);
assert.equal(await checks.apply(put(key(), owed(), 7, true)), "written");
const other = sha.replace(/0/g, "f");
assert.equal(await checks.apply(put(key(other, "novox/my=2erepo"), owed({ state: "pending", text: "y".repeat(300), target: "javascript:x" }), 8)), "written");
assert.equal(f.sent[0], "novox/mesh-sdk@01234567 mesh/check success pass 0123456789ab on aaaaaaaaaaaa https://view.example/checks/g.1");
assert.ok(f.sent[1].startsWith("novox/my.repo@f1234567 mesh/check pending yyyy"), f.sent[1]);
assert.ok(f.sent[1].length < 200, "the line is cut as every status's is");
assert.ok(!f.sent[1].includes("javascript"), "a link that is no web page is not written");
});
test("a key or value that is no owed mesh/check, and a delete, write nothing", async () => {
const { CheckStatuses } = await import("../check.ts");
const f = forge();
const checks = new CheckStatuses(f, () => {}, noSleep);
const cases = [
put(key(sha.slice(0, 12)), owed(), 1), put(`novox/mesh-sdk.12.${sha}.status=mesh/merge-gate`, owed(), 1),
put(`novox/mesh-sdk.12.${sha}.view`, owed(), 1), put(`nokey`, owed(), 1), put(`../x/y.12.${sha}.status=mesh/check`, owed(), 1),
put(key(), owed({ context: "mesh/delivery" }), 1), put(key(), owed({ act: "view" }), 1), put(key(), owed({ state: "green" }), 1),
put(key(), owed({ v: 2 }), 1), put(key(), null, 1), put(key(), "text", 1), put(key(), owed(), -1),
{ state: "mesh-delivery-checker.owed", key: key(), op: "delete" as const, revision: 2, current: false },
];
for (const c of cases) assert.equal(await checks.apply(c as any), "dropped", JSON.stringify(c));
assert.deepEqual(f.sent, []);
});
test("a revision older than one written on the key is dropped; the same one is written again", async () => {
const { CheckStatuses } = await import("../check.ts");
const f = forge();
const checks = new CheckStatuses(f, () => {}, noSleep);
await checks.apply(put(key(), owed({ state: "success" }), 10));
assert.equal(await checks.apply(put(key(), owed({ state: "pending" }), 9)), "dropped", "a stale pending never hides the verdict");
assert.equal(await checks.apply(put(key(), owed({ state: "success" }), 10)), "written");
assert.equal(await checks.apply(put(key(sha.replace(/1/g, "2")), owed({ state: "pending" }), 3)), "written", "another key is its own");
assert.equal(await checks.apply(put(key(), owed({ state: "pending" }), 11)), "written");
assert.deepEqual(f.sent.map((s) => s.split(" ").slice(0, 3).join(" ")), ["novox/mesh-sdk@01234567 mesh/check success",
"novox/mesh-sdk@01234567 mesh/check success", "novox/mesh-sdk@02234567 mesh/check pending", "novox/mesh-sdk@01234567 mesh/check pending"]);
});
test("a forge that refuses is asked again until it writes, unless a newer revision of the key came meanwhile", async () => {
const { CheckStatuses } = await import("../check.ts");
const f = forge();
const said: string[] = [];
const waits: number[] = [];
const checks = new CheckStatuses(f, (l) => said.push(l), async (ms) => { waits.push(ms); });
f.fail(2);
await checks.take(put(key(), owed({ state: "success" }), 5));
await checks.settled();
assert.deepEqual(f.sent.map((s) => s.split(" ")[2]), ["success"], "written on the third ask");
assert.deepEqual(waits, [2_000, 5_000]);
assert.match(said[0], /502.*asked again/);
// A refusal whose key moved on meanwhile is not asked again: the newer revision is written, the older never after it.
const g = forge();
let release: () => void = () => {};
const gate = new Promise<void>((r) => { release = r; });
const later = new CheckStatuses(g, () => {}, () => gate);
g.fail(1);
await later.take(put(key(), owed({ state: "pending" }), 6));
await later.take(put(key(), owed({ state: "success" }), 7));
release();
await later.settled();
assert.deepEqual(g.sent.map((s) => s.split(" ")[2]), ["success"]);
});
test("a watch refused at first is asked again, said once, and once granted writes the latest revision of each key", async () => {
const { CheckStatuses, watchOwed } = await import("../check.ts");
const f = forge();
const said: string[] = [];
const checks = new CheckStatuses(f, () => {}, noSleep);
// The checker's bucket as the bus keeps it: two revisions of one key written while the read was refused.
const kept = new Map<string, { value: unknown; revision: number }>();
kept.set(key(), { value: owed({ state: "pending" }), revision: 3 });
kept.set(key(), { value: owed({ state: "success" }), revision: 4 });
let asked = 0;
const watch = async (handler: (c: any) => Promise<void>) => {
asked++;
if (asked < 3) throw new Error("gitea keeps and reads no state called \"mesh-delivery-checker.owed\"");
for (const [k, e] of kept) await handler(put(k, e.value, e.revision, true));
};
await watchOwed(watch, checks, noSleep, (l) => said.push(l));
await checks.settled();
assert.equal(asked, 3);
assert.deepEqual(f.sent.map((s) => s.split(" ")[2]), ["success"], "only the latest revision is written");
assert.equal(said.filter((l) => /could not be bound/.test(l)).length, 1, `said once: ${said.join(" | ")}`);
assert.ok(said.some((l) => /bound after 3 attempts/.test(l)), said.join(" | "));
});
test("no tool writes mesh/check: gitea_check_status is gone and gitea_commit_status refuses it", async () => {
const delivery = await import("../delivery.ts");
assert.equal((delivery as Record<string, unknown>).checkStatusTool, undefined, "delivery.ts serves no gitea_check_status");
assert.throws(() => delivery.deliveryStatus("mesh/check", "success", "x"), /mesh-delivery-checker\.owed/);
const index = readFileSync(new URL("../index.ts", import.meta.url), "utf8");
assert.ok(!/gitea_check_status|checkStatusTool/.test(index), "index.ts serves no gitea_check_status");
});
+7 -24
View File
@@ -62,37 +62,20 @@ test("one view per pull request, found by its marker; only the mesh's statuses",
assert.equal(deliveryStatus("mesh/delivery", "success", "x", "javascript:alert(1)").target_url, undefined);
});
// Issue 1301-S4 (ADR 0447 §1; the lead's choice A): main requires mesh/check, so no hand may set it. The status tool
// every caller may use refuses it in every state; it is written only by gitea_check_status, which the forge bridge
// alone lists in its manifest's `invokes`, so only its account may call it.
test("mesh/check is refused by the status tool and written by the check's own tool alone", async () => {
const { commitStatusTool, checkStatusTool, deliveryStatus, CHECK_STATUS } = await import("../delivery.ts");
// Issue 1301-S4 (ADR 0447 §1; R4e findings K1 and R2-1): main requires mesh/check, so no hand may set it. The status tool
// every caller may use refuses it in every state; no tool writes it at all: it is written from the checker's owed state
// (check.ts, test/check-owed.test.ts; ADR 0521).
test("mesh/check is refused by the status tool in every state", async () => {
const { commitStatusTool, deliveryStatus, CHECK_STATUS } = await import("../delivery.ts");
assert.equal(CHECK_STATUS, "mesh/check");
const sha = "0123456789abcdef0123456789abcdef01234567";
const sent: string[] = [];
const setCommitStatus = async (o: string, r: string, s: string, st: any) =>
void sent.push(`${o}/${r}@${s.slice(0, 8)} ${st.context} ${st.state} ${st.description}${st.target_url ? " " + st.target_url : ""}`);
const setCommitStatus = async (o: string, r: string, s: string, st: any) => void sent.push(`${o}/${r}@${s} ${st.context}`);
const gates = { rechecked: async () => { throw new Error("not the gate writer's"); } };
for (const state of ["pending", "success", "failure", "error", "warning"]) {
assert.throws(() => deliveryStatus("mesh/check", state, "x"), /mesh\/check is the checker's status/);
await assert.rejects(commitStatusTool({ setCommitStatus, gates } as any).run({ owner: "novox", repo: "mesh-sdk", sha,
context: "mesh/check", state, description: "by hand" }), /gitea_check_status/);
context: "mesh/check", state, description: "by hand" }), /mesh-delivery-checker\.owed/);
}
assert.deepEqual(sent, [], "nothing was sent for a hand-set mesh/check");
const tool = checkStatusTool({ setCommitStatus });
assert.equal(tool.name, "gitea_check_status");
for (const k of ["owner", "repo", "sha", "state", "description", "target_url", "context"]) assert.ok(k in tool.input, k);
// The bridge's arguments as it sends them for gitea_commit_status: the context may be given, and is mesh/check.
await tool.run({ owner: "novox", repo: "mesh-sdk", sha, context: "mesh/check", state: "success", description: "pass",
target_url: "https://forge.invalid/novox/mesh-sdk/pulls/1" });
await tool.run({ owner: "novox", repo: "mesh-sdk", sha, state: "pending", description: "y".repeat(300) });
assert.equal(sent[0], "novox/mesh-sdk@01234567 mesh/check success pass https://forge.invalid/novox/mesh-sdk/pulls/1");
assert.ok(sent[1].startsWith("novox/mesh-sdk@01234567 mesh/check pending yyyy"), sent[1]);
assert.ok(sent[1].length < 200, "the description is cut as every status's is");
await assert.rejects(tool.run({ owner: "novox", repo: "mesh-sdk", sha, context: "mesh/delivery", state: "success" }), /mesh\/check alone/);
await assert.rejects(tool.run({ owner: "novox", repo: "mesh-sdk", sha: sha.slice(0, 12), state: "success" }), /whole sha/);
await assert.rejects(tool.run({ owner: "novox", repo: "mesh-sdk", sha, state: "green" }), /not a status/);
await assert.rejects(tool.run({ owner: "../x", repo: "mesh-sdk", sha, state: "success" }));
assert.equal(sent.length, 2, "nothing more was sent");
});
+2 -5
View File
@@ -15,7 +15,7 @@ import { GiteaClient } from "../dist/client.js";
import { ConfiguredToken } from "../dist/token.js";
import { getGiteaTools } from "../dist/tools/index.js";
import { mergeTool } from "../dist/merge.js";
import { checkStatusTool, commitStatusTool } from "../dist/delivery.js";
import { commitStatusTool } from "../dist/delivery.js";
import { GateWriter } from "../dist/gate.js";
import { judgeApiCall, judgeProtectionChange, ruleCovers, routedPath } from "../dist/guard.js";
@@ -168,7 +168,7 @@ const gates = { rechecked: (o: string, r: string, sha: string, s: any) => {
} };
const tools = [...getGiteaTools(client), commitStatusTool({
setCommitStatus: (o, r, sha, s) => client.setCommitStatus(o, r, sha, s), gates: writer,
}), checkStatusTool({ setCommitStatus: (o, r, sha, s) => client.setCommitStatus(o, r, sha, s) }), mergeTool(client, { gates, stateDir, emit: async (type, body) => { state.emitted.push({ type, body }); },
}), mergeTool(client, { gates, stateDir, emit: async (type, body) => { state.emitted.push({ type, body }); },
now: () => new Date(2026, 9, 12, 15, 1) })];
const tool = (name: string) => tools.find((t) => t.name === name)!;
const writes = () => state.sent.map((s) => `${s.method} ${s.path}`);
@@ -568,9 +568,6 @@ test("no tool's caller text becomes another path or another user: `..` and `?sud
["gitea_get_file", { ...repo, path: "README.md?sudo=jschoubben" }],
["gitea_commit_status", { ...repo, sha: "abc1234?sudo=jschoubben", context: "mesh/delivery", state: "success" }],
["gitea_commit_status", { ...repo, sha: "../../../admin/users", context: "mesh/delivery", state: "success" }],
["gitea_check_status", { ...repo, sha: "abc1234?sudo=jschoubben", state: "success" }],
["gitea_check_status", { ...repo, sha: "../../../admin/users", state: "success" }],
["gitea_check_status", { owner: "novox", repo: "..", sha: "0123456789abcdef0123456789abcdef01234567", state: "success" }],
["gitea_commit_status", { ...repo, sha: "0123456789abcdef0123456789abcdef01234567", context: "mesh/check", state: "success" }],
["gitea_commit_statuses", { ...repo, sha: ".." }],
["gitea_commit_statuses", { ...repo, sha: "main?sudo=jschoubben" }],
+9
View File
@@ -216,3 +216,12 @@ test("a base's required statuses decide which of the controller's statuses are w
const comment = commentFor(c, {}, ["mesh/repo-check"]);
assert.ok(comment === null || !comment.includes("mesh/merge-gate"), comment ?? "");
});
test("a rule the forge gives with no contexts (null) requires nothing, and an unread rule is unknown", async () => {
const { requiredOf } = await import("../pulls.ts");
// The forge's BranchProtection says status_check_contexts may be null (protection.ts); requiredOf takes it as it comes.
assert.deepEqual(requiredOf({ enable_status_check: true, status_check_contexts: null }), []);
assert.deepEqual(requiredOf({ enable_status_check: true, status_check_contexts: ["mesh/check", ""] }), ["mesh/check"]);
assert.deepEqual(requiredOf(null), []);
assert.equal(requiredOf(undefined), undefined);
});
+1 -1
View File
@@ -128,7 +128,7 @@ export interface TrunkForge {
branchHead(owner: string, repo: string, branch: string): Promise<string>;
/** The rule the forge applies to a branch: which of the controller's statuses are kept there (issue 1301). Not given,
* or not read: both, as before. */
appliedProtection?(owner: string, repo: string, branch: string): Promise<{ enable_status_check?: boolean; status_check_contexts?: string[] } | null>;
appliedProtection?(owner: string, repo: string, branch: string): Promise<{ enable_status_check?: boolean; status_check_contexts?: string[] | null } | null>;
}
/** What it writes with: the merge check's one writer (gate.ts), which decides in the head's own turn. */
@@ -39,12 +39,8 @@ func TestTheBaseParses(t *testing.T) {
t.Skip("logrotate is not installed here; the base configuration is not dry-run")
}
dir := t.TempDir()
content := strings.ReplaceAll(templateText(t, manifest(t).resource(t, "config")), "include "+RulesDir, "include "+filepath.Join(dir, "d"))
if err := os.Mkdir(filepath.Join(dir, "d"), 0o755); err != nil {
t.Fatal(err)
}
conf := filepath.Join(dir, "logrotate.conf")
if err := os.WriteFile(conf, []byte(content), 0o644); err != nil {
if err := os.WriteFile(conf, []byte(dryRunBase(t, dir)), 0o644); err != nil {
t.Fatal(err)
}
out, err := exec.Command(logrotate, "-d", "-s", filepath.Join(dir, "state"), conf).CombinedOutput()
@@ -52,3 +48,40 @@ func TestTheBaseParses(t *testing.T) {
t.Fatalf("logrotate -d: %v\n%s", err, out)
}
}
// The dry run proves the base parses, not what the machine's logs are: the unit level runs shut off in
// bubblewrap (mesh-sdk buildstep.Isolated), where /var/log is not there, and a base block without
// missingok then fails its stat (issue 1891).
func TestTheDryRunReadsNothingOfTheMachine(t *testing.T) {
dir := t.TempDir()
content := dryRunBase(t, dir)
for _, machine := range []string{"include " + RulesDir, "/var/log/"} {
if strings.Contains(content, machine) {
t.Fatalf("the dry run's base still names %s of the machine:\n%s", machine, content)
}
}
for _, log := range []string{"wtmp", "btmp"} {
if _, err := os.Stat(filepath.Join(dir, "log", log)); err != nil {
t.Fatalf("the base's own log %s is not in the dry run's directory: %v", log, err)
}
}
}
// dryRunBase is the base configuration pointed into dir: its include at dir/d, and its own logs at dir/log,
// both made there.
func dryRunBase(t *testing.T, dir string) string {
t.Helper()
content := strings.ReplaceAll(templateText(t, manifest(t).resource(t, "config")), "include "+RulesDir, "include "+filepath.Join(dir, "d"))
content = strings.ReplaceAll(content, "/var/log/", filepath.Join(dir, "log")+"/")
for _, d := range []string{"d", "log"} {
if err := os.Mkdir(filepath.Join(dir, d), 0o755); err != nil {
t.Fatal(err)
}
}
for _, log := range []string{"wtmp", "btmp"} {
if err := os.WriteFile(filepath.Join(dir, "log", log), nil, 0o644); err != nil {
t.Fatal(err)
}
}
return content
}
+11 -2
View File
@@ -37,8 +37,17 @@ Three read tools answer what the module stored in its own states, without the gu
forge-account provision's ssh name, so the account's own ssh files and git rewrite are used) into a workspace of the
work directory, then runs `go mod download` and `go mod verify` in every directory with a `go.mod` that does not vendor,
with `GOTOOLCHAIN=local`, `GOFLAGS=-mod=mod` and the module's cache directory. Every command runs with an
environment built from scratch (`StepRunner`): `PATH`, `HOME` and what the step passes. Workspaces older than a day
are removed.
environment built from scratch (`StepRunner`): `PATH`, `HOME` and what the step passes.
## Room to work and the workspace's end
Every step is refused, ending `refused` with `refused: <node> has N GiB free under the floor of 20 GiB (on <dir>)`, while
the work or the cache directory's filesystem has less than 20 GiB free (build-agent's floor, issue 1839); the health is
then unhealthy with the same text, so the builder chooses another node and the controller raises the condition. A
workspace is held while a step runs on it. `build_go_release {workspace}` removes it at once (no guard, no floor; one
already gone is no error, one a running step holds is refused); its caller calls it once its last step answered. A
failed fetch removes its own workspace. A workspace no caller released is removed two hours after its last use (ADR
0509), and the removal is said in the log as the caller's fault.
A pin on a pull request's head (issue 1148): when `go mod download` fails on a `git.novox.be/novox/*` pseudo-version, the
fetch looks for that commit among the forge's `refs/pull/*/head`, places the repository at it under the workspace's
@@ -23,11 +23,6 @@ var commitOK = regexp.MustCompile(`^[0-9a-f]{40}$`)
// remote of a fetch is ssh://<it>/<owner>/<repository>.git.
const ForgeAlias = "mesh-forge"
// WorkspaceAge is how long a workspace is kept: a module removes the workspaces older than a day, each time it
// fetches and every round it checks its toolchain. A workspace lies in the module's work directory, never in its
// home (ADR 0466 §1).
const WorkspaceAge = 24 * time.Hour
var (
repositoryOK = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]*/[A-Za-z0-9][A-Za-z0-9._-]*$`)
workspaceOK = regexp.MustCompile(`^[a-z0-9]+$`)
@@ -82,20 +77,37 @@ func (m *Module) fetchTool() StepTool {
}
}
func (m *Module) fetch(ctx context.Context, args map[string]any) (any, error) {
// fetch makes a workspace and fills it. A workspace lies in the module's work directory, never in its home (ADR 0466
// §1); it is held while the fetch runs, and removed when the fetch fails, since nobody learns its id to release it
// (issue 1839).
func (m *Module) fetch(ctx context.Context, args map[string]any) (res any, err error) {
// What the builder and the checker ask, in mesh-sdk's one form (ADR 0499 §4): with the module's directory and its
// named paths, the workspace is narrowed to them (buildstep.Narrow).
req, err := buildstep.ParseFetchRequest(args)
if err != nil {
return nil, err
}
repository, commit, ref, base := req.Repository, req.Commit, req.Ref, req.Base
siblings, err := siblingsArg(args["beside"])
if err != nil {
return nil, err
}
m.CleanWorkspaces(m.now())
id := buildstep.NewID()
done := m.spaces().Hold(id)
defer func() {
done()
if err != nil {
if rerr := m.spaces().Release(id); rerr != nil && m.Say != nil {
m.Say("the failed fetch's workspace %s: %v", id, rerr)
}
}
}()
return m.fetchInto(ctx, req, id, siblings)
}
func (m *Module) fetchInto(ctx context.Context, req buildstep.FetchRequest, id string, siblings []siblingArg) (any, error) {
repository, commit, ref, base := req.Repository, req.Commit, req.Ref, req.Base
var err error
root := filepath.Join(m.Work, id)
tree := filepath.Join(root, "tree")
if err := os.MkdirAll(tree, 0o700); err != nil {
@@ -236,21 +248,10 @@ func (m *Module) goEnvFor(dir string) []string {
return env
}
// CleanWorkspaces removes the workspaces older than WorkspaceAge from the work directory.
// CleanWorkspaces removes the workspaces no step holds whose last use is older than buildstep.UnheldBound: the safety
// net for a caller that never released one (issue 1839, ADR 0509).
func (m *Module) CleanWorkspaces(now time.Time) {
entries, err := os.ReadDir(m.Work)
if err != nil {
return
}
for _, e := range entries {
info, err := e.Info()
if err != nil || !e.IsDir() || !workspaceOK.MatchString(e.Name()) || now.Sub(info.ModTime()) < WorkspaceAge {
continue
}
if err := os.RemoveAll(filepath.Join(m.Work, e.Name())); err != nil && m.Say != nil {
m.Say("the workspace %s could not be removed: %v", e.Name(), err)
}
}
m.sweepSaid(m.spaces().Sweep(now, buildstep.UnheldBound))
}
// clock is the time now: Now when a test set it, else the wall clock.
@@ -265,10 +265,11 @@ func TestARepositoryThatIsNotOwnerSlashNameIsRefusedBeforeAnythingRuns(t *testin
}
}
// Workspaces older than a day are removed; younger ones, and anything that is not a workspace, stay.
func TestWorkspacesOlderThanADayAreRemoved(t *testing.T) {
// Unheld workspaces last used more than two hours ago are removed (issue 1839, ADR 0509); younger ones, and anything
// that is not a workspace, stay.
func TestUnheldWorkspacesPastTwoHoursAreRemoved(t *testing.T) {
m := &Module{Work: t.TempDir()}
for name, age := range map[string]time.Duration{"old1": 25 * time.Hour, "young": time.Hour, "Keep-This": 48 * time.Hour} {
for name, age := range map[string]time.Duration{"old1": 3 * time.Hour, "young": time.Hour, "Keep-This": 48 * time.Hour} {
dir := filepath.Join(m.Work, name)
os.MkdirAll(filepath.Join(dir, "tree"), 0o755)
when := time.Now().Add(-age)
@@ -68,6 +68,7 @@ func (m *Module) integrationShutOff(ctx context.Context, workspace string, packa
}
iso := m.isolation(workspace, false)
iso.ReadOnly = append(iso.ReadOnly, exe) // the program alone, not the directory it lies in
iso.ReadOnly = append(iso.ReadOnly, m.nodeToolchains()...)
r := buildstep.Isolated(m.Run, iso)(ctx, nil, exe, levelArg, string(spec))
var answer levelAnswer
line := lastLine(r.Stdout)
@@ -80,6 +81,25 @@ func (m *Module) integrationShutOff(ctx context.Context, workspace string, packa
return answer.Result, nil
}
// NodeToolchains is the Node build module's toolchains directory on this node: its cache directory (module.json
// resource `cache`, mode 0711) joined with toolchain, where it installs typescript, esbuild, @novox/mesh-sdk and nats
// from its own lock (ADR 0533 §1, §2; issue 1891).
const NodeToolchains = "/var/lib/mesh-build-node-cache/toolchain"
// nodeToolchains is what the integration level is given of the Node build module's toolchains: their directory,
// read-only, when it is there and this account may enter it, and nothing on a node where the Node build module has
// installed none (ADR 0533 §1). The unit level is never given it.
func (m *Module) nodeToolchains() []string {
dir := m.NodeToolchains
if dir == "" {
dir = NodeToolchains
}
if _, err := os.ReadDir(dir); err != nil {
return nil
}
return []string{dir}
}
func lastLine(s string) string {
lines := strings.Split(strings.TrimSpace(s), "\n")
return strings.TrimSpace(lines[len(lines)-1])
@@ -46,6 +46,7 @@ func main() {
Work: WorkDir,
Cache: CacheDir,
}
m.Floor = floorOf(m.Node)
if u, err := user.Current(); err == nil {
m.Home = u.HomeDir
}
@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"net/http"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
@@ -62,9 +63,18 @@ type Module struct {
Push func(ctx context.Context, registry, repository, digest string, body []byte) error
// Executable is this program's path, which the integration level starts again shut off; os.Executable when empty.
Executable string
// NodeToolchains is where the Node build module keeps its installed toolchains, bound read-only into the
// integration level (ADR 0533 §1); NodeToolchains when empty. A test points it elsewhere.
NodeToolchains string
// StartBus starts the throwaway bus of a run in dir and answers its address and what stops it (the module's own
// program, StartBus in bus.go, when nil).
StartBus func(ctx context.Context, dir string) (url string, stop func(), err error)
// Floor refuses every step while the work or cache filesystem has less free than it (issue 1839); main sets
// floorOf(node), and a zero Floor checks nothing.
Floor buildstep.Floor
spacesOnce sync.Once
workspaces *buildstep.Workspaces
}
// guard is the SDK guard for the version the probe just read: the module names no Go version (ADR 0477 §2), so the
@@ -113,6 +123,9 @@ func (m *Module) Health(ctx context.Context) health.Answer {
if err := m.checkBubblewrap(ctx); err != nil {
return health.Unhealthy(err.Error())
}
if err := m.room(); err != nil {
return health.Unhealthy(err.Error())
}
return health.Healthy(fmt.Sprintf("%s installed, the version the module builds with (recipe %s)", installed, m.Recipe.Version()))
}
@@ -125,6 +138,10 @@ func (m *Module) Long(tool string, run func(ctx context.Context, args map[string
if err := m.Check(ctx); err != nil {
return nil, err
}
if err := m.room(); err != nil {
return nil, err
}
defer m.holdArg(args)()
return run(ctx, args)
})
if err != nil {
@@ -141,6 +158,10 @@ func (m *Module) Short(run func(ctx context.Context, args map[string]any) (any,
if err := m.Check(m.Ctx); err != nil {
return nil, err
}
if err := m.room(); err != nil {
return nil, err
}
defer m.holdArg(args)()
return run(m.Ctx, args)
}
}
@@ -115,7 +115,7 @@ func TestAGoThatCannotBeRunRefusesEveryStepAndHealthIsUnhealthy(t *testing.T) {
{Name: "short_probe", Run: run},
}, m.StepTools()...)
for _, tool := range m.Tools(steps) {
if tool.Name == StatusTool || tool.Name == HealthTool || tool.Name == StepReadTool || tool.Name == StepsReadTool || tool.Name == ToolchainsReadTool {
if tool.Name == StatusTool || tool.Name == HealthTool || tool.Name == StepReadTool || tool.Name == StepsReadTool || tool.Name == ToolchainsReadTool || tool.Name == ReleaseTool {
continue
}
var reason string
@@ -13,7 +13,8 @@ import (
// ADR 0499 §4 (issue 1758): a fetch asked with the module's directory and its named paths leaves the workspace holding
// only those, the manifest and the lock files; the rest of the commit is gone before any run step; a request naming
// paths without a directory is refused before anything is fetched.
// paths without a directory is refused before anything is fetched. go.mod is kept only when named among the paths
// (mesh-sdk's narrowKept is the manifest and the lock files, review finding S1), as a Go build that names its paths does.
func TestAFetchWithNamedPathsNarrowsTheWorkspace(t *testing.T) {
f := newForge(t)
_, head := f.repo("novox/narrow", map[string]string{
@@ -26,7 +27,7 @@ func TestAFetchWithNamedPathsNarrowsTheWorkspace(t *testing.T) {
"other/o.go": "package other\n",
})
m, states := f.module(t)
res, st := fetched(t, m, states, map[string]any{"repository": "novox/narrow", "commit": head, "dir": "lib", "paths": []any{"cmd"}})
res, st := fetched(t, m, states, map[string]any{"repository": "novox/narrow", "commit": head, "dir": "lib", "paths": []any{"cmd", "go.mod"}})
if st.State != delivery.StepPassed {
t.Fatalf("the fetch ended %s: %s", st.State, st.Reason)
}
@@ -0,0 +1,128 @@
package main
import (
"context"
"encoding/json"
"os"
"path/filepath"
"slices"
"strconv"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// integrationShutOffWith runs the integration level through the fake bubblewrap and answers what its one shut-off
// process was given.
func integrationShutOffWith(t *testing.T, toolchains string) shutOff {
t.Helper()
m, states, ws, _ := workspaceFor(t, "failing")
m.Executable = "/opt/bundle/mesh-build-go"
m.NodeToolchains = toolchains
want := TestResult{Level: LevelIntegration, Passed: true, Tests: 1, Packages: []PackageResult{}, Store: "postgres 18"}
var seen []shutOff
m.Run = throughBubblewrap(t, func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
switch {
case name == "postgres" && len(args) == 1 && args[0] == "--version":
return forgekey.Ran{Stdout: "postgres (PostgreSQL) 18.6\n"}
case name == m.Executable && len(args) == 2 && args[0] == levelArg:
raw, _ := json.Marshal(levelAnswer{Result: &want})
return forgekey.Ran{Stdout: string(raw) + "\n"}
}
return forgekey.Ran{Status: 1, Err: "not expected here"}
}, &seen)
run := stepTool(t, m, "build_go_test")
if _, err := run(map[string]any{"workspace": ws, "level": LevelIntegration}); err != nil {
t.Fatal(err)
}
if st := states.waitEnd(t); st.State != delivery.StepPassed {
t.Fatalf("the step ended %s: %s", st.State, st.Reason)
}
if len(seen) != 1 {
t.Fatalf("shut off %d processes, want one: %v", len(seen), seen)
}
checkShutOff(t, m, seen[0], false)
return seen[0]
}
// 1891 (ADR 0533 §1): the integration level is given the Node build module's installed toolchains, read-only, so the
// Node build module's own integration tests build with the real tsc and esbuild and reach no registry.
func TestTheIntegrationLevelIsGivenTheNodeToolchainsReadOnly(t *testing.T) {
dir := t.TempDir()
s := integrationShutOffWith(t, dir)
if !slices.Contains(s.roBinds, dir) {
t.Errorf("the node toolchains %s are not bound read-only: %v", dir, s.roBinds)
}
if slices.Contains(s.binds, dir) || slices.Contains(s.overlays, dir) {
t.Errorf("the node toolchains %s may be written: binds %v, overlays %v", dir, s.binds, s.overlays)
}
if slices.Contains(s.roBinds, filepath.Dir(dir)) {
t.Errorf("the whole cache %s is bound, not its toolchains alone: %v", filepath.Dir(dir), s.roBinds)
}
}
// 1891 (ADR 0533 §1): a node where the Node build module has installed no toolchain runs the level as before, with
// nothing more bound (bubblewrap binds only what is there).
func TestANodeWithoutNodeToolchainsRunsTheLevelWithNothingMoreBound(t *testing.T) {
missing := filepath.Join(t.TempDir(), "none")
s := integrationShutOffWith(t, missing)
if slices.Contains(s.roBinds, missing) {
t.Errorf("a missing %s is bound: %v", missing, s.roBinds)
}
}
// 1891 (ADR 0533 §1): the unit level is not given them.
func TestTheUnitLevelIsNotGivenTheNodeToolchains(t *testing.T) {
m, states, ws, _ := workspaceFor(t, "failing")
dir := t.TempDir()
m.NodeToolchains = dir
var seen []shutOff
m.Run = throughBubblewrap(t, m.Run, &seen)
if _, st := runUnit(t, m, states, ws); st.State != delivery.StepPassed {
t.Fatalf("the step ended %s: %s", st.State, st.Reason)
}
for _, s := range seen {
if slices.Contains(s.roBinds, dir) || slices.Contains(s.binds, dir) {
t.Errorf("%s was given the node toolchains", s.command)
}
}
}
// 1891 (ADR 0533 §1, §2): the place bound is the Node build module's own: its manifest's cache directory joined with
// toolchain, and that cache directory lets other accounts enter it without listing it (0711).
func TestTheNodeToolchainsAreTheNodeBuildModulesCache(t *testing.T) {
if _, err := catalogue.ReadTreeManifest(catalogueRoot, filepath.Join(catalogueRoot, "modules", "mesh-build-node")); err != nil {
t.Fatalf("mesh-sdk's validator refuses mesh-build-node's manifest: %v", err)
}
raw, err := os.ReadFile(filepath.Join(catalogueRoot, "modules", "mesh-build-node", "module.json"))
if err != nil {
t.Fatal(err)
}
var man struct {
Resources []struct {
ID, Path, Mode string
} `json:"resources"`
}
if err := json.Unmarshal(raw, &man); err != nil {
t.Fatal(err)
}
found := false
for _, r := range man.Resources {
if r.ID != "cache" {
continue
}
found = true
if want := filepath.Join(r.Path, "toolchain"); NodeToolchains != want {
t.Errorf("NodeToolchains = %s, want %s (mesh-build-node's cache)", NodeToolchains, want)
}
mode, err := strconv.ParseUint(r.Mode, 8, 32)
if err != nil || mode&0o001 == 0 || mode&0o004 != 0 {
t.Errorf("mesh-build-node's cache has mode %q; want others to enter it and not list it (0711)", r.Mode)
}
}
if !found {
t.Fatal("mesh-build-node's manifest declares no cache directory")
}
}
@@ -0,0 +1,61 @@
package main
// A step's room to work and its workspace's end (novox/hq issue 1839, ADR 0509): every step is refused while the work
// or cache filesystem is under the floor, a workspace is held while a step runs on it, removed when its caller releases
// it, and swept unheld past buildstep.UnheldBound.
import (
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
// ReleaseTool removes a workspace whose caller's steps have ended.
const ReleaseTool = "build_go_release"
// floorOf is the floor the module keeps on node: build-agent's.
func floorOf(node string) buildstep.Floor {
return buildstep.Floor{Node: node, Bytes: buildstep.FreeSpaceFloor}
}
// room refuses while the work or cache filesystem is under the floor.
func (m *Module) room() error { return m.Floor.Check(m.Work, m.Cache) }
// spaces are the module's workspaces under Work.
func (m *Module) spaces() *buildstep.Workspaces {
m.spacesOnce.Do(func() { m.workspaces = &buildstep.Workspaces{Dir: m.Work} })
return m.workspaces
}
// holdArg holds the workspace a step's arguments name while the step runs; nothing when they name none.
func (m *Module) holdArg(args map[string]any) func() {
ws, _ := args["workspace"].(string)
if ws == "" {
return func() {}
}
return m.spaces().Hold(ws)
}
// releaseTool is build_go_release: no guard and no floor, since it only frees room.
func (m *Module) releaseTool() stdio.Tool {
return stdio.Tool{Name: ReleaseTool,
Description: "Remove a workspace whose steps have ended (fetch, test, build): its caller says so once its last step " +
"answered. A workspace already gone is no error; one a running step holds is refused. A workspace never released " +
"is removed two hours after its last use.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"workspace": map[string]any{"type": "string", "description": "the workspace id a fetch answered"},
}, "required": []string{"workspace"}},
Run: func(args map[string]any) (any, error) {
ws, _ := args["workspace"].(string)
if err := m.spaces().Release(ws); err != nil {
return nil, err
}
return map[string]any{"released": ws}, nil
}}
}
// sweepSaid is CleanWorkspaces' saying of a workspace a caller never released: a fault of the caller (ADR 0509).
func (m *Module) sweepSaid(removed []string) {
if len(removed) > 0 && m.Say != nil {
m.Say("removed %d workspaces no caller released within %s: %v", len(removed), buildstep.UnheldBound, removed)
}
}
@@ -0,0 +1,126 @@
package main
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/forgekey"
"git.novox.be/novox/mesh-sdk/go/health"
)
const underTheFloor = "refused: ace has 4 GiB free under the floor of 20 GiB"
func underFloor(node string) buildstep.Floor {
f := floorOf(node)
f.Free = func(string) (uint64, error) { return 4 << 30, nil }
return f
}
// 1839/AC4: the module as main makes it keeps build-agent's floor of 20 GiB.
func TestTheModuleKeepsTheFloorOf20GiB(t *testing.T) {
if f := floorOf("ace"); f.Bytes != 20<<30 || f.Node != "ace" {
t.Errorf("floorOf = %+v, want ace and 20 GiB", f)
}
}
// 1839/AC1: under the floor every step tool is refused, naming the node, what is free and the floor, no step runs, and
// the module is unhealthy saying so, so the builder chooses another node.
func TestUnderTheFloorEveryStepIsRefusedAndTheModuleIsUnhealthy(t *testing.T) {
states := newStates()
m := moduleWith(states, "go1.27.1")
m.Work, m.Cache, m.Floor = t.TempDir(), t.TempDir(), underFloor("ace")
ran := false
run := func(context.Context, map[string]any) (any, error) { ran = true; return "done", nil }
steps := []StepTool{{Name: "long_probe", Long: true, Run: run}, {Name: "short_probe", Run: run}}
for _, tool := range m.Tools(steps) {
switch tool.Name {
case "long_probe":
if _, err := tool.Run(map[string]any{}); err != nil {
t.Fatal(err)
}
if st := states.waitEnd(t); st.State != delivery.StepRefused || !strings.HasPrefix(st.Reason, underTheFloor) {
t.Errorf("the long step ended %s: %q, want refused: %q", st.State, st.Reason, underTheFloor)
}
case "short_probe":
if _, err := tool.Run(map[string]any{}); err == nil || !strings.HasPrefix(err.Error(), underTheFloor) {
t.Errorf("the short step answered %v, want %q", err, underTheFloor)
}
}
}
if ran {
t.Errorf("a step ran under the floor")
}
if a := m.Health(context.Background()); a.State != health.StateUnhealthy || !strings.HasPrefix(a.Why, underTheFloor) {
t.Errorf("health = %+v, want unhealthy: %q", a, underTheFloor)
}
}
func releaseTool(t *testing.T, m *Module) func(map[string]any) (any, error) {
t.Helper()
for _, tool := range m.Tools(m.StepTools()) {
if tool.Name == ReleaseTool {
return tool.Run
}
}
t.Fatalf("no tool %s", ReleaseTool)
return nil
}
// 1839/AC2: build_go_release removes the workspace at once, even under the floor; a second release is no error.
func TestReleaseRemovesTheWorkspaceAtOnce(t *testing.T) {
m, _ := fetchModule(t, &fakeStep{})
m.Floor = underFloor("ace")
ws := filepath.Join(m.Work, "abc1")
os.MkdirAll(filepath.Join(ws, "tree"), 0o700)
for i := 0; i < 2; i++ {
if _, err := releaseTool(t, m)(map[string]any{"workspace": "abc1"}); err != nil {
t.Fatalf("release %d: %v", i, err)
}
}
if _, err := os.Stat(ws); !os.IsNotExist(err) {
t.Errorf("the workspace is still there after its release")
}
}
// 1839/AC2: a fetch that fails leaves no workspace behind: nobody was told its id, so nobody could release it.
func TestAFailedFetchLeavesNoWorkspace(t *testing.T) {
f := &fakeStep{gomods: "go.mod\n", goRule: func(string) forgekey.Ran { return forgekey.Ran{Status: 1, Stderr: "boom"} }}
m, states := fetchModule(t, f)
if _, err := fetchTool(t, m)(fetchArgs); err != nil {
t.Fatal(err)
}
if st := states.waitEnd(t); st.State != delivery.StepFailed {
t.Fatalf("the step ended %s, want failed", st.State)
}
if entries, _ := os.ReadDir(m.Work); len(entries) != 0 {
t.Errorf("the failed fetch left %d entries in the work directory", len(entries))
}
}
// 1839/AC3: a workspace a running step holds is never swept, however old; an unheld one past the two-hour bound is.
func TestTheSweepKeepsAHeldWorkspaceAndRemovesAnUnheldOnePastTwoHours(t *testing.T) {
m := &Module{Work: t.TempDir()}
for _, name := range []string{"held1", "lost1"} {
dir := filepath.Join(m.Work, name)
os.MkdirAll(filepath.Join(dir, "tree"), 0o755)
}
done := m.spaces().Hold("held1")
defer done()
when := time.Now().Add(-3 * time.Hour)
for _, name := range []string{"held1", "lost1"} {
os.Chtimes(filepath.Join(m.Work, name), when, when)
}
m.CleanWorkspaces(time.Now())
if _, err := os.Stat(filepath.Join(m.Work, "held1")); err != nil {
t.Errorf("the held workspace was swept")
}
if _, err := os.Stat(filepath.Join(m.Work, "lost1")); !os.IsNotExist(err) {
t.Errorf("the unheld workspace three hours old was kept")
}
}
@@ -40,6 +40,7 @@ func (m *Module) Tools(steps []StepTool) []stdio.Tool {
tools = append(tools, t)
}
tools = append(tools, m.ReadTools()...)
tools = append(tools, m.releaseTool())
return append(tools,
stdio.Tool{Name: StatusTool,
Description: "What the Go build module is on this machine: the Go version its recipe builds with and the one installed, " +
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildgo
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023062828-6147a2045467
github.com/distribution/distribution/v3 v3.1.2
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
+4 -2
View File
@@ -1,5 +1,7 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876 h1:dtiIJzrX7ey1wmsdzapiFgCKDllCph7/vxfUp1ib/t8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9 h1:Ufmd2cYEXICQ817IGflccP9bexABE4WhIO49HPbaoSo=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023062828-6147a2045467 h1:i85+fFT0wZiTZ9CRVfMzWg1+Zr7oOLl3UH7hEaKcOIc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023062828-6147a2045467/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/alicebob/miniredis/v2 v2.35.0 h1:QwLphYqCEAo1eu1TqPRN2jgVMPBweeQcR21jeqDCONI=
github.com/alicebob/miniredis/v2 v2.35.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
+1
View File
@@ -17,6 +17,7 @@
"build_go_test",
"build_go_digest",
"build_go_build",
"build_go_release",
"build_go_step",
"build_go_steps",
"build_go_toolchains",
+6
View File
@@ -37,6 +37,12 @@ The Dockerfile's steps may use the network (ADR 0487 §6); nothing a step can re
- It refuses, naming why, an image whose build context is a whole repository (`context`) or whose `build.on` names
another module's artifact (`unsupportedImage` in `build.go`): those forms belong to the images build-agent builds
until it is removed at R4g.
- `build_image_release {workspace}` removes a workspace at once (no guard, no floor; one already gone is no error, one a
running step holds is refused); its caller calls it once its last step answered. A failed fetch removes its own
workspace, and one no caller released is removed two hours after its last use (ADR 0509).
- A fetch and a build are refused, `refused: <node> has N GiB free under the floor of 20 GiB (on <dir>)`, and the health
is unhealthy with that text, while the work directory's or the image store's filesystem has less than 20 GiB free
(issue 1839). A copy writes nothing to the disk and is not refused.
- `build_image_step`, `build_image_steps`, `build_image_status` and `build_image_health` read.
The registry's address is placed into `<home>/registry` as `${seat:mesh-artifact-store:reach}` (host:port): the provisioner's client fills a moved module's placement, and the artifact store has not moved, so a `${bound:artifact-store:...}` is refused at composition (issue 1703).
@@ -26,11 +26,6 @@ var (
// remote of a fetch is ssh://<it>/<owner>/<repository>.git.
const ForgeAlias = "mesh-forge"
// WorkspaceAge is how long a workspace is kept: a module removes the workspaces older than a day, each time it
// fetches and every round it checks itself. A workspace lies in the module's work directory, never in its home
// (ADR 0466 §1).
const WorkspaceAge = 24 * time.Hour
// FetchResult is what a passed fetch step says: the workspace it made and the commit in it, and, only when the call
// gave a base, the merge preview of ADR 0484 clause 3 (the trunk commit, the merge commit of the head onto it, the
// files in conflict, the paths the merge changes and the tree of each module directory). A head that does not
@@ -66,7 +61,9 @@ func (m *Module) fetchTool() StepTool {
}
}
func (m *Module) fetch(ctx context.Context, args map[string]any) (any, error) {
// fetch makes a workspace and fills it. The workspace is held while the fetch runs, and removed when the fetch fails,
// since nobody learns its id to release it (issue 1839).
func (m *Module) fetch(ctx context.Context, args map[string]any) (res any, err error) {
repository, _ := args["repository"].(string)
commit, _ := args["commit"].(string)
ref, _ := args["ref"].(string)
@@ -82,6 +79,19 @@ func (m *Module) fetch(ctx context.Context, args map[string]any) (any, error) {
}
m.CleanWorkspaces(m.clock())
id := buildstep.NewID()
done := m.spaces().Hold(id)
defer func() {
done()
if err != nil {
if rerr := m.spaces().Release(id); rerr != nil && m.Say != nil {
m.Say("the failed fetch's workspace %s: %v", id, rerr)
}
}
}()
return m.fetchInto(ctx, repository, commit, ref, base, id)
}
func (m *Module) fetchInto(ctx context.Context, repository, commit, ref, base, id string) (any, error) {
tree := filepath.Join(m.Work, id, "tree")
if err := os.MkdirAll(tree, 0o700); err != nil {
return nil, err
@@ -124,21 +134,10 @@ func (m *Module) merge(ctx context.Context, res *FetchResult, tree, remote, base
return err
}
// CleanWorkspaces removes the workspaces older than WorkspaceAge from the work directory.
// CleanWorkspaces removes the workspaces no step holds whose last use is older than buildstep.UnheldBound: the safety
// net for a caller that never released one (issue 1839, ADR 0509).
func (m *Module) CleanWorkspaces(now time.Time) {
entries, err := os.ReadDir(m.Work)
if err != nil {
return
}
for _, e := range entries {
info, err := e.Info()
if err != nil || !e.IsDir() || !workspaceOK.MatchString(e.Name()) || now.Sub(info.ModTime()) < WorkspaceAge {
continue
}
if err := os.RemoveAll(filepath.Join(m.Work, e.Name())); err != nil && m.Say != nil {
m.Say("the workspace %s could not be removed: %v", e.Name(), err)
}
}
m.sweepSaid(m.spaces().Sweep(now, buildstep.UnheldBound))
}
// clock is the time now: Now when a test set it, else the wall clock.
@@ -53,6 +53,11 @@ func realModule(t *testing.T) (*Module, *keptStates) {
m := &Module{Node: "ace", Account: u.Username, UID: u.Uid, Home: home, Work: t.TempDir(), Store: t.TempDir(),
Steps: states.putStep, Ctx: context.Background(), Say: t.Logf, Run: StepRunner(home),
FetchOptions: []buildstep.Option{buildstep.AllowFileRemote()}}
// buildah being installed is not enough: inside the check's sandbox rootless buildah cannot make its user
// namespace, so the module's own guard decides, as it does before every step (issue 1894).
if err := m.Check(context.Background()); err != nil {
t.Skipf("NOT RUN: this machine cannot build images where the check runs (%v); proven live on a build node", err)
}
return m, states
}
@@ -37,6 +37,7 @@ func main() {
Work: WorkDir,
Store: StoreDir,
}
m.Floor = floorOf(m.Node)
if u, err := user.Current(); err == nil {
m.Home, m.Account, m.UID = u.HomeDir, u.Username, u.Uid
}
@@ -4,8 +4,7 @@ import (
"context"
"fmt"
"net/http"
"os"
"path/filepath"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
@@ -47,6 +46,12 @@ type Module struct {
RegistryPause time.Duration
// Now is the clock; time.Now when nil.
Now func() time.Time
// Floor refuses every fetch and build while the work or the image store's filesystem has less free than it (issue
// 1839); main sets floorOf(node), and a zero Floor checks nothing.
Floor buildstep.Floor
spacesOnce sync.Once
workspaces *buildstep.Workspaces
}
// Look is what the module found out about its machine.
@@ -60,6 +65,11 @@ type Look struct {
// missing is a *buildstep.Refusal naming it, so a step ends refused and the health says the same.
func (m *Module) look(ctx context.Context) (Look, error) {
l := Look{Account: m.Account}
// The runtime directory comes first: buildah refuses before anything else when its XDG_RUNTIME_DIR is not there
// (issue 1884), so a look that made it only after `buildah --version` never made it at all.
if err := m.ensureRuntimeDir(); err != nil {
return l, &buildstep.Refusal{Text: err.Error()}
}
var err error
if l.Buildah, err = m.buildahVersion(ctx); err != nil {
return l, &buildstep.Refusal{Text: err.Error()}
@@ -67,11 +77,6 @@ func (m *Module) look(ctx context.Context) (Look, error) {
if l.UIDs, l.GIDs, err = m.ranges(); err != nil {
return l, &buildstep.Refusal{Text: err.Error(), Have: l.Buildah}
}
for _, d := range []string{m.Store, filepath.Join(m.Store, "xdg")} {
if err := os.MkdirAll(d, 0o700); err != nil {
return l, err
}
}
if err := m.userNamespace(ctx); err != nil {
return l, &buildstep.Refusal{Text: err.Error(), Have: l.Buildah}
}
@@ -106,6 +111,9 @@ func (m *Module) Health(ctx context.Context) health.Answer {
if err != nil {
return health.Unhealthy(err.Error())
}
if err := m.room(); err != nil {
return health.Unhealthy(err.Error())
}
return health.Healthy(fmt.Sprintf("buildah %s installed; %s holds %d subordinate uids and %d gids; a user namespace can be made", l.Buildah, l.Account, l.UIDs, l.GIDs))
}
@@ -128,6 +136,10 @@ func (m *Module) Long(tool string, run func(ctx context.Context, args map[string
if err := m.Check(ctx); err != nil {
return nil, err
}
if err := m.room(); err != nil {
return nil, err
}
defer m.holdArg(args)()
return run(ctx, args)
})
if err != nil {
@@ -144,6 +156,10 @@ func (m *Module) Short(run func(ctx context.Context, args map[string]any) (any,
if err := m.Check(m.Ctx); err != nil {
return nil, err
}
if err := m.room(); err != nil {
return nil, err
}
defer m.holdArg(args)()
return run(m.Ctx, args)
}
}
@@ -111,6 +111,25 @@ func (m *Module) storeFlags() []string {
// env is what buildah is given beyond the base environment: a runtime directory of its own inside the store, so
// nothing is read from the account's runtime directory, and no registry credential file anywhere (the module has none
// and REGISTRY_AUTH_FILE is never set).
//
// Every call makes the runtime directory first if it is not there (issue 1884): a directory removed after the last look
// is made again before buildah runs. A failure to make it is not lost: buildah then names the directory in its own
// error, and the next look refuses with it.
func (m *Module) env() []string {
return []string{"XDG_RUNTIME_DIR=" + filepath.Join(m.Store, "xdg")}
_ = m.ensureRuntimeDir()
return []string{"XDG_RUNTIME_DIR=" + m.runtimeDir()}
}
// runtimeDir is buildah's runtime directory, inside the store.
func (m *Module) runtimeDir() string { return filepath.Join(m.Store, "xdg") }
// ensureRuntimeDir makes the store and the runtime directory inside it, 0700 and under the module's own account (the
// process runs as it), when they are not there; it changes nothing that is.
func (m *Module) ensureRuntimeDir() error {
for _, d := range []string{m.Store, m.runtimeDir()} {
if err := os.MkdirAll(d, 0o700); err != nil {
return fmt.Errorf("buildah's runtime directory %s cannot be made: %w", m.runtimeDir(), err)
}
}
return nil
}
@@ -0,0 +1,61 @@
package main
// A step's room to work and its workspace's end (novox/hq issue 1839, ADR 0509): every fetch and build is refused while
// the work or image store's filesystem is under the floor, a workspace is held while a step runs on it, removed when its caller releases
// it, and swept unheld past buildstep.UnheldBound.
import (
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
// ReleaseTool removes a workspace whose caller's steps have ended.
const ReleaseTool = "build_image_release"
// floorOf is the floor the module keeps on node: build-agent's.
func floorOf(node string) buildstep.Floor {
return buildstep.Floor{Node: node, Bytes: buildstep.FreeSpaceFloor}
}
// room refuses while the work or the image store's filesystem is under the floor.
func (m *Module) room() error { return m.Floor.Check(m.Work, m.Store) }
// spaces are the module's workspaces under Work.
func (m *Module) spaces() *buildstep.Workspaces {
m.spacesOnce.Do(func() { m.workspaces = &buildstep.Workspaces{Dir: m.Work} })
return m.workspaces
}
// holdArg holds the workspace a step's arguments name while the step runs; nothing when they name none.
func (m *Module) holdArg(args map[string]any) func() {
ws, _ := args["workspace"].(string)
if ws == "" {
return func() {}
}
return m.spaces().Hold(ws)
}
// releaseTool is build_image_release: no guard and no floor, since it only frees room.
func (m *Module) releaseTool() stdio.Tool {
return stdio.Tool{Name: ReleaseTool,
Description: "Remove a workspace whose steps have ended (fetch, build): its caller says so once its last step " +
"answered. A workspace already gone is no error; one a running step holds is refused. A workspace never released " +
"is removed two hours after its last use.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"workspace": map[string]any{"type": "string", "description": "the workspace id a fetch answered"},
}, "required": []string{"workspace"}},
Run: func(args map[string]any) (any, error) {
ws, _ := args["workspace"].(string)
if err := m.spaces().Release(ws); err != nil {
return nil, err
}
return map[string]any{"released": ws}, nil
}}
}
// sweepSaid is CleanWorkspaces' saying of a workspace a caller never released: a fault of the caller (ADR 0509).
func (m *Module) sweepSaid(removed []string) {
if len(removed) > 0 && m.Say != nil {
m.Say("removed %d workspaces no caller released within %s: %v", len(removed), buildstep.UnheldBound, removed)
}
}
@@ -0,0 +1,117 @@
package main
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/forgekey"
"git.novox.be/novox/mesh-sdk/go/health"
)
const underTheFloor = "refused: ace has 4 GiB free under the floor of 20 GiB"
func underFloor(node string) buildstep.Floor {
f := floorOf(node)
f.Free = func(string) (uint64, error) { return 4 << 30, nil }
return f
}
// 1839/AC4: the module as main makes it keeps build-agent's floor of 20 GiB.
func TestTheModuleKeepsTheFloorOf20GiB(t *testing.T) {
if f := floorOf("ace"); f.Bytes != 20<<30 || f.Node != "ace" {
t.Errorf("floorOf = %+v, want ace and 20 GiB", f)
}
}
// 1839/AC1: under the floor a fetch and a build are refused, naming the node, what is free and the floor, nothing runs,
// and the module is unhealthy saying so, so the builder chooses another node.
func TestUnderTheFloorAFetchAndABuildAreRefusedAndTheModuleIsUnhealthy(t *testing.T) {
f := newFake(t)
states := newStates()
m := moduleWith(t, f, states)
m.Floor = underFloor("ace")
for name, args := range map[string]map[string]any{
FetchTool: {"repository": "novox/mesh-catalog", "commit": commitSHA},
BuildTool: {"workspace": "abc1", "module": "modules/m", "artifact": "image"},
} {
if _, err := tool(t, m, name)(args); err != nil {
t.Fatalf("%s: %v", name, err)
}
if st := states.waitEnd(t); st.State != delivery.StepRefused || !strings.HasPrefix(st.Reason, underTheFloor) {
t.Errorf("%s ended %s: %q, want refused: %q", name, st.State, st.Reason, underTheFloor)
}
}
for _, l := range f.lines() {
if strings.HasPrefix(l, "git ") || strings.Contains(l, " bud ") {
t.Errorf("a refused step ran %q", l)
}
}
if a := m.Health(context.Background()); a.State != health.StateUnhealthy || !strings.HasPrefix(a.Why, underTheFloor) {
t.Errorf("health = %+v, want unhealthy: %q", a, underTheFloor)
}
}
// 1839/AC2: build_image_release removes the workspace at once, even under the floor; a second release is no error.
func TestReleaseRemovesTheWorkspaceAtOnce(t *testing.T) {
m := moduleWith(t, newFake(t), newStates())
m.Floor = underFloor("ace")
ws := filepath.Join(m.Work, "abc1")
os.MkdirAll(filepath.Join(ws, "tree"), 0o700)
for i := 0; i < 2; i++ {
if _, err := tool(t, m, ReleaseTool)(map[string]any{"workspace": "abc1"}); err != nil {
t.Fatalf("release %d: %v", i, err)
}
}
if _, err := os.Stat(ws); !os.IsNotExist(err) {
t.Errorf("the workspace is still there after its release")
}
}
// 1839/AC2: a fetch that fails leaves no workspace behind: nobody was told its id, so nobody could release it.
func TestAFailedFetchLeavesNoWorkspace(t *testing.T) {
f := newFake(t)
states := newStates()
m := moduleWith(t, f, states)
m.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
if name == "git" && strings.Contains(strings.Join(args, " "), " fetch ") {
return forgekey.Ran{Status: 128, Stderr: "fatal: unreachable"}
}
return f.run(ctx, env, name, args...)
}
if _, err := tool(t, m, FetchTool)(map[string]any{"repository": "novox/mesh-catalog", "commit": commitSHA}); err != nil {
t.Fatal(err)
}
if st := states.waitEnd(t); st.State != delivery.StepFailed {
t.Fatalf("the step ended %s (%s), want failed", st.State, st.Reason)
}
if entries, _ := os.ReadDir(m.Work); len(entries) != 0 {
t.Errorf("the failed fetch left %d entries in the work directory", len(entries))
}
}
// 1839/AC3: a workspace a running step holds is never swept, however old; an unheld one past the two-hour bound is.
func TestTheSweepKeepsAHeldWorkspaceAndRemovesAnUnheldOnePastTwoHours(t *testing.T) {
m := &Module{Work: t.TempDir()}
for _, name := range []string{"held1", "lost1"} {
os.MkdirAll(filepath.Join(m.Work, name, "tree"), 0o755)
}
done := m.spaces().Hold("held1")
defer done()
when := time.Now().Add(-3 * time.Hour)
for _, name := range []string{"held1", "lost1"} {
os.Chtimes(filepath.Join(m.Work, name), when, when)
}
m.CleanWorkspaces(time.Now())
if _, err := os.Stat(filepath.Join(m.Work, "held1")); err != nil {
t.Errorf("the held workspace was swept")
}
if _, err := os.Stat(filepath.Join(m.Work, "lost1")); !os.IsNotExist(err) {
t.Errorf("the unheld workspace three hours old was kept")
}
}
@@ -0,0 +1,83 @@
package main
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/forgekey"
"git.novox.be/novox/mesh-sdk/go/health"
)
// buildahLike refuses as buildah does when the XDG_RUNTIME_DIR it is given is not there (seen on ace, issue 1884):
// "lstat <dir>: no such file or directory", before it does anything else.
func buildahLike(t *testing.T, run forgekey.Runner, seen *[]string) forgekey.Runner {
t.Helper()
return func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
if name == "buildah" {
for _, e := range env {
dir, ok := strings.CutPrefix(e, "XDG_RUNTIME_DIR=")
if !ok {
continue
}
if _, err := os.Lstat(dir); err != nil {
*seen = append(*seen, "missing")
return forgekey.Ran{Status: 125, Stderr: "Error: lstat " + dir + ": no such file or directory\n"}
}
*seen = append(*seen, "present")
}
}
return run(ctx, env, name, args...)
}
}
// 1884/AC1: given a store directory that exists without its runtime directory, when the health is looked at, then the
// module makes the runtime directory (0700) before it runs buildah, and the health is healthy.
func TestHealthMakesTheRuntimeDirectoryBeforeBuildah(t *testing.T) {
f := newFake(t)
m := moduleWith(t, f, newStates())
var seen []string
m.Run = buildahLike(t, m.Run, &seen)
xdg := filepath.Join(m.Store, "xdg")
if _, err := os.Stat(xdg); !os.IsNotExist(err) {
t.Fatalf("the test needs a store without its runtime directory: %v", err)
}
a := m.Health(context.Background())
if a.State != health.StateHealthy || !a.Healthy {
t.Fatalf("health = %+v, want healthy", a)
}
st, err := os.Stat(xdg)
if err != nil {
t.Fatalf("the runtime directory was not made: %v", err)
}
if st.Mode().Perm() != 0o700 || !st.IsDir() {
t.Errorf("the runtime directory is %v, want a directory 0700", st.Mode())
}
for _, s := range seen {
if s != "present" {
t.Fatalf("buildah ran without its runtime directory: %v", seen)
}
}
}
// 1884/AC2: given a module whose runtime directory was removed after its last look, when a build step runs buildah, then
// the runtime directory is made again before buildah runs.
func TestEveryBuildahRunMakesTheRuntimeDirectoryAgain(t *testing.T) {
f := newFake(t)
m := moduleWith(t, f, newStates())
var seen []string
m.Run = buildahLike(t, m.Run, &seen)
if err := m.Check(context.Background()); err != nil {
t.Fatal(err)
}
if err := os.RemoveAll(filepath.Join(m.Store, "xdg")); err != nil {
t.Fatal(err)
}
seen = nil
m.remove("localhost/x:y")
if len(seen) != 1 || seen[0] != "present" {
t.Fatalf("buildah's runs saw their runtime directory %v, want [present]", seen)
}
}
@@ -44,6 +44,7 @@ func (m *Module) Tools(steps []StepTool) []stdio.Tool {
tools = append(tools, t)
}
tools = append(tools, m.ReadTools()...)
tools = append(tools, m.releaseTool())
return append(tools,
stdio.Tool{Name: StatusTool,
Description: "What the image build module is on this machine: the buildah installed, the account's subordinate " +
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildimage
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023062828-6147a2045467
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+4 -2
View File
@@ -1,5 +1,7 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876 h1:dtiIJzrX7ey1wmsdzapiFgCKDllCph7/vxfUp1ib/t8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9 h1:Ufmd2cYEXICQ817IGflccP9bexABE4WhIO49HPbaoSo=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023062828-6147a2045467 h1:i85+fFT0wZiTZ9CRVfMzWg1+Zr7oOLl3UH7hEaKcOIc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023062828-6147a2045467/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
+1
View File
@@ -15,6 +15,7 @@
"build_image_fetch",
"build_image_build",
"build_image_copy",
"build_image_release",
"build_image_step",
"build_image_steps",
"build_image_status",
+5 -1
View File
@@ -57,7 +57,11 @@ toolchain install: a long step answers at once with its id and ends on `<node>.<
removed). The mesh's registry reaches `npm ci` through a private config file (`--userconfig`), so
other `@novox/` packages install. A module with packages and no lock does not fail the fetch: its result lists it
under `refused` with the reason, and a build of that bundle fails naming it (issue 1459). Every command runs through `forgekey.ExecRunnerClean(0)` with PATH, HOME, the locale and its own
words, nothing of this process's environment. Workspaces older than a day are removed at each fetch.
words, nothing of this process's environment. A failed fetch removes its own workspace; `build_node_release {workspace}` removes one at once (no guard, no
floor; its caller calls it once its last step answered), and one no caller released is removed two hours after its
last use (ADR 0509), never while a step holds it. Every step is refused, `refused: <node> has N GiB free under the
floor of 20 GiB (on <dir>)`, and the health is unhealthy with that text, while the work or the toolchain cache
filesystem has less than 20 GiB free (issue 1839).
The arguments are mesh-sdk's `buildstep.FetchRequest` (ADR 0499 §4): with `dir` and `paths` (from the builder, never
from a manifest) the workspace is narrowed to those paths, the manifest and the lock files before anything else. A
member is built against the SDK version its own lock names (the lead's decision L10 of R4e, ADR 0500 §5-§6): the
@@ -3,18 +3,25 @@
package main
import (
"bytes"
"context"
"fmt"
"io/fs"
"os"
"os/exec"
"path/filepath"
"slices"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// realMachine installs the real toolchain from the module's own lock (npm ci --ignore-scripts, from the mesh's
// registry for @novox, which reads without a credential) and makes two workspaces holding one fixture bundle.
// realMachine gives the tests the real toolchain of the module's own lock and makes two workspaces holding one fixture
// bundle. The toolchain is the one this node's installed Node build module keeps (ADR 0533 §1): the integration level
// is given its toolchains read-only and no network, so it is copied into the test's own cache from there. Only where
// no toolchain is installed (a desk) is it installed with `npm ci --ignore-scripts` from the mesh's registry, as the
// module does, which needs the network.
func realMachine(t *testing.T) (*Module, string) {
t.Helper()
for _, p := range []string{"node", "npm", "bwrap"} { // bwrap: tsc, esbuild and the scripts run shut off (1309)
@@ -40,8 +47,11 @@ func realMachine(t *testing.T) (*Module, string) {
return Registry{URL: "https://git.novox.be/api/packages/novox/npm/"}, nil
}}}
withStore(t, m)
if from := installedToolchain(t, filepath.Join(CacheDir, "toolchain"), filepath.Join(home, "toolchain", "package-lock.json")); from != "" {
copyToolchain(t, from, m.Toolchain.Paths.Dir(m.Recipe.Version()))
}
if err := m.Toolchain.Ensure(context.Background(), m.Recipe.Version()); err != nil {
t.Fatal(err)
t.Fatalf("no toolchain of this lock is installed under %s, and installing it from the registry failed: %v", filepath.Join(CacheDir, "toolchain"), err)
}
for _, id := range []string{"w1", "w2"} {
for p, body := range map[string]string{
@@ -115,3 +125,74 @@ func TestAPackageWhoseTestsImportItsBuildPassesTheUnitLevel(t *testing.T) {
t.Errorf("result = %+v", r)
}
}
// installedToolchain is the toolchain directory under toolchains that this node's Node build module installed: the
// one installed from the lock at lockPath, byte for byte, or, when the change carries a lock no delivered release has
// installed yet, the one installed last, said in the test's log (ADR 0533 §3: a new lock is proven by the module that
// installs it on its first node). "" when none is there or the directory cannot be read: no Node build module here.
func installedToolchain(t *testing.T, toolchains, lockPath string) string {
t.Helper()
want, err := os.ReadFile(lockPath)
if err != nil {
t.Fatal(err)
}
entries, err := os.ReadDir(toolchains)
if err != nil {
return ""
}
last, lastAt := "", time.Time{}
for _, e := range entries {
dir := filepath.Join(toolchains, e.Name())
mark, err := os.Stat(filepath.Join(dir, installedMark))
if err != nil {
continue
}
got, err := os.ReadFile(filepath.Join(dir, "package-lock.json"))
if err != nil {
continue
}
if bytes.Equal(got, want) {
t.Logf("the toolchain is this node's installed %s, of this change's lock", dir)
return dir
}
if mark.ModTime().After(lastAt) {
last, lastAt = dir, mark.ModTime()
}
}
if last != "" {
t.Logf("NOT THIS LOCK: no toolchain of this change's lock is installed here; the tests run with the one installed last, %s", last)
}
return last
}
// copyToolchain copies the installed toolchain from into to whole, files and folders with their modes, so the test's
// steps run behind a toolchain of their own and the installed one is only read.
func copyToolchain(t *testing.T, from, to string) {
t.Helper()
err := filepath.WalkDir(from, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
rel, _ := filepath.Rel(from, p)
dst := filepath.Join(to, rel)
info, err := d.Info()
if err != nil {
return err
}
switch {
case d.IsDir():
return os.MkdirAll(dst, 0o755)
case info.Mode().IsRegular():
raw, err := os.ReadFile(p)
if err != nil {
return err
}
return os.WriteFile(dst, raw, info.Mode().Perm()|0o600)
default:
return fmt.Errorf("%s is neither a file nor a folder", p)
}
})
if err != nil {
t.Fatalf("copying the installed toolchain %s: %v", from, err)
}
}
@@ -33,9 +33,6 @@ const ForgeAlias = "mesh-forge"
// sdkPackage is the package the toolchain carries and a module's own dependencies never supply.
const sdkPackage = "@novox/mesh-sdk"
// WorkspaceAge is how long a workspace is kept.
const WorkspaceAge = 24 * time.Hour
var commitOK = regexp.MustCompile(`^[0-9a-f]{40}$`)
var repositoryOK = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]*/[A-Za-z0-9][A-Za-z0-9._-]*$`)
@@ -105,20 +102,36 @@ func (m *Module) fetchTool() StepTool {
}
}
func (m *Module) fetch(ctx context.Context, args map[string]any) (any, error) {
// fetch makes a workspace and fills it. The workspace is held while the fetch runs, and removed when the fetch fails,
// since nobody learns its id to release it (issue 1839).
func (m *Module) fetch(ctx context.Context, args map[string]any) (res any, err error) {
// What the builder and the checker ask, in mesh-sdk's one form (ADR 0499 §4): with the module's directory and its
// named paths, the workspace is narrowed to them (buildstep.Narrow).
req, err := buildstep.ParseFetchRequest(args)
if err != nil {
return nil, err
}
repository, commit, ref, base := req.Repository, req.Commit, req.Ref, req.Base
group, err := groupPackageArg(args["package"])
if err != nil {
return nil, err
}
m.CleanWorkspaces(m.now())
id := buildstep.NewID()
done := m.spaces().Hold(id)
defer func() {
done()
if err != nil {
if rerr := m.spaces().Release(id); rerr != nil && m.Say != nil {
m.Say("the failed fetch's workspace %s: %v", id, rerr)
}
}
}()
return m.fetchInto(ctx, req, group, id)
}
func (m *Module) fetchInto(ctx context.Context, req buildstep.FetchRequest, group *groupPackage, id string) (any, error) {
repository, commit, ref, base := req.Repository, req.Commit, req.Ref, req.Base
var err error
root := filepath.Join(m.Work, id)
tree := filepath.Join(root, "tree")
if err := os.MkdirAll(tree, 0o700); err != nil {
@@ -354,19 +367,8 @@ func removeSDKCopies(nm string) error {
return nil
}
// CleanWorkspaces removes the workspaces older than WorkspaceAge from the work directory.
// CleanWorkspaces removes the workspaces no step holds whose last use is older than buildstep.UnheldBound: the safety
// net for a caller that never released one (issue 1839, ADR 0509).
func (m *Module) CleanWorkspaces(now time.Time) {
entries, err := os.ReadDir(m.Work)
if err != nil {
return
}
for _, e := range entries {
info, err := e.Info()
if err != nil || !e.IsDir() || !workspaceOK.MatchString(e.Name()) || now.Sub(info.ModTime()) < WorkspaceAge {
continue
}
if err := os.RemoveAll(filepath.Join(m.Work, e.Name())); err != nil && m.Say != nil {
m.Say("the workspace %s could not be removed: %v", e.Name(), err)
}
}
m.sweepSaid(m.spaces().Sweep(now, buildstep.UnheldBound))
}
@@ -54,6 +54,7 @@ func main() {
Bubblewrap: ProbeBubblewrap,
RegistryFile: filepath.Join(home, "registry"),
}
m.Floor = floorOf(m.Node)
// Outside a launch by the runtime — a build, a check — there is no node and no state to write.
if m.Node != "" {
go m.Start(ctx)
@@ -59,8 +59,14 @@ type Module struct {
// unhealthy, while it is missing or older than BubblewrapLeast. Nil probes nothing.
Bubblewrap func(ctx context.Context) (string, error)
mu sync.Mutex // guards Recipe
install sync.Mutex // one install at a time
// Floor refuses every step while the work or toolchain cache filesystem has less free than it (issue 1839); main
// sets floorOf(node), and a zero Floor checks nothing.
Floor buildstep.Floor
spacesOnce sync.Once
workspaces *buildstep.Workspaces
mu sync.Mutex // guards Recipe
install sync.Mutex // one install at a time
}
func (m *Module) recipe() Recipe {
@@ -122,6 +128,7 @@ func (m *Module) Start(ctx context.Context) {
if err := m.Prepare(ctx); err != nil && m.Say != nil {
m.Say("the toolchain: %v", err)
}
m.CleanWorkspaces(m.now())
select {
case <-ctx.Done():
return
@@ -138,6 +145,9 @@ func (m *Module) Health(ctx context.Context) health.Answer {
if err != nil {
return health.Unhealthy(err.Error())
}
if err := m.room(); err != nil {
return health.Unhealthy(err.Error())
}
return health.Healthy(fmt.Sprintf("node %s installed, the version the module builds with (recipe %s)", installed, m.recipe().Version()))
}
@@ -150,6 +160,10 @@ func (m *Module) Long(tool string, run func(ctx context.Context, args map[string
if err := m.Prepare(ctx); err != nil {
return nil, err
}
if err := m.room(); err != nil {
return nil, err
}
defer m.holdArg(args)()
return run(ctx, args)
})
if err != nil {
@@ -166,6 +180,10 @@ func (m *Module) Short(run func(ctx context.Context, args map[string]any) (any,
if err := m.Prepare(m.Ctx); err != nil {
return nil, err
}
if err := m.room(); err != nil {
return nil, err
}
defer m.holdArg(args)()
return run(m.Ctx, args)
}
}
@@ -117,7 +117,7 @@ func TestANodeThatCannotBeRunRefusesEveryStepAndHealthIsUnhealthy(t *testing.T)
{Name: "short_probe", Run: run},
}, (&Module{}).StepTools()...)
for _, tool := range m.Tools(steps) {
if tool.Name == StatusTool || tool.Name == HealthTool || tool.Name == StepReadTool || tool.Name == StepsReadTool || tool.Name == ToolchainsReadTool {
if tool.Name == StatusTool || tool.Name == HealthTool || tool.Name == StepReadTool || tool.Name == StepsReadTool || tool.Name == ToolchainsReadTool || tool.Name == ReleaseTool {
continue
}
var reason string
@@ -0,0 +1,67 @@
package main
// A step's room to work and its workspace's end (novox/hq issue 1839, ADR 0509): every step is refused while the work
// or toolchain cache filesystem is under the floor, a workspace is held while a step runs on it, removed when its caller releases
// it, and swept unheld past buildstep.UnheldBound.
import (
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
// ReleaseTool removes a workspace whose caller's steps have ended.
const ReleaseTool = "build_node_release"
// floorOf is the floor the module keeps on node: build-agent's.
func floorOf(node string) buildstep.Floor {
return buildstep.Floor{Node: node, Bytes: buildstep.FreeSpaceFloor}
}
// room refuses while the work or toolchain cache filesystem is under the floor.
func (m *Module) room() error {
paths := []string{m.Work}
if m.Toolchain != nil {
paths = append(paths, m.Toolchain.Paths.Cache)
}
return m.Floor.Check(paths...)
}
// spaces are the module's workspaces under Work.
func (m *Module) spaces() *buildstep.Workspaces {
m.spacesOnce.Do(func() { m.workspaces = &buildstep.Workspaces{Dir: m.Work} })
return m.workspaces
}
// holdArg holds the workspace a step's arguments name while the step runs; nothing when they name none.
func (m *Module) holdArg(args map[string]any) func() {
ws, _ := args["workspace"].(string)
if ws == "" {
return func() {}
}
return m.spaces().Hold(ws)
}
// releaseTool is build_node_release: no guard and no floor, since it only frees room.
func (m *Module) releaseTool() stdio.Tool {
return stdio.Tool{Name: ReleaseTool,
Description: "Remove a workspace whose steps have ended (fetch, test, build): its caller says so once its last step " +
"answered. A workspace already gone is no error; one a running step holds is refused. A workspace never released " +
"is removed two hours after its last use.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"workspace": map[string]any{"type": "string", "description": "the workspace id a fetch answered"},
}, "required": []string{"workspace"}},
Run: func(args map[string]any) (any, error) {
ws, _ := args["workspace"].(string)
if err := m.spaces().Release(ws); err != nil {
return nil, err
}
return map[string]any{"released": ws}, nil
}}
}
// sweepSaid is CleanWorkspaces' saying of a workspace a caller never released: a fault of the caller (ADR 0509).
func (m *Module) sweepSaid(removed []string) {
if len(removed) > 0 && m.Say != nil {
m.Say("removed %d workspaces no caller released within %s: %v", len(removed), buildstep.UnheldBound, removed)
}
}
@@ -0,0 +1,123 @@
package main
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buildstep"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/forgekey"
"git.novox.be/novox/mesh-sdk/go/health"
)
const underTheFloor = "refused: ace has 4 GiB free under the floor of 20 GiB"
func underFloor(node string) buildstep.Floor {
f := floorOf(node)
f.Free = func(string) (uint64, error) { return 4 << 30, nil }
return f
}
// 1839/AC4: the module as main makes it keeps build-agent's floor of 20 GiB.
func TestTheModuleKeepsTheFloorOf20GiB(t *testing.T) {
if f := floorOf("ace"); f.Bytes != 20<<30 || f.Node != "ace" {
t.Errorf("floorOf = %+v, want ace and 20 GiB", f)
}
}
// 1839/AC1: under the floor every step tool is refused, naming the node, what is free and the floor, no step runs, and
// the module is unhealthy saying so, so the builder chooses another node.
func TestUnderTheFloorEveryStepIsRefusedAndTheModuleIsUnhealthy(t *testing.T) {
states := newStates()
m := moduleWith(states, "v26.10.0")
m.Work, m.Floor = t.TempDir(), underFloor("ace")
ran := false
run := func(context.Context, map[string]any) (any, error) { ran = true; return "done", nil }
steps := []StepTool{{Name: "long_probe", Long: true, Run: run}, {Name: "short_probe", Run: run}}
for _, tool := range m.Tools(steps) {
switch tool.Name {
case "long_probe":
if _, err := tool.Run(map[string]any{}); err != nil {
t.Fatal(err)
}
if st := states.waitEnd(t); st.State != delivery.StepRefused || !strings.HasPrefix(st.Reason, underTheFloor) {
t.Errorf("the long step ended %s: %q, want refused: %q", st.State, st.Reason, underTheFloor)
}
case "short_probe":
if _, err := tool.Run(map[string]any{}); err == nil || !strings.HasPrefix(err.Error(), underTheFloor) {
t.Errorf("the short step answered %v, want %q", err, underTheFloor)
}
}
}
if ran {
t.Errorf("a step ran under the floor")
}
if a := m.Health(context.Background()); a.State != health.StateUnhealthy || !strings.HasPrefix(a.Why, underTheFloor) {
t.Errorf("health = %+v, want unhealthy: %q", a, underTheFloor)
}
}
func releaseTool(t *testing.T, m *Module) func(map[string]any) (any, error) {
t.Helper()
for _, tool := range m.Tools(m.StepTools()) {
if tool.Name == ReleaseTool {
return tool.Run
}
}
t.Fatalf("no tool %s", ReleaseTool)
return nil
}
// 1839/AC2: build_node_release removes the workspace at once, even under the floor; a second release is no error.
func TestReleaseRemovesTheWorkspaceAtOnce(t *testing.T) {
m, _, work := newMachine(t)
m.Floor = underFloor("ace")
ws := filepath.Join(work, "abc1")
os.MkdirAll(filepath.Join(ws, "tree"), 0o700)
for i := 0; i < 2; i++ {
if _, err := releaseTool(t, m)(map[string]any{"workspace": "abc1"}); err != nil {
t.Fatalf("release %d: %v", i, err)
}
}
if _, err := os.Stat(ws); !os.IsNotExist(err) {
t.Errorf("the workspace is still there after its release")
}
}
// 1839/AC2: a fetch that fails leaves no workspace behind: nobody was told its id, so nobody could release it.
func TestAFailedFetchLeavesNoWorkspace(t *testing.T) {
m, f := fetchMachine(t, typeScriptModule(`{}`), nil)
f.answer["git fetch"] = forgekey.Ran{Status: 128, Stderr: "fatal: unreachable"}
if _, err := m.fetch(context.Background(), fetchArgs); err == nil {
t.Fatal("the fetch passed, want it failed")
}
if entries, _ := os.ReadDir(m.Work); len(entries) != 0 {
t.Errorf("the failed fetch left %d entries in the work directory", len(entries))
}
}
// 1839/AC3: a workspace a running step holds is never swept, however old; an unheld one past the two-hour bound is,
// also at the module's own round and not only at a fetch.
func TestTheSweepKeepsAHeldWorkspaceAndRemovesAnUnheldOnePastTwoHours(t *testing.T) {
m := &Module{Work: t.TempDir()}
for _, name := range []string{"held1", "lost1"} {
os.MkdirAll(filepath.Join(m.Work, name, "tree"), 0o755)
}
done := m.spaces().Hold("held1")
defer done()
when := time.Now().Add(-3 * time.Hour)
for _, name := range []string{"held1", "lost1"} {
os.Chtimes(filepath.Join(m.Work, name), when, when)
}
m.CleanWorkspaces(time.Now())
if _, err := os.Stat(filepath.Join(m.Work, "held1")); err != nil {
t.Errorf("the held workspace was swept")
}
if _, err := os.Stat(filepath.Join(m.Work, "lost1")); !os.IsNotExist(err) {
t.Errorf("the unheld workspace three hours old was kept")
}
}
@@ -10,6 +10,7 @@ package main
import (
"context"
"fmt"
"io/fs"
"os"
"path/filepath"
"strings"
@@ -53,14 +54,17 @@ func (t Toolchain) Present(version string) bool {
// are copied in, and `npm ci --ignore-scripts` runs once, the mesh's registry named for the @novox scope. A directory
// an earlier install left unfinished is removed first.
func (t Toolchain) Ensure(ctx context.Context, version string) error {
if t.Present(version) {
return nil
}
dir := t.Paths.Dir(version)
if t.Present(version) {
if opened(dir) {
return nil
}
return t.open(dir)
}
if err := os.RemoveAll(dir); err != nil {
return err
}
if err := os.MkdirAll(dir, 0o700); err != nil {
if err := os.MkdirAll(dir, 0o755); err != nil {
return err
}
for _, name := range []string{"package.json", "package-lock.json"} {
@@ -76,18 +80,95 @@ func (t Toolchain) Ensure(ctx context.Context, version string) error {
if err != nil {
return err
}
npmrc := filepath.Join(dir, ".npmrc-install")
if err := os.WriteFile(npmrc, []byte(reg.Npmrc()), 0o600); err != nil {
// The credential is written in the account's own home (0700), never beneath the cache, whose toolchains are
// opened to every account (ADR 0533 §2), and removed as soon as npm has read it.
f, err := os.CreateTemp(t.Paths.Home, ".npmrc-install-")
if err != nil {
return err
}
npmrc := f.Name()
defer os.Remove(npmrc)
_, err = f.WriteString(reg.Npmrc())
if cerr := f.Close(); err == nil {
err = cerr
}
if err != nil {
return err
}
ran := t.Run(ctx, []string{"PATH=" + pathOrDefault(), "HOME=" + t.Paths.Home, "LC_ALL=C", "npm_config_cache=" + filepath.Join(t.Paths.Cache, "npm")},
"npm", "ci", "--ignore-scripts", "--no-bin-links", "--no-audit", "--no-fund", "--prefix", dir, "--userconfig", npmrc)
os.Remove(npmrc)
if ran.Err != "" || ran.Status != 0 {
return fmt.Errorf("npm ci --ignore-scripts for the toolchain %s failed (status %d %s): %s",
version, ran.Status, ran.Err, tail(ran.Stderr))
}
return os.WriteFile(filepath.Join(dir, installedMark), []byte(version+"\n"), 0o600)
if err := os.WriteFile(filepath.Join(dir, installedMark), []byte(version+"\n"), 0o600); err != nil {
return err
}
return t.open(dir)
}
// refuseNpmConfig refuses a toolchain folder that holds an install's npm config file (one a dead install of an older release left with
// the registry credential in it), naming it, before anything of the folder is opened.
func refuseNpmConfig(dir string) error {
return filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if !d.IsDir() && strings.HasPrefix(d.Name(), ".npmrc-install") {
return fmt.Errorf("the toolchain %s holds the npm config file %s, which may hold the registry credential, "+
"so it is not opened to other accounts; remove it", dir, p)
}
return nil
})
}
// opened says whether the toolchain directory dir was opened to other accounts: its installed mark, opened last, is
// readable by others.
func opened(dir string) bool {
st, err := os.Stat(filepath.Join(dir, installedMark))
return err == nil && st.Mode().Perm()&0o004 != 0
}
// open lets every account read the toolchain directory dir and nobody but its owner write it (ADR 0533 §2): the
// toolchains folder and every folder beneath dir 0755, every file 0644 and every program 0755, whatever umask npm ran
// under; the installed mark last, so a toolchain opened only in part is opened again. The other build module's
// integration level reads it through the cache directory, which others may enter and not list (0711); the npm cache
// beside it stays the owner's.
func (t Toolchain) open(dir string) error {
if err := refuseNpmConfig(dir); err != nil {
return err
}
if err := os.Chmod(filepath.Join(t.Paths.Cache, "toolchain"), 0o755); err != nil {
return err
}
mark := filepath.Join(dir, installedMark)
err := filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if p == mark {
return nil
}
info, err := d.Info()
if err != nil {
return err
}
switch mode := info.Mode(); {
case mode.IsDir():
return os.Chmod(p, 0o755)
case mode.IsRegular() && mode.Perm()&0o100 != 0:
return os.Chmod(p, 0o755)
case mode.IsRegular():
return os.Chmod(p, 0o644)
default:
return fmt.Errorf("%s is neither a file nor a folder, and a toolchain holds only those", p)
}
})
if err != nil {
return fmt.Errorf("the toolchain %s cannot be opened to the integration level: %w", dir, err)
}
return os.Chmod(mark, 0o644)
}
func tail(s string) string {
@@ -0,0 +1,166 @@
package main
import (
"context"
"io/fs"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// npmAsUmask077 installs as npm does under a umask of 077: node_modules with a package whose folders are 0700, its
// file 0600 and its program 0700.
func npmAsUmask077(f *fakeNpm) forgekey.Runner {
return func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
ran := f.run(ctx, env, name, args...)
for i, a := range args {
if a == "--prefix" {
pkg := filepath.Join(args[i+1], "node_modules", "typescript", "bin")
os.MkdirAll(pkg, 0o700)
os.WriteFile(filepath.Join(args[i+1], "node_modules", "typescript", "package.json"), []byte("{}"), 0o600)
os.WriteFile(filepath.Join(pkg, "tsc"), []byte("#!/bin/sh\n"), 0o700)
}
}
return ran
}
}
// checkOpen fails unless every folder from the cache's toolchain down lets others enter and read it and every file lets
// others read it, a program others run, and nothing lets anyone but the owner write.
func checkOpen(t *testing.T, tc Toolchain) {
t.Helper()
root := filepath.Join(tc.Paths.Cache, "toolchain")
err := filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
info, err := d.Info()
if err != nil {
return err
}
mode := info.Mode().Perm()
switch {
case mode&0o022 != 0:
t.Errorf("%s is %o: writable by others than its owner", p, mode)
case d.IsDir() && mode&0o005 != 0o005:
t.Errorf("%s is %o: others may not enter and read it", p, mode)
case !d.IsDir() && mode&0o004 == 0:
t.Errorf("%s is %o: others may not read it", p, mode)
case !d.IsDir() && mode&0o100 != 0 && mode&0o001 == 0:
t.Errorf("%s is %o: a program others may not run", p, mode)
}
return nil
})
if err != nil {
t.Fatal(err)
}
}
// 1891 (ADR 0533 §2): a toolchain the module installs can be read by the other build module's integration level:
// its folders 0755 and its files 0644 (programs 0755), whatever umask npm ran under, and writable by its owner alone.
func TestAnInstalledToolchainIsReadableByOthersAndWritableByItsOwnerAlone(t *testing.T) {
fake := &fakeNpm{}
tc := toolchainIn(t, fake)
tc.Run = npmAsUmask077(fake)
npm := filepath.Join(tc.Paths.Cache, "npm")
if err := os.MkdirAll(npm, 0o700); err != nil {
t.Fatal(err)
}
if err := tc.Ensure(context.Background(), "v1"); err != nil {
t.Fatal(err)
}
checkOpen(t, tc)
if st, err := os.Stat(npm); err != nil || st.Mode().Perm() != 0o700 {
t.Errorf("the npm download cache beside the toolchains is %v (%v), want 0700: only the toolchains are opened", st.Mode().Perm(), err)
}
}
// 1891 (ADR 0533 §2): a toolchain installed before this release, shut to its owner, is opened by the next Ensure
// without installing it again.
func TestAToolchainInstalledShutIsOpenedWithoutAnInstall(t *testing.T) {
fake := &fakeNpm{}
tc := toolchainIn(t, fake)
dir := tc.Paths.Dir("v1")
os.MkdirAll(filepath.Join(dir, "node_modules", "esbuild", "bin"), 0o700)
os.WriteFile(filepath.Join(dir, "node_modules", "esbuild", "bin", "esbuild"), []byte("x"), 0o700)
os.WriteFile(filepath.Join(dir, "package-lock.json"), []byte("{}"), 0o600)
os.WriteFile(filepath.Join(dir, installedMark), []byte("v1\n"), 0o600)
os.Chmod(filepath.Join(tc.Paths.Cache, "toolchain"), 0o700)
if err := tc.Ensure(context.Background(), "v1"); err != nil {
t.Fatal(err)
}
if len(fake.calls) != 0 {
t.Errorf("npm ran %d times for a toolchain already there", len(fake.calls))
}
checkOpen(t, tc)
}
// 1891 (ADR 0533 §2): the cache directory lets other accounts enter it and not list it, so the toolchains beneath it
// can be reached by their path and the npm cache beside them (0700) cannot.
func TestTheCacheDirectoryIsEnteredByOthersNotListed(t *testing.T) {
r := resource(t, readManifest(t), "cache")
if r["mode"] != "0711" {
t.Errorf("the cache directory's mode is %v, want 0711", r["mode"])
}
}
// 1891 review (ADR 0533 §2): the registry credential npm is told is never written beneath the cache, so opening a
// toolchain can never open it: npm reads it from a file in the account's own home, readable by its owner alone, and
// the file is gone when Ensure returns.
func TestTheInstallCredentialIsNeverInTheFolderThatIsOpened(t *testing.T) {
fake := &fakeNpm{}
tc := toolchainIn(t, fake)
var userconfig string
inner := npmAsUmask077(fake)
tc.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
for i, a := range args {
if a == "--userconfig" {
userconfig = args[i+1]
}
}
if st, err := os.Stat(userconfig); err != nil || st.Mode().Perm() != 0o600 {
t.Errorf("the credential file %s is %v (%v), want 0600", userconfig, st.Mode().Perm(), err)
}
return inner(ctx, env, name, args...)
}
if err := tc.Ensure(context.Background(), "v1"); err != nil {
t.Fatal(err)
}
if rel, err := filepath.Rel(tc.Paths.Home, userconfig); err != nil || rel != filepath.Base(userconfig) {
t.Errorf("the credential file %s is not in the account's home %s", userconfig, tc.Paths.Home)
}
if _, err := os.Stat(userconfig); err == nil {
t.Errorf("the credential file %s is left after the install", userconfig)
}
filepath.WalkDir(tc.Paths.Cache, func(p string, d fs.DirEntry, err error) error {
if err == nil && strings.Contains(d.Name(), "npmrc") {
t.Errorf("a credential file %s lies beneath the cache", p)
}
return nil
})
}
// 1891 review (ADR 0533 §2): open refuses a toolchain folder that holds an npm config file, naming it, and opens
// nothing of it, so a credential left by a dead install is never made readable.
func TestOpenRefusesAFolderHoldingAnNpmConfig(t *testing.T) {
fake := &fakeNpm{}
tc := toolchainIn(t, fake)
dir := tc.Paths.Dir("v1")
os.MkdirAll(filepath.Join(dir, "node_modules"), 0o700)
left := filepath.Join(dir, ".npmrc-install")
os.WriteFile(left, []byte("//x/:_auth=secret\n"), 0o600)
os.WriteFile(filepath.Join(dir, installedMark), []byte("v1\n"), 0o600)
err := tc.Ensure(context.Background(), "v1")
if err == nil || !strings.Contains(err.Error(), left) {
t.Errorf("Ensure = %v, want a refusal naming %s", err, left)
}
if st, _ := os.Stat(left); st.Mode().Perm() != 0o600 {
t.Errorf("the credential file was opened to %o", st.Mode().Perm())
}
if st, _ := os.Stat(dir); st.Mode().Perm()&0o005 != 0 {
t.Errorf("the folder holding it was opened to %o", st.Mode().Perm())
}
}
@@ -40,6 +40,7 @@ func (m *Module) Tools(steps []StepTool) []stdio.Tool {
tools = append(tools, t)
}
tools = append(tools, m.ReadTools()...)
tools = append(tools, m.releaseTool())
return append(tools,
stdio.Tool{Name: StatusTool,
Description: "What the Node build module is on this machine: the Node version its recipe builds with and the one installed, " +
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildnode
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023062828-6147a2045467
github.com/distribution/distribution/v3 v3.1.2
github.com/nats-io/nats.go v1.54.0
)
+4 -2
View File
@@ -1,5 +1,7 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876 h1:dtiIJzrX7ey1wmsdzapiFgCKDllCph7/vxfUp1ib/t8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9 h1:Ufmd2cYEXICQ817IGflccP9bexABE4WhIO49HPbaoSo=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023062828-6147a2045467 h1:i85+fFT0wZiTZ9CRVfMzWg1+Zr7oOLl3UH7hEaKcOIc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023062828-6147a2045467/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/alicebob/miniredis/v2 v2.35.0 h1:QwLphYqCEAo1eu1TqPRN2jgVMPBweeQcR21jeqDCONI=
github.com/alicebob/miniredis/v2 v2.35.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
+2 -1
View File
@@ -16,6 +16,7 @@
"build_node_fetch",
"build_node_test",
"build_node_build",
"build_node_release",
"build_node_step",
"build_node_steps",
"build_node_toolchains",
@@ -87,7 +88,7 @@
"id": "cache",
"type": "directory",
"path": "/var/lib/mesh-build-node-cache",
"mode": "0700",
"mode": "0711",
"owner": "mesh-build-node",
"unknown": "program"
},
@@ -0,0 +1,134 @@
package main
import (
"testing"
"time"
sdk "git.novox.be/novox/mesh-sdk/go/delivery"
)
// Issue 888: the checker judges on the trunk source.main names (mesh-delivery modules/mesh-delivery-checker main.go), and the
// gate moved it only for a merge it handed to the forge bridge (finish). A merge made at the forge directly moved every other
// delivery's trunk (trunkMoved) and left source.main, so the checker kept judging on the trunk before it.
const (
i888Other = "novox/other"
i888OtherH = "0000000000000000000000000000000000000001"
i888App = "novox/app"
i888Head1 = "1111111111111111111111111111111111111111"
i888Head2 = "2222222222222222222222222222222222222222"
i888Merge1 = "3333333333333333333333333333333333333333"
i888Merge2 = "4444444444444444444444444444444444444444"
)
func i888MergedAt(repo string, number int, head, merge string, at time.Time) PullEvent {
p := merged(repo, number, head, merge)
p.MergedAt = at.UTC().Format(time.RFC3339)
return p
}
// 888/AC1: a merge into main the gate did not hand is heard from the forge: source.main holds it, and keeps the others.
func TestAMergeHeardOnlyFromTheForgeMovesSourceMain(t *testing.T) {
w := newWorld(t)
w.source.main = &sdk.Source{V: sdk.Version, Group: "earlier", Heads: map[string]string{i888Other: i888OtherH}}
w.h.PullUpdated(pr(i888App, 3, i888Head1, "fix/x", "internal/a.go"))
w.settleAll()
w.h.PullMerged(i888MergedAt(i888App, 3, i888Head1, i888Merge1, w.now.Add(-time.Minute)))
w.settleAll()
if w.source.main == nil || w.source.main.Heads[i888App] != i888Merge1 || w.source.main.Heads[i888Other] != i888OtherH {
t.Fatalf("source.main is %+v; wanted %s at %s and %s kept", w.source.main, i888App, i888Merge1, i888Other)
}
n := len(w.source.moves)
w.settleAll()
w.h.PullMerged(i888MergedAt(i888App, 3, i888Head1, i888Merge1, w.now.Add(-time.Minute))) // heard again
w.settleAll()
if len(w.source.moves) != n || n != 1 {
t.Fatalf("source.main moved %d then %d times; wanted once", n, len(w.source.moves))
}
}
// 888/AC2: a group the gate merged through the bridge moves source.main once, in finish, though the forge's word of each
// member's merge is heard too.
func TestAGroupTheGateMergedMovesSourceMainOnceThoughTheForgeSaysItsMerges(t *testing.T) {
w := approvedGroup(t)
allGreen(w)
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
w.settleAll()
if len(w.forge.merges) != 2 || len(w.source.moves) != 1 {
t.Fatalf("merges %v, source.main moved %d times", w.forge.merges, len(w.source.moves))
}
commit := w.source.moves[0].Heads[gCat]
w.h.PullMerged(i888MergedAt(gCtl, 1, gHc, w.source.moves[0].Heads[gCtl], w.now))
w.h.PullMerged(i888MergedAt(gCat, 2, gHk, commit, w.now))
w.settleAll()
if len(w.source.moves) != 1 {
t.Fatalf("source.main moved %d times; wanted once, by the group: %+v", len(w.source.moves), w.source.moves)
}
}
// 888/AC3: an older merge heard after a newer one, or replayed as history, never moves source.main backwards.
func TestAnOlderMergeHeardLaterNeverMovesSourceMainBack(t *testing.T) {
w := newWorld(t)
w.h.PullMerged(i888MergedAt(i888App, 4, i888Head2, i888Merge2, w.now.Add(-time.Minute)))
w.settleAll()
w.h.PullMerged(i888MergedAt(i888App, 3, i888Head1, i888Merge1, w.now.Add(-2*time.Minute)))
w.settleAll()
if w.source.main == nil || w.source.main.Heads[i888App] != i888Merge2 || len(w.source.moves) != 1 {
t.Fatalf("source.main is %+v after %d moves; wanted the newer merge %s, moved once", w.source.main, len(w.source.moves), i888Merge2)
}
h := newWorld(t)
h.source.main = &sdk.Source{V: sdk.Version, Group: "now", Heads: map[string]string{i888App: i888Merge2}}
h.h.PullMerged(i888MergedAt(i888App, 3, i888Head1, i888Merge1, h.now.Add(-4*24*time.Hour)))
h.settleAll()
if h.source.main.Heads[i888App] != i888Merge2 || len(h.source.moves) != 0 {
t.Fatalf("a merge replayed as history moved source.main to %+v", h.source.main)
}
}
// 888/AC4: an approved group whose members were merged at the forge before the gate handed them moves source.main once.
func TestAnApprovedGroupMergedAtTheForgeMovesSourceMainOnce(t *testing.T) {
w := approvedGroup(t)
allGreen(w)
w.op.answer(w.op.open()[0], OptionApprove)
mc, mk := "5555555555555555555555555555555555555555", "6666666666666666666666666666666666666666"
w.h.PullMerged(i888MergedAt(gCtl, 1, gHc, mc, w.now))
w.h.PullMerged(i888MergedAt(gCat, 2, gHk, mk, w.now))
w.settleAll()
w.settleAll()
if len(w.source.moves) != 1 || w.source.main.Heads[gCtl] != mc || w.source.main.Heads[gCat] != mk {
t.Fatalf("source.main moved %d times to %+v; wanted once, both members at their merge commits", len(w.source.moves), w.source.main)
}
}
// 888/AC5 (review round 1): an older merge heard from the forge after the bridge merged a newer one, whose own pull.merged is
// not heard yet, never moves source.main back; nor does one older than the gate keeps records for, over a head it does not know.
func TestAnOlderForgeMergeNeverOverwritesANewerBridgeMergeOrAnUnknownHead(t *testing.T) {
w := approvedGroup(t)
allGreen(w)
w.op.answer(w.op.open()[0], OptionApprove)
w.settleAll()
w.settleAll()
if len(w.source.moves) != 1 {
t.Fatalf("the group did not move source.main once: %d", len(w.source.moves))
}
bridged := w.source.main.Heads[gCat]
w.h.PullUpdated(pr(gCat, 7, "7777777777777777777777777777777777777777", "fix/old", "modules/app/module.json"))
w.h.PullMerged(i888MergedAt(gCat, 7, "7777777777777777777777777777777777777777", "8888888888888888888888888888888888888888", w.now.Add(-10*time.Minute)))
w.settleAll()
if w.source.main.Heads[gCat] != bridged || len(w.source.moves) != 1 {
t.Fatalf("an older forge merge moved source.main from the bridge's %s to %s", bridged, w.source.main.Heads[gCat])
}
o := newWorld(t)
o.source.main = &sdk.Source{V: sdk.Version, Group: "pruned", Heads: map[string]string{i888App: i888Merge2}}
at := o.now
o.h.PullUpdated(pr(i888App, 3, i888Head1, "fix/x", "internal/a.go"))
o.later(keptFinal + 24*time.Hour)
o.h.PullMerged(i888MergedAt(i888App, 3, i888Head1, i888Merge1, at.Add(time.Minute)))
o.settleAll()
if o.source.main.Heads[i888App] != i888Merge2 || len(o.source.moves) != 0 {
t.Fatalf("a merge older than the records kept overwrote a head the gate does not know: %+v", o.source.main)
}
}
@@ -346,7 +346,8 @@ func (h *Holder) settleMerges() []func() {
}
}
})
case r.Merged != nil && !r.SourceMoved && !done[key]:
case (r.Merged != nil || d.MergedAs != "") && !r.SourceMoved && !done[key]:
// Merged through the bridge, or at the forge directly after the Approve (issue 888): the group is moved once.
if !h.allMerged(r.Heads) {
continue
}
@@ -420,3 +421,100 @@ func (h *Holder) finish(name, key string, approved map[string]string, last strin
}
h.Logf("[mesh-delivery] group %s is merged whole: source.main holds %s", name, strings.Join(members, ", "))
}
// gateMerges says whether finish moves source.main for a delivery: one the operator approved through an ask, whose merge the
// gate hands to the forge bridge.
func gateMerges(d *Delivery) bool {
return d.Ready != nil && d.Ready.Approved != nil && d.Ready.AskID != "" && len(d.Ready.Heads) > 0
}
// settleForgeMerges moves source.main for each merge into the trunk heard only from the forge (issue 888): a pull request
// merged at the forge directly, not handed by the gate on an Approve. Every other delivery's trunk moved with it (trunkMoved),
// and the checker judges on the trunk source.main names, so source.main moves too. A merge replayed as history, or older than
// a merge of its repository already heard, is never written: source.main never moves backwards. Under the lock; the writing
// happens outside it.
func (h *Holder) settleForgeMerges() []func() {
var out []func()
for _, d := range h.deliveries {
if d.MergedAs == "" || d.InSource || gateMerges(d) {
continue
}
if d.Base != MergeBranch || d.History != "" || h.newerMergeOf(d) {
d.InSource = true // not the trunk, or not its head: nothing to write
h.keep(d)
continue
}
id, repository, base, commit, at, group := d.ID, d.Repository, d.Base, d.MergedAs, d.MergedAt, firstOf(d.Branch, d.Repository)
out = append(out, func() {
err := h.Source.Move(func(prev *sdk.Source) (sdk.Source, bool) {
next := sdk.Source{Group: group, Heads: map[string]string{repository: commit}}
if prev != nil {
held := prev.Heads[repository]
if sameCommit(held, commit) {
return next, false // there already: the bridge's word and the forge's are one merge
}
if held != "" && !h.olderHead(repository, base, held, at) {
return next, false // source.main holds a newer merge, or one the gate cannot date: never moved back
}
for r, head := range prev.Heads {
if r != repository {
next.Heads[r] = head
}
}
}
return next, true
})
h.mu.Lock()
defer h.mu.Unlock()
if err != nil {
h.Logf("[mesh-delivery] %s merged at the forge, and its place in the source cannot be written yet: %v; asked again in %s", id, err, askAgainAfter)
h.again("source.main, moved for a merge heard from the forge")
return
}
if m := h.deliveries[id]; m != nil {
m.InSource = true
h.keep(m)
}
h.Logf("[mesh-delivery] %s merged at the forge: source.main holds %s at %s", id, repository, shortOf(commit, 12))
})
}
return out
}
// mergeOf is a delivery's merge into its trunk as the gate knows it: the forge's word (MergedAs), or else the bridge's
// (Ready.MergeCommit, Ready.Merged), which comes first for a merge the gate handed. Empty when it is not merged.
func mergeOf(d *Delivery) (string, time.Time) {
if d.MergedAs != "" {
return d.MergedAs, d.MergedAt
}
if d.Ready != nil && d.Ready.Merged != nil && d.Ready.MergeCommit != "" {
return d.Ready.MergeCommit, *d.Ready.Merged
}
return "", time.Time{}
}
// newerMergeOf says whether a merge into the same trunk of d's repository was heard that landed after d's.
func (h *Holder) newerMergeOf(d *Delivery) bool {
for _, o := range h.deliveries {
if commit, at := mergeOf(o); o != d && o.Repository == d.Repository && o.Base == d.Base && commit != "" &&
!sameCommit(commit, d.MergedAs) && at.After(d.MergedAt) {
return true
}
}
return false
}
// olderHead says whether the head source.main holds for a repository is a merge the gate knows landed before at, so a merge
// at that time may take its place. A head it does not know (written before its records, or pruned with its delivery) is
// older only when at is within the time final records are kept: a pruned merge ended before then (issue 888, review).
// Takes the lock.
func (h *Holder) olderHead(repository, base, head string, at time.Time) bool {
h.mu.Lock()
defer h.mu.Unlock()
for _, o := range h.deliveries {
if commit, when := mergeOf(o); o.Repository == repository && o.Base == base && commit != "" && sameCommit(commit, head) {
return !when.After(at)
}
}
return at.After(h.Now().Add(-keptFinal))
}
@@ -84,7 +84,10 @@ type Delivery struct {
Others []PullEvent `json:"others,omitempty"`
MergedAs string `json:"merged_as,omitempty"`
MergedAt time.Time `json:"merged_at"`
HeldWhy string `json:"held_why,omitempty"`
// InSource says the merge was looked at for source.main: written there, or left as older than the trunk's head (issue 888).
// A merge the gate handed to the bridge on an Approve is moved there by finish instead (merges.go).
InSource bool `json:"in_source,omitempty"`
HeldWhy string `json:"held_why,omitempty"`
// BackupsAgainAfter is the revision of its walk when a person released it for a failed backup: a record at that
// revision or before still holds the failure the controller was asked to clear, and does not hold it again
// (novox/hq ADR 0404, issue 1068).
@@ -267,6 +267,7 @@ func (h *Holder) readyAsks(now time.Time) []func() {
}
}
asks = append(asks, h.settleMerges()...)
asks = append(asks, h.settleForgeMerges()...)
asks = append(asks, h.withdrawnKeys()...)
asks = append(h.cancels, asks...)
h.cancels = nil
+1 -1
View File
@@ -3,7 +3,7 @@ module mesh-delivery-gate
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876 h1:dtiIJzrX7ey1wmsdzapiFgCKDllCph7/vxfUp1ib/t8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9 h1:Ufmd2cYEXICQ817IGflccP9bexABE4WhIO49HPbaoSo=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -120,7 +120,7 @@ type Client struct {
// Now dates the reports; nil is time.Now.
Now func() time.Time
// SocketUp wakes the client when the node-engine's write socket is made again (watchSocket); nil never does.
// A module whose write failed for want of the socket is filled again then (issue 1165), and nothing else is.
// A module whose write failed is filled again then (issue 1165, issue 1873), and nothing else is.
SocketUp <-chan struct{}
// watchBound watches a provider's bucket of entries for this machine's keys, once per run (consumers.go); set by
@@ -150,7 +150,8 @@ type state struct {
// of the consumers left out of one of its files.
providing map[string]bool
leftOut map[string]string
// unplaced is a module whose last fill failed for want of the node-engine's write socket (issue 1165).
// unplaced is a module whose last fill failed, for want of the node-engine's write socket (issue 1165) or by the
// engine's own refusal (issue 1873): filled again when the socket is made again.
unplaced map[string]bool
// bound is each provider's bucket of entries this client watches, by key: this machine's consumers' entries alone;
// boundReady is a bucket whose current entries are all read, and boundUsers the modules filled from each.
@@ -705,9 +706,10 @@ func (c *Client) reconcile(ctx context.Context, kvs map[string]jetstream.KeyValu
err = errFill(a)
}
if err != nil {
if socketAbsent(err) {
s.unplaced[module] = true
}
// Filled again when a node-engine makes its socket again: one gone (issue 1165), or one that refused for a
// reason of its own, which a newer engine may write (issue 1873: "is a directory", what a container
// runtime left at the path, cleared by the engine that followed).
s.unplaced[module] = true
failed = append(failed, fmt.Sprintf("%s: %v", f.Path, err))
continue
}
@@ -106,3 +106,38 @@ func TestWatchSocketWakesWhenTheSocketIsMadeAndWhenItsDirectoryReturns(t *testin
defer l.Close()
heard("the directory and socket back")
}
// Issue 1873: a fill the node-engine refused for a reason of its own (on ace, "read …/superuser.secret: is a
// directory", what a container runtime had left there) is tried again when a node-engine makes its socket again, as a
// newer engine may write what the older could not. Before, only a fill that failed for want of the socket was, so
// ace's postgres stayed unplaced after the engine that cures it was delivered, and its old failure was said every 30 s.
func TestNatsAFillTheEngineFailedIsTriedWhenTheSocketReturns(t *testing.T) {
b := startBus(t)
heard := b.listen(t)
e := newEngine(t, map[string]string{dunstrc: "owned"})
if err := os.MkdirAll(e.at(dunstrc), 0o755); err != nil {
t.Fatal(err)
}
stop := e.listen(t)
b.config(t, "dunst", "font-size", "13")
b.assign(t, "dunst", dunst(t))
aClient(t, b, e, t.TempDir())
eventually(t, "the engine's failure is said", func() bool {
r, _ := heard.last("dunst")
return r.Status == StatusFailed && strings.Contains(r.Reason, "is a directory")
})
// The node-engine replaces itself with one that clears what the runtime left.
stop()
if err := os.Remove(e.at(dunstrc)); err != nil {
t.Fatal(err)
}
e.listen(t)
eventually(t, "dunstrc is written by the new engine", func() bool {
return strings.Contains(e.read(t, dunstrc), "Monospace 13")
})
eventually(t, "and the report says applied", func() bool {
r, _ := heard.last("dunst")
return r.Status == StatusApplied
})
}
+2
View File
@@ -0,0 +1,2 @@
# The test script compiles the sources beside themselves before the tests import them (package.json, issue 1891).
*.js
+1 -1
View File
@@ -5,7 +5,7 @@
"type": "module",
"private": true,
"scripts": {
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
"test": "tsc --module NodeNext --moduleResolution NodeNext --target ES2022 --types node --rootDir . --outDir . $(find . -name '*.ts' ! -name '*.d.ts' ! -path './test/*' ! -path './node_modules/*' ! -path './dist/*') && node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.7"
+1 -1
View File
@@ -7,7 +7,7 @@
"scripts": {
"build": "tsc topics.ts client.ts index.ts tools/index.ts provisioner/index.ts bootstrap/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"typecheck": "tsc -p tsconfig.json",
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
"test": "npm run build && node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.1"
@@ -83,7 +83,7 @@ func TestTheKeeperIsAProcessUnderItsOwnAccountWithABusAccountOfItsOwn(t *testing
t.Errorf("the bus account is the keeper's own secret, owned by its account: %v", secrets)
}
state := m.resource(t, "state")
if state["type"] != "directory" || state["place"] != "." || state["unknown"] == nil {
if state["type"] != "directory" || state["place"] != "mesh" || state["unknown"] == nil {
t.Errorf("the state directory: %v", state)
}
order := map[string]int{}
@@ -211,3 +211,37 @@ func TestUpgradeOrderIsATrustedListOfTheMesh(t *testing.T) {
t.Error("a value falls back only to a key of its own type (ADR 0348 §4)")
}
}
// 1841/AC17: the keeper's state directory is the mesh's directory for the module (place "mesh", <data root>/mesh/pacman),
// never place "." or a placement by default: for this module both resolve under /var/lib/pacman, the package manager's
// own database directory (issue 1841: the node-engine used it as found and reported it unhealthy, and wrote keeper.json
// into it). What the keeper and its root helper keep there is still named through ${dir:state}.
func TestTheStateDirectoryIsOutsideThePackageManagersOwn(t *testing.T) {
m := rawManifest(t)
resources, _ := m["resources"].([]any)
var state map[string]any
for _, x := range resources {
if r, _ := x.(map[string]any); r["id"] == "state" {
state = r
}
}
if state == nil || state["type"] != "directory" {
t.Fatalf("no state directory: %v", state)
}
if state["place"] != "mesh" || state["path"] != nil {
t.Fatalf("the state directory says place %v and path %v; it is placed \"mesh\" with no path, so it resolves to "+
"/var/lib/mesh/pacman and not into /var/lib/pacman", state["place"], state["path"])
}
raw, err := os.ReadFile(filepath.Join(moduleDir, "module.json"))
if err != nil {
t.Fatal(err)
}
if bytes.Contains(raw, []byte("/var/lib/pacman")) {
t.Fatal("the manifest names /var/lib/pacman")
}
for _, want := range []string{"${dir:state}/keeper.json", "${dir:state}/broker-keeper.secret", RootSocketPath} {
if !bytes.Contains(raw, []byte(want)) {
t.Errorf("the manifest no longer names %s", want)
}
}
}
+7 -2
View File
@@ -248,12 +248,12 @@ func tools(m *Machine) []stdio.Tool {
Name: "pacman_upgrades",
Description: "Every full upgrade and rollback of every node, from the bus (ADR 0508 §1): each run's node, kind, what started it, " +
"its round, when it began and ended, its outcome (refused, finished, failed or running) and whether its after-upgrade check passed; " +
"narrowed to a node and to days, newest first. With a node and a run, that run whole, every step with its time.",
"narrowed to a node and to days, newest first. With a run, that run whole, every step with its time: the node's named, else this one's.",
Input: schema(map[string]any{
"node": map[string]any{"type": "string", "description": "only this node's runs"},
"from": sinceArg,
"until": map[string]any{"type": "string", "description": "a day, YYYY-MM-DD, the last one listed"},
"run": map[string]any{"type": "string", "description": "one run, by its id, with node"},
"run": map[string]any{"type": "string", "description": "one run, by its id: of node, else of this node"},
"limit": limitArg(50, 1000),
}),
Run: func(args map[string]any) (any, error) {
@@ -269,6 +269,11 @@ func tools(m *Machine) []stdio.Tool {
if err != nil {
return nil, err
}
if v[3] != "" && v[0] == "" {
// An address naming the machine has its node taken out of the arguments (the console):
// the run is this machine's (issue 1841).
v[0] = thisNode()
}
return ListUpgrades(upgradeState(StateRuns), v[0], v[1], v[2], v[3], n)
},
},
@@ -57,7 +57,7 @@ func TestPacmanConfIsWholeTheUnionOfRepositoriesAndTheImprovedOptions(t *testing
t.Skip("pacman-conf is not installed here; the rendered file is not proven")
}
file := filepath.Join(t.TempDir(), "pacman.conf")
if err := os.WriteFile(file, []byte(content), 0o644); err != nil {
if err := os.WriteFile(file, []byte(provenConf(t, content, filepath.Dir(file))), 0o644); err != nil {
t.Fatal(err)
}
repos, err := exec.Command(conf, "--config", file, "--repo-list").CombinedOutput()
@@ -74,6 +74,35 @@ func TestPacmanConfIsWholeTheUnionOfRepositoriesAndTheImprovedOptions(t *testing
}
}
// Every repository of the shipped file reads the mirror list reflector writes, and the file pacman-conf proves reads
// nothing of the machine: the unit level runs shut off (novox/hq ADR 0448), where /etc/pacman.d is not there
// (issue 1891).
func TestTheProvenConfReadsNothingOfTheMachine(t *testing.T) {
content := templateText(t, manifest(t).resource(t, "config"))
if n := strings.Count(content, "Include = "+Mirrorlist+"\n"); n != 3 || strings.Count(content, "Include") != n {
t.Fatalf("every repository reads %s, and nothing else is included: %d of 3\n%s", Mirrorlist, n, content)
}
dir := t.TempDir()
proven := provenConf(t, content, dir)
if strings.Contains(proven, "/etc/") {
t.Fatalf("the proven file still names the machine's /etc:\n%s", proven)
}
if _, err := os.Stat(filepath.Join(dir, "mirrorlist")); err != nil {
t.Fatalf("the proven file's mirror list is not in its directory: %v", err)
}
}
// provenConf is the shipped pacman.conf with its repositories reading a mirror list of its own in dir, made there,
// so pacman-conf proves the file the module ships and reads nothing of the machine.
func provenConf(t *testing.T, content, dir string) string {
t.Helper()
list := filepath.Join(dir, "mirrorlist")
if err := os.WriteFile(list, []byte("Server = https://mirror.example/$repo/os/$arch\n"), 0o644); err != nil {
t.Fatal(err)
}
return strings.ReplaceAll(content, "Include = "+Mirrorlist, "Include = "+list)
}
func TestReflectorWritesTheListPacmanReads(t *testing.T) {
content := templateText(t, manifest(t).resource(t, "mirrors"))
opts := map[string]string{}
+1 -1
View File
@@ -228,7 +228,7 @@ func (u *Upgrades) StartRollback(of string, removeAdded bool, ask string) (Run,
if o.Run.Run == r.Run.Run {
return r.Run, fmt.Errorf("run %s of %s is there already: one run a second", o.Run.Run, u.Node)
}
if !ended(o.Run) && o.From != FromLog {
if holdsTheDatabase(o.Run) {
return r.Run, u.refuse(r, fmt.Sprintf("run %s is still going (its last step is %s): one run at a time", o.Run.Run, lastStep(o.Run)))
}
}
@@ -0,0 +1,115 @@
package main
// A run read from pacman's log, or a run whose transaction ended, never holds a new run back (issue 1841, the
// refusal of 2026-10-22 22:00 on the control-node: "run 20220123T194900Z is still going (its last step is finished): one
// run at a time"); and pacman_upgrades reads one run whole at an address that names the machine.
import (
"strings"
"testing"
"time"
)
// installLogHead is the first twelve lines of the control-node's /var/log/pacman.log, captured 2026-10-22 22:10
// with node-reads.read: the install's `pacman -Syu` with nothing to do, which became run 20220123T194900Z, its
// last step finished at the moment it began (pacman_upgrades there, 2026-10-22: from log, outcome finished,
// began 2022-01-23T19:49:00Z, ended 2022-01-23T20:49:00+01:00).
const installLogHead = "[2022-01-23T20:49:00+0100] [PACMAN] Running 'pacman -Syu'\n" +
"[2022-01-23T20:49:00+0100] [PACMAN] synchronizing package lists\n" +
"[2022-01-23T20:49:00+0100] [PACMAN] starting full system upgrade\n" +
"[2022-01-23T20:50:45+0100] [PACMAN] Running 'pacman -S nano'\n" +
"[2022-01-23T20:50:49+0100] [ALPM] transaction started\n" +
"[2022-01-23T20:50:50+0100] [ALPM] installed nano (6.0-1)\n" +
"[2022-01-23T20:50:50+0100] [ALPM] transaction completed\n" +
"[2022-01-23T20:50:50+0100] [ALPM] running '30-systemd-update.hook'...\n" +
"[2022-01-23T20:55:31+0100] [PACMAN] Running 'pacman -S docker'\n" +
"[2022-01-23T20:55:36+0100] [ALPM] transaction started\n" +
"[2022-01-23T20:55:36+0100] [ALPM] installed bridge-utils (1.7.1-1)\n" +
"[2022-01-23T20:55:36+0100] [ALPM] installed libtool (2.4.6+42+gb88cebd5-16)\n"
// 1841/AC20: given the run read from the control-node's log, when an upgrade is asked, then it starts.
func TestARunReadFromTheLogIsHistoryAndNeverHoldsAnAsk(t *testing.T) {
k := newKeeperBed(t)
past := PastUpgrades("node-a", installLogHead)
if len(past) != 1 || past[0].Run != "20220123T194900Z" || steps(past[0]) != "started finished" || past[0].From != FromLog {
t.Fatalf("the captured log reads as the record the control-node holds: %+v", past)
}
if _, err := k.runs.Create(NodeKey("node-a", past[0].Run), past[0]); err != nil {
t.Fatal(err)
}
if _, err := k.u.StartUpgrade(FromAsk, "", "a1"); err != nil {
t.Fatal(err)
}
var r Run
k.runs.value("node-a.20261004T120000Z", &r)
if lastStep(r) == StepRefused || len(k.root.started) != 1 {
t.Errorf("a run read from the log held the ask back: %s %+v", steps(r), r.Steps)
}
}
// 1841/AC21: given a run the mesh ran whose last step is finished, failed or refused (no after-upgrade check
// yet, no end time of the run itself), when another run would start, then the earlier one does not hold it;
// a run whose transaction may still go on (no step, recorded, checked before, started) does.
func TestARunWhoseTransactionEndedNeverHoldsTheNext(t *testing.T) {
at := time.Date(2026, 10, 22, 20, 0, 0, 0, time.UTC)
for last, holds := range map[string]bool{
StepFinished: false, StepFailed: false, StepRefused: false, StepCheckedAfter: false,
"": true, StepRecorded: true, StepCheckedBefore: true, StepStarted: true,
} {
r := Run{Node: "node-a", Run: "20261022T200000Z", Kind: RunUpgrade, From: FromAsk}
if last != "" {
r.Steps = []Step{{Step: last, At: at}}
}
if got := holdsTheDatabase(r); got != holds {
t.Errorf("last step %q: holds %v, want %v", last, got, holds)
}
r.From = FromLog
if holdsTheDatabase(r) {
t.Errorf("a run read from the log, last step %q, holds the next", last)
}
}
k := newKeeperBed(t)
done := Run{Node: "node-a", Run: "20261004T110000Z", Kind: RunUpgrade, From: FromAsk,
Steps: []Step{{Step: StepStarted, At: k.now}, {Step: StepFinished, At: k.now}}}
if _, err := k.runs.Create(NodeKey("node-a", done.Run), done); err != nil {
t.Fatal(err)
}
if _, err := k.u.StartUpgrade(FromAsk, "", "a1"); err != nil {
t.Fatal(err)
}
var r Run
k.runs.value("node-a.20261004T120000Z", &r)
if lastStep(r) == StepRefused {
t.Errorf("a finished run held the next: %+v", r.Steps)
}
}
// 1841/AC22: given an address that names the machine (the console takes `node` out of the arguments), when
// pacman_upgrades is asked for one run, then that machine's run is answered whole.
func TestPacmanUpgradesReadsOneRunOfThisNodeWithoutANode(t *testing.T) {
runs := newBucket()
past := PastUpgrades("node-a", installLogHead)[0]
if _, err := runs.Create(NodeKey("node-a", past.Run), past); err != nil {
t.Fatal(err)
}
savedState, savedNode := upgradeState, thisNode
upgradeState = func(string) Bucket { return runs }
thisNode = func() string { return "node-a" }
defer func() { upgradeState, thisNode = savedState, savedNode }()
for _, tool := range tools(machine(nil, 1000)) {
if tool.Name != "pacman_upgrades" {
continue
}
out, err := tool.Run(map[string]any{"run": past.Run})
if err != nil {
t.Fatalf("one run without a node: %v", err)
}
whole := out.(map[string]any)["run"].(Run)
if whole.Node != "node-a" || steps(whole) != "started finished" || !strings.Contains(whole.Steps[1].Why, "no transaction") {
t.Errorf("one run whole: %+v", whole)
}
return
}
t.Fatal("no pacman_upgrades")
}
+17 -1
View File
@@ -63,6 +63,22 @@ func ended(r Run) bool {
return false
}
// holdsTheDatabase says whether a run holds a new run back (one run at a time, ADR 0508 §8): only a run the
// mesh ran whose transaction may still go on. A run read from pacman's log is history (ADR 0508 §1), and a run
// whose last step is finished, failed, refused or checked after has no transaction left: its after-upgrade
// check, still to be written by Follow, reads and changes nothing (issue 1841: every run read from the log
// ended at finished and held every ask back).
func holdsTheDatabase(r Run) bool {
if r.From == FromLog {
return false
}
switch lastStep(r) {
case StepFinished, StepFailed, StepRefused, StepCheckedAfter:
return false
}
return true
}
// ownRuns are the node's runs, oldest first, each with the revision of its key.
func (u *Upgrades) ownRuns() ([]openRun, error) {
keys, err := u.Runs.Keys()
@@ -117,7 +133,7 @@ func (u *Upgrades) startUpgrade(r *openRun) error {
if o.Run.Run == r.Run.Run {
return fmt.Errorf("run %s of %s is there already: one run a second", o.Run.Run, u.Node)
}
if !ended(o.Run) {
if holdsTheDatabase(o.Run) {
return u.refuse(r, fmt.Sprintf("run %s is still going (its last step is %s): one run at a time", o.Run.Run, lastStep(o.Run)))
}
}
+1 -1
View File
@@ -3,7 +3,7 @@ module pacman
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9
github.com/nats-io/nats-server/v2 v2.11.17
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876 h1:dtiIJzrX7ey1wmsdzapiFgCKDllCph7/vxfUp1ib/t8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022183749-875c680d9876/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9 h1:Ufmd2cYEXICQ817IGflccP9bexABE4WhIO49HPbaoSo=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
+1 -1
View File
@@ -147,7 +147,7 @@
"id": "state",
"type": "directory",
"mode": "0711",
"place": ".",
"place": "mesh",
"unknown": "program"
},
{
+2 -1
View File
@@ -119,7 +119,8 @@ what is declared, refusing secrets, taking back what is no longer declared, and
Each directory or archive this module declares says how it treats a file it did not write (ADR 0422, ADR 0416).
- `${dir:state}`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `${dir:data}` (`/var/lib/mesh/<module>`): everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `${dir:state}`: the package's own directory, declared for one delivery with the package's owner and mode so the node-engine gives it back (issue 1873); the module keeps nothing there.
- `${dir:grants}`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `/var/lib/mesh-store`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `${dir:store-data}/dumps`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
@@ -0,0 +1,94 @@
package main
// The module's directories (novox/hq issue 1873): its state directory said place ".", which is /var/lib/postgres,
// the postgresql package's own (tmpfiles.d/postgresql.conf: postgres, 0700), and its grants directory, placed by
// default, lay inside it; the node-engine gave /var/lib/postgres to the operator's account. What the module keeps
// moves to the mesh's directory for it, and for one delivery the record "state" states the package's owner and mode,
// so the node-engine, which takes a directory it recorded at the same path under the same id as its own, gives it
// back. The store's data is not touched: it is /var/lib/mesh-store, as before.
import (
"encoding/json"
"os"
"reflect"
"strings"
"testing"
)
func postgresManifest(t *testing.T) (map[string]map[string]any, map[string]any) {
t.Helper()
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(raw), "${dir:state}") {
t.Errorf("the manifest still keeps something in ${dir:state}, which is /var/lib/postgres")
}
var m map[string]any
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
out := map[string]map[string]any{}
list, _ := m["resources"].([]any)
for _, x := range list {
r, _ := x.(map[string]any)
out[r["id"].(string)] = r
}
return out, m
}
// 1873/AC2: secrets, consumers and grants are kept in <data root>/mesh/postgres; "state" gives /var/lib/postgres back
// to postgres, 0700; the store's data and dumps are as they were.
func TestTheModuleKeepsItsFilesOutsideThePackagesDirectory(t *testing.T) {
rs, m := postgresManifest(t)
data := rs["data"]
if data == nil || data["type"] != "directory" || data["place"] != "mesh" || data["path"] != nil || data["mode"] != "0700" {
t.Fatalf("the module's directory is placed \"mesh\" with no path, 0700 (/var/lib/mesh/postgres): %v", data)
}
if g := rs["grants"]; g == nil || g["path"] != "${dir:data}/grants" || g["place"] != nil {
t.Errorf("grants lies in the module's directory: %v", g)
}
if c := rs["consumers"]; c == nil || c["path"] != "${dir:data}/consumers.json" {
t.Errorf("consumers.json lies in the module's directory: %v", c)
}
own, _ := m["own-secrets"].(map[string]any)
if own["superuser"] != "${dir:data}/superuser.secret" || own["reader"] != "${dir:data}/reader.secret" {
t.Errorf("the own secrets lie in the module's directory: %v", own)
}
state := rs["state"]
if state == nil || state["type"] != "directory" || state["path"] != "/var/lib/postgres" || state["owner"] != "postgres" ||
state["mode"] != "0700" || state["place"] != nil {
t.Errorf("state gives /var/lib/postgres back as postgresql declares it, postgres 0700: %v", state)
}
// The store's data, its dumps and how the container reads them: exactly as before.
if s := rs["store-data"]; !reflect.DeepEqual(s, map[string]any{"id": "store-data", "type": "directory",
"path": "/var/lib/mesh-store", "mode": "0700", "owner": "999:70", "unknown": "program"}) {
t.Errorf("store-data changed: %v", s)
}
if d := rs["dumps"]; !reflect.DeepEqual(d, map[string]any{"id": "dumps", "type": "directory",
"path": "${dir:store-data}/dumps", "mode": "0700", "owner": "999:70", "unknown": "program"}) {
t.Errorf("dumps changed: %v", d)
}
server := rs["server"]
if !reflect.DeepEqual(server["volumes"], []any{"/var/lib/mesh-store:/var/lib/postgresql/data",
"${dir:data}/superuser.secret:/run/secrets/superuser:ro"}) {
t.Errorf("the container's volumes: %v", server["volumes"])
}
if env, _ := server["env"].(map[string]any); env["PGDATA"] != "/var/lib/postgresql/data/pgdata" {
t.Errorf("PGDATA changed: %v", env)
}
build, _ := m["build"].(map[string]any)
artifacts, _ := build["artifacts"].([]any)
env := artifacts[0].(map[string]any)["env"].(map[string]any)
for key, want := range map[string]string{
"MESH_PROVISION_PASSWORD_FILE": "${dir:data}/superuser.secret",
"MESH_POSTGRES_READER_PASSWORD_FILE": "${dir:data}/reader.secret",
"MESH_CONSUMERS_FILE": "${dir:data}/consumers.json",
"MESH_RECEIVES": "${dir:grants}/mesh.json",
} {
if env[key] != want {
t.Errorf("%s is %v, not %s", key, env[key], want)
}
}
}
+18 -9
View File
@@ -61,8 +61,8 @@
"postgres-database": "${dir:grants}"
},
"own-secrets": {
"superuser": "${dir:state}/superuser.secret",
"reader": "${dir:state}/reader.secret"
"superuser": "${dir:data}/superuser.secret",
"reader": "${dir:data}/reader.secret"
},
"data": {
"own": [
@@ -93,22 +93,31 @@
},
"resources": [
{
"id": "state",
"id": "data",
"type": "directory",
"mode": "0700",
"place": ".",
"place": "mesh",
"unknown": "program"
},
{
"id": "state",
"type": "directory",
"path": "/var/lib/postgres",
"owner": "postgres",
"mode": "0700",
"unknown": "program"
},
{
"id": "grants",
"type": "directory",
"path": "${dir:data}/grants",
"mode": "0700",
"unknown": "program"
},
{
"id": "consumers",
"type": "file",
"path": "${dir:state}/consumers.json",
"path": "${dir:data}/consumers.json",
"mode": "0600",
"template": "files/consumers.json"
},
@@ -146,7 +155,7 @@
],
"volumes": [
"/var/lib/mesh-store:/var/lib/postgresql/data",
"${dir:state}/superuser.secret:/run/secrets/superuser:ro"
"${dir:data}/superuser.secret:/run/secrets/superuser:ro"
]
},
{
@@ -169,10 +178,10 @@
],
"env": {
"MESH_PROVISION_POSTGRES": "postgres://postgres@127.0.0.1:${port:5432}/postgres?sslmode=disable",
"MESH_PROVISION_PASSWORD_FILE": "${dir:state}/superuser.secret",
"MESH_PROVISION_PASSWORD_FILE": "${dir:data}/superuser.secret",
"MESH_RECEIVES": "${dir:grants}/mesh.json",
"MESH_POSTGRES_READER_PASSWORD_FILE": "${dir:state}/reader.secret",
"MESH_CONSUMERS_FILE": "${dir:state}/consumers.json"
"MESH_POSTGRES_READER_PASSWORD_FILE": "${dir:data}/reader.secret",
"MESH_CONSUMERS_FILE": "${dir:data}/consumers.json"
}
},
{
@@ -311,6 +311,8 @@ func TestAnUnreachableHostIsNotRefreshed(t *testing.T) {
}
func TestTestSaysHowItAuthenticatedOrWhyNot(t *testing.T) {
// The test's own agent, none: the account running it may have one, and the unit level runs with none (issue 1891).
t.Setenv("SSH_AUTH_SOCK", "")
ok := "debug1: Connecting to ace.internal [10.0.0.2] port 22.\ndebug1: Server accepts key: /h/.ssh/id_ed25519 ED25519 " + edFP + "\nAuthenticated to ace.internal ([10.0.0.2]:22) using \"publickey\".\n"
f := &fake{answer: func(c call) forgekey.Ran { return forgekey.Ran{Stderr: ok} }}
got, err := client(t.TempDir(), f).Test(context.Background(), "ace")
@@ -327,9 +329,15 @@ func TestTestSaysHowItAuthenticatedOrWhyNot(t *testing.T) {
if got["ok"] != false || got["why"] != "no key it offered was accepted" || !reflect.DeepEqual(got["offered"], []string{"/h/.ssh/id_box"}) {
t.Fatalf("%v", got)
}
if !strings.Contains(got["note"].(string), "agent") {
if note, _ := got["note"].(string); !strings.Contains(note, "agent") {
t.Fatalf("no word on the agent: %v", got)
}
agent := filepath.Join(t.TempDir(), "agent.sock")
t.Setenv("SSH_AUTH_SOCK", agent)
got, _ = client(t.TempDir(), f).Test(context.Background(), "ace")
if got["agent"] != agent || got["note"] != nil {
t.Fatalf("the environment's agent is not the one it ran with, or a passphrase note without a passphrase: %v", got)
}
}
func TestCheckFindsWhatIsWrongAndSaysWhatItDidNotCheck(t *testing.T) {
+2 -1
View File
@@ -6,4 +6,5 @@ The `systemd` module of the mesh catalogue: installs the package `systemd`; clai
Each directory or archive this module declares says how it treats a file it did not write (ADR 0422, ADR 0416).
- `${dir:state}`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `${dir:data}` (`/var/lib/mesh/<module>`): everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- `${dir:state}`: the package's own directory, declared for one delivery with the package's owner and mode so the node-engine gives it back (issue 1873); the module keeps nothing there.
@@ -0,0 +1,52 @@
package main
// The module's directories (novox/hq issue 1873): its state directory said place ".", which is /var/lib/systemd,
// systemd's own directory (tmpfiles.d/systemd.conf: d /var/lib/systemd 0755 root root); the node-engine gave it to
// the operator's account, mode 0700, on every machine. What the tools keep moves to the mesh's directory for the
// module, and for one delivery the record "state" states the package's owner and mode, so the node-engine, which
// takes a directory it recorded at the same path under the same id as its own, gives it back.
import (
"encoding/json"
"os"
"testing"
)
func manifestResources(t *testing.T) (map[string]map[string]any, map[string]any) {
t.Helper()
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m map[string]any
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
out := map[string]map[string]any{}
list, _ := m["resources"].([]any)
for _, x := range list {
r, _ := x.(map[string]any)
out[r["id"].(string)] = r
}
return out, m
}
// 1873/AC1: the tools' record is kept in <data root>/mesh/systemd; "state" gives /var/lib/systemd back to root, 0755.
func TestTheToolsKeepTheirRecordOutsideSystemdsOwnDirectory(t *testing.T) {
rs, m := manifestResources(t)
data := rs["data"]
if data == nil || data["type"] != "directory" || data["place"] != "mesh" || data["path"] != nil {
t.Fatalf("the tools' directory is placed \"mesh\" with no path (/var/lib/mesh/systemd): %v", data)
}
build, _ := m["build"].(map[string]any)
artifacts, _ := build["artifacts"].([]any)
env := artifacts[0].(map[string]any)["env"].(map[string]any)
if env["MESH_SYSTEMD_STATE"] != "${dir:data}" {
t.Errorf("MESH_SYSTEMD_STATE is %v, not ${dir:data}", env["MESH_SYSTEMD_STATE"])
}
state := rs["state"]
if state == nil || state["type"] != "directory" || state["path"] != "/var/lib/systemd" || state["owner"] != "root" ||
state["mode"] != "0755" || state["place"] != nil {
t.Errorf("state gives /var/lib/systemd back as systemd declares it, root 0755: %v", state)
}
}
+11 -3
View File
@@ -41,7 +41,7 @@
"systemd-tools"
],
"env": {
"MESH_SYSTEMD_STATE": "${dir:state}"
"MESH_SYSTEMD_STATE": "${dir:data}"
}
}
]
@@ -53,10 +53,18 @@
"package": "systemd"
},
{
"id": "state",
"id": "data",
"type": "directory",
"mode": "0700",
"place": ".",
"place": "mesh",
"unknown": "program"
},
{
"id": "state",
"type": "directory",
"path": "/var/lib/systemd",
"owner": "root",
"mode": "0755",
"unknown": "program"
}
]