Compare commits

...
Author SHA1 Message Date
jschoubben 1968aba6e0 mesh-build-go, mesh-build-node: run every package of a check's level after a failure (issue 1921)
Pins mesh-sdk fix/1921-every-failure, whose check ask no longer stops a
level at its first failing test step, so one check runs every package
and names every failure. A captured fixture with a failing test in each
of two packages proves the build module's test step names both.
2026-10-24 23:47:27 +02:00
jschoubben 6fb0ceeef8 Merge pull request 'R4g finish: mesh-build-go's compose job, the reads-outside marker in the digest, merge-check.sh gone (issues 1295, 1621-1623, 1935)' (#473) from feat/r4g-finish into main
Reviewed-on: #473
2026-10-24 20:56:49 +00:00
mesh-tools c651161787 Merge pull request 'gitea's forge-account reads the forge's own host key file before scanning its sshd, so a forge restart no longer refuses every forge account (issue 1939)' (#472) from fix/1939-forge-host-key into main 2026-10-24 20:05:39 +00:00
jschoubben ee216e4e63 Merge pull request 'The gate keeps a merge it hears before it holds its seat and catches source.main up when it does, so a missed merge no longer leaves the checker on an old trunk (issue 1726)' (#474) from fix/1726-trunk-catch-up into main
Reviewed-on: #474
2026-10-24 19:47:47 +00:00
jschoubben 2fabbe556e Pin to the group's new mesh-sdk head bfc3883a, whose composer reads a fallback to a predefined key as the indexer does (ADR 0348 §7)
mesh/delivery delivered
mesh/delivery-group group feat/r4g-finish merged: every member is merged
mesh/check checking 2fabbe556e73 on 6fb0ceeef817 | novox/mesh-catalog#473 in group feat/r4g-finish
2026-10-24 14:57:35 +02:00
jschoubben 89b9212c4d Pin to the group's new mesh-sdk head 19c1d359, whose composer names the reader's pinned SDK instead of a bare (devel) (ADR 0482 clause 5)
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-catalog#473: modules/mesh-build-go/go.mod names mesh-sdk 19c1d35939bc, which is neither on mesh-sdk's trunk nor …
2026-10-24 14:12:09 +02:00
jschoubben 40da368ee3 Date an adopted merge once, by the walk it was adopted from, so a later walk it follows never makes it look newer than a forge merge heard after it (issue 1726, review)
mesh/check pass 40da368ee34e on ee216e4e63f1 | unit 246 run, 4189 kept | integration 228 run, 4249 kept | over bound: unit, integration (slowest unit …
mesh/delivery delivered
2026-10-24 13:55:10 +02:00
jschoubben 865f3b6455 Say what pacman-tools reads outside: the root predefined.json (issue 1935)
mesh/delivery-group group feat/r4g-finish rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: shanks: novox/mesh-catalog#473: modules/mesh-build-go/go.mod names mesh-sdk bc0c25e51052, which is neither on mesh-sdk's trunk nor …
2026-10-24 13:48:18 +02:00
jschoubben 03524ecd13 Mark pacman-tools as reading outside its directory: its upgrade-order test reads the root predefined.json, which its digest does not cover (issue 1935) 2026-10-24 13:48:09 +02:00
jschoubben 15101c9ce6 Move source.main for a merge the gate caught up after a gap, so the checker re-checks open heads on the trunk that is (issue 1726)
The checker takes a repository's trunk head only from the gate's source.main. On 2026-10-24 gitea's
restarted poll announced mesh-catalog #471's merge while the gate was starting and could not yet tell
whether it held the seat; the gate acknowledged the event unread. It then adopted the controller's walk
of the merge commit, which carries no merge time, so every older merge of the repository looked newer
and source.main was never moved: #472 was checked on the trunk before #471 and its merge refused.

- An event heard before the gate can tell whether it holds the seat is offered again, not taken.
- An adopted merge is dated by its walk's creation, so it settles into source.main as a forge merge does.
- Each time the gate comes to hold the seat it weighs the newest merge of each repository against
  source.main once more, so a merge marked there and never written (as #471) is written; never backwards.
2026-10-24 13:41:56 +02:00
jschoubben f910b0ceb8 Pin mesh-build-go to the group's mesh-sdk head after main's merge, so it builds on the SDK the group delivers 2026-10-24 13:28:16 +02:00
jschoubben 91c3e954ae Merge main into feat/r4g-finish (#471) 2026-10-24 13:26:24 +02:00
jschoubben 545c72bb59 Merge pull request 'gitea's watcher answers its handshake without waiting for the forge, so a slow forge at a tool-runner restart no longer kills its tools and events (issue 1726)' (#471) from fix/1726-gitea-handshake-not-on-forge into main
Reviewed-on: #471
2026-10-24 11:10:39 +00:00
jschoubben 7afcd0282d Set up the forge's webhook without holding the bundle's import, so a slow forge never costs gitea its handshake and its tools (issue 1726)
At 12:37 on 2026-10-24 node-tools on novox was replaced; gitea's watcher bundle awaited the webhook's set-up on the forge's admin API at the top of its import, outlasted the 10 s handshake and was killed, never to start again. gitea_commit_status, gitea_merge_pull_request and gitea_events were then served nowhere: the forge bridge's calls found no responders and no new pull request was announced.
2026-10-24 13:00:57 +02:00
jschoubben 75156fce00 gitea: forge-account reads the forge's own host key file before scanning its sshd (issue 1939)
mesh/delivery delivered
mesh/check pass 75156fce007e on c65116178720 | unit 246 run, 4192 kept | integration 228 run, 4252 kept | over bound: unit, integration (slowest unit …
2026-10-24 12:50:20 +02:00
jschoubben 16d7b1898d Mark the packages whose tests read files outside their directory, found by the review (issue 1935) 2026-10-24 04:38:49 +02:00
jschoubben f04465537b Say in realtree_test that a missing trunk fails, not skips (issue 1295) 2026-10-24 04:37:52 +02:00
jschoubben e870a7f65f Pin to the finish group's mesh-sdk head dd0d156b, so every member names one SDK (issue 1622) 2026-10-24 04:28:14 +02:00
jschoubben e8a59171c7 Merge remote-tracking branch 'origin/feat/r4g-finish-composer' into HEAD 2026-10-24 04:26:03 +02:00
jschoubben ebb25757a2 Merge remote-tracking branch 'origin/feat/r4g-finish-1935' into HEAD 2026-10-24 04:26:02 +02:00
jschoubben a715c8821f mesh-build-go: a catalogue module's non-Go files enter its packages' digests; markers where tests read other modules (issue 1935) 2026-10-24 03:59:08 +02:00
jschoubben b6ed8599c8 mesh-build-go: a package marked reads-outside is never kept, so tests that read files outside their directory run on every check (issue 1935) 2026-10-24 03:46:45 +02:00
jschoubben ae6f7d5b54 Make a missing trunk copy a failure again, now the checker places it (issue 1295) 2026-10-24 03:43:49 +02:00
78 changed files with 675 additions and 57 deletions
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+2 -1
View File
@@ -251,7 +251,8 @@ func main() {
}
var grown []string
if baseDir == "" {
fmt.Println("NOT COMPARED: no base branch's lists were given, so additions to them were not judged")
fmt.Fprintln(os.Stderr, "NOT COMPARED: no base branch's lists were given (-base <dir>), so additions to them were not judged; that is a failure, not a pass (issue 1295)")
os.Exit(2)
} else {
baseShared, _, err1 := list(filepath.Join(baseDir, "shared"))
baseKnown, _, err2 := list(filepath.Join(baseDir, "known"))
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+2 -2
View File
@@ -32,8 +32,8 @@ func TestNoModuleNamesAnothersState(t *testing.T) {
}
}
// And neither list gains a line the trunk's copies at the merge base do not hold, unless the line says why after `# allow:`. Skipped,
// and said, where no trunk is beside.
// And neither list gains a line the trunk's copies at the merge base do not hold, unless the line says why after `# allow:`. Fails, saying
// NOT COMPARED, where no trunk is beside (issue 1295).
func TestTheListsOfPathsHoldNoLineTheTrunksDoNot(t *testing.T) {
sharedPath, okShared := trunk.Copy(t, "checks/paths/shared")
knownPath, okKnown := trunk.Copy(t, "checks/paths/known")
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+14 -18
View File
@@ -3,14 +3,14 @@
//
// The checker fetches the change's own repository at its base commit beside the change, as it fetches a sibling, and says where
// in MESH_CHECK_BESIDE; the copy is under the repository's name (the same convention the other repositories' tests use). With
// MESH_CHECK_BESIDE unset, or set without the catalogue's copy in it, the comparison is skipped and the test says NOT COMPARED
// WITH THE TRUNK, naming why: the checker that runs before R4g's delivery places siblings but not the change's own repository,
// so R4g's own check cannot hold the copy (issue 1295 notes that a missing copy becomes a failure once that checker is gone). A trunk that holds no such file had
// MESH_CHECK_BESIDE unset, or set without the catalogue's copy in it, the test fails saying NOT COMPARED WITH THE TRUNK and
// why: the checker places the change's own repository since R4g (issue 1295), so a missing copy is a fault, never a pass. A trunk that holds no such file had
// none: the list's first copy, which nothing was added to.
package trunk
import (
"errors"
"fmt"
"os"
"path/filepath"
"testing"
@@ -23,13 +23,11 @@ const Repository = "mesh-catalog"
const BesideEnv = "MESH_CHECK_BESIDE"
// Copy is the trunk's copy of the file at rel (a path from the catalogue's root), written to a file of the test's own: its path,
// and whether the trunk holds one (false: its first copy). It skips the test, saying so, when no copy of the catalogue is beside.
// and whether the trunk holds one (false: its first copy). It fails the test when no copy of the catalogue is beside.
func Copy(t testing.TB, rel string) (string, bool) {
t.Helper()
raw, found, skip, err := Read(rel)
raw, found, err := Read(rel)
switch {
case skip != "":
t.Skip(skip)
case err != nil:
t.Fatal(err)
case !found:
@@ -43,27 +41,25 @@ func Copy(t testing.TB, rel string) (string, bool) {
return path, true
}
// Read is the trunk's copy of rel and whether the trunk holds one. skip is the words to skip the comparison with, when no trunk is
// named; err is a trunk named and not there, or unreadable.
func Read(rel string) (content []byte, found bool, skip string, err error) {
// Read is the trunk's copy of rel and whether the trunk holds one. err is a trunk not beside the check (MESH_CHECK_BESIDE unset,
// or without the catalogue's directory), or a copy unreadable: a comparison that cannot be made is never a pass.
func Read(rel string) (content []byte, found bool, err error) {
beside := os.Getenv(BesideEnv)
if beside == "" {
return nil, false, "NOT COMPARED WITH THE TRUNK: " + BesideEnv + " is not set, so the trunk's copy of " + rel +
" is not beside this run; the checker places it", nil
return nil, false, fmt.Errorf("NOT COMPARED WITH THE TRUNK: %s is not set, so the trunk's copy of %s is not beside this run; the checker places the change's own repository there (issue 1295)", BesideEnv, rel)
}
dir := filepath.Join(beside, Repository)
if _, err := os.Stat(dir); errors.Is(err, os.ErrNotExist) {
return nil, false, "NOT COMPARED WITH THE TRUNK: " + Repository + " is not beside this check in " + BesideEnv + "=" + beside +
", so " + rel + " is not judged against the trunk's copy (the checker before R4g places no copy of the change's own repository)", nil
return nil, false, fmt.Errorf("NOT COMPARED WITH THE TRUNK: %s is not beside this check in %s=%s, so %s is not judged against the trunk's copy; the checker places the change's own repository there at its trunk (issue 1295)", Repository, BesideEnv, beside, rel)
} else if err != nil {
return nil, false, "", err
return nil, false, err
}
raw, err := os.ReadFile(filepath.Join(dir, rel))
if errors.Is(err, os.ErrNotExist) {
return nil, false, "", nil
return nil, false, nil
}
if err != nil {
return nil, false, "", err
return nil, false, err
}
return raw, true, "", nil
return raw, true, nil
}
+14 -14
View File
@@ -17,27 +17,27 @@ func TestTheCloneBesideTheCheckIsTheTrunk(t *testing.T) {
t.Fatal(err)
}
t.Setenv(BesideEnv, beside)
raw, found, skip, err := Read("checks/x/known")
if err != nil || skip != "" || !found || string(raw) != "a\nb\n" {
t.Fatalf("read %q found %v skip %q err %v", raw, found, skip, err)
raw, found, err := Read("checks/x/known")
if err != nil || !found || string(raw) != "a\nb\n" {
t.Fatalf("read %q found %v err %v", raw, found, err)
}
if _, found, skip, err := Read("checks/x/absent"); err != nil || skip != "" || found {
t.Fatalf("a file the trunk does not hold: found %v skip %q err %v; wanted its first copy", found, skip, err)
if _, found, err := Read("checks/x/absent"); err != nil || found {
t.Fatalf("a file the trunk does not hold: found %v err %v; wanted its first copy", found, err)
}
}
func TestWithNothingBesideTheComparisonIsSkippedAndSaidNotMade(t *testing.T) {
func TestWithNothingBesideTheComparisonFailsAndSaysNotMade(t *testing.T) {
t.Setenv(BesideEnv, "")
_, found, skip, err := Read("checks/x/known")
if found || err != nil || len(skip) < 12 || skip[:12] != "NOT COMPARED" {
t.Fatalf("found %v, skip %q, err %v; wanted NOT COMPARED", found, skip, err)
_, found, err := Read("checks/x/known")
if found || err == nil || !strings.Contains(err.Error(), "NOT COMPARED") {
t.Fatalf("found %v, err %v; wanted a failure saying NOT COMPARED", found, err)
}
}
func TestACloneBesideWithoutTheCatalogueIsSkippedAndSaidNotMade(t *testing.T) {
t.Setenv(BesideEnv, t.TempDir()) // the siblings are there, the catalogue's own base is not (the checker before R4g)
_, found, skip, err := Read("checks/x/known")
if found || err != nil || len(skip) < 12 || skip[:12] != "NOT COMPARED" || !strings.Contains(skip, "not beside") {
t.Fatalf("found %v, skip %q, err %v; wanted NOT COMPARED naming the missing copy", found, skip, err)
func TestACloneBesideWithoutTheCatalogueFailsAndSaysNotMade(t *testing.T) {
t.Setenv(BesideEnv, t.TempDir()) // the siblings are there, the catalogue's own base is not
_, found, err := Read("checks/x/known")
if found || err == nil || !strings.Contains(err.Error(), "NOT COMPARED") || !strings.Contains(err.Error(), "not beside") {
t.Fatalf("found %v, err %v; wanted a failure naming the missing copy", found, err)
}
}
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -65,6 +65,7 @@ func configFromEnv(getenv func(string) string) (Config, error) {
HostFile: getenv("MESH_FORGE_HOST_FILE"),
OriginsFile: getenv("MESH_GITEA_ORIGINS_FILE"),
Port: port,
HostKeyFile: getenv("MESH_GITEA_HOST_KEY_FILE"), // optional
Forge: forgehttp.Client{URL: strings.TrimRight(getenv("MESH_GITEA_URL"), "/"), User: getenv("MESH_GITEA_ADMIN_USER"),
PasswordFile: getenv("MESH_GITEA_ADMIN_PASSWORD_FILE"), Timeout: 20 * time.Second},
Runner: execRunner{},
@@ -44,10 +44,13 @@ type Config struct {
// origins, one per line.
HostFile, OriginsFile string
// Port is where the forge's sshd is published on the mesh.
Port int
Forge forgehttp.Client
Runner Runner
Log func(format string, a ...any)
Port int
// HostKeyFile is the forge's own ed25519 host key (the .pub file in its data directory). It is read first; when it
// is unset or unreadable the forge's sshd is asked with ssh-keyscan instead (issue 1939).
HostKeyFile string
Forge forgehttp.Client
Runner Runner
Log func(format string, a ...any)
}
// Service answers the consumers of the provision `forge-account` (novox/hq ADR 0399, ADR 0400; issue 955): for each
@@ -448,14 +451,30 @@ func (s *Service) httpsBase() (string, error) {
return "", fmt.Errorf("%s names no https origin", s.cfg.OriginsFile)
}
// scan reads the forge's ed25519 host key from its own sshd and writes it as the machines' known_hosts line.
// scan gives the forge's ed25519 host key as the machines' known_hosts line. The key the forge itself holds is read
// from its file first; only when that cannot be read is the forge's sshd asked over the network, and when that fails
// too (the forge's container is restarting) the key read before is kept, as a host key does not change (issue 1939).
func (s *Service) scan(ctx context.Context) (string, error) {
host, err := s.hostName()
if err != nil {
return "", err
}
if s.cfg.HostKeyFile != "" {
raw, ferr := os.ReadFile(s.cfg.HostKeyFile)
if ferr == nil {
if line, perr := knownHostLine("127.0.0.1 "+strings.TrimSpace(string(raw)), host, s.cfg.Port); perr == nil {
return line, nil
}
}
}
out, err := s.cfg.Runner.Run(ctx, "ssh-keyscan", "-t", "ed25519", "-p", fmt.Sprint(s.cfg.Port), "127.0.0.1")
if err != nil {
s.mu.Lock()
kept := s.known
s.mu.Unlock()
if kept != "" {
return kept, nil
}
return "", fmt.Errorf("the forge's ssh host key cannot be read: %w", err)
}
return knownHostLine(string(out), host, s.cfg.Port)
@@ -696,3 +696,42 @@ func TestNothingIsSaidForAPerNodeUserWithoutMeshKeys(t *testing.T) {
}
}
}
// The forge's own host key file is read first: no scan over the network is needed (issue 1939).
func TestTheHostKeyIsReadFromTheForgesOwnFile(t *testing.T) {
r := newRig(t)
f := filepath.Join(t.TempDir(), "ssh_host_ed25519_key.pub")
if err := os.WriteFile(f, []byte("ssh-ed25519 AAAAC3FILEKEY forge@host\n"), 0o644); err != nil {
t.Fatal(err)
}
r.svc.cfg.HostKeyFile = f
r.runner.err = errors.New("sshd down")
got, err := r.svc.scan(context.Background())
if err != nil || got != "[novox.internal]:222 ssh-ed25519 AAAAC3FILEKEY" {
t.Fatalf("scan = %q, %v", got, err)
}
if r.runner.args != nil {
t.Errorf("the forge's sshd was asked: %v", r.runner.args)
}
}
// A file that cannot be read falls back to the forge's sshd.
func TestAnUnreadableHostKeyFileFallsBackToTheScan(t *testing.T) {
r := newRig(t)
r.svc.cfg.HostKeyFile = filepath.Join(t.TempDir(), "missing.pub")
got, err := r.svc.scan(context.Background())
if err != nil || got != "[novox.internal]:222 ssh-ed25519 AAAAC3HOSTKEY" {
t.Fatalf("scan = %q, %v", got, err)
}
}
// While the forge restarts, a key read before is kept: the sshd is not up, the key has not changed.
func TestARestartingForgeKeepsTheKeyReadBefore(t *testing.T) {
r := newRig(t)
r.svc.known = "[novox.internal]:222 ssh-ed25519 AAAAC3HOSTKEY"
r.runner.err = errors.New("connection refused")
got, err := r.svc.scan(context.Background())
if err != nil || got != r.svc.known {
t.Fatalf("scan = %q, %v", got, err)
}
}
+5 -1
View File
@@ -589,7 +589,11 @@ if (gitea) {
};
// **The forge's events first, the polls as the safety net** (novox/hq issue 729): while verified deliveries arrive the
// polls slow to every five minutes and announce only what no event said (Pace); otherwise they keep 30 s.
await listenForForge(client);
// **Not awaited** (novox/hq issue 1726): the receiver's set-up asks the forge's admin API, and the runtime kills a bundle
// whose import has not ended in its 10 s handshake and never starts it again; a forge slow to answer while every bundle
// of the node starts left gitea_commit_status, gitea_merge_pull_request and gitea_events served nowhere. Until it is
// listening, Pace keeps the polls at 30 s, so nothing is missed meanwhile; listenForForge says its own failure.
void listenForForge(client);
// Main takes no push and only the tools' account merges, in every repository, kept and not only when a token is
// minted: at the start and every ten minutes after, so a repository made later gets it too (issue 948, ADR 0398 §2).
// Only while the setting keep-main-rules is true (mainrules.ts); false, the default, leaves every repository's rule as set.
+1
View File
@@ -537,6 +537,7 @@
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
"MESH_GITEA_ORIGINS_FILE": "${dir:mesh-state}/origins",
"MESH_GITEA_SSH_PORT": "222",
"MESH_GITEA_HOST_KEY_FILE": "${dir:data}/ssh/ssh_host_ed25519_key.pub",
"MESH_FORGE_HOST_FILE": "/etc/gitea/forge-host"
}
}
+50
View File
@@ -109,3 +109,53 @@ test("a bundle whose read of the checker's owed is granted late writes the lates
assert.match(got.statuses[0], /"state":"success".*pass at 4/);
assert.match(err, /mesh-delivery-checker\.owed could not be bound/, "the refusal is said");
});
// **The bundle's import does not wait on the forge** (novox/hq issue 1726, 2026-10-24 12:37). The webhook receiver's set-up
// asks the forge's admin API for its system webhook; awaited at the top of index.ts, a forge slow to answer (busy while
// every bundle of the node starts) held the import past node-tools' 10 s handshake, the bundle was killed and never
// started again, and gitea_commit_status, gitea_merge_pull_request and gitea_events were served nowhere: the forge
// bridge's calls found no responders and no new pull request was announced. Here the forge stand-in takes every
// request and never answers: the import resolves at once all the same, with the bundle's tools registered.
test("importing the bundle resolves, its tools registered, while the forge never answers the webhook's set-up", async () => {
const index = pathToFileURL(join(here, "..", "dist", "index.js")).href;
const script = `
const http = await import("node:http");
const fs = await import("node:fs");
const os = await import("node:os");
const path = await import("node:path");
const forge = http.createServer(() => {});
await new Promise((r) => forge.listen(0, "127.0.0.1", r));
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "gitea-1726-"));
fs.writeFileSync(path.join(dir, "admin.password"), "pw");
process.env.MESH_GITEA_URL = "http://127.0.0.1:" + forge.address().port;
process.env.MESH_GITEA_STATE_DIR = dir;
process.env.MESH_GITEA_ADMIN_PASSWORD_FILE = path.join(dir, "admin.password");
process.env.MESH_GITEA_HOOK_PORT = "0";
process.env.MESH_GITEA_HOOK_HOST = "127.0.0.1";
const { useBroker } = await import("@novox/mesh-sdk/messaging");
useBroker(() => ({
subscribe: async () => () => {}, publish: async () => {}, request: async () => { throw new Error("no responders"); },
handle: async () => () => {}, close: async () => {},
}));
await import(${JSON.stringify(index)});
const { collectTools } = await import("@novox/mesh-sdk/tools");
console.log(JSON.stringify({ imported: true, tools: collectTools().flatMap((r) => r.tools.map((t) => t.name)) }));
process.exit(0);`;
const env = { ...process.env, MESH_GITEA_TOKEN: "x", MESH_GITEA_ADMIN_USER: "admin", MESH_GITEA_HOOK_URL: "http://127.0.0.1:1" };
for (const k of ["MESH_SERVED_MODULE", "MESH_GITEA_ADMIN_PASSWORD", "MESH_GITEA_CONFIG_FILE"]) delete env[k];
const child = spawn(process.execPath, ["--input-type=module", "-e", script], { cwd: join(here, ".."), env, stdio: ["ignore", "pipe", "pipe"] });
let out = "";
let err = "";
child.stdout.on("data", (d) => (out += d));
child.stderr.on("data", (d) => (err += d));
const ended = new Promise<void>((r) => child.on("exit", () => r()));
const timer = setTimeout(() => child.kill("SIGKILL"), 3000);
await ended;
clearTimeout(timer);
const line = out.split("\n").find((l) => l.startsWith("{"));
assert.ok(line, `the import did not resolve within 3 s while the forge did not answer; stdout: ${out.slice(0, 400)} stderr: ${err.slice(0, 600)}`);
const got = JSON.parse(line!) as { tools: string[] };
for (const t of ["gitea_events", "gitea_commit_status", "gitea_merge_pull_request"]) {
assert.ok(got.tools.includes(t), `${t} is registered: ${got.tools}`);
}
});
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+5 -1
View File
@@ -102,7 +102,11 @@ integration level, the postgres major and the nats-server version; and, when the
workspace (`beside`), each one's repository and commit, which the answer echoes as `beside` (the checker keeps a verdict only
for the siblings it was judged with). The text's form is `DigestVersion` 3. The Go version is not in it: the toolchain enters a
fingerprint once, as the version the build nodes agree on (ADR 0477 §3), and the standard library's packages are that
version's. A package with no tests at the level has no digest and is named in `without_tests`. A directory that vendors
version's. A package with no tests at the level has no digest and is named in `without_tests`. A catalogue module's packages also hash the module's non-Go files (the nearest ancestor directory holding `module.json`: its manifest, README, `files/` and the rest, so `../../module.json` is covered; digest form 5). A package whose directory holds a file named `reads-outside` (any content; say what the tests read) is never kept (issue 1935): its tests read
files outside it, which a digest from `go list` cannot name, so its digest is a fresh random one on every call, the ask finds no kept
verdict for it and runs it, and the answer names it in `reads_outside` and the module's log says why. The checker still writes a
verdict under that digest, which nothing ever reads again. The marker is the plain way to say it until each verdict keeps the files
its tests opened (Go's test log). A directory that vendors
is listed and tested from its vendor directory (`-mod=vendor`).
## A change to this module is judged by its candidate
@@ -116,7 +116,7 @@ func TestTheIntegrationLevelDigestsUnderBothTags(t *testing.T) {
t.Errorf("the integration digest listed %q", s.command)
}
}
if DigestVersion != "4" {
if DigestVersion != "5" {
t.Errorf("the digest's form changed in kind and is still version %s", DigestVersion)
}
}
@@ -7,6 +7,7 @@ package main
import (
"bytes"
"context"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
@@ -24,12 +25,20 @@ import (
// 2: no Go version in the text; the toolchain enters a fingerprint as the agreed version (ADR 0477 §3). 3: the siblings'
// commits (the lead's decision L5) and the integration level's tags integration,isolation (the checker's agreement).
// 4: the siblings' commits enter only the digest of a package whose tests reach a file naming MESH_CHECK_BESIDE (ADR 0536 §1).
const DigestVersion = "4"
// 5: a catalogue module's non-Go files enter the digest of its packages (issue 1935).
const DigestVersion = "5"
// besideVariable is the name a file bears when it reads where the siblings lie; a package whose tests reach such a file
// depends on the siblings' commits.
const besideVariable = "MESH_CHECK_BESIDE"
// ReadsOutsideMarker is the name of a file in a package's directory that says its tests read files outside it (a
// repository's documents, a sibling directory): such a package is never kept (issue 1935). The digest hashes only what
// the Go toolchain lists, so it cannot name what a test opens elsewhere; a marker file, beside the Go files where a
// reader of the package sees it, is the plain way to say so until the checker keeps the files a test opened (the Go
// test log) with each verdict.
const ReadsOutsideMarker = "reads-outside"
// integrationTags are the build tags of the integration level, its run and its digest alike: integration, and isolation
// for the bubblewrap proofs (mesh-sdk's and the build modules'), which a build node can run.
var integrationTags = []string{"integration", "isolation"}
@@ -89,6 +98,9 @@ type DigestResult struct {
// only when they are the siblings it gave the fetch.
Beside []string `json:"beside,omitempty"`
WithoutTests []string `json:"without_tests,omitempty"`
// ReadsOutside names the packages that carry the marker: each has a digest that no kept verdict can have (a fresh
// random one), so the check runs it every time.
ReadsOutside []string `json:"reads_outside,omitempty"`
// JudgedBy is the program that ran the digest (ADR 0535 §5): "installed", or "candidate <merge commit>" under the
// runner workspace.
JudgedBy string `json:"judged_by,omitempty"`
@@ -199,6 +211,12 @@ func (m *Module) digestHere(ctx context.Context, workspace, level string) (Diges
}
}
sort.Strings(res.WithoutTests)
sort.Strings(res.ReadsOutside)
for _, p := range res.ReadsOutside {
if m.Say != nil {
m.Say("the %s level of %s always runs: its directory carries %s, so its tests read files outside it that no digest names", level, p, ReadsOutsideMarker)
}
}
return res, nil
}
@@ -256,6 +274,17 @@ func digestModule(res *DigestResult, pkgs map[string]*goPackage, tree string, he
if testMain == nil {
return fmt.Errorf("go list gave no test binary for %s", path)
}
if _, err := os.Stat(filepath.Join(root.Dir, ReadsOutsideMarker)); err == nil {
// The digest of a package that reads outside itself is one nothing can have kept: the key is a fresh random
// hex, so the ask finds no kept verdict and runs it, and what the checker writes under it is never read again.
var nonce [32]byte
if _, err := rand.Read(nonce[:]); err != nil {
return err
}
res.Packages[path] = hex.EncodeToString(nonce[:])
res.ReadsOutside = append(res.ReadsOutside, path)
continue
}
d, err := packageDigest(root, testMain, pkgs, tree, header, beside)
if err != nil {
return fmt.Errorf("%s: %w", path, err)
@@ -303,6 +332,15 @@ func packageDigest(root, testMain *goPackage, pkgs map[string]*goPackage, tree s
}
}
}
// A catalogue module's tests read its manifest, its README and its files from the package's directory (../../module.json):
// every non-Go file of the module's directory is an input (issue 1935). The module is the nearest ancestor with a module.json.
if modDir := enclosingModule(tree, root.Dir); modDir != "" {
more, err := moduleFileLines(tree, root, modDir)
if err != nil {
return "", err
}
lines = append(lines, more...)
}
data := filepath.Join(root.Dir, "testdata")
if _, err := os.Stat(data); err == nil {
err := filepath.WalkDir(data, func(path string, e fs.DirEntry, err error) error {
@@ -328,6 +366,41 @@ func packageDigest(root, testMain *goPackage, pkgs map[string]*goPackage, tree s
return hex.EncodeToString(h.Sum(nil)), nil
}
// enclosingModule is the nearest ancestor of dir, inside tree, that holds a module.json, or "" when there is none.
func enclosingModule(tree, dir string) string {
tree = filepath.Clean(tree)
for d := filepath.Clean(dir); d != tree && strings.HasPrefix(d, tree+string(filepath.Separator)); d = filepath.Dir(d) {
if st, err := os.Stat(filepath.Join(d, "module.json")); err == nil && !st.IsDir() {
return d
}
}
return ""
}
// moduleFileLines are the content lines of every file of a module's directory that is not Go source (Go source is the
// packages' own, listed by go list), skipping .git and node_modules.
func moduleFileLines(tree string, p *goPackage, modDir string) ([]string, error) {
var lines []string
err := filepath.WalkDir(modDir, func(path string, e fs.DirEntry, err error) error {
if err != nil {
return err
}
if e.IsDir() {
if n := e.Name(); n == ".git" || n == "node_modules" {
return filepath.SkipDir
}
return nil
}
if strings.HasSuffix(path, ".go") {
return nil
}
l, err := contentLine(tree, p, path)
lines = append(lines, l)
return err
})
return lines, err
}
func fileLine(tree string, p *goPackage, name string) (string, error) {
return contentLine(tree, p, filepath.Join(p.Dir, name))
}
@@ -189,3 +189,71 @@ func TestAModuleOutsideTheTreeCountsByVersionAndChecksum(t *testing.T) {
t.Errorf("a new version left the digest as it was")
}
}
// Issue 1935: a package whose directory carries the reads-outside marker is never kept: its digest is one no kept verdict
// can have (it differs on every call, though nothing changed), and the answer names it. A package without the marker
// keeps its digest.
func TestAPackageWithTheReadsOutsideMarkerIsNeverKept(t *testing.T) {
t.Parallel()
m, tree := digestFixture(t)
plain := digests(t, m, "unit")
if len(plain.ReadsOutside) != 0 {
t.Fatalf("a package carries the marker in the fixture: %v", plain.ReadsOutside)
}
edit(t, tree, "c/"+ReadsOutsideMarker, "the tests read ../docs\n")
first, second := digests(t, m, "unit"), digests(t, m, "unit")
c := "fixture/digest/c"
if len(first.Packages[c]) != 64 || first.Packages[c] == second.Packages[c] || first.Packages[c] == plain.Packages[c] {
t.Errorf("c carries the marker and its digest is stable: %s, %s, before %s", first.Packages[c], second.Packages[c], plain.Packages[c])
}
if strings.Join(first.ReadsOutside, ",") != c {
t.Errorf("the answer does not name the package that reads outside: %v", first.ReadsOutside)
}
a := "fixture/digest/a"
if first.Packages[a] != plain.Packages[a] || first.Packages[a] != second.Packages[a] {
t.Errorf("a has no marker and its digest moved: %s, %s, before %s", first.Packages[a], second.Packages[a], plain.Packages[a])
}
}
// Issue 1935: a catalogue module's packages read ../../module.json and the module's files: a change to them changes the
// digest of the module's package, and a change to another module's files does not.
func TestACatalogueModulesFilesChangeTheDigestOfItsPackagesOnly(t *testing.T) {
t.Parallel()
m, tree := digestFixture(t)
for _, mod := range []string{"mod1", "mod2"} {
dir := filepath.Join(tree, mod, "cmd", "p")
if err := os.MkdirAll(filepath.Join(tree, mod, "files"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
edit(t, tree, mod+"/module.json", `{"module":"`+mod+`"}`)
edit(t, tree, mod+"/README.md", "# "+mod+"\n")
edit(t, tree, mod+"/files/f.env", "A=1\n")
edit(t, tree, mod+"/cmd/p/p.go", "package p\n")
edit(t, tree, mod+"/cmd/p/p_test.go", "package p\n\nimport (\n\t\"os\"\n\t\"testing\"\n)\n\nfunc TestReads(t *testing.T) {\n\tif _, err := os.ReadFile(\"../../module.json\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n")
}
p1, p2 := "fixture/digest/mod1/cmd/p", "fixture/digest/mod2/cmd/p"
before := digests(t, m, "unit")
if len(before.Packages[p1]) != 64 || len(before.Packages[p2]) != 64 {
t.Fatalf("no digest for the module packages: %+v", before.Packages)
}
if before.Packages[p1] == before.Packages[p2] {
t.Errorf("two modules, one digest")
}
for _, file := range []string{"mod1/module.json", "mod1/README.md", "mod1/files/f.env"} {
edit(t, tree, file, "changed "+file+"\n")
after := digests(t, m, "unit")
if after.Packages[p1] == before.Packages[p1] {
t.Errorf("a change to %s did not change the module's digest", file)
}
if after.Packages[p2] != before.Packages[p2] {
t.Errorf("a change to %s changed the other module's digest", file)
}
before = after
}
if got := digests(t, m, "unit"); got.Packages["fixture/digest/c"] == "" {
t.Errorf("a package outside any module lost its digest")
}
}
@@ -0,0 +1 @@
These tests read other catalogue modules' files, outside this module (issue 1935): never kept.
@@ -103,6 +103,33 @@ func TestUnitLevelNamesTheFailingTestAndFailsTheLevel(t *testing.T) {
}
}
// Issue 1921: a fixture with a failing test in each of two packages: one level's run names both, each with its own
// cause, and counts every test, the passing one beside a failure too. go test runs every package to its end; nothing
// stops at the first failure.
func TestUnitLevelNamesEveryFailingTestOfEveryPackage(t *testing.T) {
m, states, ws, calls := workspaceFor(t, "twofailing")
res, st := runUnit(t, m, states, ws)
if st.State != delivery.StepPassed || res.Passed {
t.Fatalf("step %s (%s), result %+v: want the step done and the level failed", st.State, st.Reason, res)
}
var got []string
for _, f := range res.Failed {
got = append(got, f.Package+"."+f.Test+": "+f.Cause)
}
want := []string{"fixture/twofailing/one.TestOneIsTwo: One() = 1, want 2", "fixture/twofailing/two.TestTwoIsThree: Two() = 2, want 3"}
if strings.Join(got, "\n") != strings.Join(want, "\n") {
t.Errorf("failed\n%s\nwant\n%s", strings.Join(got, "\n"), strings.Join(want, "\n"))
}
if res.Tests != 3 {
t.Errorf("%d tests counted, want 3", res.Tests)
}
for _, c := range *calls {
if strings.Contains(c.line, " test ") && strings.Contains(c.line, "-failfast") {
t.Errorf("go test ran with -failfast: %s", c.line)
}
}
}
// 1290/AC2: a fixture whose tests take more than the bound (lowered by the module's option, not by waiting five
// seconds): the level fails naming the bound and the slowest tests; under the real bound the same run passes.
func TestUnitLevelOverItsBoundFailsNamingTheBoundAndTheSlowestTests(t *testing.T) {
@@ -1,6 +1,7 @@
Each file is the standard output of `GOTOOLCHAIN=go1.27.1 GOFLAGS=-mod=mod go test -count=1 -json ./...` (go1.27.1)
run in the fixture module of the same name under ../repos, captured on 2026-10-20 for issue 1290. The unit
tests feed them to the parser through a fake runner, so no test here runs `go test`.
tests feed them to the parser through a fake runner, so no test here runs `go test`. twofailing.json was captured the
same way on 2026-10-24 for issue 1921: two packages with a failing test each, and a passing test beside one of them.
golist-external.json is `GOTOOLCHAIN=go1.27.1 GOFLAGS=-mod=mod go list -json=ImportPath,Standard,DepOnly,Module
git.novox.be/novox/mesh-sdk/go/buildstep` run in this module (go1.27.1, 2026-10-20 for issue 1291): a package of a module
@@ -0,0 +1,23 @@
{"Time":"2026-10-24T23:45:37.721718705+02:00","Action":"start","Package":"fixture/twofailing/one"}
{"Time":"2026-10-24T23:45:37.723685152+02:00","Action":"run","Package":"fixture/twofailing/one","Test":"TestOneIsTwo"}
{"Time":"2026-10-24T23:45:37.723726472+02:00","Action":"output","Package":"fixture/twofailing/one","Test":"TestOneIsTwo","Output":"=== RUN TestOneIsTwo\n","OutputType":"frame"}
{"Time":"2026-10-24T23:45:37.723743682+02:00","Action":"output","Package":"fixture/twofailing/one","Test":"TestOneIsTwo","Output":" one_test.go:6: checking One\n"}
{"Time":"2026-10-24T23:45:37.723750622+02:00","Action":"output","Package":"fixture/twofailing/one","Test":"TestOneIsTwo","Output":" one_test.go:8: One() = 1, want 2\n","OutputType":"error"}
{"Time":"2026-10-24T23:45:37.723759302+02:00","Action":"output","Package":"fixture/twofailing/one","Test":"TestOneIsTwo","Output":"--- FAIL: TestOneIsTwo (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-24T23:45:37.723889303+02:00","Action":"start","Package":"fixture/twofailing/two"}
{"Time":"2026-10-24T23:45:37.723762972+02:00","Action":"fail","Package":"fixture/twofailing/one","Test":"TestOneIsTwo","Elapsed":0}
{"Time":"2026-10-24T23:45:37.723952274+02:00","Action":"run","Package":"fixture/twofailing/one","Test":"TestOneIsOne"}
{"Time":"2026-10-24T23:45:37.723955484+02:00","Action":"output","Package":"fixture/twofailing/one","Test":"TestOneIsOne","Output":"=== RUN TestOneIsOne\n","OutputType":"frame"}
{"Time":"2026-10-24T23:45:37.723959304+02:00","Action":"output","Package":"fixture/twofailing/one","Test":"TestOneIsOne","Output":"--- PASS: TestOneIsOne (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-24T23:45:37.723964914+02:00","Action":"pass","Package":"fixture/twofailing/one","Test":"TestOneIsOne","Elapsed":0}
{"Time":"2026-10-24T23:45:37.723967704+02:00","Action":"output","Package":"fixture/twofailing/one","Output":"FAIL\n","OutputType":"frame"}
{"Time":"2026-10-24T23:45:37.724225126+02:00","Action":"output","Package":"fixture/twofailing/one","Output":"FAIL\tfixture/twofailing/one\t0.002s\n","OutputType":"frame"}
{"Time":"2026-10-24T23:45:37.724231936+02:00","Action":"fail","Package":"fixture/twofailing/one","Elapsed":0.003}
{"Time":"2026-10-24T23:45:37.725752949+02:00","Action":"run","Package":"fixture/twofailing/two","Test":"TestTwoIsThree"}
{"Time":"2026-10-24T23:45:37.725796309+02:00","Action":"output","Package":"fixture/twofailing/two","Test":"TestTwoIsThree","Output":"=== RUN TestTwoIsThree\n","OutputType":"frame"}
{"Time":"2026-10-24T23:45:37.725808239+02:00","Action":"output","Package":"fixture/twofailing/two","Test":"TestTwoIsThree","Output":" two_test.go:7: Two() = 2, want 3\n","OutputType":"error"}
{"Time":"2026-10-24T23:45:37.72581671+02:00","Action":"output","Package":"fixture/twofailing/two","Test":"TestTwoIsThree","Output":"--- FAIL: TestTwoIsThree (0.00s)\n","OutputType":"frame"}
{"Time":"2026-10-24T23:45:37.72582026+02:00","Action":"fail","Package":"fixture/twofailing/two","Test":"TestTwoIsThree","Elapsed":0}
{"Time":"2026-10-24T23:45:37.72582549+02:00","Action":"output","Package":"fixture/twofailing/two","Output":"FAIL\n","OutputType":"frame"}
{"Time":"2026-10-24T23:45:37.726319344+02:00","Action":"output","Package":"fixture/twofailing/two","Output":"FAIL\tfixture/twofailing/two\t0.002s\n","OutputType":"frame"}
{"Time":"2026-10-24T23:45:37.726362544+02:00","Action":"fail","Package":"fixture/twofailing/two","Elapsed":0.002}
@@ -0,0 +1,3 @@
module fixture/twofailing
go 1.25.0
@@ -0,0 +1,3 @@
package one
func One() int { return 1 }
@@ -0,0 +1,16 @@
package one
import "testing"
func TestOneIsTwo(t *testing.T) {
t.Log("checking One")
if One() != 2 {
t.Fatalf("One() = %d, want 2", One())
}
}
func TestOneIsOne(t *testing.T) {
if One() != 1 {
t.Fatal("one")
}
}
@@ -0,0 +1,3 @@
package two
func Two() int { return 2 }
@@ -0,0 +1,9 @@
package two
import "testing"
func TestTwoIsThree(t *testing.T) {
if Two() != 3 {
t.Errorf("Two() = %d, want 3", Two())
}
}
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildgo
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024021031-66bf26c0d721
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024214154-bc16740feb0d
github.com/distribution/distribution/v3 v3.1.2
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
+2 -4
View File
@@ -1,7 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023213200-403a5d72ab93 h1:FAiFY2KBKEoZSH49OZTt7/zRBIQxyl5g/KhYrpggbN4=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023213200-403a5d72ab93/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024021031-66bf26c0d721 h1:N//dDGmry1F3HKeBTWd+z/SfK+wb8bUUMJF0OlgR0/g=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024021031-66bf26c0d721/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024214154-bc16740feb0d h1:oJREw0ggKqJWJ5DEISSL0i2ZC5XqdXAidxo9NJ74TCc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024214154-bc16740feb0d/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/alicebob/miniredis/v2 v2.35.0 h1:QwLphYqCEAo1eu1TqPRN2jgVMPBweeQcR21jeqDCONI=
github.com/alicebob/miniredis/v2 v2.35.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
@@ -0,0 +1 @@
These tests read other catalogue modules' files, outside this module (issue 1935): never kept.
@@ -0,0 +1 @@
These tests read other catalogue modules' files, outside this module (issue 1935): never kept.
+1 -1
View File
@@ -3,7 +3,7 @@ module meshbuildnode
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023213200-403a5d72ab93
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024214154-bc16740feb0d
github.com/distribution/distribution/v3 v3.1.2
github.com/nats-io/nats.go v1.54.0
)
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023213200-403a5d72ab93 h1:FAiFY2KBKEoZSH49OZTt7/zRBIQxyl5g/KhYrpggbN4=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261023213200-403a5d72ab93/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024214154-bc16740feb0d h1:oJREw0ggKqJWJ5DEISSL0i2ZC5XqdXAidxo9NJ74TCc=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261024214154-bc16740feb0d/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/alicebob/miniredis/v2 v2.35.0 h1:QwLphYqCEAo1eu1TqPRN2jgVMPBweeQcR21jeqDCONI=
github.com/alicebob/miniredis/v2 v2.35.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
@@ -54,6 +54,9 @@ type Holder struct {
seatMu sync.Mutex
seatWhy string
// catchUp asks the next look to weigh again the newest merge of each repository against source.main (SourceCatchUp).
catchUp bool
mu sync.Mutex
deliveries map[string]*Delivery
groups map[string]*Group
@@ -1845,6 +1848,12 @@ func (h *Holder) adopt(w Walk) {
now := h.Now()
d := &Delivery{ID: IDOf(w.Repository, w.Commit), Repository: strings.ToLower(w.Repository), Commit: w.Commit,
Base: w.Branch, MergedAs: w.Commit, Created: now}
// Dated once, by the walk it is adopted from, which follows the merge: a later walk it follows (linkWalk) never redates
// it, so it never looks newer than a merge heard after it (issue 1726, review).
d.MergedAt = w.Created.UTC()
if w.Created.IsZero() {
d.MergedAt = now
}
if old := h.deliveries[d.ID]; old != nil {
if old.State.Final() {
// Never revived: an ended delivery is not the walk's; the walk, held by nothing, is listed by
@@ -0,0 +1,161 @@
package main
import (
"errors"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
sdk "git.novox.be/novox/mesh-sdk/go/delivery"
)
// Issue 1726 (2026-10-24, mesh-catalog #471 and #472): the checker takes a repository's trunk head only from source.main
// (mesh-delivery modules/mesh-delivery-checker main.go), and the gate moves source.main only on a merge it hears. While
// gitea's watcher was dead, the operator merged #471; the restarted watcher's poll announced its pull.merged while the gate
// was starting, before it could tell whether it held the seat, and the gate acknowledged it unread. The gate then adopted the
// controller's walk of the merge commit, which has no merge time, so every older merge of the repository looked newer and
// source.main was never moved: the checker judged #472 on the trunk before #471, and the gate refused its merge.
const (
i1726Repo = "novox/mesh-catalog"
i1726Head1 = "aaaa000000000000000000000000000000000001"
i1726Merge1 = "dddd000000000000000000000000000000000002"
i1726Merge2 = "bbbb000000000000000000000000000000000003"
)
// 1726c/AC1: a merge into main whose pull.merged the gate never heard, caught up as the controller's walk of its merge
// commit, moves source.main as the forge's word of it would.
func TestAMergeCaughtUpAsAnAdoptedWalkMovesSourceMain(t *testing.T) {
w := newWorld(t)
w.h.PullUpdated(pr(i1726Repo, 470, i1726Head1, "fix/a", "modules/app/a.go"))
w.h.PullMerged(i888MergedAt(i1726Repo, 470, i1726Head1, i1726Merge1, w.now.Add(-time.Hour)))
w.settleAll()
if w.source.main == nil || w.source.main.Heads[i1726Repo] != i1726Merge1 {
t.Fatalf("the heard merge did not move source.main: %+v", w.source.main)
}
// #471's pull.merged is lost; the controller walks its merge commit and the gate adopts the walk.
walk := aWalk("plan-471", i1726Repo, i1726Merge2, false)
walk.Created = w.now.Add(-time.Minute)
walk.State = walkRolling
w.ctl.put(walk)
if err := w.h.Reconcile(); err != nil {
t.Fatal(err)
}
w.settleAll()
if got := w.source.main.Heads[i1726Repo]; got != i1726Merge2 {
t.Fatalf("source.main holds %s at %s after the walk of %s was adopted; wanted the adopted merge", i1726Repo, got, i1726Merge2)
}
}
// 1726c/AC2: a merge already marked in the source without being written (as the gate before this fix left #471) is written
// when the gate next holds the seat; one older than the head source.main holds never moves it back.
func TestHoldingTheSeatWritesTheNewestMergeOfEachRepositoryNotYetInSourceMain(t *testing.T) {
w := newWorld(t)
w.h.PullUpdated(pr(i1726Repo, 470, i1726Head1, "fix/a", "modules/app/a.go"))
w.h.PullMerged(i888MergedAt(i1726Repo, 470, i1726Head1, i1726Merge1, w.now.Add(-time.Hour)))
w.settleAll()
walk := aWalk("plan-471", i1726Repo, i1726Merge2, false)
walk.Created = w.now.Add(-time.Minute)
walk.State = walkRolling
w.ctl.put(walk)
if err := w.h.Reconcile(); err != nil {
t.Fatal(err)
}
w.settleAll()
// What the gate before this fix left: the adopted merge marked in the source, source.main still at the merge before.
if d := w.delivery(IDOf(i1726Repo, i1726Merge2)); !d.InSource {
t.Fatalf("the adopted merge is not marked in the source: %+v", d)
}
w.source.main = &sdk.Source{V: sdk.Version, Group: "before", Heads: map[string]string{i1726Repo: i1726Merge1, "novox/other": "b0"}}
w.settleAll()
if got := w.source.main.Heads[i1726Repo]; got != i1726Merge1 {
t.Fatalf("source.main moved to %s before the gate held the seat again", got)
}
w.h.SourceCatchUp()
w.settleAll()
if got := w.source.main.Heads[i1726Repo]; got != i1726Merge2 || w.source.main.Heads["novox/other"] != "b0" {
t.Fatalf("source.main is %+v after the catch-up; wanted %s at %s and the other kept", w.source.main.Heads, i1726Repo, i1726Merge2)
}
n := len(w.source.moves)
w.h.SourceCatchUp()
w.settleAll()
if len(w.source.moves) != n {
t.Fatalf("a second catch-up moved source.main again: %d moves, then %d", n, len(w.source.moves))
}
o := newWorld(t)
o.source.main = &sdk.Source{V: sdk.Version, Group: "newer", Heads: map[string]string{i1726Repo: i1726Merge2}}
o.h.PullUpdated(pr(i1726Repo, 471, "cccc000000000000000000000000000000000001", "fix/b", "modules/app/b.go"))
o.h.PullMerged(i888MergedAt(i1726Repo, 471, "cccc000000000000000000000000000000000001", i1726Merge2, o.now.Add(-time.Minute)))
o.h.PullMerged(i888MergedAt(i1726Repo, 470, i1726Head1, i1726Merge1, o.now.Add(-time.Hour)))
o.settleAll()
o.h.SourceCatchUp()
o.settleAll()
if got := o.source.main.Heads[i1726Repo]; got != i1726Merge2 {
t.Fatalf("the catch-up moved source.main back to %s", got)
}
}
// seatUnknown is a seat whose occupants cannot be read yet.
type seatUnknown struct{}
func (seatUnknown) Occupants() ([]string, error) {
return nil, errors.New("the watches have not delivered yet")
}
// seatTaken is a seat today's module still stands on.
type seatTaken struct{}
func (seatTaken) Occupants() ([]string, error) { return []string{"assigned on elsewhere"}, nil }
// 1726c/AC3: an event that reaches the gate before it can tell whether it holds the seat is offered again, never
// acknowledged unread; one that reaches a gate whose seat another holds is acknowledged and left, as before.
func TestAnEventBeforeTheGateKnowsWhetherItHoldsTheSeatIsOfferedAgain(t *testing.T) {
w := newWorld(t)
w.h.Seat = seatUnknown{}
_ = w.h.SeatChanged()
ev := stdio.Envelope{Key: "gitea.pull.merged", Body: mustJSON(i888MergedAt(i1726Repo, 471, i1726Head1, i1726Merge2, w.now))}
if err := handle(w.h, ev); err == nil {
t.Fatal("an event heard before the gate could tell whether it holds the seat was taken: acknowledged unread")
}
w.h.Seat = seatTaken{}
_ = w.h.SeatChanged()
if err := handle(w.h, ev); err != nil {
t.Fatalf("an event heard while today's module holds the seat was offered again: %v", err)
}
}
// 1726c/AC4 (review round 1): an adopted merge that later follows a newer walk keeps the date of the walk it was adopted
// from, so a merge heard from the forge after it stays in source.main through a catch-up.
func TestAnAdoptedMergeFollowingANewerWalkNeverMovesSourceMainBack(t *testing.T) {
w := newWorld(t)
walk := aWalk("plan-471", i1726Repo, i1726Merge2, false)
walk.Created = w.now.Add(-10 * time.Minute)
walk.State = walkRolling
w.ctl.put(walk)
if err := w.h.Reconcile(); err != nil {
t.Fatal(err)
}
w.settleAll()
if got := w.source.main.Heads[i1726Repo]; got != i1726Merge2 {
t.Fatalf("the adopted merge did not move source.main: %s", got)
}
const later = "eeee000000000000000000000000000000000004"
w.h.PullUpdated(pr(i1726Repo, 473, "ffff000000000000000000000000000000000001", "fix/c", "modules/app/c.go"))
w.h.PullMerged(i888MergedAt(i1726Repo, 473, "ffff000000000000000000000000000000000001", later, w.now.Add(-5*time.Minute)))
w.settleAll()
if got := w.source.main.Heads[i1726Repo]; got != later {
t.Fatalf("the later forge merge did not move source.main: %s", got)
}
again := aWalk("plan-471-again", i1726Repo, i1726Merge2, false)
again.Created = w.now.Add(-time.Minute)
again.State = walkRolling
w.h.mu.Lock()
w.h.linkWalk(w.h.deliveries[IDOf(i1726Repo, i1726Merge2)], again, i1726Merge2)
w.h.mu.Unlock()
w.h.SourceCatchUp()
w.settleAll()
if got := w.source.main.Heads[i1726Repo]; got != later {
t.Fatalf("the catch-up moved source.main back to the adopted merge %s after it followed a newer walk", got)
}
}
@@ -147,6 +147,7 @@ func run(h *Holder, l *listening, occupancy *busOccupancy) {
func readOnHolding(h *Holder) {
h.wantForge()
h.hearAsks()
h.SourceCatchUp() // a merge missed or marked unwritten meanwhile reaches source.main, and so the checker (issue 1726)
h.Poke()
for wait := 2 * time.Second; ; wait = min(wait*2, time.Minute) {
err := h.Reconcile()
@@ -162,6 +163,11 @@ func readOnHolding(h *Holder) {
// handle takes one event, then asks for a look at every step the event makes possible (Work, React). An event that cannot be read is
// said and acknowledged: read again it would fail again, and the controller's walks are read back by comparison anyway.
func handle(h *Holder, e stdio.Envelope) error {
if h.Undecided() {
// Not known yet whether this gate holds the seat (starting, or carrying today's state): offered again in a few
// seconds, so an event heard at a restart is not lost (issue 1726).
return fmt.Errorf("%s: the gate cannot tell yet whether it holds the seat mesh-delivery; offered again", e.Key)
}
if !h.Holding() {
return nil // not the holder of the seat: the event is acknowledged and taken by no one (seat.go)
}
@@ -433,10 +433,20 @@ func gateMerges(d *Delivery) bool {
// and the checker judges on the trunk source.main names, so source.main moves too. A merge replayed as history, or older than
// a merge of its repository already heard, is never written: source.main never moves backwards. Under the lock; the writing
// happens outside it.
//
// A merge caught up after a gap (issue 1726) is settled the same way: the controller's walk of a merge commit whose
// pull.merged the gate never heard is adopted with the walk's time as its merge's (adopt, mergedWhen), and once each time the
// gate comes to hold the seat (SourceCatchUp) the newest merge of each repository it holds is weighed again though it is
// marked in the source, so a merge marked there and never written reaches source.main, and with it the checker.
func (h *Holder) settleForgeMerges() []func() {
var out []func()
again := map[*Delivery]bool{}
if h.catchUp {
h.catchUp = false
again = h.newestInSource()
}
for _, d := range h.deliveries {
if d.MergedAs == "" || d.InSource || gateMerges(d) {
if d.MergedAs == "" || (d.InSource && !again[d]) || gateMerges(d) {
continue
}
if d.Base != MergeBranch || d.History != "" || h.newerMergeOf(d) {
@@ -444,7 +454,7 @@ func (h *Holder) settleForgeMerges() []func() {
h.keep(d)
continue
}
id, repository, base, commit, at, group := d.ID, d.Repository, d.Base, d.MergedAs, d.MergedAt, firstOf(d.Branch, d.Repository)
id, repository, base, commit, at, group := d.ID, d.Repository, d.Base, d.MergedAs, mergedWhen(d), firstOf(d.Branch, d.Repository)
out = append(out, func() {
err := h.Source.Move(func(prev *sdk.Source) (sdk.Source, bool) {
next := sdk.Source{Group: group, Heads: map[string]string{repository: commit}}
@@ -485,7 +495,7 @@ func (h *Holder) settleForgeMerges() []func() {
// (Ready.MergeCommit, Ready.Merged), which comes first for a merge the gate handed. Empty when it is not merged.
func mergeOf(d *Delivery) (string, time.Time) {
if d.MergedAs != "" {
return d.MergedAs, d.MergedAt
return d.MergedAs, mergedWhen(d)
}
if d.Ready != nil && d.Ready.Merged != nil && d.Ready.MergeCommit != "" {
return d.Ready.MergeCommit, *d.Ready.Merged
@@ -497,7 +507,7 @@ func mergeOf(d *Delivery) (string, time.Time) {
func (h *Holder) newerMergeOf(d *Delivery) bool {
for _, o := range h.deliveries {
if commit, at := mergeOf(o); o != d && o.Repository == d.Repository && o.Base == d.Base && commit != "" &&
!sameCommit(commit, d.MergedAs) && at.After(d.MergedAt) {
!sameCommit(commit, d.MergedAs) && at.After(mergedWhen(d)) {
return true
}
}
@@ -518,3 +528,44 @@ func (h *Holder) olderHead(repository, base, head string, at time.Time) bool {
}
return at.After(h.Now().Add(-keptFinal))
}
// mergedWhen is when a merged delivery's merge landed: its merge time (the forge's, or for a merge adopted from the
// controller's walk the walk's creation, set once in adopt), else when the gate made the delivery. A zero time made every
// other merge of the repository look newer, so an adopted merge never moved source.main (issue 1726).
func mergedWhen(d *Delivery) time.Time {
if !d.MergedAt.IsZero() {
return d.MergedAt
}
return d.Created
}
// SourceCatchUp asks the next look to weigh again, once, the newest merge into the trunk of each repository though it is
// marked in the source (issue 1726): what the gate missed while it did not hold the seat, or marked without writing, reaches
// source.main. The write never moves a repository back (olderHead), and one already there is not written again.
func (h *Holder) SourceCatchUp() {
h.mu.Lock()
h.catchUp = true
h.mu.Unlock()
h.Poke()
}
// newestInSource is, for each repository, its newest merge into the trunk heard from the forge or caught up from a walk,
// marked in the source and not history. Under the lock.
func (h *Holder) newestInSource() map[*Delivery]bool {
newest := map[string]*Delivery{}
for _, d := range h.deliveries {
if d.MergedAs == "" || !d.InSource || gateMerges(d) || d.Base != MergeBranch || d.History != "" {
continue
}
if o := newest[d.Repository]; o == nil || mergedWhen(d).After(mergedWhen(o)) {
newest[d.Repository] = d
}
}
out := map[*Delivery]bool{}
for _, d := range newest {
if !h.newerMergeOf(d) {
out[d] = true
}
}
return out
}
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -105,6 +105,11 @@ func (h *Holder) SeatChanged() error {
// Holding is whether the gate acts. A holder with no Seat to ask always holds (the tests'; main sets one).
func (h *Holder) Holding() bool { return h.Seat == nil || h.seatHeld.Load() }
// Undecided is whether the gate cannot tell yet whether it holds the seat: it does not hold it, and no reason it waits was
// found (the watches have not delivered, or today's state is being carried). An event heard then is offered again, never
// acknowledged unread (issue 1726: a merge announced at a restart was lost so, and source.main never moved).
func (h *Holder) Undecided() bool { return !h.Holding() && h.SeatWhy() == "" }
// setSeatWhy keeps why the gate waits and says whether it changed.
func (h *Holder) setSeatWhy(why string) bool {
h.seatMu.Lock()
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read other catalogue modules' files, outside this module (issue 1935): never kept.
@@ -0,0 +1 @@
These tests read other catalogue modules' files, outside this module (issue 1935): never kept.
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read the repository's root predefined.json, outside this module (issue 1935): never kept.
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read other catalogue modules' files, outside this module (issue 1935): never kept.
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).