Compare commits
160
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c193d2b389 | ||
|
|
10447d8c1e | ||
|
|
5be6576117 | ||
|
|
24fad2b92a | ||
|
|
d3eb5cae80 | ||
|
|
2043200201 | ||
|
|
ef618e8199 | ||
|
|
c00005c141 | ||
|
|
bd5f170af0 | ||
|
|
6bbb8329e4 | ||
|
|
e6b913cac7 | ||
|
|
411c1b8de8 | ||
|
|
64e6688ec8 | ||
|
|
2d57c9663a | ||
|
|
5c1729523b | ||
|
|
1628db6c80 | ||
|
|
0b9f19b612 | ||
|
|
39ce149f74 | ||
|
|
1e377fe862 | ||
|
|
af89893d42 | ||
|
|
105c4d628c | ||
|
|
59cad7154a | ||
|
|
3b35e78733 | ||
|
|
e072abb35e | ||
|
|
e389e770d9 | ||
|
|
00613b438c | ||
|
|
d1f42564e3 | ||
|
|
5548e1030c | ||
|
|
7bb07ce349 | ||
|
|
4e3b140d43 | ||
|
|
b5aa226649 | ||
|
|
77b511a79a | ||
|
|
d836a31aa7 | ||
|
|
683ce7e37f | ||
|
|
3494548c88 | ||
|
|
3feeefc70f | ||
|
|
3f552a17f6 | ||
|
|
16ea3824b5 | ||
|
|
fabb62da00 | ||
|
|
8068cf74a7 | ||
|
|
6e97ba3867 | ||
|
|
448310908c | ||
|
|
58309a9729 | ||
|
|
6724dab0cf | ||
|
|
387ee59140 | ||
|
|
85bbeb37e3 | ||
|
|
363039f337 | ||
|
|
a9d96b30be | ||
|
|
644a34cf49 | ||
|
|
277a4d0060 | ||
|
|
69056948bf | ||
|
|
9cdac09ad9 | ||
|
|
86d37a14e7 | ||
|
|
d13f74df40 | ||
|
|
dfe490d9e2 | ||
|
|
4f317201c9 | ||
|
|
25f6154564 | ||
|
|
b8ba9dd966 | ||
|
|
002a0e1a12 | ||
|
|
9bb85a37f2 | ||
|
|
774fcdfa93 | ||
|
|
470687ea4c | ||
|
|
2a03c106f4 | ||
|
|
f5d4b2f86e | ||
|
|
7c2aa0a739 | ||
|
|
e057670bc4 | ||
|
|
0c6b7b5d64 | ||
|
|
9fd95693db | ||
|
|
1a9570f5e9 | ||
|
|
d871258f8b | ||
|
|
e7abe3e3e4 | ||
|
|
0974d1a57d | ||
|
|
6cea12a023 | ||
|
|
a17356a162 | ||
|
|
35ad58e6e2 | ||
|
|
da5da4f6f7 | ||
|
|
671a6a1f47 | ||
|
|
47ed384d97 | ||
|
|
3fb7951524 | ||
|
|
ae96afcaa8 | ||
|
|
0c63b4c906 | ||
|
|
354d62340d | ||
|
|
dea390ac97 | ||
|
|
00d155da60 | ||
|
|
89c30c44be | ||
|
|
bd65c839a8 | ||
|
|
b7a036e186 | ||
|
|
3abf7cd50b | ||
|
|
d7e8a8824e | ||
|
|
83e9ce5b19 | ||
|
|
653cead338 | ||
|
|
d1ca199265 | ||
|
|
bf94af0a82 | ||
|
|
476a59162f | ||
|
|
36d5993323 | ||
|
|
f8ddcc98f6 | ||
|
|
9109524502 | ||
|
|
9bd8247915 | ||
|
|
116751130f | ||
|
|
c05aa5e6ea | ||
|
|
b95b0a4750 | ||
|
|
04fa2e3286 | ||
|
|
0c61bad102 | ||
|
|
02597b66d7 | ||
|
|
457c60972c | ||
|
|
5ae337b14d | ||
|
|
2ac511ce6b | ||
|
|
e955b53bf0 | ||
|
|
454ac0fec7 | ||
|
|
3c6f2dc274 | ||
|
|
39419b1892 | ||
|
|
b51f03c4ff | ||
|
|
3d42988afa | ||
|
|
64530a8dcb | ||
|
|
e8a953818c | ||
|
|
f0f7b6737e | ||
|
|
c1cd8461bc | ||
|
|
53bf4bb84c | ||
|
|
4677b8a2f5 | ||
|
|
cc6e478c4f | ||
|
|
c17ab34a28 | ||
|
|
099605ec70 | ||
|
|
488b7bfc58 | ||
|
|
d58249d0d8 | ||
|
|
d224b74ff1 | ||
|
|
234d2d5534 | ||
|
|
78d7ef71cc | ||
|
|
1a738ed4fc | ||
|
|
c085975146 | ||
|
|
ba49a7f0f9 | ||
|
|
be7e98d25c | ||
|
|
b1413949f0 | ||
|
|
eba4bab00a | ||
|
|
216701fcba | ||
|
|
8e116cb0e6 | ||
|
|
dcf32576a8 | ||
|
|
3f8b0b4248 | ||
|
|
7c5c29421f | ||
|
|
9e41438f1b | ||
|
|
3690c149c4 | ||
|
|
671fdb7dc2 | ||
|
|
eab6918459 | ||
|
|
ff2632356b | ||
|
|
1e6b157865 | ||
|
|
5b75233406 | ||
|
|
4548b97910 | ||
|
|
1faf317069 | ||
|
|
245143d948 | ||
|
|
358ef45536 | ||
|
|
851e8782d5 | ||
|
|
175b2cd791 | ||
|
|
8530a9730f | ||
|
|
ee48e5a01e | ||
|
|
854dbfb3f3 | ||
|
|
69a3b3f7e0 | ||
|
|
3cd826830b | ||
|
|
c584dc57b9 | ||
|
|
e1e5a8ff0d | ||
|
|
c85472285d | ||
|
|
f0fbe193fa |
@@ -35,7 +35,9 @@ modules/mesh-build-image/cmd/mesh-build-image/mesh-build-image
|
||||
/checks/paths/paths
|
||||
/modules/mesh-build-go/mesh-build-go
|
||||
/modules/mesh-delivery-gate/mesh-delivery-gate
|
||||
modules/mesh-config/cmd/mesh-config/mesh-config
|
||||
|
||||
# Built by their modules' build step (issue 1577), never committed.
|
||||
modules/forticlient/cmd/forticlient-tools/forticlient-tools
|
||||
modules/networkmanager/uplink-tools
|
||||
checks/words/words
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest.
|
||||
modules/*/module.json
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest and the files it names.
|
||||
modules
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
@@ -0,0 +1,128 @@
|
||||
// Package failurekit refuses a build module whose tests do not run mesh-sdk's conformance kit (issue 1901, its thorough
|
||||
// fix): a module that claims a node-build-* seat and serves a test step must hold a test that calls conformance.Run, so
|
||||
// its parse of the test runner's output is held to the failure contract (a failing test named with its cause, never a
|
||||
// passing line in its place). A build module with no test step (mesh-build-image builds images and runs no tests) is
|
||||
// exempt from that, and said so. Every build module, test step or not, must also hold a test that calls
|
||||
// conformance.LeftRunning with its own start sweep, so a step a previous process left running is ended at its start
|
||||
// (issue 1890).
|
||||
package failurekit
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Judged is what the check says of one build module: Refused says why it fails the check, "" when it passes; Exempt
|
||||
// says why the kit does not apply.
|
||||
type Judged struct {
|
||||
Module, Refused, Exempt string
|
||||
}
|
||||
|
||||
type manifest struct {
|
||||
Claims []struct {
|
||||
Name string `json:"name"`
|
||||
} `json:"claims"`
|
||||
Tools []json.RawMessage `json:"tools"`
|
||||
}
|
||||
|
||||
// Judge reads every modules/<m>/module.json of the catalogue at root and judges each module that claims a node-build-*
|
||||
// seat.
|
||||
func Judge(root string) ([]Judged, error) {
|
||||
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sort.Strings(manifests)
|
||||
var out []Judged
|
||||
for _, p := range manifests {
|
||||
raw, err := os.ReadFile(p)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var m manifest
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", p, err)
|
||||
}
|
||||
seat := ""
|
||||
for _, c := range m.Claims {
|
||||
if strings.HasPrefix(c.Name, "node-build-") {
|
||||
seat = c.Name
|
||||
}
|
||||
}
|
||||
if seat == "" {
|
||||
continue
|
||||
}
|
||||
dir := filepath.Dir(p)
|
||||
j := Judged{Module: filepath.Base(dir)}
|
||||
var refused []string
|
||||
test := testTool(m.Tools)
|
||||
switch {
|
||||
case test == "":
|
||||
j.Exempt = fmt.Sprintf("it claims %s and serves no test step, so the failure contract's cases do not apply", seat)
|
||||
default:
|
||||
runs, err := runsTheKit(dir, "conformance.Run(")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !runs {
|
||||
refused = append(refused, fmt.Sprintf("it claims %s and serves the test step %s, and no test of it calls conformance.Run "+
|
||||
"(mesh-sdk go/buildstep/conformance): run the kit with the module's own parse path (issue 1901)", seat, test))
|
||||
}
|
||||
}
|
||||
// Every build module keeps steps, so every one runs the left-running case, test step or not (issue 1890).
|
||||
sweeps, err := runsTheKit(dir, "conformance.LeftRunning(")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !sweeps {
|
||||
refused = append(refused, fmt.Sprintf("it claims %s, and no test of it calls conformance.LeftRunning "+
|
||||
"(mesh-sdk go/buildstep/conformance): run the kit's left-running case with the module's own start sweep (issue 1890)", seat))
|
||||
}
|
||||
j.Refused = strings.Join(refused, "; ")
|
||||
out = append(out, j)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// testTool is the name of the module's test step tool (<prefix>_test), or "".
|
||||
func testTool(tools []json.RawMessage) string {
|
||||
for _, raw := range tools {
|
||||
var name string
|
||||
if json.Unmarshal(raw, &name) != nil {
|
||||
var t struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
if json.Unmarshal(raw, &t) != nil {
|
||||
continue
|
||||
}
|
||||
name = t.Name
|
||||
}
|
||||
if strings.HasSuffix(name, "_test") {
|
||||
return name
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// runsTheKit says a _test.go file under dir calls the kit's function call (such as "conformance.Run(").
|
||||
func runsTheKit(dir, call string) (bool, error) {
|
||||
found := false
|
||||
err := filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
|
||||
if err != nil || found || d.IsDir() || !strings.HasSuffix(p, "_test.go") {
|
||||
return err
|
||||
}
|
||||
raw, err := os.ReadFile(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
found = strings.Contains(string(raw), call) &&
|
||||
strings.Contains(string(raw), `"git.novox.be/novox/mesh-sdk/go/buildstep/conformance"`)
|
||||
return nil
|
||||
})
|
||||
return found, err
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
package failurekit
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// 1901/AC7 and 1890/AC8: a build module with a test step and no kit is refused, one that runs the kit passes, one with no
|
||||
// test step is exempt from the failure cases but not from the left-running case, a module that runs only the failure
|
||||
// cases is refused, and a module that claims no node-build-* seat is not judged.
|
||||
func TestABuildModuleWithATestStepMustRunTheKit(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
const imp = "package main\n\nimport \"git.novox.be/novox/mesh-sdk/go/buildstep/conformance\"\n\n"
|
||||
const sweep = "func TestSweep(t *testing.T) { conformance.LeftRunning(t, nil) }\n"
|
||||
kit := imp + "func TestKit(t *testing.T) { conformance.Run(t, conformance.GoTest, nil) }\n" + sweep
|
||||
for path, body := range map[string]string{
|
||||
"modules/a/module.json": `{"claims":[{"name":"node-build-a"}],"tools":["build_a_fetch","build_a_test"]}`,
|
||||
"modules/a/cmd/a/check_test.go": "package main\n",
|
||||
"modules/b/module.json": `{"claims":[{"name":"node-build-b"}],"tools":[{"name":"build_b_test"}]}`,
|
||||
"modules/b/cmd/b/kit_test.go": kit,
|
||||
"modules/c/module.json": `{"claims":[{"name":"node-build-c"}],"tools":["build_c_build"]}`,
|
||||
"modules/c/cmd/c/kit_test.go": imp + sweep,
|
||||
"modules/f/module.json": `{"claims":[{"name":"node-build-f"}],"tools":["build_f_build"]}`,
|
||||
"modules/g/module.json": `{"claims":[{"name":"node-build-g"}],"tools":["build_g_test"]}`,
|
||||
"modules/g/cmd/g/kit_test.go": imp + "func TestKit(t *testing.T) { conformance.Run(t, conformance.GoTest, nil) }\n",
|
||||
"modules/d/module.json": `{"claims":[{"name":"other"}],"tools":["d_test"]}`,
|
||||
"modules/e/module.json": `{"claims":[{"name":"node-build-e"}],"tools":["build_e_test"]}`,
|
||||
"modules/e/cmd/e/kit.go": kit, // not a test file
|
||||
} {
|
||||
p := filepath.Join(root, path)
|
||||
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(p, []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
got, err := Judge(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := map[string]string{"a": "refused", "b": "passes", "c": "exempt", "e": "refused", "f": "refused", "g": "refused"}
|
||||
if len(got) != len(want) {
|
||||
t.Fatalf("judged %+v, want a, b, c, e, f and g", got)
|
||||
}
|
||||
for _, j := range got {
|
||||
is := "passes"
|
||||
if j.Refused != "" {
|
||||
is = "refused"
|
||||
} else if j.Exempt != "" {
|
||||
is = "exempt"
|
||||
}
|
||||
if is != want[j.Module] {
|
||||
t.Errorf("%s %s (%+v), want it %s", j.Module, is, j, want[j.Module])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Every build module of this catalogue with a test step runs the kit.
|
||||
func TestTheCataloguesBuildModulesRunTheKit(t *testing.T) {
|
||||
got, err := Judge(filepath.Join("..", ".."))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got) == 0 {
|
||||
t.Fatal("no module claims a node-build-* seat: the check reads the wrong tree")
|
||||
}
|
||||
for _, j := range got {
|
||||
switch {
|
||||
case j.Refused != "":
|
||||
t.Errorf("%s: %s", j.Module, j.Refused)
|
||||
case j.Exempt != "":
|
||||
t.Logf("%s is exempt: %s", j.Module, j.Exempt)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
module git.novox.be/novox/mesh-catalog/checks/failurekit
|
||||
|
||||
go 1.22
|
||||
@@ -0,0 +1,7 @@
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Judge("../.."): every manifest, and the directory of each module claiming a node-build-* seat, walked for a test that
|
||||
# calls the kit. The build modules are the mesh-build-* directories; a build module named otherwise needs its line here.
|
||||
modules/*/module.json
|
||||
modules/mesh-build-*
|
||||
@@ -1 +1,10 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# every manifest through mesh-sdk's ReadTreeManifest: the root's predefined keys and each templated module's
|
||||
# templates (files/ by default, or the directory its manifest names); and the root's kept count of resources that do not say how they are ready.
|
||||
modules/*/module.json
|
||||
predefined.json
|
||||
modules/*/files
|
||||
modules/*/templates
|
||||
health-undeclared
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest.
|
||||
modules/*/module.json
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest, and whether the path each image artifact builds from is there.
|
||||
modules
|
||||
|
||||
@@ -1 +1,7 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest, a TypeScript module's package.json and whether its lock is there.
|
||||
modules/*/module.json
|
||||
modules/*/package.json
|
||||
modules/*/package-lock.json
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest.
|
||||
modules/*/module.json
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
// Package nobinaries refuses a compiled program under modules/ (issue 1577): it is build output, made by the module's build step,
|
||||
// Package nobinaries refuses a compiled program anywhere in the repository (issue 1577; first a checks/ binary slipped past a modules/-only rule): it is build output, made by the module's build step,
|
||||
// and `go build` in a module's folder writes one beside go.mod where `git add -A` takes it.
|
||||
package nobinaries
|
||||
|
||||
@@ -27,10 +27,10 @@ func Kind(head []byte) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
// Files are the files of the catalogue's root that the change holds under modules/: the tracked ones when the root is a git
|
||||
// checkout (an ignored binary a local build left is not the change's), else every file there.
|
||||
// Files are the files of the catalogue's root that the change holds, anywhere: the tracked ones when the root is a git
|
||||
// checkout (an ignored binary a local build left is not the change's), else every file there but .git's.
|
||||
func Files(root string) ([]string, error) {
|
||||
cmd := exec.Command("git", "-C", root, "ls-files", "-z", "--", "modules")
|
||||
cmd := exec.Command("git", "-C", root, "ls-files", "-z")
|
||||
if out, err := cmd.Output(); err == nil && len(out) > 0 {
|
||||
var files []string
|
||||
for _, f := range strings.Split(strings.TrimRight(string(out), "\x00"), "\x00") {
|
||||
@@ -41,10 +41,16 @@ func Files(root string) ([]string, error) {
|
||||
return files, nil
|
||||
}
|
||||
var files []string
|
||||
err := filepath.WalkDir(filepath.Join(root, "modules"), func(p string, d os.DirEntry, err error) error {
|
||||
if err != nil || d.IsDir() || !d.Type().IsRegular() {
|
||||
err := filepath.WalkDir(root, func(p string, d os.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.IsDir() && d.Name() == ".git" {
|
||||
return filepath.SkipDir
|
||||
}
|
||||
if d.IsDir() || !d.Type().IsRegular() {
|
||||
return nil
|
||||
}
|
||||
rel, _ := filepath.Rel(root, p)
|
||||
files = append(files, rel)
|
||||
return nil
|
||||
|
||||
@@ -2,6 +2,7 @@ package nobinaries
|
||||
|
||||
import (
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
@@ -16,6 +17,7 @@ func TestFindNamesEachCompiledProgramAndNothingElse(t *testing.T) {
|
||||
"modules/e/short": []byte("\x7f"),
|
||||
"modules/f/empty": {},
|
||||
"modules/g/Foo.class": {0xca, 0xfe, 0xba, 0xbe, 0, 0, 0, 52}, // Java, not Mach-O
|
||||
"checks/words/words": []byte("\x7fELF\x02\x01\x01\x00rest"), // outside modules/ too
|
||||
}
|
||||
var names []string
|
||||
for f, b := range files {
|
||||
@@ -32,8 +34,8 @@ func TestFindNamesEachCompiledProgramAndNothingElse(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("found %v; wanted modules/a/a, modules/b/cmd/b/b.exe and modules/c/c only", got)
|
||||
if len(got) != 4 {
|
||||
t.Fatalf("found %v; wanted modules/a/a, modules/b/cmd/b/b.exe and modules/c/c and checks/words/words only", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -52,3 +54,36 @@ func TestTheCatalogueHoldsNoCompiledProgram(t *testing.T) {
|
||||
t.Errorf("%s: a compiled program is build output: remove it and add it to .gitignore (issue 1577)", f)
|
||||
}
|
||||
}
|
||||
|
||||
// A compiled program outside modules/ is refused too (a binary under checks/ slipped past the modules/-only rule): Files holds
|
||||
// every tracked file of a git checkout and every file of a plain folder, and Find names the binary in both.
|
||||
func TestAProgramUnderChecksIsRefused(t *testing.T) {
|
||||
for _, git := range []bool{true, false} {
|
||||
root := t.TempDir()
|
||||
p := filepath.Join(root, "checks", "words", "words")
|
||||
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(p, []byte("\x7fELF\x02\x01\x01\x00rest"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if git {
|
||||
for _, args := range [][]string{{"init", "-q"}, {"add", "-A"}} {
|
||||
if out, err := exec.Command("git", append([]string{"-C", root}, args...)...).CombinedOutput(); err != nil {
|
||||
t.Fatalf("git %v: %v: %s", args, err, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
files, err := Files(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
found, err := Find(root, files)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(found) != 1 {
|
||||
t.Errorf("git=%v: found %v; wanted checks/words/words (ELF)", git, found)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1 +1 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# every tracked file of the repository (git ls-files, no path limit): the whole tree, so this package's verdict is never kept.
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
# The modules of this catalogue that still read the controller's assignments (mesh-controller.assignments), each with
|
||||
# why. The check of this directory (noplacement) refuses any other module that reads them, and an entry here whose module no longer
|
||||
# does: the list only shrinks, and it is empty before the controller's lease is written to a holder that is no
|
||||
# controller (novox/hq ADR 0549 §3, §6; issue 774).
|
||||
gitea the provisioner's plan of a module (go/provider) rides only in an assignment's value; its home after the lease is issue 774's provisioner-plan subtask
|
||||
keycloak the provisioner's plan of a module (go/provider), as gitea
|
||||
mesh-config unioned with the placement while both stand, as the assigner and the vault do (ADR 0549 §5); its reading placement alone is a step before the lease write
|
||||
mesh-vault the provisioner's plan, which only an assignment's value carries; its home is issue 774's provisioner-plan subtask
|
||||
postgres the provisioner's plan of a module (go/provider), as gitea
|
||||
route-adapter the provisioner's plan of a module (go/provider), as gitea
|
||||
@@ -0,0 +1,7 @@
|
||||
module git.novox.be/novox/mesh-catalog/checks/noplacement
|
||||
|
||||
go 1.22
|
||||
|
||||
require git.novox.be/novox/mesh-catalog/checks/trunk v0.0.0
|
||||
|
||||
replace git.novox.be/novox/mesh-catalog/checks/trunk => ../trunk
|
||||
@@ -0,0 +1,170 @@
|
||||
// Command noplacement refuses what the repository path of placement left behind (novox/hq ADR 0567 §10, §12).
|
||||
//
|
||||
// Placement, which module runs on which node, is bus state held by the seat mesh-placement; no file of the
|
||||
// catalogue says it, and no manifest reads it from the delivery indexer, which no longer keeps it. It refuses:
|
||||
//
|
||||
// 1. a `placement` directory (or file) at the root of the catalogue;
|
||||
//
|
||||
// 2. a `particulars` file at the root (the mesh's node names come from the checker's run, §16);
|
||||
//
|
||||
// 3. a manifest naming a state, stream, event or bucket of the indexer's placement (`mesh-delivery-indexer.placement…`
|
||||
// or `mesh-delivery-indexer_placement…`) in any string value, such as `reads`;
|
||||
//
|
||||
// 4. a manifest, other than mesh-controller's, that reads the controller's assignments (`mesh-controller.assignments`):
|
||||
// every reader takes placement from the seat mesh-placement (novox/hq ADR 0549 §3, ADR 0567), and the lease write is
|
||||
// refused while one still reads them (ADR 0549 §6; issue 774). The modules that still read them for what the
|
||||
// placement does not carry are named in checks/noplacement/assignments-readers, each with why; that list only
|
||||
// shrinks: an entry whose module no longer reads them is refused too.
|
||||
//
|
||||
// go run . <catalogue root>
|
||||
//
|
||||
// Exits 1 on a finding.
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Prefixes are the names of the indexer's placement parts; a manifest string starting with one names them.
|
||||
var Prefixes = []string{"mesh-delivery-indexer.placement", "mesh-delivery-indexer_placement"}
|
||||
|
||||
// Findings is every refusal in the catalogue at root, sorted.
|
||||
func Findings(root string) ([]string, error) {
|
||||
var out []string
|
||||
if _, err := os.Lstat(filepath.Join(root, "placement")); err == nil {
|
||||
out = append(out, "placement: the repository holds no placement; it is bus state of the seat mesh-placement (novox/hq ADR 0567 §10)")
|
||||
}
|
||||
if _, err := os.Lstat(filepath.Join(root, "particulars")); err == nil {
|
||||
out = append(out, "particulars: the repository lists no node names; the checker hands them to the check (novox/hq ADR 0567 §16)")
|
||||
}
|
||||
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sort.Strings(manifests)
|
||||
for _, file := range manifests {
|
||||
raw, err := os.ReadFile(file)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var doc any
|
||||
if err := json.Unmarshal(raw, &doc); err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", file, err)
|
||||
}
|
||||
module := filepath.Base(filepath.Dir(file))
|
||||
for _, s := range named(doc, nil) {
|
||||
out = append(out, fmt.Sprintf("%s: names %q, which the indexer no longer keeps; placement is the seat mesh-placement's (novox/hq ADR 0567 §12)", module, s))
|
||||
}
|
||||
}
|
||||
readers, err := assignmentsReaders(root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, readers...)
|
||||
sort.Strings(out)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// AssignmentsRead is the controller's assignments as a manifest reads them, and Allowed the file, relative to the
|
||||
// catalogue's root, that names the modules still reading them: `<module> <why>` per line, `#` a comment.
|
||||
const (
|
||||
AssignmentsRead = "mesh-controller.assignments"
|
||||
Allowed = "checks/noplacement/assignments-readers"
|
||||
)
|
||||
|
||||
// assignmentsReaders is every finding on the reads of the controller's assignments: a module other than the controller
|
||||
// reading them that Allowed does not name, and a module Allowed names that no longer reads them (issue 774).
|
||||
func assignmentsReaders(root string) ([]string, error) {
|
||||
allowed := map[string]bool{}
|
||||
raw, err := os.ReadFile(filepath.Join(root, Allowed))
|
||||
if err != nil && !os.IsNotExist(err) {
|
||||
return nil, err
|
||||
}
|
||||
for _, line := range strings.Split(string(raw), "\n") {
|
||||
if f := strings.Fields(line); len(f) > 0 && !strings.HasPrefix(f[0], "#") {
|
||||
allowed[f[0]] = true
|
||||
}
|
||||
}
|
||||
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var found []string
|
||||
reads := map[string]bool{}
|
||||
for _, path := range manifests {
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var m struct {
|
||||
Module string `json:"module"`
|
||||
Reads []string `json:"reads"`
|
||||
}
|
||||
if json.Unmarshal(raw, &m) != nil || m.Module == "mesh-controller" {
|
||||
continue // a manifest that does not read is the manifest checks' to refuse
|
||||
}
|
||||
for _, r := range m.Reads {
|
||||
if strings.TrimSpace(r) != AssignmentsRead {
|
||||
continue
|
||||
}
|
||||
reads[m.Module] = true
|
||||
if !allowed[m.Module] {
|
||||
found = append(found, fmt.Sprintf("%s reads %s: read where modules run from mesh-delivery-assigner.placement, "+
|
||||
"the seat mesh-placement's (novox/hq ADR 0549 §3, ADR 0567); the lease write is refused while a module reads "+
|
||||
"the assignments (ADR 0549 §6)", m.Module, AssignmentsRead))
|
||||
}
|
||||
}
|
||||
}
|
||||
for module := range allowed {
|
||||
if !reads[module] {
|
||||
found = append(found, fmt.Sprintf("%s names %s, which no longer reads %s: take it off the list, which only shrinks",
|
||||
Allowed, module, AssignmentsRead))
|
||||
}
|
||||
}
|
||||
return found, nil
|
||||
}
|
||||
|
||||
// named collects the strings of a decoded manifest that start with one of the Prefixes.
|
||||
func named(v any, into []string) []string {
|
||||
switch v := v.(type) {
|
||||
case string:
|
||||
for _, p := range Prefixes {
|
||||
if strings.HasPrefix(v, p) {
|
||||
return append(into, v)
|
||||
}
|
||||
}
|
||||
case []any:
|
||||
for _, e := range v {
|
||||
into = named(e, into)
|
||||
}
|
||||
case map[string]any:
|
||||
for _, e := range v {
|
||||
into = named(e, into)
|
||||
}
|
||||
}
|
||||
return into
|
||||
}
|
||||
|
||||
func main() {
|
||||
if len(os.Args) != 2 {
|
||||
fmt.Fprintln(os.Stderr, "usage: noplacement <catalogue root>")
|
||||
os.Exit(2)
|
||||
}
|
||||
found, err := Findings(os.Args[1])
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
os.Exit(1)
|
||||
}
|
||||
for _, f := range found {
|
||||
fmt.Println(f)
|
||||
}
|
||||
if len(found) > 0 {
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Println("no placement directory, no manifest reads the indexer's placement, and only the modules listed read the controller's assignments")
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-catalog/checks/trunk"
|
||||
)
|
||||
|
||||
func TestFindingsRefuseWhatThePlacementPathLeft(t *testing.T) {
|
||||
for _, row := range []struct {
|
||||
name string
|
||||
files map[string]string
|
||||
want string // empty: clean
|
||||
}{
|
||||
{"clean", map[string]string{"modules/a/module.json": `{"module":"a","reads":["mesh-delivery-assigner.placement"]}`}, ""},
|
||||
{"a placement directory", map[string]string{"placement/ace.json": `{}`}, "placement: the repository holds no placement"},
|
||||
{"a particulars file", map[string]string{"particulars": "ace\n"}, "particulars: the repository lists no node names"},
|
||||
{"a read of the indexer's placement", map[string]string{"modules/a/module.json": `{"module":"a","reads":["mesh-delivery-indexer.placement"]}`}, `a: names "mesh-delivery-indexer.placement"`},
|
||||
{"a read of its candidates", map[string]string{"modules/b/module.json": `{"module":"b","reads":["mesh-delivery-indexer.placement-candidates"]}`}, "placement-candidates"},
|
||||
{"its event", map[string]string{"modules/c/module.json": `{"module":"c","on":[{"event":"mesh-delivery-indexer.placement-changed"}]}`}, "placement-changed"},
|
||||
// Issue 774 (ADR 0549 §3, §6): a reader of the controller's assignments is refused unless the list names it, the
|
||||
// controller's own manifest is not judged, and a name on the list whose module no longer reads them is refused.
|
||||
{"an unlisted reader of the assignments", map[string]string{"modules/r/module.json": `{"module":"r","reads":["mesh-controller.assignments"]}`}, "r reads mesh-controller.assignments"},
|
||||
{"a listed reader and the controller", map[string]string{"modules/l/module.json": `{"module":"l","reads":["mesh-controller.assignments"]}`,
|
||||
"modules/mesh-controller/module.json": `{"module":"mesh-controller","reads":["mesh-controller.assignments"]}`, Allowed: "# why\nl its plan\n"}, ""},
|
||||
{"a listed module that reads them no more", map[string]string{Allowed: "gone no longer reads\n"}, "names gone, which no longer reads"},
|
||||
{"the seat's own state is allowed", map[string]string{"modules/d/module.json": `{"module":"d","reads":["mesh-placement.placement"]}`}, ""},
|
||||
} {
|
||||
t.Run(row.name, func(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
for path, text := range row.files {
|
||||
p := filepath.Join(root, path)
|
||||
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(p, []byte(text), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
got, err := Findings(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if all := strings.Join(got, "\n"); (row.want == "") != (all == "") || !strings.Contains(all, row.want) {
|
||||
t.Fatalf("findings = %q, want one containing %q", all, row.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The script's run of this check over the real tree (novox/hq ADR 0529 clause 2).
|
||||
func TestTheCatalogueHoldsNoPlacementOfTheRepositoryPath(t *testing.T) {
|
||||
got, err := Findings(filepath.Join("..", ".."))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, f := range got {
|
||||
t.Error(f)
|
||||
}
|
||||
// Issue 774 (ADR 0549 §3, §6; the comparison of novox/hq ADR 0529 clause 2): the list of modules still reading the
|
||||
// controller's assignments gains none the trunk's copy at the merge base does not hold. Its first copy gains nothing.
|
||||
path, ok := trunk.Copy(t, Allowed)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
names := func(path string) map[string]bool {
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out := map[string]bool{}
|
||||
for _, line := range strings.Split(string(raw), "\n") {
|
||||
if f := strings.Fields(line); len(f) > 0 && !strings.HasPrefix(f[0], "#") {
|
||||
out[f[0]] = true
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
base := names(path)
|
||||
for module := range names(filepath.Join("..", "..", Allowed)) {
|
||||
if !base[module] {
|
||||
t.Errorf("%s gains %s, which the trunk's copy does not hold: the list only shrinks", Allowed, module)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# the root, for a placement directory, and every manifest, for a read of the indexer's placement states.
|
||||
?placement
|
||||
modules/*/module.json
|
||||
@@ -1 +1,7 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# that none of the old check scripts stands at the root.
|
||||
?merge-check.sh
|
||||
?merge-check-typescript.sh
|
||||
?check-nested-modules.sh
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest; its lists are this package's own files.
|
||||
modules/*/module.json
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
// issue 1838): a module is for any mesh, so no manifest, file, code string, identifier, comment, test name or
|
||||
// fixture of a covered module names a node of this one.
|
||||
//
|
||||
// The names are in the file `particulars` at the catalogue's root, one a line, `#` comments — the one place
|
||||
// they may be written. The modules covered are in `covered` at checks/particulars/covered, one a line. A name is matched
|
||||
// The names are the mesh's own, not the repository's: the checker hands them in the environment value
|
||||
// MESH_NODE_NAMES, comma-separated, from the node records it reads (novox/hq ADR 0567 §16). A run with none fails
|
||||
// loudly, since it would pass on anything. The modules covered are in `covered` at checks/particulars/covered, one a line. A name is matched
|
||||
// whole and in any case; a hyphen, an underscore or a CamelCase boundary parts one, so a longer word holding a name is another word.
|
||||
// The forge is not a node name: its host (git.novox.be) and the organisation path (novox/<repo>) are blanked
|
||||
// before a line is read.
|
||||
@@ -34,14 +35,28 @@ func main() {
|
||||
if len(os.Args) > 1 {
|
||||
root = os.Args[1]
|
||||
}
|
||||
os.Exit(check(root))
|
||||
os.Exit(check(root, namesFromEnv()))
|
||||
}
|
||||
|
||||
// NamesEnv is the environment value that carries the mesh's node names (mesh-sdk delivery.NodeNamesEnv).
|
||||
const NamesEnv = "MESH_NODE_NAMES"
|
||||
|
||||
// namesFromEnv is the names NamesEnv holds, comma-separated, trimmed, empty entries left out.
|
||||
func namesFromEnv() []string {
|
||||
var out []string
|
||||
for _, n := range strings.Split(os.Getenv(NamesEnv), ",") {
|
||||
if n = strings.TrimSpace(n); n != "" {
|
||||
out = append(out, n)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// check runs the check over a catalogue and returns the exit code: 0 clean, 1 a finding, 2 a list unreadable.
|
||||
func check(root string) int {
|
||||
names, err := readList(filepath.Join(root, "particulars"))
|
||||
if err == nil && len(names) == 0 {
|
||||
err = fmt.Errorf("particulars lists no name — the check would pass on anything")
|
||||
func check(root string, names []string) int {
|
||||
var err error
|
||||
if len(names) == 0 {
|
||||
err = fmt.Errorf("no node names (%s is empty) — the check would pass on anything", NamesEnv)
|
||||
}
|
||||
var covered []string
|
||||
if err == nil {
|
||||
|
||||
@@ -74,26 +74,24 @@ func TestANodeNamedPathIsRefusedThroughRun(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckReturnsOneOnAFindingAndTwoOnAnUnreadableList(t *testing.T) {
|
||||
func TestCheckReturnsOneOnAFindingAndTwoOnNoNames(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
write := func(p, s string) {
|
||||
p = filepath.Join(root, p)
|
||||
os.MkdirAll(filepath.Dir(p), 0o755)
|
||||
os.WriteFile(p, []byte(s), 0o644)
|
||||
}
|
||||
write("particulars", "ace\n")
|
||||
write("checks/particulars/covered", "kernel\n")
|
||||
write("modules/kernel/a", "clean\n")
|
||||
if got := check(root); got != 0 {
|
||||
if got := check(root, []string{"ace"}); got != 0 {
|
||||
t.Errorf("clean = %d, want 0", got)
|
||||
}
|
||||
write("modules/kernel/a", "on ace\n")
|
||||
if got := check(root); got != 1 {
|
||||
if got := check(root, []string{"ace"}); got != 1 {
|
||||
t.Errorf("finding = %d, want 1", got)
|
||||
}
|
||||
os.Remove(filepath.Join(root, "particulars"))
|
||||
if got := check(root); got != 2 {
|
||||
t.Errorf("no list = %d, want 2", got)
|
||||
if got := check(root, nil); got != 2 {
|
||||
t.Errorf("no names = %d, want 2", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -117,7 +115,10 @@ func TestOnlyCoveredModulesAreRead(t *testing.T) {
|
||||
|
||||
// The script's run of this check over the real tree (novox/hq ADR 0529 clause 2): no node name of this mesh in a covered module.
|
||||
func TestTheCoveredModulesNameNoNode(t *testing.T) {
|
||||
if code := check("../.."); code != 0 {
|
||||
if _, set := os.LookupEnv(NamesEnv); !set {
|
||||
t.Skip("ADR 0567 §16: this checker hands no node names (before D14 is delivered); the skip goes once it is")
|
||||
}
|
||||
if code := check("../..", namesFromEnv()); code != 0 {
|
||||
t.Fatalf("the check of the real catalogue exits %d (see what it printed above)", code)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# check("../.."): every file of the modules the covered list names. The node names come from the run (MESH_NODE_NAMES).
|
||||
modules
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest and every file of a module.
|
||||
modules
|
||||
|
||||
@@ -1 +1,6 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# the root's list of predefined keys and every manifest that names one.
|
||||
predefined.json
|
||||
modules/*/module.json
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest and the files it names.
|
||||
modules
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest.
|
||||
modules/*/module.json
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest and its templates.
|
||||
modules
|
||||
|
||||
@@ -1 +1,5 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# Check("../.."): every manifest.
|
||||
modules/*/module.json
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
@@ -206,6 +206,9 @@ func excerpt(s string, from, to int) string {
|
||||
return strings.Join(strings.Fields(s[a:b]), " ")
|
||||
}
|
||||
|
||||
// modulePath matches a string that is only a module or import path ("git.example.org/org/repo/go/pkg").
|
||||
var modulePath = regexp.MustCompile(`^[a-z0-9.-]+\.[a-z]+/[A-Za-z0-9_./-]+$`)
|
||||
|
||||
// goStrings returns a Go file's string literals, adjacent ones joined across `+` as the program joins them.
|
||||
func goStrings(path, rel string) ([]text, error) {
|
||||
src, err := os.ReadFile(path)
|
||||
@@ -231,6 +234,11 @@ func goStrings(path, rel string) ([]text, error) {
|
||||
if err != nil {
|
||||
v = lit
|
||||
}
|
||||
if modulePath.MatchString(v) {
|
||||
// An import path or another module path names code, not words an agent reads.
|
||||
cur, joining = nil, false
|
||||
continue
|
||||
}
|
||||
if cur != nil && joining {
|
||||
cur.s += v
|
||||
} else {
|
||||
|
||||
@@ -1 +1,7 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# run("../.."): the root's lists of retired words and lines, and every module's texts.
|
||||
retired-words
|
||||
retired-lines
|
||||
modules
|
||||
|
||||
@@ -1 +1,9 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# the copies of this package in every module that carries one.
|
||||
modules/xorg/internal/desktop/*.go
|
||||
modules/lemurs/internal/desktop/*.go
|
||||
modules/i3/internal/desktop/*.go
|
||||
modules/xterm/internal/desktop/*.go
|
||||
modules/adwaita/internal/desktop/*.go
|
||||
|
||||
@@ -23,7 +23,7 @@ import (
|
||||
// manifest's.
|
||||
const NamesFileEnv = "MESH_ARTIFACT_STORE_NAMES_FILE"
|
||||
|
||||
// CatalogueModules and CatalogueModule are the catalogue's tools these definitions are read through.
|
||||
// CatalogueModules and CatalogueModule are the catalog's tools these definitions are read through.
|
||||
const (
|
||||
CatalogueModules = "mesh-catalog.catalog_modules"
|
||||
CatalogueModule = "mesh-catalog.catalog_module"
|
||||
@@ -143,21 +143,21 @@ func PinsIn(manifest json.RawMessage, names StoreNames) []pinned {
|
||||
return out
|
||||
}
|
||||
|
||||
// Definitions is what the catalogue's definitions pin in the store, and what of them could not be read.
|
||||
// Definitions is what the catalog's definitions pin in the store, and what of them could not be read.
|
||||
type Definitions struct {
|
||||
// Pins are by Key(repository, digest).
|
||||
Pins map[string][]Pin
|
||||
// Read is how many definitions were read; Unread says, per module, why its definition was not.
|
||||
Read int
|
||||
Unread map[string]string
|
||||
// NotAsked says why the catalogue could not be asked at all, when it could not.
|
||||
// NotAsked says why the catalog could not be asked at all, when it could not.
|
||||
NotAsked string
|
||||
// NamesNotRead says why the store's older names could not be read, when they could not.
|
||||
NamesNotRead string
|
||||
Names StoreNames
|
||||
}
|
||||
|
||||
// Catalogue reads the definitions through the catalogue's tools, and keeps each definition it read by its
|
||||
// Catalogue reads the definitions through the catalog's tools, and keeps each definition it read by its
|
||||
// module and commit, which never change together.
|
||||
type Catalogue struct {
|
||||
Ask Asking
|
||||
@@ -171,21 +171,21 @@ type catalogueEntry struct {
|
||||
Commit string `json:"commit"`
|
||||
}
|
||||
|
||||
// Read asks the catalogue for every module and reads each definition, bounded by DefinitionsTimeout.
|
||||
// Read asks the catalog for every module and reads each definition, bounded by DefinitionsTimeout.
|
||||
func (c *Catalogue) Read(ctx context.Context, names StoreNames) *Definitions {
|
||||
defs := &Definitions{Pins: map[string][]Pin{}, Unread: map[string]string{}, Names: names}
|
||||
if c == nil || c.Ask == nil {
|
||||
defs.NotAsked = "this bundle has no way to ask the catalogue"
|
||||
defs.NotAsked = "this bundle has no way to ask the catalog"
|
||||
return defs
|
||||
}
|
||||
raw, err := c.Ask(CatalogueModules, map[string]any{})
|
||||
if err != nil {
|
||||
defs.NotAsked = "the catalogue's modules could not be asked: " + err.Error()
|
||||
defs.NotAsked = "the catalog's modules could not be asked: " + err.Error()
|
||||
return defs
|
||||
}
|
||||
answer, output, ok := answerOf(raw)
|
||||
if !ok {
|
||||
defs.NotAsked = "the catalogue refused its modules: " + lastLine(output)
|
||||
defs.NotAsked = "the catalog refused its modules: " + lastLine(output)
|
||||
return defs
|
||||
}
|
||||
var list struct {
|
||||
@@ -193,7 +193,7 @@ func (c *Catalogue) Read(ctx context.Context, names StoreNames) *Definitions {
|
||||
Error string `json:"error"`
|
||||
}
|
||||
if err := json.Unmarshal(answer, &list); err != nil || list.Error != "" || list.Modules == nil {
|
||||
defs.NotAsked = "the catalogue's modules answer is not readable: " + firstOf(list.Error, errText(err), lastLine(output))
|
||||
defs.NotAsked = "the catalog's modules answer is not readable: " + firstOf(list.Error, errText(err), lastLine(output))
|
||||
return defs
|
||||
}
|
||||
c.mu.Lock()
|
||||
|
||||
@@ -54,7 +54,7 @@ func Tools(s *Store, c Controller, cat *Catalogue) []stdio.Tool {
|
||||
Name: "artifact_store_references",
|
||||
Description: "What the controller's records say of each manifest the artifact store holds: kept (a definition " +
|
||||
"names it, or one of the five most recent builds of a module the mesh holds), named by a definition (no " +
|
||||
"record names it, and a definition in the catalogue pins it by digest, under the store's current name or " +
|
||||
"record names it, and a definition in the catalog pins it by digest, under the store's current name or " +
|
||||
"an older one: the module is said), the holder of a kept archive, " +
|
||||
"eligible (the mesh made it and keeps it for no reason), the holder of an eligible archive, let go yet present, " +
|
||||
"a named document the controller keeps under a tag, or unrecorded — no record names it. A platform manifest " +
|
||||
|
||||
@@ -1 +1,25 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# the copies of desktop.go and its tests in every module that carries one.
|
||||
modules/blueman/cmd/blueman-tools/desktop.go
|
||||
modules/blueman/cmd/blueman-tools/desktop_test.go
|
||||
modules/blueman/cmd/blueman-tools/copies_test.go
|
||||
modules/forticlient/cmd/forticlient-tools/desktop.go
|
||||
modules/forticlient/cmd/forticlient-tools/desktop_test.go
|
||||
modules/forticlient/cmd/forticlient-tools/copies_test.go
|
||||
modules/nextcloud-client/cmd/nextcloud-client-tools/desktop.go
|
||||
modules/nextcloud-client/cmd/nextcloud-client-tools/desktop_test.go
|
||||
modules/nextcloud-client/cmd/nextcloud-client-tools/copies_test.go
|
||||
modules/nm-applet/cmd/nm-applet-tools/desktop.go
|
||||
modules/nm-applet/cmd/nm-applet-tools/desktop_test.go
|
||||
modules/nm-applet/cmd/nm-applet-tools/copies_test.go
|
||||
modules/openrazer/cmd/openrazer-tools/desktop.go
|
||||
modules/openrazer/cmd/openrazer-tools/desktop_test.go
|
||||
modules/openrazer/cmd/openrazer-tools/copies_test.go
|
||||
modules/polychromatic/cmd/polychromatic-tools/desktop.go
|
||||
modules/polychromatic/cmd/polychromatic-tools/desktop_test.go
|
||||
modules/polychromatic/cmd/polychromatic-tools/copies_test.go
|
||||
modules/slack/cmd/slack-tools/desktop.go
|
||||
modules/slack/cmd/slack-tools/desktop_test.go
|
||||
modules/slack/cmd/slack-tools/copies_test.go
|
||||
|
||||
@@ -1 +1,6 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# mesh-mail's store.json is the same as brother's; no module scan is left.
|
||||
modules/mesh-mail/files/store.json
|
||||
?modules/scan
|
||||
|
||||
@@ -45,8 +45,8 @@ whenever the node's tool runtime collects the module's tools:
|
||||
|
||||
| file | holds |
|
||||
|---|---|
|
||||
| `managed-mcp.json` | the tool servers every session loads: the mesh's console as `mesh`, and the servers set in this module's `mcp_servers` setting. **Exclusive**: a server not listed here does not load — not one added with `claude mcp add`, not a project's `.mcp.json`, not a plugin's |
|
||||
| `managed-settings.json` | the keys set in this module's `managed_settings` setting — at the controller's terminal alone (hq issue 340) — under the mesh's own keys: the repositories' attribution convention, the claude.ai connectors kept beside the managed servers, the key-helper while the node holds an API-key licence, git's credential file while the node holds the forge's read credential (below), and the two that name the `nox-mesh` marketplace and enable its plugin |
|
||||
| `managed-mcp.json` | the tool servers every session loads: the mesh's console as `mesh` (with a `timeout` of 660000 ms, mesh_wait's longest wait and a minute, so the agent's one-minute default does not cut a wait, issue 1871), and the servers set in this module's `mcp_servers` setting. **Exclusive**: a server not listed here does not load — not one added with `claude mcp add`, not a project's `.mcp.json`, not a plugin's |
|
||||
| `managed-settings.json` | the keys set in this module's `managed_settings` setting — at the controller's terminal alone (hq issue 340) — under the mesh's own keys: the repositories' attribution convention, the claude.ai connectors kept beside the managed servers, the key-helper while the node holds an API-key licence, git's credential file while the node holds the forge's read credential (below), and the two that name the `nox-mesh` marketplace and enable its plugin, and `CLAUDE_CODE_MCP_TOOL_IDLE_TIMEOUT` (660000) in `env` beside the operator's own, the mesh's key winning |
|
||||
| `CLAUDE.md` | how a session on this mesh works, this node's name and role, the conventions, the **"instead of" table** — then the instruction sections registered for every node and for this one |
|
||||
| `marketplace/` | the `nox-mesh` plugin (hq ADR 0216): the skills, subagents, commands and output styles registered for every node and for this one, offered in a session as `nox-mesh:<name>` — and the **guard on the agent's shell** (`guard/`), the mesh's own hook and the plugin's only one (hq issue 340). Replaced whole, staged beside and swapped in |
|
||||
|
||||
|
||||
@@ -83,6 +83,20 @@ func TestTheRendererWritesWhatTheTypeScriptOneWrote(t *testing.T) {
|
||||
for k := range MarketplaceKeys() {
|
||||
delete(m, k)
|
||||
}
|
||||
// The wait bound is new since the TypeScript (issue 1871), and is held below.
|
||||
if env, ok := m["env"].(map[string]any); ok {
|
||||
delete(env, "CLAUDE_CODE_MCP_TOOL_IDLE_TIMEOUT")
|
||||
if len(env) == 0 {
|
||||
delete(m, "env")
|
||||
}
|
||||
}
|
||||
}
|
||||
if m, ok := a.(map[string]any); ok && file == "managed-mcp.json" {
|
||||
if servers, ok := m["mcpServers"].(map[string]any); ok {
|
||||
if entry, ok := servers["mesh"].(map[string]any); ok {
|
||||
delete(entry, "timeout")
|
||||
}
|
||||
}
|
||||
}
|
||||
_ = json.Unmarshal([]byte(want[file]), &b)
|
||||
if !reflect.DeepEqual(a, b) {
|
||||
@@ -104,6 +118,21 @@ func TestASettingCannotReplaceTheMeshsOwnEntryAndABadNameIsLeftOut(t *testing.T)
|
||||
if mcp.MCPServers["mesh"]["url"] != "http://127.0.0.1:4270/mcp" || mcp.MCPServers["bad name"] != nil {
|
||||
t.Fatalf("%v", mcp.MCPServers)
|
||||
}
|
||||
// The entry bounds a call by mesh_wait's longest wait and the idle bound is laid over the operator's env
|
||||
// (issue 1871): a session's wait is not cut at the agent's one-minute default.
|
||||
if mcp.MCPServers["mesh"]["timeout"] != float64(660000) {
|
||||
t.Errorf("the mesh entry's timeout is %v", mcp.MCPServers["mesh"]["timeout"])
|
||||
}
|
||||
for _, held := range []map[string]any{nil, {"CLAUDE_CODE_MCP_TOOL_IDLE_TIMEOUT": "1", "FOO": "bar"}} {
|
||||
var m struct {
|
||||
Env map[string]any `json:"env"`
|
||||
}
|
||||
got := Render(Facts{Console: "x"}, Settings{ManagedSettings: map[string]any{"env": held}}, nil, "/h", nil, Config{})
|
||||
_ = json.Unmarshal([]byte(got["managed-settings.json"]), &m)
|
||||
if m.Env["CLAUDE_CODE_MCP_TOOL_IDLE_TIMEOUT"] != "660000" || (held != nil && m.Env["FOO"] != "bar") {
|
||||
t.Errorf("env %v rendered as %v", held, m.Env)
|
||||
}
|
||||
}
|
||||
if !reflect.DeepEqual(Render(Facts{Console: "x"}, Settings{}, nil, "/h", nil, Config{}), Render(Facts{Console: "x"}, Settings{}, nil, "/h", nil, Config{})) {
|
||||
t.Fatal("rendering is not deterministic")
|
||||
}
|
||||
|
||||
@@ -4,9 +4,9 @@ package main
|
||||
// the file under the agent's managed directory did not change by a byte when the module moved to Go;
|
||||
// a test holds it to that renderer's own output (testdata/rendered-by-typescript.json).
|
||||
func instructionsText(node, role string) string {
|
||||
return "# This machine is a node of a Novox mesh\n\nWritten by the mesh's `claude-code` module. Edit the module's settings or the catalogue, never this file:\nit is rewritten whenever the module renders.\n\n## Who this node is\n\n- **Node:** `" +
|
||||
return "# This machine is a node of a Novox mesh\n\nWritten by the mesh's `claude-code` module. Edit the module's settings or the catalog, never this file:\nit is rewritten whenever the module renders.\n\n## Who this node is\n\n- **Node:** `" +
|
||||
node +
|
||||
"`\n- **Role:** " +
|
||||
role +
|
||||
"\n- The other nodes, their roles and what runs where: ask the controller (`mesh-controller.nodes`,\n `mesh-controller.node`). Nothing here lists them, because a copy drifts.\n\n## How a session on this mesh works\n\nThe console is the only way to the mesh: the MCP server named `mesh`. It offers five tools, and\neverything else is an address you find and call through them:\n\n- `mesh_search` — words in, matching addresses out. `mesh_describe` — one address's arguments.\n- `mesh_call` — call an address. A seat the mesh holds once is `<seat>.<verb>` (the mesh's own verbs\n are `mesh-controller.<verb>`: `status`, `plan`, `node`, `assign`, `push`, `settings`);\n a module on a machine is `<node>/<module>.<tool>`.\n- `mesh_overview` and `mesh_machine` — the mesh's seats and machines, and what one machine runs.\n\n- **Symptom first.** For an error, a failing service or anything unexpected, search the record with the\n literal text before forming a hypothesis: the records module's `records_search`, then\n `records_read`.\n- **Ask the mesh before changing it**, and change it through the controller's verbs or the catalogue.\n- **A licence** through the `anthropic-licence-manager` seat's verbs. Never edit the agent's credentials\n file by hand, never print or ask for a token.\n\n## Hard rules\n\n- A file the mesh manages is changed through the verb or the catalogue that owns it, never on disk. If\n unsure, `mesh-controller.plan` for the node says what the mesh writes there.\n- Never write to a store's database by hand; schema changes are numbered migrations.\n- Never push to a main branch: a branch, a pull request, and a human approval for every merge.\n- The mesh creates no symlinks, and nobody else does either.\n- A package is declared in a module, never installed by hand.\n\n## Conventions\n\n- Commit messages are concise, in the imperative, about why.\n- Test before pushing: nodes update unattended.\n- The playbooks in the record say how research, decisions, designs, issues and hand-offs are done.\n"
|
||||
"\n- The other nodes, their roles and what runs where: ask the controller (`mesh-controller.nodes`,\n `mesh-controller.node`). Nothing here lists them, because a copy drifts.\n\n## How a session on this mesh works\n\nThe console is the only way to the mesh: the MCP server named `mesh`. It offers five tools, and\neverything else is an address you find and call through them:\n\n- `mesh_search` — words in, matching addresses out. `mesh_describe` — one address's arguments.\n- `mesh_call` — call an address. A seat the mesh holds once is `<seat>.<verb>` (the mesh's own verbs\n are `mesh-controller.<verb>`: `status`, `plan`, `node`, `assign`, `push`, `settings`);\n a module on a machine is `<node>/<module>.<tool>`.\n- `mesh_overview` and `mesh_machine` — the mesh's seats and machines, and what one machine runs.\n\n- **Symptom first.** For an error, a failing service or anything unexpected, search the record with the\n literal text before forming a hypothesis: the records module's `records_search`, then\n `records_read`.\n- **Ask the mesh before changing it**, and change it through the controller's verbs or the catalog.\n- **A licence** through the `anthropic-licence-manager` seat's verbs. Never edit the agent's credentials\n file by hand, never print or ask for a token.\n\n## Hard rules\n\n- A file the mesh manages is changed through the verb or the catalog that owns it, never on disk. If\n unsure, `mesh-controller.plan` for the node says what the mesh writes there.\n- Never write to a store's database by hand; schema changes are numbered migrations.\n- Never push to a main branch: a branch, a pull request, and a human approval for every merge.\n- The mesh creates no symlinks, and nobody else does either.\n- A package is declared in a module, never installed by hand.\n\n## Conventions\n\n- Commit messages are concise, in the imperative, about why.\n- Test before pushing: nodes update unattended.\n- The playbooks in the record say how research, decisions, designs, issues and hand-offs are done.\n"
|
||||
}
|
||||
|
||||
@@ -22,6 +22,7 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
@@ -98,6 +99,27 @@ func jsonFile(v any) string {
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// MeshWaitBoundMillis is how long the agent may wait on the mesh's console for one call: mesh_wait's longest
|
||||
// wait (600 s, mesh-tools node-tools console.WaitLongest) plus a minute's margin. The agent bounds an HTTP
|
||||
// server's request by the largest of 60 s, the server entry's "timeout" and MCP_TIMEOUT, and a call that
|
||||
// receives nothing for CLAUDE_CODE_MCP_TOOL_IDLE_TIMEOUT (5 minutes by default) is aborted
|
||||
// (code.claude.com/docs/en/mcp.md). Without these every mesh_wait ended "The operation timed out." at a
|
||||
// minute, whatever its timeout (issue 1871).
|
||||
const MeshWaitBoundMillis = 660000
|
||||
|
||||
// meshWaitEnv lays the idle bound into the environment the settings give every session, the mesh's key winning
|
||||
// over the operator's, like the forge's keys.
|
||||
func meshWaitEnv(managed map[string]any) {
|
||||
env := map[string]any{}
|
||||
if held, ok := managed["env"].(map[string]any); ok {
|
||||
for k, v := range held {
|
||||
env[k] = v
|
||||
}
|
||||
}
|
||||
env["CLAUDE_CODE_MCP_TOOL_IDLE_TIMEOUT"] = strconv.Itoa(MeshWaitBoundMillis)
|
||||
managed["env"] = env
|
||||
}
|
||||
|
||||
// Render composes the three files. registered — what was registered through this module and applies
|
||||
// here — is laid over the servers the operator set in its settings; config is the rest of what was
|
||||
// registered (ADR 0216): its settings laid over the operator's `managed_settings`, its instruction sections
|
||||
@@ -119,7 +141,7 @@ func RenderWithMesh(facts Facts, settings Settings, binding *Binding, helperPath
|
||||
servers[name] = entry
|
||||
}
|
||||
}
|
||||
servers[meshEntry] = map[string]any{"type": "http", "url": facts.Console}
|
||||
servers[meshEntry] = map[string]any{"type": "http", "url": facts.Console, "timeout": MeshWaitBoundMillis}
|
||||
|
||||
// The operator's `managed_settings` (ADR 0213), set at the controller's terminal alone: nothing registered
|
||||
// through this module's tools is laid over it (novox/hq issue 340).
|
||||
@@ -143,6 +165,7 @@ func RenderWithMesh(facts Facts, settings Settings, binding *Binding, helperPath
|
||||
managed[key] = entries
|
||||
}
|
||||
forgeEnv(managed, facts.ForgeCredentials)
|
||||
meshWaitEnv(managed)
|
||||
if binding != nil && binding.Kind == "api-key" {
|
||||
managed["apiKeyHelper"] = helperPath
|
||||
}
|
||||
|
||||
@@ -32,11 +32,11 @@
|
||||
"withKey": {
|
||||
"managed-mcp.json": "{\n \"mcpServers\": {\n \"anton\": {\n \"type\": \"stdio\",\n \"command\": \"node\",\n \"args\": [\n \"/a/b.js\"\n ],\n \"env\": {}\n },\n \"docs\": {\n \"type\": \"stdio\",\n \"command\": \"docs-mcp\",\n \"args\": [\n \"--x\"\n ]\n },\n \"mesh\": {\n \"type\": \"http\",\n \"url\": \"http://127.0.0.1:4270/mcp\"\n },\n \"search\": {\n \"type\": \"http\",\n \"url\": \"https://s.example/mcp\"\n }\n }\n}\n",
|
||||
"managed-settings.json": "{\n \"attribution\": {\n \"commit\": \"\",\n \"pr\": \"\"\n },\n \"allowAllClaudeAiMcps\": true,\n \"apiKeyHelper\": \"/state/api-key-helper\"\n}\n",
|
||||
"CLAUDE.md": "# This machine is a node of a Novox mesh\n\nWritten by the mesh's `claude-code` module. Edit the module's settings or the catalogue, never this file:\nit is rewritten whenever the module renders.\n\n## Who this node is\n\n- **Node:** `workstation`\n- **Role:** the laptop\n- The other nodes, their roles and what runs where: ask the controller (`mesh-controller.nodes`,\n `mesh-controller.node`). Nothing here lists them, because a copy drifts.\n\n## How a session on this mesh works\n\nThe console is the only way to the mesh: the MCP server named `mesh`. It offers five tools, and\neverything else is an address you find and call through them:\n\n- `mesh_search` — words in, matching addresses out. `mesh_describe` — one address's arguments.\n- `mesh_call` — call an address. A seat the mesh holds once is `<seat>.<verb>` (the mesh's own verbs\n are `mesh-controller.<verb>`: `status`, `plan`, `node`, `assign`, `push`, `settings`);\n a module on a machine is `<node>/<module>.<tool>`.\n- `mesh_overview` and `mesh_machine` — the mesh's seats and machines, and what one machine runs.\n\n- **Symptom first.** For an error, a failing service or anything unexpected, search the record with the\n literal text before forming a hypothesis: the records module's `records_search`, then\n `records_read`.\n- **Ask the mesh before changing it**, and change it through the controller's verbs or the catalogue.\n- **A licence** through the `anthropic-licence-manager` seat's verbs. Never edit the agent's credentials\n file by hand, never print or ask for a token.\n\n## Hard rules\n\n- A file the mesh manages is changed through the verb or the catalogue that owns it, never on disk. If\n unsure, `mesh-controller.plan` for the node says what the mesh writes there.\n- Never write to a store's database by hand; schema changes are numbered migrations.\n- Never push to a main branch: a branch, a pull request, and a human approval for every merge.\n- The mesh creates no symlinks, and nobody else does either.\n- A package is declared in a module, never installed by hand.\n\n## Conventions\n\n- Commit messages are concise, in the imperative, about why.\n- Test before pushing: nodes update unattended.\n- The playbooks in the record say how research, decisions, designs, issues and hand-offs are done.\n"
|
||||
"CLAUDE.md": "# This machine is a node of a Novox mesh\n\nWritten by the mesh's `claude-code` module. Edit the module's settings or the catalog, never this file:\nit is rewritten whenever the module renders.\n\n## Who this node is\n\n- **Node:** `workstation`\n- **Role:** the laptop\n- The other nodes, their roles and what runs where: ask the controller (`mesh-controller.nodes`,\n `mesh-controller.node`). Nothing here lists them, because a copy drifts.\n\n## How a session on this mesh works\n\nThe console is the only way to the mesh: the MCP server named `mesh`. It offers five tools, and\neverything else is an address you find and call through them:\n\n- `mesh_search` — words in, matching addresses out. `mesh_describe` — one address's arguments.\n- `mesh_call` — call an address. A seat the mesh holds once is `<seat>.<verb>` (the mesh's own verbs\n are `mesh-controller.<verb>`: `status`, `plan`, `node`, `assign`, `push`, `settings`);\n a module on a machine is `<node>/<module>.<tool>`.\n- `mesh_overview` and `mesh_machine` — the mesh's seats and machines, and what one machine runs.\n\n- **Symptom first.** For an error, a failing service or anything unexpected, search the record with the\n literal text before forming a hypothesis: the records module's `records_search`, then\n `records_read`.\n- **Ask the mesh before changing it**, and change it through the controller's verbs or the catalog.\n- **A licence** through the `anthropic-licence-manager` seat's verbs. Never edit the agent's credentials\n file by hand, never print or ask for a token.\n\n## Hard rules\n\n- A file the mesh manages is changed through the verb or the catalog that owns it, never on disk. If\n unsure, `mesh-controller.plan` for the node says what the mesh writes there.\n- Never write to a store's database by hand; schema changes are numbered migrations.\n- Never push to a main branch: a branch, a pull request, and a human approval for every merge.\n- The mesh creates no symlinks, and nobody else does either.\n- A package is declared in a module, never installed by hand.\n\n## Conventions\n\n- Commit messages are concise, in the imperative, about why.\n- Test before pushing: nodes update unattended.\n- The playbooks in the record say how research, decisions, designs, issues and hand-offs are done.\n"
|
||||
},
|
||||
"plain": {
|
||||
"managed-mcp.json": "{\n \"mcpServers\": {\n \"mesh\": {\n \"type\": \"http\",\n \"url\": \"http://127.0.0.1:4270/mcp\"\n }\n }\n}\n",
|
||||
"managed-settings.json": "{\n \"attribution\": {\n \"commit\": \"\",\n \"pr\": \"\"\n },\n \"allowAllClaudeAiMcps\": true\n}\n",
|
||||
"CLAUDE.md": "# This machine is a node of a Novox mesh\n\nWritten by the mesh's `claude-code` module. Edit the module's settings or the catalogue, never this file:\nit is rewritten whenever the module renders.\n\n## Who this node is\n\n- **Node:** `workstation`\n- **Role:** not stated — set it in this module's settings for the node\n- The other nodes, their roles and what runs where: ask the controller (`mesh-controller.nodes`,\n `mesh-controller.node`). Nothing here lists them, because a copy drifts.\n\n## How a session on this mesh works\n\nThe console is the only way to the mesh: the MCP server named `mesh`. It offers five tools, and\neverything else is an address you find and call through them:\n\n- `mesh_search` — words in, matching addresses out. `mesh_describe` — one address's arguments.\n- `mesh_call` — call an address. A seat the mesh holds once is `<seat>.<verb>` (the mesh's own verbs\n are `mesh-controller.<verb>`: `status`, `plan`, `node`, `assign`, `push`, `settings`);\n a module on a machine is `<node>/<module>.<tool>`.\n- `mesh_overview` and `mesh_machine` — the mesh's seats and machines, and what one machine runs.\n\n- **Symptom first.** For an error, a failing service or anything unexpected, search the record with the\n literal text before forming a hypothesis: the records module's `records_search`, then\n `records_read`.\n- **Ask the mesh before changing it**, and change it through the controller's verbs or the catalogue.\n- **A licence** through the `anthropic-licence-manager` seat's verbs. Never edit the agent's credentials\n file by hand, never print or ask for a token.\n\n## Hard rules\n\n- A file the mesh manages is changed through the verb or the catalogue that owns it, never on disk. If\n unsure, `mesh-controller.plan` for the node says what the mesh writes there.\n- Never write to a store's database by hand; schema changes are numbered migrations.\n- Never push to a main branch: a branch, a pull request, and a human approval for every merge.\n- The mesh creates no symlinks, and nobody else does either.\n- A package is declared in a module, never installed by hand.\n\n## Conventions\n\n- Commit messages are concise, in the imperative, about why.\n- Test before pushing: nodes update unattended.\n- The playbooks in the record say how research, decisions, designs, issues and hand-offs are done.\n"
|
||||
"CLAUDE.md": "# This machine is a node of a Novox mesh\n\nWritten by the mesh's `claude-code` module. Edit the module's settings or the catalog, never this file:\nit is rewritten whenever the module renders.\n\n## Who this node is\n\n- **Node:** `workstation`\n- **Role:** not stated — set it in this module's settings for the node\n- The other nodes, their roles and what runs where: ask the controller (`mesh-controller.nodes`,\n `mesh-controller.node`). Nothing here lists them, because a copy drifts.\n\n## How a session on this mesh works\n\nThe console is the only way to the mesh: the MCP server named `mesh`. It offers five tools, and\neverything else is an address you find and call through them:\n\n- `mesh_search` — words in, matching addresses out. `mesh_describe` — one address's arguments.\n- `mesh_call` — call an address. A seat the mesh holds once is `<seat>.<verb>` (the mesh's own verbs\n are `mesh-controller.<verb>`: `status`, `plan`, `node`, `assign`, `push`, `settings`);\n a module on a machine is `<node>/<module>.<tool>`.\n- `mesh_overview` and `mesh_machine` — the mesh's seats and machines, and what one machine runs.\n\n- **Symptom first.** For an error, a failing service or anything unexpected, search the record with the\n literal text before forming a hypothesis: the records module's `records_search`, then\n `records_read`.\n- **Ask the mesh before changing it**, and change it through the controller's verbs or the catalog.\n- **A licence** through the `anthropic-licence-manager` seat's verbs. Never edit the agent's credentials\n file by hand, never print or ask for a token.\n\n## Hard rules\n\n- A file the mesh manages is changed through the verb or the catalog that owns it, never on disk. If\n unsure, `mesh-controller.plan` for the node says what the mesh writes there.\n- Never write to a store's database by hand; schema changes are numbered migrations.\n- Never push to a main branch: a branch, a pull request, and a human approval for every merge.\n- The mesh creates no symlinks, and nobody else does either.\n- A package is declared in a module, never installed by hand.\n\n## Conventions\n\n- Commit messages are concise, in the imperative, about why.\n- Test before pushing: nodes update unattended.\n- The playbooks in the record say how research, decisions, designs, issues and hand-offs are done.\n"
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,9 @@ import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
@@ -62,3 +65,60 @@ func TestEveryFileTakesItsTextFromATemplate(t *testing.T) {
|
||||
t.Errorf("the build packs files/ into %d archives, not one", packs)
|
||||
}
|
||||
}
|
||||
|
||||
// The packaged dnsmasq.service orders itself Before= network-online.target and nss-lookup.target. A drop-in
|
||||
// that orders it After= either of them makes an ordering cycle, and systemd drops the start job: the mesh
|
||||
// resolver stayed down for 90 minutes at a boot (novox/hq issue 1126). Wants=, Requires= and BindsTo= are
|
||||
// refused too: they pull the target in, and with the packaged ordering they are the same mistake. Because the
|
||||
// tunnel is ordered after those targets, the drop-in cannot wait for the address either: a start that finds it
|
||||
// missing fails at once and Restart=on-failure brings it up (see the next test).
|
||||
func TestDropInDoesNotOrderAfterATargetThePackagedUnitOrdersBefore(t *testing.T) {
|
||||
raw, err := os.ReadFile(filepath.Join(moduleDir, "files", "mesh.conf"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
forbidden := []string{"network-online.target", "nss-lookup.target"}
|
||||
for n, line := range strings.Split(string(raw), "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if strings.HasPrefix(line, "#") {
|
||||
continue
|
||||
}
|
||||
key, val, ok := strings.Cut(line, "=")
|
||||
key, val = strings.TrimSpace(key), strings.TrimSpace(val)
|
||||
if !ok || (key != "After" && key != "Requires" && key != "Wants" && key != "BindsTo") {
|
||||
continue
|
||||
}
|
||||
for _, target := range forbidden {
|
||||
if strings.Contains(val, target) {
|
||||
t.Errorf("mesh.conf line %d: %s=%s orders dnsmasq after a target its packaged unit orders itself before: a cycle", n+1, key, val)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The wait for the address in ExecStartPre is at most two seconds: the tunnel that brings the address is ordered
|
||||
// after network-online.target, which the resolver is ordered before, so a longer wait would hold the tunnel back.
|
||||
func TestAddressWaitIsAtMostTwoSeconds(t *testing.T) {
|
||||
raw, err := os.ReadFile(filepath.Join(moduleDir, "files", "mesh.conf"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
found := false
|
||||
for _, line := range strings.Split(string(raw), "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if !strings.HasPrefix(line, "ExecStartPre=") {
|
||||
continue
|
||||
}
|
||||
m := regexp.MustCompile(`seq 1 (\d+)`).FindStringSubmatch(line)
|
||||
if m == nil {
|
||||
t.Fatalf("ExecStartPre has no bounded wait (seq 1 N): %s", line)
|
||||
}
|
||||
found = true
|
||||
if n, _ := strconv.Atoi(m[1]); n > 2 {
|
||||
t.Errorf("ExecStartPre waits up to %d s for the address, more than 2 s", n)
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("mesh.conf has no ExecStartPre")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,14 +1,15 @@
|
||||
# Written by the mesh (module dnsmasq, novox/hq issue 348). Replaced on every push.
|
||||
#
|
||||
# The resolver binds its addresses once, at start (bind-interfaces), so it starts only once this
|
||||
# machine's private address is there: it waits up to 80 seconds for it, and a start that found it
|
||||
# missing is tried again every ten seconds rather than left failed.
|
||||
# machine's private address is there: a start that finds it missing fails after at most two seconds and
|
||||
# is tried again every ten seconds (Restart=on-failure, no start limit) until the tunnel has brought it.
|
||||
# It does not order itself After= or Wants= network-online.target, and its wait is short: the packaged
|
||||
# unit is Before= network-online.target and nss-lookup.target, and the tunnel is ordered after them, so
|
||||
# a long wait would hold the tunnel back and a drop-in ordering would be a cycle (issue 1126).
|
||||
[Unit]
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
StartLimitIntervalSec=0
|
||||
|
||||
[Service]
|
||||
ExecStartPre=/usr/bin/sh -c 'for i in $(seq 1 80); do ip -4 -o addr show to ${machine:address} | grep -q . && exit 0; sleep 1; done; echo "${machine:address} is not on this machine" >&2; exit 1'
|
||||
ExecStartPre=/usr/bin/sh -c 'for i in $(seq 1 2); do ip -4 -o addr show to ${machine:address} | grep -q . && exit 0; sleep 1; done; echo "${machine:address} is not on this machine" >&2; exit 1'
|
||||
Restart=on-failure
|
||||
RestartSec=10s
|
||||
|
||||
@@ -1 +1,25 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# the copies of desktop.go and its tests in every module that carries one.
|
||||
modules/blueman/cmd/blueman-tools/desktop.go
|
||||
modules/blueman/cmd/blueman-tools/desktop_test.go
|
||||
modules/blueman/cmd/blueman-tools/copies_test.go
|
||||
modules/forticlient/cmd/forticlient-tools/desktop.go
|
||||
modules/forticlient/cmd/forticlient-tools/desktop_test.go
|
||||
modules/forticlient/cmd/forticlient-tools/copies_test.go
|
||||
modules/nextcloud-client/cmd/nextcloud-client-tools/desktop.go
|
||||
modules/nextcloud-client/cmd/nextcloud-client-tools/desktop_test.go
|
||||
modules/nextcloud-client/cmd/nextcloud-client-tools/copies_test.go
|
||||
modules/nm-applet/cmd/nm-applet-tools/desktop.go
|
||||
modules/nm-applet/cmd/nm-applet-tools/desktop_test.go
|
||||
modules/nm-applet/cmd/nm-applet-tools/copies_test.go
|
||||
modules/openrazer/cmd/openrazer-tools/desktop.go
|
||||
modules/openrazer/cmd/openrazer-tools/desktop_test.go
|
||||
modules/openrazer/cmd/openrazer-tools/copies_test.go
|
||||
modules/polychromatic/cmd/polychromatic-tools/desktop.go
|
||||
modules/polychromatic/cmd/polychromatic-tools/desktop_test.go
|
||||
modules/polychromatic/cmd/polychromatic-tools/copies_test.go
|
||||
modules/slack/cmd/slack-tools/desktop.go
|
||||
modules/slack/cmd/slack-tools/desktop_test.go
|
||||
modules/slack/cmd/slack-tools/copies_test.go
|
||||
|
||||
@@ -304,6 +304,23 @@ named and asked nothing; the start's pass is run again ten minutes apart until e
|
||||
grant by hand, now, on every repository or on those named, for every account or those named; each grant read back
|
||||
from the forge, a refusal answered in the forge's own words).
|
||||
|
||||
### Teams are read with the administrator's login
|
||||
|
||||
`gitea_team_rights` (issue 1563) answers the forge's teams of an organisation (novox when not given), or one team: its
|
||||
members, permission, units (read, write or admin per unit), whether it covers all repositories, and whether it is
|
||||
read-only. With a `user` it adds the teams that user is in and its effective right on every repository (or those named),
|
||||
listing those above read: the check that a build key's user, `nox-builder` in `builders`, reads and pushes nothing. The
|
||||
tools' token lacks `read:organization` (the forge answers 403), so the read goes with the administrator's login, GET only.
|
||||
|
||||
### Two settings have named tools, and a retargeted pull request is judged again
|
||||
|
||||
`gitea_api` refuses a repository's settings and a pull request's base (guard.ts). Issue 1422 note 7 gave each its tool:
|
||||
`gitea_default_branch_set` (owner, repo, branch, why) and `gitea_pull_base_set` (owner, repo, number, base, why). Both
|
||||
act as the tools' account, need a reason, refuse a branch the repository does not have, and answer the value before and
|
||||
after; a default must also be a branch a protection rule guards, and only an open pull request has its base changed.
|
||||
A pull request's announced head is kept under `owner/repo#number@base`, so a retarget (same head, new base) is
|
||||
announced again into the new base by the next poll instead of being skipped as already announced.
|
||||
|
||||
### The merging account writes the repositories the mesh delivers
|
||||
|
||||
`mesh-tools` sets every commit status and merges; outside the organisation it only reads (above), so a pull request of
|
||||
|
||||
+171
-9
@@ -9,6 +9,7 @@ import { ConfiguredToken, MintedToken, type TokenSource } from "./token.js";
|
||||
import { MAIN_BRANCH, mainRuleBody, protectionBody, type BranchProtection, type ProtectionWanted } from "./protection.js";
|
||||
import { Refused, checkSentPath, ruleCovers, pathSegment, pathSegments } from "./guard.js";
|
||||
import { isCommit } from "./checks.js";
|
||||
import type { ForgeTeam } from "./teamrights.js";
|
||||
|
||||
/** A repository's path on the forge, its owner and name each one segment (guard.ts pathSegment): uniform for every
|
||||
* call, whether the names came from a caller or from the forge's own answer. */
|
||||
@@ -174,8 +175,27 @@ function ignoreSecretSetting(cfg: Record<string, unknown>, key: string): void {
|
||||
console.error(`the setting ${key} is ignored: it comes from the vault`);
|
||||
}
|
||||
|
||||
/** How long the forge may take to answer one request before it is cut and said (novox/hq issue 1577: a request for a 5 MB
|
||||
* file never ended, and every read through the module timed out behind it). */
|
||||
export const FORGE_REQUEST_TIMEOUT_MS = 20_000;
|
||||
|
||||
/** The most a tool's answer carries of one file: the bus's message bound is 1 MiB and a file travels as text (issue 1577). */
|
||||
export const MAX_FILE_BYTES = 512 * 1024;
|
||||
|
||||
/** One request to the forge, cut at the limit and failing loud naming the request. */
|
||||
async function timedFetch(url: string, init: RequestInit, label: string, timeoutMs: number): Promise<Response> {
|
||||
try {
|
||||
return await fetch(url, { ...init, signal: init.signal ? AbortSignal.any([init.signal, AbortSignal.timeout(timeoutMs)]) : AbortSignal.timeout(timeoutMs) });
|
||||
} catch (err) {
|
||||
if (err instanceof Error && err.name === "TimeoutError") throw new Error(`Gitea API ${label}: no answer in ${timeoutMs / 1000} s, the request was cut`);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
export class GiteaClient {
|
||||
readonly baseUrl: string;
|
||||
/** The limit of one request; a test sets it short. */
|
||||
requestTimeoutMs = FORGE_REQUEST_TIMEOUT_MS;
|
||||
private readonly tokens: TokenSource;
|
||||
|
||||
/** A token given as a string is one somebody configured; a source decides for itself (token.ts). */
|
||||
@@ -230,14 +250,14 @@ export class GiteaClient {
|
||||
|
||||
private send(path: string, options: RequestInit, token: string): Promise<Response> {
|
||||
checkSentPath(path);
|
||||
return fetch(`${this.baseUrl}/api/v1${path}`, {
|
||||
return timedFetch(`${this.baseUrl}/api/v1${path}`, {
|
||||
...options,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `token ${token}`,
|
||||
...(options.headers as Record<string, string> | undefined),
|
||||
},
|
||||
});
|
||||
}, `${options.method ?? "GET"} ${path}`, this.requestTimeoutMs);
|
||||
}
|
||||
|
||||
/** Generic authenticated API call — the escape hatch for endpoints without a dedicated method.
|
||||
@@ -722,6 +742,15 @@ export class GiteaClient {
|
||||
return { created: true, rule };
|
||||
}
|
||||
|
||||
/** Write a branch's rule with exactly the fields given (gitea_branch_protection_set_full): an edit of the rule named
|
||||
* for the branch, or a new one. As this client's account, which the forge refuses where it is no administrator. */
|
||||
async writeBranchProtection(owner: string, repo: string, branch: string, body: Record<string, unknown>, creating: boolean): Promise<BranchProtection> {
|
||||
return creating
|
||||
? this.request<BranchProtection>(`${repoPath(owner, repo)}/branch_protections`, { method: "POST", body: JSON.stringify({ rule_name: branch, ...body }) })
|
||||
: this.request<BranchProtection>(`${repoPath(owner, repo)}/branch_protections/${pathSegments("branch", branch)}`,
|
||||
{ method: "PATCH", body: JSON.stringify(body) });
|
||||
}
|
||||
|
||||
async createPullRequest(
|
||||
owner: string,
|
||||
repo: string,
|
||||
@@ -771,6 +800,22 @@ export class GiteaClient {
|
||||
/** One file's contents at a ref (default the repository's default branch), decoded. */
|
||||
async getFile(owner: string, repo: string, path: string, ref?: string): Promise<{ path: string; ref?: string; sha: string; size: number; content: string }> {
|
||||
const qs = ref ? `?ref=${encodeURIComponent(ref)}` : "";
|
||||
// The size first, from the file's directory listing, which carries no content: a file over what an answer can carry is
|
||||
// refused without its body being fetched (issue 1577). A file the listing does not show is asked for as before.
|
||||
const clean = path.replace(/^\/+|\/+$/g, "");
|
||||
pathSegments("file path", clean); // a path that is another path or user is refused before anything reaches the forge
|
||||
const slash = clean.lastIndexOf("/");
|
||||
const dir = slash < 0 ? "" : clean.slice(0, slash);
|
||||
let listing: any[] | null = null;
|
||||
try {
|
||||
listing = await this.request<any[]>(`${repoPath(owner, repo)}/contents${dir ? "/" + pathSegments("file path", dir) : ""}${qs}`);
|
||||
} catch (err) {
|
||||
if (!(err instanceof Error) || !/: 404 /.test(err.message)) throw err;
|
||||
}
|
||||
const entry = Array.isArray(listing) ? listing.find((e) => e?.path === clean) : undefined;
|
||||
if (entry && Number(entry.size) > MAX_FILE_BYTES) {
|
||||
throw new Error(`the file is ${Number(entry.size)} bytes, over the ${MAX_FILE_BYTES / 1024} KiB a tool answer can carry; read it from a clone`);
|
||||
}
|
||||
const f = await this.request<any>(`${repoPath(owner, repo)}/contents/${pathSegments("file path", path.replace(/^\/+|\/+$/g, ""))}${qs}`);
|
||||
if (!f || f.type !== "file") throw new Error(`Gitea API: ${path} is not a file`);
|
||||
const content = f.encoding === "base64" ? Buffer.from(String(f.content ?? ""), "base64").toString("utf8") : String(f.content ?? "");
|
||||
@@ -782,6 +827,30 @@ export class GiteaClient {
|
||||
return (raw ?? []).map((b) => ({ name: String(b?.name ?? ""), commit: String(b?.commit?.id ?? ""), protected: Boolean(b?.protected) }));
|
||||
}
|
||||
|
||||
/** One branch as the forge keeps it: whether a protection rule guards it; null when the repository has no such branch. */
|
||||
async branchState(owner: string, repo: string, branch: string): Promise<{ protected: boolean } | null> {
|
||||
try {
|
||||
const b = await this.request<any>(`${repoPath(owner, repo)}/branches/${pathSegments("branch", branch)}`);
|
||||
return { protected: Boolean(b?.protected) };
|
||||
} catch (err) {
|
||||
if (/: 404 /.test(err instanceof Error ? err.message : "")) return null;
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/** Make a branch the repository's default (issue 1422): what the forge shows, clones and opens pull requests against. */
|
||||
async setDefaultBranch(owner: string, repo: string, branch: string): Promise<GiteaRepo> {
|
||||
return GiteaClient.mapRepo(await this.request<any>(repoPath(owner, repo), { method: "PATCH", body: JSON.stringify({ default_branch: branch }) }));
|
||||
}
|
||||
|
||||
/** Change the branch an open pull request merges into (issue 1422). The head stays; the forge recomputes the diff. */
|
||||
async setPullBase(owner: string, repo: string, index: number, base: string): Promise<GiteaPull> {
|
||||
indexOf(index);
|
||||
return GiteaClient.mapPull(
|
||||
await this.request<any>(`${repoPath(owner, repo)}/pulls/${index}`, { method: "PATCH", body: JSON.stringify({ base }) }),
|
||||
);
|
||||
}
|
||||
|
||||
async deleteBranch(owner: string, repo: string, branch: string): Promise<void> {
|
||||
await this.request(`${repoPath(owner, repo)}/branches/${pathSegments("branch", branch)}`, { method: "DELETE" });
|
||||
}
|
||||
@@ -940,14 +1009,14 @@ export class GiteaAdmin {
|
||||
* throwing on it — the caller decides which non-2xx codes are idempotent successes. Only an
|
||||
* unexpected status becomes an error, and only where the caller says so. */
|
||||
private async request(path: string, options: RequestInit = {}): Promise<AdminResponse> {
|
||||
const res = await fetch(`${this.baseUrl}/api/v1${path}`, {
|
||||
const res = await timedFetch(`${this.baseUrl}/api/v1${path}`, {
|
||||
...options,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: this.authorization,
|
||||
...(options.headers as Record<string, string> | undefined),
|
||||
},
|
||||
});
|
||||
}, `${options.method ?? "GET"} ${path}`, FORGE_REQUEST_TIMEOUT_MS);
|
||||
const text = await res.text();
|
||||
let body: any = null;
|
||||
if (text) {
|
||||
@@ -976,6 +1045,31 @@ export class GiteaAdmin {
|
||||
return Array.isArray(res.body) ? res.body : [];
|
||||
}
|
||||
|
||||
/** Write a branch's rule with exactly the fields given, as the forge's administrator (gitea_branch_protection_set_full;
|
||||
* the operator, 2026-10-28): an edit of the rule named for the branch, or a new one. */
|
||||
async writeBranchProtection(owner: string, repo: string, branch: string, body: Record<string, unknown>, creating: boolean): Promise<BranchProtection> {
|
||||
const path = creating ? `${repoPath(owner, repo)}/branch_protections` : `${repoPath(owner, repo)}/branch_protections/${pathSegments("branch", branch)}`;
|
||||
const res = await this.request(path, { method: creating ? "POST" : "PATCH", body: JSON.stringify(creating ? { rule_name: branch, ...body } : body) });
|
||||
if (res.status !== 200 && res.status !== 201) GiteaAdmin.fail(path, res);
|
||||
return res.body as BranchProtection;
|
||||
}
|
||||
|
||||
/** A repository as the forge answers it to its administrator: every setting (gitea_repo_settings_set_full). */
|
||||
async repoSettings(owner: string, repo: string): Promise<Record<string, unknown>> {
|
||||
const path = repoPath(owner, repo);
|
||||
const res = await this.request(path);
|
||||
if (res.status !== 200) GiteaAdmin.fail(path, res);
|
||||
return res.body as Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** Edit a repository's settings with exactly the fields given, as the forge's administrator. */
|
||||
async editRepoSettings(owner: string, repo: string, body: Record<string, unknown>): Promise<Record<string, unknown>> {
|
||||
const path = repoPath(owner, repo);
|
||||
const res = await this.request(path, { method: "PATCH", body: JSON.stringify(body) });
|
||||
if (res.status !== 200) GiteaAdmin.fail(path, res);
|
||||
return res.body as Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** Put main's rule right in every repository of an organisation (novox/hq ADR 0398 §2, issue 948): no push, only
|
||||
* the tools' account merges, and no status required (ADR 0534 §1). The rule there is edited; a repository with no
|
||||
* rule for main gets one. Answers the repositories it touched. With the administrator's login:
|
||||
@@ -1047,12 +1141,19 @@ export class GiteaAdmin {
|
||||
return true;
|
||||
}
|
||||
|
||||
/** What the forge itself says an account may do on a repository: none, read, write, admin or owner. */
|
||||
/** What the forge gives an account on a repository, effective: none, read, write, admin or owner. It is the repository as
|
||||
* that account sees it (the site administrator acts as the account with `sudo`), so access through an organisation's team
|
||||
* counts. The collaborator endpoint is not used: it names only a collaborator's own permission, and says "read" of an
|
||||
* account that writes through a team (novox/hq issue 1945). */
|
||||
async repositoryPermission(owner: string, repo: string, user: string): Promise<string> {
|
||||
const path = `${repoPath(owner, repo)}/collaborators/${encodeURIComponent(user)}/permission`;
|
||||
const path = `${repoPath(owner, repo)}?sudo=${encodeURIComponent(user)}`;
|
||||
const res = await this.request(path);
|
||||
if (res.status === 404) return "none";
|
||||
if (res.status !== 200) GiteaAdmin.fail(path, res);
|
||||
return String(res.body?.permission ?? "none");
|
||||
const p = res.body?.permissions;
|
||||
if (p?.admin === true) return res.body?.owner?.login === user ? "owner" : "admin";
|
||||
if (p?.push === true) return "write";
|
||||
return p?.pull === true ? "read" : "none";
|
||||
}
|
||||
|
||||
/** Make an account a collaborator of a repository that reads only. The forge asks a repository's administrator for
|
||||
@@ -1067,6 +1168,38 @@ export class GiteaAdmin {
|
||||
await this.addCollaborator(owner, repo, user, "write");
|
||||
}
|
||||
|
||||
/** The rule that applies to main in a repository, as its administrator reads it: a rule named main, else the first glob
|
||||
* rule covering main; null when none does. With the administrator's login. */
|
||||
async mainRule(owner: string, repo: string): Promise<BranchProtection | null> {
|
||||
const rules = await this.branchProtections(owner, repo);
|
||||
return rules.find((r) => (r.rule_name ?? r.branch_name) === MAIN_BRANCH)
|
||||
?? rules.find((r) => ruleCovers(String(r.rule_name ?? r.branch_name ?? ""), MAIN_BRANCH)) ?? null;
|
||||
}
|
||||
|
||||
/** Put an account on main's merge whitelist in a repository of a user, and nothing else (the operator, 2026-10-26:
|
||||
* write, the rule untouched). Only an existing rule whose merge whitelist is on is changed, and only its whitelist: the
|
||||
* account is added to the logins already there. Push, status checks, approvals and the administrator override stay as
|
||||
* the operator set them; with no rule, or the whitelist off, nothing is written (whoever writes may merge). Read back:
|
||||
* the forge drops a whitelist login that cannot write the repository and still answers success, so a whitelist read back
|
||||
* without the account fails loud. Answers the rule as the forge holds it after. With the administrator's login. */
|
||||
async ensureMergerListed(owner: string, repo: string, account: string): Promise<{ changed: boolean; rule: BranchProtection | null }> {
|
||||
const rule = await this.mainRule(owner, repo);
|
||||
const listed = rule?.merge_whitelist_usernames ?? [];
|
||||
if (!rule || rule.enable_merge_whitelist !== true || listed.includes(account)) return { changed: false, rule };
|
||||
const name = String(rule.rule_name ?? rule.branch_name);
|
||||
const path = `${repoPath(owner, repo)}/branch_protections/${encodeURIComponent(name)}`;
|
||||
const res = await this.request(path, { method: "PATCH", body: JSON.stringify({ merge_whitelist_usernames: [...listed, account] }) });
|
||||
if (res.status !== 200) GiteaAdmin.fail(path, res);
|
||||
const after = (await this.branchProtections(owner, repo)).find((r) => (r.rule_name ?? r.branch_name) === name);
|
||||
if (!after) throw new Error(`main's rule ${name} of ${owner}/${repo} was written, but the forge does not list it`);
|
||||
const held = after.merge_whitelist_usernames ?? [];
|
||||
if (after.enable_merge_whitelist === true && !held.includes(account)) {
|
||||
throw new Error(`${account} was added to the merge whitelist of main's rule ${name} of ${owner}/${repo}, but the forge ` +
|
||||
`holds [${held.join(", ")}]: it drops a login that cannot write the repository`);
|
||||
}
|
||||
return { changed: true, rule: after };
|
||||
}
|
||||
|
||||
private async addCollaborator(owner: string, repo: string, user: string, permission: "read" | "write"): Promise<void> {
|
||||
const path = `${repoPath(owner, repo)}/collaborators/${encodeURIComponent(user)}`;
|
||||
const res = await this.request(path, { method: "PUT", body: JSON.stringify({ permission }) });
|
||||
@@ -1135,6 +1268,35 @@ export class GiteaAdmin {
|
||||
return match ? Number(match.id) : null;
|
||||
}
|
||||
|
||||
/** The teams of an organisation, as the forge's Team objects (issue 1563). With the administrator's login: the tools'
|
||||
* account lacks read:organization, so the list answers it 403. Only a read. */
|
||||
async teams(org: string): Promise<ForgeTeam[]> {
|
||||
const all: ForgeTeam[] = [];
|
||||
for (let page = 1; page <= 50; page++) {
|
||||
const path = `/orgs/${encodeURIComponent(org)}/teams?limit=50&page=${page}`;
|
||||
const res = await this.request(path);
|
||||
if (res.status !== 200) GiteaAdmin.fail(`/orgs/${org}/teams`, res);
|
||||
const list: ForgeTeam[] = Array.isArray(res.body) ? res.body : [];
|
||||
all.push(...list);
|
||||
if (list.length < 50) break;
|
||||
}
|
||||
return all;
|
||||
}
|
||||
|
||||
/** The logins of a team's members (GET /teams/{id}/members). Only a read. */
|
||||
async teamMembers(teamId: number): Promise<string[]> {
|
||||
const all: string[] = [];
|
||||
for (let page = 1; page <= 50; page++) {
|
||||
const path = `/teams/${teamId}/members?limit=50&page=${page}`;
|
||||
const res = await this.request(path);
|
||||
if (res.status !== 200) GiteaAdmin.fail(`/teams/${teamId}/members`, res);
|
||||
const list: any[] = Array.isArray(res.body) ? res.body : [];
|
||||
all.push(...list.map((u) => String(u?.login ?? u?.username ?? "")).filter(Boolean));
|
||||
if (list.length < 50) break;
|
||||
}
|
||||
return all;
|
||||
}
|
||||
|
||||
/** Ensure a user exists with exactly this password. Created if absent; if already there, its
|
||||
* password is patched — so the mesh minting a new secret takes on the next reconcile.
|
||||
*
|
||||
@@ -1192,9 +1354,9 @@ export class GiteaAdmin {
|
||||
* other answer rejects (novox/hq issue 120).
|
||||
*/
|
||||
async holdsTeamMember(org: string, team: string, username: string, password: string): Promise<boolean> {
|
||||
const me = await fetch(`${this.baseUrl}/api/v1/user`, {
|
||||
const me = await timedFetch(`${this.baseUrl}/api/v1/user`, {
|
||||
headers: { Authorization: "Basic " + Buffer.from(`${username}:${password}`).toString("base64") },
|
||||
});
|
||||
}, "GET /user", FORGE_REQUEST_TIMEOUT_MS);
|
||||
if (me.status === 401 || me.status === 403) return false;
|
||||
if (me.status !== 200) throw new Error(`Gitea GET /user as ${username}: ${me.status}`);
|
||||
const teams = await this.request(`/orgs/${encodeURIComponent(org)}/teams?limit=50`);
|
||||
|
||||
@@ -12,6 +12,8 @@
|
||||
// forge as themselves;
|
||||
// - `gitea_branch_protection_set` lets no person push and no administrator merge past a rule, and on main keeps
|
||||
// the tools' account the only one that merges; it requires no status, as no rule of the mesh does (ADR 0534 §5);
|
||||
// - `gitea_default_branch_set` and `gitea_pull_base_set` are the two settings gitea_api refuses, as named tools: a branch that
|
||||
// exists (and, for a default, that a rule guards), a stated reason, the tools' account, the value before and after (issue 1422);
|
||||
// - `gitea_merge_pull_request` merges only into a branch that refuses its push, only a head whose mesh/check passed
|
||||
// against the trunk's head, approved by the operator where the branch requires no status (ADR 0534 §4), and only
|
||||
// that head;
|
||||
|
||||
+67
-66
@@ -53,8 +53,9 @@ import { announce, keepWatermark, readWatermark } from "./announce.js";
|
||||
import { mergeTool } from "./merge.js";
|
||||
import { GiteaAdmin, GiteaClient, movedSince } from "./client.js";
|
||||
import { keepMainRules, mainRulesStep } from "./mainrules.js";
|
||||
import { CHECK_CONTEXT, CHECK_STATUS, REPO_CHECK_CONTEXT, commentFor, composedGate, groupGateStatus, headsToAnnounce, maintained,
|
||||
requiredOf, statusesFor, writesGate, type Announced, type Checked, repoCheckFacts, reposToLook } from "./pulls.js";
|
||||
import { CHECK_CONTEXT, CHECK_STATUS, REPO_CHECK_CONTEXT, commentFor, composedGate, groupGateStatus, headKey, maintained,
|
||||
requiredOf, statusesFor, writesGate, type Announced, type Checked, repoCheckFacts } from "./pulls.js";
|
||||
import { actOn, announceNewHeads, startPass } from "./watch.js";
|
||||
import { GateWriter } from "./gate.js";
|
||||
import { HOOK_PATH, HOOK_PORT, Pace, claimHead, ensureSystemHook, eventsTool, keptSecret, receiver, type ForgeEvent } from "./webhook.js";
|
||||
import { keepSiblings, noteComposed, noteVerdict, readSiblings, siblingsFile, siblingsOf, weighSiblings } from "./siblings.js";
|
||||
@@ -138,12 +139,7 @@ async function pollMerged(client: GiteaClient): Promise<void> {
|
||||
// Closed unmerged (novox/hq ADR 0239): announced once, since the watching began, so its delivery stops.
|
||||
const closedKey = `closed:${repo.full_name}#${pull.number}`;
|
||||
if (!pull.merged && pull.state === "closed" && !announced.has(closedKey)) {
|
||||
if (primedMerges && !!pull.updated_at && !!since && pull.updated_at > since) {
|
||||
await announce("pull.closed", { owner: repo.owner, repo: repo.name, number: pull.number, title: pull.title,
|
||||
head: pull.head, head_sha: pull.head_sha, base: pull.base, html_url: pull.html_url });
|
||||
console.log(`[gitea] announced ${repo.full_name}#${pull.number} closed unmerged`);
|
||||
await siblingMoves(client, repo.full_name, pull.number);
|
||||
}
|
||||
if (primedMerges && !!pull.updated_at && !!since && pull.updated_at > since) await announceClosed(client, repo, pull);
|
||||
announced.add(closedKey);
|
||||
continue;
|
||||
}
|
||||
@@ -168,6 +164,23 @@ async function pollMerged(client: GiteaClient): Promise<void> {
|
||||
type RepoOf = { owner: string; name: string; full_name: string; clone_url?: string };
|
||||
type PullOf = Awaited<ReturnType<GiteaClient["listPullRequests"]>>[number];
|
||||
|
||||
/** A pull request closed without a merge, announced once (ADR 0239): by its event, or by the start's pass. */
|
||||
async function announceClosed(client: GiteaClient, repo: RepoOf, pull: PullOf): Promise<boolean> {
|
||||
const closedKey = `closed:${repo.full_name}#${pull.number}`;
|
||||
if (pull.merged || pull.state !== "closed" || announced.has(closedKey)) return false;
|
||||
announced.add(closedKey);
|
||||
try {
|
||||
await announce("pull.closed", { owner: repo.owner, repo: repo.name, number: pull.number, title: pull.title,
|
||||
head: pull.head, head_sha: pull.head_sha, base: pull.base, html_url: pull.html_url });
|
||||
} catch (err) {
|
||||
announced.delete(closedKey);
|
||||
throw err;
|
||||
}
|
||||
console.log(`[gitea] announced ${repo.full_name}#${pull.number} closed unmerged`);
|
||||
await siblingMoves(client, repo.full_name, pull.number);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** **A merge, announced once, whichever path saw it first** (novox/hq issue 729): the poll or the forge's event. */
|
||||
async function announceMerge(client: GiteaClient, repo: RepoOf, pull: PullOf): Promise<boolean> {
|
||||
if (!pull.merged || !pull.merge_commit_sha || announced.has(pull.merge_commit_sha)) return false;
|
||||
@@ -243,7 +256,7 @@ let heads: Announced = {};
|
||||
let primedPulls = false;
|
||||
/** **A head, announced once, whichever path saw it first** (novox/hq issue 729): the poll or the forge's event. */
|
||||
async function announceHead(client: GiteaClient, repo: RepoOf, pull: OpenOf): Promise<boolean> {
|
||||
const key = `${repo.full_name}#${pull.number}`;
|
||||
const key = headKey(repo.full_name, pull.number, pull.base);
|
||||
const sha = String(pull.head_sha ?? "");
|
||||
const before = heads[key];
|
||||
// Claimed before the first wait, so the other path finds it taken while this one is still announcing.
|
||||
@@ -281,43 +294,18 @@ async function alreadyChecked(client: GiteaClient, repo: RepoOf, pull: OpenOf):
|
||||
}
|
||||
|
||||
let lastPullLook = "";
|
||||
// **A sweep now and then looks at every repository** (novox/hq issue 347), whatever moved: the floor is a
|
||||
// heuristic, and a heuristic that misses is caught by the next sweep rather than never.
|
||||
const SWEEP_MS = 10 * 60_000;
|
||||
let lastSweep = 0;
|
||||
async function pollPulls(client: GiteaClient): Promise<void> {
|
||||
/** The pass over every open pull request: run at the start and when the receiver listens, never on a timer (issue 1422). */
|
||||
async function passPulls(client: GiteaClient): Promise<void> {
|
||||
const began = new Date().toISOString();
|
||||
const floor = lastPullLook ? new Date(Date.parse(lastPullLook) - MARGIN_MS).toISOString() : "";
|
||||
const dayAgo = new Date(Date.now() - 24 * 3600_000).toISOString();
|
||||
// A pull request opened, reopened or retargeted moves its own update time and not its repository's: the
|
||||
// pull requests updated since the floor are found by their own (issue 347). A search that fails is a sweep.
|
||||
let sweep = Date.now() - lastSweep >= SWEEP_MS;
|
||||
let updated: { owner: string; name: string; updated_at?: string }[] = [];
|
||||
if (floor && !sweep) {
|
||||
try {
|
||||
updated = await client.searchOpenPulls(floor);
|
||||
} catch (err) {
|
||||
console.error(`[gitea] the pull requests updated lately could not be searched, so every repository is looked at — ${err instanceof Error ? err.message : err}`);
|
||||
sweep = true;
|
||||
}
|
||||
}
|
||||
if (sweep) lastSweep = Date.now();
|
||||
for (const repo of reposToLook(await client.listAllRepos(), floor, updated, sweep)) {
|
||||
// Every open pull request updated since the floor, paged (novox/hq issue 579): the newest 20 alone left a head
|
||||
// beyond them unannounced. A sweep and the first pass read every page.
|
||||
const open = await client.listOpenPullsSince(repo.owner, repo.name, sweep ? "" : floor);
|
||||
for (const pull of headsToAnnounce(repo.full_name, open, heads)) {
|
||||
const key = `${repo.full_name}#${pull.number}`;
|
||||
const fresh = primedPulls || (!!pull.updated_at && pull.updated_at > dayAgo);
|
||||
// The first look after a start remembers nothing: a head the forge already shows a merge check of was announced.
|
||||
if (fresh && !primedPulls && await alreadyChecked(client, repo, pull)) {
|
||||
heads[key] = String(pull.head_sha);
|
||||
continue;
|
||||
}
|
||||
if (fresh) {
|
||||
if (await announceHead(client, repo, pull)) pace.miss(`${key} at ${String(pull.head_sha).slice(0, 8)}`);
|
||||
} else heads[key] = String(pull.head_sha);
|
||||
}
|
||||
for (const repo of await client.listAllRepos()) {
|
||||
// Every open pull request, paged (novox/hq issue 579).
|
||||
const open = await client.listOpenPullsSince(repo.owner, repo.name, "");
|
||||
await announceNewHeads(repo.full_name, open, heads, primedPulls, dayAgo, {
|
||||
alreadyChecked: (pull) => alreadyChecked(client, repo, pull),
|
||||
announceHead: (pull) => announceHead(client, repo, pull),
|
||||
missed: (what) => pace.miss(what),
|
||||
});
|
||||
if (await trunksMoved(client, repo.owner, repo.name)) changedTrunks = true;
|
||||
}
|
||||
if (changedTrunks) keepTrunks();
|
||||
@@ -497,6 +485,9 @@ const checks = new CheckStatuses({
|
||||
/** How fast the polls look: 30 s until the forge's events are heard, five minutes while they are; and what
|
||||
* gitea_events says of them (webhook.ts). */
|
||||
const pace = new Pace();
|
||||
/** Asks the pass over every repository again, shortly and once (set where the pass is made): an event that could not be
|
||||
* acted on is not lost until the next start. */
|
||||
let catchUp: () => void = () => {};
|
||||
if (!gitea) pace.unheard("the forge is not watched: no way to a token", false);
|
||||
|
||||
// The states this module keeps and which others may read, said on the bus before it serves (novox/hq ADR 0495 clause 2).
|
||||
@@ -522,22 +513,15 @@ registerModuleTools("gitea", () => [eventsTool(pace), ...readAllTools(readAll),
|
||||
gates,
|
||||
}), mergeTool(gitea, { gates })] : [])]);
|
||||
|
||||
/** What the forge's event asks of the announcer at once (issue 729): a new head is announced, and a merge. */
|
||||
async function actOn(client: GiteaClient, event: ForgeEvent): Promise<void> {
|
||||
const b = event.body as { owner?: string; repo?: string; number?: number };
|
||||
if (!b.owner || !b.repo || !b.number) return;
|
||||
if (event.name === "forge.pull-request.closed") {
|
||||
// Closed unmerged is a sibling's move; a merge is weighed with its announcement below (ADR 0382 §5).
|
||||
if (!(event.body as { merged?: boolean }).merged) await siblingMoves(client, `${b.owner}/${b.repo}`, b.number);
|
||||
return;
|
||||
}
|
||||
const head = ["forge.pull-request.opened", "forge.pull-request.synchronized", "forge.pull-request.reopened"].includes(event.name);
|
||||
const merged = event.name === "forge.pull-request.merged";
|
||||
if (!head && !merged) return;
|
||||
const [repo, pull] = await Promise.all([client.getRepo(b.owner, b.repo), client.getPullRequest(b.owner, b.repo, b.number)]);
|
||||
if (head && pull.state === "open") await announceHead(client, repo, pull);
|
||||
if (merged) await announceMerge(client, repo, pull);
|
||||
}
|
||||
/** What the forge's event asks of the announcer at once (issue 729, 1422): see watch.ts. */
|
||||
const actOnEvent = (client: GiteaClient, event: ForgeEvent): Promise<void> => actOn<RepoOf, PullOf>(event, {
|
||||
getRepo: (o, r) => client.getRepo(o, r),
|
||||
getPull: (o, r, n) => client.getPullRequest(o, r, n) as Promise<PullOf & { state?: string; merged?: boolean }>,
|
||||
announceHead: (repo, pull) => announceHead(client, repo, pull as OpenOf),
|
||||
announceMerge: (repo, pull) => announceMerge(client, repo, pull),
|
||||
announceClosed: (repo, pull) => announceClosed(client, repo, pull),
|
||||
siblingMoves: (full, n) => siblingMoves(client, full, n),
|
||||
});
|
||||
|
||||
/**
|
||||
* **The receiver, and the forge's one system webhook pointing at it** (issue 729). The secret is minted and kept beside
|
||||
@@ -563,7 +547,12 @@ async function listenForForge(client: GiteaClient): Promise<boolean> {
|
||||
// A repository made, or one not yet seen readable, is granted read before anything acts on it (readall.ts).
|
||||
await Promise.resolve().then(() => readAll().onEvent(event)).catch((err) =>
|
||||
console.error(`[gitea] whether the mesh's accounts read the repository of ${event.name} could not be settled — ${err instanceof Error ? err.message : err}`));
|
||||
await actOn(client, event);
|
||||
try {
|
||||
await actOnEvent(client, event);
|
||||
} catch (err) {
|
||||
catchUp();
|
||||
throw err;
|
||||
}
|
||||
} });
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server.once("error", reject);
|
||||
@@ -612,9 +601,9 @@ if (gitea) {
|
||||
// polls slow to every five minutes and announce only what no event said (Pace); otherwise they keep 30 s.
|
||||
// **Not awaited** (novox/hq issue 1726): the receiver's set-up asks the forge's admin API, and the runtime kills a bundle
|
||||
// whose import has not ended in its 10 s handshake and never starts it again; a forge slow to answer while every bundle
|
||||
// of the node starts left gitea_commit_status, gitea_merge_pull_request and gitea_events served nowhere. Until it is
|
||||
// listening, Pace keeps the polls at 30 s, so nothing is missed meanwhile; listenForForge says its own failure.
|
||||
void listenForForge(client);
|
||||
// of the node starts left gitea_commit_status, gitea_merge_pull_request and gitea_events served nowhere. The pass below
|
||||
// runs again once it is listening, so nothing said before is missed; listenForForge says its own failure.
|
||||
const listenOk = listenForForge(client);
|
||||
// The start's one pass over every repository (readall.ts), not awaited for the same reason. A forge not answering yet
|
||||
// is asked again, a minute apart, until one pass has run; after that only the forge's events grant.
|
||||
// An account of the list the forge does not hold yet (the forge-account provider makes nox-builder and nox-agent when
|
||||
@@ -646,8 +635,20 @@ if (gitea) {
|
||||
// Only while the setting keep-main-rules is true (mainrules.ts); false, the default, leaves every repository's rule as set.
|
||||
tick(mainRulesStep(keepMainRules(), () => GiteaAdmin.fromEnv().ensureMainMerges("novox"), (line) => console.log(line)), 600_000);
|
||||
tick(() => pollRepos(client), 60_000);
|
||||
tick(() => pollMerged(client), () => pace.every());
|
||||
tick(() => pollPulls(client), () => pace.every());
|
||||
// **No timer asks the forge for pull requests or merges** (issue 1422): the forge's events announce them. One pass over
|
||||
// every repository runs at the start and again once the receiver listens, so what happened while the module was away or
|
||||
// not yet listening is announced; passes run one at a time, a failed one is asked again a minute later.
|
||||
let passes: Promise<void> = Promise.resolve();
|
||||
const pass = (): Promise<void> => (passes = passes.catch(() => {}).then(() =>
|
||||
startPass(async () => { await pollMerged(client); await passPulls(client); }, { retryMs: 60_000 })));
|
||||
void pass();
|
||||
let catchUpAsked = false;
|
||||
catchUp = () => {
|
||||
if (catchUpAsked) return;
|
||||
catchUpAsked = true;
|
||||
setTimeout(() => { catchUpAsked = false; void pass(); }, 30_000);
|
||||
};
|
||||
void listenOk.then((ok) => (ok ? pass() : undefined));
|
||||
// Bound again until they hold, and not awaited (bind.ts, issue 1793): the runtime kills a bundle that has not answered its
|
||||
// handshake in 10 s, so the import must end while a refused bind is still asked again.
|
||||
bindInBackground([
|
||||
|
||||
@@ -569,6 +569,7 @@
|
||||
"mesh-build-node.keys",
|
||||
"mesh-build-image.keys",
|
||||
"mesh-delivery-checker.owed",
|
||||
"mesh-delivery-indexer.modules"
|
||||
"mesh-delivery-indexer.modules",
|
||||
"mesh-delivery-gate.source"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
// Every field of a branch protection rule, set through one tool (gitea_branch_protection_set_full; the operator,
|
||||
// 2026-10-28: "just make a generic tool so you can setup any configuration"). Pure, so it is tested without a forge:
|
||||
// which fields the forge's rule takes and of what kind, the body a call sends, and whether the rule the forge then
|
||||
// applies to the branch is one the controller registers a module from (judgedRule, mesh-control
|
||||
// cmd/mesh-controller/build_source.go).
|
||||
|
||||
import { TOOLS_ACCOUNT } from "./tools-account.js";
|
||||
import { MAIN_BRANCH, type BranchProtection } from "./protection.js";
|
||||
import { ruleCovers } from "./guard.js";
|
||||
|
||||
type Kind = "boolean" | "logins" | "strings" | "number" | "string";
|
||||
/** unread: the forge never answers it, so it is not read back; duration: a Go duration, compared as one ("8h" is the
|
||||
* forge's "8h0m0s"). */
|
||||
type Field = { kind: Kind; says: string; parent?: string; unread?: boolean; duration?: boolean };
|
||||
|
||||
/** Every field gitea's branch protection API takes on an edit at 1.27.3 (EditBranchProtectionOption, 32 fields), by its
|
||||
* own name, and its kind; rule_name is the branch argument, never a field. A field with a parent is read by the forge
|
||||
* only when the same call sets that parent true, and setting the parent false clears it (gitea's
|
||||
* routers/api/v1/repo/branch.go): such a field is refused without its parent true in the call. */
|
||||
export const PROTECTION_FIELDS: Record<string, Field> = {
|
||||
enable_push: { kind: "boolean", says: "whether anyone who writes may push to the branch directly (false: no direct push; also clears enable_push_whitelist and push_whitelist_deploy_keys)" },
|
||||
enable_push_whitelist: { kind: "boolean", says: "only the push whitelist may push; needs enable_push: true in the same call", parent: "enable_push" },
|
||||
push_whitelist_usernames: { kind: "logins", says: "the logins that may push directly" },
|
||||
push_whitelist_teams: { kind: "strings", says: "the teams that may push directly" },
|
||||
push_whitelist_deploy_keys: { kind: "boolean", says: "whether deploy keys may push; needs enable_push: true in the same call", parent: "enable_push" },
|
||||
enable_force_push: { kind: "boolean", says: "whether a force push is taken (false also clears the force-push allowlist flags)" },
|
||||
enable_force_push_allowlist: { kind: "boolean", says: "only the force-push allowlist may force-push; needs enable_force_push: true in the same call", parent: "enable_force_push" },
|
||||
force_push_allowlist_usernames: { kind: "logins", says: "the logins that may force-push" },
|
||||
force_push_allowlist_teams: { kind: "strings", says: "the teams that may force-push" },
|
||||
force_push_allowlist_deploy_keys: { kind: "boolean", says: "whether deploy keys may force-push; needs enable_force_push: true in the same call", parent: "enable_force_push" },
|
||||
enable_merge_whitelist: { kind: "boolean", says: "whether only the merge whitelist may merge" },
|
||||
merge_whitelist_usernames: { kind: "logins", says: "the logins that may merge (the controller's trunk rule wants mesh-tools, no agent-<node>)" },
|
||||
merge_whitelist_teams: { kind: "strings", says: "the teams that may merge" },
|
||||
enable_status_check: { kind: "boolean", says: "whether the statuses in status_check_contexts must pass to merge" },
|
||||
status_check_contexts: { kind: "strings", says: "the required commit statuses, e.g. mesh/check" },
|
||||
required_approvals: { kind: "number", says: "how many approving reviews a merge needs" },
|
||||
enable_approvals_whitelist: { kind: "boolean", says: "whether only the approvals whitelist's reviews count" },
|
||||
approvals_whitelist_username: { kind: "logins", says: "the logins whose approvals count" },
|
||||
approvals_whitelist_teams: { kind: "strings", says: "the teams whose approvals count" },
|
||||
block_on_rejected_reviews: { kind: "boolean", says: "a review asking changes blocks the merge" },
|
||||
block_on_official_review_requests: { kind: "boolean", says: "an open official review request blocks the merge" },
|
||||
block_on_outdated_branch: { kind: "boolean", says: "a head behind the branch blocks the merge" },
|
||||
dismiss_stale_approvals: { kind: "boolean", says: "a new commit dismisses earlier approvals" },
|
||||
ignore_stale_approvals: { kind: "boolean", says: "approvals of an earlier commit do not count" },
|
||||
require_signed_commits: { kind: "boolean", says: "every commit must be signed" },
|
||||
protected_file_patterns: { kind: "string", says: "files no push may change, ';'-separated globs" },
|
||||
unprotected_file_patterns: { kind: "string", says: "files a push may change even without push rights, ';'-separated globs" },
|
||||
block_admin_merge_override: { kind: "boolean", says: "true: an administrator cannot merge past the rule (the controller's trunk rule wants true)" },
|
||||
enable_bypass_allowlist: { kind: "boolean", says: "whether the bypass allowlist may merge past the rule's checks" },
|
||||
bypass_allowlist_usernames: { kind: "logins", says: "the logins that may merge past the rule's checks" },
|
||||
bypass_allowlist_teams: { kind: "strings", says: "the teams that may merge past the rule's checks" },
|
||||
priority: { kind: "number", says: "the rule's order among glob rules" },
|
||||
};
|
||||
|
||||
/** The repository settings gitea_repo_settings_set_full sets (gitea's EditRepoOption at 1.27.3), by their own names. Left
|
||||
* out: name (a rename moves every module's source address), the internal tracker, external tracker and external wiki
|
||||
* objects, and mirror_username, mirror_password and mirror_token, which are credentials. The forge ignores a
|
||||
* default_branch that does not exist; the read-back below says so. */
|
||||
export const REPO_FIELDS: Record<string, Field> = Object.fromEntries(([
|
||||
["description", "string"], ["website", "string"], ["private", "boolean"], ["template", "boolean"],
|
||||
["default_branch", "string"], ["has_code", "boolean"], ["has_issues", "boolean"], ["has_wiki", "boolean"],
|
||||
["has_pull_requests", "boolean"], ["has_projects", "boolean"], ["projects_mode", "string"], ["has_releases", "boolean"],
|
||||
["has_packages", "boolean"], ["has_actions", "boolean"], ["ignore_whitespace_conflicts", "boolean"],
|
||||
["allow_merge_commits", "boolean"], ["allow_rebase", "boolean"], ["allow_rebase_explicit", "boolean"],
|
||||
["allow_squash_merge", "boolean"], ["allow_fast_forward_only_merge", "boolean"], ["allow_manual_merge", "boolean"],
|
||||
["autodetect_manual_merge", "boolean"], ["allow_rebase_update", "boolean"], ["allow_merge_update", "boolean"],
|
||||
["default_update_style", "string"], ["default_delete_branch_after_merge", "boolean"], ["default_merge_style", "string"],
|
||||
["default_allow_maintainer_edit", "boolean"], ["archived", "boolean"], ["mirror_interval", "string"], ["enable_prune", "boolean"],
|
||||
] as [string, Kind][]).map(([name, kind]) => [name, {
|
||||
kind, says: `the repository's ${name.replace(/_/g, " ")}`,
|
||||
...(name === "projects_mode" ? { parent: "has_projects", says: "the repository's projects mode; needs has_projects: true in the same call" } : {}),
|
||||
...(name === "enable_prune" ? { unread: true, says: "a mirror's prune setting; the forge never answers it, so it is not read back" } : {}),
|
||||
...(name === "mirror_interval" ? { duration: true, says: "a mirror's interval, a Go duration such as 8h (the forge answers 8h0m0s)" } : {}),
|
||||
}]));
|
||||
|
||||
/** A Go duration ("8h", "1h30m", "8h0m0s", "90s") in milliseconds; NaN when it is none. */
|
||||
export function durationMs(v: unknown): number {
|
||||
if (typeof v !== "string" || !/^(\d+(\.\d+)?(ns|us|µs|ms|s|m|h))+$/.test(v)) return v === "0" ? 0 : NaN;
|
||||
const unit: Record<string, number> = { ns: 1e-6, us: 1e-3, "µs": 1e-3, ms: 1, s: 1e3, m: 6e4, h: 3.6e6 };
|
||||
let ms = 0;
|
||||
for (const m of v.matchAll(/(\d+(?:\.\d+)?)(ns|us|µs|ms|s|m|h)/g)) ms += Number(m[1]) * unit[m[2]];
|
||||
return ms;
|
||||
}
|
||||
|
||||
/** Fields the forge was sent but does not hold after the write, each with what was sent and what it holds: a write it
|
||||
* answered 200 and did not apply (gitea drops a field whose parent is not set, or a branch that does not exist). */
|
||||
export function unapplied(sent: Record<string, unknown>, held: Record<string, unknown>, fields: Record<string, Field> = PROTECTION_FIELDS): string[] {
|
||||
const norm = (k: string, v: unknown) => fields[k]?.duration ? String(durationMs(v))
|
||||
: Array.isArray(v) ? JSON.stringify([...v].map(String).sort()) : JSON.stringify(v ?? null);
|
||||
return Object.keys(sent).filter((k) => !fields[k]?.unread && norm(k, sent[k]) !== norm(k, held[k]))
|
||||
.map((k) => `${k} (sent ${JSON.stringify(sent[k])}, the forge holds ${JSON.stringify(held[k] ?? null)})`);
|
||||
}
|
||||
|
||||
/** Fields that changed between two reads but were not sent: what the forge changed beside the call (enable_push: false
|
||||
* clears the push whitelist flags). */
|
||||
export function alsoChanged(before: Record<string, unknown> | null, after: Record<string, unknown>, sent: Record<string, unknown>): string[] {
|
||||
if (!before) return [];
|
||||
const keys = new Set([...Object.keys(before), ...Object.keys(after)]);
|
||||
return [...keys].filter((k) => !(k in sent) && !["updated_at", "created_at"].includes(k) &&
|
||||
JSON.stringify(before[k] ?? null) !== JSON.stringify(after[k] ?? null));
|
||||
}
|
||||
|
||||
/** A forge login. */
|
||||
const LOGIN = /^[A-Za-z0-9_][A-Za-z0-9_.-]{0,39}$/;
|
||||
|
||||
/** The forge body for the fields given: each checked against its kind, a field not given left out (the forge keeps it on
|
||||
* an edit), an unknown field refused before anything is sent. Throws on the first wrong field. */
|
||||
export function fullProtectionBody(args: Record<string, unknown>, given: string[],
|
||||
fields: Record<string, Field> = PROTECTION_FIELDS, what = "a branch protection rule"): Record<string, unknown> {
|
||||
const body: Record<string, unknown> = {};
|
||||
for (const name of given) {
|
||||
const field = fields[name];
|
||||
if (!field) throw new Error(`${name} is no field of ${what}; the fields are ${Object.keys(fields).join(", ")}`);
|
||||
// The parent false clears the field, so only a field false goes beside a parent false.
|
||||
if (field.parent && args[field.parent] !== true && !(args[field.parent] === false && args[name] === false)) {
|
||||
throw new Error(`${name} is read by the forge only when the same call sets ${field.parent}: true (gitea ignores it otherwise). Nothing was sent`);
|
||||
}
|
||||
const v = args[name];
|
||||
switch (field.kind) {
|
||||
case "boolean":
|
||||
if (typeof v !== "boolean") throw new Error(`${name} is true or false`);
|
||||
body[name] = v;
|
||||
break;
|
||||
case "number":
|
||||
if (typeof v !== "number" || !Number.isInteger(v) || v < 0) throw new Error(`${name} is a whole number, 0 or more`);
|
||||
body[name] = v;
|
||||
break;
|
||||
case "string":
|
||||
if (typeof v !== "string") throw new Error(`${name} is a string`);
|
||||
if (field.duration && Number.isNaN(durationMs(v))) throw new Error(`${name} is a duration, such as 8h or 1h30m`);
|
||||
body[name] = v;
|
||||
break;
|
||||
case "strings":
|
||||
case "logins": {
|
||||
if (!Array.isArray(v) || v.some((s) => typeof s !== "string" || s.trim() === "")) throw new Error(`${name} is a list of names`);
|
||||
const names = [...new Set((v as string[]).map((s) => s.trim()))];
|
||||
if (field.kind === "logins") {
|
||||
const bad = names.find((s) => !LOGIN.test(s) || s.includes(".."));
|
||||
if (bad !== undefined) throw new Error(`${name}: ${JSON.stringify(bad)} is no forge login`);
|
||||
}
|
||||
body[name] = names;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
/** The fields of a call that are protection fields: every argument but owner, repo, branch and why. */
|
||||
export function givenFields(args: Record<string, unknown>): string[] {
|
||||
return Object.keys(args).filter((k) => !["owner", "repo", "branch", "why"].includes(k) && args[k] !== undefined);
|
||||
}
|
||||
|
||||
/** A rule as the controller reads it from gitea_branch_protection_get (protectionRule in build_source.go). */
|
||||
interface ControllerRule { rule: string; push: boolean; admin_may_override: boolean; merge_whitelist: string[] | null }
|
||||
|
||||
function asControllerRule(p: BranchProtection): ControllerRule {
|
||||
return {
|
||||
rule: String(p.rule_name ?? p.branch_name ?? ""),
|
||||
push: p.enable_push ?? false,
|
||||
admin_may_override: !(p.block_admin_merge_override ?? false),
|
||||
merge_whitelist: p.enable_merge_whitelist ? (p.merge_whitelist_usernames ?? []) : null,
|
||||
};
|
||||
}
|
||||
|
||||
/** ruleCovers of build_source.go, as the controller reads a rule's glob: `**` any, `*` within a segment, `?` one. */
|
||||
function controllerCovers(rule: string, branch: string): boolean {
|
||||
if (rule === branch) return true;
|
||||
if (!/[*?[]/.test(rule)) return false;
|
||||
let re = "^";
|
||||
for (let i = 0; i < rule.length; i++) {
|
||||
const c = rule[i];
|
||||
if (c === "*" && rule[i + 1] === "*") { re += ".*"; i++; }
|
||||
else if (c === "*") re += "[^/]*";
|
||||
else if (c === "?") re += "[^/]";
|
||||
else re += c.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
||||
}
|
||||
return new RegExp(re + "$").test(branch);
|
||||
}
|
||||
|
||||
/** Whether the controller registers a module from this branch, judged as judgedRule in mesh-control
|
||||
* cmd/mesh-controller/build_source.go does (ADR 0534 §6): the rule named for the branch, else the first glob rule that
|
||||
* covers it; it takes no push, its merge whitelist names mesh-tools and no agent-<node>, and no administrator merges
|
||||
* past it. A mirror, not the judge: the controller's own verdict is the one that counts. */
|
||||
export function trunkRule(rules: BranchProtection[], branch: string, trunks: string[] = [MAIN_BRANCH]): { holds: boolean; why: string; is_trunk: boolean } {
|
||||
const judged = judgedTrunkRule(rules, branch);
|
||||
return { ...judged, is_trunk: trunks.includes(branch) };
|
||||
}
|
||||
|
||||
function judgedTrunkRule(rules: BranchProtection[], branch: string): { holds: boolean; why: string } {
|
||||
const all = rules.map(asControllerRule);
|
||||
const r = all.find((x) => x.rule === branch) ?? all.find((x) => controllerCovers(x.rule, branch));
|
||||
const no = (why: string) => ({ holds: false, why });
|
||||
if (!r) return no(`no protection rule covers ${branch}`);
|
||||
if (r.push) return no(`the rule ${r.rule} lets a person push to ${branch} directly`);
|
||||
if (!r.merge_whitelist || r.merge_whitelist.length === 0) {
|
||||
return no(`the rule ${r.rule} keeps no merge whitelist naming ${TOOLS_ACCOUNT} alone for ${branch}`);
|
||||
}
|
||||
if (!r.merge_whitelist.includes(TOOLS_ACCOUNT)) {
|
||||
return no(`the rule ${r.rule} lets ${r.merge_whitelist.join(", ")} merge into ${branch}, and not ${TOOLS_ACCOUNT}`);
|
||||
}
|
||||
if (r.merge_whitelist.some((l) => l.startsWith("agent-"))) {
|
||||
return no(`the rule ${r.rule} lets an agent's account merge into ${branch}: ${r.merge_whitelist.join(", ")}`);
|
||||
}
|
||||
if (r.admin_may_override) return no(`the rule ${r.rule} lets an administrator merge into ${branch} past it`);
|
||||
return { holds: true, why: `the rule ${r.rule} takes no push, only ${r.merge_whitelist.join(", ")} merge, and no administrator merges past it` };
|
||||
}
|
||||
|
||||
/** The gate's source as it is read: one key per trunk, each with the heads of the repositories installed from it. */
|
||||
export type SourceRead = () => Promise<Record<string, unknown>>;
|
||||
|
||||
/** The state the trunks are read from: mesh-delivery-gate's source, whose key `main` is today's one trunk of each
|
||||
* repository it names (each repository's trunk head the checker judges on); a later trunk is a key of its own. */
|
||||
export const TRUNK_SOURCE = "mesh-delivery-gate.source";
|
||||
|
||||
/** A repository's trunks, the branches the mesh installs from (the operator, 2026-10-28 16:5x): each key of the gate's
|
||||
* source whose heads name the repository. "main" only when the source names none of its trunks or cannot be read, and
|
||||
* `from` says which. The one place trunks are read: several trunks later change only this. */
|
||||
export async function trunksOf(owner: string, repo: string, read: SourceRead): Promise<{ names: string[]; from: string }> {
|
||||
const full = `${owner}/${repo}`;
|
||||
let src: Record<string, unknown>;
|
||||
try {
|
||||
src = await read();
|
||||
} catch (err) {
|
||||
return { names: [MAIN_BRANCH], from: `${TRUNK_SOURCE} could not be read (${err instanceof Error ? err.message : String(err)}): ${MAIN_BRANCH} assumed` };
|
||||
}
|
||||
const names = Object.keys(src).filter((k) => {
|
||||
const heads = (src[k] as { heads?: Record<string, unknown> } | null)?.heads;
|
||||
return !!heads && typeof heads === "object" && full in heads;
|
||||
}).sort();
|
||||
if (names.length) return { names, from: TRUNK_SOURCE };
|
||||
return { names: [MAIN_BRANCH], from: `${TRUNK_SOURCE} names no trunk of ${full}: ${MAIN_BRANCH} assumed` };
|
||||
}
|
||||
|
||||
/** The gate's source read from the bus, every key and its value (the tools bundle's state channel). */
|
||||
export function sourceFromBus(get: (name: string) => { keys(): Promise<string[]>; get(k: string): Promise<{ value: unknown } | null> }): SourceRead {
|
||||
return async () => {
|
||||
const st = get(TRUNK_SOURCE);
|
||||
const out: Record<string, unknown> = {};
|
||||
for (const k of await st.keys()) out[k] = (await st.get(k))?.value ?? null;
|
||||
return out;
|
||||
};
|
||||
}
|
||||
|
||||
/** Why a call is the operator's, or null: it lets a force-push onto a rule covering a trunk (the operator, 2026-10-28
|
||||
* 16:4x: a call waits for their Approve only if it removes branch protection or allows a force-push to a trunk). No
|
||||
* tool of this module can ask for that Approve yet, so such a call is refused with nothing sent. */
|
||||
export function forcePushToTrunk(rule: string, body: Record<string, unknown>, trunks: string[]): string | null {
|
||||
const covered = trunks.filter((t) => ruleCovers(rule, t));
|
||||
if (!covered.length) return null;
|
||||
const opens = body.enable_force_push === true || body.force_push_allowlist_deploy_keys === true ||
|
||||
(["force_push_allowlist_usernames", "force_push_allowlist_teams"] as const).some((k) => Array.isArray(body[k]) && (body[k] as unknown[]).length > 0);
|
||||
if (!opens) return null;
|
||||
return `the rule ${rule} covers ${covered.join(", ")}, and this call lets a force-push onto it: a force-push to a trunk is ` +
|
||||
`the operator's Approve (the operator, 2026-10-28), and no tool of this module can ask it yet, so it is the operator's ` +
|
||||
`at the forge. Nothing was sent`;
|
||||
}
|
||||
@@ -141,13 +141,19 @@ export function planText(plan: ChangePlan): string {
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
/** The heads already announced, keyed `owner/repo#number`, so a restart announces nothing twice. */
|
||||
/** The heads already announced, keyed by `headKey`, so a restart announces nothing twice. */
|
||||
export type Announced = Record<string, string>;
|
||||
|
||||
/** The key a pull request's announced head is kept under: `owner/repo#number@base`. The base is in it because a pull
|
||||
* request retargeted keeps its head, and a head judged into one base is not judged into another (issue 1422, de-spiegel #6). */
|
||||
export function headKey(full: string, number: number, base?: string): string {
|
||||
return `${full}#${number}@${base ?? ""}`;
|
||||
}
|
||||
|
||||
/** Which open pull requests have a head not yet announced. A pull request merged or closed is not
|
||||
* open and is never asked about. */
|
||||
export function headsToAnnounce(full: string, pulls: GiteaPull[], announced: Announced): GiteaPull[] {
|
||||
return pulls.filter((p) => p.state === "open" && !!p.head_sha && announced[`${full}#${p.number}`] !== p.head_sha);
|
||||
return pulls.filter((p) => p.state === "open" && !!p.head_sha && announced[headKey(full, p.number, p.base)] !== p.head_sha);
|
||||
}
|
||||
|
||||
/** A pull request the forge found updated, by its repository: what `searchOpenPulls` answers. */
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
// Who is in a forge team and what the team may do (issue 1563; novox/hq ADR 0466 §3, issue 971).
|
||||
//
|
||||
// The build keys sit on one forge user, nox-builder, in the team `builders`, which reads every repository and pushes
|
||||
// nothing. The tools' account cannot read a team (the forge answers /orgs/novox/teams 403: it lacks read:organization),
|
||||
// so the read goes with the site administrator's login, and only the read, as the branch protection read does (ADR 0375):
|
||||
// GET /orgs/{org}/teams, GET /teams/{id}/members and, for a user's effective right on a repository, the repository as
|
||||
// that user sees it (GiteaAdmin.repositoryPermission, which counts access through a team). Nothing is written.
|
||||
|
||||
/** A team as the forge's Team object answers it (GET /orgs/{org}/teams): the fields read here. */
|
||||
export interface ForgeTeam {
|
||||
id: number;
|
||||
name: string;
|
||||
description?: string;
|
||||
/** The team's overall permission: none, read, write, admin, owner. */
|
||||
permission?: string;
|
||||
includes_all_repositories?: boolean;
|
||||
can_create_org_repo?: boolean;
|
||||
/** Per unit, the right the team has: units_map {"repo.code": "read", ...}. */
|
||||
units_map?: Record<string, string>;
|
||||
units?: string[];
|
||||
}
|
||||
|
||||
/** What the module asks of the forge for this, with the site administrator's login (GiteaAdmin, client.ts). */
|
||||
export interface TeamForge {
|
||||
teams(org: string): Promise<ForgeTeam[]>;
|
||||
teamMembers(teamId: number): Promise<string[]>;
|
||||
accountExists(user: string): Promise<boolean>;
|
||||
everyRepository(): Promise<{ full_name: string; owner: string; name: string }[]>;
|
||||
repositoryPermission(owner: string, repo: string, user: string): Promise<string>;
|
||||
}
|
||||
|
||||
export interface TeamRights {
|
||||
team: string;
|
||||
id: number;
|
||||
description: string;
|
||||
members: string[];
|
||||
/** The team's overall permission. */
|
||||
permission: string;
|
||||
includes_all_repositories: boolean;
|
||||
can_create_org_repo: boolean;
|
||||
/** Per unit, read / write / admin / none. */
|
||||
units: Record<string, string>;
|
||||
/** True when no unit of the team is above read: the team pushes nothing, writes no issue, no pull request. */
|
||||
read_only: boolean;
|
||||
}
|
||||
|
||||
const RANK: Record<string, number> = { none: 0, read: 1, write: 2, admin: 3, owner: 4 };
|
||||
|
||||
/** One team as a person reads it. A team the forge gives no units_map is read from its `permission`, which is what the
|
||||
* older forge versions answer. */
|
||||
export function teamRights(t: ForgeTeam, members: string[]): TeamRights {
|
||||
const units = t.units_map && Object.keys(t.units_map).length ? { ...t.units_map }
|
||||
: Object.fromEntries((t.units ?? []).map((u) => [u, t.permission ?? "none"]));
|
||||
const highest = Math.max(0, ...Object.values(units).map((r) => RANK[r] ?? 0));
|
||||
return {
|
||||
team: t.name, id: t.id, description: t.description ?? "", members: [...members].sort(), permission: t.permission ?? "none",
|
||||
includes_all_repositories: t.includes_all_repositories === true, can_create_org_repo: t.can_create_org_repo === true,
|
||||
units, read_only: highest <= RANK.read && (RANK[t.permission ?? "none"] ?? 0) <= RANK.read,
|
||||
};
|
||||
}
|
||||
|
||||
export interface UserRights {
|
||||
user: string;
|
||||
/** The teams of the organisation the user is a member of. */
|
||||
teams: string[];
|
||||
/** How many repositories the forge gives the user each right on, effective (through a team or as a collaborator). */
|
||||
repositories: Record<string, number>;
|
||||
/** The repositories where the user may do more than read: empty for a read-only user. */
|
||||
writes: { repository: string; permission: string }[];
|
||||
/** Every repository and the right on it; only when asked. */
|
||||
every?: { repository: string; permission: string }[];
|
||||
}
|
||||
|
||||
/** The teams of an organisation (or the one named), each with its members and rights; and for one user, the teams it is
|
||||
* in and its effective right on every repository (or on those named), as the forge says it. Only reads. */
|
||||
export async function readTeamRights(forge: TeamForge, org: string, opts: { team?: string; user?: string; repositories?: string[]; list?: boolean } = {}):
|
||||
Promise<{ organisation: string; teams: TeamRights[]; user?: UserRights }> {
|
||||
if (opts.user && !(await forge.accountExists(opts.user))) throw new Error(`no forge user ${opts.user}`);
|
||||
const all = await forge.teams(org);
|
||||
const named = opts.team ? all.filter((t) => t.name === opts.team) : all;
|
||||
if (opts.team && !named.length) throw new Error(`no team "${opts.team}" in ${org}: it holds ${all.map((t) => t.name).join(", ") || "none"}`);
|
||||
// Membership is read for every team when a user is asked, so the user's teams are all found.
|
||||
const wanted = opts.user ? all : named;
|
||||
const rights: TeamRights[] = [];
|
||||
for (const t of wanted) rights.push(teamRights(t, await forge.teamMembers(t.id)));
|
||||
const shown = opts.team ? rights.filter((r) => r.team === opts.team) : rights;
|
||||
const out: { organisation: string; teams: TeamRights[]; user?: UserRights } = { organisation: org, teams: shown };
|
||||
if (!opts.user) return out;
|
||||
const user = opts.user;
|
||||
const mine = rights.filter((r) => r.members.includes(user)).map((r) => r.team);
|
||||
const repos = (await forge.everyRepository()).filter((r) => !opts.repositories?.length || opts.repositories.includes(r.full_name));
|
||||
const counts: Record<string, number> = {};
|
||||
const writes: { repository: string; permission: string }[] = [];
|
||||
const every: { repository: string; permission: string }[] = [];
|
||||
for (const r of repos) {
|
||||
const permission = await forge.repositoryPermission(r.owner, r.name, user);
|
||||
counts[permission] = (counts[permission] ?? 0) + 1;
|
||||
if ((RANK[permission] ?? 0) > RANK.read) writes.push({ repository: r.full_name, permission });
|
||||
every.push({ repository: r.full_name, permission });
|
||||
}
|
||||
out.user = { user, teams: mine, repositories: counts, writes, ...(opts.list ? { every } : {}) };
|
||||
return out;
|
||||
}
|
||||
@@ -209,7 +209,7 @@ test("gitea_commit_trees takes paths as {directory: [paths]} and refuses another
|
||||
const { client, server, asked } = await serve([{ sha: COMMIT, url: "u", tree: LISTING, truncated: false, page: 1, total_count: LISTING.length }]);
|
||||
try {
|
||||
const tool = getGiteaTools(client).find((t) => t.name === "gitea_commit_trees")!;
|
||||
assert.ok("paths" in tool.input, "the tool names the argument");
|
||||
assert.ok("paths" in tool.input, "the tool names the arguments");
|
||||
const got: any = await tool.run({ owner: "novox", repo: "hq", sha: COMMIT, paths: { "modules/a": ["main.go"] } } as any);
|
||||
assert.deepEqual(got.paths, { "modules/a": { "main.go": h("6") } });
|
||||
const before = asked.length;
|
||||
|
||||
@@ -34,6 +34,12 @@ const state = {
|
||||
// refuses it the rules' list, as gitea's reqAdmin does, and it reads a branch's protection from the branch.
|
||||
admin: false,
|
||||
lastCommit: "2026-01-01T00:00:00Z",
|
||||
// The repository's default branch and pull request 7's base, as the forge keeps them (issue 1422 note 7).
|
||||
defaultBranch: "main",
|
||||
base: "main",
|
||||
prState: "open",
|
||||
// Branches the forge answers 404 for; "scratch" exists but no protection rule guards it.
|
||||
absent: ["nope"],
|
||||
// The pull request's head; the head the merge commit joined (the same, unless a test says otherwise) — the forge's
|
||||
// record of the pull request keeps saying `head` either way, as it would in the race issue 381 is about; and the
|
||||
// merge commit's own sha.
|
||||
@@ -66,7 +72,10 @@ beforeEach(() => {
|
||||
state.statuses = judgedAt(TRUNK);
|
||||
state.trunk = TRUNK;
|
||||
state.emitted = [];
|
||||
state.rules = [protectedMain()];
|
||||
state.rules = [protectedMain(), { rule_name: "release/*", enable_push: false }];
|
||||
state.defaultBranch = "main";
|
||||
state.base = "main";
|
||||
state.prState = "open";
|
||||
state.admin = false;
|
||||
state.head = REVIEWED;
|
||||
state.landed = "";
|
||||
@@ -98,12 +107,22 @@ const server = createServer(async (req, res) => {
|
||||
state.asked.push(`${method} ${path}${url.search}`);
|
||||
if (method !== "GET") state.sent.push({ method, path, body });
|
||||
const repo = "/repos/novox/mesh-catalog";
|
||||
if (method === "GET" && path === repo) return json(res, 200, { full_name: "novox/mesh-catalog", default_branch: "main" });
|
||||
if (method === "GET" && path === repo) return json(res, 200, { full_name: "novox/mesh-catalog", default_branch: state.defaultBranch });
|
||||
if (method === "PATCH" && path === repo) {
|
||||
if (body?.default_branch) state.defaultBranch = body.default_branch;
|
||||
return json(res, 200, { full_name: "novox/mesh-catalog", default_branch: state.defaultBranch });
|
||||
}
|
||||
if (method === "PATCH" && path === `${repo}/pulls/7`) {
|
||||
if (body?.base) state.base = body.base;
|
||||
return json(res, 201, { number: 7, title: "x", state: state.prState, merged: false, head: { ref: "feat/x", sha: state.head },
|
||||
base: { ref: state.base }, html_url: "http://fake/7" });
|
||||
}
|
||||
if (path.startsWith(`${repo}/branch_protections`) && !state.admin) {
|
||||
return json(res, 403, { message: "user should be an owner or a collaborator with admin write of a repository" });
|
||||
}
|
||||
if (method === "GET" && path === `${repo}/branch_protections`) return json(res, 200, state.rules);
|
||||
const branch = method === "GET" && path.startsWith(`${repo}/branches/`) ? decodeURIComponent(path.slice(`${repo}/branches/`.length)) : null;
|
||||
if (branch !== null && state.absent.includes(branch)) return json(res, 404, { message: "branch does not exist" });
|
||||
if (branch !== null) {
|
||||
// As the forge says it to any reader of the code: the rule that covers the branch, as the forge matches it.
|
||||
const rule = (state.rules as any[]).find((r) => ruleCovers(String(r.rule_name ?? r.branch_name), branch));
|
||||
@@ -125,8 +144,8 @@ const server = createServer(async (req, res) => {
|
||||
return json(res, 200, [{ commit: { committer: { date: state.lastCommit } } }]);
|
||||
}
|
||||
if (method === "GET" && path === `${repo}/pulls/7`) {
|
||||
return json(res, 200, { number: 7, title: "x", state: state.merged ? "closed" : "open", merged: state.merged,
|
||||
head: { ref: "feat/x", sha: state.head }, base: { ref: "main" }, html_url: "http://fake/7",
|
||||
return json(res, 200, { number: 7, title: "x", state: state.merged ? "closed" : state.prState, merged: state.merged,
|
||||
head: { ref: "feat/x", sha: state.head }, base: { ref: state.base }, html_url: "http://fake/7",
|
||||
merge_commit_sha: state.merged ? state.mergeCommit : null });
|
||||
}
|
||||
if (method === "GET" && path === `${repo}/git/commits/${state.mergeCommit}`) {
|
||||
@@ -640,6 +659,7 @@ test("no tool's caller text becomes another path or another user: `..` and `?sud
|
||||
await tool("gitea_delete_branch").run({ ...repo, branch: "feat/x" });
|
||||
await tool("gitea_get_file").run({ ...repo, path: "modules/gitea/module.json" }).catch(() => undefined);
|
||||
assert.deepEqual(state.asked, ["DELETE /repos/novox/mesh-catalog/branches/feat/x",
|
||||
"GET /repos/novox/mesh-catalog/contents/modules/gitea", // the file's size first (issue 1577)
|
||||
"GET /repos/novox/mesh-catalog/contents/modules/gitea/module.json"]);
|
||||
});
|
||||
|
||||
@@ -808,3 +828,47 @@ test("gitea_merge_pull_request judges a main requiring mesh/repo-check alone on
|
||||
state.statuses = [judgedAt(TRUNK)[1]];
|
||||
assert.equal(((await merge()) as Record<string, unknown>).merged, true);
|
||||
});
|
||||
|
||||
// Issue 1422 note 7: the two settings gitea_api refuses (ADR 0266) have named tools, held to the same guard as
|
||||
// gitea_branch_protection_set: the tools' account, a stated reason, a branch that exists and that a rule guards.
|
||||
test("gitea_default_branch_set answers the default before and after, and refuses a branch that is missing, unguarded or unexplained", async () => {
|
||||
const set = (args: Record<string, unknown>) =>
|
||||
tool("gitea_default_branch_set").run({ owner: "novox", repo: "mesh-catalog", why: "the trunk was recorded as unknown", ...args });
|
||||
await assert.rejects(set({ branch: "nope" }), /no branch nope/);
|
||||
await assert.rejects(set({ branch: "scratch" }), /no protection rule/);
|
||||
await assert.rejects(set({ branch: "release/1", why: " " }), /why/);
|
||||
await assert.rejects(set({ branch: ".." }), /dot segment|not a branch/);
|
||||
assert.deepEqual(writes(), []);
|
||||
assert.deepEqual(await set({ branch: "release/1" }), { branch: "release/1", before: "main", after: "release/1", changed: true,
|
||||
why: "the trunk was recorded as unknown" });
|
||||
assert.deepEqual(writes(), ["PATCH /repos/novox/mesh-catalog"]);
|
||||
assert.deepEqual(state.sent[0].body, { default_branch: "release/1" });
|
||||
assert.equal(state.defaultBranch, "release/1");
|
||||
state.sent = [];
|
||||
assert.equal(((await set({ branch: "release/1" })) as any).changed, false, "already the default: nothing is sent");
|
||||
assert.deepEqual(writes(), []);
|
||||
});
|
||||
|
||||
test("gitea_pull_base_set answers the base before and after, and refuses a base that is missing, a closed pull request or no reason", async () => {
|
||||
const set = (args: Record<string, unknown>) =>
|
||||
tool("gitea_pull_base_set").run({ owner: "novox", repo: "mesh-catalog", number: 7, why: "it was opened against the wrong trunk", ...args });
|
||||
await assert.rejects(set({ base: "nope" }), /no branch nope/);
|
||||
await assert.rejects(set({ base: "release/1", why: "" }), /why/);
|
||||
await assert.rejects(set({ base: "release/1", number: 0 }), /pull request's number/);
|
||||
state.prState = "closed";
|
||||
await assert.rejects(set({ base: "release/1" }), /not open/);
|
||||
state.prState = "open";
|
||||
assert.deepEqual(writes(), []);
|
||||
assert.deepEqual(await set({ base: "release/1" }), { number: 7, before: "main", after: "release/1", changed: true,
|
||||
why: "it was opened against the wrong trunk" });
|
||||
assert.deepEqual(writes(), ["PATCH /repos/novox/mesh-catalog/pulls/7"]);
|
||||
assert.deepEqual(state.sent[0].body, { base: "release/1" });
|
||||
assert.equal(state.base, "release/1");
|
||||
});
|
||||
|
||||
test("the two new tools are named for what mesh_search is asked, and gitea_api still refuses both settings", () => {
|
||||
assert.match(tool("gitea_default_branch_set").description, /default branch/i);
|
||||
assert.match(tool("gitea_pull_base_set").description, /pull request.s base/i);
|
||||
assert.ok("refused" in judgeApiCall("PATCH", "/repos/a/b", { default_branch: "x" }));
|
||||
assert.ok("refused" in judgeApiCall("PATCH", "/repos/a/b/pulls/7", { base: "x" }));
|
||||
});
|
||||
|
||||
@@ -45,7 +45,7 @@ test("a new head beyond the newest 20 open pull requests is announced, exactly o
|
||||
const { headsToAnnounce } = await import("../pulls.ts");
|
||||
// 25 open, every head announced before; #3 — the 23rd most recently updated — got a new head.
|
||||
const announced: Record<string, string> = {};
|
||||
for (let n = 1; n <= 25; n++) announced[`novox/hq#${n}`] = sha(n, "a");
|
||||
for (let n = 1; n <= 25; n++) announced[`novox/hq#${n}@main`] = sha(n, "a");
|
||||
const { client, server, asked } = await serve(25, { 3: sha(3, "b") });
|
||||
try {
|
||||
// The fault: the newest 20 alone do not hold it.
|
||||
@@ -58,7 +58,7 @@ test("a new head beyond the newest 20 open pull requests is announced, exactly o
|
||||
const open = await client.listOpenPullsSince("novox", "hq", "");
|
||||
for (const p of headsToAnnounce("novox/hq", open, announced)) {
|
||||
said.push(`#${p.number}@${p.head_sha}`);
|
||||
announced[`novox/hq#${p.number}`] = String(p.head_sha);
|
||||
announced[`novox/hq#${p.number}@${p.base}`] = String(p.head_sha);
|
||||
}
|
||||
}
|
||||
assert.deepEqual(said, [`#3@${sha(3, "b")}`], "the new head past the newest 20, announced once over two passes");
|
||||
@@ -90,3 +90,48 @@ test("a pass pages until it reaches back past its floor, and no further", async
|
||||
server.close();
|
||||
}
|
||||
});
|
||||
|
||||
// **One forge request never holds the others, and a file too big for an answer is not downloaded** (novox/hq issue 1577): a
|
||||
// request for a 4.9 MB file never ended and every read through the module timed out behind it.
|
||||
test("a request the forge never answers is cut at the limit and says which request", async () => {
|
||||
const { GiteaClient } = await import("../client.ts");
|
||||
const server = createServer((req) => { if (!req.url?.includes("fast")) return; });
|
||||
await new Promise<void>((r) => server.listen(0, r));
|
||||
const client = new GiteaClient(`http://127.0.0.1:${(server.address() as any).port}`, "t");
|
||||
client.requestTimeoutMs = 150;
|
||||
try {
|
||||
const started = Date.now();
|
||||
await assert.rejects(client.api("/repos/novox/hq/pulls"), /Gitea API GET \/repos\/novox\/hq\/pulls: no answer in 0\.15 s/);
|
||||
assert.ok(Date.now() - started < 3000);
|
||||
} finally {
|
||||
server.closeAllConnections();
|
||||
await new Promise<void>((r) => server.close(() => r()));
|
||||
}
|
||||
});
|
||||
|
||||
test("a file over 512 KiB is refused from its directory listing, and its body is never asked for", async () => {
|
||||
const { GiteaClient } = await import("../client.ts");
|
||||
const asked: string[] = [];
|
||||
const server = createServer((req, res) => {
|
||||
asked.push(req.url ?? "");
|
||||
res.setHeader("Content-Type", "application/json");
|
||||
if (req.url?.startsWith("/api/v1/repos/novox/hq/contents/bin?")) {
|
||||
return res.end(JSON.stringify([{ path: "bin/big", type: "file", size: 5_000_000 }, { path: "bin/small", type: "file", size: 5 }]));
|
||||
}
|
||||
if (req.url?.startsWith("/api/v1/repos/novox/hq/contents/bin/small")) {
|
||||
return res.end(JSON.stringify({ type: "file", encoding: "base64", sha: "s", size: 5, content: Buffer.from("hello").toString("base64") }));
|
||||
}
|
||||
res.statusCode = 404;
|
||||
res.end("{}");
|
||||
});
|
||||
await new Promise<void>((r) => server.listen(0, r));
|
||||
const client = new GiteaClient(`http://127.0.0.1:${(server.address() as any).port}`, "t");
|
||||
try {
|
||||
await assert.rejects(client.getFile("novox", "hq", "bin/big", "main"),
|
||||
/the file is 5000000 bytes, over the 512 KiB a tool answer can carry; read it from a clone/);
|
||||
assert.ok(!asked.some((u) => u.includes("/contents/bin/big")), asked.join("\n"));
|
||||
assert.equal((await client.getFile("novox", "hq", "bin/small", "main")).content, "hello");
|
||||
} finally {
|
||||
await new Promise<void>((r) => server.close(() => r()));
|
||||
}
|
||||
});
|
||||
|
||||
@@ -0,0 +1,250 @@
|
||||
// gitea_branch_protection_set_full (the operator, 2026-10-28: "just make a generic tool so you can setup any
|
||||
// configuration"): any field of a branch protection rule, only the fields given, as the forge's administrator; it
|
||||
// answers the rule before and after, and whether the controller's trunk rule (judgedRule, mesh-control
|
||||
// build_source.go) now holds. Against a fake forge that keeps the rules and answers them to the administrator alone.
|
||||
|
||||
import { test, after, beforeEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer } from "node:http";
|
||||
|
||||
import { GiteaAdmin, GiteaClient } from "../dist/client.js";
|
||||
import { ConfiguredToken } from "../dist/token.js";
|
||||
import { getGiteaTools } from "../dist/tools/index.js";
|
||||
import { trunkRule } from "../dist/protectfull.js";
|
||||
|
||||
const adminBasic = "Basic " + Buffer.from("mesh-admin:secret").toString("base64");
|
||||
let rules: any[] = [];
|
||||
let repoState: any = {};
|
||||
const branches = ["master", "main"];
|
||||
let sent: { method: string; path: string; body: any }[] = [];
|
||||
beforeEach(() => {
|
||||
// de-spiegel's main as found: no push, but the operator alone merges and an administrator merges past it.
|
||||
rules = [{ rule_name: "main", enable_push: false, enable_merge_whitelist: true, merge_whitelist_usernames: ["jschoubben"],
|
||||
enable_status_check: false, status_check_contexts: [], block_admin_merge_override: false, required_approvals: 0,
|
||||
enable_push_whitelist: false, push_whitelist_deploy_keys: false, enable_force_push: false, enable_force_push_allowlist: false,
|
||||
force_push_allowlist_deploy_keys: false, merge_whitelist_teams: [] }];
|
||||
source = {};
|
||||
repoState = { full_name: "novox/de-spiegel", default_branch: "master", description: "", has_wiki: true };
|
||||
sent = [];
|
||||
});
|
||||
|
||||
// What gitea 1.27.3's EditBranchProtection does with a body (routers/api/v1/repo/branch.go): the push whitelist flags are
|
||||
// read only beside enable_push true and cleared by enable_push false, the force-push allowlist flags likewise with
|
||||
// enable_force_push; teams are passed over on a repository no organisation owns (de-spiegel here, for the test).
|
||||
function giteaEdit(r: any, body: any) {
|
||||
const nested = ["enable_push", "enable_push_whitelist", "push_whitelist_deploy_keys", "enable_force_push",
|
||||
"enable_force_push_allowlist", "force_push_allowlist_deploy_keys", "merge_whitelist_teams", "push_whitelist_teams"];
|
||||
for (const [k, v] of Object.entries(body)) if (!nested.includes(k) && k !== "rule_name") r[k] = v;
|
||||
if (body.enable_push === false) Object.assign(r, { enable_push: false, enable_push_whitelist: false, push_whitelist_deploy_keys: false });
|
||||
if (body.enable_push === true) {
|
||||
r.enable_push = true;
|
||||
if (body.enable_push_whitelist !== undefined) r.enable_push_whitelist = body.enable_push_whitelist;
|
||||
if (body.push_whitelist_deploy_keys !== undefined) r.push_whitelist_deploy_keys = body.push_whitelist_deploy_keys;
|
||||
}
|
||||
if (body.enable_force_push === false) Object.assign(r, { enable_force_push: false, enable_force_push_allowlist: false, force_push_allowlist_deploy_keys: false });
|
||||
if (body.enable_force_push === true) {
|
||||
r.enable_force_push = true;
|
||||
if (body.enable_force_push_allowlist !== undefined) r.enable_force_push_allowlist = body.enable_force_push_allowlist;
|
||||
if (body.force_push_allowlist_deploy_keys !== undefined) r.force_push_allowlist_deploy_keys = body.force_push_allowlist_deploy_keys;
|
||||
}
|
||||
}
|
||||
|
||||
const server = createServer((req, res) => {
|
||||
const path = (req.url ?? "").replace(/^\/api\/v1/, "");
|
||||
const send = (status: number, body: unknown) => { res.writeHead(status, { "Content-Type": "application/json" }); res.end(JSON.stringify(body)); };
|
||||
if (req.headers.authorization !== adminBasic) return send(403, { message: "not an administrator of the repository" });
|
||||
let raw = "";
|
||||
req.on("data", (c) => (raw += c));
|
||||
req.on("end", () => {
|
||||
const body = raw ? JSON.parse(raw) : undefined;
|
||||
if (req.method !== "GET") sent.push({ method: req.method!, path, body });
|
||||
if (path === "/repos/novox/de-spiegel") {
|
||||
if (req.method === "GET") return send(200, repoState);
|
||||
if (req.method === "PATCH") {
|
||||
// gitea's updateBasicProperties: a default branch that does not exist is passed over, the call still 200.
|
||||
// enable_prune is a mirror's, kept beside the repository and never in its answer; mirror_interval is answered in Go's
|
||||
// duration form.
|
||||
const { default_branch, enable_prune, mirror_interval, ...rest } = body;
|
||||
Object.assign(repoState, rest);
|
||||
if (mirror_interval !== undefined) repoState.mirror_interval = mirror_interval === "8h" ? "8h0m0s" : mirror_interval;
|
||||
if (default_branch !== undefined && branches.includes(default_branch)) repoState.default_branch = default_branch;
|
||||
return send(200, repoState);
|
||||
}
|
||||
}
|
||||
const m = path.match(/^\/repos\/novox\/de-spiegel\/branch_protections(?:\/(.+))?$/);
|
||||
if (!m) return send(404, { message: "not found" });
|
||||
if (req.method === "GET" && !m[1]) return send(200, rules);
|
||||
if (req.method === "PATCH" && m[1]) {
|
||||
const r = rules.find((x) => x.rule_name === decodeURIComponent(m[1]));
|
||||
if (!r) return send(404, { message: "no rule" });
|
||||
giteaEdit(r, body);
|
||||
return send(200, r);
|
||||
}
|
||||
if (req.method === "POST" && !m[1]) { const r = { rule_name: body.rule_name }; giteaEdit(r, body); rules.push(r); return send(201, r); }
|
||||
send(405, { message: "no" });
|
||||
});
|
||||
});
|
||||
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
|
||||
after(() => new Promise<void>((r) => server.close(() => r())));
|
||||
const { port } = server.address() as { port: number };
|
||||
const url = `http://127.0.0.1:${port}`;
|
||||
// The gate's source as the tools read it (mesh-delivery-gate.source): one key per trunk, each naming the repositories
|
||||
// installed from it. A test sets it; empty is a source that names no trunk of de-spiegel.
|
||||
let source: Record<string, { heads: Record<string, string> }> = {};
|
||||
const readSource = async () => source;
|
||||
const tools = getGiteaTools(new GiteaClient(url, new ConfiguredToken("tools-token")), () => new GiteaAdmin(url, "mesh-admin", "secret"), readSource);
|
||||
const setRepo = (args: Record<string, unknown>) =>
|
||||
tools.find((t) => t.name === "gitea_repo_settings_set_full")!.run({ owner: "novox", repo: "de-spiegel", why: "L2", ...args });
|
||||
const setFull = (args: Record<string, unknown>) =>
|
||||
tools.find((t) => t.name === "gitea_branch_protection_set_full")!.run({ owner: "novox", repo: "de-spiegel", branch: "main", why: "L2", ...args });
|
||||
|
||||
test("it sets the merge whitelist and the administrator's override, only the fields given, and says the trunk rule now holds", async () => {
|
||||
const out: any = await setFull({ merge_whitelist_usernames: ["mesh-tools", "jschoubben"], block_admin_merge_override: true });
|
||||
assert.deepEqual(sent, [{ method: "PATCH", path: "/repos/novox/de-spiegel/branch_protections/main",
|
||||
body: { merge_whitelist_usernames: ["mesh-tools", "jschoubben"], block_admin_merge_override: true } }]);
|
||||
assert.deepEqual(out.before.merge_whitelist, ["jschoubben"]);
|
||||
assert.equal(out.before.admin_may_override, true);
|
||||
assert.deepEqual(out.after.merge_whitelist, ["mesh-tools", "jschoubben"]);
|
||||
assert.equal(out.after.admin_may_override, false);
|
||||
assert.equal(out.why, "L2");
|
||||
assert.equal(out.trunk_rule.holds, true, out.trunk_rule.why);
|
||||
});
|
||||
|
||||
test("it loosens as told too, and says the trunk rule no longer holds and why", async () => {
|
||||
const out: any = await setFull({ enable_push: true, status_check_contexts: ["mesh/check"], enable_status_check: true, required_approvals: 1 });
|
||||
assert.equal(out.after.push, true);
|
||||
assert.deepEqual(out.after.required_statuses, ["mesh/check"]);
|
||||
assert.equal(out.after.required_approvals, 1);
|
||||
assert.equal(out.trunk_rule.holds, false);
|
||||
assert.match(out.trunk_rule.why, /push to main directly/);
|
||||
});
|
||||
|
||||
test("a branch with no rule gets one with the fields given", async () => {
|
||||
const out: any = await setFull({ branch: "release/*", enable_push: false, protected_file_patterns: "*.lock" });
|
||||
assert.equal(out.created, true);
|
||||
assert.deepEqual(sent[0], { method: "POST", path: "/repos/novox/de-spiegel/branch_protections",
|
||||
body: { rule_name: "release/*", enable_push: false, protected_file_patterns: "*.lock" } });
|
||||
});
|
||||
|
||||
test("an unknown field, a field of the wrong kind, no field, no reason or a login that is none: refused, nothing sent", async () => {
|
||||
await assert.rejects(setFull({ enable_pushes: true }), /no field of a branch protection rule/);
|
||||
await assert.rejects(setFull({ enable_push: "yes" }), /true or false/);
|
||||
await assert.rejects(setFull({ required_approvals: -1 }), /whole number/);
|
||||
await assert.rejects(setFull({ merge_whitelist_usernames: ["../x"] }), /no forge login/);
|
||||
await assert.rejects(setFull({}), /at least one field/);
|
||||
await assert.rejects(setFull({ why: "", enable_push: false }), /say why/);
|
||||
assert.deepEqual(sent, []);
|
||||
});
|
||||
|
||||
// judgedRule's cases (mesh-control cmd/mesh-controller/build_source.go), each in its own words.
|
||||
test("the trunk rule is judged as the controller judges it", () => {
|
||||
const ok = { rule_name: "main", enable_push: false, enable_merge_whitelist: true, merge_whitelist_usernames: ["mesh-tools"], block_admin_merge_override: true };
|
||||
const cases: [any[], string, boolean, RegExp][] = [
|
||||
[[], "main", false, /no protection rule covers main/],
|
||||
[[{ ...ok, enable_push: true }], "main", false, /push to main directly/],
|
||||
[[{ ...ok, enable_merge_whitelist: false }], "main", false, /no merge whitelist/],
|
||||
[[{ ...ok, merge_whitelist_usernames: ["jschoubben"] }], "main", false, /and not mesh-tools/],
|
||||
[[{ ...ok, merge_whitelist_usernames: ["mesh-tools", "agent-g14"] }], "main", false, /agent's account/],
|
||||
[[{ ...ok, block_admin_merge_override: false }], "main", false, /administrator/],
|
||||
[[ok], "main", true, /takes no push/],
|
||||
[[{ ...ok, rule_name: "ma*" }], "main", true, /ma\*/],
|
||||
[[{ ...ok, rule_name: "ma*", enable_push: true }, { ...ok }], "main", true, /rule main/],
|
||||
];
|
||||
for (const [rs, branch, holds, why] of cases) {
|
||||
const got = trunkRule(rs, branch);
|
||||
assert.equal(got.holds, holds, `${JSON.stringify(rs)}: ${got.why}`);
|
||||
assert.match(got.why, why);
|
||||
}
|
||||
});
|
||||
|
||||
// Review round 1 of feat/protect-as-trunk: gitea passes over a dependent field without its parent set true, and answers 200.
|
||||
test("a field the forge reads only beside its parent is refused without it; with it, it is applied", async () => {
|
||||
await assert.rejects(setFull({ enable_push_whitelist: true }), /only when the same call sets enable_push: true/);
|
||||
await assert.rejects(setFull({ force_push_allowlist_deploy_keys: true, enable_force_push: false }), /enable_force_push: true/);
|
||||
assert.deepEqual(sent, []);
|
||||
const out: any = await setFull({ enable_push: true, enable_push_whitelist: true, push_whitelist_usernames: ["jschoubben"] });
|
||||
assert.equal(out.after.fields.enable_push_whitelist, true);
|
||||
await setFull({ enable_push: false, enable_push_whitelist: false }); // a false beside a false parent is what the forge does
|
||||
});
|
||||
|
||||
test("a field the forge answered and did not apply fails the call, naming it; what it changed beside the call is said", async () => {
|
||||
await assert.rejects(setFull({ merge_whitelist_teams: ["owners"] }), /did not apply merge_whitelist_teams \(sent \["owners"\], the forge holds \[\]\)/);
|
||||
rules[0].enable_push = true; rules[0].enable_push_whitelist = true;
|
||||
const out: any = await setFull({ enable_push: false });
|
||||
assert.deepEqual(out.also_changed, ["enable_push_whitelist"]);
|
||||
});
|
||||
|
||||
test("gitea_repo_settings_set_full sets the default branch, and fails loudly on a branch the forge passed over", async () => {
|
||||
const out: any = await setRepo({ default_branch: "main" });
|
||||
assert.deepEqual(sent.at(-1), { method: "PATCH", path: "/repos/novox/de-spiegel", body: { default_branch: "main" } });
|
||||
assert.equal(out.before.default_branch, "master");
|
||||
assert.equal(out.after.default_branch, "main");
|
||||
await assert.rejects(setRepo({ default_branch: "trunk" }), /did not apply default_branch/);
|
||||
await assert.rejects(setRepo({ name: "x" }), /no field of a repository's settings/);
|
||||
await assert.rejects(setRepo({ why: "", has_wiki: false }), /say why/);
|
||||
});
|
||||
|
||||
test("every field gitea 1.27.3 edits on a rule is taken: 32, the bypass allowlist among them", async () => {
|
||||
const { PROTECTION_FIELDS } = await import("../dist/protectfull.js");
|
||||
assert.equal(Object.keys(PROTECTION_FIELDS).length, 32);
|
||||
for (const f of ["enable_bypass_allowlist", "bypass_allowlist_usernames", "bypass_allowlist_teams"]) assert.ok(f in PROTECTION_FIELDS, f);
|
||||
});
|
||||
|
||||
// Review round 2: the settings tool's read-back meets the forge's own answer.
|
||||
test("A: enable_prune, which the forge never answers, is said as not read back, never failed", async () => {
|
||||
const pruned: any = await setRepo({ enable_prune: true });
|
||||
assert.deepEqual(pruned.not_read_back, ["enable_prune"]);
|
||||
});
|
||||
|
||||
test("B: mirror_interval is compared as a duration: 8h and the forge's 8h0m0s are one", async () => {
|
||||
const mirror: any = await setRepo({ mirror_interval: "8h" });
|
||||
assert.equal(mirror.after.mirror_interval, "8h0m0s");
|
||||
await assert.rejects(setRepo({ mirror_interval: "often" }), /duration/);
|
||||
});
|
||||
|
||||
test("C: allow_merge_update, default_update_style and has_code are taken; projects_mode needs has_projects; what is left out is named", async () => {
|
||||
const more: any = await setRepo({ allow_merge_update: true, default_update_style: "rebase", has_code: true });
|
||||
assert.deepEqual(more.set, { allow_merge_update: true, default_update_style: "rebase", has_code: true });
|
||||
await assert.rejects(setRepo({ projects_mode: "repo" }), /only when the same call sets has_projects: true/);
|
||||
await setRepo({ has_projects: true, projects_mode: "repo" });
|
||||
const desc = tools.find((t) => t.name === "gitea_repo_settings_set_full")!.description;
|
||||
for (const w of ["name", "mirror_username", "mirror_password", "mirror_token", "tracker", "wiki"]) assert.match(desc, new RegExp(w));
|
||||
});
|
||||
|
||||
// The operator, 2026-10-28 16:4x: a call waits for their Approve only if it removes branch protection or allows a
|
||||
// force-push to main. No tool of this module can ask yet (no TypeScript asker; ADR 0286's mark is unbuilt in the
|
||||
// runner), so until one can, such a call is refused, with nothing sent, and says it is the operator's; everything else
|
||||
// runs at once.
|
||||
test("a force-push onto a rule covering main is refused, nothing sent; on another branch, and tightening, it runs", async () => {
|
||||
for (const args of [{ enable_force_push: true }, { enable_force_push: true, enable_force_push_allowlist: true },
|
||||
{ force_push_allowlist_usernames: ["jschoubben"] }, { force_push_allowlist_teams: ["owners"] },
|
||||
{ branch: "ma*", enable_force_push: true }, { branch: "**", force_push_allowlist_usernames: ["x"] }]) {
|
||||
await assert.rejects(setFull(args), /force-push to a trunk is the operator's/, JSON.stringify(args));
|
||||
}
|
||||
assert.deepEqual(sent, []);
|
||||
await setFull({ branch: "release/*", enable_force_push: true });
|
||||
const tight: any = await setFull({ enable_force_push: false, force_push_allowlist_usernames: [] });
|
||||
assert.equal(tight.trunk_rule.holds, false, "trunk_rule is in the answer");
|
||||
});
|
||||
|
||||
// The operator, 2026-10-28 16:5x: the gates and trunk_rule apply to every trunk, the branches the mesh installs from, as
|
||||
// the gate's source names them; "main" only when it names none, and said.
|
||||
test("a trunk the gate's source names, beta, is gated and judged as main is; main is then no trunk of the repository", async () => {
|
||||
source = { main: { heads: { "novox/okioki": "a1" } }, beta: { heads: { "novox/de-spiegel": "b2" } } };
|
||||
await assert.rejects(setFull({ branch: "beta", enable_force_push: true }), /force-push to a trunk is the operator's/);
|
||||
await assert.rejects(setFull({ branch: "b*", force_push_allowlist_usernames: ["x"] }), /covers beta/);
|
||||
assert.deepEqual(sent, []);
|
||||
const main: any = await setFull({ enable_force_push: true });
|
||||
assert.deepEqual(main.trunks, { names: ["beta"], from: "mesh-delivery-gate.source" });
|
||||
assert.equal(main.trunk_rule.is_trunk, false);
|
||||
const beta: any = await setFull({ branch: "beta", enable_push: false, enable_merge_whitelist: true, merge_whitelist_usernames: ["mesh-tools"], block_admin_merge_override: true });
|
||||
assert.equal(beta.trunk_rule.is_trunk, true);
|
||||
assert.equal(beta.trunk_rule.holds, true, beta.trunk_rule.why);
|
||||
});
|
||||
|
||||
test("a source that names no trunk of the repository falls back to main, and says so", async () => {
|
||||
const out: any = await setFull({ block_admin_merge_override: true });
|
||||
assert.equal(out.trunks.names[0], "main");
|
||||
assert.match(out.trunks.from, /names no trunk of novox\/de-spiegel: main assumed/);
|
||||
assert.equal(out.trunk_rule.is_trunk, true);
|
||||
});
|
||||
@@ -8,16 +8,27 @@ import { createServer } from "node:http";
|
||||
test("each open pull request's new head is announced once, and nothing closed or merged", async () => {
|
||||
const { headsToAnnounce } = await import("../pulls.ts");
|
||||
const pulls = [
|
||||
{ number: 1, title: "a", state: "open", merged: false, head_sha: "aaa", html_url: "" },
|
||||
{ number: 1, title: "a", state: "open", merged: false, head_sha: "aaa", base: "main", html_url: "" },
|
||||
{ number: 2, title: "b", state: "open", merged: false, head_sha: "bbb", html_url: "" },
|
||||
{ number: 3, title: "c", state: "closed", merged: true, head_sha: "ccc", html_url: "" },
|
||||
{ number: 4, title: "d", state: "open", merged: false, html_url: "" },
|
||||
];
|
||||
const announced = { "novox/mesh-catalog#1": "aaa", "novox/mesh-catalog#2": "old" };
|
||||
const announced = { "novox/mesh-catalog#1@main": "aaa", "novox/mesh-catalog#2@": "old" };
|
||||
assert.deepEqual(headsToAnnounce("novox/mesh-catalog", pulls, announced).map((p) => p.number), [2],
|
||||
"only a head not announced, of a pull request that is open");
|
||||
});
|
||||
|
||||
test("a pull request retargeted keeps its head and is announced again, into its new base (de-spiegel #6)", async () => {
|
||||
const { headsToAnnounce, headKey } = await import("../pulls.ts");
|
||||
const was = { number: 6, title: "a", state: "open", merged: false, head_sha: "aaa", base: "old-trunk", html_url: "" };
|
||||
const announced: Record<string, string> = { [headKey("novox/de-spiegel", 6, "old-trunk")]: "aaa" };
|
||||
assert.deepEqual(headsToAnnounce("novox/de-spiegel", [was], announced), [], "the same head into the same base is announced once");
|
||||
const retargeted = { ...was, base: "main" };
|
||||
assert.deepEqual(headsToAnnounce("novox/de-spiegel", [retargeted], announced).map((p) => p.number), [6],
|
||||
"the same head into a new base is judged afresh");
|
||||
assert.notEqual(headKey("novox/de-spiegel", 6, "main"), headKey("novox/de-spiegel", 6, "old-trunk"));
|
||||
});
|
||||
|
||||
test("a verdict is the head commit's status; an error is the forge's error, never a success", async () => {
|
||||
const { statusFor, commentFor, CHECK_CONTEXT } = await import("../pulls.ts");
|
||||
const base = { owner: "novox", repo: "mesh-catalog", number: 7, commit: "0123456789abcdef", id: "build-1", on: "laptop" };
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
// Issue 1563: no tool read a forge team's members and rights, so the build keys' read-only user (nox-builder, team
|
||||
// builders, ADR 0466 §3) could not be checked live. The tools' token answers /orgs/novox/teams 403; the read goes with
|
||||
// the administrator's login, GET only.
|
||||
//
|
||||
// The answers below keep the shape of the forge's own API (gitea's swagger, "Team": id, name, description,
|
||||
// organization, includes_all_repositories, permission, units, units_map, can_create_org_repo; "User": id, login, ...;
|
||||
// repos/search: {ok, data:[Repository]}; Repository.permissions {admin, push, pull}). The live answer is read once the
|
||||
// tool is delivered (issue 1563 AC2).
|
||||
|
||||
import { test, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer } from "node:http";
|
||||
|
||||
import { GiteaAdmin, GiteaClient } from "../dist/client.js";
|
||||
import { ConfiguredToken } from "../dist/token.js";
|
||||
import { getGiteaTools } from "../dist/tools/index.js";
|
||||
|
||||
const adminBasic = "Basic " + Buffer.from("mesh-admin:secret").toString("base64");
|
||||
const org = { id: 2, username: "novox", name: "novox" };
|
||||
const team = (id: number, name: string, over: Record<string, unknown>) => ({
|
||||
id, name, description: "", organization: org, includes_all_repositories: true, permission: "read", can_create_org_repo: false,
|
||||
units: ["repo.code"], units_map: { "repo.code": "read" }, ...over,
|
||||
});
|
||||
const teams = [
|
||||
team(5, "builders", { description: "the build modules' fetch" }),
|
||||
team(3, "mesh-tools", { permission: "write", units: ["repo.code", "repo.issues", "repo.pulls"],
|
||||
units_map: { "repo.code": "write", "repo.issues": "write", "repo.pulls": "write" } }),
|
||||
];
|
||||
const members: Record<string, string[]> = { "5": ["nox-builder"], "3": ["mesh-tools"] };
|
||||
const user = (login: string) => ({ id: 23, login, username: login, email: `${login}@noreply.localhost` });
|
||||
const repo = (owner: string, name: string) => ({ full_name: `${owner}/${name}`, name, owner: { login: owner }, private: true, archived: false });
|
||||
const repos = [repo("novox", "mesh-catalog"), repo("jschoubben", "car-hunter")];
|
||||
const seen: string[] = [];
|
||||
|
||||
const server = createServer((req, res) => {
|
||||
const url = new URL(req.url ?? "", "http://x");
|
||||
const path = url.pathname.replace(/^\/api\/v1/, "");
|
||||
const as = req.headers.authorization === adminBasic ? "admin" : "tools";
|
||||
seen.push(`${req.method} ${path} as ${as}`);
|
||||
const send = (status: number, body: unknown) => { res.writeHead(status, { "Content-Type": "application/json" }); res.end(JSON.stringify(body)); };
|
||||
if (as !== "admin" && path.startsWith("/orgs/")) return send(403, { message: "token does not have at least one of required scope(s): [read:organization]" });
|
||||
if (path === "/orgs/novox/teams") return send(200, teams);
|
||||
const m = path.match(/^\/teams\/(\d+)\/members$/);
|
||||
if (m) return send(200, (members[m[1]] ?? []).map(user));
|
||||
const u = path.match(/^\/users\/([^/]+)$/);
|
||||
if (u) return send(u[1] === "nobody" ? 404 : 200, u[1] === "nobody" ? { message: "not found" } : user(u[1]));
|
||||
if (path === "/repos/search") return send(200, { ok: true, data: repos });
|
||||
const r = path.match(/^\/repos\/([^/]+)\/([^/]+)$/);
|
||||
if (r) {
|
||||
const sudo = url.searchParams.get("sudo");
|
||||
const found = repos.find((x) => x.full_name === `${r[1]}/${r[2]}`);
|
||||
// nox-builder reads the organisation's repositories through its team and the other one not at all; mesh-tools writes.
|
||||
if (!found || (sudo === "nox-builder" && r[1] !== "novox")) return send(404, { message: "not found" });
|
||||
const push = sudo === "mesh-tools";
|
||||
return send(200, { ...found, permissions: { admin: false, push, pull: true } });
|
||||
}
|
||||
send(404, { message: "not found" });
|
||||
});
|
||||
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
|
||||
after(() => new Promise<void>((r) => server.close(() => r())));
|
||||
const { port } = server.address() as { port: number };
|
||||
const url = `http://127.0.0.1:${port}`;
|
||||
const tools = getGiteaTools(new GiteaClient(url, new ConfiguredToken("tools-token")), () => new GiteaAdmin(url, "mesh-admin", "secret"));
|
||||
const run = (args: Record<string, unknown>) => tools.find((t) => t.name === "gitea_team_rights")!.run(args) as Promise<any>;
|
||||
|
||||
test("gitea_team_rights answers every team of the organisation with its members and units, read as the administrator", async () => {
|
||||
const out = await run({});
|
||||
assert.equal(out.organisation, "novox");
|
||||
assert.deepEqual(out.teams.map((t: any) => t.team), ["builders", "mesh-tools"]);
|
||||
const builders = out.teams[0];
|
||||
assert.deepEqual(builders.members, ["nox-builder"]);
|
||||
assert.deepEqual(builders.units, { "repo.code": "read" });
|
||||
assert.equal(builders.includes_all_repositories, true);
|
||||
assert.equal(builders.read_only, true);
|
||||
assert.equal(out.teams[1].read_only, false);
|
||||
assert.ok(seen.every((s) => s.startsWith("GET ") && s.endsWith("as admin")), `only reads, as the administrator: ${seen.join("; ")}`);
|
||||
});
|
||||
|
||||
test("gitea_team_rights answers one team by name, and refuses an unknown one naming the teams there are", async () => {
|
||||
const out = await run({ team: "builders" });
|
||||
assert.deepEqual(out.teams.map((t: any) => t.team), ["builders"]);
|
||||
await assert.rejects(run({ team: "nobody" }), /no team "nobody" in novox: it holds builders, mesh-tools/);
|
||||
});
|
||||
|
||||
test("gitea_team_rights answers a user's teams and effective right on each repository: nox-builder writes nowhere", async () => {
|
||||
const out = await run({ user: "nox-builder" });
|
||||
assert.deepEqual(out.user.teams, ["builders"]);
|
||||
assert.deepEqual(out.user.repositories, { read: 1, none: 1 });
|
||||
assert.deepEqual(out.user.writes, []);
|
||||
const tools2 = await run({ user: "mesh-tools", list: true });
|
||||
assert.deepEqual(tools2.user.writes, [{ repository: "novox/mesh-catalog", permission: "write" }, { repository: "jschoubben/car-hunter", permission: "write" }]);
|
||||
assert.equal(tools2.user.every.length, 2);
|
||||
});
|
||||
|
||||
test("the tools' own token is refused the teams, which is why the administrator's login reads them", async () => {
|
||||
const res = await fetch(`${url}/api/v1/orgs/novox/teams`, { headers: { Authorization: "token tools-token" } });
|
||||
assert.equal(res.status, 403);
|
||||
});
|
||||
|
||||
test("a user the forge does not know is refused, not answered as writing nowhere", async () => {
|
||||
await assert.rejects(run({ user: "nobody" }), /no forge user nobody/);
|
||||
});
|
||||
@@ -538,7 +538,8 @@ test("the tools register once there is a way to a token, and the first call mint
|
||||
"gitea_get_file",
|
||||
"gitea_list_branches",
|
||||
"gitea_delete_branch",
|
||||
"gitea_branch_protection_get", "gitea_branch_protection_set",
|
||||
"gitea_branch_protection_get", "gitea_team_rights", "gitea_branch_protection_set", "gitea_branch_protection_set_full", "gitea_repo_settings_set_full",
|
||||
"gitea_default_branch_set", "gitea_pull_base_set",
|
||||
"gitea_note_append", "gitea_delivery_view", "gitea_commit_statuses", "gitea_open_pulls", "gitea_commit_trees", "gitea_contains",
|
||||
"gitea_list_labels", "gitea_create_label",
|
||||
"gitea_api",
|
||||
|
||||
@@ -28,6 +28,7 @@ interface Team { id: number; name: string; units: any; members: Set<string> }
|
||||
|
||||
interface ForgeRepo { owner: string; name: string; full: string; private: boolean; archived?: boolean; ownerVisibility: string; collaborators: Map<string, string> }
|
||||
const forgeRepos: ForgeRepo[] = [];
|
||||
const userRules = new Map<string, any>();
|
||||
const repoFaults = { refuse: new Set<string>(), permission: new Set<string>() };
|
||||
function addRepo(full: string, isPrivate: boolean, ownerVisibility = "public"): ForgeRepo {
|
||||
const [owner, name] = full.split("/");
|
||||
@@ -84,7 +85,7 @@ function fakeForge() {
|
||||
const server = createServer(async (req, res) => {
|
||||
const url = new URL(req.url ?? "/", "http://fake");
|
||||
const p = url.pathname.replace(/^\/api\/v1/, "");
|
||||
const me = who(req);
|
||||
const me = url.searchParams.has("sudo") && isAdmin(who(req)) ? url.searchParams.get("sudo") : who(req);
|
||||
calls.push(`${req.method} ${p} as ${me}`);
|
||||
if (me === null) return json(res, 401, { message: "unauthorized" });
|
||||
|
||||
@@ -168,6 +169,23 @@ function fakeForge() {
|
||||
if (m[2] === "feat/unguarded") return json(res, 200, { name: m[2], protected: false });
|
||||
return json(res, 404, { message: "branch does not exist" });
|
||||
}
|
||||
// A repository of a user: its rules are its administrator's (a site administrator, or a collaborator with admin).
|
||||
m = p.match(/^\/repos\/(?!novox\/)([^/]+)\/([^/]+)\/branch_protections(?:\/(.*))?$/);
|
||||
if (m) {
|
||||
const key = `${m[1]}/${m[2]}`;
|
||||
const repo = forgeRepos.find((r) => r.full === key);
|
||||
if (!repo || !["admin", "owner"].includes(permissionOf(me, repo))) return json(res, 403, { message: "user should be an owner or a collaborator with admin write of a repository" });
|
||||
if (req.method === "GET" && m[3] === undefined) return json(res, 200, userRules.has(key) ? [userRules.get(key)] : []);
|
||||
// As gitea does: a whitelist login that cannot write the repository is dropped, and the write still answered.
|
||||
const keep = (b: any) => (Array.isArray(b.merge_whitelist_usernames)
|
||||
? { ...b, merge_whitelist_usernames: b.merge_whitelist_usernames.filter((u: string) => ["write", "admin", "owner"].includes(permissionOf(u, repo))) } : b);
|
||||
if (req.method === "PATCH" && m[3] === "main") {
|
||||
if (!userRules.has(key)) return json(res, 404, { message: "branch protection does not exist" });
|
||||
Object.assign(userRules.get(key), keep(await read(req)));
|
||||
return json(res, 200, userRules.get(key));
|
||||
}
|
||||
if (req.method === "POST") { const b = keep(await read(req)); userRules.set(key, b); return json(res, 201, b); }
|
||||
}
|
||||
// gitea's reqAdmin on the whole group: reading the rules is administration too.
|
||||
m = p.match(/^\/repos\/novox\/([^/]+)\/branch_protections(?:\/(.*))?$/);
|
||||
if (m) {
|
||||
@@ -205,6 +223,7 @@ function fakeForge() {
|
||||
m = p.match(/^\/repos\/([^/]+)\/([^/]+)(?:\/collaborators\/([^/]+)(\/permission)?)?$/);
|
||||
if (m) {
|
||||
const repo = forgeRepos.find((r) => r.owner === decodeURIComponent(m![1]) && r.name === decodeURIComponent(m![2]));
|
||||
if (repo && req.method === "GET" && !m[3] && url.searchParams.has("sudo") && repoFaults.permission.has(repo.full)) return json(res, 500, { message: "the forge is not well" });
|
||||
if (!repo || permissionOf(me, repo) === "none") return json(res, 404, { message: "repository does not exist" });
|
||||
if (req.method === "GET" && !m[3]) {
|
||||
const mine = permissionOf(me, repo);
|
||||
@@ -214,7 +233,8 @@ function fakeForge() {
|
||||
if (req.method === "GET" && m[4]) {
|
||||
if (repoFaults.permission.has(repo.full)) return json(res, 500, { message: "the forge is not well" });
|
||||
if (!isAdmin(me) && me !== user) return json(res, 403, { message: "only the user or an administrator" });
|
||||
return json(res, 200, { permission: permissionOf(user, repo) });
|
||||
// As gitea answers: the collaborator's own permission, not what a team gives the account.
|
||||
return json(res, 200, { permission: repo.collaborators.get(user) ?? "read" });
|
||||
}
|
||||
if (req.method === "PUT" && !m[4]) {
|
||||
// gitea's reqAdmin: the repository's administration, which on a repository of another user the site administrator alone holds.
|
||||
@@ -608,7 +628,8 @@ test("the merging account is given write on each repository the indexer's module
|
||||
await writes.take({ key: "car-hunter-2", op: "put", value: { v: 1, repository: "jschoubben/car-hunter" } });
|
||||
assert.equal(forge.calls.length, from, "a repository already written asks nothing more");
|
||||
const repo = (full: string) => forgeRepos.find((x) => x.full === full)!;
|
||||
assert.equal(repo("jschoubben/car-hunter").collaborators.get(TOOLS_ACCOUNT), "write");
|
||||
assert.equal(repo("jschoubben/car-hunter").collaborators.get(TOOLS_ACCOUNT), "write", "a user's repository is written, not administered");
|
||||
assert.equal(userRules.has("jschoubben/car-hunter"), false, "a repository with no rule for main is given none");
|
||||
assert.equal(repo("novox/hq").collaborators.size, 0, "the organisation's repository is written through its team, no collaborator");
|
||||
assert.equal((await client.getRepo("jschoubben", "car-hunter")).writes, true);
|
||||
assert.deepEqual(said, [{ type: WRITABLE_EVENT, body: { owner: "jschoubben", repo: "car-hunter", account: TOOLS_ACCOUNT } }]);
|
||||
@@ -636,9 +657,37 @@ test("the merging account is given write on each repository the indexer's module
|
||||
assert.ok(!forge.calls.slice(puts).some((c) => c.startsWith("PUT ")), "no write is asked on a repository no module builds from");
|
||||
assert.equal(repo("jschoubben/dotfiles").collaborators.get(TOOLS_ACCOUNT), "read");
|
||||
repoFaults.refuse.clear();
|
||||
// The operator's rule is left as set (the operator, 2026-10-26: "Write, rule untouched"): only mesh-tools is added to its
|
||||
// merge whitelist; a status the mesh does not set and required approvals are said as blocking the mesh's merge, unchanged.
|
||||
const operators = { rule_name: "main", enable_push: true, enable_merge_whitelist: true, merge_whitelist_usernames: ["jschoubben"],
|
||||
enable_status_check: true, status_check_contexts: ["mesh/check", "mesh/merge-gate", "ci/external"], required_approvals: 2,
|
||||
block_admin_merge_override: false, require_signed_commits: true, block_on_outdated_branch: true, block_on_rejected_reviews: true };
|
||||
userRules.set("jschoubben/snake", structuredClone(operators));
|
||||
const after = await grantTool.run({ repositories: ["jschoubben/snake"] }) as any;
|
||||
assert.deepEqual(after.granted, ["jschoubben/snake"]);
|
||||
assert.deepEqual(((await unwritableTool.run({})) as any).unwritable, []);
|
||||
assert.equal(repo("jschoubben/snake").collaborators.get(TOOLS_ACCOUNT), "write");
|
||||
assert.deepEqual(userRules.get("jschoubben/snake"), { ...operators, merge_whitelist_usernames: ["jschoubben", TOOLS_ACCOUNT] }, "only the whitelist changed");
|
||||
assert.deepEqual(after.merge_whitelist, [{ repository: "jschoubben/snake", changed: true, listed: ["jschoubben", TOOLS_ACCOUNT] }]);
|
||||
const blockedWhy = "writable, but main's rule blocks the mesh's merge: it requires the status ci/external, which the mesh does not set; " +
|
||||
"it requires 2 approving reviews on the forge, which the gate does not give; it requires signed commits, which the mesh does not make; " +
|
||||
"it refuses a branch behind main, which the mesh does not bring up to date; " +
|
||||
"it refuses a merge after a review asked for changes, which the gate does not clear";
|
||||
assert.deepEqual(after.merge_blocked, [{ repository: "jschoubben/snake", why: blockedWhy }]);
|
||||
const again = await unwritableTool.run({}) as any;
|
||||
assert.deepEqual([again.unwritable, again.merge_blocked], [[], [{ repository: "jschoubben/snake", why: blockedWhy }]]);
|
||||
const writesBefore = forge.calls.filter((c) => /^PATCH .*branch_protections/.test(c)).length;
|
||||
assert.deepEqual(((await grantTool.run({ repositories: ["jschoubben/snake"] })) as any).merge_whitelist[0].changed, false);
|
||||
assert.equal(forge.calls.filter((c) => /^PATCH .*branch_protections/.test(c)).length, writesBefore, "a listed account is not written again");
|
||||
// The mesh's own statuses never block: a rule requiring only them, with mesh-tools listed, is not reported.
|
||||
userRules.set("jschoubben/snake", { rule_name: "main", enable_merge_whitelist: true, merge_whitelist_usernames: [TOOLS_ACCOUNT],
|
||||
enable_status_check: true, status_check_contexts: ["mesh/check", "mesh/merge-gate", "mesh/repo-check"] });
|
||||
assert.deepEqual(((await unwritableTool.run({})) as any).merge_blocked, []);
|
||||
assert.deepEqual(((await grantTool.run({ repositories: ["jschoubben/snake"] })) as any).merge_blocked, []);
|
||||
// The forge drops a whitelist login that cannot write and still answers 200: the whitelist is read back and that fails loud.
|
||||
userRules.set("jschoubben/dotfiles", { rule_name: "main", enable_merge_whitelist: true, merge_whitelist_usernames: ["jschoubben"] });
|
||||
await assert.rejects(GiteaAdmin.fromEnv(env).ensureMergerListed("jschoubben", "dotfiles", TOOLS_ACCOUNT),
|
||||
/mesh-tools was added to the merge whitelist of main's rule main of jschoubben\/dotfiles, but the forge holds \[jschoubben\]: it drops a login that cannot write/);
|
||||
userRules.delete("jschoubben/dotfiles");
|
||||
// A module gone from the indexer leaves its repository out of the delivered list.
|
||||
await writes.take({ key: "snake", op: "delete" });
|
||||
assert.deepEqual(writes.delivered(), ["jschoubben/car-hunter", "novox/hq"]);
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { mock, test } from "node:test";
|
||||
import type { GiteaPull } from "../client.ts";
|
||||
|
||||
// **No poll of the forge** (issue 1422, notes 10 and 11): the forge's events announce heads and merges; a pass runs once at
|
||||
// the start and when the receiver listens.
|
||||
|
||||
const edited = (base: string) => ({ name: "forge.pull-request.edited", body: { owner: "novox", repo: "hq", number: 7, base, merged: false } });
|
||||
|
||||
function deps() {
|
||||
const calls: string[] = [];
|
||||
return { calls, d: {
|
||||
getRepo: async () => ({ r: 1 }),
|
||||
getPull: async () => ({ state: "open", base: "trunk" }) as any,
|
||||
announceHead: async () => { calls.push("head"); },
|
||||
announceMerge: async () => { calls.push("merge"); },
|
||||
announceClosed: async () => { calls.push("closed"); },
|
||||
siblingMoves: async () => { calls.push("siblings"); },
|
||||
} };
|
||||
}
|
||||
|
||||
test("an edited event announces the head into the base it now has, without a poll", async () => {
|
||||
const { actOn } = await import("../watch.ts");
|
||||
const { calls, d } = deps();
|
||||
await actOn(edited("trunk"), d);
|
||||
assert.deepEqual(calls, ["head"]);
|
||||
});
|
||||
|
||||
test("a close without a merge is announced by its event, a merged close is left to the merge event", async () => {
|
||||
const { actOn } = await import("../watch.ts");
|
||||
const { calls, d } = deps();
|
||||
await actOn({ name: "forge.pull-request.closed", body: { owner: "novox", repo: "hq", number: 7, merged: false } }, d);
|
||||
assert.deepEqual(calls, ["closed", "siblings"]);
|
||||
calls.length = 0;
|
||||
await actOn({ name: "forge.pull-request.closed", body: { owner: "novox", repo: "hq", number: 7, merged: true } }, d);
|
||||
assert.deepEqual(calls, []);
|
||||
});
|
||||
|
||||
test("no timer polls after the start: one pass ends and nothing is scheduled; a failed one is asked again", async () => {
|
||||
const { startPass } = await import("../watch.ts");
|
||||
mock.timers.enable({ apis: ["setTimeout", "setInterval"] });
|
||||
try {
|
||||
let runs = 0;
|
||||
await startPass(async () => { runs++; }, { retryMs: 60_000, log: () => {} });
|
||||
mock.timers.tick(24 * 3600_000);
|
||||
assert.equal(runs, 1, "a day later the forge was not asked again");
|
||||
let tries = 0;
|
||||
const retried = startPass(async () => { if (++tries < 2) throw new Error("forge down"); }, { retryMs: 60_000, log: () => {} });
|
||||
await Promise.resolve(); await Promise.resolve();
|
||||
mock.timers.tick(60_000);
|
||||
await retried;
|
||||
assert.equal(tries, 2);
|
||||
} finally {
|
||||
mock.timers.reset();
|
||||
}
|
||||
});
|
||||
|
||||
test("a start pass announces a head nobody announced, and only remembers one the forge shows a check of", async () => {
|
||||
const { announceNewHeads } = await import("../watch.ts");
|
||||
const pull = (n: number, sha: string): GiteaPull => ({ number: n, state: "open", head_sha: sha, base: "main",
|
||||
updated_at: new Date().toISOString() } as unknown as GiteaPull);
|
||||
const announced: number[] = [], said: string[] = [];
|
||||
const heads: Record<string, string> = {};
|
||||
await announceNewHeads("novox/hq", [pull(1, "a".repeat(40)), pull(2, "b".repeat(40))], heads, false,
|
||||
new Date(Date.now() - 24 * 3600_000).toISOString(), {
|
||||
alreadyChecked: async (p) => p.number === 2,
|
||||
announceHead: async (p) => { announced.push(p.number); return true; },
|
||||
missed: (w) => said.push(w),
|
||||
});
|
||||
assert.deepEqual(announced, [1]);
|
||||
assert.equal(heads["novox/hq#2@main"], "b".repeat(40), "the checked head is remembered, not announced");
|
||||
assert.equal(said.length, 1);
|
||||
});
|
||||
@@ -251,7 +251,8 @@ test("a forge that never delivers is said loudly once, and unhealthy, when the p
|
||||
assert.equal(r.healthy, true);
|
||||
assert.equal(r.verified.count, 1);
|
||||
assert.equal(r.verified.last.event, "push");
|
||||
assert.equal(r.polls.every_seconds, 300);
|
||||
assert.equal(r.polls.every_seconds, null);
|
||||
assert.equal(r.polls.hearing, true);
|
||||
assert.match(said.join("\n"), /heard again/);
|
||||
});
|
||||
|
||||
@@ -314,24 +315,19 @@ test("without a receiver by configuration the polls alone watch, healthy; a rece
|
||||
assert.match(r.why, /403/);
|
||||
});
|
||||
|
||||
test("the polls look every 30 s until events are heard, every five minutes while they are, and 30 s again on a miss", async () => {
|
||||
test("nothing is paced: the report says no timer asks the forge, and a miss is still counted and said", async () => {
|
||||
const { Pace } = await import("../webhook.ts");
|
||||
let now = 1_000_000;
|
||||
const said: string[] = [];
|
||||
const pace = new Pace(() => now, (l) => said.push(l));
|
||||
assert.equal(pace.every(), 30_000, "no delivery yet");
|
||||
pace.heard();
|
||||
assert.equal(pace.every(), 300_000, "a delivery heard");
|
||||
now += 11 * 60_000;
|
||||
assert.equal(pace.every(), 30_000, "ten minutes without a delivery");
|
||||
assert.equal((pace as any).every, undefined, "no pace to ask");
|
||||
assert.equal((pace.report() as any).polls.every_seconds, null);
|
||||
pace.heard();
|
||||
assert.equal((pace.report() as any).polls.hearing, true);
|
||||
now += 1000;
|
||||
pace.miss("novox/hq#7 at aaaaaaaa");
|
||||
assert.equal(pace.every(), 30_000, "the poll found what no event said");
|
||||
assert.equal((pace.report() as any).polls.hearing, false, "the pass found what no event said");
|
||||
assert.match(said.join("\n"), /MISSED novox\/hq#7/);
|
||||
now += 1000;
|
||||
pace.heard();
|
||||
assert.equal(pace.every(), 300_000, "the next delivery");
|
||||
});
|
||||
|
||||
// Issue 1079: a pull request's edit event carries its description, so mesh-delivery reads a Reviewed-by line written after
|
||||
|
||||
@@ -11,6 +11,9 @@
|
||||
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
|
||||
import { emit } from "@novox/mesh-sdk/events";
|
||||
import { GiteaAdmin, GiteaClient, type BranchProtection } from "../client.js";
|
||||
import { readTeamRights } from "../teamrights.js";
|
||||
import { state } from "@novox/mesh-sdk/state";
|
||||
import { PROTECTION_FIELDS, REPO_FIELDS, alsoChanged, forcePushToTrunk, fullProtectionBody, sourceFromBus, trunksOf, type SourceRead, givenFields, trunkRule, unapplied } from "../protectfull.js";
|
||||
import { appendNote, run, viewBody, viewComment } from "../delivery.js";
|
||||
import { containedFrom, isCommit, mergeableUnder, newestByContext } from "../checks.js";
|
||||
import {
|
||||
@@ -76,13 +79,13 @@ function namedPaths(raw: unknown): Record<string, string[]> | undefined {
|
||||
return out;
|
||||
}
|
||||
|
||||
export function getGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefinition[] {
|
||||
return namesChecked(allGiteaTools(gitea, admin));
|
||||
export function getGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin, source: SourceRead = sourceFromBus(state)): ToolDefinition[] {
|
||||
return namesChecked(allGiteaTools(gitea, admin, source));
|
||||
}
|
||||
|
||||
// gitea_merge_pull_request is served from the watcher's bundle (merge.ts, index.ts), beside the merge check's one
|
||||
// writer (novox/hq issue 456, ADR 0323).
|
||||
function allGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefinition[] {
|
||||
function allGiteaTools(gitea: GiteaClient, admin: (() => GiteaAdmin) | undefined, source: SourceRead): ToolDefinition[] {
|
||||
return [
|
||||
// ---- Repositories ----
|
||||
{
|
||||
@@ -374,7 +377,7 @@ function allGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefini
|
||||
// ---- Contents and branches ----
|
||||
{
|
||||
name: "gitea_get_file",
|
||||
description: "One file's contents from a repository, decoded, at a branch, tag or commit (default the repository's default branch).",
|
||||
description: "One file's contents from a repository, decoded, at a branch, tag or commit (default the repository's default branch). Refuses a file over 512 KiB, which a tool answer cannot carry: read it from a clone.",
|
||||
input: {
|
||||
owner: { type: "string", description: "the repository owner" },
|
||||
repo: { type: "string", description: "the repository name" },
|
||||
@@ -419,14 +422,35 @@ function allGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefini
|
||||
},
|
||||
run: async (args) => {
|
||||
const owner = String(args.owner), repo = String(args.repo);
|
||||
// The one read made as the forge's administrator (novox/hq ADR 0375, issue 772): the rules' list is the
|
||||
// repository administrator's, which the tools' account is not. No other tool uses this account.
|
||||
// Read as the forge's administrator (novox/hq ADR 0375, issue 772): the rules' list is the repository
|
||||
// administrator's, which the tools' account is not. gitea_team_rights reads, and gitea_branch_protection_set_full
|
||||
// and gitea_repo_settings_set_full write, with the same account.
|
||||
const rules = await (admin ? admin() : gitea).branchProtections(owner, repo);
|
||||
if (!args.branch) return { rules: rules.map(summarised) };
|
||||
const rule = rules.find((p) => (p.rule_name ?? p.branch_name) === String(args.branch)) ?? null;
|
||||
return { branch: String(args.branch), rule: rule ? summarised(rule) : null };
|
||||
},
|
||||
},
|
||||
// ---- Forge teams (issue 1563) ----
|
||||
{
|
||||
name: "gitea_team_rights",
|
||||
description: "Read-only: the forge's teams of an organisation, who is in each (members) and what each may do — its permission, its units read, write or admin per unit (code, issues, pull requests), whether it covers all repositories, and whether it is read-only. Give a user to see which teams it is in and its effective right (read, write, admin) on every repository, with those it can write listed: the check that a build key's forge user (nox-builder, team builders) reads and pushes nothing. Forge team members and rights; GET only, no secret answered.",
|
||||
input: {
|
||||
org: { type: "string", description: "the organisation; novox when not given" },
|
||||
team: { type: "string", description: "one team by name; every team of the organisation when not given" },
|
||||
user: { type: "string", description: "a forge user: its teams and effective right on each repository" },
|
||||
repositories: { type: "array", items: { type: "string" }, description: "owner/name of the repositories to look at for the user; every repository when not given" },
|
||||
list: { type: "boolean", description: "with a user, list the right on every repository, not only those above read" },
|
||||
},
|
||||
run: async (args) => {
|
||||
const strings = Array.isArray(args.repositories) ? args.repositories.map(String) : undefined;
|
||||
// The second read made as the forge's administrator (ADR 0375's precedent, issue 1563): the tools' token lacks
|
||||
// read:organization, so a team is not theirs to read. GET only.
|
||||
return readTeamRights(admin ? admin() : (() => { throw new Error("the forge's administrator login is not set: a team is read with it"); })(),
|
||||
String(args.org || "novox"), { team: args.team ? String(args.team) : undefined, user: args.user ? String(args.user) : undefined,
|
||||
repositories: strings, list: args.list === true });
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "gitea_branch_protection_set",
|
||||
description: "Protect a branch. Never weaker (novox/hq ADR 0266): a change that lets a person push directly or lets an administrator merge past the rule is refused, and on main so is a change that drops the merge whitelist (only mesh-tools merges there) or widens it (novox/hq ADR 0398 §2, issue 948) — that is the operator's, at the forge. The forge requires no status to merge (ADR 0534): the rule written requires none, and one the rule required is taken off; the checks are judged by mesh-delivery and the merge tool. Edits the branch's rule (everything else kept unless given) or creates one, which refuses direct pushes unless push=true. Answers the rule before and after.",
|
||||
@@ -456,6 +480,154 @@ function allGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefini
|
||||
},
|
||||
},
|
||||
|
||||
{
|
||||
name: "gitea_branch_protection_set_full",
|
||||
description: "Set any field of a branch protection rule — protect a branch as a trunk, set its merge whitelist (users and teams), " +
|
||||
"push and push whitelist, force push and its allowlist, the bypass allowlist, required status checks, required approvals and " +
|
||||
"the approvals whitelist, block on rejected reviews, block on an outdated branch, dismiss stale approvals, signed commits, " +
|
||||
"protected and unprotected file patterns, and the administrator's merge override (block_admin_merge_override): the 32 " +
|
||||
"fields gitea 1.27.3's API edits, by their own names. Only the fields given change; a branch with no rule gets one. " +
|
||||
"enable_push_whitelist and push_whitelist_deploy_keys need enable_push: true in the same call, the force-push allowlist " +
|
||||
"flags enable_force_push: true (the forge ignores them otherwise). The rule is read back after the write: a field the " +
|
||||
"forge did not apply fails the call, naming it; also_changed names what the forge changed beside the call. Generic: it " +
|
||||
"tightens or loosens as told (the operator, 2026-10-28), so a reason (why) is required and logged; but a call that lets a force-push onto a rule covering a trunk is refused, nothing sent, until the module can ask the operator's Approve for it. A repository's trunks, the branches the mesh installs from, are read from mesh-delivery-gate's source at the call (main when it names none, said in trunks.from). As the forge's " +
|
||||
"administrator. Answers the rule before and after, the repository's trunks, and trunk_rule: whether the branch is a trunk (is_trunk) and whether the controller now registers a module from " +
|
||||
"this branch (judgedRule, mesh-control build_source.go: no push, a merge whitelist naming mesh-tools and no " +
|
||||
"agent-<node>, no administrator override) and why not.",
|
||||
input: {
|
||||
owner: { type: "string", description: "the repository owner" },
|
||||
repo: { type: "string", description: "the repository name" },
|
||||
branch: { type: "string", description: "the branch (the rule's name, or a glob) whose rule is set" },
|
||||
why: { type: "string", description: "why, in a sentence: logged and answered" },
|
||||
...Object.fromEntries(Object.entries(PROTECTION_FIELDS).map(([name, f]) => [name, f.kind === "logins" || f.kind === "strings"
|
||||
? { type: "array", items: { type: "string" }, description: f.says }
|
||||
: { type: f.kind === "number" ? "number" : f.kind, description: f.says }])),
|
||||
},
|
||||
run: async (args) => {
|
||||
const owner = String(args.owner), repo = String(args.repo), branch = String(args.branch ?? "").trim();
|
||||
const why = String(args.why ?? "").trim();
|
||||
if (!branch) throw new Error("name the branch whose rule is set");
|
||||
if (!why) throw new Error("say why (why): a branch's protection is changed for a reason, which is logged");
|
||||
pathSegments("branch", branch);
|
||||
const given = givenFields(args);
|
||||
if (!given.length) throw new Error(`name at least one field to set: ${Object.keys(PROTECTION_FIELDS).join(", ")}`);
|
||||
const body = fullProtectionBody(args, given);
|
||||
const trunks = await trunksOf(owner, repo, source);
|
||||
const operators = forcePushToTrunk(branch, body, trunks.names);
|
||||
if (operators) throw new Refused(operators);
|
||||
const forge = admin ? admin() : gitea;
|
||||
const rules = await forge.branchProtections(owner, repo);
|
||||
const before = rules.find((p) => (p.rule_name ?? p.branch_name) === branch) ?? null;
|
||||
console.log(`[gitea] gitea_branch_protection_set_full ${owner}/${repo} ${branch}: ${JSON.stringify(body)} — why: ${why}`);
|
||||
const rule = await forge.writeBranchProtection(owner, repo, branch, body, !before);
|
||||
const after = await forge.branchProtections(owner, repo);
|
||||
const read = after.find((p) => (p.rule_name ?? p.branch_name) === branch) ?? rule;
|
||||
const missed = unapplied(body, read);
|
||||
if (missed.length) {
|
||||
throw new Error(`the forge answered the write of ${owner}/${repo} ${branch}'s rule but did not apply ${missed.join("; ")}. ` +
|
||||
`The rule it holds now: ${JSON.stringify(summarised(read))}`);
|
||||
}
|
||||
return {
|
||||
branch, created: !before, why, set: body, also_changed: alsoChanged(before, read, body),
|
||||
before: before ? { ...summarised(before), fields: before } : null,
|
||||
after: { ...summarised(read), fields: read },
|
||||
trunks, trunk_rule: trunkRule(after, branch, trunks.names),
|
||||
};
|
||||
},
|
||||
},
|
||||
|
||||
{
|
||||
name: "gitea_repo_settings_set_full",
|
||||
description: "Set a repository's settings as the forge's administrator — its default branch (e.g. master to main), " +
|
||||
"description, website, private, template, archived, the issues, wiki, pull requests, projects, releases, packages and " +
|
||||
"actions units, the merge styles it allows and its default merge style, delete the branch after merge, and the mirror " +
|
||||
"settings: gitea 1.27.3's EditRepoOption by its own names. Left out: name (a rename moves every module's source " +
|
||||
"address), the internal tracker, external tracker and external wiki objects, and mirror_username, mirror_password " +
|
||||
"and mirror_token, which are credentials. projects_mode needs has_projects: true in the same call. Only the fields " +
|
||||
"given change. The repository is read back: a field the forge did not apply (a default branch that does not exist) " +
|
||||
"fails the call, naming it; mirror_interval is compared as a duration, and enable_prune, which the forge never " +
|
||||
"answers, is listed in not_read_back. A reason (why) is required and logged. Answers the settings before and after.",
|
||||
input: {
|
||||
owner: { type: "string", description: "the repository owner" },
|
||||
repo: { type: "string", description: "the repository name" },
|
||||
why: { type: "string", description: "why, in a sentence: logged and answered" },
|
||||
...Object.fromEntries(Object.entries(REPO_FIELDS).map(([name, f]) => [name, { type: f.kind, description: f.says }])),
|
||||
},
|
||||
run: async (args) => {
|
||||
const owner = String(args.owner), repo = String(args.repo), why = String(args.why ?? "").trim();
|
||||
if (!why) throw new Error("say why (why): a repository's settings are changed for a reason, which is logged");
|
||||
if (!admin) throw new Error("the forge's administrator login is not set: a repository's settings are written with it");
|
||||
const given = givenFields(args);
|
||||
if (!given.length) throw new Error(`name at least one setting: ${Object.keys(REPO_FIELDS).join(", ")}`);
|
||||
const body = fullProtectionBody(args, given, REPO_FIELDS, "a repository's settings");
|
||||
const forge = admin();
|
||||
const before = await forge.repoSettings(owner, repo);
|
||||
console.log(`[gitea] gitea_repo_settings_set_full ${owner}/${repo}: ${JSON.stringify(body)} — why: ${why}`);
|
||||
await forge.editRepoSettings(owner, repo, body);
|
||||
const after = await forge.repoSettings(owner, repo);
|
||||
const missed = unapplied(body, after, REPO_FIELDS);
|
||||
if (missed.length) throw new Error(`the forge answered the write of ${owner}/${repo}'s settings but did not apply ${missed.join("; ")}`);
|
||||
const pick = (r: Record<string, unknown>) => Object.fromEntries(Object.keys(REPO_FIELDS).filter((k) => k in r).map((k) => [k, r[k]]));
|
||||
return { repository: `${owner}/${repo}`, why, set: body, before: pick(before), after: pick(after),
|
||||
not_read_back: Object.keys(body).filter((k) => REPO_FIELDS[k]?.unread) };
|
||||
},
|
||||
},
|
||||
|
||||
// ---- Two settings gitea_api refuses (issue 1422 note 7; guard.ts), each its own named tool ----
|
||||
{
|
||||
name: "gitea_default_branch_set",
|
||||
description: "Set a repository's default branch — the branch the forge shows, clones and opens pull requests against, and the one gitea_api never writes a file to. Refused when the branch does not exist, when no protection rule guards it (a default nothing protects would open it to direct writes), or when no reason (why) is given; acts as the tools' account, so the forge refuses it where that account is no administrator of the repository. Answers the default branch before and after.",
|
||||
input: {
|
||||
owner: { type: "string", description: "the repository owner" },
|
||||
repo: { type: "string", description: "the repository name" },
|
||||
branch: { type: "string", description: "the branch to make the default; it must exist and be guarded by a protection rule" },
|
||||
why: { type: "string", description: "why, in a sentence: kept in the call's record" },
|
||||
},
|
||||
run: async (args) => {
|
||||
const owner = String(args.owner), repo = String(args.repo), branch = String(args.branch ?? "").trim();
|
||||
const why = String(args.why ?? "").trim();
|
||||
if (!branch) throw new Error("name the branch to make the default");
|
||||
if (!why) throw new Error("say why (why): a default branch is changed for a reason, which the call's record keeps");
|
||||
pathSegments("branch", branch);
|
||||
const state = await gitea.branchState(owner, repo, branch);
|
||||
if (!state) throw new Refused(`${owner}/${repo} has no branch ${branch}, so it cannot be the default branch. Nothing was sent`);
|
||||
if (!state.protected) {
|
||||
throw new Refused(`no protection rule guards ${branch} of ${owner}/${repo}: a default branch nothing protects takes direct writes ` +
|
||||
`(gitea_branch_protection_set guards it first). Nothing was sent`);
|
||||
}
|
||||
const before = String((await gitea.getRepo(owner, repo)).default_branch ?? "");
|
||||
if (before === branch) return { branch, before, after: before, changed: false, why };
|
||||
const after = String((await gitea.setDefaultBranch(owner, repo, branch)).default_branch ?? "");
|
||||
return { branch, before, after, changed: before !== after, why };
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "gitea_pull_base_set",
|
||||
description: "Change an open pull request's base — the branch it merges into — keeping its head. The checks were judged against the base it had, so the pull request is announced again into the new one. Refused when the base branch does not exist, when the pull request is not open, or when no reason (why) is given; acts as the tools' account. Answers the pull request's base before and after.",
|
||||
input: {
|
||||
owner: { type: "string", description: "the repository owner" },
|
||||
repo: { type: "string", description: "the repository name" },
|
||||
number: { type: "number", description: "the pull request's number" },
|
||||
base: { type: "string", description: "the branch it should merge into; it must exist" },
|
||||
why: { type: "string", description: "why, in a sentence: kept in the call's record" },
|
||||
},
|
||||
run: async (args) => {
|
||||
const owner = String(args.owner), repo = String(args.repo), number = Number(args.number);
|
||||
const base = String(args.base ?? "").trim(), why = String(args.why ?? "").trim();
|
||||
if (!Number.isInteger(number) || number < 1) throw new Error(`${args.number} is not a pull request's number`);
|
||||
if (!base) throw new Error("name the base branch");
|
||||
if (!why) throw new Error("say why (why): a pull request's base is changed for a reason, which the call's record keeps");
|
||||
pathSegments("branch", base);
|
||||
if (!(await gitea.branchState(owner, repo, base))) throw new Refused(`${owner}/${repo} has no branch ${base}, so a pull request cannot merge into it. Nothing was sent`);
|
||||
const had = await gitea.getPullRequest(owner, repo, number);
|
||||
if (had.state !== "open" || had.merged) throw new Refused(`${owner}/${repo}#${number} is not open, so its base is not changed. Nothing was sent`);
|
||||
const before = String(had.base ?? "");
|
||||
if (before === base) return { number, before, after: before, changed: false, why };
|
||||
const after = String((await gitea.setPullBase(owner, repo, number, base)).base ?? "");
|
||||
return { number, before, after, changed: before !== after, why };
|
||||
},
|
||||
},
|
||||
|
||||
// ---- A delivery's note, view and statuses (novox/hq ADR 0239), asked by mesh-delivery ----
|
||||
{
|
||||
name: "gitea_note_append",
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
// What the forge's events ask of the announcer, and the one pass at a start (issue 1422, notes 10 and 11). Nothing polls the
|
||||
// forge: a head or merge is announced when its event is heard, and a pass over every repository runs once when the module
|
||||
// starts and again when the receiver is listening, so what happened while it was down is announced too.
|
||||
//
|
||||
// Apart from index.ts, which has side effects at import, so these are tested with the announcers injected.
|
||||
|
||||
import type { GiteaPull, GiteaRepo } from "./client.js";
|
||||
import type { Announced } from "./pulls.js";
|
||||
import { headKey, headsToAnnounce } from "./pulls.js";
|
||||
import type { ForgeEvent } from "./webhook.js";
|
||||
|
||||
export interface ActDeps<R, P> {
|
||||
getRepo(owner: string, repo: string): Promise<R>;
|
||||
getPull(owner: string, repo: string, number: number): Promise<P & { state?: string; merged?: boolean }>;
|
||||
announceHead(repo: R, pull: P): Promise<unknown>;
|
||||
announceMerge(repo: R, pull: P): Promise<unknown>;
|
||||
announceClosed(repo: R, pull: P): Promise<unknown>;
|
||||
siblingMoves(full: string, number: number): Promise<unknown>;
|
||||
}
|
||||
|
||||
const HEAD_EVENTS = ["forge.pull-request.opened", "forge.pull-request.synchronized", "forge.pull-request.reopened"];
|
||||
|
||||
/**
|
||||
* What the forge's event asks of the announcer at once: a new head is announced, a merge, a close without a merge, and a
|
||||
* pull request retargeted (an edit: the head is announced into the base it now has — the base is in the head's key, so an
|
||||
* edit that changed nothing about the base announces nothing).
|
||||
*/
|
||||
export async function actOn<R, P>(event: ForgeEvent, d: ActDeps<R, P>): Promise<void> {
|
||||
const b = event.body as { owner?: string; repo?: string; number?: number; merged?: boolean };
|
||||
if (!b.owner || !b.repo || !b.number) return;
|
||||
const edited = event.name === "forge.pull-request.edited";
|
||||
const closed = event.name === "forge.pull-request.closed";
|
||||
const head = HEAD_EVENTS.includes(event.name) || edited;
|
||||
const merged = event.name === "forge.pull-request.merged";
|
||||
if (!head && !merged && !closed) return;
|
||||
if (closed && b.merged) return; // a merge is weighed with its own event (ADR 0382 §5)
|
||||
const [repo, pull] = await Promise.all([d.getRepo(b.owner, b.repo), d.getPull(b.owner, b.repo, b.number)]);
|
||||
if (closed) {
|
||||
await d.announceClosed(repo, pull);
|
||||
await d.siblingMoves(`${b.owner}/${b.repo}`, b.number);
|
||||
return;
|
||||
}
|
||||
if (head && pull.state === "open") await d.announceHead(repo, pull);
|
||||
if (merged) await d.announceMerge(repo, pull);
|
||||
}
|
||||
|
||||
/**
|
||||
* The one pass at a start, and its retry: run now; if it fails, say so and run again after `retryMs`, until one pass has
|
||||
* ended. A retry of a failed pass, never a poll: after a pass has ended nothing is scheduled.
|
||||
*/
|
||||
export function startPass(run: () => Promise<void>, opts: { retryMs: number; log?: (l: string) => void }): Promise<void> {
|
||||
const log = opts.log ?? ((l) => console.error(`[gitea] ${l}`));
|
||||
return run().catch((err) => {
|
||||
log(`the pass over every repository failed, asked again in ${opts.retryMs / 1000} s — ${err instanceof Error ? err.message : err}`);
|
||||
return new Promise<void>((resolve) => setTimeout(() => resolve(startPass(run, opts)), opts.retryMs));
|
||||
});
|
||||
}
|
||||
|
||||
export interface HeadDeps {
|
||||
/** Whether the forge shows a merge check of the head: only an announced head has one. */
|
||||
alreadyChecked(pull: GiteaPull): Promise<boolean>;
|
||||
/** Announce it; true when it was this call that announced it (not an event before it). */
|
||||
announceHead(pull: GiteaPull): Promise<boolean>;
|
||||
/** Told of a head this pass announced that no event had. */
|
||||
missed(what: string): void;
|
||||
}
|
||||
|
||||
/**
|
||||
* The heads of one repository's open pull requests a pass announces. A head is announced when it is new and fresh (moved in
|
||||
* the last day); at the first pass after a start (`primed` false) one the forge shows a merge check of was announced
|
||||
* already, and is only remembered.
|
||||
*/
|
||||
export async function announceNewHeads(full: string, open: GiteaPull[], heads: Announced, primed: boolean, dayAgo: string, d: HeadDeps): Promise<void> {
|
||||
for (const pull of headsToAnnounce(full, open, heads)) {
|
||||
const key = headKey(full, pull.number, pull.base);
|
||||
const fresh = primed || (!!pull.updated_at && pull.updated_at > dayAgo);
|
||||
if (fresh && !primed && await d.alreadyChecked(pull)) {
|
||||
heads[key] = String(pull.head_sha);
|
||||
continue;
|
||||
}
|
||||
if (fresh) {
|
||||
if (await d.announceHead(pull)) d.missed(`${key} at ${String(pull.head_sha).slice(0, 8)}`);
|
||||
} else heads[key] = String(pull.head_sha);
|
||||
}
|
||||
}
|
||||
|
||||
export type { GiteaRepo };
|
||||
+19
-22
@@ -3,7 +3,7 @@
|
||||
// with a secret it mints and keeps beside its token; the receiver refuses whatever is unsigned or does not verify, and
|
||||
// publishes each event, trimmed, on the module's events as forge.<kind>.<action> (forge.pull-request.opened, forge.push, …),
|
||||
// on the shared EVENTS path its other events take; its own stream (ADR 0363) is a follow-up once issue 725 ships.
|
||||
// The polls stay as the safety net, slowed; the announced-heads and announced-merges records keep anything from
|
||||
// No poll is the safety net (issue 1422); the announced-heads and announced-merges records keep anything from
|
||||
// being announced twice, whichever path saw it first.
|
||||
//
|
||||
// The secret is never logged: lines say it was minted or reused and where it is kept, never what it is.
|
||||
@@ -144,7 +144,7 @@ export interface ReceiverOptions {
|
||||
publish: (event: ForgeEvent) => Promise<void>;
|
||||
/** Acts on it at once, beside publishing it (the announcer). */
|
||||
act?: (event: ForgeEvent) => Promise<void>;
|
||||
/** Told of every verified delivery — its kind (X-Gitea-Event) and its delivery id — so the polls know events are
|
||||
/** Told of every verified delivery — its kind (X-Gitea-Event) and its delivery id — so the module knows events are
|
||||
* heard, and the module can say when it last heard one. */
|
||||
heard?: (kind: string, delivery: string) => void;
|
||||
/** Told of every refused delivery, with the reason and where it came from. */
|
||||
@@ -242,7 +242,7 @@ export function receiver(opts: ReceiverOptions): Server & { idle(): Promise<void
|
||||
await opts.publish(event);
|
||||
if (opts.act) await opts.act(event);
|
||||
} catch (err) {
|
||||
log(`a forge event ${event.name} could not be published or acted on; the polls catch it — ${err instanceof Error ? err.message : err}`);
|
||||
log(`a forge event ${event.name} could not be published or acted on; the pass over every repository is asked again shortly — ${err instanceof Error ? err.message : err}`);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -276,14 +276,13 @@ export interface HookState {
|
||||
export const SILENT_AFTER = 10 * 60_000;
|
||||
|
||||
/**
|
||||
* **How fast the polls look, and whether the forge's events are heard** (issue 729): every five minutes only while the
|
||||
* forge's events are heard — a verified delivery within the last ten minutes, and none missed since. Until the first
|
||||
* delivery, after ten minutes without one, or once the poll found something no event said, every thirty seconds.
|
||||
* **Whether the forge's events are heard** (issues 729, 1422). Nothing polls the forge, so there is no pace: a delivery
|
||||
* heard within the last ten minutes, and none missed since, is `hearing`.
|
||||
*
|
||||
* It also keeps what `gitea_events` says: where the receiver listens, the hook as ensured, the verified and refused
|
||||
* deliveries, and the polls' finds no event said. **A forge whose events stop is said** (issue 729): once the receiver
|
||||
* deliveries, and the finds of the pass over every repository (at the start and when the receiver listens) that no event said. **A forge whose events stop is said** (issue 729): once the receiver
|
||||
* listens and the hook is ensured, when nothing verified for ten minutes — since the start, or since the last delivery —
|
||||
* and the poll has found at least two heads or merges no event said in that quiet (one could be the poll beating an
|
||||
* and a pass has found at least two heads or merges no event said in that quiet (one could be a pass beating an
|
||||
* event by a second), it is logged loudly once per silent spell, and the module's events are said unhealthy until a
|
||||
* delivery verifies. Deliveries refused since the last verified one are said unhealthy too.
|
||||
*/
|
||||
@@ -318,7 +317,7 @@ export class Pace {
|
||||
this.notHeard = null;
|
||||
}
|
||||
/** No events are listened for, and why: by configuration (`fails` false), or because the receiver could not listen
|
||||
* or the hook could not be ensured (`fails` true, said unhealthy). The polls alone watch. */
|
||||
* or the hook could not be ensured (`fails` true, said unhealthy). Only the pass at the start watches. */
|
||||
unheard(why: string, fails = true): void {
|
||||
this.notHeard = why;
|
||||
this.notHeardFails = fails;
|
||||
@@ -337,10 +336,10 @@ export class Pace {
|
||||
this.refusedSpell++;
|
||||
this.lastRefused = { at: this.now(), reason, from };
|
||||
}
|
||||
/** The poll found a head or merge no forge event announced. */
|
||||
/** The pass over every repository found a head or merge no forge event announced. */
|
||||
miss(what: string): void {
|
||||
const t = this.now();
|
||||
if (this.hearing()) this.log(`THE FORGE'S EVENTS MISSED ${what}: the poll found it, so the polls look every 30 s again until the next event`);
|
||||
if (this.hearing()) this.log(`THE FORGE'S EVENTS MISSED ${what}: the pass over every repository found it`);
|
||||
this.missed = t;
|
||||
this.missCount++;
|
||||
this.lastMiss = { at: t, what };
|
||||
@@ -355,8 +354,8 @@ export class Pace {
|
||||
this.log(`THE FORGE'S EVENTS ARE NOT HEARD: ${this.last ? `no delivery verified since the last at ` +
|
||||
`${new Date(this.last).toISOString()}` : "no delivery ever verified since the start"}, ${minutes} min, from the ` +
|
||||
`system webhook at ${this.hook?.url}${refusedSince ? `; ${refusedSince} refused since, the last because ` +
|
||||
`${this.lastRefused?.reason}` : ""}; the poll found ${what} — the polls alone watch the forge, every 30 s ` +
|
||||
`(novox/hq issue 729)`);
|
||||
`${this.lastRefused?.reason}` : ""}; the pass found ${what}; nothing else watches the forge ` +
|
||||
`(novox/hq issue 729, 1422)`);
|
||||
}
|
||||
}
|
||||
/** Whether a delivery was refused after the last verified one (or with none verified). */
|
||||
@@ -367,9 +366,6 @@ export class Pace {
|
||||
const t = this.now();
|
||||
return this.last > 0 && t - this.last < 10 * 60_000 && this.last > this.missed;
|
||||
}
|
||||
every(): number {
|
||||
return this.hearing() ? 5 * 60_000 : 30_000;
|
||||
}
|
||||
private expecting(): boolean {
|
||||
return this.where !== null && this.hook !== null;
|
||||
}
|
||||
@@ -383,14 +379,14 @@ export class Pace {
|
||||
if (this.silentSince) {
|
||||
state = "unhealthy";
|
||||
why = `${this.last ? `no delivery verified since the last at ${iso(this.last)}` : `the forge has never delivered an ` +
|
||||
`event since the start at ${iso(this.started)}`}, while the poll found ${this.missCount} head(s) or merge(s) no ` +
|
||||
`event since the start at ${iso(this.started)}`}, while the pass found ${this.missCount} head(s) or merge(s) no ` +
|
||||
`event said, the last ${this.lastMiss?.what} at ${iso(this.lastMiss?.at ?? t)}`;
|
||||
} else if (refusedSince) {
|
||||
state = "unhealthy";
|
||||
why = `${refusedSince} deliveries refused ${this.last ? `since the last verified one at ${iso(this.last)}` :
|
||||
"and none verified"}, the last at ${iso(this.lastRefused?.at ?? t)} because ${this.lastRefused?.reason}`;
|
||||
} else if (!this.expecting()) {
|
||||
why = this.notHeard ? `no events are listened for, so the polls alone watch the forge: ${this.notHeard}` :
|
||||
why = this.notHeard ? `no events are listened for, so only the pass at the start watches the forge: ${this.notHeard}` :
|
||||
"the receiver is not listening yet";
|
||||
if (this.notHeard && this.notHeardFails) state = "unhealthy";
|
||||
} else if (this.hearing()) {
|
||||
@@ -398,7 +394,7 @@ export class Pace {
|
||||
} else if (this.last === 0) {
|
||||
why = `no delivery heard yet; the forge is said silent if the poll finds what no event said ${SILENT_AFTER / 60_000} min after the start`;
|
||||
} else {
|
||||
why = `the last delivery was at ${iso(this.last)}; the polls look every 30 s until the next`;
|
||||
why = `the last delivery was at ${iso(this.last)}; nothing polls the forge, so what happens meanwhile is announced at the next start or when the receiver listens again`;
|
||||
}
|
||||
return {
|
||||
healthy: state !== "unhealthy", state, why,
|
||||
@@ -407,7 +403,8 @@ export class Pace {
|
||||
hook: this.hook,
|
||||
verified: { count: this.verifiedCount, last: this.lastVerified && { ...this.lastVerified, at: iso(this.lastVerified.at) } },
|
||||
refused: { count: this.refusedCount, since_verified: refusedSince, last: this.lastRefused && { ...this.lastRefused, at: iso(this.lastRefused.at) } },
|
||||
polls: { every_seconds: this.every() / 1000, hearing: this.hearing(), found_unsaid: this.missCount,
|
||||
polls: { every_seconds: null, note: "no timer asks the forge; one pass at the start and when the receiver listens (issue 1422)",
|
||||
hearing: this.hearing(), found_unsaid: this.missCount,
|
||||
last_found: this.lastMiss && { ...this.lastMiss, at: iso(this.lastMiss.at) } },
|
||||
silent: this.silentSince ? { since: iso(this.silentSince) } : null,
|
||||
};
|
||||
@@ -502,8 +499,8 @@ export function eventsTool(pace: Pace): { name: string; description: string; inp
|
||||
name: "gitea_events",
|
||||
description: "Whether the forge's events reach the mesh (novox/hq issue 729): where the receiver listens, the " +
|
||||
"system webhook as ensured at the start (its id, url, events, kind, and what it replaced), how many deliveries " +
|
||||
"verified and the last one's time and event, how many were refused and the last reason, how often the polls look " +
|
||||
"and whether events are heard, and how many heads or merges the poll found that no event said. Unhealthy when the " +
|
||||
"verified and the last one's time and event, how many were refused and the last reason, whether " +
|
||||
"events are heard (nothing polls; one pass runs at the start), and how many heads or merges that pass found that no event said. Unhealthy when the " +
|
||||
"forge has delivered nothing that verified for ten minutes (since the start or since the last delivery) while the " +
|
||||
"poll found what no event said, when deliveries were refused since the last verified one, or when the receiver " +
|
||||
"could not listen or the webhook could not be made.",
|
||||
|
||||
+100
-25
@@ -18,10 +18,18 @@
|
||||
// `repo.writable` {owner, repo, account}: mesh-delivery-gate, which does not ask the operator's Approve of a pull request
|
||||
// whose merge the forge would refuse, reads that repository again on it.
|
||||
//
|
||||
// **A repository of a user is written, and main's rule is left as the operator set it** (the operator, 2026-10-26, asked
|
||||
// "Write, rule untouched"): the account is a write collaborator there, and only added to main's merge whitelist when that
|
||||
// rule keeps one (client.ts ensureMergerListed). What else in main's rule stops the mesh's merge (a status the mesh does not
|
||||
// set, approvals the gate does not give) is changed by nobody but said: both tools list such a repository as writable, but
|
||||
// blocked, with why (merge_blocked).
|
||||
//
|
||||
// Not taken away: a repository whose last module goes from the indexer keeps its collaborator until a hand removes it;
|
||||
// gitea_unwritable_repos and gitea_grant_write say the delivered list as it is now.
|
||||
|
||||
import type { ForgeRepository } from "./client.js";
|
||||
import type { BranchProtection } from "./protection.js";
|
||||
import { CHECK_CONTEXT, CHECK_STATUS, REPO_CHECK_CONTEXT } from "./pulls.js";
|
||||
|
||||
/** The indexer's state this module reads: one key per registered module, its value naming the repository. */
|
||||
export const INDEXER_MODULES = "mesh-delivery-indexer.modules";
|
||||
@@ -34,8 +42,35 @@ export interface WriteForge {
|
||||
repository(owner: string, repo: string): Promise<ForgeRepository | null>;
|
||||
repositoryPermission(owner: string, repo: string, user: string): Promise<string>;
|
||||
addWriteCollaborator(owner: string, repo: string, user: string): Promise<void>;
|
||||
mainRule(owner: string, repo: string): Promise<BranchProtection | null>;
|
||||
ensureMergerListed(owner: string, repo: string, account: string): Promise<{ changed: boolean; rule: BranchProtection | null }>;
|
||||
}
|
||||
|
||||
/** The statuses every pull request the mesh delivers carries (mesh-delivery's checks); a rule requiring another blocks. */
|
||||
const MESH_CONTEXTS = new Set([CHECK_STATUS, CHECK_CONTEXT, REPO_CHECK_CONTEXT]);
|
||||
|
||||
/** What in main's rule stops the mesh's merge, in words; empty when nothing does. Changed by nobody here: the operator's. */
|
||||
export function mergeBlocks(rule: BranchProtection | null, account: string): string[] {
|
||||
if (!rule) return [];
|
||||
const blocks: string[] = [];
|
||||
if (rule.enable_merge_whitelist === true && !(rule.merge_whitelist_usernames ?? []).includes(account)) {
|
||||
blocks.push(`its merge whitelist [${(rule.merge_whitelist_usernames ?? []).join(", ")}] leaves out ${account} (gitea_grant_write adds it)`);
|
||||
}
|
||||
if (rule.enable_status_check === true) {
|
||||
for (const c of rule.status_check_contexts ?? []) {
|
||||
if (!MESH_CONTEXTS.has(c)) blocks.push(`it requires the status ${c}, which the mesh does not set`);
|
||||
}
|
||||
}
|
||||
const approvals = Number(rule.required_approvals ?? 0);
|
||||
if (approvals > 0) blocks.push(`it requires ${approvals} approving review${approvals === 1 ? "" : "s"} on the forge, which the gate does not give`);
|
||||
if (rule.require_signed_commits === true) blocks.push("it requires signed commits, which the mesh does not make");
|
||||
if (rule.block_on_outdated_branch === true) blocks.push("it refuses a branch behind main, which the mesh does not bring up to date");
|
||||
if (rule.block_on_rejected_reviews === true) blocks.push("it refuses a merge after a review asked for changes, which the gate does not clear");
|
||||
return blocks;
|
||||
}
|
||||
|
||||
const blockedWhy = (blocks: string[]): string => `writable, but main's rule blocks the mesh's merge: ${blocks.join("; ")}`;
|
||||
|
||||
/** One change of the indexer's modules state, as the SDK's state watch hands it. */
|
||||
export interface ModulesChange {
|
||||
key: string;
|
||||
@@ -51,8 +86,13 @@ export interface Written {
|
||||
delivered: string[];
|
||||
/** Made writable now, each read back from the forge. */
|
||||
granted: string[];
|
||||
/** Written already (the organisation's team, or a collaborator that writes). */
|
||||
/** Written already (a collaborator of a user's repository), or written through the organisation's team. */
|
||||
already: string[];
|
||||
/** Each repository of a user: whether the account was added to main's merge whitelist now, and the whitelist read back
|
||||
* (null: main's rule keeps no whitelist, or there is no rule, and whoever writes may merge). */
|
||||
merge_whitelist: { repository: string; changed: boolean; listed: string[] | null }[];
|
||||
/** Writable, but main's rule (the operator's, left as set) stops the mesh's merge, and why. */
|
||||
merge_blocked: { repository: string; why: string }[];
|
||||
/** The forge refused, in its own words; or the repository is the organisation's, whose team should write it. */
|
||||
refused: { repository: string; why: string }[];
|
||||
/** Repositories the account cannot write that are left so, and why: they build no registered module. */
|
||||
@@ -118,7 +158,7 @@ export class DeliveredWrites {
|
||||
"(the watch is not granted or not answered); nothing is granted");
|
||||
}
|
||||
const delivered = this.delivered();
|
||||
const result: Written = { account: this.account, delivered, granted: [], already: [], refused: [], left_read_only: [] };
|
||||
const result: Written = { account: this.account, delivered, granted: [], already: [], merge_whitelist: [], merge_blocked: [], refused: [], left_read_only: [] };
|
||||
const named = only && only.length ? only.map((x) => String(x).trim()) : null;
|
||||
const targets: string[] = [];
|
||||
for (const full of named ?? delivered) {
|
||||
@@ -140,36 +180,55 @@ export class DeliveredWrites {
|
||||
result.refused.push({ repository: full, why: `its permission could not be read — ${said(err)}` });
|
||||
continue;
|
||||
}
|
||||
if (WRITES.has(permission)) {
|
||||
this.writable.add(full);
|
||||
result.already.push(full);
|
||||
continue;
|
||||
}
|
||||
if (owner === this.org) {
|
||||
if (WRITES.has(permission)) {
|
||||
this.writable.add(full);
|
||||
result.already.push(full);
|
||||
continue;
|
||||
}
|
||||
// The organisation's repositories are written through its team (tools-account.ts), never a collaborator.
|
||||
result.refused.push({ repository: full, why: `a repository of ${this.org}, which ${this.account} writes through the ` +
|
||||
`team ${this.org}/${this.account}; the forge gives it "${permission}", so the team is not as it should be (it is ` +
|
||||
"set again at the next mint of the tools' token)" });
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
await this.forge.addWriteCollaborator(owner, name, this.account);
|
||||
} catch (err) {
|
||||
result.refused.push({ repository: full, why: said(err) });
|
||||
continue;
|
||||
// A repository of a user: the account writes it, and is put on main's merge whitelist when the rule keeps one.
|
||||
let changed = false;
|
||||
if (!WRITES.has(permission)) {
|
||||
try {
|
||||
await this.forge.addWriteCollaborator(owner, name, this.account);
|
||||
} catch (err) {
|
||||
result.refused.push({ repository: full, why: said(err) });
|
||||
continue;
|
||||
}
|
||||
let after: string;
|
||||
try {
|
||||
after = await this.forge.repositoryPermission(owner, name, this.account);
|
||||
} catch (err) {
|
||||
result.refused.push({ repository: full, why: `granted, but its permission could not be read back — ${said(err)}` });
|
||||
continue;
|
||||
}
|
||||
if (!WRITES.has(after)) {
|
||||
result.refused.push({ repository: full, why: `the grant was answered, but the forge still gives ${this.account} "${after}"` });
|
||||
continue;
|
||||
}
|
||||
changed = true;
|
||||
}
|
||||
let after: string;
|
||||
try {
|
||||
after = await this.forge.repositoryPermission(owner, name, this.account);
|
||||
const { changed: listedNow, rule } = await this.forge.ensureMergerListed(owner, name, this.account);
|
||||
result.merge_whitelist.push({ repository: full, changed: listedNow,
|
||||
listed: rule?.enable_merge_whitelist === true ? (rule.merge_whitelist_usernames ?? []) : null });
|
||||
const blocks = mergeBlocks(rule, this.account);
|
||||
if (blocks.length > 0) result.merge_blocked.push({ repository: full, why: blockedWhy(blocks) });
|
||||
} catch (err) {
|
||||
result.refused.push({ repository: full, why: `granted, but its permission could not be read back — ${said(err)}` });
|
||||
continue;
|
||||
}
|
||||
if (!WRITES.has(after)) {
|
||||
result.refused.push({ repository: full, why: `the grant was answered, but the forge still gives ${this.account} "${after}"` });
|
||||
result.refused.push({ repository: full, why: `${this.account} writes it, but could not be put on main's merge whitelist — ${said(err)}` });
|
||||
continue;
|
||||
}
|
||||
this.writable.add(full);
|
||||
if (!changed) {
|
||||
result.already.push(full);
|
||||
continue;
|
||||
}
|
||||
result.granted.push(full);
|
||||
await this.emit(WRITABLE_EVENT, { owner, repo: name, account: this.account }).catch((err) =>
|
||||
this.log(`${this.account} writes ${full} now, and that could not be said on the bus — ${said(err)}`));
|
||||
@@ -192,10 +251,12 @@ export class DeliveredWrites {
|
||||
}
|
||||
|
||||
/** Every delivered repository the account cannot write, as the forge says now, and why. */
|
||||
async unwritable(): Promise<{ account: string; delivered: number; unwritable: { repository: string; permission: string; why: string }[] }> {
|
||||
async unwritable(): Promise<{ account: string; delivered: number; unwritable: { repository: string; permission: string; why: string }[];
|
||||
merge_blocked: { repository: string; why: string }[] }> {
|
||||
if (!this.heard) throw new Error(`which repositories build a registered module is not known yet: ${INDEXER_MODULES} has not been read`);
|
||||
const delivered = this.delivered();
|
||||
const out: { repository: string; permission: string; why: string }[] = [];
|
||||
const blocked: { repository: string; why: string }[] = [];
|
||||
for (const full of delivered) {
|
||||
const [owner, name] = split(full);
|
||||
let permission: string;
|
||||
@@ -205,12 +266,21 @@ export class DeliveredWrites {
|
||||
out.push({ repository: full, permission: "unknown", why: `the forge did not answer ${this.account}'s permission on it — ${said(err)}` });
|
||||
continue;
|
||||
}
|
||||
if (WRITES.has(permission)) continue;
|
||||
if (WRITES.has(permission)) {
|
||||
if (owner === this.org) continue; // the organisation's main rules are the mesh's own (mainrules.ts)
|
||||
try {
|
||||
const blocks = mergeBlocks(await this.forge.mainRule(owner, name), this.account);
|
||||
if (blocks.length > 0) blocked.push({ repository: full, why: blockedWhy(blocks) });
|
||||
} catch (err) {
|
||||
blocked.push({ repository: full, why: `writable, but main's rule could not be read — ${said(err)}` });
|
||||
}
|
||||
continue;
|
||||
}
|
||||
const modules = [...this.modules].filter(([, r]) => r === full).map(([m]) => m).sort();
|
||||
out.push({ repository: full, permission, why: `builds ${modules.join(", ")}, and the forge gives ${this.account} ` +
|
||||
`"${permission}" on it: its commit status and its merge are refused until gitea_grant_write grants it` });
|
||||
}
|
||||
return { account: this.account, delivered: delivered.length, unwritable: out };
|
||||
return { account: this.account, delivered: delivered.length, unwritable: out, merge_blocked: blocked };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -224,14 +294,19 @@ export function writeTools(writes: () => DeliveredWrites): Tool[] {
|
||||
description: "Every repository that builds a registered module (the indexer's modules, read over the bus) on which " +
|
||||
"the mesh's merging forge account (mesh-tools, which sets commit statuses and merges) cannot write, with the " +
|
||||
"permission the forge gives it there and why: such a pull request can be checked but never merged, and the " +
|
||||
"delivery gate does not ask its Approve. Empty when it writes every one. Read with the site administrator's login.",
|
||||
"delivery gate does not ask its Approve. Empty when it writes every one. Its merge_blocked lists each repository of " +
|
||||
"a user it writes whose main's rule (the operator's, left as set) still stops the mesh's merge: a merge whitelist " +
|
||||
"without it, a required status the mesh does not set, required approving reviews. Read with the site administrator's login.",
|
||||
input: {},
|
||||
run: async () => writes().unwritable(),
|
||||
},
|
||||
{
|
||||
name: "gitea_grant_write",
|
||||
description: "Give the mesh's merging forge account (mesh-tools) write, as a collaborator, on every repository that " +
|
||||
"builds a registered module and that it cannot write now, or on the repositories named (each must build one). " +
|
||||
description: "Give the mesh's merging forge account (mesh-tools) write, as a collaborator, on every repository of a " +
|
||||
"user that builds a registered module, or on the repositories named (each must build one). Main's rule there is left " +
|
||||
"as the operator set it; only when it keeps a merge whitelist is mesh-tools added to it, read back (merge_whitelist). " +
|
||||
"What else in that rule stops the mesh's merge (a required status the mesh does not set, required approving reviews) " +
|
||||
"is changed by nobody and said in merge_blocked. " +
|
||||
"The module does this itself at its start and when a module of a new repository is registered; this is the same " +
|
||||
"act by hand, such as when a merge or a commit status was refused 403. A repository of the organisation novox " +
|
||||
"is written through its team and needs nothing. Answers what was granted (each read back from the forge), what " +
|
||||
|
||||
@@ -1 +1,9 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# the copies of this package in every module that carries one.
|
||||
modules/xorg/internal/desktop/*.go
|
||||
modules/lemurs/internal/desktop/*.go
|
||||
modules/i3/internal/desktop/*.go
|
||||
modules/xterm/internal/desktop/*.go
|
||||
modules/adwaita/internal/desktop/*.go
|
||||
|
||||
@@ -53,6 +53,22 @@ built for the fallback kernel too) after its boot-entries service, so the entry
|
||||
| file | `/etc/pacman.d/hooks/90-mesh-kernel-boot-entries.hook` | runs the script after a kernel image is installed, upgraded or removed |
|
||||
| packages | `linux-lts`, `linux-lts-headers` | declared after the service and the hook |
|
||||
|
||||
## Kernel modules at boot, and the running kernel's modules kept
|
||||
|
||||
A machine that runs for weeks meets a kernel upgrade that removes `/usr/lib/modules/<the running kernel>`: from then on
|
||||
no module that was not loaded yet can be loaded until the next boot. On 2026-10-28 a build node, up 24 days on
|
||||
`linux-lts` 6.18.54 with only 6.18.58's modules on disk, had `/dev/net/tun` and no tun driver, and every image build
|
||||
there failed in buildah's network setup ("Failed to open() /dev/net/tun: No such device").
|
||||
|
||||
| | what | notes |
|
||||
|---|---|---|
|
||||
| file | `/etc/modules-load.d/mesh-kernel.conf` | the kernel modules loaded at every boot by systemd-modules-load: `tun` (pasta, which buildah uses for a RUN line's network, and every user-space tunnel open `/dev/net/tun`). Read at boot only: nothing is loaded by an apply |
|
||||
| package | `kernel-modules-hook` | keeps the running kernel's modules across an upgrade (saved before the transaction, put back after), so a module first needed later still loads |
|
||||
| service | `linux-modules-cleanup.service` | the package's oneshot, enabled at boot and stopped otherwise: it moves the modules of a kernel that is neither running nor owned by a package to `/usr/lib/modules/.old`, which the package's tmpfiles rule empties after four weeks. An apply never starts it |
|
||||
|
||||
A machine whose running kernel's modules are already gone loads nothing new until it is started again; the image build
|
||||
module says so in its health and takes no image build there meanwhile.
|
||||
|
||||
## Tools
|
||||
|
||||
| tool | | answers |
|
||||
@@ -63,7 +79,7 @@ built for the fallback kernel too) after its boot-entries service, so the entry
|
||||
## Tests
|
||||
|
||||
`go test ./...`: the tool over a stand-in tree holding the control node's own files of 2026-10-14; the
|
||||
manifest's shape (the default, the order, the unit and its sandbox, reload, restart and health); the boot-entries script run with `sh` over stand-in roots (a systemd-boot entries directory, a GRUB menu, a hook that copies kernels to one fixed name): the entry from the booted command line, a menu generated again only when the kernels changed, no loader, no room, the taken-over hook; `kernel_boot` over the same roots; and the huge-pages script as the mesh
|
||||
manifest's shape (the default, the order, the unit and its sandbox, reload, restart and health); the kernel modules loaded at boot (`tun`) and the package and boot service that keep the running kernel's modules; the boot-entries script run with `sh` over stand-in roots (a systemd-boot entries directory, a GRUB menu, a hook that copies kernels to one fixed name): the entry from the booted command line, a menu generated again only when the kernels changed, no loader, no room, the taken-over hook; `kernel_boot` over the same roots; and the huge-pages script as the mesh
|
||||
writes it, run with `sh` against a stand-in directory: it sets and reads back, keeps what it found once,
|
||||
rotates the journal once and not for `always` nor while the machine starts, goes straight to a new value on a
|
||||
reload without rewriting what it found, gives back what it found and then keeps nothing, refuses a value the kernel
|
||||
|
||||
@@ -24,7 +24,7 @@ func TestTheDefaultIsMadviseAndTheUnitFollowsTheSetting(t *testing.T) {
|
||||
for _, r := range m.Resources {
|
||||
ids = append(ids, r["id"].(string))
|
||||
}
|
||||
if strings.Join(ids, " ") != "state huge-pages unit apply boot-entries boot-unit boot-hook boot-apply linux-lts linux-lts-headers" {
|
||||
if strings.Join(ids, " ") != "state huge-pages unit apply boot-entries boot-unit boot-hook boot-apply linux-lts linux-lts-headers modules-load modules-keep modules-cleanup" {
|
||||
t.Fatalf("order: %v", ids)
|
||||
}
|
||||
script := m.resource(t, "huge-pages")
|
||||
@@ -57,6 +57,31 @@ func TestTheDefaultIsMadviseAndTheUnitFollowsTheSetting(t *testing.T) {
|
||||
if h := apply["health"].(map[string]any); h["kind"] != "unit" {
|
||||
t.Fatalf("health: %v", h)
|
||||
}
|
||||
// A build node, 2026-10-28: up 24 days on linux-lts 6.18.54 after an upgrade to 6.18.58 removed the running kernel's modules,
|
||||
// so tun could not be loaded when a build first asked for it. tun is loaded at every boot, before an upgrade can take
|
||||
// its file away, and the running kernel's modules are kept across an upgrade (kernel-modules-hook) and cleaned up at
|
||||
// the next boot.
|
||||
load := m.resource(t, "modules-load")
|
||||
if load["path"] != "/etc/modules-load.d/mesh-kernel.conf" || load["mode"] != "0644" {
|
||||
t.Fatalf("modules-load: %v", load)
|
||||
}
|
||||
var names []string
|
||||
for _, l := range strings.Split(load.text(t), "\n") {
|
||||
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "#") {
|
||||
names = append(names, l)
|
||||
}
|
||||
}
|
||||
if strings.Join(names, " ") != "tun" {
|
||||
t.Fatalf("the modules loaded at boot: %v", names)
|
||||
}
|
||||
if keep := m.resource(t, "modules-keep"); keep["type"] != "package" || keep["package"] != "kernel-modules-hook" {
|
||||
t.Fatalf("modules-keep: %v", keep)
|
||||
}
|
||||
// A oneshot at boot: enabled, and stopped once it ran, never started by an apply (it moves the modules of kernels no
|
||||
// package owns away).
|
||||
if c := m.resource(t, "modules-cleanup"); c["unit"] != "linux-modules-cleanup.service" || c["state"] != "stopped" || c["boot"] != "enabled" {
|
||||
t.Fatalf("modules-cleanup: %v", c)
|
||||
}
|
||||
}
|
||||
|
||||
// bed is the script as the mesh writes it for one value of the setting, a stand-in transparent_hugepage
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
# Written by the mesh's kernel module (systemd-modules-load reads it at boot).
|
||||
# tun: buildah's network for a RUN line (pasta) and every other user-space tunnel open /dev/net/tun.
|
||||
# Loaded at boot, so an upgrade that removes the running kernel's modules cannot keep it out later.
|
||||
tun
|
||||
@@ -100,6 +100,25 @@
|
||||
"id": "linux-lts-headers",
|
||||
"type": "package",
|
||||
"package": "linux-lts-headers"
|
||||
},
|
||||
{
|
||||
"id": "modules-load",
|
||||
"type": "file",
|
||||
"path": "/etc/modules-load.d/mesh-kernel.conf",
|
||||
"mode": "0644",
|
||||
"template": "files/modules-load.conf"
|
||||
},
|
||||
{
|
||||
"id": "modules-keep",
|
||||
"type": "package",
|
||||
"package": "kernel-modules-hook"
|
||||
},
|
||||
{
|
||||
"id": "modules-cleanup",
|
||||
"type": "service",
|
||||
"unit": "linux-modules-cleanup.service",
|
||||
"state": "stopped",
|
||||
"boot": "enabled"
|
||||
}
|
||||
],
|
||||
"build": {
|
||||
|
||||
@@ -1 +1,9 @@
|
||||
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# the copies of this package in every module that carries one.
|
||||
modules/xorg/internal/desktop/*.go
|
||||
modules/lemurs/internal/desktop/*.go
|
||||
modules/i3/internal/desktop/*.go
|
||||
modules/xterm/internal/desktop/*.go
|
||||
modules/adwaita/internal/desktop/*.go
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buildstep"
|
||||
@@ -84,9 +85,17 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
|
||||
// The capacity's own change wakes the puller (issue 1887): the runtime's watch of the config value as mesh-config
|
||||
// resolved it; the settings file is read only when that watch cannot start. A test's Capacity is its own.
|
||||
var watch func(string, func(stdio.ProvisionChange) error) error
|
||||
// And the CPU bounds it (issue 1806's follow-up): the configured capacity is an upper bound, the node's build slice
|
||||
// gives floor(its quota's cores / the module's cores-per-step) at once, read live; a test judges by its own Capacity.
|
||||
// And the node's build hours (a node may build only in a window): outside them it takes
|
||||
// nothing new.
|
||||
var cores func() float64
|
||||
var hours func() (*buildstep.BuildWindow, string)
|
||||
if capacity == nil {
|
||||
capacity = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, m.Say)
|
||||
hours = buildstep.NodeBuildHours(m.Settings, m.Node, m.Say)
|
||||
capacity = buildstep.NodeCapacity(m.Settings, m.Node, m.Say)
|
||||
watch = buildstep.WatchConfigServed
|
||||
cores = buildstep.NodeCoresPerStep(m.Settings, buildstep.DefaultCoresPerStep[delivery.AskGo], m.Say)
|
||||
}
|
||||
ask := m.askSteps(steps)
|
||||
var load func(string) ([]byte, error)
|
||||
@@ -97,7 +106,7 @@ func (m *Module) Puller(steps []StepTool, source buildstep.AskSource) *buildstep
|
||||
}
|
||||
}
|
||||
return &buildstep.Puller{Source: source, Language: delivery.AskGo, Node: m.Node, Put: m.Steps, PausePut: m.Pauses, Load: load,
|
||||
Capacity: capacity, WatchConfig: watch, Logf: m.Say, Has: m.Has,
|
||||
Capacity: capacity, WatchConfig: watch, CoresPerStep: cores, BuildHours: hours, Logf: m.Say, Has: m.Has,
|
||||
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
|
||||
if err := m.Check(ctx); err != nil {
|
||||
return nil, err
|
||||
@@ -120,6 +129,13 @@ func (m *Module) SeatTools(p *buildstep.Puller) []stdio.Tool {
|
||||
// StepEndTool ends one step left running (issue 1890); the module's steps state is the one it reads and writes.
|
||||
const StepEndTool = "build_go_step_end"
|
||||
|
||||
// endLeftRunning is the module's start sweep (issue 1890): it ends the steps of its node that a previous process left
|
||||
// running (buildstep.EndStaleAtStart; every step names its owner). main runs it once at the start, and the module's test
|
||||
// runs it through the conformance kit's left-running case.
|
||||
func (m *Module) endLeftRunning(ctx context.Context, scan buildstep.Scan, booted time.Time, logf func(string, ...any)) {
|
||||
buildstep.EndStaleAtStart(ctx, scan, m.Node, booted, logf)
|
||||
}
|
||||
|
||||
// stepScan is the module's steps state as buildstep reads it.
|
||||
func (m *Module) stepScan() buildstep.Scan {
|
||||
return buildstep.ScanOf(stdio.State(delivery.StepsState))
|
||||
|
||||
@@ -3,7 +3,9 @@ package main
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
@@ -125,12 +127,39 @@ func TestManifestClaimsTheGoBuildSeatAndReadsItsCapacity(t *testing.T) {
|
||||
raw, _ := os.ReadFile(moduleDir + "/module.json")
|
||||
var doc struct {
|
||||
Config map[string]struct {
|
||||
Default string `json:"default"`
|
||||
Default json.RawMessage `json:"default"`
|
||||
Pattern, Description string
|
||||
} `json:"config"`
|
||||
}
|
||||
_ = json.Unmarshal(raw, &doc)
|
||||
if doc.Config[buildstep.CapacitySetting].Default != buildstep.DefaultCapacities {
|
||||
t.Errorf("the capacity's default = %q, want %q", doc.Config[buildstep.CapacitySetting].Default, buildstep.DefaultCapacities)
|
||||
// Issue 1806's follow-up: the cores a step needs is a setting of its own, in the settings file, with the SDK's default.
|
||||
if cores := string(doc.Config[buildstep.CoresPerStepSetting].Default); cores != fmt.Sprint(buildstep.DefaultCoresPerStep[delivery.AskGo]) {
|
||||
t.Errorf("the cores-per-step default = %s, want %g", cores, buildstep.DefaultCoresPerStep[delivery.AskGo])
|
||||
}
|
||||
if got := buildstep.NodeCoresPerStep(moduleDir+"/files/settings.json", 0, nil)(); got != buildstep.DefaultCoresPerStep[delivery.AskGo] {
|
||||
t.Errorf("the settings template holds cores-per-step %g", got)
|
||||
}
|
||||
// And the build hours (the operator, 2026-10-26): a setting of their own, empty by default (any hour), in the template.
|
||||
if hours := string(doc.Config[buildstep.BuildHoursSetting].Default); hours != `""` {
|
||||
t.Errorf("the build-hours default = %s, want empty", hours)
|
||||
}
|
||||
if w, fault := buildstep.NodeBuildHours(moduleDir+"/files/settings.json", "g14", t.Errorf)(); w != nil || fault != "" {
|
||||
t.Errorf("the settings template holds build hours %v (%s)", w, fault)
|
||||
}
|
||||
// ADR 0567 §15: build-hours names no node (one window, or empty for any hour) and capacity defaults to 0.
|
||||
window := regexp.MustCompile("^(?:" + doc.Config[buildstep.BuildHoursSetting].Pattern + ")$")
|
||||
for value, ok := range map[string]bool{"": true, "18:00-06:00": true, "g14=18:00-06:00": false, "18:00": false} {
|
||||
if window.MatchString(value) != ok {
|
||||
t.Errorf("build-hours pattern on %q: accepted %v, want %v", value, !ok, ok)
|
||||
}
|
||||
}
|
||||
if strings.Contains(doc.Config[buildstep.BuildHoursSetting].Description, "=") {
|
||||
t.Errorf("the build-hours description names a node pair")
|
||||
}
|
||||
var capacity float64
|
||||
_ = json.Unmarshal(doc.Config[buildstep.CapacitySetting].Default, &capacity)
|
||||
if capacity != 0 {
|
||||
t.Errorf("the capacity's default = %g, want 0", capacity)
|
||||
}
|
||||
settings := resource(t, m, "settings")
|
||||
if settings["path"] != "${dir:state}/settings.json" {
|
||||
|
||||
@@ -116,7 +116,7 @@ func TestTheIntegrationLevelDigestsUnderBothTags(t *testing.T) {
|
||||
t.Errorf("the integration digest listed %q", s.command)
|
||||
}
|
||||
}
|
||||
if DigestVersion != "5" {
|
||||
if DigestVersion != "6" {
|
||||
t.Errorf("the digest's form changed in kind and is still version %s", DigestVersion)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -73,6 +73,7 @@ func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
|
||||
states := newStates()
|
||||
m := moduleWith(states, "go1.26.0")
|
||||
m.Work, m.Cache, m.Inside = t.TempDir(), t.TempDir(), true
|
||||
m.Capacity = func() int { return 1 } // its own slot: no settings file, and a node with no capacity set pulls nothing (ADR 0567 §15)
|
||||
m.Run = StepRunner(t.TempDir())
|
||||
m.GoModCache = goEnvOf(t, "GOMODCACHE") // the module cache this module's own build filled, read and not fetched
|
||||
run, gocache := m.Run, goEnvOf(t, "GOCACHE") // this machine's build cache, so the composer's packages compile once
|
||||
|
||||
@@ -15,10 +15,12 @@ import (
|
||||
func (m *Module) cpuLimits() buildcpu.Limits {
|
||||
slots := 1
|
||||
switch {
|
||||
case m.Held != nil:
|
||||
slots = m.Held.CapacityNow() // the capacity the puller judges by, bounded by the node's build slice
|
||||
case m.Capacity != nil:
|
||||
slots = m.Capacity()
|
||||
case m.Settings != "":
|
||||
slots = buildstep.NodeCapacity(m.Settings, buildstep.DefaultCapacities, m.Node, nil)()
|
||||
slots = buildstep.NodeCapacity(m.Settings, m.Node, nil)()
|
||||
}
|
||||
return buildstep.NodeLimits(slots)
|
||||
}
|
||||
@@ -32,5 +34,9 @@ func (m *Module) threadEnv() []string {
|
||||
// cpuStatus is what the status tool says of it: the limits a step is given, and what the kernel holds this runtime
|
||||
// to (its unit and its slice), read from its cgroup rather than assumed.
|
||||
func (m *Module) cpuStatus() map[string]any {
|
||||
return map[string]any{"limits": m.cpuLimits(), "in-force": buildcpu.ReadInForce()}
|
||||
out := map[string]any{"limits": m.cpuLimits(), "in-force": buildcpu.ReadInForce()}
|
||||
if m.Held != nil {
|
||||
out["capacity"] = m.Held.CapacityDetail() // the capacity judged by and why (issue 1806's follow-up)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
@@ -16,6 +16,7 @@ import (
|
||||
"io"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
@@ -29,17 +30,29 @@ import (
|
||||
// commits (the lead's decision L5) and the integration level's tags integration,isolation (the checker's agreement).
|
||||
// 4: the siblings' commits enter only the digest of a package whose tests reach a file naming MESH_CHECK_BESIDE (ADR 0536 §1).
|
||||
// 5: a catalogue module's non-Go files enter the digest of its packages (issue 1935).
|
||||
const DigestVersion = "5"
|
||||
// 6: a reads-outside marker that lists what its tests read enters the digest with those files' contents (issue 1936).
|
||||
const DigestVersion = "6"
|
||||
|
||||
// besideVariable is the name a file bears when it reads where the siblings lie; a package whose tests reach such a file
|
||||
// depends on the siblings' commits.
|
||||
const besideVariable = "MESH_CHECK_BESIDE"
|
||||
|
||||
// namesVariable is the same for the mesh's node names: a package whose tests reach a file naming it depends on the node
|
||||
// list the ask carries, so its digest holds the sorted names and a new node misses a kept verdict (ADR 0567 §15).
|
||||
const namesVariable = "MESH_NODE_NAMES"
|
||||
|
||||
// ReadsOutsideMarker is the name of a file in a package's directory that says its tests read files outside it (a
|
||||
// repository's documents, a sibling directory): such a package is never kept (issue 1935). The digest hashes only what
|
||||
// the Go toolchain lists, so it cannot name what a test opens elsewhere; a marker file, beside the Go files where a
|
||||
// reader of the package sees it, is the plain way to say so until the checker keeps the files a test opened (the Go
|
||||
// test log) with each verdict.
|
||||
// repository's documents, a sibling directory), which the Go toolchain does not list (issue 1935). It lists, one per
|
||||
// line, what those tests read, as paths from the repository's root; a path may be a glob (filepath.Match), and a
|
||||
// directory it names is read whole (issue 1936). Blank lines and lines starting with # are not read. A line starting
|
||||
// with ? names a path whose absence is what the tests read (a file that must not be there): it may match nothing.
|
||||
//
|
||||
// The digest then hashes, besides what the toolchain lists, every line of the marker, every file it names (under its
|
||||
// path in the tree, by content) and every file of the package's own directory, Go files included (a file its build tags
|
||||
// exclude is one go list does not hash). A marker that lists nothing, a line that is not a clean relative path, or a
|
||||
// line without ? that matches nothing makes the package one that is never kept (its digest a fresh random hex): a list
|
||||
// nobody can trust keeps nothing. The exact fix, the files a test opened kept
|
||||
// from the Go test log with each verdict, replaces the lists (issue 1936).
|
||||
const ReadsOutsideMarker = "reads-outside"
|
||||
|
||||
// NeedsMarker is the name of a file in a package's directory that names, one per line, what its integration tests need
|
||||
@@ -128,9 +141,13 @@ type DigestResult struct {
|
||||
// only when they are the siblings it gave the fetch.
|
||||
Beside []string `json:"beside,omitempty"`
|
||||
WithoutTests []string `json:"without_tests,omitempty"`
|
||||
// ReadsOutside names the packages that carry the marker: each has a digest that no kept verdict can have (a fresh
|
||||
// random one), so the check runs it every time.
|
||||
// ReadsOutside names the packages whose marker lists nothing it can trust (ReadsOutsideMarker): each has a digest
|
||||
// that no kept verdict can have (a fresh random one), so the check runs it every time.
|
||||
ReadsOutside []string `json:"reads_outside,omitempty"`
|
||||
// neverKept is, per package of ReadsOutside, why its marker cannot be trusted; hashedFiles caches the content lines
|
||||
// of the files markers name, which several packages share (the catalogue's manifests).
|
||||
neverKept map[string]string
|
||||
hashedFiles map[string]string
|
||||
// Needs are the packages whose tests at this level need something of the node, as their NeedsMarker names it.
|
||||
Needs []delivery.PackageNeeds `json:"needs,omitempty"`
|
||||
// JudgedBy is the program that ran the digest (ADR 0535 §5): "installed", or "candidate <merge commit>" under the
|
||||
@@ -151,6 +168,8 @@ func (m *Module) digestTool() StepTool {
|
||||
"workspace": map[string]any{"type": "string", "description": "the workspace id build_go_fetch answered"},
|
||||
"level": map[string]any{"type": "string", "enum": []string{LevelUnit, LevelIntegration}, "description": "the level"},
|
||||
"runner": runnerProperty,
|
||||
delivery.NodeNamesArg: map[string]any{"type": "array", "items": map[string]any{"type": "string"},
|
||||
"description": "optional, the names of the mesh's nodes: a package whose tests read " + delivery.NodeNamesEnv + " has them in its digest"},
|
||||
},
|
||||
"required": []string{"workspace", "level"},
|
||||
},
|
||||
@@ -165,6 +184,10 @@ func (m *Module) digest(ctx context.Context, args map[string]any) (any, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
names, err := nodeNamesArg(args[delivery.NodeNamesArg])
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if runner == RunnerWorkspace {
|
||||
if err := m.workspaceThere(workspace); err != nil {
|
||||
return nil, err
|
||||
@@ -178,7 +201,7 @@ func (m *Module) digest(ctx context.Context, args map[string]any) (any, error) {
|
||||
default:
|
||||
return nil, fmt.Errorf("the level %q is neither unit nor integration", level)
|
||||
}
|
||||
a, judged, err := m.asCandidate(ctx, levelSpec{Step: stepDigest, Level: level, Workspace: workspace})
|
||||
a, judged, err := m.asCandidate(ctx, levelSpec{Step: stepDigest, Level: level, Workspace: workspace, NodeNames: names})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -188,7 +211,7 @@ func (m *Module) digest(ctx context.Context, args map[string]any) (any, error) {
|
||||
a.Digest.JudgedBy = judged
|
||||
return *a.Digest, nil
|
||||
}
|
||||
res, err := m.digestHere(ctx, workspace, level)
|
||||
res, err := m.digestHere(ctx, workspace, level, names)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -197,7 +220,7 @@ func (m *Module) digest(ctx context.Context, args map[string]any) (any, error) {
|
||||
}
|
||||
|
||||
// digestHere is the digest step as this program runs it.
|
||||
func (m *Module) digestHere(ctx context.Context, workspace, level string) (DigestResult, error) {
|
||||
func (m *Module) digestHere(ctx context.Context, workspace, level string, names []string) (DigestResult, error) {
|
||||
if !workspaceOK.MatchString(workspace) {
|
||||
return DigestResult{}, fmt.Errorf("%q is not a workspace id", workspace)
|
||||
}
|
||||
@@ -228,6 +251,14 @@ func (m *Module) digestHere(ctx context.Context, workspace, level string) (Diges
|
||||
besideHeader = append(besideHeader, "beside "+b)
|
||||
}
|
||||
|
||||
// The node names, sorted, as a header line for a package that reads them.
|
||||
sortedNames := append([]string{}, names...)
|
||||
sort.Strings(sortedNames)
|
||||
var namesHeader []string
|
||||
if len(sortedNames) > 0 {
|
||||
namesHeader = []string{"node-names " + strings.Join(sortedNames, ",")}
|
||||
}
|
||||
|
||||
dirs, err := m.goModDirs(ctx, tree)
|
||||
if err != nil {
|
||||
return DigestResult{}, err
|
||||
@@ -238,7 +269,7 @@ func (m *Module) digestHere(ctx context.Context, workspace, level string) (Diges
|
||||
if err != nil {
|
||||
return DigestResult{}, err
|
||||
}
|
||||
if err := digestModule(&res, pkgs, tree, header, besideHeader, level == LevelIntegration); err != nil {
|
||||
if err := digestModule(&res, pkgs, tree, header, besideHeader, namesHeader, level == LevelIntegration); err != nil {
|
||||
return DigestResult{}, err
|
||||
}
|
||||
}
|
||||
@@ -247,7 +278,7 @@ func (m *Module) digestHere(ctx context.Context, workspace, level string) (Diges
|
||||
sort.Slice(res.Needs, func(i, j int) bool { return res.Needs[i].Package < res.Needs[j].Package })
|
||||
for _, p := range res.ReadsOutside {
|
||||
if m.Say != nil {
|
||||
m.Say("the %s level of %s always runs: its directory carries %s, so its tests read files outside it that no digest names", level, p, ReadsOutsideMarker)
|
||||
m.Say("the %s level of %s always runs: its %s %s, so its tests read files outside it that no digest names", level, p, ReadsOutsideMarker, res.neverKept[p])
|
||||
}
|
||||
}
|
||||
return res, nil
|
||||
@@ -289,7 +320,7 @@ func parsePackages(out string) (map[string]*goPackage, error) {
|
||||
|
||||
// digestModule adds to res the digest of each root package of one go.mod directory. A root is a package of the main
|
||||
// module that is listed, not only a dependency, and not a variant of another.
|
||||
func digestModule(res *DigestResult, pkgs map[string]*goPackage, tree string, header, beside []string, needs bool) error {
|
||||
func digestModule(res *DigestResult, pkgs map[string]*goPackage, tree string, header, beside, names []string, needs bool) error {
|
||||
roots := []string{}
|
||||
for path, p := range pkgs {
|
||||
if p.ForTest == "" && !p.DepOnly && !p.Standard && !strings.HasSuffix(path, ".test") && p.Module != nil && p.Module.Main {
|
||||
@@ -316,18 +347,27 @@ func digestModule(res *DigestResult, pkgs map[string]*goPackage, tree string, he
|
||||
res.Needs = append(res.Needs, delivery.PackageNeeds{Package: path, Needs: n})
|
||||
}
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(root.Dir, ReadsOutsideMarker)); err == nil {
|
||||
// The digest of a package that reads outside itself is one nothing can have kept: the key is a fresh random
|
||||
// hex, so the ask finds no kept verdict and runs it, and what the checker writes under it is never read again.
|
||||
declared, why, err := declaredReads(tree, root, res.hashed())
|
||||
if err != nil {
|
||||
return fmt.Errorf("%s: its %s: %w", path, ReadsOutsideMarker, err)
|
||||
}
|
||||
if why != "" {
|
||||
// The digest of a package whose reads outside itself are not listed is one nothing can have kept: the key is a
|
||||
// fresh random hex, so the ask finds no kept verdict and runs it, and what the checker writes under it is never
|
||||
// read again.
|
||||
var nonce [32]byte
|
||||
if _, err := rand.Read(nonce[:]); err != nil {
|
||||
return err
|
||||
}
|
||||
res.Packages[path] = hex.EncodeToString(nonce[:])
|
||||
res.ReadsOutside = append(res.ReadsOutside, path)
|
||||
if res.neverKept == nil {
|
||||
res.neverKept = map[string]string{}
|
||||
}
|
||||
res.neverKept[path] = why
|
||||
continue
|
||||
}
|
||||
d, err := packageDigest(root, testMain, pkgs, tree, header, beside)
|
||||
d, err := packageDigest(root, testMain, pkgs, tree, header, beside, names, declared)
|
||||
if err != nil {
|
||||
return fmt.Errorf("%s: %w", path, err)
|
||||
}
|
||||
@@ -339,16 +379,17 @@ func digestModule(res *DigestResult, pkgs map[string]*goPackage, tree string, he
|
||||
// packageDigest is the digest of one package: a hash over a sorted text of the header, the files of every package its
|
||||
// tests reach in the module's own tree (by their content, under their path in the tree), the version and checksum of
|
||||
// every module outside it, and the package's own test data. The beside lines (the siblings' commits) join the header
|
||||
// only when a file the tests reach names MESH_CHECK_BESIDE. A package of the standard library is the Go version's.
|
||||
func packageDigest(root, testMain *goPackage, pkgs map[string]*goPackage, tree string, header, beside []string) (string, error) {
|
||||
// only when a file the tests reach names MESH_CHECK_BESIDE, and the node names likewise for MESH_NODE_NAMES. A package of the standard library is the Go version's.
|
||||
// The lines a package's ReadsOutsideMarker declares join them (declaredReads).
|
||||
func packageDigest(root, testMain *goPackage, pkgs map[string]*goPackage, tree string, header, beside, names, declared []string) (string, error) {
|
||||
reached := map[string]*goPackage{root.ImportPath: root}
|
||||
for _, dep := range testMain.Deps {
|
||||
if p := pkgs[dep]; p != nil {
|
||||
reached[dep] = p
|
||||
}
|
||||
}
|
||||
lines := []string{}
|
||||
readsBeside := false
|
||||
lines := append([]string{}, declared...)
|
||||
readsBeside, readsNames := false, false
|
||||
for _, p := range reached {
|
||||
if p.Standard || p.Module == nil {
|
||||
continue
|
||||
@@ -367,9 +408,10 @@ func packageDigest(root, testMain *goPackage, pkgs map[string]*goPackage, tree s
|
||||
return "", err
|
||||
}
|
||||
lines = append(lines, l)
|
||||
if !readsBeside {
|
||||
if raw, err := os.ReadFile(filepath.Join(p.Dir, f)); err == nil && bytes.Contains(raw, []byte(besideVariable)) {
|
||||
readsBeside = true
|
||||
if !readsBeside || !readsNames {
|
||||
if raw, err := os.ReadFile(filepath.Join(p.Dir, f)); err == nil {
|
||||
readsBeside = readsBeside || bytes.Contains(raw, []byte(besideVariable))
|
||||
readsNames = readsNames || bytes.Contains(raw, []byte(namesVariable))
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -401,6 +443,9 @@ func packageDigest(root, testMain *goPackage, pkgs map[string]*goPackage, tree s
|
||||
if readsBeside {
|
||||
header = append(append([]string{}, header...), beside...)
|
||||
}
|
||||
if readsNames {
|
||||
header = append(append([]string{}, header...), names...)
|
||||
}
|
||||
h := sha256.New()
|
||||
for _, l := range append(append([]string{}, header...), lines...) {
|
||||
h.Write([]byte(l + "\n"))
|
||||
@@ -469,3 +514,104 @@ func contentLine(tree string, p *goPackage, path string) (string, error) {
|
||||
sum := sha256.Sum256(raw)
|
||||
return "file " + label + " " + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
|
||||
// hashed is the cache of content lines a digest step shares across its packages.
|
||||
func (r *DigestResult) hashed() map[string]string {
|
||||
if r.hashedFiles == nil {
|
||||
r.hashedFiles = map[string]string{}
|
||||
}
|
||||
return r.hashedFiles
|
||||
}
|
||||
|
||||
// declaredReads are the digest lines a package's ReadsOutsideMarker adds: each of its lines, the content line of every
|
||||
// file it names, and those of every file of the package's own directory. Without a marker it answers nothing. why is
|
||||
// set, and the lines nil, when the marker cannot be trusted: it lists nothing, a line is not a clean relative path, or
|
||||
// a line without ? matches nothing.
|
||||
func declaredReads(tree string, root *goPackage, cache map[string]string) (lines []string, why string, err error) {
|
||||
raw, err := os.ReadFile(filepath.Join(root.Dir, ReadsOutsideMarker))
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return nil, "", nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
var patterns []string
|
||||
for _, l := range strings.Split(string(raw), "\n") {
|
||||
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "#") {
|
||||
patterns = append(patterns, l)
|
||||
}
|
||||
}
|
||||
if len(patterns) == 0 {
|
||||
return nil, "lists nothing", nil
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
add := func(path string) error {
|
||||
if seen[path] {
|
||||
return nil
|
||||
}
|
||||
seen[path] = true
|
||||
l, ok := cache[path]
|
||||
if !ok {
|
||||
var err error
|
||||
if l, err = contentLine(tree, root, path); err != nil {
|
||||
return err
|
||||
}
|
||||
cache[path] = l
|
||||
}
|
||||
lines = append(lines, "read "+l)
|
||||
return nil
|
||||
}
|
||||
for _, l := range patterns {
|
||||
pattern, absent := strings.CutPrefix(l, "?")
|
||||
if !cleanTreePattern(pattern) {
|
||||
return nil, fmt.Sprintf("names %q, which is not a clean path from the repository's root", l), nil
|
||||
}
|
||||
matches, err := filepath.Glob(filepath.Join(tree, filepath.FromSlash(pattern)))
|
||||
if err != nil {
|
||||
return nil, fmt.Sprintf("names %q, which is not a glob", l), nil
|
||||
}
|
||||
if len(matches) == 0 && !absent {
|
||||
return nil, fmt.Sprintf("names %q, which matches nothing (a path whose absence is read starts with ?)", l), nil
|
||||
}
|
||||
lines = append(lines, "reads "+l)
|
||||
for _, match := range matches {
|
||||
if err := walkFiles(match, add); err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
}
|
||||
}
|
||||
// Every file of the package's own directory: its lists, its marker, and its Go files too, since go list leaves a
|
||||
// file its build tags exclude (an integration test a unit test globs) out of what the digest hashes. A file the
|
||||
// toolchain lists is hashed twice, which changes nothing.
|
||||
err = walkFiles(root.Dir, add)
|
||||
return lines, "", err
|
||||
}
|
||||
|
||||
// cleanTreePattern says a marker's line is a path from the repository's root that stays inside it.
|
||||
func cleanTreePattern(p string) bool {
|
||||
if p == "" || strings.ContainsAny(p, " \t\\") || strings.HasPrefix(p, "/") || path.Clean(p) != p {
|
||||
return false
|
||||
}
|
||||
for _, e := range strings.Split(p, "/") {
|
||||
if e == ".." || e == "." {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// walkFiles calls each with every file at or under path, skipping .git and node_modules, in lexical order.
|
||||
func walkFiles(path string, each func(string) error) error {
|
||||
return filepath.WalkDir(path, func(p string, e fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if e.IsDir() {
|
||||
if n := e.Name(); p != path && (n == ".git" || n == "node_modules") {
|
||||
return filepath.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return each(p)
|
||||
})
|
||||
}
|
||||
|
||||
@@ -172,8 +172,9 @@ func TestAModuleOutsideTheTreeCountsByVersionAndChecksum(t *testing.T) {
|
||||
root := &goPackage{ImportPath: "x/r", Dir: dir, Module: &goModule{Path: "x", Main: true}, GoFiles: []string{"r.go"}, TestGoFiles: []string{"r.go"}}
|
||||
testMain := &goPackage{ImportPath: "x/r.test", Deps: []string{ext.ImportPath}}
|
||||
pkgs[root.ImportPath] = root
|
||||
var names []string
|
||||
digest := func() string {
|
||||
d, err := packageDigest(root, testMain, pkgs, dir, []string{"digest 1"}, nil)
|
||||
d, err := packageDigest(root, testMain, pkgs, dir, []string{"digest 1"}, nil, names, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -188,30 +189,90 @@ func TestAModuleOutsideTheTreeCountsByVersionAndChecksum(t *testing.T) {
|
||||
if d3 := digest(); d3 == d1 {
|
||||
t.Errorf("a new version left the digest as it was")
|
||||
}
|
||||
// A different node list misses the cache only for a package whose tests read MESH_NODE_NAMES (ADR 0567 §15).
|
||||
names = []string{"node-names a"}
|
||||
d4 := digest()
|
||||
names = []string{"node-names a", "node-names b"}
|
||||
if d5 := digest(); d5 != d4 {
|
||||
t.Errorf("a package that does not read the node names changed with them")
|
||||
}
|
||||
os.WriteFile(filepath.Join(dir, "r.go"), []byte("package r\n// "+namesVariable+"\n"), 0o644)
|
||||
names = []string{"node-names a"}
|
||||
d6 := digest()
|
||||
names = []string{"node-names a", "node-names b"}
|
||||
if d7 := digest(); d7 == d6 {
|
||||
t.Errorf("a new node left the digest of a package that reads the node names as it was")
|
||||
}
|
||||
}
|
||||
|
||||
// Issue 1935: a package whose directory carries the reads-outside marker is never kept: its digest is one no kept verdict
|
||||
// can have (it differs on every call, though nothing changed), and the answer names it. A package without the marker
|
||||
// keeps its digest.
|
||||
func TestAPackageWithTheReadsOutsideMarkerIsNeverKept(t *testing.T) {
|
||||
// Issue 1935, 1936: a package whose reads-outside marker lists nothing it can trust is never kept: its digest is one no
|
||||
// kept verdict can have (it differs on every call, though nothing changed), and the answer names it. A marker that lists
|
||||
// what the tests read is kept: a change to a file it names changes the package's digest, a change elsewhere does not.
|
||||
func TestAPackageWithTheReadsOutsideMarkerIsKeptOnlyByWhatItLists(t *testing.T) {
|
||||
t.Parallel()
|
||||
m, tree := digestFixture(t)
|
||||
plain := digests(t, m, "unit")
|
||||
if len(plain.ReadsOutside) != 0 {
|
||||
t.Fatalf("a package carries the marker in the fixture: %v", plain.ReadsOutside)
|
||||
}
|
||||
edit(t, tree, "c/"+ReadsOutsideMarker, "the tests read ../docs\n")
|
||||
first, second := digests(t, m, "unit"), digests(t, m, "unit")
|
||||
c := "fixture/digest/c"
|
||||
if len(first.Packages[c]) != 64 || first.Packages[c] == second.Packages[c] || first.Packages[c] == plain.Packages[c] {
|
||||
t.Errorf("c carries the marker and its digest is stable: %s, %s, before %s", first.Packages[c], second.Packages[c], plain.Packages[c])
|
||||
c, a := "fixture/digest/c", "fixture/digest/a"
|
||||
if err := os.MkdirAll(filepath.Join(tree, "docs"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Join(first.ReadsOutside, ",") != c {
|
||||
t.Errorf("the answer does not name the package that reads outside: %v", first.ReadsOutside)
|
||||
edit(t, tree, "docs/one.md", "one\n")
|
||||
edit(t, tree, "other.md", "other\n")
|
||||
for name, marker := range map[string]string{
|
||||
"an empty marker": "# nothing listed\n\n",
|
||||
"a sentence": "the tests read ../docs\n",
|
||||
"a path climbing out": "../docs\n",
|
||||
"a path matching nothing": "docs/none.md\n",
|
||||
"an absolute path": "/etc/passwd\n",
|
||||
"a path that is not clean": "docs/./one.md\n",
|
||||
"a good line and a bad line": "docs\nnope/*.md\n",
|
||||
} {
|
||||
edit(t, tree, "c/"+ReadsOutsideMarker, marker)
|
||||
first, second := digests(t, m, "unit"), digests(t, m, "unit")
|
||||
if len(first.Packages[c]) != 64 || first.Packages[c] == second.Packages[c] || first.Packages[c] == plain.Packages[c] {
|
||||
t.Errorf("%s: c's digest is stable: %s, %s, before %s", name, first.Packages[c], second.Packages[c], plain.Packages[c])
|
||||
}
|
||||
if strings.Join(first.ReadsOutside, ",") != c {
|
||||
t.Errorf("%s: the answer does not name the package that reads outside: %v", name, first.ReadsOutside)
|
||||
}
|
||||
if first.Packages[a] != plain.Packages[a] || first.Packages[a] != second.Packages[a] {
|
||||
t.Errorf("%s: a has no marker and its digest moved: %s, %s, before %s", name, first.Packages[a], second.Packages[a], plain.Packages[a])
|
||||
}
|
||||
}
|
||||
a := "fixture/digest/a"
|
||||
if first.Packages[a] != plain.Packages[a] || first.Packages[a] != second.Packages[a] {
|
||||
t.Errorf("a has no marker and its digest moved: %s, %s, before %s", first.Packages[a], second.Packages[a], plain.Packages[a])
|
||||
|
||||
// A list: a glob, a directory read whole, and a path whose absence is read.
|
||||
edit(t, tree, "c/"+ReadsOutsideMarker, "# what c's tests read\ndocs/*.md\ndocs\n?gone\n")
|
||||
before := digests(t, m, "unit")
|
||||
if len(before.ReadsOutside) != 0 || len(before.Packages[c]) != 64 || before.Packages[c] == plain.Packages[c] {
|
||||
t.Fatalf("a listed marker is not kept: %v, %s (no marker %s)", before.ReadsOutside, before.Packages[c], plain.Packages[c])
|
||||
}
|
||||
if again := digests(t, m, "unit"); again.Packages[c] != before.Packages[c] {
|
||||
t.Errorf("nothing changed and c's digest moved")
|
||||
}
|
||||
for _, step := range []struct {
|
||||
file, text string
|
||||
moves bool
|
||||
}{
|
||||
{"other.md", "changed\n", false}, // outside the list
|
||||
{"d/d.go", "package d\n\n// D.\n", false}, // a package c's tests do not reach
|
||||
{"docs/one.md", "changed\n", true}, // a listed file
|
||||
{"docs/two.md", "new\n", true}, // a new file the glob matches
|
||||
{"gone", "here\n", true}, // a path whose absence was read
|
||||
{"c/note.txt", "changed\n", true}, // the package's own file that is not Go
|
||||
{"c/x_integration_test.go", "//go:build integration\n\npackage c\n", true}, // a Go file its build tags leave out (go list's IgnoredGoFiles)
|
||||
} {
|
||||
edit(t, tree, step.file, step.text)
|
||||
after := digests(t, m, "unit")
|
||||
if moved := after.Packages[c] != before.Packages[c]; moved != step.moves {
|
||||
t.Errorf("a change to %s: c's digest moved %v, want %v", step.file, moved, step.moves)
|
||||
}
|
||||
if after.Packages[a] != plain.Packages[a] {
|
||||
t.Errorf("a change to %s moved a's digest", step.file)
|
||||
}
|
||||
before = after
|
||||
}
|
||||
|
||||
// Issue 890/AC3: a package whose directory carries the needs marker is named with its needs at the integration level,
|
||||
|
||||
@@ -1,112 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A failed test keeps the lines that say why it failed, not the lines it logged first (issue 1901). The fixture is
|
||||
// the real `go test -json` of a test that logs a line and then fails with a multi-line t.Fatalf.
|
||||
func TestAFailedTestKeepsItsFatalMessageAfterItsLogLines(t *testing.T) {
|
||||
raw, err := os.ReadFile("testdata/captured/go-test-json-log-then-fatal.jsonl")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
res := &TestResult{}
|
||||
rd := newReader(res, ".")
|
||||
rd.read(string(raw))
|
||||
rd.finish()
|
||||
if len(res.Failed) != 1 {
|
||||
t.Fatalf("failures = %+v, want one", res.Failed)
|
||||
}
|
||||
f := res.Failed[0]
|
||||
for _, want := range []string{"3 subscription(s) derive to a subject no emitter owns:", "a.b.c", "d.e.f", "g.h.i"} {
|
||||
if !strings.Contains(f.Output, want) {
|
||||
t.Errorf("1901/AC1: output lacks %q: %q", want, f.Output)
|
||||
}
|
||||
}
|
||||
if want := "3 subscription(s) derive to a subject no emitter owns:"; f.Cause != want && !strings.HasSuffix(f.Cause, want) {
|
||||
t.Errorf("1901/AC1: cause = %q, want the t.Fatalf line, not the t.Logf line", f.Cause)
|
||||
}
|
||||
}
|
||||
|
||||
// A failure longer than the bound keeps its last lines, where the cause is, and says how many earlier lines it dropped.
|
||||
func TestAFailureLongerThanTheBoundKeepsItsLastLines(t *testing.T) {
|
||||
var lines []string
|
||||
for i := 0; i < 400; i++ {
|
||||
lines = append(lines, " x_test.go:1: log line "+strings.Repeat("y", 40))
|
||||
}
|
||||
lines = append(lines, " x_test.go:9: the real cause", "--- FAIL: TestX (0.00s)")
|
||||
out := clip(lines, "")
|
||||
if len(out) > outputBytes+100 {
|
||||
t.Errorf("1901/AC1: kept %d bytes, bound is %d", len(out), outputBytes)
|
||||
}
|
||||
if !strings.HasSuffix(out, "the real cause\n--- FAIL: TestX (0.00s)") || !strings.Contains(out, "\n... (") {
|
||||
t.Errorf("1901/AC1: out = %q...%q", out[:40], out[len(out)-60:])
|
||||
}
|
||||
}
|
||||
|
||||
// An older checker, which knows no "cause" field, still reads the new build module's answer, and the output it gets
|
||||
// holds at least what the older build module kept (its first lines, for an output shorter than the bound).
|
||||
func TestAnOlderCheckerReadsTheNewFailureAndGetsAtLeastWhatItKept(t *testing.T) { // 1901/AC3
|
||||
raw, err := os.ReadFile("testdata/captured/go-test-json-log-then-fatal.jsonl")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
res := &TestResult{}
|
||||
rd := newReader(res, ".")
|
||||
rd.read(string(raw))
|
||||
rd.finish()
|
||||
b, err := json.Marshal(res.Failed)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var old []struct {
|
||||
Package string `json:"package"`
|
||||
Test string `json:"test,omitempty"`
|
||||
Output string `json:"output,omitempty"`
|
||||
}
|
||||
if err := json.Unmarshal(b, &old); err != nil || len(old) != 1 {
|
||||
t.Fatalf("1901/AC3: an older checker refused the answer: %v, %d failures", err, len(old))
|
||||
}
|
||||
first := strings.SplitN(old[0].Output, "\n", 2)[0]
|
||||
if first == "" || !strings.Contains(old[0].Output, "judged against") || !strings.Contains(old[0].Output, "3 subscription(s)") {
|
||||
t.Errorf("1901/AC3: older checker's output = %q", old[0].Output)
|
||||
}
|
||||
}
|
||||
|
||||
// A long failure, whose earlier lines are left out, still starts with a real line for an older checker, which keeps only the
|
||||
// first non-empty line of the output cut to 120 characters, and not with the marker.
|
||||
func TestALongFailureStillStartsWithARealLineForAnOlderChecker(t *testing.T) { // 1901/AC3
|
||||
var lines []string
|
||||
for i := 0; i < 400; i++ {
|
||||
lines = append(lines, " x_test.go:1: log line "+strings.Repeat("y", 40))
|
||||
}
|
||||
lines = append(lines, " x_test.go:9: the real cause", "--- FAIL: TestX (0.00s)")
|
||||
oldFirst := func(out string) string { // the older checker's rule (judge.go firstLine)
|
||||
for _, l := range strings.Split(out, "\n") {
|
||||
if l = strings.TrimSpace(l); l != "" {
|
||||
if len(l) > 120 {
|
||||
l = l[:120]
|
||||
}
|
||||
return l
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
for _, cause := range []string{"the real cause", ""} {
|
||||
out := clip(lines, cause)
|
||||
got := oldFirst(out)
|
||||
if strings.HasPrefix(got, "... (") || got == "" {
|
||||
t.Errorf("1901/AC3: cause %q: an older checker would read %q", cause, got)
|
||||
}
|
||||
if cause != "" && got != cause {
|
||||
t.Errorf("1901/AC3: older checker reads %q, want the cause", got)
|
||||
}
|
||||
if !strings.HasSuffix(out, "--- FAIL: TestX (0.00s)") {
|
||||
t.Errorf("1901/AC3: the last lines are lost: %q", out[len(out)-40:])
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -42,11 +42,13 @@ type FetchResult struct {
|
||||
// head onto it, the paths the merge changes, and the tree of each module directory (a directory holding a
|
||||
// module.json) in the merge. A head that does not merge cleanly is an answer: Merge is empty, Conflicts names the
|
||||
// files, and the workspace is left at the trunk commit.
|
||||
Base string `json:"base,omitempty"`
|
||||
Merge string `json:"merge,omitempty"`
|
||||
Conflicts []string `json:"conflicts,omitempty"`
|
||||
Changed []string `json:"changed,omitempty"`
|
||||
Trees map[string]buildstep.ModuleTree `json:"trees,omitempty"`
|
||||
Base string `json:"base,omitempty"`
|
||||
Merge string `json:"merge,omitempty"`
|
||||
Conflicts []string `json:"conflicts,omitempty"`
|
||||
Changed []string `json:"changed,omitempty"`
|
||||
// Programs are the changed files that are compiled programs (issue 1577); a merge answers it, a pointer to an empty list when there are none, absent for a fetch that made no merge.
|
||||
Programs *[]buildstep.Program `json:"programs,omitempty"`
|
||||
Trees map[string]buildstep.ModuleTree `json:"trees,omitempty"`
|
||||
// Beside are the sibling repositories placed as plain directories under the workspace's beside directory, which
|
||||
// build_go_test hands to the tests as MESH_CHECK_BESIDE (ADR 0484 clause 4).
|
||||
Beside []Sibling `json:"beside,omitempty"`
|
||||
|
||||
@@ -72,7 +72,7 @@ func (m *Module) integrationHere(ctx context.Context, workspace, tree string, pa
|
||||
return nil, fmt.Errorf("the integration level's bus: %w", err)
|
||||
}
|
||||
defer stopBus()
|
||||
env := append([]string{"MESH_TEST_POSTGRES=" + postgres, "MESH_TEST_NATS=" + nats}, m.besideEnv(workspace)...)
|
||||
env := append([]string{"MESH_TEST_POSTGRES=" + postgres, "MESH_TEST_NATS=" + nats}, m.checkEnv(workspace, nil)...)
|
||||
// Compiling counts as setup: the tests are built once, here, with the addresses a TestMain may read, and the run
|
||||
// below finds them built. Whether they compile is for the run to say; its failure is not read here.
|
||||
// Only the directories the selected packages lie in compile (ADR 0536 §3); no selection is every directory.
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buildstep"
|
||||
"git.novox.be/novox/mesh-sdk/go/buildstep/conformance"
|
||||
)
|
||||
|
||||
// 1901/AC7: the module's own parse path (readGoTest, then the level's judging) keeps the failure contract on every captured
|
||||
// go test -json case of the SDK's conformance kit, and its result names the module that sent it.
|
||||
func TestTheTestStepKeepsTheFailureContract(t *testing.T) {
|
||||
conformance.Run(t, conformance.GoTest, func(o conformance.Output) []buildstep.Failure {
|
||||
res := &TestResult{Level: LevelUnit, By: buildstep.By("mesh-build-go")}
|
||||
readGoTest(res, ".", o.Stdout)
|
||||
res.judge(Bounds{Run: UnitBound})
|
||||
if res.Passed {
|
||||
t.Errorf("a failed run judged passed: %+v", res)
|
||||
}
|
||||
return res.Failed
|
||||
})
|
||||
}
|
||||
|
||||
// 1890/AC7: the module's own start sweep (endLeftRunning, as main runs it) keeps the conformance kit's left-running
|
||||
// case: a step a previous process left running is ended at once and said, whatever its age, and a step of this process
|
||||
// is kept.
|
||||
func TestTheStartSweepEndsWhatAPreviousProcessLeft(t *testing.T) {
|
||||
conformance.LeftRunning(t, func(ctx context.Context, scan buildstep.Scan, node string, before time.Time, logf func(string, ...any)) {
|
||||
(&Module{Node: node}).endLeftRunning(ctx, scan, before, logf)
|
||||
})
|
||||
}
|
||||
@@ -43,6 +43,7 @@ type levelSpec struct {
|
||||
Run string `json:"run,omitempty"`
|
||||
Setup time.Duration `json:"setup"`
|
||||
Bound time.Duration `json:"bound"`
|
||||
NodeNames []string `json:"node-names,omitempty"`
|
||||
}
|
||||
|
||||
// levelAnswer is the one line the process prints: the level's result or the digest, or why it could not run.
|
||||
@@ -221,14 +222,14 @@ func (m *Module) answer(ctx context.Context, spec levelSpec) levelAnswer {
|
||||
tree := filepath.Join(m.Work, spec.Workspace, "tree")
|
||||
switch spec.Step {
|
||||
case stepDigest:
|
||||
d, err := m.digestHere(ctx, spec.Workspace, spec.Level)
|
||||
d, err := m.digestHere(ctx, spec.Workspace, spec.Level, spec.NodeNames)
|
||||
if err != nil {
|
||||
return levelAnswer{Error: err.Error()}
|
||||
}
|
||||
return levelAnswer{Digest: &d}
|
||||
case stepUnit:
|
||||
res, err := m.runLevel(ctx, tree, levelOpts{workspace: spec.Workspace, level: LevelUnit, packages: spec.Packages, run: spec.Run,
|
||||
bounds: Bounds{Run: m.unitBound()}, env: m.besideEnv(spec.Workspace)})
|
||||
bounds: Bounds{Run: m.unitBound()}, env: m.checkEnv(spec.Workspace, spec.NodeNames)})
|
||||
if err != nil {
|
||||
return levelAnswer{Error: err.Error()}
|
||||
}
|
||||
|
||||
@@ -6,14 +6,12 @@ package main
|
||||
// step that could not run.
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buildstep"
|
||||
)
|
||||
|
||||
// The two levels (ADR 0445 §2).
|
||||
@@ -32,39 +30,14 @@ type Bounds struct {
|
||||
Setup, Run time.Duration
|
||||
}
|
||||
|
||||
// PackageResult is one package's part of a level.
|
||||
type PackageResult struct {
|
||||
Dir string `json:"dir"`
|
||||
Package string `json:"package"`
|
||||
Passed bool `json:"passed"`
|
||||
Elapsed float64 `json:"elapsed"`
|
||||
Tests int `json:"tests"`
|
||||
// NoTests is a package with no test files (at this level: a file with the other level's build tag is none).
|
||||
NoTests bool `json:"no_tests,omitempty"`
|
||||
// Skipped is how many of the package's tests skipped: a pass that skipped proved less than it says, so the checker keeps no verdict for it.
|
||||
Skipped int `json:"skipped,omitempty"`
|
||||
}
|
||||
|
||||
// TestTime is one test and the time it took.
|
||||
type TestTime struct {
|
||||
Package string `json:"package"`
|
||||
Test string `json:"test"`
|
||||
Elapsed float64 `json:"elapsed"`
|
||||
}
|
||||
|
||||
// Failure is a test that failed, or a package that failed with no test to name (a build that failed, a TestMain that
|
||||
// panicked); Output is the last lines it wrote, up to outputBytes (issue 1901), and Cause the first line of what the
|
||||
// test reported as its error (t.Error, t.Fatal), without its file and line, when go test said which lines those were.
|
||||
// **A failure that is not one test's assertion carries its real cause too** (issue 1901, its third meeting): the
|
||||
// "panic:" line (a timeout's with the tests it found running), a "fatal error:" or "signal:" line, a build's first
|
||||
// error, or in words that the test program exited, while which tests ran or after its tests passed. Test then names the
|
||||
// tests that were running when the program stopped. A "PASS" or "ok" line is never a failure's words.
|
||||
type Failure struct {
|
||||
Package string `json:"package"`
|
||||
Test string `json:"test,omitempty"`
|
||||
Output string `json:"output,omitempty"`
|
||||
Cause string `json:"cause,omitempty"`
|
||||
}
|
||||
// PackageResult, TestTime and Failure are the SDK's failure contract (buildstep, issue 1901), with the JSON names this
|
||||
// module always sent: a Failure names its test, its file, its cause (never empty, never a passing line) and its own
|
||||
// output up to 4096 bytes, cause first.
|
||||
type (
|
||||
PackageResult = buildstep.GoPackage
|
||||
TestTime = buildstep.TestTime
|
||||
Failure = buildstep.Failure
|
||||
)
|
||||
|
||||
// TestResult is what a test step passes with.
|
||||
type TestResult struct {
|
||||
@@ -98,368 +71,23 @@ type TestResult struct {
|
||||
// JudgedBy is the program that ran the level (ADR 0535 §5): "installed", or "candidate <merge commit>" under the
|
||||
// runner workspace.
|
||||
JudgedBy string `json:"judged_by,omitempty"`
|
||||
// By is the build module and the SDK it was built with (buildstep.By): the checker names it when a failure hides
|
||||
// its cause.
|
||||
By string `json:"by,omitempty"`
|
||||
}
|
||||
|
||||
// event is one line of `go test -json` (test2json, and the build events go test adds).
|
||||
type event struct {
|
||||
Action string
|
||||
Package string
|
||||
Test string
|
||||
Output string
|
||||
OutputType string // "error" for a t.Error or t.Fatal message, "error-continue" for its further lines (go test -json)
|
||||
ImportPath string
|
||||
FailedBuild string
|
||||
Elapsed float64
|
||||
}
|
||||
|
||||
// outputBytes is the most a failure keeps of its output, and outputLineBytes the most of one line. **A failure keeps its
|
||||
// last lines** (issue 1901): a test logs before it fails, so the lines that say why are at the end, and the first
|
||||
// fifteen lines kept before lost them (the verdict of group r4g kept a t.Logf line, not the t.Fatalf after it). Four
|
||||
// kilobytes holds a long assertion message and stays far under a bus message's or a verdict's bound.
|
||||
const (
|
||||
outputBytes = 4096
|
||||
outputLineBytes = 1024
|
||||
)
|
||||
|
||||
// reader collects the events of one go.mod directory's `go test -json` into a result.
|
||||
type reader struct {
|
||||
res *TestResult
|
||||
dir string
|
||||
pkgs map[string]*PackageResult
|
||||
order []string
|
||||
builds map[string][]string // build output by import path
|
||||
pkgOut map[string][]string
|
||||
testOut map[string][]string // by package + "\x00" + test
|
||||
causes map[string]string // the first error line of a test, by the same key
|
||||
times []TestTime
|
||||
failedIn map[string]bool // packages with a failed test
|
||||
// running is the tests that started and have not ended, by package, in the order they started: when a package
|
||||
// fails with no failed test, these were running when its test program stopped (issue 1901).
|
||||
running map[string][]string
|
||||
// passed says a package's test program printed "PASS" (its tests passed) before it failed; built says its test
|
||||
// program failed to build.
|
||||
passed, built map[string]bool
|
||||
}
|
||||
|
||||
func newReader(res *TestResult, dir string) *reader {
|
||||
return &reader{res: res, dir: dir, pkgs: map[string]*PackageResult{}, builds: map[string][]string{},
|
||||
pkgOut: map[string][]string{}, testOut: map[string][]string{}, causes: map[string]string{}, failedIn: map[string]bool{},
|
||||
running: map[string][]string{}, passed: map[string]bool{}, built: map[string]bool{}}
|
||||
}
|
||||
|
||||
// read takes go test's output. Lines that are not events are kept for a failure that has nothing else to say.
|
||||
func (r *reader) read(out string) (noise []string) {
|
||||
sc := bufio.NewScanner(strings.NewReader(out))
|
||||
sc.Buffer(make([]byte, 1<<20), 1<<26)
|
||||
for sc.Scan() {
|
||||
var ev event
|
||||
if err := json.Unmarshal(sc.Bytes(), &ev); err != nil {
|
||||
if l := strings.TrimSpace(sc.Text()); l != "" {
|
||||
noise = append(noise, l)
|
||||
}
|
||||
continue
|
||||
}
|
||||
r.event(ev)
|
||||
}
|
||||
return noise
|
||||
}
|
||||
|
||||
func (r *reader) pkg(name string) *PackageResult {
|
||||
p := r.pkgs[name]
|
||||
if p == nil {
|
||||
p = &PackageResult{Dir: r.dir, Package: name}
|
||||
r.pkgs[name] = p
|
||||
r.order = append(r.order, name)
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
func keep(lines []string, out string) []string {
|
||||
if t := strings.TrimRight(out, "\r\n"); t != "" && !strings.HasPrefix(t, "=== ") {
|
||||
lines = append(lines, t)
|
||||
}
|
||||
return lines
|
||||
}
|
||||
|
||||
func (r *reader) event(ev event) {
|
||||
switch {
|
||||
case ev.Package == "" && ev.ImportPath != "":
|
||||
if ev.Action == "build-output" {
|
||||
r.builds[ev.ImportPath] = keep(r.builds[ev.ImportPath], ev.Output)
|
||||
}
|
||||
case ev.Package == "":
|
||||
case ev.Test == "":
|
||||
p := r.pkg(ev.Package)
|
||||
switch ev.Action {
|
||||
case "output":
|
||||
// go test's own summary of a package ("PASS", "FAIL", "ok <pkg>", "FAIL <pkg>") is never a failure's
|
||||
// words: a "PASS" before a "FAIL <pkg>" says only that the tests passed and the program failed after them.
|
||||
switch l := strings.TrimRight(ev.Output, "\r\n"); {
|
||||
case l == "PASS":
|
||||
r.passed[ev.Package] = true
|
||||
case summaryLine(l):
|
||||
default:
|
||||
r.pkgOut[ev.Package] = keep(r.pkgOut[ev.Package], ev.Output)
|
||||
}
|
||||
case "pass":
|
||||
p.Passed, p.Elapsed = true, ev.Elapsed
|
||||
case "fail":
|
||||
p.Passed, p.Elapsed = false, ev.Elapsed
|
||||
if ev.FailedBuild != "" {
|
||||
r.built[ev.Package] = true
|
||||
r.pkgOut[ev.Package] = append(r.builds[ev.FailedBuild], r.pkgOut[ev.Package]...)
|
||||
}
|
||||
case "skip":
|
||||
p.Passed, p.NoTests = true, true
|
||||
}
|
||||
default:
|
||||
key := ev.Package + "\x00" + ev.Test
|
||||
switch ev.Action {
|
||||
case "output":
|
||||
r.testOut[key] = keep(r.testOut[key], ev.Output)
|
||||
if ev.OutputType == "error" && r.causes[key] == "" {
|
||||
r.causes[key] = errorWords(ev.Output)
|
||||
}
|
||||
case "run":
|
||||
r.running[ev.Package] = append(r.running[ev.Package], ev.Test)
|
||||
case "pass", "fail", "skip":
|
||||
r.running[ev.Package] = slices.DeleteFunc(r.running[ev.Package], func(t string) bool { return t == ev.Test })
|
||||
p := r.pkg(ev.Package)
|
||||
p.Tests++
|
||||
r.res.Tests++
|
||||
if ev.Action == "skip" {
|
||||
r.res.Skipped++
|
||||
p.Skipped++
|
||||
}
|
||||
r.times = append(r.times, TestTime{Package: ev.Package, Test: ev.Test, Elapsed: ev.Elapsed})
|
||||
if ev.Action == "fail" {
|
||||
r.failedIn[ev.Package] = true
|
||||
cause := r.causes[key]
|
||||
if race := raceCause(r.testOut[key]); race != "" && (cause == "" || strings.HasPrefix(cause, "race detected")) {
|
||||
cause = race // the race itself, not testing's "race detected during execution of test"
|
||||
} else if cause == "" {
|
||||
cause = causeOf(r.testOut[key])
|
||||
}
|
||||
r.res.Failed = append(r.res.Failed, Failure{Package: ev.Package, Test: ev.Test, Output: clip(r.testOut[key], cause), Cause: cause})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// clip joins the last lines of a failure's output that fit outputBytes, each cut to outputLineBytes, and says how many
|
||||
// earlier lines it left out. **Its first line is the failure's cause** (issue 1901/AC3, AC4), put ahead when the kept
|
||||
// lines do not start with it, or, with no cause and lines left out, the first non-empty line of the output, then the
|
||||
// marker. An older checker takes only the first non-empty line of the output, and a marker there would be less than the
|
||||
// older pair kept.
|
||||
func clip(lines []string, cause string) string {
|
||||
size, from := 0, len(lines)
|
||||
for from > 0 {
|
||||
n := len(cutLine(lines[from-1])) + 1
|
||||
if size+n > outputBytes && from < len(lines) {
|
||||
break
|
||||
}
|
||||
size += n
|
||||
from--
|
||||
}
|
||||
kept := make([]string, 0, len(lines)-from+2)
|
||||
head := strings.TrimSpace(cause)
|
||||
for i := 0; head == "" && i < from; i++ {
|
||||
head = strings.TrimSpace(lines[i])
|
||||
}
|
||||
if head != "" && (from > 0 || len(lines) == 0 || strings.TrimSpace(lines[from]) != head) {
|
||||
kept = append(kept, cutLine(head))
|
||||
}
|
||||
if from > 0 {
|
||||
kept = append(kept, fmt.Sprintf("... (%d earlier lines)", from))
|
||||
}
|
||||
for _, l := range lines[from:] {
|
||||
kept = append(kept, cutLine(l))
|
||||
}
|
||||
return strings.Join(kept, "\n")
|
||||
}
|
||||
|
||||
// summaryLine says a line is go test's summary of a package, in go's exact forms only: "PASS" or "FAIL" alone,
|
||||
// "ok \t<pkg>\t<time>" and "FAIL\t<pkg>\t<time>" (review round 1: a line of the package's own that starts "ok " is kept).
|
||||
func summaryLine(l string) bool {
|
||||
return l == "PASS" || l == "FAIL" || strings.HasPrefix(l, "ok \t") || strings.HasPrefix(l, "FAIL\t")
|
||||
}
|
||||
|
||||
// causeOf is the cause in a failure's output that is not a test's assertion (issue 1901): a race the race detector
|
||||
// reported (raceCause); else the first "panic:" line, a timeout's with the tests it names as running; else the first
|
||||
// "fatal error:" (the runtime's) or "signal:" line (a test program killed); "" when there is none. It reads every line,
|
||||
// so a long goroutine dump after the panic cannot hide it. **It matches only lines at column 0**, where go prints the
|
||||
// runtime's: a test's indented log line holding the same words is not the cause (review round 1).
|
||||
func causeOf(lines []string) string {
|
||||
if race := raceCause(lines); race != "" {
|
||||
return race
|
||||
}
|
||||
for _, prefix := range []string{"panic: ", "fatal error: ", "signal: "} {
|
||||
for i, l := range lines {
|
||||
l = strings.TrimRight(l, "\r\n") // at column 0: an indented line is a test's own log, not the runtime's
|
||||
if !strings.HasPrefix(l, prefix) {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(l, "panic: test timed out after") {
|
||||
var names []string
|
||||
for _, n := range lines[i+1:] {
|
||||
if !strings.HasPrefix(n, "\t\t") {
|
||||
if strings.TrimSpace(n) == "running tests:" {
|
||||
continue
|
||||
}
|
||||
break
|
||||
}
|
||||
names = append(names, strings.TrimSpace(n))
|
||||
}
|
||||
if len(names) > 0 {
|
||||
l += "; running tests: " + strings.Join(names, ", ")
|
||||
}
|
||||
}
|
||||
return cutLine(l)
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// raceCause is the first race the race detector reported in a failure's output, in one line (issue 1901: a race after
|
||||
// the tests' "PASS" was worded "PASS"): its access and the frame that made it, and where the goroutine that made it was
|
||||
// created, as "WARNING: DATA RACE: Write at 0x... by goroutine 10 in pkg.F.func1() (f_test.go:13); goroutine created at
|
||||
// pkg.F() (f_test.go:13)". "" when the output holds no race.
|
||||
func raceCause(lines []string) string {
|
||||
at := -1
|
||||
for i, l := range lines {
|
||||
if strings.TrimRight(l, "\r\n") == "WARNING: DATA RACE" { // at column 0, as causeOf
|
||||
at = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if at < 0 {
|
||||
return ""
|
||||
}
|
||||
// frame is the function and the file's base name and line that follow a heading line at i.
|
||||
frame := func(i int) string {
|
||||
var fn, file string
|
||||
for _, l := range lines[i+1:] {
|
||||
if l = strings.TrimSpace(l); l == "" {
|
||||
break
|
||||
}
|
||||
if fn == "" {
|
||||
fn = l
|
||||
continue
|
||||
}
|
||||
file = l
|
||||
if j := strings.LastIndexByte(file, '/'); j >= 0 {
|
||||
file = file[j+1:]
|
||||
}
|
||||
if j := strings.Index(file, " +0x"); j >= 0 {
|
||||
file = file[:j]
|
||||
}
|
||||
break
|
||||
}
|
||||
if file != "" {
|
||||
return fn + " (" + file + ")"
|
||||
}
|
||||
return fn
|
||||
}
|
||||
words := "WARNING: DATA RACE"
|
||||
for i := at + 1; i < len(lines); i++ {
|
||||
l := strings.TrimSpace(lines[i])
|
||||
if l == "" || strings.HasPrefix(l, "===") {
|
||||
continue
|
||||
}
|
||||
words += ": " + strings.TrimSuffix(l, ":")
|
||||
if f := frame(i); f != "" {
|
||||
words += " in " + f
|
||||
}
|
||||
break
|
||||
}
|
||||
for i := at + 1; i < len(lines); i++ {
|
||||
l := strings.TrimSpace(lines[i])
|
||||
if strings.HasPrefix(l, "==================") {
|
||||
break
|
||||
}
|
||||
if strings.HasPrefix(l, "Goroutine ") && strings.HasSuffix(l, " created at:") {
|
||||
if f := frame(i); f != "" {
|
||||
words += "; its goroutine created at " + f
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
return cutLine(words)
|
||||
}
|
||||
|
||||
// buildCause is the first error line of a failed build: the first line that is not go's "# <package>" heading.
|
||||
func buildCause(lines []string) string {
|
||||
for _, l := range lines {
|
||||
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "# ") {
|
||||
return cutLine(l)
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// testNames names the tests that were running, at most three, and how many more.
|
||||
func testNames(tests []string) string {
|
||||
if len(tests) <= 3 {
|
||||
return strings.Join(tests, ", ")
|
||||
}
|
||||
return fmt.Sprintf("%s and %d more", strings.Join(tests[:3], ", "), len(tests)-3)
|
||||
}
|
||||
|
||||
// packageFailure is a package that failed with no failed test to name it (issue 1901): its test program did not build,
|
||||
// panicked, timed out, was killed or exited. It names the tests that were running when the program stopped, keeps
|
||||
// their output with the package's, and says the cause.
|
||||
func (r *reader) packageFailure(name string) Failure {
|
||||
running := r.running[name]
|
||||
var lines []string
|
||||
for _, t := range running {
|
||||
lines = append(lines, r.testOut[name+"\x00"+t]...)
|
||||
}
|
||||
lines = append(lines, r.pkgOut[name]...)
|
||||
cause := causeOf(lines)
|
||||
switch {
|
||||
case cause != "":
|
||||
case r.built[name]:
|
||||
cause = buildCause(lines)
|
||||
case len(running) > 0:
|
||||
cause = "the test program exited while " + testNames(running) + " ran, with no panic: an os.Exit or a signal (go test -json reports no exit status)"
|
||||
case r.passed[name]:
|
||||
cause = "the tests passed, then the test program exited with a failure: an os.Exit or a panic in TestMain or after the tests (go test -json reports no exit status)"
|
||||
}
|
||||
if cause == "" {
|
||||
cause = "the package failed with no test to name and no panic, signal or build error in its output"
|
||||
}
|
||||
return Failure{Package: name, Test: testNames(running), Output: clip(lines, cause), Cause: cause}
|
||||
}
|
||||
|
||||
// cutLine cuts a line to outputLineBytes, on a character boundary.
|
||||
func cutLine(l string) string {
|
||||
if len(l) <= outputLineBytes {
|
||||
return l
|
||||
}
|
||||
return strings.ToValidUTF8(l[:outputLineBytes], "") + "..."
|
||||
}
|
||||
|
||||
var fileAndLine = regexp.MustCompile(`^\S+\.go:\d+: `)
|
||||
|
||||
// errorWords is the words of an error line of a test: trimmed, without the file and line go test put before them.
|
||||
func errorWords(out string) string {
|
||||
return strings.TrimSpace(fileAndLine.ReplaceAllString(strings.TrimSpace(out), ""))
|
||||
}
|
||||
|
||||
// finish moves what was read into the result: the packages in order, a failure for each failed package that has no
|
||||
// failed test to name it, and the tests' times.
|
||||
func (r *reader) finish() {
|
||||
for _, name := range r.order {
|
||||
p := r.pkgs[name]
|
||||
if !p.Passed && !r.failedIn[name] {
|
||||
r.res.Failed = append(r.res.Failed, r.packageFailure(name))
|
||||
}
|
||||
if !p.NoTests && p.Elapsed > r.res.Elapsed {
|
||||
r.res.Elapsed = p.Elapsed
|
||||
}
|
||||
r.res.Packages = append(r.res.Packages, *p)
|
||||
}
|
||||
r.res.Slowest = append(r.res.Slowest, r.times...)
|
||||
// readGoTest reads one go.mod directory's `go test -json` into res through the SDK's reader (issue 1901: one failure
|
||||
// contract for every build module, buildstep.Failure), and answers the lines that were no event and how many packages
|
||||
// it named.
|
||||
func readGoTest(res *TestResult, dir, out string) (noise []string, packages int) {
|
||||
read := buildstep.ReadGoTest(out, dir)
|
||||
res.Tests += read.Tests
|
||||
res.Skipped += read.Skipped
|
||||
res.Failed = append(res.Failed, read.Failed...)
|
||||
res.Packages = append(res.Packages, read.Packages...)
|
||||
res.Slowest = append(res.Slowest, read.Times...)
|
||||
res.Elapsed = max(res.Elapsed, read.Elapsed)
|
||||
return read.Noise, len(read.Packages)
|
||||
}
|
||||
|
||||
// judge sets Passed and Why from what was read and the bounds. A level with no failure, over no bound, passes; with no
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buildstep"
|
||||
"git.novox.be/novox/mesh-sdk/go/forgekey"
|
||||
)
|
||||
|
||||
@@ -103,8 +104,7 @@ func (m *Module) lint(ctx context.Context, run forgekey.Runner, tree, d string,
|
||||
}
|
||||
var failed []Failure
|
||||
if len(files) > 0 {
|
||||
failed = append(failed, Failure{Package: d, Test: "gofmt",
|
||||
Output: "not as gofmt writes them (run gofmt -w): " + strings.Join(files, ", ")})
|
||||
failed = append(failed, buildstep.ConsoleFailure(d, "gofmt", "not as gofmt writes them (run gofmt -w): "+strings.Join(files, ", ")))
|
||||
}
|
||||
sets := [][]string{nil}
|
||||
if len(tags) > 0 {
|
||||
@@ -123,7 +123,7 @@ func (m *Module) lint(ctx context.Context, run forgekey.Runner, tree, d string,
|
||||
return nil, fmt.Errorf("%s in %s could not be run: %s", name, d, firstLine(r.Err+"\n"+r.Stderr))
|
||||
}
|
||||
if r.Status != 0 {
|
||||
failed = append(failed, Failure{Package: d, Test: name, Output: strings.TrimSpace(r.Stderr + "\n" + r.Stdout)})
|
||||
failed = append(failed, buildstep.ConsoleFailure(d, name, r.Stderr+"\n"+r.Stdout))
|
||||
}
|
||||
}
|
||||
return failed, nil
|
||||
@@ -132,6 +132,9 @@ func (m *Module) lint(ctx context.Context, run forgekey.Runner, tree, d string,
|
||||
// lintAll lints each directory, up to m.parallel() at once like the level's tests; the failures join in directory order
|
||||
// and the error is the first directory's, so the answer does not depend on which ended first (issue 1930).
|
||||
func (m *Module) lintAll(ctx context.Context, run forgekey.Runner, tree string, dirs []string, env func(d string) []string) ([]Failure, error) {
|
||||
// The ask's phases say it (issue 1890): lint of the directories, then their tests.
|
||||
buildstep.Phase(ctx, fmt.Sprintf("unit: lint of %d directories", len(dirs)))
|
||||
defer buildstep.Phase(ctx, fmt.Sprintf("unit: tests of %d directories", len(dirs)))
|
||||
type answer struct {
|
||||
failed []Failure
|
||||
err error
|
||||
|
||||
@@ -8,7 +8,6 @@ package main
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"git.novox.be/novox/mesh-sdk/go/buildstep"
|
||||
"os"
|
||||
"os/user"
|
||||
"strings"
|
||||
@@ -76,8 +75,9 @@ func main() {
|
||||
puller := m.Puller(steps, nil)
|
||||
m.Held = puller
|
||||
if m.Node != "" {
|
||||
// A step a previous process left running is ended (issue 1890); this process's own steps start after booted.
|
||||
go buildstep.EndStaleAtStart(ctx, m.stepScan(), m.Node, booted, logf)
|
||||
// A step a previous process left running is ended (issue 1890): one that names another owner, or none and
|
||||
// started before booted.
|
||||
go m.endLeftRunning(ctx, m.stepScan(), booted, logf)
|
||||
}
|
||||
if m.Node != "" {
|
||||
// The seat's asks, pulled once Serve has the runtime's channel; a pull before it is said and tried again.
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
)
|
||||
|
||||
// Issue 1309: a test that runs a step in bubblewrap is proof only where it runs. Where a build module judges (it hands
|
||||
// the tests MESH_BUILD_NODE=1) a missing bubblewrap is a failure, since every build node declares and has it; a missing
|
||||
// bubblewrap anywhere else, or any other missing tool, is a skip that says NOT RUN.
|
||||
func TestAMissingToolSkipsExceptBubblewrapOnABuildNode(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
name string
|
||||
tool string
|
||||
onBuildNode bool
|
||||
want string
|
||||
}{
|
||||
{"bubblewrap on a desk", "bwrap", false, "skip"},
|
||||
{"bubblewrap on a build node", "bwrap", true, "fail"},
|
||||
{"another tool on a build node", "initdb", true, "skip"},
|
||||
{"another tool on a desk", "initdb", false, "skip"},
|
||||
} {
|
||||
if got := missingToolAction(c.tool, c.onBuildNode); got != c.want {
|
||||
t.Errorf("%s: %q, want %q", c.name, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The levels a build module judges hand the tests the build node mark, with the siblings they read.
|
||||
func TestEveryLevelHandsTheTestsTheBuildNodeMark(t *testing.T) {
|
||||
m := &Module{Work: t.TempDir()}
|
||||
if env := m.checkEnv("ws1", nil); !slices.Contains(env, BuildNodeEnv+"=1") || slices.ContainsFunc(env, func(e string) bool { return strings.HasPrefix(e, delivery.NodeNamesEnv) }) {
|
||||
t.Errorf("env %v lacks %s=1, or names nodes when none were asked", env, BuildNodeEnv)
|
||||
}
|
||||
// And the mesh's node names, when the ask carried them (ADR 0567 §12).
|
||||
if env := m.checkEnv("ws1", []string{"a", "b"}); !slices.Contains(env, delivery.NodeNamesEnv+"=a,b") {
|
||||
t.Errorf("env %v lacks the node names", env)
|
||||
}
|
||||
}
|
||||
|
||||
// missingToolAction is what a test does when tool is not installed: "fail" for bubblewrap where the build module that
|
||||
// judges says so (BuildNodeEnv), otherwise "skip".
|
||||
func missingToolAction(tool string, onBuildNode bool) string {
|
||||
if tool == "bwrap" && onBuildNode {
|
||||
return "fail"
|
||||
}
|
||||
return "skip"
|
||||
}
|
||||
@@ -1,125 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A failure that is not one test's assertion carries its real cause (issue 1901, its third meeting: a try of
|
||||
// mesh-controller kept only "PASS" for a package that stopped half-way). Each fixture is the real `go test -json` of a
|
||||
// throwaway package (testdata/captured/README).
|
||||
func TestAFailureThatIsNotAnAssertionCarriesItsRealCause(t *testing.T) { // 1901/AC4
|
||||
cases := []struct {
|
||||
fixture string
|
||||
test string // the test the failure names; "" for none
|
||||
cause []string // what the cause holds
|
||||
}{
|
||||
{"panicking", "TestPanics", []string{"panic: assignment to entry in nil map"}},
|
||||
{"timeout", "TestWaitsForever", []string{"panic: test timed out after 2s", "TestWaitsForever"}},
|
||||
{"deeptimeout", "TestHoldsTheSeat, TestWaitsForTheBus", []string{"panic: test timed out after 2s", "TestHoldsTheSeat", "TestWaitsForTheBus"}},
|
||||
{"buildfail", "", []string{`cannot use "seat" (untyped string constant) as int value`}},
|
||||
{"exiting", "TestExits", []string{"exited", "TestExits"}},
|
||||
{"killed", "TestKilled", []string{"signal: killed"}},
|
||||
{"passthenfail", "", []string{"panic: the bus closed after the tests: nats: connection closed"}},
|
||||
{"mainexit", "", []string{"passed", "exited"}},
|
||||
{"race", "TestRacesOnTheSeat", []string{"DATA RACE", "Write at", "example.com/race.TestRacesOnTheSeat.func1()", "p_test.go:13", "created at example.com/race.TestRacesOnTheSeat()"}},
|
||||
{"racemain", "", []string{"DATA RACE", "Write at", "example.com/racemain.TestMain.func1()", "p_test.go:15", "created at example.com/racemain.TestMain()"}},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.fixture, func(t *testing.T) {
|
||||
raw, err := os.ReadFile("testdata/captured/go-test-json-pkg-" + c.fixture + ".jsonl")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
res := &TestResult{}
|
||||
rd := newReader(res, ".")
|
||||
rd.read(string(raw))
|
||||
rd.finish()
|
||||
if len(res.Failed) != 1 {
|
||||
t.Fatalf("failures = %+v, want one", res.Failed)
|
||||
}
|
||||
f := res.Failed[0]
|
||||
if f.Test != c.test {
|
||||
t.Errorf("1901/AC4: the failure names test %q, want %q", f.Test, c.test)
|
||||
}
|
||||
for _, want := range c.cause {
|
||||
if !strings.Contains(f.Cause, want) {
|
||||
t.Errorf("1901/AC4: cause = %q, want it to hold %q", f.Cause, want)
|
||||
}
|
||||
}
|
||||
if len(f.Cause) > outputLineBytes+3 {
|
||||
t.Errorf("1901/AC4: cause holds %d bytes, over the line bound %d", len(f.Cause), outputLineBytes)
|
||||
}
|
||||
// Never a "PASS" or "ok" line as a failure's words, and the first line of the output, all an older
|
||||
// checker reads, is the cause.
|
||||
for _, l := range strings.Split(f.Output, "\n") {
|
||||
if l = strings.TrimSpace(l); l == "PASS" || strings.HasPrefix(l, "ok ") || strings.HasPrefix(l, "ok\t") {
|
||||
t.Errorf("1901/AC4: the output keeps the line %q: %q", l, f.Output)
|
||||
}
|
||||
}
|
||||
if first := strings.SplitN(f.Output, "\n", 2)[0]; strings.TrimSpace(first) != strings.TrimSpace(f.Cause) {
|
||||
t.Errorf("1901/AC4: the output starts with %q, want the cause %q", first, f.Cause)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A package that passes, and a test that fails on its own assertion, keep what they had: no cause is made up for them.
|
||||
func TestAnAssertionKeepsItsOwnCause(t *testing.T) { // 1901/AC4
|
||||
raw, err := os.ReadFile("testdata/captured/go-test-json-log-then-fatal.jsonl")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
res := &TestResult{}
|
||||
rd := newReader(res, ".")
|
||||
rd.read(string(raw))
|
||||
rd.finish()
|
||||
if len(res.Failed) != 1 || !strings.HasSuffix(res.Failed[0].Cause, "3 subscription(s) derive to a subject no emitter owns:") {
|
||||
t.Errorf("1901/AC4: failures = %+v", res.Failed)
|
||||
}
|
||||
}
|
||||
|
||||
// go prints the runtime's lines ("panic:", "fatal error:", "signal:", "WARNING: DATA RACE") at column 0: a test's own
|
||||
// log line that holds the same words is indented, and is not the cause (review round 1). The fixture is the deeptimeout
|
||||
// capture with two log lines of the first running test added: a t.Logf and its indented continuation saying "panic:".
|
||||
func TestALoggedPanicIsNotTheCause(t *testing.T) { // 1901/AC4
|
||||
raw, err := os.ReadFile("testdata/captured/go-test-json-pkg-deeptimeout.jsonl")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lines := strings.SplitAfter(string(raw), "\n")
|
||||
added := `{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" p_test.go:9: the child said:\n"}` + "\n" +
|
||||
`{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" panic: child retried and recovered\n"}` + "\n" +
|
||||
`{"Action":"output","Package":"example.com/deeptimeout","Test":"TestHoldsTheSeat","Output":" WARNING: DATA RACE\n"}` + "\n"
|
||||
out := strings.Join(lines[:10], "") + added + strings.Join(lines[10:], "")
|
||||
res := &TestResult{}
|
||||
rd := newReader(res, ".")
|
||||
rd.read(out)
|
||||
rd.finish()
|
||||
if len(res.Failed) != 1 || !strings.HasPrefix(res.Failed[0].Cause, "panic: test timed out after 2s") {
|
||||
t.Errorf("1901/AC4: failures = %+v, want the timeout's panic as the cause", res.Failed)
|
||||
}
|
||||
}
|
||||
|
||||
// A package counts its own skipped tests, so the checker withholds a kept verdict from that package alone (issue 890).
|
||||
func TestEachPackageCountsItsOwnSkippedTests(t *testing.T) {
|
||||
log := `{"Action":"run","Package":"m/a","Test":"TestA"}
|
||||
{"Action":"skip","Package":"m/a","Test":"TestA","Elapsed":0}
|
||||
{"Action":"run","Package":"m/b","Test":"TestB"}
|
||||
{"Action":"pass","Package":"m/b","Test":"TestB","Elapsed":0}
|
||||
{"Action":"pass","Package":"m/a","Elapsed":0}
|
||||
{"Action":"pass","Package":"m/b","Elapsed":0}
|
||||
`
|
||||
res := &TestResult{}
|
||||
rd := newReader(res, ".")
|
||||
rd.read(log)
|
||||
rd.finish()
|
||||
got := map[string]int{}
|
||||
for _, p := range res.Packages {
|
||||
got[p.Package] = p.Skipped
|
||||
}
|
||||
if res.Skipped != 1 || got["m/a"] != 1 || got["m/b"] != 0 {
|
||||
t.Errorf("skipped = %d, per package %v, want 1 with m/a 1 and m/b 0", res.Skipped, got)
|
||||
}
|
||||
}
|
||||
@@ -41,6 +41,9 @@ func (m *Module) merge(ctx context.Context, res *FetchResult, tree, remote, base
|
||||
}
|
||||
res.Base = mr.Base
|
||||
res.Conflicts, res.Merge, res.Changed = mr.Conflicts, mr.Merge, mr.Changed
|
||||
if mr.Merge != "" {
|
||||
res.Programs = &mr.Programs
|
||||
}
|
||||
if res.Merge == "" {
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -77,7 +77,7 @@ func TestFetchWithABaseMergesAndAnswersThePreview(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Base != baseSHA || res.Merge != mergeSHA || res.Commit != commitSHA || len(res.Conflicts) != 0 ||
|
||||
strings.Join(res.Changed, ",") != "a.go,modules/m/main.go" {
|
||||
strings.Join(res.Changed, ",") != "a.go,modules/m/main.go" || res.Programs == nil || len(*res.Programs) != 0 { // 1577: a merge says it found no program
|
||||
t.Errorf("result = %s", st.Result)
|
||||
}
|
||||
mt, ok := res.Trees["modules/m"]
|
||||
|
||||
@@ -1 +1,7 @@
|
||||
These tests read other catalogue modules' files, outside this module (issue 1935): never kept.
|
||||
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
|
||||
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
|
||||
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
|
||||
# the manifests through mesh-sdk's validator (the predefined keys), gitea's and mesh-build-node's.
|
||||
predefined.json
|
||||
modules/gitea
|
||||
modules/mesh-build-node
|
||||
|
||||
@@ -15,6 +15,8 @@ import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buildstep"
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
"git.novox.be/novox/mesh-sdk/go/forgekey"
|
||||
)
|
||||
|
||||
@@ -37,7 +39,9 @@ func (m *Module) testTool() StepTool {
|
||||
"level": map[string]any{"type": "string", "enum": []string{LevelUnit, LevelIntegration}, "description": "the level"},
|
||||
"packages": map[string]any{"type": "array", "items": map[string]any{"type": "string"},
|
||||
"description": "package patterns of each module, ./... when none"},
|
||||
"run": map[string]any{"type": "string", "description": "optional go test -run pattern: only the tests it matches run (a try's tests)"},
|
||||
"run": map[string]any{"type": "string", "description": "optional go test -run pattern: only the tests it matches run (a try's tests)"},
|
||||
delivery.NodeNamesArg: map[string]any{"type": "array", "items": map[string]any{"type": "string"},
|
||||
"description": "optional, the names of the mesh's nodes: the tests get them comma-separated in " + delivery.NodeNamesEnv + " (a check that keeps node names out of the repository reads them there)"},
|
||||
"runner": runnerProperty,
|
||||
},
|
||||
"required": []string{"workspace", "level"},
|
||||
@@ -77,12 +81,16 @@ func (m *Module) test(ctx context.Context, args map[string]any) (any, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
names, err := nodeNamesArg(args[delivery.NodeNamesArg])
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var res *TestResult
|
||||
switch {
|
||||
case runner == RunnerWorkspace && (level == LevelUnit || level == LevelIntegration):
|
||||
return m.testAsCandidate(ctx, workspace, level, packages, pattern)
|
||||
return m.testAsCandidate(ctx, workspace, level, packages, pattern, names)
|
||||
case level == LevelUnit:
|
||||
res, err = m.runLevel(ctx, tree, levelOpts{workspace: workspace, level: LevelUnit, packages: packages, run: pattern, bounds: Bounds{Run: m.unitBound()}, env: m.besideEnv(workspace)})
|
||||
res, err = m.runLevel(ctx, tree, levelOpts{workspace: workspace, level: LevelUnit, packages: packages, run: pattern, bounds: Bounds{Run: m.unitBound()}, env: m.checkEnv(workspace, names)})
|
||||
case level == LevelIntegration:
|
||||
res, err = m.integration(ctx, workspace, tree, packages, pattern)
|
||||
default:
|
||||
@@ -97,14 +105,14 @@ func (m *Module) test(ctx context.Context, args map[string]any) (any, error) {
|
||||
|
||||
// testAsCandidate runs a level as the candidate's program (ADR 0535 §1): the integration level is refused first as the
|
||||
// installed program would refuse it (no postgres), then the whole level runs in the candidate's one shut-off process.
|
||||
func (m *Module) testAsCandidate(ctx context.Context, workspace, level string, packages []string, pattern string) (*TestResult, error) {
|
||||
spec := levelSpec{Step: stepUnit, Workspace: workspace, Packages: packages, Run: pattern, Bound: m.unitBound()}
|
||||
func (m *Module) testAsCandidate(ctx context.Context, workspace, level string, packages []string, pattern string, names []string) (*TestResult, error) {
|
||||
spec := levelSpec{Step: stepUnit, Workspace: workspace, Packages: packages, Run: pattern, Bound: m.unitBound(), NodeNames: names}
|
||||
if level == LevelIntegration {
|
||||
if _, err := m.checkPostgres(ctx); err != nil {
|
||||
return nil, err // a *buildstep.Refusal: this level only
|
||||
}
|
||||
b := m.integrationBounds()
|
||||
spec = levelSpec{Step: stepIntegration, Workspace: workspace, Packages: packages, Run: pattern, Setup: b.Setup, Bound: b.Run}
|
||||
spec = levelSpec{Step: stepIntegration, Workspace: workspace, Packages: packages, Run: pattern, Setup: b.Setup, Bound: b.Run, NodeNames: names}
|
||||
}
|
||||
a, judged, err := m.asCandidate(ctx, spec)
|
||||
if err != nil {
|
||||
@@ -127,6 +135,41 @@ func (m *Module) besideEnv(workspace string) []string {
|
||||
return []string{"MESH_CHECK_BESIDE=" + dir}
|
||||
}
|
||||
|
||||
// BuildNodeEnv is the mark a build module hands the tests it runs: this is a build node, where bubblewrap is declared
|
||||
// and present, so a test that needs it fails where it is missing instead of skipping (issue 1309).
|
||||
const BuildNodeEnv = "MESH_BUILD_NODE"
|
||||
|
||||
// checkEnv is what every level hands the tests it runs: the build node mark, the siblings (besideEnv) and, when the
|
||||
// ask carried them, the mesh's node names (delivery.NodeNamesEnv, novox/hq ADR 0567 §12).
|
||||
func (m *Module) checkEnv(workspace string, names []string) []string {
|
||||
env := append([]string{BuildNodeEnv + "=1"}, m.besideEnv(workspace)...)
|
||||
if len(names) > 0 {
|
||||
env = append(env, delivery.NodeNamesEnv+"="+strings.Join(names, ","))
|
||||
}
|
||||
return env
|
||||
}
|
||||
|
||||
// nodeNamesArg reads the node-names argument: a list of names, absent when none.
|
||||
func nodeNamesArg(raw any) ([]string, error) {
|
||||
switch v := raw.(type) {
|
||||
case nil:
|
||||
return nil, nil
|
||||
case []string:
|
||||
return v, nil
|
||||
case []any:
|
||||
var out []string
|
||||
for _, e := range v {
|
||||
n, ok := e.(string)
|
||||
if !ok || n == "" || strings.ContainsAny(n, ", \n") {
|
||||
return nil, fmt.Errorf("%s is a list of node names, not %v", delivery.NodeNamesArg, e)
|
||||
}
|
||||
out = append(out, n)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
return nil, fmt.Errorf("%s is a list of node names", delivery.NodeNamesArg)
|
||||
}
|
||||
|
||||
// packagesArg reads the packages argument: a list of package patterns, as []any when it came over the bus as JSON and
|
||||
// as []string when an ask handed it in process (issue 1925).
|
||||
func packagesArg(v any) ([]string, error) {
|
||||
@@ -181,7 +224,7 @@ func (m *Module) runLevel(ctx context.Context, tree string, o levelOpts) (*TestR
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
res := &TestResult{Level: o.level, Packages: []PackageResult{}, Setup: o.setup.Seconds(), Store: o.store, Bus: o.bus}
|
||||
res := &TestResult{Level: o.level, Packages: []PackageResult{}, Setup: o.setup.Seconds(), Store: o.store, Bus: o.bus, By: buildstep.By("mesh-build-go")}
|
||||
byDir, err := packagesByDir(tree, dirs, o.packages)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -194,10 +237,10 @@ func (m *Module) runLevel(ctx context.Context, tree string, o levelOpts) (*TestR
|
||||
// The directories that hold a selected package run side by side, up to the bound, each into a result of its own;
|
||||
// the results join in directory order, so the answer does not depend on which ended first (ADR 0536 §3).
|
||||
type dirRun struct {
|
||||
res *TestResult
|
||||
rd *reader
|
||||
r runAnswer
|
||||
noise []string
|
||||
res *TestResult
|
||||
r runAnswer
|
||||
noise []string
|
||||
packages int
|
||||
}
|
||||
var todo []string
|
||||
for _, d := range dirs {
|
||||
@@ -252,9 +295,7 @@ func (m *Module) runLevel(ctx context.Context, tree string, o levelOpts) (*TestR
|
||||
dr := &runs[i]
|
||||
dr.res = &TestResult{}
|
||||
dr.r = run(ctx, append(m.goEnvFor(filepath.Join(tree, d)), o.env...), "go", args...)
|
||||
dr.rd = newReader(dr.res, d)
|
||||
dr.noise = dr.rd.read(dr.r.Stdout)
|
||||
dr.rd.finish()
|
||||
dr.noise, dr.packages = readGoTest(dr.res, d, dr.r.Stdout)
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
@@ -263,7 +304,7 @@ func (m *Module) runLevel(ctx context.Context, tree string, o levelOpts) (*TestR
|
||||
}
|
||||
for i, d := range todo {
|
||||
dr := runs[i]
|
||||
if (dr.r.Status != 0 || dr.r.Err != "") && len(dr.rd.pkgs) == 0 {
|
||||
if (dr.r.Status != 0 || dr.r.Err != "") && dr.packages == 0 {
|
||||
// go test said nothing a test said: it could not run (a go.mod it cannot read, a toolchain it cannot find).
|
||||
return nil, fmt.Errorf("go test in %s: %s", d, firstLine(dr.r.Err+"\n"+strings.Join(dr.noise, "\n")+"\n"+dr.r.Stderr))
|
||||
}
|
||||
|
||||
@@ -33,6 +33,9 @@ func needTools(t *testing.T, what string, tools ...string) {
|
||||
t.Helper()
|
||||
for _, tool := range tools {
|
||||
if _, err := exec.LookPath(tool); err != nil {
|
||||
if missingToolAction(tool, os.Getenv(BuildNodeEnv) == "1") == "fail" {
|
||||
t.Fatalf("no %s installed on a build node (%s): every build node declares it, so the proof is not skipped here", tool, what)
|
||||
}
|
||||
t.Skipf("NOT RUN: no %s installed where this check runs (%s); proven live on a build node", tool, what)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -26,14 +26,6 @@ gunzip and sha256) at origin/main 0205b8f7 on 2026-10-21: directory f8af1d94...d
|
||||
055cffc2...428e. The digests above hold the gzip bytes of go1.27.1. The first try of issue 1589 on the merge check's build node answered
|
||||
other digests; the tests therefore compare the stream always and the digest only under go1.27.1.
|
||||
|
||||
go-test-json-pkg-*.jsonl are the standard output and error of `go test -json -count=1 ./...` (go1.27.3, the timeouts with
|
||||
`-timeout 2s`), captured on 2026-10-25 for issue 1901 in throwaway one-file modules that each fail without a test's own
|
||||
assertion: panicking (a nil map write in TestPanics), timeout (TestWaitsForever sleeps past the timeout), deeptimeout (two
|
||||
parallel tests and sixty blocked goroutines past the timeout, so the goroutine dump pushes the panic line out of the last
|
||||
4 KB), buildfail (a type error in a test file), exiting (os.Exit(3) in TestExits), killed (TestKilled sends itself SIGKILL),
|
||||
passthenfail (TestMain panics in a goroutine after m.Run passed) and mainexit (TestMain exits 2 after m.Run passed: go test
|
||||
prints "PASS" and then "FAIL <pkg>").
|
||||
go-test-json-pkg-race.jsonl and go-test-json-pkg-racemain.jsonl are the same, with `go test -race -json -count=1 ./...`
|
||||
(CGO_ENABLED=1, go1.27.3, 2026-10-25, issue 1901): race (TestRacesOnTheSeat writes a package variable from itself and a
|
||||
goroutine) and racemain (TestMain does so after m.Run passed: go test prints "PASS", then the race report and "Found 1
|
||||
data race(s)", then "FAIL <pkg>").
|
||||
The go-test-json-*.jsonl captures of issue 1901 (2026-10-23 and 2026-10-25) moved to mesh-sdk's conformance kit
|
||||
(go/buildstep/conformance/captured/go, with their description) on 2026-10-26, with the reader they test: the build module
|
||||
reads go test -json through buildstep.ReadGoTest and runs the kit (kit_test.go).
|
||||
|
||||
Vendored
-13
@@ -1,13 +0,0 @@
|
||||
{"Action":"start","Package":"x"}
|
||||
{"Action":"run","Package":"x","Test":"TestThree"}
|
||||
{"Action":"output","Package":"x","Test":"TestThree","Output":"=== RUN TestThree\n","OutputType":"frame"}
|
||||
{"Action":"output","Package":"x","Test":"TestThree","Output":" a_test.go:4: judged against /var/lib/mesh-build-go-work/id/beside/mesh-catalog as cloned\n"}
|
||||
{"Action":"output","Package":"x","Test":"TestThree","Output":" a_test.go:5: 3 subscription(s) derive to a subject no emitter owns:\n","OutputType":"error"}
|
||||
{"Action":"output","Package":"x","Test":"TestThree","Output":" a.b.c\n","OutputType":"error-continue"}
|
||||
{"Action":"output","Package":"x","Test":"TestThree","Output":" d.e.f\n","OutputType":"error-continue"}
|
||||
{"Action":"output","Package":"x","Test":"TestThree","Output":" g.h.i\n","OutputType":"error-continue"}
|
||||
{"Action":"output","Package":"x","Test":"TestThree","Output":"--- FAIL: TestThree (0.00s)\n","OutputType":"frame"}
|
||||
{"Action":"fail","Package":"x","Test":"TestThree","Elapsed":0}
|
||||
{"Action":"output","Package":"x","Output":"FAIL\n","OutputType":"frame"}
|
||||
{"Action":"output","Package":"x","Output":"FAIL\tx\t0.002s\n","OutputType":"frame"}
|
||||
{"Action":"fail","Package":"x","Elapsed":0.002}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user