novox/mesh-controller
6
0
Fork 0
Code Issues Pull Requests 5 Releases Wiki Activity

Let the module graph decide what a pull request's check runs, in two layers (hq ADR 0237)

Browse Source 
Every pull request the forge announces is mapped onto the mesh's module graph by the
merge handler's rule (issue 278): touching a module — or adding one — runs the gate
(mesh/merge-gate), its judge chosen by the graph (the controller judges itself, the
node-engine by its validator); a repository of the mesh that touches none runs only its
own merge-check.sh (mesh/repo-check), a warning when it has none. Nothing is left pending:
a repository outside the mesh touching nothing is told so as a pass.

The gate moves out of the per-repository scripts into the build seat, so a script is the
repository's own tests and declares its toolchain (go or typescript). The controller's
manifest names every verb of its seat again (ADR 0132), held by a test.
This commit is contained in:
jochen
2026-10-06 22:34:56 +02:00
parent d0580a17e5
commit 14127d4878
10 changed files with 996 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/builder/check.go
+367 -93
View File
@@ -1,11 +1,13 @@
package builder
import (
"bufio"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"os"
"os/exec"
@@ -18,27 +20,40 @@ import (
"github.com/novox/mesh-controller/internal/facts"
)
// A pull request's merge check, run on the build seat (novox/hq to-be 45 §9).
// A pull request's merge check, run on the build seat (novox/hq to-be 45 §9, ADR 0237).
//
// **The build machine already has what a check needs**: the repositories, a container runtime, the
// artifact store where the controller keeps the facts snapshot, and a Go toolchain. So a check is one
// more kind of work on the build seat's queue rather than a CI the mesh would have to run beside itself.
//
// **Two layers, each its own status on the pull request** (ADR 0237 as amended, 2026-10-06):
//
// - **the gate** (`mesh/merge-gate`) runs when the change touches a module of the mesh's graph — the
// controller, which holds the graph, says which (Modules) and which directories it adds a module in
// (New). The touched manifests through `module check`; every machine of the facts snapshot composed
// with the change and validated by the node-engine's own validator; then mesh-lab's replays. The
// judge is the controller the mesh runs — or, for a change to the controller, the change's own
// controller, and for a change to the node-engine, the running controller with the change's validator
// in place of the one it vendors. The graph decides whether this runs, never the repository.
// - **the repository's own check** (`mesh/repo-check`): its merge-check.sh, its unit tests and code
// quality, run when present in the toolchain it declares (`# mesh-check-toolchain: go|typescript`
// among its first lines; go when it declares none). A repository that reaches the build seat with
// none is said as a warning: it is the mesh's, and nothing of its own is tested before it merges.
//
// One check:
//
// 1. clones the repository at the pull request's head, and beside it the repositories its check
// reads — the controller the mesh runs, the catalogue, the host — each at the ref asked;
// reads — the controller the mesh runs, the catalogue, the host, the lab — each at the ref asked;
// 2. reads the facts snapshot the controller keeps, and with it **the versions the mesh runs**: the
// store a check's tests stand on is the store's own release, and the bus the bus's;
// 3. raises a throwaway PostgreSQL and a throwaway bus of those versions, labelled with the ask so a
// kill or a crash leaves nothing behind;
// 4. builds the judge — the controller the mesh runs, or its main while the running one predates the
// merge gate — and runs the repository's own merge-check.sh, or the merge gate alone for a
// repository that declares none. **In the mesh's Go toolchain, in a container of its own**, never in
// the build machine's: a pull request is code nobody has approved yet, and the build machine holds
// the container runtime's socket; the check's container holds none, and reaches only the
// throwaway store and bus on loopback;
// 5. answers pass, warning or fail from what ran, and error — never pass — when it could not run.
// 4. runs the gate, then the repository's script — **each in a toolchain container of its own**, never
// in the build machine: a pull request is code nobody has approved yet, and the build machine holds
// the container runtime's socket; the check's container holds none, and reaches only the throwaway
// store and bus on loopback. Only the replays — mesh-lab's main, reviewed code — get the socket;
// 5. answers each layer pass, warning or fail from what ran, and error — never pass — when it could
// not run.
// CheckSpec is one check, as the controller asks it.
type CheckSpec struct {
@@ -51,18 +66,38 @@ type CheckSpec struct {
Paths []string
// Beside are the repositories cloned next to it, by the directory they are found under.
Beside map[string]Beside
// Modules are the modules of the mesh's graph the change touches, New the directories it adds a
// module in, and Manifests the manifests among them in the change's tree: the gate runs when Modules
// or New is not empty.
Modules []string
New []string
Manifests []string
// Judge is who judges the gate: "" the controller the mesh runs, "self" the change's own, "validator"
// the running one with the change's validator.
Judge string
// Toolchain is the image a check's Go runs in: the mesh's own Go toolchain, as it holds it.
Toolchain string
// Toolchains is every toolchain the mesh holds, by language, for a script that declares another.
Toolchain string
Toolchains map[string]string
}
// Gated is whether the change touches the mesh's graph, and so whether the gate runs.
func (s CheckSpec) Gated() bool { return len(s.Modules)+len(s.New) > 0 }
// ToolchainOf is the Go toolchain image among what the mesh holds, empty when it holds none.
func ToolchainOf(held map[string]string) string {
return ToolchainsOf(held)["go"]
}
// ToolchainsOf is every toolchain image the mesh holds, by language.
func ToolchainsOf(held map[string]string) map[string]string {
out := map[string]string{}
for _, chain := range toolchains {
if chain.Language == "go" {
return held[chain.Base+"/"+chain.Artifact]
if image := held[chain.Base+"/"+chain.Artifact]; image != "" {
out[chain.Language] = image
}
}
return ""
return out
}
// Beside is one repository cloned next to the one checked.
@@ -71,15 +106,24 @@ type Beside struct {
Ref string
}
// CheckVerdict is what came of one check.
// CheckVerdict is what came of one check. Verdict and Summary are the gate's; Gate and Repo each layer.
type CheckVerdict struct {
Verdict string
Summary string
Report string
Took time.Duration
Gate *Layer
Repo *Layer
}
// CheckScript is what a repository declares its merge check as: run from its root, with the
// Layer is one layer of a check, judged.
type Layer struct {
Verdict string
Summary string
Modules []string
}
// CheckScript is what a repository declares its own merge check as: run from its root, with the
// environment below.
const CheckScript = "merge-check.sh"
@@ -94,6 +138,7 @@ const (
EnvChanged = "MESH_CHECK_CHANGED"
EnvVerdict = "MESH_CHECK_VERDICT"
EnvBeside = "MESH_CHECK_BESIDE"
EnvModules = "MESH_CHECK_MODULES"
)
// CheckTimeout bounds one check; a check that runs past it is an error, not a pass.
@@ -102,6 +147,28 @@ var CheckTimeout = 45 * time.Minute
// reportLines is how much of what a check printed travels in its verdict.
const reportLines = 200
// noModule is the gate's word for a change that touches nothing of the mesh's graph: a fact, not a
// missing check, so a pass.
const noModule = "the change touches no module of the mesh's graph"
// noScript is the repository layer's word for a repository with no merge-check.sh of its own.
const noScript = "the repository declares no " + CheckScript + ": none of its own tests run before it merges"
// toolchainLine is how a merge-check.sh declares the toolchain it runs in.
var toolchainLine = regexp.MustCompile(`^#\s*mesh-check-toolchain:\s*([a-z0-9-]+)\s*$`)
// ScriptToolchain is the language a merge-check.sh declares it runs in, among its first twenty lines;
// go when it declares none.
func ScriptToolchain(script []byte) string {
lines := bufio.NewScanner(bytes.NewReader(script))
for i := 0; i < 20 && lines.Scan(); i++ {
if m := toolchainLine.FindStringSubmatch(strings.TrimSpace(lines.Text())); m != nil {
return m[1]
}
}
return "go"
}
// Check runs one merge check. An error is that it could not run; the verdict is then "error".
func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry string, forge GitCredential,
log Log) (CheckVerdict, error) {
@@ -137,13 +204,24 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
return nil
}
name := spec.Repo
if name == "" {
if name == "" || !safeName.MatchString(name) {
name = "checked"
}
say("check", "%s/%s#%d at %s", spec.Owner, spec.Repo, spec.Number, short(spec.Ref))
if err := clone(spec.Repository, spec.Ref, name); err != nil {
return CheckVerdict{}, err
}
tree := filepath.Join(root, name)
script, scriptErr := os.ReadFile(filepath.Join(tree, CheckScript))
hasScript := scriptErr == nil
gated := spec.Gated()
if !gated && !hasScript {
// Nothing to run: said, never passed silently.
v := CheckVerdict{Verdict: "pass", Summary: noModule, Took: time.Since(began),
Gate: &Layer{Verdict: "pass", Summary: noModule}, Repo: &Layer{Verdict: "warning", Summary: noScript}}
say("check", "%s; %s", noModule, noScript)
return v, nil
}
for dir, b := range spec.Beside {
if dir == name || !safeName.MatchString(dir) {
continue
@@ -204,102 +282,297 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
if spec.Toolchain == "" {
return CheckVerdict{}, errors.New("the mesh holds no Go toolchain to run a check in")
}
inToolchain := func(dir string, env []string, command ...string) []string {
in := func(image, dir string, env []string, command ...string) []string {
// As the builder itself: what a check writes into the workspace is the builder's to remove.
args := []string{"run", "--rm", "--network", "host", "--volume", workspace + ":" + workspace, "--workdir", dir,
"--user", fmt.Sprintf("%d:%d", os.Getuid(), os.Getgid()), "--env", "HOME=" + workspace}
for _, e := range env {
args = append(args, "--env", e)
}
return append(append(args, spec.Toolchain), command...)
return append(append(args, image), command...)
}
// The judge: the controller the mesh runs, or its main while the running one has no merge gate.
gate := ""
if name != "mesh-controller" {
gate, err = judge(ctx, labelled, run, root, inToolchain, say)
if err != nil {
return CheckVerdict{}, err
}
inToolchain := func(dir string, env []string, command ...string) []string {
return in(spec.Toolchain, dir, env, command...)
}
verdictFile := filepath.Join(root, "verdict.json")
env := append([]string{},
EnvFacts+"="+factsFile, EnvGate+"="+gate, EnvGateStore+"="+storeURL, EnvTestStore+"="+storeURL,
EnvTestBus+"=nats://"+bus, EnvRepository+"="+spec.Owner+"/"+spec.Repo,
EnvChanged+"="+strings.Join(spec.Paths, ","), EnvVerdict+"="+verdictFile, EnvBeside+"="+root,
"GOCACHE="+filepath.Join(workspace, "go-cache"), "GOMODCACHE="+filepath.Join(workspace, "go-modules"))
tree := filepath.Join(root, name)
var command []string
if _, err := os.Stat(filepath.Join(tree, CheckScript)); err == nil {
say("check", "running its %s in the mesh's Go toolchain", CheckScript)
command = []string{"sh", CheckScript}
} else {
if gate == "" {
return CheckVerdict{}, fmt.Errorf("%s declares no %s and there is no judge to run", name, CheckScript)
}
say("check", "it declares no %s: the merge gate alone", CheckScript)
command = []string{"sh", "-c", `"$MESH_GATE" merge-gate --facts "$MESH_FACTS" --store "$MESH_GATE_POSTGRES" ` +
`--repository "$MESH_CHECK_REPOSITORY" --tree . --changed "$MESH_CHECK_CHANGED" --json > "$MESH_CHECK_VERDICT"`}
}
cmd := exec.CommandContext(ctx, "docker", LabelledArgs("docker", inToolchain(tree, env, command...), spec.ID)...)
inItsOwnGroup(cmd)
var out tail
cmd.Stdout, cmd.Stderr = &out, &out
runErr := cmd.Run()
// running runs one container of the check, its output into the report, in a process group of its own.
running := func(args []string) error {
cmd := exec.CommandContext(ctx, "docker", LabelledArgs("docker", args, spec.ID)...)
inItsOwnGroup(cmd)
cmd.Stdout, cmd.Stderr = &out, &out
return cmd.Run()
}
// **And the replays of what the mesh runs** (to-be 45 §9, M9): mesh-lab's, from its main — reviewed
// code, so given the container runtime the resolver replay raises containers with — against the bus
// of the release the mesh runs and the change's own catalogue when the change is to the catalogue.
var replayErr error
if lab := filepath.Join(root, "mesh-lab", "replays"); runErr == nil && ctx.Err() == nil {
// The judge. Its failing to build is the change's fault when the change is the judge or its validator,
// and the check's when it is the controller the mesh runs.
gate, judgeFault, err := judgeFor(ctx, labelled, run, spec, root, tree, inToolchain, say)
if err != nil {
return CheckVerdict{}, err
}
verdictFile := filepath.Join(root, "verdict.json")
env := []string{EnvFacts + "=" + factsFile, EnvGate + "=" + gate, EnvGateStore + "=" + storeURL,
EnvTestStore + "=" + storeURL, EnvTestBus + "=nats://" + bus, EnvRepository + "=" + spec.Owner + "/" + spec.Repo,
EnvChanged + "=" + strings.Join(spec.Paths, ","), EnvBeside + "=" + root,
EnvModules + "=" + strings.Join(append(append([]string{}, spec.Modules...), spec.New...), ","),
"GOCACHE=" + filepath.Join(workspace, "go-cache"), "GOMODCACHE=" + filepath.Join(workspace, "go-modules")}
v := CheckVerdict{}
modules := append(append([]string{}, spec.Modules...), prefixed("new:", spec.New)...)
timedOut := func() bool { return errors.Is(ctx.Err(), context.DeadlineExceeded) }
// **The gate**, when the graph says the change touches it.
if !gated {
v.Gate = &Layer{Verdict: "pass", Summary: noModule}
} else {
v.Gate = &Layer{Modules: modules}
switch {
case judgeFault != "":
v.Gate.Verdict, v.Gate.Summary = "fail", judgeFault
default:
say("check", "the gate: %d module(s) of the graph touched — %s", len(modules), strings.Join(modules, ", "))
fmt.Fprintf(&out, "--- the gate: %s\n", strings.Join(modules, ", "))
v.Gate.Verdict, v.Gate.Summary = gateLayer(ctx, spec, tree, root, gate, verdictFile, env, inToolchain,
running, &out, bus, workspace, name, say)
}
if timedOut() {
v.Gate.Verdict, v.Gate.Summary = "error", fmt.Sprintf("the check ran past %s and was ended", CheckTimeout)
}
}
if ctx.Err() != nil && !timedOut() {
return v, ctx.Err()
}
// **The repository's own check**, in the toolchain it declares.
switch {
case !hasScript:
v.Repo = &Layer{Verdict: "warning", Summary: noScript}
case timedOut():
v.Repo = &Layer{Verdict: "error", Summary: fmt.Sprintf("the check ran past %s before its %s ran", CheckTimeout, CheckScript)}
default:
language := ScriptToolchain(script)
image := spec.Toolchains[language]
if language == "go" && image == "" {
image = spec.Toolchain
}
if image == "" {
v.Repo = &Layer{Verdict: "error", Summary: fmt.Sprintf("its %s runs in the %s toolchain, which the mesh does "+
"not hold", CheckScript, language)}
break
}
say("check", "running its %s in the mesh's %s toolchain", CheckScript, language)
fmt.Fprintf(&out, "--- its %s (%s toolchain)\n", CheckScript, language)
var own tail
cmd := exec.CommandContext(ctx, "docker", LabelledArgs("docker", in(image, tree, env, "sh", CheckScript), spec.ID)...)
inItsOwnGroup(cmd)
w := io.MultiWriter(&out, &own)
cmd.Stdout, cmd.Stderr = w, w
switch err := cmd.Run(); {
case timedOut():
v.Repo = &Layer{Verdict: "error", Summary: fmt.Sprintf("its %s ran past %s and was ended", CheckScript, CheckTimeout)}
case ctx.Err() != nil:
return v, ctx.Err()
case err != nil:
v.Repo = &Layer{Verdict: "fail", Summary: "its " + CheckScript + " failed: " + lastLine(own.String())}
default:
v.Repo = &Layer{Verdict: "pass", Summary: "its " + CheckScript + " passed"}
}
}
v.Verdict, v.Summary = v.Gate.Verdict, v.Gate.Summary
v.Report, v.Took = out.String(), time.Since(began)
say("check", "gate %s — %s; repository %s — %s (%s)", strings.ToUpper(v.Gate.Verdict), v.Gate.Summary,
strings.ToUpper(v.Repo.Verdict), v.Repo.Summary, v.Took.Round(time.Second))
return v, nil
}
// gateLayer runs the gate: the touched manifests through `module check`, every machine composed with the
// change, and the replays of what the mesh runs. It answers the gate's verdict and summary.
func gateLayer(ctx context.Context, spec CheckSpec, tree, root, gate, verdictFile string, env []string,
inToolchain func(string, []string, ...string) []string, running func([]string) error, out *tail, bus, workspace,
name string, say func(step, format string, args ...any)) (string, string) {
// 1. The manifests the change touches, as the judge reads them: a manifest it cannot read fails here.
var manifests []string
for _, m := range spec.Manifests {
if _, err := os.Stat(filepath.Join(tree, m)); err == nil {
manifests = append(manifests, m)
}
}
if len(manifests) > 0 {
var own tail
cmd := exec.CommandContext(ctx, "docker", LabelledArgs("docker",
inToolchain(tree, env, append([]string{gate, "module", "check"}, manifests...)...), spec.ID)...)
inItsOwnGroup(cmd)
w := io.MultiWriter(out, &own)
cmd.Stdout, cmd.Stderr = w, w
if err := cmd.Run(); err != nil {
if ctx.Err() != nil {
return "error", "the check was ended during the module check"
}
return "fail", "a manifest the change touches fails the module check: " + firstProblem(own.String())
}
}
// 2. Every machine composed with the change.
if err := running(inToolchain(tree, append(env, EnvVerdict+"="+verdictFile), "sh", "-c",
`"$MESH_GATE" merge-gate --facts "$MESH_FACTS" --store "$MESH_GATE_POSTGRES" `+
`--repository "$MESH_CHECK_REPOSITORY" --tree . --changed "$MESH_CHECK_CHANGED" --json > "$MESH_CHECK_VERDICT"`)); err != nil {
if ctx.Err() != nil {
return "error", "the check was ended during the merge gate"
}
var said struct {
Verdict string `json:"verdict"`
Summary string `json:"summary"`
}
if raw, err := os.ReadFile(verdictFile); err == nil && json.Unmarshal(raw, &said) == nil && said.Verdict == "fail" {
return "fail", said.Summary
}
// The gate could not judge: not the change's fault, and never a pass.
return "error", "the merge gate could not judge the change: " + lastLine(out.String())
}
var said struct {
Verdict string `json:"verdict"`
Summary string `json:"summary"`
}
raw, err := os.ReadFile(verdictFile)
if err != nil || json.Unmarshal(raw, &said) != nil || said.Verdict == "" {
return "error", "the merge gate said no verdict"
}
// 3. **The replays of what the mesh runs** (to-be 45 §9, M9): mesh-lab's, from its main — reviewed code,
// so given the container runtime the resolver replay raises containers with — against the bus of the
// release the mesh runs and the change's own catalogue when the change is to the catalogue.
if lab := filepath.Join(root, "mesh-lab", "replays"); ctx.Err() == nil {
if _, err := os.Stat(lab); err == nil {
catalogue := filepath.Join(root, "mesh-catalog")
if name == "mesh-catalog" {
catalogue = tree
}
say("check", "the replays of what the mesh runs, from mesh-lab")
fmt.Fprintln(&out, "--- the replays (mesh-lab replays/)")
fmt.Fprintln(out, "--- the replays (mesh-lab replays/)")
args := inToolchain(lab, []string{EnvTestBus + "=nats://" + bus, "MESH_REPLAY_CATALOGUE=" + catalogue,
"GOCACHE=" + filepath.Join(workspace, "go-cache"), "GOMODCACHE=" + filepath.Join(workspace, "go-modules")},
"go", "test", "-count=1", "./...")
// The socket goes to the replays alone, never to the change's own script above.
// The socket goes to the replays alone, never to the change's own code above.
args = append([]string{args[0], "--volume", "/var/run/docker.sock:/var/run/docker.sock"}, args[1:]...)
replays := exec.CommandContext(ctx, "docker", LabelledArgs("docker", args, spec.ID)...)
inItsOwnGroup(replays)
replays.Stdout, replays.Stderr = &out, &out
replayErr = replays.Run()
if err := running(args); err != nil {
if ctx.Err() != nil {
return "error", "the check was ended during the replays"
}
return "fail", "a replay of a core incident fails with this change: " + lastLine(out.String())
}
}
}
v := CheckVerdict{Report: out.String(), Took: time.Since(began)}
var gateSaid struct {
Verdict string `json:"verdict"`
Summary string `json:"summary"`
if said.Verdict == "pass" || said.Verdict == "warning" || said.Verdict == "fail" {
return said.Verdict, said.Summary
}
if raw, err := os.ReadFile(verdictFile); err == nil {
_ = json.Unmarshal(raw, &gateSaid)
}
switch {
case errors.Is(ctx.Err(), context.DeadlineExceeded):
v.Verdict, v.Summary = "error", fmt.Sprintf("the check ran past %s and was ended", CheckTimeout)
case ctx.Err() != nil:
return v, ctx.Err()
case runErr != nil:
v.Verdict = "fail"
v.Summary = gateSaid.Summary
if v.Summary == "" || gateSaid.Verdict != "fail" {
v.Summary = "the merge check failed: " + lastLine(out.String())
return "error", "the merge gate said " + said.Verdict
}
// judgeFor builds the judge of a check: the change's own controller (a change to the controller), the
// running controller built with the change's validator (a change to the node-engine), or the controller
// the mesh runs. It answers the judge's path; or, when the change makes its own judge unbuildable, why —
// the change's fault, a failing gate; or an error when the check cannot build a judge at all. A check
// whose gate does not run builds a judge only to hand its scripts one, and goes on without when it cannot.
func judgeFor(ctx context.Context, labelled, run Runner, spec CheckSpec, root, tree string,
inToolchain func(string, []string, ...string) []string, say func(step, format string, args ...any)) (string, string, error) {
gated := spec.Gated()
switch spec.Judge {
case "self":
bin := filepath.Join(root, "bin", "judge-of-itself")
if _, err := labelled(ctx, root, "docker", inToolchain(tree,
[]string{"CGO_ENABLED=0", "GOFLAGS=-mod=vendor", "GOPROXY=off", "GOCACHE=" + filepath.Join(filepath.Dir(root), "go-cache")},
"go", "build", "-o", bin, "./cmd/mesh-controller")...); err != nil {
return "", "the change's controller, which judges itself, does not build: " + firstLine(err.Error()), nil
}
case replayErr != nil:
v.Verdict, v.Summary = "fail", "a replay of a core incident fails with this change: "+lastLine(out.String())
default:
v.Verdict, v.Summary = "pass", "the merge check passed"
if gateSaid.Verdict == "warning" || gateSaid.Verdict == "pass" {
v.Verdict, v.Summary = gateSaid.Verdict, gateSaid.Summary
say("check", "judged by the change's own controller")
return bin, "", nil
}
bin, judgeTree, err := judge(ctx, labelled, run, root, inToolchain, say)
if err != nil {
if !gated {
say("check", "no judge for its script: %v", err)
return "", "", nil
}
return "", "", err
}
if spec.Judge != "validator" {
return bin, "", nil
}
// The node-engine's change: its validator in place of the one the judge vendors.
vendored := filepath.Join(root, judgeTree, "vendor", "github.com", "novox", "mesh-host")
for _, pkg := range []string{"validate", filepath.Join("internal", "declaration")} {
if err := replaceGoFiles(filepath.Join(tree, pkg), filepath.Join(vendored, pkg)); err != nil {
return "", "", fmt.Errorf("the change's validator could not be put in the judge: %w", err)
}
}
say("check", "%s — %s (%s)", strings.ToUpper(v.Verdict), v.Summary, v.Took.Round(time.Second))
return v, nil
withValidator := filepath.Join(root, "bin", "judge-with-this-validator")
if _, err := labelled(ctx, root, "docker", inToolchain(filepath.Join(root, judgeTree),
[]string{"CGO_ENABLED=0", "GOFLAGS=-mod=vendor", "GOPROXY=off", "GOCACHE=" + filepath.Join(filepath.Dir(root), "go-cache")},
"go", "build", "-o", withValidator, "./cmd/mesh-controller")...); err != nil {
return "", "the controller the mesh runs does not build with this change's validator: " + firstLine(err.Error()), nil
}
say("check", "judged by the controller the mesh runs, with this change's validator")
return withValidator, "", nil
}
// replaceGoFiles puts a package's Go files — never its tests — in place of another copy's.
func replaceGoFiles(from, into string) error {
old, err := filepath.Glob(filepath.Join(into, "*.go"))
if err != nil {
return err
}
for _, f := range old {
if err := os.Remove(f); err != nil {
return err
}
}
if err := os.MkdirAll(into, 0o755); err != nil {
return err
}
files, err := filepath.Glob(filepath.Join(from, "*.go"))
if err != nil {
return err
}
if len(files) == 0 {
return fmt.Errorf("%s holds no Go files", from)
}
for _, f := range files {
if strings.HasSuffix(f, "_test.go") {
continue
}
body, err := os.ReadFile(f)
if err != nil {
return err
}
if err := os.WriteFile(filepath.Join(into, filepath.Base(f)), body, 0o644); err != nil {
return err
}
}
return nil
}
// firstProblem is the first line of a module check's output that names a problem.
func firstProblem(s string) string {
for _, line := range strings.Split(s, "\n") {
line = strings.TrimSpace(line)
if line != "" && !strings.HasSuffix(line, "tool(s)") && !strings.Contains(line, ": ok") {
return line
}
}
return lastLine(s)
}
func prefixed(prefix string, items []string) []string {
out := make([]string, 0, len(items))
for _, i := range items {
out = append(out, prefix+i)
}
return out
}
func firstLine(s string) string {
line, _, _ := strings.Cut(strings.TrimSpace(s), "\n")
return line
}
// safeName is a directory a repository beside a check may be cloned under.
@@ -377,9 +650,10 @@ func dialable(ctx context.Context, address string) error {
}
// judge builds the controller that judges a change: the one the mesh runs, beside the check as
// mesh-controller — or, when that one predates the merge gate, the controller's main, said.
// mesh-controller — or, when that one predates the merge gate, the controller's main, said. It answers
// the binary and the directory it was built from.
func judge(ctx context.Context, run, plain Runner, root string, inToolchain func(string, []string, ...string) []string,
say func(step, format string, args ...any)) (string, error) {
say func(step, format string, args ...any)) (string, string, error) {
bin := filepath.Join(root, "bin", "mesh-controller")
build := func(dir string) error {
_, err := run(ctx, root, "docker", inToolchain(filepath.Join(root, dir),
@@ -394,21 +668,21 @@ func judge(ctx context.Context, run, plain Runner, root string, inToolchain func
}
if _, err := os.Stat(filepath.Join(root, "mesh-controller")); err == nil {
if err := build("mesh-controller"); err != nil {
return "", fmt.Errorf("the controller the mesh runs does not build: %w", err)
return "", "", fmt.Errorf("the controller the mesh runs does not build: %w", err)
}
if hasGate() {
say("check", "judged by the controller the mesh runs")
return bin, nil
return bin, "mesh-controller", nil
}
}
if _, err := os.Stat(filepath.Join(root, "mesh-controller-main")); err != nil {
return "", errors.New("no controller beside the check to judge it with")
return "", "", errors.New("no controller beside the check to judge it with")
}
if err := build("mesh-controller-main"); err != nil {
return "", fmt.Errorf("the controller's main does not build: %w", err)
return "", "", fmt.Errorf("the controller's main does not build: %w", err)
}
say("check", "judged by the controller's main: the one the mesh runs predates the merge gate")
return bin, nil
return bin, "mesh-controller-main", nil
}
// tail keeps the last lines written to it.
internal/builder/check_test.go
+166 -17
View File
@@ -50,6 +50,9 @@ func aCheckedRepository(t *testing.T, files map[string]string) (string, string)
}
git("init", "--quiet", "-b", "main")
for name, body := range files {
if err := os.MkdirAll(filepath.Dir(filepath.Join(dir, name)), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o755); err != nil {
t.Fatal(err)
}
@@ -88,28 +91,31 @@ func labelled(id string) []string {
func TestACheckRunsTheRepositorysOwnScriptBesideTheMeshsVersionsAndLeavesNothing(t *testing.T) {
registry := checkEnvironment(t)
// The script proves what it was given: the facts, a store that answers, a bus that answers, and
// writes the gate's verdict where it is told to.
// The script proves what it was given: the facts, a store that answers, a bus that answers — and no
// container runtime socket.
script := `set -e
test -s "$MESH_FACTS"
grep -q '"bus": "2.11.17"' "$MESH_FACTS" || grep -q '"bus":"2.11.17"' "$MESH_FACTS"
case "$MESH_TEST_POSTGRES" in postgres://*127.0.0.1:*) ;; *) echo "no store: $MESH_TEST_POSTGRES"; exit 1;; esac
case "$MESH_TEST_NATS" in nats://127.0.0.1:*) ;; *) echo "no bus: $MESH_TEST_NATS"; exit 1;; esac
test "$MESH_CHECK_REPOSITORY" = "novox/mesh-controller"
test "$MESH_CHECK_REPOSITORY" = "novox/hq"
test "$MESH_CHECK_CHANGED" = "a.go,b.go"
test ! -S /var/run/docker.sock || { echo "the check holds the container runtime's socket"; exit 1; }
echo '{"verdict":"warning","summary":"a merge rebuilds 14 module(s)"}' > "$MESH_CHECK_VERDICT"
echo checked
`
repo, head := aCheckedRepository(t, map[string]string{CheckScript: script})
id := fmt.Sprintf("check-test-%d", time.Now().UnixNano())
v, err := Check(t.Context(), Command, CheckSpec{ID: id, Repository: repo, Ref: head, Owner: "novox",
Repo: "mesh-controller", Number: 7, Paths: []string{"a.go", "b.go"}, Toolchain: goToolchain}, t.TempDir(), registry, GitCredential{}, nil)
Repo: "hq", Number: 7, Paths: []string{"a.go", "b.go"}, Toolchain: goToolchain}, t.TempDir(), registry, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if v.Verdict != "warning" || v.Summary != "a merge rebuilds 14 module(s)" || !strings.Contains(v.Report, "checked") {
t.Fatalf("the check answered %+v", v)
// Nothing of the graph touched: the gate a pass that says so, the repository's own check run.
if v.Gate == nil || v.Gate.Verdict != "pass" || v.Gate.Summary != noModule {
t.Errorf("the gate answered %+v", v.Gate)
}
if v.Repo == nil || v.Repo.Verdict != "pass" || !strings.Contains(v.Report, "checked") {
t.Fatalf("the repository's check answered %+v\n%s", v.Repo, v.Report)
}
if left := labelled(id); len(left) > 0 {
t.Errorf("the check left %d container(s) behind", len(left))
@@ -120,12 +126,20 @@ func TestAFailingCheckFailsAndOneThatCannotRunIsNeverAPass(t *testing.T) {
registry := checkEnvironment(t)
repo, head := aCheckedRepository(t, map[string]string{CheckScript: "echo 'resource \"x.service\": refused'; exit 3\n"})
v, err := Check(t.Context(), Command, CheckSpec{ID: fmt.Sprintf("check-fail-%d", time.Now().UnixNano()),
Repository: repo, Ref: head, Owner: "novox", Repo: "mesh-controller", Toolchain: goToolchain}, t.TempDir(), registry, GitCredential{}, nil)
Repository: repo, Ref: head, Owner: "novox", Repo: "hq", Toolchain: goToolchain}, t.TempDir(), registry, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if v.Verdict != "fail" || !strings.Contains(v.Summary, "refused") {
t.Fatalf("a failing script answered %+v", v)
if v.Repo == nil || v.Repo.Verdict != "fail" || !strings.Contains(v.Repo.Summary, "refused") {
t.Fatalf("a failing script answered %+v", v.Repo)
}
// A script in a toolchain the mesh does not hold: an error, never a pass.
repo, head = aCheckedRepository(t, map[string]string{CheckScript: "# mesh-check-toolchain: cobol\nexit 0\n"})
v, err = Check(t.Context(), Command, CheckSpec{ID: fmt.Sprintf("check-cobol-%d", time.Now().UnixNano()),
Repository: repo, Ref: head, Owner: "novox", Repo: "hq", Toolchain: goToolchain}, t.TempDir(), registry, GitCredential{}, nil)
if err != nil || v.Repo == nil || v.Repo.Verdict != "error" {
t.Fatalf("a script in a toolchain nobody holds answered %+v, %v", v.Repo, err)
}
// Past its bound: an error, not a pass.
@@ -134,19 +148,154 @@ func TestAFailingCheckFailsAndOneThatCannotRunIsNeverAPass(t *testing.T) {
t.Cleanup(func() { CheckTimeout = was })
repo, head = aCheckedRepository(t, map[string]string{CheckScript: "sleep 120\n"})
v, err = Check(context.Background(), Command, CheckSpec{ID: fmt.Sprintf("check-slow-%d", time.Now().UnixNano()),
Repository: repo, Ref: head, Owner: "novox", Repo: "mesh-controller", Toolchain: goToolchain}, t.TempDir(), registry, GitCredential{}, nil)
if err == nil && v.Verdict == "pass" {
t.Fatalf("a check past its bound passed: %+v", v)
}
if err == nil && v.Verdict != "error" {
t.Fatalf("a check past its bound answered %+v", v)
Repository: repo, Ref: head, Owner: "novox", Repo: "hq", Toolchain: goToolchain}, t.TempDir(), registry, GitCredential{}, nil)
if err == nil && (v.Repo == nil || v.Repo.Verdict != "error") {
t.Fatalf("a check past its bound answered %+v", v.Repo)
}
// No facts: it cannot run, and says so.
repo, head = aCheckedRepository(t, map[string]string{CheckScript: "exit 0\n"})
_, err = Check(t.Context(), Command, CheckSpec{ID: "check-nofacts", Repository: repo, Ref: head, Owner: "novox",
Repo: "mesh-controller", Toolchain: goToolchain}, t.TempDir(), "127.0.0.1:1", GitCredential{}, nil)
Repo: "hq", Toolchain: goToolchain}, t.TempDir(), "127.0.0.1:1", GitCredential{}, nil)
if err == nil || !strings.Contains(err.Error(), "facts snapshot") {
t.Fatalf("a check with no facts said %v", err)
}
}
// A repository that touches nothing of the graph and has no script of its own runs nothing, and says
// both: the gate a pass that names why, the repository a warning — never silent.
func TestACheckWithNothingToRunSaysSo(t *testing.T) {
repo, head := aCheckedRepository(t, map[string]string{"README.md": "x"})
v, err := Check(t.Context(), Command, CheckSpec{ID: "check-nothing", Repository: repo, Ref: head, Owner: "novox",
Repo: "hq"}, t.TempDir(), "", GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if v.Gate == nil || v.Gate.Verdict != "pass" || v.Repo == nil || v.Repo.Verdict != "warning" ||
!strings.Contains(v.Repo.Summary, CheckScript) {
t.Fatalf("a check with nothing to run said %+v / %+v", v.Gate, v.Repo)
}
// A gated change never takes that path: with no store to read the facts from, it cannot run.
_, err = Check(t.Context(), Command, CheckSpec{ID: "check-gated", Repository: repo, Ref: head, Owner: "novox",
Repo: "mesh-catalog", Modules: []string{"gitea"}}, t.TempDir(), "", GitCredential{}, nil)
if err == nil {
t.Fatal("a change touching a module ran nothing and was not refused")
}
}
// A script declares its toolchain among its first lines; go when it declares none.
func TestAScriptDeclaresItsToolchain(t *testing.T) {
for script, want := range map[string]string{
"#!/bin/sh\nset -eu\n": "go",
"#!/bin/sh\n# mesh-check-toolchain: typescript\nnpm test\n": "typescript",
"#!/bin/sh\n#mesh-check-toolchain:go\n": "go",
"#!/bin/sh\necho '# mesh-check-toolchain: python'\n": "go",
} {
if got := ScriptToolchain([]byte(script)); got != want {
t.Errorf("%q declares %q, read as %q", script, want, got)
}
}
held := map[string]string{"mesh-tools/build": "reg/mesh-tools-build@sha256:a", "mesh-tools-go/build": "reg/go@sha256:b"}
chains := ToolchainsOf(held)
if chains["typescript"] != "reg/mesh-tools-build@sha256:a" || chains["go"] != "reg/go@sha256:b" || ToolchainOf(held) != chains["go"] {
t.Fatalf("the toolchains held read as %v", chains)
}
if _, held := chains["python"]; held {
t.Error("a toolchain the mesh does not hold read as held")
}
}
// A node-engine change's validator is put in place of the one the judge vendors: its Go files, never its tests.
func TestTheChangesValidatorReplacesTheVendoredOne(t *testing.T) {
from, into := t.TempDir(), t.TempDir()
for name, body := range map[string]string{"v.go": "package validate // new", "v_test.go": "package validate"} {
if err := os.WriteFile(filepath.Join(from, name), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
if err := os.WriteFile(filepath.Join(into, "old.go"), []byte("package validate // old"), 0o644); err != nil {
t.Fatal(err)
}
if err := replaceGoFiles(from, into); err != nil {
t.Fatal(err)
}
got, _ := filepath.Glob(filepath.Join(into, "*.go"))
if len(got) != 1 || filepath.Base(got[0]) != "v.go" {
t.Fatalf("the vendored validator holds %v", got)
}
}
// aJudge is a controller that judges as told: `merge-gate` answers a warning, `module check` refuses a
// manifest that says it is broken. What the gate layer is tested against without building the real one.
var aJudge = map[string]string{
"go.mod": "module example.org/judge\n\ngo 1.22\n",
"cmd/mesh-controller/main.go": `package main
import (
"fmt"
"os"
"strings"
)
func main() {
switch {
case len(os.Args) == 2 && os.Args[1] == "merge-gate":
fmt.Println("usage: merge-gate --facts <file> --store <postgres>")
os.Exit(2)
case len(os.Args) > 2 && os.Args[1] == "module":
for _, m := range os.Args[3:] {
body, _ := os.ReadFile(m)
if strings.Contains(string(body), "broken") {
fmt.Println(m + ": refused, it says it is broken")
os.Exit(1)
}
fmt.Println(m + ": ok")
}
case os.Args[1] == "merge-gate":
fmt.Println(` + "`" + `{"verdict":"warning","summary":"a merge rebuilds 14 module(s)"}` + "`" + `)
}
}
`,
}
func TestTheGateRunsWhenTheGraphIsTouchedBesideTheRepositorysOwnCheck(t *testing.T) {
registry := checkEnvironment(t)
judgeRepo, judgeHead := aCheckedRepository(t, aJudge)
beside := map[string]Beside{"mesh-controller": {Repository: judgeRepo, Ref: judgeHead}}
check := func(files map[string]string, judge string) CheckVerdict {
t.Helper()
repo, head := aCheckedRepository(t, files)
id := fmt.Sprintf("check-gate-%d", time.Now().UnixNano())
v, err := Check(t.Context(), Command, CheckSpec{ID: id, Repository: repo, Ref: head, Owner: "novox",
Repo: "mesh-catalog", Number: 9, Paths: []string{"modules/gitea/index.ts"}, Beside: beside,
Modules: []string{"gitea"}, Manifests: []string{"modules/gitea/module.json"}, Judge: judge,
Toolchain: goToolchain}, t.TempDir(), registry, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if left := labelled(id); len(left) > 0 {
t.Errorf("the check left %d container(s) behind", len(left))
}
return v
}
v := check(map[string]string{"modules/gitea/module.json": `{"module":"gitea"}`, CheckScript: "echo own; exit 0\n"}, "")
if v.Gate == nil || v.Gate.Verdict != "warning" || v.Gate.Summary != "a merge rebuilds 14 module(s)" ||
strings.Join(v.Gate.Modules, ",") != "gitea" || v.Verdict != "warning" {
t.Fatalf("the gate answered %+v\n%s", v.Gate, v.Report)
}
if v.Repo == nil || v.Repo.Verdict != "pass" {
t.Fatalf("its own check answered %+v", v.Repo)
}
v = check(map[string]string{"modules/gitea/module.json": `{"module":"gitea","broken":true}`}, "")
if v.Gate.Verdict != "fail" || !strings.Contains(v.Gate.Summary, "module check") || v.Repo.Verdict != "warning" {
t.Fatalf("a manifest the judge refuses answered %+v / %+v\n%s", v.Gate, v.Repo, v.Report)
}
// A change to the controller judges itself: one that does not build fails its own gate.
v = check(map[string]string{"go.mod": "module x\n\ngo 1.22\n", "cmd/mesh-controller/main.go": "package main\nfunc main() { nope }\n",
"modules/gitea/module.json": "{}"}, "self")
if v.Gate.Verdict != "fail" || !strings.Contains(v.Gate.Summary, "does not build") {
t.Fatalf("a controller that does not build judged itself %+v", v.Gate)
}
}