Merge pull request 'Build with the node's installed Go and Node, never a pinned or downloaded toolchain, so builds match what the nodes run (issue 1572, ADR 0477)' (#403) from fix/1572-installed-toolchain into main
This commit was merged in pull request #403.
This commit is contained in:
@@ -8,19 +8,20 @@ builder uses it.
|
||||
|
||||
## The recipe
|
||||
|
||||
`cmd/mesh-build-go/recipe.go` names the one Go version the module builds with (`RecipeGo`), the postgres major
|
||||
version the integration level tests with, and the recipe's own version. The recipe version is a hash of those and of
|
||||
the nats-server library's version, read from the program's build information, so a change of any of them moves it.
|
||||
The module builds with the `go` its node has installed (`GOTOOLCHAIN=local`; ADR 0477 §1, issue 1572) and names no Go
|
||||
version. The versions of the build nodes are agreed by the indexer, which builds a language's modules only while the
|
||||
nodes that report agree (ADR 0465), and kept equal by the weekly upgrade of every node's packages (ADR 0477 §5).
|
||||
`cmd/mesh-build-go/recipe.go` holds the steps' own version (`RecipeSteps`), the postgres major version the integration
|
||||
level tests with, and the recipe version: a hash of those and of the nats-server library's version, read from the
|
||||
program's build information, never of a Go version (ADR 0477 §3).
|
||||
|
||||
## The guard and the health
|
||||
|
||||
At start, every ten minutes, before every step and at every health look, the module reads the version of Go's own
|
||||
release of the recipe's version (`go version` run with `GOTOOLCHAIN=<RecipeGo>`, ADR 0476) and writes it with the recipe version to `<node>.go` of its `toolchains` state. The probe runs with network and
|
||||
the module's Go module cache (`GOMODCACHE`): while the release is missing from it, the installed Go downloads it there,
|
||||
verified against Go's checksum database; no step fetches it. A machine where the release cannot be had, or that does not
|
||||
report the recipe's version, refuses every step, the step ending `refused` on its key in `steps` with both versions named,
|
||||
and `build_go_health` (the module's `runtime-health`) answers unhealthy with the same text, so the controller raises
|
||||
the machine's condition. Moving to a new Go is a change of `RecipeGo`.
|
||||
At start, every ten minutes, before every step and at every health look, the module reads the installed Go (`go version`
|
||||
with `GOTOOLCHAIN=local`; nothing is downloaded) and writes it with the recipe version to `<node>.go` of its
|
||||
`toolchains` state. Whatever the version, the module is healthy and `build_go_health` (the module's `runtime-health`)
|
||||
names it. Only a `go` that cannot be run refuses every step, the step ending `refused` on its key in `steps` with the
|
||||
cause named, and the health is then unhealthy with the same text, so the controller raises the machine's condition.
|
||||
|
||||
## Tools
|
||||
|
||||
@@ -33,7 +34,7 @@ step answers at once with its id and ends on `<node>.<id>` of `steps` (ADR 0469
|
||||
`build_go_fetch {repository, commit, ref?}` fetches the commit over SSH as the module's account (`mesh-forge` is the
|
||||
forge-account provision's ssh name, so the account's own ssh files and git rewrite are used) into a workspace of the
|
||||
work directory, then runs `go mod download` and `go mod verify` in every directory with a `go.mod` that does not vendor,
|
||||
with `GOTOOLCHAIN=<RecipeGo>`, `GOFLAGS=-mod=mod` and the module's cache directory. Every command runs with an
|
||||
with `GOTOOLCHAIN=local`, `GOFLAGS=-mod=mod` and the module's cache directory. Every command runs with an
|
||||
environment built from scratch (`StepRunner`): `PATH`, `HOME` and what the step passes. Workspaces older than a day
|
||||
are removed.
|
||||
|
||||
@@ -77,8 +78,9 @@ is said until a source exists.
|
||||
`build_go_digest {workspace, level}` answers in its call, for each package of the workspace that has tests at the level,
|
||||
one digest of everything those tests read (ADR 0445 §5): the files, by content and under their path in the tree, of the
|
||||
package and every package its test binary reaches (`go list -test -deps`) that lies in the module's own tree; the version
|
||||
and checksum of every module outside it; the package's own `testdata/` files; and the Go version, the level, the build tags
|
||||
and, at the integration level, the postgres major and the nats-server version. Packages of the standard library are the Go
|
||||
and checksum of every module outside it; the package's own `testdata/` files; and the level, the build tags and, at the
|
||||
integration level, the postgres major and the nats-server version. The Go version is not in it: the toolchain enters a
|
||||
fingerprint once, as the version the build nodes agree on (ADR 0477 §3), and the standard library's packages are that
|
||||
version's. A package with no tests at the level has no digest and is named in `without_tests`. A directory that vendors
|
||||
is listed and tested from its vendor directory (`-mod=vendor`).
|
||||
|
||||
@@ -88,8 +90,8 @@ is listed and tested from its vendor directory (`-mod=vendor`).
|
||||
agent's recipe (mesh-controller's `toolchain.go`, the Go entry, copied in `testdata/captured/toolchain-go-entry.txt`, and
|
||||
`builder.go`): `CGO_ENABLED=0`, `GOFLAGS=-trimpath -buildvcs=false` (with `-mod=mod`, or `-mod=vendor` for a directory
|
||||
that vendors), `go build -ldflags "-s -w -X main.builtFor=<system>" -o .mesh-build/<artifact>/<binary> ./<from>` in the
|
||||
module's directory, by the recipe's Go release (`GOTOOLCHAIN=<RecipeGo>`, found in the module cache) from the module cache the fetch filled and no network
|
||||
module's directory, by the installed go (`GOTOOLCHAIN=local`) from the module cache the fetch filled and no network
|
||||
(`GOPROXY=off`). The binary is named as the build agent names it. The output directory is packed with mesh-sdk's `Pack`,
|
||||
so the digest is the build agent's for the same tree. The step ends with the artifact, the binary, the digest, the size and
|
||||
as the build agent packs; two build nodes on the same toolchain give the same digest for the same tree (ADR 0477 §6). The step ends with the artifact, the binary, the digest, the size and
|
||||
the path of the archive, which stays in the workspace's `artifacts` directory: publishing it is the builder's, in a later
|
||||
release. An artifact that is not a Go bundle fails the step naming it.
|
||||
|
||||
@@ -145,8 +145,8 @@ func (m *Module) build(ctx context.Context, args map[string]any) (any, error) {
|
||||
return BuildResult{Artifact: art.Name, Binary: binary, Digest: digest, Size: len(archive), Path: path}, nil
|
||||
}
|
||||
|
||||
// buildEnv is the environment of the compile: the build agent's (CGO_ENABLED=0, GOFLAGS) with the recipe's Go release
|
||||
// alone, the modules the fetch put in the module's cache and no way to fetch another (GOPROXY=off), so a build never
|
||||
// buildEnv is the environment of the compile: the build agent's (CGO_ENABLED=0, GOFLAGS) with the installed go
|
||||
// alone (GOTOOLCHAIN=local), the modules the fetch put in the module's cache and no way to fetch another (GOPROXY=off), so a build never
|
||||
// reaches the network. A directory that vendors is built from its vendor directory.
|
||||
func (m *Module) buildEnv(dir string) []string {
|
||||
mod := "-mod=mod"
|
||||
@@ -156,7 +156,7 @@ func (m *Module) buildEnv(dir string) []string {
|
||||
return []string{
|
||||
"CGO_ENABLED=0",
|
||||
"GOFLAGS=" + recipeFlags + " " + mod,
|
||||
"GOTOOLCHAIN=" + RecipeGo,
|
||||
"GOTOOLCHAIN=local",
|
||||
"GOMODCACHE=" + filepath.Join(m.Cache, "gomod"),
|
||||
"GOCACHE=" + filepath.Join(m.Cache, "gobuild"),
|
||||
"GOPROXY=off",
|
||||
|
||||
@@ -120,7 +120,7 @@ func TestTheBuildCommandCarriesTodaysFlagsAndEnvironment(t *testing.T) {
|
||||
}
|
||||
}
|
||||
env := strings.Join(c.env, "\n")
|
||||
for _, want := range []string{compile[1], "GOFLAGS=" + strings.TrimPrefix(compile[2], "GOFLAGS=") + " -mod=mod", "GOTOOLCHAIN=go1.27.1", "GOPROXY=off",
|
||||
for _, want := range []string{compile[1], "GOFLAGS=" + strings.TrimPrefix(compile[2], "GOFLAGS=") + " -mod=mod", "GOTOOLCHAIN=local", "GOPROXY=off",
|
||||
"GOMODCACHE=" + filepath.Join(m.Cache, "gomod"), "GOCACHE=" + filepath.Join(m.Cache, "gobuild")} {
|
||||
if !strings.Contains(env, want) {
|
||||
t.Errorf("the environment lacks %q:\n%s", want, env)
|
||||
|
||||
@@ -20,7 +20,7 @@ import (
|
||||
)
|
||||
|
||||
// DigestVersion is the form of the digest's text: changed whenever what the digest hashes changes in kind.
|
||||
const DigestVersion = "1"
|
||||
const DigestVersion = "2" // 2: no Go version in the text; the toolchain enters a fingerprint as the agreed version (ADR 0477 §3)
|
||||
|
||||
// goPackage is the part of `go list -json` a digest reads.
|
||||
type goPackage struct {
|
||||
@@ -106,7 +106,7 @@ func (m *Module) digest(ctx context.Context, args map[string]any) (any, error) {
|
||||
return nil, fmt.Errorf("no workspace %s: it was never fetched here, or it is older than a day and was removed", workspace)
|
||||
}
|
||||
var tags []string
|
||||
header := []string{"digest " + DigestVersion, "level " + level, "go " + m.Recipe.Go}
|
||||
header := []string{"digest " + DigestVersion, "level " + level}
|
||||
switch level {
|
||||
case LevelUnit:
|
||||
case LevelIntegration:
|
||||
|
||||
@@ -37,15 +37,7 @@ func digestFixture(t *testing.T) (*Module, string) {
|
||||
case "postgres":
|
||||
return forgekey.Ran{Stdout: "postgres (PostgreSQL) " + strconv.Itoa(m.Recipe.PostgresMajor) + ".1\n"}
|
||||
}
|
||||
// The unit suite has no network: the release (ADR 0476) is not fetched here, the installed Go runs the fixture.
|
||||
local := []string{}
|
||||
for _, e := range env {
|
||||
if e == "GOTOOLCHAIN="+RecipeGo {
|
||||
e = "GOTOOLCHAIN=local"
|
||||
}
|
||||
local = append(local, e)
|
||||
}
|
||||
return real(ctx, local, name, args...)
|
||||
return real(ctx, env, name, args...)
|
||||
}
|
||||
return m, tree
|
||||
}
|
||||
@@ -131,10 +123,6 @@ func TestTheLevelTheStoreAndTheBusAreInTheDigest(t *testing.T) {
|
||||
t.Errorf("a new nats-server version left the integration digest as it was")
|
||||
}
|
||||
m.Recipe.NatsServer = "2.11.17"
|
||||
m.Recipe.Go = "go1.27.2"
|
||||
if got := digests(t, m, "unit"); got.Packages[a] == unit.Packages[a] {
|
||||
t.Errorf("a new Go version left the unit digest as it was")
|
||||
}
|
||||
}
|
||||
|
||||
// 1291/AC4: a change in a package no test of a reaches leaves a's digest as it was; d, with no tests, has none.
|
||||
|
||||
@@ -112,13 +112,13 @@ func (m *Module) goModDirs(ctx context.Context, tree string) ([]string, error) {
|
||||
return dirs, nil
|
||||
}
|
||||
|
||||
// goEnv is what a go command of a step is given beyond the base environment: the recipe's Go release (ADR 0476: the probe put it in the module's cache; a step never fetches it),
|
||||
// goEnv is what a go command of a step is given beyond the base environment: the installed go alone (GOTOOLCHAIN=local, ADR 0477 §1: no toolchain is ever downloaded),
|
||||
// modules resolved by their go.mod and go.sum, the module's own caches, the mesh's own modules fetched directly (by
|
||||
// git, through the account's rewrite of the forge's address to ssh), and nothing that asks a question. For a directory
|
||||
// that vendors, see goEnvFor.
|
||||
func (m *Module) goEnv() []string {
|
||||
return []string{
|
||||
"GOTOOLCHAIN=" + RecipeGo,
|
||||
"GOTOOLCHAIN=local",
|
||||
"GOFLAGS=-mod=mod",
|
||||
"GOMODCACHE=" + filepath.Join(m.Cache, "gomod"),
|
||||
"GOCACHE=" + filepath.Join(m.Cache, "gobuild"),
|
||||
@@ -165,8 +165,7 @@ func (m *Module) clock() time.Time {
|
||||
func (m *Module) now() time.Time { return m.clock() }
|
||||
|
||||
// causeLine is the first line of a go command's output that says something other than Go's own progress: `go:
|
||||
// downloading ...` comes first when Go fetches a toolchain or a module, and the cause of a failure on a later line
|
||||
// (issue 1560: a refusal read only "go: downloading go1.27.1 (linux/amd64)").
|
||||
// downloading ...` comes first when Go fetches a module, and the cause of a failure on a later line.
|
||||
func causeLine(s string) string {
|
||||
for _, l := range strings.Split(s, "\n") {
|
||||
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "go: downloading ") {
|
||||
|
||||
@@ -93,7 +93,7 @@ func fetchTool(t *testing.T, m *Module) func(map[string]any) (any, error) {
|
||||
var fetchArgs = map[string]any{"repository": "novox/mesh-sdk", "commit": commitSHA}
|
||||
|
||||
// 1485/AC1: git is asked through FetchCommit with the ssh remote, and go mod download and verify run in each go.mod
|
||||
// directory (not in one that vendors) with GOTOOLCHAIN=go1.27.1, GOFLAGS=-mod=mod and the module's cache paths.
|
||||
// directory (not in one that vendors) with GOTOOLCHAIN=local, GOFLAGS=-mod=mod and the module's cache paths.
|
||||
func TestFetchAsksGitForTheSshRemoteAndDownloadsEachModule(t *testing.T) {
|
||||
f := &fakeStep{gomods: "go.mod\nsub/dir/go.mod\nvend/go.mod\n", vendors: []string{"vend"}}
|
||||
m, states := fetchModule(t, f)
|
||||
@@ -128,7 +128,7 @@ func TestFetchAsksGitForTheSshRemoteAndDownloadsEachModule(t *testing.T) {
|
||||
t.Errorf("a module that vendors was downloaded: %s", c.line)
|
||||
}
|
||||
env := strings.Join(c.env, " ")
|
||||
for _, want := range []string{"GOTOOLCHAIN=go1.27.1", "GOFLAGS=-mod=mod", "GOMODCACHE=" + filepath.Join(m.Cache, "gomod"), "GOCACHE=" + filepath.Join(m.Cache, "gobuild"), "GOPRIVATE=git.novox.be"} {
|
||||
for _, want := range []string{"GOTOOLCHAIN=local", "GOFLAGS=-mod=mod", "GOMODCACHE=" + filepath.Join(m.Cache, "gomod"), "GOCACHE=" + filepath.Join(m.Cache, "gobuild"), "GOPRIVATE=git.novox.be"} {
|
||||
if !strings.Contains(env, want) {
|
||||
t.Errorf("%s: environment lacks %s: %s", c.line, want, env)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
"git.novox.be/novox/mesh-sdk/go/forgekey"
|
||||
"git.novox.be/novox/mesh-sdk/go/health"
|
||||
)
|
||||
|
||||
// toolchainValues are the GOTOOLCHAIN settings of an environment.
|
||||
func toolchainValues(env []string) []string {
|
||||
var out []string
|
||||
for _, e := range env {
|
||||
if strings.HasPrefix(e, "GOTOOLCHAIN=") {
|
||||
out = append(out, e)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// 1572/AC1 (ADR 0477 §1): every environment a step gives its go commands says GOTOOLCHAIN=local, once, and names no
|
||||
// release.
|
||||
func TestEveryStepEnvironmentSaysGotoolchainLocal(t *testing.T) {
|
||||
m := &Module{Cache: t.TempDir()}
|
||||
vend := t.TempDir()
|
||||
os.MkdirAll(filepath.Join(vend, "vendor"), 0o755)
|
||||
os.WriteFile(filepath.Join(vend, "vendor", "modules.txt"), nil, 0o644)
|
||||
plain := t.TempDir()
|
||||
envs := map[string][]string{
|
||||
"goEnv": m.goEnv(), "goEnvFor(plain)": m.goEnvFor(plain), "goEnvFor(vendor)": m.goEnvFor(vend),
|
||||
"buildEnv(plain)": m.buildEnv(plain), "buildEnv(vendor)": m.buildEnv(vend), "probeEnv": probeEnv(t.TempDir()),
|
||||
}
|
||||
for name, env := range envs {
|
||||
if got := toolchainValues(env); len(got) != 1 || got[0] != "GOTOOLCHAIN=local" {
|
||||
t.Errorf("%s: GOTOOLCHAIN settings %v, want exactly GOTOOLCHAIN=local", name, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1572/AC1: the fetch step runs no `go version` and downloads no toolchain, and each go command says GOTOOLCHAIN=local.
|
||||
func TestTheFetchStepsGoCommandsSayGotoolchainLocalAndFetchNoToolchain(t *testing.T) {
|
||||
f := &fakeStep{gomods: "go.mod\n"}
|
||||
m, states := fetchModule(t, f)
|
||||
if _, err := fetchTool(t, m)(fetchArgs); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if st := states.waitEnd(t); st.State != delivery.StepPassed {
|
||||
t.Fatalf("the step ended %s: %s", st.State, st.Reason)
|
||||
}
|
||||
goCalls := 0
|
||||
for _, c := range f.calls {
|
||||
if strings.HasSuffix(c.line, " version") || strings.Contains(c.line, " get toolchain") || strings.Contains(c.line, " toolchain") {
|
||||
t.Errorf("the fetch step ran %q", c.line)
|
||||
}
|
||||
if strings.HasPrefix(c.line, "go ") {
|
||||
goCalls++
|
||||
if got := toolchainValues(c.env); len(got) != 1 || got[0] != "GOTOOLCHAIN=local" {
|
||||
t.Errorf("%s: GOTOOLCHAIN settings %v", c.line, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
if goCalls == 0 {
|
||||
t.Error("no go command ran")
|
||||
}
|
||||
}
|
||||
|
||||
type probeSpy struct {
|
||||
env []string
|
||||
line string
|
||||
ran forgekey.Ran
|
||||
left int64
|
||||
}
|
||||
|
||||
func (p *probeSpy) run(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
|
||||
p.env, p.line = env, name+" "+strings.Join(args, " ")
|
||||
if d, ok := ctx.Deadline(); ok {
|
||||
p.left = int64(d.Sub(time.Now()))
|
||||
}
|
||||
return p.ran
|
||||
}
|
||||
|
||||
// 1572/AC1 (ADR 0477 §1): the probe runs `go version` with GOTOOLCHAIN=local and a short bound, with an empty cache:
|
||||
// it fetches nothing, so there is no long bound for a download.
|
||||
func TestTheProbeRunsGoVersionLocalWithOneShortBound(t *testing.T) {
|
||||
p := &probeSpy{ran: forgekey.Ran{Stdout: "go version go1.27.1-X:nodwarf5 linux/amd64\n"}}
|
||||
v, err := probeGo(context.Background(), p.run, "/home/x", t.TempDir())
|
||||
if err != nil || v != "go1.27.1-X:nodwarf5" {
|
||||
t.Fatalf("= %q, %v", v, err)
|
||||
}
|
||||
if got := toolchainValues(p.env); len(got) != 1 || got[0] != "GOTOOLCHAIN=local" || p.line != "go version" {
|
||||
t.Errorf("command %q, GOTOOLCHAIN settings %v", p.line, got)
|
||||
}
|
||||
if p.left > int64(ProbeTimeout) || p.left <= 0 {
|
||||
t.Errorf("the probe's bound is %v, want at most %v", p.left, int64(ProbeTimeout))
|
||||
}
|
||||
}
|
||||
|
||||
// 1572/AC2 (ADR 0477 §2): neither the recipe nor the module names a Go version: the recipe has no field for one.
|
||||
func TestTheRecipeHasNoFieldForAToolchainVersion(t *testing.T) {
|
||||
typ := reflect.TypeOf(Recipe{})
|
||||
for i := 0; i < typ.NumField(); i++ {
|
||||
if n := typ.Field(i).Name; n == "Go" || strings.Contains(n, "Toolchain") {
|
||||
t.Errorf("Recipe has the field %s", n)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1572/AC2 and AC4: whatever version the node has installed, the module stays healthy, names that version in its health,
|
||||
// and writes it to <node>.go of the toolchains state.
|
||||
func TestAnyInstalledGoIsHealthyNamedAndReported(t *testing.T) {
|
||||
for _, installed := range []string{"go1.27.1", "go1.27.2", "go1.28.0", "go1.27.1-X:nodwarf5"} {
|
||||
t.Run(installed, func(t *testing.T) {
|
||||
states := newStates()
|
||||
m := moduleWith(states, installed)
|
||||
if err := m.Check(context.Background()); err != nil {
|
||||
t.Fatalf("Check refused: %v", err)
|
||||
}
|
||||
a := m.Health(context.Background())
|
||||
if !a.Healthy || a.State != health.StateHealthy || !strings.Contains(a.Why, installed) {
|
||||
t.Errorf("health = %+v, want healthy naming %s", a, installed)
|
||||
}
|
||||
tc, ok := states.toolchain("ace.go")
|
||||
if !ok || tc.Version != installed || tc.Recipe != m.Recipe.Version() {
|
||||
t.Errorf("ace.go = %+v, %v; want version %s and recipe %s", tc, ok, installed, m.Recipe.Version())
|
||||
}
|
||||
ran := false
|
||||
run := func(context.Context, map[string]any) (any, error) { ran = true; return "done", nil }
|
||||
if _, err := m.Short(run)(nil); err != nil || !ran {
|
||||
t.Errorf("a step on go %s: ran %v, err %v", installed, ran, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 1572/AC3 (ADR 0477 §3): the recipe version is made without a toolchain version: the module's recipe is the same
|
||||
// whatever go is installed, and a toolchain version set on the recipe by any name moves nothing; it moves with the
|
||||
// steps' version.
|
||||
func TestTheRecipeVersionIgnoresTheToolchainVersion(t *testing.T) {
|
||||
base := Current()
|
||||
v := base.Version()
|
||||
for _, installed := range []string{"go1.27.1", "go1.28.0"} {
|
||||
m := moduleWith(newStates(), installed)
|
||||
m.Recipe = Current()
|
||||
if err := m.Check(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := m.Recipe.Version(); got != v {
|
||||
t.Errorf("with go %s installed the recipe version is %s, want %s", installed, got, v)
|
||||
}
|
||||
}
|
||||
r := base
|
||||
if f := reflect.ValueOf(&r).Elem().FieldByName("Go"); f.IsValid() {
|
||||
f.SetString("go9.9.9")
|
||||
if r.Version() != v {
|
||||
t.Errorf("a toolchain version on the recipe moved its version")
|
||||
}
|
||||
}
|
||||
r = base
|
||||
r.Steps += "x"
|
||||
if r.Version() == v {
|
||||
t.Errorf("the steps' version did not move the recipe version")
|
||||
}
|
||||
}
|
||||
@@ -1,8 +1,8 @@
|
||||
// mesh-build-go: the Go build module (novox/hq ADR 0405, ADR 0466 §1, ADR 0469, ADR 0470; design 47 R4c-3, issue
|
||||
// 1483). A Go tools bundle the node's runtime launches as the module's own account (runs-as). It names the one Go
|
||||
// toolchain version it builds with, writes the version it finds to <node>.go of its `toolchains` state at start and
|
||||
// before every step, and refuses every step on a node that has another: its health is then unhealthy, so the
|
||||
// controller raises the node's condition. What it says, it says on stderr; stdout is the MCP channel.
|
||||
// 1483). A Go tools bundle the node's runtime launches as the module's own account (runs-as). It builds with the
|
||||
// go its node has installed (GOTOOLCHAIN=local, ADR 0477 §1; issue 1572), names no Go version, writes the version it
|
||||
// finds to <node>.go of its `toolchains` state at start and before every step, and refuses a step only when go
|
||||
// cannot be run: its health is then unhealthy, so the controller raises the node's condition. What it says, it says on stderr; stdout is the MCP channel.
|
||||
package main
|
||||
|
||||
import (
|
||||
|
||||
@@ -45,14 +45,21 @@ type Module struct {
|
||||
StartBus func(ctx context.Context, dir string) (url string, stop func(), err error)
|
||||
}
|
||||
|
||||
func (m *Module) guard() buildstep.Guard {
|
||||
return buildstep.Guard{Language: "go", Want: m.Recipe.Go, Recipe: m.Recipe.Version(), Probe: m.Probe, Node: m.Node, Put: m.reportingPut()}
|
||||
// guard is the SDK guard for the version the probe just read: the module names no Go version (ADR 0477 §2), so the
|
||||
// version wanted is the one installed and the guard refuses only when go cannot be run at all, writing the installed
|
||||
// version to <node>.go of the toolchains state every time.
|
||||
func (m *Module) guard(installed string, probeErr error) buildstep.Guard {
|
||||
return buildstep.Guard{Language: "go", Want: installed, Recipe: m.Recipe.Version(), Node: m.Node, Put: m.reportingPut(),
|
||||
Probe: func(context.Context) (string, error) { return installed, probeErr }}
|
||||
}
|
||||
|
||||
// Check reads the installed toolchain, writes <node>.go of the toolchains state and answers nil, or the refusal
|
||||
// (buildstep.Guard.Check). It runs at start, before every step, at every health look and every CheckRound.
|
||||
// Check reads the installed toolchain, writes <node>.go of the toolchains state and answers nil, or the refusal when
|
||||
// go cannot be run (buildstep.Guard.Check). A version other than another node's is not a refusal: the indexer waits
|
||||
// while build nodes disagree (ADR 0477 §2). It runs at start, before every step, at every health look and every
|
||||
// CheckRound.
|
||||
func (m *Module) Check(ctx context.Context) error {
|
||||
return m.guard().Check(ctx)
|
||||
installed, err := m.Probe(ctx)
|
||||
return m.guard(installed, err).Check(ctx)
|
||||
}
|
||||
|
||||
// Start checks once, as the module starts, and then every CheckRound until ctx ends.
|
||||
@@ -70,22 +77,20 @@ func (m *Module) Start(ctx context.Context) {
|
||||
}
|
||||
}
|
||||
|
||||
// Health is the module's health (the runtime-health tool, ADR 0366): unhealthy while the toolchain installed is not
|
||||
// the recipe's, saying both versions, so the controller raises the node's condition. It looks again each time it is
|
||||
// Health is the module's health (the runtime-health tool, ADR 0366): unhealthy only while go cannot be run, saying why, so the
|
||||
// controller raises the node's condition; healthy it names the installed version (ADR 0477 §2). It looks again each time it is
|
||||
// asked.
|
||||
func (m *Module) Health(ctx context.Context) health.Answer {
|
||||
err := m.Check(ctx)
|
||||
switch {
|
||||
case err == nil:
|
||||
return health.Healthy(fmt.Sprintf("%s installed, as the recipe %s builds with", m.Recipe.Go, m.Recipe.Version()))
|
||||
default:
|
||||
installed, perr := m.Probe(ctx)
|
||||
if err := m.guard(installed, perr).Check(ctx); err != nil {
|
||||
return health.Unhealthy(err.Error())
|
||||
}
|
||||
return health.Healthy(fmt.Sprintf("%s installed, the version the module builds with (recipe %s)", installed, m.Recipe.Version()))
|
||||
}
|
||||
|
||||
// Long is a tool that runs a long step (ADR 0469 §2): it answers at once with the step's id, and the step ends on
|
||||
// its key in the steps state. The guard runs inside the step, before run, so a node that refuses ends every step
|
||||
// refused with both versions named.
|
||||
// its key in the steps state. The guard runs inside the step, before run, so a node whose go cannot be run ends every step
|
||||
// refused with the cause named.
|
||||
func (m *Module) Long(tool string, run func(ctx context.Context, args map[string]any) (any, error)) func(map[string]any) (any, error) {
|
||||
return func(args map[string]any) (any, error) {
|
||||
id, err := buildstep.Start(m.Ctx, m.Steps, m.Node, tool, func(ctx context.Context) (any, error) {
|
||||
@@ -118,7 +123,7 @@ func (m *Module) Status(ctx context.Context) map[string]any {
|
||||
err := m.Check(ctx)
|
||||
installed, _ := m.Probe(ctx)
|
||||
out := map[string]any{
|
||||
"node": m.Node, "recipe": m.Recipe.Version(), "go": m.Recipe.Go, "installed": installed,
|
||||
"node": m.Node, "recipe": m.Recipe.Version(), "installed": installed,
|
||||
"postgres-major": m.Recipe.PostgresMajor, "nats-server": m.Recipe.NatsServer, "refusing": err != nil,
|
||||
}
|
||||
if err != nil {
|
||||
|
||||
@@ -3,6 +3,7 @@ package main
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
@@ -71,7 +72,7 @@ func (s *keptStates) waitEnd(t *testing.T) delivery.Step {
|
||||
}
|
||||
|
||||
func moduleWith(states *keptStates, installed string) *Module {
|
||||
return &Module{Node: "ace", Recipe: Recipe{Steps: "1", Go: "go1.27.1", PostgresMajor: 18, NatsServer: "2.11.17"},
|
||||
return &Module{Node: "ace", Recipe: Recipe{Steps: "1", PostgresMajor: 18, NatsServer: "2.11.17"},
|
||||
Probe: func(context.Context) (string, error) { return installed, nil },
|
||||
Toolchains: states.putToolchain, Steps: states.putStep, Ctx: context.Background()}
|
||||
}
|
||||
@@ -99,11 +100,14 @@ func TestStartWritesTheToolchainAndHealthIsHealthy(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// 1483/AC3: a probe answering go1.27.2: every step tool's step ends refused naming both versions, ace.go holds
|
||||
// go1.27.2, and the health tool answers unhealthy naming both.
|
||||
func TestAnotherGoRefusesEveryStepAndHealthIsUnhealthy(t *testing.T) {
|
||||
// 1572/AC2 (ADR 0477 §2): a module whose go cannot be run refuses every step tool, naming the cause, and its health is
|
||||
// unhealthy; no step runs.
|
||||
func TestAGoThatCannotBeRunRefusesEveryStepAndHealthIsUnhealthy(t *testing.T) {
|
||||
states := newStates()
|
||||
m := moduleWith(states, "go1.27.2")
|
||||
m := moduleWith(states, "")
|
||||
m.Probe = func(context.Context) (string, error) {
|
||||
return "", errors.New("exec: go: executable file not found in $PATH")
|
||||
}
|
||||
ran := false
|
||||
run := func(context.Context, map[string]any) (any, error) { ran = true; return "done", nil }
|
||||
steps := append([]StepTool{
|
||||
@@ -123,30 +127,26 @@ func TestAnotherGoRefusesEveryStepAndHealthIsUnhealthy(t *testing.T) {
|
||||
if out.(map[string]any)["step"] == "" {
|
||||
t.Fatalf("%s answered no step id: %v", tool.Name, out)
|
||||
}
|
||||
st := states.waitEnd(t)
|
||||
if st.State != delivery.StepRefused {
|
||||
if st := states.waitEnd(t); st.State != delivery.StepRefused {
|
||||
t.Errorf("%s: the step ended %s, want refused", tool.Name, st.State)
|
||||
} else {
|
||||
reason = st.Reason
|
||||
}
|
||||
reason = st.Reason
|
||||
} else {
|
||||
if err == nil {
|
||||
t.Fatalf("%s: answered %v, want a refusal", tool.Name, out)
|
||||
}
|
||||
reason = err.Error()
|
||||
}
|
||||
if !strings.Contains(reason, "go1.27.2") || !strings.Contains(reason, "go1.27.1") {
|
||||
t.Errorf("%s: the refusal %q does not name both versions", tool.Name, reason)
|
||||
if !strings.Contains(reason, "executable file not found") {
|
||||
t.Errorf("%s: the refusal %q does not name the cause", tool.Name, reason)
|
||||
}
|
||||
}
|
||||
if ran {
|
||||
t.Errorf("a step ran on a node that refuses")
|
||||
t.Errorf("a step ran on a node whose go cannot be run")
|
||||
}
|
||||
if tc, _ := states.toolchain("ace.go"); tc.Version != "go1.27.2" {
|
||||
t.Errorf("ace.go holds %q, want the installed go1.27.2", tc.Version)
|
||||
}
|
||||
a := m.Health(context.Background())
|
||||
if a.State != health.StateUnhealthy || !strings.Contains(a.Why, "go1.27.2") || !strings.Contains(a.Why, "go1.27.1") {
|
||||
t.Errorf("health = %+v, want unhealthy naming both versions", a)
|
||||
if a := m.Health(context.Background()); a.State != health.StateUnhealthy || !strings.Contains(a.Why, "executable file not found") {
|
||||
t.Errorf("health = %+v, want unhealthy naming the cause", a)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -159,19 +159,18 @@ func isLong(steps []StepTool, name string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// 1483/AC4: recipes differing only in the postgres major or the nats-server version have different recipe versions;
|
||||
// equal recipes have the same one.
|
||||
// 1483/AC4, 1572/AC3: recipes differing in the steps' version, the postgres major or the nats-server version have
|
||||
// different recipe versions; equal recipes have the same one.
|
||||
func TestRecipeVersionMovesWithEachOfItsValues(t *testing.T) {
|
||||
base := Recipe{Steps: "1", Go: "go1.27.1", PostgresMajor: 18, NatsServer: "2.11.17"}
|
||||
if base.Version() != (Recipe{Steps: "1", Go: "go1.27.1", PostgresMajor: 18, NatsServer: "2.11.17"}).Version() {
|
||||
base := Recipe{Steps: "1", PostgresMajor: 18, NatsServer: "2.11.17"}
|
||||
if base.Version() != (Recipe{Steps: "1", PostgresMajor: 18, NatsServer: "2.11.17"}).Version() {
|
||||
t.Errorf("one recipe has two versions")
|
||||
}
|
||||
seen := map[string]string{base.Version(): "base"}
|
||||
for name, r := range map[string]Recipe{
|
||||
"postgres major": {Steps: "1", Go: "go1.27.1", PostgresMajor: 19, NatsServer: "2.11.17"},
|
||||
"nats-server": {Steps: "1", Go: "go1.27.1", PostgresMajor: 18, NatsServer: "2.11.18"},
|
||||
"go": {Steps: "1", Go: "go1.27.2", PostgresMajor: 18, NatsServer: "2.11.17"},
|
||||
"steps": {Steps: "2", Go: "go1.27.1", PostgresMajor: 18, NatsServer: "2.11.17"},
|
||||
"postgres major": {Steps: "1", PostgresMajor: 19, NatsServer: "2.11.17"},
|
||||
"nats-server": {Steps: "1", PostgresMajor: 18, NatsServer: "2.11.18"},
|
||||
"steps": {Steps: "2", PostgresMajor: 18, NatsServer: "2.11.17"},
|
||||
} {
|
||||
v := r.Version()
|
||||
if was, dup := seen[v]; dup {
|
||||
@@ -181,9 +180,9 @@ func TestRecipeVersionMovesWithEachOfItsValues(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheRecipeNamesGo1271AndPostgres18AndTheLibrarysNatsVersion(t *testing.T) {
|
||||
func TestTheRecipeNamesPostgres18AndTheLibrarysNatsVersionAndNoGo(t *testing.T) {
|
||||
r := Current()
|
||||
if r.Go != "go1.27.1" || r.PostgresMajor != 18 {
|
||||
if r.PostgresMajor != 18 {
|
||||
t.Errorf("recipe = %+v", r)
|
||||
}
|
||||
if r.NatsServer == "" || strings.HasPrefix(r.NatsServer, "v") {
|
||||
|
||||
@@ -10,52 +10,34 @@ import (
|
||||
"git.novox.be/novox/mesh-sdk/go/forgekey"
|
||||
)
|
||||
|
||||
// ProbeTimeout bounds `go version` when the release is in the cache; FetchTimeout bounds it while the release is
|
||||
// missing and the installed Go downloads it (about seventy megabytes).
|
||||
const (
|
||||
ProbeTimeout = 15 * time.Second
|
||||
FetchTimeout = 5 * time.Minute
|
||||
)
|
||||
// ProbeTimeout bounds `go version`: it reads the installed go and fetches nothing (ADR 0477 §1).
|
||||
const ProbeTimeout = 15 * time.Second
|
||||
|
||||
// ProbeGo is the probe of the guard (novox/hq ADR 0476 §2, §3; issue 1560): it runs `go version` with
|
||||
// GOTOOLCHAIN=RecipeGo, so it reads the version of Go's own release of the recipe's version, which is what the
|
||||
// module builds with, and the third word of its answer is that version. It runs with network and the module's own
|
||||
// Go module cache (cache/gomod): while the release is missing from it, the installed Go downloads it there, verified
|
||||
// against Go's checksum database. No step fetches it; a step runs behind this probe and refuses when it is missing.
|
||||
// ProbeGo is the probe of the guard (novox/hq ADR 0477 §1, §2; issue 1572): it runs `go version` with
|
||||
// GOTOOLCHAIN=local, so it reads the version of the go the node's package manager installed, which is what the
|
||||
// module builds with, and the third word of its answer is that version. It downloads nothing and needs no network.
|
||||
// The environment is built from scratch like a step's (StepRunner), so the module's bus credentials are not in it.
|
||||
func ProbeGo(home, cache string) func(ctx context.Context) (string, error) {
|
||||
run := forgekey.ExecRunnerClean(0)
|
||||
return func(ctx context.Context) (string, error) { return probeGo(ctx, run, home, cache) }
|
||||
}
|
||||
|
||||
// probeEnv is what the probe gives `go version` beyond the base environment: the recipe's release as the toolchain,
|
||||
// the module's cache for it to land in (and the build cache the go command wants), and Go's own proxy and checksum
|
||||
// database, which this module's steps never leave at their default except here and in fetch.
|
||||
// probeEnv is what the probe gives `go version` beyond the base environment: the installed go alone, and the
|
||||
// module's caches, so nothing is written to the account's home.
|
||||
func probeEnv(cache string) []string {
|
||||
return []string{
|
||||
"GOTOOLCHAIN=" + RecipeGo,
|
||||
"GOTOOLCHAIN=local",
|
||||
"GOMODCACHE=" + filepath.Join(cache, "gomod"),
|
||||
"GOCACHE=" + filepath.Join(cache, "gobuild"),
|
||||
}
|
||||
}
|
||||
|
||||
// releaseCached says whether the release is in the module cache: golang.org/toolchain@v0.0.1-<version>.<os>-<arch>.
|
||||
func releaseCached(cache string) bool {
|
||||
dirs, _ := filepath.Glob(filepath.Join(cache, "gomod", "golang.org", "toolchain@v0.0.1-"+RecipeGo+".*"))
|
||||
return len(dirs) > 0
|
||||
}
|
||||
|
||||
func probeGo(ctx context.Context, run forgekey.Runner, home, cache string) (string, error) {
|
||||
bound := ProbeTimeout
|
||||
if !releaseCached(cache) {
|
||||
bound = FetchTimeout
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(ctx, bound)
|
||||
ctx, cancel := context.WithTimeout(ctx, ProbeTimeout)
|
||||
defer cancel()
|
||||
r := run(ctx, append(baseEnv(home), probeEnv(cache)...), "go", "version")
|
||||
if r.Status != 0 || r.Err != "" {
|
||||
return "", fmt.Errorf("`go version` with GOTOOLCHAIN=%s failed (the recipe builds with %s): %s", RecipeGo, RecipeGo,
|
||||
causeLine(r.Err+"\n"+r.Stderr))
|
||||
return "", fmt.Errorf("`go version` with GOTOOLCHAIN=local failed: %s", causeLine(r.Err+"\n"+r.Stderr))
|
||||
}
|
||||
return parseGoVersion(r.Stdout)
|
||||
}
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
package main
|
||||
|
||||
// The recipe of the Go build module (novox/hq ADR 0405 §1 and §7, ADR 0469 §1, ADR 0470 §4; issue 1483): the one
|
||||
// Go toolchain version it builds with, the one postgres major version its integration level tests with, and the
|
||||
// recipe's own version. A node whose installed Go differs from the recipe's refuses every step (guard.go), so all
|
||||
// builders run this version or none builds.
|
||||
// postgres major version its integration level tests with, and the recipe's own version. The module names no Go
|
||||
// version (ADR 0477 §1, §2; issue 1572): it builds with the go its node has installed (GOTOOLCHAIN=local), reports
|
||||
// that version to the toolchains state, and the indexer takes the version the build nodes agree on.
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
@@ -15,15 +15,11 @@ import (
|
||||
"github.com/nats-io/nats-server/v2/server"
|
||||
)
|
||||
|
||||
// RecipeGo is the Go version the recipe builds with: the text `go version` prints for it, which is what the guard
|
||||
// compares and what the indexer reads from <node>.go of the toolchains state.
|
||||
const RecipeGo = "go1.27.1"
|
||||
|
||||
// PostgresMajor is the postgres major version the integration level tests with (ADR 0470 §4).
|
||||
const PostgresMajor = 18
|
||||
|
||||
// RecipeSteps is the recipe's own version: changed whenever the steps of fetch, test or build change in a way
|
||||
// that can change what they produce (ADR 0405 §7). It is one of the four values the recipe version hashes.
|
||||
// that can change what they produce (ADR 0405 §7). It is one of the three values the recipe version hashes.
|
||||
const RecipeSteps = "1"
|
||||
|
||||
// natsModule is the library the integration level's bus is started from (ADR 0470 §2).
|
||||
@@ -33,8 +29,6 @@ const natsModule = "github.com/nats-io/nats-server/v2"
|
||||
type Recipe struct {
|
||||
// Steps is RecipeSteps.
|
||||
Steps string
|
||||
// Go is the toolchain's version as `go version` prints it.
|
||||
Go string
|
||||
// PostgresMajor is the integration level's postgres major version.
|
||||
PostgresMajor int
|
||||
// NatsServer is the nats-server library's version, from the program's build information.
|
||||
@@ -43,16 +37,17 @@ type Recipe struct {
|
||||
|
||||
// Current is the recipe of this build of the module.
|
||||
func Current() Recipe {
|
||||
return Recipe{Steps: RecipeSteps, Go: RecipeGo, PostgresMajor: PostgresMajor, NatsServer: natsServerVersion()}
|
||||
return Recipe{Steps: RecipeSteps, PostgresMajor: PostgresMajor, NatsServer: natsServerVersion()}
|
||||
}
|
||||
|
||||
// Version is the recipe version: a hash of the recipe's own version, the Go version, the postgres major version and
|
||||
// the nats-server version, so a change of any of the four moves it and with it every fingerprint that holds it
|
||||
// Version is the recipe version: a hash of the recipe's own version, the postgres major version and the
|
||||
// nats-server version, never a Go version (ADR 0477 §3): the toolchain enters each fingerprint once, as the version
|
||||
// the build nodes agree on (ADR 0465). A change of any of the three moves it and with it every fingerprint that holds it
|
||||
// (ADR 0405 §7, ADR 0465 §2). The fields are named in the hashed text, so no two recipes share a text.
|
||||
func (r Recipe) Version() string {
|
||||
h := sha256.New()
|
||||
for _, f := range [][2]string{
|
||||
{"steps", r.Steps}, {"go", r.Go}, {"postgres", strconv.Itoa(r.PostgresMajor)}, {"nats-server", r.NatsServer},
|
||||
{"steps", r.Steps}, {"postgres", strconv.Itoa(r.PostgresMajor)}, {"nats-server", r.NatsServer},
|
||||
} {
|
||||
h.Write([]byte(f[0] + "=" + f[1] + "\n"))
|
||||
}
|
||||
|
||||
@@ -7,14 +7,11 @@ import (
|
||||
"bytes"
|
||||
"compress/gzip"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
@@ -24,52 +21,7 @@ func TestMain(m *testing.M) {
|
||||
if len(os.Args) > 1 && os.Args[1] == busArg {
|
||||
os.Exit(runBus(os.Args[2:]))
|
||||
}
|
||||
code := m.Run()
|
||||
releaseCleanup()
|
||||
os.Exit(code)
|
||||
}
|
||||
|
||||
// The module cache the real-go tests share, filled once by the probe as the module's start does (ADR 0476 §3): the
|
||||
// steps carry GOTOOLCHAIN=<RecipeGo> and fetch no toolchain, so a test that runs the real go needs the release in its
|
||||
// cache first.
|
||||
var (
|
||||
releaseOnce sync.Once
|
||||
releaseCache string
|
||||
releaseErr error
|
||||
)
|
||||
|
||||
// needRelease answers the cache with the recipe's release in it, filled by ProbeGo, and skips loudly when the probe
|
||||
// cannot have it (no network to Go's download service where the check runs); the build node has it (issues 1486, 1560).
|
||||
func needRelease(t *testing.T, home string) string {
|
||||
t.Helper()
|
||||
releaseOnce.Do(func() {
|
||||
releaseCache, releaseErr = os.MkdirTemp("", "release-cache")
|
||||
if releaseErr != nil {
|
||||
return
|
||||
}
|
||||
var v string
|
||||
if v, releaseErr = ProbeGo(home, releaseCache)(context.Background()); releaseErr == nil && v != RecipeGo {
|
||||
releaseErr = fmt.Errorf("the probe read %s", v)
|
||||
}
|
||||
})
|
||||
if releaseErr != nil {
|
||||
t.Skipf("NOT RUN: ADR 0476: the probe could not put %s into the module cache where this check runs (%v); proven live on a build node", RecipeGo, releaseErr)
|
||||
}
|
||||
return releaseCache
|
||||
}
|
||||
|
||||
// releaseCleanup removes the shared cache; Go writes its module cache read-only.
|
||||
func releaseCleanup() {
|
||||
if releaseCache == "" {
|
||||
return
|
||||
}
|
||||
filepath.WalkDir(releaseCache, func(p string, d fs.DirEntry, err error) error {
|
||||
if err == nil && d.IsDir() {
|
||||
os.Chmod(p, 0o755)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
os.RemoveAll(releaseCache)
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
// needTools skips, loudly, a test whose tools are not installed where the check runs (the mesh's Go toolchain image
|
||||
@@ -116,7 +68,7 @@ func TestUnitLevelRunsTheRealGoOnTheFixtureRepositories(t *testing.T) {
|
||||
for name, w := range want {
|
||||
tree := repository(t, name)
|
||||
home := t.TempDir()
|
||||
m := &Module{Home: home, Work: t.TempDir(), Cache: needRelease(t, home)}
|
||||
m := &Module{Home: home, Work: t.TempDir(), Cache: t.TempDir()}
|
||||
m.Run = StepRunner(home)
|
||||
res, err := m.runLevel(context.Background(), tree, levelOpts{level: LevelUnit, bounds: Bounds{Run: UnitBound}})
|
||||
if err != nil {
|
||||
@@ -159,7 +111,7 @@ func TestIntegrationLevelRunsAgainstARealStoreAndBusAndLeavesNothing(t *testing.
|
||||
t.Fatal(err)
|
||||
}
|
||||
home := t.TempDir()
|
||||
m := &Module{Recipe: Current(), Home: home, Work: work, Cache: needRelease(t, home)}
|
||||
m := &Module{Recipe: Current(), Home: home, Work: work, Cache: t.TempDir()}
|
||||
m.Run = StepRunner(home)
|
||||
res, err := m.integration(context.Background(), "ws1", filepath.Join(work, "ws1", "tree"), nil)
|
||||
if err != nil {
|
||||
@@ -197,7 +149,7 @@ func TestBuildTwiceGivesOneDigestAndOneExecutable(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
home := t.TempDir()
|
||||
m := &Module{Home: home, Work: work, Cache: needRelease(t, home)}
|
||||
m := &Module{Home: home, Work: work, Cache: t.TempDir()}
|
||||
m.Run = StepRunner(home)
|
||||
args := map[string]any{"workspace": "ws1", "module": ".", "artifact": "tools"}
|
||||
first, err := m.build(context.Background(), args)
|
||||
|
||||
@@ -175,7 +175,7 @@ func TestUnitLevelRunsGoTestJsonInTheModulesDirectoryWithTheStepsEnvironment(t *
|
||||
if line != want {
|
||||
t.Errorf("command = %q, want %q", line, want)
|
||||
}
|
||||
for _, w := range []string{"GOTOOLCHAIN=go1.27.1", "GOFLAGS=-mod=mod", "GOMODCACHE=" + filepath.Join(m.Cache, "gomod")} {
|
||||
for _, w := range []string{"GOTOOLCHAIN=local", "GOFLAGS=-mod=mod", "GOMODCACHE=" + filepath.Join(m.Cache, "gomod")} {
|
||||
if !strings.Contains(env, w) {
|
||||
t.Errorf("environment %s lacks %s", env, w)
|
||||
}
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
)
|
||||
|
||||
// 1560 (ADR 0476 §1): every environment a step gives its go commands names the recipe's release as the toolchain
|
||||
// and never GOTOOLCHAIN=local, which would run the distribution's build.
|
||||
func TestEveryStepEnvironmentCarriesTheRecipesRelease(t *testing.T) {
|
||||
m := &Module{Cache: t.TempDir()}
|
||||
vend := t.TempDir()
|
||||
os.MkdirAll(filepath.Join(vend, "vendor"), 0o755)
|
||||
os.WriteFile(filepath.Join(vend, "vendor", "modules.txt"), nil, 0o644)
|
||||
plain := t.TempDir()
|
||||
envs := map[string][]string{
|
||||
"goEnv": m.goEnv(), "goEnvFor(plain)": m.goEnvFor(plain), "goEnvFor(vendor)": m.goEnvFor(vend),
|
||||
"buildEnv(plain)": m.buildEnv(plain), "buildEnv(vendor)": m.buildEnv(vend),
|
||||
}
|
||||
for name, env := range envs {
|
||||
joined := "\n" + strings.Join(env, "\n") + "\n"
|
||||
if !strings.Contains(joined, "\nGOTOOLCHAIN="+RecipeGo+"\n") {
|
||||
t.Errorf("%s lacks GOTOOLCHAIN=%s:\n%s", name, RecipeGo, joined)
|
||||
}
|
||||
if strings.Contains(joined, "GOTOOLCHAIN=local") {
|
||||
t.Errorf("%s still says GOTOOLCHAIN=local:\n%s", name, joined)
|
||||
}
|
||||
if !strings.Contains(joined, "\nGOMODCACHE="+filepath.Join(m.Cache, "gomod")+"\n") {
|
||||
t.Errorf("%s lacks the module cache:\n%s", name, joined)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1560 (ADR 0476 §3): the fetch step downloads the change's locked dependencies and nothing else: no `go version`
|
||||
// and no toolchain download among its commands, and each go command carries the recipe's release.
|
||||
func TestTheFetchStepsCommandsFetchNoToolchain(t *testing.T) {
|
||||
f := &fakeStep{gomods: "go.mod\n"}
|
||||
m, states := fetchModule(t, f)
|
||||
if _, err := fetchTool(t, m)(fetchArgs); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if st := states.waitEnd(t); st.State != delivery.StepPassed {
|
||||
t.Fatalf("the step ended %s: %s", st.State, st.Reason)
|
||||
}
|
||||
goCalls := 0
|
||||
for _, c := range f.calls {
|
||||
if strings.Contains(c.line, "version") || strings.Contains(c.line, "toolchain") {
|
||||
t.Errorf("the fetch step ran %q", c.line)
|
||||
}
|
||||
if strings.HasPrefix(c.line, "go ") {
|
||||
goCalls++
|
||||
if env := strings.Join(c.env, " "); !strings.Contains(env, "GOTOOLCHAIN="+RecipeGo) || strings.Contains(env, "GOTOOLCHAIN=local") {
|
||||
t.Errorf("%s: environment %s", c.line, env)
|
||||
}
|
||||
}
|
||||
}
|
||||
if goCalls == 0 {
|
||||
t.Error("no go command ran")
|
||||
}
|
||||
}
|
||||
@@ -1,55 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// 1560 (ADR 0476 §3): the probe, with a fresh module cache and network, lets the installed Go put
|
||||
// golang.org/toolchain@v0.0.1-<RecipeGo>.<os>-<arch> into the cache and reads the release's version; then a step
|
||||
// with the cache read-only and no network (GOPROXY=off) reports the recipe's version from it.
|
||||
func TestTheProbePutsTheReleaseIntoTheCacheAndAStepReadsItOffline(t *testing.T) {
|
||||
home, cache := t.TempDir(), t.TempDir()
|
||||
t.Cleanup(func() { // the Go module cache is read-only by Go's own doing
|
||||
filepath.WalkDir(cache, func(p string, d fs.DirEntry, err error) error {
|
||||
if err == nil && d.IsDir() {
|
||||
os.Chmod(p, 0o755)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
})
|
||||
v, err := ProbeGo(home, cache)(context.Background())
|
||||
if err != nil {
|
||||
t.Skipf("NOT RUN: ADR 0476: the probe could not put %s into the module cache where this check runs (%v); proven live on a build node", RecipeGo, err)
|
||||
}
|
||||
if v != RecipeGo {
|
||||
t.Fatalf("probe = %q; want %s", v, RecipeGo)
|
||||
}
|
||||
dirs, _ := filepath.Glob(filepath.Join(cache, "gomod", "golang.org", "toolchain@v0.0.1-"+RecipeGo+".*"))
|
||||
if len(dirs) != 1 {
|
||||
t.Fatalf("the cache holds %v, want golang.org/toolchain@v0.0.1-%s.*", dirs, RecipeGo)
|
||||
}
|
||||
filepath.WalkDir(cache, func(p string, d fs.DirEntry, err error) error {
|
||||
if err == nil {
|
||||
if info, e := d.Info(); e == nil {
|
||||
os.Chmod(p, info.Mode()&^0o222)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
r := StepRunner(home)(context.Background(), []string{
|
||||
"GOTOOLCHAIN=" + RecipeGo, "GOMODCACHE=" + filepath.Join(cache, "gomod"), "GOCACHE=" + t.TempDir(), "GOPROXY=off", "GOFLAGS=-mod=mod",
|
||||
}, "go", "version")
|
||||
if r.Status != 0 || r.Err != "" {
|
||||
t.Fatalf("go version offline: %s %s", r.Err, r.Stderr)
|
||||
}
|
||||
if got, err := parseGoVersion(r.Stdout); err != nil || got != RecipeGo || !strings.Contains(r.Stdout, RecipeGo) {
|
||||
t.Errorf("the step reports %q, %v; want %s", r.Stdout, err, RecipeGo)
|
||||
}
|
||||
}
|
||||
@@ -1,135 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/forgekey"
|
||||
"git.novox.be/novox/mesh-sdk/go/health"
|
||||
)
|
||||
|
||||
// probeFake is a runner that records the command and answers by rule.
|
||||
type probeFake struct {
|
||||
env []string
|
||||
line string
|
||||
ran forgekey.Ran
|
||||
}
|
||||
|
||||
func (p *probeFake) run(_ context.Context, env []string, name string, args ...string) forgekey.Ran {
|
||||
p.env, p.line = env, name+" "+strings.Join(args, " ")
|
||||
return p.ran
|
||||
}
|
||||
|
||||
// 1560 (ADR 0476 §2, §3): the probe runs `go version` with the recipe's release as the toolchain and the module's
|
||||
// cache as GOMODCACHE, in an environment built from scratch (nothing of the process's, no GOTOOLCHAIN=local).
|
||||
func TestTheProbeRunsGoVersionWithTheRecipesReleaseAndTheModuleCache(t *testing.T) {
|
||||
p := &probeFake{ran: forgekey.Ran{Stdout: "go version go1.27.1 linux/amd64\n"}}
|
||||
cache := t.TempDir()
|
||||
v, err := probeGo(context.Background(), p.run, "/home/x", cache)
|
||||
if err != nil || v != "go1.27.1" {
|
||||
t.Fatalf("= %q, %v", v, err)
|
||||
}
|
||||
env := "\n" + strings.Join(p.env, "\n") + "\n"
|
||||
for _, want := range []string{"\nGOTOOLCHAIN=" + RecipeGo + "\n", "\nGOMODCACHE=" + filepath.Join(cache, "gomod") + "\n", "\nHOME=/home/x\n"} {
|
||||
if !strings.Contains(env, want) {
|
||||
t.Errorf("the probe's environment lacks %q:\n%s", want, env)
|
||||
}
|
||||
}
|
||||
if strings.Contains(env, "GOTOOLCHAIN=local") || p.line != "go version" {
|
||||
t.Errorf("command %q, environment:\n%s", p.line, env)
|
||||
}
|
||||
}
|
||||
|
||||
// 1560: the release's own output parses to its version; the distribution's build parses to a different one.
|
||||
func TestParseGoVersionOfTheReleaseAndOfTheDistributionsBuild(t *testing.T) {
|
||||
if v, err := parseGoVersion("go version go1.27.1 linux/amd64\n"); err != nil || v != RecipeGo {
|
||||
t.Errorf("release: %q, %v", v, err)
|
||||
}
|
||||
if v, _ := parseGoVersion("go version go1.27.1-X:nodwarf5 linux/amd64\n"); v == RecipeGo {
|
||||
t.Errorf("the distribution's build parsed as the release: %q", v)
|
||||
}
|
||||
}
|
||||
|
||||
// 1560 (ADR 0476 §2): the toolchains key carries the version the probe read from the release.
|
||||
func TestTheToolchainsKeyIsTheReleasesVersion(t *testing.T) {
|
||||
states := newStates()
|
||||
m := moduleWith(states, "")
|
||||
p := &probeFake{ran: forgekey.Ran{Stdout: "go version go1.27.1 linux/amd64\n"}}
|
||||
m.Probe = func(ctx context.Context) (string, error) { return probeGo(ctx, p.run, "/home/x", t.TempDir()) }
|
||||
if err := m.Check(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tc, ok := states.toolchain("ace.go")
|
||||
if !ok || tc.Version != RecipeGo {
|
||||
t.Fatalf("ace.go = %+v, %v; want version %s", tc, ok, RecipeGo)
|
||||
}
|
||||
}
|
||||
|
||||
// 1560 (ADR 0476 §2): a probe that fails (the release cannot be had) or reads another version makes the module
|
||||
// refuse, and its health is unhealthy naming the recipe's version and what the probe got.
|
||||
func TestAFailedFetchOrAnotherVersionMakesHealthUnhealthyNamingBoth(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
ran forgekey.Ran
|
||||
want []string
|
||||
}{
|
||||
{"fetch fails", forgekey.Ran{Status: 1, Stderr: "go: downloading go1.27.1 (linux/amd64)\ngo: download go1.27.1 for linux/amd64: dial tcp: no route to host\n"},
|
||||
[]string{"go1.27.1", "no route to host"}},
|
||||
{"other version", forgekey.Ran{Stdout: "go version go1.27.1-X:nodwarf5 linux/amd64\n"},
|
||||
[]string{"go1.27.1", "go1.27.1-X:nodwarf5"}},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
m := moduleWith(newStates(), "")
|
||||
p := &probeFake{ran: c.ran}
|
||||
m.Probe = func(ctx context.Context) (string, error) { return probeGo(ctx, p.run, "/home/x", t.TempDir()) }
|
||||
a := m.Health(context.Background())
|
||||
if a.Healthy || a.State != health.StateUnhealthy {
|
||||
t.Fatalf("health = %+v, want unhealthy", a)
|
||||
}
|
||||
for _, w := range c.want {
|
||||
if !strings.Contains(a.Why, w) {
|
||||
t.Errorf("health does not name %q: %q", w, a.Why)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(a.Why, "recipe") && !strings.Contains(a.Why, RecipeGo) {
|
||||
t.Errorf("health does not name the recipe's version: %q", a.Why)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 1560: the probe is bound long enough to download while the release is missing, and short once it is cached.
|
||||
func TestTheProbeIsBoundLongOnlyWhileTheReleaseIsMissing(t *testing.T) {
|
||||
var got time.Duration
|
||||
run := func(ctx context.Context, _ []string, _ string, _ ...string) forgekey.Ran {
|
||||
d, _ := ctx.Deadline()
|
||||
got = time.Until(d)
|
||||
return forgekey.Ran{Stdout: "go version go1.27.1 linux/amd64\n"}
|
||||
}
|
||||
cache := t.TempDir()
|
||||
probeGo(context.Background(), run, "/h", cache)
|
||||
if got < time.Minute {
|
||||
t.Errorf("missing release: bound %v", got)
|
||||
}
|
||||
os.MkdirAll(filepath.Join(cache, "gomod", "golang.org", "toolchain@v0.0.1-"+RecipeGo+".linux-amd64"), 0o755)
|
||||
probeGo(context.Background(), run, "/h", cache)
|
||||
if got > ProbeTimeout {
|
||||
t.Errorf("cached release: bound %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// 1560 (review): Go's progress line is not the cause: the refusal names the line after it.
|
||||
func TestCauseLineSkipsGosDownloadingLine(t *testing.T) {
|
||||
got := causeLine("go: downloading go1.27.1 (linux/amd64)\ngo: download go1.27.1: checksum mismatch\n")
|
||||
if got != "go: download go1.27.1: checksum mismatch" {
|
||||
t.Errorf("causeLine = %q", got)
|
||||
}
|
||||
if got := causeLine("go: downloading go1.27.1 (linux/amd64)\n"); got != "go: downloading go1.27.1 (linux/amd64)" {
|
||||
t.Errorf("only a progress line: %q", got)
|
||||
}
|
||||
}
|
||||
@@ -9,9 +9,11 @@ provisioner's client (issue 1537).
|
||||
|
||||
## The recipe
|
||||
|
||||
`recipe.go` names the one Node version it builds with (`RecipeNode`, `v26.10.0`, as `node --version` prints it) and the
|
||||
recipe's own version (`RecipeSteps`). The recipe version is a hash of those two and of the bytes of the lock file
|
||||
`toolchain/package-lock.json`, so one changed byte of the lock, another Node or another recipe moves it.
|
||||
The module builds with the `nodejs` and `npm` its node has installed and names no Node version (ADR 0477 §1, issue
|
||||
1572); the versions of the build nodes are agreed by the indexer (ADR 0465) and kept equal by the weekly upgrade of
|
||||
every node's packages (ADR 0477 §5). `recipe.go` holds the recipe's own version (`RecipeSteps`). The recipe version is a
|
||||
hash of that and of the bytes of the lock file `toolchain/package-lock.json`, never of a Node version (ADR 0477 §3), so
|
||||
one changed byte of the lock or another recipe moves it, and typescript and esbuild change only when the lock does.
|
||||
|
||||
## The toolchain directory
|
||||
|
||||
@@ -34,9 +36,10 @@ separately: sdk and nats are in this directory's `node_modules`, and the build s
|
||||
|
||||
At start, every ten minutes, before every step and at every health look, the module reads the lock again and the
|
||||
installed Node (`node --version`) and writes it with the recipe version to `<node>.typescript` of its `toolchains`
|
||||
state. A machine whose Node is not the recipe's refuses every step, the step ending `refused` on its key in `steps` with
|
||||
both versions named, and `build_node_health` (the module's `runtime-health`) answers unhealthy with the same text, so
|
||||
the controller raises the machine's condition. A step runs only after the toolchain directory is installed.
|
||||
state. Whatever the version, the module is healthy and `build_node_health` (the module's `runtime-health`) names it. Only a
|
||||
`node` that cannot be run, or a lock that cannot be read, refuses every step, the step ending `refused` on its key in
|
||||
`steps` with the cause named; the health is then unhealthy with the same text, so the controller raises the machine's
|
||||
condition. A step runs only after the toolchain directory is installed.
|
||||
|
||||
## Tools
|
||||
|
||||
|
||||
@@ -70,7 +70,7 @@ func newMachine(t *testing.T) (*Module, *fakeMachine, string) {
|
||||
t.Helper()
|
||||
home, cache, work := t.TempDir(), t.TempDir(), t.TempDir()
|
||||
f := &fakeMachine{now: time.Unix(1_000_000, 0), cost: map[string]time.Duration{}, answer: map[string]forgekey.Ran{}}
|
||||
m := &Module{Node: "ace", Recipe: Recipe{Steps: "1", Node: "v26.10.0", Lock: "lock"},
|
||||
m := &Module{Node: "ace", Recipe: Recipe{Steps: "1", Lock: "lock"},
|
||||
Toolchain: &Toolchain{Paths: Paths{Home: home, Cache: cache}, Registry: func() (Registry, error) {
|
||||
return Registry{URL: "https://git.novox.be/api/packages/novox/npm/"}, nil
|
||||
}},
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/health"
|
||||
)
|
||||
|
||||
// 1572/AC2 (ADR 0477 §2): the recipe has no field for a Node version, and no constant names one.
|
||||
func TestTheRecipeHasNoFieldForAToolchainVersion(t *testing.T) {
|
||||
typ := reflect.TypeOf(Recipe{})
|
||||
for i := 0; i < typ.NumField(); i++ {
|
||||
if n := typ.Field(i).Name; n == "Node" || strings.Contains(n, "Toolchain") {
|
||||
t.Errorf("Recipe has the field %s", n)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1572/AC2 and AC4: whatever Node the machine has installed, the module stays healthy, names that version in its
|
||||
// health, runs its steps and writes the version to <node>.typescript of the toolchains state.
|
||||
func TestAnyInstalledNodeIsHealthyNamedAndReported(t *testing.T) {
|
||||
for _, installed := range []string{"v26.10.0", "v26.11.0", "v27.0.1", "v24.9.0"} {
|
||||
t.Run(installed, func(t *testing.T) {
|
||||
states := newStates()
|
||||
m := moduleWith(states, installed)
|
||||
if err := m.Check(context.Background()); err != nil {
|
||||
t.Fatalf("Check refused: %v", err)
|
||||
}
|
||||
a := m.Health(context.Background())
|
||||
if !a.Healthy || a.State != health.StateHealthy || !strings.Contains(a.Why, installed) {
|
||||
t.Errorf("health = %+v, want healthy naming %s", a, installed)
|
||||
}
|
||||
tc, ok := states.toolchain("ace.typescript")
|
||||
if !ok || tc.Version != installed || tc.Recipe != m.Recipe.Version() {
|
||||
t.Errorf("ace.typescript = %+v, %v; want version %s and recipe %s", tc, ok, installed, m.Recipe.Version())
|
||||
}
|
||||
if err := m.Prepare(context.Background()); err != nil {
|
||||
t.Errorf("no step can run on node %s: %v", installed, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 1572/AC3 (ADR 0477 §3): the recipe version is made without a Node version: the recipe read from a lock is the same
|
||||
// whatever node is installed, and a version set on the recipe by name moves nothing; it moves with the steps' version
|
||||
// and with one byte of the lock.
|
||||
func TestTheRecipeVersionIgnoresTheToolchainVersionAndMovesWithStepsAndLock(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "package-lock.json")
|
||||
os.WriteFile(path, []byte("lock-a"), 0o600)
|
||||
base, err := ReadRecipe(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
v := base.Version()
|
||||
for _, installed := range []string{"v26.10.0", "v27.0.1"} {
|
||||
m := moduleWith(newStates(), installed)
|
||||
m.Reload = func() (Recipe, error) { return ReadRecipe(path) }
|
||||
if err := m.Check(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := m.recipe().Version(); got != v {
|
||||
t.Errorf("with node %s installed the recipe version is %s, want %s", installed, got, v)
|
||||
}
|
||||
}
|
||||
r := base
|
||||
if f := reflect.ValueOf(&r).Elem().FieldByName("Node"); f.IsValid() {
|
||||
f.SetString("v99.0.0")
|
||||
if r.Version() != v {
|
||||
t.Errorf("a Node version on the recipe moved its version")
|
||||
}
|
||||
}
|
||||
r = base
|
||||
r.Steps += "x"
|
||||
if r.Version() == v {
|
||||
t.Errorf("the steps' version did not move the recipe version")
|
||||
}
|
||||
os.WriteFile(path, []byte("lock-b"), 0o600)
|
||||
if b, _ := ReadRecipe(path); b.Version() == v {
|
||||
t.Errorf("one byte of the lock did not move the recipe version")
|
||||
}
|
||||
}
|
||||
@@ -53,23 +53,35 @@ func (m *Module) recipe() Recipe {
|
||||
return m.Recipe
|
||||
}
|
||||
|
||||
func (m *Module) guard(r Recipe) buildstep.Guard {
|
||||
return buildstep.Guard{Language: "typescript", Want: r.Node, Recipe: r.Version(), Probe: m.Probe, Node: m.Node, Put: m.reportingPut()}
|
||||
// guard is the SDK guard for the version the probe just read: the module names no Node version (ADR 0477 §2), so the
|
||||
// version wanted is the one installed and the guard refuses only when node cannot be run, writing the installed
|
||||
// version to <node>.typescript of the toolchains state every time.
|
||||
func (m *Module) guard(r Recipe, installed string, probeErr error) buildstep.Guard {
|
||||
return buildstep.Guard{Language: "typescript", Want: installed, Recipe: r.Version(), Node: m.Node, Put: m.reportingPut(),
|
||||
Probe: func(context.Context) (string, error) { return installed, probeErr }}
|
||||
}
|
||||
|
||||
// Check reads the installed toolchain, writes <node>.typescript of the toolchains state and answers nil, or the
|
||||
// refusal (buildstep.Guard.Check). It runs at start, before every step, at every health look and every CheckRound.
|
||||
// refusal when node cannot be run (buildstep.Guard.Check); another version than another node's is no refusal (ADR 0477
|
||||
// §2). It runs at start, before every step, at every health look and every CheckRound.
|
||||
func (m *Module) Check(ctx context.Context) error {
|
||||
_, err := m.check(ctx)
|
||||
return err
|
||||
}
|
||||
|
||||
// check is Check that also answers the version it found installed.
|
||||
func (m *Module) check(ctx context.Context) (string, error) {
|
||||
if m.Reload != nil {
|
||||
r, err := m.Reload()
|
||||
if err != nil {
|
||||
return &buildstep.Refusal{Text: err.Error(), Want: RecipeNode}
|
||||
return "", &buildstep.Refusal{Text: err.Error()}
|
||||
}
|
||||
m.mu.Lock()
|
||||
m.Recipe = r
|
||||
m.mu.Unlock()
|
||||
}
|
||||
return m.guard(m.recipe()).Check(ctx)
|
||||
installed, err := m.Probe(ctx)
|
||||
return installed, m.guard(m.recipe(), installed, err).Check(ctx)
|
||||
}
|
||||
|
||||
// Prepare is Check and then the toolchain directory of the recipe in force: what every step runs behind.
|
||||
@@ -99,20 +111,20 @@ func (m *Module) Start(ctx context.Context) {
|
||||
}
|
||||
}
|
||||
|
||||
// Health is the module's health (the runtime-health tool, ADR 0366): unhealthy while the toolchain installed is not
|
||||
// the recipe's, saying both versions, so the controller raises the node's condition. It looks again each time it is
|
||||
// Health is the module's health (the runtime-health tool, ADR 0366): unhealthy only while node cannot be run (or the lock cannot
|
||||
// be read), saying why, so the controller raises the node's condition; healthy it names the installed version. It looks again each time it is
|
||||
// asked.
|
||||
func (m *Module) Health(ctx context.Context) health.Answer {
|
||||
if err := m.Check(ctx); err != nil {
|
||||
installed, err := m.check(ctx)
|
||||
if err != nil {
|
||||
return health.Unhealthy(err.Error())
|
||||
}
|
||||
r := m.recipe()
|
||||
return health.Healthy(fmt.Sprintf("node %s installed, as the recipe %s builds with", r.Node, r.Version()))
|
||||
return health.Healthy(fmt.Sprintf("node %s installed, the version the module builds with (recipe %s)", installed, m.recipe().Version()))
|
||||
}
|
||||
|
||||
// Long is a tool that runs a long step (ADR 0469 §2): it answers at once with the step's id, and the step ends on
|
||||
// its key in the steps state. The guard runs inside the step, before run, so a node that refuses ends every step
|
||||
// refused with both versions named.
|
||||
// its key in the steps state. The guard runs inside the step, before run, so a node whose node cannot be run ends every
|
||||
// step refused with the cause named.
|
||||
func (m *Module) Long(tool string, run func(ctx context.Context, args map[string]any) (any, error)) func(map[string]any) (any, error) {
|
||||
return func(args map[string]any) (any, error) {
|
||||
id, err := buildstep.Start(m.Ctx, m.Steps, m.Node, tool, func(ctx context.Context) (any, error) {
|
||||
@@ -145,7 +157,7 @@ func (m *Module) Status(ctx context.Context) map[string]any {
|
||||
err := m.Check(ctx)
|
||||
installed, _ := m.Probe(ctx)
|
||||
r := m.recipe()
|
||||
out := map[string]any{"node": m.Node, "recipe": r.Version(), "node-version": r.Node, "installed": installed, "refusing": err != nil}
|
||||
out := map[string]any{"node": m.Node, "recipe": r.Version(), "installed": installed, "refusing": err != nil}
|
||||
if m.Toolchain != nil {
|
||||
out["toolchain-dir"] = m.Toolchain.Paths.Dir(r.Version())
|
||||
out["toolchain-installed"] = m.Toolchain.Present(r.Version())
|
||||
|
||||
@@ -74,7 +74,7 @@ func (s *keptStates) waitEnd(t *testing.T) delivery.Step {
|
||||
}
|
||||
|
||||
func moduleWith(states *keptStates, installed string) *Module {
|
||||
return &Module{Node: "ace", Recipe: Recipe{Steps: "1", Node: "v26.10.0", Lock: "lock"},
|
||||
return &Module{Node: "ace", Recipe: Recipe{Steps: "1", Lock: "lock"},
|
||||
Probe: func(context.Context) (string, error) { return installed, nil },
|
||||
Toolchains: states.putToolchain, Steps: states.putStep, Ctx: context.Background()}
|
||||
}
|
||||
@@ -102,11 +102,14 @@ func TestStartWritesTheToolchainAndHealthIsHealthy(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// 1488/AC3: a probe answering v26.11.0: every step tool's step ends refused naming both versions, ace.typescript
|
||||
// holds v26.11.0, and the health tool answers unhealthy naming both.
|
||||
func TestAnotherNodeRefusesEveryStepAndHealthIsUnhealthy(t *testing.T) {
|
||||
// 1572/AC2 (ADR 0477 §2): a node that cannot be run: every step tool's step ends refused naming the cause, and the
|
||||
// health tool answers unhealthy naming it.
|
||||
func TestANodeThatCannotBeRunRefusesEveryStepAndHealthIsUnhealthy(t *testing.T) {
|
||||
states := newStates()
|
||||
m := moduleWith(states, "v26.11.0")
|
||||
m := moduleWith(states, "")
|
||||
m.Probe = func(context.Context) (string, error) {
|
||||
return "", errors.New("exec: node: executable file not found in $PATH")
|
||||
}
|
||||
ran := false
|
||||
run := func(context.Context, map[string]any) (any, error) { ran = true; return "done", nil }
|
||||
steps := append([]StepTool{
|
||||
@@ -137,19 +140,16 @@ func TestAnotherNodeRefusesEveryStepAndHealthIsUnhealthy(t *testing.T) {
|
||||
}
|
||||
reason = err.Error()
|
||||
}
|
||||
if !strings.Contains(reason, "v26.11.0") || !strings.Contains(reason, "v26.10.0") {
|
||||
t.Errorf("%s: the refusal %q does not name both versions", tool.Name, reason)
|
||||
if !strings.Contains(reason, "executable file not found") {
|
||||
t.Errorf("%s: the refusal %q does not name the cause", tool.Name, reason)
|
||||
}
|
||||
}
|
||||
if ran {
|
||||
t.Errorf("a step ran on a node that refuses")
|
||||
}
|
||||
if tc, _ := states.toolchain("ace.typescript"); tc.Version != "v26.11.0" {
|
||||
t.Errorf("ace.typescript holds %q, want the installed v26.11.0", tc.Version)
|
||||
}
|
||||
a := m.Health(context.Background())
|
||||
if a.State != health.StateUnhealthy || !strings.Contains(a.Why, "v26.11.0") || !strings.Contains(a.Why, "v26.10.0") {
|
||||
t.Errorf("health = %+v, want unhealthy naming both versions", a)
|
||||
if a.State != health.StateUnhealthy || !strings.Contains(a.Why, "executable file not found") {
|
||||
t.Errorf("health = %+v, want unhealthy naming the cause", a)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -162,19 +162,18 @@ func isLong(steps []StepTool, name string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// 1488/AC2: the recipe version moves with one byte of the lock and with the Node version, and with the recipe's own
|
||||
// version; equal recipes have the same one.
|
||||
// 1488/AC2, 1572/AC3: the recipe version moves with one byte of the lock and with the recipe's own version; equal
|
||||
// recipes have the same one.
|
||||
func TestRecipeVersionMovesWithEachOfItsValues(t *testing.T) {
|
||||
lock := []byte(`{"lockfileVersion":3}`)
|
||||
base := Recipe{Steps: "1", Node: "v26.10.0", Lock: LockHash(lock)}
|
||||
if base.Version() != (Recipe{Steps: "1", Node: "v26.10.0", Lock: LockHash(lock)}).Version() {
|
||||
base := Recipe{Steps: "1", Lock: LockHash(lock)}
|
||||
if base.Version() != (Recipe{Steps: "1", Lock: LockHash(lock)}).Version() {
|
||||
t.Errorf("one recipe has two versions")
|
||||
}
|
||||
seen := map[string]string{base.Version(): "base"}
|
||||
for name, r := range map[string]Recipe{
|
||||
"one byte of the lock": {Steps: "1", Node: "v26.10.0", Lock: LockHash([]byte(`{"lockfileVersion":4}`))},
|
||||
"node": {Steps: "1", Node: "v26.11.0", Lock: LockHash(lock)},
|
||||
"steps": {Steps: "2", Node: "v26.10.0", Lock: LockHash(lock)},
|
||||
"one byte of the lock": {Steps: "1", Lock: LockHash([]byte(`{"lockfileVersion":4}`))},
|
||||
"steps": {Steps: "2", Lock: LockHash(lock)},
|
||||
} {
|
||||
v := r.Version()
|
||||
if was, dup := seen[v]; dup {
|
||||
@@ -193,7 +192,7 @@ func TestReadRecipeHashesTheLockFile(t *testing.T) {
|
||||
}
|
||||
os.WriteFile(path, []byte("a"), 0o600)
|
||||
a, err := ReadRecipe(path)
|
||||
if err != nil || a.Node != "v26.10.0" {
|
||||
if err != nil || a.Steps != RecipeSteps {
|
||||
t.Fatalf("= %+v, %v", a, err)
|
||||
}
|
||||
os.WriteFile(path, []byte("b"), 0o600)
|
||||
@@ -207,7 +206,7 @@ func TestReadRecipeHashesTheLockFile(t *testing.T) {
|
||||
func TestACheckReadsTheRecipeAgain(t *testing.T) {
|
||||
states := newStates()
|
||||
m := moduleWith(states, "v26.10.0")
|
||||
r2 := Recipe{Steps: "1", Node: "v26.10.0", Lock: "other"}
|
||||
r2 := Recipe{Steps: "1", Lock: "other"}
|
||||
m.Reload = func() (Recipe, error) { return r2, nil }
|
||||
if err := m.Check(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
|
||||
@@ -11,7 +11,7 @@ import (
|
||||
// ProbeTimeout bounds `node --version`.
|
||||
const ProbeTimeout = 15 * time.Second
|
||||
|
||||
// ProbeNode reads the installed Node version: what `node --version` prints ("v26.10.0"), as RecipeNode is written.
|
||||
// ProbeNode reads the installed Node version: what `node --version` prints ("v26.10.0"), as the toolchains state holds it.
|
||||
func ProbeNode(ctx context.Context) (string, error) {
|
||||
ctx, cancel := context.WithTimeout(ctx, ProbeTimeout)
|
||||
defer cancel()
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
package main
|
||||
|
||||
// The recipe of the Node build module (novox/hq ADR 0405 §1 and §7, ADR 0469 §1 and §4; issue 1488): the one Node
|
||||
// version it builds with, and the recipe's own version, a hash of the recipe's version, that Node version and the
|
||||
// bytes of the lock file that names the TypeScript toolchain. A node whose installed Node differs from the recipe's
|
||||
// refuses every step (module.go), so all builders run this version or none builds.
|
||||
// The recipe of the Node build module (novox/hq ADR 0405 §7, ADR 0469 §4, ADR 0477 §1-§3; issues 1488, 1572): the
|
||||
// recipe's own version, and the recipe version, a hash of that and the bytes of the lock file that names the
|
||||
// TypeScript toolchain. The module names no Node version: it builds with the installed nodejs and npm, reports the
|
||||
// version installed to the toolchains state, and refuses only when node cannot be run (module.go).
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
@@ -12,20 +12,14 @@ import (
|
||||
"os"
|
||||
)
|
||||
|
||||
// RecipeNode is the Node version the recipe builds with: the text `node --version` prints for it, which is what the
|
||||
// guard compares and what the indexer reads from <node>.typescript of the toolchains state.
|
||||
const RecipeNode = "v26.10.0"
|
||||
|
||||
// RecipeSteps is the recipe's own version: changed whenever the steps of fetch, test or build change in a way that
|
||||
// can change what they produce (ADR 0405 §7). It is one of the three values the recipe version hashes.
|
||||
// can change what they produce (ADR 0405 §7). It is one of the two values the recipe version hashes.
|
||||
const RecipeSteps = "1"
|
||||
|
||||
// Recipe is every value that makes up the recipe version.
|
||||
type Recipe struct {
|
||||
// Steps is RecipeSteps.
|
||||
Steps string
|
||||
// Node is the toolchain's version as `node --version` prints it.
|
||||
Node string
|
||||
// Lock is the hex sha256 of the bytes of toolchain/package-lock.json (LockHash).
|
||||
Lock string
|
||||
}
|
||||
@@ -43,15 +37,16 @@ func ReadRecipe(lockPath string) (Recipe, error) {
|
||||
if err != nil {
|
||||
return Recipe{}, fmt.Errorf("the toolchain's lock file cannot be read: %w", err)
|
||||
}
|
||||
return Recipe{Steps: RecipeSteps, Node: RecipeNode, Lock: LockHash(raw)}, nil
|
||||
return Recipe{Steps: RecipeSteps, Lock: LockHash(raw)}, nil
|
||||
}
|
||||
|
||||
// Version is the recipe version: a hash of the recipe's own version, the Node version and the lock file's hash, so a
|
||||
// change of any of the three moves it and with it every fingerprint that holds it (ADR 0405 §7, ADR 0465 §2). The
|
||||
// Version is the recipe version: a hash of the recipe's own version and the lock file's hash, never a Node version
|
||||
// (ADR 0477 §3): the toolchain enters each fingerprint once, as the version the build nodes agree on. A change of
|
||||
// either moves it and with it every fingerprint that holds it (ADR 0405 §7, ADR 0465 §2). The
|
||||
// fields are named in the hashed text, so no two recipes share a text.
|
||||
func (r Recipe) Version() string {
|
||||
h := sha256.New()
|
||||
for _, f := range [][2]string{{"steps", r.Steps}, {"node", r.Node}, {"lock", r.Lock}} {
|
||||
for _, f := range [][2]string{{"steps", r.Steps}, {"lock", r.Lock}} {
|
||||
h.Write([]byte(f[0] + "=" + f[1] + "\n"))
|
||||
}
|
||||
return hex.EncodeToString(h.Sum(nil))[:16]
|
||||
|
||||
@@ -2,6 +2,7 @@ package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
@@ -57,7 +58,7 @@ func toolchainIn(t *testing.T, fake *fakeNpm) Toolchain {
|
||||
func TestAMissingToolchainDirectoryIsInstalledOnceFromTheLock(t *testing.T) {
|
||||
fake := &fakeNpm{}
|
||||
tc := toolchainIn(t, fake)
|
||||
version := Recipe{Steps: "1", Node: "v26.10.0", Lock: "x"}.Version()
|
||||
version := Recipe{Steps: "1", Lock: "x"}.Version()
|
||||
dir := filepath.Join(tc.Paths.Cache, "toolchain", version)
|
||||
if tc.Present(version) {
|
||||
t.Fatal("present before any install")
|
||||
@@ -137,7 +138,8 @@ func TestPrepareInstallsOnlyBehindAPassingGuard(t *testing.T) {
|
||||
fake := &fakeNpm{}
|
||||
tc := toolchainIn(t, fake)
|
||||
states := newStates()
|
||||
m := moduleWith(states, "v26.11.0")
|
||||
m := moduleWith(states, "")
|
||||
m.Probe = func(context.Context) (string, error) { return "", errors.New("no node") }
|
||||
m.Toolchain = &tc
|
||||
if err := m.Prepare(context.Background()); err == nil || len(fake.calls) != 0 {
|
||||
t.Fatalf("a refusing node: %v, %d npm calls", err, len(fake.calls))
|
||||
|
||||
Reference in New Issue
Block a user