6.8 KiB
mesh-build-go — the Go build module
Design 47, release R4c-3 (novox/hq ADR 0405, ADR 0466 §1, ADR 0469, ADR 0470). The module that builds and checks Go
code on a build machine, as an account of its own (runs-as, no login shell, home at mode 0700), with the
toolchain installed as the packages it declares (go, git, openssh, postgresql) and the capability
user-namespaces. It claims no seat: it is assigned to the build machines (ADR 0469 §6) and runs dark until the
builder uses it.
The recipe
cmd/mesh-build-go/recipe.go names the one Go version the module builds with (RecipeGo), the postgres major
version the integration level tests with, and the recipe's own version. The recipe version is a hash of those and of
the nats-server library's version, read from the program's build information, so a change of any of them moves it.
The guard and the health
At start, every ten minutes, before every step and at every health look, the module reads the installed Go
(go version) and writes it with the recipe version to <node>.go of its toolchains state. A machine whose Go
is not the recipe's refuses every step, the step ending refused on its key in steps with both versions named,
and build_go_health (the module's runtime-health) answers unhealthy with the same text, so the controller raises
the machine's condition. Moving to a new Go is a change of RecipeGo.
Tools
build_go_status, build_go_health and build_go_fetch are served now; the other step tools (build_go_test,
build_go_digest, build_go_build) are added by their own issues to StepTools and run behind the guard: a long
step answers at once with its id and ends on <node>.<id> of steps (ADR 0469 §2); the digest answers in its call.
Fetch
build_go_fetch {repository, commit, ref?} fetches the commit over SSH as the module's account (mesh-forge is the
forge-account provision's ssh name, so the account's own ssh files and git rewrite are used) into a workspace of the
work directory, then runs go mod download and go mod verify in every directory with a go.mod that does not vendor,
with GOTOOLCHAIN=local, GOFLAGS=-mod=mod and the module's cache directory. Every command runs with an
environment built from scratch (StepRunner): PATH, HOME and what the step passes. Workspaces older than a day
are removed.
Test
build_go_test {workspace, level, packages?} runs the tests of a fetched workspace by Go's convention, one level at a
time. The unit level is go test -json over every package without the integration build tag, in each directory
with a go.mod, judged by its one bound: its running time, that of the slowest package (compile not counted), within
five seconds. The step ends passed when the level ran, and its result says whether the level did: passed, the
packages with their times, the tests that failed with their first lines of output, the five slowest tests, and why
when it failed (a failed test, a build that failed) or ran over a bound. passed is the tests' own verdict: a level over its bound still passes and says over_bound (and over_which, setup or run) with the slowest tests, and the checker judges the bounds (ADR 0453 §9). A step ends failed only when go test could not
run. A repository with no tests at the level has none, and the result says so: that is a pass. testdata/captured holds
the output of the real go for the fixture repositories in testdata/repos; the unit tests read it, and the
integration-tagged test runs the real go on the fixtures.
The integration level
build_go_test with level: integration runs go test -tags integration against a store and a bus started for the
run alone (novox/hq ADR 0470). The store is a postgres of the installed package: initdb and pg_ctl into
<work>/<workspace>/run-<id>/pg, as the module's account, listening on a unix socket there and on no TCP address, with
durability off; the tests get MESH_TEST_POSTGRES (the socket directory percent-encoded in the URL, because the
controller's tests cut the URL at its last slash). The bus is a nats-server from the Go library at the version this
module's go.mod names, run by this same binary started as mesh-build-go bus <dir> (bus.go: JetStream on, a
loopback port of its own, its address on the first line, stopped when its input closes); the tests get MESH_TEST_NATS.
Both run inside the step, are stopped, and the run's directory is removed whatever the tests did.
Setup (compiling the tests, the store, the bus) is judged within 30 s and the tests' running time (the wall time of
go test) within 60 s, apart; a level over either says over_bound and names the bound, with the slowest tests for the running
time. A machine whose installed postgres major is not the recipe's refuses this level, naming both; the unit level
still runs. The result names the postgres major and the nats-server version it ran against. The mesh's own store major
is not named: no state or grant gives it to this module (ADR 0470 §4 asks for it when it differs), so only the recipe's
is said until a source exists.
Digest
build_go_digest {workspace, level} answers in its call, for each package of the workspace that has tests at the level,
one digest of everything those tests read (ADR 0445 §5): the files, by content and under their path in the tree, of the
package and every package its test binary reaches (go list -test -deps) that lies in the module's own tree; the version
and checksum of every module outside it; the package's own testdata/ files; and the Go version, the level, the build tags
and, at the integration level, the postgres major and the nats-server version. Packages of the standard library are the Go
version's. A package with no tests at the level has no digest and is named in without_tests. A directory that vendors
is listed and tested from its vendor directory (-mod=vendor).
Build
build_go_build {workspace, module, artifact} builds one Go bundle artifact of a module of the workspace with the build
agent's recipe (mesh-controller's toolchain.go, the Go entry, copied in testdata/captured/toolchain-go-entry.txt, and
builder.go): CGO_ENABLED=0, GOFLAGS=-trimpath -buildvcs=false (with -mod=mod, or -mod=vendor for a directory
that vendors), go build -ldflags "-s -w -X main.builtFor=<system>" -o .mesh-build/<artifact>/<binary> ./<from> in the
module's directory, by the installed Go (GOTOOLCHAIN=local) from the module cache the fetch filled and no network
(GOPROXY=off). The binary is named as the build agent names it. The output directory is packed with mesh-sdk's Pack,
so the digest is the build agent's for the same tree. The step ends with the artifact, the binary, the digest, the size and
the path of the archive, which stays in the workspace's artifacts directory: publishing it is the builder's, in a later
release. An artifact that is not a Go bundle fails the step naming it.