A compose ask builds the composer command of the repository it names, the node-engine's at its root (ADR 0593 §7)
The module already builds whatever repository, directory and package the ask names; the tests now say so for the node-engine's repository (a Go module at its root, the command mesh-compose) instead of mesh-sdk's composer, which leaves mesh-sdk, and the real-bus test builds a stand-in command in its fetched tree. The README tells the runner's new home and what a build says at its end. Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
This commit is contained in:
@@ -133,10 +133,10 @@ agent's recipe (mesh-controller's `toolchain.go`, the Go entry, copied in `testd
|
||||
`builder.go`): `CGO_ENABLED=0`, `GOFLAGS=-trimpath -buildvcs=false` (with `-mod=mod`, or `-mod=vendor` for a directory
|
||||
that vendors), `go build -ldflags "-s -w -X main.builtFor=<system>" -o .mesh-build/<artifact>/<binary> ./<from>` in the
|
||||
module's directory, by the installed go (`GOTOOLCHAIN=local`) from the module cache the fetch filled and no network
|
||||
(`GOPROXY=off`). The binary is named as the build agent names it. The output directory is packed with mesh-sdk's `Pack`,
|
||||
(`GOPROXY=off`). The binary is named as the build agent names it. The output directory is packed with the build runner's `Pack`,
|
||||
as the build agent packs; two build nodes on the same toolchain give the same digest for the same tree (ADR 0477 §6). The step ends with the artifact, the binary, the digest, the size and
|
||||
the reference of the archive: once the build ended, this process pushes it to the artifact store (novox/hq ADR 0498 §1 §2,
|
||||
mesh-sdk's `buildstep.PushBlob`) under `<module>/<artifact>` by its digest, a candidate's as a trunk build's, and answers
|
||||
the build runner's `buildstep.PushBlob`) under `<module>/<artifact>` by its digest, a candidate's as a trunk build's, and answers
|
||||
`artifact-store://<module>/<artifact>/blobs/<digest>`; a push that fails fails the step `not stored: <artifact>: <the
|
||||
store's words>`. The packed bytes never stay on the node. The store's address is `${dir:state}/registry`, placed by the
|
||||
manifest, which requires `artifact-store`. The go build runs shut off in bubblewrap (ADR 0448, `isolate.go`): no
|
||||
@@ -148,3 +148,20 @@ directory named by `from`, or a file pinned by `url` and `sha256`, packed with t
|
||||
(`testdata/captured/README` says how its digest was captured). A pinned file is fetched, checked against its sha256 (a
|
||||
mismatch fails the step naming both digests) and laid out as the artifact says (`file`, or unpacked from `within`; gzip, bzip2
|
||||
and xz, read in Go as the build agent reads it, issue 1590); it runs no command and needs no Go.
|
||||
|
||||
## The runner's package, and what a build says at its end (ADR 0593)
|
||||
|
||||
- The build runner (the pull of asks, the step lifecycle, the toolchain guard, the fetch, the parsers of `go test -json` and
|
||||
`node --test`, `Pack`, the conformance kit and the `mesh-pack` command) is this module's package
|
||||
`git.novox.be/novox/mesh-catalog/modules/mesh-build-go/buildstep`, a Go module of its own in `buildstep/`. mesh-build-node
|
||||
and mesh-build-image require it and replace it to the sibling directory, since it is the same repository. What the delivery
|
||||
parts read of a step (the failure contract, the fetch request, the stale-step sweep, the step id) stays in mesh-sdk's
|
||||
`buildstep`.
|
||||
- `build_go_imports` (a tool, and the last step of a build ask that built an artifact) answers the mesh packages the module's
|
||||
Go programs import: `go list -deps` of each Go bundle's program, tests left out, the packages of the forge outside the
|
||||
module's own directory (a sibling package of its own repository counts), sorted and each once. A build ask carries them as
|
||||
`BuildAnswer.Imports`; the indexer takes the module's fingerprint over those packages' content.
|
||||
- A merge preview (`build_go_fetch` with a merge) answers raw trees: each module directory's git tree, its manifest and lock
|
||||
digests and the objects of the paths its artifacts name. The indexer computes the named tree.
|
||||
- A package of the mesh's forge counts in a test digest by the content of the files the tests reach, not by its version, so a
|
||||
merge of a package whose reached files are unchanged keeps every kept verdict.
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
package main
|
||||
|
||||
// **A compose ask runs mesh-sdk's composer at a commit** (novox/hq ADR 0482 clause 4, issues 1621 and 1622): the
|
||||
// assigner asks it, to compose a group with its mesh-sdk member's head, and to read the group's manifests with the
|
||||
// mesh-sdk the running controller and node-engine pin (ADR 0482 clause 5, issue 1623). The module fetches the commit
|
||||
// **A compose ask runs a composer command at a commit** (novox/hq ADR 0482 clause 4, issues 1621 and 1622): the command is
|
||||
// the node-engine's, mesh-compose in its repository's Go module at the root, since the node-engine alone composes
|
||||
// (ADR 0593 §7); the ask names the repository, the directory and the package, and this module builds whatever it names. A
|
||||
// group that changes the node-engine's composition is composed by the command built from its head, as the running
|
||||
// engine cannot judge its own next code. The module fetches the commit
|
||||
// (its fetch step: the forge key and the network, none of the change's code), builds the composer package in the Go
|
||||
// module the ask names, in a run step shut off with nothing fetched, and keeps the program under its key
|
||||
// (delivery.ComposerKey), so each commit is built once on a node. It fills the request's manifests from the delivery
|
||||
|
||||
@@ -23,21 +23,25 @@ import (
|
||||
"git.novox.be/novox/mesh-sdk/go/testbus"
|
||||
)
|
||||
|
||||
// sdkPin is the mesh-sdk version this module's go.mod requires: the composer the test builds is that version's.
|
||||
func sdkPin(t *testing.T) string {
|
||||
t.Helper()
|
||||
raw, err := os.ReadFile("../../go.mod")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
// composerSource is a stand-in for the node-engine's composer command: it reads -request <file> and answers a verdict
|
||||
// accepting the one manifest, but only when the request's manifest was filled (no digest left in it).
|
||||
const composerSource = `package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func main() {
|
||||
raw, err := os.ReadFile(os.Args[2])
|
||||
if err != nil || strings.Contains(string(raw), "\"digest\"") {
|
||||
fmt.Println(` + "`" + `{"v":1,"sdk":"stand-in","cases":[{"name":"unfilled","manifests":{}}]}` + "`" + `)
|
||||
return
|
||||
}
|
||||
for _, l := range strings.Split(string(raw), "\n") {
|
||||
if f := strings.Fields(l); len(f) >= 2 && f[0] == "git.novox.be/novox/mesh-sdk/go" {
|
||||
return f[1]
|
||||
}
|
||||
}
|
||||
t.Fatal("go.mod requires no mesh-sdk")
|
||||
return ""
|
||||
fmt.Println(` + "`" + `{"v":1,"sdk":"stand-in","cases":[{"name":"change","manifests":{"demo":{"accepted":true}}}]}` + "`" + `)
|
||||
}
|
||||
`
|
||||
|
||||
// goEnvOf is one value of this machine's go env.
|
||||
func goEnvOf(t *testing.T, name string) string {
|
||||
@@ -49,11 +53,11 @@ func goEnvOf(t *testing.T, name string) string {
|
||||
return strings.TrimSpace(string(out))
|
||||
}
|
||||
|
||||
// 1622/AC1, AC3 against the real bus: a compose ask put on the Go build seat's check stream is pulled by the module
|
||||
// started whole on that bus, which fetches the commit (here a tree of a module requiring mesh-sdk at this module's pin,
|
||||
// as a reader repository is), builds mesh-sdk's composer there with nothing fetched, fills the request's manifests from
|
||||
// the indexer's state and runs the composer; the step key <node>.<ask id> passes with the composer's verdict. A second
|
||||
// ask of the same commit runs the kept composer: fetched and built once.
|
||||
// 1622/AC1, AC3 against the real bus (ADR 0593 §7, C9): a compose ask put on the Go build seat's check stream is pulled by
|
||||
// the module started whole on that bus, which fetches the commit (here a tree whose Go module is at the repository's root,
|
||||
// as the node-engine's repository is), builds the composer command the ask names there with nothing fetched, fills the
|
||||
// request's manifests from the indexer's state and runs the composer; the step key <node>.<ask id> passes with the
|
||||
// composer's verdict. A second ask of the same commit runs the kept composer: fetched and built once.
|
||||
func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
|
||||
nc, err := nats.Connect(testbus.URL(t))
|
||||
if err != nil {
|
||||
@@ -84,10 +88,6 @@ func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
|
||||
read, digests := indexerHolds(t, `{"module":"demo","version":"1"}`)
|
||||
m.ReadManifest = read
|
||||
|
||||
gosum, err := os.ReadFile("../../go.sum")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
fetches := 0
|
||||
steps := fakeSteps(new([]string))
|
||||
for i := range steps {
|
||||
@@ -99,11 +99,13 @@ func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
|
||||
if err := os.MkdirAll(tree, 0o700); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
mod := "module reader\n\ngo 1.26.0\n\nrequire git.novox.be/novox/mesh-sdk/go " + sdkPin(t) + "\n"
|
||||
if err := os.WriteFile(filepath.Join(tree, "go.mod"), []byte(mod), 0o600); err != nil {
|
||||
if err := os.MkdirAll(filepath.Join(tree, "cmd", "mesh-compose"), 0o700); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"workspace": ws}, os.WriteFile(filepath.Join(tree, "go.sum"), gosum, 0o600)
|
||||
if err := os.WriteFile(filepath.Join(tree, "go.mod"), []byte("module reader\n\ngo 1.26.0\n"), 0o600); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"workspace": ws}, os.WriteFile(filepath.Join(tree, "cmd", "mesh-compose", "main.go"), []byte(composerSource), 0o600)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -113,7 +115,7 @@ func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
|
||||
|
||||
ask := func() delivery.Step {
|
||||
a := composeAskOf(t, digests[0])
|
||||
a.Compose.Fetch["repository"], a.Compose.Dir = "novox/mesh-controller", delivery.RootPath
|
||||
a.Compose.Fetch["repository"], a.Compose.Dir, a.Compose.Package = "novox/mesh-host", delivery.RootPath, "reader/cmd/mesh-compose"
|
||||
raw, err := delivery.EncodeStepAsk(a)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -147,9 +149,8 @@ func TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus(t *testing.T) {
|
||||
if err := json.Unmarshal(ans.Verdict, &v); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ans.Built != wantBuilt || !strings.HasPrefix(ans.Key, "composer-") || len(v.Cases) != 1 || !v.Cases[0].Manifests["demo"].Accepted ||
|
||||
!strings.Contains(v.SDK, sdkPin(t)) {
|
||||
t.Fatalf("ask %d: answer built %v key %s, verdict %+v; want built %v and demo accepted by mesh-sdk %s", n+1, ans.Built, ans.Key, v, wantBuilt, sdkPin(t))
|
||||
if ans.Built != wantBuilt || !strings.HasPrefix(ans.Key, "composer-") || len(v.Cases) != 1 || !v.Cases[0].Manifests["demo"].Accepted || v.SDK != "stand-in" {
|
||||
t.Fatalf("ask %d: answer built %v key %s, verdict %+v; want built %v and demo accepted by the stand-in composer", n+1, ans.Built, ans.Key, v, wantBuilt)
|
||||
}
|
||||
}
|
||||
if fetches != 1 {
|
||||
|
||||
@@ -13,8 +13,13 @@ import (
|
||||
sdkstep "git.novox.be/novox/mesh-sdk/go/buildstep"
|
||||
"git.novox.be/novox/mesh-sdk/go/composer"
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
"git.novox.be/novox/mesh-sdk/go/forgekey"
|
||||
)
|
||||
|
||||
// nodeEngineComposer is the composer command of the node-engine's repository, whose Go module is at its root (ADR 0593 §7):
|
||||
// a group that changes the node-engine's composition is composed by this program built at the group's head.
|
||||
const nodeEngineComposer = "github.com/novox/mesh-host/cmd/mesh-compose"
|
||||
|
||||
const composeCommit = "4444444444444444444444444444444444444444"
|
||||
|
||||
// indexerHolds is the delivery indexer's manifests state holding each body under its digest, as the indexer writes it.
|
||||
@@ -123,3 +128,55 @@ func TestTheModuleReadsTheIndexersManifests(t *testing.T) {
|
||||
t.Errorf("reads = %v, lacks %s", m.Reads, ManifestsState)
|
||||
}
|
||||
}
|
||||
|
||||
// ADR 0593 §7 (C9): a compose ask naming the node-engine's repository builds its composer command in the Go module at the
|
||||
// repository's root, under a key of that repository, commit and package, and runs the program over the request with its
|
||||
// manifests filled; the module's answer is the program's verdict whole.
|
||||
func TestAComposeAskBuildsTheNodeEnginesComposerAtItsRoot(t *testing.T) {
|
||||
m := composeModule(t)
|
||||
read, digests := indexerHolds(t, `{"module":"demo","version":"1"}`)
|
||||
m.ReadManifest = read
|
||||
a := composeAskOf(t, digests[0])
|
||||
a.Compose.Fetch["repository"], a.Compose.Dir, a.Compose.Package = "novox/mesh-host", delivery.RootPath, nodeEngineComposer
|
||||
tree := ""
|
||||
var built []string
|
||||
real := m.Run
|
||||
m.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
|
||||
if name != "go" {
|
||||
return real(ctx, env, name, args...)
|
||||
}
|
||||
built = args
|
||||
for i, arg := range args {
|
||||
if arg == "-o" {
|
||||
script := "#!/bin/sh\necho '{\"v\":1,\"sdk\":\"node-engine\",\"cases\":[{\"name\":\"change\",\"manifests\":{\"demo\":{\"accepted\":true}}}]}'\n"
|
||||
if err := os.WriteFile(args[i+1], []byte(script), 0o700); err != nil {
|
||||
return forgekey.Ran{Status: 1, Stderr: err.Error()}
|
||||
}
|
||||
}
|
||||
}
|
||||
return forgekey.Ran{}
|
||||
}
|
||||
fetch := func(context.Context, map[string]any) (any, error) {
|
||||
tree = filepath.Join(m.Work, "ws0000000000b", "tree")
|
||||
if err := os.MkdirAll(tree, 0o700); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"workspace": "ws0000000000b"}, os.WriteFile(filepath.Join(tree, "go.mod"), []byte("module github.com/novox/mesh-host\n"), 0o600)
|
||||
}
|
||||
out, err := m.compose(context.Background(), fetch, a)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ans := out.(delivery.ComposeAnswer)
|
||||
key := delivery.ComposerKey("novox/mesh-host", composeCommit, delivery.RootPath, nodeEngineComposer, delivery.ToolchainUsed{Version: "go1.26.0", Recipe: m.Recipe.Version()})
|
||||
var v composer.Verdict
|
||||
if err := json.Unmarshal(ans.Verdict, &v); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(built) < 3 || built[0] != "-C" || built[1] != tree || built[len(built)-1] != nodeEngineComposer {
|
||||
t.Errorf("go ran as %v, want it to build %s in the tree's root %s", built, nodeEngineComposer, tree)
|
||||
}
|
||||
if !ans.Built || ans.Key != key || v.SDK != "node-engine" || !v.Cases[0].Manifests["demo"].Accepted {
|
||||
t.Errorf("answer %+v, verdict %+v, want built under %s and demo accepted", ans, v, key)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -62,7 +62,7 @@ toolchain install: a long step answers at once with its id and ends on `<node>.<
|
||||
last use (ADR 0509), never while a step holds it. Every step is refused, `refused: <node> has N GiB free under the
|
||||
floor of 20 GiB (on <dir>)`, and the health is unhealthy with that text, while the work or the toolchain cache
|
||||
filesystem has less than 20 GiB free (issue 1839).
|
||||
The arguments are mesh-sdk's `buildstep.FetchRequest` (ADR 0499 §4): with `dir` and `paths` (from the builder, never
|
||||
The arguments are the build runner's `buildstep.FetchRequest` (ADR 0499 §4): with `dir` and `paths` (from the builder, never
|
||||
from a manifest) the workspace is narrowed to those paths, the manifest and the lock files before anything else. A
|
||||
member is built against the SDK version its own lock names (the lead's decision L10 of R4e, ADR 0500 §5-§6): the
|
||||
fetch places that version in the module's `node_modules/@novox/mesh-sdk`, from the group's package when the call
|
||||
|
||||
Reference in New Issue
Block a user