Add the Node build module's manifest, ssh files, toolchain package, build step and toolchain tests (issues 1488, 1484, 1490), part 15 of 19
This commit is contained in:
+1
-1
@@ -14,7 +14,7 @@
|
||||
// differs. Trimmed of its own paths for the same reason a version comes from where it sits
|
||||
// rather than from the linker: two builds of one commit produce the same bytes.
|
||||
//
|
||||
// **And of one build source at two commits** (novox/hq ADR 0267 rule 6, issue 363): a module
|
||||
// **And of one build source at two commits** (novox/hq ADR 0267 rule 6, issue 363): a module built
|
||||
// from its repository's root is compiled with the clone's `.git` in view, and Go stamps the commit,
|
||||
// its time and whether the tree was modified into the binary unless told not to — so a merge that
|
||||
// changed only a README made a new digest of an unchanged program.
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
# mesh-build-node — the Node and TypeScript build module
|
||||
|
||||
Design 47, release R4c-3 (novox/hq ADR 0405, ADR 0466 §1, ADR 0469 §1 and §4). The module that builds and checks
|
||||
TypeScript code on a build machine, as an account of its own (`runs-as`, no login shell, home at mode 0700), with the
|
||||
packages it declares (`nodejs`, `npm`, `git`, `openssh`) and the capability `user-namespaces`. The bundle is Go
|
||||
(`cmd/mesh-build-node`). It claims no seat: it is assigned to the build machines (ADR 0469 §6) and runs dark until the
|
||||
builder uses it. Its forge key and ssh files are the same as mesh-build-go's (issue 1484).
|
||||
|
||||
## The recipe
|
||||
|
||||
`recipe.go` names the one Node version it builds with (`RecipeNode`, `v26.10.0`, as `node --version` prints it) and the
|
||||
recipe's own version (`RecipeSteps`). The recipe version is a hash of those two and of the bytes of the lock file
|
||||
`files/toolchain/package-lock.json`, so one changed byte of the lock, another Node or another recipe moves it.
|
||||
|
||||
## The toolchain directory
|
||||
|
||||
`files/toolchain/package.json` and its committed `package-lock.json` name typescript 5.9.3, esbuild 0.25.12, @types/node 22.20.1 (the types tsc checks node imports against),
|
||||
@novox/mesh-sdk 0.1.14 and nats 2.29.3. The mesh writes both into `<home>/toolchain/` (resources `toolchain-package`
|
||||
and `toolchain-lock`); the program reads the lock from there. When `<cache dir>/toolchain/<recipe version>` is missing
|
||||
the module runs `npm ci --ignore-scripts` there once (a directory without its `.installed` mark counts as missing).
|
||||
|
||||
**How the registry is reached** (the choice of issue 1488): the lock holds @novox/mesh-sdk at the forge's public
|
||||
address, and npm 12 installs a locked tarball only from the registry it is configured with. The module therefore
|
||||
requires `npm-package-registry` (read access, provided by gitea) and, like the lab module, takes its binding and sealed
|
||||
secret as files in its home. The install gets a private npm config file with the `@novox` scope at the binding's
|
||||
`public-root` (else scheme, address, port and npm path) and the bound account's credential, and removes it afterwards;
|
||||
every other package resolves from the public registry. This is the way today's build agent names it.
|
||||
|
||||
The runtime tree for bundles with externals (the pruned sdk and nats today in `/app/runtime`) is not installed
|
||||
separately: sdk and nats are in this directory's `node_modules`, and the build step (issue 1490) takes them from there.
|
||||
|
||||
## The guard and the health
|
||||
|
||||
At start, every ten minutes, before every step and at every health look, the module reads the lock again and the
|
||||
installed Node (`node --version`) and writes it with the recipe version to `<node>.typescript` of its `toolchains`
|
||||
state. A machine whose Node is not the recipe's refuses every step, the step ending `refused` on its key in `steps` with
|
||||
both versions named, and `build_node_health` (the module's `runtime-health`) answers unhealthy with the same text, so
|
||||
the controller raises the machine's condition. A step runs only after the toolchain directory is installed.
|
||||
|
||||
## Tools
|
||||
|
||||
`build_node_status` and `build_node_health` answer in their call. The step tools run behind the guard and the
|
||||
toolchain install: a long step answers at once with its id and ends on `<node>.<id>` of `steps` (ADR 0469 §2).
|
||||
|
||||
- `build_node_fetch` {repository, commit, ref} (issue 1489): the commit over ssh as the module's account
|
||||
(`buildstep.FetchCommit`, remote `ssh://mesh-forge/<owner>/<name>.git`) into `<work>/<id>/tree`, then for each
|
||||
TypeScript module (a module.json with a typescript artifact) the packages its package.json names beyond the SDK,
|
||||
`typescript` and `@types/*` (the toolchain carries those): none installs nothing; some need a package-lock.json, else
|
||||
the step fails naming the directory, and are installed as the build agent does (scratch copy without the SDK,
|
||||
devDependencies and scripts; `npm ci --omit=dev --omit=peer --ignore-scripts --no-bin-links`; copies of the SDK
|
||||
removed). A dependency on another `@novox/` package fails the step: a fetch holds no registry credential
|
||||
(ADR 0466 §4). Every command runs through `forgekey.ExecRunnerClean(0)` with PATH, HOME, the locale and its own
|
||||
words, nothing of this process's environment. Workspaces older than a day are removed at each fetch.
|
||||
- `build_node_test` {workspace, module, level} (issue 1491): the unit level is the type-check (the toolchain's tsc, the
|
||||
package's `tsconfig.json` or the build agent's flags over its .ts files) and the `test` script; the integration level
|
||||
is the `test:integration` script, the type-check being its setup. Bounds (the Go module's; ADR 0445 names none for
|
||||
TypeScript): unit 5 s, integration 30 s setup and 60 s running. A level over its bound says `over_bound` and passes;
|
||||
a failed one finishes the step with `passed` false and the output (the step fails only when the tests could not be run). Neither script nor .ts files: no tests. The toolchain makes no bin
|
||||
links, so a script runs `node` and files, not `tsc` by name.
|
||||
- `build_node_build` {workspace, module, artifact} (issue 1490): the build agent's tsc and esbuild lines
|
||||
(mesh-controller internal/builder, cited in `build.go`), the launchers, `package.json`, `buildstep.Pack`, and the
|
||||
digest. No network. The toolchain's packages (not typescript or esbuild) are copied to `<work>/<id>/node_modules`,
|
||||
above the tree, which is where tsc, esbuild and node find `@novox/mesh-sdk` and the types by walking up; a copy and
|
||||
not a link, because the mesh makes no symlink. A bundle that keeps an external gets them in its own `node_modules`.
|
||||
The packed archive is written beside the tree as `<artifact>.tar.gz`.
|
||||
@@ -0,0 +1,266 @@
|
||||
package main
|
||||
|
||||
// The build step (novox/hq ADR 0405 §3 and §7, ADR 0469 §4; issue 1490): a TypeScript bundle of a fetched workspace
|
||||
// compiled with the toolchain's tsc, bundled with its esbuild, given its launchers, packed with mesh-sdk's Pack and
|
||||
// ended with the archive's digest. No network. The commands and flags are the build agent's, copied from
|
||||
// mesh-controller origin/main internal/builder: toolchain.go:143-147 (tsc), builder.go:1637-1705 (compile),
|
||||
// :1926-1956 (writeLaunchers), :1971-2039 (bundled), :1735-1747 (sourcesFor). Where they ran in the toolchain image
|
||||
// they run here from the module's directory with the toolchain's real paths, and tsc and esbuild find the SDK in the
|
||||
// workspace's node_modules (workspace.go).
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buildstep"
|
||||
)
|
||||
|
||||
// launcherSuffix is what an entrypoint's launcher is called beside it: index.js is started as index.serve.mjs
|
||||
// (novox/hq ADR 0193).
|
||||
const launcherSuffix = ".serve.mjs"
|
||||
|
||||
// Out is where a bundle's compiled files go in the module's directory (builder toolchain.go:118).
|
||||
func Out(artifact string) string { return ".mesh-build/" + artifact }
|
||||
|
||||
// banner is esbuild's createRequire shim: CommonJS inlined into an ES module still finds `require`.
|
||||
const banner = "--banner:js=import { createRequire as __meshRequire } from 'node:module'; const require = __meshRequire(import.meta.url);"
|
||||
|
||||
// artifact is the part of a module.json artifact the build reads.
|
||||
type artifact struct {
|
||||
Name string `json:"name"`
|
||||
Language string `json:"language"`
|
||||
Entrypoints []string `json:"entrypoints"`
|
||||
External []string `json:"external"`
|
||||
}
|
||||
|
||||
// BuildResult is what a passed build step says.
|
||||
type BuildResult struct {
|
||||
Workspace string `json:"workspace"`
|
||||
Module string `json:"module"`
|
||||
Artifact string `json:"artifact"`
|
||||
// Digest is "sha256:" and the hex of the archive's bytes (buildstep.Pack).
|
||||
Digest string `json:"digest"`
|
||||
Bytes int `json:"bytes"`
|
||||
// Archive is the packed file, in the workspace beside its tree.
|
||||
Archive string `json:"archive"`
|
||||
Files []string `json:"files"`
|
||||
}
|
||||
|
||||
func (m *Module) buildTool() StepTool {
|
||||
return StepTool{
|
||||
Name: "build_node_build",
|
||||
Description: "Build a TypeScript bundle of a fetched workspace: compile it with the toolchain's tsc, bundle each " +
|
||||
"entrypoint with its esbuild, write the launchers and package.json, pack the bundle and answer its digest. " +
|
||||
"Answers a step id at once; the step ends on its key in the steps state with the digest. No network.",
|
||||
Input: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"workspace": map[string]any{"type": "string", "description": "the workspace id a fetch step answered"},
|
||||
"module": map[string]any{"type": "string", "description": "the module's directory in the workspace, '.' when it is the root"},
|
||||
"artifact": map[string]any{"type": "string", "description": "the name of the bundle in the module's module.json"},
|
||||
},
|
||||
"required": []string{"workspace", "artifact"},
|
||||
},
|
||||
Long: true,
|
||||
Run: m.build,
|
||||
}
|
||||
}
|
||||
|
||||
func (m *Module) build(ctx context.Context, args map[string]any) (any, error) {
|
||||
workspace, _ := args["workspace"].(string)
|
||||
module, _ := args["module"].(string)
|
||||
name, _ := args["artifact"].(string)
|
||||
root, dir, err := m.moduleDirOf(workspace, module)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
a, err := readArtifact(dir, name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := m.provide(root); err != nil {
|
||||
return nil, fmt.Errorf("providing the toolchain's packages to the workspace: %w", err)
|
||||
}
|
||||
out, final := Out(a.Name), Out(a.Name)+".bundled"
|
||||
if err := os.MkdirAll(filepath.Join(dir, out), 0o755); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// tsc: builder.go:1637-1705 with toolchain.go:143-147's flags, rooted at the module so an entrypoint lands where
|
||||
// it is named.
|
||||
tsc := []string{m.tsc(), "--module", "NodeNext", "--moduleResolution", "NodeNext", "--target", "ES2022",
|
||||
"--rootDir", ".", "--outDir", out}
|
||||
tsc = append(tsc, sourcesFor(a.Entrypoints, out, ".ts")...)
|
||||
if why := ranFailed(m.runIn(ctx, dir, nil, "node", tsc...)); why != "" {
|
||||
return nil, fmt.Errorf("tsc for %s: %s", a.Name, why)
|
||||
}
|
||||
launchers, err := writeLaunchers(filepath.Join(dir, out), a)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := m.bundle(ctx, dir, a, launchers); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
bundleDir := filepath.Join(dir, final)
|
||||
if len(a.External) > 0 {
|
||||
if err := copyPackages(filepath.Join(m.toolchainDir(), "node_modules"), filepath.Join(bundleDir, "node_modules")); err != nil {
|
||||
return nil, fmt.Errorf("copying the packages %s keeps external: %w", a.Name, err)
|
||||
}
|
||||
}
|
||||
archive, digest, err := buildstep.Pack(bundleDir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("packing %s: %w", a.Name, err)
|
||||
}
|
||||
file := filepath.Join(root, a.Name+".tar.gz")
|
||||
if err := os.WriteFile(file, archive, 0o600); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
files := []string{}
|
||||
_ = filepath.WalkDir(bundleDir, func(p string, d os.DirEntry, err error) error {
|
||||
if err == nil && !d.IsDir() {
|
||||
rel, _ := filepath.Rel(bundleDir, p)
|
||||
files = append(files, filepath.ToSlash(rel))
|
||||
}
|
||||
return nil
|
||||
})
|
||||
sort.Strings(files)
|
||||
return BuildResult{Workspace: workspace, Module: module, Artifact: a.Name, Digest: digest, Bytes: len(archive), Archive: file, Files: files}, nil
|
||||
}
|
||||
|
||||
// bundle makes every entrypoint and every launcher ONE file, with esbuild (builder.go:1971-2039): an entrypoint is
|
||||
// bundled in place under its own name, a launcher keeps its name with the extension .mjs.
|
||||
func (m *Module) bundle(ctx context.Context, dir string, a artifact, launchers map[string]string) error {
|
||||
out, final := Out(a.Name), Out(a.Name)+".bundled"
|
||||
if err := os.RemoveAll(filepath.Join(dir, final)); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Join(dir, final), 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
common := []string{"--bundle", "--platform=node", "--format=esm", "--target=node22",
|
||||
"--outbase=" + out, "--outdir=" + final, "--log-level=warning", banner}
|
||||
for _, x := range a.External {
|
||||
common = append(common, "--external:"+x)
|
||||
}
|
||||
var plain, launch []string
|
||||
for _, e := range a.Entrypoints {
|
||||
if strings.HasSuffix(e, ".js") {
|
||||
plain = append(plain, out+"/"+e)
|
||||
}
|
||||
}
|
||||
for _, l := range sortedValues(launchers) {
|
||||
launch = append(launch, out+"/"+l)
|
||||
}
|
||||
step := func(entries []string, extra ...string) error {
|
||||
if len(entries) == 0 {
|
||||
return nil
|
||||
}
|
||||
args := append(append(append([]string{}, entries...), common...), extra...)
|
||||
if why := ranFailed(m.runIn(ctx, dir, nil, m.esbuild(), args...)); why != "" {
|
||||
return fmt.Errorf("esbuild for %s: %s", a.Name, why)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if err := step(plain); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := step(launch, "--out-extension:.js=.mjs"); err != nil {
|
||||
return err
|
||||
}
|
||||
// Plain `.js` output is an ES module; said once.
|
||||
if err := os.WriteFile(filepath.Join(dir, final, "package.json"), []byte(`{"type":"module","private":true}`+"\n"), 0o644); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, l := range launchers {
|
||||
path := filepath.Join(dir, final, filepath.FromSlash(l))
|
||||
if _, err := os.Stat(path); err == nil {
|
||||
if err := os.Chmod(path, 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// writeLaunchers writes, beside every entrypoint, an executable that imports the entrypoint and serves what it
|
||||
// registered over MCP on stdio (builder.go:1926-1956; novox/hq ADR 0193). Its answer is each entrypoint's launcher,
|
||||
// relative to the bundle's root.
|
||||
func writeLaunchers(root string, a artifact) (map[string]string, error) {
|
||||
out := map[string]string{}
|
||||
for _, entry := range a.Entrypoints {
|
||||
if !strings.HasSuffix(entry, ".js") {
|
||||
continue
|
||||
}
|
||||
launcher := strings.TrimSuffix(entry, ".js") + launcherSuffix
|
||||
body := "#!/usr/bin/env node\n" +
|
||||
"// Written by the mesh's builder (novox/hq ADR 0193): serve what " + entry + " registers,\n" +
|
||||
"// over MCP on stdio, as the module the node's runtime names in MESH_SERVED_MODULE.\n" +
|
||||
"import { serveRegisteredOverStdio } from \"@novox/mesh-sdk/stdio\";\n" +
|
||||
"await import(\"./" + filepath.Base(entry) + "\");\n" +
|
||||
"await serveRegisteredOverStdio();\n"
|
||||
path := filepath.Join(root, filepath.FromSlash(launcher))
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := os.WriteFile(path, []byte(body), 0o755); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := os.Chmod(path, 0o755); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out[entry] = launcher
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// sourcesFor turns compiled entrypoints back into what to compile (builder.go:1735-1747).
|
||||
func sourcesFor(entrypoints []string, out, ext string) []string {
|
||||
sources := make([]string, 0, len(entrypoints))
|
||||
for _, e := range entrypoints {
|
||||
at := strings.TrimPrefix(strings.TrimPrefix(e, out), "/")
|
||||
sources = append(sources, strings.TrimSuffix(at, filepath.Ext(at))+ext)
|
||||
}
|
||||
return sources
|
||||
}
|
||||
|
||||
func sortedValues(m map[string]string) []string {
|
||||
out := make([]string, 0, len(m))
|
||||
for _, v := range m {
|
||||
out = append(out, v)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// readArtifact reads the named artifact of the module.json in dir, which must be a TypeScript bundle with entrypoints.
|
||||
func readArtifact(dir, name string) (artifact, error) {
|
||||
raw, err := os.ReadFile(filepath.Join(dir, "module.json"))
|
||||
if err != nil {
|
||||
return artifact{}, fmt.Errorf("the module has no module.json: %w", err)
|
||||
}
|
||||
var m struct {
|
||||
Build struct {
|
||||
Artifacts []artifact `json:"artifacts"`
|
||||
} `json:"build"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
return artifact{}, fmt.Errorf("module.json is not JSON: %w", err)
|
||||
}
|
||||
for _, a := range m.Build.Artifacts {
|
||||
if a.Name != name {
|
||||
continue
|
||||
}
|
||||
if a.Language != "typescript" {
|
||||
return artifact{}, fmt.Errorf("artifact %q is %q, and this module builds typescript", name, a.Language)
|
||||
}
|
||||
if len(a.Entrypoints) == 0 {
|
||||
return artifact{}, fmt.Errorf("artifact %q names no entrypoints", name)
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
return artifact{}, fmt.Errorf("module.json declares no artifact %q", name)
|
||||
}
|
||||
@@ -0,0 +1,173 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/forgekey"
|
||||
)
|
||||
|
||||
type npmCall struct {
|
||||
name string
|
||||
args []string
|
||||
env []string
|
||||
}
|
||||
|
||||
type fakeNpm struct {
|
||||
mu sync.Mutex
|
||||
calls []npmCall
|
||||
ran forgekey.Ran
|
||||
}
|
||||
|
||||
func (f *fakeNpm) run(_ context.Context, env []string, name string, args ...string) forgekey.Ran {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.calls = append(f.calls, npmCall{name, args, env})
|
||||
if f.ran.Status == 0 && f.ran.Err == "" {
|
||||
// npm ci writes node_modules into --prefix.
|
||||
for i, a := range args {
|
||||
if a == "--prefix" {
|
||||
os.MkdirAll(filepath.Join(args[i+1], "node_modules"), 0o700)
|
||||
}
|
||||
}
|
||||
}
|
||||
return f.ran
|
||||
}
|
||||
|
||||
func toolchainIn(t *testing.T, fake *fakeNpm) Toolchain {
|
||||
t.Helper()
|
||||
home, cache := t.TempDir(), t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(home, "toolchain"), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
os.WriteFile(filepath.Join(home, "toolchain", "package.json"), []byte(`{"name":"x"}`), 0o600)
|
||||
os.WriteFile(filepath.Join(home, "toolchain", "package-lock.json"), []byte(`{"lock":1}`), 0o600)
|
||||
return Toolchain{Paths: Paths{Home: home, Cache: cache}, Run: fake.run,
|
||||
Registry: func() (Registry, error) {
|
||||
return Registry{URL: "https://git.novox.be/api/packages/novox/npm/", Username: "mesh-build-node", Password: "pw"}, nil
|
||||
}}
|
||||
}
|
||||
|
||||
// 1488/AC4: given a missing toolchain directory, npm ci --ignore-scripts runs once in <cache>/toolchain/<recipe
|
||||
// version> with the module's lock; asked again with the directory present, nothing runs.
|
||||
func TestAMissingToolchainDirectoryIsInstalledOnceFromTheLock(t *testing.T) {
|
||||
fake := &fakeNpm{}
|
||||
tc := toolchainIn(t, fake)
|
||||
version := Recipe{Steps: "1", Node: "v26.10.0", Lock: "x"}.Version()
|
||||
dir := filepath.Join(tc.Paths.Cache, "toolchain", version)
|
||||
if tc.Present(version) {
|
||||
t.Fatal("present before any install")
|
||||
}
|
||||
if err := tc.Ensure(context.Background(), version); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(fake.calls) != 1 {
|
||||
t.Fatalf("npm ran %d times, want once", len(fake.calls))
|
||||
}
|
||||
c := fake.calls[0]
|
||||
joined := strings.Join(c.args, " ")
|
||||
if c.name != "npm" || c.args[0] != "ci" || !strings.Contains(joined, "--ignore-scripts") || !strings.Contains(joined, "--no-bin-links") || !strings.Contains(joined, "--prefix "+dir) {
|
||||
t.Errorf("ran %s %v, want npm ci --ignore-scripts --prefix %s", c.name, c.args, dir)
|
||||
}
|
||||
if lock, _ := os.ReadFile(filepath.Join(dir, "package-lock.json")); string(lock) != `{"lock":1}` {
|
||||
t.Errorf("the directory holds lock %q, want the module's", lock)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, ".npmrc-install")); err == nil {
|
||||
t.Errorf("the registry credential file was left in the toolchain directory")
|
||||
}
|
||||
if !tc.Present(version) {
|
||||
t.Errorf("not present after the install")
|
||||
}
|
||||
if err := tc.Ensure(context.Background(), version); err != nil || len(fake.calls) != 1 {
|
||||
t.Errorf("a present directory ran npm again: %d calls, %v", len(fake.calls), err)
|
||||
}
|
||||
}
|
||||
|
||||
// 1488/AC4: the install is told the mesh's registry for @novox with the account's credential, and only through a
|
||||
// config file of its own, never on the command line.
|
||||
func TestTheInstallNamesTheRegistryInAConfigFileNotOnTheCommandLine(t *testing.T) {
|
||||
var npmrc string
|
||||
fake := &fakeNpm{}
|
||||
tc := toolchainIn(t, fake)
|
||||
tc.Run = func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
|
||||
for i, a := range args {
|
||||
if a == "--userconfig" {
|
||||
raw, _ := os.ReadFile(args[i+1])
|
||||
npmrc = string(raw)
|
||||
}
|
||||
}
|
||||
return fake.run(ctx, env, name, args...)
|
||||
}
|
||||
if err := tc.Ensure(context.Background(), "v1"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(npmrc, "@novox:registry=https://git.novox.be/api/packages/novox/npm/\n") ||
|
||||
!strings.Contains(npmrc, "//git.novox.be/api/packages/novox/npm/:_auth=") {
|
||||
t.Errorf("npmrc = %q", npmrc)
|
||||
}
|
||||
if strings.Contains(strings.Join(fake.calls[0].args, " "), "pw") {
|
||||
t.Errorf("the credential is on the command line: %v", fake.calls[0].args)
|
||||
}
|
||||
}
|
||||
|
||||
// 1488/AC4: an install that fails leaves no directory taken for a toolchain, says why, and the next call tries again.
|
||||
func TestAFailedInstallIsNotAToolchain(t *testing.T) {
|
||||
fake := &fakeNpm{ran: forgekey.Ran{Status: 1, Stderr: "E401 unauthorized"}}
|
||||
tc := toolchainIn(t, fake)
|
||||
err := tc.Ensure(context.Background(), "v1")
|
||||
if err == nil || !strings.Contains(err.Error(), "E401") {
|
||||
t.Fatalf("err = %v, want the npm failure named", err)
|
||||
}
|
||||
if tc.Present("v1") {
|
||||
t.Errorf("a failed install is present")
|
||||
}
|
||||
fake.ran = forgekey.Ran{}
|
||||
if err := tc.Ensure(context.Background(), "v1"); err != nil || len(fake.calls) != 2 || !tc.Present("v1") {
|
||||
t.Errorf("the retry: %v, %d calls", err, len(fake.calls))
|
||||
}
|
||||
}
|
||||
|
||||
// 1488/AC4: a module's steps run the install behind the guard: a refusing node installs nothing, an accepting one
|
||||
// installs once however many steps follow.
|
||||
func TestPrepareInstallsOnlyBehindAPassingGuard(t *testing.T) {
|
||||
fake := &fakeNpm{}
|
||||
tc := toolchainIn(t, fake)
|
||||
states := newStates()
|
||||
m := moduleWith(states, "v26.11.0")
|
||||
m.Toolchain = &tc
|
||||
if err := m.Prepare(context.Background()); err == nil || len(fake.calls) != 0 {
|
||||
t.Fatalf("a refusing node: %v, %d npm calls", err, len(fake.calls))
|
||||
}
|
||||
m = moduleWith(states, "v26.10.0")
|
||||
m.Toolchain = &tc
|
||||
for i := 0; i < 3; i++ {
|
||||
if err := m.Prepare(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if len(fake.calls) != 1 {
|
||||
t.Errorf("npm ran %d times for three steps, want once", len(fake.calls))
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegistryFromTheBinding(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
b, s := filepath.Join(dir, "b.json"), filepath.Join(dir, "s")
|
||||
os.WriteFile(s, []byte("secret\n"), 0o600)
|
||||
os.WriteFile(b, []byte(`{"provision":"npm-package-registry","at":"10.42.0.1","as":"mesh-build-node-ace","serves":{"scheme":"http","port":3000,"npm-path":"/api/packages/novox/npm/"}}`), 0o600)
|
||||
r, err := ReadRegistry(b, s)
|
||||
if err != nil || r.URL != "http://10.42.0.1:3000/api/packages/novox/npm/" || r.Username != "mesh-build-node-ace" || r.Password != "secret" {
|
||||
t.Fatalf("= %+v, %v", r, err)
|
||||
}
|
||||
os.WriteFile(b, []byte(`{"at":"h","as":"a","serves":{"port":1,"npm-path":"/p/","public-root":"https://git.novox.be/p"}}`), 0o600)
|
||||
if r, _ := ReadRegistry(b, s); r.URL != "https://git.novox.be/p/" {
|
||||
t.Errorf("the public root is not preferred: %q", r.URL)
|
||||
}
|
||||
if _, err := ReadRegistry(filepath.Join(dir, "none"), s); err == nil {
|
||||
t.Errorf("no binding was accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
Host ${bound:forge-account:ssh_alias}
|
||||
HostName ${bound:forge-account:ssh_host}
|
||||
Port ${bound:forge-account:ssh_port}
|
||||
HostKeyAlias ${bound:forge-account:ssh_alias}
|
||||
User git
|
||||
IdentityFile ~/.ssh/mesh_ed25519
|
||||
IdentitiesOnly yes
|
||||
@@ -0,0 +1,2 @@
|
||||
[url "ssh://${bound:forge-account:ssh_alias}/"]
|
||||
insteadOf = ${bound:forge-account:https_base}
|
||||
@@ -0,0 +1 @@
|
||||
${bound:forge-account:ssh_alias} ${bound:forge-account:host_key}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"name": "mesh-build-node-toolchain",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"description": "The Node build module's toolchain: what npm ci installs into <cache>/toolchain/<recipe version> (issue 1488).",
|
||||
"dependencies": {
|
||||
"@novox/mesh-sdk": "0.1.14",
|
||||
"nats": "2.29.3"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "22.20.1",
|
||||
"esbuild": "0.25.12",
|
||||
"typescript": "5.9.3"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
module meshbuildnode
|
||||
|
||||
go 1.26.0
|
||||
|
||||
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261020203945-1572e50e0ef0
|
||||
@@ -0,0 +1,2 @@
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261020203945-1572e50e0ef0 h1:mr9HyARwzeZT7FxFHv1Xc2z7QErvDZ/F/pBMIpjGSd0=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261020203945-1572e50e0ef0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
|
||||
@@ -0,0 +1,177 @@
|
||||
{
|
||||
"module": "mesh-build-node",
|
||||
"version": "1",
|
||||
"runs-as": "mesh-build-node",
|
||||
"capabilities": [
|
||||
"package-manager",
|
||||
"user-namespaces"
|
||||
],
|
||||
"state": [
|
||||
"toolchains",
|
||||
"keys",
|
||||
"steps"
|
||||
],
|
||||
"tools": [
|
||||
"build_node_fetch",
|
||||
"build_node_test",
|
||||
"build_node_build",
|
||||
"build_node_status",
|
||||
"build_node_health"
|
||||
],
|
||||
"requires": [
|
||||
"forge-account",
|
||||
"npm-package-registry"
|
||||
],
|
||||
"contributes": {
|
||||
"npm-package-registry": {
|
||||
"access": "read"
|
||||
}
|
||||
},
|
||||
"binds": {
|
||||
"npm-package-registry": "${dir:state}/package-registry.json"
|
||||
},
|
||||
"secrets": {
|
||||
"npm-package-registry": "${dir:state}/package-registry.secret"
|
||||
},
|
||||
"own-secrets": {
|
||||
"broker": "${dir:state}/broker"
|
||||
},
|
||||
"secrets-owner": "mesh-build-node",
|
||||
"runtime-health": {
|
||||
"kind": "tool",
|
||||
"tool": "build_node_health",
|
||||
"interval": "60s",
|
||||
"timeout": "10s",
|
||||
"looks": 2,
|
||||
"grace": "60s"
|
||||
},
|
||||
"resources": [
|
||||
{
|
||||
"id": "account",
|
||||
"type": "user",
|
||||
"name": "mesh-build-node",
|
||||
"shell": "/usr/bin/nologin",
|
||||
"home": "/var/lib/mesh-build-node"
|
||||
},
|
||||
{
|
||||
"id": "state",
|
||||
"type": "directory",
|
||||
"mode": "0700",
|
||||
"owner": "mesh-build-node",
|
||||
"place": ".",
|
||||
"unknown": "program"
|
||||
},
|
||||
{
|
||||
"id": "work",
|
||||
"type": "directory",
|
||||
"path": "/var/lib/mesh-build-node-work",
|
||||
"mode": "0700",
|
||||
"owner": "mesh-build-node",
|
||||
"unknown": "program"
|
||||
},
|
||||
{
|
||||
"id": "cache",
|
||||
"type": "directory",
|
||||
"path": "/var/lib/mesh-build-node-cache",
|
||||
"mode": "0700",
|
||||
"owner": "mesh-build-node",
|
||||
"unknown": "program"
|
||||
},
|
||||
{
|
||||
"id": "ssh-dir",
|
||||
"type": "directory",
|
||||
"path": "${dir:state}/.ssh",
|
||||
"mode": "0700",
|
||||
"owner": "mesh-build-node",
|
||||
"unknown": "program"
|
||||
},
|
||||
{
|
||||
"id": "ssh-config",
|
||||
"type": "file",
|
||||
"path": "${dir:ssh-dir}/config",
|
||||
"owner": "mesh-build-node",
|
||||
"mode": "0600",
|
||||
"template": "files/forge"
|
||||
},
|
||||
{
|
||||
"id": "known-hosts",
|
||||
"type": "file",
|
||||
"path": "${dir:ssh-dir}/known_hosts",
|
||||
"owner": "mesh-build-node",
|
||||
"mode": "0644",
|
||||
"template": "files/known_hosts"
|
||||
},
|
||||
{
|
||||
"id": "git-config",
|
||||
"type": "file",
|
||||
"path": "${dir:state}/.config/git/config",
|
||||
"owner": "mesh-build-node",
|
||||
"mode": "0644",
|
||||
"template": "files/gitconfig"
|
||||
},
|
||||
{
|
||||
"id": "toolchain-dir",
|
||||
"type": "directory",
|
||||
"path": "${dir:state}/toolchain",
|
||||
"mode": "0700",
|
||||
"owner": "mesh-build-node",
|
||||
"unknown": "program"
|
||||
},
|
||||
{
|
||||
"id": "toolchain-package",
|
||||
"type": "file",
|
||||
"path": "${dir:toolchain-dir}/package.json",
|
||||
"owner": "mesh-build-node",
|
||||
"mode": "0644",
|
||||
"template": "files/toolchain/package.json"
|
||||
},
|
||||
{
|
||||
"id": "toolchain-lock",
|
||||
"type": "file",
|
||||
"path": "${dir:toolchain-dir}/package-lock.json",
|
||||
"owner": "mesh-build-node",
|
||||
"mode": "0644",
|
||||
"template": "files/toolchain/package-lock.json"
|
||||
},
|
||||
{
|
||||
"id": "nodejs",
|
||||
"type": "package",
|
||||
"package": "nodejs"
|
||||
},
|
||||
{
|
||||
"id": "npm",
|
||||
"type": "package",
|
||||
"package": "npm"
|
||||
},
|
||||
{
|
||||
"id": "git",
|
||||
"type": "package",
|
||||
"package": "git"
|
||||
},
|
||||
{
|
||||
"id": "openssh",
|
||||
"type": "package",
|
||||
"package": "openssh"
|
||||
}
|
||||
],
|
||||
"build": {
|
||||
"artifacts": [
|
||||
{
|
||||
"name": "tools",
|
||||
"kind": "bundle",
|
||||
"language": "go",
|
||||
"system": "arch",
|
||||
"from": "cmd/mesh-build-node",
|
||||
"binary": "mesh-build-node",
|
||||
"loads": [
|
||||
"mesh-build-node"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "files",
|
||||
"kind": "archive",
|
||||
"from": "files"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user