The installer carries a builder and builds the control plane it raises

It carried the thing it was going to run; it now carries the thing that makes
it. One artifact either way — but a mesh raised this way holds a control plane
it built from a repository and a commit it can name, and can therefore build
again. A mesh handed a finished image could not, and had no way to find that
out until somebody needed it to.

A build step sits between load and bundle, because the bundle must name an
image and that image no longer arrives finished. Everything after it is
unchanged: a locally built image is named by the digest of its own
configuration, which is exactly what the carried one was named by.

Refused in preflight when nothing says what to build, so a run that cannot
finish says so before it has changed anything.

.gitignore

@@ -3,4 +3,4 @@
 /dist/
 # The placeholder `make bootstrap` moves aside while a saved image is embedded. Ignored so an
 # interrupted release build cannot commit a twenty-megabyte tar by accident.
-/internal/image/control-plane.tar.placeholder
+/internal/image/builder.tar.placeholder

Makefile

@@ -58,13 +58,17 @@ host:
 	@echo "built for $(SYSTEM) carrying $(BUNDLE)"
 
 # The installer, carrying the control plane's image:
-#   make bootstrap IMAGE=mesh-control:v1.2.3
+#   make bootstrap IMAGE=mesh-builder:v1.2.3
 #
-# The image is BUILT ELSEWHERE and handed over — mesh-control's own `make image` — and embedded
+# **The carried image is the BUILDER** (novox/hq ADR 0073). It used to be the control plane, on the
-# here at release time. Not built on the machine being bootstrapped, and not fetched: the forge
+# argument that the forge holding the source runs on the mesh, so building at genesis would need a
-# that holds mesh-control's source runs on the mesh, so a bootstrap that had to fetch or build the
+# mesh in order to raise one. That argument was about the *control plane's* source, and it is
-# control plane would need a mesh in order to raise one. Carrying it breaks that cycle, the same
+# answered by ADR 0071: the source comes from a mesh that already exists, which is not the one being
-# way carrying the bundle breaks the "copy it onto a machine and run it" one (novox/hq ADR 0005).
+# raised. What cannot be fetched is the thing that does the fetching, and that is what is carried.
+#
+# The image is BUILT ELSEWHERE and handed over — mesh-control's own `make builder-image` — and
+# embedded here at release time, the same way carrying the bundle breaks the "copy it onto a machine
+# and run it" cycle (novox/hq ADR 0005).
 #
 # The saved image occupies the embed slot for the length of one build and the placeholder goes
 # back, exactly as `host:` does with the bundle. Nothing large is ever committed.
@@ -84,15 +88,15 @@ host:
 BOOTSTRAP_OUT ?= mesh-bootstrap
 
 bootstrap:
-	@test -n "$(IMAGE)" || { echo "IMAGE= is required; an installer carrying no control-plane image cannot raise a mesh"; exit 1; }
+	@test -n "$(IMAGE)" || { echo "IMAGE= is required; an installer carrying no builder image cannot raise a mesh"; exit 1; }
 	@case "$(IMAGE)" in sha256:*) echo "IMAGE=$(IMAGE) is an image id. The installer identifies the carried image by its tag, because an id is the digest of a configuration that a runtime rewrites as it loads. Pass a name:tag"; exit 1;; esac
 	@docker image inspect "$(IMAGE)" >/dev/null 2>&1 || { echo "this machine does not hold $(IMAGE) — build it in mesh-control with 'make image'"; exit 1; }
-	@test -n "$$(docker image inspect --format '{{len .RepoTags}}' "$(IMAGE)" | grep -v '^0$$')" || { echo "$(IMAGE) has no repository tag, so the saved archive would carry no name the installer can ask a runtime about. Tag it first: docker tag $(IMAGE) mesh-control:<version>"; exit 1; }
+	@test -n "$$(docker image inspect --format '{{len .RepoTags}}' "$(IMAGE)" | grep -v '^0$$')" || { echo "$(IMAGE) has no repository tag, so the saved archive would carry no name the installer can ask a runtime about. Tag it first: docker tag $(IMAGE) mesh-builder:<version>"; exit 1; }
-	@cp internal/image/control-plane.tar internal/image/control-plane.tar.placeholder
+	@cp internal/image/builder.tar internal/image/builder.tar.placeholder
-	@docker save --output internal/image/control-plane.tar "$(IMAGE)"
+	@docker save --output internal/image/builder.tar "$(IMAGE)"
 	@CGO_ENABLED=0 go build -ldflags="-s -w -X main.version=$(VERSION)" -o "$(BOOTSTRAP_OUT)" ./cmd/mesh-bootstrap; \
 	  status=$$?; \
-	  mv internal/image/control-plane.tar.placeholder internal/image/control-plane.tar; \
+	  mv internal/image/builder.tar.placeholder internal/image/builder.tar; \
 	  exit $$status
 	@echo "built $(BOOTSTRAP_OUT) carrying $(IMAGE)"
 

cmd/mesh-bootstrap/main.go

@@ -51,15 +51,16 @@ const usage = `mesh-bootstrap — make a bare machine into a mesh
   version
 
      1 preflight  what has to be true before anything is changed
-     2 load       the control plane's image, carried in this installer
+     2 load       the builder's image, carried in this installer
-     3 bundle     the substrate, named for this machine
+     3 build      the control plane, from its own repository and a commit
-     4 apply      raise it
+     4 bundle     the substrate, named for this machine
-     5 verify     it is up, and the control plane replies
+     5 apply      raise it
-     6 enrol      this machine becomes the mesh's first node
+     6 verify     it is up, and the control plane replies
-     7 registry   install the module that gives this mesh an image store
+     7 enrol      this machine becomes the mesh's first node
-     8 publish    push the control plane's image into it, for its first digest
+     8 registry   install the module that gives this mesh an image store
-     9 control    reinstall the control plane as an ordinary module, pinned to that digest
+     9 publish    push the control plane's image into it, for its first digest
-    10 retire     drop the temporary control plane; the host removes it
+    10 control    reinstall the control plane as an ordinary module, pinned to that digest
+    11 retire     drop the temporary control plane; the host removes it
 
   --bundle    the substrate template to build this machine's bundle from
               (default ` + defaultTemplate + `)
@@ -67,8 +68,14 @@ const usage = `mesh-bootstrap — make a bare machine into a mesh
               (default ` + defaultOut + `)
   --state     where this node records what it has applied
               (default ` + store.DefaultPath + `)
+  --source    the repository the control plane is built from, on a mesh that
+              already exists — not the one being raised
+  --source-ref  the commit to build. A branch is a moving target somebody else
+              controls, and what is cloned here is the trust anchor for
+              everything this mesh will ever run
+  --source-path  the module's directory inside that repository, if not its root
   --catalog   a checkout of the mesh's catalogue, holding the registry's and the
-              control plane's manifests. Without it this stops after step 5
+              control plane's manifests. Without it this stops after step 6
   --node      the name this machine is known by (default: its hostname)
   --registry  where this mesh keeps its own images (default ` + defaultRegistry + `)
               every node pulls the control plane from this, so on a mesh of more
@@ -83,6 +90,10 @@ const usage = `mesh-bootstrap — make a bare machine into a mesh
   --dry-run   everything that does not change the machine
   --json      machine-readable output
 
+The installer carries a builder, not a control plane. What raises a mesh is therefore
+the same thing that will maintain it, and the control plane a mesh ends up running is
+one it built itself, from a repository and a commit it can name and build again.
+
 Genesis is a pivot: a temporary control plane installs the registry that makes it
 permanent. The temporary one is called temp-mesh-control and the permanent one is
 called mesh-control, so they are two containers with two owners and there is nothing
@@ -175,6 +186,12 @@ func newFlagSet(opts *bootstrap.Options, jsonOut *bool) *flag.FlagSet {
 	set.StringVar(&opts.State, "state", opts.State, "where this node records what it has applied")
 	set.StringVar(&opts.Catalogue, "catalog", opts.Catalogue,
 		"a checkout of the mesh's catalogue; without it this stops after the substrate")
+	set.StringVar(&opts.Source.Repository, "source", opts.Source.Repository,
+		"the repository the control plane is built from, on a mesh that already exists")
+	set.StringVar(&opts.Source.Ref, "source-ref", opts.Source.Ref,
+		"the commit to build; a branch is a moving target somebody else controls")
+	set.StringVar(&opts.Source.Path, "source-path", opts.Source.Path,
+		"the module's directory inside that repository, if not its root")
 	set.StringVar(&opts.Node, "node", opts.Node, "the name this machine is known by")
 	set.StringVar(&opts.Registry, "registry", opts.Registry, "where this mesh keeps its own images")
 	set.StringVar(&opts.Host, "host", opts.Host, "the mesh-host binary on this machine")

internal/bootstrap/bootstrap.go

@@ -43,6 +43,7 @@ type Step string
 const (
 	StepPreflight    Step = "preflight"
 	StepLoad         Step = "load"
+	StepBuild        Step = "build"
 	StepBundle       Step = "bundle"
 	StepApply        Step = "apply"
 	StepVerify       Step = "verify"
@@ -53,14 +54,19 @@ const (
 	StepRetire       Step = "retire"
 )
 
-// Steps in the order they happen, so a failure can say "step 2 of 10".
+// Steps in the order they happen, so a failure can say "step 2 of 11".
 //
-// The first five make a machine; the last five make a mesh that can maintain itself. They are one
+// The first six make a machine; the last five make a mesh that can maintain itself. They are one
-// program because they are one procedure — the whole reason the pivot exists is that steps 7 to 9
+// program because they are one procedure — the whole reason the pivot exists is that steps 8 to 10
-// cannot happen without steps 1 to 5, and steps 1 to 5 leave something that cannot be upgraded
+// cannot happen without steps 1 to 6, and steps 1 to 6 leave something that cannot be upgraded
-// without steps 7 to 9 (novox/hq ADR 0067).
+// without steps 8 to 10 (novox/hq ADR 0067).
+//
+// **Build sits between load and bundle**, because the bundle has to name an image and that image
+// no longer arrives finished. The installer carries the builder, loads it, and uses it to produce
+// the control plane from source (novox/hq ADR 0073) — so what the bundle names is something this
+// mesh made, out of a repository and a commit it can name, and can therefore make again.
 var Steps = []Step{
-	StepPreflight, StepLoad, StepBundle, StepApply, StepVerify,
+	StepPreflight, StepLoad, StepBuild, StepBundle, StepApply, StepVerify,
 	StepEnrol, StepRegistry, StepPublish, StepControlPlane, StepRetire,
 }
 
@@ -118,6 +124,11 @@ type Options struct {
 	// looks installed and cannot upgrade itself.
 	Catalogue string
 
+	// Source is where the control plane is built from — a repository on a mesh that already
+	// exists, and a commit. The installer carries the builder rather than a finished control
+	// plane (novox/hq ADR 0073), so this is what it is told to make.
+	Source Source
+
 	// Registry is where this mesh's own images live, as this machine reaches it. Every node will
 	// pull the control plane from what this says, so on a mesh of more than one machine it must be
 	// an address the others can reach.
@@ -171,6 +182,14 @@ type Result struct {
 	ImageTags []string `json:"image-tags,omitempty"`
 	// ImageHeld is true when the machine already held it and nothing was loaded.
 	ImageHeld bool `json:"image-already-held,omitempty"`
+	// Built is what the genesis build produced, and BuiltFrom is the commit it actually built.
+	//
+	// Reported because they are the difference between a mesh that can rebuild its control plane
+	// and one that cannot: a machine holding these can be asked for the same thing again and get
+	// the same thing back.
+	Built     string `json:"built,omitempty"`
+	BuiltFrom string `json:"built-from,omitempty"`
+
 	// ImagePredicted is true when Image is the archive's id because nothing was loaded — a dry run
 	// only, and the reason a dry run does not claim to know what would be applied.
 	ImagePredicted bool `json:"image-id-is-a-prediction,omitempty"`
@@ -287,7 +306,7 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
 	say("  system            " + sys.Name())
 
 	// ---- 2. load ------------------------------------------------------------------------
-	say("load — the control plane's image, carried in this installer")
+	say("load — the builder's image, carried in this installer")
 	loaded, err := Load(ctx, d.Run, o.DryRun, say)
 	if err != nil {
 		return result, failed(StepLoad, err)
@@ -296,9 +315,23 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
 	result.ImageArchive, result.ImageTag = loaded.Archive, loaded.Tag
 	result.ImagePredicted = loaded.Predicted
 
-	// ---- 3. bundle ----------------------------------------------------------------------
+	// ---- 3. build -----------------------------------------------------------------------
+	say("build — the control plane, from its own repository and a commit")
+	built, err := BuildControlPlane(ctx, d.Run, loaded.Tag, o.Source, o.DryRun, say)
+	if err != nil {
+		return result, failed(StepBuild, err)
+	}
+	result.Built, result.BuiltFrom = built.Module, built.Commit
+	controlPlaneImage := built.Image
+	if o.DryRun {
+		// Nothing was built, so there is no id to name. The carried builder's own is used only so
+		// the remaining steps have something well-formed to describe; nothing is applied.
+		controlPlaneImage = loaded.ID
+	}
+
+	// ---- 4. bundle ----------------------------------------------------------------------
 	say("bundle — what this machine will be asked to be")
-	rewritten, err := Rewrite(template, loaded.ID)
+	rewritten, err := Rewrite(template, controlPlaneImage)
 	if err != nil {
 		return result, failed(StepBundle, err)
 	}

internal/bootstrap/build.go

@@ -0,0 +1,165 @@
+package bootstrap
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"strings"
+)
+
+// Source is where the control plane is built from.
+//
+// **A commit, not a branch** (novox/hq ADR 0071). The forge a mesh installs from is the trust
+// anchor for everything that mesh will ever run, and a branch is a moving target somebody else
+// controls. The installer names what it wants and the builder checks what it got.
+type Source struct {
+	// Repository is the clone URL, on a mesh that already exists. Not the one being raised.
+	Repository string
+	// Ref is the commit to build.
+	Ref string
+	// Path is the module's directory inside that repository. Empty is its root.
+	Path string
+}
+
+// Named reports whether a source was given at all.
+func (s Source) Named() bool { return strings.TrimSpace(s.Repository) != "" }
+
+// Check is whether this installer was told enough to build anything.
+//
+// **Its own function so it can be asked twice**: once in preflight, before the machine has been
+// touched, and once at the build, which is where it would otherwise be discovered. The first is
+// what a person wants — a run that cannot finish should say so before it changes anything — and
+// the second is what keeps the build honest if it is ever called from somewhere else.
+func (s Source) Check() error {
+	if !s.Named() {
+		return errors.New(
+			"this installer carries a builder and was not told what to build. Give it --source " +
+				"(a repository on a mesh that already exists) and --source-ref (a commit). It " +
+				"does not guess: what it clones is the trust anchor for everything this mesh " +
+				"will ever run")
+	}
+	if strings.TrimSpace(s.Ref) == "" {
+		return errors.New(
+			"--source was given without --source-ref. Genesis names a commit, because a branch " +
+				"is a moving target somebody else controls and what is cloned here is the trust " +
+				"anchor for everything this mesh will ever run (novox/hq ADR 0071)")
+	}
+	return nil
+}
+
+// Built is what one genesis build produced.
+type Built struct {
+	// Module is what the manifest called itself, so the installer can say it built the right thing.
+	Module string
+	// Commit is what was actually built, which may not be what was asked for if a ref moved.
+	Commit string
+	// Image is the artifact, named by the digest of its own configuration — the identity a machine
+	// can use with nothing serving it, and the same one the installer used for a carried image.
+	Image string
+}
+
+// builderOutput is the part of the builder's one-shot result this needs.
+type builderOutput struct {
+	Module string `json:"module"`
+	Commit string `json:"commit"`
+	Made   []struct {
+		Name      string `json:"name"`
+		Kind      string `json:"kind"`
+		Reference string `json:"reference"`
+	} `json:"made"`
+}
+
+// BuildControlPlane runs the carried builder once, to produce the control plane from source.
+//
+// **This is the step that makes a raised mesh able to maintain itself** (novox/hq ADR 0073). What
+// comes out is not merely an image: it came from a named repository, a path and a commit, which is
+// the same description every later rebuild of the control plane will use. A mesh raised this way
+// can rebuild the thing that runs it. A mesh handed a finished image cannot, and has no way to
+// discover that until somebody needs it to.
+//
+// The builder is given the machine's container runtime and nothing else. It is not given a registry:
+// there is none yet, and none is needed — the image it produces stays in the runtime of the machine
+// that will run it, which is this one.
+func BuildControlPlane(ctx context.Context, run Runner, builderTag string, source Source,
+	dryRun bool, say func(string)) (Built, error) {
+
+	if err := source.Check(); err != nil {
+		return Built{}, err
+	}
+
+	args := []string{
+		"run", "--rm",
+		// The build runs containers of its own, which is the whole of what it needs.
+		"-v", "/var/run/docker.sock:/var/run/docker.sock",
+		builderTag,
+		"build", source.Repository, "--ref", source.Ref,
+	}
+	if source.Path != "" {
+		args = append(args, "--path", source.Path)
+	}
+
+	say(fmt.Sprintf("building the control plane from %s at %s", source.Repository, shortRef(source.Ref)))
+	if dryRun {
+		say("  dry run: not built")
+		return Built{}, nil
+	}
+
+	out, err := run(ctx, "docker", args...)
+	if err != nil {
+		return Built{}, fmt.Errorf("the control plane could not be built from %s at %s: %w",
+			source.Repository, shortRef(source.Ref), err)
+	}
+
+	// The builder writes its result to standard output and everything else to standard error, so
+	// what is parsed here is the whole of what it said. Trimmed rather than searched: a parser that
+	// hunts for the first `{` will happily read a brace out of a progress line.
+	var result builderOutput
+	if err := json.Unmarshal([]byte(strings.TrimSpace(out)), &result); err != nil {
+		return Built{}, fmt.Errorf(
+			"the builder finished and what it said is not a result: %w. What it said was: %s",
+			err, firstLine(out))
+	}
+
+	var images []string
+	for _, made := range result.Made {
+		if made.Kind == "image" {
+			images = append(images, made.Reference)
+		}
+	}
+	switch len(images) {
+	case 1:
+	case 0:
+		return Built{}, fmt.Errorf(
+			"%s built, and produced no image. The installer raises the control plane from an "+
+				"image, so there is nothing here to raise", result.Module)
+	default:
+		// Refused rather than guessed at. Picking one of several would work until the day the
+		// order changed, and then raise the wrong thing without saying so.
+		return Built{}, fmt.Errorf(
+			"%s produced %d images, and the installer cannot tell which one is the control "+
+				"plane. A module raised at genesis declares exactly one",
+			result.Module, len(images))
+	}
+
+	say(fmt.Sprintf("  built %s from %s", result.Module, shortRef(result.Commit)))
+	return Built{Module: result.Module, Commit: result.Commit, Image: images[0]}, nil
+}
+
+func shortRef(ref string) string {
+	if len(ref) > 8 {
+		return ref[:8]
+	}
+	return ref
+}
+
+func firstLine(s string) string {
+	s = strings.TrimSpace(s)
+	if i := strings.IndexByte(s, '\n'); i >= 0 {
+		return s[:i]
+	}
+	if len(s) > 200 {
+		return s[:200]
+	}
+	return s
+}

internal/bootstrap/build_test.go

@@ -0,0 +1,41 @@
+package bootstrap
+
+import (
+	"strings"
+	"testing"
+)
+
+// An installer that carries a builder and was told nothing refuses, and says what is missing.
+//
+// **Exercised rather than assumed.** This is the refusal that stands between a person and a
+// machine left holding a store, a broker and no control plane — the failure the whole preflight
+// exists to prevent — and a refusal nothing tests is a refusal nobody has read.
+func TestAnInstallerWithNothingToBuildRefuses(t *testing.T) {
+	err := Source{}.Check()
+	if err == nil {
+		t.Fatal("a source naming no repository was accepted; nothing would have been built")
+	}
+	for _, want := range []string{"--source", "--source-ref"} {
+		if !strings.Contains(err.Error(), want) {
+			t.Errorf("the refusal does not name %s, so it does not say how to fix it: %v", want, err)
+		}
+	}
+}
+
+// A repository without a commit refuses too, because a branch is somebody else's moving target.
+func TestABranchIsNotACommit(t *testing.T) {
+	err := Source{Repository: "https://example.invalid/mesh-control.git"}.Check()
+	if err == nil {
+		t.Fatal("a source with no ref was accepted; genesis would have built whatever a branch pointed at")
+	}
+	if !strings.Contains(err.Error(), "--source-ref") {
+		t.Errorf("the refusal does not name the flag that fixes it: %v", err)
+	}
+}
+
+// And a repository with a commit is enough.
+func TestARepositoryAndACommitIsEnough(t *testing.T) {
+	if err := (Source{Repository: "https://example.invalid/mesh-control.git", Ref: "a1b2c3d4"}).Check(); err != nil {
+		t.Fatalf("a repository and a commit were refused: %v", err)
+	}
+}

internal/bootstrap/load.go

@@ -30,7 +30,7 @@ type Loaded struct {
 	Predicted bool
 }
 
-// Load puts the carried control-plane image into this machine's container runtime, and reports
+// Load puts the carried builder image into this machine's container runtime, and reports
 // what the runtime decided to call it.
 //
 // **The digest of a configuration is not portable across runtimes, and that is why the id is read

internal/bootstrap/preflight.go

@@ -32,6 +32,15 @@ func Preflight(ctx context.Context, o Options, d Deps, say func(string)) ([]byte
 	if image.IsEmpty() {
 		return nil, image.ErrEmpty
 	}
+	// And was it told what to build?
+	//
+	// Asked here rather than at the build, for the same reason as the line above: a run that
+	// cannot finish should say so before it has changed anything. The installer carries a builder
+	// and nothing else (novox/hq ADR 0073), so an installer with no source is an installer that
+	// would raise a store and a broker and then have nothing to raise a control plane from.
+	if err := o.Source.Check(); err != nil {
+		return nil, fmt.Errorf("%w. Nothing has been changed on this machine", err)
+	}
 	saved, err := image.Saved()
 	if err != nil {
 		return nil, err

internal/image/image.go

@@ -30,6 +30,12 @@ import (
 
 // The saved image, replaced at release time by `make bootstrap`.
 //
+// **It is the builder, not the control plane** (novox/hq ADR 0073). The installer used to carry
+// the thing it was going to run; it now carries the thing that makes it. One artifact either way —
+// but a mesh raised by the second one holds a control plane it built from source, out of the same
+// repository and path every later rebuild of it will use, and can therefore rebuild it. A mesh
+// raised by the first held an artifact it could not reproduce and knew nothing about.
+//
 // What is committed here is a placeholder, for the same reason `internal/bundle` commits locks
 // that are only comments: `go:embed` refuses to compile against a file that is not there, so a
 // checkout with nothing embedded would not build at all — and someone reading this repository or
@@ -41,18 +47,18 @@ import (
 // the length of one build and then puts the placeholder back — exactly what `make host` does with
 // the bundle it embeds.
 //
-//go:embed control-plane.tar
+//go:embed builder.tar
 var saved []byte
 
-// ErrEmpty means this installer carries no control-plane image.
+// ErrEmpty means this installer carries no builder image.
 //
 // A separate error rather than a message, so the caller can refuse in preflight — before a
 // machine has been touched — instead of discovering it at the load, after the runtime has been
 // probed and a bundle has been written.
 var ErrEmpty = errors.New(
-	"this mesh-bootstrap carries no control-plane image, so it cannot raise a mesh. A release " +
+	"this mesh-bootstrap carries no builder image, so it cannot raise a mesh. A release build " +
-		"build embeds one: `make bootstrap IMAGE=<image>` in the mesh-host repository, where " +
+		"embeds one: `make bootstrap IMAGE=<image>` in the mesh-host repository, where <image> " +
-		"<image> is a control-plane image already built from the mesh-control source")
+		"is a mesh-builder image already built from the mesh-control source")
 
 // IsEmpty reports whether anything was built in.
 //