mesh-bootstrap: the first-node procedure, as a program rather than a test
The only complete written-down copy of how a mesh is stood up was an integration test in the lab. That is why every bootstrap gap kept being found late: an install procedure that lives as a test fixture is exercised by whoever writes tests, never by whoever installs. This is that procedure. A separate binary, not a mesh-host subcommand. mesh-host says of itself that it connects to nothing and listens on nothing and that what it applies comes from a file, and that sentence is what makes an always-running root daemon auditable. An installer loads images and interrogates a control plane. Same tier, different program. The control plane's image is carried, not built and not fetched. The forge that holds its source runs on the mesh, so a bootstrap that had to fetch it would need a mesh in order to raise one. Embedding breaks that cycle the way the carried bundle breaks "copy it onto a machine and run it". The image id is read out of the saved tar before the runtime is asked anything, which is what makes the load idempotent: the installer can ask whether the machine already holds exactly this. Five steps, each idempotent and each saying whether it found or changed something, because this is run over and over by somebody getting a machine working. It stops at a running substrate with a control plane that replies — enrolment, the module catalogue and assignment are the next stage and are deliberately absent. Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
This commit is contained in:
@@ -0,0 +1,8 @@
|
||||
This is not a saved image. It is the placeholder that keeps this repository buildable.
|
||||
|
||||
A release build replaces this file with the output of `docker save` and puts it back afterwards:
|
||||
|
||||
make bootstrap IMAGE=mesh-control:<version>
|
||||
|
||||
An installer built with this file present carries no control plane, and says so in preflight
|
||||
rather than getting a machine part-way to being a mesh and stopping.
|
||||
@@ -0,0 +1,173 @@
|
||||
// Package image is the control plane's image, carried inside the installer.
|
||||
//
|
||||
// **Carried rather than built, and that is not an optimisation.** The mesh's forge runs on the
|
||||
// mesh. A bootstrap that needed the control plane's source in order to build it would need a
|
||||
// forge to fetch that source from, and the forge is one of the things the mesh raises — so the
|
||||
// mesh would be required in order to raise the mesh. Embedding the image breaks that cycle the
|
||||
// same way `internal/bundle` breaks it for the declaration: the installer arrives holding
|
||||
// everything a bare machine has to be given, and a bare machine is given one file
|
||||
// (novox/hq ADR 0005).
|
||||
//
|
||||
// It is also the rule the rest of tier 0 already follows. Nobody compiles `mesh-host` on the
|
||||
// machine it will run on; the binary is put there. The image it raises arrives the same way.
|
||||
//
|
||||
// What is embedded is the output of `docker save` — a tar holding the image's config, its layers
|
||||
// and a `manifest.json` naming them. The control plane's image is `FROM scratch` with one static
|
||||
// binary in it (novox/hq ADR 0006), so this is tens of megabytes rather than hundreds.
|
||||
package image
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
_ "embed"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"path"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The saved image, replaced at release time by `make bootstrap`.
|
||||
//
|
||||
// What is committed here is a placeholder, for the same reason `internal/bundle` commits locks
|
||||
// that are only comments: `go:embed` refuses to compile against a file that is not there, so a
|
||||
// checkout with nothing embedded would not build at all — and someone reading this repository or
|
||||
// running `go test ./...` would meet a compile error instead of a program. The placeholder keeps
|
||||
// the tree buildable and makes the absence a thing the installer *says*, at the earliest moment
|
||||
// it can, rather than a thing a compiler says to the wrong person.
|
||||
//
|
||||
// It is small and it is committed. A saved image is not, and `make bootstrap` puts one here for
|
||||
// the length of one build and then puts the placeholder back — exactly what `make host` does with
|
||||
// the bundle it embeds.
|
||||
//
|
||||
//go:embed control-plane.tar
|
||||
var saved []byte
|
||||
|
||||
// ErrEmpty means this installer carries no control-plane image.
|
||||
//
|
||||
// A separate error rather than a message, so the caller can refuse in preflight — before a
|
||||
// machine has been touched — instead of discovering it at the load, after the runtime has been
|
||||
// probed and a bundle has been written.
|
||||
var ErrEmpty = errors.New(
|
||||
"this mesh-bootstrap carries no control-plane image, so it cannot raise a mesh. A release " +
|
||||
"build embeds one: `make bootstrap IMAGE=<image>` in the mesh-host repository, where " +
|
||||
"<image> is a control-plane image already built from the mesh-control source")
|
||||
|
||||
// IsEmpty reports whether anything was built in.
|
||||
//
|
||||
// It asks whether the bytes are a tar rather than comparing them against the placeholder's text,
|
||||
// because the question that matters is "can this be loaded", and a truncated or corrupted embed
|
||||
// answers no to that while matching no placeholder. A tar's first header carries the string
|
||||
// `ustar` at offset 257 and nothing else does by accident.
|
||||
func IsEmpty() bool {
|
||||
const magicAt, magic = 257, "ustar"
|
||||
return len(saved) < magicAt+len(magic) || string(saved[magicAt:magicAt+len(magic)]) != magic
|
||||
}
|
||||
|
||||
// Saved returns the embedded tar, for loading into a container runtime.
|
||||
func Saved() ([]byte, error) {
|
||||
if IsEmpty() {
|
||||
return nil, ErrEmpty
|
||||
}
|
||||
return saved, nil
|
||||
}
|
||||
|
||||
// manifestEntry is the part of a saved image's `manifest.json` this needs.
|
||||
//
|
||||
// One field. The layers are the runtime's business and the repository tags are decoration — what
|
||||
// is wanted is the config, because the digest of the config IS the image id.
|
||||
type manifestEntry struct {
|
||||
Config string `json:"Config"`
|
||||
RepoTags []string `json:"RepoTags"`
|
||||
}
|
||||
|
||||
// ID is the image id the runtime will give this image once it is loaded, read out of the tar.
|
||||
//
|
||||
// **Read here rather than parsed out of what `docker load` prints.** The load prints a sentence
|
||||
// for a person — `Loaded image: name:tag` or `Loaded image ID: sha256:…`, depending on whether the
|
||||
// image was saved with a tag — and a program that scraped it would be depending on which of those
|
||||
// a particular runtime version chose. The id is a fact about the file, available before the
|
||||
// runtime is asked anything, which is also what makes the load idempotent: the installer can ask
|
||||
// whether the machine already holds THIS image before loading it.
|
||||
//
|
||||
// An image id is the sha256 of the image's configuration document (novox/hq ADR 0006, and see
|
||||
// `internal/declaration`'s checkImage). `manifest.json` names that document by its digest — as
|
||||
// `<64hex>.json` in the older layout and `blobs/sha256/<64hex>` in the OCI one — so both forms
|
||||
// reduce to the same sixty-four characters.
|
||||
func ID(saved []byte) (string, error) {
|
||||
manifest, err := fileFromTar(saved, "manifest.json")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
var entries []manifestEntry
|
||||
if err := json.Unmarshal(manifest, &entries); err != nil {
|
||||
return "", fmt.Errorf(
|
||||
"the carried image has a manifest.json this cannot read, so the image it holds "+
|
||||
"cannot be named: %w", err)
|
||||
}
|
||||
// One image, deliberately. A tar holding several would leave the installer choosing which
|
||||
// one is the control plane, and a bootstrap must not be the thing that guesses.
|
||||
if len(entries) != 1 {
|
||||
return "", fmt.Errorf(
|
||||
"the carried image holds %d images, and the installer raises exactly one control "+
|
||||
"plane. Save a single image: `docker save --output … <image>`", len(entries))
|
||||
}
|
||||
|
||||
digest := strings.TrimSuffix(path.Base(entries[0].Config), ".json")
|
||||
if !isSHA256(digest) {
|
||||
return "", fmt.Errorf(
|
||||
"the carried image names its configuration %q, which is not a sha256 digest. An "+
|
||||
"image id is the digest of that configuration, so there is nothing to call this "+
|
||||
"image", entries[0].Config)
|
||||
}
|
||||
return "sha256:" + digest, nil
|
||||
}
|
||||
|
||||
// Tags is what the saved image was called when it was saved, for a person reading a report.
|
||||
//
|
||||
// Decoration, and said so: the installer names the image by its id everywhere it matters, because
|
||||
// a tag is exactly what a pinned bundle may not rely on (novox/hq ADR 0006). This is here so a
|
||||
// report can say which build somebody embedded, which is otherwise sixty-four characters of hex.
|
||||
func Tags(saved []byte) []string {
|
||||
manifest, err := fileFromTar(saved, "manifest.json")
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var entries []manifestEntry
|
||||
if err := json.Unmarshal(manifest, &entries); err != nil || len(entries) == 0 {
|
||||
return nil
|
||||
}
|
||||
return entries[0].RepoTags
|
||||
}
|
||||
|
||||
func fileFromTar(archive []byte, want string) ([]byte, error) {
|
||||
reader := tar.NewReader(bytes.NewReader(archive))
|
||||
for {
|
||||
header, err := reader.Next()
|
||||
if errors.Is(err, io.EOF) {
|
||||
return nil, fmt.Errorf(
|
||||
"the carried image has no %s, so it is not something `docker save` produced. "+
|
||||
"Embed the output of `docker save`, not a layer or a build context", want)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the carried image cannot be read as a tar: %w", err)
|
||||
}
|
||||
if path.Clean(header.Name) == want {
|
||||
return io.ReadAll(reader)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func isSHA256(s string) bool {
|
||||
if len(s) != 64 {
|
||||
return false
|
||||
}
|
||||
for _, c := range s {
|
||||
if (c < '0' || c > '9') && (c < 'a' || c > 'f') {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,152 @@
|
||||
package image
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Each test names the decision it defends (novox/hq ADR 0017).
|
||||
|
||||
// savedImage builds what `docker save` produces, as far as this package reads it.
|
||||
func savedImage(t *testing.T, files map[string]string) []byte {
|
||||
t.Helper()
|
||||
var buffer bytes.Buffer
|
||||
writer := tar.NewWriter(&buffer)
|
||||
for name, content := range files {
|
||||
header := &tar.Header{Name: name, Mode: 0o644, Size: int64(len(content))}
|
||||
if err := writer.WriteHeader(header); err != nil {
|
||||
t.Fatalf("building the fixture: %v", err)
|
||||
}
|
||||
if _, err := writer.Write([]byte(content)); err != nil {
|
||||
t.Fatalf("building the fixture: %v", err)
|
||||
}
|
||||
}
|
||||
if err := writer.Close(); err != nil {
|
||||
t.Fatalf("building the fixture: %v", err)
|
||||
}
|
||||
return buffer.Bytes()
|
||||
}
|
||||
|
||||
func manifest(t *testing.T, config string, tags ...string) string {
|
||||
t.Helper()
|
||||
raw, err := json.Marshal([]manifestEntry{{Config: config, RepoTags: tags}})
|
||||
if err != nil {
|
||||
t.Fatalf("building the fixture: %v", err)
|
||||
}
|
||||
return string(raw)
|
||||
}
|
||||
|
||||
// The image id is read from the FILE, before any runtime is asked anything.
|
||||
//
|
||||
// That is what makes the load idempotent: knowing the id in advance lets the installer ask "do you
|
||||
// already hold exactly this" instead of loading and then finding out. Scraping it from what
|
||||
// `docker load` prints would only be possible after loading, so the second run of an installer
|
||||
// would load again every time and be unable to say it had not.
|
||||
func TestTheImageIdIsReadFromTheSavedFile(t *testing.T) {
|
||||
digest := strings.Repeat("a", 64)
|
||||
// Both layouts `docker save` has used. The older one names the config `<digest>.json`; the OCI
|
||||
// one names it `blobs/sha256/<digest>`. They carry the same sixty-four characters, and a
|
||||
// reader that understood only one would work until somebody upgraded their runtime.
|
||||
for _, config := range []string{digest + ".json", "blobs/sha256/" + digest} {
|
||||
saved := savedImage(t, map[string]string{"manifest.json": manifest(t, config)})
|
||||
id, err := ID(saved)
|
||||
if err != nil {
|
||||
t.Fatalf("config %q: %v", config, err)
|
||||
}
|
||||
if id != "sha256:"+digest {
|
||||
t.Errorf("config %q gave id %q, want sha256:%s", config, id, digest)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheSavedTagsAreReadForAPersonToRecognise(t *testing.T) {
|
||||
saved := savedImage(t, map[string]string{
|
||||
"manifest.json": manifest(t, strings.Repeat("b", 64)+".json", "mesh-control:v1"),
|
||||
})
|
||||
got := Tags(saved)
|
||||
if len(got) != 1 || got[0] != "mesh-control:v1" {
|
||||
t.Errorf("tags = %v, want [mesh-control:v1]", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A tar that is not a saved image is refused with what is wrong, not with a nil id.
|
||||
//
|
||||
// The installer names the control plane by this id in the bundle it writes. An id it could not
|
||||
// read, treated as empty, would produce a bundle naming nothing — refused by the host two steps
|
||||
// later, with a message about a declaration rather than about what somebody embedded.
|
||||
func TestSomethingThatIsNotASavedImageIsRefused(t *testing.T) {
|
||||
notAnImage := savedImage(t, map[string]string{"hello": "world"})
|
||||
if _, err := ID(notAnImage); err == nil {
|
||||
t.Error("a tar with no manifest.json was accepted as a saved image")
|
||||
} else if !strings.Contains(err.Error(), "docker save") {
|
||||
t.Errorf("the refusal does not say what to embed instead: %v", err)
|
||||
}
|
||||
|
||||
if _, err := ID([]byte("this is not a tar at all")); err == nil {
|
||||
t.Error("bytes that are not a tar were accepted")
|
||||
}
|
||||
}
|
||||
|
||||
// Exactly one image. A bootstrap that chose between several would be the thing that guesses which
|
||||
// one is the control plane, and it would guess right until the day somebody saved two.
|
||||
func TestATarHoldingSeveralImagesIsRefused(t *testing.T) {
|
||||
entries, err := json.Marshal([]manifestEntry{
|
||||
{Config: strings.Repeat("a", 64) + ".json"},
|
||||
{Config: strings.Repeat("b", 64) + ".json"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
saved := savedImage(t, map[string]string{"manifest.json": string(entries)})
|
||||
if _, err := ID(saved); err == nil {
|
||||
t.Error("a tar holding two images was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
// An id is a digest or it is nothing. A truncated one names several images, and which one ran
|
||||
// would be whichever the runtime matched first — the same reasoning `internal/declaration` gives
|
||||
// for refusing a short image reference.
|
||||
func TestAConfigThatIsNotADigestIsRefused(t *testing.T) {
|
||||
for _, config := range []string{"config.json", "abc.json", "blobs/sha256/" + strings.Repeat("a", 63)} {
|
||||
saved := savedImage(t, map[string]string{"manifest.json": manifest(t, config)})
|
||||
if _, err := ID(saved); err == nil {
|
||||
t.Errorf("config %q was accepted and is not a digest", config)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The committed placeholder must read as "carries nothing", so an installer built from a plain
|
||||
// checkout says so in preflight rather than getting a machine as far as a running store and
|
||||
// stopping. This is the same guarantee `internal/bundle` makes about a lock file of only comments.
|
||||
func TestAnInstallerBuiltFromAPlainCheckoutCarriesNothing(t *testing.T) {
|
||||
if !IsEmpty() {
|
||||
// Not a failure of this checkout: `make bootstrap` embeds a real image and puts the
|
||||
// placeholder back, so a real image here means a build was interrupted.
|
||||
t.Skip("this checkout has a saved image embedded, so there is no placeholder to check")
|
||||
}
|
||||
if _, err := Saved(); err == nil {
|
||||
t.Fatal("an installer carrying only the placeholder reported it carries an image")
|
||||
}
|
||||
}
|
||||
|
||||
// And "empty" is decided by whether the bytes could be loaded, not by matching the placeholder's
|
||||
// text. A truncated or corrupted embed is equally unloadable and equally worth refusing early.
|
||||
func TestEmptyMeansUnloadableRatherThanEqualToThePlaceholder(t *testing.T) {
|
||||
restore := saved
|
||||
defer func() { saved = restore }()
|
||||
|
||||
saved = []byte("half a tar, cut off")
|
||||
if !IsEmpty() {
|
||||
t.Error("bytes that are not a tar were reported as a carried image")
|
||||
}
|
||||
|
||||
saved = savedImage(t, map[string]string{
|
||||
"manifest.json": manifest(t, strings.Repeat("c", 64)+".json"),
|
||||
})
|
||||
if IsEmpty() {
|
||||
t.Error("a real saved image was reported as no image at all")
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user