novox/mesh-host
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

mesh-bootstrap: the first-node procedure, as a program rather than a test

Browse Source 
The only complete written-down copy of how a mesh is stood up was an integration
test in the lab. That is why every bootstrap gap kept being found late: an install
procedure that lives as a test fixture is exercised by whoever writes tests, never
by whoever installs. This is that procedure.

A separate binary, not a mesh-host subcommand. mesh-host says of itself that it
connects to nothing and listens on nothing and that what it applies comes from a
file, and that sentence is what makes an always-running root daemon auditable. An
installer loads images and interrogates a control plane. Same tier, different
program.

The control plane's image is carried, not built and not fetched. The forge that
holds its source runs on the mesh, so a bootstrap that had to fetch it would need
a mesh in order to raise one. Embedding breaks that cycle the way the carried
bundle breaks "copy it onto a machine and run it". The image id is read out of the
saved tar before the runtime is asked anything, which is what makes the load
idempotent: the installer can ask whether the machine already holds exactly this.

Five steps, each idempotent and each saying whether it found or changed something,
because this is run over and over by somebody getting a machine working. It stops
at a running substrate with a control plane that replies — enrolment, the module
catalogue and assignment are the next stage and are deliberately absent.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
This commit is contained in:
jschoubben
2026-09-10 23:17:30 +02:00
parent 4af9483219
commit b82ab95f74
18 changed files with 2620 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/bootstrap/rewrite.go
+241
View File
@@ -0,0 +1,241 @@
package bootstrap
import (
"bytes"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/novox/mesh-host/internal/declaration"
)
// ControlPlaneID is the resource the installer replaces the image of.
//
// A resource id rather than a container name or a guess at the image, because the id is the one
// thing a declaration promises is stable — it is what lets the store say *this is the same
// resource I applied last time* (`internal/declaration`, Resource.Identity). A bundle that does not
// name one is refused rather than applied without a control plane, which would raise a store and a
// broker and no mesh.
const ControlPlaneID = "control-plane"
// brokerAddressVar is what a token tells an enrolling node to dial.
//
// Not rewritten here — see the note in Rewrite — but reported, because it is the field most likely
// to be wrong on a machine that is not the one the template was written for, and it is wrong in a
// way nothing notices until a second node tries to join.
const brokerAddressVar = "MESH_BROKER_ADDRESS"
// Rewritten is the bundle this machine will apply, and what was done to produce it.
type Rewritten struct {
// Bundle is the produced file's bytes — the template with one image reference replaced,
// comments and all.
Bundle []byte
// Declaration is that bundle, parsed. Carried so the apply and the verify are talking about
// the same document rather than each re-reading the file and hoping.
Declaration *declaration.Declaration
// Was is the image the template named the control plane by; Now is the one it names it by.
Was string
Now string
// Places is how many times that reference appeared, and therefore how many were replaced.
Places int
// Changed is false when the template already named this image — a re-run on a bundle this
// installer produced earlier.
Changed bool
// Kept is every other container image, unchanged, as "<name> <image>". Reported rather than
// assumed: "postgres was left alone" is a claim, and this is the evidence for it.
Kept []string
// Resources is how many things the produced bundle asks for.
Resources int
// BrokerAddress is what the control plane will tell enrolling nodes to dial, or empty.
BrokerAddress string
}
// Rewrite produces the bundle this machine will apply from the template it was given.
//
// **One substitution, and it is textual.** The control plane's image becomes the id of the image
// this machine now holds; nothing else changes. Textual rather than parse-and-re-serialise because
// the produced file has to be *read* — a person getting a machine working must be able to open it,
// see the substrate they recognise, and see exactly one thing different. Re-serialising a parsed
// declaration would drop every comment in the template, and those comments are where the reasons
// live.
//
// **Every place that reference appears, not only the container.** The bundle names the control
// plane's image twice: once as the container that runs `serve`, and once inside the action that
// runs `migrate` to create the contexts' schemas. Replacing only the container would leave the
// migration pointing at an image no registry serves, and the apply would fail in the middle —
// after the store is up, before the broker. They are one image and they move together.
//
// **Third-party images are not touched.** postgres and lavinmq keep the `name@sha256:` references
// the template carries and are pulled from wherever those name (novox/hq ADR 0006). This is
// checked afterwards rather than merely intended: the produced bundle is re-parsed and every other
// container's image is compared against what it was.
//
// **What this deliberately does NOT rewrite:** MESH_BROKER_ADDRESS, the endpoint every enrolment
// token will carry. It differs per machine and it is silently fatal when wrong — a node enrols
// against a dead address and nothing complains until it fails to come back. It belongs to the
// enrolment stage, which is not built yet, so this reports it loudly and leaves it alone rather
// than guessing an address for a machine it has not been told about.
func Rewrite(template []byte, imageID string) (Rewritten, error) {
if !isImageID(imageID) {
return Rewritten{}, fmt.Errorf(
"%q is not an image id. The control plane is named by the digest of its own "+
"configuration — sha256: and sixty-four hex characters — because nothing serves "+
"it and there is no manifest digest to use instead", imageID)
}
before, err := declaration.ParseFileTrusted(template)
if err != nil {
return Rewritten{}, fmt.Errorf("the bundle template is not a declaration: %w", err)
}
control, err := controlPlaneIn(before)
if err != nil {
return Rewritten{}, err
}
out := Rewritten{Was: control.Image, Now: imageID, BrokerAddress: control.Env[brokerAddressVar]}
occurrences := bytes.Count(template, []byte(control.Image))
if occurrences == 0 {
// The parser found the image and the bytes do not contain it, which means the two are
// reading different things. Refused rather than replaced-zero-times-and-reported-success.
return Rewritten{}, fmt.Errorf(
"the control plane's image is %q according to the parsed template, and that text is "+
"not in the file. Nothing was rewritten", control.Image)
}
out.Places = occurrences
switch {
case control.Image == imageID:
// Already this image. The idempotent case, and the one that happens whenever somebody
// re-runs the installer against a bundle it produced earlier.
out.Bundle = template
default:
out.Bundle = bytes.ReplaceAll(template, []byte(control.Image), []byte(imageID))
out.Changed = true
}
// Read back, on the bytes that will actually be applied. Everything above is an intention
// until the produced file is parsed and asked what it says.
after, err := declaration.ParseFileTrusted(out.Bundle)
if err != nil {
return Rewritten{}, fmt.Errorf(
"the bundle this produced is not a declaration, so the substitution broke it: %w", err)
}
out.Declaration, out.Resources = after, len(after.Resources)
produced, err := controlPlaneIn(after)
if err != nil {
return Rewritten{}, err
}
if produced.Image != imageID {
return Rewritten{}, fmt.Errorf(
"the produced bundle still names the control plane %q, not %q", produced.Image, imageID)
}
// And nothing else moved. A substitution on text can in principle catch more than it was
// aimed at, and "postgres was left exactly as it was" is the claim this checks rather than
// asserts.
was := containerImages(before)
for id, image := range containerImages(after) {
if id == ControlPlaneID {
continue
}
if was[id] != image {
return Rewritten{}, fmt.Errorf(
"rewriting the control plane's image also changed %q, from %q to %q. Only the "+
"mesh's own image may move; everything else is somebody else's image at "+
"somebody else's registry", id, was[id], image)
}
out.Kept = append(out.Kept, fmt.Sprintf("%s %s", id, image))
}
sortStrings(out.Kept)
return out, nil
}
// controlPlaneIn finds the container this installer replaces the image of.
func controlPlaneIn(d *declaration.Declaration) (*declaration.Container, error) {
for _, r := range d.Resources {
if r.Identity() != ControlPlaneID {
continue
}
container, ok := r.(*declaration.Container)
if !ok {
return nil, fmt.Errorf(
"this bundle's %q is a %s, and the control plane has to be a container for its "+
"image to be named. Nothing was rewritten", ControlPlaneID, r.Kind())
}
return container, nil
}
return nil, fmt.Errorf(
"this bundle names no %q, so there is no control plane to give this machine's image to. "+
"A substrate without one raises a store and a broker and no mesh. It declares: %s",
ControlPlaneID, strings.Join(identities(d), ", "))
}
func containerImages(d *declaration.Declaration) map[string]string {
images := map[string]string{}
for _, r := range d.Resources {
if container, ok := r.(*declaration.Container); ok {
images[container.ID] = container.Image
}
}
return images
}
func identities(d *declaration.Declaration) []string {
var ids []string
for _, r := range d.Resources {
ids = append(ids, r.Identity())
}
return ids
}
// isImageID is the same shape `internal/declaration` accepts for an image the machine holds. Asked
// here as well so the refusal names the installer's own mistake, rather than surfacing as a
// declaration refusal about a bundle this program wrote.
func isImageID(s string) bool {
const prefix = "sha256:"
if !strings.HasPrefix(s, prefix) || len(s) != len(prefix)+64 {
return false
}
for _, c := range s[len(prefix):] {
if (c < '0' || c > '9') && (c < 'a' || c > 'f') {
return false
}
}
return true
}
func sortStrings(values []string) {
for i := 1; i < len(values); i++ {
for j := i; j > 0 && values[j] < values[j-1]; j-- {
values[j], values[j-1] = values[j-1], values[j]
}
}
}
// writeBundleFile puts the produced bundle where a person can read it, creating the directory it
// lives in.
//
// 0644, and that is deliberate: this file names an image and describes a substrate, and it holds
// the bootstrap credentials the template happens to carry — which are the same ones anybody can
// read in the template itself. It is meant to be read. What must not be world-readable is the
// node's identity, and that lives elsewhere and is written elsewhere (`internal/identity`).
func writeBundleFile(path string, content []byte) error {
if dir := filepath.Dir(path); dir != "" && dir != "." {
if err := os.MkdirAll(dir, 0o755); err != nil {
return fmt.Errorf("cannot make %s to write the produced bundle into: %w", dir, err)
}
}
if err := os.WriteFile(path, content, 0o644); err != nil {
return fmt.Errorf(
"cannot write the produced bundle to %s: %w\nIt is what is about to be applied, and "+
"applying something nobody can read afterwards is how a machine becomes a mystery",
path, err)
}
return nil
}