novox/mesh-host
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

bootstrap: the rest of the pivot — enrol, registry, publish, reinstall, retire

Browse Source 
Steps 6 to 10, which turn a substrate into a mesh that can maintain itself
(novox/hq ADR 0067).

 6 enrol      a node record, a token, `mesh-host enrol`, and the host agent
              running. Proved by the mesh having HEARD from the node, not by a
              process existing: a host that cannot reach the broker looks exactly
              like a successful install until the first push applies nothing.
 7 registry   the module that gives this mesh an image store, registered from a
              --catalog checkout, assigned and pushed. Its image is upstream and
              never built (04-ISSUES/029) — a placeholder digest there is refused.
              Verified by asking `/v2/`, because a container that is up is not a
              registry that serves.
 8 publish    the carried image pushed into that registry, which assigns it the
              first manifest digest it has ever had. This is the hinge: without
              it the mesh works and can never upgrade itself.
 9 control    the control plane registered as an ordinary module pinned to that
              digest, with the substrate's own store connections delivered
              through `secret accept` — read out of the bundle that made them,
              because the mesh cannot invent a credential that predates it.
10 retire     the temporary control plane dropped from the bundle and removed by
              the host's ordinary removal pass.

Every step asks before it acts and reports "already done". No step leaves the
machine without a control plane: steps 9 and 10 overlap deliberately, and two
stateless control planes are untidy rather than broken.

mesh-control's `internal/builder`.PublishImage is mirrored rather than imported —
tier 0 depends on nothing that must be installed first — with one correction: the
digest is chosen from RepoDigests by repository instead of taken as element zero,
so an image pushed to two registries cannot silently pin this mesh to the wrong
one.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
This commit is contained in:
jschoubben
2026-09-10 23:59:57 +02:00
parent af953dbb0d
commit f534cf8b42
15 changed files with 2910 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/bootstrap/bootstrap.go
+203 -26
View File
@@ -25,6 +25,7 @@ package bootstrap
import (
"context"
"fmt"
"strings"
"time"
)
@@ -33,15 +34,28 @@ import (
type Step string
const (
StepPreflight Step = "preflight"
StepLoad Step = "load"
StepBundle Step = "bundle"
StepApply Step = "apply"
StepVerify Step = "verify"
StepPreflight Step = "preflight"
StepLoad Step = "load"
StepBundle Step = "bundle"
StepApply Step = "apply"
StepVerify Step = "verify"
StepEnrol Step = "enrol"
StepRegistry Step = "registry"
StepPublish Step = "publish"
StepControlPlane Step = "control-plane"
StepRetire Step = "retire"
)
// Steps in the order they happen, so a failure can say "step 2 of 5".
var Steps = []Step{StepPreflight, StepLoad, StepBundle, StepApply, StepVerify}
// Steps in the order they happen, so a failure can say "step 2 of 10".
//
// The first five make a machine; the last five make a mesh that can maintain itself. They are one
// program because they are one procedure — the whole reason the pivot exists is that steps 7 to 9
// cannot happen without steps 1 to 5, and steps 1 to 5 leave something that cannot be upgraded
// without steps 7 to 9 (novox/hq ADR 0067).
var Steps = []Step{
StepPreflight, StepLoad, StepBundle, StepApply, StepVerify,
StepEnrol, StepRegistry, StepPublish, StepControlPlane, StepRetire,
}
// Error is a failure, named by the step it happened in.
type Error struct {
@@ -86,8 +100,37 @@ type Options struct {
// Wait is how long something that is merely starting is given: a socket-activated container
// runtime, a control plane opening its stores.
Wait time.Duration
// Node is the name this machine is known by in the mesh. Everything after the substrate names
// it: the record, the token, the assignment, the push.
Node string
// Catalogue is a checkout of the mesh's catalogue repository, which is where the registry's and
// the control plane's manifests are read from. Empty stops the installer after the substrate:
// there is no pivot without manifests, and pretending otherwise would leave a machine that
// looks installed and cannot upgrade itself.
Catalogue string
// Registry is where this mesh's own images live, as this machine reaches it. Every node will
// pull the control plane from what this says, so on a mesh of more than one machine it must be
// an address the others can reach.
Registry string
// Host is the `mesh-host` binary on this machine — the program that enrols and then holds the
// machine to what the mesh says. The installer runs it; it does not contain it.
Host string
// HostService is the unit that supervises it. Started and enabled, never written: what a unit
// says is a packaging decision, and an installer inventing one would put a file on the machine
// that whatever installed the host will disagree with.
HostService string
// HostInBackground starts the host unsupervised instead, which is what the lab does and what no
// real machine should do — it does not survive a reboot.
HostInBackground bool
}
// pivots reports whether this run goes past the substrate.
func (o Options) pivots() bool { return strings.TrimSpace(o.Catalogue) != "" }
// Deps are the ways this program reaches outside itself. Injected so the whole of it can be
// tested without a container runtime, a network, or a machine to break — the same reason
// `internal/apply` takes a Runner (novox/hq ADR 0017).
@@ -97,6 +140,10 @@ type Deps struct {
// Dial reports whether a TCP address answers, for "can this machine reach the registries the
// bundle names".
Dial func(ctx context.Context, address string) error
// Fetch asks an HTTP endpoint and reports what it said. Used only against the mesh's own
// registry: a container that is up is not a registry that serves, and `/v2/` is the one
// question whose answer means it is.
Fetch func(ctx context.Context, url string) (int, string, error)
}
// Result is what the bootstrap did, in the shape `--json` prints.
@@ -123,10 +170,34 @@ type Result struct {
// Running is the substrate's containers, confirmed up.
Running []string `json:"running,omitempty"`
// Answered is what the control plane said back — not merely that it is up.
Answered string `json:"control-plane,omitempty"`
// Answered is what the temporary control plane said back — not merely that it is up.
Answered string `json:"temporary-control-plane,omitempty"`
// Temporary is what the substrate's control plane is called, which is not what the module's is.
Temporary string `json:"temporary-container,omitempty"`
// Stopped names why a dry run went no further. Empty on a real run.
// Node is this machine's name in the mesh, and how it came to be enrolled and heard from.
Node string `json:"node,omitempty"`
NodeAdded bool `json:"node-record-created,omitempty"`
Enrolled bool `json:"enrolled-now,omitempty"`
Agent string `json:"host-agent,omitempty"`
// Registry is the mesh's own artifact store, once it answers.
Registry string `json:"registry,omitempty"`
RegistryReplied int `json:"registry-replied,omitempty"`
RegistryKnown bool `json:"registry-already-registered,omitempty"`
RegistryRunning string `json:"registry-container,omitempty"`
PublishedAs string `json:"control-plane-image,omitempty"`
PublishedAlready bool `json:"control-plane-image-already-published,omitempty"`
// Permanent is the control plane as an ordinary module.
Permanent string `json:"permanent-container,omitempty"`
PermanentAnswered string `json:"permanent-control-plane,omitempty"`
StoresDelivered []string `json:"stores-delivered,omitempty"`
TemporaryRetired bool `json:"temporary-retired,omitempty"`
TemporaryWasGone bool `json:"temporary-was-already-gone,omitempty"`
TemporaryRemovedAt int `json:"resources-removed,omitempty"`
// Stopped names why a run went no further. Empty on a run that pivoted.
Stopped string `json:"stopped,omitempty"`
}
@@ -141,9 +212,41 @@ type Result struct {
// answer to it is to run this again: re-running is the retry, and it is one a person chooses after
// reading which step failed and why.
//
// **What this does NOT do: enrolment, the module catalogue, and assignment.** It stops at a running
// substrate with a control plane that replies — a mesh of one node with nothing joined to it. See
// the marker at the end.
// **Genesis is a pivot** (novox/hq ADR 0067). Steps 1 to 5 raise a substrate whose control plane is
// named by the digest of its own configuration, because nothing has ever served that image and
// nothing could have. Steps 6 to 10 turn that into a mesh that can maintain itself: this machine
// enrols, the registry module is installed, the carried image is pushed INTO that registry — which
// gives it a manifest digest, its first — and the control plane is reinstalled as an ordinary
// module pinned to it. The temporary one is then dropped from the bundle and the host removes it.
//
// **What makes the last part expressible is a name.** The substrate's control plane is called
// `temp-mesh-control` and the module's is called `mesh-control`. Two containers, two owners:
// nothing is handed over, nothing has to stop being owned without being destroyed, and destruction
// by omission is the right end for something named "temp".
//
// Without --catalog it stops after step 5 and says so, because there are no manifests to install
// and a machine that looks installed and cannot upgrade itself is worse than one that stopped.
//
// **What an interruption leaves, at every step, and how a re-run continues.** This matters more
// here than anywhere else in the repository, because a machine left without a control plane cannot
// be fixed remotely — so no step may leave one:
//
// 1–3 nothing on the machine but a written file. Re-run: the bundle is produced again.
// 4 a partly-raised substrate, recorded in the state file. Re-run: apply converges the rest.
// 5 everything up; something did not answer yet. Re-run: it is asked again.
// 6 a node record and possibly a spent token. Re-run: `node list` finds the record, the
// identity file says whether this machine enrolled, and a fresh token is issued if not.
// 7 the registry registered, assigned, maybe not applied. Re-run: registered again (an
// upsert), pushed again, waited for again. The temporary control plane is untouched.
// 8 the image pushed and the digest unread. Re-run: the registry is asked what it holds and
// the answer is the same digest; nothing is pushed twice.
// 9 the module registered and the container not yet up, OR up beside the temporary one. Both
// are working states: the mesh has a control plane throughout. Re-run: continues.
// 10 the bundle rewritten and the container still there. Re-run: the bundle already omits it
// and the apply removes it; a bundle that already omits it reports "already dropped".
//
// The only step that cannot be undone by re-running is enrolment, and that is refused rather than
// repeated: a second identity is one the mesh does not know, and the mesh believes the first.
func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, error) {
if say == nil {
say = func(string) {}
@@ -181,12 +284,21 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
return result, failed(StepBundle, err)
}
result.BundleWas, result.BundlePlaces, result.Bundle = rewritten.Was, rewritten.Places, o.Out
result.Temporary = rewritten.TempName
if rewritten.Renamed {
say(fmt.Sprintf(" control plane %s, renamed from %s",
rewritten.TempName, rewritten.WasCalled))
say(" the permanent one is a module and takes the plain name; " +
"this one is dropped at the end")
} else {
say(" control plane " + rewritten.TempName + " — the template already named it that")
}
if rewritten.Changed {
say(fmt.Sprintf(" control plane %s", rewritten.Now))
say(fmt.Sprintf(" its image %s", rewritten.Now))
say(fmt.Sprintf(" replacing %s, named in %d place(s)",
rewritten.Was, rewritten.Places))
} else {
say(fmt.Sprintf(" control plane %s — the template already named it, nothing rewritten",
say(fmt.Sprintf(" its image %s — the template already named it, nothing rewritten",
rewritten.Now))
}
for _, kept := range rewritten.Kept {
@@ -207,6 +319,15 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
// loading, applying and asking the result questions, and none of those can be answered by
// not doing them.
say(fmt.Sprintf(" would write %s (%d resources)", o.Out, rewritten.Resources))
if o.pivots() {
// Named rather than attempted. Everything from step 6 on is a conversation with a
// control plane that a dry run has not raised, so there is nothing to ask and nothing
// honest to report about the answers.
say(" would then enrol " + o.Node + ", install the registry from " +
o.Catalogue + ", push the control plane's image into it,")
say(" reinstall the control plane as a module, and drop " +
rewritten.TempName)
}
result.Stopped = "dry run: the bundle was produced and checked, and nothing was written, " +
"loaded or applied"
say("\n" + result.Stopped)
@@ -242,19 +363,75 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
return result, failed(StepVerify, err)
}
say("\nthis machine is a mesh of one node, with nothing joined to it yet.")
if !o.pivots() {
// Stopped, and said plainly. What has been raised works and cannot be upgraded: its
// control plane is named by an image id, which no registry serves, so nothing can ever
// replace it with a newer one. That is the whole of what the pivot fixes, and it needs
// manifests, and manifests come from a checkout somebody has to point this at.
result.Stopped = "no --catalog was given, so this stopped at the substrate. " +
"The control plane is named by the digest of its own configuration and no registry " +
"serves it, so this mesh cannot yet upgrade itself. Run again with " +
"--catalog <a checkout of the mesh's catalogue> to finish the pivot; every step " +
"above will say it is already done"
say("\nthis machine is a mesh of one node, with nothing joined to it yet.")
say(result.Stopped)
return result, nil
}
// NEXT STAGE — NOT IMPLEMENTED HERE.
// ---- 6. enrol -------------------------------------------------------------------------
//
// What remains between "a mesh exists" and "a mesh does something": issuing this machine a
// token and enrolling it as its own first node, registering the module catalogue with the
// control plane, and assigning modules to nodes. All three are conversations with the control
// plane that has just been proved to reply, so they belong after this point and inside none of
// the steps above.
//
// Left out rather than half-written. Everything above changes a machine; all of that changes a
// mesh, and a program that did both would have two jobs and one name.
say("not done here: enrolment, the module catalogue, and assignment.")
// From here on the mesh is being told things, and the way to tell it anything is to run its
// own binary inside its own container. `temporary` is the substrate's control plane; the
// module's is a different container with a different name and does not exist yet.
temporary := controlPlane{container: rewritten.TempName, run: d.Run, timeout: o.Timeout}
say("enrol — this machine joins the mesh it is running")
enrolled, err := Enrol(ctx, o, sys, temporary, say)
result.Node, result.NodeAdded, result.Enrolled = enrolled.Node, enrolled.Added, enrolled.Joined
result.Agent = enrolled.Agent
if err != nil {
return result, failed(StepEnrol, err)
}
// ---- 7. registry ----------------------------------------------------------------------
say("registry — somewhere for this mesh to keep its own images")
registry, err := InstallRegistry(ctx, o, d, temporary, say)
result.Registry, result.RegistryRunning = registry.Address, registry.Container
result.RegistryKnown, result.RegistryReplied = registry.Known, registry.Answered
if err != nil {
return result, failed(StepRegistry, err)
}
// ---- 8. publish -----------------------------------------------------------------------
say("publish — the control plane's image gets its first manifest digest")
published, err := PublishControlPlane(ctx, o, d, loaded.ID, say)
result.PublishedAs, result.PublishedAlready = published.Reference, published.Already
if err != nil {
return result, failed(StepPublish, err)
}
// ---- 9. control plane -----------------------------------------------------------------
say("control plane — installed as an ordinary module, pinned to that digest")
permanent, err := InstallControlPlane(ctx, o, d, temporary, rewritten.Declaration,
published.Reference, say)
result.Permanent, result.PermanentAnswered = permanent.Container, permanent.Answered
result.StoresDelivered = permanent.Delivered
if err != nil {
return result, failed(StepControlPlane, err)
}
// ---- 10. retire -----------------------------------------------------------------------
say("retire — the temporary control plane is dropped from the bundle")
retired, err := RetireTheTemporaryControlPlane(ctx, o, sys, rewritten.Bundle, d.Run, say)
result.TemporaryRetired = retired.Gone || retired.Already
result.TemporaryWasGone, result.TemporaryRemovedAt = retired.Already, retired.Removed
if err != nil {
return result, failed(StepRetire, err)
}
say("\nthis machine is a mesh of one node, and the control plane it runs is a module " +
"pinned to an image its own registry serves.")
say("what remains is somebody else's: adding nodes, and assigning what they should run.")
return result, nil
}