package bootstrap import ( "bytes" "fmt" "os" "path/filepath" "strings" "github.com/novox/mesh-host/internal/declaration" ) // ControlPlaneID is the resource the installer replaces the image of. // // A resource id rather than a container name or a guess at the image, because the id is the one // thing a declaration promises is stable — it is what lets the store say *this is the same // resource I applied last time* (`internal/declaration`, Resource.Identity). A bundle that does not // name one is refused rather than applied without a control plane, which would raise a store and a // broker and no mesh. const ControlPlaneID = "control-plane" // TempPrefix is what the foundation's control plane is renamed with. // // **This is the whole of how a carried resource becomes a declared one.** The foundation raises a // control plane and a module later declares one, and for a moment both exist — which looked like a // handover problem needing a way for the host to stop owning something without destroying it. It is // not one. The temporary control plane is called `temp-mesh-controller` and the permanent one is // called `mesh-controller`: two containers, two owners, nothing shared and nothing to hand over. At // the end the temporary one is dropped from the bundle and the host removes it, which is exactly // what should happen to something named "temp" (novox/hq ADR 0067). // // The name is also the audit. After the pivot, a machine running `mesh-controller` and not // `temp-mesh-controller` has completed it; one running both stopped in the middle; one running only // the temp has not started. That is readable from `docker ps` by somebody who knows nothing else. const TempPrefix = "temp-" // ControlPlaneModule is the module the permanent control plane is installed as, and the name its // container takes — the name the foundation's own control plane gives up here so that it can. // // Declared beside the rename rather than beside the step that uses it, because this is where the // two names are decided together and where the reason for both of them is written down. const ControlPlaneModule = "mesh-controller" // brokerAddressVar is what a token tells an enrolling node to dial. // // Not rewritten here — see the note in Rewrite — but reported, because it is the field most likely // to be wrong on a machine that is not the one the template was written for, and it is wrong in a // way nothing notices until a second node tries to join. const brokerAddressVar = "MESH_BROKER_ADDRESS" // Rewritten is the bundle this machine will apply, and what was done to produce it. type Rewritten struct { // Bundle is the produced file's bytes — the template with one image reference replaced, // comments and all. Bundle []byte // Declaration is that bundle, parsed. Carried so the apply and the verify are talking about // the same document rather than each re-reading the file and hoping. Declaration *declaration.Declaration // Was is the image the template named the control plane by; Now is the one it names it by. Was string Now string // Places is how many times that reference appeared, and therefore how many were replaced. Places int // Changed is false when the template already named this image — a re-run on a bundle this // installer produced earlier. Changed bool // WasCalled is what the template called the control plane's container; TempName is what the // produced bundle calls it. Renamed is false when the template already used the temporary name. WasCalled string TempName string Renamed bool // Kept is every other container image, unchanged, as " ". Reported rather than // assumed: "postgres was left alone" is a claim, and this is the evidence for it. Kept []string // Resources is how many things the produced bundle asks for. Resources int // BrokerAddress is what the control plane will tell enrolling nodes to dial, or empty. BrokerAddress string } // Rewrite produces the bundle this machine will apply from the template it was given. // // **Two substitutions, and both are textual.** The control plane's image becomes the id of the image // this machine now holds, and its container is renamed `temp-mesh-controller`; nothing else changes. // The rename is what makes the pivot expressible at all — see TempPrefix. Textual rather than // parse-and-re-serialise because // the produced file has to be *read* — a person getting a machine working must be able to open it, // see the foundation they recognise, and see exactly one thing different. Re-serialising a parsed // declaration would drop every comment in the template, and those comments are where the reasons // live. // // **Every place that reference appears, not only the container.** The bundle names the control // plane's image twice: once as the container that runs `serve`, and once inside the action that // runs `migrate` to create the contexts' schemas. Replacing only the container would leave the // migration pointing at an image no registry serves, and the apply would fail in the middle — // after the store is up, before the broker. They are one image and they move together. // // **Third-party images are not touched.** postgres and lavinmq keep the `name@sha256:` references // the template carries and are pulled from wherever those name (novox/hq ADR 0006). This is // checked afterwards rather than merely intended: the produced bundle is re-parsed and every other // container's image is compared against what it was. // // **What this deliberately does NOT rewrite:** MESH_BROKER_ADDRESS, the endpoint every enrolment // token will carry. It differs per machine and it is silently fatal when wrong — a node enrols // against a dead address and nothing complains until it fails to come back. It belongs to the // enrolment stage, which is not built yet, so this reports it loudly and leaves it alone rather // than guessing an address for a machine it has not been told about. func Rewrite(template []byte, imageID string) (Rewritten, error) { if !isImageID(imageID) { return Rewritten{}, fmt.Errorf( "%q is not an image id. The control plane is named by the digest of its own "+ "configuration — sha256: and sixty-four hex characters — because nothing serves "+ "it and there is no manifest digest to use instead", imageID) } before, err := declaration.ParseFileTrusted(template) if err != nil { return Rewritten{}, fmt.Errorf("the bundle template is not a declaration: %w", err) } control, err := controlPlaneIn(before) if err != nil { return Rewritten{}, err } out := Rewritten{Was: control.Image, Now: imageID, BrokerAddress: control.Env[brokerAddressVar]} occurrences := bytes.Count(template, []byte(control.Image)) if occurrences == 0 { // The parser found the image and the bytes do not contain it, which means the two are // reading different things. Refused rather than replaced-zero-times-and-reported-success. return Rewritten{}, fmt.Errorf( "the control plane's image is %q according to the parsed template, and that text is "+ "not in the file. Nothing was rewritten", control.Image) } out.Places = occurrences switch { case control.Image == imageID: // Already this image. The idempotent case, and the one that happens whenever somebody // re-runs the installer against a bundle it produced earlier. out.Bundle = template default: out.Bundle = bytes.ReplaceAll(template, []byte(control.Image), []byte(imageID)) out.Changed = true } // And the container is renamed, for the reason TempPrefix records. Done here rather than // anywhere later because the bundle is the only place the name is decided: the apply creates // the container from it, the verify asks that container questions, and the retirement takes // this same resource back out. One name, one place, read by everything. out.WasCalled, out.TempName = control.Name, TempPrefix+control.Name if strings.HasPrefix(control.Name, TempPrefix) { out.TempName = control.Name } renamed, err := renameContainer(out.Bundle, control.Name, out.TempName) if err != nil { return Rewritten{}, err } out.Bundle, out.Renamed = renamed, out.TempName != control.Name // Read back, on the bytes that will actually be applied. Everything above is an intention // until the produced file is parsed and asked what it says. after, err := declaration.ParseFileTrusted(out.Bundle) if err != nil { return Rewritten{}, fmt.Errorf( "the bundle this produced is not a declaration, so the substitution broke it: %w", err) } out.Declaration, out.Resources = after, len(after.Resources) produced, err := controlPlaneIn(after) if err != nil { return Rewritten{}, err } if produced.Image != imageID { return Rewritten{}, fmt.Errorf( "the produced bundle still names the control plane %q, not %q", produced.Image, imageID) } if produced.Name != out.TempName { return Rewritten{}, fmt.Errorf( "the produced bundle still calls the control plane's container %q, not %q. The "+ "permanent one is a module and takes the plain name, so a foundation that kept it "+ "would put two owners on one container", produced.Name, out.TempName) } // And nothing else moved. A substitution on text can in principle catch more than it was // aimed at, and "postgres was left exactly as it was" is the claim this checks rather than // asserts. Both the image AND the name, because there are now two substitutions. wasImage, wasName := containerImages(before), containerNames(before) for id, image := range containerImages(after) { if id == ControlPlaneID { continue } if wasImage[id] != image { return Rewritten{}, fmt.Errorf( "rewriting the control plane's image also changed %q, from %q to %q. Only the "+ "mesh's own image may move; everything else is somebody else's image at "+ "somebody else's registry", id, wasImage[id], image) } out.Kept = append(out.Kept, fmt.Sprintf("%s %s", id, image)) } for id, name := range containerNames(after) { if id == ControlPlaneID || wasName[id] == name { continue } return Rewritten{}, fmt.Errorf( "renaming the control plane's container also renamed %q, from %q to %q. Only the "+ "control plane moves out of the way; every other container keeps the name the "+ "foundation gave it", id, wasName[id], name) } sortStrings(out.Kept) return out, nil } // renameContainer changes one container's name in the bundle's text. // // **The quoted name, not the bare word.** `mesh-controller` also appears inside the image reference // the template carries (`…/mesh-controller@sha256:…`) and could appear inside a command line; a bare // substitution would catch those too. What is wanted is a JSON string that IS the name, so the // quotes are part of what is matched — `"mesh-controller"` matches the container's `name` and an // action's `in`, which are exactly the places the name means the container, and nothing else. // // It refuses when the text does not contain what the parse says is there, for the same reason the // image substitution does: the two would then be reading different things, and a rename that // replaced nothing and reported success would leave the module and the foundation fighting over one // container three steps later. func renameContainer(bundle []byte, from, to string) ([]byte, error) { if from == to { return bundle, nil } quoted := []byte(`"` + from + `"`) if bytes.Count(bundle, quoted) == 0 { return nil, fmt.Errorf( "the control plane's container is called %q according to the parsed template, and %s "+ "is not in the file. Nothing was renamed, and the foundation would raise a "+ "container the module also wants", from, quoted) } return bytes.ReplaceAll(bundle, quoted, []byte(`"`+to+`"`)), nil } // controlPlaneIn finds the container this installer replaces the image of. func controlPlaneIn(d *declaration.Declaration) (*declaration.Container, error) { for _, r := range d.Resources { if r.Identity() != ControlPlaneID { continue } container, ok := r.(*declaration.Container) if !ok { return nil, fmt.Errorf( "this bundle's %q is a %s, and the control plane has to be a container for its "+ "image to be named. Nothing was rewritten", ControlPlaneID, r.Kind()) } return container, nil } return nil, fmt.Errorf( "this bundle names no %q, so there is no control plane to give this machine's image to. "+ "A foundation without one raises a store and a broker and no mesh. It declares: %s", ControlPlaneID, strings.Join(identities(d), ", ")) } func containerImages(d *declaration.Declaration) map[string]string { images := map[string]string{} for _, r := range d.Resources { if container, ok := r.(*declaration.Container); ok { images[container.ID] = container.Image } } return images } func containerNames(d *declaration.Declaration) map[string]string { names := map[string]string{} for _, r := range d.Resources { if container, ok := r.(*declaration.Container); ok { names[container.ID] = container.Name } } return names } func identities(d *declaration.Declaration) []string { var ids []string for _, r := range d.Resources { ids = append(ids, r.Identity()) } return ids } // isImageID is the same shape `internal/declaration` accepts for an image the machine holds. Asked // here as well so the refusal names the installer's own mistake, rather than surfacing as a // declaration refusal about a bundle this program wrote. func isImageID(s string) bool { const prefix = "sha256:" if !strings.HasPrefix(s, prefix) || len(s) != len(prefix)+64 { return false } for _, c := range s[len(prefix):] { if (c < '0' || c > '9') && (c < 'a' || c > 'f') { return false } } return true } func sortStrings(values []string) { for i := 1; i < len(values); i++ { for j := i; j > 0 && values[j] < values[j-1]; j-- { values[j], values[j-1] = values[j-1], values[j] } } } // writeBundleFile puts the produced bundle where a person can read it, creating the directory it // lives in. // // 0600: the produced bundle carries the credentials genesis made — the store's and the broker's, // inside the temporary control plane's connection strings (novox/hq issue 071). It used to be // 0644 and say so was fine because the template's credentials were the same ones anybody could // read in the template; they are not any more. Still meant to be read, by root. func writeBundleFile(path string, content []byte) error { if dir := filepath.Dir(path); dir != "" && dir != "." { if err := os.MkdirAll(dir, 0o755); err != nil { return fmt.Errorf("cannot make %s to write the produced bundle into: %w", dir, err) } } if err := os.WriteFile(path, content, 0o600); err != nil { return fmt.Errorf( "cannot write the produced bundle to %s: %w\nIt is what is about to be applied, and "+ "applying something nobody can read afterwards is how a machine becomes a mystery", path, err) } // The mode above applies only when the file is created. A bundle an earlier installer left at // 0644 would keep that while now carrying real credentials, with this function saying 0600. if err := os.Chmod(path, 0o600); err != nil { return fmt.Errorf("cannot make the produced bundle %s readable by root alone: %w", path, err) } return nil }