novox/mesh-host
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge pull request 'Self-upgrade installer: genesis pivot, rename, adopt store+broker' (#13) from feat/a-bed-that-hands-over-nothing into main

Browse Source
This commit was merged in pull request #13.
This commit is contained in:
jschoubben
2026-09-16 23:22:06 +02:00
parent 200c6c127f 56124c38b7
commit 1f483fcacd
61 changed files with 2395 additions and 348 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Makefile
+8 -8
View File
@@ -36,24 +36,24 @@ test:
go test ./... -count=1
# A default build carries no bundle and refuses to reconcile, which is the honest state for a
# host nobody has told what a substrate is.
# host nobody has told what a foundation is.
build:
CGO_ENABLED=0 go build -ldflags="$(LDFLAGS)" -o mesh-host ./cmd/mesh-host
# A host for a real machine, carrying a real bundle:
# make host SYSTEM=arch BUNDLE=path/to/substrate.lock
# make host SYSTEM=arch BUNDLE=path/to/foundation.lock
#
# The bundle replaces the one for SYSTEM, because its contents are per operating system —
# package names and unit names differ (novox/hq ADR 0005).
host:
@test -n "$(BUNDLE)" || { echo "BUNDLE= is required; a host with no bundle cannot raise a first node"; exit 1; }
@test -f "$(BUNDLE)" || { echo "no such bundle: $(BUNDLE)"; exit 1; }
@test -f internal/bundle/substrate-$(SYSTEM).lock || { echo "no bundle slot for SYSTEM=$(SYSTEM)"; exit 1; }
@cp internal/bundle/substrate-$(SYSTEM).lock internal/bundle/substrate-$(SYSTEM).lock.default
@cp "$(BUNDLE)" internal/bundle/substrate-$(SYSTEM).lock
@test -f internal/bundle/foundation-$(SYSTEM).lock || { echo "no bundle slot for SYSTEM=$(SYSTEM)"; exit 1; }
@cp internal/bundle/foundation-$(SYSTEM).lock internal/bundle/foundation-$(SYSTEM).lock.default
@cp "$(BUNDLE)" internal/bundle/foundation-$(SYSTEM).lock
@CGO_ENABLED=0 go build -ldflags="$(LDFLAGS)" -o mesh-host ./cmd/mesh-host; \
status=$$?; \
mv internal/bundle/substrate-$(SYSTEM).lock.default internal/bundle/substrate-$(SYSTEM).lock; \
mv internal/bundle/foundation-$(SYSTEM).lock.default internal/bundle/foundation-$(SYSTEM).lock; \
exit $$status
@echo "built for $(SYSTEM) carrying $(BUNDLE)"
@@ -66,7 +66,7 @@ host:
# answered by ADR 0071: the source comes from a mesh that already exists, which is not the one being
# raised. What cannot be fetched is the thing that does the fetching, and that is what is carried.
#
# The image is BUILT ELSEWHERE and handed over — mesh-control's own `make builder-image` — and
# The image is BUILT ELSEWHERE and handed over — mesh-controller's own `make builder-image` — and
# embedded here at release time, the same way carrying the bundle breaks the "copy it onto a machine
# and run it" cycle (novox/hq ADR 0005).
#
@@ -90,7 +90,7 @@ BOOTSTRAP_OUT ?= mesh-bootstrap
bootstrap:
@test -n "$(IMAGE)" || { echo "IMAGE= is required; an installer carrying no builder image cannot raise a mesh"; exit 1; }
@case "$(IMAGE)" in sha256:*) echo "IMAGE=$(IMAGE) is an image id. The installer identifies the carried image by its tag, because an id is the digest of a configuration that a runtime rewrites as it loads. Pass a name:tag"; exit 1;; esac
@docker image inspect "$(IMAGE)" >/dev/null 2>&1 || { echo "this machine does not hold $(IMAGE) — build it in mesh-control with 'make image'"; exit 1; }
@docker image inspect "$(IMAGE)" >/dev/null 2>&1 || { echo "this machine does not hold $(IMAGE) — build it in mesh-controller with 'make image'"; exit 1; }
@test -n "$$(docker image inspect --format '{{len .RepoTags}}' "$(IMAGE)" | grep -v '^0$$')" || { echo "$(IMAGE) has no repository tag, so the saved archive would carry no name the installer can ask a runtime about. Tag it first: docker tag $(IMAGE) mesh-builder:<version>"; exit 1; }
@cp internal/image/builder.tar internal/image/builder.tar.placeholder
@docker save --output internal/image/builder.tar "$(IMAGE)"
README.md
+10 -10
View File
@@ -111,7 +111,7 @@ the fault this exists to prevent.
A host built for a machine carries its declaration **inside the binary**:
```
make host BUNDLE=path/to/substrate.lock
make host BUNDLE=path/to/foundation.lock
```
`mesh-host reconcile` then applies it. That is the first node's path — no mesh present, nothing
@@ -126,21 +126,21 @@ Stages 3 and 4 — the link, and enrolment — are designed and not built.
## What stage 2 does not yet prove
The design defines stage 2 as *the host applies `substrate.lock` with no mesh present*, and
calls out the claim underneath it: **that one host can raise the substrate alone**.
The design defines stage 2 as *the host applies `foundation.lock` with no mesh present*, and
calls out the claim underneath it: **that one host can raise the foundation alone**.
The mechanism is proved — a sealed machine, one binary, and it configures itself from what it
carries. The claim is not. The substrate is four container services, and:
carries. The claim is not. The foundation is four container services, and:
- the vocabulary has no container type, because a container needs an image and where images
come from is open ([`novox/hq` research 012](https://git.novox.be/novox/hq));
- what belongs in a substrate is not known — the closure for a one-node mesh is what
- what belongs in a foundation is not known — the closure for a one-node mesh is what
research 011 and 012 exist to answer;
- and the machine used to test this has no container runtime, because a sealed network cannot
install one.
So `substrate.lock` here is a real bundle with a placeholder's content. Saying that plainly
beats shipping a host that claims a substrate it has never raised.
So `foundation.lock` here is a real bundle with a placeholder's content. Saying that plainly
beats shipping a host that claims a foundation it has never raised.
## A capability is detected, never assumed
@@ -198,7 +198,7 @@ repository carries implementation and does not carry decisions.
## Checks that cross into the control plane's repository
Two things are agreed between this repository and `novox/mesh-control`, and each is a separate
Two things are agreed between this repository and `novox/mesh-controller`, and each is a separate
struct on each side. A field renamed on one of them fails **silently** — the crossing succeeds and
something is simply absent — so both are checked by handing one side's real output to the other's
real parser. Neither runs by default; each skips with a reason, because a repository that fails
@@ -207,7 +207,7 @@ without its neighbour checked out is a repository nobody can build.
**What the mesh sends, read by this host:**
```
mesh-control: ./build/mesh-control plan <node> --json > /tmp/d.json
mesh-controller: ./build/mesh-controller plan <node> --json > /tmp/d.json
mesh-host: MESH_EMITTED=/tmp/d.json go test ./internal/declaration/ -v
```
@@ -215,7 +215,7 @@ mesh-host: MESH_EMITTED=/tmp/d.json go test ./internal/declaration/ -v
```
mesh-host: MESH_ENROL_OUT=/tmp/enrol.json go test ./internal/link/
mesh-control: MESH_ENROL=/tmp/enrol.json make check
mesh-controller: MESH_ENROL=/tmp/enrol.json make check
```
The second writes the private half of the sealing key beside the request, so the mesh's suite can
cmd/mesh-bootstrap/main.go
+111 -12
View File
@@ -8,11 +8,11 @@
// cannot be folded into it without making that sentence false. Same tier, same repository,
// different program.
//
// Genesis is a pivot (novox/hq ADR 0067). It raises a substrate whose control plane is named by the
// Genesis is a pivot (novox/hq ADR 0067). It raises a foundation whose control plane is named by the
// digest of its own configuration — legal exactly where nothing could have served an image — then
// enrols this machine, installs the registry module, pushes that image into it to get the manifest
// digest it has never had, reinstalls the control plane as an ordinary module pinned to it, and
// drops the temporary one. Without --catalog it stops after the substrate and says why.
// drops the temporary one. Without --catalog it stops after the foundation and says why.
package main
import (
@@ -28,9 +28,11 @@ import (
"syscall"
"time"
"bufio"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/bootstrap"
"github.com/novox/mesh-host/internal/store"
"strings"
)
// version is stamped at build time. Unset in a development build, and said so rather than
@@ -38,8 +40,8 @@ import (
var version = "development build"
const (
defaultTemplate = "substrate.lock"
defaultOut = "/var/lib/mesh-host/substrate.lock"
defaultTemplate = "foundation.lock"
defaultOut = "/var/lib/mesh-host/foundation.lock"
defaultRegistry = "127.0.0.1:5000"
defaultHost = "/usr/local/bin/mesh-host"
// The unit this project actually packages, in `packaging/`. It said `mesh-host.service`, which
@@ -50,13 +52,13 @@ const (
const usage = `mesh-bootstrap — make a bare machine into a mesh
bootstrap the twelve steps below (the default)
bootstrap the eighteen steps below (the default)
version
1 preflight what has to be true before anything is changed
2 load the builder's image, carried in this installer
3 build the control plane, from its own repository and a commit
4 bundle the substrate, named for this machine
4 bundle the foundation, named for this machine
5 apply raise it
6 verify it is up, and the control plane replies
7 enrol this machine becomes the mesh's first node
@@ -67,7 +69,21 @@ const usage = `mesh-bootstrap — make a bare machine into a mesh
12 builder publish the carried builder and install it, so this mesh can
make the rest of the catalogue rather than be handed it
--bundle the substrate template to build this machine's bundle from
Twelve make a mesh that RUNS. The rest make one that WORKS, asking where a
human must choose — a run without a terminal answers with the flags below:
13 base build the shared toolchain and runtime everything with code
stands on
14 store build and install postgres — a database provider, which the
foundation's own store is not
15 catalogue build and install the module graph
16 network choose the private network (--private-network), place this
machine as its hub (--endpoint, --site)
17 filter choose the packet filter (--packet-filter) — required, so the
question is which, not whether
18 extras anything beyond the floor (--extras)
--bundle the foundation template to build this machine's bundle from
(default ` + defaultTemplate + `)
--out where the produced bundle is written, for a person to read
(default ` + defaultOut + `)
@@ -96,13 +112,27 @@ const usage = `mesh-bootstrap — make a bare machine into a mesh
--dry-run everything that does not change the machine
--json machine-readable output
--tools-source the repository the shared base is built from
--tools-ref what of it to build (default main)
--catalog-source the catalogue REPOSITORY, for building its modules;
--catalog is the checkout that says what they are
--catalog-ref what of it to build (default main)
--sdk-source the repository the shared library is built from
--sdk-ref what of it to build (default main)
--private-network which private network to run (wireguard)
--endpoint host:port other machines dial for it; derived from the
broker address when unsaid
--site where this machine sits (default main)
--packet-filter which packet filter to run (nftables)
--extras catalogue modules beyond the floor, comma-separated
The installer carries a builder, not a control plane. What raises a mesh is therefore
the same thing that will maintain it, and the control plane a mesh ends up running is
one it built itself, from a repository and a commit it can name and build again.
Genesis is a pivot: a temporary control plane installs the registry that makes it
permanent. The temporary one is called temp-mesh-control and the permanent one is
called mesh-control, so they are two containers with two owners and there is nothing
permanent. The temporary one is called temp-mesh-controller and the permanent one is
called mesh-controller, so they are two containers with two owners and there is nothing
to hand over.
Every step is idempotent: run it again after fixing whatever it named, and the steps
@@ -187,11 +217,11 @@ func newFlagSet(opts *bootstrap.Options, jsonOut *bool) *flag.FlagSet {
set := flag.NewFlagSet("mesh-bootstrap", flag.ContinueOnError)
set.SetOutput(os.Stderr)
set.Usage = func() { fmt.Fprint(os.Stderr, usage) }
set.StringVar(&opts.Template, "bundle", opts.Template, "the substrate template to build from")
set.StringVar(&opts.Template, "bundle", opts.Template, "the foundation template to build from")
set.StringVar(&opts.Out, "out", opts.Out, "where the produced bundle is written")
set.StringVar(&opts.State, "state", opts.State, "where this node records what it has applied")
set.StringVar(&opts.Catalogue, "catalog", opts.Catalogue,
"a checkout of the mesh's catalogue; without it this stops after the substrate")
"a checkout of the mesh's catalogue; without it this stops after the foundation")
set.StringVar(&opts.Source.Repository, "source", opts.Source.Repository,
"the repository the control plane is built from, on a mesh that already exists")
set.StringVar(&opts.Source.Ref, "source-ref", opts.Source.Ref,
@@ -209,9 +239,71 @@ func newFlagSet(opts *bootstrap.Options, jsonOut *bool) *flag.FlagSet {
set.DurationVar(&opts.Wait, "wait", opts.Wait, "how long something merely starting is given")
set.BoolVar(&opts.DryRun, "dry-run", false, "everything that does not change the machine")
set.BoolVar(jsonOut, "json", false, "machine-readable output")
// Phase two — the installer goes as far as it can, and asks where a human must choose. A run
// without a terminal answers with these; a required choice nothing answered is a refusal.
set.StringVar(&opts.ToolsSource.Repository, "tools-source", opts.ToolsSource.Repository,
"the repository the shared base is built from")
set.StringVar(&opts.ToolsSource.Ref, "tools-ref", opts.ToolsSource.Ref,
"what of it to build (default main)")
set.StringVar(&opts.CatalogSource.Repository, "catalog-source", opts.CatalogSource.Repository,
"the catalogue REPOSITORY, for building its modules — --catalog is the checkout that says what they are")
set.StringVar(&opts.CatalogSource.Ref, "catalog-ref", opts.CatalogSource.Ref,
"what of it to build (default main)")
set.StringVar(&opts.SDKSource.Repository, "sdk-source", opts.SDKSource.Repository,
"the repository the shared library is built from, published before the base resolves it")
set.StringVar(&opts.SDKSource.Ref, "sdk-ref", opts.SDKSource.Ref,
"what of it to build (default main)")
set.StringVar(&opts.Site, "site", "main", "where this machine sits, for the private network")
if opts.Answers == nil {
opts.Answers = map[string]string{}
}
answers := opts.Answers
set.Func("private-network", "which private network to run (wireguard)", func(v string) error {
answers["private-network"] = v
return nil
})
set.Func("packet-filter", "which packet filter to run (nftables)", func(v string) error {
answers["packet-filter"] = v
return nil
})
set.Func("endpoint", "host:port other machines dial for the private network (derived from the broker address if unsaid)", func(v string) error {
answers["endpoint"] = v
return nil
})
set.Func("extras", "catalogue modules beyond the floor, comma-separated", func(v string) error {
for _, e := range strings.Split(v, ",") {
if e = strings.TrimSpace(e); e != "" {
opts.Extras = append(opts.Extras, e)
}
}
return nil
})
return set
}
// askOn is how a person is asked a choice, when there is a person: the question, the options, a
// read line. Wired only when stdin is a terminal, so the lab and unattended runs are never left
// waiting on a prompt nobody will answer.
func askOn(in *bufio.Reader, out io.Writer) func(bootstrap.Choice) (string, error) {
return func(c bootstrap.Choice) (string, error) {
fmt.Fprintf(out, "\n%s\n", c.Question)
if len(c.Options) > 0 {
fmt.Fprintf(out, " options: %s\n", strings.Join(c.Options, ", "))
}
if c.Default != "" {
fmt.Fprintf(out, " [%s] ", c.Default)
} else {
fmt.Fprint(out, " > ")
}
line, err := in.ReadString('\n')
if err != nil {
return "", fmt.Errorf("the terminal went away mid-question: %w", err)
}
return strings.TrimSpace(line), nil
}
}
func run(ctx context.Context, command string, opts bootstrap.Options, jsonOut bool) error {
switch command {
case "bootstrap":
@@ -220,6 +312,13 @@ func run(ctx context.Context, command string, opts bootstrap.Options, jsonOut bo
fmt.Println(line)
}
}
// A person at a terminal is asked the choices; anything else answers with flags. `--json`
// counts as "anything else": a run whose output is being parsed has no one reading a
// question.
if info, err := os.Stdin.Stat(); err == nil &&
info.Mode()&os.ModeCharDevice != 0 && !jsonOut {
opts.Prompt = askOn(bufio.NewReader(os.Stdin), os.Stdout)
}
result, err := bootstrap.Run(ctx, opts, bootstrap.Deps{
Run: apply.ExecRunner,
Dial: dial,
@@ -268,7 +367,7 @@ func hostname() string {
//
// Plain HTTP, and only at the mesh's own registry: it is reached over the mesh's private network,
// which is already the encrypted and authenticated thing, and a second layer inside it would be
// certificates to issue and rotate for no property the first does not have (mesh-control's
// certificates to issue and rotate for no property the first does not have (mesh-controller's
// `internal/builder` pushes to it on the same reasoning).
//
// The body is read with a limit. What is asked for is a status and a short JSON answer, and a
cmd/mesh-bootstrap/main_test.go
+4 -2
View File
@@ -57,7 +57,7 @@ func TestAMistypedFlagIsRefusedNotIgnored(t *testing.T) {
}
func TestAnUnexpectedArgumentIsRefused(t *testing.T) {
if _, _, _, err := parseArgs([]string{"bootstrap", "substrate.lock"}); err == nil {
if _, _, _, err := parseArgs([]string{"bootstrap", "foundation.lock"}); err == nil {
t.Fatal("a stray argument was ignored rather than refused — the bundle is --bundle")
}
}
@@ -108,6 +108,8 @@ func TestTheUsageTextAndTheFlagsAgree(t *testing.T) {
for _, promised := range []string{
"bundle", "out", "state", "system", "timeout", "wait", "dry-run", "json",
"catalog", "node", "registry", "host", "host-service", "host-in-background",
"tools-source", "tools-ref", "catalog-source", "catalog-ref",
"private-network", "endpoint", "site", "packet-filter", "extras",
} {
if !declared[promised] {
t.Errorf("the usage text promises --%s and no such flag exists", promised)
@@ -139,7 +141,7 @@ func TestThePivotsDefaultsAreTheDocumentedOnes(t *testing.T) {
// be a checkout somebody else made, at whatever commit they left it on — and it decides which
// image the mesh's control plane is pinned to for ever after.
if opts.Catalogue != "" {
t.Errorf("--catalog defaults to %q; without one the installer stops at the substrate",
t.Errorf("--catalog defaults to %q; without one the installer stops at the foundation",
opts.Catalogue)
}
// The machine's own name, because that is what a person already calls it.
cmd/mesh-host/main.go
+1 -1
View File
@@ -823,7 +823,7 @@ func sealOpener(statePath string) apply.Unseal {
// carriedPorts is every machine port held by what this host raised from its own bundle.
//
// **What the mesh must assign around** (novox/hq ADR 0038). The substrate is not a module: a node
// **What the mesh must assign around** (novox/hq ADR 0038). The foundation is not a module: a node
// raises it before any mesh exists, so the control plane has never heard of the store or the
// broker. Told this, it can put a module somewhere else; not told, it hands out a port one of them
// holds and finds out from a container runtime.
examples/README.md
+5 -5
View File
@@ -1,17 +1,17 @@
# Examples
## `substrate-first-node.lock`
## `foundation-first-node.lock`
What a machine must be before a mesh exists — the bootstrap in
[novox/hq `07-the-substrate.md`](https://git.novox.be/novox/hq), whole:
[novox/hq `07-the-foundation.md`](https://git.novox.be/novox/hq), whole:
```
0 a container runtime
1 the store runs
2 a database per context `inventory` and `identity`
3 those contexts' schemas mesh-control migrate
3 those contexts' schemas mesh-controller migrate
4 the broker runs with a certificate it generated itself
5 the control plane runs mesh-control serve
5 the control plane runs mesh-controller serve
```
**A machine that applies this is a mesh** — one node, with nothing joined to it yet, which is
@@ -24,7 +24,7 @@ a comment about what something does not do is a comment nobody updates.
Build a host carrying it:
```
make host SYSTEM=arch BUNDLE=examples/substrate-first-node.lock
make host SYSTEM=arch BUNDLE=examples/foundation-first-node.lock
```
**The registry address and digests have to be replaced before this is useful.** They are written
examples/bundle_test.go
+9 -9
View File
@@ -1,6 +1,6 @@
// Package examples checks the bundles shipped in this directory.
//
// **Nothing checked them before.** `substrate-first-node.lock` is what a machine becomes when
// **Nothing checked them before.** `foundation-first-node.lock` is what a machine becomes when
// there is no mesh to ask — the one declaration applied with nothing to verify it against — and
// it was edited by hand and read by nobody but a running host.
package examples
@@ -16,7 +16,7 @@ import (
func bundle(t *testing.T) *declaration.Declaration {
t.Helper()
raw, err := os.ReadFile("substrate-first-node.lock")
raw, err := os.ReadFile("foundation-first-node.lock")
if err != nil {
t.Fatal(err)
}
@@ -27,12 +27,12 @@ func bundle(t *testing.T) *declaration.Declaration {
return d
}
// Defends novox/hq ADR 0028: the substrate supplies the control plane and nothing else.
// Defends novox/hq ADR 0028: the foundation supplies the control plane and nothing else.
//
// The object store was substrate for months on the strength of "it cannot grant itself a bucket",
// The object store was foundation for months on the strength of "it cannot grant itself a bucket",
// which answers half the test. The control plane never needed one, and nothing noticed because
// nothing counted what the bundle holds.
func TestTheBundleCarriesTheSubstrateAndTheControlPlaneAndNothingElse(t *testing.T) {
func TestTheBundleCarriesTheFoundationAndTheControlPlaneAndNothingElse(t *testing.T) {
var images []string
for _, r := range bundle(t).Resources {
c, ok := r.(*declaration.Container)
@@ -48,7 +48,7 @@ func TestTheBundleCarriesTheSubstrateAndTheControlPlaneAndNothingElse(t *testing
}
sort.Strings(images)
want := []string{"lavinmq", "mesh-control", "postgres"}
want := []string{"lavinmq", "mesh-controller", "postgres"}
if strings.Join(images, ",") != strings.Join(want, ",") {
t.Fatalf("the bundle carries %v; expected exactly %v.\n\n"+
"Adding one is a change to what every first node becomes, and to ADR 0006's "+
@@ -57,13 +57,13 @@ func TestTheBundleCarriesTheSubstrateAndTheControlPlaneAndNothingElse(t *testing
}
// The control plane is in the bundle, and the design overlooked it once by reasoning about
// substrate services rather than counting containers (novox/hq 03-DESIGN/01-to-be/07).
// foundation services rather than counting containers (novox/hq 03-DESIGN/01-to-be/07).
func TestTheControlPlaneIsCarriedToo(t *testing.T) {
for _, r := range bundle(t).Resources {
if c, ok := r.(*declaration.Container); ok && strings.Contains(c.Image, "mesh-control") {
if c, ok := r.(*declaration.Container); ok && strings.Contains(c.Image, "mesh-controller") {
return
}
}
t.Fatal("nothing in the bundle starts the control plane, so the machine would raise a " +
"substrate and stop")
"foundation and stop")
}
examples/substrate-first-node.lock → examples/foundation-first-node.lock
+8 -8
View File
@@ -1,6 +1,6 @@
// substrate-first-node.lock — what a machine must be before a mesh exists.
// foundation-first-node.lock — what a machine must be before a mesh exists.
//
// The whole bootstrap (novox/hq 03-DESIGN/01-to-be/07-the-substrate.md): a container runtime, a
// The whole bootstrap (novox/hq 03-DESIGN/01-to-be/07-the-foundation.md): a container runtime, a
// store, a database per context, those contexts' schemas, the broker, and the control plane
// running on top of them.
//
@@ -55,7 +55,7 @@
"POSTGRES_PASSWORD": "bootstrap",
"PGDATA": "/var/lib/postgresql/data/pgdata"
},
"ports": ["127.0.0.1:5432:5432"],
"ports": ["5432:5432"],
"volumes": ["mesh-store-data:/var/lib/postgresql/data"]
},
// Over TCP, not the socket. While the store initialises it runs a temporary server on the
@@ -85,7 +85,7 @@
},
// Each context owns its own database (novox/hq ADR 0008). A third one is a third database,
// created the same way and named the same way — which is the whole of adding a context to the
// bootstrap, and is why the count is not something the substrate has an opinion about.
// bootstrap, and is why the count is not something the foundation has an opinion about.
{
"id": "licences-database",
"type": "action",
@@ -100,7 +100,7 @@
"-e", "MESH_STORE_INVENTORY=postgres://postgres:bootstrap@127.0.0.1:5432/inventory?sslmode=disable",
"-e", "MESH_STORE_IDENTITY=postgres://postgres:bootstrap@127.0.0.1:5432/identity?sslmode=disable",
"-e", "MESH_STORE_LICENCES=postgres://postgres:bootstrap@127.0.0.1:5432/licences?sslmode=disable",
"192.0.2.250:5000/mesh-control@sha256:c67db38439ff0aee242b467486765467bb95801f52175fc5727cc4e437338ace",
"192.0.2.250:5000/mesh-controller@sha256:c67db38439ff0aee242b467486765467bb95801f52175fc5727cc4e437338ace",
"migrate"],
"verify": ["sh", "-c", "docker exec mesh-store psql -U postgres -d inventory -tAc \"select to_regclass('public.node')\" | grep -qx node && docker exec mesh-store psql -U postgres -d identity -tAc \"select to_regclass('public.signing_key')\" | grep -qx signing_key && docker exec mesh-store psql -U postgres -d licences -tAc \"select to_regclass('public.licence')\" | grep -qx licence"]
},
@@ -119,7 +119,7 @@
"type": "container",
"name": "mesh-broker",
"image": "192.0.2.250:5000/cloudamqp/lavinmq@sha256:b117c254e6e269a29db479e6b410ca4e46e035b4981e49d24b159673ef09d336",
"ports": ["5671:5671", "127.0.0.1:5672:5672", "127.0.0.1:15672:15672"],
"ports": ["5671:5671", "5672:5672", "127.0.0.1:15672:15672"],
"volumes": ["mesh-broker-data:/var/lib/lavinmq", "mesh-broker-tls:/tls:ro"],
"args": ["--amqps-port=5671", "--cert=/tls/tls.crt", "--key=/tls/tls.key"]
},
@@ -133,8 +133,8 @@
{
"id": "control-plane",
"type": "container",
"name": "mesh-control",
"image": "192.0.2.250:5000/mesh-control@sha256:c67db38439ff0aee242b467486765467bb95801f52175fc5727cc4e437338ace",
"name": "mesh-controller",
"image": "192.0.2.250:5000/mesh-controller@sha256:c67db38439ff0aee242b467486765467bb95801f52175fc5727cc4e437338ace",
"network": "host",
"args": ["serve"],
"volumes": ["mesh-broker-tls:/broker-tls:ro"],
internal/apply/apply.go
+2
View File
@@ -270,6 +270,8 @@ func applyOne(ctx context.Context, sys system.System, r declaration.Resource, ru
return applyUser(ctx, sys, res, run)
case *declaration.Archive:
return applyArchive(ctx, res, previous)
case *declaration.Process:
return applyProcess(ctx, res, run, changed, previous)
case *declaration.Action:
return applyAction(ctx, res, run)
case *declaration.Network:
internal/apply/apply_test.go
+1 -1
View File
@@ -459,7 +459,7 @@ func TestForgettingAUnitThatIsGoneDoesNotStrandTheNode(t *testing.T) {
}
}
// --- package, container and action (novox/hq 07-the-substrate.md, ADR 0006, ADR 0005) ---
// --- package, container and action (novox/hq 07-the-foundation.md, ADR 0006, ADR 0005) ---
func parseTrusted(t *testing.T, raw string) *declaration.Declaration {
t.Helper()
internal/apply/process.go
+291
View File
@@ -0,0 +1,291 @@
package apply
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
// Running the mesh's own code, without the module choosing how.
//
// **A daemon is an intent and this is one answer to it.** A module says what to run and the
// machine's own supervisor is how — which means the module is not writing a unit file, and not
// choosing between a container and a service before it can declare anything
// (novox/hq 03-DESIGN/01-to-be/18-building-a-module.md).
//
// What this does, in order: fetch the bundle, refuse it unless it hashes to what was declared,
// unpack it where the mesh keeps such things, write the unit, and put it in the state asked for.
// The unit is the mesh's — an operator editing it loses the edit at the next declaration, which is
// the same rule every managed file on a machine follows (ADR 0011).
// daemonRoot is where unpacked daemons live.
//
// Under the mesh's own directory rather than somewhere a distribution owns: these are files the
// mesh puts there and replaces, and putting them where a package manager also writes is how two
// owners end up disagreeing about one path.
const daemonRoot = "/var/lib/mesh/daemons"
// unitDir is where the mesh writes the units it owns.
const unitDir = "/etc/systemd/system"
func applyProcess(ctx context.Context, r *declaration.Process, run Runner,
changed map[string]bool, previous store.Applied) (Outcome, error) {
out := begin(r)
out.Action = "unchanged"
body, err := fetch(ctx, r.Source)
if err != nil {
return out, err
}
sum := sha256.Sum256(body)
got := "sha256:" + hex.EncodeToString(sum[:])
if got != r.Digest {
// Refused before anything is written or started. What is at that address is not what was
// declared, and running it would be running something nobody reviewed.
return out, fmt.Errorf(
"%s was declared as %s and what arrived is %s; nothing was unpacked or started",
r.Source, r.Digest, got)
}
// **Its identity is its bytes AND how it is run.** Two processes from one bundle differing
// only in their command are different, and a record tracking the digest alone would call the
// second one unchanged.
want := got + " " + unitFor(r)
at := filepath.Join(daemonRoot, r.Name)
// **Something it reads changed, so it must be restarted even though it is unchanged.** A
// running process does not re-read its configuration: replace the file, find the daemon
// already up, do nothing, and the machine keeps behaving the way it did before while every
// check passes. The same rule a service follows, for the same reason.
var because string
for _, id := range r.RestartOn {
if changed[id] {
because = id
break
}
}
if previous.Wrote == want && because == "" {
// Everything about it is as declared. Still asked whether it is RUNNING, because a
// declaration that is satisfied by a record rather than by the machine is how a stopped
// service reports success.
if active, err := run(ctx, "systemctl", "is-active", "--quiet", r.Name+".service"); err == nil {
_ = active
return out, nil
}
if _, err := run(ctx, "systemctl", "start", r.Name+".service"); err != nil {
return out, fmt.Errorf("%s is installed and would not start: %w", r.Name, err)
}
out.Action = "updated"
out.Detail = "restarted a daemon that had stopped"
out.wrote = want
return out, nil
}
// Replaced rather than merged: the bundle is the whole of what it runs, and files left from a
// previous version would be loaded by a runtime that walks a directory.
if err := os.RemoveAll(at); err != nil {
return out, err
}
if err := os.MkdirAll(at, 0o755); err != nil {
return out, err
}
written, err := unpack(body, at)
if err != nil {
return out, err
}
if err := ownAll(at, r.User); err != nil {
return out, err
}
// **A step is run to completion, not installed.** What follows it is gated on it finishing,
// so the machine is not asked to start something that needed a migration that did not happen.
// Nothing is left behind to ask afterwards: the record that it ran is the digest, which is why
// the identity above includes the command.
if r.RunOnce {
if _, err := run(ctx, r.Run[0], r.Run[1:]...); err != nil {
return out, fmt.Errorf("the %s step did not complete: %w", r.Name, err)
}
out.Action = "created"
if previous.Wrote != "" {
out.Action = "updated"
}
out.Detail = fmt.Sprintf("%d file(s), step completed", written)
out.wrote = want
return out, nil
}
unit := filepath.Join(unitDir, r.Name+".service")
if err := os.WriteFile(unit, []byte(unitFor(r)), 0o644); err != nil {
return out, err
}
if _, err := run(ctx, "systemctl", "daemon-reload"); err != nil {
return out, err
}
// Enabled and restarted, in that order: enabled so it survives a reboot, restarted rather than
// started because this path is also how a new version arrives and the old one is still running.
// **Scheduled means a timer, not a service that stays up.** The unit above is written either
// way and describes what to run; what differs is whether the machine is asked to keep it
// running or to start it when the timer says so.
if r.Schedule != "" {
timer := filepath.Join(unitDir, r.Name+".timer")
if err := os.WriteFile(timer, []byte(timerFor(r)), 0o644); err != nil {
return out, err
}
if _, err := run(ctx, "systemctl", "daemon-reload"); err != nil {
return out, err
}
// The timer is enabled and started; the service is neither. Enabling the service too
// would have it run at boot as well as on its cadence, which is a second schedule nobody
// asked for.
if _, err := run(ctx, "systemctl", "enable", r.Name+".timer"); err != nil {
return out, err
}
if _, err := run(ctx, "systemctl", "restart", r.Name+".timer"); err != nil {
return out, fmt.Errorf("%s was installed and its timer would not start: %w", r.Name, err)
}
out.Action = "updated"
if previous.Wrote == "" {
out.Action = "created"
}
out.Detail = fmt.Sprintf("%d file(s), scheduled as %s.timer", written, r.Name)
out.wrote = want
return out, nil
}
if _, err := run(ctx, "systemctl", "enable", r.Name+".service"); err != nil {
return out, err
}
if _, err := run(ctx, "systemctl", "restart", r.Name+".service"); err != nil {
return out, fmt.Errorf("%s was installed and would not start: %w", r.Name, err)
}
out.Action = "updated"
if previous.Wrote == "" {
out.Action = "created"
}
out.Detail = fmt.Sprintf("%d file(s), running as %s.service", written, r.Name)
if because != "" {
out.Detail += ", restarted because " + because + " changed"
}
out.wrote = want
return out, nil
}
// unitFor is the unit the mesh writes for a daemon.
//
// **Generated whole and never edited in place**, the same rule as every other managed file: an
// edit survives until the next declaration and then vanishes, which is worse than not being
// allowed at all, so the file says so.
//
// Deterministic — environment sorted — because this string is half the daemon's identity, and a
// map iterated in Go's order would make every apply look like a change.
func unitFor(r *declaration.Process) string {
var b strings.Builder
b.WriteString("# Generated by the mesh. Do not edit — this file is replaced whenever the\n")
b.WriteString("# declaration changes, and an edit would survive until then and vanish.\n")
b.WriteString("[Unit]\n")
fmt.Fprintf(&b, "Description=%s, a mesh daemon\n", r.Name)
b.WriteString("After=network-online.target\n")
b.WriteString("Wants=network-online.target\n\n")
b.WriteString("[Service]\n")
b.WriteString("Type=simple\n")
fmt.Fprintf(&b, "WorkingDirectory=%s\n", filepath.Join(daemonRoot, r.Name))
for _, file := range r.EnvFile {
fmt.Fprintf(&b, "EnvironmentFile=%s\n", file)
}
for _, key := range sortedKeys(r.Env) {
fmt.Fprintf(&b, "Environment=%s\n", unitValue(key, r.Env[key]))
}
if r.User != "" {
fmt.Fprintf(&b, "User=%s\n", r.User)
}
fmt.Fprintf(&b, "ExecStart=%s\n", strings.Join(r.Run, " "))
if r.Schedule != "" {
// Started by its timer and expected to finish. Restarting it would have it run
// continuously between fires, which is the opposite of a schedule.
b.WriteString("Type=oneshot\n")
b.WriteString("\n")
return strings.Replace(b.String(), "Type=simple\n", "", 1)
}
// Restarted when it exits, because something that stops is not something that stays up.
// Delayed, so a process that fails at once does not spin the machine.
b.WriteString("Restart=always\nRestartSec=5\n\n")
b.WriteString("[Install]\nWantedBy=multi-user.target\n")
return b.String()
}
// timerFor is the cadence a scheduled process runs on.
//
// **The mesh's cron expression, handed to the machine's own timer.** The host already parses and
// evaluates five-field cron (ADR 0053) for a scheduled container; a machine with a supervisor can
// be told the cadence directly rather than have the host wake up and decide.
func timerFor(r *declaration.Process) string {
var b strings.Builder
b.WriteString("# Generated by the mesh. Do not edit — this file is replaced whenever the\n")
b.WriteString("# declaration changes, and an edit would survive until then and vanish.\n")
b.WriteString("[Unit]\n")
fmt.Fprintf(&b, "Description=%s, on a schedule the mesh set\n\n", r.Name)
b.WriteString("[Timer]\n")
fmt.Fprintf(&b, "OnCalendar=%s\n", calendarFor(r.Schedule))
// A fire missed because the machine was off happens when it comes back, rather than being
// skipped silently — which is the difference between a machine that was down and a schedule
// that quietly stopped.
b.WriteString("Persistent=true\n\n")
b.WriteString("[Install]\nWantedBy=timers.target\n")
return b.String()
}
// calendarFor turns five-field cron into what a systemd timer reads.
//
// minute hour day-of-month month day-of-week -> DayOfWeek Year-Month-Day Hour:Minute:Second
func calendarFor(cron string) string {
fields := strings.Fields(cron)
if len(fields) != 5 {
// Refused at validation, so this is unreachable — and returning something that would fire
// constantly is worse than returning something that never does.
return "*-*-* 00:00:00"
}
minute, hour, dom, month, dow := fields[0], fields[1], fields[2], fields[3], fields[4]
day := dow
if dow == "*" {
day = ""
} else {
day += " "
}
return fmt.Sprintf("%s*-%s-%s %s:%s:00", day, month, dom, hour, minute)
}
// unitValue renders one environment assignment so a unit file means what the declaration said.
//
// **Three things a unit file does to a value that nothing else does**, and a module's environment
// routinely contains all three — a generated password is arbitrary bytes.
//
// - `%` begins a specifier. `%H` is the hostname, `%i` the instance. A password containing one
// is silently replaced by something else, and the failure is an authentication error nobody
// can explain by looking at the declaration.
// - whitespace separates assignments. `Environment=K=a b` sets K to "a" and then tries to read
// "b" as another assignment.
// - a newline ends the line. What follows it is read as a unit DIRECTIVE, so a value carrying
// one could write ExecStart= and have the machine run something nobody declared.
//
// Quoted, with quotes and backslashes escaped and percent doubled. A container needs none of this
// because `--env` is passed through literally, which is why this had to be found here rather than
// noticed in both.
func unitValue(key, value string) string {
escaped := strings.NewReplacer(
`\`, `\\`,
`"`, `\"`,
"%", "%%",
).Replace(value)
return `"` + key + "=" + escaped + `"`
}
internal/apply/process_test.go
+165
View File
@@ -0,0 +1,165 @@
package apply
import (
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
)
func aProcess() *declaration.Process {
return &declaration.Process{
ID: "server", Type: declaration.TypeProcess, Name: "greeter",
Source: "https://store.invalid/greeter/daemon",
Digest: "sha256:" + strings.Repeat("a", 64),
Run: []string{"node", "index.js"},
Env: map[string]string{"MESH_NODE": "anchor", "A_FIRST": "1"},
}
}
// The unit the mesh writes says what it runs, where, and that it comes back.
func TestTheUnitRunsWhatTheDaemonSaid(t *testing.T) {
unit := unitFor(aProcess())
for _, want := range []string{
"ExecStart=node index.js",
"WorkingDirectory=/var/lib/mesh/daemons/greeter",
"Restart=always",
"WantedBy=multi-user.target",
} {
if !strings.Contains(unit, want) {
t.Fatalf("the unit does not say %q:\n%s", want, unit)
}
}
}
// **Generated whole and saying so.** Every managed file on a machine carries this, because an edit
// that survives until the next declaration and then vanishes is worse than one that is refused.
func TestTheUnitSaysItIsTheMeshs(t *testing.T) {
unit := unitFor(aProcess())
if !strings.HasPrefix(unit, "#") || !strings.Contains(unit, "Do not edit") {
t.Fatalf("the unit does not say it is generated:\n%s", unit)
}
}
// **Deterministic, because the unit is half the daemon's identity.** Environment held in a map
// would be written in Go's iteration order, so every apply would see a different unit and call an
// unchanged daemon changed — restarting it on every declaration for ever.
func TestTheUnitIsTheSameEveryTime(t *testing.T) {
first := unitFor(aProcess())
for i := 0; i < 20; i++ {
if again := unitFor(aProcess()); again != first {
t.Fatalf("two renderings of one daemon differ:\n%s\n---\n%s", first, again)
}
}
// And sorted, so the order is a decision rather than luck.
if strings.Index(first, "A_FIRST") > strings.Index(first, "MESH_NODE") {
t.Fatalf("environment is not in a stable order:\n%s", first)
}
}
// **Two daemons from one bundle differing only in their command are different daemons.** Tracking
// the digest alone would call the second one unchanged and leave the first one running.
func TestAProcesssIdentityIncludesHowItIsRun(t *testing.T) {
one := aProcess()
two := aProcess()
two.Run = []string{"node", "other.js"}
if unitFor(one) == unitFor(two) {
t.Fatal("two daemons with different commands render one unit, so a change would be missed")
}
}
// A process that runs as somebody says so, and one that does not says nothing — rather than naming
// root explicitly, which would be a claim the mesh does not need to make.
func TestAProcessRunsAsWhoItSaid(t *testing.T) {
as := aProcess()
as.User = "greeter"
if !strings.Contains(unitFor(as), "User=greeter") {
t.Fatalf("the unit does not run as the user it named:\n%s", unitFor(as))
}
if strings.Contains(unitFor(aProcess()), "User=") {
t.Fatalf("a process that named no user had one written for it:\n%s", unitFor(aProcess()))
}
}
// **Three modes, one kind.** A scheduled process is a timer plus a unit that finishes, not a unit
// that stays up — and the difference has to be in what is written, or a schedule becomes a second
// copy running continuously between fires.
func TestAScheduledProcessRunsOnItsCadenceRatherThanContinuously(t *testing.T) {
every := aProcess()
every.Schedule = "0 3 * * *"
unit := unitFor(every)
if strings.Contains(unit, "Restart=always") {
t.Fatalf("a scheduled process is restarted whenever it exits, so it never stops:\n%s", unit)
}
if !strings.Contains(unit, "Type=oneshot") {
t.Fatalf("a scheduled process is not a step that finishes:\n%s", unit)
}
timer := timerFor(every)
if !strings.Contains(timer, "OnCalendar=") {
t.Fatalf("a scheduled process has no cadence:\n%s", timer)
}
// A fire missed while the machine was off happens when it returns, rather than being skipped —
// the difference between a machine that was down and a schedule that quietly stopped.
if !strings.Contains(timer, "Persistent=true") {
t.Fatalf("a missed fire is skipped silently:\n%s", timer)
}
}
// Five-field cron becomes what a timer reads, rather than the host waking to decide.
func TestACronBecomesATimersCalendar(t *testing.T) {
for cron, want := range map[string]string{
"0 3 * * *": "*-*-* 3:0:00",
"30 4 1 * *": "*-*-1 4:30:00",
"0 0 * * mon": "mon *-*-* 0:0:00",
} {
if got := calendarFor(cron); got != want {
t.Fatalf("%q became %q rather than %q", cron, got, want)
}
}
}
// And the long-running mode is unchanged by any of it: it stays up and comes back.
func TestAProcessThatStaysUpIsStillRestartedWhenItExits(t *testing.T) {
unit := unitFor(aProcess())
if !strings.Contains(unit, "Restart=always") {
t.Fatalf("a process that should stay up is not restarted when it exits:\n%s", unit)
}
if strings.Contains(unit, "Type=oneshot") {
t.Fatalf("a process that should stay up is declared a step:\n%s", unit)
}
}
// **A unit file reinterprets a value in three ways nothing else does**, and a module's environment
// routinely contains all three — a generated password is arbitrary bytes.
func TestAnEnvironmentValueMeansWhatTheDeclarationSaid(t *testing.T) {
// A percent begins a specifier: %H is the hostname. A password containing one would be
// silently replaced, failing as an authentication error nobody can explain from the
// declaration.
percent := aProcess()
percent.Env = map[string]string{"PASSWORD": "a%Hb"}
if !strings.Contains(unitFor(percent), "%%H") {
t.Fatalf("a percent was left as a systemd specifier:\n%s", unitFor(percent))
}
// Whitespace separates assignments: unquoted, K=a b sets K to "a" and reads "b" as another.
spaced := aProcess()
spaced.Env = map[string]string{"GREETING": "hello there"}
if !strings.Contains(unitFor(spaced), `"GREETING=hello there"`) {
t.Fatalf("a value with a space was not quoted:\n%s", unitFor(spaced))
}
// A quote would end the quoting early, and what follows would be read as unit syntax.
quoted := aProcess()
quoted.Env = map[string]string{"TOKEN": `a"b`}
line := ""
for _, l := range strings.Split(unitFor(quoted), "\n") {
if strings.HasPrefix(l, "Environment=") {
line = l
}
}
if !strings.Contains(line, `\"`) {
t.Fatalf("a quote was not escaped, so the value ends early: %s", line)
}
}
internal/bootstrap/apply.go
+4 -4
View File
@@ -15,7 +15,7 @@ import (
// Runner is the same runner every applier in this repository takes.
type Runner = apply.Runner
// ApplyBundle raises the substrate, through the host's own apply.
// ApplyBundle raises the foundation, through the host's own apply.
//
// **This calls `internal/apply` rather than running the `mesh-host` binary**, and that is worth
// stating because shelling out would have been easier. The installer and the host must apply a
@@ -25,7 +25,7 @@ type Runner = apply.Runner
// raising, which would make the installer depend on the thing it installs.
//
// It applies under `store.OriginCarried`, which is the same origin `mesh-host reconcile` uses and
// is not a detail: what the substrate raised must be invisible to the removal pass of a
// is not a detail: what the foundation raised must be invisible to the removal pass of a
// declaration that later arrives from the control plane, or the first thing the mesh tells this
// node would tear down the mesh (novox/hq 04-ISSUES/010).
//
@@ -71,12 +71,12 @@ func ApplyBundle(ctx context.Context, o Options, sys system.System, d *declarati
//
// It cannot happen and it is refused with a sentence rather than a nil dereference. A sealing key
// is generated at enrolment (`internal/identity`), and enrolment is something that happens on a
// mesh — which is the thing this program is raising. A substrate bundle carrying a sealed file
// mesh — which is the thing this program is raising. A foundation bundle carrying a sealed file
// would be a bundle written for a node that has already joined.
func refuseSealed(string) ([]byte, error) {
return nil, errors.New(
"this bundle contains a file sealed to a node's key, and a machine that has not enrolled " +
"has no such key. A substrate is applied before any mesh exists, so it can carry no " +
"has no such key. A foundation is applied before any mesh exists, so it can carry no " +
"secret the mesh sealed")
}
internal/bootstrap/apply_test.go
+3 -3
View File
@@ -78,12 +78,12 @@ func TestASystemNobodyHasBuiltIsRefusedByName(t *testing.T) {
}
}
// A substrate is applied before any mesh exists, so it can carry no secret the mesh sealed — there
// A foundation is applied before any mesh exists, so it can carry no secret the mesh sealed — there
// is no key to open one with. Refused with a sentence rather than a nil dereference.
func TestASealedFileInASubstrateIsRefusedWithAReason(t *testing.T) {
func TestASealedFileInAFoundationIsRefusedWithAReason(t *testing.T) {
_, err := refuseSealed("anything")
if err == nil {
t.Fatal("a sealed file in a substrate bundle was accepted")
t.Fatal("a sealed file in a foundation bundle was accepted")
}
if !strings.Contains(err.Error(), "has not enrolled") {
t.Errorf("the refusal does not say why there is no key: %v", err)
internal/bootstrap/bootstrap.go
+103 -17
View File
@@ -53,6 +53,14 @@ const (
StepControlPlane Step = "control-plane"
StepRetire Step = "retire"
StepBuilder Step = "builder"
StepPackages Step = "packages"
StepSDK Step = "sdk"
StepBase Step = "base"
StepStore Step = "store"
StepCatalogue Step = "catalogue"
StepNetwork Step = "network"
StepFilter Step = "filter"
StepExtras Step = "extras"
)
// Steps in the order they happen, so a failure can say "step 2 of 11".
@@ -69,6 +77,11 @@ const (
var Steps = []Step{
StepPreflight, StepLoad, StepBuild, StepBundle, StepApply, StepVerify,
StepEnrol, StepRegistry, StepPublish, StepControlPlane, StepRetire, StepBuilder,
StepPackages, StepSDK,
// Phase two. The twelve above make a mesh that RUNS; these make one that WORKS — able to
// build, to say what it holds, on its network, filtering. They used to be things somebody
// typed afterwards, which is how they went missing without anything complaining.
StepBase, StepStore, StepCatalogue, StepNetwork, StepFilter, StepExtras,
}
// Error is a failure, named by the step it happened in.
@@ -98,7 +111,7 @@ func failed(step Step, err error) error {
// Options are the things that differ between machines.
type Options struct {
// Template is the substrate bundle this machine's own bundle is made from.
// Template is the foundation bundle this machine's own bundle is made from.
Template string
// Out is where the produced bundle is written, so a person can read what was applied.
Out string
@@ -115,12 +128,12 @@ type Options struct {
// runtime, a control plane opening its stores.
Wait time.Duration
// Node is the name this machine is known by in the mesh. Everything after the substrate names
// Node is the name this machine is known by in the mesh. Everything after the foundation names
// it: the record, the token, the assignment, the push.
Node string
// Catalogue is a checkout of the mesh's catalogue repository, which is where the registry's and
// the control plane's manifests are read from. Empty stops the installer after the substrate:
// the control plane's manifests are read from. Empty stops the installer after the foundation:
// there is no pivot without manifests, and pretending otherwise would leave a machine that
// looks installed and cannot upgrade itself.
Catalogue string
@@ -145,9 +158,30 @@ type Options struct {
// HostInBackground starts the host unsupervised instead, which is what the lab does and what no
// real machine should do — it does not survive a reboot.
HostInBackground bool
// ---- phase two — a mesh that runs becomes a mesh that works ----
// ToolsSource is where the shared base is built from. Everything with code of its own
// compiles against it, so it is the first thing the mesh builds for itself.
ToolsSource Source
// CatalogSource is where the catalogue REPOSITORY is, for building its modules. The catalogue
// CHECKOUT (Catalogue, above) says what a module is; this is where a builder clones it.
CatalogSource Source
// SDKSource is where the mesh's shared library is built from. It is published to the package
// registry before the base is built, because the base resolves it by version (novox/hq ADR 0076).
SDKSource Source
// Site is where this machine sits, for the private network's placement.
Site string
// Answers are the choices, answered by flag: name → answer. What a terminal would be asked.
Answers map[string]string
// Prompt asks a person one choice. Nil is an unattended run: flags and defaults answer, and a
// required choice nothing answered is a refusal rather than a guess.
Prompt func(Choice) (string, error)
// Extras are catalogue modules beyond the floor, asked for by name.
Extras []string
}
// pivots reports whether this run goes past the substrate.
// pivots reports whether this run goes past the foundation.
func (o Options) pivots() bool { return strings.TrimSpace(o.Catalogue) != "" }
// Deps are the ways this program reaches outside itself. Injected so the whole of it can be
@@ -211,11 +245,11 @@ type Result struct {
Applied int `json:"applied,omitempty"`
Changed bool `json:"changed,omitempty"`
// Running is the substrate's containers, confirmed up.
// Running is the foundation's containers, confirmed up.
Running []string `json:"running,omitempty"`
// Answered is what the temporary control plane said back — not merely that it is up.
Answered string `json:"temporary-control-plane,omitempty"`
// Temporary is what the substrate's control plane is called, which is not what the module's is.
// Temporary is what the foundation's control plane is called, which is not what the module's is.
Temporary string `json:"temporary-container,omitempty"`
// Node is this machine's name in the mesh, and how it came to be enrolled and heard from.
@@ -255,15 +289,15 @@ type Result struct {
// answer to it is to run this again: re-running is the retry, and it is one a person chooses after
// reading which step failed and why.
//
// **Genesis is a pivot** (novox/hq ADR 0067). Steps 1 to 5 raise a substrate whose control plane is
// **Genesis is a pivot** (novox/hq ADR 0067). Steps 1 to 5 raise a foundation whose control plane is
// named by the digest of its own configuration, because nothing has ever served that image and
// nothing could have. Steps 6 to 10 turn that into a mesh that can maintain itself: this machine
// enrols, the registry module is installed, the carried image is pushed INTO that registry — which
// gives it a manifest digest, its first — and the control plane is reinstalled as an ordinary
// module pinned to it. The temporary one is then dropped from the bundle and the host removes it.
//
// **What makes the last part expressible is a name.** The substrate's control plane is called
// `temp-mesh-control` and the module's is called `mesh-control`. Two containers, two owners:
// **What makes the last part expressible is a name.** The foundation's control plane is called
// `temp-mesh-controller` and the module's is called `mesh-controller`. Two containers, two owners:
// nothing is handed over, nothing has to stop being owned without being destroyed, and destruction
// by omission is the right end for something named "temp".
//
@@ -275,7 +309,7 @@ type Result struct {
// be fixed remotely — so no step may leave one:
//
// 1–3 nothing on the machine but a written file. Re-run: the bundle is produced again.
// 4 a partly-raised substrate, recorded in the state file. Re-run: apply converges the rest.
// 4 a partly-raised foundation, recorded in the state file. Re-run: apply converges the rest.
// 5 everything up; something did not answer yet. Re-run: it is asked again.
// 6 a node record and possibly a spent token. Re-run: `node list` finds the record, the
// identity file says whether this machine enrolled, and a fresh token is issued if not.
@@ -424,7 +458,7 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
o.Out, rewritten.Resources))
// ---- 4. apply -----------------------------------------------------------------------
say("apply — raising the substrate")
say("apply — raising the foundation")
report, err := ApplyBundle(ctx, o, sys, rewritten.Declaration, d.Run, say)
result.Applied, result.Changed = len(report.Outcomes), report.Changed()
if err != nil {
@@ -438,7 +472,7 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
}
// ---- 5. verify ----------------------------------------------------------------------
say("verify — the substrate is up, and the control plane replies")
say("verify — the foundation is up, and the control plane replies")
verified, err := Verify(ctx, rewritten.Declaration, d.Run, o.Timeout, o.Wait, say)
result.Running, result.Answered = verified.Running, verified.Answered
if err != nil {
@@ -450,7 +484,7 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
// control plane is named by an image id, which no registry serves, so nothing can ever
// replace it with a newer one. That is the whole of what the pivot fixes, and it needs
// manifests, and manifests come from a checkout somebody has to point this at.
result.Stopped = "no --catalog was given, so this stopped at the substrate. " +
result.Stopped = "no --catalog was given, so this stopped at the foundation. " +
"The control plane is named by the digest of its own configuration and no registry " +
"serves it, so this mesh cannot yet upgrade itself. Run again with " +
"--catalog <a checkout of the mesh's catalogue> to finish the pivot; every step " +
@@ -463,7 +497,7 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
// ---- 6. enrol -------------------------------------------------------------------------
//
// From here on the mesh is being told things, and the way to tell it anything is to run its
// own binary inside its own container. `temporary` is the substrate's control plane; the
// own binary inside its own container. `temporary` is the foundation's control plane; the
// module's is a different container with a different name and does not exist yet.
temporary := controlPlane{container: rewritten.TempName, run: d.Run, timeout: o.Timeout}
@@ -535,9 +569,61 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
return result, failed(StepBuilder, err)
}
say("\nthis machine is a mesh of one node, and the control plane it runs is a module " +
"pinned to an image its own registry serves.")
say("it holds a builder, so it can make the rest of the catalogue rather than be handed it.")
// ---- packages — the registry, before the base that resolves the SDK from it ----------
say("packages — a registry answers, and the SDK is in it, before the base is built")
if err := RaisePackageRegistry(ctx, o, d, permanentControl, say); err != nil {
return result, failed(StepPackages, err)
}
// ---- sdk — published on a public base, so this needs no toolchain --------------------
say("sdk — the shared library, published by version so the base can resolve it")
if err := PublishTheSDK(ctx, o, permanentControl, say); err != nil {
return result, failed(StepSDK, err)
}
// ---- 13. base -------------------------------------------------------------------------
say("base — the shared toolchain and runtime everything with code stands on")
if err := BuildBase(ctx, o, permanentControl, say); err != nil {
return result, failed(StepBase, err)
}
// ---- 14. store ------------------------------------------------------------------------
// The foundation's own store, ADOPTED as the `postgres` module (novox/hq issue 051): one
// server holds the control plane's contexts and the database of each module that asks for one,
// rather than the
// foundation's store beside a second one a module raised. Adopted in place — the module names
// the container the foundation is already running, and the applier leaves it be (phase3.go).
say("store — the foundation's store, adopted as the postgres module: one server, not two")
if err := InstallStore(ctx, o, permanentControl, rewritten.Declaration, say); err != nil {
return result, failed(StepStore, err)
}
// ---- 15. catalogue --------------------------------------------------------------------
say("catalogue — the module graph: what is held, what a change reaches, what to rebuild")
if err := InstallFromCatalogue(ctx, o, permanentControl, "mesh-catalog", say); err != nil {
return result, failed(StepCatalogue, err)
}
// ---- 16. network ----------------------------------------------------------------------
say("network — the private network, and this machine's name on it")
if err := PlaceOnTheNetwork(ctx, o, permanentControl, rewritten.BrokerAddress, say); err != nil {
return result, failed(StepNetwork, err)
}
// ---- 17. filter -----------------------------------------------------------------------
say("filter — required, so the question is which, not whether")
if err := ChooseAndInstallFilter(ctx, o, permanentControl, say); err != nil {
return result, failed(StepFilter, err)
}
// ---- 18. extras -----------------------------------------------------------------------
say("extras — beyond the floor, if asked")
if err := InstallExtras(ctx, o, permanentControl, say); err != nil {
return result, failed(StepExtras, err)
}
say("\nthis machine is a mesh of one node: it builds its own software, holds its graph, " +
"sits on its private network, and filters what modules declared.")
say("what remains is somebody else's: adding nodes, and assigning what they should run.")
return result, nil
internal/bootstrap/build_test.go
+2 -2
View File
@@ -24,7 +24,7 @@ func TestAnInstallerWithNothingToBuildRefuses(t *testing.T) {
// A repository without a commit refuses too, because a branch is somebody else's moving target.
func TestABranchIsNotACommit(t *testing.T) {
err := Source{Repository: "https://example.invalid/mesh-control.git"}.Check()
err := Source{Repository: "https://example.invalid/mesh-controller.git"}.Check()
if err == nil {
t.Fatal("a source with no ref was accepted; genesis would have built whatever a branch pointed at")
}
@@ -35,7 +35,7 @@ func TestABranchIsNotACommit(t *testing.T) {
// And a repository with a commit is enough.
func TestARepositoryAndACommitIsEnough(t *testing.T) {
if err := (Source{Repository: "https://example.invalid/mesh-control.git", Ref: "a1b2c3d4"}).Check(); err != nil {
if err := (Source{Repository: "https://example.invalid/mesh-controller.git", Ref: "a1b2c3d4"}).Check(); err != nil {
t.Fatalf("a repository and a commit were refused: %v", err)
}
}
internal/bootstrap/builder.go
+1 -1
View File
@@ -53,7 +53,7 @@ func InstallBuilder(ctx context.Context, o Options, d Deps, control controlPlane
"This is the manifest that makes the builder an ordinary module. Without it the mesh "+
"has the image and no way to run it, so nothing can be built here", err)
}
pinned, places, err := pinImage(manifest, published.Reference)
pinned, places, err := pinImage(manifest, published.Reference, BuilderModule)
if err != nil {
return out, err
}
internal/bootstrap/choices.go
+79
View File
@@ -0,0 +1,79 @@
package bootstrap
import (
"fmt"
"strings"
)
// What the installer asks a person, and how an unattended run answers.
//
// **The installer goes as far as it can, and where a human must choose, it asks** — a packet
// filter is required, so the question is not whether but which. A run with a terminal is asked;
// a run without one (the lab, an unattended machine) answers with flags, and a required choice
// with no flag, no terminal and no lone option is a refusal rather than a guess.
// Choice is one question the installer needs answered.
type Choice struct {
// Name is the flag that answers it unattended: --packet-filter, --private-network.
Name string
// Question is what a person is asked, ending with what the options are.
Question string
// Options are the acceptable answers. Empty means free-form (an address, a list).
Options []string
// Default is used when nothing and nobody answered. Empty means the choice is required.
Default string
}
// decide resolves one choice, in the order a person would expect:
//
// an explicit answer (the flag) wins; a lone option answers itself, said aloud; a terminal is
// asked; a default fills in; and a required choice nothing answered is refused naming its flag.
func decide(c Choice, answered string, prompt func(Choice) (string, error), say func(string)) (string, error) {
if got := strings.TrimSpace(answered); got != "" {
return validated(c, got)
}
if len(c.Options) == 1 {
// The only answer there is. Said rather than silent, because "it chose for me" and "there
// was nothing to choose" read identically afterwards unless one of them says so.
say(fmt.Sprintf(" %-17s %s — the only option there is", c.Name, c.Options[0]))
return c.Options[0], nil
}
if prompt != nil {
got, err := prompt(c)
if err != nil {
return "", err
}
if strings.TrimSpace(got) == "" && c.Default != "" {
say(fmt.Sprintf(" %-17s %s (default)", c.Name, c.Default))
return c.Default, nil
}
return validated(c, strings.TrimSpace(got))
}
if c.Default != "" {
say(fmt.Sprintf(" %-17s %s (default)", c.Name, c.Default))
return c.Default, nil
}
return "", fmt.Errorf(
"%s must be chosen and nothing chose it: no --%s, no terminal to ask on%s",
c.Name, c.Name, orOptions(c))
}
func validated(c Choice, got string) (string, error) {
if len(c.Options) == 0 {
return got, nil
}
for _, option := range c.Options {
if got == option {
return got, nil
}
}
return "", fmt.Errorf("%q is not a %s this mesh offers. It has %s",
got, c.Name, strings.Join(c.Options, ", "))
}
func orOptions(c Choice) string {
if len(c.Options) == 0 {
return ""
}
return ". It offers " + strings.Join(c.Options, ", ")
}
internal/bootstrap/choices_test.go
+64
View File
@@ -0,0 +1,64 @@
package bootstrap
import (
"strings"
"testing"
)
func quietly(string) {}
// A flag answers, and answers wrongly is refused naming what would have worked.
func TestAFlagAnswersAndAWrongOneIsRefused(t *testing.T) {
filter := Choice{Name: "packet-filter", Options: []string{"nftables", "ufw"}}
got, err := decide(filter, "ufw", nil, quietly)
if err != nil || got != "ufw" {
t.Fatalf("an explicit answer was not taken: %q, %v", got, err)
}
_, err = decide(filter, "iptables", nil, quietly)
if err == nil {
t.Fatal("an answer nothing offers was accepted")
}
if !strings.Contains(err.Error(), "nftables") || !strings.Contains(err.Error(), "ufw") {
t.Fatalf("the refusal does not say what would have worked: %v", err)
}
}
// A lone option answers itself — and says so, because "it chose for me" and "there was nothing to
// choose" read identically afterwards unless one of them speaks.
func TestALoneOptionAnswersItselfAloud(t *testing.T) {
var said []string
got, err := decide(Choice{Name: "private-network", Options: []string{"wireguard"}},
"", nil, func(line string) { said = append(said, line) })
if err != nil || got != "wireguard" {
t.Fatalf("the only option was not taken: %q, %v", got, err)
}
if len(said) == 0 || !strings.Contains(said[0], "only option") {
t.Fatalf("choosing silently: %v", said)
}
}
// A terminal is asked; an empty answer takes the default when there is one.
func TestATerminalIsAskedAndEmptyTakesTheDefault(t *testing.T) {
asked := 0
prompt := func(c Choice) (string, error) { asked++; return "", nil }
got, err := decide(Choice{Name: "extras", Default: "none"}, "", prompt, quietly)
if err != nil || got != "none" || asked != 1 {
t.Fatalf("empty answer at a prompt did not take the default: %q asked=%d %v", got, asked, err)
}
}
// **Unattended and required is a refusal, not a guess.** The lab and any machine without a
// terminal must be answerable by flags — and a required choice with no flag has to stop the run
// naming the flag, because a guessed packet filter is a machine somebody else configured.
func TestUnattendedAndRequiredRefusesNamingTheFlag(t *testing.T) {
_, err := decide(Choice{Name: "packet-filter", Options: []string{"nftables", "ufw"}},
"", nil, quietly)
if err == nil {
t.Fatal("a required choice was guessed for an unattended run")
}
if !strings.Contains(err.Error(), "--packet-filter") {
t.Fatalf("the refusal does not name the flag that answers it: %v", err)
}
}
internal/bootstrap/control.go
+27 -27
View File
@@ -14,7 +14,7 @@ import (
// storeFileSuffix is how a manifest asks for a store connection in a file rather than in the
// environment.
//
// `MESH_STORE_<CONTEXT>` is what the control plane reads (mesh-control's `internal/store`.Variable)
// `MESH_STORE_<CONTEXT>` is what the control plane reads (mesh-controller's `internal/store`.Variable)
// and putting a password in a container's environment puts it in `docker inspect` for ever. So a
// module manifest names a file per context and points at it with `…_FILE`; the mesh seals the value
// into that file on the machine, and nothing but the process reads it.
@@ -41,20 +41,20 @@ type Permanent struct {
// **The host performs the replacement, not the control plane** (novox/hq ADR 0067). The temporary
// control plane composes a declaration naming the registry-pinned image, publishes it, and this
// node's host creates the container. Nothing is asked to replace itself while running, which is
// what makes the whole thing expressible: the container being created is called `mesh-control` and
// the one composing it is called `temp-mesh-control`, so there are two of them and neither is in
// what makes the whole thing expressible: the container being created is called `mesh-controller` and
// the one composing it is called `temp-mesh-controller`, so there are two of them and neither is in
// the other's way.
//
// **The store connections are the substrate's, made at genesis, and the mesh cannot invent them.**
// **The store connections are the foundation's, made at genesis, and the mesh cannot invent them.**
// Every other secret in a mesh is one the mesh made; these existed before the mesh did — they are
// the credentials the substrate bundle created the databases with. Generating replacements would
// the credentials the foundation bundle created the databases with. Generating replacements would
// put thirty-two random bytes where a working connection string has to be, and the control plane
// would come up unable to open a single context. So they go in through `secret accept`, which is
// exactly the path for a value the mesh must carry and could not have invented — and they are read
// out of the bundle this installer produced rather than reconstructed, because the bundle is what
// created them and a second opinion about what a DSN should say is a second chance to be wrong.
func InstallControlPlane(ctx context.Context, o Options, d Deps, control controlPlane,
substrate *declaration.Declaration, image string, say func(string)) (Permanent, error) {
foundation *declaration.Declaration, image string, say func(string)) (Permanent, error) {
out := Permanent{Image: image}
@@ -63,12 +63,12 @@ func InstallControlPlane(ctx context.Context, o Options, d Deps, control control
return out, fmt.Errorf(
"%w\n"+
"This is the manifest that makes the control plane an ordinary module. Without it "+
"the machine keeps the temporary control plane the substrate raised, which works "+
"the machine keeps the temporary control plane the foundation raised, which works "+
"and cannot be upgraded — so the install stops here rather than pretending to "+
"have pivoted", err)
}
pinned, places, err := pinImage(manifest, image)
pinned, places, err := pinImage(manifest, image, ControlPlaneModule)
if err != nil {
return out, err
}
@@ -92,7 +92,7 @@ func InstallControlPlane(ctx context.Context, o Options, d Deps, control control
}
// The connections, before the push that would otherwise deliver random bytes for them.
delivered, err := deliverStores(ctx, o, control, pinned, substrate, say)
delivered, err := deliverStores(ctx, o, control, pinned, foundation, say)
out.Delivered = delivered
if err != nil {
return out, err
@@ -107,7 +107,7 @@ func InstallControlPlane(ctx context.Context, o Options, d Deps, control control
}
// And it answers, which is the same question step 5 asked of the temporary one and for the
// same reason: `status` opens all three stores, so a reply proves the sealed connections it
// was given are the ones the substrate made. Asked of the NEW container — this is the only
// was given are the ones the foundation made. Asked of the NEW container — this is the only
// moment in the program where two control planes are running, and asking the wrong one would
// report the temporary one's health as the permanent one's.
answered, err := waitForTheControlPlane(ctx, control.run, o.Timeout, o.Wait, container, say)
@@ -130,19 +130,19 @@ func InstallControlPlane(ctx context.Context, o Options, d Deps, control control
// carrying a real digest is one somebody pinned by hand, and quietly registering it would install a
// control plane that is not the image this machine just published — which is the one thing this
// step exists to guarantee.
func pinImage(manifest []byte, reference string) ([]byte, int, error) {
func pinImage(manifest []byte, reference, module string) ([]byte, int, error) {
places := bytes.Count(manifest, []byte(placeholderDigest))
if places == 0 {
return nil, 0, fmt.Errorf(
"the %s module's manifest carries no placeholder digest (%s), so there is nothing to "+
"pin to the image this machine just published.\n"+
"A manifest already naming a digest was pinned by somebody else, to some other "+
"build. Registering it would install a control plane that is not the one this "+
"installer carried and pushed", ControlPlaneModule, placeholderDigest)
"build. Registering it would install a module that is not the one this "+
"installer carried and pushed", module, placeholderDigest)
}
// The reference the registry gave back is `<registry>/<repository>@sha256:…`, and what the
// manifest holds is `<something>@sha256:0…0`. Replacing only the digest would leave the
// manifest's own repository name in front of it — which may be `mesh-control` with no
// manifest's own repository name in front of it — which may be `mesh-controller` with no
// registry, and a runtime would then pull it from the internet. The whole reference moves.
var out bytes.Buffer
rest := manifest
@@ -157,7 +157,7 @@ func pinImage(manifest []byte, reference string) ([]byte, int, error) {
if start < 0 {
return nil, 0, fmt.Errorf(
"the %s module's manifest has a placeholder digest that is not inside a JSON "+
"string, so the installer cannot tell what image it belongs to", ControlPlaneModule)
"string, so the installer cannot tell what image it belongs to", module)
}
out.Write(rest[:start+1])
out.WriteString(reference)
@@ -170,12 +170,12 @@ func pinImage(manifest []byte, reference string) ([]byte, int, error) {
var checked map[string]any
if err := json.Unmarshal(pinned, &checked); err != nil {
return nil, 0, fmt.Errorf(
"pinning the %s module's image broke its manifest: %w", ControlPlaneModule, err)
"pinning the %s module's image broke its manifest: %w", module, err)
}
if bytes.Contains(pinned, []byte(placeholderDigest)) {
return nil, 0, fmt.Errorf(
"the %s module's manifest still carries a placeholder digest after pinning",
ControlPlaneModule)
module)
}
return pinned, places, nil
}
@@ -215,21 +215,21 @@ func controlPlaneResourceIn(manifest []byte) string {
return ""
}
// deliverStores carries the substrate's own database connections into the module.
// deliverStores carries the foundation's own database connections into the module.
//
// The pairing is read from the manifest rather than assumed, so that whatever the catalogue calls
// these secrets is what is delivered. The installer does not guess that the secret holding the
// inventory connection is called `inventory`; it follows the manifest from the variable to the
// secret, and a manifest whose two ends do not meet is refused rather than half-delivered.
//
// **What is delivered is what the substrate already has, and only that.** The mesh generates an
// **What is delivered is what the foundation already has, and only that.** The mesh generates an
// own-secret nobody supplied, which is right for something coming into existence and wrong for
// something that already exists. So every variable the module fills from a secret is looked up in
// the substrate's control plane: what it names is accepted, what it does not is left for the mesh
// to make. A store connection missing from the substrate is the one exception and is an error —
// the foundation's control plane: what it names is accepted, what it does not is left for the mesh
// to make. A store connection missing from the foundation is the one exception and is an error —
// a control plane that cannot open a context is not a control plane.
func deliverStores(ctx context.Context, o Options, control controlPlane, manifest []byte,
substrate *declaration.Declaration, say func(string)) ([]string, error) {
foundation *declaration.Declaration, say func(string)) ([]string, error) {
wanted, err := secretsByVariableIn(manifest)
if err != nil {
@@ -246,7 +246,7 @@ func deliverStores(ctx context.Context, o Options, control controlPlane, manifes
ControlPlaneModule, storeVariablePrefix, storeVariablePrefix, storeFileSuffix)
}
temporary, err := controlPlaneIn(substrate)
temporary, err := controlPlaneIn(foundation)
if err != nil {
return nil, err
}
@@ -260,13 +260,13 @@ func deliverStores(ctx context.Context, o Options, control controlPlane, manifes
return delivered, fmt.Errorf(
"the %s module wants %s and the bundle this installer produced does not name "+
"one.\n"+
"That connection is the substrate's, created at genesis — the mesh cannot "+
"That connection is the foundation's, created at genesis — the mesh cannot "+
"invent it and the installer will not guess at one",
ControlPlaneModule, variable)
}
// Not something the substrate made. The mesh generates its own, which is exactly what
// Not something the foundation made. The mesh generates its own, which is exactly what
// an own-secret is for; said so that nothing about the delivery is silent.
say(" the mesh will make " + secret + " — the substrate names no " + variable)
say(" the mesh will make " + secret + " — the foundation names no " + variable)
continue
}
@@ -282,7 +282,7 @@ func deliverStores(ctx context.Context, o Options, control controlPlane, manifes
return delivered, err
}
delivered = append(delivered, secret)
say(" accepted " + secret + " — " + variable + ", as the substrate made it")
say(" accepted " + secret + " — " + variable + ", as the foundation made it")
}
return delivered, nil
}
internal/bootstrap/control_test.go
+52 -52
View File
@@ -9,37 +9,37 @@ import (
// Step 9 is where the control plane stops being a special case. These tests defend the two things
// that could go wrong quietly: pinning it to the wrong image, and delivering it store connections
// the mesh invented rather than the ones the substrate actually made.
// the mesh invented rather than the ones the foundation actually made.
// theControlPlaneModule is the catalogue's manifest, trimmed to what this installer reads.
//
// A fixture rather than the file itself, unlike the substrate example the rewrite tests use: the
// A fixture rather than the file itself, unlike the foundation example the rewrite tests use: the
// catalogue is a different repository on a different branch, and a test that read it would pass or
// fail according to what somebody else had checked out. What it must stay faithful to is the
// SHAPE — the placeholder digest, the own-secret per context, the mount from the machine's path to
// the container's, and the environment file that fills what is not a path.
const theControlPlaneModule = `{
"module": "mesh-control",
"module": "mesh-controller",
"version": "1",
"slug": "control",
"capabilities": ["container-runtime"],
"claims": [{"name": "the-control-plane", "scope": "mesh"}],
"claims": [{"name": "the-controller", "scope": "mesh"}],
"own-secrets": {
"inventory": "/var/lib/mesh/mesh-control/inventory",
"identity": "/var/lib/mesh/mesh-control/identity",
"licences": "/var/lib/mesh/mesh-control/licences",
"broker": "/var/lib/mesh/mesh-control/broker",
"broker-management": "/var/lib/mesh/mesh-control/broker-management"
"inventory": "/var/lib/mesh/mesh-controller/inventory",
"identity": "/var/lib/mesh/mesh-controller/identity",
"licences": "/var/lib/mesh/mesh-controller/licences",
"broker": "/var/lib/mesh/mesh-controller/broker",
"broker-management": "/var/lib/mesh/mesh-controller/broker-management"
},
"resources": [
{"id": "mesh-state", "type": "directory", "path": "/var/lib/mesh/mesh-control", "mode": "0700"},
{"id": "broker-env", "type": "file", "path": "/var/lib/mesh/mesh-control/broker.env",
{"id": "mesh-state", "type": "directory", "path": "/var/lib/mesh/mesh-controller", "mode": "0700"},
{"id": "broker-env", "type": "file", "path": "/var/lib/mesh/mesh-controller/broker.env",
"mode": "0600",
"content": "MESH_BROKER_AMQP=${secret:broker}\nMESH_BROKER_MANAGEMENT=${secret:broker-management}\nMESH_BROKER_ADDRESS=${machine:at}:5671\n"},
{"id": "server", "type": "container", "name": "mesh-control",
"image": "mesh-control@` + placeholderDigest + `",
{"id": "server", "type": "container", "name": "mesh-controller",
"image": "mesh-controller@` + placeholderDigest + `",
"network": "host", "args": ["serve"],
"env-file": ["/var/lib/mesh/mesh-control/broker.env"],
"env-file": ["/var/lib/mesh/mesh-controller/broker.env"],
"env": {
"MESH_STORE_INVENTORY_FILE": "/run/secrets/inventory",
"MESH_STORE_IDENTITY_FILE": "/run/secrets/identity",
@@ -48,22 +48,22 @@ const theControlPlaneModule = `{
},
"volumes": [
"mesh-broker-tls:/broker-tls:ro",
"/var/lib/mesh/mesh-control/inventory:/run/secrets/inventory:ro",
"/var/lib/mesh/mesh-control/identity:/run/secrets/identity:ro",
"/var/lib/mesh/mesh-control/licences:/run/secrets/licences:ro"
"/var/lib/mesh/mesh-controller/inventory:/run/secrets/inventory:ro",
"/var/lib/mesh/mesh-controller/identity:/run/secrets/identity:ro",
"/var/lib/mesh/mesh-controller/licences:/run/secrets/licences:ro"
]}
]
}`
const pushedReference = "127.0.0.1:5000/mesh-control@sha256:" +
const pushedReference = "127.0.0.1:5000/mesh-controller@sha256:" +
"eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
// **The whole reference moves, not only the digest.** The manifest's placeholder names a
// repository too, and replacing sixty-four zeros inside it would leave `mesh-control@sha256:…`
// repository too, and replacing sixty-four zeros inside it would leave `mesh-controller@sha256:…`
// with no registry in front — which a runtime would go to the internet for, and this mesh's
// control plane exists in no public registry by design.
func TestTheControlPlaneIsPinnedToWhatThisMeshsRegistryAssigned(t *testing.T) {
pinned, places, err := pinImage([]byte(theControlPlaneModule), pushedReference)
pinned, places, err := pinImage([]byte(theControlPlaneModule), pushedReference, "mesh-controller")
if err != nil {
t.Fatal(err)
}
@@ -73,7 +73,7 @@ func TestTheControlPlaneIsPinnedToWhatThisMeshsRegistryAssigned(t *testing.T) {
if !strings.Contains(string(pinned), `"image": "`+pushedReference+`"`) {
t.Errorf("the manifest does not name the pushed image:\n%s", pinned)
}
if strings.Contains(string(pinned), `"mesh-control@sha256:`) {
if strings.Contains(string(pinned), `"mesh-controller@sha256:`) {
t.Errorf("the digest was replaced and the manifest's own repository name was left in "+
"front of it, so nothing says which registry serves it:\n%s", pinned)
}
@@ -85,7 +85,7 @@ func TestTheControlPlaneIsPinnedToWhatThisMeshsRegistryAssigned(t *testing.T) {
func TestAManifestAlreadyPinnedByHandIsRefused(t *testing.T) {
already := strings.Replace(theControlPlaneModule, placeholderDigest,
"sha256:"+strings.Repeat("9", 64), 1)
if _, _, err := pinImage([]byte(already), pushedReference); err == nil {
if _, _, err := pinImage([]byte(already), pushedReference, "mesh-controller"); err == nil {
t.Fatal("a manifest already pinned to some other image was accepted")
}
}
@@ -95,12 +95,12 @@ func TestAManifestAlreadyPinnedByHandIsRefused(t *testing.T) {
// otherwise be left half pinned, and fail inside an apply rather than here.
func TestEveryPlaceTheManifestNamesTheImageIsPinned(t *testing.T) {
twice := strings.Replace(theControlPlaneModule,
`{"id": "mesh-state", "type": "directory", "path": "/var/lib/mesh/mesh-control", "mode": "0700"},`,
`{"id": "mesh-state", "type": "directory", "path": "/var/lib/mesh/mesh-control", "mode": "0700"},
{"id": "migrate", "type": "container", "name": "mesh-control-migrate", "run-once": true,
"image": "mesh-control@`+placeholderDigest+`", "args": ["migrate"]},`, 1)
`{"id": "mesh-state", "type": "directory", "path": "/var/lib/mesh/mesh-controller", "mode": "0700"},`,
`{"id": "mesh-state", "type": "directory", "path": "/var/lib/mesh/mesh-controller", "mode": "0700"},
{"id": "migrate", "type": "container", "name": "mesh-controller-migrate", "run-once": true,
"image": "mesh-controller@`+placeholderDigest+`", "args": ["migrate"]},`, 1)
pinned, places, err := pinImage([]byte(twice), pushedReference)
pinned, places, err := pinImage([]byte(twice), pushedReference, "mesh-controller")
if err != nil {
t.Fatal(err)
}
@@ -112,8 +112,8 @@ func TestEveryPlaceTheManifestNamesTheImageIsPinned(t *testing.T) {
}
}
// **The connections are the substrate's, and they are read out of the bundle that made them.**
// The mesh cannot invent them: they are the credentials the substrate created the databases with,
// **The connections are the foundation's, and they are read out of the bundle that made them.**
// The mesh cannot invent them: they are the credentials the foundation created the databases with,
// and thirty-two random bytes in their place would leave the control plane unable to open a single
// context. The pairing is read from the manifest so that whatever the catalogue calls these
// secrets is what is delivered.
@@ -141,38 +141,38 @@ func TestTheStoreConnectionsComeFromTheBundleThatMadeThem(t *testing.T) {
t.Error("the address the mesh composes from the machine was treated as a secret")
}
// The values are the substrate's own, taken from the produced bundle rather than composed.
// The values are the foundation's own, taken from the produced bundle rather than composed.
rewritten, err := Rewrite(theRealBundle(t), held)
if err != nil {
t.Fatal(err)
}
runtime := &asked{answer: aMeshThatAgrees(nil)}
control := controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second}
control := controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second}
delivered, err := deliverStores(context.Background(), Options{Node: "anchor"}, control,
[]byte(theControlPlaneModule), rewritten.Declaration, func(string) {})
if err != nil {
t.Fatal(err)
}
// Three stores and both halves of the broker: everything the substrate made and nothing else.
// Three stores and both halves of the broker: everything the foundation made and nothing else.
if len(delivered) != 5 {
t.Fatalf("%d values were delivered, and the substrate names five: %v",
t.Fatalf("%d values were delivered, and the foundation names five: %v",
len(delivered), delivered)
}
for _, secret := range delivered {
if !runtime.ran("secret accept anchor mesh-control " + secret + " --from") {
if !runtime.ran("secret accept anchor mesh-controller " + secret + " --from") {
t.Errorf("%s was not accepted through `secret accept`: %v", secret, runtime.commands)
}
}
}
// A secret the substrate did not make is left for the mesh to make, and said so. Every other
// A secret the foundation did not make is left for the mesh to make, and said so. Every other
// secret in a mesh is one the mesh made; `secret accept` is only for what predates the mesh.
func TestASecretTheSubstrateNeverMadeIsLeftToTheMesh(t *testing.T) {
func TestASecretTheFoundationNeverMadeIsLeftToTheMesh(t *testing.T) {
extra := strings.Replace(theControlPlaneModule,
`"broker": "/var/lib/mesh/mesh-control/broker",`,
`"broker": "/var/lib/mesh/mesh-control/broker",
"something-new": "/var/lib/mesh/mesh-control/something-new",`, 1)
`"broker": "/var/lib/mesh/mesh-controller/broker",`,
`"broker": "/var/lib/mesh/mesh-controller/broker",
"something-new": "/var/lib/mesh/mesh-controller/something-new",`, 1)
extra = strings.Replace(extra,
`"content": "MESH_BROKER_AMQP=${secret:broker}\n`,
`"content": "MESH_SOMETHING_NEW=${secret:something-new}\nMESH_BROKER_AMQP=${secret:broker}\n`, 1)
@@ -182,7 +182,7 @@ func TestASecretTheSubstrateNeverMadeIsLeftToTheMesh(t *testing.T) {
t.Fatal(err)
}
runtime := &asked{answer: aMeshThatAgrees(nil)}
control := controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second}
control := controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second}
var said []string
delivered, err := deliverStores(context.Background(), Options{Node: "anchor"}, control,
@@ -192,7 +192,7 @@ func TestASecretTheSubstrateNeverMadeIsLeftToTheMesh(t *testing.T) {
}
for _, secret := range delivered {
if secret == "something-new" {
t.Error("a value the substrate never made was accepted as though it had")
t.Error("a value the foundation never made was accepted as though it had")
}
}
if !strings.Contains(strings.Join(said, "\n"), "the mesh will make something-new") {
@@ -205,8 +205,8 @@ func TestASecretTheSubstrateNeverMadeIsLeftToTheMesh(t *testing.T) {
// be — which presents as a control plane that will not start, three steps from the cause.
func TestAConnectionFileNothingWritesIsRefused(t *testing.T) {
mismatched := strings.Replace(theControlPlaneModule,
`"inventory": "/var/lib/mesh/mesh-control/inventory",`,
`"inventory": "/var/lib/mesh/mesh-control/somewhere-else",`, 1)
`"inventory": "/var/lib/mesh/mesh-controller/inventory",`,
`"inventory": "/var/lib/mesh/mesh-controller/somewhere-else",`, 1)
_, err := secretsByVariableIn([]byte(mismatched))
if err == nil {
@@ -226,11 +226,11 @@ func TestAManifestWantingNoStoresIsRefusedWithTheShapeItShouldHave(t *testing.T)
t.Fatal(err)
}
runtime := &asked{answer: aMeshThatAgrees(nil)}
control := controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second}
control := controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second}
bare := `{"module":"mesh-control","version":"1","resources":[
{"id":"container","type":"container","name":"mesh-control",
"image":"mesh-control@` + placeholderDigest + `"}]}`
bare := `{"module":"mesh-controller","version":"1","resources":[
{"id":"container","type":"container","name":"mesh-controller",
"image":"mesh-controller@` + placeholderDigest + `"}]}`
_, err = deliverStores(context.Background(), Options{Node: "anchor"}, control,
[]byte(bare), rewritten.Declaration, func(string) {})
@@ -244,13 +244,13 @@ func TestAManifestWantingNoStoresIsRefusedWithTheShapeItShouldHave(t *testing.T)
// The permanent control plane is asked a question, not merely looked at — the same question the
// temporary one was asked at step 5, and for the same reason: `status` opens all three stores, so
// a reply proves the sealed connections it was given are the ones the substrate made.
// a reply proves the sealed connections it was given are the ones the foundation made.
func TestThePermanentControlPlaneIsAskedTheSameQuestion(t *testing.T) {
runtime := &asked{answer: aMeshThatAgrees(map[string]string{
"module list": "",
"exec mesh-control /mesh-control": "1 node, 0 waiting\n",
"exec mesh-controller /mesh-controller": "1 node, 0 waiting\n",
})}
control := controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second}
control := controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second}
rewritten, err := Rewrite(theRealBundle(t), held)
if err != nil {
t.Fatal(err)
@@ -265,11 +265,11 @@ func TestThePermanentControlPlaneIsAskedTheSameQuestion(t *testing.T) {
if out.Answered != "1 node, 0 waiting" {
t.Errorf("the permanent control plane's reply is reported as %q", out.Answered)
}
if !runtime.ran("docker exec mesh-control " + controlPlaneBinary + " status") {
if !runtime.ran("docker exec mesh-controller " + controlPlaneBinary + " status") {
t.Errorf("the permanent control plane was never asked anything: %v", runtime.commands)
}
// And the module was registered with the digest, not with the placeholder.
if !runtime.ran("module add /mesh-control-module.json") {
if !runtime.ran("module add /mesh-controller-module.json") {
t.Errorf("the module was never registered: %v", runtime.commands)
}
}
internal/bootstrap/enrol.go
+1 -1
View File
@@ -13,7 +13,7 @@ import (
// hereIs what `node list` says about a machine the mesh has heard from recently.
//
// mesh-control prints one of three words per node: "here", "never spoken", or "out of touch <age>".
// mesh-controller prints one of three words per node: "here", "never spoken", or "out of touch <age>".
// The installer waits for the first, and it is the only honest proof that the host agent is
// running: an enrolled machine whose host is not running looks exactly like an enrolled machine
// whose host has crashed, and both look exactly like a successful install until the first push
internal/bootstrap/enrol_test.go
+6 -6
View File
@@ -70,7 +70,7 @@ func TestAMachineThatHasAlreadyEnrolledIsNotEnrolledAgain(t *testing.T) {
out, err := Enrol(context.Background(), Options{
Node: "anchor", State: alreadyEnrolled(t, "anchor"), Timeout: time.Second,
Host: "/usr/local/bin/mesh-host", HostInBackground: true,
}, arch(t), controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second},
}, arch(t), controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second},
func(string) {})
if err != nil {
t.Fatal(err)
@@ -109,7 +109,7 @@ func TestAMeshThatHasHeardFromAMachineWithNoAgentStartsOne(t *testing.T) {
out, err := Enrol(context.Background(), Options{
Node: "anchor", State: alreadyEnrolled(t, "anchor"), Timeout: time.Second,
Host: "/usr/local/bin/mesh-host", HostInBackground: true,
}, arch(t), controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second},
}, arch(t), controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second},
func(string) {})
if err != nil {
t.Fatal(err)
@@ -135,7 +135,7 @@ func TestAMachineEnrolledUnderAnotherNameIsRefused(t *testing.T) {
_, err := Enrol(context.Background(), Options{
Node: "anchor", State: alreadyEnrolled(t, "somewhere-else"), Timeout: time.Second,
}, arch(t), controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second},
}, arch(t), controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second},
func(string) {})
if err == nil {
t.Fatal("a machine already enrolled as something else was enrolled again")
@@ -173,7 +173,7 @@ func TestAHostThatIsRunningAndUnheardOfIsNotAnInstall(t *testing.T) {
_, err := Enrol(context.Background(), Options{
Node: "anchor", State: alreadyEnrolled(t, "anchor"), HostService: "mesh-host.service",
Timeout: time.Second, Wait: 0,
}, arch(t), controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second},
}, arch(t), controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second},
func(string) {})
if err == nil {
t.Fatal("a node the mesh has never heard from was reported enrolled and running")
@@ -202,7 +202,7 @@ func TestAMachineWithNoHostServiceIsRefusedRatherThanGivenOne(t *testing.T) {
_, err := Enrol(context.Background(), Options{
Node: "anchor", State: alreadyEnrolled(t, "anchor"), HostService: "mesh-host.service",
Timeout: time.Second, Wait: 0,
}, arch(t), controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second},
}, arch(t), controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second},
func(string) {})
if err == nil {
t.Fatal("a machine with no host service was reported as having a running host")
@@ -222,7 +222,7 @@ func TestAMachineWithNoNameIsRefusedBeforeAnythingIsAsked(t *testing.T) {
return "", fmt.Errorf("nothing should have been asked")
}}
_, err := Enrol(context.Background(), Options{Timeout: time.Second}, arch(t),
controlPlane{container: "temp-mesh-control", run: runtime.run}, func(string) {})
controlPlane{container: "temp-mesh-controller", run: runtime.run}, func(string) {})
if err == nil {
t.Fatal("a machine with no name was enrolled")
}
internal/bootstrap/load.go
+1 -1
View File
@@ -120,7 +120,7 @@ func loadImage(ctx context.Context, run Runner, saved []byte, dryRun bool, say f
// Through a file rather than through stdin: the runner this repository shares runs a command
// and captures its output, and giving it a second mouth for one caller would change every
// applier's contract for the sake of one step (internal/apply's Runner).
tarball, err := os.CreateTemp("", "mesh-control-*.tar")
tarball, err := os.CreateTemp("", "mesh-controller-*.tar")
if err != nil {
return loaded, fmt.Errorf("nowhere to put the carried image while loading it: %w", err)
}
internal/bootstrap/load_test.go
+8 -8
View File
@@ -41,12 +41,12 @@ func (a *asked) ran(fragment string) bool {
return false
}
// savedImageFixture builds what `docker save` produces, tagged `mesh-control:test` unless a test
// savedImageFixture builds what `docker save` produces, tagged `mesh-controller:test` unless a test
// asks for something else. Pass no tags for an archive saved without one.
func savedImageFixture(t *testing.T, digest string, tags ...string) []byte {
t.Helper()
if tags == nil {
tags = []string{"mesh-control:test"}
tags = []string{"mesh-controller:test"}
}
entries, err := json.Marshal([]struct {
Config string
@@ -75,7 +75,7 @@ func savedImageFixture(t *testing.T, digest string, tags ...string) []byte {
// fixtureDigest is what the ARCHIVE calls the image, and runtimeDigest is what a runtime calls it
// after loading the same bytes. They differ on purpose, because they differ in reality: an image
// id is the digest of the image's configuration, and a runtime rewrites that configuration as it
// loads. Measured on a live raise, `mesh-control:development` was `sha256:b86bb81c…` on the
// loads. Measured on a live raise, `mesh-controller:development` was `sha256:b86bb81c…` on the
// workstation that saved it and `sha256:2dc21904…` on the machine that loaded it.
const (
fixtureDigest = "3333333333333333333333333333333333333333333333333333333333333333"
@@ -107,7 +107,7 @@ func TestTheIdComesFromTheRuntimeAndNotFromTheArchive(t *testing.T) {
// Loaded — and stored under a configuration of the runtime's own making.
return "sha256:" + runtimeDigest + "\n", nil
case len(args) > 0 && args[0] == "load":
return "Loaded image: mesh-control:test\n", nil
return "Loaded image: mesh-controller:test\n", nil
}
return "", fmt.Errorf("unexpected command: %v", args)
}
@@ -128,7 +128,7 @@ func TestTheIdComesFromTheRuntimeAndNotFromTheArchive(t *testing.T) {
t.Error("a real run reported its id as a prediction")
}
// The runtime was asked BY THE TAG, which is the only name that survives the transfer.
if !runtime.ran("docker image inspect --format {{.Id}} mesh-control:test") {
if !runtime.ran("docker image inspect --format {{.Id}} mesh-controller:test") {
t.Errorf("the runtime was never asked what the tag resolves to: %v", runtime.commands)
}
// And the difference is said out loud, or somebody comparing this against `docker images` on
@@ -183,7 +183,7 @@ func TestALoadThatLeftNothingBehindIsAFailure(t *testing.T) {
if len(args) > 1 && args[0] == "image" && args[1] == "inspect" {
return "", errors.New("Error: No such image")
}
return "Loaded image: mesh-control:test\n", nil
return "Loaded image: mesh-controller:test\n", nil
}}
_, err := loadImage(context.Background(), runtime.run,
@@ -192,7 +192,7 @@ func TestALoadThatLeftNothingBehindIsAFailure(t *testing.T) {
t.Fatal("a load that left nothing on the machine was reported as success")
}
// Named by the tag, because that is what was asked about and what is missing.
if !strings.Contains(err.Error(), "mesh-control:test") {
if !strings.Contains(err.Error(), "mesh-controller:test") {
t.Errorf("the failure does not say what this machine holds nothing of: %v", err)
}
}
@@ -298,7 +298,7 @@ func TestAnInstallerCarryingNoImageSaysSoRatherThanRaisingHalfAMesh(t *testing.T
// check anything against, so it is checked where the refusal can say whose mistake it is.
func TestARuntimeAnsweringSomethingThatIsNotAnImageIdIsRefused(t *testing.T) {
for _, nonsense := range []string{
"mesh-control:test",
"mesh-controller:test",
"sha256:" + strings.Repeat("9", 63),
"<no value>",
} {
internal/bootstrap/module.go
+1 -1
View File
@@ -80,7 +80,7 @@ func installModule(ctx context.Context, o Options, control controlPlane, module
// Split out because one module needs something in between: the control plane's own store
// connections have to be accepted before its declaration is composed, or the mesh would seal
// thirty-two random bytes into the file it expects a connection string in and the container would
// come up unable to open anything (mesh-control's `secret accept`, and what it exists for).
// come up unable to open anything (mesh-controller's `secret accept`, and what it exists for).
//
// **`module add` is run every time and is not skipped when the module is already known.** It is an
// upsert on the manifest, and the manifest is exactly what changes between runs — step 9 registers
internal/bootstrap/phase2.go
+218
View File
@@ -0,0 +1,218 @@
package bootstrap
import (
"context"
"encoding/json"
"fmt"
"net"
"strings"
"time"
)
// Phase two — a mesh that runs becomes a mesh that works.
//
// Genesis ends with a control plane, a store, a broker, a registry and a builder — a mesh that
// RUNS. It holds no module graph, has no private network, and filters nothing. Those used to be
// things somebody typed afterwards, which is how they went missing for weeks without anything
// complaining (novox/hq 03-DESIGN/01-to-be/21-the-installation-in-full.md). The installer goes as
// far as it can instead, and asks where a human must choose.
//
// Everything here is `module add`, `build`, `assign` and `push` — the same verbs a person types,
// through the same commands, so what the installer does and what an operator does remain one act.
// buildWait bounds one module build. Generous, because the first build compiles a toolchain.
const buildWait = 20 * time.Minute
// BuildBase asks the mesh to build the shared base every module with code of its own stands on.
//
// **First, because until it exists nothing else with code can be built.** Not registered as a
// module here: it is never assigned — it runs nowhere — and the build itself records what was
// made, which is all anything downstream reads.
func BuildBase(ctx context.Context, o Options, control controlPlane, say func(string)) error {
if o.ToolsSource.Repository == "" {
return fmt.Errorf("phase two needs --tools-source: the shared base is built from its " +
"own repository, and an installer told nothing cannot know where that is")
}
say(" building " + o.ToolsSource.Repository + " at " + refOr(o.ToolsSource.Ref))
_, err := control.within(buildWait).tell(ctx,
"build", o.ToolsSource.Repository, "--ref", refOr(o.ToolsSource.Ref), "--wait", "1200s")
return err
}
// InstallFromCatalogue builds a catalogue module and installs it on this machine.
//
// The order matters and is the one the lab proved: register the manifest, build (so the artifact
// exists before anything resolves it), issue its broker account (a runtime without one starts,
// parses a password as a credential document, and loops), assign, push.
func InstallFromCatalogue(ctx context.Context, o Options, control controlPlane,
module string, say func(string)) error {
manifest, err := readManifest(o.Catalogue, module)
if err != nil {
return err
}
remote := "/" + module + "-module.json"
if err := control.carrying(ctx, module+"-module.json", manifest, remote); err != nil {
return err
}
if _, err := control.tell(ctx, "module", "add", remote); err != nil {
return err
}
say(" registered " + module)
if builds(manifest) {
if o.CatalogSource.Repository == "" {
return fmt.Errorf("%s has to be built and there is no --catalog-source to build it "+
"from: the catalogue CHECKOUT says what it is, the catalogue REPOSITORY is where "+
"a builder clones it", module)
}
say(" building " + module)
if _, err := control.within(buildWait).tell(ctx, "build", o.CatalogSource.Repository,
"--path", "modules/"+module, "--ref", refOr(o.CatalogSource.Ref),
"--wait", "1200s"); err != nil {
return err
}
}
if _, err := control.tell(ctx, "module", "issue", module, "--node", o.Node); err != nil {
// Not every module consumes the broker; one that does not is refused an account and that
// is fine. Said rather than silent, so a module that SHOULD have one and was refused is
// visible here rather than as a crash-loop later.
say(" no account " + module + " — it declares nothing to say on the broker")
} else {
say(" account issued " + module)
}
if _, err := control.tell(ctx, "assign", o.Node, module); err != nil {
return err
}
if _, err := pushNode(ctx, o, control, say); err != nil {
return err
}
say(" installed " + module)
return nil
}
// PlaceOnTheNetwork chooses a private-network provider, assigns it, and places this machine as
// the hub.
//
// **Assigning is not being on the network** — a lesson paid for: the module installed, the names
// file was written with no names in it, and everything reported success, because nobody had said
// where this machine IS. So placement is part of the step, not a separate act.
func PlaceOnTheNetwork(ctx context.Context, o Options, control controlPlane,
brokerAddress string, say func(string)) error {
network, err := decide(Choice{
Name: "private-network",
Question: "Which private network should this mesh run?",
Options: []string{"wireguard"},
}, o.Answers["private-network"], o.Prompt, say)
if err != nil {
return err
}
// Today the one provider is the control plane's own computed module. The choice exists so
// that the day there are two, this asks instead of assuming.
module := "networking"
_ = network
endpoint, err := decide(Choice{
Name: "endpoint",
Question: "Where do other machines reach this one for the private network? " +
"(host:port; the host other machines dial)",
Default: derivedEndpoint(brokerAddress),
}, o.Answers["endpoint"], o.Prompt, say)
if err != nil {
return err
}
if endpoint == "" {
return fmt.Errorf("the private network needs an endpoint other machines can dial, and " +
"nothing said one: pass --endpoint, or --broker-address so one can be derived")
}
if _, err := control.tell(ctx, "assign", o.Node, module); err != nil {
return err
}
if _, err := control.tell(ctx, "overlay", "place", o.Node,
"--hub", "--endpoint", endpoint, "--site", o.Site); err != nil {
return err
}
if _, err := pushNode(ctx, o, control, say); err != nil {
return err
}
say(" on the network " + o.Node + " is the hub, at " + endpoint)
return nil
}
// ChooseAndInstallFilter picks the packet filter — required, so the question is which, not
// whether — and installs it.
func ChooseAndInstallFilter(ctx context.Context, o Options, control controlPlane, say func(string)) error {
filter, err := decide(Choice{
Name: "packet-filter",
Question: "Which packet filter should this machine run?",
Options: []string{"nftables"},
}, o.Answers["packet-filter"], o.Prompt, say)
if err != nil {
return err
}
return InstallFromCatalogue(ctx, o, control, filter, say)
}
// InstallExtras installs what was asked for beyond the floor.
//
// One refusal per act: an extra that cannot be installed fails the run, because somebody asked
// for it by name and a mesh that reports success minus one thing is reporting the wrong thing.
func InstallExtras(ctx context.Context, o Options, control controlPlane, say func(string)) error {
asked, err := decide(Choice{
Name: "extras",
Question: "Anything beyond the floor? (comma-separated catalogue modules — " +
"gitea, step-ca, dnsmasq — or nothing)",
Default: "none",
}, strings.Join(o.Extras, ","), o.Prompt, say)
if err != nil {
return err
}
if asked == "" || asked == "none" {
say(" extras none")
return nil
}
for _, extra := range strings.Split(asked, ",") {
if extra = strings.TrimSpace(extra); extra == "" {
continue
}
if err := InstallFromCatalogue(ctx, o, control, extra, say); err != nil {
return fmt.Errorf("%s was asked for and could not be installed: %w", extra, err)
}
}
return nil
}
// builds says whether a manifest declares anything to build.
func builds(manifest []byte) bool {
var m struct {
Build *struct {
Artifacts []json.RawMessage `json:"artifacts"`
} `json:"build"`
}
if err := json.Unmarshal(manifest, &m); err != nil {
return false
}
return m.Build != nil && len(m.Build.Artifacts) > 0
}
// derivedEndpoint is the default place other machines dial for the private network: the same host
// they already dial for the broker, on WireGuard's ordinary port. One fact, not two.
func derivedEndpoint(brokerAddress string) string {
host, _, err := net.SplitHostPort(brokerAddress)
if err != nil || host == "" {
return ""
}
return net.JoinHostPort(host, "51820")
}
func refOr(ref string) string {
if ref == "" {
return "main"
}
return ref
}
internal/bootstrap/phase2_test.go
+25
View File
@@ -0,0 +1,25 @@
package bootstrap
import "testing"
// The endpoint other machines dial defaults to the host they already dial — the broker's — on
// WireGuard's port. One fact, not two that drift.
func TestTheEndpointDerivesFromTheBrokerAddress(t *testing.T) {
if got := derivedEndpoint("192.0.2.10:5671"); got != "192.0.2.10:51820" {
t.Fatalf("derived %q", got)
}
if got := derivedEndpoint(""); got != "" {
t.Fatalf("an endpoint was invented from nothing: %q", got)
}
}
// A manifest with artifacts builds; one without does not — which is what separates postgres (a
// bundle to compile) from nftables (a package and a service).
func TestOnlyAManifestWithArtifactsBuilds(t *testing.T) {
if !builds([]byte(`{"build":{"artifacts":[{"name":"x"}]}}`)) {
t.Fatal("a manifest with artifacts was not built")
}
if builds([]byte(`{"resources":[{"id":"p","type":"package","package":"nftables"}]}`)) {
t.Fatal("a manifest with nothing to build was built anyway")
}
}
internal/bootstrap/phase3.go
+205
View File
@@ -0,0 +1,205 @@
package bootstrap
import (
"context"
"encoding/json"
"fmt"
"strings"
"github.com/novox/mesh-host/internal/declaration"
)
// Phase three — nothing is special after installation (novox/hq issue 051).
//
// Genesis raises a store and a broker before any module system exists, because the control plane
// cannot ask provisioning for the store it keeps its own records in or the broker it is reached over
// (novox/hq ADR 0006). That leaves two servers behind: the foundation's, and a second one the
// `postgres`/`lavinmq` modules used to raise for other modules to use. This turns the foundation's
// own servers into those modules, so a mesh runs ONE postgres and ONE lavinmq — the control plane's
// contexts and the database of each module that asks for one, in the same server (WBS 3.1/3.2).
//
// **Adopted in place, not replaced.** The control plane is stateless and is swapped for a fresh
// container (control.go); the store and broker hold the mesh's memory and its bus, so they are kept.
// The module declares a container with the same name, image and spec the foundation raised, and the
// applier — which keys on the container name and compares a spec digest (mesh-host internal/apply) —
// finds it already running and leaves it be. The image is pinned to the one the foundation is
// running, read from the bundle this installer produced, so the two specs are the same digest and
// nothing is recreated. A recreate happens only on a real upgrade, which is where a stated window
// belongs (WBS 3.3).
// StoreID and BrokerID are what the foundation bundle calls the two servers it raises; the modules
// that adopt them are found by these ids in the bundle this installer produced, the same way the
// control plane's own container is (ControlPlaneID).
const (
StoreID = "store"
BrokerID = "broker"
)
// InstallStore makes the foundation's store the `postgres` module, adopted in place.
//
// The order is InstallFromCatalogue's, with two additions the store needs and an ordinary provider
// does not: the server image is pinned to the one the foundation is already running (so the module's
// container is the same spec and is adopted, not a second one raised), and the superuser password —
// the foundation's, made at genesis — is carried in through `secret accept`, because the mesh cannot
// invent a credential that already created the databases (the same reasoning as the control plane's
// store connections, control.go deliverStores).
func InstallStore(ctx context.Context, o Options, control controlPlane,
foundation *declaration.Declaration, say func(string)) error {
const module = "postgres"
manifest, err := readManifest(o.Catalogue, module)
if err != nil {
return err
}
store, err := storeIn(foundation)
if err != nil {
return err
}
// The module adopts the running store rather than raising a second one, so its server container
// has to BE the foundation's — same name, same image. The applier keys on the name and compares
// a spec digest (internal/apply), so a mismatch here would not adopt the mesh's memory but
// replace it. Checked before anything is registered, so a drift between the two pinned upstream
// images (the catalogue's and the foundation bundle's) fails fast and by name, rather than
// surfacing as the mesh's store being torn down and recreated.
//
// Not rewritten to the foundation's: the server image travels through the builder, which reads
// the manifest from the repository and leaves a concrete image alone but would carry a rewrite
// nowhere. The two are kept equal at the source — one pinned postgres, named in both places.
if err := serverMatchesFoundation(manifest, store, module); err != nil {
return err
}
say(" adopting " + store.Name + " — the store the foundation raised, unchanged")
remote := "/" + module + "-module.json"
if err := control.carrying(ctx, module+"-module.json", manifest, remote); err != nil {
return err
}
if _, err := control.tell(ctx, "module", "add", remote); err != nil {
return err
}
say(" registered " + module)
if o.CatalogSource.Repository == "" {
return fmt.Errorf("%s has to be built and there is no --catalog-source to build it from: "+
"the catalogue CHECKOUT says what it is, the catalogue REPOSITORY is where a builder "+
"clones it", module)
}
say(" building " + module + " (the provisioner; the server is adopted, not built)")
if _, err := control.within(buildWait).tell(ctx, "build", o.CatalogSource.Repository,
"--path", "modules/"+module, "--ref", refOr(o.CatalogSource.Ref), "--wait", "1200s"); err != nil {
return err
}
if _, err := control.tell(ctx, "module", "issue", module, "--node", o.Node); err != nil {
say(" no account " + module + " — it declares nothing to say on the broker")
} else {
say(" account issued " + module)
}
if _, err := control.tell(ctx, "assign", o.Node, module); err != nil {
return err
}
// The superuser is the foundation's, made at genesis — carried in before the push, or the push
// would seal random bytes where a working password has to be and the provisioner would not open
// the store it is meant to manage.
if err := deliverSuperuser(ctx, o, control, module, store, say); err != nil {
return err
}
if _, err := pushNode(ctx, o, control, say); err != nil {
return err
}
say(" adopted mesh-store — the foundation's store is now the " + module + " module")
return nil
}
// storeIn finds the store container in the bundle this installer produced.
func storeIn(d *declaration.Declaration) (*declaration.Container, error) {
return foundationContainer(d, StoreID, "store")
}
// brokerIn finds the broker container in the bundle this installer produced.
func brokerIn(d *declaration.Declaration) (*declaration.Container, error) {
return foundationContainer(d, BrokerID, "broker")
}
func foundationContainer(d *declaration.Declaration, id, what string) (*declaration.Container, error) {
for _, r := range d.Resources {
if r.Identity() != id {
continue
}
container, ok := r.(*declaration.Container)
if !ok {
return nil, fmt.Errorf(
"this bundle's %q is a %s, not a container, so the %s module has nothing to adopt",
id, r.Kind(), what)
}
return container, nil
}
return nil, fmt.Errorf(
"this bundle names no %q, so there is no %s for a module to adopt. It declares: %s",
id, what, strings.Join(identities(d), ", "))
}
// serverMatchesFoundation checks that the module's adopting container is the one the foundation
// raised — same name, same image — so the applier reconciles it in place rather than replacing it.
func serverMatchesFoundation(manifest []byte, store *declaration.Container, module string) error {
var m struct {
Resources []struct {
Type string `json:"type"`
Name string `json:"name"`
Image string `json:"image"`
} `json:"resources"`
}
if err := json.Unmarshal(manifest, &m); err != nil {
return fmt.Errorf("the %s module's manifest is not readable: %w", module, err)
}
for _, r := range m.Resources {
if r.Type != "container" || r.Name != store.Name {
continue
}
if r.Image != store.Image {
return fmt.Errorf(
"the %s module's %q container is pinned to %q, and the foundation is running %q.\n"+
"The module adopts the foundation's store in place, so the two must name the same "+
"image — a different one would tear down the mesh's store and raise a new one on "+
"its data. Pin both to the same postgres image",
module, store.Name, r.Image, store.Image)
}
return nil
}
return fmt.Errorf(
"the %s module declares no container named %q, so it has nothing to adopt the foundation's "+
"store with. Its server container has to carry the name the foundation raised",
module, store.Name)
}
// deliverSuperuser carries the store's superuser password into the module.
//
// It is the foundation's, set on the bundle's store container at genesis; the mesh cannot invent a
// credential that already made the databases, so it goes in through `secret accept`, exactly as the
// control plane's store connections do (control.go deliverStores).
func deliverSuperuser(ctx context.Context, o Options, control controlPlane, module string,
store *declaration.Container, say func(string)) error {
const secret = "superuser"
value := strings.TrimSpace(store.Env["POSTGRES_PASSWORD"])
if value == "" {
return fmt.Errorf(
"the foundation's store names no POSTGRES_PASSWORD, so the %s module has no superuser "+
"to open it with — and the mesh cannot invent the one that already made the databases",
module)
}
at := "/accepting-" + secret
if err := control.carrying(ctx, "mesh-accepting-"+secret, []byte(value), at); err != nil {
return err
}
if _, err := control.tell(ctx, "secret", "accept", o.Node, module, secret, "--from", at); err != nil {
return err
}
say(" accepted " + secret + " — the store's superuser, as the foundation made it")
return nil
}
internal/bootstrap/phase_packages.go
+298
View File
@@ -0,0 +1,298 @@
package bootstrap
import (
"context"
"fmt"
"net/http"
"os"
"strings"
"time"
)
// Raising the package registry, before the base is built.
//
// The base (mesh-tools) resolves the SDK by version from the mesh's package registry rather than
// cloning it from a git URL (novox/hq ADR 0076, issue 053). So the registry has to answer, and the
// SDK has to be in it, before the base build runs. That is a pivot like the control plane's: gitea's
// SERVER is raised directly here, on the foundation's own postgres, and adopted as an ordinary module
// only after the base exists (which is what lets its provisioner image — built on the base — run).
//
// Nothing here is the steady state. It is the smallest set of acts that puts a working npm registry
// in front of the base build: a database, a server, an admin, one org, the builder's own account,
// and the SDK published under it. The gitea MODULE, installed after the base, takes all of this over.
const (
// foundationStore is the foundation's postgres container — the mesh's own memory, raised from the
// bundle. gitea's bootstrap database lives here too, so a mesh runs one postgres (issue 051).
foundationStore = "mesh-store"
// giteaBootstrap is the gitea server raised directly at genesis, before gitea is a module.
giteaBootstrap = "mesh-gitea-server"
// giteaImage is the same upstream image the gitea module runs, pinned identically so the module
// adopts the running server rather than replacing it.
giteaImage = "gitea/gitea@sha256:dfc61e347c8b582df918f4556401bf2cecdfbdb56c5282ae9488dd76fca3e41c"
// packagesOrg is the npm owner: every module consumes `@novox/*` from this gitea org.
packagesOrg = "novox"
// packagesTeam is the org team whose members may read and write the org's packages.
packagesTeam = "packages"
// giteaAdminUser is the admin the bootstrap creates and the provisioner later authenticates as.
giteaAdminUser = "mesh-admin"
// builderGiteaUser is the gitea account the builder publishes and pulls with at genesis. It is
// the `as` the builder's static package binding names.
builderGiteaUser = "mesh-builder"
// giteaDBRole/giteaDBName is gitea's own database in the foundation store.
giteaDBRole = "mesh_gitea"
giteaDBName = "mesh_gitea"
// giteaPort is where the raised server answers on the machine.
giteaPort = 3000
)
// RaisePackageRegistry puts a working npm registry in front of the base build. It is idempotent:
// every step tolerates having been done, because genesis is safe to run again.
func RaisePackageRegistry(ctx context.Context, o Options, d Deps, control controlPlane,
say func(string)) error {
run := d.Run
// The passwords this pivot mints, kept once so a re-run is the same run: gitea already holds
// them, so regenerating would lock the mesh out of the forge it just raised.
dbPassword, adminPassword, builderPassword, err := packagePasswords()
if err != nil {
return err
}
say(" seeding gitea's database in the foundation store")
if err := seedGiteaDatabase(ctx, run, o.Timeout, dbPassword, say); err != nil {
return err
}
say(" raising the gitea server on that database")
if err := raiseGiteaServer(ctx, run, o.Timeout, dbPassword, say); err != nil {
return err
}
say(" waiting for gitea to answer")
base := fmt.Sprintf("http://127.0.0.1:%d", giteaPort)
if err := waitForGitea(ctx, d, o, base, say); err != nil {
return err
}
say(" creating the gitea admin")
if err := createGiteaAdmin(ctx, run, o.Timeout, adminPassword, say); err != nil {
return err
}
admin := &giteaAdmin{base: base, user: giteaAdminUser, password: adminPassword,
client: &http.Client{Timeout: o.Timeout}}
say(" ensuring the npm org, its package team, and the builder's account")
if err := admin.ensureOrg(ctx, packagesOrg); err != nil {
return err
}
teamID, err := admin.ensureTeam(ctx, packagesOrg, packagesTeam)
if err != nil {
return err
}
if err := admin.ensureUser(ctx, builderGiteaUser, builderPassword); err != nil {
return err
}
if err := admin.addToTeam(ctx, teamID, builderGiteaUser); err != nil {
return err
}
say(" delivering the builder its registry credential")
if err := deliverBuilderNpm(ctx, o, control, builderPassword, say); err != nil {
return err
}
return nil
}
// seedGiteaDatabase creates gitea's role and database inside the foundation postgres, the same way
// the foundation creates its own — psql run through the store container (the map's Route B). The role
// is created before the database because the database is owned by it. Both are tolerant of already
// existing, so a re-run changes nothing.
func seedGiteaDatabase(ctx context.Context, run Runner, timeout time.Duration, password string,
say func(string)) error {
asking, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
// Single statements through psql -c, not one script: CREATE DATABASE cannot run in a
// transaction and \gexec does not parse through -c. The password is base64url, so it carries no
// quote or backslash to escape inside a SQL literal.
psql := func(sql string) (string, error) {
return run(asking, "docker", "exec", foundationStore, "psql", "-U", "postgres", "-tAc", sql)
}
// The role: create it, and if it is already there (create fails) reset its password so a re-run
// converges on this run's credential.
create := fmt.Sprintf("CREATE ROLE %s LOGIN PASSWORD '%s'", giteaDBRole, password)
if _, err := psql(create); err != nil {
alter := fmt.Sprintf("ALTER ROLE %s LOGIN PASSWORD '%s'", giteaDBRole, password)
if _, err := psql(alter); err != nil {
return fmt.Errorf("could not create gitea's role in %s: %w", foundationStore, err)
}
}
// The database: created only if absent, because CREATE DATABASE has no IF NOT EXISTS and a
// second create is an error rather than a no-op.
present, err := psql(fmt.Sprintf("SELECT 1 FROM pg_database WHERE datname='%s'", giteaDBName))
if err != nil {
return fmt.Errorf("could not check for gitea's database in %s: %w", foundationStore, err)
}
if strings.TrimSpace(present) != "1" {
if _, err := psql(fmt.Sprintf("CREATE DATABASE %s OWNER %s", giteaDBName, giteaDBRole)); err != nil {
return fmt.Errorf("could not create gitea's database in %s: %w", foundationStore, err)
}
}
return nil
}
// raiseGiteaServer starts the gitea server container against the foundation store. It joins the
// store's network namespace so `127.0.0.1:5432` reaches postgres, and publishes its own port on the
// machine so the builder and this installer can reach it. Started if absent, left alone if present.
func raiseGiteaServer(ctx context.Context, run Runner, timeout time.Duration, dbPassword string,
say func(string)) error {
asking, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
// Already there: a re-run does not raise a second one. `docker start` is a no-op on a running
// container and revives a stopped one.
if out, _ := run(asking, "docker", "inspect", "--format", "{{.Id}}", giteaBootstrap); strings.TrimSpace(out) != "" {
_, _ = run(asking, "docker", "start", giteaBootstrap)
return nil
}
env := []string{
"-e", "GITEA__database__DB_TYPE=postgres",
// The store is reached on the shared network namespace's loopback.
"-e", "GITEA__database__HOST=127.0.0.1:5432",
"-e", "GITEA__database__NAME=" + giteaDBName,
"-e", "GITEA__database__USER=" + giteaDBRole,
"-e", "GITEA__database__PASSWD=" + dbPassword,
// Skip the install wizard: the mesh configures gitea, not a person at a browser.
"-e", "GITEA__security__INSTALL_LOCK=true",
// The forge answers on the machine's loopback, and its own links must say so: gitea's
// package metadata hands npm a tarball URL built from ROOT_URL, and a client only sends its
// stored credential to the host it was stored for. A default ROOT_URL of localhost is a
// different host than the binding's 127.0.0.1, so the credential would not be sent.
"-e", fmt.Sprintf("GITEA__server__ROOT_URL=http://127.0.0.1:%d/", giteaPort),
"-e", "USER_UID=1000", "-e", "USER_GID=1000",
}
args := append([]string{
"run", "-d", "--name", giteaBootstrap,
// Host network, like the control plane: it reaches the foundation store on the machine's
// loopback (where the store publishes 5432) and answers on the machine's own 3000, which is
// where mesh-bootstrap and the builder's build containers look for it.
"--network", "host",
"--restart", "unless-stopped",
}, env...)
args = append(args, giteaImage)
if _, err := run(asking, "docker", args...); err != nil {
return fmt.Errorf("could not raise the gitea server: %w", err)
}
return nil
}
// waitForGitea polls gitea's version endpoint until it answers or the wait runs out. A container
// that is up is not a forge that serves; `/api/v1/version` is the question whose answer means it is.
func waitForGitea(ctx context.Context, d Deps, o Options, base string, say func(string)) error {
deadline := time.Now().Add(o.Wait)
url := base + "/api/v1/version"
for {
status, _, err := d.Fetch(ctx, url)
if err == nil && status == http.StatusOK {
return nil
}
if time.Now().After(deadline) {
return fmt.Errorf("gitea did not answer at %s within %s", url, o.Wait)
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(2 * time.Second):
}
}
}
// createGiteaAdmin creates the mesh's gitea admin through the server's own CLI. Tolerant of the
// admin already existing, because a re-run must not fail on it — and it resets the password every
// run, so a rotated admin secret takes.
func createGiteaAdmin(ctx context.Context, run Runner, timeout time.Duration, password string,
say func(string)) error {
asking, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
// Create once, with the password kept across re-runs, and do not follow with change-password:
// change-password re-enables must-change-password, which then refuses every API call as
// "you must change your password". Tolerant of "already exists", because the kept password
// means the existing admin is already the one this run authenticates as.
create := fmt.Sprintf(
"gitea admin user create --admin --username %s --email %s@localhost --password %q --must-change-password=false",
giteaAdminUser, giteaAdminUser, password)
if _, err := run(asking, "docker", "exec", "-u", "git", giteaBootstrap,
"sh", "-c", create+" || true"); err != nil {
return fmt.Errorf("could not create the gitea admin: %w", err)
}
return nil
}
// packagePasswords loads the pivot's three passwords, minting and keeping them the first time. Kept
// on the machine because gitea, once raised, holds them: a second genesis that minted fresh ones
// would raise a forge it cannot then log into.
func packagePasswords() (db, admin, builder string, err error) {
const dir = "/var/lib/mesh/packages"
const file = dir + "/bootstrap.env"
if raw, e := os.ReadFile(file); e == nil {
vals := map[string]string{}
for _, line := range strings.Split(string(raw), "\n") {
if k, v, ok := strings.Cut(strings.TrimSpace(line), "="); ok {
vals[k] = v
}
}
if vals["DB"] != "" && vals["ADMIN"] != "" && vals["BUILDER"] != "" {
return vals["DB"], vals["ADMIN"], vals["BUILDER"], nil
}
}
db, admin, builder = newPassword(), newPassword(), newPassword()
if err = os.MkdirAll(dir, 0o700); err != nil {
return "", "", "", err
}
content := fmt.Sprintf("DB=%s\nADMIN=%s\nBUILDER=%s\n", db, admin, builder)
if err = os.WriteFile(file, []byte(content), 0o600); err != nil {
return "", "", "", err
}
return db, admin, builder, nil
}
// deliverBuilderNpm seals the builder's registry password to this node as its `npm-password`
// own-secret, the same way the control plane's store connections are delivered — carry the value in,
// `secret accept`, and the next push writes it sealed where the builder reads it.
func deliverBuilderNpm(ctx context.Context, o Options, control controlPlane, password string,
say func(string)) error {
at := "/accepting-npm-password"
if err := control.carrying(ctx, "mesh-accepting-npm-password", []byte(password), at); err != nil {
return err
}
if _, err := control.tell(ctx, "secret", "accept", o.Node, BuilderModule, "npm-password", "--from", at); err != nil {
return err
}
// Push so the sealed secret reaches the builder, which restarts on it and comes back credentialed.
if _, err := control.tell(ctx, "push", o.Node); err != nil {
return err
}
return nil
}
// PublishTheSDK dispatches a build of the SDK to the builder. The builder has its registry
// credential by now, so the build's `npm publish` authenticates; the artifact is a `package`, built
// on a public base, so this needs no toolchain — which is the whole point of doing it before the base.
func PublishTheSDK(ctx context.Context, o Options, control controlPlane, say func(string)) error {
if o.SDKSource.Repository == "" {
return fmt.Errorf("raising the registry needs --sdk-source: the SDK is built from its own " +
"repository, and an installer told nothing cannot know where that is")
}
say(" publishing " + o.SDKSource.Repository + " at " + refOr(o.SDKSource.Ref))
_, err := control.within(buildWait).tell(ctx,
"build", o.SDKSource.Repository, "--ref", refOr(o.SDKSource.Ref), "--wait", "1200s")
return err
}
internal/bootstrap/phase_packages_gitea.go
+186
View File
@@ -0,0 +1,186 @@
package bootstrap
import (
"bytes"
"context"
"crypto/rand"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
)
// A minimal gitea admin client, for the genesis pivot only. The gitea MODULE carries the real one
// (its TS provisioner); this exists because at genesis that module cannot be built yet — its image
// stands on the base, which is what this is helping to build. It does the few acts the pivot needs
// and nothing more: an org, a team, a user, a membership. Everything is idempotent, because genesis
// is safe to run again.
type giteaAdmin struct {
base string
user string
password string
client *http.Client
}
func (g *giteaAdmin) do(ctx context.Context, method, path string, body any) (int, []byte, error) {
var payload io.Reader
if body != nil {
raw, err := json.Marshal(body)
if err != nil {
return 0, nil, err
}
payload = bytes.NewReader(raw)
}
req, err := http.NewRequestWithContext(ctx, method, g.base+"/api/v1"+path, payload)
if err != nil {
return 0, nil, err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Basic "+base64.StdEncoding.EncodeToString([]byte(g.user+":"+g.password)))
res, err := g.client.Do(req)
if err != nil {
return 0, nil, err
}
defer res.Body.Close()
out, _ := io.ReadAll(res.Body)
return res.StatusCode, out, nil
}
// ok reports whether a status is one this pivot treats as success — the create succeeded, or the
// thing already exists (422/409), which for an idempotent step is the same outcome.
func ensured(status int) bool {
return status/100 == 2 || status == http.StatusUnprocessableEntity || status == http.StatusConflict
}
func (g *giteaAdmin) ensureOrg(ctx context.Context, name string) error {
status, body, err := g.do(ctx, http.MethodPost, "/orgs",
map[string]any{"username": name, "visibility": "private"})
if err != nil {
return err
}
if !ensured(status) {
return fmt.Errorf("could not create the gitea org %q: %d %s", name, status, body)
}
return nil
}
// ensureTeam creates the org's package team with write on packages and returns its id, finding the
// existing one when a create loses to a concurrent one.
func (g *giteaAdmin) ensureTeam(ctx context.Context, org, team string) (int, error) {
if id, err := g.findTeam(ctx, org, team); err != nil {
return 0, err
} else if id != 0 {
return id, nil
}
status, body, err := g.do(ctx, http.MethodPost, "/orgs/"+org+"/teams", map[string]any{
"name": team,
"permission": "read",
"units_map": map[string]string{"repo.packages": "write"},
"includes_all_repositories": true,
"can_create_org_repo": false,
})
if err != nil {
return 0, err
}
if status/100 == 2 {
var made struct {
ID int `json:"id"`
}
if err := json.Unmarshal(body, &made); err == nil && made.ID != 0 {
return made.ID, nil
}
}
// A lost race, or a body without an id: re-find.
if id, err := g.findTeam(ctx, org, team); err == nil && id != 0 {
return id, nil
}
return 0, fmt.Errorf("could not create the gitea team %q in %q: %d %s", team, org, status, body)
}
func (g *giteaAdmin) findTeam(ctx context.Context, org, team string) (int, error) {
status, body, err := g.do(ctx, http.MethodGet, "/orgs/"+org+"/teams?limit=50", nil)
if err != nil {
return 0, err
}
if status != http.StatusOK {
return 0, nil
}
var teams []struct {
ID int `json:"id"`
Name string `json:"name"`
}
if err := json.Unmarshal(body, &teams); err != nil {
return 0, err
}
for _, t := range teams {
if t.Name == team {
return t.ID, nil
}
}
return 0, nil
}
// ensureUser creates a gitea user with the mesh's minted password, or resets that user's password
// when it already exists, so a rotation takes.
func (g *giteaAdmin) ensureUser(ctx context.Context, name, password string) error {
// A dotted domain: gitea's API validates the address, and an @localhost with no dot is refused
// as malformed — which comes back as the same 422 an "already exists" does, so the email is
// chosen to not provoke it and existence is checked directly rather than inferred from a status.
status, body, err := g.do(ctx, http.MethodPost, "/admin/users", map[string]any{
"username": name,
"email": name + "@packages.mesh.local",
"password": password,
"must_change_password": false,
})
if err != nil {
return err
}
if status/100 == 2 {
return nil
}
exists, err := g.userExists(ctx, name)
if err != nil {
return err
}
if exists {
// Already there: reset the password so this run's credential is the one that works.
reset, rbody, err := g.do(ctx, http.MethodPatch, "/admin/users/"+name,
map[string]any{"login_name": name, "password": password, "must_change_password": false})
if err != nil {
return err
}
if reset/100 == 2 {
return nil
}
return fmt.Errorf("could not reset the gitea user %q: %d %s", name, reset, rbody)
}
return fmt.Errorf("could not create the gitea user %q: %d %s", name, status, body)
}
func (g *giteaAdmin) userExists(ctx context.Context, name string) (bool, error) {
status, _, err := g.do(ctx, http.MethodGet, "/users/"+name, nil)
if err != nil {
return false, err
}
return status == http.StatusOK, nil
}
func (g *giteaAdmin) addToTeam(ctx context.Context, teamID int, user string) error {
status, body, err := g.do(ctx, http.MethodPut, fmt.Sprintf("/teams/%d/members/%s", teamID, user), nil)
if err != nil {
return err
}
if !ensured(status) {
return fmt.Errorf("could not add %q to team %d: %d %s", user, teamID, status, body)
}
return nil
}
// newPassword is a mesh-minted secret: 32 bytes of randomness, URL-safe so it survives a connection
// string and an .npmrc without escaping.
func newPassword() string {
b := make([]byte, 32)
_, _ = rand.Read(b)
return base64.RawURLEncoding.EncodeToString(b)
}
internal/bootstrap/preflight.go
+4 -4
View File
@@ -26,7 +26,7 @@ func Preflight(ctx context.Context, o Options, d Deps, say func(string)) ([]byte
// 1. Does this installer carry what it claims to?
//
// Asked before the machine is touched, for the same reason `mesh-host bundle` exists: a host
// that carries no substrate must say so when somebody asks, not on a first node
// that carries no foundation must say so when somebody asks, not on a first node
// (internal/bundle). An installer built without an image would otherwise get a machine as far
// as a running store and a running broker and stop.
if image.IsEmpty() {
@@ -67,13 +67,13 @@ func Preflight(ctx context.Context, o Options, d Deps, say func(string)) ([]byte
}
say(fmt.Sprintf(" builder %s carried (the archive calls it %s)", tag, carriedID))
// 2. Is the template there, and is it a substrate?
// 2. Is the template there, and is it a foundation?
template, err := os.ReadFile(o.Template)
if err != nil {
return nil, fmt.Errorf(
"the bundle template could not be read: %w\n"+
"It is what this machine will be asked to be, so there is nothing to do without "+
"it. Point --bundle at one; mesh-host's examples/substrate-first-node.lock is "+
"it. Point --bundle at one; mesh-host's examples/foundation-first-node.lock is "+
"the shape", err)
}
// Parsed here as well as at the rewrite, because a template that is not a declaration should
@@ -120,7 +120,7 @@ func Preflight(ctx context.Context, o Options, d Deps, say func(string)) ([]byte
// with the id of the image this installer carries. Whatever the slot held is therefore never
// pulled, never fetched, and never reached; requiring it to be reachable refuses a correct
// install because of a string that is about to be thrown away. Found on the first real run: the
// lab's template still carried `192.0.2.250:5000/mesh-control@…`, the address of a registry that
// lab's template still carried `192.0.2.250:5000/mesh-controller@…`, the address of a registry that
// no longer exists, and preflight timed out dialling it.
for _, host := range registriesIn(parsed) {
dialing, cancel := context.WithTimeout(ctx, o.Timeout)
internal/bootstrap/preflight_test.go
+3 -3
View File
@@ -82,7 +82,7 @@ func TestOnlyTheRegistriesTheBundleNamesAreAskedAbout(t *testing.T) {
strings.Repeat("7", 64) + `"},
{"id":"broker","type":"container","name":"mesh-broker","image":"192.0.2.250:5000/lavinmq@sha256:` +
strings.Repeat("8", 64) + `"},
{"id":"control-plane","type":"container","name":"mesh-control","image":"` + held + `"}
{"id":"control-plane","type":"container","name":"mesh-controller","image":"` + held + `"}
]}`))
if err != nil {
t.Fatal(err)
@@ -111,7 +111,7 @@ func TestTheControlPlanesOwnRegistryIsNeverAskedAbout(t *testing.T) {
parsed, err := declaration.ParseFileTrusted([]byte(`{"declaration":1,"resources":[
{"id":"store","type":"container","name":"mesh-store","image":"postgres@sha256:` +
strings.Repeat("7", 64) + `"},
{"id":"control-plane","type":"container","name":"mesh-control","image":"192.0.2.250:5000/mesh-control@sha256:` +
{"id":"control-plane","type":"container","name":"mesh-controller","image":"192.0.2.250:5000/mesh-controller@sha256:` +
strings.Repeat("8", 64) + `"}
]}`))
if err != nil {
@@ -137,7 +137,7 @@ func TestWhereAnImageWouldBeFetchedFrom(t *testing.T) {
{"postgres@sha256:" + strings.Repeat("a", 64), DefaultRegistry, true},
{"cloudamqp/lavinmq@sha256:" + strings.Repeat("a", 64), DefaultRegistry, true},
{"192.0.2.250:5000/postgres@sha256:" + strings.Repeat("a", 64), "192.0.2.250:5000", true},
{"localhost/mesh-control@sha256:" + strings.Repeat("a", 64), "localhost:443", true},
{"localhost/mesh-controller@sha256:" + strings.Repeat("a", 64), "localhost:443", true},
{"registry.example.com/a/b@sha256:" + strings.Repeat("a", 64), "registry.example.com:443", true},
// Held by this machine. Nothing serves it, and nothing can.
{"sha256:" + strings.Repeat("a", 64), "", false},
internal/bootstrap/publish.go
+5 -5
View File
@@ -9,19 +9,19 @@ import (
)
// ControlPlaneRepository is what the control plane's image is called in the mesh's own registry.
const ControlPlaneRepository = "mesh-control"
const ControlPlaneRepository = "mesh-controller"
// genesisTag is the tag the first push uses.
//
// A tag is not a pin and is never what anything is deployed from — the digest the registry assigns
// is (novox/hq ADR 0006). This exists so a person reading `/v2/mesh-control/tags/list` can see
// is (novox/hq ADR 0006). This exists so a person reading `/v2/mesh-controller/tags/list` can see
// which image this mesh started from, and so the push has something to name. Everything downstream
// uses the digest that comes back.
const genesisTag = "genesis"
// Published is what step 8 did.
type Published struct {
// Reference is `<registry>/mesh-control@sha256:…` — the first manifest digest this image has
// Reference is `<registry>/mesh-controller@sha256:…` — the first manifest digest this image has
// ever had, and the thing that makes the control plane an ordinary module.
Reference string
// Tagged is where it was pushed, tag and all.
@@ -34,7 +34,7 @@ type Published struct {
//
// **This is the pivot's hinge.** Every image must be pinned by digest, and a digest a pin can mean
// is one a REGISTRY assigned when something was pushed to it. The control plane's image is built
// from source and pushed nowhere, so it has none — which is why the substrate names it by the
// from source and pushed nowhere, so it has none — which is why the foundation names it by the
// digest of its own configuration, and why that is legal exactly where nothing could have served
// one. The moment this push completes, that stops being true: the image has a manifest digest, so
// the control plane can be named the way every other module is named, so the mesh can build and
@@ -42,7 +42,7 @@ type Published struct {
// upgrade itself, which is the check novox/hq ADR 0067 states: after installing, the running
// control plane must be pinned by a digest the mesh's own registry assigned, not by an image id.
//
// **It mirrors mesh-control's `internal/builder`.PublishImage rather than importing it.** Tag,
// **It mirrors mesh-controller's `internal/builder`.PublishImage rather than importing it.** Tag,
// push, read back `RepoDigests`, refuse anything without `@sha256:` — the same four steps, because
// there is exactly one right way to learn what a registry will serve something as, and it is to
// ask the registry. It is not imported because that code is tier 2: the host and its installer
internal/bootstrap/publish_test.go
+9 -9
View File
@@ -41,7 +41,7 @@ func TestAnImageNoRegistryHasEverHeldIsPushed(t *testing.T) {
if !pushed {
return http.StatusNotFound, "", nil
}
return http.StatusOK, `{"name":"mesh-control","tags":["genesis"]}`, nil
return http.StatusOK, `{"name":"mesh-controller","tags":["genesis"]}`, nil
},
func(_ string, args []string) (string, error) {
switch args[0] {
@@ -51,7 +51,7 @@ func TestAnImageNoRegistryHasEverHeldIsPushed(t *testing.T) {
pushed = true
return "", nil
case "inspect":
return `["127.0.0.1:5000/mesh-control@sha256:` + strings.Repeat("a", 64) + `"]`, nil
return `["127.0.0.1:5000/mesh-controller@sha256:` + strings.Repeat("a", 64) + `"]`, nil
}
return "", fmt.Errorf("unexpected: %v", args)
})
@@ -63,10 +63,10 @@ func TestAnImageNoRegistryHasEverHeldIsPushed(t *testing.T) {
if out.Already {
t.Error("an image no registry held was reported as already published")
}
if !strings.HasPrefix(out.Reference, "127.0.0.1:5000/mesh-control@sha256:") {
if !strings.HasPrefix(out.Reference, "127.0.0.1:5000/mesh-controller@sha256:") {
t.Errorf("the control plane is pinned as %q", out.Reference)
}
if !runtime.ran("docker push 127.0.0.1:5000/mesh-control:genesis") {
if !runtime.ran("docker push 127.0.0.1:5000/mesh-controller:genesis") {
t.Errorf("nothing was pushed: %v", runtime.commands)
}
}
@@ -77,11 +77,11 @@ func TestAnImageNoRegistryHasEverHeldIsPushed(t *testing.T) {
func TestAnImageTheRegistryAlreadyServesIsNotPushedAgain(t *testing.T) {
o, d, runtime := publishing(t,
func(string) (int, string, error) {
return http.StatusOK, `{"name":"mesh-control","tags":["genesis"]}`, nil
return http.StatusOK, `{"name":"mesh-controller","tags":["genesis"]}`, nil
},
func(_ string, args []string) (string, error) {
if args[0] == "inspect" {
return `["127.0.0.1:5000/mesh-control@sha256:` + strings.Repeat("b", 64) + `"]`, nil
return `["127.0.0.1:5000/mesh-controller@sha256:` + strings.Repeat("b", 64) + `"]`, nil
}
return "", fmt.Errorf("unexpected: %v", args)
})
@@ -103,8 +103,8 @@ func TestAnImageTheRegistryAlreadyServesIsNotPushedAgain(t *testing.T) {
// listed first — which would pin this mesh's control plane to somebody else's registry, silently,
// which is the dependency the whole pivot exists to remove.
func TestTheDigestComesFromThisMeshsOwnRegistry(t *testing.T) {
elsewhere := "some.other.registry/mesh-control@sha256:" + strings.Repeat("c", 64)
ours := "127.0.0.1:5000/mesh-control@sha256:" + strings.Repeat("d", 64)
elsewhere := "some.other.registry/mesh-controller@sha256:" + strings.Repeat("c", 64)
ours := "127.0.0.1:5000/mesh-controller@sha256:" + strings.Repeat("d", 64)
o, d, _ := publishing(t,
func(string) (int, string, error) {
@@ -167,7 +167,7 @@ func TestATagIsNotAPin(t *testing.T) {
},
func(_ string, args []string) (string, error) {
if args[0] == "inspect" {
return `["127.0.0.1:5000/mesh-control:genesis"]`, nil
return `["127.0.0.1:5000/mesh-controller:genesis"]`, nil
}
return "", nil
})
internal/bootstrap/registry.go
+8 -3
View File
@@ -10,7 +10,12 @@ import (
)
// RegistryModule is the module that provides the mesh's artifact store.
const RegistryModule = "registry"
//
// **Named for the software, not the job.** The job is `artifact-store`, which is the provision this
// module offers; the software is Distribution, the OCI reference implementation. Calling the module
// `registry` named neither — and promised that any registry could sit there, which is the false
// genericity the naming rule forbids (novox/hq ADR 0040).
const RegistryModule = "distribution"
// registryResource is the id of the resource in that module's manifest that runs the registry.
// What the container is CALLED is read from the manifest rather than assumed, because the name is
@@ -39,7 +44,7 @@ type Registry struct {
//
// **No credentials, and that is deliberate.** The registry is reached over the mesh's own private
// network, which is already the encrypted and authenticated thing; a second layer inside it would
// be certificates to issue and rotate for no property the first does not have (mesh-control's
// be certificates to issue and rotate for no property the first does not have (mesh-controller's
// `internal/builder`, which pushes to it the same way). So there is nothing here to configure and
// nothing to seal — which is also why step 8 can push without the mesh having issued anything.
//
@@ -131,7 +136,7 @@ func waitForTheRegistry(ctx context.Context, d Deps, o Options, say func(string)
// waitForContainer waits for a container the mesh was asked to create to be running.
//
// Unlike the substrate's own verify, this one waits: the mesh applies through a node's host, over
// Unlike the foundation's own verify, this one waits: the mesh applies through a node's host, over
// the broker, asynchronously. A push that the control plane accepted has not yet happened on the
// machine, and refusing on the first look would refuse every correct install.
func waitForContainer(ctx context.Context, run Runner, probe, wait time.Duration, name string,
internal/bootstrap/registry_test.go
+10 -10
View File
@@ -18,7 +18,7 @@ import (
// catalogueWith writes a fake catalogue checkout holding one module's manifest.
//
// A fixture here rather than the real catalogue, unlike the substrate example the rewrite tests
// A fixture here rather than the real catalogue, unlike the foundation example the rewrite tests
// use: the catalogue is a different repository on a different branch, and a test that read it
// would pass or fail according to what somebody else had checked out.
func catalogueWith(t *testing.T, module, manifest string) string {
@@ -35,7 +35,7 @@ func catalogueWith(t *testing.T, module, manifest string) string {
}
const upstreamRegistryManifest = `{
"module": "registry",
"module": "distribution",
"version": "1",
"provides": [{"name": "artifact-store", "scope": "mesh"}],
"capabilities": ["container-runtime"],
@@ -100,7 +100,7 @@ func TestARegistryContainerThatIsUpIsNotARegistryThatServes(t *testing.T) {
_, err := InstallRegistry(context.Background(),
installing(t, catalogueWith(t, RegistryModule, upstreamRegistryManifest)),
deps, controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second},
deps, controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second},
func(string) {})
if err == nil {
t.Fatal("a registry whose container is up and which answers 500 was accepted")
@@ -124,7 +124,7 @@ func TestARegistryThatAnswersIsAccepted(t *testing.T) {
out, err := InstallRegistry(context.Background(),
installing(t, catalogueWith(t, RegistryModule, upstreamRegistryManifest)),
deps, controlPlane{container: "temp-mesh-control", run: runtime.run, timeout: time.Second},
deps, controlPlane{container: "temp-mesh-controller", run: runtime.run, timeout: time.Second},
func(string) {})
if err != nil {
t.Fatal(err)
@@ -137,8 +137,8 @@ func TestARegistryThatAnswersIsAccepted(t *testing.T) {
}
// Registered, assigned and pushed, through the same three commands a person types.
for _, wanted := range []string{
"module add /registry-module.json",
"assign anchor registry",
"module add /distribution-module.json",
"assign anchor distribution",
"push anchor",
} {
if !runtime.ran(wanted) {
@@ -159,7 +159,7 @@ func TestARegistryManifestThatWantsBuildingIsRefused(t *testing.T) {
runtime := &asked{answer: aMeshThatAgrees(nil)}
_, err := InstallRegistry(context.Background(),
installing(t, catalogueWith(t, RegistryModule, wants)),
Deps{Run: runtime.run}, controlPlane{container: "temp-mesh-control", run: runtime.run},
Deps{Run: runtime.run}, controlPlane{container: "temp-mesh-controller", run: runtime.run},
func(string) {})
if err == nil {
t.Fatal("a registry manifest naming an image the mesh would have to build was accepted")
@@ -178,7 +178,7 @@ func TestACatalogueThatIsNotThereIsSaidPlainly(t *testing.T) {
runtime := &asked{answer: aMeshThatAgrees(nil)}
_, err := InstallRegistry(context.Background(),
installing(t, filepath.Join(t.TempDir(), "nowhere")),
Deps{Run: runtime.run}, controlPlane{container: "temp-mesh-control", run: runtime.run},
Deps{Run: runtime.run}, controlPlane{container: "temp-mesh-controller", run: runtime.run},
func(string) {})
if err == nil {
t.Fatal("a catalogue that does not exist was accepted")
@@ -188,7 +188,7 @@ func TestACatalogueThatIsNotThereIsSaidPlainly(t *testing.T) {
}
}
// A refusal from the control plane is repeated verbatim. mesh-control refuses in paragraphs —
// A refusal from the control plane is repeated verbatim. mesh-controller refuses in paragraphs —
// "nothing provides route, wanted by registry" — and an installer that reported "exit status 1"
// would throw away the only thing a person can act on.
func TestWhatTheMeshRefusedIsRepeated(t *testing.T) {
@@ -202,7 +202,7 @@ func TestWhatTheMeshRefusedIsRepeated(t *testing.T) {
_, err := InstallRegistry(context.Background(),
installing(t, catalogueWith(t, RegistryModule, upstreamRegistryManifest)),
Deps{Run: runtime.run}, controlPlane{container: "temp-mesh-control", run: runtime.run,
Deps{Run: runtime.run}, controlPlane{container: "temp-mesh-controller", run: runtime.run,
timeout: time.Second}, func(string) {})
if err == nil {
t.Fatal("a push the mesh refused was reported as successful")
internal/bootstrap/retire.go
+4 -4
View File
@@ -32,8 +32,8 @@ type Retired struct {
// bundle is applied again, and the removal pass does what it does for every other resource that
// leaves a declaration.
//
// That is the whole of why the rename at step 3 mattered. Had the substrate and the module both
// called their container `mesh-control`, this apply would have removed the module's container —
// That is the whole of why the rename at step 3 mattered. Had the foundation and the module both
// called their container `mesh-controller`, this apply would have removed the module's container —
// the host would have been asked to take away something it believed it owned, and it would have
// been right. Two names, two owners, and the removal is unambiguous.
//
@@ -63,7 +63,7 @@ func RetireTheTemporaryControlPlane(ctx context.Context, o Options, sys system.S
// Textual, for the reason the rewrite at step 3 is textual: the produced bundle is meant to be
// READ, and a person coming to a machine after a pivot should be able to open the file the
// installer applied and see the substrate they recognise with the control plane gone from it.
// installer applied and see the foundation they recognise with the control plane gone from it.
// Re-serialising a parsed declaration would drop every comment in it.
bundle, err := removeResource(produced, ControlPlaneID)
if err != nil {
@@ -124,7 +124,7 @@ func RetireTheTemporaryControlPlane(ctx context.Context, o Options, sys system.S
// removeResource takes one resource out of a bundle's text, comments and all.
//
// It walks the `resources` array counting braces, skipping over strings and comments so that a
// `//` inside a connection string is not read as the start of one — the substrate's own bundle
// `//` inside a connection string is not read as the start of one — the foundation's own bundle
// contains `postgres://…` several times, and a scanner that did not know the difference would
// treat the rest of the line as a comment and lose a brace.
//
internal/bootstrap/retire_test.go
+5 -5
View File
@@ -46,7 +46,7 @@ func TestTheTemporaryControlPlaneLeavesTheBundleAndNothingElseDoes(t *testing.T)
t.Error("the bundle still declares a control plane")
}
// The store and the broker are still exactly what they were. A retirement that took the
// substrate with it would leave the machine with a module and nothing under it.
// foundation with it would leave the machine with a module and nothing under it.
for id, name := range containerNames(before) {
if id == ControlPlaneID {
continue
@@ -57,7 +57,7 @@ func TestTheTemporaryControlPlaneLeavesTheBundleAndNothingElseDoes(t *testing.T)
}
}
// **A `//` inside a string is not a comment.** The substrate's own bundle carries
// **A `//` inside a string is not a comment.** The foundation's own bundle carries
// `postgres://…` several times, and a scanner that read the rest of those lines as a comment
// would lose braces and cut the wrong thing out — silently, because what it produced would still
// look like a file.
@@ -144,7 +144,7 @@ func TestAContainerStillThereAfterRemovalIsNotGone(t *testing.T) {
stillThere := &asked{answer: func(_ string, _ []string) (string, error) {
return "true running\n", nil
}}
gone, err := isGone(context.Background(), stillThere.run, time.Second, 0, "temp-mesh-control")
gone, err := isGone(context.Background(), stillThere.run, time.Second, 0, "temp-mesh-controller")
if err != nil {
t.Fatal(err)
}
@@ -153,9 +153,9 @@ func TestAContainerStillThereAfterRemovalIsNotGone(t *testing.T) {
}
removed := &asked{answer: func(_ string, _ []string) (string, error) {
return "", errors.New("No such object: temp-mesh-control")
return "", errors.New("No such object: temp-mesh-controller")
}}
gone, err = isGone(context.Background(), removed.run, time.Second, 0, "temp-mesh-control")
gone, err = isGone(context.Background(), removed.run, time.Second, 0, "temp-mesh-controller")
if err != nil {
t.Fatal(err)
}
internal/bootstrap/rewrite.go
+19 -19
View File
@@ -19,27 +19,27 @@ import (
// broker and no mesh.
const ControlPlaneID = "control-plane"
// TempPrefix is what the substrate's control plane is renamed with.
// TempPrefix is what the foundation's control plane is renamed with.
//
// **This is the whole of how a carried resource becomes a declared one.** The substrate raises a
// **This is the whole of how a carried resource becomes a declared one.** The foundation raises a
// control plane and a module later declares one, and for a moment both exist — which looked like a
// handover problem needing a way for the host to stop owning something without destroying it. It is
// not one. The temporary control plane is called `temp-mesh-control` and the permanent one is
// called `mesh-control`: two containers, two owners, nothing shared and nothing to hand over. At
// not one. The temporary control plane is called `temp-mesh-controller` and the permanent one is
// called `mesh-controller`: two containers, two owners, nothing shared and nothing to hand over. At
// the end the temporary one is dropped from the bundle and the host removes it, which is exactly
// what should happen to something named "temp" (novox/hq ADR 0067).
//
// The name is also the audit. After the pivot, a machine running `mesh-control` and not
// `temp-mesh-control` has completed it; one running both stopped in the middle; one running only
// The name is also the audit. After the pivot, a machine running `mesh-controller` and not
// `temp-mesh-controller` has completed it; one running both stopped in the middle; one running only
// the temp has not started. That is readable from `docker ps` by somebody who knows nothing else.
const TempPrefix = "temp-"
// ControlPlaneModule is the module the permanent control plane is installed as, and the name its
// container takes — the name the substrate's own control plane gives up here so that it can.
// container takes — the name the foundation's own control plane gives up here so that it can.
//
// Declared beside the rename rather than beside the step that uses it, because this is where the
// two names are decided together and where the reason for both of them is written down.
const ControlPlaneModule = "mesh-control"
const ControlPlaneModule = "mesh-controller"
// brokerAddressVar is what a token tells an enrolling node to dial.
//
@@ -85,11 +85,11 @@ type Rewritten struct {
// Rewrite produces the bundle this machine will apply from the template it was given.
//
// **Two substitutions, and both are textual.** The control plane's image becomes the id of the image
// this machine now holds, and its container is renamed `temp-mesh-control`; nothing else changes.
// this machine now holds, and its container is renamed `temp-mesh-controller`; nothing else changes.
// The rename is what makes the pivot expressible at all — see TempPrefix. Textual rather than
// parse-and-re-serialise because
// the produced file has to be *read* — a person getting a machine working must be able to open it,
// see the substrate they recognise, and see exactly one thing different. Re-serialising a parsed
// see the foundation they recognise, and see exactly one thing different. Re-serialising a parsed
// declaration would drop every comment in the template, and those comments are where the reasons
// live.
//
@@ -182,7 +182,7 @@ func Rewrite(template []byte, imageID string) (Rewritten, error) {
if produced.Name != out.TempName {
return Rewritten{}, fmt.Errorf(
"the produced bundle still calls the control plane's container %q, not %q. The "+
"permanent one is a module and takes the plain name, so a substrate that kept it "+
"permanent one is a module and takes the plain name, so a foundation that kept it "+
"would put two owners on one container", produced.Name, out.TempName)
}
@@ -209,7 +209,7 @@ func Rewrite(template []byte, imageID string) (Rewritten, error) {
return Rewritten{}, fmt.Errorf(
"renaming the control plane's container also renamed %q, from %q to %q. Only the "+
"control plane moves out of the way; every other container keeps the name the "+
"substrate gave it", id, wasName[id], name)
"foundation gave it", id, wasName[id], name)
}
sortStrings(out.Kept)
return out, nil
@@ -217,15 +217,15 @@ func Rewrite(template []byte, imageID string) (Rewritten, error) {
// renameContainer changes one container's name in the bundle's text.
//
// **The quoted name, not the bare word.** `mesh-control` also appears inside the image reference
// the template carries (`…/mesh-control@sha256:…`) and could appear inside a command line; a bare
// **The quoted name, not the bare word.** `mesh-controller` also appears inside the image reference
// the template carries (`…/mesh-controller@sha256:…`) and could appear inside a command line; a bare
// substitution would catch those too. What is wanted is a JSON string that IS the name, so the
// quotes are part of what is matched — `"mesh-control"` matches the container's `name` and an
// quotes are part of what is matched — `"mesh-controller"` matches the container's `name` and an
// action's `in`, which are exactly the places the name means the container, and nothing else.
//
// It refuses when the text does not contain what the parse says is there, for the same reason the
// image substitution does: the two would then be reading different things, and a rename that
// replaced nothing and reported success would leave the module and the substrate fighting over one
// replaced nothing and reported success would leave the module and the foundation fighting over one
// container three steps later.
func renameContainer(bundle []byte, from, to string) ([]byte, error) {
if from == to {
@@ -235,7 +235,7 @@ func renameContainer(bundle []byte, from, to string) ([]byte, error) {
if bytes.Count(bundle, quoted) == 0 {
return nil, fmt.Errorf(
"the control plane's container is called %q according to the parsed template, and %s "+
"is not in the file. Nothing was renamed, and the substrate would raise a "+
"is not in the file. Nothing was renamed, and the foundation would raise a "+
"container the module also wants", from, quoted)
}
return bytes.ReplaceAll(bundle, quoted, []byte(`"`+to+`"`)), nil
@@ -257,7 +257,7 @@ func controlPlaneIn(d *declaration.Declaration) (*declaration.Container, error)
}
return nil, fmt.Errorf(
"this bundle names no %q, so there is no control plane to give this machine's image to. "+
"A substrate without one raises a store and a broker and no mesh. It declares: %s",
"A foundation without one raises a store and a broker and no mesh. It declares: %s",
ControlPlaneID, strings.Join(identities(d), ", "))
}
@@ -316,7 +316,7 @@ func sortStrings(values []string) {
// writeBundleFile puts the produced bundle where a person can read it, creating the directory it
// lives in.
//
// 0644, and that is deliberate: this file names an image and describes a substrate, and it holds
// 0644, and that is deliberate: this file names an image and describes a foundation, and it holds
// the bootstrap credentials the template happens to carry — which are the same ones anybody can
// read in the template itself. It is meant to be read. What must not be world-readable is the
// node's identity, and that lives elsewhere and is written elsewhere (`internal/identity`).
internal/bootstrap/rewrite_test.go
+27 -27
View File
@@ -15,16 +15,16 @@ const (
otherHeld = "sha256:2222222222222222222222222222222222222222222222222222222222222222"
)
// theRealBundle is this repository's own substrate example, used rather than a fixture.
// theRealBundle is this repository's own foundation example, used rather than a fixture.
//
// A fixture would agree with whatever this code does. The example is what an installer is actually
// pointed at, it names the control plane twice, and it is the file that changes when the substrate
// pointed at, it names the control plane twice, and it is the file that changes when the foundation
// changes — so a rewrite that stops working on it is a rewrite that has stopped working.
func theRealBundle(t *testing.T) []byte {
t.Helper()
raw, err := os.ReadFile("../../examples/substrate-first-node.lock")
raw, err := os.ReadFile("../../examples/foundation-first-node.lock")
if err != nil {
t.Fatalf("reading the substrate example: %v", err)
t.Fatalf("reading the foundation example: %v", err)
}
return raw
}
@@ -48,7 +48,7 @@ func TestTheControlPlaneIsNamedByTheImageThisMachineHolds(t *testing.T) {
// **Every place the bundle names that image, not only the container.**
//
// The substrate names the control plane's image twice: the container that runs `serve`, and the
// The foundation names the control plane's image twice: the container that runs `serve`, and the
// action that runs `migrate` to create the contexts' schemas. Rewriting only the container leaves
// the migration pointing at an image no registry serves, and the apply dies in the middle — after
// the store is up and before the broker. This is the test that would have caught that.
@@ -60,7 +60,7 @@ func TestEveryPlaceTheBundleNamesTheControlPlaneIsRewritten(t *testing.T) {
t.Fatal(err)
}
if out.Places < 2 {
t.Fatalf("the control plane's image was found in %d place(s); the substrate names it in "+
t.Fatalf("the control plane's image was found in %d place(s); the foundation names it in "+
"the container AND in the migration action", out.Places)
}
if remaining := strings.Count(string(out.Bundle), out.Was); remaining != 0 {
@@ -86,7 +86,7 @@ func TestPostgresAndTheBrokerAreLeftExactlyAsTheyWere(t *testing.T) {
for _, id := range []string{"store", "broker"} {
image, named := produced[id]
if !named {
t.Fatalf("the substrate example no longer declares a %q container", id)
t.Fatalf("the foundation example no longer declares a %q container", id)
}
// Compared against the template's own text rather than against an expectation written
// here: what is being defended is "unchanged", and the template is the only thing that
@@ -126,7 +126,7 @@ func TestABundleThatNamesNoControlPlaneIsRefused(t *testing.T) {
func TestAControlPlaneThatIsNotAContainerIsRefused(t *testing.T) {
template := []byte(`{"declaration":1,"resources":[
{"id":"control-plane","type":"package","package":"mesh-control"}
{"id":"control-plane","type":"package","package":"mesh-controller"}
]}`)
if _, err := Rewrite(template, held); err == nil {
t.Fatal("a control plane declared as a package was accepted, and a package has no image")
@@ -175,7 +175,7 @@ func TestANewImageReplacesAnOlderHeldOne(t *testing.T) {
}
// The produced bundle is meant to be READ. Re-serialising a parsed declaration would drop every
// comment in the template, and the substrate example is mostly comments — each one recording why a
// comment in the template, and the foundation example is mostly comments — each one recording why a
// resource is the way it is, several of them paid for in the lab.
func TestTheProducedBundleKeepsTheTemplatesComments(t *testing.T) {
template := theRealBundle(t)
@@ -194,11 +194,11 @@ func TestTheProducedBundleKeepsTheTemplatesComments(t *testing.T) {
func TestSomethingThatIsNotAnImageIdIsRefused(t *testing.T) {
for _, bad := range []string{
"",
"mesh-control:latest",
"mesh-controller:latest",
"sha256:abc",
"sha256:" + strings.Repeat("1", 63),
"sha256:" + strings.Repeat("g", 64),
"mesh-control@sha256:" + strings.Repeat("1", 64),
"mesh-controller@sha256:" + strings.Repeat("1", 64),
} {
if _, err := Rewrite(theRealBundle(t), bad); err == nil {
t.Errorf("image id %q was accepted", bad)
@@ -216,7 +216,7 @@ func TestTheAddressNodesWillDialIsReportedAndNotRewritten(t *testing.T) {
t.Fatal(err)
}
if out.BrokerAddress == "" {
t.Fatal("the substrate example no longer says what address enrolling nodes will dial")
t.Fatal("the foundation example no longer says what address enrolling nodes will dial")
}
if !strings.Contains(string(out.Bundle), out.BrokerAddress) {
t.Errorf("the produced bundle no longer carries %q — it was rewritten, and nothing here "+
@@ -228,21 +228,21 @@ func TestTheAddressNodesWillDialIsReportedAndNotRewritten(t *testing.T) {
// The rename, which is what makes genesis a pivot rather than a handover (novox/hq ADR 0067).
// ---------------------------------------------------------------------------------------------
// **This is the test that dissolves the blocker.** The substrate raises a control plane and a
// module later declares one; if both are called `mesh-control` then for one moment two owners hold
// **This is the test that dissolves the blocker.** The foundation raises a control plane and a
// module later declares one; if both are called `mesh-controller` then for one moment two owners hold
// one container, and the host — which tracks what it owns — has no way to stop owning something
// without destroying it. Nothing here invents such a mechanism. The substrate's container is
// called `temp-mesh-control` instead, and there are simply two containers.
func TestTheSubstratesControlPlaneMovesOutOfTheModulesWay(t *testing.T) {
// without destroying it. Nothing here invents such a mechanism. The foundation's container is
// called `temp-mesh-controller` instead, and there are simply two containers.
func TestTheFoundationsControlPlaneMovesOutOfTheModulesWay(t *testing.T) {
out, err := Rewrite(theRealBundle(t), held)
if err != nil {
t.Fatal(err)
}
if !out.Renamed {
t.Error("the rewrite reported nothing renamed, and the template named it mesh-control")
t.Error("the rewrite reported nothing renamed, and the template named it mesh-controller")
}
if out.TempName != "temp-mesh-control" {
t.Errorf("the substrate's control plane is called %q", out.TempName)
if out.TempName != "temp-mesh-controller" {
t.Errorf("the foundation's control plane is called %q", out.TempName)
}
control, err := controlPlaneIn(out.Declaration)
if err != nil {
@@ -260,22 +260,22 @@ func TestTheSubstratesControlPlaneMovesOutOfTheModulesWay(t *testing.T) {
}
}
// The image reference contains the string `mesh-control` too, and it is not a container name. A
// substitution that caught it would produce `…/temp-mesh-control@sha256:…`, which no registry
// The image reference contains the string `mesh-controller` too, and it is not a container name. A
// substitution that caught it would produce `…/temp-mesh-controller@sha256:…`, which no registry
// serves — and it would be found inside a pull rather than here.
func TestTheImageReferenceIsNotMistakenForTheContainerName(t *testing.T) {
out, err := Rewrite(theRealBundle(t), held)
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(out.Bundle), TempPrefix+"mesh-control@") ||
strings.Contains(string(out.Bundle), "/"+TempPrefix+"mesh-control") {
if strings.Contains(string(out.Bundle), TempPrefix+"mesh-controller@") ||
strings.Contains(string(out.Bundle), "/"+TempPrefix+"mesh-controller") {
t.Error("the rename reached inside an image reference")
}
}
// Everything else keeps the name the substrate gave it. The store and the broker are containers
// too, and a rename that moved them would leave a machine whose substrate the host cannot find.
// Everything else keeps the name the foundation gave it. The store and the broker are containers
// too, and a rename that moved them would leave a machine whose foundation the host cannot find.
func TestRenamingTheControlPlaneLeavesEveryOtherContainerAlone(t *testing.T) {
before, err := declaration.ParseFileTrusted(theRealBundle(t))
if err != nil {
@@ -309,7 +309,7 @@ func TestRewritingABundleThisAlreadyProducedRenamesNothing(t *testing.T) {
t.Fatal(err)
}
if second.Renamed {
t.Error("a bundle already naming temp-mesh-control was renamed again")
t.Error("a bundle already naming temp-mesh-controller was renamed again")
}
if second.TempName != first.TempName {
t.Errorf("the second pass calls it %q and the first called it %q",
internal/bootstrap/talk.go
+6 -6
View File
@@ -13,13 +13,13 @@ import (
//
// **Through `docker exec`, not over a network.** The control plane listens on nothing — `serve` is
// a broker consumer, and every administrative verb is a subcommand of the same binary that opens
// the stores directly (mesh-control's own usage). So the way to tell a mesh anything, from the
// the stores directly (mesh-controller's own usage). So the way to tell a mesh anything, from the
// machine the mesh is on, is to run its binary inside its own container. That is also what the lab
// does, and having the installer and the lab drive the mesh identically is the point: the lab is
// meant to exercise the installer, not a second procedure that resembles it.
//
// It carries which container, because the whole pivot turns on there being two of them: the
// substrate's `temp-mesh-control` for steps 6 to 9, and the module's `mesh-control` afterwards.
// foundation's `temp-mesh-controller` for steps 6 to 9, and the module's `mesh-controller` afterwards.
type controlPlane struct {
container string
run Runner
@@ -35,9 +35,9 @@ func (c controlPlane) within(timeout time.Duration) controlPlane {
return c
}
// tell runs a mesh-control subcommand and gives back what it said.
// tell runs a mesh-controller subcommand and gives back what it said.
//
// The failure carries the command AND the output. A mesh-control refusal is a paragraph explaining
// The failure carries the command AND the output. A mesh-controller refusal is a paragraph explaining
// what is wrong — "nothing provides route, wanted by registry" — and an installer that reported
// only "exit status 1" would throw away the one thing a person needs.
func (c controlPlane) tell(ctx context.Context, args ...string) (string, error) {
@@ -83,7 +83,7 @@ func (c controlPlane) carry(ctx context.Context, local, remote string) error {
// crash-looped on material it never received. There is no shell in the image to chown it with.
//
// What goes through here is a module manifest and a store connection string. The connection is the
// same value the produced bundle already holds in the clear — a substrate names its own bootstrap
// same value the produced bundle already holds in the clear — a foundation names its own bootstrap
// credentials, and at genesis there is nowhere else for them to be — so this widens nothing. The
// file on the machine is removed at once, and the copy inside the container goes when the
// container does, which for the temporary control plane is step 10.
@@ -107,7 +107,7 @@ func indent(s string) string {
//
// Line-and-word rather than a substring search, because these listings are columns and a
// substring match would find `registry` inside `registry-mirror` and report a module installed
// that is not. Every one of mesh-control's `list` verbs prints the name first on the line.
// that is not. Every one of mesh-controller's `list` verbs prints the name first on the line.
func mentions(listing, name string) bool {
for _, line := range strings.Split(listing, "\n") {
first, _, _ := strings.Cut(strings.TrimSpace(line), " ")
internal/bootstrap/verify.go
+5 -5
View File
@@ -13,14 +13,14 @@ import (
//
// A path rather than a shell command, because the image is `FROM scratch` and holds one static
// binary and nothing else — no shell to invoke, nothing to interpret a command line
// (mesh-control's Dockerfile, novox/hq ADR 0006). That is a property of the image this installer
// (mesh-controller's Dockerfile, novox/hq ADR 0006). That is a property of the image this installer
// carries, which is why the path can be written down here.
const controlPlaneBinary = "/mesh-control"
const controlPlaneBinary = "/mesh-controller"
// answerEvery is how often the control plane is asked again while it is starting.
var answerEvery = 2 * time.Second
// Verified is what the substrate was found to be.
// Verified is what the foundation was found to be.
type Verified struct {
// Running is every long-running container the bundle declares, confirmed up.
Running []string
@@ -29,7 +29,7 @@ type Verified struct {
Answered string
}
// Verify proves the substrate is up and the control plane replies.
// Verify proves the foundation is up and the control plane replies.
//
// **A container that is up is not a control plane that replies**, and this project has paid for
// that distinction more than once: a runtime reports a container running from the moment the
@@ -146,7 +146,7 @@ func containerRunning(ctx context.Context, run Runner, probe time.Duration, name
//
// A run-once step has exited by design and a scheduled step has deliberately never been started
// (novox/hq ADR 0052, ADR 0053), so asking either of them to be running would be asking the
// substrate to be something other than what it declared.
// foundation to be something other than what it declared.
func longRunning(d *declaration.Declaration) []string {
var names []string
for _, r := range d.Resources {
internal/bootstrap/verify_test.go
+15 -15
View File
@@ -11,7 +11,7 @@ import (
"github.com/novox/mesh-host/internal/declaration"
)
func substrate(t *testing.T) *declaration.Declaration {
func foundation(t *testing.T) *declaration.Declaration {
t.Helper()
out, err := Rewrite(theRealBundle(t), held)
if err != nil {
@@ -39,12 +39,12 @@ func TestAContainerThatIsUpIsNotAControlPlaneThatReplies(t *testing.T) {
return "", fmt.Errorf("unexpected command: %v", args)
}}
_, err := Verify(context.Background(), substrate(t), runtime.run,
_, err := Verify(context.Background(), foundation(t), runtime.run,
time.Second, 0, func(string) {})
if err == nil {
t.Fatal("every container was running, nothing answered, and the substrate was reported up")
t.Fatal("every container was running, nothing answered, and the foundation was reported up")
}
for _, wanted := range []string{"mesh-control", "Running is not replying", "docker logs"} {
for _, wanted := range []string{"mesh-controller", "Running is not replying", "docker logs"} {
if !strings.Contains(err.Error(), wanted) {
t.Errorf("the failure does not mention %q:\n%v", wanted, err)
}
@@ -65,14 +65,14 @@ func TestAControlPlaneThatSaysNothingHasNotAnswered(t *testing.T) {
return " \n", nil
}}
if _, err := Verify(context.Background(), substrate(t), runtime.run,
if _, err := Verify(context.Background(), foundation(t), runtime.run,
time.Second, 0, func(string) {}); err == nil {
t.Fatal("a control plane that exited zero without saying anything was accepted")
}
}
// The substrate answering is the whole point, and what it said is reported rather than asserted.
func TestASubstrateThatIsUpAndAnsweringIsAccepted(t *testing.T) {
// The foundation answering is the whole point, and what it said is reported rather than asserted.
func TestAFoundationThatIsUpAndAnsweringIsAccepted(t *testing.T) {
runtime := &asked{answer: func(_ string, args []string) (string, error) {
if args[0] == "inspect" {
return "true running\n", nil
@@ -80,16 +80,16 @@ func TestASubstrateThatIsUpAndAnsweringIsAccepted(t *testing.T) {
return "1 node, 0 waiting\n", nil
}}
verified, err := Verify(context.Background(), substrate(t), runtime.run,
verified, err := Verify(context.Background(), foundation(t), runtime.run,
time.Second, 0, func(string) {})
if err != nil {
t.Fatal(err)
}
// Three long-running containers: the store, the broker and the TEMPORARY control plane. The
// run-once and scheduled shapes are excluded on purpose — a step that has exited is not a
// fault. The name is `temp-mesh-control` because the permanent one is a module and takes the
// fault. The name is `temp-mesh-controller` because the permanent one is a module and takes the
// plain name (novox/hq ADR 0067), which is what makes the two of them coexist at all.
want := []string{"mesh-store", "mesh-broker", "temp-mesh-control"}
want := []string{"mesh-store", "mesh-broker", "temp-mesh-controller"}
if len(verified.Running) != len(want) {
t.Fatalf("confirmed %v running, want %v", verified.Running, want)
}
@@ -122,7 +122,7 @@ func TestAControlPlaneThatIsStillStartingIsWaitedFor(t *testing.T) {
return "1 node\n", nil
}}
if _, err := Verify(context.Background(), substrate(t), runtime.run,
if _, err := Verify(context.Background(), foundation(t), runtime.run,
time.Second, time.Second, func(string) {}); err != nil {
t.Fatalf("a control plane that answered on the third ask was refused: %v", err)
}
@@ -141,10 +141,10 @@ func TestAContainerThatExitedIsNamedWithItsState(t *testing.T) {
return "", fmt.Errorf("unexpected command: %v", args)
}}
_, err := Verify(context.Background(), substrate(t), runtime.run,
_, err := Verify(context.Background(), foundation(t), runtime.run,
time.Second, 0, func(string) {})
if err == nil {
t.Fatal("a container that had exited was reported as part of a running substrate")
t.Fatal("a container that had exited was reported as part of a running foundation")
}
if !strings.Contains(err.Error(), "mesh-broker") || !strings.Contains(err.Error(), "exited") {
t.Errorf("the failure does not say which container is in what state: %v", err)
@@ -160,11 +160,11 @@ func TestTheControlPlaneIsAskedByRunningItsOwnBinary(t *testing.T) {
}
return "1 node\n", nil
}}
if _, err := Verify(context.Background(), substrate(t), runtime.run,
if _, err := Verify(context.Background(), foundation(t), runtime.run,
time.Second, 0, func(string) {}); err != nil {
t.Fatal(err)
}
if !runtime.ran("docker exec " + TempPrefix + "mesh-control " + controlPlaneBinary + " status") {
if !runtime.ran("docker exec " + TempPrefix + "mesh-controller " + controlPlaneBinary + " status") {
t.Errorf("the control plane was never asked anything: %v", runtime.commands)
}
}
internal/bundle/bundle.go
+4 -4
View File
@@ -27,13 +27,13 @@ import (
// nothing and reporting success — a host that silently did nothing on a first node would look
// exactly like one that worked.
//
//go:embed substrate-arch.lock
//go:embed foundation-arch.lock
var archLock []byte
//go:embed substrate-alpine.lock
//go:embed foundation-alpine.lock
var alpineLock []byte
//go:embed substrate-android.lock
//go:embed foundation-android.lock
var androidLock []byte
var locks = map[string][]byte{
@@ -52,7 +52,7 @@ func Raw(system string) []byte { return locks[system] }
// IsEmpty reports whether anything was built in. A bundle of only comments and whitespace is
// empty for this purpose: a placeholder is a comment, and treating it as content would mean a
// host claims to carry a substrate it does not.
// host claims to carry a foundation it does not.
func IsEmpty(system string) bool {
for _, line := range strings.Split(string(locks[system]), "\n") {
line = strings.TrimSpace(line)
internal/bundle/bundle_test.go
+1 -1
View File
@@ -13,7 +13,7 @@ func TestADefaultBuildCarriesNothingAndSaysSo(t *testing.T) {
// success would look exactly like a host that raised a first node — and the difference
// would surface as a mesh that never came up, with nothing to point at.
if !IsEmpty("arch") {
t.Fatal("the default build claims to carry a substrate")
t.Fatal("the default build claims to carry a foundation")
}
_, err := Load("arch")
if !errors.Is(err, ErrEmpty) {
internal/bundle/substrate-alpine.lock → internal/bundle/foundation-alpine.lock
+1 -1
View File
@@ -1,4 +1,4 @@
// substrate-alpine.lock — the pinned tier-1 descriptor the ALPINE host carries.
// foundation-alpine.lock — the pinned tier-1 descriptor the ALPINE host carries.
//
// Per system, because its CONTENTS are: this one names apk packages and OpenRC services where
// the arch bundle names pacman packages and systemd units (novox/hq ADR 0005).
internal/bundle/substrate-android.lock → internal/bundle/foundation-android.lock
+3 -3
View File
@@ -1,10 +1,10 @@
// substrate-android.lock — deliberately not a bundle.
// foundation-android.lock — deliberately not a bundle.
//
// An android host cannot raise a mesh, and this file says so rather than being an empty
// placeholder waiting to be filled in.
//
// The substrate is a container runtime, a store and the control plane (novox/hq
// 07-the-substrate.md). An android host implements `file`, `directory` and `action` and refuses
// The foundation is a container runtime, a store and the control plane (novox/hq
// 07-the-foundation.md). An android host implements `file`, `directory` and `action` and refuses
// `package`, `container` and `service` (ADR 0005) — so every step of the bootstrap is a shape it
// does not have. No amount of filling this in changes that.
//
internal/bundle/substrate-arch.lock → internal/bundle/foundation-arch.lock
+1 -1
View File
@@ -1,4 +1,4 @@
// substrate-arch.lock — the pinned tier-1 descriptor the ARCH host carries.
// foundation-arch.lock — the pinned tier-1 descriptor the ARCH host carries.
//
// Per system, because its CONTENTS are: package names, unit names and service names all differ
// (novox/hq ADR 0005). The mechanism is shared; what it names is not.
internal/declaration/cron_test.go
+20 -20
View File
@@ -11,19 +11,19 @@ import (
func TestParseCronRefusesMalformed(t *testing.T) {
for _, expr := range []string{
"", // nothing
"* * * *", // four fields
"* * * * * *", // six fields
"60 * * * *", // minute out of range
"* 24 * * *", // hour out of range
"* * 0 * *", // day-of-month below 1
"* * 32 * *", // day-of-month above 31
"* * * 13 *", // month out of range
"* * * * 8", // day-of-week above 7
"a * * * *", // not a number
"*/0 * * * *", // zero step
"5-1 * * * *", // inverted range
"1,,2 * * * *", // empty element
"", // nothing
"* * * *", // four fields
"* * * * * *", // six fields
"60 * * * *", // minute out of range
"* 24 * * *", // hour out of range
"* * 0 * *", // day-of-month below 1
"* * 32 * *", // day-of-month above 31
"* * * 13 *", // month out of range
"* * * * 8", // day-of-week above 7
"a * * * *", // not a number
"*/0 * * * *", // zero step
"5-1 * * * *", // inverted range
"1,,2 * * * *", // empty element
} {
if _, err := ParseCron(expr); err == nil {
t.Errorf("a malformed cron %q was accepted", expr)
@@ -33,13 +33,13 @@ func TestParseCronRefusesMalformed(t *testing.T) {
func TestParseCronAcceptsTheOrdinaryForms(t *testing.T) {
for _, expr := range []string{
"* * * * *", // every minute
"0 3 * * *", // 03:00 daily
"*/15 * * * *", // every 15 minutes
"0 0 1 1 *", // new year
"0 9-17 * * 1-5", // business hours, weekdays
"0 0 * * 7", // Sunday as 7
"0,30 * * * *", // twice an hour
"* * * * *", // every minute
"0 3 * * *", // 03:00 daily
"*/15 * * * *", // every 15 minutes
"0 0 1 1 *", // new year
"0 9-17 * * 1-5", // business hours, weekdays
"0 0 * * 7", // Sunday as 7
"0,30 * * * *", // twice an hour
} {
if _, err := ParseCron(expr); err != nil {
t.Errorf("a valid cron %q was refused: %v", expr, err)
internal/declaration/declaration.go
+149 -3
View File
@@ -59,6 +59,25 @@ const (
// (04-ISSUES/026) — and leaves everything about it alone. Several modules declaring one
// access is ordinary, because none of them owns it.
TypeAccess Type = "access"
// TypeProcess is the module's own code, run on the machine, in one of three modes.
//
// **The intent, not the mechanism.** Until this, an author decided the hosting before they
// could declare anything: code of their own meant a `container` built from an image, a script
// meant a `service` and a unit somebody else had to install. Same intent — run this — with
// the choice baked into which kind was picked.
//
// **And three modes rather than three kinds**, exactly as a container has. A first draft of
// this added a `daemon` for the long-running case alone, which would have meant a new kind for
// each of the others — a scheduled task, a run-once migration, a health check. They are one
// thing run at different cadences, and that is a field, not a vocabulary entry. Every addition
// to this vocabulary widens what a compromised control plane can express.
//
// What a module declares is a bundle and a command. The mesh unpacks the bundle where it keeps
// such things and runs it — as a unit that stays up, as a step that must finish, or on a
// cadence. Tools, hooks and event consumers are not separate modes: they are loaded by a tool
// host, which is itself a process that stays up.
TypeProcess Type = "process"
)
// Resource is one thing that should be true of the machine.
@@ -393,6 +412,131 @@ func (a *Archive) validate(where string, _ bool) []string {
return problems
}
// Process is the module's own code, run on the machine, in one of three modes.
//
// The difference from Service is who owns the unit: a Service puts an EXISTING unit into a state
// and deliberately does not install one, which is right for software that ships its own. This is
// the mesh's own code — a bundle it built — so there is no unit until the mesh writes it, and
// nothing else will.
//
// The difference from Container is the hosting, and a module should not have to choose: what this
// says is what to run, and the machine's own supervisor is how. Code that genuinely needs a
// container's isolation declares a container and says so.
//
// **Three modes, matching a container's**, because they are the same thing at different cadences:
// stays up, runs once, runs on a schedule. A module's scheduled task, its run-once migration, its
// health check and its tool host are all this — and each being its own resource kind would be four
// entries in a vocabulary where every entry widens what a compromised control plane can express.
type Process struct {
ID string `json:"id"`
Type Type `json:"type"`
// Name is what the unit is called, and what an operator will see in the process table.
Name string `json:"name"`
// Source is where to fetch the bundle from, and Digest is what it must hash to. The same
// discipline as an archive, for the same reason: this crosses a network the mesh does not
// control.
Source string `json:"source"`
Digest string `json:"digest"`
// Run is the command, relative to the unpacked bundle. The first element is the program.
//
// **Named by the module, never inferred.** Guessing an entrypoint from which files exist makes
// a daemon change what it runs when somebody adds a file.
Run []string `json:"run"`
// Env and EnvFile are what it runs with. A file rather than inline values is how a credential
// reaches a daemon without passing through the declaration.
Env map[string]string `json:"env,omitempty"`
EnvFile []string `json:"env-file,omitempty"`
// User is who it runs as. Absent means root, which is what the mesh's own modules need for
// the things they do to a machine.
User string `json:"user,omitempty"`
// RestartOn names resources whose change means this must be restarted — the same rule a
// service follows, and for the same reason: a running process does not re-read its
// configuration, so replacing a file and finding the process already up leaves the machine
// behaving the way it did before while every check passes.
RestartOn []string `json:"restart-on,omitempty"`
// RunOnce marks code the host runs to completion rather than leaves running: a migration, a
// seed, a first-boot step. What follows it is gated on it finishing, because a step that did
// not make the machine ready must not be followed by the thing that needed it.
RunOnce bool `json:"run-once,omitempty"`
// Schedule runs it on a cadence — a five-field cron expression (novox/hq ADR 0053). The
// recurring twin of RunOnce: the same code, run again rather than left running.
//
// Exclusive with RunOnce and with RestartOn, for the same reason a container's is: something
// that runs once does not run on a schedule, and something that is not running cannot be
// restarted when a file changes.
Schedule string `json:"schedule,omitempty"`
}
func (d *Process) Identity() string { return d.ID }
func (d *Process) Kind() Type { return TypeProcess }
func (d *Process) Target() string { return d.Name }
func (d *Process) validate(where string, _ bool) []string {
var problems []string
if d.Name == "" {
problems = append(problems, where+": a process needs a name, which is what its unit is called")
}
if strings.ContainsAny(d.Name, "/ \t") {
// It becomes a unit name and a file on disk. A name with a separator in it would write
// somewhere nobody meant.
problems = append(problems, where+": a process name becomes a unit name, so it cannot "+
"contain a path separator or a space")
}
if d.Source == "" {
problems = append(problems, where+": a process needs somewhere to fetch its bundle from")
}
if !strings.HasPrefix(d.Digest, "sha256:") || len(d.Digest) != len("sha256:")+64 {
// The same rule an archive follows, and for the same reason: this crosses a network the
// mesh does not control, and a reference that can be made to point elsewhere is not one.
problems = append(problems, where+
": a process bundle is pinned by digest, as sha256:<64 hex characters>")
}
if len(d.Run) == 0 {
problems = append(problems, where+": a process needs to say what to run")
}
for _, part := range d.Run {
if part == "" {
problems = append(problems, where+": a process command has an empty element")
break
}
}
// **A newline cannot be represented in a unit's environment, so it is refused rather than
// mangled.** Everything else a unit file reinterprets — a percent specifier, whitespace
// splitting assignments, a quote ending one early — can be escaped. A newline cannot: it ends
// the line, and what follows is read as a unit DIRECTIVE. A value carrying one could write
// ExecStart= and have the machine run something nobody declared.
//
// Refused here, near whoever wrote it, rather than at the far end of a declaration.
for key, value := range d.Env {
if strings.ContainsAny(value, "\n\r") {
problems = append(problems, fmt.Sprintf(
"%s: the value of %s contains a line break, which cannot be written into a unit's "+
"environment — what followed it would be read as a unit directive", where, key))
}
if key == "" {
problems = append(problems, where+": an environment value with no name")
}
}
if d.Schedule != "" {
if d.RunOnce {
problems = append(problems, where+
": a process runs once or on a schedule, not both")
}
if len(d.RestartOn) > 0 {
problems = append(problems, where+
": a scheduled process is not running between its fires, so there is nothing to "+
"restart when something it reads changes")
}
if _, err := ParseCron(d.Schedule); err != nil {
problems = append(problems, where+": "+err.Error())
}
}
return problems
}
// Service is a unit the host puts into a state. It does not install the unit.
//
// Two states, and they are orthogonal rather than one scale. A unit can be enabled and stopped
@@ -654,6 +798,8 @@ func newOf(t Type) Resource {
return &Archive{}
case TypeAccess:
return &Access{}
case TypeProcess:
return &Process{}
}
return nil
}
@@ -661,8 +807,8 @@ func newOf(t Type) Resource {
// Vocabulary is every kind this host speaks.
func Vocabulary() []Type {
return []Type{
TypeAccess, TypeAction, TypeArchive, TypeContainer, TypeDirectory, TypeFile, TypeNetwork,
TypePackage, TypeService, TypeUser,
TypeAccess, TypeAction, TypeArchive, TypeContainer, TypeDirectory, TypeFile,
TypeNetwork, TypePackage, TypeProcess, TypeService, TypeUser,
}
}
@@ -930,7 +1076,7 @@ func vocabulary() string {
// ParseFileTrusted reads a declaration from a file somebody handed this host.
//
// The same as ParseTrusted, and it allows whole-line `//` comments first. A pinned, hand-authored
// artefact that nobody can annotate is one nobody can review — the substrate bundle is mostly
// artefact that nobody can annotate is one nobody can review — the foundation bundle is mostly
// explanation of why each digest is what it is.
//
// **Only for a file, never for the link.** Over the link the format stays exactly JSON, because
internal/declaration/declaration_test.go
+20 -9
View File
@@ -157,7 +157,7 @@ func TestAnEmptyDeclarationIsAMistake(t *testing.T) {
refusalFor(t, `{"declaration":1,"resources":[]}`)
}
// --- the vocabulary the substrate bootstrap needs (novox/hq 07-the-substrate.md) ---
// --- the vocabulary the foundation bootstrap needs (novox/hq 07-the-foundation.md) ---
func TestAnActionOverTheLinkIsRefused(t *testing.T) {
// novox/hq ADR 0005. The link may push declarations of known shape and never a command to
@@ -229,7 +229,7 @@ func TestAnImageMustBePinnedByDigest(t *testing.T) {
func TestAnImageTheMachineHoldsIsNamedByItsOwnDigest(t *testing.T) {
held := "sha256:" + strings.Repeat("b", 64)
if _, err := ParseTrusted([]byte(`{"declaration":1,"resources":[
{"id":"control","type":"container","name":"mesh-control","image":"` + held + `"}
{"id":"control","type":"container","name":"mesh-controller","image":"` + held + `"}
]}`)); err != nil {
t.Errorf("an image named by its own digest was refused: %v", err)
}
@@ -238,7 +238,7 @@ func TestAnImageTheMachineHoldsIsNamedByItsOwnDigest(t *testing.T) {
// whichever the runtime happened to match first.
for _, bad := range []string{"sha256:abc", "sha256:", "sha256:" + strings.Repeat("b", 63)} {
if _, err := ParseTrusted([]byte(`{"declaration":1,"resources":[
{"id":"control","type":"container","name":"mesh-control","image":"` + bad + `"}
{"id":"control","type":"container","name":"mesh-controller","image":"` + bad + `"}
]}`)); err == nil {
t.Errorf("image id %q was accepted and is not a digest", bad)
}
@@ -266,8 +266,8 @@ func TestAFieldTheNewTypesDoNotUseIsRefused(t *testing.T) {
}
}
func TestTheVocabularyIsTheEightShapesTheMeshNeeds(t *testing.T) {
// Six of them the bootstrap uses (novox/hq 07-the-substrate.md), and removing one is a
func TestTheVocabularyIsTheElevenShapesTheMeshNeeds(t *testing.T) {
// Six of them the bootstrap uses (novox/hq 07-the-foundation.md), and removing one is a
// failing test rather than a discovery during a first-node install.
//
// Two were added on 2026-08-30 and the count is asserted precisely because adding one is a
@@ -282,7 +282,7 @@ func TestTheVocabularyIsTheEightShapesTheMeshNeeds(t *testing.T) {
}
for _, want := range []Type{
TypeDirectory, TypeFile, TypeService, TypePackage, TypeContainer, TypeAction,
TypeUser, TypeArchive, TypeNetwork, TypeAccess,
TypeUser, TypeArchive, TypeNetwork, TypeAccess, TypeProcess,
} {
if !speaks[want] {
t.Errorf("the host no longer speaks %q", want)
@@ -298,8 +298,19 @@ func TestTheVocabularyIsTheEightShapesTheMeshNeeds(t *testing.T) {
// `access` is the tenth, and novox/hq ADR 0051 is its decision: shared, pre-existing data is
// the operator's, and a module is granted use of it without owning it — a shape the host must
// tell apart from a directory precisely because it must NOT create, chown or remove it.
if len(speaks) != 10 {
t.Errorf("the vocabulary is %d shapes rather than 10; every addition widens what a compromised "+
//
// `daemon` is the eleventh, and it exists because the mechanism was leaking into every module.
// Running code of one's own meant a `container` and therefore an image; running a script meant
// a `service` and a unit somebody else had to install. One intent — run this and keep it
// running — expressed two unrelated ways, with the hosting chosen before anything could be
// declared. A daemon says what to run; the machine's own supervisor is how, and the mesh owns
// the unit because it is the mesh's own code (novox/hq 03-DESIGN/01-to-be/18-building-a-module.md).
//
// It is a full-host shape rather than a portable one: it needs a process supervisor to install
// into. It does NOT need a container runtime, which is the point — only software that
// genuinely needs isolation asks for a container.
if len(speaks) != 11 {
t.Errorf("the vocabulary is %d shapes rather than 11; every addition widens what a compromised "+
"control plane can express, so a change here is a decision: %s",
len(speaks), vocabulary())
}
@@ -353,7 +364,7 @@ func TestSomethingInsideAValueIsNotAComment(t *testing.T) {
// parses as the declaration and the rest is never looked at. The machine applies something,
// reports success, and what it applied is not what the file says.
//
// Not hypothetical: a test harness appended a line to the substrate bundle by accident, every
// Not hypothetical: a test harness appended a line to the foundation bundle by accident, every
// apply kept working, and nothing said so for the entire time it was wrong.
func TestSomethingAfterTheDeclarationIsRefused(t *testing.T) {
good := `{"declaration":1,"resources":[{"id":"a","type":"file","path":"/tmp/a",` +
internal/declaration/process_test.go
+160
View File
@@ -0,0 +1,160 @@
package declaration
import (
"strings"
"testing"
)
func aProcess() *Process {
return &Process{
ID: "server", Type: TypeProcess, Name: "greeter",
Source: "https://store.invalid/greeter/daemon",
Digest: "sha256:" + strings.Repeat("a", 64),
Run: []string{"node", "index.js"},
}
}
// A process is part of the vocabulary, or a declaration carrying one is refused whole.
func TestAProcessIsSomethingTheHostSpeaks(t *testing.T) {
var found bool
for _, kind := range Vocabulary() {
if kind == TypeProcess {
found = true
}
}
if !found {
t.Fatal("a process cannot be declared, so a module that declares one is refused")
}
if newOf(TypeProcess) == nil {
t.Fatal("the decoder has no daemon, so one would be refused as an unknown kind")
}
}
// **Pinned by digest, like everything else that crosses a network.** A bundle fetched by a
// reference somebody can repoint is not pinned, and it is the one thing on a machine that would
// then be running code nobody reviewed.
func TestAProcesssBundleMustBePinned(t *testing.T) {
for _, bad := range []string{"", "latest", "sha256:short", strings.Repeat("a", 64)} {
d := aProcess()
d.Digest = bad
if problems := d.validate("a process", false); len(problems) == 0 {
t.Fatalf("a process pinned by %q was accepted", bad)
}
}
}
// What to run is named, never inferred. Guessing an entrypoint from which files are present makes
// a process change what it runs when somebody adds a file.
func TestAProcessMustSayWhatToRun(t *testing.T) {
d := aProcess()
d.Run = nil
if problems := d.validate("a process", false); len(problems) == 0 {
t.Fatal("a process with no command was accepted")
}
}
// Its name becomes a unit name and a path, so a separator in it would write somewhere nobody meant.
func TestAProcesssNameCannotEscapeItsUnit(t *testing.T) {
for _, bad := range []string{"", "../escape", "two words", "a/b"} {
d := aProcess()
d.Name = bad
if problems := d.validate("a process", false); len(problems) == 0 {
t.Fatalf("a process called %q was accepted", bad)
}
}
}
// And a well-formed one is accepted, or the tests above prove only that everything is refused.
func TestAWellFormedDaemonIsAccepted(t *testing.T) {
if problems := aProcess().validate("a process", false); len(problems) != 0 {
t.Fatalf("a well-formed daemon was refused: %v", problems)
}
}
// **The modes are exclusive, and saying so is the point of having one kind.** Something that runs
// once does not run on a schedule; something not running between fires cannot be restarted when a
// file changes. A container's modes carry the same rule, and this is the same rule because it is
// the same thing hosted differently.
func TestTheModesAreExclusive(t *testing.T) {
both := aProcess()
both.RunOnce = true
both.Schedule = "0 3 * * *"
if problems := both.validate("a process", false); len(problems) == 0 {
t.Fatal("a process that runs once and on a schedule was accepted")
}
watching := aProcess()
watching.Schedule = "0 3 * * *"
watching.RestartOn = []string{"some-file"}
if problems := watching.validate("a process", false); len(problems) == 0 {
t.Fatal("a scheduled process was given something to restart on, and it is never running")
}
}
// A cadence that is not a cadence is refused near its author, rather than by a machine at the far
// end of a declaration.
func TestAScheduleMustBeACadence(t *testing.T) {
for _, bad := range []string{"often", "0 3 * *", "99 3 * * *"} {
p := aProcess()
p.Schedule = bad
if problems := p.validate("a process", false); len(problems) == 0 {
t.Fatalf("a process scheduled %q was accepted", bad)
}
}
}
// And each mode on its own is accepted, or the tests above prove only that everything is refused.
func TestEachModeOnItsOwnIsAccepted(t *testing.T) {
once := aProcess()
once.RunOnce = true
if problems := once.validate("a process", false); len(problems) != 0 {
t.Fatalf("a step was refused: %v", problems)
}
every := aProcess()
every.Schedule = "0 3 * * *"
if problems := every.validate("a process", false); len(problems) != 0 {
t.Fatalf("a scheduled process was refused: %v", problems)
}
}
// **The one that cannot be escaped, only refused.**
//
// Everything else a unit file reinterprets can be escaped: a percent specifier doubled, whitespace
// quoted, a quote backslashed. A newline cannot — it ends the line, and what follows is read as a
// unit DIRECTIVE. A value carrying one could write ExecStart= and have the machine run something
// nobody declared.
//
// So it is refused here, near whoever wrote it, rather than rendered into a unit at the far end of
// a declaration.
func TestAnEnvironmentValueCannotCarryALineBreak(t *testing.T) {
for _, bad := range []string{
"safe\nExecStart=/usr/bin/whatever",
"carriage\rreturn",
} {
p := aProcess()
p.Env = map[string]string{"X": bad}
problems := p.validate("a process", false)
if len(problems) == 0 {
t.Fatalf("a value containing %q was accepted", bad)
}
var said bool
for _, problem := range problems {
if strings.Contains(problem, "line break") {
said = true
}
}
if !said {
t.Fatalf("refused for some other reason, which would stop being true: %v", problems)
}
}
}
// And ordinary awkward values are accepted, because escaping is what handles those — refusing them
// too would make a module unable to hold a generated password.
func TestOrdinaryAwkwardValuesAreAccepted(t *testing.T) {
p := aProcess()
p.Env = map[string]string{"PASSWORD": `a%H b"c\d`}
if problems := p.validate("a process", false); len(problems) != 0 {
t.Fatalf("a password containing the characters passwords contain was refused: %v", problems)
}
}
internal/image/builder.tar
+1 -1
View File
@@ -2,7 +2,7 @@ This is not a saved image. It is the placeholder that keeps this repository buil
A release build replaces this file with the output of `docker save` and puts it back afterwards:
make bootstrap IMAGE=mesh-control:<version>
make bootstrap IMAGE=mesh-controller:<version>
An installer built with this file present carries no control plane, and says so in preflight
rather than getting a machine part-way to being a mesh and stopping.
internal/image/image.go
+1 -1
View File
@@ -58,7 +58,7 @@ var saved []byte
var ErrEmpty = errors.New(
"this mesh-bootstrap carries no builder image, so it cannot raise a mesh. A release build " +
"embeds one: `make bootstrap IMAGE=<image>` in the mesh-host repository, where <image> " +
"is a mesh-builder image already built from the mesh-control source")
"is a mesh-builder image already built from the mesh-controller source")
// IsEmpty reports whether anything was built in.
//
internal/image/image_test.go
+3 -3
View File
@@ -72,11 +72,11 @@ func TestTheArchivesOwnIdIsReadFromTheSavedFile(t *testing.T) {
// does not.
func TestTheSavedTagsAreRead(t *testing.T) {
saved := savedImage(t, map[string]string{
"manifest.json": manifest(t, strings.Repeat("b", 64)+".json", "mesh-control:v1"),
"manifest.json": manifest(t, strings.Repeat("b", 64)+".json", "mesh-controller:v1"),
})
got := Tags(saved)
if len(got) != 1 || got[0] != "mesh-control:v1" {
t.Errorf("tags = %v, want [mesh-control:v1]", got)
if len(got) != 1 || got[0] != "mesh-controller:v1" {
t.Errorf("tags = %v, want [mesh-controller:v1]", got)
}
}
internal/link/messages.go
+1 -1
View File
@@ -60,7 +60,7 @@ type Report struct {
// Carried are the machine's ports held by what this host raised from its own bundle.
//
// **So the mesh can assign around what it did not put here** (novox/hq ADR 0038). A node
// raises its substrate before any mesh exists, so the control plane has never heard of the
// raises its foundation before any mesh exists, so the control plane has never heard of the
// store or the broker — and would hand a module a port one of them holds, discovering it only
// when a container runtime refused to start.
//
internal/store/store.go
+3 -3
View File
@@ -35,7 +35,7 @@ type Applied struct {
Type string `json:"type"`
// Origin is who asked for this: the bundle this host carries, or the mesh.
//
// Recorded because the two must not remove each other. A node raises its own substrate from
// Recorded because the two must not remove each other. A node raises its own foundation from
// the bundle before any mesh exists, then enrols and is sent declarations — and a
// declaration naming two resources would otherwise remove the store, the broker and the
// control plane, which is 04-ISSUES/010 and happened on the first end-to-end run.
@@ -47,7 +47,7 @@ type Applied struct {
// Holds are the machine's own ports this resource occupies.
//
// **So the mesh can assign around what it did not put here** (novox/hq ADR 0038). A node
// raises its substrate from the bundle before any mesh exists, so the control plane has never
// raises its foundation from the bundle before any mesh exists, so the control plane has never
// heard of the store, the broker or the control plane's own container — and a module assigned
// afterwards would be given a port one of them already holds, and would be told so by a
// container runtime rather than by anything that could have prevented it.
@@ -226,7 +226,7 @@ const (
//
// State written before origins existed was all bundle-applied: a host had no other way to be
// told anything. Guessing wrong in the other direction would have a first upgrade remove the
// substrate, which is the fault this field exists to prevent.
// foundation, which is the fault this field exists to prevent.
func originOf(r Applied) string {
if r.Origin == "" {
return OriginCarried
internal/store/store_test.go
+2 -2
View File
@@ -136,7 +136,7 @@ func TestNothingIsAnOrphanWhenEverythingIsDeclared(t *testing.T) {
}
func TestADeclarationDoesNotOrphanWhatTheBundleRaised(t *testing.T) {
// 04-ISSUES/010. A first node raises its substrate from the bundle it carries, then enrols
// 04-ISSUES/010. A first node raises its foundation from the bundle it carries, then enrols
// and is sent a declaration naming two resources. Before origins, that removed the store, the
// broker and the control plane that had sent it — the mesh deleting itself over the link the
// message arrived on, in under a second, on the first end-to-end run.
@@ -175,7 +175,7 @@ func TestTheBundleDoesNotOrphanWhatTheMeshDeclared(t *testing.T) {
func TestStateWrittenBeforeOriginsExistedIsTreatedAsCarried(t *testing.T) {
// Every resource a host had applied before this field existed came from its bundle, because
// there was no other way to tell it anything. Guessing the other way would have the first
// declaration remove the substrate — which is the fault this exists to prevent, arriving
// declaration remove the foundation — which is the fault this exists to prevent, arriving
// through the upgrade that fixes it.
s := State{Resources: []Applied{{ID: "store", Type: "container", Target: "mesh-store"}}}
internal/system/android.go
+1 -1
View File
@@ -35,7 +35,7 @@ import (
// while disconnected, reconcile already happens on start, and the mesh already reports *last
// heard from* rather than alarming on silence.
//
// It also **cannot be the first node** — every step of raising a substrate is a shape it
// It also **cannot be the first node** — every step of raising a foundation is a shape it
// refuses — and its bundle says so rather than being an empty placeholder.
type android struct{}
internal/system/system.go
+5
View File
@@ -173,6 +173,11 @@ func everyShape() []declaration.Type {
// bind mount, and a bind mount needs the container runtime a full host has. So it sits
// here with the container it guards, not at the portable floor (novox/hq ADR 0051).
declaration.TypeAccess,
// A daemon needs a process supervisor to install a unit into, which is what separates a
// full host from the floor. It does NOT need a container runtime, which is the point of
// it: the mesh's own code runs as a process on the machine, and only software that
// genuinely needs isolation asks for a container.
declaration.TypeProcess,
}
}