The bootstrap reaches the broker, and survives a reboot
Five steps now instead of three. A sealed machine goes from bare to a container runtime, a store, the inventory database, that database's schema applied by mesh-control, and LavinMQ running and answering. The database is called inventory rather than mesh. ADR 0008 grants a context only what it exclusively owns and ADR 0006 says the mesh database names a thing that will not exist -- so one database per context, and there is one context. The broker is in the bundle because ADR 0006 now says it must be: the control plane reaches a node only over the link, the link is the broker, so nothing can provision the broker. Two images, which is the cost that record accepts. Verified by reading the system rather than the report: inventory present and mesh absent, the node table with its indexes, the migration row, lavinmqctl answering, 5672 listening. Sealed confirmed both ways -- the internet times out, the lab registry returns 200. Then rebooted, which was the part worth doing rather than assuming. Everything returned: docker from boot: enabled, both containers because this host creates every container --restart unless-stopped, the schema intact in its volume. Three reconciles before and one after all report no change. One thing that reads as a success and was not: the first sealed check said the machine could reach example.com. It was the test that was wrong -- a helper script pasted arguments into a shell line, so a command with quotes was re-split and ran on the workstation. The machine had been sealed the whole time. The helper now requotes each argument.
This commit is contained in:
+35
-13
@@ -2,13 +2,20 @@
|
||||
|
||||
## `substrate-first-node.lock`
|
||||
|
||||
What an Arch machine must be before a mesh exists — the first steps of the bootstrap in
|
||||
[novox/hq `07-the-substrate.md`](https://git.novox.be/novox/hq): a container runtime, a store
|
||||
running, and the control plane's database created inside it.
|
||||
What a machine must be before a mesh exists — steps 0 to 4 of the bootstrap in
|
||||
[novox/hq `07-the-substrate.md`](https://git.novox.be/novox/hq):
|
||||
|
||||
**It stops there**, and the file says why: loading the control plane's schema and starting the
|
||||
control plane are the next two steps, and there is no control plane yet. A bundle naming one
|
||||
would be a bundle that cannot be applied.
|
||||
```
|
||||
0 a container runtime
|
||||
1 the store runs
|
||||
2 a database per context one today, `inventory`
|
||||
3 that context's schema mesh-control migrate
|
||||
4 the broker runs
|
||||
```
|
||||
|
||||
**It stops there, and the file says why.** Step 5 is a virtual host, a credential and a
|
||||
certificate; step 6 is the control plane running. Nothing consumes any of them yet, and a bundle
|
||||
whose last step cannot be checked is worse than a shorter one.
|
||||
|
||||
Build a host carrying it:
|
||||
|
||||
@@ -16,11 +23,26 @@ Build a host carrying it:
|
||||
make host SYSTEM=arch BUNDLE=examples/substrate-first-node.lock
|
||||
```
|
||||
|
||||
**The image reference has to be replaced before this is useful.** It is written as
|
||||
`REGISTRY/postgres@DIGEST` because the digest belongs to whatever registry serves it — in the
|
||||
lab, one the scenario raises, which reports its digests when it comes up. That is not a
|
||||
placeholder to be tidied away: a bundle is built *for a target*, and which registry that target
|
||||
pulls from is part of the target.
|
||||
**The registry address and digests have to be replaced before this is useful.** They are written
|
||||
as `192.0.2.250:5000/…@sha256:…` because a digest belongs to whatever registry serves it — here,
|
||||
one a lab scenario raises, which reports its digests when it comes up. That is not a placeholder
|
||||
to be tidied away: a bundle is built *for a target*, and which registry that target pulls from is
|
||||
part of the target.
|
||||
|
||||
Verified end to end in a lab machine with no route out: five resources applied, idempotent on a
|
||||
second run, and `mesh` present in the store afterwards.
|
||||
### What was verified, and how
|
||||
|
||||
On a lab machine confirmed sealed — `curl https://example.com` times out, the lab registry answers
|
||||
200 — the whole bundle applied from bare: eight resources, `inventory` created and `mesh` nowhere,
|
||||
the `node` table present with its indexes, the migration recorded, and LavinMQ answering
|
||||
`lavinmqctl status` with AMQP listening on 5672.
|
||||
|
||||
Three consecutive reconciles after that: **already matches — 8 resource(s) checked**, each time.
|
||||
|
||||
Then the machine was **rebooted**, and everything came back: docker from `boot: enabled`, both
|
||||
containers because the host creates every container `--restart unless-stopped`
|
||||
(`internal/apply/apply.go`), the schema intact in its named volume, and reconcile still finding
|
||||
nothing to do.
|
||||
|
||||
The reboot is worth doing rather than assuming. Nothing in the declaration asks for a container to
|
||||
return, so that it does is a property of the host, and the only way to know it holds is to take
|
||||
the machine away and give it back.
|
||||
|
||||
@@ -1,18 +1,22 @@
|
||||
// substrate-arch.lock — what an Arch machine must be before a mesh exists.
|
||||
// substrate-first-node.lock — what a machine must be before a mesh exists.
|
||||
//
|
||||
// The first three steps of the bootstrap (novox/hq 03-DESIGN/01-to-be/07-the-substrate.md).
|
||||
// Steps four and five — load the control plane's schema, start the control plane — are absent
|
||||
// because the control plane does not exist yet. A bundle that named it would be a bundle that
|
||||
// cannot be applied.
|
||||
// Steps 0 to 4 of the bootstrap (novox/hq 03-DESIGN/01-to-be/07-the-substrate.md): a container
|
||||
// runtime, a store, a database per context, that context's schema, and the broker.
|
||||
//
|
||||
// It stops before step 5 (a virtual host, a credential, a certificate) and step 6 (the control
|
||||
// plane runs), because nothing consumes them yet. A bundle naming a control plane that serves
|
||||
// nothing would be a bundle whose last step cannot be checked.
|
||||
//
|
||||
// PINNED BY DIGEST, and the digest is not decoration: a tag can be made to point at a different
|
||||
// image, and this file is applied on a machine with no mesh to ask about anything.
|
||||
// image, and this file is applied on a machine with no mesh to ask about anything. These digests
|
||||
// belong to the registry the lab raises, which is what a real node pulls from anyway — what is
|
||||
// required is a reference that is exact and cannot move (novox/hq ADR 0006).
|
||||
//
|
||||
// The store's data is a NAMED VOLUME rather than a directory on the machine. A directory the
|
||||
// host creates is owned by root, and the database runs as somebody else inside the container —
|
||||
// so it could not write, and the container crash-looped. A named volume lets the image set up
|
||||
// its own ownership, which is what it is for. It also outlives the container, which is what you
|
||||
// want for the thing holding the mesh's state.
|
||||
// The store's data is a NAMED VOLUME, not a directory on the machine. A directory the host
|
||||
// creates is owned by root, and the database runs as somebody else inside the container — so it
|
||||
// could not write, and the container crash-looped. A named volume lets the image set up its own
|
||||
// ownership, and outlives the container, which is what you want for the thing holding the mesh's
|
||||
// state.
|
||||
{
|
||||
"declaration": 1,
|
||||
"resources": [
|
||||
@@ -32,7 +36,7 @@
|
||||
"id": "store",
|
||||
"type": "container",
|
||||
"name": "mesh-store",
|
||||
"image": "REGISTRY/postgres@DIGEST",
|
||||
"image": "192.0.2.250:5000/postgres@sha256:7abf537131b66ed5af448d90653abf1679b0c7e9a1f07efdd4c3108a401b259a",
|
||||
"env": {
|
||||
"POSTGRES_PASSWORD": "bootstrap",
|
||||
"PGDATA": "/var/lib/postgresql/data/pgdata"
|
||||
@@ -47,11 +51,35 @@
|
||||
"verify": ["pg_isready", "-U", "postgres"]
|
||||
},
|
||||
{
|
||||
"id": "control-plane-database",
|
||||
"id": "inventory-database",
|
||||
"type": "action",
|
||||
"in": "mesh-store",
|
||||
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE mesh'"],
|
||||
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw mesh"]
|
||||
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE inventory'"],
|
||||
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw inventory"]
|
||||
},
|
||||
{
|
||||
"id": "inventory-schema",
|
||||
"type": "action",
|
||||
"command": ["docker", "run", "--rm", "--network", "container:mesh-store",
|
||||
"-e", "MESH_STORE_INVENTORY=postgres://postgres:bootstrap@127.0.0.1:5432/inventory?sslmode=disable",
|
||||
"192.0.2.250:5000/mesh-control@sha256:1c27a43c4431c2b580404e8e1768cd858009e265e80b6f1591eb6de1123fc411",
|
||||
"migrate"],
|
||||
"verify": ["sh", "-c", "docker exec mesh-store psql -U postgres -d inventory -tAc \"select to_regclass('public.node')\" | grep -qx node"]
|
||||
},
|
||||
{
|
||||
"id": "broker",
|
||||
"type": "container",
|
||||
"name": "mesh-broker",
|
||||
"image": "192.0.2.250:5000/cloudamqp/lavinmq@sha256:b117c254e6e269a29db479e6b410ca4e46e035b4981e49d24b159673ef09d336",
|
||||
"ports": ["5672:5672"],
|
||||
"volumes": ["mesh-broker-data:/var/lib/lavinmq"]
|
||||
},
|
||||
{
|
||||
"id": "broker-ready",
|
||||
"type": "action",
|
||||
"in": "mesh-broker",
|
||||
"command": ["sh", "-c", "for i in $(seq 1 60); do lavinmqctl status >/dev/null 2>&1 && exit 0; sleep 1; done; exit 1"],
|
||||
"verify": ["lavinmqctl", "status"]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user